How close are we to AGI?
Is the current model already training the next one?
Do all models converge?
Where does the energy come from?
What are the other bottlenecks?
Is China actually behind?
Are we already at war?
Is cancer actually solved?
Are humanoids real?
Is it a bubble?
Will we nationalize the labs?
Could we even?
What happens to college?
Universal basic income?
Should I sell my AI exposure?
Should I buy a doomsday compound?
Should I just sell to a lab?
Why can’t I sleep?
Will I still have a job?
What should I tell my kids to study?
Should I have a kid?
When was the last time you had a conversation that was not about this?
Is this a religion?
Do we really think we’re building a new god?
Will I be okay?
Will any of us?
“We are approaching here a crucial question. The declared aim of modern science is to establish a strictly detached, objective knowledge…But suppose that tacit thought forms an indispensable part of all knowledge, then the ideal of eliminating all personal elements of knowledge would, in effect, aim at the destruction of all knowledge. The ideal of exact science would turn out to be fundamentally misleading and possibly a source of devastating fallacies”
- Michael Polanyi, The Tacit Dimension
There is a particular anxiety, psychosis, and stress emanating from the internet, from San Francisco, from technology as a tool and an industry, from the news, from society.
It shows up in two forms from people in my spheres across both coasts.
The first, a psychosis and anxiety born from the outside. The world is changing faster than ever before, AGI is coming to reorient society and value and our lives, peak instability.
The second, a psychosis from the inside. A sense of nihilism leaving people wondering what the point is, whether we are societally on a runaway freight train, and whether there is any way to crawl out of the depths of the past 20 years of societal change.
And thus, we experience a very particular kind of self-protectionism.
We search for ways to survive the doom loop of our internal and external states, reaching for flickers of utopias that feel precarious at best and naive at worst. We tell ourselves, this time is different, that the universe is conspiring at this exact moment to knock humans off the prior trajectory we were allegedly on.
To deal with this, we look aggressively away from, or obsessively toward tomorrow. We either stop caring, or we build theoretical futures elaborate enough to calm thoughts today.
Escapism, mostly. Which works until a real person walks into the model and breaks it.
World-First Thinking
“It seems very likely that we could in the future build a computer system which has approximately the same set of capabilities as a human. Whether we would want to is another matter. This would probably be a very complex system with lots of parts that don’t really play well together, just like our brain, and very hard to fine-tune.”
- Julian Togelius, A very short history of some times we solved AI
I have long had an obsession with imagining possible futures. It is, in many ways, the reason why I have had any semblance of success professionally.
Imagining and forecasting futures is endlessly fun because it is a complex art of turning uncertainty into shape. The craft rewards modeling gradients of possibilities, incentives, and cascading effects of technological and scientific breakthroughs.
This is seductive right now to the collective tech consciousness, staring into a dark forest with shadows of abundance, existentialism, and god knows what else. The specifics are too fuzzy and too overwhelming, and so people start building neat worlds.
I’ve started to call this world-first thinking. Ideal future worlds, pulled from childhood, prior art, the current thing, science fiction, all to help tighten the volatility bands of life. You get to model concern for humanity without getting too close to human beings, build the worlds and conditions for said humanity, then slot the human beings in.1
But the poetic reality, or maybe the problem, is that the volatile thing in any model is the human beings. And by modeling them last and confining them to worlds built before them, we never let any actual person seem real enough to induce meaningful, personal volatility in the world, in our lives, or in the philosophies of how we think we want to live.
A Crisis of Specificity
“Man developed his intellect to gain control of nature. He conquered the sea, the jungle, the desert, disease. But in the exertion of this power of the mind he lost, at times, his contact with nature. Nature is irrational. It creates earthquakes, floods, disasters over which we have no control.”
- Anaïs Nin, The Novel of the Future
When you artificially confine the most volatile thing in the model, you create a crisis of specificity.
Nobody can convincingly walk through the specifics of what could, should, or might happen in the world being built, nor how we bridge the gap to this world.2
The lab leaders themselves have started admitting it. Dario put Work and Meaning as the closing section of Machines of Loving Grace, conceding the brevity came from a lack of clear answers rather than a lack of seriousness. Sam, less pithy in recent interviews, says we need to stop yadda-yadding the human side. Demis, on stage, told a room we will have philosophical questions to answer and will need great new philosophers.
It is true that it is almost entirely downside for anyone in these positions to give specifics right now on how the world could or should change beyond vague warnings of “we’re fucked, it’s fine.”
But these framings, however sincere, still admissions. The technological future is described with great precision surrounding capex, scaling laws, and model capabilities while the human condition gets deferred into platitudes like meaning, usefulness, philosophy, and adaptation.
Words that show an understanding of the importance, but an inability to convey it; oddly reminiscent of Barthes take on writing about love:3
“On the one hand, this is saying nothing; on the other, it is saying too much: impossible to adjust. My expressive needs oscillate between the mild little haiku summarizing a huge situation, and a great flood of banalities. I am both too big and too weak for writing”.
Perhaps we should look back to old philosophers, not theoretical new ones.
Means or Ends: A philosophical progression
As I’ve continued to talk with friends about this gnawing feeling4, I’ve been pushed to read and re-read a variety of philosophers with the crux being the Kantian framing of whether or not we consider human beings as a means or an end.
That question is annoyingly hard to escape because you eventually end up in some discussion of if humans are the thing the world is for, or if we are merely passing through until something else optimizes the world? And should we (or should I) care?
Arendt’s The Human Condition has a perhaps better way of saying this meandering point: “Nobody is ever the same as anyone else who ever lived, lives, or will live.”
The Effective Altruist and MacAskill longtermist framing complicates this, because it is not exactly wrong or even misaligned to Arendt but still feels off.
I get it. Future people matter and we should think beyond ourselves.5
But there is a threshold where “humanity” becomes the thing we romanticize and actual humans begin to blur. Their significance, outline, feelings, eye color, passing expressions, and wandering thoughts fade into nothingness.
We then can progress into further philosophies.
Accelerationism is perhaps the darker abstraction. And to be clear, I am not anti-acceleration. But there is some version of this ideology where the future seems to matter more than the people living through it. Where technological progress becomes the thing, and humans are just supposed to deal with whatever force it exerts on the world.6 Protected, subsidized, entertained, optimized, maybe even loved, but still an afterthought of sorts.
Total Eclipse
Annie Dillard wrote an essay titled Total Eclipse in 1982 which has stuck with me as a canonical piece of writing on humanity.
She watches the eclipse, screams involuntarily with everyone around her, and ends up at a roadside diner needing to talk it through over breakfast before she can do anything else. While the story is about an eclipse, the essay really is about the same question underneath all of this: what does significance mean if the significant change only erodes the core humanness of it all?
She writes one of my favorite paragraphs (ever?):
“If you were to glance out one day and see a row of mushroom clouds rising on the horizon, you would know at once that what you were seeing, remarkable as it was, was intrinsically not worth remarking. No use running to tell anyone. Significant as it was, it did not matter a whit. For what is significance? It is significance for people. No people, no significance. This is all I have to tell you.”
We keep modeling that horizon and that world, perfectly sensible and strangely bloodless.
A whole world.
And then, somewhere after all of that, the human being.
thanks to andy, jess, and smac for thoughts
Almost as if humanity has no choice in how worlds emerge.
Mass unemployment? Incels? Breakaway income disparity? Crisis of connection? These things are meaningful but not exactly felt at scale without specificity.
I know this is so random but bear with me.
They are tired of it, hence the essay.
Especially in tech, where people far too often manage to be both wildly ambitious and at times, shockingly myopic.
I couldn’t bring myself to directly quote the unabomber manifesto but Ted Kaczynski has some meditations that might make people who say stuff like “it’ll happen no matter what so we might as well get there faster” think. One in particular “The system does not and cannot exist to satisfy human needs. Instead, it is human behavior that has to be modified to fit the needs of the system. This has nothing to do with the political or social ideology that may pretend to guide the technological system. It is not the fault of capitalism and it is not the fault of socialism. It is the fault of technology, because the system is guided not by ideology but by technical necessity…When skilled workers are put out of a job by technical advances and have to undergo “retraining,” no one asks whether it is humiliating for them to be pushed around in this way. It is simply taken for granted that everyone must bow to technical necessity. and for good reason: If human needs were put before technical necessity there would be economic problems, unemployment, shortages or worse. The concept of “mental health” in our society is defined largely by the extent to which an individual behaves in accord with the needs of the system and does so without showing signs of stress.”
