Below is Part I of a multi-part essay that I’ll be rolling out on NZN over the next couple of weeks. The essay is a kind of call to action—an argument about what we need to do to successfully navigate the AI revolution and why we need to do it.
Part I: The kind of singularity that’s approaching
In late May, Demis Hassabis, head of Google DeepMind, said “We’re at the foothills of the singularity.” In late July, Sam Altman, head of OpenAI, said “We’re now, like, in the singularity.” That same week, Elon Musk said, “We are in the singularity, just the very early stages of it.” If you’re not steeped in the culture of artificial intelligence, you may have a question: What are these people talking about?
They’re talking about an idea that people in their milieu have been talking about earnestly for decades—and an idea that most people outside of that milieu have been either unaware of or dismissive of. But over the past year, and especially the past few months, more and more people have come to take the idea seriously—and many now believe that, as futurist Ray Kurzweil put it in the title of his 2005 bestseller, “the singularity is near.”
In one sense, it’s hard to argue with them. A big part of the meaning of “the singularity” is accelerating and ultimately transformative technological change—change that happens faster and faster until, at some point, you’re in a whole new world. And there are definitely signs that growth in AI capability is accelerating, portending big and possibly breathtaking progress in science and dramatic change in other realms. The resulting sense of momentous acceleration is why, though some people seem giddy about the singularity, eager for the blessings progress can bring, others are terrified, imagining a world that spins out of control.
But there’s also a sense in which Hassabis, Altman, Musk, and the many other heralds of the singularity are wrong. There’s an important part of the meaning of “the singularity” that doesn’t apply to the current situation. And understanding that fact—seeing the difference between “the singularity” in its full modern meaning and the watershed in technological evolution we’re actually approaching—could be salvific. It could empower us to keep the world from spinning out of control as the AI revolution unfolds.
The first person to use the term “singularity” in the modern sense of the term seems to have been John von Neumann. Von Neumann, a physicist and mathematician who helped design the atom bomb and co-invented game theory (among many other feats), used the word while talking to the mathematician Stanislaw Ulam. The only record we have of the discussion is Ulam’s recollection, written down after Von Neumann’s death in 1957:
“One conversation centered on the ever accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue.”
The term “singularity” would have been familiar to von Neumann from physics as well as math. In both, it refers to a point where existing analytical tools lose their usefulness; what lies beyond is unclear—unpredictable and maybe incomprehensible. A common example is a black hole: As you approach its center (which isn’t recommended), the strength of gravity approaches infinity, and the known laws of physics break down.
Hence the full modern meaning of the term “the singularity”—not just the relentless and ultimately transformative acceleration of technological change, but an acceleration whose transformative consequences are inherently unknowable: “Human affairs as we know them,” as von Neumann put it, won’t continue, and it’s not clear what will replace them. Beyond a black hole’s “event horizon”—the point where gravity prevents even light from escaping—lies deep mystery.
This image—of an imminent and unpredictable metamorphosis, a future that is opaque but assuredly strange—helps explain the allure of the concept of the singularity. If an episode of a sci-fi TV series ended with titans of technology announcing the arrival of such a thing, you’d tune in next week, right?
Here’s my version of a plot spoiler:
I agree with these tech titans that AI will bring transformative change—a whole new world, socially and politically—and that the technological force behind this change is growing faster and faster. But as for the other half of the meaning of “the singularity:” I don’t think there is a metaphorical “event horizon” that shields the future entirely from our view. I think the singularity is clear.
Or, at least, clear enough. The broad contours of the future are visible. It’s possible to discern the paths available to us and the different kinds of worlds they will lead to. Then, if we can summon the requisite political and moral resources, we can create the best of those worlds.
Here is our menu of options: (1) a world of chaos and conflict and intermittent catastrophe, conceivably featuring human extinction; (2) a world that features expanding and deepening international governance that eventually spans the planet and qualifies for the term “global governance.”
In a way, that’s too simple, because option two—global governance—subdivides into two options: good global governance and bad global governance. And bad global governance, in the age of AI, could be very bad, because this technology can be a hugely powerful tool of authoritarian surveillance and control. Even if we leave aside the most cinematic sci-fi doomer scenarios, in which the authoritarian ruler is made of silicon, there are global governance scenarios—realistic scenarios, I’m afraid—that qualify as dystopian.
So here is the full menu of options: (1) good global governance; (2) bad global governance; (3) an enduringly chaotic hellscape.
In all of these scenarios, the current world order—a bunch of nation-states and loose blocs of nation-states that function without systematic global coordination yet together keep the planet largely stable—would cease to exist. Either there would be considerable migration of governance up to the global level (even as nation-states retained significant powers) or stability would no longer be the rule. In any event, “human affairs as we know them” would not continue.
To understand why change this dramatic is not just plausible but inescapable, it helps to understand a dimension of the concept of the singularity that von Neumann, so far as we know, didn’t mention: a positive feedback mechanism that drives the acceleration of change.
Coming later this week: Part II: The roots and branches of acceleration. All parts of the essay accumulate here.
[Meanwhile: If you want to check out my book on AI, The God Test, you can read the introductory chapter and excerpts from all other chapters at thegodtest.net.]
Banners and graphics by Clark McGillis.
I read the book. Very good distillation of the landscape. I highly recommend it.
Picky point, Ulam didn't co-invent game theory. That was Morgenstern.
Von Neumann said that Edward Teller was the father of the H bomb, but Ulam slept with its mother.