Save the date: I'll be hosting a Zoom discussion for NonZero members on Saturday, April 25th, at noon US Eastern Time. The subject will be AI and, to some extent, war—and the particular connection between the two that’s laid out in the piece below. A link to the Zoom call is at the bottom of this newsletter.
—Bob
“The week that changed everything” is a phrase that shouldn’t be tossed around lightly. After all, “everything” covers a lot!
Also: In what may be the phrase’s most common context—a Christian context—it refers to the week that brought the Crucifixion and Resurrection of the Messiah and hence the promise of salvation for humankind. And that’s a high bar.
So I’m going to hold off for now on calling this past week the week that changed everything. But I’m going to argue that, depending on how things play out, a title this exalted could wind up being in order. This was a week that saw seismic tremors at two different levels—one geopolitical (in the Middle East) and one technological (in artificial intelligence). And there was a connection between the two that isn’t yet visible but could prove momentous. The connection could even prove salvific—salvific as in salvation, as in the salvation of humankind. And I’m not kidding, so long as “salvation” refers not to an afterlife or the cleansing of our sins but to saving humankind from various apocalyptic outcomes, up to and including extinction.
It’s possible that I’m reading too much into this. I have a book coming out in June called The God Test: Artificial Intelligence and Our Coming Cosmic Reckoning. And when you have a book coming out you tend to see the world through that lens, and this can lead to distortion. You can, for example, see corroboration of your book’s thesis everywhere you look. And such corroboration is what I see when I look at this week’s news. That’s not necessarily a cheering thought, because this thesis isn’t wholly upbeat. Still, I do see, in this week’s news, some cause for hope—reason to think that maybe the dark scenarios laid out in my book won’t come to pass and the brighter scenarios laid out in it will.
But, again, I could be wrong; AIs aren’t the only beings capable of hallucination. You be the judge.
One of the book’s central points is that if we’re going to successfully navigate the AI revolution—avoid traumas and catastrophes that range from social chaos to planet-wide authoritarian rule to nuclear war to complete annihilation—we’ll have to cross the threshold to true global community. The world’s nations have to confront this challenge collectively—build new international rules and norms—or else watch in dismay and intermittent terror as a technology that accelerates without constraint or guidance strips us of agency. (If this sounds too dramatic, please suspend judgment. I spend a fair amount of time in the book laying out the evidence and logic behind my more dramatic-sounding claims.)
Much of what we need to do in controlling AI comes under the rubric “international governance,” but I think this phrase, radical though it sounds to some people, runs the risk of understating the challenge. After all, we have international governance now. Various treaties, such as the Nuclear Non-Proliferation Treaty and the Law of the Sea Treaty, qualify for that label. And—don’t get me wrong!—these treaties are nothing to sneeze at. But few of them are global, and they often lack enforcement mechanisms, and compliance can be spotty, and they have a tendency to fray. I think AI will ultimately call for structures of international governance more ambitious and nuanced than the kind we’ve had. And, as if that weren’t a tall enough order, these structures will have to be bolstered by new norms, and these norms will have to get the kind of respect that a society’s most important norms get.
This is why I used the phrase “true global community” above. I don’t think our species will be able to muster the focus and commitment this mission demands unless we can sustain at least the minimal degree of harmony that the term “community” implies. That doesn’t mean kumbaya-level harmony, but it does mean we can’t keep getting bogged down in the kinds of wars that have plagued humanity since (at least) the dawn of history.
This points to one reason I found this week’s Iran war ceasefire, partial and tenuous though it is, heartening. I’m hoping it can usher in not just peace, but a peace that has a particular kind of psychological impact—a peace that highlights the frequent futility and absurdity of war. After all: The two aggressors, the US and Israel, spent God knows how much money and killed God knows how many innocent people, and not only failed to reach their stated war aims but may have handed their enemy a new and potent form of regional influence: ongoing control of the Strait of Hormuz. (I elaborate on some of this in a video op-ed I posted Friday about my hopes for the ceasefire—and argue that this new Iranian influence, though anathema to Israel and America, could be a stabilizing and pacifying force in the near term.)
What’s more, this war has been nothing short of a political debacle for the leader of the more powerful of the two aggressor nations, Donald Trump. So the perpetrator of an egregious violation of the UN Charter got powerfully negative feedback. That’s the kind of feedback we need if we’re going to restore some respect for international rules and norms, a pre-requisite for building a true global community.
As if Trump’s humiliation weren’t enough cause for celebration, the transparency of his motivations helped highlight the absurdity of the political dynamics that so often lead to war: National leaders inflate threats abroad to foment fear at home and thus benefit politically by posing as guardians of national security. Both Trump and Benjamin Netanyahu are veteran practitioners of this craft, but in this case Trump practiced it so ineptly as to invite widespread ridicule. He literally couldn’t think of a single quasi-plausible threat that Iran posed to the US, even though he deemed the Iranian menace so grave as to warrant an assault that killed thousands of people, wiped out untold infrastructure, and destroyed numerous cultural treasures.
Trump’s crudeness—the playground taunts and macho chest-pounding—can get tiresome, but it does have the virtue of clearly demonstrating what drives him and so many other warmongering politicians: They’re just alpha males trying to stay on top of the heap. It would be hard for him to more vividly convey the primitive and visceral energies that fuel wars without actually putting on an ape costume.
I don’t want to get carried away. We are not on the verge of ending war forever, and in fact it’s not even clear that we’ll succeed in ending this war. Still, this war had some of the hallmarks of a landmark. In The God Test I write, “Seeing the absurdity of the average conflict doesn’t make any particular conflict easy to resolve, and it doesn’t mean there’s never one side in a conflict that deserves support. Still, if the amount of change we need on this planet is as great as I think it is, it may be necessary for large numbers of people to look around and say, ‘Folks, this is ridiculous. We can’t go on like this.’ ” What we’ve seen over the past six weeks is a war that, more than any American war I’ve lived through, invites that kind of reaction. Folks, this was ridiculous. It was almost a caricature of the folly of war, and I hope it will be seen that way more and more as it recedes into the past.
So that’s the seismic activity at the geopolitical level that I alluded to above—a war that seized the world’s attention by imperiling the global economy and that, however unwelcome in itself, could have a welcome impact on the perception of war itself, an impact that I hope is the beginning of truly tectonic change, of a sustained drift away from war.
Of course, a hope is just a hope. One conspicuously absurd war won’t by itself transform the world’s view of war, much less build a global community. That construction project will require additional impetus. Which leads us to the other level at which seismic change was evident this week: the technological level, where that very impetus began to materialize.
This week Anthropic announced that it has a new large language model that’s way more powerful than past models—so powerful that it poses a threat to the world’s information infrastructure. This model can find otherwise unfindable vulnerabilities in software—and in fact, said Anthropic, has found previously unknown holes in every major operating system and browser. And in most cases, it is able to then build a tool to exploit those vulnerabilities. So if this model fell into the wrong hands, that could be big trouble—which is why, says Anthropic, there are no plans to release the model for the time being.
Some people have wondered whether Anthropic’s claims about the terrifying power of this new model are mainly marketing hype—a suspicion that isn’t exactly discouraged by the model’s name: Mythos. But Anthropic is making Mythos available to big companies that maintain important parts of the digital infrastructure, like Microsoft, Apple, and Cisco. So if these claims were hugely exaggerated, word of that would get out—especially since one of the companies, Google, is a direct competitor of Anthropic’s and would presumably prefer not to admit that Mythos is significantly more powerful than its own frontier model, Gemini, in such an important domain. (Mythos, says Anthropic, wasn’t designed to facilitate cyberattacks—this capability was just a byproduct of designing it to write computer code.)
AI, as Mythos demonstrates, is a good example of a technology that can’t be adequately governed at the national level alone. Today it’s America that has the most potent AI hacker, but tomorrow it could be China. In fact, we can’t be quite sure that China doesn’t have a potent AI hacker now, because the two countries are making no serious effort to collaboratively address the emerging challenge they face and so aren’t doing a lot of communicating on this subject.
That gives me an uneasy feeling. As I write in The God Test:
These kinds of dangers—AI-abetted biological virus, AI-abetted computer virus, AI-infused cyberweapon, rampantly destructive AI agent—and various others make it harder for any nation to feel safe unless it has some confidence that things are under control in other nations. And it’s hard to get that kind of confidence without international agreements that qualify, in at least some sense, for the term “international governance.”
I’m happy to report that the need for international coordination on AI is appreciated by some influential people. New York Times columnist Tom Friedman, assessing the Anthropic news, wrote this week:
The solution—this may shock people—must begin with the two AI superpowers, the US and China. It is now urgent that they learn to collaborate to prevent bad actors from gaining access to this next level of cyber capability. Such a powerful tool would threaten them both, leaving them exposed to criminal actors inside their countries and terrorist groups and other adversaries outside. It could easily become a greater threat to each country than the two countries are to each other.
But other observers have reacted in roughly the opposite fashion, saying there’s no realistic hope of deep trans-Pacific cooperation, so America must double down on its AI race with China. This will no doubt be a common reaction.
Another common reaction, I suspect, will be that we should immediately set about deploying Mythos as a weapon against China. This week the well known social media influencer Noah Smith, in reference to a story about recent data theft from a state-run Chinese computer, tweeted: “Just today, someone asked me: If our government knew about Mythos ahead of time, why didn’t it team up with Anthropic to hack China before announcing the model’s capabilities to the public?”
If our government hasn’t done that, the reason may be the Trump administration’s ongoing spat with Anthropic. But you can bet that people in the administration are pondering the question Smith raised. And you can bet that people in Beijing are pondering it. And you can pretty much bet that nobody in the Trump administration is wise enough to ponder the implications of Beijing pondering it. Here’s one such implication: When nations get insecure about the emerging capabilities of another nation, they sometimes take dramatic action. Just ask Bibi Netanyahu. And when the emerging capability is ultra-powerful hacking that, for all you know, is already being deployed against you, a nation can get insecure and reactive, and interpret ambiguous evidence as attack and respond accordingly.
The principle this points to—the potentially stabilizing effects of mutual transparency between the US and China—is just one of many AI-related issues that deserve lots of attention and aren’t getting much. The same might be said of this week’s Anthropic news itself. (The Times, notwithstanding Friedman’s column, ran its news story about Mythos in the business section—and, in the physical edition, below the fold.) But this story is still filtering through tech podcasts and other niche media and is destined to get more mainstream attention for some time to come.
I’m hoping that, as more and more people process the story, they’ll see its significance. I’m also hoping that they’ll understand one reason its significance took a while to sink in: Because news about it was obscured by news of war. To put a finer point on it: Tuesday—the day Anthropic released the news—was also the day Trump declared that if Iran didn’t bend to his will, “a whole civilization will die tonight.”
It’s understandable that this bizarre and vile and ultimately ephemeral threat to a great civilization absorbed our attention while news of a more deeply rooted and enduring threat to civilization writ large flew under the radar. Still, folks, this is ridiculous—that we’re letting primitive and malicious creatures like Donald Trump and Bibi Netanyahu dominate the stage while issues of critical importance to the whole planet languish in relative obscurity.
Maybe reflecting on the absurdity of Tuesday’s allocation of attention will help us eventually get our priorities straight. So remember the date: April 7, 2026. We can call it Planet of the Apes day.
PS: I taped a podcast conversation this week with Shakeel Hakim, editor of Transformer, a newsletter about AI. We taped the conversation Tuesday, right before the Mythos news broke (or at least before it reached my news feed). But we did agree on some things relevant to Mythos: (1) AI demands international governance; (2) It may take a catastrophe to focus the world’s attention on that fact—but, as Shakeel pointed out, we can hope that this service will instead be performed by a “near miss”—a narrowly averted catastrophe. Well, so far, at least, Mythos hasn’t brought a catastrophe and so is the closest thing we’ve had to a “near miss.” Let’s not waste it.
PPS: I notice that the Amazon page for my book offers a “pre-order price guarantee”—an opportunity to lock in the current price just in case the price changes between now and June. Now, TBH, the chances of the price rising between now and June are pretty low even amid high oil prices. But you never know…
Note: Those final data points are from before the Iran war.
Banners and graphics by Clark McGillis.