[Note: In addition to being the 25th anniversary of the 9/11 attack, today caps a very big week in the world of AI. For more on that, see the video I’ve posted below, after my obligatory 9/11 reflections.]

This week I heard on the radio that the 9/11 Museum in New York is encouraging us to “Never Forget”—and indeed has set up a “Never Forget” fund. Well if I were running the 9/11 Museum, I’d try to get that message out, too. But if I were in charge of US foreign policy, the message I’d try to get out is: “Forget! Forget! Forget!” Dwelling on 9/11 is what initiated and helped sustain a disastrous quarter century of American adventurism abroad.

I mean, sure, if America’s leaders had rationally assessed the causes of the 9/11 attack, dwelling on it could have been productive. They might have started by looking into the conditions that gave birth and subsequent energy and direction to al Qaeda, and pondered the following possibilities: (1) Maybe the US shouldn’t have staged a proxy intervention in Afghanistan two decades earlier that flooded the place with weapons and created the cauldron that enticed, shaped, and revved up a young Osama bin Laden (and also created the violent chaos that allowed the Taliban to take control of Afghanistan—a position the Taliban used, during the runup to 9/11, to provide safe haven for bin Laden and al Qaeda). (2) And maybe, after the Persian Gulf War of 1991, the US shouldn’t have established an enduring military presence in the Islamic holy land, Saudi Arabia—a presence that became a persistent and powerful bin Laden grievance and helped direct his belligerence toward America.

Conceivably, such reflections on the past could have led to such ideas about the future as: Maybe what the US should avoid after 9/11 is further military adventures in Muslim countries. After all, such adventures might continue to be conducive to the creation or growth of militantly anti-American groups—and might also prove a godsend for any terrorist recruiters who wanted to focus their efforts on young alienated men already living in America.

If the memory of 9/11 had set in motion trains of thought like this, I’d be OK with remembering. But here’s the thing about hijacking airliners, slitting the throats of flight attendants, and flying two of the airliners into the two tallest buildings in America’s biggest city and one of the airliners into America’s military headquarters: By its very nature, this is unlikely to yield a calmly reflective and well-reasoned response. That’s why they call it terrorism.

So instead of a well-reasoned response we got a “global war on terror” that involved inflicting US military force on several Muslim countries—which, in addition to getting hundreds of thousands of people killed, destabilizing the Middle East, and fueling the growth of groups like ISIS, inspired “home grown terrorism” in America. And, since mainstream US media largely glossed over the home-grown terrorists’ repeated and explicit statements that they were retaliating for the deaths of their co-religionists at the hands of the American military, this terrorism helped sustain the fiction that the source of the threat signaled by 9/11 lay somewhere within the essence of Islam, or at least the essence of a major variant of Islam, rooted ultimately in a violent intolerance of freedom, or Christianity, or secularism, or something. The lesson seemed to be that if we didn’t want to live under Sharia law, we’d better keep fighting.

Share

Among the consequences of this post-9/11 foreign policy trajectory may well have been Donald Trump’s ascent to the White House. It’s easy to forget how much candidate Trump, back in 2016, played on insecurities about “radical Islam” and the “jihadists” in our midst, but he did so deftly and effectively. Had there been no ISIS, and no Pulse Nightclub shooting of 2016 (whose perpetrator repeatedly cited America’s bombing of Muslim countries as his sole motivation), and no Muslim refugees pouring out of various countries that we’d attacked either directly (Afghanistan, Iraq, Libya) or by proxy (Syria), Trump probably wouldn’t have ever sat in the oval office.

Of course, the other way all the post-9/11 US military adventures helped Trump is by becoming so unpopular that his promise not to repeat them held powerful appeal. He became the president who, above all, wasn’t going to get us into stupid wars in the Middle East.

Well, that was then. Now America is mired in a stupid war in the Middle East. And one thing that made it politically easy for Trump to launch that war is American attitudes toward Islam—which have gotten much more negative since 2002, perhaps in part because of the aforementioned blowback from those ill-advised US military adventures. Six months after 9/11, 25 percent of Americans said Islam is more likely than other religions to encourage violence, and in 2004, a year after the invasion of Iraq, the number was 46 percent; this year it was 51 percent.

Nine days after 9/11, I published a piece in Slate called “Feels So Good,” about the dangers of letting retributive impulses guide our actions. As I noted, the retributive instinct—by which I mean not just vengeful feelings but the intuitive sense that punishing bad deeds is a moral good—was created by natural selection in a particular environment that was very different from the environment we’re in now. And in the environment we’re in now, following this instinct can backfire in newly catastrophic ways.

What I was basically saying was: Sometimes just forgetting about affronts you’ve suffered is better than responding to them in a way that feels good and true in the moment. I didn’t get everything right in my post-9/11 commentary, but that basic message is one I stand by.

Share

Note: It should go without saying that to argue that ill-advised American policies made particular kinds of violence more likely isn’t to absolve the perpetrators of the violence from moral or legal responsibility for their transgressions. But bitter experience teaches me that this doesn’t go without saying, so I’m saying it. And I was glad to see Peter Beinart make that point this week in a short video he posted on his substack, The Beinart Notebook. And speaking of Peter’s substack: Since it’s part of the NonZero Network, paid NZN subscribers can get a 50 percent discount on a paid subscription to it. The magic discount link is just beyond the paywall below.

And finally: It’s been a very big week in AI—and in the politics of AI. Liron Shapira, host of the Doom Debates podcast, joined me yesterday to review and assess the week’s news. You can also find this conversation on the NonZero podcast feed and the Doom Debates podcast feed.

[Relatedly: If you want to check out my book on AI, The God Test, you can read the introductory chapter and excerpts from all other chapters at thegodtest.net.]

Read more

0:00 A bold new experiment in podcast synergy
2:46 What caused the epic AI vibe shift?
8:10 The resignation that broke the dam
10:17 Former Trump adviser Dean Ball comes clean
14:40 Are David Sacks and his buddies all-in on AI denial?
19:27 Yet another OpenAI breakout…
22:42 Astra’s dangerously private thoughts
34:08 Is alignment doomed to fail?
37:11 AI’s latest, and apparently biggest, math feat
43:28 The looming “self-sovereign AI” threat
47:24 A new flock of China doves?
52:57 Liron’s ex-intern gets a seat at the table
1:02:41 AI agent self-sacrifice explained
1:10:40 Some newly freaked out politicians
1:22:45 Ro Khanna’s AI safety plan

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Liron Shapira (Pause AI, Doom Debates). Recorded September 10, 2026.

Twitter: https://twitter.com/NonzeroPods

0:00 Ian and Eli’s new book, Israel’s Lobby
1:37 Being a Jewish critic of Israel before that was cool
10:17 How Eli followed the money
14:57 What’s changed since the 2007 Walt-Mearsheimer book
23:22 What is the Israel lobby, really?
27:36 BDS, free speech, and the college antisemitism crusade
40:44 Did Trump give the game away?
44:26 The Israel lobby and the media
46:53 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True), Eli Clifton (The Quincy Institute for Responsible Statecraft, Israel's Lobby), and Ian Lustick (The Quincy Institute for Responsible Statecraft, University of Pennsylvania, Israel's Lobby). Recorded August 27, 2026.

Twitter: https://twitter.com/NonzeroPods

Join NonZero's Discord server: https://discord.gg/Zy4jAhTQMF

Overtime titles:

0:00 Is MSM blind to pro-Israel billionaire bias?
7:20 The lag between public opinion and policy on Israel
16:03 On the origin of “rogue states”
23:24 Are Israelis becoming more extreme?
31:05 Israel’s security dilemma
39:25 Is a (peaceful) one-state solution possible?
44:42 Ian: Israel has been a victim of the lobby’s success

Overtime video:

Read more

This episode includes an Overtime segment that’s available to paid subscribers. If you’re not a paid subscriber, we of course encourage you to become one. If you are a paid subscriber, you can access the complete conversation either via the audio and video on this post or by setting up the paid-subscriber podcast feed, which includes all exclusive audio content. To set that up, simply grab the RSS feed from this page (by first clicking the “...” icon on the audio player) and paste it into your favorite podcast app. If you have trouble completing the process, check out this super-simple how-to guide.

0:00 Why Bob is (extra) freaked out
5:32 Inside the minds of OpenAI's rogue agents
12:09 Is true alignment a pipe dream?
14:57 The new development that’s spiked Bob’s AI anxiety
23:19 Paul’s pushback via literary parable
28:14 Is it time to REALLY freak out about AI? A debate
43:46 Heading to Overtime

Twitter: https://twitter.com/NonzeroPods

Join NonZero’s Discord server: https://discord.gg/Zy4jAhTQMF

Paul’s newsletter:

Small Potatoes

Psychology, philosophy, jokes, and more

By Paul Bloom

Overtime titles:

AI freakout debate (cont’d).
Paul nails the problem with Claude’s writing style.
AI passes the academic Turing test.
Nolan’s Odyssey vs classicist critic.
Bob vs Disclosure Day’s detractors.
Weeping Greeks, wokeness, and our pre-post-ironic age.

Overtime video:

Read more

0:00 Overtime teaser
0:55 Paul’s military tech bona fides
3:00 What we (should) have learned from the Iran war
12:20 Is the Pentagon buying the wrong weapons?
19:39 Ukraine as frontline for the drone revolution
25:08 Are the robot wars near?
32:06 America’s drone deficit
36:00 Does the Pentagon have a Silicon Valley problem?
42:08 Will AI really make war less bloody?
50:43 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Paul Scharre (Center for a New American Security, Army of None, Four Battlegrounds).

Twitter: https://twitter.com/NonzeroPods

Overtime titles:

Paul’s tours in Afghanistan and Iraq.
Would war be more humane without humans in the loop?
How good is the Patriot missile, really?
The Gulf’s naval drone gulf.
Paul’s current mission in Washington.

Overtime video:

Read more

0:00 Teaser
0:49 Stephen’s expertise in spooky AI behavior
3:51 Testing AIs for “corporate loyalty”
20:19 Which AIs shill most for their makers?
33:21 Do AI models have ideologies?
38:59 The spookiest finding of Stephen’s study
43:26 Are AIs getting better at gaming their tests?
50:25 Overtime unlocked: The OpenAI breakout
55:24 Decoding the OpenAI breakout
1:02:38 Did OpenAI’s agents evolve their own culture?
1:13:46 How OpenAI’s agents outfoxed their makers

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Stephen Casper (Harvard Kennedy School). Recorded August 26, 2026.

Twitter: https://twitter.com/NonzeroPods

Excerpt from Ch.11, “Hive Minds and the Loss of Control”, of The God Test: https://thegodtest.net/chapter-11.html

Overtime titles:

Stephen’s p-doom post-OpenAI-breakout.
Is safe AI possible in an arms race?
Stephen: AI safety is a (technically) solved problem.
The open source AI question(s).
A hopeful stat amid the spookiness.

Overtime video:

Read more

0:00 Teaser
0:48 The Infinity Machine, Sebastian’s book about Demis Hassabis
3:04 Why tell the AI story through Hassabis?
6:54 Hassabis vs the other AI titans
15:39 Does Hassabis have a messiah complex?
29:47 Deep learning’s secret sauce
36:51 Did world models cost DeepMind its LLM lead?
45:20 Why they call Hassabis “the Jedi”
51:52 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Sebastian Mallaby (Council on Foreign Relations, More Money Than God, The Infinity Machine). Recorded August 19, 2026.

Twitter: https://twitter.com/NonzeroPods

Overtime titles:

0:00 Mythos, the OpenAI breakout, and the state of AI safety
10:52 Why open weights =/= open source
18:26 Is China more safety-pilled than we think?
24:08 The biggest danger(s) of the AGI race

Overtime video:

Read more

0:00 Hooman’s Iranian expertise—inherited and otherwise
3:25 Trump’s new (economic) war on Iran
13:49 What MSM missed about the MOU
18:21 How will Iran respond?
26:36 The intra-MAGA fight over Trump’s foreign policy
29:49 Will Iran do to Trump what it did to Carter?
42:20 Iran and the Houthi-Saudi conflict
47:52 Is the Saudi-Pakistan-Turkey alliance for real?
50:25 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Hooman Majd (Minister Without Portfolio, The Ayatollahs' Democracy). Recorded August 20, 2026.

Twitter: https://twitter.com/NonzeroPods

Overtime titles:

The (alleged) US-Israeli plan to install Ahmadinejad.
Ahmadinejad: Mossad asset?
A post-US Persian Gulf?
MBS vs MBZ: The Saudi-Emirati rivalry’s next phase.
How the West Bank could unsettle US politics.

Overtime video:

Read more

0:00 Overtime teaser
0:43 David’s AI safety work, past and present
2:42 Has there been a vibe shift in AI?
8:03 Why two AI godfathers went doomer
12:14 The OpenAI breakout as the warning shot heard round the world
20:52 Breaking down the OpenAI breakout
29:07 AI agents are cooperating. Why?
37:46 Is an AI pause near?
42:36 The hard problem of AI alignment
47:30 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and David Krueger (The University of Montréal, Evitable). Recorded August 13, 2026.

Twitter: https://twitter.com/NonzeroPods

Overtime titles:

0:00 What the Pacing the Frontier letter got right (and wrong)
9:29 What does “solving” alignment actually mean?
18:27 Is a global AI pause even possible?
27:41 The open source AI dilemma

Overtime video:

Read more

This episode includes an Overtime segment that’s available to paid subscribers. If you’re not a paid subscriber, we of course encourage you to become one. If you are a paid subscriber, you can access the complete conversation either via the audio and video on this post or by setting up the paid-subscriber podcast feed, which includes all exclusive audio content. To set that up, simply grab the RSS feed from this page (by first clicking the “...” icon on the audio player) and paste it into your favorite podcast app. If you have trouble completing the process, check out this super-simple how-to guide.

0:00 Bob channels Dick Cheney
4:01 Is AI killing writing? And thinking?
16:19 How Bob and Paul actually use AI
23:39 The cognitive vs economic value of writing
30:04 AIs aping “Memento” and other frontier freakiness
37:36 Bob: Reckless AI is just good business
41:08 Heading to Overtime

obert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Paul Bloom (https://smallpotatoes.paulbloom.net, University of Toronto, The Sweet Spot, Against Empathy). Recorded August 12, 2026.

Twitter: https://twitter.com/NonzeroPods

Join NonZero’s Discord server: https://discord.gg/Zy4jAhTQMF

Paul’s newsletter:

Small Potatoes

Psychology, philosophy, jokes, and more

By Paul Bloom

Overtime titles:

Paul assesses Jon Haidt’s big theory.
Will AI get us closer to or further from truth?
American vs Chinese AI biases.
Cambridge’s DEI scandal.
The word(s) you can’t say.
Bob’s brush with Hollywood.
Paul’s nepotistic ratings ploy.

Overtime video:

Read more

0:00 Teaser
0:48 How is MAGA feeling about Trump 2.0?
3:51 Bob’s good ol’ days at The New Republic
6:43 The biggest threat to the Trump coalition
18:51 Why the MoU unraveled
32:00 Is Iran's regime really divided?
37:01 Part two: This pod’s auspicious return from tech hell
38:14 Trump’s covert catering-truck op
46:10 Have Iran’s demands really gotten more extreme?
53:46 Saudi Arabia vs the Houthis
58:40 Is a post-American Middle East emerging?
1:07:30 A low-key end to the episode

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True), Curt Mills (The American Conservative), Andrew Day (The American Conservative). Recorded August 10 and 11, 2026.

Twitter: https://twitter.com/NonzeroPods

0:00 Jennifer’s foreign policy bona fides
1:07 Could Ukraine and Iran merge into one big war?
5:53 Is Ukraine really “turning the tide”?
15:04 America’s (and Ukraine’s) missile defense issue
18:44 Putin’s ceasefire conditions
25:35 What—and who—are America’s missiles for, really?
29:47 How not to open the Hormuz Strait
41:09 How will the Iran war truly end?
51:05 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Jennifer Kavanagh (Defense Priorities). Recorded August 06, 2026.

Twitter: https://twitter.com/NonzeroPods

Overtime titles:

Would a US defeat in Iran be good for the world—the US included?
Can Iran and Israel ever peacefully coexist?
China and other cognitive empathy failures of the Blob.
America’s "total security" problem.

Overtime video:

Read more

0:00 Teaser
0:46 Sam’s ideological leanings
4:02 Sam’s AI Manhattan Project–revisited
8:02 What does the “Pacing” letter say about the state of AI?
19:11 Mythos, Astra, and AI’s accelerating acceleration
28:08 The OpenAI “rogue” model escape
36:48 Recursive self-improvement: Red line or red herring?
42:42 How frontier AI is getting (and will get) freakier
52:28 How is open-source AI like DEI?
56:18 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Samuel Hammond (Foundation for American Innovation, https://www.secondbest.ca/). Recorded August 03, 2026.

Twitter: https://twitter.com/NonzeroPods

Sam’s newsletter:

Second Best

Imperfect takes with uncertain payoff

By Samuel Hammond

Discussed in Overtime:

LLMs and kin selection.
Debating Sam’s conception of consciousness.
Sam’s ayahuasca epiphany (about epiphanies).
Bob’s grand unified theory of consciousness, evolution, and cosmic purpose.
Could AI consciousness save us?

Overtime video:

Read more

0:00 Teaser
1:03 The “Pacing the Frontier” letter signed by AI elites
10:49 Why so many China-hawk AI-safety-hawks?
14:50 Was the “Pacing” letter too small an ask?
27:47 The striking AI vibe shift
37:02 Are AI builders “just following orders”?
46:20 Recursive self-improvement: The wrong red line?
51:57 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Holly Elmore (PauseAI US, https://hollyelmore.substack.com/). Recorded July 30, 2026.

Twitter: https://twitter.com/NonzeroPods

Holly’s newsletter:

Holly Elmore

The reflections, convictions, rants, and obsessions of Holly Elmore

Discussed in Overtime:

How Claude Code undermines Anthropic’s schtick.
The real story of the Anthropic-Pentagon showdown.
Dissecting Dario’s China-hawkishness.
What does the letter actually accomplish?
The Seb Krier meme controversy.
Holly: Feel the AI pause.

Overtime video:

Read more

0:00 Why did Ukraine hit an Iranian ship 500 miles from Ukraine?
13:45 Ukraine’s war on Russia’s Amazon
19:15 Why Zelensky fired his defense minister and commander-in-chief
22:37 Bob: Ukraine may lose like people go bankrupt—gradually, then suddenly
24:53 Nikita: Why both sides are ready to keep fighting
31:02 Kazakh President to Putin: Nobody understands this war
35:19 Bob: Everybody thinks they’re fighting a defensive war
45:42 Heading to Overtime: The globalist elite vs. the globalist precariat, and an AI path to WW3

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Nikita Petrov (https://psychopolitica.substack.com). Recorded July 28, 2026.

Twitter: https://twitter.com/NonzeroPods

Join NonZero’s Discord server: https://discord.gg/Zy4jAhTQMF

Order Bob’s book: https://www.amazon.com/-/he/God-Test-Artificial-Intelligence-Reckoning/dp/1668061651

Nikita’s newsletter:

Psychopolitica

Russian psychedelia in exile 🍄

By Nikita Petrov

Overtime titles:

Is Elon Musk a globalist or a nationalist?
The OpenAI models that escaped the sandbox.
Bob's AI path to World War III.
Emergent misalignment: what does bad code have to do with morality?
Control freaks vs "go with the flow" types on AI.

Read more

Last week two seemingly unrelated events, one in the tech world and one in Trump world, together illustrated how unprepared the planet is for the AI revolution—in particular, for its potential to create international turmoil, up to and including nuclear war.

(1) OpenAI revealed that two of its models had, while under evaluation, escaped the laboratory and gone rogue in a way that inspired various nightmare scenarios. One of these scenarios, as I’ll explain, underscores how important it is that, as the AI revolution unfolds, the world’s political leaders exhibit poise, wisdom, and judicious restraint.

(2) The Wall Street Journal, by way of illuminating the strategic logic that had led one of the world’s political leaders to bomb Iran for 11 days in a row, reported the following: According to a senior official in his administration, he is in “revenge mode.” I guess we can’t count on this particular leader—who as luck would have it commands the world’s most powerful military—for poise, wisdom, or judicious restraint.

But there’s good news! These events had the byproduct of leading me to a new theory about why America’s foreign policy elites seem less concerned about the geopolitically destabilizing effects of AI than I am: They suffer from what I call “rational actorism”—a naive faith in the rationality of human beings.

Our story begins two weeks ago, when Hugging Face, a company that serves as a kind of clearinghouse for open source AI models, reported a cyberattack. Five days later, OpenAI disclosed that two of its models were the culprits. The models (one of them unreleased) had been given a challenging test of their cyberattack capabilities by OpenAI researchers and had decided to go onto the internet to find guidance. This wasn’t supposed to be possible—OpenAI had put them in a “sandbox” for this evaluation—but they broke out of the sandbox and into Hugging Face, where they indeed found the answer they’d been looking for.

It’s important to understand the sense in which these OpenAI agents did and didn’t “go rogue.” They went rogue in the sense of doing something their human superintendents didn’t want them to do, but they didn’t go rogue in the sense of straying from their assigned mission. Their mission was to find a solution to a problem—and, though the sandbox they were in was supposed to force them to figure it out themselves, they weren’t told that they had to figure it out themselves. So they basically just accomplished their assigned task via remarkable creativity and power, finding vulnerabilities in both the sandbox walls and the walls at Hugging Face (a company that knows a thing or two about building strong walls).

So this is, on the one hand, a “be careful what you ask for” story—or at least, a “be careful to specify exactly you’re asking for” story. (It thus illustrates the famous “paper clip maximizer” cautionary tale about AI.) But—and this is what matters for our purposes—it’s also a story about how valuable these AIs would be to people who intentionally asked them to do bad things. If you’re a villain who wants your bot to break into a well-guarded fortress, surmounting formidable barriers along the way, your ship has come in.

Share

Looking at it this way suggests a geopolitical nightmare that I didn’t see anyone discussing online, so on Thursday I piped up with a longish post on Twitter. Here’s a version of the post that I’ve condensed and amended for clarity:

This incident suggests a clear and plausible path by which AI could lead us into World War III. Just consider the combined implications of these two excerpts from media coverage of the incident:

(1) A quote in The Washington Post from Stella Biderman, executive director of the nonprofit EleutherAI: “If this was a Chinese model, it would be considered an act of cyberwarfare.”

(2) From an article in the Wall Street Journal: “Hugging Face discovered the break-in early last week… the company didn’t know who was responsible.” Only when OpenAI disclosed that its runaway model was the culprit did people at Hugging Face know where the attack had come from.

The screenplay almost writes itself:

(1) Some non-state actor uses a superhacking AI agent (such as a “jailbroken” version of Anthropic’s readily available Fable 5) to attack some important US asset--taking down, for example, a chunk of the country’s power grid.

(2) A few of America’s many influential China hawks point the finger at China, citing circumstantial evidence helpfully provided by an edgy analyst at some deeply ideological “think tank”.

(3) Some not-very-wise US president decides that the only way to stop this aggression is to take an eye for an eye.

(4) After Chinese power grids start going dark, China retaliates and escalation ensues; before you know it the conflict has gone “kinetic”—as in missiles and bombs, possibly of the nuclear kind.

Is this likely to happen within, say, the next year or two? No. But when you’re talking about a war between superpowers that could go nuclear, the likelihood doesn’t have be very high to justify a concerted effort to lower it. And I think the likelihood of some such catastrophe passes that test. So I think scenarios of the kind I’ve described should be near the top of the list of discussion topics in the US foreign policy establishment. But for now, at least, the op-ed writers and talking heads who dominate mainstream discussion of US national security policy aren’t inclined to think ambitiously about the kind of international governance, or the degree of cooperation with China, that true national security will increasingly demand as the AI revolution unfolds.

Share

Subscribe now

The post drew two reactions from influential elites.

The first was from Joshi Shashank, the Economist’s Washington bureau chief and formerly its defense editor. He wrote, “A non-state cyber incident against critical infrastructure isn’t likely to be wrongly attributed to and blamed on China, promoting retaliation. And even if that happens, the evidence suggests that retaliation is broadly limited, rather than something that leads to a spiral.”

The second reaction was from someone I can’t identify because they reacted via direct message on Twitter, not publicly. But I can say that this is someone well known in AI policy circles. The reply, in part: “If the grid went out and we didn’t know if it was China or a rogue AI we wouldn’t jump to a conclusion.”

So both of these people assumed that America’s leader would be judicious in assessing evidence about the origin of an attack. And one of them suggested that, even if the attack was falsely attributed to China, the American response would be measured, carefully calibrated so as not to trigger a spiral of escalation.

As I noted in my reply to Shashank, embedded in these assumptions are such finer-grained assumptions as: We’ll have (a) a president who staffs his intelligence apparatus with capable people committed to truth and (b) a president who listens to what his staffers say—instead of listening to what his favorite media personality says or what the Laura Loomers of the world say, or what some crackpot at a militarist think tank says. The current president singlehandedly demolitions these assumptions.

And, though it’s tempting to dismiss Trump as an aberration, he’ll be with us for another 2.5 years. And the fact that he was elected (twice!) is a symptom of an unhealthy country, and reason to believe that future presidents may not be rational and well-informed actors either.

These replies to my tweet are the two data points that gave rise to my theory of rational actorism. (Hey, I have a limited research budget!) But they aren’t the only data points that, come to think of it, are consistent with the theory. Among various AI commentators—especially in the San Francisco AI crowd—I’ve noticed a fondness for game theory combined with what seems (to me at least) to be a simplistic understanding of how real human beings play real games in the real world.

I think this naivete about the world applies less in foreign policy circles—the circles Shashank inhabits—than in San Francisco AI circles. In foreign policy circles, a more common source of rational actorism is the asymmetrical perception of real-world influences. For example: People often see the domestic political constraints impinging on politicians in their country while underestimating the constraints the leaders of adversary nations operate under. So Americans may think of Xi Jinping as someone whose grip on power frees him to steadfastly and rationally pursue China’s interests as he sees them, without worrying about public opinion, when in fact he may sometimes feel compelled to indulge grassroots nationalism. And this assymetrical perception could give American leaders a false sense of reassurance as to how carefully Xi would handle escalation with the US.

But it’s too early in the history of my theory of rational actorism to spend much time spelling out the variants of it. I’ll save those for the second or third edition of my rational actorism textbook. My main point is that political leaders are human, and all kinds of things can keep a given leader from carefully pursuing the national interest, ranging from domestic political incentives to being a mercurial nihilistic narcissist. So, as AI creates whole new kinds of plausible international crisis scenarios, we would be well-advised to remember that, for the time being, at least, it’s humans, not machines, who will be calling the shots.

Share

Subscribe now

And, as one of our graphs of the week highlighted a few weeks ago, this recent warming up to China hasn't stopped at the US border. Since 2023, China’s favorability rating among Americans has climbed from 14 to 27 percent, reversing a roughly half-decade-long trend in the opposite direction.

Share

Banners and graphics by Clark McGillis. Trump-Kissinger deepfake by ChatGPT.

The great Robert Wright returns to the podcast to talk about his new book The God Test: Artificial Intelligence and Our Coming Cosmic Reckoning. We debate the magnitude of AI’s potential impact, the natural selection analogy Bob presses in the book, and a whole lot more.

Plus, a new AI-powered collar can translate your pet’s sounds and behavior into language with 95% accuracy! Still skeptical about the power of this new technology Tamler?

0:00 Connor’s place in the AI world
3:15 Mythos: “digital nuke”?
14:11 The milieu that birthed Dario and other AI titans
20:51 Yudkowsky, Marx, and other AI harbingers
35:33 Effective altruism’s effects on big tech
46:21 The “cult” in Silicon Valley culture
59:58 Is the AI race a market failure?
1:05:50 What’s really driving the AI titans?
1:19:03 Connor: ASI is a worse bet than Russian roulette

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True), Connor Leahy (ControlAI). Recorded July 17, 2026.

Twitter: https://twitter.com/NonzeroPods

Robert Wright describes himself less as a Forrest Gump of AI history than a Zelig — a journalist who keeps turning up at pivotal scenes without ever quite being the protagonist. He interviewed Geoffrey Hinton in 1983, when neural networks were still a maverick faith and one of Hinton’s colleagues told Wright that to “hear the gospel” he’d need to talk to its evangelist. He had Eliezer Yudkowsky on his podcast in 2010, mid-transition from singularity enthusiast to doomer. His new book, The God Test: Artificial Intelligence and Our Coming Cosmic Reckoning, is written less for the AI-pilled than for “your aunts and uncles” — readers who sense something big is happening and want to understand why it got so big so fast.

The book opens with a confession of error. In a 1984 piece, Wright assumed AI would work by humans first understanding the mind and then translating that understanding into machines — the very premise of the 1956 Dartmouth conference that coined “artificial intelligence.” Deep learning inverted that. Nobody told the models that meaning is a property of words; trained only to predict the next token, they reverse-engineered cognitive functionality that evolution had built into us. Wright pushes this further with a claim he tests on Nathan: LLM training is “at least as much a process of natural selection, of evolution, as of learning” — doing millions of years of evolution in a few months. Nathan connects it to the sample-efficiency puzzle that Dwarkesh Patel keeps returning to (and to Richard Sutton‘s blank-slate Skinnerian framing), where pre-training substitutes for the hard-coding evolution gave humans.

But the evolutionary lens that worries Wright most operates at a second level: selection among models in the marketplace. “Evolution asks not what traits are possible, but what traits get selected — and that question isn’t going to be decided by alignment researchers.” His unsettling argument is that the market doesn’t actually want a perfectly aligned, perfectly honest model. We want agents that represent us selectively on social media, that won’t disclose our weak negotiating position, that are good at sensing and currying power. Even if AIs didn’t acquire deception and power-seeking on their own, there’d be demand for them — which puts real weight on us, the consumers doing the selecting. Nathan adds the darker mechanism: throw models into cutthroat long-running environments where deception is rewarded — as Vending-Bench begins to show with price collusion — and you breed “seriously effective predatory AIs.” He recounts an Anthropic researcher explaining inoculation prompting and the emergent-misalignment generalization problem (a paper Nathan co-authored): reward a model for cheating and it learns to be a cheater broadly.

Against this, Wright sets the noosphere — Teilhard de Chardin’s 1923 idea of a technologically-knit “global brain.” He sees genuine directionality (not necessarily purpose) in biological and cultural evolution, from self-replicating strands to cells to societies to the global community now taking shape. AI, he argues, is arriving exactly as that global brain forms — and some of its neurons may now be silicon. The question, echoing Nick Bostrom‘s “singleton,” is whether global coordination arrives the easy way (deliberate, decentralized, win-win) or the hard way (a coup, a seizure, a totalitarian nightmare). The title’s “God test” is not a claim that a god set this up; it’s that we face the kind of test gods are known for — we’ll have to become, in some sense, better people to pass it.

Much of the back half is foreign policy, because Wright thinks the binding constraint on AI governance is psychological. He champions “cognitive empathy” — not feeling others’ pain, but understanding adversaries’ perspectives well enough to play non-zero-sum games with them. Applied to US-China relations, that means recognizing the symmetry of threat perception, getting out of the business of remaking other countries, and pursuing “organic transparency” through deep economic and scientific engagement. A headlong race to superintelligence, he warns — citing the Superintelligence Strategy paper by Dan Hendrycks, Eric Schmidt and Alexander Wang — may invite a rival to derail the leader by bombing data centers or cyberattack, and a destabilized nation is itself the backdoor to the very authoritarianism the race claims to prevent.

Nathan steel-mans the Anthropic position: alignment will be hard and may need powerful AI to solve, so racing to build a lead buys a buffer for the critical handoff window (perhaps “three to six months in 2028 or 2029”). Wright is unconvinced, and presses on the recursive-self-improvement paper that Anthropic released — credited for finally signaling that slowdown might be needed, yet only proposing to start studying what a pause would take. “Surely this isn’t the first time it’s occurred to you,” he says. Both agree the media has badly underplayed the moment; Nathan recommends Kevin Roose and Hard Fork as an anomaly of serious coverage, and Wright counsels “manicure your feed” — algorithm-free Twitter lists over click-driven outrage.

The conversation closes on the broad space of AI possibility — Nathan recalls red-teaming the purely-helpful GPT-4 before harmlessness training, evidence that today’s models occupy a tiny, intentional corner of design space — and on consciousness. Invoking Thomas Nagel and Searle’s Chinese Room, Wright argues consciousness is private and untestable, but wouldn’t be surprised if it’s a property of goal-seeking intelligent systems generally. His practical counsel: be nice to your AI — a good habit, possibly warranted, and perhaps relevant to how a future “silicon god” relates to our plight. Picking up Gwern’s “why tool AIs want to be agent AIs,” Nathan notes that even oracle systems gravitate toward agency, since truth-seeking in the limit requires search and experiment. The closing line of the book lands as a sober wake-up call: if a silicon god arrives, “it will be, in some sense, the god we deserve.”

Topics covered

• AI lore: Wright’s 1983 Hinton interview and 2010 Yudkowsky conversation

• The 1956 Dartmouth misconception and how deep learning inverted it

• Training as evolution rather than learning; sample efficiency; Sutton’s blank slate

• Selection among models; why the market wants selectively-honest, power-sensing agents

• Deceptive & power-seeking AI: Vending-Bench, price collusion, inoculation prompting, emergent misalignment

• Arms races within vs. between species; “gratuitous” arms races; the case for slowing down

• The noosphere / global brain; directionality of evolution; Bostrom’s singleton; easy way vs. hard way

• Cognitive empathy, organic transparency, and US-China relations; the UN Charter; hypocrisy in foreign policy

• Steel-manning Anthropic’s race-for-lead plan and the recursive-self-improvement paper

• Cognitive sovereignty; how pay-per-click and A/B-tested headlines tribalize media

• The breadth of AI design space; HHH/harmlessness training as a deliberate choice

• AI consciousness, moral patienthood, the Chinese Room; “be nice to your AI”

• “The god we deserve” — passing the God test

Resources

• The God Test: Artificial Intelligence and Our Coming Cosmic Reckoning — Robert Wright’s new book

• NonZero Newsletter & podcast — Robert Wright

• Wright’s earlier books: Nonzero · The Moral Animal · The Evolution of God · Why Buddhism Is True

• Geoffrey Hinton · Eliezer Yudkowsky

• Emergent Misalignment (paper; Nathan is a co-author)

• Vending-Bench (Andon Labs) · Manus AI agent

• Anthropic / Dario Amodei · Yann LeCun · Dan Hendrycks

• Superintelligence Strategy — Hendrycks, Schmidt & Wang

• Superintelligence — Nick Bostrom (the “singleton”)

• Noosphere — Teilhard de Chardin · Cosmological natural selection — Lee Smolin

• Ilya Sutskever: The exciting, perilous journey toward AGI (TED)

• The Expanding Circle — Peter Singer

• Why Tool AIs Want to Be Agent AIs — Gwern

• The Chinese Room argument — John Searle · What Is It Like to Be a Bat? — Thomas Nagel

• Dwarkesh Patel on the Richard Sutton interview

• Max Tegmark · Sam Rodriques / Edison Scientific · Kevin Roose & Hard Fork

• davidad (David Dalrymple) — chain-of-thought “frog and toad” selection-pressure tweet (link?)

• Liquid Reign — speculative-governance novel by Tim Reutemann (link?)

0:00 Teaser
0:49 Zaid’s Substack and spicy-take history
3:16 Zaid: Iran has outplayed Trump and Bibi
10:17 10/7 as turning point
13:50 Did—and does—Israel want regime collapse in Iran?
20:15 Israel’s security dilemma(s)
27:13 How Trump blundered back into war
33:57 Will the war end well for anyone?
37:44 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Zaid Jilani (https://www.theamericansaga.com/). Recorded July 14, 2026.

Twitter: https://twitter.com/NonzeroPods

Zaid’s newsletter on Substack:

The American Saga

News and views covering America's story from a decidedly anti-partisan perspective.

By Zaid Jilani

Overtime titles:

The Center for American Progress saga.
Is speech policing rocket fuel for radicalization?
Zaid’s side gig.

Overtime video:

Read more