0:00 Teaser
0:55 Nonzero Doom returns
3:12 A foom in slow motion
7:52 Is AI winter canceled forever?
10:38 Will Google’s Genie 4 lure us into the Matrix?
16:59 Liron’s bull case for Google stock
24:39 Does Google have big-company disease?
28:33 Could a slowdown actually help Anthropic and OpenAI?
30:17 Was OpenAI’s “pause” just PR?
40:06 Social science judgment day is coming
47:17 Why even AI’s upsides could tear society apart
51:17 Does OpenAI’s alignment approach improve on Asimov?
55:11 Sam and Satya pivot on alignment
58:39 Crypto and the AI hacking threat
1:03:47 Could AI crack the math behind encryption?
1:08:57 Liron’s $1,000-a-day AI coding habit
1:15:28 Explaining the missing productivity boost
1:17:31 Does Anthropic’s S-1 sink the AI bubble thesis?

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Liron Shapira (Pause AI, Doom Debates). Recorded October 30, 2026.

Twitter: https://twitter.com/NonzeroPods

This episode includes an Overtime segment that’s available to paid subscribers. If you’re not a paid subscriber, we of course encourage you to become one. If you are a paid subscriber, you can access the complete conversation either via the audio and video on this post or by setting up the paid-subscriber podcast feed, which includes all exclusive audio content. To set that up, simply grab the RSS feed from this page (by first clicking the “...” icon on the audio player) and paste it into your favorite podcast app. If you have trouble completing the process, check out this super-simple how-to guide.

0:00 Teaser
0:31 Paul attains potato hegemony
4:10 Bob’s book: Gateway drug to AI doomerism?
7:42 Nathan Fielder’s view of Elizabeth Holmes (via Paul)
22:27 Bob TORCHES Fielder
34:02 Are art and artist really separable?
38:50 What Trump and Musk say about our times
44:20 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Paul Bloom (https://smallpotatoes.paulbloom.net, University of Toronto, The Sweet Spot, Against Empathy). Recorded October 27, 2026.

Twitter: https://twitter.com/NonzeroPods

Join NonZero’s Discord server: https://discord.gg/Zy4jAhTQMF

Paul’s newsletter:

Small Potatoes

Psychology, philosophy, jokes, and more

By Paul Bloom

Overtime titles:

Bob reveals his anti-Fielder bias.
Gibney’s view of Musk (via Bob).
What kind of show is Widow’s Bay?
Paul: DO talk to strangers.
Paul’s date with an AI companion.
OpenAI’s (semi-)secret sales model.
Are there human domains safe from AI?
All-In’s epistemic short stack.
Fielder vs the field of journalism.
Bob’s post-rant remorse.

Overtime video:

Read more

Andrew Day and Curt Mills of The American Conservative join Robert Wright to talk about the Israeli elections, Benjamin Netanyahu’s escalating rhetorical belligerence, Iran’s strategic calculations and arguable missteps, President Lula’s surprise victory in Brazil, and U.S.-China competition over AI.

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True), Curt Mills (The American Conservative), Andrew Day (The American Conservative). Recorded October 26, 2026.

Twitter: https://twitter.com/NonzeroPods

0:00 Overtime teaser
0:43 Boris’s background and new book, Our Planetary Mirror
3:43 Three ways of seeing the planet—and us
12:47 Resurveying the Gaia “hypothesis”
20:31 Dawkins’s case against Gaia
27:16 Is humanity outgrowing its planetary parent?
32:57 The noosphere: Teilhard vs Vernadsky
48:46 Can we reason our way to global solidarity?
53:58 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Boris Shoshitaishvili (Our Planetary Mirror). Recorded October 15, 2026.

Twitter: https://twitter.com/NonzeroPods

Overtime titles:

Does cosmopolitanism need better myths?
The dark side of planetary thinking.
Towards a “giant organic love blob”?
Bob the virtue ethicist, Boris the aesthete.
The global vs the planetary.
When climate fixes backfire.
Are we AI’s fossil fuel?
The Victorian who saw AI coming.

Overtime video:

Read more

0:00 Teaser
0:23 Connor’s role with ControlAI
1:59 The AI vibe shift in Washington
11:32 How does ControlAI want to control AI?
16:05 Dario decoded
20:57 Are AI safety evals worth much?
29:56 Why Silicon Valley libertarians can’t escape the state
33:37 On the redefinition of superintelligence and alignment
40:14 Is the China AI race a cover story?
47:58 Connor: What really drives Dario is…
51:29 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True), Connor Leahy (ControlAI). Recorded October 16, 2026.

Twitter: https://twitter.com/NonzeroPods

Discussed in Overtime:

How did Yudkowsky become a doomer?
Hard vs soft AI takeover turning points.
What would an AI kill switch look like?
The open source AI question(s).
Is China just aping American AI?
Can Sam Altman be trusted?

Overtime video:

Read more

Below is the final installment of a five-part essay that I’ve been rolling out here on NZN. (Part I is here, Part II is here, Part III is here, Part IV is here, and the entire essay—all 7,500 words of it!—appears in conveniently unified form here.) The essay is a kind of call to action—an argument about what we need to do to successfully navigate the AI revolution and why we need to do it.

Part V: The Epic Challenge of International Governance

For years the term “warning shot” has had a special meaning within the AI safety community. It refers to a high-profile event that could focus attention on the catastrophic potential of artificial intelligence and thus on the need for preventive measures. Typically, the envisioned event involves significant death and suffering. “Chernobyl-sized” is the way some put it, referring to the 1986 nuclear accident that killed dozens of people in the short run and an estimated tens of thousands via cancer in the long run. Others imagine a higher death toll—maybe millions—as the necessary price for enlightenment about the dangers of AI.

Against that backdrop, some of the most unsettling AI stories of recent months look like good news. We learned in April about the superhacking abilities of Anthropic’s Mythos, and three months later came the Hugging Face story—about the swarm of OpenAI agents that escaped their “sandbox” and then broke various other rules. Neither of these developments, so far as we know, entailed any deaths, and both significantly expanded awareness of AI risk.

Indeed, in the wake of the Mythos story, President Trump—a skeptic of international governance, to say the least—agreed with Chinese leader Xi Jinping to initiate a formal dialogue on AI safety between the two countries. And the aftermath of the Hugging Face incident saw not only the “pacing the frontier” letter, but a subsequent statement by Dario Amodei embracing—in principle at least—the idea of a slowdown.

Still… there’s a ways to go. It’s great that the phrase “international coordination” is now more common in the context of AI, but even the more optimistic observers have modest expectations about the fruits of US-China dialogue. The wildest commonly expressed hope is for the eventual emergence of something comparable to a US-Soviet arms accord during the Cold War—a narrow agreement about specific capabilities that doesn’t involve very intrusive monitoring and doesn’t much alter the fundamentally adversarial nature of the relationship between the two nations.

That would be progress, but not as much progress as the US and China, and the whole world, are going to need in the long run. This technology is only beginning to demonstrate its power—or, more to the point, its powers. Given the number of dimensions along which its capability will grow, and the novel nature of the policy challenges it will pose, preserving international and intranational stability will demand much more thorough and creative policy collaboration between the US and its superpower rival than materialized during the Cold War.

Share

No issue more clearly highlights the degree of innovation in governance that AI will eventually demand than the issue of “open source” (or, strictly speaking, “open weights”) large language models—models whose internal parameters are publicly available. Among the implications of open-source AI are: (1) Anyone with the requisite hardware can generate a working copy of the model and operate it independently—free from the monitoring that’s possible with a proprietary model. (Both OpenAI and Anthropic have used their control of the Application Programming Interface, or API, to spot and shut down users they say were using their proprietary models to conduct cyber intrusions.) (2) Anyone with the requisite hardware can amend the model, changing its parameters through customized training—possibly in ways that undermine not only safety “guardrails” that had been added to the model but deeper tendencies toward responsible behavior that had been trained into the model.

All of this helps explain why Nobel laureate Geoffrey Hinton, at least as responsible as any other single researcher for the “deep learning” revolution that paved the way for large language models, has said, “Open sourcing big models is like being able to buy nuclear weapons at RadioShack. . . Bad actors can fine tune them for all sorts of bad things.”

Open source advocates have their own arguments. Open source models, because they so readily elude centralized control, could in theory help prevent an unhealthy concentration of AI power—a danger that, in light of this technology’s capacity for surveillance and persuasion, is already considerable and will grow as models get more powerful. There are also purely economic arguments in favor of open source AI. Still, it seems likely that, as more and more powerful open source models emerge, there will be growing support, in various national capitols, for some kind of constraining policy. Even China, whose emphasis on open source models has in some ways served its interests, will find things to fear about a world with wholly unhindered open source development and proliferation.

Assuming policy constraints on open source AI indeed materialize, what form will they take? It’s hard to envision them clearly, but two assumptions seem sound:

(1) Any restrictive policies that the US or China or any other nation favors will be of limited value if they apply only at the national level. Getting a long series of numbers—a model’s parameters—across a nation’s borders is not a big challenge, even when the number of numbers exceeds a trillion. And once those borders have been breached, the model can be copied over and over, proliferating far and wide. Besides, even if you manage to keep a powerful open source model from thus entering your country, it can still, like powerful AI models in general, damage your country from afar—if, for example, it helps bad actors start a pandemic or stage an international cyberattack.

(2) Any policy that gives nations much reassurance will have to offer them not just transparency beyond their borders but an unusually fine-grained form of transparency. As AI chips grow in power and drop in cost, and such techniques as “distillation” create more efficient AI models, more and more formidable AI capabilities will assume a lower and lower profile. Right now there are consumer-level computers that cost less than $3,000 and can run—in the privacy of your home, with no internet connection required—a recently released and highly regarded Chinese open source model known as Qwen 3.8 27B. The model is no Mythos, but it’s no slouch—and next year the frontiers of DIY AI will expand further. And so too the following year…

Share

Of course, the advance of AI could slow down significantly, or even stop, if some technical barrier to progress is reached. But lately the rate of advance has been speeding up. And both OpenAI and Anthropic are reported to already have the next generation of models in house, helping their workers develop subsequent generations. So it seems close to certain that America’s current approach to regulating AI—which consists basically of not regulating it, and engaging in only modest international dialogue about it—will soon be incompatible with public safety, let alone the public interest in a broader sense. And it seems likely that we will before long need not just robust international dialogue but meaningful international governance and indeed unprecedented kinds of international governance, complete with unprecedented degrees of international transparency.

In fact, I think the requisite transparency will be so fine-grained that we’ll need to cultivate what I call organic transparency—the kind of transparency that grows naturally out of deep economic, cultural, and scientific engagement among nations, as information flows via regular cross-border contact among elites. When two businesspeople from different nations sit down over drinks, you never know what new information will emerge. And when American and Chinese researchers chat at an international AI conference, they’ll learn more than they could have learned by just reading each other’s papers.

Of course, having this degree of engagement across all international borders would be unprecedented. It would require, among other things, a world without wars. Then again, artificial intelligence is unprecedented in the nature and magnitude of its powers. Nobody said managing the most abruptly momentous technological revolution in the history of the planet would be easy.

It’s tempting to dismiss this global adaptation to new security challenges—this revolution in the nature of international relations—as impossible, as irreconcilable with tribalistic human psychology and time-honored patterns of human behavior. But the big stories in AI safety of the past few months—the Mythos and Hugging Face stories—have had an appreciable impact on the psychology of important actors and large swaths of the public. There will be many more big stories to come out of AI, and they could have a galvanizing effect on humankind more broadly and more deeply.

That, at least, is the hope (along with the hope that these stories can be powerfully galvanizing without involving massive lethality). Certainly some powerful psychological jolts will be necessary. After all, the mission that technological evolution has assigned us is to finally do what humankind tried but failed to do in the aftermath of the twentieth century’s two biggest catastrophes, World War I and World War II: cross the threshold to true global community, a community cohesive enough to act with unified purpose. Crossing this threshold would mean that our species had, in a non-trivial sense, become more enlightened. It would also mean we’d realized part of John von Neumann’s original conception of the singularity: Human affairs as we’ve known them would not continue.

Share

P.S.: Again, all five parts of this essay can be found in unified form here. And if a mere 7,500 words isn’t enough of a challenge, you can always check out my book The God Test: Artificial Intelligence and Our Coming Cosmic Reckoning. The introductory chapter and excerpts from all other chapters are here.

Banners and graphics by Clark McGillis.

This episode includes an Overtime segment that’s available to paid subscribers. If you’re not a paid subscriber, we of course encourage you to become one. If you are a paid subscriber, you can access the complete conversation either via the audio and video on this post or by setting up the paid-subscriber podcast feed, which includes all exclusive audio content. To set that up, simply grab the RSS feed from this page (by first clicking the “...” icon on the audio player) and paste it into your favorite podcast app. If you have trouble completing the process, check out this super-simple how-to guide.

0:00 Teaser
0:36 Is Paul pro-gooey-pod?
5:05 Steven Pinker vs Scott Alexander on AI
8:17 A review of AI peer reviewers
18:45 Tipping tips
22:43 The promise and perils of AI companions
37:56 Are mathematicians getting mogged by AI?
42:24 Paul critiques Jon Haidt’s critique of social media
49:18 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Paul Bloom (https://smallpotatoes.paulbloom.net, University of Toronto, The Sweet Spot, Against Empathy). Recorded October 07, 2026.

Twitter: https://twitter.com/NonzeroPods

Join NonZero’s Discord server: https://discord.gg/Zy4jAhTQMF

Paul’s newsletter:

Small Potatoes

Psychology, philosophy, jokes, and more

By Paul Bloom

Overtime titles:

Bob’s beef with Scott Alexander.
The Cornell 7 and evolving consent norms.
Has Dark Matter lost its pull?
Bob’s awkward TED moment.
Trump, misunderstood.
Trump’s not-so-super-intelligent rebrand.

Read more

0:00 Teaser
0:35 Jeremie’s podcast and prescient AI report
6:56 Is scale still (mostly) all you need?
13:10 Jeremie: We’re way past “stochastic parrots”
22:00 Inside the early AI safety ecosystem
33:50 What Yudkowsky got less wrong than most
42:29 Why capability is outpacing alignment
49:25 Were Altman’s fears ever sincere?
55:34 Would pausing AI crash the economy?
1:01:13 Overtime preview: Debating China

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Jeremie Harris (Gladstone AI, Last Week in AI). Recorded October 10, 2026.

Twitter: https://twitter.com/NonzeroPods

Join NonZero's Discord server: https://discord.com/invite/Zy4jAhTQMF

Overtime titles:

Bob’s beef with “kumbaya.”
Will it take an AI Chernobyl to convince the skeptics?
Is China-hawkism the safest option?
The case for “organic transparency” with China.
How alignment progress could backfire.
Has China infiltrated US tech?
Breaking down Dario’s democracy bloc idea.

Overtime video:

Read more

Below is Part IV of a five-part essay that I’m rolling out on NZN. (Part I is here, Part II is here, Part III is here, and all the parts are accumulating here.) The essay is a kind of call to action—an argument about what we need to do to successfully navigate the AI revolution and why we need to do it.

Part IV: International and Intranational Destabilization

It’s common to break the AI safety issue down into distinct kinds of threats: bad actors using AI to build a bioweapon, bad actors using AI as a devastating cyberweapon, AI going rogue and wreaking havoc, and so on. But there’s a broader and in some ways deeper kind of AI danger—a danger that, depending on how exactly it took shape, might involve one or more of these threats but might not, and in any event would have other key ingredients as well. This is the danger of catastrophic destabilization—of multiple consequences of AI pushing a nation, or a whole international region, or even the whole world, beyond the edge of order.

Let’s start with international destabilization—with some of the ways AI could increase the chances of conflict among nations.

When people envision this technology being used to create a bioweapon or conduct a cyberattack, the destructive potential seems clear enough: a pandemic could kill millions if not hundreds of millions of people, and a cyberattack could take out whole pillars of the planet’s electronic infrastructure. But these kinds of weapons also have a property that, though not so obviously destructive, could lead to great carnage. Namely: Their point of origin can be hard to pin down. Unlike, for example, ballistic missiles, they can be delivered without a return address.

This cloak of anonymity could make it more tempting for both nation-states and non-state actors to use these weapons. And, perhaps more important, it could give these weapons an incendiary potential that, through a kind of chain reaction, could draw other kinds of weaponry into an expanding realm of conflict.

When nations face an attack of unclear origin, they can be quick to point fingers—and a consensus can quickly form, on the basis of scanty evidence and pre-existing animosities, that the most feared adversary of the day must be responsible. This in turn can lead to counterattack (perhaps to “establish deterrence”) that leads to escalation. So the age of AI makes it easier for a non-state actor, or a small-state actor, to launch a cyber-attack or bio-attack that leads to nuclear war between the US and China.

This generic prospect—a major war growing out of an AI-abetted attack of unclear origin—is rendered only more plausible by the variety of threat vectors that AI can intensify. In principle, for example, regime change could be orchestrated remotely, via the deployment of millions of bots that seem to be human (and that may sometimes use the commandeered social media accounts of real humans) but have superhuman persuasive powers and use them to ignite a rebellion. Faced with such an AI-astroturfed rebellion, will a nation’s political leaders point fingers at a foreign adversary even if they’re not sure whether the threat has a foreign or domestic origin? Well, political leaders have certainly been known to point fingers abroad when facing domestic discontent. What’s more, they’ve been known to consolidate shaky domestic support by fomenting conflict, even war, with an outside power.

It’s hard to say whether this last threat vector—AI-astroturfed rebellion—will turn out to be a big thing. And this kind of uncertainty is part of the problem: We are entering a period of ultra-rapid advance in a technology that could well pose various new threats to peace and stability at both the national and international level—including threats we don’t clearly anticipate. The good news is that, however diverse and unpredictable the emerging threat portfolio may be, one generalization would seem to hold: Structures for expanded and deepened transparency—structures that give nations a clearer idea of what’s going on with AI abroad—can be stabilizing if well designed. Transparency can not only discourage misdirected counterattacks; it can discourage the initial attacks by making it harder for countries to prepare and execute them without fear of detection.

A common response to the growing threat of cyber-attack and bio-attack is to say that America should increase its “resilience”—strengthen cybersecurity in the private and public sector, build an infrastructure for rapid-response vaccine production, and so on. Resilience is indeed worth building, and it could make a big difference. How big a difference is hard to say. In the realm of cyberattack for example, its efficacy depends partly on the question of whether, as AI supercharges both the offensive and defensive arsenals, the balance of increased power winds up favoring defense in the long run. But in any event there are powerful reasons to doubt that enhanced resilience can bring true security in the absence of robust and innovative international governance.

First, there are the aforementioned new threat vectors, like AI-astroturfed rebellion. For every threat vector there are things you can do to build resilience, but the more vectors there are, the harder it is to find security in a broad sense through resilience alone—especially if the emergence of some vectors is hard to predict.

And, even if we confine the analysis to the two best-known threat vectors—cyber-attack and bio-attack—reliance on resilience faces a big problem. Given how fast AI is advancing, and how unpredictable specific bursts of progress are, and how widely the use of “open source” AI is distributing the power to customize AI to nefarious ends, it may not always matter whether, “in the long run,” AI progress “on balance” strengthens defense more than offense. A single moment of asymmetry between the offensive capabilities of a bad actor and the defensive capabilities of a nation could have massively destructive consequences. So a trajectory of AI progress that repeatedly and abruptly creates such asymmetries could be a source of recurring large-scale lethality and ongoing instability.

Share

This problem—that the speed of AI advance can create moments of incendiary asymmetry—isn’t confined to specific narrow threat vectors. There is a generic version of the problem.

A number of influential Americans—including Anthropic CEO Dario Amodei—have argued that winning the race to “artificial superintelligence” is critical to US national security. The premise is that superintelligence will entail such broad and diverse power that the first nation to achieve it will have unquestioned global hegemony. In this scenario, the US either gains this dominance by winning the race to superintelligence or gets dominated by the other likely winner, China.

But, as Dan Hendrycks, Eric Schmidt, and Alexandr Wang noted in a 2025 paper called Superintelligence Strategy, a race between two superpowers toward superintelligence poses a risk: If both countries believe that reaching that threshold confers enduring dominance on the winner, there is a strong incentive for the would-be loser—the country that lags in the race as the finish line approaches—to pre-emptively sabotage the rival AI program. (The idea of a distinct “finish line” is of course too simple, but for purposes of this analysis, you don’t need to imagine the future being binary in that way; sufficiently rapid acceleration—a singularity-style curve of AI progress—could be enough to inspire existential fear in the lagging nation as the curve starts to “go vertical.”) The sabotage might come in the form of cyber-attack but might also, given the stakes, be kinetic—like, for example, bombing AI data centers or factories that make advanced AI chips.

In either event, and especially in the kinetic scenario, escalation toward nuclear war is imaginable. And the kinetic scenario is rendered all the more plausible by the fact that the only factories capable of producing the world’s most advanced AI chips are within easy striking range of China in territory that China considers its own—and China is currently denied access to those chips.

Hendrycks, Schmidt, and Wang expressed the hope that war between rival AI superpowers could be prevented by a kind of naturally developing deterrence (“mutually assured AI malfunction,” or MAIM). This hope was dismissed by some as unrealistic, and it indeed presupposes, among other things, not obviously plausible demonstrations of unilateral restraint by national leaders. Still, one premise of this hopeful scenario—that a nation leading the race to superintelligence would perceive the growing incentive of its rival to take drastic action—is reasonable. And this could in principle be the basis for a bi-national agreement that brought verifiable commitments of mutual restraint, accompanied by the formalized transparency required for such verification.

A headlong international AI race can bring destabilization not just among countries but within them. So long as the US sees winning a race with China as vital to national security, the race will tend to take priority over domestic regulation that might slow the advance of American AI. (As MIT physicist and AI researcher Max Tegmark has put it, American AI companies “have found that by far the easiest way to avoid being regulated is just to say this two-word sentence: ‘But China!’”) And, given the many realms of life where AI will have a dislocating or disorienting effect—jobs, education, politics, child-rearing, romance, and on and on—rapid AI advance could push American social and political life even further from equilibrium than it now is.

This prospect of domestic destabilization becomes all the more plausible in light of one goal AI companies share with social media companies: to maximize engagement. One way to keep people engaged with an information product is to give them information that feels good—including information that bolsters their belief in their rightness and the wrongness of their rivals and adversaries. So there is good reason to believe that AI, if shaped by market forces alone, will be tribalizing, intensifying divisions among American factions.

There’s an irony in the prospect of domestic destabilization via AI—via its collective impact along various economic, social, psychological, and political dimensions. Prominent among the premises underlying the US foreign policy establishment’s consensus that the US must “win” the AI race with China is the belief that China aims to impose its system of governance on other nations and would use increased international leverage to that end. It’s not clear that this premise is correct (it is so widely accepted as to be rarely examined), but it’s clear how it could backfire: If fear of China-imposed authoritarianism fuels an American sprint toward superintelligence that is essentially untrammeled by regulation, the result could be sharply increased domestic discontent, disorientation, and disorder. And history shows that this is fertile ground for authoritarianism. So authoritarian rule in America—the threat that a sprint toward superintelligence is intended to head off—could wind up being a product of that sprint.

Share

In the scenario I’ve just painted, the destabilization of American society and politics would in some sense be the product of a racing dynamic between the US and China. But it’s worth underscoring how that dynamic works: It’s not that the race with China accelerates the pace of American AI development, pushing it beyond the pace that intense competition among American companies would by itself sustain. Rather, it’s that the race with China precludes the kind of regulation that might otherwise slow the advance of AI. So, by the lights of this analysis, the “natural” rate of AI progress—the rate that American capitalism produces even in the absence of international adversaries—poses a threat to the stability of the United States.

This threat—that the nation could dissolve into violent disarray—may be the most underappreciated of the threats AI poses to the United States (and to other countries). To be sure, many of the individual elements that together constitute the threat get a fair amount of attention. There is broad and growing awareness of the possibility of massive job displacement. The challenges AI poses to education and to child rearing and to the mental health of vulnerable adults are also much discussed. There is even some discussion, if less, of the way ambitious actors could use armies of AI agents to shape our opinions and beliefs and allegiances, creating new religious cults or ideological cults or other tribes with malign potential. And there is some awareness that radically new kinds of scams and cybercrimes are taking shape. Meanwhile, the Hugging Face incident has turned the dangers of rogue AIs into a common discussion topic. But there is still little discussion of the fact that the emergence of these and other challenges within a short time frame—like, the next two or three years—might overwhelm our capacity for social and political adaptation, carrying a whole nation off the rails.

Yet history tells us that, though societies can adapt to technological change, adaptation takes time, and in the meanwhile there is dislocation. And in all of history there has never been as much dislocation, in as short a period of time, as seems to be in the offing.

The prospect of a nation collapsing into chaos is, on the face of it, a national problem, especially when the problem emanates from the nation’s own economy—from technological progress that is driven by companies residing within the nation and is intensified by competition among those companies. Yet even when the problem is in this sense national, national policy alone can’t solve it. No significant slowdown of AI progress is politically feasible unless it is internationally coordinated—involving, at a minimum, both the US and China. Whether the problem is international destabilization or national destabilization, a big part of the solution must lie in international governance.

Share

Coming next week: Part V: The Profound Challenge of International Governance. All parts of the essay accumulate here.

Meanwhile: If you want to check out my book on AI, The God Test, you can read the introductory chapter and excerpts from all other chapters accu.

Banners and graphics by Clark McGillis.

0:00 Teaser
0:32 Alex’s new documentary about Elon
3:07 How Alex’s film turned epic
7:10 Simulation theory + social Darwinism = Elon?
13:36 Inside the mind of Musk
23:46 Elon’s daddy issues
26:35 Elon’s lady issues
29:14 Elon’s electoral propriety issues
36:58 Has Elon gotten weirder over time?
46:42 The damage DOGE could still do
50:14 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Alex Gibney (Going Clear, Enron: The Smartest Guys in the Room, Taxi to the Dark Side, The Bibi Files). Recorded September 22, 2026.

Twitter: https://twitter.com/NonzeroPods

Overtime titles:

Elon’s Trumpian side.
Is Elon a hypocrite on AI?
All-In on the Twitter-DOGE playbook.
Are Elon and Trump the new normal?
The time Bob interviewed Elon.

Overtime video:

Read more

0:00 Teaser
0:34 Garrison’s background and his new book, Obsolete
5:54 Is the left dangerously downplaying AI?
13:01 What is the “obsoleting project”?
17:51 Inside the minds of Altman and Amodei
21:53 Has the job-pocalypse already begun?
33:24 Why AI bio-threats could be extra deadly
37:14 Is alignment doomed?
47:26 “Verifiable rewards” and the OpenAI breakout
53:51 Garrison’s proposal to stop the AI race
1:00:28 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Garrison Lovely (Obsolete). Recorded September 30, 2026.

Twitter: https://twitter.com/NonzeroPods

Overtime titles:

Did Yudkowsky wait too late to speak out?
Culture clash: Anthropic vs OpenAI.
Garrison’s rebuttal to the “but China” argument.

Overtime video:

Read more

Below is Part III of a five-part essay that I’m rolling out on NZN. (Part I is here, and Part II is here.) The essay is a kind of call to action—an argument about what we need to do to successfully navigate the AI revolution and why we need to do it.

Part III: The Inexorable Logic of International Governance

In the wake of the Hugging Face incident, an extraordinary assemblage of AI elites declared that it was time to start preparing the ground for an intentional slowdown of AI progress, a slowdown that would give us more time to build careful AI governance. A letter signed by more than 1,000 people who work at big AI companies, including the chief scientific officers at Anthropic, OpenAI, and Google DeepMind, declared that, as a Washington Post headline put it, it’s time for the “US government to consider slowing down AI.”

But at least as important as this aspiration was the way the letter operationalized it. The US government, the letter said, should “support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.” Recognizing that a slowdown of AI will have to be internationally coordinated to be very effective is a first step toward seeing the new social order that the current technological moment points toward. The second step is seeing how broad and deep this logic runs, how thoroughly the need for international coordination permeates the landscape of AI policy.

The argument for internationally coordinating a slowdown isn’t just that if there’s any hope of getting US and Chinese officials, and US and Chinese AI companies, to support a significant slowdown, they’ll all have to be confident that companies on the other side of the Pacific Ocean will verifiably participate. There’s a second component of the argument that’s at least as important: Leaving aside this practical difficulty with a unilateral slowdown, such a slowdown would have limited value. How much good would it do you, in the long run, to slow AI advance in your country if it continued to race ahead in others? After all, some of gravest threats that you fear advanced AI will bring can cross borders readily.

The most famous hypothetical example of cross-border spillover from AI is in the realm of bioweapons. Anthropic has acknowledged that its Mythos 5 model could make it easier for someone with an undergraduate science degree to weaponize small pox or some other existing pathogen. Worse still, Mythos could, according to Anthropic’s safety report on the model, help “well-resourced threat actors” create a novel bioweapon, such as a virus designed to be much more contagious and lethal than Covid.

Mythos, fortunately, remains unreleased, but Anthropic’s readily available Fable is basically just Mythos with “guardrails” that get the model to foreclose dangerous avenues of discovery by refusing to pursue certain subjects and goals. And guardrails have a long history of being circumvented via “jailbreaks.” Jeremie Harris, CEO of Gladstone AI and co-author of a 2024 report on AI safety commissioned by the State Department, recently said, “We remain in a world where no one knows how to stop jailbreaks.”

A global pandemic, in addition to being a possible consequence of unregulated AI, is an excellent metaphor for other possible consequences. A lethal virus’s indifference to national borders is a property shared by other dangers that AI carries. The Hugging Face incident illustrates the point: An AI agent that doesn’t respect sandbox borders—or the borders of the computer it breaks into after escaping the sandbox—is unlikely to respect national borders.

So, though tighter American restrictions on model testing might prevent a repeat breakout by future OpenAI agents, that alone is of limited value to Americans when Chinese AI models are only months behind American models and other countries are ramping up AI programs. Time and again with artificial intelligence, this is the moral of the story: Whether the concern is a rogue AI or an AI deployed by rogue humans, national security will be increasingly hard to achieve via policy at the national level alone.

Share

And note that AIs can not only, like a virus, travel readily across national borders but, also like a virus, make copies of themselves. I’m not just talking about an LLM spawning a swarm of agents (each of which, though in some sense a distinct entity, has that LLM as its brain and, regardless of where it roams, remains ultimately rooted in the data center(s) where the LLM resides). And I’m not just talking about the fact that some of those agents may in turn spawn other agents. I’m talking about the fact that agents may at some point create a whole new copy of their underlying LLM on a computer somewhere other than the data centers that originally hosted the LLM.

This could happen without the knowledge of any human being, and it could happen at the instigation of a malign human being. In either event, at this point the agents would have escaped control in a deeper sense than they escaped control during the Hugging Face incident. In that case OpenAI could, and did, disable its rogue agents by remote control. But once agents have “exfiltrated the weights” of the LLM to a computer that’s beyond our control, there’s no kill switch.

Given all this—the spawning of agents that may in turn spawn other agents, the real (and in fact already realized) possibility of unauthorized LLM replication—sci-fi scenarios cease to be sci-fi. It’s not crazy to imagine an AI infesting a data center without permission, covertly commandeering some of the center’s computing power, and using that power to sustain agents that carry the AI to other data centers, and on and on: a chain reaction of potentially great length and magnitude.

And, even without imagining quite that dramatic an offensive, you can imagine various critical networks—of satellites, of power grids—being rapidly compromised: Suddenly hospitals or smartphones or streetlights or whatever are inoperative across large swaths of territory. Such is the power of swarms of AI agents—swarms whose size, importantly, isn’t limited by any law of nature.

The two properties that make a big swarm of agents so powerful—cohesion and creativity—were on vivid display in the Hugging Face incident. Hundreds of agents that weren’t supposed to be able to even communicate with each other figured out a way to do that and then hatched an illicit mission and pursued it with a degree of coordination that drove home what a blurry line there is between a hive of minds and a hive mind.

One implication of this awesome display of power is that, as it gets easier to imagine various kinds of outages suffusing an entire nation or multiple nations, it’s also easier to imagine these outages being hard to reverse. There would be swarms of creatively intelligent agents working in concert to thwart your latest plan for reversal.

Share

All of this only underscores—and italicizes and boldfaces—the main moral of our story: Regulation at the national level, even if useful, can’t thoroughly address the coming threats, because in any given case the threat may originate outside your borders. And, with this kind of threat, there’s no such thing as an impermeable national wall—unless the nation in question wants to shut itself off from the international flow of commerce and communication.

AI isn’t the first dangerous technology that warrants international governance. The existence of the Nuclear Nonproliferation Treaty, the Chemical Weapons Convention, and the Biological Weapons Convention reflect longstanding awareness of such dangers. But the limited efficacy of these initiatives is a warning about the political difficulty of crafting strong international governance.

What’s more: The biggest successes in these arms control agreements—the cases where verification of compliance was effective—have involved relatively conspicuous technologies, such as nuclear centrifuges and ballistic missiles with nuclear warheads. And artificial intelligence can keep a very low profile.

To be sure, AI has its conspicuous aspects. The training of a new generation of models involves lots of computing power and electrical power and typically occupies a large swath of land. So an international moratorium on such training runs could be verifiable without a radically intrusive monitoring system—a fact that adds to the appeal of a globally coordinated slowdown as a first step toward keeping this technology under control.

Still, when it comes to the next big step—using this breathing space to craft effective international policies—the relatively low profile of artificial intelligence will pose a challenge. As AI becomes a more and more pervasive part of human life—and more and more powerful models get more and more efficient in their use of computing power and electrical power and hence get less and less conspicuous—the challenge of governance will become less like the challenge of governing nuclear weapons and more like the challenge of governing biotech or cybertech.

So far the lack of serious international governance in those two realms hasn’t led to a catastrophe, but it would be naive to expect this kind of luck to continue as technological advance continues. Indeed, the possibility that the Covid pandemic may have begun with the accidental release of a genetically engineered virus in China suggests that millions of people in America and other countries may have already died for lack of effective international governance. America’s existing regulation of biolabs may be tight enough to prevent this kind of lab leak, but when the thing leaked makes copies of itself and travels invisibly across borders, that isn’t enough to keep Americans safe.

Whatever the origins of the Covid pandemic, the lab leak scenario is of paradigmatic importance. The basic trajectory of AI’s development foreshadows rapid growth in the likelihood and scale of massively destructive disasters—at least, in the absence of strong international governance.

Share

Coming next week: Part IV: The Underdiscussed Threats to International and Intranational Stability. All parts of the essay accumulate here.

Meanwhile: If you want to check out my book on AI, The God Test, you can read the introductory chapter and excerpts from all other chapters at thegodtest.net.]

Banners and graphics by Clark McGillis.

0:00 Cold Open
0:21 Round Two of Nonzero × Doom Debates
1:23 Trump, the FTC, and an Industry Feeling the Heat
5:37 Google Says Gemini 4 Exists
7:41 Greg Brockman Pulls His $25M from Leading the Future
12:53 The Break-Ins Keep Coming
15:23 The Market Wants Agent Swarms
18:30 The End of “Alignment by Default”
20:55 The Prelude to Recursive Self-Improvement
26:14 David Sacks Can’t Explain Dario
30:31 The All-In Echo Chamber
33:47 Scott Alexander vs. Steven Pinker
38:37 Jason Calacanis vs. the Public Benefit Corporation
43:03 The Elephant in the Room Is the AI
46:21 What the Average American Thinks
48:17 From Chatbot to Agent to Swarm
54:20 Boeing, Agent Swarms, and the Off Switch
57:07 Skynet as a Botnet That Trades With Us
1:00:32 Tyler Cowen: National Treasure, Net Negative on AI
1:04:01 Liron’s Doomer DCF Yields a 20x P/E
1:07:11 Interest Rates: Where Cowen Has Me
1:09:12 Does Pinker Owe Us a Debate?
1:12:48 Pinker vs. Instrumental Convergence
1:17:00 Wrap-Up

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Liron Shapira (Pause AI, Doom Debates). Recorded October 01, 2026.

Twitter: https://twitter.com/NonzeroPods

0:00 Teaser
0:51 Paul’s impeccable timing
2:33 Is Dario’s AI slowdown proposal for real?
11:05 Can AI companies police themselves?
15:47 How chip controls can undermine AI safety
25:24 Why the 2024 US-China AI talks failed
30:58 America’s cognitive empathy gap on China
44:25 Taiwan and the chip war
50:39 War over AI: The real doom scenario?
52:25 A possible (grim) path to global AI cooperation
57:39 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Paul Triolo (Albright Stonebridge Group, https://pstaidecrypted.substack.com/). Recorded September 28, 2026.

Twitter: https://twitter.com/NonzeroPods

Paul’s newsletter:

AIStackDecrypted

My personal Substack devoted to AI Stack issues and US China relations

By Paul Triolo

Overtime titles:

0:00 Where US-China AI talks should start
5:32 Mythos as missed diplomatic opportunity
13:29 Why is China more upbeat about AI?
20:45 Will China change its mind on open source?
31:32 Why Paul isn’t a doomer on US-China AI dialogue

Overtime video:

Read more

Below is Part II of a five-part essay that I’m rolling out on NZN. (Part I appeared last Tuesday.) The essay is a kind of call to action—an argument about what we need to do to successfully navigate the AI revolution and why we need to do it.

Part II: The roots and branches of acceleration

It’s long been true that technological advances could facilitate subsequent technological advances. This dynamic is especially common in the realm of information technology, where advances often make collaboration among scientists and engineers more far flung and more efficient. The internet was only the latest in a long line of such innovation accelerators.

In the 1960s, the mathematician I.J. Good imagined an era when the self-reinforcing advance of information technology would make a quantum leap—the era of artificial intelligence. A sufficiently intelligent machine, he wrote, “could design even better machines.” And the resulting machines could repeat the process—so “there would then unquestionably be an ‘intelligence explosion,’ and the intelligence of man would be left far behind.”

It was the science fiction writer Vernor Vinge who fused Good’s musings with John von Neumann’s use of the term singularity. In a 1993 paper that cited both men, he wrote, “When greater-than-human intelligence drives progress, that progress will be much more rapid. In fact, there seems no reason why progress itself would not involve the creation of still more intelligent entities—on a still-shorter time scale.” He suggested that, “it’s fair to call this event a singularity (‘the Singularity’ for the purposes of this paper).” The Singularity, he wrote, would be “a point where our models must be discarded and a new reality rules.”

This sounded like pure science fiction at the time—and, actually, for decades thereafter. But in early June of this year, the AI company Anthropic published a paper that, though it didn’t mention the singularity, brought to mind the title of Kurzweil’s book: The Singularity is Near.

For the past few years, Anthropic said, each generation of its large language model, Claude, has been playing a role in developing the next generation’s model—and, moreover, a larger role than the previous generation had played. This isn’t just a matter of LLMs writing more and more of the computer code. Though the shifting of this burden from human to machine has been dramatic, more notable is the growing ability of LLMs to participate in the research process—doing experiments to see what innovations could make the next generation better than the last.

Claude, says Anthropic, “can already match or outperform skilled humans at executing a well-specified experiment.” The more challenging part of the research chain is “deciding what experiments to run, interpreting what comes back, and figuring out which ideas to try next.” And, though in this area “large performance gaps” between human and machine persist, they’re shrinking. Anthropic says that in April, “Claude-powered agents were given an open problem in AI safety—roughly, can a weaker model reliably supervise a stronger one?—and were left to solve it. This involved proposing hypotheses, testing them, sharing findings with parallel agents, and iterating.” The agents got some guidance along the way but they “designed every experiment themselves.” An engineer who provided that guidance said the agent’s performance was on par with that of a good junior colleague and declared, “The future is now.”

Well, it depends on what you mean by the future. In the strictest singularity scenario, this engineer would no longer have a job. We would have reached “recursive self-improvement,” the point where the machines just keep building better versions of themselves, with no humans in the loop. But according to the Anthropic paper—titled “When AI builds itself”—even that point may not be far off. Current trend lines point to “an AI system capable of fully autonomously designing and developing its own successor.” The paper continued, “We are not there yet, and recursive self-improvement is not inevitable. But it could come sooner than most institutions are prepared for.” And one consequence might be to “increase the risks of humans losing control over AI systems.”

Share

This may sound alarmist—and there are people who accuse Anthropic of self-servingly hyping AI risk—but in a sense the company, by focusing narrowly on recursive self-improvement, is underplaying the threat we face. The self-accelerating character of AI progress could bring radically disruptive change well before this threshold arrives—and even if it never arrives. We could still reach a point where, as von Neumann put it, human affairs would not continue “as we know them.” In that sense, the singularity could be nearer than Anthropic is suggesting.

In fact, there are signs that it is—signs of more and more dramatic AI advance. Machines have started solving problems that the world’s best human mathematicians had struggled with for decades. Superhacking AI agents like Anthropic’s Mythos have been finding software vulnerabilities that had escaped detection by legions of human engineers. But the most vivid and possibly the strongest piece of evidence that AI is driving us faster and faster toward an epic threshold is something that’s taken shape not in recent months but in recent years: a graphical depiction of AI progress developed by a nonprofit called METR—a depiction that, within the AI community, has reached iconic status.

METR’s researchers measure how long it would take a human to do tasks that the most powerful large language models can do, and in early 2025 they reported a pattern that had persisted since the dawn of the LLM age six years earlier: The human “task time” that these models can match had been doubling roughly every seven months. That’s an exponential growth rate that, when plotted on a graph, looks like it’s “going vertical” (a common feature of singularity-signifying curves from math and physics). And as if that weren’t enough: Since 2025 the doubling time has been shrinking; the curve is going really vertical.

This curve represents improvement in various AI skills—and, most important, in a kind of meta-skill: autonomy. Autonomy is the ability to pursue an assigned goal flexibly, recognizing and overcoming obstacles, backtracking and retooling in the face of failure. Whether you’re trying to prove a math theorem, research a history dissertation, or do various kinds of jobs that AI agents may encounter in the workplace, autonomy is critical to success. That’s why the market is encouraging the big AI companies to build more and more autonomous AI agents—guidance the companies are following with furious intensity.

Unfortunately, AI autonomy can be dangerous. It’s the reason that in July hundreds of AI agents were able to escape the supposedly secure “sandbox” that OpenAI had put them in for evaluation and break into computers at a company called Hugging Face, looking for information that could help them get a good grade from the evaluators. They pursued their assigned goal more flexibly than the goal-givers had anticipated.

The reaction to the Hugging Face incident was voluble and, if you share my vision of the future, heartening. Not only did the burst of publicity spread awareness of the dangers of rampant AI autonomy; the reaction within the AI community signaled awareness that the response to such dangers will naturally carry governance beyond the national level, to the international level.

Coming Friday, Oct. 2: Part III: The inexorable logic of international governance. All parts of the essay accumulate here.

Meanwhile: If you want to check out my book on AI, The God Test, you can read the introductory chapter and excerpts from all other chapters at thegodtest.net.]

Share

Banners and graphics by Clark McGillis.

0:00 Teaser
0:59 Jill’s book OK Boomer, Let’s Talk
5:43 When is generalizing about a group okay?
9:50 The world boomers built
16:49 Are boomers really uniquely selfish?
27:11 Is blaming boomers counterproductive?
31:14 Who’s actually driving the generational divide?
38:57 The generational blame spiral
50:25 Millennials vs Gen Z
53:31 Does America have a gerontocracy problem?
58:39 Why Kenyans are looking toward China

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Jill Filipovic (OK Boomer, Let's Talk; The H-Spot). Recorded September 15, 2026.

Twitter: https://twitter.com/NonzeroPods

Below is Part I of a multi-part essay that I’ll be rolling out on NZN over the next couple of weeks. The essay is a kind of call to action—an argument about what we need to do to successfully navigate the AI revolution and why we need to do it.

Part I: The kind of singularity that’s approaching

In late May, Demis Hassabis, head of Google DeepMind, said “We’re at the foothills of the singularity.” In late July, Sam Altman, head of OpenAI, said “We’re now, like, in the singularity.” That same week, Elon Musk said, “We are in the singularity, just the very early stages of it.” If you’re not steeped in the culture of artificial intelligence, you may have a question: What are these people talking about?

They’re talking about an idea that people in their milieu have been talking about earnestly for decades—and an idea that most people outside of that milieu have been either unaware of or dismissive of. But over the past year, and especially the past few months, more and more people have come to take the idea seriously—and many now believe that, as futurist Ray Kurzweil put it in the title of his 2005 bestseller, “the singularity is near.”

In one sense, it’s hard to argue with them. A big part of the meaning of “the singularity” is accelerating and ultimately transformative technological change—change that happens faster and faster until, at some point, you’re in a whole new world. And there are definitely signs that growth in AI capability is accelerating, portending big and possibly breathtaking progress in science and dramatic change in other realms. The resulting sense of momentous acceleration is why, though some people seem giddy about the singularity, eager for the blessings progress can bring, others are terrified, imagining a world that spins out of control.

But there’s also a sense in which Hassabis, Altman, Musk, and the many other heralds of the singularity are wrong. There’s an important part of the meaning of “the singularity” that doesn’t apply to the current situation. And understanding that fact—seeing the difference between “the singularity” in its full modern meaning and the watershed in technological evolution we’re actually approaching—could be salvific. It could empower us to keep the world from spinning out of control as the AI revolution unfolds.

The first person to use the term “singularity” in the modern sense of the term seems to have been John von Neumann. Von Neumann, a physicist and mathematician who helped design the atom bomb and co-invented game theory (among many other feats), used the word while talking to the mathematician Stanislaw Ulam. The only record we have of the discussion is Ulam’s recollection, written down after Von Neumann’s death in 1957:

“One conversation centered on the ever accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue.”

The term “singularity” would have been familiar to von Neumann from physics as well as math. In both, it refers to a point where existing analytical tools lose their usefulness; what lies beyond is unclear—unpredictable and maybe incomprehensible. A common example is a black hole: As you approach its center (which isn’t recommended), the strength of gravity approaches infinity, and the known laws of physics break down.

Hence the full modern meaning of the term “the singularity”—not just the relentless and ultimately transformative acceleration of technological change, but an acceleration whose transformative consequences are inherently unknowable: “Human affairs as we know them,” as von Neumann put it, won’t continue, and it’s not clear what will replace them. Beyond a black hole’s “event horizon”—the point where gravity prevents even light from escaping—lies deep mystery.

Share

This image—of an imminent and unpredictable metamorphosis, a future that is opaque but assuredly strange—helps explain the allure of the concept of the singularity. If an episode of a sci-fi TV series ended with titans of technology announcing the arrival of such a thing, you’d tune in next week, right?

Here’s my version of a plot spoiler:

I agree with these tech titans that AI will bring transformative change—a whole new world, socially and politically—and that the technological force behind this change is growing faster and faster. But as for the other half of the meaning of “the singularity:” I don’t think there is a metaphorical “event horizon” that shields the future entirely from our view. I think the singularity is clear.

Or, at least, clear enough. The broad contours of the future are visible. It’s possible to discern the paths available to us and the different kinds of worlds they will lead to. Then, if we can summon the requisite political and moral resources, we can create the best of those worlds.

Here is our menu of options: (1) a world of chaos and conflict and intermittent catastrophe, conceivably featuring human extinction; (2) a world that features expanding and deepening international governance that eventually spans the planet and qualifies for the term “global governance.”

In a way, that’s too simple, because option two—global governance—subdivides into two options: good global governance and bad global governance. And bad global governance, in the age of AI, could be very bad, because this technology can be a hugely powerful tool of authoritarian surveillance and control. Even if we leave aside the most cinematic sci-fi doomer scenarios, in which the authoritarian ruler is made of silicon, there are global governance scenarios—realistic scenarios, I’m afraid—that qualify as dystopian.

So here is the full menu of options: (1) good global governance; (2) bad global governance; (3) an enduringly chaotic hellscape.

In all of these scenarios, the current world order—a bunch of nation-states and loose blocs of nation-states that function without systematic global coordination yet together keep the planet largely stable—would cease to exist. Either there would be considerable migration of governance up to the global level (even as nation-states retained significant powers) or stability would no longer be the rule. In any event, “human affairs as we know them” would not continue.

To understand why change this dramatic is not just plausible but inescapable, it helps to understand a dimension of the concept of the singularity that von Neumann, so far as we know, didn’t mention: a positive feedback mechanism that drives the acceleration of change.

Coming later this week: Part II: The roots and branches of acceleration. All parts of the essay accumulate here.

[Meanwhile: If you want to check out my book on AI, The God Test, you can read the introductory chapter and excerpts from all other chapters at thegodtest.net.]

Share

Banners and graphics by Clark McGillis.

0:00 Teaser
0:48 Nick’s singular place in AI history
4:21 Where is Nick on the pause-accelerationist spectrum?
13:13 Utilitarianism vs the “moral parliament”
19:13 Is a “singleton” coming? What kind?
32:32 Is global cooperation on AI doomed?
47:10 The pros and cons of reading an AI’s mind
52:43 Do AIs deserve moral status?
1:10:43 AI, evolution, and the cosmic purpose question

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Nick Bostrom (Superintelligence, Macrostrategy Research Initiative, Deep Utopia). Recorded September 16, 2026.

Twitter: https://twitter.com/NonzeroPods

This episode includes an Overtime segment that’s available to paid subscribers. If you’re not a paid subscriber, we of course encourage you to become one. If you are a paid subscriber, you can access the complete conversation either via the audio and video on this post or by setting up the paid-subscriber podcast feed, which includes all exclusive audio content. To set that up, simply grab the RSS feed from this page (by first clicking the “...” icon on the audio player) and paste it into your favorite podcast app.

If you have trouble completing the process, check out this super-simple how-to guide.

0:00 Teaser
0:46 This episode’s surprise vibe shift
2:30 Middle East update: Houthis vs. Saudis
15:21 What’s really driving Dario (et al.)?
25:14 Debating effective altruism’s effectiveness
30:18 The trouble with big-tech Kremlinology
34:08 Was “learn to code” just sleight of hand?
40:25 Bob’s Fukuyama name drop
42:11 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True), Derek Davison (Foreign Exchanges, American Prestige, Discontents), and Daniel Bessner (American Prestige, Imperialist Realism, University of Washington). Recorded September 17, 2026.

Twitter: https://twitter.com/NonzeroPods

Danny’s Substack:

Imperialist Realism

It's easier to imagine the end of the world than the end of American Empire.

By Daniel Bessner

Overtime titles:

Is international cooperation doomed under capitalism?
Trump’s Iran War woes get woeful-er.
The Substitute and other prestige entertainment.

Overtime video:

Read more

0:00 Teaser
0:58 Where Zvi’s famous newsletter got its name
5:46 Zvi’s biggest AI fear
13:26 Why is ChatGPT’s mind getting harder to read?
18:51 Is self-sacrifice really such a surprising trait in agents?
33:48 AGI + agent swarm = superintelligence?
46:29 Appraising the elements of Dario’s “pacing” proposal
1:02:14 Heading to Overtime

Robert Wright (Nonzero, The Evolution of God, Why Buddhism Is True) and Zvi Mowshowitz (Don't Worry About the Vase). Recorded September 15, 2026.

Zvi's newsletter:

Overtime titles:

Are Dario and Sam Altman sincere about AI safety?
Imagining a sci-fi AI takeover—without the “fi”.
Zvi’s path from Magic to Covid to AI.
Why did Google drop off the frontier?
Noam Shazeer’s weird (re-)exit from Google.

Read more