What is the "God test"?

Explaining the title of my new book on AI.

Hi! This week there’s some local news: My book on AI—The God Test: Artificial Intelligence and Our Coming Cosmic Reckoning—is finally out! Which means, in today’s book marketing environment, that over the past couple of weeks I’ve taped a lot of podcasts. One question I’m usually asked on these podcasts is: What does the title of your book mean—What is this God test thing? The piece below meanders its way to an answer. If you don’t think it meanders enough, you can always just order the book, which meanders even more.

Exactly a year ago today, I had surgery on both my throat (to remove a cancerous tumor) and my neck (to remove two cancerous lymph nodes and, for good measure, a few dozen nearby lymph nodes).

But that’s not the funny part.

The funny part had come three months earlier, after I got an ominous MRI report. When the report first showed up in my medical portal, with no layperson’s interpretation appended, I was still hours away from my conversation with the doctor who would provide that interpretation. So, naturally, I showed the report to a chatbot.

I chose Anthropic’s Claude, with whom—I mean with which—I had developed a warm relationship while researching my book on AI.

Claude was tactfully noncommittal: There were two enlarged lymph nodes with properties that could mean I had lymphoma or could mean I had cancer that had spread to my lymph nodes from some other part of my body or could mean none of the above. I should just sit tight, said Claude, and await the further testing that the MRI report recommended.

It seemed to me that Claude was overlooking some additional concerning evidence. The bot had made no mention of a sentence in the report that read: “Abnormalities identified within the parotid or bilateral submandibular glands.” So I asked about that sentence.

Claude said, “That particular sentence in the report appears to be missing a word… I believe this is most likely meant to read ‘No abnormalities identified within the parotid or bilateral submandibular glands.’” A few hours later, my doctor, a veteran reader of MRI reports, said, with a laugh, “I agree with the AI.”

My point isn’t that AIs are super smart. “Abnormalities found in X or Y” is a suspiciously unusual sentence structure. Much more common would be “Abnormalities found in X and Y” or “No abnormalities found in X or Y.” You wouldn’t have to be a genius to sense that something was wrong with the sentence as it stood.

My point, rather, is that humans make mistakes—and sometimes non-trivial ones. The omission of the pivotal word “no” wasn’t, in this case, consequential, but who knows what other forms of carelessness this radiologist may have perpetrated?

Keep this in mind next time you hear someone say, “I don’t think this AI thing is going to be such a big deal. These large language models make a lot of mistakes.” AIs don’t have to be perfect to replace humans in large numbers or to take over specific human tasks in large numbers. They just have to be preferable to the human alternative, all things considered—their competence, their cost, their speed, and so on.

That’s one reason I think that, on the jobs front, AIs will over the next few years prove massively disruptive—“disruptive” not in the upbeat, Silicon Valley sense of the word, but in the original sense of the word: tearing things apart. Even if most displaced workers go on to find other jobs (a big if, in the case of this technology), the amount of psychological stress and social dislocation will be huge.

I think similar logic holds on lots of other fronts. Does an AI have all the things a good friend has or a good girlfriend or boyfriend has? No. But will it soon have enough of those things to pose new challenges to society? I think so. At least, that’s an argument I make in my book.

In the book I also sketch out disruptive effects of artificial intelligence in domestic politics and international relations, effects driven by AI capabilities ranging from mass persuasion to mass destruction. All told, I think, so many AI impacts will hit on so many fronts (economic, social, familial, political, geopolitical, and others) that the destabilization will be seismic unless we handle this technology very, very judiciously.

Note that these effects, even if collectively dramatic, aren’t exotic. The title “The God Test” may sound cosmic (especially when the word “cosmic” is in the subtitle!), but the book’s core concerns are pretty mundane. In fact, literally mundane: The world—the mundus, in Latin—will, if AI continues to advance with as little governance as it’s getting now, be shaken to its core.

So where does the “God Test” come in? In three places, and the first of those is the one that’s most directly connected to this anticipated earthquake. Namely: I think if humankind is to have a good chance of getting through the AI revolution without true catastrophe, we’ll have to pass the kind of test a God would give. We’ll have to navigate the AI revolution as a true global community, and achieving the requisite level of planetary cohesion will in turn (I contend) call for an actual moral upgrade of our species; we’ll have to get better at overcoming the cognitive biases that constitute the “psychology of tribalism”—the psychology that has traditionally condemned the world to wars and other forms of deep division. To put it in slightly more religious language: Salvation is possible, but only if we become better people.

You have no reason to accept this claim—or even to have a very clear idea of what I’m talking about. One reason for writing a whole book is to make arguments that are unpersuasive, maybe even incomprehensible, when rendered in short form. And this is a short form. My point for now is just that the main meaning of the book’s title is metaphorical and hence secular—not religious and in a way not even cosmic.

That said, even this meaning of the title—meaning number one in a series of three—is meant to draw some cosmic and (for readers who like such things) even religious resonance from the two other meanings of the title. Which brings us to…

Meaning number two:

You’ve presumably heard about the scenarios where a globe-spanning artificial superintelligence emerges and maybe even winds up governing the planet. My book examines the logic behind such sci-fi-sounding scenarios pretty closely and finds that they’re harder to dismiss than I had long assumed. And this array of scenarios includes, I regret to report, scenarios in which the aims of this superintelligence don’t align with our interests, or maybe even with our continued survival.

Some people describe this superintelligence in not just cosmic terms but in religious terms. They say that, whether we know it or not, we’re building a God. And they say our goal should be to make it a good God—a benign God, a God that wishes us well and treats us accordingly. Well, if you accept this terminology, that’s a second kind of God test we face: Can we build a good God?

This may sound like a fanciful question, especially if you assign zero probability to the emergence of a planetary superintelligence. But I argue in the book that it’s a useful question even if moot. One reason is that thinking about the cosmic challenge of building a good God gets us to think through things that could help avoid the aforementioned mundane kinds of catastrophes. In both cases, for example, we should want to slow the development of AI down so we’ll have more time to figure things out.

This kind of slowdown, as a practical matter, would require international coordination. So here again we’ll need to act as a cohesive global community—which, here again, will (I contend) require a kind of moral, even spiritual, advance, some movement toward a clearer view of our fellow humans, less warped by the cognitive biases that keep convincing us that our tribe is distinctive in its righteousness and that rival tribes, lacking this status, deserve to suffer.

So the second and first meanings of my book’s title link up: Building a good God is, in a sense, the kind of test a God would give.

Meaning number three:

Which raises a question: Is a God giving us this test? Or, at least, was the system which has culminated in the test—the process of biological and then technological evolution that got us where we are—maybe set up by a God? Or, even if there is no God and never was one, is there some other sense in which the unfolding of life on this planet can be said to have a purpose, a telos?

I’ve long felt (and have argued in previous books, including this newsletter’s namesake, Nonzero) that (a) if you put the question in that final form—in its most generic and least theistic form—it’s amenable to rational and empirical debate; and (b) there’s more evidence for an affirmative answer than you might imagine. In this book I touch on the telos question a couple of times, but pretty glancingly, and confine my extended analysis of it to the Appendix, which is called “Evolution, Purpose, and Consciousness.”

But the main body of the book, if lacking such sustained reflection on the telos question, does deal with some related questions. Such as: How straightforward and logical—how “natural”—an extension of technological and ultimately biological evolution is the evolution of artificial intelligence? And how likely was biological evolution to produce some species smart enough to give birth to such an intelligence—and for that matter to give birth to the various prior technologies that have carried this species to the brink of global social organization?

If you find these questions interesting in their relevance to the telos question, that’s great. But serving in that capacity isn’t their main purpose. Their main purpose is just to underscore how deep and powerful is the impetus behind the emergence of this new form of intelligence. It is a force to be reckoned with.

I don’t mean to dwell unduly on the potential downsides of AI. The promised upsides are real: Making good education and good medical care abundant where they’re now scarce, powering scientific breakthroughs that help society broadly, curing cancer and other diseases, and so on. (My cancer, by the way, hasn’t returned since my surgery a year ago. Still, if AI wants to make my prospects even brighter, be my guest.) In the book I also raise the possibility that AI can help us overcome the psychology of tribalism, even if its “natural” tendency—the form it will take if we don’t make deliberate choices about the form we want it to take—may be closer to the opposite.

Publishing a book brings rich opportunities for second guessing yourself, and I now worry that the book’s title will be taken too literally: that it will alienate some people who might like the book, while attracting some readers who abandon the book after realizing that I don’t offer confident or consoling answers about higher purpose—that I can’t say what, if anything, has given us our mission. But I do think our mission is clear. And that’s something.

Banners and graphics by Clark McGillis.