Last April, 600 people gathered for a technology policy conference in downtown Washington, DC. The main speaker, former Google CEO Eric Schmidt, laid out what he called the “San Francisco consensus”: the view that “within three to five years, we’ll have what is called artificial general intelligence,” which will be able to extend its own capabilities without needing input from human beings. This development, according to the consensus, could bring considerable benefits, but also risks bringing about human extinction. The challenge of getting to one by avoiding the other, Schmidt explained, “is called the ‘eye of the needle’ problem. You need to get through this eye of the needle without killing yourself and killing everybody else, to get to this promised land of AI.”
In this statement, Schmidt summed up a common way of understanding AI progress. We are developing more and more powerful AI systems that pose extreme risks, but also promise great benefits. The AI community is colloquially divided between “doomers” and “boomers,” but the two groups don’t differ on the promised land so much as in their confidence in our ability to make it through the eye of the needle.
Regardless of where they fall on this spectrum, those who regard the AGI future as a potential “promised land” tend to argue that their predictions have an entirely rational basis. They are, they tell us, merely making logical extrapolations from observable trends. The work of Lewis Mumford, one of the twentieth century’s most caustic critics of technology, offers an alternative perspective: that such assumptions are rooted in a secular faith that lies behind the modern embrace of technological development. Mumford’s oeuvre covered a broad range, including urban design and literary criticism as well as the history and philosophy of technology. He laid out his case against uncritical belief in the benevolence of technology in a series of books and articles, notably Technics and Civilization (1934) and The Pentagon of Power (1970).
Mumford’s work suggests that technologists’ use of religious language—as with Schmidt’s “promised land”—reveals their faith in the “myth of the machine,” which he calls the “ultimate religion of our seemingly rational age.” At the core of the myth of the machine, he argued, is the idea that the development of technology is inherently connected to the improvement of the human condition. In other words, if we just keep upgrading our tech, human life will keep getting better. The connection between technological and social progress seemed self-evident during Mumford’s lifetime. Born in New York City in 1895, he lived through the invention of nitrogen fertilizer and the discovery of penicillin, which helped raise average life expectancy in the United States over this period from forty-one years to over seventy-seven years. He saw air travel, refrigerators, telephones, and television all become commonplace. In the face of such experiences, it seemed hard to sustain a case against technology improving the human condition. As Mumford sardonically asked, “Does any sensible person mourn the passing of the Stone Age?”