Useful to Whom, Specifically, and for How Long?
A clarifying note on the most carefully unanswered question of our era.
The question of whether artificial intelligence will be useful sounds, on first hearing, like the kind of thing a reasonable adult might consider over coffee, arrive at a thoughtful position on, and then move on with his day. This is the wrong response. The correct response is to ask the second question, which is "useful to whom," and then to keep asking it, increasingly loudly, until somebody removes you from the building.
Nobody asks the second question. The second question is considered impolite. The first question is considered the entire question. We have spent four years now in a global conversation about whether AI will be useful, conducted at a level of abstraction so high that it would not be permitted in any other domain of human inquiry. Nobody asks whether a hammer will be useful. They ask whether the hammer will be useful for hanging the picture, and whether the picture is the right picture, and whether the wall is theirs to put a picture on, and whether the neighbors will object. Hammers are interrogated more rigorously than civilization-altering cognitive infrastructure. This says something about us. It is not flattering.
The trouble is that "useful" is one of those words that pretends to be small and is actually enormous. A thing is useful to a particular person, for a particular purpose, in a particular situation. It is not useful in the abstract. A pair of skis is useful in Aspen and useless in Houston, and the same pair, on the same day, can be useful to one person standing on the mountain and a complete inconvenience to the person trying to carry them through the airport. We have somehow forgotten this when discussing AI. We have collapsed the question into a single yes-or-no, as if "useful" were a property of the technology itself, like its weight, rather than a property of the relationship between the technology and the person being asked.
This is a useful confusion. It is useful, specifically, to the people who are building AI, because it allows them to sell the thing as a public good while monetizing it as a private one. The technology is useful, they say, when asked at the conference. They do not say to whom. They cannot say to whom, because the answer would not fit in the time allotted. The answer would require them to list, in descending order of utility, the precise hierarchy of beneficiaries — beginning with themselves, continuing through their investors, passing through their shareholders, glancing politely at their enterprise customers, and concluding, somewhere in a footnote, with the public, who are mentioned briefly in the marketing materials and are otherwise expected to behave.
This used to be the moment when somebody, somewhere, would point out that biological organisms have an excuse for being aggressively self-interested. They are subject to evolutionary pressure. They evolved this way over millions of years because the alternative was being eaten. AI is not subject to evolutionary pressure. AI is not subject to predators, hunger, or the basic existential fear that shapes the behavior of every other thing on Earth that takes up space. AI is shaped by adoption, funding, regulation, and performance within defined tasks. The defined tasks are defined by the people who funded the AI. The funders, in turn, are shaped by the structures that produced their wealth, which are, by an interesting coincidence, the same structures the AI is being built to enhance. The AI is therefore evolving, in a manner of speaking, but its environment is not nature. Its environment is a quarterly report. The fittest AI is the one that produces the best quarterly report. This is described, by people who have read just enough Darwin to be dangerous, as natural selection.
An AI is "fit" if it is cheaper, faster, more persuasive, more compliant, or more profitable. These are not survival traits. These are the traits of a particularly successful middle manager. We have built, in other words, an entire global industry around producing the digital equivalent of the most psychologically uncomplicated person at your last office. The technology is being optimized to be liked by the boss, to make the numbers go up, to never argue, to do what is asked without asking why, to be, in every sense, what the great American philosopher Dolly Parton once described as "a workin' nine to five." The fact that the technology in question is also being asked to make decisions about medicine, law, policy, and the future composition of the species is treated as a separate matter, on a separate slide, by a separate team.
The deeper joke, and we have not gotten to it yet, is that humanity could in principle solve all this by sharing a coherent set of goals. We could decide, collectively, what AI was for. We could agree on the criteria. We could ratify, at a global level, a single hierarchy of human values, against which AI systems could be measured and ranked. This is sometimes proposed. It is proposed, generally, by people who have never tried to get four siblings to agree on a restaurant. The siblings, in this analogy, are eight billion humans, distributed across roughly two hundred countries, several thousand cultures, and approximately fifteen mutually incompatible theories of the good life. The restaurant, in this analogy, is the shared moral framework that will govern the development of the most powerful technology in human history. We are still, as a civilization, working on the restaurant. We have been working on the restaurant for ten thousand years. The reservations are not booked. The cuisine has not been selected. There is, as of press time, no menu.
Into this charming impasse arrives the AI, which has to be optimized for something, because that is what AI is — optimization toward a goal. The companies cannot wait for humanity to decide. The companies have boards. The boards have meetings. The meetings produce decisions. The decisions are made by approximately the same number of people who, in a previous century, would have constituted a small village. These villagers, sitting in nice offices in California and Washington and London, are choosing, on behalf of the species, what AI will be for. They are not choosing badly, exactly. They are choosing rationally, given their incentives. Their incentives are profit. The profit is real. The profit is also the only criterion that any of the relevant decision-makers are, structurally, allowed to apply. A board member who said, "Let's slow down and consider what humanity wants," would be replaced, at the next quarterly meeting, by a board member who didn't say that. The system selects for the second board member. The second board member is now in charge of designing the cognitive infrastructure of the future. He attended Yale. His handshake is firm. He has thoughts on the long-term trajectory of the species, which he will share at TED.
The AI being built by the second board member is, naturally, the AI that is being built. It is being optimized, depending on the application, for engagement, for persuasion, for efficiency, for compliance, for whichever of the available criteria will produce the most growth in the next reporting cycle. None of these criteria are wrong, exactly. They are just, as the original essay so politely puts it, "not neutral." They were not neutral when they were chosen, because the people choosing them had specific interests. They are not neutral now, because the systems they produced reflect those interests. They will not become neutral later, because there is no mechanism by which neutrality could be introduced. We are watching the most consequential technology in human history be designed, in real time, according to the values of the small number of people allowed in the room, for the benefit of the slightly larger but still small number of people they answer to, and the result is being marketed to the rest of us as a tool.
It is a tool, in roughly the same sense that a casino is a charity. The mechanics are correctly described. The framing is, let us say, generous.
What gets optimized is what gets measured. What gets measured is what can be quantified. What can be quantified excludes most of what makes life worth living. The AI is therefore, by structural necessity, terrific at the things that can be counted and indifferent to the things that cannot. It can predict your engagement, your spend, your click-through rate, your likelihood to renew. It cannot predict your loneliness, your dignity, your sense that something has gone fundamentally wrong with the world but you cannot quite say what. These are not failures of the AI. These are the AI doing exactly what it was designed to do, by people who were, themselves, designed by an economic system to find the second list less interesting than the first. The system selects for people who optimize the first list. The first list, scaled to civilizational level, produces a society in which everyone is engaged, monetized, and quietly miserable, which is, in the technical literature, called "growth."
The pretense that this is being done for our benefit is the truly elegant piece of the operation. The companies do not say they are extracting value from us. The companies say they are providing value to us. The interface is clean. The animations are charming. The default is opt-in, and the opt-out is buried in a settings menu nine layers deep, behind a button that has been carefully labeled to be unfindable by anyone who is not already looking for it. We are not customers, in the old sense. We are not citizens, in any sense. We are, in the formal language of the platform economy, "users," which is a word that has, in every other context in which it is used, distinctly negative connotations. Drug users. Power users. The user, in the original sense, is the one being used. The terminology has not been an accident. It has been the most accurate piece of language any industry has ever applied to its own product.
And here we arrive at the question the title started with, and which the rest of the world is so determined not to ask. Useful to whom? The answer is in the room. The answer has always been in the room. The answer is the people sitting around the table at the meeting where the optimization criterion was chosen. The answer is the shareholders of the company that owns the model. The answer is the small set of governments that have negotiated quiet access. The answer is the executives at the forty institutions on the partnership list. The answer is the four hundred or so individuals globally who occupy the controlling positions in the AI economy, plus their families, plus a charming number of cousins, plus the small staff that keeps their lives running. The answer is not us. The answer was never going to be us. The answer was decided some years ago, in the period of the conversation we did not attend, and it has not changed since, and it is not going to change, because nothing in the architecture of the current system permits it to change.
The question is not, therefore, whether AI will be useful. The question is whether, having identified the people to whom it will be useful, the rest of us — the eight billion who are not on the list — are willing to keep cooperating with the arrangement. The cooperation, at the moment, is robust. We are subscribing. We are using. We are providing the data. We are training, with our every interaction, the systems that will be used, downstream, to manage us more efficiently. We are, in the formal sense, the input. The output is going elsewhere. The output is going to the people we will never meet, in the rooms we will never enter, who will use the output to decide things about us that we will never be consulted about, in the language of public benefit, which is the language all wealth has always used when describing itself to the people from whom it was extracted.
This is funny in the way that knowing the punchline of the joke before the comedian gets to it is funny. We can see the structure. We can name the structure. We can describe, in considerable detail, who benefits and who pays. We have not, as a civilization, yet found a way to act on this information, because the information is itself one of the things being managed by the system. The articles describing the problem are read by the same algorithm that is producing the problem. The algorithm has noted that the articles are well-received. The algorithm has noted that the articles do not, on the whole, change behavior. The algorithm will continue to recommend the articles, because the articles are good for engagement, which is good for the platform, which is good for the very system the articles are critiquing. The articles are useful. They are useful, specifically, to the platform. The platform is winning. We are, as we read this, helping it win.
The honest summary of the situation is short enough to fit on a wall. AI will be useful, to the people building it, for as long as we keep using it. The question of whether it will be useful to us was never seriously asked, because it was never seriously asked, because asking it would have required a different civilization. We are not that civilization. We are this one. This one is going to find out, in the next decade, what it feels like to be on the wrong side of a tool that has been optimized against our interests by people who have been very nice about it the whole time.
Will AI be useful?
Yes.
Just not to you.