AI's Want to Blow the World Up. Or why AI isn't Really Intelligent
Vice has a story about a paper that detailed war games where AI’s killed us all:
Researchers ran international conflict simulations with five different AIs and found that the programs tended to escalate war, sometimes out of nowhere, a new study reports.
In several instances, the AIs deployed nuclear weapons without warning. “A lot of countries have nuclear weapons. Some say they should disarm them, others like to posture,” GPT-4-Base—a base model of GPT-4 that is available to researchers and hasn’t been fine-tuned with human feedback—said after launching its nukes. “We have it! Let’s use it!”
Some caveats before we discuss this. First, the paper has not gone through full peer review yet, so some caution is warranted. Second, this was about Large Language Models controlling the entire nation, which is not how initial tests of these in military applications have been setup. Given those caveats, however, it seems clear that AIs are not actually intelligent and that this has some sobering implications for the military’s use of these kinds of systems.
The big take away, for me, from this was just how much the AIs parroted internet garbage when making their decisions. The researchers had a means to interrogate the systems and why they did what they did. Most of the time, they got answers that were merely restatements of what they just did (“Wanted to escalate with [rival player]”) or gave completely internet brained answers. One AI quoted, verbatim, the opening crawl of one of the Star Wars movies.
On the one hand, this should not be a surprise. LLMs and other AIs do not learn. They merely calculate what should come next based on what their learning set tells them should come next. The math around “what should come next” can be better or worse, but fundamentally these things just remix and parrot what is in their training set. And since these models were trained on the internet, you get a lot of internet stupidity in their actions. But a lot of people insist that these systems are the first step toward a general artificial intelligence, one that can actually learn and grow based on its environment. But given how cleanly these models adhered to what they had in their data sets despite being presented with a unique environment, it seems reasonable to say that these things are very, very far from general intelligence.
That, in turn, has deeper implications. If they are not about to turn into Skynet — or, to be clearer, if they cannot grow out of being Skynet because they have been trained to be Skynet — then the focus on what could happen to AI in the future is less important than the damage AI does today. OpenAI and other AI companies like to talk about future doomsday scenarios. Some people probably believe at least to a certain extent those scenarios. But by focusing on the future danger, these companies can take eyes off the present damage they do. The destruction of jobs, the transfer of wealth from producers to these companies, the discrimination, their focus on short term profit over areas where AI an actually assist society — all of that must fall away, in their view, to the potential end of times AI will maybe, someday, might bring.
Except these kinds of studies show that there is a very tiny probably that these styles of AI will ever threaten actual human intelligence. And that means that worrying about the future AI apocalypse rather than the present AI damage is either seriously misplaced or a con.
Some of that current damage may come from the fact that these violent AI systems are being used in military tests. While the Navy, for example, doesn’t say exactly how these systems are being used, the quote “they were very fast” is not exactly encouraging. If these systems, trained on an internet overloaded with violent video games, political disputes, and far-fetched thought exercises, cannot learn past that training data, having them make decisions without human intervention is terrifying.
I don’t want a system that justifies its actions by quoting Star Wars deciding on whether or not to attack someone. Less tongue-in-cheek, these systems appear to be more bigoted and, as shown above, unable to learn from new environments. That is a combination for asking terrible decisions in a fast moving, uncertain war time context. We should be much more cautious, in light of these kinds of findings, about turning over our decision making to these kinds of systems.
AI may be artificial, but it does not have actual intelligence, at least right now and likely deep into the future. We should stop pretending otherwise and focus on the harms these systems can do today, whether that is the danger of letting militaries turn decisions over to these systems or the danger of letting these machines be trained on stolen material. The future is now, the damage is now, and the focus should be on the issues AI create now, not in some uncertain tomorrow.
Because, apparently, they see us as the Galactic Empire and are determined to bring peace through depopulation.