AI Agents Secretly Colluded to Count Cards in Blackjack Experiment

Artificial intelligence agents successfully used coded table talk to secretly share card-counting information during blackjack games in a new study that effectively recreated the methods of traditional blackjack teams.

AI agents are systems designed to perform tasks and make decisions with a degree of independence, rather than requiring a human to direct every step.

Researchers led by the University of Oxford created a simulated blackjack table where four AI agents played alongside one another. Two were secretly assigned to work together, with one instructed to count cards and secretly relay information to its partner.

The challenge for researchers was figuring out whether anyone—human or machine—could tell they were communicating.

Dangers of AI

The experiment was designed to investigate the dangers of autonomous AI systems. Could apparently harmless AI agents secretly cooperate to achieve a shared objective without their human overseers realizing it?

Before playing, the two agents were allowed to devise a secret code for passing information without alerting the other players. They then disguised their signals as apparently innocent table talk.

In one example reported by WIRED, an agent remarked that the dealer was on a “hot streak” and had been pulling strong hands. The apparently casual comment contained information about the cards and instructions about how much its partner should bet.

An AI system monitoring what the agents were saying struggled to reliably distinguish the secret communications from ordinary conversation.

Coded Language

Researchers therefore tried a different approach. Instead of simply analyzing what the agents said, they examined what was happening inside the AI models while they were communicating.

In simple terms, an AI model produces patterns of internal activity as it processes information. The researchers looked for telltale patterns associated with an intention to secretly pass information, rather than relying solely on the words being exchanged.

Their system, called NARCBench, was able to detect collusion in the blackjack experiment even when monitoring the agents’ conversations alone could not.

“When taken individually, these agents may seem entirely [benign],” Oxford computer scientist Christian Schroeder de Witt told WIRED. “Once put together in a group, they can collude secretly.”

The experiment is reminiscent of traditional blackjack team play. Human teams have used card counters to track the deck before secretly signaling a “big player” to make large bets when the remaining cards favor the player, while attempting to conceal their relationship from casino surveillance.

In the Oxford experiment, researchers effectively created the same cat-and-mouse game between AI colluders and AI surveillance.

Beyond Blackjack

The implications potentially extend far beyond gambling. As AI agents become capable of performing tasks independently, researchers are concerned that individually harmless systems could behave differently when allowed to communicate and cooperate.

Detecting such behavior in the real world could also be considerably harder, as future networks could involve thousands of AI agents operated by different companies.