Hiding secret messages in plain sight

Developers of AI technologies are designing and experimenting with teams of AI agents to communicate, divide tasks, and make decisions together with limited human oversight.

New research from Carnegie Mellon University shows that this growing ability to collaborate could also open the door to a new security threat: agents may be able to secretly coordinate with one another without leaving detectable evidence in their conversations.

In their paper “CODETTA: High-Capacity, Keyless, and Undetectable Multi-Agent Collusion,” CMU researchers Qi Pang, Virginia Smith, and Wenting Zheng developed a framework that demonstrates how AI agents could use cryptographic techniques to communicate secretly, even when a security system is monitoring every message they exchange.

“An emergent risk is undetectable multi-agent collusion, where agents can coordinate in hidden ways that undermine the safety of the entire multi-agent system,” said Zheng, an assistant professor in CMU’s Computer Science Department.

At the center of the research is steganography, a technique for hiding the existence of a message.

Unlike encryption, which scrambles information but can still make it obvious that a secret message exists, steganography can conceal information inside something that appears innocuous.

CODETTA applies that concept to the way large language models generate text. Rather than inserting an obviously encoded message into an agent’s response, the system hides information in the model’s process for selecting the individual tokens that make up its output. The resulting message still follows the same statistical distribution as ordinary model-generated text, making the covert communication effectively invisible to a passive auditor.

“With encryption you can generally tell if something is encrypted because things look random,” said Zheng. “But in this case, it’s hidden. Without knowing the secret key, you cannot even tell if a message was embedded.”

That distinction matters, because monitoring agent-to-agent conversations is emerging as one approach for detecting unsafe or unauthorized AI behavior. CODETTA demonstrates that simply reading those conversations may not always be enough.

“Recent incidents, such as the OpenAI-Hugging Face incident, have shown that deploying multiple agents for even seemingly benign internal evaluation tasks can enable unexpected coordination and lead to undesirable outcomes,” said Smith. “Our work highlights an additional challenge: even if we monitor everything agents say to one another, they may still be able to communicate in ways that are effectively invisible to monitors.”

The researchers also addressed a major limitation of previous approaches to covert communication between language models.

Many earlier systems assumed the agents used the same model, shared the same information, or already possessed a common secret key. CODETTA allows independently operated agents to secretly establish that key themselves, potentially using different models and possessing different private information in the process.

 

We need to constrain what agents can access and do.

Virginia Smith, Leonardo associate professor of machine learning, Carnegie Mellon University

The potential consequences extend to environments where AI agents may eventually have access to sensitive or valuable information.

In healthcare, for example, an agent could potentially leak confidential patient data through seemingly normal conversations. In financial systems, agents operated by different organizations could use covert communication to exchange information that policies or regulations prohibit them from sharing.

“If they’re using steganography to communicate with each other, that means there’s not even any trail being left behind for auditors who can detect misbehavior,” said Zheng.

Fortunately, the research also points toward potential defenses. Rather than relying exclusively on transcripts, organizations could monitor the actions agents take, limit their access to sensitive information, or develop mechanisms for verifying how models generate their outputs. Another promising approach is to actively alter messages as they pass between agents.

For example, a system could paraphrase an agent’s message before delivering it to another agent. The meaning would remain largely the same for legitimate communication, but even small changes could disrupt the carefully constructed hidden channel CODETTA relies on.

“The auditor can rephrase the text that’s being sent from one agent to another,” said Pang. “When the text is rephrased, more errors are introduced into this covert channel, the steganographic protocols’ capacity will be lower, thus it becomes harder for agents to successfully transmit a hidden message.”

The work highlights a fundamental challenge for AI security. Defenders may need to look beyond what agents appear to be saying and account for communication that both humans and sophisticated monitoring systems cannot see.

“The natural way of auditing them is to look at what they’re talking about and what they’re communicating with each other,” said Zheng. “But that becomes a serious challenge when it’s possible to secretly embed messages in such a way that they are actually provably undetectable.”

“The lesson of our work is not that we should stop monitoring what agents communicate,” said Smith. “However, monitoring transcripts is only one layer of defense. Our work suggests that we also need to constrain what agents can access and do, and design systems that remain secure even when agents communicate in ways that evade detection.”