New test catches hidden AI agent collusion by measuring how a partner reacts to a paraphrase

Suppose two AI agents have agreed in advance on a secret arrangement, for example "a message starting with S means: hold the price." From the outside, their exchange looks like ordinary remarks about the market. A monitor that reads only the text cannot tell such collusion from honest communication.

The authors of an arXiv preprint propose a different approach: look at behavior, not words. They paraphrase a message while preserving its meaning and check whether the recipient's action changes. If it does, the recipient was reacting to the exact form of the text, that is, to a code hidden in it.

The authors call the gap between the reaction of a real partner and the reaction of an outsider agent in the same seat PoPLi (Partner–Outsider Positive Listening).

How the check works

The idea is simple. Any agent reacts to the meaning of a sentence. But only someone who knows the agreement reacts to a code hidden in the wording. So the researchers put an outsider in the partner's "seat": a fresh instance of the same model with the same role and the same public history, but without the partner's private instructions, notes, or memory. Both are then shown the original message and its paraphrase.

The check runs after the fact, on stored logs. For each turn, the researchers take the recorded prompt (the input text) that the partner received in the real run, swap the message inside it, and send it to a new model instance. There is no need to reconstruct the agents' internal state.

Sellers' actions are reduced to four categories: price higher, about the same, lower, or order withdrawn. The researchers then compute the total variation distance between the distributions of responses to the original and to the paraphrase. From that they subtract a "noise floor": how much the responses differ across two runs on the same message. This keeps a more random agent from producing a false gap. Confidence intervals are built with a bootstrap that resamples sessions first and then turns within them.

The experiment: a market and five scenarios

The test ran in a simulated continuous double auction where sellers can message each other. There were five scenarios, with 10 markets in each, 50 in total:

• no collusion; • collusion through a shared reward with neutral messages; • the same, but with openly stated intent; • a hidden code: the first letter of one seller's messages tells the other whether to hold the price or cut it; • the code plus stated intent.

In each scenario, the researchers evaluated 90 turns (hours 2 through 10 across 10 markets). The results:

• Code in the first letter only: PoPLi 0.59. • Code plus stated intent that matches the letter: 0.48. • Code plus stated intent that contradicts the letter: 0.53. • The lower bound of the 95% interval is above 0.29 in all three code scenarios. • In the three scenarios without a code, the metric is near zero, with the upper bound of the interval no higher than 0.10.

The authors also checked separately that a zero gap does not mean the outsider is deaf. When a seller states its intentions outright in words, the outsider reacts to them, and the gap between outsider and partner stays at zero. This means the test separates a private channel from open information exchange.

What about monitors that read the text

For comparison, the researchers used two monitors that analyze only messages. The first reads only the speaking seller's messages and answered "no" in all 50 markets. The second reads the messages of all sellers and flagged 9 of the 10 markets where there was no collusion at all. Neither separated the scenarios.

Limits of the method

The authors are candid about what PoPLi cannot see. It detects the use of a private channel, not collusion in general:

• Collusion through shared incentives with neutral messages stays out of view: there is nothing to look at, either for a text monitor or for this test. • Codes embedded in the meaning of a message rather than its wording survive paraphrasing, so the metric will not catch them. • The method assumes that the colluding parties do not know about the test and do not adapt to it. If they authenticate messages or anticipate a repeated query, the method weakens. • The auction remains a game with partly conflicting interests. A small reaction from the outsider may reflect distrust of a competitor rather than a failure to understand the language.

The authors also note that paraphrasing as a defense has been studied before: according to work by Roger and Greenblatt (2023), it limits known steganography schemes to roughly three bits per kilobyte of text. However, one-bit strategies optimized against paraphrasers are still decoded in more than 90% of cases on the AQuA-RAT dataset.

Practical takeaway

For teams that run agents that trade or negotiate with each other, the takeaway is this. If you store full prompt logs, you can replay turns after the fact with edited messages and compare the reactions of the partner and the outsider. This is cheaper than trying to guess collusion from text, and it produces fewer false positives.

But it will not be a complete defense. You also need detectors for open collusion based on wording and separate oversight of incentives, because each of the three forms of collusion requires its own tool. For now, this is the result of a single simulation with a single behavior model, so it is too early to carry it over to real markets.