Communication emerges between frozen ai language models using prompt optimization

Communication between Frozen Large Language Models via Prompt Optimization in a Referential Game

Computation and LanguageArtificial IntelligenceMachine LearningMultiagent Systems

Summary

This paper looks at how two separate large language models from different providers can communicate with each other without changing their internal setup. One model describes an object in a short message using a limited set of symbols, and the other model tries to identify the object based on that message. Instead of changing the models themselves, the researchers optimize the prompts given to each model to improve communication. They find that with careful prompt tuning, the models can develop a shared code that helps them understand each other even when messages are short and limited. The success varies depending on optimization methods, but when it works, the communication protocol can be read directly in the prompts.

What this means in practice

  • For api developers: Enable different large language models to exchange compact, interpretable messages without retraining their weights through optimized prompting.
  • For security engineers: Audit and verify emergent communication protocols between black-box language models to ensure transparency in multi-model systems.

Authors

Vivek Anand, Muthu Chandrasekaran, Shiva Chaitanya

Abstract

We study communication between two frozen large language models from different providers, with different tokenizers, accessed through their API endpoints. The two play a referential game: one sees an object and describes it in a short fixed-length message over a small alphabet; the other must pick that object out of a candidate set. Neither model's weights are updated. Each agent's prompt is rewritten by an isolated prompt optimizer whose reflection model reads that agent's scored interactions. In the positional setting, optimized prompts carry a shared code that generalizes to held-out objects above a measured no-codebook baseline, including when the memory window is removed. In a second setting, independent per-letter blocks no longer fit within the message, although a whole-object place value code does. The base system fails to establish reliable communication: the sender struggles to retain an injective rule, and the receiver has too few confirmed examples in view. A sender collision penalty, retention of successful interactions, and sequential optimization enable successful place value communication in some runs. Outcomes vary across runs and reflection models. In successful runs, the protocol is written into the optimized prompts, where it can be read and audited directly.
View PDFOpen arXiv