Latent Communication

Fig 1 · The channel

The grey parts are two ordinary frozen models. The dashed top row is what every multi-agent system does today. The teal row is the addition: pick a layer of the sender, pass its states (or its keys and values) through a bridge, and inject them into a layer of the receiver. The three published ways to inject are adding into the residual stream, prepending to the receiver's KV cache, and letting the receiver cross-attend over the sender's states. The bridge is the only thing with gradients; in the training-free variants it is the identity or a closed-form rotation.

The math in three lines

\[\text{text:}\quad m = \operatorname{decode}\!\big(h^{S}\big) \in V^{T},\qquad h^{R} = \operatorname{Enc}_{R}(m)\] // one token per step, at most \(\log_2 |V| \approx 17\) bits; \(h^{S}_{k}\) holds \(d\) reals per position and is discarded \[\text{latent:}\quad \tilde h = f_{\theta}\!\big(h^{S}_{k}\big),\qquad h^{R}_{j} \leftarrow h^{R}_{j} + \tilde h \quad\text{or}\quad \mathrm{KV}^{R}_{j} \leftarrow \big[\,f_{\theta}(\mathrm{KV}^{S}_{k})\,;\,\mathrm{KV}^{R}_{j}\big] \quad\text{or}\quad h^{R}_{j} \leftarrow h^{R}_{j} + \operatorname{Attn}\!\big(h^{R}_{j},\, f_{\theta}(h^{S})\big)\] // add (AC, SDE) · prepend to the cache (C2C, LatentMAS, KVComm, StateBridge) · cross-attend (Mixture of Thoughts, XBridge) \[\min_{\theta}\ \mathbb{E}_{(x,y)}\Big[-\log p_{R}\big(y \mid x,\ f_{\theta}(h^{S}(x))\big)\Big],\qquad \phi_{S},\ \phi_{R}\ \text{frozen}\] // training-free special cases: \(f = \mathrm{Id}\) when S and R are the same model (AC, LatentMAS); \(f = W^{\star} = \arg\min_{W^{\top}W = I}\lVert W H^{S} - H^{R}\rVert_F\) fitted on paired states (StateBridge)

\(\theta\) is the only knob, and it is small. Zero parameters when sender and receiver are the same model, about 1% of the combined pair in the Bicameral model, 3.8% of the receiver (264M) in XBridge. Everything else about the two models is untouched, which is the whole point: the bridge is retrained, not the models.

Why it is kept

The text is a summary of the state, and the state carries more. Hanna & Ameisen show Qwen-3 holds a representation of a planned word (“accountant”) several tokens before writing it, and that this is what produces the “an” in front of it; the effect grows from 0.6B to 14B. Gurnee et al. find a small, shifting set of concepts the model is poised to say but does not (the “J-space”), which it can hold silently while the visible output continues. Both are the interpretability results the communication papers, and Mostik's page, lean on.
It beats the text hand-off in every paper that measures it. Activation communication: +27% over text on coordination and reasoning tasks. Cache-to-Cache: 3–5 points over text. XBridge: better than text on all seven tasks for every model pair. StateBridge: best or tied on 22 of 26 model–task pairs, including against trained projectors.
No decoding of the message. The receiver never waits for the sender to generate its report token by token: 2× faster in C2C, 11× lower latency in XBridge, under a quarter of the compute in activation communication, up to 24× with Interlat's compression. The sender's prefill is still paid, see the pitfalls.
The asymmetric case is the useful one. A large model reads, a small model writes. That is LLM-to-SLM from 2024 (frozen large encoder, small fine-tuned decoder, up to 4× faster at 1–2 points cost), C2C's Qwen3-4B → Qwen3-0.6B pair, and the setup Mostik is selling.

Fig 2 · Why a cheap bridge should exist, and what got built on it

The left column is the older representation-learning result that makes each channel plausible: two networks trained separately end up with spaces that a linear (often orthogonal) map joins. Stitching is the direct ancestor of the trained bridges; relative representations and Procrustes-style alignment are the ancestor of the training-free ones; the Platonic and vec2vec results are what make cross-family bridges credible; the affine-map paper is the cross-size result. The last row is the interpretability argument for why the channel carries anything worth sending, and the amber box is the claim this note is checking.

The papers

Paper	What is sent	Bridge and fusion	Trained	Models	Against text
CIPHER · Liu et al., ICLR 2024	expected embedding: the output distribution's weighted sum of the embedding table	none; concatenated as input	no	same model, debate setting	+0.5–5.0 points over natural-language debate on five reasoning tasks
LLM-to-SLM · Bergner et al., 2024	the large model's encoding of the whole prompt	projector; small model conditions on projected states	projector + small model; large model frozen	encoder LLM → small encoder-decoder or decoder-only	vs the LLM itself: up to 4× faster, 1–2 points lower
Activation communication (AC) · Ramesh & Li, 2025	hidden state of the last token at a chosen layer	identity; added into the receiver at the same layer mid-forward	no	same model as sender and receiver	+27% on coordination and reasoning at under ¼ of the compute
Mixture of Thoughts · Fein-Ashley et al., 2025	hidden states of the top-K routed experts	interaction layers project every expert into one shared space; the primary expert cross-attends	router + interaction layers	heterogeneous open models, first cross-architecture method without pairwise translators	vs routing and aggregation baselines, in- and out-of-distribution
Cache-to-Cache (C2C) · ICLR 2026	the sender's whole KV cache	per-layer projection + gated fuser into the receiver's cache	fuser only; both models frozen	Qwen2.5-0.5B, Llama3.2-1B, Qwen3-4B → Qwen3-0.6B	+3–5 points, 2× faster; +8.5–10.5 over the receiver alone
KVComm · Wang et al., ICLR 2026	a subset of KV pairs picked by attention importance with a Gaussian layer prior	none; prepend	no	same model	matches the full-cache upper bound at up to 3× less communication
Thought Communication · Zheng et al., 2025	latent “thoughts” recovered from hidden states, with an identifiability result	learned encoder; gated exposure of shared and private thoughts	encoder	—	improves over text collaboration
Interlat · Du et al., ACL 2026	last-layer hidden states	learned compression / projection; concatenated along the token axis	compressor; models frozen	heterogeneous	beats fine-tuned CoT and single-agent; up to 24× faster with compression
LatentMAS · Zou, Yang et al., 2025	autoregressive latent thoughts (last-layer embeddings) plus a shared KV working memory	none; prefill + decode cache exposed	no	Qwen3 4B / 8B / 14B, same backbone	+2.8–4.6 points over text MAS, 71–84% fewer output tokens, 4× faster
Vision Wormhole · 2026	the reasoning trace rendered through a universal visual codec	codec; the receiver is a VLM that reads it as an image	codec, label-free distillation	fully heterogeneous, cross-architecture	—
Bicameral model · Flamant, Ghai, Shimizu, 2026	intermediate activations, both directions, every decoding step	translation network + learned suppression gate	≈1% of combined params; bases frozen	pairs of 0.5–0.6B models, one coupled to a tool	vs unaugmented: arithmetic 36 → 96%, 1.7× on ZebraLogic
Beyond Tokens · survey, 2026	Taxonomy of the field along three axes: what (embeddings, hidden states, KV caches), which (sender–receiver and layer alignment), how (concat, prepend, add, cross-attention, cache restoration). Notes that most methods assume a shared backbone, that cross-architecture bridges are rare and always trained, and that size asymmetry is not studied.
XBridge · Yang et al., Aug 2026	sender hidden states plus lexical anchors mapped into the receiver's vocabulary	gated cross-attention: the receiver queries the sender's states	264M params, 3.8% of the receiver	Llama, Qwen, Mistral pairs	beats text on 7/7 tasks per pair, 11× lower latency; beats KV sharing on 6/7 same-family
StateBridge · Peng et al., Aug 2026	final-layer hidden states	closed-form orthogonal map + norm calibration + vocabulary anchoring; prepended as a continuous prefix	no	four models from two families	best or tied on 22/26 model–task pairs, incl. vs trained projectors
Mostik · web page, Sep 2026	hidden states (“two megabytes of internal state” per token)	“a small trained bridge”; fusion not described	bridge only; both models frozen	GLM-5.2 753B → Qwen-3.5 4B	claims ≈10 points over a text hand-off, half the gap to the large model closed, 2.5× cheaper than a mid-size model; no paper

Latent chain-of-thought inside one model (Coconut, Hao et al. 2024) is the same trick applied step to step rather than model to model; the survey keeps the two apart, and so does this table.

Mostik against the literature

The design is Cache-to-Cache and XBridge's. Two frozen models, a small trained bridge, hidden states instead of tokens. The “a million numbers thrown away per token” argument is the activation-communication paper's opening motivation. The two interpretability citations are real and say what the page says they say.
What would be new: the scale of the sender (753B where the literature stops at 14B), the cross-family pair (GLM → Qwen; the survey lists cross-family bridges as rare), and a 4B receiver where C2C uses 0.6B. Size asymmetry at this ratio has not been published by anyone.
What is unverified: there is no paper, no bridge size, no benchmark names, and no named baseline behind “a mid-sized model”. The ≈10-point gain over a text hand-off sits in the C2C / XBridge band, so it is plausible. “Closes half the gap to the 753B model” is stronger than anything published and is the number to ask for.
The compute claim needs the prefill counted. The 753B sender still runs over the prompt every time. LLM-to-SLM accounts for this honestly: the saving is in decoding only, so “2.5× cheaper than a mid-sized model” is true only for long generations after short prompts, if at all.

Two readings of “AR to diffusion”

“Converting” can mean rewriting the AR weights into a diffusion LM, or keeping the AR model frozen and attaching a diffusion decoder that reads its latents. The bridge design maps onto the second cleanly, and that version already exists: it is DFlash, the fastest lossless speculative decoder published, and it is the MTP head widened from one token to a block.

DFlash's arrangement, drawn in the vocabulary of Fig 1. The target model is the sender; a learned mix of a few of its layers is the bridge; the receiver is a small block-diffusion model that fills k mask slots in one non-causal forward pass instead of k autoregressive draft steps. The target verifies the block as in ordinary speculative decoding, so the output distribution is unchanged. DFlare keeps the design and gives every draft layer its own mix of target layers instead of one shared fusion.

Attach, don't convert

DFlash, ICML 2026. A lightweight block-diffusion drafter conditioned on context features from the target. Over 6× lossless speedup, up to 2.5× more than EAGLE-3, because one forward pass drafts the whole block where EAGLE-3 needs one per token.
DFlare, June 2026. Per-layer fusion: each draft layer attends to its own learnable combination of target layers, trained on 2.4M samples instead of DFlash's 800K. 5.5× on Qwen3-4B and Qwen3-8B, 3.9× on GPT-OSS-20B.
Why it sidesteps conversion. The trunk stays causal and frozen, so nothing is lost to the bidirectional-attention switch or to denoising at random mask rates. Only the head is trained, and the AR model verifies, so quality is exact by construction.
It inherits the bridges' open questions. The head is fit to the trunk's states and goes stale when the trunk is post-trained, MTP healing again. A learned gate could let the diffusion head emit easy spans unverified. RLVR on the head with the trunk frozen is untried.

If you do convert

Method	Start	Budget	Trick	Result
DiffuGPT / DiffuLLaMA · ICLR 2025	GPT-2, LLaMA-2 7B	continued pretraining, tens of billions of tokens	attention-mask annealing from causal to full; shifted targets	first 7B-class adapted diffusion LM
Dream 7B · 2025	Qwen2.5-7B	580B tokens	shift operation kept from AR; context-adaptive token-level noise rescheduling	strongest open 7B full-attention diffusion LM at release
SDAR · 2025	Qwen3 1.7B – 30B-A3B	50B tokens at 30B	blockwise diffusion, brief adaptation after AR pretraining	AR-level quality with parallel decoding inside blocks
Fast-dLLM v2 · 2025	Qwen2.5	≈1B tokens	block diffusion with a complementary attention mask; hierarchical block + sub-block KV cache	matches AR accuracy, up to 2.5× faster than AR decoding; 500× less adaptation data than Dream
LLaDA 2.0 · Dec 2025	Ling AR models, 16B and 100B	warmup–stable–decay schedule	treat the AR model as block diffusion with block size 1, grow it 1 → 4 → 32 → 64 → 4096, train, then decay back to small blocks for inference	first 100B diffusion LM
NBDiff-7B · Dec 2025	off-the-shelf 7B AR	—	causal attention kept on the prefix; AR guidance during adaptation; gradual block growth	state of the art among 7B diffusion LMs
Don't Retrain, Align · May 2026	any AR model, frozen as teacher; same architecture	—	masked denoising plus a per-layer cosine alignment of the diffusion model's hidden states to the frozen AR model's	up to 4× faster conversion, largest gain in low-data regimes

The trend is “keep the causal prefix, grow the block”. Every 2025–26 method treats the AR model as block diffusion with block size 1 and widens it. The adaptation budget fell from 580B tokens to about 1B in a year, so knowledge transfer is no longer the bottleneck.
Alignment is already a conversion regularizer, in its simplest form only. Don't Retrain, Align uses the identity map, the same architecture, every layer. Untried: an affine or cross-size bridge (align a small diffusion model to a large AR model's states, the size-asymmetric transfer the survey on the Papers tab calls unstudied), selective layers, and the uniform-noise setting. Von Rütte et al. find uniform diffusion more parameter-heavy than masked, which suggests its representations sit further from AR's, so whether an AR teacher helps or hurts under GIDD-style noise is open.
A bridge alone converts nothing. It transports state between two working models; it does not change how either one generates. The hard part of conversion is the factorization, not the representation.

Verdict

Most promising: the attachment. A diffusion head on a frozen AR trunk is where the bridge design is cleanest and the wins are already lossless and large.
Next: alignment beyond the identity map. Cross-size, selective-layer, and non-masked-noise versions of Don't Retrain, Align are open and cheap to run.
Caveat. The speedups are small-batch, memory-bound numbers, the same regime where MTP pays. Diffusion LMs still do not beat AR plus speculative decoding per FLOP at scale, and parallel decoding has its own quality trade-offs (ParallelBench).

Pitfalls

Parameter-free is not assumption-free. Even the training-free methods choose a layer, rescale norms, and decide what position the injected states occupy. The survey's line: “parameter-free does not mean assumption-free.” Get any of the three wrong and the receiver reads noise.
Same backbone is the default. Most results are one model talking to a copy of itself. Cross-family needs training, one bridge per ordered pair unless a shared space is used (Mixture of Thoughts), and transfer to unseen pairs is open.
Post-training on either side breaks the bridge. The bridge was fit against two fixed spaces; RL or SFT on the receiver moves its residual stream and the alignment goes stale, the same way an MTP head loses acceptance after RLVR. Budget a re-fit.
It is a high-privilege channel. Nothing in the receiver checks what arrives; the survey flags modification, replay and malicious senders, and latent injection attacks on these systems have already been demonstrated.
The numbers across papers do not compare. Different backbones, prompts, context lengths and hardware; every “vs text” figure above is against that paper's own text baseline. Only the survey's taxonomy transfers, not the deltas.
Latent communication is not latent reasoning. Coconut-style continuous thoughts flow step to step inside one model; these methods flow agent to agent. Papers and product pages blur the two.

Latent communication between models