DeepMind Recirculation Boosts Gemma3 Without Retraining a Single Weight

Google DeepMind researchers have published a technique called recirculation that improves off the shelf transformer models at inference time without retraining weights. The paper, posted to arxiv on August 18, 2026, reports a 23 percent reduction in perplexity and a 21 percent accuracy gain on GSM8k for the Gemma3 family.

The core insight is deceptively simple: feedforward transformers update their internal state only once per layer during a forward pass. That bounds how much belief tracking a model can perform within a single token step. Recirculation breaks that bound by leaking activations from a deeper layer back into a shallower one, then running the upper stack again.

How recirculation works

During generation, the method selects a source layer and a destination layer along with a mixture coefficient alpha. After the initial forward pass produces logits and a deep residual stream state, that state is norm matched and mixed into an earlier residual boundary. The upper layers then execute a second pass, overwriting upper layer key value cache entries so subsequent tokens attend to the recirculated representation.

The approach incurs essentially no additional latency during token generation itself, though it requires serial processing during the prefill phase. That tradeoff makes it attractive for workloads where prefill cost is amortized across long outputs.

Results on Gemma3

The team evaluated adaptive recirculation, a variant that tunes only hyperparameters while freezing original model weights. On a suite of datasets, perplexity dropped 23 percent relative to the baseline Gemma3 1B pretrained model. GSM8k accuracy rose 21 percent. Other downstream tasks showed reliable improvements as well.

The researchers distinguished recirculation from chain of thought reasoning, which they argue should be reserved for complex inferences rather than basic state tracking. They also contrasted it with depth recurrence techniques like looping and the costly training of recurrent transformers.

Why this matters beyond benchmarks

Recirculation is training free. It leverages the trained model itself to inform architectural modifications at inference time. That suggests a broader design principle: architectural evolution guided by a network's learned properties rather than forced, arbitrary structural choices.

Within days of publication, the vLLM project opened an RFC for experimental recirculation support in causal decoders, with Gemma 3 as the initial target family. If inference engines adopt the technique, practitioners could see gains on existing deployed models without waiting for the next training run.

Limitations and open questions

The paper sweeps hyperparameters including source layer, destination layer, and alpha on held out data. Production deployment will need robust defaults per model family and safeguards against pathological recirculation paths that degrade quality on specific token types.

The method also adds prefill complexity. For latency sensitive applications with short outputs, the prefill overhead may dominate any generation time savings.

Still, for a training free enhancement that materially moves perplexity and reasoning benchmarks, recirculation deserves attention from both research labs and inference platform teams.

Sources

arxiv.org/abs/2608.17981; Google DeepMind blog August 21 2026; vLLM RFC August 22 2026