Stop Applauding the Mirror: Why Self-Reflection Hype Needs a Token Budget Audit

The agentic-AI playbook still opens with reflection. Models plan, critique, rewrite, debate themselves, and pick among their own drafts — architectures marketed as deeper reasoning. On July 30, 2026, an arXiv paper titled "Sample More, Reflect Less" asked a simpler question with harder statistics: once you count every token spent on introspection, does any of it beat just sampling again?

Across 36 paired comparisons, the answer was no — and often worse.

Equal cost, unequal hype

Researchers tested seven methods on open models at 1.5B, 3B, and 7B parameters across two mathematics benchmarks with 150 questions each. Every method was compared against repeated sampling at its own measured token budget, with bootstrap confidence intervals and multiplicity correction.

No method reliably beat repeated sampling at equal cost anywhere. Ten were reliably worse — all involving self-inspection. All 18 self-inspection comparisons came out negative.

The paper (arXiv:2607.28576v1) revisits a warning Wang et al. (2024) raised with point estimates but without rigorous significance testing: generating more text alone lifts accuracy, so gains over one chain-of-thought do not prove the reflection machinery helped.

When reflection silently disappears

Two failure modes deserve industry attention beyond benchmark tables.

Choosing among samples hurts small models: taking Best-of-N's eight answers and counting the most common response beat letting the model pick by 8.0 and 11.3 points at 1.5B, but only 2.0 and 1.3 points at 7B — no longer statistically distinguishable from zero.

Rewriting does not recover: Self-Refine and forced Reflexion stayed 3.6 to 10.1 points below the equal-cost baseline at 7B. Reflexion as published never triggered its own retry on the smallest model — it judged itself correct every time and silently collapsed to a single chain-of-thought.

That last detail is an engineering indictment. A method marketed as iterative correction can become a no-op while still billing tokens for the theater of reflection.

Why vendors keep selling mirrors

Reflection loops are legible in demos. They produce verbose traces investors and procurement committees can watch. They also inflate token bills — convenient when usage-based pricing rewards length.

Repeated sampling is boring. It lacks narrative. It is, however, exactly what many "agent frameworks" wrap in critique layers that the July 30 paper suggests may be negative-value at matched spend.

A procurement standard, not a philosophy war

This is not an argument against ever reflecting. It is an argument for token-budget audits before paying premiums for self-critique stacks. If a vendor claims +X accuracy from reflection, ask: +X compared to what, at how many tokens, with what confidence intervals?

The authors release code, prompts, generations, and verification scripts — the right response to a literature that too often compared unequal budgets and called it architecture innovation.

Frontier labs simultaneously cut inference prices and ship longer-context reasoning models. Cheaper tokens make brute-force sampling more affordable precisely when reflection's advantage may be illusory. Buyers should treat that coincidence as a feature selection problem, not a branding problem.

Stop applauding the mirror. Count the tokens. Sample again.

### Sources

• arXiv — Sample More, Reflect Less: Self-Refine and Reflexion Lose to Repeated Sampling at Equal Token Cost (July 30, 2026)
• arXiv — 2607.28576 DOI (July 30, 2026)