Env Rethink Paper Shows Agent Failures Start in Noisy File Environments
Shanghai Jiao Tong University and Tencent Hunyuan researchers posted Env Rethink on 24 September 2026, reporting that noisy file environments cut agent pass rates from 83.9% to 57.6% across nine model configurations.
What changed
Researchers from Shanghai Jiao Tong University, Theseus Labs, and Tencent Hunyuan submitted Breaking the Environment Wall: Evolving LLM Agent Environments for Recursive Self Improvement to arXiv on 24 September 2026 (arXiv:2609.29773). The paper introduces Env Rethink, a 27B post trained system that organizes scattered files, learns to judge source validity, and evolves environments to create harder but verifiable training instances.
On 30 difficult tasks adapted from Workspace Bench, nine model and harness configurations averaged 83.9% rubric pass rate in a clean environment versus 57.6% when all files were exposed, a 26.3 percentage point drop. Env Rethink raised mean evaluation check pass rate from 59.4% with Qwen3.8 27B to 72.7% across nine downstream configurations.
Why it matters
Frontier labs are racing to deploy agents that edit code, browse repositories, and run multi step workflows. This paper argues the bottleneck is often the environment, not the model: agents trust misleading filenames, miss withdrawal notices buried in attachments, and treat duplicate records as independent confirmation. Teams building enterprise agents should treat workspace hygiene as a first class product surface.
Who is affected
Agent platform engineers, enterprise IT teams exposing document stores to coding agents, and eval owners benchmarking SWE or office automation tasks. Vendors selling agent harnesses without environment preparation layers may see reliability gaps that better models alone cannot close.
What to do next
Audit one production agent workspace for the three failure modes the paper documents: incomplete search, filename based version trust, and superficial file reuse. Measure pass rate delta between a curated file subset and the full noisy corpus before your next model upgrade.
What to watch
Whether major agent vendors adopt Collection Map and Event Log style preparation in commercial products, and independent replication of the 26.3 point noise penalty on customer data.
The authors identify three recurring agent failures in noisy settings: stopping search too early, assuming a file named final is authoritative, and copying similar files without checking scope. Env Rethink responds with Collection Maps for semantic file grouping, Event Logs that reconstruct revision relationships, and event driven evolution that generates harder instances while preserving verifiable reference outcomes.
Sources
- Primary. arXiv, Breaking the Environment Wall: Evolving LLM Agent Environments for Recursive Self Improvement (24 September 2026). Establishes Env Rethink architecture, noise benchmark results, and pass rate improvements.