Research · 11 min read

Memory Is a Policy Problem: MemCon Learns When to Retrieve, Forget, and Inject Plans

UCLA-led MemCon treats agent memory as a learned MDP: a tabular bandit decides when to retrieve, inject plans, or forget — up to 15.2-point gains with 5–20% fewer tokens and zero extra LLM calls.

By Classy AI News Staff — Research Desk · August 4, 2026

Most agent memory systems still behave like static pipelines: retrieve once with a fixed top-k, then reason. A new paper argues that optimal memory access is context-dependent — and that the fix is not a fancier vector store, but a learned controller that decides when to retrieve, re-retrieve, inject a plan, consolidate, or forget.

MemCon (Memory as a Controlled Process), posted to arXiv by researchers at UCLA, the University of Washington, and Northwestern, formalizes agent memory management as a Markov Decision Process and learns an online policy with a lightweight tabular contextual bandit — without additional LLM calls.

The static-pipeline bottleneck

Large language model agents increasingly rely on external memory to accumulate experience across tasks. Yet nearly all existing approaches — from graph-structured memories to reflective insight stores — access memory through fixed, hand-designed heuristics.

The authors identify four regimes where any single fixed setting fails:

  1. Early tasks should retrieve less, because memory is sparse and indiscriminate retrieval dilutes the prompt.
  2. Recurring goal types should prefer plan reuse — replaying a distilled success template — rather than nearest-neighbor lookup over raw trajectories.
  3. Stuck agents that repeat actions should trigger re-retrieval with an alternative query, not re-read the same top-k that already failed.
  4. Long task streams require consolidation and forgetting so a growing, noisy memory store remains useful.

MemCon occupies a design point between MemGPT — which promotes the LLM itself to memory controller at the cost of extra calls per operation — and static retrievers. It adds a learned but lightweight controller that is adaptive without any extra LLM calls.

Abstract network visualization representing memory graph structures

How MemCon works

MemCon casts memory operations as actions in a Memory MDP. The state fuses task-progress signals (goal type, step phase, stuck indicator, locations visited) with memory-status signals (store size, plan availability, learning phase).

The action space includes:

  • Retrieve (with varying top-k, insight-k, graph hop depth)
  • PlanInject — inject a distilled, object-generalized success plan
  • Re-Retrieve — alternative query when stuck
  • Consolidate and Forget
  • NoOp

The policy uses UCB exploration over a tabular contextual bandit, warm-started from human-readable priors, updated via reverse-discounted credit assignment from binary task success. Memory control is a millisecond-scale table lookup, not a second LLM invocation.

Critically, MemCon is backend-agnostic: it wraps any existing memory implementation's retrieve and store endpoints, separating what is stored from how it is accessed.

Evaluation breadth

The paper reports results across:

  • 6 benchmarks: ALFWorld, PDDL planning, ScienceWorld, TriviaQA, WebWalkerQA, GAIA
  • 3 agent frameworks: Lobster, LangGraph, Microsoft Agent-Framework
  • 3 LLM backbones: GPT-4.1-mini, Claude Sonnet-4, DeepSeek-V3.2
  • 9+ memory baselines including G-Memory, MemGPT, Voyager, LatentMem, and others

MemCon consistently outperforms baselines by up to 15.2 points in task success while reducing token consumption by 5–20%. On ALFWorld with GPT-4.1-mini (Lobster), the paper reports 67.9% — the top score among all memories evaluated in that cell.

Developer reviewing code on a laptop in a quiet workspace

Why this matters beyond benchmarks

The MemCon framing shifts the research question from "what should agents remember?" to "when and how should they access what they remember?" That is closer to how human expertise develops: early tasks need exploration, familiar patterns deserve plan reuse, and long careers require pruning.

The zero-extra-LLM-call constraint also matters for production economics. MemGPT-style LLM-as-controller approaches improve adaptivity but inflate inference cost. MemCon's bandit policy converges within tens of tasks using only binary success feedback.

Code is available at github.com/ericjiang18/MemCon.

Limits and open questions

The paper's discretization yields on the order of a few hundred distinct states per benchmark — sufficient for online convergence but potentially coarse for highly heterogeneous enterprise workflows. Independent replication on proprietary agent stacks remains to be seen.

Still, for a field that has spent two years optimizing storage formats while leaving access policies hand-tuned, MemCon offers a concrete alternative: treat memory management as a sequential decision problem and learn the policy online.

Overhead view of a desk with notes and a computer

### Sources