Remembering Is a Control Problem: Proactive Memory for Long-Horizon Agents

Long-horizon agents often fail without literally losing the relevant information. A requirement may still be present in the transcript; a failed command may remain inside the model’s context window; a correct diagnosis may have been stated only a few turns earlier. Yet the next action ignores it. The paper Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents gives this failure mode a useful name: behavioral state decay.

The distinction matters for system design. Availability is not influence. If a fact is stored but no longer reliably constrains action selection, increasing context length or attaching another retrieval layer may not solve the operational problem. Wu et al. instead formulate memory as an intervention policy: maintain decision-relevant execution state separately, then decide whether a grounded reminder should enter the next action-agent call—or whether the correct action is silence.

Behavioral state decay

The paper defines execution state broadly: task requirements, stable environment facts, prior attempts, failure diagnoses, intermediate discoveries, and open subgoals that should continue to shape future actions. During a long trajectory, this state can become behaviorally inactive even when it remains technically accessible.

Typical manifestations are familiar to anyone running coding or tool-use agents:

• a constraint identified early is violated while fixing an unrelated bug;
• a command or implementation path that already failed is retried with only superficial changes;
• a diagnosed error pattern is later treated as new;
• a policy fact observed through a tool stops constraining a state-changing operation;
• an unfinished subgoal disappears behind a local debugging loop.

This separates three problems. Storage asks what persists; retrieval asks which records match a query; intervention asks whether remembered state should affect the next decision. Under-intervention permits repeated errors, while over-intervention adds cost and distraction. A useful policy needs a calibrated no-op.

Architecture: a sidecar memory agent

The proposal adds a separate memory agent, (pi_M), alongside an unchanged action agent, (pi_A). The action agent continues to interact with the environment through its existing tool scaffold. At a memory timestep, the sidecar observes the task description, a recent trajectory window, and its current memory bank. It first edits the bank, then chooses an intervention:

1. update persistent execution state;
2. emit either a short, memory-grounded reminder or a null intervention;
3. if non-null, insert that reminder as transient context in the next action-agent call.

The bank has three components. A private status field tracks progress, unresolved issues, and risks, but is never exposed to the action agent. Knowledge memory stores relatively stable facts: requirements, paths, configuration details, API properties, verified user or tool facts, and evaluator-relevant observations. Procedural memory stores attempts and outcomes: failed commands, successful fixes, ruled-out hypotheses, diagnostics, and empirical improvements.

The interface is constrained. Phase 1 returns predefined operations—update status, save knowledge, save procedure, or delete—rather than unconstrained prose. Identifiers let stale entries be corrected. Phase 2 cannot edit the bank; it selects a grounded reminder or silence. The prompt discourages broad strategy, redundancy, and taking over the action model’s planning.

The main experiments invoke memory at the first step and every subsequent step. The memory agent sees the latest (k=8) messages, the task, and the bank. The method section allows a general fixed interval and discusses event-driven triggers—tool errors, failed tests, repeated commands, or context shifts—but those selective triggers are not evaluated in the primary setup.

Main quantitative results

The evaluation covers two distinct long-horizon settings. Terminal-Bench 2.0 exercises autonomous command-line work: file inspection, code editing, command execution, debugging, and hidden verifier tests. The benchmark has 89 tasks, but the paper reports the 85 paired tasks for which baseline and memory runs both yielded valid evaluations; four Docker failures unrelated to agent behavior were excluded.

τ²-Bench evaluates conversational tool use under domain policies, with a user simulator and mutable environment. The base split contains 50 airline, 114 retail, and 114 telecom tasks, or 278 episodes per configuration. All reported scores are single-run pass@1 percentages, and deltas are percentage points. Claude Opus 4.6 is the memory agent in Table 1.

• Table 1 prints +9.6 pp, although subtraction of the displayed rounded values gives 9.7. This article preserves the paper’s reported delta rather than silently “correcting” it. The prose later says +9.7 pp, revealing the same internal rounding/editorial inconsistency.

The largest gains occur with the weaker Sonnet action model: +8.3 pp on Terminal-Bench and +6.8 pp on the task-weighted τ²-Bench average. The effect does not disappear for Opus: +2.4 and +2.5 pp respectively. However, domain variation is material. Opus gains nothing on airline, while Sonnet gains substantially on airline and retail and only +2.6 pp on telecom. These results support usefulness across two task classes, not a claim of uniform improvement.

Qualitative traces show plausible mechanisms. Terminal-Bench reminders resurface boundary conditions, environment failures, and edit outcomes. In τ²-Bench, one intervention distinguishes claimed Gold status from tool-verified Regular status before compensation; another reactivates the rule that a basic-economy flight cannot be modified before a state-changing call.

What Table 2 actually establishes

The τ²-Bench ablations use Sonnet 4.5 as action agent and Opus 4.6 for all memory-model variants. Importantly, Table 2 is not numerically identical to Table 1: its full system scores 57.0% on retail and 61.2% micro, versus 58.8% and 61.8% in the main table. The paper does not explicitly explain the run-level difference, so comparisons should remain within each table.

The Table 2 baseline is 68.0 airline, 49.1 retail, and 55.3 telecom, yielding 57.5 macro and 55.0 task-weighted micro. The full two-phase system reaches 78.0, 57.0, and 57.9, with 64.3 macro and 61.2 micro.

Four comparisons isolate the control choices:

• Full-bank context retains agentic bank management but exposes the entire bank every step. It reaches 61.5 macro and 58.6 micro, trailing the full system by 2.8 and 2.6 points. Stored state helps, but indiscriminate exposure is weaker than targeted reactivation.
• Always inject retains the bank and synthesized reminders but removes silence. It reaches 63.5 macro and 61.5 micro. It is 0.3 points above the full system on micro, while losing on domain-balanced macro and scoring notably lower on airline (72.0 versus 78.0). The authors treat the micro edge as within expected run variance. Thus the ablation does not show selective intervention dominating every scalar metric; it shows the most balanced cross-domain behavior.
• Injection-only, no bank acts like an advisor observing recent trajectory without maintained state. It scores 61.0 macro and 60.8 micro. Telecom rises to 66.7, but airline falls below baseline, from 68.0 to 62.0. Advice without persistent grounding is less robust.
• Mem0 writes through ADD and retrieves vector+BM25 top-10 items. It scores 62.1 macro and 60.8 micro. It improves retail and telecom but leaves airline at baseline. General retrieval is useful, yet does not explicitly decide when a memory should enter the control loop or synthesize it as a targeted reminder.

The defensible takeaway is not “retrieval fails” or “always-on reminders never work.” Most variants improve averages; maintained state plus calibrated intervention is simply the most balanced across domains.

Training an open-weight intervention policy

A frontier model call at every memory step is an expensive deployment assumption, so the authors conduct a preliminary open-weight study. They freeze a Qwen3.5-122B-A10B action agent and train Qwen3.5-27B as the memory agent on SETA executable terminal tasks. Terminal-Bench remains held out for transfer evaluation.

SFT distills prompted-memory trajectories, supervising both Phase 1 bank operations and Phase 2 reminder/null decisions. GRPO then optimizes downstream verifier reward. Because task-level rewards are sparse over many memory calls, training focuses updates on pivot turns that offline labeled rollouts identify as likely to influence final success.

The SETA validation results make the calibration requirement explicit. Action-only obtains average reward 0.709 and solves 56 tasks. Adding an untrained base 27B memory model hurts: 0.693, 54 solved, a -0.016 reward delta. SFT reaches 0.720, 58 solved, or +0.011 over action-only. GRPO reaches 0.734, also 58 solved, or +0.025. On the held-out 85-task Terminal-Bench set, trained 27B memory raises the frozen action model from 37.6% to 41.1% pass@1, a +3.5 pp transfer gain.

This suggests the interface and some calibration are learnable. It does not show that the open-weight policy matches Opus or transfers broadly; the study is preliminary.

Cost and limitations

The architecture is plug-and-play because it leaves the action agent and its decoding/tool scaffold unchanged. That operational simplicity should not be confused with zero cost. A separate memory inference adds latency, tokens, and monetary or compute expense. In the main configuration it runs every step, and the paper provides no token, wall-clock, or dollar-cost table. The pass@1 gains therefore cannot yet be translated into deployment ROI.

Interventions also introduce a new error channel. The qualitative analysis reports speculative inferences expressed too confidently, reminders of facts already active, and plausible but unnecessary concerns that trigger extra verification. These are calibration failures rather than pure storage failures. The negative result from the untrained Qwen memory model reinforces the point: adding a memory model can make the system worse.

The study uses one sampled episode per task, two benchmark families, limited model combinations, and frequent fixed triggering. Four Terminal-Bench tasks are excluded under a paired-validity criterion for infrastructure failures. Domain variation and the Table 1/Table 2 discrepancy argue against overgeneralizing exact gains.

The paper’s open directions are well chosen: jointly train action and memory policies; learn when to invoke memory rather than using a fixed schedule; and determine when verbatim reminders are preferable to task-specific abstractions.

Practical implementation lessons

For teams building long-running coding, operations, or service agents, the paper suggests a concrete design checklist.

Represent execution state, not just conversation history. Maintain separate channels for stable facts, attempt/outcome evidence, and private progress/risk tracking. A generic rolling summary can blur these semantics.

Make edits explicit. Use identifiers and constrained save/update/delete operations. Long-horizon memory must support correction; stale state that cannot be removed becomes a persistent source of bad control.

Keep interventions transient and grounded. Inject the smallest reminder needed for the next action, with a traceable basis in the bank. Do not let the sidecar become an unconstrained planner.

Treat silence as a first-class output. Evaluate false-positive interventions, extra tool calls, and redundant verification—not only recall of useful facts.

Measure end-to-end outcomes and overhead together. Bank quality is not the objective. Task success is, subject to token, latency, compute, and intervention-risk budgets.

Start with fixed triggering, then optimize it. Frequent invocation isolates policy quality and is easy to implement, but production systems should test event-driven or learned triggers once they can measure missed interventions versus avoided calls.

The paper’s central contribution is a change of abstraction: memory is not merely a database attached to an agent. It is a control mechanism that restores causal influence from prior execution state to the present decision. For long-horizon systems, the key question is no longer only “Can the agent retrieve this?” It is “Will the right fact become behaviorally active before the next irreversible action?”

Sources

• Wu et al., Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents, Meta AI, 2026.
• Official code: github.com/yifannnwu/proactive-memory-agent
• SETA repository cited by the paper: github.com/camel-ai/seta