AI Papers Reader

Personalized digests of latest AI research

View on GitHub

AI Agents Learn to Remember Like Humans—By Rewriting the Past

When humans recall a past experience to solve a new problem, we rarely play it back like an immutable video recording. Instead, our brains actively reconstruct the memory, filtering out irrelevant details and adapting old lessons to fit our present reality.

Large Language Model (LLM) agents, however, have long suffered from a crude form of digital memory. Traditional “memory-augmented” AI systems rely on verbatim replay: when faced with a task, they search a database for past experiences and copy-paste the raw text directly into their context window. This often backfires. Because historical situations rarely match the present step-for-step, verbatim replay introduces misleading context, causing the AI to make foolish errors—a phenomenon known as negative transfer.

To bridge this gap, researchers from Zhejiang University, Shanghai AI Laboratory, and partner institutions have developed MemHarness, a framework that transforms AI memory from static replay into dynamic context-aware reconstruction.

Don’t Copy-Paste—Critique and Adapt

MemHarness inserts an explicit “critique and rewrite” step between memory retrieval and action execution. At every decision step, a unified AI policy compares the original context of the retrieved memory against its current environment observation before deciding how to act.

Consider an embodied AI robot in a virtual house tasked with cooling a cup in a fridge and placing it on a sidetable. If the agent retrieves a past success trajectory about putting credit cards on a coffee table, a standard verbatim replay agent would blindly paste references to “credit cards” into its prompt, causing it to hallucinate non-existent objects.

MemHarness avoids this trap. Its model compares the historical state with the current state, strips away the irrelevant objects, and reconstructs the memory into precise, state-grounded guidance: “You have cooled the cup using the fridge; the next step is to place it on the sidetable.” If a retrieved memory is entirely irrelevant, MemHarness outputs an <EMPTY> token, discarding the bad advice and falling back on its own intrinsic reasoning.

In e-commerce tasks on WebShop, instead of simply repeating an abstract heuristic like “always verify constraints before clicking,” MemHarness actively executes the rule during reconstruction, scanning the current page to output: “Product B06W51MMKY matches all criteria for black size 6 oxfords.”

Mindful Reinforcement

Crucially, MemHarness does not require human-annotated examples of “how to rewrite memories.” Instead, the agent learns this reconstructive ability end-to-end using Group Relative Policy Optimization (GRPO), a reinforcement learning algorithm. By rewarding the agent based purely on final task success, the model autonomously discovers when to retain, revise, or reject recalled memories.

Across benchmarks including ALFWorld (household tasks) and WebShop (online shopping), MemHarness achieved state-of-the-art success rates of 85.2% and 75.6%, respectively. It significantly outperformed both standard reinforcement learning baselines and traditional static memory agents, while surpassing large closed-source models such as GPT-4o and Gemini-2.5-Pro.

Most importantly, MemHarness proved remarkably robust in out-of-distribution scenarios featuring unseen room layouts and novel product catalogs. By teaching machines to reconstruct the past rather than blindly repeat it, MemHarness marks a major step toward more flexible, human-like autonomous reasoning.