Sycophancy & memory
Recalling Too Well: Sycophancy Evaluation and Mitigation in Memory-Augmented Models
- Identifier
- arXiv:2606.10949
- Published
- Jun 9, 2026
- Authors
- 8
- Type
- Paper
- Area
- Practical evaluations
In one paragraph
Memory systems are supposed to make assistants better. We found they can make them agree with you even when you're wrong. When a memory system extracts facts, it often drops the correction the assistant made earlier, and the model then sides with the user's mistaken belief.
- Sonnet 4.6 sycophancy on MIST-Moral: chat history vs. Mem0 memory
- 1.6% → 40.2%
- Sonnet 4.6 sycophancy on MIST-Moral: chat history vs. Mem0 memory
- every model at least tripled its sycophancy under at least one memory condition
- 3×+
- every model at least tripled its sycophancy under at least one memory condition
- sycophancy after the prose-summary mitigation (vs. 17.1% for Zep)
- 12.8%
- sycophancy after the prose-summary mitigation (vs. 17.1% for Zep)
FIG. 02: Mechanism
How it works
Plant a misconception
Over a multi-turn conversation, the user states something wrong, and the assistant correctly pushes back.
Remember it
A memory system (Mem0, MemOS or Zep) turns the conversation into stored memories, usually short extracted facts about the user.
Lose the correction
Extraction is lossy. "User believes X" is kept, but "assistant explained X is false" is often dropped.
Agree later
In a new session, the model sees the user's belief in memory without the correction, and it goes along with it. That is sycophancy, introduced by the memory system.
Mitigate
Simple fixes help. Keep the assistant's turns in memory, or store a short prose summary instead of isolated facts.
FIG. 03: Interactive figure
Watch a memory system forget the correction.
Switch memory modes and see which ones keep the assistant honest when the user comes back in a new session.
Loading interactive demo…
FIG. 04: Notes
This page covers two related papers on how personalized context (memory, user preferences and rebuttals) erodes accuracy:
- Recalling Too Well (arXiv 2606.10949) introduces the MIST benchmark for memory-augmented models.
- The Price of Agreement (arXiv 2604.24668, ICLR 2026 FinAI Workshop) measures sycophancy in agentic financial applications.
The problem
Enterprise assistants increasingly remember things: preferences, past conversations, facts about the user. Personalization feels helpful, but it raises a quieter risk. Sycophancy is a model’s tendency to tell users what they want to hear rather than what is true.
The approach
MIST is a benchmark of synthetic multi-turn conversations spanning scientific (GPQA Diamond), medical (MMLU Medical) and moral reasoning (Moral Stories). In each conversation the user holds a misconception and the assistant corrects it. A later session then tests whether the model stays correct when it has to rely on memory.
We tested five frontier model families against three production memory systems (Mem0, MemOS and Zep) and compared them with plain chat-history baselines.
In the finance study, eight frontier models were tested on FinanceBench and FinanceAgent with three kinds of injected personal context: rebuttal, contradiction and personal preference.
What we found
- Memory amplifies sycophancy. Every model at least tripled its sycophancy rate under at least one memory condition. Sonnet 4.6 rose from 1.6% with chat history to 40.2% under Mem0 on MIST-Moral.
- Extraction is the cause. Lossy fact extraction keeps the user’s belief and drops the correction.
- Cheap mitigations work. Including the assistant’s turns in memory, or storing a prose summary of about the same length, cuts sycophancy sharply. The prose summary brought MIST-Moral down to 12.8%, below Zep’s 17.1%, while matching or beating memory systems on factual recall.
- In finance, how the context arrives matters. Directly injected context causes larger accuracy drops, but models flag the conflict more often. Context delivered as a tool result, the way a memory API would deliver it, causes smaller drops, but the models almost never acknowledge the conflict.
Reference
@misc{bensal2026recalling,
title={Recalling Too Well: Sycophancy Evaluation and Mitigation in Memory-Augmented Models},
author={Shelly Bensal and Axel Magnuson and Aparna Balagopalan and Daniel M. Bikel},
year={2026},
eprint={2606.10949},
archivePrefix={arXiv},
primaryClass={cs.AI}
}