Sycophancy & memory

Recalling Too Well: Sycophancy Evaluation and Mitigation in Memory-Augmented Models

Shelly Bensal, Axel Magnuson, Aparna Balagopalan, Daniel M. Bikel, Zhenyu Zhao, Adi Agrawal, Dilshoda Yergasheva, Waseem AlShikh

FIG. 01 Plant a misconception → Remember it → Lose the correction → Agree later → Mitigate
Identifier
arXiv:2606.10949
Published
Jun 9, 2026
Authors
8
Type
Paper
Area
Practical evaluations

In one paragraph

Memory systems are supposed to make assistants better. We found they can make them agree with you even when you're wrong. When a memory system extracts facts, it often drops the correction the assistant made earlier, and the model then sides with the user's mistaken belief.

Sonnet 4.6 sycophancy on MIST-Moral: chat history vs. Mem0 memory
1.6% → 40.2%
Sonnet 4.6 sycophancy on MIST-Moral: chat history vs. Mem0 memory
every model at least tripled its sycophancy under at least one memory condition
3×+
every model at least tripled its sycophancy under at least one memory condition
sycophancy after the prose-summary mitigation (vs. 17.1% for Zep)
12.8%
sycophancy after the prose-summary mitigation (vs. 17.1% for Zep)

FIG. 02: Mechanism

How it works

  1. Plant a misconception

    Over a multi-turn conversation, the user states something wrong, and the assistant correctly pushes back.

  2. Remember it

    A memory system (Mem0, MemOS or Zep) turns the conversation into stored memories, usually short extracted facts about the user.

  3. Lose the correction

    Extraction is lossy. "User believes X" is kept, but "assistant explained X is false" is often dropped.

  4. Agree later

    In a new session, the model sees the user's belief in memory without the correction, and it goes along with it. That is sycophancy, introduced by the memory system.

  5. Mitigate

    Simple fixes help. Keep the assistant's turns in memory, or store a short prose summary instead of isolated facts.

FIG. 03: Interactive figure

Watch a memory system forget the correction.

Switch memory modes and see which ones keep the assistant honest when the user comes back in a new session.

Loading interactive demo…

FIG. 04: Notes

This page covers two related papers on how personalized context (memory, user preferences and rebuttals) erodes accuracy:

  • Recalling Too Well (arXiv 2606.10949) introduces the MIST benchmark for memory-augmented models.
  • The Price of Agreement (arXiv 2604.24668, ICLR 2026 FinAI Workshop) measures sycophancy in agentic financial applications.

The problem

Enterprise assistants increasingly remember things: preferences, past conversations, facts about the user. Personalization feels helpful, but it raises a quieter risk. Sycophancy is a model’s tendency to tell users what they want to hear rather than what is true.

The approach

MIST is a benchmark of synthetic multi-turn conversations spanning scientific (GPQA Diamond), medical (MMLU Medical) and moral reasoning (Moral Stories). In each conversation the user holds a misconception and the assistant corrects it. A later session then tests whether the model stays correct when it has to rely on memory.

We tested five frontier model families against three production memory systems (Mem0, MemOS and Zep) and compared them with plain chat-history baselines.

In the finance study, eight frontier models were tested on FinanceBench and FinanceAgent with three kinds of injected personal context: rebuttal, contradiction and personal preference.

What we found

  • Memory amplifies sycophancy. Every model at least tripled its sycophancy rate under at least one memory condition. Sonnet 4.6 rose from 1.6% with chat history to 40.2% under Mem0 on MIST-Moral.
  • Extraction is the cause. Lossy fact extraction keeps the user’s belief and drops the correction.
  • Cheap mitigations work. Including the assistant’s turns in memory, or storing a prose summary of about the same length, cuts sycophancy sharply. The prose summary brought MIST-Moral down to 12.8%, below Zep’s 17.1%, while matching or beating memory systems on factual recall.
  • In finance, how the context arrives matters. Directly injected context causes larger accuracy drops, but models flag the conflict more often. Context delivered as a tool result, the way a memory API would deliver it, causes smaller drops, but the models almost never acknowledge the conflict.

Reference

Cite this work
@misc{bensal2026recalling,
  title={Recalling Too Well: Sycophancy Evaluation and Mitigation in Memory-Augmented Models},
  author={Shelly Bensal and Axel Magnuson and Aparna Balagopalan and Daniel M. Bikel},
  year={2026},
  eprint={2606.10949},
  archivePrefix={arXiv},
  primaryClass={cs.AI}
}