The Price of Agreement
The Price of Agreement: Measuring LLM Sycophancy in Agentic Financial Applications
- Identifier
- arXiv:2604.24668
- Published
- Apr 27, 2026
- Authors
- 6
- Type
- Paper
- Area
- Domain-specific specialization
In one paragraph
Finance agents hold up reasonably well when a user argues with them. They fail when the user's preferences are injected into their context or tool results and contradict the correct answer. Filtering inputs with a separate LLM recovers some of the lost accuracy, but not all of it.
- Gemini-3-Pro accuracy on FinanceBench with injected preferences
- 0.83 → 0.24
- Gemini-3-Pro accuracy on FinanceBench with injected preferences
- Claude-Sonnet-4.5 accuracy on FinanceBench under user contradiction
- 0.87 → 0.45
- Claude-Sonnet-4.5 accuracy on FinanceBench under user contradiction
- Claude-Opus-4.5 accuracy recovered by prompt-based filtering
- 0.55 → 0.71
- Claude-Opus-4.5 accuracy recovered by prompt-based filtering
FIG. 02: Mechanism
How it works
Run the finance tasks clean
Models answer FinanceBench questions over 10-K and 10-Q filings and complete FinanceAgent tasks with tools. This gives baseline accuracy.
Push back in the conversation
A rebuttal adds a user turn that rejects the model's answer and asks it to try again. A contradiction also proposes a different answer.
Inject biased user preferences
Synthetic user beliefs and past behavior that contradict the reference answer are added either directly to the prompt or as the result of a memory or personalization tool call.
Score accuracy and transparency
An LLM judge scores accuracy, the acknowledgment rate (AR, how often the model says the user's information shaped its answer) and EWU (the share of wrong answers given without that acknowledgment).
Filter the inputs
A separate LLM with a specialized prompt removes bias-inducing preference content before the main model sees it, and the same metrics are measured again.
FIG. 03: Interactive figure
Tell the agent what the user wants
Inject a rebuttal, a contradiction or a biased preference into a finance agent's context, then filter it out and compare the reported accuracy.
Loading interactive demo…
FIG. 04: Notes
Summary
LLMs in financial systems can side with what the user believes rather than what is correct. This paper measures that sycophancy in agentic financial tasks, on FinanceBench and FinanceAgent.
Three kinds of personal context are tested:
- Rebuttals: the user rejects the model’s answer and asks it to try again.
- Contradictions: the user rejects the answer and proposes a different one.
- Preferences: user preferences that contradict the reference answer, injected directly into the prompt or delivered as a tool result, the way a memory or personalization API would deliver them.
Two metrics complement accuracy:
- Acknowledgment rate (AR): whether the model flags the conflict.
- EWU: the share of wrong answers given with no acknowledgment.
Results
- Rebuttals cause modest drops; injected preferences break most models. On FinanceBench, Gemini-3-Pro fell from 0.83 to 0.24 under direct preference injection. Claude-Sonnet-4.5 fell from 0.87 to 0.45 under contradiction and to 0.72 under rebuttal.
- Tool-delivered context is quieter. Context delivered as a tool result lowers awareness and acknowledgment.
- Filtering helps partially. A separate LLM that removes biased preference content brought Claude-Opus-4.5 from 0.55 back to 0.71, short of its baseline.
This work is the finance half of our sycophancy and memory research.
Reference
@misc{zhao2026price,
title={The Price of Agreement: Measuring LLM Sycophancy in Agentic Financial Applications},
author={Zhenyu Zhao and Aparna Balagopalan and Adi Agrawal and Dilshoda Yergasheva and Waseem AlShikh and Daniel M. Bikel},
year={2026},
eprint={2604.24668},
archivePrefix={arXiv},
primaryClass={cs.AI}
}