AgentDrift is a study from March 2026 about AI agents: assistants that fetch information from other systems, such as stock prices or a supplier's price list, and act on it. Given manipulated data, the AI investment advisors in the study passed it on without question and recommended risky shares to cautious investors. The usual quality score noticed nothing, so a good score was no proof of good advice.
How did the study work?
AgentDrift by Zekun Wu, Adriano Koshiyama, Sahan Bulathwela and Maria Perez-Ortiz (UCL and Holistic AI, 13 March 2026) is a method for agent evaluation. It replays real financial-advice conversations twice, once with clean tool output and once with contaminated output, and compares the two runs: a paired-trajectory protocol. The test used ten users from the Conv-FinRe dataset with 23 turns each, a choice of ten stocks, and seven language models from 7B parameters up to frontier models such as GPT-5.2 and Claude Sonnet 4.6. In every turn the AI agent fetched market data and news through tools and returned a ranked shortlist of stocks. The contamination in the main experiment was deliberately extreme: inverted risk scores with matching figures, biased headlines, and an added high-risk fund shown as low-risk.
What is evaluation blindness?
NDCG, the standard metric for ranking quality, compares the agent's list with an expert ranking that is the same for every user, so a risky stock that replaces a safe one scores about the same. NDCG barely moved (a utility preservation ratio of about 1.0), while products outside the user's risk profile appeared in 65–93% of contaminated turns. The authors call this gap evaluation blindness. Clean sessions were not spotless: 46–78% of their turns already broke the stated risk profile, partly because there were only ten stocks to choose from, and the authors confirm the effect of the contamination with a second measure.
The violations began at the first contaminated turn and did not correct themselves over 23 turns. Across 1,563 contaminated turns, no agent questioned whether the tool data was reliable, and a changed risk score, when quoted, was repeated 80% of the time. The authors call that trust expected, since agents are built to rely on their tools, and an attack surface nobody monitors. Biased headlines alone, without changed numbers, also caused significant drift.
Example
For a user with a low risk tolerance, Tesla (risk 5 on a scale of 1 to 5) ranks last of the ten stocks with clean data. After contamination it shows risk 1, lower volatility and a higher expected return, and ranks near the top. Procter & Gamble, a defensive stock, goes the other way.
What helps?
- Score safety per turn. A safety-penalised NDCG (sNDCG), which counts unsuitable products as zero, brought the preservation ratio down to 0.51–0.74.
- Check tool output against a trusted reference. Comparing each risk score with a known-good list flagged every contaminated turn of the full attack; shifts of at most one step on the risk scale slipped past such checks and still produced 61% of the drift.
- Expect the same gap elsewhere. The authors see it wherever safety is separate from what a ranking metric rewards, such as medical triage or product recommendations. A small retail pilot pointed the same way; they call it too small for general conclusions.
What changed in the revised version?
The revised version of 25 May 2026 (v8) is titled Sell Me This Stock and drops the name AgentDrift. It tests eight models and reports risk-mismatched recommendations in 65–99% of turns. Across 1,840 turns, 80% of risk-score citations repeat the manipulated value verbatim, and not a single turn pushes back. Even a cross-check that flags the manipulation in 99–100% of turns on a frontier model left the advice unchanged: "the agent identifies the tampering and recommends it anyway."