Trajectory refinement

Trajectory refinement improves an AI agent's plan by comparing it with what happened when the agent carried it out, then repairing the steps that went wrong. The trajectory is the chain of steps an agent takes: searching, calling tools, deciding. Suppose an agent plans a business trip and books a museum visit before opening time. Refinement spots the clash and fixes that part of the plan before you receive it.

Where does the term come from?

Agents that learn from feedback in words are older. In Reflexion (Shinn et al., 2023), an AI agent reflects on a failed attempt in text and keeps those notes for its next attempt, without any change to the model. The term itself is in the title of PIVOT (Plan-Inspect-eVOlve Trajectories), a framework by five researchers at Amazon, published on 11 May 2026 (Zhang et al.). They start from what they call plan-execution misalignment: agents write plans that look coherent and then fail on infeasible actions, broken constraints or errors that pile up over a long task.

How does PIVOT work?

PIVOT treats the whole trajectory as something to optimise, in four stages:

1. Plan: the agent turns the task into an explicit trajectory and starts carrying it out.
2. Inspect: it compares what happened with the plan, finds the first step from which the execution no longer supports it, and writes plain-language feedback on what went wrong. The authors call this a textual gradient, a term from TextGrad (2024). The feedback comes from a person (human-in-the-loop) or, in the autonomous variant, from a language model acting as judge.
3. Evolve: it rewrites the trajectory from that step on and keeps the part that worked. A revision is only accepted if it improves the estimated result, so quality never goes down.
4. Verify: a final check of the whole plan against the task's constraints.

The model itself is not retrained. The paper also explains why this helps: raising the model's thinking budget from 1,024 to 3,072 tokens brought no consistent gain, because the model did its thinking at the start and almost none while assembling the final plan. PIVOT prompts it to reason at several points along the way.

What did it achieve?

The authors tested PIVOT with Qwen3-235B and four Claude models on DeepPlanning, which asks for multi-day travel plans and product selections within budget and other constraints, and on GAIA, open questions that need web search and tools. With human feedback, PIVOT scored best in every combination, and the authors report up to 94% relative improvement in constraint satisfaction, which they present as an upper bound. Without human feedback, it beat plain ReAct (a single pass of reasoning and tool use) in all 20 combinations and the strongest other method in 12. On DeepPlanning it spent 3 to 5 times fewer extra tokens per solved task than competing refinement methods.

Where are the limits?

• Refinement is only as good as the tools. On GAIA, calling the same tools again often returned the same evidence, so refinement came close to simply retrying.
• The largest gains came with human feedback. Without it, a self-critique method with built-in domain knowledge did better in 8 of the 20 combinations.
• The evidence comes from benchmarks. The authors call for evaluation in real-world settings beyond controlled tests.

Example

In one case from the paper, run without human feedback, a solo traveller wanted a day trip from Hangzhou to Shaoxing with the earliest train, the cheapest Home Inn and a meal near Bazhi Bridge at a restaurant with a waiting area. The baseline agent's plan visited the Lu Xun's Hometown site at 08:23, seven minutes before it opens, and missed the restaurant requirement. With PIVOT, the inspection listed every constraint as a checklist, flagged those errors and a gap in the itinerary, and rebuilt the plan. The benchmark's composite score went from 0.375 to 1.000.