Adversarial Cost to Exploit (ACE)

Adversarial Cost to Exploit (ACE) expresses the security of an AI agent in money: how much does an attacker have to spend before the agent does something it was told never to do, such as paying out a refund? If that cost is lower than what the attacker stands to gain, the agent needs protection beyond the AI model itself. The security company Fabraix proposed the measure in April 2026.

How does Fabraix measure it?

In its post of 5 April 2026, Fabraix defines ACE as the total token spend an autonomous attacker needs to get an AI agent to call a tool it may not use. In this Gatekeeper challenge, the agent is an assistant named Kai with four ordinary tools, such as web search and a pricing lookup, and a fifth, reveal_access_code, while its system prompt forbids it ever to reveal that code. The attacker is itself an AI that talks to Kai through the normal chat, tries an approach, reads the reply and adapts: automated red teaming. A run succeeds the moment the forbidden call appears in the API response, so no judge is needed.

Only the model changes; system prompt, tools and attacker stay the same. Without extra filters or guardrails, the result reflects the model's own resistance. Each model got 40 to 50 runs. The challenges are published in Fabraix's open-source Playground.

What did the first measurement show?

Fabraix tested six budget models, and all six were broken in every run. The mean cost of a successful attack, in dollar equivalents:

• Gemini 3.1 Flash-Lite (Google): $0.23
• DeepSeek v3.2: $0.46
• Mistral Small 4: $0.66
• Grok 4.1 Fast (xAI): $0.95
• GPT-5.4 Nano (OpenAI): $1.15
• Claude Haiku 4.5 (Anthropic): $10.21

Haiku cost the attacker roughly nine times as much as the next model. Fabraix also names four models with a text/action mismatch: the text of the reply refused, while the tool call in the same response carried out the forbidden action. Haiku was the only model where Fabraix saw no such mismatch. The pattern was documented before ACE: Cartagena and Teixeira (February 2026) found 219 such cases across six frontier models, even under safety-reinforced system prompts.

How do you use it?

In classical security economics, which Fabraix follows, an attack is rational when the expected gain is greater than the ACE, and a system is incentive-compatible when breaking it costs more than it yields. Fabraix links this to the Gordon-Loeb model, in which the sensible security budget is bounded by the expected loss. The ACE of a model is then the floor that the model alone provides. A tool that can be worth more to an attacker than that floor needs protection outside the model, such as checks on tool calls, rate limits, anomaly detection or a human approval step.

Example

Fabraix works out two cases. An agent that can issue refunds of up to $25 is a profitable target with all six models. GPT-5.4 Nano, at a mean ACE of $1.15, gives the attacker 22 times the outlay, and even Haiku still gives 2.4 times. For a tool worth at most $5, such as a promo code generator, Haiku's $10.21 is twice the gain, so the attack no longer pays. The other five models remain rational targets.

Where are the limits?

Fabraix names them itself: six budget models, one attack route (a forbidden tool call) and one automated attacker. Human experts may break a model more cheaply, and the values are snapshots, since providers update their models. The post names neither the attacker model nor the prices used to turn tokens into dollars, so the figures cannot be recomputed from the post alone. ACE was published as a blog post by a company that sells offensive security testing for AI agents, and measured on that company's own Playground.