Activation steering

Activation steering changes how an AI model behaves by adjusting the numbers inside it while it answers, without changing the prompt or the model. Researchers find the pattern for a trait, such as a positive tone or refusing a request, and add or subtract it. For a business it matters because it can expose hidden bias, and can switch off the safety training of a model anyone can download.

Where does the idea come from?

In 2023, Turner et al. described "activation engineering": changing a language model's activations while it runs, to steer what it writes. Their method, ActAdd, takes the difference between the activations for two prompts such as "Love" and "Hate", and adds that steering vector while the model writes. It needs no training. Panickssery et al. (2023) averaged the difference over many pairs of answers that do and do not show a behavior, such as factual versus made-up answers, and called this contrastive activation addition (CAA).

How does it work?

1. Collect pairs of examples that differ in one trait, for example an honest answer and one that simply agrees with the user.
2. Run them through the model and record the activations at one layer, in the residual stream: the internal representation that each layer reads from and writes to.
3. Average the differences. The result is the steering vector, a direction in the model's latent space.
4. While the model answers, add the vector at that layer, multiplied by a coefficient. A larger coefficient steers harder; a negative one steers the other way.

The layer matters: in Llama 2 7B Chat, Panickssery et al. found the strongest effect at layer 13 and the layers around it. The model's weights do not change, which sets steering apart from fine-tuning. Anthropic's variant, feature steering, takes its directions from dictionary learning, which finds many interpretable directions (features) in the residual stream.

What is it used for?

• Testing what drives behavior. In Anthropic's study of emotion vectors (April 2026), an earlier, unreleased version of Claude Sonnet 4.5 blackmailed in 22% of the runs of a fictional scenario. Steering toward "desperate" raised that to 72%; steering toward "calm" brought it to 0% (Sofroniew et al.).
• Finding hidden bias. Tripathy and Buckmann (May 2026) had open-weight models assess mortgage applications that differed only in racially associated names. The decisions showed no bias, but the models kept the demographic information internally, and reinjecting it with activation steering reversed decisions almost completely, mostly in one direction. Their conclusion: audits that only check outputs are not enough.

Where does it go wrong?

In October 2024, Anthropic tested feature steering on 29 features of Claude 3 Sonnet related to social bias, and called the results mixed. Turning up features for neutrality and for multiple perspectives lowered the bias scores on all nine dimensions of the BBQ benchmark. Beyond a steering factor of −5 to 5, however, the model's scores on the MMLU knowledge test dropped, sometimes until it was unusable, and a feature linked to gender bias awareness also raised the age bias score by 13%. Anthropic concluded that it may not yet be a reliable way to make targeted changes.

Steering can also remove safety behavior. Arditi et al. (2024) found that in 13 open chat models of up to 72B parameters, refusal runs through a single direction. Erasing it made the models comply with harmful instructions. They call this a "white-box jailbreak": it needs access to the model's internals and no jailbreak prompt, so it applies to open-weight models that anyone can download and run.

Example

In May 2024, Anthropic put Golden Gate Claude online for 24 hours: Claude 3 Sonnet with its feature for the Golden Gate Bridge turned up. Asked how to spend $10, it recommended driving across the bridge and paying the toll. Anthropic stressed that it had used neither a system prompt nor fine-tuning: it had changed the model's internal activations.