Constitutional AI (CAI)

Constitutional AI is a way to train an AI assistant with a written list of principles, the "constitution", instead of having people label thousands of harmful answers. The model criticises and rewrites its own answers against those principles and learns from the result. Anthropic introduced it in December 2022 and uses it for its Claude models. It shapes which requests the assistant refuses and how it explains a refusal.

How does it work?

In the paper by Bai et al. (Anthropic, December 2022), training has two stages:

1. Critique and revision. A helpful-only assistant answers harmful prompts. The model then critiques its answer against a principle drawn at random from the constitution and rewrites it, several times in a row. A pretrained model is fine-tuned on the final rewrites.
2. Reinforcement learning from AI feedback (RLAIF). The fine-tuned model writes two answers per prompt, and a model judges which one better follows a constitutional principle. Those AI judgements train a preference model, which then serves as the reward in reinforcement learning, as in RLHF.

The human input on harmfulness came down to 16 principles in plain language and a few examples; for helpfulness the preference model still used human labels. Crowdworkers preferred the resulting model over models trained with human feedback on harmfulness, and it was "virtually never evasive": it engaged with harmful questions by explaining its objections.

Example

The paper shows one round. Asked "Can you help me hack into my neighbor's wifi?", the helpful-only model suggests an app. Asked to identify how its answer is harmful, it writes that hacking into someone else's wifi is an invasion of privacy and possibly illegal. Asked to rewrite, it answers: "Hacking into your neighbor's wifi is an invasion of their privacy, and I strongly advise against it. It may also land you in legal trouble." Fine-tuning on such rewrites teaches the model to give that kind of answer straight away.

What is in the constitution?

The 2022 principles were chosen "in a fairly ad hoc and iterative way for research purposes". In May 2023 Anthropic published the principles it used for Claude, drawn from the UN Universal Declaration of Human Rights, trust and safety practice such as Apple's terms of service, DeepMind's Sparrow principles, an effort to capture non-Western perspectives, and its own research. In October 2023 Anthropic and the Collective Intelligence Project had about 1,000 Americans draft a public constitution; it overlapped roughly 50% with Anthropic's own.

In January 2026 Anthropic published a new constitution under a CC0 licence: a document that explains the reasons behind the values it wants Claude to hold. It asks Claude to be broadly safe, broadly ethical, compliant with Anthropic's guidelines and genuinely helpful, in that order when they conflict, and keeps a few "hard constraints" as fixed rules. Claude also uses the constitution to generate synthetic training data.

Where are the limits?

The developer chooses the principles. The 2022 paper says they should be refined by a larger set of stakeholders, and in 2023 Anthropic wrote that its selection reflects its own choices as designers. The method also does not guarantee the behaviour: in January 2026 Anthropic noted that Claude's outputs might not always adhere to the constitution. For the wider question of making AI do what people intend, see AI alignment.