Structured output

Structured output means that an AI model fills in a fixed form instead of writing free text. You define the fields, for example supplier, invoice date and amount, and the model returns exactly those fields in a format that software can read, usually JSON. An accounting package can then process an invoice without anyone copying numbers by hand. The form is fixed; the values still need checking.

How do you describe the form?

With a JSON Schema, "a declarative language for defining structure and constraints for JSON data" (JSON Schema). It lists the fields, their types (text, number, true or false), which ones are required and, where needed, the allowed values (an enum). Allowed values also stop a model from inventing codes, the problem described under semantic training gap. Providers support only part of JSON Schema: Anthropic's documentation (September 2026), for example, has no minimum or maximum for numbers and no recursive schemas.

How does it work?

There are three levels, from loose to strict:

1. Asking for it in the prompt ("answer in JSON with these fields"). Nothing enforces it.
2. JSON mode: the output is valid JSON, but not necessarily in your schema. OpenAI introduced it in 2023; its guide (September 2026) calls it a more basic version of structured outputs (OpenAI guide).
3. Constrained decoding: the API turns the schema into a grammar and, after every token, allows only the tokens that keep the output valid.

In OpenAI's tests with complex schemas, gpt-4o-2024-08-06 with structured outputs scored 100%, against less than 40% for the older gpt-4-0613 (OpenAI, August 2024).

In September 2026 OpenAI (since August 2024), Anthropic (beta in November 2025, generally available since February 2026) and Google's Gemini API all accept a JSON Schema. The same technique works for function calling: with strict: true, the arguments of a tool call follow the schema.

For models you run yourself, Outlines (Willard and Louf, 2023) and the GBNF grammars of llama.cpp block invalid tokens in the same way. Instructor works differently: it checks the answer against a Pydantic model afterwards and asks again when the check fails.

Where does it go wrong?

• A valid form can hold wrong values. Structured output "doesn't prevent all kinds of model mistakes", OpenAI writes, and Google advises checking the values in your application.
• Input that does not fit. The model always tries to fill in the schema, which can produce made-up values when the input has nothing to do with it (OpenAI). Give it a way out, such as a field that may stay empty.
• Exceptions. A refusal for safety reasons, or hitting the token limit, can still produce output that does not match.
• Speed and cost. The first request with a new schema is slower while the grammar is compiled. In September 2026 Anthropic caches the grammar for 24 hours and adds an extra system prompt about the format, which costs input tokens.
• Reasoning. Tam et al. (2024) found that strict formats lowered scores on reasoning tasks and raised them on classification; one cause was that the model gave its answer before its reasoning. The Outlines team repeated three of those tests and found structured output slightly better, for example 0.78 against 0.77 on the maths set GSM8K (November 2024). Either way, put a field for the reasoning before the answer field: according to OpenAI's guide, the keys come out in schema order.

Example

Suppose an accountancy firm lets a model read incoming PDFs, with this schema:

1
{
2
  "type": "object",
3
  "properties": {
4
    "document_type": {"type": "string", "enum": ["invoice", "other"]},
5
    "supplier": {"type": ["string", "null"]},
6
    "invoice_date": {"type": ["string", "null"], "format": "date"},
7
    "total_incl_vat": {"type": ["number", "null"]}
8
  },
9
  "required": ["document_type", "supplier", "invoice_date", "total_incl_vat"],
10
  "additionalProperties": false
11
}

For an invoice the model returns, for example, {"document_type": "invoice", "supplier": "Drukkerij Jansen", "invoice_date": "2026-09-01", "total_incl_vat": 1250.0}. For a delivery note it can answer "other" and leave the fields empty instead of inventing an amount. The firm's software still checks the values, for example against the purchase order, before anything is booked.