Model card

A model card is a short document that comes with an AI model, like the leaflet in a box of medicine. It says what the model is for, what data it learned from, how well it did in tests, where it goes wrong and what its license allows. A business that wants a chatbot for Dutch customers can read in it whether the model was made and tested for Dutch.

Where does the idea come from?

Researchers at Google and the University of Toronto proposed model cards in October 2018 (Mitchell et al., presented at the FAT* conference in January 2019). Their motive: systematic errors, such as face recognition that failed far more often for darker-skinned women, had only come to light after the systems were in use. They proposed a record of one to two pages with every released model, with test results broken down by group and condition. In their example card for a smile detector, that breakdown showed the model wrongly labelling older men as smiling far more often than other groups: bias that an average score hides.

What is in a model card?

The paper proposes nine sections, here grouped:

• Model details: maker, date, version, type and license.
• Intended use: what and whom it was made for, and uses that are out of scope.
• Factors and metrics: which groups and conditions were tested, and how performance was measured.
• Evaluation and training data: what the tests ran on and, where possible, what the model learned from.
• Quantitative analyses: the results per group and for combinations of groups.
• Ethical considerations, caveats and recommendations.

On Hugging Face the model card is the README.md file of a model's repository, with a block of metadata at the top (license, languages, datasets) that feeds the site's search filters. Its template also has a section for the estimated CO2 emissions of training.

Who publishes them?

Google DeepMind publishes model cards for Gemini and Gemma, and Meta for Llama. Anthropic and OpenAI call theirs system cards; Anthropic's cover the capabilities, safety evaluations and deployment decisions for each Claude model.

Is a model card required by law?

The EU AI Act mentions model cards once, in recital 89, as a practice it encourages among developers of open-source AI components. What is binding is a duty for makers of general-purpose AI models: since 2 August 2025 they must give businesses that build the model into a product information on its intended tasks, license, architecture and training data (Article 53 and Annex XII). Models released under an open-source license with public weights are exempt from that duty, unless they pose a systemic risk.

Where does it fall short?

The maker writes the card, and cards differ widely in what they cover. Researchers at Stanford and Hugging Face (Liang et al., 2024) examined Hugging Face as of October 2022: 44.2% of its 74,970 model repositories had a model card, and those models accounted for 90.5% of downloads. Within those cards, the training section was filled in 74.3% of the time, limitations 17.4%, evaluation 15.4% and environmental impact 2.0%.

Example

Meta's model card for Llama 3 (April 2024) says the model was trained on more than 15 trillion tokens of publicly available data, without Meta user data. It gives test scores per size, such as 66.6 and 79.5 on the MMLU knowledge benchmark for the base models with 8 and 70 billion parameters, and estimates the emissions of pretraining at 2,290 tonnes of CO2 equivalent. It states that Llama 3 is intended for commercial and research use in English, that other languages are out of scope unless a developer fine-tunes the model within the license, and that testing so far was in English. For a Dutch web shop that is the line that matters: a Dutch customer chatbot on Llama 3 falls outside the use Meta intended and tested.