There is a running joke among AI governance practitioners that documentation is the thing everyone agrees is essential and nobody wants to do. This is not entirely unfair. Documentation is unglamorous work. It is time away from building. It ages the moment you finish writing it. And it is the first thing to slip when deadlines squeeze.
But here is what happens when documentation is missing. Regulators ask for it, and it is not there. Internal audit tries to verify a claim, and there is no evidence. A vendor changes their model, and you have no baseline to compare against. An incident happens, and the incident response team cannot answer basic questions about what the system was supposed to do. A new team inherits an AI system, and they have to reverse-engineer what it does. Every one of those failure modes has real, sometimes very expensive, consequences.
This article is about the documentation formats that have emerged as standard practice for AI systems: model cards, datasheets for datasets, system cards for larger deployments, and audit trails for evidence. Each has a specific purpose, a specific audience, and a specific format. Getting them right is not extra work; it is what makes the rest of your governance defensible.
Why documentation is different for AI
Traditional software documentation focuses on functionality: what the system does, how to use it, how it fails. This is necessary but not sufficient for AI systems.
AI documentation has to answer additional questions. What was the system trained on? What are its known limitations? What performance characteristics does it exhibit, and on which populations? What are its intended and unintended uses? What are its ethical considerations? How does it interact with the humans in its workflow? What happens when it fails, and how would you know?
These questions are not answerable from the source code, no matter how clean. They require deliberate documentation practices that go beyond software engineering norms.
The response to this challenge has emerged from a series of proposals in academic and industry research. Model cards, first proposed by Mitchell et al. in 2019, provide a standard structure for model-level documentation. Datasheets for datasets, proposed by Gebru et al., provide the equivalent for training data. System cards, developed by organisations including OpenAI, extend the model card concept to full deployed systems including safety mitigations and use restrictions. All of these have been picked up by regulatory frameworks in various ways.
Model cards: what they are and what they include
A model card is a short, structured document that describes an AI model. It is designed to be readable by non-specialists while being technically precise enough for practitioners.
The standard sections of a model card cover:
Model details. Model name, version, date, type, and the organisation that developed it. Contact information for questions or issues. Licence conditions.
Intended use. What the model is designed for. Explicit call-out of out-of-scope uses. Primary intended users.
Factors. Groups, instrumentation, or environments the model is meant to be evaluated across. This includes demographic factors relevant to fairness, environmental factors relevant to robustness, and instrumentation factors relevant to reproducibility.
Metrics. Model performance metrics chosen, why they were chosen, and how they were computed. For fairness, the specific mathematical definitions used.
Evaluation data. Datasets used for evaluation, why they were chosen, and how they were preprocessed. This should include disaggregated evaluation across the factors above.
Training data. What the model was trained on, at least at a summary level compatible with privacy and IP constraints.
Quantitative analyses. Performance results, disaggregated by factors. Any fairness analyses. Confidence intervals or uncertainty quantification where appropriate.
Ethical considerations. Foreseeable harms, mitigation strategies, and residual risks. This is often the most important section and often the shortest in practice, which is a mistake.
Caveats and recommendations. Known limitations. Recommended use practices. Situations where the model should not be used.
The strength of the model card format is that it is short and structured. A well-written model card fits on a few pages. It is designed to be read, updated, and referred to, not to be filed once and forgotten.
The relationship to regulation
Model cards did not begin as a regulatory instrument, but they have been substantially adopted by the emerging regulatory landscape.
The EU AI Act requires technical documentation for high-risk AI systems (Article 11) and describes the required content in Annex IV. This required content substantially overlaps with what a model card provides, though the Act’s technical documentation is more comprehensive. For GPAI providers, Article 53 requires a summary of training data content, following a template maintained by the AI Office. Model cards are compatible with these requirements without substituting for them.
The NIST AI RMF Generative AI Profile (NIST AI 600-1) references documentation as a specific action under the MAP function. Model documentation is expected to identify AI system context and information used to inform stakeholders.
The MAS Consultation Paper on AI Risk Management Guidelines identifies documentation of key AI concepts, processes, and responsibilities as an expected practice.
ISO/IEC 42001 Annex A control A.6 (AI system lifecycle) includes documentation requirements at each stage of the lifecycle. Model cards are one form these can take.
Given this convergence, model cards have become de facto standard practice. Even where not strictly required, they are increasingly expected.
Figure 1: The documentation stack
Figure 1 shows the AI documentation stack. At the bottom are dataset-level documents (datasheets for datasets) covering training and evaluation data. Above them are model-level documents (model cards) covering individual models. Above them are system-level documents (system cards) covering deployed systems with their safety mitigations and use context. At the top are program-level documents (AI policies, risk assessments, governance framework documentation) covering the organisation’s overall AI management. Each layer supports the layer above, and each has different audiences (data scientists, developers, deployers, regulators, and executives).
Datasheets for datasets
Datasheets serve the same purpose for training data that model cards serve for models. They document what a dataset is, where it came from, how it was collected, and what it is suitable for.
Standard datasheet sections cover:
Motivation. Why was the dataset created? Who funded it? What tasks was it intended to support?
Composition. What does the dataset represent? What instances does it contain? Are there any relationships between instances? What are the recommended splits?
Collection process. How was the data acquired? What mechanisms were used? Was informed consent obtained where applicable? What was the timeframe of collection?
Preprocessing, cleaning, labelling. What was done to the data after collection? Was anonymisation or other privacy processing applied? Who labelled the data and how were labellers trained?
Uses. Has the dataset been used for particular tasks? What tasks should it not be used for? Are there any concerns about future uses?
Distribution. How is the dataset distributed? Under what licence? Are there restrictions on use?
Maintenance. Who maintains the dataset? Will it be updated? What version control is applied?
For enterprise AI, datasheets rarely need to be comprehensive research documents. What matters is that the questions have been asked, the answers are captured somewhere, and the information is available when needed for risk assessment, regulatory response, or incident investigation.
For foundation models used under commercial terms, the deployer typically cannot produce datasheets for the training data (which they do not have access to). They rely on the provider’s disclosures. But for any data the deployer contributes (fine-tuning data, retrieval corpora, evaluation sets), datasheets are the deployer’s responsibility.
System cards
For deployed systems that go beyond a single model, system cards have emerged as the extended format. They cover:
- Everything a model card covers
- Deployment context (where, how, for whom)
- Safety mitigations built into the deployment
- Access controls and use restrictions
- Monitoring and evaluation in production
- Incident response and update procedures
- User-facing behaviour and guidance
The system card format was popularised by AI safety publications from major AI labs describing their model deployments. It has been adopted more slowly than the model card format in enterprise contexts, but for high-risk AI systems (particularly under the EU AI Act) it is increasingly used.
Where a model card describes what a model is, a system card describes how it is being used in a specific deployment. Both are needed for a full documentation picture.
Audit trails: the operational documentation
Documentation is not just about the “what” and the “why.” It is also about the “when” and the “who.” This is where audit trails come in.
An audit trail for an AI system is a record of significant events over time. What has happened, when, and who did what. Effective audit trails cover:
Access. Who has accessed the system, when, and from where. For high-risk systems, this covers both administrative access (developers, deployers, operators) and use access (the end users).
Changes. Every material change to the system: model version updates, prompt changes, configuration changes, retrieval corpus updates, monitoring threshold changes. Who made the change, when, and what the change was.
Decisions. For systems that make or influence decisions, an audit trail of the decisions themselves, the inputs that produced them, and the outputs the system generated. This is critical for high-risk systems where individual decisions may need to be defended or explained later.
Incidents. All incidents identified in monitoring or reported by users. What happened, what was investigated, what was concluded, what was done.
Reviews. All periodic reviews, risk reassessments, and validation exercises. When they occurred, who conducted them, what they concluded.
Under the EU AI Act, Article 12 requires automatic logging for high-risk AI systems. Deployers must retain logs for at least six months (Article 26). Providers must maintain post-market monitoring logs (Article 72). Audit trails support these requirements directly.
The engineering implementation of audit trails is non-trivial for AI systems. LLM interactions produce large volumes of text; agentic systems produce sequences of tool calls; retrieval-augmented systems produce document access patterns. Logging everything at full fidelity may be prohibitively expensive; logging too little produces gaps that come back to bite you.
Reasonable practice is to log everything at reduced fidelity by default, with the ability to increase logging depth for specific investigations or for high-risk systems. Retention periods should reflect regulatory requirements and organisational needs. Privacy considerations may constrain what can be logged and where.
Documentation that scales
The critique most often levelled at model cards and datasheets is that they do not scale. Producing a rigorous document for every model or dataset is prohibitive for organisations with many systems. This is a real problem, and here are the practical responses.
Tier your documentation intensity. Full model cards for high-tier systems. Standard cards for medium-tier. Minimal records for low-tier. The tiering matches the risk classification we have used throughout the track.
Automate what you can. Many documentation fields can be populated from the training pipeline, the evaluation pipeline, or the deployment configuration. Investment in automation pays back at scale.
Templatise ruthlessly. Standard templates for each type of documentation reduce the effort per instance. Well-designed templates also make information easier to find and cross-reference.
Integrate into workflows. Documentation produced after the fact is documentation that will not be produced. Integrating documentation requirements into the development, review, and deployment workflows produces documentation as a by-product of doing the work.
Use documentation platforms. Multiple commercial platforms now provide AI documentation management, evaluation tracking, and model registry functions. Depending on scale, these can materially reduce the marginal cost of documentation.
What auditors actually look for
Two observations from real audit engagements, worth flagging.
First, auditors look at consistency. If your policy says all high-tier systems must have model cards, and your inventory shows twelve high-tier systems, the auditors will ask to see twelve model cards. Nine is a problem. So is twelve, if three of them are placeholder templates. Consistency between policy and practice is what defensibility rests on.
Second, auditors look at recency. Documentation dated three years ago for a system that has been updated monthly since is a red flag. Live documentation, updated as the system changes, is the standard. Version control on documentation is not an academic nicety; it is what makes the documentation defensible.
Figure 2: The documentation lifecycle
Figure 2 shows the documentation lifecycle for a typical AI system. At initiation, an intended use statement and preliminary risk assessment are created. During development, model cards and datasheets are drafted. At validation, evaluation results are added. At deployment, the system card is finalised and the audit trail begins. In production, monitoring outputs and change records feed continuously into the documentation. At review points (typically quarterly for high-tier systems), documentation is reviewed and updated. At decommissioning, the historical record is preserved for the retention period specified by policy or regulation. The lifecycle emphasises that documentation is not a one-time artefact but a continuous discipline.
Getting started
If your organisation is not producing structured AI documentation today, here is the pragmatic starting point.
Pilot with one high-tier system. Do the model card, the datasheet (or a summary if training data details are proprietary), and the initial system card properly. Learn what works and what does not.
Templatise from the pilot. Take what you learned and produce standard templates that fit your organisation’s tone, terminology, and workflow.
Roll out with the next batch of systems. Apply the templates to systems in development or entering production, capturing the documentation as part of the workflow rather than retrofitting it.
Backfill selectively. For existing systems, apply documentation retrospectively only to the highest-tier cases. Backfilling everything is usually not worth it; focus on the systems where the documentation gap presents real risk.
Automate. Once the templates and workflows are stable, invest in automation for the fields that can be populated automatically.
The end state is a documentation practice where every AI system in your organisation has documentation appropriate to its tier, that documentation is current, and it can be produced on demand for regulators, auditors, or internal review.
The next article moves from documentation to the substantive practices that documentation captures: responsible AI in operation. Fairness, explainability, and human oversight are the disciplines your model cards and system cards document. Getting them right on paper is only useful if they are right in practice.