There is a version of model risk management that has existed in banks for decades. It grew up around credit scoring models, market risk models, and regulatory capital models. Its bible in the United States is the Federal Reserve’s SR 11-7 guidance from 2011. In the UK it is the PRA’s SS1/23. In Australia it appears in APRA’s expectations across CPS 220, CPS 230, and CPG 235. In Singapore it lives in MAS Guidelines on model risk management and Technology Risk Management.
That existing discipline is one of the most valuable inheritances the AI governance movement has, because it already knows a great deal about how to manage risk in mathematical models that produce decisions affecting real people. Independent validation. Ongoing monitoring. Change management. Documentation standards. Reporting hierarchies. These are not new problems. They are old problems with new subjects.
The new subject changes some things. Traditional models are deterministic (given inputs, they produce reproducible outputs) and transparent (their coefficients and structure can be inspected). Modern AI models are probabilistic (the same input can produce different outputs) and opaque (their behaviour emerges from parameters no human can inspect meaningfully). Large language models compound the problem by operating on unstructured inputs, producing unstructured outputs, and behaving as if they had general capabilities across many tasks at once.
This article is about how to extend the discipline of model risk management to modern AI, including large language models. It builds on the traditional MRM foundations, adapts them to AI-specific realities, and points to the practices that matter most.
The traditional MRM foundation
If you have never worked with an established model risk management framework, three principles are worth understanding upfront. They come primarily from SR 11-7 and its equivalents, but the principles travel.
Effective challenge. Model owners, model developers, and model users all have incentives to trust their models. Effective challenge requires a second set of eyes, structurally independent from the first, whose job is to find problems with the model. Independent validation is the institutional expression of this principle. Without effective challenge, model risk management degrades into self-review, which is not risk management.
Fitness for purpose. A model is only valid for the purpose it is intended for and the data it was designed to handle. A model trained on retail credit data is not a wholesale credit model. A model calibrated on pre-2020 conditions may not perform in 2026 conditions. Fitness for purpose is not a one-time assessment; it is a continuous discipline.
Documentation and reproducibility. A model that cannot be documented cannot be governed. A model whose training and validation cannot be reproduced cannot be defended when questioned. Documentation is not overhead; it is the substrate on which everything else rests.
These principles apply to modern AI. They apply differently, because the systems are different, but they apply.
The tiering question, again
Just as with the overall AI governance program (article 2), model risk management needs to be proportionate. Not every model deserves the same intensity of oversight.
Traditional MRM frameworks use a tiering approach based on materiality and complexity. A high-materiality, high-complexity model gets independent validation, ongoing monitoring, formal documentation, and regular reporting. A low-materiality model may get a lighter touch: developer validation, periodic review, simpler documentation.
For AI systems, materiality should incorporate the harm categories from article 3. Financial impact is one dimension. Impact on individuals through algorithmic decisions is another. Reputational impact is a third. Regulatory impact (particularly under the EU AI Act’s high-risk classification, or MAS materiality, or APRA operational impact assessment) is a fourth.
Complexity should incorporate the technology-specific factors. A traditional logistic regression is low complexity. A gradient-boosted ensemble is medium. A neural network is high. A large language model is very high. Complexity affects how difficult validation is, what monitoring is possible, and what interpretability tools are available.
The result is a tiering matrix. Systems in the high-materiality, high-complexity cell get the full weight of MRM discipline. Systems in the low-materiality, low-complexity cell get the minimum. The middle gets calibrated. Article 2 covered how to design the tiering; here I am extending it specifically to models.
Figure 1: The MRM lifecycle for AI
Figure 1 shows the AI model risk management lifecycle: initiation (identifying the need and defining the use case), development (data, training, validation), independent challenge (second-line review), approval (formal sign-off with defined authority), deployment (production integration), monitoring (continuous), review (periodic reassessment), and decommissioning (formal retirement). Each stage produces artefacts that feed the next; the flow is not linear but iterative. The figure also shows where GOVERN, MAP, MEASURE, and MANAGE from the NIST framework operate across the lifecycle.
Independent validation for AI
Independent validation is the hardest thing to get right for AI, particularly for foundation models and LLMs. Let me walk through why and how.
For a traditional statistical model, independent validation is well-established. The validator has access to the training data, the model specification, the validation results, and the mathematical properties of the technique. The validator can rebuild the model, compare results, run stress tests, and produce a defensible opinion.
For a modern AI model, particularly one built on a foundation model, the validator often cannot access the training data. They cannot rebuild the model. They may not have full visibility into the model’s structure. The mathematical properties are not always well characterised. What can independent validation actually mean in this setting?
The pragmatic answer is that validation for AI focuses on behaviour rather than internal structure. The validator cannot fully audit the model’s internals, but they can rigorously test its behaviour against defined criteria. This shift is fundamental and has several implications.
Validation becomes evaluation-heavy. You need test sets that are held out from any tuning, red-team evaluations for adversarial robustness, fairness evaluations across relevant demographic slices, and stress tests for unusual inputs. For LLMs, you also need evaluations of hallucination, refusal behaviour, and prompt injection resistance.
Validation becomes documentation-dependent. Because the model itself cannot be fully inspected, the validator relies on the developer’s documentation of what was done, how it was done, and why. Poor documentation produces validation opinions that are qualified or negative.
Validation becomes continuous. Traditional models could be validated at deployment and re-validated periodically. AI models require continuous validation because their behaviour can shift with input drift, retraining, or upstream provider changes.
Validation may include third-party attestations. For foundation models used under commercial terms, the validator may need to rely on provider representations, third-party audits (including SOC 2 and ISO 42001), and public safety documentation, in combination with the deployer’s own evaluation on the specific use case.
The end result is a validation practice that looks different from traditional MRM but achieves the same purpose: providing an independent, defensible view of whether the model is fit for its intended use.
Data governance for models
Data risk is inseparable from model risk. Traditional MRM has always required data quality, provenance, and appropriateness assessments. AI intensifies these requirements.
For traditional models, data risk focused on completeness, accuracy, and representativeness of a defined set of features.
For AI models, data risk extends to:
Training data provenance. Where did the data come from? What consent or lawful basis supports its use? What licence conditions apply? For foundation models, this is the question at the centre of ongoing copyright litigation. For enterprise fine-tuning, it is a question you need to answer for your own risk assessment.
Training data appropriateness. Is the data suitable for the model’s intended purpose? Data appropriate for a general chatbot may be inappropriate for medical advice; data appropriate for English-speaking users may be inappropriate for other languages.
Training data bias. Article 3 covered this in the risk sense. In the MRM sense, the training data’s biases become the model’s biases, and identifying them requires deliberate effort at the data stage.
Feedback data. For models that learn from user interactions, the feedback loop can amplify biases and drift the model in unintended directions over time. This needs specific governance.
Retrieval-augmented data. For RAG systems, the retrieval corpus is effectively an extension of the model’s knowledge. Its governance requirements are similar to training data governance but often less rigorous in practice. This is a common gap.
For foundation models used under commercial terms, most of these questions cannot be answered by the deployer alone. They require reliance on the provider’s disclosures, the emerging Code of Practice compliance under the EU AI Act, and other third-party mechanisms. Building this reliance defensibly is part of the deployer’s own MRM discipline.
Monitoring in production
The traditional MRM concept of ongoing monitoring is where AI most obviously requires adaptation.
For a traditional model, ongoing monitoring watches for input drift (are the input characteristics changing?), output drift (are the outputs changing?), and performance degradation (is the model still accurate against ground truth?). The monitoring is typically monthly or quarterly.
For an AI model, monitoring needs to be:
Continuous. Batch monitoring on monthly cycles is inadequate for probabilistic systems whose failure modes can appear and evolve quickly. Real-time or near-real-time monitoring is expected.
Multi-dimensional. Beyond input drift and output drift, monitoring should watch for fairness drift, hallucination indicators, tool-use anomalies (for agentic systems), latency changes, cost changes (particularly for API-based models), and refusal rate changes.
Behaviour-focused. For LLMs, monitoring should include sampling of actual model behaviour against test suites. Some organisations run daily or weekly evaluation sets against production models to detect changes.
Human-in-the-loop. For high-risk use cases, human review of a sample of model outputs continues to be part of the monitoring approach. Automation cannot fully substitute for human judgement in the monitoring layer.
The tooling for AI monitoring has matured significantly through 2025 and 2026. Multiple commercial platforms now offer AI observability, evaluation, and monitoring capabilities. The build-versus-buy question depends on scale and specificity of your use cases.
Change management
Traditional MRM has strong change management: any material change to a model requires documented justification, validation of the change, and formal re-approval before deployment.
AI systems present specific change management challenges:
Foundation model updates. When your provider updates the underlying model, your system’s behaviour can change without any code change on your side. You need mechanisms to detect these changes and processes to reassess model behaviour when they occur. Many providers now offer version pinning; use it where the use case requires stability.
Fine-tuning cycles. Fine-tuning intended to improve performance can inadvertently change behaviour in unintended ways. Formal validation before deployment is essential.
Prompt changes. Changes to prompts, system messages, or retrieval configurations can materially affect model behaviour. These are typically not treated as model changes but they should be, particularly for high-tier use cases.
Adjacent system changes. For AI systems integrated with other systems, changes upstream (in data sources, retrieval systems, or connected tools) can affect model behaviour. Change management must consider these dependencies.
Article 15 goes into agentic AI-specific change management, where the challenges compound.
Model documentation
Documentation is where MRM meets audit and regulatory review. It also happens to be where most MRM programs struggle.
For AI systems, effective model documentation should cover:
- Intended use, including in-scope and out-of-scope cases
- Model architecture and version
- Training data (with appropriate provenance and privacy considerations)
- Training methodology
- Evaluation results, including fairness and adversarial testing
- Known limitations and failure modes
- Deployment configuration
- Monitoring and incident response
- Change history
For foundation models used under commercial terms, deployer documentation supplements provider documentation. Provider model cards, system cards, and safety documentation form part of the record; deployer-specific documentation covers the use case, integration, and evaluation on that specific application.
Article 13 covers documentation in depth, including specific formats (model cards, datasheets, and audit trails) that have become standard.
Governance interfaces
Model risk management does not operate in isolation. It sits at the intersection of several other governance disciplines, and getting the interfaces right matters.
With operational risk. Model failures are a subset of operational risk. Serious model incidents feed into the operational risk framework and its escalation processes.
With information security. Model vulnerabilities (prompt injection, model theft, data poisoning) are cybersecurity concerns. MRM and infosec need to be integrated, not parallel.
With data governance. The data flowing into models is the data covered by data governance. The two disciplines share substantive responsibilities and should share evidence.
With third-party risk. Foundation model providers are third parties. Their risk management flows into your MRM through vendor assessment, contracts, and monitoring. Article 16 covers this in depth.
With regulatory compliance. For regulated entities, MRM is where regulatory expectations meet operational practice. APRA, MAS, and other regulators do not distinguish sharply between MRM discipline and general AI governance; the practitioner has to build both.
Figure 2: MRM tiering matrix for AI
Figure 2 shows the tiering matrix for AI model risk management. The horizontal axis is model complexity (low: statistical models; medium: gradient-boosted or neural network; high: foundation models; very high: LLMs and agentic systems). The vertical axis is materiality (low: internal productivity; medium: customer support; high: automated decisions; very high: high-risk decisions with regulatory implications). The four quadrants illustrate the intensity of MRM discipline required. The intersection of high complexity and high materiality (top-right) is where the discipline is most demanding, and where the traditional MRM inheritance is most valuable.
The direction of travel
Two observations to close on.
First, foundation model providers are increasingly meeting deployer MRM needs partway. Model cards, safety documentation, capability disclosures, and third-party attestations are all improving. This trend will continue and should be encouraged through procurement.
Second, the MRM discipline is going to keep converging with AI governance broadly. In 2020, model risk management was a specialist discipline within financial services. By 2026, it is a foundational element of AI governance in every industry. In 2028, the vocabulary will have merged. The specific expertise that traditional MRM brings (independent validation, ongoing monitoring, documentation, effective challenge) will remain valuable throughout that convergence.
The next article covers documentation specifically: model cards, datasheets for datasets, and audit trails, which have emerged as the practical formats for AI documentation across the industry.