Continuous AI Assurance
Launch approval is a snapshot of a system that keeps changing: data drifts, vendors update models and users find new uses. Assurance pays off when monitoring is tied to the risks you accepted at approval, with thresholds that trigger action and evidence produced as a by-product rather than assembled for each audit.
Part of Security, Assurance & Governance, in the Run mode. 9 sources, checked .
- of 128 model and dataset pairs showed quality degradation over time in a study across four industries (published 2022)
- 91%
- of 128 model and dataset pairs showed quality degradation over time in a study across four industries (published 2022)
- Scientific Reports (Vela D, Sharp A, Zhang R, Nguyen T, Hoang A, Pianykh OS), 2022(opens in a new tab)
- AI incidents recorded by the AI Incident Database in 2025, up from 233 in 2024
- 362
- AI incidents recorded by the AI Incident Database in 2025, up from 233 in 2024
- Stanford Institute for Human-Centered Artificial Intelligence, 2026(opens in a new tab)
- of organizations rated their AI incident response as excellent in 2025, down from 28% in 2024
- 18%
- of organizations rated their AI incident response as excellent in 2025, down from 28% in 2024
- Stanford Institute for Human-Centered Artificial Intelligence, 2026(opens in a new tab)
The problem
Models degrade after launch, often quietly. In a study published in Scientific Reports, researchers ran four standard model types against 32 datasets from healthcare, transportation, finance and weather, and observed quality degradation over time in 91% of the 128 model and dataset pairs.1 Hosted language models change too. Researchers found GPT-4's accuracy at identifying prime numbers fell from 84% in March 2023 to 51% in June 2023, and concluded that LLM services need continuous monitoring.2
Incidents are rising while confidence in handling them falls. The AI Incident Database recorded 362 AI incidents in 2025, up from 233 in 2024.3 In the 2025 survey behind the Stanford AI Index, the share of organizations rating their AI incident response as excellent dropped from 28% to 18%. Among organizations that reported incidents, the share with three to five of them rose from 30% to 50%.3
Regulators expect monitoring for the life of the system. The EU AI Act treats risk management for high-risk systems as a continuous process across the lifecycle, requires providers to collect and analyse performance data through post-market monitoring, and requires serious incidents to be reported no later than 15 days after awareness.4 The 2026 Digital Omnibus moved Annex III high-risk obligations to 2 December 2027 and requires the Commission to publish post-market monitoring guidance, including a template, by 2 September 2027.5 US federal policy requires ongoing monitoring of high-impact AI for changes after deployment.6
Banks face a gap. US banking agencies replaced SR 11-7 with SR 26-2 in April 2026. It expects ongoing monitoring of models, including vendor models, but places generative and agentic AI outside its scope.7 For those systems you have to define the monitoring yourself. Open tooling helps: the UK AI Security Institute's Inspect framework provides more than 200 pre-built evaluations that can be run on any model.8
What you get
- 01
Monitoring plan for each system
For every production AI system, the metrics, thresholds and owners tied to the risks accepted at approval, covering accuracy, drift, fairness, safety and security, and the action each threshold triggers.
- 02
Scheduled evaluation suites
Automated test suites, including red-team and jailbreak tests for generative systems, rerun on every change of model, prompt, retrieval data or vendor version, and on a fixed schedule.
- 03
Evidence pipeline
Logs, test results, approvals and incident records collected automatically into one evidence store, mapped to your controls and to frameworks such as the NIST AI RMF, ISO/IEC 42001 and the EU AI Act.
- 04
Quarterly assurance report
A report to the governance council and board on each system's status against its thresholds, open findings, incidents and material changes since the last quarter.
- 05
AI incident runbook
A working definition of an AI incident, severity levels, who decides, rollback and shutdown steps, and the regulatory reporting clocks that apply to you.
Where it goes wrong
The failure modes we design against from the first week, and the control for each.
Monitoring uptime instead of behavior
Infrastructure dashboards show the service is up while its answers get worse. Monitor output quality against labeled samples and the risk metrics accepted at approval, because degradation over time is the norm, not the exception.1
Missing vendor model changes
Hosted models can change behind the same product name, and behavior can shift a lot within months.2 Pin model versions where you can, track vendor release notes, and rerun your evaluation suite on every change before it reaches users.
Assuming model risk guidance covers generative AI
SR 26-2 places generative and agentic AI outside its scope,7 so a bank's model validation team may not own them by default. Assign monitoring and sign-off for these systems explicitly through the governance council.
Evidence assembled for the audit
Evidence pulled together after the fact is incomplete and expensive. The EU AI Act expects post-market data to be actively and systematically collected and analysed.4 Capture logs, test results and approvals as the system runs, so quarterly reports and regulator requests draw on the same record.
Standards and rules
EU AI Act, Articles 9, 72 and 73
Primary text on eur-lex.europa.eu(opens in a new tab)Providers of high-risk AI must run risk management across the lifecycle, operate post-market monitoring and report serious incidents within 15 days of awareness, with shorter limits for the gravest cases.
SR 26-2, Revised Guidance on Model Risk Management (April 2026)
Primary text on federalreserve.gov(opens in a new tab)US banking agencies expect ongoing monitoring of models, including vendor models, scaled to materiality; generative and agentic AI are outside its scope.
OMB Memorandum M-25-21
Primary text on whitehouse.gov(opens in a new tab)US federal agencies must monitor high-impact AI for performance and adverse impacts, designed to detect changes to the system, its data or its context of use after deployment.
NIST AI RMF 1.0, MEASURE 2.4 and MANAGE 4.1
Primary text on nvlpubs.nist.gov(opens in a new tab)AI system behavior is monitored in production, and post-deployment monitoring plans cover user input, override, incident response, recovery and change management.
How to measure it
- Share of production AI systems with a current monitoring plan and a named owner
- Threshold breaches caught by monitoring versus problems reported by users or customers
- Time from a threshold breach to a decision to retrain, roll back or accept
- Share of model, prompt and vendor version changes that passed the evaluation suite before release
- Open assurance findings past their due date, by severity
Questions buyers ask
How is continuous AI assurance different from model validation?
Validation is a point-in-time judgment that a model is fit for purpose before it is used. Continuous assurance checks that it stays fit: it monitors performance, drift, fairness and security in production, reruns tests whenever something changes, and produces evidence on a schedule. Validation tells you the system was sound at launch. Assurance tells you whether it still is, and flags the moment it is not.
How often should AI systems be re-tested?
On every material change, and on a schedule set by risk. A new model version, prompt, retrieval source or vendor release should trigger the evaluation suite automatically. Beyond that, high-risk systems warrant frequent checks of key metrics and at least a quarterly review, while low-risk internal tools can be reviewed less often. What matters is that the frequency is written down, justified and actually followed.
What counts as an AI incident?
Any event where an AI system causes or nearly causes harm or breaks a requirement: wrong decisions that affect customers, discriminatory outcomes, data leaking through prompts or outputs, unsafe content, or a security compromise of the model or its data. Define severity levels in advance and map them to your reporting duties, such as the EU AI Act serious incident rules or your sector regulator, so nobody debates definitions mid-incident.
Sources
Every figure on this page comes from the sources below. We checked each link on .
- 1.Temporal quality degradation in AI models(opens in a new tab)Scientific Reports (Vela D, Sharp A, Zhang R, Nguyen T, Hoang A, Pianykh OS), July 8, 2022 · nature.com
- 2.How Is ChatGPT's Behavior Changing Over Time?(opens in a new tab)Harvard Data Science Review (Chen L, Zaharia M, Zou J), March 12, 2024 · hdsr.mitpress.mit.edu
- 3.The 2026 AI Index Report, Chapter 3: Responsible AI(opens in a new tab)Stanford Institute for Human-Centered Artificial Intelligence, 2026 · hai.stanford.edu
- 4.Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act)(opens in a new tab)European Parliament and Council of the European Union, July 12, 2024 · eur-lex.europa.eu
- 5.Regulation (EU) 2026/1744 amending Regulations (EU) 2024/1689, (EU) 2018/1139 and (EU) 2023/1230 (Digital Omnibus on AI)(opens in a new tab)European Parliament and Council of the European Union, July 8, 2026 · eur-lex.europa.eu
- 6.M-25-21: Accelerating Federal Use of AI through Innovation, Governance, and Public Trust(opens in a new tab)Executive Office of the President, Office of Management and Budget, April 3, 2025 · whitehouse.gov
- 7.SR 26-2: Revised Guidance on Model Risk Management(opens in a new tab)Board of Governors of the Federal Reserve System, April 17, 2026 · federalreserve.gov
- 8.Inspect: An open-source framework for large language model evaluations(opens in a new tab)UK AI Security Institute, 2026 · inspect.aisi.org.uk
- 9.Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1(opens in a new tab)National Institute of Standards and Technology, January 2023 · nvlpubs.nist.gov