- Documented AI incidents reported in Stanford's 2026 AI Index, up from 233 in 2024
- 362
- Documented AI incidents reported in Stanford's 2026 AI Index, up from 233 in 2024
- Stanford Institute for Human-Centered Artificial Intelligence, 2026(opens in a new tab)
- Organizations that reported a breach targeting AI models or applications (IBM, 2026)
- 20%+
- Organizations that reported a breach targeting AI models or applications (IBM, 2026)
- IBM (Cost of a Data Breach Report 2026, Ponemon Institute), 2026(opens in a new tab)
- FinOps practitioners who now manage AI spend, up from 31% two years earlier (State of FinOps 2026)
- 98%
- FinOps practitioners who now manage AI spend, up from 31% two years earlier (State of FinOps 2026)
- FinOps Foundation, 2026(opens in a new tab)
The problem
Google engineers warned in 2015 that only a small fraction of a real-world machine learning system is the model code, and that it is common to incur massive ongoing maintenance costs in such systems1. Security data now shows the same pattern. In IBM's 2026 breach study, more than 20% of organizations reported a breach targeting AI models or applications, and the most common causes were compromised APIs, applications or plug-ins (27%) and cloud misconfigurations affecting AI workloads (27%)2.
Incidents and costs are both climbing. Stanford's 2026 AI Index counts 362 documented AI incidents, up from 233 in 2024, and says responsible AI is not keeping pace with capability3. In the FinOps Foundation's 2026 survey of 1,192 practitioners, 98% now manage AI spend, up from 31% two years earlier. Granular monitoring of AI spend, by tokens, LLM requests and GPU use, topped the list of tool features practitioners want and do not yet have4.
What good looks like is borrowed from site reliability engineering: define service level indicators, set objectives against them, and allow an error budget instead of chasing perfection5. For AI, the indicators must include answer quality, escalation and cost per transaction, not just uptime. NIST's AI RMF asks for post-deployment monitoring plans covering incident response, recovery and change management, and for assigned responsibility to deactivate systems that perform inconsistently with intended use6. NIST's generative AI profile adds vendor contracts that assign responsibility for incidents and for changes over time such as drift7.
What you get
- 01
Service definition and service levels
For each AI system, the indicators and targets we run it against: availability, latency, answer quality, escalation rate and cost per transaction, with error budgets and the agreed service levels.
- 02
Monitoring across the whole system
Telemetry for the model, retrieval, data feeds, integrations and agent tools, plus scheduled quality sampling of real outputs, so you see wrong answers as well as outages.
- 03
AI incident response
Runbooks for AI-specific failures such as harmful or wrong outputs, prompt injection, data exposure and vendor model changes, with severity levels, fallback modes, post-incident reviews and support for regulatory reporting.
- 04
Cost control
Spend tracked per use case down to tokens and compute, with budgets, alerts and choices on model routing and caching that keep cost per transaction inside agreed limits.
- 05
Controlled change and improvement
Every prompt, model, data or tool change passes a regression evaluation before release, and a regular service review turns incidents and user feedback into a ranked improvement backlog.
Where it goes wrong
The failure modes we design against from the first week, and the control for each.
Watching uptime, not answers
An AI system can be fully available and consistently wrong. Ordinary tests are not enough in a changing world, and live monitoring with automated response is critical1. Add quality indicators from sampled evaluation and treat a quality breach as an incident.
Silent vendor model changes
Hosted models change underneath you and behavior shifts with them. Pin model versions, rerun a regression set before any switch, and write responsibility for incidents and for changes such as drift into vendor contracts, as NIST suggests7.
Securing the model but not the plumbing
The most common causes of AI breaches in IBM's 2026 study were compromised APIs, applications or plug-ins and cloud misconfigurations2. Bring AI integrations, keys and agent permissions into the same patching, access review and logging as other production systems.
Cost surprises
AI spend scales with usage in ways budgets do not expect, and practitioners still lack granular monitoring of it4. Track unit cost per use case, set budgets with alerts, and review routing and caching before scaling a system to more users.
Standards and rules
NIST AI RMF, MANAGE function
Primary text on nvlpubs.nist.gov(opens in a new tab)Post-deployment monitoring plans, incident response, recovery and change management must be in place, with assigned responsibility to supersede, disengage or deactivate systems that do not perform as intended.
EU AI Act, post-market monitoring and incident reporting
Primary text on digital-strategy.ec.europa.eu(opens in a new tab)For high-risk systems, providers run post-market monitoring, deployers ensure human oversight and monitoring, and both report serious incidents; these rules apply from 2 December 2027 (Annex III) and 2 August 2028 (Annex I).
ISO/IEC 42001
Primary text on iso.org(opens in a new tab)Requires an AI management system that is maintained and continually improved, which a managed operations service should feed with monitoring results, incidents and corrective actions.
How to measure it
- Service level attainment per AI system, including a quality objective, not only availability
- Mean time to detect and mean time to resolve AI incidents
- Cost per transaction or per resolved case against budget
- Share of model, prompt and data changes released only after passing regression evaluation
- Escalation and fallback rates over time
Questions buyers ask
What does managed AI operations include?
It covers everything needed to keep an AI system useful after launch: monitoring of quality, latency and cost; incident response for wrong or harmful outputs, security events and vendor changes; controlled releases of model, prompt and data changes; cost control; and a regular service review. It runs under service levels agreed in advance, with a named owner accountable for each system.
How is running an AI system different from running ordinary software?
Ordinary software fails loudly; AI often fails quietly. Outputs vary from run to run, quality can degrade without a single error in the logs, hosted models can change without notice, and cost depends on how people use the system. That means you monitor sampled answers as well as infrastructure, test every change against a fixed evaluation set, and track unit cost.
What service levels should we set for an AI application?
Keep the usual availability and latency targets, then add a quality objective measured on sampled outputs, an escalation or fallback rate, and a cost per transaction ceiling. Agree how quality is scored and who scores it before launch. An error budget for each objective tells you when to pause new features and fix reliability instead.
Sources
Every figure on this page comes from the sources below. We checked each link on .
- 1.Hidden Technical Debt in Machine Learning Systems(opens in a new tab)Advances in Neural Information Processing Systems 28 (NeurIPS 2015), 2015 · proceedings.neurips.cc
- 2.IBM Study: One in Four Malicious Breaches are AI-Enabled, Costing Companies $6 Million on Average(opens in a new tab)IBM (Cost of a Data Breach Report 2026, Ponemon Institute), July 29, 2026 · newsroom.ibm.com
- 3.The 2026 AI Index Report(opens in a new tab)Stanford Institute for Human-Centered Artificial Intelligence, 2026 · hai.stanford.edu
- 4.State of FinOps 2026 Report(opens in a new tab)FinOps Foundation, 2026 · data.finops.org
- 5.Service Level Objectives (Site Reliability Engineering, Chapter 4)(opens in a new tab)Google, 2016 · sre.google
- 6.Artificial Intelligence Risk Management Framework (AI RMF 1.0)(opens in a new tab)National Institute of Standards and Technology, January 2023 · nvlpubs.nist.gov
- 7.Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1)(opens in a new tab)National Institute of Standards and Technology, July 2024 · nvlpubs.nist.gov
- 8.AI Act(opens in a new tab)European Commission, October 19, 2026 · digital-strategy.ec.europa.eu
- 9.ISO/IEC 42001:2023 - AI management systems(opens in a new tab)International Organization for Standardization, December 2023 · iso.org