Risk modeling and model validation
A machine learning risk model pays off only if its extra accuracy survives effective challenge. If you cannot show a validator why it behaves as it does, where it breaks and that its gain justifies its complexity, it does not reach production, however good the backtest looks.
Part of AI for Financial Services & Capital Markets. 9 sources, checked .
- of UK financial firms surveyed in 2024 report only partial understanding of the AI technologies they use
- 46%
- of UK financial firms surveyed in 2024 report only partial understanding of the AI technologies they use
- Bank of England and Financial Conduct Authority, 2024(opens in a new tab)
- AI use cases at UK financial firms are third-party implementations, up from 17% in 2022
- 1 in 3
- AI use cases at UK financial firms are third-party implementations, up from 17% in 2022
- Bank of England and Financial Conduct Authority, 2024(opens in a new tab)
- of UK financial firms using AI apply at least one explainability method (2024)
- 81%
- of UK financial firms using AI apply at least one explainability method (2024)
- Bank of England and Financial Conduct Authority, 2024(opens in a new tab)
The problem
Machine learning is already inside bank risk functions, but understanding has not kept pace. In the 2024 Bank of England and FCA survey, 75% of UK financial firms were using AI and 81% of those users applied some kind of explainability method.1 Yet 46% of firms said they had only partial understanding of the AI technologies they use, and a third of all use cases were third-party implementations.1 That gap between use and understanding is exactly what model validation exists to close, and it is where most AI risk models stall.
The rules changed in 2025 and 2026. On 17 April 2026 the Federal Reserve, OCC and FDIC replaced SR 11-7 with revised, risk-based guidance, expected to be most relevant to banks with over $30 billion in assets.2 It ties the rigor of oversight to model materiality and covers non-generative machine learning, while leaving generative and agentic AI out of scope.3 The ECB added machine learning expectations to its internal models guide so that such models are adequately explainable and their performance justifies their complexity.5
Consumer credit carries extra duties. The EU AI Act classes AI that evaluates the creditworthiness of individuals or sets their credit score as high risk, and the 2026 Digital Omnibus moved the start date for stand-alone high-risk systems to 2 December 2027.6,7 In the US, Regulation B requires the reasons on an adverse action notice to be specific and to indicate the principal reasons for the decision.8 Respondents to Treasury's 2024 request for information warned that opaque models may conflict with such notice requirements.9
What AI changes is the cost of producing evidence, not the need for it. Both the US guidance and the PRA treat model risk as rising with complexity, and the PRA asks firms to rate complexity partly on interpretability, explainability and the potential for bias.3,4 The work that pays is building the challenger, benchmarks, stability tests and documentation alongside the model, so validators review evidence instead of reconstructing it.
What we build
- 01
Machine learning challengers for credit and loss models
A gradient-boosted or neural challenger built next to your champion PD, LGD or loss-forecasting model, with like-for-like benchmarks that show where any gain comes from and which segments it helps.
- 02
Validation evidence generated with every build
Out-of-time and out-of-sample tests, sensitivity and stability analysis, and outcomes analysis produced automatically on each model run and filed against your model risk policy, so validators start from evidence.
- 03
Reason codes that hold up on an adverse action notice
Per-decision reasons drawn from the model's actual drivers, tested for consistency across similar applicants and mapped to the principal-reason language Regulation B requires.
- 04
Vendor model test bench
A standing test bench that runs third-party scores and models on your own portfolio, tracks performance and drift, and documents any customization or overlay for validation.
- 05
Model inventory with materiality and complexity tiering
An inventory that records each model's purpose, exposure, complexity rating and monitoring thresholds, and flags when a model's use changes enough to need revalidation.
Where it goes wrong
The failure modes we design against from the first week, and the control for each.
A gain that disappears out of time
Flexible models can fit the development window and fade when conditions shift. The control is out-of-time and out-of-sample testing at build, documented performance thresholds, and ongoing monitoring that triggers recalibration or redevelopment when results drift outside them.3
Explanations nobody has challenged
Validators often test the score and accept the explanation. If reason codes change for near-identical applicants, the notice you send cannot be specific and accurate. Test explanation stability as a validation item in its own right, and constrain features where the business logic demands it.8,4
Vendor models treated as black boxes
Vendors may withhold code, data or methodology, but the guidance still expects you to understand a vendor model's conceptual soundness, development data and performance. Write evidence rights into the contract and run your own outcomes analysis and monitoring.3
Generative AI slipped into the model chain
Generative and agentic AI sit outside the US guidance, but the agencies expect your own risk management to govern them. An LLM that extracts features or writes credit memos becomes an ungoverned dependency unless you inventory it and set controls for it under your own policy.3
Rules that apply
SR 26-2 / OCC Bulletin 2026-13, Revised Guidance on Model Risk Management (US)
Primary text on occ.gov(opens in a new tab)Scales oversight to model materiality and expects validation of conceptual soundness, outcomes analysis and ongoing monitoring, including for vendor models; it replaced SR 11-7 in April 2026.
PRA SS1/23, Model risk management principles for banks (UK)
Primary text on bankofengland.co.uk(opens in a new tab)Requires firm-wide model tiering by materiality and complexity, independent validation, and controls for the risks of using AI in modelling techniques.
EU AI Act, Annex III point 5(b)
Primary text on ai-act-service-desk.ec.europa.eu(opens in a new tab)Treats AI that evaluates the creditworthiness of individuals or establishes their credit score as high risk, with obligations for stand-alone systems applying from 2 December 2027.
Regulation B, 12 CFR 1002.9 (US)
Primary text on consumerfinance.gov(opens in a new tab)Requires the statement of reasons for adverse action to be specific and to indicate the principal reasons, which a credit model must be able to support.
How to measure it
- Share of material models validated before first use, by tier
- Lift over the champion model on out-of-time data, after overlays
- Number and age of open validation findings
- Performance and population stability against documented thresholds
- Weeks from model build to validation sign-off
Questions buyers ask
Does SR 11-7 still apply to AI models?
No. On 17 April 2026 the Federal Reserve, OCC and FDIC replaced SR 11-7 with revised model risk guidance, issued as SR 26-2 and OCC Bulletin 2026-13. It covers traditional statistical models and non-generative machine learning, and is most relevant to banks with over $30 billion in assets. Generative and agentic AI are outside its scope, but the agencies still expect your own risk management to set controls for them.
Can we use machine learning for credit decisions and still explain declines?
Yes, if you design for it from the start. Regulation B requires adverse action reasons to be specific and to name the principal reasons. That means reason codes drawn from the model's real drivers, tested for consistency, and reviewed by validators. In the EU, creditworthiness models for individuals are high-risk AI systems, with the Act's high-risk obligations applying from 2 December 2027.
How do we validate a vendor model we cannot see inside?
The 2026 US guidance keeps vendor models in scope even when the vendor withholds code or data. You are expected to understand the model's conceptual soundness, design, development data and performance, then run ongoing monitoring and outcomes analysis on your own portfolio. Contract for the evidence you need, and document every customization you make as part of validation.
Sources
Every figure on this page comes from the sources below. We checked each link on .
- 1.Artificial intelligence in UK financial services - 2024(opens in a new tab)Bank of England and Financial Conduct Authority, November 21, 2024 · bankofengland.co.uk
- 2.SR 26-2: Revised Guidance on Model Risk Management(opens in a new tab)Board of Governors of the Federal Reserve System, April 17, 2026 · federalreserve.gov
- 3.Supervisory Guidance on Model Risk Management (attachment to OCC Bulletin 2026-13)(opens in a new tab)Federal Reserve, FDIC and Office of the Comptroller of the Currency, April 17, 2026 · occ.gov
- 4.SS1/23 Model risk management principles for banks(opens in a new tab)Bank of England, Prudential Regulation Authority, May 17, 2023 · bankofengland.co.uk
- 5.ECB publishes revised guide to internal models(opens in a new tab)European Central Bank, Banking Supervision, July 28, 2025 · bankingsupervision.europa.eu
- 6.AI Act (Regulation (EU) 2024/1689), Annex III(opens in a new tab)European Commission, AI Act Service Desk, 2024 · ai-act-service-desk.ec.europa.eu
- 7.Digital Omnibus on AI (Legislative Train Schedule)(opens in a new tab)European Parliament, July 24, 2026 · europarl.europa.eu
- 8.12 CFR 1002.9 Notifications (Regulation B)(opens in a new tab)Consumer Financial Protection Bureau · consumerfinance.gov
- 9.Report on the Uses, Opportunities, and Risks of Artificial Intelligence in the Financial Services Sector(opens in a new tab)U.S. Department of the Treasury, December 2024 · home.treasury.gov