Loading
Skip to content
Miracle Minds AI
  • Try “HIPAA”, “model risk”, “FinOps” or “agentic”.

Data Platform Modernization

A new platform pays off only if it fixes the data paths your AI use cases depend on, with lineage and access controls carried all the way to the model. Migrations that move everything first and govern later recreate the old problems on a newer bill.

Part of AI & Data, in the Build mode. 10 sources, checked .

Top perceived AI risks that relate to data, UK financial services firms (Bank of England and FCA survey, 2024)
4 of 5
Top perceived AI risks that relate to data, UK financial services firms (Bank of England and FCA survey, 2024)
Bank of England and Financial Conduct Authority, 2024(opens in a new tab)
Chief data officers confident their organization can use unstructured data in a way that delivers business value (IBM, 2025)
26%
Chief data officers confident their organization can use unstructured data in a way that delivers business value (IBM, 2025)
IBM Institute for Business Value, 2025(opens in a new tab)
AI projects unsupported by AI-ready data that Gartner predicts organizations will abandon through 2026
60%
AI projects unsupported by AI-ready data that Gartner predicts organizations will abandon through 2026
Gartner, 2025(opens in a new tab)

The problem

The data under AI is still the weak point. In the Bank of England and FCA's 2024 survey, four of the top five risks firms perceived in AI were about data: privacy and protection, quality, security, and bias and representativeness1. The BIS Financial Stability Institute wrote in 2026 that the sector's many data sources hold rich but fragmented information, that this has historically produced inconsistent data quality, and that scaling generative AI amplifies the weakness2. Gartner predicts organizations will abandon 60% of AI projects unsupported by AI-ready data through 20263.

Generative AI moves the problem into documents. IBM's 2025 survey of 1,700 chief data officers found only 26% confident their organization can use unstructured data in a way that delivers business value, and 81% said they bring AI to data rather than centralizing it4. Retrieval adds a failure that older platforms never had: OWASP warns that similarity search does not respect access controls, so permissions must be checked before content is retrieved, because filtering after generation cannot undo what the model has already seen5.

For banks the standard is old and still unmet. BCBS 239 expects data architecture that supports risk data aggregation in normal times and in stress, with data aggregated on a largely automated basis6. In January 2026 the Basel Committee reported that legacy systems and distributed data estates still complicate lineage, and that balancing manual and automated processes across fragmented estates is resource-intensive7. Good modernization builds the governed paths the priority use cases need, removes the manual steps on them, and keeps lineage intact from source to model.

What you get

  1. 01

    Target architecture tied to use cases

    A platform design covering storage, streaming, features and retrieval, sized to the data paths your priority use cases need, with a clear call on what to keep, migrate or retire.

  2. 02

    Governed pipelines

    Ingestion and transformation with automated quality checks, captured lineage and data contracts between producers and consumers, so upstream changes are caught before they reach a model.

  3. 03

    Retrieval layer for generative AI

    Document ingestion, chunking, embedding and indexing with source permissions carried into the index and enforced before retrieval, plus versioning so answers can be traced to the documents behind them.

  4. 04

    Features and streams for machine learning

    Shared, versioned feature definitions and event streams, so models train and serve on the same data and freshness matches what each decision needs.

  5. 05

    Migration and decommissioning plan

    A phased cutover with reconciliation against the legacy system at each step, and the old pipelines and stores switched off so you stop paying for both.

Where it goes wrong

The failure modes we design against from the first week, and the control for each.

  • Moving the mess to a new platform

    Data migrated without owners, definitions or quality checks arrives broken in a more expensive place. The Basel Committee found several banks still lack a common taxonomy and complete data lineage8. The control: agree owners, definitions and quality thresholds for each domain before it moves, and reconcile against the source after.

  • Retrieval that ignores permissions

    Indexing documents without their access rights lets the model show people content they could never open directly. OWASP says to authorize before retrieval5. The control: carry source permissions into the index, filter at query time, and test with users in different roles before launch.

  • Manual steps left on critical paths

    Spreadsheets and hand-offs between systems break lineage and add errors. BCBS 239 expects risk data to be aggregated on a largely automated basis6, and supervisors still find the balance of manual and automated work hard for banks7. The control: list every manual touchpoint on the paths that feed AI and automate them, with reconciliation.

  • Blind spots in third-party data and services

    Vendors and cloud services sit inside your data paths, and the BIS notes third-party dependencies limit data visibility across the AI supply chain2. The control: require lineage, change notices and quality reporting from providers in the contract, and monitor their feeds like your own.

Standards and rules

  • BCBS 239, Principles 2, 3 and 6

    Primary text on bis.org(opens in a new tab)

    Data architecture must support risk data aggregation in normal times and in stress, aggregate data on a largely automated basis, and meet ad hoc and supervisory requests, which sets the bar for any bank platform that also feeds AI.

  • EU AI Act, Article 10 (data and data governance)

    Primary text on ai-act-service-desk.ec.europa.eu(opens in a new tab)

    High-risk systems need governance over data collection and origin and over preparation steps such as annotation, labelling, cleaning and aggregation, so the platform must record them.

  • CDMC, Cloud Data Management Capabilities (EDM Association)

    Primary text on edmcouncil.org(opens in a new tab)

    Defines 14 key controls for protecting sensitive data in cloud environments, a practical checklist for platform design and audit evidence.

How to measure it

  • Data freshness: time from source event to availability for models, against the target for each use case
  • Share of pipelines feeding AI with automated quality checks and captured lineage
  • Data incidents reaching production models each quarter, and time to detect them
  • Share of retrieval queries with permissions enforced before retrieval
  • Legacy pipelines and stores decommissioned against plan, and run cost per workload

Questions buyers ask

Do we need a lakehouse for AI?

Not necessarily. You need governed, timely data for the use cases you plan to run, with lineage and access controls you can show an auditor. A lakehouse is one way to get there and often a good one, but the choice should follow from the data paths, volumes, latency and skills you have, not from a vendor roadmap. Start from the use cases and work back to the architecture.

What does a data platform need for generative AI and retrieval?

A pipeline that ingests documents with their metadata, versions and access rights; chunking and embedding you can rerun when models change; an index that enforces permissions before content is retrieved; and lineage from each answer back to its source documents. It also needs retention and deletion rules that reach the index, so removed or expired content does not keep turning up in answers.

How do you modernize a data platform without disrupting regulatory reporting?

Run old and new side by side for each domain, reconcile outputs until they match within agreed tolerances, and cut over one domain at a time with a rollback plan. Keep lineage and controls documented throughout so reporting owners and auditors can follow the change. Decommission the legacy path only after the new one has passed a full reporting cycle.

Sources

Every figure on this page comes from the sources below. We checked each link on .

  1. 1.Artificial intelligence in UK financial services - 2024(opens in a new tab)Bank of England and Financial Conduct Authority, November 21, 2024 · bankofengland.co.uk
  2. 2.In data we trust? Emerging policy and supervisory approaches to AI data use in financial services (FSI Insights No 73)(opens in a new tab)Financial Stability Institute, Bank for International Settlements, March 2026 · bis.org
  3. 3.Lack of AI-Ready Data Puts AI Projects at Risk(opens in a new tab)Gartner, February 26, 2025 · gartner.com
  4. 4.IBM Study: Chief Data Officers Redefine Strategies as AI Ambitions Outpace Readiness(opens in a new tab)IBM Institute for Business Value, November 13, 2025 · newsroom.ibm.com
  5. 5.OWASP Top 10 for LLM Applications 2026(opens in a new tab)OWASP GenAI Security Project, August 3, 2026 · genai.owasp.org
  6. 6.Principles for effective risk data aggregation and risk reporting (BCBS 239)(opens in a new tab)Basel Committee on Banking Supervision, Bank for International Settlements, January 9, 2013 · bis.org
  7. 7.Implementation of the Principles for effective risk data aggregation and risk reporting (BCBS 239 Principles), BCBS Newsletter No 36(opens in a new tab)Basel Committee on Banking Supervision, Bank for International Settlements, January 6, 2026 · bis.org
  8. 8.Progress in adopting the Principles for effective risk data aggregation and risk reporting(opens in a new tab)Basel Committee on Banking Supervision, Bank for International Settlements, November 28, 2023 · bis.org
  9. 9.AI Act (Regulation (EU) 2024/1689), Article 10: Data and data governance(opens in a new tab)European Commission, AI Act Service Desk, 2024 · ai-act-service-desk.ec.europa.eu
  10. 10.CDMC: Cloud Data Management Capabilities framework(opens in a new tab)EDM Association (formerly EDM Council), September 2021 · edmcouncil.org