Data Quality and Migration Testing on Databricks

A Tier 2 financial services organisation partnered with Innablr to support a major data modernisation programme

Platform: Databricks | Power BI | SAP HANA | Pega BIX

Overview

A Tier 2 financial services organisation partnered with Innablr to support a major data modernisation programme, migrating compliance reporting and legacy data systems to a new Databricks environment. Innablr deployed a blended onshore and offshore team of 10 specialists, embedded directly into the client's Agile (SAFe) delivery squads.

The Challenge

The client's data office was managing several concurrent, testing-heavy initiatives driven by upstream system changes:

•Legacy compliance report migration from manual processes to automated Power BI reports on Databricks

•SAP HANA upgrade, requiring extensive data transformation and re-validation of reporting structures to ensure continuity of critical financial reporting

•Master Data Management (MDM) implementation to establish a single source of truth and validate data flow across downstream systems

•Pega BIX integration, ingesting Pega BIX files into the client's Enterprise Data Management Platform (EDMP), requiring thorough data quality validation across ingestion pipelines

The work was predominantly testing-heavy rather than data engineering, reflecting the volume and complexity of ensuring data integrity across regulated systems during transition.

What Innablr Delivered

Innablr's hybrid team provided end-to-end data quality and migration testing across all workstreams. For the SAP HANA upgrade, the team validated that transformed data and updated reporting structures produced accurate outputs after the platform switch. For MDM, testers verified that data updates propagated correctly to downstream systems, confirming the integrity of the single source of truth. For the Pega BIX integration, the team validated the ingestion of Pega BIX files into the EDMP, ensuring data completeness, accuracy, and consistency across pipelines.

Alongside delivery, Innablr developed a foundational automated data testing framework capable of generating test plans, test cases, and Python-based testing scripts automatically upon development completion — reducing manual testing effort and setting the foundation for scalable data quality practices.

Outcomes

•Seamless continuity of compliance and regulatory reporting through the SAP HANA upgrade

•Validated MDM data governance, with confirmed single source of truth across systems

•Successful ingestion and quality assurance of Pega BIX data into the client's EDMP

•Automated data testing framework projected to reduce testing resource requirements by up to 50%

•Reusable framework and methodology applicable across future client engagements