← All customer stories

Detecting data drift before it impacts business insights

An illustrative use case: how Validatum monitors a health plan’s data lake for the problems that only appear between systems.

Customer story
·
Data & Eligibility
Detecting data drift before it impacts business insights
Nightly
Threshold-Based Alerts
Checks run on schedule as data changes and new sources are added.
40% → 5%
Distribution Break Caught
Illustrative example: one state’s share of records after an ETL load.
11
Supported Data Sources
Seven databases and warehouses, plus CSV, fixed-width, X12 and SQLite files.
CLIENT PERSPECTIVE

Overview

Illustrative use case.

A national health plan runs a data lake built up over 10 years, fed by many source systems. Each source system validates its own data, and each one looks healthy on its own.

The risk sits between them. Key Software’s Validatum monitors aggregate signals across systems on a nightly schedule and raises an alert when data quality, completeness, freshness or consistency drifts beyond an agreed threshold, so the data team can investigate before claims processing, reporting or AI models use the data.

The challenge

Constraints removed by design

Warehouses and lakes drop database validation to accept volume from many sources, so nothing enforces integrity once data lands.

Errors live between systems

Each system looks healthy in isolation. Errors appear between systems, where no one owns both the source and the target data model.

Drift is invisible at record level

The difference between 97% and 95% completeness cannot be seen by a user querying millions of rows.

Data integrity for AI model training

As clients intend to deploy AI models, the underlying data needs to be consistent and accurate.

The Validatum solution

Cross-database comparison

Source and target are configured as a pair, so checks run across the systems, not just inside one.

Distribution drift detection

Statistical divergence compares the shape of the data and alerts when it moves beyond a threshold.

Scalar, validity and schema checks

Values, business rules and structure are checked within a system.

Built-in SQL editor

Teams write custom rules for checks specific to their data.

Automated nightly scheduling

Checks run on schedule as data changes and new sources are added.

Category-level quality dashboard

Scores for each quality category are tracked over time.

Example: a distribution break no record-level check can catch

Records by state, before and after an ETL load. Every value is valid, but the distribution is not. Validatum measures the statistical distance between the two and alerts when it passes the agreed threshold.

  • California: 40% in the source system, 5% in the target after ETL
  • Texas: 20% in the source system, 20% in the target after ETL
  • Arizona: 15% in the source system, 50% in the target after ETL
  • Others: 25% in the source system, 25% in the target after ETL

Illustrative figures.

Where Validatum fits

Within a single system

Scalar, validity, schema and row-level checks inside one database, where drift builds up between releases.

Across distributed systems

Comparison where several systems feed a common warehouse with no live connection between them, for example after acquisitions or across subsidiaries.

Warehouse, OLAP and data lake

Platforms such as Snowflake and Databricks, catching drift before it reaches dashboards, regulatory reports or AI model pipelines.

Supported data sources

SQL Server, MySQL, Oracle, PostgreSQL, MongoDB, Snowflake and Databricks, plus CSV, fixed-width, X12 and SQLite files.

Explore Validatum
Learn More