Data contracts: making source changes safer for the business

A source-system update can pass its own tests and still disrupt reporting, planning, or an AI workflow downstream. A field keeps the same name but changes meaning. An identifier starts arriving with a new prefix. A delivery moves from early morning to midday. The pipeline may remain green while a business decision becomes less reliable.

A data contract makes these expectations explicit between the people producing data and the people relying on it. It describes what is delivered, what it means, when it is usable, who owns it, and how changes will be handled. The purpose is not to freeze systems. It is to let them evolve without making every consumer discover the consequences independently.

For executives, this is a continuity and accountability issue. For engineers, it is a testable interface and change-management problem. Both perspectives are necessary. A contract that only lists column types misses business meaning; a document that nobody validates misses operational protection.

Start with a decision that cannot tolerate surprises

Choose a dataset supporting an important recurring decision: a replenishment review, an account-margin report, or a customer-service workflow. Identify the producer and the consumers. Ask what a late, incomplete, or differently interpreted delivery would change for those users.

An inventory feed used for weekly planning may tolerate a delayed enrichment field but not a missing location balance. A finance-close dataset may require reproducibility after adjustments. A near-real-time operational view may value freshness more than finality. The contract should reflect that purpose rather than promise one service level for every use.

Keep the first scope manageable. One source, one important dataset, and a few known consumers are enough to establish the pattern. The business owner should approve the meaning and tolerance, while the technical owners agree on how to detect and communicate a breach.

Describe more than the schema

A useful contract includes a small set of related expectations:

• Identity and grain. What one row represents and which keys identify it.
• Meaning. Definitions, units, currency, time zone, and important exclusions.
• Structure. Required fields, types, allowed values, and nullability.
• Delivery. Expected schedule, covered period, and freshness requirement.
• Quality. Checks for duplicates, completeness, reconciliation, and business validity.
• Access and retention. Authorized uses, sensitive fields, and lifecycle responsibilities.
• Ownership and change. Contacts, versioning, notification, and escalation.

Some expectations can be checked automatically. Others need review. A timestamp can be present and correctly typed while referring to the wrong business event. Technical validation and business acceptance should reinforce one another rather than compete for ownership.

Separate a model contract from the broader agreement

Tools can enforce part of this approach. For example, dbt model contracts can validate declared column names and data types for supported models. Constraints and enforcement vary by platform and materialization. The dbt documentation distinguishes these structural guarantees from data tests that check other properties of the built dataset.

That is useful, but it is not the whole producer-consumer agreement. A passing model contract does not prove that revenue has the agreed exclusions, all stores arrived before the planning meeting, or a source owner gave adequate notice. Select tools for specific checks and keep the remaining expectations visible in the operating process. Avoid labeling a dataset fully governed simply because one schema test passes.

Work through a business change

Consider an illustrative inventory source that reports available units by product and location. Today, "available" excludes damaged stock but includes stock reserved for open orders. The source team plans to exclude reservations as well. The column name and numeric type will not change.

A structural test will not detect the new meaning. Without coordination, planners may see a sudden shortage and place unnecessary orders. The contract should identify the definition owner and require review of this semantic change. The producer and consumers can then decide whether to introduce a new field, publish a new version, or coordinate a clearly dated cutover.

During the transition, compare representative locations and products, including those with large reservations. Explain the expected differences in the planning view. Once consumers have accepted the change, retire the older definition through an agreed process. Keeping both indefinitely without clear labels creates a different kind of ambiguity.

Distinguish compatible and breaking changes

Adding an optional field can be compatible when consumers select explicit columns and ignore unknown fields. It may still break a consumer that assumes a fixed column order. Removing a field, changing a key, or altering units is more likely to require migration. Compatibility depends on actual consumers, not only on the producer's view of the change.

Maintain a consumer inventory for important datasets. It should include scheduled reports, extracts, applications, and automated workflows, not just warehouse dependencies visible to one tool. A spreadsheet distributed outside the platform can remain a significant consumer even when lineage software cannot see it.

For a breaking change, publish the intended behavior, affected consumers, migration action, and retirement date. Let consumers test a representative sample or parallel version. Record acceptance and any exceptions. A version number helps only when people know what changed and what they need to do.

Decide how failures affect publication

Not every failed check should stop every output. Classify failures by decision impact. A missing optional description might create a warning. Duplicate transaction keys or an incomplete inventory partition may justify holding the affected report until reviewed. Make that distinction before an incident creates pressure to publish something quickly.

Quarantine questionable data when appropriate, preserve evidence, and show a visible freshness or coverage status. Do not silently substitute zero for unavailable data. If the team uses the last accepted version, label its timestamp and explain which decisions it can still support.

The recovery procedure should include reconciliation, controlled reprocessing, and communication to consumers who already used an affected output. A contract breach is not resolved merely because the next run succeeds. Someone must decide whether previous reports or recommendations need correction.

Review contracts through an operating view

A practical review grid can list dataset, owner, consumer, latest accepted version, freshness status, open breaches, and upcoming changes. Keep the business exposure beside the technical status. A delayed feed serving a low-use exploratory report should not obscure a problem affecting a purchasing decision.

Track a few measures over time: unannounced breaking changes, time to acknowledge a breach, time to restore accepted output, and repeated incidents with the same cause. These are improvement measures, not a reason to punish teams for reporting problems. If alerts repeatedly produce no useful action, revise the checks or their thresholds.

Review the contract when the business use changes. A dataset originally built for monthly reporting may need different controls when an agent begins querying it throughout the day. Reuse should trigger a fit check, not an assumption that the original guarantees cover every new workflow.

Begin with a shared acceptance conversation

The first deliverable can be a short agreement and a small set of executable checks. Ask the producer to explain what can change, the consumer to explain what matters, and the business owner to explain what happens when the data is not ready. That conversation often surfaces undocumented assumptions before a new tool is needed.

Data contracts complement reliable pipeline operations. They do not replace monitoring, source reconciliation, or good modeling. Their value is making responsibility and change visible enough that those practices work together.

VeriVal helps teams define source expectations, build validation into ingestion and analytical models, and establish practical ownership for business-critical data. If source changes repeatedly interrupt reporting or planning, start a conversation.

References

• dbt: model contracts
• dbt: data tests