Data Vault 2.0

Agile, auditable enterprise data warehouse methodology

What is Data Vault?

Data Vault 2.0 is a hybrid approach to modeling enterprise data warehouses that combines the best aspects of third normal form (3NF) and dimensional modeling. Developed by Dan Linstedt in the 1990s and refined into version 2.0, it emphasizes scalability, flexibility, and auditability while supporting agile development practices.

Data Vault is designed specifically for enterprise data warehouses that need to integrate data from multiple sources while maintaining complete historical tracking and adapting quickly to changing business requirements. It separates business keys, descriptive data, and relationships into distinct table types.

Core Philosophy:

Data Vault prioritizes adaptability and auditability over query performance. It is optimized for data warehouse loading and change management, with the understanding that query performance can be addressed through separate presentation layers or information marts.

Three Core Entity Types

Hubs

Business Keys

Hub_Customer
customer_hash_key (PK)
customer_id (Business Key)
load_date
record_source
  • •Contain unique business keys
  • •No descriptive attributes
  • •Never deleted or updated
  • •Track data lineage
Links

Relationships

Link_Order
order_hash_key (PK)
customer_hash_key (FK)
product_hash_key (FK)
load_date
record_source
  • •Connect hubs together
  • •Many-to-many relationships
  • •No descriptive data
  • •Transaction associations
Satellites

Descriptive Attributes

Sat_Customer
customer_hash_key (PK, FK)
load_date (PK)
customer_name
email
phone
address
end_date
record_source
  • •Store descriptive data
  • •Track history over time
  • •One per data source
  • •Multiple per hub/link

Data Vault Model Example

E-Commerce Data Vault

Hub_Customer
customer_hk (PK)
customer_id (BK)
load_dts
rec_source
Hub_Product
product_hk (PK)
product_id (BK)
load_dts
rec_source
Hub_Order
order_hk (PK)
order_id (BK)
load_dts
rec_source
Link_Order_Customer
order_customer_hk (PK)
order_hk (FK)
customer_hk (FK)
load_dts
rec_source
Link_Order_Product
order_product_hk (PK)
order_hk (FK)
product_hk (FK)
load_dts
rec_source
Sat_Customer_CRM
customer_hk (PK, FK)
load_dts (PK)
end_dts
customer_name
email
phone
rec_source
Sat_Product_Details
product_hk (PK, FK)
load_dts (PK)
end_dts
product_name
description
price
rec_source
Sat_Order_Details
order_hk (PK, FK)
load_dts (PK)
end_dts
order_date
status
total_amount
rec_source

Data Vault 2.0 Principles

Insert-Only Architecture

Data Vault follows an append-only pattern where records are never deleted or updated. All changes are tracked as new records with timestamps, providing complete audit trails and historical context.

Benefits:

  • • Complete audit trail of all changes
  • • Point-in-time reconstruction
  • • Simplified ETL processes
  • • Parallel loading capability
Hash Keys for Performance

Data Vault 2.0 uses hash keys (MD5, SHA-1) generated from business keys for primary keys. This provides consistent key sizes, eliminates key collisions, and enables parallel processing.

customer_hash_key =
MD5(customer_id +
record_source)
Source-Centric Approach

Every record includes metadata about its source system. Multiple satellites can exist for the same hub, each containing data from different source systems without integration conflicts.

  • •Sat_Customer_CRM (from Salesforce)
  • •Sat_Customer_ERP (from SAP)
  • •Sat_Customer_Web (from website)
Agile & Scalable

Data Vault supports agile development by allowing teams to add new data sources, attributes, or relationships without impacting existing structures. New hubs, links, or satellites can be added independently.

Parallel Development:

Multiple teams can work on different parts of the model simultaneously without conflicts.

Advantages & Challenges

Advantages

✓Complete Auditability: Every change is tracked with timestamps and source information
✓Flexible & Agile: Easy to add new sources and attributes without restructuring
✓Parallel Loading: Multiple ETL processes can load data simultaneously
✓Source Traceability: Know exactly where each piece of data originated
✓Scalable: Handles massive data volumes and complex integration scenarios

Challenges

✗Complex for End Users: Not designed for direct querying by business users
✗Query Performance: Requires information marts or views for optimal query speed
✗Steeper Learning Curve: Teams need training on Data Vault methodology
✗Storage Overhead: Insert-only approach requires more storage space
✗Tooling Gap: Fewer specialized tools compared to dimensional modeling

When to Use Data Vault

Ideal Scenarios:
  • 1.Enterprise Data Warehouses: Large organizations with multiple source systems that need comprehensive integration
  • 2.Regulatory Compliance: Industries requiring complete audit trails (finance, healthcare, government)
  • 3.Agile Environments: Organizations that need to rapidly integrate new data sources and adapt to changing requirements
  • 4.Historical Analysis: Need for point-in-time reporting and historical reconstruction
  • 5.Master Data Management: Creating a single source of truth from disparate systems

Data Vault Architecture Layers

1. Raw Vault (Staging)

Initial landing zone for source data. Data loaded as-is without transformation, maintaining complete historical records.

2. Business Vault

Adds calculated fields, reference data, and business rules. Bridges the gap between raw data and business needs.

3. Information Marts

Dimensional models or aggregated views for end-user consumption. Optimized for query performance and business user understanding.