The Data Vault method to model a data warehouse was born out of necessity. Data warehousing projects often have to deal with excessively long implementation times. This implies that Business needs may change over the course of the project, jeopardizing meeting implementation deadlines and project costs..
To improve implementation times, method was introduced Data vault to model the data warehouse. The design principle involves separating the business keys, the context and relationships in different tables, as hub, satellite and links.
Hub concepts, satellite and link
A hub contains the actual business key (one or more fields that uniquely identify an entity for business, for instance, a customer number) and a surrogate key that is used to connect this table with other structures (equivalente a una Primary KeyThe primary key is a fundamental concept in databases, used to uniquely identify each record within a table. It consists of one or more attributes that cannot contain null values and must be unique. Its correct design is crucial to maintain data integrity, facilitating relationships between tables and optimizing queries. Without a primary key, ambiguities and errors could be generated in the...). What's more, también puede contener metadatos como marcas de tiempo o información sobre la Data SourceA "Data Source" refers to any place or medium where information can be obtained. These sources can be both primary and, such as surveys and experiments, as secondary, as databases, academic articles or statistical reports. The right choice of a data source is crucial to ensure the validity and reliability of information in research and analysis..... The links below connect the hubs with a simple many-to-many table consisting of the respective replacement keys..
Hereby, Hubs and links represent the most stable part of a model and are enriched by so-called satellites.. A satellite is connected to a hub with its replacement key and contains one or more descriptive attributes that are generally grouped by a source system, a business context or an exchange rate.. What's more, a satellite table can also comprise different types of metadata, such as valid date periods and origin information.
A basic Data Vault model could consist of a very simple order center and customer center, as well as related links and satellites.. In this model, the customer center could have two satellites: one with master data that possibly comes from the CRM system and another with an attribute called smartphone that can come from a web analytics system.
Benefits of a data warehouse in the scope of Data Vault
- Ease of expansion enables an agile project approach.
- The models created are highly scalable.
- Loading processes can be optimally parallelized because there are few synchronization points.
- Models are easy to audit
But along with the many benefits, Data Vault also presents a number of challenges.
Challenges of a data warehouse at the data warehouse level
- There is a large increase in the number of data objects. (boards, columns) as a result of separating the types of information and enriching them with meta information for loading.
- This results in increased modeling effort. comprising numerous unsophisticated mechanical tasks
How can you overcome these challenges using a standard data modeling tool? The highly schematic structure of the models provides the ideal prerequisites for generating models. This allows to automate important parts of the modeling process, allowing you to dramatically accelerate Data Vault projects.
Possibility of automating the Data Warehouse
What specific parts of the model can be automated? The standard architecture of a data warehouse includes the following layers:
- Origin system: Operating system, like ERP or CRM systems
- Staging area: This is where the data is delivered from the operating systems. The structure of the data model generally corresponds to the source system, with enhancements to document loading.
- Warehouse core: Here data from various systems is integrated. This layer is based on the Data Vault model and is subdivided into the Raw Vault and Business Vault areas. This involves implementing all business rules in Business Vault so that only very simple transformations are used in Raw Vault.
- Dating Markets: The structure of the data markets is based on the analysis requirements and is modeled as a star schema.
Both the staging area and the Raw Vault are suitable for automation, as clearly defined derivation rules can be set from the previous layer.
(function(d, s, id) {
var js, fjs = d.getElementsByTagName(s)[0];
if (d.getElementById(id)) return;
js = d.createElement(s); js.id = id;
js.src = “//connect.facebook.net/es_ES/all.js#xfbml=1&status=0”;
fjs.parentNode.insertBefore(js, fjs);
}(document, ‘script’, 'facebook-jssdk'));
