To trust Big Data you need to understand your Data lineage. Sin Data Lineage, Big data becomes the last sentence of the broken phone game. The original data changes along the way to become something else entirely when it reaches the end. And very few understand how they came to be so different from the original version..
More than ninety percent of the world's data today has been created in recent years. This explosion of data is the result of the growing number of systems and automation at all levels and in institutions of all sizes.. Although this information facilitates access to data in the world of work, it has also contributed to creating a new set of problems..
What is data lineage?
Data lineage describe the origin, the movements, the characteristics and quality of the data. Arguably, Data Lineage has regularly outlined where each piece of data begins and how it transforms to achieve results in different business projects..
Data lineage can be compared to a combination table and map, allowing you to guide which SQL to use to choose, summarize or group the data. Even though this It is a very traditional approach that, nowadays, is not sufficient to explain the scope of Data lineage.
In reality, applying only the traditional approach to data lineage encounters crashes, especially with regard to data lines. master data, as information about people, processes and items that form the core of the business.
For a more realistic view and a more meaningful lineage, it is necessary to include additional aspects of the Data lineagelike who uses what data, What do they mean, when the data is accessed, why data is stored and how data items are related. Having this more holistic perspective helps mitigate the clutter in data projects., shortening the time period for development and testing.
Between the dimensions of Data lineage that should not be missing are:
- WHO
- That
- Where
- Why
- How
Why track the lineage of your data?
The Data lineage is linked to numerous business benefits, which include the following:
- More effective data governance. Data governance needs metadata management. This is necessary to ensure that Big Data meets business standards.. A data lineage solution ties metadata together and provides understanding and validation of optimal data use and information risks that need to be mitigated.
- Greater compliance capacity. Data Lineage provides evidence that reports adequately reflect data, essential to allow business users, clients or auditors trust the reported data as the organization responds quickly to emerging possibilities and faces regulatory challenges.
- A boost to data quality. Challenges to data quality include moving, The transformation, the interpretation and selection of data through people and processes. The pressure to reliably demonstrate the origin and transformation of data across the organization can only be managed through a solution of Data Lineage, providing end-to-end visibility.
