AI glossary · Basic · Data engineering
Data lineage
Data lineage is recording and showing where data came from, what processing it went through and where it flowed to. It lets you trace a single number back to its source and the path it took.
About this term
- Level
- Basic · Technical terms you often meet in AI news
- Category
- Data, ops & platforms
- Fields
- Data engineering
- Added
- 2026-10-09
How it works
Company data is usually collected from source databases, cleaned and transformed, loaded into a data warehouse, and then used for reports, dashboards or machine learning model training. Data lineage tools record, as connecting lines, 'which table produced which table' and 'which column was changed by which calculation' along the way. Recording methods include automatically analyzing SQL or pipeline code, collecting the logs left when jobs run, and having people write it down manually. Viewing at the table level is called table-level lineage, and viewing down to individual columns is called column-level lineage.
Why it matters
When a dashboard number suddenly looks wrong, you can trace the lineage backward to quickly find the step where the problem occurred. In the other direction, before changing a source table, you can check in advance which reports and models use it and prevent incidents; this is called impact analysis. It is also used for regulatory compliance that requires knowing how far personal data has spread. In the AI era it has become more important, as there are more occasions to explain what data a model was trained on.
Things to know
Lineage is not a picture you draw once and finish; it must be updated whenever the pipeline changes. When maintained by hand, it quickly drifts from reality, so it is better to use automatic collection wherever possible. Movements that happen outside the system, such as passing around Excel files or copying by hand, leave gaps that are not captured. It is usually used together with data catalog and data governance tools.
Example
Data catalog services and data transformation tools offer a screen where clicking a table shows its 'upstream (sources)' and 'downstream (where this table is used)' as a connected graph. For example, when an analyst gets a report that the values on a 'monthly revenue' dashboard are wrong, they follow this graph from the revenue aggregation table → the cleaned orders table → the source orders DB to find which transformation step went wrong.