What is Data Observability?
Borrowing from software observability, data observability means having complete visibility into the health of your data pipelines and datasets — knowing when something breaks, what broke, and why.
The Five Pillars of Data Observability
- Freshness: Is your data up to date? Has the table been updated within its expected window?
- Volume: Are the expected number of rows arriving? A sudden drop could indicate an upstream failure.
- Distribution: Have the statistical properties of columns changed unexpectedly?
- Schema: Have column names, types, or counts changed without notice?
- Lineage: Which downstream dashboards and models are affected by this pipeline failure?