Data Quality and Testing Flashcards
6 cards from real Data Engineering practice questions. Tap to flip, then mark Knew It or Still Learning โ missed cards come back until you master them.
Read the first 6 Data Quality and Testing flashcards as text
What is 'data lineage' and why is it important for data quality?
Answer: The end-to-end tracking of data from origin through all transformations to its final destination
Data lineage maps the journey of data from source to destination, enabling root cause analysis when data quality issues occur.
Which approach to data quality testing runs assertions directly in SQL against your data warehouse tables?
Answer: In-warehouse testing (e.g., dbt tests)
In-warehouse testing (used by dbt and similar tools) compiles assertions into SQL queries that run directly against the target tables.
What is 'referential integrity' in the context of data warehouse quality?
Answer: Ensuring all fact table foreign keys have matching dimension table records
Referential integrity ensures every foreign key in the fact table points to an existing record in the corresponding dimension table.
What is an 'anomaly detection' test in data quality pipelines?
Answer: A test that flags unexpected deviations from historical data patterns
Anomaly detection tests compare current data metrics against historical baselines to flag unusual volumes, distributions, or values.
In dbt, what is the difference between a 'test' and a 'snapshot'?
Answer: Tests validate data quality assertions; snapshots capture slowly changing dimension history
dbt tests assert data quality conditions (like uniqueness or not-null), while snapshots implement SCD Type 2 by capturing row changes over time.
What does 'data profiling' accomplish in a data engineering workflow?
Answer: Analyzes dataset characteristics like cardinality, null rates, and distributions to understand data quality
Data profiling examines datasets to produce statistical summaries (null counts, distinct values, min/max) that reveal data quality issues before transformation.