Fundamentals Flashcards

What is a slowly changing dimension (SCD) Type 2 designed to do?

Answer: Preserve full history by adding new rows for changed attributes

SCD Type 2 keeps historical changes by inserting a new row each time a tracked attribute changes.

Which guarantee does 'exactly-once' processing provide in a streaming system?

Answer: Each record is processed once with no duplicates or loss

Exactly-once semantics ensure every record affects the result a single time, even across failures and retries.

What is the role of a message broker like Apache Kafka in a data architecture?

Answer: It decouples producers and consumers by buffering streams of events

Kafka acts as a durable, scalable buffer that decouples data producers from consumers via topic-based event streams.

Which normalization concept aims to reduce data redundancy in a relational design?

Answer: Normalization to higher normal forms

Normalization organizes tables to minimize redundancy and dependency by progressing through normal forms.

What does 'backfilling' a pipeline mean?

Answer: Processing past time periods that were missed or need reprocessing

Backfilling runs a pipeline over historical date ranges to populate or correct data that was missing or wrong.

In CAP theorem, what must a distributed system trade off when a network partition occurs?

Answer: Consistency and availability

CAP theorem states that during a network partition a system must choose between consistency and availability.

What is the primary benefit of using a columnar storage engine for OLAP workloads?

Answer: Efficient reads and compression of specific columns for aggregations

Columnar storage lets analytical queries read only needed columns and compress them well, speeding up aggregations.