Data Engineering Real-Time Streaming Architectures 2 — Questions and Answers
Question 1: In Apache Kafka, what guarantees message ordering?
- Ordering within a single partition (Correct answer)
- Ordering across the entire topic
- Ordering by producer timestamp globally
- Ordering by consumer group
Correct answer: Ordering within a single partition
Kafka only guarantees order within a partition, not across an entire topic.
Question 2: What is the primary purpose of a watermark in stream processing?
- To track progress of event time and handle late data (Correct answer)
- To encrypt messages in transit
- To compress the event payload
- To assign partitions to consumers
Correct answer: To track progress of event time and handle late data
Watermarks signal how far event time has advanced so windows can close and handle lateness.
Question 3: Which delivery semantic ensures a message is processed once and only once, even after failures?
- Exactly-once (Correct answer)
- At-least-once
- At-most-once
- Best-effort
Correct answer: Exactly-once
Exactly-once semantics guarantee each record affects state precisely one time despite failures.
Question 4: A tumbling window in stream processing is characterized by:
- Fixed-size, non-overlapping intervals (Correct answer)
- Fixed-size, overlapping intervals
- Variable-size gaps based on activity
- A single window covering all time
Correct answer: Fixed-size, non-overlapping intervals
Tumbling windows partition time into fixed, contiguous, non-overlapping buckets.
Question 5: What does backpressure refer to in a streaming pipeline?
- A downstream consumer signaling it cannot keep up with input rate (Correct answer)
- A producer flooding extra replicas
- Compression applied before network transit
- Rebalancing partitions across brokers
Correct answer: A downstream consumer signaling it cannot keep up with input rate
Backpressure is the mechanism by which slow consumers throttle faster producers.
Question 6: In Kafka, what role does the consumer group play?
- It distributes partitions among members for parallel consumption (Correct answer)
- It stores the topic's replication factor
- It controls broker leader election
- It defines the message serialization format
Correct answer: It distributes partitions among members for parallel consumption
Partitions are divided among members of a consumer group so each is read by one member.
Question 7: Which architecture uses both a batch layer and a speed layer to serve queries?
- Lambda architecture (Correct answer)
- Kappa architecture
- Star schema
- Medallion architecture
Correct answer: Lambda architecture
Lambda architecture combines a batch layer for accuracy with a speed layer for low latency.
Data Engineering Real-Time Streaming Architectures 2
In Apache Kafka, what guarantees message ordering?