MS-DS Master of Data science Big Data 2 — Questions and Answers
Question 1: In the Lambda architecture, which layer handles real-time, low-latency data processing?
- Batch layer
- Speed layer (Correct answer)
- Serving layer
- Ingestion layer
Correct answer: Speed layer
The speed layer processes incoming data in real time to provide low-latency views that complement the batch layer.
Question 2: Which compression codec is best suited for Hadoop workloads that require splittable compressed files?
- Gzip
- Bzip2 (Correct answer)
- Snappy
- LZ4
Correct answer: Bzip2
Bzip2 produces splittable compressed files, allowing Hadoop to divide them across multiple mappers.
Question 3: In Apache Kafka, what guarantees message ordering within a topic?
- Replication factor
- Consumer group ID
- Partition assignment (Correct answer)
- Broker ID
Correct answer: Partition assignment
Kafka guarantees ordering only within a single partition, so messages with the same key are routed to the same partition.
Question 4: Which HBase component stores the write-ahead log (WAL) to ensure durability?
- MemStore
- HFile
- RegionServer
- HLog (Correct answer)
Correct answer: HLog
The HLog (WAL) records all mutations before they are applied to the MemStore, enabling recovery after a crash.
Question 5: The CAP theorem states that a distributed system can simultaneously guarantee at most how many of its three properties?
- 1
- 2 (Correct answer)
- 3
- All 3 with trade-offs
Correct answer: 2
CAP theorem proves that during a network partition, a system must sacrifice either consistency or availability, so only 2 of 3 properties can be fully guaranteed.
Question 6: Which Apache Spark RDD operation is a wide transformation that triggers a shuffle across the cluster?
- map()
- filter()
- groupByKey() (Correct answer)
- flatMap()
Correct answer: groupByKey()
groupByKey() requires data with the same key to be co-located, forcing a shuffle across partitions.
Question 7: In HDFS, what is the default block replication factor?
- 1
- 2
- 3 (Correct answer)
- 5
Correct answer: 3
HDFS replicates each block 3 times by default to tolerate node failures while maintaining data availability.
MS-DS Master of Data science Big Data 2
In the Lambda architecture, which layer handles real-time, low-latency data processing?