MS-DS Master of Data science Big Data 3 — Questions and Answers
Question 1: Which technique does Apache Spark use to optimize query execution plans in Spark SQL?
- MapReduce optimization
- Catalyst optimizer (Correct answer)
- Zookeeper coordination
- YARN scheduling
Correct answer: Catalyst optimizer
Spark SQL uses the Catalyst optimizer to apply rule-based and cost-based optimizations on logical and physical query plans.
Question 2: In stream processing, what does 'exactly-once' semantics guarantee?
- Each message is processed at least once
- Each message is processed at most once
- Each message produces one and only one output effect (Correct answer)
- Messages are delivered in order exactly once
Correct answer: Each message produces one and only one output effect
Exactly-once semantics ensures that despite failures and retries, each input message affects system state only once.
Question 3: Which data format is columnar and widely used in the Hadoop ecosystem for efficient analytical queries?
- Avro
- JSON
- Parquet (Correct answer)
- CSV
Correct answer: Parquet
Parquet is a columnar storage format that enables efficient column pruning and predicate pushdown for analytical queries.
Question 4: What is the primary role of Apache ZooKeeper in a distributed big data system?
- Data storage
- Distributed coordination and consensus (Correct answer)
- Query optimization
- Stream ingestion
Correct answer: Distributed coordination and consensus
ZooKeeper provides distributed coordination services such as leader election, configuration management, and distributed locks.
Question 5: Which windowing strategy in stream processing assigns each event to all windows that contain its timestamp?
- Tumbling window
- Session window
- Sliding window (Correct answer)
- Global window
Correct answer: Sliding window
Sliding windows overlap, so a single event can belong to multiple windows depending on the slide interval.
Question 6: In MapReduce, the combiner function is best described as a:
- Local reducer that runs on mapper output before the shuffle (Correct answer)
- Secondary sort mechanism
- Custom partitioner for keys
- Distributed cache manager
Correct answer: Local reducer that runs on mapper output before the shuffle
The combiner acts as a mini-reducer on the mapper side, aggregating local output to reduce data transferred during shuffle.
Question 7: Which consistency model does Apache Cassandra use by default for reads and writes?
- Strong consistency
- Linearizability
- Eventual consistency (Correct answer)
- Serializability
Correct answer: Eventual consistency
Cassandra defaults to eventual consistency, prioritizing availability and partition tolerance over immediate consistency.
MS-DS Master of Data science Big Data 3
Which technique does Apache Spark use to optimize query execution plans in Spark SQL?