MS-DS Master of Data science FREE MS-DS Master of Data science Big Data Technologies Questions and Answers 2 — Questions and Answers
Question 1: Which component of the Hadoop ecosystem is specifically designed for real-time stream processing of large-scale data?
- Apache Storm (Correct answer)
- Apache Hive
- Apache Sqoop
- Apache Oozie
Correct answer: Apache Storm
Apache Storm is a distributed real-time computation system designed for processing unbounded streams of data at scale.
Question 2: In Apache Spark, what is the primary abstraction for a distributed collection of items that can be processed in parallel?
- RDD (Resilient Distributed Dataset) (Correct answer)
- DataFrame only
- SparkSession
- Driver Program
Correct answer: RDD (Resilient Distributed Dataset)
RDD is Spark's fundamental data structure representing an immutable, partitioned collection of elements that can be operated on in parallel.
Question 3: What is the default replication factor for data blocks in a standard HDFS configuration?
- 3 (Correct answer)
- 2
- 1
- 5
Correct answer: 3
HDFS defaults to a replication factor of 3, storing three copies of each data block across different nodes for fault tolerance.
Question 4: Which big data storage format uses columnar storage and is optimized for analytical queries in distributed systems?
- Apache Parquet (Correct answer)
- CSV
- JSON Lines
- Apache Avro
Correct answer: Apache Parquet
Apache Parquet is a columnar storage format that enables efficient compression and encoding schemes, making it ideal for analytical workloads.
Question 5: In a MapReduce job, what happens during the shuffle and sort phase between the Map and Reduce stages?
- Intermediate key-value pairs are grouped by key and transferred to reducer nodes (Correct answer)
- Data is written back to HDFS for checkpointing
- The input splits are recalculated based on data skew
- Map output is compressed and archived before reduction
Correct answer: Intermediate key-value pairs are grouped by key and transferred to reducer nodes
The shuffle and sort phase transfers mapper output across the network, grouping all values associated with the same key together before sending them to the appropriate reducer.
Question 6: Which CAP theorem property does Apache Cassandra primarily sacrifice to maintain high availability and partition tolerance?
- Strong consistency (Correct answer)
- Availability
- Partition tolerance
- Durability
Correct answer: Strong consistency
Cassandra is an AP system that prioritizes availability and partition tolerance, offering eventual consistency rather than strong consistency by default.
MS-DS Master of Data science FREE MS-DS Master of Data science Big Data Technologies Questions and Answers 2
Which component of the Hadoop ecosystem is specifically designed for real-time stream processing of large-scale data?