Apache Spark Spark Core and RDDs 1 — Questions and Answers
Question 1: What does RDD stand for in Apache Spark?
- Resilient Distributed Dataset (Correct answer)
- Redundant Data Distribution
- Remote Data Driver
- Reactive Distributed Database
Correct answer: Resilient Distributed Dataset
RDD stands for Resilient Distributed Dataset, which is the fundamental data structure in Apache Spark.
Question 2: Which of the following is a transformation in Spark RDDs?
- collect()
- count()
- map() (Correct answer)
- save()
Correct answer: map()
map() is a transformation that returns a new RDD, while collect(), count(), and save() are actions.
Question 3: What is the default number of partitions when creating an RDD from a collection using sc.parallelize()?
- 1
- 2
- Depends on the SparkContext default parallelism (Correct answer)
- 10
Correct answer: Depends on the SparkContext default parallelism
By default, sc.parallelize() uses the SparkContext's default parallelism, which is typically the number of cores.
Question 4: Which RDD operation returns all elements of the RDD to the driver program?
- take()
- first()
- collect() (Correct answer)
- count()
Correct answer: collect()
collect() returns all elements of the RDD to the driver program as an array.
Question 5: What does the flatMap() transformation do in Spark?
- Maps each element to exactly one output element
- Maps each element to zero or more output elements and flattens the result (Correct answer)
- Flattens nested RDDs into a single RDD
- Filters elements based on a condition
Correct answer: Maps each element to zero or more output elements and flattens the result
flatMap() applies a function to each element that returns an iterator and flattens all iterators into a single RDD.
Question 6: Which storage level in Spark caches RDD data in memory as deserialized Java objects?
- MEMORY_ONLY_SER
- DISK_ONLY
- MEMORY_ONLY (Correct answer)
- MEMORY_AND_DISK_SER
Correct answer: MEMORY_ONLY
MEMORY_ONLY stores RDD data in memory as deserialized Java objects, offering the fastest access but highest memory usage.
Apache Spark Spark Core and RDDs 1
What does RDD stand for in Apache Spark?