DSE Big Data Technologies 4 — Questions and Answers
Question 1: Which consistency model does Apache Cassandra use by default to optimize for availability?
- Strong consistency
- Eventual consistency (Correct answer)
- Linearizability
- Strict serializability
Correct answer: Eventual consistency
Cassandra defaults to eventual consistency, trading strict consistency for high availability and partition tolerance in line with the CAP theorem.
Question 2: What does the term 'hot partition' mean in a distributed NoSQL system?
- A partition stored on a high-temperature server rack
- A partition receiving disproportionately high read or write traffic, creating a bottleneck (Correct answer)
- A partition that has been encrypted with AES-256
- A partition containing the most recently inserted records
Correct answer: A partition receiving disproportionately high read or write traffic, creating a bottleneck
A hot partition occurs when one partition key attracts far more traffic than others, overloading the node responsible for that partition.
Question 3: In Apache Spark, what is a 'shuffle' operation?
- Randomly sampling rows from an RDD
- Redistributing data across partitions, typically during wide transformations like groupBy or join (Correct answer)
- Compressing data before writing to disk
- Caching an RDD in memory for reuse
Correct answer: Redistributing data across partitions, typically during wide transformations like groupBy or join
A shuffle redistributes data across the network to group records with the same key onto the same partition, and is one of the most expensive Spark operations.
Question 4: What is Apache NiFi primarily designed for?
- Running distributed machine learning training jobs
- Automating and managing data flow between systems with a visual interface (Correct answer)
- Providing a SQL interface to Kafka topics
- Monitoring cluster resource utilization
Correct answer: Automating and managing data flow between systems with a visual interface
Apache NiFi provides a web-based graphical interface for designing, managing, and monitoring data flows between systems with built-in provenance tracking.
Question 5: Which of the following best describes a Delta Lake?
- A proprietary AWS data warehouse service
- An open-source storage layer that adds ACID transactions and versioning to data lakes (Correct answer)
- A streaming framework built on top of Apache Kafka
- A columnar file format similar to Parquet
Correct answer: An open-source storage layer that adds ACID transactions and versioning to data lakes
Delta Lake is an open-source storage layer that brings ACID transactions, schema enforcement, and time-travel versioning to data lakes built on cloud object storage.
Question 6: What is 'compaction' in Apache Cassandra?
- Compressing keyspace definitions to reduce metadata size
- Merging SSTables on disk to reclaim space and improve read performance (Correct answer)
- Rebalancing token ranges across nodes
- Removing expired TTL entries from the commit log
Correct answer: Merging SSTables on disk to reclaim space and improve read performance
Compaction merges multiple SSTables into fewer, larger ones, removing deleted or overwritten data (tombstones) and improving read efficiency.
Question 7: In the context of stream processing, what is 'watermarking' used for?
- Adding visible branding to data visualizations
- Handling late-arriving events by defining how long the system waits before closing a time window (Correct answer)
- Encrypting sensitive fields in a data stream
- Throttling the rate of messages produced to Kafka
Correct answer: Handling late-arriving events by defining how long the system waits before closing a time window
Watermarks define the maximum lateness a stream processor will tolerate, after which a time window is finalized and late data is either dropped or handled separately.
DSE Big Data Technologies 4
Which consistency model does Apache Cassandra use by default to optimize for availability?