Apache Spark GraphX and Graph Processing 2 — Questions and Answers
Question 1: What computation model does the Pregel API in GraphX implement?
- MapReduce-based batch aggregation
- Bulk-synchronous message-passing between vertices (Correct answer)
- Streaming micro-batch graph updates
- SQL-style declarative graph traversal
Correct answer: Bulk-synchronous message-passing between vertices
The Pregel API implements a bulk-synchronous parallel (BSP) message-passing model where vertices send and receive messages in iterative supersteps until convergence.
Question 2: In GraphX's `aggregateMessages`, what is the role of the `sendMsg` function?
- To send final computation results back to the driver
- To define what messages are sent along each edge to neighboring vertices (Correct answer)
- To broadcast vertex attributes to all partitions
- To serialize edge data for network transfer
Correct answer: To define what messages are sent along each edge to neighboring vertices
The `sendMsg` function is called for each edge triplet and specifies what messages, if any, to send to the source and/or destination vertices for aggregation.
Question 3: What does GraphX's `connectedComponents()` algorithm store as each vertex's attribute in the result graph?
- The lowest VertexId in its connected component (Correct answer)
- A list of all neighbor IDs in its component
- The number of vertices in its component
- A Boolean flag indicating if the vertex is a component root
Correct answer: The lowest VertexId in its connected component
connectedComponents() labels each vertex with the lowest VertexId found in its connected component, allowing grouping of all vertices in the same component.
Question 4: What does the `mapVertices` operation in GraphX do?
- Moves vertices to different partitions for load balancing
- Transforms vertex attributes using a user-defined function without changing graph structure (Correct answer)
- Remaps vertex IDs to new sequential integer values
- Creates a sorted copy of the graph ordered by vertex attribute
Correct answer: Transforms vertex attributes using a user-defined function without changing graph structure
mapVertices applies a function to each vertex's attribute and returns a new Graph with the transformed properties while preserving the original topology.
Question 5: What is the purpose of the `subgraph` operation in GraphX?
- To extract a subset of vertices and edges that satisfy a given predicate (Correct answer)
- To split a graph into two equal halves for parallel processing
- To create a coarser hierarchical summary of the graph
- To merge two separate graphs into one larger graph
Correct answer: To extract a subset of vertices and edges that satisfy a given predicate
The `subgraph` operation filters vertices and edges using user-supplied predicates, returning a new graph that contains only the elements satisfying both conditions.
Question 6: Which GraphX algorithm ranks vertices based on the number and quality of their incoming edges?
- Triangle Count
- Shortest Paths
- PageRank (Correct answer)
- Label Propagation
Correct answer: PageRank
PageRank in GraphX iteratively scores vertices: vertices receiving edges from many highly-ranked vertices accumulate higher rank scores.
Question 7: What does calling `Graph.reverse` on a GraphX graph produce?
- A graph with vertices listed in reverse order
- A new graph with all edge directions flipped while keeping vertex and edge attributes unchanged (Correct answer)
- A graph with edges sorted from destination to source alphabetically
- A backup replica of the graph on a secondary cluster
Correct answer: A new graph with all edge directions flipped while keeping vertex and edge attributes unchanged
Graph.reverse returns a new Graph with identical vertices and edges but with every edge's source and destination swapped, reversing the direction of all edges.
Apache Spark GraphX and Graph Processing 2
What computation model does the Pregel API in GraphX implement?