Databricks Certified Data Engineer Associate Auto Loader and Data Ingestion 2

Question 1

Which Auto Loader option limits processing to a maximum number of files per trigger interval?

Accepted Answer

cloudFiles.maxFilesPerTrigger

Answer

cloudFiles.maxBytesPerTrigger

Answer

trigger.maxFiles

Answer

cloudFiles.batchSize

Question 2

Auto Loader's file notification mode on AWS relies on which combination of managed services?

Accepted Answer

S3 + SNS + SQS

Answer

S3 + CloudWatch + Lambda

Answer

S3 + Kinesis + DynamoDB

Answer

S3 + EventBridge + Step Functions

Question 3

When `cloudFiles.inferColumnTypes` is set to false in Auto Loader schema inference, how are columns typed?

Accepted Answer

All inferred columns default to StringType

Answer

Auto Loader fails to start without an explicit schema

Answer

Only primitive types are inferred; complex types are dropped

Answer

Auto Loader uses the last known schema from schemaLocation

Question 4

Which statement BEST compares COPY INTO versus Auto Loader for data ingestion?

Accepted Answer

Auto Loader is better suited for large-scale continuous streaming; COPY INTO is designed for idempotent batch loads

Answer

COPY INTO supports exactly-once semantics natively; Auto Loader does not

Answer

COPY INTO supports schema evolution; Auto Loader does not

Answer

Auto Loader requires a Delta table target; COPY INTO works with any file format

Question 5

Which of the following file formats is NOT natively supported as input by Auto Loader?

Accepted Answer

Microsoft Excel (.xlsx)

Answer

JSON

Answer

CSV

Answer

Parquet

Question 6

What is the purpose of the `rescuedDataColumn` option in Auto Loader?

Accepted Answer

It stores rows or fields that could not be parsed due to schema mismatches in a dedicated column

Answer

It defines a backup storage location for files that failed to load

Answer

It specifies a partition column used to segregate problematic data

Answer

It enables automatic recovery and reprocessing of corrupted source files

Question 7

How does Auto Loader prevent duplicate file processing when a streaming job restarts after a failure?

Accepted Answer

It uses a checkpoint directory to track previously processed files, ensuring each file is processed exactly once

Answer

It relies on Delta Lake's ACID properties to detect and remove duplicates on write

Answer

Deduplication must be handled entirely by application-level logic after ingestion

Answer

Auto Loader uses cloud storage object versioning to detect already-processed files