Natural Language Processing Flashcards
7 cards from real MS-DS Master of Data science practice questions. Tap to flip, then mark Knew It or Still Learning — missed cards come back until you master them.
Read the first 7 Natural Language Processing flashcards as text
Which attention mechanism allows a model to attend to all positions in a sequence simultaneously, rather than sequentially?
Answer: Self-attention (scaled dot-product)
Self-attention computes attention weights between all token pairs in parallel using query, key, and value matrices scaled by the square root of the dimension.
In the context of language models, what does 'perplexity' measure?
Answer: How well a probability model predicts a sample
Perplexity is the exponentiated average negative log-likelihood of a test set; lower perplexity indicates a better-fitting language model.
What is the primary purpose of the positional encoding added to token embeddings in a Transformer model?
Answer: To inject information about token order since self-attention is permutation-invariant
Because self-attention has no inherent notion of sequence order, sinusoidal or learned positional encodings are added to embeddings to convey position.
Which NLP task involves assigning a semantic label such as ARG0 or ARG1 to constituents in a sentence relative to a predicate?
Answer: Semantic role labeling
Semantic role labeling (SRL) identifies who did what to whom by labeling predicate-argument structures using frameworks like PropBank.
In byte-pair encoding (BPE) tokenization, what determines when to stop merging character pairs?
Answer: When a predefined vocabulary size limit is reached
BPE iteratively merges the most frequent pair of adjacent symbols until the target vocabulary size is achieved.
What distinguishes extractive summarization from abstractive summarization?
Answer: Extractive copies spans from the source; abstractive generates novel text
Extractive summarization selects and concatenates existing sentences from the source document, while abstractive summarization generates new text that may paraphrase or compress content.
Which evaluation metric for machine translation computes n-gram precision with a brevity penalty?
Answer: BLEU
BLEU (Bilingual Evaluation Understudy) measures modified n-gram precision against reference translations and penalizes outputs shorter than the reference.