NLP Machine Translation 3 — Questions and Answers
Question 1: Which approach to MT does NOT require any parallel bilingual data during training?
- Supervised NMT
- Unsupervised MT using monolingual corpora only (Correct answer)
- Transfer learning from multilingual models
- Phrase-based SMT with phrase tables
Correct answer: Unsupervised MT using monolingual corpora only
Unsupervised MT methods (e.g., using denoising autoencoders and back-translation on monolingual data) require no parallel sentences.
Question 2: What is the main advantage of using subword tokenization (e.g., BPE) in NMT?
- Eliminates the need for an encoder
- Reduces vocabulary size while handling rare and OOV words (Correct answer)
- Speeds up beam search decoding
- Improves alignment between source and target
Correct answer: Reduces vocabulary size while handling rare and OOV words
Byte-pair encoding splits rare words into frequent subword units, giving the model coverage of open-vocabulary words without an extremely large vocabulary.
Question 3: In multilingual NMT, what is the 'language token' prepended to the source sentence used for?
- Indicating the domain of the text
- Signaling the desired target language to a single shared model (Correct answer)
- Marking sentence boundaries for the tokenizer
- Weighting the loss function per language
Correct answer: Signaling the desired target language to a single shared model
A target-language tag (e.g., <2fr>) prepended to the input tells a universal NMT model which language to generate.
Question 4: Which IBM Model introduced the concept of word fertility in MT alignment?
- IBM Model 1
- IBM Model 2
- IBM Model 3 (Correct answer)
- IBM Model 4
Correct answer: IBM Model 3
IBM Model 3 introduced fertility, the number of target words that a single source word generates.
Question 5: What does 'zero-shot translation' mean in a multilingual NMT system?
- Translating with zero training examples by few-shot prompting
- Translating between a language pair never seen together in training (Correct answer)
- Running inference without a decoder
- Achieving BLEU=0 on a test set
Correct answer: Translating between a language pair never seen together in training
Zero-shot translation is the ability to translate between two languages that were never paired together in training data.
Question 6: Which technique allows an NMT model to incorporate external knowledge such as a domain-specific glossary at inference time?
- Hard lexical constraints via constrained beam search (Correct answer)
- Fine-tuning on domain data
- Vocabulary reduction
- Increasing beam width
Correct answer: Hard lexical constraints via constrained beam search
Constrained decoding forces the beam search to include specified terms, implementing hard lexical constraints without retraining.
Question 7: In domain adaptation for MT, what is 'catastrophic forgetting'?
- The model forgets padding tokens during fine-tuning
- Fine-tuning on in-domain data degrades performance on the original general domain (Correct answer)
- Attention weights become zero for infrequent tokens
- The optimizer forgets gradient history between epochs
Correct answer: Fine-tuning on in-domain data degrades performance on the original general domain
Catastrophic forgetting occurs when fine-tuning on a new domain overwrites general knowledge learned during pretraining, hurting out-of-domain performance.
NLP Machine Translation 3
Which approach to MT does NOT require any parallel bilingual data during training?