AI Ultimate AI Engineer 4 — Questions and Answers
Question 1: Which serving optimization packs multiple independent requests into a single GPU forward pass without waiting for a full batch to fill?
- Tensor parallelism
- Continuous batching (iteration-level scheduling) (Correct answer)
- Quantization-aware training
- Flash Attention
Correct answer: Continuous batching (iteration-level scheduling)
Continuous batching inserts new requests mid-generation at the token level, maximizing GPU utilization without fixed batch wait times.
Question 2: An AI engineer wants to reduce an LLM's memory footprint by 4x with minimal accuracy loss. Which technique is most appropriate?
- Doubling the number of attention heads
- INT4 weight quantization (Correct answer)
- Increasing the learning rate during fine-tuning
- Using a larger tokenizer vocabulary
Correct answer: INT4 weight quantization
INT4 quantization represents weights in 4 bits instead of 16, achieving approximately 4x memory reduction with small accuracy degradation.
Question 3: What is the key advantage of Flash Attention over standard attention implementation?
- It reduces the number of attention heads required
- It computes attention in tiles to avoid materializing the full N×N attention matrix in HBM (Correct answer)
- It replaces softmax with a linear approximation
- It eliminates positional encodings entirely
Correct answer: It computes attention in tiles to avoid materializing the full N×N attention matrix in HBM
Flash Attention tiles the computation to keep intermediates in SRAM rather than writing the full attention matrix to GPU high-bandwidth memory, dramatically reducing memory I/O.
Question 4: Which technique enables an AI system to use external tools like calculators or web search during inference?
- Pre-training on tool documentation only
- Function calling / tool use via structured output parsing (Correct answer)
- Increasing the model's context window to 1M tokens
- Training with contrastive loss on tool examples
Correct answer: Function calling / tool use via structured output parsing
Function calling allows the model to emit structured JSON specifying a tool and arguments, which the host application executes and returns results for.
Question 5: In MLOps, what does 'model drift' refer to?
- Weights gradually changing during serving due to online learning
- Degradation in model performance caused by changes in real-world data distribution (Correct answer)
- Exceeding GPU memory limits during training
- A bug where model versions are mislabeled in the registry
Correct answer: Degradation in model performance caused by changes in real-world data distribution
Model drift occurs when the statistical properties of production inputs diverge from the training distribution, causing accuracy to degrade over time.
Question 6: Which safety technique trains the model to refuse harmful requests by generating refusals as positive examples?
- Constitutional AI / RLAIF using AI-generated critiques (Correct answer)
- Prompt injection filtering at the API gateway
- Weight pruning of neurons associated with unsafe content
- Adversarial data augmentation during pretraining only
Correct answer: Constitutional AI / RLAIF using AI-generated critiques
Constitutional AI uses a set of principles and AI-generated critiques/revisions to self-improve responses toward safer behavior without human labelers for every example.
Question 7: A RAG pipeline returns irrelevant chunks. Which component should an engineer investigate first?
- The LLM's decoding temperature
- The embedding model and chunking strategy (Correct answer)
- The inference server's batch size setting
- The tokenizer's special token configuration
Correct answer: The embedding model and chunking strategy
Retrieval quality depends on how well the embedding model represents semantic meaning and whether chunk size/overlap preserves coherent context.
AI Ultimate AI Engineer 4
Which serving optimization packs multiple independent requests into a single GPU forward pass without waiting for a full batch to fill?