AI Azure AI Engineer 3 — Questions and Answers
Question 1: A model trained on historical loan data consistently under-predicts default risk for a specific demographic group. This is an example of:
- Overfitting
- Data drift
- Algorithmic bias (Correct answer)
- Underfitting
Correct answer: Algorithmic bias
Algorithmic bias occurs when a model produces systematically skewed outcomes for a protected group, often due to imbalanced or unrepresentative training data.
Question 2: Which Azure tool helps detect when a deployed model's input data distribution shifts significantly from its training data?
- Azure Monitor Alerts
- Azure Machine Learning Data Drift Monitor (Correct answer)
- Azure Policy
- Azure Advisor
Correct answer: Azure Machine Learning Data Drift Monitor
Azure ML Data Drift Monitor compares baseline training data statistics against live inference data and alerts when drift exceeds a configured threshold.
Question 3: In the Azure AI Foundry prompt flow, what is the primary role of a 'tool node'?
- Rendering the UI for user input
- Executing code or calling external APIs as a step in the flow (Correct answer)
- Storing conversation history
- Managing authentication tokens
Correct answer: Executing code or calling external APIs as a step in the flow
Tool nodes in prompt flow encapsulate executable logic—Python functions or API calls—that transform data between LLM steps.
Question 4: Which Azure service provides pre-built AI models for document understanding, including invoice and receipt extraction, without custom training?
- Azure Form Recognizer (Document Intelligence) (Correct answer)
- Azure Custom Vision
- Azure Machine Learning AutoML
- Azure Bot Service
Correct answer: Azure Form Recognizer (Document Intelligence)
Azure AI Document Intelligence includes pre-built models for common document types like invoices, receipts, and ID documents that work out of the box.
Question 5: When fine-tuning an Azure OpenAI model, what file format is required for the training dataset?
- CSV with headers
- JSONL with chat completion format (Correct answer)
- Parquet
- XML
Correct answer: JSONL with chat completion format
Azure OpenAI fine-tuning requires JSONL files where each line is a JSON object with a 'messages' array following the chat completion schema.
Question 6: An engineer wants to evaluate multiple LLM responses for groundedness against source documents at scale. Which Azure capability supports this?
- Azure OpenAI Playground
- Azure AI Evaluation SDK with GPT-based evaluators (Correct answer)
- Azure Logic Apps
- Azure Stream Analytics
Correct answer: Azure AI Evaluation SDK with GPT-based evaluators
The Azure AI Evaluation SDK provides built-in GPT-powered evaluators including groundedness, relevance, and coherence that can run at scale in pipelines.
Question 7: Which feature of Azure Machine Learning allows you to reuse and share preprocessing steps and feature engineering logic across multiple experiments?
- Datasets
- Pipelines with registered components (Correct answer)
- Environments
- Endpoints
Correct answer: Pipelines with registered components
Registered components in Azure ML pipelines encapsulate reusable steps that can be versioned and shared across teams and experiments.
AI Azure AI Engineer 3
A model trained on historical loan data consistently under-predicts default risk for a specific demographic group.
This is an example of: