AML Reinforcement Learning & Decision Making 1 — Questions and Answers
Question 1: In reinforcement learning, what does the agent seek to maximize over time?
- Cumulative discounted reward (Correct answer)
- Prediction accuracy of the next state
- Number of actions taken per episode
- Shortest path to a terminal state
Correct answer: Cumulative discounted reward
An RL agent maximizes cumulative discounted reward over time, balancing immediate and future gains according to the discount factor.
Question 2: Which of the following is NOT a standard component of a Markov Decision Process (MDP)?
- State space
- Transition probability function
- Neural network architecture (Correct answer)
- Reward function
Correct answer: Neural network architecture
An MDP is formally defined by states, actions, transition probabilities, rewards, and a discount factor — neural network architecture is an implementation choice, not part of the MDP definition.
Question 3: What effect does setting the discount factor (γ) close to 0 have on a reinforcement learning agent?
- The agent learns faster due to stronger gradients
- The agent becomes myopic, prioritizing only immediate rewards (Correct answer)
- The agent explores more aggressively
- The agent treats all future rewards equally to immediate ones
Correct answer: The agent becomes myopic, prioritizing only immediate rewards
A discount factor near 0 causes the agent to heavily discount future rewards, making it focus almost entirely on immediate rewards (myopic behavior).
Question 4: What is the key distinction between model-based and model-free reinforcement learning?
- Model-based RL uses deep neural networks; model-free uses tabular methods only
- Model-based RL learns or uses an environment model for planning; model-free learns directly from interactions (Correct answer)
- Model-free RL requires more labeled data than model-based approaches
- Model-based RL only works in discrete action spaces
Correct answer: Model-based RL learns or uses an environment model for planning; model-free learns directly from interactions
Model-based RL builds or leverages an explicit model of environment dynamics for planning, while model-free RL learns policies or value functions directly from experience without modeling transitions.
Question 5: What is the exploration-exploitation dilemma in reinforcement learning?
- Choosing between GPU and CPU for training
- Balancing trying new actions to gather information against using known high-reward actions (Correct answer)
- Selecting between policy gradient and value-based methods
- Allocating compute between training and inference phases
Correct answer: Balancing trying new actions to gather information against using known high-reward actions
The exploration-exploitation dilemma involves balancing the need to explore unknown actions (to discover better rewards) against exploiting already-known high-reward actions.
Question 6: What does a stochastic policy output for a given state in reinforcement learning?
- The single action with the highest Q-value
- A probability distribution over all possible actions (Correct answer)
- A random action drawn uniformly from the action space
- The expected cumulative reward for that state
Correct answer: A probability distribution over all possible actions
A stochastic policy maps each state to a probability distribution over actions, enabling exploration by sampling from this distribution during training.
Question 7: What does the Bellman equation fundamentally express in reinforcement learning?
- The gradient of the policy loss with respect to network parameters
- The recursive relationship between a state's value and the values of its successor states (Correct answer)
- The backpropagation update rule for deep Q-networks
- The optimal epsilon decay schedule for exploration
Correct answer: The recursive relationship between a state's value and the values of its successor states
The Bellman equation expresses value recursively: the value of a state equals the immediate reward plus the discounted value of successor states, enabling dynamic programming solutions.
AML Reinforcement Learning & Decision Making 1
In reinforcement learning, what does the agent seek to maximize over time?