Data Science Deep Learning and Neural Networks 1 — Questions and Answers
Question 1: What is the primary role of an activation function in a neural network?
- To initialize weights
- To introduce non-linearity into the model (Correct answer)
- To normalize input data
- To reduce overfitting
Correct answer: To introduce non-linearity into the model
Activation functions introduce non-linearity, allowing neural networks to learn complex patterns beyond simple linear relationships.
Question 2: Which optimization algorithm adapts the learning rate for each parameter using estimates of first and second moments of the gradients?
- SGD
- RMSProp
- Adam (Correct answer)
- Adagrad
Correct answer: Adam
Adam (Adaptive Moment Estimation) combines momentum and RMSProp by tracking both the first and second moments of gradients.
Question 3: What does the term 'epoch' mean in the context of training a neural network?
- A single weight update step
- One complete pass through the entire training dataset (Correct answer)
- A batch of training samples
- The learning rate schedule
Correct answer: One complete pass through the entire training dataset
An epoch is one full pass through all training examples, after which weights have been updated based on every sample.
Question 4: Which technique randomly drops neurons during training to reduce overfitting in neural networks?
- Batch normalization
- Weight decay
- Dropout (Correct answer)
- Early stopping
Correct answer: Dropout
Dropout randomly sets a fraction of neuron outputs to zero during each training step, forcing the network to learn redundant representations.
Question 5: In a convolutional neural network (CNN), what does a pooling layer primarily do?
- Adds more learnable parameters
- Reduces spatial dimensions of feature maps (Correct answer)
- Applies the activation function
- Connects every neuron to every other neuron
Correct answer: Reduces spatial dimensions of feature maps
Pooling layers downsample feature maps by summarizing regions, reducing computation and providing translation invariance.
Question 6: What is the vanishing gradient problem in deep neural networks?
- Gradients become very large and destabilize training
- Gradients shrink exponentially as they propagate back through layers, slowing learning (Correct answer)
- The loss function reaches a local minimum too early
- Weights converge to zero due to regularization
Correct answer: Gradients shrink exponentially as they propagate back through layers, slowing learning
Vanishing gradients occur when repeated multiplication of small derivatives causes gradients to approach zero in early layers, preventing effective weight updates.
Data Science Deep Learning and Neural Networks 1
What is the primary role of an activation function in a neural network?