AI AI Engineer: Neural Networks and Deep Learning 2 — Questions and Answers
Question 1: What is a recurrent neural network (RNN) primarily designed to handle?
- Image classification
- Sequential and time-series data with temporal dependencies (Correct answer)
- Tabular data regression
- Clustering unlabeled data
Correct answer: Sequential and time-series data with temporal dependencies
RNNs maintain a hidden state across time steps, making them suited for sequential data like text, audio, and time series.
Question 2: What problem do LSTM (Long Short-Term Memory) networks solve compared to vanilla RNNs?
- Slow matrix multiplication
- Long-range dependency learning by mitigating vanishing gradients over long sequences (Correct answer)
- Overfitting on small datasets
- High memory consumption during inference
Correct answer: Long-range dependency learning by mitigating vanishing gradients over long sequences
LSTMs use gating mechanisms (input, forget, output gates) to selectively retain or discard information, enabling learning of long-range dependencies.
Question 3: In deep learning, what is transfer learning?
- Moving a model from one server to another
- Reusing a model pretrained on a large dataset as a starting point for a new task (Correct answer)
- Transferring data between training and validation sets
- Sharing weights between two different models
Correct answer: Reusing a model pretrained on a large dataset as a starting point for a new task
Transfer learning leverages knowledge from a pretrained model (e.g., ImageNet-trained ResNet) by fine-tuning it on a smaller task-specific dataset.
Question 4: What is the role of the 'softmax' activation function in a multi-class classification output layer?
- Introduces non-linearity in hidden layers
- Converts raw logits into a probability distribution summing to 1 (Correct answer)
- Normalizes input features
- Reduces dimensionality of the output
Correct answer: Converts raw logits into a probability distribution summing to 1
Softmax exponentiates each logit and divides by the sum of all exponentiated logits, producing class probabilities that sum to 1.
Question 5: Which deep learning architecture introduced skip (residual) connections to enable training of very deep networks?
- VGG
- AlexNet
- ResNet (Correct answer)
- GoogLeNet
Correct answer: ResNet
ResNet introduced residual connections that add layer inputs directly to outputs, allowing gradients to flow more easily and enabling hundreds of layers.
Question 6: What is the purpose of weight initialization in neural networks?
- To set hyperparameters before training
- To set initial parameter values that promote stable gradient flow at the start of training (Correct answer)
- To define the network architecture
- To normalize the training data
Correct answer: To set initial parameter values that promote stable gradient flow at the start of training
Good weight initialization (e.g., Xavier, He) prevents vanishing or exploding gradients from the first forward pass, enabling stable training.
AI AI Engineer: Neural Networks and Deep Learning 2
What is a recurrent neural network (RNN) primarily designed to handle?