MS-DS Master of Data science Master of Data science Data Wrangling and Preprocessing 1 — Questions and Answers
Question 1: Which pandas function is used to detect missing values in a DataFrame, returning a boolean mask?
- df.dropna()
- df.isnull() (Correct answer)
- df.fillna()
- df.notna()
Correct answer: df.isnull()
df.isnull() returns a DataFrame of the same shape with True where values are NaN and False elsewhere, making it the standard method for detecting missing data.
Question 2: What is the main difference between StandardScaler and MinMaxScaler in scikit-learn?
- StandardScaler clips outliers while MinMaxScaler does not
- StandardScaler transforms features to zero mean and unit variance; MinMaxScaler scales features to a fixed range like [0,1] (Correct answer)
- MinMaxScaler is only for categorical data; StandardScaler is for numerical data
- StandardScaler requires normally distributed data; MinMaxScaler works only on integers
Correct answer: StandardScaler transforms features to zero mean and unit variance; MinMaxScaler scales features to a fixed range like [0,1]
StandardScaler centers data by subtracting the mean and dividing by standard deviation, producing zero mean and unit variance. MinMaxScaler maps values to a bounded range (default [0,1]) by subtracting the minimum and dividing by the range.
Question 3: In pandas, what is the effect of setting the parameter how='outer' in pd.merge()?
- It returns only rows with matching keys in both DataFrames
- It returns all rows from the left DataFrame only
- It returns all rows from both DataFrames, filling NaN where there is no match (Correct answer)
- It raises an error if keys do not match perfectly
Correct answer: It returns all rows from both DataFrames, filling NaN where there is no match
An outer join returns the union of keys from both DataFrames. Where a key exists in one DataFrame but not the other, the missing columns are filled with NaN.
Question 4: Which technique is best suited for imputing missing values in a numerical column when the data is believed to be missing at random and the column has a moderate skew?
- Mean imputation
- Median imputation (Correct answer)
- Forward fill
- Dropping all rows with missing values
Correct answer: Median imputation
Median imputation is preferred over mean imputation when data is skewed because the median is robust to outliers and better represents the central tendency of a skewed distribution.
Question 5: What does the pandas groupby() method return before an aggregation function is applied?
- A new DataFrame with sorted rows
- A DataFrameGroupBy object (Correct answer)
- A pivot table
- A Series of group labels
Correct answer: A DataFrameGroupBy object
df.groupby() returns a DataFrameGroupBy object, which is a lazy grouping structure. The actual computation only occurs when an aggregation function such as .sum(), .mean(), or .agg() is called on it.
Question 6: When applying label encoding to an ordinal categorical feature with categories ['Low', 'Medium', 'High'], which concern is most important to address?
- Label encoding always introduces multicollinearity
- The assigned integer values must preserve the natural order of the categories (Correct answer)
- Label encoding requires the feature to be normally distributed first
- Label encoding cannot be applied to features with more than two categories
Correct answer: The assigned integer values must preserve the natural order of the categories
For ordinal features, the integer labels assigned by encoding must reflect the correct ordering (e.g., Low=0, Medium=1, High=2). Using arbitrary or reversed numeric assignments would mislead models that interpret numeric magnitude as meaningful.
MS-DS Master of Data science Master of Data science Data Wrangling and Preprocessing 1
Which pandas function is used to detect missing values in a DataFrame, returning a boolean mask?