Python · easy

How do you handle missing values in a pandas DataFrame?

Asked in Data Analyst interviews, in the Python round.

Short answer

fillna (mean/median/mode/forward-fill), dropna, or model-based imputation depending on context.

Sample data

These tables are loaded into an in-browser engine on this page, so you can run your own answer against them. No account needed.

survey

How to answer it

Definition

There is no single right answer. The right handling depends on why the value is missing, and saying that first is most of what is being tested.

Why it is missing

• Missing completely at random. Dropping is safe, if you can afford the rows.
• Missing at random, explainable by other columns. Impute using those columns.
• Missing not at random, where the absence itself carries signal. Income missing because someone declined to answer is information. Add an indicator column rather than papering over it.

Syntax

df.isna().sum()                        # where and how much
df.dropna(subset=["revenue"])          # drop rows missing a specific column
df["x"] = df["x"].fillna(df["x"].median())
df["x"] = df["x"].ffill()              # ordered data: carry the last value
df["x_missing"] = df["x"].isna().astype(int)   # keep the fact of absence

Example

print(df.isna().mean().sort_values(ascending=False).head())

df["signup_source"] = df["signup_source"].fillna("unknown")
df["age_missing"] = df["age"].isna().astype(int)
df["age"] = df["age"].fillna(df["age"].median())

Output

age              0.184
signup_source    0.061
revenue          0.000
dtype: float64

Key points

• Look before you fill. df.isna().mean() tells you whether this is a rounding error or half the column.
• Median over mean for skewed numerics, because the mean is already distorted by the tail.
• Forward fill belongs to ordered data and nothing else. On an unsorted frame it invents values from unrelated rows.
• Imputing with the column mean shrinks the variance and understates uncertainty downstream.
• Fit the imputation on the training set only. Using the full dataset's median leaks the test set.

FAQ

• Is dropping ever fine? Yes, when the rows are few and plausibly random. Say what fraction you dropped.
• What about NaN in a categorical? An explicit "unknown" level usually beats dropping, and it keeps the row.
• Does the model care? Some tree libraries handle missing natively. Linear models do not, so this has to happen first.

The 30-second answer

The first thing I would say is that it depends on why the value is missing, so I would start with isna per column to see how much is gone and where.

If it is missing at random and rare I might drop those rows. For a numeric column I usually impute with the median rather than the mean, because the mean is already pulled around by the skew that made the column messy. For ordered data I would forward fill, but only after sorting.

The part people skip is that the absence is often signal itself, so I add a boolean column recording that the value was missing before I fill it. And I fit the imputation on the training set only, because using the whole dataset's median leaks the test set into training.

In the interview

• What they are checking: that you ask why before you fill, and that you know imputation must be fit on train only.
• Common mistake: fillna(0) on everything. A zero income and a missing income are different facts.

Practice this for real

• Check your resume against a Data Analyst job description, free, no account needed.
• Every data interview question, by role and topic
• Scored AI mock interviews, two free, no card