What is data leakage in ML experiments and how can it affect evaluation?

• A. The model's outputs are post-processed with external validation.
• B. Data leakage refers to overfitting due to too many parameters.
• C. Training data inadvertently included in test data leads to inflated performance.
• D. Data leakage occurs when the dataset has missing values.

Answer: (C)

Data leakage happens when information from the evaluation data ends up influencing the training process, so the model learns from data that it wouldn’t have access to in real use. This makes the model perform unusually well on the test set because it has effectively seen or inferred answers during training, which never happens with truly unseen data. As a result, the evaluation metrics become overly optimistic and don’t reflect how the model will generalize to new, real-world data.

A common way this leakage shows up is when training data inadvertently includes data from the test set. For example, if you create features or normalize data using the entire dataset before splitting, the model can pick up patterns that tie directly to the test examples. Similarly, using test data to select features, tune hyperparameters, or reveal future information (like a future value or label) during training are leakage scenarios. When you later evaluate on fresh data, the model’s performance drops, revealing that the earlier high scores were inflated by the leakage.

To prevent leakage, keep a strict separation between training and evaluation data from the start and apply all preprocessing within each split (or fit preprocessors on the training data only and apply them to the test data). Be careful with feature engineering—don’t base features on the test set, and avoid using future or target information during training. This ensures the evaluation reflects genuine generalization to unseen data.

The scenario described—training data contains information from the test set, leading to inflated performance—fits data leakage precisely, which is why it’s the right interpretation.