Which element is essential in a robust evaluation plan for prompt-based tasks?

• A. Define success criteria and use objective metrics such as accuracy and F1.
• B. Rely only on user satisfaction surveys.
• C. Use proper data splits, cross-validation, ablations, and statistical testing.
• D. Avoid human evaluation altogether.

Answer: (C)

Prompt-based tasks bring variability from how prompts are worded and from non-deterministic model outputs. A robust evaluation plan tackles this by using proper data splits so the test data is independent and by employing cross-validation to check that results hold across different data subsets. Ablations reveal what parts of the setup actually drive performance—whether changes come from the prompt, the few-shot template, or the model itself—so improvements are understood and not just incidental. Statistical testing then checks that observed differences aren’t due to random chance, strengthening claims about what works. This combination makes results more trustworthy and generalizable beyond a single dataset or prompt. Relying solely on user satisfaction surveys or evaluating with a single metric without a solid experimental design can misrepresent capability and overlook important qualitative or statistical factors.