How can you measure factuality in LLM outputs, and what evaluation approaches are commonly used?

• A. Factuality is irrelevant for LLM outputs.
• B. Compare generated facts to trusted sources and consider citation-aware generation.
• C. Factuality can be judged solely by the length of the generated text.
• D. Only the model's internal confidence score determines factuality.

Answer: (B)

Measuring factuality in LLM outputs centers on grounding what the model says to verifiable sources and providing traceable evidence. The best approach is to compare generated facts to trusted sources such as primary literature, official records, or reputable databases, and to use citation-aware generation so the model can indicate where a fact comes from. This directly enables verification and helps reduce hallucinations by attaching verifiable provenance to statements. In practice, evaluation blends automatic checks with human judgment. Automatic methods include QA-based verification (issuing questions whose answers should be supported by sources), retrieval-augmented evaluation (checking facts against retrieved documents), and fact-focused metrics like entity-level accuracy or belief consistency. Human evaluation uses clear rubrics to judge factual correctness, completeness, and relevance, often by domain experts. Benchmarks like FEVER and related datasets illustrate different facets of factuality and help expose where models stray. Relying on length or on the model’s internal confidence alone is misleading, since neither guarantees truth or traceability; external verification and evidence are essential.