Term
Faithfulness
Faithfulness is a RAG evaluation metric (0–1) measuring how many claims in a response can actually be derived from the retrieved contexts — a direct hallucination indicator.
Faithfulness — explained in detail
Faithfulness checks whether a generated answer is supported by the provided source contexts. The score sits between 0 and 1: 1 means every claim in the response can be backed by the contexts, 0 means none are. Computation has three steps: 1) claim extraction — the answer is broken into atomic statements. 2) claim verification — each statement is checked for logical derivability from the contexts. 3) score = supported claims / total claims.
Example / Practical use
Question: “When was Einstein born?”. Context: “Einstein was born on March 14, 1879 in Germany”. Answer A: “Einstein was born on March 14, 1879 in Germany” — every claim grounded → score 1.0. Answer B: “Einstein was born on March 20, 1879 in Germany” — the place is grounded, the date contradicts → score 0.5. Frameworks like RAGAS compute this either via an LLM judge or via a specialized classifier such as HHEM-2.1-Open.
Distinction from related terms
Faithfulness measures only adherence to the given context — not whether the answer is objectively true. An answer can be faithful to a flawed context and still be wrong in the real world. Related metrics: answer relevancy (does the answer match the question), context precision (how much of the context is relevant at all), and factual correctness (comparison against a gold answer). In production these are typically tracked together, because each metric exposes a different failure mode.
Discover more
AI Workflows by Keyword: How We Make Recurring Routines Enforceable
A typed keyword triggers a fixed AI routine — and every single step must be committed before the next one appears. Why that's the actual trick.
GlossaryGround Truth
A reference taken to be correct, against which model outputs are measured. In AI code analysis it is a piece of code with deliberately injected, known defects, so you can measure how many real problems a model actually finds.
EncyclopediaAgent Evaluation — Task Success Rate, Eval Harnesses, Benchmarks
How to evaluate AI agents: eval harnesses, task success rate, benchmarks like GAIA and Tau-Bench, LLM-as-a-judge — and how it differs from pure model quality.