The distinction teams often blur is between an eval that produces a number and one that guides a decision. Most people building LLM applications start running evaluations within their first few months with a production feature. Far fewer run those same evaluations a year later. The ones abandoned usually share one problem: their number never informed a decision.
A score you never use isn't an evaluation. It's noise with extra steps.
While building Monogram's eval engine, I've reviewed many eval setups over the past year. Certain properties consistently distinguish useful evaluations from ignored ones. This is my attempt to define those properties precisely.
Validity: does the eval reflect production needs
The key first question is whether the eval measures something tied to production quality. That is harder than it sounds: the easiest evaluations to build often measure the wrong thing.
Take a summarization feature. An easy eval measures ROUGE or similar n-gram overlap between a generated summary and a reference. ROUGE is computable, deterministic, and well-understood. Yet it often fails to show whether a reader finds the summary useful. A summary may score poorly by using synonyms while clearly improving on the reference, or score well by using the right words in the wrong order of importance.
The validity test is: when this eval score rises, does production quality rise too? If you cannot confidently say yes, the eval measures a proxy rather than the real thing. Proxies belong in an eval suite when direct measurement is difficult, but they are partial evidence, not primary signals.
The strongest validity test uses outputs you have already reviewed manually. Run the eval on them and compare its rankings with yours. If they usually match, the eval is valid for your task distribution. If they differ, it measures something other than what you value.
Reliability: same inputs, same scores
A reliable evaluation returns consistent scores across runs. Deterministic scorers, including exact match, schema validation, and regex checks, do this automatically. LLM-as-judge evaluations need deliberate design.
LLM-as-judge methods are increasingly common because they capture nuance that rubric-based scoring can miss. But a judge scoring the same response 7/10 once and 5/10 next time is unreliable. Judge-score variance becomes indistinguishable from model-behavior variance, so a score change cannot be trusted as a real behavior change.
Why judges become unreliable:
- Temperature set too high, introducing randomness into scoring
- Underspecified rubrics that allow the judge to weight criteria differently across runs
- No fixed examples in the judge prompt to anchor its interpretation of the scale
- Context ordering effects when the judge sees multiple responses together
The practical test is to run your judge ten times on a fixed response set. If per-response score variance exceeds 10 to 15 percent of the scale range, add constraints. Temperature 0.0, a detailed rubric with anchored points, such as what specifically separates a 3 from a 4, and fixed few-shot examples for edge cases usually reduce variance to an acceptable level.
Coverage: are important cases included
A frequent failure: teams build eval suites from examples already available or cases considered during development. Production failures usually come from edge cases nobody considered.
Coverage does not mean a large eval set. A 500-example eval drawn from the same distribution as easy training data is less useful than a 50-example eval that intentionally includes the input types you fear most. Coverage means deciding which failures matter most and representing them.
Review production failures, user complaints, or support tickets from the past several months. Group them by failure type, then include examples of every type in your eval suite. Any production failure absent from the eval is a coverage gap. An eval cannot catch failures it was not built to detect.
Actionability: does failure suggest a next step
An eval that detects a problem but offers no place to investigate is frustrating. Good evals connect failure results to a specific hypothesis: the output was too long, the citation was missing, or entity extraction missed a type.
Monolithic evals that give one 1-10 response score fail this test. A score of 4 instead of 8 does not reveal which of ten possible problems occurred. Decomposed evals score each criterion separately, locating the failure. You can see that citation recall dropped while factuality and length stayed unchanged, pointing to the prompt changes most likely to help.
The objection is that decomposed evals cost more to build and maintain. True. The choice is between designing good evaluation structure upfront and later debugging changes from one aggregate number. In my experience, upfront work pays off quickly when prompt changes ship frequently.
Stability with time
An eval needing constant upkeep to remain meaningful will be abandoned. Two causes create most maintenance burden: changing external ground truth and scorer assumptions that no longer hold.
If a scorer depends on output formats your prompts use today, a quality-improving prompt change that alters the format can break the eval even as quality rises. Evals should test behavior, not format, unless format is what truly matters.
Ask when the eval last changed. A change made deliberately to measure something new is fine. A change made because something stopped working for unclear reasons signals a fragile eval.
Answering the usual objection
The objection I hear most: "We're moving too fast to build evals with all these properties. We'll do it properly once things stabilize."
I understand the reasoning, but it does not hold up empirically. The teams I have seen struggle most with LLM quality built fast, shallow eval infrastructure early, then could not change it because deployment depended on it. A shallow eval gating production is worse than no gate: it creates false confidence.
An effective eval needs less initial investment than the full ideal above. You do not need decomposed criteria or a calibrated LLM judge on day one. You do need validity, consistent runs, and coverage of the input types you worry about most. For most features, that takes a day or two. These properties help assess whether the investment is worthwhile, not define prerequisites.
A decision-guiding score is worth much more than one that guides nothing, whatever the methodology underneath.