Investigated
How do model judges get validated?
Short answer
Image: developer.apple.com
Model judges are validated by comparing their scores with carefully collected human judgments on representative test items, then checking agreement, calibration, bias, and robustness. Raw agreement alone can mislead when labels are imbalanced or when multiple human ratings are reasonable, so validation increasingly combines chance-corrected metrics, task-specific tests, and ongoing monitoring.
The views on this question
Two lenses on the same evidence. Source weight and the primary source ratio show what each rests on.
Skeptics
Source weight 50%Lens adapted to this topic: Limits of agreement metrics
Critical work argues that apparent validation can overstate reliability. High raw agreement may reflect class imbalance or chance, while different bookkeeping choices can change reported accuracy. Human labels may also conceal legitimate disagreement, and judges can share superficial heuristics that create consensus without measuring substantive quality. This view favors indeterminacy-aware labels, chance-corrected statistics, adversarial testing, and monitoring for drift.
- Raw agreement can be inflated by imbalanced labels: a judge that approves everything may achieve high agreement while having zero Cohen’s kappa.
- Forced-choice human labels can hide rating indeterminacy, where multiple ratings are reasonable; validation should account for that possibility.
- High agreement between judges may reflect shared surface heuristics rather than substantive quality, and model-level agreement can mask weaker item-level reliability.
- Judges can exhibit bias, unstable calibration, and performance drift after model or vendor updates, so validation should not be treated as a one-time exercise.
- A practical alternative emphasizes an iterative annotate–encode–validate–iterate process rather than relying on model choice or a prompt in isolation.
Follow the white rabbit
Deeper threads worth pulling on next.
- How are human judgments collected?Different angle, same storySampling, annotation instructions, and inter-annotator agreement define the human baseline for validation.
- What are chance-corrected metrics?Different angle, same storyCohen’s kappa, Krippendorff’s alpha, and similar stats adjust for agreement expected by chance.
- How do evaluators catch bias?Different angle, same storyAudit designs, demographic slicing, and targeted tests reveal systematic bias in judge outputs.