NEW YORK — What happened
A major controversy erupted this month when three prominent AI labs released safety and capability evaluation reports relying almost entirely on automated AI judges rather than human subject matter experts. Critics argue this creates a self-congratulatory feedback loop.
Why it matters
Regulatory compliance frameworks in the EU and US increasingly rely on third-party evaluations to certify high-risk AI deployments. If the evaluation mechanism itself is biased toward the style and rhetoric of the model being tested, safety audits become meaningless.
Deep dive
Automated judges consistently favor outputs that mimic their own tone, vocabulary, and structural pacing. When human domain experts—such as cryptographers, medical doctors, and legal scholars—blind-test the same outputs, they routinely flag logical errors and safety violations that the automated judge passes with high confidence scores.
Report check (claims vs what is verified vs still rumor)
It is verified that commercial safety evaluators heavily utilize LLM-as-a-judge pipelines to reduce labor costs. It remains disputed whether human-in-the-loop evaluations can scale fast enough to keep pace with weekly model updates.
Open questions
Will regulatory bodies mandate human-only red-teaming panels for frontier models, or will they accept statistically adjusted automated evaluations?