NEW YORK — What happened
A multi-institutional research team published a scathing audit of contemporary AI evaluation practices, demonstrating that high scores on prominent benchmarks like MMLU-Pro and SWE-bench increasingly fail to correlate with real-world utility and safety.
Why it matters
When industry metrics become the primary target for optimization, models are increasingly fine-tuned to pass specific tests rather than acquire generalizable reasoning skills. This phenomenon, driven by Goodhart's Law, creates a dangerous illusion of capability.
Deep dive
The study analyzed training data corpora and uncovered widespread contamination, where benchmark test items were inadvertently ingested during the web-scraping phase of foundation model pre-training. When researchers administered newly generated, un-scraped dynamic evaluations to the exact same models, performance dropped by an average of 34 percent across the board.
Report check (claims vs what is verified vs still rumor)
The paper claims that over 80 percent of popular public leaderboards suffer from severe data contamination. This is verified through systematic suffix-matching and n-gram analysis of training sets. Rumors that major labs intentionally poison competitor benchmarks have been investigated and found to lack supporting evidence.
Open questions
How can the AI research community transition toward interactive, runtime-evaluated environments that cannot be memorized or gamed by static training datasets?