Artificial Analysis 改版 Intelligence Index，开源与闭源模型差距扩大
New Artificial Analysis index widens the gap between open and closed models
AI 导读
Artificial Analysis 改版 Intelligence Index 后，Kimi K3.1 Pro 从第 4 名降至第 9 名，排在 8 个闭源模型之后。
摘要与补充说明
新指数中开源模型与榜首的差距从 3 分扩大到 12 分，Grok 4.7 也下降了 12 分。改版后 60% 的指数权重来自私有测试集，用于防止针对已知测试题训练的 benchmaxxing。AI 研究员 Nathan Lambert 认为，benchmaxxing 只是一部分原因，许多训练收益来自与基准任务相似的强化学习环境，而同时维持多领域水平需要复杂的 RL 基础设施。
正文
Artificial Analysis overhauled its Intelligence Index, causing Kimi K3.1 Pro to drop five spots in the rankings. The model originally launched in fourth place, but it falls to ninth behind eight closed models in version 5. Artificial Analysis now puts the gap between open models and the top of the leaderboard at 12 points, up from 3 points previously.
For the first time, 60 percent of the index relies on private test sets to prevent "benchmaxxing," the practice of training models specifically on known test questions. Kimi K3.1 Pro's drop suggests the old index made some models look better than they were. Grok 4.7 also lost 12 points.
AI researcher Nathan Lambert thinks benchmaxxing is only part of the story. Many training gains today happen in reinforcement learning environments with tasks that resemble the benchmarks. Keeping up across multiple domains at once takes sophisticated RL infrastructure, and not every lab has it.
Closed test sets do have a downside, since outsiders can no longer spot errors in the prompts. The AA-Omniscience knowledge test proves those errors exist. Artificial Analysis had to revise ambiguous and flawed questions in that benchmark for the new index.
AI News Without the Hype – Curated by Humans
Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.