Epoch AI's InnovationEval: even the latest models struggle to match human algorithmic innovation (per community reports)

What happened

Epoch AI's InnovationEval tests whether AI can carry out the full process on its own, from coming up with ideas through implementation, experiments and analysis, to produce a new method that matches SDPO, a human-developed post-training technique. Claude Fable 5 and GPT-5.6 Sol were each allowed 3,000 GPU-hours and 10 billion tokens per evaluation run. According to community summaries, both models struggled to reach human-level algorithmic innovation.

Why it matters

With more claims that AI can do research on its own and deliver results, this could be evidence to the contrary: AI's ability to actually create new techniques is still limited.

Sources

• Community 최신 AI 모델들, 인간의 알고리듬 혁신에 필적하는 성과를 내는 데 어려움 겪어 GeekNews