Two benchmarks on the Hugging Face blog: Ai2's 'StateBench' for tracking state changes in long documents and 'CWE-bench v2' with improved scope, quality and transparency
Announced November 3, 2026
Verified (fact)
Ai2 introduced 'StateBench' on the Hugging Face blog, which tests how well language models track state changes in long documents. The same day, 'CWE-bench v2', a benchmark based on security weakness types (CWE), was released with improved coverage, quality, and transparency. The collected material for both posts had no body excerpts, so detailed figures could not be confirmed.
Why it matters (interpretation)
Long-context tracking and secure code evaluation are considered weak spots in practical agent work, so the new evaluation tools fill a gap.
Confidence medium
Sources
- Official StateBench: Testing how language models track state changes over long documents Hugging Face
- Official CWE-bench v2: More coverage, quality, and transparency Hugging Face
More about Hugging Face
-
ERNIE 5.5 Flash Base follow-up: weights license excludes South Korea and Europe from permitted regions (report)
According to AI Times, the 'ERNIE 5.5 Flash Base' weights Baidu uploaded to Hugging Face on the 5th exclude South Korea and Europe from...
-
Hugging Face's 'Harness Adapters' train LoRA adapters per harness and task to tailor agent performance (community report)
A poster from Hugging Face introduced 'Harness Adapters', an approach that trains LoRA adapters on a model to fit the agent harness in...
-
Hugging Face's Open ASR Leaderboard adds Egyptian and Nigerian English evaluations
According to the Hugging Face blog, Egyptian English and Nigerian English have been added to the Open ASR Leaderboard, a ranking of...
-
Ai2 releases 'ToyBlocks', a controlled environment for robot learning and evaluation
Ai2 introduced 'ToyBlocks' on the Hugging Face blog. ToyBlocks is an environment with controlled conditions for robot learning and...
-
NVIDIA unveils Cosmos 3, an open model for interactive world simulation
NVIDIA published a post on the Hugging Face blog introducing Cosmos 3, a new open model for interactive world simulation. The collected...