AI glossary · Basic · Data & evaluation · Agents · Language models
METR time horizon
The METR time horizon is a metric created by METR, a nonprofit that evaluates AI. It expresses, as a length of time, how long a task (measured by how long it takes a human) an AI model can complete.
About this term
- Level
- Basic · Technical terms you often meet in AI news
- Category
- Foundations & applications
- Fields
- Data & evaluation, Agents, Language models
- Added
- 2026-11-21
What it is
Benchmarks usually compare models with a score such as accuracy. The time horizon expresses ability as "task length" instead of a score. Here, length means the time a human took to finish the same task. Looking at how long a task a model can complete gives an intuitive way to compare "how long a job AI can handle on its own." Graphs plotting this metric over time were widely shared, but because the measurement method is complex, they were also often misread.
How it works
First, human testers solve a set of tasks themselves, and each task is rated for difficulty by how long it took to finish. Next, AI models are given the same tasks. Finally, models are ranked by the length of task they can complete. Research materials call this value the "METR 50% task horizon," meaning it is based on the task length at which the model succeeds about half the time. So a "two-hour horizon" does not mean the model always completes two-hour tasks.
How the time horizon is measured
- 1Humans solve tasksRecord time taken per task
- 2Rate difficultyTime taken = task length
- 3Same tasks given to the modelRecord success · failure
- 4Set the horizonLength with about 50% success
Why it matters
MIT Technology Review reported that in the first graph, the task length frontier models could complete doubled roughly every four months. Because the graph showed the pace of AI progress at a glance, it was often cited in forecasts. An update released in November 2026 added tasks that take humans much longer and open-ended tasks without clear-cut answers. According to reports, some capabilities are still improving exponentially fast, but tasks that require judgment were not handled as well as coding. It draws strong interest because it relates to whether AI could carry out research and development on its own.
Things to know (limitations)
AI has improved especially quickly on "verifiable" tasks such as coding and math, where success and failure can be checked automatically. These tasks are easy to train on repeatedly with reinforcement learning. Whether AI improves at the same pace on tasks with fuzzy success criteria, such as choosing a new research direction, remains a big open question. An arXiv paper released in August 2026 pointed out limits in the measurement record itself: no open-weight model had a measured METR horizon, and when the metric version changes (1.0 vs. 1.1), the trend lines are hard to join directly. So forecasts that pin a date on "when AI will do what" based on this one graph should be read with caution.
Verifiable tasks vs judgment tasks
Verifiable tasksCoding · math
- Success checked automatically
- Easy to train with reinforcement learning
- AI improves especially fast
Tasks requiring judgmente.g., choosing research directions
- Fuzzy success criteria
- Hard to train and hard to evaluate
- Not as good as at coding
공통 Both are included in the updated METR task suite
Example
When a new model comes out, news that its "METR time horizon has grown to so many hours" often circulates. When reading it, it helps to check three things: whether the tasks used were mainly verifiable tasks like coding, whether it uses the roughly-half-success (50%) criterion, and which version of the task suite was used. Adding open-ended tasks that require judgment, as in the November 2026 update, can make the same model look lower. So when a company decides whether to hand long tasks to an AI agent, it is safer not to rely on this number alone but to test directly on tasks similar to its own work.