← AI glossary

AI glossary · Basic · Data & evaluation · Agents · Language models

METR time horizon

The METR time horizon is a metric created by METR, a nonprofit that evaluates AI. It expresses, as a length of time, how long a task (measured by how long it takes a human) an AI model can complete.

About this term
Level
Basic · Technical terms you often meet in AI news
Added
2026-11-21
In plain words It is similar to describing a new employee's skill by "the size of the job you can leave to them alone," as in: "A 30-minute job can be handed over, but a full-day job is still a coin flip." However, a human's "full-day job" and the kinds of work AI is good at can differ, so the analogy is not exact.

What it is

Benchmarks usually compare models with a score such as accuracy. The time horizon expresses ability as "task length" instead of a score. Here, length means the time a human took to finish the same task. Looking at how long a task a model can complete gives an intuitive way to compare "how long a job AI can handle on its own." Graphs plotting this metric over time were widely shared, but because the measurement method is complex, they were also often misread.

How it works

First, human testers solve a set of tasks themselves, and each task is rated for difficulty by how long it took to finish. Next, AI models are given the same tasks. Finally, models are ranked by the length of task they can complete. Research materials call this value the "METR 50% task horizon," meaning it is based on the task length at which the model succeeds about half the time. So a "two-hour horizon" does not mean the model always completes two-hour tasks.

How the time horizon is measured

  1. 1
    Humans solve tasksRecord time taken per task
  2. 2
    Rate difficultyTime taken = task length
  3. 3
    Same tasks given to the modelRecord success · failure
  4. 4
    Set the horizonLength with about 50% success
Task difficulty is set by how long humans took, and then the task length at which the model succeeds about half the time is found.

Why it matters

MIT Technology Review reported that in the first graph, the task length frontier models could complete doubled roughly every four months. Because the graph showed the pace of AI progress at a glance, it was often cited in forecasts. An update released in November 2026 added tasks that take humans much longer and open-ended tasks without clear-cut answers. According to reports, some capabilities are still improving exponentially fast, but tasks that require judgment were not handled as well as coding. It draws strong interest because it relates to whether AI could carry out research and development on its own.

Things to know (limitations)

AI has improved especially quickly on "verifiable" tasks such as coding and math, where success and failure can be checked automatically. These tasks are easy to train on repeatedly with reinforcement learning. Whether AI improves at the same pace on tasks with fuzzy success criteria, such as choosing a new research direction, remains a big open question. An arXiv paper released in August 2026 pointed out limits in the measurement record itself: no open-weight model had a measured METR horizon, and when the metric version changes (1.0 vs. 1.1), the trend lines are hard to join directly. So forecasts that pin a date on "when AI will do what" based on this one graph should be read with caution.

Verifiable tasks vs judgment tasks

Verifiable tasksCoding · math

  • Success checked automatically
  • Easy to train with reinforcement learning
  • AI improves especially fast

Tasks requiring judgmente.g., choosing research directions

  • Fuzzy success criteria
  • Hard to train and hard to evaluate
  • Not as good as at coding

공통 Both are included in the updated METR task suite

Compares tasks that are scored automatically with tasks whose success criteria are fuzzy.

Example

When a new model comes out, news that its "METR time horizon has grown to so many hours" often circulates. When reading it, it helps to check three things: whether the tasks used were mainly verifiable tasks like coding, whether it uses the roughly-half-success (50%) criterion, and which version of the task suite was used. Adding open-ended tasks that require judgment, as in the November 2026 update, can make the same model look lower. So when a company decides whether to hand long tasks to an AI agent, it is safer not to rely on this number alone but to test directly on tasks similar to its own work.

Related terms

References