← AI glossary

AI glossary · Basic · Agents · Data & evaluation

Remote Labor Index

The Remote Labor Index (RLI) is an evaluation metric that measures how often an AI agent takes on a real freelance project from start to finish and completes it to a level fit for delivery to a client. It looks not at knowledge or problem solving but at the ability to carry out paid, real-world work in its entirety.

About this term
Level
Basic · Technical terms you often meet in AI news
Added
2026-11-13
In plain words It is like giving a new hire an actual outsourced job instead of a test, then having three senior colleagues check whether the result could be handed to the client as is. However, communication and collaboration with the client, which make up a large part of real work, are not part of this review.

What it is

RLI is an AI agent evaluation metric first released by the Center for AI Safety in the US and Scale AI. It was built from 240 projects completed by real professionals across fields such as game development, product design, architecture, data analysis and video production. Of these, 230 are used for private evaluation and only 10 are released as public examples. The total workload was described as more than 6,000 hours and the total value of the projects as over $140,000. It also includes hard tasks that take more than 100 hours or cost more than $10,000.

How it works

The AI is given the materials and tools needed for the work and assigned a project. Execution cost per project is capped at $30. The AI's output is placed side by side with the finished work made by a professional, and three evaluators directly judge whether it is good enough to deliver to a real client. The share of projects judged equal to or better than the human professional's work becomes the model's "automation rate."

How an RLI evaluation proceeds

  1. 1
    Assign projectProvide materials and tools
  2. 2
    Produce outputExecution cost capped at $30
  3. 3
    Compare with finished workCompare with professional's output
  4. 4
    Evaluator judgment3 people judge deliverability
  5. 5
    Tally automation rateShare at or above human level
After a real freelance project is assigned, human evaluators judge whether the result is deliverable to produce the automation rate.

Why it matters

When RLI was first released in 2025, Manus held the top score at 2.5%. In the latest results, Anthropic's Claude Fable 5.2 was reported highest at 41.3%, followed by Claude Opus 5.5 at 21.3% and GPT-6 Astra at 20.8%. That is more than double the 17.9% of the previous generation, Fable 5.1. The jump in one year from "1 in 40" to "2 in 5" is taken as a sign that AI is moving beyond solving single problems toward carrying out complex real-world work in its entirety.

RLI automation rate by model

Claude Fable 5.2
Claude Opus 5.5
GPT-6 Astra
Claude Fable 5.1
Manus (2025)
The share of evaluated projects judged equal to or better than a human professional. The Manus figure is from the first release in 2025.

Things to know

41.3% does not mean 41.3% of freelance jobs will disappear. It is only the share of evaluated projects where the AI's output was judged equal to or better than a human professional's. Successes were concentrated in sound effects production, image generation, report writing and data search. On the other hand, there were also failures where quality fell short, the work was not finished, or the required file format was not followed. Also, the same name (Remote Labor Index) was used in 2020 COVID-19 economic research for an index measuring "how feasible remote work is by occupation," so it should not be confused with RLI in the AI evaluation context.

Example

Suppose a company is considering handing part of its outsourced video editing or data analysis to AI agents. Rather than general benchmark scores, the person in charge can refer to results based on actual delivery quality, like RLI, to get a sense of which models are usable for which kinds of jobs. For example, the news that Fable 5.2 scored 41.3% on RLI can be read as "about four in ten jobs were good enough to hand straight to the client." However, given that successes were concentrated in sound effects, images, reports and data search, they should separately check whether their own company's work is close to those areas.

Related terms

References