AI glossary · Basic · Agents · Data & evaluation
Remote Labor Index
The Remote Labor Index (RLI) is an evaluation metric that measures how often an AI agent takes on a real freelance project from start to finish and completes it to a level fit for delivery to a client. It looks not at knowledge or problem solving but at the ability to carry out paid, real-world work in its entirety.
About this term
- Level
- Basic · Technical terms you often meet in AI news
- Category
- Perception & action
- Fields
- Agents, Data & evaluation
- Added
- 2026-11-13
What it is
RLI is an AI agent evaluation metric first released by the Center for AI Safety in the US and Scale AI. It was built from 240 projects completed by real professionals across fields such as game development, product design, architecture, data analysis and video production. Of these, 230 are used for private evaluation and only 10 are released as public examples. The total workload was described as more than 6,000 hours and the total value of the projects as over $140,000. It also includes hard tasks that take more than 100 hours or cost more than $10,000.
How it works
The AI is given the materials and tools needed for the work and assigned a project. Execution cost per project is capped at $30. The AI's output is placed side by side with the finished work made by a professional, and three evaluators directly judge whether it is good enough to deliver to a real client. The share of projects judged equal to or better than the human professional's work becomes the model's "automation rate."
How an RLI evaluation proceeds
- 1Assign projectProvide materials and tools
- 2Produce outputExecution cost capped at $30
- 3Compare with finished workCompare with professional's output
- 4Evaluator judgment3 people judge deliverability
- 5Tally automation rateShare at or above human level
Why it matters
When RLI was first released in 2025, Manus held the top score at 2.5%. In the latest results, Anthropic's Claude Fable 5.2 was reported highest at 41.3%, followed by Claude Opus 5.5 at 21.3% and GPT-6 Astra at 20.8%. That is more than double the 17.9% of the previous generation, Fable 5.1. The jump in one year from "1 in 40" to "2 in 5" is taken as a sign that AI is moving beyond solving single problems toward carrying out complex real-world work in its entirety.
RLI automation rate by model
Things to know
41.3% does not mean 41.3% of freelance jobs will disappear. It is only the share of evaluated projects where the AI's output was judged equal to or better than a human professional's. Successes were concentrated in sound effects production, image generation, report writing and data search. On the other hand, there were also failures where quality fell short, the work was not finished, or the required file format was not followed. Also, the same name (Remote Labor Index) was used in 2020 COVID-19 economic research for an index measuring "how feasible remote work is by occupation," so it should not be confused with RLI in the AI evaluation context.
Example
Suppose a company is considering handing part of its outsourced video editing or data analysis to AI agents. Rather than general benchmark scores, the person in charge can refer to results based on actual delivery quality, like RLI, to get a sense of which models are usable for which kinds of jobs. For example, the news that Fable 5.2 scored 41.3% on RLI can be read as "about four in ten jobs were good enough to hand straight to the client." However, given that successes were concentrated in sound effects, images, reports and data search, they should separately check whether their own company's work is close to those areas.
Related terms
References
- AI performs 41.3% of freelance work at professional level, up sharply from 2.5% a year ago (AI Times)
- Supply and demand shocks in the COVID-19 pandemic: An industry and occupation perspective (arxiv.org)
- Anthropic's Claude Fable 5.2 completes 41% of real freelance jobs on the Remote Labor Index, double the next model (MLQ.ai)
- Anthropic’s Fable 5.2 Does 41% of Freelance Jobs in RLI Test (FourWeekMBA)