AI glossary · Beginner · Data & evaluation
Leaderboard
A leaderboard evaluates multiple AI models using the same benchmark or user votes and shows the results as a ranking table. It serves as a map for narrowing down initial candidates when choosing a model.
About this term
- Level
- Beginner · Core ideas for first-time readers
- Category
- Infrastructure & using models
- Fields
- Data & evaluation
- Added
- 2026-10-12
How it works
There are two main types of leaderboards. One has every model solve a fixed set of test questions (a benchmark) and assigns scores. The other, like LMArena, has people anonymously compare two models' answers to the same question and vote, then aggregates the results to calculate a score (rating). The former is easier to reproduce; the latter is closer to what real users experience.
Where you will see them
There are leaderboard spaces on Hugging Face, voting-based sites such as LMArena, and leaderboards for specific areas such as coding, math and search. Companies announcing models also frequently cite leaderboard rankings and scores in their blogs or model cards. Ranking tables usually show the model name, score, confidence interval or number of votes, and license. It is better to get a sense of the 'top group' than to focus on 'the single number one.'
Things to know
When score differences are small, a change in rank may reflect almost no real difference. Rankings can be inflated by data contamination, where test questions leak into training data, or by tuning aimed only at leaderboard scores. Voting-based leaderboards have been criticized for tending to favor long, friendly answers or attractive formatting. So after narrowing down candidates with a leaderboard, you should evaluate them directly on your own work data (Evals) and also weigh cost, speed and licensing.
What to check when reading a leaderboard
Reading the rankings
- Is the score difference within the confidence interval?
- Is the ranking for the same area as my use case?
- Is there any possibility of data contamination?
Adoption decision
- Does the license allow commercial use?
- Do cost and speed fit the budget?
- Have I evaluated it directly on my own data?
Example
Leaderboards have become a metric the industry watches closely, to the point that this site covered a report that LMArena's operator raised a large investment. In practice, when choosing a model for an internal chatbot, teams narrow the top of the leaderboard down to three or four that fit their license and budget requirements, then compare them directly on a set of frequently asked questions to decide on the final model.