How LMRank ranks models

What the ranking means

LMRank orders active models using a maintained composite score from 1 to 10. The public site emphasizes rank because it is easier to compare and less precise-looking than the underlying calibration. The score is an editorial synthesis of available evidence, not a simple average of every benchmark and not a claim that one number captures every workload.

Evidence used

We favor public, inspectable evidence and record source links on model pages. Inputs can include:

• Independent benchmark suites such as Artificial Analysis and SWE-bench
• Official model cards, system cards, release notes, and provider documentation
• Published context limits, modalities, API availability, and list pricing
• LMRank evaluation generations when a complete, versioned run is available

How ordering is produced

New models are calibrated against nearby models with comparable published evidence. Active models are ordered by composite score, with a stable slug-based tie break. When LMRank has a complete validated leaderboard generation, its versioned entries can supply the active score and rank. Models without enough trustworthy evidence can remain listed while their placement is conservative or their page is excluded from search indexing.

What the ranking does not guarantee

A higher overall rank does not guarantee lower latency, lower total task cost, better local deployment, or better results for every prompt. Benchmark coverage differs by model and provider-reported results can use different settings. Use category pages and model source links to check the constraints that matter for your workload.

Updates and corrections

Models, pricing, and rankings are reviewed when releases or credible new evidence arrive. To report stale pricing, a broken source, or a ranking-data problem, open a public correction issue on GitHub. Include the affected URL and a primary source so the change can be verified.