Chatbot Arena

This leaderboard is based on the following benchmarks.

• Chatbot Arena - a crowdsourced, randomized battle platform for large language models (LLMs). We use 3.1M+ user votes to compute Elo ratings.
• MMLU - a test to measure a model’s multitask accuracy on 57 tasks.
• Arena-Hard-Auto - an automatic evaluation tool for instruction-tuned LLMs.

| Vote | Blog | GitHub | Paper | Dataset | Twitter | Discord |

OpenLM

Model	Arena Elo	MMLU	License
DeepSeek-R1-0528	1424	90.8	MIT
Qwen3-235B-A22B-no-thinking	1389	88.5	Apache 2.0
DeepSeek-V3-0324	1384	88.5	MIT
Minimax-M1	1374		Apache 2.0
Qwen3-235B-A22B	1367	88.5	Apache 2.0
Gemma-3-27B-it	1361		Gemma
Gemma-3n-e4b-it	1309		Gemma

Full Leaderboard

Model	Arena Elo	Coding	Vision	Arena Hard	MMLU	Votes	Organization	License
🥇 Gemini-2.5-Pro	1473	1491	1345	96.4		14062	Google	Proprietary
🥇 ChatGPT-4o-latest (2025-03-26)	1428	1439	1307			23599	OpenAI	Proprietary
🥇 o3-2025-04-16	1426	1441	1301			20095	OpenAI	Proprietary
🥇 DeepSeek-R1-0528	1424	1437		93.2	90.8	12396	DeepSeek	MIT
🥇 Grok-3-Preview-02-24	1423	1435			92.7	25067	xAI	Proprietary
🥇 Gemini-2.5-Flash	1418	1425	1302			19451	Google	Proprietary
🥇 GPT-4.5-Preview	1415	1418	1254			15271	OpenAI	Proprietary
🥈 Gemini-2.0-Flash-Thinking-Exp-01-21	1398	1380	1275			27613	Google	Proprietary
🥈 Gemini-2.0-Pro-Exp-02-05	1397	1396	1239			20120	Google	Proprietary
🥈 Qwen3-235B-A22B-no-thinking	1389	1413		95.6	88.5	14090	Alibaba	Apache 2.0
🥈 GPT-4.1-2025-04-14	1385	1391	1276			17456	OpenAI	Proprietary
🥈 DeepSeek-V3-0324	1384	1400		85.5	88.5	20171	DeepSeek	MIT
🥈 Hunyuan-Turbos-20250416	1380	1385				8244	Tencent	Proprietary
🥈 DeepSeek-R1	1375	1380		93.2	90.8	19430	DeepSeek	MIT
🥈 Minimax-M1	1374	1373				6117	MiniMax	Apache 2.0
🥈 Claude Opus 4 (20250514)	1372	1413	1230			19708	Anthropic	Proprietary
🥈 Mistral Medium 3	1369	1388	1202			18439	Mistral	Proprietary
🥈 Qwen3-235B-A22B	1367	1393		95.6	88.5	13817	Alibaba	Apache 2.0
🥈 o1-2024-12-17	1367	1375	1229	92.1	91.8	29038	OpenAI	Proprietary
🥈 o4-mini-2025-04-16	1364	1382	1256			17150	OpenAI	Proprietary
🥈 Gemini-2.0-Flash-001	1364	1362	1216			36673	Google	Proprietary
🥈 Qwen2.5-Max	1363	1368				31778	Alibaba	Proprietary
🥈 Grok-3-Mini-beta	1361	1381				9759	xAI	Proprietary
🥈 Gemma-3-27B-it	1361	1344	1224			25902	Google	Gemma
🥈 Claude Sonnet 4 (20250514)	1345	1385	1224			15543	Anthropic	Proprietary
🥈 o3-mini-high	1342	1379				19404	OpenAI	Proprietary
🥈 GPT-4.1-mini-2025-04-14	1338	1372	1241			16454	OpenAI	Proprietary
🥉 Gemma-3-12B-it	1338	1309				3976	Google	Gemma
🥉 DeepSeek-V3	1336	1336		85.5	88.5	22841	DeepSeek	DeepSeek
🥉 QwQ-32B	1334	1347				18063	Alibaba	Apache 2.0
🥉 Amazon-Nova-Experimental-Chat-05-14	1330	1347				6343	Amazon	Proprietary
🥉 Gemini-2.0-Flash-Lite	1330	1338	1157			26104	Google	Proprietary
🥉 Qwen-Plus-0125	1328	1336				6055	Alibaba	Proprietary
🥉 GLM-4-Plus-0111	1328	1306				6028	Zhipu	Proprietary
🥉 Command A (03-2025)	1327	1335				23879	Cohere	CC-BY-NC-4.0
🥉 o3-mini	1322	1360				36135	OpenAI	Proprietary
🥉 Step-2-16K-Exp	1322	1311				5126	StepFun	Proprietary
🥉 o1-mini	1321	1369		92		54951	OpenAI	Proprietary
🥉 Gemini-1.5-Pro-002	1320	1307	1222			58645	Google	Proprietary
🥉 Claude 3.7 Sonnet (thinking-32k)	1315	1345	1221			25145	Anthropic	Proprietary
🥉 Hunyuan-Turbo-0110	1314	1333				2510	Tencent	Proprietary
🥉 Llama-3.3-Nemotron-Super-49B-v1	1314	1317		88.3	86	2371	Nvidia	Nvidia
🥉 Gemma-3n-e4b-it	1309	1292				6311	Google	Gemma
🥉 Claude 3.7 Sonnet	1308	1344	1207			29775	Anthropic	Proprietary
🥉 Yi-Lightning	1305	1318		81.5		28968	01 AI	Proprietary
🥉 Grok-2-08-13	1305	1298			87.5	67084	xAI	Proprietary
🥉 GPT-4o-2024-05-13	1302	1308	1206	79.21	88.7	117747	OpenAI	Proprietary
🥉 Claude 3.5 Sonnet (20241022)	1301	1340	1186	85.2	88.7	77078	Anthropic	Proprietary
Deepseek-v2.5-1210	1297	1313				7243	DeepSeek	DeepSeek
Athene-v2-Chat-72B	1293	1316		85		26074	NexusFlow	NexusFlow
Llama-4-Maverick-17B-128E-Instruct	1293	1309	1197			17075	Meta	Llama 4
Gemma-3-4B-it	1293	1264				4321	Google	Gemma
Hunyuan-Large-2025-02-10	1289	1309				3856	Tencent	Proprietary
GPT-4o-mini-2024-07-18	1289	1298	1124	74.94	82	72531	OpenAI	Proprietary
Gemini-1.5-Flash-002	1289	1270	1206			37021	Google	Proprietary
GPT-4.1-nano-2025-04-14	1288	1311	1118			6302	OpenAI	Proprietary
Llama-3.1-405B-Instruct-bf16	1286	1296			88.6	43788	Meta	Llama 3.1
Llama-3.1-Nemotron-70B-Instruct	1286	1287		84.9		7577	Nvidia	Llama 3.1
Llama-3.1-405B-Instruct-fp8	1285	1292		69.3	88.6	63038	Meta	Llama 3.1
Grok-2-Mini-08-13	1284	1278				55442	xAI	Proprietary
Yi-Lightning-lite	1282	1282				17067	01 AI	Proprietary
Qwen-Max-0919	1281	1296				17432	Alibaba	Qwen
Hunyuan-Standard-2025-02-10	1278	1286				4014	Tencent	Proprietary
Qwen2.5-72B-Instruct	1275	1299		78		41519	Alibaba	Qwen
Llama-3.3-70B-Instruct	1275	1274				47256	Meta	Llama-3.3
GPT-4-Turbo-2024-04-09	1274	1278	1151	82.63		102133	OpenAI	Proprietary
Mistral-Small-3.1-24B-Instruct-2503	1273	1296	1175			6154	Mistral	Apache 2.0
Athene-70B	1268	1269		77.6		20580	NexusFlow	CC-BY-NC-4.0
GPT-4-1106-preview	1267	1268				103748	OpenAI	Proprietary
Mistral-Large-2411	1266	1282		70.42		29633	Mistral	MRL
Llama-3.1-70B-Instruct	1265	1267		55.73	86	58637	Meta	Llama 3.1
Claude 3 Opus	1265	1266	1076	60.36	86.8	202641	Anthropic	Proprietary
magistral-medium-2506	1262	1313				3602	Mistral	Proprietary
Amazon Nova Pro 1.0	1262	1279	1044			26371	Amazon	Proprietary
GPT-4-0125-preview	1262	1259		77.96		97079	OpenAI	Proprietary
Llama-3.1-Tulu-3-70B	1262	1249				3010	Ai2	Llama 3.1
Claude 3.5 Haiku (20241022)	1257	1285	1158			48556	Anthropic	Propretary
Reka-Core-20240904	1253	1237				7948	Reka AI	Proprietary
Jamba-1.5-Large	1239	1243			81.2	9125	AI21 Labs	Jamba Open
Deepseek-v2-API-0628	1237	1257				19508	DeepSeek AI	DeepSeek
Gemma-2-27B-it	1237	1225		57.51		79538	Google	Gemma license
Hunyuan-Large-Vision	1236	1258	1260			3905	Tencent	Proprietary
Qwen2.5-Coder-32B-Instruct	1235	1277				5730	Alibaba	Apache 2.0
Mistral-Small-24B-Instruct-2501	1235	1248				15321	Mistral	Apache 2.0
Amazon Nova Lite 1.0	1234	1251	1061			20646	Amazon	Proprietary
Gemma-2-9B-it-SimPO	1234	1212				10548	Princeton	MIT
Command R+ (08-2024)	1233	1197				10535	Cohere	CC-BY-NC-4.0
Deepseek-Coder-v2-0724	1232	1282		62.3		11725	DeepSeek	Proprietary
Gemini-1.5-Flash-8B-001	1230	1224	1106			37697	Google	Proprietary
Llama-3.1-Nemotron-51B-Instruct	1229	1226				3889	Nvidia	Llama 3.1
Nemotron-4-340B-Instruct	1227	1214				20608	Nvidia	Nvidia
Aya-Expanse-32B	1227	1209				28768	Cohere	CC-BY-NC-4.0
GLM-4-0520	1224	1232		63.84		10221	Zhipu AI	Proprietary
Llama-3-70B-Instruct	1224	1215		46.57	82	163629	Meta	Llama 3
Phi-4	1223	1238				25213	Microsoft	MIT
OLMo-2-0325-32B-Instruct	1223	1214				3460	Allen AI	Apache-2.0
Reka-Flash-20240904	1223	1207				8132	Reka AI	Proprietary
Claude 3 Sonnet	1218	1229	1048	46.8	79	113067	Anthropic	Proprietary
Amazon Nova Micro 1.0	1215	1226				20654	Amazon	Proprietary
Gemma-2-9B-it	1209	1189				57197	Google	Gemma license
Hunyuan-Standard-256K	1206	1243				2901	Tencent	Proprietary
Qwen2-72B-Instruct	1205	1202		46.86	84.2	38872	Alibaba	Qianwen LICENSE
GPT-4-0314	1204	1211		50	86.4	55962	OpenAI	Proprietary
Llama-3.1-Tulu-3-8B	1203	1195				3074	Ai2	Llama 3.1
Ministral-8B-2410	1200	1217				5111	Mistral	MRL
Claude 3 Haiku	1197	1205	1000	41.47	75.2	122309	Anthropic	Proprietary
Aya-Expanse-8B	1197	1181				10391	Cohere	CC-BY-NC-4.0
Command R (08-2024)	1197	1177				10851	Cohere	CC-BY-NC-4.0
DeepSeek-Coder-V2-Instruct	1196	1255				15753	DeepSeek AI	DeepSeek License
Llama-3.1-8B-Instruct	1193	1202		21.34	73	52578	Meta	Llama 3.1
Jamba-1.5-Mini	1193	1196			69.7	9274	AI21 Labs	Jamba Open
GPT-4-0613	1180	1183		37.9		91614	OpenAI	Proprietary
Qwen1.5-110B-Chat	1179	1190			80.4	27430	Alibaba	Qianwen LICENSE
Yi-1.5-34B-Chat	1175	1178			76.8	25135	01 AI	Apache-2.0
Llama-3-8B-Instruct	1169	1162		20.56	68.4	109056	Meta	Llama 3
InternLM2.5-20B-chat	1166	1174				10599	InternLM	Other
Claude-1	1166	1151			77	21149	Anthropic	Proprietary
Qwen1.5-72B-Chat	1165	1176		36.12	77.5	40658	Alibaba	Qianwen LICENSE
Mixtral-8x22b-Instruct-v0.1	1165	1168		36.36	77.8	53751	Mistral	Apache 2.0
Mistral Medium	1165	1168		31.9	75.3	35556	Mistral	Proprietary
Gemma-2-2b-it	1161	1123			51.3	48892	Google	Gemma license
Granite-3.1-8B-Instruct	1160	1189				3289	IBM	Apache 2.0
Claude-2.0	1149	1150		23.99	78.5	12763	Anthropic	Proprietary
Gemini-1.0-Pro-001	1149	1118			71.8	18800	Google	Proprietary
Zephyr-ORPO-141b-A35b-v0.1	1145	1140				4854	HuggingFace	Apache 2.0
Qwen1.5-32B-Chat	1143	1165			73.4	22765	Alibaba	Qianwen LICENSE
Phi-3-Medium-4k-Instruct	1140	1141		33.37	78	26105	Microsoft	MIT
Granite-3.1-2B-Instruct	1137	1164				3380	IBM	Apache 2.0
Claude-2.1	1136	1147		22.77		37699	Anthropic	Proprietary
Starling-LM-7B-beta	1136	1145		23.01		16676	Nexusflow	Apache-2.0
GPT-3.5-Turbo-0613	1134	1150		24.82		38955	OpenAI	Proprietary
Mixtral-8x7B-Instruct-v0.1	1131	1130		23.4	70.6	76126	Mistral	Apache 2.0
Claude-Instant-1	1129	1124			73.4	20631	Anthropic	Proprietary
Yi-34B-Chat	1129	1122		23.15	73.5	15917	01 AI	Yi License
Qwen1.5-14B-Chat	1126	1141			67.6	18687	Alibaba	Qianwen LICENSE
WizardLM-70B-v1.0	1124	1087			63.7	8383	Microsoft	Llama 2
DBRX-Instruct-Preview	1121	1134		24.63	73.7	33743	Databricks	DBRX LICENSE
Llama-3.2-3B-Instruct	1120	1096				8390	Meta	Llama 3.2
Phi-3-Small-8k-Instruct	1119	1123		29.77	75.7	18476	Microsoft	MIT
Tulu-2-DPO-70B	1116	1109		14.99		6658	AllenAI/UW	AI2 ImpACT Low-risk
Granite-3.0-8B-Instruct	1111	1113				7002	IBM	Apache 2.0
Llama-2-70B-chat	1110	1088		11.55	63	39595	Meta	Llama 2
OpenChat-3.5-0106	1109	1118			65.8	12990	OpenChat	Apache-2.0
Vicuna-33B	1108	1083		8.63	59.2	22936	LMSYS	Non-commercial
Snowflake Arctic Instruct	1107	1093		17.61	67.3	34173	Snowflake	Apache 2.0
Starling-LM-7B-alpha	1106	1095		12.8	63.9	10415	UC Berkeley	CC-BY-NC-4.0
Nous-Hermes-2-Mixtral-8x7B-DPO	1102	1095				3836	NousResearch	Apache-2.0
Gemma-1.1-7B-it	1101	1100		12.09	64.3	25070	Google	Gemma license
NV-Llama2-70B-SteerLM-Chat	1098	1038			68.5	3636	Nvidia	Llama 2
pplx-70B-online	1095	1044				6898	Perplexity AI	Proprietary
DeepSeek-LLM-67B-Chat	1094	1095			71.3	4988	DeepSeek AI	DeepSeek License
OpenChat-3.5	1094	1070			64.3	8106	OpenChat	Apache-2.0
OpenHermes-2.5-Mistral-7B	1092	1073				5088	NousResearch	Apache-2.0
Granite-3.0-2B-Instruct	1091	1103				7191	IBM	Apache 2.0
Mistral-7B-Instruct-v0.2	1090	1089		12.57		20067	Mistral	Apache-2.0
Phi-3-Mini-4K-Instruct-June-24	1088	1098			70.9	12808	Microsoft	MIT
Qwen1.5-7B-Chat	1087	1105			61	4872	Alibaba	Qianwen LICENSE
Phi-3-Mini-4k-Instruct	1084	1102			68.8	21097	Microsoft	MIT
Llama-2-13b-chat	1081	1067			53.6	19722	Meta	Llama 2
SOLAR-10.7B-Instruct-v1.0	1080	1063			66.2	4286	Upstage AI	CC-BY-NC-4.0
Dolphin-2.2.1-Mistral-7B	1080	1041				1714	Cognitive Computations	Apache-2.0
WizardLM-13b-v1.2	1076	1041			52.7	7176	Microsoft	Llama 2
Llama-3.2-1B-Instruct	1071	1062				8523	Meta	Llama 3.2
Qwen2.5-VL-32B-Instruct			1214			1505	Alibaba	Apache 2.0
Step-1o-Vision-32k (highres)			1186			2891	StepFun	Proprietary
Llama-4-Scout-17B-16E-Instruct			1175			1542	Meta	Llama
Qwen2.5-VL-72B-Instruct			1169			3884	Alibaba	Qwen
Pixtral-Large-2411			1153			5546	Mistral	MRL
Qwen-VL-Max-1119			1128			1449	Alibaba	Proprietary
Qwen2-VL-72b-Instruct			1111			6028	Alibaba	Qwen
Step-1V-32K			1111			1553	StepFun	Proprietary
Molmo-72B-0924			1076			3092	AI2	Apache 2.0
Pixtral-12B-2409			1072			7623	Mistral	Apache 2.0
Llama-3.2-90B-Vision-Instruct			1070			8829	Meta	Llama 3.2
InternVL2-26B			1067			5265	OpenGVLab	MIT
Hunyuan-Standard-Vision-2024-12-31			1064			811	Tencent	Proprietary
Aya-Vision-32B			1058			849	Cohere	CC-BY-NC-4.0
Qwen2-VL-7B-Instruct			1054			5854	Aliaba	Apache 2.0
Yi-Vision			1046			1237	01 AI	Proprietary
Llama-3.2-11B-Vision-Instruct			1032			4893	Meta	Llama 3.2

If you want to see more models, please help us add them.

💻 Code: The Arena Elo ratings are computed by this notebook. The MT-bench scores (single-answer grading on a scale of 10) are computed by fastchat.llm_judge. The MMLU scores are computed by InstructEval. Higher values are better for all benchmarks. Empty cells mean not available. The latest and detailed leaderboard is here.

More Statistics for Chatbot Arena

🔗 Arena Statistics

Transition from online Elo rating system to Bradley-Terry model

We adopted the Elo rating system for ranking models since the launch of the Arena. It has been useful to transform pairwise human preference to Elo ratings that serve as a predictor of winrate between models. Specifically, if player A has a rating of R_A and player B a rating of R_B, the probability of player A winning is

ELO rating has been used to rank chess players by the international community for over 60 years. Standard Elo rating systems assume a player’s performance changes overtime. So an online algorithm is needed to capture such dynamics, meaning recent games should weigh more than older games. Specifically, after each game, a player’s rating is updated according to the difference between predicted outcome and actual outcome.

This algorithm has two distinct features:

1. It can be computed asynchronously by players around the world.
2. It allows for players performance to change dynamically – it does not assume a fixed unknown value for the players rating.

This ability to adapt is determined by the parameter K which controls the magnitude of rating changes that can affect the overall result. A larger K essentially put more weight on the recent games, which may make sense for new players whose performance improves quickly. However as players become more senior and their performance “converges” then a smaller value of K is more appropriate. As a result, USCF adopted K based on the number of games and tournaments completed by the player (reference). That is, the Elo rating of a senior player changes slower than a new player.

When we launched the Arena, we noticed considerable variability in the ratings using the classic online algorithm. We tried to tune the K to be sufficiently stable while also allowing new models to move up quickly in the leaderboard. We ultimately decided to adopt a bootstrap-like technique to shuffle the data and sample Elo scores from 1000 permutations of the online plays. You can find the details in this notebook. This provided consistent stable scores and allowed us to incorporate new models quickly. This is also observed in a recent work by Cohere. However, we used the same samples to estimate confidence intervals which were therefore too wide (effectively CI’s for the original online Elo estimates).

In the context of LLM ranking, there are two important differences from the classic Elo chess ranking system. First, we have access to the entire history of all games for all models and so we don’t need a decentralized algorithm. Second, most models are static (we have access to the weights) and so we don’t expect their performance to change. However, it is worth noting that the hosted proprietary models may not be static and their behavior can change without notice. We try our best to pin specific model API versions if possible.

To improve the quality of our rankings and their confidence estimates, we are adopting another widely used rating system called the Bradley–Terry (BT) model. This model actually is the maximum likelihood (MLE) estimate of the underlying Elo model assuming a fixed but unknown pairwise win-rate. Similar to Elo rating, BT model is also based on pairwise comparison to derive ratings of players to estimate win rate between each other. The core difference between BT model vs the online Elo system is the assumption that player’s performance does not change (i.e., game order does not matter) and the computation takes place in a centralized fashion.