Chatbot Arena

This leaderboard is based on the following benchmarks.

• Chatbot Arena - a crowdsourced, randomized battle platform for large language models (LLMs). We use 3M+ user votes to compute Elo ratings.
• MMLU - a test to measure a model’s multitask accuracy on 57 tasks.
• Arena-Hard-Auto - an automatic evaluation tool for instruction-tuned LLMs.

| Vote | Blog | GitHub | Paper | Dataset | Twitter | Discord |

Best Open LM

Model	Arena Elo	MMLU	License
DeepSeek-V3-0324	1382	88.5	MIT
DeepSeek-R1	1371	90.8	MIT
Qwen3-235B-A22B	1360	88.5	Apache 2.0
Gemma-3-27B-it	1355		Gemma

Full Leaderboard

Model	Arena Elo	Coding	Vision	Arena Hard	MMLU	Votes	Organization	License
🥇 Gemini-2.5-Pro-Preview-06-05	1478	1494	1353	96.4		7343	Google	Proprietary
🥇 o3-2025-04-16	1425	1442	1300			15210	OpenAI	Proprietary
🥇 ChatGPT-4o-latest (2025-03-26)	1423	1431	1309			19762	OpenAI	Proprietary
🥇 Gemini-2.5-Flash-Preview-05-20	1420	1431	1305			12614	Google	Proprietary
🥇 Grok-3-Preview-02-24	1417	1429			92.7	21879	xAI	Proprietary
🥇 GPT-4.5-Preview	1411	1416	1253			15271	OpenAI	Proprietary
🥈 Gemini-2.0-Pro-Exp-02-05	1394	1394	1239			20120	Google	Proprietary
🥈 Gemini-2.0-Flash-Thinking-Exp-01-21	1394	1379	1275			27618	Google	Proprietary
🥈 GPT-4.1-2025-04-14	1384	1393	1276			13830	OpenAI	Proprietary
🥈 DeepSeek-V3-0324	1382	1397		85.5	88.5	16550	DeepSeek	MIT
🥈 Claude Opus 4 (20250514)	1373	1412	1232			13850	Anthropic	Proprietary
🥈 Hunyuan-Turbos-20250416	1372	1383				5944	Tencent	Proprietary
🥈 DeepSeek-R1	1371	1379		93.2	90.8	19430	DeepSeek	MIT
🥈 Mistral Medium 3	1363	1385				12003	Mistral	Proprietary
🥈 o1-2024-12-17	1363	1374	1229	92.1	91.8	29038	OpenAI	Proprietary
🥈 Gemini-2.0-Flash-001	1362	1362	1218			34240	Google	Proprietary
🥈 Grok-3-Mini-beta	1361	1388				6636	xAI	Proprietary
🥈 o4-mini-2025-04-16	1361	1381	1253			13554	OpenAI	Proprietary
🥈 Qwen3-235B-A22B	1360	1385		95.6	88.5	10677	Alibaba	Apache 2.0
🥈 Qwen2.5-Max	1358	1365				29484	Alibaba	Proprietary
🥈 Gemma-3-27B-it	1355	1335				20295	Google	Gemma
🥈 Claude Sonnet 4 (20250514)	1345	1384	1220			10740	Anthropic	Proprietary
🥈 o3-mini-high	1338	1378				19404	OpenAI	Proprietary
🥈 GPT-4.1-mini-2025-04-14	1336	1373	1236			12702	OpenAI	Proprietary
🥉 Gemma-3-12B-it	1334	1306				3976	Google	Gemma
🥉 DeepSeek-V3	1332	1335		85.5	88.5	22841	DeepSeek	DeepSeek
🥉 Amazon-Nova-Experimental-Chat-05-14	1330	1345				2595	Amazon	Proprietary
🥉 QwQ-32B	1330	1344				15930	Alibaba	Apache 2.0
🥉 Gemini-2.0-Flash-Lite	1326	1336	1157			26104	Google	Proprietary
🥉 Qwen-Plus-0125	1324	1335				6055	Alibaba	Proprietary
🥉 GLM-4-Plus-0111	1324	1305				6028	Zhipu	Proprietary
🥉 Command A (03-2025)	1323	1333				20084	Cohere	CC-BY-NC-4.0
🥉 o3-mini	1319	1362				32421	OpenAI	Proprietary
🥉 Step-2-16K-Exp	1318	1310				5126	StepFun	Proprietary
🥉 o1-mini	1317	1368		92		54951	OpenAI	Proprietary
🥉 Gemini-1.5-Pro-002	1316	1306	1222			58645	Google	Proprietary
🥉 Claude 3.7 Sonnet (thinking-32k)	1313	1347	1225			21310	Anthropic	Proprietary
🥉 Hunyuan-Turbo-0110	1310	1331				2510	Tencent	Proprietary
🥉 Llama-3.3-Nemotron-Super-49B-v1	1310	1316		88.3	86	2371	Nvidia	Nvidia
🥉 Claude 3.7 Sonnet	1306	1343	1206			25983	Anthropic	Proprietary
🥉 Yi-Lightning	1301	1318				28968	01 AI	Proprietary
🥉 Grok-2-08-13	1301	1297			87.5	67084	xAI	Proprietary
🥉 Gemma-3n-e4b-it	1301	1278				3913	Google	Gemma
🥉 GPT-4o-2024-05-13	1298	1308	1206	79.21	88.7	117747	OpenAI	Proprietary
🥉 Claude 3.5 Sonnet (20241022)	1297	1341	1187	85.2	88.7	73327	Anthropic	Proprietary
Deepseek-v2.5-1210	1293	1312				7243	DeepSeek	DeepSeek
Athene-v2-Chat-72B	1289	1315		85		26074	NexusFlow	NexusFlow
Gemma-3-4B-it	1289	1262				4321	Google	Gemma
Llama-4-Maverick-17B-128E-Instruct	1287	1306	1155			13750	Meta	Llama 4
Hunyuan-Large-2025-02-10	1285	1308				3856	Tencent	Proprietary
GPT-4o-mini-2024-07-18	1285	1298	1123	74.94	82	72536	OpenAI	Proprietary
Gemini-1.5-Flash-002	1285	1269	1206			37021	Google	Proprietary
GPT-4.1-nano-2025-04-14	1284	1309	1117			6302	OpenAI	Proprietary
Llama-3.1-405B-Instruct-bf16	1282	1295			88.6	43788	Meta	Llama 3.1
Llama-3.1-Nemotron-70B-Instruct	1282	1286		84.9		7577	Nvidia	Llama 3.1
Llama-3.1-405B-Instruct-fp8	1281	1291		69.3	88.6	63038	Meta	Llama 3.1
Grok-2-Mini-08-13	1280	1277				55442	xAI	Proprietary
Yi-Lightning-lite	1278	1282				17067	01 AI	Proprietary
Qwen-Max-0919	1277	1295				17432	Alibaba	Qwen
Hunyuan-Standard-2025-02-10	1274	1285				4014	Tencent	Proprietary
Qwen2.5-72B-Instruct	1271	1298		78		41519	Alibaba	Qwen
GPT-4-Turbo-2024-04-09	1270	1278	1151	82.63		102133	OpenAI	Proprietary
Llama-3.3-70B-Instruct	1270	1272				44800	Meta	Llama-3.3
Athene-70B	1264	1268		77.6		20580	NexusFlow	CC-BY-NC-4.0
Mistral-Small-3.1-24B-Instruct-2503	1263	1282				2484	Mistral	Apache 2.0
GPT-4-1106-preview	1263	1268				103748	OpenAI	Proprietary
Mistral-Large-2411	1262	1281		70.42		29633	Mistral	MRL
Llama-3.1-70B-Instruct	1261	1266		55.73	86	58637	Meta	Llama 3.1
Claude 3 Opus	1261	1265	1076	60.36	86.8	202641	Anthropic	Proprietary
Amazon Nova Pro 1.0	1258	1278	1044			26371	Amazon	Proprietary
GPT-4-0125-preview	1258	1258		77.96		97079	OpenAI	Proprietary
Llama-3.1-Tulu-3-70B	1258	1248				3010	Ai2	Llama 3.1
Claude 3.5 Haiku (20241022)	1251	1283	1157			44893	Anthropic	Propretary
Reka-Core-20240904	1249	1236				7948	Reka AI	Proprietary
Gemini-1.5-Flash-001	1240	1247	1072	49.61	78.9	65661	Google	Proprietary
Jamba-1.5-Large	1235	1242			81.2	9125	AI21 Labs	Jamba Open
Deepseek-v2-API-0628	1233	1257				19508	DeepSeek AI	DeepSeek
Gemma-2-27B-it	1233	1224		57.51		79538	Google	Gemma license
Qwen2.5-Coder-32B-Instruct	1231	1276				5730	Alibaba	Apache 2.0
Mistral-Small-24B-Instruct-2501	1231	1247				15321	Mistral	Apache 2.0
Amazon Nova Lite 1.0	1230	1250	1061			20646	Amazon	Proprietary
Gemma-2-9B-it-SimPO	1230	1211				10548	Princeton	MIT
Command R+ (08-2024)	1229	1196				10535	Cohere	CC-BY-NC-4.0
Deepseek-Coder-v2-0724	1228	1282		62.3		11725	DeepSeek	Proprietary
Gemini-1.5-Flash-8B-001	1226	1223	1107			37697	Google	Proprietary
Llama-3.1-Nemotron-51B-Instruct	1225	1226				3889	Nvidia	Llama 3.1
Nemotron-4-340B-Instruct	1223	1213				20608	Nvidia	Nvidia
Aya-Expanse-32B	1223	1208				28768	Cohere	CC-BY-NC-4.0
GLM-4-0520	1220	1231		63.84		10221	Zhipu AI	Proprietary
Llama-3-70B-Instruct	1220	1215		46.57	82	163629	Meta	Llama 3
Phi-4	1219	1237				25213	Microsoft	MIT
OLMo-2-0325-32B-Instruct	1219	1211				3460	Allen AI	Apache-2.0
Reka-Flash-20240904	1219	1206				8132	Reka AI	Proprietary
Claude 3 Sonnet	1214	1228	1048	46.8	79	113067	Anthropic	Proprietary
Amazon Nova Micro 1.0	1211	1225				20654	Amazon	Proprietary
Gemma-2-9B-it	1206	1188				57197	Google	Gemma license
Hunyuan-Standard-256K	1202	1242				2901	Tencent	Proprietary
Qwen2-72B-Instruct	1201	1202		46.86	84.2	38872	Alibaba	Qianwen LICENSE
GPT-4-0314	1200	1210		50	86.4	55962	OpenAI	Proprietary
Llama-3.1-Tulu-3-8B	1199	1194				3074	Ai2	Llama 3.1
Ministral-8B-2410	1196	1216				5111	Mistral	MRL
Claude 3 Haiku	1193	1204	1000	41.47	75.2	122309	Anthropic	Proprietary
Aya-Expanse-8B	1193	1180				10391	Cohere	CC-BY-NC-4.0
Command R (08-2024)	1193	1176				10851	Cohere	CC-BY-NC-4.0
DeepSeek-Coder-V2-Instruct	1192	1254				15753	DeepSeek AI	DeepSeek License
Llama-3.1-8B-Instruct	1189	1201		21.34	73	52578	Meta	Llama 3.1
Jamba-1.5-Mini	1189	1196			69.7	9274	AI21 Labs	Jamba Open
GPT-4-0613	1177	1182		37.9		91614	OpenAI	Proprietary
Qwen1.5-110B-Chat	1175	1190			80.4	27430	Alibaba	Qianwen LICENSE
Yi-1.5-34B-Chat	1171	1177			76.8	25135	01 AI	Apache-2.0
Llama-3-8B-Instruct	1165	1161		20.56	68.4	109056	Meta	Llama 3
InternLM2.5-20B-chat	1162	1173				10599	InternLM	Other
Claude-1	1162	1151			77	21149	Anthropic	Proprietary
Qwen1.5-72B-Chat	1161	1175		36.12	77.5	40658	Alibaba	Qianwen LICENSE
Mixtral-8x22b-Instruct-v0.1	1161	1168		36.36	77.8	53751	Mistral	Apache 2.0
Mistral Medium	1161	1168		31.9	75.3	35556	Mistral	Proprietary
Gemma-2-2b-it	1157	1122			51.3	48892	Google	Gemma license
Granite-3.1-8B-Instruct	1156	1188				3289	IBM	Apache 2.0
Claude-2.0	1145	1150		23.99	78.5	12763	Anthropic	Proprietary
Gemini-1.0-Pro-001	1145	1118			71.8	18800	Google	Proprietary
Zephyr-ORPO-141b-A35b-v0.1	1141	1139				4854	HuggingFace	Apache 2.0
Qwen1.5-32B-Chat	1139	1164			73.4	22765	Alibaba	Qianwen LICENSE
Phi-3-Medium-4k-Instruct	1136	1140		33.37	78	26105	Microsoft	MIT
Granite-3.1-2B-Instruct	1133	1163				3380	IBM	Apache 2.0
Claude-2.1	1132	1147		22.77		37699	Anthropic	Proprietary
Starling-LM-7B-beta	1132	1144		23.01		16676	Nexusflow	Apache-2.0
GPT-3.5-Turbo-0613	1130	1150		24.82		38955	OpenAI	Proprietary
Mixtral-8x7B-Instruct-v0.1	1128	1129		23.4	70.6	76126	Mistral	Apache 2.0
Claude-Instant-1	1125	1124			73.4	20631	Anthropic	Proprietary
Yi-34B-Chat	1125	1121		23.15	73.5	15917	01 AI	Yi License
Qwen1.5-14B-Chat	1122	1141			67.6	18687	Alibaba	Qianwen LICENSE
GPT-3.5-Turbo-0314	1120	1130		18.05	70	5640	OpenAI	Proprietary
WizardLM-70B-v1.0	1120	1086			63.7	8383	Microsoft	Llama 2
GPT-3.5-Turbo-0125	1119	1139		23.34		68867	OpenAI	Proprietary
DBRX-Instruct-Preview	1117	1133		24.63	73.7	33743	Databricks	DBRX LICENSE
Phi-3-Small-8k-Instruct	1116	1122		29.77	75.7	18476	Microsoft	MIT
Llama-3.2-3B-Instruct	1116	1095				8390	Meta	Llama 3.2
Tulu-2-DPO-70B	1113	1108		14.99		6658	AllenAI/UW	AI2 ImpACT Low-risk
Granite-3.0-8B-Instruct	1107	1112				7002	IBM	Apache 2.0
Llama-2-70B-chat	1106	1087		11.55	63	39595	Meta	Llama 2
OpenChat-3.5-0106	1105	1117			65.8	12990	OpenChat	Apache-2.0
Vicuna-33B	1104	1082		8.63	59.2	22936	LMSYS	Non-commercial
Snowflake Arctic Instruct	1103	1092		17.61	67.3	34173	Snowflake	Apache 2.0
Starling-LM-7B-alpha	1102	1095		12.8	63.9	10415	UC Berkeley	CC-BY-NC-4.0
Nous-Hermes-2-Mixtral-8x7B-DPO	1098	1094				3836	NousResearch	Apache-2.0
Gemma-1.1-7B-it	1097	1099		12.09	64.3	25070	Google	Gemma license
NV-Llama2-70B-SteerLM-Chat	1094	1038			68.5	3636	Nvidia	Llama 2
pplx-70B-online	1091	1043				6898	Perplexity AI	Proprietary
DeepSeek-LLM-67B-Chat	1090	1094			71.3	4988	DeepSeek AI	DeepSeek License
OpenChat-3.5	1090	1069			64.3	8106	OpenChat	Apache-2.0
OpenHermes-2.5-Mistral-7B	1088	1073				5088	NousResearch	Apache-2.0
Granite-3.0-2B-Instruct	1087	1103				7191	IBM	Apache 2.0
Mistral-7B-Instruct-v0.2	1086	1089		12.57		20067	Mistral	Apache-2.0
Phi-3-Mini-4K-Instruct-June-24	1084	1097			70.9	12808	Microsoft	MIT
Qwen1.5-7B-Chat	1083	1104			61	4872	Alibaba	Qianwen LICENSE
GPT-3.5-Turbo-1106	1081	1110		18.87		17036	OpenAI	Proprietary
Phi-3-Mini-4k-Instruct	1080	1101			68.8	21097	Microsoft	MIT
Llama-2-13b-chat	1077	1066			53.6	19722	Meta	Llama 2
SOLAR-10.7B-Instruct-v1.0	1076	1062			66.2	4286	Upstage AI	CC-BY-NC-4.0
Dolphin-2.2.1-Mistral-7B	1076	1040				1714	Cognitive Computations	Apache-2.0
WizardLM-13b-v1.2	1072	1041			52.7	7176	Microsoft	Llama 2
Llama-3.2-1B-Instruct	1067	1061				8523	Meta	Llama 3.2
Qwen2.5-VL-32B-Instruct			1212			1505	Alibaba	Apache 2.0
Step-1o-Vision-32k (highres)			1186			2891	StepFun	Proprietary
Qwen2.5-VL-72B-Instruct			1169			3884	Alibaba	Qwen
Pixtral-Large-2411			1153			5546	Mistral	MRL
Llama-4-Scout-17B-16E-Instruct			1150			1029	Meta	Llama
Qwen-VL-Max-1119			1128			1449	Alibaba	Proprietary
Qwen2-VL-72b-Instruct			1111			6028	Alibaba	Qwen
Step-1V-32K			1111			1553	StepFun	Proprietary
Molmo-72B-0924			1076			3092	AI2	Apache 2.0
Pixtral-12B-2409			1072			7623	Mistral	Apache 2.0
Llama-3.2-90B-Vision-Instruct			1070			8829	Meta	Llama 3.2
InternVL2-26B			1067			5265	OpenGVLab	MIT
Hunyuan-Standard-Vision-2024-12-31			1064			811	Tencent	Proprietary
Aya-Vision-32B			1058			849	Cohere	CC-BY-NC-4.0
Qwen2-VL-7B-Instruct			1054			5854	Aliaba	Apache 2.0
Yi-Vision			1045			1237	01 AI	Proprietary
Llama-3.2-11B-Vision-Instruct			1032			4893	Meta	Llama 3.2

If you want to see more models, please help us add them.

💻 Code: The Arena Elo ratings are computed by this notebook. The MT-bench scores (single-answer grading on a scale of 10) are computed by fastchat.llm_judge. The MMLU scores are computed by InstructEval. Higher values are better for all benchmarks. Empty cells mean not available. The latest and detailed leaderboard is here.

More Statistics for Chatbot Arena

🔗 Arena Statistics

Transition from online Elo rating system to Bradley-Terry model

We adopted the Elo rating system for ranking models since the launch of the Arena. It has been useful to transform pairwise human preference to Elo ratings that serve as a predictor of winrate between models. Specifically, if player A has a rating of R_A and player B a rating of R_B, the probability of player A winning is

ELO rating has been used to rank chess players by the international community for over 60 years. Standard Elo rating systems assume a player’s performance changes overtime. So an online algorithm is needed to capture such dynamics, meaning recent games should weigh more than older games. Specifically, after each game, a player’s rating is updated according to the difference between predicted outcome and actual outcome.

This algorithm has two distinct features:

1. It can be computed asynchronously by players around the world.
2. It allows for players performance to change dynamically – it does not assume a fixed unknown value for the players rating.

This ability to adapt is determined by the parameter K which controls the magnitude of rating changes that can affect the overall result. A larger K essentially put more weight on the recent games, which may make sense for new players whose performance improves quickly. However as players become more senior and their performance “converges” then a smaller value of K is more appropriate. As a result, USCF adopted K based on the number of games and tournaments completed by the player (reference). That is, the Elo rating of a senior player changes slower than a new player.

When we launched the Arena, we noticed considerable variability in the ratings using the classic online algorithm. We tried to tune the K to be sufficiently stable while also allowing new models to move up quickly in the leaderboard. We ultimately decided to adopt a bootstrap-like technique to shuffle the data and sample Elo scores from 1000 permutations of the online plays. You can find the details in this notebook. This provided consistent stable scores and allowed us to incorporate new models quickly. This is also observed in a recent work by Cohere. However, we used the same samples to estimate confidence intervals which were therefore too wide (effectively CI’s for the original online Elo estimates).

In the context of LLM ranking, there are two important differences from the classic Elo chess ranking system. First, we have access to the entire history of all games for all models and so we don’t need a decentralized algorithm. Second, most models are static (we have access to the weights) and so we don’t expect their performance to change. However, it is worth noting that the hosted proprietary models may not be static and their behavior can change without notice. We try our best to pin specific model API versions if possible.

To improve the quality of our rankings and their confidence estimates, we are adopting another widely used rating system called the Bradley–Terry (BT) model. This model actually is the maximum likelihood (MLE) estimate of the underlying Elo model assuming a fixed but unknown pairwise win-rate. Similar to Elo rating, BT model is also based on pairwise comparison to derive ratings of players to estimate win rate between each other. The core difference between BT model vs the online Elo system is the assumption that player’s performance does not change (i.e., game order does not matter) and the computation takes place in a centralized fashion.