Chatbot Arena

This leaderboard is based on the following benchmarks.

• Chatbot Arena - a crowdsourced, randomized battle platform for large language models (LLMs). We use 2.9M+ user votes to compute Elo ratings.
• MMLU - a test to measure a model’s multitask accuracy on 57 tasks.
• Arena-Hard-Auto - an automatic evaluation tool for instruction-tuned LLMs.

| Vote | Blog | GitHub | Paper | Dataset | Twitter | Discord |

Best Open LM

Model	Arena Elo	MMLU	License
DeepSeek-V3-0324	1372	88.5	MIT
DeepSeek-R1	1358	90.8	MIT
Qwen3-235B-A22B	1342	88.5	Apache 2.0
Gemma-3-27B-it	1341		Gemma
Gemma-3-12B-it	1321		Gemma
QwQ-32B	1313		Apache 2.0
Llama-3.3-Nemotron-Super-49B-v1	1296	86	Nvidia

Full Leaderboard

Model	Arena Elo	Coding	Vision	Arena Hard	MMLU	Votes	Organization	License
🥇 Gemini-2.5-Pro-Exp-03-25	1437	1430	1328	96.4		12541	Google	Proprietary
🥇 o3-2025-04-16	1411	1426	1305			5714	OpenAI	Proprietary
🥇 ChatGPT-4o-latest (2025-03-26)	1408	1407	1311			10290	OpenAI	Proprietary
🥇 Grok-3-Preview-02-24	1402	1408			92.7	14845	xAI	Proprietary
🥇 GPT-4.5-Preview	1398	1400	1256			15276	OpenAI	Proprietary
🥇 Gemini-2.5-Flash-Preview-04-17	1393	1401	1274			5303	Google	Proprietary
🥈 Gemini-2.0-Pro-Exp-02-05	1380	1379	1240			20120	Google	Proprietary
🥈 Gemini-2.0-Flash-Thinking-Exp-01-21	1380	1364	1278			26892	Google	Proprietary
🥈 DeepSeek-V3-0324	1372	1393			88.5	8127	DeepSeek	MIT
🥈 GPT-4.1-2025-04-14	1367	1372	1282			4432	OpenAI	Proprietary
🥈 DeepSeek-R1	1358	1365		93.2	90.8	17957	DeepSeek	MIT
🥈 Gemini-2.0-Flash-001	1355	1353	1224			24289	Google	Proprietary
🥈 o4-mini-2025-04-16	1352	1372	1264			4412	OpenAI	Proprietary
🥈 o1-2024-12-17	1350	1358	1231	92.1	91.8	29042	OpenAI	Proprietary
🥈 Qwen3-235B-A22B	1342	1365		95.6	88.5	2680	Alibaba	Apache 2.0
🥈 Qwen2.5-Max	1341	1342				23169	Alibaba	Proprietary
🥈 Gemma-3-27B-it	1341	1313				11801	Google	Gemma
🥈 o3-mini-high	1325	1362				19405	OpenAI	Proprietary
🥈 GPT-4.1-mini-2025-04-14	1323	1361	1233			4293	OpenAI	Proprietary
🥉 Gemma-3-12B-it	1321	1286				2466	Google	Gemma
🥉 DeepSeek-V3	1318	1320		85.5	88.5	22833	DeepSeek	DeepSeek
🥉 QwQ-32B	1313	1317				9719	Alibaba	Apache 2.0
🥉 Gemini-2.0-Flash-Lite	1312	1320	1159			24427	Google	Proprietary
🥉 GLM-4-Plus-0111	1311	1290				6027	Zhipu	Proprietary
🥉 Qwen-Plus-0125	1310	1320				6062	Alibaba	Proprietary
🥉 o3-mini	1305	1349				24925	OpenAI	Proprietary
🥉 Command A (03-2025)	1305	1310				9885	Cohere	CC-BY-NC-4.0
🥉 Step-2-16K-Exp	1305	1295				5126	StepFun	Proprietary
🥉 o1-mini	1304	1353		92		54952	OpenAI	Proprietary
🥉 Hunyuan-TurboS-20250226	1302	1326				2450	Tencent	Proprietary
🥉 Gemini-1.5-Pro-002	1302	1291	1222			58642	Google	Proprietary
🥉 Claude 3.7 Sonnet (thinking-32k)	1301	1339				11345	Anthropic	Proprietary
🥉 Hunyuan-Turbo-0110	1296	1315				2515	Tencent	Proprietary
🥉 Llama-3.3-Nemotron-Super-49B-v1	1296	1299			86	2368	Nvidia	Nvidia
🥉 Claude 3.7 Sonnet	1291	1328	1208			16696	Anthropic	Proprietary
🥉 Grok-2-08-13	1288	1282			87.5	67085	xAI	Proprietary
🥉 Yi-Lightning	1287	1302		81.5		28970	01 AI	Proprietary
🥉 GPT-4o-2024-05-13	1285	1293	1206	79.21	88.7	117742	OpenAI	Proprietary
🥉 Claude 3.5 Sonnet (20241022)	1283	1326	1183	85.2	88.7	65446	Anthropic	Proprietary
Deepseek-v2.5-1210	1279	1297				7243	DeepSeek	DeepSeek
Athene-v2-Chat-72B	1275	1300		85		26078	NexusFlow	NexusFlow
Hunyuan-Large-2025-02-10	1272	1293				3858	Tencent	Proprietary
GPT-4o-mini-2024-07-18	1272	1283	1124	74.94	82	71373	OpenAI	Proprietary
Gemma-3-4B-it	1272	1254				2773	Google	Gemma
GPT-4.1-nano-2025-04-14	1271	1288	1123			4452	OpenAI	Proprietary
Gemini-1.5-Flash-002	1271	1254	1206			37025	Google	Proprietary
Llama-4-Maverick-17B-128E-Instruct	1270	1286				7121	Meta	Llama 4
Llama-3.1-405B-Instruct-bf16	1269	1280			88.6	43791	Meta	Llama 3.1
Llama-3.1-Nemotron-70B-Instruct	1269	1271		84.9		7577	Nvidia	Llama 3.1
Llama-3.1-405B-Instruct-fp8	1267	1276		69.3	88.6	63035	Meta	Llama 3.1
Grok-2-Mini-08-13	1266	1262				55435	xAI	Proprietary
Yi-Lightning-lite	1264	1267				17071	01 AI	Proprietary
Hunyuan-Standard-2025-02-10	1261	1269				4017	Tencent	Proprietary
Qwen2.5-72B-Instruct	1257	1283		78		41515	Alibaba	Qwen
Llama-3.3-70B-Instruct	1257	1258				38102	Meta	Llama-3.3
GPT-4-Turbo-2024-04-09	1256	1263	1151	82.63		102148	OpenAI	Proprietary
Mistral-Large-2407	1251	1269		70.42		48216	Mistral	Mistral Research
GPT-4-1106-preview	1250	1253				103748	OpenAI	Proprietary
Athene-70B	1250	1253		77.6		20577	NexusFlow	CC-BY-NC-4.0
Mistral-Large-2411	1249	1265				29655	Mistral	MRL
Llama-3.1-70B-Instruct	1248	1251		55.73	86	58643	Meta	Llama 3.1
Claude 3 Opus	1247	1250	1076	60.36	86.8	202653	Anthropic	Proprietary
Amazon Nova Pro 1.0	1245	1262	1044			25484	Amazon	Proprietary
GPT-4-0125-preview	1245	1243		77.96		97072	OpenAI	Proprietary
Llama-3.1-Tulu-3-70B	1244	1233				3009	Ai2	Llama 3.1
Yi-Large-preview	1240	1245		71.48		51637	01 AI	Proprietary
Claude 3.5 Haiku (20241022)	1237	1265				35694	Anthropic	Propretary
Reka-Core-20240904	1235	1221				7946	Reka AI	Proprietary
Reka-Core-20240722	1231	1208				13280	Reka AI	Proprietary
Qwen-Plus-0828	1227	1245				14624	Alibaba	Proprietary
Gemini-1.5-Flash-001	1227	1232	1072	49.61	78.9	65665	Google	Proprietary
Jamba-1.5-Large	1222	1227			81.2	9127	AI21 Labs	Jamba Open
Deepseek-v2-API-0628	1220	1242				19510	DeepSeek AI	DeepSeek
Gemma-2-27B-it	1220	1209		57.51		79526	Google	Gemma license
Qwen2.5-Coder-32B-Instruct	1217	1261				5734	Alibaba	Apache 2.0
Amazon Nova Lite 1.0	1217	1235	1061			20654	Amazon	Proprietary
Mistral-Small-24B-Instruct-2501	1217	1231				15317	Mistral	Apache 2.0
Gemma-2-9B-it-SimPO	1216	1196				10551	Princeton	MIT
Command R+ (08-2024)	1215	1181				10538	Cohere	CC-BY-NC-4.0
Deepseek-Coder-v2-0724	1214	1266		62.3		11726	DeepSeek	Proprietary
Yi-Large	1213	1220		63.7		16624	01 AI	Proprietary
Gemini-1.5-Flash-8B-001	1213	1208	1106			37696	Google	Proprietary
Llama-3.1-Nemotron-51B-Instruct	1211	1211				3888	Nvidia	Llama 3.1
Nemotron-4-340B-Instruct	1209	1198				20615	Nvidia	NVIDIA Open Model
Aya-Expanse-32B	1209	1193				28753	Cohere	CC-BY-NC-4.0
GLM-4-0520	1207	1216		63.84		10221	Zhipu AI	Proprietary
Llama-3-70B-Instruct	1207	1199		46.57	82	163650	Meta	Llama 3
Reka-Flash-20240904	1206	1191				8136	Reka AI	Proprietary
Phi-4	1205	1221				25224	Microsoft	MIT
OLMo-2-0325-32B-Instruct	1205	1197				2775	Allen AI	Apache-2.0
Claude 3 Sonnet	1201	1213	1048	46.8	79	113062	Anthropic	Proprietary
Reka-Flash-20240722	1201	1187				13729	Reka AI	Proprietary
Reka-Core-20240501	1199	1190	1015		83.2	62563	Reka AI	Proprietary
Amazon Nova Micro 1.0	1198	1210				20650	Amazon	Proprietary
Gemma-2-9B-it	1192	1173				57212	Google	Gemma license
Command R+ (04-2024)	1190	1164		33.07		80856	Cohere	CC-BY-NC-4.0
Hunyuan-Standard-256K	1189	1227				2900	Tencent	Proprietary
Qwen2-72B-Instruct	1187	1187		46.86	84.2	38871	Alibaba	Qianwen LICENSE
GPT-4-0314	1186	1195		50	86.4	55974	OpenAI	Proprietary
Llama-3.1-Tulu-3-8B	1185	1179				3074	Ai2	Llama 3.1
GLM-4-0116	1183	1191		55.72		7578	Zhipu AI	Proprietary
Qwen-Max-0428	1183	1189				25697	Alibaba	Proprietary
Ministral-8B-2410	1182	1201				5109	Mistral	MRL
Aya-Expanse-8B	1180	1165				10397	Cohere	CC-BY-NC-4.0
Command R (08-2024)	1180	1161				10850	Cohere	CC-BY-NC-4.0
Claude 3 Haiku	1179	1189	1000	41.47	75.2	122319	Anthropic	Proprietary
DeepSeek-Coder-V2-Instruct	1178	1239				15754	DeepSeek AI	DeepSeek License
Llama-3.1-8B-Instruct	1176	1186		21.34	73	52581	Meta	Llama 3.1
Jamba-1.5-Mini	1176	1181			69.7	9272	AI21 Labs	Jamba Open
GPT-4-0613	1163	1167		37.9		91624	OpenAI	Proprietary
Qwen1.5-110B-Chat	1161	1175			80.4	27445	Alibaba	Qianwen LICENSE
Mistral-Large-2402	1157	1170		37.71	81.2	64909	Mistral	Proprietary
Yi-1.5-34B-Chat	1157	1162			76.8	25134	01 AI	Apache-2.0
Reka-Flash-21B-online	1156	1147				16029	Reka AI	Proprietary
Llama-3-8B-Instruct	1152	1146		20.56	68.4	109100	Meta	Llama 3
InternLM2.5-20B-chat	1149	1158				10593	InternLM	Other
Claude-1	1149	1136			77	21150	Anthropic	Proprietary
Command R (04-2024)	1149	1123		17.02		56388	Cohere	CC-BY-NC-4.0
Mixtral-8x22b-Instruct-v0.1	1148	1153		36.36	77.8	53753	Mistral	Apache 2.0
Mistral Medium	1148	1152		31.9	75.3	35567	Mistral	Proprietary
Qwen1.5-72B-Chat	1147	1160		36.12	77.5	40660	Alibaba	Qianwen LICENSE
Reka-Flash-21B	1147	1141			73.5	25804	Reka AI	Proprietary
Gemma-2-2b-it	1144	1107			51.3	48893	Google	Gemma license
Granite-3.1-8B-Instruct	1143	1173				3294	IBM	Apache 2.0
Claude-2.0	1132	1135		23.99	78.5	12759	Anthropic	Proprietary
Gemini-1.0-Pro-001	1131	1103			71.8	18802	Google	Proprietary
Zephyr-ORPO-141b-A35b-v0.1	1127	1124				4855	HuggingFace	Apache 2.0
Qwen1.5-32B-Chat	1125	1149			73.4	22766	Alibaba	Qianwen LICENSE
Mistral-Next	1124	1132		27.37		12376	Mistral	Proprietary
Phi-3-Medium-4k-Instruct	1123	1125		33.37	78	26112	Microsoft	MIT
Granite-3.1-2B-Instruct	1119	1147				3380	IBM	Apache 2.0
Starling-LM-7B-beta	1119	1129		23.01		16669	Nexusflow	Apache-2.0
Claude-2.1	1118	1132		22.77		37697	Anthropic	Proprietary
GPT-3.5-Turbo-0613	1117	1135		24.82		38955	OpenAI	Proprietary
Mixtral-8x7B-Instruct-v0.1	1114	1114		23.4	70.6	76141	Mistral	Apache 2.0
Claude-Instant-1	1111	1109			73.4	20626	Anthropic	Proprietary
Yi-34B-Chat	1111	1106		23.15	73.5	15918	01 AI	Yi License
Gemini Pro	1111	1091		17.8	71.8	6556	Google	Proprietary
Qwen1.5-14B-Chat	1109	1126			67.6	18688	Alibaba	Qianwen LICENSE
GPT-3.5-Turbo-0314	1107	1115		18.05	70	5641	OpenAI	Proprietary
GPT-3.5-Turbo-0125	1106	1124		23.34		68858	OpenAI	Proprietary
WizardLM-70B-v1.0	1106	1071			63.7	8385	Microsoft	Llama 2
DBRX-Instruct-Preview	1103	1118		24.63	73.7	33738	Databricks	DBRX LICENSE
Llama-3.2-3B-Instruct	1103	1080				8395	Meta	Llama 3.2
Phi-3-Small-8k-Instruct	1102	1107		29.77	75.7	18473	Microsoft	MIT
Tulu-2-DPO-70B	1099	1093		14.99		6659	AllenAI/UW	AI2 ImpACT Low-risk
Granite-3.0-8B-Instruct	1093	1097				7002	IBM	Apache 2.0
Llama-2-70B-chat	1093	1072		11.55	63	39609	Meta	Llama 2
OpenChat-3.5-0106	1092	1102			65.8	12991	OpenChat	Apache-2.0
Vicuna-33B	1091	1067		8.63	59.2	22938	LMSYS	Non-commercial
Snowflake Arctic Instruct	1090	1077		17.61	67.3	34177	Snowflake	Apache 2.0
Starling-LM-7B-alpha	1088	1080		12.8	63.9	10416	UC Berkeley	CC-BY-NC-4.0
Gemma-1.1-7B-it	1084	1084		12.09	64.3	25066	Google	Gemma license
Nous-Hermes-2-Mixtral-8x7B-DPO	1084	1079				3836	NousResearch	Apache-2.0
NV-Llama2-70B-SteerLM-Chat	1080	1023			68.5	3635	Nvidia	Llama 2
pplx-70B-online	1078	1028				6898	Perplexity AI	Proprietary
DeepSeek-LLM-67B-Chat	1077	1079			71.3	4988	DeepSeek AI	DeepSeek License
OpenChat-3.5	1076	1054			64.3	8107	OpenChat	Apache-2.0
Granite-3.0-2B-Instruct	1074	1088				7187	IBM	Apache 2.0
OpenHermes-2.5-Mistral-7B	1074	1058				5087	NousResearch	Apache-2.0
Mistral-7B-Instruct-v0.2	1072	1073		12.57		20061	Mistral	Apache-2.0
Phi-3-Mini-4K-Instruct-June-24	1071	1082			70.9	12803	Microsoft	MIT
Qwen1.5-7B-Chat	1070	1089			61	4873	Alibaba	Qianwen LICENSE
GPT-3.5-Turbo-1106	1068	1095		18.87		17038	OpenAI	Proprietary
Phi-3-Mini-4k-Instruct	1066	1086			68.8	21091	Microsoft	MIT
Llama-2-13b-chat	1063	1051			53.6	19715	Meta	Llama 2
Dolphin-2.2.1-Mistral-7B	1063	1024				1713	Cognitive Computations	Apache-2.0
SOLAR-10.7B-Instruct-v1.0	1062	1047			66.2	4288	Upstage AI	CC-BY-NC-4.0
WizardLM-13b-v1.2	1059	1026			52.7	7176	Microsoft	Llama 2
Llama-3.2-1B-Instruct	1054	1046				8519	Meta	Llama 3.2
Qwen2.5-VL-32B-Instruct			1212				Alibaba	Apache 2.0
Step-1o-Vision-32k (highres)			1187				StepFun	Proprietary
Qwen2.5-VL-72B-Instruct			1171				Alibaba	Qwen
Pixtral-Large-2411			1154				Mistral	MRL
Qwen-VL-Max-1119			1128				Alibaba	Proprietary
Qwen2-VL-72b-Instruct			1111				Alibaba	Qwen
Step-1V-32K			1111				StepFun	Proprietary
Molmo-72B-0924			1076				AI2	Apache 2.0
Pixtral-12B-2409			1072				Mistral	Apache 2.0
Llama-3.2-90B-Vision-Instruct			1070				Meta	Llama 3.2
Aya-Vision-8B			1069				Cohere	CC-BY-NC-4.0
InternVL2-26B			1067				OpenGVLab	MIT
Hunyuan-Standard-Vision-2024-12-31			1066				Tencent	Proprietary
Aya-Vision-32B			1060				Cohere	CC-BY-NC-4.0
Qwen2-VL-7B-Instruct			1054				Aliaba	Apache 2.0
Yi-Vision			1045				01 AI	Proprietary
Llama-3.2-11B-Vision-Instruct			1032				Meta	Llama 3.2

If you want to see more models, please help us add them.

💻 Code: The Arena Elo ratings are computed by this notebook. The MT-bench scores (single-answer grading on a scale of 10) are computed by fastchat.llm_judge. The MMLU scores are computed by InstructEval. Higher values are better for all benchmarks. Empty cells mean not available. The latest and detailed leaderboard is here.

More Statistics for Chatbot Arena

🔗 Arena Statistics

Transition from online Elo rating system to Bradley-Terry model

We adopted the Elo rating system for ranking models since the launch of the Arena. It has been useful to transform pairwise human preference to Elo ratings that serve as a predictor of winrate between models. Specifically, if player A has a rating of R_A and player B a rating of R_B, the probability of player A winning is

ELO rating has been used to rank chess players by the international community for over 60 years. Standard Elo rating systems assume a player’s performance changes overtime. So an online algorithm is needed to capture such dynamics, meaning recent games should weigh more than older games. Specifically, after each game, a player’s rating is updated according to the difference between predicted outcome and actual outcome.

This algorithm has two distinct features:

1. It can be computed asynchronously by players around the world.
2. It allows for players performance to change dynamically – it does not assume a fixed unknown value for the players rating.

This ability to adapt is determined by the parameter K which controls the magnitude of rating changes that can affect the overall result. A larger K essentially put more weight on the recent games, which may make sense for new players whose performance improves quickly. However as players become more senior and their performance “converges” then a smaller value of K is more appropriate. As a result, USCF adopted K based on the number of games and tournaments completed by the player (reference). That is, the Elo rating of a senior player changes slower than a new player.

When we launched the Arena, we noticed considerable variability in the ratings using the classic online algorithm. We tried to tune the K to be sufficiently stable while also allowing new models to move up quickly in the leaderboard. We ultimately decided to adopt a bootstrap-like technique to shuffle the data and sample Elo scores from 1000 permutations of the online plays. You can find the details in this notebook. This provided consistent stable scores and allowed us to incorporate new models quickly. This is also observed in a recent work by Cohere. However, we used the same samples to estimate confidence intervals which were therefore too wide (effectively CI’s for the original online Elo estimates).

In the context of LLM ranking, there are two important differences from the classic Elo chess ranking system. First, we have access to the entire history of all games for all models and so we don’t need a decentralized algorithm. Second, most models are static (we have access to the weights) and so we don’t expect their performance to change. However, it is worth noting that the hosted proprietary models may not be static and their behavior can change without notice. We try our best to pin specific model API versions if possible.

To improve the quality of our rankings and their confidence estimates, we are adopting another widely used rating system called the Bradley–Terry (BT) model. This model actually is the maximum likelihood (MLE) estimate of the underlying Elo model assuming a fixed but unknown pairwise win-rate. Similar to Elo rating, BT model is also based on pairwise comparison to derive ratings of players to estimate win rate between each other. The core difference between BT model vs the online Elo system is the assumption that player’s performance does not change (i.e., game order does not matter) and the computation takes place in a centralized fashion.