Chatbot Arena

This leaderboard is based on the following benchmarks.

• Chatbot Arena - a crowdsourced, randomized battle platform for large language models (LLMs). We use 3M+ user votes to compute Elo ratings.
• MMLU - a test to measure a model’s multitask accuracy on 57 tasks.
• Arena-Hard-Auto - an automatic evaluation tool for instruction-tuned LLMs.

| Vote | Blog | GitHub | Paper | Dataset | Twitter | Discord |

Best Open LM

Model	Arena Elo	MMLU	License
DeepSeek-V3-0324	1368	88.5	MIT
DeepSeek-R1	1354	90.8	MIT
Gemma-3-27B-it	1339		Gemma
Qwen3-235B-A22B	1337	88.5	Apache 2.0
Qwen3-32B	1324		Apache 2.0

Full Leaderboard

Model	Arena Elo	Coding	Vision	Arena Hard	MMLU	Votes	Organization	License
🥇 Gemini-2.5-Pro-Preview-05-06	1446	1446	1375	96.4		6115	Google	Proprietary
🥇 Gemini-2.5-Flash-Preview-05-20	1418	1431	1318			3892	Google	Proprietary
🥇 o3-2025-04-16	1409	1419	1303			7921	OpenAI	Proprietary
🥇 ChatGPT-4o-latest (2025-03-26)	1405	1395	1310			10280	OpenAI	Proprietary
🥇 Grok-3-Preview-02-24	1399	1396			92.7	14840	xAI	Proprietary
🥇 GPT-4.5-Preview	1394	1389	1256			15276	OpenAI	Proprietary
🥈 Gemini-2.0-Flash-Thinking-Exp-01-21	1377	1353	1278			26885	Google	Proprietary
🥈 Gemini-2.0-Pro-Exp-02-05	1376	1368	1240			20124	Google	Proprietary
🥈 DeepSeek-V3-0324	1368	1382		85.5	88.5	9741	DeepSeek	MIT
🥈 GPT-4.1-2025-04-14	1365	1358	1283			6094	OpenAI	Proprietary
🥈 Hunyuan-Turbos-20250416	1356	1351				5111	Tencent	Proprietary
🥈 DeepSeek-R1	1354	1352		93.2	90.8	19339	DeepSeek	MIT
🥈 Gemini-2.0-Flash-001	1351	1341	1224			24928	Google	Proprietary
🥈 o1-2024-12-17	1346	1347	1231	92.1	91.8	29041	OpenAI	Proprietary
🥈 o4-mini-2025-04-16	1343	1357	1270			6102	OpenAI	Proprietary
🥈 Mistral Medium 3	1343	1346				3327	Mistral	Proprietary
🥈 Gemma-3-27B-it	1339	1304				12989	Google	Gemma
🥈 Qwen3-235B-A22B	1337	1351		95.6	88.5	4942	Alibaba	Apache 2.0
🥈 Qwen2.5-Max	1337	1331				23170	Alibaba	Proprietary
🥈 Qwen3-32B	1324	1345				3960	Alibaba	Apache 2.0
🥈 o3-mini-high	1321	1351				19403	OpenAI	Proprietary
🥈 GPT-4.1-mini-2025-04-14	1319	1349	1236			5929	OpenAI	Proprietary
🥉 Gemma-3-12B-it	1317	1278				3882	Google	Gemma
🥉 QwQ-32B	1310	1307				9936	Alibaba	Apache 2.0
🥉 Gemini-2.0-Flash-Lite	1309	1309	1159			25984	Google	Proprietary
🥉 Qwen-Plus-0125	1307	1309				6056	Alibaba	Proprietary
🥉 GLM-4-Plus-0111	1307	1279				6028	Zhipu	Proprietary
🥉 Command A (03-2025)	1303	1301				11530	Cohere	CC-BY-NC-4.0
🥉 o3-mini	1302	1338				24916	OpenAI	Proprietary
🥉 Step-2-16K-Exp	1301	1284				5128	StepFun	Proprietary
🥉 o1-mini	1300	1342		92		54953	OpenAI	Proprietary
🥉 Llama-3.1-Nemotron-Ultra-253B-v1	1300	1314				2658	Nvidia	Nvidia
🥉 Gemini-1.5-Pro-002	1299	1280	1222			58639	Google	Proprietary
🥉 Claude 3.7 Sonnet (thinking-32k)	1296	1324				13027	Anthropic	Proprietary
🥉 Hunyuan-Turbo-0110	1293	1304				2513	Tencent	Proprietary
🥉 Llama-3.3-Nemotron-Super-49B-v1	1293	1288		88.3	86	2368	Nvidia	Nvidia
🥉 Claude 3.7 Sonnet	1287	1315	1208			18395	Anthropic	Proprietary
🥉 Qwen3-30B-A3B	1286	1290				4159	Alibaba	Apache 2.0
🥉 Gemma-3n-e4b-it	1286	1255				3583	Google	Gemma
🥉 Yi-Lightning	1284	1292		81.5		28971	01 AI	Proprietary
🥉 Grok-2-08-13	1284	1271			87.5	67081	xAI	Proprietary
🥉 GPT-4o-2024-05-13	1281	1282	1206	79.21	88.7	117749	OpenAI	Proprietary
🥉 Claude 3.5 Sonnet (20241022)	1280	1315	1183	85.2	88.7	65436	Anthropic	Proprietary
Deepseek-v2.5-1210	1276	1286				7245	DeepSeek	DeepSeek
Athene-v2-Chat-72B	1272	1289		85		26074	NexusFlow	NexusFlow
Gemma-3-4B-it	1272	1233				4206	Google	Gemma
GPT-4.1-nano-2025-04-14	1269	1282	1123			6181	OpenAI	Proprietary
GPT-4o-mini-2024-07-18	1269	1272	1124	74.94	82	71356	OpenAI	Proprietary
Hunyuan-Large-2025-02-10	1268	1282				3858	Tencent	Proprietary
Gemini-1.5-Flash-002	1268	1243	1206			37020	Google	Proprietary
Llama-4-Maverick-17B-128E-Instruct	1266	1278	1181			8797	Meta	Llama 4
Llama-3.1-405B-Instruct-bf16	1265	1269			88.6	43790	Meta	Llama 3.1
Llama-3.1-Nemotron-70B-Instruct	1265	1260		84.9		7579	Nvidia	Llama 3.1
Llama-3.1-405B-Instruct-fp8	1264	1265		69.3	88.6	63035	Meta	Llama 3.1
Grok-2-Mini-08-13	1263	1251				55442	xAI	Proprietary
Yi-Lightning-lite	1261	1256				17066	01 AI	Proprietary
Hunyuan-Standard-2025-02-10	1257	1259				4013	Tencent	Proprietary
Qwen2.5-72B-Instruct	1254	1272		78		41520	Alibaba	Qwen
GPT-4-Turbo-2024-04-09	1253	1252	1151	82.63		102136	OpenAI	Proprietary
Llama-3.3-70B-Instruct	1253	1247				38078	Meta	Llama-3.3
Athene-70B	1247	1242		77.6		20581	NexusFlow	CC-BY-NC-4.0
Mistral-Small-3.1-24B-Instruct-2503	1246	1251				2352	Mistral	Apache 2.0
GPT-4-1106-preview	1246	1242				103743	OpenAI	Proprietary
Mistral-Large-2411	1245	1254		70.42		29641	Mistral	MRL
Llama-3.1-70B-Instruct	1244	1240		55.73	86	58638	Meta	Llama 3.1
Claude 3 Opus	1244	1239	1076	60.36	86.8	202635	Anthropic	Proprietary
Amazon Nova Pro 1.0	1241	1252	1044			26298	Amazon	Proprietary
GPT-4-0125-preview	1241	1233		77.96		97077	OpenAI	Proprietary
Llama-3.1-Tulu-3-70B	1240	1222				3012	Ai2	Llama 3.1
Llama-4-Scout-17B-16E-Instruct	1239	1240	1161			3397	Meta	Llama
Claude 3.5 Haiku (20241022)	1234	1255				37402	Anthropic	Propretary
Reka-Core-20240904	1232	1210				7950	Reka AI	Proprietary
Gemini-1.5-Flash-001	1223	1221	1072	49.61	78.9	65662	Google	Proprietary
Jamba-1.5-Large	1218	1216			81.2	9125	AI21 Labs	Jamba Open
Deepseek-v2-API-0628	1216	1231				19511	DeepSeek AI	DeepSeek
Gemma-2-27B-it	1216	1198		57.51		79529	Google	Gemma license
Qwen2.5-Coder-32B-Instruct	1214	1250				5731	Alibaba	Apache 2.0
Mistral-Small-24B-Instruct-2501	1214	1220				15322	Mistral	Apache 2.0
Amazon Nova Lite 1.0	1213	1224	1061			20643	Amazon	Proprietary
Gemma-2-9B-it-SimPO	1213	1185				10547	Princeton	MIT
Command R+ (08-2024)	1212	1170				10536	Cohere	CC-BY-NC-4.0
Deepseek-Coder-v2-0724	1211	1256		62.3		11725	DeepSeek	Proprietary
Gemini-1.5-Flash-8B-001	1209	1197	1106			37699	Google	Proprietary
Llama-3.1-Nemotron-51B-Instruct	1208	1200				3888	Nvidia	Llama 3.1
Nemotron-4-340B-Instruct	1206	1187				20610	Nvidia	Nvidia
Aya-Expanse-32B	1206	1182				28760	Cohere	CC-BY-NC-4.0
GLM-4-0520	1203	1205		63.84		10222	Zhipu AI	Proprietary
Llama-3-70B-Instruct	1203	1189		46.57	82	163623	Meta	Llama 3
Phi-4	1202	1210				25213	Microsoft	MIT
OLMo-2-0325-32B-Instruct	1202	1184				3456	Allen AI	Apache-2.0
Reka-Flash-20240904	1202	1180				8135	Reka AI	Proprietary
Claude 3 Sonnet	1198	1202	1048	46.8	79	113058	Anthropic	Proprietary
Amazon Nova Micro 1.0	1194	1199				20656	Amazon	Proprietary
Gemma-2-9B-it	1189	1162				57200	Google	Gemma license
Hunyuan-Standard-256K	1185	1216				2900	Tencent	Proprietary
Qwen2-72B-Instruct	1184	1176		46.86	84.2	38873	Alibaba	Qianwen LICENSE
GPT-4-0314	1183	1184		50	86.4	55961	OpenAI	Proprietary
Llama-3.1-Tulu-3-8B	1182	1168				3075	Ai2	Llama 3.1
Ministral-8B-2410	1179	1190				5109	Mistral	MRL
Qwen-Max-0428	1179	1178				25693	Alibaba	Proprietary
Claude 3 Haiku	1176	1178	1000	41.47	75.2	122311	Anthropic	Proprietary
Aya-Expanse-8B	1176	1154				10391	Cohere	CC-BY-NC-4.0
Command R (08-2024)	1176	1150				10852	Cohere	CC-BY-NC-4.0
DeepSeek-Coder-V2-Instruct	1175	1228				15753	DeepSeek AI	DeepSeek License
Jamba-1.5-Mini	1173	1170			69.7	9273	AI21 Labs	Jamba Open
Llama-3.1-8B-Instruct	1172	1175		21.34	73	52585	Meta	Llama 3.1
GPT-4-0613	1160	1156		37.9		91619	OpenAI	Proprietary
Qwen1.5-110B-Chat	1158	1164			80.4	27431	Alibaba	Qianwen LICENSE
Yi-1.5-34B-Chat	1154	1151			76.8	25136	01 AI	Apache-2.0
Llama-3-8B-Instruct	1148	1135		20.56	68.4	109061	Meta	Llama 3
InternLM2.5-20B-chat	1146	1147				10599	InternLM	Other
Claude-1	1145	1125			77	21149	Anthropic	Proprietary
Qwen1.5-72B-Chat	1144	1149		36.12	77.5	40655	Alibaba	Qianwen LICENSE
Mixtral-8x22b-Instruct-v0.1	1144	1142		36.36	77.8	53756	Mistral	Apache 2.0
Mistral Medium	1144	1142		31.9	75.3	35562	Mistral	Proprietary
Gemma-2-2b-it	1140	1096			51.3	48894	Google	Gemma license
Granite-3.1-8B-Instruct	1139	1162				3289	IBM	Apache 2.0
Claude-2.0	1128	1124		23.99	78.5	12764	Anthropic	Proprietary
Gemini-1.0-Pro-001	1128	1092			71.8	18801	Google	Proprietary
Zephyr-ORPO-141b-A35b-v0.1	1124	1113				4854	HuggingFace	Apache 2.0
Qwen1.5-32B-Chat	1122	1138			73.4	22765	Alibaba	Qianwen LICENSE
Phi-3-Medium-4k-Instruct	1119	1114		33.37	78	26106	Microsoft	MIT
Granite-3.1-2B-Instruct	1116	1136				3380	IBM	Apache 2.0
Claude-2.1	1115	1121		22.77		37697	Anthropic	Proprietary
Starling-LM-7B-beta	1115	1118		23.01		16675	Nexusflow	Apache-2.0
GPT-3.5-Turbo-0613	1113	1124		24.82		38955	OpenAI	Proprietary
Mixtral-8x7B-Instruct-v0.1	1111	1103		23.4	70.6	76134	Mistral	Apache 2.0
Claude-Instant-1	1108	1098			73.4	20630	Anthropic	Proprietary
Yi-34B-Chat	1108	1095		23.15	73.5	15916	01 AI	Yi License
Qwen1.5-14B-Chat	1105	1115			67.6	18682	Alibaba	Qianwen LICENSE
GPT-3.5-Turbo-0314	1104	1104		18.05	70	5640	OpenAI	Proprietary
WizardLM-70B-v1.0	1103	1060			63.7	8383	Microsoft	Llama 2
GPT-3.5-Turbo-0125	1102	1113		23.34		68874	OpenAI	Proprietary
DBRX-Instruct-Preview	1100	1107		24.63	73.7	33743	Databricks	DBRX LICENSE
Llama-3.2-3B-Instruct	1099	1069				8390	Meta	Llama 3.2
Phi-3-Small-8k-Instruct	1098	1096		29.77	75.7	18473	Microsoft	MIT
Tulu-2-DPO-70B	1096	1082		14.99		6659	AllenAI/UW	AI2 ImpACT Low-risk
Granite-3.0-8B-Instruct	1090	1086				7002	IBM	Apache 2.0
Llama-2-70B-chat	1089	1061		11.55	63	39594	Meta	Llama 2
OpenChat-3.5-0106	1088	1091			65.8	12990	OpenChat	Apache-2.0
Vicuna-33B	1087	1056		8.63	59.2	22936	LMSYS	Non-commercial
Snowflake Arctic Instruct	1086	1066		17.61	67.3	34175	Snowflake	Apache 2.0
Starling-LM-7B-alpha	1085	1069		12.8	63.9	10415	UC Berkeley	CC-BY-NC-4.0
Nous-Hermes-2-Mixtral-8x7B-DPO	1081	1068				3838	NousResearch	Apache-2.0
Gemma-1.1-7B-it	1080	1073		12.09	64.3	25072	Google	Gemma license
NV-Llama2-70B-SteerLM-Chat	1077	1012			68.5	3636	Nvidia	Llama 2
pplx-70B-online	1074	1017				6896	Perplexity AI	Proprietary
DeepSeek-LLM-67B-Chat	1073	1068			71.3	4988	DeepSeek AI	DeepSeek License
OpenChat-3.5	1073	1043			64.3	8107	OpenChat	Apache-2.0
OpenHermes-2.5-Mistral-7B	1071	1047				5088	NousResearch	Apache-2.0
Granite-3.0-2B-Instruct	1070	1077				7189	IBM	Apache 2.0
Mistral-7B-Instruct-v0.2	1069	1063		12.57		20071	Mistral	Apache-2.0
Phi-3-Mini-4K-Instruct-June-24	1067	1071			70.9	12808	Microsoft	MIT
Qwen1.5-7B-Chat	1066	1078			61	4872	Alibaba	Qianwen LICENSE
GPT-3.5-Turbo-1106	1064	1084		18.87		17035	OpenAI	Proprietary
Phi-3-Mini-4k-Instruct	1063	1075			68.8	21097	Microsoft	MIT
Llama-2-13b-chat	1060	1040			53.6	19725	Meta	Llama 2
SOLAR-10.7B-Instruct-v1.0	1059	1036			66.2	4286	Upstage AI	CC-BY-NC-4.0
Dolphin-2.2.1-Mistral-7B	1059	1014				1714	Cognitive Computations	Apache-2.0
WizardLM-13b-v1.2	1055	1015			52.7	7176	Microsoft	Llama 2
Llama-3.2-1B-Instruct	1050	1035				8523	Meta	Llama 3.2
Qwen2.5-VL-32B-Instruct			1214				Alibaba	Apache 2.0
Step-1o-Vision-32k (highres)			1187				StepFun	Proprietary
Qwen2.5-VL-72B-Instruct			1171				Alibaba	Qwen
Pixtral-Large-2411			1154				Mistral	MRL
Qwen-VL-Max-1119			1128				Alibaba	Proprietary
Qwen2-VL-72b-Instruct			1111				Alibaba	Qwen
Step-1V-32K			1111				StepFun	Proprietary
Molmo-72B-0924			1076				AI2	Apache 2.0
Pixtral-12B-2409			1072				Mistral	Apache 2.0
Llama-3.2-90B-Vision-Instruct			1070				Meta	Llama 3.2
Aya-Vision-8B			1069				Cohere	CC-BY-NC-4.0
InternVL2-26B			1067				OpenGVLab	MIT
Hunyuan-Standard-Vision-2024-12-31			1066				Tencent	Proprietary
Aya-Vision-32B			1060				Cohere	CC-BY-NC-4.0
Qwen2-VL-7B-Instruct			1054				Aliaba	Apache 2.0
Yi-Vision			1045				01 AI	Proprietary
Llama-3.2-11B-Vision-Instruct			1032				Meta	Llama 3.2

If you want to see more models, please help us add them.

💻 Code: The Arena Elo ratings are computed by this notebook. The MT-bench scores (single-answer grading on a scale of 10) are computed by fastchat.llm_judge. The MMLU scores are computed by InstructEval. Higher values are better for all benchmarks. Empty cells mean not available. The latest and detailed leaderboard is here.

More Statistics for Chatbot Arena

🔗 Arena Statistics

Transition from online Elo rating system to Bradley-Terry model

We adopted the Elo rating system for ranking models since the launch of the Arena. It has been useful to transform pairwise human preference to Elo ratings that serve as a predictor of winrate between models. Specifically, if player A has a rating of R_A and player B a rating of R_B, the probability of player A winning is

ELO rating has been used to rank chess players by the international community for over 60 years. Standard Elo rating systems assume a player’s performance changes overtime. So an online algorithm is needed to capture such dynamics, meaning recent games should weigh more than older games. Specifically, after each game, a player’s rating is updated according to the difference between predicted outcome and actual outcome.

This algorithm has two distinct features:

1. It can be computed asynchronously by players around the world.
2. It allows for players performance to change dynamically – it does not assume a fixed unknown value for the players rating.

This ability to adapt is determined by the parameter K which controls the magnitude of rating changes that can affect the overall result. A larger K essentially put more weight on the recent games, which may make sense for new players whose performance improves quickly. However as players become more senior and their performance “converges” then a smaller value of K is more appropriate. As a result, USCF adopted K based on the number of games and tournaments completed by the player (reference). That is, the Elo rating of a senior player changes slower than a new player.

When we launched the Arena, we noticed considerable variability in the ratings using the classic online algorithm. We tried to tune the K to be sufficiently stable while also allowing new models to move up quickly in the leaderboard. We ultimately decided to adopt a bootstrap-like technique to shuffle the data and sample Elo scores from 1000 permutations of the online plays. You can find the details in this notebook. This provided consistent stable scores and allowed us to incorporate new models quickly. This is also observed in a recent work by Cohere. However, we used the same samples to estimate confidence intervals which were therefore too wide (effectively CI’s for the original online Elo estimates).

In the context of LLM ranking, there are two important differences from the classic Elo chess ranking system. First, we have access to the entire history of all games for all models and so we don’t need a decentralized algorithm. Second, most models are static (we have access to the weights) and so we don’t expect their performance to change. However, it is worth noting that the hosted proprietary models may not be static and their behavior can change without notice. We try our best to pin specific model API versions if possible.

To improve the quality of our rankings and their confidence estimates, we are adopting another widely used rating system called the Bradley–Terry (BT) model. This model actually is the maximum likelihood (MLE) estimate of the underlying Elo model assuming a fixed but unknown pairwise win-rate. Similar to Elo rating, BT model is also based on pairwise comparison to derive ratings of players to estimate win rate between each other. The core difference between BT model vs the online Elo system is the assumption that player’s performance does not change (i.e., game order does not matter) and the computation takes place in a centralized fashion.