LLM Leaderboard

Intelligence

o4-mini (high)
70
Gemini 2.5 Pro Preview
68
Grok 3 mini Reasoning (high)
67
4
o3-mini (high)
66

Output

Nova Micro
335.6t/s
Gemini 1.5 Flash (May)
328.5t/s
DeepSeek R1 Distill Qwen 1.5B
315.9t/s
4
Gemini 1.5 Flash-8B
285.9t/s

Latency

o4-mini (high)
39.16t/s
o1
33.95t/s
o3-mini (high)
31.67t/s
4
Gemini 2.5 Pro Preview
27.4t/s

Price

o1-pro
$262.5/1M
GPT-4.5 (Preview)
$93.75/1M
GPT-4
$37.5/1M
4
Claude 3 Opus
$30.0/1M

ContextWindow

Llama 4 Scout
10m
MiniMax-Text-01
4m
Gemini 1.5 Pro (May)
2m
4
Gemini 2.0 Flash Thinking exp. (Dec '24)
2m

		Intelligence	Price	Output tokens/s	Latency
																											Further Analysis
o4-mini (high)	Proprietary	200k	70	0.83%	0.78%	0.18%	0.8%	0.47%	0.99%	0.99%	0.94%	0%	0%	$1.93/1M	4.4/1M	1.1/1M	126.5t/s	77.1t/s	105.3t/s	148t/s	187.4t/s	39.16t/s	18.64t/s	28.35t/s	56.43t/s	79.18t/s
Gemini 2.5 Pro Preview	Proprietary	1m	68	0.86%	0.84%	0.17%	0.7%	0.4%	0.99%	0.98%	0.87%	0%	0%	$3.44/1M	10.0/1M	1.25/1M	220.5t/s	196.1t/s	212.7t/s	228.4t/s	234.1t/s	27.4t/s	12.91t/s	17.47t/s	30.22t/s	34.85t/s
Grok 3 mini Reasoning (high)	Proprietary	1m	67	0.83%	0.79%	0.11%	0.7%	0.41%	0.98%	0.99%	0.93%	0%	0%	$0.35/1M	0.5/1M	0.3/1M	112.5t/s	86.6t/s	98.7t/s	141.5t/s	212.8t/s	0.36t/s	0.3t/s	0.33t/s	0.43t/s	0.5t/s
o3-mini (high)	Proprietary	200k	66	0.8%	0.77%	0.12%	0.73%	0.4%	0%	0.99%	0.86%	0%	0%	$1.93/1M	4.4/1M	1.1/1M	208.2t/s	99.9t/s	185.4t/s	225.3t/s	278.5t/s	31.67t/s	19.82t/s	27.54t/s	36.86t/s	48.16t/s
o3-mini	Proprietary	200k	63	0.79%	0.75%	0.09%	0.72%	0.4%	0.97%	0.97%	0.77%	0%	0%	$1.93/1M	4.4/1M	1.1/1M	219t/s	151.2t/s	197.2t/s	249.9t/s	302.1t/s	10.7t/s	6.75t/s	9.25t/s	13.08t/s	18.01t/s
o1	Proprietary	200k	62	0.84%	0.75%	0.08%	0.68%	0.36%	0.97%	0.97%	0.72%	0.88%	0%	$26.25/1M	60.0/1M	15.0/1M	86.6t/s	57t/s	70.9t/s	97.5t/s	118.7t/s	33.95t/s	21.22t/s	30.97t/s	38.1t/s	60.95t/s
DeepSeek R1	Open	128k	60	0.84%	0.71%	0.09%	0.62%	0.36%	0.98%	0.97%	0.68%	0%	0%	$0.96/1M	2.19/1M	0.55/1M	25.3t/s	21.4t/s	23.4t/s	30t/s	35.1t/s	3.38t/s	2.82t/s	3.11t/s	4.5t/s	6.36t/s
QwQ-32B	Open	131k	58	0.76%	0.59%	0.08%	0.63%	0.36%	0.98%	0.96%	0.78%	0%	0%	$0.47/1M	0.55/1M	0.4/1M	89.1t/s	21t/s	40.9t/s	361.2t/s	429.7t/s	0.49t/s	0.19t/s	0.29t/s	1.33t/s	3.31t/s
Claude 3.7 Sonnet Thinking	Proprietary	200k	57	0.84%	0.77%	0.1%	0.47%	0.4%	0.98%	0.95%	0.49%	0%	0%	$6.0/1M	15.0/1M	3.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
o1-mini	Proprietary	128k	54	0.74%	0.6%	0.05%	0.58%	0.32%	0.97%	0.94%	0.6%	0.83%	1308%	$1.93/1M	4.4/1M	1.1/1M	215.4t/s	104t/s	179.7t/s	222.8t/s	235.4t/s	9.68t/s	1.4t/s	5.55t/s	11.5t/s	14.12t/s
GPT-4.1 mini	Proprietary	1m	53	0.78%	0.66%	0.05%	0.48%	0.4%	0.95%	0.93%	0.43%	0%	0%	$0.7/1M	1.6/1M	0.4/1M	84.2t/s	67.4t/s	75.5t/s	93.2t/s	117.6t/s	0.35t/s	0.26t/s	0.32t/s	0.41t/s	0.58t/s
DeepSeek V3 (Mar' 25)	Open	128k	53	0.82%	0.66%	0.05%	0.41%	0.36%	0.92%	0.94%	0.52%	0%	0%	$0.48/1M	1.1/1M	0.27/1M	26t/s	23.6t/s	24.3t/s	28.9t/s	34.1t/s	3.68t/s	3.08t/s	3.27t/s	4t/s	4.26t/s
GPT-4.1	Proprietary	1m	53	0.81%	0.67%	0.05%	0.46%	0.38%	0.96%	0.91%	0.44%	0%	0%	$3.5/1M	8.0/1M	2.0/1M	130.8t/s	66.8t/s	102.3t/s	144.7t/s	175.3t/s	0.4t/s	0.34t/s	0.36t/s	0.56t/s	0.8t/s
DeepSeek R1 Distill Qwen 32B	Open	128k	52	0.74%	0.62%	0.06%	0.27%	0.38%	0.95%	0.94%	0.69%	0%	0%	$0.22/1M	0.24/1M	0.21/1M	33.8t/s	18.3t/s	20.7t/s	46.3t/s	49.9t/s	1.02t/s	0.2t/s	0.24t/s	1.12t/s	1.23t/s
Gemini 2.0 Flash Thinking exp. (Jan '25)	Proprietary	1m	52	0.8%	0.7%	0.07%	0.32%	0.33%	0%	0.94%	0.5%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Llama 4 Maverick	Open	1m	51	0.81%	0.67%	0.05%	0.4%	0.33%	0.88%	0.89%	0.39%	0%	0%	$0.36/1M	0.82/1M	0.2/1M	123.9t/s	54.2t/s	94t/s	136t/s	806.5t/s	0.4t/s	0.16t/s	0.34t/s	0.53t/s	0.92t/s
Grok 3	Proprietary	1m	51	0.8%	0.69%	0.05%	0.43%	0.37%	0.91%	0.87%	0.33%	0%	0%	$6.0/1M	15.0/1M	3.0/1M	53.9t/s	32.6t/s	44.6t/s	59t/s	66.7t/s	0.55t/s	0.47t/s	0.52t/s	0.68t/s	3.8t/s
GPT-4o (March 2025)	Proprietary	128k	50	0.8%	0.66%	0.05%	0.43%	0.37%	0.96%	0.89%	0.33%	0%	0%	$7.5/1M	15.0/1M	5.0/1M	211.3t/s	120.5t/s	194t/s	229.9t/s	240.7t/s	0.56t/s	0.35t/s	0.41t/s	0.63t/s	1t/s
DeepSeek R1 Distill Qwen 14B	Open	128k	49	0.74%	0.48%	0.04%	0.38%	0.24%	0.93%	0.95%	0.67%	0%	0%	$0.88/1M	0.88/1M	0.88/1M	72.7t/s	41.8t/s	44.8t/s	152.9t/s	171t/s	0.61t/s	0.27t/s	0.38t/s	0.74t/s	0.86t/s
Gemini 2.0 Pro Experimental	Proprietary	2m	49	0.81%	0.62%	0.07%	0.35%	0.31%	0.95%	0.92%	0.36%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	218.3t/s	203.3t/s	211.6t/s	225.1t/s	233.8t/s	27.15t/s	12.32t/s	17.21t/s	30.74t/s	36.09t/s
Gemini 2.0 Flash	Proprietary	1m	48	0.78%	0.62%	0.05%	0.33%	0.31%	0.9%	0.93%	0.33%	0%	0%	$0.17/1M	0.4/1M	0.1/1M	255.2t/s	230.6t/s	243.5t/s	265.3t/s	284.3t/s	0.33t/s	0.25t/s	0.28t/s	0.4t/s	0.41t/s
Claude 3.7 Sonnet	Proprietary	200k	48	0.8%	0.66%	0.05%	0.39%	0.38%	0.95%	0.85%	0.22%	0%	0%	$6.0/1M	15.0/1M	3.0/1M	78.4t/s	65.4t/s	74.2t/s	79.7t/s	83.1t/s	1.01t/s	0.71t/s	0.81t/s	1.53t/s	2.52t/s
DeepSeek R1 Distill Llama 70B	Open	128k	48	0.8%	0.4%	0.06%	0.27%	0.31%	0.97%	0.94%	0.67%	0%	0%	$0.6/1M	0.87/1M	0.54/1M	83.3t/s	29.3t/s	56.6t/s	347.4t/s	2313.00t/s	0.5t/s	0.17t/s	0.34t/s	0.76t/s	1.72t/s
Reka Flash 3	Open	128k	47	0.67%	0.53%	0.05%	0.44%	0.27%	0.95%	0.89%	0.51%	0%	0%	$0.35/1M	0.8/1M	0.2/1M	56.2t/s	55.2t/s	55.5t/s	57.1t/s	58t/s	0.96t/s	0.87t/s	0.95t/s	0.98t/s	1.07t/s
DeepSeek V3 (Dec '24)	Open	128k	46	0.75%	0.56%	0.04%	0.36%	0.35%	0.91%	0.89%	0.25%	0.86%	0%	$0.48/1M	1.1/1M	0.27/1M	26.2t/s	23.7t/s	24.4t/s	29.6t/s	35.7t/s	3.66t/s	3.05t/s	3.21t/s	3.9t/s	4.78t/s
Gemini 2.0 Flash (exp)	Proprietary	1m	46	0.78%	0.64%	0.05%	0.21%	0.34%	0.91%	0.91%	0.3%	0.84%	0%	$0.0/1M	0.0/1M	0.0/1M	250.8t/s	227.9t/s	241.9t/s	257.3t/s	268.1t/s	0.27t/s	0.23t/s	0.25t/s	0.29t/s	0.36t/s
Gemini 1.5 Pro (Sep)	Proprietary	2m	45	0.75%	0.59%	0.05%	0.32%	0.3%	0.9%	0.88%	0.23%	0.85%	1301%	$2.19/1M	5.0/1M	1.25/1M	99.6t/s	90.3t/s	94.7t/s	103.9t/s	106.6t/s	0.46t/s	0.41t/s	0.44t/s	0.54t/s	0.92t/s
Qwen2.5 Max	Proprietary	32k	45	0.76%	0.59%	0.05%	0.36%	0.34%	0.93%	0.84%	0.23%	0%	0%	$2.8/1M	6.4/1M	1.6/1M	52.4t/s	45.9t/s	50.7t/s	53.6t/s	54t/s	1.27t/s	1.14t/s	1.19t/s	1.37t/s	1.49t/s
Claude 3.5 Sonnet (Oct)	Proprietary	200k	44	0.77%	0.6%	0.04%	0.38%	0.37%	0.93%	0.77%	0.16%	0.88%	1282%	$6.0/1M	15.0/1M	3.0/1M	80.3t/s	68.1t/s	77.7t/s	81.5t/s	83.3t/s	0.87t/s	0.65t/s	0.78t/s	1.22t/s	1.64t/s
QwQ 32B-Preview	Open	33k	43	0.65%	0.56%	0.05%	0.34%	0.04%	0.87%	0.91%	0.45%	0%	0%	$0.58/1M	0.75/1M	0.55/1M	54.6t/s	38.4t/s	46.2t/s	82.5t/s	98t/s	0.41t/s	0.2t/s	0.25t/s	0.56t/s	1.68t/s
Sonar Pro	Proprietary	200k	43	0.76%	0.58%	0.08%	0.28%	0.23%	0.85%	0.75%	0.29%	0%	0%	$6.0/1M	15.0/1M	3.0/1M	105.5t/s	50t/s	77.8t/s	137.8t/s	178.5t/s	2.45t/s	1.71t/s	1.96t/s	2.87t/s	3.84t/s
Llama 4 Scout	Open	10m	43	0.75%	0.59%	0.04%	0.3%	0.17%	0.83%	0.84%	0.28%	0%	0%	$0.26/1M	0.5/1M	0.15/1M	121.3t/s	39.2t/s	89.5t/s	155.8t/s	2733.30t/s	0.41t/s	0.2t/s	0.32t/s	0.52t/s	0.83t/s
Sonar	Proprietary	127k	43	0.69%	0.47%	0.07%	0.3%	0.23%	0.82%	0.82%	0.49%	0%	0%	$1.0/1M	1.0/1M	1.0/1M	77.1t/s	35.5t/s	57.7t/s	86.6t/s	118.9t/s	2.28t/s	1.78t/s	2.02t/s	2.69t/s	3.51t/s
Llama 3.3 70B	Open	128k	41	0.71%	0.5%	0.04%	0.29%	0.26%	0.86%	0.77%	0.3%	0.84%	0%	$0.6/1M	0.7/1M	0.59/1M	121t/s	25.7t/s	45.9t/s	184t/s	2104.30t/s	0.47t/s	0.24t/s	0.35t/s	0.6t/s	1.49t/s
Gemini 2.0 Flash-Lite (Feb '25)	Proprietary	1m	41	0.72%	0.54%	0.04%	0.19%	0.25%	0.88%	0.87%	0.28%	0%	0%	$0.13/1M	0.3/1M	0.07/1M	203.7t/s	178.7t/s	193.5t/s	210.5t/s	216t/s	0.29t/s	0.25t/s	0.28t/s	0.31t/s	0.35t/s
GPT-4o (Nov '24)	Proprietary	128k	41	0.75%	0.54%	0.03%	0.31%	0.33%	0.93%	0.76%	0.15%	0.84%	1361%	$4.38/1M	10.0/1M	2.5/1M	174.6t/s	107.8t/s	144.1t/s	200.7t/s	219t/s	0.61t/s	0.46t/s	0.51t/s	0.66t/s	0.8t/s
GPT-4o (May '24)	Proprietary	128k	41	0.74%	0.53%	0.03%	0.33%	0.31%	0.94%	0.79%	0.11%	0%	1285%	$7.5/1M	15.0/1M	5.0/1M	124.4t/s	77.5t/s	89.6t/s	138.7t/s	154.7t/s	0.52t/s	0.42t/s	0.47t/s	0.61t/s	0.85t/s
GPT-4.1 nano	Proprietary	1m	41	0.66%	0.51%	0.04%	0.33%	0.26%	0.88%	0.85%	0.24%	0%	0%	$0.17/1M	0.4/1M	0.1/1M	242.1t/s	158.6t/s	218.6t/s	294.6t/s	360.9t/s	0.46t/s	0.25t/s	0.34t/s	0.86t/s	0.9t/s
Tulu3 405B	Open	128k	40	0.72%	0.52%	0.04%	0.29%	0.3%	0.89%	0.78%	0.13%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Command A	Open	256k	40	0.71%	0.53%	0.05%	0.29%	0.28%	0.82%	0.82%	0.1%	0%	0%	$4.38/1M	10.0/1M	2.5/1M	55.7t/s	49.4t/s	53.2t/s	57t/s	58.8t/s	0.26t/s	0.23t/s	0.24t/s	0.26t/s	0.38t/s
Phi-4	Open	16k	40	0.71%	0.57%	0.04%	0.23%	0.26%	0.87%	0.81%	0.14%	0%	0%	$0.22/1M	0.5/1M	0.13/1M	45.2t/s	39.1t/s	44t/s	45.5t/s	45.5t/s	0.48t/s	0.4t/s	0.45t/s	0.52t/s	18.07t/s
MiniMax-Text-01	Open	4m	40	0.76%	0.58%	0.04%	0.25%	0.25%	0.86%	0.75%	0.13%	0%	0%	$0.42/1M	1.1/1M	0.2/1M	28.8t/s	20.6t/s	23.8t/s	33.7t/s	39.7t/s	0.85t/s	0.63t/s	0.72t/s	1.03t/s	1.53t/s
Qwen2.5 72B	Open	131k	40	0.72%	0.49%	0.04%	0.28%	0.27%	0.88%	0.86%	0.16%	0.83%	1259%	$0.0/1M	0.0/1M	0.0/1M	63.1t/s	59t/s	62.7t/s	63.2t/s	63.5t/s	1.17t/s	1.02t/s	1.12t/s	1.27t/s	1.49t/s
Llama 3.1 405B	Open	128k	40	0.73%	0.52%	0.04%	0.31%	0.3%	0.85%	0.7%	0.21%	0.77%	1266%	$3.5/1M	3.5/1M	3.5/1M	31.8t/s	11.8t/s	27t/s	78.3t/s	171.4t/s	0.78t/s	0.37t/s	0.46t/s	1.35t/s	2.15t/s
Llama 3.3 Nemotron Super 49B v1	Open	128k	39	0.7%	0.52%	0.04%	0.28%	0.23%	0.83%	0.78%	0.19%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Gemini 1.5 Flash (Sep)	Proprietary	1m	39	0.68%	0.46%	0.04%	0.27%	0.27%	0.84%	0.83%	0.18%	0.81%	1271%	$0.13/1M	0.3/1M	0.07/1M	198t/s	187.3t/s	194.1t/s	201.2t/s	207.2t/s	0.21t/s	0.19t/s	0.2t/s	0.22t/s	0.32t/s
Grok 2	Proprietary	131k	39	0.71%	0.51%	0.04%	0.27%	0.28%	0.86%	0.78%	0.13%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Grok Beta	Proprietary	128k	38	0.7%	0.47%	0.05%	0.24%	0.3%	0.87%	0.74%	0.1%	0%	1289%	$7.5/1M	15.0/1M	5.0/1M	65.4t/s	56.4t/s	61.8t/s	66.4t/s	68.2t/s	0.28t/s	0.24t/s	0.26t/s	0.32t/s	0.39t/s
Gemma 3 27B	Open	128k	38	0.67%	0.43%	0.05%	0.14%	0.21%	0.89%	0.88%	0.25%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Mistral Large 2 (Nov '24)	Open	128k	38	0.7%	0.49%	0.04%	0.29%	0.29%	0.9%	0.74%	0.11%	0.83%	0%	$3.0/1M	6.0/1M	2.0/1M	36.6t/s	15.3t/s	29.9t/s	41.2t/s	46.3t/s	0.39t/s	0.36t/s	0.37t/s	0.44t/s	1.32t/s
Llama 3.1 Nemotron 70B	Open	128k	37	0.69%	0.47%	0.05%	0.17%	0.23%	0.82%	0.73%	0.25%	0%	1269%	$0.24/1M	0.4/1M	0.18/1M	41t/s	32.5t/s	39.2t/s	64.8t/s	71.7t/s	0.53t/s	0.26t/s	0.44t/s	0.63t/s	0.87t/s
Qwen2.5 Instruct 32B	Open	128k	37	0.7%	0.47%	0.04%	0.25%	0.23%	0.9%	0.81%	0.11%	0%	0%	$0.15/1M	0.3/1M	0.1/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Pixtral Large	Open	128k	37	0.7%	0.51%	0.04%	0.26%	0.29%	0.85%	0.71%	0.07%	0%	0%	$3.0/1M	6.0/1M	2.0/1M	28.3t/s	14.4t/s	21.8t/s	37.4t/s	41.4t/s	0.48t/s	0.36t/s	0.4t/s	0.59t/s	9.39t/s
Mistral Large 2 (Jul '24)	Open	128k	37	0.68%	0.47%	0.03%	0.27%	0.27%	0.89%	0.71%	0.09%	0%	1251%	$3.0/1M	6.0/1M	2.0/1M	35.8t/s	10.2t/s	28.3t/s	40.6t/s	43.4t/s	0.46t/s	0.39t/s	0.41t/s	0.71t/s	2.85t/s
Nova Pro	Proprietary	300k	37	0.69%	0.5%	0.03%	0.23%	0.21%	0.83%	0.79%	0.11%	0.83%	0%	$1.75/1M	4.0/1M	1.0/1M	113.2t/s	98.4t/s	109.2t/s	116.4t/s	123.5t/s	0.4t/s	0.36t/s	0.38t/s	0.43t/s	0.53t/s
GPT-4o mini	Proprietary	128k	36	0.65%	0.43%	0.04%	0.23%	0.23%	0.88%	0.79%	0.12%	0.80%	1273%	$0.26/1M	0.6/1M	0.15/1M	88.4t/s	60.6t/s	74t/s	99.9t/s	116.1t/s	0.34t/s	0.28t/s	0.31t/s	0.38t/s	0.55t/s
Qwen2.5 Coder 32B	Open	131k	36	0.64%	0.42%	0.04%	0.3%	0.27%	0.9%	0.77%	0.12%	0%	1220%	$0.2/1M	0.2/1M	0.2/1M	52.9t/s	13.8t/s	41.2t/s	67.8t/s	87.6t/s	0.46t/s	0.21t/s	0.29t/s	0.66t/s	1.64t/s
Claude 3 Opus	Proprietary	200k	35	0.7%	0.49%	0.03%	0.28%	0.23%	0.85%	0.64%	0.03%	0%	1248%	$30.0/1M	75.0/1M	15.0/1M	29.1t/s	25.3t/s	27.2t/s	30.1t/s	31.5t/s	1.06t/s	0.88t/s	0.99t/s	1.14t/s	1.75t/s
Mistral Small 3	Open	32k	35	0.65%	0.46%	0.04%	0.25%	0.24%	0.85%	0.72%	0.08%	0%	0%	$0.15/1M	0.3/1M	0.1/1M	139.3t/s	69.8t/s	125t/s	156t/s	165.5t/s	0.27t/s	0.25t/s	0.27t/s	0.3t/s	1.2t/s
Mistral Small 3.1	Open	128k	35	0.66%	0.45%	0.05%	0.21%	0.27%	0.86%	0.71%	0.09%	0%	0%	$0.15/1M	0.3/1M	0.1/1M	147.9t/s	68.2t/s	106.1t/s	162t/s	179.5t/s	0.28t/s	0.24t/s	0.26t/s	0.31t/s	0.41t/s
Llama 3.1 70B	Open	128k	35	0.68%	0.41%	0.05%	0.23%	0.27%	0.81%	0.65%	0.17%	0%	1249%	$0.66/1M	0.73/1M	0.66/1M	47.7t/s	10.7t/s	31.7t/s	108.1t/s	161.7t/s	0.54t/s	0.23t/s	0.41t/s	0.78t/s	7.22t/s
Claude 3.5 Haiku	Proprietary	200k	35	0.63%	0.41%	0.04%	0.31%	0.27%	0.86%	0.72%	0.03%	78%	0%	$1.6/1M	4.0/1M	0.8/1M	66.1t/s	60.7t/s	64.4t/s	68t/s	69.2t/s	5.91t/s	1.59t/s	3.83t/s	7.37t/s	9.13t/s
Qwen Turbo	Proprietary	1m	34	0.63%	0.41%	0.04%	0.16%	0.15%	0.85%	0.81%	0.12%	0%	0%	$0.09/1M	0.2/1M	0.05/1M	110.4t/s	96.4t/s	106.9t/s	111.6t/s	114.9t/s	1.11t/s	0.96t/s	0.99t/s	1.19t/s	1.38t/s
Gemini 1.5 Pro (May)	Proprietary	2m	34	0.66%	0.37%	0.04%	0.24%	0.27%	0.83%	0.67%	0.08%	0%	1260%	$2.19/1M	5.0/1M	1.25/1M	68.1t/s	63.7t/s	66.1t/s	69.4t/s	70.7t/s	0.37t/s	0.34t/s	0.35t/s	0.39t/s	0.46t/s
Gemma 3 12B	Open	128k	34	0.6%	0.35%	0.05%	0.14%	0.17%	0.83%	0.85%	0.22%	0%	0%	$0.06/1M	0.1/1M	0.05/1M	32.4t/s	19.9t/s	26.3t/s	42.9t/s	53.5t/s	0.53t/s	0.4t/s	0.45t/s	0.67t/s	0.89t/s
DeepSeek R1 Distill Llama 8B	Open	128k	34	0.54%	0.3%	0.04%	0.23%	0.12%	0.84%	0.85%	0.33%	0%	0%	$0.04/1M	0.04/1M	0.04/1M	48.2t/s	25.9t/s	34t/s	53.6t/s	56.1t/s	0.81t/s	0.6t/s	0.68t/s	15.94t/s	50.16t/s
Nova Lite	Proprietary	300k	33	0.59%	0.43%	0.05%	0.17%	0.14%	0.84%	0.77%	0.11%	0.76%	0%	$0.1/1M	0.24/1M	0.06/1M	282.7t/s	244.3t/s	264.1t/s	305.4t/s	323.9t/s	0.33t/s	0.3t/s	0.33t/s	0.34t/s	0.38t/s
Qwen2 72B	Open	131k	33	0.62%	0.37%	0.04%	0.16%	0.23%	0.83%	0.7%	0.15%	0%	1187%	$0.0/1M	0.0/1M	0.0/1M	31t/s	30.8t/s	30.9t/s	31.1t/s	31.2t/s	1.36t/s	1.22t/s	1.29t/s	1.44t/s	1.75t/s
Llama 3.2 90B (Vision)	Open	128k	33	0.67%	0.43%	0.05%	0.21%	0.24%	0.82%	0.63%	0.05%	0%	0%	$0.81/1M	0.81/1M	0.81/1M	39.9t/s	24.6t/s	32t/s	45.8t/s	60.9t/s	0.35t/s	0.17t/s	0.21t/s	0.48t/s	0.61t/s
Gemini 1.5 Flash-8B	Proprietary	1m	31	0.57%	0.36%	0.05%	0.22%	0.23%	0.12%	0.69%	0.03%	0.74%	1211%	$0.07/1M	0.15/1M	0.04/1M	285.9t/s	273.6t/s	280t/s	293.8t/s	302.7t/s	0.19t/s	0.16t/s	0.18t/s	0.23t/s	0.26t/s
DeepHermes 3 - Mistral 24B	Open	32k	30	0.58%	0.38%	0.04%	0.2%	0.23%	0.75%	0.6%	0.05%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Jamba 1.6 Large	Open	256k	29	0.56%	0.39%	0.04%	0.17%	0.18%	0.7%	0.58%	0.05%	0%	0%	$3.5/1M	8.0/1M	2.0/1M	66.2t/s	42.7t/s	62.4t/s	68.8t/s	70.6t/s	0.57t/s	0.37t/s	0.49t/s	0.68t/s	1.02t/s
Hermes 3 - Llama-3.1 70B	Open	128k	29	0.57%	0.4%	0.04%	0.19%	0.23%	0.75%	0.54%	0.02%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Jamba 1.5 Large	Open	256k	29	0.57%	0.43%	0.04%	0.14%	0.16%	0.24%	0.61%	0.05%	0%	1221%	$3.5/1M	8.0/1M	2.0/1M	68t/s	52.3t/s	66.8t/s	68.3t/s	69.5t/s	0.5t/s	0.35t/s	0.39t/s	0.58t/s	0.91t/s
Yi-Large	Proprietary	32k	28	0.59%	0.36%	0.03%	0.11%	0.19%	0.74%	0.56%	0.07%	0%	1213%	$3.0/1M	3.0/1M	3.0/1M	69t/s	63.7t/s	66.5t/s	79.2t/s	81.8t/s	0.39t/s	0.32t/s	0.36t/s	0.47t/s	3.19t/s
Nova Micro	Proprietary	130k	28	0.53%	0.36%	0.05%	0.14%	0.09%	0.8%	0.7%	0.08%	0.71%	0%	$0.06/1M	0.14/1M	0.04/1M	335.6t/s	304.7t/s	314.6t/s	362t/s	383.6t/s	0.32t/s	0.28t/s	0.31t/s	0.33t/s	0.38t/s
Gemini 1.5 Flash (May)	Proprietary	1m	28	0.57%	0.32%	0.04%	0.2%	0.18%	0.72%	0.55%	0.09%	0%	1227%	$0.13/1M	0.3/1M	0.07/1M	328.5t/s	300.2t/s	310.4t/s	339.8t/s	356.4t/s	0.29t/s	0.24t/s	0.26t/s	0.3t/s	0.38t/s
Codestral (Jan '25)	Proprietary	256k	28	0.45%	0.31%	0.05%	0.24%	0.25%	0.85%	0.61%	0.04%	0%	0%	$0.45/1M	0.9/1M	0.3/1M	202.6t/s	93.5t/s	194.8t/s	206t/s	210.8t/s	0.25t/s	0.23t/s	0.24t/s	0.27t/s	0.56t/s
Claude 3 Sonnet	Proprietary	200k	28	0.58%	0.4%	0.04%	0.18%	0.23%	0.71%	0.41%	0.05%	0%	1201%	$6.0/1M	15.0/1M	3.0/1M	61.5t/s	55.3t/s	59.3t/s	62.6t/s	63.9t/s	0.52t/s	0.48t/s	0.5t/s	0.6t/s	1.17t/s
Qwen2.5 Coder 7B	Open	131k	27	0.47%	0.34%	0.05%	0.13%	0.15%	0.9%	0.66%	0.05%	0%	0%	$0.03/1M	0.06/1M	0.02/1M	204.6t/s	201.4t/s	202.9t/s	218t/s	219.9t/s	0.48t/s	0.45t/s	0.47t/s	0.52t/s	0.66t/s
Phi-4 Multimodal	Open	128k	27	0.49%	0.32%	0.04%	0.13%	0.11%	0.73%	0.69%	0.09%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	22.1t/s	19.9t/s	21.2t/s	23.3t/s	23.7t/s	0.35t/s	0.28t/s	0.33t/s	0.36t/s	0.68t/s
Mistral Small (Sep '24)	Open	33k	27	0.53%	0.38%	0.04%	0.14%	0.16%	0.81%	0.56%	0.06%	0%	0%	$0.3/1M	0.6/1M	0.2/1M	67.1t/s	11.1t/s	52.4t/s	81.2t/s	99.5t/s	0.33t/s	0.28t/s	0.3t/s	0.49t/s	0.96t/s
Llama 3 70B	Open	8k	27	0.57%	0.38%	0.04%	0.2%	0.19%	0.79%	0.48%	0%	0%	1206%	$0.88/1M	0.88/1M	0.8/1M	47.7t/s	19t/s	33t/s	154t/s	335.8t/s	0.52t/s	0.23t/s	0.38t/s	0.74t/s	1.76t/s
Mistral Large (Feb '24)	Proprietary	33k	26	0.52%	0.35%	0.03%	0.18%	0.21%	0.71%	0.53%	0%	0%	1157%	$6.0/1M	12.0/1M	4.0/1M	30.1t/s	22.2t/s	29.2t/s	30.9t/s	31.5t/s	0.46t/s	0.42t/s	0.45t/s	0.5t/s	1.37t/s
Phi-4 Mini	Open	128k	26	0.47%	0.33%	0.04%	0.13%	0.11%	0.74%	0.7%	0.03%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	56.7t/s	52.5t/s	55t/s	57.5t/s	59t/s	0.34t/s	0.31t/s	0.33t/s	0.37t/s	0.49t/s
Mixtral 8x22B	Open	65k	26	0.54%	0.33%	0.04%	0.15%	0.19%	0.72%	0.55%	0%	0%	1148%	$3.0/1M	6.0/1M	2.0/1M	55.8t/s	35.1t/s	49.2t/s	66.4t/s	80.4t/s	0.35t/s	0.31t/s	0.32t/s	0.4t/s	0.79t/s
Phi-3 Medium 14B	Open	128k	25	0.54%	0.33%	0.05%	0.15%	0.12%	0%	0.46%	0.01%	0%	1123%	$0.3/1M	0.68/1M	0.17/1M	51.9t/s	34.4t/s	50.6t/s	53.1t/s	54.6t/s	0.43t/s	0.39t/s	0.42t/s	0.46t/s	0.52t/s
Llama 3.1 8B	Open	128k	24	0.48%	0.26%	0.05%	0.12%	0.13%	0.67%	0.52%	0.08%	0.61%	1172%	$0.1/1M	0.1/1M	0.1/1M	155.1t/s	53.3t/s	74.1t/s	465.8t/s	2039.00t/s	0.36t/s	0.16t/s	0.23t/s	0.5t/s	1.13t/s
Claude 2.1	Proprietary	200k	24	0.5%	0.32%	0.04%	0.2%	0.18%	0.16%	0.37%	0.03%	0%	1118%	$12.0/1M	24.0/1M	8.0/1M	13.8t/s	13.5t/s	13.7t/s	14.1t/s	14.4t/s	0.85t/s	0.79t/s	0.81t/s	0.87t/s	1.2t/s
Gemma 3 4B	Open	128k	24	0.42%	0.29%	0.05%	0.07%	0.06%	0.72%	0.77%	0.05%	0%	0%	$0.03/1M	0.04/1M	0.02/1M	95.8t/s	50.5t/s	86.6t/s	101.5t/s	108.8t/s	0.3t/s	0.25t/s	0.27t/s	0.34t/s	0.64t/s
Pixtral 12B	Open	128k	23	0.47%	0.34%	0.05%	0.12%	0.14%	0.78%	0.46%	0%	0%	0%	$0.15/1M	0.15/1M	0.15/1M	110.2t/s	101.5t/s	106.4t/s	112.7t/s	117.1t/s	0.3t/s	0.26t/s	0.27t/s	0.35t/s	0.5t/s
Mistral Medium	Proprietary	33k	23	0.49%	0.35%	0.03%	0.1%	0.12%	0%	0.41%	0.04%	0%	1148%	$4.09/1M	8.1/1M	2.75/1M	41.6t/s	22.5t/s	40.3t/s	42.5t/s	43.5t/s	0.38t/s	0.34t/s	0.36t/s	0.44t/s	0.99t/s
Mistral Small (Feb '24)	Proprietary	33k	23	0.42%	0.3%	0.04%	0.11%	0.13%	0.79%	0.56%	0.01%	0%	0%	$1.5/1M	3.0/1M	1.0/1M	150.4t/s	62.4t/s	116.2t/s	161.7t/s	172.3t/s	0.29t/s	0.24t/s	0.26t/s	0.35t/s	2.19t/s
LFM 40B	Proprietary	32k	22	0.43%	0.33%	0.05%	0.1%	0.07%	0.51%	0.48%	0.02%	0%	0%	$0.15/1M	0.15/1M	0.15/1M	171.6t/s	154.1t/s	165.4t/s	174.6t/s	177.1t/s	0.46t/s	0.15t/s	0.25t/s	0.49t/s	0.58t/s
Phi-3 Mini	Open	4k	22	0.44%	0.32%	0.04%	0.12%	0.09%	0.25%	0.46%	0.04%	0%	1037%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Gemma 2 9B	Open	8k	22	0.5%	0.31%	0.04%	0.13%	0.01%	0.65%	0.52%	0%	0%	1190%	$0.12/1M	0.15/1M	0.12/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Ministral 8B	Open	128k	22	0.39%	0.28%	0.05%	0.11%	0.12%	0.77%	0.57%	0.04%	0%	1183%	$0.1/1M	0.1/1M	0.1/1M	142.3t/s	79.8t/s	137.3t/s	145.7t/s	147.6t/s	0.28t/s	0.26t/s	0.27t/s	0.29t/s	0.67t/s
Llama 3 8B	Open	8k	21	0.41%	0.3%	0.05%	0.1%	0.12%	0.71%	0.5%	0%	0%	1152%	$0.09/1M	0.14/1M	0.06/1M	92t/s	26.2t/s	73.8t/s	123.5t/s	1350.80t/s	0.36t/s	0.21t/s	0.3t/s	0.5t/s	1.59t/s
Command-R+	Open	128k	21	0.43%	0.34%	0.05%	0.11%	0.12%	0.63%	0.4%	0%	0%	1215%	$4.38/1M	10.0/1M	2.5/1M	53.5t/s	50.2t/s	51.8t/s	54t/s	55.3t/s	0.27t/s	0.25t/s	0.26t/s	0.29t/s	0.32t/s
Gemini 1.0 Pro	Proprietary	33k	21	0.43%	0.28%	0.05%	0.12%	0.12%	0.02%	0.4%	0.01%	0%	1111%	$0.75/1M	1.5/1M	0.5/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Mistral NeMo	Open	128k	20	0.4%	0.31%	0.04%	0.06%	0.1%	0.65%	0.4%	0%	0%	0%	$0.15/1M	0.15/1M	0.15/1M	140.7t/s	75.8t/s	134.6t/s	145.1t/s	148.8t/s	0.29t/s	0.26t/s	0.27t/s	0.33t/s	0.48t/s
Ministral 3B	Proprietary	128k	20	0.34%	0.26%	0.06%	0.07%	0.09%	0.74%	0.54%	0%	0%	0%	$0.04/1M	0.04/1M	0.04/1M	225.5t/s	132.8t/s	220.8t/s	231.6t/s	239.7t/s	0.27t/s	0.24t/s	0.25t/s	0.35t/s	0.75t/s
DBRX	Open	33k	20	0.4%	0.33%	0.07%	0.09%	0.12%	0.67%	0.28%	0.03%	0%	1103%	$1.13/1M	2.25/1M	0.75/1M	69.6t/s	51.3t/s	69.2t/s	70.1t/s	70.7t/s	0.55t/s	0.44t/s	0.46t/s	0.73t/s	0.97t/s
Command-R+ (Apr '24)	Open	128k	20	0.43%	0.32%	0.05%	0.12%	0.12%	0.64%	0.28%	0.01%	0%	1190%	$6.0/1M	15.0/1M	3.0/1M	67.8t/s	58.3t/s	61.9t/s	72t/s	77.6t/s	0.23t/s	0.21t/s	0.22t/s	0.24t/s	0.25t/s
Llama 2 Chat 13B	Open	4k	20	0.41%	0.32%	0.05%	0.1%	0.12%	0%	0.33%	0.02%	0%	1063%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Aya Expanse 32B	Open	128k	20	0.38%	0.23%	0.05%	0.14%	0.15%	0.68%	0.45%	0%	0.65%	1207%	$0.75/1M	1.5/1M	0.5/1M	122t/s	118.1t/s	120.6t/s	127t/s	128.8t/s	0.16t/s	0.14t/s	0.15t/s	0.17t/s	0.22t/s
Codestral (May '24)	Open	33k	20	0.33%	0.26%	0.05%	0.21%	0.22%	0.8%	0.35%	0%	0%	0%	$0.3/1M	0.6/1M	0.2/1M	106.5t/s	46.6t/s	103.1t/s	108.4t/s	110t/s	0.3t/s	0.28t/s	0.29t/s	0.35t/s	0.87t/s
Llama 3.2 3B	Open	128k	20	0.35%	0.26%	0.05%	0.08%	0.05%	0.56%	0.49%	0.07%	0%	1103%	$0.05/1M	0.06/1M	0.04/1M	160.4t/s	62.1t/s	87.2t/s	244.1t/s	1584.50t/s	0.45t/s	0.18t/s	0.28t/s	0.53t/s	1.11t/s
DeepSeek R1 Distill Qwen 1.5B	Open	128k	19	0.27%	0.1%	0.03%	0.07%	0.07%	0.45%	0.69%	0.18%	0%	0%	$0.18/1M	0.18/1M	0.18/1M	315.9t/s	259.4t/s	275.1t/s	373.7t/s	390.3t/s	0.25t/s	0.17t/s	0.22t/s	0.27t/s	0.35t/s
Jamba 1.6 Mini	Open	256k	18	0.37%	0.3%	0.05%	0.07%	0.1%	0.43%	0.26%	0.03%	0%	0%	$0.25/1M	0.4/1M	0.2/1M	198.9t/s	183.2t/s	196.8t/s	203.1t/s	205t/s	0.37t/s	0.19t/s	0.3t/s	0.4t/s	0.8t/s
Jamba 1.5 Mini	Open	256k	18	0.37%	0.3%	0.05%	0.06%	0.08%	0.63%	0.36%	0.01%	0%	1176%	$0.25/1M	0.4/1M	0.2/1M	167.4t/s	140.4t/s	160.2t/s	184.8t/s	197.8t/s	0.34t/s	0.16t/s	0.22t/s	0.38t/s	0.58t/s
Mixtral 8x7B	Open	33k	17	0.39%	0.29%	0.05%	0.07%	0.03%	0.01%	0.3%	0%	0%	1114%	$0.7/1M	0.7/1M	0.7/1M	92t/s	28.8t/s	77.8t/s	99.9t/s	102.4t/s	0.3t/s	0.28t/s	0.29t/s	0.35t/s	0.8t/s
Aya Expanse 8B	Open	8k	16	0.31%	0.25%	0.05%	0.07%	0.08%	0.44%	0.32%	0%	0.49%	0%	$0.75/1M	1.5/1M	0.5/1M	168.7t/s	162.2t/s	166.6t/s	172.5t/s	176.6t/s	0.12t/s	0.1t/s	0.11t/s	0.12t/s	0.18t/s
DeepHermes 3 - Llama-3.1 8B	Open	128k	16	0.37%	0.27%	0.04%	0.09%	0.09%	0.54%	0.22%	0%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Command-R	Open	128k	15	0.34%	0.29%	0.05%	0.04%	0.09%	0.42%	0.15%	0%	0%	1179%	$0.26/1M	0.6/1M	0.15/1M	86.9t/s	80.6t/s	84.4t/s	88.3t/s	89.6t/s	0.19t/s	0.18t/s	0.18t/s	0.21t/s	0.22t/s
Command-R (Mar '24)	Open	128k	15	0.34%	0.28%	0.05%	0.05%	0.06%	0.4%	0.16%	0.01%	0%	1149%	$0.75/1M	1.5/1M	0.5/1M	179.5t/s	166.4t/s	176.6t/s	182t/s	185.2t/s	0.14t/s	0.13t/s	0.14t/s	0.16t/s	0.18t/s
Codestral-Mamba	Open	256k	14	0.21%	0.21%	0.05%	0.13%	0.11%	0.8%	0.24%	0%	0%	0%	$0.25/1M	0.25/1M	0.25/1M	93.9t/s	54.6t/s	89.9t/s	95.5t/s	97.5t/s	0.44t/s	0.41t/s	0.42t/s	0.5t/s	0.86t/s
Llama 3.2 1B	Open	128k	10	0.2%	0.2%	0.05%	0.02%	0.02%	0.4%	0.14%	0%	0%	1054%	$0.03/1M	0.05/1M	0.03/1M	181.6t/s	95.4t/s	118.5t/s	275.8t/s	2632.70t/s	0.44t/s	0.17t/s	0.2t/s	0.48t/s	2.51t/s
Mistral 7B	Open	8k	10	0.25%	0.18%	0.04%	0.05%	0.02%	0.4%	0.12%	0%	0%	1008%	$0.25/1M	0.25/1M	0.25/1M	113.5t/s	51.6t/s	110.6t/s	116.9t/s	120.2t/s	0.28t/s	0.26t/s	0.27t/s	0.32t/s	0.65t/s
Llama 2 Chat 7B	Open	4k	8	0.16%	0.23%	0.06%	0%	0%	0%	0.06%	0%	0%	1037%	$0.1/1M	0.25/1M	0.05/1M	132t/s	108t/s	127.8t/s	134.1t/s	135t/s	0.43t/s	0.38t/s	0.41t/s	8.08t/s	20.73t/s
Reka Flash (Feb '24)	Proprietary	128k	0	0%	0%	0%	0%	0%	0.61%	0.33%	0%	0%	1165%	$0.35/1M	0.8/1M	0.2/1M	46.1t/s	45t/s	45.8t/s	46.3t/s	46.4t/s	0.93t/s	0.86t/s	0.88t/s	0.97t/s	1.03t/s
Claude 3.5 Sonnet (June)	Proprietary	200k	0	0.75%	0.56%	0.04%	0%	0.32%	0.9%	0.7%	0.1%	0%	1268%	$6.0/1M	15.0/1M	3.0/1M	80.3t/s	69.8t/s	78.5t/s	82.6t/s	85.7t/s	0.62t/s	0.5t/s	0.54t/s	0.71t/s	1.17t/s
Llama 2 Chat 70B	Open	4k	0	0.41%	0.33%	0.05%	0.1%	0%	0%	0.32%	0%	0%	1093%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
R1 1776	Open	128k	0	0%	0%	0%	0%	0%	0%	0.95%	0%	0%	0%	$3.5/1M	8.0/1M	2.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
o3	Proprietary	128k	0	0%	0.83%	0.18%	0.53%	0%	0%	0.99%	0.9%	0%	0%	$17.5/1M	40.0/1M	10.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Reka Core	Proprietary	128k	0	0%	0%	0%	0%	0%	0.73%	0.56%	0%	0%	1199%	$2.0/1M	2.0/1M	2.0/1M	27.8t/s	26.6t/s	27.3t/s	28.1t/s	28.4t/s	0.96t/s	0.87t/s	0.91t/s	1.02t/s	1.16t/s
Gemini 2.5 Flash Preview	Proprietary	1m	0	0%	0.7%	0%	0%	0%	0%	0%	0%	0%	0%	$0.26/1M	0.6/1M	0.15/1M	214.9t/s	176.4t/s	195.8t/s	229.5t/s	375.8t/s	10.76t/s	6.08t/s	9.77t/s	11.45t/s	14.38t/s
GPT-4	Proprietary	8k	0	0%	0%	0%	0%	0%	0%	0%	0%	0%	1186%	$37.5/1M	60.0/1M	30.0/1M	32.1t/s	21.6t/s	26.2t/s	34.7t/s	37.7t/s	0.65t/s	0.55t/s	0.59t/s	0.7t/s	0.89t/s
Sonar Reasoning Pro	Proprietary	127k	0	0%	0%	0%	0%	0%	0%	0.96%	0.79%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
GPT-4o (Aug '24)	Proprietary	128k	0	0%	0.52%	0.03%	0.32%	0%	0.93%	0.8%	0.12%	0.87%	1337%	$4.38/1M	10.0/1M	2.5/1M	126.1t/s	76.9t/s	95t/s	144.4t/s	163.3t/s	0.52t/s	0.44t/s	0.48t/s	0.54t/s	0.58t/s
Reka Flash	Proprietary	128k	0	0%	0%	0%	0%	0%	0.74%	0.53%	0%	0%	0%	$0.35/1M	0.8/1M	0.2/1M	46.2t/s	45.4t/s	46t/s	46.2t/s	46.4t/s	0.95t/s	0.86t/s	0.89t/s	1t/s	1.07t/s
Gemini 2.0 Flash Thinking exp. (Dec '24)	Proprietary	2m	0	0%	0%	0%	0%	0%	0.94%	0.48%	0%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
GPT-4o (ChatGPT)	Proprietary	128k	0	0.77%	0.51%	0.04%	0%	0.33%	0.94%	0.8%	0.1%	0%	0%	$7.5/1M	15.0/1M	5.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
DeepSeek Coder V2 Lite	Open	128k	0	0.43%	0.32%	0.05%	0.16%	0.14%	0%	0%	0%	0%	0%	$0.09/1M	0.18/1M	0.06/1M	115.7t/s	105t/s	112.5t/s	118.5t/s	121.5t/s	0.6t/s	0.55t/s	0.58t/s	0.67t/s	0.81t/s
o1-preview	Proprietary	128k	0	0%	0%	0%	0%	0%	0.96%	0.92%	0%	0.86%	1334%	$26.25/1M	60.0/1M	15.0/1M	162.9t/s	77t/s	142.5t/s	171t/s	180.2t/s	18.56t/s	12.6t/s	15.57t/s	21.4t/s	27.82t/s
Jamba Instruct	Proprietary	256k	0	0.34%	0.27%	0.05%	0.05%	0.08%	0%	0.24%	0%	0%	0%	$0.55/1M	0.7/1M	0.5/1M	164.6t/s	142t/s	155.6t/s	175.9t/s	188.8t/s	0.33t/s	0.17t/s	0.25t/s	0.41t/s	0.55t/s
Llama 3.1 Nemotron Ultra 253B Reasoning	Open	128k	0	0%	0.73%	0.08%	0.64%	0%	0%	0.96%	0.75%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Gemini 2.5 Flash Preview (Reasoning)	Proprietary	1m	0	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	$0.99/1M	3.5/1M	0.15/1M	203.6t/s	145.3t/s	178.3t/s	299.4t/s	389.4t/s	11.44t/s	6.67t/s	10.27t/s	12.07t/s	15.76t/s
GPT-4 Turbo	Proprietary	128k	0	0.69%	0%	0.03%	0.29%	0.32%	0.92%	0.74%	0.15%	0%	1256%	$15.0/1M	30.0/1M	10.0/1M	46t/s	21.5t/s	35.9t/s	51.1t/s	59.7t/s	0.73t/s	0.56t/s	0.61t/s	0.89t/s	1.52t/s
DeepSeek LLM 67B (V1)	Open	4k	0	0%	0%	0%	0%	0%	0.75%	0%	0%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
OpenChat 3.5	Open	8k	0	0.31%	0.23%	0.05%	0.12%	0%	0.68%	0.31%	0%	0%	1076%	$0.06/1M	0.06/1M	0.06/1M	47.4t/s	30.4t/s	40.7t/s	52.5t/s	68t/s	0.24t/s	0.15t/s	0.2t/s	0.32t/s	0.46t/s
Gemma 3 1B	Open	32k	0	0.1%	0.19%	0%	0.01%	0%	0.32%	0.14%	0.01%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Qwen Chat 72B	Open	34k	0	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	$1.0/1M	1.0/1M	1.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
DeepSeek-Coder-V2	Open	128k	0	0%	0%	0%	0%	0%	0.87%	0.74%	0%	0%	1178%	$0.17/1M	0.28/1M	0.14/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
DeepSeek-V2	Open	128k	0	0%	0%	0%	0%	0%	0.87%	0%	0%	0%	1220%	$0.17/1M	0.28/1M	0.14/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Solar Mini	Open	4k	0	0%	0%	0%	0%	0%	0.59%	0.33%	0%	0%	1062%	$0.15/1M	0.15/1M	0.15/1M	49.9t/s	42.7t/s	46.7t/s	86.6t/s	89.2t/s	1.07t/s	0.99t/s	1.04t/s	1.12t/s	1.24t/s
Gemma 2 27B	Open	8k	0	0.57%	0.36%	0.04%	0%	0.13%	0.76%	0.54%	0.09%	0%	1219%	$0.26/1M	0.51/1M	0.17/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Qwen1.5 Chat 110B	Open	32k	0	0%	0.29%	0%	0%	0%	0%	0%	0%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	23.7t/s	23.6t/s	23.6t/s	23.8t/s	23.8t/s	1.61t/s	1.48t/s	1.52t/s	1.68t/s	1.74t/s
DeepSeek-V2.5 (Dec '24)	Open	128k	0	0%	0%	0%	0%	0%	0.88%	0.76%	0%	0%	0%	$0.17/1M	0.28/1M	0.14/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
DeepSeek-V2.5	Open	128k	0	0%	0%	0%	0%	0%	0.87%	0%	0%	0%	1258%	$0.17/1M	0.28/1M	0.14/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Grok 3 mini	Proprietary	1m	0	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
GPT-4o mini Realtime (Dec '24)	Proprietary	128k	0	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Llama 3.2 11B (Vision)	Open	128k	0	0.46%	0.22%	0.05%	0.11%	0%	0.69%	0.52%	0.09%	0%	0%	$0.16/1M	0.16/1M	0.16/1M	103.6t/s	41t/s	82.7t/s	128.2t/s	144.1t/s	0.35t/s	0.13t/s	0.24t/s	0.44t/s	0.51t/s
Arctic	Open	4k	0	0%	0%	0%	0%	0%	0%	0%	0%	0%	1090%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
Mistral Saba	Proprietary	32k	0	0.61%	0.42%	0.04%	0%	0.24%	0.85%	0.68%	0.13%	0%	0%	$0.3/1M	0.6/1M	0.2/1M	91.6t/s	47.5t/s	76.5t/s	95.3t/s	99.3t/s	0.32t/s	0.29t/s	0.3t/s	0.38t/s	0.72t/s
Claude 2.0	Proprietary	100k	0	0.49%	0.34%	0%	0.17%	0.19%	0%	0%	0%	0%	1132%	$12.0/1M	24.0/1M	8.0/1M	30.9t/s	29.8t/s	30.4t/s	31.6t/s	33.3t/s	0.83t/s	0.79t/s	0.8t/s	0.87t/s	0.95t/s
Grok 3 Reasoning Beta	Proprietary	1m	0	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
GPT-4o Realtime (Dec '24)	Proprietary	128k	0	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	$0.0/1M	0.0/1M	0.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s
GPT-4.5 (Preview)	Proprietary	128k	0	0%	0.71%	0%	0%	0%	0%	0%	0%	0%	0%	$93.75/1M	150.0/1M	75.0/1M	74.9t/s	51.3t/s	61.3t/s	84.2t/s	91.4t/s	0.93t/s	0.84t/s	0.9t/s	1.08t/s	1.34t/s
Reka Edge	Proprietary	128k	0	0%	0%	0%	0%	0%	0.41%	0.22%	0%	0%	0%	$0.1/1M	0.1/1M	0.1/1M	84.6t/s	81.7t/s	82.9t/s	86.6t/s	87.9t/s	0.92t/s	0.83t/s	0.86t/s	0.97t/s	1.05t/s
Claude 3 Haiku	Proprietary	200k	0	0%	0%	0%	0.15%	0.19%	0.76%	0.39%	0.01%	0.68%	1179%	$0.5/1M	1.25/1M	0.25/1M	140.9t/s	116.3t/s	134.7t/s	147.9t/s	157t/s	0.47t/s	0.36t/s	0.4t/s	0.64t/s	0.98t/s
Gemini 2.0 Flash-Lite (Preview)	Proprietary	1m	0	0%	0.54%	0.04%	0.18%	0.25%	0.9%	0.87%	0.3%	0%	0%	$0.13/1M	0.3/1M	0.07/1M	198.2t/s	177t/s	189.9t/s	209.6t/s	216.6t/s	0.29t/s	0.24t/s	0.26t/s	0.31t/s	0.36t/s
Sonar Reasoning	Proprietary	127k	0	0%	0.62%	0%	0%	0%	0%	0.92%	0.77%	0%	0%	$2.0/1M	5.0/1M	1.0/1M	78t/s	67.9t/s	75.8t/s	80.9t/s	85.9t/s	1.8t/s	1.43t/s	1.54t/s	2.05t/s	2.46t/s
o1-pro	Proprietary	200k	0	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	$262.5/1M	600.0/1M	150.0/1M	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s	0t/s

🚀Home 📦Tools 📖Blog leaderboards