Artificial Analysis

FLUX.2 [dev] is the new leading open weights text to image model, surpassing HunyuanImage 3.0, Qwen-Image, and HiDream-I1-Dev in the Artificial Analysis Image Arena! Black Forest Labs latest release claims the top spot for open weights text to image generation, while also ranking #2 in open weights Image Editing, trailing only Alibaba's Qwen Image Edit 2509. FLUX.2 [dev] is released under the FLUX [dev] Non-Commercial License with weights available on Hugging Face. Commercial applications require a separate license from Black Forest Labs. The model is available via API on fal, Replicate, Runware, Verda, Together, Cloudflare, and Deepinfra. Black Forest Labs has also announced FLUX.2 [klein], which will be released under the Apache 2.0 license, enabling developers and businesses to build commercial applications without separate licensing requirements from Black Forest Labs

DeepSeek V3.2 is the #2 most intelligent open weights model and also ranks ahead of Grok 4 and Claude Sonnet 4.5 (Thinking) - it takes DeepSeek Sparse Attention out of ‘experimental’ status a couples it with a material boost to intelligence DeepSeek V3.2 scores 66 on the Artificial Analysis Intelligence Index; a substantial intelligence uplift over DeepSeek V3.2-Exp (+9 points) released in September 2025. DeepSeek has switched its main API endpoint to V3.2, with no pricing change from the V3.2 Exp pricing - this puts pricing at just $0.28/$0.42 per 1M input/output tokens, with 90% off for cached input tokens. Since the original DeepSeek V3 release ~11 moths ago in late December 2024, DeepSeek’s V3 architecture with 671B total/37B active parameters has seen them go from a model scoring a 32 to scoring a 66 in Artificial Analysis Intelligence Index. DeepSeek has also released V3.2-Speciale, a reasoning-only variant with enhanced capabilities but significantly higher token usage. This is a common tradeoff in reasoning models, where more enhanced reasoning generally yields higher intelligence scores and more output tokens. V3.2-Speciale is available via DeepSeek's first-party API until December 15.

Amazon has launched a new speech-to-speech model, Nova Sonic 2.0, which ranks #2 on our Artificial Analysis Big Bench Audio Speech Reasoning benchmark! The new model achieves a reasoning accuracy score of 87.1% on Big Bench Audio, placing second overall behind Google’s Gemini 2.5 Flash Native Audio Thinking and above other offerings including GPT Realtime Performance: ➤ Reasoning: Achieves 87.1% on Big Bench Audio, ranking second on the Artificial Analysis Speech to Speech reasoning leaderboard between Google’s Gemini 2.5 Flash Native Audio Thinking and OpenAI’s GPT Realtime, Aug ‘25 ➤ Latency: At an average time to first audio of 1.39 seconds, the new model is >2 seconds faster than the leading reasoning model, Gemini 2.5 Flash Native Audio Thinking, but slower than OpenAI’s most recent models Model details: ➤ Provided via an API with bidirectional audio streaming, enabling real-time, multi-turn conversation ➤ Uses adaptive speech response that dynamically adjusts delivery based on the prosody of input speech ➤ Supports five languages including English (US, UK), French, Italian, German, and Spanish Benchmark context: Big Bench Audio is the first dedicated dataset for evaluating reasoning performance of speech models. Big Bench Audio comprises 1,000 audio questions adapted from the Big Bench Hard text test set, chosen for its rigorous testing of advanced reasoning, translated into the audio domain.

Mistral just launched their new large open weights model, Mistral Large 3 (675B total, 41B active), alongside a set of three Ministral models (3B, 8B, 14B) Mistral has released Instruct (non-reasoning) variants of all four models, as well as reasoning variants of the three Ministral models. All models support multimodal inputs and are available with an Apache 2.0 license today on Hugging Face. We evaluated Mistral Large 3 and the Instruct variants of the three Ministral models prior to launch. Mistral’s highest scoring model in Artificial Analysis Intelligence Index remains the proprietary Magistral Medium 1.2, launched a couple of months back in September - this is due to reasoning giving models a significant advantage in many evals we use. Mistral discloses that a reasoning version of Mistral Large 3 is already in training and we look forward to evaluating it soon! Key highlights: ➤ Large and small models: at 675B total with 41B active, Mistral Large 3 is Mistral’s first open weights mixture-of-experts model since Mixtral 8x7B and 8x22B in late 2023 to early 2024. The Ministral releases are dense with 3B, 8B, and 14B parameter variants ➤ Significant intelligence increase but not amongst leading models (including proprietary): Mistral Large 3 represents a significant upgrade compared to the previous Mistral Large 2 with a +11 point increase on the Intelligence Index up to 38. However, Large 3 still trails leading proprietary reasoning & non-reasoning models ➤ Versatile small models: the Ministral models are released with Base, Instruct, and Reasoning variant weights - we tested only the Instruct variants ahead of release, which achieved Index scores of 31 (14B), 28 (8B), and 22 (3B). This places Ministral 14B ahead of the previous Mistral Small 3.2 with 40% fewer parameters. We are working on evaluating the reasoning variants and will share their intelligence results soon. ➤ Multi-modal capabilities: all models in the release support text and image inputs - this is a significant differentiator for Mistral Large 3, as few open weight models in its size class have support for image input. Context length also increases to 256k, enabling larger-input tasks. These new models from Mistral are not a step change from open weights competition, but they represent a strong performance base with vision capabilities. The Ministral 8B and 14B variants offer particularly compelling performance for their size, and we’re excited to see how the community uses and builds on these models. At launch, the new models are available for serverless inference on Mistral AI and a range of other providers including Amazon Web Services (AWS) Bedrock, Microsoft Azure AI Foundry, IBM watsonx, Fireworks AI, Together AI, and Modal. Mistral Large 3 trails the frontier, but notably is one of the most intelligent open weights multimodal non-reasoning models. Recent models from DeepSeek (v3.2) and Moonshot (Kimi K2) continue to only support text input and output.

All three FLUX.2 variants rank in the top 10 of the Artificial Analysis Image Editing Leaderboard, with [flex] and [pro] beating out GPT-5 and the previous generation FLUX.1 Kontext models FLUX.2 is a family of image models from Black Forest Labs, coming in pro, flex and dev variants. All variants support both text to image and image editing. One interesting point is that the pricing for image editing is effectively doubled, with the listed price per megapixel applying to both input and output megapixels. FLUX.2 [flex] tested at 50 inference steps with 4.5 guidance scale ranks #4 in Image Editing at $120/1k images, trailing Nano Banana Pro ($134/1k images), and Seedream 4.0 ($30/1k images). At its highest settings, [flex] offers the best editing quality in the FLUX.2 family for users willing to pay the premium. FLUX.2 [pro] comes in at #5 at $45/1k images for editing, offering strong performance at a more accessible price point than [flex]. Both [pro] and [flex] surpass GPT-5 and the previous FLUX.1 Kontext [max] ($80/1k) in editing quality. FLUX.2 [dev] ranks #9 at $24/1k images on hosted providers, making it the most cost-effective option in the family while still outperforming FLUX.1 Kontext [max] and [pro]. Notably, all three FLUX.2 variants surpass the previous generation FLUX.1 Kontext models, with even [dev] beating Kontext [max] despite costing less than a third of the price.

FLUX.2 [pro] ranks #2 in the Artificial Analysis Text to Image Leaderboard, trailing only Nano Banana Pro (Gemini 3.0 Pro Image) while costing less than a quarter of the price! FLUX.2 is a family of image models from Black Forest Labs, coming in pro, flex and dev variants. All variants support both text to image and image editing. FLUX.2 [pro] comes in at #2 in the Text to Image Leaderboard and is positioned by BFL as the best balance of generation speed and quality. We observe generation times of ~10s from Black Forest Labs' API, comparable to FLUX.1 Kontext [max] (10s) and Seedream 4.0 (12s). FLUX.2 [pro] is priced at $30/1k 1MP images, matching Seedream 4.0 and substantially cheaper than Nano Banana Pro (Gemini 3.0 Pro Image) at $39/1k. FLUX.2 [flex] ranks #4 in Text to Image, tested at 50 inference steps with 4.5 guidance scale. This variant offers the most control, with adjustable guidance scale and inference steps for maximum quality. The model is priced higher than the pro variant at $60/1k 1MP images regardless of settings, making it more expensive than Nano Banana (Gemini 2.5 Flash Image) at $39/1k. Generation times run ~20s at default settings, among the slowest diffusion models in our benchmarking. FLUX.2 [dev] comes in at #8 in the Text to Image Leaderboard and is the open weights variant under the FLUX [dev] Non-Commercial license. The 32B parameter model is sized up from FLUX.1 [dev]'s 12B, designed for professional hardware with fp8 quantized versions available for consumer use. FLUX.2 [klein] is also planned, a size-distilled variant under Apache 2.0 license that may succeed the popular FLUX.1 [schnell].

Amazon is back with Nova 2.0, a substantial upgrade over prior Amazon Nova models and demonstrating particular strength in agentic capabilities Amazon has released Nova 2.0 Pro (Preview), its new flagship model; Nova 2.0 Lite, focused on speed and lower cost; and Nova 2.0 Omni, a multimodal model handling text, image, video and speech inputs with text and image outputs. Key benchmarking takeaways: Amazon back amongst top AI players: This is Amazon’s latest release since Nova Premier and Amazon’s first release of reasoning models. Nova 2.0 Pro jumps 30 points in the Artificial Analysis Intelligence Index over Premier and Lite 38 points. This represents a huge increase in capabilities and Amazon’s return to being amongst the top AI players. Strengths in agentic capabilities: Agentic capabilities including tool calling is a strength of the models, Nova 2.0 Pro scores 93% on τ²-Bench Telecom and 80% on IFBench on medium and high reasoning budgets respectively (complete benchmarks for high reasoning coming soon). This places Nova 2.0 Pro Preview amongst the leading models in these benchmarks. Multimodal: Nova 2.0 Omni is one of few models, alongside most notably the Gemini model series, that can natively handle text, image, video and speech inputs. This is a new differentiator for Amazon’s Nova model series. Competitive pricing: Amazon has priced Nova 2.0 Pro at $1.25/$10 per million input/output tokens, and considering token usage the model took $662 to run our Artificial Analysis Intelligence Index. This is substantially less than other frontier models like Claude 4.5 Sonnet ($817) and Gemini 3 Pro ($1201), but remains above others including Kimi K2 Thinking ($380). Nova 2.0 Lite and Omni are both priced at $0.3/$2.5 per million input/output tokens. See below for further analysis

Vidu Q2 ranks #4 in Image Editing in the Artificial Analysis Image Editing Arena, surpassing GPT-5 and Qwen Image Edit 2509! Vidu AI, best known for their Vidu Q2 video generation model, has released an image model with the same name (Vidu Q2) that supports text to image, image editing, and multi-image input editing with up to 7 image inputs. In image editing (with single image inputs), Vidu Q2 ranked #4 in the Artificial Analysis Image Leaderboard delivering outputs surpassing GPT-5 and Qwen Image Edit 2509. In Text to Image, Vidu Q2 ranked #13, delivering quality comparable to FLUX.1 Kontext [max]. The model supports 1K, 2K, and 4K resolutions across a variety of aspect ratios. Vidu Q2 is available in the Vidu application, with 1K resolution generation free until the end of 2025. Once generated, they can be used as input images for the Vidu Q2 video generation model’s Image to Video capability. Vidu Q2 is also available on the Vidu API Platform at $30/1k images for text to image, and $40/1k images for image editing.

Introducing the Artificial Analysis Openness Index: a standardized and independently assessed measure of AI model openness across availability and transparency Openness is not just the ability to download model weights. It is also licensing, data and methodology - we developed a framework underpinning the Artificial Analysis Openness Index to incorporate these elements. It allows developers, users, and labs to compare across all these aspects of openness on a standardized basis, and brings visibility to labs advancing the open AI ecosystem. A model with a score of 100 in Openness Index would be open weights and permissively licensed with full training code, pre-training data and post-training data released - allowing users to not just use the model but reproduce its training in full, or take inspiration from some or all of the model creator’s approach to build their own model. We have not yet awarded any models a score of 100! Key details: 🔒 Few models and providers take a fully open approach. We see a strong and growing ecosystem of open weights models, including leading models from Chinese labs such as Kimi K2, Minimax M2, and DeepSeek V3.2. However, releases of data and methodology are much rarer - OpenAI’s gpt-oss family is a prominent example of open weights and Apache 2.0 licensing, but minimal disclosure otherwise. 🥇 OLMo from Ai2 leads the Openness Index at launch. Living up to AI2’s mission to provide ‘truly open’ research, the OLMo family achieves the top score of 89 on the Index (16 of a maximum of 18 points) by prioritizing full replicability and permissive licensing across weights, training data, and code. With the recent launch of OLMo 3, this included the latest version of AI2’s data, utilities and software, full details on reasoning model training, and the new Dolci post-training dataset. 🥈 NVIDIA’s Nemotron family also performs strongly for openness. NVIDIA AI models such as NVIDIA Nemotron Nano 9B v2 reach a score of 67 on the Index due to their release alongside extensive technical reports detailing their training process, open source tooling for building models like them, and the Nemotron-CC and Nemotron post-training datasets. Methodology & Context: ➤ We analyze openness using a standardized framework covering model availability (weights & license) and model transparency (data and methodology). This means we capture not just how freely a model can be used, but visibility into its training and knowledge, and potential to replicate or build on its capabilities or data. ➤ AI model developers may choose not to fully open their models for a wide range of reasons. We feel strongly that there are important advantages to the open AI ecosystem and supporting the open ecosystem is a key reason we developed the Openness Index. We do not, however, wish to dismiss the legitimacy of the tradeoffs that greater openness comes with, and we do not intend to treat Openness Index as a strictly ‘higher is better’ scale.

Google TPU v6e vs AMD MI300X vs NVIDIA H100/B200: Artificial Analysis’ Hardware Benchmarking shows NVIDIA achieving a ~5x tokens-per-dollar advantage over TPU v6e (Trillium), and a ~2x advantage over MI300X, in our key inference cost metric In our metric for inference cost called Cost Per Million Input and Output Tokens at Reference Speed, we see NVIDIA H100 and B200 systems achieving lower overall cost than TPU v6e and MI300X. For Llama 3.3 70B running with vLLM at a Per-Query Reference Speed of 30 output tokens/s, NVIDIA H100 achieves a Cost Per Million Input and Output Tokens of $1.06, compared to MI300X at $2.24 and TPU v6e at $5.13. This analysis relies on results of the Artificial Analysis System Load Test for system inference throughput across a range of concurrency levels, and GPU instance pricing data we collect from a range of GPU cloud providers. “Cost Per Million Input and Output Tokens at Reference Speed” uses the system throughput that the system can achieve while maintaining 30 output tokens per second per query, and divides the system’s rental cost by that throughput (scaled to a million tokens). Full results across a range of concurrency and speed levels are available on the Artificial Analysis Hardware Benchmarking page. Important context: ➤ We are only reporting results for TPU v6e running Llama 3.3 70B because this is the only model on our hardware page for which vLLM on TPU is officially supported. We report results for NVIDIA Hopper and Blackwell systems, and now for AMD MI300X, across all four models on our hardware page: gpt-oss-120b, Llama 4 Maverick, DeepSeek R1 and Llama 3.3 70B. ➤ These results are based on what companies can rent now in the cloud - next generation MI355X and TPU v7 accelerators are not yet widely available. We take the lowest price across a reference set of GPU cloud providers. TPU v6e is priced for on-demand at $2.70 per chip per hour, which is cheaper than our lowest tracked price for NVIDIA B200 ($5.50 per hour) but similar to NVIDIA H100 ($2.70 per hour) and AMD MI300X ($2 per hour). ➤ Google’s TPU v7 (Ironwood) is becoming generally available in the coming weeks. We would anticipate TPU v7 outperforming v6e substantially, given leaps in compute (918 TFLOPS to 4,614 TFLOPS), memory (32GB to 192GB) and memory bandwidth (1.6 TB/s to 7.4 TB/s). However, we don’t yet know what Google will charge for these instances - so the impact on implied per token costs is not yet clear. ➤ Our Cost per Million Input and Output Tokens metric can’t be directly compared to serverless API pricing. The overall implied cost per million tokens for a given deployment is affected by the per-query speed you want to aim for (driven by batch size/concurrency) and the ratio of input to output tokens. ➤ These results are all for systems with 8 accelerators - ie. 8xH100, 8xB200, 8xTPU v6e, 8xMI300X. We’ve also recently published updated Blackwell results - more analysis of these coming soon.