Kimi K3.1 Pro Leads Open-Weight Models in Artificial Analysis Index v5

What changed in the Intelligence Index?

On October 27, Artificial Analysis announced Intelligence Index v5, a major revision to its composite measure of AI model capabilities. The index now combines nine evaluations across four equally weighted areas: knowledge work, coding, science, and general capabilities.

Two evaluations are new: FrontierCode 2.0, which tests real-world coding tasks, and Terminal-Bench-Science, which tests research workflows in a terminal. The update also replaces Humanity’s Last Exam with HLE-Diamond and revises AA-Omniscience and CritPt.

Artificial Analysis says 60% of the index now comes from evaluations using private test sets or held-out answers. This can reduce the risk of models benefiting from exposure to public benchmark questions, although private material also makes independent inspection harder.

Each of the four areas accounts for 25% of the result. Within coding, FrontierCode 2.0 contributes 15% of the total index and Terminal-Bench 4.0 contributes 10%. Science combines Terminal-Bench-Science at 15% and the CritPt v2 hard subset at 10%. Knowledge work and general capabilities each combine several assessments as well.

Those choices matter. An aggregate is useful for a broad comparison, but it expresses the evaluator’s selection and weighting of tasks. A model that is excellent at one coding workflow can still score lower overall if it is weaker on knowledge or scientific reasoning. The new version also removes GDPval-AA v2.1, SciCode and AA-LCR v1.1 from the composite.

Where Kimi K3.1 Pro stands

In the v5 announcement, Kimi K3.1 Pro scores 46, making it the highest-scoring model in Artificial Analysis’s open-weight category. Claude Opus 5.5 leads the overall index at 58, with GPT-6 Astra at 57 and GPT-6.1 Sol at 55, all at the stated maximum effort setting.

This is a selected comparison, not the full ranking. Artificial Analysis notes that Kimi K3.1 Pro’s weights were not yet released at the time of its announcement; DeepSeek V4.1 Flash 1017 and GLM-5.3 led among models with released weights. Kimi says it plans to release the full weights by November 4. Read Kimi’s announcement.

The gap between the overall leader and Kimi widened from three points in v4.3 to 12 points in v5, according to Artificial Analysis. Because the tests and weighting changed, a lower score across versions should not be read as the model itself becoming less capable.

For context, the Reddit post compares earlier and v5 scores for several models, including DeepSeek V4.1 Flash 1017 and GLM-5.3 at 45 versus 32. Those figures describe a change in index methodology as well as a change in reported numbers; they do not measure a loss in the underlying model’s ability. Artificial Analysis also introduced a separate Progress Index designed to track capability over time on a continuous scale.

Why the new scores sparked debate

In a Reddit discussion, developers questioned whether the new rankings reflect their own experience, while others argued that benchmarks need to evolve as older tests become saturated. These are community opinions, not evidence that the index is biased or that a model has changed.

The practical lesson is to look beyond a single aggregate: v5 gives coding one quarter of the overall weight, alongside knowledge work, science and general capabilities. A model that suits your workflow may perform differently on the particular tasks you care about.

A standout result for coding agents

The headline score is only part of the story. Artificial Analysis reports that Kimi K3.1 Pro leads Terminal-Bench 4.0 at 67% among the models it measured. That result is particularly relevant to developers interested in agents that work through terminal-based tasks.

Terminal-Bench is one evaluation, not a guarantee that a model will win every coding task. Repository size, tool configuration, prompts, latency, and the work itself all affect real-world results. Still, Kimi’s result makes it a compelling model to put through its paces on your own projects.

In AA’s breakdown, Kimi’s lead is specific to Terminal-Bench 4.0. On the new FrontierCode 2.0 assessment, Claude Opus 5.5 leads at 46%; the two benchmarks examine different coding tasks, so neither result should be substituted for the other. In the overall index, the two evaluations are also weighted differently.

Where the harder tests show a gap

AA’s new science and knowledge assessments provide a useful counterpoint. On Terminal-Bench-Science, Kimi scores 34%, compared with 63% for GPT-6 Astra (max). On HLE-Diamond, Kimi scores 37%, compared with 68% for Claude Opus 5.5 (max). These figures explain why a model can lead a coding benchmark while trailing in a broader composite.

The scale in this figure ends at 70% to make the comparison easier to read; the labels show actual success rates. A different benchmark may produce a different ordering, and a scientific research score is not a proxy for everyday coding performance.

Cost and reasoning effort matter too

AA reports that costs per index task vary by more than 13 times among the ten highest-scoring models shown. Its measured cost per task is specific to the evaluation, provider and configuration; it is not the price of a typical coding request or a SiAPI.ai plan. Reasoning effort also changes both performance and usage, so compare models with settings that make sense for your work.

What developers can try

Start with work you can evaluate: ask Kimi to explain an unfamiliar codebase, investigate a reproducible bug, implement a small feature, or propose tests for an existing module. Review the diff and run your normal checks, then compare the outcome, time, and usage with your current workflow.

Kimi describes K3.1 Pro as a 3T-class model with a context window of up to one million tokens. On SiAPI.ai, the API model ID is kimi-k3.1-pro; the model reference lists Low, High, and Max reasoning choices. The available context and actual results depend on your configuration and task.

For a useful comparison, give each model the same task and repository context, set a time or usage budget, and assess whether the resulting change passes tests and is maintainable. Note where an agent needs help rather than judging only the first response. Long context can help with complex projects, but supplying more context is not a substitute for choosing relevant files and reviewing generated changes.