AI
“Moonshot AI's Kimi K3.1 Pro scored 65.7% on Terminal-Bench 4.0, ahead of GPT-6 Astra's 58.2% but just behind Claude Opus 5.5's 66.4%.”
Plain restatementKimi K3.1 Pro achieved 65.7% on the Terminal-Bench 4.0 evaluation, a result that ranks it above GPT-6 Astra (58.2%) and marginally below Claude Opus 5.5 (66.4%).
The numbers in this post are quoted correctly, but they are not what they look like. All three come straight from Moonshot AI's own launch page for its new Kimi K3.1 Pro model, and Moonshot's own footnote explains that it ran its model in one setup while simply taking each competitor's best previously published score from their blogs or a public board. That means the three figures were produced by three different parties using different settings, so lining them up as a single ranking is not a fair comparison. The claim that Kimi came in "just behind" Claude Opus 5.5 also rests on a 0.7 point gap, which is smaller than the margin of error Anthropic publishes for its own score, so the two results cannot really be separated. Notably, the independent evaluator Artificial Analysis ran all the models through one identical setup and found Kimi ahead of Claude Opus 5.5, not behind it, which cuts against the post's ordering in Kimi's favour. Kimi K3.1 Pro does appear to be a genuinely strong model, and independent testing supports that. What is not supported is treating a company's own marketing table as a measured head to head result.
[drifted from the evidence:] Moonshot AI's Kimi K3.1 Pro [drifted from the evidence:] scored 65.7% on Terminal-Bench 4.0, [drifted from the evidence:] ahead of GPT-6 [drifted from the evidence:] Astra's 58.2% [drifted from the evidence:] but just behind Claude Opus [drifted from the evidence:] 5.5's 66.4%.
Kimi K3.1 Pro [added by the neutral restatement:] achieved 65.7% on [added by the neutral restatement:] the Terminal-Bench 4.0 [added by the neutral restatement:] evaluation, a result that ranks it above GPT-6 [added by the neutral restatement:] Astra (58.2%) [added by the neutral restatement:] and marginally below Claude Opus [added by the neutral restatement:] 5.5 (66.4%).
Red-tinted words in the claim drifted from the evidence. Green-tinted words are what a neutral restatement needs.
The trace / claim to source
- Moonshot AI published exactly these three figures in exactly this relationship. The claim's sentence is a faithful transcription of the company's own launch blog.
- Kimi K3.1 Pro is a real model, released by Moonshot AI on 2026-10-23, and is live on the Kimi API and listed on third-party routers.
- The 66.4% figure for Claude Opus 5.5 is a real published number from Anthropic.
- GPT-6 Astra does score in the high 50s on Terminal-Bench 4.0 across several sources, so that leg of the comparison is directionally supported.
- The broader thrust, that K3.1 Pro performs near the strongest proprietary models on agentic coding, is supported by an independent evaluator: Artificial Analysis places it first on its own Terminal-Bench 4.0 run at 67.2% and fourth on its Intelligence Index at 55.
- The claim sentence states the three scores as plain fact with no attribution. Every number in it comes from a vendor's own product launch page. The post's caption does add "according to Kimi" on its coding bullet, but the headline sentence that travels on its own carries no such label, and a reader meets it as a measured result rather than as a company's self-report.
- The three numbers were not produced by one evaluation. Moonshot's own footnote says K3.1 Pro was run in the Claude Code harness while all other scores were taken as each competitor's maximum reported value from their blogs or the public board. Three producers, three setups, presented as a single ranking.
- Margin versus noise: "just behind Claude Opus 5.5" asserts an ordering that the numbers cannot carry. The 0.7-point gap is less than one task on a 66-task benchmark and well inside the plus or minus 2.6 point standard error Anthropic itself publishes for that very figure. On the evidence the two results are indistinguishable, not ranked.
- The 65.7% was obtained at Moonshot's maximum reasoning effort setting. Moonshot's own cost chart is reported to show the same model at 52.8% at high effort. The headline figure is the top of a range that the claim does not mention, and the cost of reaching it is not surfaced.
- Benchmark cherry picking, comparator selection: the comparison set is Opus 5.5 and GPT-6 Astra. Claude Sonnet 5.5, which sits at or above Opus 5.5 on both independent boards checked here, does not appear in it.
- Which ordering is correct is genuinely open. The claim puts K3.1 Pro behind Opus 5.5. Artificial Analysis, running all models through one harness, puts K3.1 Pro ahead of Opus 5.5 by a wide margin. The two cannot both describe the same contest, and no single-harness run reconciling Moonshot's 65.7% with Anthropic's 66.4% exists.
- Whether 65.7% replicates outside the Claude Code harness is unknown. No independent party has reproduced Moonshot's specific figure under Moonshot's specific setup.
- The exact provenance of the 58.2% for GPT-6 Astra is not fully pinned. OpenAI's own launch post lists 57.9%, and 58.2% appears to be a public-board maximum, but the deciding board page was not retrieved here.
- Public boards disagree with each other on the same models, partly over how provider-side fallbacks are counted, so no single current ranking for Terminal-Bench 4.0 can be stated with confidence as of 2026-10-25.
The three numbers in the claim are a verbatim transcription of a sentence Moonshot AI published in its own launch blog. Moonshot writes that on Terminal-Bench 4.0, "K3.1 Pro scores 65.7%, ahead of GPT-6 Astra (58.2%) and narrowly behind Claude Opus 5.5 (66.4%)." The same blog's footnote states how those numbers were produced: "Kimi K3.1 Pro is evaluated with the Claude-Code harness," and "All remaining scores are taken as the maximum value reported in either their blog posts or the official Terminal-Bench 4.0 leaderboard." So the row is not one evaluation. It is Moonshot's own run of its own model placed beside each competitor's best previously published figure, drawn from different harnesses and different producers. Anthropic's own page confirms the provenance of the 66.4%: it reports that Terminal-Bench 4.0 results are given for Opus 5.5 at xhigh effort and for GPT-6 Astra at high effort as reported by OpenAI, and that these represent each model's highest score. Anthropic also publishes the uncertainty: the standard error is plus or minus 2.6 points for Claude Opus 5.5. Independent runners using one fixed harness across all models produce a different ordering. Artificial Analysis places Kimi K3.1 Pro first on Terminal-Bench 4.0 at 67.2%, ahead of Claude Sonnet 5.5 at 63.6% and Claude Opus 5.5 at 59.6%. Vals AI, as of 2026-10-22 and before K3.1 Pro existed, had Opus 5.5 at 65.15%, Sonnet 5.5 at 64.14% and GPT-6 Astra at 59.60% under Mini-SWE-agent. Artificial Analysis separately places K3.1 Pro fourth on its Intelligence Index with a score of 55, three points behind Opus 5.5.
Complete reasoning
The reply is formatted for pasting into the thread where the claim is circulating.
Ask this case
Answers come only from the case file above; nothing is added.
Are the numbers in the claim accurate?
Yes, all three figures are exact quotes from Moonshot AI's own launch blog for Kimi K3.1 Pro. The problem is not the numbers themselves but how they are being compared.
Why is comparing these three scores misleading?
Moonshot's own footnote says it ran Kimi K3.1 Pro itself in one setup, but just took each competitor's best previously published score from their own blogs or a public leaderboard. That means the three figures came from three different producers using different setups, so treating them as one fair ranking is not accurate.
Does the 0.7 point gap really mean Kimi is behind Claude Opus 5.5?
No. Anthropic itself publishes a standard error of plus or minus 2.6 points for that score, which is much larger than the 0.7 point gap. The two results are statistically indistinguishable, not clearly ranked.
What did independent testing find?
Artificial Analysis, which ran all the models through one identical setup, actually placed Kimi K3.1 Pro ahead of Claude Opus 5.5 on Terminal-Bench 4.0, the opposite order from the claim. Vals AI's leaderboard, taken before K3.1 Pro existed, shows a different ranking among the other models too.
Is Kimi K3.1 Pro actually a strong model?
The case file indicates it is. Independent evaluator Artificial Analysis ranks it first on its own Terminal-Bench 4.0 run and fourth on its Intelligence Index, supporting the broader idea that it performs near the top proprietary models, even though the specific claim's comparison method is flawed.