AI Team Defeats Opus 4.8 in Programming Challenge

Sakana AI released Fugu, a tool that outperforms the best individual models by working together with a group of models. Fugu Ultra achieves a score of 73.7 on SWE Bench Pro (a benchmark for debugging code in real projects) compared to 69.2 from Anthropic’s Opus 4.8, 58.6 from GPT 5.5, and 54.2 from Gemini 3.1 Pro. It also leads the pack on TerminalBench 2.1 (82.1), LiveCodeBench (93.2), and the GPQA-Diamond science exam (95.5), according to Sakana, and the company claims that it can match Anthropic’s Fable 5 and Mythos (which aren’t included in Fugu’s model set) in performance.

How does it work? Fugu is a trained model that manages other models. When presented with a query, Fugu either responds itself or passes the job along to another model in its interchangeable set. It handles choosing which model to use, sending the request, checking results internally, and aggregating responses. From the perspective of the user, Fugu acts as a single model accessible via an OpenAI-compatible API. There are currently two variants: Fugu Base, optimized for low latency tasks such as coding and conversation, and Fugu Ultra, designed for complex tasks such as research, cybersecurity, and patent research.

1Why a committee beats a soloist

These results aren’t theoretical. According to one developer, Fugu Ultra found over twenty problems in a code review while GPT-5.5 only identified around three. This makes sense because each model has its own weaknesses, so a system that can direct requests to the correct expert and verify the result will catch mistakes that a single model might overlook. Sakana backs this claim up with references to their papers Trinity and Conductor, published at ICLR 2026. Founded by ex-Google researchers Llion Jones (co-author of the 2017 Transformer paper) and David Ha.

Both models are accessible via an API and console now with pricing options based on subscription and usage. You can even instruct Fugu to remove certain models from consideration for regulatory purposes.

2Why it matters for you

This has two advantages. First, quality. If a group of models consistently beats the best individual model, then you know that your potential results are higher when you make requests to a single API. Second, freedom. As Sakana puts it, “Fugu is a safeguard against single-provider dependence,” referencing situations where everyone lost access to a top model overnight. Having a system that can automatically switch providers gives you peace of mind, even if you don’t care about beating benchmarks.

Thoughts: Orchestration is more interesting than benchmark scores. For two years we’ve been wondering which single model is the best. Now we know that asking which combination is the best might be the right question. And it just got some results.