New Benchmark Exposes Coding Agents' Weak Spot

The Alibaba group, together with several Asian universities, has developed a new benchmark called Co-Design Bench to evaluate the performance of code agents on open-domain programming tasks. The benchmark is based on actual user requests and requires the agent to modify web apps so that the implemented features are functional upon testing via a web browser. The best performing agent on this benchmark was OpenAI’s GPT-6 Astra with a score of 26.4%. Other notable performers include Anthropic’s Claude Opus 5 and Alibaba’s Qwen3.8-Max.

Agents like these have been achieving over 99% scores on traditional benchmarks like WebCompass-edit. However, there is a significant disparity (over 35 percentage points) between their performance on these benchmarks and on Co-Design Bench. This highlights the limitations of current benchmarks in measuring the ability of code agents to collaborate with others on existing codebases. If you are planning to deploy code agents for your organization, make sure to evaluate them on relevant benchmarks rather than relying on generalized ones.