GitHub radar
Auto-BenchMax: 2× LLM Score on Tool-Use Benchmarks
An open-source pipeline for synthesizing training data that doubles an LLM's score on tool-use and agentic benchmarks. One training iteration lifted Qwen3-Coder-30B on MCP-Atlas from 19.1 to 40.4 — more than 2× the baseline.
Auto-BenchMax is an open-source pipeline by community contributor 'sunny-glow' for synthesizing training data that doubles LLM performance on tool-use and agentic benchmarks. The repository — with documentation partly in Chinese — includes the complete pipeline, training scripts (Qwen3-Coder-30B via axolotl, DeepSpeed ZeRO-3), and the first-iteration synthetic dataset of 1,312 samples. On the MCP-Atlas public benchmark, one training iteration raised the score from 19.1 to 40.4. The pipeline uses two paradigms: Execute-then-Extract for rule-based benchmarks and Construct-then-Verify for LLM-judged ones. It can be driven by Claude Code with a single instruction via an included skill.
Why a vibe-coder should care
If you're fine-tuning a model for agentic or tool-use tasks, this repo gives a concrete, reproducible path to 2× benchmark performance — with the data, code, and environment all open-sourced for the community.
▌ More finds