GitHub radar
The AI Agent Eval Bible Is Now Open
443+ verified resources on AI agent evaluation — papers, tools, benchmarks — with deep reading notes and a runnable playbook. The gap-audited list that the field was missing.
This is a collection of over 443 carefully selected & maintained resources on evaluating AI agents (research papers, frameworks, interview clips w/ timestamps, and more). What makes this resource unique is that it’s based on a depth-4 citation analysis of 11,600 papers + adversarial gap audits instead of being a random assortment of links. This resource went viral when frontier labs started publishing big posts about evaluation and evaluating AI agents became the limiting factor in training RL agents.
Why a vibe-coder should care
This resource is perfect if you’re building something that uses AI agents or if you’re interested in understanding what makes certain tools “trustworthy” while others hallucinate all the time. In this resource, you’ll learn how to quantify the differences between these types of tools. There are even code snippets to show you how to implement these ideas in your own work!
▌ More finds