Vals Raises $40M for Neutral AI Benchmarks

Vals, a company that tests AI models for legal and financial firms, closed a $40M Series A round led by Andreessen Horowitz. Test questions are kept secret so that vendors cannot train their models on the correct answers. Revenue grew eightfold over the past year.

Public benchmarks are becoming harder to take seriously, because labs know exactly what they are being tested on. Vals is building a business modeled on College Board, offering a paid, independent evaluation that is difficult to cheat on. I personally suspect every time that the numbers on a model's marketing sheet are slightly inflated, and this kind of independent verification has been long overdue.