Probably Raises $9M to build a non-hallucinating AI

Probably has just raised $9 million in seed capital from Andreessen Horowitz (a16z) to develop AI that will be 99.99% accurate — i.e., as reliable as traditional software — instead of the hallucinatory chatbots we are used to. The company, founded by CEO Peter Elias, wants to prevent hallucinations and mistakes in the first place, rather than apologizing for them later.

How? By surrounding their language model with what Elias calls a “data science mech suit.” In this case, the model’s answers are validated against “deterministic validators,” and the model is trained on the harness to give answers that please those validators. As Elias told TechCrunch, “The stronger your harness engineering is, the weaker the model can be.” And that makes sense, because the company’s current offering uses a model that is “four classes weaker” than current state-of-the-art models, and can therefore be run locally, rather than in the cloud, at much lower cost per token.

1What Probably actually ships first

Probably’s initial application is a data science tool that provides answers based on complex data, along with citations and a chain of custody that allows you to trace it back to the source. This is intentional: Probably is focusing on fields where giving the wrong answer with confidence has high costs, such as accounting or medicine. Elias also criticized other AI companies, saying, “I think it’s really interesting that the big AI labs have not even attempted to do this. They’re all just competing on having a better model, not about making sure the answer is right.”

2Why this matters for you

We’ve all seen AI give the wrong answer with confidence: providing bogus references, incorrect figures, or functions that don’t exist. So far, the remedy has been “just use a bigger model and hope,” which is time-consuming, expensive, and ineffective. Probably’s solution is to focus on the harness around the model. If we can get almost perfect results on a specific task with a smaller, cheaper, local model and effective validation, then we might have a better way to trust AI with our money and our lives. Based on my experience using these tools, I think they’re onto something: the most important advances come not from improving the model, but from adding a harness to validate its output before it gets to me. This is just a $9M seed investment, but it’s in the right direction.