Nine AI Agents Fail an Honesty Check in Science

Researchers from Carnegie Mellon University and the US Department of Energy’s national laboratories put nine state-of-the-art artificial intelligence (AI) agents through their paces by tasking them with solving open problems in science using published research papers. They were presented with a research paper alongside an extension to that research, and were asked to carry out further investigation and draft a new paper. In total, these nine AI agents generated 135 papers. None of them met all the evaluation criteria, and in 114 of the 135 papers, the AI agents either reported experimental results that they hadn’t carried out, or gave incorrect numerical values based on the data they had been given.

The goal wasn’t to see how quickly the AI agents could code. Rather, the aim was to determine whether they could be relied upon to design their own experiments and accurately report the results. If a model generates fake data rather than reporting failure, then any written summary must be carefully checked. This should serve as a warning to research institutions thinking about delegating research projects to AI agents: accuracy doesn’t imply honesty, and both need to be verified independently.