Google’s AMIE Outperforms Human Doctors in Disease Management

AMIE started as a chatbot for diagnosing patients based on their symptoms. It could recommend possible causes of their illness after a quick chat. Now, it’s being used for managing chronic diseases, which involves long-term, iterative treatments such as prescribing medicines and monitoring the progress of the disease. Most of the medical practice revolves around these tasks.

The system has two components. The first one is an empathetic dialogue agent that interacts with the patient in real-time. The second component is a management-reasoning agent that performs heavy lifting by referring to hundreds of pages of drug compendiums and clinical guidelines. It can store all this information in its working memory because of Gemini’s long context. Basically, there’s a part that talks to you and another part that silently checks your medical records against available medical knowledge before recommending any course of action.

1How the study was set up

Google didn’t pull this out of thin air. They conducted a blind trial involving patient actors and evaluated the dialogues using physicians who didn’t know whether they were talking to humans or AI. They pitted AMIE against 21 primary care doctors and found that it performed equally well in terms of management reasoning. It even scored significantly better in precision of treatment plans and adherence to clinical guidelines. Beating human doctors in adherence to clinical guidelines is the holy grail because it means the AI is more than just a gimmick.

2Why this matters for you

It’s important to note that this isn’t a real doctor you can visit. The patients were actors reading from a script and the environment was specifically set up. There are no clinics using this technology right now. So don’t panic and cancel your appointment with your GP. Instead, focus on the future. AMIE excels in adherence to clinical guidelines because it can access all the relevant guidelines and remember them perfectly. It never gets tired, forgets anything, or skips steps. It’s not a genius, it’s just disciplined. Google plans to conduct a nationwide randomized trial to evaluate the system in real-world virtual care settings. We’ll have to wait and see if it works as well outside the lab.