Microsoft AI diagnosis tool MAI-DxO beats doctors on hard cases in Science study

Microsoft says its newest AI diagnosis system got the right answer on hard medical cases far more often than the doctors it was tested against. The Microsoft AI diagnosis results, published in the journal Science this week, show a tool called MAI-DxO solving 83% of complex cases from the New England Journal of Medicine, while generalist physicians solved between 22% and 36%. It is a striking number. It also comes with real caveats, and those matter just as much for anyone trying to understand what this means for their next trip to the doctor.

What Microsoft announced

The system is called the Microsoft AI Diagnostic Orchestrator, or MAI-DxO. According to a post from Microsoft's MAI Health team dated October 22, it works less like a single chatbot and more like a small panel of AI agents. Each agent has a role. Some propose possible diagnoses, some decide which question to ask or which test to order next, and others push back on the group's reasoning. They go back and forth over several rounds, and the system weighs how useful each test is likely to be against what it costs.

That design mirrors how a real diagnosis happens. A doctor rarely gets every lab result up front. They start with a few symptoms, ask questions, order a test, and narrow things down. Microsoft built a benchmark around that idea, called SDBench (the Sequential Diagnosis Benchmark). A case begins with limited information, and the AI or the physician has to ask for what they want to see next. The benchmark scores three things: whether the final diagnosis is right, how much evidence was requested, and the estimated cost of the tests ordered.

Microsoft has been pushing deeper into AI lately, from its Decision-1 model to CEO Satya Nadella's call for an AI emergency brake. This study is its most direct claim yet that AI can outperform clinicians at core medical reasoning.

The Microsoft AI diagnosis numbers

The study used two sets of cases. The first was 304 complex cases drawn from the NEJM's published case records, the kind of puzzling cases doctors read to sharpen their skills. The second came from MIMIC-IV-ED, a dataset of more typical emergency department visits.

Here is how Microsoft reports the results:

• Complex NEJM cases: MAI-DxO reached the correct diagnosis 83% of the time. Physicians working without outside help got 22%. Physicians allowed to use textbooks and the internet got 36%.
• Emergency department cases: MAI-DxO scored 80%, compared with 47% for physicians.

Microsoft describes that as nearly four times as accurate as doctors working alone on the hard cases, and more than twice as accurate as doctors who could look things up.

It also ordered fewer tests

Accuracy was only half the pitch. Microsoft says MAI-DxO got to the right answer at half or even one third of the cost of comparison approaches, largely because it ordered fewer tests. One cost focused setup reached 81% accuracy on the NEJM cases at less than half the cost of other systems, the company says. Microsoft also claims no other human or AI approach in the study was both more accurate and cheaper across both datasets.

TechCrunch reported per case figures from the study. A plain GPT-5 model with a basic prompt scored 78.9% on the complex cases, at an estimated cost of about $8,400 in tests per case. Running GPT-5 inside the orchestrator cut that cost by roughly $3,000 while adding only about four points of accuracy.

Why the underlying model matters most

That last detail points to the most interesting finding in the study, and it is one Microsoft's own headline numbers do not make obvious. According to TechCrunch's reading of the data, most of the improvement seems to come from the AI models underneath the orchestrator, not from the orchestrator itself.

Microsoft tested MAI-DxO on top of models from OpenAI, Google, Anthropic, Meta and DeepSeek. The best results came from OpenAI's GPT-5 and GPT-5.5. A few examples TechCrunch highlighted:

• GPT-5.5 went from 72% on its own to nearly 87% on complex cases when run through the orchestrator.
• On emergency cases, GPT-5.5 scored about 85% with or without the orchestrator.
• Google's Gemini 2.5 Flash jumped from 46.4% alone to nearly 69% with it.

So the orchestrator helps a weaker model a lot and a strong model somewhat. TechCrunch notes the gains shrink as models get closer to the ceiling, and that the boost from the orchestrator is smaller than the gap between weak and strong models. On emergency cases with GPT-5 or GPT-5.5, it added little or nothing. As frontier models like GPT-6 keep improving, the value of the extra layer may keep narrowing.

The caveats you should know

None of this happened with real patients. Both the NEJM and emergency results come from simulated cases run through the benchmark.

The human comparison also deserves a closer look. The doctors in the study were generalists, primary care physicians and general internists working alone. TechCrunch points out that cases this complex are usually handled by specialists or teams of physicians, who were not part of the comparison. Beating a solo generalist on a rare, puzzling case is impressive, but it is not the same as beating the care a patient would actually get at a major hospital.

There is also a broader question about how people use these tools. A Berkeley study found that short bursts of AI help can make people quit hard tasks sooner, which is worth keeping in mind as diagnostic AI moves closer to clinics.

What Microsoft leaders said

Mustafa Suleyman, the CEO of Microsoft AI, called the result a step toward what the company calls "medical superintelligence," according to TechCrunch. Microsoft defines that as "the capacity to surpass the strongest contemporary human-led health systems in medical reasoning and coordination across the full arc of care." Suleyman also told TechCrunch he believes that in 2027 "we will be on the verge of a coding style breakthrough for medical records."

Dominic King, vice president of health at Microsoft AI, told TechCrunch, "There is something very substantial and exciting here," and said the technology "is ready for real world evaluations."

Who is affected and what happens next

For now, nobody can use MAI-DxO as a patient. Microsoft has not announced a product. Here is what it has laid out instead:

• A real hospital trial. Microsoft plans a prospective study with Beth Israel Deaconess Medical Center in Boston. Patients will get standard care, and those who consent will also interact with the AI in parallel. Its recommendations will be compared against doctors' independent assessments. No start date was given in the coverage we reviewed.
• An open benchmark. Microsoft is releasing SDBench for a selection of NEJM cases, including the code that plays the gatekeeper, judges answers and estimates costs, so other researchers can test their own systems.
• A Mayo Clinic model. TechCrunch reports Microsoft is training a healthcare foundation model with Mayo Clinic, with a release planned for next year.
• A companion paper. A second paper, "The Building Blocks of Medical Superintelligence," appeared in BMJ Digital Health and AI.

Microsoft already sells clinical tools such as Dragon Copilot for dictation and PowerScribe for radiology, and its post links this research to those products. That suggests a path where diagnostic reasoning shows up inside software doctors already use, rather than as a standalone app.

Microsoft is not alone here. TechCrunch notes that Google, with its AMIE research published in The Lancet, and OpenAI have made similar claims about AI diagnosis. The real test for all of them is the same: how these systems perform with real patients, real messy records and real specialists in the room. Follow more of these stories on our AI news page.

Frequently asked questions

What is MAI-DxO?

It is Microsoft's AI Diagnostic Orchestrator, a system where several AI agents propose diagnoses, choose tests and challenge each other before settling on an answer. It runs on top of existing large language models.

Did Microsoft's AI really beat doctors?

On simulated cases, yes. It scored 83% on complex NEJM cases versus 22% to 36% for generalist physicians. The doctors were not specialists, and no real patients were involved.

Can I use Microsoft's diagnosis AI today?

No. Microsoft has not released it as a product. A clinical study with Beth Israel Deaconess Medical Center is planned first.

Why does the underlying AI model matter so much?

The study data shows most of the accuracy comes from the base model. The orchestrator gives weaker models a big lift but adds less to top models like GPT-5.5.