MAI-DxO Review: Microsoft's AI Diagnostic Orchestrator

Introduction

Microsoft AI has published its diagnostic research in the journal Science. The paper introduces two things: SDBench, the Sequential Diagnosis Benchmark, and MAI-DxO, the Microsoft AI Diagnostic Orchestrator. A companion vision paper, "The Building Blocks of Medical Superintelligence," appears in BMJ Digital Health and AI. The work builds on an earlier Microsoft AI post, "The Path to Medical Superintelligence," so MAI-DxO itself is not a new system name; Science is where the work now appears in peer-reviewed form.

Microsoft defines medical superintelligence as "the capacity to surpass the strongest contemporary human-led health systems in medical reasoning and coordination across the full arc of care." Two caveats belong at the top. The results come from benchmark cases and not from live clinical deployment, and Microsoft's own post says the true test is performance in real clinical environments. And TechCrunch's reading of the study data suggests that much of the improvement comes from stronger underlying models. Both points are covered below.

Feature Overview

SDBench: an interactive diagnostic test. SDBench uses 304 complex case proceedings from the New England Journal of Medicine (NEJM), plus a set of typical emergency-department presentations from MIMIC-IV-ED. It is interactive: the model or physician starts with limited information and chooses what to ask or which test to order next. SDBench measures diagnostic accuracy, the evidence requested and the estimated cost of the tests. Microsoft is open-sourcing SDBench for select NEJM cases, including the benchmark, the Gatekeeper, the Judge and the cost-estimation code, at microsoft.github.io/sdbench.

MAI-DxO: a panel of virtual physicians. MAI-DxO works like a panel of virtual physicians. Each one proposes diagnoses, selects questions and tests, and challenges the emerging view. Over several rounds of debate the panel weighs each test's value against its cost. The design treats diagnosis as a sequence of decisions, not a single answer.

Reported results. Microsoft reports the following:

Microsoft puts the NEJM gap at about 3.7 times and 2.3 times the physicians' accuracy, and the emergency gap at more than 1.7 times. A cost-optimised configuration reached 81% on NEJM cases at less than half the cost of comparative systems. Microsoft adds that "across both datasets, no other human or AI approach was both more accurate and less costly than MAI-DxO," and that the gains persisted when the evaluation was repeated with newer models, including GPT-5.5, without changing the orchestration method.

Model-agnostic testing. According to TechCrunch, Microsoft tested the orchestrator with models from OpenAI, Google, Anthropic, Meta and DeepSeek, with the best results from OpenAI's GPT-5 and GPT-5.5.

Reading the Numbers Carefully

Two different comparisons appear in the coverage, and they answer different questions. Microsoft compares MAI-DxO against physicians: 80% versus 47% on emergency cases. TechCrunch, working from the study's data, compares a base model alone against the same model with the orchestrator. The two are compatible. A system can beat individual physicians by a wide margin even if a strong base model would have scored nearly as well without the orchestration layer.

TechCrunch's figures: GPT-5 with a short, basic prompt scored 78.9% on complex cases at about $8,400 per case. Adding DxO cut cost by about $3,000 and raised accuracy by only about four points. On emergency cases there was little improvement in accuracy or cost between GPT-5 alone and with DxO. For GPT-5.5, DxO raised complex-case accuracy from 72% to nearly 87% at slightly lower cost, while emergency cases came out at about 85% both with and without DxO at similar cost. For Gemini 2.5 Flash, accuracy rose from 46.4% to nearly 69%, but TechCrunch notes the gap between weaker and stronger models was larger than DxO's boost. We have not reconciled these figures against the paper's tables, so treat them as TechCrunch's reading.

The physician group also deserves attention. TechCrunch reports that the comparison group was primary care doctors and general internists working individually, and notes that cases this complex are usually handled by specialists or physician teams, who were not part of the comparison. Microsoft's post does not give the number of physicians or the exact protocol.

Usability Analysis

MAI-DxO is a research system, not something a patient or clinician can use today. The material we reviewed describes research results and a planned prospective study, not a released clinical product. The most immediate usable piece is SDBench: because it is interactive and tracks cost as well as accuracy, other groups can test their own systems on the same format using the released code for select NEJM cases.

For health systems, the useful lesson is the framing. Microsoft evaluates diagnosis as an ordering problem, asking which question or test is worth its price at each step, which is closer to real workflow than a single multiple-choice answer. Dominic King, Microsoft AI's VP of Health, told TechCrunch: "The biggest challenge in healthcare delivery at the moment is the growing mismatch between demand and supply of medical expertise."

Pros and Cons

The strengths are a peer-reviewed venue, an interactive benchmark that prices tests, released benchmark code and results that Microsoft says held across newer models. The limits are that the evidence is benchmark-based, the physician baseline excludes specialists and teams, and, in TechCrunch's reading, a large share of the gain tracks the strength of the underlying model, with little added accuracy on emergency cases for the stronger models.

Outlook

The next test is real patients. Microsoft announced a partnership with Beth Israel Deaconess Medical Center in Boston for a prospective study in which patients interact with the system directly, with performance measured against physicians' independent assessments. Microsoft also collaborates with Mayo Clinic on a frontier healthcare model. Mustafa Suleyman, Microsoft AI's CEO, told TechCrunch that the Mayo model is planned for release next year, and he expects healthcare models to get "an order of magnitude better in capability within the next 24 to 36 months." That is a prediction, not a result. Microsoft is also consolidating health teams, including Microsoft Research scientists and the Dragon Copilot and PowerScribe teams, into Microsoft AI.

The field is moving in parallel: TechCrunch notes that Google published a real-world urgent-care study of its AMIE system with Beth Israel in The Lancet this month.

Conclusion

The Science paper gives MAI-DxO and SDBench a firmer evidentiary footing than a company blog post, and the open benchmark is a real contribution. The headline gap over physicians is large but comes from an individual-physician baseline on benchmark cases, and, in TechCrunch's reading of the data, the orchestration layer's own contribution varies widely by model and case type, from about 15 points on complex cases for GPT-5.5 to little or none on emergency cases. It is most relevant to medical AI researchers and health-system technology leaders, who should watch the Beth Israel study. We rate it 4 out of 5 as research, with no claim about clinical readiness.

Editor's Verdict

MAI-DxO Review: Microsoft's AI Diagnostic Orchestrator earns a solid recommendation within the research space.

The strongest case for paying attention: a peer-reviewed publication in Science, with a companion vision paper in BMJ Digital Health and AI, gives the work an outside review route. That puts the claims in front of outside reviewers rather than resting on a company blog post. Reinforcing that, a cost-optimised configuration reached 81% on NEJM cases at less than half the cost of comparative systems, which makes cost a first-class metric — practical value rather than just headline appeal. The broader signal worth registering is straightforward: a panel-of-virtual-physicians design turns diagnosis into a sequential, cost-aware process in which agents debate which question or test is worth its price. On the other side of the ledger, one constraint is real rather than a marketing footnote: the results come from benchmark cases and not from live clinical deployment, and Microsoft itself says the true test is performance in real clinical environments. It is the main reason not to read these numbers as clinical readiness. Layered on top of that, the physician baseline consisted of primary care doctors and general internists working individually, not specialists or physician teams — which limits how far the headline comparison with physicians can be generalised.

For medical AI researchers, the paper is worth reading closely and SDBench is worth reusing as a benchmark. Health-system technology leaders should treat it as a signal, not a tool. The evidence that matters for clinical use will come from the Beth Israel Deaconess prospective study.