Microsoft AI Panel Outperforms Generalist Doctors in Complex Diagnostics

The landscape of artificial intelligence in healthcare has shifted from simple administrative tools to active diagnostic participants, with major tech firms now asserting that their systems can rival human expertise in clinical reasoning. This week, Microsoft provided concrete evidence of that claim, releasing data that positions its new diagnostic architecture as a significant step toward what its leadership is calling medical superintelligence.

The findings, published in the journal Science on Thursday, detail the performance of MAI-DxO, a system designed to mimic a multidisciplinary medical team. In a rigorous simulation using complex cases from the New England Journal of Medicine, the AI system correctly identified the diagnosis 83% of the time. By comparison, primary care physicians and general internists, working individually with access to standard reference materials, reached the correct conclusion only 36% of the time. While specialists or physician teams typically handle such difficult cases in real-world settings, the gap between the automated system and individual generalists highlights a substantial leap in diagnostic capability.

The system operates by coordinating multiple AI agents that function as a virtual panel of doctors. These agents engage in a structured process of questioning the patient, ordering necessary tests, and debating potential diagnoses before reaching a final conclusion. This collaborative approach is central to Microsoft’s strategy, which aims to move beyond single-model responses to a more robust, multi-agent orchestration that can handle the nuance of complex medical histories.

Dominic King, vice president of Health at Microsoft AI, described the results as a significant development in addressing a critical industry challenge. He noted that the primary issue in current healthcare delivery is the widening gap between the growing demand for medical expertise and the limited supply of skilled practitioners. King stated, “There is something very substantial and exciting here,” emphasizing that the technology targets the precise area where human resources are most strained.

Microsoft’s entry into this specific domain places it in direct competition with Google and OpenAI, which have both recently claimed that their AI systems can match or exceed human physician performance. However, the study’s data suggests that the bulk of the performance improvement stems from the underlying power of the large language models used, with Microsoft’s orchestration layer providing additional, though variable, benefits. The company tested the system using models from OpenAI, Google, Anthropic, Meta, and DeepSeek, finding the highest accuracy scores when utilizing OpenAI’s GPT-5 and GPT-5.5 models.

Cost efficiency was also a key metric in the evaluation. When using GPT-5 with a basic prompt, the system achieved 78.9% accuracy at an estimated cost of $8,400 per case. The MAI-DxO orchestration reduced this cost by approximately $3,000 while improving accuracy by only four points. Results varied with the newer GPT-5.5 model; on complex cases, the system boosted accuracy from 72% to nearly 87% at a slightly lower cost, but showed little difference in performance or cost for emergency department cases. The data indicates that the orchestration layer is most beneficial when working with less capable models, such as Gemini 2.5 Flash, which saw its accuracy rise from 46.4% to nearly 69% with the assistance of the system.

Mustafa Suleyman, CEO of Microsoft AI, framed these results as part of a broader infrastructure build-out aimed at achieving real breakthroughs in healthcare. He revealed that Microsoft is collaborating with the Mayo Clinic to train a dedicated foundation model for healthcare, with a release planned for next year. Suleyman expressed confidence that this specialized model will allow the company to make significant strides in the field, predicting a coding-style breakthrough for medical records by 2027 that could provide clinicians with highly accurate predictions of patient outcomes.

To validate these capabilities in real-world settings, Microsoft has established a partnership with Beth Israel Deaconess Medical Center in Boston. The upcoming evaluation will assess the clinical reasoning of MAI-DxO across various hospital environments, including emergency departments and urgent care settings. In this study, patients will receive standard human care while consenting participants will also be evaluated by the AI system, allowing researchers to compare the two approaches directly. King described this as potentially the largest study of its kind, aiming to provide compelling evidence for the large-scale deployment of such systems in the future.

The initiative aligns with a broader manifesto released by Microsoft defining medical superintelligence as the capacity to surpass the strongest human-led health systems in reasoning and coordination. By integrating with clinicians and healthcare systems rather than just benchmarking individual tasks, Microsoft aims to differentiate itself from other frontier labs. Suleyman, a veteran of AI research who previously co-founded DeepMind, remains optimistic that healthcare models will improve by an order of magnitude within the next 24 to 36 months, driven by the same rapid iteration cycles that have transformed coding and chat interfaces.