AI tool reads doctor notes to predict patient outcomes with high accuracy

An AI tool developed by a team at NYU Grossman School of Medicine has shown the ability to read doctors’ notes and accurately predict patients’ risk of death, readmission to hospital, and other significant outcomes. The software, called NYUTron, is currently being used at affiliated hospitals throughout New York, with the aim of becoming a standard part of health care. The study on the tool’s predictive value was recently published in the journal Nature.

Eric Oermann, an NYU neurosurgeon and computer scientist, explained that while non-AI predictive models have been used in medicine for a long time, they are rarely implemented in practice due to the need for complex data reorganisation and formatting. However, doctors’ notes are a common source of data in medicine. The team’s primary insight was to use medical notes as the data source and build predictive models on top of them.

NYUTron is a large language model trained on millions of clinical notes from the health records of 387,000 people who received care within NYU Langone hospitals between January 2011 and May 2020. The notes included patient progress notes, radiology reports, and discharge instructions, resulting in a 4.1-billion-word corpus. One of the main challenges for the software was interpreting the natural language that physicians write in, which varies greatly among individuals, including the abbreviations they use.

By looking back at records of what happened, researchers were able to calculate how often the software’s predictions turned out to be accurate. They also tested the tool in live environments, training it on the records from, for example, a hospital in Manhattan then seeing how it fared in a Brooklyn hospital, with different patient demographics.

Overall, NYUTron identified an impressive 95% of people who died in hospital before they were discharged and 80% of patients who would be readmitted within 30 days. It outperformed most doctors on its predictions, as well as the non-AI computer models used today. However, the most senior physician achieved better results than the model, which surprised the team.

Oermann emphasised that AI will never be a substitute for the physician-patient relationship. Instead, it will help “provide more information for physicians seamlessly at the point-of-care so they can make more informed decisions.”