Large language model GPT-4 matched the performance of radiologists in detecting errors in radiology reports, according to research published in Radiology

Getty Images


April 19, 2024 — Large language model GPT-4 matched the performance of radiologists in detecting errors in radiology reports, according to research published in Radiology, a journal of the Radiological Society of North America (RSNA).

Errors in radiology reports may occur due to resident-to-attending discrepancies, speech recognition inaccuracies and high workload. Large language models, such as GPT-4, have the potential to enhance the report generation process.

“Our research offers a novel examination of the potential of OpenAI’s GPT-4,” said study lead author Roman J. Gertz, M.D., resident in the Department of Radiology at University Hospital of Cologne, in Cologne, Germany. “Prior studies have demonstrated potential applications of GPT-4 across various stages of the patient journey in radiology: for instance, selecting the correct imaging exam and protocol based on a patient’s medical history, transforming free-text radiology reports into structured reports or automatically generating the impression section of a report.”

However, this is the first study to distinctively compare GPT-4 and human performance in error detection in radiology reports, assessing its capabilities against radiologists of varied experience levels in terms of accuracy, speed and cost-effectiveness, Dr. Gertz noted.

Dr. Gertz and colleagues set out to assess GPT-4’s effectiveness in identifying common errors in radiology reports, focusing on performance, time and cost-efficiency.

For the study, 200 radiology reports (X-rays and CT/MRI imaging) were gathered between June 2023 and December 2023 at a single institution. The researchers intentionally inserted 150 errors from five error categories (omission, insertion, spelling, side confusion and “other”) into 100 of the reports. Six radiologists (two senior radiologists, two attending physicians and two residents) and GPT-4 were tasked with detecting these errors.

Researchers found that GPT-4 had a detection rate of 82.7% (124 of 150). The error detection rates were 89.3% for senior radiologists (134 out of 150) and 80.0% for attending radiologists and radiology residents (120 out of 150), on average.

In the overall analysis, GPT-4 detected less errors compared with the best performing senior radiologist (82.7% vs 94.7%). However, there was no evidence of a difference in the percentage of average performance in error detection rate between GPT-4 and all the other radiologists.

GPT-4 required less processing time per radiology report than even the fastest human reader, and the use of GPT-4 resulted in lower mean correction cost per report than the most cost-efficient radiologist.

“This efficiency in detecting errors may hint at a future where AI can help optimize the workflow within radiology departments, ensuring that reports are both accurate and promptly available,” Dr. Gertz said, “thus enhancing the radiology department’s capacity to deliver timely and reliable diagnostics.”

Dr. Gertz notes that the study’s findings are significant for their potential to improve patient care by enhancing the accuracy of radiology reports through GPT-4 assisted proofreading. Demonstrating that GPT-4 can match the error detection performance of radiologists—while significantly reducing the time and cost associated with report correction—this research shows the potential benefits of integrating AI into radiology departments.

“The study addresses critical health care challenges such as the increasing demand for radiology services and the pressure to reduce operational costs,” he said. “Ultimately, our research provides a concrete example of how AI, specifically through applications like GPT-4, can revolutionize health care by boosting efficiency, minimizing errors and ensuring broader access to reliable, affordable diagnostic services—fundamental steps toward improving patient care outcomes.”

For more information: www.rsna.org


Related Content

News | ASTRO

May 17, 2024 — Registration opens today for the American Society for Radiation Oncology's (ASTRO) 66th Annual Meeting ...

Time May 17, 2024
arrow
News | Enterprise Imaging

May 16, 2024 — AGFA HealthCare announced that St. Vincent’s Private Hospital in Dublin, Ireland, has chosen to implement ...

Time May 16, 2024
arrow
News | Artificial Intelligence

May 16, 2024 — deepc, the globally recognized digital medicine pioneer and market leader behind the leading AI operating ...

Time May 16, 2024
arrow
Sponsored Content | Case Study | Enterprise Imaging

Having the most efficient clinical workflows with enhanced diagnostic capabilities is a major goal for clinicians and ...

Time May 16, 2024
arrow
News | Enterprise Imaging

May 15, 2024 — etherFAX announced the expansion of its partnership with Hyland, a leading global provider of intelligent ...

Time May 15, 2024
arrow
News | Prostate Cancer

May 13, 2024 — Avenda Health, an AI healthcare company creating the future of personalized prostate cancer care, unveils ...

Time May 13, 2024
arrow
News | Cybersecurity

May 13, 2024 — In the wake of the cybersecurity breach targeting the prominent healthcare system Ascension, a new study ...

Time May 13, 2024
arrow
News | Radiology Imaging

May 13, 2024 — National Basketball Association (NBA), the National Basketball Players Association (NBPA), and the Next ...

Time May 13, 2024
arrow
News | Radiation Oncology

May 10, 2024 — Mariana Oncology, a fully integrated biotechnology company pioneering a new era of radiopharmaceutical ...

Time May 10, 2024
arrow
News | ARRS

May 8, 2024 — Compared to males, women in radiology are at a consistently higher risk of not matching into diagnostic ...

Time May 08, 2024
arrow
Subscribe Now