News | Artificial Intelligence | July 05, 2023

Researchers detail the first study to date finding that ChatGPT can support the clinical decision-making process, including when picking the correct radiological imaging tests for breast cancer screening or breast pain 

Researchers detail the first study to date finding that ChatGPT can support the clinical decision-making process, including when picking the correct radiological imaging tests for breast cancer screening or breast pain

Getty Images


July 5, 2023 — A new study by investigators from Mass General Brigham has found that artificial intelligence (AI) language models like ChatGPT can accurately identify appropriate imaging services for two important clinical presentations: breast cancer screening and breast pain. Their results suggest that large language models have the potential to assist decision-making for primary care doctors and referring providers in evaluating patients and ordering imaging tests for breast pain and breast cancer screenings. Their results are published in the Journal of the American College of Radiology.  

"In this scenario, ChatGPT's abilities were impressive," said corresponding author Marc D. Succi, MD, associate chair of Innovation and Commercialization at Mass General Brigham Radiology and executive director of the MESH Incubator. "I see it acting like a bridge between the referring healthcare professional and the expert radiologist — stepping in as a trained consultant to recommend the right imaging test at the point of care, without delay. This could reduce administrative time on both referring and consulting physicians in making these evidence-backed decisions, optimize workflow, reduce burnout, and reduce patient confusion and wait times." 

ChatGPT is a large language model (LLM) built on data from the internet to answer questions in a human-like way. Since ChatGPT was introduced in November 2022, researchers worldwide are diving into learning how these AI tools can be used in medical scenarios. Published as a preprint on February 7, 2023, this study is the first of its kind to test ChatGPT's clinical decision-making abilities, and the first to test GPT 4 as opposed to older iterations.  

When a primary care doctor orders specialized testing, say for a patient who complains of breast pain, they may not know the best imaging test to choose. It might be an MRI, an ultrasound, a mammogram, or another imaging test. Radiologists generally follow the American College of Radiology's Appropriateness Criteria to make these decisions. These evidence-backed guidelines are well-known to specialists, but less known for non-specialists who many need to pick the best imaging test during a patient’s visit. This can cause confusion on the patient's side and can lead to patients getting tests they don't need or getting the wrong tests.  

The researchers asked OpenAI's ChatGPT 3.5 and 4 to help them decide what imaging tests to use for 21 made-up patient scenarios involving the need for breast cancer screening or the reporting of breast pain using the appropriateness criteria.  

They asked the AI in an open-ended way and by giving ChatGPT a list of options. They tested ChatGPT 3.5 as well as ChatGPT 4, a newer, more advanced version. ChatGPT 4 outperformed 3.5, especially when given the available imaging options. For example, when asked about breast cancer screenings, and given multiple choice imaging options, ChatGPT 3.5 answered an average of 88.9% of prompts correctly, and ChatGPT 4 got about 98.4% right. 

"This study doesn't compare ChatGPT to existing radiologists because the existing gold standard is actually a set of guidelines from the American College of Radiology, which is the comparison we performed,” Succi said. “This is purely an additive study, so we are not arguing that the AI is better than your doctor at choosing an imaging test but can be an excellent adjunct to optimize a doctor’s time on non-interpretive tasks." 

Integrating AI into medical decision making could happen at the point of care. When a primary care doctor enters data into an electronic health record, the program could alert them to the best imaging options — providing an answer to the patient for what to expect when they go for the test and suggesting to the doctor the right test to order.  

Researchers added that a more advanced medical AI could be created using datasets from hospitals and other research institutions to make it more specific to health-focused applications.  

"We may be able to finetune ChatGPT with different patient and therapeutic data and knowledge sets to tailor it to specific patient populations," Succi said. "At Mass General Brigham, we have specialized centers of excellence where we care for patients with some of the most complex and rare diseases. We can leverage our experience and lessons learned from caring for these patient cases to train a model to provide support for rare and complex diagnoses and then make that model available to centers around the world, especially centers that may treat these conditions less frequently." 

But before any AI would be involved in medical decision-making, it would need to be extensively tested for bias, privacy concerns, and approved for use in medical setting. New regulations around medical AI could also play a big role in what makes it into patient care interactions.  

For more information: massgeneralbrigham.org


Related Content

News | Breast Imaging

June 20, 2024 — The technologically advanced VELA Mammography Chair, offered by Enable Me, a VELA Medical company ...

Time June 20, 2024
arrow
News | Enterprise Imaging

June 20, 2024 — Agfa announces its participation in the upcoming Society for Imaging Informatics in Medicine (SIIM) 2024 ...

Time June 20, 2024
arrow
News | SIIM

June 20, 2024 — The Society for Imaging Informatics in Medicine (SIIM) is gearing up for its SIIM24 Annual Meeting and ...

Time June 20, 2024
arrow
News | Cybersecurity

June 19, 2024 —Karamba Security, a world leader in end-to-end product security, today announced the signing of a ...

Time June 19, 2024
arrow
News | Mobile C-Arms

June 19, 2024 —Royal Philips, a global leader in health technology, today announced the launch of Philips Image Guided ...

Time June 19, 2024
arrow
News | Breast Imaging

June 18, 2024 — Delphinus Medical Technologies, a pioneering medical imaging company that developed the SoftVue Breast ...

Time June 18, 2024
arrow
News | Artificial Intelligence

June 18, 2024 — The advancement of Artificial Intelligence (AI) in healthcare to support diagnostic decision making ...

Time June 18, 2024
arrow
News | PET Imaging

June 18, 2024 — Positron Corporation, a leading molecular imaging medical device company offering PET and PET-CT ...

Time June 18, 2024
arrow
News | Lung Imaging

June 18, 2024 — A new study led by American Cancer Society (ACS) researchers shows less than one-in-five eligible ...

Time June 18, 2024
arrow
News | Magnetic Resonance Imaging (MRI)

June 17, 2024 — MRI has transformed neuroscience research over the past 50 years, but research participants have had to ...

Time June 17, 2024
arrow
Subscribe Now