News | Artificial Intelligence | February 01, 2024

A new paper in JAMA describes how, despite widespread enthusiasm about AI’s potential and the current use of AI-powered tools, no federal regulations require that AI-powered tools be evaluated for potential harm or benefit to patients 

A new paper in JAMA describes how, despite widespread enthusiasm about AI’s potential and the current use of AI-powered tools, no federal regulations require that AI-powered tools be evaluated for potential harm or benefit to patients

John W. Ayers, Ph.D., is deputy director of informatics in Altman Clinical and Translational Research Institute and a Qualcomm Institute affiliate at University of California, San Diego. Image courtesy of Qualcomm Institute at UC San Diego 


February 1, 2024 — Ever wonder if the latest and greatest artificial intelligence (AI) tool you read about in the morning paper is going to save your life? A new study published in JAMA led by John W. Ayers, Ph.D., of the Qualcomm Institute within the University of California San Diego, finds that question can be difficult to answer since AI products in healthcare do not universally undergo any externally evaluated approval process assessing how it might benefit patient outcomes before coming to market. 

The research team evaluated the recent White House Executive Order that instructed the Department of Health and Human Services to develop new AI-specific regulatory strategies addressing equity, safety, privacy, and quality for AI in healthcare before April 27, 2024. However, team members were surprised to find the order did not once mention patient outcomes, the standard metric by which healthcare products are judged before being allowed to access the healthcare marketplace.  

“The goal of medicine is to save lives,” said Davey Smith, M.D., head of the Division of Infectious Disease and Global Public Health at UC San Diego School of Medicine, co-director of the university’s Altman Clinical and Translational Research Institute, and study senior author. “AI tools should prove clinically significant improvements in patient outcomes before they are widely adopted.” 

According to the team, AI-powered early warning systems for sepsis, a fatal acute illness among hospitalized patients that affects 1.7 million Americans each year, demonstrates the consequences of inadequate prioritization of patient outcomes in regulations. A third-party evaluation of the most widely adopted AI sepsis prediction model revealed 67% of patients who developed sepsis were not identified by the system. Would hospital administrators have chosen this sepsis prediction system if trials assessing patient outcomes data were mandated, the team wondered, considering the array of available early warning systems for sepsis? 

“We are calling for a revision to the White House Executive Order that prioritizes patient outcomes when regulating AI products,” added John W. Ayers, Ph.D., who is deputy director of informatics in Altman Clinical and Translational Research Institute in addition to his Qualcomm Institute affiliation. “Similar to pharmaceutical products, AI tools that impact patient care should be evaluated by federal agencies for how they improve patients' feeling, function, and survival.” 

The team points to its 2023 study in JAMA Internal Medicine on using AI-powered chatbots to respond to patient messages as an example of what patient outcome-centric regulations can achieve. “Theoretically, a study comparing standard care versus standard care enhanced by AI conversational agents might find differences in downstream care utilization in some patient populations, such as heart failure patients,” said Nimit Desai, B.S., who is a research affiliate at the Qualcomm Institute, UC San Diego School of Medicine student, and study coauthor. “But studies like this don’t just happen unless regulators appropriately incentivize them. With a patient outcomes-centric approach, AI for patient messaging and all other clinical applications can truly enhance people’s lives.” 

The team recognizes that its proposed regulatory strategy can be a significant lift for AI and healthcare industry partners and may not be necessary for every flavor of AI use case in healthcare. However, the researchers say, excluding patient outcomes-centric rules in the White House Executive Order is a serious omission.  

The piece, “Regulate Artificial Intelligence in Health Care by Prioritizing Patient Outcomes,” (doi:10.1001/jama.2024.0549) was published by JAMA on January 29, 2024. 

For more information: https://ucsd.edu/ 


Related Content

News | Radiology Imaging

July 25, 2025 — Data in recent staffing surveys from the American Society of Radiologic Technologists show that vacancy ...

Time July 25, 2025
arrow
News | Prostate Cancer

July 9, 2025 — Artera, the developer of multimodal artificial intelligence (MMAI)-based prognostic and predictive cancer ...

Time July 09, 2025
arrow
News | Magnetic Resonance Imaging (MRI)

July 2, 2025 — Philips has received FDA 510(k) clearance for SmartSpeed Precise[1] MR’s latest deep learning ...

Time July 03, 2025
arrow
News | Ultrasound Imaging

July 1, 2025 — UPDATE: The final paper is now available at: JMIR AI - ChatGPT-4–Driven Liver Ultrasound Radiomics ...

Time July 01, 2025
arrow
News | Magnetic Resonance Imaging (MRI)

June 26, 2025 — Siemens Healthineers has received Food and Drug Administration clearance for the Magnetom Flow.Ace, its ...

Time June 26, 2025
arrow
News | Prostate Cancer

June 26, 2025 – Quibim, a global provider of quantitative medical imaging solutions, has launched AI-QUAL, a new feature ...

Time June 26, 2025
arrow
News | Bone Densitometry Systems

June 19, 2025 — Naitive Technologies has published results demonstrating the diagnostic performance of its AI-powered ...

Time June 18, 2025
arrow
News | Lung Imaging

June 18, 2025 — Exo recently announced that now included on its Exo Iris is the first ever FDA 510(k) cleared AI for ...

Time June 18, 2025
arrow
News | Digital Pathology

June 11, 2025 — Diagnostic laboratory leaders view digital pathology and artificial intelligence (AI) as pivotal to ...

Time June 12, 2025
arrow
News | Lung Imaging

June 11, 2025 — To prepare healthcare workforces and providers for an AI-driven future, Qure.ai has expanded its Global ...

Time June 11, 2025
arrow
Subscribe Now