News | Artificial Intelligence | September 02, 2021

Insufficient Evidence That AI Breast Cancer Screening is Accurate Enough to Replace Human Scrutiny

Better evidence needed before considering the future integration of AI into breast cancer screening programs, say experts

The researchers say there is currently a lack of good quality evidence to support a policy of replacing human radiologists with artificial intelligence (AI) technology when screening for breast cancer.

Getty Images

September 2, 2021 — Humans still seem to be better than technology when it comes to the accuracy of spotting possible cases of breast cancer during screening, suggests a review published online in The BMJ.

The researchers say there is currently a lack of good quality evidence to support a policy of replacing human radiologists with artificial intelligence (AI) technology when screening for breast cancer.

Breast cancer is a leading cause of death among women worldwide and many countries have introduced mammography screening programmes to detect and treat it early. But examining mammograms for early signs of cancer is high volume repetitive work for radiologists, and some cancers are missed.

Previous research has suggested that AI systems outperform humans and might soon be used instead of experienced radiologists. Yet a recent review of 23 studies highlighted evidence gaps and concerns about the methods used.

To address this uncertainty, the UK National Screening Committee commissioned a team of researchers from the University of Warwick to examine the accuracy of AI for the detection of breast cancer in mammography screening practice.

The researchers reviewed 12 studies carried out since 2010 involving data for 131,822 screened women in Sweden, the United States, Germany, the Netherlands and Spain.

Overall, the quality of the methods used in the 12 studies was poor and their applicability to European or UK breast cancer screening programmes was low.

Three large studies involving 79,910 women compared AI systems with the clinical decisions of the original radiologist. Of these, 1,878 had screen detected cancer or interval cancer (cancer diagnosed in-between routine screening appointments) within 12 months of screening.

The majority (34 out of 36 or 94%) of AI systems evaluated in these three studies were less accurate than a single radiologist, and all were less accurate than the consensus of two or more radiologists, which is the standard practice in Europe.

In contrast, five smaller studies involving 1,086 women reported that all of the AI systems evaluated were more accurate than a single radiologist. But the researchers note that these studies were at high risk of bias and their promising results are not replicated in larger studies.

In three studies, AI used as a pre-screen to triage which mammograms need to be examined by a radiologist and which do not screened out 53%, 45%, and 50% of women at low risk but also 10%, 4%, and 0% of cancers detected by radiologists.

The authors point to some study limitations such as excluding non-English studies that might have contained relevant evidence, and they acknowledge that AI algorithms are short lived and constantly improving, so reported assessments of AI systems might be out of date by the time of study publication.

Nevertheless, use of stringent study inclusion criteria together with rigorous and systematic evaluation of study quality suggests their conclusions are robust.

As such, they said: “Current evidence on the use of AI systems in breast cancer screening is a long way from having the quality and quantity required for its implementation into clinical practice.”

They add: “Well designed comparative test accuracy studies, randomised controlled trials, and cohort studies in large screening populations are needed which evaluate commercially available AI systems in combination with radiologists in clinical practice.”

For more information: www.bmj.com/content/374/bmj.n1872

Related content:

AI DBT Impact on Mammography Post-breast Therapy

Artificial Intelligence Aids in Discovery of New Prognostic Biomarkers for Breast Cancer

Technology Report: Artificial Intelligence in Radiology 2021

 


 

Related Content

This is an example of 3-D ultrasound imaging on a breast, designed to help increase efficiency and diagnostic accuracy in any practice. Image courtesy of Hologic.

This is an example of TriVu ultrasound imaging on a breast, designed to help increase efficiency and diagnostic accuracy in any practice. Image courtesy of Hologic.

Feature | Breast Imaging | September 15, 2021 | By Jennifer Meade
The...
While the Mammography Quality Standards Act (MQSA) and the introduction of EQUIP (Enhancing Quality Using the Inspection Program) have been successful in standardizing and enhancing mammographic imaging quality, inadequate breast positioning can dramatically impact the ability of radiologists and technicians to quickly and accurately detect breast cancer and potentially malignant lesions in their patients

Getty Images

Feature | Mammography | September 15, 2021 | By Christopher Austin, M.D. and Randy D. Hicks, M.D., MBA
Revenues for teleradiology reading service providers are forecast to follow a similar profile over this period.

Outlook for 2021 and Beyond. As displayed in the figure below, these six market drivers are projected to result in teleradiology reading service volumes increasing by 21% in 2021 and nearly doubling by 2025. Revenues for teleradiology reading service providers are forecast to follow a similar profile over this period.

Feature | Teleradiology | September 15, 2021 | By Arun Gill
The closely tied relationship between...
To get more flexibility and cost savings from storage, healthcare organizations are increasing their investments in the cloud
Feature | Information Technology | September 15, 2021 | By Kumar Goswami
Healthcare organizations today are storing petabytes of medical imaging data — lab slides,...
Cloud services have been utilized within healthcare organizations for more than a decade. Now with the growth of artificial intelligence (AI) it is very common to see organizations adopting cloud services.

Getty Images

Feature | Information Technology | September 14, 2021 | By Jef Williams
Figure 1: MWT Schematic of a typical setup for detecting malignant tissues/tumors.

Figure 1: MWT Schematic of a typical setup for detecting malignant tissues/tumors.

Feature | Radiology Imaging | September 14, 2021 | By Brendon McHugh
As with all imaging technologies, COVID-19 is expected to continue to negatively impact the market.

Courtesy of Grand View Research

Feature | Magnetic Resonance Imaging (MRI) | September 14, 2021 | By Melinda Taschetta-Millane
Us2.ai, a Singapore-based medtech firm backed by Sequoia India and EDBI, has received U.S. Food and Drug Administration (FDA) clearance for Us2.v1, a completely automated AI decision support tool for cardiac ultrasound.
News | Cardiovascular Ultrasound | September 14, 2021
September 14, 2021 – Us2.ai, a Singapore-based medtech firm backed by Sequoia India and EDBI, has received U.S.
Plan to attend RSNA21 at McCormick Place Chicago, Nov. 28 – Dec. 2, 2021

Getty Images

News | RSNA | September 13, 2021
September 13, 2021 — The Radiological Society of North America (RSNA) today announced highlights of the Technical Exh