News | Artificial Intelligence | April 30, 2020

RSNA AI Challenge Breaks New Ground

A complex multicompartmental cerebral hemorrhage on a single axial CT image displayed using the annotation tool in a single portal window. Hemorrhage labels (left column) relevant to the image display on the bottom of the image once selected. ASNR = American Society of Neuroradiology RSNA = Radiological Society of North America.

A complex multicompartmental cerebral hemorrhage on a single axial CT image displayed using the annotation tool in a single portal window. Hemorrhage labels (left column) relevant to the image display on the bottom of the image once selected. ASNR = American Society of Neuroradiology RSNA = Radiological Society of North America. Image courtesy of RSNA

April 30, 2020 — An unprecedented collaboration among two medical societies and over 60 volunteer neuroradiologists has resulted in the generation of the largest public collection of expert-annotated brain hemorrhage computed tomography (CT) images, according to a report published in Radiology: Artificial Intelligence. Leaders of the project expect the dataset to help speed the development of machine learning (ML) algorithms to aid in the detection and characterization of this potentially life-threatening condition.

The creation of the dataset stems from the most recent edition of the Radiology Society of North America (RSNA) Artificial Intelligence (AI) Challenge. For the 2019 edition, participants were asked to create an ML algorithm that could assist in the detection and characterization of intracranial hemorrhage on brain CT. Accuracy in diagnosing the presence and type of intracranial hemorrhage is a vital part of effective treatment, as even a small hemorrhage can lead to death if it is in a critical location.

Rather than using an existing dataset, as had been done for the first two challenges, the competition's organizers set out to create one from scratch. They compiled the brain hemorrhage CT dataset from three institutions: Stanford University in Palo Alto, California, Universidade Federal de São Paulo in São Paulo, Brazil, and Thomas Jefferson University Hospital in Philadelphia, Pennsylvania.

"The value of this challenge is to create a dataset that might lead to a generalizable solution, and the best way to do that is to train a model from data originating from multiple institutions that use a variety of CT scanners from various manufacturers, scanning protocols and a heterogeneous patient population," said the paper's lead author, Adam E. Flanders, M.D., neuroradiologist and professor at Thomas Jefferson University Hospital. "In this case, we had data from three institutions and international participation. The dataset is unique, not only in terms of the volume of abnormal images but also the heterogeneity of where they all came from."

RSNA and the American Society of Neuroradiology (ASNR) collaborated to curate the dataset and organizers issued an open call for volunteers within the ASNR membership to annotate the images. A day-and-a-half later, they had 140 volunteers from which they selected 60 to annotate a vast trove of 874,035 brain hemorrhage CT images in 25,312 unique exams. The volunteers marked each image as normal or abnormal. For the abnormal images, they indicated the hemorrhage subtype.

"It was a nail-biter all the way along," Flanders said of the process. "We were building the airplane while it was in flight. When you consider the number of images that we had to de-identify locally, consume, curate, label, cross-check and then organize into just the right datasets to release to the contestants, there was a lot of work involved by the volunteer workforce, the RSNA Machine Learning Subcommittee, data scientists, contractors and RSNA staff."

The dataset's release attracted interest from far and wide. Organizers received more than 22,200 submissions from 1,787 individual competitors in 1,345 teams from 75 countries. Dr. Flanders was particularly struck by the international reach of the project and the level of enthusiasm even from people outside of the medical realm.

"The 10 top solutions came from all over the world," he said. "Some of the winners had absolutely no background in medical imaging."

The dataset was released under a non-commercial license, meaning it is freely available to the AI research community for non-commercial use and further enhancement.

Flanders said the objective of engaging with a subspecialty society to leverage their unique expertise in developing a high-quality dataset is an effective and useful pathway to follow for future collaborations. The model worked so well that organizers are using it again for this year's competition, a collaboration with the Society of Thoracic Radiology seeking improved detection and characterization of pulmonary embolism on chest CT.

"I was really impressed by the huge volunteer effort and the tremendous worldwide interest in this project," Flanders said. "The dataset we created for this challenge will endure as a valuable ML research resource for years to come."

For more information: www.rsna.org

Related Content

AIR Recon DL delivers shorter scans and better image quality (Photo: Business Wire)

AIR Recon DL delivers shorter scans and better image quality (Photo: Business Wire).

News | Artificial Intelligence | May 29, 2020
May 29, 2020 — GE Healthcare announced U.S.
Largest case series (n=30) to date yields high frequency (77%) of negative chest CT findings among pediatric patients (10 months-18 years) with COVID-19, while also suggesting common findings in subset of children with positive CT findings

A and B, Unenhanced chest CT scans show minimal GGOs (right lower and left upper lobes) (arrows) and no consolidation. Only two lobes were affected, and CT findings were assigned CT severity score of 2. Image courtesy of American Journal of Roentgenology (AJR)

News | Coronavirus (COVID-19) | May 29, 2020
May 29, 2020 — An investigation published open-access in the ...
AI has the potential to help radiologists improve the efficiency and effectiveness of breast cancer imaging

Getty Images

Feature | Breast Imaging | May 28, 2020 | By January Lopez, M.D.
Headlines around the world the past several months declared that...
United Imaging's uMR OMEGA is designed to provide greater access to magnetic resonance imaging (MRI) with the world’s first ultra-wide 75-cm bore 3T MRI.
News | Magnetic Resonance Imaging (MRI) | May 27, 2020
May 27, 2020 — United Imaging's...
There were several new developments in digital radiography (DR) technology at the 2019 Radiological Society of North America (RSNA) annual meeting. These trends included integration of artificial intelligence (AI) auto detection technologies, more durable glassless detector plates, and technologies to pull more diagnostic data out of X-ray imaging. Some vendors also have redesigned their DR systems to make them more user-friendly and ergonomic. 
Feature | Digital Radiography (DR) | May 26, 2020 | By Dave Fornell
There were several new developments in digital rad...
An example of DiA'a automated ejection fraction AI software on the GE vScan POCUS system at RSNA 2019.

An example of DiA'a automated ejection fraction AI software on the GE vScan POCUS system at RSNA 2019. Photo by Dave Fornell.

News | Ultrasound Imaging | May 26, 2020
May 12, 2020 — DiA Imaging Analysis, a provider of AI based ultrasound analysis solutions, said it received a governm
 Recently the versatility of mixed and augmented reality products has come to the forefront of the news, with an Imperial led project at the Imperial College Healthcare NHS Trust. Doctors have been wearing the Microsoft Hololens headsets whilst working on the front lines of the COVID pandemic, to aid them in their care for their patients. IDTechEx have previously researched this market area in its report “Augmented, Mixed and Virtual Reality 2020-2030: Forecasts, Markets and Technologies”, which predicts th

Doctors wearing the Hololens Device. Source: Imperial.ac.uk

News | Artificial Intelligence | May 22, 2020
May 22, 2020 — Recently the versatility of