News | Artificial Intelligence | August 27, 2018

RSNA Announces Pneumonia Detection Machine Learning Challenge

Teams will develop algorithms to identify and localize pneumonia in chest X-rays

RSNA Announces Pneumonia Detection Machine Learning Challenge

August 27, 2018 — The Radiological Society of North America (RSNA) has launched its second annual machine learning challenge. The RSNA Pneumonia Detection challenge invites teams to develop algorithms to identify and localize pneumonia in chest X-rays.

“The expectation that artificial intelligence [AI] will soon provide valuable tools for radiology continues to grow,” said Luciano Prevedello, M.D., MPH, chief of the Division of Medical Imaging Informatics at The Ohio State University and chair of the Machine Learning Steering Subcommittee of the RSNA Radiology Informatics Committee (RIC). “By organizing machine learning data challenges, RSNA is playing an important role in fostering and demonstrating these capabilities.”

A machine learning data challenge is a competition among AI researchers to create applications that perform a defined task according to specified performance measures. The Pneumonia Detection challenge will be based on a publicly available dataset published by the National Institutes of Health (NIH) that has been carefully annotated by multiple expert reviewers. The RSNA Machine Learning Steering Subcommittee collaborated with volunteer specialists from the Society of Thoracic Radiology to annotate the dataset, identifying abnormal areas in the lung images and assessing the probability of pneumonia.

“A successful Machine Learning challenge needs to begin with a data set accurate and large enough to provide ‘ground truth,’” said Safwan Halabi, M.D., medical director of radiology informatics at Stanford University and chair of the Machine Learning Data Standards Subcommittee of the RIC. “Developers build their applications by training them on the dataset and challenge organizers use a segment of the dataset to measure their performance.”

This year’s challenge will be conducted using the competition platform provided by Kaggle Inc., a subsidiary of Alphabet Inc., which is also the parent company of Google. The Kaggle platform will provide a home page for the challenge, controlled access to the challenge datasets, a discussion forum for participants and the repository where they submit their results. Kaggle has recognized the RSNA Pneumonia Detection Challenge as a public good and will provide $30,000 in prize money for the winning entries.

The challenge will have two phases: training and evaluation. During the training phase, which starts Aug. 27 and runs until Oct. 17, participants will use the training portion of the dataset to develop algorithms that duplicate the annotations provided by the radiologist observers. In the evaluation phase, which runs Oct. 18-24, participants will run their algorithms on the testing portion of the dataset, from which the annotations are withheld. Oct. 24 is the deadline for submission of test results, which will then be compared to the “ground-truth” values supplied by the annotations of the expert observers to determine the winners. The results will be announced in early November, and the top submissions will be recognized Nov. 26 in a session at the Machine Learning Showcase during the RSNA 2018 annual meeting, Nov. 25-30 in Chicago.

Last year’s challenge had more than 200 participants, many of whom formed teams comprising both data scientists and radiologists. “Radiologists have a critical role to play in the challenge, even if they don’t have extensive experience with machine learning,” Prevedello said. “They can provide insight, based on their clinical experience, to refine or supplement the annotations on the training data set and help developers identify systematic errors in the results of their algorithms.”

Radiologists interested in participating can set up a free account on the Kaggle site and express their interests and qualifications in the forum for the RSNA Pneumonia Detection challenge.

For more information: www.kaggle.com/c/rsna-pneumonia-detection-challenge

Related Artificial Intelligence Content

RSNA Launches Artificial Intelligence Initiatives for 2018 and Beyond

Related Content

The Caption Guidance software uses artificial intelligence to guide users to get optimal cardiac ultrasound images in a point of care ultrasound (POCUS) setting.

The Caption Guidance software uses artificial intelligence to guide users to get optimal cardiac ultrasound images in a point of care ultrasound (POCUS) setting.

News | Artificial Intelligence | February 13, 2020
February 13, 2020 — The U.S.
Varian announced it has received FDA 510(k) clearance for its Ethos therapy, an Adaptive Intelligence solution. Ethos therapy is an artificial intelligence (AI)-driven holistic solution that provides an opportunity to transform cancer care.
News | Image Guided Radiation Therapy (IGRT) | February 11, 2020
February 11, 2020 — Varian announced it has received FDA 510(k) c
Mammograms of a 49-year-old woman with invasive lobular carcinoma on the right-side breast

Mammograms of a 49-year-old woman with invasive lobular carcinoma on the right-side breast. A small mass with micro-calcifications on the right-side breast was detected correctly by AI with an abnormality score of 96%. This case was recalled by 7 out of 14 radiologists (4 breast radiologists and 3 general radiologists) initially (without AI) and all 14 radiologists recalled this case correctly with the assistance of AI.

News | Artificial Intelligence | February 11, 2020
February 11, 2020 — A new study, published in...
PaxeraHealth enterprise imaging, PACS, VNA solutions
News | Enterprise Imaging | February 11, 2020
February 11, 2020 — Enterprise Imaging developer PaxeraHealth
aycan completed the install of a second aycan xray-print solution at Inspira Health in New Jersey
News | X-Ray | February 10, 2020
February 10, 2020 — aycan completed the install of a second...
An example of artificial intelligence (AI) being developed by Hitachi to automatically review and identify nodules on lung CT scans. This is part of a suite of AI apps Hitachi is developing. This example was being shown as a work in progress at RSNA 2019.

An example of artificial intelligence (AI) being developed by Hitachi to automatically review and identify nodules on lung CT scans. This is part of a suite of AI apps Hitachi is developing. This example was being shown as a work in progress at RSNA 2019. Photo by Dave Fornell.

Feature | Artificial Intelligence | February 07, 2020 | Sanjay Parekh, Ph.D. 
February 7, 2020 – At the 2019 Radiological Society...
Sponsored Content | Videos | Artificial Intelligence | February 07, 2020
At RSNA19, GE Healthcare introduced its...
Sponsored Content | Videos | Artificial Intelligence | February 06, 2020
ProFound AI is an FDA-cleared artificial intelligence (AI) system for reading 3-D breast tomosynthesis images.
Infervision’s deep learning medical imaging platform is helping screen patients for the coronavirus in China. It acts as second pair of eyes to identify multiple diseases from one set of chest scans. The artificial intelligence (AI) can provide a complete view of the nodule, including volume and density.

Infervision’s deep learning medical imaging platform is helping screen patients for the coronavirus in China. It acts as second pair of eyes to identify multiple diseases from one set of chest scans. The artificial intelligence (AI) can provide a complete view of the nodule, including volume and density.

News | Artificial Intelligence | February 04, 2020
February 4, 2020 — Since January 2020, the...
While electronic medical record systems have helped consolidate most patient data into one location, medical imaging IT systems has proved to be more difficult to replicate by large EMR vendors. This has made room in the market for third-party radiology IT vendors that allow easy integration with the larger EMRs like Epic and Cerner. This image shows Agfa's enterprise imaging system, leveraging its ability to be accessed anywhere with internet connection and pull images from radiology and surgery.

While electronic medical record systems have helped consolidate most patient data into one location, medical imaging IT systems has proved to be more difficult to replicate by large EMR vendors. This has made room in the market for third-party radiology information system vendors that allow easy integration with the larger EMRs like Epic and Cerner. This image shows Agfa's enterprise imaging system, leveraging its ability to be accessed anywhere with an internet connection and able to pull in images from both radiology and surgery. 

Feature | Enterprise Imaging | February 02, 2020 | Steve Holloway
The growing influence and uptake of electronic medical records (EMRs) in healthcare has driven debate over the future