News | Artificial Intelligence | August 27, 2018

RSNA Announces Pneumonia Detection Machine Learning Challenge

Teams will develop algorithms to identify and localize pneumonia in chest X-rays

RSNA Announces Pneumonia Detection Machine Learning Challenge

August 27, 2018 — The Radiological Society of North America (RSNA) has launched its second annual machine learning challenge. The RSNA Pneumonia Detection challenge invites teams to develop algorithms to identify and localize pneumonia in chest X-rays.

“The expectation that artificial intelligence [AI] will soon provide valuable tools for radiology continues to grow,” said Luciano Prevedello, M.D., MPH, chief of the Division of Medical Imaging Informatics at The Ohio State University and chair of the Machine Learning Steering Subcommittee of the RSNA Radiology Informatics Committee (RIC). “By organizing machine learning data challenges, RSNA is playing an important role in fostering and demonstrating these capabilities.”

A machine learning data challenge is a competition among AI researchers to create applications that perform a defined task according to specified performance measures. The Pneumonia Detection challenge will be based on a publicly available dataset published by the National Institutes of Health (NIH) that has been carefully annotated by multiple expert reviewers. The RSNA Machine Learning Steering Subcommittee collaborated with volunteer specialists from the Society of Thoracic Radiology to annotate the dataset, identifying abnormal areas in the lung images and assessing the probability of pneumonia.

“A successful Machine Learning challenge needs to begin with a data set accurate and large enough to provide ‘ground truth,’” said Safwan Halabi, M.D., medical director of radiology informatics at Stanford University and chair of the Machine Learning Data Standards Subcommittee of the RIC. “Developers build their applications by training them on the dataset and challenge organizers use a segment of the dataset to measure their performance.”

This year’s challenge will be conducted using the competition platform provided by Kaggle Inc., a subsidiary of Alphabet Inc., which is also the parent company of Google. The Kaggle platform will provide a home page for the challenge, controlled access to the challenge datasets, a discussion forum for participants and the repository where they submit their results. Kaggle has recognized the RSNA Pneumonia Detection Challenge as a public good and will provide $30,000 in prize money for the winning entries.

The challenge will have two phases: training and evaluation. During the training phase, which starts Aug. 27 and runs until Oct. 17, participants will use the training portion of the dataset to develop algorithms that duplicate the annotations provided by the radiologist observers. In the evaluation phase, which runs Oct. 18-24, participants will run their algorithms on the testing portion of the dataset, from which the annotations are withheld. Oct. 24 is the deadline for submission of test results, which will then be compared to the “ground-truth” values supplied by the annotations of the expert observers to determine the winners. The results will be announced in early November, and the top submissions will be recognized Nov. 26 in a session at the Machine Learning Showcase during the RSNA 2018 annual meeting, Nov. 25-30 in Chicago.

Last year’s challenge had more than 200 participants, many of whom formed teams comprising both data scientists and radiologists. “Radiologists have a critical role to play in the challenge, even if they don’t have extensive experience with machine learning,” Prevedello said. “They can provide insight, based on their clinical experience, to refine or supplement the annotations on the training data set and help developers identify systematic errors in the results of their algorithms.”

Radiologists interested in participating can set up a free account on the Kaggle site and express their interests and qualifications in the forum for the RSNA Pneumonia Detection challenge.

For more information: www.kaggle.com/c/rsna-pneumonia-detection-challenge

Related Artificial Intelligence Content

RSNA Launches Artificial Intelligence Initiatives for 2018 and Beyond

Related Content

Videos | PET-CT | December 04, 2020
This is an example of Canon's...
Interactive VR models complement 3D printing as a faster, low-cost solution for remote planning and collaboration
News | Virtual and Augmented Reality | December 04, 2020
December 4, 2020 — Materialise NV, a global leader in medical 3-D printing software and services, introduced its new
The Nanox.ARC next-generation cold cathode X-ray prototype at the 2020 Radiology Society of North America (RSNA) virtual meeting. The live demonstration featured a range of 2-D and 3-D imaging procedures using its groundbreaking digital X-ray tube in a unique multi-source array structure. #RSNA20 #RSNA2020

The Nanox.ARC next-generation cold cathode X-ray prototype at the 2020 Radiology Society of North America (RSNA) virtual meeting. The live demonstration featured a range of 2-D and 3-D imaging procedures using its groundbreaking digital X-ray tube in a unique multi-source array structure.

News | X-Ray | December 03, 2020
December 3, 2020 — Nanox said today it successfully demonstrated its Nanox.ARC next-generation cold cathode...
The company’s ClearRead Xray Confirm (Enhanced) and ClearRead CT – breakthrough lung nodule detection applications­ – have received clearance for sale in Australia
News | Computed Tomography (CT) | December 03, 2020
December 3, 2020 — Riverain Technologies, a U.S.-based developer of...
A new hybrid X-ray detector developed by the University of Surrey outperforms commercial devices — and could lead to more accurate cancer therapy.
News | X-Ray | December 03, 2020
December 3, 2020 — A new hybrid X-ray detector deve
Generated heatmaps appropriately highlighted abnormalities in the lung fields in those images accurately labeled as COVID-19 positive (A-C) in contrast to images which were accurately labeled as negative for COVID-19 (D). Intensity of colors on the heatmap correspond to features of the image that are important for prediction of COVID-19 positivity. Image courtesy of Northwestern University

Generated heatmaps appropriately highlighted abnormalities in the lung fields in those images accurately labeled as COVID-19 positive (A-C) in contrast to images which were accurately labeled as negative for COVID-19 (D). Intensity of colors on the heatmap correspond to features of the image that are important for prediction of COVID-19 positivity. Image courtesy of Northwestern University

News | Artificial Intelligence | December 03, 2020
December 3, 2020 — Northwestern University researchers ha
An example of a body composition analysis of an abdominal CT slice with the subcutaneous fat in green, skeletal muscle red and visceral fat in yellow. This was automatically identified and analyzed via a deep learning algorithm to assess the risk for heart attack and stroke in more than 12,000 patients. #RSNA2020 #RSNA20 #RSNA

An example of a body composition analysis of an abdominal CT slice with the subcutaneous fat in green, skeletal muscle red and visceral fat in yellow. This was automatically identified and analyzed via a deep learning algorithm to assess the risk for heart attack and stroke in more than 12,000 patients.

Feature | Artificial Intelligence | December 02, 2020
December 2, 2020 – Automated deep learning analysis of abdominal...
 Riverain Technologies will showcase its artificial intelligence solutions and enhancements at the now-virtual 2020 Radiological Society of North America Annual Meeting (RSNA).
News | Artificial Intelligence | December 02, 2020
December 2, 2020 — Riverain Technologies will showcase its...
Studies show new deep-learning algorithm helps radiologists detect breast cancers in their early stages when used with the Genius 3D Mammography exam
News | Breast Imaging | December 02, 2020
December 2, 2020 — Hologic, Inc., an innovative medical technology company primarily focused on improving women’s hea