Feature | Advanced Visualization | July 02, 2019 | By Jeff Zagoudis

Augmented Reality Versus 3-D Printing for Radiology

New research explores potential of advanced visualization technologies to improve procedural outcomes and increase patient satisfaction

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model.

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model. Photo courtesy of Nicole Wake, Ph.D.

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model.

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model. Photo courtesy of Nicole Wake, Ph.D.

Three-dimensional (3-D) printing and augmented reality (AR) are two of the more exciting technologies being explored in radiology today. In their own way, each technology gives form and depth to medical imaging data, offering a new perspective for physicians and patients. Early explorations have promised new ways to plan surgical and interventional procedures, and new methods to help patients understand what care is being delivered and how. For now, however, both technologies are still largely viewed as novel, with at best anecdotal evidence of how they can be translated into clinical use. A pair of recent large-scale studies may offer a sense of how to collect quantitative evidence that these technologies can benefit all members of the healthcare team.

Augmented reality technology — which superimposes virtual images onto the user’s real-world surroundings, creating a “mixed reality” — took off in 2016 when Microsoft released the first consumer-grade mixed reality headset, the HoloLens. The U.S. Food and Drug Administration (FDA) approved the first healthcare-focused AR technology, Novarad’s OpenSight AR system, in November 2018. OpenSight uses a HoloLens headset to register a virtual overlay of a patient’s magnetic resonance (MR) or computed tomography (CT) scans onto the patient.

 

The Path to AR and 3-D

Nicole Wake, Ph.D., an assistant professor in the Department of Radiology at Albert Einstein College of Medicine in New York, began exploring advanced visualization while working on her Ph.D. thesis at New York University School of Medicine. “In our lab we had an industrial 3-D printer that was bought to print custom-made housings for radiofrequency (RF) coils. I started using this printer to make patient-specific anatomical models,” she said. “Then the pathway to create AR and virtual reality (VR) models from radiological imaging data is similar to printing, so it made sense to explore those technologies as well.”

When the HoloLens was first released, Wake was already working in 3-D printing but recognized the limitations of the technology — primarily the amount of time required to create a 3-D printed model. Building the digital model that serves as a blueprint for the printer might only take an hour or two, according to Wake, but the actual printing process can take several hours. “The printing times vary based on the technology used and the volumes. Kidneys can take approximately 10 hours for printing, maybe a little more or less depending on how big each one is,” she said. Creating a three-dimensional model of an entire pelvis could take as long as 30 hours.

Seeing the potential of AR to address these limitations, Wake applied for a HoloLens developer license and taught herself how to program with the technology. The process is similar to building a digital model prior to the 3-D printing process. After the initial CT or MR image is acquired, the user performs image segmentation. “Once you have the images, you have to divide out the regions of interest you want to show in the model,” she said. This step can be time-consuming depending on the type of image acquired, the segmentation software and the experience of the user. From there, the model is brought into the AR or 3-D printing software for a full workup.

Wake noted that while prior programming experience was helpful in working with the HoloLens, much of the interface was fairly intuitive.

 

3-D Printing for Procedure Planning

One of the first applications for 3-D modeling that Wake investigated was for procedural planning. At the time, she was working on her Ph.D. thesis with Hersh Chandarana, M.D., an abdominal radiologist with NYU Langone Health, so her initial research has focused on 3-D modeling to aid kidney and prostate cancer patients.

In a study published in 2017, Wake explored using 3-D printing and augmented reality to help surgeons with procedural planning. Retrospectively evaluating the cases of patients who received clinically indicated and research MRI prior to surgery for renal masses, the research team created patient-specific 3-D printed models. Three experienced urologists reviewed all patient cases, first with 2-D imaging alone and then with conventional imaging plus the 3-D models. Data indicated that the 3-D models resulted in a change to the type of procedure planned as much as 50 percent of the time.1

 

AR Versus 3-D Printing for Patient Education

Wake and colleagues undertook a second study, published this year, looking at how 3-D printing and AR might influence patient understanding of an upcoming procedure. The research team worked with 200 patients in two disease/procedure groups:

•    One group had magnetic resonance (MR)-visible prostate cancer and was undergoing either robotic-assisted radical prostatectomy or focal ablative therapy; and

•    One group had renal masses and were undergoing partial nephrectomy.

 

All patients went through routine imaging protocols; prostate cancer patients received a multiparametric MRI exam, while kidney patients received either a 1.5T MR scan or a CT scan. All patients were then randomized to pre-planning with imaging alone, or imaging plus a patient-specific 3-D model. Models for the second group were either 3-D printed, visualized in AR, or viewed in 3-D on a 2-D monitor. To measure patient response, Wake and colleagues used a five-point Likert scale survey — on a scale of 1 to 5, patients rated their understanding of:

•    Their cancer/disease itself;

•    The size of the tumor/cancer;

•    The location of the tumor/cancer;

•    Why the surgeon chose their given treatment plan; and

•    Their comfort with the surgical plan.

 

Patients randomized to 3-D models took the survey twice — once after viewing conventional imaging only, and a second time after viewing the 3-D model. Regardless of disease type and across all variables measured, patient comprehension was highest with 3-D printed models compared to any other visualization method. In the kidney patient cohort, the 3-D printed models actually helped patient decision-making in regards to the procedure: One patient ended up going through with the nephrectomy rather than forego the procedure, and another was able to decide between radical and partial nephrectomy.2   

“Just seeing the anatomy in 3-D, whether a virtual model or a physical tactile model, is much more powerful than simply viewing grayscale medical images on a 2-D screen, especially for patients who might not have seen a medical image before so they don’t really know what they’re looking at,” Wake said.   

While all three-dimensional models were deemed helpful by patients, none found augmented reality models to be as helpful as a physical 3-D printed model. Wake noted that the AR setup can be difficult to work with, due to a heavy headset and a limited field of view with the first-generation HoloLens. “Depending on the size of your model, you might have to move your head to see different parts, or step further back so you can see more in the window,” she stated.

Despite the utility of both technologies, cost remains a prohibitive factor blocking widespread adoption. Wake said a planned future prospective study will provide a deeper cost analysis in hopes of expanding the utility of 3-D printing and AR.

 

References

1.  Wake N., Rude T., Kang S.K., et al. 3D Printed Renal Cancer Models Derived from MRI Data: Application in Pre-Surgical Planning. Abdominal Radiology, May 1, 2018. doi:10.1007/s00261-016-1022-2.

2.  Wake N., Rosenkrantz A.B., Huang R., et al. Patient-specific 3D printed and augmented reality kidney and prostate cancer models: impact on patient education. 3D Printing in Medicine, Feb. 2019

Related Content

United Imaging's uMR OMEGA is designed to provide greater access to magnetic resonance imaging (MRI) with the world’s first ultra-wide 75-cm bore 3T MRI.
News | Magnetic Resonance Imaging (MRI) | May 27, 2020
May 27, 2020 — United Imaging's...
Off-site imaging companies are playing a key role in the fight against COVID-19
Feature | Coronavirus (COVID-19) | May 26, 2020 | By Sean Zahniser
After the worst of the COVID-19 pandemic has pas
There were several new developments in digital radiography (DR) technology at the 2019 Radiological Society of North America (RSNA) annual meeting. These trends included integration of artificial intelligence (AI) auto detection technologies, more durable glassless detector plates, and technologies to pull more diagnostic data out of X-ray imaging. Some vendors also have redesigned their DR systems to make them more user-friendly and ergonomic. 
Feature | Digital Radiography (DR) | May 26, 2020 | By Dave Fornell
There were several new developments in digital rad...
Lymphocytic Inflammation in a Lung from a Patient Who Died from Covid-19. The gross appearance of a lung from a patient who died from coronavirus disease 2019 (Covid-19) is shown in Panel A (the scale bar corresponds to 1 cm). The histopathological examination, shown in Panel B, revealed interstitial and perivascular predominantly lymphocytic pneumonia with multifocal endothelialitis (hematoxylin–eosin staining; the scale bar corresponds to 200 μm).

Lymphocytic Inflammation in a Lung from a Patient Who Died from Covid-19. The gross appearance of a lung from a patient who died from coronavirus disease 2019 (Covid-19) is shown in Panel A (the scale bar corresponds to 1 cm). The histopathological examination, shown in Panel B, revealed interstitial and perivascular predominantly lymphocytic pneumonia with multifocal endothelialitis (hematoxylin–eosin staining; the scale bar corresponds to 200 μm). Image courtesy of The New England Journal of Medicine

News | Coronavirus (COVID-19) | May 22, 2020
May 22, 2020 — In a new study in the New
Butterfly iQ devices provide revolutionary portable ultrasound capabilities for faster and easier screening and monitoring
News | Coronavirus (COVID-19) | May 20, 2020
May 20, 2020 —Atrium Health is one of the first major health systems in the nation to put into wide practice a new...
Examples of chest CT images of COVID-19 (+) patients and visualization of features correlated to COVID-19 positivity. For each pair of images, the left image is a CT image showing the segmented lung used as input for the CNN (convolutional neural network algorithm) model trained on CT images only, and the right image shows the heatmap of pixels that the CNN model classified as having SARS-CoV-2 infection (red indicates higher probability). (a) A 51-year-old female with fever and history of exposure to SARS-

Figure 1: Examples of chest CT images of COVID-19 (+) patients and visualization of features correlated to COVID-19 positivity. For each pair of images, the left image is a CT image showing the segmented lung used as input for the CNN (convolutional neural network algorithm) model trained on CT images only, and the right image shows the heatmap of pixels that the CNN model classified as having SARS-CoV-2 infection (red indicates higher probability). (a) A 51-year-old female with fever and history of exposure to SARS-CoV-2. The CNN model identified abnormal features in the right lower lobe (white color), whereas the two radiologists labeled this CT as negative. (b) A 52-year-old female who had a history of exposure to SARS-CoV-2 and presented with fever and productive cough. Bilateral peripheral ground-glass opacities (arrows) were labeled by the radiologists, and the CNN model predicted positivity based on features in matching areas. (c) A 72-year-old female with exposure history to the animal market in Wuhan presented with fever and productive cough. The segmented CT image shows ground-glass opacity in the anterior aspect of the right lung (arrow), whereas the CNN model labeled this CT as negative. (d) A 59-year-old female with cough and exposure history. The segmented CT image shows no evidence of pneumonia, and the CNN model also labeled this CT as negative.  

News | Coronavirus (COVID-19) | May 19, 2020
May 19, 2020 — Mount Sinai researchers are the first in the country to use...
Advanced imaging data exchange is now live in Colorado due to the partnership of Health Images and the Colorado Regional Health Information Organization

Getty Images

News | Radiology Business | May 18, 2020
May 18, 2020 —