Feature | Advanced Visualization | July 02, 2019 | By Jeff Zagoudis

Augmented Reality Versus 3-D Printing for Radiology

New research explores potential of advanced visualization technologies to improve procedural outcomes and increase patient satisfaction

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model.

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model. Photo courtesy of Nicole Wake, Ph.D.

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model.

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model. Photo courtesy of Nicole Wake, Ph.D.

Three-dimensional (3-D) printing and augmented reality (AR) are two of the more exciting technologies being explored in radiology today. In their own way, each technology gives form and depth to medical imaging data, offering a new perspective for physicians and patients. Early explorations have promised new ways to plan surgical and interventional procedures, and new methods to help patients understand what care is being delivered and how. For now, however, both technologies are still largely viewed as novel, with at best anecdotal evidence of how they can be translated into clinical use. A pair of recent large-scale studies may offer a sense of how to collect quantitative evidence that these technologies can benefit all members of the healthcare team.

Augmented reality technology — which superimposes virtual images onto the user’s real-world surroundings, creating a “mixed reality” — took off in 2016 when Microsoft released the first consumer-grade mixed reality headset, the HoloLens. The U.S. Food and Drug Administration (FDA) approved the first healthcare-focused AR technology, Novarad’s OpenSight AR system, in November 2018. OpenSight uses a HoloLens headset to register a virtual overlay of a patient’s magnetic resonance (MR) or computed tomography (CT) scans onto the patient.

 

The Path to AR and 3-D

Nicole Wake, Ph.D., an assistant professor in the Department of Radiology at Albert Einstein College of Medicine in New York, began exploring advanced visualization while working on her Ph.D. thesis at New York University School of Medicine. “In our lab we had an industrial 3-D printer that was bought to print custom-made housings for radiofrequency (RF) coils. I started using this printer to make patient-specific anatomical models,” she said. “Then the pathway to create AR and virtual reality (VR) models from radiological imaging data is similar to printing, so it made sense to explore those technologies as well.”

When the HoloLens was first released, Wake was already working in 3-D printing but recognized the limitations of the technology — primarily the amount of time required to create a 3-D printed model. Building the digital model that serves as a blueprint for the printer might only take an hour or two, according to Wake, but the actual printing process can take several hours. “The printing times vary based on the technology used and the volumes. Kidneys can take approximately 10 hours for printing, maybe a little more or less depending on how big each one is,” she said. Creating a three-dimensional model of an entire pelvis could take as long as 30 hours.

Seeing the potential of AR to address these limitations, Wake applied for a HoloLens developer license and taught herself how to program with the technology. The process is similar to building a digital model prior to the 3-D printing process. After the initial CT or MR image is acquired, the user performs image segmentation. “Once you have the images, you have to divide out the regions of interest you want to show in the model,” she said. This step can be time-consuming depending on the type of image acquired, the segmentation software and the experience of the user. From there, the model is brought into the AR or 3-D printing software for a full workup.

Wake noted that while prior programming experience was helpful in working with the HoloLens, much of the interface was fairly intuitive.

 

3-D Printing for Procedure Planning

One of the first applications for 3-D modeling that Wake investigated was for procedural planning. At the time, she was working on her Ph.D. thesis with Hersh Chandarana, M.D., an abdominal radiologist with NYU Langone Health, so her initial research has focused on 3-D modeling to aid kidney and prostate cancer patients.

In a study published in 2017, Wake explored using 3-D printing and augmented reality to help surgeons with procedural planning. Retrospectively evaluating the cases of patients who received clinically indicated and research MRI prior to surgery for renal masses, the research team created patient-specific 3-D printed models. Three experienced urologists reviewed all patient cases, first with 2-D imaging alone and then with conventional imaging plus the 3-D models. Data indicated that the 3-D models resulted in a change to the type of procedure planned as much as 50 percent of the time.1

 

AR Versus 3-D Printing for Patient Education

Wake and colleagues undertook a second study, published this year, looking at how 3-D printing and AR might influence patient understanding of an upcoming procedure. The research team worked with 200 patients in two disease/procedure groups:

•    One group had magnetic resonance (MR)-visible prostate cancer and was undergoing either robotic-assisted radical prostatectomy or focal ablative therapy; and

•    One group had renal masses and were undergoing partial nephrectomy.

 

All patients went through routine imaging protocols; prostate cancer patients received a multiparametric MRI exam, while kidney patients received either a 1.5T MR scan or a CT scan. All patients were then randomized to pre-planning with imaging alone, or imaging plus a patient-specific 3-D model. Models for the second group were either 3-D printed, visualized in AR, or viewed in 3-D on a 2-D monitor. To measure patient response, Wake and colleagues used a five-point Likert scale survey — on a scale of 1 to 5, patients rated their understanding of:

•    Their cancer/disease itself;

•    The size of the tumor/cancer;

•    The location of the tumor/cancer;

•    Why the surgeon chose their given treatment plan; and

•    Their comfort with the surgical plan.

 

Patients randomized to 3-D models took the survey twice — once after viewing conventional imaging only, and a second time after viewing the 3-D model. Regardless of disease type and across all variables measured, patient comprehension was highest with 3-D printed models compared to any other visualization method. In the kidney patient cohort, the 3-D printed models actually helped patient decision-making in regards to the procedure: One patient ended up going through with the nephrectomy rather than forego the procedure, and another was able to decide between radical and partial nephrectomy.2   

“Just seeing the anatomy in 3-D, whether a virtual model or a physical tactile model, is much more powerful than simply viewing grayscale medical images on a 2-D screen, especially for patients who might not have seen a medical image before so they don’t really know what they’re looking at,” Wake said.   

While all three-dimensional models were deemed helpful by patients, none found augmented reality models to be as helpful as a physical 3-D printed model. Wake noted that the AR setup can be difficult to work with, due to a heavy headset and a limited field of view with the first-generation HoloLens. “Depending on the size of your model, you might have to move your head to see different parts, or step further back so you can see more in the window,” she stated.

Despite the utility of both technologies, cost remains a prohibitive factor blocking widespread adoption. Wake said a planned future prospective study will provide a deeper cost analysis in hopes of expanding the utility of 3-D printing and AR.

 

References

1.  Wake N., Rude T., Kang S.K., et al. 3D Printed Renal Cancer Models Derived from MRI Data: Application in Pre-Surgical Planning. Abdominal Radiology, May 1, 2018. doi:10.1007/s00261-016-1022-2.

2.  Wake N., Rosenkrantz A.B., Huang R., et al. Patient-specific 3D printed and augmented reality kidney and prostate cancer models: impact on patient education. 3D Printing in Medicine, Feb. 2019

Related Content

A cardiac MRI of athletes who had COVID-19 is seven times more effective in detecting inflammation of the heart than symptom-based testing, according to a study led by researchers at The Ohio State University Wexner Medical Center and College of Medicine with 12 other Big Ten programs.

Cardiac Magnetic Resonance Imaging in Athletes With Clinical and Subclinical Myocarditis A-D, Athlete A with subclinical possible myocarditis was asymptomatic with normal electrocardiogram (ECG), echocardiogram, and high-sensitivity troponin findings. A, T2 mapping showing elevated T2 in basal-mid inferolateral wall in short axis view. B, late gadolinium enhancement (LGE) in the basal inferolateral wall in short axis view. C, Postcontrast steady state-free precession (SSFP) images showing contrast uptake in the basal-mid inferolateral wall in short axis view. D, LGE in the inferolateral wall in 3-chamber view. E-H, Athlete B with subclinical probable myocarditis was asymptomatic with normal ECG, normal echocardiogram, and elevated high-sensitivity troponin findings. E, T2 mapping showing elevated T2 in the anteroseptal wall in short axis view. F, LGE in the anteroseptal wall in 3-chamber view. G, T2 mapping showing elevated T2 in the anteroseptal wall in 3-chamber view. F, Postcontrast SSFP image showing pericardial effusion in short axis view. I-K, Athlete C with clinical myocarditis and chest pain, dyspnea, abnormal ECG, normal echocardiogram, and normal troponin findings. I, T2 mapping showing elevated T2 in the lateral wall short axis view. J, Postcontrast SSFP images showing contrast uptake in midlateral wall in short axis view. K, LGE in the epicardial midlateral wall in short axis view. L-N, Athlete D with clinical myocarditis, chest pain, abnormal ECG, echocardiogram, and troponin findings. L, T1 mapping showing elevated native T1 in midlateral wall in short axis view. M, T2 mapping showing elevated T2 in the midlateral wall in short axis view. N, LGE in the epicardial midlateral wall in short axis view. IR indicates inferior right view; IRP, inferior, right, posterior view; PLI, posterior, left, inferior view; SL, superior left view; SLA, superior, left, anterior view. Image courtesy of JAMA Cardiol. Published online May 27, 2021. doi:10.1001/jamacardio.2021.2065

News | Cardiac Imaging | June 15, 2021
June 15, 2021 — A...
Rensselaer algorithm can identify risk of cardiovascular disease using lung cancer scan #CT
News | Computed Tomography (CT) | June 14, 2021
June 14, 2021 — Heart disease and cancer are the ...
A new imaging technique has the potential to detect neurological disorders — such as Alzheimer's disease — at their earliest stages, enabling physicians to diagnose and treat patients more quickly. Termed super-resolution, the imaging methodology combines position emission tomography (PET) with an external motion tracking device to create highly detailed images of the brain.

Result of the Hoffman brain phantom study. Top row: same PET slice reconstructed with A) 2mm static OSEM, B) 1mm static OSEM, C) proposed SR method and D) corresponding CT slice (note that the CT image can be treated as a high-resolution reference). Middle row: zoom on region of interest for corresponding images. Bottom row: Line profiles for corresponding data. Image created by Y Chemli, et al., Gordon Center for Medical Imaging: Department of Radiology Massachusetts General Hospital, Harvard Medical School, Boston, MA.

News | PET Imaging | June 14, 2021
June 14, 2021 — A new imaging technique has the potential to detect neurological disorders — such as...
Prediction performance of DL compared to quantitative measures and Kaplan-Meier curves for quartiles of DL. Image created by Singh et al., Cedars-Sinai Medical Center, Los Angeles, CA.

Prediction performance of DL compared to quantitative measures and Kaplan-Meier curves for quartiles of DL. Image created by Singh et al., Cedars-Sinai Medical Center, Los Angeles, CA.

News | SPECT Imaging | June 14, 2021
June 14, 2021 — An advanced artificial i...
Accuray Incorporated announced the company has received CE Mark certification for its ClearRT helical fan-beam kVCT imaging capability.
News | Radiation Therapy | June 11, 2021
June 11, 2021 — Accuray Incorporated announced the company has received CE Mark certification for its...
The new X-ray scanner can provide detailed information about the internal makeup of rocks, which could be useful for archaeologists studying fossils or miners making decisions about which ore to use in their extraction facilities. Image courtesy of Joel Greenberg, Duke University

The new X-ray scanner can provide detailed information about the internal makeup of rocks, which could be useful for archaeologists studying fossils or miners making decisions about which ore to use in their extraction facilities. Image courtesy of Joel Greenberg, Duke University

News | X-Ray | June 10, 2021
June 10, 2021 — Engineers at Duke University have demonstrated a prot
News | PET-CT | June 10, 2021
June 10, 2021 — Bringing the power of...
Richard Ernst was considered the father of nuclear magnetic resonance imaging (MRI)
News | Magnetic Resonance Imaging (MRI) | June 10, 2021
June 10, 2021 — The Washington Post has reported that Richard R.