Feature | Advanced Visualization | July 02, 2019 | By Jeff Zagoudis

Augmented Reality Versus 3-D Printing for Radiology

New research explores potential of advanced visualization technologies to improve procedural outcomes and increase patient satisfaction

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model.

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model. Photo courtesy of Nicole Wake, Ph.D.

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model.

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model. Photo courtesy of Nicole Wake, Ph.D.

Three-dimensional (3-D) printing and augmented reality (AR) are two of the more exciting technologies being explored in radiology today. In their own way, each technology gives form and depth to medical imaging data, offering a new perspective for physicians and patients. Early explorations have promised new ways to plan surgical and interventional procedures, and new methods to help patients understand what care is being delivered and how. For now, however, both technologies are still largely viewed as novel, with at best anecdotal evidence of how they can be translated into clinical use. A pair of recent large-scale studies may offer a sense of how to collect quantitative evidence that these technologies can benefit all members of the healthcare team.

Augmented reality technology — which superimposes virtual images onto the user’s real-world surroundings, creating a “mixed reality” — took off in 2016 when Microsoft released the first consumer-grade mixed reality headset, the HoloLens. The U.S. Food and Drug Administration (FDA) approved the first healthcare-focused AR technology, Novarad’s OpenSight AR system, in November 2018. OpenSight uses a HoloLens headset to register a virtual overlay of a patient’s magnetic resonance (MR) or computed tomography (CT) scans onto the patient.

 

The Path to AR and 3-D

Nicole Wake, Ph.D., an assistant professor in the Department of Radiology at Albert Einstein College of Medicine in New York, began exploring advanced visualization while working on her Ph.D. thesis at New York University School of Medicine. “In our lab we had an industrial 3-D printer that was bought to print custom-made housings for radiofrequency (RF) coils. I started using this printer to make patient-specific anatomical models,” she said. “Then the pathway to create AR and virtual reality (VR) models from radiological imaging data is similar to printing, so it made sense to explore those technologies as well.”

When the HoloLens was first released, Wake was already working in 3-D printing but recognized the limitations of the technology — primarily the amount of time required to create a 3-D printed model. Building the digital model that serves as a blueprint for the printer might only take an hour or two, according to Wake, but the actual printing process can take several hours. “The printing times vary based on the technology used and the volumes. Kidneys can take approximately 10 hours for printing, maybe a little more or less depending on how big each one is,” she said. Creating a three-dimensional model of an entire pelvis could take as long as 30 hours.

Seeing the potential of AR to address these limitations, Wake applied for a HoloLens developer license and taught herself how to program with the technology. The process is similar to building a digital model prior to the 3-D printing process. After the initial CT or MR image is acquired, the user performs image segmentation. “Once you have the images, you have to divide out the regions of interest you want to show in the model,” she said. This step can be time-consuming depending on the type of image acquired, the segmentation software and the experience of the user. From there, the model is brought into the AR or 3-D printing software for a full workup.

Wake noted that while prior programming experience was helpful in working with the HoloLens, much of the interface was fairly intuitive.

 

3-D Printing for Procedure Planning

One of the first applications for 3-D modeling that Wake investigated was for procedural planning. At the time, she was working on her Ph.D. thesis with Hersh Chandarana, M.D., an abdominal radiologist with NYU Langone Health, so her initial research has focused on 3-D modeling to aid kidney and prostate cancer patients.

In a study published in 2017, Wake explored using 3-D printing and augmented reality to help surgeons with procedural planning. Retrospectively evaluating the cases of patients who received clinically indicated and research MRI prior to surgery for renal masses, the research team created patient-specific 3-D printed models. Three experienced urologists reviewed all patient cases, first with 2-D imaging alone and then with conventional imaging plus the 3-D models. Data indicated that the 3-D models resulted in a change to the type of procedure planned as much as 50 percent of the time.1

 

AR Versus 3-D Printing for Patient Education

Wake and colleagues undertook a second study, published this year, looking at how 3-D printing and AR might influence patient understanding of an upcoming procedure. The research team worked with 200 patients in two disease/procedure groups:

•    One group had magnetic resonance (MR)-visible prostate cancer and was undergoing either robotic-assisted radical prostatectomy or focal ablative therapy; and

•    One group had renal masses and were undergoing partial nephrectomy.

 

All patients went through routine imaging protocols; prostate cancer patients received a multiparametric MRI exam, while kidney patients received either a 1.5T MR scan or a CT scan. All patients were then randomized to pre-planning with imaging alone, or imaging plus a patient-specific 3-D model. Models for the second group were either 3-D printed, visualized in AR, or viewed in 3-D on a 2-D monitor. To measure patient response, Wake and colleagues used a five-point Likert scale survey — on a scale of 1 to 5, patients rated their understanding of:

•    Their cancer/disease itself;

•    The size of the tumor/cancer;

•    The location of the tumor/cancer;

•    Why the surgeon chose their given treatment plan; and

•    Their comfort with the surgical plan.

 

Patients randomized to 3-D models took the survey twice — once after viewing conventional imaging only, and a second time after viewing the 3-D model. Regardless of disease type and across all variables measured, patient comprehension was highest with 3-D printed models compared to any other visualization method. In the kidney patient cohort, the 3-D printed models actually helped patient decision-making in regards to the procedure: One patient ended up going through with the nephrectomy rather than forego the procedure, and another was able to decide between radical and partial nephrectomy.2   

“Just seeing the anatomy in 3-D, whether a virtual model or a physical tactile model, is much more powerful than simply viewing grayscale medical images on a 2-D screen, especially for patients who might not have seen a medical image before so they don’t really know what they’re looking at,” Wake said.   

While all three-dimensional models were deemed helpful by patients, none found augmented reality models to be as helpful as a physical 3-D printed model. Wake noted that the AR setup can be difficult to work with, due to a heavy headset and a limited field of view with the first-generation HoloLens. “Depending on the size of your model, you might have to move your head to see different parts, or step further back so you can see more in the window,” she stated.

Despite the utility of both technologies, cost remains a prohibitive factor blocking widespread adoption. Wake said a planned future prospective study will provide a deeper cost analysis in hopes of expanding the utility of 3-D printing and AR.

 

References

1.  Wake N., Rude T., Kang S.K., et al. 3D Printed Renal Cancer Models Derived from MRI Data: Application in Pre-Surgical Planning. Abdominal Radiology, May 1, 2018. doi:10.1007/s00261-016-1022-2.

2.  Wake N., Rosenkrantz A.B., Huang R., et al. Patient-specific 3D printed and augmented reality kidney and prostate cancer models: impact on patient education. 3D Printing in Medicine, Feb. 2019

Related Content

3D Systems Earns Additional FDA Clearance for D2P Medical 3-D Printing Software

3D Systems’ D2P FDA-cleared software allows clinicians to 3-D-print diagnostic patient-specific anatomic models. Image courtesy of 3D Systems.

Technology | Medical 3-D Printing | September 12, 2019
3D Systems has received additional U.S. Food and Drug Administration (FDA) 510(k) clearance for its D2P software...
Sudhen Desai, M.D.

Sudhen Desai, M.D.

Feature | Pediatric Imaging | September 04, 2019 | By Jeff Zagoudis
Burnout has become a popular buzzword in today’s business world, meant to describe prolonged periods of stress in the
Medical Imaging Rates Continue to Rise Despite Push to Reduce Their Use
News | Radiology Imaging | September 03, 2019
Despite a broad campaign among physician groups to reduce the amount of medical imaging, use rates of various scans...
The Siemens Ysio Max digital radiography system.
Webinar | Digital Radiography (DR) | September 03, 2019
The Webinar "Benefits of Advanced Automation in X-ray" will be presented Wednesday, Sept. 25 at 1 p.m.
A SPECT nuclear scan of the heart to show perfusion defects in the myocardium due to coronary artery blockages or heart attack. The imaging uses the Mo-99 based medical imaging isotope Tc-99m. The U.S. government has created policy to move away from use of highly enriched uranium (HEU) to low-enriched uranium (LEU) for Mo-99 isotope production, but there is one hold out who has not yet converted before a 2020 deadline. Photo courtesy of Philips Healthcare.

A SPECT nuclear scan of the heart to show perfusion defects in the myocardium due to coronary artery blockages or heart attack. The imaging uses the Mo-99 based medical imaging isotope Tc-99m. The U.S. government has created policy to move away from use of highly enriched uranium (HEU) to low-enriched uranium (LEU) for Mo-99 isotope production, but there is one holdout who has not yet converted before a 2020 deadline. Photo courtesy of Philips Healthcare.

Feature | Nuclear Imaging | August 30, 2019 | Dave Fornell, Editor
In a surprising move, the National Institute for Radioelements (IRE) has applied for a new license to export highly e
Delaware Imaging Network Now Offers NeuroQuant Brain Imaging MRI Software
News | Neuro Imaging | August 29, 2019
Delaware Imaging Network (DIN), Delaware’s largest network of outpatient medical imaging centers, has added NeuroQuant...
Smoldering Spots in the Brain May Signal Severe MS

NIH researchers found that dark rimmed spots representing ongoing, “smoldering” inflammation, may be a hallmark of more disabling forms of multiple sclerosis. Image courtesy of Reich lab, NIH/NINDS.

News | Neuro Imaging | August 22, 2019
Aided by a high-powered brain scanner and a 3-D printer, National Institutes of Health (NIH) researchers peered inside...
Videos | Computed Tomography (CT) | August 21, 2019
This is a quick walk around of a mobile 32-slice...
RSNA and ACR to Collaborate on Landmark Medical 3D Printing Registry
News | Medical 3-D Printing | August 08, 2019
The Radiological Society of North America (RSNA) and the American College of Radiology (ACR) will launch a new medical...
Synaptive Medical Launches Modus Plan With Automated Tractography Segmentation
Technology | Neuro Imaging | August 07, 2019
Synaptive Medical announced the U.S. launch and availability of Modus Plan featuring BrightMatter AutoSeg. This release...