Feature | Advanced Visualization | July 02, 2019 | By Jeff Zagoudis

Augmented Reality Versus 3-D Printing for Radiology

New research explores potential of advanced visualization technologies to improve procedural outcomes and increase patient satisfaction

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model.

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model. Photo courtesy of Nicole Wake, Ph.D.

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model.

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model. Photo courtesy of Nicole Wake, Ph.D.

Three-dimensional (3-D) printing and augmented reality (AR) are two of the more exciting technologies being explored in radiology today. In their own way, each technology gives form and depth to medical imaging data, offering a new perspective for physicians and patients. Early explorations have promised new ways to plan surgical and interventional procedures, and new methods to help patients understand what care is being delivered and how. For now, however, both technologies are still largely viewed as novel, with at best anecdotal evidence of how they can be translated into clinical use. A pair of recent large-scale studies may offer a sense of how to collect quantitative evidence that these technologies can benefit all members of the healthcare team.

Augmented reality technology — which superimposes virtual images onto the user’s real-world surroundings, creating a “mixed reality” — took off in 2016 when Microsoft released the first consumer-grade mixed reality headset, the HoloLens. The U.S. Food and Drug Administration (FDA) approved the first healthcare-focused AR technology, Novarad’s OpenSight AR system, in November 2018. OpenSight uses a HoloLens headset to register a virtual overlay of a patient’s magnetic resonance (MR) or computed tomography (CT) scans onto the patient.

 

The Path to AR and 3-D

Nicole Wake, Ph.D., an assistant professor in the Department of Radiology at Albert Einstein College of Medicine in New York, began exploring advanced visualization while working on her Ph.D. thesis at New York University School of Medicine. “In our lab we had an industrial 3-D printer that was bought to print custom-made housings for radiofrequency (RF) coils. I started using this printer to make patient-specific anatomical models,” she said. “Then the pathway to create AR and virtual reality (VR) models from radiological imaging data is similar to printing, so it made sense to explore those technologies as well.”

When the HoloLens was first released, Wake was already working in 3-D printing but recognized the limitations of the technology — primarily the amount of time required to create a 3-D printed model. Building the digital model that serves as a blueprint for the printer might only take an hour or two, according to Wake, but the actual printing process can take several hours. “The printing times vary based on the technology used and the volumes. Kidneys can take approximately 10 hours for printing, maybe a little more or less depending on how big each one is,” she said. Creating a three-dimensional model of an entire pelvis could take as long as 30 hours.

Seeing the potential of AR to address these limitations, Wake applied for a HoloLens developer license and taught herself how to program with the technology. The process is similar to building a digital model prior to the 3-D printing process. After the initial CT or MR image is acquired, the user performs image segmentation. “Once you have the images, you have to divide out the regions of interest you want to show in the model,” she said. This step can be time-consuming depending on the type of image acquired, the segmentation software and the experience of the user. From there, the model is brought into the AR or 3-D printing software for a full workup.

Wake noted that while prior programming experience was helpful in working with the HoloLens, much of the interface was fairly intuitive.

 

3-D Printing for Procedure Planning

One of the first applications for 3-D modeling that Wake investigated was for procedural planning. At the time, she was working on her Ph.D. thesis with Hersh Chandarana, M.D., an abdominal radiologist with NYU Langone Health, so her initial research has focused on 3-D modeling to aid kidney and prostate cancer patients.

In a study published in 2017, Wake explored using 3-D printing and augmented reality to help surgeons with procedural planning. Retrospectively evaluating the cases of patients who received clinically indicated and research MRI prior to surgery for renal masses, the research team created patient-specific 3-D printed models. Three experienced urologists reviewed all patient cases, first with 2-D imaging alone and then with conventional imaging plus the 3-D models. Data indicated that the 3-D models resulted in a change to the type of procedure planned as much as 50 percent of the time.1

 

AR Versus 3-D Printing for Patient Education

Wake and colleagues undertook a second study, published this year, looking at how 3-D printing and AR might influence patient understanding of an upcoming procedure. The research team worked with 200 patients in two disease/procedure groups:

•    One group had magnetic resonance (MR)-visible prostate cancer and was undergoing either robotic-assisted radical prostatectomy or focal ablative therapy; and

•    One group had renal masses and were undergoing partial nephrectomy.

 

All patients went through routine imaging protocols; prostate cancer patients received a multiparametric MRI exam, while kidney patients received either a 1.5T MR scan or a CT scan. All patients were then randomized to pre-planning with imaging alone, or imaging plus a patient-specific 3-D model. Models for the second group were either 3-D printed, visualized in AR, or viewed in 3-D on a 2-D monitor. To measure patient response, Wake and colleagues used a five-point Likert scale survey — on a scale of 1 to 5, patients rated their understanding of:

•    Their cancer/disease itself;

•    The size of the tumor/cancer;

•    The location of the tumor/cancer;

•    Why the surgeon chose their given treatment plan; and

•    Their comfort with the surgical plan.

 

Patients randomized to 3-D models took the survey twice — once after viewing conventional imaging only, and a second time after viewing the 3-D model. Regardless of disease type and across all variables measured, patient comprehension was highest with 3-D printed models compared to any other visualization method. In the kidney patient cohort, the 3-D printed models actually helped patient decision-making in regards to the procedure: One patient ended up going through with the nephrectomy rather than forego the procedure, and another was able to decide between radical and partial nephrectomy.2   

“Just seeing the anatomy in 3-D, whether a virtual model or a physical tactile model, is much more powerful than simply viewing grayscale medical images on a 2-D screen, especially for patients who might not have seen a medical image before so they don’t really know what they’re looking at,” Wake said.   

While all three-dimensional models were deemed helpful by patients, none found augmented reality models to be as helpful as a physical 3-D printed model. Wake noted that the AR setup can be difficult to work with, due to a heavy headset and a limited field of view with the first-generation HoloLens. “Depending on the size of your model, you might have to move your head to see different parts, or step further back so you can see more in the window,” she stated.

Despite the utility of both technologies, cost remains a prohibitive factor blocking widespread adoption. Wake said a planned future prospective study will provide a deeper cost analysis in hopes of expanding the utility of 3-D printing and AR.

 

References

1.  Wake N., Rude T., Kang S.K., et al. 3D Printed Renal Cancer Models Derived from MRI Data: Application in Pre-Surgical Planning. Abdominal Radiology, May 1, 2018. doi:10.1007/s00261-016-1022-2.

2.  Wake N., Rosenkrantz A.B., Huang R., et al. Patient-specific 3D printed and augmented reality kidney and prostate cancer models: impact on patient education. 3D Printing in Medicine, Feb. 2019

Related Content

Ceiling-mounted X-ray system includes MyExam Companion intelligent user interface to guide technologist through exam workflow
News | X-Ray | October 26, 2020
October 26, 2020 — Siemens Healthineers has announced the Food and Drug Administration (FDA) clearance of the...
This illustration show the complexity of the data obtained from one single patient with moderate/severe traumatic brain injury. Different imaging approaches and techniques have their own unique sensitivity in assessing different aspects of neuroanatomy and neuropathology. What can be seen on images also changes with time since injury. Data from comprehensive clinical and functional assessments using a range of other tools is also important for evaluating patient outcome. Through data harmonization and large

This illustration show the complexity of the data obtained from one single patient with moderate/severe traumatic brain injury. Different imaging approaches and techniques have their own unique sensitivity in assessing different aspects of neuroanatomy and neuropathology. What can be seen on images also changes with time since injury. Data from comprehensive clinical and functional assessments using a range of other tools is also important for evaluating patient outcome. Through data harmonization and large-scale analyses of data combined across multiple research sites, the ENIGMA Brain Injury will develop and test methods and procedures for making sense of the complexity in this data. Images courtesy of Olsen et al., Brain Imaging and Behavior, 2020

News | Magnetic Resonance Imaging (MRI) | October 23, 2020
October 23, 2020 — Trau...
A, Initial conventional axial CT image shows no noticeable lung damage (within red box) in right upper lobe. B, Electron density spectral CT image obtained at same time as image in A shows lesions (within red box) in right upper lobe. C, Follow-up conventional axial chest CT image obtained 5 days after images in A and B confirm presence of lesions (within red box) in right upper lobe.

A, Initial conventional axial CT image shows no noticeable lung damage (within red box) in right upper lobe. B, Electron density spectral CT image obtained at same time as image in A shows lesions (within red box) in right upper lobe. C, Follow-up conventional axial chest CT image obtained 5 days after images in A and B confirm presence of lesions (within red box) in right upper lobe. Image courtesy of the American Roentgen Ray Society (ARRS), American Journal of Roentgenology (AJR)

News | Coronavirus (COVID-19) | October 22, 2020
October 22, 2020 — According to an open-...
The fMRI hyperscanning environment.

(A) The fMRI hyperscanning environment. The clinician (1) and patient (2) were positioned in two different 3T MRI scanners. An audio-video link enabled online communication between the two scanners (3), and video images were used to extract frame-by-frame facial expression metrics. During simultaneous acquisition of blood oxygen level–dependent (BOLD)–fMRI data, the clinician used a button box (4) to apply electroacupuncture (EA) treatment (real/sham, double-blind) to the patient (5) to alleviate evoked pressure pain to the leg (6; Hokanson cuff inflation). Pain and affect related to the treatment were rated after each trial. (B) Study overview. After an initial behavioral visit, each individual participated in a Clinical-Interaction (hyperscan preceded by a clinical intake) and No-Interaction condition (hyperscan without a preceding intake), in a counterbalanced order, with two different partners. (C) Experimental protocol. Each hyperscan was composed of 12 repeated trials (four verum EA, four sham EA, and four no treatment) in a pseudo-randomized order. After a resting period (far left), both participants were shown a visual cue to indicate whether the next pain stimulus would be treated (green frame) or not treated (red frame) by the clinician. These cues prompted clinicians prepare to either apply or not apply treatment while evoking corresponding anticipation for the patient. Following the anticipation cue, moderately painful pressure pain was applied to the patient’s left leg, while the clinician applied or did not apply treatment, respectively. After another resting period, participants rated pain (patients), vicarious pain (clinicians), and affect (both) using a visual analog scale (VAS).

News | Clinical Trials | October 22, 2020
October 22, 2020 — The potential impact of the patient-clinician relationship on a patient's response to treatment is
Lesion was originally reported as indeterminate enhancing mass, and outside report recommended biopsy. Classic features of benign hemangioma are shown. Error was attributed to faulty reasoning. A, Axial MR image obtained 5 minutes after contrast agent administration shows peripheral nodular discontinuous enhancement. B, Axial MR image obtained 10 minutes after contrast agent administration shows centripetal progression of enhancement (arrow). C, Axial fast imaging employing steady-state acquisition (FIESTA)

Lesion was originally reported as indeterminate enhancing mass, and outside report recommended biopsy. Classic features of benign hemangioma are shown. Error was attributed to faulty reasoning. A, Axial MR image obtained 5 minutes after contrast agent administration shows peripheral nodular discontinuous enhancement. B, Axial MR image obtained 10 minutes after contrast agent administration shows centripetal progression of enhancement (arrow). C, Axial fast imaging employing steady-state acquisition (FIESTA) MR image shows lesion is homogeneously hyperintense compared with liver parenchyma. Image courtesy of American Roentgen Ray Society (ARRS), American Journal of Roentgenology (AJR)

News | Magnetic Resonance Imaging (MRI) | October 21, 2020
October 21, 2020 — According to an artic...
According to an inquest, a man with a heart disorder and chest pain died two days after a doctor viewed the wrong scan and sent him home
News | Computed Tomography (CT) | October 21, 2020
October 21, 2020 — The BBC News
Flowchart of patient inclusion and exclusion.

Figure 1. Flowchart of patient inclusion and exclusion.

News | Coronavirus (COVID-19) | October 20, 2020
October 20, 2020 — A new multi-institutional study published in the journal ...
Rensselaer, First-Imaging, and GE Global researchers develop a deep neural network to perform nearly as well as more complex dual-energy CT imaging technology
News | Computed Tomography (CT) | October 20, 2020
October 20, 2020 — Bioimaging technologies are the eyes that allow doctors to see inside the body in order to diagnos
Lesion was originally reported as indeterminate enhancing mass, and outside report recommended biopsy. Classic features of benign hemangioma are shown. Error was attributed to faulty reasoning. A, Axial MR image obtained 5 minutes after contrast agent administration shows peripheral nodular discontinuous enhancement. B, Axial MR image obtained 10 minutes after contrast agent administration shows centripetal progression of enhancement (arrow). C, Axial fast imaging employing steady-state acquisition (FIESTA)

56-Year-Old Woman With Benign Hemangioma: Lesion was originally reported as indeterminate enhancing mass, and outside report recommended biopsy. Classic features of benign hemangioma are shown. Error was attributed to faulty reasoning. A, Axial MR image obtained 5 minutes after contrast agent administration shows peripheral nodular discontinuous enhancement. B, Axial MR image obtained 10 minutes after contrast agent administration shows centripetal progression of enhancement (arrow). C, Axial fast imaging employing steady-state acquisition (FIESTA) MR image shows lesion is homogeneously hyperintense compared with liver parenchyma.

News | Magnetic Resonance Imaging (MRI) | October 16, 2020
October 16, 2020 — 
Carestream Health will offer insights on technology, innovative new products and research in medical imaging through a new webinar series titled Carestream Talks. #RSNA20
News | Radiology Education | October 14, 2020
October 14, 2020 — Carestrea