Feature | Advanced Visualization | July 02, 2019 | By Jeff Zagoudis

Augmented Reality Versus 3-D Printing for Radiology

New research explores potential of advanced visualization technologies to improve procedural outcomes and increase patient satisfaction

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model.

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model. Photo courtesy of Nicole Wake, Ph.D.

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model.

A 3-D printed model (left) and a model constructed in augmented reality (right), both of a kidney with a tumor. In both models, the kidney is clear; the tumor is visible in purple on the AR model and in white on the 3-D printed model. Photo courtesy of Nicole Wake, Ph.D.

Three-dimensional (3-D) printing and augmented reality (AR) are two of the more exciting technologies being explored in radiology today. In their own way, each technology gives form and depth to medical imaging data, offering a new perspective for physicians and patients. Early explorations have promised new ways to plan surgical and interventional procedures, and new methods to help patients understand what care is being delivered and how. For now, however, both technologies are still largely viewed as novel, with at best anecdotal evidence of how they can be translated into clinical use. A pair of recent large-scale studies may offer a sense of how to collect quantitative evidence that these technologies can benefit all members of the healthcare team.

Augmented reality technology — which superimposes virtual images onto the user’s real-world surroundings, creating a “mixed reality” — took off in 2016 when Microsoft released the first consumer-grade mixed reality headset, the HoloLens. The U.S. Food and Drug Administration (FDA) approved the first healthcare-focused AR technology, Novarad’s OpenSight AR system, in November 2018. OpenSight uses a HoloLens headset to register a virtual overlay of a patient’s magnetic resonance (MR) or computed tomography (CT) scans onto the patient.

 

The Path to AR and 3-D

Nicole Wake, Ph.D., an assistant professor in the Department of Radiology at Albert Einstein College of Medicine in New York, began exploring advanced visualization while working on her Ph.D. thesis at New York University School of Medicine. “In our lab we had an industrial 3-D printer that was bought to print custom-made housings for radiofrequency (RF) coils. I started using this printer to make patient-specific anatomical models,” she said. “Then the pathway to create AR and virtual reality (VR) models from radiological imaging data is similar to printing, so it made sense to explore those technologies as well.”

When the HoloLens was first released, Wake was already working in 3-D printing but recognized the limitations of the technology — primarily the amount of time required to create a 3-D printed model. Building the digital model that serves as a blueprint for the printer might only take an hour or two, according to Wake, but the actual printing process can take several hours. “The printing times vary based on the technology used and the volumes. Kidneys can take approximately 10 hours for printing, maybe a little more or less depending on how big each one is,” she said. Creating a three-dimensional model of an entire pelvis could take as long as 30 hours.

Seeing the potential of AR to address these limitations, Wake applied for a HoloLens developer license and taught herself how to program with the technology. The process is similar to building a digital model prior to the 3-D printing process. After the initial CT or MR image is acquired, the user performs image segmentation. “Once you have the images, you have to divide out the regions of interest you want to show in the model,” she said. This step can be time-consuming depending on the type of image acquired, the segmentation software and the experience of the user. From there, the model is brought into the AR or 3-D printing software for a full workup.

Wake noted that while prior programming experience was helpful in working with the HoloLens, much of the interface was fairly intuitive.

 

3-D Printing for Procedure Planning

One of the first applications for 3-D modeling that Wake investigated was for procedural planning. At the time, she was working on her Ph.D. thesis with Hersh Chandarana, M.D., an abdominal radiologist with NYU Langone Health, so her initial research has focused on 3-D modeling to aid kidney and prostate cancer patients.

In a study published in 2017, Wake explored using 3-D printing and augmented reality to help surgeons with procedural planning. Retrospectively evaluating the cases of patients who received clinically indicated and research MRI prior to surgery for renal masses, the research team created patient-specific 3-D printed models. Three experienced urologists reviewed all patient cases, first with 2-D imaging alone and then with conventional imaging plus the 3-D models. Data indicated that the 3-D models resulted in a change to the type of procedure planned as much as 50 percent of the time.1

 

AR Versus 3-D Printing for Patient Education

Wake and colleagues undertook a second study, published this year, looking at how 3-D printing and AR might influence patient understanding of an upcoming procedure. The research team worked with 200 patients in two disease/procedure groups:

•    One group had magnetic resonance (MR)-visible prostate cancer and was undergoing either robotic-assisted radical prostatectomy or focal ablative therapy; and

•    One group had renal masses and were undergoing partial nephrectomy.

 

All patients went through routine imaging protocols; prostate cancer patients received a multiparametric MRI exam, while kidney patients received either a 1.5T MR scan or a CT scan. All patients were then randomized to pre-planning with imaging alone, or imaging plus a patient-specific 3-D model. Models for the second group were either 3-D printed, visualized in AR, or viewed in 3-D on a 2-D monitor. To measure patient response, Wake and colleagues used a five-point Likert scale survey — on a scale of 1 to 5, patients rated their understanding of:

•    Their cancer/disease itself;

•    The size of the tumor/cancer;

•    The location of the tumor/cancer;

•    Why the surgeon chose their given treatment plan; and

•    Their comfort with the surgical plan.

 

Patients randomized to 3-D models took the survey twice — once after viewing conventional imaging only, and a second time after viewing the 3-D model. Regardless of disease type and across all variables measured, patient comprehension was highest with 3-D printed models compared to any other visualization method. In the kidney patient cohort, the 3-D printed models actually helped patient decision-making in regards to the procedure: One patient ended up going through with the nephrectomy rather than forego the procedure, and another was able to decide between radical and partial nephrectomy.2   

“Just seeing the anatomy in 3-D, whether a virtual model or a physical tactile model, is much more powerful than simply viewing grayscale medical images on a 2-D screen, especially for patients who might not have seen a medical image before so they don’t really know what they’re looking at,” Wake said.   

While all three-dimensional models were deemed helpful by patients, none found augmented reality models to be as helpful as a physical 3-D printed model. Wake noted that the AR setup can be difficult to work with, due to a heavy headset and a limited field of view with the first-generation HoloLens. “Depending on the size of your model, you might have to move your head to see different parts, or step further back so you can see more in the window,” she stated.

Despite the utility of both technologies, cost remains a prohibitive factor blocking widespread adoption. Wake said a planned future prospective study will provide a deeper cost analysis in hopes of expanding the utility of 3-D printing and AR.

 

References

1.  Wake N., Rude T., Kang S.K., et al. 3D Printed Renal Cancer Models Derived from MRI Data: Application in Pre-Surgical Planning. Abdominal Radiology, May 1, 2018. doi:10.1007/s00261-016-1022-2.

2.  Wake N., Rosenkrantz A.B., Huang R., et al. Patient-specific 3D printed and augmented reality kidney and prostate cancer models: impact on patient education. 3D Printing in Medicine, Feb. 2019

Related Content

According to the new market research report "Radiation Dose Management Market by Products & Services (Standalone Solutions, Integrated Solutions, Services), Modality (Computed Tomography, Nuclear Medicine), Application (Oncology, Cardiology, Orthopedic), End User (Hospitals) - Global Forecast to 2025", published by MarketsandMarkets, the radiation dose management market is projected to reach USD 422.65 million by 2025 from USD 220.22 million in 2020, at a CAGR of 13.9%.

Courtesy of MarketsandMarkets Research

News | Radiation Dose Management | August 03, 2020
August 3, 2020 — According to the new market research report "...
Franco Fontana, CEO of the Esaote Group, and Xie Yufeng, Chairman of WDM.

Franco Fontana, CEO of the Esaote Group, and Xie Yufeng, Chairman of WDM.

News | Digital Radiography (DR) | July 31, 2020
July 31, 2020 — In the thick of the COVID-19 eme
Older Americans may be receiving cancer screenings not recommended by the U.S. Preventive Services Task Force, according to Penn State College of Medicine researchers.

Getty Images

News | Radiology Imaging | July 31, 2020
July 31, 2020 — Older Americans may be receiving cancer screenings not recommended by the...
A World Health Organization (WHO) rapid advice guide on the use of chest imaging in the diagnosis and management of COVID-19 was published in the journal Radiology.

Getty Images

News | Coronavirus (COVID-19) | July 30, 2020
July 30, 2020 — A World Health Organization (WHO) rapid advice gui
It covers every major modality, including breast imaging/mammography, fixed and portable C-arms (cath, IR/angio, hybrid, OR), CT, MRI, nuclear medicine, radiographic fluoroscopy, ultrasound and X-ray
News | Radiology Imaging | July 29, 2020
July 29, 2020 — IMV Medical Information, part of Scien...
Left, a 3-D rendering of a heart from a cardiac CT exam. Right, a lung-CT exam showing the heart and ground glass lesions in the lungs of a COVID-19 patient. CT has become a front-line imaging modality in the COVID era because it offers both cardiac and lung information to help determine a patients disposition with chest pain, COVID-19 and COVID-caused myocarditis and pulmonary embolism. #COVID19 #CCTfirst #YesCCT

Left, a 3-D rendering of a heart from a cardiac CT exam. Right, a lung-CT exam showing the heart and ground glass lesions in the lungs of a COVID-19 patient. CT has become a front-line imaging modality in the COVID era because it offers both cardiac and lung information to help determine a patients disposition with chest pain, COVID-19 and COVID-caused myocarditis and pulmonary embolism.

Feature | Computed Tomography (CT) | July 28, 2020
July 28, 2020 — The use of cardiova...
 The Radiological Society of North America (RSNA) has been awarded a $750,000 grant by the Department of Energy’s National Nuclear Security Administration (NNSA) to develop a Global Learning Center (GLC) in Sub-Saharan Africa. The grant will help address the growing need in Sub-Saharan Africa for training in radiology.
News | Radiology Education | July 28, 2020
July 28, 2020 — The Radiological Society of North America (RSNA)
ams Commended by United Imaging for Accelerated Supply of Unprecedented Levels of Essential CT Detectors to Fight Covid-19 (SARS-CoV-2)
News | Coronavirus (COVID-19) | July 28, 2020
July 28, 2020 — ams, a leading worldwide supplier of high performance sensor solutions, has been thanked by long-term
A cardiac CT of a patient with pacemaker leads, which can be challenging to get good images due to metal artifact. This image was rendered from using Canon's AiCE AI-assisted interactive reconstruction with Global Illumination 3-D rendering from a scan on an Aquilion One Genesis SP system.

A cardiac CT of a patient with pacemaker leads, which can be challenging to get good images due to metal artifact. This image was rendered from using Canon's AiCE AI-assisted interactive reconstruction with Global Illumination 3-D rendering from a scan on an Aquilion One Genesis SP system.

Feature | Computed Tomography (CT) | July 27, 2020
There has been tremendous growth in the field of cardiovascular...