News | Advanced Visualization | March 06, 2017

European Team Developing Augmented Reality Visor to Dramatically Improve Surgery

Project team claims finished product will reduce surgery times by over 20 minutes for every three hours; mass production hoped to begin by 2022

VOSTARS, augmented reality surgical visor, Euopean project

Image courtesy of Photonics21

March 6, 2017 — Employing new photonics technology, European scientists are developing a new augmented reality surgical visor in a bid to improve accuracy of interventions. The visor shows anesthetic and medical data while superimposing a patient’s X-ray in perfect unison with their body, meaning surgeons never have to look away during an operation and surgery times are reduced by over 20 minutes for every three hours.

The VOSTARS (Video Optical See-Through Augmented Reality surgical System) medical visor is a head-mounted display (HMD) system that is capable of superimposing the patient’s X-ray images in perfect 3-D unison with their anatomy.

The visor also presents a patient’s anesthetic data, heart rate, body temperature, blood pressure and breathing rates conveniently into the surgeon’s field of vision, in a drive to increase accuracy by focusing on the operation and reduce time by never having to look away.

The project forecasts a significant improvement of the intervention accuracy coupled with a reduction in times spent in an operation and under anesthetic by at least 11 percent. 

Although augmented reality for surgical procedures has been talked about in academic and industrial research since the 1990s, a tool joining a surgeon’s natural perception with patient data has not yet been widely implemented.

However, it has been the use of photonics components — with the small, high-luminous micro display, the LED optical waveguide and the array of microns to project a 2-D X-ray image in front of the user — that has been fundamental to realizing this one-time science fiction, according to the research team.

Project coordinator Vincenzo Ferrari, biomedical engineering researcher at the Department of Information Engineering, at the University of Pisa, explained, “With this state-of-the-art, highly ergonomic visor, we intend to provide all the information required to improve surgery. The primary goal is to reduce not just surgery times, but also the time spent under anesthetic and the cost involved in any operation.”

In the same way that a facing camera on a smartphone films moving images, the VOSTARS system works by capturing what the surgeon sees from a head-mounted camera. The system then merges this footage of reality with the patient’s medical images, from computed tomography (CT), magnetic resonance imaging (MRI) or 3-D ultrasound scans.

The central processor, using the most advanced registration techniques available for surgical navigation, then presents a real-time hybrid image on the visor ‘dashboard’ to the surgeon.

“Imagine driving with a sat-nav: we know how to drive and roughly where to go, but with real-time information, like speed, distance and time presented to us, we can take the most efficient route in the quickest time and perhaps more safely,” said Ferrari. “However, rather than having that information on a small GPS screen, our important patient data, like the surgical target in the anatomy, anesthetic info, breathing and heart rates for example, are all integrated instantly into the surgical visor.

“A clinician can move freely while still seeing the patient, the hybrid X-ray image and all of critical data all at once in a surgical ‘dashboard’ inside the screen. The surgeon, the patient and the procedure are all at one,” he added

Scientists at VOSTARS are building a hybrid of the two existing AR headsets approaches that combines all the benefits of both a video see-through (VST) system and optical see-through (OST) head-mounted display.

Fabrizio Cutolo, engineer and expert in wearable augmented reality systems, believes neither VST nor OST alone was suitable for operating on a live patient. “When operating on a real person, it was clear to us that the benefits of OST and VST systems could be combined to make a hybrid device,” he said.

In ‘optical see-through’ (OST) systems, as in Microsoft Hololens, the user has a direct view of the natural environment, with the computer-generated images superimposed on a user’s field of vision using a semi-transparent mirror. This gives the user a more naturalistic experience, superimposing small amounts of the virtual onto the real world.

With ‘video see-through’ (VST) systems, as seen in the Occulus Rift headsets, the user is submerged in the virtual world, seeing through a closed HMD together with stereo cameras, experiencing life through screens. Since VST systems capture the video-image as seen by the user in real time, it is good for aligning the real and virtual worlds.

“For something as critical as an operation, we had to have the naturalistic ‘feel’ of OST, while having the fluid interaction of the VST. Therefore a brand new device had to be made from scratch, rather than extend an existing technology. The VST-OST Augmented Reality hybrid was born,” said Cutolo.

Already three months into the three-year project, VOSTARS aims to have a working prototype of the hybrid device ready for May of 2018. The project is initially being trialed on a number of procedures to the head, including maxillofacial (jaws and face) surgery, neurosurgery, ENT (ear, nose and throat) and orthopedic surgery. The project team hopes the technology will be available to end users in three years, with mass production by 2022.

For more information: www.vostars.eu

Related Content

Of all the buzzwords one would have guessed would dominate 2020, few expected it to be “virtual”. We have been virtualizing various aspects of our lives for many years, but the circumstances of this one has moved almost all of our lives into the virtual realm.

Getty Images

Feature | Radiology Education | September 18, 2020 | By Jef Williams
Of all the buzzwords one would have guessed would dominate 2020, few expected it to be “virtual”.
Videos | Cardiac Imaging | August 12, 2020
Advanced visualization company...
Siemens Partnership will make better health easier throughout Pennsylvania and in all communities that Geisinger serves

Getty Images

News | Radiology Business | June 08, 2020
June 8, 2020 — Siemens Healthineers and Geisinger have estab
Figure 4 for the study. Images of a 65-year-old man (patient 6). (a) Cardiac MRI perfusion shows perfusion deficit of anterior/anterolateral wall attributed to left anterior descending artery/left circumflex artery (*). (b) CT coronary angiography. (c) Coronary angiography, left anterior oblique projection with caudal angulation. (d) Three-dimensional image fusion helped refine diagnosis: perfusion deficits (*) were most likely caused by narrow first diagonal branch and its first, stented side branch.

Figure 4 for the study. Images of a 65-year-old man (patient 6). (a) Cardiac MRI perfusion shows perfusion deficit of anterior/anterolateral wall attributed to left anterior descending artery/left circumflex artery (*). (b) CT coronary angiography. (c) Coronary angiography, left anterior oblique projection with caudal angulation. (d) Three-dimensional image fusion helped refine diagnosis: perfusion deficits (*) were most likely caused by narrow first diagonal branch and its first, stented side branch (arrowhead). Retrospectively, denoted lesion could also be found at CT coronary angiography and coronary angiography (arrowheads in b and c, respectively). CT FFR = CT-derived fractional flow reserve, LGE = late gadolinium enhancement. Image courtesy of RSNA, Radiology.

News | Cardiac Imaging | May 04, 2020
May 4, 2020 – A new technique that combines computed tomography (CT) and magnetic resonance imaging MRI can bolster c
Two examples of CT myocardial perfusion (CTP) imaging assessment software. Canon is on the left and GE Healthcare is on the right. Both of these technologies have been around for a few years, but there have been an increasing amount of clinical data from studies showing the accuracy of the technology compared to nuclear imaging, the current stand of care for myocardial perfusion imaging, and cardiac MRI. #SCCT #perfusionimaging 

Two examples of CT myocardial perfusion (CTP) imaging assessment software. Canon is on the left and GE Healthcare is on the right. Both of these technologies have been around for a few years, but there have been an increasing amount of clinical data from studies showing the accuracy of the technology compared to nuclear imaging, the current stand of care for myocardial perfusion imaging, and cardiac MRI.

News | Computed Tomography (CT) | March 16, 2020
March 16, 2020 — The Society of Cardiovascular Computed Tomography (SCCT) released a new...
Virtual reality during chemotherapy has been shown to improve breast cancer patients’ quality of life during the most stressful treatments
News | Virtual and Augmented Reality | January 21, 2020
January 21, 2020 — Virtual reality during chemotherapy has been shown to improve...
Carestream’s X-ray digital tomosynthesis functionality creates three-dimensional datasets from digital radiography (DR) that can be scrolled through similar to computed tomography (CT) imaging. It received 510(k) clearance from the U.S. Food and Drug Administration (FDA) in January 2020. Digital tomosynthesis uses a single sweep of X-ray exposures and streamlines operator workflow by separating the process of DT exposure acquisition from image volume formation.
News | Digital Radiography (DR) | January 15, 2020
January 15, 2020 — Carestream’s X-ray digital tomosynthesis (DT) functionality, which creates three-dimensional datas