Gesture Technology: From Xbox to OR

“Keys, scumbag. It’s the universal symbol for keys.”

Mike Ehrmantraut, his hand outstretched to Walter White (Episode 1, Season 5, Breaking Bad)

Some gestures are easy to figure out. Others less so. Under Christmas trees around the world, the latest version of Xbox is waiting to take a crack at some of what may be the first signs of a new way to manipulate medical images.

“Gesture technology” will allow owners of Xbox One to zoom in and out by closing a hand over an area, then pulling back and pushing away. It will allow them to scroll through a menu by moving a hand left or right, up or down.

This technology will soon be applied en masse around the globe in Call of Duty, BioShock and Halo. Christian Schaller, founder and CEO of the German tech start-up Metrilus, would like them used to view radiological images in OR and interventional suites.

Schaller was gesturing the possibilities in the German Pavilion on the exhibit floor of RSNA 2013. Using time-of-flight 3-D cameras similar to technology built into the Xbox Kinect motion sensor, he was able to call up an image by pointing to it with a single finger, switch displayed images with the swipe of a hand, and zoom an image by motioning up and down.

The interface was in prototype and a long way from application. But the allure was clear.

Gesture navigation offers surgeons the promise of remote control over display technology without physical contact. The advantage is obvious to anyone who has tried to maneuver the tight spaces of an operating theater, much of which is draped in blue, signifying scrubbed and sterilized surfaces that better not be touched, if the one doing the touching knows what’s good for him.

Like voice recognition, which over the last decade has become a staple in radiological circles, gesture recognition appears as inevitable as its beginning is sure to be rocky.  My nephew — an accomplished gamer — tells me an upright foot, crossed on an adjacent knee while sitting in front of an Xbox can be confused with a hand, leading to results that are anything but helpful. Unintended hand movements in the OR could have similarly unexpected effects. But, hey, gesture technology is early in its development.

The world may be on the doorstep of, as yet, unimagined interfaces, ones that will dramatically cut the time from thought to action. These may be adopted by the gaming world long before they are by the radiological community, just as medical image display on mobile devices were preceded by YouTube videos.

Not surprisingly, then, the first place to look this Christmas for the future of radiology may be under old tannenbaum.