In This Section      

Promise and Progress - From the Family Room to the Operating Room

Engineering Cures: Physicians and Engineers Working Together to Fight Cancer
Issue No. 2012

From the Family Room to the Operating Room

By: Valerie Matthews Mehl
Date: December 20, 2011

Surgeon Finds New Uses for Video Game Systems

1 2
Kinect, the optical hardware from Microsoft, Inc.

Liver cancer surgeon Michael Choti is moving gaming systems from the family room to the operating room.  In collaboration with experts from engineering, Choti and team are applying the image-guided technology used in interactive gaming systems like the Nintendo Wii® and  Microsoft Kinect® systems to medicine.

These newer iterations of video games use cameras, motion sensors, tracking systems and accelerometers to allow players to interact directly with the game.  Choti, Emad Boctor, a radiology and computer science engineer, and Philipp Stolka, a post-doctoral engineering fellow,  are working to apply the same technology to create image guidance and tracking devices for use in cancer surgery and treatments.  “Current robotic and image-guided tracking systems tend to be cumbersome and extremely expensive,” says Choti.  “Our approach makes use of sophisticated and mobile, but relatively cheap technology, which should broaden its appeal.”
A Wii remote contains accelerometers and infra-red cameras that calculate relative position and orientation and beam the information continuously to the game controller via a “Bluetooth” radio link.  A computer receives and processes the Bluetooth signals.  In one adaptation, Choti and team used the Wii remote to transform standard ultrasound imaging into a more readable 3-D imaging.  While 3-D ultrasound probes exist, Choti says they are unwieldy and expensive. However, with the position and orientation information from the Wii remote and local optical sensors, a computer could straightforwardly turn the 2-D ultrasound image into a more readable and useful 3-D ultrasound image.  “Our setup, in principle would, allow clinicians to convert a 2-D ultrasound probe into a 3-D probe that’s portable and easy to use in a variety of applications,” says Choti.

In another application, Choti and his engineering team showed that Kinect, the optical hardware from Microsoft, Inc., and image-recognition algorithms used to identify and track the location and orientation of objects in its visual field could be adapted to locate and track the motions of a biopsy needle.

One of the next steps in the development of these systems will be to combine the tracking of an imaging device such as an ultrasound probe with the tracking and 3-D guidance of a surgical instrument such as a needle, says Choti.  “That combination would give us a relatively inexpensive and portable system for image-guided surgical procedures,” he says.  Microsoft executives have expressed interest in working with Choti and team to develop the technology specifically for medical applications.

“This shows the value of looking at other fields, whether it is engineering, geology, or computerized gaming,” says Choti.  “Collaborations—and Hopkins is good at collaboration—can lead to unexpected uses of technology that are of value to patients,”

Articles in this Issue

Director's Letter

Headline Makers

Cover Story: Engineering Cures

In the News

Honors & Awards


Web Exclusives


Find Physicians Specializing In...