A Hands-Free Controller for Video Games

Published in Insight - November 2016

Six years ago, Gyorgy Levay overcame a devastating meningitis infection that robbed him of one arm and one hand. An avid gamer, he was no longer able to play video games like World of Warcraft.

“I can play several video games, but not games that require forward, backward, left or right movements,” says Levay, a biomedical engineering master’s degree candidate. “You need fingers for that.”

That’s why Levay helped design a new hands-free control system, GEAR, that’s worn like a pair of sandals and allows users to execute more than 15 commands with their feet.

Products like feet-controlled button pads exist but can be uncomfortable to operate because they require users to keep their legs positioned in a certain way. GEAR lets users move their legs freely without impacting the controller’s performance.

GEAR sprouted out of an assignment in biomedical engineering professor Nitish Thakor’s Principles of Design of Medical Instrumentation class.

“What this does is take buttons and put them on your feet,” explains Levay, who worked with biomedical engineering grad students Adam Li and Nhat Tran.

At the bottom of each sandal are three pressure sensors that detect movement. Wires relay that information to an Intel Edison microcontroller tucked into the left heel, which then communicates via USB cable to the user’s computer. Applying weight on the front right sandal, for instance, causes players to move forward in a game.

GEAR’s developers see the controller being used in a variety of computer tasks.

In May, the team won the 2016 Intel-Cornell Cup’s $7,500 grand prize for an innovative application of embedded technology.

The team members worked with Johns Hopkins Technology Ventures to obtain a provisional patent for their invention. They’re hoping a company will license GEAR so production of the controller can begin.