Just a Twist (and Tilt) of the Wrist
WHEN Scott Forstall, a senior vice president at Apple, demonstrated a new space game for the iPhone, he let the spacecraft cruise through a field of stars for a bit. As the audience finished absorbing the look of the shooter set in a distant galaxy, he asked them, “I don’t have a joystick on here, or any four-button toggle, so how do I steer?”
A few seconds later, he tilted the phone just a few degrees and the ship shifted course on the screen. “We have a full three-axis accelerometer in here, so all I’ve got to do is move the phone around and now I’m steering it,” he said. The crowd of programmers cheered.
The iPhone is not unique. Nintendo’s popular Wii game console uses similar technology, and many cellphones, computers and other electronic gadgets are gaining a sensitivity to motion.
Manufacturers are increasingly embedding accelerometers and other sensors into the machines, which allow them to respond to movement without waiting for their humans to push a button. Game designers and other programmers are jumping to remake user interfaces so that users can direct gadgets with a nudge, a tilt or a shake.
Some of the applications are silly. The programmer Graham Oldfield turned his Nokia N95 cellphone into a virtual light saber by writing software that tracked the phone’s movement using the built-in accelerometer.
When the phone is still, it emits a low hum, but when the user waves it, the pitch and volume increase just like the weapons in the “Star Wars” movies. If the phone is abruptly stopped, it assumes it encountered something and provides a proper cracking sound. Version 1.5 of the software, available free from Mr. Oldfield’s Web site (graho.wordpress.com), adds tactile feedback through the vibrating ringer, a feature Mr. Oldfield calls SaberTingle.
Versions of the popular video game Snake from the 1970s are now available for iPhone and Nokia phones with accelerometers. Tilting the phone guides a snake to dinner, a process that gets harder and harder as the snake grows.
Andreas Jakl and Stephan Selinger, professors at the University of Applied Sciences in Hagenberg, Austria, transformed a Nokia N95 cellphone into a steering wheel for a radio-controlled car (www.symbianresources.com/projects). Turn the phone to the left and the car turns left.
Professor Jakl also worked with a student to develop software that turned a Nokia phone into a device that records the movement of a skier or a snowboarder. “We’re from Austria and we’re a skiing nation,” he said. “You put your phone into your pocket. It will record your jumps, how often you jump, how often you crash. You can compare it with your friends to see who jumped longer.”
Not all of the applications are about having fun. FlipSilent and ShakeSMS are two freeware tools that let the accelerometer control the user interface. If the phone rings, FlipSilent will shut off the ringer when the phone is turned over. ShakeSMS will page through text messages with a shake and a tilt with no pushing of buttons. Both are available from flipsilent.com.
Accelerometers started appearing in cellphones in the early 1990s. Michael Markowitz, a spokesman for STMicroelectronics, says the company’s line of accelerometers helps to activate air bags in cars and protect hard drives in a laptop should it be suddenly dropped.
The current generation of accelerometers are tiny blocks of silicon carved out of wafers using many of the same techniques used to create transistors and circuitry. Circuits built on the same chips sense how movement pushes the blocks along. For instance, the LIS302DL from STMicroelectronics, used in some phones, is only 3 millimeters by 5 millimeters by 1 millimeter, and can measure forces up to 8 times the earth’s gravity along three axes.
Translating the data generated by the chips into results that a user can see requires some artful software. Johnny Chung Lee, a researcher at Carnegie Mellon University, recently built a pen that gives users the illusion they are pushing hard or soft 3-D buttons on a flat computer screen, enhancing their feel of virtual objects. The software follows the user’s movement of the pen and a computer calculates how much pressure should be applied.
One problem with accelerometers is that they detect acceleration — the rate of change in velocity — but not the movement itself, Dr. Lee said. If a computer wants to know the speed or the position, it must apply common equations from high school calculus. Any noise or error is compounded over time.
Dr. Lee says that error is why many designers pair accelerometers with other sensors, like magnetometers that follow the earth’s magnetic field or G.P.S. radio systems that use a network of satellites to detect geographic position. For example, the controllers for the Sony PlayStation 3 come with sensors that detect angles, tilting, thrusting and pulling. Fusing the results of several sensors gives the device a better sense of where it is going and where it has been.
Game designers are just beginning to tap into the power of accelerometers to transform cellphones into more intuitive game devices. Travis Boatman, a vice president at EA Mobile, a division of Electronic Arts, says that placing the accelerometer in the same place as the screen helps simplify the interface while eliminating the need for the user’s brain to make a connection between a push of a button and an action on a screen across the room.
In E.A.’s forthcoming game Spore, the company modified the control system in the iPhone version to eliminate buttons used in the PC version. “You control where the spore goes, but everything else is controlled for you. It eats automatically,” said Mr. Boatman.
Mr. Boatman says one challenge is that accelerometers are more sensitive than humans, who often cannot tell whether the device is tilting 5, 10 or 12 degrees. The solution lies in blurring measurements and programming the devices to react to general movement.
Moving a device around can be more intuitive than pressing buttons, said Ethan Einhorn, a developer for Sega of America who has been experimenting with bringing a game called Super Monkey Ball to the iPhone.
“I’ve been able to hand the game off to people who are not gamers and they’re instantly able to understand what they’re supposed to do. There’s a direct connection: I tilt it and the monkey moves,” Mr. Einhorn said.These opportunities are just beginning to inspire others. Microsoft’s research division recently demonstrated a small PC with a screen that will detect and respond to twisting, stretching, bending and squeezing to move a mouse, switch programs or perform whatever other functions that programmers can dream up.