While Sony managed to earn more than a few impressed exclamations from press, fans, and members of the gaming community with its new virtual reality headset code named “Project Morpheus,” one of its more incredible pieces of hardware sat unnoticed in a back room and had nothing to do with virtual reality.
The tech in question is a piece of unnamed prototype hardware that may or may not ever make it to retail, but it should – it literally offers a new way to look at games. Using a specially designed camera, one of Sony’s research and development teams (the group known as Magic Labs) has come up with a way for you to control the camera in a game with your eyes. It isn’t something that changes the nature of gaming in the same way that virtual reality does, but it is one of the coolest pieces of gaming tech on display at GDC.
The eye tracking technology is still a long way from being released – and there is no guarantee it will ever be – but the potential is incredible. Eye tracking is becoming a common feature of smartphones, but the scale of Sony’s device eclipses anything a smartphone currently offers. In the demo on display, Sony’s eye tracking tools (which, again, don’t technically have a name at the moment) were used in conjunction with the PlayStation 4 game Infamous: Second Son.
To help the energy blasts reach their targets, you simply look at what you want to hit.
It all starts with a quick calibration that’s as simple as looking at one dot in the upper left corner then another on the diagonally opposite side. The camera then begins to send data to a nearby laptop, which in turn takes over for the game’s camera.
It takes a few moments to orient to the new method of seeing. In a third-person game, it’s natural for your eyes to gravitate to your character that is below the center of your screen. With the eye tracking software, looking at your character therefore causes the camera to pan down. Once you compensate, though, the effects are amazing.
Beyond the ability to look around with your eyes in the same way you would normally pan the camera around with the right thumbstick, the tracking can also be far more precise. In Infamous, quickly tapping the R2 button sends a stream of energy blasts at opponents similar to firing a machine gun, but with slower projectiles. To help the energy blasts reach their targets, you simply look at what you want to hit. If you are facing two enemies, one on either side of you, just glance one direction then the other and the energy blasts will find their mark. It registers your movements as fast as your eyes move, creating a responsive camera that could alter the way we visually input games forever.
While the demo focused on what, for all intents and purposes, is a third-person shooter (at least in the demo, which showed an early section of the game where energy blasts were among your most effective weapons), the possibilities with Sony’s eye-tracking tech are endless. Imagine playing a soccer game and, instead of trying to awkwardly pass to one particular player running in an unusual pattern, you just look at them and hit the pass button. In football, while controlling the quarterback, you could simply look at your receivers and hit pass to the one you want. In first-person shooters, you could theoretically aim faster than your thumb could move.
There are also possible uses for the tech with the Project Morpheus virtual reality headset. It’s still just theoretical right now (as far as we know), but we imagine that you may be able to one day put on the VR headset and look around the virtual world by moving your head, but also look around as you naturally would, with your eyes.
The effects could go beyond gaming as well. Imagine watching a 3D telecast of a basketball game using a virtual reality headset. You could look around a limited field of view, but when the action happened, your eyes would command the video where to focus. If a shot is blocked and a fast break begins, it’s more natural to follow with your eyes than whip your head around to see the ball race down court.
There are some drawbacks, however. For one, it requires a new camera separate from the PlayStation 4’s camera. That said, the device on display was a pure prototype, and there could always be a chance they find a way to incorporate existing cameras to use the tech via software updates. But if it were released tomorrow, you’d need to buy a new peripheral.
Another, more significant, drawback is the software’s issues with tracking eyes behind glasses. It’s hit or miss on whether or not it recognizes the movement of the eyes behind lenses, and people who wear glasses will run into issues. That could eventually be fixed. But even if you are free of spectacles, you’ll need to sit close to the camera. In order to track the movement of eyes, a camera needs to be close enough to follow the movements, potentially creating an awkward issue with positioning. The alternative would be a high-resolution camera capable of zooming in enough to make out your eyes from a distance. That’s certainly possible, but it would likely cost more than it would be worth. Still, these are all solvable issues.
Sony’s eye-tracking tech was first shown back in November 2013 as the world prepared for the arrival of the PlayStation 4. It was highlighted as part of the look at Magic Labs, the company also responsible for devices like the Move, the Eye Toy Camera, and most recently the DualShock 4. At the time, the eye tracking was little more than a gimmick – a neat gimmick – but compared to the release of the new system, a theoretical prototype was not a major focus. Now that the PlayStation 4 is on the market, however, the boundaries of gaming are ready to be pushed by devices like this one, Project Morpheus, and others we haven’t considered yet (but are likely being worked on in labs around the world).
With the Project Morpheus casting a long shadow, eye tracking is still just a sideshow, especially since there are no firm plans for it at the moment. Give it time, though. This may just be part of the future of gaming.
