Last year, the Darmstadt University of Technology in Germany and Intel Labs collaborated on a method of pulling visual information from Grand Theft Auto V, according to a report from Bloomberg. The game has now been modified such that it can be used in research projects designed to perfect self-driving programs.
The AI currently trying its luck at driving around Los Santos is called Charles. It uses pixel-level-analysis of on-screen information to make decisions about how to drive around the fictional city. In essence, it uses the same visual data that a human driver would. For an idea of what Charles “sees,” the developer has put together a short image library to explain it.
Crafted in Python — hence the Stream name, “Python Plays” — Charles currently doesn’t have any short-term memory, so doesn’t have much of an ability to learn. That’s why its driving is so poor right now and doesn’t seem to get much better no matter how long you view the stream. It does occasionally make it a few blocks without hitting anything and long-term viewers claim it is getting better, but it is far from being a safe or capable self-driving system.
Despite its obvious problems though, for a homebrew project by a single developer, the fact that it’s able to get around without any specialized information is reasonably impressive. You can imagine with time and further development, how an AI could become more proficient at driving by using a virtual toolkit like GTA V.
This kind of testing will hopefully help teach automated driving systems how to respond to unexpected obstacles with the same alertness and awareness as a skilled human driver. Anything can happen when you’re on the road, and it’s essential that an AI is able to respond appropriately, no matter what the eventuality.
At some point, self-driving technology will be advanced enough to stand up to trials on the same roads as every other human motorist. However, until we reach a certain level of sophistication, it will be the pedestrians roaming the streets in Grand Theft Auto V who bear the brunt of any unexpected occurrences.
Updated on 05-19-2017 by Jon Martindale: Added Twitch stream and information on it.
