Intelligently designed

Playing the game NERO doesn’t seem, on the surface, so different from many other popular computer games of the day.

Robot soldiers battle it out in a post-apocalyptic world in a “struggle over the relics of human civilization.”

Guns are fired and carnage happens.

But for Computer Sciences Professor Risto Miikkulainen, NERO (Neuro-Evolving Robotic Operatives) is a model for creating fully autonomous, artificially intelligent agents that can learn in a rapidly changing environment. The game is based on neuroevolution, which combines the principles of neural networks (computer programs that run massively parallel, simple operations) with a learning process that mimics biological evolution.

Evolving neural networks like those used in NERO could one day be used in real time for robots handling the challenges of a search-and-rescue mission.

“This project started with hard-core machine learning research,” says Miikkulainen. “But we had this great idea: maybe we can make learning itself a game and it can be fun. It’s one way to make evolution and machine learning less mysterious.”

In NERO’s virtual world, each soldier has a neural network “brain.” Players train teams of soldiers by adjusting the soldiers’ rewards for their behaviors, like approaching or firing at an enemy. The soldiers are naïve at first and are easily killed-off during training maneuvers. But they are replaced by their descendents, and those soldier-bots that do best have more offspring populating the evolving team. The entire soldier population evolves battle behaviors over time. They learn through evolution.

Once the troops are ready for battle, evolution stops and players around the world pit their virtual teams against each other. The most highly evolved robot soldiers win.

The game was created by Miikulainen’s former post-doc Dr. Ken Stanley, now an assistant professor at the University of Central Florida, and a creative collaborative of art and computer sciences students. When the lab released NERO 1.0 in June 2005, there were so many downloads—20,000—that they had to shut the server down. There are still around 200 downloads per day and the game just won the student middleware award at the 7th annual Interactive Games Festival in San Jose, Calif.

Photo by Matt Lankes.