Google DeepMind's latest AI agent learned to drive 'Goat Simulator 3'


“SIMA goes a step further and shows strong generalization to new games,” he says. “The number of environments is still very small, but I think SIMA is on the right track.

A new way to play

SIMA shows that DeepMind is bringing a new twist to game-playing agents, an AI technology the company has developed in the past.

In 2013, before DeepMind was acquired by Google, the London-based startup had shown how a technique called reinforcement learning, which involves training an algorithm with positive and negative feedback on its performance, could teach computers how to play classic Atari video games. Can help. In 2016, as part of Google, DeepMind developed AlphaGo, a program that used the same approach to defeat the world champion of Go, an ancient board game that requires subtle and intuitive skills.

For the SIMA project, the Google DeepMind team collaborated with several game studios to collect keyboard and mouse data from humans playing 10 different games with 3D environments, including no man's sky, tear down, hydroneerAnd satisfactory, DeepMind later added descriptive labels to that data to associate clicks and taps with actions users took, for example whether they were a goat looking for their jetpack or a human character digging for gold. .

Data obtained from human players was fed into the kind of language models that power modern chatbots, which gained the ability to process language by digesting vast databases of text. SIMA can then take actions in response to typed commands. And finally, humans evaluated SIMA's efforts inside different games, generating data that was used to improve its performance.

The SIMA AI software was trained using data from humans playing 10 different games with 3D environments.

Courtesy of Google DeepMind

After all that training, SIMA is able to take actions in response to hundreds of commands given by a human player, like “turn left” or “go to the spacecraft” or “go through the gate” or “cut down a tree.” ” “The program can perform over 600 actions ranging from exploration to combat to equipment use. In line with Google's ethical guidelines on AI, the researchers avoided games that involve violent activities.

“It's still a research project,” says Tim Harley, another member of the Google DeepMind team. “However, one can imagine that one day agents like SIMA will play in the game with you and your friends.”

Video games provide a relatively safe environment for AI agents to work. In order for agents to perform useful office or everyday administrative tasks, they will need to become more reliable. Harley and Besse at DeepMind say they are working on technologies to make agents more trustworthy.

UPDATE 3/13/2024, 10:20am ET: Comment added from Linxi “Jim” Fan.