DeepMind made it a mission to prove that not only can an AI become truly proficient in a game, but it can do so without even being taught the rules. Its brand new AI agent, called MuZero, does this not only with visually simple games with complex strategies, such as Go, Chess and Shogi, but with visually complex Atari Games.
The success of DeepMind's early AI was at least in part due to very efficient navigation of the immense decision trees that represent possible actions in a game. In Go or Chess these trees are governed by very specific rules, such as where the pieces can move, what happens when this piece does and so on.
The AI that beat the world champions at Go, AlphaGo, knew these rules and kept them in mind (or perhaps in RAM) as they studied the matches between and against human players, forming a series of best practices and strategies. The sequel, AlphaGo Zero, did so without human data, playing only against itself. AlphaZero did the same with Go, Chess, and Shogi in 2018, creating a single model of artificial intelligence that can play all of these games competently.
But in all of these cases the AI was presented with a set of immutable and known rules for games, creating a framework around which to build its strategies. Think about it: if you are told that a pawn can become a queen, you plan it from the start, but if you have to find out, you could develop completely different strategies.
As the company explains in a blog post about their new research, if AIs are told the rules in advance, "this makes it difficult to apply them to messy real-world problems that are typically complex and difficult to distill into simple rules."
The company's latest advancement, therefore, is MuZero, which plays not only the aforementioned games but a variety of Atari games, and does so without any regulation being provided. The final model learned to play all of these games not only by experimenting on its own (no human data) but without even being taught the most basic rules.
Instead of using the rules to find the best scenario (because it can't), MuZero learns to take every aspect of the game environment into consideration, observing for itself whether it matters or not. In millions of games he learns not just the rules, but the general value of a position, the general policies to move forward and a way of evaluating one's actions with hindsight.
This latter ability helps him learn from his mistakes, rewinding and re-playing games to try different approaches that further refine the position and values of politics.
You might remember Agent57, another creation from DeepMind that excelled at a set of 57 Atari games. MuZero takes the best of that artificial intelligence and combines it with the best of AlphaZero. MuZero differs from the former in that it does not shape the entire game environment, but focuses on the parts that influence its decision-making process, and from the latter in that it bases its rules model purely on its own experimentation and first-hand knowledge.
Understanding the game world allows MuZero to effectively plan its actions even when the game world is, like many Atari games, partly randomized and visually complex. This brings it closer to an artificial intelligence that can safely and intelligently interact with the real world, learning to understand the world around it without needing to be told every detail (although it is likely that some, such as "not crush humans ", will be engraved in stone). As one of the researchers told the BBC, the team is already experimenting with how MuZero could improve video compression - obviously a very different problem than Ms. Pac-Man.
MuZero details were published today in the journal Nature.
