Earlier this year, DeepMind’s AI Alphastar defeated two professional players at StarCraft II, a real-time strategy video game. Now, European Starcraft II players will get a chance to face off experimental versions of AlphaStar, as part of ongoing research into AI.
So now I can just write off all my losses as “probably AlphaStar” and feel good about myself again today is a good day for Starcraft
— Max (@MaxBakerTV) July 10, 2019
AlphaStar learns by imitating the basic micro and macro-strategies used by players on the StarCraft ladder. A neural network was trained initially using supervised learning from anonymised human games released by Blizzard. Once the agents get trained from human game replays, they’re then trained against other competitors in the “AlphaStar league”. This is where a multi-agent reinforcement learning process starts. New competitors are added to the league (branched from existing competitors). Each of these agents then learns from games against other competitors. This ensures that each competitor performs well against the strongest strategies, and does not forget how to defeat earlier ones.
Anyone who wants to participate in this experiment will have to opt into the chance to play against the StarCraft II program. There will be an option provided in the in-game pop-up window. Users can alter their opt-in selection at any time. To ensure anonymity, all games will be blind test matches. European players that opt-in won’t know if they’ve been matched up against AlphaStar. This will help ensure that all games are played under the same conditions, as players may tend to react differently when they know they’re against an AI. A win or a loss against AlphaStar will affect a player’s MMR (Matchmaking Rating) like any other game played on the ladder.
“DeepMind is currently interested in assessing AlphaStar’s performance in matches where players use their usual mix of strategies,” Blizzard said in its blog post. “Having AlphaStar play anonymously helps ensure that it is a controlled test, so that the experimental versions of the agent experience gameplay as close to a normal 1v1 ladder match as possible. It also helps ensure all games are played under the same conditions from match to match.”
Some people have appreciated the anonymous testing feature. A Hacker News user commented, “Of course the anonymous nature of the testing is interesting as well. Big contrast to OpenAI’s public play test. I guess it will prevent people from learning to exploit the bot’s weaknesses, as they won’t know they are playing a bot at all. I hope they eventually do a public test without the anonymity so we can see how its strategies hold up under focused attack.”
Others find it interesting to see what happens if players know they are playing against AlphaStar.
I think it’ll also be a very interesting experiment if the players *knew* they are actually playing against AlphaStar, just to observe if humans can come up with different strategies specifically against A.I., and measure the difference in performance when this anonymity is gone.
— hardmaru (@hardmaru) July 10, 2019
AlphaStar will play in Starcraft’s three in-universe races (Terran, Zerg, or Protoss). Pairings on the ladder will be decided according to normal matchmaking rules, which depend on how many players are online while AlphaStar is playing. It will not be learning from the games it plays on the ladder, having been trained from human replays and self-play. The Alphastar will also use a camera interface and more restricted APMs. Per the blog post, “AlphaStar has built-in restrictions, which cap its effective actions per minute and per second. These caps, including the agents’ peak APM, are more restrictive than DeepMind’s demonstration matches back in January, and have been applied in consultation with pro players.”
Notable changes include: AlphaStar now plays all three races (rather than just playing Protoss as before), and AlphaStar can play against all three races (instead of just playing versus Protoss). Also, it uses the camera view that is used when it lost to Mana in the live match. https://t.co/9LxjifWcDw
— Eric Wallace (@Eric_Wallace_) July 10, 2019
Oh! @DeepMindAI is anonymously entering ladder on Europe server. I am very curious how the games are going to look like and if anybody will be able to detect they have played against AlphaStar. Super excited about this project! https://t.co/upoakBiEMn
— Grzegorz Komincz (@Liquid_MaNa) July 10, 2019
DeepMind will be benchmarking the performance of a number of experimental versions of AlphaStar to enable DeepMind to gather a broad set of results during the testing period.
DeepMind will use a player’s replays and the game data (skill level, MMR, the map played, race played, time/date played, and game duration) to assess and describe the performance of the AlphaStar system. However, Deepmind will remove identifying details from the replays including usernames, user IDs and chat histories. Other identifying details will be removed to the extent that they can be without compromising the research DeepMind is pursuing.
For now, AlphaStar agents will play only in Europe. The research results will be released in a peer-reviewed scientific paper along with replays of AlphaStar’s matches.
Read Next
Google DeepMind’s AI AlphaStar beats StarCraft II pros TLO and MaNa; wins 10-1 against the gamers
Deepmind’s AlphaZero shows unprecedented growth in AI, masters 3 different games
