Unity releases ML-Agents toolkit v0.5 with Gym interface, a new suite of learning environments

2 min read

In their commitment to become the go-to platform for Artificial Intelligence, Unity has released a new version of their ML-Agents Toolkit. ML-Agents toolkit v0.5 comes with more flexible action specification, Gym interface for researchers to more easily integrate ML-Agents environments into their training workflows, and a new suite of learning environments replicating some of the Continuous Control benchmarks used in Deep Reinforcement Learning.

They have also released a research paper on ML-Agents which the Unity platform has titled “Unity: A General Platform for Intelligent Agent.”

Changes to the ML-Agents toolkit v0.5

A lot of changes have been made pertaining to ML-Agents toolkit v0.5.

Highlighted changes to repository structure

The python folder has been renamed ml-agents. It now contains a python package called mlagents.
The unity-environment folder, containing the Unity project, has been renamed UnitySDK.
The protobuf definitions used for communication have been added to a new protobuf-definitions folder.
Example curricula and the trainer configuration file have been moved to a new config sub-directory.

New features

New package gym-unity which provides gym interface to wrap UnityEnvironment.
The ML-Agents toolkit v0.5 can now run multiple concurrent training sessions with the –num-runs=<n> command line option.
Added Meta-Curriculum which supports curriculum learning in multi-brain environments.
Action Masking for Discrete Control which makes it possible to mask invalid actions each step to limit the actions an agent can take.

Fixes & Performance Improvements

Replaced some activation functions to swish.
Visual Observations use PNG instead of JPEG to avoid compression losses.
Improved python unit tests.
Multiple training sessions are available on single GPU.
Curriculum lessons are now tracked correctly.
Developers can now visualize value estimates when using models trained with PPO from Unity with GetValueEstimate().
It is now possible to specify which camera the Monitor displays to.
Console summaries will now be displayed even when running inference mode from python.
Minimum supported Unity version is now 2017.4.

You can read all about the new version of ML-Agents Toolkit on the Unity Blog.