OpenAI released Spinning Up yesterday. It is an educational resource for anyone who wants to become a skilled deep learning practitioner. Spinning Up has many examples in reinforcement learning, documentation, and tutorials.
The inspiration to build Spinning Up comes from OpenAI Scholars and Fellows initiatives. They observed that it’s possible for people with little-to-no experience in machine learning to rapidly become practitioners with the right guidance and resources. Spinning Up in Deep RL is also integrated into the curriculum for OpenAI 2019 cohorts of Scholars and Fellows.
A quick overview of Spinning Up course content
- A short introduction to reinforcement learning. What is it? The terminology used, different types of algorithms and basic theory to develop an understanding.
- An essay that lays out points and requirements to grow into a reinforcement learning research role. It explains the background, practice learning, and developing a project.
- A list of important research papers organized by topic for learning.
- A well-documented code repository of short, standalone implementations of various algorithms. These include Vanilla Policy Gradient (VPG), Trust Region Policy Optimization (TRPO), Proximal Policy Optimization (PPO), Deep Deterministic Policy Gradient (DDPG), Twin Delayed DDPG (TD3), and Soft Actor-Critic (SAC).
- And finally, a few exercises to solve and start applying what you’ve learned.
Support plan for Spinning Up
Fast-paced support period
For the first three weeks after release OpenAI will quickly work on bug-fixes, installation issues, and resolving errors in the docs. They will work to streamline the user experience so that it as easy as possible to self-study with Spinning Up.
A major review in April 2019
Around April next year, OpenAI will perform a serious review of the state of package based on feedback received from the community. After that any plans for future modification will be announced.
Public release of internal development
On making changes to Spinning Up in Deep RL with OpenAI Scholars and Fellows, the changes will also be pushed to the public repository so that it is available to everyone immediately.
In Spinning Up, running deep reinforcement learning algorithms is as easy as:
python -m spinup.run ppo --env CartPole-v1 --exp_name hello_worldFor more details on Spinning Up, visit the OpenAI Blog.
Read next
This AI generated animation can dress like humans using deep reinforcement learning