Introducing “Pyro” for deep probabilistic modeling

Last year when Uber set up its ambitious facility in San Francisco as Uber AI Labs, the aim was to leverage cutting-edge research in artificial intelligence and machine learning to move people and things in the real world — a challenge that is more complex and uncertain than it appears on paper. It extends to, as the firm admitted, teaching a self-driven machine to safely and autonomously navigate the world, whether a car on the roads or an aircraft through busy airspace or new types of robotic devices.

Well, the first big initiative to come out of the Labs is Pyro, a deep universal probabilistic programming language. “Pyro is a tool for deep probabilistic modeling, unifying the best of modern deep learning and Bayesian modeling,” writes Stanford researcher Noah Goodman, a member of Uber AI Labs.

Written in Python, the Pyro programming language supports PyTorch in the backend. Among the key principles underlying Pyro’s design, it is a flexible and scalable programming library, implemented with a small core of powerful, composable abstractions.

Pyro: Design principles and insights

In Uber’s own words, Pyro was developed to satisfy the following four design principles:

• Universal: Pyro is a universal PPL—it can represent any computable probability distribution. How? By starting from a universal language with iteration and recursion (arbitrary Python code), and then adding random sampling, observation, and inference.
• Scalable: Pyro scales to large data sets with little overhead above hand-written code. How? By building modern black box optimization techniques, which use mini-batches of data, to approximate inference.
• Minimal: Pyro is agile and maintainable. How? Pyro is implemented with a small core of powerful, composable abstractions. Wherever possible, the heavy lifting is delegated to PyTorch and other libraries.
• Flexible: Pyro aims for automation when you want it and control when you need it. How? Pyro uses high-level abstractions to express generative and inference models, while allowing experts to easily customize inference.

In a way, Pyro is going to reflect on interesting aspects in PPL research, starting from dynamic computational graphs to deep generative models, or even programmable inference.

“In Pyro, both the generative models and the inference guides can include deep neural networks as components,” Goodman wrote. “The resulting deep probabilistic models have shown great promise in recent work, especially for unsupervised and semi-supervised machine learning problems.”

Pyro: Installation

(Remember to first install PyTorch)

Install via pip:

Python 2.7.*:

pip install pyro-ppl

Python 3.5:

pip3 install pyro-ppl

Install from source:

git clone git@github.com:uber/pyro.git

cd pyro

pip install .

Still in alpha release, Pyro may see several possible enhancements in coming days with more and more engagement from probabilistic programming and deep learning communities.