Uber and GM Cruise are open sourcing their Automation Visualization Systems

3 min read

Yesterday, Uber and GM Cruise announced that they are open sourcing their respective Autonomous Visualization Systems (AVS). It is a new way for the industry to understand and share its data.

AVS has become a new standard for describing and visualizing autonomous vehicle perception, motion, and planning data while offering a web-based toolkit for building applications. AVS makes it easier for developers to make a decision with respect to their development. It is free for users, so it might encourage developers to come up with interesting developments for the autonomous industry.

AVS acts as a standardized visualization layer that frees developers from building custom visualization software for their autonomous vehicles. Developers can now focus on core autonomy capabilities for drive systems, remote assistance, mapping, and simulation, with the help of AVS abstracting visualization.

There is a need to have operators understand why their cars make certain decisions. The visualization system helps engineers to break out and playback certain trip intervals for closer inspection. AV operators rely on off-the-shelf visualization systems that aren’t designed with self-driving cars in mind. They are usually limited to bulky desktop computers that are difficult to navigate. Uber has taken move towards its web-based visualization platform so operators don’t have to learn complex computer graphics and data visualization.

Uber opted for XVIZ and streetscape.gl

Autonomous vehicle development is rapidly evolving with new services, data sets, and many use cases that require new solutions. The team at Uber had unique requirements that needed to be addressed. The team wanted to manage the data while retaining performance comparable to desktop-based systems.

So the team built a system around two key pieces: XVIZ, that provides the data (including management and specification) and streetscape.gl which is the component toolkit to power web applications. Uber’s new tool seems to be more geared to AV operators specifically.

While talking about its Autonomous Visualization System, the company said, “It is a customizable web-based platform that allows self-driving technology developers — big or small — to transform their vehicle data into an easily digestible visual representation of what the vehicle is seeing in the real world.”

XVIZ

XVIZ provides a stream-oriented view of a scene changing over time and also a user interface display system. Users can randomly seek and understand the state of the world at that point. Just like an HTML document, it’s presentation is focused and structured according to a schema that allows for introspection. It also allows for easy exploration and interrogation of the data.

streetscape.gl

streetscape.gl is a toolkit used for developing web applications that consume data in the XVIZ protocol. It also offers components for visualizing XVIZ streams in 3D viewports, charts, tables, videos, and more. It addresses common visualization challenges such as time synchronization across data streams, coordinate systems, cameras, dynamic styling, and interaction with 3D objects and cross components.

Voyage co-founder Warren Ouyang said, “We’re excited to use Uber’s autonomous visualization system and collaborate on building better tools for the community going forward.”

Last week, in a Medium post, Cruise introduced its graphics library of two- and three-dimensional scenes called “Worldview.” It provides 2D and 3D cameras, keyboard and mouse movement controls, click interaction, and a suite of built-in drawing commands. Developers can build custom visualizations easily, without having to learn complex graphics APIs or write wrappers to make them work with React.

In statement to Medium, Cruise said, “We hope Worldview will lower the barrier to entry into the powerful world of WebGL, giving web developers a simple foundation and empowering them to build more complex visualizations.”