Tesla v9 to incorporate neural networks for autopilot
Tesla, the car maker founded by Elon Musk is incorporating larger neural networks for autopilot in the new Tesla v9.
Based on the new Autopilot capabilities of version 9, it was known that the new neural net was a significant upgrade over the v8. It can now track vehicles and other objects around the car by making better use of the eight cameras around the car.
Tesla motors club member Jimmy_d, a deep learning expert has shared his thoughts on v9 and the neural network used in it. Tesla has now deployed a new camera network to handle all 8 cameras.
Like V8 the V9 neural network system consists of a set of ‘camera networks’ which process camera output directly. There is a separate set of ‘post processing’ networks that take output from the camera networks and turn it into higher level actionable abstractions. V9 is a pretty big change from V8.
Other major changes from V8 to V9 as stated by Jimmy are:
- Same weight file being used for all cameras (this has pretty interesting implications and previously V8 main/narrow seems to have had separate weights for each camera)
- Processed resolution of 3 front cameras and back camera: 1280×960 (full camera resolution)
- Processed resolution of pillar and repeater cameras: 640×480 (1/2×1/2 of camera’s true resolution)
- all cameras: 3 color channels, 2 frames (2 frames also has very interesting implications) (was 640×416, 2 color channels, 1 frame, only main and narrow in V8)
These camera changes mean a much larger neural network that require more processing power.
The V9 network takes images in a resolution of 1280×960 with 3 color channels and 2 frames per camera. That’s 1280x960x3x2 as an input which is 7.3MB. The V8 main camera processing frame was 640x416x2 that is, 0.5MB. The v9 camera will have access to more details.
About the network size, Jimmy said: “This V9 network is a monster, and that’s not the half of it. When you increase the number of parameters (weights) in an NN by a factor of 5 you don’t just get 5 times the capacity and need 5 times as much training data. In terms of expressive capacity increase it’s more akin to a number with 5 times as many digits. So if V8’s expressive capacity was 10, V9’s capacity is more like 100,000.”
Tesla CEO Elon Musk had something to say about the estimates made by Jimmy:
To be clear, actual NN improvement is significantly overestimated in this article. V9.0 vs V8.1 is more like a ~400% increase in useful ops/sec due to enabling integrated GPU & better use of discrete GPU.
— Elon Musk (@elonmusk) October 16, 2018
The amount of training data doesn’t go up by a mere 5x. It takes at least thousands and even millions of times more data to fully utilize a network that has 5x as many parameters.
We will see this new neural network implementation on the road in new cars about six months down the line.
For more details, you can view the discussion on the Tesla motors club website.
Read next
Tesla is building its own AI hardware for self-driving cars
Elon Musk reveals big plans with Neuralink
DeepMind, Elon Musk, and others pledge not to build lethal AI