Samsung speeds up on-device AI processing with a 4x lighter and 8x faster algorithm

Yesterday, Samsung announced an on-device AI lightweight algorithm that can deliver optimization of low-power and high-speed computations. It uses an NPU (Neural Processing Unit) solution for speeding processing to enable 4 times lighter and 8 times faster computing than the existing algorithms of 32-bit deep learning data used in servers.

Last month, Samsung Electronics had announced their goal of expanding its proprietary NPU technology development, in order to strengthen Samsung’s leadership in the global system semiconductor industry by 2030. Recently, the company also delivered an update to this goal, at the conference on Computer Vision and Pattern Recognition (CVPR), with paper titled “Learning to Quantize Deep Networks by Optimizing Quantization Intervals With Task Loss”.

A Neural Processing Unit (NPU) is a processor which is optimized for deep learning algorithm computation, and designed to efficiently process thousands of computations simultaneously.

The Vice President and head of Computer Vision Lab of Samsung Advanced Institute of Technology, Chang-Kyu Choi says that, “Ultimately, in the future we will live in a world where all devices and sensor-based technologies are powered by AI. Samsung’s On-Device AI technologies are lower-power, higher-speed solutions for deep learning that will pave the way to this future. They are set to expand the memory, processor and sensor market, as well as other next-generation system semiconductor markets.”

Last year, Samsung had introduced Exynos 9 (9820), which featured a Samsung NPU inside the mobile System on Chip (SoC). This product allows mobile devices to perform AI computations independent of any external cloud server.

Samsung uses Quantization Interval Learning (QIL) to retain data accuracy

The Samsung Advanced Institute of Technology (SAIT) developed the on-device AI lightweight technology by adjusting data into groups of under 4 bits, while maintaining accurate data recognition. The technology is using ‘Quantization Interval Learning (QIL)’ to retain data accuracy. The QIL allows the quantized networks to maintain the accuracy of the full-precision (32-bit) networks with bit-width as low as 4-bit and minimize the accuracy degeneration with further bit width reduction like 3-bit and 2-bit.

• The 4-bit networks preserve the accuracy of the full-precision networks with various architectures.
• The 3-bit networks yield comparable accuracy to the full-precision networks.
• The 2-bit networks suffer from minimal accuracy loss.

The quantizer also achieves good quantization performance that outperforms the existing methods even when trained on a heterogeneous dataset and applied to a pretrained network.

When the data of a deep learning computation is presented in bit groups lower than 4 bits, computations of ‘and’ and ‘or’ are allowed, on top of the simpler arithmetic calculations of addition and multiplication.

By using the QIL process, the 4-bit computation gives the same results as existing processes while using 1/40 to 1/120 fewer transistors. As the system requires less hardware and less electricity, it can be mounted directly in-device at the place where the data for an image or fingerprint sensor is being obtained.

Benefits of Samsung’s on-device AI technology

• A large amount of data can be computed at a high speed without consuming excessive amounts of electricity.
• Samsung’s system semiconductor capacity will be developed and strengthened by directly computing data from within the device itself.
• By reduction in the cost of cloud construction for AI operations, Samsung’s on-device AI technology will provide quick and stable performance for use cases such as virtual reality and autonomous driving.
• It will save personal biometric information used for device authentication, such as fingerprint, iris and face scans, onto mobile devices safely.

Earlier this month, Samsung Electronics announced a multi-year strategic partnership with AMD. The strategic alliance is for ultra low power, high-performance mobile graphics IP based on AMD Radeon graphics technologies.

Surprisingly though, many users are not impressed with Samsung’s new technology, due to poor performances of Samsung’s previous devices.

This technology is not yet implemented in Samsung phones. It remains to be seen if the new on-device AI technology can make users change their opinion about Samsung.

Visit the Samsung Newsroom site for more details.

Read Next

Samsung AI lab researchers present system that can animate heads with one-shot learning

Facebook app is undeletable on Samsung phones and can possibly track your movements, reports Bloomberg

Samsung opens its AI based Bixby voice assistant to third-party developers