OpenAI’s Gradient-checkpointing, Ethercraft, an Ethereum based RPG, pytorch implementation of faster R-CNN, and more in today’s top stories around artificial intelligence, blockchain, and data science news.
1. OpenAI releases gradient-checkpointing – a package for making huge neural nets fit in memory
OpenAI has released gradient checkpointing – a package for fitting bigger Tensorflow models on GPUs. The package is developed jointly by Tim Salimans and Yaroslav and allows trading off some memory usage with computation to make a model fit into memory more easily. The package was able to fit more than 10x larger models onto GPU for feed-forward models, at only a 20% increase in computation time.
The memory intensive part of training deep neural networks is computing the gradient of the loss by backpropagation. The package checkpoints nodes in the computation graph and recomputes the parts of the graph in between those nodes during backpropagation. Thus making it possible to calculate the gradient at reduced memory cost. Using this way, researchers were able to reduce the memory consumption to O(sqrt(n)), at the cost of performing one additional forward pass when training deep feed-forward neural networks consisting of n layers.
The complete working of the model can be found at the Github Repo.
2. A new project enables faster pytorch implementation of faster R-CNN
There is a new project which allows complete pytorch implementation of faster R-CNN. This project aims to accelerate the training of faster R-CNN object detection models. Here are some highlights:
- It is pure Pytorch code. All numpy implementations are converted to pytorch
- Supports multi-image batch training.
- Supports multiple GPUs training. A multiple GPU wrapper (nn.DataParallel here) is used to make it flexible to use one or more GPUs, as a merit of the above two features.
- Supports three pooling methods, roi pooling, roi align and roi crop.
- It is memory efficient.
- It is faster.
More information can be found at the Github repo.
3. Nordcloud teams up with Microsoft Azure to help customers to complete AI projects faster
Nordcloud has partnered with Microsoft for jointly deploying Azure AI-based solutions for their enterprise customers. Nordcloud provides public cloud infrastructure solutions and cloud-native application services. It plans to help a much wider range of businesses implement artificial intelligence by leveraging Azure’s AI infrastructure and services. Microsoft’s Azure cloud computing service offers Platform Services for AI, machine learning, and IoT development. Nordcloud will work with Azure to allow its clients to spot opportunities to build cost-effective, scalable and intelligent AI-powered digital solutions. Customers can now use the entire Azure AI stack to create scalable intelligent services. They can also add smart Azure features to their existing solutions in a fast and agile manner.
4. Microsoft creates AI for state-of-the-art natural language processing
A team at Microsoft Research Asia have reached the human parity milestone by creating an AI that can read a document and answer questions as well as a person. Microsoft has submitted a model based on the Stanford Question Answering Dataset (SQuAD). SQuAD is a machine reading comprehension dataset that is made up of questions about a set of Wikipedia articles. Microsoft’s model was able to reach the score of 82.650 on the exact match portion. The human performance on the same set of questions and answers is 82.304. Microsoft is already applying earlier versions of the models that were submitted for the SQuAD dataset leaderboard in its Bing search engine, and more complex problems.
5. Introducing Ethercraft, a decentralized Role Play Game based on Ethereum blockchain
There is a new Role Play Game (RPG) in town. Ethercraft is a new decentralized RPG game based on the Ethereum Blockchain. Although this game is still being developed, players can already start trading and purchasing items. The game and its development can be divided into three main components:
-
- Items: Buying, selling, and trading.
- Crafting: Combining items according to a recipe
- The playable video game
Players will be able to buy, sell, and trade items beginning immediately. All of the items are independent tokens adhering to the ERC20 token standard. Players can also buy, sell, and trade Gold Pieces. Gold Pieces (XGP) is an ERC20 token, which can be used as an intermediary for everything involving ETH transfer. It also acts as a governance token.
The Crafting and the playable video game is currently undergoing development and testing.