PyTorch-based HyperLearn Statsmodels aims to implement a faster and leaner GPU Sklearn

HyperLearn is a Statsmodel, a result of the collaboration of languages such as PyTorch, NoGil Numba, Numpy, Pandas, Scipy & LAPACK, and has similarities to Scikit Learn.

This project started last month by Daniel Hanchen and still has some unstable packages. He aims to make Linear Regression, Ridge, PCA, LDA/QDA faster, which then flows onto other algorithms being faster.

This Statsmodels combo incorporates novel algorithms to make it 50% more faster and enables it to use 50% lesser RAM along with a leaner GPU Sklearn.

HyperLearn also has an embedded statistical inference measures, and can be called similar to a Scikit Learn’s syntax (model.confidence_interval_)

HyperLearn’s Speed/ Memory comparison

There is a  50%+ improvement on Quadratic Discriminant Analysis (similar improvements for other models) as can be seen below:

Source: GitHub

Time(s) is Fit + Predict. RAM(mb) = max( RAM(Fit), RAM(Predict) )

Key Methodologies and Aims of the HyperLearn project

#1 Parallel For Loops

• Hyperlearn for loops will include Memory Sharing and Memory Management
• CUDA Parallelism will be made possible through PyTorch & Numba

#2 50%+ faster and leaner

• Matrix operations that have been improved include  Matrix Multiplication Ordering, Element Wise Matrix Multiplication reducing complexity to O(n^2) from O(n^3), reducing Matrix Operations to Einstein Notation and Evaluating one-time Matrix Operations in succession to reduce RAM overhead.
• Applying QR Decomposition and then SVD(Singular Value decomposition) might be faster in some cases.
• Utilise the structure of the matrix to compute faster inverse
• Computing SVD(X) and then getting pinv(X) is sometimes faster than pure pinv(X)

#3 Statsmodels is sometimes slow

• Confidence, Prediction Intervals, Hypothesis Tests & Goodness of Fit tests for linear models are optimized.
• Using Einstein Notation & Hadamard Products where possible.
• Computing only what is necessary to compute (Diagonal of matrix only)
• Fixing the flaws of Statsmodels on notation, speed, memory issues and storage of variables.

#4 Deep Learning Drop In Modules with PyTorch

• Using PyTorch to create Scikit-Learn like drop in replacements.

#5 20%+ Less Code along with Cleaner Clearer Code

• Using Decorators & Functions wherever possible.
• Intuitive Middle Level Function names like (isTensor, isIterable).
• Handles Parallelism easily through hyperlearn.multiprocessing

#6 Accessing Old and Exciting New Algorithms

• Matrix Completion algorithms – Non Negative Least Squares, NNMF
• Batch Similarity Latent Dirichelt Allocation (BS-LDA)
• Correlation Regression and many more!

       
Daniel further went on to publish some prelim algorithm timing results on a range of algos from MKL Scipy, PyTorch, MKL Numpy, HyperLearn’s methods + Numba JIT compiled algorithms

Here are his key findings on the HyperLearn statsmodel:

1. HyperLearn’s Pseudoinverse has no speed improvement
2. HyperLearn’s PCA will have over 200% improvement in speed boost.
3. HyperLearn’s Linear Solvers will be over 1 times faster i.e  it will show a 100% improvement in speed

You can find all the details of the test on reddit.com

For more insights on HyperLearn, check out the release notes on Github.

Read Next

A new geometric deep learning extension library for PyTorch releases!

NVIDIA leads the AI hardware race. But which of its GPUs should you use for deep learning?

Introduction to Sklearn