PyTorch
PyTorch Lightning & Lightning Hydra Template are neat. Tensor Puzzles is great intro.
title: PyTorch#
PyTorch Lightning ↗ & Lightning Hydra Template ↗ are neat. Tensor Puzzles ↗ is great intro.
Notes#
Links#
- PyTorch Tutorials ↗
- RL Adventure ↗
- PyTorch template project ↗
- PyTorch GAN ↗ - PyTorch implementations of Generative Adversarial Networks.
- PyTorch examples ↗
- Grokking PyTorch ↗
- Awesome PyTorch list ↗
- OCaml PyTorch ↗ - OCaml bindings for PyTorch.
- Apex ↗ - PyTorch Extension: Tools for easy mixed precision and distributed training in PyTorch.
- Translate ↗ - PyTorch Language Library.
- What is torch.nn really? ↗
- Introduction to PyTorch Workshop at the AMLD 2019 ↗
- Neural Pipeline ↗ - Neural networks training pipeline based on PyTorch. Designed to standardize training process and accelerate experiments.
- Faster R-CNN and Mask R-CNN in PyTorch ↗
- tch-rs ↗ - Rust bindings for PyTorch.
- PyTorch Geometric ↗ - Geometric deep learning extension library for PyTorch.
- PyTorch Geometric Temporal ↗ - Temporal (dynamic) extension library for PyTorch Geometric.
- BigGAN-PyTorch ↗ - Contains code for 4-8 GPU training of BigGANs from Large Scale GAN Training for High Fidelity Natural Image Synthesis.
- Incredible PyTorch ↗ - Curated list of tutorials, papers, projects, communities and more relating to PyTorch.
- NimTorch ↗ - Nim frontend for pytorch, aiming to be mostly auto-generated and internally using ATen.
- PyTorch adds new dev tools as it hits production scale (2019) ↗
- PyTorch tutorials demonstrating modern techniques with readable code ↗
- PyTorch internals ↗
- Towards Reproducible Research with PyTorch Hub (2019) ↗
- XLM ↗ - PyTorch original implementation of Cross-lingual Language Model Pretraining.
- BoTorch ↗ - Bayesian optimization in PyTorch.
- PyTorch Lightning ↗ - Lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate. (Tutorial ↗) (Finding why Pytorch Lightning made my training 4x slower ↗) (PyTorch Lightning for Dummies ↗) (Tutorial ↗) (Docs ↗) (Web ↗) (Intro ↗) (Intro Code ↗)
- PointNet.pytorch ↗ - PyTorch implementation for “PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation”.
- former ↗ - Simple transformer implementation from scratch in PyTorch.
- PyTorch Implementation of Differentiable ODE Solvers ↗
- PyTorch BigGraph ↗ - Software used for generating embeddings from large-scale graph-structured data.
- Deep Learning (PyTorch) ↗ - Projects and exercises for the latest Deep Learning ND program.
- PyWarm ↗ - Cleaner way to build neural networks for PyTorch.
- PyTorch implementations of algorithms for density estimation ↗
- Higher ↗ - PyTorch library allowing users to obtain higher order gradients over losses spanning training loops rather than individual training steps.
- Detectron2 ↗ - FAIR’s next-generation research platform for object detection and segmentation.
- Deep learning with PyTorch: a 60 minute blitz ↗ (HN ↗)
- Neural Network Programming - Deep Learning with PyTorch (2019) ↗ - Series is all about neural network programming and PyTorch.
- A quick tour of Torch internals (2015) ↗
- How to Run PyTorch with TPUs ↗
- EfficientNet PyTorch ↗
- Captum ↗ - Model interpretability and understanding for PyTorch.
- Pytorch-Struct ↗ - Fast, general, and tested differentiable structured prediction in PyTorch.
- Dive Into Deep Learning 2 PyTorch ↗
- PyTorch at Tesla - Andrej Karpathy, Tesla (2019) ↗ (HN ↗)
- 3D Ken Burns Effect ↗ - Implementation of 3D Ken Burns Effect from a Single Image using PyTorch.
- Fast LSTMs in PyTorch (2019) ↗
- F8 2019: Developing and Scaling AI Experiences at Facebook with PyTorch ↗
- Kaolin ↗ - PyTorch Library for Accelerating 3D Deep Learning Research.
- SpConv ↗ - PyTorch Spatially Sparse Convolution Library.
- Segmentation models ↗ - Python library with Neural Networks for Image Segmentation based on PyTorch.
- LSTM and QRNN Language Model Toolkit ↗
- cvxpylayers ↗ - Python library for constructing differentiable convex optimization layers in PyTorch and TensorFlow using CVXPY.
- Frustum ConvNet ↗ - Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection.
- PyTorch Elastic ↗ - Framework that enables distributed training jobs to be executed in a fault tolerant and elastic manner.
- torchgpipe ↗ - GPipe implementation in PyTorch.
- C++ Implementation of PyTorch Tutorial for Deep Learning Researchers ↗
- Neural Painters: A learned differentiable constraint for generating brushstroke paintings ↗
- CycleGAN and pix2pix in PyTorch ↗ - Image-to-Image Translation in PyTorch.
- Easily Extendable Basic Deep Metric Learning Pipeline ↗
- TTAch ↗ - Image Test Time Augmentation with PyTorch.
- Hyperspherical Variational Auto-Encoders in PyTorch ↗
- Hyperbolic Graph Convolutional Networks in PyTorch ↗
- Deep Learning with PyTorch Book (2020) ↗ - Provides a detailed, hands-on introduction to building and training neural networks with PyTorch. (HN ↗) (Code ↗)
- OpenProtein ↗ - PyTorch framework for tertiary protein structure prediction.
- GPyTorch ↗ - Highly efficient and modular implementation of Gaussian Processes in PyTorch. (Docs ↗)
- Memory-efficient implementation of DenseNets ↗
- PyTorch Toolbelt ↗ - PyTorch extensions for fast R&D prototyping and Kaggle farming.
- Reformer ↗ - Efficient Transformer, in PyTorch.
- Which PyTorch-compatible training abstraction libraries do you use and why? (2020) ↗
- Deep Learning with PyTorch Course ↗ (Code ↗)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library (2019) ↗
- Open Graph Benchmark (OGB) ↗ - Collection of benchmark datasets, data-loaders and evaluators for graph machine learning in PyTorch.
- Ignite ↗ - High-level library to help with training neural networks in PyTorch.
- PyTorch Forum ↗
- Collection of PyTorch notebooks for learning and practicing deep learning ↗
- face.evoLVe ↗ - High-Performance Face Recognition Library based on PyTorch.
- PyTorch implementation for 3D Bounding Box Estimation Using Deep Learning and Geometry ↗
- Fast AutoAugment ↗ - Official Implementation of ‘Fast AutoAugment’ in PyTorch.
- Rethinking the Value of Network Pruning ↗
- PyTorch Metric Learning ↗
- Deep Tutorials for PyTorch ↗
- Matchbox ↗ - Enables deep learning researchers to write PyTorch code at the level of individual examples, then run it efficiently on minibatches.
- torchtext ↗ - Data loaders and abstractions for text and NLP.
- Relational Networks ↗ - PyTorch implementation of “A simple neural network module for relational reasoning”.
- nflows ↗ - Normalizing flows in PyTorch.
- torchlayers ↗ - Shape inference for PyTorch (like in Keras) + new SoTA layers.
- PyTorch re-implementation of Neural Radiance Fields ↗
- Torchmeta ↗ - Collection of extensions and data-loaders for few-shot learning & meta-learning in PyTorch.
- PyTorch SimCLR: A Simple Framework for Contrastive Learning of Visual Representations ↗
- Neural Spline Flow, RealNVP, Autoregressive Flow, 1x1Conv in PyTorch ↗
- PyTorch implementation of NeRF (Neural Radiance Fields) ↗
- PyTorch learning rate finder ↗
- PyTorch Scatter ↗ - PyTorch Extension Library of Optimized Scatter Operations.
- TorchServe ↗ - Flexible and easy to use tool for serving PyTorch models.
- torch-optimizer ↗ - Collection of optimizers for PyTorch.
- PyTorch 1.5 (2020) ↗ (HN ↗)
- Tacotron 2 (without wavenet) ↗ - PyTorch implementation of Natural TTS Synthesis By Conditioning Wavenet On Mel Spectrogram Predictions.
- Tacotron 2 latest papers ↗
- pytorch-crf ↗ - Conditional random field in PyTorch.
- torchprof ↗ - PyTorch layer-by-layer model profiler.
- A Taste of PyTorch C++ frontend API (2020) ↗
- Tutorial: Mixture Density Networks with JAX (2020) ↗
- Deep learning CNN feature visualization using PyTorch ↗
- rectorch ↗ - PyTorch-based framework for state-of-the-art top-N recommendation.
- neural-dream ↗ - PyTorch implementation of DeepDream algorithm.
- PyProf2 ↗ - PyTorch Profiling tool.
- Blitz ↗ - Simple and extensible library to create Bayesian Neural Network layers on PyTorch.
- PyContrast ↗ - PyTorch implementation of Contrastive Learning methods.
- PyVacy ↗ - Differentially Private Optimization Algorithms for PyTorch.
- Gaussian mixture models in PyTorch ↗
- Effective PyTorch ↗ - PyTorch tutorials and best practices.
- PyTorch for Numpy Users ↗ (Code ↗)
- PyTorch implementation of Fully Convolutional Networks ↗
- Lite Transformer with Long-Short Range Attention ↗
- PyTorch implementation of the KWNG estimator ↗
- Classification with PyTorch ↗
- PyTorch Toolbox ↗ - Aiming to make you write Pytorch code more easier, readable and concise.
- Sinusoidal Representation Networks (SIREN) ↗
- PyTorch Hub ↗ - Discover and publish models to a pre-trained model repository designed for research exploration. (Code ↗)
- Pretrained models for PyTorch ↗
- DrQA ↗ - Reading Wikipedia to Answer Open-Domain Questions. In PyTorch.
- minGPT ↗ - Minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training. (Code ↗)
- MiniTorch ↗ - Teaching library for machine learning engineers who wish to learn about the internal concepts underlying deep learning systems.
- PyTorch Performance Tuning Guide - Szymon Migacz, NVIDIA (2020) ↗ (Summary ↗)
- Opacus ↗ - Training PyTorch models with differential privacy. (Web ↗) (Opacus Lab Research ↗)
- Towards Fast, Accurate and Stable 3D Dense Face Alignment ↗
- Hivemind ↗ - Library to train large neural networks across the internet. Imagine training one huge transformer on thousands of computers from universities, companies, and volunteers.
- Prediction and Policy-learning Under Uncertainty (PPUU) ↗
- PyTorch Lightning Bolts ↗ - Toolbox of models, callbacks, and datasets for AI/ML researchers.
- LabML Neural Networks ↗ - Collection of simple PyTorch implementation of various neural network architectures and layers. (Code ↗) (HN ↗)
- PyTorch Interpret ↗ - PyTorch library for interpreting your deep learning results, using both visualisations and attributions.
- Lambda Networks - PyTorch ↗ - Implementation of Lambda Networks, an attention-based solution for image recognition that reaches SOTA.
- PyTorch image data augmentation ↗
- Bootstrap Your Own Latent (BYOL), in PyTorch ↗
- Vision Transformer - PyTorch ↗ - Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in PyTorch.
- Benchmkarking Optimizers in PyTorch (2020) ↗
- ShrinkBench ↗ - PyTorch library to facilitate development and standardized evaluation of neural network pruning methods.
- Asteroid ↗ - PyTorch-based audio source separation toolkit for researchers.
- The Hitchhikers’s Guide to PyTorch for Data Scientists (2020) ↗
- Pointnet2/Pointnet++ PyTorch ↗
- Dreamer PyTorch ↗ - Dream to Control: Learning Behaviors by Latent Imagination, implemented in PyTorch.
- project-NII-pytorch-scripts ↗
- fairscale ↗ - PyTorch extension library for high performance and large scale training for optimizing training on one or across multiple machines/nodes.
- FastSpeech PyTorch ↗
- Lightwood ↗ - Pytorch based framework that breaks down machine learning problems into smaller blocks that can be glued together.
- PyTorch Loss Functions: Guide (2020) ↗
- David’s Tips on How to Read PyTorch ↗
- MeshCNN in PyTorch ↗ - General-purpose deep neural network for 3D triangular meshes, which can be used for tasks such as 3D shape classification or segmentation.
- tinygrad ↗ - Something in between a PyTorch and a karpathy/micrograd. (tinygrad: tiny efficientnet is recognize cat? ↗) (HN ↗)
- Lucent ↗ - Lucid library adapted for PyTorch.
- pytorch-pfn-extras ↗ - Supplementary components to accelerate research and development in PyTorch.
- PyHessian ↗ - PyTorch library for second-order based analysis and training of Neural Networks.
- AdaHessian ↗ - Second order based optimizer for the neural network training based on PyTorch.
- TorchMTL ↗ - Lightweight module to do Multi-Task Learning in PyTorch.
- TRTorch ↗ - PyTorch/TorchScript compiler for NVIDIA GPUs using TensorRT. (Docs ↗)
- Mastering TorchScript: Tracing vs Scripting, Device Pinning, Direct Graph Modification (2020) ↗
- Flow Forecast ↗ - Deep learning PyTorch library for time series forecasting, classification, and anomaly detection.
- SpykeTorch ↗ - High-speed simulator of convolutional spiking neural networks with at most one spike per neuron.
- PyTorch Loss ↗ - Label-smooth, amsoftmax, focal-loss, triplet-loss.
- Cleora ↗ - Faster alternative to PyTorch-BigGraph.
- The Original Transformer (PyTorch) ↗
- PyTorch implementation of the hamburger module from the ICLR 2020 paper “Is Attention Better Than Matrix Decomposition” ↗
- PyTorch Developer Day 2020 ↗
- MLflow and PyTorch integration – autologging and TorchServe plugin (2020) ↗
- HiddenLayer ↗ - Lightweight library for neural network graphs and training metrics for PyTorch, Tensorflow, and Keras.
- LabML ↗ - Monitor PyTorch & TensorFlow model training on mobile phones. (Docs ↗)
- Old Photo Restoration (Official PyTorch Implementation) ↗
- Torch-Pruning ↗ - PyTorch pruning toolkit for structured neural network pruning and layer dependency maintaining.
- torchac ↗ - Fast Arithmetic Coding for PyTorch.
- AdaHessian ↗ - Easy-to-use AdaHessian optimizer (PyTorch).
- PyTorch Dynamic Quantization (2020) ↗
- Performer in PyTorch ↗ - Implementation of Performer, a linear attention-based transformer, in PyTorch.
- GoTch ↗ - Go binding for Pytorch C++ API (libtorch).
- auraloss ↗ - Collection of audio-focused loss functions in PyTorch.
- Normalizing flows with PyTorch ↗ - Implementation and tutorials of normalizing flows with the novel distributions module.
- PyTorch Lightning Snippets ↗
- Stylized Neural Painting (2021) ↗ (PyTorch Code ↗)
- PyTorch Tools, Best Practices & Style Guide ↗
- NeuralProphet ↗ - Neural Network based Time-Series model, inspired by Facebook Prophet and AR-Net, built on PyTorch.
- Pytorch-Lightning Implementation of Self-Supervised algorithms ↗
- Reimplementation of LiteFlowNet using PyTorch ↗
- Julius ↗ - Fast PyTorch based DSP for audio and 1D signals.
- PyTorch Points 3D ↗ - PyTorch framework for doing deep learning on point clouds.
- PyTorch Lightning project template ↗
- TorchData ↗ - PyTorch dataset extended with map, cache etc. (tensorflow.data like).
- pytorch_memlab ↗ - Profiling and inspecting memory in PyTorch.
- NeRF (Neural Radiance Fields) using PyTorch (PyTorch Lightning) ↗
- Faster PyTorch Implementation of Faster R-CNN ↗
- Axial Attention in PyTorch ↗ - Implementation of Axial Attention - attending to multi-dimensional data efficiently.
- torchaudio ↗ - Audio library for PyTorch.
- Benchmark Suite for Deep Learning ↗ - Provides a wrapper to run PyTorch benchmarks using NVidia’s Deep Learning Examples repo.
- Fast Transformers ↗ - PyTorch library for fast transformer implementations.
- PyTorch Quantization Aware Training (2020) ↗
- Unsupervised Learning of Image Segmentation Based on Differentiable Feature Clustering ↗
- WebDataset ↗ - PyTorch Dataset (IterableDataset) implementation providing efficient access to datasets stored in POSIX tar archives and uses only sequential/streaming data access.
- Semantic Segmentation in PyTorch ↗
- torchlambda ↗ - Lightweight tool to deploy PyTorch models to AWS Lambda.
- torchfunc ↗ - PyTorch functions and utilities to make your life easier.
- PyTorch-Reparam-Module ↗ - Reparameterize your PyTorch modules.
- Pytorch Forecasting ↗ - Time series forecasting with PyTorch.
- DALL-E in PyTorch ↗ - Implementation / replication of DALL-E, OpenAI’s Text to Image Transformer, in PyTorch.
- PyTorch Architectures ↗ - PyTorch implementations of research paper model architectures.
- Ways of making PyTorch training faster (2021) ↗
- RepVGG: Making VGG-style ConvNets Great Again (PyTorch) ↗
- PyTorch-OpCounter ↗ - Count the MACs / FLOPs of your PyTorch model.
- PyTorch Memory optimizations via gradient checkpointing ↗
- Efficient PyTorch ↗ - Best practices of training large datasets using PyTorch.
- SAM: Sharpness-Aware Minimization (PyTorch) ↗
- Factorization Machine models in PyTorch ↗
- graftr ↗ - Interactive shell to view and edit PyTorch checkpoints.
- EagerPy ↗ - Writing Code That Works Natively with PyTorch, TensorFlow, JAX, and NumPy. (Web ↗)
- Evidential Deep Learning to Quantify Classification Uncertainty (2018) ↗ (Code ↗)
- PyProf ↗ - GPU performance profiling tool for PyTorch models.
- ctcdecode ↗ - Implementation of CTC (Connectionist Temporal Classification) beam search decoding for PyTorch.
- OpenMM PyTorch Plugin ↗ - OpenMM plugin to define forces with neural networks.
- Totally Versatile Miscellania for PyTorch ↗
- Lightning Flash ↗ - Collection of tasks for fast prototyping, baselining, finetuning and solving problems with deep learning.
- FlashTorch ↗ - Python visualization toolkit, built with PyTorch, for neural networks in PyTorch.
- torchinfo ↗ - View model summaries in PyTorch.
- Compressive transformer PyTorch implementation with notes ↗
- DALL-E ↗ - PyTorch package for the discrete VAE used for DALL·E. (HN ↗)
- PyTorch: From Centralized To Federated (2021) ↗
- PyTorch 1.8, with AMD ROCm support (2021) ↗ (HN ↗)
- Linear Attention Transformer in PyTorch ↗ - Transformer based on a variant of attention that is linear complexity in respect to sequence length.
- GeoTorch ↗ - Constrained optimization toolkit for PyTorch.
- LieTorch: Tangent Space Backpropagation ↗
- FastMoE ↗ - Fast MoE implementation for PyTorch. (Code ↗)
- NN Template ↗ - Generic template to bootstrap your PyTorch project with PyTorch Lightning, Hydra, W&B, DVC, and Streamlit.
- Lightning Hydra Template ↗ - PyTorch Lightning + Hydra. A very general, feature-rich template for rapid, scalable ML experimentation process and best practices.
- Memformer - PyTorch ↗ - Implementation of Memformer, a Memory-augmented Transformer, in PyTorch.
- torchtyping ↗ - Runtime type annotations for the shape, dtype etc. of PyTorch Tensors.
- torchbench ↗ - Easily benchmark machine learning models in PyTorch.
- pytorch-world ↗ - Nix scripts for pytorch-related libraries.
- Butterfly matrix multiplication in PyTorch ↗
- GLOM - PyTorch ↗ - Geoffrey Hinton’s new idea that integrates concepts from neural fields, top-down-bottom-up processing, and attention (consensus between columns) for learning emergent part-whole heirarchies from data.
- nnAudio ↗ - Audio processing by using PyTorch 1D convolution network.
- Lightning-Transformers ↗ - Flexible interface for high performance research using SOTA Transformers leveraging Pytorch Lightning, Transformers, and Hydra.
- PyTorch Tutorial for Deep Learning Researchers ↗
- torchsynth ↗ - Based upon traditional modular synthesis written in PyTorch. It is GPU-optional and differentiable. (Docs ↗)
- PyTorch Tabular ↗ - Standard framework for modelling Deep Learning Models for tabular data.
- PyTorch implementation of EfficientDet ↗
- Transfer Learning for Semantic Segmentation using PyTorch DeepLab v3 ↗
- Class Activation Map methods implemented in PyTorch ↗
- Neural Network Pruning ↗ - PyTorch-based pruning and research.
- Deep Learning with PyTorch Step-by-Step: Beginner’s Guide (2021) ↗ (Code ↗)
- learn2learn ↗ - PyTorch Library for Meta-learning Research.
- PyTorch builds the future of AI and machine learning at Facebook (2021) ↗
- allRank ↗ - Framework for training learning-to-rank neural models based on PyTorch.
- KD_Lib ↗ - PyTorch library to easily facilitate knowledge distillation for custom deep learning models.
- PyTorch Sparse ↗ - PyTorch Extension Library of Optimized Autograd Sparse Matrix Operations.
- train-CLIP ↗ - PyTorch Lightning solution to training CLIP from both scratch and fine-tuning.
- Norse ↗ - Deep learning with spiking neural networks (SNNs) in PyTorch.
- Dense Matching ↗ - Dense matching library based on PyTorch.
- TorchShard ↗ - Slicing a PyTorch Tensor Into Parallel Shards.
- Growing open-source: from Torch to PyTorch (2021) ↗
- PyTorch implementation of “Supervised Contrastive Learning” ↗
- Equinox ↗ - Callable PyTrees and filtered JIT/grad transformations => neural networks in JAX.
- PonderNet: Learning to Ponder ↗ (HN ↗)
- PyNIF3D ↗ - Open-source PyTorch-based library for research on neural implicit functions (NIF)-based 3D geometry representation.
- PyGCL ↗ - Graph Contrastive Learning Library for PyTorch.
- Fast Transformer - PyTorch ↗
- Fast Transformer - TensorFlow ↗
- Bonito ↗ - PyTorch Basecaller for Oxford Nanopore Reads.
- Working through “What is torch.nn really?” (2021) ↗
- PyTorch WideDeep ↗ - Flexible package to combine tabular data with text and images using Wide and Deep models in PyTorch.
- Vector Quantization - PyTorch ↗
- PytorchLightning Tutorials ↗ - Collection of PyTorch lightning tutorial form as rich scripts automatically transformed to ipython notebooks.
- torchkit ↗ - Lightweight library containing PyTorch utilities useful for day-to-day research.
- PyTorch implementation of various Attention Mechanisms, MLP, Re-parameter, Convolution ↗
- stoke ↗ - Lightweight wrapper for PyTorch that provides a simple declarative API for context switching between devices, distributed modes, mixed-precision, and PyTorch extensions.
- pystiche ↗ - Framework for Neural Style Transfer (NST) built upon PyTorch.
- TorchData ↗ - PyTorch repo for data loading and utilities to be shared by the PyTorch domain libraries.
- Efficient Video Dataset Loading and Augmentation in PyTorch ↗
- Accelerate PyTorch models with ONNX Runtime ↗
- Bagua ↗ - Deep learning training acceleration framework for PyTorch.
- TorchMetrics ↗ - Machine learning metrics for distributed, scalable PyTorch applications. (Docs ↗) (Article ↗)
- AutoDL-Projects ↗ - Automated deep learning algorithms implemented in PyTorch.
- PyTorch 1.10 (2021) ↗ (HN ↗)
- TorchX ↗ - Library containing standard DSLs for authoring and running PyTorch related components for an E2E production ML pipeline.
- classy ↗ - PyTorch-based library for fast prototyping and sharing of deep neural network models.
- Probabilistic Torch ↗ - Library for deep generative models that extends PyTorch.
- jax2torch ↗ - Use JAX functions in PyTorch with DLPack.
- Differentiable Neural Computers, Sparse Access Memory and Sparse Differentiable Neural Computers, for Pytorch ↗
- Toroidal ↗ - Lightweight transformer library for PyTorch.
- Ultimate PyTorch Source-Build Template ↗ - Template repository to build PyTorch projects from source on any version of PyTorch/CUDA/cuDNN.
- hf-hub-lightning ↗ - PyTorch Lightning Callback for pushing models to the Hugging Face Hub.
- Audio Augmentations ↗ - Library for PyTorch for audio in the time-domain, with support for stochastic data augmentations.
- e2cnn ↗ - PyTorch extension for equivariant deep learning.
- Deep Learning With PyTorch - Full Course (2021) ↗
- All You Need to Know About PyTorch (2021) ↗
- TorchSnooper ↗ - Debug PyTorch code using PySnooper.
- PyTorch-LIT ↗ - Lite Inference Toolkit (LIT) for PyTorch which focuses on easy and fast inference of large models on end-devices.
- Interactive Deep Colorization in PyTorch ↗
- PyTorch Batched SVD ↗ - 100x faster SVD for PyTorch.
- PyTorch Adapt ↗ - Modular domain adaptation library written in PyTorch.
- Torch Memory-adaptive Algorithms (TOMA) ↗ - Helps you write algorithms in PyTorch that adapt to the available (CUDA) memory.
- Bayesian-Torch ↗ - Bayesian neural network layers for uncertainty estimation.
- PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly) ↗ (HN ↗)
- torchfilter ↗ - Bayesian filters in PyTorch.
- PopTorch ↗ - PyTorch integration for the Graphcore IPU.
- FlowTorch ↗ - PyTorch library for learning and sampling from complex probability distributions using a class of methods called Normalizing Flows.
- Attention is all you need implemented from scratch in PyTorch ↗
- Winners of the 2021 PyTorch Annual Hackathon ↗
- PyTorch vs TensorFlow in 2022 ↗ (HN ↗)
- Perceiver IO ↗ - PyTorch implementation of DeepMind’s Perceiver IO with PyTorch Lightning scripts for distributed training.
- Neural network inferencing for PyTorch and TensorFlow with ONNX, WebAssembly System Interface, and WASI NN (2021) ↗
- hamiltorch ↗ - PyTorch-based library for Riemannian Manifold Hamiltonian Monte Carlo (RMHMC) and inference in Bayesian neural networks.
- Machine Learning with PyTorch and Scikit-Learn Book (2021) ↗ (Tweet ↗) (Code ↗)
- Web Application to generate your training scripts with PyTorch Ignite ↗
- torchinterp1d ↗ - CUDA 1-D interpolation for PyTorch.
- torchquad ↗ - High-performance numerical integration on the GPU with PyTorch.
- Efficient PyTorch: Tensor Memory Format Matters (2021) ↗
- TorchSSL ↗ - PyTorch-based Toolbox for Semi-Supervised Learning.
- PyTea ↗ - PyTorch Tensor Shape Error Analyzer.
- PyTorch Builder ↗ - Scripts to build PyTorch binaries and do end-to-end integration tests.
- PyTorch VAE ↗ - Collection of Variational Autoencoders (VAE) in PyTorch.
- Pytorch MS-SSIM ↗ - Fast and differentiable MS-SSIM and SSIM for PyTorch.
- PyTorch Live ↗ - Easy to use library of tools for creating on-device ML demos on Android and iOS.
- Modeling uncertainty with PyTorch (2022) ↗
- PyTorch distributions examples ↗
- moolib ↗ - Library for distributed ML training with PyTorch.
- v objective diffusion inference code for PyTorch ↗
- Texar-PyTorch ↗ - Integrating the Best of TF into PyTorch, for Machine Learning, Natural Language Processing, and Text Generation.
- solo-learn ↗ - Library of self-supervised methods for unsupervised visual representation learning powered by PyTorch Lightning.
- torch-max-mem ↗ - Decorators for maximizing memory utilization with PyTorch & CUDA.
- PyTorch implementation of various Knowledge Distillation (KD) methods ↗
- Padertorch ↗ - Designed to simplify the training of deep learning models written with PyTorch. Focusing on speech and audio processing.
- VFormer ↗ - Modular PyTorch library for Vision Transformers.
- BindsNET ↗ - Simulation of spiking neural networks (SNNs) using PyTorch.
- Hammer ↗ - Efficient PyTorch-based library for training deep models.
- (Generic) EfficientNets for PyTorch ↗ - Pretrained EfficientNet, EfficientNet-Lite, MixNet, MobileNetV3 / V2, MNASNet A1 and B1, FBNet, Single-Path NAS.
- sbi ↗ - PyTorch package for simulation-based inference.
- torchmd ↗ - PyTorch differentiable molecular dynamics.
- TorchCAM ↗ - Simple way to leverage the class-specific activation of convolutional layers in PyTorch.
- torch2trt ↗ - PyTorch to TensorRT converter which utilizes the TensorRT Python API.
- PyTorch Toolbox for Image Quality Assessment ↗
- Human Segmentation PyTorch ↗ - Human segmentation models, training/inference code, and trained weights, implemented in PyTorch.
- Breaching ↗ - Framework for Attacks against Privacy in Federated Learning in PyTorch.
- PyTorchViz ↗ - Small package to create visualizations of PyTorch execution graphs.
- PyTorch on Apple M1 MAX GPUs with SHARK – faster than TensorFlow-Metal (2022) ↗ (HN ↗)
- Deep Robust ↗ - PyTorch adversarial library for attack and defense methods on images and graphs.
- TorchRec ↗ - Library for modern production recommendation systems. (Article ↗) (HN ↗)
- TorchStudio ↗ - IDE for PyTorch. (Web ↗) (Beginner’s Guide ↗)
- sk2torch ↗ - Convert scikit-learn models to PyTorch modules.
- Multi-Task Learning in PyTorch ↗
- PyTorch C++ Samples ↗ - Deep Learning sample programs of PyTorch written in C++.
- SparseLinear ↗ - PyTorch package that allows a user to create extremely wide and sparse linear layers efficiently.
- Complete Guide of Swin Transformer with Full PyTorch Implementation (2022) ↗
- Kernel Point Convolution implemented in PyTorch ↗
- executorch ↗ - Simple and portable executor of PyTorch programs.
- Image 2 Vec with PyTorch ↗
- Tez ↗ - Super-simple and lightweight Trainer for PyTorch.
- TorchMultimodal ↗ - PyTorch library for training state-of-the-art multimodal multi-task models at scale.
- MiniTorch Module 0 ↗
- baukit ↗ - Kit of David’s secret tools to help with productive research prototyping with PyTorch.
- Cyclemoid PyTorch ↗ - Implementation of the cyclemoid activation function for PyTorch.
- Graph Embedding ↗ - Reimplementation of Graph Embedding methods by PyTorch.
- Distributed PyTorch Lightning Training on Ray ↗
- Lightning Pose ↗ - Accelerated pose estimation and tracking using semi-supervised convolutional networks.
- Disent ↗ - Modular VAE disentanglement framework for python built with PyTorch Lightning.
- MQBench ↗ - Open-source model quantization toolkit based on PyTorch fx.
- Self-supervised learning tutorial: Implementing SimCLR with PyTorch lightning (2022) ↗ (Code ↗)
- PyTracking ↗ - Visual tracking library based on PyTorch.
- TorchDynamo ↗ - Python-level JIT compiler designed to make unmodified PyTorch programs faster.
- Einops Extensions ↗ - Implementation of some personal helper functions for Einops.
- TorchOpt ↗ - High-performance optimizer library built upon PyTorch for easy implementation of functional optimization and gradient-based meta-learning.
- PyTorch Connectomics ↗ - Segmentation toolbox for EM connectomics.
- TedNet ↗ - PyTorch Toolkit for Tensor Decomposition Networks.
- TorchData: PyTorch Composable Data loading utility library (2022) ↗
- Equivariant Steerable CNNs Library for PyTorch ↗
- PyTorch C++ inference with LibTorch ↗
- pl_cross ↗ - Cross validation in PyTorch lightning made easy.
- diffsptk ↗ - Differentiable version of SPTK based on the PyTorch framework.
- minDiffusion ↗ - Self-contained, minimalistic implementation of diffusion models with PyTorch.
- torchdistX ↗ - Torch Distributed Experimental.
- grafog ↗ - Graph Data Augmentation Library for PyTorch Geometric.
- High-fidelity performance metrics for generative models in PyTorch ↗
- Denoising Diffusion Probabilistic Models in PyTorch ↗
- PyTorch Geometric Install Helper ↗
- PyTorch Benchmarks ↗ - Collection of open source benchmarks used to evaluate PyTorch performance.
- Torchprofile ↗ - General and accurate MACs / FLOPs profiler for PyTorch models.
- K Means using PyTorch ↗
- surgeon-pytorch ↗ - Library to inspect itermediate layers of PyTorch models. (Reddit ↗)
- PyMDE ↗ - Minimum-distortion embedding with PyTorch.
- torchdistill ↗ - Modular, Configuration-Driven Framework for Knowledge Distillation.
- PyTorch on Angel ↗ - Arming PyTorch with a powerful Parameter Server, which enable PyTorch to train very big models.
- NITorch ↗ - Library written in PyTorch aimed at medical image processing and analysis, with a focus on neuroimaging.
- torch-ppr ↗ - Page-Rank computation using PyTorch.
- PyTorch-Energizer ↗ - Active learning library for PyTorch based on PyTorch-Lightning.
- Torchshow ↗ - Visualize PyTorch tensor in one-line of code.
- Accelerated PyTorch Training on M1 Mac (2022) ↗ (HN ↗) (Reddit ↗)
- Metatorch ↗ - Library and framework for prototyping PyTorch models with Haskell.
- Running PyTorch on the M1 GPU (2022) ↗ (HN ↗)
- PyTorch Topological ↗ - Topological machine learning framework based on PyTorch.
- Optimizers ↗ - PyTorch optimization algorithms.
- MinT ↗ - Minimal Transformer Library and Tutorials.
- Apple Neural Engine (ANE) Transformers ↗ - Reference implementation of the Transformer architecture optimized for Apple Neural Engine (ANE.
- Taking Datasets, DataLoaders, and PyTorch’s New DataPipes for a Spin (2022) ↗ (Code ↗)
- Lightning EcoSystem CI ↗ - Automated Testing for Lightning EcoSystem Projects.
- Run PyTorch models in the browser using ONNX.js ↗
- torchsnapshot ↗ - Light-weight library for adding fault tolerance to large-scale PyTorch distributed training workloads.
- PyTorch Project Template ↗
- PyTorch Model Compare ↗ - Tiny package to compare two neural networks in PyTorch.
- PyEPO (PyTorch-based End-to-End Predict-and-Optimize Tool) ↗ - Python-based, open-source software that supports modeling and solving predict-and-optimize problems with the linear objective function.
- Parti - PyTorch ↗ - Implementation of Parti, Google’s pure attention-based text-to-image neural network, in PyTorch.
- OpenMMLab ↗ - Open source optical flow toolbox based on PyTorch.
- simpleT5 ↗ - Quickly train T5/mT5/byT5 models in just 3 lines of code.
- ngp_pl ↗ - Instant-ngp (only NeRF) in pytorch+cuda trained with pytorch-lightning (high quality with high speed).