Machine learning

Phil Wang ↗ always recreates many cutting edge ML papers with PyTorch. This course ↗ & Math for ML ↗ seem great. Hugging Face ↗ is incredible community & tools ↗. Transformer architectures ↗ & Diffusion Models ↗ are great. MLU-Explain ↗ is a great ML visual explainer ↗.

Looking into using envd ↗ together with PyTorch Lightning ↗ for my ML experiments.

Notes#

• A big part of the utility of math (especially in ML) is having breadth rather than depth. The strategy of picking out specific things you don’t know from papers and looking them up is only effective if you have the breadth in your background to understand the answers you find.
  • Broad knowledge is also what helps you manage the exponential tree of complexity you’re encountering.
    • You won’t have seen all the things you come across, but you’ll develop the ability to make good judgements about what you need to read to achieve your goals. You’ll learn how to recognize when a reference you’re reading is more (or less) technical than you need, and how to search for something more appropriate. You’ll also learn how and when you can use results without understanding the details.
  • Finally, as a general grad student strategy trying to learn everything just in time is not a path to success. Even if you had the perfect math oracle that you want it would be setting you up to be left behind. All the oracle gives you is the ability to catch up quickly to the ideas of others. Your job as a grad student is to generate new knowledge and to do that you need to seek things out on your own, not just follow along the latest trend. Part of your job is to go out hunting for ideas that your peers haven’t found yet and bring them back to your field.
• In supervised learning, you have a bunch of data, a specific question you want to answer, and access to the correct answer to many instances of that question. In unsupervised learning, you have a bunch of data points, and you want to find meaningful patterns in the structure of that data. In reinforcement learning, you have a task you want to take actions to accomplish, and you don’t have any access to knowing what the best action is, but after each action you get a rough idea of how good the result was. ↗
• AI doesn’t need to follow the human model, just like planes don’t need to flap their wings like a bird. For most jobs AI will be very different from humans. Even when AI acts as human for entertainment I would imagine them being very different internally, as their job is to mimic aspects of human behaviors, not actually a human as a whole.
• Almost all of machine learning is about representing data as vectors and performing linear and non-linear transformations in order to perform classification, regression, etc.
• Most of ML is fitting models to data. To fit a model you minimize some error measure as a function of its real valued parameters, e.g. the weights of the connections in a neural network. The algorithms to do the minimization are based on gradient descent, which depends on derivatives, i.e. differential calculus.
• Learn optimization before studying machine learning if you really want to understand what’s going on. ↗
• What idiot called it “machine learning” instead of “bias automation”. ↗
• If you were to learn only 1 method for explaining machine learning models, it should be Shapley values (SHAP): 1. Model-agnostic: Use with any model. 2. Theoretic foundation: Game theory. 3. Good software ecosystem. 4. Local and global explanations. ↗
• What idiot called it “machine learning” instead of “bias automation”. ↗

Links#

• Neural Networks and Deep Learning book ↗
• Deep Learning Papers Reading Roadmap ↗
• Ask HN: Best way to get started with AI? ↗
• Ask HN: What maths are critical to pursuing ML/AI? ↗
• Ask HN: ‘Crash Courses’ for Mathematics Related to DL, ML and Data Analysis ↗
• Computational Statistics and Machine Learning Revision Notes ↗
• Stanford CS229 course ↗
• Readings in applied data science ↗
• Learn ML in 3 months ↗
• Deep Learn ↗ - Implementation of research papers on Deep Learning+ NLP+ CV in Python using Keras, TensorFlow and Scikit Learn.
• Building Brundage Bot ↗
• Summaries of ML papers ↗
• Code and data for paper “Deep Painterly Harmonization” ↗
• FB AI Tools ↗
• Best Practices for ML Engineering ↗
• Machine Learning From Scratch ↗
• Dive Into ML ↗
• Fermat’s Library NIPS comments ↗
• Heroes of Deep Learning: Andrew Ng interviews Ian Goodfellow ↗
• Machine Learning for Humans ↗ - Great article.
• Deep Learning for Siri’s Voice: On-device Deep Mixture Density Networks for Hybrid Unit Selection Synthesis ↗
• The Google Brain Team — Looking Back on 2017 ↗
• Building the Software 2.0 Stack by Andrej Karpathy from Tesla (2018) ↗
• Deep Learning World ↗
• Machine Learning cheatsheets for Stanford’s CS 229 ↗
• RAAIS 2018 - François Chollet (Creator of Keras) ↗
• KubeFlow ↗ - Machine Learning Toolkit for Kubernetes. (Winding Road to Better Machine Learning Infrastructure Through Tensorflow Extended and Kubeflow ↗)
• KALE (Kubeflow Automated pipeLines Engine) ↗ - Aims at simplifying the Data Science experience of deploying Kubeflow Pipelines workflows.
• MIT AGI: Deep Learning (Yoshua Bengio) (2018) ↗
• TL-GAN: transparent latent-space GAN ↗ - Use supervised learning to illuminate the latent space of GAN for controlled generation and edit.
• Grokking Deep Learning ↗ - Repository accompanying “Grokking Deep Learning” book.
• HN: Can we rule out near-term AGI? (2018) ↗
• Introduction to Grenade (Haskell library for Deep Learning) ↗
• Grenade ↗ - Deep Learning in Haskell.
• Deep Learning 1: Introduction to Machine Learning Based AI ↗
• Deep Learning cheatsheets for Stanford’s CS 230 (2018) ↗
• Deep Learning Book Chapter Summaries ↗ - Attempting to make the Deep Learning Book easier to understand.
• PracticalAI ↗ - Practical approach to learning machine learning.
• Ask HN: How to incorporate machine learning into day job? (2018) ↗
• RLgraph ↗ - Flexible computation graphs for deep reinforcement learning.
• Nevergrad ↗ - Gradient-free optimization platform.
• Machine Learning Cheat Sheet ↗
• GANs and Divergence Minimization (2018) ↗
• Convolution arithmetic ↗ - Technical report on convolution arithmetic in the context of deep learning.
• FloydHub ↗ - Managed cloud platform for data scientists.
• Style Transfer as Optimal Transport ↗ - Algorithm that transfers the distribution of visual characteristics, or style, of a reference image onto a subject image via an Optimal Transport plan.
• Looking Back at Google’s Research Efforts in 2018 ↗
• Recommenders ↗ - Examples and best practices for building recommendation systems, provided as Jupyter notebooks.
• Deep Learning State of the Art (2019) - MIT ↗
• AdaNet ↗ - Lightweight TensorFlow-based framework for automatically learning high-quality models with minimal expert intervention.
• DAWNBench ↗ - Benchmark suite for end-to-end deep learning training and inference.
• Interpretable Machine Learning (2022) ↗ - Guide for Making Black Box Models Explainable. (Code ↗) (2nd edition ↗)
• All You Need to Know About Deep Learning - A kick-starter (2019) ↗
• KubeFlow Pipelines ↗ - Machine learning (ML) toolkit that is dedicated to making deployments of ML workflows on Kubernetes simple, portable, and scalable.
• Summary of some ML papers ↗
• Practical Deep Learning for Coders 2019 ↗ - (HN ↗) (GitHub ↗)
• Notebooks for the “A walk with fastai2” Study Group and Lecture Series ↗
• The Matrix Calculus You Need For Deep Learning ↗ (HN ↗) (HN 2 ↗) (Paper ↗)
• Machine Learning Feynman Experience ↗ - Collection of concepts I tried to implement using only Python, NumPy and SciPy on Google Colaboratory.
• Tensor2Tensor ↗ - Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.
• Deep learning drizzle ↗ - Various ML, reinforcement learning video lectures. (Code ↗)
• Xfer ↗ - Transfer Learning library for Deep Neural Networks.
• List of summer schools in machine learning + related fields ↗
• Most Cited Deep Learning Papers ↗
• Machine Learning CS Columbia Course (2019) ↗
• Learning to Discover Efficient Mathematical Identities ↗ - Exploring how machine learning techniques can be applied to the discovery of efficient mathematical identities.
• CleverHans ↗ - Adversarial example library for constructing attacks, building defenses, and benchmarking both.
• Google AI Research ↗ - Contains code released by Google AI Research.
• Machine Learning Mindmap / Cheatsheet ↗
• Curated list of network embedding techniques ↗
• Deploying Deep Learning ↗ - Training guide for inference and deep vision runtime library for NVIDIA DIGITS and Jetson Xavier/TX1/TX2.
• “Adversarial Machine Learning” with Ian Goodfellow (2018) ↗
• HN: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award (2019) ↗
• Large scale K-means and K-nn implementation on NVIDIA GPU / CUDA ↗
• fairseq ↗ - Sequence-to-sequence learning toolkit for Torch from Facebook AI Research tailored to Neural Machine Translation (NMT).
• TinyFlow ↗ - Tutorial code on how to build your own Deep Learning System in 2k Lines.
• Deep Learning Models ↗ - Collection of various deep learning architectures, models, and tips.
• Multi-Level Intermediate Representation Overview ↗ - MLIR project aims to define a common intermediate representation (IR) that will unify the infrastructure required to execute high performance machine learning models in TensorFlow and similar ML frameworks. (Talks ↗) (HN ↗) (Slides ↗)
• PySparNN ↗ - Approximate Nearest Neighbor Search for Sparse Data in Python.
• Machine Learning Course with Python ↗
• ICML ↗ - International Conference on Machine Learning.
• Integrating Domain Knowledge into Deep Learning by Ruslan Salakhutdinov (2019) ↗
• Differentiation for Hackers ↗ - The goal of this handbook is to demystify algorithmic differentiation, the tool that underlies modern machine learning.
• Homemade Machine Learning ↗
• Machine Learning Systems are Stuck in a Rut (2019) ↗ (HN ↗)
• Literature of Deep Learning for Graphs ↗
• Supplementary Materials for “Unsupervised word embeddings capture latent knowledge from materials science literature”, Nature ↗
• Solution of assignment in 2011 Stanford Machine Learning Class ↗
• Autocompletion with deep learning (2019) ↗
• Rules of Machine Learning: Best Practices for ML Engineering ↗
• Python Machine Learning Book (2019) ↗ (3rd edition code ↗) (2nd edition code ↗)
• ML and DS Applications in Industry ↗ - Curated list of applied machine learning and data science notebooks and libraries across different industries.
• Awesome production machine learning ↗
• Awesome Gradient Boosting Research Papers ↗
• HoloClean ↗ - Machine Learning System for Data Enrichment.
• Snorkel ↗ - System for quickly generating training data with weak supervision.
• RAdam ↗ - On The Variance Of The Adaptive Learning Rate And Beyond.
• Google ML/AI Comic ↗
• Machine Learning Notebooks ↗ - Series of Jupyter notebooks that walk you through the fundamentals of Machine Learning and Deep Learning in Python using Scikit-Learn, Keras and TensorFlow 2.
• Streamlit ↗ - Fastest way to build custom ML tools. (Web ↗) (Awesome Streamlit ↗) (Streamlit Cheat Sheet ↗) (Tips, tricks, methods, and techniques for building apps with streamlit ↗) (Best of Streamlit ↗)
• A Gentle Introduction to Bayes’ Theorem for Machine Learning (2019) ↗ (HN ↗)
• Practical Deep Learning for Coders ↗ (Notes ↗)
• Part 2: Deep Learning from the Foundations ↗
• Computational Linear Algebra for Coders ↗
• Introduction to Machine Learning for Coders ↗
• Papers with Code ↗ - The latest in machine learning. (HN ↗)
• Gradient Descent Derivation (2014) ↗
• Best of Machine Learning Newsletter ↗
• TASO ↗ - Tensor Algebra SuperOptimizer for Deep Learning.
• An Exponential Learning Rate Schedule for Deep Learning (2019) ↗
• Billion-scale semi-supervised learning for state-of-the-art image and video classification (2019) ↗
• TRAINS ↗ - Auto-Magical Experiment Manager & Version Control for AI.
• Differentiable Optimization-Based Modeling for Machine Learning (2019) ↗
• Notebooks and code for the book “Introduction to Machine Learning with Python” ↗
• Awesome Deep Learning Project Ideas ↗
• Top-down, practical guide to learn AI, Deep learning and Machine Learning ↗
• Most Aesthetically Pleasing ML Research Papers ↗
• Polyaxon ↗ - Platform for reproducible and scalable machine learning and deep learning on Kubernetes. (GitHub ↗) (Code ↗)
• Different projects built using fast.ai ↗
• Spell ↗ - Fastest and most powerful end-to-end platform for machine learning and deep learning.
• ML portfolio tips (2019) ↗
• DeepMind Research ↗ - Contains implementations and illustrative code to accompany DeepMind publications.
• Deep Learning Tutorials ↗
• Prodify ↗ - Radically efficient machine teaching. An annotation tool powered by active learning.
• Runway ↗ - Professional video editing powered by machine learning — all on the web. (HN ↗)
• An Extended Version Of The Scikit-Learn Cheat Sheet (2014) ↗
• Notes on Machine Learning ↗
• Notes on Deep Learning ↗
• Awesome free deep learning papers ↗
• Teachable Machine ↗ - Fast, easy way to create machine learning models for your sites, apps, and more – no expertise or coding required.
• Deep Learning Interview Topics ↗
• Ask HN: Why do so many startups claim machine learning is their long game? (2019) ↗
• End-to-End Machine Learning Courses ↗
• End-to-End Machine Learning Library ↗
• Course material for STAT 479: Machine Learning (FS 2019) taught by Sebastian Raschka at University Wisconsin-Madison ↗
• Clipper ↗ - Prediction serving system that sits between user-facing applications and a wide range of commonly used machine learning models and frameworks.
• AI building blocks - from scratch with Python (2018) ↗
• ARUBA: Learning-to-Learn with Less Regret (2019) ↗
• Machine Learning Systems Design ↗
• Guide to Production Level Deep Learning ↗
• Lessons learned building an ML trading system that turned $5k into$200k (2019) ↗ (HN ↗)
• Deep Learning for Programmers ↗
• Arcade Learning Environment ↗ - Simple object-oriented framework that allows researchers and hobbyists to develop AI agents for Atari 2600 games.
• Space to discuss the future of the ML ecosystem in Rust ↗
• Awesome System for Machine Learning ↗
• Collection of explainer tutorials on how machine learning and statistical concepts work ↗
• 2019’s Top Machine and Deep Learning Research Papers ↗ (HN ↗)
• NeurIPS 2019 Schedule ↗
• Machine Learning Crash Course with TensorFlow APIs ↗ - Google’s fast-paced, practical introduction to machine learning.
• What was your favorite paper of 2019 and why? (2019) ↗
• Ask HN: Full-on machine learning for 2020, what are the best resources? ↗
• Dive into Deep Learning ↗ - Interactive deep learning book with code, math, and discussions, based on the NumPy interface. (HN ↗) (Code ↗)
• Resources to learn more about Machine Learning and Artificial Intelligence ↗
• Foundations of Machine Learning book ↗ - New edition of a graduate-level machine learning textbook that focuses on the analysis and theory of algorithms.
• Deep Learning book ↗ - Resource intended to help students and practitioners enter the field of machine learning in general and deep learning in particular. (Code ↗) (Notes ↗) (Exercises ↗) (LaTeX files for book notation ↗) (PDF ↗) (PDF 2 ↗)
• Introduction to Deep Learning - Eugene Charniak ↗ - Project-based guide to the basics of deep learning.
• Hands-On Machine Learning with Scikit-Learn, Keras, and Tensorflow: Concepts, Tools, and Techniques to Build Intelligent Systems ↗
• Topological Techniques for Unsupervised Learning (2019) ↗
• Meet AdaMod: a new deep learning optimizer with memory (2020) ↗
• Deep Learning State of the Art (2020) ↗
• The Case for Bayesian Deep Learning (2020) ↗ (HN ↗) (Reddit ↗)
• Machine Learning Summer School (2020) ↗
• Machine Learning Summer School videos (2009) ↗
• Turi Create ↗ - Simplifies the development of custom machine learning models.
• Private machine learning progress ↗
• Demucs ↗ - Code for the paper Music Source Separation in the Waveform Domain.
• Apple at NeurIPS 2019 ↗
• Magenta ↗ - Make Music and Art Using Machine Learning. (JS Code ↗) (GitHub ↗)
• An overview of gradient descent optimization algorithms (2016) ↗
• What are the current significant trends in ML that are NOT Deep Learning related? (2020) ↗
• Trax ↗ - Helps you understand and explore advanced deep learning.
• ATMSeer: Increasing Transparency and Controllability in Automated Machine Learning ↗ (Code ↗)
• Cambridge Machine Learning Group ↗
• Convolutional Conditional Neural Processes (2020) ↗
• Privacy Preserving AI (Andrew Trask) (2020) ↗
• Emil’s Story as a Self-Taught AI Researcher (2020) ↗
• Humans of Machine Learning ↗
• Machine Learning Flashcards ↗ (HN ↗)
• Awesome Software Engineering for Machine Learning ↗
• Awesome Data Labeling ↗ - Curated list of awesome data labeling tools.
• Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence (2020) ↗
• fastai: A Layered API for Deep Learning (2020) ↗ (HN ↗)
• The Deep Learning Compiler: A Comprehensive Survey (2020) ↗
• Perceptrons explained ↗
• A Simple Framework for Contrastive Learning of Visual Representations (2020) ↗ (Tweet ↗) (Code ↗)
• MediaPipe ↗ - Cross-platform framework for building multimodal applied machine learning pipelines.
• ML courses by Zico Kolter ↗
• List of AI Residency Programs ↗
• MIT Introduction to Deep Learning course (2020) ↗ (Code ↗) (Videos ↗)
• Stanford Deep Multi-Task and Meta Learning course (2019) ↗
• FastAI book draft (2020) ↗ (HN ↗) (Notebooks ↗)
• Pattern Recognition and Machine Learning by Christopher M. Bishop Book ↗ (Code ↗) (PDF ↗) (Code/Notes ↗)
• Google Colaboratory ↗ - Research project created to help disseminate machine learning education and research.
• Resources for teaching machine learning ↗
• Population Based Augmentation ↗ - Algorithm that quickly and efficiently learns data augmentation functions for neural network training.
• NVIDIA Deep Learning Examples for Tensor Cores ↗
• Introduction to Deep Learning and Generative Models course ↗ (Code ↗)
• Collection of Conference & School Notes in Machine Learning ↗
• AutoML-Zero ↗ - Open source code for the paper: “AutoML-Zero: Evolving Machine Learning Algorithms From Scratch”. (HN ↗)
• fastAI course v4 ↗
• Ask HN: What is your ML stack like? (2020) ↗
• MLflow ↗ - Open source platform to manage the ML lifecycle, including experimentation, reproducibility and deployment. (Code ↗)
• Deep Unsupervised Learning (2020) ↗
• Machine Learning: a Probabilistic Perspective book ↗ (Code ↗)
• Introduction to Machine Learning - Carnegie Mellon University (2019) ↗
• Notes on ML courses ↗
• Made With ML ↗ - Share what you’ve Made With ML. (Code ↗) (MLOps Course ↗)
• Backpropagation 101 (2020) ↗ - How to trick yourself into understanding backprop without even trying.
• A Spacetime Approach to Generalized Cognitive Reasoning in Multi-scale Learning (2017) ↗
• Open Source Deep Learning Glossary ↗
• Awesome Graph Classification ↗ - Collection of important graph embedding, classification and representation learning papers with implementations.
• fast.ai ↗ - Making neural nets uncool again. (Code ↗) (Docs ↗) (Course launch ↗) (HN ↗)
• Interactive Tools for ML, DL and Math ↗
• Deep learning with graph-structured representations (2020) ↗
• Interactive Machine Learning Experiments ↗ (Web ↗)
• AxCell: Automatic Extraction of Results from Machine Learning Papers (2020) ↗
• Awesome Machine Learning and AI Courses ↗
• How do you keep up with new research? (2020) ↗
• OpenLTH: A Framework for Lottery Tickets and Beyond ↗
• Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning ↗
• Awesome Interpretable Machine Learning ↗
• DLPack: Open In Memory Tensor Structure ↗
• SVM tutorial ↗ (HN ↗)
• DeepMind x UCL | Deep Learning Lecture Series (2020) ↗
• Nicholas Carlini: Making and Measuring Progress in Adversarial Machine Learning (2019) ↗
• Deep Learning in Production ↗ - Notes and references about deploying deep learning-based models in production.
• Stanford Class on Deep Multi-Task and Meta-Learning (2019) ↗ (HN ↗)
• Weights & Biases ↗ - Developer tools for ML. Experiment tracking, hyperparameter optimization, model and dataset versioning. (Code ↗) (Docs ↗) (Examples ↗) (Community ↗)
• Protocols and Structures for Inference (PSI) spec ↗ - Aims to develop an architecture for presenting machine learning algorithms, their inputs (data) and outputs (predictors) as resource-oriented RESTful web services.
• ML Engineer Roadmap ↗
• Machine Learning for Everyone - In simple words. With real-world examples. ↗
• Deep learning на пальцах ↗ (Code ↗)
• How to add AI to your app without knowing anything about AI (2020) ↗
• What I learned from looking at 200 machine learning tools (2020) ↗ (HN ↗)
• Interactive Machine Learning List ↗ (Code ↗)
• Sema ↗ - Live Code Language Design Playground. (Code ↗)
• Model Zoo ↗ - Deploy your model with a single line of code. (HN ↗)
• Introduction to Machine Learning with Julia ↗
• Reverb ↗ - Efficient and easy-to-use data storage and transport system designed for machine learning research.
• System design patterns for machine learning ↗
• Income strategies to support your ML research (2020) ↗
• Synthetic Data for Deep Learning (2019) ↗
• NeurIPS ↗ - Conference on Neural Information Processing Systems.
• Machine Learning Mastery ↗
• Distill ↗ - Latest articles about machine learning.
• Compute access for ML training (2020) ↗
• Scann: Scalable Nearest Neighbors ↗ (HN ↗)
• CML ↗ - Continuous Machine Learning: Bring DevOps to Data Science. (HN ↗) (Code ↗) (Web Code ↗)
• TensorFlow, Keras and deep learning, without a PhD ↗ (HN ↗)
• Apple Machine Learning Research ↗ (HN ↗)
• Model Card Toolkit ↗ - Streamlines and automates generation of Model Cards, machine learning documents that provide context and transparency into a model’s development and performance. (Article ↗)
• Top ML Books recommended by experts ↗
• Best practices for performance and cost optimization for machine learning ↗
• Bethge Lab ↗ - Perceiving Neural Networks.
• Graphcore code examples ↗
• Machine learning examples and tutorials ↗
• Vintage Factor Analysis with Varimax Performs Statistical Inference (2020) ↗
• An Opinionated Guide to ML Research (2020) ↗
• Amazon’s Machine Learning University ↗ (HN ↗)
• Progress, Notes, Summaries and a lot of Questions on Machine Learning ↗
• SciML ↗ - Open Source Software for Scientific Machine Learning.
• Mihaela Rosca ML notes ↗
• Compression of Deep Learning Models for Text: A Survey (2020) ↗
• The Computational Limits of Deep Learning (2020) ↗ (HN ↗)
• ML Ops: Machine Learning Operations ↗ (Awesome ↗) (References ↗)
• Benchmarks of approximate nearest neighbor libraries in Python ↗ (Web ↗)
• MIT Deep Learning course ↗
• Visualizing Deep Learning (2020) ↗
• Books for Machine Learning, Deep Learning, and related topics ↗
• Graph Representation Learning Book ↗ (Reddit ↗)
• Effective testing for machine learning systems (2020) ↗ (HN ↗)
• Machine Learning from Scratch ↗ (HN ↗) (Code ↗)
• Compose ↗ - Machine learning tool for automated prediction engineering. It allows you to structure prediction problems and generate labels for supervised learning.
• Daily scikit-learn tips ↗
• Applied ML ↗ - Curated papers, articles, and blogs on data science & machine learning in production.
• Understanding Convolution in Deep Learning (2015) ↗
• 21 Habits I Picked Up While Learning Machine Learning (2019) ↗
• Hacker News for ML ↗
• Think Fast: Tensor Streaming Processor for Accelerating Deep Learning Workloads (2020) ↗
• My Deep Learning Toolchain (2020) ↗
• igel ↗ - Machine learning tool that allows you to train/fit, test and use models without writing code. (HN ↗)
• Gradient Boosted Decision Trees (2020) ↗ (HN ↗)
• NeurIPS 2020 Accepted Papers ↗
• explained.ai ↗ - Deep explanations of machine learning and related topics.
• How to visualize decision trees (2018) ↗
• How to explain gradient boosting (2018) ↗
• Beware Default Random Forest Importances (2018) ↗
• The Mechanics of Machine Learning ↗
• Yann LeCun Spring 2020 DL Course (Videos, Slides, Jupyter Notebooks) ↗ (HN ↗) (Article ↗)
• Yann LeCun Deep Learning Course 2021 ↗ (HN ↗)
• ML Guide: Feature Store vs Data Warehouse (2020) ↗ (HN ↗)
• Grid AI ↗ - Seamlessly train hundreds of Machine Learning models on the cloud from your laptop. (HN ↗)
• Getting Started with Applied ML Research (2020) ↗
• Machine Learning Engineering book by Andriy Burkov ↗
• Determined ↗ - Deep Learning Training Platform. (Web ↗)
• Phasic Policy Gradient ↗
• Juergen Schmidhuber ML reading list (2015) ↗
• Machine Learning Primer for Interviews ↗
• Confetti AI ↗ - Ace Your Machine Learning Interviews.
• Intro to Deep Learning: Use TensorFlow and Keras to build and train neural networks by Kaggle ↗
• Manifold ↗ - Model-agnostic visual debugging tool for machine learning. (Web ↗)
• Making With ML YouTube series ↗
• Topological Autoencoders (2020) ↗ (Code ↗)
• Awesome Teachable Machine List ↗ - Curated list of awesome machine learning projects built with Google’s Teachable Machine.
• How to put machine learning models into production (2020) ↗
• Example Machine Learning Scripts for Numerai’s Tournament (2020) ↗
• Synthetic Data Vault (SDV) ↗ - Synthetic Data Generation for tabular, relational, time series data. (Web ↗)
• Penn Machine Learning Benchmarks ↗ - Large, curated repository of benchmark datasets for evaluating supervised machine learning algorithms. (Web ↗)
• 2020 Machine Learning Roadmap ↗
• Responsible Machine Learning ↗ - Collection of tools for eXplainable AI (XAI). (Web ↗)
• MI2 DataLab ↗ - Group of mathematicians and computer scientists that love to play with data. (GitHub ↗)
• Papers of Robust ML ↗ - Mainly focus on defenses.
• Why Deep Learning Works Even Though It Shouldn’t (2020) ↗ (Lobsters ↗)
• Some Notable Recent ML Papers and Future Trends (2020) ↗
• Wiki: 2020 ML Interviews Resources & Advice(s) ↗
• Interpretable Machine Learning — A Brief History, State-of-the-Art and Challenges (2020) ↗
• Over 200 of the Best Machine Learning, NLP, and Python Tutorials (2018) ↗
• What is a Feature Store? (2020) ↗
• Awesome AutoML Papers ↗ - Curated list of automated machine learning papers, articles.
• Machine Learning Systems Design at Stanford course (2020) ↗
• Preferred Networks ↗ - Develops practical applications of deep learning and other cutting-edge technologies. (GitHub ↗)
• ML Art ↗ - Curated showcase of creative machine learning artworks and projects.
• Decision Making under Uncertainty course ↗ (Algorithms for Decision Making book ↗) (Concise Deep Learning Overview (2020) ↗ (HN ↗)) (Book Website ↗)
• Effective testing for machine learning systems (2020) ↗
• Reading List for Topics in Multimodal Machine Learning ↗
• Awesome Multimodal Research ↗
• ML and DL related contests, competitions and conference challenges ↗
• ML Visuals ↗ - Contains figures and templates which you can reuse and customize to improve your scientific writing.
• DL Visuals ↗ - Deep Learning Visuals.
• Deep Learning with Catalyst course ↗
• OpenMined Courses ↗ - Learn how privacy technology is changing our world and how you can lead the charge.
• Adversarial ML Threat Matrix ↗
• DeepMind Educational Resources ↗
• Deep Learning (for Audio) with Python ↗ (Code ↗)
• Awesome Tensor Compilers ↗
• Adversarial Robustness Toolbox ↗ - Python library for Machine Learning Security.
• AI Summer ↗ - Learn Deep Learning and Artificial Intelligence. (GitHub ↗)
• How Attention works in Deep Learning (2020) ↗
• Brain Tokyo Workshop ↗ - Research materials released by members of the Google Brain team in Tokyo.
• A Novel Framework for Explaining Machine Learning Using Shapley Values (2020) ↗ (HN ↗)
• create-ml-app ↗ - Template Makefile for ML projects in Python.
• telesto.ai ↗ - Competitive marketplace, where you can work on real-life machine learning challenges.
• ML from the Fundamentals ↗ - Machine learning in a “from the first principles” style. (Code ↗)
• Interpretability in Machine Learning: An Overview (2020) ↗
• Implicit Rank-Minimizing Autoencoder (2020) ↗ (Code ↗)
• ML/CV/NLP study resources ↗
• MIT Mądry Lab ↗ - Towards a Principled Science of Deep Learning. (GitHub ↗)
• Scaling Down Deep Learning (2020) ↗ (HN ↗)
• Every Model Learned by Gradient Descent Is Approximately a Kernel Machine (2020) ↗ (HN ↗)
• DeepFaceLab ↗ - Leading software for creating deepfakes.
• Deep Learning DIY ↗ (Code ↗) (GitHub ↗) (Website Code ↗)
• Using JAX to accelerate our research (2020) ↗ (HN ↗)
• Stanford MLSys Seminar Series (2020) ↗
• MLCommons ↗ - Machine learning innovation to benefit everyone.
• Automated discovery of machine learning optimizations (2020) ↗
• A Visual Tour of Backpropagation (2020) ↗
• Deep Learning GPU Benchmarks ↗ (Code ↗)
• What I wish someone had told me about tensor computation libraries (2020) ↗ (HN ↗)
• Machine learning could be fundamentally unexplainable (2020) ↗
• Minimum Viable Study Plan for Machine Learning Interviews ↗
• Awesome JAX ↗
• Machine Learning Productivity Hacks (2019) ↗
• Learn Deep Learning: Powerful Mental Models to Accelerate Your Journey (2020) ↗ (Tweet ↗)
• OpenNMT ↗ - Open source ecosystem for neural machine translation and neural sequence learning. (GitHub ↗)
• How much hyperparameter tuning do you typically end up doing? (2020) ↗
• ZenML ↗ - Extensible, open-source MLOps framework for using production-ready Machine Learning pipelines. (Web ↗)
• MIT Parallel Computing and Scientific Machine Learning course (2020) ↗
• Invariant Risk Minimization (2019) ↗ (Code ↗)
• Awesome Federated Learning ↗
• Awesome Fraud Detection Research Papers ↗
• Probabilistic Machine Learning: An Introduction ↗ (HN ↗)
• Awesome Machine Learning Interpretability ↗
• Reflections on my (Machine Learning) PhD Journey (2020) ↗
• ML beyond Curve Fitting: An Intro to Causal Inference and do-Calculus (2018) ↗ (HN ↗)
• Distributed deep learning and inference without sharing raw data ↗ (Code ↗)
• Famous ML papers/concepts that are hard to understand (2021) ↗
• MLJAR Automated Machine Learning ↗ - Automates Machine Learning Pipeline with Feature Engineering and Hyper-Parameters Tuning. (Web ↗) (HN ↗)
• Best of Machine Learning with Python ↗
• Deep Learning’s Most Important Ideas - A Brief Historical Review (2020) ↗
• Awesome Anomaly Detection ↗
• Extending JAX with custom C++ and CUDA code ↗
• Mathematical Engineering of Deep Learning Course (2021) ↗
• Paper List for Style Transfer in Text ↗
• Machine learning with large-scale graphs course ↗ (Notes ↗)
• Single-Machine Simulation of Federated Learning Systems (2021) ↗
• Awesome AutoML ↗
• Optimization Methods for Machine Learning and Engineering (2021) ↗
• Three mysteries in deep learning: Ensemble, knowledge distillation, and self-distillation (2021) ↗
• Awesome Federated Computing ↗
• Noah ML Research ↗ - Research related code released by Huawei Noah’s Ark Lab.
• Prototypical Networks for Few-shot Learning (2017) ↗ (Code ↗)
• Domain generalization papers/resources ↗
• The Universal Training Loop of Machine Learning (2021) ↗
• Learning Curve Theory (2021) ↗
• ML Surveys ↗ - Survey papers summarizing advances in deep learning, NLP, CV, graphs, reinforcement learning, recommendations, graphs, etc.
• Practical Deep Learning Course ↗
• Diverse Counterfactual Explanations (DiCE) for ML ↗ - Generate Diverse Counterfactual Explanations for any machine learning model. (Docs ↗)
• Interpretable Machine Learning ↗ - Techniques & resources for training interpretable ML models, explaining ML models, and debugging ML models.
• Awesome Causality ↗ - Resources related to causality.
• How to Understand ML Papers Quickly (2021) ↗
• Patterns, Predictions, and Actions Book ↗ - A story about machine learning.
• Full Stack Deep Learning Course (2021) ↗ (Code ↗)
• Physical Principles for Scalable Neural Recording ↗
• Learn About Transformers: A Recipe (2021) ↗
• An Inferential Perspective on Federated Learning (2021) ↗
• Free Lunch for Few-shot Learning: Distribution Calibration (2020) ↗ (Code ↗)
• Enlightening Guide to Machine Learning Interviews ↗
• How Machine Language Works (2021) ↗
• Sliced Score Matching: A Scalable Approach to Density and Score Estimation (2019) ↗
• Accelerating Natural Gradient with Higher-Order Invariance (2018) ↗
• Some interesting observations about machine learning publication practices from an outsider (2021) ↗
• Understanding deep learning requires rethinking generalization (2021) ↗ (HN ↗)
• MIT HAN Lab ↗ - Accelerate Deep Learning Computing. (GitHub ↗)
• Feature Stores - A Hierarchy of Needs (2021) ↗
• Visualizing Representations: Deep Learning and Human Beings (2015) ↗ (Tweet ↗)
• Transformers ↗ - Collection of resources to study Transformers in depth.
• Incomplete Deep Learning Guide ↗
• A Year at Google Brain (2020) ↗ (HN ↗)
• Pretrained Transformers as Universal Computation Engines (2021) ↗ (Code ↗)
• Testing Machine Learning Systems: Code, Data and Models ↗ (Tweet ↗)
• Finding Mona Lisa in the Game of Life (2021) ↗ (HN ↗)
• Geometric deep learning, from Euclid to drug design (2021) ↗ (Reddit ↗)
• GeoGuessing with Deep Learning (2021) ↗
• Awesome Incremental Learning / Lifelong learning ↗
• Out of Distribution Generalization in Machine Learning (2021) ↗
• Label Errors ↗ - Label errors in benchmark ML test sets. (Lobsters ↗) (Code ↗)
• What will the major ML research trends be in the 2020s? ↗
• Machine Learning and Deep Learning Courses (2021) ↗
• Awesome Domain Adaptation ↗
• Why machine learning struggles with causality (2021) ↗
• TabNet: Attentive Interpretable Tabular Learning (2020) ↗ (Code ↗)
• AutoML.org ↗ (GitHub ↗)
• How I built a €25K Machine Learning Rig (2021) ↗ (Reddit ↗)
• Learning Fast Algorithms for Linear Transforms Using Butterfly Factorizations (2019) ↗
• Machine Learning Collection ↗ - Resource for learning about ML, DL, PyTorch and TensorFlow.
• Joint Universal Syntactic and Semantic Parsing (2021) ↗
• A Comprehensive Introduction to Bayesian Deep Learning (2021) ↗
• Stanford Machine Learning with Graphs Course (2021) ↗
• Keepsake ↗ - Version control for machine learning. (Web ↗)
• Learning Versatile Neural Architectures by Propagating Network Codes ↗
• Machine learning is going real-time (2020) ↗
• Awesome Normalizing Flows ↗
• Differentiable Model Compression via Pseudo Quantization Noise (2021) ↗ (Code ↗)
• The Rise of HuggingFace (2021) ↗
• See through Gradients: Image Batch Recovery via GradInversion (2021) ↗ (Reddit ↗)
• Recommendation System using ML and DL ↗
• Mathematical Foundations of Machine Learning (2020) ↗
• How to Write Design Docs for Machine Learning Systems ↗ (Code ↗)
• Reproducible Deep Learning (2021) ↗ - PhD Course in Data Science. (Code ↗)
• Unsupervised Contrastive Learning of Sound Event Representations (2020) ↗ (Code ↗)
• Differentially Private Learning Needs Better Features (or Much More Data) (2021) ↗
• Bias, variance, and their relationship with machine learning algorithms explained (2021) ↗
• Unsupervised Contrastive Learning of Sound Event Representations ↗
• Kobra ↗ - Visual programming language for machine learning. (HN ↗)
• Delving into Deep Imbalanced Regression (2021) ↗ (Code ↗)
• Comprehensive Survey on Transfer Learning ↗
• Applied Deep Learning Course ↗ (Videos ↗)
• Pay Attention to MLPs (2021) ↗ (Tweet ↗) (Code ↗)
• E(n) Equivariant Normalizing Flows for Molecule Generation in 3D (2021) ↗ (Tweet ↗)
• Fast and Slow Learning of Recurrent Independent Mechanisms (2021) ↗ (Tweet ↗)
• Weekly Papers ↗ (Daily ↗)
• Sharpness-Aware Minimization for Efficiently Improving Generalization (2020) ↗ (Code ↗)
• Apple - Making Mobile Applications Accessible with Machine Learning (2021) ↗
• NYU Deep Learning Spring 2021 ↗ (Videos ↗)
• JAX learning resources (2021) ↗
• Machine Learning with Python Cookbook Book (2018) ↗
• Understanding Dimensionality Reduction with UMAP ↗
• Neural Algorithmic Reasoning (2021) ↗ (Tweet ↗)
• A Pragmatic Look at Deep Imitation Learning ↗
• Stanford CS229: Machine Learning Course (2021) ↗ (Notes ↗)
• Informative Dropout for Robust Representation Learning: A Shape-bias Perspective (2020) ↗ (Code ↗)
• Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval (2020) ↗ (Code ↗)
• Fairification: Making Unfair Programs Fair (2017) ↗
• Fairness and machine learning: Limitations and Opportunities ↗
• Practical Deep Learning for Cloud, Mobile, and Edge (2019) ↗
• Pretrained Encoders are All You Need (2021) ↗ (Code ↗)
• The Modern Mathematics of Deep Learning (2021) ↗ (HN ↗)
• An Attention Free Transformer (2021) ↗ (Code ↗)
• Towards Causal Representation Learning (2021) ↗
• SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training (2021) ↗ (Code ↗)
• AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation (2021) ↗ (Video ↗)
• Distributed Machine Learning Patterns Book (2021) ↗ (Code ↗)
• Revisiting Deep Learning Models for Tabular Data (2021) ↗
• Introduction to Machine Learning Interviews Book ↗ (Code ↗)
• Introduction to Machine Learning 2019 at ETH Zürich ↗ (Summary ↗)
• Awesome Community Detection Research Papers ↗
• Understanding Deep Learning (2021) ↗
• The Scaling Hypothesis ↗
• ML YouTube Courses ↗
• Neuromatch Academy Deep Learning (NMA-DL) syllabus ↗
• Stanford AI Lab Papers and Talks at CVPR 2021 ↗
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption (2021) ↗
• Machine Learning for Beginners - A Curriculum ↗
• You Don’t Need Math For Machine Learning (2021) ↗
• Data Augmentation Resources ↗
• Extremely revealing books that explains everything behind machine learning? (2021) ↗
• ML@B ↗ - Machine Learning at Berkeley.
• Awesome Graph Self-Supervised Learning ↗
• Contrastive Representation Learning (2021) ↗ (HN ↗)
• Learn Machine Learning Resources ↗
• Popular Machine Learning Interview Questions (2021) ↗
• Solving Machine Learning Performance Anti-Patterns: a Systematic Approach (2021) ↗
• Bayesian learning via stochastic gradient langevin dynamics (2011) ↗
• Megaverse: Simulating Embodied Agents at One Million Experiences per Second ↗ (Paper ↗) (Code ↗)
• Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Them (2021) ↗ (Tweet ↗)
• Linear unit-tests for invariance discovery (2021) ↗ (Code ↗)
• In Search of Lost Domain Generalization (2020) ↗ (Code ↗)
• Understanding and improving out-of-distribution generalisation with Agnieszka Słowik (2021) ↗
• Algorithmic Concept-based Explainable Reasoning (2021) ↗ (Tweet ↗)
• Tsinghua Machine Learning Group ↗ (GitHub ↗)
• Zero-Shot Learning Resources ↗
• Machine Learning Collection ↗
• Open MLOps – Open-Source Production Machine Learning (2021) ↗ (HN ↗)
• A Gentle Introduction To Gradient Descent Procedure (2021) ↗
• Stanford MLSys Seminars ↗
• Painless Uncertainty for Deep Learning (2021) ↗
• How to avoid machine learning pitfalls: a guide for academic researchers (2021) ↗
• MLOps-Basics ↗
• Machine & Deep Learning Compendium ↗ (Code ↗)
• Differentiable Factor Graph Optimization for Learning Smoothers (2021) ↗
• A visual introduction to Gaussian Belief Propagation (2021) ↗ (HN ↗)
• Bootstrap your own latent: A new approach to self-supervised Learning (2020) ↗ (Code ↗)
• Tutorial: Performance Engineering for Machine Learning and Scientific Computing (2017) ↗
• Must-read papers on Recommender System ↗
• Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts (2018) ↗ (Code ↗)
• Machine Learning Bookcamp: Build a portfolio of real-life projects (2021) ↗ (Code ↗)
• Product Recommendations (2019) ↗
• Stanford CRFM ↗ - Stanford Center for Research on Foundation Models. (GitHub ↗)
• [1810.04650] Multi-Task Learning as Multi-Objective Optimization (2019) ↗ (Code ↗)
• Hyperparameter Search with spaCy and Weights & Biases (2021) ↗ (Tweet ↗)
• Dive into Machine Learning ↗
• The Values Encoded in Machine Learning Research (2021) ↗ (Tweet ↗)
• An overview of the theory of overparameterized machine learning (2021) ↗
• Geometric Deep Learning Blueprint (2021) ↗
• Multi-Armed Bandits and Pure-Exploration (2020) ↗
• The First Rule of Machine Learning: Start Without Machine Learning (2021) ↗ (HN ↗)
• An Introduction to Weighted Automata in Machine Learning (2021) ↗ (Code ↗)
• Ultimate FREE Machine Learning Study Plan ↗
• node2vec: Scalable Feature Learning for Networks ↗ (Code ↗)
• Learning to Superoptimize Real-world Programs (2021) ↗
• Data Movement Is All You Need: A Case Study on Optimizing Transformers (2020) ↗ (Code ↗)
• Reading List for Topics in Representation Learning ↗
• Курс по машинному обучению для 3 курса факультета ВМК МГУ (2021) ↗
• Deep Learning with Python, Second Edition (2021) ↗ (Code ↗)
• Some ML tools (2021) ↗
• Unsolved Problems in ML Safety (2021) ↗ (HN ↗)
• Evaluating Predictive Distributions: Does Bayesian Deep Learning Work? (2021) ↗ (Code ↗) (Tweet ↗)
• Arsenii Ashukha: Ensemble Generation (2020) ↗
• Imitating Deep Learning Dynamics via Stochastic Differential Equations (2021) ↗ (HN ↗)
• What are some ideas that are hyped up in machine learning research but don’t actually get used in industry (and vice versa)? (2021) ↗
• Variational Graph Auto-Encoders (2016) ↗ (Code ↗)
• Diffusion Normalizing Flow (2021) ↗
• Approximately Correct Machine Intelligence (ACMI) Lab ↗ - Research on machine learning, its social impacts, and applications to healthcare. (GitHub ↗) (Twitter ↗)
• Machine Learning, Kolmogorov Complexity, and Squishy Bunnies (2019) ↗
• Fast Machine Learning Lab ↗ (GitHub ↗)
• Meaning of interpolation in ML (2021) ↗
• Learning in High Dimension Always Amounts to Extrapolation (2021) ↗ (Tweet ↗)
• Awesome Transformer Architecture Search ↗
• Katana ML Skipper ↗ - Simple and flexible ML workflow engine. It helps to orchestrate events across a set of microservices and create executable flow to handle requests.
• PaRoT: A Practical Framework for Robust Deep Neural Network Training (2021) ↗
• Superposition of many models into one (2019) ↗ (Tweet ↗)
• Meta-Learning Requires Meta-Augmentation (2020) ↗
• Shaking the foundations: delusions in sequence models for interaction and control (2021) ↗ (Tweet ↗)
• Introduction to Deep Learning (I2DL) (2021) ↗
• Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data (2021) ↗ (Code ↗)
• Advanced-Foundations-of-ML ↗
• Implicit MLE: Backpropagating Through Discrete Exponential Family Distributions (2021) ↗ (Code ↗)
• Deep Learning Recommendation Model for Personalization and Recommendation Systems ↗
• MLflow Examples ↗
• 8-bit Optimizers via Block-wise Quantization (2021) ↗ (Code ↗)
• Efficiently Modeling Long Sequences with Structured State Spaces (2021) ↗ (Code ↗)
• Get started with JAX ↗ (Videos ↗)
• How does Jax allocate memory on a TPU? An interactive C++ walkthrough ↗ (HN ↗)
• Skyformer: Remodel Self-Attention with Gaussian Kernel and Nyström Method (2021) ↗ (Code ↗)
• Introduction to Deep Learning (2021) ↗ - 170 Video Lectures from Adaptive Linear Neurons to Zero-shot Classification with Transformers.
• Gradients are Not All You Need (2021) ↗ (Tweet ↗)
• FC2T2: The Fast Continuous Convolutional Taylor Transform with Applications in Vision and Graphics (2021) ↗ (Summary ↗)
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingvon (2018) ↗ (Code ↗)
• Model compression via distillation and quantization (2019) ↗ (Code ↗)
• ML Collective ↗ - Independent, nonprofit organization with a mission to make research opportunities accessible and free. (Classics and Trends ↗)
• Awesome MLOps ↗
• Machine Learning Zoomcamp ↗ (Code ↗)
• Yann LeCun’s 2021 Deep Learning Course at CDS (2021) ↗ (HN ↗)
• Interactive Gradient Descent Demo (2021) ↗
• Why Momentum Really Works (2017) ↗
• HNPE: Leveraging Global Parameters for Neural Posterior Estimation (2021) ↗ (Tweet ↗)
• Facebook AI Similarity Search (Faiss) ↗ (HN ↗)
• Combined Scaling for Zero-shot Transfer Learning (2021) ↗
• Awesome Distributed Deep Learning ↗
• Neuro Evolution Of Augmented Topologies (2021) ↗ (HN ↗)
• machine-config-operator ↗ - Managing updates and configuration changes to essentially everything between the kernel and kubelet.
• Factorized Fourier Neural Operators (2021) ↗ (Code ↗)
• NeurIPS 2021 Best Paper Awards (2021) ↗
• What areas of deep learning are under-explored? (2021) ↗
• cnvrg.io ↗ - Full Stack Machine Learning Operating System.
• Learning with not Enough Data Part 1: Semi-Supervised Learning (2021) ↗ (HN ↗)
• Efficient Training of Audio Transformers with Patchout (2021) ↗ (Code ↗)
• Machine Learning for Creativity and Design ↗
• Maximum Likelihood Training of Score-Based Diffusion Models (2021) ↗ (Code ↗)
• Learning Gradient Fields for Shape Generation (2020) ↗ (Code ↗)
• Making Friends with Machine Learning - YouTube ↗
• Schedule - NeurIPS 2021 ↗
• The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization (2021) ↗ (Code ↗)
• Momentum Residual Neural Networks (2021) ↗ (Code ↗)
• Awesome-Zero-Shot-Learning ↗
• Awesome Treasure of Transformers Models Collection ↗
• Self-attention Does Not Need $O(n^2)$ Memory (2021) ↗ (Code ↗) (Code ↗)
• Neural Discrete Representation Learning (2021) ↗ (Code ↗)
• VSE++: Improving Visual-Semantic Embeddings with Hard Negatives (2018) ↗ (Code ↗)
• JAX ResNet ↗ - Implementations and checkpoints for ResNet, Wide ResNet, ResNeXt, ResNet-D, and ResNeSt in JAX (Flax).
• Best AI and Deep learning books to read in 2022 ↗
• Machine Learning for Combinatorial Optimization - NeurIPS 2021 Competition ↗ (Code ↗)
• Tutorial and Summary of Machine Learning ↗
• Never Give Up: Learning Directed Exploration Strategies (2020) ↗ (Code ↗)
• A Step Toward Quantifying Independently Reproducible Machine Learning Research (2019) ↗ (Tweet ↗)
• ML Hub ↗ - Multi-user development platform for machine learning teams. Simple to setup within minutes.
• Adversarial Explainable AI ↗
• AI/ML Tutorials List ↗
• Awesome Diffusion Models ↗
• Autoencoder-based deep metric learning for network intrusion detection (2021) ↗ (Code ↗)
• Graph Adversarial Learning Literature ↗
• Trustworthy Machine Learning by Kush R. Varshney ↗
• Recommender System Suits ↗ - Open source toolkit for recommender system.
• Randomized Ensembled Double Q-Learning: Learning Fast Without a Model (2021) ↗ (Code ↗)
• STATS320: Machine Learning Methods for Neural Data Analysis Course (2021) ↗
• Deep Learning Interviews: Hundreds of fully solved job interview questions from a wide range of key topics in AI (2021) ↗ (Code ↗) (HN ↗)
• Diffusion-Models-Seminar ↗
• Awesome Graph Representation Learning ↗
• Machine Learning System Resources (2022) ↗
• Dynamic Tensor Rematerialization (2021) ↗ (Review ↗)
• Designing a Practical Degradation Model for Deep Blind Image Super-Resolution (2021) ↗ (Code ↗)
• A Theoretical Framework for Target Propagation (2020) ↗ (Code ↗)
• SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks (2020) ↗ (Code ↗)
• Iterative SE(3)-Transformers (2021) ↗ (Article ↗)
• Permutation Invariance, DeepSets and Universal Function Approximation ↗
• Review: Deep Learning on Sets ↗
• Contrastive Self-Supervised Learning (2020) ↗
• Top AI Conference Papers with Code ↗
• Awesome Deep Learning papers for industrial Search, Recommendation and Advertising ↗
• JaxTon ↗ - 100 exercises to learn JAX.
• Deep Multi-attribute Graph Representation Learning on Protein Structures (2020) ↗
• Variational Diffusion Models (2021) ↗ (Code ↗)
• Cheat sheet for the “Deep Learning” course at ETH Zürich ↗
• Convolutional Networks on Graphs for Learning Molecular Fingerprints (2015) ↗ (Code ↗)
• Cheatsheet for the Advanced Machine Learning Lecture 2020, ETH Zurich ↗
• Tutorial on amortized optimization for learning to optimize over continuous domains (2022) ↗ (Code ↗)
• Growing 3D Artefacts and Functional Machines with Neural Cellular Automata (2021) ↗ (Code ↗)
• Generative Flow Networks for Discrete Probabilistic Modeling (2022) ↗ (Code ↗)
• Awesome Contrastive Learning ↗
• ML-fairness-gym ↗ - Components for building simple simulations that explore the potential long-run impacts of deploying machine learning-based decision systems in social environments.
• MultiBench: Multiscale Benchmarks for Multimodal Representation Learning ↗
• Learning Features with Parameter-Free Layers (2022) ↗
• Gaussian Processes for Machine Learning: Book ↗
• IQ-Learn: Inverse soft-Q Learning for Imitation (2021) ↗ (Code ↗)
• On Neural Differential Equations (2022) ↗ (Reddit ↗) (Tweet ↗)
• Datamodels: Predicting Predictions from Training Data (2022) ↗ (Data ↗)
• Understanding Black-box Predictions via Influence Functions (2020) ↗ (PyTorch Code ↗)
• Parameter Prediction for Unseen Deep Architectures (NeurIPS 2021) ↗
• Language-Agnostic Representation Learning of Source Code from Structure and Context (2021) ↗ (Code ↗)
• Machine Learning Open Source University ↗
• Exploring hyperparameter meta-loss landscapes with Jax (2021) ↗
• Transformers Can Do Bayesian Inference (2022) ↗
• Norm-based Analysis of Transformer ↗ - Implementations for 2 papers introducing to analyze Transformers using vector norms.
• Point-NeRF: Point-based Neural Radiance Fields (2022) ↗ (Code ↗)
• ML training compute has been doubling every 6 months since 2010 ↗ (HN ↗)
• Using JAX in 2022 ↗ (HN ↗)
• Should We Be Using JAX in 2022? ↗
• Parallel Computing and Scientific Machine Learning (SciML): Methods and Applications ↗ (Code ↗)
• Awesome Neural ODE ↗ - Resources regarding the interplay between differential equations, deep learning, dynamical systems, control and numerical methods.
• LLC: Accurate, Multi-purpose Learnt Low-dimensional Binary Codes (2021) ↗ (Code ↗)
• TinyML Paper and Projects ↗
• Multiplicative Filter Networks (2021) ↗ (Code ↗)
• Class-incremental learning: survey and performance evaluation on image classification (2020) ↗ (Code ↗)
• Appendix: More Is Different In Other Domains (2022) ↗
• Robustness and Accuracy Could Be Reconcilable by (Proper) Definition (2022) ↗ (Code ↗)
• Nebullvm ↗ - All-in-one library for easy-to-use DL compilers.
• SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies (2021) ↗ (Code ↗)
• Open Platform for AI (OpenPAI) ↗ - Resource scheduling and cluster management for AI.
• Hugging Face Optimum ↗ - Accelerate Transformer models for training and inference on targeted hardware. (Tweet ↗)
• Gaussian Processes and Statistical Decision-making in Non-Euclidean Spaces (2022) ↗ (Tweet ↗)
• What’s hot for Machine Learning Research in 2022? ↗
• Machine Learning with PyTorch and Scikit-Learn (2022) ↗ (HN ↗) (Author AMA ↗)
• General Cyclical Training of Neural Networks (2022) ↗ (Code ↗)
• Machine Learning Interview Questions ↗
• Intro to Continual Learning ↗
• Gradients without Backpropagation (2022) ↗ (HN ↗)
• Variational Autoencoders Without the Variation (2022) ↗
• What’s your favorite unpopular/forgotten Machine Learning method? (2022) ↗
• Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels (2022) ↗ (Tweet ↗)
• Probabilistic Machine Learning: Advanced Topics ↗
• AST: Audio Spectrogram Transformer (2021) ↗ (Code ↗)
• Practical Machine Learning ↗ - Learn by experimenting on state-of-the-art machine learning models and algorithms.
• Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer (2022) ↗ (Code ↗)
• Optimization for machine learning course (2022) ↗ (Code ↗)
• Awesome Long-Tailed Learning ↗
• Offline Reinforcement Learning as One Big Sequence Modeling Problem (2021) ↗ (Code ↗)
• Deep Learning’s New Infrastructure (2022) ↗
• Gensyn ↗ - Hyperscale, cost-efficient compute protocol for the world’s deep learning models. (Twitter ↗)
• Software Engineering for AI/ML — An Annotated Bibliography ↗
• Physics-Based Deep Learning ↗
• Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation (2019) ↗ (Code ↗)
• Tensil ↗ - Open-Source ML Accelerators. (HN ↗) (Code ↗)
• Ask HN: What ML platform are you using? (2022) ↗
• Graphsignal ↗ - Machine Learning Profiler. (HN ↗)
• Machine Learning and AI with Go ↗
• Everything about Transfer Learning ↗
• Making Deep Learning Go Brrrr From First Principles ↗
• Machine Learning Guide ↗
• Building a ML Transformer in a Spreadsheet (2022) ↗
• Lecture Notes for Machine Learning Theory (2021) ↗
• On Embeddings for Numerical Features in Tabular Deep Learning (2022) ↗ (Code ↗)
• Automated Machine Learning in Action Book (2022) ↗ (Code ↗)
• ML Course Notes ↗
• The Mathematics of Artificial Intelligence (2022) ↗
• Differentiable Sorting Networks for Scalable Sorting and Ranking Supervision (2021) ↗ (Code ↗)
• Fast TreeSHAP: Accelerating SHAP Value Computation for Trees (2021) ↗ (Code ↗)
• Memorizing Transformers (2022) ↗ (Code ↗) (HN ↗)
• DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latents (2022) ↗ (Code ↗)
• Metarank ↗ - Low-code Machine Learning personalization service. (Code ↗) (HN ↗)
• Research on Tabular Deep Learning ↗
• HuggingFace Blog ↗ (Code ↗)
• Learning to Prompt for Continual Learning (2022) ↗ (Code ↗)
• Group Equivariant Deep Learning (2022) ↗ (Tweet ↗)
• Human-Centered Machine Learning (2022) ↗
• Transformer models: an introduction and catalog — 2022 Edition ↗
• Bayesian Structure Learning with Generative Flow Networks (2022) ↗ (Code ↗)
• Neural Networks with Recurrent Generative Feedback (2020) ↗ (Code ↗)
• Coursera Machine Learning MOOC by Andrew Ng ↗ (Code ↗)
• Chaos is a Ladder: A New Understanding of Contrastive Learning (2022) ↗ (Code ↗)
• Efficient-VDVAE: Less is more (2022) ↗ (Code ↗)
• Transformer Quality in Linear Time (2022) ↗ (Code ↗)
• Machine Learning for Big Code and Naturalness ↗ - Survey of Machine Learning for Big Code and Naturalness. (Code ↗)
• Randomized Smoothing of All Shapes and Sizes (2020) ↗ (Code ↗)
• Redesigning the Transformer Architecture with Insights from Multi-particle Dynamical Systems (2021) ↗ (Code ↗)
• Deep Maths - machine learning and mathematics (2022) ↗
• Manim Machine Learning ↗ - Focused on providing animations and visualizations of common machine learning concepts with the Manim Community Library.
• Anyscale ↗ - Effortlessly develop, scale and deploy AI, at any scale. (GitHub ↗)
• Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting (2021) ↗ (Code ↗)
• On the Bottleneck of Graph Neural Networks and its Practical Implications (2021) ↗ (Code ↗)
• ML Notebooks ↗ - Series of code examples for all sorts of machine learning tasks and applications.
• Awesome Semi-Supervised Learning ↗
• FedScale: Benchmarking Model and System Performance of Federated Learning at Scale (2021) ↗ (Code ↗)
• Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting (2021) ↗ (Code ↗)
• Understand transformer architectures (2022) ↗
• The Future of Machine Learning Tools (2021) ↗
• Transformers Are All You Need: Quick tour through the most popular Neural Net architecture (2022) ↗
• Transformers in Time Series ↗ - Curated list of awesome resources (paper, code, data, etc.) on transformers in time series.
• SRBench: A Living Benchmark for Symbolic Regression (2022) ↗ (Code ↗)
• How FAANG etc. architect their recommendation systems at scale ↗
• BigScience Research Workshop ↗
• SmallScience ↗ - My journey to training a large(ish) transformer model.
• Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networks (TSSL-BP) ↗ (Code ↗)
• Recommender Systems, Not Just Recommender Models (2022) ↗
• Reading Lists of Machine Learning, Natural Language Processing and etc. ↗
• Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions (2021) ↗ (Code ↗)
• The Principles of Deep Learning Theory (2021) ↗ (HN ↗)
• Understanding the Limitations of Variational Mutual Information Estimators (2019) ↗ (Code ↗)
• Tensor Puzzles ↗ - Solve puzzles. Improve your PyTorch. (Tweet ↗)
• ACM FAccT - 2022 Accepted Papers ↗
• How Attention works, in the field of artificial intelligence ↗
• Awesome Weak-Shot Learning ↗
• Ultimate Awesome Transformer Attention ↗
• What are Diffusion Models? (2022) ↗
• Apollo: An Adaptive Parameter-wise Diagonal Quasi-Newton Method for Nonconvex Stochastic Optimization (2020) ↗
• Awesome Conformal Prediction ↗
• The Machine Learning Job Market (2022) ↗ (HN ↗)
• Admin-Torch ↗ - Understanding the Difficulty of Training Transformers.
• Sampling with Mirrored Stein Operators (2022) ↗ (Code ↗) (Tweet ↗)
• data2vec and the future of multimodal learning (2022) ↗
• Mapping Fair ML ↗ - Curated list of links and resources for Fair ML and Data Ethics.
• Open Source MLOps ↗ - Fuzzy Labs guide to the universe of free and open source MLOps tools.
• Why train when you can optimize? ↗ (HN ↗) (Code ↗)
• Data Engineering & Machine Learning Knowledge Hub ↗
• Graph Contrastive Learning with Augmentations (2020) ↗ (Code ↗)
• Pseudo Numerical Methods for Diffusion Models on Manifolds (2022) ↗ (Code ↗)
• Practical MLOps Book (2021) ↗ (Code ↗)
• Disco Diffusion ↗ - Notebooks, models and techniques for the generation of AI Art and Animations.
• The Annotated Transformer: Attention is All You Need ↗ (Code ↗)
• Metaseq ↗ - Codebase for working with Open Pre-trained Transformers.
• Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoders (2022) ↗ (Code ↗)
• Learning with Noisy Labels ↗
• Pathways: Asynchronous Distributed Dataflow for ML (2022) ↗ (Pathways: Google’s New ML System ↗)
• Google AI Blog: Alpa: Automated Model-Parallel Deep Learning (2022) ↗
• Compositional Attention: Disentangling Search and Retrieval (2021) ↗ (Code ↗) (Code ↗) (Code ↗)
• Awesome Active Learning ↗
• Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning (2022) ↗ (Code ↗)
• Introduction to Diffusion Models for Machine Learning (2022) ↗ (HN ↗)
• BGU-CS-VIL/DeepDPM: “DeepDPM: Deep Clustering With An Unknown Number of Clusters” (2022) ↗ (Code ↗)
• Towards a Learning-Based Query Optimizer (2022) ↗ (HN ↗)
• Scaling-up Diverse Orthogonal Convolutional Networks with a Paraunitary Framework (2021) ↗
• Data Distributional Properties Drive Emergent Few-Shot Learning in Transformers (2022) ↗ (Tweet ↗)
• Machine Learning Specialization - DeepLearning.AI ↗ (HN ↗)
• MLU-Explain ↗ - Visual explanations of core machine learning concepts. (HN ↗)
• Planning with Diffusion for Flexible Behavior Synthesis (2022) ↗ (Code ↗)
• Solutions to Recommender Systems competitions ↗
• Recipe for a General, Powerful, Scalable Graph Transformer (2022) ↗ (Code ↗)
• How to properly handle hyperparameter configs in ML repos (2022) ↗
• FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (2022) ↗ (Code ↗)
• Machine Learning Design patterns ↗ - Software Architecture for ML engineers.
• Hopular: Modern Hopfield Networks for Tabular Data (2022) ↗ (Code ↗)
• Some thoughts on machine learning with small data (2022) ↗ (HN ↗)
• Most elegant/beautiful ideas in ML? (2022) ↗
• Apple Silicon DL benchmarks ↗ - Collection of ML scripts to test the M1 Pro MacBook Pro.
• Post-Modern ML Stack ↗
• Improving Discrete Latent Representations With Differentiable Approximation Bridges (2019) ↗ (Code ↗)
• Semantic Search and GIFs ↗ (HN ↗)
• Materials for workshops on the Hugging Face ecosystem ↗
• Deploying Transformers on the Apple Neural Engine (2022) ↗ (HN ↗)
• A comprehensive review of Binary Neural Network (2022) ↗
• Self-organising Systems from Google ↗
• Improved Denoising Diffusion Probabilistic Models (2021) ↗ (Code ↗)
• What topics to learn to get ‘cutting edge AI’ (2022) ↗
• Probability flow solution of the Fokker-Planck equation (2022) ↗
• Review of latest Score Based Generative Modeling papers ↗ - All diffusion papers reverse chronological.
• Intro resources on diffusion/score-matching models (2022) ↗
• Meta Optimal Transport (2022) ↗ (Code ↗)
• envd ↗ - Development environment for machine learning.
• Awesome Open Source MLOps ↗
• Generalised Implicit Neural Representations (2022) ↗ (Code ↗)
• Diffusers ↗ - Provides pretrained diffusion models across multiple modalities, such as vision and audio, and serves as a modular toolbox for inference and training of diffusion models.
• Lightning AI ↗ - Use Lightning Apps to build everything from production-ready, multi-cloud ML systems to simple research demos.
• Latent World Models For Intrinsically Motivated Exploration (2020) ↗ (Code ↗)
• Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt (2022) ↗ (Code ↗)
• Self-Supervised Learning from Images: Up-to-date reading list ↗
• How robust are pre-trained models to distribution shift? (2022) ↗
• MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge (2022) ↗ (Code ↗)
• MLOPs Primer ↗ - Collection of resources to learn about MLOPs.
• Are wider nets better given the same number of parameters? (2020) ↗ (Code ↗)
• Brandon Amos’s presentation slides ↗
• Ethical Principles for Web Machine Learning ↗ (Code ↗)
• MLC ↗ - Machine Learning Compiler.
• Golan Levin’s lectures ↗
• RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network (2022) ↗ (Code ↗)
• Pen and paper exercises in machine learning ↗ (Paper ↗) (HN ↗)
• Dual Curriculum Design ↗ - Implementations of robust Dual Curriculum Design (DCD) algorithms for unsupervised environment design.
• gDDIM: Generalized denoising diffusion implicit models (2022) ↗ (Code ↗)
• Awesome Active Learning ↗
• Designing Machine Learning Systems: An Iterative Process for Production-Ready Applications ↗ (Code ↗)
• The Berkeley Crossword Solver ↗
• MLC ↗ - Machine Learning Compiler.
• Recommender System on MovieLens dataset ↗
• Awesome Radar Perception ↗ - Curated list of radar datasets, detection, tracking and fusion.