Dask

---

title: Dask#

Parallel computing with task scheduling.

Links#

• Dask Tutorial ↗ (Code ↗)
• Deploy Dask on Job Queueing systems ↗
• Dask Gateway ↗ - Provides a secure, multi-tenant server for managing Dask clusters.
• Jim Crist: Introducting Dask-Gateway: Dask clusters as a service (2019) ↗
• Distributed ↗ - Distributed task scheduler for Dask.
• dask-ml ↗ - Scalable Machine Learning with Dask.
• Tom Augspurger: Scalable Machine Learning with Dask (2019) ↗
• Data Processing with Dask (2020) ↗
• Dask Updates 2020 ↗
• Deploying and Scaling Data Science Tools on Distributed Systems (2020) ↗
• dask-sql ↗ - SQL Engine for Dask. (Web ↗)
• Coiled: Dask for Everyone, Everywhere (2020) ↗
• Coiled ↗ - Scaling Python Simply.
• Coiled Cloud ↗
• Runtime vs Scheduler: Analyzing Dask’s Overheads (2020) ↗
• Dask Helm Charts ↗ (Web ↗)
• Dask CUDA ↗ - Utilities for Dask and CUDA interactions.
• Dask Web Code ↗
• Dask Example Notebooks ↗ - Includes easy-to-run example notebooks for Dask. (Docs ↗)
• Dask Kubernetes Operator ↗
• dask-geopandas ↗ - Parallel GeoPandas with Dask.
• Dask Cloud Provider ↗ - Native Cloud integration for Dask. (Docs ↗)
• Jupyter, Microscopy, and On-demand Dask Cloud Clusters (2021) ↗
• Tackling unmanaged memory with Dask (2021) ↗
• Hacking Dask: Diving into Dask’s Internals ↗
• Dask Binder template ↗ - Template for Dask-enabled repositories for use on Binder.
• Afar ↗ - Run code on a Dask cluster via a context manager or IPython magic.
• Dask as a Spark Replacement (2021) ↗
• 2021 Dask User Survey ↗
• Dask Working Notes ↗
• Choosing good chunk sizes in Dask (2021) ↗
• HN: Dask (2021) ↗
• dask-awkward ↗ - Connecting Dask with Awkward Arrays.
• flox ↗ - Fast & furious GroupBy operations for dask.array.
• dask-ctl ↗ - Tools to provide a control plane for managing the lifecycle of Dask clusters.
• dask-geomodeling ↗ - On-the-fly operations on geographical maps.