Parsl: Enabling Scalable Interactive Computing in Python

Loading

Follow to receive video recommendations   a   A


Scientists and engineers like to use Python for interactive data science, machine learning, and online computing. However, computations that are simple to perform at small scales (e.g., on a laptop) can easily become prohibitively difficult as data sizes and analysis complexity grows. For example, efficient interactive analysis at scale can require real-time management of parallel and/or cloud computing resources, orchestration of remote task execution, and data staging across wide area networks. In this talk we introduce Parsl (Parallel Scripting Library), a pure Python library for orchestrating the concurrent and parallel execution of interactive and many-task workloads, and demonstrate how it integrates with the scientific Python ecosystem and how it is being used in a variety of scientific domains. The talk is intended for attendees interested in interactive and parallel computing.

Editors Note:

I am looking for editors/curators to help with branches of the tree. Please send me an email  if you are interested.