pythontaskviews

Python task views aim to provide guidance on which python packages are relevant for tasks related to a certain data science topic

View project on GitHub

Python Task Views: High Performance Computing

Packages that help support HPC

Functionality (PyPi Link) Description / Remarks
Bindings to C/C++ Cython, pybind11 Python is slow compared to lower level / compiled languages. A common approach to make full use of existing CPU is to extend the language via bindings to a faster language
Bindings to other languages (Java, Rust) py4j , pyO3  
Multi-threading Built-in (thread)  
Multi-core multiprocessing  
Spark interface pySpark  
GPU Computing pyCUDA Offered also built-in in some packages (e.g pytorch, tensorflow)
Distributed Data dask