Python is becoming increasingly popular, even in parallel and high-performance computing, although performance is often worse than compiled languages. In our previous work, we looked at Cuda Numba and showed that it can achieve good performance for single kernels. We are now looking at multi-GPU applications that require data exchange. We show that by using stream-aware communication, as enabled by NCCL, performance can be many times better than with MPI, where the Python interpreter poses a significant performance problem.

Lena Oden is a professor for Computer Engeneering at the FernUniversität in Hagen and scientist at the Jülich Supercomuting Center.

Her research interests are programming models and runtime systems, with a special interest in (multi-) GPU computing and the design and implementation of federated research infrastructures, combining the benefits of cloud and HPC computing.

Copyright © P2S2. All rights reserved.