Interactive debugger brings easier testing to Python high performance code
Interactive Debugger for Performance Portable Python HPC Kernels
Distributed, Parallel, and Cluster Computing
Summary
Debugging programs that run on multiple computer cores or graphics cards can be very hard, especially with Python code used for scientific and high performance computing. The authors created PKDB, a tool that lets programmers pause, step through, and check variables in the exact part of the code running on CPUs or GPUs without changing the program. It also lets them test new code or fix bugs right inside the running program without restarting everything. Tests showed PKDB works well on different processors and adds only a little extra work for the computer, making it a practical help for programmers.
Pythonhigh performance computingGPUmultithreadingdebuggerPyKokkosOpenMPinteractive debuggingcode evaluationkernel recompilation
Authors
Ivan Grigorik, Gabriel Kosmacher, George Biros, Milos Gligoric
Abstract
We propose PKDB, the first interactive debugger for GPU and multithreaded low-level kernels written in Python. Python is widely used in high performance computing (HPC), with frameworks such as PyKokkos translating Python-embedded domain-specific languages to native code that runs across OpenMP-threaded CPUs and various GPUs. Yet interactive debugging support for such code is absent: developers resort to print statements, framework-specific assertions, or CPU-only execution, the last of which requires altering the program or its data and can mask device-specific bugs. PKDB enables standard interactive debugging like breakpoints, stepping, and variable inspection while preserving actual on-device execution without source modification. Beyond these fundamentals, PKDB introduces two advanced capabilities that exploit the dynamic nature of Python and PyKokkos: (i) Live code evaluation, which lets developers execute arbitrary Python expressions or entire kernels in the middle of a paused kernel without restarting the process; (ii) Kernel call site substitution, which allows an actively running kernel to be updated and reloaded on the fly, so only the kernel is recompiled and re-executed without restarting the application. Our performance evaluation on Intel, AMD, and NVIDIA CPUs, and NVIDIA and AMD GPUs shows that PKDB introduces limited overhead and is practical for everyday use while introducing critical debugging features to the Python HPC ecosystem.