Gpu runtime system improves task handling and latency hiding
HEL: A Heterogeneous Extensible Layer for GPU-Resident Runtime Orchestration
Distributed, Parallel, and Cluster Computing
Summary
Running many small tasks efficiently on a graphics processing unit (GPU) can be tricky because tasks must be managed carefully. The authors created HEL, a system that lives on the GPU and organizes these tasks like little programs that can pause and resume. HEL can handle tasks that wait for others, retry failed tasks automatically, and run multiple tasks together. Their tests show HEL works better when GPUs are not fully busy and helps hide delays between tasks, making some programs run faster.
What this means in practice
- •For gpu software developers: Schedule and run many interdependent GPU tasks efficiently to improve performance when GPUs are underutilized.
- •For compression software teams: Integrate HEL to reduce latency in GPU-accelerated compression workloads for better throughput.
Authors
Agnieszka M. Łupińska, Serban D. Porumbescu, John D. Owens
Abstract
We describe a GPU-resident runtime system, HEL, built on stateful services that persist throughout the lifespan of a user application. A HEL programmer expresses a workload as an application that spawns coroutine-like tasks, which are then scheduled and run by the HEL runtime. HEL supports synchronous and asynchronous tasks, tasks with runtime dependencies, batched task submission, and automatic retries. Applications and tasks run alongside each other on the spatially partitioned GPU. Using producer-consumer and host-RPC benchmarks together with a PFPL compression case study, we show that HEL demonstrates a performance advantage for underutilized GPUs and for scenarios where HEL enables hiding task latency.