LooseLeaf frees up GPU time in interactive notebooks
LooseLeaf: Enabling Serverless GPUs for Computational Notebooks
Distributed, Parallel, and Cluster Computing
Summary
Computational notebooks usually keep a GPU reserved the entire time, even though it’s only used briefly when a cell runs. This means GPUs sit idle a lot, wasting resources. The authors present LooseLeaf, which lets notebooks release the GPU after each cell by quickly rebuilding just the parts needed for the next cell. This approach cuts GPU usage significantly without making users wait longer when running code.
What this means in practice
- •For cloud platform engineers: Provision GPUs more efficiently for interactive notebook users by reclaiming GPU time between executed cells without perceptible delays.
- •For data analysts using high-performance compute: Run GPU-powered analyses in notebooks with lower cost and better resource reuse by allowing GPU release after each cell execution.
Authors
Yusen Wu, Tingfeng Lan, Benjamin Carver, Haoliang Wang, Geoffrey Fox, Yue Cheng
Abstract
Computational notebooks reserve a GPU for the whole session, yet the GPU computes only while a cell executes and sits idle otherwise. Serverless execution would release the GPU between cells, but reconstructing the accumulated notebook state on a fresh worker imposes substantial cost on every GPU cell. This paper presents LooseLeaf, a serverless GPU platform for interactive notebooks. LooseLeaf derives the state and import closures of each cell, the minimal variables and packages that the cell needs. Our analysis on 5,000 real-world notebooks shows that these closures are small, and user-cell editing leaves seconds to prepare them. LooseLeaf prepares a GPU worker with these closures while the user edits the cell. After execution, the worker saves the updated state within seconds, so the GPU is reclaimable after every cell. LooseLeaf starts 95-98% of GPU cells with no visible delay, matches the interactivity of a pinned session, and provisions 32-55% fewer GPU-hours than state-of-the-art notebook platforms.