PredictIO improves storage prefetching by separating prediction from execution
PredictIO: Decoupling Workload Prediction from OS-Controlled Storage Prefetching
Operating Systems
Summary
Storage delays slow down many computer applications, even with fast devices. Prefetching data ahead of time can help, but operating systems usually guess future data needs based only on recent file activity, which isn't effective for complex workloads. The authors created PredictIO, a system that lets programs predict what data they'll need separately from how that data is fetched, making prefetching smarter and more reusable across different applications. Their tests show it reduces delays and speeds up tasks like launching apps, running database scans, and loading large models.
What this means in practice
- •For database engineers: Integrate PredictIO to speed up range scans in databases like SQLite and LMDB by separating prediction from storage fetch execution.
- •For ai infrastructure teams: Use PredictIO to cut down loading times of large language model data and metadata by improving prefetching efficiency.
Authors
Jun Lu, Yancan Mao, Zhenya Ma, Sheng Yue, Yongjian Fu, Ju Ren
Abstract
Storage stalls remain a significant source of application latency even on fast storage devices. Prefetching can hide part of this latency by fetching data before it is demanded, but general-purpose operating-system mechanisms typically infer future accesses from recent file offsets. This approach is ineffective when accesses are determined by workload structure, such as recurring sparse accesses during application launch, logical traversal in database scans, or model-layout metadata in LLM runtimes. Existing workload-aware prefetchers usually embed prediction logic inside individual applications and couple execution to application-level I/O paths, making policies difficult to reuse across workloads and systems. We present PredictIO, a programmable interface that separates workload prediction from prefetch execution. PredictIO delivers kernel-observed accesses and application hints to user-space policies, which express predictions as uniform page-range requests. A kernel-space executor validates, bounds, and filters these requests against cached and in-flight pages before issuing asynchronous prefetch I/O. We implement PredictIO for application launch, SQLite range scans, and LLM model loading. Across these workloads, PredictIO reduces latency by 5.3-26.3%, achieves up to 2.99x speedup for SQLite scans, and reduces LLM loading latency by 8.8-47.9%. We further show that a common prediction policy can be reused across SQLite and LMDB through lightweight adapters, that an application-integrated RocksDB readahead policy can be migrated while preserving its workload selectivity, and that the execution backend determines how much benefit migration provides for different access patterns and storage devices.