Operating system changes speed up hardware accelerators in chips
Operating System Support for Accelerators as First-Class Compute Elements in Heterogeneous Systems
Hardware ArchitectureDistributed, Parallel, and Cluster ComputingOperating Systems
Summary
Many modern computer chips have special helper parts called hardware accelerators to make tasks faster and use less energy. But current operating systems treat these helpers like slower devices instead of powerful helpers, which slows things down. The authors built a system called MEERKAT to test and find ways to make operating systems work better with these helpers. Their improvements help accelerators talk to software faster and share work better with regular processors, making programs run up to six times faster in some cases. This shows it's important for operating systems to fully support these helpers as real computing parts.
What this means in practice
- •For embedded systems engineers: Improve performance of embedded devices by integrating special hardware accelerators more efficiently into operating systems.
- •For cloud infrastructure operators: Enhance server efficiency by better scheduling and communication between CPUs and hardware accelerators to speed up workload processing.
Authors
Mohammad Shahidzadeh, Lesley Shannon
Abstract
Heterogeneous systems-on-chip (SoCs) increasingly rely on domain-specific hardware accelerators (HAs) to deliver performance and energy efficiency gains beyond what general-purpose processors can provide. However, operating systems (O/S), such as Linux, assume that processors are the Compute Elements (CEs) and HAs are peripheral devices rather than specialized CEs. As a result, HA execution incurs substantial software (SW) overheads that reduce the effective benefits of hardware (HW) acceleration. This work demonstrates that significant inefficiencies arise from ``O/S peripheral device taxes'' (O/S taxes), including SW stack-accelerator communication, invocation, interrupt handling, and scheduling. We built MEERKAT, a Field-Programmable Gate Array (FPGA)-based framework for configurable multicore systems that include HAs; it allows the user to migrate Linux O/S support from SW to HW and identify, study, and quantify the impact. Here we study four major sources of O/S tax and introduce techniques to mitigate them. These include a virtual-memory-aware (VMA) communication method that enables efficient HA access to application data, a lightweight HA invocation interface, Interrupt Policy Adaptation (IPA) for dynamic polling and interrupt selection, and an HA-aware scheduler that dynamically distributes work across Central Processing Units (CPUs) and multiple HA instances. Experimental results demonstrate up to $260\times$ faster communication and $2$--$6\times$ performance improvements in wall-clock time over conventional schedulers. Furthermore, our study reveals that overall system performance depends not only on HA speed, but also on O/S overheads, resource availability, and scheduling decisions. These findings highlight the importance of treating HAs as first-class computational resources and motivate a more accelerator-centric O/S design for future heterogeneous systems.