Near data strands speed up loops without software changes
NDS: Programmer-Free Offload of High-Performance Near Data Strands
Hardware Architecture
Summary
Moving data around in computers can slow things down and waste energy. This paper introduces a new method called Near Data Strands (NDS) that automatically finds parts of a program that can run close to where the data lives, making things faster without making programmers change their code. The system breaks down loops in programs into parallel pieces, handles the data, and runs them near the data automatically. Tests show it can make programs over three times faster than running on a single core, and even faster than eight cores working together, all without any software changes.
What this means in practice
- •For system architects: Design hardware systems that automatically accelerate loops in existing applications without software changes using near-data execution.
- •For high performance computing teams: Run legacy compute-intensive loops faster by leveraging hardware that orchestrates execution near the data without rewriting software.
Authors
Shreyas Singh, Pratyush Nandi, Lin Jia, Shankar Balachandran, Rajeev Balasubramonian
Abstract
Near Data Processing (NDP) has the potential to significantly improve system performance and energy by alleviating data movement bottlenecks. However, most NDP proposals pose heavy requirements for the software stack, data layout, and/or the underlying hardware. To broaden NDP adoption, this work focuses on a modular hardware-centric approach that automates the above steps and can target unmodified binaries. This paper presents Near Data Strands (NDS), a framework that orchestrates tasks and data automatically without involving the programmer and the software stack. As a first step towards this ambitious goal, this work focuses on regular loops that meet specific criteria. The framework identifies potential offloadable loops, decomposes them into parallel execution strands, creates a high performance instruction schedule, performs the required data marshaling and orchestration, and initiates near-data execution. We demonstrate that NDS achieves a 3.14x geomean speedup over a single-core host baseline and a 1.82x geomean speedup over 8-core host-parallel variants, all without any modifications to the software stack. These gains grow as repeated loop invocations amortize offload costs, showing that legacy applications can leverage a transparent hardware approach to extract benefits from NDP.