Scatter-gather model simplifies programming on mixed computer systems
SGL: Towards a Bridging Model for Heterogeneous Hierarchical Platforms
Distributed, Parallel, and Cluster ComputingPerformanceProgramming Languages
Summary
Computers now often combine different kinds of processors, like multi-core CPUs and graphics accelerators, making programming complicated. The authors introduce SGL, a simple programming language based on a well-known model called Bulk-Synchronous Parallel, designed to work well on these mixed systems. SGL aims to make it easier to write parallel programs that run efficiently and predictably without the complexity of detailed message passing. It also helps balance work automatically by explicitly managing how data is spread out across processors.
What this means in practice
- •For parallel software developers: Write parallel programs for systems combining CPUs and GPUs with less complex messaging and predictable performance.
- •For high performance computing teams: Manage workload distribution automatically on large multiprocessor systems using explicit data layouts for better efficiency.
Authors
Chong Li, Gaétan Hains
Abstract
This paper introduces the Scatter-Gather parallel-programming and parallel-execution model in the form of a simple imperative Scatter-Gather Language (SGL). Its design is based on past experience with bulk-synchronous parallel (BSP) programming and BSP language design. SGL's novel features are motivated by the last decade's move towards multi-level and heterogeneous parallel architectures involving multi-core processors, graphics accelerators and hierarchical routing networks in the largest multiprocessing systems. The design of SGL is coherent with L. Valiant's Multi-BSP while offering a programming interface that is even simpler than the primitives of Bulk-Synchronous Parallel ML (BSML). SGL appears to cover a large subset of all BSP algorithms while avoiding complex message-passing programming. Like all BSP-inspired systems, it supports predictable, portable and scalable performance. Moreover, SGL's explicit data distribution will allow automatic or programmable load-balancing.