Productivity measures and challenges in high performance computing
Productivity in HPC: A Critical Review from SLOC and Performance Portability to AI-Assisted Programming
Distributed, Parallel, and Cluster Computing
Summary
Measuring productivity in high-performance computing (HPC) has been difficult because there is no standard way to define or quantify it. The authors review how productivity has been assessed, from counting lines of code to looking at code complexity and usability studies. They discuss newer programming tools that use automation and artificial intelligence, which shift human effort from writing code to verifying and tuning it. The paper compares different productivity measurement methods and highlights ongoing challenges in accurately assessing effort and performance in HPC programming.
What this means in practice
- •For hpc software developers: Use insights on productivity metrics to choose programming approaches that reduce effort in writing and tuning HPC code.
- •For compiler and language designers: Apply the review’s findings to improve tools that balance high performance with ease of programming and portability.
A survey. It maps existing work.
Authors
Ami Marowka
Abstract
Productivity has long been recognized as a central concern in high-performance computing (HPC), yet it still lacks a workable formal definition and a broadly accepted metric for quantified assessment. Recent attempts to incorporate productivity into performance-portability analysis have introduced code-divergence, convergence, and semantic measures, but have not established a convincing formal relationship between productivity and performance portability. More recent language and compiler systems, including JACC, Mojo, JLIR, and Accl++, pursue productive performance portability through higher-level abstractions, multi-level intermediate representations, automatic kernel generation, and runtime compilation. These systems demonstrate substantial engineering progress, while productivity is still commonly represented through programmability, ecosystem reuse, code size, or reduced implementation burden rather than direct measurement of human effort. This paper presents a critical review of HPC productivity measurement, tracing its development from source lines of code (SLOC), static complexity, and empirical usability studies to development-time models and semantic approaches. We also examine how automated migration and artificial-intelligence(AI)-assisted programming weaken the correspondence between code transformation and human effort, shifting effort toward verification, correction, and performance tuning. The reviewed methods are compared in terms of what they measure, directness, objectivity, reproducibility, practicality, and their relationship to performance and portability. We conclude by identifying the principal limitations and open challenges for quantified HPC productivity assessment.