Performance and portability tradeoffs in running scientific software on HPC and cloud
Performance vs Portability in Heterogeneous HPC Environments: Why Pre-execution Benchmarking is Required
Distributed, Parallel, and Cluster Computing
Summary
Running scientific programs on different computer systems can be tricky because they use different setups and tools. The authors studied how running quantum chemistry software inside containers compares to running it directly on high-performance computers. They found that how the software is built, how long it takes to start, and how it’s run affect how fast it works. For short tasks, setting up the container can take more time than running the calculation itself, while longer tasks need careful monitoring. This shows the need to test software performance before using it in these mixed environments.
What this means in practice
- •For hpc system administrators: Determine workload-specific setup and execution strategies to optimize scientific software performance in containerized HPC environments.
- •For cloud infrastructure engineers: Coordinate deployment approaches across cloud and HPC platforms to balance software portability and computational efficiency.
Authors
Mindaugas Macernis
Abstract
Cloud computing and high-performance computing (HPC) typically follow different paradigms: cloud services are often orchestrated using Kubernetes, whereas HPC workloads are managed through batch schedulers such as Slurm. Growing demand for shared computational resources increases the need for interoperability between these environments. This study uses Podman as a user-accessible tool to benchmark quantum chemistry software distributed as precompiled executables. Containerized and on-system execution are compared across CPU-optimized builds and multiple compute nodes, without additional network optimizations. The results demonstrate that performance depends on software compilation, startup overhead, and execution configuration. Container preparation can dominate short-task workflows, while long-running calculations require monitoring of computational progress. These findings highlight the importance of workload-specific benchmarking and user-level monitoring when deploying scientific applications in containers on HPC systems.