Papers for
performance optimization teams
Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.
Tool improves reliability of gpu kernel results in large language models
God Does Not Play Dice: Enforcing Batch Invariance in Agentic Kernel Generation
Abstract: Large language model (LLM) serving systems dynamically batch user requests for GPU execution, so the same request may be processed alongside different requests across runs. If the GPU kernels lack batch invariance (BI), this changing batch context can alter the numerical output for an unchanged input, undermining reproducibility. Existing kernel generation agents focus on correctness and efficiency but \emph{overlook BI}. We find 19 confirmed BI violations among 83 batch-applicable released artifacts. To address this limitation, we propose \tool, which combines contract-guided static dependency analysis with dynamic validation of bitwise consistency across batch contexts. \tool is designed as a plug-in component for general kernel agents, allowing their original optimization capabilities to be reused while adding BI enforcement. We integrate it as successive acceptance gates in the optimization loop, rejecting structurally risky or dynamically non-invariant candidates before performance benchmarking and returning feedback to the agent. Experiments across three agent frameworks under baseline and BI-aware generation on real world LLM serving tasks show that \tool improves the BI pass rate from 67.6\% to 91.9\% while retaining 91.8\% of the baseline geometric mean speedup. These results demonstrate that \tool can be incorporated into kernel optimization by agents as an additional reliability objective alongside correctness and speed.
Kubernetes tuning cuts costs using expert knowledge and fast solvers
Domain-Knowledge Driven Configuration Tuning of Kubernetes Workloads
Abstract: Industrial software applications often require engineers to tune the software deployment platform. Configuration tuning identifies valuable configurations that minimize costs while ensuring the application performance. However, finding these configurations is difficult. The number of tunable configuration parameters in industrial applications is large, and engineers cannot assess different configurations in production because users may experience performance degradation. This work targets Kubernetes: A system for automating the deployment, scaling, and management of containerized applications. This paper proposes a domain-knowledge-driven solution for finding configurations of industrial applications in a practical time. The solution relies on a catalog of constraints encountered by domain experts in similar applications. Depending on the problem under analysis, engineers can customize the optimization problem by selecting the relevant constraints from the catalog for the application at hand. We rely on two alternative solvers to find an optimal configuration: The first is based on heuristics, while the second reuses existing optimization-based solutions. We implemented our solution within an industrial tool from a software company specializing in performance engineering and optimization. We compare our solution to a baseline, considering a set of predefined configuration values provided by engineers. Our results confirm the effectiveness of our solution: Compared to the baseline, the heuristic-based and optimization-based solutions provide significant average savings of 11591 USD/mo and 15874 USD/mo per Kubernetes, respectively, accounting for 24.03% and 32.90% of the costs, in practical time (0.71 seconds per extraction for the heuristic-based and 45.54 seconds per extraction for the optimization-based solution).