Optimizing ML Workload Partitioning between CPUs and CIM Accelerators for Heterogeneous Computing
2026-07-06 • Emerging Technologies
Emerging TechnologiesArtificial IntelligenceHardware ArchitectureDistributed, Parallel, and Cluster ComputingMachine Learning
AI summaryⓘ
The authors developed a new method to split machine learning tasks between regular CPUs and special memory units called CIM accelerators that do math inside memory. Their approach considers real limits of the memory hardware, like slow writing and limited usage cycles, which previous methods ignored. They use a math-based optimization to reduce the time it takes to run tasks by carefully sharing work between CPU and memory units. Their method speeds up processing by up to about 31 times compared to using only the CPU, and also gives ideas for designing better future systems.
Computing-in-Memory (CIM)Matrix-Vector Multiplication (MVM)Resistive RAM (RRAM)Integer Linear Programming (ILP)Workload PartitioningInference LatencyHeterogeneous ComputingDesign Space Exploration (DSE)Machine Learning AcceleratorsEdge Computing
Authors
Joel Klein, Rebecca Pelke, Roberto Laudani, Jan Moritz Joseph, Rainer Leupers
Abstract
Computing-in-Memory (CIM) accelerators execute Matrix-Vector Multiplications (MVMs) in memory, making them a compelling solution for Machine Learning (ML) workloads. However, existing ML workload partitioning approaches for CIM accelerators do not fully account for Resistive Random Access Memory (RRAM) constraints such as limited memory, high write latency, and limited endurance. They also neglect parallelism, low-level architectural effects, or the Central Processing Unit (CPU) as a complementary compute resource. To address these limitations, we propose an Integer Linear Programming (ILP)-based workload partitioning framework for heterogeneous CPU-CIM systems. It minimizes end-to-end inference latency under RRAM constraints, captures parallelism, and combines empirical profiling with analytical models. Using our framework, heterogeneous CPU-CIM execution achieves speedups of up to 30.9x over CPU-only execution on an edge CPU and 7.3x over a high-performance CPU. A Design Space Exploration (DSE) yields further design insights for future CIM accelerators.