Apple M6 chip FFT speed limited by memory bandwidth not computing power
Bandwidth, Not FLOPS: FFT Kernels, Matrix Units and SAR Imaging on Apple M6
PerformanceHardware Architecture
Summary
Computing the fast Fourier transform (FFT) is important for radar and imaging but can be slow on many devices. The authors found that on Apple’s M6 chip, the real bottleneck is how fast data can be moved from memory, not how fast the chip can do calculations. They showed that keeping data in very fast on-chip registers can make FFT much faster, but trying to use certain matrix units in the GPU doesn’t help. Using the matrix units on the CPU side does help and can speed things up significantly. Overall, this research explains where to focus effort to improve FFT speeds on modern processors.
What this means in practice
- •For radar system engineers: Optimize radar signal processing software by focusing on reducing memory data movement instead of arithmetic operations for better FFT performance on Apple M6 devices.
- •For mobile graphics programmers: Develop faster FFT routines by exploiting on-chip registers and CPU matrix units on Apple M6 chips, improving image processing and scientific computations.
Authors
Mohamed Amine Bergach
Abstract
The fast Fourier transform (FFT) underlies radar, imaging and scientific computing. A classic rule for fast GPU FFTs is to compute the largest block that fits on chip and compose larger transforms from such blocks. We test this rule on Apple's M6 chip, whose GPU performs 29 arithmetic operations in the time it reads one byte from main memory, and which adds matrix units to both its GPU and CPU. Comparing our kernels with Apple's MPSGraph and vDSP libraries and MLX, we find that data movement, not arithmetic, sets FFT speed. Large batches run at or near the main-memory bandwidth limit in every GPU library, and benchmarks that keep data in cache overstate real throughput by up to $3.7\times$. The on-chip rule still predicts where speed collapses: MPSGraph and MLX lose half or more of their speed once a transform outgrows the GPU's 32\,KiB local memory. Keeping such transforms in registers avoids an extra trip through memory: $2.2\times$ faster than MPSGraph, and $4.4\times$ with half-precision storage. The GPU's matrix units do not help, because recasting the FFT as matrix products adds as much arithmetic as they save. The CPU's matrix unit, 47 times faster than its vector units, does: our kernel for it beats vDSP by up to $5.3\times$. End to end, a $4096\times4096$ synthetic aperture radar image takes 8.1\,ms on the GPU (6.0\,ms with half-precision intermediates), $14$--$19\times$ faster than a CPU reference. Running the CPU's matrix unit alongside the GPU gains nothing: both share one memory bandwidth.