Methods to measure compression limits of quantized Gaussian signals
Computing the entropy rate of a quantized stationary Gaussian process
Information Theory
Summary
When a smooth, repeating signal is turned into numbers by rounding, it becomes harder to perfectly compress without losing any information. The authors studied how to calculate the minimum possible size for such compressed data when the original signal is a Gaussian type with certain spectral properties. They developed a new formula that works well even when older approaches fail, and a computer simulation method that estimates the same limit accurately. Their work helps understand how close actual compression methods come to the theoretical best.
What this means in practice
- •For data compression engineers: Calculate exact limits on lossless compression rates for signals processed through filtering and quantization to optimize codec design.
- •For audio codec developers: Evaluate how closely audio compression methods approach theoretical entropy limits for quantized Gaussian-like sound signals after filtering.
Authors
Jeremy Magland
Abstract
We consider a stationary Gaussian process observed after uniform quantization. The entropy rate of the resulting integer sequence is the fundamental limit on lossless compression of the quantized signal, but it has no closed form, and the classical high-resolution approximation breaks down whenever the spectral density is small or vanishing on part of the band, as happens routinely after smoothing or filtering. Here we present two methods for computing the rate. The first is an analytical approximation obtained by combining an exact dithering identity with the Kolmogorov-Szegő formula; the quantization noise power acts as a floor on the spectral density, so the rate remains finite where the classical formula fails. The second is a Monte Carlo estimate of the exact rate. Its minimum-phase spectral factor represents the process as a finite moving average of Gaussian innovations, making the quantized sequence a hidden Markov process whose optimal (fully adapted) particle filter is available in closed form; by the Shannon-McMillan-Breiman theorem the rate follows from the filter's log-likelihood on a single long sequence. In experiments across six process families, the approximation agrees with the estimate to within a few millibits per sample over most of the parameter range for quantization steps up to the signal standard deviation, including strongly filtered processes on which the classical formula fails; at the most strongly filtered points the discrepancy grows to a few percent of the rate, and for much coarser steps the estimator should be used. We also compare lossless coders against this limit. At a step of one quarter of the standard deviation, linear predictive coding followed by an entropy coder comes within 2 to 4 percent of the entropy rate and FLAC within 3 to 32 percent, whereas five general-purpose compressors on the raw samples remain 12 to 89 percent above it.