From Sweep to Seam: Interleaved Cross-Block Post-Training Quantization
2026-08-10 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors propose a new method called Interleaved Cross-Block Quantization (ICBQ) to improve how large language models are compressed into very small sizes using two-bit precision. Unlike previous methods that only visit each pair of blocks once, ICBQ revisits the boundary between blocks twice to reduce errors. This helps keep the model's performance better during compression, as shown by experiments where ICBQ improves perplexity compared to the baseline. Their approach also works with different low-bit quantization settings and fits into existing compression workflows.
large language modelspost-training quantizationblock-wise quantizationTransformerperplexityternary quantizationmodel compressionGPTQlow-bit quantizationcross-block quantization
Authors
Achille Jacquemond, Yuma Ichikawa, Akira Sakai
Abstract
Compressing large language models to two bits or fewer is increasingly feasible through block-wise post-training quantization; cross-block variants reconstruct neighboring Transformer blocks within a moving window. In the fixed two-block setting studied here, the matched sequential baseline moves this window through the network once, so errors introduced early in the sweep are not revisited. We propose Interleaved Cross-Block Quantization (ICBQ), a scheduling modification that revisits the boundary pair between consecutive chunks. Each seam pair is refined twice: first at the end of one chunk and again at the start of the next. The method retains the local two-block objective and reuses the calibration inputs of existing block-wise PTQ pipelines. Under stated local contraction and smoothness assumptions, we derive a depth-wise upper-bound comparison in which seam revisits multiply the propagated term while the residual remains bounded independently of depth. In the reported experiments, ICBQ reduces ternary-quantization perplexity relative to the matched Sequential CBQ baseline, yields finite perplexity in configurations where the baseline has severe degradation, and can also be used with 3-bit and 2-bit GPTQ.