Papers for

music generation developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

LatticeSMC improves long sequence generation with better inference compute use

LatticeSMC: Where to Spend Inference-Time Compute in Chunked Sequence Generators

Abstract: Long-form generators for music, motion and video produce sequences chunk by chunk, with each chunk generated by iterative denoising while rewards are defined over the full sequence. Existing inference-time steering methods typically act on one axis at a time: best-of-N at the end, Feynman-Kac steering across denoising steps, or streaming pruning across chunks, and are often compared under unmatched compute or different return rules. We introduce budget-matched chunked steering and propose LatticeSMC, a sampler derived from a Feynman-Kac model on the two-dimensional lattice of chunk index and denoising step. Two telescoping results make its design exact: for chunk-additive rewards, the two axes induce identical weights, so resampling should occur where lookahead is cheapest; for terminal rewards, any prefix score defines an exact intermediate potential, making prefix-evaluable rewards twists with no estimation or extra denoiser calls. LatticeSMC resamples on these potentials at chunk boundaries and, when scoring is free, within chunks, returning either a weighted draw or the best particle. Under matched compute, on music-to-dance diffusion and 40-second text-to-music generation, it raises beat alignment from 0.234 to 0.441 (best-of-N: 0.354) and prompt adherence from 0.470 to 0.560 at 32 particles, while preserving held-out quality. It also retains its advantage on long-range rewards and is preferred by human raters in 60-77 percent of pairwise comparisons. Finally, we show that commitment strength should follow the information in the current potential, while the value of lookahead is predicted by the within-set predictability of future reward.

Fri 2 OctMachine Learning
The gist
Generating long sequences of music, motion, or video chunk by chunk is hard because the best results depend on the whole sequence, not just parts. The authors introduced LatticeSMC, a new method that smartly decides when and where to spend computing power during the inference process to improve results. By organizing the generation steps on a two-dimensional grid and resampling at the right moments, LatticeSMC produces more coherent and aligned outputs. Tests showed it improves musical beat alignment and prompt accuracy while keeping quality stable, and humans often preferred its results.
Open → 2610.02774v1