Separating signals needs enough varied environments and samples

Separation Is a Riesz Energy: How Many Environments Does ICA Need?

Machine LearningInformation Theory

Summary

Some signals can be hidden inside mixtures of many noises, and how well we can separate them depends on how their strengths vary in different situations called environments. The authors show exactly how many different environments and how many data samples we need as the number of hidden signals grows. They introduce a way to measure the difficulty of separating each signal based on how different its strengths are across environments and prove that with enough well-designed environments, fewer samples are needed. They also test their findings on large simulated data and brain scan data to show it predicts which signals are easy or hard to separate.

What this means in practice

  • For signal processing engineers: Design signal separation setups that know how many different environments and samples are needed for effective separation as signal numbers grow.
  • For neuroimaging analysts: Identify which brain activity components are harder to separate based on their variance patterns across different experimental conditions.

Tested on simulated data.

Authors

Yuda Bi, Yubo Guo, Jiaxiao Cai, Xiaonai Li, Zhuolin Li, Vince D. Calhoun

Abstract

Can independent components be recovered when all of them are Gaussian? Yes, if their variances change across environments; but how many environments, and how many samples as the number of components $d$ grows? We answer both for the joint diagonalization of $K$ covariances. Each component has a signature, its variances across the environments. Separating component $j$ costs the Riesz 2-energy $E_j$ of this signature, a sum, over the other components, of inverse pairwise Fisher informations. For Gaussian sources no estimator in $O(d)$ beats $cE_j/(n+E_j)$ at any $n$ and $d$, and above an explicit sample size a spectral estimator attains $E_j/n$ up to a constant, for every mixing and without the signatures (given a lower bound on the pairwise information). Since signatures live in $K-1$ dimensions, the rate obeys a dimension law: $d^2/n$, $d\log d/n$ and $d/n$ for $K=2$, $3$ and $K\ge4$ well-spread environments. What counts is the dimension the signatures span, not their number, and $\log d$ well-designed environments make $d\log d$ samples suffice. Projecting the covariances onto one linear combination pays $d^2/n$ at first order instead. In simulations up to $d=1000$ the predictions hold and joint diagonalization with $K\ge4$ attains the rate once $n/d$ is in the hundreds; on task fMRI the energy says which components are hard to separate.