Polynomial density neurons clarify exact outputs for missing inputs

Evidence or Marginal? What Moment-Vector Propagation Computes in Polynomial Density Neurons

Machine LearningArtificial Intelligence

Summary

The paper looks at how certain neurons that use polynomial models should respond when some input data is missing or uncertain. The researchers compare two ways to calculate the neuron output: one that conditions on the missing input as evidence, and another that treats the missing input as changing the overall input distribution. They show mathematically which approach corresponds to which interpretation and test this on various data setups. Their findings help clarify how these neurons propagate information when not all inputs are known exactly.

What this means in practice

  • For machine learning engineers: Implement more precise output computation in polynomial-based neural models with uncertain or missing input data.
  • For statistical modelers: Distinguish when to apply Bayesian updates versus marginal adjustments in models handling partial evidence or missing inputs.

Tested on simulated data.

Authors

Przemysław Spyra

Abstract

When a neuron's input is lost, or known through a belief $q$, what should it output? For a neuron storing a joint polynomial density $N$ with input density $D$, two formulas compete. $p_\mathrm{CD}=E_q[N]/E_q[D]$ costs one matrix-vector product; Hierarchical Correlation Reconstruction introduces it as a constant-denominator approximation to $p_\mathrm{EU}=E_q[N/D]$. For a density model both are exact answers to different questions: $p_\mathrm{CD}$ is Bayesian updating with the belief as virtual evidence; $p_\mathrm{EU}$ installs the belief as a new marginal (Jeffrey's rule). With a missing input $p_\mathrm{CD}$ returns the model's conditional; $p_\mathrm{EU}$ generally does not. This post hoc reading is proved, and illustrated outside the pair family up to $k=8$ inputs; it has no pre-registered test, only one exploratory real-data pilot. Post hoc, closed forms fall within a factor 2 of all 112 measured gaps, and negative density begins near input correlation $1/k$. In a pre-registered synthetic study (up to 32 inputs), 5 of 15 hypotheses are supported, 2 inconclusive and 8 not supported, including all three on transfer beyond equicorrelated Gaussians. Post hoc, where $p_\mathrm{EU}$ is defined, truncation usually outweighs the gap for Beta beliefs in pair neurons; triple neurons under flat and data beliefs are exceptions.