Neural Computation, Vol 11, 193-213, Copyright © 1999 by The MIT Press
Variational Learning in Nonlinear Gaussian Belief Networks
Brendan J. Frey and Geoffrey E. Hinton
We view perceptual tasks such as vision and speech recognition as inference
problems where the goal is to estimate the posterior distribution over
latent variables (e.g., depth in stereo vision) given the sensory input.
The recent flurry of research in independent component analysis exemplifies
the importance of inferring the continuous-valued latent variables of input
data. The latent variables found by this method are linearly related to the
input, but perception requires nonlinear inferences such as classification
and depth estimation. In this article, we present a unifying framework for
stochastic neural networks with nonlinear latent variables. Nonlinear units
are obtained by passing the outputs of linear gaussian units through
various nonlinearities. We present a general variational method that
maximizes a lower bound on the likelihood of a training set and give
results on two visual feature extraction problems. We also show how the
variational method can be used for pattern classification and compare the
performance of these nonlinear networks with other methods on the problem
of handwritten digit recognition.
