Neural Computation, Vol 10, 2085-2101, Copyright © 1998 by The MIT Press

LETTERS

Information Maximization and Independent Component Analysis: Is There a Difference?

D. Obradovic and G. Deco

This article provides a detailed and rigorous analysis of the two commonly used methods for redundancy reduction: linear independent component analysis (ICA) posed as a direct minimization of a suitably chosen redundancy measure and information maximization (InfoMax) of a continuous stochastic signal transmitted through an appropriate nonlinear network. The article shows analytically that ICA based on the Kullback-Leibler information as a redundancy measure and InfoMax lead to the same solution if the parameterization of the output nonlinear functions in the latter method is sufficiently rich. Furthermore, this work discusses the alternative redundancy measures not based on the Kullback-Leibler information distance. The practical issues of applying ICA and InfoMax are also discussed and illustrated on the problem of extracting statistically independent factors from a linear, pixel-by-pixel mixture of images.

This article has been cited by other articles:

				C. Kayser, K. P. Kording, and P. Konig Learning the Nonlinearity of Neurons from Natural Visual Stimuli Neural Comput., August 1, 2003; 15(8): 1751 - 1759. [Abstract] [Full Text]

				N. Ay Locality of Global Stochastic Interaction in Directed Acyclic Networks Neural Comput., December 1, 2002; 14(12): 2959 - 2980. [Abstract] [Full Text]