| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Sloan Center for Theoretical Neurobiology and W. M. Keck Foundation Center for Integrative Neuroscience, University of California at San Francisco, San Francisco, CA 94143-0444, U.S.A.
We introduce the independent factor analysis (IFA) method for recovering independent hidden sources from their observed mixtures. IFA generalizes and unifies ordinary factor analysis (FA), principal component analysis (PCA), and independent component analysis (ICA), and can handle not only square noiseless mixing but also the general case where the number of mixtures differs from the number of sources and the data are noisy. IFA is a two-step procedure. In the first step, the source densities, mixing matrix, and noise covariance are estimated from the observed data by maximum likelihood. For this purpose we present an expectation-maximization (EM) algorithm, which performs unsupervised learning of an associated probabilistic model of the mixing situation. Each source in our model is described by a mixture of gaussians; thus, all the probabilistic calculations can be performed analytically. In the second step, the sources are reconstructed from the observed data by an optimal nonlinear estimator. A variational approximation of this algorithm is derived for cases with a large number of sources, where the exact algorithm becomes intractable. Our IFA algorithm reduces to the one for ordinary FA when the sources become gaussian, and to an EM algorithm for PCA in the zero-noise limit. We derive an additional EM algorithm specifically for noiseless IFA. This algorithm is shown to be superior to ICA since it can learn arbitrary source densities from the data. Beyond blind separation, IFA can be used for modeling multidimensional data by a highly constrained mixture of gaussians and as a tool for nonlinear signal encoding.
This article has been cited by other articles:
|
|
 |
|
  R. Turner and M. Sahani A maximum-likelihood interpretation for slow feature analysis. Neural Comput., April 1, 2007; 19(4): 1022 - 1038. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. TUFAIL, M. ABE, and M. KAWAMATA A Characteristic Function Based Contrast Function for Blind Extraction of Statistically Independent Signals IEICE Trans A: Fundamentals, August 1, 2006; E89-A(8): 2149 - 2157. [Abstract] [PDF]
|
|
|
|
 |
|
 M. C. Tresch, V. C. K. Cheung, and A. d'Avella Matrix Factorization Algorithms for the Identification of Muscle Synergies: Evaluation on Simulated and Experimental Data Sets J Neurophysiol, April 1, 2006; 95(4): 2199 - 2212. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. ZHANG, J. LU, and T. YAHAGI Nonlinear Blind Source Separation Method for X-Ray Image Separation IEICE Trans A: Fundamentals, April 1, 2006; E89-A(4): 924 - 931. [Abstract] [PDF]
|
|
|
|
|
|
S. Haykin and Z. Chen The Cocktail Party Problem Neural Comput., September 1, 2005; 17(9): 1875 - 1902. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. Zhang and L.-W. Chan Extended Gaussianization Method for Blind Separation of Post-Nonlinear Mixtures Neural Comput., February 1, 2005; 17(2): 425 - 452. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S.-i. Maeda, W.-J. Song, and S. Ishii Nonlinear and Noisy Extension of Independent Component Analysis: Theory and Its Application to a Pitch Sensation Model Neural Comput., January 1, 2005; 17(1): 115 - 144. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Z.-Y. Liu, K.-C. Chiu, and L. Xu One-Bit-Matching Conjecture for Independent Component Analysis Neural Comput., February 1, 2004; 16(2): 383 - 399. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. Chan, T.-W. Lee, and T. J. Sejnowski Variational Bayesian Learning of ICA with Missing Data Neural Comput., August 1, 2003; 15(8): 1991 - 2011. [Abstract] [Full Text]
|
|
|
|
|
|
K. Kreutz-Delgado, J. F. Murray, B. D. Rao, K. Engan, T.-W. Lee, and T. J. Sejnowski Dictionary Learning Algorithms for Sparse Representation Neural Comput., February 1, 2003; 15(2): 349 - 396. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. A. Choudrey and S. J. Roberts Variational Mixture of Bayesian Independent Component Analyzers Neural Comput., January 1, 2003; 15(1): 213 - 252. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. Valpola and J. Karhunen An Unsupervised Ensemble Learning Method for Nonlinear Dynamic State-Space Models Neural Comput., November 1, 2002; 14(11): 2647 - 2692. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. A.d.F.R. Hojen-Sorensen, O. Winther, and L. K. Hansen Mean-Field Approaches to Independent Component Analysis Neural Comput., April 1, 2002; 14(4): 889 - 918. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Girolami A Variational Method for Learning Sparse and Overcomplete Representations Neural Comput., November 1, 2001; 13(11): 2517 - 2532. [Abstract] [Full Text]
|
|
|
|
|
|
V. Koivunen, M. Enescu, and E. Oja Adaptive Algorithm for Blind Separation from Noisy Time-Varying Mixtures Neural Comput., October 1, 2001; 13(10): 2339 - 2357. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Zibulevsky and B. A. Pearlmutter Blind Source Separation by Sparse Decomposition in a Signal Dictionary Neural Comput., April 1, 2001; 13(4): 863 - 882. [Abstract] [Full Text]
|
|
|
|
|
|
M. Welling and M. Weber A Constrained EM Algorithm for Independent Component Analysis Neural Comput., March 1, 2001; 13(3): 677 - 689. [Abstract] [Full Text]
|
|
|
|
|
|
D. Husmeier The Bayesian Evidence Scheme for Regularizing Probability-Density Estimating Neural Networks Neural Comput., November 1, 2000; 12(11): 2685 - 2717. [Abstract] [Full Text]
|
|
|
|
|
|
R. Everson and S. Roberts Independent Component Analysis: A Flexible Nonlinearity and Decorrelating Manifold Approach Neural Comput., November 15, 1999; 11(8): 1957 - 1983. [Abstract] [Full Text]
|
|
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| J COGNITIVE NEUROSCIENCE | NEURAL COMPUTATION | MIT PRESS JOURNALS |
