Neural Comp. Sign up for ETOCS
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

This Article
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Bell, A. J.
Right arrow Articles by Sejnowski, T. J.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Bell, A. J.
Right arrow Articles by Sejnowski, T. J.

Neural Computation, Vol 7, 1129-1159, Copyright © 1995 by The MIT Press

ARTICLES

An information-maximization approach to blind separation and blind deconvolution

AJ Bell and TJ Sejnowski
Howard Hughes Medical Institute, Computational Neurobiology Laboratory, Salk Institute, La Jolla, CA 92037, USA.

We derive a new self-organizing learning algorithm that maximizes the information transferred in a network of nonlinear units. The algorithm does not assume any knowledge of the input distributions, and is defined here for the zero-noise limit. Under these conditions, information maximization has extra properties not found in the linear case (Linsker 1989). The nonlinearities in the transfer function are able to pick up higher-order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation. This enables the network to separate statistically independent components in the inputs: a higher-order generalization of principal components analysis. We apply the network to the source separation (or cocktail party) problem, successfully separating unknown mixtures of up to 10 speakers. We also show that a variant on the network architecture is able to perform blind deconvolution (cancellation of unknown echoes and reverberation in a speech signal). Finally, we derive dependencies of information transfer on time delays. We suggest that information maximization provides a unifying framework for problems in "blind" signal processing.


This article has been cited by other articles:


Home page
 Neural Comput.Home page
M. M. Van Hulle
Sequential Fixed-Point ICA Based on Mutual Information Minimization
Neural Comput., May 1, 2008; 20(5): 1344 - 1365.
[Abstract] [Full Text] [PDF]

Home page
 Proc. Natl. Acad. Sci. USAHome page
T. Eichele, S. Debener, V. D. Calhoun, K. Specht, A. K. Engel, K. Hugdahl, D. Y. von Cramon, and M. Ullsperger
Prediction of human errors by maladaptive changes in event-related brain networks
PNAS, April 22, 2008; 105(16): 6173 - 6178.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurosci.Home page
S. A. Overduin, A. d'Avella, J. Roh, and E. Bizzi
Modulation of Muscle Synergy Recruitment in Primate Grasping
J. Neurosci., January 23, 2008; 28(4): 880 - 892.
[Abstract] [Full Text] [PDF]

Home page
 BioinformaticsHome page
D. Mantini, F. Petrucci, P. Del Boccio, D. Pieragostino, M. Di Nicola, A. Lugaresi, G. Federici, P. Sacchetta, C. Di Ilio, and A. Urbani
Independent component analysis for the extraction of reliable protein signal profiles from MALDI-TOF mass spectra
Bioinformatics, January 1, 2008; 24(1): 63 - 70.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurosci.Home page
A. Delorme, M. Westerfield, and S. Makeig
Medial Prefrontal Theta Bursts Precede Rapid Motor Responses during Visual Selective Attention
J. Neurosci., October 31, 2007; 27(44): 11949 - 11959.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
X. Miao and R. P. N. Rao
Learning the Lie Groups of Visual Invariance
Neural Comput., October 1, 2007; 19(10): 2665 - 2693.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
A. S. Klyubin, D. Polani, and C. L. Nehaniv
Representations of space and time in the maximization of information flow in the perception-action loop.
Neural Comput., September 1, 2007; 19(9): 2387 - 2432.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
C.-H. Zheng, D.-S. Huang, K. Li, G. Irwin, and Z.-L. Sun
MISEP Method for Postnonlinear Blind Source Separation.
Neural Comput., September 1, 2007; 19(9): 2557 - 2578.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
R. Baragona and F. Battaglia
Outliers detection in multivariate time series by independent component analysis.
Neural Comput., July 1, 2007; 19(7): 1962 - 1984.
[Abstract] [Full Text] [PDF]

Home page
 Cereb CortexHome page
J. J. Peissig, J. Singer, K. Kawasaki, and D. L. Sheinberg
Effects of Long-Term Object Familiarity on Event-Related Potentials in the Monkey
Cereb Cortex, June 1, 2007; 17(6): 1323 - 1334.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
J. Triesch
Synergies between intrinsic and synaptic plasticity mechanisms.
Neural Comput., April 1, 2007; 19(4): 885 - 909.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
M. Dyrholm, S. Makeig, and L. K. Hansen
Model Selection for Convolutive ICA with an Application to Spatiotemporal Analysis of EEG.
Neural Comput., April 1, 2007; 19(4): 934 - 955.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
M. M. Van Hulle
Constrained Subspace ICA Based on Mutual Information Optimization Directly
Neural Comput., April 1, 2007; 20(4): 964 - 973.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
T. Toyoizumi, J.-P. Pfister, K. Aihara, and W. Gerstner
Optimality Model of Unsupervised Spike-Timing-Dependent Plasticity: Synaptic Memory and Weight Distribution.
Neural Comput., March 1, 2007; 19(3): 639 - 671.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
L. Xu
One-Bit-Matching Theorem for ICA, Convex-Concave Programming on Polyhedral Set, and Distribution Approximation for Combinatorics.
Neural Comput., February 1, 2007; 19(2): 546 - 569.
[Abstract] [Full Text] [PDF]

Home page
 Cereb CortexHome page
R. M. Siegel, J.-R. Duann, T.-P. Jung, and T. Sejnowski
Spatiotemporal Dynamics of the Functional Architecture for Gain Fields in Inferior Parietal Lobule of Behaving Monkey
Cereb Cortex, February 1, 2007; 17(2): 378 - 390.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurosci.Home page
K. A. Celone, V. D. Calhoun, B. C. Dickerson, A. Atri, E. F. Chua, S. L. Miller, K. DePeau, D. M. Rentz, D. J. Selkoe, D. Blacker, et al.
Alterations in Memory Networks in Mild Cognitive Impairment and Alzheimer's Disease: An Independent Component Analysis
J. Neurosci., October 4, 2006; 26(40): 10222 - 10231.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurophysiol.Home page
N. Krouchev, J. F. Kalaska, and T. Drew
Sequential Activation of Muscle Synergies During Locomotion in the Intact Cat as Revealed by Cluster Analysis and Direct Decomposition
J Neurophysiol, October 1, 2006; 96(4): 1991 - 2010.
[Abstract] [Full Text] [PDF]

Home page
 IEICE Trans FundamentalsHome page
M. TUFAIL, M. ABE, and M. KAWAMATA
An Extension to the Natural Gradient Algorithm for Robust Independent Component Analysis in the Presence of Outliers
IEICE Trans A: Fundamentals, September 1, 2006; E89-A(9): 2429 - 2432.
[Abstract] [PDF]

Home page
 Neural Comput.Home page
N. Slonim, N. Friedman, and N. Tishby
Multivariate Information Bottleneck
Neural Comput., August 1, 2006; 18(8): 1739 - 1789.
[Abstract] [Full Text] [PDF]

Home page
 NeuroscientistHome page
Y. P. Ivanenko, R. E. Poppele, and F. Lacquaniti
Motor Control Programs and Walking
Neuroscientist, August 1, 2006; 12(4): 339 - 348.
[Abstract] [PDF]

Home page
 J. Neurosci.Home page
H. Asari, B. A. Pearlmutter, and A. M. Zador
Sparse representations for the cocktail party problem.
J. Neurosci., July 12, 2006; 26(28): 7477 - 7490.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
J.-P. Pfister, T. Toyoizumi, D. Barber, and W. Gerstner
Optimal spike-timing-dependent plasticity for precise action potential firing in supervised learning.
Neural Comput., June 1, 2006; 18(6): 1318 - 1348.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurophysiol.Home page
G. Cappellini, Y. P. Ivanenko, R. E. Poppele, and F. Lacquaniti
Motor Patterns in Human Walking and Running
J Neurophysiol, June 1, 2006; 95(6): 3426 - 3437.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurophysiol.Home page
B. B. Averbeck and D. Lee
Effects of Noise Correlations on Information Encoding and Decoding
J Neurophysiol, June 1, 2006; 95(6): 3633 - 3644.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurophysiol.Home page
K. H. Knuth, A. S. Shah, W. A. Truccolo, M. Ding, S. L. Bressler, and C. E. Schroeder
Differentially Variable Component Analysis: Identifying Multiple Evoked Components Using Trial-to-Trial Variability
J Neurophysiol, May 1, 2006; 95(5): 3257 - 3276.
[Abstract] [Full Text] [PDF]

Home page
 GutHome page
A M Drewes, S A K Sami, G Dimcevski, K D Nielsen, P Funch-Jensen, M Valeriani, and L Arendt-Nielsen
Cerebral processing of painful oesophageal stimulation: a study based on independent component analysis of the EEG
Gut, May 1, 2006; 55(5): 619 - 629.
[Abstract] [Full Text] [PDF]

Home page
 Journal of Vibration and ControlHome page
P. W. Tse, J. Y. Zhang, and X. J. Wang
Blind Source Separation and Blind Equalization Algorithms for Mechanical Signal Separation and Identification
Journal of Vibration and Control, April 1, 2006; 12(4): 395 - 423.
[Abstract] [PDF]

Home page
 J. Neurophysiol.Home page
M. C. Tresch, V. C. K. Cheung, and A. d'Avella
Matrix Factorization Algorithms for the Identification of Muscle Synergies: Evaluation on Simulated and Experimental Data Sets
J Neurophysiol, April 1, 2006; 95(4): 2199 - 2212.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
X.-L. Zhu, X.-D. Zhang, and J.-M. Ye
A generalized contrast function and stability analysis for overdetermined blind separation of instantaneous mixtures.
Neural Comput., March 1, 2006; 18(3): 709 - 728.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
M. C. Ozturk, D. Xu, and J. C. Principe
Analysis and Design of Echo State Networks
Neural Comput., January 1, 2006; 19(1): 111 - 138.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
Y. Matsuda and K. Yamaguchi
Linear Multilayer ICA Generating Hierarchical Edge Detectors
Neural Comput., January 1, 2006; 19(1): 218 - 230.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
T. Wennekers and N. Ay
Finite State Automata Resulting from Temporal Information Maximization and a Temporal Learning Rule
Neural Comput., October 1, 2005; 17(10): 2258 - 2290.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
K. B. Petersen, O. Winther, and L. K. Hansen
On the Slow Convergence of EM and VBEM in Low-Noise Linear Models
Neural Comput., September 1, 2005; 17(9): 1921 - 1926.
[Abstract] [Full Text] [PDF]

Home page
 IEICE Trans FundamentalsHome page
H. SARUWATARI, H. YAMAJO, T. TAKATANI, T. NISHIKAWA, and K. SHIKANO
Blind Separation and Deconvolution for Convolutive Mixture of Speech Combining SIMO-Model-Based ICA and Multichannel Inverse Filtering
IEICE Trans A: Fundamentals, September 1, 2005; E88-A(9): 2387 - 2400.
[Abstract] [PDF]

Home page
 J. Neurosci.Home page
Y. P. Ivanenko, G. Cappellini, N. Dominici, R. E. Poppele, and F. Lacquaniti
Coordination of Locomotion with Voluntary Movements in Humans
J. Neurosci., August 3, 2005; 25(31): 7238 - 7253.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurosci.Home page
V. C. K. Cheung, A. d'Avella, M. C. Tresch, and E. Bizzi
Central and Sensory Contributions to the Activation and Organization of Muscle Synergies during Natural Motor Behaviors
J. Neurosci., July 6, 2005; 25(27): 6419 - 6434.
[Abstract] [Full Text] [PDF]

Home page
 IEICE Trans FundamentalsHome page
S. MAKINO, H. SAWADA, R. MUKAI, and S. ARAKI
Blind Source Separation of Convolutive Mixtures of Speech in Frequency Domain
IEICE Trans A: Fundamentals, July 1, 2005; E88-A(7): 1640 - 1655.
[Abstract] [PDF]

Home page
 IEICE Trans FundamentalsHome page
A. ANDO, M. IWAKI, K. ONO, and K. KUROZUMI
Separation of Sound Sources Propagated in the Same Direction
IEICE Trans A: Fundamentals, July 1, 2005; E88-A(7): 1665 - 1672.
[Abstract] [PDF]

Home page
 IEICE Trans FundamentalsHome page
T. TAKATANI, S. UKAI, T. NISHIKAWA, H. SARUWATARI, and K. SHIKANO
A Self-Generator Method for Initial Filters of SIMO-ICA Applied to Blind Separation of Binaural Sound Mixtures
IEICE Trans A: Fundamentals, July 1, 2005; E88-A(7): 1673 - 1682.
[Abstract] [PDF]

Home page
 J. Neurophysiol.Home page
B. Feige, K. Scheffler, F. Esposito, F. Di Salle, J. Hennig, and E. Seifritz
Cortical and Subcortical Correlates of Electroencephalographic Alpha Rhythm Modulation
J Neurophysiol, May 1, 2005; 93(5): 2864 - 2872.
[Abstract] [Full Text] [PDF]

Home page
 Proc. Natl. Acad. Sci. USAHome page
T. Toyoizumi, J.-P. Pfister, K. Aihara, and W. Gerstner
Generalized Bienenstock-Cooper-Munro rule for spiking neurons that maximizes information transmission
PNAS, April 5, 2005; 102(14): 5239 - 5244.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
S. Fiori
Nonlinear Complex-Valued Extensions of Hebbian Learning: An Essay
Neural Comput., April 1, 2005; 17(4): 779 - 838.
[Abstract] [Full Text] [PDF]

Home page
 StrokeHome page
D. C. Alsop, E. Makovetskaya, S. Kumar, M. Selim, and G. Schlaug
Markedly Reduced Apparent Blood Volume on Bolus Contrast Magnetic Resonance Imaging as a Predictor of Hemorrhage After Thrombolytic Therapy for Acute Ischemic Stroke
Stroke, April 1, 2005; 36(4): 746 - 750.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
Y. Xia and G. Feng
On Convergence Conditions of an Extended Projection Neural Network
Neural Comput., March 1, 2005; 17(3): 515 - 525.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
S. Xie, Z. He, and Y. Fu
A Note on Stone's Conjecture of Blind Signal Separation
Neural Comput., February 1, 2005; 17(2): 321 - 330.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
J. Ma, Z. Liu, and L. Xu
A Further Result on the ICA One-Bit-Matching Conjecture
Neural Comput., February 1, 2005; 17(2): 331 - 334.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
Y. Karklin and M. S. Lewicki
A Hierarchical Bayesian Model for Learning Nonlinear Statistical Regularities in Nonstationary Natural Signals
Neural Comput., February 1, 2005; 17(2): 397 - 423.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
K. Zhang and L.-W. Chan
Extended Gaussianization Method for Blind Separation of Post-Nonlinear Mixtures
Neural Comput., February 1, 2005; 17(2): 425 - 452.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
S. Osindero, M. Welling, and G. E. Hinton
Topographic Product Models Applied to Natural Scene Statistics
Neural Comput., February 1, 2005; 18(2): 381 - 414.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
S.-i. Maeda, W.-J. Song, and S. Ishii
Nonlinear and Noisy Extension of Independent Component Analysis: Theory and Its Application to a Pitch Sensation Model
Neural Comput., January 1, 2005; 17(1): 115 - 144.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
Md. N. H. Mollah, M. Minami, and S. Eguchi
Exploring Latent Structure of Mixture ICA Models by the Minimum {beta}-Divergence Method
Neural Comput., January 1, 2005; 18(1): 166 - 190.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
K. Zhang and L.-W. Chan
An Adaptive Method for Subband Decomposition ICA
Neural Comput., January 1, 2005; 18(1): 191 - 223.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
F. J. Theis
A New Concept for Separability Problems in Blind Source Separation
Neural Comput., September 1, 2004; 16(9): 1827 - 1850.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
J.-M. Ye, X.-L. Zhu, and X.-D. Zhang
Adaptive Blind Separation with an Unknown Number of Sources
Neural Comput., August 1, 2004; 16(8): 1641 - 1660.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurosci.Home page
C. B. Hart and S. F. Giszter
Modular Premotor Drives and Unit Bursts as Primitives for Frog Motor Behaviors
J. Neurosci., June 2, 2004; 24(22): 5269 - 5282.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurosci.Home page
J.-M. Fellous, P. H. E. Tiesinga, P. J. Thomas, and T. J. Sejnowski
Discovering Spike Patterns in Neuronal Responses
J. Neurosci., March 24, 2004; 24(12): 2989 - 3001.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
Z.-Y. Liu, K.-C. Chiu, and L. Xu
One-Bit-Matching Conjecture for Independent Component Analysis
Neural Comput., February 1, 2004; 16(2): 383 - 399.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
G. de A. Barreto, A. F. R. Araujo, and S. C. Kremer
A Taxonomy for Spatiotemporal Connectionist Networks Revisited: The Unsupervised Case
Neural Comput., June 1, 2003; 15(6): 1255 - 1320.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurosci.Home page
R. Gutig, R. Aharonov, S. Rotter, and H. Sompolinsky
Learning Input Correlations through Nonlinear Temporally Asymmetric Hebbian Plasticity
J. Neurosci., May 1, 2003; 23(9): 3697 - 3714.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
K. Kreutz-Delgado, J. F. Murray, B. D. Rao, K. Engan, T.-W. Lee, and T. J. Sejnowski
Dictionary Learning Algorithms for Sparse Representation
Neural Comput., February 1, 2003; 15(2): 349 - 396.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
E. Doi, T. Inui, T.-W. Lee, T. Wachtler, and T. J. Sejnowski
Spatiochromatic Receptive Field Properties Derived from Information-Theoretic Analyses of Cone Mosaic Responses to Natural Scenes
Neural Comput., February 1, 2003; 15(2): 397 - 417.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
F. J. Theis, A. Jung, C. G. Puntonet, and E. W. Lang
Linear Geometric ICA: Fundamentals and Algorithms
Neural Comput., February 1, 2003; 15(2): 419 - 439.
[Abstract] [Full Text] [PDF]

Home page
 Cereb CortexHome page
S. Zeki, R.J. Perry, and A. Bartels
The Processing of Kinetic Contours in the Brain
Cereb Cortex, February 1, 2003; 13(2): 189 - 202.
[Abstract] [Full Text] [PDF]