Neural Comp. Sign up for ETOCS
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

This Article
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Hochreiter, S.
Right arrow Articles by Schmidhuber, J.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Hochreiter, S.
Right arrow Articles by Schmidhuber, J.

Neural Computation, Vol 11, 679-714, Copyright © 1999 by The MIT Press

LETTERS

Feature Extraction Through LOCOCODE

Sepp Hochreiter and Juergen Schmidhuber

Low-complexity coding and decoding (LOCOCODE) is a novel approach to sensory coding and unsupervised learning. Unlike previous methods, it explicitly takes into account the information-theoretic complexity of the code generator. It computes lococodes that convey information about the input data and can be computed and decoded by low-complexity mappings. We implement LOCOCODE by training autoassociators with flat minimum search, a recent, general method for discovering low-complexity neural nets. It turns out that this approach can unmix an unknown number of independent data sources by extracting a minimal number of low-complexity features necessary for representing the data. Experiments show that unlike codes obtained with standard autoencoders, lococodes are based on feature detectors, never unstructured, usually sparse, and sometimes factorial or local (depending on statistical properties of the data). Although LOCOCODE is not explicitly designed to enforce sparse or factorial codes, it extracts optimal codes for difficult versions of the "bars" benchmark problem, whereas independent component analysis (ICA) and principal component analysis (PCA) do not. It produces familiar, biologically plausible feature detectors when applied to real-world images and codes with fewer bits per pixel than ICA and PCA. Unlike ICA, it does not need to know the number of independent sources. As a preprocessor for a vowel recognition benchmark problem, it sets the stage for excellent classification performance. Our results reveal an interesting, previously ignored connection between two important fields&colon: regularizer research and ICA-related research. They may represent a first step toward unification of regularization and unsupervised learning.


This article has been cited by other articles:


Home page
 Neural Comput.Home page
J. Lucke and C. von der Malsburg
Rapid Processing and Unsupervised Learning in a Model of the Cortical Macrocolumn
Neural Comput., March 1, 2004; 16(3): 501 - 533.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
M. W. Spratling and M. H. Johnson
Preintegration Lateral Inhibition Enhances Unsupervised Learning
Neural Comput., September 1, 2002; 14(9): 2157 - 2179.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
A. Hyvärinen
Complexity Pursuit: Separating Interesting Components from Time Series
Neural Comput., April 1, 2001; 13(4): 883 - 898.
[Abstract] [Full Text]


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
J COGNITIVE NEUROSCIENCE NEURAL COMPUTATION MIT PRESS JOURNALS
Copyright © 1999 by The MIT Press.