HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Porr, B. Articles by Wörgötter, F. Search for Related Content PubMed PubMed Citation Articles by Porr, B. Articles by Wörgötter, F.

Strongly Improved Stability and Faster Convergence of Temporal Sequence Learning by Using Input Correlations Only

B.Porr{at}elec.gla.ac.uk Department of Electronics and Electrical Engineering, University of Glasgow, Glasgow, GT12 8LT, Scotland

Florentin Wörgötter

worgott{at}chaos.gwdg.de Department of Psychology, University of Stirling, Stirling FK9 4LA, Scotland, and Bernstein Center of Computational Neuroscience, University Göttingen, Germany

Currently all important, low-level, unsupervised network learning algorithms follow the paradigm of Hebb, where input and output activity are correlated to change the connection strength of a synapse. However, as a consequence, classical Hebbian learning always carries a potentially destabilizing autocorrelation term, which is due to the fact that every input is in a weighted form reflected in the neuron's output. This self-correlation can lead to positive feedback, where increasing weights will increase the output, and vice versa, which may result in divergence. This can be avoided by different strategies like weight normalization or weight saturation, which, however, can cause different problems. Consequently, in most cases, high learning rates cannot be used for Hebbian learning, leading to relatively slow convergence. Here we introduce a novel correlation-based learning rule that is related to our isotropic sequence order (ISO) learning rule (Porr & Wörgötter, 2003a), but replaces the derivative of the output in the learning rule with the derivative of the reflex input. Hence, the new rule uses input correlations only, effectively implementing strict heterosynaptic learning. This looks like a minor modification but leads to dramatically improved properties. Elimination of the output from the learning rule removes the unwanted, destabilizing autocorrelation term, allowing us to use high learning rates. As a consequence, we can mathematically show that the theoretical optimum of one-shot learning can be reached under ideal conditions with the new rule. This result is then tested against four different experimental setups, and we will show that in all of them, very few (and sometimes only one) learning experiences are needed to achieve the learning goal. As a consequence, the new learning rule is up to 100 times faster and in general more stable than ISO learning.

This article has been cited by other articles:

				B. Porr and F. Worgotter Learning with "Relevance": Using a Third Factor to Stabilize Hebbian Learning Neural Comput., October 1, 2007; 19(10): 2694 - 2719. [Abstract] [Full Text] [PDF]