Neural Computation, Vol 8, 1677-1710, Copyright © 1996 by The MIT Press

ARTICLES

Hebbian learning of context in recurrent neural networks

N Brunel
Istituto di Fisica, Universita di Roma I La Sapienza, Italy.

Single electrode recording in the inferotemporal cortex of monkeys during delayed visual memory tasks provide evidence for attractor dynamics in the observed region. The persistent elevated delay activities could be internal representations of features of the learned visual stimuli shown to the monkey during training. When uncorrelated stimuli are presented during training in a fixed sequence, these experiments display significant correlations between the internal representations. Recently a simple model of attractor neural network has reproduced quantitatively the measured correlations. An underlying assumption of the model is that the synaptic matrix formed during the training phase contains in its efficacies information about the contiguity of persistent stimuli in the training sequence. We present here a simple unsupervised learning dynamics that produces such a synaptic matrix if sequences of stimuli are repeatedly presented to the network at fixed order. The resulting matrix is then shown to convert temporal correlations during training into spatial correlations between attractors. The scenario is that, in the presence of selective delay activity, at the presentation of each stimulus, the activity distribution in the neural assembly contain information of both the current stimulus and the previous one (carried by the attractor). Thus the recurrent synaptic matrix can code not only for each of the stimuli presented to the network but also for their context. We combine the idea that for learning to be effective, synaptic modification should be stochastic, with the fact that attractors provide learnable information about two consecutive stimuli. We calculate explicitly the probability distribution of synaptic efficacies as a function of training protocol, that is, the order in which stimuli are presented to the network. We then solve for the dynamics of a network composed of integrate-and-fire excitatory and inhibitory neurons with a matrix of synaptic collaterals resulting from the learning dynamics. The network has a stable spontaneous activity, and stable delay activity develops after a critical learning stage. The availability of a learning dynamics makes possible a number of experimental predictions for the dependence of the delay activity distributions and the correlations between them, on the learning stage and the learning protocol. In particular it makes specific predictions for pair-associates delay experiments.

This article has been cited by other articles:

				A. Akrami, Y. Liu, A. Treves, and B. Jagadeesh Converging Neuronal Activity in Inferior Temporal Cortex during the Classification of Morphed Stimuli Cereb Cortex, July 31, 2008; (2008) bhn125v1. [Abstract] [Full Text] [PDF]

				A. Bernacchia and D. J. Amit Impact of spatiotemporally correlated images on the structure of memory PNAS, February 27, 2007; 104(9): 3544 - 3549. [Abstract] [Full Text] [PDF]

				E. Curti, G. Mongillo, G. La Camera, and D. J. Amit Mean Field and Capacity in Realistic Networks of Spiking Neurons Storing Sparsely Coded Random Memories Neural Comput., December 1, 2004; 16(12): 2597 - 2637. [Abstract] [Full Text] [PDF]

				C. R. Laing and A. Longtin Dynamics of Deterministic and Stochastic Paired Excitatory-Inhibitory Delayed Feedback Neural Comput., December 1, 2003; 15(12): 2779 - 2822. [Abstract] [Full Text] [PDF]

				D. J. Amit and G. Mongillo Selective Delay Activity in the Cortex: Phenomena and Interpretation Cereb Cortex, November 1, 2003; 13(11): 1139 - 1150. [Abstract] [Full Text] [PDF]

				N. Brunel Dynamics and Plasticity of Stimulus-selective Persistent Activity in Cortical Network Models Cereb Cortex, November 1, 2003; 13(11): 1151 - 1161. [Abstract] [Full Text] [PDF]

				D. J. Amit and G. Mongillo Spike-Driven Synaptic Dynamics Generating Working Memory States Neural Comput., March 1, 2003; 15(3): 565 - 596. [Abstract] [Full Text] [PDF]

				F. van der Velde and M. de Kamps From Knowing What to Knowing Where: Modeling Object-Based Attention with Feedback Disinhibition of Activation J. Cogn. Neurosci., May 1, 2001; 13(4): 479 - 491. [Abstract] [Full Text]