HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Rodriguez, P. Search for Related Content PubMed PubMed Citation Articles by Rodriguez, P.

Simple Recurrent Networks Learn Context-Free and Context-Sensitive Languages by Counting

Department of Cognitive Science, University of California at San Diego, La Jolla, CA 92093, U.S.A.

It has been shown that if a recurrent neural network (RNN) learns to process a regular language, one can extract a finite-state machine (FSM) by treating regions of phase-space as FSM states. However, it has also been shown that one can construct an RNN to implement Turing machines by using RNN dynamics as counters. But how does a network learn languages that require counting? Rodriguez, Wiles, and Elman (1999) showed that a simple recurrent network (SRN) can learn to process a simple context-free language (CFL) by counting up and down. This article extends that to show a range of language tasks in which an SRN develops solutions that not only count but also copy and store counting information. In one case, the network stores information like an explicit storage mechanism. In other cases, the network stores information more indirectly in trajectories that are sensitive to slight displacements that depend on context. In this sense, an SRN can learn analog computation as a set of interdependent counters. This demonstrates how SRNs may be an alternative psychological model of language or sequence processing.

This article has been cited by other articles:

				A. Gruning Elman Backpropagation as Reinforcement for Simple Recurrent Networks Neural Comput., November 1, 2007; 19(11): 3108 - 3131. [Abstract] [Full Text] [PDF]

				G. Capi and K. Doya Evolution of Neural Architecture Fitting Environmental Dynamics Adaptive Behavior, March 1, 2005; 13(1): 53 - 66. [Abstract] [PDF]

				J. Schmidhuber, F. Gers, and D. Eck Learning Nonregular Languages: A Comparison of Simple Recurrent Networks and LSTM Neural Comput., September 1, 2002; 14(9): 2039 - 2041. [Abstract] [Full Text] [PDF]