HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Poirazi, P. Articles by Mel, B. W. Search for Related Content PubMed PubMed Citation Articles by Poirazi, P. Articles by Mel, B. W.

Choice and Value Flexibility Jointly Contribute to the Capacity of a Subsampled Quadratic Classifier

Department of Biomedical Engineering, University of Southern California, Los Angeles, CA 90089, U.S.A.

Bartlett W. Mel

Department of Biomedical Engineering, University of Southern California, Los Angeles, CA 90089, U.S.A.

Biophysical modeling studies have suggested that neurons with active dendrites can be viewed as linear units augmented by product terms that arise from interactions between synaptic inputs within the same dendritic subregions. However, the degree to which local nonlinear synaptic interactions could augment the memory capacity of a neuron is not known in a quantitative sense. To approach this question, we have studied the family of subsampled quadratic classifiers: linear classifiers augmented by the best k terms from the set of K=(d²+d)/2 second-order product terms available in d dimensions. We developed an expression for the total parameter entropy, whose form shows that the capacity of an SQ classifier does not reside solely in its conventional weight values—the explicit memory used to store constant, linear, and higher-order coefficients. Rather, we identify a second type of parameter flexibility that jointly contributes to an SQ classifier's capacity: the choice as to which product terms are included in the model and which are not. We validate the form of the entropy expression using empirical studies of relative capacity within families of geometrically isomorphic SQ classifiers. Our results have direct implications for neurobiological (and other hardware) learning systems, where in the limit of high-dimensional input spaces and low-resolution synaptic weight values, this relatively little explored form of choice flexibility could constitute a major source of trainable model capacity.

This article has been cited by other articles:

				M. Shamir and H. Sompolinsky Nonlinear Population Codes Neural Comput., June 1, 2004; 16(6): 1105 - 1136. [Abstract] [Full Text] [PDF]