Neural Computation, Vol 10, 671-716, Copyright © 1998 by The MIT Press

LETTERS

Constrained Optimization for Neural Map Formation: A Unifying Framework for Weight Growth and Normalization

Laurenz Wiskott and Terrence J. Sejnowski

Computational models of neural map formation can be considered on at least three different levels of abstraction: detailed models including neural activity dynamics, weight dynamics that abstract from the neural activity dynamics by an adiabatic approximation, and constrained optimization from which equations governing weight dynamics can be derived. Constrained optimization uses an objective function, from which a weight growth rule can be derived as a gradient flow, and some constraints, from which normalization rules are derived. In this article, we present an example of how an optimization problem can be derived from detailed nonlinear neural dynamics. A systematic investigation reveals how different weight dynamics introduced previously can be derived from two types of objective function terms and two types of constraints. This includes dynamic link matching as a special case of neural map formation. We focus in particular on the role of coordinate transformations to derive different weight dynamics from the same optimization problem. Several examples illustrate how the constrained optimization framework can help in understanding, generating, and comparing different models of neural map formation. The techniques used in this analyses may also be useful in investigating other types of neural dynamics.

This article has been cited by other articles:

				T. Elliott An Analysis of Synaptic Normalization in a General Class of Hebbian Models Neural Comput., April 1, 2003; 15(4): 937 - 963. [Abstract] [Full Text] [PDF]

				T. Elliott and N. R. Shadbolt Multiplicative Synaptic Normalization and a Nonlinear Hebb Rule Underlie a Neurotrophic Model of Competitive Synaptic Plasticity Neural Comput., June 1, 2002; 14(6): 1311 - 1322. [Abstract] [Full Text]

				R. Kempter, W. Gerstner, and J. L. van Hemmen Intrinsic Stabilization of Output Rates by Spike-Based Hebbian Learning Neural Comput., December 1, 2001; 13(12): 2709 - 2741. [Abstract] [Full Text]