Neural Computation, Vol 9, 441-460, Copyright © 1997 by The MIT Press
Average-Case Learning Curves for Radial Basis Function Networks
Sean B. Holden and Mahesan Niranjan
The application of statistical physics to the study of the learning curves
of feedforward connectionist networks has to date been concerned mostly
with perceptron-like networks. Recent work has extended the theory to
networks such as committee machines and parity machines, and an important
direction for current and future research is the extension of this body of
theory to further connectionist networks. In this article, we use this
formalism to investigate the learning curves of gaussian radial basis
function networks (RBFNs) having fixed basis functions. (These networks
have also been called generalized linear regression models.) We address
the problem of learning linear and nonlinear, realizable and unrealizable,
target rules from noise-free training examples using a stochastic training
algorithm. Expressions for the generalization error, defined as the
expected error for a network with a given set of parameters, are derived
for general gaussian RBFNs, for which all parameters, including centers and
spread parameters, are adaptable. Specializing to the case of RBFNs with
fixed basis functions (basis functions having parameters chosen without
reference to the training examples), we then study the learning curves for
these networks in the limit of high temperature.
