Neural Computation, Vol 10, 1481-1505, Copyright © 1998 by The MIT Press
Extended Kalman Filter-Based Pruning Method for Recurrent Neural Networks
John Sum, Lai-wan Chan, Chi-sing Leung and Gilbert H. Young
Pruning is one of the effective techniques for improving the generalization
error of neural networks. Existing pruning techniques are derived mainly
from the viewpoint of energy minimization, which is commonly used in
gradient-based learning methods. In recurrent networks, extended Kalman
filter (EKF)-based training has been shown to be superior to gradient-based
learning methods in terms of speed. This article explains a pruning
procedure for recurrent neural networks using EKF training. The sensitivity
of a posterior probability is used as a measure of the importance of a
weight instead of error sensitivity since posterior probability density is
readily obtained from this training method. The pruning procedure is tested
using three problems: (1) the prediction of a simple linear time series,
(2) the identification of a nonlinear system, and (3) the prediction of an
exchange-rate time series. Simulation results demonstrate that the proposed
pruning method is able to reduce the number of parameters and improve the
generalization ability of a recurrent network.
