Neural Computation, Vol 8, 583-593, Copyright © 1996 by The MIT Press
Predictive minimum description length criterion for time series modeling with neural networks
M Lehtokangas, J Saarinen, P Huuhtanen and K Kaski
Microelectronics Laboratory, Tampere University of Technology, Finland.
Nonlinear time series modeling with a multilayer perceptron network is
presented. An important aspect of this modeling is the model selection,
i.e., the problem of determining the size as well as the complexity of the
model. To overcome this problem we apply the predictive minimum description
length (PMDL) principle as a minimization criterion. In the neural network
scheme it means minimizing the number of input and hidden units. Three time
series modeling experiments are used to examine the usefulness of the PMDL
model selection scheme. A comparison with the widely used cross-validation
technique is also presented. In our experiments the PMDL scheme and the
cross-validation scheme yield similar results in terms of model complexity.
However, the PMDL method was found to be two times faster to compute. This
is significant improvement since model selection in general is very time
consuming.
