HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by de Freitas, J. F. G. Articles by Gee, A. H. Search for Related Content PubMed PubMed Citation Articles by de Freitas, J. F. G. Articles by Gee, A. H.

Hierarchical Bayesian Models for Regularization in Sequential Learning

Cambridge University Engineering Department, Cambridge CB2 1PZ, U.K.

M. Niranjan

Cambridge University Engineering Department, Cambridge CB2 1PZ, U.K.

A. H. Gee

Cambridge University Engineering Department, Cambridge CB2 1PZ, U.K.

We show that a hierarchical Bayesian modeling approach allows us to perform regularization in sequential learning. We identify three inference levels within this hierarchy: model selection, parameter estimation, and noise estimation. In environments where data arrive sequentially, techniques such as cross validation to achieve regularization or model selection are not possible. The Bayesian approach, with extended Kalman filtering at the parameter estimation level, allows for regularization within a minimum variance framework. A multilayer perceptron is used to generate the extended Kalman filter nonlinear measurements mapping. We describe several algorithms at the noise estimation level that allow us to implement on-line regularization. We also show the theoretical links between adaptive noise estimation in extended Kalman filtering, multiple adaptive learning rates, and multiple smoothing regularization coefficients.