HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Saito, K. Articles by Nakano, R. Search for Related Content PubMed PubMed Citation Articles by Saito, K. Articles by Nakano, R.

Second-Order Learning Algorithm with Squared Penalty Term

NTT Communication Science Laboratories, Seika-cho, Soraku-gun, Kyoto 619-0237 Japan

Ryohei Nakano

NTT Communication Science Laboratories, Seika-cho, Soraku-gun, Kyoto 619-0237 Japan

Correspondence: Present address: Nagoya Institute of Technology, Gokiso-cho, Showa-Ku, Nagoya 466-8555, Japan

This article compares three penalty terms with respect to the efficiency of supervised learning, by using first- and second-order off-line learning algorithms and a first-order on-line algorithm. Our experiments showed that for a reasonably adequate penalty factor, the combination of the squared penalty term and the second-order learning algorithm drastically improves the convergence performance in comparison to the other combinations, at the same time bringing about excellent generalization performance. Moreover, in order to understand how differently each penalty term works, a function surface evaluation is described. Finally, we show how cross validation can be applied to find an optimal penalty factor.

This article has been cited by other articles:

				T. Karkkainen and E. Heikkola Robust Formulations for Training Multilayer Perceptrons Neural Comput., April 1, 2004; 16(4): 837 - 862. [Abstract] [Full Text] [PDF]

				T. Karkkainen MLP in Layer-Wise Form with Applications to Weight Decay Neural Comput., June 1, 2002; 14(6): 1451 - 1480. [Abstract] [Full Text]