Neural Comp. NEW Faster Access
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

This Article
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Yang, H. H.
Right arrow Articles by Amari, S.-i.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Yang, H. H.
Right arrow Articles by Amari, S.-i.

Neural Computation, Vol 10, 2137-2157, Copyright © 1998 by The MIT Press

LETTERS

Complexity Issues in Natural Gradient Descent Method for Training Multi-Layer Perceptrons

Howard Hua Yang and Shun-ichi Amari

The natural gradient descent method is applied to train an n-m-1 mul-tilayer perceptron. Based on an efficient scheme to represent the Fisher information matrix for an n-m-1 stochastic multilayer perceptron, a new algorithm is proposed to calculate the natural gradient without inverting the Fisher information matrix explicitly. When the input dimension n is much larger than the number of hidden neurons m, the time complexity of computing the natural gradient is O(n).


This article has been cited by other articles:


Home page
 Neural Comput.Home page
N. N. Schraudolph
Fast Curvature Matrix-Vector Products for Second-Order Gradient Descent
Neural Comput., July 1, 2002; 14(7): 1723 - 1738.
[Abstract] [Full Text] [PDF]

Home page
 Neural Comput.Home page
S.-i. Amari, H. Park, and K. Fukumizu
Adaptive Method of Realizing Natural Gradient Learning for Multilayer Perceptrons
Neural Comput., June 1, 2000; 12(6): 1399 - 1409.
[Abstract] [Full Text]


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
J COGNITIVE NEUROSCIENCE NEURAL COMPUTATION MIT PRESS JOURNALS
Copyright © 1998 by The MIT Press.