HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Ueda, N. Articles by Hinton, G. E. Search for Related Content PubMed PubMed Citation Articles by Ueda, N. Articles by Hinton, G. E.

SMEM Algorithm for Mixture Models

NTT Communication Science Laboratories, Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0237 Japan

Ryohei Nakano

NTT Communication Science Laboratories, Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0237 Japan

Zoubin Ghahramani

Gatsby Computational Neuroscience Unit, University College London, London WC1N 3AR, U.K.

Geoffrey E. Hinton

Gatsby Computational Neuroscience Unit, University College London, London WC1N 3AR, U.K.

Correspondence: Present address: Nagoya Institute of Technology, Gokiso-cho, Showa-Ku, Nagoya 466–8555 Japan

We present a split-and-merge expectation-maximization (SMEM) algorithm to overcome the local maxima problem in parameter estimation of finite mixture models. In the case of mixture models, local maxima often involve having too many components of a mixture model in one part of the space and too few in another, widely separated part of the space. To escape from such configurations, we repeatedly perform simultaneous split-and-merge operations using a new criterion for efficiently selecting the split-and-merge candidates. We apply the proposed algorithm to the training of gaussian mixtures and mixtures of factor analyzers using synthetic and real data and show the effectiveness of using the split-and-merge operations to improve the likelihood of both the training data and of held-out test data. We also show the practical usefulness of the proposed algorithm by applying it to image compression and pattern recognition problems.

This article has been cited by other articles:

				A. S. Tolias, A. S. Ecker, A. G. Siapas, A. Hoenselaar, G. A. Keliris, and N. K. Logothetis Recording Chronically From the Same Neurons in Awake, Behaving Primates J Neurophysiol, December 1, 2007; 98(6): 3780 - 3790. [Abstract] [Full Text] [PDF]

				S. Vijayakumar, A. D'Souza, and S. Schaal Incremental Online Learning in High Dimensions Neural Comput., December 1, 2005; 17(12): 2602 - 2634. [Abstract] [Full Text] [PDF]

				Z. Chen, S. Becker, J. Bondy, I. C. Bruce, and S. Haykin A Novel Model-Based Hearing Compensation Design Using a Gradient-Free Optimization Method Neural Comput., December 1, 2005; 17(12): 2648 - 2671. [Abstract] [Full Text] [PDF]

				J. J. Verbeek, N. Vlassis, and B. Krose Efficient Greedy Learning of Gaussian Mixture Models Neural Comput., February 1, 2003; 15(2): 469 - 485. [Abstract] [Full Text] [PDF]

				M. K. Titsias and A. Likas Mixture of Experts Classification Using a Hierarchical Mixture Model Neural Comput., September 1, 2002; 14(9): 2221 - 2244. [Abstract] [Full Text] [PDF]

				A. Minagawa, N. Tagawa, and T. Tanaka SMEM Algorithm Is Not Fully Compatible with Maximum-Likelihood Framework Neural Comput., June 1, 2002; 14(6): 1261 - 1266. [Abstract] [Full Text]