HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Yuille, A. L. Articles by Rangarajan, A. Search for Related Content PubMed PubMed Citation Articles by Yuille, A. L. Articles by Rangarajan, A.

The Concave-Convex Procedure

yuille{at}ski.org, Smith-Kettlewell Eye Research Institute, San Francisco, CA 94115, U.S.A.

Anand Rangarajan

anand{at}cise.ufl.edu, Department of Computer and Information Science and Engineering, University of Florida, Gainesville, FL 32611, U.S.A.

The concave-convex procedure (CCCP) is a way to construct discrete-time iterative dynamical systems that are guaranteed to decrease global optimization and energy functions monotonically. This procedure can be applied to almost any optimization problem, and many existing algorithms can be interpreted in terms of it. In particular, we prove that all expectation-maximization algorithms and classes of Legendre minimization and variational bounding algorithms can be reexpressed in terms of CCCP. We show that many existing neural network and mean-field theory algorithms are also examples of CCCP. The generalized iterative scaling algorithm and Sinkhorn's algorithm can also be expressed as CCCP by changing variables. CCCP can be used both as a new way to understand, and prove the convergence of, existing optimization algorithms and as a procedure for generating new algorithms.

This article has been cited by other articles:

				S. Ikeda, T. Tanaka, and S.-i. Amari Stochastic Reasoning, Free Energy, and Information Geometry Neural Comput., September 1, 2004; 16(9): 1779 - 1810. [Abstract] [Full Text] [PDF]