HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Full Text Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Silva, L. M. Articles by Sá, J. M. d. Search for Related Content PubMed PubMed Citation Articles by Silva, L. M. Articles by Sá, J. M. d.

Error Entropy in Classification Problems: A Univariate Data Analysis

lmsilva{at}fe.up.pt

Carlos S. Felgueiras

casf{at}fe.up.pt Instituto de Engenharia Biomédica, Laboratório Sinal e Imagem Biomédica, 4200-465, Porto, Portugal

Luís A. Alexandre

lfbaa{at}di.ubi.pt Departamento de Informática, Universidade da Beira Interior, Covilhã, Portugal, and Instituto de Telecomunicações, Networks and Multimedia Group, Covilhã, Portugal

J. Marques de Sá

jmsa{at}fe.up.pt Instituto de Engenharia Biomédica, Laboratório Sinal e Imagem Biomédica, 4200-465, Porto, Portugal, and Faculdade de Engenharia da Universidade do Porto, Departamento de Engenharia Electrotécnica e Computadores, 4200-465, Porto, Portugal

Entropy-based cost functions are enjoying a growing attractiveness in unsupervised and supervised classification tasks. Better performances in terms both of error rate and speed of convergence have been reported. In this letter, we study the principle of error entropy minimization (EEM) from a theoretical point of view. We use Shannon's entropy and study univariate data splitting in two-class problems. In this setting, the error variable is a discrete random variable, leading to a not too complicated mathematical analysis of the error entropy. We start by showing that for uniformly distributed data, there is equivalence between the EEM split and the optimal classifier. In a more general setting, we prove the necessary conditions for this equivalence and show the existence of class configurations where the optimal classifier corresponds to maximum error entropy. The presented theoretical results provide practical guidelines that are illustrated with a set of experiments with both real and simulated data sets, where the effectiveness of EEM is compared with the usual mean square error minimization.