Supervised and Unsupervised Classification Using Mixture Models

¹ Inria Grenoble Rhône-Alpes & Laboratoire Jean Kuntzmann, Grenoble-Alpes University, Grenoble, France
² Bordeaux Institute of Technology (Bordeaux INP) & Inria Bordeaux Sud Ouest, CQFD team & Bordeaux Institute of Mathematics (IMB, UMR 5251 CNRS), France

Abstract

This chapter is dedicated to model-based supervised and unsupervised classification. Probability distributions are defined over possible labels as well as over the observations given the labels. To this end, the basic tools are the mixture models. This methodology yields a posterior distribution over the labels given the observations which allows to quantify the uncertainty of the classification. The role of Gaussian mixture models is emphasized leading to Linear Discriminant Analysis and Quadratic Discriminant Analysis methods. Some links with Fisher Discriminant Analysis and logistic regression are also established. The Expectation-Maximization algorithm is introduced and compared to the K-means clustering method. The methods are illustrated both on simulated datasets as well as on real datasets using the R software.