MONOGRAPHS ON APPLl~[) PROBABILITY AND STATISTICS FINITE MIXTURE DISTRIBUTIONS
MONOGRAPHS ON APPLIED PROBABILITY AND STATISTICS General Editor D.R. COX, FRS Also available in the series Probability, Statistics and Time M.S. Bartlett The Statistical Analysis of Spatial Pattern M.S. Bartlett Stochastic Population Models in Ecology and Epidemiology M.S. Bartlett Risk Theory R.E. Beard, T. Pentikilinen and E. Pesonen Point Processes O.R. Cox and V. Isham Analysis of Binary Data D.R. Cox The Statistical Analysis of Series of Events D.R. Cox and P.A.W. Lewis Queues O.R. Cox and W.L. Smith Stochastic Abundance Models E. Engen The Analysis of Contingency Tables B.S. Everitt Finite Mixture Distributions B.S. Everitt and O.J. Hand Population Genetics W.J. Ewens
Classification A.D. Gordon Monte Carlo Methods J.M. Hammersley and D.C. Handscomb Identification of Outliers D.M. Hawkins Multivariate Analysis in Behavioural Research A.E. Maxwell Applications of Queueing Theory G.F. Newell Some Basic Theoryfor Statistical Inference E.J.G. Pitman Statistical Inference S.D. Silvey Models in Regression and Related Topics P. Sprent Sequential Methods in Statistics G.B. Wetherill
Finite Mixture Distributions B. S. EVERITT Head of Biometrics Unit, Institute of Psychiatry and DJ. HAND Lecturer, Biometrics Unit, Institute of Psychiatry LONDON NEW YORK CHAPMAN AND HALL
First published in 1981 by Chapman and Hall Ltd II New Fetter Lane, London EC4P 4EE Published in the USA by Chapman and Hall in association with Methuen, Inc. 733 Third Avenue, New York. NY 10017 1981 B.S. Everitt and 0.1. Hand Softcover reprint of the hardcover 1st edition 1981 ISBN-13: 978-94-009-5899-9 All rights reserved. No part of this book may be reprinted, or reproduced or utilized in any form or by any electronic, mechanical or other means, now known or hereafter invented, including photocopying and recording, or in any information storage and retrieval system, without permission in writing from the Publisher. British Library Cataloguing in Publication Data Everitt. B S Finite mixture distributions. - (Monographs on applied probability and statistics). I. Distribution (Probability theory) I. Title II. Series 5195'32 QA273.6 80-41131 ISBN-13: 978-94-009-5899-9 e-isbn-13: 978-94-009-5897-5 001: 10.1007/978-94-009-5897-5
Contents Preface page xi 1 General introduction 1 1.1 Introduction 1 1.2 Some applications of finite mixture distributions 2 1.3 Definition 4 1.4 Estimation methods 7 1.4.1 Maximum likelihood 8 1.4.2 Bayesian estimation 11 1.4.3 Inversion and error minimization 13 1.4.4 Other methods 20 1.4.5 Estimating the number of components 22 1.5 Summary 23 2 Mixtures of normal distributions 25 2.1 Introduction 25 2.2 Some descriptive properties of mixtures of normal distributions 27 2.3 Estimating the parameters in normal mixture distributions 30 2.3.1 Method of moments estimation 31 2.3.2 Maximum likelihood estimation 36 2.3.3 Maximum likelihood estimates for grouped data 45 2.3.4 Obtaining initial parameter values for the maximum likelihood estimation algorithms 47 2.3.5 Graphical estimation techniques 48 2.3.6 Other estimation methods 51 2.4 Summary 57 3 Mixtures of exponential and other continuous distributions 59 3.1 Exponential mixtures 59
VIll CONTENTS 3.2 Estimating exponential mixture parameters 60 3.2.1 The method of moments and generalizations 60 3.2.2 Maximum likelihood 73 3.3 Properties of exponential mixtures 79 3.4 Other continuous distributions 80 3.4.1 Non-central chi-squared distribution 81 3.4.2 Non-central F distribution 81 3.4.3 Beta distributions 82 3.4.4 Doubly non-central t distribution 83 3.4.5 Planck's distribution 83 3.4.6 Logistic 84 3.4.7 Laplace 84 3.4.8 Weibull 84 3.4.9 Gamma 86 3.5 Mixtures of different component types 86 3.6 Summary 87 4 Mixtures of discrete distributions 89 4.1 Introduction 89 4.2 Mixtures of binomial distributions 89 4.2.1 Moment estimators for binomial mixtures 90 4.2.2 Maximum likelihood estimators for mixtures of binomial distributions 94 4.2.3 Other estimation methods for mixtures of binomial distributions 97 4.3 Mixtures of Poisson distributions 97 4.3.1 Moment estimators for mixtures of Poisson distributions 98 4.3.2 Maximum likelihood estimators for a Poisson mixture 100 4.4 Mixtures of Poisson and binomial distributions 102 4.5 Mixtures of other discrete distributions 102 4.6 Summary 105 5 Miscellaneous topics 107 5.1 Introduction \07 5.2 Determining the number of components in a mixture 107 5.2.1 Informal diagnostic tools for the detection of mixtures \07 5.2.2 Testing hypotheses on the number of components in a mixture 116
CONTENTS 5.3 Probability density function estimation 5.4 Miscellaneous problems 5.5 Summary References Index ix 118 124 127 129 139
Preface Finite mixture distributions arise in a variety of applications ranging from the length distribution of fish to the content of DNA in the nuclei of liver cells. The literature surrounding them is large and goes back to the end of the last century when Karl Pearson published his well-known paper on estimating the five parameters in a mixture of two normal distributions. In this text we attempt to review this literature and in addition indicate the practical details of fitting such distributions to sample data. Our hope is that the monograph will be useful to statisticians interested in mixture distributions and to research workers in other areas applying such distributions to their data. We would like to express our gratitude to Mrs Bertha Lakey for typing the manuscript. Institute oj Psychiatry University of London 1980 B.S. Everitt D.l Hand