Probability and Statistics Volume II
Didier Dacunha-Castelle Marie Duflo Probability and Statistics Volume II Translated by David McHale Springer-Verlag New York Berlin Heidelberg Tokyo
Didier Dacunha-Castelle Universite de Paris-Sud Equipe de Recherche Associee au C.N.R.S. 532 Statistique Applique Mathematique 91405 Orsay Cedex France Marie Duflo Universite de Paris-Nord 93430 Villetaneuse France David McHale (Translator) Linslade, Leighton Buzzard Bedfordshire LU7 7XW United Kingdom With 6 Illustrations AMS Classification 60-01 Library of Congress Cataloging-in-Publication Data Dacunha-Castelle, Didier. Probability and statistics. Translation of: Probabilites et statistiques. Includes bibliographies and index. 1. Probabilities. 2. Mathematical statistics. I. Dufio, Marie. I. Title. QA273.D23 1986 519.2 85-25094 French Edition, "Probabilities et statistiques," Masson, Editeur, Paris, 1983 1986 by Springer-Verlag New York Inc. Softcover reprint of the hardcover I st edition 1986 All rights reserved. No part of this book may be translated or reproduced in any form without written permission from Springer-Verlag, 175 Fifth Avenue, New York, New York 10010, U.S.A. 987654321 ISBN-13: 978-1-4612-9339-2 e-isbn-13: 978-1-4612-4870-5 DOl 10.1007/978-1-4612-4870-5
Note: In this second volume of Probability and Statistics, reference is sometimes made to Volume I. These references appear in the form [Vol. I. X.Y.Z.] and refer to Chapter X, paragraph Y, sub-paragraph Z. Similarly, for the references to Volume II, denoted only [X.Y.Z.].
INTRODUCTION How can we predict the future without asking an astrologer? When a phenomenon is not evolving, experiments can be repeated and observations therefore accumulated; this is what we have done in Volume I. However history does not repeat itself. Prediction of the future can only be based on the evolution observed in the past. Yet certain phenomena are stable enough so that observation in a sufficient interval of time gives usable information on the future or the mechanism of evolution. Technically, the keys to asymptotic statistics are the following: laws of large numbers, central limit theorems, and likelihood calculations. We have sought the shortest route to these theorems by neglecting to present the most general models. The future statistician will use the foundations of the statistics of processes and should satisfy himself about the unity of the methods employed. At the same time, we have adhered as closely as possible to present day ideas of the theory of processes. For those who wish to follow the study of probabilities to postgraduate level, it is not a waste of time to begin with the least difficult technical situations. This book for final year mathematics courses is not the end of the matter. It acts as a springboard either for dealing concretely with the problems of the statistics of processes, or
viii In trod uction to study in depth the more subtle aspects of probabilities. Finally, let us note that a more classical probability course can easily be organized around Chapter 2 which is central, on Chapter 4 on Markov chains and Chapters 5 to 8 for the important parts which do not call on statistical concepts.
CONTENTS CHAPTER 0 Introduction to Random Processes 0.1. Random Evolution Through Time 0.2. Basic Measure Theory 0.3. Convergence in Distribution I 3 9 CHAPTER 1 Time Series I. I. Second Order Processes 1.2. Spatial Processes with Orthogonal Increments 1.3. Stationary Second Order Processes 1.4. Time Series Statistics 13 13 23 29 47 CHAPTER 2 Martingales in Discrete Time 2.1. Some Examples 2.2. Martingales 2.3. Stopping 2.4. Convergence of a Submartingale 2.5. Likelihoods 2.6. Square Intergrable Martingales 2.7. Almost Sure Asymptotic Properties 2.8. Central Limit Theorems 62 63 64 69 74 82 96 101 107
x Contents CHAPTER 3 Asymptotic Statistics 3. I. Models Dominated at Each Instant 3.2. Contrasts 3.3. Rate of Convergence of an Estimator 3.4. Asymptotic Properties of Tests CHAPTER 4 Markov Chains 4.1. Introduction and First Tools 4.2. Recurrent or Transient States 4.3. The Study of a Markov Chain Having a Recurrent State 4.4. Statistics of Markov Chains CHAPTER 5 Step by Step Decisions 5.1. Optimal Stopping 5.2. Control of Markov Chains 5.3. Sequential Statistics 5.4. Large Deviations and Likelihood Tests CHAPTER 6 Counting Processes 6.1. Renewal Processes and Random Walks 6.2. Counting Processes 6.3. Poisson Processes 6.4. Statistics of Counting Processes CHAPTER 7 Processes in Continuous Time 7.1. Stopping Times 7.2. Martingales in Continuous Time 7.3. Processes with Continuous Trajectories 7.4. Functional Central Limit Theorems CHAPTER 8 Stochastic Integrals 8.1. Stochastic Integral with Respect to a Square Integrable Martingale 8.2. Ito's Formula and Stochastic Calculus 8.3. Asymptotic Study of Point Processes 115 116 119 129 148 157 157 164 171 187 207 207 215 224 238 249 250 264 280 283 289 289 294 304 321 331 332 350 364
Contents 8.4. Brownian Motion 8.5. Regression and Diffusions Bibliography Notations and Conventions Index xi 372 380 389 397 406
SUMMARY OF VOLUME I Censuses Census of two qualitative characteristics Census of quantitative characteristics First definitions of discrete probabilities Pairs of random variables and correspondence analysis Heads or Tails. Quality Control Repetition of n independent experiments A Bernoulli sample Estimation Tests, confidence intervals for a Bernoulli sample, and quality control Observations of indeterminate duration Probabilistic Vocabulary of Measure Theory. Inventory of The Most Useful Tools Probabilistic models Integration The distribution of a measurable function Convergence in distribution
XIV Summary of Volume I Independence: Sample Statistics Based on the Observation of a A sequence of n-observations - Product measure spaces Independence Distribution of the sum of independent random vectors A sample from a distribution and estimation of this distri bu tion Non-parametric tests Gaussian Samples, Regression, and Analysis of Variance Gaussian samples Gaussian random vectors Central limit theorem on frk The X 2 test Regression Conditional Expectation, Markov Chains, Information Approximation in the least squares sense by functions of an observation Conditional expectation - extensions Markov chains Information carried by one distribution on another Dominated Statistical Models and Estimation Dominated statistical models Dissimilarity in a dominated model Likelihood Statistical Decisions Decisions Ba yesian statistics Optimality properties of some likelihood ratio tests Invariance