Probability and Statistics. Volume II

Transcription

1 Probability and Statistics Volume II

2 Didier Dacunha-Castelle Marie Duflo Probability and Statistics Volume II Translated by David McHale Springer-Verlag New York Berlin Heidelberg Tokyo

3 Didier Dacunha-Castelle Universite de Paris-Sud Equipe de Recherche Associee au C.N.R.S. 532 Statistique Applique Mathematique Orsay Cedex France Marie Duflo Universite de Paris-Nord Villetaneuse France David McHale (Translator) Linslade, Leighton Buzzard Bedfordshire LU7 7XW United Kingdom With 6 Illustrations AMS Classification Library of Congress Cataloging-in-Publication Data Dacunha-Castelle, Didier. Probability and statistics. Translation of: Probabilites et statistiques. Includes bibliographies and index. 1. Probabilities. 2. Mathematical statistics. I. Dufio, Marie. I. Title. QA273.D French Edition, "Probabilities et statistiques," Masson, Editeur, Paris, by Springer-Verlag New York Inc. Softcover reprint of the hardcover I st edition 1986 All rights reserved. No part of this book may be translated or reproduced in any form without written permission from Springer-Verlag, 175 Fifth Avenue, New York, New York 10010, U.S.A ISBN-13: e-isbn-13: DOl /

4 Note: In this second volume of Probability and Statistics, reference is sometimes made to Volume I. These references appear in the form [Vol. I. X.Y.Z.] and refer to Chapter X, paragraph Y, sub-paragraph Z. Similarly, for the references to Volume II, denoted only [X.Y.Z.].

5 INTRODUCTION How can we predict the future without asking an astrologer? When a phenomenon is not evolving, experiments can be repeated and observations therefore accumulated; this is what we have done in Volume I. However history does not repeat itself. Prediction of the future can only be based on the evolution observed in the past. Yet certain phenomena are stable enough so that observation in a sufficient interval of time gives usable information on the future or the mechanism of evolution. Technically, the keys to asymptotic statistics are the following: laws of large numbers, central limit theorems, and likelihood calculations. We have sought the shortest route to these theorems by neglecting to present the most general models. The future statistician will use the foundations of the statistics of processes and should satisfy himself about the unity of the methods employed. At the same time, we have adhered as closely as possible to present day ideas of the theory of processes. For those who wish to follow the study of probabilities to postgraduate level, it is not a waste of time to begin with the least difficult technical situations. This book for final year mathematics courses is not the end of the matter. It acts as a springboard either for dealing concretely with the problems of the statistics of processes, or

6 viii In trod uction to study in depth the more subtle aspects of probabilities. Finally, let us note that a more classical probability course can easily be organized around Chapter 2 which is central, on Chapter 4 on Markov chains and Chapters 5 to 8 for the important parts which do not call on statistical concepts.

7 CONTENTS CHAPTER 0 Introduction to Random Processes 0.1. Random Evolution Through Time 0.2. Basic Measure Theory 0.3. Convergence in Distribution I 3 9 CHAPTER 1 Time Series I. I. Second Order Processes 1.2. Spatial Processes with Orthogonal Increments 1.3. Stationary Second Order Processes 1.4. Time Series Statistics CHAPTER 2 Martingales in Discrete Time 2.1. Some Examples 2.2. Martingales 2.3. Stopping 2.4. Convergence of a Submartingale 2.5. Likelihoods 2.6. Square Intergrable Martingales 2.7. Almost Sure Asymptotic Properties 2.8. Central Limit Theorems

8 x Contents CHAPTER 3 Asymptotic Statistics 3. I. Models Dominated at Each Instant 3.2. Contrasts 3.3. Rate of Convergence of an Estimator 3.4. Asymptotic Properties of Tests CHAPTER 4 Markov Chains 4.1. Introduction and First Tools 4.2. Recurrent or Transient States 4.3. The Study of a Markov Chain Having a Recurrent State 4.4. Statistics of Markov Chains CHAPTER 5 Step by Step Decisions 5.1. Optimal Stopping 5.2. Control of Markov Chains 5.3. Sequential Statistics 5.4. Large Deviations and Likelihood Tests CHAPTER 6 Counting Processes 6.1. Renewal Processes and Random Walks 6.2. Counting Processes 6.3. Poisson Processes 6.4. Statistics of Counting Processes CHAPTER 7 Processes in Continuous Time 7.1. Stopping Times 7.2. Martingales in Continuous Time 7.3. Processes with Continuous Trajectories 7.4. Functional Central Limit Theorems CHAPTER 8 Stochastic Integrals 8.1. Stochastic Integral with Respect to a Square Integrable Martingale 8.2. Ito's Formula and Stochastic Calculus 8.3. Asymptotic Study of Point Processes

9 Contents 8.4. Brownian Motion 8.5. Regression and Diffusions Bibliography Notations and Conventions Index xi

10 SUMMARY OF VOLUME I Censuses Census of two qualitative characteristics Census of quantitative characteristics First definitions of discrete probabilities Pairs of random variables and correspondence analysis Heads or Tails. Quality Control Repetition of n independent experiments A Bernoulli sample Estimation Tests, confidence intervals for a Bernoulli sample, and quality control Observations of indeterminate duration Probabilistic Vocabulary of Measure Theory. Inventory of The Most Useful Tools Probabilistic models Integration The distribution of a measurable function Convergence in distribution

11 XIV Summary of Volume I Independence: Sample Statistics Based on the Observation of a A sequence of n-observations - Product measure spaces Independence Distribution of the sum of independent random vectors A sample from a distribution and estimation of this distri bu tion Non-parametric tests Gaussian Samples, Regression, and Analysis of Variance Gaussian samples Gaussian random vectors Central limit theorem on frk The X 2 test Regression Conditional Expectation, Markov Chains, Information Approximation in the least squares sense by functions of an observation Conditional expectation - extensions Markov chains Information carried by one distribution on another Dominated Statistical Models and Estimation Dominated statistical models Dissimilarity in a dominated model Likelihood Statistical Decisions Decisions Ba yesian statistics Optimality properties of some likelihood ratio tests Invariance