An Introduction to Probability Theory and Its Applications

Transcription

1 An Introduction to Probability Theory and Its Applications WILLIAM FELLER ( ) Eugene Higgins Professor of Mathematics Princeton University VOLUME II SECOND EDITION JOHN WILEY & SONS

2 Contents I THE EXPONENTIAL AND THE UNIFORM DENSITIES Introduction 1 2. Densities. Convolutions 3 3. The Exponential Density 8 4. Waiting Time Paradoxes. The Poisson Process The Persistence of Bad Luck Waiting Times and Order Statistics The Uniform Distribution Random Splittings Convolutions and Covering Theorems Random Directions The Use of Lebesgue Measure Empirical Distributions Problems for Solution 39 II SPECIAL DENSITIES. RANDOMIZATION Notations and Conventions Gamma Distributions 47 *3. Related Distributions of Statistics Some Common Densities Randomization and Mixtures Discrete Distributions 55 * Starred sections are not required for the understanding of the sequel and should be omitted at first reading. xvii

3 xviii CONTENTS 7. Bessel Functions and Random Walks Distributions on a Circle Problems for Solution 64 III DENSITIES IN HIGHER DIMENSIONS. NORMAL DENSITIES AND PROCESSES Densities Conditional Distributions Return to the Exponential and the Uniform Distributions A Characterization of the Normal Distribution Matrix Notation. The Covariance Matrix Normal Densities and Distributions Stationary Normal Processes Markovian Normal Densities Problems for Solution 99 IV PROBABILITY MEASURES AND SPACES Baire Functions Interval Functions and Integrals in 3i r a-algebras. Measurability Probability Spaces. Random Variables The Extension Theorem Product Spaces. Sequences of Independent Variables Null Sets. Completion 125 V PROBABILITY DISTRIBUTIONS IN 3T Distributions and Expectations Preliminaries Densities Convolutions 143

4 CONTENTS XIX 5. Symmetrization Integration by Parts. Existence of Moments Chebyshev's Inequality Further Inequalities. Convex Functions Simple Conditional Distributions. Mixtures *10. Conditional Distributions 160 *11. Conditional Expectations Problems for Solution 165 VI A SURVEY OF SOME IMPORTANT DISTRIBUTIONS AND PROCESSES Stable Distributions in ft Examples Infinitely Divisible Distributions in & Processes with Independent Increments 179 *5. Ruin Problems in Compound Poisson Processes Renewal Processes Examples and Problems Random Walks The Queuing Process Persistent and Transient Random Walks General Markov Chains Martingales Problems for Solution 215 VII LAWS OF LARGE NUMBERS. APPLICATIONS IN ANALYSIS Main Lemma and Notations Bernstein Polynomials. Absolutely Monotone Functions Moment Problems 224 *4. Application to Exchangeable Variables 228 *5. Generalized Taylor Formula and Semi-Groups Inversion Formulas for Laplace Transforms

5 XX CONTENTS *7. Laws of Large Numbers for Identically Distributed Variables 234 *8. Strong Laws 237 *9. Generalization to Martingales Problems for Solution 244 VIII THE BASIC LIMIT THEOREMS Convergence of Measures Special Properties Distributions as Operators The Central Limit Theorem 258 *5. Infinite Convolutions Selection Theorems 267 *7. Ergodic Theorems for Markov Chains Regular Variation Asymptotic Properties of Regularly Varying Functions Problems for Solution 284 IX INFINITELY DIVISIBLE DISTRIBUTIONS AND SEMI-GROUPS Orientation Convolution Semi-Groups Preparatory Lemmas Finite Variances The Main Theorems Example: Stable Semi-Groups Triangular Arrays with Identical Distributions Domains of Attraction Variable Distributions. The Three-Series Theorem Problems for Solution 318

6 CONTENTS XXI X MARKOV PROCESSES AND SEMI-GROUPS The Pseudo-Poisson Type A Variant: Linear Increments Jump Processes Diffusion Processes in The Forward Equation. Boundary Conditions Diffusion in Higher Dimensions Subordinated Processes Markov Processes and Semi-Groups The "Exponential Formula" of Semi-Group Theory Generators. The Backward Equation 356 XI RENEWAL THEORY The Renewal Theorem Proof of the Renewal Theorem 364 *3. Refinements Persistent Renewal Processes The Number N t of Renewal Epochs Terminating (Transient) Processes Diverse Applications Existence of Limits in Stochastic Processes 379 *9. Renewal Theory on the Whole Line Problems for Solution 385 XII RANDOM WALKS IN ft Basic Concepts and Notations Duality. Types of Random Walks Distribution of Ladder Heights. Wiener-Hopf Factorization 398 3a. The Wiener-Hopf Integral Equation 402

7 XX11 CONTENTS 4. Examples Applications A Combinatorial Lemma Distribution of Ladder Epochs The Arc Sine Laws Miscellaneous Complements Problems for Solution 425 XIII LAPLACE TRANSFORMS. TAUBERIAN THEOREMS. RESOLVENTS Definitions. The Continuity Theorem Elementary Properties Examples Completely Monotone Functions. Inversion Formulas Tauberian Theorems Stable Distributions 448 *7. Infinitely Divisible Distributions 449 *8. Higher Dimensions Laplace Transforms for Semi-Groups The Hille-Yosida Theorem Problems for Solution 463 XIV APPLICATIONS OF LAPLACE TRANSFORMS The Renewal Equation: Theory Renewal-Type Equations: Examples Limit Theorems Involving Arc Sine Distributions Busy Periods and Related Branching Processes Diffusion Processes Birth-and-Death Processes and Random Walks The Kolmogorov Differential Equations Example: The Pure Birth Process Calculation of Ergodic Limits and of First-Passage Times Problems for Solution 495

8 CONTENTS XXIU XV CHARACTERISTIC FUNCTIONS Definition. Basic Properties Special Distributions. Mixtures 502 2a. Some Unexpected Phenomena Uniqueness. Inversion Formulas Regularity Properties The Central Limit Theorem for Equal Components The Lindeberg Conditions Characteristic Functions in Higher Dimensions *8. Two Characterizations of the Normal Distribution Problems for Solution 526 XVI* EXPANSIONS RELATED TO THE CENTRAL LIMIT THEOREM Notations Expansions for Densities Smoothing Expansions for Distributions The Berry-Esseen Theorems Expansions in the Case of Varying Components Large Deviations 548 XVII INFINITELY DIVISIBLE DISTRIBUTIONS Infinitely Divisible Distributions Canonical Forms. The Main Limit Theorem a. Derivatives of Characteristic Functions Examples and Special Properties Special Properties Stable Distributions and Their Domains of Attraction. 574 *6. Stable Densities Triangular Arrays 583

9 XXIV CONTENTS 8. The Class L Partial Attraction. "Universal Laws" Infinite Convolutions Higher Dimensions Problems for Solution 595 XVIII APPLICATIONS OF FOURIER METHODS TO RANDOM WALKS The Basic Identity 598 *2. Finite Intervals. Wald's Approximation The Wiener-Hopf Factorization Implications and Applications Two Deeper Theorems Criteria for Persistency Problems for Solution 616 XIX HARMONIC ANALYSIS The Parseval Relation Positive Definite Functions Stationary Processes Fourier Series 626 *5. The Poisson Summation Formula Positive Definite Sequences L 2 Theory Stochastic Processes and Integrals Problems for Solution 647 ANSWERS TO PROBLEMS 651 SOME BOOKS ON COGNATE SUBJECTS 655 INDEX 657