An Introduction to Stochastic Modeling

Size: px

Start display at page:

Download "An Introduction to Stochastic Modeling"

Posy Chandler
6 years ago
Views:

1 F An Introduction to Stochastic Modeling Fourth Edition Mark A. Pinsky Department of Mathematics Northwestern University Evanston, Illinois Samuel Karlin Department of Mathematics Stanford University Stanford, California PT ~P\TTPV AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO

2 F Contents 1.:: Preface to the Fourth Edition Preface to the Third Edition Preface to the First Edition To the Instructor Acknowledgments xi xiii XV xvii xix 1 Introduction Stochastic Modeling Stochastic Processes Probability Review Events and Probabilities Random Variables Moments and Expected Values Joint Distribution Functions Sums and Convolutions Change of Variable Conditional Probability Review of Axiomatic Probability Theory The Major Discrete Distributions Bernoulli Distribution Binomial Distribution Geometric and Negative Binominal Distributions The Poisson Distribution The Multinomial Distribution Important Continuous Distributions The Normal Distribution The Exponential Distribution The Uniform Distribution The Gamma Distribution The Beta Distribution The Joint Normal Distribution Some Elementary Exercises Tail Probabilities The Exponential Distribution Useful Functions, Integrals, and Sums 42

3 vi Content 2 Conditional Probability and Conditional Expectation 4~ 2.1 The Discrete Case The Dice Game Craps s 2.3 Random Sums 5" Conditional Distributions: The Mixed Case The Moments of a Random Sum The Distribution of a Random Sum Conditioning on a Continuous Random Variable Martingales The Definition The Markov Inequality The Maximal Inequality for Nonnegative Martingales 7 3 Markov Chains: Introduction Definitions Transition Probability Matrices of a Markov Chain Some Markov Chain Models An Inventory Model The Ehrenfest Urn Model Markov Chains in Genetics g A Discrete Queueing Markov Chain s 3.4 First Step Analysis s Simple First Step Analyses s The General Absorbing Markov Chain 1C 3.5 Some Special Markov Chains 1 J The Two-State Markov Chain 1 J Markov Chains Defined by Independent Random Variables One-Dimensional Random Walks l! Success Runs 1: 3.6 Functionals of Random Walks and Success Runs J: The General Random Walk 1: Cash Management J: The Success Runs Markov Chain J: 3.7 Another Look at First Step Analysis J: 3.8 Branching Processes J Examples of Branching Processes I The Mean and Variance of a Branching Process I Extinction Probabilities '], 3.9 Branching Processes and Generating Functions Generating Functions and Extinction Probabilities Probability Generating Functions and Sums of Tndeoendent Random Variables I I

4 F Contents vii 4 The Long Run Behavior of Markov Chains Regular Transition Probability Matrices Doubly Stochastic Matrices Interpretation of the Limiting Distribution Examples Including History in the State Description Reliability and Redundancy A Continuous Sampling Plan Age Replacement Policies Optimal Replacement Rules The Classification of States Irreducible Markov Chains Periodicity of a Markov Chain Recurrent and Transient States The Basic Limit Theorem of Markov Chains Reducible Markov Chains Poisson Processes The Poisson Distribution and the Poisson Process The Poisson Distribution The Poisson Process Nonhomogeneous Processes Cox Processes The Law of Rare Events The Law of Rare Events and the Poisson Process Proof of Theorem Distributions Associated with the Poisson Process The Uniform Distribution and Poisson Processes Shot Noise Sum Quota Sampling Spatial Poisson Processes Compound and Marked Poisson Processes Compound Poisson Processes Marked Poisson Processes Continuous Time Markov Chains Pure Birth Processes Postulates for the Poisson Process Pure Birth Process The Yule Process Pure Death Processes The Linear Death Process 287

5 8 Brownian Motion and Related Processes Brownian Motion and Gaussian Processes 3S A Little History 3~ The Brownian Motion Stochastic Process The Central Limit Theorem and the Invariance Principle 3! Gaussian Processes 3! 8.2 The Maximum Variable and the Reflection Principle The Reflection Principle The Time to First Reach a Level The Zeros of Brownian Motion Variations and Extensions Reflected Brownian Motion Absorbed Brownian Motion 4 Rii The Brownian Bridge 4 viii Content 6.3 Birth and Death Processes Postulates Sojourn Times Differential Equations of Birth and Death Processes The Limiting Behavior of Birth and Death Processes Birth and Death Processes with Absorbing States Probability of Absorption into State 0 31E Mean Time Until Absorption 31~ 6.6 Finite-State Continuous Time Markov Chains A Poisson Process with a Markov Intensity 33~ 7 Renewal Phenomena 34'i 7.1 Definition of a Renewal Process and Related Concepts 34~ 7.2 Some Examples of Renewal Processes 35: Brief Sketches of Renewal Situations 35: Block Replacement 35< 7.3 The Poisson Process Viewed as a Renewal Process 35: 7.4 The Asymptotic Behavior of Renewal Processes 36: The Elementary Renewal Theorem The Renewal Theorem for Continuous Lifetimes The Asymptotic Distribution of N(t) The Limiting Distribution of Age and Excess Life Generalizations and Variations on Renewal Processes Delayed Renewal Processes Stationary Renewal Processes Cumulative and Related Processes Discrete Renewal Theory The Discrete Renewal Theorem 3~ Deterministic Population Growth with Age Distribution 3~

6 f Contents ix 8.4 Brownian Motion with Drift The Gambler's Ruin Problem Geometric Brownian Motion The Omstein-Uhlenbeck Process A Second Approach to Physical Brownian Motion The Position Process The Long Run Behavior Brownian Measure and Integration Queueing Systems 9.1 Queueing Processes The Queueing Formula L = A. W A Sampling of Queueing Models 9.2 Poisson Arrivals, Exponential Service Times The MIMil System TheMIMioo System The MIMI s System 9.3 General Service Time Distributions TheMIGI1 System The MIGioo System 9.4 Variations and Extensions Systems with Balking Variable Service Rates A System with Feedback A Two-Server Overflow Queue Preemptive Priority Queues 9.5 Open Acyclic Queueing Networks The Basic Theorem Two Queues in Tandem Open Acyclic Networks Appendix: Time Reversibility Proof of Theorem General Open Networks The General Open Network Random Evolutions Two-State Velocity Model Two-State Random Evolution The Telegraph Equation Distribution Functions and Densities in the Two-State Model Passage Time Distributions N-State Random Evolution Finite Markov Chains and Random Velocity Models 507

7 X Content Random Evolution Processes Existence-Uniqueness of the First-Order System (10.26) Single Hyperbolic Equation Spectral Properties of the Transition Matrix Recunence Properties of Random Evolution 10.3 Weak Law and Central Limit Theorem 10.4 Isotropic Transport in Higher Dimensions The Rayleigh Problem of Random Flights Three-Dimensional Rayleigh Model C : 51( : 11 Characteristic Functions and Their Applications 52! 11.1 Definition of the Characteristic Function 52~ Two Basic Properties of the Characteristic Function 52< 11.2 Inversion Formulas for Characteristic Functions Fourier Reciprocity/Local Non-Uniqueness 53( Fourier Inversion and Parseval's Identity Inversion Formula for General Random Variables 53: 11.4 The Continuity Theorem Proof of the Continuity Theorem Proof of the Central Limit Theorem Stirling's Formula and Applications Poisson Representation of n! Proof of Stirling's Formula Local demoivre-laplace Theorem 53 Further Reading Answers to Exercises Index

Fundamentals of Applied Probability and Random Processes

Fundamentals of Applied Probability and Random Processes,nd 2 na Edition Oliver C. Ibe University of Massachusetts, LoweLL, Massachusetts ip^ W >!^ AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS