Adventures in Stochastic Processes

Size: px

Start display at page:

Download "Adventures in Stochastic Processes"

Sheena Cameron
5 years ago
Views:

1 Sidney Resnick Adventures in Stochastic Processes with Illustrations Birkhäuser Boston Basel Berlin

Table of Contents Preface ix CHAPTER 1. PRELIMINARIES: DISCRETE INDEX SETS AND/OR DISCRETE STATE SPACES 1.1. Non-negative integer valued random variables 1 1.2. Convolution 5 1.3.

2 Table of Contents Preface ix CHAPTER 1. PRELIMINARIES: DISCRETE INDEX SETS AND/OR DISCRETE STATE SPACES 1.1. Non-negative integer valued random variables Convolution Generating functions Differentiation of generating functions Generating functions and moments Generating functions and convolution Generating functions, compounding and random sums The simple branching process Limit distributions and the continuity theorem The law of rare events The simple random walk The distribution of a process* Stopping times* Wald's identity Splitting an iid sequence at a stopping time * 48 Exercises for Chapter 1 51 CHAPTER 2. MARKOV CHAINS 2.1. Construction and first properties Examples Higher order transition probabilities Decomposition of the State space The dissection principle Transience and recurrence Periodicity Solidarity properties Examples Canonical decomposition Absorption probabilities Invariant measures and stationary distributions 116

vi CONTENTS 2.12.1. Time averages 122 2.13. Limit distributions 126 2.13.1 More on null recurrence and transience*. 134 2.14. Computation of the stationary distribution 137 2.15.

3 vi CONTENTS Time averages Limit distributions More on null recurrence and transience* Computation of the stationary distribution Classification techniques 142 Exercises for Chapter CHAPTER 3. RENEWAL THEORY 3.1. Basics Analytic interlude Integration Convolution Laplace transforms Counting renewals Renewal reward processes The renewal equation Risk processes* The Poisson process as a renewal process Informal discussion of renewal limit theorems; regenerative processes An informal discussion of regenerative processes Discrete renewal theory Stationary renewal processes* Blackwell and key renewal theorems* Direct Riemann integrability* Equivalent forms of the renewal theorems* Proofof the renewal theorem* Improper renewal equations More regenerative processes* Definitions and examples* The renewal equation and Smith's theorem* Queueing examples 269 Exercises for Chapter CHAPTER 4. POINT PROCESSES 4.1. Basics The Poisson process Transforming Poisson processes 308

CONTENTS vii 4.3.1. Max-stable and stable random variables* 313 4.4. More transformation theory; marking and thinning 316 4.5. The order statistic property 321 4.6. Variants of the Poisson process 327 4.

4 CONTENTS vii Max-stable and stable random variables* More transformation theory; marking and thinning The order statistic property Variants of the Poisson process Technical basics* The Laplace functional* More on the Poisson process* A general construction of the Poisson process; a simple derivation of the order statistic property* More transformation theory; location dependent thinning* Records* 346 Exercises for Chapter CHAPTER 5. CONTINUOUS TIME MARKOV CHAINS 5.1. Definitions and construction Stability and explosions The Markov property* Dissection More detail on dissection* The backward equation and the generator matrix Stationary and limiting distributions More on invariant measures* Laplace transform methods Calculations and examples Queueing networks Time dependent Solutions* Reversibility Uniformizability The linear birth process as a point process 439 Exercises for Chapter CHAPTER 6. BROWNIAN MOTION 6.1. Introduction Preliminaries Construction of Brownian motion* Simple properties of Standard Brownian motion The reflection principle and the distribution of the maximum The strong independent increment property and reflection* Escape from a strip 508

viii CONTENTS 6.8. Brownian motion with drift 511 6.9. Heavy traffic approximations in queueing theory 514 6.10. The Brownian bridge and the Kolmogorov-Smirnov statistic. 524 6.11. Path properties* 539 6.

5 viii CONTENTS 6.8. Brownian motion with drift Heavy traffic approximations in queueing theory The Brownian bridge and the Kolmogorov-Smirnov statistic Path properties* Quadratic Variation Khintchine's law of the iterated logarithm for Brownian motion* 546 Exercises for Chapter CHAPTER 7. THE GENERAL RANDOM WALK* 7.1. Stopping times Global properties Prelude to Wiener-Hopf: Probabilistic interpretations of transforms Dual pairs of stopping times Wiener-Hopf decompositions Consequences of the Wiener-Hopf factorization The maximum of a random walk Random walks and the G/G/l queue Exponential right tail Application to G/M/l queueing model Exponential left tail The M/G/l queue Queue lengths 607 References 613 Index 617

Stochastic Processes. Theory for Applications. Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS

Stochastic Processes. Theory for Applications. Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS Stochastic Processes Theory for Applications Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS Contents Preface page xv Swgg&sfzoMj ybr zmjfr%cforj owf fmdy xix Acknowledgements xxi 1 Introduction and review