Stochastic Models. Edited by D.P. Heyman Bellcore. MJ. Sobel State University of New York at Stony Brook

Size: px

Start display at page:

Download "Stochastic Models. Edited by D.P. Heyman Bellcore. MJ. Sobel State University of New York at Stony Brook"

Jessica Butler
6 years ago
Views:

1 Stochastic Models Edited by D.P. Heyman Bellcore MJ. Sobel State University of New York at Stony Brook 1990 NORTH-HOLLAND AMSTERDAM NEW YORK OXFORD TOKYO

Contents Preface CHARTER 1 Point Processes R.F. Serfozo 1. Introduction 2. Poisson processes and some relatives 3. Renewal theory 4. Stationary point processes 5.

2 Contents Preface CHARTER 1 Point Processes R.F. Serfozo 1. Introduction 2. Poisson processes and some relatives 3. Renewal theory 4. Stationary point processes 5. Point processes characterized by martingales 6. Convergence in distribution of sums of point Acknowledgement CHAPTER 2 Markov Processes A.F. Karr 1. Introduction 2. Markov chains 3. Markov processes 4. Diffusion processes: Selected complements 5. Markov random fields CHAPTER 3 Martingales and Random Walks H.M. Taylor Introduction 1. Martingales 2. The major results of martingale theory 3. Sample applications of martingale theory 4. Continuous time martingales

XU Contents 5. Random walks 141 Bibliography 143 144 CHARTER 4 Diffusion Approximations P.W. Glynn 145 1. Introduction 145 2. Weak convergence of stochastic processes 146 3.

3 XU Contents 5. Random walks 141 Bibliography CHARTER 4 Diffusion Approximations P.W. Glynn Introduction Weak convergence of stochastic processes Verification criteria for weak convergence of stochastic processes Donsker's theorem Weak convergence theorems for the single-server queue Background on diffusion processes Diffusion approximations for an open network of queues in heavy traffic Diffusion approximations for a closed network of queues in heavy traffic Approximations for queues with many Servers Conditional weak convergence theorems CHAPTER 5 Computational Methods in Probability Theory W.K. Grassmann Basic concepts Markov processes Markov modelling Markov processes with repetitive structures CHAPTER 6 Statistical Methods J. Lehoczky Introduction Parametric Statistical inference Hypothesis testing Inference for stochastic processes Sequential analysis Random parameter stochastic process modeis

Contents xm CHAPTER 7 Simulation Experiments B. Schmeiser 295 1. Introduction 295 2. Sources of randomness 297 3. Random-variate generation 303 4. Input modeling 309 5. Point estimation 312 6.

4 Contents xm CHAPTER 7 Simulation Experiments B. Schmeiser Introduction Sources of randomness Random-variate generation Input modeling Point estimation Output analysis Variance reduction 324 Acknowledgements CHAPTER 8 Markov Decision Processes M.L. Puterman Introduction Problem formulation Examples The finite horizon case Foundations of infinite horizon modeis Discounted Markov decision problems Undiscounted Markov decision problems I Undiscounted Markov decision problems II Semi-Markov decision processes and Markov renewal programming 423 Bibliography 429 CHAPTER 9 ControUed Continuous Time Markov Processes R. Rishel Introduction Examples of controlled processes Transition probabilities semigroups generators Markov processes Diffusion processes Piecewise deterministic processes Controlled Markov processes Dynamic programming optimality conditions Control computations, Example Control computations, Example Control computations, Example Control computations, Example 4 463

XIV Contents 13. The linear quadratic Gaussian control problem 14. Concluding comments 465 466 467 CHAPTER 10 Queueing Theory R.B. Cooper 1. Introduction 2. Some general theorems 3.

5 XIV Contents 13. The linear quadratic Gaussian control problem 14. Concluding comments CHAPTER 10 Queueing Theory R.B. Cooper 1. Introduction 2. Some general theorems 3. Performance measures 4. Length-biased sampling and the role of the exponential distribution 5. One-dimensional birth-and-death modeis 6. Multidimensional birth-and-death modeis 7. The M/G/l queue 8. The GI/M/s queue 9. Other topics 10. Concluding remarks Acknowledgements CHAPTER 11 Queueing Networks J. Walrand Introduction An example: Tandem M/M/1 queues with feedback Product-form networks Non-product-form networks Optimization and comments 590 CHAPTER 12 Stochastic Inventory Theory E.L. Porteus Introduction and overview The deterministic (EOQ) model The single period (newsvendor) model The dynamic linear model The dynamic convex model The dynamic concave model Other modeis

Contents xv CHARTER 13 Rehability and Maintainability M. Shaked and J.G. Shanthikumar 653 1. Introduction 653 2. Reliability Systems: Monotonicity and coherency 654 3.

6 Contents xv CHARTER 13 Rehability and Maintainability M. Shaked and J.G. Shanthikumar Introduction Reliability Systems: Monotonicity and coherency Reliability and availability measures Performability measures Univariate aging notions Maintenance policies Stochastically dependent components Systems with repairable components Multivariate aging notions Statistical inference in reliability theory Subject Index 715 Contents of the Previous Volume 725

COPYRIGHTED MATERIAL CONTENTS. Preface Preface to the First Edition

COPYRIGHTED MATERIAL CONTENTS. Preface Preface to the First Edition Preface Preface to the First Edition xi xiii 1 Basic Probability Theory 1 1.1 Introduction 1 1.2 Sample Spaces and Events 3 1.3 The Axioms of Probability 7 1.4 Finite Sample Spaces and Combinatorics 15