Stochastic Processes. Theory for Applications. Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS

Size: px

Start display at page:

Download "Stochastic Processes. Theory for Applications. Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS"

Emery Nelson
6 years ago
Views:

1 Stochastic Processes Theory for Applications Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS

2 Contents Preface page xv Swgg&sfzoMj ybr zmjfr%cforj owf fmdy xix Acknowledgements xxi 1 Introduction and review of probability Probability models The sample space of a probability model Assigning probabilities for finite sample spaces The axioms of probability theory Axioms for events Axioms of probability Probability review Conditional probabilities and Statistical independence Repeated idealized experiments Random variables Multiple random variables and conditional probabilities Stochastic processes The Bernoulli process Expectations and more probability review Random variables as functions of other random variables Conditional expectations Typical values of random variables; mean and median Indicator random variables Moment generating functions and other transforms Basic inequalities The Markov inequality The Chebyshev inequality Chernoff bounds The laws of large numbers Weak law of large numbers with a finite variance Relative frequency The central limit theorem (CLT) Weak law with an infinite variance Convergence of random variables Convergence with probability 1 48

3 viii Contents 1.8 Relation of probability models to the real world 5' Relative frequencies in a probability model Relative frequencies in the real world Statistical independence of real-world experiments Limitations of relative frequencies Subjective probability Summary $ Exercises ^ 2 Poisson processes Introduction Arrival processes Definition and properties of a Poisson process Memoryless property Probability density of 5, 7 and joint density of 5i, The probability mass function (PMF) for N(t) Alternative definitions of Poisson processes The Poisson process as a Ii mit of shrinking Bernoulli processes Combining and Splitting Poisson processes Subdividing a Poisson process Examples using independent Poisson processes Non-homogeneous Poisson processes Conditional arrival densities and order statistics Summary Exercises 97 3 Gaussian random vectors and processes Introduction Gaussian random variables Gaussian random vectors Generating functions of Gaussian random vectors HD normalized Gaussian random vectors Jointly-Gaussian random vectors Joint probability density for Gaussian ;;-rv s (special case) Properties of covariance matrices I Symmetrie matrices Positive definite matrices and covariance matrices I Joint probability density for Gaussian n-rv s (general case) I Geometry and principal axes for Gaussian densities Conditional PDFs for Gaussian random vectors Gaussian processes Stationarity and related concepts Orthonormal expansions Continuous-time Gaussian processes Gaussian sine processes 132

4 Contents ix Filtered Gaussian sine processes Filtered continuous-time stochastic processes Interpretation of spectral density and covariance White Gaussian noise The Wiener process/brownian motion Circularly-symmetric complex random vectors Circular symmetry and complex Gaussian random variables Covariance and pseudo-covariance of complex n-dimensional random vectors Covariance matrices of complex n-dimensional random vectors Linear transformations of W ~ CÄf(0, [Ii]) Linear transformations of Z ~ CW(0, [Ä]) The PDF of circularly-symmetric Gaussian n-dimensional random vectors Conditional PDFs for circularly-symmetric Gaussian random vectors Circularly-symmetric Gaussian processes Summary Exercises Finite-state Markov chains Introduction Classification of states The matrix representation Steady State and [P n \ for large n Steady State assuming [P] > Ergodic Markov chains Ergodic unichains Arbitrary finite-state Markov chains The eigenvalues and eigenvectors of stochastic matrices Eigenvalues and eigenvectors for M 2 states Eigenvalues and eigenvectors for M > 2 states Markov chains with rewards Expected first-passage times The expected aggregate reward over multiple transitions The expected aggregate reward with an additional final reward Markov decision theory and dynamic programming Dynamic programming algorithm Optimal stationary policies Policy improvement and the search for optimal stationary policies Summary Exercises 202

5 X Contents 5 Renewal processes ^' Introduction The strong law of large numbers and convergence with probability I Convergence with probability 1 (WP1) Strong law of large numbers Strong law for renewal processes Renewal-reward processes; time averages General renewal-reward processes Stopping times for repeated experiments Wald's equality Applying Wald's equality to E [7V(0J Generalized stopping trials, embedded renewals, and G/G/l queues Little's theorem M/G/l queues Expected number of renewals; ensemble averages Laplace transform approach The elementary renewal theorem Renewal-reward processes; ensemble averages Age and duration for arithmetic processes Joint age and duration: non-arithmetic case Age Z{t) for finite t: non-arithmetic case Age Z(f) as f» oo: non-arithmetic case Arbitrary renewal-reward functions: non-arithmetic case Delayed renewal processes Delayed renewal-reward processes Transient behavior of delayed renewal processes The equilibrium process Summary Exercises Countable-state Markov chains Introductory examples First-passage times and recurrent states Renewal theory applied to Markov chains Renewal theory and positive recurrence Steady State Blackwell's theorem applied to Markov chains Age of an arithmetic renewal process Birth-death Markov chains Reversible Markov chains The M/M/l sampled-time Markov chain Branching processes 3Q9 6.8 Round-robin service and processor sharing 312

6 Contents xi 6.9 Summary Exercises Markov processes with countable-state spaces Introduction The sampled-time approximation to a Markov process Steady-state behavior of irreducible Markov processes Renewals on successive entries to a given State The limiting fraction of time in each State Rinding {pj(i); j > 0} in terms of {izy, j > 0} Solving for the steady-state process probabilities directly The sampled-time approximation again Pathological cases The Kolmogorov differential equations Uniformization Birth-death processes The M/M/1 queue again Other birth-death systems Reversibility for Markov processes Jackson networks Closed Jackson networks Semi-Markov processes Example - the M/G/l queue Summary Exercises Detection, decisions, and hypothesis testing Decision criteria and the maximum a posteriori probability (MAP) criterion Binary MAP detection Sufficient statistics I Binary detection with a one-dimensional Observation Binary MAP detection with vector observations Sufficient statistics II Binary detection with a minimum-cost criterion The error curve and the Neyman-Pearson rule The Neyman-Pearson detection rule The min-max detection rule Finitely many hypotheses Sufficient statistics with M > 2 hypotheses More general minimum-cost tests Summary Exercises 410

7 xii Contents 9 Random walks, large deviations, and martingales 41 ' 9.1 Introduction Simple random walks Integer-valued random walks Renewal processes as special cases of random walks The queueing delay in a G/G/l queue Threshold crossing probabilities in random walks The Chernoff bound Tilted probabilities Large deviations and compositions Back to threshold crossings Thresholds, stopping rules, and Wald's identity Wald's identity for two thresholds The relationship of Wald's identity to Wald's equality Zero-mean random walks Exponential bounds on the probability of threshold crossing Binary hypotheses with HD observations Binary hypotheses with a fixed number of observations Sequential decisions for binary hypotheses Martingales Simple examples of martingales Scaled branching processes Partial Isolation of past and future in martingales Submartingales and supermartingales Stopped processes and stopping tri als The Wald identity The Kolmogorov inequalities TheSLLN The martingale convergence theorem A simple model for Investments Portfolios with constant fractional allocations Portfolios with time-varying allocations Markov modulated random walks Generating functions for Markov random walks Stopping trials for martingales relative to a process Markov modulated random walks with thresholds Summary Exercises Estimation 4%% 10.1 Introduction 4%% The squared-cost function Other cost functions 49O 10.2 MMSE estimation for Gaussian random vectors 49]

8 Contents xiii Scalar iterative estimation Scalar Kaiman filter LLSE estimation Filtered vector signal plus noise Estimate of a Single random variable in HD vector noise Estimate of a Single random variable in arbitrary vector noise Vector iterative estimation Vector Kaiman filter Estimation for circularly-symmetric Gaussian rv s The vector space of random variables; orthogonality MAP estimation and sufficient statistics Parameter estimation Fisher Information and the Cramer-Rao bound Vector observations Information Summary Exercises 523 References Index

DISCRETE STOCHASTIC PROCESSES Draft of 2nd Edition

DISCRETE STOCHASTIC PROCESSES Draft of 2nd Edition R. G. Gallager January 31, 2011 i ii Preface These notes are a draft of a major rewrite of a text [9] of the same name. The notes and the text are outgrowths