Mathematical Framework for Stochastic Processes

Transcription

1 Mathematical Foundations of Discrete-Time Markov Chains Tuesday, February 04, :04 PM Homework 1 posted, due Friday, February 21. Reading: Lawler, Ch. 1 Mathematical Framework for Stochastic Processes At an intuitive level, a stochastic process is just a mapping from a parameter domain T to a state space S with the property that the mapping is itself "random" How can one extend the concepts of measure theory for random variables, to give them a concrete mathematical foundation, to stochastic processes? There are several rigorous ways to proceed A useful standard way to represent a stochastic process for the purpose of building up the theory of stochastic processes is as follows: View as a collection of (not usually independent) random variables, indexed by the "time" parameter t (by which we mean the value of the parameter in the parameter domain T). This can work OK in practice for discrete parameter domains T, but gets a bit awkward to work with in practice when the parameter domain T is continuous because then we're dealing with an uncountable infinity of random variables. Nonetheless, one builds the general mathematical theory of stochastic processes from this viewpoint by referring to "finite dimensional distributions" as the defining objects of the stochastic process, and then applying the Kolmogorov extension theorem to show that the definitions in terms of finite dimensional distributions in fact can, under certain assumptions of reasonability ("separability") define the stochastic process precisely. A facile abstract formulation is to simply view the stochastic process itself as a random Stoch14 Page 1

2 3. variable in function space. This is done sometimes in mathematical and theoretical physics treatments, i.e., path space integrals. But hard to work with in most applications. "Monte Carlo" formulation: Think about the stochastic process as a joint map from sample space and the parameter domain into state space: For high-quality pseudorandom number generators and the statistical tests behind them, see the papers and website of Pierre L'Ecuyer. 4. "Weak" formulation: Reasonable observables of the stochastic process are random variables, i.e., Now let's turn to some explicit simple classes of stochastic models, and we will stay for a while with a discrete parameter domain though sometimes we'll find it useful to think about negative epochs as well Simplest useful class of stochastic processes are simply collections of independent (perhaps identically distributed) random variables on this discrete parameter domain. That is, is a collection of independent random variables. Note the notation; when we work with discrete parameter domains, we usually index time by its epoch n (rather than by t), and we write the stochastic process with the time parameter in the subscript rather than as a function, i.e., X(n). Stoch14 Page 2

3 Such a stochastic process has no regularity in its variation with respect to time; it just jumps around between independent values at successive times. Applications where this is a reasonable model: velocities of particles undergoing Brownian motion, provided that the time step between observations is not too small (i.e., successive scatterings of neutrons off of some profile of atoms outcome of successive pollination attempts successive coin or die tosses intervals between successive cars on a road times between successive decays of radioactive material But this complete independence assumption precludes any form of the memory in the models for the tracked variables, and this limits its applicability. In practice, it's often changes in certain variables that might be treated as independent from epoch to epoch, and one wants to define a model that describes how the system responds to these independent disturbances. A very useful mathematical framework for describing such systems, which have memory but can be described in terms of an accumulation of a sequence of independent disturbances at different epochs is known as a Markov process. We will spend most of the class within the framework of Markov processes. The reason for this is that the assumptions defining the Markov process are applicable enough and powerful enough to be useful for many applications and for a good deal of mathematical results to be developed. In fact there's a mathematical theorem by Skorokhod that more or less that any stochastic process can be formulated as a Markov process if one introduces a rich enough state space If one wants to work with non-markov processes, then other classes of stochastic processes with well-developed theories are: Gaussian processes (all finite dimensional distributions are Gaussian) stationary random processes (statistics don't change under rigid translation of the observation times/locations) Stoch14 Page 3

4 These are particularly important for describing spatial rather than temporal randomness. An excellent reference for both of these is Yaglom, Correlation Theory of Stationary and Related Random Functions Finite state, discrete time (FSDT) Markov chain This is a Markov process on a discrete parameter domain T with discrete state space S. For concreteness, we'll take, at least for now: There are two equivalent ways to define an FSDT Markov chain. Stochastic update rule (Resnick Sec. 2.1) This mathematically encodes the intuition behind a Markov process described in the previous paragraphs. Prescribe arbitrary updating functions, for Define a sequence of independent, identically distributed random variables with. "identically distributed" means that all random variables have the same probability distribution. these variables represent the unpredictable (at the level of resolution of the model) noise entering the system at epoch n Then the dynamics of the Markov chain are given by the mapping: Stoch14 Page 4

5 This is a random dynamical system, and it also needs, like any dynamical system, an initial condition. We'll discuss this next time. Stoch14 Page 5