M4A42 APPLIED STOCHASTIC PROCESSES

Transcription

1 M4A42 APPLIED STOCHASTIC PROCESSES G.A. Pavliotis Department of Mathematics Imperial College London, UK LECTURE 1 12/10/2009

2 Lectures: Mondays 09:00-11:00, Huxley 139, Tuesdays 09:00-10:00, Huxley 144. Office Hours: Mondays 14:00-15:00, Tuesdays 14:00-15:00 or by appointment. Course webpage: Text: Lecture notes, available from the course webpage. Also, recommended reading from various textbooks.

3 This is an introductory course on stochastic processes and their applications, aimed towards students in applied mathematics. The emphasis of the course will be on the presentation of analytical tools that are useful in the study of stochastic models that appear in various problems in applied mathematics, physics, chemistry and biology. We will also discuss about numerical methods for simulating stochastic processes, solving stochastic differential equations etc. Time permitting, some applications of stochastic processes such as the modeling of molecular motors or chemical reactions will be discussed.

4 Prerequisites Elementary probability theory. Ordinary and partial differential equations. Linear algebra. Some familiarity with analysis (measure theory, linear functional analysis) is desirable but not necessary. Course Objectives By the end of the course you are expected to be familiar with the basic concepts of the theory of stochastic processes in continuous time and to be able to use various analytical and computational techniques to study stochastic models that appear in applications. Course assessment Coursework: 3 sets of assessed coursework 10% of the final mark. Final exam (May/June 2010).

5 Probability theory and random variables (3 lectures). Basic definitions, probability spaces, probability measures etc. Random variables, conditional expectation, characteristic functions, limits theorems. Stochastic processes (8 lectures). Basic definitions. Brownian motion. Stationary processes. Other examples of stationary processes. The Karhunen-Loeve expansion. Markov processes (4 lectures). Introduction and examples. Basic definitions. The Chapman-Kolmogorov equation. The generator of a Markov process and its adjoint. Ergodic and stationary Markov processes. Diffusion processes (4 lectures). Basic definitions and examples. The backward and forward (Fokker-Planck) Kolmogorov equations. Connection between diffusion processes and stochastic differential equations.

6 The Fokker-Planck equation (7 lectures). Basic properties of the FP equation. Examples of diffusion processes and of the FP equation. The Ornstein-Uhlenbeck process. Gradient flows and eigenfunction expansions. Stochastic Differential Equations (4 lectures). Basic properties of SDEs. Itô s formula. Numerical solution of SDEs.

7 Lecture notes will be provided for all the material that we will cover in this course. The notes are available from the course webpage. There are many excellent textbooks/review articles on applied stochastic processes, at a level and style similar to that of this course. Standard textbooks that cover the material on probability theory, Markov chains and stochastic processes are: Grimmett and Stirzaker: Probability and Random Processes. Karlin and Taylor: A First Course in Stochastic Processes. Lawler: Introduction to Stochastic Processes. Resnick: Adventures in Stochastic Processes.

8 Books on stochastic processes with a view towards applications, mostly to physics, are: Horsthemke and Lefever: Noise induced transitions. Risken: The Fokker-Planck equation. Gardiner: Handbook of stochastic methods. van Kampen: Stochastic processes in physics and chemistry. Mazo: Brownian motion: fluctuations, dynamics and applications. Chorin and Hald: Stochastic tools for mathematics and science. Gillespie; Markov Processes.

9 The rigorous mathematical theory of probability and stochastic processes is presented in Koralov and Sinai: Theory of probability and random processes. Karatzas and Shreeve: Brownian motion and stochastic calculus. Revuz and Yor: Continuous martingales and Brownian motion. Stroock: Probability theory, an analytic view. Books on stochastic differential equations and their numerical solution are Oksendal: Stochastic differential equations. Kloeden and Platen, Numerical Solution of Stochastic Differential Equations. An excellent book on the theory and the applications of stochastic processes is Bhatthacharya and Waymire: Stochastic processes and applications.

10 A stochastic process is used to model systems that evolve in time and whose laws of evolution are probabilistic in nature. The state of the system evolves in time and can be described through a state variable x(t). The evolution of the state of the system depends on the outcome of an experiment. We can write x = x(t, ω), where ω denotes the outcome of the experiment. Examples: The random walk in one dimension. Brownian motion. The exchange rate between the British pound and the US dollar. Photon emission. The spread of the SARS epidemic.

11 The One-Dimensional Random Walk We let time be discrete, i.e. t = 0, 1,... Consider the following stochastic process S n : S 0 = 0; at each time step it moves to ±1 with equal probability 1 2. In other words, at each time step we flip a fair coin. If the outcome is heads, we move one unit to the right. If the outcome is tails, we move one unit to the left. Alternatively, we can think of the random walk as a sum of independent random variables: S n = n X j, j=1 where X j { 1, 1} with P(X j = ±1) = 1 2.

12 We can simulate the random walk on a computer: We need a (pseudo)random number generator to generate n independent random variables which are uniformly distributed in the interval [0,1]. If the value of the random variable is 1 2 then the particle moves to the left, otherwise it moves to the right. We then take the sum of all these random moves. The sequence {S n } N n=1 indexed by the discrete time T = {1, 2,...N} is the path of the random walk. We use a linear interpolation (i.e. connect the points {n, S n } by straight lines) to generate a continuous path.

13 50 step random walk Figure: Three paths of the random walk of length N = 50.

14 1000 step random walk Figure: Three paths of the random walk of length N = 1000.

15 Every path of the random walk is different: it depends on the outcome of a sequence of independent random experiments. We can compute statistics by generating a large number of paths and computing averages. For example, E(S n ) = 0, E(S 2 n) = n. The paths of the random walk (without the linear interpolation) are not continuous: the random walk has a jump of size 1 at each time step. This is an example of a discrete time, discrete space stochastic processes. The random walk is a time-homogeneous (the probabilistic law of evolution is independent of time) Markov (the future depends only on the present and not on the past) process. If we take a large number of steps, the random walk starts looking like a continuous time process with continuous paths.

16 First 50 steps First 1000 steps First 100 steps First 5000 steps First 200 steps First steps Figure: Space-time rescaled random walks.

17 Consider the sequence of continuous time stochastic processes Zt n := 1 S nt. n In the limit as n, the sequence {Zt n } converges (in some appropriate sense) to a Brownian motion with diffusion coefficient D = x2 2 t = 1 2.

18 2 1.5 mean of 1000 paths 5 individual paths 1 U(t) t Figure: Sample Brownian paths.

19 Brownian motion W(t) is a continuous time stochastic processes with continuous paths that starts at 0 (W(0) = 0) and has independent, normally. distributed Gaussian increments. We can simulate the Brownian motion on a computer using a random number generator that generates normally distributed, independent random variables.

20 We can write an equation for the evolution of the paths of a Brownian motion X t with diffusion coefficient D starting at x: dx t = 2DdW t, X 0 = x. This is an example of a stochastic differential equation. The probability of finding X t at y at time t, given that it was at x at time t = 0, the transition probability density ρ(y, t) satisfies the PDE ρ t = D 2 ρ y 2, ρ(y, 0) = δ(y x). This is an example of the Fokker-Planck equation. The connection between Brownian motion and the diffusion equation was made by Einstein in 1905.

21 Why introduce randomness in the description of physical systems? To describe outcomes of a repeated set of experiments. Think of tossing a coin repeatedly or of throwing a dice. To describe a deterministic system for which we have incomplete information: we have imprecise knowledge of initial and boundary conditions or of model parameters. ODEs with random initial conditions are equivalent to stochastic processes that can be described using stochastic differential equations. To describe systems for which we are not confident about the validity of our mathematical model.

22 To describe a dynamical system exhibiting very complicated behavior (chaotic dynamical systems). Determinism versus predictability. To describe a high dimensional deterministic system using a simpler, low dimensional stochastic system. Think of the physical model for Brownian motion (a heavy particle colliding with many small particles). To describe a system that is inherently random. Think of quantum mechanics.