Session 1: Probability and Markov chains 1. Probability distributions and densities. 2. Relevant distributions. 3. Change of variable. 4. Stochastic processes. 5. The Markov property. 6. Markov finite chains. 7. Evolution equation. 8. Transient and stationary regimes. 9. A simple example Juan MR Parrondo. 1
1. Probability distributions and densities Probability distribution (for discrete variables): Probability density (for continuous variables): Joint probability: Conditional probability: Juan MR Parrondo. 2
2.Relevant distributions Binomial (discrete): Distribution of the number of 1 s in a string with N random bits taking on value 1 with probability p and value 0 with probability 1-p. Distribution of: where X i = 1 with prob. p and X i =0 with prob. 1-p Juan MR Parrondo. 3
2. Relevant distributions (II) Poisson (discrete): It is the limit of the binomial for: Juan MR Parrondo. 4
2. Relevant distributions (III) Uniform (continuous): Exponential (continuous): Juan MR Parrondo. 5
2. Relevant distributions (IV) Gaussian (continuous): Juan MR Parrondo. 6
3. Change of variable Juan MR Parrondo. 7
4. Stochastic processes X(t) Different realizations of the same process. t Probability density: Joint probability densities: Conditional probability: Juan MR Parrondo. 8
5. The Markov property If we know the present (t 1 ), information on the past (t 2, t 3, ) does not improve our predictions of the future (t). For Markov processes, the statistics at two times determines the statistics of the whole process. Juan MR Parrondo. 9
6. Marxov finite chains 2 1 6 3 X(t) a Markov process that only takes on a 4 finite number of values. 5 The process jumps in discrete time steps: t = 1,2,3, The process is defined by the jump probability: Juan MR Parrondo. 10
7. Evolution equation Juan MR Parrondo. 11
8. Transient and stationary regimes Juan MR Parrondo. 12
9. A simple example α 1-α 1-α α Juan MR Parrondo. 13
Exercise: paradoxical games. Find the winning probability following the indications. Juan MR Parrondo. 14
Session 2: The Master Equation 1. From discrete to continuous time. 2. Distribution of residence times. 3. Master equation. 4. Probability current. 5. Detailed balance. Juan MR Parrondo. 15
1. From discrete to continuous time. An example 2 1-β α β 1 1-α Juan MR Parrondo. 16
2. Distribution of residence times. 1 T 1 T 2 2 t Poisson statistics Juan MR Parrondo. 17
3. Master equation 3 2 4 5 1 6 Juan MR Parrondo. 18
4. Probability current Net input Net output Juan MR Parrondo. 19
5. Detailed balance i j Juan MR Parrondo. 20
Exercise: demographic model o Read: o Eq. (1): can be converted into a master equation by considering q m,t as the probability that a given city has a population m at time t, and including q 0,t. o Find the stationary solution of the master equation using detailed balance. Compare with the results of the paper. Juan MR Parrondo. 21
Session 3: Stochastic differential equations 1. Introduction. 2. The Wiener process 3. Gaussian white noise. 4. Stochastic differential equations. 5. Stochastic integrals. 6. Ito calculus. 7. Ito versus Stratonovich. 8. Fokker-Planck Equation. 9. Stationary solutions. Juan MR Parrondo. 22
1. Introduction o Our aim is to study differential equations: with: (t) dx(t) dt = f(x(t); (t)) a stochastic process. o One possibility: solve the equation for a specific realization of the process. BUT o x(t) is Markovian if ad only if ξ(t) is white. Juan MR Parrondo. 23
1. Introduction (II) o x(t) is Markovian if ad only if ξ(t) is white. Proof for discrete time: x(t) ξ(t) t+1 t t-1 If ξ is not white: with x(t-1) and x(t) I can guess ξ(t) and have a better prediction of x(t+1) Juan MR Parrondo. 24
2. The Wiener process It is the position of a free Brownian particle: Juan MR Parrondo. 25
3. Gaussian white noise White noise is the derivative of the Wiener process: The problem is that the realizations of the Wiener process are not differentiable. W(t) t Juan MR Parrondo. 26
3. Gaussian white noise (II). Averaging: t t White noise is a generalized stochastic process. It makes sense only if integrated. Juan MR Parrondo. 27
4. Stochastic differential equations t i 0 t Δt t i * Juan MR Parrondo. 28
5. Stochastic integrals. An example. Juan MR Parrondo. 29
6. Ito calculus Juan MR Parrondo. 30
7. Ito vs Stratonovich Ito <x(t)ξ(t)>=0 Ito s lemma Good for systems with intrinsic discrete-time (percolation, stock market, ) Stratonovich <x(t)ξ(t)> 0 (Novikov theorem) Standard calculus Good for systems with intrinsic continuous time (chemistry, physics, ) Relation between Ito and Stratonovich: Theorem: If x(t) is a solution of ẋ = f(x)+g(x) Ito then x(t) is also a solution of ẋ = f(x) 2 2 g(x)g0 (x)+g(x) Stratonovich Proof: Use Ito s lemma with (x) = R x dx 0 /g(x 0 ) to obtain an equation with additive noise. Then go back to x using standard calculus. Máster en Física Teórica (UCM). Juan MR Parrondo. 31
8. The Fokker-Planck equation Consider the equation: ẋ = f(x)+g(x) Ito By Ito s lemma, for any arbitrary function A(x): Averaging: A = A 0 (x)[f(x)+g(x) ]+ hai = A 0 (x)f(x)+ 2 2 g(x)2 A 00 (x) Ito 2 2 g(x)2 A 00 (x) In terms of the probability density, ha(x(t))i = R dxa(x) (X, t): Z Z dxa(x)@ t (x, t) = apple dx A 0 (x)f(x)+ 2 2 g(x)2 A 00 (x) (x, t) Integrating by parts: Z Z dxa(x)@ t (x, t) = apple dxa(x) @ x (f(x) (x, t)) + 2 2 @2 x(g(x) 2 (x, t)) Since this expression is valid for any function A(x), we have Juan MR Parrondo. 32
8. The Fokker-Planck equation (II) ẋ = f(x)+g(x) Ito ẋ = f(x)+ g(x) Stratonovich Juan MR Parrondo. 33
9. Stationary solutions Gibbs state Juan MR Parrondo. 34
Exercise: noise-induced transitions. Juan MR Parrondo. 35