STOCHASTIC MODELS LECTURE 1 MARKOV CHAINS. Nan Chen MSc Program in Financial Engineering The Chinese University of Hong Kong (ShenZhen) Sept.

Transcription

1 STOCHASTIC MODELS LECTURE 1 MARKOV CHAINS Nan Chen MSc Program in Financial Engineering The Chinese University of Hong Kong (ShenZhen) Sept. 6, 2016

2 Outline 1. Introduction 2. Chapman-Kolmogrov Equations 3. The Gambler s ruin Problem

3 1.1 INTRODUCTION

4 What is a Stochastic Process? A stochastic process is a collection of random variables that are indexed by time. Usually, we denote it by (X 1, X 2,!, X n,!) or (X t,t 0). Examples: Daily average temperature on CUHK-SZ campus Real-time stock price of Google

5 Motivation of Markov Chains Stochastic processes are widely used to characterize the temporal relationship between random variables. The simplest model should be that independent of each other. X n are But, the above model may not be able to provide a reasonable approximation to financial markets.

6 What is a Markov Chain? Let (X n, n = 0,1, 2,!) be a stochastic process that takes on a finite/countable number of possible states. We call it by a Markov chain, if the conditional distribution of X n+1 depends on the past observations (X 1, X 2,!, X n ) only through. Namely, X n P(X n+1 = j X 1 = i 1, X 2 = i 2,!, X n = i n ) = P(X n+1 = j X n = i n ) for all n.

7 The Markovian Property It can be shown that the definition of Markov chains is equivalent to stating that P(X n+1 = j, X 1 = i 1, X 2 = i 2,!, X n 1 = i n 1 X n = i n ) = P(X n+1 = j X n = i n )P(X 1 = i 1, X 2 = i 2,!, X n 1 = i n 1 X n = i n ) In words, given the current state of the process, its future and historical movements are independent.

8 Financial Rationale: Efficient Market Hypothesis The Markovian property turns out to be highly relevant to financial modeling in light of one of the most profound theory in the history of modern finance --- efficient market hypothesis. It states Market information, such as the information reflected in the past record or the information published in financial press, must be absorbed and reflected quickly in the stock price.

9 More about EMH: a Thought Experiment Let us start with the following thought experiment: Assume that Prof. Chen had invented an formula which we could use to predict the movements of Google stock price very accurately. What would happen if this formula was disclosed to the public?

10 More about EMH: a Thought Experiment Suppose that it predicted that Google s stock price would rise dramatically in three days to US$700 from US$650. The prediction would induce a great wave of immediate buy orders. Huge demands on Google s stocks would push its price to jump to $700 immediately. The formula fails! A true story of Edward Thorp and the Black- Scholes formula

11 Implication of Efficient Market Hypothesis One implication of EMH is that given the current stock price, knowing its history will help very little in predicting its future. Therefore, we should use Markov processes to model the dynamic of financial variables.

12 Transition Matrix In this lecture, we only consider timehomogenous Markov chains; that is, the transition probabilities P(X n+1 = j X n = i n ) are independent of time n. Denote p ij := P(X n+1 = j X n = i). We then can use the following matrix to characterize the process.! # # P := # # " p 11 p 12! p 1n p 21 p 22! p 2n!!!! p n1 p n2! p nn $ & & & & %

13 Transition Matrix The transition matrix of a Markov chain must be a stochastic matrix: p ij 0. n p ij =1. j=1

14 Example I: Forecasting the Weather Suppose that the chance of rain tomorrow in Shenzhen depends on previous weather conditions only through whether or not it is raining today. Assume that if it rains today, then it will rain tomorrow with probability 70%; and if it does not rain today, then it will rain tomorrow with prob. 50%. How can we use a Markov chain to model it?

15 Example II: 1-dimensional Random Walk A Markov chain whose state space is given by the integers 0,±1,±2,... is said to be a random walk if, for some number 0 < p <1, P i,i+1 = p =1 P i,i 1, i = 0,±1,±2,... We say the random walk is symmetric if p =1/ 2 ; asymmetric if p 1/ 2.

16 1.2 CHAPMAN- KOLMOGOROV EQUATIONS

17 The Chapman-Kolmogorov Equations The CK equations provide a method for computing the n -step transition probabilities of a Markov chain. P n+m ij = P( X n+m = j X 0 = i) = P n m ik P kj or in a matrix form, P n+m = P n P m. k

18 Example III: Rain Probability Reconsider the situation in Example I. Given that it is raining today, what is the probability that it will rain four days from today?

19 Example IV: Urn and Balls An urn always contains 2 balls. Ball colors are red and blue. At each stage a ball is randomly chosen and then replaced by a new ball, which with prob. 80% is the same color, and with prob. 20% is the opposite color. If initially both balls are red, find the probability that the fifth ball selected is red.

20 1.3 THE GAMBLER S RUIN PROBLEM

21 The Gambler s Ruin Problem Consider a gambler who at each play of the game has probability p of winning one dollar and probability q =1 p of losing one dollar. Assuming that successive plays of the game are independent, what is the probability that, starting with i dollars, the gambler fortune will win N dollars before he ruins (i.e., his fortune reaches 0)?

22 Markov Description of the Model If we let X n denote the player s fortune at time n, then the process {X n, n =1, 2,...} is a Markov chain with transition probabilities P 00 = P NN =1, P i,i+1 = p =1 P i,i 1 i =1, 2,..., N 1. The Markov chain has three classes: {0},{1, 2,..., N 1},{N}

23 Solution Let P i be the probability that, starting with dollars, the gambler fortune will eventually reach. By conditioning on the outcome of the initial play and N P i = pp i+1 + qp i 1 P 0 = 0, P N =1., i =1, 2,..., N 1. i

24 Solution (Continued) Hence, we obtain from the preceding slide that P 2 P 1 = q p (P 1 P 0 ) = q p P 1; P 3 P 2 = q p (P " 2 P 1 ) = q % $ ' # p & 2 P 1 ; P N P N 1 = q p (P " N 1 P N 2 ) = q % $ ' # p & N 1 P 1.

25 Solution (Continued) Adding all the equalities up, we obtain " $ P 1 = # $ % 1/ N, 1 (q / p) 1 (q / p) N, p =1/ 2; p 1/ 2. P i = " $ # $ % i / N, 1 (q / p) i 1 (q / p) N, p =1/ 2; p 1/ 2.

26 Solution (Continued) Note that, as N +, P i ) + * +, + 0, # 1 q & % ( $ p ' i, p 1/ 2; p >1/ 2. Thus, if p >1/ 2, there is a positive probability that the gambler s fortune will increase indefinitely; while if p 1/ 2, the gambler will, with probability 1, go ruin against an infinitely rich adversary (say, a casino).

27 Homework Assignments Read Ross Chapter 4.1, 4.2, and Answer Questions: Exercises 5, 6 (Page 261, Ross) Exercises 13, 14 (Page 262, Ross) Exercises 56, 57 (Page 270, Ross) Due on Sept. 13, Wed.