Problems. HW problem 5.7 Math 504. Spring CSUF by Nasser Abbasi

Transcription

1 Problems HW problem 5.7 Math 504. Spring CSUF by Nasser Abbasi 1

2 Problem 6.3 Part(A) Let I n be an indicator variable de ned as 1 when (n = jj I n = 0 = i) 0 otherwise Hence Now we see that E (V ) = E E (I n ) = P ( n = jj 0 = i) I n = E (I n ) = P ( n = jj 0 = i) n=0 n=0 n=0 Now, let b be entry in matrix B where b = E (V ), then the above can be written as b = P ( 0 = jj 0 = i) + P ( 1 = jj 0 = i) + P ( 2 = jj 0 = i) + P ( 3 = jj 0 = i) + (1) Which is the same as writing When i = j, then P (0) b = P (0) + P (1) + P (2) + P (3) + = 1 otherwise it is 0. Hence b = + P (1) + P (2) + P (3) + P (4) + (2) Let the set of transient states be T, written as and using chapman-kolmogorov, the above can be P (2) b = + P (1) + P (2) z } { P (1) + P (3) z } { P (2) + P (4) z } { P (3) + (2) z } { But P (1) is multiplying the i th row of the Q matrix by the j th column of the Q matrix. z } { which is the (i; j) entry of the matrix Q 2, and P (2) is multiplying the i th row of the Q 2 matrix we just obtained, by the j th column of the Q matrix, which is the (i; j) entry of the matrix Q 3. Continue this way, we obtain that P (4) is the entry i; j in matrix Q 4 and so on. Hence we see that b is the (i; j) entry of a matrix resulting from I + Q + Q 2 + Q 3 + QED. 2 P (3)

3 Part(B) From part(a), we obtained that E (V ) is the (i; j) entry in the matrix resulting from the sum I + Q + Q 2 + Q 3 +. Since this is a Q matrix, then we know its elements will all go to zero an n gets very large, so this is a convergent sum, hence I +Q+Q 2 +Q 3 +. (I Q) 1 : Therefore E (V ) is the (i; j) entry in the matrix (I Q) 1 : 3

4 Problem 6.5 Part(A) I solve this part in 2 ways, rst by conditioning on next state, as required, and then by the counting method explained in the lecture. by conditioning on next state. Let I be the set of all states. Then E (T ) = k2i E (T j 1 = k; 0 = i) P ( 1 = 0 = i) (1) But by Markov property, chain state on next step depends only on current state. Hence E (T j 1 = k; 0 = i) = E (T j 1 = k) and also since P ( 1 = 0 = i) = P ki then (1) can be written as E (T ) = k2i E (T ) P ki (2) Now, when 1 = j, then E (T ) = 1 since chain already in state j after one step. Therefore (2) can be rewritten as E (T ) = 1 + E (T j 1 = k) P ki k2i But E (T j 1 = k) is the same as writing E (T ), so the above becomes E (T ) = 1 + k2i E (T ) P ki Using the notation shown in the problem, the above becomes QED. m = 1 + k2i m P ki 4

5 Now solve part(a) using rst a counting argument, and using the following diagram as a guide Then we write (letting E (T ) = m and E (T ) = m ) m = N (P ) + NP ik (m + 1) N = P + P ik (m + 1) = P + P ik m + P ik But P + P ik = 1 hence the above becomes m = 1 + P ik m (1) 5

6 Part(B) We start from the result of part (A) which is m = 1 + P ik m Multiply both sides by w i and obtain Sum over all possible states i and obtain But w i = 1 and hence (2) becomes w i m = w i + w i P ik m w i m = w i + w i P ik m = w i m = 1 + m w i P ik m w i P ik m w i P ik = m (2) w i P ik, Now, since w = fw 1 ; w 2 ; ; w r g is the stationary state vector, then it satis es the following relation w = wp Where P is the one step probability transition matrix. The solution to the above is given by w k = w i P ik (4) Where k is any state. Using (4) into RHS of (3), we can rewrite (3) as A z } { w i m w i m = 1 + m w k k=1 (3) B z } { m w k = 1 (5) k=1 Now looking at the LHS, we see that the rst sum labeled A counts for all the w 0 s and the second sum labeled B also counts for all the w 0 s except for the j term. Hence if we subtract B from A, only the term m jj w j will survive. Hence (5) becomes or QED. m jj w j = 1 m jj = 1 w j 6

7 Part (C) If we wait for the chain to arrive at its steady state (i.e. we the chain probability state vector does not change, or w = wp ), then we observe the chain from that point on, for a long period of time, say T. The number of times the chain will be in state j during this time T is then given by w j T, since w j is the probability of the chain being in state j. So, to nd the average number of time units (steps) it took for the chain for go from state j back to state j we need to divide T by the number of times the chain was in state j during this time, which we just found as w j T Hence m jj = T w j T = 1 w j Intuitively this makes sense. Since the smaller the probability that the chain will be in state j we would expect the time between the events that the chain is in state j to become larger, So the relation should be an inverse one, as was found. QED 7