Stochastic Processes

Transcription

1 MTHE/STAT455, STAT855 Fall 202 Stochastic Processes Final Exam, Solutions (5 marks) (a) (0 marks) Condition on the first pair to bond; each of the n adjacent pairs is equally likely to bond Given that (m r, m r+ ) is the first pair to bond the probability that m will end up isolated is p r ; this holds for r 2,, n If (m, m 2 ) is the first pair to bond then the probability that m ends up isolated is 0 By the law of total probability we have p n n (p + p p n 2 ) The rest is working out p n Since p n n 2 (p + p p n 3 ), by subtraction we get (n )p n (n 2)p n p n 2 (n )(p n p n ) (p n p n 2 ) Therefore, recursively we have p n p n n (p n p n 2 ) ( ) 2 (n )(n 2) (p n 2 p n 3 ) ( ) n 2 (n ) (2) (p 2 p ) ( )n (n )!, where the last equality follows since p and p 2 0 Recursively again, we get p n ( )n (n )! p n ( )n (n )! + ( )n 2 (n 2)! + p n n 2 r0 ( ) r r! (b) (5 marks) For n 5 we can go through all cases of which pair bonds first If (m, m 2 ) bonds first, then m 3 is isolated with probability p 3 (it is at the end of a 3 molecule line) If (m 2, m 3 ) bonds first or if (m 3, m 4 ) bonds first then m 3 is isolated with probability 0 If (m 4, m 5 ) bonds first then m 3 is isolated with probability p 3 Thus, we get that the probability that m 3 is isolated is 4 (p 3 + p 3 ) p ( ) ( )

2 MTHE/STAT455, STAT Final Exam Solutions, 202 Page 2 of 7 2 (5 marks) (a) (2 marks) If p ij > 0 then from the definition of p ij we have that q ij > 0 and a ij > 0 Since a ij > 0, from the definition of a ij we have that q ji > 0 (since f(j) > 0) But then a ji must be positive also, hence p ji q ji a ji > 0 The reverse direction is similar (b) (8 marks) We check the local balance equations: p ij p ji, (which we only need to consider for pairs of states i and j such that p ij > 0 and p ji > 0), which are q ij a ij q ji a ji Now, if then a ij balance equation for states i and j is and a ji In this case the local q ij q ji f(i) f(j) If then a ij and a ji, and in this case the local balance equation for states i and j is q ij q ji f(i) f(j) So in all cases where p ij > 0 and p ji > 0 the local balance equation for states i and j reduces to Thus, π f(i) f(j) i Cf(i), where C is the appropriate normalizing constant: C [ i S f(i)] (and the sum is convergent since i S f(i) < by assumption) Since the local balance equations are satisfied the process X is time reversible (c) (5 marks) It is easy to see that with a ij as given, p ij > 0 if and only if p ji > 0 For states i and j such that p ij > 0 and p ji > 0, the local balance equation for states i and j is q ij q ji + + f(i) f(j) So we see that the stationary distribution of X here is the same as in part(b), and since the local balance equations are satisfied X is time reversible

3 MTHE/STAT455, STAT Final Exam Solutions, 202 Page 3 of 7 3 (5 marks) (a) (8 marks) We check the local balance equations: for n 0,,, which give π n λ n + π n+µ π n+ λ n + µ π n (n + )n (n + )! ( λ µ ) 2 π n ( ) n+ λ π 0 µ Thus, we see that π n (λ/µ)n e λ/µ for n 0 (ie, s the Poisson distribution with parameter λ/µ) (b) (7 marks) Let ψ (ψ n ) n 0 denote the stationary distribution of the embedded discrete time jump chain We can use the fact that ψ n is proportional to π n v n, where v n is the parameter of the exponential holding time in state n We have v n λ + µ for n and v n+ 0 λ Therefore, for an appropriate normalizing constant C, we have ) ψ n Cπ n v n C (µ (λ/µ)n+ (n + )! e λ/µ + µ (λ/µ)n e λ/µ for n and ψ 0 Cµ (λ/µ) e λ/µ! To find C we sum ψ n over n, giving ψ n Cµ( e λ/µ + ) Cµ(2 e λ/µ ), giving C µ(2 e λ/µ ), and for n and ψ 0 ( ) + (λ/µ)/(n + ) (λ/µ) n ψ n 2 e λ/µ λ/µ 2 e λ/µ e λ/µ e λ/µ

4 MTHE/STAT455, STAT Final Exam Solutions, 202 Page 4 of 7 4 (5 marks) (a) (8 marks) For t, conditioned on N(t) n, for n, the probability that the firefly is illuminated at time t is P (U (n) > t ), where U (n) max(u,, U n ) is the maximum of n independent Uniform(0, t) random variables U,, U n We compute P (U (n) > t ) P (U (n) t ) P (U t,, U n t ) P (U t ) P (U n t ) P (U t ) n ( ) n t, t where the second equality follows because the maximum is less than or equal to t if and only if all are, the third equality follows by independence, and the fourth equality follows because the U i are identically distributed Conditioned on N(t) 0 the probability that the firefly is illuminated at time t is 0, so the above works for n 0 as well Then the unconditional probability that the firefly is illuminated at time t is [ ( ) n ] t (λt) n e λt t (λ(t )) n e λt e λ (λ(t )) n e λ e λ(t ) For t < the firefly is illuminated at time t if and only if there was at least one event in [0, t], and this probability is e λt (b) (7 marks) Define an event to be a type f event if it is a Type event and it occurred in the interval (t, t) The probability that an event at time s is a type f event is p (s)i (t,t) (s) The number of type f events by time t is exactly the number of illuminated fireflies at time t So (by a proposition in class) the distribution of N (t) is Poisson with parameter λ t 0 t p (s)i (t,t) (s)ds λ p (s)ds t

5 MTHE/STAT455, STAT Final Exam Solutions, 202 Page 5 of 7 5 (5 marks) (a) (2 marks) If a discrete time Markov chain is time reversible then we must have p ji > 0 whenever p ij > 0 So if p ij > 0 then p ii p ij p ji > 0 So for any state i that can be left with positive probability, it can be returned to in 2 steps with positive probability Thus the period of state i cannot be greater than 2 If the probability of leaving state i is 0, then it will be returned to in step with probability ; again the period of state i cannot be greater than 2 (in this case it would be ) (b) (3 marks) (i) (4 marks) Since x xp we have that x xp n for any n (multiply both sides by P n times, each time reducing xp to x), or x i j S x jp ji (n) for all i S and all n Therefore, (ii) (4 marks) We have q ij (n) x j p ji (n) x i x i x i j S j S q ik ()q kj (n) x k p ki () x j p jk (n) x i x k x j p ki ()p jk (n) x i x j x i p ji (n + ) q ij (n + ), where p ki()p jk (n) p ji (n+) by the Chapman Kolmogorov equations In matrix form this is Q(n + ) Q()Q(n) QQ(n) Recursing this down we get Q(n + ) Q n+ (iii) (5 marks) By irreducibility, p ji (n) > 0 for some n and so q ij (n) > 0 for this n Since i and j are arbitrary we get irreducibility for the Markov chain with transition matrix Q For recurrence use the fact that q ii(n) p ii(n), where the last equality is true because all states in the X chain are recurrent (and we are using the proposition from class that a state i is recurrent if and only if p ii(n) ) By this proposition again state i in the chain with transition matrix Q is recurrent