6.041/6.431 Fall 2010 Final Exam Solutions Wednesday, December 15, 9:00AM - 12:00noon.

Transcription

1 604/643 Fall 200 Final Exam Solutions Wednesday, December 5, 9:00AM - 2:00noon Problem (32 points) Consider a Markov chain {X n ; n 0,, }, specified by the following transition diagram (4 points) Given that the chain starts with X 0, find the probability that X 2 2 The two-step transition probability is: r 2 (2) p p 2 + p 2 p (4 points) Find the steady-state probabilities π, π 2, π 3 of the different states We set up the balance equations of a birth-death process and the normalization equation as such: π p 2 π 2 p 2 π 2 p 23 π 3 p 32 π + π 2 + π 3 Solving the system of equations yields the following steady state probabilities: π /9 π 2 2/9 π 3 6/9 In case you did not do part 2 correctly, in all subsequent parts of this problem you can just use the symbols π i : you do not need to plug in actual numbers 3 (4 points) Let Y n X n X n Thus, Y n indicates that the nth transition was to the right, Y n 0 indicates it was a self-transition, and Y n indicates it was a transition to the left Find lim P(Y n ) n Using the total probability theorem and steady state probabilities, 3 lim P(Y n ) π i P(Y n X n i) n i π p 2 + π 2 p 23 /9 Page of 9

2 4 (4 points) Is the sequence Y n a Markov chain? Justify your answer No Assume the Markov process is in steady state To satisfy the Markov property, For large n, P(Y n Y n, Y n 2 ) P(Y n Y n ) P(Y n Y n, Y n 2 ) 0, since it is not possible to move upwards 3 times in a row However in steady state, P({Y n } {Y n }) P(Y n Y n ) P(Y n ) π p 2 p 23 π p 2 + π 2 p 23 0 Therefore, the sequence Y n is not a Markov chain 5 (4 points) Given that the nth transition was a transition to the right (Y n ), find the probability that the previous state was state (You can assume that n is large) Using Bayes Rule, P(X n Y n ) 3 P(X n )P(Y n X n ) i P(X n i)p(y n X n i) π p 2 π p 2 + π 2 p 23 2/5 6 (4 points) Suppose that X 0 Let T be defined as the first positive time at which the state is again equal to Show how to find E[T ] (It is enough to write down whatever equation(s) needs to be solved; you do not have to actually solve it/them or to produce a numerical answer) In order to find the the mean recurrence time of state, the mean first passage times to state are first calculated by solving the following system of equations: t 2 + p 22 t 2 + p 23 t 3 t 3 + p 32 t 2 + p 33 t 3 The mean recurrence time of state is then t + p 2t 2 Solving the system of equations yields t 2 20 and t 3 30 and t 9 7 (4 points) Does the sequence X, X 2, X 3, converge in probability? If yes, to what? If not, just say no without explanation No Page 2 of 9

3 8 (4 points) Let Z n max{x,, X n } Does the sequence Z, Z 2, Z 3, converge in probability? If yes, to what? If not, just say no without explanation Yes The sequence converges to 3 in probability For the original markov chain, states {, 2, 3} form one single recurrent class Therefore, the Markov process will eventually visit each state with probability In this case, the sequence Z n will, with probability, converge to 3 once X n visits 3 for the first time Problem 2 (68 points) Alice shows up at an Athena * cluster at time zero and spends her time exclusively in typing s The times that her s are sent are a Poisson process with rate λ A per hour (3 points) What is the probability that Alice sent exactly three s during the time interval [, 2]? The number of s Alice sends in the interval [, 2] is a Poisson random variable with parameter λ A So we have: λ A 3 e λ A P(3, ) 3! 2 Let Y and Y 2 be the times at which Alice s first and second s were sent (a) (3 points) Find E[Y 2 Y ] Define T 2 as the second inter-arrival time in Alice s Poisson process Then: Y 2 Y + T 2 E[Y 2 Y ] E[Y + T 2 Y ] Y + E[T 2 ] Y + /λ A (b) (3 points) Find the PDF of Y 2 Let Z Y 2 Then we first find the CDF of Z and differentiate to find the PDF of Z: { F Z (z) P(Y 2 z) P( z Y e z) λ A z z 0 0 z < 0 ( ) f Z (z) df Z (z) λ A e λ A z z /2 dz 2 { λ A f Z (z) 2 z e λ A z z 0 0 z < 0 (z 0) (c) (3 points) Find the joint PDF of Y and Y 2 f Y,Y 2 (y, y 2 ) f Y (y )f Y2 Y (y 2 y ) f Y (y )f T2 (y 2 y ) λ { A e λ Ay λ A e λ A(y 2 y ) λ 2 A e λ Ay 2 y 2 y 0 0 otherwise y 2 y 0 *Athena is MIT's UNIX-based computing environment OCW does not provide access to it Page 3 of 9

4 3 You show up at time and you are told that Alice has sent exactly one so far (Only give answers here, no need to justify them) (a) (3 points) What is the conditional expectation of Y 2 given this information? Let A be the event {exactly one arrival in the interval [0,]} Looking forward from time t, the time until the next arrival is simply an exponential random variable (T ) So, E[Y 2 A] + E[T ] + /λ A (b) (3 points) What is the conditional expectation of Y given this information? Given A, the times in this interval are equally likely for the arrival Y Thus, E[Y A] /2 4 Bob just finished exercising (without access) and sits next to Alice at time He starts typing s at time, and fires them according to an independent Poisson process with rate λ B (a) (5 points) What is the PMF of the total number of s sent by the two of them together during the interval [0, 2]? Let K be the total number of s sent in [0, 2] Let K be the total number of s sent in [0, ), and let K 2 be the total number of s sent in [, 2] Then K K + K 2 where K is a Poisson random variable with parameter λ A and K 2 is a Poisson random variable with parameter λ A + λ B (since the s sent by both Alice and Bob after time t arrive according to the merged Poisson process of Alice s s and Bob s s) Since K is the sum of independent Poisson random variables, K is a Poisson random variable with parameter 2λ A + λ B So K has the distribution: (2λ A + λ B ) k e (2λ A+λ B ) p K (k) k 0,, k! (b) (5 points) What is the expected value of the total typing time associated with the that Alice is typing at the time that Bob shows up? (Here, total typing time includes the time that Alice spent on that both before and after Bob s arrival) The total typing time Q associated with the that Alice is typing at the time Bob shows up is the sum of S 0, the length of time between Alice s last or time 0 (whichever is later) and time, and T, the length of time from to the time at which Alice sends her current T is exponential with parameter λ A and S 0 min{t 0, }, where T 0 is exponential with parameter λ A Then, Q S 0 + T min{t 0, } + T and E[Q] E[S 0 ] + E[T ] We have: E[T ] /λ A Page 4 of 9

5 We can find E[S 0 ] via the law of total expectations: E[S 0 ] E[min{T 0, }] P(T 0 )E[T 0 T 0 ] + P(T 0 > )E[ T 0 > ] ( ) e λ A tf T T0 (t) dt + e λ A 0 ( ) λ A e λ At e λ A t dt + e λ A ( e λ A 0 ) tλ A e λ At dt + e λ A 0 tλ 2 Ae λ At dt + e λ A λ A 0 ( ) e λ A λ A e λ A + e λ A λ A ( ) e λ A λ A where the above integral is evaluated by manipulating the integrand into an Erlang order 2 PDF and equating the integral of this PDF from 0 to to the probability that there are 2 or more arrivals in the first hour (ie P(Y 2 < ) P(0, ) P(, )) Alternatively, one can integrate by parts and arrive at the same result Combining the above expectations: ( ) ( ) E[Q] E[S 0 ] + E[T ] e λ A + 2 e λ A λ A λ A λ A (c) (5 points) What is the expected value of the time until each one of them has sent at least one ? (Note that we count time starting from time 0, and we take into account any s possibly sent out by Alice during the interval [0, ]) Define U as the time from t 0 until each person has sent at least one Define V as the remaining time from when Bob arrives (time ) until each person has sent at least one (so V U ) Define S as the time until Bob sends his first after time Define the event A {Alice sends one or more s in the time interval [0, ]} {Y }, where Y is the time Alice sends her first Define the event B {After time, Bob sends his next before Alice does}, which is equivalent to the event where the next arrival in the merged process from Alice and Bob s orginal processes (starting from time ) comes from Bob s process We have: P(A) P(Y ) e λ A P(B) λ B λ A + λ B Page 5 of 9

6 Then, E[U] P(A)E[U A] + P(A c )E[U A c ] ( e λ A )( + E[V A]) + e λ A ( + E[V A c ]) ( e λ A )( + E[V A]) + e λ A ( + P(B A c )E[V B A c ] + P(B c A c )E[V B c A c ]) ( e λ A )( + E[V A]) + e λ A ( + P(B)E[V B A c ] + P(B c )E[V B c A c ]) ( ) ( e λ A )( + E[V A]) + e λ λ A B + E[V B A c λa ] + E[V B c A c ] λa + λ B λa + λ B Note that E[V B c A c ] is the expected value of the time until each of them sends one after time (since, given A c, Alice did not send any in the interval [0, ]) and given Alice sends an before Bob Then this is the expected time until an arrival in the merged process followed by the expected time until an arrival in Bob s process So, E[V B c A c ] + λ A +λ B λ B Similarly, E[V B A c ] is the time until each sends an after time, given Bob sends an before Alice So E[V B A c ] λ A +λ B + λ A Also, E[V A] is the expected time it takes for Bob to send his first after time (since, given A, Alice already sent an in the interval [0, ]) So E[V A] E[S] /λ B Combining all of this with the above, we have: E[U] ( e λ A )( + /λ B ) ( ( ) ( )) +e λ λ A B + + λ A + + λ A + λ B λ A + λ B λ A λ A + λ B λ A + λ B λ B (d) (5 points) Given that a total of 0 s were sent during the interval [0, 2], what is the probability that exactly 4 of them were sent by Alice? P(Alice sent 4 in [0, 2] total 0 sent in [0, 2]) P(Alice sent 4 in [0, 2] total 0 sent in [0, 2]) P(total 0 sent in [0, 2]) P(Alice sent 4 in [0, 2] Bob sent 6 [0, 2]) P(total 0 sent in [0, 2]) ( ) ( ) (2λ A ) 4 e 2λ A (λ B ) 6 e λ B 4! 6! (2λ A +λ B ) 0 e 2λ A +λ B 0! ( )( ) 4 ( ) 0 2λ 6 A λ B 4 2λ A + λ B 2λ A + λ B As the form of the solution suggests, the problem can be solved alternatively by computing the probability of a single being sent by Alice, given it was sent in the interval [0, 2] This can be found by viewing the number of s sent by Alice in [0, 2] as the number of arrivals arising from a Poisson process with twice the rate (2λ A ) in an interval of half the duration (particularly, the interval [, 2]), then merging this process with Bob s process Then the probability that an sent in the interval [0, 2] was sent by Alice is the probability that an arrival in this new merged process came from the newly constructed 2λ A rate process: Page 6 of 9

7 2λ A p 2λ A + λ B Then, out of 0 s, the probability that 4 came from Alice is simply a binomial probability with 4 successes in 0 trials, which agrees with the solution above 5 (5 points) Suppose that λ A 4 Use Chebyshev s inequality to find an upper bound on the probability that Alice sent at least 5 s during the time interval [0, ] Does the Markov inequality provide a better bound? Let N be the number of s Alice sent in the interval [0, ] Since N is a Poisson random variable with parameter λ A, E[N] var(n) λ A 4 To apply the Chebyshev inequality, we recognize: P(N 5) P(N 4 ) P( N 4 ) var(n) 4 2 In this case, the upper-bound of 4 found by application of the Chebyshev inequality is uninformative, as we already knew P(N 5) To find a better bound on this probability, use the Markov inequality, which gives: P(N 5) E[ N] (5 points) You do not know λ A but you watch Alice for an hour and see that she sent exactly 5 s Derive the maximum likelihood estimate of λ A based on this information λˆa arg max log (p N (5; λ)) λ ( ) λ 5 e λ arg max log λ 5! arg max log(5!) + 5log(λ) λ λ Setting the first derivative to zero 5 λ 0 λˆa 5 Page 7 of 9

8 7 (5 points) We have reasons to believe that λ A is a large number Let N be the number of s sent during the interval [0, ] Justify why the CLT can be applied to N, and give a precise statement of the CLT in this case With λ A large, we assume λ A For simplicity, assume λ A is an integer We can divide the interval [0, ] into λ A disjoint intervals, each with duration /λ A, so that these intervals span the entire interval from [0, ] Let N i be the number of arrivals in the ith such interval, so that the N i s are independent, identically distributed Poisson random variables with parameter Since N is defined as the number of arrivals in the interval [0, ], then N N + + N λa Since λ A, then N is the sum of a large number of independent and identically distributed random variables, where the distribution of N i does not change as the number of terms in the sum increases Hence, N is approximately normal with mean λ A and variance λ A If λ A is not an integer, the same argument holds, except that instead of having λ A intervals, we have an integer number of intervals equal to the integer part of λ A (λ A floor(λ A )) of length /λ A and an extra interval of a shorter length (λ A λ A)/λ A Now, N is a sum of λ A independent, identically distributed Poisson random variables with parameter added to another Poisson random variable (also independent of all the other Poisson random variables) with parameter (λ A λ A) In this case, N would need a small correction to apply the central limit theorem as we are familiar with it; however, it turns out that even without this correction, adding the extra Poisson random variable does not preclude the distribution of N from being approximately normal, for large λ A, and the central limit theorem still applies To arrive at a precise statement of the CLT, we must standardize N by subtracting its mean then dividing by its standard deviation After having done so, the CDF of the standardized version of N should converge to the standard normal CDF as the number of terms in the sum approaches infinity (as λ A ) Therefore, the precise statement of the CLT when applied to N is: ( ) N λ A lim P z Φ(z) λ A where Φ(z) is the standard normal CDF λa 8 (5 points) Under the same assumption as in last part, that λ A is large, you can now pretend that N is a normal random variable Suppose that you observe the value of N Give an (approximately) 95% confidence interval for λ A State precisely what approximations you are making Possibly useful facts: The cumulative normal distribution satisfies Φ(645) 095 and Φ(96) 0975 We begin by estimating λ A with its ML estimator Λˆ A N, where E[N] λ A With λ A large, the CLT applies, and we can assume N has an approximately normal distribution Since var(n) λ A, we can also approximate the variance of N with ML estimator for λ A, so var(n) N, and σ N N Page 8 of 9

9 To find the 95% confidence interval, we find β such that: 095 P ( N λ A β) ) N λ P A β N N ( ) β 2Φ N So, we find: Thus, we can write: β NΦ (0975) 96 N P(N 96 N λ A N + 96 N) 095 So, the approximate 95% confidence interval is: [N 96 N, N + 96 N] 9 You are now told that λ A is actually the realized value of an exponential random variable Λ, with parameter 2: f Λ (λ) 2e 2λ, λ 0 (a) (5 points) Find E[N 2 ] E[N 2 ] E[E[N 2 Λ]] E[var(N Λ) + (E[N Λ]) 2 ] E[Λ + Λ 2 ] E[Λ] + var(λ) + (E[Λ]) (b) (5 points) Find the linear least squares estimator of Λ given N ˆ cov(n, Λ) Λ LLMS E[Λ] + (N E[N]) var(n) Solving for the above quantities: E[Λ] 2 E[N] E[E[N Λ]] E[Λ] 2 3 var(n) E[N 2 ] (E[N]) cov(n, Λ) E[NΛ] E[N]E[Λ] E[E[NΛ Λ]] (E[Λ]) 2 E[Λ 2 ] (E[Λ]) 2 var(λ) 4 Page 9 of 9

10 Substituting these into the equation above: ˆΛ LLMS cov(n, Λ) E[Λ] + (N E[N]) var(n) 2 + /4 ( N ) 3/4 2 (N + ) 3 Page 0 of 9

11 MIT OpenCourseWare / 643 Probabilistic Systems Analysis and Applied Probability Fall 200 For information about citing these materials or our Terms of Use, visit: