Problem Set - Solutons Due: Aprl 27, 2005. (a) Frst note that spam messages, nvtatons and other e-mal are all ndependent Posson processes, at rates pλ, qλ, and ( p q)λ. The event of the tme T at whch you decde about your popularty beng greater than some value t s the same as the event of havng less than 0 messages n each of the IMPORTANT and OTHER folders at tme t. Snce arrvals nto the IMPORTANT and OTHER folders are ndependent, the probablty that nether has had 0 arrvals by tme t s the product of the probabltes of each recevng less than 0 by that tme. Snce for each, the number of messages receved n [0,t] s a Posson random varable, we have P(T > t) = (λqt) e λqt (λ( p q)t) j e λ( p q)t! j! =0 j=0 (λqt) (λ( p q)t) j = e λ( p)t! j! =0 j=0 and the CDF s F T (t) = P(T > t). (b) At least 0 of the frst non-spam messages must be party nvtatons. Note that f I receve my 0th nvtaton by the tme Iget the th non-spam message, snce there wll be fewer than 0 messages collected n the OTHER folder by that tme, Iwll conclude that Iam very popular. Conversely, f Ihad fewer than 0 tems n the INVITATION folder by the tme Ihave receved non-spam messages, then Imust have at least 0 n the OTHER folder and would conclude that Iam not very popular (c) Havng showed n part (b) that gettng at least 0 nvtatons among the frst non-spam messages s necessary and suffcent to conclude very popular overall, we are lookng for the probablty of that event. The number of nvtatons n the frst non-spam messages s a bnomal random varable, snce each non-spam message s ndependently an nvtaton wth probablty q/( p). So, the probablty of concludng very popular s P(very popular) = (q/( p)) (( p q)/( p)). =0 Here s an alternatve soluton: The probablty that Iconclude that Iam very popular upon recevng kth non-spam message, 0 k, s the probablty that exactly ( out ) of the frst k are nvtatons and and the kth s also an nvtaton,.e., k (q/( p)) 0 (( p q)/( p)) k 0. Summng over k, k P(very popular) = (q/( p)) 0 (( p q)/( p)) k 0. k=0 It can be shown that ths s equal to the frst answer found above. Page of 5
2. (a) We have modeled the fouls as a Posson process wth parameter λ = 8. Ths mples that, neglectng foulng out, the number of fouls n the nterval [0,t] s a Posson random varable wth parameter λt = 8t. All we have to do s adjust for the fact that more than 6 fouls cannot be commted by assgnng the probabltes of 7, 8,... fouls to 6 to obtan: t/8 e (t/8)k, for k = 0,,...,5; k! p Xt (k) = t/8 (t/8) l e, for k = 6, l=6 t/8 e (t/8)k, for k = 0,,...,5; k! = 5 t/8 (t/8) l e, for k = 6. 5 48/8 (48/8)l p X48 (6) = e l=0 l=0 (b) Wallace has fouled out f at the end of the game he has sx fouls. Thus the probablty of foulng out s 6 = e 6 e 6 6 2 e 6 6 3 e 6 6 4 e 6 6 5 e 6 0.5543. 2 6 24 20 (c) When Wallace does not foul out, hs tme wns by (0.25 ponts/mnute) (48 mnutes) = 2 ponts. When Wallace fouls out at tme t, the score dfferental n favor of Wallace s team s 0.25t }{{} 0.5(48 t) 6 }{{}}{{} =0.75t 30. Wallace playng Wallace not playng Wallace s three techncal fouls Thus Wallace s team wns whenever 0.75t 30 > 0or t> 40. In terms of our model for Wallace s fouls, we are nterested n the sxth arrval tme. Wthout restrctng the game length to 48 mnutes for the moment, ths sxth arrval tme has the Erlang PDF of order 6 and parameter λ = 8. Denotng the sxth arrval tme by U, wehave λ 6 u 5 e λu f U (u) =, u 0. 5! The event of nterest s {U > 40} because ths captures Wallace not foulng out and Wallace foulng out late enough n the game that hs team wns. λ 6 5 λu u e P({Wallace s team wns}) = P(U > 40) = f U (u) du = du 6 5 = u e 8 u du 8 5! 40 07 = e 5 0.66 2 40 40 5! where the ntegral can be computed by repeated ntegraton by parts. Page 2 of 5
(d) Ths s very smlar to the prevous part. When Wallace commts hs ffth foul at tme t, the score dfferental n favor of Wallace s team s } 0.25t {{} 0.5(48 t) =0.75t 24. }{{} Wallace playng Wallace not playng Thus Wallace s team wns whenever 0.75t 24 > 0or t> 32. We are now nterested n the ffth arrval tme n the Wallace-foul process. Ths ffth arrval tme has the Erlang PDF of order 5 and parameter λ = 8. Denotng the ffth arrval tme by V, wehave λ 5 4 λv v e f V (v) =, v 0. 4! λ 5 v 4 e λv P({Wallace s team wns}) = P(V > 32) = f V (v) dv = dv 32 32 4! 5 4 = v e 8 v dv 8 4! 32 03 4 = e 0.62 3 where agan the ntegral can be computed by repeated ntegraton by parts. Incdentally, the strategy proposed n part (d) s clearly not optmal for maxmzng the probablty of vctory because the decson of whether or not to play Wallace wth fve fouls should depend also on the tme remanng n the game. For example, assumng the present model, removng Wallace less than 32 mnutes nto the game does not make sense because t leads to certan defeat. 3. (a) The state dagram of the Markov chan s: 0.5 0.5 0.5 2 3 4 5 6 7 0.5 (b) State 5 s reachable from state n a mnmum of three transtons. Paths from state to state 5 also nclude paths wth a loop from back to (of length 3) and/or a loop from 5 back to 5 by way of state 7 (ether length 2 or length 3). Therefore potental path lengths are 3 + 2m +3n, for m, n 0. Therefore, r 5 (n) > 0 for n = 3 or n 5. (c) From states, 2, and 3, all states are accessble because there s a non-zero probablty path from these states by way of state 3 to any other state. From states, 4, 5, 6, and 7, paths only exst to states 5, 6, and 7. Page 3 of 5
(d) States 5-7 are recurrent because by the logc n (c), they can be reached from any other state. States -4 are transent; once the system has transtoned out of state 4, t cannot return to any state other than states 5, 6, or 7. States 5, 6, and 7 form a recurrent class. Because t can be traversed from state 5 back to 5 n ether 2 or 3 steps (as dscussed n (b)), the system can return to state 5 after n steps for any n 2; therefore t s aperodc. (e) One transton must be added to create a sngle recurrent class: for example, addng a transton from state 5 to state would allow every state to be reached from every other state. Any transton from the recurrent class states 5,6, or 7 to any of the states, 2, or 3 would work. 4. (a) Gven L n, the hstory of the process (.e., L n 2,L n 3,...) s rrelevant for determnng the probablty dstrbuton of L n, the number of remanng unlocked doors at tme n. Therefore, L n s Markov. More precsely, P(L n = j L n =, L 2 2 = k,...,l = q) = P(L n = j L n = ) = p j. Clearly, at one step the number of unlocked doors can only decrease by one or stay constant. So, for d, f j =, then p j = P(selectng an unlocked door on day n+ L n = ) = d. For 0 d, f j =, then p j = P(selectng an locked door on day n + L n = ) = d d. Otherwse, p j = 0. To summarze, for 0, j d, we have the followng: d d j = p j = j = d 0 otherwse (b) The state wth 0 unlocked doors s the only recurrent state. All other states are then transent, because from each, there s a postve probablty of gong to state 0, from whch t s not possble to return. (c) Note that once all the doors are locked, none wll ever be unlocked agan. So the state 0 s an absorbng state: there s a postve probablty that the system wll enter t, and once t does, t wll reman there forever. Then, clearly, lm n r 0 (n) = and lm n r j (n) = 0 for all j = 0 and all,. (d) Now, f Ichoose a locked door, the number of unlocked doors wll ncrease by one the next day. Smlarly, the number of unlocked doors wll decrease by f and only f I choose an unlocked door. Hence, d d j = + p j = j = d 0 otherwse Clearly, from each state one can go to any other state and return wth postve probablty, hence all the states n ths Markov chan communcate and thus form one recurrent class. There are no transent states or absorbng states. Note however, that from an even-numbered state (states 0, 2, 4, etc) one can only go to an odd-numbered state Page 4 of 5
n one step, and smlarly all one-step transtons from odd-numbered states lead to even-numbered states. Snce the states can be grouped nto two groups such that all transtons from one lead to the other and vce versa, the chan s perodc wth perod 2. Ths wll lead to r j (n) oscllatng and not convergng as n. For example, r (n) = 0 for all odd n, but postve for even n. (e) In ths case L n s not a Markov process. To see ths, note that P(L n = + L n =, L n 2 = ) = 0 snce accordng to my strategy Ido not unlock doors two days n a row. But clearly, P(L n = + L n = ) > 0for <d snce t s possble to go from a state of unlocked doors to a state of + unlocked doors n general. Thus P(L n = + L n =, L n 2 = ) = P(L n = + L n = ), whch shows that L n does not have the Markov property. Page 5 of 5