Homework 4 (Sas 62, Winer 217) Due Tuesday Feb 14, in class Quesions are derived from problems in Sochasic Processes by S. Ross. 1. Le A() and Y () denoe respecively he age and excess a. Find: (a) P{Y () > x A() s}. (b) P{Y () > x A( + x/2) s}. (c) P{Y () > x A( + x) > s} for a Poisson process. (d) P{Y () > x, A() > y}. (e) If µ <, show ha, wih probabiliy 1, A()/ as. Hin: For (d), use a regeneraive process argumen (E.g. Ross, secion 3.7) o find P(Y () > x, A() > y). For (e), you may use wihou proof he following resuls on convergence wih probabiliy 1: (L1) n S n /n µ; (L2) N() ; (L3) N()/ 1/µ. Soluion: (a) P[Y () > x A() s] P[X N()+1 > s + x S N() s] Here is a more formal soluion: (b) P[Y () > x A() s] P[X 1 > s + x X 1 > s] F (s + x)/f (s). P[S N()+1 > + x S N() s] P[X N()+1 > s + x S N() s] P[X n+1 > s + x S n s, N() n]p[n() n S N() s] n P[X n+1 > s + x S n s, X n+1 > s]p[n() n S N() s] n P[X n+1 > s + x X n+1 > s]p[n() n S N() s](by independence) n P[X 1 > s + x X 1 > s] F (s + x)/f (s) p : P[Y () > x A( + x/2) s] For s x/2, an argumen similar o (a) allows us o wrie p P[no even in ( + x/2 s, + x] even a + x/2 s, no evens in ( + x/2 s, + x/2]] P[X N()+1 > s + x/2 S N() + x/2 s, X N()+1 > s] P[X 1 > s + x/2 X 1 > s] F (s + x/2)/f (s) For s < x/2, {A( + x/2) s} {Y () s x/2}. I follows ha p. 1
(c) q P[Y () > x A( + x) > s] P [ no even in [, + x] no evens in [ + x s, + x] ] for s x, since he process is Poisson wih independen incremens, q P [ no even in [, + x s] ] exp λ(x s), where λ is he rae of he Poisson process. For s > x, q 1. (d) P(Y () > x, A() > y) P(X N()+1 > S N() + x, S N() > y) P(X 1 > + x, > y S N() )P(S N() ) + P(X N()+1 > S N() + x, S N() > y S N() s, X N()+1 > S N() )df SN() (s) 1 {>y} P(X 1 > + x X 1 > )P(S N() ) + 1 { s>y} P(X > + x s X > s)df SN() (s) 1 {>y} F ( + x) + y F ( + x s)dm(s). Le P P[Y () > x, A() > y]. Define a regeneraive process o be on a if S N() < y and S N()+1 > + x. Thus, P is he probabiliy ha he process is on a ime. By he regeneraive process i heorem (e) P E[ime on during a cycle] E[ime of he cycle] E[max(X 1 (x + y), )] µ 1 µ x+y (z x y) df (z) A() S N() N() 1 µ/µ (by (L1), (L2) and (L3)) S N() 1 N() 2. Consider a single-server bank in which poenial cusomers arrive in accordance wih a renewal process having inerarrival disribuion F. However, an arrival only eners he bank if he 2
server is free when he or she arrives; oherwise, he individual goes elsewhere wihou being served. Would he number of evens by ime consiue a (possibly delayed) renewal process if an even corresponds o a cusomer: (a) enering he bank? (b) leaving he bank afer being served? Wha if F were exponenial? Soluion: Le X i denoe he lengh of he i-h service and le Y i denoe he ime from he end of i-h service unil he sar of he i + 1-h service. Le Y denoe he ime when firs arrival eners he bank (and ges service). Noe ha X i and Y i may be dependen when he arrival is no a Poisson process. (a) In his case, each cycle consiss of Z i X i + Y i, i 1, 2,... and Z Y. Since X i and Y i are independen of X j and Y j wih j 1,..., i 1, {Z i } i N are i.i.d copies. We hus have a delayed renewal process. (b) In his case, Z i Y i 1 +X i. When X i and Y i are dependen, {Z i } i N are no i.i.d. copies. We do no have a (delayed) renewal process. One couner example can be consruced as in he sequel. Suppose he service disribuion is given by { 1 w.p..5 Y 1 1 w.p..5 and he inerarrival imes of he cusomers o he bank Z n F are given by, Z 1 6 w.p. 1. Then, given a previous inerval beween deparures S n S n 1 3, we know ha he nex arrival will ener he bank a ime S n + 4. If F is exponenial (a) sill gives a delayed renewal process. (b) now resuls in a (non-delayed) renewal process, since he memoryless propery implies ha Y i is independen of X i, i N. Hence, {Z i } i N are i.i.d. copies. 3. On each be a gambler, independenly of he pas, eiher wins or loses 1 uni wih respecive probabiliy p and 1 p. Suppose he gambler s sraegy is o qui playing he firs ime she wins k consecuive bes. A he momen she quis (a) find her expeced winnings. (b) find he expeced number of bes ha she has won. Hin: I may help you o look a Example 3.5(A) in Ross. Soluion: Le and { 1 if nh game is a win Y n else { 1 if nh game is a win X n 1 else and le N inf{n k : n mn k+1 X m k} inf{n k : n mn k+1 Y m k}, he firs ime k consecuive games are won. Le W N i1 X i, he gamblers oal winnings. Also le N W N i1 Y i, he number of games won. 3
(a) From Ross, Example 3.5A, E[N] k i1 (1/p)i. Also N is a sopping ime w.r. X i, i 1, 2,.... By Wald s equaion, we have ( k ) 1 E[W ] E[N] E[X 1 ] p i (2p 1) (b) N is also a sopping ime w.r.. Y i, i 1,, so Wald s equaion gives E[N W ] E[N]E[Y 1 ] i1 ( k i1 ) 1 k 1 1 p i p p i i 4. Prove Blackwell s heorem for renewal reward processes. Tha is, assuming ha he cycle disribuion is no laice, show ha, as, E [ reward in(, + a) ] a E[reward incycle] E[ime of cycle]. Assume ha any relevan funcion is direcly Riemann inegrable. Hin: You may adop an informal approach by assuming ha one can wrie [ +a ] +a E dr(s) E [ dr(s) ], and hen developing he ideniy E[dR()] E[R 1 X 1 ]df () + {E[R 1 X 1 x]df ( x)} dm(x). If you can find a more elegan or more rigorous soluion, ha would also be good! Soluion: Le R() be he reward accumulaed by ime. Then, E[reward in (, + a)] E[R( + a) R()] E[ +a +a dr(s)] E[ dr(s)] assuming ha he inerchange is allowed, e.g. if R() is increasing. Now, E[ dr()] E[E[ dr() S N() ]] E[ dr() S N() ]P[S N() ] + E[ dr() S N() ]F () + 4 E[ dr() S N() y] df SN() (y) E[ dr() S N() y]f ( y) dm(y).
Now, since R() only increases when an even occurs, E[ dr() S N() y] E[R N()+1 X N()+1 y]df XN()+1 S N() y( y) This esablished he ideniy E[R 1 X 1 y] E[ dr()] E[R 1 X 1 ] df () + Now he key renewal heorem gives Thus df ( y) F ( y). (E[R 1 X 1 y] df ( y)) dm(x). E[ dr()] E[R 1 X 1 ] df () d E[R 1 ] d. +a Anoher approach: Noe ha E[ dr()] +a N(+a) N() E[reward in (, + a)] E[ R n R n ] E[R 1 ] ds a E[R 1] µ. N(+a)+1 N()+1 E[ R n ] E[ R n ] + E[R N()+1 ] E[R N(+a)+1 ] Now N( + a) + 1 and N() + 1 are sopping imes for he sequence (X i, R i ), i 1,. Thus from (generalized) Wald s equaion and N()+1 E[ N(+a)+1 E[ R n ] E[N() + 1]E[R] where E[R] is he expeced reward in a cycle. Thus R n ] E[N( + a) + 1]E[R], E[reward in (, + a)] E[N( + a) + 1]E[R] E[N() + 1]E[R] + E[R N()+1 ] E[R N(+a)+1 ] (m( + a) m())e[r] + E[R N()+1 ] E[R N(+a)+1 ]. Now (m( + a) m())e[r] ae[r]/e[x]. 5
from Blackwell s heorem. Now, i suffices o show ha E[R N()+1 ] exiss and is finie. Indeed, E[R N()+1 ] E[R N()+1 S N() ]F () + E[R X > ]F () + h() + h( s) dm(s), E[R N()+1 S N() s]f ( s)dm(s) E[R X > s]f ( s) dm(s) where h() E[R X > ]F (). Then by he Key Renewal heorem, we have E[R N()+1] 1 E[X] Here we assumed ha h() is direcly Riemann inegrable. h(s) ds. 5. The life of a car is a random variable wih disribuion F. An individual has a policy of rading in his car eiher when i fails or reaches he age of A. Le R(A) denoe he resale value of an A-year-old car. There is no resale value of a failed car. Le C 1 denoe he cos of a new car and suppose ha an addiional cos C 2 is incurred whenever he car fails. (a) Say ha a cycle begins each ime a new car is purchased. Compue he long-run average cos per uni ime. (b) Say ha a cycle begins each ime a car in use fails. Compue he long-run average cos per uni ime. Noe: In boh (a) and (b) you are expeced o compue he raio of he expeced cos incurred in a cycle o he expeced ime of a cycle. The answer should, of course, be he same in boh pars. Soluion: and (a) Clearly, E[cos per cycle] C 1 F (A)R(A) + F (A)C 2 E[ime of cycle] A xdf (x) + A(1 F (A)). So, reaing he cos as he reward, he renewal reward heorem gives E[accumulaed cos by ] E[cos per cycle] E[ime of cycle] C 1 F (A)R(A) + F (A)C 2 x df (x) + AF (A) (b) The chance ha a car fails is F (A), so he number, N, of cars bough beween failures has he geomeric disribuion wih parameer p F (A). We have, E[cos per cycle] E[NC 1 (N 1)R(A) + C 2 ] C 1 /F (A) + (1 1/F (A))R(A) + C 2 A 6
and E[ime of cycle] E[(N 1)A]+E[car life car life < A] F (A)A/F (A)+ Thus, E[accumulaed cos by ] A C 1/F (A) + (1 1/F (A))R(A) + C 2 F (A)A/F (A) +. A x df (x)/f (A) x df (x)/f (A). Muliplying numeraor and denominaor by F (A) gives he same expression as in (a). Recommended reading: Secions 3.4 hrough 3.7, excluding subsecions 3.4.3, 3.6.1, 3.7.1. We will no cover he maerial in Secion 3.8, hough you may like o look hrough i. Supplemenary exercises: 3.24, 3.27, 3.35. These are opional, bu recommended. Do no urn in soluions hey are in he back of he book. 7