Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli s with parameter p, ad these have MGF pe s +q. Therefore, { M (s) = Eexp s X } p = e s [ p/q pe s/ pq +q ] = [ pe s q/p +qe s p/q ]. pq (b) Compute the limit lim M (s), directly, without usig the cetral limit theorem. We wat to write M (s) i the form (1+ a ). Solvig for a gives a = [ pe s q/p +qe s p/q 1 ]. Recalligthattheexpasiooftheexpoetial ise x = 1+x+x 2 /2+ suggests that this should be rewritte i the form Sice a = p [ e s q/p 1 s q/p ] +q [ e s p/q 1+s p/q ]. (1) lim x e x 1 x x 2 = 1 2 (by applyig L Hopital s rule twice), lim a = s2 2. So, lim M (s) = e s2 /2, which is the mgf of the N(,1) distributio. F 5. Suppose X is Poisso with parameter. (a) Fid the momet geeratig fuctio M (s) of (X )/. Sice the MGF of the Poisso with parameter is e (es 1), M (s) = e s +(e s/ 1). (b) Compute the limit lim M (s), directly, without usig the cetral limit theorem. Takig logs gives [ logm (s) = e s/ s ] 1, 1
2 so usig (1) agai gives so that as i Problem F 4. lim logm (s) = s2 2, lim M (s) = e s2 /2 G 2. Suppose the radom variables X satisfy EX =, EX 2 1, ad Cov(X,X m ) for m. Show that S = X 1 + +X coverges to i probability. Solutio: The proof follows the proof of the WLLN uder a secod momet assumptio: By Chebyshev, But var(s ) = Combiig these gives P( S / ǫ) 1 ǫ 2 2var(S ). cov(x i,x j ) i,j=1 var(x i ). i=1 P( S / ǫ) 1 ǫ 2, which teds to zero as. G 3. Show that i each of the cases (a), (c), ad (d) of Problem 5 o page 288, the sequece actually coverges a.s. Solutio: For (a): EY 2 = 1, so 3 2 EY 2 <. Therefore Y 2 a.s., so Y a.s. For (c), E Y = 1, so 2 E Y <. Therefore Y a.s. For (d), there are two possible approaches: Oe is to show that E(1 Y ) 2 8 =, ad proceed as i the other cases. The (+1)(+2) other is to ote that Y is odecreasig i ad is bouded above by 1. Therefore, Y = lim Y exists for every ω, ad satisfies Y 1. To show that Y = 1 a.s., take < ǫ < 1 ad write P(Y 1 ǫ) P(Y 1 ǫ) = (1 (ǫ/2)), which teds to zero as. Therefore P(Y < 1) = lim ǫ P(Y 1 ǫ) =.
G 4. Suppose each X takes the values ±1 with probability 1 each. 2 Show that the radom series X p coverges a.s. (which meas that the partial sums coverge a.s.) if p > 1. Solutio: The series coverges absolutely, sice X 1 = p <. p Now use the fact that absolute covergece of a series implies covergece. H 1. Suppose X are i.i.d. o-egative radom variables. (a) Show that X i probability with o further assumptios. (You did this before i case they are uiformly distributed o [ 1, 1].) Solutio: P(X / > ǫ) = P(X 1 > ǫ) as. Cosider ow two cases: (i) EX < ad (ii) EX =. Recall that the series P(X 1 > k) = P(X k > k) k k coverges i case (i) ad diverges i case (ii). (See Problem 3 o page 184. This gives the statemet i terms of itegrals rather tha sums, but there is o real differece.) (b) Express P(X k k for all k ) i terms of the probabilities P(X k > k). Solutio: [1 P(X 1 > k)]. (c) Show that k= lim P(X k k for all k ) = 1 i case (i) ad P(X k k for all k ) = for all i case (ii). (Suggestio: take logs.) 3
4 log(1+x) Solutio: Sice lim x = 1 by L Hopital, there is a ǫ > so x that x 2 log(1 x) 3x for x ǫ. 2 Therefore, for sufficietly large k, 1 2 P(X 1 > k) log[1 P(X 1 > k)] 3 2 P(X 1 > k). The required statemet ow follows from the compariso theorem for series. (d) Coclude that ( ) P {X k k for all k } = 1 i case (i) ad = i case (ii). Solutio: I case (i), this follows from ( ) P {X k k for all k } P(X k k for all k m) for ay m. I case (ii), it follows from ( ) P {X k k for all k } P(X k k for all k ). Note that by applyig this to the radom variables X /ǫ, the case (i) statemet ca be stregtheed to ( ) P {X k ǫk for all k } = 1 Bycosiderig asequece ofǫ stedig to, itcabefurtherstregtheed to P( ǫ > 1 such that k,x k ǫk) = 1. (e) Use part (d) to show that X coverges to a.s. i case (i) but ot i case (ii). Solutio: I case (i), this ow follows from part (d) ad the defiitio of the limit: for every ω { ǫ > 1 such that k,x k ǫk}, X (ω)/. Case (ii) is similar.
H 2. LetU beuiformo[,1],addefieradomvariablesx 1,X 2,... by writig the decimal expasio of U as U =.X 1 X 2 X 3. (a) Show that X 1,X 2,X 3 are idepedet. Solutio: For k =,1,...,9, ( k P(X 1 = k) = P 1 < U < k +1 ) 1 Similarly, = 1 1. P(X 1 = k,x 2 = l,x 3 = m) = 1 1 3. (b) Let P be the proportio of 3 s i the first decimal digits of U. Usig the fact that the full sequece X 1,X 2,... is i.i.d., show that P 1 a.s. 1 Solutio: Let Y i be the idicator of the evet {X i = 3}. The P = 1 Y i. Therefore, this follows from the SLLN. (c) If we take the probability space to be Ω = [,1] with the usual assigmet of probabilities ad U(ω) = ω, is it true that P 1 for 1 every ω Ω? Explai. Solutio: No, e.g., ω =.5. (d) Let Q be the proportio of 3 s i the first decimal digits of U that are followed immediately by a 7. Show that Q 1 a.s. 1 (Suggestio: cosider separately the eve k s for which X k = 3,X k+1 = 7 ad the odd k s for which X k = 3,X k+1 = 7.) Solutio: Now let Y i be the idicator of the evet {X i = 3,X i+1 = 7}. The Y i s are o loger idepedet, but the sequeces Y 1,Y 3,... ad Y 2,Y 4,... are each i.i.d. Therefore, by the SLLN, each coverges to 1 1 1 1 i= Q 2 = 1 1 2 Y 2i+1 i=1 ad 1 a.s. It follows that Y 2i+1 + 1 2 i= i=1 i=1 Y 2i Y 2i 1 1 The same argumet works for Q 2+1. You do t eed to show it, but the same argumet ca be used to showthatforayfiiteblockofdigits(say238 47), thatblockoccurs a.s. 5
6 with limitig frequecy 1 1 a.s., where is the legth of the block. A umber i [,1] is called ormal to the base 1 if it has this property (for all fiite blocks). A example of a ormal umber is obtaied by listig the positive itegers i order:.12345678911112131415161718. (e) Show that the set of ormal umbers to the base 1 i [,1] has probability 1. (This is kow as the Borel Law of Normal Numbers.) Solutio: For each fiite block B, let A B = {ω : B does ot occur with the right limitig frequecy i ω}. There are coutably may such blocks, ad P(A B ) = for each B, so ( ) P A B =. B Ay ω / B A B is ormal. Of course, the same is true for ay base b = 1,2,3,... A umber is called completely ormal if it is ormal to every base. (f) Show that the set of completely ormal umbers i [,1] has probability 1. Solutio: The argumet is the same as that for part (e), sice there are coutably may bases. K 2. Cosider a sequece of idepedet trials, each of which has three possible outcomes, A, B, C, with respective probabilities p, q, r (p+q +r = 1). Fid the probability of the evet D that a A ru of legth m occurs before a B ru of legth. Solutio: Let u = P(D X 1 = A), v = P(D X 1 = B), w = P(D X 1 = C) = P(D). The u = = P(D X 1 = A,...,X k 1 = A,X k = B)p k 2 q k=2 + P(D X 1 = A,...,X k 1 = A,X k = C)p k 2 r k=2 m vp k 2 q + k=2 k=m+1 p k 2 q + = qv+rw q +r (1 pm 1 )+p m 1. m wp k 2 r + k=2 k=m+1 p k 2 r
7 Similarly, Solvig gives P(D) = w = v = pu+rw p+r (1 q 1 ). (q +r)p m (1 q ) (q +r)p m +(p+r)q (p+q)p m q. Recall that a Poisso process with parameter λ is a radom collectio of poits o [, ) whose distributio is determied by the followig equivalet properties: (A) If T 1,T 2,... are the successive spacigs betwee poits, the T 1,T 2,... are i.i.d. with the expoetial distributio with parameter λ. (B)IfN(t)istheumber ofpoitsi[,t], thefort 1 < t 2 <, the radom variables N(t 1 ),N(t 2 ) N(t 1 ),N(t 3 ) N(t 2 ),... are idepedet Poisso radom variables with parameters λt 1,λ(t 2 t 1 ),λ(t 3 t 2 ),... I class, we checked part of the equivalece: (i) If (B) holds, the T 1 is Expoetial (λ), ad (ii) If (A) holds, the N(t) is Poisso (λ). I the ext two problems, you will check aother case of the equivalece. K 3. Suppose (B) holds. (a) Write the evet {T 1 > s,t 1 +T 2 > s+t} i terms of the radom variables N(s) ad N(s+t), ad use this to compute its probability. Solutio: P(T 1 > s,t 1 +T 2 > s+t) = P(N(s) =,N(s+t) 1) = e λ(s+t) (1+λt). (b) Write P(T 1 > s,t 1 +T 2 > s+t) i terms of the joit desity of T 1 ad T 2. Solutio: Lettig f be the joit desity, P(T 1 > s,t 1 +T 2 > s+t) = s f(u, v)dudv t s+t v s f(u, v)dudv. (c) Use the fact that the aswers to parts (a) ad (b) are equal to show that T 1 ad T 2 are idepedet Expoetial (λ). Solutio: Equatig the above expressios ad differetiatig with respect to s gives λe λ(s+t) (1+λt) = t f(s+t v,v)dv+ t f(s,v)dv.
8 Differetiatig this idetity with respect to t gives (where f 1 is the partial derivative of f with respect to the first variable) t f 1 (w v,v)dv = λ 3 te λw, where w = s+t. Differetiatig with respect to t gives i.e. Itegratig gives f 1 (w t,t) = λ 3 e λw, f 1 (s,t) = λ 3 e λ(s+t). f(s,t) = λ 2 e λ(s+t). K 4. Suppose (A) holds. (a) Write the evet {N(s) = k,n(s+t) N(s) = l} i terms of the radom variables T 1,T 2,... Solutio: Lettig S = T 1 + +T, P(N(s) = k,n(s+t) N(s) = l) = P(S k < s < S k+1,s k+l < s+t < S k+l+1 ). (b) Use the fact that the sum of k idepedet Expoetial (λ) distributed radom variables is Gamma (k,λ) to show that N(s) ad N(s+t) N(s) are idepedet Poisso distributed radom variables with parameters λs ad λt respectively. Solutio: Coditioig o the values of S k,t k+1,s k+l S k+1, ad lettigf k (x)bethegamma(k,λ)desity, givesthefollowigexpressio for the above probability: (WLOG, assume l 1) f k (x)f 1 (y)f l 1 (z)e λ(s+t x y z) dzdydx, A where A = {x < s < x+y,x+y +z < s+t}. The itegrad is e λ(s+t) λ k+l (k 1)!(l 2)! xk 1 z l 2. Itegratig o < z < s+t x y gives the followig expressio for the itegral: e λ(s+t) λ k+l x k 1 (s+t x y) l 1 dydx. (k 1)!(l 1)! x<s<x+y<s+t Itegratig s x < y < s+t x gives e λ(s+t) λ k+l (k 1)!(l 1)! s x k 1tl l dx.
9 Itegratig o < x < s gives as required. e λ(s+t)λk+l k!l! tl s k