Law of Large Numbers

Transcription

1 Toss a co tmes. Law of Large Numbers Suppose 0 f f th th toss came up H toss came up T s are Beroull radom varables wth p ½ ad E( ) ½. The proporto of heads s. Itutvely approaches ½ as. week 2

2 Markov s Iequalty If s a o-egatve radom varable wth E() < ad a >0 the, P ( a) E( ) a week 2 2

3 Chebyshev s Iequalty For a radom varable wth E() < ad V() <, for ay a >0 Proof: P ( E( ) a) V a ( ) 2 week 2 3

4 week 2 4 Back to the Law of Large Numbers Iterested sequece of radom varables, 2, 3, such that the radom varables are depedet ad detcally dstrbuted (..d). Let Suppose E( ) μ, V( ) σ 2, the ad Itutvely, as, so ( ) ( ) μ E E E ( ) ( ) V V V 2 2 σ ( ) 0 V ( ) μ E

5 Formally, the Weak Law of Large Numbers (WLLN) states the followg: Suppose, 2, 3, are..d wth E( ) μ <, V( ) σ 2 <, the for ay postve umber a as. Ths s called Covergece Probablty. Proof: ( a) 0 P μ week 2 5

6 Example Flp a co 0,000 tmes. Let f 0 f th th toss came up H toss came up T E( ) ½ ad V( ) ¼. Take a 0.0, the by Chebyshev s Iequalty P ,000 ( ) ( ) Chebyshev Iequalty gves a very weak upper boud. Chebyshev Iequalty works regardless of the dstrbuto of the s. week 2 6

7 Strog Law of Large Number Suppose, 2, 3, are..d wth E( ) μ <, the coverges to μ as wth probablty. That s P lm 2 ( + + L + ) μ Ths s called covergece almost surely. week 2 7

8 Cotuty Theorem for MGFs Let be a radom varable such that for some t 0 > 0 we have m (t) < for t ( t,t 0 0 ). Further, f, 2, s a sequece of radom varables wth () t m < ( ) ( ) ad lm m t m t for all the { } coverges dstrbuto to. t ( t,t 0 0 ) Ths theorem ca also be stated as follows: Let F be a sequece of cdfs wth correspodg mgf m. Let F be a cdf wth mgf m. If m (t) m(t) for all t a ope terval cotag zero, the F (x) F(x) at all cotuty pots of F. Example: Posso dstrbuto ca be approxmated by a Normal dstrbuto for large λ. week 2 8

9 Example to llustrate the Cotuty Theorem Let λ, λ 2, be a creasg sequece wth λ as ad let { } be a sequece of Posso radom varables wth the correspodg parameters. We kow that E( ) λ V( ). ( ) ( ) E λ Let Z the we have that E(Z ) 0, V(Z ). V λ We ca show that the mgf of Z s the mgf of a Stadard Normal radom varable. We say that Z covergece dstrbuto to Z ~ N(0,). week 2 9

10 Example Suppose s Posso(900) radom varable. Fd P( > 950). week 2 0

11 Cetral Lmt Theorem The cetral lmt theorem s cocered wth the lmtg property of sums of radom varables. If, 2, s a sequece of..d radom varables wth mea μ ad varace σ 2 ad, S the by the WLLN we have that μ probablty. The CLT cocered ot just wth the fact of covergece but how S / fluctuates aroud μ. Note that E(S ) μ ad V(S ) σ 2. The stadardzed verso of S s S Z S μ σ ad we have that E(Z ) 0, V(Z ). week 2

12 The Cetral Lmt Theorem Let, 2, be a sequece of..d radom varables wth E( ) μ < ad Var( ) σ 2 <. Suppose the commo dstrbuto fucto F (x) ad the commo momet geeratg fucto m (t) are defed a eghborhood of 0. Let The, S μ lm P σ S x Φ for - < x < where Ф(x) s the cdf for the stadard ormal dstrbuto. Ths s equvalet to sayg that S μ Z coverges dstrbuto to Z ~ N(0,). σ Also, lm P σ μ x Φ ( x).e. μ Z coverges dstrbuto to Z ~ N(0,). σ ( x) week 2 2

13 Example Suppose, 2, are..d radom varables ad each has the Posso(3) dstrbuto. So E( ) V( ) 3. ( ) ( ) The CLT says that P x 3 Φ x as. L week 2 3

14 Examples A very commo applcato of the CLT s the Normal approxmato to the Bomal dstrbuto. Suppose, 2, are..d radom varables ad each has the Beroull(p) dstrbuto. So E( ) p ad V( ) p(-p). ( ( )) ( ) The CLT says that P + + p x p p x as. + Φ L Let Y + + the Y has a Bomal(, p) dstrbuto. So for large, P ( Y y) P Y p p y p Φ y p ( p) p( p) p( p) Suppose we flp a based co 000 tmes ad the probablty of heads o ay oe toss s 0.6. Fd the probablty of gettg at least 550 heads. Suppose we toss a co 00 tmes ad observed 60 heads. Is the co far? week 2 4