,m = 1,...,n; 2 ; p m (1 p) n m,m = 0,...,n; E[X] = np; n! e λ,n 0; E[X] = λ.

Size: px

Start display at page:

Download ",m = 1,...,n; 2 ; p m (1 p) n m,m = 0,...,n; E[X] = np; n! e λ,n 0; E[X] = λ."

Chloe Nichols
5 years ago
Views:

1 CS70: Lecture 21. Revew: Dstrbutos Revew: Idepedece Varace; Iequaltes; WLLN 1. Revew: Dstrbutos 2. Revew: Idepedece 3. Varace 4. Iequaltes Markov Chebyshev 5. Weak Law of Large Numbers U[1,...,] : Pr[X = m] = 1,m = 1,...,; E[X] = +1 2 ; B(,p) : Pr[X = m] = ( m) p m (1 p) m,m = 0,...,; E[X] = p; G(p) : Pr[X = ] = (1 p) 1 p, = 1,2,...; E[X] = 1 p ; P(λ) : Pr[X = ] = λ! e λ, 0; E[X] = λ. Defto X ad Y are depedet Pr[X = x,y = y] = Pr[X = x]pr[y = y], x,y Pr[X A,Y B] = Pr[X A]Pr[Y B], A,B. Theorem X ad Y are depedet f (X),g(Y ) are depedet f ( ),g( ) E[XY ] = E[X]E[Y ]. Revew: Mutual Idepedece Varace Varace ad Stadard Devato Defto X,Y,Z,... are mutually depedet Pr[X = x,y = y,z = z,...] = Pr[X = x]pr[y = y]pr[z = z], x,y,z,... Pr[X A,Y B,Z C,...] = Pr[X A]Pr[Y B]Pr[Z C], A,B,C,... Theorem X,Y,Z,V,W,U... are mutually depedet f (X,Y ),g(z,v,w ),h(u,...),... are mutually depedet E[XYZ ] = E[X]E[Y ]E[Z ] The varace measures the devato from the mea value. Defto: The varace of X s σ 2 (X) := var[x] = E[(X E[X]) 2 ]. σ(x) s called the stadard devato of X. Fact: var[x] = E[X 2 ] E[X] 2. Ideed: var(x) = E[(X E[X]) 2 ] = E[X 2 2XE[X] + E[X] 2 ) = E[X 2 ] 2E[X]E[X] + E[X] 2, by learty = E[X 2 ] E[X] 2.

2 A smple example Ths example llustrates the term stadard devato. Cosder the radom varable X such that { µ σ, w.p. 1/2 X = µ + σ, w.p. 1/2. The, E[X] = µ ad (X E[X]) 2 = σ 2. Hece, var(x) = σ 2 ad σ(x) = σ. Varace of geometrc dstrbuto. X s a geometrcally dstrbuted RV wth parameter p. Thus, Pr[X = ] = (1 p) 1 p for 1. Recall E[X] = 1/p. E[X 2 ] = p + 4p(1 p) + 9p(1 p) (1 p)e[x 2 ] = [p(1 p) + 4p(1 p) ] pe[x 2 ] = p + 3p(1 p) + 5p(1 p) = 2(p + 2p(1 p) + 3p(1 p) 2 +..) E[X]! pe[x 2 ] = 2E[X] 1 (p + p(1 p) + p(1 p) ) = 2( 1 p ) 1 = 2 p p = E[X 2 ] = (2 p)/p 2 ad var[x] = E[X 2 ] E[X] 2 = 2 p σ(x) = 1 p p p 2 1 p 2 = 1 p p 2. E[X] whe p s small(sh). Dstrbuto. Example Cosder X wth The Also, { 1, w. p X = 99, w. p E[X] = = 0. E[X 2 ] = (99) Var(X) 100 = σ(x) 10. E( X ) = = Thus, σ(x) E[ X E[X] ]! Exercse: How bg ca you make σ(x) E[ X E[X] ]? Fxed pots. Number of fxed pots a radom permutato of tems. Number of studet that get homework back. X = X 1 + X 2 + X where X s dcator varable for th studet gettg hw back. E(X 2 ) = E(X ) + j E(X X j ). = 1 + ()( 1) 1 ( 1) = = 2. E(X 2 ) = 1 Pr[X = 1] + 0 Pr[X = 0] = 1 E(X X j ) = 1 Pr[X = 1 X j = 1] + 0 Pr[ aythg else ] = 1 1 ( 2)!! = 1 ( 1) Var(X) = E(X 2 ) (E(X)) 2 = 2 1 = 1. Uform Assume that Pr[X = ] = 1/ for {1,...,}. The Also, Ths gves E[X] = E[X 2 ] = = var(x) = Varace: bomal. Too hard! = 1 E[X 2 ] = Pr[X = ] = 1 ( + 1) 2 2 Pr[X = ] = 1 = , as you ca verfy ( + 1)2 = =0 2 ( ) p (1 p). = Really???!!##... Ok.. fe. Let s do somethg else. Maybe ot much easer...but there s a payoff.

Propertes of varace. 1. Var(cX) = c 2 Var(X), where c s a costat. Scales by c 2. 2. Var(X + c) = Var(X), where c s a costat. Shfts ceter.

3 Propertes of varace. 1. Var(cX) = c 2 Var(X), where c s a costat. Scales by c Var(X + c) = Var(X), where c s a costat. Shfts ceter. Var(cX) = E((cX) 2 ) (E(cX)) 2 = c 2 E(X 2 ) c 2 (E(X)) 2 = c 2 (E(X 2 ) E(X) 2 ) = c 2 Var(X) Var(X + c) = E((X + c E(X + c)) 2 ) = E((X + c E(X) c) 2 ) = E((X E(X)) 2 ) = Var(X) Varace of sum of two depedet radom varables If X ad Y are depedet, the Var(X +Y) = Var(X) +Var(Y ). Sce shftg the radom varables does ot chage ther varace, let us subtract ther meas. That s, we assume that E(X) = 0 ad E(Y ) = 0. The, by depedece, Hece, E(XY ) = E(X)E(Y ) = 0. var(x + Y ) = E((X + Y ) 2 ) = E(X 2 + 2XY + Y 2 ) = E(X 2 ) + 2E(XY ) + E(Y 2 ) = E(X 2 ) + E(Y 2 ) = var(x) + var(y ). Varace of sum of depedet radom varables If X,Y,Z,... are parwse depedet, the var(x + Y + Z + ) = var(x) + var(y ) + var(z ) +. Sce shftg the radom varables does ot chage ther varace, let us subtract ther meas. That s, we assume that E[X] = E[Y ] = = 0. The, by depedece, Hece, E[XY ] = E[X]E[Y ] = 0. Also, E[XZ ] = E[YZ ] = = 0. var(x + Y + Z + ) = E((X + Y + Z + ) 2 ) = E(X 2 + Y 2 + Z XY + 2XZ + 2YZ + ) = E(X 2 ) + E(Y 2 ) + E(Z 2 ) = var(x) + var(y ) + var(z ) +. Varace of Bomal Dstrbuto. Flp co wth heads probablty p. X- how may heads? Iequaltes: A Overvew Adrey Markov { 1 f th flp s heads X = 0 otherwse E(X 2 ) = 1 2 p (1 p) = p. Var(X ) = p (E(X)) 2 = p p 2 = p(1 p). p = 0 = Var(X ) = 0 p = 1 = Var(X ) = 0 X = X 1 + X X. X ad X j are depedet: Pr[X = 1 X j = 1] = Pr[X = 1]. Dstrbuto p p µ Markov p a Pr[X >a] Chebyshev p µ Pr[ X µ > ] Adrey Markov s best kow for hs work o stochastc processes. A prmary subject of hs research later became kow as Markov chas ad Markov processes. Pafuty Chebyshev was oe of hs teachers. Markov was a athest. I 1912 he protested Leo Tolstoy s excommucato from the Russa Orthodox Church by requestg hs ow excommucato. The Church compled wth hs request. Var(X) = Var(X 1 + X ) = p(1 p).

Markov s equalty The equalty s amed after Adrey Markov, although t appeared earler the work of Pafuty Chebyshev. It should be (ad s sometmes) called Chebyshev s frst equalty.

Ideed, f X < a, the equalty reads 0 f (X)/f (a), whch holds sce f ( ) 0. Also, f X a, t reads 1 f (X)/f (a), whch holds sce f ( ) s odecreasg.

4 Markov s equalty The equalty s amed after Adrey Markov, although t appeared earler the work of Pafuty Chebyshev. It should be (ad s sometmes) called Chebyshev s frst equalty. Theorem Markov s Iequalty Assume f : R [0, ) s odecreasg. The, Observe that Pr[X a] E[f (X)], for all a such that f (a) > 0. f (a) 1{X a} f (X) f (a). Ideed, f X < a, the equalty reads 0 f (X)/f (a), whch holds sce f ( ) 0. Also, f X a, t reads 1 f (X)/f (a), whch holds sce f ( ) s odecreasg. Takg the expectato yelds the equalty, because expectato s mootoe. A pcture Markov Iequalty Example: G(p) Let X = G(p). Recall that E[X] = 1 p ad E[X 2 ] = 2 p p 2. Choosg f (x) = x, we get Pr[X a] E[X] a = 1 ap. Choosg f (x) = x 2, we get a 2 = 2 p p 2 Markov Iequalty Example: P(λ) Let X = P(λ). Recall that E[X] = λ ad E[X 2 ] = λ + λ 2. Choosg f (x) = x, we get Pr[X a] E[X] = λ a a. Choosg f (x) = x 2, we get a 2 = λ + λ 2 Chebyshev s Iequalty Ths s Pafuty s equalty: Pr[ X E[X] > a] var[x] a 2, for all a > 0. Let Y = X E[X] ad f (y) = y 2. The, Pr[Y a] E[f (Y )] f (a) = var[x] Ths result cofrms that the varace measures the devatos from the mea. Chebyshev ad Posso Let X = P(λ). The, E[X] = λ ad var[x] = λ. Thus, Pr[ X λ ] var[x] 2 = λ 2.

5 Chebyshev ad Posso (cotued) Let X = P(λ). The, E[X] = λ ad var[x] = λ. By Markov s equalty, a 2 = λ + λ 2 Also, f a > λ, the X a X λ a λ > 0 X λ a λ. Hece, for a > λ, Pr[X a] Pr[ X λ a λ] λ. (a λ ) 2 Weak Law of Large Numbers Theorem Weak Law of Large Numbers Let X 1,X 2,... be parwse depedet wth the same dstrbuto ad mea µ. The, for all ε > 0, Pr[ X X µ ε] 0, as. Let Y = X 1+ +X. The Pr[ Y µ ε] var[y ] ε 2 = var[x X ] 2 ε 2 = var[x 1] 2 ε 2 = var[x 1] ε 2 0, as. Fracto of H s Here s a classcal applcato of Chebyshev s equalty. How lkely s t that the fracto of H s dffers from 50%? Let X m = 1 f the m-th flp of a far co s H ad X m = 0 otherwse. Defe Y = X X, for 1. We wat to estmate By Chebyshev, Now, Summary Pr[ Y ] = Pr[Y 0.4 or Y 0.6]. Pr[ Y ] var[y ] (0.1) 2 = 100var[Y ]. var[y ] = 1 2 (var[x 1] + + var[x ]) = 1 var[x 1] = 1 4. Varace; Iequaltes; WLLN Varace: var[x] := E[(X E[X]) 2 ] = E[X 2 ] E[X] 2 Fact: var[ax + b]a 2 var[x] Sum: X,Y,Z mutually d. var[x + Y + Z ] = Markov: Pr[X a] E[f (X)]/f (a) where... Chebyshev: Pr[ X E[X] a] var[x]/a 2 WLLN: X m..d. X 1+ +X E[X] Fracto of H s Y = X X, for 1. Pr[ Y ] 25. For = 1,000, we fd that ths probablty s less tha 2.5%. As, ths probablty goes to zero. I fact, for ay ε > 0, as, the probablty that the fracto of Hs s wth ε > 0 of 50% approaches 1: Pr[ Y 0.5 ε] 1. Ths s a example of the Law of Large Numbers. We look at a geeral case ext.

Discrete Mathematics and Probability Theory Fall 2016 Seshia and Walrand DIS 10b

Discrete Mathematics and Probability Theory Fall 2016 Seshia and Walrand DIS 10b CS 70 Dscrete Mathematcs ad Probablty Theory Fall 206 Sesha ad Walrad DIS 0b. Wll I Get My Package? Seaky delvery guy of some compay s out delverg packages to customers. Not oly does he had a radom package