Independent random variables

Transcription

1 Will Monroe July 4, 017 with materials by Mehran Sahami and Chris Piech Independent random variables

2 Announcements: Midterm Tomorrow! Tuesday, July 5, 7:00-9:00pm Building (main quad, Geology Corner) One page of notes (front & back) No books/computers/calculators

3 Review: Joint distributions A joint distribution combines multiple random variables. Its PDF or PMF gives the probability or relative likelihood of both random variables taking on specific values. p X,Y (a, b)=p ( X =a,y =b)

4 Review: Joint PMF A joint probability mass function gives the probability of more than one discrete random variable each taking on a specific value (an AND of the + values). p X,Y (a, b)=p ( X =a,y =b) Y X

5 Review: Joint PDF A joint probability density function gives the relative likelihood of more than one continuous random variable each taking on a specific value. P (a1 < X a, b1 <Y b )= a b dx dy f X,Y ( x, y ) a1 b1

6 Review: Joint CDF F X, Y (x, y)=p( X x, Y y ) to 1 as x +, y +, y to 0 as x -, y -, x plot by Academo

7 Probabilities from joint CDFs P (a1 < X a, b1 <Y b )=F X,Y (a, b ) F X,Y (a1, b ) F X,Y (a, b1 ) + F X,Y (a1, b1 ) b b1 a1 a

8 Review: Marginalization Marginal probabilities give the distribution of a subset of the variables (often, just one) of a joint distribution. Sum/integrate over the variables you don t care about. p X (a)= p X,Y (a, y) y f X (a)= dy f X,Y (a, y)

9 Review: Non-negative RV expectation lemma You can integrate y times the PMF, or you can integrate 1 minus the CDF! E [Y ]= dy P(Y > y) 0 = dy (1 F Y ( y)) 0 y FY ( y )

10 Non-negative RV expectation lemma: Rearranging terms E [ X ]=0 P ( X=0)+1 P( X =1)+ P( X =)+3 P( X =3)+ =0 P( X =0)+1 P( X =1)+ P( X =)+3 P ( X =3)+ +1 P( X1 =1)+ P( X =)+3 P ( X =3)+ +1 P( X =1)+ P( X =)+3 P ( X =3) =0 P( X =0)+1 P( X =1)+ P( X =)+3 P ( X =3)+ +1 P( X =1)+ P( X =)+3 P ( X =3)+ +1 P( X =1)+ P( X =)+3 P ( X =3)+ =0 P( X =0)+1 P( X 1) +1 P( X =1)+ P( X ) +1 P( X =1)+ P( X =)+3 P ( X 3)+ = P( X i) [Addendum] i=1

11 Non-negative RV expectation lemma: Graphically p_x(x) E [ X ]= P(X 4) P(X 3) P(X ) P(X 1) 0 0 P(X=0) 1 P(X=1) P(X=) 3 P(X=3) x [Addendum] 4 P(X=4)

12 Review: Multinomial random variable An multinomial random variable records the number of times each outcome occurs, when an experiment with multiple outcomes (e.g. die roll) is run multiple times. X 1,, X m MN (n, p1, p,, p m) vector! P ( X 1 =c 1, X =c,, X m=c m) c c c n = p1 p p m c 1, c,, c m ( ) 1 m

13 A question from last class p X,Y (a, b)=p ( X =a,y =b) Y X Are X and Y independent? P ( X =0, Y =0)=P ( X =0) P (Y =0)

14 Independence of discrete random variables Two random variables are independent if knowing the value of one tells you nothing about the value of the other (for all values!). X Y iff x, y : P ( X = x, Y = y)=p ( X =x) P (Y = y) - or - p X,Y (x, y )= p X ( x) py ( y )

15 Coin flips m flips n flips X: number of heads Y: number of heads in first n flips in next m flips x n x m y m y n P( X =x, Y = y)= p (1 p) p (1 p) x y () () =P( X =x) P(Y = y) X Y

16 Coin flips n flips X: number of heads in first n flips m flips Z: total number of heads in n + m flips Z=0 X =0 X Z

17 Web server hits Your web server gets N requests in a day. N ~ Poi(λ). Each request comes independently from human (prob. p) or bot (1 p). X: # requests from humans in day Y: # requests from bots in day Knowing N: X Bin ( N, p) Y Bin ( N,1 p) i j = i+ j p (1 p) i ( ) =e λ i+ j λ (i+ j)! P( X =i,y = j)=p ( X =i,y = j N =i+ j)p ( N =i+ j) + P ( X =i,y = j N i+ j) P( N i+ j)

18 Web server hits i+ j i j λ λ i+ j P( X =i,y = j)= p (1 p) e (i+ j)! i i j (i+ j)! i j λ λ λ = p (1 p) e i! j! (i+ j)! ( ) i λ =e i j p λ (1 p) λ i! j! j i λ p (λ p) λ (1 p) [λ (1 p)] =e e i! j! X Poi (λ p) Y Poi (λ (1 p)) =P ( X =i) P(Y = j) X Y j

19 Independence of continuous random variables Two random variables are independent if knowing the value of one tells you nothing about the value of the other (for all values!). X Y iff x, y : f X,Y ( x, y )=f X ( x) f Y ( y) - or - f X,Y (x, y )=g(x)h( y ) - or - F X,Y (x, y)=f X (x) F Y ( y)

20 Density functions and independence g(x) h(y) 3 x y f X,Y (x, y)=6 e e X Y : yes! for 0<{x, y }< g(x) h(y) f X,Y (x, y)=4 x y for 0<{x, y }<1 X Y : yes! f X,Y (x, y)=4 x y for 0< x <1 y <1 X Y : no! Room: CS109SUMMER17

21 The Joy of Meetings people set up a meeting for 1pm. Each arrives independently, uniformly between 1:00 and 1:30. X ~ Uni(0, 30): mins after 1 for person 1 Y ~ Uni(0, 30): mins after 1 for person P(first to arrive waits > 10 minutes for the other) =? P( X +10<Y )+ P(Y +10< X )= P( X +10<Y ) (symmetry) = x, y : x+10< y dx dy f X,Y ( x, y)

22 The Joy of Meetings (symmetry) P( X +10<Y )+ P(Y +10< X )= P( X +10<Y ) = dx dy f X,Y ( x, y) dx dy f X ( x) f Y ( y) x, y : x+10< y = (independence) x, y : x+10< y 30 y 10 = dy y=10 30 x=0 1 dx 30 ( ) = dy ( y 10) 30 y=10 y = 10 y 30 [ 30 ] [( )] = = 9 30 y=10 )(

23 Independence of continuous random variables Two random variables are independent if knowing the value of one tells you nothing about the value of the other (for all values!). X Y iff x, y : f X,Y ( x, y )=f X ( x) f Y ( y) - or - f X,Y (x, y )=g(x)h( y ) - or - F X,Y (x, y)=f X (x) F Y ( y)

24 Setting records Let X₁, X₂, be a sequence of independent and identically distributed (I.I.D.) continuous random variables. record value : an Xₙ that beats all previous Xᵢ Xₙ = max(x₁,, Xₙ) Aᵢ: event that Xᵢ is a record value A n+1 A n? Room: CS109SUMMER17

25 Independence is symmetric X Y Y X E F F E Let X₁, X₂, be a sequence of independent and identically distributed (I.I.D.) continuous random variables. record value : an Xₙ that beats all previous Xᵢ Xₙ = max(x₁,, Xₙ) Aᵢ: event that Xᵢ is a record value A n+1 A n? A n A n+1? P( A n A n+1 ) 1 1 = n n+1 yes! =P ( A n ) P( A n+1) Room: CS109SUMMER17

26 Break time!

27 Sum of independent binomials m flips n flips X: number of heads Y: number of heads in first n flips in next m flips X Bin (n, p) Y Bin (m, p) X +Y Bin (n+m, p) More generally: N X i Bin (ni, p) all X i independent ( N X i Bin ni, p i=1 i=1 )

28 Sum of independent Poissons λ₁ chips/cookie X: number of chips in first cookie λ₂ chips/cookie Y: number of chips in second cookie X Poi (λ 1 ) Y Poi (λ ) X +Y Poi(λ 1 +λ ) More generally: N X i Poi (λ i ) all X i independent N ( ) X i Poi λ i i=1 i=1

29 Convolution A convolution is the distribution of the sum of two independent random variables. f X +Y (a)= dy f X (a y) f Y ( y)

30 Dance Dance Convolution X, Y: independent discrete random variables (law of total probability) P( X +Y =a)= P( X +Y =a, Y = y) y = P( X=a y) P(Y = y ) y X, Y: independent continuous random variables f X+Y (a)= dy f X (a y) f Y ( y)

31 Blurring a photo images: Stig Nygaard (left), Daniel Paxton (right)

32 Sum of independent uniforms X Uni (0,1) 0 1 Y Uni (0,1) f X+Y (a)= dy f X (a y) f Y ( y) 1 = dy f X (a y) f Y ( y) 1 0 Case 1: if 0 a 1, then we need 0 y a (for a y to be in [0, 1]) Case : if 1 a, then we need a 1 y 1 = { a 0 dy 1=a 1 a 1 dy 1= a 0 0 a 1 1 a otherwise

33 Sum of independent normals X N (μ 1,σ 1 ) Y N (μ, σ ) X +Y N (μ 1+μ, σ 1 +σ ) More generally: X i N (μ i, σ i ) all X i independent N ( N N X i N μ i, σ i i=1 i=1 i=1 )

34 Virus infections 150 computers in a dorm: 50 Macs (each independently infected with probability 0.1) 100 PCs (each independently infected with probability 0.4) What is P( 40 machines infected)? M: # infected Macs M Bin (50, 0.1) X N (5, 4.5) P: # infected PCs P Bin (100, 0.4) Y N (40, 4) P( M + P 40) P( X +Y 39.5) W =X +Y N (5+40, 4.5+4)=N (45, 8.5) W P(W 39.5)=P 1 Φ( 1.03) ( )