Properties of Joints Chris Piech CS109, Stanford University

Size: px

Start display at page:

Download "Properties of Joints Chris Piech CS109, Stanford University"

Russell Lawrence
5 years ago
Views:

1 Properties of Joits Chris Piech CS09, Staford Uiversity

3 Titaic Probability

5 7% of passegers were from the Ottoma Empire

7 Biometric Keystroes

8 Altruism? Scores for a stadardized test that studets i Polad are required to pass before movig o i school See if you ca guess the miimum score to pass the test. 8

Joit Radom Variables Use a joit table, desity fuctio or CDF to solve probability questio Thi about coditioal probabilities with joit variables

9 Joit Radom Variables Use a joit table, desity fuctio or CDF to solve probability questio Thi about coditioal probabilities with joit variables (which might be cotiuous) Use ad fid expectatio of radom variables Use ad fid idepedece of radom variables What happes whe you add radom variables?

10 Zero Sum Games What is the probability that the Warriors wi? How do you model zero sum games?

11 Motivatig Idea: Zero Sum Games How it wors: Each team has a ELO score S, calculated based o their past performace. Each game, the team has ability A ~ N(S, ) The team with the higher sampled ability wis. Arpad Elo A B N (555, ) A W N (797, ) p ability P (Warriors wi) P (A W >A B )

12 Motivatig Idea: Zero Sum Games A W N (797, ) A B N (555, ) P (Warriors wi) P (A W >A B ) I class we solved this by samplig. But that is a bit of a cheat ad is computatioally expesive. Sums (or subtractios) of radom variables show up all the time. But we have o explicit tools for dealig with them! Challege: try ad come up with the way to solve this by the ed of class

13 You may have ot see this ZZ f x,y dy dx Z p y 2 Z f x,y dy dx x 2 +y 2 < p y 2 -

14 Four Prototypical Trajectories Expectatio of Multiple RVs

15 Joit Expectatio E[] x xp(x) Expectatio over a joit is t icely defied because it is ot clear how to compose the multiple variables: Add them? Multiply them? Lemma: For a fuctio g(,y) we ca calculate the expectatio of that fuctio: E[g(, Y )] x,y g(x, y)p(x, y) Recall, this also holds for sigle radom variables: E[g()] x g(x)p(x)

16 E[ + Y] E[] + E[Y] Geeralized: Holds regardless of depedecy betwee i s å å ú û ù ê ë é i i i i E E ] [ Expected Values of Sums Big deal lemma: first stated without proof

17 Septical Chris Wats a Proof! Let g(,y) [ + Y] E[ + Y ]E[g(, Y )] g(x, y)p(x, y) x,y [x + y]p(x, y) x,y What a useful lemma By the defiitio of g(x,y) Brea that sum ito parts! Chage the sum of (x,y) ito separate sums That is the defiitio of margial probability That is the defiitio of expectatio xp(x, y)+ yp(x, y) x,y x,y x p(x, y)+ y p(x, y) x y y x xp(x)+ yp(y) x y E[]+E[Y ]

18 Four Prototypical Trajectories Idepedece ad Radom Variables

19 Idepedet Discrete Variables Two discrete radom variables ad Y are called idepedet if: p( x, y) p ( x) py ( y) for all x, P ( x, Y y) P ( x) P (Y y) Ituitively: owig the value of tells us othig about the distributio of Y (ad vice versa) If two variables are ot idepedet, they are called depedet Similar coceptually to idepedet evets, but we are dealig with multiple variables Keep your evets ad variables distict (ad clear)! y

20 Is Year Idepedet of Luch? For all values of Year, Status: P(Year y, Luch s) P(Year y)p(luch s) Yes!

21 Is Year Idepedet of Luch? For all values of Year, Status: P(Year y, Luch s) P(Year y)p(luch s) No L 0.08

22 Aside: Butterfly Effect

23 Coi Flips Flip coi with probability p of heads Flip coi a total of + m times Let umber of heads i first flips Let Y umber of heads i ext m flips P( x, Y y) æ ç è ad Y are idepedet Let Z umber of total heads i + m flips Are ad Z idepedet? x ö ø p o What if you are told Z 0? x ( - p) -x æ ç è P ( x) P( Y y) mö p y ø y ( - p) m- y

24 Idepedet Cotiuous Variables Two cotiuous radom variables ad Y are called idepedet if: P( a, Y b) P( a) P(Y b) for ay a, b Equivaletly: F f, Y ( a, b) ( a, b), Y F f ( a) F ( a) f ( b) ( b) More geerally, joit desity factors separately: f, Y ( x, y) Y Y for all a, b for all a, b h( x) g( y) where - < x, y <

It is a widely used effect i graphics software, typically to reduce image oise.

25 Is the Blur Distributio Idepedet? I image processig, a Gaussia blur is the result of blurrig a image by a Gaussia fuctio. It is a widely used effect i graphics software, typically to reduce image oise. Gaussia blurrig with StDev 3, is based o a joit probability distributio: Joit PDF f,y (x, y) e x 2 +y Joit CDF F,Y (x, y) x 3 y 3 Used to geerate this weight matrix

26 Pop Quiz (just iddig) Cosider joit desity fuctio of ad Y: -3x -2 y f ( x, y) 6e e for 0 < x y < Cosider joit desity fuctio of ad Y: Are ad Y idepedet? Yes! Let h( x) 2x ad g( y) 2y, so f, ( x, y) h( x) g( y) Now add costrait that: 0 < (x + y) < Are ad Y idepedet? No! o, Y, Are ad Y idepedet? Yes! -3x -2 y Let h( x) 3e ad g( y) 2e, so f, f Y, ( x, y) 4xy for 0 < x, y < ( x, y) h( x) g( y) Caot capture costrait o x + y i factorizatio! Y Y

27 Four Prototypical Trajectories What happes whe you add radom variables?

28 Sum of Idepedet Biomials Let ad Y be idepedet biomials with the same value for p: ~ Bi(, p) ad Y ~ Bi( 2, p) + Y ~ Bi( + 2, p) Ituitio: has trials ad Y has 2 trials o Each trial has same success probability p Defie Z to be + 2 trials, each with success prob. p Z ~ Bi( + 2, p), ad also Z + Y

29 Four Prototypical Trajectories If oly it were always that simple

30 The Isight to Covolutio Imagie a game where each player idepedetly scores betwee 0 ad 00 poits: Let be the amout of poits you score. Let Y be the amout of poits your oppoet scores. Let s say you ow P( x) ad P(Y y). What is the probability of a tie?

31 The Isight to Covolutio Proofs P( + Y )? What is the probability that + Y? Y P( 0, Y ) P(, Y -) P( 2, Y -2) 0 P(, Y 0)

32 The Isight to Covolutio Proofs P( + Y )? What is the probability that + Y? P ( + Y ) 0 P (, Y ) Sice this is the OR or mutually exclusive evets 0 P ( )P (Y ) If the radom variables are idepedet

33 Sum of Idepedet Poissos Recall the Biomial Theorem (a + b) a b 0

34 Let ad Y be idepedet radom variables ~ Poi(l ) ad Y ~ Poi(l 2 ) + Y ~ Poi(l + l 2 ) Proof: (just for referece) Rewrite ( + Y ) as (, Y ) where 0 Notig Biomial theorem: so, + Y ~ Poi(l + l 2 ) å å Y P P Y P Y P 0 0 ) ( ) ( ), ( ) ( å å å e e e e 0 2 ) ( 0 2 ) ( 0 2 )!!(!! )!!( )! (! l l l l l l l l l l l l ( ) e Y P 2 ) (! ) ( 2 l l l l å )!!(! ) ( l l l l Sum of Idepedet Poissos

35 Referece: Sum of Idepedet RVs Let ad Y be idepedet Biomial RVs ~ Bi(, p) ad Y ~ Bi( 2, p) + Y ~ Bi( + 2, p) More geerally, let i ~ Bi( i, p) for i N, the Let ad Y be idepedet Poisso RVs ~ Poi(l ) ad Y ~ Poi(l 2 ) æ ç è å i + Y ~ Poi(l + l 2 ) N i ö ø æ ~ Biç è More geerally, let i ~ Poi(l i ) for i N, the æ ç è N å i i ö ø N å i N æ ö ~ Poiçå li è i ø i, ö p ø

36 Covolutio of Probability Distributios CON We taled about sum of Biomial ad Poisso who s missig from this party? Uiform.

37 Four Prototypical Trajectories Summatio: ot just for the %

38 Dace, Dace Covolutio Let ad Y be idepedet radom variables Probability Desity Fuctio (PDF) of + Y: f + Y ( a) ò f ( a - y- y) f Y ( y) dy ò y- I discrete case, replace with, ad f(y) with p(y) å y

39 Itegratio with Costrait ZZ f x,y dy dx Z p y 2 Z f x,y dy dx x 2 +y 2 < p y 2 -

40 Dace, Dace Covolutio Let ad Y be idepedet radom variables Cumulative Distributio Fuctio (CDF) of + Y: F + Y CDF of ( a) P( + Y a) òò f x+ y a ò F y- ( x) f Y ( a - y) ( y) f Y dx dy ( y) dy ò y- a- y ò f x- ( x) dx PDF of Y f Y ( y) dy ò y- I discrete case, replace with, ad f(y) with p(y) å y

41 Sum of Idepedet Uiforms Let ad Y be idepedet radom variables ~ Ui(0, ) ad Y ~ Ui(0, ) à f(x) for 0 x f(x) For both ad Y

42 Sum of Idepedet Uiforms Let ad Y be idepedet radom variables ~ Ui(0, ) ad Y ~ Ui(0, ) à f(x) for 0 x What is PDF of + Y? Whe a 0.5: f +Y (0.5) f + Y ( a) ò f ( a - y) fy ( y) dy ò f ( a - y 0 y 0 Z y? y? Z Z f (0.5 f (0.5 dy y)dy y)dy f Y (a) + y) dy 2 a

43 Sum of Idepedet Uiforms Let ad Y be idepedet radom variables ~ Ui(0, ) ad Y ~ Ui(0, ) à f(x) for 0 x What is PDF of + Y? Whe a.5: f +Y (.5) f + Y ( a) ò f ( a - y) fy ( y) dy ò f ( a - y 0 y 0 Z y? y? Z 0.5 Z f (.5 f (.5 dy y)dy y)dy f Y (a) + y) dy 2 a

44 Sum of Idepedet Uiforms Let ad Y be idepedet radom variables ~ Ui(0, ) ad Y ~ Ui(0, ) à f(x) for 0 x What is PDF of + Y? Whe a : f +Y () f + Y ( a) ò f ( a - y) fy ( y) dy ò f ( a - y 0 y 0 Z y? y? Z 0 Z 0 f ( f ( dy y)dy y)dy f Y (a) + y) dy 2 a

45 Let ad Y be idepedet radom variables ~ Ui(0, ) ad Y ~ Ui(0, ) à f(x) for 0 x What is PDF of + Y? Whe 0 a ad 0 y a, 0 a y à f (a y) Whe a 2 ad a y, 0 a y à f (a y) Combiig: ò ò ) ( ) ( ) ( ) ( y y Y Y dy y a f dy y f y a f a f a dy a f a y Y ò + 0 ) ( a dy a f a y Y - ò ) ( ï î ï í ì < - + otherwise ) ( a a a a a f Y a 2 f Y (a) + Sum of Idepedet Uiforms

46 Sum of Idepedet Normals Let ad Y be idepedet radom variables ~ N(µ, s 2 ) ad Y ~ N(µ 2, s 22 ) + Y ~ N(µ + µ 2, s 2 + s 22 ) Geerally, have idepedet radom variables i ~ N(µ i, s i2 ) for i, 2,..., : æ ç è å i ö æ çå å i ~ N µ i, s i ø è i i 2 ö ø

47 Say you are worig with the WHO to pla a respose to a the iitial coditios of a virus: Two exposed groups Virus Ifectios P: 50 people, each idepedetly ifected with p 0. P2: 00 people, each idepedetly ifected with p 0.4 Questio: Probability of more tha 40 ifectios? Saity chec: Should we use the Biomial Sum-of-RVs shortcut? A. YES! B. NO! C. Other/oe/more

48 Say you are worig with the WHO to pla a respose to a the iitial coditios of a virus: Two exposed groups Virus Ifectios P: 50 people, each idepedetly ifected with p 0. P2: 00 people, each idepedetly ifected with p 0.4 A # ifected i P A ~ Bi(50, 0.)» ~ N(5, 4.5) B # ifected i P2 B ~ Bi(00, 0.4)» Y ~ N(40, 24) What is P( 40 people ifected)? P(A + B 40)» P( + Y 39.5) + Y W ~ N( , ) P( W ³ 39.5) æw Pç > è ö ø - F( -.03)»

Liear Trasform N(µ, 2 ) Y + 2 Y N(2µ, 4 2 ) Y + 2

49 Liear Trasform N(µ, 2 ) Y + 2 Y N(2µ, 4 2 ) Y N(µ + µ, ) Y N(2µ, 2 2 ) is ot idepedet of

50 Motivatig Idea: Zero Sum Games How it wors: Each team has a ELO score S, calculated based o their past performace. Each game, the team has ability A ~ N(S, ) The team with the higher sampled ability wis. Arpad Elo A B N (555, ) A W N (797, ) p ability P (Warriors wi) P (A W >A B )

Motivatig Idea: Zero Sum Games A B N (555, 200 2 ) P

51 Motivatig Idea: Zero Sum Games A B N (555, ) P (Warriors wi) A W N (797, ) P (A W >A B ) p ability

52 Four Prototypical Trajectories That s all fols!

Econ 325: Introduction to Empirical Economics

Econ 325: Introduction to Empirical Economics Eco 35: Itroductio to Empirical Ecoomics Lecture 3 Discrete Radom Variables ad Probability Distributios Copyright 010 Pearso Educatio, Ic. Publishig as Pretice Hall Ch. 4-1 4.1 Itroductio to Probability