Continuous Joint Distributions Chris Piech CS109, Stanford University

Transcription

1 Continuous Joint Distriutions Chris Piech CS09, Stnford University

2 CS09 Flow Tody Discrete Joint Distriutions: Generl Cse Multinomil: A prmetric Discrete Joint Cont. Joint Distriutions: Generl Cse

3 Lerning Gols. Know how to use multinomil. Be le to clculte lrge yes prolems using computer 3. Use Joint CDF

4 Motivting Exmples

5 Four Prototypicl Trjectories Recll logs

6 Log Review e y = x log(x) =y

7 Log Identities log( ) = log() + log() log(/) = log() log() log( n )=n log()

8 Products ecome Sums! log( ) = log() + log() log( Y i i )= X i log( i ) * Spoiler lert: This is importnt ecuse the product of mny smll numers gets hrd for computers to represent.

9 Four Prototypicl Trjectories Where we left off

10 Joint Proility Tle Wlk Bike Scooter Drive Mrginl Yer Freshmn Sophomore Junior Senior Mrginl Mode

11 The Multinomil Multinomil distriution n independent trils of experiment performed Ech tril results in one of m outcomes, with respective proilities: p, p,, p m where X i = numer of trils with outcome i m å i = p i = P c c ( X = c, X = c,..., X m = cm) = ç p p c, c,..., cm where æ n ö ç è ø Joint distriution m åci = i= n nd Multinomil # wys of ordering the successes æ ç èc, c n,..., c ö ø = c! c n!! c m m!... p c m m Proilities of ech ordering re equl nd mutully exclusive

12 The Multinomil Multinomil distriution n independent trils of experiment performed Ech tril results in one of m outcomes, with respective proilities: p, p,, p m where X i = numer of trils with outcome i m å i = p i = Count of ech word Joint distriution where m åci = i= n nd Multinomil # wys of ordering the successes æ ç èc, c n,..., c ö ø = c! c n!! c m m Proilities of ech word!

13 6-sided die is rolled 7 times Roll results: one, two, 0 three, four, 0 five, 3 six This is generliztion of Binomil distriution Binomil: ech tril hd possile outcomes Multinomil: ech tril hs m possile outcomes !!0!!0!3! 7! 3) 0,, 0,,, ( ø ö ç è æ = ø ö ç è æ ø ö ç è æ ø ö ç è æ ø ö ç è æ ø ö ç è æ ø ö ç è æ = = = = = = = X X X X X X P Hello Die Rolls, My Old Friends

14 Proilistic Text Anlysis According to the Glol Lnguge Monitor there re 988,968 words in the english lnguge used on the internet. The

15 Text is Multinomil P Exmple document: Py for Vigr with credit-crd. Vigr is gret. So re credit-crds. Risk free Vigr. Click for free. n = 8 Vigr = Free = Risk = Credit-crd: For = Proility of seeing this document spm spm = It s Multinomil! n!!!...! p vigrp free...p for The proility of word in spm emil eing vigr

16 Four Prototypicl Trjectories Who wrote the federlist ppers?

17

18 Old nd New Anlysis Authorship of Federlist Ppers 85 essys dvocting rtifiction of US constitution Written under pseudonym Pulius o Relly, Alexnder Hmilton, Jmes Mdison nd John Jy Who wrote which essys? o Anlyzed proility of words in ech essy versus word distriutions from known writings of three uthors

19 Four Prototypicl Trjectories Let s write progrm!

20 Text is Multinomil P Exmple document: Py for Vigr with credit-crd. Vigr is gret. So re credit-crds. Risk free Vigr. Click for free. n = 8 Vigr = Free = Risk = Credit-crd: For = Proility of seeing this document spm spm = It s Multinomil! n!!!...! p vigrp free...p for The proility of word in spm emil eing vigr

21 Continuous Rndom Vriles Joint Distriutions

22 Four Prototypicl Trjectories Continuous Joint Distriution

23 Riding the Mrguerite You re running to the us stop. You don t know exctly when the us rrives. You rrive t :0pm. Wht is P(wit < 5 min)?

24 Joint Drt Distriution P(hit within R pixels of center)? Wht is the proility tht drt hits t ( , )?

25 Joint Drt Distriution P(hit within R pixels of center)? Drt y loction Drt x loction

26 Joint Drt Distriution P(hit within R pixels of center)? Drt y loction Drt x loction

27 Joint Drt Distriution P(hit within R pixels of center)? Drt y loction Drt x loction

28 Joint Drt Distriution 900 y 0 x 900 In the limit, s you rek down continuous vlues into intestinlly smll uckets, you end up with multidimensionl proility density

29 Joint Proility Density Funciton A joint proility density function gives the reltive likelihood of more thn one continuous rndom vrile ech tking on specific vlue y P ( < X, < Y ) = f X, ò ò Y ( x, x 900 y) dy dx

30 ò ò = < < ), ( ), ( P, Y X dx dy y x f Y X f X,Y (x, y) x y Joint Proility Density Funciton

31 plot y Acdemo Joint Proility Density Funciton P ( < X, < Y ) = f X, ò ò Y ( x, y) dy dx!

32 Multiple Integrls Without Ters l Let X nd Y e two continuous rndom vriles where 0 X nd 0 Y l We wnt to integrte g(x,y) = xy w.r.t. X nd Y: First, do innermost integrl (tret y s constnt): ò ò xy dx dy = ò æ ç è ò ö xy dx dy ø 0 y= 0 x= 0 y= 0 x= 0 y= 0 y= 0 Then, evlute remining (single) integrl: = ò é x yê ë ù ú û dy = ò y dy ò y= 0 y dy = é ê ë y 4 ù ú û 0 = - 0 =

33 Mrginliztion Mrginl proilities give the distriution of suset of the vriles (often, just one) of joint distriution. Sum/integrte over the vriles you don t cre out. p X () = X y p X,Y (, y) f X () = Z f X,Y (, y) dy f Y () = Z f X,Y (x, ) dx

34 Drts! Drt PDF y X-Pixel Mrginl x Y-Pixel Mrginl X N 900, 900 Y N 900 3, 900 5

35 Cumultive Density Function (CDF): ò ò - - = Y X Y X dx dy y x f F ), ( ), (,, ), ( ), (,, F f Y X Y X = Joint Cumultive Density Function

36 Joint CDF to s x +, y +, " to 0 s x -, y -,! plot y Acdemo

37 Jointly Continuous ò ò = < < ), ( ), ( P, Y X dx dy y x f Y X f X,Y (x, y) x y

38 Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) '

39 Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) '

40 Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) '

41 Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) ', -,. " #, ) '

42 Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) ', -,. " #, ) '

43 Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) ', -,. " #, ) ', -,. " ', ) #

44 Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) ', -,. " #, ) ', -,. " ', ) #

45 Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) ', -,. " #, ) ', -,. " ', ) # +, -,. " #, ) #

46 Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) ', -,. " #, ) ', -,. " ', ) # +, -,. " #, ) #

47 Proility for Instgrm!

48 Gussin Blur In imge processing, Gussin lur is the result of lurring n imge y Gussin function. It is widely used effect in grphics softwre, typiclly to reduce imge noise. Gussin lurring with StDev = 3, is sed on joint proility distriution: Joint PDF f X,Y (x, y) = 3 e x +y 3 Joint CDF F X,Y (x, y) = x 3 y 3 Used to generte this weight mtrix

49 Gussin Blur Joint PDF f X,Y (x, y) = 3 e x +y 3 Joint CDF F X,Y (x, y) = Weight Mtrix x 3 y 3 Ech pixel is given weight equl to the proility tht X nd Y re oth within the pixel ounds. The center pixel covers the re where -0.5 x 0.5 nd -0.5 y 0.5 Wht is the weight of the center pixel? P ( 0.5 < X < 0.5, 0.5 < Y < 0.5) =P (X < 0.5,Y < 0.5) P (X < 0.5,Y < 0.5) P (X < 0.5,Y <0.5) + P (X < 0.5,Y < 0.5) = = =0.06

50 Four Prototypicl Trjectories Hve gret weekend!