Continuous Joint Distriutions Chris Piech CS09, Stnford University
CS09 Flow Tody Discrete Joint Distriutions: Generl Cse Multinomil: A prmetric Discrete Joint Cont. Joint Distriutions: Generl Cse
Lerning Gols. Know how to use multinomil. Be le to clculte lrge yes prolems using computer 3. Use Joint CDF
Motivting Exmples
Four Prototypicl Trjectories Recll logs
Log Review e y = x log(x) =y
Log Identities log( ) = log() + log() log(/) = log() log() log( n )=n log()
Products ecome Sums! log( ) = log() + log() log( Y i i )= X i log( i ) * Spoiler lert: This is importnt ecuse the product of mny smll numers gets hrd for computers to represent.
Four Prototypicl Trjectories Where we left off
Joint Proility Tle Wlk Bike Scooter Drive Mrginl Yer Freshmn 0.04 0.04 0.0 0.03 0. Sophomore 0.03 0.34 0.03 0.00 0.40 Junior 0.04 0. 0.0 0.00 0.5 Senior 0.07 0.08 0.0 0.00 0.6 5+ 0.04 0.07 0.00 0.0 0. Mrginl Mode 0. 0.73 0.06 0.05
The Multinomil Multinomil distriution n independent trils of experiment performed Ech tril results in one of m outcomes, with respective proilities: p, p,, p m where X i = numer of trils with outcome i m å i = p i = P c c ( X = c, X = c,..., X m = cm) = ç p p c, c,..., cm where æ n ö ç è ø Joint distriution m åci = i= n nd Multinomil # wys of ordering the successes æ ç èc, c n,..., c ö ø = c! c n!! c m m!... p c m m Proilities of ech ordering re equl nd mutully exclusive
The Multinomil Multinomil distriution n independent trils of experiment performed Ech tril results in one of m outcomes, with respective proilities: p, p,, p m where X i = numer of trils with outcome i m å i = p i = Count of ech word Joint distriution where m åci = i= n nd Multinomil # wys of ordering the successes æ ç èc, c n,..., c ö ø = c! c n!! c m m Proilities of ech word!
6-sided die is rolled 7 times Roll results: one, two, 0 three, four, 0 five, 3 six This is generliztion of Binomil distriution Binomil: ech tril hd possile outcomes Multinomil: ech tril hs m possile outcomes 7 3 0 0 6 5 4 3 6 40 6 6 6 6 6 6!!0!!0!3! 7! 3) 0,, 0,,, ( ø ö ç è æ = ø ö ç è æ ø ö ç è æ ø ö ç è æ ø ö ç è æ ø ö ç è æ ø ö ç è æ = = = = = = = X X X X X X P Hello Die Rolls, My Old Friends
Proilistic Text Anlysis According to the Glol Lnguge Monitor there re 988,968 words in the english lnguge used on the internet. The
Text is Multinomil P Exmple document: Py for Vigr with credit-crd. Vigr is gret. So re credit-crds. Risk free Vigr. Click for free. n = 8 Vigr = Free = Risk = Credit-crd: For = Proility of seeing this document spm spm = It s Multinomil! n!!!...! p vigrp free...p for The proility of word in spm emil eing vigr
Four Prototypicl Trjectories Who wrote the federlist ppers?
Old nd New Anlysis Authorship of Federlist Ppers 85 essys dvocting rtifiction of US constitution Written under pseudonym Pulius o Relly, Alexnder Hmilton, Jmes Mdison nd John Jy Who wrote which essys? o Anlyzed proility of words in ech essy versus word distriutions from known writings of three uthors
Four Prototypicl Trjectories Let s write progrm!
Text is Multinomil P Exmple document: Py for Vigr with credit-crd. Vigr is gret. So re credit-crds. Risk free Vigr. Click for free. n = 8 Vigr = Free = Risk = Credit-crd: For = Proility of seeing this document spm spm = It s Multinomil! n!!!...! p vigrp free...p for The proility of word in spm emil eing vigr
Continuous Rndom Vriles Joint Distriutions
Four Prototypicl Trjectories Continuous Joint Distriution
Riding the Mrguerite You re running to the us stop. You don t know exctly when the us rrives. You rrive t :0pm. Wht is P(wit < 5 min)?
Joint Drt Distriution P(hit within R pixels of center)? Wht is the proility tht drt hits t (456.34334355, 53.3443456)?
Joint Drt Distriution P(hit within R pixels of center)? Drt y loction 0. 0.005 Drt x loction
Joint Drt Distriution P(hit within R pixels of center)? Drt y loction 0. 0.005 Drt x loction
Joint Drt Distriution P(hit within R pixels of center)? Drt y loction Drt x loction
Joint Drt Distriution 900 y 0 x 900 In the limit, s you rek down continuous vlues into intestinlly smll uckets, you end up with multidimensionl proility density
Joint Proility Density Funciton A joint proility density function gives the reltive likelihood of more thn one continuous rndom vrile ech tking on specific vlue. 900 900 y 0 900 0 P ( < X, < Y ) = f X, ò ò Y ( x, x 900 y) dy dx
ò ò = < < ), ( ), ( P, Y X dx dy y x f Y X f X,Y (x, y) x y Joint Proility Density Funciton
plot y Acdemo Joint Proility Density Funciton P ( < X, < Y ) = f X, ò ò Y ( x, y) dy dx!
Multiple Integrls Without Ters l Let X nd Y e two continuous rndom vriles where 0 X nd 0 Y l We wnt to integrte g(x,y) = xy w.r.t. X nd Y: First, do innermost integrl (tret y s constnt): ò ò xy dx dy = ò æ ç è ò ö xy dx dy ø 0 y= 0 x= 0 y= 0 x= 0 y= 0 y= 0 Then, evlute remining (single) integrl: = ò é x yê ë ù ú û dy = ò y dy ò y= 0 y dy = é ê ë y 4 ù ú û 0 = - 0 =
Mrginliztion Mrginl proilities give the distriution of suset of the vriles (often, just one) of joint distriution. Sum/integrte over the vriles you don t cre out. p X () = X y p X,Y (, y) f X () = Z f X,Y (, y) dy f Y () = Z f X,Y (x, ) dx
Drts! Drt PDF y X-Pixel Mrginl x Y-Pixel Mrginl X N 900, 900 Y N 900 3, 900 5
Cumultive Density Function (CDF): ò ò - - = Y X Y X dx dy y x f F ), ( ), (,, ), ( ), (,, F f Y X Y X = Joint Cumultive Density Function
Joint CDF to s x +, y +, " to 0 s x -, y -,! plot y Acdemo
Jointly Continuous ò ò = < < ), ( ), ( P, Y X dx dy y x f Y X f X,Y (x, y) x y
Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) '
Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) '
Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) '
Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) ', -,. " #, ) '
Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) ', -,. " #, ) '
Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) ', -,. " #, ) ', -,. " ', ) #
Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) ', -,. " #, ) ', -,. " ', ) #
Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) ', -,. " #, ) ', -,. " ', ) # +, -,. " #, ) #
Proilities from Joint CDF! " # < % " ', ) # < * ) ' =, -,. " ', ) ', -,. " #, ) ', -,. " ', ) # +, -,. " #, ) #
Proility for Instgrm!
Gussin Blur In imge processing, Gussin lur is the result of lurring n imge y Gussin function. It is widely used effect in grphics softwre, typiclly to reduce imge noise. Gussin lurring with StDev = 3, is sed on joint proility distriution: Joint PDF f X,Y (x, y) = 3 e x +y 3 Joint CDF F X,Y (x, y) = x 3 y 3 Used to generte this weight mtrix
Gussin Blur Joint PDF f X,Y (x, y) = 3 e x +y 3 Joint CDF F X,Y (x, y) = Weight Mtrix x 3 y 3 Ech pixel is given weight equl to the proility tht X nd Y re oth within the pixel ounds. The center pixel covers the re where -0.5 x 0.5 nd -0.5 y 0.5 Wht is the weight of the center pixel? P ( 0.5 < X < 0.5, 0.5 < Y < 0.5) =P (X < 0.5,Y < 0.5) P (X < 0.5,Y < 0.5) P (X < 0.5,Y <0.5) + P (X < 0.5,Y < 0.5) 0.5 0.5 0.5 0.5 = 3 3 3 3 0.5 0.5 + 3 3 =0.566 0.566 0.4338 + 0.4338 =0.06
Four Prototypicl Trjectories Hve gret weekend!