7. Joint Distributions

Transcription

1 7. Jot Dstrbutos Chrs Pech ad Mehra Saham Ma 2017 Ofte ou wll work o problems where there are several radom varables (ofte teractg wth oe aother. We are gog to start to formall look at how those teractos pla out. For ow we wll thk of jot probabltes wth two radom varables X ad Y. 1 Dscrete Jot Dstrbutos I the dscrete case a jot probablt mass fucto tells ou the probablt of a combato of evets X a ad Y b: p X,Y (a,b P(X a,y b Ths fucto tells ou the probablt of all combatos of evets (the, meas ad. If ou wat to back calculate the probablt of a evet ol for oe varable ou ca calculate a margal from the jot probablt mass fucto: p X (a P(X a P X,Y (a, p Y (b P(Y b P X,Y (,b I the cotuous case a jot probablt dest fucto tells ou the relatve probablt of a combato of evets X a ad Y. I the dscrete case, we ca defe the fucto p X,Y o-parametrcall. Istead of usg a formula for p we smpl state the probablt of each possble outcome. 2 Cotuous Jot Dstrbutos Radom varables X ad Y are Jotl Cotuous f there ests a Probablt Dest Fucto (PDF f X,Y such that: P(a 1 < X a 2,b 1 < Y b 2 a2 b2 a 1 b 1 f X,Y (d d Usg the PDF we ca compute margal probablt destes: f X (a f Y (b f X,Y (a,d f X,Y (,bd Let F(a,b be the Cumulatve Dest Fucto (CDF: P(a 1 < X a 2,b 1 < Y b 2 F(a 2,b 2 F(a 1,b 2 + F(a 1,b 1 F(a 2,b 1 1

2 3 Multomal Dstrbuto Sa ou perform depedet trals of a epermet where each tral results oe of m outcomes, wth respectve probabltes: p 1, p 2,..., p m (costraed so that p 1. Defe X to be the umber of trals wth outcome. A multomal dstrbuto s a closed form fucto that aswers the questo: What s the probablt that there are c trals wth outcome. Mathematcall: ( P(X 1 c 1,X 2 c 2,...,X m c m p c 1 1 c 1,c 2,...,c pc pc m m m Eample 1 A -sded de s rolled 7 tmes. What s the probablt that ou roll: 1 oe, 1 two, 0 threes, 2 fours, 0 fves, 3 ses (dsregardg order. P(X 1 1,X 2 1,X 3 0,X 4 2,X 5 0,X 3 7! 2!3! ( ( 1 1 ( 1 1 ( 1 0 ( 1 2 ( 1 0 ( 1 3 Fedaralst Papers I class we wrote a program to decde whether or ot James Madso or Aleader Hamlto wrote Fedaralst Paper 49. Both me have clamed to be have wrtte t, ad hece the authorshp s dspute. Frst we used hstorcal essas to estmate p, the probablt that Hamlto geerates the word (depedet of all prevous ad future choces or words. Smlarl we estmated q, the probablt that Madso geerates the word. For each word we observe the umber of tmes that word occurs Fedaralst Paper 49 (we call that cout c. We assume that, gve o evdece, the paper s equall lkel to be wrtte b Madso or Hamlto. Defe three evets: H s the evet that Hamlto wrote the paper, M s the evet that Madso wrote the paper, ad D s the evet that a paper has the collecto of words observed Fedaralst Paper 49. We would lke to kow whether P(H D s larger tha P(M D. Ths s equvalet to trg to decde f P(H D/P(M D s larger tha 1. The evet D H s a multomal parameterzed b the values p. The evet D M s also a multomal, ths tme parameterzed b the values q. Usg Baes Rule we ca smplf the desred probablt. P(H D P(M D P(D HP(H P(D P(D MP(M P(D ( c 1,c 2,...,c m p c ( c 1,c 2,...,c m q c P(D HP(H P(D MP(M P(D H P(D M p c q c Ths seems great! We have our desred probablt statemet epressed terms of a product of values we have alread estmated. However, whe we plug ths to a computer, both the umerator ad deomator come out to be zero. The product of ma umbers close to zero s too hard for a computer to represet. To f ths problem, we use a stadard trck computatoal probablt: we appl a log to both sdes ad appl 2

3 some basc rules of logs. ( P(H D log log P(M D ( p c q c log( p c log(p c log( q c log(q c c log(p c log(q Ths epresso s umercall stable ad m computer retured that the aswer was a egatve umber. We ca use epoetato to solve for P(H D/P(M D. Sce the epoet of a egatve umber s a umber smaller tha 1, ths mples that P(H D/P(M D s smaller tha 1. As a result, we coclude that Madso was more lkel to have wrtte Federalst Paper Epectato wth Multple RVs Epectato over a jot s t cel defed because t s ot clear how to compose the multple varables. However, epectatos over fuctos of radom varables (for eample sums or multplcatos are cel defed: E[g(X,Y ] g(p( for a fucto g(x,y. Whe ou epad that result for the fucto g(x,y X +Y ou get a beautful result: E[X +Y ] E[g(X,Y ] g(p( [ + ]p( p( + p(, p( + p( + p( E[X] + E[Y ] Ths ca be geeralzed to multple varables: E [ 1X ] 1 E[X ] p(, Asde: A Lovel Lemma Here s a lovel lemmas: Ad dd ou kow that f Y s a o-egatve radom varable the followg hold (for dscrete ad cotuous radom varables respectvel: E[Y ] E[Y ] 1 0 P(Y P(Y d 3

4 Eample 3 A dsk surface s a crcle of radus R. A sgle pot mperfecto s uforml dstrbuted o the dsk wth jot PDF: { 1 f R 2 f X,Y ( πr 2 0 else Let D be the dstace from the org: D X 2 +Y 2. What s E[D]? Ht: use the lemmas 5 Idepedece wth Multple RVs Dscrete Two dscrete radom varables X ad Y are called depedet f: P(X,Y P(X P(Y for all Itutvel: kowg the value of X tells us othg about the dstrbuto of Y. If two varables are ot depedet, the are called depedet. Ths s a smlar coceptuall to depedet evets, but we are dealg wth multple varables. Make sure to keep our evets ad varables dstct. Cotuous Two cotuous radom varables X ad Y are called depedet f: P(X a,y b P(X ap(y b for all a,b Ths ca be stated equvaletl as: F X,Y (a,b F X (af Y (b for all a,b f X,Y (a,b f X (a f Y (b for all a,b More geerall, f ou ca factor the jot dest fucto the our cotuous radom varable are depedet: f X,Y ( h(g( where < < Eample 2 Let N be the # of requests to a web server/da ad that N Po(λ. Each request comes from a huma (probablt p or from a bot (probablt (1 p, depedetl. Defe X to be the # of requests from humas/da ad Y to be the # of requests from bots/da. Sce requests come depedetl, the probablt of X codtoed o kowg the umber of requests s a Bomal. Specfcall: (X N B(N, p (Y N B(N,1 p Calculate the probablt of gettg eactl huma requests ad j bot requests. Start b epadg usg the cha rule: P(X,Y j P(X,Y j X +Y + jp(x +Y + j 4

5 We ca calculate each term ths epresso: ( + j P(X,Y j X +Y + j p (1 p j P(X +Y + j e λ λ + j ( + j! Now we ca put those together ad smplf: ( + j P(X,Y j p (1 p j λ λ + j e ( + j! As a eercse ou ca smplf ths epresso to two depedet Posso dstrbutos. Smmetr of Idepedece Idepedece s smmetrc. That meas that f radom varables X ad Y are depedet, X s depedet of Y ad Y s depedet of X. Ths clam ma seem meagless but t ca be ver useful. Image a sequece of evets X 1,X 2,... Let A be the evet that X s a record value (eg t s larger tha all prevous values. Is A +1 depedet of A? It s easer to aswer that A s depedet of A +1. B smmetr of depedece both clams must be true. Covoluto of Dstrbutos Covoluto s the result of addg two dfferet radom varables together. For some partcular radom varables computg covoluto has tutve closed form equatos. Importatl covoluto s the sum of the radom varables themselves, ot the addto of the probablt dest fuctos (PDFs that correspod to the radom varables. Idepedet Bomals wth equal p For a two Bomal radom varables wth the same success probablt: X B( 1, p ad Y B( 2, p the sum of those two radom varables s aother bomal: X +Y B( 1 + 2, p. Ths does ot hold whe the two dstrbuto have dfferet parameters p. Idepedet Possos For a two Posso radom varables: X Po(λ 1 ad Y Po(λ 2 the sum of those two radom varables s aother Posso: X +Y Po(λ 1 + λ 2. Ths holds whe λ 1 s ot the same as λ 2. Idepedet Normals For a two ormal radom varables X N (µ 1,σ1 2 ad Y N (µ 2,σ2 2 the sum of those two radom varables s aother ormal: X +Y N (µ 1 + µ 2,σ1 2 + σ 2 2. Geeral Idepedet Case For two geeral depedet radom varables (aka cases of depedet radom varables that do t ft the above specal stuatos ou ca calculate the CDF or the PDF of the sum of two radom varables usg the 5

6 followg formulas: F X+Y (a P(X +Y a f X+Y (a f X (a f Y (d F X (a f Y (d There are drect aaloges the dscrete case where ou replace the tegrals wth sums ad chage otato for CDF ad PDF. Eample 1 Calculate the PDF of X +Y for depedet uform radom varables X U(0,1 ad Y U(0,1? Frst plug the equato for geeral covoluto of depedet radom varables: 1 f X+Y (a f X (a f Y (d f X+Y (a f X (a d Because f Y ( 1 It turs out that s ot the easest thg to tegrate. B trg a few dfferet values of a the rage [0,2] we ca observe that the PDF we are trg to calculate s dscotuous at the pot a 1 ad thus wll be easer to thk about as two cases: a < 1 ad a > 1. If we calculate f X+Y for both cases ad correctl costra the bouds of the tegral we get smple closed forms for each case: a f 0 < a 1 f X+Y (a 2 a f 1 < a 2 0 else 7 Codtoal Dstrbutos Before we looked at codtoal probabltes for evets. Here we formall go over codtoal probabltes for radom varables. The equatos for both the dscrete ad cotuous case are tutve etesos of our uderstadg of codtoal probablt: Dscrete The codtoal probablt mass fucto (PMF for the dscrete case: p X Y ( P(X Y P(X,Y P(Y P X,Y ( p Y ( The codtoal cumulatve dest fucto (CDF for the dscrete case: F X Y (a P(X a Y a p X,Y ( p Y ( p X Y ( a Cotuous The codtoal probablt dest fucto (PDF for the cotuous case: f X Y ( f X,Y ( f Y (

7 The codtoal cumulatve dest fucto (CDF for the cotuous case: a F X Y (a P(X a Y f X Y ( d Eample 2 Let s sa we have two depedet radom Posso varables for requests receved at a web server a da: X # requests from humas/da, X Po(λ 1 ad Y # requests from bots/da, Y Po(λ 2. Sce the covoluto of Posso radom varables s also a Posso we kow that the total umber of requests (X +Y s also a Posso (X +Y Po(λ 1 + λ 2. What s the probablt of havg k huma requests o a partcular da gve that there were total requests? P(X k X +Y P(X k,y k P(X +Y P(X kp(y k P(X +Y e λ 1λ1 k e λ 2λ2 k k! ( k!! e 1(λ 1+λ 2 (λ 1 + λ 2 ( ( k ( k λ1 λ2 k λ 1 + λ 2 λ 1 + λ 2 ( λ 2 B, λ 1 + λ 2 7