Revew for the prevous lecture: Theorems ad Examples: How to obta the pmf (pdf) of U = g (, Y) ad V = g (, Y) Chapter 4 Multple Radom Varables Chapter 44 Herarchcal Models ad Mxture Dstrbutos Examples: Bomal-Posso Herarchy: =umber survved ad Y =umber of eggs lad The we ca use the models Y ~ bomal( Y, p ) ad Y ~ Posso( λ ) I ths case, ~ Posso( λ p) Posso-Expoetal Herarchy: Let Y =umber of eggs lad ad Λ =varablty across dfferet mothers Y Λ ~Posso( Λ ) ad Λ ~expoetal( β ) I ths case, Y ~ egatve bomal( r =, p= /( + β )) 3 Bomal-Posso-Expoetal Herarchy (Three-stage herarchy): Let =umber survved, Y =umber of eggs lad ad, Λ = varablty across dfferet mothers Y ~ bomal( Y, p ) ad Y Λ ~ Posso( Λ ) ad Λ ~expoetal( β ) Equvaletly, we ca look at ths as a two-stage herarchy where Y ~ bomal( Y, p ) ad Y ~ egatve bomal( r =, p= /( + β )) 4 Beta-Bomal Herarchy: P~ bomal( P, ) ad P ~ beta( α, β ) Defto 444: A radom varable s sad to have a mxture dstrbuto f the dstrbuto of depeds o a quatty that also has a dstrbuto
IMPORTANT RESULTS: These results are useful whe oe s terested oly the expectato ad varaces of a radom varable Theorem 443: If ad Y are ay two radom varables, the E = E( E( Y)), provded that the expectatos exst Proof: We prove ths theorem the cotuous stuato Let f, ( xy, ) deote the jot pdf of ad Y The we have E = xf, ( xydxdy, ) = [ xf( x ydxf ) ] Y( ydy ) = E ( y) fy( ydy ) = EE ( ( Y)) Theorem 447: For ay two radom varables ad Y, the Var = E( Var( Y )) + Var( E( Y )), provded that the expectatos exst Proof Frst we have, Var = E( E) = E( E( Y) + E( Y) E) = E( E( Y)) + E( E( Y) E) + E([ E( Y)][ E( Y) E]) I addto, E([ E( Y)][ E( Y) E]) = E( E{[( E( Y))( E( Y) E)] Y}),
ad Fally, we have E{[ E( Y)][ E( Y) E] Y} = ( E( Y) E)( E{[ E( Y)] Y}) = ( E( Y) E)( E( Y) E( Y)) = 0 E E Y E E E Y Y = EVar ( ( Y)), ( ( )) = ( [ ( )] ) E E Y E VarE Y ([ ( ) ] ) = ( ( ), Var = E( Var( Y )) + Var( E( Y )) Illustratos: Bomal-Posso Herarchy: models Y ~ bomal(( Y, p ) ad Y ~ Posso( λ ) (Recall that ~ Posso( λ p) ) Usg Theorem 443 ad 445, we get E = E( E( Y)) = E( py) = λ p ad Var E Var Y Var E Y E Yp p Var py p p λ p λ = ( ( )) + ( ( )) = ( ( )) + ( ) = ( ) + Beta-Bomal Herarchy: P~ bomal( P, ) ad P ~ beta( α, β ) Fd E ad Var 3 No-cetral ch-squared dstrbuto wth degrees of freedom, k, ad ocetralty parameter λ p/+ k x/ k λ x e λ e f( x λ, p) = k= 0 p/+ k Note that K ~ χ p+ k ad K ~Posso( λ ) Therefore, Γ ( p / + k) k! E = E( E( P)) = E( p+ K) = p+ λ Chapter 45 Covarace ad Correlato 3
Defto 45: The covarace of ad Y s the umber defed by Cov(, Y) = E( E )( Y EY ) Some Notes: < Cov(, Y ) < Cov(, Y ) = Cov( Y, ) 3 Postve (egatve) covarace s whe small values of ted to be observed wth small (large) values of Y Defto 45: The correlato of Cov(, Y ) = E( ) μ μy ad Y s the umber defed by ρ (correlato coeffcet) Cov(, Y ) = σ σ Y Some Notes: ρ xy ρ = ad ρ = mply the lear relatoshp betwee ad Y s perfectly egatve ad postve, respectvely Theorem 453: For ay radom varables ad Y, Cov(, Y ) == E( ) E( ) E( Y) = E( ) μ μy Example 454: Let (, ) be bvarate radom varables wth jot pdf f ( xy, ) =,0 < x<, x< y< x+ 4
() Fd the margal pdf of ad Y () Fd the meas ad varaces of ad Y (3) Fd the covarace ad correlato of ad Y Theorem 455: If ad Y are depedet radom varables, the Cov(, Y ) = 0 ad ρ = 0 IMPORTANT NOTE: Theorem 454 does ot say that f Cov(, Y ) = 0 or ρ = 0, the ad Y are depedet Covarace ad correlato measure oly a partcular kd of lear relatoshp Theorem 456: If ad Y are ay two radom varables ad a ad b are ay two costats, the Var( a + by ) = a Var + b VarY + abcov(, Y ) If ad Y are depedet, the Var( a by ) a Var b VarY + = + Theorem 457: For ay radom varables ad Y, a ρ b ρ = f ad oly f there exst umbers a 0 ad bsuch that PY ( = a+ b) = If ρ =, the a > 0, ad f ρ =, the a < 0 See Example 459 (p 74-75) for a example where ad Y are strogly related (o-lear) but ther covarace s 0 5
Defto 450: Let < μ <, < μy <, σ > 0, σ Y > 0, ad < ρ < be fve real umbers The bvarate ormal pdf wth meas μ ad μ Y, varaces σ ad σ Y, ad correlato ρ s the bvarate pdf gve by x μ x μ y μ y μ f( x, y) = exp ( ) ρ( )( ) + ( ) πσ σ Y for < x <, < y < Y Y ρ ( ρ ) σ σ σy σy, Alteratvely, we ca re-wrte ths formula a geeral way Defe ρσ σ, whch the varace- covarace of the bvarate ormal dstrbuto, the we have σ Σ= ρσ σ Y Y σ Y T f( x, y) = exp (, ) (, ) / x μ y μy Σ x μ y μy ( π ) Σ Propertes: The margal dstrbutos of ad Y are μ σ ad The correlato betwee ad Y s ρ = ρ (, ) μ σ ( Y, Y), respectvely 3 For ay costats a ad b, the dstrbuto of a + by s a + b a + b + ab ( μ μy, σ σy ρσ σy) 4 The codtoal dstrbutos of gve Y = y ad of Y gve = x are also ormal dstrbutos The pdf of Y = y s ( μ ρσ ( / σy)( y μy), σ ( ρ )) + 5 ad Y are depedet f ad oly f ρ = 0 6
Example (Margal Normalty Does ot Imply Jot Normalty): Let ad Y are depedet radom varables wth the pdf (0,), defe, f >0; Z = -, f <0 The Z has ormal dstrbuto but Z ad Y s ot bvarate ormal Chapter 46 Multvarate Dstrbutos Cosder a radom vector =(,,, ) wth sample space that s a subset of x = ( x, x,, x ) to deote a sample R I addto, we wrte Jot pmf (Dscrete Case): The jot pmf of (,,, ) s defed as f (x) = P ( = x, = x,, = x) for each ( x, x,, x) R Hece for A R, P ( A) f(x) = x A Jot pdf (Cotuous Case): The jot pdf of (,,, ) s the fucto that satsfes P ( A ) = f (x) d x = f ( x, x,, x ) dx dx dx A A Jot cdf: The jot cdf of (,,, ) s defed as 7
F(x) = P( x, x,, x ) for each ( x, x,, x) R Expected Values: Eg() g(x) f(x) dx or = Eg() = R g(x) f(x) x Margal dstrbuto of (,,, ): or f ( x, x,, x ) f( x,, x ) dx dx, = k k+ f ( x, x,, x ) = f ( x,, x ) k k ( x,, ) R k+ x Codtoal pdf or pmf of (,,, ): If f( x, x,, x ) > 0, the the codtoal pdf or pmf of (, k+, ) gve,, = x k = xk s f ( x, x x,, x ) f( x, x,, x )/ f( x,, x ) k+, k = k Defto 46: Let m ad be postve tegers ad let p, p,, p be umbers satsfyg 0 p, =,, ad p = = The the radom vector (,,, ) has a multomal dstrbuto wth m trals ad cell probabltes p, p,, p f the jot pmf s (,,, ) m! x x f x x x = p p x! x! 8
o the set of ( x,, x ) such that each x s a oegatve teger ad x = m = Notes: m! x! x! s kow as the multomal coeffce x t whch deotes the umber of ways m objects ca be dvded to groups wth x the frst, the secod,, x the th group Bomal s a specal case of the multomal dstrbuto where = Theorem 464 (Multomal Theorem): Let m ad be postve tegers Let A be the set of vectors x = ( x,, x ) such that each x s a oegatve teger ad x = m The for ay real umbers p,, p, ( ) = m! m x x p p p p + + = x A x! x! Results: The margal dstrbuto of s bomal( mp, ) p p The codtoal dstrbuto of,, gve = x s multomal( m x;,, ) p p 3 The covarace of ad j s Cov(, j) = mpp j 9
Defto 465: Let,, be radom vectors wth jot pdf or pmf f ( x,, x ) Let f ( ) x deote the margal pdf or pmf of The,, are called mutually depedet radom vectors, f for every ( x,, x ), f ( x,, x ) f ( x ) f ( x ) f ( x ) = = = If the are all oe-dmesoal, the,, are called mutually depedet radom varables Note: Parwse depedece does ot mply mutual depedece Theorem 466: Let,, be mutually depedet E( g( ) g( )) = E( g( )) E( g( )) t radom varables The Theorem 467: Let,, be mutually depedet radom varables wth Z = + + The M () t = M () t M () t Z M (), t, M () t Defe Example 468: Usg the mgf techque, show that f ~ gamma( α, β ), the Z = + + ~gamma( α + + α, β) 0
Corollary 469: Let,, be mutually depedet radom varables wth ad b,, b be fxed costats Defe Z = ( a + b) + + ( a + b) The t b M () t = e M ( at) M ( a t) Z M (), t, M () t Let a,, a Corollary 460: Let,, be mutually depedet radom varables wth b,, b fxed costats Defe Z = ( a + b) + + ( a + b) The ~ ( ( ), ) μ + σ = = Z a b a ~ ( μ, σ ) Let a,, a ad Theorem 46: Let,, be radom vectors The Let,, be mutually depedet radom vectors f ad oly f there exst fuctos g( x)( =,, ), such that the jot pdf or pmf of,, ca be wrtte as f ( x,, x ) = g ( x ) g ( x ) Theorem 46: Let = be radom vectors Let g( x)( =,, ) be a fucto oly of x The the radom varables U = g ( )( =,, ) are mutually depedet Theorem: Let (,, ) be radom varables wth jot pdf f( x,, x ) Let A be the support set Cosder the ew radom vector U = ( U,, U ), defed by U = g() For a partto A0, A0,, A k of A : P( A 0) = 0 ad o each A, the trasformato s oe-oe from A to B, the we have x j = hj( u,, u)( j =,, ; =,, k) Let J be the Jacoba o A, the the jot pdf of ( U,, U ) s k fu( u,, u ) = f ( h ( u,, u),, h( u,, u)) J =
Example 46 ad 463 Example 46 (Multvarate pdfs) Let = 4 ad 3 f ( x, x, x3, x4) = ( x + x + x3 + x4) for 0 < x <, =,,3,4 4 Ths jot pdf ca be used to compute probabltes such as 3/4 /3 P( < /, < 3/4, 4 > /) = / 0 0 ( x 0 + x + x3 + x4) dxdxdx3dx4 4 3 3 3 / 3 3/4 3 3 3 3 7 = ( x 0 + x 0 + x3 0 + x4 /) = 43 4 4 4 04 The margal pdf of (, ) by tegratg out the varables x 3 ad x 4 to obta 3 3 f( x, x) = ( x 0 0 + x + x3 + x4) dx3dx4 = ( x + x) + for 0< x < ad 0< x < 4 4 The expected value that volves oly (, ) ca be computed usg ths margal pdf For example: 3 5 E ( ) = ( xx 0 0 )[ ( x + x) + ] dxdx = 4 6 The codtoal pdf of ( 3, 4) gve = x, = x s: f ( x, x, x, x ) x + x + x + x 3 4 3 4 3 4 x x = = f( x, x) x + x + /3 f( x, x, ) For example, the codtoal pdf of ( 3, 4) gve = /3, = /3 s: for 0 < x <, =,,3,4
5 9 9 f ( x3, x4 = /3, = /3) = + x3 + x4 Ths ca used to compute 5 9 9 7 E ( 3 4 = /3, = /3) = xx 0 0 3 4( + x3 + x4) dxdx 3 4 = Defto: Let u = ( u,, u ) ad be the covarace matrx, whch σ j = σσ jρj The multvarate ormal pdf s gve by T f( x) = exp ( - ) ( - ) / x u Σ x u ( π ) Σ Propertes: The margal dstrbuto s the multvarate ormal The correlato s ρ j 3 Ay lear combato has the ormal dstrbuto 4 The codtoal dstrbuto s the multvarate ormal 5 They are mutually mpedet f ad oly f they are parwse depedet (parwse u-correlated) 6 If ay lear combato has the ormal dstrbuto, the the jot pdf s the multvarate ormal 3