Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for Chapter 4-5 Notes: Although all deftos ad theorems troduced our lectures ad ths ote are mportat ad you should be famlar wth, but I put those deftos ad theorems that you must be very famlar wth ad you ca use them wthout referrg to ay books or otes red ths ote I also put those deftos ad theorems that you should kow but you do ot eed to memorze blue ths ote Chapter 4 Multple Radom Varables Chapter 4 Jot ad Margal Dstrbutos Defto 4: A -dmesoal radom vector s a fucto from a sample space S to Eucldea space R, -dmesoal Defto 43: Let (, ) be a dscrete bvarate radom vector The the fucto f ( xyfrom, ) R to R defed by f ( xy, ) = f, ( xy, ) = P ( = x, = y) s called the jot probablty mass fucto or jot pmf of (, ) I ths case, f ( xymust, ) satsfy ()0 f ( xy, ) ad () R f( x, y) = How do we compute expected value of a fucto g(, ): = ( xy, ) R ( xy, ) Eg(, ) g( x, y) f( x, y) Theorem 46: Let (, ) be a dscrete bvarate radom vector wth jot pmf f, ( x, y ) The the margal pmfs of ad, f ( x) = P( = x) ad f ( y) = P( = y), are gve by f ( x) = R f ( x, y) ad f( y) = R f, ( x, y) y, x
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Example 49 (Same margals, dfferet jot pmf) Defto 40: A fucto f ( xy, ) from cotuous bvarate radom vector (, ) f, for every R to R s called a jot desty fucto or jot pdf of the A R, P((, ) A) f( x, y) dxdy = Defto: The jot cdf of((, ) s defed by F( x, y) = P( x, y) for all (, ) For the cotuous case, F( x, y) f( s, t) dsdt x y = whch by the Fudametal Theorem of Calculus mples A F( x, y) = xy f ( xy, ) Example 4: Let the cotuous radom vector (, ) have jot pdf y e,0 < x< y<, f( x, y) = 0, otherwse y Equvaletly, f ( xy, ) = e I{( uv, ):0 < u< v< } ( xy, ) Fd P ( + ) xy R Chapter 4 Codtoal Dstrbutos ad Idepedece Defto 4: Let (, ) be a dscrete bvarate radom vector wth jot pmf f ( xy, ) ad margal pmfs f ( x ) ad f ( y ) For ay such x such that P ( = x) = f ( x) > 0, the codtoal pmf of gve that = x s the fucto of y deoted by f ( y x ) : f ( y x) = P( = y = x) = f( x, y)/ f ( x) For ay such y such that P ( = y) = f ( y) > 0, the codtoal pmf of gve that = y s the fucto of x deoted by f ( x y ) : f ( x y) = P( = x = y) = f( x, y)/ f ( y)
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Defto 43: Let (, ) be a cotuous bvarate radom vector wth jot pdf f ( xy, ) ad margal pdfs f ( x ) ad f ( y ) For ay such x such that f ( x ) > 0, the codtoal pdf of gve that = x s the fucto of y deoted by f ( y x ) : f ( y x) = f( x, y)/ f ( x) For ay y such that f ( y ) > 0, the codtoal pdf of gve that = y s the fucto of x deoted by f ( x y ) : f ( x y) = f( x, y)/ f ( y) Calculatg Expected Values Usg Codtoal pmfs or pdfs: Let (, ) be a dscrete (cotuous) bvarate radom vector wth jot pmf (pdf) f ( xy, ) ad margal pmfs (pdfs) f ( x ) ad f ( y ), ad g ( ) s a fucto of, the the codtoal expected value of g( ) gve that = x s deoted by E( g ( ) x ) ad s gve by E( g ( ) x) = g( y) f( y x) ad ( E( g ( ) x) = g( y) f( y xdy ) ) Example 44: (cotued) (c) Fd E( = x) ; (d) Fd Var( = x) y Defto 45: Let (, ) be a bvarate radom vector wth jot pdf or pmf f ( xy, ) ad margal pdfs or pmfs f ( x ) ad f ( y ) The ad are called depedet radom varables, f for every x R ad y R, f ( xy, ) = f ( xf ) ( y) Cosequetly, f ad are depedet, f ( y x) = f ( y) ad f ( x y) = f ( x) Techcal Note: If f ( xy, ) s the jot pdf for the cotuous radom vector (, ) where f ( xy, ) f( xf ) ( y) o a set Asuch that f ( x, y ) dxdy= 0, the ad are stll called depedet radom A varables Lemma 47: Let (, ) be a bvarate radom vector wth jot pdf or pmf f ( x, y ) The ad are depedet radom varables f ad oly f there exst fuctos g( x ) ad h( y) such that, for every x R ad y R, f ( xy, ) = gxhy ( ) ( ) I ths case f ( x) = cg( x) ad f ( y) = dh( y), where c ad d are some costats that would make f ( x ) ad f ( y ) vald pdfs or pmfs 3
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 4 y ( x/) Example 48: Cosder the jot pdf f( x, y) = x y e, x > 0 ad y > 0 By Lemma 47, ad are 384 depedet radom varables Notes: Cosder the set {( x, y) : x A ad y B}, where A= { x: f ( x) > 0} ad B= { y: f ( x) > 0}, the ths set s called a cross-product deoted by A B If f ( xy, ) s a jot pdf or pmf such that the set {( x, y) : f( x, y ) > 0} s ot a cross-product, the the radom varables ad wth jot pdf or pmf f ( xy, ) are ot depedet 3 If t s kow that ad are depedet radom varables wth margal pdfs (or pmfs) f ( x ) ad f ( y ), the the jot pdf (or pmf) of ad s gve by f ( xy, ) = f( xf ) ( y) (See Example 49 for dscrete case) Theorem 40: Let ad be depedet radom varables a For ay A R ad B R, P ( A, B) = P ( AP ) ( B) b Let g( x ) be a fucto oly of x ad h( y ) be a fucto oly of y The E( g( ) h( )) = E( g( )) E( h( )) Theorem 4: Let ad be depedet radom varables wth momet geeratg fuctos M () t ad M () t The the momet geeratg fucto of the radom varable Z = + s gve by M Z() t = M () t M() t Example 43: Let ~ N( μ, σ ) ad ~ ( γ, τ ) Fd the pdf of Z = + (Ths s a very mportat result!!) Chapter 43 Bvarate Trasformatos Notes: 4
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 If oly oe fucto s of terest, say U = g ( x, y) I ths case, choose a coveet V (, ) = g so that ( UV, ) s a - trasformato from A oto B ad obta the jot pdf of U ad V gve by fuv, ( uv, ) Fally, oe ca obta the margal of U by tegratg over all values of V What s the trasformato s ot -? We use a geeralzed verso of Theorem 8 from uvarate to may-to-oe fuctos by fdg parttos A0, A,, A of A where the trasformato U = g ( x, y) ad V = g (, ) s - from A ( =,,, ) oto B ad PA ( 0) = 0 Therefore, the jot pdf of U ad V s gve by UV, =, f ( uv, ) = f ( h( uv, ), h ( uv, )) J 3 Eve the trasformato s ot -, we ca always use the method the proof of Theorem of 435 to calculate the jot cdf ad pdf of U ad V Example 436: Show that the dstrbuto of the rato of two depedet ormal varables s a Cauchy radom varable That s, f ~ (0,), ~ (0,), the U = / ~ Cauchy(0) Chapter 44 Herarchcal Models ad Mxture Dstrbutos Examples: Bomal-Posso Herarchy: =umber survved ad =umber of eggs lad The we ca use the models ~ bomal(, p ) ad ~ Posso( λ ) I ths case, ~ Posso( λ p) Posso-Expoetal Herarchy: Let =umber of eggs lad ad Λ =varablty across dfferet mothers Λ ~Posso( Λ ) ad Λ ~expoetal( β ) I ths case, ~ egatve bomal( r =, p= /( + β )) 3 Bomal-Posso-Expoetal Herarchy (Three-stage herarchy): Let =umber survved, =umber of eggs lad ad, Λ = varablty across dfferet mothers ~ bomal(, p ) ad Λ ~ Posso( Λ ) ad 5
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Λ ~expoetal( β ) Equvaletly, we ca look at ths as a two-stage herarchy where ~ bomal(, p ) ad ~ egatve bomal( r =, p= /( + β )) 4 Beta-Bomal Herarchy: P~ bomal( P, ) ad P ~ beta( α, β ) Defto 444: A radom varable s sad to have a mxture dstrbuto f the dstrbuto of depeds o a quatty that also has a dstrbuto IMPORTANT RESULTS: These results are useful whe oe s terested oly the expectato ad varaces of a radom varable Theorem 443: If ad are ay two radom varables, the E = E( E( )), provded that the expectatos exst Theorem 447: For ay two radom varables ad, the Var = E( Var( )) + Var( E( )), provded that the expectatos exst Chapter 45 Covarace ad Correlato Defto 45: The covarace of ad s the umber defed by Cov(, ) = E( E )( E ) Some Notes: < Cov(, ) < Cov(, ) = Cov(, ) 3 Postve (egatve) covarace s whe small values of ted to be observed wth small (large) values of 6
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Defto 45: The correlato of ad s the umber defed by Some Notes: ρ xy ρ Cov(, ) = (correlato coeffcet) σ σ ρ = ad ρ = mply the lear relatoshp betwee ad s perfectly egatve ad postve, respectvely Theorem 453: For ay radom varables ad, Cov(, ) = E μμ Theorem 455: If ad are depedet radom varables, the Cov(, ) = 0 ad ρ = 0 IMPORTANT NOTE: Theorem 454 does ot say that f Cov(, ) = 0 or ρ = 0, the ad are depedet Covarace ad correlato measure oly a partcular kd of lear relatoshp Theorem 456: If ad are ay two radom varables ad a ad b are ay two costats, the Var( a + b ) = a Var + b Var + abcov(, ) If ad are depedet, the Var( a + b ) = a Var + b Var Theorem 457: For ay radom varables ad, a ρ b ρ = f ad oly f there exst umbers a 0 ad b such that P ( = a+ b) = If ρ =, the a > 0, ad f ρ =, the a < 0 Defto 450: Let < μ <, < μ <, σ > 0, σ > 0, ad < ρ < be fve real umbers The bvarate ormal pdf wth meas μ ad μ, varaces σ ad σ, ad correlato ρ s the bvarate pdf gve by x μ x μ y μ y μ f( x, y) = exp ( ) ρ( )( ) + ( ) πσ σ < x < < y < for, ρ ( ρ ) σ σ σ σ, 7
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Alteratvely, we ca re-wrte ths formula a geeral way Defe ρσ σ, whch the varace- covarace of the bvarate ormal dstrbuto, the we have σ Σ= ρσ σ σ Propertes: T f( x, y) = exp (, ) (, ) / x μ y μ Σ x μ y μ ( π ) Σ The margal dstrbutos of ad are ( μ, σ ) ad ( μ, σ ), respectvely The correlato betwee ad s ρ = ρ 3 For ay costats a ad b, the dstrbuto of a + b s a ( μ + bμ, aσ + bσ + abρσ σ ) 4 The codtoal dstrbutos of gve = y ad of gve = x are also ormal dstrbutos The pdf of = y s ( μ + ρσ ( / σ )( y μ ), σ ( ρ )) 5 ad are depedet f ad oly f ρ = 0 Chapter 46 Multvarate Dstrbutos Cosder a radom vector =(,,, ) wth sample space that s a subset of x = ( x, x,, x ) to deote a sample Jot pmf (Dscrete Case): The jot pmf of (,,, ) s defed as f (x) = P ( = x, = x,, = x) for each ( x, x,, x ) R Hece for A R, R I addto, we wrte P ( A) f(x) = x A 8
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Jot pdf (Cotuous Case): The jot pdf of (,,, ) s the fucto that satsfes P ( A ) = f (x) d x = f ( x, x,, x ) dx dx dx Jot cdf: The jot cdf of (,,, ) s defed as Expected Values: A A F(x) = P( x, x,, x ) for each ( x, x,, x) R Eg() g(x) f(x) dx or = Margal dstrbuto of (,,, ): or Eg() = R g(x) f(x) f ( x, x,, x ) f( x,, x ) dx dx, = x k k+ f ( x, x,, x k) = f k ( x (,, ),, x ) x x R k+ Codtoal pdf or pmf of (,,, ): If f( x, x,, x ) > 0, the the codtoal pdf or pmf of (, k+, ) gve,, = x k = xk s f ( x, x x,, x ) f( x, x,, x )/ f( x,, x ) k+, k = k Defto 46: Let m ad be postve tegers ad let p, p,, p be umbers satsfyg 0 p, =,, ad p = = The the radom vector (,,, ) has a multomal dstrbuto wth m trals ad cell probabltes p, p,, p f the jot pmf s (,,, ) m! x x f x x x = p p x! x! 9
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 o the set of ( x,, x ) such that each x s a oegatve teger ad Notes: x = m = m! s kow as the multomal coeffcet whch deotes the umber of ways m objects ca be x! x! dvded to groups wth x the frst, x the secod,, x the th group Bomal s a specal case of the multomal dstrbuto where = Theorem 464 (Multomal Theorem): Let m ad be postve tegers Let A be the set of vectors x = ( x,, x ) such that each x s a oegatve teger ad x = m The for ay real umbers p,, p, ( ) = m! m x x p p p p + + = x A x! x! Results: The margal dstrbuto of s bomal( mp, ) p p The codtoal dstrbuto of,, gve = x s multomal( m x;,, ) p p 3 The covarace of ad j s Cov(, j) = mpp j Defto 465: Let,, be radom vectors wth jot pdf or pmf f (x,,x ) Let f (x ) deote the margal pdf or pmf of The,, are called mutually depedet radom vectors, f for every (x,,x ), f (x,,x) f (x) (x) (x) f f = = = If the are all oe-dmesoal, the,, are called mutually depedet radom varables 0
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Note: Parwse depedece does ot mply mutual depedece Theorem 466: Let,, are mutually depedet radom varables The E( g ( ) g ( )) = E( g ( )) E( g ( )) Theorem 467: Let,, are mutually depedet radom varables wth Z = + + The M () t = M () t M () t Z Example 468: Usg the mgf techque, show that f ~ gamma( α, β ), the Z = + + ~gamma( α + + α, β) Corollary 469: Let,, are mutually depedet radom varables wth ad b,, b fxed costats Defe Z = ( a + b) + + ( a + b) The t b M () t = e M ( at) M ( a t) Z M (), t, M () t Defe M (), t, M () t Let a,, a Corollary 460: Let,, be mutually depedet radom varables wth ~ ( μ, σ ) Let a,, a ad,, b b fxed costats Defe Z = ( a + b) + + ( a + b) The Z ~ ( ( a μ + b ), a σ ) = = Theorem 46: Let,, be radom vectors The Let,, are mutually depedet radom vectors f ad oly f there exst fuctos g(x )( =,, ), such that the jot pdf or pmf of Let,, ca be wrtte f(x,,x ) = g (x ) g (x ) as Theorem 46: Let,, be radom vectors Let g(x )( =,, ) be a fucto oly of x The the radom varables U = g ( )( =,, ) are mutually depedet
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Theorem: Let (,, ) are radom varables wth jot pdf f( x,, x ) Let A s the support set Cosder the ew radom vector U = ( U,, U ), defed by U = g() For a partto A0, A0,, A k of A : P( A 0) = 0 ad o each A, the trasformato s oe-oe from A to B, the we have x j = hj( u,, u)( j =,, ; =,, k) Let J be the Jacoba o A, the the jot pdf of ( U,, U ) s Example 46 ad 463 k f ( u,, u ) = = f ( h ( u,, u ),, h ( u,, u )) J U Chapter 5 Propertes of a Radom Sample Secto 5 Basc Cocepts of Radom Samples Defto 5: The radom varables,, are called a radom sample of sze from the populato f ( x ) f,, are mutually depedet radom varables ad the margal pdf or pmf of each s the same fucto f ( x ) Alteratvely,,, are called depedet ad detcally dstrbuted (d) radom varables wth pdf or pmf f ( x ) The jot pdf or pmf of the radom sample,, s gve by f ( x,, x ) = = f( x) If the pdf or pmf s a member of a parametrc famly, the f( x,, x θ ) = = f( x θ ), where the same value of θ s used for all sce,, are detcally dstrbuted Notes:
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Samplg from the fte populato: d radom varables Samplg from the fte populato: x, x, N o Samplg wth replacemet: d radom varables o Samplg wthout replacemet:,, have the same margal dstrbuto, but ot mutually depedet, they are close to be depedet f N s very large Example 53 (Fte populato model): Secto 5 Sums of Radom Varables from a Radom Sample Defto 5: Let,, be a radom sample of sze from a populato ad let T(,, ) be a realvalued or vector-valued fucto whose doma cludes the sample space of (,, ) The the radom varable or radom vector = T(,, ) s called a statstc The probablty dstrbuto of a statstc s called the samplg dstrbuto of Remarks: A statstc should ot be a fucto of a ukow parameter A statstc s a quatty computed from sample data 3 A statstc s a radom varable 4 A statstc s typcally deoted by captal letters of the Eglsh alphabet as cotrasted to parameters whch are typcally deoted by Greek letters Defto 5: The sample mea s the algorthmetrc average of the values a radom sample It s usually + + deoted by = = = 3
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Defto 53: The sample varace s the statstc defed by S = ( ) = The sample stadard devato s defed by S = S Theorem 54: Let x,, x be ay umbers ad x = ( x + + x)/, the a x a = x x = = m ( ) ( ) = = ( ) s = ( x x ) = x x Lemma 55: Let,, be a radom sample from a populato ad let g( x ) be a fucto such that E( g( )) ad Var( g( )) exst The E( g( )) = E( g( )) ad Var( g( )) = Var( g( )) = Theorem 56: Let,, be a radom sample from a populato wth mea μ ad varace E = μ 3 Var ES = σ / = σ Remark: E = μ s true eve f,, are ot depedet = σ < The Theorem 57: Let,, be a radom sample from a populato wth mgf M () t The M () t = [ M (/ t )] Example: Let,, be a radom sample from a gamma( α, β ) populato The M () t = [ M (/ t )] = ( β / t ) α 4
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Theorem 59: If ad are depedet cotuous radom varables wth pdfs f ( x ) ad f ( y ), the the pdf of Z = + s fz( z) = f( w) f( z w) dw Two mportat famles of dstrbutos: x μ locato-scale famly - f( x μσ, ) = fz( ): σ σ Cosder a radom sample,, ad Z,, Z where = μ + σ Z( =,, ), the = σ Z + μ Therefore, f g ( z ) s the pdf of Z, the has the pdf (a locato-scale famly dstrbuto): Z x μ f ( x) = g ( ) Z σ σ Expoetal famly - f ( x θ) = h ( x ) c ( θ)exp( w ( θ) t ( x )): Theorem 5: Suppose,, s a radom varable from a pdf or pmf f ( x θ ) where s a member of a expoetal famly Defe T, T by k = f ( x k θ) = h ( x ) c ( θ)exp( w ( θ) t ( x )), k j= j = T(,, ) = t ( ), =,, k If the set {( w ( θ ),, w ( θ)), θ Θ} cotas a ope subset of k R, the the dstrbuto of s expoetal famly of the form ft( u,, uk θ) = H( u,, uk)[ c( θ)] exp( w( ) ) = θ u Notes: c( θ ) ad w ( θ ) are the same for f ad f T k ( T,, T k ) a 5
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Ths s ot true for the curved expoetal famly Secto 53 Samplg from the Normal Dstrbuto Theorem 53: Let,, be a radom varable from a = [/( )] ( ) = S a ad b has a S are depedet The ( μσ, / ) dstrbuto ( μ, σ ) dstrbuto, ad let = (/ ) ad = c ( ) S / σ has a ch squared dstrbuto wth degrees of freedom Lemma 53 (Facts about ch squared dstrbuto): We use χ p deote a ch squared radom varable wth p degrees of freedom a If Z s (0,) radom varable, the b If,, are depedet ad p + + p Z ~ χ ~ χ, the + + s ch squared dstrbuted wth degrees of freedom p Lemma 533: Let j ~ ( μ j, σ j), j =,,, depedet For costats a j ad b rj ( j =,, ; =,, kr ; =,, m), where k + m, defe U = a,,,, j j j = k ad V,,, = r = b j rj j r = m = a U ad V r are depedet f ad oly f Cov( U, V r) = 0 Furthermore, r a jb j rjσ = j Cov( U, V ) = 6
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 b The vectors ( U,, U k ) ad ( V,, V m ) are depedet f ad oly f U s depedet of V r for all pars (, r)( =,, kr ; =,, m) Note: Ths result (a) of ths Lemma mples that for two ormal radom varables to be depedet, we oly eed to show that ther covarace s 0 Defto 534 (Studet s T dstrbuto) Let,, be a radom sample from a ( μ, σ ) populato The μ quatty has a Studet s t dstrbuto wth degrees of freedom Equvaletly, a radom varable S/ p + Γ( ) T ~ t p f t has a pdf gve by f () T t =, < t < / ( p+ )/ p Γ( ) ( pπ ) ( + t / p) Notes: If p =, we get a Cauchy(0,) dstrbuto I the radom sample settg, ths happes whe = If T ~ t the oly p momets exst I partcular, p p E( T ) = 0, p> ad Var( T ) = 3 Mgf for Studet s t dstrbuto does ot exst 4 I geeral, f U ~ (0,), ~ p p p p p f p > V χ ad U ad V are depedet, the T = U / V / p ~ t Defto 535: (Sedecor s F dstrbuto amed hoor of Sr Roald Fsher) Let,, be a radom sample from a ( μ, σ ) populato ad,, m be a radom sample from a depedet ( μ, σ ) populato p p 7
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 S / S The radom varable F = has a F dstrbuto wth (umerator degrees of freedom) ad σ / σ (deomator degrees of freedom) m Equvaletly, the radom varable F has the F dstrbuto wth p ad p+ q Γ( ) ( p /) / q degrees of freedom f t has pdf ( ) p p x ff x = ( ),0 < x< ( p+ q)/ p q Γ( ) Γ( ) q [ + p/ q) x] Notes: Kelker (970) showed that as log as the paret populatos have a certa type of symmetry (sphercal symmetry), the the rato the defto wll have a F dstrbuto I geeral, f U ~ p p ad q degrees of freedom χ ad V ~ χ q ad U ad V are depedet, the F, 3 F dstrbuto s commoly used Aalyss of Varace methods 4 If ~ F pq, the / ~ F qp, pq U / p = has a F dstrbuto wth V / q 5 If ~ t q, the ~ F, q Secto 54 Order Statstcs Defto 54: The order statstcs of a radom sample,, are the sample values placed ascedg order They are deoted by (),, ( ), where () ( ) 8
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Defto 54: The otato { b }, whe appearg a subscrpt, s defed to be the umber b rouded to the earest teger the usual way More precsely, f s a teger ad 05 b< + 05, the { b} = Some statstcs whch are fuctos of order statstcs: sample rage: R = ( ) () sample meda: M (( + ) / ),f s odd = ( ( /) + ( /+ ) )/,f s eve 3 (00 p) th sample percetle, s ({ p}) f /( ) < p< 05 ad ( + {( ( p)}) f 05 < p < /( ), whch s the observato such that approxmately p of the observatos are less tha ths observato ad ( p) of the observatos are greater 4 lower quartle, Q L s the 5 th percetle whle upper quartle, Q U s the 75 th percetle 5 terquartle rage, IQR = QU QL Theorem 543: Let,, be a radom sample from a dscrete dstrbuto wth pmf f ( x) = P( = x) = p, where x< x < < x are the possble values of ascedg order Defe P 0 = 0, P = p (,, ) k = Let = (),, ( ) deote the order statstcs from the sample The the cdf of ( j) s ad k k P ( ( j) x) = P ( P) k= j k, k k k k P ( ( j) = x) = [ P ( P) P ( P ) ] k= j k 9
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Theorem 544: Let (),, ( ) deote the order statstcs of a radom sample,,,, from a cotuous populato wth cdf F ( x ) ad pdf f ( x ) The the pdf of ( j) s! f x f x F x F x ( j ) ( j )!( j)! j j ( ) = ( )[ ( )] [ ( )] Example 545: (Uform order statstc pdf) The j th order statstc from uform(0,) sample has a j j ( j+ ) beta( j, j+ ) Cosequetly, E ( ( j) ) = ad Var ( j) = + ( + ) ( + ) Theorem 546: Let (),, ( ) deote the order statstcs of a radom sample,,,, from a cotuous populato wth cdf F ( x ) ad pdf f ( x ) The the jot pdf of () ad ( j) < j, s! f u v f u f v F u F v F u F v ( ) ( j) ( )!( j )!( j)! The jot pdf of (),, ( ) s gve by j j, (, ) = ( ) ( )[ ( )] [ ( ) ( )] [ ( )]! f ( x) f( x), < x< < x; f(),, ( x ( ),, x ) = 0, otherwse for < u < v < Example: (Order statstcs from expoetal from Example 463): Let,, 3, 4 be a radom sample from expoetal() Fd the jot dstrbuto of the order statstcs (), (), (3), (4) Secto 55 Covergece Cocepts Defto 55: A sequece of radom varables,,,, coverges probablty to a radom varable f, for every ε > 0, 0
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 lm P( ε ) = 0, or equvaletly, lm P( < ε ) = Theorem 55 (Weak Law of Large Numbers): Let,,, be d radom varables wth Var = σ < Defe = = The for every ε > 0, lm P( μ < ε ) = Example 553 (Cosstecy of S ) Let,,, be d radom varables wth ad defe S = ( ), ca we prove a WLLN for = Ad thus, a suffcet codto that P E( S σ ) Var( S) ( S σ ε) = ε ε S coverges probablty to E E = μ ad Var = μ ad = σ < S? Usg Chebychev s Iequalty, we have σ s that Var S whe ( ) 0 Theorem 554: Suppose that,, coverges probablty to a radom varable ad that h s cotuous fucto The h ( ), h ( ), coverges probablty to h ( ) Example 555 (Cosstecy of S ) If S s a cosstet estmator of σ, the by Theorem 554, the sample stadard devato S = S s a cosstet estmator of σ Note that E( S ) s a based estmator of σ (Exercse 5) Defto 556: A sequece of radom varables,,,, coverges almost surely to radom varable f, for every ε > 0, P(lm < ε ) = Defto 550: A sequece of radom varables,,,, coverges dstrbuto to a radom varable f lm F ( x) = F ( x) for all pots x where F ( x ) s cotuous
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Theorem 55: If the sequece of radom varables,,,, coverges probablty to, the the sequece coverges dstrbuto to Theorem 553: The sequece of radom varable,,,, coverges probablty to a costat μ f ad oly f the sequece also coverges dstrbuto to μ That s, the statemet lm P( μ > ε ) = 0 for every ε > 0 s equvalet to 0, x < u, lm P ( x) =, x > u Theorem 554 (Cetral Lmt Theorem) Let,, be a sequece of d radom varables whose mgfs exts eghbor of 0 (that s, M () t exsts for t < h, for some postve h ) Let E = μ ad Var = σ > 0 (Both μ ad σ are fte sce the mgf exsts) Defe for ay < x <, = Let G ( x ) deote the cdf of ( μ)/ σ The, x y lm G( x) = exp( ) dy π That s, ( μ)/ σ has a lmtg stadard ormal dstrbuto Theorem 555 (Stroger form of the Cetral Lmt Theorem) Let,, be a sequece of d radom varables wth E = μ ad σ 0 < Var = < Defe ( μ)/ σ The, for ay < x <, = Let G ( x ) deote the cdf of
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 x y lm G( x) = exp( ) dy π That s, ( μ)/ σ has a lmtg stadard ormal dstrbuto Theorem 557 (Slutsky s Theorem) If dstrbuto ad a, a costat, probablty, the a a dstrbuto b + + a dstrbuto Example 558 (Normal approxmato wth estmated varace) Suppose that S ( μ) (0,), but the value of σ s ukow We have see example 553 that f Var( S ) 0, the σ σ probablty By exercse 53, we have σ / probablty By theorem 557, we have ( μ) σ ( μ) = (0,) S σ S Notes (relatoshp betwee several covergeces) coverges almost surely coverges probablty coverges dstrbuto coverges probablty exsts a subsequece that coverges almost surely 3 coverges probablty to a costat coverges dstrbuto to a costat 4 Slutsky s Theorem Defto 550: If a fucto gx ( ) has dervatves of order r, that s, costat a, the Tyalor polyomal of order r about a s S r ( r d g ) ( x) = g( x) exsts, the for ay r dx 3
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 () r g ( a) Tr ( x) = ( x a) = 0! r ( r d Theorem 55 (Taylor) If g ) ( a) = g( x) r x= a exsts, the dx gx ( ) Tr ( x) lmx a = 0 r ( x a) Example 55 (Cotuato of Example 559) Recall that we are terested the propertes of t gt () =, θ = E( ) = p, the g'( t) =, thus t ( t) p Eg ( ( )) Egp ( ( )) = ; p ad p Var g = g p Var = p p = ( p) ( p) ( p) ( ( )) [ '( )] ( ) ( ) Let Example 553 (Approxmate mea ad varace) Suppose s a radom varable wth E( ) = μ 0 If we wat to estmate the mea ad varace of the radom varable g( ), we have E( g( )) g( μ) ad Var( g( )) [ g '( μ)] Var( ) Specfcally, for g( ) = /, we have E(/ ) / μ ad 4 Var(/ ) (/ μ) Var( ) Theorem 554 (Delta Method) Let be a sequece of radom varable that satsfes ( θ ) (0, σ ) dstrbuto For a gve g ad a specfc value of θ, suppose that g '( θ ) exsts ad s ot 0 The g g g dstrbuto ' [ ( ) ( θ)] (0, σ [ ( θ)] ) 4
Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Example 555 (Cotuato of Example 553) Suppose ow that we have the mea of a radom sample 4 the we have ( ) (0,(/ μ) Var( )) μ, 5