Lecture 18: Samplig distributios I may applicatios, the populatio is oe or several ormal distributios (or approximately). We ow study properties of some importat statistics based o a radom sample from a ormal distributio. If X 1,...,X is a radom sample from N(µ,σ 2 ), the the joit pdf is ( 1 (2π) /2 σ exp 1 2σ 2 (x i µ) ), 2 x i R,i = 1,..., Theorem 5.3.1. Let X 1,...,X be a radom sample from N(µ,σ 2 ) ad let X ad S 2 be the sample mea ad sample variace. The a. X ad S 2 are idepedet radom variables; b. X N(µ,σ 2 /); c. ( 1)S 2 /σ 2 has the chi-square distributio with 1 degrees of freedom. UW-Madiso (Statistics) Stat 609 Lecture 18 2015 1 / 18
Proof. We have already established property b (Chapter 4). To prove property a, it is eough to show the idepedece of Z ad SZ 2, the sample mea ad variace based o Z i = (X i µ)/σ N(0,1), i = 1,...,, because we ca apply Theorem 4.6.12 ad X = σ Z µ ad S 2 = σ 2 1 (Z i Z ) 2 = σ 2 SZ 2 Cosider the trasformatio The Y 1 = Z, Y i = Z i Z, i = 2,...,, Z 1 = Y 1 (Y 2 + + Y ), Z i = Y i + Y 1, i = 2,...,, ad (Z 1,...,Z ) (Y 1,...,Y ) = 1 Sice the joit pdf of Z 1,...,Z is ( ) 1 exp 1 (2π) /2 2 zi 2 z i R,i = 1,...,, UW-Madiso (Statistics) Stat 609 Lecture 18 2015 2 / 18
the joit pdf of (Y 1,...,Y ) is ( exp 1 y (2π) /2 1 2 i=2 ( = (2π) /2 exp ) 2 y 1 2 exp 1 2 ) 2 ( y i exp y 2 i=2 ) 1 2 (y i + y 1 ) 2 i=2 ) 2 y i i + ( i=2 y i R i = 1,...,. Sice the first exp factor ivolves y 1 oly ad the secod exp factor ivolves y 2,...,y, we coclude (Theorem 4.6.11) that Y 1 is idepedet of (Y 2,...,Y ). Sice Z 1 Z = we have i=2 SZ 2 = 1 1 (Z i Z ) = i=2 (Z i Z ) 2 = 1 1 Y i ad Z i Z = Y i, i = 2,...,, ( ) 2 Y i + 1 1 i=2 i=2 Y 2 i UW-Madiso (Statistics) Stat 609 Lecture 18 2015 3 / 18
which is a fuctio of (Y 2,...,Y ). Hece, Z ad SZ 2 are idepedet by Theorem 4.6.12. This proves a. Fially, we prove c (the proof i the textbook ca be simplified). Note that ( 1)S 2 = (X i X) 2 = (X i µ +µ X) 2 = (X i µ) 2 +(µ X) 2 The ( ) 2 X µ ( ) ( 1)S2 Xi µ 2 + σ σ 2 = = σ Zi 2 Sice Z i N(0,1) ad Z 1,...,Z are idepedet, we have previously show that each Zi 2 chi-square with degree of freedom 1, the sum Z i 2 chi-square with degrees of freedom, ad its mgf is (1 2t) /2, t < 1/2, ( X µ)/σ N(0,1) ad hece [( X µ)/σ] 2 chi-square with degree of freedom 1. UW-Madiso (Statistics) Stat 609 Lecture 18 2015 4 / 18
The left had side of the previous expressio is a sum of two idepedet radom variables ad, hece, if f (t) is the mgf of ( 1)S 2 /σ 2, the the mgf of the sum o the left had side is (1 2t) 1/2 f (t) Sice the right had side of the previous expressio has mgf (1 2t) /2, we must have f (t) = (1 2t) /2 /(1 2t) 1/2 = (1 2t) ( 1)/2 t < 1/2 This is the mgf of the chi-square with degrees of freedom 1, ad the result follows. The idepedece of X ad S 2 ca be established i other ways. t-distributio Let X 1,...,X be a radom sample from N(µ,σ 2 ). Usig the result i Chapter 4 about a ratio of idepedet ormal ad chi-square radom variables, the ratio X µ S/ = ( X µ)/(σ/ ) [( 1)S 2 /σ 2 ]/( 1) UW-Madiso (Statistics) Stat 609 Lecture 18 2015 5 / 18
has the cetral t-distributio with 1 degrees of freedom. What is the distributio of T 0 = X µ 0 S/ for a fixed kow costat µ 0 R which is ot ecessarily equal to µ? Note that T is ot a statistic while T 0 is a statistic. Sice X µ 0 N(µ µ 0,σ 2 /), from the discussio i Chapter 4 we kow that the distributio of T 0 is the ocetral t-distributio with degrees of freedom 1 ad ocetrality parameter δ = (µ µ 0 )/σ. F-distributio Let X 1,...,X be a radom sample from N(µ x,σ 2 x ), Y 1,...,Y m be a radom sample from N(µ y,σ 2 y ), X i s ad Y i s be idepedet, ad S 2 x ad S 2 y be the sample variaces based o X i s ad Y i s, respectively. From the previous discussio, ( 1)S 2 x /σ 2 x ad (m 1)S 2 y /σ 2 y are both chi-square distributed, ad the ratio S2 x /σx 2 has the F-distributio Sy 2 /σy 2 with degrees of freedom 1 ad m 1 (deoted by F 1,m 1 ). UW-Madiso (Statistics) Stat 609 Lecture 18 2015 6 / 18
Theorem 5.3.8. Let F p,q deote the F-distributio with degrees of freedom p ad q. a. If X F p,q, the 1/X F q,p. b. If X has the t-distributio with degrees of freedom q, the X 2 F 1,q. c. If X F p.q, the (p/q)x/[1 + (p/q)x] beta(p/2,q/2). Proof. We oly eed to prove c, sice properties a ad b follow directly from the defiitios of F- ad t-distributios. Note that Z = (p/q)x has pdf Γ[(p + q)/2] z p/2 1 Γ(p/2)Γ(q/2) (1 + z) (p+q)/2, z > 0 If u = z/(1 + z), the z = u/(1 u), dz = (1 u) 2 du, ad the pdf of U = Z /(1 + Z ) is = Γ[(p + q)/2] Γ(p/2)Γ(q/2) ( u ) p/2 1 1 1 1 u (1 u) (p+q)/2 (1 u) 2 Γ[(p + q)/2] Γ(p/2)Γ(q/2) up/2 1 (1 u) q/2 1 u > 0 UW-Madiso (Statistics) Stat 609 Lecture 18 2015 7 / 18
Defiitio 5.4.1 (Order statistics). The order statistics of a radom sample of uivariate X 1,...,X are the sample values placed i a o-decreasig order, ad they are deoted by X (1),...,X (). Oce X (1),...,X () are give, the iformatio left i the sample is the particular positios from which X (i) is observed, i = 1,...,. Fuctios of order statistics May useful statistics are fuctios of order statistics. Both sample mea ad variace are fuctios of order statistics, because X i = X (i) ad Xi 2 = X(i) 2 The sample rage R = X () X (1), the distace betwee the smallest ad largest observatios, is a measure of the dispersio i the sample ad should reflect the dispersio i the populatio. UW-Madiso (Statistics) Stat 609 Lecture 18 2015 8 / 18
For ay fixed p (0,1), the (100p)th sample percetile is the observatio such that about p of the observatios are less tha this observatio ad (1 p) of the observatios are greater: X (1) if p (2) 1 X ({p}) if (2) 1 < p < 0.5 X ((+1)/2) if p = 0.5 ad is odd (X (/2) + X (/2+1) )/2 if p = 0.5 ad is eve if 0.5 < p < 1 (2) 1 X (+1 {(1 p)}) X () if p 1 (2) 1 where {b} is the umber b rouded to the earest iteger, i.e., if k is a iteger ad k 0.5 b < k + 0.5, the {b} = k. Other textbooks may defie sample percetiles differetly. The sample media is the 50th sample percetile. It is a measure of locatio, alterative to the sample mea. The sample lower quartile is the 25th sample percetile ad the upper quartile is the 75th sample percetile. The sample mid-rage is defied as V = (X (1) + X () )/2. UW-Madiso (Statistics) Stat 609 Lecture 18 2015 9 / 18
If X 1,...,X is a radom sample of discrete radom variables, the the calculatio of probabilities for the order statistics is maily a coutig task. Theorem 5.4.3. Let X 1,...,X be a radom sample from a discrete distributio with pmf f (x i ) = p i, where x 1 < x 2 < are the possible values of X 1. Defie P 0 = 0, P 1 = p 1,..., P i = p 1 + + p i,... The, for the jth order statistic X (j), Proof. P(X (j) = x i ) = P(X (j) x i ) = k=j k=j ( k ) P k i (1 P i ) k ( ) [Pi k (1 P i ) k Pi 1 k k (1 P i 1) k ] For ay fixed i, let Y be the umber of X 1,...,X that are less tha or equal to x i. UW-Madiso (Statistics) Stat 609 Lecture 18 2015 10 / 18
If the evet {X j x i } is a success", the Y is the umber of successes i trials ad is distributed as biomial(,p i ). The, the result follows from {X (j) x i } = {Y j}, P(X (j) x i ) = P(Y j) = k=j ( k ad P(X (j) = x i ) = P(X (j) x i ) P(X (j) x i 1 ). ) P k i (1 P i ) k If X 1,...,X is a radom sample from a cotiuous populatio with pdf f (x), the P(X (1) < X (2) < < X () ) = 1 i.e., we do ot eed to worry about ties, ad the joit pdf of (X (1),...,X () ) is {!f (x1 ) f (x ) x 1 < x 2 < < x h(x 1,...,x ) = 0 otherwise The! aturally comes ito this formula because, for ay set of values x 1,...,x, there are! equally likely assigmets for these values to X 1,...,X that all yield the same values for the order statistics. UW-Madiso (Statistics) Stat 609 Lecture 18 2015 11 / 18
Theorem 5.4.4. Let X (1),...,X () be the order statistics of a radom sample X 1,...,X from a cotiuous populatio with cdf F ad pdf f. The the pdf of X (j) is Proof. f X(j) (x) =! (j 1)!( j)! [F(x)]j 1 [1 F(x)] j f (x) x R Let Y be the umber of X 1,...,X less tha or equal to x. The, similar to the proof of Theorem 5.4.3, Y biomial(,f(x)), {X (j) x} = {Y j} ad F X(j) (x) = P(X (j) x) = P(Y j) = k=j k=j ( ) [F(x)] k [1 F(x)] k k We ow obtai the pdf of X (j) by differetiatig the cdf F X(j) : f X(j) (x) = d ( ) d dx F X (j) (x) = k dx [F(x)]k [1 F(x)] k UW-Madiso (Statistics) Stat 609 Lecture 18 2015 12 / 18
( ) {k[f(x)] = k 1 [1 F(x)] k ( k)[f(x)] k [1 F(x)] k 1} f (x) k k=j ( ) = j[f(x)] j 1 [1 F(x)] j ( ) f (x) + j l[f(x)] l 1 [1 F(x)] l f (x) l l=j+1 ( ) ( k)[f(x)] k [1 F (x)] k 1 f (x) k 1 k=j! = (j 1)!( j)! [F(x)]j 1 [1 F(x)] j f (x) 1( ) + (k + 1)[F(x)] k [1 F(x)] k 1 f (x) k + 1 k=j ( ) ( k)[f(x)] k [1 F(x)] k 1 f (x) k 1 k=j The result follows from the fact that the last two terms cacel, because ( ) ( )! (k + 1) = k + 1 k!( k 1)! = ( k) k UW-Madiso (Statistics) Stat 609 Lecture 18 2015 13 / 18
Example 5.4.5. Let X 1,...,X be a radom sample from uiform(0,1) so that f (x) = 1 ad F(x) = x for x [0,1]. By Theorem 5.4.4, the pdf of X (j) is! (j 1)!( j)! x j 1 (1 x) j Γ( + 1) = Γ(j)Γ( j + 1) x j 1 (1 x) j+1 1 0 < x < 1 which is the pdf of beta(j, j + 1). Theorem 5.4.6. Let X (1),...,X () be the order statistics of a radom sample X 1,...,X from a cotiuous populatio with cdf F ad pdf f. The the joit pdf of X (i) ad X (j), 1 i < j, is f X(i),X (j) (x,y) =! (i 1)!(j i 1)!( j)! [F(x)]i 1 [F(y) F (x)] j i 1 [1 F(y)] j f (x)f (y) x < y, (x,y) R 2 The proof is left to Exercise 5.26. UW-Madiso (Statistics) Stat 609 Lecture 18 2015 14 / 18
Example 5.4.7. Let X 1,...,X be a radom sample from uiform(0,a), R = X () X (1) be the rage, ad V = (X (1) + X () )/2 be the midrage. We wat to obtai the joit pdf of R ad V as well as the margial distributios of R ad V. By Theorem 5.4.6, the joit pdf of Z = X (1) ad Y = X () is ( 1) ( y f Z,Y (z,y) = a 2 a z ) 2 ( 1)(y z) 2 = a a, 0 < z < y < a Sice R = Y Z ad V = (Y + Z )/2, we obtai Z = V R/2 ad Y = V + R/2, (Z,Y ) (R,V ) = 1 2 1 1 = 1 2 1 The trasformatio from (Z,Y ) to (R,V ) maps the sets {(z,y) : 0 < z < y < a} {(r,v) : 0 < r < a,r/2 < v < a r/2} Obviously 0 < r < a, ad for a fixed r, the smallest value of v is r/2 (whe z = 0 ad y = r) ad the largest value of v is a r/2 (whe z = a r ad y = a). UW-Madiso (Statistics) Stat 609 Lecture 18 2015 15 / 18
Thus, the joit pdf of R ad V is ( 1)r 2 f R,V (r,v) = a, 0 < r < a, r/2 < v < a r/2 The margial pdf of R is f R (r) = a r/2 r/2 ( 1)r 2 The margial pdf of V is f V (v) = = 2v 0 2(a v) 0 a dv = ( 1)r 2 (a r) a, 0 < r < a ( 1)r 2 a dr = (2v) 1 a 0 < v < a/2 ( 1)r 2 (2(a v) 1 a dr = a a/2 < v < a because the set where f R,V (r,v) > 0 is {(r,v) : 0 < r < a,r/2 < v < a r/2} = {(r,v) : 0 < v a/2,0 < r < 2v} {(r,v) : a/2 < v a,0 < r < 2(a v)} UW-Madiso (Statistics) Stat 609 Lecture 18 2015 16 / 18
Example. Let X 1,...,X be a radom sample from uiform(0,1). We wat to fid the distributio of X 1 /X (1). For s > 1, P ( X1 X (1) > s ) = = i=2 ( X1 P ( X1 P = ( 1)P X (1) > s,x (1) = X i ) ) > s,x X (1) = X i (1) ( X1 > s,x X (1) = X (1) = ( 1)P (X 1 > sx,x 2 > X,...,X 1 > X ) = ( 1)P (sx < 1,X 1 > sx,x 2 > X,...,X 1 > X ) [ 1/s ( ] 1 1 1 = ( 1) 0 sx dx i )dx 1 dx i=2 x 1/s = ( 1) (1 x ) 2 (1 sx )dx 0 ) UW-Madiso (Statistics) Stat 609 Lecture 18 2015 17 / 18
Thus, for s > 1, ( ) d ds P X1 s = d X (1) ds [ 1/s 1 ( 1) 1/s = ( 1) = ( 1) = ( 1) 0 1/s 0 1/s 0 ( = 1 1 1 s 0 (1 t) 2 tdt ] (1 t) 2 (1 st)dt 1/s (1 t) 2 tdt ( 1) 0 1/s (1 t) 2 tdt ( 1) 0 ) [ 1 1 ( 1 1 1 s (1 t) 1 dt (1 t) 1 dt ) ] 1 For s 1, obviously ( ) X1 P s = 0 X (1) ( ) d ds P X1 s = 0 X (1) UW-Madiso (Statistics) Stat 609 Lecture 18 2015 18 / 18