17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15

Size: px

Start display at page:

Download "17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15"

Augustus Morgan
5 years ago
Views:

1 17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig oly the defiitio of covergece i distributio without relyig o ay results other tha the fact that 1 + c ) b e c 3) b if c c ad b. I a similar way, we may derive the joit asymptotic distributio of several order statistics, as see i the followig example. Example 17.1 Rage of uiform sample: Let X 1,..., X be iid from Uiform0, 1). Let R = X ) X 1) deote the rage of the sample. What is the asymptotic distributio of R? We begi to aswer this questio by fidig the joit asymptotic distributio of X ), X 1) ), as follows. For certai sequeces k ad l, as yet uspecified, cosider P k X 1) > x ad l 1 X ) ) > y) = P X 1) > x/k ad X ) < 1 y/l ) = P x/k < X 1) < < X ) < 1 y/l ), where we have assumed that k ad l are positive. Sice the probability above is simply the probability that the etire sample is to be foud i the iterval x/k, 1 y/l ), we coclude that as log as 0 < x k < 1 y l < 1, we have P k X 1) > x ad l 1 X ) ) > y) = 1 y x ). l k Expressio 3) ow makes it clear that k = ad l = are sesible choices for k ad l, ad the result is that P X 1) > x ad 1 X ) ) > y) = 1 y x ) as log as 0 < x < 1 y < 1. 33) Notice that coditio 33) will be satisfied for large if ad oly if x ad y are both positive. We coclude that for x > 0, y > 0, P X 1) > x ad 1 X ) ) > y) e x e y. Sice this is the joit distributio of iid stadard expoetial radom variables, say, Y 1 ad Y, we coclude that ) ) X1) L Y1. 1 X ) ) Therefore, applyig the cotiuous fuctio fa, b) = a + b to both sides gives Y 1 X ) + X 1) ) = 1 R ) L Y 1 + Y, ad the sum of idepedet gamma1, 1) variables is gamma, 1). 46

2 Let s cosider a similar example i which the asymptotic joit distributio does ot ivolve idepedet radom variables. Example 17. As i Example 17.1, let X 1,..., X be iid from uiform0,1). We ow cosider the joit asymptotic distributio of X 1) ad X ). Omittig the step i which k ad l are to be foud, sice they would agai both be set to, we proceed as follows: P {1 X 1) ) > x ad 1 X ) ) > y)} = P X 1) < 1 x ad X ) < 1 y ). 34) We cosider two separate cases: If 0 < x < y, the the right had side of 34) is simply P X ) < 1 y/), which coverges to e y. O the other had, if 0 < y < x, the P X 1) < 1 x ad X ) < 1 y ) = P X ) < 1 x ) + P X 1) < 1 x < X ) < 1 y ) = 1 x ) + 1 x ) 1 x ) y e x 1 + x y). What is this joit asymptotic distributio? Suppose that Y 1 ad Y are iid stadard expoetial variables as i the previous example. Cosider the joit distributio of Y 1 ad Y 1 + Y : If 0 < x < y, the O the other had, if 0 < y < x, the P Y 1 + Y > x ad Y 1 > y) = P Y 1 > y) = e y. P Y 1 + Y > x ad Y 1 > y) = P Y 1 > max{y, x Y }) = E e max{y,x Y} Therefore, we coclude that = e y P y > x Y ) + ) 1 X 1) ) L 1 X ) ) x y 0 Y1 + Y Y 1 ). e t x e t dt = e x 1 + x y). Notice that this meas that the asymptotic margial distributio of 1 X 1) ) is gamma,1). Theorem 15 i Ferguso s book is a geeralizatio of Example 17.. Recall that if F is a cotiuous, ivertible cdf ad U is a stadard uiform radom variable, the F 1 U) F. This is easy to prove, sice P {F 1 U) t} = P {U F t)} = F t). We may use this fact i cojuctio with the result of Example 17. as i the followig example. Example 17.3 Suppose X 1,..., X are iid stadard expoetial radom variables. What is the joit asymptotic distributio of X 1), X ) )? Sice the cdf of a stadard expoetial distributio is F t) = 1 e t, whose iverse is F 1 u) = log1 u), clearly { log1 U 1) ), log1 U ) )} D ={X 1), X ) }, where D = meas has the same distributio. Therefore, log{1 U 1) ), 1 U ) )} = { log1 U 1) ) log, log1 U ) ) log } D = {X 1) log, X ) log }. We coclude by the result of Example 17. that ) ) X 1) log L logy1 + Y ), X ) log log Y 1 where Y 1 ad Y are iid stadard expoetial variables. 47

3 Problems Problem 17.1 If X 1,..., X are iid stadard uiform variables, fid the joit asymptotic distributio of {X ), 1 X 1) )}. Hit: To fid a probability such as P a < X ) < X ) < b), cosider the triomial distributio with parameters [; a, b a, 1 b)] ad ote that the probability i questio is the same as the probability that the umbers i the first ad third categories are each 1. Problem 17. Let X 1,..., X be a radom sample from the distributio with cdf F x) = [1 1/x)]I{x > 1}. a) Fid the joit asymptotic distributio of X 1) /, X ) /). b) Fid the asymptotic distributio of X 1) /X ). Hit: I part a), proceed as i Example Problem 17.3 Problem 17.4 If X 1,..., X are iid uiform0,1) variables, prove that X 1) /X ) L uiform0,1). Let X 1,..., X be iid from uiform 0, θ). a) Let M = X 1) + X ) )/. Fid the asymptotic distributio of M θ). b) Compare the asymptotic performace of the three estimators M, X, ad the sample media X by cosiderig their relative efficiecies. c) For {101, 1001, 10001}, geerate 500 samples of size, takig θ = 1. Keep track of M, X, ad X for each sample. Costruct a 3 3 table i which you report the sample variace of each estimator for each value of. Do your simulatio results agree with your theoretical results i part b)? Problem 17.5 Let X 1,..., X be a iid sample from a logistic distributio with cdf F t) = e t/θ /1 + e t/θ ) for all t. a) Fid the asymptotic distributio of X ) X 1). b) Based o part a), costruct a approximate 95% cofidece iterval for θ. Use the fact that the.05 ad.975 quatiles of the stadard expoetial distributio are ad , respectively. c) Simulate 1000 samples of size = 40 with θ =. How may cofidece itervals cotai θ? Hit: zero. I part a), use the fact that log U ) ad log U 1) both coverge i probability to 48

4 18. The multivariate ormal distributio Lehma 5., 5.4 We begi with a desity for a multivariate ormal distributio o R k. However, ote that ot all multivariate ormal distributios have desities cosider the uivariate example of N0, 0). Give a mea vector ξ R k ad a positive defiite k k covariace matrix Σ, X has a multivariate ormal distributio with mea ξ ad covariace Σ, writte X N k ξ, Σ), if its desity o R k is { fx) = C exp 1 } x ξ)t Σ 1 x ξ). 35) I expressio 35), the costat is C = k π k Σ ) 1/, where Σ deotes the determiat of Σ. Because of the assumptio that Σ is positive defiite, a assumptio that we will later relax, Σ is guarateed to be positive. As a special case, cosider the bivariate ormal distributio, where for some σ > 0, τ > 0, ad 1 < ρ < 1 we have ) σ ρστ Σ = ρστ τ ad thus Σ = σ τ 1 ρ ) ad Σ 1 = ) 1 τ ρστ σ τ 1 ρ ) ρστ σ. I this case, of course, X 1 ad X have correlatio ρ ad margial variaces σ ad τ, respectively. To see the bivariate ormal desity really writte out, see Lehma page 87. However, here we will use matrix otatio wheever possible because of its elegace compared to the compoetwise expasios. To defie the multivariate ormal distributio i full geerality, we first cosider the case i which Σ is diagoal, say Σ = D = diagd 1,..., d k ). Of course, if D is a legitimate covariace matrix it must be oegative defiite or positive semidefiite), which meas that all of its eigevalues are oegative. Sice the eigevalues of a diagoal matrix are simply its diagoal elemets, we see immediately that d i 0 for all i. We may ow defie a multivariate ormal distributio with covariace matrix D. Defiitio 18.1 Suppose that D = diagd 1,..., d k ) for oegative real umbers d 1,..., d k. The for ξ R k, the multivariate ormal distributio with mea ξ ad covariace D, deoted N k ξ, D), is the joit distributio of the idepedet radom variables X 1,..., X k, where X i Nξ i, d i ). To defie a multivariate ormal distributio for a geeral covariace matrix Σ, we make use of the fact that ay symmetric matrix may be diagoalized by a orthogoal matrix. We first defie orthogoal, the state the diagoalizability result as a lemma. Defiitio 18. A square matrix Q is orthogoal if Q 1 exists ad is equal to Q t. Lemma 18.1 If A is a symmetric k k matrix, the there exists a orthogoal matrix Q such that QAQ t is diagoal. Note that the diagoal elemets of the matrix QAQ t i the matrix above must be the eigevalues of A. This is easy to prove, sice if λ is a diagoal elemet of QAQ t the it is a eigevalue of QAQ t ad hece there exists a vector x such that QAQ t x = λx, which implies that AQ t x) = λq t x) ad so λ is a eigevalue of A. 49

5 Defiitio 18.3 Suppose Σ is a arbitrary symmetric k k matrix with oegative eigevalues. Let Q be a orthogoal matrix such that QΣQ t is diagoal. The for ξ R k, the multivariate ormal distributio with mea ξ ad covariace Σ, deoted N k ξ, Σ), is the distributio of ξ + Q t Y, where Y N k 0, QΣQ t ). It is ot immediately clear that N k ξ, Σ) is well-defied by Defiitio 18.3, sice it is ot clear that ξ + Q t 1Y ad ξ + Q t Y must have the same distributio wheever Q 1 ΣQ t 1 ad Q ΣQ t are both diagoal for orthoal Q 1 ad Q. However, this is ideed true though we will ot prove it here), so Defiitio 18.3 is a valid defiitio. Now that N k ξ, Σ) is defied, we may state the multivariate versio of the cetral limit theorem: Theorem 18.1 If X 1), X ),... are iid with mea vector µ R k ad covariace Σ where Σ has fiite etries), the X µ) L N k 0, Σ). Example 18.1 Let X 1, X,... be iid with E X i = ξ, Var X i = σ, E X i ξ) 3 = γ, ad Var X i ξ) = τ <. Let S = 1 X i X ). 36) i=1 We have show earlier that S σ ) L N0, τ ). The same fact may be prove usig the multivariate results of Chapter 5 as follows. First, let Y i = X i ξ ad Z i = Yi. We may use the multivariate cetral limit theorem to fid the joit asymptotic distributio of Y ad Z, amely { ) Y Z 0 σ )} )} L σ γ N {0, γ τ. Note that the above result uses the fact that Cov Y 1, Z 1 ) = γ, which is easy to check. We may write S = Z Y ). Therefore, defie the fuctio ga, b) = b a ad ote that this gives ġa, b) = a, 1). To use the delta method, we should evaluate ) ) ) ġ0, σ σ γ ) γ τ ġ0, σ ) t σ γ 0 = 0, 1) γ τ = ġ0, σ ) t = τ 1 We coclude that { g Y Z ) 0 g σ )} = S σ ) L N0, τ ) as foud earlier. Problems Problem 18.1 Suppose X N k ξ, Σ), where Σ is ivertible. Prove that X ξ) t Σ 1 X ξ) χ k. Hit: If Q diagoalizes Σ, say QΣQ t = Λ, let Λ 1/ have the obvious defiitio ad cosider Y t Y, where Y = Λ 1/ ) 1 QX ξ). 50

6 Problem 18. Let X 1, X,... be iid from Nξ, σ ) where ξ 0. Let S be defied as i equatio 36). Fid the asymptotic distributio of the coefficiet of variatio S /X. Problem 18.3 If X ad Y are stadard ormal distributios, the ay mixture of these distributios is trivially stadard ormal sice αf X x) + 1 α)f Y x) is a stadard ormal desity. Is the same true of bivariate ormal distributios? I other words, if X ad Y are bivariate ormal, where each of the margial distributios is stadard ormal that is, X 1, X, Y 1, ad Y are all stadard ormal), is αf X x) + 1 α)f Y x) a bivariate ormal desity? If yes, prove it; if o, provide a couterexample. 51

Since X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain

Since X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the