Qualfyg Exam Statstcal Theory Problem Solutos August 5. Let X, X,..., X be d uform U(,), <<. (a) Assume a quadratc loss fucto. Calculate the Bayes estmate for the mproper pror π(), << Verfy whether the Bayes estmate s cosstet for. (b) Idetfy (wthout proof) a complete suffcet statstc T. Usg completeess ad suffcecy of T,calculateE(X T)whereX X /. Soluto: (a) The lkelhood s I (X () <)I (X() >) Sce X () > wth probablty for ay, we take ths lkelhood to be The Bayes estmate s I (X () <) X () d X () d X () P, The Bayes estmate s cosstet for. (b) X () s a complete ad suffcet statstc. By completeess ad suffcecy, E(X X () ) Ψ(X () ), where Ψ s the uque fucto wth E (Ψ(X () )) E (X) /. Now, E (X () ). Therefore E(X X + ()) +X ().. Let X,..., X be a sample from the beta dstrbuto β(, ). (a) Fd the MLE of /. Is t ubased? Calculate the formato equalty lower boud ad check whether the MLE acheves the lower boud? (b)fdaubasedestmateof/( + ). Does the ubased estmate acheve the formato equalty lower boud? Soluto: (a) The desty fucto of the beta dstrbuto β(, ) s f(x; ) x, >, <x<. The lkelhood from the samples s L exp{ log X + log l logl log X } log X + log log X ad
The MLE of s the root of the equato log X + /. Therefore MLE / log X ad the MLE of / s log X /. Note E( log X ) E(log X ) (log x)x dx (log x)d(x ) (log x)x + x dx x x dx hece ths MLE s ubased. Moreover, var( log X ) var(log X ) (E(log X ) ) ( ) ad the Fsher formato umber s I() E( log f(x, )) The formato equalty lower boud s ( ( ) ) 4 I() var( log X ) Therefore the MLE acheves the lower boud. (b) Note that E(X) xx dx x dx + x+ therefore X s the ubased estmate of /( + ). Its varace s ( var(x) ( ) ) E(X ) ( ) + + ( +) + ( +)( +) The lower boud s ( ( ) ) + I() ( ) (+) ( +) 4
It s easy to show that ( +) < + ad therefore the lower boud ( +) 4 < ( +)( +) var(x) Remark: For a estmate to atta the lower boud, t s ecessary ad suffcet that there s a lear relato betwee the estmate ad the dervatve of the log lkelhood. Ths shows the frst estmate attas the lower boud but the secod does ot. However (a) requres the calculato of the lower boud. 3. Let X (X,..., X ) be a d sample from a expoetal desty wth mea. Cosder testg H : vs. H : >.LetP(X) your p-value for a approprate test. (a) What s E (P (X))? Derve your aswer explctly. (b) Derve E (P (X)) for. Specfcally, assumg oly oe sample,.e., calculate E (P (X)) as explctly as possble for. (c) Whe there s oly oe sample, s E (P (X)) a decreasg fucto of? I geeral, ca you prove your result for a arbtrary MLR famly? Soluto: (a) Let T be the test statstc. For the expoetal desty, T X. Let the dstrbuto fucto of T be F (t), whch s actually gamma(, ) dstrbuto. The p-value equals F (T ). Sce F (T ) s a stadard uform radom varable uder the ull hypothess (ths ca be easly proved), F (T ) s also uform (, ). Therefore E (P (X)) E( F (T )) /. (b) Let T deote a radom varable wth dstrbuto F ad let T be depedet of T.The E (P (X)) P (T t T t)f (t)dt P (T >T) t s e s/ ds t e t/ Γ() Γ() dt where the desty of gamma(, ) s used. Whe there s oly oe sample, the above tegral s smplfed: E (P (X)) e s/ t + ds e t/ dt e t( + ) dt (c) Whe there s oly oe sample, E (P (X)) s a decreasg fucto of. + Suppose we defe MLR as meag f (t)/f (t) s odecreasg fucto of t, for >. The the expectato of p-value s odecreasg. To prove ths ote that p-value F (T ), 3
where F () s the dstrbuto fucto of T uder ull. Ths s a decreasg fucto of T hece ts expectato s a descreasg fucto of. (Ths s a property of MLR famles. It s gve as a problem Cassella Berger. The proof s otrval. We gve ths problem ust to see f ay oe ca do t. Ths part should carry less tha half of the pots for part (c)). 4. Let X, X be d uform o to +. (a) Show that for ay gve <α<, you ca fd c> such that P {X c<<x + c} α (b) Show that for ɛ postve ad suffcetly small P {X c<<x + c X X ɛ} (c) The statemet (a) s used to assert that X ± c s a ( α)% cofdece terval for. Does the asserto (b) cotradct ths? If your sample observatos are X, X, would you use the cofdece terval (a)? Soluto: (a) Note that P {X C<<X + C} P { C <X <C} P { C <X<C} (*) whch s a cotuous creasg fucto of C ad vares betwee zero (whe C )ad oe (whe C ). Hece oe ca fd C>s.t. the above probablty α, for ay <α<. (b) Note that for the order statstcs X () m(x,x )adx () max(x,x ), X () <X () +,.e. X () X () + (**) If X () X () > ɛ, the (**) mples + X X () X X X ().e. ( ) ( ) X() X () X() X () X.e. ɛ X ɛ Hece for suffcet small ɛ, < ɛ < C the above evet wll mply X < C,.e., (X C, X + C). Hece the codtoal probablty. (c) Say, X () X (). Sce X () X () ( + ) ( )weowmusthave X () +, X (). So we kow for sure X. To say that we have cofdece α<that les our terval s coutertutve. 5. The verse Gaussa dstrbutos have the followg desty fucto, f(y;, σ) (πσ) / y 3/ exp{ (σy) (y ) }, y >, >,σ > 4
wth mea ad varace 3 σ.lety,y,...,y be a d sample draw from f(y :,σ) ad Y,Y,...,Y a d sample draw from f(y :,σ). Assume that µ+α, for, ; ad α + α. (a). Wrte dow the lkelhood fucto ad derve the MLEs for µ, α,α ad σ. (b). Derve the lkelhood rato test statstc (LRTS) for H : α α. Soluto: (a) The log-lkelhood fucto s l c log σ Y (Y (µ + α ) ) σ Takg the drevatves of l wth respect to µ ad α,wehave l µ (Y (µ + α ) ); σ l (Y (µ + α ) ) for,. α σ Let Y.. Y ad Y. Y. Settg l l ad µ α to be zeros ad cosderg that α + α,wehave ˆµ ( + ) ; Y. Y. ˆα ( ) ; Y. y. ˆα ( + ). Y. Y. Takg the dervatve of l wth respect to σ ad settg t to be zero, we have The l σ σ + σ ˆσ Y (Y (µ + α ) ). Y (Y (ˆµ +ˆα ) ) Y Y. Y. (b) Uder H : α α, the log-lkelhood fucto s l c log σ Y (Y µ ), σ ad the MLEs for µ ad σ are µ Y.. ; 5
σ Y (Y µ ) Hece the logarthm of the lkelhood rato s λ l(ˆµ, ˆα, ˆα, ˆσ) l ( µ, σ) log σˆσ Y Y.. log. Y Y. Y. Y Y... 6. Let X,X,...,X deote a sample of depedet observatos geerated from the followg model ( ) ( ) ( ) ( ) X µ λ ε X + F + λ X µ where µ,µ ad λ are ukow parameters ad F, ε ad ε are radom varables depedet ad detcally dstrbuted as N(, ). Let X bethesamplemeaad S ( ) (X X)(X X) S S S S be the sample covarace matrx. (a). Wrte dow the log-lkelhood fucto for µ, µ ad λ. (b). Show that S + S +S ad X are suffcet statstcs for λ, µ ad µ. Soluto: (a) It s easy to see that X follows a bvarate ormal dstrbuto wth mea µ ( µ ) µ ad varace matrx Σ ( λ λ So the log-lkelhood fucto s (b) The verse of Σ s l ca be smpled as follows: ) (λ, λ)+ l c log Σ Σ ε ( ) ( ) +λ λ λ +λ. (X µ) Σ (X µ). ( ) +λ λ +λ λ +λ. l c log Σ tr(σ S) ( X µ) Σ ( X µ) c log( + λ ) ( + λ )(S + S ) λ S +λ ( X µ) Σ ( X µ) c log( + λ ) (S + S S ) S + S +S 4 +λ ( X µ) Σ ( X µ). 6
By the factorazato theorem, the suffcet statstcs for µ ad λ are X ad S +S +S. 7. Let β ( β,..., β p ) T be the set of LSE uder the geeral model E(Y )Xβ p β x Σ Y σ I where Y ad x are d-dmesoal vectors ad x s the th colum of X. LetH deote the hypothess that E(Y ) m β x or β m+... β p Show that ( β,..., β m ) s a set of LSE for (β,..., β m ) uder the hypothess H f ad oly f p x β x m+,..., m Soluto: LetV deote the space spaed by all colums of X, adv V be the space spaed by the frst m colums of X. Moreover, let P be the orthogoal proecto from to R to V,adP be the orthogoal proecto from V to V, ad lastly, let P be the orthogoal proecto from R to V. Clearly, P P P,sofwedeote(ˆβ, ˆβ,..., ˆβ m)be the LSE of (β,β,...,β m ) uder H, the [ m p ] P ˆβ x + ˆβ x m+ m ˆβ x ; sce P ( m ˆβ x )( m ˆβ x ), we coclude: P ( p m+ ˆβ x ) m ( ˆβ β )x. Now ( ˆβ,..., ˆβ m ) s LSE s equvalet to that ( ˆβ,..., ˆβ m )(ˆβ,..., ˆβ m) ad s equvalet to [ p ] P ˆβ x, m+ whchstheequvalettox p m+ ˆβ x for,,...,m; ths fshes the proof. 8. For a multple lear regresso Y Xβ + ɛ, wherex s by p full rak matrx (p <), let the hat matrx H X(X X) X. Moreover, wrte X [W, V ], ad let H be the ew hat matrx of the lear model Y Wβ + ɛ. 7
a). Show that H H H for ay,. b). Show that H H s symmetrc ad sem-defte. c). If the frst colum of X are all s, show that H / for all. Soluto: (a). Lettg x be the -th row of X, otcethath x (X X) x whch defes a er product <x,x >; (a) follows drectly. (b). To abuse the symbol a lttle, use X, V,adW for the spaces spaed by the colum vectors of X, V ad W correspodgly. It s suffcet to show that for ay o-zero vector ξ V,(I H )ξ. Splt ξ η + τ where η W ad τ W, clearly (I H )ξ η ; the last equalty follows from the full rak assumpto of X. (c). Clearly H ad H commute wth each other so H H tself s symmetrc ad dempotet ad thus sem-defte. (d). Let H be the hat matrx wth W be the frst colum of X, theh H /. 8