Fall 007 Soluton to Mdterm Examnaton STAT 7 Dr. Goel. [0 ponts] For the general lnear model = X + ε, wth uncorrelated errors havng mean zero and varance σ, suppose that the desgn matrx X s not necessarly of full rank. Let ˆ = Xˆ denote the projecton of onto the space spanned by the columns of X, where ˆ s any OLS soluton to the normal equatons XX = X. Fnd the varance-covarance matrx ˆ of ˆ. Furthermore show thattrace( ) = σ rank( X). ˆ For the GLM = X + ε, E[ ε] = 0, Cov[ ε] = ((cov( ε, ε j))) = σ I, the estmator ˆ ˆ = X = XXX ( ) X = P, where P = XXX ( ) X s the orthogonal projecton matrx onto the space spanned by the columns of X, s unquely defned, even f the column rank(x) s not full. Now, P P P PIP. ˆ = Cov( ) = Cov( ) = σ However, snce P s a symmetrc dempotent matrx, = PIP = P = P. ˆ σ σ σ Furthermore, the egen-values λ, =,, L, p, of P are ether 0 or [Egen-values of P are equal to the square of the egen-values of P, and P = P mples that these values must be ether 0 or ]. Now, n p Var( ˆ ) = trace[ ˆ ] = trace[ σ P] = σ trace[ P] = σ λ σ rank( ). = X = =. [50 ponts] Consder a two-way ANOVA model y = α + τ + ε, =,; j =,. j j j The parameters α and τ are unknown. j a. [5 ponts] Suppose that the errorsε j have mean zero, varance σ, and are uncorrelated. Express these observatons nto a general lnear model form = X + ε, where = ( α, α, τ, τ),.e., defne the vector and the matrx X, and the covarance matrx of the error vectorε. Let us stack the data ponts n the vector gven by = (,,, ), and let e = ( ε, ε, ε, ε ),and ß= ( α, α, τ, τ ). 0 0 0 0 = 0 0 0 0 Then the desgn matrx X=. Furthermore, e σ I.
b. [0 ponts] Show that for ths desgn, a parametrc functon cα+ cα + dτ+ dτs estmable f and only f c + c = d + d. The parametrc functon l ß = cα+ cα+ d τ+ d τs estmable, ff a vector t such that l = tx c c d d = ( t + t t + t t + t t + t ). ( ) Therefore, c + c = t + t + t + t = d + d. 3 3 3 Now suppose that c+ c= d+ d holds, then c = d + d c. Now ( c c d d ) d d c c d d ( d c d c ) = ( + ) = + X. c. [5 ponts] Consder the parametrc functonsα α, τ τand α + α + τ + τ. Show that the coeffcent vectors n these parametrc functons form an orthogonal bass of the row space of X. 0 0 Note that these functons can be expressed as Kß,where K = 0 0. Easy to check that KK s a dagonal matrx. Hence the rows of K are orthogonal. Furthermore, rank( K)=3, and X= K. Thus, each row of X can be wrtten as lnear combnatons of rows of K. R X. Therefore, rows of K form an orthogonal bass of the row space [ ] d. [5 ponts] A g-nverse of the matrx XX for the above model s gven below: 0 0 0 0 0 ( XX ) = 0 3 0 3 Fnd the best lnear unbased estmators of three parametrc functons n part (c) above and ther varance covarance matrx. Gven a generalzed nverse of XX, the BLUEs of Kßare gven by ˆ Kß= K[ XX ] X = L,where L =. Note that the three rows of L are orthogonal, and the length of each row vector equals one. Hence the var-cov matrx of Kß ˆ =Σ = σ LL = σ I. K߈ 3
e. [5 ponts] Is the best lnear unbased estmator ofα + τ? Explan No, t s not. Snce α + τs estmable, therefore ts BLUE s unque. It s easy to check that ( 0 0 ) ߈ = (3 + + ). 9 Note that the varance of the BLUE s σ < Var( ). 6 f. [0 ponts] Consder a reduced model for ths problem under the restrcton α α = 0. Fnd the dfference of the ERROR Sum of Squares for the reduced model and the full model. From Part (c) above, ( αˆ αˆ) = ( ),wth Var( αˆ ˆ α) = σ. The handout on Optmzaton of Error SS under Lnear Restrctons on parameter vector, t s known for estmable lnear restrctons, Error SS(Reduced model under the restrcton α α = 0- Error SS(Full model) = σ ( αˆ αˆ )/Var( αˆ αˆ ) = ( αˆ αˆ ). 3. [0 ponts] Assume that the -dmensonal random vector follows the model = µ + ε, =,, 3,, where the errors have mean zero, and gven the scalar c, the varance covarance matrx of ε s gven by c c 0 c 0 c σ. c 0 c 0 c c a) [5 ponts] Fnd all values of c for whch the above matrx s a covarance matrx. For the matrx V above to be a covarance matrx, t must be n.n.d. Thus all ts egenvalues must be non-negatve. Consderng the approprate x parttoned matrces n V, λ c λ c c 0 λ c c 0 V λi =. c λ c λ 0 c c λ 0 c + ci ci λ c ( λ) c ( c λ) ( λ) ( c λ) = c λ = = ( c λ) ( λ) + c ( c λ) ( λ) = = ( λ) c ( λ) ( λ)(( λ) c ).
Thus the roots of the characterstc polynomal V λi = 0 are λ = (wth multplcty ) and - λ= c. Now λ 0 - c 0 c. b) [0 ponts] Fnd the Gauss Markov estmator ˆµ of µ based on the vector. Note that for ths model the desgn matrx s, a column of s. Furthermore, VX = (+ ). c Therefore, the column space of VX s same as the column space of X. Hence the Gauss Markov estmator of µ = OLS of µ = X / XX = =. c) [5 ponts] Fnd the rato of the varances of ˆµ and the OLS of µ. (+ ) c Now Var( ) = σ. Of course, snce the two estmators are same, the rato of ther varances equals. Note that for c=, Var ( ) = 0,and = µ wth Prob.. [5 ponts each] Explan why each of the followng statement s True or False. If you make correct choce, but provde ncorrect explanaton, you wll not receve any credt. a. [True/False] In a general lnear model, = X + ε, let x denote the th column of X, =,, p. The parametrc functon c+ cs estmable f the vectors { x, =,} do not belong to the space spanned by the vectors{ cx cx, x3, L, x p }. False, but somethng close to ths holds. Snce the vectors ( c c ) and ( c c) are orthogonal, one can reparametrze = (,,, L, ) by * = (,,, L, ) 3 p 3 usng = c+ c and = c c. Now solve for, n terms of p, * c c.e., * = c c c c. Assume, wthout loss of generalty, that + c + c =, and substtute for, n the orgnal model n terms of the,. Then frst two columns n the desgn matrx of the reparametrzed model are cx + cx, cx cx. Now s estmable f the frst column of the new matrx, *.e., cx + cx does not belong to the space spanned by the last (p-) columns of the new matrx,.e. cx + cx R[ cx cx, X 3, L, X p ]. [Ths was a home work problem, and dscussed n the class.] The key s to thnk of reparametrzaton. The problem statement sad b. [True/False] If the margnal dstrbuton of X and Xare normal wth means zero and varance, then ther jont dstrbuton must be a bvarate normal dstrbuton.
False, snce the margnals do not determne the jont dstrbuton. c. [True/False] In the sample model = x + ε, =,, L,, wth errors{} ε havng mean zero, varance t σ, and par-wse correlatons ρ, the B.L.U.E. of the parameter s gven by the rato estmator ˆ = / x. = = In ths case, snce the X does not contan the column of s, the GLS and OLS may be dfferent, unless ρ =0. However, snce [( ρ) I + ρj] = ci + cj,where c's are non-zero, XV X = cxx + c ( xx ) and XV = cxy + c ( x )(' y). Thus, the G-M estmator s gven by ther rato. Ths s not equal to the rato estmator (' y)/( x ). d. [True/False] Let c and d be both BLUE of some parametrc functon l. Then c and d must be equal. Ths s false f the functon l. s not estmable, snce 0 l = l ( XX ) X depends on the choce of g-nverse. However, f t s estmable, the BLUE s unque. Thus c = d for all and hence c= d.