Multivariate Regression: Estimating & Reporting Certainty, Hypotheses Tests

Transcription

1 Multivariate Regressio: Estimatig & Reportig Certaity, Hypotheses Tests I. The C(N)LRM: A. Core Assumptios:. y=xβ +. E () = 0 3. V () = σ I 4. E ( X) = 0 5. X of full-colum rak. B. Coveiece Assumptios (Uecessary):. X o-stochastic. (Uec. to ubiasedess; relaxed to coditioig o X for variace estimatio.) ( ). ~ MVN, σ 0 I. (& if ot: CLT & asymptopia!) II. Properties of OLS Estimator uder CLRM: A. ( ) A( Xβ + ) ˆ LS LS β b = XX Xy Ay = = β+a Page of

2 B. LS E β ˆ = E β+a = β+ E ( A ) = β +AE = β, if X o-stoch. = β + E( A) = β, if X stoch. & E( X) = 0 C. LS V βˆ = V β+a = AV() A D. NOTES: = ( XX ) X V XXX ( ) = ( XX ) X σ I X( XX ) = σ ( XX ) XX ( XX ) σ ( XX ), if Xo-stoch. = σ E{ ( XX) }, if Xstoch. + + V ( b ) k = f σ, V, R X x, x k ~ k + + ˆ ˆ V b = f ˆ σ, V, R X x x k ~ k k,.. V βls = σ XX ˆ is the Cramer-Rao lower-boud; i.e., βˆ is efficiet; i.e., βˆ is BLUE. LS I fact, if ~ MVN, the βˆ is BUE. Page of LS LS

3 E. If ~ ( MVN, σ ) b LS 0 I, the: = β +A = costat + liear-sum of ormals ( σ ) bls ~ MVN β, ( XX ) ; if ot, the bls = β +A = costat + liear-sum of? A bls ~ MVN β, ( XX ). III. Estimatig σ usig ( σ ) k ee : e= M( Xβ + ) = MXβ + M = 0β + M = M E( ee ) = E ( M) M = E{ MM } = E{ M } { } { ( M) } E{ trace( M )} = E trace = { M ( )} { Mσ I} = trace E = trace { I N} σ { I } { N} = σ trace = trace trace { XXX X } { } = σ { trace XX XX [ ] = σ { trace = σ trace( I ) = σ ( k) ee ee E = σ use se as LS (ubiased) est. σ. k k k Page 3 of

4 ee A. NOTE: k divided by degrees freedom, so is sum of squared (asympt.) ormals, se is (asympt.) B. Therefore, std Wald t-tests & cof. its. by: T bj c0 bj c0 = = se..( b ) s ( XX). { } j e jj ~ ( A) t k χ k k.. { } b t se b b t s XX α α j ± k..( j) = j ± k e ( ) { α}% (asympt.) cof. it. jj 3. Tests of liear restrictios (by Wald strategy): a) For istace, H o : + =. If ull-hypoth. true, the estimate ot far (i std. err. uits) from ull: b) b + b b + b b + b T = = = se.. b+ b Vˆ bˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ + b V b + V b + C b, b ˆ But ote how we could use matrix algebra to geeralize: ~ A t k Page 4 of

5 H o : β = 0, β = r β= q; for istace, if β is (4 ), the: β0 β r [ ] = 0 0 ad β = β β3 rb rb A H : r β = q; eg.., H : r β = T = = ~ t Vˆ( rb ) r Vˆ( b) r Notice how r r plucks Vˆ( b ), Vˆ( b ), Cˆ( b, b ), Cˆ( b, b o o k 4. Test of joit hypotheses (by Wald strategy): Ho: β= q, β = q Rβ = q; for istace, H : β = 0, β = ad β is (4 ), the: o Page 5 of ), just as i scalar! β β 0 R =,, ad β = q = β β3 H : ˆ o Rβ = q C= Rb q V Rb q ( Rb q) is a ratio of chi-squares. "Deomiator" has -k deg. free, ad has -k i its deom. "Numerator" is square ormals. So, C/, or, more geerally, C/J where J=rows( R) is F! J, k ˆ ~ A F= Rb q V Rb q Rb q J F J, k ad J F= ( ) Vˆ Rb q ( ) Rb q ( Rb q) ~ A χ k

6 C. Cofidece Itervals & Cofidece Regios. Recall the simple formula for cofidece itervals: ( α)% cofidece iterval ( ci..) for β: ˆ β ± T se..( ˆ β). Recall/Note, too, -for- correspodece b/w hypothesis level ad the (- )% c.i.: a) c.i. overlaps 0gfail-to-reject; 0 lies outside c.i.greject b) The (- )% c.i. also correspods to set of hypothesized that oe would fail-to-reject at level give estimated b. 3. By latter uderstadig, ca see how might costruct a, β, β (- )% cofidece regio for ( β β ) as set of ( ) that would fail-to-reject at give estimates (, ) b b : b β Vˆ ( b) Cˆ ( b, b) b β J b ˆ β C( b, b) Vˆ ( b) b β F J, k Page 6 of

7 a) Usig F J,-k critical value for from desired (- )% c.i., b) J is the umber of estimates at issue ( here) & so J is also the dimesioality of the resultig cofidece regio. c) Multiply & solve for (, ) 4. They geerally look like this: β β that just-satisfy iequality. 5. Are properties of cof. reg. ituitive to you? a) Ellipsoidal ad cetered o (b,b ). b) Go top-left to bottom-right if C(b,b )<0, ad this will be whe (partial) C(x,x )>0; go from bottom-right to top-left if C(b,b )>0, ad this will be whe (partial) C(x,x )<0. c) Appear thier as C(b,b ) &/or V(b )-V(b ) greater. d) Circular if C(b,b )=0 ad V(b )=V(b ). 6. Short-cut Approximatio: Rectagular regio give by k uivariate (- k )% c.i. s cotais at least ( Σ α k )% : Page 7 of

8 a) Ex: two 95% c.i. s e regio w/ mi. ( )%=90% b) Worse approx. (area too big) the more slated ad thier the actual cofidece regio. Never right area (ultimately, b/c rectagular ot ellipsoidal). D. Measures of Fit ( Goodess of Fit Statistics). Std. Err. Est./Reg. (S.E.E., S.E.R., s.e.e., s.e.r.): ee A A χ k se = se = ote: If ~ MVN, the se ~ k k a) Sometimes also deoted or ˆ σ, w/ or w/o sub e or, but best to reserve ˆe σ for ML est. ad to use the hat & e ot : σ = σ = ee ˆe e b) Notes: () Kida measure of typical or avg error or mistake. (Act ly, measures square root of average squared mistake ) Page 8 of

9 () I same uits as dep var. E.g., if dep var i $, s.e.e. i $. R (3) Not costruct models to mi s e ay more tha to max R.. R : share of the variatio i y explaied (liearly accouted) by the model (X ). ( ˆ ) [ ( ) ] ( ) ( y y) ( ) ( ) ( ( ) ) + = ( y y) ( ye ˆ + e ye) e = ( y y) ( y y) ( y y) [ ] ( ) SSR y y y e y y y e = = = = SST y y y y y y y y e e e e e SSE = = = SST a) R is also the square of the correlatio of y & ŷ, i.e., r yy., ˆ b) χ MVN R F. A A If ~, the ~ χ = 3. Adjusted R, Adj. R, R-bar squared: a) Ca always icrease R just by addig variables. Wat some pealty for lack parsimoy. Commo adj. to R is to replace umerator & deomiator w/ ubiased estimates. Page 9 of

10 R ubiased( SSE) e ( k) s = = = ubiased( SST ) y y Page 0 of ( ) e sy b) Weak pealty. Ca show that addig variable w/ coeff. havig t> icreases R. Alterative adj. s with stroger pealties, based o Iformatio Criterio : A kaike IC, B ayesia /S chwartz IC, c) Although ot directly used (that I m aware), otice that: χ ( k) If ~ MVN, the R ~ = F ( ) A A k χ k, 4. (Log) Likelihood from ML est. is also measure of fit. 5. Use & Abuse of Fit Statistics/Measures: a) Ca use to compare model performace i same sample; use to compare across samples oly w/ great attetio ad care how much V(y) to explai varies across samples. b) At ed, (relative) fit of model more somethig to estimate give a model, tha somethig to model to maximize. E. Degradatio of Fit Strategy for Testig:. Logic: If ull hypothesis were true, the imposig it as true rather tha estimatig its parameters should result i little loss of fit.. Strategy: Measure fit-loss, the determie how that measure or some fuctio of it would be distributed

11 uder the ull hypothesis, so we ca determie how likely this much fit-loss is to have occurred by chace. 3. The chage-i-r or delta-r or R Test: a) Determie how to impose the ull hypothesis. Example: Ho: β3 = β4 = 0 Ho: y= β0 + βx+ βx + H: β3 0 or β4 0 H: y= β0 + βx+ βx + β3x3+ β4x4 + b) Measure loss of explaatory power relative to gap from big-model explaatory power to oe, ad divide each umerator ad deomiator by its degrees of freedom: SSE k SSE SSE SSE Δ R = R R0 = = SST SST SST 0 SSE k SSE k SSE 0 k fit-gap: R = ratio: SST SSE loss of fit: χ k k χ k 0 0 k k k 0 0 k χ ( k ) ( k ) k k Δk free: k k k = = ΔR Δk So: F = ~ ( R ) ( k ) ( A) F Δk, k 4. Tests usig other measures of fit, s e or l(l), also 5. Third logic, Lagrage-Multiplier Tests: if ull hypoth true, the impose it as costrait o max l(l) or mi SSE should ot bid, implyig: Lagrage multipliers, =0, ad Øl(L)/Ø at β ull =0 or ØSSE/Ø at β ull =0. Page of