Y = X + " E [X 0 "] = 0 K E ["" 0 ] = = 2 I N : 2. So, you get an estimated parameter vector ^ OLS = (X 0 X) 1 X 0 Y:

Transcription

1 1 Ecent OLS 1. Consder te model Y = X + " E [X 0 "] = 0 K E ["" 0 ] = = 2 I N : Ts s OLS appyland! OLS s BLUE ere. 2. So, you get an estmated parameter vector ^ OLS = (X 0 X) 1 X 0 Y: 3. You know tat t s te lowest varance estmator, but wat s ts varance? Its bas s E ^OLS = E (X 0 X) 1 X 0 X + (X 0 X) 1 X 0 " = E (X 0 X) 1 X 0 " = (X 0 X) 1 0 K = 0 K Te varance of te estmated parameter vector s te expectaton of te square of te quantty n square brackets: 0 V ^OLS = E ^OLS ^OLS = E (X 0 X) 1 X 0 ""X (X 0 X) 1 = (X 0 X) 1 X 0 E ["" 0 ] X (X 0 X) 1 = (X 0 X) 1 X 0 2 I N X (X 0 X) 1 = 2 (X 0 X) 1 X 0 X (X 0 X) 1 = 2 (X 0 X) 1 4. Precson s good. Low varance sprecson. How do you get a precse estmate? One can tnk about V ^OLS = 2 (X 0 X) 1 n 3 peces: (a) Te varance of " s te varance of Y condtonal on X. Less varaton of Y around te regresson lne yelds greater precson. (b) N s te number of observatons. It sows up, mplctly, nsde X 0 X. Ts s easest to see f X as just one column: n ts case, X 0 X = P N =1 (x ) 2, wc for x drawn from some densty f(x) as an expectaton tat ncreases lnearly wt N. So, V ^OLS goes nversely proportonally wt N. 1

2 (c) X 0 X s related to te covarance matrx te vectors x 0, = 1; :::; N. If eac column of X s mean-zero, ten X 0 X s te covarance matrx of te K columns of X. For ts reason, f X asa lot of varance, ten X 0 X s bgger, so (X 0 X) 1 s smaller, so V ^OLS s smaller and te estmate ^ OLS s more precse. 5. Te only problem ere s tat 2 s not observed. However, we ave a sample analog: te sample resdual e: 6. So ow exactly does e relate to "? e = Y X ^ OLS : e = Y X (X 0 X) 1 X 0 Y = I X (X 0 X) 1 X 0 Y = I X (X 0 X) 1 X 0 X + = X X + = I X (X 0 X) 1 X 0 " I X (X 0 X) 1 X 0 " I X (X 0 X) 1 X 0 " e s a lnear transformaton of ". However, altoug I X (X 0 X) 1 X 0 s an NxN matrx, t s not a full rank matrx: ts columns are related. Indeed, ts NxN wegtng matrx s all drven by te dentty matrx, wc as rank N, and te matrx X, wc only as K columns. Te full matrx I X (X 0 X) 1 X 0 as rank N K. 7. Matrces lke I X (X 0 X) 1 X 0 and X (X 0 X) 1 X 0 are called projecton matrces, and tey come up a lot. (a) for any matrx Z, denote ts projecton matrx P Z = Z(Z 0 Z) 1 Z 0 and ts error projecton as M Z = I Z(Z 0 Z) 1 Z 0 (b) Tese are convenent. We can wrte te OLS estmate of X as X ^ OLS = P X Y; and te OLS resduals Y X ^ OLS as e = M X Y and also, e = M X " 2

3 (c) We say stu lke "Te matrx P X projects X onto Y." (d) Tese matrces ave a few useful propertes:. tey are symmetrc.. tey are dempotent, wc means tey equal ter own square: P Z P Z = P Z, M Z M Z = M Z 8. Compute e 0 e n terms of ": so, because P X as rank K. So, we can use an estmate e = M X " E [e 0 e] = E [" 0 M X M X "] = E [" 0 M X "] = E [" 0 "] E = N 2 K 2 Consequently, E [e 0 e] N K = 2 : ^ 2 = " 0 X (X 0 X) 1 X 0 " e0 e N K 9. Te estmated varance of te OLS estmator s tus gven by ^V ^OLS = ^ 2 (X 0 X) 1 : Now, we can compute te BLUE estmate, and say sometng about ts bas (zero) and ts samplng varablty f we ave spercal dsturbances. 2 NonSpercal Dsturbances 1. In a model f dsturbances satsfy Y = g (X; ) + "; E ["" 0 ] = 2 I N ; we call tem spercal. Independently normal dsturbances are spercal, but te assumpton of ndepenent normalty s muc stronger tan te assumpton tat dsturbances are spercal, because normalty restrcts all products of all powers of all dsturbances. In contrast, te restrcton tat dsturbances are spercal restrcts only te squares of dsturbances and cross-products of dsturbances: 3

4 (a) Te rst mplcaton of spercal dsturbances s E (" ) 2 = 2 ; for all = 1; :::; N, wc usually call omoskedastcty. Homoskedastc dsturbances ave te same varance for all observatons. (b) Te second mplcaton s tat E [" " j ] = 0; for all 6= j. Ts means tat tere are no correlatons n dsturbances across observatons. Ts rules out over-tme correlatons n tme-seres data, and spatal correlatons n cross-sectonal data. 2. OLS s necent f dsturbances are nonspercal. Ts s easy to see by example. Imagne tat we ave a lnear model wt a constant and one regressor (te vector X): were Y = + X + "; E ["] = 0 N E ["" 0 ] = 6= 2 I N = I N Tat s, we ave an envronment were we know tat te rst and last observaton ave a dsturbance term of zero, and all te rest are of te usual knd. (a) Consder a regresson lne tat connects te rst and last data ponts, and gnores all te rest. Ts regresson lne s exactly rgt. Includng oter data n te estmate only adds wrongness. Tus, te best lnear unbased estmator n ts case s te lne connectng te rst and last dots. Consequently, OLS s necent t does not ave te lowest varance. (b) Te pont s tat you want to pay close attenton were te dsturbances ave low varance and not pay muc attenton were te dsturbances ave g varance. (c) Alternatvely, magne tat : = N 1 0 N : 4

5 Here, te notaton K ndcates a K vector of ones. Tus, N 2 0 N 2 s an N 2xN 2 matrx of ones, and 2 N 1 0 N 1 s a matrx lled wt 2. Ts covarance matrx would arse f observatons 1 and N ad ndependent dsturbances wt varance 2, and observatons 2; :::; N 1 ad te same dsturbance term. Not just dsturbance terms drawn from te same dstrbuton, but lterally te same value of " for eac of tose observatons. (d) In ts case, you'd want to treat observatons 2; :::; N 1 as f tey were just one observaton: for example, tey all ad a bg postve dsturbance, you wouldn't want to pull te regresson lne up very muc, because you'd know tat wat seemed lke a lot of postve dsturbances was really just one bg outler. Consequently, snce OLS wouldn't do any groupng lke ts, OLS s not ecent. 3 Generalsed Least Squares 1. Generalsed Least Squares (GLS) s used wen we face a model lke Y = + X + "; E ["] = 0 N E ["" 0 ] = Here, f 6= 2 I N, you ave some form of nonspercal dsturbances: eter eteroskestcty, or correlatons across observatons. 2. We know tat OLS s te ecent estmator gven omoskedastc dsturbances, but wat about te above case? 3. Te trck s to convert ts problem back to a omoskedastc problem. Consder premultplyng Y and X by 1=2 1=2 Y = 1=2 X + 1=2 " Here s a model wt te dsturbance term premultpled by ts werd nverse-matrx-square-root tng. 4. Wat s te mean and varance of ts new transformed dsturbance term? E 1=2 " = 1=2 E ["] = 0 E 1=2 "" 0 1=2 = 1=2 E ["" 0 ] 1=2 = 1=2 1=2 = 1=2 1=2 1=2 1=2 = I N I N = I N (see Kennedy's appendx "All About Varance" for more rules on varance computatons). 5

6 5. So te premultpled model s omoskedastc wt unt varance dsturbances. 6. Gven tat te coecents n te transformed model are te same as tose n te untransformed model, we can estmate tem by usng OLS on te transformed model. 7. Tranformng data by a known varance matrx and ten applyng OLS s called Generalsed Least Squares. 8. We refer to te matrx as te Transformaton Matrx. T = 1=2 9. One suc known varance matrx s tat assocated wt dependent varable data wose elements are group means: eg, average ncome n a country. In ts case, te averages ave known relatve varances: te varance of te mean of sometng goes wt te square root of te sample sze used to compute t. If every country as te same varance n eac observaton t uses to calculate ts average ncome, te averages wll ave varances nversely proportonal to te sample szes used to compute tem. So, n te model were ndexes countres, and eac country computes ts mean o of a sample wt sze S, and te dsturbances are not correlated across countres, te covarance matrx must be = 2 S S S N and, terefore, 2 T = 1 4 p S1 p S p SN 10. Te transformaton matrx wc amounts to multplyng eac Y and eac X by te square root of te sample sze used n eac country. 11. Ts strategy, n wc you premultply eac observaton separately, rater tan premultplyng a wole vector of Y and a wole matrx of X, s approprate wen te covarance matrx s dagonal as t s n te grouped mean data case. Ts strategy s referred to as Wegted Least Squares (WLS). 12. GLS s all great f you know te covarance matrx of te dsturbances, but usually, you don't. A smlar strategy, called Feasble Generalsed Least Squares (FGLS) covers te case were you don't know ts covarance matrx, but you can estmate t

7 13. FGLS uses two steps: (a) Get a consstent estmate ^ of.. A consstent estmate s one wc s asymptotcally unbased and wose varance declnes as te sample sze ncreases.. Not all tngs can be estmated consstently. Examples wll come somewat later. (b) Compute ^T = ^ 1=2, and run GLS. 14. Te Random Eects Model uses FGLS (a) Assume tat E Y t = X t + + " t E [ ] j Xt = E [" t ] j Xt = E [ " js ] j Xt = 0; ( ) 2 j Xt = 2 E (" t ) 2 j Xt = " 2 (Actually, ts s a bt stronger tan wat s needed: you just need ortogonal to X t, but te derng subscrpts makes tat assumpton notatonally cumbersome.) Te fact tat are mean zero no matter wat value X takes s strong. For example, f X ncludes educaton and s meant to capture smartness, we would expect correlaton between tem. We also need te varance of to be ndependent of X. For example, f alf of all people are lazy and lazy people never go to college, ten te varance of would covary postvely wt X observed post-secondary scoolng. (b) Gven te assumpton on, we get were Y t = X t + u t u t = + " t s a composte error term wc satses exogenety, but does not satsfy te spercal error term requrement for ecency of OLS. (c) One could use OLS of Y on X and get unbased consstent estmates of. Te reason s tat te nonspercal error term only urts te ecency of te OLS estmator; t s stll unbased. (d) However, ts approac leaves out mportant nformaton tat could mprove te precson of our estmate. In partcular, we ave assumed tat te composte errors ave a cunk wc s te same for every t for a gven. Tere s a GLS approac to take advantage of ts 7

8 assumpton. If we knew te varance of te terms, 2, and knew te varance of te true dsturbances, ", 2 we could take advantage of ts fact. (e) Under te model, we can compute te covarance of errors of any two observatons: = E [u t u js ] = E[( + " t )( j + " js )] = I[ = j] 2 + I[s = t] 2 " were I[:] s te ndcator functon. Ts covarance matrx s block dagonal, were eac block conssts of te sum of te two varances 2 and 2 " on te dagonal, and just 2 o te dagonal. Tese blocks le on te dagonal of te bg matrx, and te o-dagonal blocks are all zero. (see Green around p 295 for furter exposton). So, as dagonal elements equal to " and wtn-person o-dagonal elements equal to 2 and across-person o-dagonal elements equal to 0. (f) Te GLS transformaton matrx s computed as T = 1=2, wc s te matrx square-root of ts composte error covarance matrx. Ten, FGLS regresses transformed Y on transformed X: T Y = T X + T u E [XT 0 T u] = 0 E [T uu 0 T ] = 1 N Note tat E [XT 0 T u] = 0 because we mposed te strong condtonal mean-ndependence condton above: E [ ] j Xt = E [" t ] j Xt = 0 mples bot E [X 0 u] = 0 and E [XT 0 T u] = 0 (ceck for yourself!). In constrast, te weaker ortogonalty condton E [X 0 u] = 0 does not mply E [XT 0 T u] = 0: (g) Ts GLS approac s only easy to mplement wt a balanced panel n wc eac and every observaton s observed for te same number of perods (so tat T s well-dened). But, even wt an unbalanced panel, you can stll create te block dagonal matrx and nvert t. () FGLS requres a consstent estmate of te two varances. A xed eects model can be run n advance to get estmates of tese varances. Or, one could run OLS and construct an estmate of te error covarance matrx drectly. Eter yelds a consstent estmate. 15. Te trck wt FGLS s tat te covarance matrx as N(N 1)=2 elements (t s symmetrc, so t doesn't ave N xn elements). Tus, t always as more elements tan you ave observatons. So, you cannot estmate te covarance matrx of te dsturbances wtout puttng some structure on t. We'll do ts over and over later on. 8

9 4 Inecent OLS 1. Wat f dsturbances are not spercal? OLS s necent, but so wat? Qut your bellyacn' t stll mnmzes predcton error, t stll forces ortogonalty of dsturbances to regressors, t s stll easy to do, easy to explan, just plan easy. 2. But, wt non-spercal dsturbances, te OLS estmated coecent varance s derent from wen dsturbances are spercal. Consder te model Y = X + " E [X 0 "] = 0 K E ["" 0 ] = 6= 2 I N : Recall tat E ^OLS = E (X 0 X) 1 X 0 X + (X 0 X) 1 X 0 " = E (X 0 X) 1 X 0 " = (X 0 X) 1 0 K = 0 K Te varance of te estmated parameter vector s te expectaton of te square of ts quantty: 0 V ^OLS = E ^OLS ^OLS = E (X 0 X) 1 X 0 ""X (X 0 X) 1 = (X 0 X) 1 X 0 E ["" 0 ] X (X 0 X) 1 = (X 0 X) 1 X 0 X (X 0 X) 1 : If = 2 I N, a par of X 0 X's cancel leavng 2 (X 0 X) 1. not. If not, ten 3. It seems lke you could do sometng lke wt te spercal case to get rd of te bt wt : After all E ["" 0 ] =, so peraps we could just substtute n some errors. For example, we could compute (X 0 X) 1 X 0 ee 0 X (X 0 X) 1 : Unfortunately, snce OLS satses te moment condton X 0 e = 0, ts would result n So, tat's not gonna work. (X 0 X) 1 0 K 0 0 KX (X 0 X) 1 = 0 K 0 0 K: 9

10 4. Te problem for estmatng s te same as wt FGLS: as too many parameters to consstently estmate wtout structure. You mgt tnk tat a model lke tat used for WLS mgt be restrctve enoug: you reduce to just N varance parameters and no o-dagonal terms. Unfortunately, wt N observatons, you cannot estmate N parameters consstently. 5. Te trck ere s to come up wt an estmate of X 0 X. Tere are many strateges, and tey are typcally referred to as 'robust' varance estmates (because tey are robust to nonspercal dsturbances) or as 'sandwc' varance estmates, because you sandwc an estmate X0 dx nsde a par of (X 0 X) 10 's. For te same reason as above, you cannot substtute ee 0 for, because you'd get X0 dx = X 0 ee 0 X = 0. (a) General Heteroskedastc dsturbances. Imagne tat dsturbances are not correlated wt eac oter, but tey don't ave dentcal varances. We use te Ecker-Wte Heterorobust varance estmator: restrct to be a dagonal matrx, construct d X 0 X = X 0 DX were D s a dagonal matrx wt (e ) 2 on te man dagonal. (b) You cannot get a consstent estmate of D, because D as N elements: addng observatons wll not ncrease te precson of te estmate of any element of D. (c) However, X 0 DX s only KxK, wc does not grow n sze wt N. Recall tat asymptotc varance s equal to te varance dvded by N, and t s used because te varance goes to 0 as te sample sze goes to nnty. To talk about varance as te sample sze grows, you ave to reate t by sometng, n ts case N. (Te coce of wat to reate t by underles muc of nonparametrc econometrc teory n some models, you ave to reate by N rased to a power less tan 1). So, asy:v ^OLS = 1 N (X0 X) 1 X 0 X (X 0 X) 1 ; and asy: ^V ^OLS = 1 N (X0 X) 1 d X0 X (X 0 X) 1 = (X 0 X) 1 d X0 X N (X0 X) 1 : Consder a model were X = 1, a column of ones. Xd 0 X N = X d P 0 N DX N = =1 (e ) 2 : N 10 Ten,

11 As N grows, ts tng gets closer and closer to 2.. (d) Spatally correlated dsturbances. Imagne tat wtn groups of observatons, dsturbances are correlated, but across groups, tey are not. We use te clustered varance estmator: restrct to be a block-dagonal matrx, construct d X 0 X = X 0 CX were C s block dagonal, wt elements equal to e e j (or ter average) n te blocks and zero elsewere. 11