Linear Regression Model

Transcription

1 Lnear Regresson Model Dependent Varable - focs of std; want to now how other factors called regressors, "ndependent" varables, eogenos varables, or covarates affect the dependent varable; also called endogenos varable or regressand Eogenos Varables - have no control over the at te of decson; also called characterstc varables, ndependent varables, or regressors Deand Eaple - eogenos varables n theor nclde prces and ncoe, bt gets ore coplcated n real lfe: Hosehold vs. Indvdal - nber and age of ds nflence deand Other Factors - edcaton, relgon, gender, ethnct Sppl - coplcatons arse here too: captal strctre, organzatonal strctre, technolog, locaton Reslt - eprcal wor needs to worr abot all possble eogenos varables Specf Model - f ; where s a coln vector of eogenos varables Specfcaton - econoc theor gets s an approaton of f, bt we never actall now f becase we cant now or collect data on all possble eogenos varables Data - collect saple data to chec f f,..., Proble - well never fnd f that wors for all data e.g., two ndvdals can have sae characterstcs, bt dfferent deand becase of tlt fncton ndvdal taste whch depends on nobserved factors Add Error Ter - f, ; 4 eplanatons & 4 are nttve eplanatons for ndergrads, not ths corse. Unobserved Factors - heterogenet; proble descrbed above of data ponts wth sae characterstc varables resltng n dfferent deand.e., sae elds dfferent. Approaton - wrong fnctonal for; dont reall now f, jst estatng as close as we can. Errors n Data - ether recorded wrong or reported ncorrectl. Average vs. Indvdal - no wa to forecast ndvdal deand so onl focs on average deand for gven characterstcs; ths s "good enogh" for frs and governent becase there worred abot aret/poplaton deand; spl ltpl average deand b nber n poplaton Transton - steps to get fro theoretcal odel to econoetrc odel:. Add Error Ter. Pro Varables - f we cant observe varables fro theoretcal odel we se a pro that s hghl correlated to theoretcal varable t replaces "Health" Eaple - "health stats" s not clear or cold be too sbjectve so se pro varable: # tes sc, # doctor vsts, etc. "Qalt" Eaple - # defects per llon nts of otpt cold be a pro * f *, g,... here and are pro for * and * Econoetrcs - nfcaton of econoc theor, atheatcs, and statstcs Theor - sed to develop the odel and agan to nterpret and dscss the reslts ote: f theres no theoretcal odel, rel on coon sense or nstttonal nowledge echans that generated the data to develop a odel; ths cold lead to a new theor dependng on reslts o get Statstcs/Math - sed to solve the odel and derve the reslts of

2 Lnear Regresson - refers to lnear n paraeters j, not necessarl the regressors eogenos varables Basc Model - was to wrte t: paraeters; observatons or or Y X U Y X U s coln vector of eogenos varables; can nclde sqared ters, nteracton ters, d varables, etc.; when theres ore than one eogenos varable, ths s called ltple regresson X - atr of all observed eogenos varables; each row conssts of a sngle observaton of the eogenos varables for a total of rows; there are colns, one for each eogenos varable j - the th observaton of the j th eogenos varable; eleents of the X atr; note the bacwards notaton here... refers to the row, bt ts the second nber s coln vector of paraeters s ncontrolled factor or error ter Fnctonal For - cold be step fncton, polnoal, log, etc.; whch s best s topc for a ore advanced corse Lnear - ost portant becase all fnctons can be locall approated b a lnear fncton... on-lnear - allows nteractons between varables and polnoal ters ; portant becase Talor Seres allows p to approate an fncton wth a polnoal... n theor, of corse; t cold be too dffclt to nterpret n practce f ts a ver hgh order polnoal 4 Step Fncton - ses d varables to represent a step fncton d, where d f < ; otherwse Log-Lnear - sall sed when left hand sde varables s alwas > ln ln Cobb-Doglas Trans-Log - allows hgher order ters snce an people arge Cobb-Doglas snt realstc for prodcton fnctons ln ln ln ln ln Bldng and Interpretng Model - Sales Ptch - need to sa wh ore nterested n and wh other people shold care Jstfcaton - wh s each nclded n the odel; ost crtcs of eprcal wor coes fro whch varables are sed or not sed n the odel Objectve - estate and ae nferences; want to now how characterstc varables affect the average Interpretaton of Coeffcents - depends on fnctonal for of

3 Deand Eaple -..., where s deand for good for ndvdal ; j s prce of good j j and s ncoe for ndvdal ; when other varables dont change, how does prce of good affect deand? Margnal effect of on... / Warnng - ths snt alwas the case; eaples where t doesn t happen: D P I 4 I... here / I becase of I ter D P I 4 P I... here / P and / I have P I ter Stll OK - can stll answer qeston of argnal effect on deand; ts jst not as sple as a sngle coeffcent Estatng Paraeters Mltple Regresson - statstcal technqe to solate how ndvdal affects wth others nchanged; wors even f are correlated jst not hghl correlated; "hgh" s relatve to saple sze; larger saple allows hgher correlaton; ote: were stll dong lnear regresson; ts called ltple regresson n the general case when theres ore than one regressor 4 Assptons - the frst are essental; the second are to ae coptatons easer and arent reqred nless ore sng a statstcal pacage that ses the. Error Uncorrelated to Conteporar Regressors -..., we want to change whle holdng all other j s constant; we loo at change n and se that to estate... bt onl f doesnt change ether.e., & not correlated; snce & are both fro the th observaton, the asspton deals wth "conteporar" regressors all the j are fro the sae observaton; there are a cople was to wrte ths asspton: E j,..., ; j,..., E,..., a coln vector shown on prevos page Zero Error on Average - f we se a constant ter, we bascall have a coln of s n the X atr.e., j sch that j,..., ; n sch a case, we get the plct asspton that the epected vale of the error ter s zero: E,..., Stronger Condton - soe tetboos start wth E,,..., ; ths s a ch stronger condton as well see n a nte, bt t ples the basc asspton. X and EX/ onsnglar - estate for paraeters: Start wth asspton: E Sbsttte odel : E Mltpl that ot and brea p the epectaton: E E We plled of ot of the epectaton becase ts a constant ow solve for : [ E ] E Ths brngs s to the theoretcal part of asspton ; there are a cople was to wrte t: E s nonsnglar X E s nonsnglar Theres a proble wth sng E : we cant reall get epected vales so we tr to approate sng saple average:

4 4 of... to get we se nstead of : ow we can solve for : Ths brngs s to the practcal part of asspton oll now t doesnt hold f the copter crashes when o tr to estate the paraeters: s nonsnglar X s nonsnglar Meanng - practcall, what ths asspton eans s that no two rows or colns n X can be dentcal or no row or coln n X can be a lnear cobnaton of the other rows or colns... n practcal ters: each eogenos varable st brng soe new nforaton.e., cant jst repeat what other varables tell o Estatng - there are several was to wrte t ot: X Y X X U X X X To get fro estate on left to the one on the rght, we sbsttte for or Y: Brea p the second saton: Mltpl t ot note the nverse cancels n the frst ter: Intton - close to f saple average s close to poplaton average; we now fro statstcs that saple average s consstent and nbased estator of poplaton average Consstent - s consstent.e., P as ; consstent eans that as gets larger, s "ore lel" to be close to Identfcaton Condtons - assptons & cobned are called the dentfcaton or reglart condtons; the garantee that we can calclate and that t s consstent

5 oral Dstrbton - assptons & can be cobned wth other techncal statstcs stff le central lt theore; specfcs not portant to ths corse to sa that: ~, ote: we dd not have to asse ~ oral Varance ter coes fro U and the fact that Var z E z [ E z ] : Var [ U] [ ] U U E E In the rght ter, we can pll X ot of the epectaton becase ts nown fro the saple; wth EU a lttle wor wth the atrces wll show: E E X U E fro asspton E ow we have Var U E [ U] E[ UU X ] X We can pll the X ot of the epectaton and get E UU, so f we let E UU, we get the varance ter we had above If we wor ot the atr stff we can fnd that [ ] E UU E U E E j j j at least thats what I wrote down n class... loos hard to prove. o Atocorrelaton - ths s a ver techncal asspton that s prel statstcal to splf calclatons; t doesnt reall have econoc eanng and we dont need t to get a good estate of to ; bascall ths asspton eans that an two error ters and j are ncorrelated e.g., an two frs or ndvdals dont affect each other; there are several was to wrte ths, each slghtl dfferent; there lsted here fro weaest to strongest a. E j - ths allows s to splf the varance ter: j j E j j E all the ters wth j go awa j b. E, j j c. E,..., and and j are ndependent - ths s the strongest verson sed n classcal lnear regresson; t eans that and are orthogonal; n a graph, t eans that the average of the error ters for each vale of the eogenos varables s zero Unbased - E ; doesnt depend on saple sze; doesnt ean estate s close to tre vale; sngle estate can be ver wrong, bt average of estates wll be close to Start wth Average n each coln s zero 5 of

6 ow tae the epectaton: E E In order for to be nbased, that ter on the rght st eqal zero; ts too coplcated to wor wth so we obvosl se the teratve epectaton rle: EUV E V E U U V E E E,,, s nbased f E,,...,,,...,.e., orthogonal that s, s ncorrelated to all, not jst le we need for to be consstent ot Reqred - an nbased estator s better to have for sall saple szes, bt a consstent estator s better for large saples; large saples are now coon so consstenc s better also allows less restrctve assptons abot data 4. Hoosedastct - to splf the coptatons even frther, the net asspton sas that the error ter has the sae varance for all.e. there are no patterns n the error ters wth respect to... doesnt get bgger or saller E σ whch can also be wrtten σ X Gong bac to the whole pont of splfng these calclatons, loo at the dstrbton now wth assptons & 4: ~, or Varance of n each coln s the sae ~, σ or ~, σ Sar of Assptons - these are added to the assptons that we have the correct odel and that ts lnear n the paraeters:. E,..., - error ncorrelated to conteporar regressors. X and EX/ onsnglar - cobned wth asspton, these are the dentfcaton or reglart condtons.e., we can fnd n theor or n practce ~, E UU. E j - error ters are nrelated to each other 4. & wth soe techncal stat stff sa [ ] j j σ E - hoosedastct error ters have the sae varance & 4 sa ~, σ X X 6 of

7 Testng Paraeters What to Estate - ost copter pacages start wth the for assptons we jst covered so the se ~, σ ; the onl thng we need to estate here s σ Var E... replace wth saple average... Estate - now proble s that we dont now we dont now Regresson Resdals - estate wth e Estate Varance - σ e... and proble there s that Saple Varance - S e, where # of paraeters s ote: both σ and S converge to σ as, bt S gves better estates for sall saple sze; ost pacages se S Chec Assptons - f theres enogh evdence n the data to thn or 4 dont hold, then we cant se the reslts fro the pacage Te Seres - serall correlated whch volates asspton Refresher - heres what were worng wth n order to test the paraeters: Var Cov, Cov, Var S X Cov Cov, Cov, Cov, Cov, Var Standard Error - also called standard devaton; Var t Test - se to loo at sngle restrcton on paraeters Standardzed Paraeter - / Var ~ t ; dstrbted as a t dstrbton wth - degrees of freedo nder the nll hpothess that Hpothess Test - H : H a : Rejecton Regon: f Var > tα,, then reject H If we cant reject we sa a s statstcall nsgnfcant; b regressor has no statstcall sgnfcant effect on ; c theres not enogh evdence n the 7 of

8 data to sggest theres an assocaton between and ; or d the th eogenos varable doesnt provde an sefl nforaton for eplanng the varaton n p-vale - gves level at whch s statstcall nsgnfcant so we dont need to go to tables to get crtcal vale of t; f p <.5 or desred level, reject H Pr t > > α, Var Paraeters Eqal to Other Vales - H : a H a : a Rejecton Regon: a Var > t α, Test Mltple Paraeters - sngle test nvolvng ore than one H : a H a : a Hard Wa - Rejecton Regon: a Var Var Cov, > t α, "Eas" Wa - re-rn the regresson b enforcng the restrcton: a a ~ ~ ~ Use reglar t-test to chec H : on-lnear Restrcton - cant se the eas trc fro before becase ts not lnear n H : a General Case - H : h, where h s a contnos fncton of Delta Method - A h h h ~, Cov Rejecton Regon: h h h Cov 8 of > t α, Proble - rght hand sde regressor a be scaled e.g., per capta GDP n,s of dollars

9 t rato standardzed paraeter wont change, bt delta ethod doesn t wor Wald Test - se to test ltple restrctons on the paraeters; we ght need to do that becase a paraeter a be nsgnfcant ndvdall, bt sgnfcant jontl General - R r Eaples - a H :, R, r H a : or b H :, R H a : or, r R r ~, RCov R Weghted Qadratc Dstance - R r RCov R R r ~ χ # restrctons Prof A doesn t le the Wald Test... too ch wor F Test - se to test ltple restrctons on the paraeters jst le wth the Wald Test; steps:. Start wth orgnal odel. Solve for a dfferent n each of the restrctons sng R a b and. Plg nto the orgnal odel a b 4. Cobne ters a b In general wll have ~ ~ ~ ~ ~ ~ 5. Re-rn regresson Test Statstc - F ~ / ~ F /, As Favorte - "ver good fnte saple propertes" Proble - assptons & 4 no atocorrelaton and hoosedastct; F test doesnt actall se the saple varance, bt t reles on these assptons other tests wll be fne as long as we can copte the new varance 9 of

10 S of Sqares Start wth basc odel: ow sbtract eans: snce ow sqare t: ow add over all : ote: becase of asspton S of Sqared Resdals SSR - ; also called s of sqared error SSE S of Sqared Model SSM - S of Sqared Total SST - Centered R - ; also called s of sqared regresson SSR SSR ; how ch varaton n n % ters s eplaned b the odel; cold SST û, b bad odel wrong fnctonal for or ssng be low becase a large varaton n regressors Cross-Sectonal Data - bg dfference aong ndvdals so theres lots of heterogenet n data; R >. s sall prett good Te Seres - R >.8 Econoc Theor - R s ncreasng fncton of R and decreasng fncton of R ; statstcans bascall add and reove varables to prove R ; econoetrcans dont do that becase econoc theor tells s whch varables to nclde SSR / Adjsted R - ; solves proble of R SST / Uncentered R - ; sed when we dont se a constant ter Reportng Conventon for reportng a paraeter: Varable nae Paraeter Estate Eper.98 **. Sgnfcance Level: ** % * 5% % Std Error or T-rato of