Refresher course Regression Analysis

Transcription

1 Refresher course Regressio Aalysis Ursia Kuh Swiss Household Pael (SHP), FORS 3.6.9, Uiversity of ausae

2 Aim ad cotet of the course Refresher course o liear regressio What is a regressio? How do we obtai regressio coefficiets? How to iterpret regressio coefficiets? Iferece from sample to populatio of iterest (sigificace tests) Assumptios of liear regressio Cosequeces whe assumptios are violated Regressio with pael data

3 What is a regressio? A regressio is a statistical method for studyig the relatioship betwee a sigle depedet variable ad oe or more idepedet variables. Simplest form: liear relatioship betwee a depedet ad oe idepedet variable for a give set of observatios (bivariate liear regressio) 3

4 iear regressio: fittig a lie Y yi ŷi uit X slope xi ei Y = + x 4

5 yearly icome from employmet umber of years spet i paid work 5

6 yearly icome from employmet a b uit x umber of years spet i paid work Regressio lie: ŷ i = a + bx i = *xi 6

7 yearly icome from employmet umber of years spet i paid work Regressio lie: ŷi = a + bxi = *xi Estimated regressio equatio: y i = a + bx i + e i 7

8 yearly icome from employmet umber of years spet i paid work How to fid the lie? iimise squared errors (Ordiary least squares, OS) 8

9 Iterpretig the (liear) regressio equatio Estimated regressio equatio: y i = a + bx i + e i I the social scieces, a regressio is geerally used to represet a causal process. y represets the depedet variable x is the idepedet variable (also called predictor or regressor) a is the itercept (it represets the predicted value of Y if X is equal zero) b is called the regressio coefficiets ad provide a measure of the effect of the idepedet variable o Y (they measure the slope of the lie) e is the part of y ot explaied by the causal model (residual) ca cosist of Omitted variables easuremet errors Stochastic shock Disturbace 9

10 ultivariate regressio Effect of x holdig all other x s costat Portio of y explaied by x that is ot explaied by the other x s Bivariate model y = a + bx + e i ultivariate model y = a + b x + b x + b 3 x 3 + e i Example: geder wage gap sample: full-time employed, yearly salary betwee ad CHF) bivariate: salary = a + b sex + e i (sex: =male, =female) salary = sex (female) multivariate b costat 45'369 female -9'9 educatio (Ref: compulsory) secodary educatio 9'97 tertiary educatio 3'786 supervisio 7'8 fiacial sector 5'59 umber of years i paid work 79

11 Assumptios for OS-estimatios: coefficiets Assumptios for OS-estimatio (ecessary to calculate slope coefficiets) ) No perfect multicolliearity (Noe of the regressors ca be writte as a liear fuctio of the other regressors) ) E(e) = 3) Noe of the x is correlated with e; Cov(x,e) = (all x s are exogeous, oe of x s is edogeous) If assumptios -3 hold: OS is cosistet, regressio coefficiets (b s) ubiased

12 Edogeeity Reasos for misspecificatio of model Omitted variables easuremet error (i explaatory variables) Simultaeity Noliearity i parameters Detectio of edogeeity Difficult to detect ad correct! Cautio for causal iterpretatio Theory, literature (variable selectio ad iterpretatio)!!!! Correctio for edogeeity Test for oliear relatioship, iteractios iclude oliear terms, iteractios (but: still liearity i parameters!) Omitted variables: istrumetal variables, pael data Simultaeity: Structural equatios modellig, pael data for time orderig Theory, literature (variable selectio ad iterpretatio)!!!!

13 Iferece from liear regressio Iferece from OS-estimatios if radom sample But: OS coefficiets are estimatios Estimated regressio equatio: y i = a + bx i + e i True regressio equatio: y i = α + βx i + ε i True coefficiets (α, β) ukow, true «error term» ukow Distributio of coefficiets (a, b) E( b) = Var( b) β = β β E(β) 3

14 Iferece from liear regressio Var ( b) ( ε i ) ε = β = where = ε ( xi x) Variatio of b ( β ): decreases if icreases x are more spread out Squared residuals decrease Distributio of b Studet t-distributio Depeds o ad umber of x s Normal distributio if is large p E(β) β 4

15 Iferece from liear regressio: testig whether b If β = (i populatio), there is o relatioship betwee x ad y we have to test how likely it is, that β = H : Distributio if β = critical values for coefficiets compare estimated coefficiet with critical value if b. > critical value the b sigificat Critical value for stadardized Normal distributio ad 95% cofidece level:.96 stadardisatio: b stad b = t value = b 5 b

16 Iferece from liear regressio: example yearly icome from employmet umber of years spet i paid work Regressio lie: ŷ i = a + bx i ; example: ŷi = *xi 6

17 Sample =53 Iferece from liear regressio: example Coef. st.e. t P> t [95% Cof. Iterval] years work _cos R :. Sample =787 Coef. St.e. t P> t [95% Cof. Iterval] years work _cos R :.59 7

18 8 Iferece from liear regressio: assumptios Assumptios for iferece: assumptio o error terms Idepedece of error terms, o autocorrelatio: Cov (ε i, ε k ) = for all i,k, i k Costat error variace : Var(ε i )= ε for all i; (Homoscedasticity) Preferetially: e is ormally distributed atrix of error terms ; O k i

19 9 Autocorrelatio Reaso: Nested observatios (e.g. households, schools, time, commuities) stadard errors uderestimated OS, adjust stadard errors O ; k i ; O k i autocorrelatio o autocorrelatio

20 Heteroskedasticity Variace is ot cosistet stadard errors overestimated or uderestimated OS, adjust stadard errors (White stadard errors) Weighted least squares (WS) ; O k i ; O k i Homoskedasticity Heteroskedasticity

21 Agai: assumptios of liear regressio Geeral Cotiuous depedet variable Radom sample Coefficiet estimatio No perfect multicolliearity E(e) = No edogeeity Cov(x,e) = Omitted variables easuremet error Simultaeity Noliearity i parameters Iferece No autocorrelatio Cov (ei, ek)= Costat variace (o heterogeeity) Coefficiets biased (icosistet) Stadard errors of coefficiets biased

22 Pael data: The Swiss Household-Pael (SHP) Sample of ca. 574 households (7799 idividuals) i 999 (SHP I) Sample of ca. 538 households (443 idividuals) i 4 (SHP II) Yearly observatios of the same idividuals Up to observatio poits per idividual But: attritio, gaps betwee waves

23 Structure of pael data Wide data format og data format (perso-period-file) idpers i4empy i5empy i6empy i7empy idpers year iempy

24 OS with pael data OS for cross-sectioal aalysis (oe wave) o particular problem! OS for pooled data (differet years i oe file) Problem: assuptio of idepedece of observatios violated (autocorrelatio) Correct clusterig i error terms (leaves coefficiets uaffected) But: OS is ot the best estimator for pooled data (ot efficiet) yearly icome from OS OS, cluster i se employmet b t b t female -5'55 (-.38) -5'55 (-6.6) secodary educatio '383 (6.58) '383 (9.89) tertiary educatio 3'67 (43.5) 3'67 (.69) supervisio 3'79 (35.48) 3'79 (.46) age i 999 '98 (.43) '98 (.44) age i 999 squared -7 (-3.65) -7 (-7.79) time (999-7) 987 (4.6) 987 (.66) fiacial sector 3'93 (.3) 3'93 (.5) married 8'68 (7.5) 8'68 (9.45) wome*married -9'6 (-.6) -9'6 (-6.7) costat 5'55 (.84) 5'55 (.9) 4

25 How to aalyse pael data? Two differet types of variatio i pael data Variatio withi idividuals Variatio betwee idividuals OS does ot take accout of differece betwee the two types of variatios Take advatage of pael data characteristics! Cotrol for uobservable variables (stable persoal characteristics) reduce bias from omitted variables Fixed Effects odels (oly withi variatio) Radom itercept odels ulti level aalysis /radom effects/ fraility for evet history 5

26 yearly icome from OS OS, cluster i se Radom effects Fixed effects employmet b t b t b t b t female -5'55 (-.38) -5'55 (-6.6) -8'4 (-.8) (dropped) secodary educatio '383 (6.58) '383 (9.89) '8 (.33) 3'35 (.34) tertiary educatio 3'67 (43.5) 3'67 (.69) 8'3 (7.33) 6'53 (.33) supervisio 3'79 (35.48) 3'79 (.46) 3'74 (.6) 53 (.69) age i 999 '98 (.43) '98 (.44) ' (6.33) (dropped) age i 999 squared -7 (-3.65) -7 (-7.79) - (-.5) (dropped) time (999-7) 987 (4.6) 987 (.66) '76 (5.36) '3 (6.7) fiacial sector 3'93 (.3) 3'93 (.5) 7'549 (9.87) '45 (.48) married 8'68 (7.5) 8'68 (9.45) 6'636 (.) 4'48 (5.77) wome*married -9'6 (-.6) -9'6 (-6.7) -7'848 (-7.5) -3'767 (-.44) costat 5'55 (.84) 5'55 (.9) 7'94 (.7) 6'59 (6.77) 6

27 Workig with lagged variables iclude lags of idepedet variables y it = a + b x it + b x it + b 3 x it- + e it Do ot iclude lags of depedet variable o the right had side of the equatio! Causes edogeity y = a + b x + e y t t = a + b x t t t + e t y y t t = a + b x = a + b x t t + b + b y t + e t ( a + b x + e ) + e t t If e t ad e t- are correlated (which is likely because they are residuals of the same perso) Cov(x,e), all b s are likely to be biased But: there are more sophisticated methods which allow icludig lags of the depedet variabler t 7

28 No-liear regressio e.g. logistic regressio P(y=) y = a+ b x + b x + e) + e ( x 8

29 No liear models Depedet variable is ot cotiuous: o liear regressio Dummy variable (e.g. yes-o) ultiomial (e.g. vote for SPS, vote for FDP, vote for SVP, vote for others) Ordial (low educatio, itermediate educatio, higher educatio) Cout variable (umber of visits to the doctor) ogistic Regressio Probit Regressio ultiomal logistic regressio ultiomial probit regressio Ordial regressio Poisso regressio 9

30 Additioal possibilities with pael data ogitudial model for growth: radom itercept ad radom slope odellig evets, duratio Evet history aalysis, survival aalysis (trasitios as depedet variable) Differetiate cohort, time period ad age (two out of three) arkov Chai models 3

31 Thak you! For questios: 3

32 Refereces Itroductio Wooldridge, Jeffrey,, Ecoometric Aalysis of Cross Sectio ad Pael Data, IT Press. Further Readig Camero, A. C., ad Trivedi, P.K. 5. icroecoometrics: ethods ad Applicatios, Cambridge Uiversity Press, sectio V. Verbeek,. 4. A Guide to oder Ecoometrics. (d ed.) Wiley, ch.. Baltagi Badi H. 5. Ecoometric Aalysis of Pael Data, (3rd ed.) Wiley. 3