ECON 3150/4150, Spring term Lecture 1

Transcription

1 ECON 3150/4150, Sprig term Lecture 1 Ragar Nymoe Uiversity of Oslo 15 Jauary / 42

2 Refereces to Lecture 1 ad 2 Hill, Griffiths ad Lim, 4 ed (HGL) Ch 1-1.5; Ch , Bårdse ad Nymoe (BN) Kap 1-2 Kap 3 Referece to ECON 2130 : Om ekel lieær regresjo I by Harald Goldstei A ote about data types i ecoometrics by Erik Bior (posted o the web page) 2 / 42

3 The goal of ecoometrics The Ecoometric Project: Use real world data ad statistical theory to obtai empirical kowledge about relatioships that hold outside the give sample. Statistical iferece is a mai cocept: Geeralizatio of empirical evidece from the data to the populatio Historically, iferece theory was imported from mathematical statistics, ad further developed to icrease its relevace for ecoomic data ad theories. Iferece is ot oly about cofirmatio of ecoomic theory: I priciple equally valuable to reject as to cofirm that a theory holds i the populatio. Ecoometric models are the hallmark of ecoometrics 3 / 42

4 Ecoometrics is a combied disciplie I Several of the itersectios are of iterest, but area 4 represets geuie ecoometric models 4 / 42

5 Ecoometric models I The mai ecoometric models that we lear i this course are regressio models with determiistic ad stochastic (i.e. radom) explaatory variables To begi with, we simplify ad look at the model where there is a sigle explaatory variable which is determiistic (there is o ucertaity about which values the variable takes) We dub this Regressio Model 1 (RM1) Next we explai the statistical theory eeded to establish the regressio model with stochastic explaatory variables, (RM2) The we expad RM2 i differet directios that are importat for the relevace of the model for real world ecoomic data. 5 / 42

6 Ecoometric models II We also explore the limits of the regressio model, but the full itroductio to other ecoometric model tha the regressio model is left for more advaced courses i ecoometrics 6 / 42

7 Data types I Cross sectio: A data set where the variables vary across idividuals i = 1, 2,..., Time series: A data set where the variables vary over T time periods: t = 1,..., T Pael data: Variatio both across idividuals ad over time. There are other distictios betwee data types as well Micro/macro Experimetal/o-experimetal 7 / 42

8 Data types II I this course we will cocetrate o the commo groud betwee cross-sectio ad time series data Will use otatio like (Y i,x i ) i = 1, 2,..., for the most But will use (Y t,x t ) t = 1, 2,..., T whe it is relevat. 8 / 42

9 Ecoometrics courses i our programmes ECON 2130 Statistikk 1 ECON 3145/4150 Itroductory Ecoometrics ECON 4136 Applied Statistics ad Ecoometrics ECON 4160 Ecoometrics-Modellig ad System Estimatio ECON 4130 Statistics 2 ECON 5101/02/03 Advaced coursed i time series, pael data, micro ecoometrics 9 / 42

10 Ecoometric software Stata A itroductio to Stata is give i the Computer classes that start ext week Used i ECON 4136, ad i advaced courses i micro ad pael data OxMetrics PcGive EViews TSP MicroFit RATS Gretl R Used i ECON 4160 (itegrated CC ad semiars) ECON 5101, dyamic ecoometrics 10 / 42

11 Historical ad methodological backgroud The sigular cotributio to moder ecoometrics is The Probability Approach to Ecoometrics by Trygve Haavelmo from / 42

12 Historical ad methodological backgroud The sigular cotributio to moder ecoometrics is The Probability Approach to Ecoometrics by Trygve Haavelmo from / 42

13 Ecoometric models ad regressio Ecoometric models are the cojuctio that Haavelmo spoke of. As metioed: I this course, regressio models will be cetral But we start with a review of regressio as a method of fittig a straight lie to the observed data poits To keep it apart from regressio models, we call this mere curve fittig, or regressio without statistical iferece. For those who took Statistikk 1, a very good referece for review is Harald Goldstei s Om ekel lieær regresjo I 13 / 42

14 Basic ideas Scatter plot ad least squares fit Y X 14 / 42

15 Basic ideas Y Which lie is best? Idea: Miimize sum of squared errors! But which errors? X 15 / 42

16 Basic ideas Which squared error? Y x ( X i, Y i ) 1: Least vertical distace to lie 2. Least horizotal 3. Shortest distace to lie X X i 16 / 42

17 Basic ideas ^Y i Y i Y x ( X i, Y i ) X i X Choose 1 whe wat to miimize squared errors from predictig Y i liearly from X i Residual: ˆε i = Y i Ŷ i, where Ŷ i is predicted value 17 / 42

18 Y Itroductio Fidig the best fit by regressio Residuals ad R-sq Regressio ad causality Basic ideas Regressio lie ad predictio errors (projectios) X 18 / 42

19 Least squares algebra Ordiary least squares (OLS) estimates I The differet lies that we cosidered placig i the scatter-plot correspod to differet values of the parameter β 0 ad β 1 i the liear fuctio that coects give umbers X 1,X 2,... X with Y1 fitted,y2 fitted,..., Y fitted : Y fitted i = β 0 β 1 X i, i = 1, 2,..., We obtai the best fit Y fitted i Ŷ i (i = 1, 2,..., ) Ŷ i = ˆβ 0 ˆβ 1 X i, i = 1, 2,..., (1) by fidig the estimates of β 0 ad β 1 that miimizes the sum of squared residuals ( Yi Yi fitted ) 2: S(β 0,β 2 ) = (Y i β 0 β 1 X i ) 2 (2) 19 / 42

20 Least squares algebra Ordiary least squares (OLS) estimates II where Cosequetly ˆβ 0 ad ˆβ 1 are determied by the 1oc s: Y ˆβ 0 ˆβ 1 X = 0 (3) X i Y i ˆβ 0 X i ˆβ 1 Xi 2 = 0 (4) X = 1 is the sample mea (empirical mea) of X. X i (5) It is expected that you ca solve the simultaeous equatio system (3)-(4). Work with exercises to Semiar 1! 20 / 42

21 Least squares algebra A trick ad a simplified derivatio I The trick is to ote that β 0 + β 1 X i α + β 1 (X i X ) (6) whe the itercept parameter α is defied as α β 0 + β 1 X (7) This meas that the best predictio Ŷ i give X i ca be writte as Ŷ i = ˆβ 0 + ˆβ 1 X i ˆα + ˆβ 1 (X i X ) where ˆα ˆβ 0 + ˆβ 1 X (8) ad we therefore choose the α ad β 1 that miimize S(α,β 1 ) = [Y i α β 1 (X i X )] 2 (9) 21 / 42

22 Least squares algebra A trick ad a simplified derivatio II Calculate the two partial derivatives ( kjereregele for each elemet i the sums): S(α,β 1 ) α = 2 S(α,β 1 ) = 2 β 1 [Y i α β 1 (X i X )] ( 1) ad choose ˆα ad ˆβ 1 as the solutios of 2 2 [Y i α β 1 (X i X )] (X i X ) [ Yi ˆα ˆβ 1 (X i X ) ] ( 1) = 0 (10) [ Yi ˆα ˆβ 1 (X i X ) ] (X i X ) = 0 (11) 22 / 42

23 Least squares algebra A trick ad a simplified derivatio III ˆα Ȳ = 0 (12) where i X )Y i ˆβ 1 (X i X ) (X 2 = 0 (13) Ȳ = 1 Y i (14) the empirical mea of Y. DIY exercise 1: Show that (10) gives (12), ad (11) gives (13) ad that the solutios of (12) ad (13) are ˆα = Ȳ, (15) ˆβ 1 = (X i X )Y i (X i X ) 2 (16) 23 / 42

24 Least squares algebra A trick ad a simplified derivatio IV Note that for (16) to make sese, we eed to assume (X i X ) 2 > 0 (i.e., X is a variable, ot a costat) A geeralizatio of this will be importat later, ad is the called absece of perfect multicolliearity. To obtai ˆβ 0 we simply use ˆβ 0 = ˆα ˆβ 1 X (17) 24 / 42

25 Residuals ad total sum of squares I Defiitio of OLS residuals: ˆε i = Y i Ŷ i, i = 1, 2,..., (18) Usig this defiitio i the 1oc s (10) ad (13) gives ˆε i = 0 (19) ˆε i (X i X ) = 0. (20) 25 / 42

26 Residuals ad total sum of squares II ˆε i = 0 = ε = 1 ˆε i (X i X ) = 0 = ˆσ εx = 1 ˆε i = 0 (21) (ˆε i ε)(x i X ) = 0 (22) where ˆσ εx deotes the (empirical) covariace betwee the residuals ad the explaatory variable. These properties always hold whe we iclude the itercept (β 0 or α) i the model They geeralize to the case of multiple regressio as we shall later (22) is a orthogoality coditio. It says that the OLS residuals are ucorrelated with the explaatory variable. 26 / 42

27 Residuals ad total sum of squares III ˆσ εx = 0 occurs because we have defied the OLS residuals i such a way that they measure what is left uexplaied i Y whe we have extract all the explaatory power of X 27 / 42

28 Total Sum of Squares ad Residual Sum of Squares I We defie the Total Sum of Squares for Y as TSS = (Y i Ȳ ) 2 (23) We ca guess that TSS ca be split i Explaied Sum of Squares ESS = ad Residual Sum of Squares RSS = (Ŷ i Ŷ ) 2 (24) (ˆε i ε) 2 (25) 28 / 42

29 Total Sum of Squares ad Residual Sum of Squares II so that TSS = ESS + RSS (26) To show this importat decompositio, start with (Y i Ȳ ) 2 = (Y i Ŷ i ) + (Ŷ }{{} i Ŷ ) ˆε i where we have used that Ȳ = 1 Y i = 1 (ˆε i + Ŷ i ) = Ŷ because of (19). Completig the square gives 2 29 / 42

30 Total Sum of Squares ad Residual Sum of Squares III (Y i Ȳ ) 2 = RSS + 2 ˆε i (Ŷ i Ŷ ) + ESS } {{ } TSS Expad the middle term: ˆε i (Ŷ i Ŷ ) = ˆε i (ˆα + ˆβ 1 (X i X ) Ŷ ) = ˆα ˆε i + ˆβ 1 ˆε i (X i X ) Ŷ }{{}}{{} (19) (20) ˆε i }{{} (19) 30 / 42

31 Total Sum of Squares ad Residual Sum of Squares IV Therefore ˆε i (Ŷ i Ŷ ) = 0 The residuals are ucorrelated with the predictios Ŷ i. Could it be differet? Hece we have the desired result: TSS = ESS + RSS (27) 31 / 42

32 The coefficiet of determiatio I To summarize the goodess of fit i the form of a sigle umber, the coefficiet of determiatio, almost everywhere deoted R 2, is used: R 2 = ESS TSS RSS = = 1 RSS TSS TSS TSS = 1 rate of uexplaied Y variatio (28) If ˆβ 1 = 0, RSS = (Y i Ŷ i ε) 2 = (Y i ˆα) 2 = (Y i Ȳ ) 2 = TSS. ad R 2 = 0 If RSS = 0,a perfect fit, the R 2 = 1 32 / 42

33 The coefficiet of determiatio II Hece we have the property 0 R 2 1 (29) These results deped o defiig the regressio fuctio as as i (1). If we istead use Ŷ i = ˆβ 0 ˆβ 1 X i, Ŷ o i i = ˆβ o i 1 X i which forces the regressio lie trough the origi: the correspodig residuals do ot sum to zero, the decompositio of TSS breaks dow. R 2 (as defied above) ca be egative! Work with Exercises to Semiar 1 to lear more! 33 / 42

34 Regressio ad correlatio I We defie the empirical correlatio coefficiet betwee X ad Y as r X,Y 1 1 (Y i Y )(X i X ) = (X i X ) 2 1 (Y i Y ) 2 ˆσ X,Y ˆσ X ˆσ Y, (30) ˆσ X,Y deotes the empirical covariace betwee Y ad X ˆσ X ad ˆσ Y deote the two empirical stadard deviatios They are square roots of the empirical variaces, e.g., ˆσ X = ˆσ 2X = 1/ (X i X ) 2 34 / 42

35 Regressio ad correlatio II ˆσ X,Y ca be writte i three equivalet ways: ˆσ X,Y = 1 i X )(Y i Ȳ ) = (X 1 = 1 (Y i Ȳ )X i (X i X )Y i The regressio coefficiet ca therefore be re-expressed as ˆβ 1 = 1 (X i X )Y i (X i X ) 2 = (X i X )Y i 1 (X i X ) = ˆσ X,Y 2 ˆσ X 2 = ˆσ Y ˆσ X ˆσ X,Y ˆσ X ˆσ Y = ˆσ Y ˆσ X r X,Y (31) This shows that 35 / 42

36 Regressio ad correlatio III r X,Y = 0 is ecessary for ˆβ 1 = 0. Correlatio is ecessary for fidig regressio relatioships Still, ˆβ 1 = r XY (i geeral) ad regressio aalysis is differet from correlatio aalysis. 36 / 42

37 Regressio ad causality I Three possible theoretical causal relatioships betwee X ad Y. Our regressio is causal if I is true, ad II (joit causality) ad III are ot true r XY = 0 i all three cases Ca also be that a third variable (Z) causes both Y ad X (spurious correlatio) 37 / 42

38 Causal iterpretatio of regressio aalysis I Regressio aalysis ca refute a causal relatioship, sice correlatio is ecessary for causality But caot cofirm or discover a causal relatioship by regressio aalysis aloe However, regressio aalysis is usually doe with referece to a coceptual framework (a theory) that poits out oe directio of causality as more likely or relevat tha others. For example, whe we choose X as the regressor ad Y as the regressad, that choice is usually doe with referece to a theory that idepedet chages i X cause a respose i Y. If that theory is supported by other evidece, that stregthes the relevace (belief i) a causal iterpretatio of our regressio (combied disciplie!) 38 / 42

39 Causal iterpretatio of regressio aalysis II Causality represets the sought after holy grail i may ecoometric studies, ad recetly big advaces has bee made i studies that utilize atural experimets, ofte with the use of large micro data set. I time series, there are also possibilities of ivestigatig causality further. The cetral cocept is autoomy with respect to structural breaks. We will look at simple example 39 / 42

40 Cosider a sample with time series data for X t ad Y t (t = 1, 2,... T ) Time series data has a atural orderig of observatios, from past to preset, ad this is helpful i causality aalysis Start with recordig the two possible regressio coefficiets: Regress Y o X ˆβ 1 = T t=1 (X t X )Y t T t=1 (X t X ) 2 Regress X o Y : ˆβ 1 = T t=1 (X t X )Y t t=1 (Y t Ȳ ) 2 40 / 42

41 Therefore: ˆβ 1 ˆβ 1 = r 2 XY (32) Let us ow sharpe the otio of causality by demadig that a causal relatioship i characterized by parameters that are ivariat to structural breaks i the sample (t = 1, 2,... T ). A cadidate for a structural break is the correlatio coefficiet r XY. (32) says that if there is a break i r XY the at least oe of ˆβ 1 ad ˆβ 1 must chage at the same poit i time. However if either ˆβ 1 or ˆβ 1s are costat despite the break i r XY, we see that the ivariace property is validated, ad that we have evidece of a oe-way causal relatioship. 41 / 42

42 A example of ivariace testig I Regressio coeff for X o Y Regressio coeff for Y o X The graphs show recursive estimates of ˆβ 1 ad ˆβ 1 There is a structural break i period 50 (a higher σ X that reduces r XY ) ˆβ 1 is ot ivariat to the break, but ˆβ 1 is ivariat, supportig that X Y 42 / 42