Econometrics II. Andrea Beccarini. Winter2011/2012

Transcription

1 Econometrics II Andrea Beccarini Winter2011/2012 1

2 Introduction Econometrics: application of statistical methods to empirical research in economics Compare theory with facts (data) Statistics: foundation of econometrics 2

3 Module Statistics and Module Empirical Methods Descriptive statistics (Statistik I) How to process data? How to display data? Probability theory and statistical inference (Statistik II) Estimation of unknown parameters from random samples; hypothesis tests Empirical research in economics (Empirische Wirtschaftsforschung) Applications of the linear model; statistical software 3

4 Module Statistics/Econometrics/Empirical Economics I Advanced Statistics Probability theory; multidimensional random variables; estimation and hypothesis testing Econometrics I Simple and multiple linear regression model Econometrics II Extensions of the multivariate linear regression model; simultaneous equation systems; dynamic models 4

5 Module Statistics/Econometrics/Empirical Economics II Time series analysis: Stochastic processes; stationarity; ergodicity; linear processes; unit root processes; cointegration; vector-autoregressive models One further special course or seminar, e.g. Financial econometrics Panel data econometrics Introduction to R Poverty and inequality Statistical inference, bootstrap Wage and earnings dynamics 5

6 Literature: Statistical basics Karl Mosler and Friedrich Schmid, Wahrscheinlichkeitsrechnung and schließende Statistik, 2. Au., Springer, Aris Spanos, Statistical Foundations of Econometric Modelling, Cambridge University Press, Mood, A.M., Graybill, F.A. and D.C. Boes (1974). Introduction to the theory of statistics, 3rd ed., McGraw-Hill, Tokyo. 6

7 Literature: Econometrics Main book for this course: Ludwig von Auer, Ökonometrie: Eine Einführung, 4. Au., Springer, Alternatively: William E. Gri ths, R. Carter Hill and George G. Judge, Learning and Practicing Econometrics, John Wiley & Sons, James Stock and Mark Watson, Introduction to Econometrics, Addison Wesley, Russell Davidson and James MacKinnon, Econometric Theory and Methods, Oxford University Press,

8 Class Class teacher: Rainer Schüssler Time and location: Tue, , CAWM1 A detailed schedule is available on the home page of this course Studium! Aktuelle Veranstaltungen! Econometrics II 8

9 Outline Very brief revision of Econometrics I (chap. 8 to 14) Violations of model assumptions (chap. 15 to 19, 21) Stochastic exogenous variables (chap. 20) Dynamic models (chap. 22) Interdependent equation systems (chap. 23) 9

10 Multiple linear regression model (revision) Assumption A1: No relevant exogenous variable is omitted from the econometric model, and all exogenous variables in the model are relevant Assumption A2: The true functional dependence between X and y is linear Assumption A3: The parameters are constant for all T observations (x t ; y t ) Assumptions B1 to B4: u N 0; 2 I T 10

11 Assumption C1: The exogenous variables x 1t ; : : : ; x Kt are not stochastic, but can be controlled as in an experimental situation Assumption C2: No perfect multicollinearity: rank(x) = K + 1 Econometric model: y = X + u Point estimator (OLS): ^ = X 0 X 1 X 0 y 11

12 Estimated model: ^y = X^ Residuals: ^u = y ^y Coe cient of determination: R 2 = S yy S yy S^u^u = S^y^y S yy = P Kk=1 b k S ky S yy 12

13 Unbiasedness: E(^) = Covariance matrix of ^ V(^) = 2 X 0 X 1 Gauss-Markov theorem: ^ is BLUE 13

14 Distribution of y: y N(X; 2 I T ) Distribution of ^: ^ N ; 2 X 0 X 1 Estimator of error term variance: ^ 2 = S^u^u T K 1 Unbiasedness: E(^ 2 ) = 2 14

15 Interval estimator of the component k of h^k t a=2 cse(^k ) ; ^k + t a=2 cse(^k ) i t-test: where H 0 : r 0 = q H 1 : r 0 6= q r = [r 0 ; r 1 ; : : : ; r K ] 0 Test statistic: t = r0^ q cse(r 0^) 15

16 F -test: H 0 : R = q H 0 : R 6= q Test statistic: or F = S 0 bubu S bubu. L S bubu / (T K 1) ; R^ q 0 h R X 0 X 1 R 0 i 1 R^ q =L F = ^u 0^u= (T K 1) where L is the number of restrictions in H 0 16

17 Forecasting: Let x 0 = [1; x 10 ; x 20 ; : : : ; x K0 ] 0 be the vector of exogenous variables Point forecast: ^y 0 = x 0 0^ Variance of the forecast error: V ar (^y 0 y 0 ) = x 0 0 X 0 X 1 x0 Violation of A1: Omitted or redundant variables Violation of A2: Nonlinear functional forms 17

18 Qualitative exogenous variables A3: The parameters are constant for all T observations (x t ; y t ) Example: Wage y t depends on both education x 1t and age x 2t y t = + 1 x 1t + 2 x 2t + u t Suppose the parameters di er between men and women y t = M + M1 x 1t + M2 x 2t + u t y t = F + F 1 x 1t + F 2 x 2t + u t What happens if the gender di erence is ignored? [dummy.r] 18

19 Introduce a dummy variable D t = ( 0 if male 1 if female Extended model y t = + D t + 1 x 1t + 1 D t x 1t + 2 x 2t + 2 D t x 2t + u t Submodels for men (D t = 0) and women (D t = 1) y t = + 1 x 1t + 2 x 2t + u t y t = ( + ) + ( ) x 1t + ( ) x 2t + u t Interpretation of the coe cients ; 1 ; 2 19

20 Estimation of the model by OLS? How does the matrix of exogenous variables X look like? Apply t- or F -tests to check parameter constancy, e.g. H 0 : = 1 = 2 = 0 Often, the models just include a level e ect, i.e. (use a t-test for ) y t = + D t + 1 x 1t + 2 x 2t + u t 20

21 If the qualitative exogenous variable has more than two values, we need more than one dummy variable Example: Religion (protestant, catholic, other) D prot t = D cath t = 8 >< >: 8 >< >: 0 if other 1 if protestant 0 if catholic 0 if other 0 if protestant 1 if catholic Interpretation of the coe cients? 21

22 If there are two or more qualitative exogenous variables, interaction terms can be added Example: Gender and citizenship D 1t = D 2t = ( 0 if male 1 if female ( 0 if German citizenship 1 else Interpretation of the coe cients 1 ; 2 ; in the two models y t = + 1 D 1t + 2 D 2t + x t + u t y t = + 1 D 1t + 2 D 2t + D 1t D 2t + x t + u t 22

23 What happens if there are two dummy variables D female t = D male t = ( 0 if male 1 if female ( 1 if male 0 if female What happens if the dummy variable is coded as D t = ( 1 if male 2 if female 23

24 Compare the joint dummy variable model y t = + D t + 1 x 1t + 1 D t x 1t + 2 x 2t + 2 D t x 2t + u t with the two separated models [dummycomparison.r] y t = M + M1 x 1t + M2 x 2t + u t y t = F + F 1 x 1t + F 2 x 2t + u t for men for women Questions: 1. Why are the point estimates identical? [1] 2. Why is the sum of squared residuals identical? [2] 3. Why are the standard errors di erent? [3] 24

25 Heteroskedasticity Assumption B2: V ar(u t ) = 2 for t = 1; : : : ; T Rent example: The rent y t depends on the distance x t from the city center t x t y t t x t y t 1 0,50 16,80 7 3,10 12,80 2 1,40 16,20 8 4,40 12,20 3 1,10 15,90 9 3,70 15,00 4 2,20 15, ,00 13,60 5 1,30 16, ,50 14,10 6 3,20 13, ,10 13,30 25

26 The scatterplot suggests that there might be heteroskedasticity: What are the properties of ^ if there is heteroskedasticity? [4] 26

27 Transformation of the model (Restrictive and arbitrary) assumption: 2 t = 2 x t Transformation of the model: y t p = p 1 + x t p + xt xt xt y t = z t + x t + u t u t p xt {z } error term Properties of the new error term u t [5] 27

28 The transformed model satis es all A-, B- and C-assumptions! OLS estimation of the transformed model: b = S z y S z z b = S x y = S x x P (x t x ) (y t y) P x t x 2 = P 1xt (x t x) (y t y) P 1xt (x t x) 2 28

29 The usual estimators ^ = P (xt x) (y t y) P (xt x) 2 ^ = y ^x are ine cient An unbiased estimator of V ar(u t ) = 2 is ^ 2 = S^u ^u T 2 29

30 From 2 t = 2 x t we conclude that is an unbiased estimator of V ar(u t ) ^ 2 t = ^2 x t It can be shown that [6] V ar(^) = P (xt S 2 xx x) 2 2 t The usual equations 2 V ar(^) = and ^ 2 = S^u^u S xx T 2 are wrong under heteroskedasticity 30

31 Goldfeld-Quandt test Step 1: Re-order the observations according to their x t -values (or some other source of heteroskedasticity ) Step 2: De ne two groups: T 1 observations with low x t -values; T 2 observations with high x t -values Often, T 1 + T 2 = T 31

32 Step 3: We assume 2 2 > 2 1 ; hence H 0 : 2 2 = 2 1 H 1 : 2 2 > 2 1 Step 4: Separate OLS estimation for both groups; compute S and 1^u^u S2^u^u Step 5: Goldfeld and Quandt (1972) show that ander H 0 F = S2^u^u = (T 2 K 1) S 1^u^u = (T 1 K 1) follows an F (T2 K 1;T 1 K 1) -distribution Step 6: Compare F to the critical level F a. If F > F a ; reject H 0 32

33 Numeric illustration: rentexample.r 1. Order the observations according to their x t -values 2. Group Z: City center (T Z = 5) Group P: Periphery (T P = 7) 3. Null hypothesis: H 0 : 2 P 2 Z 4. Sums of squared residuals S Z^u^u = 0:246 and SP^u^u = 4:666 33

34 5. Hence, F = 4:666=5 0:246=3 = 11:4 6. At level a = 5% the critical value is Reject the null hypothesis. The data indicate heteroscedasticity. The null hypothesis that the error term variance is the same in the center and the periphery, is rejected at the 5% level. Heteroscedasticity should be taken into account. 34

35 White test Consider the linear regression model with two exogenous variables y t = + 1 x 1t + 2 x 2t + u t Step 1: H 0 : the error terms are homoskedastic Step 2: Calculate the OLS residuals ^u t Step 3: Estimate the auxilliary regression ^u 2 t = x 1t + 2 x 2t + 3 x 2 1t + 4x 2 2t + 5x 1t x 2t + v t 35

36 Step 4: It can be shown that under H 0 R 2 T 2 r where r is the number of slope parameters in the auxilliary regression If T R 2 is larger than the critical value of the 2 r-distribution, reject H 0 The squared residuals can be explained (at least partially) by the exogenous variables Illustration [rentexample.r] 36

37 Question: Given that heteroskedasticity has been detected, how shall we proceed? Answer 1: Adjust the estimation procedure! GLS or feasible GLS Answer 2: Still use OLS but compute the correct standard errors! White s heteroskedasticiy-consistent covariance maxtrix estimator 37

38 Generalized least squares method (GLS) Verallgemeinerte Kleinste-Quadrate-Methode (VKQ) Regression model y = X + u Covariance matrix of the error terms V(u) 6= 2 I, but V(u) = 2 Example: 2 t = 2 x kt ; then = x k1 : : : : : : x kt

39 Transformation of the model: Since is positive de nit, there is a (T T )-matrix P with P 0 P = 1 Example: If then P = = x k1 : : : : : : x kt ; 1= p x k1 : : : : : : 1= p x kt

40 From P 0 P = 1 it follows that PP 0 = I T Pre-multiplication of y = X + u by P yields Py = PX + Pu y = X + u Properties of u [7] The transformed model satis es all A-, B- and C-assumptions 40

41 Derivation of the GLS estimator ^ V KQ [8] Covariance matrices of ^ V KQ and ^ [9] Estimation of 2 by ^ 2 = ^u0^u T K 1 = ^u0 1^u T K 1 Ignoring heteroskedasticity one would use V(^) = 2 (X 0 X) 1 ^ 2 = ^u 0^u T K 1 41

42 Interval estimators and hypothesis tests would not work correctly What happens if is unknown? Example: W = 2 = I 0 : : : : : : : : : I.. 2 II : : : : : : : : : 0 2 II

43 Feasible Generalized Least Squares (FGLS), Geschätzte verallgemeinerte Kleinste-Quadrate (GVKQ) First, estimate the unknown quantities in W = 2 The FGLS estimator is ^ F GLS = X 0 ^W 1 X 1 X 0 ^W 1 y Estimated covariance matrix ^V(^ F GLS ) = X 0 ^W 1 X 1 What to do if there is no information at all about the form of heteroskedasticity? 43

44 White s heteroskedasticiy-consistent covariance maxtrix estimator Davidson and MacKinnon, chap. 5.5 Econometric model y = X + u Covariance matrix V (u) = W with W =diag 2 1 ; : : : ; 2 T OLS estimator ^ = (X 0 X) 1 X 0 y Covariance matrix V(^) = X 0 X 1 X 0 WX 0 X 0 X 1 44

45 Consistent estimation of W is impossible White (1980): Consistent estimation of = 1 T X0 WX is possible! = 1 T TX t=1 2 i x ix 0 i Consistent estimator of ^ = 1 T TX t=1 ^u 2 i x ix 0 i 45

46 Estimated covariance matrix ^V(^) = (X 0 X) 1 X 0 ^WX(X 0 X) 1 with ^W = ^u ^u 2 T Sandwich estimator Illustration [rentexample.r] 46

47 Autocorrelation Assumption B3: The error terms are uncorrelated, for all t 6= s Cov(u t ; u s ) = 0 Example [water lter.r]: Demand function y t = + x t + u t for water lters; quantity sold y t and prices x t for the months January 2001 to December

48 Assumption about the form of autocorrelation: with 1 < < 1 u t = u t 1 + e t Assumption about e t et NID(0; 2 e) Properties of u t [10] 48

49 Moment functions of u t E(u t ) = 0 V ar(u t ) = 2 e 1 2 Cov(u t ; u t 1 ) = 2 e 1 2 Cov(u t ; u t j ) = j 2 e 1 2!! B1, B2 and B4 are still satis ed But B3 is violated 49

50 Transformation of the model [11] y t y t 1 = (1 ) + (x t x t 1 ) + e t De ne yt = y t y t 1 = (1 ) x t = x t x t 1 Then yt = + x t + e t satis es all A-, B- and C-assumptions (if would be known) 50

51 Hence, OLS estimation is ine cient Consequences for interval estimation and hypothesis tests? The usual OLS formulas and are invalid V ar(^) = 2 ^ 2 = S xx S^u^u T 2 Consequences are the same as in the case of heteroskedasticity 51

52 Diagnosis Plot the residuals ^u t over time, or plot the pairs (^u t 1 ; ^u t ) Example (demand function) Estimator for : Because of u t = u t regression 1 + e t we can estimate by the bu t = bu t 1 + e t 52

53 Least squares estimator ^ = P Tt=2 bu t bu t 1 P Tt=2 bu 2 t 1 Numeric illustration: From the residuals we calculate ^ = = 0:58 Due to the two-step approach the ordinary t-test is no longer exact 53

54 Durbin-Watson test Step 1: Set up the hypotheses H 0 : 0 H 1 : > 0 Step 2: Compute the Durbin-Watson test statistic d = P Tt=2 (bu t bu t 1 ) 2 P Tt=1 bu 2 t 54

55 Numeric illustration: d = = 0:76 Relation between d and ^ [12] d 2(1 ^) Step 3: Find the critical value d a (using econometric software). If d < d a, reject H 0 55

56 Problem: The critical value d a depends on X If the software cannot compute d a there are tables providing an upper boandary d H a and a lower boandary d L a for d a Step 4: Compare the test statistic d to d L a and dh a Decision rule: if d < d L 0;05, reject H 0 : 0; if d > d H 0;05, do not reject H 0 : 0; if d L 0;05 d dh 0;05, leave the decision open 56

57 Numeric illustration: For K = 1 and T = 24 Table T5 gives d L 0:05 = 1:27 and dl 0:05 = 1:45 Since d = 0:76 < d L 0:05, reject the null hypothesis that the residuals are positively correlated Disadvantages of the Durbin-Watson test: no decision in some cases lagged endogenous variables are not allowed only applicable for AR(1)-processes Alternative tests for autocorrelation are available in many software packages 57

58 GLS and autocorrelation Regression model y = X + u Covariance matrix V(u) = 2 with = : : : T 1 1 : : : T T 1 T 2 : : :

59 Transformation of the model using the matrix P satisfying P 0 P = 1 One can verify that P = q : : : : : : : : : : : : The GLS estimator is the same as in the case of heteroskedasticity 59

60 GLS estimator with covariance matrix ^ GLS = X 0 1 X 1 X 0 1 y V(^ GLS ) = 2 X 0 1 X 1 Estimator of the error term variance ^ 2 = ^u0 1^u T K 1 GLS is not possible as (and hence P) is unknown 60

61 Hildreth-Lu approach: Lege für ein feines Gitter über [ von ^ 2 1; 1]; wähle das mit dem kleinsten Wert Cochrane-Orcutt-Verfahren: Schätze ^ aus den KQ-Residuen, dann GVKQ mit ^; anschließend Iterationen 61

62 Heteroskedasticiy and autocorrelation consistent covariance maxtrix estimation Newey W.K. and West, K.D. (1987), A Simple Positive De nite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix, Econometrica, 55: Econometric model y = X + u Covariance matrix with arbitrary covariance matrix W V (u) = W 62

63 OLS estimator ^ = (X 0 X) 1 X 0 y Covariance matrix of ^ (as before) V(^) = (X 0 X) 1 X 0 WX(X 0 X) 1 The matrix W cannot be estimated consistently But 1 T X0 WX can estimated consistently 63

64 Consistent estimation of V(^) by TX t=1 ^u 2 t x t x 0 t + qx b=1 1 b q + 1 where q is the number of autocorrelations to be taken into account! A b The matrices A b = TX t=b+1 are estimators of autocorrelation matrices xt^u t^u t b x 0 t b + x t b^u t b^u t x 0 t The White estimator is a special case of V(^) (if A b = 0 for all b) 64

65 Nonnormal error terms Assumption B4: The error terms are normally distributed This assumption is necessary to derive the normality of ^ to derive the t-distribution of the t-statistic to derive the F -distribution of the F -statistic Remember that ^ is a linear estimator ^ = X 0 X 1 X 0 y = Cy 65

66 For a single component of ^ we nd ^ k = TX t=1 c kt y t The random variables y 1 ; : : : ; y T are stochastically independent Hence, ^k is the sum of independent (but not identically distributed) random variables Question: How is the sum of random variables distributed? 66

67 Central limit theorem: The sum of many i.i.d. random variables is approximately normally distributed The central limit theorem does also hold for nonidentical distribution: ^ k is approximately normally distributed, even if the error terms are nonnormal Further: ^ is approximately multivariate normal, ^ appr N ; 2 X 0 X 1 Careful: There are some (weak) regularity conditions that must be satis ed; normality can break down (but usually does not) 67

68 Simulation [b4.r]: Gratuituy example (from last semester): y t = 0:5 + 0:1 x t + u t satisfying all A-, B-, C-assumptions apart from B4 Distribution of error terms: f ut (u) = exp ( (u + 1)) 68

69 Since ^ appr N ; 2 X 0 X 1 we nd for single components ^k of ^ for k = 1; : : : ; K ^ k k SE(^k ) d! U N(0; 1) Con dence intervals and t-tests are asymptotically valid (use quantiles of N(0; 1) instead of t-distribution) F -tests are asymptotically valid (convergence to 2 -distribution) 69

70 Stochastic convergence and limit theorems Comvergence of real sequences: Let a 1 ; a 2 ; : : : be a sequence of real numbers De nition: The sequence fa n g n2n converges to its limit a, if for any (arbitrarily small) " > 0 there is a number N(") such that ja n aj < " for all n N(") Notation: lim n!1 a n = a or a n! a Examples: lim n!1 1=n = 0 lim n!1 h n 2 + n + 6 = 3n 2 2n + 2 i = 1=3 70

71 Graph of the convergent sequence (n 2 + n + 6)=(3n 2 2n + 2): 71

72 Questions: How can the idea of convergence be transferred to sequences of random variables? What is a sequence of random variables? What does convergence of sequences of random variables mean? Which sequences of random variables do we typically encounter in econometrics? 72

73 De nition: Let X 1 ; X 2 ; : : : be random variables X i :! R We call X 1 ; X 2 ; : : : a sequences of random variables X 1 ; X 2 ; : : : are (countably in nitely many) multivariate random variables Formally, this is a sequences of functions (not of real variables) 73

74 De nition: The sequence X 1 ; X 2 ; : : : converges almost surely (fast sicher) to a random variable X, if P n! : lim n!1 X n(!) = X(!) o = 1 Notation X n X n f:s:! X a:s:! X This kind of convergence is only of minor importance in econometrics 74

75 De nition: The sequences X 1 ; X 2 ; : : : converges in probability (nach Wahrscheinlichkeit) to a random variables X, if lim n!1 P (jx n Xj < ") = 1 Notation X n p! X plim X n = X This kind of convergence is very important in econometrics 75

76 Special case: Convergence in probability to a constant The sequence X 1 ; X 2 ; : : : converges in probability (nach Wahrscheinlichkeit) to a constant a, if lim n!1 P (jx n aj < ") = 1 Notation X n p! a plim X n = a In econometrics we usually need this kind of convergence in probability 76

77 De nition: The sequence X 1 ; X 2 ; : : : (with distribution functions F 1 ; F 2 ; : : :) converges in distribution, in law (nach Verteilung) to a random variable X (with distribution function F ), if lim n!1 F n(x) = F (x) for all x 2 R where F (x) is continuous Notation X n d! X Relation between types of convergence f:s: X n! X ) p X n! X ) d Xn! X 77

78 Limit theorems: laws of large numbers (LLN, Gesetze der großen Zahl); central limit theorems (CLT, zentrale Grenzwertsätze) Let X 1 ; X 2 ; : : : be a sequence of random variables De ne a new sequence X 1 ; X 2 ; : : : where X n = 1 n nx X i i=1 De ne another new sequence Z 1 ; Z 2 ; : : : where Z n = S n E(S n ) q V ar(s n ) with S n = nx X i i=1 78

79 Strong law of large numbers (SLLN) Let X 1 ; X 2 ; : : : be a sequence of independent random variables with i = E(X i ) < 1 and V ar(x i ) < 1 for i = 1; 2; : : : If P 1 k=1 V ar(x k )=k 2 < 1; then P lim n!1 X n 1 n 1 nx i i=1 1 A = 0A = 1 Special case: iid sequences, X n f:s:! 79

80 Weak law of large numbers (Chebyshev, WLLN) Let X 1 ; X 2 ; : : : be a sequence of independent random variables with i = E(X i ) < 1 and V ar(x i ) < c < 1 Then lim n!1 P X n 1 n nx i i=1 1 < " A = 1 Special case: iid sequences, plim X n = 80

81 Weak law of large numbers (Khinchin) Let X 1 ; X 2 ; : : : be a sequence of iid random variables with E(X i ) = Then lim n!1 P X n < " = 1 There are also laws of large numbers for stochastic processes, e.g. for martingale di erence sequences 81

82 The weak laws of large numbers can easily be generalized to the multivariate case, e.g. Khinchin: Let X 1 ; X 2 ; : : : be a sequences of iid random vectors with E(X i ) = For each component k = 1; : : : ; K lim n!1 P X nk k < " = 1 Notation plim X n = 82

83 Central limit theorem Let X 1 ; X 2 ; : : : be a sequence of random variables Consider the sequence of standardized cumulative sums Z n = S n E(S n ) q V ar(s n ) with S n = nx X i i=1 How is Z n distributed for n! 1? Impose only a few assumptions about the distribution of the X i s 83

84 Central limit theorem (Lindeberg-Levy) Let X 1 ; X 2 ; : : : be a sequence of iid random variables with E(X i ) = and V ar(x i ) = 2 < 1 Let F n (z) = P (Z n z) denote the distribution function of Z n Then lim n!1 F n(z) = Z z p exp 2 2 u2 du Convergence in distribution: Z n d! Z N(0; 1) 84

85 Central limit theorem (Liapunov) Let X 1 ; X 2 ; : : : be a sequence of independent random variables with E(X i ) = i, V ar(x i ) = 2 i < 1, and E(jX ij 2+ ) < 1 for (arbitrarily small) > 0 De ne c n = q Pni=1 2 i If lim n!1 1 c 2+ n nx i=1 1 E (jx i i j) 2+ A = 0; then Z n d! Z N(0; 1) 85

86 The heart of the central limit theorem: no single random variable must dominate the sum Each (X i i )= i is only a negligibly small contribution to the sum (S n E(S n ))=c n Frequent notation (in the iid case) appr S n N(n; n 2 ) appr X n N(; 2 =n) We can deal with the sum as if it is normally distributed (if n is large enough) 86

87 The central limit theorem also applies to empirical moments! Let k = E(X k ) denote the k-th (theoretical) moment of X The k-th empirical moment m k = 1 n nx Xi k i=1 is an estimator for k According to the CLT, m k is asymptotically normal if the variance of X k exists (i.e., the 2k-th moment 2k ) 87

88 The central limit theorem can easily be generalized to the multivariate case, e.g. Lindeberg-Levy: Let X 1 ; X 2 ; : : : be a sequence of iid random vectors with E(X i ) = and Cov(X i ) = Then p n X n d! Z N(0; ) Remark: In the univariate case we can also write p n( X n ) d! Z N(0; 2 ) 88

89 Further central limit theorems The assumptions about the sequence X 1 ; X 2 ; : : : can be weakened Central limit theorems for stochastic processes Central limit theorems for products of random variables Central limit theorems for maxima (extreme value theory) 89

90 Useful rules of calculus If plim X n = a and plim Y n = b, then plim(x n Y n ) = a b plim(x n Y n ) = ab plim Xn Y n = a ; if b 6= 0 b If a function g is continuous at a, then plim g (X n ) = g (a) 90

91 If Y n d! Z and h is a continuous function, then h (Y n ) d! h (Z) Cramér s theorem: If X n p! a and Yn d! Z, then X n + Y n d! a + Z X n Y n d! az Cramér s theorem is very useful if there are unknown parameters in the asymptotic distribution that can be estimated consistently (more on consistency later) 91

92 Example for Cramér s theorem: Let X 1 ; : : : ; X n be a random sample from X; we know that S 2 n = S 2 n = 1 n 1 n 1 nx i=1 nx i=1 Xi X 2 p! 2 Xi X 2 p! 2 Hence S n p! 1 and S n p! 1 92

93 According to the central limit theorem p X n n d! Z N (0; 1) Due to and =S n p! 1 we have p X n n = p X n n S n S n p X n d n! Z 1 = Z N (0; 1) S n Similarly for p n( X n )=S n 93

94 Multivariate version: According to the central limit theorem p n X n d! Z N (0; ) Due to ^ n = 1 n X Xi X n Xi X n 0 p! we can use the following approximation for large n; appr X n N ; ^ n (Careful: the notation is bad, but it helps the intuition) 94

95 Stochastic exogenous variables Assumption C1: The matrix X ist non-stochastic What happens if X is (at least partially) stochastic? We distinguish three cases: 1. X and u are stochastically independent 2. Contemporaneous uncorrelatedness: Cov(x kt ; u t ) = 0 for all t; k 3. X and u are contemporaneously correlated 95

96 Conditional expectation Let (X; Y ) be jointly continuous with density function f X;Y (x; y) Marginal distributions (marginal densities) f X (x) = f Y (y) = Z 1 Z 1 1 f X;Y (x; y)dy 1 f X;Y (x; y)dx Conditional density of X given Y = y f XjY =y (x) = f X;Y (x; y) f Y (y) 96

97 Conditional expectation (bedingter Erwartungswert) of X given Y = y E (XjY = y) = Z 1 1 xf XjY =y (x) dx Conditional expectation (bedingte Erwartung) of X given Y : E (XjY ) is a random variable realizing as E (XjY = y) if Y = y The conditional expectation E(XjY = y) is a real number (for given y) The conditional expectation E(XjY ) is a random variable 97

98 Useful rules for conditional expectations 1. Law of iterated expectations: E (E (XjY )) = E (X) 2. Independence: If X and Y are independent, then E (XjY ) = E (X) 3. Linearity: For a 1 ; a 2 2 R, E (a 1 X 1 + a 2 X 2 jy ) = a 1 E (X 1 jy ) + a 2 E (X 2 jy ) 4. The conditioned random variables can be treated like constants, E (f (X) g (Y ) jy ) = g (Y ) E (f (X) jy ) 98

99 Stochastic exogenous variables, case 1 Model y = X + u with X and u stochastically independent The estimators ^ and ^ 2 are unbiased and consistent (Estimated) covariance matrix of ^ Asymptotic normality: p T (^ ) N(0; 2 uq 1 XX ) Conclusion: If X and u are independent there are no problems 99

100 Stochastic exogenous variables, case 2 The error term and the exogenous variables are contemporaneously uncorrelated (but may be correlated over time) Typical case: lagged endogenous variables on the right hand side Unbiasedness is lost Consistency and asymptotic normality still hold Conclusion: If there is contemporaneous uncorrelatedness, there are hardly any problems if the sample is large enough 100

101 Stochastic exogenous variables, case 3 Contemporaneous correlation between error terms and exogenous variables Example: 101

102 Why might there be contemporaneous correlation? Errors-in-variables: Model: y t = + x t + e t Measurement: x t = x t + v t Simultaneous equation systems: c t = + y t + u t y t = c t + i t 102

103 Instrumental variables (IV estimation) Model y = X + u with contemporaneous correlation between X and u Instrumental variables: contemporaneously uncorrelated with u, but correlated with X Let Z denote the (T (L + 1))-matrix of instruments, and P = Z Z 0 Z 1 Z 0 103

104 The matrix P is symmetric and idempotent, P 0 P = P Number of columns L K (often L = K) Transformed model Py = PX + Pu The least squares estimators of the transformed model are called IV estimators ^ IV = X 0 P 0 PX 1 X 0 P 0 Py = X 0 PX 1 X 0 Py 104

105 If L = K then ^ IV = (X 0 Z(Z 0 Z) 1 Z 0 X) 1 X 0 Z(Z 0 Z) 1 Z 0 y = (Z 0 X) 1 (Z 0 Z)(X 0 Z) 1 X 0 Z(Z 0 Z) 1 Z 0 y = (Z 0 X) 1 Z 0 y Simple linear regression (L = K = 1) ^ IV = P (zt z) (y t y) P (zt z) (x t x) 105

106 Assumptions about Z Existing limit, plim Z0 Z T = lim T!1 E! Z0 Z T = Q ZZ with Q ZZ positive de nite Asymptotic correlation with exogenous variables plim Z0 X T = Q ZX; rang(q ZX ) = K

107 Asymptotic uncorrelatedness with error terms plim Z0 u T = lim T!1 E! Z0 u T = 0 107

108 IV estimators are consistent but not unbiased Hausman test (Hausman-Wu test): Hypotheses H 0 : plim X0 u T = 0 H 1 : plim X0 u T 6= 0 Test idea: Under H 0 both OLS and IV are consistent, under H 1 only IV is consistent If ^IV deviates too much from ^, reject H0 108

109 Test statistic: ^IV 0 ^ ^V(^ IV ) 1 ^IV ^V(^) ^ Asymptotic distribution under H 0 is 2 K, where K is the number of columns in Z that are not included in X 109

110 Multicollinearity Perfect vs imperfect multicollinearity Graphical illustration 110

111 Dynamic models Stochastic process: x 1 ; : : : ; x T Moment functions: E(x t ), V ar(x t ), Cov(x t ; x t+ ) (Weak) stationarity E(x t ) = V ar(x t ) = 2 x Cov(x t ; x t+ ) = Order of integration of a process, I(d) 111

112 Simplest dynamic model: lagged exogenous variables y t = + 0 x t + 1 x t 1 + : : : + K x t K + v t Interpretation of the parameters (short-term and long-term multiplier) Problems: many parameters Multicollinearity no precise estimation of individual components k 112

113 Note: The variance of the long-term multiplier may be small even if all components ^k have a large variance Functional form for 0 ; 1 ; : : : ; K Polynomial lags (Almon lags) geometric lags (Koyck lags) 113

114 Polynomial lags The k are a polynomial function of k Example: Quadratic function: for k = 0; : : : ; K k = k + 2 k 2 114

115 There are less than K parameters, since y t = + = + KX k=0 KX k=0 k x t k + v t k + 2 k 2 x t k + v t X K X K = + 0 x t k + 1 kx t k k=0 k=0 + 2 K X k=0 k 2 x t k + v t = + 0 x 1t + 1x 2t + 2x 3t + v t The validity of the linear restrictions can be tested 115

116 Geometric lags The k depend on k as follows, k = 0 k where 0 < < 1 It is possible to set K = 1 y t = + 0 x t + 1 x t x t 2 + : : : + v t = + 0 x t + 0 x t x t 2 + : : : + v t 116

117 Short-term multiplier: 0 Long-term multiplier: 1X k=0 k = 0 1X k=0 k =

118 Koyck transformation: y t = + 0 x t + 0 x t x t 2 + : : : + v t minus y t 1 = + 0 x t x t 2 + : : : + v t 1 yields y t y t 1 = ( ) + 0 x t + (v t v t 1 ) y t = x t + y t 1 + u t Estimation problematic since B3 and C1 are violated 118

119 Models with rational lag distribution y t = x t + 1 x t 1 + : : : + K x t K + 1 y t 1 + : : : + M y t M + u t Special case K = M = 1 y t = x t + x t 1 + y t 1 + u t From y t = x t + x t 1 + y t 1 + u t we nd y t y t 1 = x t + x t 1 + u t 119

120 Long-term (undisturbed) equilibrium y = x Error correction formulation with error (disequilibrium) term y t = 0 x t (1 ) e t 1 + u t e t 1 = y t x t 1 If x t and y t are both I (1), and if e t cointegrated 1 is I(0), then x t and y t are called 120

121 Estimation of error correction models (ECM) 1. Determine the order of integration of x t and y t 2. Estimate by OLS y t 1 = 0 1 and calculate the residuals ^e t x t 1 + e t 1 3. Determine the order of integration of ^e t 1 4. If there is cointegration, estimate y t = 0 x t (1 ) ^e t 1 + u t 121

122 Interdependent equation systems Illustration by a simple example Pharmacy company: Advertisement expenditures w t, quantity sold a t, price p t, advertising price (per page) q t Model equations a t = + 1 w t + 2 p t + u t w t = + 1 a t + 2 q t + v t Error terms satisfy all B-assumptions; further we assume Cov (u t ; v t ) = uv and Cov (u s ; v t ) = 0 for s 6= t 122

123 In the rst equation, u t and w t are correlated! Hence the OLS estimators are inconsistent Structural form vs reduced form From the structural form we derive the reduced form a t = + 1 w t + 2 p t + u t w t = + 1 a t + 2 q t + v t a t = p t + 3 q t + u t w t = p t + 6 q t + vt 123

124 Reduced form: all endogenous variables are on the left hand side, all exogenous are on the right hand side The equations of the reduced form can be estimated by the OLS method. From the estimated values ^ 1 ; : : : ; ^ 6 one obtains the following values ^; ^1 ; ^2 ; ^; ^1 ; ^2 The estimators ^; ^1 ; ^2 ; ^; ^1 ; ^2 are consistent It is not always possible to derive the structural parameters from the reduced form parameters (identi cation problem) 124

125 From the structural form a t = + 1 w t + 2 p t + u t w t = + 1 a t + v t one obtains the reduced form a t = p t + u t w t = p t + vt Five structural parameters but only for reduced parameters Sometimes there are more reduced parameters than structural parameters 125

126 Condition of indeterminacy _K = Number of the exogenous variables in the general model K = Number of the exogenous variables in the considered equation M = Number of the endogenous variables in the considered equation An equation is underidenti ed, if M 1 > _K K exactly identi ed, if M 1 = _K K overidenti ed, if M 1 < _K K M 1 is the number of the explanatory endogneous variables (on the right side); _K K is the number of the exogenous variables in the other equations 126

127 Estimation of an exactly or overidenti ed equation Two-stages LS method (2SLS) Idea: obtain instrumental variables from the reduced form Example for the 2SLS method In the system a t = + 1 w t + 2 p t + u t w t = + 1 a t + 2 q t + v t The second equation has to be estimated 127

128 First step: estimate by the LS and obtains ^a t = ^ 1 + ^ 2 p t + ^ 3 q t a t = p t + 3 q t + u t Second step: estimate by the LS w t = + 1^a t + 2 q t + v t The 2SLS estimators are consistent (IV-estimator) The standard errors have to be adjusted The properties of the estimators in nite samples are complicated 128

129 Interdipendent euquationsystems in matrix notation General representation Let M the number of equations in the system The endogenous variables are set in a (T M)-Matrix Y = [y 1 y 2 : : : y M ] The exogenous variables (and the intercept) are set in a (T _K)-Matrix X = [x 0 x 1 : : : x K ] 129

130 The m-th equation is y m = m x 0 + 1m x 1 + 2m x 2 + : : : + Km x K + 1m y 1 + : : : + m 1m y m 1 + m+1m y m+1 + : : : + Mm y M +u m Setting mm = 1, yields 1m y 1 + : : : + Mm y M + m x 0 + 1m x 1 + : : : + Km x K + u m = 0 Pile the coe cients in vectors m = ( 1m ; 2m ; : : : ; Mm ) 0 m = ( m ; 1m ; 2m ; : : : ; Km ) 0 130

131 Compact notation of the complete system Y 1 + X 1 + u 1 = 0 Y 2 + X 2 + u 2 = 0. Y M + X M + u M = 0 and accordingly Y + XB + U = 0 with dimensions! = [ 1 : : : M ] B = [ 1 : : : K ] U = [u 1 : : : u M ] 131

132 The noise terms u m, m = 1; : : : ; M, satisfy all B-assumptions Dependencies between noise terms of di erent equations are permitted Assumption E(u m u 0 m ) = 2 mi T für m = 1; : : : ; M E(u m u 0 n) = mn I T für m 6= n How could one write these asumptions in a compact notation for the matrix U? 132

133 Reduced form (all endogenous variables on the left side and all exogenous ones on the right side) From Y + XB + U = 0 follows Y 1 + XB 1 + U 1 = 0 and accordingly Y = X + V with = B 1 and V = U 1 133

134 The structural coe cients in and B are identi able only when their values can be distinctly deduced from Number of the coe cients: : _KM : M 2 M B : _KM So one needs (at least) M 2 M appropriate restrictions in and/or B In what follows zero-restrictions are assumed 134

135 Estimations of interdependent equationsystems Reduced form y 1 = X 1 + v 1. y M = X M + v M LS-estimation of one equation ^ m = (X 0 X) 1 X 0 y m LS-estimation of all equations ^ = (X 0 X) 1 X 0 Y 135

136 ILS-method: if equation m is exactly identi ed, one can derive the estimators of the structural coe cients from the matrix ^ If equation m is exactly or overidenti ed, one uses the 2SLS-method Resort and matrix partition h ym Y m Y m i m0 5 + X m + u m = 0 Y m : in equation m included endogenous variables; Y m : excluded variables 136

137 Equation m can be rewritten in this way y m = Y m m + X m + u m = h Y m X i " m m # + u m First step: estimate ^ = (X 0 X) 1 X 0 Y and accordingly partition ^ m c m c m = (X 0 X) 1 X 0 h y m Y m Y m i 137

138 The system of the endogenous variables are estimated by cy m = X c m Second step: substitute Y m through c Y m in y m = h Y m X i " m m # + u m 2SLS estimator 2 4 bzskq m ^ ZSKQ m 3 5 = " c 0 Y m X c Y m # 1 0 X c Y m X ym 138

139 The covariance matrix of the estimated vector is 2 4 bzskq m ^ ZSKQ m " c 0 ^ 2 Y c m X # 1 Y m X 3 5 with and NOT ^ 2 = 1 T ^ 2 = 1 T TX t=1 TX t=1 0 m Y m X i 2 4 bzskq m ^ ZSKQ m m c Y m X 2 4 bzskq m ^ ZSKQ m 312 5A 312 5A 139