Econometrics. Andrés M. Alonso. Unit 1: Introduction: The regression model. Unit 2: Estimation principles. Unit 3: Hypothesis testing principles.

Transcription

1 Andrés M. Alonso Unit 1: Introduction: The regression model. Unit 2: Estimation principles. Unit 3: Hypothesis testing principles. Unit 4: Heteroscedasticity in the regression model. Unit 5: Endogeneity of regressors.

2 Bibliography 2 Creel, M. (2006), (Available at Hayashi, F. (2000), Princeton University Press. (Library code: D HAY). Wooldridge, J.M. (2000) Introductory : A Modern Approach, South Western College Publishing (Library code: D WOO). Software: I recommend to use MATLAB (or Octave, its freeware version) or S-Plus (or R, its freeware version). Causin, M.C. (2005) MATLAB Tutorials. (Available at LeSage, J.P. (1999) Applied using MATLAB. (Available at

3 The outline for today 3 Unit 1. Introduction: The regression model. 1.1 Conditional expectation. 1.2 Assumptions. 1.3 Interpretation of the model. Addendum. Resampling methods in i.i.d. data. The Jackknife. The Bootstrap. Unit 2. Estimation principles. Unit 3: Hypothesis testing principles. Unit 4: Heteroscedasticity in the regression model. Unit 5: Endogeneity of regressors.

4 Introduction 4 is a branch of Economy concerning the empirical study of relations among economic variables. In econometrics is used: An econometric model: Based on the economic theory. Data: Facts. techniques: Statistical inference. Recommended readings: Chapter 2 and 3 of Creel (2006) and Chapter 1 and 6 of Greene (2000).

5 Introduction 5 Objectives of an econometric study Structural Analysis: It consists on the use of models to measure specific economic relations. Its an important objective since we can compare different theories under the fact s (data) information. Prediction: It consists on the use of models to predict future values of economic variables using the fact s (data) information. Policies evaluation: alternatives policies. It consists on the use of models to select among These three objectives are interrelated.

6 Introduction 6 Economic and econometric models: Example # 1: Economic theory tells us that demand functions are something like: x i = x i (p i, m i, z i ) x i is G 1 vector of quantities demanded. p i is G 1 vector of prices. m i is income. z i is a vector of individual characteristics related to preferences. Suppose we have a sample consisting of one observation on n individuals demands at time period t (this is a cross section, where i = 1, 2,..., n indexes the individuals in the sample).

7 Introduction 7 Example # 1 (cont.): This is nice economic model but it is not estimable as it stands, since: The form of the demand function is different for all i. Some components of z i may not be observable to an outside modeler. For example, people don t eat the same lunch every day, and you can t tell what they will order just by looking at them. Suppose we can break z i into the observable components w i and a single unobservable component ε i. What can we do?

8 Introduction 8 Example # 1 (cont.): An estimable (e.g., econometric) model is x i = β 0 + p iβ p + m i β m + w iβ w + ε i Here, we impose a number of restrictions on the theoretical model: The functions x i ( ) which may differ for all i have been restricted to all belong to the same parametric family. Of all parametric families of functions, we have restricted the model to the class of linear in the variables functions. There is a single unobservable component, and we assume it is additive.

9 Introduction 9 Example # 1 (cont.): These are very strong restrictions, compared to the theoretical model. Furthermore, these restrictions have no theoretical basis. For this reason, specification testing will be needed, to check that the model seems to be reasonable. First conclusion: Only when we are convinced that the model is at least approximately correct should we use it for economic analysis.

10 Introduction 10 The classical linear model is based upon several assumptions: 1. Linearity: the model is a linear function of the parameter vector β 0 : y t = β 0 1x 1 + β 0 2x β 0 kx k + ε t, (1) or in matrix form, y = x β 0 + ε, where y is the dependent variable, x = (x 1, x 2,, x k ), where x t is a k 1 vector of explanatory variables and β 0 and ε are conformable vectors. The subscript 0 in β 0 parameter. means this is the true value of the unknown

11 Introduction 11 Suppose that we have n observations, then model (1) can be written as: y = Xβ + ε, (2) where y is a n 1 vector and X is a n k matrix. 2. IID mean zero errors: E(ε X) = 0 Var(ε X) = E(εε X) = σ 2 I n. What are the implications of this second assumption?

12 Introduction Nonstochastic, linearly independent regressors: (a) X n k has rank k, its number of columns. (b) X is nonstochastic. (c) lim n 1 n X X = Q X, a finite positive definite matrix. What are the implications of this third assumption? 4. Normality (Optional): ε X is normally distributed. For economic data, the assumption of nonstochastic regressors is obviously unrealistic. Likewise, normality of errors will often be unrealistic.

13 Introduction 13 Linear models are more general than they might first appear, since one can employ nonlinear transformations of the variables: ϕ 0 (z t ) = [ ϕ 1 (w t ) ϕ 2 (w t ) ϕ p (w t ) ] β 0 + ε t where the φ i () are known functions. Defining y t = ϕ 0 (z t ), x t1 = ϕ 1 (w t ), etc, leads to a model in the form of equation (1). For example, the Cobb-Douglas model: z = Aw β 2 2 wβ 3 3 exp(ε) can be transformed logarithmically to obtain ln z = ln A + β 2 ln w 2 + β 3 ln w 3 + ε.

14 Introduction 14 Example: The Nerlove model Theoretical background: For a firm that takes input prices w and the output level q as given, the cost minimization problem is to choose the quantities of inputs x to solve the problem min w x x subject to the restriction f(x) = q. The solution is the vector of factor demands x(w, q). The cost function is obtained by substituting the factor demands into the criterion function: C(w, q) = w x(w, q).

15 Introduction 15 Example: The Nerlove model Monotonicity Increasing factor prices cannot decrease cost, so C(w,q) w 0. Homogeneity Because the cost of production is a linear function of w, it has the property of homogeneity of degree 1 in input prices: C(tw, q) = tc(w, q), where t is a scalar constant. Returns to scale The returns to scale parameter γ is defined as the inverse of the elasticity of cost with respect to output: γ = ( C(w, q) q ) 1 q. C(w, q) Constant returns to scale is the case where increasing production q implies that cost increases in the proportion 1:1. If this is the case, then γ = 1.

16 Introduction 16 Example: The Nerlove model Cobb-Douglas approximating model: The Cobb-Douglas functional form is linear in the logarithms of the regressors and the dependent variable. For a cost function, if there are g factors, the Cobb-Douglas cost function has the form: C = Aw β wβ g g q β q e ε. After a logarithmic transformation we obtain: where α = ln A. ln C = α + β 1 ln w β g ln w g + β q ln q + ɛ,

17 Introduction 17 Example: The Nerlove model Company cost Company cost - logs Output x Output - logs

18 Introduction 18 Example: The Nerlove model Using the Cobb-Douglas functional form we can: Verify that the property of HOD1, since it implies that g i=1 β g = 1. Verify the hypothesis that the technology exhibits CRTS, since it implies that γ = 1 β q = 1 so β q = 1. Verify the hypothesis of monotonicity, since it implies that the coefficients β i 0, i = 1,..., g. BUT, the β s are unknowns, then we must to estimate them.

19 Estimation principles 19 Estimation methods Ordinary Least Squares Method: The ordinary least squares (OLS) estimator is defined as the value that minimizes the sum of the squared errors: 5 4 ˆβ = arg min s(β) where s(β) = n (y t x tβ) 2 t=1 = (y Xβ) (y Xβ) = y y 2y Xβ + β X Xβ = y Xβ 2. Company cost - logs Using this distance, the OLS line is defined Output - logs

20 Estimation principles 20 Ordinary Least Squares Method (2): To minimize the criterion s(β), find the derivative with respect to β and set it to zero: D β s( ˆβ) = 2X y + 2X X ˆβ = 0, so ˆβ = (X X) 1 X y. To verify that this is a minimum, check the s.o.c.: D 2 βs( ˆβ) = 2X X. Since rank of X is equal k, this matrix is positive definite, since it s a quadratic form in a p.d. matrix (the identity matrix of order n), so ˆβ is in fact a minimizer.

21 Estimation principles 21 Ordinary Least Squares Method (3): The fitted values are in the vector ŷ = X ˆβ. The residuals are in the vector ˆε = y X ˆβ Note that y = Xβ + ε = X ˆβ + ˆε. The first order conditions can be written as: X y + X X ˆβ = 0 X (y + X ˆβ) = 0 X ˆε = 0, then, the OLS residuals are orthogonal to X.

22 Estimation principles 22 Ordinary Least Squares Method (4): We have that X ˆβ is the projection of y on the span of X, or X ˆβ = X (X X) 1 X y. Therefore, the matrix that projects y onto the span of X is since X ˆβ = P X y. P X = X(X X) 1 X, ˆε is the projection of y onto the n k dimensional space orthogonal to the span of X. We have that ˆε = y X ˆβ = y X(X X) 1 X y = [ I n X(X X) 1 X ] y.

23 Estimation principles 23 So the matrix that projects y onto the space orthogonal to the span of X is Then ˆε = M X y. Therefore M X = I n X(X X) 1 X = I n P X. y = P X y + M X y = X ˆβ + ˆε. Note that both P X and M X are symmetric and idempotent. A symmetric matrix A is one such that A = A. An idempotent matrix A is one such that A = AA. The only nonsingular idempotent matrix is the identity matrix.

24 Estimation principles 24 Example: The Nerlove model (estimation) The file <nerlovedata.m> contains data on 145 electric utility companies cost of production, output and input prices. The data are for the U.S., and were collected by M. Nerlove. The observations are by row, and the columns are COMPANY CODE, COST (C), OUTPUT (Q), PRICE OF LABOR (P L ), PRICE OF FUEL (P F ) and PRICE OF CAPITAL (P K ). Data Example: Let s MATLAB works...

25 Estimation principles 25 We will estimate the Cobb-Douglas model: using OLS. ln C = β 1 + β 2 ln Q + β 3 ln P L + β 4 ln P F + β 5 ln P K + ɛ (3) The <prg1sl1.m> results are the following: % OLS estimation (using the formula). B =

26 Estimation principles Company cost - logs Residuals Fitted values Fitted values Do the theoretical restrictions hold? Does the model fit well? What do you think about CRTS? Influential observations or outliers?

27 Estimation principles 27 Influential observations and outliers: The OLS estimator of the i th element of the vector β is simply: ˆβ i = [ (X X) 1 X ] y i = c iy Then ˆβ i variable. is a linear estimator, i.e., it s a linear function of the dependent Since it s a linear combination of the observations on the dependent variable, where the weights are determined by the observations on the regressors, some observations may have more influence than others.

28 Estimation principles 28 Influential observations and outliers (2): Define h t = (P X ) tt = e tp X e t = P X e t 2 = e t 2 = 1 h t is the t th element on the main diagonal of P X and e t is t th -unit vector. So 0 < h t < 1, and Trace(P X ) = k h = k/n. So, on average, the weight on the y t s is k/n. If the weight is much higher, then the observation is potentially influential. However, an observation may also be influential due to the value of y t, rather than the weight it is multiplied by, which only depends on the x t s.

29 Estimation principles 29 Influential observations and outliers (3): Consider the estimation of β without using the t th observation (denote this estimator by ˆβ (t) ). Then, ˆβ (t) = ˆβ ( ) 1 (X X) 1 X 1 h tˆε t t so the change in the t th observations fitted value is X t ˆβ Xt ˆβ(t) = ( ht 1 h t ) ˆε t While an observation may be influential if it doesn t affect its own fitted value, it certainly is influential if it does. A fast means of identifying influential observations is to plot function of t. ( ht 1 h t ) ˆε t as a

30 Estimation principles 30 Influential observations and outliers (4): The following example is based on Table 13.8 of Page 423 in Peña, D. (1995) Estadística. Modelos y Métodos. It consists on three data sets that differ in only one observation (see file <penadata.m>). Data Set ˆβ1 ˆβ2 ˆβ3 ŝ ε R 2 h t ˆε t ( ) ht 1 h ˆε t t Original A B C A is an outlier but is not an influential observation. B is an influential but is not an outlier observation. C is an outlier influential observation.

31 Estimation principles 31 Example: The Nerlove model (influential observations) 6 Company cost - logs Fitted values Leverage Influence

32 Estimation principles 32 Example: The Nerlove model (outliers observations) 5 Residuals Standardized residuals Residuals Fitted values Standardized residuals: r t = ˆε t ŝ ε 1 ht t n k 2.

33 Estimation principles 33 Example: The Nerlove model (outliers observations) 5 4 Warning: Multiple testing Residuals Standardized residuals 3 2 Residuals Fitted values Instead of using the α-critical value, we will use α/n-critical value.

34 Estimation principles 34 Properties of OLS estimator Small sample properties of the least squares estimator: Unbiasedness: For ˆβ we have ˆβ = (X X) 1 X (Xβ + ε) = β + (X X) 1 X ε, then applying the strong exogeneity assumption and the law of iterated expectation, we have E[(X X) 1 X ε] = E[E[(X X) 1 X ε X]] So the OLS estimator is unbiased. = E[(X X) 1 X E[ε X]] = 0,

35 Estimation principles 35 Small sample properties of the least squares estimator (2): Normality: ˆβ X N (β, (X X) 1 σ 2 ), where σ 2 = E[ε 2 t]. Proof: Note that ˆβ = β + (X X) 1 X ε, then ˆβ is a linear function of ε. It only rest to obtain the conditional variance. Var( ˆβ X) = E[( ˆβ β)( ˆβ β) X] = E[(X X) 1 X εε XX X) 1 X] = (X X) 1 X E[εε X] XX X) 1 = (X X) 1 σ 2. Notice that the above results is true regardless the distribution of ε.

36 Estimation principles 36 Small sample properties of the least squares estimator (3): Efficiency (Gauss-Markov theorem): In the classical linear regression model, the OLS estimator, ˆβ, is the minimum variance linear unbiased estimator of β. Moreover, for any vector of constants w, the minimum variance linear unbiased estimator of w β is w ˆβ. Proof: The OLS estimator is a linear estimator, i.e., it is a linear function of the dependent variable. It is also unbiased, as we proved two slides ago. Now, consider a new linear estimator β = Cy. If this estimator is unbiased, then we must have CX = I since E[Cy] = E(CXβ + Cε) = CXβ Therefore, CX = I.

37 Estimation principles 37 The variance of an unbiased β is V ( β) = CC σ 2. Define D = C (X X) 1 X Since CX = I, then DX = 0 so Var( β X) = ( D + (X X) 1 X ) ( D + (X X) 1 X ) σ 2 ( = DD + (X X) 1) σ 2 = DD σ 2 + Var( ˆβ X) Finally, Var( β) V ( ˆβ). Notice that the above results is true regardless the distribution of ε.

38 Estimation principles 38 Small sample properties of the least squares estimator (3): Unbiasedness of ˆσ 2 = 1 n k ˆε ˆε = 1 n k ε M X ε, where M X is idempotent matrix that projects onto the space orthogonal to the span of X and ˆε = M X y. Proof: E[ˆσ 2 ] = = = = = σ 2. 1 n k E[Trace(ε M X ε)] 1 n k E(Trace(M Xεε )] 1 n k Trace(E[M Xεε ]) 1 n k σ2 Trace(M X )

39 Resampling methods in i.i.d. data. 39 Resampling methods - Introduction Problem: Let X = (X 1,..., X N ) be observations generated by a model P, and let T (X) be the statistics of interest that estimate the parameter θ. We want to know: Bias: b T = E [T (X)] θ, Variance: v T, Distribution: L(T, P).

40 Resampling methods in i.i.d. data. 40 Resampling methods in i.i.d. data Jackknife: Let X = (X 1, X 2,..., X N ) be a sample of size N and let T N = T N (X) be an estimator of θ. In the jackknife samples an observation of X is excluded each time, i.e. X (i) = (X 1, X 2,..., X i 1, X i+1,..., X N ) for i = 1, 2,..., N, and we calculate the i-th jackknife statistic T N 1,i = T N 1 ( X(i) ). Jackknife bias estimator: Jackknife variance estimator: b Jack = (N 1)( T N T N ) v Jack = N 1 N N ( TN 1,i T ) 2 N, i=1 where T N = N 1 N i=1 T N 1,i

41 Resampling methods in i.i.d. data. 41 Jackknife Outline of the resampling procedure: (X 1,..., X N ) X\X 1 T N 1,1 X\X 2 T N 1, X\X N T N 1,N b Jack = (N 1)( T N T N ) N ( v Jack = N 1 N TN 1,i T ) 2 N i=1 Possible uses: Bias reduction: T Jack = T N b Jack. If N(T N θ) is asymptotically normal distributed, then we can obtain an estimate of the asymptotic variance by v Jack.

42 Resampling methods in i.i.d. data. 42 Example # 1: Let X 1,..., X N be i.i.d. N (µ, σ 2 ) observations and T N = X be the statistic of interest. In this case, we know that: b T = 0, v T = σ2 N, σ2 L(T ) = N (µ, N ). Jackknife results: N = 100, µ = 0 and σ 2 = 1. jackknife(data = x, statistic = mean) Number of Replications: 100 Summary Statistics: Observed Bias Mean SE mean e b jack = 1.374e and v jack = 0, =

43 Resampling methods in i.i.d. data. 43 d-jackknife: Let S n,r be the subsets of {1, 2,..., N} of size r. For any s = {i 1, i 2,..., i r } S N,r we obtain the d-jackknife replica by T r,s c = T r (X i1, X i2,..., X ir ). d-jackknife variance estimator: v Jack d = r dc s S N,r ( T r,s c C 1 s S N,r T r,s c) 2, where C = ( N d ). d-jackknife as estimator of the distribution of T N : Let H n (x) = Pr{ n(t n θ) x} be the distribution we want to estimate. We define the jackknife histogram by: H Jack (x) = 1 C s S N,d I ( Nr/d(Tr,s T N ) x). To learn more: Shao and Tu (1995).

44 Resampling methods in i.i.d. data. 44 Example # 2: We consider the following statistics: T n = X 2 n and the sampling median T n = F 1 n ( 1 2 ) = Q 2 : [ ] RB = Ê vjack v T v T and RM = Ê [ (v Jack v T ) 2].

45 Resampling methods in i.i.d. data. 45 Resampling methods in i.i.d. data Bootstrap: Let X = (X 1,..., X N ) be observations generated by model P and let T (X) be the statistic whose distribution L(T, P ) we want to estimate. The bootstrap proposes the distribution L (T ; P N ) of T = T (X ) as estimator of L(T, P ), where X are observations generated by the estimated model PN. Possible estimators of P in the i.i.d. case: Standard bootstrap: F N (x) = N 1 N i=1 I (X i x). Parametric bootstrap: F ϑ (Its assumed that P = F ϑ ). Smoothed bootstrap: F n,h is a kernel estimator of F. To learn more: Efron y Tibshirani (1993), Shao and Tu (1995), and Davison and Hinkley (1997).

46 Resampling methods in i.i.d. data. 46 Standard bootstrap Outline of the resampling procedure: (X 1,..., X N ) F N (X (1) 1,..., X (1) N ) T (1) N (X (2) 1,..., X (2) N ) T (2) N (X (B) 1,..., X (B) N ) T (B) N b Boot = E [T N ] T N v Boot = E [ (T N E [T N ])2] L T (x) = Pr {(T N T N) x} where E and Pr are the bootstrap expectation and the bootstrap probability. Remark: The X i are i.i.d. F N observations, therefore the resampling procedure could be interpreted as a sampling with replacement from the original sample (X 1,..., X N ).

47 Resampling methods in i.i.d. data. 47 Example # 1 (cont.): T N = X and B = 1000 resamples of size N. bootstrap(data = x.f, statistic = mean) Number of Replications: 1000 Summary Statistics: Observed Bias Mean SE mean Empirical Percentiles: 2.5% 5% 95% 97.5% mean b boot = y v boot = 0, =

48 Resampling methods in i.i.d. data. 48 Example # 1 (cont.): mean mean Density Quantiles of Replicates Value Quantiles of Standard Normal

49 Resampling methods in i.i.d. data. 49 Example # 3: Let X 1,..., X N be i.i.d. observations N (µ = 0, σ 2 = 1) and T N = Q 2. In this case, we know that the asymptotic distribution of N( Q2 Q 2 ) is N (0, 1/4φ(0) 2 ) and v T 1/( ) = Jackknife results: jackknife(data = x.f, statistic = median) Number of Replications: 100 Summary Statistics: Observed Bias Mean SE median e Empirical Percentiles: 2.5% 5% 95% 97.5% median

50 Resampling methods in i.i.d. data. 50 Example # 3 (cont.): Bootstrap results: bootstrap(data = x.f, statistic = median) Number of Replications: 1000 Summary Statistics: Observed Bias Mean SE median , Empirical Percentiles: 2.5% 5% 95% 97.5% median v jack = and v boot =

51 Resampling methods in i.i.d. data. 51 Example # 3 (cont.): median median median Density Density Density Value Value Value

52 Resampling methods in i.i.d. data. 52 Bootstrap always? One example where the bootstrap fails: Let X 1,..., X N be i.i.d. U(0, θ). We know that the m.l.e. of θ is T N = θ = max 1 i N X i and that the density function of T N is: f θ(x) = { nx n 1 θ n if 0 x θ 0 otherwise Bootstrap results: bootstrap(data = x.f2, statistic = max) Summary Statistics: Observed Bias Mean SE max Empirical Percentiles: 2.5% 5% 95% 97.5% max

53 Resampling methods in i.i.d. data. 53 Bootstrap always? One example where the bootstrap fails (cont.): Funcion de densidad de Max(X_i) max f.y Density f.x Value Pr{ θ = θ} = 0, but Pr { θ = θ} = 1 (1 1/n) n 1 e 1.

54 Resampling methods in i.i.d. data. 54 Bootstrap always? One example where the bootstrap fails (cont.): A bootstrap solution is the parametric bootstrap: Funcion de densidad de Max(X_i) Param f.y Density f.x Value

55 Resampling methods in i.i.d. data. 55 ( Bootstrap in Linear Regression: Let (Y1, X 1,1,..., X 1,k ), (Y 2, X 2,1,..., X 2,k ),..., (Y N, X N,1,..., X N,k ) ) be observations generated by the model: Y i = β 0 + β 1 X i,1 + + β k X i,k + ε i, where the ε i are i.i.d. with E[ε i ] = 0, and E[ε 2 i ] = σ2. Resampling of residuals (based on model): 1. We generate resamples ε 1, ε 2,..., ε N i.i.d. from F N, ε. 2. We construct bootstrap replicas by: Y i = β 0 + β 1 X i,1 + + β k X i,k + ε i, where the β i are estimates of the parameters β i. 3. Finally, we compute the statistic of interest in the bootstrap replicas.

56 Resampling methods in i.i.d. data. 56 Bootstrap in Linear Regression: Resampling of pairs (not based on model): 1. We generate bootstrap replicas (Y 1, X 1,1,..., X 1,k ), (Y 2, X 2,1,..., X 2,k ),..., (Y N, X N,1,..., X N,k )) i.i.d. from F N,(Y,X1,...,X k ). 2. We calculate the statistic of interest in the bootstrap replicas. Remark: The bootstrap observations in the resampling of pairs could be interpreted as a sampling with replacement from the N vectors p 1 in the original data. Remark: What happens if ε i is not an i.i.d. sequence? For example, if the ε i is an AR(1) process? or if it is an heteroscedastic sequence?

57 Resampling methods in i.i.d. data. 57 Example # 4: The data are taken from an operation of a plant for the oxidation of ammonia to nitric acid, measured on 21 consecutive days (Chapter 6 of Draper and Smith (1966)): loss Air.Flow Water.Temp Acid.Conc. percent of ammonia lost air flow to the plant cooling water inlet temperature acid concentration as a percentage Bootstrap results: bootstrap(data = stack, statistic = coef(lm(stack.loss ~ Air.Flow + Water.Temp + Acid.Conc., stack)), B = 1000, seed = 0, trace = F) Summary Statistics: Observed Bias Mean SE (Intercept) Air.Flow Water.Temp Acid.Conc

58 Resampling methods in i.i.d. data. 58 Example # 4 (cont): Summary Statistics: Observed Bias Mean SE (Intercept) Air.Flow Water.Temp Acid.Conc Empirical Percentiles: 2.5% 5% 95% 97.5% (Intercept) Air.Flow Water.Temp Acid.Conc

59 Resampling methods in i.i.d. data. 59 Example # 4 (cont): (Intercept) Air.Flow Density Density Value Water.Temp Value Acid.Conc. Density Density Value Value