MBF1923 Econometrics Prepared by Dr Khairul Anuar

Transcription

1 MBF1923 Econometrics Prepared by Dr Khairul Anuar L4 Ordinary Least Squares

2 Ordinary Least Squares The bread and butter of regression analysis is the estimation of the coefficient of econometric models with a technique called ordinary least squares. Regression users rely on computers to do the actual OLS calculations, so the emphasis here is on understanding what OLS attempts to do and how it goes about doing it. 1

3 Estimating Single-Independent-Variable Models with OLS Recall that the objective of regression analysis is to take a purely theoretical equation like: And, use a set of data to create an estimated equation like: (2.1) (2.2) Recall that equation 2.1 is purely theoretical, while equation (2.2) is it empirical counterpart. Note: each hat indicates a sample estimate of the true population value 2

4 Estimating Single-Independent-Variable Models with OLS (cont.) How to move from (2.1) to (2.2)? One of the most widely used methods is Ordinary Least Squares (OLS) OLS minimizes (i = 1, 2,., N) (2.3) Or, the sum of squared deviations of the vertical distance between the residuals (i.e. the estimated error terms) and the estimated regression line We also denote this term the Residual Sum of Squares (RSS) 3

5 Estimating Single-Independent-Variable Models with OLS (cont.) Similarly, OLS minimizes: N i Y Yˆ ( i i 2 ) Why use OLS? Relatively easy to use The goal of minimizing RSS is intuitively / theoretically appealing This basically says we want the estimated regression equation to be as close as possible to the observed data OLS estimates have a number of useful characteristics 4

6 Estimating Single-Independent-Variable Models with OLS (cont.) OLS estimates have at least two useful characteristics: The sum of the residuals is exactly zero OLS can be shown to be the best estimator when certain specific conditions hold (we ll get back to this in Chapter 4) Ordinary Least Squares (OLS) is an estimator A given produced by OLS is an estimate Estimator is a mathematical technique that is applied to a sample of data to produce realworld numerical estimates of the true population) Why not just total the residuals? 5

7 Estimating Single-Independent- Variable Models with OLS (cont.) How does OLS work? How would OLS estimate a single-independent-variable regression model (Recall equation 2.1) For an equation with just one independent variable, the coefficients are: (2.4) (2.5) 6

8 Estimating Multivariate Regression Models with OLS The general multivariate regression model with K independent variables is: Y i = β 0 + β 1 X 1i + β 2 X 2i β K X Ki + ε i (i = 1,2,,N) (1.13) Biggest difference with single-explanatory variable regression model is in the interpretation of the slope coefficients Now a slope coefficient indicates the change in the dependent variable associated with a one-unit increase in the explanatory variable holding the other explanatory variables constant 7

9 Estimating Multivariate Regression Models with OLS In the real world one explanatory variable is not enough The general multivariate regression model with K independent variables is: Y i = β 0 + β 1 X 1i + β 2 X 2i β K X Ki + ε i (i = 1,2,,N) (1.13) Biggest difference with single-explanatory variable regression model is in the interpretation of the slope coefficients Now a slope coefficient indicates the change in the dependent variable associated with a one-unit increase in the explanatory variable holding the other explanatory variables constant 8

10 Estimating Multivariate Regression Models with OLS (cont.) Omitted (and relevant!) variables are therefore not held constant The intercept term, β 0, is the value of Y when all the Xs and the error term equal zero Nevertheless, the underlying principle of minimizing the summed squared residuals remains the same 9

11 Example: financial aid awards at a liberal arts college Dependent variable: FINAID i : financial aid (measured in dollars of grant) awarded to the ith applicant 10

12 Example: financial aid awards at a liberal arts college Theoretical Model: (2.9) where: (2.10) PARENT i : The amount (in dollars) that the parents of the ith student are judged able to contribute to college expenses HSRANK i : The ith student s GPA rank in high school, measured as a percentage (i.e. between 0 and 100) 11

13 Example: financial aid awards at a liberal arts college (cont.) Estimate model using the data in Table 2.2 to get: (2.11) Interpretation of the slope coefficients? Graphical interpretation in Figures 2.1 and

14 Figure 2.1 Financial Aid as a Function of Parents Ability to Pay 13

15 Figure 2.2 Financial Aid as a Function of High School Rank 2011 Pearson Addison-Wesley. All rights reserved. 14

16 Total, Explained, and Residual Sums of Squares (2.12) (2.13) TSS = ESS + RSS This is usually called the decomposition of variance TSS= Total sum of squares ESS=Explained sum of squares RSS=Residual sum of squares 15

17 Figure 2.3 Decomposition of the Variance in Y 2011 Pearson Addison-Wesley. All rights reserved. 16

18 Evaluating the Quality of a Regression Equation Checkpoints here include the following: 1. Is the equation supported by sound theory? 2. How well does the estimated regression fit the data? 3. Is the data set reasonably large and accurate? 4. Is OLS the best estimator to be used for this equation? 5. How well do the estimated coefficients correspond to the expectations developed by the researcher before the data were collected? 6. Are all the obviously important variables included in the equation? 7. Has the most theoretically logical functional form been used? 8. Does the regression appear to be free of major econometric problems? *These numbers roughly correspond to the relevant chapters in the book 17

19 Describing the Overall Fit of the Estimated Model The simplest commonly used measure of overall fit is the coefficient of determination, R 2 : (2.14) Since OLS selects the coefficient estimates that minimizes RSS, OLS provides the largest possible R2 (within the class of linear models) (R 2 is the ratio of the explained sum of squares to the total sum of squares) 18

20 Figure 2.4 Illustration of Case Where R2 = Pearson Addison-Wesley. All rights reserved. 19

21 Figure 2.5 Illustration of Case Where R2 = Pearson Addison-Wesley. All rights reserved. 20

22 Figure 2.6 Illustration of Case Where R2 = Pearson Addison-Wesley. All rights reserved. 21

23 The Simple Correlation Coefficient, r This is a measure related to R 2 r measures the strength and direction of the linear relationship between two variables: r = +1: the two variables are perfectly positively correlated r = 1: the two variables are perfectly negatively correlated r = 0: the two variables are totally uncorrelated 22

24 The adjusted coefficient of determination A major problem with R 2 is that it can never decrease if another independent variable is added An alternative to R 2 that addresses this issue is the adjusted R 2 or R 2 : Where N K 1 = degrees of freedom (2.15) 2011 Pearson Addison-Wesley. All rights reserved. 2-23

25 The adjusted coefficient of determination (cont.) So, R 2 measures the share of the variation of Y around its mean that is explained by the regression equation, adjusted for degrees of freedom R 2 can be used to compare the fits of regressions with the same dependent variable and different numbers of independent variables As a result, most researchers automatically use instead of R 2 when evaluating the fit of their estimated regressions equations 24

26 Table 2.1a The Calculation of Estimated Regression Coefficients for the Weight/Height Example 25

27 Table 2.1b The Calculation of Estimated Regression Coefficients for the Weight/Height Example 26

28 Table 2.2a Data for the Financial Aid Example 27

29 Table 2.2b Data for the Financial Aid Example 28

30 Table 2.2c Data for the Financial Aid Example 29

31 Table 2.2d Data for the Financial Aid Example 30