Statistics II. Management Degree Management Statistics IIDegree. Statistics II. 2 nd Sem. 2013/2014. Management Degree. Simple Linear Regression

Transcription

1

2 Model 1 2 Ordinary Least Squares 3 4 Non-linearities 5 of the coefficients and their

3 to the model We saw that econometrics studies E (Y x). More generally, we shall study regression analysis. : The regression of Y on X is any characteristic of the conditional distribution f (y x) expressed as a function of x.

4 to the model Frequently the term regression is incorrectly used to denote other quantities. We could be interested in other measures of location, say for instance the median. In this course we shall focus our attention mainly on the conditional expected value.

5 Choice of the conditional expected value Why? Because E (Y x) is the function of x that minimises [ E (Y h (x)) 2]. (The median minimises E [ Y h (x) ], and it allows for non-symmetric loss functions.) Let us start with the simplest case of a univariate x (this is helpful as it allows a simple graphical representation).

6 Definition of the model We need to define three items: 1. The functional form of E (Y x) (we shall assume linearity) 2. How to treat the effect of the remaining variables on Y 3. How to perform a ceteris paribus analysis

7 Definition of the model Our starting point is: E (Y x) = β 0 + β 1 x Y = β 0 + β 1 x + u Example: wage as a function of schooling

8 Y E (Y x) = u Properties 1. E (u x) = 0 2. E (u) = 0 3. Var (u x) = σy 2 x 4. Cov (u, x) = 0 5. Cov (u, g (x)) = 0 Only the deviations of Y from E (Y x) have property 5.

9 Observations In the case of the conditional expected value, a ceteris paribus inferential analysis is possible, since we can consider variations of x with constant u. The problem is that many times the economic analysis is not interested in the parameters appearing in conditional expected values, or at least not in those we can estimate.

10 Model s hypotheses SLR.1: Model linear in the parameters Y = β 0 + β 1 x + u SLR.2: Random sample (i.i.d.) {(y i, x i ) : i = 1... n} y i = β 0 + β 1 x i + u i SLR.3: Sample variation: x i is not constant in the sample SLR.4: Errors uncorrelated with x: E (u x) = 0 SLR.5: Homoskedasticity: Var (u x) = σ 2 constant

11 Notice that E (u) = 0 and that Cov (u, x) = E (xu) = 0 Applying the method of moments: 1 ( y i n ˆβ 0 ˆβ ) 1 x i = 0 1 ( ) y i ˆβ 0 ˆβ 1 x i x i = 0 n

12 From the first equation we obtain ȳ = ˆβ 0 + ˆβ 1 x ˆβ 0 = ȳ ˆβ 1 x

13 Substituting into the second equation 1 ( ( y i ȳ n ˆβ ) 1 x ˆβ ) 1 x i x i = 0 1 (yi ȳ) x i = 1 ˆβ 1 (x i x) x i n n ˆβ 1 = (yi ȳ) x i (xi x) 2 = (yi ȳ) (x i x) (xi x) 2 This expression corresponds to to the ratio between the sample covariance of y and x and the sample variance of x (necessarily positive).

14 The fitted values of y are given by: ŷ = ˆβ 0 + ˆβ 1 x i which is called regression line. From ŷ, the residuals can be computed: ( ) û i = y i ŷ i = y i ˆβ 0 + ˆβ 1 x i

15 Consider now the sum of the squared residuals: SSR = ( ) 2 y i ˆβ 0 ˆβ 1 x i The ordinary least squares ˆβ 0 and ˆβ 1 minimise the SSR.

16 To prove the above, consider SSR = ( y i ˆβ 0 ˆβ 1 x i ) 2 SSR ˆβ 0 = 2 ( y i ˆβ 0 ˆβ 1 x i ) = 0 SSR ˆβ 1 = 2 ( y i ˆβ 0 ˆβ 1 x i ) x i = 0 You can find the second order condition in the book.

17 Properties: 1 1. ûi = 0 n 2. 1 xi û i = 0 n 3. ȳ = ˆβ 0 + ˆβ 1 x 4. ŷ = ȳ 5. 1 ŷi û i = 0 n

18 Important comment Noticing that y i = ŷ i + û i, this last property: 1 ŷi û i = 0 n implies that the least squares method provides a decomposition of y i into two parts that are orthogonal in the sample.

19 Decomposition of y i

20 Consider now the following quantities: Total sum of squares - total variation in dependent variable SST = (y i ȳ) 2 Explained sum of squares - variation explained by regression SSE = (ŷ i ȳ) 2 Residual sum of squares - variation not explained by regression SSR = ûi 2

21 It is possible to prove that SST = SSE + SSR. This decomposition allows to evaluate the goodness of fit using the following coefficient: R 2 = SSE SST = 1 SSR SST 0 R 2 1 In the simple linear regression model R 2 = ρ 2 y,x.

22 SST = (y i ȳ) 2 = [(y i ŷ i ) + (ŷ i ȳ)] 2 = [û i + (ŷ i ȳ)] 2 = û 2 i + 2 û i (ŷ i ȳ) + (ŷ i ȳ) 2 = SSR + 2 û i (ŷ i ȳ) + SSE From properties 1 to 5 it is easy to see that û i (ŷ i ȳ) = 0, so that SST = SSE + SSR.

23 Example

24 Example

25 Example

26 Change of location and scale What happens when we change the unit of measurement of the variables? Y = β 0 + β 1 α (αx) + u αy = αβ 0 + αβ 1 x + αu αy + θ = (αβ 0 + θ) + αβ 1 x + αu In any of the above cases the R 2 coefficient remains the same.

27 Non-linearities So far, it looked like the kind of model we considered, namely Y = β 0 + β 1 x + u only allows to describe linear relationships. Luckily, this is not the case, and the model can accommodate for non-linear relationships between y and x (as long as it keeps being linear in the parameters).

28 Non-linearities For example, the model wage = β 0 + β 1 edu + u implies that any added year of schooling has the same effect on wage. This is not very close to reality! Luckily, the model only needs to be linear in the parameters, but not in the variables, so we can consider more sensible alternatives.

29 Non-linearities Consider for instance: ln (wage) = β 0 + β 1 edu + u Now: edu = 1 = ln (wage) = β 1 Since ln (wage) wage wage this specification of the model implies that any additional year of schooling has the same percentage effect on wage.

30 Example 2.10

31 Non-linearities Consider now a demand function of the type Q = β 0 + β 1 P Elasticity in this model is given by ε Q,P = β 1P β 0 + β 1 P

32 Non-linearities In order to obtain a constant elasticity, we can use the following specification: ln (Q) = β 0 + β 1 ln (P) Q = exp (β 0 + β 1 ln (P)) Q P = β 1 P Q = ε Q,P = β 1Q P P Q = β 1 This model falls within our framework, because it could be linearised.

33 Example 2.11

34 Non-linearities Some models cannot be linearised. An example is the following specification of the probability of an individual being vegetarian as a function of age: Pr = exp (β 0 + β 1 age) 1 + exp (β 0 + β 1 age)

35 of the coefficients and their Case 1: lin-lin model Consider the model y = β 0 + β 1 x + u or E(Y x) = β 0 + β 1 x estimated by ŷ = ˆβ 0 + ˆβ 1 x

36 of the coefficients and their From the above we obtain: so that E(Y x) = β 1 x ŷ = ˆβ 1 x x = 1 = Cp E(Y x) = β 1 and ŷ = ˆβ 1

37 of the coefficients and their Example: lin-lin model educ = 1 = Cp ŝal = educ ŝal = 0.64

38 of the coefficients and their Case 2: log-linear model Consider the model estimated by log(y) = ˆβ 0 + ˆβ 1 x Now a change in x has the following effect: x = 1 = Cp log(y) = ˆβ 1

39 of the coefficients and their But we can approximate the log function as follows: so that log(y) ŷ ŷ log(y) 100% ŷ ŷ 100% = % ŷ

40 of the coefficients and their Substituting into the effect of a change in x we obtain x = 1 = Cp % ŷ ˆβ 1 100% which represents an approximate semi-elasticity.

41 of the coefficients and their Example: log-linear model log(sal) = educ educ = 1 = Cp % ŝal 10%

42 of the coefficients and their Case 3: log-log model Consider now the model The change log(x) leads to log(y) = ˆβ 0 + ˆβ 1 log(x) log(y) = β 1 log(x) log(y) log(x) = β 1 elasticity % x = 1% = Cp % y β 1 %

43 of the coefficients and their Example: log-log model log(price) = log(sqrft) % sqrft = 1% = Cp % price 0.70%

44 of the coefficients and their Case 4: lin-log model In the model: ŷ = ˆβ 0 + ˆβ 1 log(x) A change log(x) gives If % x = 1% then ŷ = ˆβ 1 log(x) log(x) 100 = 1 log(x) = ŷ = β 1 100

45 of the coefficients and their Example: lin-log model price = log(sqrft) % sqrft = 1% = Cp price

46 Table 2.3

47 Exercise 2.6