5.1 Model Specification and Data 5.2 Estimating the Parameters of the Multiple Regression Model 5.3 Sampling Properties of the Least Squares

Transcription

1

2 5.1 Model Specification and Data 5. Estimating the Parameters of the Multiple Regression Model 5.3 Sampling Properties of the Least Squares Estimator 5.4 Interval Estimation 5.5 Hypothesis Testing for a Single Coefficient 5.6 Measuring Goodness-of-Fit

3 S = β+β 1 P+β3A β = the change in monthly sales S ($1000) when the price index P is increased by one unit ($1), and advertising expenditure A is held constant = ΔS ΔP ( A held constant) S = P β 3 = the change in monthly sales S ($1000) when advertising expenditure A is increased by one unit ($1000), and the price index P is held constant = ΔS ΔA ( P held constant) S = A

4 Figure 5.1 The multiple regression plane Slide 5-4

5 Slide 5-5

6 S = E( S ) + e =β +β P +β A + e i i i 1 i 3 i i The introduction of the error term, and assumptions about its probability distribution, turn the economic model into the econometric model in (5.).

7 y =β +β x +β x + L+β x + e i 1 i 3 i3 K ik i ΔE y β k = = Δx ( ) ( ) k other x's held constant E y x k y = β+β x +β x + e i 1 i 3 i3

8 1. Ee ( i ) = 0 Each random error has a probability distribution with zero mean. Some errors will be positive, some will be negative; over a large number of observations they will average out to zero.

9 . var( e i ) =σ Each random error has a probability distribution with variance σ. The variance σ is an unknown parameter and it measures the uncertainty in the statistical model. It is the same for each observation, so that for no observations will the model uncertainty be more, or less, nor is it directly related to any economic variable. Errors with this property are said to be homoskedastic.

10 3. cov( e, e ) = 0 i j The covariance between the two random errors corresponding to any two different observations is zero. The size of an error for one observation has no bearing on the likely size of an error for another observation. Thus, any pair of errors is uncorrelated.

11 4. ( ) e ~ N 0, σ i We will sometimes further assume that the random errors have normal probability distributions.

12 The statistical properties of y i follow from the properties of e i. 1. E( y ) = β+β x +βx i 1 i 3 i 3 The expected (average) value of y i depends on the values of the explanatory variables and the unknown parameters. It is equivalent to Ee ( i ) = 0. This assumption says that the average value of y i changes for each observation and is given by the regression E( y i ) function = β+β 1 x i +β3x i 3.

13 . var( y ) = var( e) =σ i i The variance of the probability distribution of y i does not change with each observation. Some observations on y i are not more likely to be further from the regression function than others.

14 3. cov( y, y ) = cov( e, e ) = 0 i j i j Any two observations on the dependent variable are uncorrelated. For example, if one observation is above E(y i ), a subsequent observation is not more or less likely to be above E(y i ).

15 4. yi ~ N ( β+β 1 xi +β3xi3), σ We sometimes will assume that the values of y i are normally distributed about their ( ) e ~ N 0, σ mean. This is equivalent to assuming that i.

16 Assumptions of the Multiple Regression Model MR1. MR. MR3. MR4. MR5. MR6. yi =β 1+β xi + L+β KxiK + ei, i= 1, K, N E( y ) =β +β x + L+β x E( e) = 0 i 1 i K ik i var( y ) = var( e) =σ i cov( y, y ) = cov( e, e ) = 0 i j i j i The values of each x tk are not random and are not exact linear functions of the other explanatory variables y N x x e N i ~ ( β 1+β i + L+βK ik), σ i ~ (0, σ ) Slide 5-16

17 y = β+β x +β x + e i 1 i 3 i3 N S β, β, β = y E( y ) ( ) ( ) 1 3 i i i= 1 N = y β β x β x ( ) i 1 i 3 i3 i= 1

18 Slide 5-18

19 E( y ) = β+β x +βx i 1 i 3 i 3 yˆ = b + b x + b x i 1 i 3 i3 = x x i i3 Sˆ = P A i i i SALES = PRICE ADVERT

20 Sˆ = PRICE ADVERT = = Slide 5-0

21

22 σ = var( e) = E( e ) i i 垐 i = i i = i 1+ i + 3 i3 e y y y b b x b x ( ) σ ˆ = N i= 1 N eˆ i K

23 75 eˆ i i= ˆ σ = = = N K 75 3 SSE N = e = i= 1 ˆi σ= ˆ =

24 Slide 5-4

25 var( b ) = σ N 3 i i= 1 (1 r ) ( x x ) r 3 = ( x x )( x x ) i i3 3 ( x x ) ( x x ) i i3 3

26 1. Larger error variances σ lead to larger variances of the least squares estimators.. Larger sample sizes N imply smaller variances of the least squares estimators. 3. More variation in an explanatory variable around its mean, leads to a smaller variance of the least squares estimator. 4. A larger correlation between x and x 3 leads to a larger variance of b. Slide 5-6

27 The covariance matrix for K=3 is ( b1) ( b1 b) ( b1 b3) ( ) ( ) ( ) ( b b ) ( b b ) ( b ) var cov, cov, cov ( b1, b, b3) = cov b1, b var b cov b, b3 cov 1, 3 cov, 3 var 3 The estimated variances and covariances in the example are cov ( b1, b, b3) = Slide 5-7

28 Therefore, we have var b = cov b, b = ( ) ( ) 1 1 var b = 1.01 cov b, b =.7484 ( ) ( ) 1 3 var b =.4668 cov b, b =.0197 ( ) ( ) 3 3 Slide 5-8

29 Slide 5-9

30 The standard errors are se( b) = var( b) = = se( b ) = var( b ) = 1.01 = se( b ) = var( b ) =.4668 = Slide 5-30

31 y =β +β x +β x + L+β x + e i 1 i 3 i3 K ik i y N x L x e N i ~ ( β+β 1 i + +βk ik), σ i ~ (0, σ) b ~,var( ) k N β k bk Slide 5-31

32 bk βk z = ~ N( 0,1 ), for k = 1,,, K var b ( ) k t b β = = var b ( ) b β se( b ) k k k k k k ~ t ( N K) Slide 5-3

33 P( t < t < t ) =.95 c (7) c b β P = se( b ) [ ] Pb se( b) β b se( b) =.95 [ b se( b ), b se( b )] Slide 5-33

34 A 95% interval estimate for β based on our sample is given by ( 10.09, 5.74) A 95% interval estimate for β 3 based on our sample is given by ( , ) = (.501, 3.4) The general expression for a 100(1 α)% confidence interval is [ b t se( b ), b + t se( b ) k (1 α/, N K) k k (1 α/, N K) k Slide 5-34

35 STEP-BY-STEP PROCEDURE FOR TESTING HYPOTHESES 1.Determine the null and alternative hypotheses..specify the test statistic and its distribution if the null hypothesis is true. 3.Select α and determine the rejection region. 4.Calculate the sample value of the test statistic and, if desired, the p-value. 5.State your conclusion. Slide 5-35

36 t = H : 0 0 β k = H : 0 1 βk bk se b ( ) k ~ t ( N K) For a test with level of significance α t = t and t = t c (1 α/, N K) c ( α/, N K) Slide 5-36

37 Big Andy s Burger Barn example 1. The null and alternative hypotheses are: H : β = 0 and H : β The test statistic, if the null hypothesis is true, is ( ) t = b se b ~ t N K ( ) 3. Using a 5% significance level (α=.05), and 7 degrees of freedom, the critical values that lead to a probability of 0.05 in each tail of the distribution are t = and t = (.975, 7) (.05, 7) Slide 5-37

38 4. The computed value of the t-statistic is the p-value in this case can be found as ( ( ) ) Pt 7 ( 7) Pt 10 ( ) t = = > < 7.15 = (. 10 ) = Since , we reject H : β = 0 and conclude that there is evidence < 0 from the data to suggest sales revenue depends on price. Using the p-value to perform the test, we reject H because.000 < Slide 5-38

39 Testing whether sales revenue is related to advertising expenditure 1. H : β = 0 and H : β The test statistic, if the null hypothesis is true, is ( ) t = b se b ~ t N K 3 3 ( ) 3. Using a 5% significance level, we reject the null hypothesis if t or t In terms of the p-value, we reject H 0 if p.05. Slide 5-39

40 Testing whether sales revenue is related to advertising expenditure 4. The value of the test statistic is t = = ; the p-value in given by ( ( ) ) Pt 7 ( 7) Pt.76 ( ) >.76 + <.76 =.004 = Because.76 > 1.993, we reject the null hypothesis; the data support the conjecture that revenue is related to advertising expenditure. Using the p-value we reject H because.008 < Slide 5-40

41 5.5.a Testing for elastic demand We wish to know if β : a decrease in price leads to a decrease in sales revenue (demand is price 0 inelastic), or β < : a decrease in price leads to an increase in sales revenue (demand is price 0 elastic) Slide 5-41

42 1. H : β 0 (demand is unit elastic or inelastic) 0 H1: β < 0 (demand is elastic). To create a test statistic we assume that H : β = 0 is true and use t = b b t se( ) ~ N K ( ) 0 3. At a 5% significance level, we reject H if t or if the p value <.05 0 Slide 5-4

43 4. The value of the test statistic is t b se = = = ( b ) 7.15 The corresponding p-value is Pt ( (7) < 7.15) = Since 7.15 < we reject H : β 0. Since.000 <.05, the same conclusion is reached using the p-value. 0 Slide 5-43

44 b Testing Advertising Effectiveness H : β 1 and H : β > To create a test statistic we assume that H : β = 1 is true and use b3 1 t = ~ t se( b ) 3 ( N K) At a 5% significance level, we reject H if t or if the p value.05 0 Slide 5-44

45 5.5.b Testing Advertising Effectiveness 1. The value of the test statistic is t b β se = = = ( b ) 3 The corresponding p-value is Pt ( (7) > 1.63) = Since 1.63<1.666 we do not reject H 0. Since.105>.05, the same conclusion is reached using the p-value. Slide 5-45

46 R SSR = = SST N i= 1 N ( ) i ( ) i i= 1 yˆ y y y SSE = 1 = 1 SST N i= 1 i ( ) i i= 1 N y eˆ y Slide 5-46

47 y = b + b x + b x + L+ b x ˆi 1 i 3 i 3 k ik σ 1 ˆ = N ( ) SST y yi y N 1 = N 1 i= 1 SST = ( N 1) σˆ y Slide 5-47

48 For Big Andy s Burger Barn we find that SST = = SSE = R eˆ i = 1 = 1 =.448 y y i= 1 N ( ) i i= 1 N Slide 5-48

49 An alternative measure of goodness-of-fit called the adjusted-r, is usually reported by regression programs and it is computed as SSE /( N K) R = 1 SST /( N 1) Slide 5-49

50 If the model does not contain an intercept parameter, then the measure R given in (5.16) is no longer appropriate. The reason it is no longer appropriate is that, without an intercept term in the model, N N N y 垐 i y yi y + ei i= 1 i= 1 i= 1 ( ) ( ) SST SSR + SSE Slide 5-50

51 SALES = PRICE + ADVERT R = (se) (6.35) (1.096) (.683) From this summary we can read off the estimated effects of changes in the explanatory variables on the dependent variable and we can predict values of the dependent variable for given values of the explanatory variables. For the construction of an interval estimate we need the least squares estimate, its standard error, and a critical value from the t-distribution. Slide 5-51

52 Slide 5-5

53 Slide 5-53

54 N S( β, β, β ) = ( y β β x β x ) 1 3 i 1 i 3 i3 i= 1 (A.1) S β 1 S β S β 3 = Nβ + β x + β x y 1 i 3 i3 i = β x + β x + β x x x y 1 i i 3 i i3 i i = β x + β x x + β x x y 1 i3 i i3 3 i3 i3 i Slide 5-54

55 Nb + x b + x b = y 1 i i3 3 i x b + x b + x x b = x y i 1 i i i3 3 i i (5A.1) x b + x x b + x b = x y i3 1 i i3 i3 3 i3 i let y = y y, x = x x, x = x x i i i i i3 i3 3 Slide 5-55

56 b = y b x b x b b = ( yx i i)( xi3 ) ( yx i i3)( xixi3) ( xi )( xi3 ) ( xixi3) = ( yx i i3)( xi ) ( yx i i)( xi3xi) ( xi )( xi3 ) ( xixi3) 3 Slide 5-56