The regression model with one fixed regressor cont d

The regression model with one fixed regressor cont d 3150/4150 Lecture 4 Ragnar Nymoen 27 January 2012

The model with transformed variables Regression with transformed variables I References HGL Ch 2.8 BN: Kap 3 Transformation of the variables prior to estimation is often used in econometric analysis.

The model with transformed variables Regression with transformed variables II Variable transformation is done to 1. suite the purpose of the analysis, or 2. to make the estimated model corespond to the assumptions of the econometric model (in our instance RM1) Later in the course (Topic 9) we will look more in detail on what can be achieved by 2. At this stage we briefly mention the most popular transformations that are used when practising regression analysis.

De-meaning, scaling and standardization De-meaning I We have already encountered de-meaning of x as a way of simplifying some derivations. Have seen that when x i is replaced by (x i x) the only change is in intercept term. De-meaning both variables can be motivated the following way: Consider first the linear relationship y i = β 1 + β 2 x i + e i, i = 1, 2,..., n (1) Take the mean on both sides:

De-meaning, scaling and standardization De-meaning II and subtract from (1) to get: y i ȳ = β 1 + β 2 x + ē = β 2 x i + e i, i = 1, 2,..., n (2) where we define the de-meaned variables: yi xi ei = y i ȳ = x i x = e i ē

De-meaning, scaling and standardization De-meaning III The OLS estimator β 2 from the de-meaned relationship (2) is therefore identical to β 2 from the original model formulation (1). If the classical assumptions of RM1 holds for (1) with disturbance e i, the same is true for (2). DIY exercise 1. Show however: var(e i ) = σ 2 (1 2 1 n + 1 n 2 ) + σ2 n 1 n 2 = σ 2 (1 1 n ) Note the important difference between de-meaning of the variables and the mindless omission of the intercept β 1 from the original equation.

De-meaning, scaling and standardization De-meaning IV The latter forces the regression line through origo and leads to the biased estimator β 2 = n i=1 y i x i n i=1 x 2 i in DYI exercise 4 and 5 in Lecture 3.

De-meaning, scaling and standardization Scaling I This is the situation when we multiply the original variables with the fixed (deterministic) factors ω y and ω x. For example change units from thousand to million or billion. If we again start with (1) and multiply by e.g. ω y : ω y y i = ω y β 1 + ω x β 2 x i + ω y e i, i = 1, 2,..., n let y i, x i and e i denote the scaled variables yi xi ei = ω y y i = ω x x = ω y e i

De-meaning, scaling and standardization Scaling II The equation in terms of these variables becomes: where y i = β 1 + β 2x i + e i, i = 1, 2,..., n (3) β 1 = ω y β 1 (4) β 2 = ω y ω x β 2 (5) σ 2 = ω 2 y σ 2 (6) Scaling of one or both of the variables will affect the OLS estimates.

De-meaning, scaling and standardization Scaling III If for example x i is in thousands, and x i is in millions then ω x = 0.001. If ω y = 1, no scaling of y i, for example ˆβ 2 = 0.005 is changed to ˆβ 2 = 5 after the scaling. If on the other hand, ω x = ω y the slope estimate is unchanged by the scaling, but the intercept and the variance of the disturbance are affected. The statistical properties of the OLS estimators ˆβ 1 and ˆβ 2 are not affected by scaling, since the ei has classical properties. DIY exercise 2 Imagine estimation (1) and then the model with scaled variables (3). Will the coeffi cient of determination (R 2 ) be the same in the two regression? Explain.

De-meaning, scaling and standardization Standardarized variables I Finally imagine first de-meaning y i and x i,and second scaling the de-meaned variables by ω y = 1ˆσ y ω x = 1ˆσ x where ˆσ y and ˆσ x are empirical standard deviations of y and x: ˆσ y = 1 n n i=1 (y i ȳ) 2, and ˆσ x = 1 n n i=1 (x i ˆx) 2

De-meaning, scaling and standardization Standardarized variables II y i x i = y i ȳ ˆσ y = x i x ˆσ x This standardized version of the regression model becomes y i = β 2x i + e i (7) In this interpretation, the estimated ˆβ 2 it is called the beta-coeffi cient. has a separate name,

De-meaning, scaling and standardization Standardarized variables III Since standardization is a combination of de-meaning and scaling we have that ˆβ 2 = ω y ω x ˆβ 2 = ˆσ x ˆσ y ˆβ 2 (8) for the relationship between OLS estimate for the original slope coeffi cient and the beta-coeffi cient.

Non-linear variable transformations Estimating non-linear relationships (with a linear estimator) I By applying a non-linear transformation of y i and x i before estimation we can estimate many interesting non-linear functions with OLS. After transformation the model is linear in the parameters β 1 and β 2. In this way we obtain great flexibility in fitting different non-linear relationships between y and x. The main practical issue is the relevance of the transformation for the economic problem And to be clear about the interpretation of the parameter β 2

Non-linear variable transformations Some popular transformations I Quadratic transform of x y i = β 1 + β 2 x 2 i This equation becomes linear when we define xi = xi 2. The slope coeffi cient β 2 is no longer the derivate. The derivative is instead y x = 2β 2x i which is increasing in x if β 2 > 0. + ε If y is a measure of price or of unit-costs, and x is a measure of production (or of capacity), this model may be relevant to estimate a cost-function. See HGL Figure 2.13 and 2.14

Non-linear variable transformations Some popular transformations II If one or both of the variables are log transformed, we speak of log-linear models. i y i = β 1 + β 2 ln x i + e i ii ln y i = β 1 + β 2 x i + e i iii ln y i = β 1 + β 2 ln x i + e i The two first are sometimes called semi-logarithmic models. The third is sometimes called the log-log model.

Non-linear variable transformations Some popular transformations III All three models have different derivatives, different elasticites El x y and different semi-elasticities y 1 x y y x y 1 x y El x y 1 i β 2 x x β 2 y ii β 2 y β 2 β 2 x y 1 iii β 2 x β 2 x β 2 β 2 y

Non-linear variable transformations GDP per capita GDP per capita in Norway (million fixed kroner) 300000 200000 100000 1840 1860 1880 1900 1920 1940 1960 1980 2000 2020 13 Log GDP per capita in Norway. 12 11 10 1840 1860 1880 1900 1920 1940 1960 1980 2000 2020 Blue graph: GDP per capita Y against time, t Clearly non-linear function Red graph shows ln Y against time. Still not completely linear Better on sub-samples

Non-linear variable transformations 300000 GDP per capita in Norway (million fixed kroner) 200000 100000 1840 1860 1880 1900 1920 1940 1960 1980 2000 2020 13 Log GDP per capita in Norway. 12 11 10 1840 1860 1880 1900 1920 1940 1960 1980 2000 2020

Non-linear variable transformations Attempt semi-log model Y t = Ae g Y t+e t on two sub-samples 1830-1939 1948-2010 Hence estimate ln Y t = β 1 + β 2 trend t + e t, t = 1830,..., to obtain estimates of the semi-elasticity β 2, the growth-rate. Note that trend = is a deterministic variable that take the values trend = 1, 2,..., 181.

Regression with indicator variable (dummies) Dummy variable as regressor I Often the x variable is a qualitative explanatory variable For example: Availability effect on alcohol consumption y i. Make this operational by defining an indicator variable, also called a dummy: { 1 if pol in town x i = 0 else We estimate the model y i = β 1 + β 2 x i + e i Under the assumption of RM1, the classical assumptions in particular, the OLS estimators will be BLUE as before.

Regression with indicator variable (dummies) Dummy variable as regressor II The interpretation of β 1 changes: It is no longer a slope coeffi cient. Instead it measures the difference in mean alcohol consumption between pol and no-pol towns. Starting from the standard expressions for the OLS estimators, they can be shown to reduce to ˆβ 1 = ȳ 0 ˆβ 2 = ȳ 1 ȳ 0 where ȳ 0 is the average consumption in no-pol towns and ȳ 1 is the is the average consumption in pol towns.

Regression with indicator variable (dummies) Dummy variable as regressor III The expression for ˆβ 1 has been rejuvenated in recent years, due to the availability of large data set that allow the estimation of treatment-effects. It is then referred to as the difference-estimator The use of dummies as regressors will be developed in interesting directions when we come to multiple regression models. NOTE: While dummies are unproblematic to use as left hand side variables in RM1, it is a different matter when the dependent variable is an qualitative variable. A different econometric model than RM1 is then required!

Statistical inference in the regression model Starting Topic 3 on the lecture plan HGL Chap 3 BN Kap 5.4-5.5. (building on kap 4.5-4.6) Review of course in probabillty theory: One important form of statistical inference is that we form a hypothesis about the value of a parameter θ in the probability distribution ( fordelingsfunksjonen ) to a stochastic variable. Let θ 0 denote the hypothesized parameter value The idea is that we place our hypothesis about the value of θ in danger of being rejected, it is the null hypothesis, H 0 : θ = θ 0. The alternative hypothesis may be one-sided, for example H 1 : θ > θ 0 or two-sided H 1 : θ = θ 0.

Statistical inference in the regression model Hence, after specification of both the null-hypothesis and H 0 and the alternative, we have for example H 0 : θ = θ 0 against H 1 : θ = θ 0. for the case of a two-sided alternative. The next step is to use the assumed probability distribution to find a test statistic ˆθ that has a known probability distribution. We reject H 0 is the test statistic differs significantly from what we expect when H 0 is true. This translates into choosing a low significance level for the test, meaning that we reject H 0 if ˆθ θ takes a value that has a low a probability (for example 5 % or 2.5 % or 1 %) under the assumption that H 0 is true.

Statistical inference in the regression model The decision: Compare value of test statistic with the critical values that correspond to the chosen significance level from the relevant distribution. The significance level is the probability of Type 1 error in hypothesis testing. Rejecting H 0 when H 0 is in fact true. Type 2 error is the failure to reject when H 0 is wrong. To have low risk of Type 2 error we would like to have test statistic that have high statistical power. Powerful tests give a high probability for rejecting H 0 when H 0 is false.

Hypothesis testing in a special case of RM 1 Consider the following special case of RM1, where we assume that β 2 = 0: y i = β 1 + e i a. x i are fixed numbers, (i = 1, 2,..., n) b. E (e i ) = 0, i, ( for al i ) c. var (e i ) = σ 2, i d. cov (e i, e j ) = 0, i = j e. β 1 and σ 2 are constant parameters f. e i N(0, σ 2 ).

Hypothesis testing in a special case of RM 1 In this special case of RM1 (with assumption f. included) it is implied that E (y i ) = β 1 and Testing y i N(β 1, σ 2 ) H 0 : β 1 = β 0 1 against H 1 : β 1 > β 0 1. is therefore the same as performing a one-sided test of the expectations parameter in a normal distribution. Know from course in statistics that we perform this test by the use of 1. A test statistic that has a standard normal distribution when we know σ 2 2. A test statistic that has a t distribution when we do not know σ 2, but estimate it

Hypothesis testing in a special case of RM 1 What we do next is to generalize the t-test in 2. to the case where the model is y i = β 1 + β 2 x i + e i and the RM1 assumptions a.-f. hold.

t-tests in RM1 The Z and t statistics I We have the following hypothesis test for the slope parameter β 2 of the model: H 0 : β 2 = β 0 2 against H 1 : β 2 > β 0 2. A logical test-statistic to use (based on the assumption of the model!) could be where Z = ˆβ 2 β 0 2 se( β 2 ) se( β 2 ) = var( β 2 ) (9) is the standard error of β 2.

t-tests in RM1 The Z and t statistics II Why would Z a natural statistic? Reject H 0 when Z > critical value corresponding to the chosen significance level Find critical value from the quantiles of the standard normal distribution, since Z N(0, 1) under H 0 from the fact ˆβ 2 N(β 2, var( β 2 ) in RM1. However Z cannot be used in practice since var( β 2 ) depends on the unknown parameter σ 2.

t-tests in RM1 The Z and t statistics III Therefore we need to find an estimator ˆσ 2 of σ 2 and then (hopefully) we are able to find the distribution of another statistic (that we by custom call t): t = ˆβ 2 β 0 2 ŝe( β 2 ) (10) where ŝe( β 2 ) denotes the estimated standard error of β 2 : ŝe( β 2 ) = var( β ˆσ 2 ) = 2 n i=1(x i x) 2 (11)

t-tests in RM1 Estimating the variance the disturbance I Heuristically, we can use the residuals ê i to estimate σ 2, they can be written as ê i = y i Ê (y i ), i = 1, 2,.., n Let us regard the estimated expectation of y i as fixed (a small short-cut) then we have that (ê i 0) σ 2 N(0, 1) and they are stochastically independent We then have from the definition of the Chi-square distribution that the variable U U = n i=1 ê 2 i σ 2 χ 2 (n 2) (12) is Chi-square distributed with n 2 degrees of freedom.

t-tests in RM1 Estimating the variance the disturbance II The reduction of 2 in the degrees of freedom is explained by the linear restrictions that are the 1 order conditions of the OLS principle. In the simplified model y i = β 1 + e i, the corresponding variable has n 1 degrees of freedom.

t-tests in RM1 Estimating the variance the disturbance III We choose the unbiased estimator of σ 2 DIY exercise 3. Show that ˆσ 2 = 1 n n 2 êi 2 i=1 E (ˆσ 2 ) = σ 2

t-tests in RM1 The t-statistic and the t-value I The t-statistic is the stochastic variable t = ˆβ 2 β 0 2 ˆσ 2 n i=1 (x i x ) 2 = ˆβ 2 β 0 2 σ 2 n i=1 (x i x )2 n i=1 ê i 2 σ 2 n 2 where ˆβ 2 and ˆσ 2 are interpreted as estimators Hence the stochastic variable t, has the structure t = Z U n 2 (13) which defines t as t-distributed variable with n 2 degrees of freedom.

t-tests in RM1 The t-statistic and the t-value II In RM1 we test H 0 : β 2 = β 0 2 against H 1 : β 2 > β 0 2. by comparing the calculated t-value t = ˆβ 2 β 0 2 ŝe( β 2 ) where ˆβ 2 and ŝe( β 2 ) are estimates, with the critical values from the t-distribution with n 2 degrees of freedom.