The general linear regression with k explanatory variables is just an extension of the simple regression as follows
|
|
- Cody Murphy
- 6 years ago
- Views:
Transcription
1 3. Multiple Regression Analysis The general linear regression with k explanatory variables is just an extension of the simple regression as follows (1) y i = β 0 + β 1 x i1 + + β k x ik + u i. Because (2) y i x ij = β j j = 1,..., k, coefficient β j indicates the marginal effect of variable x j, and indicates the amount y is expected to change as x j changes by one unit and other variables are kept constant (ceteris paribus). The multiple regression opens up several additional options to enrich analysis and make modeling more realistic compared to the simple regression. 1
2 Example 3.1: Consider the hourly wage example. Enhance the model as (3) log(w) = β 0 + β 1 x 1 + β 2 x 2 + β 3 x 3, where w = average hourly earnings, x 1 = years of education (educ), x 2 = years of labor market experience (exper), and x 3 (tenure). Dependent Variable: LOG(WAGE) Method: Least Squares Date: 08/21/12 Time: 09:16 Sample: Included observations: 526 = years with the current employer Variable Coefficient Std. Error t-statistic Prob. C EDUC EXPER TENURE R-squared Mean dependent var Adjusted R-squared S.D. dependent var S.E. of regression Akaike info criterion Sum squared resid Schwarz criterion Log likelihood Hannan-Quinn criter F-statistic Durbin-Watson stat Prob(F-statistic) For example the coefficient means that, holding exper and tenure fixed, another year of education is predicted to increase wage by approximately 9.2%. Staying another year at the same firm (educ fixed, exper= tenure=1) is expected to result in a salary increase by approximately 0.4% + 2.2% = 2.6%. 2
3 Example 3.2: Consider the consumption function C = f(y ), where Y is income. Suppose the assumption is that as incomes grow the marginal propensity to consume decreases. In simple regression we could try to fit a level-log model or log-log model. One possibility also could be β 1 = β 1l + β 1q Y, where according to our hypothesis β 1q < 0. Thus the consumption function becomes C = β 0 + (β 1l + β 1q Y )Y + u = β 0 + β 1l Y + β 1q Y 2 + u This is a multiple regression model with x 1 = Y and x 2 = Y 2. This simple example demonstrates that we can meaningfully enrich simple regression analysis (even though we have essentially only two variables, C and Y ) and at the same time get a meaningful interpretation to the above polynomial model. The response of C to a one unit change in Y is now C Y = β 1l + 2β 1q Y. 3
4 Estimation In order to estimate the model we replace the classical assumption 3 as 3. None of the independent variables is constant, and no observation vector of any independent variable can be written as a linear combination of the observation vectors of any other independent variables. The estimation method again is the OLS, which produces estimates ˆβ 0, ˆβ 1,..., ˆβ k by minimizing (4) n i=1 (y i β 0 β 1 x i1 β k x ik ) 2 with respect to the parameters. Again the first order solution is to set the (k + 1) partial derivatives equal to zero. The solution is straightforward although the explicit form of the estimators become complicated. 4
5 Matrix form Using matrix algebra simplifies considerably the notations in multiple regression. Denote the observation vector on y as (5) y = (y 1,..., y n ), where the prime denotes transposition. In the same manner denote the data matrix on x-variables enhanced with ones in the first column as an n (k + 1) matrix (6) X = 1 x 11 x 12 x 1k 1. x 21. x 22. x 2k. 1 x n1 x n2 x nk, where k < n (the number of observations, n, is larger than the number of x-variables, k). 5
6 Then we can present the whole set of regression equations for the sample (7) y 1 = β 0 + β 1 x β k x 1k + u 1 y 2. = β 0 + β 1 x β k x 2k + u 2 y n = β 0 + β 1 x n1 + + β k x nk + u n in the matrix form as (8) y 1 y 2. y n = or shortly 1 x 11 x 12 x 1k 1. x 21. x 22. x 2k. 1 x n1 x n2 x nk β 0 β 1 β 2. β k + u 1 u 2. u n (9) y = Xb + u, where b = (β 0, β 1,..., β k ) is the parameter vector and is the error vector. u = (u 1, u 2,..., u n ) 6
7 The normal equations for the first order conditions of the minimization of (4) in matrix form are simply (10) X Xˆb = X y which gives the explicit solution for the OLS estimator of b as (11) ˆb = (X X) 1 X y, where ˆb = (ˆβ 0, ˆβ 1,..., ˆβ k ) and existence of (X X) 1 is granted by assumption 3. The fitted model is (12) ŷ i = ˆβ 0 + ˆβ 1 x i1 + + ˆβ k x ik, i = 1,..., n. 7
8 Remark 3.1: Single and multiple regression do in general not produce the same parameter estimates on the same independent variables. For example, if you fit the simple regression ỹ = β 0 + β 1 x 1, where β 0 and β 1 are the OLS estimators, and fit a multiple regression ŷ = ˆβ 0 + ˆβ 1 x 1 + ˆβ 2 x 2 then it turns out that β 1 = ˆβ 1 + ˆβ 2 δ 1, where δ 1 is the slope coefficient from regressing x 2 on x 1. This implies that β 1 ˆβ 1 unless ˆβ 2 = 0, or x 1 and x 2 are uncorrelated. 8
9 Goodness-of-Fit Again in the same manner as with the simple regression we have n n n (13) (y i ȳ) 2 = (ŷ i ȳ) 2 + (y i ŷ i ) 2 or i=1 i=1 i=1 (14) SST = SSE + SSR, where (15) ŷ i = ˆβ 0 + ˆβ 1 x i1 + + ˆβ k x ik. 9
10 R-square: R-square denotes again the sample variation in the fitted ŷ i s as a proportion of the sample variation in the original y i s: (16) R 2 = SSE SST = 1 SSR SST. Again as in the case of the simple regression, R 2 can be shown to be the squared correlation coefficient between the actual y i and fitted ŷ i. This correlation is called the multiple correlation (17) R = (yi ȳ)(ŷ i ŷ) (yi ȳ) 2 (ŷi ŷ) 2. Remark 3.2: ŷ = ȳ. Remark 3.3: R 2 never decreases when an explanatory variable is added to the model. 10
11 Adjusted R-square: (18) R 2 = 1 s2 u s 2 y = 1 n 1 n k 1 (1 R2 ), where (19) s 2 u = 1 n k 1 n i=1 (y i ŷ i ) 2 = 1 n k 1 n û 2 i i=1 is an estimator of the error variance σu 2 = Var[u i ]. The square root of (19) is the so called standard error of the regression. 11
12 3.3 Expected values of the OLS estimators Given observation tuples (x i1, x i1,..., x ik, y i ), i = 1,..., n the classical assumptions read now: Assumptions (classical assumptions): 1. y = β 0 + k i=1 β i x + u in the population. 2. {(x i1, x i1,..., x ik, y i )} is a random sample of the model above, implying uncorrelated residuals: Cov(u i, u j ) = 0 for all i j. 3. All independent variables including the vector of constants are linearly independent, implying that (X X) 1 exists. 4. E[u x 1,..., x k ] = 0 implying E[u] = 0 and Cov(u, x 1 ) =... = Cov(u, x k ) = Var[u x 1,..., x k ] = σ 2 implying Var[u] = σ 2. Under these assumptions we can show that the estimators of the regression coefficients are unbiased. That is Theorem 3.1: Given observations on the x-variables (20) E[ˆβ j ] = βj 12
13 Using matrix notations, the proof of Theorem 3.1 is pretty straightforward. To do this write ˆb = (X X) 1 X y (21) = (X X) 1 X (Xb + u) = (X X) 1 X Xb + (X X) 1 X u = b + (X X) 1 X u. Given X, the expected value of ˆb is (22) E[ˆb] = b + (X X) 1 X E[u] = b because by Assumption 4 E[u] = 0. Thus the OLS estimators of the regression coefficients are unbiased. Remark 3.4: If z = (z 1,..., z k ) is a random vector, then E[z] = (E[z 1 ],..., E[z n ]). That is the expected value of a vector is a vector whose components are the individual expected values. 13
14 Irrelevant variables in a regression Suppose the correct model is (23) y = β 0 + β 1 x 1 + u but we estimate the model as (24) y = β 0 + β 1 x 1 + β 2 x 2 + u. Thus, β 2 = 0 in reality. The OLS estimation results yield (25) ŷ = ˆβ 0 + ˆβ 1 x 1 + ˆβ 2 x 2 By Theorem 3.1 E[ˆβ j ] = β j, thus in particular E[ˆβ 2 ] = β 2 = 0, implying that inclusion of extra variables to a regression does not bias the results. However, as will be seen later, they decrease accuracy of estimation by increasing variance of the OLS estimates. 14
15 Omitted variable bias Suppose now as an example that the correct model is (26) y = β 0 + β 1 x 1 + β 2 x 2 + u, but we misspecify the model as (27) y = β 0 + β 1 x 1 + v, where the omitted variable is embedded into the residual term v = β 2 x 2 + u. OLS estimator for β 1 for specification (27)is (28) β 1 = (xi1 x 1 )y i (xi1 x 1 ) 2. From Equation (2.37) we have (29) β 1 = β 1 + where n i=1 a i v i, (30) a i = (x i1 x 1 ) (xi1 x 1 ) 2. 15
16 Thus because E[v i ] = E[β 2 x i2 + u i ] = β 2 x i2 E[ β 1 ] = β 1 + a i E[v i ] (31) i.e., = β 1 + a i β 2 x i2 = β 1 + β 2 (xi1 x 1 )x i2 (xi1 x 1 ) 2 }{{} δ 1 (32) E[ β 1 ] = β 1 + β 2 δ 1, where δ 1 is the slope coefficient of regressing x 2 upon x 1, implying that β 1 is biased for β 1 unless x 1 and x 2 are uncorrelated (or β 2 = 0). This is called the omitted variable bias. The direction of the omitted variable bias is as follows: Corr(x 1, x 2 ) > 0 Corr(x 1, x 2 ) < 0 β 2 > 0 positive bias negative bias β 2 < 0 negative bias positive bias 16
17 3.4 The variance of OLS estimators Write the regression model (33) y i = β 0 + β 1 x i1 + + β k x ik + u i i = 1,..., n in the matrix form (34) y = Xb + u. Then we can write the OLS estimators compactly as (35) ˆb = (X X) 1 X y. 17
18 Under the classical assumption 1 5, and assuming X fixed we can show that the variancecovariance matrix b is (36) Cov[ˆb] = (X X) 1 σ 2 u. Variances of the individual coefficients are obtained form the main diagonal of the matrix, and can be shown to be of the form (37) Var[ˆβ j ] = σ 2 u (1 R 2 j ) n i=1 (x ij x j ) 2, j = 1,..., k, where Rj 2 is the R-square when regressing x j on the other explanatory variables and the constant term. 18
19 Multicollinearity In terms of linear algebra, we say that vectors x 1, x 2,..., x k are linearly independent if a 1 x a k x k = 0 holds only if a 1 = = a k = 0. Otherwise x 1,..., x k are linearly dependent. In such a case some a l 0 and we can write x l = c 1 x c l 1 x l 1 + c l+1 x l c k x k, where c j = a j /a l that is x l can be represented as a linear combination of the other variables. 19
20 In statistics the multiple correlation measures the degree of linear dependence. If the variables are perfectly linearly dependent. That is, if for example, x j is a linear combination of other variables, the multiple correlation R j = 1. A perfect linear dependence is rare between random variables. However, particularly between macro economic variables dependencies are often high. 20
21 From the variance equation (37) Var[ˆβ j ] = σ 2 u (1 R 2 j ) n i=1 (x ij x j ) 2, we see that Var[ˆβ j ] as Rj 2 1. That is, the more the explanatory variables are linearly dependent the larger the variance becomes. This implies that the coefficient estimates become increasingly instable. High (but not perfect) correlation between two or more explanatory variables is called multicollinearity. 21
22 Symptoms of multicollinearity: (1) High correlations between explanatory variables. (2) R 2 is relatively high, but the coefficient estimates tend to be insignificant (see the section of hypothesis testing) (3) Some of the coefficients are of wrong sign and some coefficients are at the same time unreasonably large. (4) Coefficient estimates change much from one model alternative to another. 22
23 Example 3.3: Variable E t denotes cost expenditures in a sample of Toyota Mark II cars at time point t, M t denotes the milage and A t age. Consider model alternatives: Model A: Model B: Model C: E t = α 0 + α 1 A t + u 1t E t = β 0 + β 1 M t + u 2t E t = γ 0 + γ 1 M t + γ 2 A t + u 3t 23
24 Estimation results: (t-values in parentheses) Variable Malli A Malli B Malli C Constant (-5.98) (-5.91) (0.06) Age (22.16) (9.58) Miles (18.27) (-7.06) df R ˆσ Findings: Apriori, coefficients α 1, β 1, γ 1, and γ 2 should be positive. However, ˆγ 2 = (!!?), but ˆβ 1 = Correlation r M,E = 0.996! 24
25 Remedies: In the collinearity problem the question is the there is not enough information to reliably identify each variables contribution as an explanatory variable in the model. Thus in order to alleviate the problem: (1) Use non-sample information if available to impose restrictions between coefficients. (2) Increase the sample size if possible. (3) Drop the most collinear (on the base of R 2 j ) variables. (4) If a linear combination (usually a sum) of the most collinear variables is meaningful, replace the collinear variables by the linear combination. 25
26 Variances of misspecified models Consider again as in (26) the regression model (38) y = β 0 + β 1 x 1 + β 2 x 2 + u. Suppose the following models are estimated by OLS (39) ŷ = ˆβ 0 + ˆβ 1 x 1 + ˆβ 2 x 2 and (40) ỹ = β 0 + β 1 x 1. Then by (37) (41) Var[ˆβ 1 ] = and in analogy to (2.40) (42) Var[ β 1 ] = Var[β 2x 2 + u] (xi1 x 1 ) 2 = σ 2 u (1 r 2 12 ) (x i1 x 1 ) 2 σ 2 u (xi1 x 1 ) 2, where r 12 is the sample correlation between x 1 and x 2. 26
27 Thus Var[ β 1 ] Var[ˆβ 1 ], and the inequality is strict if r In summary (assuming r 12 0): (1) If β 2 0, then β 1 is biased, ˆβ 1 is unbiased, and Var[ β 1 ] < Var[ˆβ 1 ] (2) If β 2 = 0, then both β 1 and ˆβ 1 are unbiased, but Var[ β 1 ] < Var[ˆβ 1 ] 27
28 Estimating error variance σ 2 u An unbiased estimator of the error variance Var[u] = σ 2 u is (43) ˆσ 2 u = where 1 n k 1 n i=1 û 2 i, (44) û i = y i ˆβ 0 ˆβ 1 x i1 ˆβ k u ik. The term n k 1 in (42) is the degrees of freedom (df). It can be shown that (45) E[ˆσ 2 u ] = σ2 u, i.e., ˆσ 2 u is unbiased estimator of σ2 u. ˆσ u is called the standard error of the regression. 28
29 Standard errors of ˆβ k Standard deviation of ˆβ j is the square root of (37), i.e. (46) σˆβ j = Var[β j ] = σ u (1 R 2 j ) (x ij x j ) 2 Substituting σ u by its estimate ˆσ u = the standard error of ˆβ j ˆσ 2 u gives (47) se(ˆβ j ) = ˆσ u (1 R 2 j ) (x ij x j ) 2. 29
30 3.5 The Gauss-Markov Theorem Theorem 3.2: Under the classical assumptions 1 5 ˆβ 0, ˆβ 1,... ˆβ k are best linear unbiased estimators (BLUEs) of β 0, β 1,... β k, respectively. BLUE: Best: The variance of the OLS estimator is smallest among all linear unbiased estimators of β j Linear: ˆβ j = n i=1 w ij y i. Unbiased: E[ˆβ j ] = β j. 30
Multiple Regression Analysis. Part III. Multiple Regression Analysis
Part III Multiple Regression Analysis As of Sep 26, 2017 1 Multiple Regression Analysis Estimation Matrix form Goodness-of-Fit R-square Adjusted R-square Expected values of the OLS estimators Irrelevant
More informationThe Simple Regression Model. Part II. The Simple Regression Model
Part II The Simple Regression Model As of Sep 22, 2015 Definition 1 The Simple Regression Model Definition Estimation of the model, OLS OLS Statistics Algebraic properties Goodness-of-Fit, the R-square
More informationCHAPTER 6: SPECIFICATION VARIABLES
Recall, we had the following six assumptions required for the Gauss-Markov Theorem: 1. The regression model is linear, correctly specified, and has an additive error term. 2. The error term has a zero
More informationPractical Econometrics. for. Finance and Economics. (Econometrics 2)
Practical Econometrics for Finance and Economics (Econometrics 2) Seppo Pynnönen and Bernd Pape Department of Mathematics and Statistics, University of Vaasa 1. Introduction 1.1 Econometrics Econometrics
More informationHeteroskedasticity. Part VII. Heteroskedasticity
Part VII Heteroskedasticity As of Oct 15, 2015 1 Heteroskedasticity Consequences Heteroskedasticity-robust inference Testing for Heteroskedasticity Weighted Least Squares (WLS) Feasible generalized Least
More informationStatistical Inference. Part IV. Statistical Inference
Part IV Statistical Inference As of Oct 5, 2017 Sampling Distributions of the OLS Estimator 1 Statistical Inference Sampling Distributions of the OLS Estimator Testing Against One-Sided Alternatives Two-Sided
More informationEconometrics I KS. Module 2: Multivariate Linear Regression. Alexander Ahammer. This version: April 16, 2018
Econometrics I KS Module 2: Multivariate Linear Regression Alexander Ahammer Department of Economics Johannes Kepler University of Linz This version: April 16, 2018 Alexander Ahammer (JKU) Module 2: Multivariate
More informationECNS 561 Multiple Regression Analysis
ECNS 561 Multiple Regression Analysis Model with Two Independent Variables Consider the following model Crime i = β 0 + β 1 Educ i + β 2 [what else would we like to control for?] + ε i Here, we are taking
More information3. Linear Regression With a Single Regressor
3. Linear Regression With a Single Regressor Econometrics: (I) Application of statistical methods in empirical research Testing economic theory with real-world data (data analysis) 56 Econometrics: (II)
More informationModel Specification and Data Problems. Part VIII
Part VIII Model Specification and Data Problems As of Oct 24, 2017 1 Model Specification and Data Problems RESET test Non-nested alternatives Outliers A functional form misspecification generally means
More informationThe Multiple Regression Model Estimation
Lesson 5 The Multiple Regression Model Estimation Pilar González and Susan Orbe Dpt Applied Econometrics III (Econometrics and Statistics) Pilar González and Susan Orbe OCW 2014 Lesson 5 Regression model:
More informationMultiple Linear Regression CIVL 7012/8012
Multiple Linear Regression CIVL 7012/8012 2 Multiple Regression Analysis (MLR) Allows us to explicitly control for many factors those simultaneously affect the dependent variable This is important for
More information5. Erroneous Selection of Exogenous Variables (Violation of Assumption #A1)
5. Erroneous Selection of Exogenous Variables (Violation of Assumption #A1) Assumption #A1: Our regression model does not lack of any further relevant exogenous variables beyond x 1i, x 2i,..., x Ki and
More information2. Linear regression with multiple regressors
2. Linear regression with multiple regressors Aim of this section: Introduction of the multiple regression model OLS estimation in multiple regression Measures-of-fit in multiple regression Assumptions
More informationHomoskedasticity. Var (u X) = σ 2. (23)
Homoskedasticity How big is the difference between the OLS estimator and the true parameter? To answer this question, we make an additional assumption called homoskedasticity: Var (u X) = σ 2. (23) This
More informationMultivariate Regression Analysis
Matrices and vectors The model from the sample is: Y = Xβ +u with n individuals, l response variable, k regressors Y is a n 1 vector or a n l matrix with the notation Y T = (y 1,y 2,...,y n ) 1 x 11 x
More informationAnswers to Problem Set #4
Answers to Problem Set #4 Problems. Suppose that, from a sample of 63 observations, the least squares estimates and the corresponding estimated variance covariance matrix are given by: bβ bβ 2 bβ 3 = 2
More informationTHE MULTIVARIATE LINEAR REGRESSION MODEL
THE MULTIVARIATE LINEAR REGRESSION MODEL Why multiple regression analysis? Model with more than 1 independent variable: y 0 1x1 2x2 u It allows : -Controlling for other factors, and get a ceteris paribus
More informationHeteroscedasticity 1
Heteroscedasticity 1 Pierre Nguimkeu BUEC 333 Summer 2011 1 Based on P. Lavergne, Lectures notes Outline Pure Versus Impure Heteroscedasticity Consequences and Detection Remedies Pure Heteroscedasticity
More informationstatistical sense, from the distributions of the xs. The model may now be generalized to the case of k regressors:
Wooldridge, Introductory Econometrics, d ed. Chapter 3: Multiple regression analysis: Estimation In multiple regression analysis, we extend the simple (two-variable) regression model to consider the possibility
More informationIntroductory Econometrics
Based on the textbook by Wooldridge: : A Modern Approach Robert M. Kunst robert.kunst@univie.ac.at University of Vienna and Institute for Advanced Studies Vienna November 23, 2013 Outline Introduction
More information2) For a normal distribution, the skewness and kurtosis measures are as follows: A) 1.96 and 4 B) 1 and 2 C) 0 and 3 D) 0 and 0
Introduction to Econometrics Midterm April 26, 2011 Name Student ID MULTIPLE CHOICE. Choose the one alternative that best completes the statement or answers the question. (5,000 credit for each correct
More informationECON The Simple Regression Model
ECON 351 - The Simple Regression Model Maggie Jones 1 / 41 The Simple Regression Model Our starting point will be the simple regression model where we look at the relationship between two variables In
More information6. Assessing studies based on multiple regression
6. Assessing studies based on multiple regression Questions of this section: What makes a study using multiple regression (un)reliable? When does multiple regression provide a useful estimate of the causal
More informationRegression with Qualitative Information. Part VI. Regression with Qualitative Information
Part VI Regression with Qualitative Information As of Oct 17, 2017 1 Regression with Qualitative Information Single Dummy Independent Variable Multiple Categories Ordinal Information Interaction Involving
More informationOutline. 2. Logarithmic Functional Form and Units of Measurement. Functional Form. I. Functional Form: log II. Units of Measurement
Outline 2. Logarithmic Functional Form and Units of Measurement I. Functional Form: log II. Units of Measurement Read Wooldridge (2013), Chapter 2.4, 6.1 and 6.2 2 Functional Form I. Functional Form: log
More informationEmpirical Economic Research, Part II
Based on the text book by Ramanathan: Introductory Econometrics Robert M. Kunst robert.kunst@univie.ac.at University of Vienna and Institute for Advanced Studies Vienna December 7, 2011 Outline Introduction
More informationIntermediate Econometrics
Intermediate Econometrics Heteroskedasticity Text: Wooldridge, 8 July 17, 2011 Heteroskedasticity Assumption of homoskedasticity, Var(u i x i1,..., x ik ) = E(u 2 i x i1,..., x ik ) = σ 2. That is, the
More informationMultiple Regression Analysis
Chapter 4 Multiple Regression Analysis The simple linear regression covered in Chapter 2 can be generalized to include more than one variable. Multiple regression analysis is an extension of the simple
More informationReview of Econometrics
Review of Econometrics Zheng Tian June 5th, 2017 1 The Essence of the OLS Estimation Multiple regression model involves the models as follows Y i = β 0 + β 1 X 1i + β 2 X 2i + + β k X ki + u i, i = 1,...,
More informationApplied Econometrics. Applied Econometrics Second edition. Dimitrios Asteriou and Stephen G. Hall
Applied Econometrics Second edition Dimitrios Asteriou and Stephen G. Hall MULTICOLLINEARITY 1. Perfect Multicollinearity 2. Consequences of Perfect Multicollinearity 3. Imperfect Multicollinearity 4.
More information1 Linear Regression Analysis The Mincer Wage Equation Data Econometric Model Estimation... 11
Econ 495 - Econometric Review 1 Contents 1 Linear Regression Analysis 4 1.1 The Mincer Wage Equation................. 4 1.2 Data............................. 6 1.3 Econometric Model.....................
More informationEconomics 113. Simple Regression Assumptions. Simple Regression Derivation. Changing Units of Measurement. Nonlinear effects
Economics 113 Simple Regression Models Simple Regression Assumptions Simple Regression Derivation Changing Units of Measurement Nonlinear effects OLS and unbiased estimates Variance of the OLS estimates
More information10. Time series regression and forecasting
10. Time series regression and forecasting Key feature of this section: Analysis of data on a single entity observed at multiple points in time (time series data) Typical research questions: What is the
More informationAdvanced Quantitative Methods: ordinary least squares
Advanced Quantitative Methods: Ordinary Least Squares University College Dublin 31 January 2012 1 2 3 4 5 Terminology y is the dependent variable referred to also (by Greene) as a regressand X are the
More informationLinear Regression. Junhui Qian. October 27, 2014
Linear Regression Junhui Qian October 27, 2014 Outline The Model Estimation Ordinary Least Square Method of Moments Maximum Likelihood Estimation Properties of OLS Estimator Unbiasedness Consistency Efficiency
More informationMaking sense of Econometrics: Basics
Making sense of Econometrics: Basics Lecture 2: Simple Regression Egypt Scholars Economic Society Happy Eid Eid present! enter classroom at http://b.socrative.com/login/student/ room name c28efb78 Outline
More informationcoefficients n 2 are the residuals obtained when we estimate the regression on y equals the (simple regression) estimated effect of the part of x 1
Review - Interpreting the Regression If we estimate: It can be shown that: where ˆ1 r i coefficients β ˆ+ βˆ x+ βˆ ˆ= 0 1 1 2x2 y ˆβ n n 2 1 = rˆ i1yi rˆ i1 i= 1 i= 1 xˆ are the residuals obtained when
More informationEconometrics I KS. Module 1: Bivariate Linear Regression. Alexander Ahammer. This version: March 12, 2018
Econometrics I KS Module 1: Bivariate Linear Regression Alexander Ahammer Department of Economics Johannes Kepler University of Linz This version: March 12, 2018 Alexander Ahammer (JKU) Module 1: Bivariate
More informationStatistical Inference with Regression Analysis
Introductory Applied Econometrics EEP/IAS 118 Spring 2015 Steven Buck Lecture #13 Statistical Inference with Regression Analysis Next we turn to calculating confidence intervals and hypothesis testing
More informationEconometrics Summary Algebraic and Statistical Preliminaries
Econometrics Summary Algebraic and Statistical Preliminaries Elasticity: The point elasticity of Y with respect to L is given by α = ( Y/ L)/(Y/L). The arc elasticity is given by ( Y/ L)/(Y/L), when L
More informationBrief Suggested Solutions
DEPARTMENT OF ECONOMICS UNIVERSITY OF VICTORIA ECONOMICS 366: ECONOMETRICS II SPRING TERM 5: ASSIGNMENT TWO Brief Suggested Solutions Question One: Consider the classical T-observation, K-regressor linear
More informationEconometrics Multiple Regression Analysis: Heteroskedasticity
Econometrics Multiple Regression Analysis: João Valle e Azevedo Faculdade de Economia Universidade Nova de Lisboa Spring Semester João Valle e Azevedo (FEUNL) Econometrics Lisbon, April 2011 1 / 19 Properties
More informationECON2228 Notes 2. Christopher F Baum. Boston College Economics. cfb (BC Econ) ECON2228 Notes / 47
ECON2228 Notes 2 Christopher F Baum Boston College Economics 2014 2015 cfb (BC Econ) ECON2228 Notes 2 2014 2015 1 / 47 Chapter 2: The simple regression model Most of this course will be concerned with
More informationMultiple Regression. Midterm results: AVG = 26.5 (88%) A = 27+ B = C =
Economics 130 Lecture 6 Midterm Review Next Steps for the Class Multiple Regression Review & Issues Model Specification Issues Launching the Projects!!!!! Midterm results: AVG = 26.5 (88%) A = 27+ B =
More informationMaking sense of Econometrics: Basics
Making sense of Econometrics: Basics Lecture 4: Qualitative influences and Heteroskedasticity Egypt Scholars Economic Society November 1, 2014 Assignment & feedback enter classroom at http://b.socrative.com/login/student/
More informationAbout the seasonal effects on the potential liquid consumption
About the seasonal effects on the potential liquid consumption Lucie Ravelojaona Guillaume Perrez Clément Cousin ENAC 14/01/2013 Consumption raw data Figure : Evolution during one year of different family
More informationEconometrics - Slides
1 Econometrics - Slides 2011/2012 João Nicolau 2 1 Introduction 1.1 What is Econometrics? Econometrics is a discipline that aims to give empirical content to economic relations. It has been defined generally
More informationHeteroskedasticity and Autocorrelation
Lesson 7 Heteroskedasticity and Autocorrelation Pilar González and Susan Orbe Dpt. Applied Economics III (Econometrics and Statistics) Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity
More informationSample Problems. Note: If you find the following statements true, you should briefly prove them. If you find them false, you should correct them.
Sample Problems 1. True or False Note: If you find the following statements true, you should briefly prove them. If you find them false, you should correct them. (a) The sample average of estimated residuals
More informationMultiple Regression Analysis
Multiple Regression Analysis y = β 0 + β 1 x 1 + β 2 x 2 +... β k x k + u 2. Inference 0 Assumptions of the Classical Linear Model (CLM)! So far, we know: 1. The mean and variance of the OLS estimators
More information7. Prediction. Outline: Read Section 6.4. Mean Prediction
Outline: Read Section 6.4 II. Individual Prediction IV. Choose between y Model and log(y) Model 7. Prediction Read Wooldridge (2013), Chapter 6.4 2 Mean Prediction Predictions are useful But they are subject
More informationMultiple Linear Regression
Multiple Linear Regression Simple linear regression tries to fit a simple line between two variables Y and X. If X is linearly related to Y this explains some of the variability in Y. In most cases, there
More informationMultiple Regression Analysis: Inference MULTIPLE REGRESSION ANALYSIS: INFERENCE. Sampling Distributions of OLS Estimators
1 2 Multiple Regression Analysis: Inference MULTIPLE REGRESSION ANALYSIS: INFERENCE Hüseyin Taştan 1 1 Yıldız Technical University Department of Economics These presentation notes are based on Introductory
More informationApplied Econometrics (QEM)
Applied Econometrics (QEM) The Simple Linear Regression Model based on Prinicples of Econometrics Jakub Mućk Department of Quantitative Economics Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple
More informationIntermediate Econometrics
Intermediate Econometrics Markus Haas LMU München Summer term 2011 15. Mai 2011 The Simple Linear Regression Model Considering variables x and y in a specific population (e.g., years of education and wage
More informationLeast Squares Estimation-Finite-Sample Properties
Least Squares Estimation-Finite-Sample Properties Ping Yu School of Economics and Finance The University of Hong Kong Ping Yu (HKU) Finite-Sample 1 / 29 Terminology and Assumptions 1 Terminology and Assumptions
More informationMotivation for multiple regression
Motivation for multiple regression 1. Simple regression puts all factors other than X in u, and treats them as unobserved. Effectively the simple regression does not account for other factors. 2. The slope
More informationx = 1 n (x = 1 (x n 1 ι(ι ι) 1 ι x) (x ι(ι ι) 1 ι x) = 1
Estimation and Inference in Econometrics Exercises, January 24, 2003 Solutions 1. a) cov(wy ) = E [(WY E[WY ])(WY E[WY ]) ] = E [W(Y E[Y ])(Y E[Y ]) W ] = W [(Y E[Y ])(Y E[Y ]) ] W = WΣW b) Let Σ be a
More informationWooldridge, Introductory Econometrics, 4th ed. Chapter 15: Instrumental variables and two stage least squares
Wooldridge, Introductory Econometrics, 4th ed. Chapter 15: Instrumental variables and two stage least squares Many economic models involve endogeneity: that is, a theoretical relationship does not fit
More informationApplied Quantitative Methods II
Applied Quantitative Methods II Lecture 4: OLS and Statistics revision Klára Kaĺıšková Klára Kaĺıšková AQM II - Lecture 4 VŠE, SS 2016/17 1 / 68 Outline 1 Econometric analysis Properties of an estimator
More information4. Nonlinear regression functions
4. Nonlinear regression functions Up to now: Population regression function was assumed to be linear The slope(s) of the population regression function is (are) constant The effect on Y of a unit-change
More information11. Simultaneous-Equation Models
11. Simultaneous-Equation Models Up to now: Estimation and inference in single-equation models Now: Modeling and estimation of a system of equations 328 Example: [I] Analysis of the impact of advertisement
More informationBusiness Statistics. Tommaso Proietti. Linear Regression. DEF - Università di Roma 'Tor Vergata'
Business Statistics Tommaso Proietti DEF - Università di Roma 'Tor Vergata' Linear Regression Specication Let Y be a univariate quantitative response variable. We model Y as follows: Y = f(x) + ε where
More informationOutline. 11. Time Series Analysis. Basic Regression. Differences between Time Series and Cross Section
Outline I. The Nature of Time Series Data 11. Time Series Analysis II. Examples of Time Series Models IV. Functional Form, Dummy Variables, and Index Basic Regression Numbers Read Wooldridge (2013), Chapter
More informationMultiple Regression Analysis
Multiple Regression Analysis y = 0 + 1 x 1 + x +... k x k + u 6. Heteroskedasticity What is Heteroskedasticity?! Recall the assumption of homoskedasticity implied that conditional on the explanatory variables,
More informationApplied Econometrics (QEM)
Applied Econometrics (QEM) based on Prinicples of Econometrics Jakub Mućk Department of Quantitative Economics Jakub Mućk Applied Econometrics (QEM) Meeting #3 1 / 42 Outline 1 2 3 t-test P-value Linear
More information13. Time Series Analysis: Asymptotics Weakly Dependent and Random Walk Process. Strict Exogeneity
Outline: Further Issues in Using OLS with Time Series Data 13. Time Series Analysis: Asymptotics Weakly Dependent and Random Walk Process I. Stationary and Weakly Dependent Time Series III. Highly Persistent
More informationLecture 5: Omitted Variables, Dummy Variables and Multicollinearity
Lecture 5: Omitted Variables, Dummy Variables and Multicollinearity R.G. Pierse 1 Omitted Variables Suppose that the true model is Y i β 1 + β X i + β 3 X 3i + u i, i 1,, n (1.1) where β 3 0 but that the
More informationWooldridge, Introductory Econometrics, 4th ed. Chapter 2: The simple regression model
Wooldridge, Introductory Econometrics, 4th ed. Chapter 2: The simple regression model Most of this course will be concerned with use of a regression model: a structure in which one or more explanatory
More informationIntroductory Econometrics
Based on the textbook by Wooldridge: : A Modern Approach Robert M. Kunst robert.kunst@univie.ac.at University of Vienna and Institute for Advanced Studies Vienna October 16, 2013 Outline Introduction Simple
More informationMaking sense of Econometrics: Basics
Making sense of Econometrics: Basics Lecture 7: Multicollinearity Egypt Scholars Economic Society November 22, 2014 Assignment & feedback Multicollinearity enter classroom at room name c28efb78 http://b.socrative.com/login/student/
More informationLinear models. Linear models are computationally convenient and remain widely used in. applied econometric research
Linear models Linear models are computationally convenient and remain widely used in applied econometric research Our main focus in these lectures will be on single equation linear models of the form y
More informationSimple Linear Regression
Simple Linear Regression In simple linear regression we are concerned about the relationship between two variables, X and Y. There are two components to such a relationship. 1. The strength of the relationship.
More informationLecture 8. Using the CLR Model. Relation between patent applications and R&D spending. Variables
Lecture 8. Using the CLR Model Relation between patent applications and R&D spending Variables PATENTS = No. of patents (in 000) filed RDEP = Expenditure on research&development (in billions of 99 $) The
More informationMultiple Regression Analysis: Further Issues
Multiple Regression Analysis: Further Issues Ping Yu School of Economics and Finance The University of Hong Kong Ping Yu (HKU) MLR: Further Issues 1 / 36 Effects of Data Scaling on OLS Statistics Effects
More informationLECTURE 10. Introduction to Econometrics. Multicollinearity & Heteroskedasticity
LECTURE 10 Introduction to Econometrics Multicollinearity & Heteroskedasticity November 22, 2016 1 / 23 ON PREVIOUS LECTURES We discussed the specification of a regression equation Specification consists
More informationF3: Classical normal linear rgression model distribution, interval estimation and hypothesis testing
F3: Classical normal linear rgression model distribution, interval estimation and hypothesis testing Feng Li Department of Statistics, Stockholm University What we have learned last time... 1 Estimating
More informationIn Chapter 2, we learned how to use simple regression analysis to explain a dependent
C h a p t e r Three Multiple Regression Analysis: Estimation In Chapter 2, we learned how to use simple regression analysis to explain a dependent variable, y, as a function of a single independent variable,
More informationECO220Y Simple Regression: Testing the Slope
ECO220Y Simple Regression: Testing the Slope Readings: Chapter 18 (Sections 18.3-18.5) Winter 2012 Lecture 19 (Winter 2012) Simple Regression Lecture 19 1 / 32 Simple Regression Model y i = β 0 + β 1 x
More informationSimple Linear Regression
Simple Linear Regression ST 430/514 Recall: A regression model describes how a dependent variable (or response) Y is affected, on average, by one or more independent variables (or factors, or covariates)
More informationEconometrics. Week 4. Fall Institute of Economic Studies Faculty of Social Sciences Charles University in Prague
Econometrics Week 4 Institute of Economic Studies Faculty of Social Sciences Charles University in Prague Fall 2012 1 / 23 Recommended Reading For the today Serial correlation and heteroskedasticity in
More informationApplied Econometrics (MSc.) Lecture 3 Instrumental Variables
Applied Econometrics (MSc.) Lecture 3 Instrumental Variables Estimation - Theory Department of Economics University of Gothenburg December 4, 2014 1/28 Why IV estimation? So far, in OLS, we assumed independence.
More informationEastern Mediterranean University Department of Economics ECON 503: ECONOMETRICS I. M. Balcilar. Midterm Exam Fall 2007, 11 December 2007.
Eastern Mediterranean University Department of Economics ECON 503: ECONOMETRICS I M. Balcilar Midterm Exam Fall 2007, 11 December 2007 Duration: 120 minutes Questions Q1. In order to estimate the demand
More informationSimple Linear Regression: The Model
Simple Linear Regression: The Model task: quantifying the effect of change X in X on Y, with some constant β 1 : Y = β 1 X, linear relationship between X and Y, however, relationship subject to a random
More informationBrief Sketch of Solutions: Tutorial 3. 3) unit root tests
Brief Sketch of Solutions: Tutorial 3 3) unit root tests.5.4.4.3.3.2.2.1.1.. -.1 -.1 -.2 -.2 -.3 -.3 -.4 -.4 21 22 23 24 25 26 -.5 21 22 23 24 25 26.8.2.4. -.4 - -.8 - - -.12 21 22 23 24 25 26 -.2 21 22
More informationin the time series. The relation between y and x is contemporaneous.
9 Regression with Time Series 9.1 Some Basic Concepts Static Models (1) y t = β 0 + β 1 x t + u t t = 1, 2,..., T, where T is the number of observation in the time series. The relation between y and x
More informationSolution to Exercise E6.
Solution to Exercise E6. The Multiple Regression Model. Inference Exercise E6.1 Beach umbrella rental Part I. Simple Linear Regression Model. a. Regression model: U t = α + β T t + u t t = 1,..., 22 Model
More informationTopic 4: Model Specifications
Topic 4: Model Specifications Advanced Econometrics (I) Dong Chen School of Economics, Peking University 1 Functional Forms 1.1 Redefining Variables Change the unit of measurement of the variables will
More informationThe Simple Regression Model. Simple Regression Model 1
The Simple Regression Model Simple Regression Model 1 Simple regression model: Objectives Given the model: - where y is earnings and x years of education - Or y is sales and x is spending in advertising
More information7. Integrated Processes
7. Integrated Processes Up to now: Analysis of stationary processes (stationary ARMA(p, q) processes) Problem: Many economic time series exhibit non-stationary patterns over time 226 Example: We consider
More informationEconometric Methods. Prediction / Violation of A-Assumptions. Burcu Erdogan. Universität Trier WS 2011/2012
Econometric Methods Prediction / Violation of A-Assumptions Burcu Erdogan Universität Trier WS 2011/2012 (Universität Trier) Econometric Methods 30.11.2011 1 / 42 Moving on to... 1 Prediction 2 Violation
More informationEconometrics. Final Exam. 27thofJune,2008. Timeforcompletion: 2h30min
Econometrics Final Exam 27thofJune,2008 João Valle e Azevedo António José Morgado Tiago Silva Vieira Timeforcompletion: 2h30min Give your answers in the space provided. Usedraftpapertoplanyouranswersbeforewritingthemontheexampaper.
More informationL2: Two-variable regression model
L2: Two-variable regression model Feng Li feng.li@cufe.edu.cn School of Statistics and Mathematics Central University of Finance and Economics Revision: September 4, 2014 What we have learned last time...
More informationApplied Regression Analysis
Applied Regression Analysis Chapter 3 Multiple Linear Regression Hongcheng Li April, 6, 2013 Recall simple linear regression 1 Recall simple linear regression 2 Parameter Estimation 3 Interpretations of
More informationIntroductory Econometrics
Based on the textbook by Wooldridge: : A Modern Approach Robert M. Kunst robert.kunst@univie.ac.at University of Vienna and Institute for Advanced Studies Vienna December 17, 2012 Outline Heteroskedasticity
More informationIn Chapter 2, we learned how to use simple regression analysis to explain a dependent
3 Multiple Regression Analysis: Estimation In Chapter 2, we learned how to use simple regression analysis to explain a dependent variable, y, as a function of a single independent variable, x. The primary
More informationRegression Models - Introduction
Regression Models - Introduction In regression models there are two types of variables that are studied: A dependent variable, Y, also called response variable. It is modeled as random. An independent
More informationSimple Regression Model Setup Estimation Inference Prediction. Model Diagnostic. Multiple Regression. Model Setup and Estimation.
Statistical Computation Math 475 Jimin Ding Department of Mathematics Washington University in St. Louis www.math.wustl.edu/ jmding/math475/index.html October 10, 2013 Ridge Part IV October 10, 2013 1
More informationMultiple Linear Regression
Multiple Linear Regression Asymptotics Asymptotics Multiple Linear Regression: Assumptions Assumption MLR. (Linearity in parameters) Assumption MLR. (Random Sampling from the population) We have a random
More information