Simple Regression Model (Assumptions)

Size: px

Start display at page:

Download "Simple Regression Model (Assumptions)"

Scarlett Evans
6 years ago
Views:

1 Simple Regression Model (Assumptions) Lecture 18 Reading: Sections 18.1, 18., Logarithms in Regression Analysis with Asiaphoria, (Optional: Normal probability plot pp ) 1 Height son, inches Remember Regression? son_hat = *father n = 1078, R = 0.51, s_e = Height father, inches (s.d. of residuals).437 inches: measures scatter about OLS line 0.51: 5.1% of variation in sons heights explained by variation in their fathers heights OLS intercept : No interpretation b/c father cannot be 0 inches tall OLS slope 0.514: For every extra 1 inch of father s height, son is on average about ½ inch taller (y-hat): Predicted y, given x; E.g. son of a 7 inch tall father predicted to be inches (= *7) (residual): ; E.g. if is but is , then residual is inches Descriptive & Inferential Statistics Chap. 6: Scatterplots, Association, and Correlation Chap. 7: Introduction to Linear Regression / Simple Reg.: Chaps. 18 & 19 (Inference for Regression & Understanding Regression Residuals) Multiple Reg.: Chaps. 0 & 1 (Multiple Regression & Building Multiple Regression Models) BUT, multiple regression is also a new way to describe data: descriptive statistics 3 Lecture 18, Page 1 of 9

2 Questions and Data: Still Important Which kind of question? Research question: What is causal effect of a change in X (e.g. match) on Y (e.g. amount given) Descriptive question: what are patterns in data (e.g. how does household spending on food vary with income?) Which kind of data? Observational or experimental Correlation causation is a cliché Instead, apply understanding of data and specific context to fully interpret quantitative results Cross-sectional, time series, or panel 4 Which kind of data are these? On this slide, what is review of Fall-term material and what is new material? Unsurprisingly, we find that higher university density is associated with higher GDP per capita levels. p. 9 Why associated instead of correlated? Also, does quote imply causality? Source: The Economic Impact of Universities: Evidence from Across the Globe a 016 NBER Working Paper 5 It is interesting that countries with more universities in 1960 generally had higher growth rates over p. 9 But is it a strong association? 6 Lecture 18, Page of 9

3 X-variable is defined as Log(1 + universities per million people) Logs can straighten curved scatter plot Plus one addresses countries with 0 universities Example 1: x-value of 1 is a country with 1.7 universities per million: ln( ) 1 E.g. 10 universities w/ pop. 5.8 million: 1.710/5.8 Example : x-value of 3 is a country with universities per million: ln( ) 3 E.g. 5 universities w/ pop million: /1.31 University density is over 11 times bigger in Example, but x-value only 3 times as big (diminishing returns) 7 How to interpret b=.64? On average, countries with a university density (universities per million people) that is 10% higher have average years of schooling that is 0.64 years higher. 8 Frozen Pizza (p. 67) How does the volume of sales depend on the price of frozen pizza? What is the economic name of this relationship? Weekly data on price and quantity for each of four cities ( ); 156 weeks Raw data: ch18_mcsp_frozen_pizza.csv Are these data observational? Cross-sectional, time series, or panel? 9 Lecture 18, Page 3 of 9

4 Observational Data Price Quantity Demanded Demand Shifters What are some demand shifters for frozen pizza? Why do these demand shifters affect the price the firm chooses to set? 10 Frozen Pizza: OLS Interpret the line? Is the OLS line an estimate of the demand equation? Quantity, 10,000s Denver, n = 156 weeks Price, $1s 11 Simple Linear Regression: One x-variable Model: : dependent var., regressand, y-var., LHS-var. : independent var., regressor, explanatory var., x- var., RHS-var. (i.e. right-hand side variable) : observation index (often or cross-sectional data; time series data; or panel data) : intercept (constant) parameter : slope parameter : error term, residual, disturbance 1 Lecture 18, Page 4 of 9

5 Error term in Line is expected value: E Error explains deviations from expectations includes all other factors that affect aside from Impossible to collect data on everything: some variables unobserved to the researcher It reflects reality: model cannot control for everything In the above graph is positive or negative? 13 Assumptions Tame Elusive Epsilon We cannot observe but we can observe Notice how many of the six assumptions are about the unobservable Some assumptions can be checked by analyzing (the statistic tied to the parameter, but some cannot In general, models make assumptions about unknowns For example, a model could assume the outcome of the role of a die follows a discrete Uniform distribution: i.e. it s fair with a 1/6 probability of each outcome {1,, 3, 4, 5, 6} 14 Six Assumptions Six assumptions of the linear regression model Your book gives only four assumptions: I suspect it leaves one out because it is fairly obvious I am sure the other one is left out because it is only required if you wish to make a causal interpretation To minimize confusion, number the extra two as 5 & 6 Econometrics addresses violations in assumptions ECO374H Applied Econometrics (Commerce) ECO375H & ECO475H Applied Econometrics I & II 15 Lecture 18, Page 5 of 9

6 Assumption #1 Regression equation is linear in the error and parameters; the variables (in boxes) are linearly related to each other Not assuming that what is in boxes is linear (so long as no nonlinear functions of parameters or nonlinear functions of the error) Example of a linear regression: Example of a linear regression: ln 16 Frozen Pizza (Chapter 18) Weekly Q, 10,000s Frozen Pizza, Denver, Q-hat = *P n = 156, R = 0.59, s.e.(b) = Weekly price, $1s e (residuals) Q_hat Which violations can we see? 17 Natural Log Transformations ln(weekly Q, 10,000s) Frozen Pizza, Denver, ln(q)-hat = *ln(P) n = 156, R = 0.631, s.e.(b) = ln(weekly price, $1s) e (residuals) ln(q)_hat 18 Lecture 18, Page 6 of 9

7 Assumption # No autocorrelation / no serial correlation:, 0if Common problem in time-series data E.g. higher than expected inflation today, likely high tomorrow Errors assumed not systematically related across observations residual (e) residual (e) Assumption # Holds No Autocorrelation t Assumption # Violated Positive Autocorrelation t 19 Assumption #3 Homoscedasticity:, 1,, Equal variance assumption Error is just as noisy for all values of x Violation is called heteroscedasticity Common problem in cross-sectional data Y Y Homoscedasticity X1 Heteroscedasticity X 0 ln(weekly Q, 10,000s) Fix Assumption #1 issues before checking Assumption #3 Frozen Pizza, Denver, ln(q)-hat = *ln(P) n = 156, R = 0.631, s.e.(b) = ln(weekly price, $1s) e (residuals) ln(q)_hat Heteroscedasticity unequal variance of the residuals is often a byproduct of a violation of the linearity assumption Is Denver pizza regression an example? Remember that Chapter 18 advises you to check the assumptions in order: start with the linearity assumption 1 Lecture 18, Page 7 of 9

8 Assumptions #4 & #5 Galton s data (Lec. 5) Assumptions 1-3 hold? Normality: is Normal is unobserved so check Error has mean zero: 0,1,, Constant term (i.e. or ) picks up any constant effects, not the error Density Son, inches80 Galton's Heights, n = Father, inches n = residuals (e) Graphical Summary Would the elements in the population (not shown) lie on the line?,,, Is a reflection of sampling error? Assumptions #3, #4, and #5 combined: ~ 0, ON Public Sector Disclosure of 016 salaries for University of Waterloo employees Sex n Mean S.d. F 416 $139.74K $33.74K M 941 $155.36K $36.96K OLS Results: Salary-hat = *Male R = , n = 1,357, = Do Assumptions #1 - #5 hold? Salary (1,0000's CAN$) Salary and Salary-hat Waterloo, 016 Salaries Male=0 Male=1 Waterloo, 016 Salaries Male 4 Lecture 18, Page 8 of 9

9 Assumption #6 x uncorrelated w/ error:, 0 Exogeneity: x variable(s) unrelated with error Dosage is exogenous: Experimental data can est. causal effect: Endogeneity: x variable(s) related with error With observational data, lurking/unobserved/omitted/ confounding variables mean x and error are related Price of pizza is endogenous: Endogeneity bias means: 5 Short-Hand Assumptions 1) Linear relationship between variables (possibly non-linearly transformed) ) No correlation amongst errors (no autocorrelation for time-series data) 3) Homoscedasticity (single variance) of errors 4) Normally distributed errors 5) Constant included (error has mean 0) 6) No relationship between x and error 6 Lecture 18, Page 9 of 9

Lecture: Simultaneous Equation Model (Wooldridge s Book Chapter 16)

Lecture: Simultaneous Equation Model (Wooldridge s Book Chapter 16) 1 2 Model Consider a system of two regressions y 1 = β 1 y 2 + u 1 (1) y 2 = β 2 y 1 + u 2 (2) This is a simultaneous equation model