Ordinary Least Squares (OLS): Multiple Linear Regression (MLR) Assessment I What s New? & Goodness-of-Fit
|
|
- Curtis Marsh
- 5 years ago
- Views:
Transcription
1 Ordinary Least Squares (OLS): Multiple Linear egression (ML) Assessment I What s New? & Goodness-of-Fit Introduction OLS: A Quick Comparison of SL and ML Assessment Not much that's new! ML Goodness-of-Fit: Adjusted -squared and MSE (MSE) Comparing ML Models I: Using Goodness of Fit Metrics Introduction When we looked earlier at assessment in the context of SL models we focused on two types of measures of performance, goodness-of-fit and precision/inference metrics. The goodness of fit metrics told you something about how well the predicteds from your model fit the actuals, and the precision/inference metrics said something about how precisely the slope parameter was estimated: SL: Goodness-of-Fit Mean Squared Error (MSE): SS MSE = an average squared residual, sort of n oot Mean Squared Error (MSE): MSE = MSE sort of an average residual, but more like a square root of an average squared residual, sort of Coefficient of Determination ( ) : SS SSE S = = = = = proportion ˆˆ ρxy ρ ˆ SST SST S of the variance of the actuals explained by the predicted, as well as the correlation (squared) between the predicteds and the actuals. SL: Precision/Inference se β : Standard Error ( ˆ ) t β : t stat ( ˆ ) MSE MSE se = ˆ β ( x x) = Sx n ˆ ˆ β i ˆ β SSE t = = ( n ) = ( n ) β se SS The MSE, MSE and Standard Error metrics are not in standardized units, making it difficult to interpret the magnitudes. But and t ˆ β are both standardized to some extent, making them perhaps more useful in assessing the performance of the model:
2 : 0 closer to one is better. closer to zero, not so much t ˆ β : above ish: nice precision; less than ish: not so nice; and between and : there's hope moving to ML models: We continue to turn to these assessment metrics when we move to ML models, with the formulas changing by not very much, as you'll see below. Of the metrics, however, proves to be far less useful when assessing performance of ML models and so we address that shortcoming with a new Goodness-of-Fit metric, adjusted (sometimes adj, or ). Some Intuition: The shortcoming of in the ML world: When additional explanatory variables are added to a ML model, SSs will typically decrease, or at worst, stay the same but SSs can never increase (and can never decrease) with additional explanatory variables in the model. And so in the context of the metric, HS variables get credit for just showing up irrespective of their explanatory power. Consider a ML model with min SSs of SS 0. If you have an additional HS variable, then one option in minimizing SSs is to keep the coefficient of that new variable equal to zero. But when you minimize SSs with that restriction, you are solving the previous min SSs problem, and so the minimum SS (restricting the new coefficient to be zero) is SS 0. So with the additional explanatory variable, you can never do worse in minimizing SSs than SS, and you can probably do better once you drop the restriction of that zero coefficient. 0 If it turns out that the when minimizing SSs for the new model, the new variable has a coefficient of zero, then SSs will remain at SS 0 and the new variable has added nothing (no explanatory content) to the model. is unchanged. Alternatively, if the new coefficient is non-zero when minimizing SSs, then SSs will necessarily have decreased (so long as the new variable is not perfectly collinear with the other HS variables in the model). increases. When new explanatory variables are added to a model their coefficients will typically be non-zero and will typically increase. So don t be impressed if increases when new HS variables are added to the ML analysis that's entirely to be expected. Certainly McKayla Maroney is not impressed! Assuming no changes to the dependent variable.
3 Here's an application, which illustrates the point and tests your understanding: ML Application: Correlations provide a lower bound on ML. Suppose you are considering a ML box office revenues analysis, with explanatory variables wk, wk and wk3. Here are the correlations of the variables in the model:. corr rtotgross wk wk wk3 (obs=7,730) rtotgr~s wk wk wk rtotgross.0000 wk wk wk Notice that the largest correlation between a HS variable and rtotgross is (wk3). Then as shown below, the in the full model must be no less than.9474 =.898, and most likely will be greater. And so the correlations (squared) provide a lower bound on the ML model. Or put differently: you can often get a pretty good sense of in a ML model just by looking at the correlations (squared) amongst the variables. If you understand the previous comment about HS variable getting credit for just showing up, you'll understand why I claim that the in the full ML model will have an of at least.9474 =.898. Here's why: To get to the ML model, let's start with the SL model in which rtotgross has been regressed on wk3. I pick wk3 because of the three HS variables, it has the highest correlation with rtotgross. Since = ρ xy for SL models, we know that the for this SL model will be.9474 =.898. See Model () below. and build to Model (3): () () (3) rtotgross rtotgross rtotgross wk3 7.75*** 5.47*** 4.778*** (60.3) (.38) (59.84) wk 0.735*** 0.540*** (46.60) (.36) wk 0.745*** (9.79) _cons * -0.60** (0.5) (-.36) (-.64) N sq SS
4 As predicted, ' s are increasing moving left to right, since the coefficients for the new variables are non-zero: increases from.898 in Model () to.90 in Model (), and to.9 in Model (3). And also as predicted, SSs are decreasing. And so we can use simple pairwise correlations together with the fact that will never decrease when additional HS variable are added to a model, to place a lower bound on for the final ML model or put differently, the simple correlations alone tell you that the final ML model will have a very high close to. The following table compares the various Goodness-of-Fit concepts/definitions/formulas in SL and ML models. (Note that I assume that there is always a constant term in the SL and ML models.) OLS: A Quick Comparison of SL and ML Assessment Goodness of Fit SL ML Sum Squares SST = SSE + SS SST = SSE + SS (Coefficient of Determination) (w/ intercept term) SS SSE = = SST SST SS SSE = = SST SST SampleVar( predicted) = SampleVar( actual) SampleVar( predicted) = SampleVar( actual) Degrees of freedom () = n = n k MSE SS SS MSE = = SS SS MSE = = n n k MSE MSE = MSE MSE = MSE Adjusted SS n MSE = = SST n k S 4
5 As you can see, there are a few differences between SL and ML models, but not many!. The definitions of SS, SSE, SST are the same for SL and ML models as is the definition of, and the fact that SST = SSE + SS (since there is a constant term in the model).. In the MSE calculation, we now divide by n-k-, the degrees of freedom () in the ML model. This reflects an interest in unbiasedness to be discussed later. This is in fact consistent with the SL metric, since there were n- in those models. 3. We have new metric for ML models, Adjusted, discussed in more detail below. As discussed above, this will not give new HS variables goodness-of-fit credit merely for just showing up. New HS variables have to impress (in reducing SSs by more than some trivial amount) for Adjusted to increase. ML Goodness of Fit: Adjusted -squared As discussed above, when you are adding explanatory variables to a ML model (and not changing the y's or number of observations), SSs will always decrease (and -sq will always increase) unless the estimated ML coefficient for the new variable is exactly 0 (or the new variable is perfectly collinear with the other HS variables already in the model). So nobody should be impressed if increases when additional explanatory variables are brought into the analysis. The question should be: By how much did increase? If increased a lot, then you should be impressed; but if it increased by not so much, then maybe you'll want to hold your applause. Adjusted is an attempt to adjust the coefficient of determination for this shortcoming. You ll discover that smallish decreases in SSs will not generate a higher adj ; but larger decreases will and what is small or large will depend in part on how many additional variables were added to the model.. n Adj is often (and rather opaquely) defined as = ( ) n k. ecall that we sometimes refer to n-k- as the (number of) degrees of freedom () in the model. Since = SS, a more easily interpreted expression for Adj is: SST = SST, SS n which looks a lot like the definition of (with a n adjustment). 5
6 ( n ) ( n ) Note that since = >, < for k > 0, with the difference inversely ( n k ) related to k. And so adjusted is always bounded above by. Adjusted can be negative, though this rarely happens in practice.. If you see that, you have a really really really bad model! Time to find a new profession! Interpretation of Adj : It's all about the rates of change of SSs and. We can rewrite the previous expression for Adjusted : ( n ) SS = SST. As you add explanatory variables to the model, only the terms in the square brackets (SS and ) are changing, and both ( SSs and ) are typically declining. And so whether increases or decreases will depend on the relative rates of change of SSs and : SS If the decline in SSs is faster than the decline in, then will decline and increase with the additional explanatory variables. will SS But if the decline in SSs is slower than the decline in, then will increase, and will decrease. So for Adjusted to decrease, it must be the case that SSs are dropping faster than. and MSE (MSE) SS / ( n k ) MSE Since = =, adjusted and MSE will always move in opposite SST / ( n ) S directions when S is fixed. So if you are adding (or subtracting) HS variables to (or from) a ML model (and not impacting S ), you should expect to see and MSE moving in exactly opposite directions. Accordingly, the two goodness-of-fit metrics are effectively redundant in the sense that knowing the movements patterns of one tells you the movements of the other. An important difference however is that while we don t necessarily have a good sense of when MSEs (MSEs) are small or large, we do know that, and so we typically have an easier time evaluating magnitudes of. Note however that since and do not necessarily move in the same direction, MSEs and will not necessarily move in opposite directions. That was not the case for SL models. 6
7 Comparing ML Models I: Using Goodness of Fit Metrics To illustrate Goodness-of-Fit metrics in action, here s an example using the bodyfat dataset. In Model (), the Brozek measure of bodyfat had been regressed on hgt and wgt.. esttab, r ar scalar (rmse) compress () () (3) (4) Brozek Brozek Brozek Brozek hgt *** (-6.9) (-.43) (-.5) (-.55) wgt 0.87*** -0.0*** -0.08** -0.00* (4.48) (-5.4) (-3.8) (-.5) abd 0.880*** 0.883*** 0.898*** (5.9) (5.3) (.6) hip (-0.49) (-0.58) chest (-0.38) _cons 3.6*** -3.66*** -8.64** -5.86* (4.5) (-5.0) (-.7) (-.0) N sq adj. -sq rmse t statistics in parentheses * p<0.05, ** p<0.0, *** p<0.00 Note the esttab options: r ( ), ar ( ), and rmse ( ) MSE. In Model (), abd has been added to Model (), and -sq and adj. -sq both increase, while MSE declines. In Model (3) hip has been added in, with -sq continuing to increase as it almost always will. Now, however, adj. -sq declines and MSE increases. As always, adj -sq and MSE are moving in opposite directions. And in going to Model (4), with chest added to the model, -sq continues to (slightly) increase, while adj -sq again declines and MSE again increases. 7
8 ecall that with SL models, we could use -sq to compare the performance of different models having the same dependent variable. In the ML world, we often use adj 's to compare models, so long as the dependent variables are the same though I'd be the last to suggest that you only look at adj. Applying this criterion to the previous set of four ML models, Model () is the best performer since it has the highest adj -sq and the lowest MSE. But all of the Models tell you something so don't ignore them, just because their performance stats aren't as impressive! Comparing the performance of ML models is as much art as science and in truth, we typically look at a number of different aspects/properties of the model. But certainly adj -sq and MSE are in the conversation. We'll return to this topic later, and focus on the different criteria at play in assessing the performance of the three types of econometrics models discussed in the Introduction: Forecasting models (focus on out-of-sample forecasting, and don t over-fit the data) Behavioral models (the challenging art form) Favorite coefficient models (focus on the favorite coefficient and don t worry about the other aspects of the model other than making sure that you really have included every possible relevant explanatory variable, and accordingly that you have minimized the possibility of omitted variable impact/bias) 8
F Tests and F statistics
F Tests and F statistics Testing Linear estrictions F Stats and F Tests F Distributions F stats (w/ ) F Stats and tstat s eported F Stat's in OLS Output Example I: Bodyfat Babies and Bathwater F Stats,
More informationOrdinary Least Squares (OLS): Multiple Linear Regression (MLR) Analytics What s New? Not Much!
Ordinary Least Squares (OLS): Multiple Linear Regression (MLR) Analytics What s New? Not Much! OLS: Comparison of SLR and MLR Analysis Interpreting Coefficients I (SRF): Marginal effects ceteris paribus
More informationσ σ MLR Models: Estimation and Inference v.3 SLR.1: Linear Model MLR.1: Linear Model Those (S/M)LR Assumptions MLR3: No perfect collinearity
Comparison of SLR and MLR analysis: What s New? Roadmap Multicollinearity and standard errors F Tests of linear restrictions F stats, adjusted R-squared, RMSE and t stats Playing with Bodyfat: F tests
More informationstatistical sense, from the distributions of the xs. The model may now be generalized to the case of k regressors:
Wooldridge, Introductory Econometrics, d ed. Chapter 3: Multiple regression analysis: Estimation In multiple regression analysis, we extend the simple (two-variable) regression model to consider the possibility
More informationRegression, part II. I. What does it all mean? A) Notice that so far all we ve done is math.
Regression, part II I. What does it all mean? A) Notice that so far all we ve done is math. 1) One can calculate the Least Squares Regression Line for anything, regardless of any assumptions. 2) But, if
More informationStatistical Inference with Regression Analysis
Introductory Applied Econometrics EEP/IAS 118 Spring 2015 Steven Buck Lecture #13 Statistical Inference with Regression Analysis Next we turn to calculating confidence intervals and hypothesis testing
More informationLecture 4: Multivariate Regression, Part 2
Lecture 4: Multivariate Regression, Part 2 Gauss-Markov Assumptions 1) Linear in Parameters: Y X X X i 0 1 1 2 2 k k 2) Random Sampling: we have a random sample from the population that follows the above
More informationLecture 4: Multivariate Regression, Part 2
Lecture 4: Multivariate Regression, Part 2 Gauss-Markov Assumptions 1) Linear in Parameters: Y X X X i 0 1 1 2 2 k k 2) Random Sampling: we have a random sample from the population that follows the above
More informationRegression, Part I. - In correlation, it would be irrelevant if we changed the axes on our graph.
Regression, Part I I. Difference from correlation. II. Basic idea: A) Correlation describes the relationship between two variables, where neither is independent or a predictor. - In correlation, it would
More informationLectures 5 & 6: Hypothesis Testing
Lectures 5 & 6: Hypothesis Testing in which you learn to apply the concept of statistical significance to OLS estimates, learn the concept of t values, how to use them in regression work and come across
More informationGreene, Econometric Analysis (7th ed, 2012)
EC771: Econometrics, Spring 2012 Greene, Econometric Analysis (7th ed, 2012) Chapters 2 3: Classical Linear Regression The classical linear regression model is the single most useful tool in econometrics.
More informationIntroduction to Statistical modeling: handout for Math 489/583
Introduction to Statistical modeling: handout for Math 489/583 Statistical modeling occurs when we are trying to model some data using statistical tools. From the start, we recognize that no model is perfect
More informationECON The Simple Regression Model
ECON 351 - The Simple Regression Model Maggie Jones 1 / 41 The Simple Regression Model Our starting point will be the simple regression model where we look at the relationship between two variables In
More informationInferences for Regression
Inferences for Regression An Example: Body Fat and Waist Size Looking at the relationship between % body fat and waist size (in inches). Here is a scatterplot of our data set: Remembering Regression In
More informationdf=degrees of freedom = n - 1
One sample t-test test of the mean Assumptions: Independent, random samples Approximately normal distribution (from intro class: σ is unknown, need to calculate and use s (sample standard deviation)) Hypotheses:
More informationNotes 11: OLS Theorems ECO 231W - Undergraduate Econometrics
Notes 11: OLS Theorems ECO 231W - Undergraduate Econometrics Prof. Carolina Caetano For a while we talked about the regression method. Then we talked about the linear model. There were many details, but
More informationHypothesis testing I. - In particular, we are talking about statistical hypotheses. [get everyone s finger length!] n =
Hypothesis testing I I. What is hypothesis testing? [Note we re temporarily bouncing around in the book a lot! Things will settle down again in a week or so] - Exactly what it says. We develop a hypothesis,
More informationAt this point, if you ve done everything correctly, you should have data that looks something like:
This homework is due on July 19 th. Economics 375: Introduction to Econometrics Homework #4 1. One tool to aid in understanding econometrics is the Monte Carlo experiment. A Monte Carlo experiment allows
More informationSTA121: Applied Regression Analysis
STA121: Applied Regression Analysis Linear Regression Analysis - Chapters 3 and 4 in Dielman Artin Department of Statistical Science September 15, 2009 Outline 1 Simple Linear Regression Analysis 2 Using
More informationSTAT 350: Summer Semester Midterm 1: Solutions
Name: Student Number: STAT 350: Summer Semester 2008 Midterm 1: Solutions 9 June 2008 Instructor: Richard Lockhart Instructions: This is an open book test. You may use notes, text, other books and a calculator.
More informationR 2 and F -Tests and ANOVA
R 2 and F -Tests and ANOVA December 6, 2018 1 Partition of Sums of Squares The distance from any point y i in a collection of data, to the mean of the data ȳ, is the deviation, written as y i ȳ. Definition.
More informationECO220Y Simple Regression: Testing the Slope
ECO220Y Simple Regression: Testing the Slope Readings: Chapter 18 (Sections 18.3-18.5) Winter 2012 Lecture 19 (Winter 2012) Simple Regression Lecture 19 1 / 32 Simple Regression Model y i = β 0 + β 1 x
More informationSection Least Squares Regression
Section 2.3 - Least Squares Regression Statistics 104 Autumn 2004 Copyright c 2004 by Mark E. Irwin Regression Correlation gives us a strength of a linear relationship is, but it doesn t tell us what it
More informationSeptember 12, Math Analysis Ch 1 Review Solutions. #1. 8x + 10 = 4x 30 4x 4x 4x + 10 = x = x = 10.
#1. 8x + 10 = 4x 30 4x 4x 4x + 10 = 30 10 10 4x = 40 4 4 x = 10 Sep 5 7:00 AM 1 #. 4 3(x + ) = 5x 7(4 x) 4 3x 6 = 5x 8 + 7x CLT 3x = 1x 8 +3x +3x = 15x 8 +8 +8 6 = 15x 15 15 x = 6 15 Sep 5 7:00 AM #3.
More information( )( b + c) = ab + ac, but it can also be ( )( a) = ba + ca. Let s use the distributive property on a couple of
Factoring Review for Algebra II The saddest thing about not doing well in Algebra II is that almost any math teacher can tell you going into it what s going to trip you up. One of the first things they
More informationECON2228 Notes 2. Christopher F Baum. Boston College Economics. cfb (BC Econ) ECON2228 Notes / 47
ECON2228 Notes 2 Christopher F Baum Boston College Economics 2014 2015 cfb (BC Econ) ECON2228 Notes 2 2014 2015 1 / 47 Chapter 2: The simple regression model Most of this course will be concerned with
More informationMITOCW ocw f99-lec05_300k
MITOCW ocw-18.06-f99-lec05_300k This is lecture five in linear algebra. And, it will complete this chapter of the book. So the last section of this chapter is two point seven that talks about permutations,
More informationLECTURE 6. Introduction to Econometrics. Hypothesis testing & Goodness of fit
LECTURE 6 Introduction to Econometrics Hypothesis testing & Goodness of fit October 25, 2016 1 / 23 ON TODAY S LECTURE We will explain how multiple hypotheses are tested in a regression model We will define
More informationRegression Models. Chapter 4. Introduction. Introduction. Introduction
Chapter 4 Regression Models Quantitative Analysis for Management, Tenth Edition, by Render, Stair, and Hanna 008 Prentice-Hall, Inc. Introduction Regression analysis is a very valuable tool for a manager
More informationMITOCW MITRES_18-007_Part5_lec3_300k.mp4
MITOCW MITRES_18-007_Part5_lec3_300k.mp4 The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high-quality educational resources
More information1 Multiple Regression
1 Multiple Regression In this section, we extend the linear model to the case of several quantitative explanatory variables. There are many issues involved in this problem and this section serves only
More informationB. Weaver (24-Mar-2005) Multiple Regression Chapter 5: Multiple Regression Y ) (5.1) Deviation score = (Y i
B. Weaver (24-Mar-2005) Multiple Regression... 1 Chapter 5: Multiple Regression 5.1 Partial and semi-partial correlation Before starting on multiple regression per se, we need to consider the concepts
More informationLinear Regression with Multiple Regressors
Linear Regression with Multiple Regressors (SW Chapter 6) Outline 1. Omitted variable bias 2. Causality and regression analysis 3. Multiple regression and OLS 4. Measures of fit 5. Sampling distribution
More informationBiostatistics and Design of Experiments Prof. Mukesh Doble Department of Biotechnology Indian Institute of Technology, Madras
Biostatistics and Design of Experiments Prof. Mukesh Doble Department of Biotechnology Indian Institute of Technology, Madras Lecture - 39 Regression Analysis Hello and welcome to the course on Biostatistics
More informationLECTURE 15: SIMPLE LINEAR REGRESSION I
David Youngberg BSAD 20 Montgomery College LECTURE 5: SIMPLE LINEAR REGRESSION I I. From Correlation to Regression a. Recall last class when we discussed two basic types of correlation (positive and negative).
More informationsociology 362 regression
sociology 36 regression Regression is a means of studying how the conditional distribution of a response variable (say, Y) varies for different values of one or more independent explanatory variables (say,
More informationNonrecursive Models Highlights Richard Williams, University of Notre Dame, https://www3.nd.edu/~rwilliam/ Last revised April 6, 2015
Nonrecursive Models Highlights Richard Williams, University of Notre Dame, https://www3.nd.edu/~rwilliam/ Last revised April 6, 2015 This lecture borrows heavily from Duncan s Introduction to Structural
More information9. Linear Regression and Correlation
9. Linear Regression and Correlation Data: y a quantitative response variable x a quantitative explanatory variable (Chap. 8: Recall that both variables were categorical) For example, y = annual income,
More informationSection 3: Simple Linear Regression
Section 3: Simple Linear Regression Carlos M. Carvalho The University of Texas at Austin McCombs School of Business http://faculty.mccombs.utexas.edu/carlos.carvalho/teaching/ 1 Regression: General Introduction
More informationBusiness Economics BUSINESS ECONOMICS. PAPER No. : 8, FUNDAMENTALS OF ECONOMETRICS MODULE No. : 3, GAUSS MARKOV THEOREM
Subject Business Economics Paper No and Title Module No and Title Module Tag 8, Fundamentals of Econometrics 3, The gauss Markov theorem BSE_P8_M3 1 TABLE OF CONTENTS 1. INTRODUCTION 2. ASSUMPTIONS OF
More informationCPSC 320 Sample Solution, Reductions and Resident Matching: A Residentectomy
CPSC 320 Sample Solution, Reductions and Resident Matching: A Residentectomy August 25, 2017 A group of residents each needs a residency in some hospital. A group of hospitals each need some number (one
More informationECNS 561 Multiple Regression Analysis
ECNS 561 Multiple Regression Analysis Model with Two Independent Variables Consider the following model Crime i = β 0 + β 1 Educ i + β 2 [what else would we like to control for?] + ε i Here, we are taking
More information1 Warm-Up: 2 Adjusted R 2. Introductory Applied Econometrics EEP/IAS 118 Spring Sylvan Herskowitz Section #
Introductory Applied Econometrics EEP/IAS 118 Spring 2015 Sylvan Herskowitz Section #10 4-1-15 1 Warm-Up: Remember that exam you took before break? We had a question that said this A researcher wants to
More informationGov 2000: 9. Regression with Two Independent Variables
Gov 2000: 9. Regression with Two Independent Variables Matthew Blackwell Harvard University mblackwell@gov.harvard.edu Where are we? Where are we going? Last week: we learned about how to calculate a simple
More informationMITOCW ocw f99-lec30_300k
MITOCW ocw-18.06-f99-lec30_300k OK, this is the lecture on linear transformations. Actually, linear algebra courses used to begin with this lecture, so you could say I'm beginning this course again by
More informationSlope Fields: Graphing Solutions Without the Solutions
8 Slope Fields: Graphing Solutions Without the Solutions Up to now, our efforts have been directed mainly towards finding formulas or equations describing solutions to given differential equations. Then,
More informationModule 2 Study Guide. The second module covers the following sections of the textbook: , 4.1, 4.2, 4.5, and
Module 2 Study Guide The second module covers the following sections of the textbook: 3.3-3.7, 4.1, 4.2, 4.5, and 5.1-5.3 Sections 3.3-3.6 This is a continuation of the study of linear functions that we
More informationEC4051 Project and Introductory Econometrics
EC4051 Project and Introductory Econometrics Dudley Cooke Trinity College Dublin Dudley Cooke (Trinity College Dublin) Intro to Econometrics 1 / 23 Project Guidelines Each student is required to undertake
More informationChapter 1: Linear Regression with One Predictor Variable also known as: Simple Linear Regression Bivariate Linear Regression
BSTT523: Kutner et al., Chapter 1 1 Chapter 1: Linear Regression with One Predictor Variable also known as: Simple Linear Regression Bivariate Linear Regression Introduction: Functional relation between
More information28. SIMPLE LINEAR REGRESSION III
28. SIMPLE LINEAR REGRESSION III Fitted Values and Residuals To each observed x i, there corresponds a y-value on the fitted line, y = βˆ + βˆ x. The are called fitted values. ŷ i They are the values of
More informationApplied Statistics and Econometrics
Applied Statistics and Econometrics Lecture 6 Saul Lach September 2017 Saul Lach () Applied Statistics and Econometrics September 2017 1 / 53 Outline of Lecture 6 1 Omitted variable bias (SW 6.1) 2 Multiple
More informationEmpirical Application of Simple Regression (Chapter 2)
Empirical Application of Simple Regression (Chapter 2) 1. The data file is House Data, which can be downloaded from my webpage. 2. Use stata menu File Import Excel Spreadsheet to read the data. Don t forget
More informationMathematical Notation Math Introduction to Applied Statistics
Mathematical Notation Math 113 - Introduction to Applied Statistics Name : Use Word or WordPerfect to recreate the following documents. Each article is worth 10 points and should be emailed to the instructor
More informationECON3150/4150 Spring 2015
ECON3150/4150 Spring 2015 Lecture 3&4 - The linear regression model Siv-Elisabeth Skjelbred University of Oslo January 29, 2015 1 / 67 Chapter 4 in S&W Section 17.1 in S&W (extended OLS assumptions) 2
More informationMITOCW ocw f99-lec01_300k
MITOCW ocw-18.06-f99-lec01_300k Hi. This is the first lecture in MIT's course 18.06, linear algebra, and I'm Gilbert Strang. The text for the course is this book, Introduction to Linear Algebra. And the
More informationMITOCW MITRES18_005S10_DiffEqnsGrowth_300k_512kb-mp4
MITOCW MITRES18_005S10_DiffEqnsGrowth_300k_512kb-mp4 GILBERT STRANG: OK, today is about differential equations. That's where calculus really is applied. And these will be equations that describe growth.
More informationMITOCW MITRES18_006F10_26_0501_300k-mp4
MITOCW MITRES18_006F10_26_0501_300k-mp4 ANNOUNCER: The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational
More informationThe Simple Regression Model. Part II. The Simple Regression Model
Part II The Simple Regression Model As of Sep 22, 2015 Definition 1 The Simple Regression Model Definition Estimation of the model, OLS OLS Statistics Algebraic properties Goodness-of-Fit, the R-square
More informationsociology 362 regression
sociology 36 regression Regression is a means of modeling how the conditional distribution of a response variable (say, Y) varies for different values of one or more independent explanatory variables (say,
More informationMultiple Linear Regression CIVL 7012/8012
Multiple Linear Regression CIVL 7012/8012 2 Multiple Regression Analysis (MLR) Allows us to explicitly control for many factors those simultaneously affect the dependent variable This is important for
More informationMITOCW MITRES_18-007_Part3_lec5_300k.mp4
MITOCW MITRES_18-007_Part3_lec5_300k.mp4 The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources
More informationThe Simple Regression Model. Simple Regression Model 1
The Simple Regression Model Simple Regression Model 1 Simple regression model: Objectives Given the model: - where y is earnings and x years of education - Or y is sales and x is spending in advertising
More informationPROFESSOR: WELCOME BACK TO THE LAST LECTURE OF THE SEMESTER. PLANNING TO DO TODAY WAS FINISH THE BOOK. FINISH SECTION 6.5
1 MATH 16A LECTURE. DECEMBER 9, 2008. PROFESSOR: WELCOME BACK TO THE LAST LECTURE OF THE SEMESTER. I HOPE YOU ALL WILL MISS IT AS MUCH AS I DO. SO WHAT I WAS PLANNING TO DO TODAY WAS FINISH THE BOOK. FINISH
More informationMultiple Regression. Midterm results: AVG = 26.5 (88%) A = 27+ B = C =
Economics 130 Lecture 6 Midterm Review Next Steps for the Class Multiple Regression Review & Issues Model Specification Issues Launching the Projects!!!!! Midterm results: AVG = 26.5 (88%) A = 27+ B =
More informationMultiple Regression Analysis
Multiple Regression Analysis y = β 0 + β 1 x 1 + β 2 x 2 +... β k x k + u 2. Inference 0 Assumptions of the Classical Linear Model (CLM)! So far, we know: 1. The mean and variance of the OLS estimators
More informationMultiple Regression Theory 2006 Samuel L. Baker
MULTIPLE REGRESSION THEORY 1 Multiple Regression Theory 2006 Samuel L. Baker Multiple regression is regression with two or more independent variables on the right-hand side of the equation. Use multiple
More informationNote: Please use the actual date you accessed this material in your citation.
MIT OpenCourseWare http://ocw.mit.edu 18.06 Linear Algebra, Spring 2005 Please use the following citation format: Gilbert Strang, 18.06 Linear Algebra, Spring 2005. (Massachusetts Institute of Technology:
More informationLecture 16 - Correlation and Regression
Lecture 16 - Correlation and Regression Statistics 102 Colin Rundel April 1, 2013 Modeling numerical variables Modeling numerical variables So far we have worked with single numerical and categorical variables,
More informationMITOCW MITRES_18-007_Part1_lec3_300k.mp4
MITOCW MITRES_18-007_Part1_lec3_300k.mp4 The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high-quality educational resources
More informationOrdinary Least Squares Regression Explained: Vartanian
Ordinary Least Squares Regression Explained: Vartanian When to Use Ordinary Least Squares Regression Analysis A. Variable types. When you have an interval/ratio scale dependent variable.. When your independent
More informationLinear Regression with Multiple Regressors
Linear Regression with Multiple Regressors (SW Chapter 6) Outline 1. Omitted variable bias 2. Causality and regression analysis 3. Multiple regression and OLS 4. Measures of fit 5. Sampling distribution
More informationChapter 4: Regression Models
Sales volume of company 1 Textbook: pp. 129-164 Chapter 4: Regression Models Money spent on advertising 2 Learning Objectives After completing this chapter, students will be able to: Identify variables,
More informationappstats27.notebook April 06, 2017
Chapter 27 Objective Students will conduct inference on regression and analyze data to write a conclusion. Inferences for Regression An Example: Body Fat and Waist Size pg 634 Our chapter example revolves
More informationMITOCW ocw f99-lec17_300k
MITOCW ocw-18.06-f99-lec17_300k OK, here's the last lecture in the chapter on orthogonality. So we met orthogonal vectors, two vectors, we met orthogonal subspaces, like the row space and null space. Now
More informationSection I. Define or explain the following terms (3 points each) 1. centered vs. uncentered 2 R - 2. Frisch theorem -
First Exam: Economics 388, Econometrics Spring 006 in R. Butler s class YOUR NAME: Section I (30 points) Questions 1-10 (3 points each) Section II (40 points) Questions 11-15 (10 points each) Section III
More informationThe Multiple Regression Model
Multiple Regression The Multiple Regression Model Idea: Examine the linear relationship between 1 dependent (Y) & or more independent variables (X i ) Multiple Regression Model with k Independent Variables:
More informationInstructor (Brad Osgood)
TheFourierTransformAndItsApplications-Lecture26 Instructor (Brad Osgood): Relax, but no, no, no, the TV is on. It's time to hit the road. Time to rock and roll. We're going to now turn to our last topic
More informationBusiness Statistics. Lecture 9: Simple Regression
Business Statistics Lecture 9: Simple Regression 1 On to Model Building! Up to now, class was about descriptive and inferential statistics Numerical and graphical summaries of data Confidence intervals
More informationACCESS TO SCIENCE, ENGINEERING AND AGRICULTURE: MATHEMATICS 1 MATH00030 SEMESTER / Lines and Their Equations
ACCESS TO SCIENCE, ENGINEERING AND AGRICULTURE: MATHEMATICS 1 MATH00030 SEMESTER 1 017/018 DR. ANTHONY BROWN. Lines and Their Equations.1. Slope of a Line and its y-intercept. In Euclidean geometry (where
More informationChapter 27 Summary Inferences for Regression
Chapter 7 Summary Inferences for Regression What have we learned? We have now applied inference to regression models. Like in all inference situations, there are conditions that we must check. We can test
More informationEconometrics. Week 11. Fall Institute of Economic Studies Faculty of Social Sciences Charles University in Prague
Econometrics Week 11 Institute of Economic Studies Faculty of Social Sciences Charles University in Prague Fall 2012 1 / 30 Recommended Reading For the today Advanced Time Series Topics Selected topics
More information10 Model Checking and Regression Diagnostics
10 Model Checking and Regression Diagnostics The simple linear regression model is usually written as i = β 0 + β 1 i + ɛ i where the ɛ i s are independent normal random variables with mean 0 and variance
More informationLesson 6: Algebra. Chapter 2, Video 1: "Variables"
Lesson 6: Algebra Chapter 2, Video 1: "Variables" Algebra 1, variables. In math, when the value of a number isn't known, a letter is used to represent the unknown number. This letter is called a variable.
More informationECON3150/4150 Spring 2016
ECON3150/4150 Spring 2016 Lecture 4 - The linear regression model Siv-Elisabeth Skjelbred University of Oslo Last updated: January 26, 2016 1 / 49 Overview These lecture slides covers: The linear regression
More informationChapter 19 Sir Migo Mendoza
The Linear Regression Chapter 19 Sir Migo Mendoza Linear Regression and the Line of Best Fit Lesson 19.1 Sir Migo Mendoza Question: Once we have a Linear Relationship, what can we do with it? Something
More information1-Way ANOVA MATH 143. Spring Department of Mathematics and Statistics Calvin College
1-Way ANOVA MATH 143 Department of Mathematics and Statistics Calvin College Spring 2010 The basic ANOVA situation Two variables: 1 Categorical, 1 Quantitative Main Question: Do the (means of) the quantitative
More informationEQ: How do I convert between standard form and scientific notation?
EQ: How do I convert between standard form and scientific notation? HW: Practice Sheet Bellwork: Simplify each expression 1. (5x 3 ) 4 2. 5(x 3 ) 4 3. 5(x 3 ) 4 20x 8 Simplify and leave in standard form
More informationLecture 3: Multivariate Regression
Lecture 3: Multivariate Regression Rates, cont. Two weeks ago, we modeled state homicide rates as being dependent on one variable: poverty. In reality, we know that state homicide rates depend on numerous
More informationMITOCW MITRES18_005S10_DerivOfSinXCosX_300k_512kb-mp4
MITOCW MITRES18_005S10_DerivOfSinXCosX_300k_512kb-mp4 PROFESSOR: OK, this lecture is about the slopes, the derivatives, of two of the great functions of mathematics: sine x and cosine x. Why do I say great
More informationLectures on Simple Linear Regression Stat 431, Summer 2012
Lectures on Simple Linear Regression Stat 43, Summer 0 Hyunseung Kang July 6-8, 0 Last Updated: July 8, 0 :59PM Introduction Previously, we have been investigating various properties of the population
More information1 The basics of panel data
Introductory Applied Econometrics EEP/IAS 118 Spring 2015 Related materials: Steven Buck Notes to accompany fixed effects material 4-16-14 ˆ Wooldridge 5e, Ch. 1.3: The Structure of Economic Data ˆ Wooldridge
More informationECON3150/4150 Spring 2016
ECON3150/4150 Spring 2016 Lecture 6 Multiple regression model Siv-Elisabeth Skjelbred University of Oslo February 5th Last updated: February 3, 2016 1 / 49 Outline Multiple linear regression model and
More informationMITOCW MITRES18_006F10_26_0602_300k-mp4
MITOCW MITRES18_006F10_26_0602_300k-mp4 FEMALE VOICE: The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational
More informationEconometrics I KS. Module 2: Multivariate Linear Regression. Alexander Ahammer. This version: April 16, 2018
Econometrics I KS Module 2: Multivariate Linear Regression Alexander Ahammer Department of Economics Johannes Kepler University of Linz This version: April 16, 2018 Alexander Ahammer (JKU) Module 2: Multivariate
More information1 Regression with Time Series Variables
1 Regression with Time Series Variables With time series regression, Y might not only depend on X, but also lags of Y and lags of X Autoregressive Distributed lag (or ADL(p; q)) model has these features:
More informationInference for the Regression Coefficient
Inference for the Regression Coefficient Recall, b 0 and b 1 are the estimates of the slope β 1 and intercept β 0 of population regression line. We can shows that b 0 and b 1 are the unbiased estimates
More informationMITOCW ocw-18_02-f07-lec02_220k
MITOCW ocw-18_02-f07-lec02_220k The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
More informationChapter 14 Simple Linear Regression (A)
Chapter 14 Simple Linear Regression (A) 1. Characteristics Managerial decisions often are based on the relationship between two or more variables. can be used to develop an equation showing how the variables
More informationGeneral Linear Model (Chapter 4)
General Linear Model (Chapter 4) Outcome variable is considered continuous Simple linear regression Scatterplots OLS is BLUE under basic assumptions MSE estimates residual variance testing regression coefficients
More informationOrdinary Least Squares Regression Explained: Vartanian
Ordinary Least Squares Regression Eplained: Vartanian When to Use Ordinary Least Squares Regression Analysis A. Variable types. When you have an interval/ratio scale dependent variable.. When your independent
More information