Multiple Regression Analysis

Size: px
Start display at page:

Download "Multiple Regression Analysis"

Transcription

1 Multiple Regression Analysis

2 Where as simple linear regression has 2 variables (1 dependent, 1 independent): y ˆ = a + bx Multiple linear regression has >2 variables (1 dependent, many independent): ˆ 2 y = a + b1 x1 + b2 x +... b n x n The problems and solutions are the same as bivariate regression, except there are more parameters to estimate.

3 In bivariate regression we fit a line through points plotted in 2- dimenstional space:

4 In multiple regression with 3 variables we fit a plane through points plotted in 3-dimenstional space: Additional variables add additional dimensions to the variable space.

5 In addition to the assumptions of bivariate regression, multiple regression has the assumption of no multicollinearity among the independent variables. Multicollinearity when two or more of the independent variables are highly correlated, making it difficult to separate their effects on the dependent variable.

6 Example: Determine the strength of the relationship between native American male standing height, average yearly minimum temperature, and annual temperature range. Variables: MHT Male Standing Height(cm) Dependent AnnMinTemp Annual Minimum Temp (ºF) Independent AnnRange Annual Temp Range (ºF) Independent

7

8 Model 1 Model Summary b Adjusted Std. Error of Durbin- R R Square R Square the Estimate Watson.654 a a. Predictors: (Constant), AnnRange, AnnMinTemp b. Dependent Variable: MHT 41.6% of height explained by min temp and range. ANOVA b Model 1 Regres sion Residual Total Sum of Squares df Mean Square F Sig a a. Predic tors: (Constant), AnnRange, AnnMinTemp b. Dependent Variable: MHT Model is significant. Model 1 (Constant) AnnMinTemp AnnRange a. Dependent Variable: MHT Unstandardized Coeffic ients Coefficients a Standardiz ed Coeffic ients Collinearity Statistics B Std. Error Beta t Sig. Tolerance VIF Slopes are not zero. Some collinearity.

9 We interpret the regression equation as follows: Male Standing Height = (º F min temp) (º F temp range) The equation can be interpreted as follows: Every 1ºF increase in minimum temperature adds 4.49 centimeters in male standing height holding constant the temperature range. Conversely, every increase of 1ºF in the annual temperature range adds 1.57 centimeters in male standing height, holding constant the minimum temperature.

10 Normality of the residuals is one of the most important assumptions of linear regression. In this case the residuals are normally distributed.

11 The observed and predicted residuals do not display any systematic bias, which would indicate that the independent variables vary systematically with each other.

12 Coefficients a Unstandardized Coeffic ients Standardiz ed Coeffic ients Collinearity Statistics Model B Std. Error Beta t Sig. Tolerance VIF 1 (Constant) AnnMinTemp AnnRange a. Dependent Variable: MHT Tolerance is the amount of the variance in a given independent variable that can not be explained by other independent variables. In this case 46.2% of the variance in one can not be explained by the other meaning that 53.8% of the variance IS shared or collinear. Model Summary b Model R R Square Adjusted R Square Std. Error of the Estimate Durbin- Watson a a. Predictors: (Constant), AnnRange, AnnMinTemp b. Dependent Variable: MHT This is why the standard error of the estimate is so large. The standard error of the estimate is the average error expressed in the original units (e.g. centimeters). 30cm is 1 foot of error... in a person s height.

13 Coefficients a Unstandardized Coeffic ients Standardiz ed Coeffic ients Collinearity Statistics Model B Std. Error Beta t Sig. Tolerance VIF 1 (Constant) AnnMinTemp AnnRange a. Dependent Variable: MHT VIFs (variance inflation factors) higher than 2 are considered problematic (according to SPSS) and our VIFs are over just over 2.1.

14 Coefficients a Unstandardized Coeffic ients Standardiz ed Coeffic ients Collinearity Statistics Model B Std. Error Beta t Sig. Tolerance VIF 1 (Constant) AnnMinTemp AnnRange a. Dependent Variable: MHT The standard beta values indicate the strength of the relationship between the independent and dependent variables. Minimum temperature is a much stronger predictor of height than annual range.

15 The question becomes: do these collinearity statistics rise to the level of indicating multicollinearity among the independent variables? In this example they do. Correlations AnnMinTemp AnnRange AnnMinTemp Pearson Correlation Sig. (2-tailed) **.000 N AnnRange Pearson Correlation -.734** 1 Sig. (2-tailed) N **. Correlation is s ignificant at the 0.01 level (2-tailed).

16 Misspecification an error in the regression equation due to the exclusion of an independent variable that influences the dependent variable OR the inclusion of an independent variable that does not influence the dependent variable. Misspecification errors are common since it is difficult to know a priori what factors influence the dependent variable. Misspecification is a hypothesis, not statistical issue.

17 Data Transformation

18 Often the association between two variables in not linear. Data transformation (log, etc ) is perfectly acceptable. The type of transformation must be stated in your summary statement.

19 In this case, log transforming the population data created a linear relationship.

20 Converting to natural log is easy. For example, the mining town of Argentine has a population of 100, its natural log would be: pop(ln) = ln(100) = Converting back to the original units is also easy: pop = e = 100

21 Calculator transformations: Converting to a log: use the ln key. Converting from a log: use the e x key. SPSS transformations: Converting to a log: Transform>Compute variable> Arithmetic>Ln Converting from a log: Transform>Compute variable> Arithmetic>Exp

22 Population and Elevation in Colorado Mining Towns. The model is significant. What is the standard error of the estimate telling us? What are the units? Population = (-4.238)(Elevation)

23 Population and Elevation in Colorado Mining Towns: Log Transformation The model is significant. What is the standard error of the estimate telling us? What are the units? Ln(population) = (elevation)

24 Town Population Elevation (ft) (ln)population (ln)predicted (ln)residual Argentine Boreas Breckenridge Buckskin Joe Chihuahua Dudley Fairplay Hamilton Horseshoe Lamartine Lincoln Montezuma Mosquito Park City Parkville Quartzville Rexford Sacramento Saints John Silverheels Swandyke Silver Plume

25 Converting to Original Units from a Log Transformation Town = Horseshoe Population = 800 Elevation = 10,544ft Predicted ln(population) = Calculated ln(residual) = Converting to original units (people): population = e ( ) = 640 Converting the residual: residual = e = A A This is the ratio of the difference between the actual and predicted value. Original population = (640)( ) = Observed Predicted = residual Residual in original units (people): difference = = 160 e.g. the equation under-predicted Horseshoe s population by 160 people.

26 Town Population Elevation (ft) (ln)population (ln)predicted (ln)residual Argentine Observed Population = 100 (ln)predicted Population = (ln)residual = Predicted population ( e predicted ) Residual = observed predicted = What are the predicted population and residual values, in the original units?

27 Iterative Regression

28 If you are exploring a data base for associations, one method is to use iterative regression. Iterative Regression a iterative procedure which either adds or removes variables from a regression model based on their significance.

29 IMPORTANT: The SPSS stepwise procedure give results that are inconsistent with the other methods. Due to this inconsistency it is recommended that the stepwise procedure not be used. A better method of performing iterative regression is to use all variables with the enter procedure, then remove insignificant variables individually. OR use the backwards or forwards procedures.

30 Types of Iterative Regression: Enter all variables are entered in a single step. Stepwise independent variables are entered based on the smallest F probability. Variables already in the equation are removed if their probability of F becomes too large. Backward all variables are entered into the equation and then sequentially removed based on the smallest partial correlation. Forward - A stepwise variable selection procedure in which variables are sequentially entered into the model.

31 Harrisburg Housing Value (Iterative using the Enter procedure) Not significant Not significant

32 With insignificant variables removed. No changes here. All slopes are significant. Predicted value ($) = (Square Feet) (Year Built) (Bedrooms) (Half Baths) (Parcel Size) (Distance to Front St)

33 Standardized Coefficients Standardized or beta coefficients are slope values that have been standardized so that their variances are 1. They can be used to determine which of the independent variables have a greater effect on the dependent variable when the variables are measured in different units of measurement. In this case, Square Feet and Distance to Front Street are having the greatest effect.

34 705 ½ South Front Street Value = $133,900 Square Feet = 2380 Parcel Size = 2975 Distance to Front Street = 84 Year Built = 1900 Bedrooms = 3 Half Baths = 1 Predicted value ($) = (2380) (1900) (3) (1) (2975) (84) Predicted value ($) = Predicted value ($) = Residual ($) = = This is not surprising considering that the r 2 was Over 40% of the variation in housing value is not explained by this model.

35 Mapping Regression Residuals

36 Temperature Recording Sites Kyrgystan Region

37 Average yearly temperature is influenced by: Elevation: 6.4 C per 1000 m elevation change. Latitude: 4.0 C per 1000 km latitude change. To what degree can we predict temperature based on both elevation and latitude?

38 Elevation Latitude

39 Model Summary b Model R R Square Adjusted R Square Std. Error of the Estimate a a. Predictors: (Constant), Elevation b. Dependent Variable: Average Temperature Model: Elevation ANOVA a Model 1 Sum of Mean Square Squares df F Sig. Regression b Residual Total a. Dependent Variable: Average temperature b. Predictors: (Constant), Elevation Coefficients a Model 1 a. Dependent Variable: Average Temperature Unstandardized Coefficients Standardized Coefficients B Std. Error Beta t Sig. (Constant) Elevation Temperature P = (0.005)Elevation

40

41 The standard error of the estimate is about 3.5 C, which is half of the number of degree change per 1000m elevation. Model Summary b Model R R Square Adjusted R Square Std. Error of the Estimate a a. Predictors: (Constant), Elevation b. Dependent Variable: Average Temperature This model is not very accurate.

42 Unknown Missing explanatory variable.

43 Model Summary b Model R R Square Adjusted R Square Std. Error of the Estimate a a. Predictors: (Constant), Latitude b. Dependent Variable: Average Temperature This R 2 is really low. Model: Latitude ANOVA a Model 1 Sum of Mean Square Squares df F Sig. Regression b Residual Total a. Dependent Variable: Average Temperature b. Predictors: (Constant), Latitude Coefficients a Model 1 a. Dependent Variable: Average Temperature Unstandardized Coefficients Standardized Coefficients B Std. Error Beta t Sig. (Constant) Latitude

44 The standard error of the estimate is about 6 C, which is nearly the number of degree change per 1000m elevation. Model Summary b Model R R Square Adjusted R Square Std. Error of the Estimate a a. Predictors: (Constant), Latitude b. Dependent Variable: Average Temperature This model is also not very accurate. By itself, the variable latitude is not a good predictor of temperature.

45

46 This similarity in pattern suggests that together elevation and latitude combined may produce a strong predictive model.

47 Model Summary b Model R R Square Adjusted R Square Std. Error of the Estimate a a. Predictors: (Constant), Elevation, Latitude b. Dependent Variable: Average Temperature Model: Elevation + Latitude ANOVA a Model 1 Sum of Mean Square Squares df F Sig. Regression b Residual Total a. Dependent Variable: Average Temperature b. Predictors: (Constant), Elevation, Latitude Coefficients a Model Unstandardized Coefficients Standardized Coefficients Correlations Collinearity Statistics Std. Error Zero-order Tolerance VIF B Beta t Sig. Partial Part (Constant) Elevation Latitude a. Dependent Variable: Average Temperature

48 The standard error of the estimate is less than 2 C, which is by far the best estimator. Model Summary b Model R R Square Adjusted R Square Std. Error of the Estimate a a. Predictors: (Constant), Latitude b. Dependent Variable: Average Temperature This model is very accurate (90%).

49 Outlier?

50 Significantly over/under predicted locations.

51 There does not appear to be any spatial pattern to the distribution of residuals. The residuals appear to be spatially random. The number of large over/under predictions is about equal. It might be a good idea to examine large over/under predicted locations in greater detail.

52 Over-prediction Name Lon Lat Elev Temp Resid Humrogi Dzhergetal Gasan-kuli Under-prediction Name Lon Lat Elev Temp Resid Kushka Susamyr Aksai An initial inspection does not show any locational influences, with the exception of Gasan-kuli, which is located far from the other sites.

53 Gasan-kuli Dzhergetal Susamyr Aksai Kushka Humrogi

54 Key Points: 1. Let theory drive your selection of independent variables. Individual variable analyses (regressions) were misleading. 2. Use the tools available. Both statistics and graphs. 3. Map residuals and look for patterns. Patterns may be of interest. The absence of patterns is NOT a failure.

Regression ( Kemampuan Individu, Lingkungan kerja dan Motivasi)

Regression ( Kemampuan Individu, Lingkungan kerja dan Motivasi) Regression (, Lingkungan kerja dan ) Descriptive Statistics Mean Std. Deviation N 3.87.333 32 3.47.672 32 3.78.585 32 s Pearson Sig. (-tailed) N Kemampuan Lingkungan Individu Kerja.000.432.49.432.000.3.49.3.000..000.000.000..000.000.000.

More information

Parametric Test. Multiple Linear Regression Spatial Application I: State Homicide Rates Equations taken from Zar, 1984.

Parametric Test. Multiple Linear Regression Spatial Application I: State Homicide Rates Equations taken from Zar, 1984. Multiple Linear Regression Spatial Application I: State Homicide Rates Equations taken from Zar, 984. y ˆ = a + b x + b 2 x 2K + b n x n where n is the number of variables Example: In an earlier bivariate

More information

Multiple linear regression S6

Multiple linear regression S6 Basic medical statistics for clinical and experimental research Multiple linear regression S6 Katarzyna Jóźwiak k.jozwiak@nki.nl November 15, 2017 1/42 Introduction Two main motivations for doing multiple

More information

Multiple Regression. More Hypothesis Testing. More Hypothesis Testing The big question: What we really want to know: What we actually know: We know:

Multiple Regression. More Hypothesis Testing. More Hypothesis Testing The big question: What we really want to know: What we actually know: We know: Multiple Regression Ψ320 Ainsworth More Hypothesis Testing What we really want to know: Is the relationship in the population we have selected between X & Y strong enough that we can use the relationship

More information

x3,..., Multiple Regression β q α, β 1, β 2, β 3,..., β q in the model can all be estimated by least square estimators

x3,..., Multiple Regression β q α, β 1, β 2, β 3,..., β q in the model can all be estimated by least square estimators Multiple Regression Relating a response (dependent, input) y to a set of explanatory (independent, output, predictor) variables x, x 2, x 3,, x q. A technique for modeling the relationship between variables.

More information

Item-Total Statistics. Corrected Item- Cronbach's Item Deleted. Total

Item-Total Statistics. Corrected Item- Cronbach's Item Deleted. Total 45 Lampiran 3 : Uji Validitas dan Reliabilitas Reliability Case Processing Summary N % Valid 75 00.0 Cases Excluded a 0.0 Total 75 00.0 a. Listwise deletion based on all variables in the procedure. Reliability

More information

Multivariate Correlational Analysis: An Introduction

Multivariate Correlational Analysis: An Introduction Assignment. Multivariate Correlational Analysis: An Introduction Mertler & Vanetta, Chapter 7 Kachigan, Chapter 4, pps 180-193 Terms you should know. Multiple Regression Linear Equations Least Squares

More information

Prepared by: Prof. Dr Bahaman Abu Samah Department of Professional Development and Continuing Education Faculty of Educational Studies Universiti

Prepared by: Prof. Dr Bahaman Abu Samah Department of Professional Development and Continuing Education Faculty of Educational Studies Universiti Prepared by: Prof Dr Bahaman Abu Samah Department of Professional Development and Continuing Education Faculty of Educational Studies Universiti Putra Malaysia Serdang M L Regression is an extension to

More information

ECON 497 Midterm Spring

ECON 497 Midterm Spring ECON 497 Midterm Spring 2009 1 ECON 497: Economic Research and Forecasting Name: Spring 2009 Bellas Midterm You have three hours and twenty minutes to complete this exam. Answer all questions and explain

More information

Sociology 593 Exam 1 February 14, 1997

Sociology 593 Exam 1 February 14, 1997 Sociology 9 Exam February, 997 I. True-False. ( points) Indicate whether the following statements are true or false. If false, briefly explain why.. There are IVs in a multiple regression model. If the

More information

Predictive Analytics : QM901.1x Prof U Dinesh Kumar, IIMB. All Rights Reserved, Indian Institute of Management Bangalore

Predictive Analytics : QM901.1x Prof U Dinesh Kumar, IIMB. All Rights Reserved, Indian Institute of Management Bangalore What is Multiple Linear Regression Several independent variables may influence the change in response variable we are trying to study. When several independent variables are included in the equation, the

More information

Introduction to Regression

Introduction to Regression Regression Introduction to Regression If two variables covary, we should be able to predict the value of one variable from another. Correlation only tells us how much two variables covary. In regression,

More information

STAT 3900/4950 MIDTERM TWO Name: Spring, 2015 (print: first last ) Covered topics: Two-way ANOVA, ANCOVA, SLR, MLR and correlation analysis

STAT 3900/4950 MIDTERM TWO Name: Spring, 2015 (print: first last ) Covered topics: Two-way ANOVA, ANCOVA, SLR, MLR and correlation analysis STAT 3900/4950 MIDTERM TWO Name: Spring, 205 (print: first last ) Covered topics: Two-way ANOVA, ANCOVA, SLR, MLR and correlation analysis Instructions: You may use your books, notes, and SPSS/SAS. NO

More information

Dr. Maddah ENMG 617 EM Statistics 11/28/12. Multiple Regression (3) (Chapter 15, Hines)

Dr. Maddah ENMG 617 EM Statistics 11/28/12. Multiple Regression (3) (Chapter 15, Hines) Dr. Maddah ENMG 617 EM Statistics 11/28/12 Multiple Regression (3) (Chapter 15, Hines) Problems in multiple regression: Multicollinearity This arises when the independent variables x 1, x 2,, x k, are

More information

Review of Multiple Regression

Review of Multiple Regression Ronald H. Heck 1 Let s begin with a little review of multiple regression this week. Linear models [e.g., correlation, t-tests, analysis of variance (ANOVA), multiple regression, path analysis, multivariate

More information

Multiple Regression and Model Building (cont d) + GIS Lecture 21 3 May 2006 R. Ryznar

Multiple Regression and Model Building (cont d) + GIS Lecture 21 3 May 2006 R. Ryznar Multiple Regression and Model Building (cont d) + GIS 11.220 Lecture 21 3 May 2006 R. Ryznar Model Summary b 1-[(SSE/n-k+1)/(SST/n-1)] Model 1 Adjusted Std. Error of R R Square R Square the Estimate.991

More information

Ref.: Spring SOS3003 Applied data analysis for social science Lecture note

Ref.:   Spring SOS3003 Applied data analysis for social science Lecture note SOS3003 Applied data analysis for social science Lecture note 05-2010 Erling Berge Department of sociology and political science NTNU Spring 2010 Erling Berge 2010 1 Literature Regression criticism I Hamilton

More information

Area1 Scaled Score (NAPLEX) .535 ** **.000 N. Sig. (2-tailed)

Area1 Scaled Score (NAPLEX) .535 ** **.000 N. Sig. (2-tailed) Institutional Assessment Report Texas Southern University College of Pharmacy and Health Sciences "An Analysis of 2013 NAPLEX, P4-Comp. Exams and P3 courses The following analysis illustrates relationships

More information

Assoc.Prof.Dr. Wolfgang Feilmayr Multivariate Methods in Regional Science: Regression and Correlation Analysis REGRESSION ANALYSIS

Assoc.Prof.Dr. Wolfgang Feilmayr Multivariate Methods in Regional Science: Regression and Correlation Analysis REGRESSION ANALYSIS REGRESSION ANALYSIS Regression Analysis can be broadly defined as the analysis of statistical relationships between one dependent and one or more independent variables. Although the terms dependent and

More information

LINEAR REGRESSION ANALYSIS. MODULE XVI Lecture Exercises

LINEAR REGRESSION ANALYSIS. MODULE XVI Lecture Exercises LINEAR REGRESSION ANALYSIS MODULE XVI Lecture - 44 Exercises Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur Exercise 1 The following data has been obtained on

More information

Multiple OLS Regression

Multiple OLS Regression Multiple OLS Regression Ronet Bachman, Ph.D. Presented by Justice Research and Statistics Association 12/8/2016 Justice Research and Statistics Association 720 7 th Street, NW, Third Floor Washington,

More information

ECON 4230 Intermediate Econometric Theory Exam

ECON 4230 Intermediate Econometric Theory Exam ECON 4230 Intermediate Econometric Theory Exam Multiple Choice (20 pts). Circle the best answer. 1. The Classical assumption of mean zero errors is satisfied if the regression model a) is linear in the

More information

Regression: Main Ideas Setting: Quantitative outcome with a quantitative explanatory variable. Example, cont.

Regression: Main Ideas Setting: Quantitative outcome with a quantitative explanatory variable. Example, cont. TCELL 9/4/205 36-309/749 Experimental Design for Behavioral and Social Sciences Simple Regression Example Male black wheatear birds carry stones to the nest as a form of sexual display. Soler et al. wanted

More information

Single and multiple linear regression analysis

Single and multiple linear regression analysis Single and multiple linear regression analysis Marike Cockeran 2017 Introduction Outline of the session Simple linear regression analysis SPSS example of simple linear regression analysis Additional topics

More information

Chapter 9 - Correlation and Regression

Chapter 9 - Correlation and Regression Chapter 9 - Correlation and Regression 9. Scatter diagram of percentage of LBW infants (Y) and high-risk fertility rate (X ) in Vermont Health Planning Districts. 9.3 Correlation between percentage of

More information

Psychology Seminar Psych 406 Dr. Jeffrey Leitzel

Psychology Seminar Psych 406 Dr. Jeffrey Leitzel Psychology Seminar Psych 406 Dr. Jeffrey Leitzel Structural Equation Modeling Topic 1: Correlation / Linear Regression Outline/Overview Correlations (r, pr, sr) Linear regression Multiple regression interpreting

More information

Practical Biostatistics

Practical Biostatistics Practical Biostatistics Clinical Epidemiology, Biostatistics and Bioinformatics AMC Multivariable regression Day 5 Recap Describing association: Correlation Parametric technique: Pearson (PMCC) Non-parametric:

More information

Simple Linear Regression

Simple Linear Regression Simple Linear Regression 1 Correlation indicates the magnitude and direction of the linear relationship between two variables. Linear Regression: variable Y (criterion) is predicted by variable X (predictor)

More information

36-309/749 Experimental Design for Behavioral and Social Sciences. Sep. 22, 2015 Lecture 4: Linear Regression

36-309/749 Experimental Design for Behavioral and Social Sciences. Sep. 22, 2015 Lecture 4: Linear Regression 36-309/749 Experimental Design for Behavioral and Social Sciences Sep. 22, 2015 Lecture 4: Linear Regression TCELL Simple Regression Example Male black wheatear birds carry stones to the nest as a form

More information

y response variable x 1, x 2,, x k -- a set of explanatory variables

y response variable x 1, x 2,, x k -- a set of explanatory variables 11. Multiple Regression and Correlation y response variable x 1, x 2,, x k -- a set of explanatory variables In this chapter, all variables are assumed to be quantitative. Chapters 12-14 show how to incorporate

More information

Simple Linear Regression Using Ordinary Least Squares

Simple Linear Regression Using Ordinary Least Squares Simple Linear Regression Using Ordinary Least Squares Purpose: To approximate a linear relationship with a line. Reason: We want to be able to predict Y using X. Definition: The Least Squares Regression

More information

General linear models. One and Two-way ANOVA in SPSS Repeated measures ANOVA Multiple linear regression

General linear models. One and Two-way ANOVA in SPSS Repeated measures ANOVA Multiple linear regression General linear models One and Two-way ANOVA in SPSS Repeated measures ANOVA Multiple linear regression 2-way ANOVA in SPSS Example 14.1 2 3 2-way ANOVA in SPSS Click Add 4 Repeated measures The stroop

More information

Analysing data: regression and correlation S6 and S7

Analysing data: regression and correlation S6 and S7 Basic medical statistics for clinical and experimental research Analysing data: regression and correlation S6 and S7 K. Jozwiak k.jozwiak@nki.nl 2 / 49 Correlation So far we have looked at the association

More information

REVIEW 8/2/2017 陈芳华东师大英语系

REVIEW 8/2/2017 陈芳华东师大英语系 REVIEW Hypothesis testing starts with a null hypothesis and a null distribution. We compare what we have to the null distribution, if the result is too extreme to belong to the null distribution (p

More information

bivariate correlation bivariate regression multiple regression

bivariate correlation bivariate regression multiple regression bivariate correlation bivariate regression multiple regression Today Bivariate Correlation Pearson product-moment correlation (r) assesses nature and strength of the linear relationship between two continuous

More information

1 Correlation and Inference from Regression

1 Correlation and Inference from Regression 1 Correlation and Inference from Regression Reading: Kennedy (1998) A Guide to Econometrics, Chapters 4 and 6 Maddala, G.S. (1992) Introduction to Econometrics p. 170-177 Moore and McCabe, chapter 12 is

More information

2 Prediction and Analysis of Variance

2 Prediction and Analysis of Variance 2 Prediction and Analysis of Variance Reading: Chapters and 2 of Kennedy A Guide to Econometrics Achen, Christopher H. Interpreting and Using Regression (London: Sage, 982). Chapter 4 of Andy Field, Discovering

More information

Ordinary Least Squares Regression Explained: Vartanian

Ordinary Least Squares Regression Explained: Vartanian Ordinary Least Squares Regression Eplained: Vartanian When to Use Ordinary Least Squares Regression Analysis A. Variable types. When you have an interval/ratio scale dependent variable.. When your independent

More information

Correlation. A statistics method to measure the relationship between two variables. Three characteristics

Correlation. A statistics method to measure the relationship between two variables. Three characteristics Correlation Correlation A statistics method to measure the relationship between two variables Three characteristics Direction of the relationship Form of the relationship Strength/Consistency Direction

More information

Multicollinearity Richard Williams, University of Notre Dame, https://www3.nd.edu/~rwilliam/ Last revised January 13, 2015

Multicollinearity Richard Williams, University of Notre Dame, https://www3.nd.edu/~rwilliam/ Last revised January 13, 2015 Multicollinearity Richard Williams, University of Notre Dame, https://www3.nd.edu/~rwilliam/ Last revised January 13, 2015 Stata Example (See appendices for full example).. use http://www.nd.edu/~rwilliam/stats2/statafiles/multicoll.dta,

More information

Regression Diagnostics Procedures

Regression Diagnostics Procedures Regression Diagnostics Procedures ASSUMPTIONS UNDERLYING REGRESSION/CORRELATION NORMALITY OF VARIANCE IN Y FOR EACH VALUE OF X For any fixed value of the independent variable X, the distribution of the

More information

Regression Model Building

Regression Model Building Regression Model Building Setting: Possibly a large set of predictor variables (including interactions). Goal: Fit a parsimonious model that explains variation in Y with a small set of predictors Automated

More information

Multiple Regression and Model Building Lecture 20 1 May 2006 R. Ryznar

Multiple Regression and Model Building Lecture 20 1 May 2006 R. Ryznar Multiple Regression and Model Building 11.220 Lecture 20 1 May 2006 R. Ryznar Building Models: Making Sure the Assumptions Hold 1. There is a linear relationship between the explanatory (independent) variable(s)

More information

Chapter Goals. To understand the methods for displaying and describing relationship among variables. Formulate Theories.

Chapter Goals. To understand the methods for displaying and describing relationship among variables. Formulate Theories. Chapter Goals To understand the methods for displaying and describing relationship among variables. Formulate Theories Interpret Results/Make Decisions Collect Data Summarize Results Chapter 7: Is There

More information

A discussion on multiple regression models

A discussion on multiple regression models A discussion on multiple regression models In our previous discussion of simple linear regression, we focused on a model in which one independent or explanatory variable X was used to predict the value

More information

Sociology 593 Exam 1 February 17, 1995

Sociology 593 Exam 1 February 17, 1995 Sociology 593 Exam 1 February 17, 1995 I. True-False. (25 points) Indicate whether the following statements are true or false. If false, briefly explain why. 1. A researcher regressed Y on. When he plotted

More information

Sociology 593 Exam 1 Answer Key February 17, 1995

Sociology 593 Exam 1 Answer Key February 17, 1995 Sociology 593 Exam 1 Answer Key February 17, 1995 I. True-False. (5 points) Indicate whether the following statements are true or false. If false, briefly explain why. 1. A researcher regressed Y on. When

More information

Using the Regression Model in multivariate data analysis

Using the Regression Model in multivariate data analysis Bulletin of the Transilvania University of Braşov Series V: Economic Sciences Vol. 10 (59) No. 1-2017 Using the Regression Model in multivariate data analysis Cristinel CONSTANTIN 1 Abstract: This paper

More information

Multiple Regression. Peerapat Wongchaiwat, Ph.D.

Multiple Regression. Peerapat Wongchaiwat, Ph.D. Peerapat Wongchaiwat, Ph.D. wongchaiwat@hotmail.com The Multiple Regression Model Examine the linear relationship between 1 dependent (Y) & 2 or more independent variables (X i ) Multiple Regression Model

More information

A particularly nasty aspect of this is that it is often difficult or impossible to tell if a model fails to satisfy these steps.

A particularly nasty aspect of this is that it is often difficult or impossible to tell if a model fails to satisfy these steps. ECON 497: Lecture 6 Page 1 of 1 Metropolitan State University ECON 497: Research and Forecasting Lecture Notes 6 Specification: Choosing the Independent Variables Studenmund Chapter 6 Before we start,

More information

Correlation and Regression Bangkok, 14-18, Sept. 2015

Correlation and Regression Bangkok, 14-18, Sept. 2015 Analysing and Understanding Learning Assessment for Evidence-based Policy Making Correlation and Regression Bangkok, 14-18, Sept. 2015 Australian Council for Educational Research Correlation The strength

More information

MULTIPLE REGRESSION AND ISSUES IN REGRESSION ANALYSIS

MULTIPLE REGRESSION AND ISSUES IN REGRESSION ANALYSIS MULTIPLE REGRESSION AND ISSUES IN REGRESSION ANALYSIS Page 1 MSR = Mean Regression Sum of Squares MSE = Mean Squared Error RSS = Regression Sum of Squares SSE = Sum of Squared Errors/Residuals α = Level

More information

Daniel Boduszek University of Huddersfield

Daniel Boduszek University of Huddersfield Daniel Boduszek University of Huddersfield d.boduszek@hud.ac.uk Introduction to moderator effects Hierarchical Regression analysis with continuous moderator Hierarchical Regression analysis with categorical

More information

TOPIC 9 SIMPLE REGRESSION & CORRELATION

TOPIC 9 SIMPLE REGRESSION & CORRELATION TOPIC 9 SIMPLE REGRESSION & CORRELATION Basic Linear Relationships Mathematical representation: Y = a + bx X is the independent variable [the variable whose value we can choose, or the input variable].

More information

LAB 3 INSTRUCTIONS SIMPLE LINEAR REGRESSION

LAB 3 INSTRUCTIONS SIMPLE LINEAR REGRESSION LAB 3 INSTRUCTIONS SIMPLE LINEAR REGRESSION In this lab you will first learn how to display the relationship between two quantitative variables with a scatterplot and also how to measure the strength of

More information

EDF 7405 Advanced Quantitative Methods in Educational Research MULTR.SAS

EDF 7405 Advanced Quantitative Methods in Educational Research MULTR.SAS EDF 7405 Advanced Quantitative Methods in Educational Research MULTR.SAS The data used in this example describe teacher and student behavior in 8 classrooms. The variables are: Y percentage of interventions

More information

Available online at (Elixir International Journal) Statistics. Elixir Statistics 49 (2012)

Available online at   (Elixir International Journal) Statistics. Elixir Statistics 49 (2012) 10108 Available online at www.elixirpublishers.com (Elixir International Journal) Statistics Elixir Statistics 49 (2012) 10108-10112 The detention and correction of multicollinearity effects in a multiple

More information

Example: Forced Expiratory Volume (FEV) Program L13. Example: Forced Expiratory Volume (FEV) Example: Forced Expiratory Volume (FEV)

Example: Forced Expiratory Volume (FEV) Program L13. Example: Forced Expiratory Volume (FEV) Example: Forced Expiratory Volume (FEV) Program L13 Relationships between two variables Correlation, cont d Regression Relationships between more than two variables Multiple linear regression Two numerical variables Linear or curved relationship?

More information

Bivariate Regression Analysis. The most useful means of discerning causality and significance of variables

Bivariate Regression Analysis. The most useful means of discerning causality and significance of variables Bivariate Regression Analysis The most useful means of discerning causality and significance of variables Purpose of Regression Analysis Test causal hypotheses Make predictions from samples of data Derive

More information

STATISTICS 110/201 PRACTICE FINAL EXAM

STATISTICS 110/201 PRACTICE FINAL EXAM STATISTICS 110/201 PRACTICE FINAL EXAM Questions 1 to 5: There is a downloadable Stata package that produces sequential sums of squares for regression. In other words, the SS is built up as each variable

More information

Chapter 19: Logistic regression

Chapter 19: Logistic regression Chapter 19: Logistic regression Self-test answers SELF-TEST Rerun this analysis using a stepwise method (Forward: LR) entry method of analysis. The main analysis To open the main Logistic Regression dialog

More information

Last updated: Oct 18, 2012 LINEAR REGRESSION PSYC 3031 INTERMEDIATE STATISTICS LABORATORY. J. Elder

Last updated: Oct 18, 2012 LINEAR REGRESSION PSYC 3031 INTERMEDIATE STATISTICS LABORATORY. J. Elder Last updated: Oct 18, 2012 LINEAR REGRESSION Acknowledgements 2 Some of these slides have been sourced or modified from slides created by A. Field for Discovering Statistics using R. Simple Linear Objectives

More information

12.12 MODEL BUILDING, AND THE EFFECTS OF MULTICOLLINEARITY (OPTIONAL)

12.12 MODEL BUILDING, AND THE EFFECTS OF MULTICOLLINEARITY (OPTIONAL) 12.12 Model Building, and the Effects of Multicollinearity (Optional) 1 Although Excel and MegaStat are emphasized in Business Statistics in Practice, Second Canadian Edition, some examples in the additional

More information

CHAPTER 6: SPECIFICATION VARIABLES

CHAPTER 6: SPECIFICATION VARIABLES Recall, we had the following six assumptions required for the Gauss-Markov Theorem: 1. The regression model is linear, correctly specified, and has an additive error term. 2. The error term has a zero

More information

Ridge Regression. Summary. Sample StatFolio: ridge reg.sgp. STATGRAPHICS Rev. 10/1/2014

Ridge Regression. Summary. Sample StatFolio: ridge reg.sgp. STATGRAPHICS Rev. 10/1/2014 Ridge Regression Summary... 1 Data Input... 4 Analysis Summary... 5 Analysis Options... 6 Ridge Trace... 7 Regression Coefficients... 8 Standardized Regression Coefficients... 9 Observed versus Predicted...

More information

SPSS Output. ANOVA a b Residual Coefficients a Standardized Coefficients

SPSS Output. ANOVA a b Residual Coefficients a Standardized Coefficients SPSS Output Homework 1-1e ANOVA a Sum of Squares df Mean Square F Sig. 1 Regression 351.056 1 351.056 11.295.002 b Residual 932.412 30 31.080 Total 1283.469 31 a. Dependent Variable: Sexual Harassment

More information

ESP 178 Applied Research Methods. 2/23: Quantitative Analysis

ESP 178 Applied Research Methods. 2/23: Quantitative Analysis ESP 178 Applied Research Methods 2/23: Quantitative Analysis Data Preparation Data coding create codebook that defines each variable, its response scale, how it was coded Data entry for mail surveys and

More information

Chapter 4 Regression with Categorical Predictor Variables Page 1. Overview of regression with categorical predictors

Chapter 4 Regression with Categorical Predictor Variables Page 1. Overview of regression with categorical predictors Chapter 4 Regression with Categorical Predictor Variables Page. Overview of regression with categorical predictors 4-. Dummy coding 4-3 4-5 A. Karpinski Regression with Categorical Predictor Variables.

More information

9. Linear Regression and Correlation

9. Linear Regression and Correlation 9. Linear Regression and Correlation Data: y a quantitative response variable x a quantitative explanatory variable (Chap. 8: Recall that both variables were categorical) For example, y = annual income,

More information

Multiple Linear Regression I. Lecture 7. Correlation (Review) Overview

Multiple Linear Regression I. Lecture 7. Correlation (Review) Overview Multiple Linear Regression I Lecture 7 Image source:http://commons.wikimedia.org/wiki/file:vidrarias_de_laboratorio.jpg Survey Research & Design in Psychology James Neill, 017 Creative Commons Attribution

More information

SMAM 314 Practice Final Examination Winter 2003

SMAM 314 Practice Final Examination Winter 2003 SMAM 314 Practice Final Examination Winter 2003 You may use your textbook, one page of notes and a calculator. Please hand in the notes with your exam. 1. Mark the following statements True T or False

More information

(4) 1. Create dummy variables for Town. Name these dummy variables A and B. These 0,1 variables now indicate the location of the house.

(4) 1. Create dummy variables for Town. Name these dummy variables A and B. These 0,1 variables now indicate the location of the house. Exam 3 Resource Economics 312 Introductory Econometrics Please complete all questions on this exam. The data in the spreadsheet: Exam 3- Home Prices.xls are to be used for all analyses. These data are

More information

Review of Statistics 101

Review of Statistics 101 Review of Statistics 101 We review some important themes from the course 1. Introduction Statistics- Set of methods for collecting/analyzing data (the art and science of learning from data). Provides methods

More information

SPSS LAB FILE 1

SPSS LAB FILE  1 SPSS LAB FILE www.mcdtu.wordpress.com 1 www.mcdtu.wordpress.com 2 www.mcdtu.wordpress.com 3 OBJECTIVE 1: Transporation of Data Set to SPSS Editor INPUTS: Files: group1.xlsx, group1.txt PROCEDURE FOLLOWED:

More information

Final Exam - Solutions

Final Exam - Solutions Ecn 102 - Analysis of Economic Data University of California - Davis March 19, 2010 Instructor: John Parman Final Exam - Solutions You have until 5:30pm to complete this exam. Please remember to put your

More information

Univariate analysis. Simple and Multiple Regression. Univariate analysis. Simple Regression How best to summarise the data?

Univariate analysis. Simple and Multiple Regression. Univariate analysis. Simple Regression How best to summarise the data? Univariate analysis Example - linear regression equation: y = ax + c Least squares criteria ( yobs ycalc ) = yobs ( ax + c) = minimum Simple and + = xa xc xy xa + nc = y Solve for a and c Univariate analysis

More information

Inter Item Correlation Matrix (R )

Inter Item Correlation Matrix (R ) 7 1. I have the ability to influence my child s well-being. 2. Whether my child avoids injury is just a matter of luck. 3. Luck plays a big part in determining how healthy my child is. 4. I can do a lot

More information

Regression of Inflation on Percent M3 Change

Regression of Inflation on Percent M3 Change ECON 497 Final Exam Page of ECON 497: Economic Research and Forecasting Name: Spring 2006 Bellas Final Exam Return this exam to me by midnight on Thursday, April 27. It may be e-mailed to me. It may be

More information

Project Report for STAT571 Statistical Methods Instructor: Dr. Ramon V. Leon. Wage Data Analysis. Yuanlei Zhang

Project Report for STAT571 Statistical Methods Instructor: Dr. Ramon V. Leon. Wage Data Analysis. Yuanlei Zhang Project Report for STAT7 Statistical Methods Instructor: Dr. Ramon V. Leon Wage Data Analysis Yuanlei Zhang 77--7 November, Part : Introduction Data Set The data set contains a random sample of observations

More information

Interactions and Centering in Regression: MRC09 Salaries for graduate faculty in psychology

Interactions and Centering in Regression: MRC09 Salaries for graduate faculty in psychology Psychology 308c Dale Berger Interactions and Centering in Regression: MRC09 Salaries for graduate faculty in psychology This example illustrates modeling an interaction with centering and transformations.

More information

Technical Appendix C: Methods. Multilevel Regression Models

Technical Appendix C: Methods. Multilevel Regression Models Technical Appendix C: Methods Multilevel Regression Models As not all readers may be familiar with the analytical methods used in this study, a brief note helps to clarify the techniques. The firewall

More information

Chapter 3 Multiple Regression Complete Example

Chapter 3 Multiple Regression Complete Example Department of Quantitative Methods & Information Systems ECON 504 Chapter 3 Multiple Regression Complete Example Spring 2013 Dr. Mohammad Zainal Review Goals After completing this lecture, you should be

More information

CHAPTER 5 FUNCTIONAL FORMS OF REGRESSION MODELS

CHAPTER 5 FUNCTIONAL FORMS OF REGRESSION MODELS CHAPTER 5 FUNCTIONAL FORMS OF REGRESSION MODELS QUESTIONS 5.1. (a) In a log-log model the dependent and all explanatory variables are in the logarithmic form. (b) In the log-lin model the dependent variable

More information

Chapter 14 Student Lecture Notes 14-1

Chapter 14 Student Lecture Notes 14-1 Chapter 14 Student Lecture Notes 14-1 Business Statistics: A Decision-Making Approach 6 th Edition Chapter 14 Multiple Regression Analysis and Model Building Chap 14-1 Chapter Goals After completing this

More information

Multivariate Lineare Modelle

Multivariate Lineare Modelle 0-1 TALEB AHMAD CASE - Center for Applied Statistics and Economics Humboldt-Universität zu Berlin Motivation 1-1 Motivation Multivariate regression models can accommodate many explanatory which simultaneously

More information

Correlation and simple linear regression S5

Correlation and simple linear regression S5 Basic medical statistics for clinical and eperimental research Correlation and simple linear regression S5 Katarzyna Jóźwiak k.jozwiak@nki.nl November 15, 2017 1/41 Introduction Eample: Brain size and

More information

Can you tell the relationship between students SAT scores and their college grades?

Can you tell the relationship between students SAT scores and their college grades? Correlation One Challenge Can you tell the relationship between students SAT scores and their college grades? A: The higher SAT scores are, the better GPA may be. B: The higher SAT scores are, the lower

More information

Lecture 4: Multivariate Regression, Part 2

Lecture 4: Multivariate Regression, Part 2 Lecture 4: Multivariate Regression, Part 2 Gauss-Markov Assumptions 1) Linear in Parameters: Y X X X i 0 1 1 2 2 k k 2) Random Sampling: we have a random sample from the population that follows the above

More information

The simple linear regression model discussed in Chapter 13 was written as

The simple linear regression model discussed in Chapter 13 was written as 1519T_c14 03/27/2006 07:28 AM Page 614 Chapter Jose Luis Pelaez Inc/Blend Images/Getty Images, Inc./Getty Images, Inc. 14 Multiple Regression 14.1 Multiple Regression Analysis 14.2 Assumptions of the Multiple

More information

Applied Econometrics. Applied Econometrics Second edition. Dimitrios Asteriou and Stephen G. Hall

Applied Econometrics. Applied Econometrics Second edition. Dimitrios Asteriou and Stephen G. Hall Applied Econometrics Second edition Dimitrios Asteriou and Stephen G. Hall MULTICOLLINEARITY 1. Perfect Multicollinearity 2. Consequences of Perfect Multicollinearity 3. Imperfect Multicollinearity 4.

More information

Hypothesis testing Goodness of fit Multicollinearity Prediction. Applied Statistics. Lecturer: Serena Arima

Hypothesis testing Goodness of fit Multicollinearity Prediction. Applied Statistics. Lecturer: Serena Arima Applied Statistics Lecturer: Serena Arima Hypothesis testing for the linear model Under the Gauss-Markov assumptions and the normality of the error terms, we saw that β N(β, σ 2 (X X ) 1 ) and hence s

More information

Day 4: Shrinkage Estimators

Day 4: Shrinkage Estimators Day 4: Shrinkage Estimators Kenneth Benoit Data Mining and Statistical Learning March 9, 2015 n versus p (aka k) Classical regression framework: n > p. Without this inequality, the OLS coefficients have

More information

FinQuiz Notes

FinQuiz Notes Reading 10 Multiple Regression and Issues in Regression Analysis 2. MULTIPLE LINEAR REGRESSION Multiple linear regression is a method used to model the linear relationship between a dependent variable

More information

MORE ON SIMPLE REGRESSION: OVERVIEW

MORE ON SIMPLE REGRESSION: OVERVIEW FI=NOT0106 NOTICE. Unless otherwise indicated, all materials on this page and linked pages at the blue.temple.edu address and at the astro.temple.edu address are the sole property of Ralph B. Taylor and

More information

( ), which of the coefficients would end

( ), which of the coefficients would end Discussion Sheet 29.7.9 Qualitative Variables We have devoted most of our attention in multiple regression to quantitative or numerical variables. MR models can become more useful and complex when we consider

More information

DEVELOPMENT OF CRASH PREDICTION MODEL USING MULTIPLE REGRESSION ANALYSIS Harshit Gupta 1, Dr. Siddhartha Rokade 2 1

DEVELOPMENT OF CRASH PREDICTION MODEL USING MULTIPLE REGRESSION ANALYSIS Harshit Gupta 1, Dr. Siddhartha Rokade 2 1 DEVELOPMENT OF CRASH PREDICTION MODEL USING MULTIPLE REGRESSION ANALYSIS Harshit Gupta 1, Dr. Siddhartha Rokade 2 1 PG Student, 2 Assistant Professor, Department of Civil Engineering, Maulana Azad National

More information

Research Design - - Topic 19 Multiple regression: Applications 2009 R.C. Gardner, Ph.D.

Research Design - - Topic 19 Multiple regression: Applications 2009 R.C. Gardner, Ph.D. Research Design - - Topic 19 Multiple regression: Applications 2009 R.C. Gardner, Ph.D. Curve Fitting Mediation analysis Moderation Analysis 1 Curve Fitting The investigation of non-linear functions using

More information

Regression Analysis. A statistical procedure used to find relations among a set of variables.

Regression Analysis. A statistical procedure used to find relations among a set of variables. Regression Analysis A statistical procedure used to find relations among a set of variables. Understanding relations Mapping data enables us to examine (describe) where things occur (e.g., areas where

More information

Lecture 4: Multivariate Regression, Part 2

Lecture 4: Multivariate Regression, Part 2 Lecture 4: Multivariate Regression, Part 2 Gauss-Markov Assumptions 1) Linear in Parameters: Y X X X i 0 1 1 2 2 k k 2) Random Sampling: we have a random sample from the population that follows the above

More information

: The model hypothesizes a relationship between the variables. The simplest probabilistic model: or.

: The model hypothesizes a relationship between the variables. The simplest probabilistic model: or. Chapter Simple Linear Regression : comparing means across groups : presenting relationships among numeric variables. Probabilistic Model : The model hypothesizes an relationship between the variables.

More information