Ch14. Multiple Regression Analysis
|
|
- Maude Elliott
- 5 years ago
- Views:
Transcription
1 Ch14. Multiple Regression Analysis 1
2 Goals : multiple regression analysis Model Building and Estimating More than 1 independent variables Quantitative( 量 ) independent variables Qualitative( ) independent variables: dummy variables Regression coefficients Multiple standard error of estimate Model Evaluation Goodness-of-fit: global and individual linearity Multi-collinearity Model assumptions diagnostic: analysis of residuals, residual plot 2
3 略 remedies. If not ok, try a new model. If not ok, need some adequate 立 立 度 1. Linearity 2. Multi-collinearity 1. 立 異數 3
4 A multiple regression analysis : When there are k independent variables, X1,X2,,Xk, the multiple regression equation is : µ = α + β X + β X + L+ Y β k X k 4
5 A multiple regression analysis : Where µ Y = the Y-intercept = when X1=X2= =0. 1, 2,, k = net/partial regression coefficients, 1 = the net change in mean of Y for each unit change in X1 when other variables X2,,Xk are kept constants. 數 X1 Y 數 量 5
6 If there are k=2 independent variables, see Chart
7 For qualitative X Recall that a regression model establishes systematic( ) relationship between two continuous variables, independent and dependent variable. What if some independent variables are nominal-scale/ qualitative( )? Ans. Using a dummy variable( 數 ) to replace the original variable. 7
8 Example. X1=X2= Y= X1, Y are continuous variables, while X2 is nominal. If X2=male, y = X1 If X2=female, y = X1 How to express such a model? 8
9 Dummy variable : a variable with only two possible outcomes, 0 or 1 I=1, if success ; I=0, if failure. 9
10 Example. Let I2 = 1, if X2=female, I2=0, if X2=male, Multiple regression model : previous model is expressed as X 1, X2 = male µ Y = X 1, X2 = female X 1, X2 = male, I2 = 0 = X1+ 5, X2 = female, I2 = 1 = X + 5I = α + β X + β I =0.6 : I2=0, 0 數 0.6 不 女 數 1=0.9 2=5 X1 女 (I=1)(I=0) 數 2=5 10
11 female male 11
12 數 數 Ex X 1, X2 = male, I2 = 0 µ Y = X1+ 0.3X 1, X2 = female, I2 = 1 = X + 0.3X I = α + β X + β X I µ Y = x female male 0.6 µ Y = x 12
13 (1%) Bonus 1 : 數 數 數 µ Y X 1, X2 = male,i2 = 0 = X 1, X2 = female, I2 = 1 Bonus 2 : X1= X2= ( 金 ) Y= 列 X1 X2 = 金 µ Y = X1 X2 = X1 X2 = X Y 13
14 14 Estimating the regression equation : The multiple regression equation is estimated by where a, b1,, bk are the least squared estimates (LSE). The calculations are tedious as k becomes large. Example. K=2, two independent variable, solving the equations : Many software packages provide LSEs. k k X b X b X b a ' Y = L + + = + + = + + = x b x x b x a yx x x b x b x a yx x b x b an y
15 Example. P477 Salsberry Realty sells homes along the east coast of USA. How much can one expect to pay to heat it during the winter? frequently asked by customers. Independent variables : X s 1. The mean daily outside temperature 2. The number of inches of insulation( ) in the attic( 樓 ) 3. The age of the furnace( 爐 ) Dependent variable : Y = heating cost n=20 houses were sampled and investigated. 15
16 Answer the following questions : 1. Determine the multiple regression equation 2. Discuss the regression coefficients 3. What does it indicate that some are positive and some are negative? What is the intercept? 4. What is the estimated heating cost for a home if the mean outside temperature is 30 degrees, there are 5 inches of insulation in the attic, and the furnace is 10 years old? 16
17 Home Heating cost (Y) Mean outside temp. (X1) Attic insulation (X2) Age of furnace (X3) 17
18 數 數 度 數 Y = X X X3 18
19 Findings : 1. Y = X X X3 2. The intercept is b1, b2 are negative, X1, X2 have inverse relationship. As the outside temperature X1 increases, the mean heating cost will go down. reasonable. For each degree the mean temperature increases, the mean heating cost decreases 4.58 per month. The more insulation in the attic, the less the heating cost. 4. b3=6.1 > 0, X3 has a direct relationship. An older furnace, more heating cost. 5. If X1=30, X2=5, X3=10, the estimated heating cost is Y = (30) (5) (10) =
20 More on estimation : Multiple standard error of estimate: A measure of the error or variability in the prediction. Formula : S 2 y 12Kk = = (Y Y') = n (k + 1) SSE n (k + 1) MSE Residual = Y-Y = The standard error of estimate helps to construct confidence intervals and prediction intervals. Why the degrees of freedom is n-(k+1)? There are n responses, Y1,,Yn. The Y is determined by the predicted equation with (k+1) estimated coefficients 20
21 Home Heating cost (Y) (Y-Y') (Y-Y')^2 sum S 2 y 123 = = (Y Y') = n (k + 1) (3 + 1)
22 Or, the estimate can be found in the output of EXCEL 數 數 S y 123 = 度 S 123 y = MSE = 2606 =
23 Model fit Model Evaluation 1. There is a linear relationship between each X1, Xk and Y Global test : (X1, Xk) vs. Y Individual regression coefficient : Xi vs. Y 2. There is no correlation among X1,,Xk If there is, multicollinearity exists. Diagnosed by correlation matrix Model assumptions diagnostic 1. All pairs of observation (X1,,Xk, Y) are independent. Residual plots 2. The random error = Y Y ~ Normal(0, 2 ), equal variance Residual plot, normal plot Homoscedasticity = equal variance 23
24 Linearity Linear relationship between X1, Xk and Y Global linearity : jointly, (X1,, Xk), has linear relationship with Y. Individual linearity : each X1,X2,,Xk, has linear relationship with Y Methods : Subjective : eyeball, r 2 Objective : statistical tests 24
25 Linearity : subjective methods Individual linearity between Xi and Y. Scatter diagrams : Plots of (X1, Y), (X2, Y),, (Xk, Y) linear relationship : positively linear, negatively linear Correlation matrix : a matrix showing the correlation coefficient r between all pairs of variables. Off-diagonal : correlation coefficients The correlation between (X1, Y),, (Xk, Y) should be r 1 or r -1 25
26 Example. P485 scatter plots 26
27 Example. P486 EXCEL : correlation matrix X1, X2 are negatively related to Y, while X3 is positively related. X1 has strongest correlation with Y X2 has weakest correlation with Y. 27
28 Linearity : subjective methods Global linear relationship between (X1,.., Xk) and Y. Coefficient of multiple determination r 2 : ANOVA table The proportion of the total variation of Y explained by X1,, Xk r 2 = SSR SStotal = 1 SSE SStotal ANOVA Table Source of Variation Sum of Squares Degrees of Freedom Mean Square F Regression SSR k SSR/k=MSR MST/MSE Error SSE n-k-1 SSE/(n-k-1)=MSE Total SS total n-1 28
29 Example. r 2 can be found in the output of EXCEL 數 數 r 2 = 80.42% 度 or SSR r 2 = = = SStotal X1, X2, X3 80% Y 異 29
30 Linearity : objective methods Objectively, the hypothesis of linearity are tested. Global linearity : whether jointly, (X 1,, X k ), has linear relationship with Y? Whether all population coefficients 1,, k are not 0? H0 : β1 =... = βk = 0 --F test in ANOVA! Individual linearity : whether individually, each X 1,X 2,,X k, has linear relationship with Y? Whether any of the population coefficient 1,, k is not 0? Eg, for X1, testing H : 0 vs H : 0 --t-test! 0 β1 = 0 β1 30
31 Individual test (P490) Step 1. Hypotheses H : β = 0 vs H : β Step 2. Significant level Step 3. Test statistic : t-test statistic b1 0 t = = SE(b ) 1 b s 1 b 1 31
32 Step 4. Decision rule : A two-sided t-test Since under null hypothesis, t ~ t distribution with d.f. n-(k+1). H0 is rejected if t t Or if p-value ( n (k+ 1), α / 2), t t(n (k+ 1), α / 2) Step 5. Conclusion : 32
33 Example. P518 Since n-k-1=16, =0.05,critical values = t 16,0.025 = 2.12 數 Conclusion : if = For intercept, a = , SE(a)=59.6, t=7.17, p-value=0, significant! 2. For X1, b1=-4.58, SE(b1)=0.77, t = -5.93, p-value = 0.00, significant! 3. For X2, b2=-14.83, SE(b2)=4.75, t = -3.12, p-value=0.0066, significant! 4. For X3, b3=6.10, SE(b3)=4.01, t = 1.52, p-value= > 0.05, not significant! Recall : in the correlation matrix, r(y,x3)=0.53 is quite large, why the linearity is insignificant here? r(y,x2)=-0.25 is close to 0, why the linearity is significant here? 33
34 Global test (P ) Step 1. Hypotheses H0 : β1 = β2 =... = βk = 0 Step 2. Significant level Step 3. Test statistic : F-test statistic SSR / k F = = SSE /(n (k + 1)) MSR MSE 34
35 Step 4. Decision rule : A one-sided F-test : significant if F is large Since under null hypothesis, F ~ F distribution with d.f. (k, n-(k+1)). H0 is rejected if F Or if p-value F (k,n (k+ 1), α) Step 5. Conclusion : 35
36 Example. P488 Since k=3, n-k-1=16, =0.05,critical values =F (3,16,0.05) =3.24 度 Conclusion : at =0.05, H0 is rejected since F=21.9 > 3.24 or p-value = < 0.05, 36
37 Strategy for model selection : (P490) how many independent variables should be in the model? 1. Develop a multiple regression equation based on all independent variables. 1) Global test : significant? If not : stop and conclude that (X1,,Xk) are uncorrelated with Y. If yes : continue to 1-2). 2) Individual test : significant? If all are, go to 3. If some are, some are not: go to Remove the X with the largest p-value, back to 1. Delete the most insignificant independent variable. 3. The global and individual linearity are significant, check the model assumptions. 37
38 Is there any nonlinear relationship between X and Y? Residual = e = Y-Y = unexplained error/variation Is there any systematic pattern in a residual plot : (X, e)? If the model is right, Y ~ N( µ e The residuals are around 0 and independent with X. If the model is not right, e.g. Y ~ N( µ e Y = Y Y' Y = α + β x, σ Y µ = α + β x + β = Y (a + bx) 1 1 Y ),Y' = ~ N(0, σ Y ( α + β ),Y' = x) ~ N( β a + bx The residual is a quadratic function of X If the nonlinear relationship exists, the model should be modified. 2 2 x 2, σ a + bx ) 2 x 2, σ 2 ) 38
39 39
40 2. Check the multicollinearity between Xs Multicollinearity : correlation exists among the independent variables Xs. Multicollinearity can distort the SE(b) and lead to incorrect conclusions in hypotheses testing. SE(b) becomes large, the conclusion is insignificant. In previous example, X1, X3 are correlated. Method : check the X part in the correlation matrix Multicollinearity exists if r > 0.7, or r < -0.7 Strategy : If multicollinearity exists, drop one of the independent variables and rebuild the model. 40
41 Example. P486 EXCEL Slight correlations between (X1, X2) (X2, X3) Moderate negative correlation between (X1, X3) Recall that H 0 : 3 =0 is not rejected. 41
42 Model assumptions : If the model is correct, Y ~ N( µ = α + β x, σ e = Y Y' Y 1,..,Y n are independent e i,,e n are independent approximately Y 1,..,Y n ~Normal population distribution e i,,e n ~normal Y Y µ ),Y' = ~ N(0, σ a + bx Y 1,..,Y n has constant variance at each level of x 1 Y 2 2 ) 42
43 Under independency, 3. Assumption of independence 1. The observed value should be independent with the sampling order (i), Residual plot : ( i, e) 2. The successive observations should be uncorrelated. Residual plot : (e(i), e(i+1)) 43
44 3. Assumption of independence-1 The residuals ei should be independent with the order i Plot ( i, e i ) 44
45 3. Assumption of independence -2 There should be no systematic pattern between successive obsn s Plot e,e ) ( i i+ 1 45
46 4. Assumption of normality and equal variance Normal distribution : the residuals, e s ~ normal 1. Histogram of e s. : bell-shaped, symmetric Example. Model : X1, X2, X3 P505 residual 率 率 46
47 2. Normal probability plot, p-p plot : 率 : linear Example. Model : X1, X2, X3 P505 率 Nearly a straight line, we can conclude that normal is true. 47
48 Equal variance/homoscedasticity The distributions of Y at different X-levels have equal variances. homoscedasticity If the variances are not equal, SE(regression coeff.) is understated t-statistic is too large incorrectly conclude the significance of X. If the variances are not equal, Select other independent variables Some transformations on X or Y 48
49 The residuals should have equal variations at different X- levels. Check the residual plot (X, e) or (, e) Ŷ P526 Example 1. Unequal variance : increased. Example 2. Other association, quadratic, may exists. 49
50 50
51 Example. An analyst is studying the effect of tire pressure on fuel economy (Mpg) for a fleet of 24 sedans used by regional supervisors. There are four different cars driven with a tire pressure of 30, 31, 32, 33, 34 and 35 pounds per square inch. Develop an appropriate regression model to relate tire pressure to fuel effectiveness. What appears to be the best level for tire pressure? 51
52 The mileage seems to be curvilinear to the pressure. 52
53 數 數 The R 2 is low. 度 數 Y = (Pressure) 53
54 According to the residual plot, there is a non-linear relation between the residual and the pressure. 54
55 數 數 度 數 Y = (Pressure)-1.15(pressure)^2 55
56 56
57 率 According to the residual plots, there is no severe departure from the model assumptions. 57
58 EXCEL : 料 數 Exercise : 9, 10, 11, 13, 14, 15 Excel: 17, 21, 23, 25 58
59 Bonus : (1%) Exercise 利 EXCEL 料 立 1. Linear relationship between X and Y? Global linearity Individual linearity 2. Multicollinearity? 3. Independent observations (X1,,Xk, Y)? 4. Normal distribution? 5. Equal variance? EXCEL output 59
Chapter 4. Regression Models. Learning Objectives
Chapter 4 Regression Models To accompany Quantitative Analysis for Management, Eleventh Edition, by Render, Stair, and Hanna Power Point slides created by Brian Peterson Learning Objectives After completing
More informationChapter 14 Student Lecture Notes 14-1
Chapter 14 Student Lecture Notes 14-1 Business Statistics: A Decision-Making Approach 6 th Edition Chapter 14 Multiple Regression Analysis and Model Building Chap 14-1 Chapter Goals After completing this
More informationRegression Models. Chapter 4. Introduction. Introduction. Introduction
Chapter 4 Regression Models Quantitative Analysis for Management, Tenth Edition, by Render, Stair, and Hanna 008 Prentice-Hall, Inc. Introduction Regression analysis is a very valuable tool for a manager
More informationChapter 7 Student Lecture Notes 7-1
Chapter 7 Student Lecture Notes 7- Chapter Goals QM353: Business Statistics Chapter 7 Multiple Regression Analysis and Model Building After completing this chapter, you should be able to: Explain model
More informationCorrelation Analysis
Simple Regression Correlation Analysis Correlation analysis is used to measure strength of the association (linear relationship) between two variables Correlation is only concerned with strength of the
More informationChapter 4: Regression Models
Sales volume of company 1 Textbook: pp. 129-164 Chapter 4: Regression Models Money spent on advertising 2 Learning Objectives After completing this chapter, students will be able to: Identify variables,
More informationInferences for Regression
Inferences for Regression An Example: Body Fat and Waist Size Looking at the relationship between % body fat and waist size (in inches). Here is a scatterplot of our data set: Remembering Regression In
More informationChapter 3 Multiple Regression Complete Example
Department of Quantitative Methods & Information Systems ECON 504 Chapter 3 Multiple Regression Complete Example Spring 2013 Dr. Mohammad Zainal Review Goals After completing this lecture, you should be
More informationStatistics for Managers using Microsoft Excel 6 th Edition
Statistics for Managers using Microsoft Excel 6 th Edition Chapter 13 Simple Linear Regression 13-1 Learning Objectives In this chapter, you learn: How to use regression analysis to predict the value of
More informationBasic Business Statistics 6 th Edition
Basic Business Statistics 6 th Edition Chapter 12 Simple Linear Regression Learning Objectives In this chapter, you learn: How to use regression analysis to predict the value of a dependent variable based
More informationChapter 16. Simple Linear Regression and dcorrelation
Chapter 16 Simple Linear Regression and dcorrelation 16.1 Regression Analysis Our problem objective is to analyze the relationship between interval variables; regression analysis is the first tool we will
More informationChapter 16. Simple Linear Regression and Correlation
Chapter 16 Simple Linear Regression and Correlation 16.1 Regression Analysis Our problem objective is to analyze the relationship between interval variables; regression analysis is the first tool we will
More informationBusiness Statistics. Chapter 14 Introduction to Linear Regression and Correlation Analysis QMIS 220. Dr. Mohammad Zainal
Department of Quantitative Methods & Information Systems Business Statistics Chapter 14 Introduction to Linear Regression and Correlation Analysis QMIS 220 Dr. Mohammad Zainal Chapter Goals After completing
More informationKeller: Stats for Mgmt & Econ, 7th Ed July 17, 2006
Chapter 17 Simple Linear Regression and Correlation 17.1 Regression Analysis Our problem objective is to analyze the relationship between interval variables; regression analysis is the first tool we will
More informationThe Multiple Regression Model
Multiple Regression The Multiple Regression Model Idea: Examine the linear relationship between 1 dependent (Y) & or more independent variables (X i ) Multiple Regression Model with k Independent Variables:
More informationMathematics for Economics MA course
Mathematics for Economics MA course Simple Linear Regression Dr. Seetha Bandara Simple Regression Simple linear regression is a statistical method that allows us to summarize and study relationships between
More informationChapter 14 Student Lecture Notes Department of Quantitative Methods & Information Systems. Business Statistics. Chapter 14 Multiple Regression
Chapter 14 Student Lecture Notes 14-1 Department of Quantitative Methods & Information Systems Business Statistics Chapter 14 Multiple Regression QMIS 0 Dr. Mohammad Zainal Chapter Goals After completing
More informationEstimating σ 2. We can do simple prediction of Y and estimation of the mean of Y at any value of X.
Estimating σ 2 We can do simple prediction of Y and estimation of the mean of Y at any value of X. To perform inferences about our regression line, we must estimate σ 2, the variance of the error term.
More informationCh 2: Simple Linear Regression
Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component
More informationChapter 13. Multiple Regression and Model Building
Chapter 13 Multiple Regression and Model Building Multiple Regression Models The General Multiple Regression Model y x x x 0 1 1 2 2... k k y is the dependent variable x, x,..., x 1 2 k the model are the
More informationPredictive Analytics : QM901.1x Prof U Dinesh Kumar, IIMB. All Rights Reserved, Indian Institute of Management Bangalore
What is Multiple Linear Regression Several independent variables may influence the change in response variable we are trying to study. When several independent variables are included in the equation, the
More informationBayesian Analysis LEARNING OBJECTIVES. Calculating Revised Probabilities. Calculating Revised Probabilities. Calculating Revised Probabilities
Valua%on and pricing (November 5, 2013) LEARNING OBJECTIVES Lecture 7 Decision making (part 3) Regression theory Olivier J. de Jong, LL.M., MM., MBA, CFD, CFFA, AA www.olivierdejong.com 1. List the steps
More informationUnit 10: Simple Linear Regression and Correlation
Unit 10: Simple Linear Regression and Correlation Statistics 571: Statistical Methods Ramón V. León 6/28/2004 Unit 10 - Stat 571 - Ramón V. León 1 Introductory Remarks Regression analysis is a method for
More informationRegression Analysis II
Regression Analysis II Measures of Goodness of fit Two measures of Goodness of fit Measure of the absolute fit of the sample points to the sample regression line Standard error of the estimate An index
More informationFinal Review. Yang Feng. Yang Feng (Columbia University) Final Review 1 / 58
Final Review Yang Feng http://www.stat.columbia.edu/~yangfeng Yang Feng (Columbia University) Final Review 1 / 58 Outline 1 Multiple Linear Regression (Estimation, Inference) 2 Special Topics for Multiple
More informationCHAPTER EIGHT Linear Regression
7 CHAPTER EIGHT Linear Regression 8. Scatter Diagram Example 8. A chemical engineer is investigating the effect of process operating temperature ( x ) on product yield ( y ). The study results in the following
More informationLI EAR REGRESSIO A D CORRELATIO
CHAPTER 6 LI EAR REGRESSIO A D CORRELATIO Page Contents 6.1 Introduction 10 6. Curve Fitting 10 6.3 Fitting a Simple Linear Regression Line 103 6.4 Linear Correlation Analysis 107 6.5 Spearman s Rank Correlation
More informationLecture 10 Multiple Linear Regression
Lecture 10 Multiple Linear Regression STAT 512 Spring 2011 Background Reading KNNL: 6.1-6.5 10-1 Topic Overview Multiple Linear Regression Model 10-2 Data for Multiple Regression Y i is the response variable
More informationInference for Regression Inference about the Regression Model and Using the Regression Line
Inference for Regression Inference about the Regression Model and Using the Regression Line PBS Chapter 10.1 and 10.2 2009 W.H. Freeman and Company Objectives (PBS Chapter 10.1 and 10.2) Inference about
More informationBasic Business Statistics, 10/e
Chapter 4 4- Basic Business Statistics th Edition Chapter 4 Introduction to Multiple Regression Basic Business Statistics, e 9 Prentice-Hall, Inc. Chap 4- Learning Objectives In this chapter, you learn:
More informationTrendlines Simple Linear Regression Multiple Linear Regression Systematic Model Building Practical Issues
Trendlines Simple Linear Regression Multiple Linear Regression Systematic Model Building Practical Issues Overfitting Categorical Variables Interaction Terms Non-linear Terms Linear Logarithmic y = a +
More informationConcordia University (5+5)Q 1.
(5+5)Q 1. Concordia University Department of Mathematics and Statistics Course Number Section Statistics 360/1 40 Examination Date Time Pages Mid Term Test May 26, 2004 Two Hours 3 Instructor Course Examiner
More informationSTAT Chapter 10: Analysis of Variance
STAT 515 -- Chapter 10: Analysis of Variance Designed Experiment A study in which the researcher controls the levels of one or more variables to determine their effect on the variable of interest (called
More informationChapter 14 Multiple Regression Analysis
Chapter 14 Multiple Regression Analysis 1. a. Multiple regression equation b. the Y-intercept c. $374,748 found by Y ˆ = 64,1 +.394(796,) + 9.6(694) 11,6(6.) (LO 1) 2. a. Multiple regression equation b.
More informationThe simple linear regression model discussed in Chapter 13 was written as
1519T_c14 03/27/2006 07:28 AM Page 614 Chapter Jose Luis Pelaez Inc/Blend Images/Getty Images, Inc./Getty Images, Inc. 14 Multiple Regression 14.1 Multiple Regression Analysis 14.2 Assumptions of the Multiple
More informationRegression Analysis. BUS 735: Business Decision Making and Research. Learn how to detect relationships between ordinal and categorical variables.
Regression Analysis BUS 735: Business Decision Making and Research 1 Goals of this section Specific goals Learn how to detect relationships between ordinal and categorical variables. Learn how to estimate
More informationRegression Models. Chapter 4
Chapter 4 Regression Models To accompany Quantitative Analysis for Management, Eleventh Edition, by Render, Stair, and Hanna Power Point slides created by Brian Peterson Introduction Regression analysis
More informationBasic Statistics Exercises 66
Basic Statistics Exercises 66 42. Suppose we are interested in predicting a person's height from the person's length of stride (distance between footprints). The following data is recorded for a random
More informationFormal Statement of Simple Linear Regression Model
Formal Statement of Simple Linear Regression Model Y i = β 0 + β 1 X i + ɛ i Y i value of the response variable in the i th trial β 0 and β 1 are parameters X i is a known constant, the value of the predictor
More informationSIMPLE REGRESSION ANALYSIS. Business Statistics
SIMPLE REGRESSION ANALYSIS Business Statistics CONTENTS Ordinary least squares (recap for some) Statistical formulation of the regression model Assessing the regression model Testing the regression coefficients
More informationChapter Learning Objectives. Regression Analysis. Correlation. Simple Linear Regression. Chapter 12. Simple Linear Regression
Chapter 12 12-1 North Seattle Community College BUS21 Business Statistics Chapter 12 Learning Objectives In this chapter, you learn:! How to use regression analysis to predict the value of a dependent
More informationLinear regression. We have that the estimated mean in linear regression is. ˆµ Y X=x = ˆβ 0 + ˆβ 1 x. The standard error of ˆµ Y X=x is.
Linear regression We have that the estimated mean in linear regression is The standard error of ˆµ Y X=x is where x = 1 n s.e.(ˆµ Y X=x ) = σ ˆµ Y X=x = ˆβ 0 + ˆβ 1 x. 1 n + (x x)2 i (x i x) 2 i x i. The
More informationOutline. Remedial Measures) Extra Sums of Squares Standardized Version of the Multiple Regression Model
Outline 1 Multiple Linear Regression (Estimation, Inference, Diagnostics and Remedial Measures) 2 Special Topics for Multiple Regression Extra Sums of Squares Standardized Version of the Multiple Regression
More informationSTA121: Applied Regression Analysis
STA121: Applied Regression Analysis Linear Regression Analysis - Chapters 3 and 4 in Dielman Artin Department of Statistical Science September 15, 2009 Outline 1 Simple Linear Regression Analysis 2 Using
More informationInference for Regression Simple Linear Regression
Inference for Regression Simple Linear Regression IPS Chapter 10.1 2009 W.H. Freeman and Company Objectives (IPS Chapter 10.1) Simple linear regression p Statistical model for linear regression p Estimating
More informationRegression Analysis. BUS 735: Business Decision Making and Research
Regression Analysis BUS 735: Business Decision Making and Research 1 Goals and Agenda Goals of this section Specific goals Learn how to detect relationships between ordinal and categorical variables. Learn
More informationTable of z values and probabilities for the standard normal distribution. z is the first column plus the top row. Each cell shows P(X z).
Table of z values and probabilities for the standard normal distribution. z is the first column plus the top row. Each cell shows P(X z). For example P(X.04) =.8508. For z < 0 subtract the value from,
More informationUnit 11: Multiple Linear Regression
Unit 11: Multiple Linear Regression Statistics 571: Statistical Methods Ramón V. León 7/13/2004 Unit 11 - Stat 571 - Ramón V. León 1 Main Application of Multiple Regression Isolating the effect of a variable
More informationSimple Linear Regression
9-1 l Chapter 9 l Simple Linear Regression 9.1 Simple Linear Regression 9.2 Scatter Diagram 9.3 Graphical Method for Determining Regression 9.4 Least Square Method 9.5 Correlation Coefficient and Coefficient
More informationSMAM 314 Exam 3d Name
SMAM 314 Exam 3d Name 1. Mark the following statements True T or False F. (6 points -2 each) T A. A process is out of control if at a particular point in time the reading is more than 3 standard deviations
More informationCh 3: Multiple Linear Regression
Ch 3: Multiple Linear Regression 1. Multiple Linear Regression Model Multiple regression model has more than one regressor. For example, we have one response variable and two regressor variables: 1. delivery
More informationEcon 3790: Business and Economics Statistics. Instructor: Yogesh Uppal
Econ 3790: Business and Economics Statistics Instructor: Yogesh Uppal yuppal@ysu.edu Sampling Distribution of b 1 Expected value of b 1 : Variance of b 1 : E(b 1 ) = 1 Var(b 1 ) = σ 2 /SS x Estimate of
More informationFinQuiz Notes
Reading 10 Multiple Regression and Issues in Regression Analysis 2. MULTIPLE LINEAR REGRESSION Multiple linear regression is a method used to model the linear relationship between a dependent variable
More informationBiostatistics 380 Multiple Regression 1. Multiple Regression
Biostatistics 0 Multiple Regression ORIGIN 0 Multiple Regression Multiple Regression is an extension of the technique of linear regression to describe the relationship between a single dependent (response)
More informationSimple Linear Regression: One Qualitative IV
Simple Linear Regression: One Qualitative IV 1. Purpose As noted before regression is used both to explain and predict variation in DVs, and adding to the equation categorical variables extends regression
More informationInference for the Regression Coefficient
Inference for the Regression Coefficient Recall, b 0 and b 1 are the estimates of the slope β 1 and intercept β 0 of population regression line. We can shows that b 0 and b 1 are the unbiased estimates
More information5. Multiple Regression (Regressioanalyysi) (Azcel Ch. 11, Milton/Arnold Ch. 12) The k-variable Multiple Regression Model
5. Multiple Regression (Regressioanalyysi) (Azcel Ch. 11, Milton/Arnold Ch. 12) The k-variable Multiple Regression Model The population regression model of a dependent variable Y on a set of k independent
More informationInference for Regression
Inference for Regression Section 9.4 Cathy Poliak, Ph.D. cathy@math.uh.edu Office in Fleming 11c Department of Mathematics University of Houston Lecture 13b - 3339 Cathy Poliak, Ph.D. cathy@math.uh.edu
More informationholding all other predictors constant
Multiple Regression Numeric Response variable (y) p Numeric predictor variables (p < n) Model: Y = b 0 + b 1 x 1 + + b p x p + e Partial Regression Coefficients: b i effect (on the mean response) of increasing
More informationMultiple Regression. Peerapat Wongchaiwat, Ph.D.
Peerapat Wongchaiwat, Ph.D. wongchaiwat@hotmail.com The Multiple Regression Model Examine the linear relationship between 1 dependent (Y) & 2 or more independent variables (X i ) Multiple Regression Model
More informationWhat is a Hypothesis?
What is a Hypothesis? A hypothesis is a claim (assumption) about a population parameter: population mean Example: The mean monthly cell phone bill in this city is μ = $42 population proportion Example:
More informationA discussion on multiple regression models
A discussion on multiple regression models In our previous discussion of simple linear regression, we focused on a model in which one independent or explanatory variable X was used to predict the value
More information3. Diagnostics and Remedial Measures
3. Diagnostics and Remedial Measures So far, we took data (X i, Y i ) and we assumed where ɛ i iid N(0, σ 2 ), Y i = β 0 + β 1 X i + ɛ i i = 1, 2,..., n, β 0, β 1 and σ 2 are unknown parameters, X i s
More informationDiagnostics and Remedial Measures
Diagnostics and Remedial Measures Yang Feng http://www.stat.columbia.edu/~yangfeng Yang Feng (Columbia University) Diagnostics and Remedial Measures 1 / 72 Remedial Measures How do we know that the regression
More informationSimple Linear Regression: One Quantitative IV
Simple Linear Regression: One Quantitative IV Linear regression is frequently used to explain variation observed in a dependent variable (DV) with theoretically linked independent variables (IV). For example,
More informationCorrelation & Simple Regression
Chapter 11 Correlation & Simple Regression The previous chapter dealt with inference for two categorical variables. In this chapter, we would like to examine the relationship between two quantitative variables.
More informationSociology 6Z03 Review II
Sociology 6Z03 Review II John Fox McMaster University Fall 2016 John Fox (McMaster University) Sociology 6Z03 Review II Fall 2016 1 / 35 Outline: Review II Probability Part I Sampling Distributions Probability
More informationStatistics and Quantitative Analysis U4320
Statistics and Quantitative Analysis U3 Lecture 13: Explaining Variation Prof. Sharyn O Halloran Explaining Variation: Adjusted R (cont) Definition of Adjusted R So we'd like a measure like R, but one
More informationSimple Linear Regression
Chapter 2 Simple Linear Regression Linear Regression with One Independent Variable 2.1 Introduction In Chapter 1 we introduced the linear model as an alternative for making inferences on means of one or
More informationChapte The McGraw-Hill Companies, Inc. All rights reserved.
12er12 Chapte Bivariate i Regression (Part 1) Bivariate Regression Visual Displays Begin the analysis of bivariate data (i.e., two variables) with a scatter plot. A scatter plot - displays each observed
More informationMultiple Regression Methods
Chapter 1: Multiple Regression Methods Hildebrand, Ott and Gray Basic Statistical Ideas for Managers Second Edition 1 Learning Objectives for Ch. 1 The Multiple Linear Regression Model How to interpret
More informationSummary of Chapter 7 (Sections ) and Chapter 8 (Section 8.1)
Summary of Chapter 7 (Sections 7.2-7.5) and Chapter 8 (Section 8.1) Chapter 7. Tests of Statistical Hypotheses 7.2. Tests about One Mean (1) Test about One Mean Case 1: σ is known. Assume that X N(µ, σ
More informationAMS 315/576 Lecture Notes. Chapter 11. Simple Linear Regression
AMS 315/576 Lecture Notes Chapter 11. Simple Linear Regression 11.1 Motivation A restaurant opening on a reservations-only basis would like to use the number of advance reservations x to predict the number
More informationWe like to capture and represent the relationship between a set of possible causes and their response, by using a statistical predictive model.
Statistical Methods in Business Lecture 5. Linear Regression We like to capture and represent the relationship between a set of possible causes and their response, by using a statistical predictive model.
More informationEconometrics. 4) Statistical inference
30C00200 Econometrics 4) Statistical inference Timo Kuosmanen Professor, Ph.D. http://nomepre.net/index.php/timokuosmanen Today s topics Confidence intervals of parameter estimates Student s t-distribution
More informationMULTIPLE REGRESSION AND ISSUES IN REGRESSION ANALYSIS
MULTIPLE REGRESSION AND ISSUES IN REGRESSION ANALYSIS Page 1 MSR = Mean Regression Sum of Squares MSE = Mean Squared Error RSS = Regression Sum of Squares SSE = Sum of Squared Errors/Residuals α = Level
More informationLecture 9: Linear Regression
Lecture 9: Linear Regression Goals Develop basic concepts of linear regression from a probabilistic framework Estimating parameters and hypothesis testing with linear models Linear regression in R Regression
More informationChapter 14 Simple Linear Regression (A)
Chapter 14 Simple Linear Regression (A) 1. Characteristics Managerial decisions often are based on the relationship between two or more variables. can be used to develop an equation showing how the variables
More informationEcon 3790: Statistics Business and Economics. Instructor: Yogesh Uppal
Econ 3790: Statistics Business and Economics Instructor: Yogesh Uppal Email: yuppal@ysu.edu Chapter 14 Covariance and Simple Correlation Coefficient Simple Linear Regression Covariance Covariance between
More informationApplied Regression Analysis
Applied Regression Analysis Chapter 3 Multiple Linear Regression Hongcheng Li April, 6, 2013 Recall simple linear regression 1 Recall simple linear regression 2 Parameter Estimation 3 Interpretations of
More informationRegression Analysis IV... More MLR and Model Building
Regression Analysis IV... More MLR and Model Building This session finishes up presenting the formal methods of inference based on the MLR model and then begins discussion of "model building" (use of regression
More informationLinear models and their mathematical foundations: Simple linear regression
Linear models and their mathematical foundations: Simple linear regression Steffen Unkel Department of Medical Statistics University Medical Center Göttingen, Germany Winter term 2018/19 1/21 Introduction
More informationSTAT Chapter 11: Regression
STAT 515 -- Chapter 11: Regression Mostly we have studied the behavior of a single random variable. Often, however, we gather data on two random variables. We wish to determine: Is there a relationship
More informationRegression analysis is a tool for building mathematical and statistical models that characterize relationships between variables Finds a linear
Regression analysis is a tool for building mathematical and statistical models that characterize relationships between variables Finds a linear relationship between: - one independent variable X and -
More informationThe Standard Linear Model: Hypothesis Testing
Department of Mathematics Ma 3/103 KC Border Introduction to Probability and Statistics Winter 2017 Lecture 25: The Standard Linear Model: Hypothesis Testing Relevant textbook passages: Larsen Marx [4]:
More informationLecture 15 Multiple regression I Chapter 6 Set 2 Least Square Estimation The quadratic form to be minimized is
Lecture 15 Multiple regression I Chapter 6 Set 2 Least Square Estimation The quadratic form to be minimized is Q = (Y i β 0 β 1 X i1 β 2 X i2 β p 1 X i.p 1 ) 2, which in matrix notation is Q = (Y Xβ) (Y
More informationMULTIPLE REGRESSION ANALYSIS AND OTHER ISSUES. Business Statistics
MULTIPLE REGRESSION ANALYSIS AND OTHER ISSUES Business Statistics CONTENTS Multiple regression Dummy regressors Assumptions of regression analysis Predicting with regression analysis Old exam question
More informationINFERENCE FOR REGRESSION
CHAPTER 3 INFERENCE FOR REGRESSION OVERVIEW In Chapter 5 of the textbook, we first encountered regression. The assumptions that describe the regression model we use in this chapter are the following. We
More informationK. Model Diagnostics. residuals ˆɛ ij = Y ij ˆµ i N = Y ij Ȳ i semi-studentized residuals ω ij = ˆɛ ij. studentized deleted residuals ɛ ij =
K. Model Diagnostics We ve already seen how to check model assumptions prior to fitting a one-way ANOVA. Diagnostics carried out after model fitting by using residuals are more informative for assessing
More informationF-tests and Nested Models
F-tests and Nested Models Nested Models: A core concept in statistics is comparing nested s. Consider the Y = β 0 + β 1 x 1 + β 2 x 2 + ǫ. (1) The following reduced s are special cases (nested within)
More informationDEMAND ESTIMATION (PART III)
BEC 30325: MANAGERIAL ECONOMICS Session 04 DEMAND ESTIMATION (PART III) Dr. Sumudu Perera Session Outline 2 Multiple Regression Model Test the Goodness of Fit Coefficient of Determination F Statistic t
More informationInteractions. Interactions. Lectures 1 & 2. Linear Relationships. y = a + bx. Slope. Intercept
Interactions Lectures 1 & Regression Sometimes two variables appear related: > smoking and lung cancers > height and weight > years of education and income > engine size and gas mileage > GMAT scores and
More informationSimple Linear Regression
Simple Linear Regression ST 430/514 Recall: A regression model describes how a dependent variable (or response) Y is affected, on average, by one or more independent variables (or factors, or covariates)
More informationSimple Linear Regression. Material from Devore s book (Ed 8), and Cengagebrain.com
12 Simple Linear Regression Material from Devore s book (Ed 8), and Cengagebrain.com The Simple Linear Regression Model The simplest deterministic mathematical relationship between two variables x and
More informationRegression Models for Quantitative and Qualitative Predictors: An Overview
Regression Models for Quantitative and Qualitative Predictors: An Overview Polynomial regression models Interaction regression models Qualitative predictors Indicator variables Modeling interactions between
More informationUnivariate analysis. Simple and Multiple Regression. Univariate analysis. Simple Regression How best to summarise the data?
Univariate analysis Example - linear regression equation: y = ax + c Least squares criteria ( yobs ycalc ) = yobs ( ax + c) = minimum Simple and + = xa xc xy xa + nc = y Solve for a and c Univariate analysis
More informationCh 13 & 14 - Regression Analysis
Ch 3 & 4 - Regression Analysis Simple Regression Model I. Multiple Choice:. A simple regression is a regression model that contains a. only one independent variable b. only one dependent variable c. more
More informationLECTURE 6. Introduction to Econometrics. Hypothesis testing & Goodness of fit
LECTURE 6 Introduction to Econometrics Hypothesis testing & Goodness of fit October 25, 2016 1 / 23 ON TODAY S LECTURE We will explain how multiple hypotheses are tested in a regression model We will define
More informationChapter 12 - Lecture 2 Inferences about regression coefficient
Chapter 12 - Lecture 2 Inferences about regression coefficient April 19th, 2010 Facts about slope Test Statistic Confidence interval Hypothesis testing Test using ANOVA Table Facts about slope In previous
More informationSection 3: Simple Linear Regression
Section 3: Simple Linear Regression Carlos M. Carvalho The University of Texas at Austin McCombs School of Business http://faculty.mccombs.utexas.edu/carlos.carvalho/teaching/ 1 Regression: General Introduction
More information