Applied Regression Analysis
|
|
- Kory Short
- 6 years ago
- Views:
Transcription
1 Applied Regression Analysis Chapter 3 Multiple Linear Regression Hongcheng Li April, 6, 2013
2 Recall simple linear regression 1 Recall simple linear regression 2 Parameter Estimation 3 Interpretations of Regression Coefficients 4 Properties of the Least Squares Estimators 5 Multiple correlation coefficient
3 Recall simple linear regression Multiple Linear Regression I In the last lesson, we have learned that ŵage = eper How about other variables besides experience that are related to the wages? How about the level of education?
4 Recall simple linear regression Multiple Linear Regression II ŵage = β 0 + β 1 educ + β 2 eper + ɛ where exper is years of labor market experience and wage is the level of education. Multiple regression analysis is also useful for generalizing functional relationships between variables. For example, considering the relationship between consumption(cons) and family income(inc): cons = β 0 + β 1 inc + β 2 inc 2 + ɛ
5 Recall simple linear regression Multiple Linear Regression III After taking x 1 = inc and x 2 = inc 2, it is still a multiple linear regression problem. Y = β 0 + β 1 X 1 + β 2 X β p X p + ε (3.1)
6 Recall simple linear regression Multiple Linear Regression IV According to above equation, each observation can be written as y i = β 0 + β 1 x i1 + β 2 x i2 + + β p x ip + ε i
7 Recall simple linear regression Multiple Linear Regression V The key assumption of multiple linear regression is : E(ε X 1,, X p ) = 0
8 Parameter Estimation 1 Recall simple linear regression 2 Parameter Estimation 3 Interpretations of Regression Coefficients 4 Properties of the Least Squares Estimators 5 Multiple correlation coefficient
9 Parameter Estimation Parameter Estimation I The errors can be written as ε i = y i (β 0 + β 1 x i1 + β 2 x i2 + + β p x ip ) The sum of squares of these errors is S(β 0, β 1,, β p ) = n ε 2 i = i=1 n (y i (β 0 +β 1 x i1 +β 2 x i2 + +β p x ip )) 2 i=1
10 Parameter Estimation Parameter Estimation I In the general case with k independent variables, we seek estimates in the equation of ˆβ 0, ˆβ 1,, ˆβ p ŷ = ˆβ 0 + ˆβ 1 x 1 + ˆβ 2 x ˆβ p x p + ε (3.1)
11 Parameter Estimation Parameter Estimation II The OLS estimates, of the p + 1 parameters,are chosen to minimize the sum of squared residuals: S(β 0, β 1,, β p ) 1 By a direct application of calculus, it can be show that the least squares estimates ˆβ 0, ˆβ 1,, ˆβ p
12 Parameter Estimation Parameter Estimation III which minimize S(β 0, β 1,, β p ), are given by the solution of the following system of equations: s 11 ˆβ 1 + s 12 ˆβ s 1p ˆβ p = s y1 s 12 ˆβ 1 + s 22 ˆβ s 2p ˆβ p = s y2. s 1p ˆβ 1 + s 2p ˆβ s pp ˆβ p = s yp
13 Parameter Estimation Parameter Estimation I where n s ij = (x αi x i )(x αj x j ) s yj = α=1 n (y α ȳ)(x αj x j ) α=1 n α=1 x j = x αj n n α=1 ȳ = y α n
14 Parameter Estimation Parameter Estimation II and β 0 = ȳ ˆβ 1 x 1 ˆβ 2 x 2 ˆβ p x p.
15 Parameter Estimation The equations in the above system are called the normal equations. β 0 is usually referred to as the intercept or constant. β j, j = 1, 2,, p, is usually referred to as the regression coefficients or partial coefficients.
16 Interpretations of Regression Coefficients 1 Recall simple linear regression 2 Parameter Estimation 3 Interpretations of Regression Coefficients 4 Properties of the Least Squares Estimators 5 Multiple correlation coefficient
17 Interpretations of Regression Coefficients Interpretations of Regression Coefficients I 1 β 0 is the value of Y when X 1 = X 2 = = X p = 0, as in the simple regression.
18 Interpretations of Regression Coefficients Interpretations of Regression Coefficients II 2 β j, j = 1, 2,, p: has several interpretations: the change in Y corresponding to a unit change in X j when all other predictor variables are held constant. Magnitude of the change is not depend on the values at which the other predictor variables are fixed. partial regression coefficient-represents the contribution of X j to the response variable Y after it has been adjusted for the other predictor variables.
19 Interpretations of Regression Coefficients Interpretations of Regression Coefficients III 3 Ref P57 Explain: Partial regression coefficients
20 Interpretations of Regression Coefficients Check the data I Y X X2
21 Interpretations of Regression Coefficients Check the data II X1 Y
22 Interpretations of Regression Coefficients Check the data III X2 Y
23 Interpretations of Regression Coefficients Explain: Partial regression coefficients I Supervisor data > pairs( Y~ X1 + X2, pch = 16, col ="blue", data = ch3) > lm1 <- lm(y ~ X1+X2, data = ch3) > summary(lm1)
24 Interpretations of Regression Coefficients Explain: Partial regression coefficients II Call: lm(formula = Y ~ X1 + X2, data = ch3) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) *
25 Interpretations of Regression Coefficients Explain: Partial regression coefficients III X e-07 *** X Signif. codes: 0 *** ** 0.01 * Residual standard error: on 27 degrees of freedom Multiple R-squared: ,
26 Interpretations of Regression Coefficients Explain: Partial regression coefficients IV Adjusted R-squared: F-statistic: 29.1 on 2 and 27 DF, p-value: 1.833e-07
27 Properties of the Least Squares Estimators 1 Recall simple linear regression 2 Parameter Estimation 3 Interpretations of Regression Coefficients 4 Properties of the Least Squares Estimators 5 Multiple correlation coefficient
28 Properties of the Least Squares Estimators Properties of the Least Squares Estimators I 1 The estimator ˆβ j, j = 0, 1,, p, is an unbiased estimate of β j and has a variance of σ 2 c jj, where c jj is the j th diagonal element of (X T X ) 1. The least square estimators are BLUE(best linear unbiased estimator has the smallest variance among all unbiased estimators).
29 Properties of the Least Squares Estimators Properties of the Least Squares Estimators II 2 The estimator ˆβ j, j = 0, 1,, p, is normally distributed with mean β j and variance σ 2 c jj.
30 Properties of the Least Squares Estimators Properties of the Least Squares Estimators III 3 W = SSE/σ 2 has a χ 2 distribution with n p 1 degree of freedom, and ˆβ j s and ˆσ 2 are distributed independently of each other.
31 Properties of the Least Squares Estimators Properties of the Least Squares Estimators IV 4 The vector ˆβ = ( ˆβ 0, ˆβ 1,, ˆβ p ) has a (p + 1)-variate normal distribution with mean vector β = (β 0, β 1,, β p ) and variance covariance matrix with elements σ 2 c ij.
32 Multiple correlation coefficient 1 Recall simple linear regression 2 Parameter Estimation 3 Interpretations of Regression Coefficients 4 Properties of the Least Squares Estimators 5 Multiple correlation coefficient
33 Multiple correlation coefficient Multiple correlation coefficient I 1 The strength of the linear relationship between Y and the set of predictors X 1, X 2,, X p can be assessed through the examination of the scatter plot of Y versus Ŷ and
34 Multiple correlation coefficient Multiple correlation coefficient II 2 the correlation coefficient between Y and Ŷ (yi ȳ)(ŷ i ŷ) Cor(Y, Ŷ ) = (yi ȳ) 2 (ŷ i ŷ) 2
35 Multiple correlation coefficient Multiple correlation coefficient III 3 Goodness-of-Fit: The coefficient of determination SST: Total Sum of Squares SSE: Explained Sum of Squares SSR: Residual Sum of Squares (or Sum of Squared Residuals)
36 Multiple correlation coefficient Multiple correlation coefficient IV SST SSE SSR n (y i ȳ) 2 i=1 n (ŷ i ȳ) 2 i=1 n (y i ŷ i ) 2 i=1 SST = SSR + SSE R 2 = SSR SST = 1 SSE SST = 1 (yi ŷ i ) 2 (yi ȳ) 2
37 Multiple correlation coefficient Inference for individual regression coefficients I 1 H 0 : β j = β 0 j P61 2 Test statistic t j = ˆβ j β 0 j s.e.( ˆβ j )
38 Multiple correlation coefficient Inference for individual regression coefficients II 3 C.I. for β j The confidence limits for β j with confidence coefficient α are given by ˆβ j ± ˆσ c jj t (n p 1,α/2)
39 Multiple correlation coefficient Supervisor Performance I The fitted regression equation is Ŷ = x X X X X X 6 1 How to interpret the output Variable Coefficient s.e. t-test p-value Constant X 1 X 2 X 3 X 4 X 5 X 6 n = 30 R 2 = 0.73 Ra 2 = 0.60 ˆσ = d.f. =23
40 Multiple correlation coefficient Supervisor Performance II
41 Multiple correlation coefficient Test of Hypothesis in a linear model I 1 All the regression coefficients associated with the predictor variables are zero. 2 Some of the regression coefficients are zero. 3 Some of the regression coefficients are equal to each other. 4 the regression parameters satisfy certain specified constraints.
42 Multiple correlation coefficient Model Compare I The full model: Y = β 0 + β 1 X 1 + β 2 X β p X p + ε (Full Model-FM) If we set some of the regression coefficients to be 0, then we get a reduced model-rm Like, for a given k, β k = 0, then we get a reduced model. The number of distinct parameters to be estimated in the reduced model is smaller than the number of parameters to be estimated in the full model.
43 Multiple correlation coefficient Model Compare II Accordingly, we wish to test: H 0 : Reduced model is adequate against H 1 : Full model is adequate 1 What s nested model. A set of models are said to be nested if they can be obtained from a larger model as special cases. 2 P64 The sum of squares due to error associated with the FM (p + 1 parameters), SSE(FM) = (y i ŷ i ) 2.
44 Multiple correlation coefficient Model Compare III 3 P64 The sum of squares due to error associated with the RM(k distinct parameters), SSE(RM) = (y i ŷ i ) 2.
45 Multiple correlation coefficient Model Compare IV Here for sure SSE(RM) SSE(FM), the point is how large is the difference between the residual sum of squares. If the difference is large, the reduced model is inadequate. F = [SSE(RM) SSE(FM)]/(p + 1 k) SSE(FM)/(n p 1) H 0 is rejected if F F (p+1 k,n p 1;α). or, equivalently, if p(f ) α
46 Multiple correlation coefficient Testing all regression coefficients equal to zero I RM: H 0 : Y = β 0 + ε FM: H 1 : Y = β 0 + β 1 X β p X p + ε The F-test reduced to F = [SST SSE]/p SSE/(n p 1) = SSR/p SSE/(n p 1) = MSR MSE
47 Multiple correlation coefficient Testing a subset of regression coefficients equal to zero I An important goal in regression analysis is to arrive at adequate descriptions of observed phenomenon in terms of as few meaningful variables as possible. Simplicity of description or the principle of parsimony is one of the important guiding principles in regression analysis.
48 Multiple correlation coefficient Testing a subset of regression coefficients equal to zero II 1 RM: Y = β 0 + β 1 X 1 + β 3 X 3 + ε which corresponds to hypothesis 2 In simple regression, p = 1. H 0 : β 2 = β 4 = β 5 = β 6 = 0 t 1 = ˆβ 1 s.e.( ˆβ 1 ) Therefore,
49 Multiple correlation coefficient Testing a subset of regression coefficients equal to zero III F = t 2 1
50 Multiple correlation coefficient Testing the Equality of Regression coefficients I 1 H 0 : β 1 = β 3 (= β 1) (β 2 = β 4 = β 5 = β 6 = 0) Under H 0 2 Y = β 0 + β 1(X 1 + X 3 ) + ε
51 Multiple correlation coefficient Estimating and Testing of regression parameters under constrains I 1 H 0 : β 1 + β 3 = 1 (β 2 = β 4 = β 5 = β 6 = 0) Under H 0 2 Y = β 0 + β 1 X 1 + (1 β 1 )X 3 ) + ε
52 Multiple correlation coefficient Predictions 1 suppose x 0 = (x 01, x 02,, x 0p ), the predicted value, ŷ 0, corresponding to x 0 is given by ŷ 0 = ˆβ 0 + ˆβ 1 x 01 + ˆβ 2 x ˆβ p x 0p 2 The C.I with confidence coefficient α, ŷ 0 ± t (n p 1,α/2) s.e.(ŷ 0 ).
53 Multiple correlation coefficient Homework 1 P
Applied Regression. Applied Regression. Chapter 2 Simple Linear Regression. Hongcheng Li. April, 6, 2013
Applied Regression Chapter 2 Simple Linear Regression Hongcheng Li April, 6, 2013 Outline 1 Introduction of simple linear regression 2 Scatter plot 3 Simple linear regression model 4 Test of Hypothesis
More informationCh 3: Multiple Linear Regression
Ch 3: Multiple Linear Regression 1. Multiple Linear Regression Model Multiple regression model has more than one regressor. For example, we have one response variable and two regressor variables: 1. delivery
More informationCh 2: Simple Linear Regression
Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component
More informationInference for Regression
Inference for Regression Section 9.4 Cathy Poliak, Ph.D. cathy@math.uh.edu Office in Fleming 11c Department of Mathematics University of Houston Lecture 13b - 3339 Cathy Poliak, Ph.D. cathy@math.uh.edu
More informationLecture 6 Multiple Linear Regression, cont.
Lecture 6 Multiple Linear Regression, cont. BIOST 515 January 22, 2004 BIOST 515, Lecture 6 Testing general linear hypotheses Suppose we are interested in testing linear combinations of the regression
More informationMultiple Linear Regression
Multiple Linear Regression Simple linear regression tries to fit a simple line between two variables Y and X. If X is linearly related to Y this explains some of the variability in Y. In most cases, there
More informationLinear Regression Model. Badr Missaoui
Linear Regression Model Badr Missaoui Introduction What is this course about? It is a course on applied statistics. It comprises 2 hours lectures each week and 1 hour lab sessions/tutorials. We will focus
More informationSimple Linear Regression
Simple Linear Regression In simple linear regression we are concerned about the relationship between two variables, X and Y. There are two components to such a relationship. 1. The strength of the relationship.
More informationMATH 644: Regression Analysis Methods
MATH 644: Regression Analysis Methods FINAL EXAM Fall, 2012 INSTRUCTIONS TO STUDENTS: 1. This test contains SIX questions. It comprises ELEVEN printed pages. 2. Answer ALL questions for a total of 100
More informationFigure 1: The fitted line using the shipment route-number of ampules data. STAT5044: Regression and ANOVA The Solution of Homework #2 Inyoung Kim
0.0 1.0 1.5 2.0 2.5 3.0 8 10 12 14 16 18 20 22 y x Figure 1: The fitted line using the shipment route-number of ampules data STAT5044: Regression and ANOVA The Solution of Homework #2 Inyoung Kim Problem#
More informationLinear models and their mathematical foundations: Simple linear regression
Linear models and their mathematical foundations: Simple linear regression Steffen Unkel Department of Medical Statistics University Medical Center Göttingen, Germany Winter term 2018/19 1/21 Introduction
More informationLecture 15 Multiple regression I Chapter 6 Set 2 Least Square Estimation The quadratic form to be minimized is
Lecture 15 Multiple regression I Chapter 6 Set 2 Least Square Estimation The quadratic form to be minimized is Q = (Y i β 0 β 1 X i1 β 2 X i2 β p 1 X i.p 1 ) 2, which in matrix notation is Q = (Y Xβ) (Y
More informationVariance Decomposition and Goodness of Fit
Variance Decomposition and Goodness of Fit 1. Example: Monthly Earnings and Years of Education In this tutorial, we will focus on an example that explores the relationship between total monthly earnings
More informationSimple and Multiple Linear Regression
Sta. 113 Chapter 12 and 13 of Devore March 12, 2010 Table of contents 1 Simple Linear Regression 2 Model Simple Linear Regression A simple linear regression model is given by Y = β 0 + β 1 x + ɛ where
More informationEstimating σ 2. We can do simple prediction of Y and estimation of the mean of Y at any value of X.
Estimating σ 2 We can do simple prediction of Y and estimation of the mean of Y at any value of X. To perform inferences about our regression line, we must estimate σ 2, the variance of the error term.
More informationCoefficient of Determination
Coefficient of Determination ST 430/514 The coefficient of determination, R 2, is defined as before: R 2 = 1 SS E (yi ŷ i ) = 1 2 SS yy (yi ȳ) 2 The interpretation of R 2 is still the fraction of variance
More informationSimple Linear Regression
Simple Linear Regression ST 430/514 Recall: A regression model describes how a dependent variable (or response) Y is affected, on average, by one or more independent variables (or factors, or covariates)
More information14 Multiple Linear Regression
B.Sc./Cert./M.Sc. Qualif. - Statistics: Theory and Practice 14 Multiple Linear Regression 14.1 The multiple linear regression model In simple linear regression, the response variable y is expressed in
More informationBiostatistics 380 Multiple Regression 1. Multiple Regression
Biostatistics 0 Multiple Regression ORIGIN 0 Multiple Regression Multiple Regression is an extension of the technique of linear regression to describe the relationship between a single dependent (response)
More informationVariance Decomposition in Regression James M. Murray, Ph.D. University of Wisconsin - La Crosse Updated: October 04, 2017
Variance Decomposition in Regression James M. Murray, Ph.D. University of Wisconsin - La Crosse Updated: October 04, 2017 PDF file location: http://www.murraylax.org/rtutorials/regression_anovatable.pdf
More informationMultiple Linear Regression
Multiple Linear Regression ST 430/514 Recall: a regression model describes how a dependent variable (or response) Y is affected, on average, by one or more independent variables (or factors, or covariates).
More informationEconometrics I KS. Module 2: Multivariate Linear Regression. Alexander Ahammer. This version: April 16, 2018
Econometrics I KS Module 2: Multivariate Linear Regression Alexander Ahammer Department of Economics Johannes Kepler University of Linz This version: April 16, 2018 Alexander Ahammer (JKU) Module 2: Multivariate
More informationMeasuring the fit of the model - SSR
Measuring the fit of the model - SSR Once we ve determined our estimated regression line, we d like to know how well the model fits. How far/close are the observations to the fitted line? One way to do
More informationLecture 4 Multiple linear regression
Lecture 4 Multiple linear regression BIOST 515 January 15, 2004 Outline 1 Motivation for the multiple regression model Multiple regression in matrix notation Least squares estimation of model parameters
More informationLECTURE 6. Introduction to Econometrics. Hypothesis testing & Goodness of fit
LECTURE 6 Introduction to Econometrics Hypothesis testing & Goodness of fit October 25, 2016 1 / 23 ON TODAY S LECTURE We will explain how multiple hypotheses are tested in a regression model We will define
More informationST430 Exam 1 with Answers
ST430 Exam 1 with Answers Date: October 5, 2015 Name: Guideline: You may use one-page (front and back of a standard A4 paper) of notes. No laptop or textook are permitted but you may use a calculator.
More informationLinear regression. We have that the estimated mean in linear regression is. ˆµ Y X=x = ˆβ 0 + ˆβ 1 x. The standard error of ˆµ Y X=x is.
Linear regression We have that the estimated mean in linear regression is The standard error of ˆµ Y X=x is where x = 1 n s.e.(ˆµ Y X=x ) = σ ˆµ Y X=x = ˆβ 0 + ˆβ 1 x. 1 n + (x x)2 i (x i x) 2 i x i. The
More informationFormal Statement of Simple Linear Regression Model
Formal Statement of Simple Linear Regression Model Y i = β 0 + β 1 X i + ɛ i Y i value of the response variable in the i th trial β 0 and β 1 are parameters X i is a known constant, the value of the predictor
More informationRegression Review. Statistics 149. Spring Copyright c 2006 by Mark E. Irwin
Regression Review Statistics 149 Spring 2006 Copyright c 2006 by Mark E. Irwin Matrix Approach to Regression Linear Model: Y i = β 0 + β 1 X i1 +... + β p X ip + ɛ i ; ɛ i iid N(0, σ 2 ), i = 1,..., n
More informationStatistics for Engineers Lecture 9 Linear Regression
Statistics for Engineers Lecture 9 Linear Regression Chong Ma Department of Statistics University of South Carolina chongm@email.sc.edu April 17, 2017 Chong Ma (Statistics, USC) STAT 509 Spring 2017 April
More informationCategorical Predictor Variables
Categorical Predictor Variables We often wish to use categorical (or qualitative) variables as covariates in a regression model. For binary variables (taking on only 2 values, e.g. sex), it is relatively
More informationSTAT 540: Data Analysis and Regression
STAT 540: Data Analysis and Regression Wen Zhou http://www.stat.colostate.edu/~riczw/ Email: riczw@stat.colostate.edu Department of Statistics Colorado State University Fall 205 W. Zhou (Colorado State
More informationSTAT420 Midterm Exam. University of Illinois Urbana-Champaign October 19 (Friday), :00 4:15p. SOLUTIONS (Yellow)
STAT40 Midterm Exam University of Illinois Urbana-Champaign October 19 (Friday), 018 3:00 4:15p SOLUTIONS (Yellow) Question 1 (15 points) (10 points) 3 (50 points) extra ( points) Total (77 points) Points
More informationRegression Models - Introduction
Regression Models - Introduction In regression models there are two types of variables that are studied: A dependent variable, Y, also called response variable. It is modeled as random. An independent
More informationLecture 14 Simple Linear Regression
Lecture 4 Simple Linear Regression Ordinary Least Squares (OLS) Consider the following simple linear regression model where, for each unit i, Y i is the dependent variable (response). X i is the independent
More informationCorrelation Analysis
Simple Regression Correlation Analysis Correlation analysis is used to measure strength of the association (linear relationship) between two variables Correlation is only concerned with strength of the
More informationAMS 315/576 Lecture Notes. Chapter 11. Simple Linear Regression
AMS 315/576 Lecture Notes Chapter 11. Simple Linear Regression 11.1 Motivation A restaurant opening on a reservations-only basis would like to use the number of advance reservations x to predict the number
More informationTests of Linear Restrictions
Tests of Linear Restrictions 1. Linear Restricted in Regression Models In this tutorial, we consider tests on general linear restrictions on regression coefficients. In other tutorials, we examine some
More informationChapter 14. Linear least squares
Serik Sagitov, Chalmers and GU, March 5, 2018 Chapter 14 Linear least squares 1 Simple linear regression model A linear model for the random response Y = Y (x) to an independent variable X = x For a given
More informationMultivariate Regression
Multivariate Regression The so-called supervised learning problem is the following: we want to approximate the random variable Y with an appropriate function of the random variables X 1,..., X p with the
More informationMathematics for Economics MA course
Mathematics for Economics MA course Simple Linear Regression Dr. Seetha Bandara Simple Regression Simple linear regression is a statistical method that allows us to summarize and study relationships between
More informationChapter 14 Student Lecture Notes Department of Quantitative Methods & Information Systems. Business Statistics. Chapter 14 Multiple Regression
Chapter 14 Student Lecture Notes 14-1 Department of Quantitative Methods & Information Systems Business Statistics Chapter 14 Multiple Regression QMIS 0 Dr. Mohammad Zainal Chapter Goals After completing
More informationMS&E 226: Small Data
MS&E 226: Small Data Lecture 15: Examples of hypothesis tests (v5) Ramesh Johari ramesh.johari@stanford.edu 1 / 32 The recipe 2 / 32 The hypothesis testing recipe In this lecture we repeatedly apply the
More informationMath 3330: Solution to midterm Exam
Math 3330: Solution to midterm Exam Question 1: (14 marks) Suppose the regression model is y i = β 0 + β 1 x i + ε i, i = 1,, n, where ε i are iid Normal distribution N(0, σ 2 ). a. (2 marks) Compute the
More informationDensity Temp vs Ratio. temp
Temp Ratio Density 0.00 0.02 0.04 0.06 0.08 0.10 0.12 Density 0.0 0.2 0.4 0.6 0.8 1.0 1. (a) 170 175 180 185 temp 1.0 1.5 2.0 2.5 3.0 ratio The histogram shows that the temperature measures have two peaks,
More informationLecture 18: Simple Linear Regression
Lecture 18: Simple Linear Regression BIOS 553 Department of Biostatistics University of Michigan Fall 2004 The Correlation Coefficient: r The correlation coefficient (r) is a number that measures the strength
More informationSTAT5044: Regression and Anova. Inyoung Kim
STAT5044: Regression and Anova Inyoung Kim 2 / 47 Outline 1 Regression 2 Simple Linear regression 3 Basic concepts in regression 4 How to estimate unknown parameters 5 Properties of Least Squares Estimators:
More informationIntroduction and Single Predictor Regression. Correlation
Introduction and Single Predictor Regression Dr. J. Kyle Roberts Southern Methodist University Simmons School of Education and Human Development Department of Teaching and Learning Correlation A correlation
More informationLinear Regression. Simple linear regression model determines the relationship between one dependent variable (y) and one independent variable (x).
Linear Regression Simple linear regression model determines the relationship between one dependent variable (y) and one independent variable (x). A dependent variable is a random variable whose variation
More informationSTAT 3A03 Applied Regression With SAS Fall 2017
STAT 3A03 Applied Regression With SAS Fall 2017 Assignment 2 Solution Set Q. 1 I will add subscripts relating to the question part to the parameters and their estimates as well as the errors and residuals.
More informationMultiple Regression Analysis. Part III. Multiple Regression Analysis
Part III Multiple Regression Analysis As of Sep 26, 2017 1 Multiple Regression Analysis Estimation Matrix form Goodness-of-Fit R-square Adjusted R-square Expected values of the OLS estimators Irrelevant
More information6. Multiple Linear Regression
6. Multiple Linear Regression SLR: 1 predictor X, MLR: more than 1 predictor Example data set: Y i = #points scored by UF football team in game i X i1 = #games won by opponent in their last 10 games X
More informationApplied Econometrics (QEM)
Applied Econometrics (QEM) based on Prinicples of Econometrics Jakub Mućk Department of Quantitative Economics Jakub Mućk Applied Econometrics (QEM) Meeting #3 1 / 42 Outline 1 2 3 t-test P-value Linear
More informationThe Multiple Regression Model
Multiple Regression The Multiple Regression Model Idea: Examine the linear relationship between 1 dependent (Y) & or more independent variables (X i ) Multiple Regression Model with k Independent Variables:
More informationMultivariate Regression (Chapter 10)
Multivariate Regression (Chapter 10) This week we ll cover multivariate regression and maybe a bit of canonical correlation. Today we ll mostly review univariate multivariate regression. With multivariate
More informationHomework 2: Simple Linear Regression
STAT 4385 Applied Regression Analysis Homework : Simple Linear Regression (Simple Linear Regression) Thirty (n = 30) College graduates who have recently entered the job market. For each student, the CGPA
More informationSimple linear regression
Simple linear regression Biometry 755 Spring 2008 Simple linear regression p. 1/40 Overview of regression analysis Evaluate relationship between one or more independent variables (X 1,...,X k ) and a single
More informationECON 450 Development Economics
ECON 450 Development Economics Statistics Background University of Illinois at Urbana-Champaign Summer 2017 Outline 1 Introduction 2 3 4 5 Introduction Regression analysis is one of the most important
More informationComparing Nested Models
Comparing Nested Models ST 370 Two regression models are called nested if one contains all the predictors of the other, and some additional predictors. For example, the first-order model in two independent
More informationLectures on Simple Linear Regression Stat 431, Summer 2012
Lectures on Simple Linear Regression Stat 43, Summer 0 Hyunseung Kang July 6-8, 0 Last Updated: July 8, 0 :59PM Introduction Previously, we have been investigating various properties of the population
More informationHomoskedasticity. Var (u X) = σ 2. (23)
Homoskedasticity How big is the difference between the OLS estimator and the true parameter? To answer this question, we make an additional assumption called homoskedasticity: Var (u X) = σ 2. (23) This
More informationChapter 4: Regression Models
Sales volume of company 1 Textbook: pp. 129-164 Chapter 4: Regression Models Money spent on advertising 2 Learning Objectives After completing this chapter, students will be able to: Identify variables,
More informationUNIVERSITY OF MASSACHUSETTS. Department of Mathematics and Statistics. Basic Exam - Applied Statistics. Tuesday, January 17, 2017
UNIVERSITY OF MASSACHUSETTS Department of Mathematics and Statistics Basic Exam - Applied Statistics Tuesday, January 17, 2017 Work all problems 60 points are needed to pass at the Masters Level and 75
More informationRegression Analysis. Regression: Methodology for studying the relationship among two or more variables
Regression Analysis Regression: Methodology for studying the relationship among two or more variables Two major aims: Determine an appropriate model for the relationship between the variables Predict the
More informationBasic Business Statistics 6 th Edition
Basic Business Statistics 6 th Edition Chapter 12 Simple Linear Regression Learning Objectives In this chapter, you learn: How to use regression analysis to predict the value of a dependent variable based
More informationSimple Linear Regression
Simple Linear Regression September 24, 2008 Reading HH 8, GIll 4 Simple Linear Regression p.1/20 Problem Data: Observe pairs (Y i,x i ),i = 1,...n Response or dependent variable Y Predictor or independent
More informationInference for Regression Simple Linear Regression
Inference for Regression Simple Linear Regression IPS Chapter 10.1 2009 W.H. Freeman and Company Objectives (IPS Chapter 10.1) Simple linear regression p Statistical model for linear regression p Estimating
More informationwhere x and ȳ are the sample means of x 1,, x n
y y Animal Studies of Side Effects Simple Linear Regression Basic Ideas In simple linear regression there is an approximately linear relation between two variables say y = pressure in the pancreas x =
More informationInference. ME104: Linear Regression Analysis Kenneth Benoit. August 15, August 15, 2012 Lecture 3 Multiple linear regression 1 1 / 58
Inference ME104: Linear Regression Analysis Kenneth Benoit August 15, 2012 August 15, 2012 Lecture 3 Multiple linear regression 1 1 / 58 Stata output resvisited. reg votes1st spend_total incumb minister
More informationChapter 12 - Lecture 2 Inferences about regression coefficient
Chapter 12 - Lecture 2 Inferences about regression coefficient April 19th, 2010 Facts about slope Test Statistic Confidence interval Hypothesis testing Test using ANOVA Table Facts about slope In previous
More informationChapter 4. Regression Models. Learning Objectives
Chapter 4 Regression Models To accompany Quantitative Analysis for Management, Eleventh Edition, by Render, Stair, and Hanna Power Point slides created by Brian Peterson Learning Objectives After completing
More informationCorrelation and the Analysis of Variance Approach to Simple Linear Regression
Correlation and the Analysis of Variance Approach to Simple Linear Regression Biometry 755 Spring 2009 Correlation and the Analysis of Variance Approach to Simple Linear Regression p. 1/35 Correlation
More informationHandout 4: Simple Linear Regression
Handout 4: Simple Linear Regression By: Brandon Berman The following problem comes from Kokoska s Introductory Statistics: A Problem-Solving Approach. The data can be read in to R using the following code:
More informationLecture 10 Multiple Linear Regression
Lecture 10 Multiple Linear Regression STAT 512 Spring 2011 Background Reading KNNL: 6.1-6.5 10-1 Topic Overview Multiple Linear Regression Model 10-2 Data for Multiple Regression Y i is the response variable
More informationStatistics for Managers using Microsoft Excel 6 th Edition
Statistics for Managers using Microsoft Excel 6 th Edition Chapter 13 Simple Linear Regression 13-1 Learning Objectives In this chapter, you learn: How to use regression analysis to predict the value of
More informationST430 Exam 2 Solutions
ST430 Exam 2 Solutions Date: November 9, 2015 Name: Guideline: You may use one-page (front and back of a standard A4 paper) of notes. No laptop or textbook are permitted but you may use a calculator. Giving
More informationLecture 15. Hypothesis testing in the linear model
14. Lecture 15. Hypothesis testing in the linear model Lecture 15. Hypothesis testing in the linear model 1 (1 1) Preliminary lemma 15. Hypothesis testing in the linear model 15.1. Preliminary lemma Lemma
More informationChapter 12: Multiple Linear Regression
Chapter 12: Multiple Linear Regression Seungchul Baek Department of Statistics, University of South Carolina STAT 509: Statistics for Engineers 1 / 55 Introduction A regression model can be expressed as
More informationECO220Y Simple Regression: Testing the Slope
ECO220Y Simple Regression: Testing the Slope Readings: Chapter 18 (Sections 18.3-18.5) Winter 2012 Lecture 19 (Winter 2012) Simple Regression Lecture 19 1 / 32 Simple Regression Model y i = β 0 + β 1 x
More informationSTAT 511. Lecture : Simple linear regression Devore: Section Prof. Michael Levine. December 3, Levine STAT 511
STAT 511 Lecture : Simple linear regression Devore: Section 12.1-12.4 Prof. Michael Levine December 3, 2018 A simple linear regression investigates the relationship between the two variables that is not
More informationMultiple Regression Methods
Chapter 1: Multiple Regression Methods Hildebrand, Ott and Gray Basic Statistical Ideas for Managers Second Edition 1 Learning Objectives for Ch. 1 The Multiple Linear Regression Model How to interpret
More informationInferences for Regression
Inferences for Regression An Example: Body Fat and Waist Size Looking at the relationship between % body fat and waist size (in inches). Here is a scatterplot of our data set: Remembering Regression In
More information17: INFERENCE FOR MULTIPLE REGRESSION. Inference for Individual Regression Coefficients
17: INFERENCE FOR MULTIPLE REGRESSION Inference for Individual Regression Coefficients The results of this section require the assumption that the errors u are normally distributed. Let c i ij denote the
More informationProblems. Suppose both models are fitted to the same data. Show that SS Res, A SS Res, B
Simple Linear Regression 35 Problems 1 Consider a set of data (x i, y i ), i =1, 2,,n, and the following two regression models: y i = β 0 + β 1 x i + ε, (i =1, 2,,n), Model A y i = γ 0 + γ 1 x i + γ 2
More informationSimple Linear Regression
Simple Linear Regression Reading: Hoff Chapter 9 November 4, 2009 Problem Data: Observe pairs (Y i,x i ),i = 1,... n Response or dependent variable Y Predictor or independent variable X GOALS: Exploring
More informationLecture 1: Linear Models and Applications
Lecture 1: Linear Models and Applications Claudia Czado TU München c (Claudia Czado, TU Munich) ZFS/IMS Göttingen 2004 0 Overview Introduction to linear models Exploratory data analysis (EDA) Estimation
More informationRegression Models. Chapter 4. Introduction. Introduction. Introduction
Chapter 4 Regression Models Quantitative Analysis for Management, Tenth Edition, by Render, Stair, and Hanna 008 Prentice-Hall, Inc. Introduction Regression analysis is a very valuable tool for a manager
More information13 Simple Linear Regression
B.Sc./Cert./M.Sc. Qualif. - Statistics: Theory and Practice 3 Simple Linear Regression 3. An industrial example A study was undertaken to determine the effect of stirring rate on the amount of impurity
More informationMAT2377. Rafa l Kulik. Version 2015/November/26. Rafa l Kulik
MAT2377 Rafa l Kulik Version 2015/November/26 Rafa l Kulik Bivariate data and scatterplot Data: Hydrocarbon level (x) and Oxygen level (y): x: 0.99, 1.02, 1.15, 1.29, 1.46, 1.36, 0.87, 1.23, 1.55, 1.40,
More informationApplied Regression Analysis. Section 2: Multiple Linear Regression
Applied Regression Analysis Section 2: Multiple Linear Regression 1 The Multiple Regression Model Many problems involve more than one independent variable or factor which affects the dependent or response
More informationChapter 14 Simple Linear Regression (A)
Chapter 14 Simple Linear Regression (A) 1. Characteristics Managerial decisions often are based on the relationship between two or more variables. can be used to develop an equation showing how the variables
More informationECON3150/4150 Spring 2015
ECON3150/4150 Spring 2015 Lecture 3&4 - The linear regression model Siv-Elisabeth Skjelbred University of Oslo January 29, 2015 1 / 67 Chapter 4 in S&W Section 17.1 in S&W (extended OLS assumptions) 2
More informationassumes a linear relationship between mean of Y and the X s with additive normal errors the errors are assumed to be a sample from N(0, σ 2 )
Multiple Linear Regression is used to relate a continuous response (or dependent) variable Y to several explanatory (or independent) (or predictor) variables X 1, X 2,, X k assumes a linear relationship
More informationSimple Linear Regression. Material from Devore s book (Ed 8), and Cengagebrain.com
12 Simple Linear Regression Material from Devore s book (Ed 8), and Cengagebrain.com The Simple Linear Regression Model The simplest deterministic mathematical relationship between two variables x and
More informationData Mining Stat 588
Data Mining Stat 588 Lecture 02: Linear Methods for Regression Department of Statistics & Biostatistics Rutgers University September 13 2011 Regression Problem Quantitative generic output variable Y. Generic
More informationMultiple Regression. Inference for Multiple Regression and A Case Study. IPS Chapters 11.1 and W.H. Freeman and Company
Multiple Regression Inference for Multiple Regression and A Case Study IPS Chapters 11.1 and 11.2 2009 W.H. Freeman and Company Objectives (IPS Chapters 11.1 and 11.2) Multiple regression Data for multiple
More informationOrdinary Least Squares Regression Explained: Vartanian
Ordinary Least Squares Regression Explained: Vartanian When to Use Ordinary Least Squares Regression Analysis A. Variable types. When you have an interval/ratio scale dependent variable.. When your independent
More informationLinear Models and Estimation by Least Squares
Linear Models and Estimation by Least Squares Jin-Lung Lin 1 Introduction Causal relation investigation lies in the heart of economics. Effect (Dependent variable) cause (Independent variable) Example:
More informationRegression Models - Introduction
Regression Models - Introduction In regression models, two types of variables that are studied: A dependent variable, Y, also called response variable. It is modeled as random. An independent variable,
More informationSTAT 4385 Topic 03: Simple Linear Regression
STAT 4385 Topic 03: Simple Linear Regression Xiaogang Su, Ph.D. Department of Mathematical Science University of Texas at El Paso xsu@utep.edu Spring, 2017 Outline The Set-Up Exploratory Data Analysis
More information