STAT763: Applied Regression Analysis. Multiple linear regression. 4.4 Hypothesis testing
|
|
|
- Letitia Baker
- 5 years ago
- Views:
Transcription
1 STAT763: Applied Regression Analysis Multiple linear regression 4.4 Hypothesis testing Chunsheng Ma
2 4.4.1 Significance of regression Null hypothesis (Test whether all β j = 0, j = 1,..., k) H 0 : β 1 = β 2 =... = β k = 0
3 4.4.1 Significance of regression Null hypothesis (Test whether all β j = 0, j = 1,..., k) H 0 : β 1 = β 2 =... = β k = 0 Alternative hypothesis H a : Not all βs are equal to zero
4 4.4.1 Significance of regression Null hypothesis (Test whether all β j = 0, j = 1,..., k) H 0 : β 1 = β 2 =... = β k = 0 Alternative hypothesis H a : Not all βs are equal to zero Test statistic F = MS R MS Res
5 4.4.1 Significance of regression Null hypothesis (Test whether all β j = 0, j = 1,..., k) H 0 : β 1 = β 2 =... = β k = 0 Alternative hypothesis H a : Not all βs are equal to zero Test statistic F = MS R MS Res ANOVA table Source SS df MS Test statistic Regression SS R k MS R F Residuals SS Res n-k-1 MS Res Total SS T n-1
6 Delivery time data (Example 3.1) y = c(16.68, 11.50, 12.03, 14.88, 13.75, 18.11, 8.00, 17.83, 79.24, 21.50, 40.33, 21.00, 13.50, 19.75, 24.00,29.00, 15.35, 19.00, 9.50, 35.10, 17.90, 52.32, 18.75, 19.83, 10.75) x1 = c(7, 3, 3, 4, 6, 7, 2, 7, 30, 5, 16, 10, 4, 6, 9, 10, 6, 7, 3, 17, 10, 26, 9, 8, 4) x2 = c(560, 220, 340, 80, 150, 330, 110, 210, 1460, 605, 688, 215, 255, 462, 448, 776, 200, 132, 36, 770, 140, 810, 450, 635, 150) myfit = lm ( y ~ x1+x2)
7 ANOVA table > anova (myfit) Analysis of Variance Table Response: y Df Sum Sq Mean Sq F value Pr(>F) x < 2.2e-16 x Residuals
8 ANOVA table > anova (myfit) Analysis of Variance Table Response: y Df Sum Sq Mean Sq F value Pr(>F) x < 2.2e-16 x Residuals From the above table, we obtain SS Res = 233.7, or > anova(myfit)[3,2] [1]
9 SS R SS R = = , or > SS_R = anova(myfit)[1,2]+anova(myfit)[2,2] [1]
10 SS R SS R = = , or > SS_R = anova(myfit)[1,2]+anova(myfit)[2,2] [1] The observed F value F = SS R/2 = /2 = MS Res 10.6
11 SS R SS R = = , or > SS_R = anova(myfit)[1,2]+anova(myfit)[2,2] [1] The observed F value F = SS R/2 = /2 = MS Res 10.6 MS_R = SS_R/(anova(myfit)[1,1]+anova(myfit)[2,1]) F = MS_R/anova(myfit)[3,3]
12 SS R SS R = = , or > SS_R = anova(myfit)[1,2]+anova(myfit)[2,2] [1] The observed F value F = SS R/2 = /2 = MS Res 10.6 MS_R = SS_R/(anova(myfit)[1,1]+anova(myfit)[2,1]) F = MS_R/anova(myfit)[3,3] p-value (tail probability) > 1-pf(F,2,22) [1] e-16 P(F 2,22 > observed F value)
13 Two ways to make decision (1) Report p value, which is evaluated from the data. A small p-value leads to reject H 0. (2) Use the significance level α, which is previously given. Take α = 0.05, say. From the F table, we find the threshold, qf (1 0.05, 2, 22) > qf(1-0.05,2,22) [1] Then, compare the observed F value with this threshold. Finally, make the decision.
14 Derive the F test via the likelihood-ratio-test method H 0 : β 1 = β 2 =... = β k = 0 H a : Not all βs are equal to zero The whole parameter space in this case is Θ = {(β 0, β 1,..., β k, σ 2 ) : (β 0, β 1,..., β k ) R k+1, σ 2 (0, )}
15 Derive the F test via the likelihood-ratio-test method H 0 : β 1 = β 2 =... = β k = 0 H a : Not all βs are equal to zero The whole parameter space in this case is Θ = {(β 0, β 1,..., β k, σ 2 ) : (β 0, β 1,..., β k ) R k+1, σ 2 (0, )} The parameter space associated with H 0 is Θ 0 = {(β 0, 0,..., 0, σ 2 ) : β 0 R, σ 2 (0, )}
16 Derive the F test via the likelihood-ratio-test method H 0 : β 1 = β 2 =... = β k = 0 H a : Not all βs are equal to zero The whole parameter space in this case is Θ = {(β 0, β 1,..., β k, σ 2 ) : (β 0, β 1,..., β k ) R k+1, σ 2 (0, )} The parameter space associated with H 0 is Θ 0 = {(β 0, 0,..., 0, σ 2 ) : β 0 R, σ 2 (0, )} Likelihood function of parameter (β 0, β 1,..., β k, σ 2 ) { l(β 0, β 1,..., β k, σ 2 1 ) = ( 2πσ) exp 1 } n 2σ 2 (Y Xβ) (Y Xβ)
17 Likelihood ratio Λ = sup l(β 0, β 1,..., β k, σ 2 ) Θ 0 sup l(β 0, β 1,..., β k, σ 2 ) Θ
18 Likelihood ratio Λ = sup l(β 0, β 1,..., β k, σ 2 ) Θ 0 sup l(β 0, β 1,..., β k, σ 2 ) Θ sup l(β 0, β 1,..., β k, σ 2 ) = l Θ 0 ( ȳ, 0,..., 0, 1 n i=1 ( = l ȳ, 0,..., 0, 1 n SS T ) n (y i ȳ) 2 )
19 Likelihood ratio Λ = sup l(β 0, β 1,..., β k, σ 2 ) Θ 0 sup l(β 0, β 1,..., β k, σ 2 ) Θ sup l(β 0, β 1,..., β k, σ 2 ) = l Θ 0 ( ȳ, 0,..., 0, 1 n i=1 ( = l ȳ, 0,..., 0, 1 n SS T ) n (y i ȳ) 2 ) ( ) sup l(β 0, β 1,..., β k, σ 2 ) = l ˆβ 0, ˆβ 1,..., ˆβ k, 1 n (y i ŷ i ) 2 Θ n i=1 ( = l ˆβ 0, ˆβ 1,..., ˆβ k, 1 ) n SS Res
20 The likelihood ratio Λ = = sup l(β 0, β 1,..., β k, σ 2 ) Θ 0 sup l(β 0, β 1,..., β k, σ 2 ) Θ ( ) n k 1 + n k 1 F 2, which is a decreasing function of F.
21 The likelihood ratio Λ = = sup l(β 0, β 1,..., β k, σ 2 ) Θ 0 sup l(β 0, β 1,..., β k, σ 2 ) Θ ( ) n k 1 + n k 1 F 2, which is a decreasing function of F. Thus, a larger F would lead us to conclude H a.
22 4.4.2 Test whether a single β j = 0 H 0 : β j = 0 H a : β j 0 Two ways for this hypothesis testing question (1). Student s t test T = ˆβ j s{ ˆβ j } The corresponding value can be obtained in R by summary(lm(y x1+...)), where s{ ˆβ j } is the standard error of ˆβ j.
23 Rational A Student s t variable is formulated based on the following three facts ˆβ j N ( β j, σ 2 (X X) 1 ) ˆβj β j+1,j+1, or j N(0, 1) σ 2 (X X) 1 j+1,j+1
24 Rational A Student s t variable is formulated based on the following three facts ˆβ j N ( β j, σ 2 (X X) 1 ) ˆβj β j+1,j+1, or j N(0, 1) SS Res σ 2 χ 2 n k 1, σ 2 (X X) 1 j+1,j+1
25 Rational A Student s t variable is formulated based on the following three facts ˆβ j N ( β j, σ 2 (X X) 1 ) ˆβj β j+1,j+1, or j N(0, 1) SS Res σ 2 χ 2 n k 1, ˆβ j and SS Res are independent each other. σ 2 (X X) 1 j+1,j+1
26 Rational A Student s t variable is formulated based on the following three facts ˆβ j N ( β j, σ 2 (X X) 1 ) ˆβj β j+1,j+1, or j N(0, 1) SS Res σ 2 χ 2 n k 1, ˆβ j and SS Res are independent each other. σ 2 (X X) 1 j+1,j+1 Student s t variable or ˆβ j β j σ 2 (X X) 1 j+1,j+1 SS Res t n k 1 σ /(n k 1) 2 ˆβ j β j MS Res (X X) 1 j+1,j+1 = ˆβ j β j s( ˆβ j ) t n k 1
27 When H 0 : β j = 0 is true, ˆβ j s( ˆβ j ) t n k 1, where s( ˆβ j ) = MS Res (X X) 1 j+1,j+1 Test statistic for H 0 : β j = 0 T = ˆβ j s( ˆβ j ) Two ways could be used to make a decision: p-value or significance level.
28 (2). Partial F test F test statistic F = SS R(X j X 1,..., X j 1, X j+1,..., X k ) 1 = SS R(X j X 1,..., X j 1, X j+1,..., X k ) MS Res SS Res(X 1,..., X k ) n k 1 SS R (X j X 1,..., X j 1, X j+1,..., X k ) = SS R (use all predictor variables in the model) SS R (use predictor variables X 1,..., X j 1, X j+1,..., X k )
29 Sum of squares SS T : WITHOUT taking any predictor variables into account. n SS T = (Y i Ȳ )2 i=1
30 Sum of squares SS T : WITHOUT taking any predictor variables into account. n SS T = (Y i Ȳ )2 i=1 (i) Taking X 1 into account, Y is regressed on X 1 alone (simple linear regression). SS R (X 1 ) = the regression sum of squares when X 1 only is in the model SS Res (X 1 ) = the error sum of squares when X 1 only is in the model SS T = SS R (X 1 ) + SS Res (X 1 ).
31 Sum of squares SS T : WITHOUT taking any predictor variables into account. n SS T = (Y i Ȳ )2 i=1 (i) Taking X 1 into account, Y is regressed on X 1 alone (simple linear regression). SS R (X 1 ) = the regression sum of squares when X 1 only is in the model SS Res (X 1 ) = the error sum of squares when X 1 only is in the model SS T = SS R (X 1 ) + SS Res (X 1 ). (ii) Taking both X 1 and X 2 into account, Y is now regressed on X 1 and X 2 (two predictor variables). SS R (X 1, X 2 ) = the regression sum of squares when X 1 and X 2 are in the model SS Res (X 1, X 2 ) = the error sum of squares when X 1 and X 2 are in the model SS T = SS R (X 1, X 2 ) + SS Res (X 1, X 2 ).
32 Questions Which one is larger, SS Res (X 1 ) or SS Res (X 1, X 2 )? Which one is larger, SS R (X 1 ) or SS R (X 1, X 2 )?
33 Questions Which one is larger, SS Res (X 1 ) or SS Res (X 1, X 2 )? Which one is larger, SS R (X 1 ) or SS R (X 1, X 2 )? Answers SS Res (X 1 ) SS Res (X 1, X 2 ) SS R (X 1 ) SS R (X 1, X 2 )
34 Questions Which one is larger, SS Res (X 1 ) or SS Res (X 1, X 2 )? Which one is larger, SS R (X 1 ) or SS R (X 1, X 2 )? Answers SS Res (X 1 ) SS Res (X 1, X 2 ) Extra sum of squares SS R (X 1 ) SS R (X 1, X 2 ) SS Res (X 2 X 1 ) = SS Res (X 1 ) SS Res (X 1, X 2 ) It measures the marginal effect of adding X 2 to the regression model, when X 1 is already in the model (and X 2 is regarded as an extra variable). SS R (X 2 X 1 ) = SS R (X 1, X 2 ) SS R (X 1 ) it is the marginal increase in the regression sum of squares.
35 Questions Which one is larger, SS Res (X 1 ) or SS Res (X 1, X 2 )? Which one is larger, SS R (X 1 ) or SS R (X 1, X 2 )? Answers SS Res (X 1 ) SS Res (X 1, X 2 ) Extra sum of squares SS R (X 1 ) SS R (X 1, X 2 ) SS Res (X 2 X 1 ) = SS Res (X 1 ) SS Res (X 1, X 2 ) It measures the marginal effect of adding X 2 to the regression model, when X 1 is already in the model (and X 2 is regarded as an extra variable). SS R (X 2 X 1 ) = SS R (X 1, X 2 ) SS R (X 1 ) it is the marginal increase in the regression sum of squares. SSTO = SSR(X 1 ) + SSR(X 2 X 1 ) + SSE(X 1, X 2 )
36 4.4.3 Test several β j = 0, j = 1,..., r H 0 : β 1 =... = β r = 0 H a : not all β j 0
37 4.4.3 Test several β j = 0, j = 1,..., r H 0 : β 1 =... = β r = 0 H a : not all β j 0 Test statistic F = SS R(X 1,..., X r X r+1,..., X k ) r = SS R(X 1,..., X r X r+1,..., X k ) r MS Res SS Res(X 1,..., X k ) n k 1 SS R (X 1,..., X r X r+1,..., X k ) = SS R (use all predictor variables in the model) SS R (use predictor variables X r+1,..., X k ) = SS Res (use predictor variables X r+1,..., X k ) SS Res (use all predictor variables in the model)
38 Derive the F test via the likelihood-ratio-test method H 0 : β 1 = β 2 =... = β r = 0 H 1 : Not all βs are equal to zero The whole parameter space in this case is Θ = {(β 0, β 1,..., β k, σ 2 ) : (β 0, β 1,..., β k ) R k+1, σ 2 (0, )}
39 Derive the F test via the likelihood-ratio-test method H 0 : β 1 = β 2 =... = β r = 0 H 1 : Not all βs are equal to zero The whole parameter space in this case is Θ = {(β 0, β 1,..., β k, σ 2 ) : (β 0, β 1,..., β k ) R k+1, σ 2 (0, )} The parameter space associated with H 0 is Θ 0 = {(β 0, 0,..., 0, β r+1,..., β k, σ 2 ) : β 0 R, β r+1 R,..., β k R, σ 2 (0, )}
40 Derive the F test via the likelihood-ratio-test method H 0 : β 1 = β 2 =... = β r = 0 H 1 : Not all βs are equal to zero The whole parameter space in this case is Θ = {(β 0, β 1,..., β k, σ 2 ) : (β 0, β 1,..., β k ) R k+1, σ 2 (0, )} The parameter space associated with H 0 is Θ 0 = {(β 0, 0,..., 0, β r+1,..., β k, σ 2 ) : β 0 R, β r+1 R,..., β k R, σ 2 (0, )} Likelihood function of parameter (β 0, β 1,..., β k, σ 2 ) { l(β 0, β 1,..., β k, σ 2 1 ) = ( 2πσ) exp 1 } n 2σ 2 (Y Xβ) (Y Xβ)
41 Likelihood ratio Λ = sup l(β 0, β 1,..., β k, σ 2 ) Θ 0 sup l(β 0, β 1,..., β k, σ 2 ) Θ
42 Likelihood ratio Λ = sup l(β 0, β 1,..., β k, σ 2 ) Θ 0 sup l(β 0, β 1,..., β k, σ 2 ) Θ sup l(β 0, β 1,..., β k, σ 2 ) = l(, 0,..., 0,,...,,?) Θ 0
43 Likelihood ratio Λ = sup l(β 0, β 1,..., β k, σ 2 ) Θ 0 sup l(β 0, β 1,..., β k, σ 2 ) Θ sup l(β 0, β 1,..., β k, σ 2 ) = l(, 0,..., 0,,...,,?) Θ 0 ( ) sup l(β 0, β 1,..., β k, σ 2 ) = l ˆβ 0, ˆβ 1,..., ˆβ k, 1 n (y i ŷ i ) 2 Θ n i=1 ( = l ˆβ 0, ˆβ 1,..., ˆβ k, 1 ) n SS Res
44 The likelihood ratio Λ = sup l(β 0, β 1,..., β k, σ 2 ) Θ 0 sup l(β 0, β 1,..., β k, σ 2 ) Θ = (1+?F ) n 2, which is a decreasing function of F.
45 The likelihood ratio Λ = sup l(β 0, β 1,..., β k, σ 2 ) Θ 0 sup l(β 0, β 1,..., β k, σ 2 ) Θ = (1+?F ) n 2, which is a decreasing function of F. Thus, a larger F would lead us to conclude H a.
Outline. Remedial Measures) Extra Sums of Squares Standardized Version of the Multiple Regression Model
Outline 1 Multiple Linear Regression (Estimation, Inference, Diagnostics and Remedial Measures) 2 Special Topics for Multiple Regression Extra Sums of Squares Standardized Version of the Multiple Regression
Biostatistics 380 Multiple Regression 1. Multiple Regression
Biostatistics 0 Multiple Regression ORIGIN 0 Multiple Regression Multiple Regression is an extension of the technique of linear regression to describe the relationship between a single dependent (response)
Ch 3: Multiple Linear Regression
Ch 3: Multiple Linear Regression 1. Multiple Linear Regression Model Multiple regression model has more than one regressor. For example, we have one response variable and two regressor variables: 1. delivery
STA 4210 Practise set 2b
STA 410 Practise set b For all significance tests, use = 0.05 significance level. S.1. A linear regression model is fit, relating fish catch (Y, in tons) to the number of vessels (X 1 ) and fishing pressure
Multiple Linear Regression
Multiple Linear Regression Simple linear regression tries to fit a simple line between two variables Y and X. If X is linearly related to Y this explains some of the variability in Y. In most cases, there
Linear models and their mathematical foundations: Simple linear regression
Linear models and their mathematical foundations: Simple linear regression Steffen Unkel Department of Medical Statistics University Medical Center Göttingen, Germany Winter term 2018/19 1/21 Introduction
STAT5044: Regression and Anova
STAT5044: Regression and Anova Inyoung Kim 1 / 25 Outline 1 Multiple Linear Regression 2 / 25 Basic Idea An extra sum of squares: the marginal reduction in the error sum of squares when one or several
Chapter 12 - Lecture 2 Inferences about regression coefficient
Chapter 12 - Lecture 2 Inferences about regression coefficient April 19th, 2010 Facts about slope Test Statistic Confidence interval Hypothesis testing Test using ANOVA Table Facts about slope In previous
Chapter 8: Hypothesis Testing Lecture 9: Likelihood ratio tests
Chapter 8: Hypothesis Testing Lecture 9: Likelihood ratio tests Throughout this chapter we consider a sample X taken from a population indexed by θ Θ R k. Instead of estimating the unknown parameter, we
Inferences for Regression
Inferences for Regression An Example: Body Fat and Waist Size Looking at the relationship between % body fat and waist size (in inches). Here is a scatterplot of our data set: Remembering Regression In
STA 4210 Practise set 2a
STA 410 Practise set a For all significance tests, use = 0.05 significance level. S.1. A multiple linear regression model is fit, relating household weekly food expenditures (Y, in $100s) to weekly income
Math 423/533: The Main Theoretical Topics
Math 423/533: The Main Theoretical Topics Notation sample size n, data index i number of predictors, p (p = 2 for simple linear regression) y i : response for individual i x i = (x i1,..., x ip ) (1 p)
Ch 2: Simple Linear Regression
Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component
PubH 7405: REGRESSION ANALYSIS. MLR: INFERENCES, Part I
PubH 7405: REGRESSION ANALYSIS MLR: INFERENCES, Part I TESTING HYPOTHESES Once we have fitted a multiple linear regression model and obtained estimates for the various parameters of interest, we want to
Inference for Regression
Inference for Regression Section 9.4 Cathy Poliak, Ph.D. cathy@math.uh.edu Office in Fleming 11c Department of Mathematics University of Houston Lecture 13b - 3339 Cathy Poliak, Ph.D. cathy@math.uh.edu
School of Mathematical Sciences. Question 1
School of Mathematical Sciences MTH5120 Statistical Modelling I Practical 8 and Assignment 7 Solutions Question 1 Figure 1: The residual plots do not contradict the model assumptions of normality, constant
Correlation Analysis
Simple Regression Correlation Analysis Correlation analysis is used to measure strength of the association (linear relationship) between two variables Correlation is only concerned with strength of the
Sections 7.1, 7.2, 7.4, & 7.6
Sections 7.1, 7.2, 7.4, & 7.6 Adapted from Timothy Hanson Department of Statistics, University of South Carolina Stat 704: Data Analysis I 1 / 25 Chapter 7 example: Body fat n = 20 healthy females 25 34
Lecture 15 Multiple regression I Chapter 6 Set 2 Least Square Estimation The quadratic form to be minimized is
Lecture 15 Multiple regression I Chapter 6 Set 2 Least Square Estimation The quadratic form to be minimized is Q = (Y i β 0 β 1 X i1 β 2 X i2 β p 1 X i.p 1 ) 2, which in matrix notation is Q = (Y Xβ) (Y
MATH 644: Regression Analysis Methods
MATH 644: Regression Analysis Methods FINAL EXAM Fall, 2012 INSTRUCTIONS TO STUDENTS: 1. This test contains SIX questions. It comprises ELEVEN printed pages. 2. Answer ALL questions for a total of 100
Lecture 15. Hypothesis testing in the linear model
14. Lecture 15. Hypothesis testing in the linear model Lecture 15. Hypothesis testing in the linear model 1 (1 1) Preliminary lemma 15. Hypothesis testing in the linear model 15.1. Preliminary lemma Lemma
Lecture 6 Multiple Linear Regression, cont.
Lecture 6 Multiple Linear Regression, cont. BIOST 515 January 22, 2004 BIOST 515, Lecture 6 Testing general linear hypotheses Suppose we are interested in testing linear combinations of the regression
Final Review. Yang Feng. Yang Feng (Columbia University) Final Review 1 / 58
Final Review Yang Feng http://www.stat.columbia.edu/~yangfeng Yang Feng (Columbia University) Final Review 1 / 58 Outline 1 Multiple Linear Regression (Estimation, Inference) 2 Special Topics for Multiple
MA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7
MA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7 1 Random Vectors Let a 0 and y be n 1 vectors, and let A be an n n matrix. Here, a 0 and A are non-random, whereas y is
6. Multiple Linear Regression
6. Multiple Linear Regression SLR: 1 predictor X, MLR: more than 1 predictor Example data set: Y i = #points scored by UF football team in game i X i1 = #games won by opponent in their last 10 games X
Figure 1: The fitted line using the shipment route-number of ampules data. STAT5044: Regression and ANOVA The Solution of Homework #2 Inyoung Kim
0.0 1.0 1.5 2.0 2.5 3.0 8 10 12 14 16 18 20 22 y x Figure 1: The fitted line using the shipment route-number of ampules data STAT5044: Regression and ANOVA The Solution of Homework #2 Inyoung Kim Problem#
We like to capture and represent the relationship between a set of possible causes and their response, by using a statistical predictive model.
Statistical Methods in Business Lecture 5. Linear Regression We like to capture and represent the relationship between a set of possible causes and their response, by using a statistical predictive model.
LECTURE 5 HYPOTHESIS TESTING
October 25, 2016 LECTURE 5 HYPOTHESIS TESTING Basic concepts In this lecture we continue to discuss the normal classical linear regression defined by Assumptions A1-A5. Let θ Θ R d be a parameter of interest.
CAS MA575 Linear Models
CAS MA575 Linear Models Boston University, Fall 2013 Midterm Exam (Correction) Instructor: Cedric Ginestet Date: 22 Oct 2013. Maximal Score: 200pts. Please Note: You will only be graded on work and answers
" M A #M B. Standard deviation of the population (Greek lowercase letter sigma) σ 2
Notation and Equations for Final Exam Symbol Definition X The variable we measure in a scientific study n The size of the sample N The size of the population M The mean of the sample µ The mean of the
Nested 2-Way ANOVA as Linear Models - Unbalanced Example
Linear Models Nested -Way ANOVA ORIGIN As with other linear models, unbalanced data require use of the regression approach, in this case by contrast coding of independent variables using a scheme not described
Lecture 18 MA Applied Statistics II D 2004
Lecture 18 MA 2612 - Applied Statistics II D 2004 Today 1. Examples of multiple linear regression 2. The modeling process (PNC 8.4) 3. The graphical exploration of multivariable data (PNC 8.5) 4. Fitting
Chapter 3: Multiple Regression. August 14, 2018
Chapter 3: Multiple Regression August 14, 2018 1 The multiple linear regression model The model y = β 0 +β 1 x 1 + +β k x k +ǫ (1) is called a multiple linear regression model with k regressors. The parametersβ
Topic 22 Analysis of Variance
Topic 22 Analysis of Variance Comparing Multiple Populations 1 / 14 Outline Overview One Way Analysis of Variance Sample Means Sums of Squares The F Statistic Confidence Intervals 2 / 14 Overview Two-sample
AMS 315/576 Lecture Notes. Chapter 11. Simple Linear Regression
AMS 315/576 Lecture Notes Chapter 11. Simple Linear Regression 11.1 Motivation A restaurant opening on a reservations-only basis would like to use the number of advance reservations x to predict the number
Formal Statement of Simple Linear Regression Model
Formal Statement of Simple Linear Regression Model Y i = β 0 + β 1 X i + ɛ i Y i value of the response variable in the i th trial β 0 and β 1 are parameters X i is a known constant, the value of the predictor
Lecture 13 Extra Sums of Squares
Lecture 13 Extra Sums of Squares STAT 512 Spring 2011 Background Reading KNNL: 7.1-7.4 13-1 Topic Overview Extra Sums of Squares (Defined) Using and Interpreting R 2 and Partial-R 2 Getting ESS and Partial-R
ST430 Exam 2 Solutions
ST430 Exam 2 Solutions Date: November 9, 2015 Name: Guideline: You may use one-page (front and back of a standard A4 paper) of notes. No laptop or textbook are permitted but you may use a calculator. Giving
Lecture 1: Linear Models and Applications
Lecture 1: Linear Models and Applications Claudia Czado TU München c (Claudia Czado, TU Munich) ZFS/IMS Göttingen 2004 0 Overview Introduction to linear models Exploratory data analysis (EDA) Estimation
Problems. Suppose both models are fitted to the same data. Show that SS Res, A SS Res, B
Simple Linear Regression 35 Problems 1 Consider a set of data (x i, y i ), i =1, 2,,n, and the following two regression models: y i = β 0 + β 1 x i + ε, (i =1, 2,,n), Model A y i = γ 0 + γ 1 x i + γ 2
Lecture 10 Multiple Linear Regression
Lecture 10 Multiple Linear Regression STAT 512 Spring 2011 Background Reading KNNL: 6.1-6.5 10-1 Topic Overview Multiple Linear Regression Model 10-2 Data for Multiple Regression Y i is the response variable
Basic Business Statistics 6 th Edition
Basic Business Statistics 6 th Edition Chapter 12 Simple Linear Regression Learning Objectives In this chapter, you learn: How to use regression analysis to predict the value of a dependent variable based
Unbalanced Data in Factorials Types I, II, III SS Part 1
Unbalanced Data in Factorials Types I, II, III SS Part 1 Chapter 10 in Oehlert STAT:5201 Week 9 - Lecture 2 1 / 14 When we perform an ANOVA, we try to quantify the amount of variability in the data accounted
Chapter 14 Student Lecture Notes Department of Quantitative Methods & Information Systems. Business Statistics. Chapter 14 Multiple Regression
Chapter 14 Student Lecture Notes 14-1 Department of Quantitative Methods & Information Systems Business Statistics Chapter 14 Multiple Regression QMIS 0 Dr. Mohammad Zainal Chapter Goals After completing
Linear regression. We have that the estimated mean in linear regression is. ˆµ Y X=x = ˆβ 0 + ˆβ 1 x. The standard error of ˆµ Y X=x is.
Linear regression We have that the estimated mean in linear regression is The standard error of ˆµ Y X=x is where x = 1 n s.e.(ˆµ Y X=x ) = σ ˆµ Y X=x = ˆβ 0 + ˆβ 1 x. 1 n + (x x)2 i (x i x) 2 i x i. The
STAT 540: Data Analysis and Regression
STAT 540: Data Analysis and Regression Wen Zhou http://www.stat.colostate.edu/~riczw/ Email: riczw@stat.colostate.edu Department of Statistics Colorado State University Fall 205 W. Zhou (Colorado State
Analysis of Covariance. The following example illustrates a case where the covariate is affected by the treatments.
Analysis of Covariance In some experiments, the experimental units (subjects) are nonhomogeneous or there is variation in the experimental conditions that are not due to the treatments. For example, a
Table 1: Fish Biomass data set on 26 streams
Math 221: Multiple Regression S. K. Hyde Chapter 27 (Moore, 5th Ed.) The following data set contains observations on the fish biomass of 26 streams. The potential regressors from which we wish to explain
Estimating σ 2. We can do simple prediction of Y and estimation of the mean of Y at any value of X.
Estimating σ 2 We can do simple prediction of Y and estimation of the mean of Y at any value of X. To perform inferences about our regression line, we must estimate σ 2, the variance of the error term.
Political Science 236 Hypothesis Testing: Review and Bootstrapping
Political Science 236 Hypothesis Testing: Review and Bootstrapping Rocío Titiunik Fall 2007 1 Hypothesis Testing Definition 1.1 Hypothesis. A hypothesis is a statement about a population parameter The
Regression Analysis. Regression: Methodology for studying the relationship among two or more variables
Regression Analysis Regression: Methodology for studying the relationship among two or more variables Two major aims: Determine an appropriate model for the relationship between the variables Predict the
Mathematical Notation Math Introduction to Applied Statistics
Mathematical Notation Math 113 - Introduction to Applied Statistics Name : Use Word or WordPerfect to recreate the following documents. Each article is worth 10 points and should be emailed to the instructor
STAT 213 Interactions in Two-Way ANOVA
STAT 213 Interactions in Two-Way ANOVA Colin Reimer Dawson Oberlin College 14 April 2016 Outline Last Time: Two-Way ANOVA Interaction Terms Reading Quiz (Multiple Choice) If there is no interaction present,
Applied Regression Analysis
Applied Regression Analysis Chapter 3 Multiple Linear Regression Hongcheng Li April, 6, 2013 Recall simple linear regression 1 Recall simple linear regression 2 Parameter Estimation 3 Interpretations of
Handout 4: Simple Linear Regression
Handout 4: Simple Linear Regression By: Brandon Berman The following problem comes from Kokoska s Introductory Statistics: A Problem-Solving Approach. The data can be read in to R using the following code:
Statistical Inference. Part IV. Statistical Inference
Part IV Statistical Inference As of Oct 5, 2017 Sampling Distributions of the OLS Estimator 1 Statistical Inference Sampling Distributions of the OLS Estimator Testing Against One-Sided Alternatives Two-Sided
No other aids are allowed. For example you are not allowed to have any other textbook or past exams.
UNIVERSITY OF TORONTO SCARBOROUGH Department of Computer and Mathematical Sciences Sample Exam Note: This is one of our past exams, In fact the only past exam with R. Before that we were using SAS. In
2. Regression Review
2. Regression Review 2.1 The Regression Model The general form of the regression model y t = f(x t, β) + ε t where x t = (x t1,, x tp ), β = (β 1,..., β m ). ε t is a random variable, Eε t = 0, Var(ε t
Concordia University (5+5)Q 1.
(5+5)Q 1. Concordia University Department of Mathematics and Statistics Course Number Section Statistics 360/1 40 Examination Date Time Pages Mid Term Test May 26, 2004 Two Hours 3 Instructor Course Examiner
Section 4.6 Simple Linear Regression
Section 4.6 Simple Linear Regression Objectives ˆ Basic philosophy of SLR and the regression assumptions ˆ Point & interval estimation of the model parameters, and how to make predictions ˆ Point and interval
Variance Decomposition and Goodness of Fit
Variance Decomposition and Goodness of Fit 1. Example: Monthly Earnings and Years of Education In this tutorial, we will focus on an example that explores the relationship between total monthly earnings
14 Multiple Linear Regression
B.Sc./Cert./M.Sc. Qualif. - Statistics: Theory and Practice 14 Multiple Linear Regression 14.1 The multiple linear regression model In simple linear regression, the response variable y is expressed in
STAT420 Midterm Exam. University of Illinois Urbana-Champaign October 19 (Friday), :00 4:15p. SOLUTIONS (Yellow)
STAT40 Midterm Exam University of Illinois Urbana-Champaign October 19 (Friday), 018 3:00 4:15p SOLUTIONS (Yellow) Question 1 (15 points) (10 points) 3 (50 points) extra ( points) Total (77 points) Points
STA 108 Applied Linear Models: Regression Analysis Spring Solution for Homework #6
STA 8 Applied Linear Models: Regression Analysis Spring 011 Solution for Homework #6 6. a) = 11 1 31 41 51 1 3 4 5 11 1 31 41 51 β = β1 β β 3 b) = 1 1 1 1 1 11 1 31 41 51 1 3 4 5 β = β 0 β1 β 6.15 a) Stem-and-leaf
F-tests and Nested Models
F-tests and Nested Models Nested Models: A core concept in statistics is comparing nested s. Consider the Y = β 0 + β 1 x 1 + β 2 x 2 + ǫ. (1) The following reduced s are special cases (nested within)
Lecture 18: Analysis of variance: ANOVA
Lecture 18: Announcements: Exam has been graded. See website for results. Lecture 18: Announcements: Exam has been graded. See website for results. Reading: Vasilj pp. 83-97. Lecture 18: Announcements:
MEI Exam Review. June 7, 2002
MEI Exam Review June 7, 2002 1 Final Exam Revision Notes 1.1 Random Rules and Formulas Linear transformations of random variables. f y (Y ) = f x (X) dx. dg Inverse Proof. (AB)(AB) 1 = I. (B 1 A 1 )(AB)(AB)
ST430 Exam 1 with Answers
ST430 Exam 1 with Answers Date: October 5, 2015 Name: Guideline: You may use one-page (front and back of a standard A4 paper) of notes. No laptop or textook are permitted but you may use a calculator.
Chapter 4. Regression Models. Learning Objectives
Chapter 4 Regression Models To accompany Quantitative Analysis for Management, Eleventh Edition, by Render, Stair, and Hanna Power Point slides created by Brian Peterson Learning Objectives After completing
The Multiple Regression Model
Multiple Regression The Multiple Regression Model Idea: Examine the linear relationship between 1 dependent (Y) & or more independent variables (X i ) Multiple Regression Model with k Independent Variables:
PART I. (a) Describe all the assumptions for a normal error regression model with one predictor variable,
Concordia University Department of Mathematics and Statistics Course Number Section Statistics 360/2 01 Examination Date Time Pages Final December 2002 3 hours 6 Instructors Course Examiner Marks Y.P.
Matrix Approach to Simple Linear Regression: An Overview
Matrix Approach to Simple Linear Regression: An Overview Aspects of matrices that you should know: Definition of a matrix Addition/subtraction/multiplication of matrices Symmetric/diagonal/identity matrix
The linear model is the most fundamental of all serious statistical models encompassing:
Linear Regression Models: A Bayesian perspective Ingredients of a linear model include an n 1 response vector y = (y 1,..., y n ) T and an n p design matrix (e.g. including regressors) X = [x 1,..., x
Chapter 11 - Lecture 1 Single Factor ANOVA
Chapter 11 - Lecture 1 Single Factor ANOVA April 7th, 2010 Means Variance Sum of Squares Review In Chapter 9 we have seen how to make hypothesis testing for one population mean. In Chapter 10 we have seen
Master s Written Examination - Solution
Master s Written Examination - Solution Spring 204 Problem Stat 40 Suppose X and X 2 have the joint pdf f X,X 2 (x, x 2 ) = 2e (x +x 2 ), 0 < x < x 2
Homework 2: Simple Linear Regression
STAT 4385 Applied Regression Analysis Homework : Simple Linear Regression (Simple Linear Regression) Thirty (n = 30) College graduates who have recently entered the job market. For each student, the CGPA
Workshop 7.4a: Single factor ANOVA
-1- Workshop 7.4a: Single factor ANOVA Murray Logan November 23, 2016 Table of contents 1 Revision 1 2 Anova Parameterization 2 3 Partitioning of variance (ANOVA) 10 4 Worked Examples 13 1. Revision 1.1.
Chapter 14. Linear least squares
Serik Sagitov, Chalmers and GU, March 5, 2018 Chapter 14 Linear least squares 1 Simple linear regression model A linear model for the random response Y = Y (x) to an independent variable X = x For a given
Statistics for Managers using Microsoft Excel 6 th Edition
Statistics for Managers using Microsoft Excel 6 th Edition Chapter 13 Simple Linear Regression 13-1 Learning Objectives In this chapter, you learn: How to use regression analysis to predict the value of
Simple Regression Model Setup Estimation Inference Prediction. Model Diagnostic. Multiple Regression. Model Setup and Estimation.
Statistical Computation Math 475 Jimin Ding Department of Mathematics Washington University in St. Louis www.math.wustl.edu/ jmding/math475/index.html October 10, 2013 Ridge Part IV October 10, 2013 1
Master s Written Examination
Master s Written Examination Option: Statistics and Probability Spring 016 Full points may be obtained for correct answers to eight questions. Each numbered question which may have several parts is worth
(ii) Scan your answer sheets INTO ONE FILE only, and submit it in the drop-box.
FINAL EXAM ** Two different ways to submit your answer sheet (i) Use MS-Word and place it in a drop-box. (ii) Scan your answer sheets INTO ONE FILE only, and submit it in the drop-box. Deadline: December
STAT2012 Statistical Tests 23 Regression analysis: method of least squares
23 Regression analysis: method of least squares L23 Regression analysis The main purpose of regression is to explore the dependence of one variable (Y ) on another variable (X). 23.1 Introduction (P.532-555)
Example: Poisondata. 22s:152 Applied Linear Regression. Chapter 8: ANOVA
s:5 Applied Linear Regression Chapter 8: ANOVA Two-way ANOVA Used to compare populations means when the populations are classified by two factors (or categorical variables) For example sex and occupation
INTERVAL ESTIMATION AND HYPOTHESES TESTING
INTERVAL ESTIMATION AND HYPOTHESES TESTING 1. IDEA An interval rather than a point estimate is often of interest. Confidence intervals are thus important in empirical work. To construct interval estimates,
36-707: Regression Analysis Homework Solutions. Homework 3
36-707: Regression Analysis Homework Solutions Homework 3 Fall 2012 Problem 1 Y i = βx i + ɛ i, i {1, 2,..., n}. (a) Find the LS estimator of β: RSS = Σ n i=1(y i βx i ) 2 RSS β = Σ n i=1( 2X i )(Y i βx
I i=1 1 I(J 1) j=1 (Y ij Ȳi ) 2. j=1 (Y j Ȳ )2 ] = 2n( is the two-sample t-test statistic.
![I i=1 1 I(J 1) j=1 (Y ij Ȳi ) 2. j=1 (Y j Ȳ )2 ] = 2n( is the two-sample t-test statistic.](/thumbs/85/92729432.jpg "I i=1 1 I(J 1) j=1 (Y ij Ȳi ) 2. j=1 (Y j Ȳ )2 ] = 2n( is the two-sample t-test statistic.")
Serik Sagitov, Chalmers and GU, February, 08 Solutions chapter Matlab commands: x = data matrix boxplot(x) anova(x) anova(x) Problem.3 Consider one-way ANOVA test statistic For I = and = n, put F = MS
Lectures on Simple Linear Regression Stat 431, Summer 2012
Lectures on Simple Linear Regression Stat 43, Summer 0 Hyunseung Kang July 6-8, 0 Last Updated: July 8, 0 :59PM Introduction Previously, we have been investigating various properties of the population
Basic Business Statistics, 10/e
Chapter 4 4- Basic Business Statistics th Edition Chapter 4 Introduction to Multiple Regression Basic Business Statistics, e 9 Prentice-Hall, Inc. Chap 4- Learning Objectives In this chapter, you learn:
Analysis of Variance Bios 662
Analysis of Variance Bios 662 Michael G. Hudgens, Ph.D. mhudgens@bios.unc.edu http://www.bios.unc.edu/ mhudgens 2008-10-21 13:34 BIOS 662 1 ANOVA Outline Introduction Alternative models SS decomposition
Simple and Multiple Linear Regression
Sta. 113 Chapter 12 and 13 of Devore March 12, 2010 Table of contents 1 Simple Linear Regression 2 Model Simple Linear Regression A simple linear regression model is given by Y = β 0 + β 1 x + ɛ where
Area1 Scaled Score (NAPLEX) .535 ** **.000 N. Sig. (2-tailed)
Institutional Assessment Report Texas Southern University College of Pharmacy and Health Sciences "An Analysis of 2013 NAPLEX, P4-Comp. Exams and P3 courses The following analysis illustrates relationships
Formulary Applied Econometrics
Department of Economics Formulary Applied Econometrics c c Seminar of Statistics University of Fribourg Formulary Applied Econometrics 1 Rescaling With y = cy we have: ˆβ = cˆβ With x = Cx we have: ˆβ
UNIVERSITY OF MASSACHUSETTS. Department of Mathematics and Statistics. Basic Exam - Applied Statistics. Tuesday, January 17, 2017
UNIVERSITY OF MASSACHUSETTS Department of Mathematics and Statistics Basic Exam - Applied Statistics Tuesday, January 17, 2017 Work all problems 60 points are needed to pass at the Masters Level and 75
Statistics 203: Introduction to Regression and Analysis of Variance Course review
Statistics 203: Introduction to Regression and Analysis of Variance Course review Jonathan Taylor - p. 1/?? Today Review / overview of what we learned. - p. 2/?? General themes in regression models Specifying
Foundations of Statistical Inference
Foundations of Statistical Inference Julien Berestycki Department of Statistics University of Oxford MT 2015 Julien Berestycki (University of Oxford) SB2a MT 2015 1 / 16 Lecture 16 : Bayesian analysis
Correlation and the Analysis of Variance Approach to Simple Linear Regression
Correlation and the Analysis of Variance Approach to Simple Linear Regression Biometry 755 Spring 2009 Correlation and the Analysis of Variance Approach to Simple Linear Regression p. 1/35 Correlation
8. Hypothesis Testing
FE661 - Statistical Methods for Financial Engineering 8. Hypothesis Testing Jitkomut Songsiri introduction Wald test likelihood-based tests significance test for linear regression 8-1 Introduction elements
UNIVERSITY OF TORONTO SCARBOROUGH Department of Computer and Mathematical Sciences Midterm Test, October 2013
UNIVERSITY OF TORONTO SCARBOROUGH Department of Computer and Mathematical Sciences Midterm Test, October 2013 STAC67H3 Regression Analysis Duration: One hour and fifty minutes Last Name: First Name: Student
Variance Decomposition in Regression James M. Murray, Ph.D. University of Wisconsin - La Crosse Updated: October 04, 2017
Variance Decomposition in Regression James M. Murray, Ph.D. University of Wisconsin - La Crosse Updated: October 04, 2017 PDF file location: http://www.murraylax.org/rtutorials/regression_anovatable.pdf