Ch 2: Simple Linear Regression
|
|
- Lawrence Lawson
- 5 years ago
- Views:
Transcription
1 Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component with zero mean and unknown variance σ 2. The primary Goal is to do statistical inference on f(x) = β 0 + β 1 x 1. Estimate β 0 and β Do testing H 0 : β 1 = 0. Note that the regressor x is not random variable. But the response y is random variable. With the error assumption, the mean and variance response given x are E(y x) = β 0 + β 1 x and V ar(y x) = σ 2. The true regression line, β 0 + β 1 x is a line of the mean response. The parameters, β 0 (intercept) and β 1 (slope) are called regression coefficients. 2. Least-Squares Estimation Suppose that we observe n pairs of data (x 1, y 1 ),..., (x n, y n ) from the following model y i = β 0 + β 1 x i + ɛ i, i = 1,..., n. The question is how to estimate β 0 and β 1. We will use least squares method. Find β 0 and β 1 to minimize the sum of the squares of the differences between the observations and a linear function. 1
2 y x Least-Squares Estimation of β 0 and β 1 Use least squares (LS) method: estimate β 0 and β 1 to minimize LS estimators, ˆβ0 and ˆβ 1 satisfy S(β 0, β 1 ) = (y i β 0 β 1 x i ) 2 = ε 2 i S β 0 ˆβ0, ˆβ 1 = 2 S β 1 ˆβ0, ˆβ 1 = 2 = Least-squares normal equations ˆβ 0 n ˆβ 0 + ˆβ 1 n x i + ˆβ 1 The solution to the normal equation is where ȳ = n y i /n, x = n x i /n, (y i ˆβ 0 ˆβ 1 x i ) = 0 (y i ˆβ 0 ˆβ 1 x i )x i = 0 n n x i = x 2 i = ˆβ 0 = ȳ ˆβ 1 x ˆβ 1 = S xy, y i y i x i S xy = y i (x i x) and = (x i x) 2 2
3 Define the fitted values and the residuals as ŷ i = ˆβ 0 + ˆβ 1 x i and e i = y i ŷ i, respectively, for i = 1, 2,..., n. Both quantities will be used for model adequacy later. Example (The rocket propellant data, n = 20) Question: statistical relationship between y (Shear strength) and x (Age of propellant). Use a simple linear regression model. Shear strength Age of propellant Example for estimating parameters: To estimate the model parameters, first calculate x = = S xy = Then, we find that ȳ = (x i x) 2 = y i (x i x) = ˆβ 1 = S xy = = ˆβ 0 = ȳ ˆβ 0 x = =
4 The least-squares fit is ŷ = x Shear strength Age of propellant Output from R (the rocket propellant data) Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) < 2e-16 *** x e-10 *** Properties of ˆβ 0 and ˆβ 1 ˆβ 0 and ˆβ 1 are unbiased estimators, that is, E( ˆβ 0 ) = β 0 and E( ˆβ 1 ) = β 1 The variances of ˆβ 0 and ˆβ 1 are ( ) V ar( ˆβ 1 ) = σ2 and V ar( S ˆβ 1 0 ) = σ 2 xx n + x2 4
5 Properties of the least-squares fit, ŷ The sum of the residuals is always zero, that is, (y i ŷ i ) = e i = 0, which results in y i = ŷ i The least squares fit always passes through the point ( x, ȳ) The sum of the residuals weighted by the corresponding x i always equals zero, x i e i = 0. The sum of the residuals weighted by the corresponding fitted values ŷ i always equals zero, ŷ i e i = 0 Estimation of σ 2 An estimation of σ 2 is required to do hypothesis testing on the β 0 and β 1. To that end, decompose the corrected SS of y i into residual (error) SS and model (regression) SS, that is, (y i ȳ) 2 = (y i ŷ i ) 2 + (ŷ i ȳ) (y i ŷ i )(ŷ i ȳ) (y i ȳ) 2 = (y i ŷ i ) 2 + (ŷ i ȳ) 2 SS T = SS E + SS R For the SSE term, SS E = = e 2 i = (y i ŷ i ) 2 yi 2 nȳ 2 ˆβ 1 S xy = SS T ˆβ 1 S xy 5
6 SS E has n 2 degree of freedom As an unbiased estimator of σ 2, use MS E is called the mean square error. ˆσ 2 = SS E n 2 = MS E. ˆσ is called the standard error of regression. Notice that it is a model-dependent estimate of σ 2. For the rocket propellant data, find Then compute SS T = 20 y 2 i 20 ȳ 2 = SS E = SS T ˆβ 1 S xy = ( 37.15) ( ) = Finally, the estimate of σ 2 is ˆσ 2 = SS E n 2 = = Alternative form of the model Consider an alternative model by rewriting the original model as y i = (β 0 + β 1 x) + β 1 (x i x) + ɛ i = β 0 + β 1 (x i x) + ɛ i which is a shifted version of the original model by x. ˆβ 0 = ȳ and ˆβ 1 = S xy /. The least squares estimators are uncorrelated, that is, Cov( ˆβ 0, ˆβ 1 ) = 0. 6
7 3. Hypothesis testing on the slope Now we need the assumption ε i.i.d N(0, σ 2 ). Wish to test the hypothesis that the slope is 0 (that is, the significance of regression) H 0 : β 1 = 0 H 1 : β 1 0 ˆβ 1 is distributed in N(β 1, σ 2 / ). In case that σ 2 is known, use the statistic Z 0 = ˆβ 1 0 σ 2 /, which is distributed N(0, 1) under the null hypothesis is true. t-tests Typically, σ 2 is unknown Use t-test t 0 = ˆβ 1 0 se( ˆβ 1 ), which is distributed t-distribution with d.f. n 2 under the null below. Here se( ˆβ 1 ) = MS E / is called the standard error of the slope H 0 : β 1 = 0 H 1 : β 1 0. Reject the null hypothesis if t 0 > t α/2,n 2 If we fail to reject H 0 : β 1 = 0, there is no linear relationship between x and y Example (The rocket propellant data): 7
8 With ˆβ 1 = 37.15, MS E = and = , The test statistic is t 0 = / = Under that α = 0.05, t 0.025,18 = Since t 0 > 2.101, we would reject H 0. Example with n = 20: With ˆβ 1 0, MS E = and = , The test statistic is t 0 = / = 0 Under that α = 0.05, t 0.025,18 = Since t 0 < 2.101, we fail to reject H 0. new.y x Output from a Statistical Software Many statistical softwares use P-value approach for decision making. P-value is the probability (when H 0 is true) that t 0 takes a value as extreme or more extreme than the actually observed value. 8
9 Decision rule: if P-value α, reject H 0. Output from R (the rocket propellant data) Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) < 2e-16 *** x e-10 *** = Since p-value 0(< 0.05), reject H 0. The Analysis of Variance (ANOVA) An approach to test significance of regression (that is, test H 0 : β 1 = 0). ANOVA provides the same result as that of t-test. But it will be very useful for multiple regression model. ANOVA is based on a partition of total variability in the response variable y. Consider the following partition y i ȳ = (y i ŷ i ) + (ŷ i ȳ) Squaring the both sides and summing (y i ȳ) 2 = (y i ŷ i ) 2 + (ŷ i ȳ) 2 SS T = SS E + SS R Note that the cross-product term is equal to 0 and SS R = ˆβ 1 S xy. Degree of freedom (df) df SST = df SSR + df SSE (n 1) = 1 + (n 2) 9
10 Do F-test for testing H 0 : β 1 = 0 F 0 = SS R /1 SS E /(n 2) = MS R MS E where F 0 follows the F 1,n 2 distribution under the H 0. Decision rule is that reject H 0 if F 0 > F α,1,n 2 The benefit to use the analysis of variance is in Multiple regression Example (The rocket propellant data): To obtain the statistic, F 0, first compute SS T = (y i ȳ) 2 = i 1 SS R = ˆβ 1 S xy = ( 37.15) ( ) = And then find SS E = SS T SS R = Finally, compute Reject H 0 because F 0 > F 0.05,1,18 = 8.29 Output from R (P-value approach) F 0 = / /18 = F-statistic: on 1 and 18 DF, p-value: 1.643e Interval Estimation Confidence intervals on β 0 and β 1 Under the assumption that the errors are normally and independently distributed, both ˆβ 1 β 1 se( ˆβ 1 ) and ˆβ 0 β 0 se( ˆβ 0 ) follow t distribution with n 2 degrees of freedom. 10
11 Therefore, a 100(1 α) percent confidence interval on β 1 is ˆβ 1 t α/2,n 2 se( ˆβ 1 ) β 1 ˆβ 1 + t α/2,n 2 se( ˆβ 1 ) and a 100(1 α) percent confidence interval on β 0 is ˆβ 0 t α/2,n 2 se( ˆβ 0 ) β 0 ˆβ 0 + t α/2,n 2 se( ˆβ 0 ), where ( ) se( ˆβ 1 0 ) = MS E n + x2 and se( ˆβ 1 ) = MSE Example (The rocket propellant data): Construct 95% confidence intervals on β 0 and β 1 With se( ˆβ 0 ) = , se( ˆβ 1 ) = 2.89 and t 0.025,18 = 2.101, ˆβ 0 t 0.025,18 se( ˆβ 0 ) β 0 ˆβ 0 + t 0.025,18 se( ˆβ 0 ) (2.101)(44.184) β (2.101)(44.184) β ˆβ 1 t 0.025,18 se( ˆβ 1 ) β 1 ˆβ 1 + t 0.025,18 se( ˆβ 1 ) (2.101)(2.89) β (2.101)(2.89) β Confidence intervals on σ 2 If the errors ɛ i are normally and independently distributed, it can be shown that (n 2)MS E σ 2 χ 2 (n 2). Thus, a 100(1 α) percent confidence interval on σ 2 is (n 2)MS E χ 2 α/2,n 2 σ 2 (n 2)MS E. χ 2 1 α/2,n 2 11
12 Interval estimation of the mean response A major use of a regression model is to estimate mean response E(y) for a particular value of x. Want to estimate mean response E(y x 0 ), where x 0 is any value of regressor variable with in the range of the original data on x to used to fit the model Use the estimates E(y x 0 ) = ˆµ y x0 = ˆβ 0 + ˆβ 1 x 0 ˆµ y x0 are normally distributed with mean E(y x 0 ) and variance ( 1 σ 2 n + (x 0 x) 2 ) The sampling distribution of ˆµ y x0 E(y x 0 ) ( MS 1 E + ) (x 0 x) 2 n is t distribution with n 2 degrees of freedom. Therefore, a 100(1 α) percent confidence interval on the mean response at the point x = x 0 is ˆµ y x0 t α/2,n 2 A E(y x 0 ) ˆµ y x0 + t α/2,n 2 A, where ( 1 A = MS E n + (x ) 0 x) 2 For the rocket propellant data, a 95% confidence interval on E(y x 0 ) is ˆµ y x0 (2.101)A(x 0 ) E(y x 0 ) ˆµ y x0 + (2.101)A(x 0 ), where ( 1 A(x 0 ) = (x ) ) For example, if x 0 = , then the confidence interval is E(y )
13 Prediction of new observation Prediction is an important application of the regression model If x 0 is the value of the regressor variable of interest, then ŷ 0 = ˆβ 0 + ˆβ 1 x 0 is the point prediction of the future observation y 0. The variability of ŷ 0 y 0 is E(ŷ 0 y 0 ) 2 = E{[ŷ 0 E(y 0 )] [y 0 E(y 0 )]} 2 = E[ŷ 0 E(y 0 )] 2 + E[y 0 E(y 0 )] 2 ( 1 = σ 2 n + (x 0 x) 2 ) + σ 2 Therefore, a 100(1 α) percent prediction interval on a future observation at the point x = x 0 is ŷ 0 t α/2,n 2 B y 0 ŷ 0 + t α/2,n 2 B, where ( B = MS E n + (x ) 0 x) 2 For the rocket propellant data, a 95% prediction interval on y 0 is ŷ 0 (2.101)B(x 0 ) y 0 ŷ 0 + (2.101)B(x 0 ), where ( B(x 0 ) = (x ) ) If x 0 = 10, then a 95% prediction interval is [ , ] 13
14 Coefficient of determination R 2 The coefficient of determination is defined as R 2 = SS R SS T (0 R 2 1) R 2 is the proportion of variation explained by the regressor x R 2 1 implies that most of variability in y is explained by the regression model For the rocket propellant data, R 2 = SS R SS T = = that is, 90.18% of the variability in strength is explained by the regression model Relationship to the correlation coefficient R 2 = SS R SS T = ˆβ 1 S xy S yy = S xys xy S yy 2 S = xy Sxx Syy = r 2 xy = the square of the sample correlation between x and y For the rocket propellant data, R 2 = (r xy ) 2 = ( ) 2 =
Ch 3: Multiple Linear Regression
Ch 3: Multiple Linear Regression 1. Multiple Linear Regression Model Multiple regression model has more than one regressor. For example, we have one response variable and two regressor variables: 1. delivery
More informationSimple Linear Regression
Simple Linear Regression In simple linear regression we are concerned about the relationship between two variables, X and Y. There are two components to such a relationship. 1. The strength of the relationship.
More informationSimple Linear Regression
Simple Linear Regression ST 430/514 Recall: A regression model describes how a dependent variable (or response) Y is affected, on average, by one or more independent variables (or factors, or covariates)
More informationMultiple Linear Regression
Multiple Linear Regression Simple linear regression tries to fit a simple line between two variables Y and X. If X is linearly related to Y this explains some of the variability in Y. In most cases, there
More informationInference for Regression
Inference for Regression Section 9.4 Cathy Poliak, Ph.D. cathy@math.uh.edu Office in Fleming 11c Department of Mathematics University of Houston Lecture 13b - 3339 Cathy Poliak, Ph.D. cathy@math.uh.edu
More informationLinear models and their mathematical foundations: Simple linear regression
Linear models and their mathematical foundations: Simple linear regression Steffen Unkel Department of Medical Statistics University Medical Center Göttingen, Germany Winter term 2018/19 1/21 Introduction
More informationMeasuring the fit of the model - SSR
Measuring the fit of the model - SSR Once we ve determined our estimated regression line, we d like to know how well the model fits. How far/close are the observations to the fitted line? One way to do
More informationSimple and Multiple Linear Regression
Sta. 113 Chapter 12 and 13 of Devore March 12, 2010 Table of contents 1 Simple Linear Regression 2 Model Simple Linear Regression A simple linear regression model is given by Y = β 0 + β 1 x + ɛ where
More informationEstimating σ 2. We can do simple prediction of Y and estimation of the mean of Y at any value of X.
Estimating σ 2 We can do simple prediction of Y and estimation of the mean of Y at any value of X. To perform inferences about our regression line, we must estimate σ 2, the variance of the error term.
More informationMAT2377. Rafa l Kulik. Version 2015/November/26. Rafa l Kulik
MAT2377 Rafa l Kulik Version 2015/November/26 Rafa l Kulik Bivariate data and scatterplot Data: Hydrocarbon level (x) and Oxygen level (y): x: 0.99, 1.02, 1.15, 1.29, 1.46, 1.36, 0.87, 1.23, 1.55, 1.40,
More informationSimple Linear Regression
Simple Linear Regression MATH 282A Introduction to Computational Statistics University of California, San Diego Instructor: Ery Arias-Castro http://math.ucsd.edu/ eariasca/math282a.html MATH 282A University
More informationAMS 315/576 Lecture Notes. Chapter 11. Simple Linear Regression
AMS 315/576 Lecture Notes Chapter 11. Simple Linear Regression 11.1 Motivation A restaurant opening on a reservations-only basis would like to use the number of advance reservations x to predict the number
More informationProblems. Suppose both models are fitted to the same data. Show that SS Res, A SS Res, B
Simple Linear Regression 35 Problems 1 Consider a set of data (x i, y i ), i =1, 2,,n, and the following two regression models: y i = β 0 + β 1 x i + ε, (i =1, 2,,n), Model A y i = γ 0 + γ 1 x i + γ 2
More informationCorrelation Analysis
Simple Regression Correlation Analysis Correlation analysis is used to measure strength of the association (linear relationship) between two variables Correlation is only concerned with strength of the
More informationApplied Regression Analysis
Applied Regression Analysis Chapter 3 Multiple Linear Regression Hongcheng Li April, 6, 2013 Recall simple linear regression 1 Recall simple linear regression 2 Parameter Estimation 3 Interpretations of
More informationLecture 14 Simple Linear Regression
Lecture 4 Simple Linear Regression Ordinary Least Squares (OLS) Consider the following simple linear regression model where, for each unit i, Y i is the dependent variable (response). X i is the independent
More informationLecture 6 Multiple Linear Regression, cont.
Lecture 6 Multiple Linear Regression, cont. BIOST 515 January 22, 2004 BIOST 515, Lecture 6 Testing general linear hypotheses Suppose we are interested in testing linear combinations of the regression
More informationMath 3330: Solution to midterm Exam
Math 3330: Solution to midterm Exam Question 1: (14 marks) Suppose the regression model is y i = β 0 + β 1 x i + ε i, i = 1,, n, where ε i are iid Normal distribution N(0, σ 2 ). a. (2 marks) Compute the
More informationREGRESSION ANALYSIS AND INDICATOR VARIABLES
REGRESSION ANALYSIS AND INDICATOR VARIABLES Thesis Submitted in partial fulfillment of the requirements for the award of degree of Masters of Science in Mathematics and Computing Submitted by Sweety Arora
More informationLecture 15. Hypothesis testing in the linear model
14. Lecture 15. Hypothesis testing in the linear model Lecture 15. Hypothesis testing in the linear model 1 (1 1) Preliminary lemma 15. Hypothesis testing in the linear model 15.1. Preliminary lemma Lemma
More informationCoefficient of Determination
Coefficient of Determination ST 430/514 The coefficient of determination, R 2, is defined as before: R 2 = 1 SS E (yi ŷ i ) = 1 2 SS yy (yi ȳ) 2 The interpretation of R 2 is still the fraction of variance
More informationLinear regression. We have that the estimated mean in linear regression is. ˆµ Y X=x = ˆβ 0 + ˆβ 1 x. The standard error of ˆµ Y X=x is.
Linear regression We have that the estimated mean in linear regression is The standard error of ˆµ Y X=x is where x = 1 n s.e.(ˆµ Y X=x ) = σ ˆµ Y X=x = ˆβ 0 + ˆβ 1 x. 1 n + (x x)2 i (x i x) 2 i x i. The
More informationLinear Models and Estimation by Least Squares
Linear Models and Estimation by Least Squares Jin-Lung Lin 1 Introduction Causal relation investigation lies in the heart of economics. Effect (Dependent variable) cause (Independent variable) Example:
More informationSTAT 511. Lecture : Simple linear regression Devore: Section Prof. Michael Levine. December 3, Levine STAT 511
STAT 511 Lecture : Simple linear regression Devore: Section 12.1-12.4 Prof. Michael Levine December 3, 2018 A simple linear regression investigates the relationship between the two variables that is not
More informationBusiness Statistics. Chapter 14 Introduction to Linear Regression and Correlation Analysis QMIS 220. Dr. Mohammad Zainal
Department of Quantitative Methods & Information Systems Business Statistics Chapter 14 Introduction to Linear Regression and Correlation Analysis QMIS 220 Dr. Mohammad Zainal Chapter Goals After completing
More informationSimple Linear Regression
Simple Linear Regression September 24, 2008 Reading HH 8, GIll 4 Simple Linear Regression p.1/20 Problem Data: Observe pairs (Y i,x i ),i = 1,...n Response or dependent variable Y Predictor or independent
More informationChapter 14 Simple Linear Regression (A)
Chapter 14 Simple Linear Regression (A) 1. Characteristics Managerial decisions often are based on the relationship between two or more variables. can be used to develop an equation showing how the variables
More informationConcordia University (5+5)Q 1.
(5+5)Q 1. Concordia University Department of Mathematics and Statistics Course Number Section Statistics 360/1 40 Examination Date Time Pages Mid Term Test May 26, 2004 Two Hours 3 Instructor Course Examiner
More informationLinear Regression. Simple linear regression model determines the relationship between one dependent variable (y) and one independent variable (x).
Linear Regression Simple linear regression model determines the relationship between one dependent variable (y) and one independent variable (x). A dependent variable is a random variable whose variation
More informationChapter 12 - Lecture 2 Inferences about regression coefficient
Chapter 12 - Lecture 2 Inferences about regression coefficient April 19th, 2010 Facts about slope Test Statistic Confidence interval Hypothesis testing Test using ANOVA Table Facts about slope In previous
More informationLinear Regression Model. Badr Missaoui
Linear Regression Model Badr Missaoui Introduction What is this course about? It is a course on applied statistics. It comprises 2 hours lectures each week and 1 hour lab sessions/tutorials. We will focus
More informationSimple Linear Regression
Simple Linear Regression Reading: Hoff Chapter 9 November 4, 2009 Problem Data: Observe pairs (Y i,x i ),i = 1,... n Response or dependent variable Y Predictor or independent variable X GOALS: Exploring
More informationSTAT420 Midterm Exam. University of Illinois Urbana-Champaign October 19 (Friday), :00 4:15p. SOLUTIONS (Yellow)
STAT40 Midterm Exam University of Illinois Urbana-Champaign October 19 (Friday), 018 3:00 4:15p SOLUTIONS (Yellow) Question 1 (15 points) (10 points) 3 (50 points) extra ( points) Total (77 points) Points
More informationMultiple Linear Regression
Multiple Linear Regression ST 430/514 Recall: a regression model describes how a dependent variable (or response) Y is affected, on average, by one or more independent variables (or factors, or covariates).
More information13 Simple Linear Regression
B.Sc./Cert./M.Sc. Qualif. - Statistics: Theory and Practice 3 Simple Linear Regression 3. An industrial example A study was undertaken to determine the effect of stirring rate on the amount of impurity
More informationInference for Regression Simple Linear Regression
Inference for Regression Simple Linear Regression IPS Chapter 10.1 2009 W.H. Freeman and Company Objectives (IPS Chapter 10.1) Simple linear regression p Statistical model for linear regression p Estimating
More informationSummary of Chapter 7 (Sections ) and Chapter 8 (Section 8.1)
Summary of Chapter 7 (Sections 7.2-7.5) and Chapter 8 (Section 8.1) Chapter 7. Tests of Statistical Hypotheses 7.2. Tests about One Mean (1) Test about One Mean Case 1: σ is known. Assume that X N(µ, σ
More informationHomework 2: Simple Linear Regression
STAT 4385 Applied Regression Analysis Homework : Simple Linear Regression (Simple Linear Regression) Thirty (n = 30) College graduates who have recently entered the job market. For each student, the CGPA
More informationLECTURE 6. Introduction to Econometrics. Hypothesis testing & Goodness of fit
LECTURE 6 Introduction to Econometrics Hypothesis testing & Goodness of fit October 25, 2016 1 / 23 ON TODAY S LECTURE We will explain how multiple hypotheses are tested in a regression model We will define
More informationSimple linear regression
Simple linear regression Biometry 755 Spring 2008 Simple linear regression p. 1/40 Overview of regression analysis Evaluate relationship between one or more independent variables (X 1,...,X k ) and a single
More informationChapter 3: Multiple Regression. August 14, 2018
Chapter 3: Multiple Regression August 14, 2018 1 The multiple linear regression model The model y = β 0 +β 1 x 1 + +β k x k +ǫ (1) is called a multiple linear regression model with k regressors. The parametersβ
More informationInferences for Regression
Inferences for Regression An Example: Body Fat and Waist Size Looking at the relationship between % body fat and waist size (in inches). Here is a scatterplot of our data set: Remembering Regression In
More informationSTAT2012 Statistical Tests 23 Regression analysis: method of least squares
23 Regression analysis: method of least squares L23 Regression analysis The main purpose of regression is to explore the dependence of one variable (Y ) on another variable (X). 23.1 Introduction (P.532-555)
More informationSTAT 540: Data Analysis and Regression
STAT 540: Data Analysis and Regression Wen Zhou http://www.stat.colostate.edu/~riczw/ Email: riczw@stat.colostate.edu Department of Statistics Colorado State University Fall 205 W. Zhou (Colorado State
More informationTable 1: Fish Biomass data set on 26 streams
Math 221: Multiple Regression S. K. Hyde Chapter 27 (Moore, 5th Ed.) The following data set contains observations on the fish biomass of 26 streams. The potential regressors from which we wish to explain
More informationStatistics for Managers using Microsoft Excel 6 th Edition
Statistics for Managers using Microsoft Excel 6 th Edition Chapter 13 Simple Linear Regression 13-1 Learning Objectives In this chapter, you learn: How to use regression analysis to predict the value of
More informationBasic Business Statistics 6 th Edition
Basic Business Statistics 6 th Edition Chapter 12 Simple Linear Regression Learning Objectives In this chapter, you learn: How to use regression analysis to predict the value of a dependent variable based
More informationDensity Temp vs Ratio. temp
Temp Ratio Density 0.00 0.02 0.04 0.06 0.08 0.10 0.12 Density 0.0 0.2 0.4 0.6 0.8 1.0 1. (a) 170 175 180 185 temp 1.0 1.5 2.0 2.5 3.0 ratio The histogram shows that the temperature measures have two peaks,
More informationwhere x and ȳ are the sample means of x 1,, x n
y y Animal Studies of Side Effects Simple Linear Regression Basic Ideas In simple linear regression there is an approximately linear relation between two variables say y = pressure in the pancreas x =
More informationApplied Econometrics (QEM)
Applied Econometrics (QEM) based on Prinicples of Econometrics Jakub Mućk Department of Quantitative Economics Jakub Mućk Applied Econometrics (QEM) Meeting #3 1 / 42 Outline 1 2 3 t-test P-value Linear
More informationUNIVERSITY OF TORONTO SCARBOROUGH Department of Computer and Mathematical Sciences Midterm Test, October 2013
UNIVERSITY OF TORONTO SCARBOROUGH Department of Computer and Mathematical Sciences Midterm Test, October 2013 STAC67H3 Regression Analysis Duration: One hour and fifty minutes Last Name: First Name: Student
More informationLecture 11: Simple Linear Regression
Lecture 11: Simple Linear Regression Readings: Sections 3.1-3.3, 11.1-11.3 Apr 17, 2009 In linear regression, we examine the association between two quantitative variables. Number of beers that you drink
More informationFigure 1: The fitted line using the shipment route-number of ampules data. STAT5044: Regression and ANOVA The Solution of Homework #2 Inyoung Kim
0.0 1.0 1.5 2.0 2.5 3.0 8 10 12 14 16 18 20 22 y x Figure 1: The fitted line using the shipment route-number of ampules data STAT5044: Regression and ANOVA The Solution of Homework #2 Inyoung Kim Problem#
More informationApplied Regression. Applied Regression. Chapter 2 Simple Linear Regression. Hongcheng Li. April, 6, 2013
Applied Regression Chapter 2 Simple Linear Regression Hongcheng Li April, 6, 2013 Outline 1 Introduction of simple linear regression 2 Scatter plot 3 Simple linear regression model 4 Test of Hypothesis
More informationStatistics for Engineers Lecture 9 Linear Regression
Statistics for Engineers Lecture 9 Linear Regression Chong Ma Department of Statistics University of South Carolina chongm@email.sc.edu April 17, 2017 Chong Ma (Statistics, USC) STAT 509 Spring 2017 April
More informationOverview Scatter Plot Example
Overview Topic 22 - Linear Regression and Correlation STAT 5 Professor Bruce Craig Consider one population but two variables For each sampling unit observe X and Y Assume linear relationship between variables
More informationOct Simple linear regression. Minimum mean square error prediction. Univariate. regression. Calculating intercept and slope
Oct 2017 1 / 28 Minimum MSE Y is the response variable, X the predictor variable, E(X) = E(Y) = 0. BLUP of Y minimizes average discrepancy var (Y ux) = C YY 2u C XY + u 2 C XX This is minimized when u
More informationBNAD 276 Lecture 10 Simple Linear Regression Model
1 / 27 BNAD 276 Lecture 10 Simple Linear Regression Model Phuong Ho May 30, 2017 2 / 27 Outline 1 Introduction 2 3 / 27 Outline 1 Introduction 2 4 / 27 Simple Linear Regression Model Managerial decisions
More informationLecture 10 Multiple Linear Regression
Lecture 10 Multiple Linear Regression STAT 512 Spring 2011 Background Reading KNNL: 6.1-6.5 10-1 Topic Overview Multiple Linear Regression Model 10-2 Data for Multiple Regression Y i is the response variable
More informationFormal Statement of Simple Linear Regression Model
Formal Statement of Simple Linear Regression Model Y i = β 0 + β 1 X i + ɛ i Y i value of the response variable in the i th trial β 0 and β 1 are parameters X i is a known constant, the value of the predictor
More informationSTAT5044: Regression and Anova. Inyoung Kim
STAT5044: Regression and Anova Inyoung Kim 2 / 47 Outline 1 Regression 2 Simple Linear regression 3 Basic concepts in regression 4 How to estimate unknown parameters 5 Properties of Least Squares Estimators:
More informationSTAT Chapter 11: Regression
STAT 515 -- Chapter 11: Regression Mostly we have studied the behavior of a single random variable. Often, however, we gather data on two random variables. We wish to determine: Is there a relationship
More informationInference in Regression Analysis
Inference in Regression Analysis Dr. Frank Wood Frank Wood, fwood@stat.columbia.edu Linear Regression Models Lecture 4, Slide 1 Today: Normal Error Regression Model Y i = β 0 + β 1 X i + ǫ i Y i value
More informationMa 3/103: Lecture 25 Linear Regression II: Hypothesis Testing and ANOVA
Ma 3/103: Lecture 25 Linear Regression II: Hypothesis Testing and ANOVA March 6, 2017 KC Border Linear Regression II March 6, 2017 1 / 44 1 OLS estimator 2 Restricted regression 3 Errors in variables 4
More information22s:152 Applied Linear Regression. Take random samples from each of m populations.
22s:152 Applied Linear Regression Chapter 8: ANOVA NOTE: We will meet in the lab on Monday October 10. One-way ANOVA Focuses on testing for differences among group means. Take random samples from each
More informationST430 Exam 1 with Answers
ST430 Exam 1 with Answers Date: October 5, 2015 Name: Guideline: You may use one-page (front and back of a standard A4 paper) of notes. No laptop or textook are permitted but you may use a calculator.
More informationEstadística II Chapter 4: Simple linear regression
Estadística II Chapter 4: Simple linear regression Chapter 4. Simple linear regression Contents Objectives of the analysis. Model specification. Least Square Estimators (LSE): construction and properties
More informationMathematics for Economics MA course
Mathematics for Economics MA course Simple Linear Regression Dr. Seetha Bandara Simple Regression Simple linear regression is a statistical method that allows us to summarize and study relationships between
More informationECO220Y Simple Regression: Testing the Slope
ECO220Y Simple Regression: Testing the Slope Readings: Chapter 18 (Sections 18.3-18.5) Winter 2012 Lecture 19 (Winter 2012) Simple Regression Lecture 19 1 / 32 Simple Regression Model y i = β 0 + β 1 x
More informationSTAT 4385 Topic 03: Simple Linear Regression
STAT 4385 Topic 03: Simple Linear Regression Xiaogang Su, Ph.D. Department of Mathematical Science University of Texas at El Paso xsu@utep.edu Spring, 2017 Outline The Set-Up Exploratory Data Analysis
More informationSTAT 350 Final (new Material) Review Problems Key Spring 2016
1. The editor of a statistics textbook would like to plan for the next edition. A key variable is the number of pages that will be in the final version. Text files are prepared by the authors using LaTeX,
More informationInference for the Regression Coefficient
Inference for the Regression Coefficient Recall, b 0 and b 1 are the estimates of the slope β 1 and intercept β 0 of population regression line. We can shows that b 0 and b 1 are the unbiased estimates
More informationLectures on Simple Linear Regression Stat 431, Summer 2012
Lectures on Simple Linear Regression Stat 43, Summer 0 Hyunseung Kang July 6-8, 0 Last Updated: July 8, 0 :59PM Introduction Previously, we have been investigating various properties of the population
More informationSimple Linear Regression
Simple Linear Regression ST 370 Regression models are used to study the relationship of a response variable and one or more predictors. The response is also called the dependent variable, and the predictors
More informationMath 423/533: The Main Theoretical Topics
Math 423/533: The Main Theoretical Topics Notation sample size n, data index i number of predictors, p (p = 2 for simple linear regression) y i : response for individual i x i = (x i1,..., x ip ) (1 p)
More informationPART I. (a) Describe all the assumptions for a normal error regression model with one predictor variable,
Concordia University Department of Mathematics and Statistics Course Number Section Statistics 360/2 01 Examination Date Time Pages Final December 2002 3 hours 6 Instructors Course Examiner Marks Y.P.
More informationMA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7
MA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7 1 Random Vectors Let a 0 and y be n 1 vectors, and let A be an n n matrix. Here, a 0 and A are non-random, whereas y is
More information(ii) Scan your answer sheets INTO ONE FILE only, and submit it in the drop-box.
FINAL EXAM ** Two different ways to submit your answer sheet (i) Use MS-Word and place it in a drop-box. (ii) Scan your answer sheets INTO ONE FILE only, and submit it in the drop-box. Deadline: December
More informationy ˆ i = ˆ " T u i ( i th fitted value or i th fit)
1 2 INFERENCE FOR MULTIPLE LINEAR REGRESSION Recall Terminology: p predictors x 1, x 2,, x p Some might be indicator variables for categorical variables) k-1 non-constant terms u 1, u 2,, u k-1 Each u
More informationMATH 644: Regression Analysis Methods
MATH 644: Regression Analysis Methods FINAL EXAM Fall, 2012 INSTRUCTIONS TO STUDENTS: 1. This test contains SIX questions. It comprises ELEVEN printed pages. 2. Answer ALL questions for a total of 100
More informationChapter 1: Linear Regression with One Predictor Variable also known as: Simple Linear Regression Bivariate Linear Regression
BSTT523: Kutner et al., Chapter 1 1 Chapter 1: Linear Regression with One Predictor Variable also known as: Simple Linear Regression Bivariate Linear Regression Introduction: Functional relation between
More informationCategorical Predictor Variables
Categorical Predictor Variables We often wish to use categorical (or qualitative) variables as covariates in a regression model. For binary variables (taking on only 2 values, e.g. sex), it is relatively
More informationHypothesis Testing hypothesis testing approach
Hypothesis Testing In this case, we d be trying to form an inference about that neighborhood: Do people there shop more often those people who are members of the larger population To ascertain this, we
More informationSimple Linear Regression Analysis
LINEAR REGRESSION ANALYSIS MODULE II Lecture - 6 Simple Linear Regression Analysis Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur Prediction of values of study
More informationLecture 3: Inference in SLR
Lecture 3: Inference in SLR STAT 51 Spring 011 Background Reading KNNL:.1.6 3-1 Topic Overview This topic will cover: Review of hypothesis testing Inference about 1 Inference about 0 Confidence Intervals
More informationECON The Simple Regression Model
ECON 351 - The Simple Regression Model Maggie Jones 1 / 41 The Simple Regression Model Our starting point will be the simple regression model where we look at the relationship between two variables In
More informationRegression Analysis. Regression: Methodology for studying the relationship among two or more variables
Regression Analysis Regression: Methodology for studying the relationship among two or more variables Two major aims: Determine an appropriate model for the relationship between the variables Predict the
More informationChapter 14 Student Lecture Notes Department of Quantitative Methods & Information Systems. Business Statistics. Chapter 14 Multiple Regression
Chapter 14 Student Lecture Notes 14-1 Department of Quantitative Methods & Information Systems Business Statistics Chapter 14 Multiple Regression QMIS 0 Dr. Mohammad Zainal Chapter Goals After completing
More information14 Multiple Linear Regression
B.Sc./Cert./M.Sc. Qualif. - Statistics: Theory and Practice 14 Multiple Linear Regression 14.1 The multiple linear regression model In simple linear regression, the response variable y is expressed in
More informationSIMPLE REGRESSION ANALYSIS. Business Statistics
SIMPLE REGRESSION ANALYSIS Business Statistics CONTENTS Ordinary least squares (recap for some) Statistical formulation of the regression model Assessing the regression model Testing the regression coefficients
More informationThe Multiple Regression Model
Multiple Regression The Multiple Regression Model Idea: Examine the linear relationship between 1 dependent (Y) & or more independent variables (X i ) Multiple Regression Model with k Independent Variables:
More informationLinear Regression. In this problem sheet, we consider the problem of linear regression with p predictors and one intercept,
Linear Regression In this problem sheet, we consider the problem of linear regression with p predictors and one intercept, y = Xβ + ɛ, where y t = (y 1,..., y n ) is the column vector of target values,
More informationFrom last time... The equations
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this
More informationChapter 1. Linear Regression with One Predictor Variable
Chapter 1. Linear Regression with One Predictor Variable 1.1 Statistical Relation Between Two Variables To motivate statistical relationships, let us consider a mathematical relation between two mathematical
More informationChapter 14. Linear least squares
Serik Sagitov, Chalmers and GU, March 5, 2018 Chapter 14 Linear least squares 1 Simple linear regression model A linear model for the random response Y = Y (x) to an independent variable X = x For a given
More informationChapter 2. Continued. Proofs For ANOVA Proof of ANOVA Identity. the product term in the above equation can be simplified as n
Chapter 2. Continued Proofs For ANOVA Proof of ANOVA Identity We are going to prove that Writing SST SSR + SSE. Y i Ȳ (Y i Ŷ i ) + (Ŷ i Ȳ ) Squaring both sides summing over all i 1,...n, we get (Y i Ȳ
More informationA discussion on multiple regression models
A discussion on multiple regression models In our previous discussion of simple linear regression, we focused on a model in which one independent or explanatory variable X was used to predict the value
More information20.1. Balanced One-Way Classification Cell means parametrization: ε 1. ε I. + ˆɛ 2 ij =
20. ONE-WAY ANALYSIS OF VARIANCE 1 20.1. Balanced One-Way Classification Cell means parametrization: Y ij = µ i + ε ij, i = 1,..., I; j = 1,..., J, ε ij N(0, σ 2 ), In matrix form, Y = Xβ + ε, or 1 Y J
More informationSection 3: Simple Linear Regression
Section 3: Simple Linear Regression Carlos M. Carvalho The University of Texas at Austin McCombs School of Business http://faculty.mccombs.utexas.edu/carlos.carvalho/teaching/ 1 Regression: General Introduction
More information6. Multiple Linear Regression
6. Multiple Linear Regression SLR: 1 predictor X, MLR: more than 1 predictor Example data set: Y i = #points scored by UF football team in game i X i1 = #games won by opponent in their last 10 games X
More information