Applied Regression Analysis

Size: px
Start display at page:

Download "Applied Regression Analysis"

Transcription

1 Applied Regression Analysis Lecture 2 January 27, 2005 Lecture #2-1/27/2005 Slide 1 of 46

2 Today s Lecture Simple linear regression. Partitioning the sum of squares. Tests of significance.. Regression diagnostics (introduction). hen X is random. How to do today s topics in. rapping Up Lecture #2-1/27/2005 Slide 2 of 46

3 Previously on EPSY Recall from our first meeting the hand calculated sample problem with the following scatterplot: Small Example The Basics The Basics Estimation Example Properties rapping Up r XY = Correlation shows the degree of association between X and Y......but does so without scale. hat if we wanted to predict Y given a value of X? Lecture #2-1/27/2005 Slide 3 of 46

4 The Basics Small Example The Basics The Basics Estimation Example Properties rapping Up Assume (for now) X is fixed at pre-determined levels in an experiment - independent variable. For example, we have an experiment where subjects are given X cups of coffee. Jon the Mad Scientist decides that subjects should be randomly assigned to a group drinking either 1, 2, 3, 4,or 5 cups of coffee. The we want to estimate the linear effect of the independent variable X on the dependent variable Y. For our example, we want to see how coffee drinking affects blood pressure. Blood pressure = Y = dependent variable. Lecture #2-1/27/2005 Slide 4 of 46

5 The Basics Small Example The Basics The Basics Estimation Example Properties The linear regression model (for observation i = 1,...,N): Y i = α + βx i + ǫ i rapping Up Don t be confused by the Greek alphabet, this is simply the equation for a line (y = a + bx). α is the mean of the population when X is zero...the Y intercept. β is the slope of the line, the amount of increase in Y brought about by a unit increase (X = X + 1) in X. ǫ i is the random error, specific to each observation. Lecture #2-1/27/2005 Slide 5 of 46

6 Parameter Estimates Small Example The Basics The Basics Estimation Example Properties Chapter 2 parameterizes the linear regression model as: Y = a + bx + e To find estimates for a and b consider several possible choices: N So that e is minimized. So that i N e 2 is minimized. i From some guy in the hallway. rapping Up Lecture #2-1/27/2005 Slide 6 of 46

7 And The inner Is... Small Example The Basics The Basics Estimation Example Properties Finding a and b that minimize: N i e 2 Using calculus, these happen to be: b = (X X)(Y Ȳ ) (X X) 2 = r xy s y s x = xy x 2 And: rapping Up a = Ȳ b X LS estimates are considered BLUE: Best Linear Unbiased Estimators. Lecture #2-1/27/2005 Slide 7 of 46

8 An Example of Small Example The Basics The Basics Estimation Example Properties Table 2.1 in Pedhazur lists data from an experiment where X was the number of hours given for study, and Y is the score on a test Test Score 6.00 rapping Up Hours of Study Lecture #2-1/27/2005 Slide 8 of 46

9 Example (continued) Small Example The Basics The Basics Estimation Example Properties e can tell that: (X X) 2 = x 2 = 40 (X X)(Y Ȳ ) = xy = 30 X = 3.0 Ȳ = 7.3 So: b = xy x 2 = = 0.75 a = Ȳ b X = 7.3 ( ) = 5.05 Given these estimates, the linear regression line is given by: Y = X rapping Up Lecture #2-1/27/2005 Slide 9 of 46

10 Example (continued) Small Example The Basics The Basics Estimation Example Properties Test Score = * X R Square = 0.17 Test Score rapping Up Hours of Study Lecture #2-1/27/2005 Slide 10 of 46

11 Properties of the LS Estimates Notice: Small Example The Basics The Basics Estimation Example Properties Y = a + bx = (Ȳ b X) + bx = This means when X = X, Y = Ȳ. Ȳ + b(x X) Furthermore, this implies that for every simple linear regression, the point ( X, Ȳ ) will always fall on the regression line. Furthermore, if b = 0 (no relation between X and Y ), the best guess (in a LS sense) for a value of Y would be Ȳ. rapping Up Lecture #2-1/27/2005 Slide 11 of 46

12 (Y Ȳ ) 2 = ss reg + ss res How Good Is Our Line? Model Accuracy Sums of Squares Example Variance Accounted For VAC Example You ve fit the model...you have your estimates...now what? e need to determine how well Y is predicted by X. Specifically, how much variability in Y can be accounted for by the linear regression where X is a predictor? A math adventure: Y = Ȳ + (Y Ȳ ) + (Y Y ) Y Ȳ = (Y Ȳ ) + (Y Y ) (Y Ȳ ) 2 = (Y Ȳ )2 + (Y Y ) 2 rapping Up Lecture #2-1/27/2005 Slide 12 of 46

13 Sums of Squares Model Accuracy Sums of Squares Example Variance Accounted For VAC Example The Sums of Squares for our dependent variable, Y, is denoted in Pedhazur by y 2. As shown before, in a regression, these can be partitioned into two components: Part due to the regression line: ss reg rapping Up Part due to error: ss res Dividing both components by y 2 gives the proportion of sum of squares due accounted for by the regression and the proportion accounted for by error. Lecture #2-1/27/2005 Slide 13 of 46

14 Example Computation Model Accuracy Sums of Squares Example Variance Accounted For VAC Example In our studying v. test score example from before, we know that: y 2 = b = 0.75 xy= 30 To compute ss reg, one of several formulas can be used: ss reg = ( xy) 2 x 2 = b xy = b 2 x 2 = = 22.5 rapping Up ss res = y 2 ss reg = = Lecture #2-1/27/2005 Slide 14 of 46

15 Example Continued Dividing both by y 2 gives: Proportion explained by regression: Model Accuracy Sums of Squares Example Variance Accounted For VAC Example Proportion explained by error: rapping Up Lecture #2-1/27/2005 Slide 15 of 46

16 Variance Accounted For Model Accuracy Sums of Squares Example Variance Accounted For VAC Example Instead of approaching fit via Sums of Squares, one could look at it as a spinoff of the Pearson correlation: r 2 xy = ( xy) 2 x 2 y 2 ss reg = r 2 xy y 2 rapping Up Lecture #2-1/27/2005 Slide 16 of 46

17 Variance Accounted For - Example From our studying v. test score example from before, we know that: Model Accuracy Sums of Squares Example Variance Accounted For VAC Example So, r 2 xy = r xy = hich is the percent of variance accounted for by the regression. Therefore, 1 r 2 xy = 0.827; the percent of variance due to error, or unexplained variance. rapping Up Lecture #2-1/27/2005 Slide 17 of 46

18 Statistical Significance Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test rapping Up Statistical significance is literally the likelihood of the null hypothesis in a hypothesis test. H 0 : b = 0 H 1 : b 0 Statistical significance is not related to the meaningfulness of the result. Take b = ith se(b) = b is statistically significant but in reality, there is a negligible relationship between X and Y. Lecture #2-1/27/2005 Slide 18 of 46

19 Hypothesis Tests in Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test There are several types of hypothesis tests that can be used in simple linear regression. Most of these tests are all related: a test for a non-zero relationship between two variables, X and Y. Today, we have covered three different measures of association: b -ss reg r 2 r xy rapping Up Lecture #2-1/27/2005 Slide 19 of 46

20 Hypothesis Tests in Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test Each of the following tests evolve statistically from placing distributional assumptions on the disturbance term. Y = a + bx + e Typically, we say that errors follow a normal distribution: e N(0, σ 2 e) By placing distributional assumptions Sampling distributions are defined. Hypothesis tests are enabled. must be recognized and checked for validity. rapping Up Lecture #2-1/27/2005 Slide 20 of 46

21 Testing the Entire Regression Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test A test of all meaningful regression parameters is as follows: H 0 : b = 0 H 1 : b 0 Because we only have one predictor variable, this is very straight forward. The test statistic is F -distributed, and is computed by: F = ss reg df reg ss res df res = ss reg k ss res N k 1 F-table gives significance for a given Type I error rate α, OR USE THE COMPUTER. rapping Up Lecture #2-1/27/2005 Slide 21 of 46

22 Testing the Entire Regression - Example From our example, the test of the entire regression is: H 0 : b = 0 H 1 : b 0 Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test rapping Up F = ss reg df reg ss res df res = ss reg k ss res N k 1 = = 3.76 Typing =fdist(3.76,1,18) into MS Excel gives a p-value of e could infer that there is no statistical evidence to suggest b is different from 0. This means there is no statistical relationship between hours of study and the score on a test. Lecture #2-1/27/2005 Slide 22 of 46

23 Testing the VAC Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test Because of the relationship of r 2 xy with ss reg the previous test could be phrased as: F = r 2 xy df reg 1 r 2 xy df res = r 2 xy k 1 r 2 xy N k 1 = = 3.76 Notice this is identical to the F we received before. Therefore our conclusions are the same... rapping Up Lecture #2-1/27/2005 Slide 23 of 46

24 Testing the Regression Coefficient Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test Finally, the regression coefficient between X and Y can tested for difference from zero: H 0 : b = 0 H 1 : b 0 The variance of Y conditional on X is given by: (Y Y s 2 ) y.x = N k 1 = ss res N k 1 The standard error of the regression coefficient, s b, is based on the conditional variance of Y X: s 2 y.x s b = x 2 rapping Up Lecture #2-1/27/2005 Slide 24 of 46

25 Testing the Regression Coefficient Finally, the test statistic for the following hypothesis test is: H 0 : b = 0 H 1 : b 0 Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test t = b s b This statistic is t distributed, with N-k-1 degrees of freedom (which is the denominator of s 2 y.x). rapping Up Lecture #2-1/27/2005 Slide 25 of 46

26 Testing the Regression Coefficient - Example From our example, the test of the regression coefficient is: H 0 : b = 0 H 1 : b 0 Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test s 2 y.x = (Y Y ) N k 1 = ss res N k 1 = x 2 = 40 = rapping Up t = b s 2 y.x x 2 = = 1.94 Typing in =tdist(1.94,18,2) (2 for a two-tailed test), gives a p-value of the same as before. Recall F = t 2 from somewhere before... Lecture #2-1/27/2005 Slide 26 of 46

27 Testing the Regression Coefficient Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test In our example, we tested against zero. In reality, any value can be tested against: H 0 : b = B H 1 : b B t = b B s 2 y.x x 2 rapping Up Lecture #2-1/27/2005 Slide 27 of 46

28 Key Behind Basic assumptions: X is fixed - measured without error. The mean of Y X has a linear relationship with X Error assumptions: Mean of errors is zero. All errors independent. The variance of the errors does not change as a function of X - homoscedasticity. All errors are uncorrelated with X. Statistically speaking, errors are IID: e N(0, σ 2 e). rapping Up Lecture #2-1/27/2005 Slide 28 of 46

29 : ays of Checking An easy way to check many error based assumptions a visualization: A plot of the standardized residual versus the predicted value. The next class will focus on other diagnostic measures. Example rapping Up Lecture #2-1/27/2005 Slide 29 of 46

30 Z Curvilinear Example hat if we were to fit a linear regression to these data? Example rapping Up Hours of Study Lecture #2-1/27/2005 Slide 30 of 46

31 Z Curvilinear Example Fitting the simple linear regression model gives: Example rapping Up 2.50 Z = * X R Square = Hours of Study Not good? Lecture #2-1/27/2005 Slide 31 of 46

32 Curvilinear Example Let s take a look at a plot of the standardized residuals versus the predicted values: Example Standardized Residual rapping Up Unstandardized Predicted Value Lecture #2-1/27/2005 Slide 32 of 46

33 hen X is Random Commonly, we have no control over our predictor variable. Observational Studies Statistically speaking, having X being random adds: The potential for measurement error in X - reliability. Limitations to the causal inferences that can be drawn from a regression analysis. All test statistics and measures are used equivalently. e can predict Y from X or X from Y. Correlations are more common place: common variance rather than causal influence. rapping Up Lecture #2-1/27/2005 Slide 33 of 46

34 rap Up Example Example taken from eisberg (1985, p. 230): Old Faithful Scatterplot Descriptives Regression Line Hypothesis Tests Perhaps the most famous geyser is Old Faithful, in Yellowstone National Park, yoming. The intervals of eruptions of Old Faithful range from about 30 to 90 minutes. ater shoots to heights generally over 35 meters, with eruptions lasting from 1 to 5.5 minutes. Data on Old Faithful has been collected for many years by ranger/naturalists in the park, using a stopwatch. The duration measurements have been rounded to the nearest 0.1 minute or 6 seconds, while intervals reported are to the nearest minute. The National Park Service uses x (values of the duration of an eruption) to predict y (the interval to the next eruption). rapping Up Lecture #2-1/27/2005 Slide 34 of 46

35 Old Faithful Postcard Lecture #2-1/27/2005 Slide 35 of 46

36 Old Faithful Scatterplot Old Faithful Scatterplot Descriptives Regression Line Hypothesis Tests Interval to Next Eruption rapping Up Duration of Eruption Lecture #2-1/27/2005 Slide 36 of 46

37 Descriptive Statistics Old Faithful Scatterplot Descriptives Regression Line Hypothesis Tests Variable N Mean SD x - Duration y - Interval r xy = r 2 xy = x 2 = y 2 = 17, xy = 1, rapping Up Lecture #2-1/27/2005 Slide 37 of 46

38 Regression Line b = xy x 2 = 1, = a = Ȳ b X = 71.0 (10.74) = Y = X Old Faithful Scatterplot Descriptives Regression Line Hypothesis Tests rapping Up Lecture #2-1/27/2005 Slide 38 of 46

39 Regression Line Old Faithful Scatterplot Descriptives Regression Line Hypothesis Tests rapping Up Interval to Next Eruption Interval to Next Eruption = * X R Square = Duration of Eruption Lecture #2-1/27/2005 Slide 39 of 46

40 Hypothesis Tests ss reg = b xy = , = 13, ss res = y 2 ss reg = 17, , = 4, F = ss reg k ss res N k 1 = 13, /1 4, /( ) = Old Faithful Scatterplot Descriptives Regression Line Hypothesis Tests rapping Up Lecture #2-1/27/2005 Slide 40 of 46

41 Residual Plot Old Faithful Scatterplot Descriptives Regression Line Hypothesis Tests Standardized Residual rapping Up Unstandardized Predicted Value Lecture #2-1/27/2005 Slide 41 of 46

42 Data Data More Data Scatterplots rapping Up Lecture #2-1/27/2005 Slide 42 of 46

43 Data Data More Data Scatterplots rapping Up Lecture #2-1/27/2005 Slide 43 of 46

44 Plots from Today Scatterplots were made with the interactive graphs: Graphs...Interactive...Scatterplot. Data More Data Scatterplots rapping Up Lecture #2-1/27/2005 Slide 44 of 46

45 Analyze...Regression...Linear. Data More Data Scatterplots rapping Up Lecture #2-1/27/2005 Slide 45 of 46

46 Saving Residuals Analyze...Regression...Linear Save Button. Data More Data Scatterplots rapping Up Lecture #2-1/27/2005 Slide 46 of 46

47 Final Thought rapping Up Final Thought Next Class Simple linear regression can be simple and straight forward. Need to understand difference between statistical significance and practical relevance. The tests of significance and model estimates are only valid if all assumptions are met. Regression is fairly robust to violations of assumptions, but... Usually when assumptions are violated, something else is wrong with the model. Listen to your data, they are trying to tell you something. Lecture #2-1/27/2005 Slide 47 of 46

48 Next Time Regression diagnostics - Chapter 3. More. More practice with fitting models. QUESTION: Do you have any data you are willing to share with class? rapping Up Final Thought Next Class Lecture #2-1/27/2005 Slide 48 of 46

Advanced Regression Topics: Violation of Assumptions

Advanced Regression Topics: Violation of Assumptions Advanced Regression Topics: Violation of Assumptions Lecture 7 February 15, 2005 Applied Regression Analysis Lecture #7-2/15/2005 Slide 1 of 36 Today s Lecture Today s Lecture rapping Up Revisiting residuals.

More information

Regression With a Categorical Independent Variable

Regression With a Categorical Independent Variable Regression ith a Independent Variable ERSH 8320 Slide 1 of 34 Today s Lecture Regression with a single categorical independent variable. Today s Lecture Coding procedures for analysis. Dummy coding. Relationship

More information

Regression With a Categorical Independent Variable

Regression With a Categorical Independent Variable Regression With a Independent Variable Lecture 10 November 5, 2008 ERSH 8320 Lecture #10-11/5/2008 Slide 1 of 54 Today s Lecture Today s Lecture Chapter 11: Regression with a single categorical independent

More information

Regression With a Categorical Independent Variable

Regression With a Categorical Independent Variable Regression With a Categorical Independent Variable Lecture 15 March 17, 2005 Applied Regression Analysis Lecture #15-3/17/2005 Slide 1 of 29 Today s Lecture» Today s Lecture» Midterm Note» Example Regression

More information

ASSIGNMENT 3 SIMPLE LINEAR REGRESSION. Old Faithful

ASSIGNMENT 3 SIMPLE LINEAR REGRESSION. Old Faithful ASSIGNMENT 3 SIMPLE LINEAR REGRESSION In the simple linear regression model, the mean of a response variable is a linear function of an explanatory variable. The model and associated inferential tools

More information

Inferences for Regression

Inferences for Regression Inferences for Regression An Example: Body Fat and Waist Size Looking at the relationship between % body fat and waist size (in inches). Here is a scatterplot of our data set: Remembering Regression In

More information

Scatter plot of data from the study. Linear Regression

Scatter plot of data from the study. Linear Regression 1 2 Linear Regression Scatter plot of data from the study. Consider a study to relate birthweight to the estriol level of pregnant women. The data is below. i Weight (g / 100) i Weight (g / 100) 1 7 25

More information

Simple Linear Regression

Simple Linear Regression Simple Linear Regression ST 430/514 Recall: A regression model describes how a dependent variable (or response) Y is affected, on average, by one or more independent variables (or factors, or covariates)

More information

Simple Linear Regression

Simple Linear Regression Simple Linear Regression 1 Correlation indicates the magnitude and direction of the linear relationship between two variables. Linear Regression: variable Y (criterion) is predicted by variable X (predictor)

More information

REVIEW 8/2/2017 陈芳华东师大英语系

REVIEW 8/2/2017 陈芳华东师大英语系 REVIEW Hypothesis testing starts with a null hypothesis and a null distribution. We compare what we have to the null distribution, if the result is too extreme to belong to the null distribution (p

More information

Regression With a Categorical Independent Variable: Mean Comparisons

Regression With a Categorical Independent Variable: Mean Comparisons Regression With a Categorical Independent Variable: Mean Lecture 16 March 29, 2005 Applied Regression Analysis Lecture #16-3/29/2005 Slide 1 of 43 Today s Lecture comparisons among means. Today s Lecture

More information

Review 6. n 1 = 85 n 2 = 75 x 1 = x 2 = s 1 = 38.7 s 2 = 39.2

Review 6. n 1 = 85 n 2 = 75 x 1 = x 2 = s 1 = 38.7 s 2 = 39.2 Review 6 Use the traditional method to test the given hypothesis. Assume that the samples are independent and that they have been randomly selected ) A researcher finds that of,000 people who said that

More information

MORE ON SIMPLE REGRESSION: OVERVIEW

MORE ON SIMPLE REGRESSION: OVERVIEW FI=NOT0106 NOTICE. Unless otherwise indicated, all materials on this page and linked pages at the blue.temple.edu address and at the astro.temple.edu address are the sole property of Ralph B. Taylor and

More information

Chapter 16. Simple Linear Regression and dcorrelation

Chapter 16. Simple Linear Regression and dcorrelation Chapter 16 Simple Linear Regression and dcorrelation 16.1 Regression Analysis Our problem objective is to analyze the relationship between interval variables; regression analysis is the first tool we will

More information

Scatter plot of data from the study. Linear Regression

Scatter plot of data from the study. Linear Regression 1 2 Linear Regression Scatter plot of data from the study. Consider a study to relate birthweight to the estriol level of pregnant women. The data is below. i Weight (g / 100) i Weight (g / 100) 1 7 25

More information

Lecture 2 Simple Linear Regression STAT 512 Spring 2011 Background Reading KNNL: Chapter 1

Lecture 2 Simple Linear Regression STAT 512 Spring 2011 Background Reading KNNL: Chapter 1 Lecture Simple Linear Regression STAT 51 Spring 011 Background Reading KNNL: Chapter 1-1 Topic Overview This topic we will cover: Regression Terminology Simple Linear Regression with a single predictor

More information

Lecture Slides. Elementary Statistics Tenth Edition. by Mario F. Triola. and the Triola Statistics Series. Slide 1

Lecture Slides. Elementary Statistics Tenth Edition. by Mario F. Triola. and the Triola Statistics Series. Slide 1 Lecture Slides Elementary Statistics Tenth Edition and the Triola Statistics Series by Mario F. Triola Slide 1 Chapter 10 Correlation and Regression 10-1 Overview 10-2 Correlation 10-3 Regression 10-4

More information

Six Sigma Black Belt Study Guides

Six Sigma Black Belt Study Guides Six Sigma Black Belt Study Guides 1 www.pmtutor.org Powered by POeT Solvers Limited. Analyze Correlation and Regression Analysis 2 www.pmtutor.org Powered by POeT Solvers Limited. Variables and relationships

More information

Lecture 14 Simple Linear Regression

Lecture 14 Simple Linear Regression Lecture 4 Simple Linear Regression Ordinary Least Squares (OLS) Consider the following simple linear regression model where, for each unit i, Y i is the dependent variable (response). X i is the independent

More information

Linear Regression & Correlation

Linear Regression & Correlation Linear Regression & Correlation Jamie Monogan University of Georgia Introduction to Data Analysis Jamie Monogan (UGA) Linear Regression & Correlation POLS 7012 1 / 25 Objectives By the end of these meetings,

More information

Multiple Regression. More Hypothesis Testing. More Hypothesis Testing The big question: What we really want to know: What we actually know: We know:

Multiple Regression. More Hypothesis Testing. More Hypothesis Testing The big question: What we really want to know: What we actually know: We know: Multiple Regression Ψ320 Ainsworth More Hypothesis Testing What we really want to know: Is the relationship in the population we have selected between X & Y strong enough that we can use the relationship

More information

determine whether or not this relationship is.

determine whether or not this relationship is. Section 9-1 Correlation A correlation is a between two. The data can be represented by ordered pairs (x,y) where x is the (or ) variable and y is the (or ) variable. There are several types of correlations

More information

Business Statistics. Lecture 9: Simple Regression

Business Statistics. Lecture 9: Simple Regression Business Statistics Lecture 9: Simple Regression 1 On to Model Building! Up to now, class was about descriptive and inferential statistics Numerical and graphical summaries of data Confidence intervals

More information

Psychology 282 Lecture #4 Outline Inferences in SLR

Psychology 282 Lecture #4 Outline Inferences in SLR Psychology 282 Lecture #4 Outline Inferences in SLR Assumptions To this point we have not had to make any distributional assumptions. Principle of least squares requires no assumptions. Can use correlations

More information

Regression: Main Ideas Setting: Quantitative outcome with a quantitative explanatory variable. Example, cont.

Regression: Main Ideas Setting: Quantitative outcome with a quantitative explanatory variable. Example, cont. TCELL 9/4/205 36-309/749 Experimental Design for Behavioral and Social Sciences Simple Regression Example Male black wheatear birds carry stones to the nest as a form of sexual display. Soler et al. wanted

More information

AMS 7 Correlation and Regression Lecture 8

AMS 7 Correlation and Regression Lecture 8 AMS 7 Correlation and Regression Lecture 8 Department of Applied Mathematics and Statistics, University of California, Santa Cruz Suumer 2014 1 / 18 Correlation pairs of continuous observations. Correlation

More information

Ch 2: Simple Linear Regression

Ch 2: Simple Linear Regression Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component

More information

Statistical Distribution Assumptions of General Linear Models

Statistical Distribution Assumptions of General Linear Models Statistical Distribution Assumptions of General Linear Models Applied Multilevel Models for Cross Sectional Data Lecture 4 ICPSR Summer Workshop University of Colorado Boulder Lecture 4: Statistical Distributions

More information

Chapter 16. Simple Linear Regression and Correlation

Chapter 16. Simple Linear Regression and Correlation Chapter 16 Simple Linear Regression and Correlation 16.1 Regression Analysis Our problem objective is to analyze the relationship between interval variables; regression analysis is the first tool we will

More information

Regression Models - Introduction

Regression Models - Introduction Regression Models - Introduction In regression models there are two types of variables that are studied: A dependent variable, Y, also called response variable. It is modeled as random. An independent

More information

Simple linear regression

Simple linear regression Simple linear regression Biometry 755 Spring 2008 Simple linear regression p. 1/40 Overview of regression analysis Evaluate relationship between one or more independent variables (X 1,...,X k ) and a single

More information

THE PEARSON CORRELATION COEFFICIENT

THE PEARSON CORRELATION COEFFICIENT CORRELATION Two variables are said to have a relation if knowing the value of one variable gives you information about the likely value of the second variable this is known as a bivariate relation There

More information

Tutorial 6: Linear Regression

Tutorial 6: Linear Regression Tutorial 6: Linear Regression Rob Nicholls nicholls@mrc-lmb.cam.ac.uk MRC LMB Statistics Course 2014 Contents 1 Introduction to Simple Linear Regression................ 1 2 Parameter Estimation and Model

More information

y = a + bx 12.1: Inference for Linear Regression Review: General Form of Linear Regression Equation Review: Interpreting Computer Regression Output

y = a + bx 12.1: Inference for Linear Regression Review: General Form of Linear Regression Equation Review: Interpreting Computer Regression Output 12.1: Inference for Linear Regression Review: General Form of Linear Regression Equation y = a + bx y = dependent variable a = intercept b = slope x = independent variable Section 12.1 Inference for Linear

More information

36-309/749 Experimental Design for Behavioral and Social Sciences. Sep. 22, 2015 Lecture 4: Linear Regression

36-309/749 Experimental Design for Behavioral and Social Sciences. Sep. 22, 2015 Lecture 4: Linear Regression 36-309/749 Experimental Design for Behavioral and Social Sciences Sep. 22, 2015 Lecture 4: Linear Regression TCELL Simple Regression Example Male black wheatear birds carry stones to the nest as a form

More information

Quantitative Techniques - Lecture 8: Estimation

Quantitative Techniques - Lecture 8: Estimation Quantitative Techniques - Lecture 8: Estimation Key words: Estimation, hypothesis testing, bias, e ciency, least squares Hypothesis testing when the population variance is not known roperties of estimates

More information

Basic Business Statistics 6 th Edition

Basic Business Statistics 6 th Edition Basic Business Statistics 6 th Edition Chapter 12 Simple Linear Regression Learning Objectives In this chapter, you learn: How to use regression analysis to predict the value of a dependent variable based

More information

Regression. Marc H. Mehlman University of New Haven

Regression. Marc H. Mehlman University of New Haven Regression Marc H. Mehlman marcmehlman@yahoo.com University of New Haven the statistician knows that in nature there never was a normal distribution, there never was a straight line, yet with normal and

More information

Wed, June 26, (Lecture 8-2). Nonlinearity. Significance test for correlation R-squared, SSE, and SST. Correlation in SPSS.

Wed, June 26, (Lecture 8-2). Nonlinearity. Significance test for correlation R-squared, SSE, and SST. Correlation in SPSS. Wed, June 26, (Lecture 8-2). Nonlinearity. Significance test for correlation R-squared, SSE, and SST. Correlation in SPSS. Last time, we looked at scatterplots, which show the interaction between two variables,

More information

y response variable x 1, x 2,, x k -- a set of explanatory variables

y response variable x 1, x 2,, x k -- a set of explanatory variables 11. Multiple Regression and Correlation y response variable x 1, x 2,, x k -- a set of explanatory variables In this chapter, all variables are assumed to be quantitative. Chapters 12-14 show how to incorporate

More information

Keller: Stats for Mgmt & Econ, 7th Ed July 17, 2006

Keller: Stats for Mgmt & Econ, 7th Ed July 17, 2006 Chapter 17 Simple Linear Regression and Correlation 17.1 Regression Analysis Our problem objective is to analyze the relationship between interval variables; regression analysis is the first tool we will

More information

23. Inference for regression

23. Inference for regression 23. Inference for regression The Practice of Statistics in the Life Sciences Third Edition 2014 W. H. Freeman and Company Objectives (PSLS Chapter 23) Inference for regression The regression model Confidence

More information

Business Statistics. Lecture 10: Correlation and Linear Regression

Business Statistics. Lecture 10: Correlation and Linear Regression Business Statistics Lecture 10: Correlation and Linear Regression Scatterplot A scatterplot shows the relationship between two quantitative variables measured on the same individuals. It displays the Form

More information

INFERENCE FOR REGRESSION

INFERENCE FOR REGRESSION CHAPTER 3 INFERENCE FOR REGRESSION OVERVIEW In Chapter 5 of the textbook, we first encountered regression. The assumptions that describe the regression model we use in this chapter are the following. We

More information

Lecture 2 Linear Regression: A Model for the Mean. Sharyn O Halloran

Lecture 2 Linear Regression: A Model for the Mean. Sharyn O Halloran Lecture 2 Linear Regression: A Model for the Mean Sharyn O Halloran Closer Look at: Linear Regression Model Least squares procedure Inferential tools Confidence and Prediction Intervals Assumptions Robustness

More information

Correlation & Simple Regression

Correlation & Simple Regression Chapter 11 Correlation & Simple Regression The previous chapter dealt with inference for two categorical variables. In this chapter, we would like to examine the relationship between two quantitative variables.

More information

1 Correlation and Inference from Regression

1 Correlation and Inference from Regression 1 Correlation and Inference from Regression Reading: Kennedy (1998) A Guide to Econometrics, Chapters 4 and 6 Maddala, G.S. (1992) Introduction to Econometrics p. 170-177 Moore and McCabe, chapter 12 is

More information

Chapter 1: Linear Regression with One Predictor Variable also known as: Simple Linear Regression Bivariate Linear Regression

Chapter 1: Linear Regression with One Predictor Variable also known as: Simple Linear Regression Bivariate Linear Regression BSTT523: Kutner et al., Chapter 1 1 Chapter 1: Linear Regression with One Predictor Variable also known as: Simple Linear Regression Bivariate Linear Regression Introduction: Functional relation between

More information

BIOSTATISTICS NURS 3324

BIOSTATISTICS NURS 3324 Simple Linear Regression and Correlation Introduction Previously, our attention has been focused on one variable which we designated by x. Frequently, it is desirable to learn something about the relationship

More information

Can you tell the relationship between students SAT scores and their college grades?

Can you tell the relationship between students SAT scores and their college grades? Correlation One Challenge Can you tell the relationship between students SAT scores and their college grades? A: The higher SAT scores are, the better GPA may be. B: The higher SAT scores are, the lower

More information

Simple Linear Regression. (Chs 12.1, 12.2, 12.4, 12.5)

Simple Linear Regression. (Chs 12.1, 12.2, 12.4, 12.5) 10 Simple Linear Regression (Chs 12.1, 12.2, 12.4, 12.5) Simple Linear Regression Rating 20 40 60 80 0 5 10 15 Sugar 2 Simple Linear Regression Rating 20 40 60 80 0 5 10 15 Sugar 3 Simple Linear Regression

More information

Chapter 4: Regression Models

Chapter 4: Regression Models Sales volume of company 1 Textbook: pp. 129-164 Chapter 4: Regression Models Money spent on advertising 2 Learning Objectives After completing this chapter, students will be able to: Identify variables,

More information

Mathematics for Economics MA course

Mathematics for Economics MA course Mathematics for Economics MA course Simple Linear Regression Dr. Seetha Bandara Simple Regression Simple linear regression is a statistical method that allows us to summarize and study relationships between

More information

Simple Linear Regression

Simple Linear Regression Simple Linear Regression EdPsych 580 C.J. Anderson Fall 2005 Simple Linear Regression p. 1/80 Outline 1. What it is and why it s useful 2. How 3. Statistical Inference 4. Examining assumptions (diagnostics)

More information

Correlation Analysis

Correlation Analysis Simple Regression Correlation Analysis Correlation analysis is used to measure strength of the association (linear relationship) between two variables Correlation is only concerned with strength of the

More information

STAT 135 Lab 11 Tests for Categorical Data (Fisher s Exact test, χ 2 tests for Homogeneity and Independence) and Linear Regression

STAT 135 Lab 11 Tests for Categorical Data (Fisher s Exact test, χ 2 tests for Homogeneity and Independence) and Linear Regression STAT 135 Lab 11 Tests for Categorical Data (Fisher s Exact test, χ 2 tests for Homogeneity and Independence) and Linear Regression Rebecca Barter April 20, 2015 Fisher s Exact Test Fisher s Exact Test

More information

SSR = The sum of squared errors measures how much Y varies around the regression line n. It happily turns out that SSR + SSE = SSTO.

SSR = The sum of squared errors measures how much Y varies around the regression line n. It happily turns out that SSR + SSE = SSTO. Analysis of variance approach to regression If x is useless, i.e. β 1 = 0, then E(Y i ) = β 0. In this case β 0 is estimated by Ȳ. The ith deviation about this grand mean can be written: deviation about

More information

Chapter 11 Linear Regression

Chapter 11 Linear Regression Chapter 11 Linear Regression Linear regression is a methodology that allows us to examine the relationship between two continuously measured variables where we believe that values of one variable may influence

More information

Density Temp vs Ratio. temp

Density Temp vs Ratio. temp Temp Ratio Density 0.00 0.02 0.04 0.06 0.08 0.10 0.12 Density 0.0 0.2 0.4 0.6 0.8 1.0 1. (a) 170 175 180 185 temp 1.0 1.5 2.0 2.5 3.0 ratio The histogram shows that the temperature measures have two peaks,

More information

Correlation 1. December 4, HMS, 2017, v1.1

Correlation 1. December 4, HMS, 2017, v1.1 Correlation 1 December 4, 2017 1 HMS, 2017, v1.1 Chapter References Diez: Chapter 7 Navidi, Chapter 7 I don t expect you to learn the proofs what will follow. Chapter References 2 Correlation The sample

More information

Simple Linear Regression

Simple Linear Regression Simple Linear Regression OI CHAPTER 7 Important Concepts Correlation (r or R) and Coefficient of determination (R 2 ) Interpreting y-intercept and slope coefficients Inference (hypothesis testing and confidence

More information

Statistics for Engineers Lecture 9 Linear Regression

Statistics for Engineers Lecture 9 Linear Regression Statistics for Engineers Lecture 9 Linear Regression Chong Ma Department of Statistics University of South Carolina chongm@email.sc.edu April 17, 2017 Chong Ma (Statistics, USC) STAT 509 Spring 2017 April

More information

Regression Models. Chapter 4. Introduction. Introduction. Introduction

Regression Models. Chapter 4. Introduction. Introduction. Introduction Chapter 4 Regression Models Quantitative Analysis for Management, Tenth Edition, by Render, Stair, and Hanna 008 Prentice-Hall, Inc. Introduction Regression analysis is a very valuable tool for a manager

More information

28. SIMPLE LINEAR REGRESSION III

28. SIMPLE LINEAR REGRESSION III 28. SIMPLE LINEAR REGRESSION III Fitted Values and Residuals To each observed x i, there corresponds a y-value on the fitted line, y = βˆ + βˆ x. The are called fitted values. ŷ i They are the values of

More information

Dr. Junchao Xia Center of Biophysics and Computational Biology. Fall /1/2016 1/46

Dr. Junchao Xia Center of Biophysics and Computational Biology. Fall /1/2016 1/46 BIO5312 Biostatistics Lecture 10:Regression and Correlation Methods Dr. Junchao Xia Center of Biophysics and Computational Biology Fall 2016 11/1/2016 1/46 Outline In this lecture, we will discuss topics

More information

Chapter 4. Regression Models. Learning Objectives

Chapter 4. Regression Models. Learning Objectives Chapter 4 Regression Models To accompany Quantitative Analysis for Management, Eleventh Edition, by Render, Stair, and Hanna Power Point slides created by Brian Peterson Learning Objectives After completing

More information

Regression Analysis: Basic Concepts

Regression Analysis: Basic Concepts The simple linear model Regression Analysis: Basic Concepts Allin Cottrell Represents the dependent variable, y i, as a linear function of one independent variable, x i, subject to a random disturbance

More information

AP Statistics Unit 6 Note Packet Linear Regression. Scatterplots and Correlation

AP Statistics Unit 6 Note Packet Linear Regression. Scatterplots and Correlation Scatterplots and Correlation Name Hr A scatterplot shows the relationship between two quantitative variables measured on the same individuals. variable (y) measures an outcome of a study variable (x) may

More information

y n 1 ( x i x )( y y i n 1 i y 2

y n 1 ( x i x )( y y i n 1 i y 2 STP3 Brief Class Notes Instructor: Ela Jackiewicz Chapter Regression and Correlation In this chapter we will explore the relationship between two quantitative variables, X an Y. We will consider n ordered

More information

Regression Models - Introduction

Regression Models - Introduction Regression Models - Introduction In regression models, two types of variables that are studied: A dependent variable, Y, also called response variable. It is modeled as random. An independent variable,

More information

AMS 315/576 Lecture Notes. Chapter 11. Simple Linear Regression

AMS 315/576 Lecture Notes. Chapter 11. Simple Linear Regression AMS 315/576 Lecture Notes Chapter 11. Simple Linear Regression 11.1 Motivation A restaurant opening on a reservations-only basis would like to use the number of advance reservations x to predict the number

More information

CS 5014: Research Methods in Computer Science

CS 5014: Research Methods in Computer Science Computer Science Clifford A. Shaffer Department of Computer Science Virginia Tech Blacksburg, Virginia Fall 2010 Copyright c 2010 by Clifford A. Shaffer Computer Science Fall 2010 1 / 207 Correlation and

More information

Regression and correlation. Correlation & Regression, I. Regression & correlation. Regression vs. correlation. Involve bivariate, paired data, X & Y

Regression and correlation. Correlation & Regression, I. Regression & correlation. Regression vs. correlation. Involve bivariate, paired data, X & Y Regression and correlation Correlation & Regression, I 9.07 4/1/004 Involve bivariate, paired data, X & Y Height & weight measured for the same individual IQ & exam scores for each individual Height of

More information

Sociology 6Z03 Review II

Sociology 6Z03 Review II Sociology 6Z03 Review II John Fox McMaster University Fall 2016 John Fox (McMaster University) Sociology 6Z03 Review II Fall 2016 1 / 35 Outline: Review II Probability Part I Sampling Distributions Probability

More information

Chapter 1 Statistical Inference

Chapter 1 Statistical Inference Chapter 1 Statistical Inference causal inference To infer causality, you need a randomized experiment (or a huge observational study and lots of outside information). inference to populations Generalizations

More information

PS2: Two Variable Statistics

PS2: Two Variable Statistics PS2: Two Variable Statistics LT2: Measuring Correlation and Line of best fit by eye. LT3: Linear regression LT4: The χ 2 test of independence. 1 Pearson's Correlation Coefficient In examinations you are

More information

Nature vs. nurture? Lecture 18 - Regression: Inference, Outliers, and Intervals. Regression Output. Conditions for inference.

Nature vs. nurture? Lecture 18 - Regression: Inference, Outliers, and Intervals. Regression Output. Conditions for inference. Understanding regression output from software Nature vs. nurture? Lecture 18 - Regression: Inference, Outliers, and Intervals In 1966 Cyril Burt published a paper called The genetic determination of differences

More information

Section 3: Simple Linear Regression

Section 3: Simple Linear Regression Section 3: Simple Linear Regression Carlos M. Carvalho The University of Texas at Austin McCombs School of Business http://faculty.mccombs.utexas.edu/carlos.carvalho/teaching/ 1 Regression: General Introduction

More information

Unit 6 - Introduction to linear regression

Unit 6 - Introduction to linear regression Unit 6 - Introduction to linear regression Suggested reading: OpenIntro Statistics, Chapter 7 Suggested exercises: Part 1 - Relationship between two numerical variables: 7.7, 7.9, 7.11, 7.13, 7.15, 7.25,

More information

Intro to Linear Regression

Intro to Linear Regression Intro to Linear Regression Introduction to Regression Regression is a statistical procedure for modeling the relationship among variables to predict the value of a dependent variable from one or more predictor

More information

Multiple Regression. Dr. Frank Wood. Frank Wood, Linear Regression Models Lecture 12, Slide 1

Multiple Regression. Dr. Frank Wood. Frank Wood, Linear Regression Models Lecture 12, Slide 1 Multiple Regression Dr. Frank Wood Frank Wood, fwood@stat.columbia.edu Linear Regression Models Lecture 12, Slide 1 Review: Matrix Regression Estimation We can solve this equation (if the inverse of X

More information

Chapter 8. Linear Regression. The Linear Model. Fat Versus Protein: An Example. The Linear Model (cont.) Residuals

Chapter 8. Linear Regression. The Linear Model. Fat Versus Protein: An Example. The Linear Model (cont.) Residuals Chapter 8 Linear Regression Copyright 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide 8-1 Copyright 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Fat Versus

More information

Chapter 8. Linear Regression. Copyright 2010 Pearson Education, Inc.

Chapter 8. Linear Regression. Copyright 2010 Pearson Education, Inc. Chapter 8 Linear Regression Copyright 2010 Pearson Education, Inc. Fat Versus Protein: An Example The following is a scatterplot of total fat versus protein for 30 items on the Burger King menu: Copyright

More information

Review of Multiple Regression

Review of Multiple Regression Ronald H. Heck 1 Let s begin with a little review of multiple regression this week. Linear models [e.g., correlation, t-tests, analysis of variance (ANOVA), multiple regression, path analysis, multivariate

More information

Chapter Learning Objectives. Regression Analysis. Correlation. Simple Linear Regression. Chapter 12. Simple Linear Regression

Chapter Learning Objectives. Regression Analysis. Correlation. Simple Linear Regression. Chapter 12. Simple Linear Regression Chapter 12 12-1 North Seattle Community College BUS21 Business Statistics Chapter 12 Learning Objectives In this chapter, you learn:! How to use regression analysis to predict the value of a dependent

More information

Chapter 9. Correlation and Regression

Chapter 9. Correlation and Regression Chapter 9 Correlation and Regression Lesson 9-1/9-2, Part 1 Correlation Registered Florida Pleasure Crafts and Watercraft Related Manatee Deaths 100 80 60 40 20 0 1991 1993 1995 1997 1999 Year Boats in

More information

UNIT 12 ~ More About Regression

UNIT 12 ~ More About Regression ***SECTION 15.1*** The Regression Model When a scatterplot shows a relationship between a variable x and a y, we can use the fitted to the data to predict y for a given value of x. Now we want to do tests

More information

Warm-up Using the given data Create a scatterplot Find the regression line

Warm-up Using the given data Create a scatterplot Find the regression line Time at the lunch table Caloric intake 21.4 472 30.8 498 37.7 335 32.8 423 39.5 437 22.8 508 34.1 431 33.9 479 43.8 454 42.4 450 43.1 410 29.2 504 31.3 437 28.6 489 32.9 436 30.6 480 35.1 439 33.0 444

More information

Linear Regression. Linear Regression. Linear Regression. Did You Mean Association Or Correlation?

Linear Regression. Linear Regression. Linear Regression. Did You Mean Association Or Correlation? Did You Mean Association Or Correlation? AP Statistics Chapter 8 Be careful not to use the word correlation when you really mean association. Often times people will incorrectly use the word correlation

More information

Chapter 27 Summary Inferences for Regression

Chapter 27 Summary Inferences for Regression Chapter 7 Summary Inferences for Regression What have we learned? We have now applied inference to regression models. Like in all inference situations, there are conditions that we must check. We can test

More information

Lecture 3: Inference in SLR

Lecture 3: Inference in SLR Lecture 3: Inference in SLR STAT 51 Spring 011 Background Reading KNNL:.1.6 3-1 Topic Overview This topic will cover: Review of hypothesis testing Inference about 1 Inference about 0 Confidence Intervals

More information

Correlation. A statistics method to measure the relationship between two variables. Three characteristics

Correlation. A statistics method to measure the relationship between two variables. Three characteristics Correlation Correlation A statistics method to measure the relationship between two variables Three characteristics Direction of the relationship Form of the relationship Strength/Consistency Direction

More information

The Simple Linear Regression Model

The Simple Linear Regression Model The Simple Linear Regression Model Lesson 3 Ryan Safner 1 1 Department of Economics Hood College ECON 480 - Econometrics Fall 2017 Ryan Safner (Hood College) ECON 480 - Lesson 3 Fall 2017 1 / 77 Bivariate

More information

Statistics for Managers using Microsoft Excel 6 th Edition

Statistics for Managers using Microsoft Excel 6 th Edition Statistics for Managers using Microsoft Excel 6 th Edition Chapter 13 Simple Linear Regression 13-1 Learning Objectives In this chapter, you learn: How to use regression analysis to predict the value of

More information

Any of 27 linear and nonlinear models may be fit. The output parallels that of the Simple Regression procedure.

Any of 27 linear and nonlinear models may be fit. The output parallels that of the Simple Regression procedure. STATGRAPHICS Rev. 9/13/213 Calibration Models Summary... 1 Data Input... 3 Analysis Summary... 5 Analysis Options... 7 Plot of Fitted Model... 9 Predicted Values... 1 Confidence Intervals... 11 Observed

More information

Chapter 8: Correlation & Regression

Chapter 8: Correlation & Regression Chapter 8: Correlation & Regression We can think of ANOVA and the two-sample t-test as applicable to situations where there is a response variable which is quantitative, and another variable that indicates

More information

SLR output RLS. Refer to slr (code) on the Lecture Page of the class website.

SLR output RLS. Refer to slr (code) on the Lecture Page of the class website. SLR output RLS Refer to slr (code) on the Lecture Page of the class website. Old Faithful at Yellowstone National Park, WY: Simple Linear Regression (SLR) Analysis SLR analysis explores the linear association

More information

Correlation and the Analysis of Variance Approach to Simple Linear Regression

Correlation and the Analysis of Variance Approach to Simple Linear Regression Correlation and the Analysis of Variance Approach to Simple Linear Regression Biometry 755 Spring 2009 Correlation and the Analysis of Variance Approach to Simple Linear Regression p. 1/35 Correlation

More information

Module 03 Lecture 14 Inferential Statistics ANOVA and TOI

Module 03 Lecture 14 Inferential Statistics ANOVA and TOI Introduction of Data Analytics Prof. Nandan Sudarsanam and Prof. B Ravindran Department of Management Studies and Department of Computer Science and Engineering Indian Institute of Technology, Madras Module

More information

Correlation and simple linear regression S5

Correlation and simple linear regression S5 Basic medical statistics for clinical and eperimental research Correlation and simple linear regression S5 Katarzyna Jóźwiak k.jozwiak@nki.nl November 15, 2017 1/41 Introduction Eample: Brain size and

More information