Applied Regression Analysis
|
|
- David Walton
- 6 years ago
- Views:
Transcription
1 Applied Regression Analysis Lecture 2 January 27, 2005 Lecture #2-1/27/2005 Slide 1 of 46
2 Today s Lecture Simple linear regression. Partitioning the sum of squares. Tests of significance.. Regression diagnostics (introduction). hen X is random. How to do today s topics in. rapping Up Lecture #2-1/27/2005 Slide 2 of 46
3 Previously on EPSY Recall from our first meeting the hand calculated sample problem with the following scatterplot: Small Example The Basics The Basics Estimation Example Properties rapping Up r XY = Correlation shows the degree of association between X and Y......but does so without scale. hat if we wanted to predict Y given a value of X? Lecture #2-1/27/2005 Slide 3 of 46
4 The Basics Small Example The Basics The Basics Estimation Example Properties rapping Up Assume (for now) X is fixed at pre-determined levels in an experiment - independent variable. For example, we have an experiment where subjects are given X cups of coffee. Jon the Mad Scientist decides that subjects should be randomly assigned to a group drinking either 1, 2, 3, 4,or 5 cups of coffee. The we want to estimate the linear effect of the independent variable X on the dependent variable Y. For our example, we want to see how coffee drinking affects blood pressure. Blood pressure = Y = dependent variable. Lecture #2-1/27/2005 Slide 4 of 46
5 The Basics Small Example The Basics The Basics Estimation Example Properties The linear regression model (for observation i = 1,...,N): Y i = α + βx i + ǫ i rapping Up Don t be confused by the Greek alphabet, this is simply the equation for a line (y = a + bx). α is the mean of the population when X is zero...the Y intercept. β is the slope of the line, the amount of increase in Y brought about by a unit increase (X = X + 1) in X. ǫ i is the random error, specific to each observation. Lecture #2-1/27/2005 Slide 5 of 46
6 Parameter Estimates Small Example The Basics The Basics Estimation Example Properties Chapter 2 parameterizes the linear regression model as: Y = a + bx + e To find estimates for a and b consider several possible choices: N So that e is minimized. So that i N e 2 is minimized. i From some guy in the hallway. rapping Up Lecture #2-1/27/2005 Slide 6 of 46
7 And The inner Is... Small Example The Basics The Basics Estimation Example Properties Finding a and b that minimize: N i e 2 Using calculus, these happen to be: b = (X X)(Y Ȳ ) (X X) 2 = r xy s y s x = xy x 2 And: rapping Up a = Ȳ b X LS estimates are considered BLUE: Best Linear Unbiased Estimators. Lecture #2-1/27/2005 Slide 7 of 46
8 An Example of Small Example The Basics The Basics Estimation Example Properties Table 2.1 in Pedhazur lists data from an experiment where X was the number of hours given for study, and Y is the score on a test Test Score 6.00 rapping Up Hours of Study Lecture #2-1/27/2005 Slide 8 of 46
9 Example (continued) Small Example The Basics The Basics Estimation Example Properties e can tell that: (X X) 2 = x 2 = 40 (X X)(Y Ȳ ) = xy = 30 X = 3.0 Ȳ = 7.3 So: b = xy x 2 = = 0.75 a = Ȳ b X = 7.3 ( ) = 5.05 Given these estimates, the linear regression line is given by: Y = X rapping Up Lecture #2-1/27/2005 Slide 9 of 46
10 Example (continued) Small Example The Basics The Basics Estimation Example Properties Test Score = * X R Square = 0.17 Test Score rapping Up Hours of Study Lecture #2-1/27/2005 Slide 10 of 46
11 Properties of the LS Estimates Notice: Small Example The Basics The Basics Estimation Example Properties Y = a + bx = (Ȳ b X) + bx = This means when X = X, Y = Ȳ. Ȳ + b(x X) Furthermore, this implies that for every simple linear regression, the point ( X, Ȳ ) will always fall on the regression line. Furthermore, if b = 0 (no relation between X and Y ), the best guess (in a LS sense) for a value of Y would be Ȳ. rapping Up Lecture #2-1/27/2005 Slide 11 of 46
12 (Y Ȳ ) 2 = ss reg + ss res How Good Is Our Line? Model Accuracy Sums of Squares Example Variance Accounted For VAC Example You ve fit the model...you have your estimates...now what? e need to determine how well Y is predicted by X. Specifically, how much variability in Y can be accounted for by the linear regression where X is a predictor? A math adventure: Y = Ȳ + (Y Ȳ ) + (Y Y ) Y Ȳ = (Y Ȳ ) + (Y Y ) (Y Ȳ ) 2 = (Y Ȳ )2 + (Y Y ) 2 rapping Up Lecture #2-1/27/2005 Slide 12 of 46
13 Sums of Squares Model Accuracy Sums of Squares Example Variance Accounted For VAC Example The Sums of Squares for our dependent variable, Y, is denoted in Pedhazur by y 2. As shown before, in a regression, these can be partitioned into two components: Part due to the regression line: ss reg rapping Up Part due to error: ss res Dividing both components by y 2 gives the proportion of sum of squares due accounted for by the regression and the proportion accounted for by error. Lecture #2-1/27/2005 Slide 13 of 46
14 Example Computation Model Accuracy Sums of Squares Example Variance Accounted For VAC Example In our studying v. test score example from before, we know that: y 2 = b = 0.75 xy= 30 To compute ss reg, one of several formulas can be used: ss reg = ( xy) 2 x 2 = b xy = b 2 x 2 = = 22.5 rapping Up ss res = y 2 ss reg = = Lecture #2-1/27/2005 Slide 14 of 46
15 Example Continued Dividing both by y 2 gives: Proportion explained by regression: Model Accuracy Sums of Squares Example Variance Accounted For VAC Example Proportion explained by error: rapping Up Lecture #2-1/27/2005 Slide 15 of 46
16 Variance Accounted For Model Accuracy Sums of Squares Example Variance Accounted For VAC Example Instead of approaching fit via Sums of Squares, one could look at it as a spinoff of the Pearson correlation: r 2 xy = ( xy) 2 x 2 y 2 ss reg = r 2 xy y 2 rapping Up Lecture #2-1/27/2005 Slide 16 of 46
17 Variance Accounted For - Example From our studying v. test score example from before, we know that: Model Accuracy Sums of Squares Example Variance Accounted For VAC Example So, r 2 xy = r xy = hich is the percent of variance accounted for by the regression. Therefore, 1 r 2 xy = 0.827; the percent of variance due to error, or unexplained variance. rapping Up Lecture #2-1/27/2005 Slide 17 of 46
18 Statistical Significance Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test rapping Up Statistical significance is literally the likelihood of the null hypothesis in a hypothesis test. H 0 : b = 0 H 1 : b 0 Statistical significance is not related to the meaningfulness of the result. Take b = ith se(b) = b is statistically significant but in reality, there is a negligible relationship between X and Y. Lecture #2-1/27/2005 Slide 18 of 46
19 Hypothesis Tests in Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test There are several types of hypothesis tests that can be used in simple linear regression. Most of these tests are all related: a test for a non-zero relationship between two variables, X and Y. Today, we have covered three different measures of association: b -ss reg r 2 r xy rapping Up Lecture #2-1/27/2005 Slide 19 of 46
20 Hypothesis Tests in Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test Each of the following tests evolve statistically from placing distributional assumptions on the disturbance term. Y = a + bx + e Typically, we say that errors follow a normal distribution: e N(0, σ 2 e) By placing distributional assumptions Sampling distributions are defined. Hypothesis tests are enabled. must be recognized and checked for validity. rapping Up Lecture #2-1/27/2005 Slide 20 of 46
21 Testing the Entire Regression Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test A test of all meaningful regression parameters is as follows: H 0 : b = 0 H 1 : b 0 Because we only have one predictor variable, this is very straight forward. The test statistic is F -distributed, and is computed by: F = ss reg df reg ss res df res = ss reg k ss res N k 1 F-table gives significance for a given Type I error rate α, OR USE THE COMPUTER. rapping Up Lecture #2-1/27/2005 Slide 21 of 46
22 Testing the Entire Regression - Example From our example, the test of the entire regression is: H 0 : b = 0 H 1 : b 0 Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test rapping Up F = ss reg df reg ss res df res = ss reg k ss res N k 1 = = 3.76 Typing =fdist(3.76,1,18) into MS Excel gives a p-value of e could infer that there is no statistical evidence to suggest b is different from 0. This means there is no statistical relationship between hours of study and the score on a test. Lecture #2-1/27/2005 Slide 22 of 46
23 Testing the VAC Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test Because of the relationship of r 2 xy with ss reg the previous test could be phrased as: F = r 2 xy df reg 1 r 2 xy df res = r 2 xy k 1 r 2 xy N k 1 = = 3.76 Notice this is identical to the F we received before. Therefore our conclusions are the same... rapping Up Lecture #2-1/27/2005 Slide 23 of 46
24 Testing the Regression Coefficient Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test Finally, the regression coefficient between X and Y can tested for difference from zero: H 0 : b = 0 H 1 : b 0 The variance of Y conditional on X is given by: (Y Y s 2 ) y.x = N k 1 = ss res N k 1 The standard error of the regression coefficient, s b, is based on the conditional variance of Y X: s 2 y.x s b = x 2 rapping Up Lecture #2-1/27/2005 Slide 24 of 46
25 Testing the Regression Coefficient Finally, the test statistic for the following hypothesis test is: H 0 : b = 0 H 1 : b 0 Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test t = b s b This statistic is t distributed, with N-k-1 degrees of freedom (which is the denominator of s 2 y.x). rapping Up Lecture #2-1/27/2005 Slide 25 of 46
26 Testing the Regression Coefficient - Example From our example, the test of the regression coefficient is: H 0 : b = 0 H 1 : b 0 Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test s 2 y.x = (Y Y ) N k 1 = ss res N k 1 = x 2 = 40 = rapping Up t = b s 2 y.x x 2 = = 1.94 Typing in =tdist(1.94,18,2) (2 for a two-tailed test), gives a p-value of the same as before. Recall F = t 2 from somewhere before... Lecture #2-1/27/2005 Slide 26 of 46
27 Testing the Regression Coefficient Statistical Significance Hypothesis Tests Regression Test VAC Test Coefficient Test In our example, we tested against zero. In reality, any value can be tested against: H 0 : b = B H 1 : b B t = b B s 2 y.x x 2 rapping Up Lecture #2-1/27/2005 Slide 27 of 46
28 Key Behind Basic assumptions: X is fixed - measured without error. The mean of Y X has a linear relationship with X Error assumptions: Mean of errors is zero. All errors independent. The variance of the errors does not change as a function of X - homoscedasticity. All errors are uncorrelated with X. Statistically speaking, errors are IID: e N(0, σ 2 e). rapping Up Lecture #2-1/27/2005 Slide 28 of 46
29 : ays of Checking An easy way to check many error based assumptions a visualization: A plot of the standardized residual versus the predicted value. The next class will focus on other diagnostic measures. Example rapping Up Lecture #2-1/27/2005 Slide 29 of 46
30 Z Curvilinear Example hat if we were to fit a linear regression to these data? Example rapping Up Hours of Study Lecture #2-1/27/2005 Slide 30 of 46
31 Z Curvilinear Example Fitting the simple linear regression model gives: Example rapping Up 2.50 Z = * X R Square = Hours of Study Not good? Lecture #2-1/27/2005 Slide 31 of 46
32 Curvilinear Example Let s take a look at a plot of the standardized residuals versus the predicted values: Example Standardized Residual rapping Up Unstandardized Predicted Value Lecture #2-1/27/2005 Slide 32 of 46
33 hen X is Random Commonly, we have no control over our predictor variable. Observational Studies Statistically speaking, having X being random adds: The potential for measurement error in X - reliability. Limitations to the causal inferences that can be drawn from a regression analysis. All test statistics and measures are used equivalently. e can predict Y from X or X from Y. Correlations are more common place: common variance rather than causal influence. rapping Up Lecture #2-1/27/2005 Slide 33 of 46
34 rap Up Example Example taken from eisberg (1985, p. 230): Old Faithful Scatterplot Descriptives Regression Line Hypothesis Tests Perhaps the most famous geyser is Old Faithful, in Yellowstone National Park, yoming. The intervals of eruptions of Old Faithful range from about 30 to 90 minutes. ater shoots to heights generally over 35 meters, with eruptions lasting from 1 to 5.5 minutes. Data on Old Faithful has been collected for many years by ranger/naturalists in the park, using a stopwatch. The duration measurements have been rounded to the nearest 0.1 minute or 6 seconds, while intervals reported are to the nearest minute. The National Park Service uses x (values of the duration of an eruption) to predict y (the interval to the next eruption). rapping Up Lecture #2-1/27/2005 Slide 34 of 46
35 Old Faithful Postcard Lecture #2-1/27/2005 Slide 35 of 46
36 Old Faithful Scatterplot Old Faithful Scatterplot Descriptives Regression Line Hypothesis Tests Interval to Next Eruption rapping Up Duration of Eruption Lecture #2-1/27/2005 Slide 36 of 46
37 Descriptive Statistics Old Faithful Scatterplot Descriptives Regression Line Hypothesis Tests Variable N Mean SD x - Duration y - Interval r xy = r 2 xy = x 2 = y 2 = 17, xy = 1, rapping Up Lecture #2-1/27/2005 Slide 37 of 46
38 Regression Line b = xy x 2 = 1, = a = Ȳ b X = 71.0 (10.74) = Y = X Old Faithful Scatterplot Descriptives Regression Line Hypothesis Tests rapping Up Lecture #2-1/27/2005 Slide 38 of 46
39 Regression Line Old Faithful Scatterplot Descriptives Regression Line Hypothesis Tests rapping Up Interval to Next Eruption Interval to Next Eruption = * X R Square = Duration of Eruption Lecture #2-1/27/2005 Slide 39 of 46
40 Hypothesis Tests ss reg = b xy = , = 13, ss res = y 2 ss reg = 17, , = 4, F = ss reg k ss res N k 1 = 13, /1 4, /( ) = Old Faithful Scatterplot Descriptives Regression Line Hypothesis Tests rapping Up Lecture #2-1/27/2005 Slide 40 of 46
41 Residual Plot Old Faithful Scatterplot Descriptives Regression Line Hypothesis Tests Standardized Residual rapping Up Unstandardized Predicted Value Lecture #2-1/27/2005 Slide 41 of 46
42 Data Data More Data Scatterplots rapping Up Lecture #2-1/27/2005 Slide 42 of 46
43 Data Data More Data Scatterplots rapping Up Lecture #2-1/27/2005 Slide 43 of 46
44 Plots from Today Scatterplots were made with the interactive graphs: Graphs...Interactive...Scatterplot. Data More Data Scatterplots rapping Up Lecture #2-1/27/2005 Slide 44 of 46
45 Analyze...Regression...Linear. Data More Data Scatterplots rapping Up Lecture #2-1/27/2005 Slide 45 of 46
46 Saving Residuals Analyze...Regression...Linear Save Button. Data More Data Scatterplots rapping Up Lecture #2-1/27/2005 Slide 46 of 46
47 Final Thought rapping Up Final Thought Next Class Simple linear regression can be simple and straight forward. Need to understand difference between statistical significance and practical relevance. The tests of significance and model estimates are only valid if all assumptions are met. Regression is fairly robust to violations of assumptions, but... Usually when assumptions are violated, something else is wrong with the model. Listen to your data, they are trying to tell you something. Lecture #2-1/27/2005 Slide 47 of 46
48 Next Time Regression diagnostics - Chapter 3. More. More practice with fitting models. QUESTION: Do you have any data you are willing to share with class? rapping Up Final Thought Next Class Lecture #2-1/27/2005 Slide 48 of 46
Advanced Regression Topics: Violation of Assumptions
Advanced Regression Topics: Violation of Assumptions Lecture 7 February 15, 2005 Applied Regression Analysis Lecture #7-2/15/2005 Slide 1 of 36 Today s Lecture Today s Lecture rapping Up Revisiting residuals.
More informationRegression With a Categorical Independent Variable
Regression ith a Independent Variable ERSH 8320 Slide 1 of 34 Today s Lecture Regression with a single categorical independent variable. Today s Lecture Coding procedures for analysis. Dummy coding. Relationship
More informationRegression With a Categorical Independent Variable
Regression With a Independent Variable Lecture 10 November 5, 2008 ERSH 8320 Lecture #10-11/5/2008 Slide 1 of 54 Today s Lecture Today s Lecture Chapter 11: Regression with a single categorical independent
More informationRegression With a Categorical Independent Variable
Regression With a Categorical Independent Variable Lecture 15 March 17, 2005 Applied Regression Analysis Lecture #15-3/17/2005 Slide 1 of 29 Today s Lecture» Today s Lecture» Midterm Note» Example Regression
More informationASSIGNMENT 3 SIMPLE LINEAR REGRESSION. Old Faithful
ASSIGNMENT 3 SIMPLE LINEAR REGRESSION In the simple linear regression model, the mean of a response variable is a linear function of an explanatory variable. The model and associated inferential tools
More informationInferences for Regression
Inferences for Regression An Example: Body Fat and Waist Size Looking at the relationship between % body fat and waist size (in inches). Here is a scatterplot of our data set: Remembering Regression In
More informationScatter plot of data from the study. Linear Regression
1 2 Linear Regression Scatter plot of data from the study. Consider a study to relate birthweight to the estriol level of pregnant women. The data is below. i Weight (g / 100) i Weight (g / 100) 1 7 25
More informationSimple Linear Regression
Simple Linear Regression ST 430/514 Recall: A regression model describes how a dependent variable (or response) Y is affected, on average, by one or more independent variables (or factors, or covariates)
More informationSimple Linear Regression
Simple Linear Regression 1 Correlation indicates the magnitude and direction of the linear relationship between two variables. Linear Regression: variable Y (criterion) is predicted by variable X (predictor)
More informationREVIEW 8/2/2017 陈芳华东师大英语系
REVIEW Hypothesis testing starts with a null hypothesis and a null distribution. We compare what we have to the null distribution, if the result is too extreme to belong to the null distribution (p
More informationRegression With a Categorical Independent Variable: Mean Comparisons
Regression With a Categorical Independent Variable: Mean Lecture 16 March 29, 2005 Applied Regression Analysis Lecture #16-3/29/2005 Slide 1 of 43 Today s Lecture comparisons among means. Today s Lecture
More informationReview 6. n 1 = 85 n 2 = 75 x 1 = x 2 = s 1 = 38.7 s 2 = 39.2
Review 6 Use the traditional method to test the given hypothesis. Assume that the samples are independent and that they have been randomly selected ) A researcher finds that of,000 people who said that
More informationMORE ON SIMPLE REGRESSION: OVERVIEW
FI=NOT0106 NOTICE. Unless otherwise indicated, all materials on this page and linked pages at the blue.temple.edu address and at the astro.temple.edu address are the sole property of Ralph B. Taylor and
More informationChapter 16. Simple Linear Regression and dcorrelation
Chapter 16 Simple Linear Regression and dcorrelation 16.1 Regression Analysis Our problem objective is to analyze the relationship between interval variables; regression analysis is the first tool we will
More informationScatter plot of data from the study. Linear Regression
1 2 Linear Regression Scatter plot of data from the study. Consider a study to relate birthweight to the estriol level of pregnant women. The data is below. i Weight (g / 100) i Weight (g / 100) 1 7 25
More informationLecture 2 Simple Linear Regression STAT 512 Spring 2011 Background Reading KNNL: Chapter 1
Lecture Simple Linear Regression STAT 51 Spring 011 Background Reading KNNL: Chapter 1-1 Topic Overview This topic we will cover: Regression Terminology Simple Linear Regression with a single predictor
More informationLecture Slides. Elementary Statistics Tenth Edition. by Mario F. Triola. and the Triola Statistics Series. Slide 1
Lecture Slides Elementary Statistics Tenth Edition and the Triola Statistics Series by Mario F. Triola Slide 1 Chapter 10 Correlation and Regression 10-1 Overview 10-2 Correlation 10-3 Regression 10-4
More informationSix Sigma Black Belt Study Guides
Six Sigma Black Belt Study Guides 1 www.pmtutor.org Powered by POeT Solvers Limited. Analyze Correlation and Regression Analysis 2 www.pmtutor.org Powered by POeT Solvers Limited. Variables and relationships
More informationLecture 14 Simple Linear Regression
Lecture 4 Simple Linear Regression Ordinary Least Squares (OLS) Consider the following simple linear regression model where, for each unit i, Y i is the dependent variable (response). X i is the independent
More informationLinear Regression & Correlation
Linear Regression & Correlation Jamie Monogan University of Georgia Introduction to Data Analysis Jamie Monogan (UGA) Linear Regression & Correlation POLS 7012 1 / 25 Objectives By the end of these meetings,
More informationMultiple Regression. More Hypothesis Testing. More Hypothesis Testing The big question: What we really want to know: What we actually know: We know:
Multiple Regression Ψ320 Ainsworth More Hypothesis Testing What we really want to know: Is the relationship in the population we have selected between X & Y strong enough that we can use the relationship
More informationdetermine whether or not this relationship is.
Section 9-1 Correlation A correlation is a between two. The data can be represented by ordered pairs (x,y) where x is the (or ) variable and y is the (or ) variable. There are several types of correlations
More informationBusiness Statistics. Lecture 9: Simple Regression
Business Statistics Lecture 9: Simple Regression 1 On to Model Building! Up to now, class was about descriptive and inferential statistics Numerical and graphical summaries of data Confidence intervals
More informationPsychology 282 Lecture #4 Outline Inferences in SLR
Psychology 282 Lecture #4 Outline Inferences in SLR Assumptions To this point we have not had to make any distributional assumptions. Principle of least squares requires no assumptions. Can use correlations
More informationRegression: Main Ideas Setting: Quantitative outcome with a quantitative explanatory variable. Example, cont.
TCELL 9/4/205 36-309/749 Experimental Design for Behavioral and Social Sciences Simple Regression Example Male black wheatear birds carry stones to the nest as a form of sexual display. Soler et al. wanted
More informationAMS 7 Correlation and Regression Lecture 8
AMS 7 Correlation and Regression Lecture 8 Department of Applied Mathematics and Statistics, University of California, Santa Cruz Suumer 2014 1 / 18 Correlation pairs of continuous observations. Correlation
More informationCh 2: Simple Linear Regression
Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component
More informationStatistical Distribution Assumptions of General Linear Models
Statistical Distribution Assumptions of General Linear Models Applied Multilevel Models for Cross Sectional Data Lecture 4 ICPSR Summer Workshop University of Colorado Boulder Lecture 4: Statistical Distributions
More informationChapter 16. Simple Linear Regression and Correlation
Chapter 16 Simple Linear Regression and Correlation 16.1 Regression Analysis Our problem objective is to analyze the relationship between interval variables; regression analysis is the first tool we will
More informationRegression Models - Introduction
Regression Models - Introduction In regression models there are two types of variables that are studied: A dependent variable, Y, also called response variable. It is modeled as random. An independent
More informationSimple linear regression
Simple linear regression Biometry 755 Spring 2008 Simple linear regression p. 1/40 Overview of regression analysis Evaluate relationship between one or more independent variables (X 1,...,X k ) and a single
More informationTHE PEARSON CORRELATION COEFFICIENT
CORRELATION Two variables are said to have a relation if knowing the value of one variable gives you information about the likely value of the second variable this is known as a bivariate relation There
More informationTutorial 6: Linear Regression
Tutorial 6: Linear Regression Rob Nicholls nicholls@mrc-lmb.cam.ac.uk MRC LMB Statistics Course 2014 Contents 1 Introduction to Simple Linear Regression................ 1 2 Parameter Estimation and Model
More informationy = a + bx 12.1: Inference for Linear Regression Review: General Form of Linear Regression Equation Review: Interpreting Computer Regression Output
12.1: Inference for Linear Regression Review: General Form of Linear Regression Equation y = a + bx y = dependent variable a = intercept b = slope x = independent variable Section 12.1 Inference for Linear
More information36-309/749 Experimental Design for Behavioral and Social Sciences. Sep. 22, 2015 Lecture 4: Linear Regression
36-309/749 Experimental Design for Behavioral and Social Sciences Sep. 22, 2015 Lecture 4: Linear Regression TCELL Simple Regression Example Male black wheatear birds carry stones to the nest as a form
More informationQuantitative Techniques - Lecture 8: Estimation
Quantitative Techniques - Lecture 8: Estimation Key words: Estimation, hypothesis testing, bias, e ciency, least squares Hypothesis testing when the population variance is not known roperties of estimates
More informationBasic Business Statistics 6 th Edition
Basic Business Statistics 6 th Edition Chapter 12 Simple Linear Regression Learning Objectives In this chapter, you learn: How to use regression analysis to predict the value of a dependent variable based
More informationRegression. Marc H. Mehlman University of New Haven
Regression Marc H. Mehlman marcmehlman@yahoo.com University of New Haven the statistician knows that in nature there never was a normal distribution, there never was a straight line, yet with normal and
More informationWed, June 26, (Lecture 8-2). Nonlinearity. Significance test for correlation R-squared, SSE, and SST. Correlation in SPSS.
Wed, June 26, (Lecture 8-2). Nonlinearity. Significance test for correlation R-squared, SSE, and SST. Correlation in SPSS. Last time, we looked at scatterplots, which show the interaction between two variables,
More informationy response variable x 1, x 2,, x k -- a set of explanatory variables
11. Multiple Regression and Correlation y response variable x 1, x 2,, x k -- a set of explanatory variables In this chapter, all variables are assumed to be quantitative. Chapters 12-14 show how to incorporate
More informationKeller: Stats for Mgmt & Econ, 7th Ed July 17, 2006
Chapter 17 Simple Linear Regression and Correlation 17.1 Regression Analysis Our problem objective is to analyze the relationship between interval variables; regression analysis is the first tool we will
More information23. Inference for regression
23. Inference for regression The Practice of Statistics in the Life Sciences Third Edition 2014 W. H. Freeman and Company Objectives (PSLS Chapter 23) Inference for regression The regression model Confidence
More informationBusiness Statistics. Lecture 10: Correlation and Linear Regression
Business Statistics Lecture 10: Correlation and Linear Regression Scatterplot A scatterplot shows the relationship between two quantitative variables measured on the same individuals. It displays the Form
More informationINFERENCE FOR REGRESSION
CHAPTER 3 INFERENCE FOR REGRESSION OVERVIEW In Chapter 5 of the textbook, we first encountered regression. The assumptions that describe the regression model we use in this chapter are the following. We
More informationLecture 2 Linear Regression: A Model for the Mean. Sharyn O Halloran
Lecture 2 Linear Regression: A Model for the Mean Sharyn O Halloran Closer Look at: Linear Regression Model Least squares procedure Inferential tools Confidence and Prediction Intervals Assumptions Robustness
More informationCorrelation & Simple Regression
Chapter 11 Correlation & Simple Regression The previous chapter dealt with inference for two categorical variables. In this chapter, we would like to examine the relationship between two quantitative variables.
More information1 Correlation and Inference from Regression
1 Correlation and Inference from Regression Reading: Kennedy (1998) A Guide to Econometrics, Chapters 4 and 6 Maddala, G.S. (1992) Introduction to Econometrics p. 170-177 Moore and McCabe, chapter 12 is
More informationChapter 1: Linear Regression with One Predictor Variable also known as: Simple Linear Regression Bivariate Linear Regression
BSTT523: Kutner et al., Chapter 1 1 Chapter 1: Linear Regression with One Predictor Variable also known as: Simple Linear Regression Bivariate Linear Regression Introduction: Functional relation between
More informationBIOSTATISTICS NURS 3324
Simple Linear Regression and Correlation Introduction Previously, our attention has been focused on one variable which we designated by x. Frequently, it is desirable to learn something about the relationship
More informationCan you tell the relationship between students SAT scores and their college grades?
Correlation One Challenge Can you tell the relationship between students SAT scores and their college grades? A: The higher SAT scores are, the better GPA may be. B: The higher SAT scores are, the lower
More informationSimple Linear Regression. (Chs 12.1, 12.2, 12.4, 12.5)
10 Simple Linear Regression (Chs 12.1, 12.2, 12.4, 12.5) Simple Linear Regression Rating 20 40 60 80 0 5 10 15 Sugar 2 Simple Linear Regression Rating 20 40 60 80 0 5 10 15 Sugar 3 Simple Linear Regression
More informationChapter 4: Regression Models
Sales volume of company 1 Textbook: pp. 129-164 Chapter 4: Regression Models Money spent on advertising 2 Learning Objectives After completing this chapter, students will be able to: Identify variables,
More informationMathematics for Economics MA course
Mathematics for Economics MA course Simple Linear Regression Dr. Seetha Bandara Simple Regression Simple linear regression is a statistical method that allows us to summarize and study relationships between
More informationSimple Linear Regression
Simple Linear Regression EdPsych 580 C.J. Anderson Fall 2005 Simple Linear Regression p. 1/80 Outline 1. What it is and why it s useful 2. How 3. Statistical Inference 4. Examining assumptions (diagnostics)
More informationCorrelation Analysis
Simple Regression Correlation Analysis Correlation analysis is used to measure strength of the association (linear relationship) between two variables Correlation is only concerned with strength of the
More informationSTAT 135 Lab 11 Tests for Categorical Data (Fisher s Exact test, χ 2 tests for Homogeneity and Independence) and Linear Regression
STAT 135 Lab 11 Tests for Categorical Data (Fisher s Exact test, χ 2 tests for Homogeneity and Independence) and Linear Regression Rebecca Barter April 20, 2015 Fisher s Exact Test Fisher s Exact Test
More informationSSR = The sum of squared errors measures how much Y varies around the regression line n. It happily turns out that SSR + SSE = SSTO.
Analysis of variance approach to regression If x is useless, i.e. β 1 = 0, then E(Y i ) = β 0. In this case β 0 is estimated by Ȳ. The ith deviation about this grand mean can be written: deviation about
More informationChapter 11 Linear Regression
Chapter 11 Linear Regression Linear regression is a methodology that allows us to examine the relationship between two continuously measured variables where we believe that values of one variable may influence
More informationDensity Temp vs Ratio. temp
Temp Ratio Density 0.00 0.02 0.04 0.06 0.08 0.10 0.12 Density 0.0 0.2 0.4 0.6 0.8 1.0 1. (a) 170 175 180 185 temp 1.0 1.5 2.0 2.5 3.0 ratio The histogram shows that the temperature measures have two peaks,
More informationCorrelation 1. December 4, HMS, 2017, v1.1
Correlation 1 December 4, 2017 1 HMS, 2017, v1.1 Chapter References Diez: Chapter 7 Navidi, Chapter 7 I don t expect you to learn the proofs what will follow. Chapter References 2 Correlation The sample
More informationSimple Linear Regression
Simple Linear Regression OI CHAPTER 7 Important Concepts Correlation (r or R) and Coefficient of determination (R 2 ) Interpreting y-intercept and slope coefficients Inference (hypothesis testing and confidence
More informationStatistics for Engineers Lecture 9 Linear Regression
Statistics for Engineers Lecture 9 Linear Regression Chong Ma Department of Statistics University of South Carolina chongm@email.sc.edu April 17, 2017 Chong Ma (Statistics, USC) STAT 509 Spring 2017 April
More informationRegression Models. Chapter 4. Introduction. Introduction. Introduction
Chapter 4 Regression Models Quantitative Analysis for Management, Tenth Edition, by Render, Stair, and Hanna 008 Prentice-Hall, Inc. Introduction Regression analysis is a very valuable tool for a manager
More information28. SIMPLE LINEAR REGRESSION III
28. SIMPLE LINEAR REGRESSION III Fitted Values and Residuals To each observed x i, there corresponds a y-value on the fitted line, y = βˆ + βˆ x. The are called fitted values. ŷ i They are the values of
More informationDr. Junchao Xia Center of Biophysics and Computational Biology. Fall /1/2016 1/46
BIO5312 Biostatistics Lecture 10:Regression and Correlation Methods Dr. Junchao Xia Center of Biophysics and Computational Biology Fall 2016 11/1/2016 1/46 Outline In this lecture, we will discuss topics
More informationChapter 4. Regression Models. Learning Objectives
Chapter 4 Regression Models To accompany Quantitative Analysis for Management, Eleventh Edition, by Render, Stair, and Hanna Power Point slides created by Brian Peterson Learning Objectives After completing
More informationRegression Analysis: Basic Concepts
The simple linear model Regression Analysis: Basic Concepts Allin Cottrell Represents the dependent variable, y i, as a linear function of one independent variable, x i, subject to a random disturbance
More informationAP Statistics Unit 6 Note Packet Linear Regression. Scatterplots and Correlation
Scatterplots and Correlation Name Hr A scatterplot shows the relationship between two quantitative variables measured on the same individuals. variable (y) measures an outcome of a study variable (x) may
More informationy n 1 ( x i x )( y y i n 1 i y 2
STP3 Brief Class Notes Instructor: Ela Jackiewicz Chapter Regression and Correlation In this chapter we will explore the relationship between two quantitative variables, X an Y. We will consider n ordered
More informationRegression Models - Introduction
Regression Models - Introduction In regression models, two types of variables that are studied: A dependent variable, Y, also called response variable. It is modeled as random. An independent variable,
More informationAMS 315/576 Lecture Notes. Chapter 11. Simple Linear Regression
AMS 315/576 Lecture Notes Chapter 11. Simple Linear Regression 11.1 Motivation A restaurant opening on a reservations-only basis would like to use the number of advance reservations x to predict the number
More informationCS 5014: Research Methods in Computer Science
Computer Science Clifford A. Shaffer Department of Computer Science Virginia Tech Blacksburg, Virginia Fall 2010 Copyright c 2010 by Clifford A. Shaffer Computer Science Fall 2010 1 / 207 Correlation and
More informationRegression and correlation. Correlation & Regression, I. Regression & correlation. Regression vs. correlation. Involve bivariate, paired data, X & Y
Regression and correlation Correlation & Regression, I 9.07 4/1/004 Involve bivariate, paired data, X & Y Height & weight measured for the same individual IQ & exam scores for each individual Height of
More informationSociology 6Z03 Review II
Sociology 6Z03 Review II John Fox McMaster University Fall 2016 John Fox (McMaster University) Sociology 6Z03 Review II Fall 2016 1 / 35 Outline: Review II Probability Part I Sampling Distributions Probability
More informationChapter 1 Statistical Inference
Chapter 1 Statistical Inference causal inference To infer causality, you need a randomized experiment (or a huge observational study and lots of outside information). inference to populations Generalizations
More informationPS2: Two Variable Statistics
PS2: Two Variable Statistics LT2: Measuring Correlation and Line of best fit by eye. LT3: Linear regression LT4: The χ 2 test of independence. 1 Pearson's Correlation Coefficient In examinations you are
More informationNature vs. nurture? Lecture 18 - Regression: Inference, Outliers, and Intervals. Regression Output. Conditions for inference.
Understanding regression output from software Nature vs. nurture? Lecture 18 - Regression: Inference, Outliers, and Intervals In 1966 Cyril Burt published a paper called The genetic determination of differences
More informationSection 3: Simple Linear Regression
Section 3: Simple Linear Regression Carlos M. Carvalho The University of Texas at Austin McCombs School of Business http://faculty.mccombs.utexas.edu/carlos.carvalho/teaching/ 1 Regression: General Introduction
More informationUnit 6 - Introduction to linear regression
Unit 6 - Introduction to linear regression Suggested reading: OpenIntro Statistics, Chapter 7 Suggested exercises: Part 1 - Relationship between two numerical variables: 7.7, 7.9, 7.11, 7.13, 7.15, 7.25,
More informationIntro to Linear Regression
Intro to Linear Regression Introduction to Regression Regression is a statistical procedure for modeling the relationship among variables to predict the value of a dependent variable from one or more predictor
More informationMultiple Regression. Dr. Frank Wood. Frank Wood, Linear Regression Models Lecture 12, Slide 1
Multiple Regression Dr. Frank Wood Frank Wood, fwood@stat.columbia.edu Linear Regression Models Lecture 12, Slide 1 Review: Matrix Regression Estimation We can solve this equation (if the inverse of X
More informationChapter 8. Linear Regression. The Linear Model. Fat Versus Protein: An Example. The Linear Model (cont.) Residuals
Chapter 8 Linear Regression Copyright 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide 8-1 Copyright 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Fat Versus
More informationChapter 8. Linear Regression. Copyright 2010 Pearson Education, Inc.
Chapter 8 Linear Regression Copyright 2010 Pearson Education, Inc. Fat Versus Protein: An Example The following is a scatterplot of total fat versus protein for 30 items on the Burger King menu: Copyright
More informationReview of Multiple Regression
Ronald H. Heck 1 Let s begin with a little review of multiple regression this week. Linear models [e.g., correlation, t-tests, analysis of variance (ANOVA), multiple regression, path analysis, multivariate
More informationChapter Learning Objectives. Regression Analysis. Correlation. Simple Linear Regression. Chapter 12. Simple Linear Regression
Chapter 12 12-1 North Seattle Community College BUS21 Business Statistics Chapter 12 Learning Objectives In this chapter, you learn:! How to use regression analysis to predict the value of a dependent
More informationChapter 9. Correlation and Regression
Chapter 9 Correlation and Regression Lesson 9-1/9-2, Part 1 Correlation Registered Florida Pleasure Crafts and Watercraft Related Manatee Deaths 100 80 60 40 20 0 1991 1993 1995 1997 1999 Year Boats in
More informationUNIT 12 ~ More About Regression
***SECTION 15.1*** The Regression Model When a scatterplot shows a relationship between a variable x and a y, we can use the fitted to the data to predict y for a given value of x. Now we want to do tests
More informationWarm-up Using the given data Create a scatterplot Find the regression line
Time at the lunch table Caloric intake 21.4 472 30.8 498 37.7 335 32.8 423 39.5 437 22.8 508 34.1 431 33.9 479 43.8 454 42.4 450 43.1 410 29.2 504 31.3 437 28.6 489 32.9 436 30.6 480 35.1 439 33.0 444
More informationLinear Regression. Linear Regression. Linear Regression. Did You Mean Association Or Correlation?
Did You Mean Association Or Correlation? AP Statistics Chapter 8 Be careful not to use the word correlation when you really mean association. Often times people will incorrectly use the word correlation
More informationChapter 27 Summary Inferences for Regression
Chapter 7 Summary Inferences for Regression What have we learned? We have now applied inference to regression models. Like in all inference situations, there are conditions that we must check. We can test
More informationLecture 3: Inference in SLR
Lecture 3: Inference in SLR STAT 51 Spring 011 Background Reading KNNL:.1.6 3-1 Topic Overview This topic will cover: Review of hypothesis testing Inference about 1 Inference about 0 Confidence Intervals
More informationCorrelation. A statistics method to measure the relationship between two variables. Three characteristics
Correlation Correlation A statistics method to measure the relationship between two variables Three characteristics Direction of the relationship Form of the relationship Strength/Consistency Direction
More informationThe Simple Linear Regression Model
The Simple Linear Regression Model Lesson 3 Ryan Safner 1 1 Department of Economics Hood College ECON 480 - Econometrics Fall 2017 Ryan Safner (Hood College) ECON 480 - Lesson 3 Fall 2017 1 / 77 Bivariate
More informationStatistics for Managers using Microsoft Excel 6 th Edition
Statistics for Managers using Microsoft Excel 6 th Edition Chapter 13 Simple Linear Regression 13-1 Learning Objectives In this chapter, you learn: How to use regression analysis to predict the value of
More informationAny of 27 linear and nonlinear models may be fit. The output parallels that of the Simple Regression procedure.
STATGRAPHICS Rev. 9/13/213 Calibration Models Summary... 1 Data Input... 3 Analysis Summary... 5 Analysis Options... 7 Plot of Fitted Model... 9 Predicted Values... 1 Confidence Intervals... 11 Observed
More informationChapter 8: Correlation & Regression
Chapter 8: Correlation & Regression We can think of ANOVA and the two-sample t-test as applicable to situations where there is a response variable which is quantitative, and another variable that indicates
More informationSLR output RLS. Refer to slr (code) on the Lecture Page of the class website.
SLR output RLS Refer to slr (code) on the Lecture Page of the class website. Old Faithful at Yellowstone National Park, WY: Simple Linear Regression (SLR) Analysis SLR analysis explores the linear association
More informationCorrelation and the Analysis of Variance Approach to Simple Linear Regression
Correlation and the Analysis of Variance Approach to Simple Linear Regression Biometry 755 Spring 2009 Correlation and the Analysis of Variance Approach to Simple Linear Regression p. 1/35 Correlation
More informationModule 03 Lecture 14 Inferential Statistics ANOVA and TOI
Introduction of Data Analytics Prof. Nandan Sudarsanam and Prof. B Ravindran Department of Management Studies and Department of Computer Science and Engineering Indian Institute of Technology, Madras Module
More informationCorrelation and simple linear regression S5
Basic medical statistics for clinical and eperimental research Correlation and simple linear regression S5 Katarzyna Jóźwiak k.jozwiak@nki.nl November 15, 2017 1/41 Introduction Eample: Brain size and
More information