R 2 and F -Tests and ANOVA
|
|
- Doris Lawson
- 5 years ago
- Views:
Transcription
1 R 2 and F -Tests and ANOVA December 6, Partition of Sums of Squares The distance from any point y i in a collection of data, to the mean of the data ȳ, is the deviation, written as y i ȳ. Definition. We define the following sum of squares SS T Total Sum of Squares. It measures total variation in y SS T (y i ȳ) 2 SS Res Residual Sum-of-Squares or Error Sum of Squares. It measures amount of residual variation in y SS Res e 2 i (y i ŷ i ) 2 SS Res is an aggregate measure of mis-fit of the line of best fit. SS R Sum of Squares due to Regression or Explained Sum of Squares. It measures amount of systematic variation in y due to the x y relationship SS R (ŷ i ȳ) 2 1
2 Figure 1: SST
3 Figure 2: SSRes
4 Figure 3: SSR
5 Then we have the following partition of the total sum of squares (y i y) 2 (y i y + ŷ i ŷ i ) 2 ((ŷ i ȳ) 2 + 2e i (ŷ i ȳ) + e 2 i ) (ŷ i ȳ) 2 + (ŷ i ȳ) 2 + (ŷ i ȳ) 2 + (ŷ i ȳ) 2 + e 2 i + 2 e 2 i + 2 ((ŷ i ȳ) + (y i ŷ i )) 2 }{{} e i e i (ŷ i ȳ) e i ( ˆβ 0 + ˆβ 1 x i y) e 2 i + 2( ˆβ 0 y) e 2 i e i }{{} 0 +2 ˆβ 1 e i x i }{{} 0
6 Therefore SS T SS R + SS Res Note: Partitioning of the sum of squared deviations into various components allows the overall variability in a dataset to be ascribed to different types or sources of variability, with the relative importance of each being quantified by the size of each component of the overall sum of squares. 2 R 2 Global Goodness-of-fit For a global measure of adequacy, we can reconsider the sums-of-squares decomposition. SS T SS Res + SS R If we want to get a global measure of how well x predicts y, we may consider the proportion of variation explained by the regression model which is called R 2 statistic. R 2 SS R SS T SS T SS Res SS T 1 SS Res SS T 2.1 Range Because we have
7 SS R (ŷ i ȳ) 2 (( ˆβ 0 + ˆβ 1 x i ) ( ˆβ 0 + ˆβ 1 x)) 2 ˆβ 2 1 ( cxy (x i x) 2 s 2 X n ˆβ 1 c XY ) 2 ns 2 X R 2 will be 0 when β 1 0. On the other hand, if all the residuals are zero, SS Res 0 then R 2 1. This marked out the limits of R 2 : a sample slope of 0 gives an R 2 of 0, the lowest possible, and all the data points falling exactly on a straight line gives an R 2 of 1, the largest possible. 2.2 Interpretation R 2 value tells you how much variation is explained by your model (whether the predictors are sufficient in explaining the variation in the response). So 0.1 R 2 means that your model explains 10% of variation within the data. The greater R 2 the better the model. Whereas p-value tells you about the hypothesis testing of the fit of the intercept-only model and your model are equal. So if the p-value is less than the significance level (usually 0.05) then your model fits the data well. (whether the predictors are necessary in explaining the variation in the response) Thus you have four scenarios: 1. low R 2 and low p-value (p-value α) 2. low R 2 and high p-value (p-value > α) 3. high R 2 and low p-value 4. high R 2 and high p-value
8 Interpretation: 1. means that your model does not explain much of variation of the data but it is significant (better than not having a model) 2. means that your model does not explain much of variation of the data and it is not significant (worst scenario) 3. means your model explains a lot of variation within the data and is significant (best scenario) 4. means that your model explains a lot of variation within the data but is not significant (model is worthless) 2.3 Adjusted R 2 We know that E Y X [SS Res X] σ 2 (n p) If we increase the model complexity by increasing the number of parameters in the model p n, the value of SS Res will approach 0, hence the value R 2 1 SS Res SS T approaches 1. This fact implies that the R 2 statistics does not account for the model complexity, so often the adjusted R 2 statistics is used R 2 Adj 1 SS Res/(n p) SS T /(n 1) If R 2 or RAdj 2 is near 1, then the predictor x explains a large proportion of the observed variation in response y, i.e. x is a good predictor of y. Note: When p 1, R 2 Adj R2. Adjusted R 2 is particularly useful in the feature selection stage of model building. We can get the relationship between R 2 and R 2 Adj
9 Therefore and hence R 2 Adj 1 SS Res/(n p) SS T /(n 1) SS Res SS T R 2 1 SS Res SS T n p n 1 (1 R2 Adj) 1 n p n 1 (1 R2 Adj) Figure 4: Six summary graphs. R 2 is an appropriate measure for a c, but inappropriate for d f.
10 2.4 Limitation What does R 2 converge to as n. The population version is R 2 Var[m(x)] Var[Y ] Var[β 0 + β 1 X] Var[β 0 + β 1 X + ɛ] Var[β 1X] Var[β 1 X + ɛ] β2 1Var[X] β 2 1Var[X] + σ 2 Since all our parameter estimates are consistent, and this formula is continuous in all the parameters, the R 2 we get from our estimate will converge on this limit. Unfortunately, a lot of myths about R 2 have become endemic in the scientific community, and it is vital at this point to immunize you against them. 1. The most fundamental is that R 2 does not measure goodness of fit. (a) R 2 can be arbitrarily low when the model is completely correct. By making Var[x] small, or σ 2 large, we drive R 2 towards 0, even when every assumption of the simple linear regression model is correct in every particular. (b) R 2 can be arbitrarily close to 1 when the model is totally wrong. There is, indeed, no limit to how high R 2 can get when the true model is nonlinear. All that s needed is for the slope of the best linear approximation to be non-zero, and for Var[X] to get big. 2. R 2 is also pretty useless as a measure of predictability. (a) R 2 says nothing about prediction error. R 2 can be anywhere between 0 and 1 just by changing the range of X. Mean squared error is a much better measure of how good predictions are; better yet are estimates of out-of-sample error which we ll cover later in the course.
11 (b) says nothing about interval forecasts. In particular, it gives us no idea how big prediction intervals, or confidence intervals for m(x), might be. 3. R 2 cannot be compared across data sets. 4. cannot be compared between a model with untransformed Y and one with transformed Y, or between different transformations of Y. 5. The one situation where R 2 can be compared is when different models are fit to the same data set with the same, untransformed response variable. Then increasing R 2 is the same as decreasing in-sample MSE. In that case, however, you might as well just compare the MSEs. 6. It is very common to say that R 2 is the fraction of variance explained by the regression. But it is also extremely easy to devise situations where R 2 is high even though neither one could possible explain the other. At this point, you might be wondering just what R 2 is good for what job it does that isn t better done by other tools. The only honest answer I can give you is that I have never found a situation where it helped at all. If I could design the regression curriculum from scratch, I would never mention it. Unfortunately, it lives on as a historical relic, so you need to know what it is, and what mis-understandings about it people suffer from. 2.5 The Correlation Coefficient As you know, the correlation coefficient between X and Y is ρ XY Cov[X, Y ] Var[X]Var[Y ] which lies between 1 and 1. It takes its extreme values when Y is a linear function of X. The sample version is ρ XY c XY s X s Y n (x i x)(y i ȳ) n (x i x) 2
12 ( ) 2 c We know that SS R XY s ns 2 2 X. Therefore the sample correlation is X ρ 2 XY c2 XY s 2 X s2 Y ( cxy s 2 X SS R n (y i ȳ) 2 SS R SS T R 2 ) 2 ns 2 X ns 2 Y Therefore R 2 equals to the sample correlation coefficient squared. 3 F -Tests and ANOVA The idea is to compare two models: H 0 : Y β 0 + ɛ versus Y β 0 + β 1 X + ɛ If we fit the first model, the least squares estimator is β 0 ȳ. The idea is to create a statistic that measures how much better the second model is than the first model. Consider now the F test statistic F obs SS R/(p 1) SS Res /(n p) MS R MS Res Under H 0, the statistic has a known distribution called the F distribution. This distribution depends on two parameters. These are called the degrees of freedom for the F distribution. We denote the distribution by F F (p 1, n p) F (1, n 2) (p 2) We reject H 0 if F is large enough. To quantify this, we set significance level α (0.01, 0.05, etc.) and reject H 0 if F obs > F α,p 1,n p (p 2)
13 where F α,p 1,n p is the 1 α quantile of the F -distribution with degree of freedom p 1, n p. Equivalent, we can conduct the test using p-value. The p-value is P (F > F obs ) where F F (1, n 2) and F obs is the actual observed value you compute from the data. We reject H 0 if p-value is less than α. In the olden days, people summarized the quantities used by F -test in a ANOVA table: Source SS df M S F Regression SS R p 1 MS R SS R /(p 1) Residual SS Res n p MS Res SS Res /(n p) SS R /(p 1) SS Res /(n p) Total SS T n 1 In deriving the F distribution, it is absolutely vital that all of the assumptions of the Gaussian-noise simple linear regression model hold: the true model must be linear, the noise must be Gaussian, the noise variance must be constant, the noise must be independent of X and independent across measurements. The only hypothesis being tested is whether, maintaining all these assumptions, we must reject the flat model β 1 in favor of a line at an angle. In particular, the test never doubts that the right model is a straight line. ANOVA is an historical relic. In serious applied work from the modern (say, post-1985) era, I have never seen any study where filling out an ANOVA table for a regression, etc., was at all important. 3.1 ANOVA in R The ANOVA table arranges the requires information in tabular form: Source SS df MS F Regression SS R p 1 SS R /(p 1) F Residual SS Res n p SS Res /(n p) Total SS T n 1
14 Source df SS MS F Regression p 1 SS R SS R /(p 1) F Residual n p SS Res SS Res /(n p) Total n 1 SS T Note that R swaps the positions of columns 1 and 2. In either case we have summation results in columns 1 and 2 SS R + SS Res SS T and (p 1) + (n p) (n 1) from the previous theory. R also adds the p-value from the test of the null hypothesis in a further final column. The easiest way to do this in R is to use the anova function. This will give you an analysis-of-variance table for the model. The actual object the function returns is an anova object, which is a special type of data frame. The columns record, respectively, degrees of freedom, sums of squares, mean squares, the actual F statistic, and the p value of the F statistic. What we ll care about will be the first row of this table, which will give us the test information for the slope on X. library(gamair) data(chicago) out lm(death ~ tmpd, data chicago) anova(out) ## Analysis of Variance Table ## ## Response: death ## Df Sum Sq Mean Sq F value Pr(>F) ## tmpd < 2.2e-16 ## Residuals ##
15 ## tmpd *** ## Residuals ## --- ## Signif. codes: ## 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Here is another example regarding F -test and ANOVA 1 > x<- RocketProp$Age 2 > y<- RocketProp$Strength 3 > n<- length (x) 4 > fit.rp <-lm(y x) 5 > summary ( fit.rp) 6 7 Call : 8 lm( formula y x) 9 10 Residuals : 11 Min 1 Q Median 3 Q Max Coefficients : 15 Estimate Std. Error t value Pr ( > t ) 16 ( Intercept ) < 2e -16 *** 17 x e -10 *** Signif. codes : 0 *** ** 0.01 * Residual standard error : on 18 degrees of freedom 22 Multiple R- squared : , Adjusted R- squared : F- statistic : on 1 and 18 DF, p- value : 1.643e -10 Line 23 contains the computation of the F statistic (165.4), the degrees of freedom values
16 p 2 and n p 18, and the p-value in the test of the hypothesis H 0 : β 1 0 ( ); the hypothesis is strongly rejected. 24 > anova ( fit.rp) 25 Analysis of Variance Table Response : y 28 Df Sum Sq Mean Sq F value Pr( > F) 29 x e -10 *** 30 Residuals Signif. codes : 0 *** ** 0.01 * > 34 > anova ( fit.rp )[,'Df '] 35 [1] > anova ( fit.rp )[,' Sum Sq '] 37 [1] > 39 > sum ( anova ( fit.rp )[,' Sum Sq ']) -(n -1)* var (y) 40 [1] e -10 The ANOVA table is contained between lines 25 and 32. The Regression entry is listed on line 29 as x. Note that the F statistic and p-values, listed in the columns headed F value and Pr(>F), are identical to the ones from the lm summary (line 23). 3.2 Equivalence of F -test and t-test for simple linear regression For simple linear regression, the test H 0 : β 1 0 H a : β 1 0 can be carried out using
17 t-test F -test as in this model Since by { } 2 [ ] T obs 2 ˆβ1 1 ŝe( ˆβ F obs 1 ) SS R (ŷ i ȳ) 2 (( ˆβ 0 + ˆβ 1 x i1 ) ( ˆβ 0 + ˆβ 1 x 1 )) 2 ˆβ 2 1 (x i1 x 1 ) 2 ˆβ 2 1ns 2 X and ŝe( ˆβ ˆσ 1 ) 2 (n 2)s 2 X we have F obs SS R/(p 1) SS Res /(n p) SS R/(2 1) SS Res /(n 2) SS R nˆσ 2 /(n 2) ˆβ 2 1ns 2 X ˆσ 2 The p-value for F -test is exactly the same as the p-value for t-test. Thus F > F obs T 2 > [ ] T1 obs 2 T > T obs 1 { ˆβ1 ŝe( ˆβ 1 ) } 2 [ ] T1 obs 2 Thus P (F > F obs ) P ( T > T obs 1 )
18 Therefore, the test results based on F and t statistics are the same. Note: this result is only true for simple linear regression model. 3.3 What the F -Test Really Tests Suppose first that we retain the null hypothesis, i.e., we do not find any significant share of variance associated with the regression. This could be because (i) the intercept-only model is right; (iii) β 1 0 but the test doesn t have enough power to detect departures from the null. We don t know which it is. There is also possibility that the real relationship is nonlinear, but the best linear approximation to it has slope (nearly) zero, in which case the F test will have no power to detect the nonlinearity. Suppose instead that we reject the null, intercept-only hypothesis. This does not mean that the simple linear model is right. It means that the latter model predicts better than the intercept-only model too much better to be due to chance. The simple linear regression model can be absolute garbage, with every single one of its assumptions flagrantly violated, and yet better than the model which makes all those assumptions and thinks the optimal slope is zero. Neither the F test of β 1 0 vs. β 1 0 nor the Wald/t test of the same hypothesis tell us anything about the correctness of the simple linear regression model. All these tests presume the simple linear regression model with Gaussian noise is true, and check a special case (flat line) against the general one (titled line). They do not test linearity, constant variance, lack of correlation, or Gaussianity.
Lecture 10: F -Tests, ANOVA and R 2
Lecture 10: F -Tests, ANOVA and R 2 1 ANOVA We saw that we could test the null hypothesis that β 1 0 using the statistic ( β 1 0)/ŝe. (Although I also mentioned that confidence intervals are generally
More informationVariance Decomposition and Goodness of Fit
Variance Decomposition and Goodness of Fit 1. Example: Monthly Earnings and Years of Education In this tutorial, we will focus on an example that explores the relationship between total monthly earnings
More informationCh 2: Simple Linear Regression
Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component
More informationLecture 6 Multiple Linear Regression, cont.
Lecture 6 Multiple Linear Regression, cont. BIOST 515 January 22, 2004 BIOST 515, Lecture 6 Testing general linear hypotheses Suppose we are interested in testing linear combinations of the regression
More informationSimple Linear Regression
Simple Linear Regression ST 370 Regression models are used to study the relationship of a response variable and one or more predictors. The response is also called the dependent variable, and the predictors
More informationSimple Linear Regression
Simple Linear Regression In simple linear regression we are concerned about the relationship between two variables, X and Y. There are two components to such a relationship. 1. The strength of the relationship.
More informationVariance Decomposition in Regression James M. Murray, Ph.D. University of Wisconsin - La Crosse Updated: October 04, 2017
Variance Decomposition in Regression James M. Murray, Ph.D. University of Wisconsin - La Crosse Updated: October 04, 2017 PDF file location: http://www.murraylax.org/rtutorials/regression_anovatable.pdf
More informationSimple Linear Regression
Simple Linear Regression MATH 282A Introduction to Computational Statistics University of California, San Diego Instructor: Ery Arias-Castro http://math.ucsd.edu/ eariasca/math282a.html MATH 282A University
More informationApplied Regression. Applied Regression. Chapter 2 Simple Linear Regression. Hongcheng Li. April, 6, 2013
Applied Regression Chapter 2 Simple Linear Regression Hongcheng Li April, 6, 2013 Outline 1 Introduction of simple linear regression 2 Scatter plot 3 Simple linear regression model 4 Test of Hypothesis
More informationSimple Linear Regression
Simple Linear Regression ST 430/514 Recall: A regression model describes how a dependent variable (or response) Y is affected, on average, by one or more independent variables (or factors, or covariates)
More informationLecture 11: Simple Linear Regression
Lecture 11: Simple Linear Regression Readings: Sections 3.1-3.3, 11.1-11.3 Apr 17, 2009 In linear regression, we examine the association between two quantitative variables. Number of beers that you drink
More informationLinear models and their mathematical foundations: Simple linear regression
Linear models and their mathematical foundations: Simple linear regression Steffen Unkel Department of Medical Statistics University Medical Center Göttingen, Germany Winter term 2018/19 1/21 Introduction
More informationInference for Regression
Inference for Regression Section 9.4 Cathy Poliak, Ph.D. cathy@math.uh.edu Office in Fleming 11c Department of Mathematics University of Houston Lecture 13b - 3339 Cathy Poliak, Ph.D. cathy@math.uh.edu
More informationEstimating σ 2. We can do simple prediction of Y and estimation of the mean of Y at any value of X.
Estimating σ 2 We can do simple prediction of Y and estimation of the mean of Y at any value of X. To perform inferences about our regression line, we must estimate σ 2, the variance of the error term.
More informationCorrelation and the Analysis of Variance Approach to Simple Linear Regression
Correlation and the Analysis of Variance Approach to Simple Linear Regression Biometry 755 Spring 2009 Correlation and the Analysis of Variance Approach to Simple Linear Regression p. 1/35 Correlation
More information22s:152 Applied Linear Regression. Take random samples from each of m populations.
22s:152 Applied Linear Regression Chapter 8: ANOVA NOTE: We will meet in the lab on Monday October 10. One-way ANOVA Focuses on testing for differences among group means. Take random samples from each
More informationRegression, Part I. - In correlation, it would be irrelevant if we changed the axes on our graph.
Regression, Part I I. Difference from correlation. II. Basic idea: A) Correlation describes the relationship between two variables, where neither is independent or a predictor. - In correlation, it would
More informationMS&E 226: Small Data
MS&E 226: Small Data Lecture 15: Examples of hypothesis tests (v5) Ramesh Johari ramesh.johari@stanford.edu 1 / 32 The recipe 2 / 32 The hypothesis testing recipe In this lecture we repeatedly apply the
More informationExample: Poisondata. 22s:152 Applied Linear Regression. Chapter 8: ANOVA
s:5 Applied Linear Regression Chapter 8: ANOVA Two-way ANOVA Used to compare populations means when the populations are classified by two factors (or categorical variables) For example sex and occupation
More informationCh 3: Multiple Linear Regression
Ch 3: Multiple Linear Regression 1. Multiple Linear Regression Model Multiple regression model has more than one regressor. For example, we have one response variable and two regressor variables: 1. delivery
More informationRegression. Marc H. Mehlman University of New Haven
Regression Marc H. Mehlman marcmehlman@yahoo.com University of New Haven the statistician knows that in nature there never was a normal distribution, there never was a straight line, yet with normal and
More informationBiostatistics 380 Multiple Regression 1. Multiple Regression
Biostatistics 0 Multiple Regression ORIGIN 0 Multiple Regression Multiple Regression is an extension of the technique of linear regression to describe the relationship between a single dependent (response)
More information22s:152 Applied Linear Regression. There are a couple commonly used models for a one-way ANOVA with m groups. Chapter 8: ANOVA
22s:152 Applied Linear Regression Chapter 8: ANOVA NOTE: We will meet in the lab on Monday October 10. One-way ANOVA Focuses on testing for differences among group means. Take random samples from each
More informationSchool of Mathematical Sciences. Question 1
School of Mathematical Sciences MTH5120 Statistical Modelling I Practical 8 and Assignment 7 Solutions Question 1 Figure 1: The residual plots do not contradict the model assumptions of normality, constant
More informationFigure 1: The fitted line using the shipment route-number of ampules data. STAT5044: Regression and ANOVA The Solution of Homework #2 Inyoung Kim
0.0 1.0 1.5 2.0 2.5 3.0 8 10 12 14 16 18 20 22 y x Figure 1: The fitted line using the shipment route-number of ampules data STAT5044: Regression and ANOVA The Solution of Homework #2 Inyoung Kim Problem#
More informationSimple and Multiple Linear Regression
Sta. 113 Chapter 12 and 13 of Devore March 12, 2010 Table of contents 1 Simple Linear Regression 2 Model Simple Linear Regression A simple linear regression model is given by Y = β 0 + β 1 x + ɛ where
More informationLectures on Simple Linear Regression Stat 431, Summer 2012
Lectures on Simple Linear Regression Stat 43, Summer 0 Hyunseung Kang July 6-8, 0 Last Updated: July 8, 0 :59PM Introduction Previously, we have been investigating various properties of the population
More informationComparing Nested Models
Comparing Nested Models ST 370 Two regression models are called nested if one contains all the predictors of the other, and some additional predictors. For example, the first-order model in two independent
More informationMultiple Linear Regression
Multiple Linear Regression Simple linear regression tries to fit a simple line between two variables Y and X. If X is linearly related to Y this explains some of the variability in Y. In most cases, there
More informationMath 423/533: The Main Theoretical Topics
Math 423/533: The Main Theoretical Topics Notation sample size n, data index i number of predictors, p (p = 2 for simple linear regression) y i : response for individual i x i = (x i1,..., x ip ) (1 p)
More informationSTAT420 Midterm Exam. University of Illinois Urbana-Champaign October 19 (Friday), :00 4:15p. SOLUTIONS (Yellow)
STAT40 Midterm Exam University of Illinois Urbana-Champaign October 19 (Friday), 018 3:00 4:15p SOLUTIONS (Yellow) Question 1 (15 points) (10 points) 3 (50 points) extra ( points) Total (77 points) Points
More informationStatistical Inference with Regression Analysis
Introductory Applied Econometrics EEP/IAS 118 Spring 2015 Steven Buck Lecture #13 Statistical Inference with Regression Analysis Next we turn to calculating confidence intervals and hypothesis testing
More informationSIMPLE REGRESSION ANALYSIS. Business Statistics
SIMPLE REGRESSION ANALYSIS Business Statistics CONTENTS Ordinary least squares (recap for some) Statistical formulation of the regression model Assessing the regression model Testing the regression coefficients
More informationSTAT 215 Confidence and Prediction Intervals in Regression
STAT 215 Confidence and Prediction Intervals in Regression Colin Reimer Dawson Oberlin College 24 October 2016 Outline Regression Slope Inference Partitioning Variability Prediction Intervals Reminder:
More informationTests of Linear Restrictions
Tests of Linear Restrictions 1. Linear Restricted in Regression Models In this tutorial, we consider tests on general linear restrictions on regression coefficients. In other tutorials, we examine some
More informationInferences for Regression
Inferences for Regression An Example: Body Fat and Waist Size Looking at the relationship between % body fat and waist size (in inches). Here is a scatterplot of our data set: Remembering Regression In
More informationMultiple Regression Introduction to Statistics Using R (Psychology 9041B)
Multiple Regression Introduction to Statistics Using R (Psychology 9041B) Paul Gribble Winter, 2016 1 Correlation, Regression & Multiple Regression 1.1 Bivariate correlation The Pearson product-moment
More informationECON3150/4150 Spring 2015
ECON3150/4150 Spring 2015 Lecture 3&4 - The linear regression model Siv-Elisabeth Skjelbred University of Oslo January 29, 2015 1 / 67 Chapter 4 in S&W Section 17.1 in S&W (extended OLS assumptions) 2
More informationInference. ME104: Linear Regression Analysis Kenneth Benoit. August 15, August 15, 2012 Lecture 3 Multiple linear regression 1 1 / 58
Inference ME104: Linear Regression Analysis Kenneth Benoit August 15, 2012 August 15, 2012 Lecture 3 Multiple linear regression 1 1 / 58 Stata output resvisited. reg votes1st spend_total incumb minister
More informationChapter 12 - Lecture 2 Inferences about regression coefficient
Chapter 12 - Lecture 2 Inferences about regression coefficient April 19th, 2010 Facts about slope Test Statistic Confidence interval Hypothesis testing Test using ANOVA Table Facts about slope In previous
More informationSwarthmore Honors Exam 2012: Statistics
Swarthmore Honors Exam 2012: Statistics 1 Swarthmore Honors Exam 2012: Statistics John W. Emerson, Yale University NAME: Instructions: This is a closed-book three-hour exam having six questions. You may
More informationK. Model Diagnostics. residuals ˆɛ ij = Y ij ˆµ i N = Y ij Ȳ i semi-studentized residuals ω ij = ˆɛ ij. studentized deleted residuals ɛ ij =
K. Model Diagnostics We ve already seen how to check model assumptions prior to fitting a one-way ANOVA. Diagnostics carried out after model fitting by using residuals are more informative for assessing
More informationRegression Analysis: Basic Concepts
The simple linear model Regression Analysis: Basic Concepts Allin Cottrell Represents the dependent variable, y i, as a linear function of one independent variable, x i, subject to a random disturbance
More informationDensity Temp vs Ratio. temp
Temp Ratio Density 0.00 0.02 0.04 0.06 0.08 0.10 0.12 Density 0.0 0.2 0.4 0.6 0.8 1.0 1. (a) 170 175 180 185 temp 1.0 1.5 2.0 2.5 3.0 ratio The histogram shows that the temperature measures have two peaks,
More informationLinear Regression Model. Badr Missaoui
Linear Regression Model Badr Missaoui Introduction What is this course about? It is a course on applied statistics. It comprises 2 hours lectures each week and 1 hour lab sessions/tutorials. We will focus
More information13 Simple Linear Regression
B.Sc./Cert./M.Sc. Qualif. - Statistics: Theory and Practice 3 Simple Linear Regression 3. An industrial example A study was undertaken to determine the effect of stirring rate on the amount of impurity
More informationLecture 9: Predictive Inference for the Simple Linear Model
See updates and corrections at http://www.stat.cmu.edu/~cshalizi/mreg/ Lecture 9: Predictive Inference for the Simple Linear Model 36-401, Fall 2015, Section B 29 September 2015 Contents 1 Confidence intervals
More informationSection 3: Simple Linear Regression
Section 3: Simple Linear Regression Carlos M. Carvalho The University of Texas at Austin McCombs School of Business http://faculty.mccombs.utexas.edu/carlos.carvalho/teaching/ 1 Regression: General Introduction
More informationRegression and the 2-Sample t
Regression and the 2-Sample t James H. Steiger Department of Psychology and Human Development Vanderbilt University James H. Steiger (Vanderbilt University) Regression and the 2-Sample t 1 / 44 Regression
More informationConfidence Intervals, Testing and ANOVA Summary
Confidence Intervals, Testing and ANOVA Summary 1 One Sample Tests 1.1 One Sample z test: Mean (σ known) Let X 1,, X n a r.s. from N(µ, σ) or n > 30. Let The test statistic is H 0 : µ = µ 0. z = x µ 0
More informationAMS 315/576 Lecture Notes. Chapter 11. Simple Linear Regression
AMS 315/576 Lecture Notes Chapter 11. Simple Linear Regression 11.1 Motivation A restaurant opening on a reservations-only basis would like to use the number of advance reservations x to predict the number
More informationHomework 2: Simple Linear Regression
STAT 4385 Applied Regression Analysis Homework : Simple Linear Regression (Simple Linear Regression) Thirty (n = 30) College graduates who have recently entered the job market. For each student, the CGPA
More information36-707: Regression Analysis Homework Solutions. Homework 3
36-707: Regression Analysis Homework Solutions Homework 3 Fall 2012 Problem 1 Y i = βx i + ɛ i, i {1, 2,..., n}. (a) Find the LS estimator of β: RSS = Σ n i=1(y i βx i ) 2 RSS β = Σ n i=1( 2X i )(Y i βx
More information22s:152 Applied Linear Regression. Chapter 5: Ordinary Least Squares Regression. Part 1: Simple Linear Regression Introduction and Estimation
22s:152 Applied Linear Regression Chapter 5: Ordinary Least Squares Regression Part 1: Simple Linear Regression Introduction and Estimation Methods for studying the relationship of two or more quantitative
More informationMultiple Regression: Example
Multiple Regression: Example Cobb-Douglas Production Function The Cobb-Douglas production function for observed economic data i = 1,..., n may be expressed as where O i is output l i is labour input c
More informationNature vs. nurture? Lecture 18 - Regression: Inference, Outliers, and Intervals. Regression Output. Conditions for inference.
Understanding regression output from software Nature vs. nurture? Lecture 18 - Regression: Inference, Outliers, and Intervals In 1966 Cyril Burt published a paper called The genetic determination of differences
More informationST430 Exam 2 Solutions
ST430 Exam 2 Solutions Date: November 9, 2015 Name: Guideline: You may use one-page (front and back of a standard A4 paper) of notes. No laptop or textbook are permitted but you may use a calculator. Giving
More informationSTA 302 H1F / 1001 HF Fall 2007 Test 1 October 24, 2007
STA 302 H1F / 1001 HF Fall 2007 Test 1 October 24, 2007 LAST NAME: SOLUTIONS FIRST NAME: STUDENT NUMBER: ENROLLED IN: (circle one) STA 302 STA 1001 INSTRUCTIONS: Time: 90 minutes Aids allowed: calculator.
More informationMathematics for Economics MA course
Mathematics for Economics MA course Simple Linear Regression Dr. Seetha Bandara Simple Regression Simple linear regression is a statistical method that allows us to summarize and study relationships between
More informationInference for Regression Simple Linear Regression
Inference for Regression Simple Linear Regression IPS Chapter 10.1 2009 W.H. Freeman and Company Objectives (IPS Chapter 10.1) Simple linear regression p Statistical model for linear regression p Estimating
More informationRegression Analysis. Regression: Methodology for studying the relationship among two or more variables
Regression Analysis Regression: Methodology for studying the relationship among two or more variables Two major aims: Determine an appropriate model for the relationship between the variables Predict the
More informationStat 135, Fall 2006 A. Adhikari HOMEWORK 10 SOLUTIONS
Stat 135, Fall 2006 A. Adhikari HOMEWORK 10 SOLUTIONS 1a) The model is cw i = β 0 + β 1 el i + ɛ i, where cw i is the weight of the ith chick, el i the length of the egg from which it hatched, and ɛ i
More informationStatistiek II. John Nerbonne. March 17, Dept of Information Science incl. important reworkings by Harmut Fitz
Dept of Information Science j.nerbonne@rug.nl incl. important reworkings by Harmut Fitz March 17, 2015 Review: regression compares result on two distinct tests, e.g., geographic and phonetic distance of
More informationAnalysis of Variance. Source DF Squares Square F Value Pr > F. Model <.0001 Error Corrected Total
Math 221: Linear Regression and Prediction Intervals S. K. Hyde Chapter 23 (Moore, 5th Ed.) (Neter, Kutner, Nachsheim, and Wasserman) The Toluca Company manufactures refrigeration equipment as well as
More informationSimple Linear Regression
Simple Linear Regression EdPsych 580 C.J. Anderson Fall 2005 Simple Linear Regression p. 1/80 Outline 1. What it is and why it s useful 2. How 3. Statistical Inference 4. Examining assumptions (diagnostics)
More informationReview. One-way ANOVA, I. What s coming up. Multiple comparisons
Review One-way ANOVA, I 9.07 /15/00 Earlier in this class, we talked about twosample z- and t-tests for the difference between two conditions of an independent variable Does a trial drug work better than
More informationSTAT 3A03 Applied Regression With SAS Fall 2017
STAT 3A03 Applied Regression With SAS Fall 2017 Assignment 2 Solution Set Q. 1 I will add subscripts relating to the question part to the parameters and their estimates as well as the errors and residuals.
More informationInference for Regression Inference about the Regression Model and Using the Regression Line
Inference for Regression Inference about the Regression Model and Using the Regression Line PBS Chapter 10.1 and 10.2 2009 W.H. Freeman and Company Objectives (PBS Chapter 10.1 and 10.2) Inference about
More informationMultiple Linear Regression
Multiple Linear Regression ST 430/514 Recall: a regression model describes how a dependent variable (or response) Y is affected, on average, by one or more independent variables (or factors, or covariates).
More informationTentative solutions TMA4255 Applied Statistics 16 May, 2015
Norwegian University of Science and Technology Department of Mathematical Sciences Page of 9 Tentative solutions TMA455 Applied Statistics 6 May, 05 Problem Manufacturer of fertilizers a) Are these independent
More information1 Least Squares Estimation - multiple regression.
Introduction to multiple regression. Fall 2010 1 Least Squares Estimation - multiple regression. Let y = {y 1,, y n } be a n 1 vector of dependent variable observations. Let β = {β 0, β 1 } be the 2 1
More informationIES 612/STA 4-573/STA Winter 2008 Week 1--IES 612-STA STA doc
IES 612/STA 4-573/STA 4-576 Winter 2008 Week 1--IES 612-STA 4-573-STA 4-576.doc Review Notes: [OL] = Ott & Longnecker Statistical Methods and Data Analysis, 5 th edition. [Handouts based on notes prepared
More information1-Way ANOVA MATH 143. Spring Department of Mathematics and Statistics Calvin College
1-Way ANOVA MATH 143 Department of Mathematics and Statistics Calvin College Spring 2010 The basic ANOVA situation Two variables: 1 Categorical, 1 Quantitative Main Question: Do the (means of) the quantitative
More informationMAT2377. Rafa l Kulik. Version 2015/November/26. Rafa l Kulik
MAT2377 Rafa l Kulik Version 2015/November/26 Rafa l Kulik Bivariate data and scatterplot Data: Hydrocarbon level (x) and Oxygen level (y): x: 0.99, 1.02, 1.15, 1.29, 1.46, 1.36, 0.87, 1.23, 1.55, 1.40,
More informationSection Least Squares Regression
Section 2.3 - Least Squares Regression Statistics 104 Autumn 2004 Copyright c 2004 by Mark E. Irwin Regression Correlation gives us a strength of a linear relationship is, but it doesn t tell us what it
More informationSix Sigma Black Belt Study Guides
Six Sigma Black Belt Study Guides 1 www.pmtutor.org Powered by POeT Solvers Limited. Analyze Correlation and Regression Analysis 2 www.pmtutor.org Powered by POeT Solvers Limited. Variables and relationships
More informationECON2228 Notes 2. Christopher F Baum. Boston College Economics. cfb (BC Econ) ECON2228 Notes / 47
ECON2228 Notes 2 Christopher F Baum Boston College Economics 2014 2015 cfb (BC Econ) ECON2228 Notes 2 2014 2015 1 / 47 Chapter 2: The simple regression model Most of this course will be concerned with
More informationSTAT 350: Summer Semester Midterm 1: Solutions
Name: Student Number: STAT 350: Summer Semester 2008 Midterm 1: Solutions 9 June 2008 Instructor: Richard Lockhart Instructions: This is an open book test. You may use notes, text, other books and a calculator.
More informationST430 Exam 1 with Answers
ST430 Exam 1 with Answers Date: October 5, 2015 Name: Guideline: You may use one-page (front and back of a standard A4 paper) of notes. No laptop or textook are permitted but you may use a calculator.
More informationAMS 7 Correlation and Regression Lecture 8
AMS 7 Correlation and Regression Lecture 8 Department of Applied Mathematics and Statistics, University of California, Santa Cruz Suumer 2014 1 / 18 Correlation pairs of continuous observations. Correlation
More informationUnbalanced Data in Factorials Types I, II, III SS Part 1
Unbalanced Data in Factorials Types I, II, III SS Part 1 Chapter 10 in Oehlert STAT:5201 Week 9 - Lecture 2 1 / 14 When we perform an ANOVA, we try to quantify the amount of variability in the data accounted
More informationANOVA: Analysis of Variation
ANOVA: Analysis of Variation The basic ANOVA situation Two variables: 1 Categorical, 1 Quantitative Main Question: Do the (means of) the quantitative variables depend on which group (given by categorical
More informationInference for the Regression Coefficient
Inference for the Regression Coefficient Recall, b 0 and b 1 are the estimates of the slope β 1 and intercept β 0 of population regression line. We can shows that b 0 and b 1 are the unbiased estimates
More informationApplied Regression Analysis
Applied Regression Analysis Chapter 3 Multiple Linear Regression Hongcheng Li April, 6, 2013 Recall simple linear regression 1 Recall simple linear regression 2 Parameter Estimation 3 Interpretations of
More informationMeasuring the fit of the model - SSR
Measuring the fit of the model - SSR Once we ve determined our estimated regression line, we d like to know how well the model fits. How far/close are the observations to the fitted line? One way to do
More informationLab 3 A Quick Introduction to Multiple Linear Regression Psychology The Multiple Linear Regression Model
Lab 3 A Quick Introduction to Multiple Linear Regression Psychology 310 Instructions.Work through the lab, saving the output as you go. You will be submitting your assignment as an R Markdown document.
More informationChapter 16. Simple Linear Regression and dcorrelation
Chapter 16 Simple Linear Regression and dcorrelation 16.1 Regression Analysis Our problem objective is to analyze the relationship between interval variables; regression analysis is the first tool we will
More informationExam Applied Statistical Regression. Good Luck!
Dr. M. Dettling Summer 2011 Exam Applied Statistical Regression Approved: Tables: Note: Any written material, calculator (without communication facility). Attached. All tests have to be done at the 5%-level.
More informationIntroduction and Single Predictor Regression. Correlation
Introduction and Single Predictor Regression Dr. J. Kyle Roberts Southern Methodist University Simmons School of Education and Human Development Department of Teaching and Learning Correlation A correlation
More informationDealing with Heteroskedasticity
Dealing with Heteroskedasticity James H. Steiger Department of Psychology and Human Development Vanderbilt University James H. Steiger (Vanderbilt University) Dealing with Heteroskedasticity 1 / 27 Dealing
More informationThe scatterplot is the basic tool for graphically displaying bivariate quantitative data.
Bivariate Data: Graphical Display The scatterplot is the basic tool for graphically displaying bivariate quantitative data. Example: Some investors think that the performance of the stock market in January
More informationdf=degrees of freedom = n - 1
One sample t-test test of the mean Assumptions: Independent, random samples Approximately normal distribution (from intro class: σ is unknown, need to calculate and use s (sample standard deviation)) Hypotheses:
More informationOverview Scatter Plot Example
Overview Topic 22 - Linear Regression and Correlation STAT 5 Professor Bruce Craig Consider one population but two variables For each sampling unit observe X and Y Assume linear relationship between variables
More informationWe like to capture and represent the relationship between a set of possible causes and their response, by using a statistical predictive model.
Statistical Methods in Business Lecture 5. Linear Regression We like to capture and represent the relationship between a set of possible causes and their response, by using a statistical predictive model.
More informationLinear Regression. In this lecture we will study a particular type of regression model: the linear regression model
1 Linear Regression 2 Linear Regression In this lecture we will study a particular type of regression model: the linear regression model We will first consider the case of the model with one predictor
More informationLecture 3: Inference in SLR
Lecture 3: Inference in SLR STAT 51 Spring 011 Background Reading KNNL:.1.6 3-1 Topic Overview This topic will cover: Review of hypothesis testing Inference about 1 Inference about 0 Confidence Intervals
More informationUNIVERSITY OF MASSACHUSETTS. Department of Mathematics and Statistics. Basic Exam - Applied Statistics. Tuesday, January 17, 2017
UNIVERSITY OF MASSACHUSETTS Department of Mathematics and Statistics Basic Exam - Applied Statistics Tuesday, January 17, 2017 Work all problems 60 points are needed to pass at the Masters Level and 75
More informationRegression Models. Chapter 4. Introduction. Introduction. Introduction
Chapter 4 Regression Models Quantitative Analysis for Management, Tenth Edition, by Render, Stair, and Hanna 008 Prentice-Hall, Inc. Introduction Regression analysis is a very valuable tool for a manager
More informationHandout 4: Simple Linear Regression
Handout 4: Simple Linear Regression By: Brandon Berman The following problem comes from Kokoska s Introductory Statistics: A Problem-Solving Approach. The data can be read in to R using the following code:
More informationECO220Y Simple Regression: Testing the Slope
ECO220Y Simple Regression: Testing the Slope Readings: Chapter 18 (Sections 18.3-18.5) Winter 2012 Lecture 19 (Winter 2012) Simple Regression Lecture 19 1 / 32 Simple Regression Model y i = β 0 + β 1 x
More information