SLR output RLS. Refer to slr (code) on the Lecture Page of the class website.

Transcription

1 SLR output RLS Refer to slr (code) on the Lecture Page of the class website. Old Faithful at Yellowstone National Park, WY: Simple Linear Regression (SLR) Analysis SLR analysis explores the linear association between an explanatory (independent) variable, usually denoted as x, and a response (dependent) variable, usually denoted as y. The point is to see if we can use a mathematical linear model to describe the association (relationship) between the two variables; using one known value to estimate the other value. The variables are the eruption duration and the waiting time between eruptions of Old Faithful; eruptions will be the explanatory (independent) variable and waiting will be the response (dependent) variable, modelling waiting time by erption duration. You are familiar with y = mx + b from algebra, where m is the slope and b is the y-intercept (value of y when x = 0), which is a mathematical linear equation, a deterministic equation. The population regression model used in statistics Notice that it is basically the same as the one from algebra: y i = β 0 + β 1 x i + ɛ i Where: * y i : value of the response (dependent) variable * β 0 : the value of the y-intercept (when x = 0) * β 1 : the value of the slope (the change in y due to a one unit increase in x. Not rise run ) * ɛ i : the (squared) residual (error) term (( ŷ i y i ) 2 ), the total (sum) squared distance between each estimated y and the true (observed) value of y. A residual (error) is calculated as e i = ŷ i y i Assumptions of SLR: 1. E(ɛ i ) = 0: the mean of the residuals is 0 2. V (ɛ i ) = σ 2 ɛ : the variance of the residuals is constant (the same) for all values of y. Also called constant variance, homogeneity of variance (means same variance) 3. Cov(ɛ i, ɛ j ) = 0: independence of residuals 4. ɛ i N(0, σ 2 ɛ ): Residuals have an approximately normal distribution with mean 0 and homogeneous variance 1

2 The sample regression model Is used once there are estimated values from the data: ŷ i = ˆβ 0 + ˆβ 1 x i Where: * ŷ i : estimate of the value of the response (dependent) variable * ˆβ 0 : the estimate of the value of the y-intercept (ŷ when x = 0) * ˆβ 1 : the estimate of the value of the slope (the change in y due to a one unit increase in x. Not rise run ) * Note that ɛ i dropped off from the other model. This is because of the first assumption of regression, E(ɛ i ) = 0: the mean of the residuals is 0. This example uses a dataset that is built into R along with the needed commands. data(faithful) head(faithful) eruptions waiting With all datasets that are read in from R (or other commands for different file types 1 ), accessing the variable names uses something called a two-level name. A two-level name will include both the dataset name and the variable you want to access. Without a two-level name or without attach(), you d have to type datasetname$variablename. For example, if you want to make a histogram of just the waiting variable from the faithful dataset, you d have to use a two-level name. data(faithful) # hist(waiting) gives an error "Error in hist(waiting) : object 'waiting' not found" # but faithful$ waiting exists hist(faithful$waiting) 1 Can use read.table(), read.csv(), etc. 2

3 Histogram of faithful$waiting Frequency faithful$waiting The command attach() allows you just to access them with their name. There are other options including using with() or some commands allow the use of the data= option (not all commands but a fair number of them), hist() is one that does not have a data= option. attach(faithful) hist(waiting) detach(faithful) # undo the attach() command so hist(waiting) will give an error again # hist(waiting) another error # using with() with(faithful,hist(waiting)) Histogram of waiting Frequency waiting 3

4 Scatterplot A scatterplot of the data is to see if there is a linear association between the eruption duration and the waiting time between eruptions of Old Faithful; eruptions will be the explanatory (independent) variable and waiting will be the response (dependent) variable, modelling waiting time by eruption duration. Let x=eruptions and y=waiting. Use the following commands: plot(y~x,main='title goes here',...) y~x is the formula (use the variable names exact case) main=' ': a title, such as Scatterplot of Y by X...: other options (including color (col()=''), symbols (pch=), etc. Just enter?option in the console for help files) attach(faithful) plot(waiting~eruptions,pch=17,main="raw Data Scatterplot for Old Faithful") Raw Data Scatterplot for Old Faithful waiting eruptions Many times in regression, we want to see what the line of the regression equation will look like on the scatterplot of the raw data. It s not strictly necessary but the point of this analysis is to explore and understand the linear relationship between two variables. If you do not have one, then use of this analysis is not useful as the results cannot be used with the given dataset. plot(waiting~eruptions,pch=17,main="raw Data Scatterplot for Old Faithful") abline(lm(waiting~eruptions)) 4

5 Raw Data Scatterplot for Old Faithful waiting eruptions Correlation Is a calculation that measures the strength and direction (positive or negative) of the linear relationship between x and y. It is bounded between -1 and 1, where -1 and +1 are perfect linear relationships and 0 implies both no linear relationship and x and y would be independent. Correlation is denoted as r for sample correlation and ρ for the population correlation. r = 1 (xi x)(y i ȳ) n 1 s x s y The R command is cor(). cor(x,y,method=' ',...) Where: * x,y: two numeric vectors of data or dataset name if all variables are numeric * method=' ': pearson (default), kendall, spearman for differeny types of correlation (we use pearson, the default) * data=: the dataset name *...: other options available cor(faithful) # or cor(waiting,eruptions) eruptions waiting eruptions waiting

6 R Regression commands R has commands to do a regression analysis. The correlation r is not calculated in the regression analysis, but a number that is related to it is provided. It is denoted as R 2, called the coefficient of determination. It is the proportion (or 100%) of observed variation that can be explained by the relationship between x and y. It is bounded between 0 (0%) and 1 (100%). The closer to 1 (100%), the more variation we can explain and also the stronger the linear relationship between x and y. R 2 = (r) 2 so you can find it by taking the square root of R 2 and if the slope is positive, then r is positive, if the slope is negative, then r is negative. The regression output table is calculated by the linear model command and its output displayed with a summary command. First, create the model with lm(), usually naming it so that you can call it in the summary() command. The syntax for lm() with a name: fit=lm(y~x,data= ) Then display the regression results from lm() with summary() summary(fit) faith.fit=lm(waiting~eruptions,data=faithful) summary(faith.fit) Call: lm(formula = waiting ~ eruptions, data = faithful) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) <2e-16 *** eruptions <2e-16 *** --- Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual standard error: on 270 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: 1162 on 1 and 270 DF, p-value: < 2.2e-16 The hypothesis test for the estimated intercept (β 0 ) and slope (β 1 ) are located on the regression output. The hypotheses are: For β 1, the slope: H 0 : β 1 = 0 vs. H a : β 1 0 The alternative hypothesis is asking if the slope is just significant (a slope of 0 is a horizontal line and for any value of x, y would be constant). You can do upper or lower tail tests if you are asking if the slope has increased or decreased. Most often the interest is whether or not it is significant. Realistically the hypothesized value (here it is 0) could be something other than 0 if there is a need. 6

7 For β 0, the intercept: H 0 : β 0 = 0 vs. H a : β 0 0 The alternative hypothesis is asking if the intercept is significant (an intercept at 0 just means that there is a value of x that yields a y = 0 value). Frequently, the intercept is of little to no interest because often there is no value of x = 0 in the sampled datasets, either due to random chance or just that x = 0 does not or cannot exist in the context of the data (even if it exists mathematically, it may have no realistic or practical meaning). There are some economic datasets and many others that utilize the intercept because it make sense both mathematically and realistically. You can do upper or lower tail tests if you are asking if the intercept has increased or decreased. Most often the interest is whether or not it is significant. Realistically the hypothesized value (here it is 0) could be something other than 0 if there is a need. The following picture is a printout of a regression summary table from fit=lm(y~x,data= ) and summary(fit) Diagnostic plots used to check assumptions of slr We need to create vectors of residuals and predicted values (ŷ i ) and then plot them to see if we have met the assumptions that allow us to use SLR to describe the relationship in our data. res=rstudent(faith.fit); pred=fitted(faith.fit) # Assumption 1: histogram should be centered around (approximately) 0, # or the mean of the residuals should be approximately = 0 hist(res,main='histogram of residuals') 7

8 Histogram of residuals Frequency res mean(res) [1] # Assumption 2: plot of x=predicted and y=residuals should have no discerable pattern (random scatter) plot(pred,res,main=' Residuals vs. Predicted'); abline(0,0) Residuals vs. Predicted res pred 8

9 # Assumption 3: independence of residuals -- no need to check; assume it's met # Assumption 4: normality of residuals so histogram should be approximately symmetric/bell-shaped # or QQplot (normal probability plot) where most points should be mostly along y=x line qqnorm(res,main='qqplot of Residuals'); qqline(res) QQPlot of Residuals Sample Quantiles Theoretical Quantiles R 2 (Multiple R-squared) is meaning that 81.15%. r could be calculated as r = + R 2 = = 0.9. It is positive since the slope is positive (if r > 0 then β 1 > 0, if r < 0 then β 1 < 0, and vice versa). 9