Math Section MW 1-2:30pm SR 117. Bekki George 206 PGH

Transcription

1 Math 3339 Section MW 1-2:30pm SR 117 Bekki George 206 PGH Office Hours: M 11-12:30pm & T,TH 10:00 11:00 am and by appointment

2 Linear Regression (again) Consider the relationship between the weight, x, of an automobile and the amount of fuel needed, y, to drive it 100 miles. We expect a linear relationship between these two variables (W=Fd). Thus, we use the model ŷ = a + bx + ε or Y = β 0 + β 1 x + ε where ε is a random component. The variable x is called the explanatory variable, and the variable Y is called the response variable. We have E Y = β + β x 0 1, and var( Y ) = σ 2. In many cases, it is appropriate to assume that Y is normally distributed, and we make that assumption in this course when certain conditions are met (we will give these later).

3 If the relationship between car weight and gas required is linear according to the model Y = β 0 + β 1 x + ε, then each sample of a particular car has Y normally distributed. The diagram below illustrates.

4 Suppose we have n pairs of observations ( x 1, y 1 ),( x 2, y 2 ),..., x n, y n sample to estimate unknown parameters of the distribution. Example: Take the fuel consumption rates of the 10 cars below: Car: Weight (1000s lbs) Fuel Consumption (gpm) AMC Concord Chevy Caprice Ford Squire Chevette Toyota Corona Ford Mustang Mazda GLC AMC Sprint VW Rabbit Buick Century ( ). You may use this Create a scatter plot of this data. Do you notice a relationship between weight and fuel consumption?

5 How would we estimate β 0,β 1, the y-intercept and slope of the line, based on our sample points? ( ) = The distance from each sample point to the least squares line is y i β 0 + β 1 x i If we want to find the best line for our data, we need to minimize the sum of these distances. This would be equivalent to minimizing the sum of the squares of the differences. n 0 1 i 0 1 i i= 1 ( β, β ) = ( β + β ) 2 S y x Taking the partial derivatives equal to zero and solving for β 0,β 1 gives the least-squares estimates: ˆ Σxy i i ( Σxi)( Σyi) / n β1 = 2 2 Σx Σx / n ˆ β i ( i) = y ˆ β x 0 1 ŷ= ˆ β + ˆ β x 0 1

6 Example: Determine the least-squares estimate for the fuel economy example. Use these values to determine an estimated regression line. Residuals and Fitted Values For each i we call y i the fitted value and e i = y i ŷ i the residual. Determine the fitted values and residuals for our fuel consumption problem:

7 The sum of squares residual (SS(resid)), is found by:! Where β 1 = Sxy / S xx The regression sum of squares is So the total sum of squares is The estimate of the variance is s2 = SS(resid) n 2

8 Estimate the variance in the previous example: The Coefficient of Determination The coefficient of determination, denoted by r 2 can be found by r 2 = 1 SS(resid) SS(tot) This is interpreted as the proportion of observed y variation that can be explained by the simple linear regression model.

9 Distribution of the Parameter Estimators For each i, the random variables Y i are normally distributed with means β 0 + β 1 x i and 2 variances σ. This gives us normally distributed estimators ˆβ 0 ~ N β 0,σ ˆβ 1 ~ N β 1,σ / ( 1/ n) + x 2 / Σ x i x Σ( x i x ) 2 ( ) 2 We can use these to determine confidence intervals for β0, β 1. A ( 1 α )100% confidence interval for the slope, β 0 is ˆβ 0 ± z α /2 σ 1 n + x 2 Σ( x i x ) 2 And, a ( 1 α )100% confidence interval for β 1 is ˆβ 1 ± z α /2 σ ( ) 2 or Σ x i x ˆβ 1 ± t α /2 SE β (df = n-2)

10 Example: Determine a 99% confidence interval for the slope in the fuel consumption example. Use the calculations previously made on the fuel consumption problem to estimate the number of gallons per mile (gpm) required to operate a 3000 pound vehicle.

11 We can also test hypotheses about the value of the slope. The most common hypothesis is that the slope is zero that is there is no true linear relationship between x and y. (the mean of y does not change at all when x changes) The test statistic is another t statistic: b t = SEb You will not have to do calculations for this test by hand. We will let R-Studio do all of the calculations for us. Example: How well do golfers scores on the first round of a two-round tournament predict their scores for a second round? (Remember, low scores are better.) Golfer Round Round

12 To use regression inference the data must satisfy the regression model assumptions. Check these assumptions one by one before doing inference: 1. The true relationship is linear * Look at the scatterplot to check that the overall pattern is roughly linear. * Plot the residuals and make sure there is no pattern. 2. The standard deviation of the response about the true line is the same everywhere * The scatterplot should show roughly the same scatter of data points about the line over the entire range of data. * Check the residual plot. The scatter of points should not increase as x increases. 3. The response varies normally about the true regression line. * The residuals should follow a normal distribution. Check the histogram of the residuals for deviations from normality.

13 Example: A midterm exam in Applied Mathematics consists of problems in 8 topical areas. One of the teachers believes that the most important of these is the section on problem solving. She analyzes the scores of 36 randomly chosen students using computer software and produces the following print-out relating the total score to the problem-solving subscore, ProbSolv: Predictor Coef S tdev T p s = Constant R-sq = 62.0% ProbSolv R-sq (adj) = 60.9% a.) What is the regression equation? b.) Interpret the slope of the regression in the context of the problem. c.) Interpret the value of R-Sq in words. d.) Interpret the value of S in words. e.) Calculate the 95% confidence interval of the slope of the regression line for all students.

14 f.) Use the information provided to test whether there is a significant relationship between the problem solving subsection and the total score at the 5% level. Complete the list of components we have given as required for full credit.

15 The F test for Significance of Regression The F distribution with v 1 and v 2 degrees of freedom is found by F = χ 2 1 / v 1 χ 2 2 / v 2 χ 1 2 has v 1 degrees of freedom and χ 2 2 has v 2 degrees of freedom For a regression model with 2 parameters (let p represent the number of parameters for the general formula and n is the number of values), v 1 = p 1 = 2 1 = 1 and v 2 = n p = n 2 For a regression model with 2 parameters, the F test statistic can also be calculated by F = MS(regr) MS(resid) This is also equal to the square of the t statistic on the test for H 0 :β = 0 vs. H a :β 0.

16 Example. The file fire_theft.csv contains the following data: the number of fires per 1000 housing units and the number of thefts per 1000 population within the same Zip code in the Chicago metro area. (Reference: U.S. Commission on Civil Rights) The data can be found here: Test whether there is a significant relationship between fire and thefts in that zip code.

17 Popper 20 The effects of a toxic pollutant upon fish was examined by placing fish in a two-liter solution of water with various concentrations of the pollutant. The time (in minutes) until the fish showed distress was recorded, at which time the fish were removed from the container. A total of 18 different experiments were performed. Note that the pollutant is measured on a logarithmic scale where a change of one unit represents an increase of tenfold in the pollution concentration. A preliminary plot of the data showed that the relationship of time versus log(pollution) was approximately linear. The computer output appears below: SOURCE DF SUM OF SQUARES MEAN SQUARE F VALUE PR > F MODEL ERROR CORR. TOTAL T FOR HO: PR > T STD ERROR OF PARAMETER ESTIMATE PARAMETER=0 ESTIMATE INTERCEPT LOGPOLLUT The fitted regression line is: ^ a) Y = X b) Y = X ^ c) Y = X d) Y = X ^ e) Y = X ^ ^

18 Example: The table below displays the performance of 10 randomly selected students on the SAT Verbal and SAT Math tests taken last year. Here is the scatter plot: Student Math Verbal

19 Popper What can be said about this scatter plot? a. There is a strong negative linear relationship b. There is a weak negative linear relationship c. There is a strong positive linear relationship d. There is a weak positive linear relationship e. None of these

20 Here is the computer output:

21 Popper Calculate the least-squares regression line for this data. a. ŷ = x b. ŷ = x c. ŷ = x d. ŷ = x e. none of these 4. What is the value of r? a b c d e. none of these