SIMPLE LINEAR REGRESSION

Size: px

Start display at page:

Download "SIMPLE LINEAR REGRESSION"

Carmel Terry
6 years ago
Views:

1 SIMPLE LINEAR REGRESSION In linear regreion, we conider the frequency ditribution of one variable (Y) at each of everal level of a econd variable (). Y i known a the dependent variable. The variable for which you collect data. i known a the independent variable. The variable for the treatment. Determining the Regreion Equation One goal of regreion i to draw the bet line through the data point. The bet line uually i obtained uing mean intead of individual obervation. Example Effect of hour of mixing on temperature of wood pulp Hour of mixing () Temperature of wood pulp (Y) Y Y39 Y8 364 Y,967 n6

2 Effect of hour of mixing on temperature of w ood pulp 8 Temperature Hour of m ixing The equation for any traight line can be written a: Ŷ b + b where: b o Y intercept, and b regreion coefficient lope of the line The linear model can be written a: where: e i reidual Y Ŷ i i Y β + β + ε With the data provided, our firt goal i to determine the regreion equation i i Step. Solve for b ( )(Y Y) ( ) ( Y) Y n SS Cro Product b SSCP SS ( ) SS n for the data in thi example

3 4 Y 39 Y,8 364 Y,967 ( Y) Y n ( ) n (4x39) b 8. The number calculated for b, the regreion coefficient, indicate that for each unit increae in (i.e., hour of mixing), Y (i.e., wood pulp temperature) will increae 8. unit (i.e., degree). The regreion coefficient can be a poitive or negative number. To complete the regreion equation, we need to calculate b o Y - b b Therefore, the regreion equation i: Ŷ i Temperature ( o F) Hour of mixing

4 Aumption of Regreion. There i a linear relationhip between and Y. The value of are known contant and preumably are meaure without error. 3. For each value of, Y i independent and normally ditributed: Y~N(, σ ). 4. Sum of deviation from the regreion line equal zero: ( Y i Yˆi ). 5. Sum of quare for error are a minimum. Effect of hour of mixing on temperature of wood pulp Temperature Hour of mixing If you quare the deviation and um acro all obervation, you obtain the definition formula for the following um of quare: ( Ŷ i Y) ( Y ) i Ŷ i ( Y i Y) Sum Square Due to Regreion Sum Square Due to Deviation from Regreion (Reidual) Sum Square Total

5 Teting the hypothei that a linear relationhip between and Y exit The hypothee to tet that a linear relationhip between and Y exit are: H o : ß H A : ß Thee hypothee can be teted uing three different method:. F-tet. t-tet 3. Confidence interval Method. F-tet The ANOVA to tet H o : $ can be done uing the following ource of variation, degree of freedom, and um of quare: SOV df Sum of Square Due to regreion ( Y) Y n SSCP ( ) SS Reidual n- Determined by ubtraction n ( Y) Total n- Y n SS Y Uing data from the example: 4 Y 39 Y,8 364 Y,967 Step. Calculate Total SS Y ( Y) n 39, ,6.833

6 Step. Calculate SS Due to Regreion ( Y) Y n ( ) n ( 4x39) , ,59.7 Step 3. Calculate Reidual SS SS Deviation from Regreion Total SS - SS Due to Regreion Step 4. Complete ANOVA SOV df SS MS F Due to Regreion Due to Reg. MS/Reidual MS ** Reidual Total The reidual mean quare i an etimate of σ Y, read a variance of Y given. Thi parameter etimate the tatitic σ Y. Step 5. Becaue the F-tet on the Due to Regreion SOV i ignificant, we reject H o : ß at the 99% level of confidence and can conclude that there i a linear relationhip between and Y. Coefficient of Determination - r From the ANOVA table, the coefficient of variation can be calculated uing the formula r SS Due to Regreion / SS Total Thi value alway will be poitive and range from to.. A r approache., the aociation between and Y improve. r x i the percentage of the variation in Y that can be explained by having in the model. For our example: r / We can conclude that 9.7% (i.e..97 x ) of the variation in wood pulp temperature can be explained by hour of mixing.

7 Method. t-tet The formula for the t-tet to tet the hypothei H o : ß i: b t b where: b the regreion coefficient, and b Y SS For our example: Step. Calculate Remember that Y Reidual MS [SS Y - (SSCP / SS )] / (n-) b We know from previou part of thi example: Therefore, SS Y SSCP 567. SS 7. b ( Y / SS ) SSCP SS Y - SS n - SS

8 Step. Calculate t tatitic b t b Step 3. Look up table t value Table t "/, (n-) df t.5/, 4df.776 Step 4. Draw concluion Since the table t value (.776) i le that the calculated t-value (6.66), we reject H o : ß at the 95% level of confidence. Thu, we can conclude that there i a linear relationhip between hour of mixing and wood pulp temperature at the 95% level of confidence. Method 3. Confidence Interval The hypothei H o : ß can be teted uing the confidence interval: CI b ± t α,( n ) df ( b ) For thi example: CI b ± t α,( n ) df ( b ) 8.± β.476 We reject Ho: ß at the 95% level of confidence ince the CI doe not include.

9 Predicting Y Given Regreion analyi alo can be ued to predict a value for Y given. Uing the example, we can predict the temperature of one batch of wood pulp after mixing hour. In thi cae, we predict an individual outcome of Y drawn from the ditribution of Y. Thi etimate i ditinct from etimating mean or average of a ditribution of Y. The value of an individual Y at a given will take on the form of the confidence interval: CI Ŷ ± tα,( n ) df (Y ) where Y Y, and Example Y ( ) + + Remember Y Y i the Reidual Mean Square n SS We wih to determine the temperature of the one batch of wood pulp after mixing two hour (i.e., Y ). Step. Uing the regreion equation, olve for Ŷ when. Remember Ŷ Ŷ ().667 Step. Solve for Y Y Y + n ( ) + SS ( 7) + 7

10 Step 3. Calculate the confidence interval CI Ŷ ± t α,( n ) df ( Y ).667 ± ± Therefore: LCI -.and UCI Note: Thi CI i not ued to tet a hypothei Thi CI tate that if we mix the wood pulp for two hour, we would expect the temperature to fall within the range of -. and degree 95% of the time. We would expect the temperature to fall outide of thi range 5% of the time due to random chance. Example We wih to determine the temperature of the one batch of wood pulp after mixing even hour (i.e., Y 7 ). Step. Uing the regreion equation, olve for Ŷ when 7. Remember Ŷ Ŷ (7) Step. Solve for Y 7 Y Y + n ( ) + SS (7 7) + 7

11 Step 3. Calculate the confidence interval CI Ŷ ± t α,( n ) df ( Y ) ± ± 3.59 Therefore: LCI.658 and UCI Note: For 7 (i.e., at the mean of ), the variance Y o i at a minimum. Thi CI tate that if we mix the wood pulp for even hour, we would expect the temperature to fall within the range of.658 and degree 95% of the time. We would expect the temperature to fall outide of thi range 5% of the time due to random chance. Predicting Y Given Regreion analyi alo can be ued to predict a value for Y given. Uing the example, we can predict the average temperature of wood pulp after mixing hour. In thi cae, we predict an individual outcome of Y drawn from the ditribution of Y. Thi etimate i ditinct from ditribution of Y for a. The value of an individual Y at a given will take on the form of the confidence interval: CI Ŷ ± tα,( n ) df ( Y where Y Y Y Y ), and ( ) + n SS

12 Example We wih to determine the average temperature of the wood pulp after mixing two hour (i.e., Y ). Step. Uing the regreion equation, olve for Ŷ when. Remember Ŷ Ŷ ().667 Step. Solve for Y Y Y ( ) + n SS ( 7) Step 3. Calculate the confidence interval CI Ŷ ± t α,( n ) df ( Y ).667 ± ±.443 Therefore: LCI and UCI 33. Note: Thi CI i not ued to tet a hypothei Thi CI tate that if we mix the wood pulp for two hour any number of time, we would expect the average temperature to fall within the range of and 33. degree 95% of the time. We would expect the temperature to fall outide of thi range 5% of the time due to random chance.

13 Example We wih to determine the average temperature of wood pulp after mixing even hour. Step. Uing the regreion equation, olve for Ŷ when 7. Remember Ŷ Ŷ (7) Step. Solve for Y 7 Y 7 Y ( ) + n SS (7 7) Step 3. Calculate the confidence interval CI Ŷ ± t α,( n ) df ( Y ) ± ±.53 Therefore: LCI and UCI Note: For 7 (i.e., at the mean of ), the variance i at a minimum. Y

14 Comparing and Y Y i alway greater than Y. Y Comparing the formula: Y Y ( ) + + and Y n SS ( ) +. Y n SS Notice that in the formula for you add one while in the formula for Y you do not. Y Comparion of Y and Y. Y Y

15 We can draw the two confidence interval a confidence belt about the regreion line. Confidence belt for the effect of hour of mixing on temperature of wood pulp 3 Temperature LL - Ind. Y UL - Ind. Y LL - Y Bar UL - Y Bar Hour of mixing Notice that:. The confidence belt are ymmetrical about the regreion line. The confidence belt are narrowet at the mean of, and 3. The confidence belt for the ditribution baed on mean are narrower than the ditribution baed on an individual obervation.

16 Determining if Two Independent Regreion Coefficient are Different It may be deirable to tet the homogeneity of two b ' to determine if they are etimate of the ame ß. Thi can be done uing a t-tet to tet the hypothee: H o : ß ß ' H A : ß ß ' b b' Where t (ReidualMS + Reidual MS) The Table t-value ha (n - ) + (n - ) df. Example SS + SS Y Y Y 8 Y Y 46 Y 787 Step. Determine regreion coefficient for each Y for Y Y 7 Thu b [7 - (5x8)/5] / [55-5 /5].6 for Y Y 65 Thu b ' [65 - (5x87)/5] / [55-5/5] 9.

17 Step. Calculate Reidual MS for each Y Remember Reidual MS SSCP SS Y SS n Reidual MS Reidual MS Step 3. Solve for t.6 9. ( )x (.) 4. Step 4. Look up table t-value with (n - ) + (n - ) df t.5/, 6 df Step 4. Make concluion Becaue the abolute value of the calculated t-value (-4.) i greater than the abolute value of the tabular t-value (.776), we can conclude at the 95% level of confidence that the two regreion coefficient are not etimating the ame ß.

18 Summary - Some Ue of Regreion. Determine if there i a linear relationhip between an independent and dependent variable.. Predict value of Y at a given Mot accurate near the mean of. Should avoid predicting value of Y outide the range of the independent variable that were ued. 3. Can adjut Y to a common bae by removing the effect of the independent variable (Analyi of Covariance). 4. ANOVA (CRD, RCBD, and LS) can be done uing regreion 5. Compare homogeneity of two regreion coefficient. SAS Command option pagneo; data reg; input x y; dataline; ; proc reg; model yx/cli clm; title 'SAS Output for Linear Regreion Example in Cla'; run;

19 Printout of Data Ob x y xy

20 Decriptive tatitic The MEANS Procedure Variable Sum Mean Uncorrected SS Corrected SS x y xy Thi ummary table include the value that were calculated for the data on page of the regreion note. The uncorrected SS give you the value when you quare the value for each obervation and then um the quared value. Diregard the value for Mean, Uncorrected SS, and Corrected SS for xy.

21 Simple Linear Regreion, Including Confidence Interval The REG Procedure Model: MODEL Dependent Variable: y Number of Obervation Read 6 Number of Obervation Ued 6 Source DF Analyi of Variance Sum of Square Mean Square F Value Pr > F Model Error Corrected Total Root MSE.753 R- Square.973 Dependent Mean Coeff Var Variable DF Adj R- Sq.8966 Parameter Etimate Parameter Etimate Standard Error t Value Pr > t Intercept x Reult from the F- tet of the hypothei Ho: ß. r value the coefficient of determination Value ued to generate the regreion equation. Intercept Y- intercept x regreion coefficient Reult from the t- tet of the hypothei Ho: ß

22 Simple Linear Regreion, Including Confidence Interval The REG Procedure Model: MODEL Dependent Variable: y Ob Dependent Variable Output Statitic Predicted Value Std Error Mean Pre dict 95% CL Mean 95% CL Predict Reidual Sum of Reidual Sum of Squared Reidual Predicted Reidual SS (PRESS) Dependent variable i Yi. Predicted value i baed on the regreion equation. 95% CL Mean i the confidence limit for a predicted mean value of Y at a given level of. 95% CL for the confidence limit for the predicted value of an individual value of Y at a given level of.

23 SAS Command for Data with Multiple Replicate option pageno; data reg; input x rep y; xyx*y; dataline; ;; od graphic off; od rtf file'repreg.rtf'; proc print; title 'Printout of Data - All Data'; run; proc reg; model yx; title 'Regreion on All Data - Rep Kept Separate'; run; proc ort; by x; proc mean mean noprint; by x; var y; output outnew meanmeany; *Comment: The previou tatement take the average of Y and Y to create a new variable Y. Performing regreion on mean provide a better coefficient of determination'; run; proc reg; model meanyx; title 'Regreion on the Mean of Y'; run; od rtf cloe;

24 Printout of Data - All Data Ob x rep y xy

25 Regreion on All Data - Rep Kept Separate The REG Procedure Model: MODEL Dependent Variable: y Number of Obervation Read Number of Obervation Ued Source DF Analyi of Variance Sum of Square Mean Square F Value Pr > F Model Error Corrected Total 747 Root MSE 5.4 R- Square.7766 Dependent Mean Adj R- Sq.7543 Coeff Var Variable DF Parameter Etimate Parameter Etimate Standard Error t Value Pr > t Intercept <. x

26 Regreion on the Mean of Y The REG Procedure Model: MODEL Dependent Variable: meany Number of Obervation Read 6 Number of Obervation Ued 6 Source DF Analyi of Variance Sum of Square Mean Square F Value Pr > F Model Error Corrected Total Root MSE R- Square.9377 Dependent Mean Adj R- Sq.9 Coeff Var Variable DF Parameter Etimate Parameter Etimate Standard Error t Value Pr > t Intercept <. x

REVIEW OF SIMPLE LINEAR REGRESSION SIMPLE LINEAR REGRESSION

REVIEW OF SIMPLE LINEAR REGRESSION SIMPLE LINEAR REGRESSION I liear regreio, we coider the frequecy ditributio of oe variable (Y) at each of everal level of a ecod variable (X). Y i kow a the depedet variable.