Regression. What is regression? Linear Regression. Cal State Northridge Ψ320 Andrew Ainsworth PhD

Transcription

1 Regreion Cal State Northridge Ψ30 Andrew Ainworth PhD What i regreion? How do we predict one variable from another? How doe one variable change a the other change? Caue and effect Linear Regreion A technique we ue to predict the mot likely core on one variable from thoe on another variable Ue the nature of the relationhip (i.e. correlation) between two (or more; next chapter) variable to enhance your prediction 3 1

2 Linear Regreion: Part Y - the variable you are predicting i.e. dependent variable X - the variable you are uing to predict i.e. independent variable Ŷ- your prediction (alo known a Y ) 4 Why Do We Care? We may want to make a prediction. More likely, we want to undertand the relationhip. How fat doe CHD mortality rie with a one unit increae in moking? Note: we peak about predicting, but often don t actually predict. 5 An Example Cigarette and CHD Mortality from Chapter 9 Data repeated on next lide We want to predict level of CHD mortality in a country averaging 10 cigarette per day. 6

3 The Data Baed on the data we have what would we predict the rate of CHD be in a country that moked 10 cigarette on average? Firt, we need to etablih a prediction of CHD from moking Country Cigarette CHD CHD Mortality per 10, We predict a CHD rate of about 14 Regreion Line 0 For a country that moke 6 C/A/D Cigarette Conumption per Adult per Day 8 Formula Regreion Line Y ˆ = bx + a Yˆ = the predicted value of Y (e.g. CHD mortality) X = the predictor variable (e.g. average cig./adult/country) 9 3

4 Regreion Coefficient Coefficient are a and b b = lope Change in predicted Y for one unit change in X a = intercept value of Ŷ when X = 0 10 Intercept Calculation Slope cov XY y b = or b = r X x or b N XY X Y = N X ( X ) a = Y bx 11 For Our Data Cov XY = 11.1 X =.33 = b = 11.1/5.447 =.04 a = *5.95 =.3 See SPSS printout on next lide Anwer are not exact due to rounding error and deire to match SPSS. 1 4

5 SPSS Printout 13 Note: The value we obtained are hown on printout. The intercept i the value in the B column labeled contant The lope i the value in the B column labeled by name of predictor variable. 14 Making a Prediction Second, once we know the relationhip we can predict Yˆ = bx + a =.04X Yˆ =.04* =.787 We predict.77 people/10,000 in a country with an average of 10 C/A/D will die of CHD 15 5

6 Accuracy of Prediction Finnih moker moke 6 C/A/D We predict: Yˆ = bx + a =.04X Yˆ =.04* = They actually have 3 death/10,000 Our error ( reidual ) = = 8.38 a large error Reidual CHD Mortality per 10, Prediction Cigarette Conumption per Adult per Day 17 Reidual When we predict Ŷ for a given X, we will ometime be in error. Y Ŷ for any X i a an error of etimate Alo known a: a reidual We want to Σ(Y-Ŷ) a mall a poible. BUT, there are infinitely many line that can do thi. Jut draw ANY line that goe through the mean of the X and Y value. Minimize Error of Etimate How? 18 6

7 Minimizing Reidual Again, the problem lie with thi definition of the mean: ( X X ) = 0 So, how do we get rid of the 0? Square them. 19 Regreion Line: A Mathematical Definition The regreion line i the line which when drawn through your data et produce the mallet value of: ( Y Yˆ ) Called the Sum of Squared Reidual or SS reidual Regreion line i alo called a leat quare line. 0 Summarizing Error of Prediction Reidual variance The variability of predicted value Y Yˆ Σ( Y Yˆ ) SS = = N N i i reidual 1 7

8 Standard Error of Etimate Standard error of etimate The tandard deviation of predicted value Σ( Y ˆ i Yi ) SSreidual ˆ = = Y Y N N A common meaure of the accuracy of our prediction We want it to be a mall a poible. Country X (Cig.) Y (CHD) Y' (Y - Y') (Y - Y') Mean SD Sum Y Yˆ Y Yˆ Example ˆ Σ( Yi Yi ) = = = N 1 ˆ Σ( Yi Yi ) = = = N 1 = = Regreion and Z Score When your data are tandardized (linearly tranformed to z-core), the lope of the regreion line i called β DO NOT confue thi β with the β aociated with type II error. They re different. When we have one predictor, r = β Z y = βz x, ince A now equal 0 4 8

9 Partitioning Variability Sum of quare deviation Total SS = ( Y Y ) total Regreion SS = ( Yˆ Y ) regreion Reidual we already covered SS ˆ reidual = ( Y Y ) SS total = SS regreion + SS reidual 5 Partitioning Variability Degree of freedom Total df total = N - 1 Regreion df regreion = number of predictor Reidual df reidual = df total df regreion df total = df regreion + df reidual 6 Partitioning Variability Variance (or Mean Square) Total Variance total = SS total / df total Regreion Variance regreion = SS regreion / df regreion Reidual Variance reidual = SS reidual / df reidual 7 9

10 Country X (Cig.) Y (CHD) Y' (Y - Y') (Y - Y') (Y' - Ybar) (Y - Ybar) Mean SD Sum Y' = (.04*X) +.37 Example 8 Example SS = ( Y Y ) = ; df = 1 1 = 0 Total SS = ( Yˆ Y ) = ; df = 1 (only 1 predictor) regreion SS = ( Y Yˆ ) = ; df = 0 1 = 19 total reidual regreion reidual total regreion reidual ( Y Y ) = = = N 1 0 ( Yˆ Y ) = = = ( Y Yˆ ) = = = N 19 Note : 9 reidual = Y Yˆ Coefficient of Determination It i a meaure of the percent of predictable variability r = the correlation quared or SS regreion r = SSY The percentage of the total variability in Y explained by X 30 10

11 r =.713 r for our example r =.713 =.508 or SS regreion r = = =.507 SS Y Approximately 50% in variability of incidence of CHD mortality i aociated with variability in moking. 31 Coefficient of Alienation It i defined a 1 - r or SSreidual 1 r = SS Example =.49 Y SSreidual r = = =.49 SS Y 3 r, SS and Y-Y r * SS total = SS regreion (1 - r ) * SS total = SS reidual We can alo ue r to calculate the tandard error of etimate a: N 1 0 Y Yˆ = y r = = N 19 (1 ) 6.690* (.49)

12 Hypothei Teting Tet for overall model Null hypothee b = 0 a = 0 population correlation (ρ) = 0 We aw how to tet the lat one in Chapter Teting Overall Model We can tet for the overall prediction of the model by forming the ratio: regreion = F tatitic reidual If the calculated F value i larger than a tabled value (Table D.3 α =.05 or Table D.4 α =.01 ) we have a ignificant prediction 35 Teting Overall Model Example regreion = = reidual Table D.3 F critical i found uing thing df regreion (numerator) and df reidual. (demoninator) Table D.3 our F crit (1,19) = > 4.38, ignificant overall Should all ound familiar 36 1

13 SPSS output Model 1 Model Summary Adjuted Std. Error of R R Square R Square the Etimate.713 a a. Predictor: (Contant), CIGARETT Model 1 Regreion Reidual Total a. Predictor: (Contant), CIGARETT b. Dependent Variable: CHD ANOVA b Sum of Square df Mean Square F Sig a Teting Slope and Intercept The regreion coefficient can be teted for ignificance Each coefficient divided by it tandard error equal a t value that can alo be looked up in a table (Table D.6) Each coefficient i teted againt 0 38 Teting Slope With only 1 predictor, the tandard error for the lope i: e b = X Y Yˆ N 1 For our Example: e b = = =

14 Teting Slope and Intercept With only 1 predictor, the tandard error for the intercept i: 1 X 1 X e = a Y Yˆ Y Yˆ N + ( X X ) = N + SS For our Example: = = = * X 40 Teting Slope Thee are given in computer printout a a t tet. 41 Teting The t value in the econd from right column are tet on lope and intercept. The aociated p value are next to them. The lope i ignificantly different from zero, but not the intercept. Why do we care? 4 14

15 Teting What doe it mean if lope i not ignificant? How doe that relate to tet on r? What if the intercept i not ignificant? Doe ignificant lope mean we predict quite well? 43 15