Psychology 282 Lecture #3 Outline

Transcription

1 Psychology 8 Lecture #3 Outline Simple Linear Regression (SLR) Given variables,. Sample of n observations. In study and use of correlation coefficients, and are interchangeable. In regression analysis, one variable is defined as dependent () and the other as independent (). SLR represents as linear function of. (If we have more than one IV, then method is called multiple linear regression, or MLR.) Relationship may be causal, but not necessarily. Use regression for various purposes: Explanation Prediction Will study SLR in terms of both geometric and algebraic representations.

2 Geomtric representation of SLR Recall scatterplot: Consider representation or approximation of relationship between and using straight line Using this line, for any selected individual, can obtained predicted, called ˆ.

3 3 Can then define residual or error in prediction, e, as vertical deviation of point from line. Can define residuals for entire sample. Note that different lines produce different residuals. Consider objective of choosing the best line: Find line that produces smallest residuals for sample. Algebraic representation of SLR Equation for a straight line: ˆ B 0 + B where ˆ is predicted value of. B 0 is intercept: value of where line crosses -axis, or predicted value of when 0. B is regression coefficient, or slope of line. Slope is defined as the ratio of vertical change to horiontal change, or. This value represents the change in predicted corresponding to a 1-unit increase in. For any individual, given and and an equation of the form ˆ B 0 + B, we can then obtain predicted value ˆ and residual e ˆ.

4 Can obtain residuals for each of n individuals in sample. Problem: Would like to find equation B 0 + B that makes these residuals small, thus providing best approximation of observed values. Need to define aggregate measure of sie of residuals in sample. Cannot just sum them to obtain e, because positive and negative residuals will cancel out. Consider squaring residuals, then summing: e This value will become smaller as selected equation, and corresponding line, approximates relationship better. Objective: Given scores for n individuals on and, find B 0 and B such that e is as small as possible, where e ˆ and ˆ B 0 + B. This is the principle of least squares, and e is the sum of squared residuals, to be minimied. ˆ 4

5 5 Minimiing the sum of squared residuals: Define θ e. ( ˆ) ( ( B0 + B )) ( B B 0 ) Problem: Find B 0 and B that minimie this quantity. Solution obtained by basic calculus: Obtain partial derivatives of θ with respect to B 0 and B. Set these two partial derivatives equal to ero. Solve this system of two equations for B 0 and B. Result: n B and B B n 0 These equations provide linear regression coefficient and intercept for SLR that minimie sum of squared residuals. Any other intercept and slope would produce higher e. See applet:

6 6 Relationship between regression coefficient and correlation coefficient: B r Given r, can compute B easily. Expressing regression equation in terms of : Given ˆ B 0 + B Residuals: e ˆ Then: Implies: ˆ + e B + B + e 0 Relationships among,, ˆ, and e: r observed from data. r 1 ˆ r 0 e r ˆ r

7 7 SLR for standardied variables Development of SLR above was for raw scores on and. Suppose we wished to do SLR after standardiing IV and DV: ( ) ( ) Means are ero: 0 and 0 Variances are 1.0: 1 and 1 Regression equation: ( n 1) ( n 1) ˆ B + Residuals: e 0 ˆ B Wish to find SLR equation that will make residuals as small as possible. Apply principle of least squares. Problem: Find B 0 and B that will minimie e

8 8 Solution: Use solution for raw score regression coefficient and intercept, converting to standardied variables. Results: Standardied regression coefficient: n B n n(0)(0) ( n 1) n(0) ( n 1 ) r Standardied intercept: B 0 B 0 B (0) 0 In SLR for standardied variables, the intercept will be ero, meaning the regression line must pass through the origin. The slope, or standardied regression coefficient, will be equal to the Pearson correlation coefficient.

9 9 SLR for standard scores can be represented as: ˆ r r + e Z Thus, the Pearson correlation coefficient has two distinct interpretations and uses: Measure of linear relationship. Standardied regression coefficient in SLR. SLR using raw vs. standard scores: Choice based on context, purpose. Desire to predict raw score, vs. desire to predict relative standing with respect to the mean. Relationship: B r r B

10 10 Regression toward the mean Standard scores represent deviation from the mean, in units. SLR for standard scores: ˆ r Bounds on value of r : r 1. 0 Implies that: ˆ Predicted score on must be relatively closer to mean than was observed score on. This is regression to the mean. A statistical phenomenon associated with regression and least squares. Substantive interpretation of this phenomenon is not justified.

11 11 Measures of strength of association In SLR we are interested in strength of association between IV and DV. Can be measured by r and by e. Consider other possible measures. Use notion of partitioning variance in. Variance in is partially accounted for by, with the remainder unaccounted for. Determine these portions. Consider first in standardied SLR: r + e Z ˆ + e Z Var ( ) Var( ˆ ) + Var( ez ) Var ( ) Var( r ) + Var( ez ) 1 r Var( ) + Var( e Z ) 1 r +Var( e Z )

12 1 This expression shows that the variance in, which is 1.0, can be partitioned into two parts: Variance accounted for by, which is equal to r. Variance not accounted for by, which is equal to (1-r ). Thus, r indicates proportion of variance in DV accounted for by its linear relationship with IV. This is an important device for interpreting strength of relationship implied by r. We can define similar partitioning of variance in using raw variance. Divide that variance into two portions: Variance in accounted for by its linear relationship with : ˆ r Variance in not accounted for by its linear relationship with : r ( 1 ) ( ˆ)

13 13 In regression analysis we are often interested in the standard deviation of the residuals. From this last term we can define standard deviation of raw residuals: r ( 1 ) ( ˆ) This is a sample statistic. We often wish to estimate the standard deviation of residuals that would be obtained in the population. This value is called the standard error of estimate (SE). The sample value of ( ˆ ) tends to overestimate this value; i.e., it is a biased estimate of the true standard error of estimate. An unbiased estimate of SE can be obtained by: SE ( ˆ) ( n ) This value is provided by regression software and is used to construct confidence intervals for predicted scores and for other purposes.