Math 1710 Class 20. V2u. Last Time. Graphs and Association. Correlation. Regression. Association, Correlation, Regression Dr. Back. Oct.

Transcription

1 ,, Dr. Back Oct. 14, 2009

2 Son s Heights from Their Fathers Galton s Original 1886 Data

3 If you know a father s height, what can you say about his son s? Son s Heights from Their Fathers Galton s Original 1886 Data

4 Son s Heights from Their Fathers If you know a father s height, what can you say about his son s? > hts V2 V

5 Son s Heights from Their Fathers If you know a father s height, what can you say about his son s? >summary(hts$v3) (Sons) Min. 1stQu. Median Mean 3rdQu. Max >summary(hts$v2) (Fathers) Min. 1stQu. Median Mean 3rdQu. Max

6 Son s Heights from Their Fathers If you know a father s height, what can you say about his son s? >summary(hts$v3) (Sons) Min. 1stQu. Median Mean 3rdQu. Max >summary(hts$v2) (Fathers) > sd(hts$v3) [1] > sd(hts$v2) [1] Min. 1stQu. Median Mean 3rdQu. Max

7 Son s Heights from Their Fathers If you know a father s height, what can you say about his son s? >summary(hts$v3) (Sons) Min. 1stQu. Median Mean 3rdQu. Max >summary(hts$v2) (Fathers) Min. 1stQu. Median Mean 3rdQu. Max > cor(hts$v3, hts$v2) [1]

8 Son s Heights from Their Fathers Galton Data with Line of

9 Son s Heights from Their Fathers Suppose y = height son were linearly related to x = height father by y = β 1 x + β 0.

10 Son s Heights from Their Fathers Suppose y = height son were linearly related to x = height father by y = β 1 x + β 0. We d then have ȳ = β 1 x + β 0 as well and (y ȳ) = β 1 (x x).

11 Son s Heights from Their Fathers Suppose y = height son were linearly related to x = height father by y = β 1 x + β 0. We d then have ȳ = β 1 x + β 0 as well and (y ȳ) = β 1 (x x). Furthermore, the slope β 1 would be the ratio of standard deviations: β 1 = s son s father

12 Son s Heights from Their Fathers Galton Data with line of slope s son s father added in blue.

13 Son s Heights from Their Fathers The fact that the red best fitting line (termed the line of regression) actually has less slope than the blue line is one form of Galton s regression to the mean. Tall fathers tend to have tall sons, but the sons are typically not as extreme in their tallness as their fathers were.

14 Son s Heights from Their Fathers The fact that the red best fitting line (termed the line of regression) actually has less slope than the blue line is one form of Galton s regression to the mean. Tall fathers tend to have tall sons, but the sons are typically not as extreme in their tallness as their fathers were. In modern terms, we see two usages of the word regression here.

15 Son s Heights from Their Fathers Galton Data with Other Line of in pink: (predict father s height from son s)

16 Given: paired data (x 1, y 1 ),..., (x n, y n )

17 Given: paired data (x 1, y 1 ),..., (x n, y n ) Scatterplot: Plot x horizontally, y vertically.

18 Given: paired data (x 1, y 1 ),..., (x n, y n ) If one variable is potentially explanatory for the response of the other, choose the explanatory variable as x.

19 Given: paired data (x 1, y 1 ),..., (x n, y n ) is what we care about. If we know the x value of a point, does it tell us something about the likely y value?

20 Given: paired data (x 1, y 1 ),..., (x n, y n ) is what we care about. If we know the x value of a point, does it tell us something about the likely y value? (a number) and regression (a line) are just techniques to study association.

21 Given: paired data (x 1, y 1 ),..., (x n, y n ) Principal Aspects of : Direction: Strength: Form:

22 Given: paired data (x 1, y 1 ),..., (x n, y n ) Principal Aspects of : Direction: positive or negative Strength: Form:

23 Given: paired data (x 1, y 1 ),..., (x n, y n ) Principal Aspects of : Direction: positive or negative Strength: Form: Negative means as x increases, y generally decreases.

24 Given: paired data (x 1, y 1 ),..., (x n, y n ) Principal Aspects of : Direction: Strength: strong, moderate, or weak Form:

25 Given: paired data (x 1, y 1 ),..., (x n, y n ) Principal Aspects of : Direction: Strength: Form: linear, curved, or clustered

26 Examples Example b

27 Examples Example b My call: Direction: positive Strength: moderate Form: curved

28 Examples Example b using Data Desk

29 Examples Example c

30 Examples Example c My call: Direction: negative Strength: strong Form: linear

31 Examples Example d

32 Examples Example d My call: Direction: negative Strength: moderate Form: (perhaps some outliers)

33 Examples Example d using Data Desk

34 Examples Example e

35 Examples Example e My call: Direction: positive Strength: strong Form: linear

36 Examples Example f

37 Examples Example f My call: Direction: negative Strength: weak Form: curved, maybe an outlier

38 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ s y

39 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ s y Positive association means as x increases, so does y (And similarly when x decreases.)

40 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ Positive association means as x increases, so does y (And similarly when x decreases.) So for pos. association, most terms in the sum are either (+) (+) or ( ) ( ). s y

41 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ Positive association means as x increases, so does y (And similarly when x decreases.) So for pos. association, most terms in the sum are either (+) (+) or ( ) ( ). Thus with pos. association, r tends to be positive. s y

42 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ s y pos r pos. association neg. r neg. association

43 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ s y 1 <= r <= 1 (= ±1 only for perfect linear association)

44 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ s y 1 <= r <= 1 (= ±1 only for perfect linear association)

45 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ s y 1 <= r <= 1 (= ±1 only for perfect linear association) To see = ±1 for perfect linear association y = β 1 x + β 0 means s y = β 1 s x and ȳ = β 1 x + β 0.

46 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ To see = ±1 for perfect linear association y = β 1 x + β 0 means s y = β 1 s x and ȳ = β 1 x + β 0. r = 1 ( ) ( ) n 1 Σ xi x yi ȳ s x s y = 1 ( ) ( ) n 1 Σ xi x xi x β 1 = 1 n 1 Σ s x ( xi x s x ) 2 β 1 β 1 s y β 1 s x = β 1 β 1 where the last line used the definition of the variance.

47 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ If you ve studied vectors, the fact 1 <= r <= 1 comes from the same mathematics which explains why s y cos θ = v w v w has right hand side between 1 and +1.

48 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ s y r is unchanged if x and y are exchanged.

49 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ s y invariant under rescaling

50 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ s y Curved association and r=0 are consistent!

51 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ Curved association and r=0 are consistent! Five points along y = x 2. ( x = 0 and ȳ = 2.) s y x y (x-0) (y-2) (x-0)(y-2)

52 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ Curved association and r=0 are consistent! Five points along y = x 2. ( x = 0 and ȳ = 2.) s y x y (x-0) (y-2) (x-0)(y-2) r is exactly zero even though the association is very strong.

53 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ s y r is strongly affected by outliers.

54 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ s y samples from independent RV s r 0

55 Properties r = 1 ( n 1 Σ xi x s x ) ( ) yi ȳ s y X,Y indep std normal RV s ; set Y = ρx + 1 ρ 2 Y Then (X, Y ) will tend to generate data with r ρ. (e.g. ρ =.99 (1 ρ => 2 ) ρ =.14!)

56 What does Best Fitting Mean? Given any point (x i, y i )

57 What does Best Fitting Mean? Given any point (x i, y i ) and any line y = c 0 + c 1 x

58 What does Best Fitting Mean? Given any point (x i, y i ) and any line y = c 0 + c 1 x we can use the line to get a predicted value ŷ i = c 0 + c 1 x i

59 What does Best Fitting Mean? Given any point (x i, y i ) and any line y = c 0 + c 1 x we can use the line to get a predicted value and define the residual d i by ŷ i = c 0 + c 1 x i d i = y i ŷ i.

60 What does Best Fitting Mean? Best fitting means the line minimizing the sum Σdi 2 squares of the vertical distances. of the

61 What does Best Fitting Mean? Best fitting means the line minimizing the sum Σdi 2 of the squares of the vertical distances. Answer: We ll show the best fitting line ŷ = b 1 x + b 0 is given by ( ) sy b 1 = r s x

62 What does Best Fitting Mean? Best fitting means the line minimizing the sum Σdi 2 of the squares of the vertical distances. Answer: We ll show the best fitting line ŷ = b 1 x + b 0 is given by ( ) sy b 1 = r i.e.: Slope is r in standard deviation units. And b 0 = ȳ b 1 x i.e.: The point ( x, ȳ) lies on the line. s x

63 What does Best Fitting Mean? Strictly speaking, the word residual refers to this vertical distance d i JUST in the case that the line is the answer line ŷ = b 0 + b 1 x.