Associations between variables I: Correlation

Transcription

1 Associations between variables I: Correlation Part 2 will describe linear regression.

2 Variable Associations Previously, we asked questions about whether samples might be from populations with the same mean for a specific variable. Now, we are interested in relationships among variables within a sample.

3 Data Structure variables linked to same individual > X = read.table("army.rdata") > X SEX AGE RACE HEAD_LENGTH HEAD_BREADTH EAR_LENGTH

4 Why might variable values be coupled? LENGTH genes Leg Length Arm Length

5 Why might variable values be coupled? LENGTH genes Developmental nutrition Leg Length Arm Length

6 Independence Almost everything one can measure on a subject has some degree of association, but if there is none, the values are said to be independent. E.g., Handedness (R/L) and Ear Length (I have no idea if these are independent, but seems reasonable)

7 Independence : examples How many hours of sleep versus outside humidity Scholarly aptitude versus bone density

8 Dependence : examples Average number of cigarette packs a day versus number of years lived (for smokers) Class attendance versus amount of rain...

9 Correlation Correlation is the measure of linear dependence between two variables as one increases, the other is observed to increase (or decrease), as well. Pearson's Product-Moment Correlation is probably the most familiar measure of this association. PPMC, r, measures the strength of linear dependence between two variables.

10 Correlation A lack of independence means there is some association (or relationship) between values for different variables. One type of such an association is called correlation which is measured by a number of different statistical measures. NOTE! There are forms of association other than that measured by a particular correlation coefficient Independent does not imply non-correlated Non-correlated does not imply independent

11 Example y1 = x x in range [-1 to 1] y2 = x*x y1 and y2 are not correlated y1 and y2 are clearly dependent!!! If I know y1, I can calculate y2 with 100% certainty

12 Correlation/Causation Higher scores are correlated with higher fluency of FCAT passages Higher fluency is not necessarily the cause of higher scores, although it is certainly possible Can we quantify this correlation?

13 Correlation/Causation dfr$score > cor(dfr$score, dfr$fluency) cor(x, y) = cov(x, y) sd(x)sd(y) dfr$fluency

14 Variance/Covariance cov(x, y) = E((x µ)(y µ)) var(x, x) = E((x µ) 2 )= 2 The above formulas apply to populations

15 Correlation between two random variables cor(x, y) = E((x µ)(y µ)) (x) (y) If x and y are identical, cor(x, x) = E((x µ)2 ) (x) 2 =1 E(x) is calculated by choosing n values of x from the population, computing the average, and letting n go to (infinity)

16 Correlation between two samples s1 and s2 Correlation: cor(s1,s2) Covariance: cov(s1,s2) N1 = (s1-mean(s1)) / sd(s1) N2 = (s2-mean(s2)) / sd(s2) correlation: r = cov(n1,n2) N1 and N2 are normalized to have zero mean and standard deviation of 1

17 Independence vs correlation X = N(0,1) Y = X*X X and Y are clearly not independent since given X, the value of Y is known exactly y Yet, mathematically, X and Y have zero correlation > x = rnorm(1000) > y = x*x > cor(x,y) [1] x

18 r If two values are perfectly, linearly correlated, then r = 1.0. If two values are perfectly, negatively, linearly correlated, r = If two variables are independent (no relationship), r = 0.0 Remember, though, uncorrelated is not the same as independent. Two values can be dependent, yet have zero correlation!!!

19 r

20 Y1, Y2 axes are not normalized

21 Y1-mean(y1), Y2 y1bar=mean(y1)

22 Y1-Y1bar, Y2-Y2bar

23 (Y1-Y1bar)/sd(Y1), (Y2-Y2bar)/sd(Y2) variables are now normalized

24 Y1,Y2 (Y1-Y1bar)/sd(Y1), (Y2-Y2bar)/sd(Y2) Axis ranges are not consistent Axis ranges are consistent

25 ?cor.test cor.test {stats} Test for Association/Correlation Between Paired Samples Description Test for association between paired samples, using one of Pearson's product moment correlation coefficient, Kendall's tau or Spearman's rho. Usage cor.test(x,...) ## Default S3 method: cor.test(x, y, alternative = c("two.sided", "less", "greater"), method = c("pearson", "kendall", "spearman"), exact = NULL, conf.level = 0.95, continuity = FALSE,...) Test for association between paired samples, using one of Pearson's product moment correlation coefficient, Kendall's tau or Spearman's rho. ## S3 method for class 'formula' cor.test(formula, data, subset, na.action,...) Arguments x, y numeric vectors of data values. x and y must have the same length.... Details... Value...

26 cor.test(y1,y2) > cor.test(y1,y2) Pearson's product-moment correlation data: y1 and y2 t = , df = 98, p-value = 2.247e-13 alternative hypothesis: true correlation is not equal to 0 95 percent confidence interval: sample estimates: cor The correlation is significantly different from zero. There is sufficient evidence to accept the alternative hypothesis

27 > r = 0.3 > y1 = rnorm(100) > y.temp = rnorm(100) > y2 = (y1*r) + y.temp*(sqrt(1-r*r)) > cor.test(y1,y2) Play Time... Pearson's product-moment correlation data: y1 and y2 t = , df = 98, p-value = 4.366e-05 alternative hypothesis: true correlation is not equal to 0 95 percent confidence interval: sample estimates: cor Generate your own pair of variables with a given correlation

28 Play Time... > r = 0.3 > y1 = rnorm(100) > y.temp = rnorm(100) > y2 = (y1*r) + y.temp*(sqrt(1-r*r)) Generate and plot and compute r,... y1,y2 ~ N(0,1); r=1.0, -1.0, 0, 0.3, -0.7 y1,y2 ~ N(0,0.2),N(0,2.3); r=1.0, 0, -0.7 y1,y2 ~ N(1,0.7),N(100,12.5); r=0.3 y1', y2' subtract off means and divide by sd

29 User R functions I wish to repeat a sequence of commands many times with different parameters For example: generate n random numbers plot a histogram of these 100 random numbers (create 6 plots per page) Let n take the values 100, 500, 5000, 10000

30 Method 1 par(mfrow=c(2,3)) n=100 hist(rnorm(n)) n=500 hist(rnorm(n)) n=5000 hist(rnorm(n))... Simply type everything out Not difficult since very few commands n = 100 rand = rnorm(n) hist(rand)

31 Method 2: scripts Put the following command in a script called histo.r plot(hist(rnorm(n)) par(mfrow=c(2,3)) n=100 source( histo.r ) n=500 source( histo.r )... Note that running the script histo.r without a value for n will give and error (undefined variable)

32 Method 3: functions In a script called histof.r, add the following: histof = function(n=100) { hist(rnorm(n)) } In R, > par(mfrow=c(2,3) > source( histof.r ) > histof() > histof(500) > histof(n=5000) >... Most efficient and easy to use

33 Use Apply to run all cases (script: histof.r) histof = function(n=100) { title = paste("normal histogram: ", n, " points") hist(rnorm(n), main=title) } # par(mfrow=c(2,3)) par(ask=t) histof() histof(n=500) histof(n=2000) histof(n=5000) # par(mfrow=c(2,3)) apply(as.matrix(c(100,200,300,400,500,5000)), 1, histof) histof(n) is a user-defined function Use the function like any R function Use apply() to simplify your code

34 For loops histof = function(n=100) { title = paste("normal histogram: ", n, " points") hist(rnorm(n), main=title) } # par(mfrow=c(2,3)) par(ask=t) counts = c(100,200,300,400,500,5000) for (i in counts) { histof(i) }

35 Return to: Play Time... > r = 0.3 > y1 = rnorm(100) > y.temp = rnorm(100) > y2 = (y1*r) + y.temp*(sqrt(1-r*r)) Generate and plot and compute r,... y1,y2 ~ N(0,1); r=1.0, -1.0, 0, 0.3, -0.7 y1,y2 ~ N(0,0.2),N(0,2.3); r=1.0, 0, -0.7 y1,y2 ~ N(1,0.7),N(100,12.5); r=0.3 y1', y2' subtract off means and divide by sd

36 Functions Create a function to store > r = 0.3 > y1 = rnorm(100) > y.temp = rnorm(100) > y2 = (y1*r) + y.temp*(sqrt(1-r*r))

37 play = function(r=r, n=100, mean=0, sd=1) { y1 = rnorm(n, mean=mean, sd=sd) y.temp = rnorm(n, mean=mean, sd=sd) y2 = (y1*r) + y.temp*(sqrt(1-r*r)) } correlation = cor(y1,y2) cat("r=",r," correlation= ", correlation, "\n") return(correlation)

38 Using the function play(...) print(" ") print(" r = c(.1,.5,.8) ") corre = as.matrix(c(.1,.5,.8)) apply(corre, 1, play, 100, 0, 1) print(" ") print(" r = seq(-1,1,.1) ") function arguments: r = corre (a matrix column or row) n= 100 mean= 0 sd=1 corre = as.matrix(seq(-1,1,.1)) apply(corre, 1, play, 100, 0, 1)

39 Criminals What is the correlation between middle finger length and height? Is it statistically significant? Are the mean finger length and mean height significantly different? (t.test)

40 Criminals Standardize (y->y') (subtract mean/divide by variance) Plot Compute r directly (manual calculation) Compare cor()

41 Criminals > X = read.table("criminal_cambridge.rdata") > dim(x) [1] > X1 = subset(x,source=="criminal") > dim(x1) [1] > head(x1,3) source height.cm middle.finger.cm 1 criminal criminal criminal Is there a correlation between h and mfl???

42 Criminals Extract h and mfl for criminals (see above) Plot h vs. mfl Compute r for (h and mfl) and (mfl and h) Test r for h, mfl Note, plot(x) where x=full data set cor(h,l) v. cor(y.hl) cov()

43 Assumptions Significance testing of r assumes that the sample pairs are independent and identically distributed and follow a bivariate normal distribution. What if they are not? Transformations? Non-parametric tests usually based on ranks Randomization test

44 Ranks A rank is the position of an observation in a list of observations sorted by magnitude. E.g., see?rank?sort > y1 = rnorm(5) > y2 = rnorm(5) > y1 [1] > y2 [1] > rank(y1) [1] > rank(y2) [1] * Ties are require special handling.

45 Spearman's Correlation Spearman's rank correlation coefficient or Spearman's rho,...is a non-parametric measure of statistical dependence between two variables. It is computed as the product-moment correlation of the ranks of the two variables.

46 Spearman's rho > y1 = rnorm(5); y2 = rnorm(5) > y1 [1] > y2 [1] > rank(y1) [1] > rank(y2) [1] > cor(y1,y2) [1] > cor(y1,y2,method="sp") # Spearman Correlation (rank-based) [1] -0.1 > cor(rank(y1),rank(y2)) [1] -0.1

47 Kendall's tau Kendall rank correlation coefficient, commonly referred to as Kendall's tau (τ) coefficient, is a statistic used to measure the association between two measured quantities. A tau test is a nonparametric hypothesis test for statistical dependence based on the tau coefficient. Specifically, it is a measure of rank correlation, i.e., the similarity of the orderings of the data when ranked by each of the quantities.

48 Kendall's tau (part one) (part two) Comparison between Tau and Spearman

49 Both Spearman and Tau methods can be used for two variables?cor cor(x, y = NULL, use = "everything", method = c("pearson", "kendall", "spearman"))