Some Notes on ANOVA for Correlations. James H. Steiger Vanderbilt University

Transcription

1 Some Notes on ANOVA for Correlations James H. Steiger Vanderbilt University

2 Over the years, a number of people have asked me about doing analysis of variance on correlations. Consider, for example, a way ANOVA type setup, in which there are independent samples of size N = 65 in each cell, and the following correlations between two variables are observed (1.1) In this case, a main effect for rows simply asks whether the average correlation varies over the two levels of the row effect, and the main effect for columns asks the corresponding question about the 4 levels of the column effect. The interaction effect asks whether the change in correlation across columns is the same for both rows. The question of whether ideas about means generalize meaningfully to correlation coefficients (i.e., whether one should be doing ANOVA type hypothesis tests on correlations) is a complicated one, and I will not address it here. However, the computation of test statistics corresponding to ANOVA is a straightforward special case of a linear model for correlations discussed in Steiger and Browne (1984). In what follows, we will assume multivariate normality. The method may be extended to the ADF case using results in Steiger and Hakstian (1982). A key result, dating back at least to Pearson and Filon (1898), is that under multivariate normality, a correlation coefficient ri, jhas an asymptotic distribution that is normal, with mean ρ i, jand that 1/2 N ( ri, j ρi, j) (1.2) has a variance of ( 1 ρ 2 ) 2 i, j (1.3) If correlations come from independent samples, they of course have a covariance of zero. If they are measured on the same sample, they have a covariance with a rather complicated formula. Specifically, the covariance of ψ = Cov N 1/2 ( r ρ ), N 1/2 ( r ρ ) is ψ ( ) ( ρjh, ρjk, ρkh, )( ρkm, ρkh, ρhm, ) + ( ρjm, ρjh, ρhm, )( ρkh, ρk, jρjh, ) + ( ρjh, ρjm, ρmh, )( ρkm, ρk, jρjm, ) + ( ρjm, ρjk, ρkm, )( ρkh, ρkm, ρmh, ) ijkh, i, j i, j kh, kh, 1 jk, hm 2 = (1.4) Steiger and Browne (1984) discuss Asymptotic Tests of Linear Hypotheses on correlations. They deal specifically with hypotheses of the form H 0 : Mρ = h (1.5) where M is a specified g k matrix of rank g, and h is a specified g 1 vector of constants, usually zero. Let Ψ be contain elements defined as in equations (1.2) (1.4). Then (see Steiger & Browne, 1984, equation (3.3),

3 2 1 χ = N( Mr h)( MΨM ) ( Mr h ) (1.6) will have a chi-square distribution with g degrees of freedom. Defining M for a factorial ANOVA design is a straightforward application of results on Kronecker products. For an excellent introduction, see Woodward, Bonett, and Brecht (1990). In what follows, I give a condensed but hopefully adequate description. For simplicity, suppose we have a 2 2 ANOVA, with 2 observations per cell. The cell means are μ1,1 μ1,2 U = μ2,1 μ (7) 2,2 Notice that it is easy to write the null hypothesis in the form AUB = 0. We can write the null hypothesis for rows as μ1,1 μ1,2 1 H0 :1 [ 1] μ2,1 μ 2,2 1 (8) = μ + μ ( μ + μ ) = 0 1,1 1,2 2,1 2,2 This can be written H 0 : OU1 = 0 (9) We call O an omnibus contrast matrix, and it is always of the same basic form, i.e., O= [ I 1 ] (10) Notice that we have an omnibus contrast matrix on the left, and a summing vector on the right which collapses across columns. In a similar vein, we can write the column effect null hypothesis as μ1,1 μ1,2 1 H0 :1 [ 1] μ2,1 μ 2,2 1 (11) = μ + μ ( μ + μ ) = 0 1,1 2,1 1,2 2,2 Now we are comparing columns after collapsing across rows. This is of the form H 0 : 1UO = 0 (12) The interaction null hypothesis is that there are no differences of differences, and consequently is of the form H 0 : OUO = 0 (13) As straightforward as this system is, unfortunately the null hypothesis requires all the means in a single vector μ, not in a matrix, and there is only one hypothesis matrix, not two as in the examples above. So how do we proceed? First, we need to define the Kronecker Product.

4 The Kronecker product of two matrices A and B is denoted A B, and can be written in partitioned form as a1,1b a1,2 B a1, qb a2,1b a2,2b a2, qb paq rbs = (14) ap,1 B ap,2 B ap, qb The Kronecker product above is of order pr qs. Kronecker products have some interesting properties. Let vec c ( X ) be a column vector consisting of the columns of X stacked on top of each other. Let vec r ( X ) be a column vector consisting of the rows of X transposed into columns and stacked on top of each other. Then vec c( BSA ) = ( A B) vec c( S ) (15) and, since vec r( S ) = vec c( S ), we also have vec r( ASB ) = ( A B) vec r( S ) (16) vec r( ASB) = ( A B ) vec r( S ) (17) Consider the null hypothesis of Equation (9). This can be written as H 0 : OU1 = 0, or as H 0 : ( O 1 ) vec r ( U) = 0 (18) Example. Consider the null hypothesis of no row effect in the 2 2 ANOVA. μ1,1 μ 1,2 ( O 1 ) μ = ([ 1 1] [ 1 1] ) μ2,1 μ 2,2 (19) μ1,1 μ 1,2 = [ ] μ2,1 μ2,2 Equation (19) gives ( μ1,1 μ1,2 ) ( μ2,1 μ2,2 ) + +, which is the quantity that is zero when there is no row main effect.

5 General Specification of the Hypothesis Matrix in Factorial Anova 1. Call the effects A, B, C etc. 2. A conformable Omnibus Matrix for effect A is denoted O A. 3. A conformable Summing Vector for effect A is denoted 1 A. 4. A selection vector s A, j is a row vector conformable with factor A with a 1 in position j, and zeroes elsewhere. 1. To construct an M for a main effect, use an O matrix for that effect and a summing vector for all other effects. 2. To construct an M for an interaction effect, use an O matrix for all effects in the interaction, and a summing vector for all effects not in the interaction. 3. To construct the M for a simple main effect, use selection vector(s) to select the levels for the simple main, then use an O matrix for the main effect tested. Examples. 2-way ( A B) ANOVA A Main Effect O A 1 B B Main Effect 1 A O B AB Interaction Effect OA O B Simple Main Effect of A at Level 1 of B O A s B,1

6 Examples. 3-way ( A B C) ANOVA ABC Interaction Effect OA OB O C A Main Effect OA 1 B 1 C Now that we understand how to express univariate ANOVA in matrix notation, we move on to MANOVA. Returning to the original question how would we perform a 2 4 ANOVA? We stack the correlations row-wise, so r = (1.20) Since each correlation is from an independent sample, Ψ= (1.21) Suppose we wish to test for a main effect for columns. In this case, M is given by M = [ 1 1] = (1.22)

7 Computing the test statistic requires a program like Mathematica capable of matrix algebra. It should be noted that Mathematica does not include a Kronecker (direct) product routine as part of its basic functionality, or indeed part of its Matrix Algebra routines! There is a package downloadable from the Wolfram user-supported library called OPMatrix that implements a Kronecker product function. Below is the code for computing our test statistic. The chi-square, with 3 df, is , obviously not significant.

8 References Pearson, K., & Filon, L.N.G. (1898). Mathematical contributions to the theory of evolution: IV. On the probable error of frequency constants and on the influence of random selection on variation and correlation. Philosophical Transactions of the Royal Society of London, Series A, 191, Steiger, J.H., & Browne, M.W. (1984). The comparison of interdependent correlations between optimal linear composites. Psychometrika, 49, Steiger, J.H., & Hakstian, A.R. (1983). A historical note on the asymptotic distribution of correlations. British Journal of Mathematical and Statistical Psychology, 36, 157. Woodward, J. A., Bonett, D. G., & Brecht, M-L. (1990). Introduction to linear models and experimental design. San Diego, CA: Harcourt Brace Jovanovich.