Rejection regions for the bivariate case The rejection region for the T 2 test (and similarly for Z 2 when Σ is known) is the region outside of an ellipse, for which there is a (1-α)% chance that the test statistic would fall for a random data set. If instead you reject the null hypothesis if either of two univiariate t-tests fails, then the rejection rejection is the area outside of a rectangle. STAT476/STAT576 February 20, 2015 1 / 46
Rejection regions STAT476/STAT576 February 20, 2015 2 / 46
Rejection regions STAT476/STAT576 February 20, 2015 3 / 46
Rejection regions The graphic illustrates the idea that for a given α, if you don t correct for multiple testing, then there are points where you don t reject µ 1 = µ 01 and µ 1 = µ 01 (or µ 11 = µ 12 and µ 21 = µ 22 for the two-sample problem) when you should (dark grey). This indicates lower power for the univariate approach. Similarly, the points in the light grey areas are points where you reject the null hypothesis using the univariate approach but shouldn t, and wouldn t using the multivariate approach. The white region indicates cases where both approaches would not reject the null hypothesis. Points outside both the rectangle and the ellipse are cases where both approaches would reject the null hypothesis. STAT476/STAT576 February 20, 2015 4 / 46
Properties of the T 2 statistic We need n 1 > p (one sample) or n 1 + n 2 2 > p (two samples) for S and S pl, respectively, to be nonsingular The degrees of freedom is the same as for the univariate t-tests: n 1 for one sample, n 1 + n 2 2 for the two-sample test The density for T 2 is a scaled version of the F distribution, so it is skewed, and you reject H 0 : µ 1 = µ 0 or H 0 : µ 1 = µ 2 for large values of T 2 even though the test is not one-sided The T 2 approach turns out to be equivalent to the likelihood ratio approach, which has good properties on theoretical grounds. STAT476/STAT576 February 20, 2015 5 / 46
Tests on individual variables conditional on rejecting the multivariate test As illustrated in the graphic with the dark grey region, there are cases where H 0 : µ 1 = µ 2 is rejected but H 0 : µ 1j = µ 2j is not rejected using univariate t-tests for any particular j. However, it is possible to construct a linear combination of the variables, z = a y, for which a univariate test will reject H 0 : µ z1 = µ z2. We could also write this hypothesis as H 0 : a µ 1 = a µ 2. STAT476/STAT576 February 20, 2015 6 / 46
Tests on individual variables conditional on rejecting the multivariate test To test the linear combination, we z j = a y j, j = 1, 2, and the pooled sample variance of z 1 and z 2 is a Sa. Therefore the t test statistic, as a function of a, is: t(a) = z 1 z 2 (1/n1 + 1/n 2 )s 2 z = a y 1 a y 2 (1/n1 + 1/n 2 )a S pl a The value of a which maximizes t 2 (a) is any multiple of a = S 1 (y 1 y 2 ) STAT476/STAT576 February 20, 2015 7 / 46
The discriminant function For a = S 1 (y 1 y 2 ), z = a y is called the discriminant function. If H 0 : µ 1 = µ 2 is rejected, then constructing a = S 1 (y 1 y 2 ) can be done to examine the a j values to determine which variables contribute most to the rejection of the null hypothesis. STAT476/STAT576 February 20, 2015 8 / 46
Other procedures to follow up a multivariate rejection of H 0 : µ 1 = µ 2 do univariate t-tests with some risk of inflated type I error do univariate t-tests with Bonferroni corrections for α levels. If testing p variables, then use α/p instead of α. For example, if p = 5, then use α =.01 instead of α =.05. Note that the book notates this as t α/2p,n1 +n 2 2, which means t α/(2p),n1 +n 2 2, for the critical t values. Use T α,p,n1 +n 2 2 = Tα,p,n 2 1 +n 2 2. That is use the square root of the critical value of Hotelling s T 2 as a the critical value for univariate t-tests, which is more conservative than Bonferroni corrections, but allows looking at linear combinations as well. There are other methods as well that we will see later in the course. STAT476/STAT576 February 20, 2015 9 / 46
Follow up tests The first procedure is too liberal (rejects too easiy), while the second and third tests are too conservative (often don t reject when they should). If you carry out individual tests only after an overall T 2 test says to reject, then you reject less often (because you might not reject on the first step), so it makes the first procedure less liberal and more acceptable, while it makes the second and third tests more conservative. The probability of rejecting one of the p univariate tests is called the overall α or experimentwise error rate. Simulations have been done to compare the performance of these different methods. Here we compare doing univariate t-tests either conditional or not conditional on rejection using T 2. No Bonferroni correction is used. STAT476/STAT576 February 20, 2015 10 / 46
Rejection regions STAT476/STAT576 February 20, 2015 11 / 46
Follow up tests The results suggest that univariate tests conditional on rejection using T 2 is quite acceptable (still slightly conservative since error rate is less than α), and would be expected to be more powerful than using a Bonferroni correction. STAT476/STAT576 February 20, 2015 12 / 46
The discriminant function: example Recall the data from the psychological testing on men and women. For this data, to find the discriminant function, we have 0.5104 a = S 1 pl (y 1 y 2 ) = 0.2033 0.4660 0.3097 The linear combination that has the greatest separation of the two groups is a y = 0.5104y 1 0.2033y 2 + 0.4660y 3 0.3097y 4 Thus, y 1 an y 3, pictorial inconsistencies and tool recognition, are the variables that separate the men and women samples the most for these data. STAT476/STAT576 February 20, 2015 13 / 46
Computing T 2 and matched pairs The book has some sections on how to compute T 2 using other computer output, such as multiple regression output, which we will skip. It is easy enough to compute T 2 these days. The book also has a section on paired differences which will we skip. The basic idea is that we can have multivariate analogues of the matched pairs t-test. This is appropriate for the couples data that we looked at earlier. Basically, you can look at differences within a pair and analyze these as raw data. For the couples data, we could look the age difference and height difference for couples and analyze these two differences as a single bivariate normal sample. STAT476/STAT576 February 20, 2015 14 / 46
Test for additional information For a two-sample problem, one thing we may wish to do is to determin, given a set of p + q variables, whether q of the variables are redundant. In other words, does the addition of the q variables significantly increase T 2? Let y j denote the p-variable data and let x j denote the q-variable data where there are two samples, j = 1, 2. The samples can be described as follows: STAT476/STAT576 February 20, 2015 15 / 46
Test for additional information STAT476/STAT576 February 20, 2015 16 / 46
Test for additional information STAT476/STAT576 February 20, 2015 17 / 46
Test for additional information In the previous slide, v = n 1 + n 2 2. T 2 (x y) is distributed as T 2 q,v p, but it is probably easier to convert to an F using T 2 p+q and T 2 p. The F test has q and v p q + 1 numerator and denominator degrees of freedom. The (null) hypothesis that the additional information in x is redundant is rejected for F values that are sufficiently large. STAT476/STAT576 February 20, 2015 18 / 46
Example with psychological data For this data set, you could test whether any of the variables is redundant (so do a separate test for each of the four variables, conditional on the other three being present. The critical value is 4.002. Calculating the statistics for each variable yields: STAT476/STAT576 February 20, 2015 19 / 46
Example with psychological data Since variable y 2 has a test statistic lower than 4.002, it appears to not contribute significantly to the separation between men and women, given that the other three variables are present. Thus, if the goal was to create a psychological test that could distinguish between men and women, the other three variables would have been sufficient. On the other hand, given that three other variables were present, y 1, y 3 and y 4 all contributed significantly to increasing T 2 and threfore to the separation between men and women. STAT476/STAT576 February 20, 2015 20 / 46
Profile Analysis When multivariate data are the result of repeated measures, you might want to see how the mean is changing over time. As an example, you might have data on blood pressure taken once per month for the same set of patients. It can be a little bit ambiguous whether you think of this as multivariate or univariate data. Suppose we just keep track of systolic blood pressure. You could think of the response as blood pressure, with the time (month) and individual as predictors. In this case, the number of observations is the number of individuals times the number of time points. Or you could think of the number of observations as the number of individuals, with each observation as being multivariate with one variable (or response) for each time point. STAT476/STAT576 February 20, 2015 21 / 46
Profile analysis Profile analysis can also be used when variables are not ordered in time (or any other way), but the values are on a similar scale, such as in psychological testing. A profile plot uses the subscript of the variable on the x-axis and the means µ 1,..., µ p on the y-axis. We can do both one-sample and two-sample profile plots. We ll start with one sample. STAT476/STAT576 February 20, 2015 22 / 46
Profile plot STAT476/STAT576 February 20, 2015 23 / 46
Profile plot The null hypothesis that all means are equal H 0 : µ 1 = = µ 2 can be interpreted graphically as the hypothesis that the profile plot is flat. The alternative hypothesis is H 1 : µ i µ j for some i, j. Note that these hypotheses are the same as for ANOVA, but the analysis will be different because the columns are not assumed to be independent. In ANOVA, you would have p populations that are assumed to be independent samples. Here we have a single population but are sampling correlated variables from this population. STAT476/STAT576 February 20, 2015 24 / 46
Profile plot Two equivalent ways of expressing the null hypothesis are µ 1 µ 2 0 H 0 : µ 2 µ 3 = 0 µ p 1 µ p 0 µ 1 µ 2 H 0 : µ 1 µ 3 = µ 1 µ p 0 0 0 STAT476/STAT576 February 20, 2015 25 / 46
Profile plot The left-hand matrices have entries that are linear combinations of µ 1,..., µ p, so we can represent them using matrix multiplications: 1 1 0 0 1 1 0 0 0 1 1 0 C 1 :...., C 1 0 1 0 2 :...., 0 0 0 1 1 0 0 1 Here C 1 and C 2 are (p 1) p and have rank p 1. The null hypotheses can then be written as H 0 : C 1 µ = 0 and H 0 : C 2 µ = 0 STAT476/STAT576 February 20, 2015 26 / 46
Profile plot More generally, if a matrix satisfies Cj = 0, then the rows of C sum to 0. If C has rank p 1, then Cµ = 0 can only be satisfied if µ 1 = = µ 2. In particular if one of the rows of C, say the ith row, sums to 0, then c i µ is a contrast. For example, we might have c i µ = µ 1 + µ 2 2 µ 1 + µ 2 + µ 3 3 Here C, and C 1 and C 2 from the previous slide, are called contrast matrices. STAT476/STAT576 February 20, 2015 27 / 46
Profile plot To test the null hypothesis, we let z = Cy and S z = CSC Assuming y N p (µ, Σ), then z N p 1 (0, CΣC /n) STAT476/STAT576 February 20, 2015 28 / 46
Profile plot To convert this to a T 2 squared statistic, you can use T 2 = z (CSC /n) 1 z and this has a Tp 1,n 1 2 distribution. To convert to an F statistic, you can use v p + 1 Tp,v 2 = F p,v p+1 vp (n 1) (p 1) + 1 T 2 = F (p 1)(n 1) p 1,n 1 (p 1)+1 n p + 1 (p 1)(n 1) T 2 = F p 1,n p+1 STAT476/STAT576 February 20, 2015 29 / 46
Profile plots: two samples We can also do profile analysis with two plots. Here we have two profiles. Graphically, we might be interested in the following questions: Are the two plots both flat? Are the two plots both parallel? Are the two plots at the same level (is one higher than the other)? STAT476/STAT576 February 20, 2015 30 / 46
Profile plot: two samples STAT476/STAT576 February 20, 2015 31 / 46
Profile plot: two samples For two samples, the null hypothesis that the two plots have the same slopes (are parallel) can be tested by checking whether differences between means are equal for the two groups. This can be written as H 0 : µ 1,j µ 1,j 1 = µ 2,j µ 2,j 1 for j = 1,..., p. In matrix notation, this is STAT476/STAT576 February 20, 2015 32 / 46
Profile analysis: two samples The test statistic is T 2 = (Cy 1 Cy 2 ) [( 1 n 1 + 1 n 2 ) CS pl C ] 1 (Cy 1 Cy 2 ) = n 1n 2 n 1 + n 2 (y 1 y 2 ) C [CS pl C ] 1 C(y 1 y 2 ) Note that if you try to distribute the inverse, you might be tempted to have terms like C 1 C, but this incorrect as C doesn t have an inverse since it is not square. The test statistic has a T 2 distribution with p 1 and n 1 + n 2 2 degrees of freedom, which again can be converted to an F. STAT476/STAT576 February 20, 2015 33 / 46
Profile analysis: two samples The biggest descrepancy in the slopes can be found by looking at the discriminant function, for which we use a = (CS pl C ) 1 C(y 1 y 2 ) If the largest component of a is a i, then the i slope has largest violation of parallel slopes. This approach is similar to testing for an interaction in two-way ANOVA, but again ANOVA assumes independent populations, where here we allow correlation between the p variables. STAT476/STAT576 February 20, 2015 34 / 46
Profile analysis: two samples A second question is whether two samples have the same overall level, ignoring possible interaction. This hypothesis can be expressed as or or H 0 : µ 11 + + µ 1p = µ 21 + + µ 2p H 0 : j µ 1 = j µ 2 H 0 : j (µ 1 µ 2 ) = 0 This null hypothesis can be false even when there is no interaction, such as when the plots are parallel but one plot is higher than the higher. On the other hand, when there is interaction, but the two plots have the same average value, this null hypothesis can be satisfied. STAT476/STAT576 February 20, 2015 35 / 46
Profile plot: two samples STAT476/STAT576 February 20, 2015 36 / 46
Profile analysis: two samples In this case the test statistic is t = j (y 1 y 2 ) j S pl j(1/n 1 + 1/n 2 ) which has a t distribution with n 1 + n 2 2 degrees of freedom. STAT476/STAT576 February 20, 2015 37 / 46
Profile analysis: two samples Finally, a third hypothesis is that the two profiles are both flat, which would imply that there is no interaction but is stronger in requiring that the slopes also don t change. This hypothesis does not require the same overall mean for the two profiles. STAT476/STAT576 February 20, 2015 38 / 46
Profile analysis: two samples Because lack of parallelism automatically implies that at least one curve is not flat, the hypothesis that both curves are flat is more interesting if the hypothesis of parallelism can t be rejected. If it is rejected, then you could test individually whether each curve was flat using a one-sample technique. If the curves are roughly parallel, then it makes sense to test for flatness (if this is of interest). This is a bit like testing for a main effect in an ANOVA after finding no interaction. If there is an interaction, then you normally wouldn t also test for a main effect. STAT476/STAT576 February 20, 2015 39 / 46
Profile plot: two samples STAT476/STAT576 February 20, 2015 40 / 46
Profile plot: two samples Recall the example of the psychological profile of men and women: STAT476/STAT576 February 20, 2015 41 / 46
Profile plot: two samples STAT476/STAT576 February 20, 2015 42 / 46
Profile plot: two samples STAT476/STAT576 February 20, 2015 43 / 46
Profile plot: two samples For this data, the profile plot appears to not be very parallele, and one group has consistently larger values than the other at each variable. To formally do some tests, we can test for parallelism using 1 1 0 0 C = 0 1 1 0 0 0 1 0 Then the null is H 0 : C(µ 1 µ 2 ) = 0 STAT476/STAT576 February 20, 2015 44 / 46
Profile plot: two samples STAT476/STAT576 February 20, 2015 45 / 46
Profile plot: two samples The book uses the T 2 table to compare this to a critical value. You can do this or convert to an F. To convert to an F, we again use v p + 1 T 2 = F p,v vp Here we use p 1 in place of p because p 1 is the rank of C, and we use v = n 1 + n 2 2 This yields F = n 1 + n 2 2 (p 1) + 1 T 2 = (n 1 + n 2 2)(p 1) This gives a p-value of about 2 10 10 > 1-pf(23.94,3,62) [1] 2.06982e-10 so there is clear evidence against parallelism. 32 + 32 4 T 2 = 60 74.24 = 23.9 62 3 186 STAT476/STAT576 February 20, 2015 46 / 46
HW2: due in two weeks: March 2nd (continues to the next slide) STAT476/STAT576 February 20, 2015 47 / 46
HW2: due in two weeks: March 2nd STAT476/STAT576 February 20, 2015 48 / 46