z = β βσβ Statistical Analysis of MV Data Example : µ=0 (Σ known) consider Y = β X~ N 1 (β µ, β Σβ) test statistic for H 0β is

Transcription

1 Example X~N p (µ,σ); H 0 : µ=0 (Σ known) consider Y = β X~ N 1 (β µ, β Σβ) H 0β : β µ = 0 test statistic for H 0β is y z = β βσβ /n And reject H 0β if z β > c [suitable critical value] 301

2 Reject H 0 if any H 0β is rejected H 0 not rejected if no H 0β is rejected i.e. if no z β > critical value i.e. if max{z β } < c Now 2 z 2 = ny = nβ xx β β βσβ βσβ and this is maximized [using The Procedure ] when β = [only] eigenvector of Σ 1 xx 302

3 The [only] eigenvector of is Σ 1 x This direction exhibits the greatest deviation from µ=0 Σ 1 xx Examining coefficients of variables in Σ 1 x will shew which variables in which combinations cause rejection of H 0 303

4 Substituting x in expression for gives β=σ 1 z β = nx Σ x x Σ x = Σ 1 β 1 opt x Σ x z nx x and under H 0 this has an [exact] distribution χ 2 n (not just an asymptotic as given by general MLE theory) NB 1 x Σ x is 1 p p p p 1 = 1 1 i.e. a scalar, so cancels from top and bottom 304

5 Other Examples 2-sample test (c.f. task sheet 10) Test on data projected into 1-dimension gives a standard two-sample t-test. Maximizing wrt projection gives two-sample T 2 -test Direction of greatest difference is 1 S (x x ) 1 2 Example: Iris Versicolor vs Virginica 305

6 Example: Iris Versicolor vs Virginica (cf task sheet 9) We have x x = S * = * * * * *

7 and then S (x x ) 5610 = c.f. eigenvector from MANOVA differs only by a factor ~

8 Interpretation: Variable coefficient in Sepal-l Sepal-w Petal-l Petal-w S (x1 x 2) Greatest difference between varieties is exhibited in a direction contrasting size of sepals with size of petals & particularly widths of sepals and petals 308

9 Other Examples Test of H 0 : Σ = Σ 0 vs Σ Σ 0 with µ unknown c.f UIT gives directions of deviations from H 0 as the smallest and largest eigenvectors of Σ 0-1 S (i.e. corres. to λ 1 & λ p ) with test statistics λ 1 & λ p rejecting H 0 if λ 1 is improbably big or if λ p is improbably small Assess by simulating from N p (0, Σ 0 ) and calculating λ 1 & λ p many times 309

10 Example: Iris Versicolor vs Virginica To test whether Σ = Σ 0 =0.19 I 4 (artificial example: 0.19 chosen as average trace of S, tests whether lengths and widths of sepals & petals are independent with equal variances) As before S * = * * * * *

11 Eigenvalues of Σ 0-1 S are (0.591, 0.087, 0.056, ) =(3.11, 0.46, 0.29, 0.13) So LRT statistic is ~ (see 5.5.2) First eigenvector is (-0.72, -0.25, -0.62, -0.16) and last is (-0.21, 0.44, 0.29, -0.83) 311

12 To assess whether the UIT rejects H 0 : Σ = 0.19 I 4 need to simulate 100 samples from N 4 (0, 0.19 I 4 ), calculate the sample covariance matrix S find the largest and smallest eigenvalues of S repeat this 1000 times (say) see how many of these are > 3.11 or < 0.13 (the observed values) Then, reject H 0 at the 5% level if fewer than 5% of the simulated values are outside these bounds 312

13 Interpretation (v.diff) First eigenvector is (-0.72, -0.25, -0.62, -0.16) and last is (-0.21, 0.44, 0.29, -0.83) Direction of deviation is weighted towards 1 st & 3 rd comps (i.e.lengths) *... S = * * * * *. 057 (which have largest covariance) & 1 st & 4 th [with max & min variances] different from 2 nd & 3 rd 313

14 Multisample Tests MANOVA Setup: k known groups n i observations from each group, Σn i = n Each observation is p-dimensional S is the total variance W is the within groups variance B is the between groups variance Then it can be shewn that (n 1)S = (K 1)B + (n k)w the one-way multivariate analysis of variance 314

15 Model is that data from i th group ~N p (µ i, Σ) (common Σ note) Standard hypothesis is H 0 : = µ 1 = µ 2 =. = µ k vs at least one µ i different Various test statistics all based on eigenvalues of W and B 315

16 Likelihood Ratio Test Wilks Λ-test Test statistic is W 1 B = product of all eigenvalues of W 1 B UIT Test Roy s test Test statistic is largest eigenvalue of W 1 B Direction exhibiting greatest deviation from H 0 is corresponding eigenvector 316

17 Others Pillai Trace Trace of B(B+W) 1 = sum of eigenvlaues of B(B+W) 1 Lawley-Hotelling Trace Trace of W 1 B = sum of eigenvlaues of W 1 B In case k = 2 all tests are the same For k > 2 they are different For k = 2 the Lawley-Hotelling trace = (n 2) Hotelling s T 2 317

18 Key Advantage of MANOVA: Partially overcomes problem of multiple testing of all variables separately False positives, non-independent tests &c. Use of UIT principle allows interpretation of which [combination] of variables exhibits deviation Generalizations to 2-way MANOVA & general multivariate linear model 318

19 Summary & Conclusions Univariate results extended to multivariate data Introduction of Multivariate Distributions: Normal Wishart Hotelling s T 2 sample mean and variance unbiased for population mean and variance Normal sample mean Normal variance is Wishart and they are independent 319

20 One and two-sample t-tests T 2 -tests Generalized likelihood ratio tests (LRTs) for constructing hypotheses which cannot arise in one dimension Union-Intersection Tests (UITs) provide an alternative strategy similarities with multivariate EDA techniques such as PCA and LDA in construction and interpretation of directions standard packages have all standard tests 320