Lecture 12: Multiple Hypothesis Testing

Transcription

1 ECE 830 Fall 00 Saisical Signal Processing insrucor: R. Nowak, scribe: Xinjue Yu Lecure : Muliple Hypohesis Tesing Inroducion In many applicaions we consider muliple hypohesis es a he same ime. Example FMRI (Funcional Magneic Resonance Imaging) Figure : Brain image by FMRI wih m voxels Example Microarrays Figure : Gene es plo Is here a difference in he mean expression level in he healhy and diseased cells? For he overall m genes ha are expressed, : X i N (0, ) H i : X i N (µ, ) where i,,...m, µ 0

2 Lecure : Muliple Hypohesis Tesing Suppose we have m ess and each is designed o guaranee P F A α. Then for any one es, he chance of a false alarm is α. However he prob of a leas one false alarm among all he ess is much higher. This is he muliple esing problem. Consider m hypohesis ess (a family of ess): v.s. H i, where i,,..., m Definiion The family-wise error rae (FWER) is he probabiliy of one or more false alarms. F W ER P( m {decide when H i is rue}) i In many applicaions we wan o conrol he FWER. For example, we wan o be confiden ha all deeced voxels or genes are ruly relevan. The Bonferroni Correcion Suppose we have wo ess: γ and γ are ses such ha P F A α in boh cases. H 0 v.s. H ; H 0 v.s. H H H γ ; γ H 0 H 0 P( > γ H 0 ) α ; P( > γ H 0 ) α Recall ha for any wo evens A and B, we have P(A B) P(A) + P(B) wih equaliy iff A B. This is called he union of evens bound, union bound or Bonferroni s inequaliy. Thus, F W ER P( > γ or > γ H 0, H 0 ) P({ > γ H 0 } { > γ H 0 }) α More generally, if we have m ess and each has an individual P F A α, hen F W ER mα. Definiion Borferroni Correcion To guaranee a F W ER α for a family of m ess, we can se he P F A α m Example 3 Threshold for muliple hypohesis esing : X i N (0, ) H i : X i N (µ, ) where i,,...m, µ H i For he es: x i γ, if we use γ Q ( α m ) insead of γ Q (α), hen F W ER α for each individual es. Example 4 Tesing for differenial expression in genes Suppose we have 638 genes or ess wih : X N (0, ). If we wan a F W ER 0.05, hen he hreshold γ F W Q ( ) 4.9 will suffice. For comparison, if we only consider one gene, hen he hreshold γ Q (0.05).6449 would guaranee P F A α. Thus, he Bonferroni correcion is a quie conservaive hreshold.

3 Lecure : Muliple Hypohesis Tesing 3 3 Muliple Tesing When m Is Large To ge a beer insigh ino his problem, le s consider he siuaion when m is very large and consider he composie ess: : X i N (0, ) H i : X i N (µ, ) where i,,...m, µ 0 The Bonferroni Correcion is conservaive and he union bound may be oo loose. Can we do beer? Consider he following bound on he ail of N (0, ) γ e x dx e γ H i Wih his bound, we can conclude ha he P F A of he es x i γ saisfies P F Ai e γ. Now he union bound implies F W ER To guaranee F W ER, we mus have γ m i For large m, his is he Bonferoni hreshold. P F Ai m e γ log m γ e( log m) Can we conrol he FWER wih a smaller hreshold? The answer, a leas for large m, is no. I can be shown ha if ω i N (0, ), i,,..., m, hen max i {ω i } a.s. lim m log m lim F W ER(γ m, m) if γ m log m m So, for large m, we mus ake γ log m if we wan o keep F W ER 4 False Discover Rae (FDR) Conrol FWER conrol: P(one or more false alarms) α. Perhaps he FWER consrain is oo conservaive. Alernaive, α (number of declared deecions) The declared deecions includes boh he false alarms and he correc deecions. This consrain is less conservaive han FWER. In oher words, we could insead aim o guaranee ha among all he ess for which we decide H, only a small fracion α are false alarms. Definiion 3 The false-discovery proporion is F DP number of discoveries + number of correc deecions

4 Lecure : Muliple Hypohesis Tesing 4 Definiion 4 The false-discovery rae is F DR E[ number of discoveries ] A discovery is made anyime we decide H, wheher i is he correc decision or no. We can now aim o conrol he FDR as: F ER α Generally, γ F DR γ F W ER since i allows a small fracion of false-alarms. So how o choose γ o guaranee F DR α? 5 Benjamini-Hochberg (BH) Threshold The BH hreshold is an adapively seleced hreshold ha conrols he F DR α. I is generally lower han he bonferroni hreshold for F W ER α. The BH hreshold is compleed as follows. Assume : X i N (0, ), i,,..., m. Sep Compue he values of he ail probabiliies p p i : P(N (0, ) x i ),i,,..., m Sep Sor p-values such ha p () p ()... p (m) where p () represens he mos exreme ail probabiliy wih larges x i, p (m) represens he leas exreme ail probabiliy wih smalles x i. Sep 3 Se hreshold i : max {i : p i < iα m } γ BH p i H i Then he es x i γ BH have F DR α 6 Gaussian Tail Bound The ail of he sandard Gaussian N (0, ) disribuion saisfies he bound for any 0, Proof e x dx min( e, e )

5 Lecure : Muliple Hypohesis Tesing 5 Consider For he firs bound, le y x, R e x dx e () e (x ) dx () e (x )(x+) dx (3) R e y(y+) dy 0 0 e y dy (4) For he second bound, noe ha e (x )(x+) dx e (x ) dx (5) e e x dx (6) e e x (7) (8)