ASYMPTOTIC PROPERTIES OF PARTIAL AREAS UNDER THE RECEIVER OPERATING CHARACTERISTIC CURVE WITH APPLICATIONS IN MICROARRAY EXPERIMENTS

Transcription

1 Uiversiy of Keucky UKowledge Uiversiy of Keucky Docoral Disseraios Graduae School 6 ASYMPTOTIC PROPERTIES OF PARTIAL AREAS UNDER THE RECEIVER OPERATING CHARACTERISTIC CURVE WITH APPLICATIONS IN MICROARRAY EXPERIMENTS Hua Liu Uiversiy of Keucky, hualiu@s.uky.edu Click here o le us kow how access o his docue beefis you. Recoeded Ciaio Liu, Hua, "ASYMPTOTIC PROPERTIES OF PARTIAL AREAS UNDER THE RECEIVER OPERATING CHARACTERISTIC CURVE WITH APPLICATIONS IN MICROARRAY EXPERIMENTS" (6. Uiversiy of Keucky Docoral Disseraios hs://ukowledge.uky.edu/gradschool_diss/463 This Disseraio is brough o you for free ad oe access by he Graduae School a UKowledge. I has bee acceed for iclusio i Uiversiy of Keucky Docoral Disseraios by a auhorized adiisraor of UKowledge. For ore iforaio, lease coac UKowledge@lsv.uky.edu.

2 ABSTRACT OF DISSERTATION Hua Liu The Graduae School Uiversiy of Keucky 6

3 ASYMPTOTIC PROPERTIES OF PARTIAL AREAS UNDER THE RECEIVER OPERATING CHARACTERISTIC CURVE WITH APPLICATIONS IN MICROARRAY EXPERIMENTS ABSTRACT OF DISSERTATION A disseraio subied i arial fulfille of he requirees for he degree of Docor of Philosohy i he College of Agriculure a he Uiversiy of Keucky By Hua Liu Lexigo, Keucky Co-Direcors: Dr. Arold J. Sroberg, Professor of Saisics ad Dr. Cosace L. Wood, Associae Professor of Saisics Lexigo, Keucky 6 Coyrigh Hua Liu 6

4 ABSTRACT OF DISSERTATION ASYMPTOTIC PROPERTIES OF PARTIAL AREAS UNDER THE RECEIVER OPERATING CHARACTERISTIC CURVE WITH APPLICATIONS IN MICROARRAY EXPERIMENTS Receiver oeraig characerisic (ROC curves are widely used i edical decisio akig. I was recogized i he las decade ha oly a secific regio of he ROC curve is of cliical ieres, which ca be suarized by he arial area uder he ROC curve (arial AUC. Early saisical ehods for evaluaig arial AUC assue ha he daa are fro a secified uderlyig disribuio. Noaraeric esiaors of he arial AUC eerged recely, bu here are heoreical issues o be addressed. I his disseraio, we roose wo ew oaraeric saisics, arially iegraed ROC ad arially iegraed weighed ROC, for esiaig arial AUC. We show ha our arially iegraed ROC saisic is a cosise esiaor of he arial AUC, ad derive is asyoic disribuio which is disribuio free uder he ull hyohesis. I he arially iegraed ROC saisic, whe he ROC curve crosses he Uifor disribuio fucio (CDF ad if he arial area evaluaed coais he crossig oi, or whe here are ulile crossig, he arially iegraed ROC saisic igh o erfor well. To address his issue, we roose he arially iegraed weighed ROC saisic. This saisic evaluaes he arially weighed AUC, where larger weigh is give whe he ROC curve is above he Uifor CDF ad saller weigh is give whe he ROC curve is below he Uifor CDF. We show ha our arially iegraed weighed ROC saisic is a cosise esiaor of he arially weighed AUC. We derive is asyoic disribuio which is disribuio free uder he ull hyohesis.

5 We roose o aly our wo oaraeric saisics o fucioal caegory aalysis i icroarray exeries. We defie he fucioal caegory aalysis o be he saisical ideificaio of over-rereseed fucioal gee caegories i a icroarray exerie based o differeial gee exressio. We coare our saisics wih exisig ehods for he fucioal caegory aalysis boh via siulaio sudy ad alicaio o a real icroarray daa, ad deosrae ha our wo saisics are effecive for ideifyig over-rereseed fucioal gee caegories. We also ehasize he esseial role of he eirical disribuio fucio los ad he ROC curves i he fucioal caegory aalysis. KEYWORDS: Parial Area uder he ROC Curve, Parially Iegraed ROC, Parially Iegraed Weighed ROC, Fucioal Caegory Aalysis, Microarray Hua Liu Jue 5, 6

6 ASYMPTOTIC PROPERTIES OF PARTIAL AREAS UNDER THE RECEIVER OPERATING CHARACTERISTIC CURVE WITH APPLICATIONS IN MICROARRAY EXPERIMENTS By Hua Liu Arold J. Sroberg Co-Direcor of Disseraio Cosace L. Wood Co-Direcor of Disseraio Arold J. Sroberg Direcor of Graduae Sudies Jue 5, 6

7 RULES FOR THE USE OF DISSERTATION Uublished disseraios subied for he Docor s degree ad deosied i he Uiversiy of Keucky Library are as a rule oe for isecio, bu are o be used oly wih due regard o he righs of he auhors. Bibliograhical refereces ay be oed, bu quoaios or suaries of ars ay be ublished oly wih he erissio of he auhor, ad wih he usual scholarly ackowledgees. Exesive coyig or ublicaio of he disseraio i whole or i ar also requires he cose of he Dea of he Graduae School of he Uiversiy of Keucky. A library ha borrows his disseraio for use by is aros is execed o secure he sigaure of each user. Nae Dae

8 DISSERTATION Hua Liu The Graduae School Uiversiy of Keucky 6

9 ASYMPTOTIC PROPERTIES OF PARTIAL AREAS UNDER THE RECEIVER OPERATING CHARACTERISTIC CURVE WITH APPLICATIONS IN MICROARRAY EXPERIMENTS DISSERTATION A disseraio subied i arial fulfille of he requirees for he degree of Docor of Philosohy i he College of Agriculure a he Uiversiy of Keucky By Hua Liu Lexigo, Keucky Co-Direcors: Dr. Arold J. Sroberg, Professor of Saisics ad Dr. Cosace L. Wood, Associae Professor of Saisics Lexigo, Keucky 6 Coyrigh Hua Liu 6

10 ACKNOWLEDGEMENTS I would like o exress y sicere areciaio o y advisors, Drs. Arold J. Sroberg ad Cosace L. Wood, for heir excelle acadeic guidace, isighful advice, exceioal aiece, ad geerous suor. Wihou heir hel, his disseraio would o have bee wrie. I also would like o hak oher ebers of y disseraio coiee, Drs. Ker Viele, Thoas V. Gechell, ad Willia S. Rayes, ad y ouside exaier, Dr. Gary Va Za, for heir helful coes. I also hak Dr. Are Bahke for his hel. I would esecially hak Dr. Arold J. Sroberg for rovidig e wih he grea research assisashi ooruiy i UK bioiforaics grou. I would like o exress y graefuless o Chrisoher P. Sauders for his isighful discussio, hel, ad friedshi. I would also like o exress y areciaio o Drs. Kueychu Che ad Marily L. Gechell for heir suor. I would like o dedicae his disseraio o y husbad, Cog Zhu, ad y ares, for heir coiuous ecouragees ad suors hroughou y sudy. I would also like o hak y fried, Yaxi She ad her kids, Jessica, Mia, ad Eha, for he grea suor ad joy ha hey rovided e hroughou he as year. iii

11 TABLE OF CONTENTS ACKNOWLEDGEMENTS...iii LIST OF TABLES...vi LIST OF FIGURES...vii Chaer Iroducio.... Previous work o arial area uder he ROC curve.... Fucioal caegory aalysis i icroarray exeries Saisical ehods alied o he fucioal caegory aalysis Mai resuls Suary of he disseraio...6 Chaer Preliiary resuls o asyoic heories...8. Eirical rocesses...8. Geeralized U-saisics Relaed aroxiaio heores...5 Chaer 3 Parially iegraed ROC Asyoic disribuio of he arially iegraed ROC wih fixed liis Cosisecy of he arially iegraed ROC wih esiaed liis Asyoic disribuio of he arially iegraed ROC wih esiaed liis Alicaio i fucioal caegory aalysis...48 Chaer 4 Parially iegraed weighed ROC Cosisecy of he arially iegraed weighed ROC Cosisecy of he arially iegraed weighed ROC wih chagig weigh fucios Asyoic disribuio of he arially iegraed weighed ROC Alicaio i fucioal caegory aalysis...79 Chaer 5. Siulaio sudies Coariso of exac ad asyoic variaces Noial alha levels Power aalysis wih fixed aleraives Power aalysis wih coiguous aleraives Power aalysis wih chagig...9 Chaer 6. Alicaio o a icroarray exerie Microarray daa ad saisical aalysis Fucioal caegory aalysis...97 Chaer 7. Suary ad discussio Suary Fuure direcios...8 iv

12 Refereces... Via...3 v

13 LIST OF TABLES Table. False osiive ad false egaive....3 Table. A exale of a sall roorio of a gee lis...7 Table.3 A exale of a by coigecy able i Fisher s exac es... Table 5. Exac ad asyoic variaces of he arially iegraed ROC saisic ( ˆP whe ( ( F = G =Uifor(,...8 Table 5. Exac ad asyoic variaces of he arially iegraed ROC saisic ( ˆP whe F ( = Bea(, ad G ( = Uifor(,...8 Table 5.3 Exac ad asyoic variaces of he arially iegraed weighed ROC saisic ( ˆP whe ( ( F = G = Uifor(,....8 Table 5.4 Exac ad asyoic variaces of he arially iegraed weighed ROC saisic ( ˆP uder he aleraive hyoheses....8 Table 5.5 Noial alha levels of he saisical ehods whe daa are geeraed fro Uifor (, Table 5.6 Noial alha levels of he saisical ehods whe daa are geeraed fro Bea (.8,...83 Table 5.7 Noial alha levels of he saisical ehods whe daa are geeraed fro Noral (,...83 Table 5.8 Power aalysis wih coiguous aleraives, where he curves cross ad he ROC curve is above he Uifor CDF a he lower ails....9 Table 5.9 Power aalysis wih coiguous aleraives, where he curves cross ad he ROC curve is above he Uifor CDF a he lower ails...9 Table 5. Coariso of exac ad asyoic ower wih chagig...9 Table 6. The by able used by Fisher s exac es is cosruced based o he - values fro coras 4 for he fucioal caegory iracellular rasor...99 Table 6. The EASE aalyses ad he Fisher s exac ess a various α levels Table 6.3 P-values obaied by he fucioal caegory aalysis for he iracellular rasor caegory usig various ehods... Table 6.4 P-values obaied by he fucioal caegory aalysis for he fucioal caegory hoshorylaio usig differe ehods...5 vi

14 LIST OF FIGURES Figure. A exale of a ROC curve....3 Figure. Exales of icroarray gee chis....6 Figure.3 Grahical rereseaios of he KS es ad he WMW es used for he fucioal caegory aalysis i icroarray exeries.... Figure.4 Grahical rereseaios of he cases whe he wo-sale KS es ad he WMW es ca give isleadig resuls...4 Figure 5. The ROC curve for case I where he ROC curve is above he Uifor CDF84 Figure 5. Power curves for case I ha he ROC curve is above he Uifor CDF...85 Figure 5.3 Power curves for case I ha he ROC curve is above he Uifor CDF...86 Figure 5.4 The ROC curve for case II whe he ROC curve crosses he Uifor CDF ad is above he Uifor CDF a he lower ails...87 Figure 5.5 Power curves for case II ha he ROC curve crosses he Uifor CDF ad is above he Uifor CDF a he lower ails Figure 5.6 Power curves for case II ha he ROC curve crosses he Uifor CDF ad is above he Uifor CDF a he lower ails Figure 5.7 The ROC curve for he ower aalysis wih coiguous aleraive...9 Figure 5.8 Asyoic ower curve of our arially iegraed ROC saisic wih chagig...94 Figure 5.9 The curve of rejecio oi for asyoic ower of our arially iegraed ROC saisic wih chagig...95 Figure 6. Coefficies i he orhogoal corass for daa aalysis...97 Figure 6. Grahical illusraio for esig over-rereseaio of he iracellular rasor caegory based o coras 4... Figure 6.3 Grahical illusraio for esig over-rereseaio of he fucioal caegory iracellular rasor based o coras....4 Figure 6.4 Grahical illusraio for esig over-rereseaio of he fucioal caegory hoshorylaio...5 vii

15 Chaer Iroducio Receiver oeraig characerisic (ROC curves are widely used i edical decisio akig. I was recogized i he las decade ha oly a secific regio of he ROC curve is of cliical ieres, which ca be suarized by he arial area uder he ROC curve (arial AUC. Early saisical ehods for evaluaig arial AUC assue ha he daa are fro a secified uderlyig disribuio. Noaraeric esiaors of he arial AUC eerged recely, bu here are heoreical issues o be addressed. I his disseraio, we roose wo oaraeric esiaors of he arial AUC, ivesigae heir asyoic roeries, ad cosider heir alicaios i icroarray exeries. Oe ajor issue i icroarray exeries is he biological ierreaio of differeial gee exressio. Wih yically hudreds o housads of gees differeially exressed siulaeously, i is difficul o exlore he biological heoea derived fro he icroarray exerie. Biological ierreaio ca be doe by grouig gees based o heir fucios or ahways, he sudyig he saisical sigificace of he differeial exressio of he fucioal grou. I his disseraio, we defie his ye of aroach as fucioal caegory aalysis. Exisig saisical ehods alied o he fucioal caegory aalysis are o desiged for his roble. Here we roose he alicaio of our arial AUC esiaors o address issues i he fucioal caegory aalysis i icroarray exeries.. Previous work o arial area uder he ROC curve The ROC curve was develoed i 95 s as a byroduc of he research of radio sigal deecio i he resece of oise ad has gaied oulariy i edical research for screeig ad diagosic ess (Haley, 989. I is ofe used o evaluae he abiliy of a laboraory es o discriiae bewee he diseased aies ad he healhy aies. A ROC curve is a grahical rereseaio of he rade off bewee false osiive rae ad false egaive rae for every ossible cu off ha is used for akig a decisio (Table.. I has he rue osiive rae ( false egaive rae o he verical axis; ad he false osiive rae o he horizoal axis. I edical research, rue osiive rae is ofe called

16 sesiiviy, which is he abiliy o ick ou aies wih disease, ad false osiive rae is ofe called secificiy, which is he abiliy o ick ou aies wihou disease. For a exale of a ROC curve, see Figure.. A diagosic es ha has a good discriiaig abiliy bewee diseased aies ad o-diseased aies will have a ROC curve close o coaiig he oi (, (Figure.. A diagosic es ha has o discriiaig abiliy will have a ROC curve close o he Uifor CDF (for exale, see he blue lie i Figure.. To cosruc a ROC curve for he seig discussed here, le F ( be he disribuio fucio of he aboral oulaio, ad G ( be he disribuio fucio of he oral oulaio, R. Le The he ROC curve is { } G ( = if : G(, < <. ( ROC( = F G (, < <. Le X,, X be ideede ad ideically disribued (i.i.d. rado variables wih disribuio fucio F ( ad eirical cuulaive disribuio fucio (ECDF F (, where R, ad F ( = Ι[ x ]. i = Le Y, Y be i.i.d. rado variables wih disribuio fucio G ( ad ECDF G (,, where R, ad G ( = Ι[ y ]. i = Le X,, X ad Y,, Y be uually ideede. To cosruc a ROC curve based o a sale, oice ha F ( corresods o he rue osiive rae ad G ( corresods o he false osiive rae i he sale. Therefore, a ROC curve ca be loed by F ( agais G (, which is equivale o lo i j F G ( ( agais. The ROC curve i Figure. is loed by ( ( F G agais, where F ( is geeraed fro Bea(3,, ad G ( is geeraed fro Uifor (,.

17 Table. False osiive ad false egaive. Decisio Posiive Negaive Posiive True Posiive False Negaive Truh Negaive False Posiive True Negaive Figure. A exale of a ROC curve. The red curve is he ROC curve. The blue lie is he Uifor (, CDF. A recoeded way o suarize he ROC curve is he area uder he ROC curve (AUC (Swes ad Picke, 98. The AUC rereses he robabiliy ha he diagosic esig value of a radoly chose idividual fro he aboral oulaio is less ha (or greaer ha he diagosic esig value of a radoly chose idividual fro he oral oulaio. Usig he above oaio, he corresodig AUC is ( F G ( d. Whe G ( is coiuous, he above AUC is equivale o FdG ( (. There are boh araeric ad oaraeric ehods of esiaig he AUC. Aog araeric ehods, bioral odel assuio ad axiu likelihood esiaio of 3

18 he AUC had bee widely eloyed (Dorfa ad Elf, 969; Mez 978; Swes ad Picke, 98. The bioral odel assuio assues ha boh diseased ad odiseased oulaios follow oral disribuios. The corresodig ROC ad AUC are give as: ( ROC( =Φ a + bφ (, ad a AUC =Φ. + b I he above AUC esiaor, a = ( µ µ / ad b / (, ND ND D ND D =, where (, ND D µ ad µ deoe he eas ad sadard deviaios of he es resul of diseased ad o-diseased oulaio, resecively. a ad b ca be esiaed usig axiu likelihood aroach. Noaraeric ehods, which do o have ay disribuioal assuios, are a ideal aleraive for esiaig he AUC. The oaraeric esiaor of he AUC ca be defied as or F G d ( (, F ( dg (. This AUC esiaor is equivale o he Ma-Whiey U-saisic, which is a ye of geeralized U-saisic (Hoeffdig, 948. DeLog e al. (988 exloied he heory of geeralized U-saisic for coarig he areas uder correlaed ROC curves. I he alicaio of ROC curves, i is well recogized ha os ars of he ROC curve are o releva i ay cases. For exale, oly he lower ail of he ROC curve is of ieres for cacer screeig because he false osiive rae us be very sall o be acceable (Lilliefeld, 974. Parial area uder he ROC curve was roosed o address such issues (McClish, 989; Thoso ad Zucchii, 989. Boh of hese wo aers roosed calculaig arial AUC based o bioral odel assuio ha is aki o he above-eioed bioral odel for esiaig he full AUC. The corresodig arial AUC esiaor is defied as D D 4

19 ( Φ a+ bφ ( d, which assues he daa has uderlyig oral disribuio. Jiag e al. (996 described a arial area idex for highly sesiive diagosic ess by exedig McClish s work. Baker ad Pisky ( odified he raezoidal rule of Baber (975 o esiae he arial AUC, ad cosruced a saisic o he raio of he wo arial AUCs by assuig he daa follow a ulioial disribuio. The firs oaraeric ehod for esiaig arial AUC was roosed by Zhag e al. (. Based o he heory of DeLog e al (988, hey worked ou he asyoic disribuio of heir esiae of he arial AUC for discree x ad y, where hey used raig scales such ha heir esiae of he arial AUC is a su of k raezoidal areas whe here are k classes based o he raig scale. Dodd ad Pee (3 roosed a oaraeric esiaor of arial AUC for a regio of false osiive raes. I heir ehod, wih he cocer of low rue osiive rae whe he false osiive rae is oo sall, hey cosidered a regio (, of false osiive raes. Assuig G ( o be coiuous, he arial AUC ha hey cosidered is which is equivale o ( (, F G d G ( G ( Their arial AUC esiaor is defied as (. which ca also be wrie as i= j= G ( G ( FdG ( (. F ( dg (, Ι[ x y ] Ι[ G ( y G ( ], i j j whe G ( ad G ( are kow. Whe hese wo quailes are o kow, hey suggesed o use eirical quaile esiaes G ( ad G (. However, hey did o derive he asyoic disribuio for heir esiaor, bu raher derived he asyoic disribuio for he regressio araeers i arial AUC regressio (Dodd,, which is o alicable for evaluaig arial AUC. I his disseraio, we 5

20 roose a odified versio of he esiaor of Dodd ad Pee (3 for arial AUC, ad a esiaor of a weighed arial AUC. We derive he asyoic roeries of hese wo esiaors ad roose heir alicaio o he fucioal caegory aalysis i icroarray exeries. Figure. Exales of icroarray gee chis. The iage o he lef is a Affyerix oligoucleoide gee chi. The iage o he righ is a wo-color cdna gee chi.. Fucioal caegory aalysis i icroarray exeries Each cell i a livig orgais carries he geeic iforaio of he eire geoe, bu oly a cerai fracio of he gees are ured o (exressed i a cell a a give ie. A cell resods o is chagig eeds ad eviroe siuli by orchesraig gees ad is roducs hrough corollig wheher a gee is ured o ad he aou of he exressio. Gee exressio is he ass of geeic iforaio fro DNA o RNA he o roei; ad roeis erfor os of he criical fucios of cells. The level of RNA is a easure of gee exressio. This used o be a rocess of a few gees a a ie by echologies such as orher lo or RT-PCR. Microarray, a echology breakhrough, oiors he level of RNA for es of housads of gees siulaeously hrough a sigle gee chi (Schea e al., 995; Lockhar e al., 996. A icroarray gee chi is a icroscoic slide which coais es of housads of sos. Each so coais a shor oligoucleoide sequece, which ca be a sall frage of a 6

21 gee, or a sequece wih ukow fucio. A RNA sale, exraced fro exerieal subjecs, ca be labeled by fluoresce dye or bioi, ad hybridized o he gee chi. If a gee is exressed i he cell, he corresodig so o he gee chi will be icked u by hybridizig o he RNA sale. The level of he gee exressio ca be visualized hrough he iesiy of he dye a ha so. For exale, see Figure. for wo ajor yes of icroarray chis. For furher deails of hese wo yes of icroarray, see h://affyerix.co ad h://browlab.saford.edu. Table. A exale of a sall roorio of a gee lis. The lis is obaied by erforig wo-sale -es o each idividual gee i a icroarray exerie coarig kock-ou ice (KO wih wild ye ice (WT. robese Gee Sybol -value WT ea WT SE KO ea KO SE 4378_a Morf4l 4.644E _a_a Ed.683E _a Ebf.853E _x_a Gs 3.33E _s_a Tuba E _a_a Aig E _a Pra E _a Cd E _a Igfb E _a B E _a Aeb 6.88E _a_a Igfb E _a_a Aoe E _a Bre 8.5E _a_a Us 9.75E _s_a Fl.4E _s_a Hdgf.933E _a Pk9.37E While icroarrays excie biologiss by rovidig huge aou of iforaio, hey also ose ay saisical challeges. Oe ajor saisical challege lies i he biological ierreaio of differeial gee exressios i a icroarray exerie. A icroarray exerie usually coais various reaes wih several relicaios er reae. Afer oralizig ad sadardizig he raw daa o ake chis coarable wihi he exerie, hyohesis esig ca be alied o all exressio iesiies of 7

22 each idividual gee o deec he sigificace of differeial gee exressio due o effec of he reaes. Hyohesis esig will resul i a lis of gees wih corresodig -values for differeial exressio. Table. gives a sall orio of a gee lis obaied by erforig wo-sale -es o each gee i a icroarray exerie for coarig kock-ou ice wih wild ye ice. Tyically, here are hudreds or housads of gees reseig saisically sigifica differeial exressio. The ex esseial ask is o exlore he fudaeal biological echaiss derived fro icroarray exeries. To udersad he biological eaig of a gee lis, he fucio of each idividual gee i he lis eeds o be sudied. This used o be doe via exesive search of relaed lieraure ad daabases, which is very ie-cosuig ad difficul, esecially whe he syses of oeclaure for gees ad heir roducs were diverge ad he uderlyig siilariies are o obvious. I is he obsacle of he lack of ieroerabiliy of geoic daabases ha he Gee Oology (GO Cosoriu (Ashburer e al., was fored o address. GO is a dyaic ad corolled vocabulary for describig he roles of gees ad heir roducs i ay orgais. The use of GO ers by several collaboraig daabases faciliaes Uifor queries across he. GO works a hree srucured, corolled oologies for each gee: biological rocess, cellular cooe, ad olecular fucio, ad orgaizes gees io hierarchical caegories based o hese hree oologies. GO Cosoriu allows auoaed searchig of he fucio of a gee, hus has becoe a ajor ool for searchig gee fucios. Oe icoveiece of he GO Cosoriu is ha he iforaio of gees ca be queried for oly oe gee a a ie. I works well, for exale, whe a biologis was o search he fucios of several gees ha were obaied by a RT-PCR exerie. I icroarray exeries, however, he gee lis usually coais hudreds or eve housads of gees. Thus, he rocess of usig GO Cosoriu ca sill be iecosuig. Eve if all he gees have bee queried i GO, he GO resuls sill eed o be suarized o obai isigh io he biological eaig of he gee lis. Suarizig he large uber of GO resuls for all gees i he lis ca be overwhelig. May sofware ackages have bee ublished o bach rocess such searches ha are based o GO (Hosack e al., 3; Zeeberg e al., 3; ec.. These sofware ackages caegorize 8

23 gees io fucioal caegories based o GO ers, for exale, i a gee lis, all gees ha fucio i a biological rocess cell deah will be u io a fucioal caegory called cell deah. The quesio o ask is ha wheher he fucioal caegory, for exale, cell deah, is differeially exressed, raher ha he differeial exressio of each idividual gee i he fucioal caegory. The aalysis of fucioal caegories allows he ui of he aalysis o be shifed fro idividual gees o grouigs of gees. Oe goal of fucioal caegory aalysis is o es wheher a grou of fucioally relaed gees is over-rereseed or eriched i he icroarray exerie. A overrereseed fucioal caegory is likely o be differeial exressed due o he effec of he reae i he icroarray exerie. Ofe, biologiss have soe secific grous of gees ha hey have ideified before hey coduc he icroarray exerie, for exale, gees i a cerai regio o he chroosoe or gees ivolved i a secific ahway. The exressio of such resecified grous of gees ca also be sudied via he fucioal caegory aalysis..3 Saisical ehods alied o he fucioal caegory aalysis The fucioal caegory aalysis ca be erfored based o cerai characerisics of differeial gee exressio, for exale, -values for differeial gee exressio, or corresodig es saisics, or cerai correlaios. Throughou his disseraio, -values for differeial gee exressio are used for he fucioal caegory aalysis. Le he - values of he gees i a fucio caegory be i.i.d. wih disribuio fucio F ( ad ECDF F (, for i [,]; he reaiig -values i he icroarray exerie be i.i.d. wih disribuio fucio G ( ad ECDF G (, for i [,]. Le F ( ad G ( be coiuous. The hyohesis ha he fucioal caegory aalysis ess is: H : F ( = G ( for all, [,] H : F ( > G ( for a leas oe, [,] Aog various sofware ackages ha coduc fucioal caegory aalysis, he ioeers are sofware usig saisical ehods based o by coigecy ables, for exale, hyergeoeric disribuios, chi-square disribuios (Hosack e al., 3; Zeeberg e al., 3; Doiger e al., 3; Beissbarh ad Seed, 4; ec.. Aog 9

24 he, Fisher s exac es, which is based o he hyergeoeric disribuio, is ofe used. Exressio Aalysis Syseaic Exlorer (EASE is oe such sofware ackage. Afer uloadig a give lis of ieresed gees ad selecig he ye of he gee chi i EASE, for exale, ouse gee chi U74Av, EASE uses GO Cosoriu o caegorize gees io differe fucioal grous. The i uilizes Jackkife ehod ad he Fisher s exac es o evaluae he over-rereseaio of he fucioal caegories. I is erfored by ealizig (reovig oe gee fro he fucioal caegory i he gee lis, ad erforig he Fisher s exac es o coare he fucioal caegory wih all reaiig gees i he secific ye of he gee chi. The reaso for usig Jackkife i EASE is o ealize he caegory wih few gees, sice he auhors clai ha a caegory wih oe gee is eiher global or sable ad is rarely ieresig (Hosack e al., 3. Here, a give gee lis ha is uloaded io EASE is a grou of gees ha is seleced based o a cerai cu-off. The cu-off usually is he crierio for deeriig differeial gee exressio, ha is, a -value cu-off or a es saisic cu-off. For exale, all gees ha have -values of differeial exressio less ha.5 ca be groued io a gee lis. I his disseraio, we use -values for differeial gee exressio i he fucioal caegory aalysis. A fucioal gee caegory is called over-rereseed if i has a sigificaly larger roorio of differeially exressed gees ha he reaiig ar of he exerie. The sigificace of over-rereseaio is rereseed by a EASE score, which is a uer boud of he Jackkife Fisher s exac robabiliies based o by coigecy ables. Defie α o be he level of sigificace for deeriig he sigificace of differeial exressio for each idividual gee. Table.3 illusraes he by able used by he Fisher s exac es, whe α =.5 ad he fucioal caegory is cell deah. I his case, he by able is cosruced by couig he uber a, which is he uber of sigifica gees (-value.5 i cell deah; he uber d, which is he uber of o-sigifica gees (-value >.5 i he reaiig ar of he icroarray exerie (called oher i he by able, ec. The Fisher s exac es coares he raios a/(a+c wih b/(b+d, ha is, coares he roorio of sigifica gees (a.5 level i he fucioal caegory cell deah wih he roorio of sigifica gees (a.5 level i he reaiig ar of he exerie. I is obvious ha he Fisher s exac es deeds o differe cuoffs. I Chaer 6, his deedecy o he arbirary chose

25 cu-off is illusraed i deail by a alicaio o a real icroarray daa. Furherore, by selecig a secific cu-off, i is oly releva wheher he -value or he es saisic of a gee is below or above he cu-off, bu o how uch he level of exressio of he gee is chaged. Thus oly arial iforaio fro he gee lis is rerieved for he fucioal caegory aalysis usig saisical ehods based o by coigecy ables. Table.3 A exale of a by coigecy able i Fisher s exac es. cell deah oher sigifica a α =.5 a b a+b o-sigifica a α =.5 c a+c d b+d c+d Sofware usig saisical ehods ha are based o coiuous disribuios eerged recely (Lab e al., 3; Mooha e al., 4; Bresli e al., 4. Corresodig saisical ehods iclude wo-sale Kologorov-Sirov (KS es ad Wilcoxo Ma-Whiey (WMW es. These ehods are ore effecive ha he cu-off based ehods because he iforaio fro all gees i he exerie is uilized i he fucioal caegory aalysis. The wo-sale KS es deeries he overrereseaio of a fucioal gee caegory by coarig he ECDF of he -values i a fucioal caegory wih he ECDF of he -values i he reaiig ar of he exerie. The oe-sided wo-sale KS es evaluaes [ F G ] su ( (, which is he axiu differece bewee he wo ECDFs whe F ( G (. Figure.3a gives a grahical rereseaio of he oe-sided wo-sale KS es saisic, which is he axiu verical differece bewee he agea ad black curves whe he agea ECDF curve is above he black ECDF curve. Whe here is o differeial exressio for ay gee i a icroarray exerie, he -values for differeial gee exressio follow a Uifor disribuio, ad he agea ad black ECDF curves should be close o he Uifor CDF (blue lie. The WMW es deeries overrereseaio by coarig he raks of he gees i a fucioal caegory wih he

26 raks of he reaiig gees i he exerie, where he rak is he osiio of a gee i a sored lis ha coais all gees i he exerie ad he sorig ca be based o a cerai crierio (-values, es saisics, correlaios, ec. I his disseraio, he rak is based o -values for differeial gee exressio. The es saisic of he oe-sided WMW is F ( dg ( = Ι[ x y ], i j i = j = which is he oaraeric esiaor of he AUC (see Figure.3b. Whe here is o differeial exressio for ay gee i a icroarray exerie, he ROC curve should be close o he Uifor CDF (blue lie i Figure.3b, ad he esiaed AUC should be close o.5. Figure.3 Grahical rereseaios of he KS es ad he WMW es used for he fucioal caegory aalysis i icroarray exeries. (a. The ECDF lo. The blue lie is he Uifor (, CDF, he agea curve is he ECDF of he -values i a fucioal caegory, he black curve is he ECDF of he reaiig -values i he exerie; (b. The ROC curve lo. The blue lie is he Uifor (, CDF. The red curve is he ROC curve.

27 Boh he KS es ad he WMW es are ore effecive ha he cu-off based ehods such as he Fisher s exac es for ideifyig over-rereseed fucioal caegories sice iforaio fro all gees i he exerie are used. However, boh ehods ca give isleadig resuls i cerai cases. For exale, he KS es ay show a fucioal caegory is over-rereseed whe a fucioal caegory has a siilar or lower roorio of sall -values bu a higher roorio of large -values ha he reaiig ar of he exerie. Seakig grahically, his is he siuaio whe he lower ail of he ECDF curve of a fucioal caegory is close o or below he lower ail of he ECDF curve of he reaiig ar of he exerie, bu is uer ail is above he uer ail of he ECDF curve of he reaiig ar of he exerie (Figure.4a. Sice he oe-sided wo-sale KS es is deecig ay differece bewee he wo disribuios, i his case, KS es will show sigificace due o he fac ha he agea ECDF curve is above he black ECDF curve a he uer ails of he ECDF curves. However, he fucioal caegory aalysis based o he KS es or he WMW es o he above-eioed case could be isleadig because he over-rereseaio is defied as a higher roorio of sall -values i a fucioal gee caegory ha i he reaiig ar of he icroarray exerie, whereas he relaioshi bewee he roorios of large -values is irreleva. Furherore, for he WMW es, i is well kow ha i ca be isleadig whe he ROC curve crosses he Uifor CDF. For exale, whe he curves cross ad he ROC curve is above he Uifor CDF a he lower ail, if he area uder he ROC curve is close o.5, he WMW es will show o-sigificace (Figure.4b. However, he fucioal caegory does have a higher roorio of sall -values coarig wih he reaiig ar of he icroarray exerie. Therefore, focusig o sall -values, or he lower ail of he ECDF or ROC curve, is esseial for akig reasoable deecio of over-rereseed fucioal caegories. To address his issue, oe ca u a large weigh o he lower ail of he disribuio ad a sall weigh o he uer ail. Sofware ackage GSEA ha becae available recely (Subraaia e al., 5 uilized he idea of weighig. They roosed a weighed KS-like saisic ad used i i GSEA. The roble wih his ehod is ha heir es saisic, which is he axiu differece bewee weighed disribuio fucios, does o coverge o zero uder he ull hyohesis, ha is, whe he wo disribuio fucios are he sae. 3

28 Therefore i is o a reasoable sei-eric for coarig wo disribuio fucios. I his disseraio, based o he idea of weigh fucios, we roose wo oaraeric esiaors of arial AUC. We aly hese wo esiaors o he fucioal caegory aalysis i icroarray exeries. Figure.4 Grahical rereseaios of he cases whe he wo-sale KS es ad he WMW es ca give isleadig resuls. (a. The ECDF lo. The blue lie is he Uifor (, CDF, he agea curve is he ECDF of he -values i a fucioal caegory, he black curve is he ECDF of he reaiig -values i he exerie; (b. The ROC curve lo. The blue lie is he Uifor (, CDF. The red curve is he ROC curve..4 Mai resuls I his disseraio, i order o focus o he lower ail of he ROC curve, we cosider a arial AUC G ( P = F( dg(. We roose a oaraeric esiaor of his arial AUC, arially iegraed ROC, which is defied as G ( Pˆ = F( dg(, 4

29 where R. The weigh fucio used i his esiaor is Ι, which gives a [ G ( ] weigh of a he lower ail of he ROC curve whe G ( ad a weigh of a he uer ail whe > G (. The differece bewee his esiaor wih he esiaor of Dodd ad Pee (3 is ha we focus o he eire lower ail of he ROC curve. I Chaer 3, we show ha ˆP is a cosise esiaor of P, derive is asyoic disribuio, ad show ha uder he ull hyohesis, i is asyoically disribuio free. The above arially iegraed ROC ( ˆP is based o esiaed liis i is iegraio. Whe G ( wih fixed liis, which is is kow, we roose a esiaor, arially iegraed ROC G ( Pˆ = F( dg(. We derive is asyoic disribuio, ad show ha uder he ull hyohesis, i is asyoically disribuio free i Chaer 3. We aly he arially iegraed ROC saisic, whe he rage of is [,], o he fucioal caegory aalysis i icroarray exeries. Whe he ROC curve crosses wih he Uifor CDF ad if he cross occurs before, he cross could ifluece he erforace of he arial AUC esiaio ad give isleadig resul, ha is, he es based o ˆP saisic could show o-sigificace whe here exiss a sigificace. I soe cases, ROC curve crosses he Uifor CDF ore ha oce, which akes he cross ore likely o hae before. To address his issue, we evaluae he followig weighed arial AUC G ( ( ( P F( F G = Φ dg(, where R, Φ ( deoes Noral (, CDF, ad is a cosa. Noe ha his weigh fucio gives large weigh whe F ( G ( ad sall weigh whe F ( < G (. We roose he followig esiaor of P, arially iegraed weighed ROC, which is defied as 5

30 G ( ˆ F( G( P = F( Φ dg(. I Chaer 4, we show ha ˆP is a cosise esiaor of P, derive is asyoic disribuio, ad show ha uder he ull hyohesis, i is asyoically disribuio free. We also show he cosisecy of G ( ˆ F( G( P = F( dg( Φ o P, where he rae of covergece of is ( loglog,. We aly he arially iegraed weighed ROC saisic, whe he rage of is [,], o he fucioal caegory aalysis i icroarray exeries..5 Suary of he disseraio I his disseraio, we roose wo ew oaraeric saisics for esiaig arial area uder he ROC curves wih alicaio i he fucioal caegory aalysis i icroarray exeries. Chaer coais soe backgroud heories fro Serflig (, va der Vaar (998, ad va der Vaar ad Weller (996, which are used i Chaers 3 ad 4. I Chaer 3, we roose a oaraeric esiaor of he arial AUC P, he arially iegraed ROC ( ˆP, we show ha ˆP is a cosise esiaor of P, derive is asyoic disribuio, show ha uder he ull hyohesis, i is asyoically disribuio free, ad aly ˆP ad is asyoic disribuio o he fucioal caegory aalysis i icroarray exeries. We also derive he asyoic disribuio of he arially iegraed ROC wih fixed liis i his chaer. I Chaer 4, we roose a oaraeric esiaor of weighed arial AUC P, he arially iegraed weighed ROC ( ˆP, we show ha ˆP is a cosise esiaor of P, derive is asyoic disribuio, show ha uder he ull hyohesis, i is asyoically disribuio free, ad aly ˆP ad is asyoic disribuio o he fucioal caegory aalysis i icroarray exeries. We also exaie he cosisecy of he arially iegraed 6

31 weighed ROC wih chagig weigh fucio i his chaer. I Chaer 5, we exaie oial alha levels ad ower of our ew saisics ogeher wih exisig ehods via siulaio sudy, where ower is evaluaed wih fixed aleraives ad coiguous aleraives; we also coare exac ad asyoic ower as chages. I Chaer 6, we alied he asyoic disribuios of our wo ew saisics, he arially iegraed ROC ( ˆP ad he arially iegraed weighed ROC ( ˆP ha are derived i Chaers 3 ad 4 o he fucioal caegory aalysis of a icroarray sudy of olfacory sesory euros. Chaer 7 coais suary ad discussio. 7

32 Chaer Preliiary Resuls o Asyoic Theories Our aroaches of ivesigaig he asyoic roeries of our oaraeric esiaors of arial area uder he ROC curve i Chaers 3 ad 4 are aily based o he heories i Serflig (, va der Vaar (998, ad va de Vaar ad Weller (996. I his chaer, we will sae he ai resuls fro hose books ha we have used.. Eirical rocesses Le X,, X be a rado sale fro a robabiliy disribuio P o a easurable sace (, X A. Give a easurable fucio f : X R, deoe P f o be he execaio of f uder he eirical easure, ad P f o be he execaio uder P. Thus, (. ad P f = f ( x i, i = (. P f = f dp. By he law of large ubers, P f P f alos surely, as, if P f exiss. By ceral lii heore, he eirical rocess ( f Pf P is asyoically oral if Pf <. A iora asec i eirical rocess heory is he uiforiy of a class of fucios. The uiforiy of a class of fucios deeds o he size of he class. Aroriae easures of he size are wo yes of eroy ubers: coverig ubers ad brackeig ubers. Before we iroduce he defiiios of he eroy ubers, firs we eed o defie Lr ( P -or as i va de Vaar ad Weller (996. Defiiio. (va der Vaar ad Weller 996 For a fucio Q : F R, defie { } Q = su Qf, f F. The ( -or F r r L P is defied as f = Pr, ( P f / r. 8

33 N F be a subse of a ored sace of real fucios f : X R o soe se. Le (,. The followig defiiios of coverig ubers ad brackeig ubers are give by va de Vaar ad Weller (996. Defiiio. (va der Vaar ad Weller 996, Defiiio..5 The coverig ubers N ( ε, F,. is he iial uber of balls { g: g f ε} eeded o cover F. < of radius ε Defiiio.3 (va der Vaar ad Weller 996, Defiiio..6 Give wo fucios l ad u, he brake [ lu, ] is he se of all fucios f wih l f u. A ε -bracke is a bracke [ lu, ] wih u l < ε. The brackeig uber N [ ] ( ε, F,. is he iiu uber of ε -bracke eeded o cover F. Noe ha brackeig ubers ad coverig ubers are relaed by (.3 N( ε F N[ ]( ε F,,.,,.. va de Vaar (998 furher gives he followig defiiios. Defiiio.4 (va der Vaar 998 A eveloe fucio of a class F is ay fucio x F( x such ha f ( x F( x, for every x X ad f F. Defiiio.5 (va der Vaar 998 The brackeig iegral is defied as δ [ ]( F [ ]( F J δ,, L ( P = log N ε,, L ( P dε. Defiiio.6 (va der Vaar 998 Defie uifor coverig ubers o be ( ε F Qr, r su N F,, L ( Q, he he uifor eroy iegral is defied as Q δ ( F ( F Q, J δ,, L = logsu N ε F,, L ( Q d ε. Q 9

34 The uifor eroy iegral has he followig roery give by va de Vaar (998. Exale. (va der Vaar 998, Exale 9.9 Le F ad G ossess a fiie uifor eroy iegral, relaive o eveloe fucios F ad G, he so does he class FG of all fucios x f( x g( x, relaive o he eveloe fucio FG. Furher oe ha i var der Vaar ad Weller (996, he ouer robabiliy of a arbirary subse B of X is defied as { } P ( B = if P ( A : A B, A A. Now we iroduce 3 classes of fucios, which are Gliveco-Caelli, Dosker, ad VC classes of fucios. We sae he roeries of hese classes of fucios ha are used i Chaer 3 ad Chaer 4. Defiiio.7 (va der Vaar 998 A class of easurable fucios f : called P-Gliveco-Caelli if (.4 P f Pf = su P f Pf,. F f F as X R is Theore. (va der Vaar 998, Theore 9.3 Le F be a suiably easurable class of easurable fucios wih su (,, ( Q N ε F L Q, Q PF<, he F is P-Gliveco-Caelli. F < for every ε >. If Defiiio.8 (va der Vaar 998 A class F of easurable fucios f : { f Pf : f F } called P-Dosker if he sequeces of rocesses ( X R is P coverges i disribuio o a Gaussia rocess wih zero ea of covariace fucio P fg P f Pg i he sace of ( F.

35 Exale. (va der Vaar 998, Exale 9. Le F be he collecio of all oooe fucios ha are of variaio bouded by. The here exiss a cosa K such ha, for every r ad robabiliy easure P such ha log N[ ]( ε, F, L( P K. ε Thus, his class of fucios is P-Dosker for every P. Theore. (va der Vaar 998, Theore 9.4 Le F be a suiably easurable class of easurable fucios wih (, F, J L <. If PF <, he F is P-Dosker. Lea.3 (va der Vaar 998, Lea 9.4 Suose ha F is a P-Dosker class of easurable fucios ad f is a sequece of rado fucios ha ake heir values i F such ha ( f ( x f ( x dp( x coverges i robabiliy o for soe f L ( P. The, ( Pf Pf Pf + Pf = o(,. Theore.4 (va der Vaar ad Weller 996, Theore..6 Le F,, F k be Dosker classes wih P < for each i. For a fixed a φ : R R, le F φ i ( F,, F k deoe he class of fucios x φ ( f (,, ( x f k x as f = ( f,, f k i F F k. Le φ : k R R saisfy φ f ( x φ k g( x ( f l( x g l( x l =. The he class φ ( F,, F k is Dosker, rovided ( f,, f k for a leas oe ( f,, f k. φ is square iegrable k Exale.3 (va der Vaar ad Weller 996, Exale..7 If F ad G are Dosker classes ad P <, he he airwise sus F + G is a Dosker class. F G

36 Exale.4 (va der Vaar ad Weller 996, Exale..8 If F ad G are uiforly bouded Dosker, he he airwise roducs F G for a Dosker class. A VC class fucios is defied hrough cobiaorial roeries. var der Vaar (998 gives he followig defiiios. Defiiio.9 (va der Vaar 998 A collecio C of subses of he sale sace X icks ou a cerai subse A of he fiie se { } x x X if i ca be wrie as, { } C for soe C C. The collecio C is said o shaer { x x } A= x x, if C, icks ou each of is subses. The VC idex V ( C of C is he salles for which o se of size is shaered by C. A collecio C of easurable ses is called a VC class if is idex V ( C is fiie. Defiiio. (va der Vaar 998 A collecio F is a VC class of fucios if he collecio of all subgrahs {( x, : f( x } ses i X R. <, if f rages over F, fors a VC class of Releva roeries of he VC classes of fucios are saed below. Lea.5 shows ha VC class of fucios have fiie coverig ubers, which are bouded by a olyoial i /ε. Exale.5 shows ha { [ x ] : } Ι R is a VC class of fucios. Lea.6 shows ha a secific rasforaio of a VC class of fucios is also a VC class. Lea.5 (va der Vaar 998, Lea 9.5 There exiss a uiversal cosa K such ha for ay VC class F of fucios, ay r ad < ε <, rv ( ( F V ( F su N F,, L ( (6. Q, r r Q KV e Q ε (.5 ( ε F ( F

37 Exale.5 (va der Vaar 998, Exale 9.6 The collecio of all cells (, i he real lie is a VC class of idex V ( C =. Lea.6 (va der Vaar ad Weller 996, Lea.6.8 (vi Le F ad G be VC classes of fucios o a se X ad a fixed fucio g : { : } F g = f g f F is VC class. X R. The,. Geeralized U-saisics Serflig ( saes he asyoic heory of he geeralized U-saisics of Hoeffdig (948 i secios 5..3 ad 5.5. for k-sale case. I his secio, we iroduce he corresodig wo-sale case of he geeralized U-saisics. Cosider wo ideede sales of ideede observaios { X, X,, X } ad { Y, Y,, Y } ake fro disribuios F ad G, resecively. Le { h( X X X } Y Y Y θ = E,,,,,,,,, where h is assued wihou loss of geeraliy o be syeric wihi each sale. For ad, he corresodig U-saisics is defied as Here { ij,, ijj} ad ( i i i i U = h X,, X, Y,, Y. j c Π j = j deoes a se of j disic elees of he se {,,, j}, j, deoes suaio over all such cobiaios. Le c { X, X,, X } { Y, Y,, Y } ad have exacly c X i s ad d Y j s i coo, where c =,, ad d =,,. Le, ( hcd, = E h X,, X, Y,, Y X,, Xc, Y,, Y d, 3

38 ad Scd, = var ( hcd,. For E( h he <, if + λ, λ < <, ad i (,, S, S, + ( U θ is AN, +. λ λ Nex we give he asyoic disribuio of Wilcoxo Ma-Whiey U-saisic. Exale.6 (Wilcoxo Ma-Whiey U-saisic Le { X X X } { Y Y Y },,, ad,,, be ideede rado fro coiuous disribuios F ad G, resecively. The for A ubiased esiaor is ( FG FdG P( X Y θ, = =, U = Ι Xi Y j. = = i j The, ad Hece, ad S = var ( h, = E h X, Y X [ ] = E X Y X Ι = GX (, ( h, = E h X, Y Y [ ] = E X Y Y Ι = FY (. ( h,, ( = G ( df( G( df(, 4

39 Thus, for S = var λ, λ + ( h,, ( = F ( dg( F( dg(. < <, i (, Wilcoxo Ma-Whiey U-saisic is, he asyoic disribuio of he d S S + ( U θ N + λ λ,,,. Uder H : F ( = G (, he asyoic disribuio becoes d + ( U θ N,. λ( λ.3 Relaed aroxiaio heores The followig Theores i Serflig ( are used i he roofs i Chaer 3 ad Chaer 4. k Theore.7 (Serflig, Theore.7, Coiuous aig Theore Le X, X, Xk be rado k-vecors defied o a robabiliy sace ad le g be a vecorvalued Borel fucio defied o R. Suose ha g is coiuous wih P -robabiliy. The, w w ( ( X X g X g X. X Theore.8 A (Serflig, Theore.. A Le he fucio g have a fiie h derivaive ( g everywhere i he oe ierval ( ab, ad ( h derivaio ( g coiuous i he closed ierval [ ab, ]. Le x [ ab, ]. For each oi y [ ab, ], y x, here exiss a oi z ierior o he ierval joiig x ad y such ha ( k ( g ( x k g ( z g( y = g( x + ( y x + ( y x. k!! k = 5

40 Theore.8 B (Serflig, Theore.. C, Youg s for of Taylor s Theore Le g have a fiie h derivaive a he oi x. The, k = ( ( k g ( x k g( y g( x ( y x = o y x, y x. k! The sale disribuio fucio F ( has he followig asyoic roeries. Theore.9 (Serflig, Theore..3 A Reark (iii Le F be defied o R. There exiss a fiie osiive cosa C (o deedig o F such ha for all =,,. ( P su F ( F( d > Cex d, d >, Theore. (Serflig, Theore..4 B Wih robabiliy, su ( ( = { F [ F] } F F li su ( (. ( loglog Defie he h quaile of F o be { } he asyoic roeries of he sale quaile. ξ = if x : F( x. The followig Theores give Theore. (Serflig, Theore.3. Le < <. If ξ is he uique soluio x of F( x F( x, he ˆ ξ w ξ. Corollary. (Serflig, Corollary.3.3 A Le < <. If F is differeiable a ξ ad F ( ξ >, he ˆ ( ξ is AN ξ,. F ( ξ 6

41 Theore.3 (Serflig, Theore.5., Bahadur (966 Le < <. Suose ha F is wice differeiable a ( = >. The, ξ, wih F ξ f ( ξ ( ξ F ˆ ξ = ξ + + R, f ( ξ where wih robabiliy 3/4 ( ( R = O 3/4 log,. 7

42 Chaer 3 Parially Iegraed ROC I his chaer, we roose a oaraeric esiaor of arial area uder he ROC curve, arially iegraed ROC, ad cosider is asyoic roeries. Le X,, X be i.i.d. rado variables wih disribuio fucio F ( ad sale disribuio fucio F (, R. Le Y,, Y be i.i.d. rado variables wih disribuio fucio G ( ad sale disribuio fucio G (, R. Le X,, X ad Y, Y be uually ideede. Defie, F ( = if{ x: F( x }, < <. Le < <. Deoe G ( P = F( dg( o be he arial area uder he ROC curve. We roose he esiaor of P, he arially iegraed ROC, o be Noe ha ˆP is equivale o G ( Pˆ = F( dg(. Ι[ Xi Yj] Ι[ Yj G ( ]. i= j= Whe he quaile G ( is kow, we have he followig esiaor of P G ( Pˆ = F( dg(. I Secio 3., we derive he asyoic disribuio of ˆP. I Secios 3. ad 3.3, we derive he asyoic roeries of ˆP. I Secio 3.4, we aly ˆP ad is asyoic disribuio o he fucioal caegory aalysis i icroarray exeries. 3. Asyoic disribuio of he arially iegraed ROC wih fixed liis I his secio, we derive he asyoic disribuio of geeralized U-saisics. ˆP usig he heory of 8

43 Lea 3. Le F ( ad G ( be coiuous disribuio fucios. Le ad be sequeces of iegers such ha where λ, < λ <, as., The, + d S, S, + ( Pˆ P N,,,, + λ λ ad G ( ( S, = [ G (] df ( [ G (] df (, G ( ( S, = F ( dg( F( dg(. Uder H : F ( = G (, for all R, Proof: Noe ha d 3 4 ( ˆ + P P N,. λ( λ 3 4 G ( Pˆ = F ( dg ( = Ι X Y Ι Y G i= j= [ i j] [ j ( ] is a geeralized U-saisic. Now we aly he heory of geeralized U-saisics. Le hx Y X Y Y G ( i, j =Ι[ i j] Ι[ j ( ]. The, ad h ( X = E[ h( X, Y X ], = E Ι[ X Y] Ι[ Y G ( ] X = E Ι[ X Y G ( ] X = GG ( ( GX ( Ι[ X G ( ] [ ] = G( X Ι[ X G ( ], 9

44 Hece, S = Var( h ( X ad Thus, for,, h,( Y = E[ h( X, Y Y ] = E Ι[ X Y] Ι[ Y G ( ] Y =Ι[ Y G ( ] E[ Ι[ X Y] Y ] =Ι[ Y G ( ] F( Y. ([ G X ] X G = Var ( Ι[ ( ] [ ] [ ] { } = E G( X Ι[ X G ( ] E G( X Ι[ X G ( ] G ( G ( = [ G( X ] df( X [ G( X ] df( X G ( G ( = [ G (] df ( [ G (] df (, S = Var( h ( Y,, ( Y G F Y = Var Ι[ ( ] ( ( Y ( G F Y E Y G F Y = E Ι[ ( ] ( Ι[ ( ] ( ( = Ι[ Y G ( ] F ( Y dg( Y Ι[ Y G ( ] F( Y dg( Y = G ( G ( F ( Y dg( Y F( Y dg( Y ( G ( ( G ( = F dg F ( dg (. λ, λ + ( ˆ + P P is Uder H : F ( = G (, we have < <, i (,, he asyoic disribuio of d,, ( ˆ S S + P P N,. + λ λ 3

45 ad G ( G [ ] ( S, = G ( dg ( [ G (] dg ( G ( G ( = G( + G ( dg( G( ( 3 G ( ( ( = G G + 3 = , ( The, G ( ( G S, = F ( dg( F( dg( G ( ( G = G ( dg( G( dg( 3 ( ( G ( G ( = 3 = S, S, + = +. λ λ λ λ 3 4 Thus, uder H : F ( = G ( for all R, he asyoic disribuio becoes d 3 4 ( ˆ + P P N,. λ( λ Cosisecy of he arially iegraed ROC wih esiaed liis Noe ha ˆP i Secio 3. deeds o he disribuio fucio G (. Whe G ( is ukow, G ( ca be esiaed by sale quaile G (, which is used i our esiaor ˆP. I his secio, we show ha ˆP is a srogly cosise esiaor of P, as., Firs, we show he followig wo resuls. 3

46 Lea 3. The class of fucios, = { F ( Ι[ c] : c } fucios. Proof: As i Exale.5, { [ c] : c } F R, is a VC-class of Ι R is a VC-class of fucios. Sice F ( is a fucio ha as R o R, by Lea.6, = { F ( Ι[ c] : c } F R is a VC-class of fucios. Lea 3.3 Le G ( be coiuous. Le G ( G (, < <, R. The, G ( be he uique soluio of G ( G ( FdG ( ( FdG ( ( w,. Proof: By Lea 3., = { F ( Ι[ c] : c } F R is a VC-class of fucios. Sice he eveloe fucio of F is, by Lea.5, for ay easure Q, ay r, < ε <, a give eveloe fucio H, we have ( ε F Qr, r su N H,, L ( Q <. Thus by Theore., = { F ( Ι[ c] : c } fucios. By (., (., ad (.4, we have This ilies for c R, Noe ha Q F R is a P-Gliveco-Caelli class of { P } su g Pg w,. g F su FdG ( ( FdG ( ( w,. c c c ( G G ( c FdG ( ( FdG ( ( su FdG ( ( FdG ( ( c w,. Therefore, for ay give ε >, here exiss N such ha for all N, (3. G ( G ( G ( FdG ε FdG FdG + ε Nex oe ha by Theore., ( ( ( ( ( (. c 3

47 G ( G (, w,,. The for ay give ε >, here exiss N such ha for all N, Tha is, G ( G ( ε w,. (3. G ( ε G ( G ( + ε w,. Cobiig (3. ad (3., for ay give ε > ad ε >, for all ax( N, N have, we (3.3 ε G ( G ( FdG ε ( ( FdG ( ( ε G ( + FdG ( ( + ε w,. Nex subracig G ( FdG ( (, we have (3.4 ε G ( G ( G ( ε G ( FdG ( ( FdG ( ( FdG ( ( G ( + FdG ( ( + ε w,. G ( ε ε ε Therefore, for a give ε >, choose ε ad ε, here exiss δ, N, N such ha for ay give se A i Borel ses, P ( A G δ ilies FdG ( ( ε ad, for all A ax( N, N, based o (3.4, we have Hece, G ( G ( ε FdG ( ( FdG ( ( ε w,. G ( G ( FdG ( ( FdG ( ( w,. 33

48 Theore 3.4 Le G ( be coiuous. Le G ( be he uique soluio of G ( G ( Proof:, < <, R. The ˆP P, w, as., G ( G ( Pˆ P = F ( dg ( F( dg( G ( G ( G ( [ ] = F ( F( dg ( + F( dg ( F( dg( G ( G ( G ( [ ] F ( F( dg ( + F( dg ( F( dg(. G ( Firs cosider [ ] (3.5 By Gliveco-Caelli Theore, Thus, F ( F( dg (. Noe ha G ( G ( [ ] F ( F( dg ( su F ( F( dg ( G ( = su F ( F( dg ( su F ( F(. su F ( F( w,. G ( Also oe ha by Lea 3.3, Thus, Tha is, [ ] F ( F( dg ( w,. G ( G ( FdG ( ( FdG ( ( w,. P ˆ P w,,. Pˆ P w,,. 34

49 3.3 Asyoic disribuio of he arially iegraed ROC wih esiaed liis Now we derive he asyoic disribuio of ( Pˆ P +, as., Le, be sequeces of iegers such ha λ, < λ <, as., Firs we + cosider a class of fucios = { H ( Ι[ c] : c R, H } where H = { H : H is a disribuio fucio} disribuio of ( P ˆ P F H by Leas 3.5 ad 3.6, +, as.,. Nex we derive he asyoic Lea 3.5 The class of fucios = { H ( Ι[ c] : c R, H } F H is a P-Dosker class of fucios, where = { H : H is a disribuio fucio} H. Proof: As i Exale., { H (: H H } has fiie brackeig iegral. Noe ha by (.3, N( ε,,. < N[] ( ε,,. F F, hus his class of fucios has fiie coverig ubers. Therefore he class of fucios { H (: H H } has fiie uifor eroy iegral wih eveloe fucio. As i Exale.5, { [ c] : c } Ι R is a VC-class of fucios. By Lea.5, his class of fucios has fiie coverig ubers, hus has fiie uifor eroy iegral wih eveloe fucio. { c : c } Cobiig he above resuls, boh { H (: H H } ad [ ] Ι R have fiie uifor eroy iegral wih eveloe fucio. Hece, by Exale., { H ( [ c] : c R, H } F = Ι H has fiie uifor eroy iegral wih eveloe fucio. Sice he execaio of he squared eveloe fucio is fiie, by Theore., = { H ( Ι[ c] : c R, H } F H is a P-Dosker class of fucios. 35