A PROBABILTY NEURAL NETWORK FOR CONTINUOUS AND CATEGORICAL DATA. Shuang Cang 1 and Hongnian Yu 2

Transcription

1 PROBBITY EUR ETWORK FOR COTIUOUS D CTEGORIC DT Shug Cg d Hog Yu Dertet o Couter See Uversty o Wles berystyth Y3 3DB UK Fulty o Coutg Egeerg d Tehology Stordshre Uversty Stord ST8 DG UK bstrt: I ost lto o the dt lsstos the dt sets ot both otuous d tegorl vrbles. I other ord ultvrte dt sets otg tures o otuous d tegorl vrbles rse requetly rte. Ths er resets ovel Probblty eurl etor P hh lssy the dt or both otuous d tegorl ut dt tyes. The se th ether otuous or tegorl ut vrbles s sel se o the tures o otuous d tegorl ut vrbles. Thereore the roosed P be lso led to these to sel ses. Eetto sto E lgorth s dely used or ture odels o otuous vrbles but ot lble or tegorl vrbles. ture odel o otuous d tegorl vrbles s used to ostrut Probblty Desty Futo PDF hh s the ey rt or the P. The roosed P hs to dvtges org th the ovetol lgorths suh s the ultlyer Peretro P eurl etor. Oe dvtge s tht the P rodue better results org th the P eurl etor eve usg the orlzed ut vrbles or the P. orlly the orlzed ut vrbles geerte better result th the o-orlzed ut vrbles or the P eurl etor. other dvtge s tht the P does ot eed the ross vldto dt set d does ot rodue the over trg le the P eurl etor does. These hve bee rove our eeretl study. The roosed P lso be used to eror the usuervsed luster lyss. The suerorty o P org the P eurl etor s deostrted by lyg the to rel-le dt set the Tru dt set hh ludes both otuous d tegorl vrbles. Coyrght 5 IFC Keyords: eurl etors Probblty Desty Futo PDF Clssto Ptter Reogto ture odels Eetto sto E lgorth.. ITRODUCTIO Probblty desty utos PDF ly ortt role tter reogto. I e o PDF or eh lss the the robblty o the e tter belogg to eh lss be obted by usg the Byes rule. ture odels Yulle 994 hls Bsho 995 re dely used to rote true desty uto or otuous vrbles. The Prze do esttor Prze 96 s udetl tehque or esttg PDF. The ture odel or the bry vrbles s studed by Yg. Hoever t s qute ote tht the dt set o rel lto ots both otuous vrbles d tegorl vrbles th vlues >. It s oted tht the bry vrbles re the sel se o tegorl vrbles. Ths er resets e roh o dg PDF or ture o otuous d tegorl ut vrbles hh e ll the robblty eurl etor P or the tures o otuous d tegorl ut vrbles. U to o ost o the ors re delg th ether the bry vrbles or otuous vrbles hh re sel ses o our roh. The uber o ooets be detered by usg the lgorths roosed

2 by Cg hh hs suessvely be led or deterg the uber o ooets PDF. Ths er ossts o ve setos. I seto the dely used Eetto sto E lgorth s brely desrbed. Seto 3 resets e ture odel or the ture o otuous d tegorl vrbles. The robblty eurl etor d the roedures o trg robblty eurl etor re reseted Seto 4. eeretl result s reseted Seto 5. The olusos re reseted Seto 6.. GUSSI ITURE ODES FOR COTIUOUS VRIBES The E lgorth s dely used to estte reters ture odels or otuous vrbles. For ooets the ture desty or d desol vetor be rtte s ler obto o ooet desty utos the or P. here P re the reters the ture odel d stsy the ollog odtos P P. The ooet desty utos. stsy d.3 The ost dely used dstrbuto or eh ooet desty s the Guss dstrbuto. The Guss ture odel s oly lble or otuous vrbles. The or o the Guss desty uto or eh ooet s π T µ Σ µ / e d / / Σ.4 here the reters µ d re the es o d- desol vetor d dd ovre tr resetvely. Vlues or the reters P µ d be detered eh ooet usg the E lgorth Bsho 995 s ollos. Frst K-es lusterg ethod s used or ed uber o ooets. to detere reters P µ d or eh ooet.4. Clerly the odto.3 s stsed. The the reters P µ d re obted or eh ooet usg the ollog reursve oruls. P µ µ µ Σ.5 here s the sze o the dt set d the eght s ' P P.6 s deed.4. Iterte E-ste -ste stble ot or the reters the ture odel be rehed. The E lgorth reseted bove requres tht ll the vrbles re otuous d the E ot be led to the ses th ture o otuous d tegorl vrbles. We roose oded E lgorth hh del th the se or ture o otuous d tegorl vrbles et seto. 3. ITURE ODES FOR COTIUOUS D CTEGORIC VRIBES 3. ture odels I. e ssue tht s d-desol vetor hh ots d otuous vrbles d bry vrbles d vrbles th 3 tegorl vlues d geerl d - vrbles th tegorl vlues thus d d d. The E lgorth desrbed seto ot be led dretly. We roose e odel to hdle ths ssue. The be rereseted s [ ]. d d d ssug tht the tegorl vrbles re deedet o eh other or ture o otuous d tegorl vrbles ture odel. e roose to rereset ooet s the ollog or / d Π d d / Π.. Π 3. here re the otuous vrbles d s uto o vrble. We sho tht the ooet desty utos stsy d. To sel ses re: I the vrbles re ll otuous vrbles 3. redues to d stses Guss dstrbuto.4.

3 I the vrbles re ll bry vrbles 3. redues to d Π. Thereore the roosed odel 3. ludes the otuous or bry odels hh re dely led s sel se. et e sho tht ho to detere the reters 3. or ture o otuous d tegorl vrbles y uber o tegorl vrbles. The geerl or o the utos s vestgted seto Deterg ture odel 3. I ths seto e ll develo geerl or or the utos 3.. For s bol robblty dstrbuto thus -. For 3 - d. I geerl utos be rereseted s order olyol We rerte the uto s ollos: T here... d. I vrble tes the tegorl vlue } { the let. So the oeets d stsy the ollog equtos. C 3.4 here T d C d re the eleets o tr d re dsrete vlues suh s. For ele d the d C I d the d C. The oeets o vetor be obted usg equto 3.4 C 3.5 The oeets... or y be lulted by usg 3.5. It s oted tht equto 3.5 s esy to ly. We sho tht the verse o ests d the

4 outto s ot ssue se t ll be outed o-le. 3.3 u elhood d E lgorth or ture odel I ths seto e detere the reters the th ooet deed 3.. Guss ooets re used or the otuous vrbles. The the ture odel 3. ots the ollog dustble reters: P µ d ro Guss ooets d ro bry ooets d d ro terry vlues ooets d so o. The egtve log-lelhood Bsho 995 or the set o tters { } s E l l{ P } 3.6 Fro 3.6 t s esy to see tht zg the lelhood s equvlet to zg E. Settg the dervtves th reset to eh reter 3.6 to zero e obt P µ µ µ Σ 3.7 here s the sze o the dt set d the eght s P / 3.8 P here s deed 3. s. It s oted tht equtos 3.7 d 3.8 re the se s equtos.5 d.6 but the lulto o eghts deed 3.8 eed hh s ture o otuous d tegorl vrble 3.7 d 3.8. For geerl tegorl vlue hh hs reters d d 3. settg the dervtves th reset to eh o the reters 3.6 e obt the ollog equtos 3.9 ' Deg F ro equto 3.9 e obt F u F u 3. here d d. ote tht. For se hh s bol robblty dstrbuto e hve d 3. I the oded E lgorth or the u lelhood 3.6 e use d 3. to udte the reters. The ture odel or otuous d tegorl Vrbles s detered. 4. PROBBIITY EUR ETWORK The rhteture o robblty eurl etor s desrbed Fgure. Tylly ths robblty eurl etor hs d uts d oututs oe or eh lss. The deret th lssl eurl etor les o the se utol or o the bse utos hh re osdered to be desty utos or eh lss s ell s o soe ostrts volvg ro the hdde to the outut lyer. C C P C P C P C P C C C C C C d Fgure : Probblty eurl etor Follog the dsusso seto 3 d Fgure e surse the outg lgorth or 3. s belo.

5 oded E lgorth:. For the suervsed lerg dvde the dt set to to rts the trg d test dt sets. The sze o the trg dt set s.. Prtto the trg dt set or eh lss. Thus e obt uber o sub-trg dt sets. Eh sub-trg dt set belogs to oe lss. The sze o the sub-trg dt set s C or lss C [ K] d K s totl uber o lsses. 3. For eh sub-trg dt set hh belogs to oe lss deterg the uber o ooets usg the lgorths roosed by Cg d the oute the reters or eh ooets 3. usg d 3.. The obt the ture odels / C deed. or eh lss here C dtes lss. 4. Coute the ror robblty or eh C lss P C here P C. 5. For eh e tter tter test dt set oute the vlues / C P C [ K] d detere C by tg / C P C. 5. EPERIET STUDY I ths seto e ly the e P to relst dt set Tru dt set d e lso ore P th the stdrd P eurl etor. The ross vldto ethod s used or these robles. The dt s rttoed to ed uber o rttos or olds eh old s hold out s test dt tur hle the other - olds re esured d the the esttes re verged to gve l ury. I the ollog lyses the detos or the ouso tr sestvty sety true ostve rte TPR d lse ostve rte FPR or to lsses re gve s ollos. Tble : The ouso tr or to lsses: True Clss bel Predto Clss Clss Clss bel Clss Clss here Sestvty uber o true ostve desos/uber o tully ostve ses /. True ostve rte TPR Sestvty. Flse ostve rte FPR Sety. The Heloter Eergey edl Serve HES tthed to the Royl odo Hostl hs gthered dt ro re-hostl tru tets over teyer erod. The sze o the tru dt s 44 eludg 3 tters tht ot ssg dt soehere. The outoe s lved/ded redto o dvdul tets. The tru dt s ubled dt th oly 58 ded ses og 44 tters. There re 6 etures ludg 5 otuous etures ths dt set. The etures re desrbed s Tble here Co dtes Cotuous eture d Ct Idtes tegorl eture. Tble : Fetures the tru dt set e Tye Vlues Desrto ge Co. ge ro to Geder Ct. lefele Iury Tye Ct. BlutPeetrtg Hed Ct Hed ury 6 Fl Ct. 3 4 Fl ury Chest Ct Chest ury 6 bdol Ct bdol or elv otets ury bs Ct bs or boy elvs ury Eterl Ct. 3 Eterl ury Resrto Co. Resrto rte Rte Systol Co. Systol blood ressure Blood GCS Eye Ct. 3 4 Glsgo o sore GCS eye resose GCS otor Ct GCS otor resose 6 GCS Ct GCS verbl resose Verbl Oetry Co. Oetry % red blood ell O sturto Hert Rte Co. Hert rte Clss Ct. Clssto s ded d s lved. I order to test every sgle tter d e orsos or these ethods e dvded the tru dt to ve lost eqully sze old thout overlg eh other. The szes o eh old re d resetvely. We used oe old s the test dt set d the rest o our old s the trg dt set tur. Ths hs gurteed tht every tter the tru dt ould test oe. Ths rdo rttog s doe tes. For eh rtto dt the es o the sestvty sety d lssto rtes or ll 5 olds o the test dt re lulted by usg to ethods. Oe s the e P. The other ethod s the stdrd P eurl etor th oe hdde lyer. The results o the sestvty sety d lssto rte set re sho Tble 3 or the test dt. Sety uber o true egtve/uber o tully egtve ses /.

6 Tble 3: The trg d test erore results o eh dt rtto there re rdo rttos SEsestvty SPseto Clssto Trg Dt Probblty eurl etor P ultlyer Peretro eurl etor P SE SP C SE SP C e Stdrd Dvso Test Dt Probblty eurl etor P ultlyer Peretro eurl etor P SE SP C SE SP C e Stdrd Dvso Fro Tble 3 e see tht our e roh P th 88.4% lssto rte overll e gve better result or the test dt set th the P eurl etor th 85.8% eve use the olzed ut vrbles or P eurl etor. For the trg dt e see tht there s over trg or P eurl etor hle there s o robles o ths tter or the P. 6 COCUSIOS Ths er hs roosed e robblty eurl etor P or ture o otuous d tegorl vrbles uts. We hve used relst dt set Tru dt set to deostrte tht the roosed ethod s ore relble d ore urte th the stdrd P eurl etor. It be led y robles. The roble or the P eurl etor s the over trg hle the P overoes ths roble d the P does ot eed to use the ross vldto dt set. The P be used s usuervsed lerg s ell. We obt the lusters thout osderg the lss lbel d esure the slrty d deret og the lusters. The uber o the lusters be detered by the lgorths roosed by Cg. We oud tht or usuervsed lerg usg the P s ore urte d ore robust th the rle ooet lyss PC or the ost olte dt sets. REFERECES S. Cg d D. Prtrdge Deterg the uber o ooets ture odels Usg Wlls Sttstl Test Pro. o the 8 th Itertol Coeree o eurl Iorto Proessg Ch Prze E. 96 O estto o robblty desty uto d ode. ls o thets Sttsts Yulle.. Stolorz P. d Uts J. 994 Sttstl hyss tures o dstrbutos d E lgorths. eurl Coutto hls K. Ttss d rstds C. s Shred erel odels or lss odtol desty estto IEEE Trstos o eurl etors VO. O Z. R. Yg bry robblst odel d geet lgorth or HIV rotese levge stes redto d serh Pro. o the 8 th Itertol Coeree o eurl Iorto Proessg Ch Y. Youg d G. Corlu 97 Stohst estto o ture o orl desty utos usg orto rtero. IEEE Trs. O Iorto Theory Crrer-Per.. ode-dg or tures o Guss dstrbutos IEEE Trs. o Ptter lyss d he Itellgee Rhrdso S. d GreeP.J 997 O Byes lyss o tures th Uo uber o Cooets. Jourl o the Royl Sttstl Soety B Detro 989 er Jourl o Crdology C. Bsho 995 eurl etors or tter reogto Oord Press. The P be led to ult-lss d hgh desol dt set. I order to redue outto te or hgh desol dt set e redue deso by usg der rge o eture seleto lgorths or rl ooets lyss PC rst the ly P to ths redug desol dt set s ut.