Structural Risk Minimization Principle Based on Complex Fuzzy Random Samples *

Transcription

1 ISSN Engand UK Jouna of Infomaion and Compuing Siene Vo 5 No 00 pp Suua Ris Minimizaion Pinipe Based on Compex Fuzzy Random Sampes * Zhiming Zhang +a * Jingfeng Tian b a Coege of Mahemais and Compue Sienes Hebei Univesiy 80 Wu Si Eas Road Baoding 0700 Hebei Povine PR China b Siene and Tehnoogy Coege Noh China Eei Powe Univesiy 8 Rui Xiang See Baoding 0705 Hebei Povine PR China (Reeived Febuay aeped Api 5 009) Absa Saisia Leaning Theoy is ommony egaded as a sound famewo wihin whih we hande a vaiey of eaning pobems in pesene of sma size daa sampes I has beome a apidy pogessing eseah aea in mahine eaning The heoy is based on ea andom sampes and as suh is no eady o dea wih he saisia eaning pobems invoving ompex fuzzy andom sampes whih we may enoune in ea wod senaios This pape expoes saisia eaning heoy based on ompex fuzzy andom sampes Fisy he definiion of ompex fuzzy andom vaiabe is inodued Nex he oneps and some popeies of he mahemaia expeaion and independene of ompex fuzzy andom vaiabes ae povided Seondy he oneps of anneaed enopy gowh funion and VC dimension of measuabe ompex fuzzy se vaued funions ae poposed and he bounds on he ae of unifom onvegene of eaning poess based on ompex fuzzy andom sampes ae onsued Thidy on he basis of hese bounds he idea of he ompex fuzzy suua is minimizaion pinipe is pesened Finay he onsiseny of his pinipe is poven and he bound on he asympoi ae of onvegene is deived Keywods: ompex fuzzy andom vaiabe anneaed enopy gowh funion VC dimension ompex fuzzy suua is minimizaion pinipe bound on he asympoi ae of onvegene Inoduion Saisia Leaning Theoy (SLT fo sho) poposed in 960s and fuy esabished in 990s by Vapni e a [7-9] has emeged as an ineesing and sound heoy ha suppos he deveopmen of aws of saisia eaning fo sma daa sampes I has povided effeive souions obained in pesene of sma sampes whee suh sampes ae inheeny assoiaed wih uia issues suh as ovefiing and undefiing high-dimensionaiy of assifiaion pobems exisene of muipiiy of oa minima and ohe impoan pobems enouneed in paie of mahine eaning mehods and hei ahieues suh as eg neua newos In he ae 990s SLT had beome one of he fases-gowing disipines in mahine eaning Is essene was o mae he eaning mahines wo effeivey wih he imied sampes and hen impove he geneaizaion abiiies of he eaning mahines By doing his we esabish a meaningfu heoeia famewo fo saisia eaning fo sma daa sampes Meanwhie SLT gave ise o a new aegoy of genea eaning agoihms namey Suppo Veo Mahine (SVM fo sho) A pesen he SLT and SVM onsiue ineesing eseah avenues in mahine eaning [ ] Despie he fa ha SLT has eahed a subsania eve of mauiy hee ae si a numbe of open issue as eg he deveopmen of he SLT and SVM eaized on a basis of pobabiiy measue spae and he ea-vaued andom sampes (ea numbes-vaued andom vaiabes) In ea wod senaios hee ofen ae many non-pobabiiy spaes (suh as fuzzy measue spaes [33] edibiiy measue spae [9] e) and non-ea vaued andom sampes (suh as fuzzy andom sampes [6] ompex andom sampes [36] e) * This wo was suppoed by he Naiona Naua Siene Foundaion of China (Gan No ) he Naua Siene Foundaion of Hebei Povine of China (Gan No ) a Key Sienifi Reseah Poe of Depamen of Eduaion of Hebei Povine of China (Gan No 00500D) and he Key Sienifi and Tehnia Reseah Poe of Minisy of Eduaion of China (Gan No 060) + Coesponding auho E-mai addess: zhangzm978@yahooomn Pubished by Wod Aademi Pess Wod Aademi Union

2 0 Zhiming Zhang e a: Suua Ris Minimizaion Pinipe Based on Compex Fuzzy Random Sampes To aeviae hese pobems i beomes impeaive o move fowad wih exensions and geneaizaions aong he ine of deaing wih he saisia eaning heoies esabished on non-pobabiiy spae and based on non-ea vaued andom sampes Some eseah [8-805] has been aeady eaized aong his ine Fo exampe Ha e a [8-0] geneaized he ey heoem of eaning heoy and onsued he bounds on he ae of onvegene of eaning poess of saisia eaning heoy fom pobabiiy spae o Sugeno measue spae edibiiy spae and quasi-pobabiiy spaes whee hese hee ae ypia non-pobabiiy spaes; Lin and Wang [8] onsued suppo veo mahine based on fuzzy andom sampes; Liu and Chen [0] disussed fae eogniion using oa magin-based adapive fuzzy suppo veo mahines; Jin Tang and Zhang [] onsued suppo veo mahines wih genei fuzzy feaue ansfomaion fo biomedia daa assifiaion Howeve hee has been ahe ie wo ompeed fo saisia eaning heoy based on ompex fuzzy andom sampes I is we nown ha ompex numbes onsiue a subsania and paiay eevan geneaizaion of ea numbes By he same oen sampes fomed by ompex fuzzy andom vaiabes onsiue he impoan geneaizaion of ea andom vaiabes The wo on fuzzy ompex anaysis was saed by he onep of fuzzy ompex numbes whih inodued a fis by Buey [] In he seque hee have been fuhe eseahes on fuzzy ompex anaysis [ ] Fo exampe Buey [3] disussed he diffeeniabiiy and inegabiiy of fuzzy ompex vaued funions; Zhang [37] pesened he imi heoy of he sequene of fuzzy ompex numbes and given a seies of esus abou imi heoy; Qiu Wu and Li [3] evisied he idea of fuzzy ompex anaysis in sevea diffeen ways They ooed a diffeen oneps of onvegene and eaionships beween hem and disussed he oninuiy and diffeeniaion of fuzzy ompex funions Zhang [38] poposed he oneps of measuabe ompex fuzzy se vaued funion and ompex andom vaiabe and disussed some popeies in deai A hese obsevaions ead o he onusion of eevane and appiabiiy of he geneaizaion of he SLT o ompex fuzzy andom sampes (we noe ha he fuzzy andom sampes and he ompex andom sampes ae wo speia ases) In he SLT one of he enes onen is a new induion pinipe he soaed suua is minimizaion pinipe whih is a bee induion pinipe of he eaning mahine han he empiia is minimizaion pinipe This pinipe minimizes bounds wih espe o wo faos he vaue of empiia is and he apaiy Moeove his pinipe aows us o find he funion ha ahieves he guaaneed minimum of he expeed is using a finie numbe of obsevaions Howeve in he assia saisia eaning heoy he onusions abou he suua is minimizaion pinipe wee based on ea andom sampes whih ae no eady o dea wih he saisia eaning pobems invoving ompex fuzzy andom sampes whih we may enoune in ea wod senaios This sudy fis poposes he onep of suua is minimizaion pinipe of ompex fuzzy andom sampes by ombining fuzzy ompex anaysis and SLT hen he onsiseny of he suua is minimizaion pinipe of ompex fuzzy andom sampes and asympoi bounds on he ae of onvegene ae pesened and poven The sudy wi hep ay essenia heoeia foundaions fo suppo veo mahine based on ompex fuzzy andom sampes This pape is oganized as foows Seion inodues some basi definiions and popeies whih wi be used in he sudy In Seion 3 he oneps of apaiy fo he se of measuabe ompex fuzzy se vaued funions ae poposed In he seque in Seion we give he bounds on he ae of unifom onvegene of eaning poess based on ompex fuzzy andom sampes In Seion 5 we popose he oneps of ompex fuzzy suua is minimizaion pinipe In Seion 6 we pove he onsiseny of he ompex fuzzy suua is minimizaion pinipe and onsu asympoi bound on he ae of onvegene The fina seion offes he onusions and bings pospes of poenia fuue deveopmens Peiminaies Thoughou his pape we assume ha P is a pobabiiy measue spae is he ea numbes fied and is a famiy of nonempy ompa onvex subses of Le denoe he famiy of a funions X : 0 and Le denoe he famiy of a funions X whih saisfies he foowing ondiions: () X is noma ie hee exiss x suh ha X x ; () X is uppe semi-oninuous; JIC emai fo onibuion: edio@iogu

3 Jouna of Infomaion and Compuing Siene Vo 5 (00) No pp (3) supp X x : X x 0 is ompa; () X is a onvex fuzzy se ie X x y min X x X y fo xy and 0 Fo a fuzzy se X if we define hen i foows ha X X X if and ony if X and x : X x 0 supp X 0 Theefoe X is ompeey deemined by he ineva X X X If AB Le XY and se whee hen he Hausdoff mei is defined by X is a osed bounded ineva fo eah 0 dh ( A B) max supinf x y supinf x y xa yb yb xa sup H d X Y d X Y 0 d X Y max X Y X Y Aso he nom X of fuzzy numbe X wi be defined as H 0 max 0 0 X d X X X Definiion [38] Le be a ompex numbes fied The mapping Z : 0 ompex se The u of Z is We sepaaey define Z 0 he Z Z z Zz 0 0 u of Z as he osue of he union of he is aed a fuzzy Z fo 0 Definiion [38] If XY wih membeship funions x X and yy espeivey hen is aed a ompex fuzzy se wih membeship funion zz min xx yy Z X iy whee z xiy We denoe he ass of a he ompex fuzzy ses by Espeiay if XY hen we a Z X i Z X Y X iy fo 0 and Y a bounded osed ompex fuzzy numbe Beause he -us of Z ae eanges a bounded osed ompex fuzzy numbe is aso aed a eangua fuzzy he ass of a he bounded osed ompex fuzzy ompex numbe (see Ref []) And we denoe by numbes ie Z X iy X Y Le be a famiy of nonempy ompa onvex subses of If AB hen he Hausdoff mei is defined by whee dh ( A B) max supinf dx ysupinf dx y xa yb yb xa denoes he disane beween wo ompex numbes d x y x and y Le us define a onsisen Hausdoff mei in whee Z X iy W U iv o be in he foowing fom d Z W d X iy U iv sup d X iy U iv Theoem If Z X iy W U iv 0 hen we have H dx U dy Vd Z W d X U dy V max JIC emai fo subsipion: pubishing@wauogu

4 Zhiming Zhang e a: Suua Ris Minimizaion Pinipe Based on Compex Fuzzy Random Sampes Poof: We have Beause we have Beause we have 0 d Z W d X iy U iv sup d X iy U iv 0 H sup max sup inf dz z sup inf dz z 0 z z U iv z X iy X iy zu iv sup max d X iy U iv d X iy U iv d X iy U iv d X iy U iv X U X U Y V Y V max sup max X U X U sup max Y V Y V dz W 0 0 d X U dy V dz W max 0 X U X U Y V Y V 0 0 d Z W sup max X U X U max Y V Y V sup max sup max H d X U d Y V H d Z W d X U d Y V The heoem is poven Definiion 3 [38] Le be a measuabe spae U be a univese of disouse and U be a fuzzy measuabe spae F is a fuzzy se vaued mapping fom o U If D and 0 he foowing eaion F D F D is vaid whee is a fuzzy ageba of some fuzzy subses ofu is efeeed as he empy se and F uu F u hen we say F is a measuabe fuzzy se vaued mapping fom o U Le be he ea ine is omposed of a he Boe ses of I 0 BI B I onsiss of a he Boe ses of I We denoes B= BB B B B I B Le be he ompex pane Bz onsiss of a he ompex Boe ses of We denoes C A ib A B A B = B B B B I z I I Definiion [38] If F is a measuabe fuzzy se vaued mapping fom ( A) o I B hen we say F is a measuabe ea fuzzy se vaued funion; If F is a measuabe fuzzy se vaued mapping fom B z o hen we say F is a measuabe ompex fuzzy se vaued funion Definiion 5 [38] Le P be a pobabiiy measue spae A fuzzy se vaued mapping : is aed ea fuzzy andom vaiabe if is a measuabe ea fuzzy se vaued funion ie 0 BB fo JIC emai fo onibuion: edio@iogu

5 Jouna of Infomaion and Compuing Siene Vo 5 (00) No pp B Definiion 6 [38] Le P is a pobabiiy measue spae A ompex fuzzy se vaued mapping C A ib A B : i is aed ompex fuzzy andom vaiabe if is a measuabe ompex fuzzy se vaued funion defined on P is a ea pa of Re and is an imaginay pa of ha is Im Theoem [38] i is a ompex fuzzy andom vaiabe defined on P if and ony if and ae he ea fuzzy andom vaiabes defined on P = 0 is aed inegabe if fo eah [0] ae inegabe In his ase he mahemaia expeaion of is defined in he foowing manne E dp dp dp 0 Definiion 8 [38] Le be a ompex fuzzy andom vaiabe on P We a E dp Re dpi Im dp E Re ie Im + Definiion 7 [6] A ea fuzzy andom vaiabe and he expeaion of if E Re and E Im boh exis Theoem 3 [38] If is a ea (o ompex) fuzzy andom vaiabe hen he foowing equaiies E E E E E E wheneve ; 3 E E E fo 0 ; hod ue Definiion 9 Suppose ha T is a famiy of ea fuzzy andom vaiabes andt is any se of indexes ) If fo any posiive inege n n T he ageba famiy s s n is muua independen hen we say ha T T is a ounabe se ha is T andom vaiabes is a famiy of muua independen ea fuzzy andom vaiabes If hen we say ) If fo any posiive inege n T and 0 s s n T is a sequene of independen ea fuzzy he ageba famiy s n is muua independen hen we say ha T is a famiy of evewise independen ea fuzzy andom vaiabes If T is a ounabe se ha is T hen we say T is a sequene of eve-wise independen ea fuzzy andom vaiabes 3) If 0 T is a famiy of ideniay disibued ea andom veos hen we say ha T is a famiy of eve-wise ideniay disibued ea fuzzy andom vaiabes Definiion 0 Suppose ha T is a famiy of ompex fuzzy andom vaiabes T is any se of JIC emai fo subsipion: pubishing@wauogu

6 Zhiming Zhang e a: Suua Ris Minimizaion Pinipe Based on Compex Fuzzy Random Sampes indexes and i T ) If Tis a famiy of independen (o eve-wise independen) ea fuzzy andom veos hen we say ha Tis a famiy of independen (o eve-wise independen) ompex fuzzy andom vaiabes ) If T is a famiy of eve-wise ideniay disibued ea fuzzy andom vaiabes and T T is aso a famiy of eve-wise ideniay disibued ea fuzzy andom vaiabes hen we say ha is a famiy of eve-wise ideniay disibued ompex fuzzy andom vaiabes Theoem [38] Le G be a measuabe ompex fuzzy se vaued funion defined on B z and be a ompex fuzzy andom vaiabe defined on P If PA 0 whee hen G 0 A G is aso a ompex fuzzy andom vaiabe on P Theoem 5 Suppose ha i is a sequene of eve-wise independen and evewise ideniay disibued ompex fuzzy andom vaiabes Le be oay bounded ha is hee exis A B A B suh ha A B and We aso use he noaion A A A hods ue Poof: P d E A B 0 min and B max B B The foowing inequaiy Pd E exp B A Pd E ie i Pd E d E Pd E Pd E P sup d H E 0 P sup H d E 0 Psup max E E 0 Psup max E E 0 JIC emai fo onibuion: edio@iogu

7 Jouna of Infomaion and Compuing Siene Vo 5 (00) No pp Psup max E E 0 Psup max E E 0 exp exp exp B A B A BA 3 Leaning apabiiy fo he se of measuabe ompex fuzzy se vaued funions In his seion we wi inodue oneps of apaiy of he se of measuabe ompex fuzzy se vaued funions In hese oneps anneaed enopy and gowh funion ae non onsuive and VC dimension is onsuive Given hese oneps we wi obain expessions fo he bounds of he ae of unifom onvegene of eaning poess Definiion 3 Le Z X iy be a sequene of eve-wise independen and eve-wise Q ideniay disibued ompex fuzzy andom vaiabes and e Z be a se of measuabe R QZ dp ompex fuzzy se vaued funions We a EQ Z he expeed is funiona based on ompex fuzzy andom sampes I oud be efeed o as he ompex fuzzy expeed is R emp Q Z funiona We a he expession he ompex fuzzy empiia is funiona Definiion 3 Le Z X iy be a sequene of eve-wise independen and eve-wise ideniay disibued ompex fuzzy andom vaiabes and e QZ be a se of measuabe ompex fuzzy se vaued funions be any se of index If hee exiss suh ha he eaion R R 0 inf is vaid hen we say ha R is he geaes owe bound of R 0 R R 0 inf Simiay if hee exiss suh ha he eaion inf R R emp emp is vaid hen we say ha R is he geaes owe bound of R emp inf R R emp emp emp 0 denoed by denoed by The pinipe of empiia is minimizaion an be desibed in he foowing manne Le us insead of R minimize he ompex fuzzy empiia is R Conside ha he minimum of he ompex fuzzy expeed is funiona is aained a minimizing he ompex fuzzy expeed is funiona funiona emp 0 and suppose ha he minimum of he ompex fuzzy empiia is funiona is aained a Q Z Q Z as an appoximaion of he oigina funion Q Z 0 Q Z We view he funion The pinipe of soving he is minimizaion pobem is aed he ompex fuzzy empiia is minimizaion pinipe he CFERM pinipe o be bief Definiion 33 Le Q X be a se of measuabe ea fuzzy se vaued funions and X be a ea fuzzy andom vaiabe We a he se of indiao funions QX JIC emai fo subsipion: pubishing@wauogu

8 6 Zhiming Zhang e a: Suua Ris Minimizaion Pinipe Based on Compex Fuzzy Random Sampes whee inf QX sup QX X X X u u 0 0 u 0 0 he ef ompee se of indiaos fo a se of measuabe ea fuzzy se vaued funions QX Definiion 3 Le Q X X be a se of measuabe ea fuzzy se vaued funions Le N X X X of indiaos: Le he funion be he numbe of diffeen sepaaions of veos Q X inf QX sup QX X X X 0 n H X X X N X X X be measuabe wih espe o pobabiiy P on X X X We a he quaniy X ann H n EN X X X n max X X X X G N X X X X X X by a ef ompee se he ef anneaed enopy of he se indiaos of measuabe ea fuzzy se vaued funions Definiion 35 We a he quaniy he ef gowh funion of a se of measuabe ea fuzzy se vaued funions Q X X Definiion 36 We a he maxima numbe h of veo X X X ha an be shaeed by he ef ompee se of indiaos Q X inf QX sup QX X X X he ef VC dimension of he se of measuabe ea fuzzy se vaued funions Simiay we have he foowing definiions: 0 Q X Definiion 37 Le Q X be a se of measuabe ea fuzzy se vaued funions and X be a ea fuzzy andom vaiabe We a he se of indiao funions Q X inf QX sup QX X X X 0 he igh ompee se of indiaos fo a se of measuabe ea fuzzy se vaued funions QX Le Definiion 38 Le Q X N X X X X be a se of measuabe ea fuzzy se vaued funions be he numbe of diffeen sepaaions of veos X X X by a igh ompee JIC emai fo onibuion: edio@iogu

9 Jouna of Infomaion and Compuing Siene Vo 5 (00) No pp se of indiaos: Q X inf QX sup QX X X X 0 Le he funion H X X X n N X X X be measuabe wih espe o pobabiiy We a he quaniy on P X X X X H n ann EN X X X he igh anneaed enopy of he se indiaos of measuabe ea fuzzy se vaued funions Definiion 39 We a he quaniy n max X G N X X X X X X he igh gowh funion of a se of measuabe ea fuzzy se vaued funions Q X X Definiion 30 We a he maxima numbe h of veo X X X ha an be shaeed by he ompee se of indiaos Q X inf QX sup QX X X X 0 igh VC dimension of he se of measu Q X he abe ea fuzzy se vaued funions Definiion 3 We a espeivey X X X H ann max Hann H ann X X X G max G G h max h h X he anneaed enopy he gowh funion and he VC dimension of a se of measuabe ea fuzzy se vaued fun Q X ions Definiion 3 Le Z X iy be a ompex fuzzy andom sampe and e QZ Re Q Z iim Q Z be a se of measuabe ompex fuzzy se vaued funions Suppose ha he anneaed enopy he gowh funion and VC dimension of he se of measuabe ompex fuzzy se vaued funions QZ ae Z espeivey H Z ann G and hz he anneaed enopy he gowh funion and VC dimension of he ea pas Re Q Z of Q Z ae espeivey X H ann G X and hx and he anneaed enopy he gowh funion and VC dimension of he imaginay pas Im Q Z of QZ Y ae espeivey H Y ann G and hy We define ha Z X Y H ann max Hann H ann JIC emai fo subsipion: pubishing@wauogu

10 8 Zhiming Zhang e a: Suua Ris Minimizaion Pinipe Based on Compex Fuzzy Random Sampes Z Theoem 3 The foowing inequaiies X max G Y G G X X ann h max h h Z X Y n if h X H G h n if X X h h X n if h Y H G h n if Y Y h h Y Y Y ann n if h Z H G h n if h Z Z h Z Z Z ann hod ue The poof goes he same way as in he poof of Theoem 3 of Ref [8] wih he aid of he Definiions 33-3 and wi be omied Rema 3 In assia saisia eaning heoy he oneps and some impoan popeies (see Ref [7 8]) of he anneaed enopy he gowh funion and VC dimension of ea measuabe is Q funions z ae he speia ases of his sudy Deeminaion of bounds on he ae of unifom onvegene of eaning poess In he SLT he impoan onusions abou he eaion beween he empiia is and paia is ae expessed in he fom of he bounds of geneaizaion They beome essenia when anayzing he apaiy of eaning mahines and deveoping new eaning agoihms The bounds on he aes of unifom onvegene of eaning poess ae impoan omponens of he bounds of geneaizaion In his seion we disuss he bounds on he ae of unifom onvegene of eaning poess based on ompex fuzzy andom sampes To ahieve his goa we need empoy he oneps poposed in Seion 3 We wi obain expessions fo he bounds of he ae of unifom onvegene of eaning poess hough he foowing wo ases Z H ann n 8 In his seion we use he noaion Q Z is a se of oay bounded measuabe ompex fuzzy se vaued funions Theoem Suppose ha Q Z is a se of oay bounded measuabe ompex fuzzy se vaued funions ha is hee exis A B AB suh ha A Re Q Z Band A Im Q Z B 0 We aso use he noaion A min A A and B max B B hods ue Poof: The foowing inequaiy Z Hann P sup dr R emp 8exp B A () JIC emai fo onibuion: edio@iogu

11 Jouna of Infomaion and Compuing Siene Vo 5 (00) No pp sup sup emp P d R R P de Q Z Q Z P d E Q Z Q Z sup Re Re P sup d EIm QZ Im Q Z Psup sup dh E Re Q Z Re Q Z 0 Psup sup dh E Im Q Z Im Q Z 0 P max sup sup E Re Q Z Re Q Z sup sup E Re Q Z Re Q Z 0 0 Z P max sup sup E Im Q Z Im Q Z sup sup E Im Q Z Im Q 0 0 X ann Y ann H H exp exp B A B A Z H ann 8exp B A Theoem Fo a funions in a se of measuabe ompex fuzzy se vaued funions Q Z whih saisfy he ondiions fo heoem we have () he foowing inequaiy hods ue wih pobabiiy () he inequaiy emp d R R B A () n R R 0 B A B A (3) hods wih pobabiiy a eas Poof Le us ewie he inequaiy () in a eain equivaen fom To do his we inodue a posiive vaue 0 and he equaiy 8exp whih we sove wih espe o We obain Z H ann B A B A Now he asseion omes in he foowing equivaen fom: Wih pobabiiy is vaid simuaneousy fo a funions in he se QZ emp d R R B A he inequaiy JIC emai fo subsipion: pubishing@wauogu

12 30 Le Q Z 0 funiona R funiona Zhiming Zhang e a: Suua Ris Minimizaion Pinipe Based on Compex Fuzzy Random Sampes and e Q Z R emp funion QZ is vaid be a funion fom he se of funions ha minimizes he ompex fuzzy expeed is be a funion fom his se ha minimizes he ompex fuzzy empiia is Sine he inequaiy is ue fo a funions in he se i is ue as we fo he Thus wih pobabiiy a eas he foowing inequaiies Fo he funion QZ 0 hods ue wih pobabiiy a eas ae R R B A () emp whih minimizes R aoding o Theoem 5 he foowing eaionship n R emp 0 R 0 B A (5) Aoding o () and (5) we onude ha wih pobabiiy a eas he inequaiies R R R R R R R R emp emp emp 0 emp 0 0 saisfied wih pobabiiy a eas n B A B A Q Z is a se of oay bounded nonnegaive measuabe ompex fuzzy se vaued funions Theoem 3 Suppose Q Z is a se of oay bounded nonnegaive measuabe ompex ha fuzzy se vaued funions ha is hee exis B B suh ha Q Z B and 0 Im Q Z 0 Re B 0 We aso use he noaion B max B B The foowing inequaiy hods ue Poof: Z Hann 0 d R R emp P sup 8exp R 6B d R P sup R R dere QZ Re QZ deim QZ Im QZ P sup P sup R R emp E Re Q Z Re Q Z E Re Q Z Re Q Z P max sup sup sup sup 0 0 E Re QZ E Re QZ (6) JIC emai fo onibuion: edio@iogu

13 Jouna of Infomaion and Compuing Siene Vo 5 (00) No pp E Im Q Z Im Q Z E Im Q Z Im Q Z P max sup sup sup sup 0 0 E Im QZ E Im QZ Y exp exp 6B 6 B X H H ann ann 8exp Theoem Fo a funions in a se of measuabe ompex Q Z whih saisfy he ondiions fo heoem 3 we have () he foowing inequaiy hods ue wih pobabiiy () he inequaiy hods wih pobabiiy a eas Z H ann 6B fuzzy se vaued funions R emp R R emp B (7) B R emp n R R 0 B B (8) B Poof: We an pove his heoem in he same way as done in Theoem and wi be omied Rema The bounds desibed by he inequaiies () and (7) depend on he pobabiiy disibuion P One an deive boh disibuion-fee non onsuive bounds and disibuion-fee onsuive bounds To obain hese bounds i is suffiien in he inequaiies () and ( 7) o use he expession Z G n 8 (his expession povides disibuion-fee non onsuive bounds) o o use he expession h n n Z h Z 8 (his expession povides disibuion-fee onsuive bounds) 5 Suua is minimizaion pinipe based on ompex fuzzy andom sampes Aoding o Theoem () we obained ha wih pobabiiy a eas simuaneousy fo a funions in a se of measuabe ompex fuzzy se vaued funions Q Z wih finie VC dimension he inequaiy R R B A emp h n n Z h 8 Z hods ue Aoding o Theoem () We obained aso ha wih pobabiiy a eas simuaneousy fo a Q Z he inequaiy funions in a se of measuabe ompex fuzzy se vaued funions (5) JIC emai fo subsipion: pubishing@wauogu

14 3 Zhiming Zhang e a: Suua Ris Minimizaion Pinipe Based on Compex Fuzzy Random Sampes R emp R R emp B (5) h n n Z h Z 8 B hods ue Aoding o inequaiies (5) and (5) he uppe bound on he is deeases wih deeasing he vaue of empiia is This is he eason why he pinipe of ompex fuzzy empiia is minimizaion ofen gives good esus fo age sampe size Howeve if aio of he numbe of he aining paens o he VC dimension of he se of funions of he eaning mahines is sma a sma vaue of he ompex fuzzy empiia is R emp does no guaanee a sma vaue of he ompex fuzzy aua is In his ase o minimize he ompex fuzzy aua is R one has o minimize he igh-hand side of inequaiy (5) (o (5)) simuaneousy ove boh ems Noe ha he fis em in inequaiy (5) depends on a speifi funion of he se of funions whie fo a fixed numbe of obsevaions he seond em depends mainy on he VC dimension of he whoe se of measuabe ompex fuzzy se vaued funions Theefoe o minimize he igh-hand side of he bound of is (5) (o (5)) simuaneousy ove boh ems one has o mae he VC dimension a onoing vaiabe To do his we onside he foowing sheme Le us impose he suue on he se S of measuabe ompex fuzzy se vaued funions Q Z Conside he se of nesed subses of funions S S S n whee S = { Q ( Z a ) a ÎL * } and S = S Conside admissibe suues-he suues ha saisfy he foowing popeies: Any eemen S of suue has a finie VC dimension h Any eemen S of he suue (53) onains eihe (i) a se of oay bounded measuabe ompex fuzzy se vaued funions QZ ( a ) aîl saisfying he foowing: A Re QZ B and A Im Q Z B 0 whee A A B B A min A A B max B B We aso use he noaion and Z (ii) o a se of oay bounded nonnegaive measuabe ompex fuzzy se vaued funions saisfying he foowing: Q ( Z a ) aîl Q Z B 0 Re and 0 Im Q Z B 0 whee B B We aso use he noaion B max B B S S Q Z " e> hee exiss a funion QZa ( * ) Î S * suh ha d( Q ( ) ( ) ) * 3 The se is eveywhee dense in he se S ( ) (53) in he Hausdoff mei d ha is " Q Z a Î S 0 ò Z a Q Z a * dp< e Noe ha in view of he suue (53) he foowing asseions ae ue: The sequene of vaues of VC dimension h Z fo he eemens S of he suue is nondeeasing wih ineasing : n h h h Z Z Z The sequene of vaues of he bounds B fo he eemens S of he suue is nondeeasing wih JIC emai fo onibuion: edio@iogu

15 Jouna of Infomaion and Compuing Siene Vo 5 (00) No pp ineasing : B B B n Denoe by Q Z he funion ha minimizes he ompex fuzzy empiia is in he se of funions S Fo a given se of obsevaions Z Z he SRM mehod hooses he eemen S of he suue fo whih he smaes bound on he is (he smaes guaaneed is) is ahieved Theefoe he idea of he suua is minimizaion pinipe of ompex fuzzy andom sampes is he foowing: To povide he given se of funions wih an admissibe suue and hen o find he funion ha minimizes guaaneed is ove given eemens of he suue To sess he impoane of hoosing he eemen of he suue ha possesses an appopiae apaiy we a his pinipe he ompex fuzzy suua is minimizaion pinipe of saisia eaning heoy he CFSRM pinipe o be bief The CFSRM pinipe maes a ompomise beween he auay of appoximaion of he aining daa and he ompexiy on he se of appoximaed funions The ompex fuzzy empiia is is deeased wih he ineased of he index of eemen of he suue whie he onfidene ineva is ineased The smaes bound of he is is ahieved on some appopiae eemen of he suue 6 Consiseny of he ompex fuzzy suua is minimizaion pinipe and asympoi bounds on he ae of onvegene In his seion we anayze asympoi popeies of he CFSRM pinipe Hee we answe wo quesions: Is he CFSRM pinipe onsisen? Do he iss fo he funions hosen aoding o his pinipe onvege o he smaes possibe is fo he se S wih ineasing amoun of obsevaions?wha is he bound on he (asympoi) ae of onvegene? Le S be a se of measuabe ompex fuzzy se vaued funions and e be an admissibe suue Conside now he ase whee he suue onains an infinie numbe of eemens We denoe by Q Z he measuabe ompex fuzzy se vaued funion whih minimizes he ompex fuzzy empiia is ove he funions in he se S and denoe by Q Z 0 he measuabe ompex fuzzy se vaued funion whih minimizes he ompex fuzzy expeed is ove he funions in he se Ss ; we denoe aso by Q Z 0 he measuabe ompex fuzzy se vaued funion whih minimizes he ompex fuzzy expeed is ove he se of funion S In he foowing ex we pove he onsiseny of he CFSRM pinipe Conside he a pioi ue n nfo hoosing he numbe of eemen of he suue depending on he numbe of given sampes n Theoem 6 The ue n n povides appoximaions Q Z fo whih he sequene of iss R n onveges as ends o infiniy o he smaes is: wih asympoi ae of onvegene whee R R 0 inf V n n n Z B h n n (6) n 0 R R (6) 0 0 n (ha is he equaiy P im supv R R hods ue) if n Z B h n n 0 n (63) JIC emai fo subsipion: pubishing@wauogu

16 3 Zhiming Zhang e a: Suua Ris Minimizaion Pinipe Based on Compex Fuzzy Random Sampes Poof: Conside a suue wih eemens dimension We have ha he foowing inequaiy S onaining oay bounded funions wih he finie VC h n n Z n h 8 Z R R 0 B A B A hods ue wih pobabiiy a eas Le ha is Then wih pobabiiy he inequaiy is vaid Fo any eemens h n n 8 Z h Z R R 0 B A B A wih pobabiiy a eas he addiive bound S n h n n 8 Z h Z R R 0 B A is vaid Then wih pobabiiy he inequaiy n hods whee Sine posiive inege n h n n 8 Z n n h Z n R R 0 B A (6) n n n n n 0 0 R R S S eveywhee dense in S Q Z fo Q ( Z a 0 ) Î S fo any e > 0 hee exiss a K suh ha QZ ( a *) Î SK and d Q( Z ò ) R 0 ( 0 a Q (Z * a )) dp< e We have Re Im Re 0 Im 0 Re 0 Re Im Im 0 d R d Q Z dp i Q Z dp Q Z dp i Q Z dp d Q Z dp Q Z dp d Q Z dp Q Z dp Aoding o Theoem 33 of Ref [7] we obain When Re 0 Re Im Im 0 Re Re 0 Im Im 0 dqz 0 QZ dp dq Z 0 Q Z dp d Q Z dp Q Z dp d Q Z dp Q Z dp d Q Z Q Z dp d Q Z Q Z dp n () ³ K he foowing eaionships JIC emai fo onibuion: edio@iogu

17 Jouna of Infomaion and Compuing Siene Vo 5 (00) No pp R R 0 dr R 0 n K im im R 0 R 0 im R n 0 R im im 0 ae saisfied Theefoe he ondiion deemines onvegene o zeo Denoe n Z B h n n 0 n h n n 8 Z n h Z V B A n n n Le we ewie he asseion (6) in he fom Sine n PV R R 0 0 n 0 PV R R 0 0 n aoding o he ooay fom he Boe-Canei emma (see Ref [8]) one an asse ha he inequaiy imv R n R 0 is vaid wih pobabiiy one The nex heoem is devoed o asympoi popeies of he ompex fuzzy suua is minimizaion pinipe Theoem 6 If he suue is suh ha B n n hen fo any disibuion funion he CFSRM mehod povides onvegene o he bes possibe souion wih pobabiiy one (ie he CFSRM mehod is Q Z beongs o some eemen of univesay songy onsisen) Moeove if he opiona souion 0 * he suue Q Z 0 Q Z and B n n foowing asympoi ae of onvegene: V O hen using he CFSRM mehod one ahieves he Poof To avoid hoosing he minimum of funiona (5) ove he infinie numbe of eemens of he suue we inodue one addiiona onsain on he CFSRM mehod: we wi hoose he minimum fom he fis eemens of he suue whee is equa o he numbe of obsevaions Theefoe we appoximae he souion by funion Q Z Q Z minimizing whih among funions empiia is on oesponding eemens S of he suue povide he smaes guaaneed (wih pobabiiy ) is: S * JIC emai fo subsipion: pubishing@wauogu

18 36 Zhiming Zhang e a: Suua Ris Minimizaion Pinipe Based on Compex Fuzzy Random Sampes Denoe by deomposiion h R emp min R emp B A he paamee ha minimizes guaaneed is R Z emp n n 8 h Z using obsevaions Conside he R R R R R R 0 emp emp 0 Fo he fis em of his deomposiion we have emp emp P R R P R R P R R B emp h exp B Z n n 8 h Z h n n 8 Z h Z h h n n 8 Z Z e h Z 3 exp h B Z e x p exp exp B B whee we ae ino aoun ha B Using he Boe-Canei emma we obain ha fis summand of he deomposiion onveges amos suey o he non-posiive vaue Now onside he seond em of he deomposiion Sine S S eveywhee dense in S Q Z fo evey hee exiss an eemens of he suue suh ha R S 0 R 0 Theefoe we wi pove ha he seond em in he deomposiion does no exeed zeo if we show ha wih pobabiiy one Noe ha fo any hee exiss R 0 im min R 0 suh ha fo any 0 emp 0 Fo 0 we have B h n n 8 Z h Z (65) JIC emai fo onibuion: edio@iogu

19 Jouna of Infomaion and Compuing Siene Vo 5 (00) No pp emp 0 emp 0 P R R P R R min P R emp R 0 B h n n 8 Z h Z P R emp R 0 Psup R R emp h n Z h Z h Z h Z e e 6exp 6 exp 6 exp 8B h 8B h 8 Z Z Again appying he Boe-Canei emma one onudes ha seond em of he deomposiion onveges amos suey o he nonposiive vaue Sine he sum of wo ems is nonnegaive we obain am R o R 0 This poves he fis pa of he heoem os sue onvegene To pove he seond pa noe ha when he opima souion beongs o one of he eemens of he suue S he equaiy R R hods ue Com bining bounds fo boh ems one obains ha 0 0 fo saisfying (65) he foowing inequaiies ae vaid: 0 P R R P R Remp P Remp R 0 h Z exp 6 exp 6 3 e h Z Fom his inequaiy we obain he ae of onvegene: 7 Conusions V n O Consideing he exisene and he signifiane of ompex fuzzy andom vaiabes in ea wod his pape poposes he oneps of apaiy of he se of measuabe ompex fuzzy se vaued funions and he suua is minimizaion pinipe based on ompex fuzzy andom sampes Fuhemoe he onsiseny of he ompex fuzzy suua is minimizaion pinipe is poven and bound on he asympoi ae of onvegene is pesened Aogehe hese findings have aid he foundaion fo fuhe eseah in saisia eaning heoy invoving ompex fuzzy andom sampes Fuhe invesigaions migh fous on some appied aspes suh as eg ompex fuzzy suppo veo mahines 8 Anowedgemens The auho woud ie o han he efeees fo hei vauabe ommens and suggesions 9 Refeenes [] M Bunao R Baii Saisia eaning heoy fo oaion fingepining in wieess LANs Compue Newos 005 7: [] J J Buey Fuzzy ompex numbes Fuzzy Ses and Sysems : JIC emai fo subsipion: pubishing@wauogu

20 38 Zhiming Zhang e a: Suua Ris Minimizaion Pinipe Based on Compex Fuzzy Random Sampes [3] J J Buey Fuzzy ompex anaysis II: inegaion Fuzzy Ses and Sysems 99 9: 7-79 [] J J Buey Y X Qu Fuzzy ompex anaysis I: diffeeniaion Fuzzy Ses and Sysems 99 : 69-8 [5] J L Caso L D Foes-Hidago C J Manas J M Puhe Exaion of fuzzy ues fom suppo veo mahines Fuzzy Ses and Sysems : [6] A Ceiyimaz I B Tusen Fuzzy funions wih suppo veo mahines Infomaion Sienes : [7] T Evgeniou T Poggio M Poni e a Reguaizaion and saisia eaning heoy fo daa anaysis Compuaiona Saisis & Daa Anaysis 00 38: -3 [8] M H Ha Y C Bai P Wang e a The ey heoem and he bounds on he ae of unifom onvegene of saisia eaning heoy on a edibiiy spae Advanes in Fuzzy Ses and Sysems 006 : 3-7 [9] M H Ha Z F Feng S J Song L Q Gao The ey heoem and he bounds on he ae of unifom onvegene of saisia eaning heoy on quasi-pobabiiy spaes Chinese Jouna of Compues 008 3: [0] M H Ha Y Li J Li D Z Tian The ey heoem and he bounds on he ae of unifom onvegene of eaning heoy on Sugeno measue spae Siene in China: Seies F 006 9: - [] M H Ha J Tian The heoeia foundaions of saisia eaning heoy based on fuzzy numbe sampes Infomaion Sienes : [] C L Huang MC Chen CJ Wang Cedi soing wih a daa mining appoah based on suppo veo mahines Expe Sysem wih Appiaions : [3] C Hwang D H Hong K H Seo Suppo veo ineva egession mahine fo isp inpu and oupu daa Fuzzy Ses and Sysems : -5 [] B Jin Y C Tang Y Q Zhang Suppo veo mahines wih genei fuzzy feaue ansfomaion fo biomedia daa assifiaion Infomaion Sienes : [5] T Kiuhi S Abe Compaison beween eo oeing oupu odes and fuzzy suppo veo mahines Paen Reogniion Lees 005 6: [6] Y K Kim A song aw of age numbes fo fuzzy andom vaiabes Fuzzy Ses and Sysems 000 : [7] Y K Kim B M Ghi Inegas of fuzzy-numbe-vaued funions Fuzzy Ses and Sysems : 3- [8] C F Lin S D Wang Fuzzy suppo veo mahines IEEE Tansaions on Neua Newos 00 3: 6-7 [9] B D Liu Y K Liu Expeed vaue of fuzzy vaiabe and fuzzy expeed vaue modes IEEE Tansaions on Fuzzy Sysems 00 0: 5-50 [0] Y H Liu Y T Chen Fae eogniion using oa magin-based adapive fuzzy suppo veo mahines IEEE Tansaions on Neua Newos 007 8: 78-9 [] D Qiu L Shu Noes on On he esudy of fuzzy ompex anaysis: Pa I and Pa II Fuzzy Ses and Sysems : [] J Q Qiu C X Wu F C Li On he esudy of fuzzy ompex anaysis: Pa I The sequene and seies of fuzzy ompex numbes and hei onvegenes Fuzzy Ses and Sysems 000 5: 5-50 [3] J Q Qiu C X Wu F C Li On he esudy of fuzzy ompex anaysis: Pa II The oninuiy and diffeeniaion of fuzzy ompex funions Fuzzy Ses and Sysems 00 0: 57-5 [] C Su C H Yang Feaue seeion fo he SVM: an appiaion o hypeension diagnosis Expe Sysems wih Appiaions 008 3: [5] D Z Tian Z M Zhang M H Ha The ey heoem of ompex saisia eaning heoy Dynamis of Coninuous Disee and Impusive Sysems Seies A: Mahemaia Anaysis 007 (S): 6-50 [6] J J Tsong Hybid appoah of seeing hype-paamees of suppo veo mahine fo egession IEEE Tansaions on Sysems Man and Cybeneis Pa B: Cybeneis : [7] V N Vapni The Naue of Saisia Leaning Theoy New Yo: Spinge-Veag 995 [8] V N Vapni Saisia Leaning Theoy New Yo: A Wiey-Inesiene Pubiaion 998 [9] V N Vapni An oveview of saisia eaning heoy IEEE Tansaions on Neua Newos 999 0: [30] M Vidyasaga Saisia eaning heoy and andomized agoihms fo ono IEEE Cono Sysems Magazine 998 : [3] G J Wang X P Li Geneaized Lebesgue inegas of fuzzy ompex vaued funions Fuzzy Ses and Sysems 00 7: [3] Y Q Wang S Y Wang K K Lai A new fuzzy suppo veo mahine o evauae edi is IEEE Tansaions on Fuzzy Sysems 005 3: JIC emai fo onibuion: edio@iogu

21 Jouna of Infomaion and Compuing Siene Vo 5 (00) No pp [33] Z Y Wang J K Geoge Fuzzy Measue Theoy New Yo: Penum Pess 99 [3] H Wehse Z Dui F Y Li V Cheassy Moion esimaion using saisia eaning heoy IEEE Tansaions on Paen Anaysis and Mahine Ineigene 00 6: [35] C X Wu J Q Qiu Some emas fo fuzzy ompex anaysis Fuzzy Ses and Sysems : 3-38 [36] S J Yan J X Wang Q X Liu Foundaions of Pobabiiy Theoy Beiing: Siene Pess 999 [37] G Q Zhang Fuzzy imi heoy of fuzzy ompex numbes Fuzzy Ses and Sysems 998 6: 7-35 [38] Y Zhang G Y Wang Fuzzy Random Powe Sysem Theoy Beiing: Siene Pess 993 JIC emai fo subsipion: pubishing@wauogu

22 0 Zhiming Zhang e a: Suua Ris Minimizaion Pinipe Based on Compex Fuzzy Random Sampes JIC emai fo onibuion: edio@iogu