A Novel Hybrid Method for Learning Bayesian Network

Transcription

1 A Noel Hybrd Mehod for Learnn Bayesan Nework Wan Chun-Fen *, Lu Ku Dearmen of Mahemacs, Henan Normal Unersy, Xnxan, 4537, PR Chna * Corresondn auhor Tel: ; emal: wanchunfen1@16com Manuscr submed February, 14; acceed March 9, 15 do: 11776/jc Absrac: Ths aer resens a new hybrd aroach for learnn Bayesan neworks (BNs) based on arfcal bee colony alorhm and arcle swarm omzaon Frsly, an unconsraned omzaon roblem s esablshed, whch can rode a smaller search sace Secondly, he defnon and encodn of he basc mahemacal elemens of our alorhm are en, and he basc oeraons are desned, whch rode uaranee of conerence Thrdly, from a known ornal Bayesan nework wh robablsc loc samln, full samles for he rann se and esn se are eneraed, and hen he srucure of Bayesan nework s learned from comlee rann se by usn our mehod The smulaon exermenal resuls show ha our mehod s effece Key words: Bayesan nework srucure learnn, ABC, PSO, unconsraned omzaon 1 Inroducon Wh he communy of arfcal nellence, Bayesan neworks(bn) s an moran model, and also a owerful formalsm o model he uncerany knowlede n racce [1] Snce he sandard mehod o consruc BN s a labor-nense rocess of elcn knowlede from exers, he aroach o learn BN from daa has become an ncreasnly ace area of research Unforunaely, hh comuaonal requremens and he lack of ower []-[6] The learnn ask of he Bayesan neworks can be decomosed no wo subasks whch are he learnn of rah srucure and arameers Srucure learnn s o denfy he ooloy of he nework, and arameer learnn s o deermne he condonal robables for a en nework ooloy Ths aer focus on srucure learnn From he reew of soln mechansm, hese mehods for learnn BN can be classfed no wo caeores: he deendency analyss aroaches and he score-and-search aroaches [7]-[1] The frs caeory s based on consrans, whch oses he learnn rocess as a consran sasfacon roblem, and hen consrucs a nework srucure by esn he condonal ndeendence relaons The second aroach oses he learnn roblem as a srucure omzaon roblem, e uses a score merc o ealuae eery canddae nework srucure, and hen fnds a nework srucure wh he bes score The score used o ealuae he nework could be dered from nformaon heory, Bayesan sascs and MDL rncle Thouh he mlemen of he former aroach s relaely smle, he comuaons for hher-order esns are comlex and rresonsble Moreoer, he recson of learnn a model s hard o ensure, hus he score-and-search aroach radually becomes a oular aroach for learnn Bayesan nework The alorhm roosed n hs aer belons o he score-and-search aroach Snce searchn he bes nework srucure s an NP-hard roblem, he roblem s usually aroached by 13 Volume 1, Number, March 15

2 usn heursc mehods, such as reedy hll-clmbn, smulaed annealn, enec alorhm and PSO alorhm[11], [1] Amon hese heursc mehods, PSO hae been nensely researched and roed ben effece n learnn Bayesan nework Howeer, PSO alorhm has wo drawbacks n learnn Bayesan nework whch are s exense comuaonal cos and remaure conerence Recenly, ABC alorhm has been successfully aled n many areas [13] Karaboa and Basurk used ABC alorhm o omze a lare se of numercal es funcon Comarn wh PSO alorhm, dfferenal eoluon alorhm and eoluon sraees, ABCs adanaes no only le n s easy mlemenaon and few arameers o adjus, bu also le n s sron search caably n he roblem sace and he ably of quck dscoery of omal soluon Thouh ABC alorhm has many adanaes, here exs fewer work use o learn Bayesan nework By our knowlede, he aer [14] s he only aer o learn Bayesan nework by usn ABC alorhm In hs aer, we roose a new hybrd mehod for learnn Bayesan nework based on PSO and ABC alorhms In hs alorhm, he mechansm of PSO, such as he moemen of arcle ec, wll be used by emloyed bee, onlooker bee and scou bee o fnd beer food source In a number of exermens, comuaonal resuls on benchmark daases demonsrae he effeceness of our aroach The res of he aer s oranzed as follows The conce of BNs and srucural learnn s nroduced n Secon Then, we e a bref nroducon on ABC alorhm, and dscuss he deal of roosed hybrd alorhm n Secon 3 In Secon 4, some numercal resuls are reored o show he effcency of our alorhm Fnally, we conclude he aer n Secon5 Bayesan Nework Srucure Learnn 1 Bayesan Neworks Bayesan nework (BN) s a dreced acyclc rah(dag), whch can be denoed as a rle rou ( X, A, ), where X, A defnes a dreced acyclc rah srucure G:X s he se of nodes, x X reresens a random arable n a secal doman; A s a se of dreced arcs, robably dsrbuon of x en he aren se condonal robably of node en he sae of s arens a j A descrbes a drec x of he arable x P( x x ) s he x As he rah srucure G qualaely characerzes he ndeendence relaonshs amon random arables, and hese condonal robably dsrbuons quanfy he srenh of deendences beween each node and s arens nodes, hus, he jon robably can be wren as follows: P( x, x,, xn ) P( x x ) (1) 1 1 Gen daase D, he objece of srucure learnn s o denfy he bes nework srucure G ha bes maches D For search-and-scorn aroaches, how oodness he srucure maches daase s measured by adoed scorn merc The mos common scorn aroach o ealuae srucure s by he oseror robably of he srucure en he daa The oseror robably of a canddae Bayesan nework srucure BN can be obaned by alyn Bayes rules: n P( D Bs ) P( Bs ) P( Bs D) () P( D) where P D B ) s he lkelhood of he daa en he BN srucure, and P B ) s he ror robably of ( s he srucure BN, and P (D) s he robably of he obsered daa D Snce P (D) s no deenden on he srucure, can be nored when ryn o fnd he bes scorn funcon ( s 131 Volume 1, Number, March 15

3 The roblem s now reduced o fnd he srucure wh he maxmum lkelhood P D B ) In oher word, ( s en a srucure, hese srucure are ealuaed accordn o how robable s ha he daa are eneraed from he srucure Because of he decomoson characersc of Bayesan nework as shown n (1), scorn merc commonly used such as Bayesan score, BIC score and MDL score could be decomosed no summaon of sunscore of each arable For examle, k score s defned as follows: where x en he saes of her arens xx x : Score ( G) Score( x ) (3) x n q r ( r 1)! Score( G) (lo( lo( Njk!)) (4) ( N r 1)! r denoes each arable confuraons for he arables n 1 j1 x has j k1 r ossble alue assnmens; x relae o D ; x are nsanaed o he j -h confuraon BIC score s e as follows: q s he number of ossble N jk $ s he number of cases n D where x and n q r N n jk q ( r 1) Score( G) Njk lo lon (5) N 1 j1 k1 If a score merc follows (3), s decomosable For any decomosable score mercs, s easy o see ha, he bes nework srucure eery arable x bes j 1 G could be obaned by searchn he bes conbanon of aren for In hs aer, he BIC score wll be used o benchmark he mee-heursc omzaon sraees 3 Hybrd Alorhm for BN Srucure Learnn Ths secon es a hybrd alorhm based on he ABC alorhm and PSO alorhm In hs alorhm, emloyed bee, onlooker bee and scou bee use he way of PSO o deermne food source Snce he adanaes of wo alorhms are used, he new alorhm has a ery ood characerscs, e can fnd he omal Bayesan nework srucure quckly In he follown, we e he deals of our alorhm 31 Classcal ABC Alorhm In ABC alorhm, he oson of a food source reresens a ossble soluon o he omzaon roblem and he necar amoun of a food source denoes he qualy (fness) of he assocaed soluon The number of he emloyed bees andhe onlooker bees s equal o he number of soluons A he ben, ABC alorhm eneraes he nal oulaon of SN soluons (food source osons)randomly Suose ha each soluon x ( 1,,, SN) s a n -dmensonal ecor Here, n s he number of omzaon arameers Afer nalzaon, he oulaon of he osons (soluons) s subjec o reeaed cycles of he search rocesses of he emloyed bees, he onlooker bees and he scou bees An emloyed bee roduces a modfcaon on he oson (soluon) n her memory deendn on he local nformaon (sual nformaon) and ess he necar amoun (fness alue) of he new source (new soluon) If he necar amoun of he new one s hher han ha of he reous one, he bee memorzes he new oson and fores he old one Oherwse she kees he oson of he reous one n her memory Afer all emloyed bees comlee he search rocess, hey share he necar nformaon of he food sources and her oson nformaon wh he onlooker bees An onlooker bee ealuaes he necar nformaon aken from all emloyed bees and chooses a x 13 Volume 1, Number, March 15

4 food source wh a robably relaed o s necar amoun As n he case of he emloyed bee, she roduces a modfcaon on he oson n her memory and checks he necar amoun of he canddae source If he necar s hher han ha of he reous one, he bee memorzes he new oson and fores he old one The man ses of he alorhm are en as below: Se 1 Inalze Poulaon Se Whle so creron s no me do Se 3 Place he emloyed bees on her food sources Se 4 Place he onlooker bees on he food sources deendn on her necar amouns Se 5 Send he scous o he search area for dscoern new food sources Se 6 Memorze he bes food source found so far Se 7 End whle In ABC alorhm, each cycle of he search consss of hree ses: sendn he emloyed bees ono her food sources and ealuan her necar amouns; afer sharn he necar nformaon of food sources, he selecon of food source reons by he onlookers and ealuan he necar amoun of he food sources; deermnn he scou bees and hen sendn hem randomly ono ossble new food sources 3 The Mechansm of PSO In our alorhm, for deermnn food sources for emloyed bees, onlooker bees and scou bees, he mechansm of PSO wll be adoed The deals of he mechansm are en below o enerae beer nal food sources, we frs consruc an unconsraned omzaon roblem, s omal soluon can be used o reduce he searchn sace The deal s exlaned below As we all known, a Bayesan nework can be reresened by an adjacency marx, so, n hs aer, adjacency marx wll be used o reresen food source (soluon or canddae Bayesan nework) n ABC alorhm For examle, he adjacency marx of he Bayesan nework n F 1 s F 1 Four nodes nework Thus, for conssence wh he defnon of food source of ABC alorhm, he oson of a arcle n PSO also be reresened by a marx Suose ha a se, he arcle s denoed by G, n hs aer, he bes reous arcle denoed by G,, and he bes reous oson amon arcle swarm s denoed by In PSO alorhm, he elocy s used o moe he arcle and measure he dfference beween wo osons In order o conenen oeraon, he elocy of a arcle s also defned as an n n marx Here he elocy marx for arcle a se s denoed by, The elemen of elocy marx n row and column j s denoed by j, where { 1,,1} when he elocy s aled o a oson, j G 133 Volume 1, Number, March 15

5 1 means ha he ede from node o node j n he DAG would be remoed, means ha j he corresondn ede would reman unchaned, and j 1 means ha he ede from node o node j would be added Noe ha n DAG, an ede from nodeo self s forbdden, hus Wh he defnons of "oson" and "elocy" saed aboe, we e he oher oeraons: moe ( oson, elocy ) and subracon ( oson, oson ) When a oson G s moed o The defnon of he oeraon moe (G, ) j If j, when 1, hen added; oherwse, would make j j j 1 j G by a elocy G s en:, he elemen 1, j, j 1 or j 1, j,, j, j or j 1, j 1, 1, j 1, j 1,, j, j 1 j j s chaned accordn o j j 1, whch means ha he ede from node node o node j would be If If 1, 1 would make hs ede o be remoed, and or 1 j j j In addon, n our alorhm, an oeraon add(,, ) s used o combne he hree endences for mon a arcle Here,, a b c new a s ornal elocy of a arcle b s elocy resuled from subracon( G, G ) by whch he arcle would moe oward s bes reous oson c s resuled from subracon( G, G ) by whch he arcle would moe o lobal bes reous oson amon he arcle swarm, By he decomosable score merc, a nework G s an alomeraon of he arens ecor ( 1,, n) For ealuan Bayesan nework srucures, a lle chane n ( 1,, n) may cause score( ) chaned realy When combnn a, b and kee beer srucure feaures of G, and c he elocy column ecor G should be aken as he basc u o Suose a [ a,1, a,,, a, n], b [ b,1, b,,, b, n], c [ c,1, c,,, c, n], and [,,, ], he oeraon add(,, ) s defned new new,1 new, new, n subjec o r r r 1 a b c a b c new a,, rand ra, new, b,, rand rb, c,, rand rc, I brns a roblem ha how o assn he alues o he robables reasonably whch are shown n (6) An emrcal heursc s aken n hs aer In nal hase, arcles mon o search new area And durn he searchn, j (6) r a could be se a larer alue o make r b and ha he nehbor areas of bes reous osons could be searched more nensely 33 Alorhm Saemens r c should be ncreased such 134 Volume 1, Number, March 15

6 Gen he mahemacal objecs and oeraons defned aboe, we roose he hybrd alorhm, whch s named afer "ABC-PSO-BN" In hs alorhm, for mron he conerence, we consruc and sole an unconsraned omzaon roblem by adon he mehod of aer [15], and oban an undrec rah Based on hs undrec rah, he nal food source are eneraed randomly Ths rocess can reduce he search sace of our alorhm realy The seudo code of ABC-PSO-BN s shown as follows: Alorhm ABC-PSO-BN Inu: 1 N Trann daa D { x, x,, x }, N s he number of daa case; he nal oulaon sze s s, food sources, emloyee bees and onlooker bees are maxcycle; snfcal leel ; saus s ; u s he uer bound of he number of arens for each node; he max eraon number s l for each food source Se 1 Sole an unconsraned omzaon roblem, and oban an undrec rah by adon he mehod of [16] Afer ha, enerae he nal food sources randomly s X( 1,, ) Se For c=1 o maxcycle do /* emloyee bee hase*/ For each emloyed bee, nalze oson G,, G, G, G, =maxscor e( G, ), s 1,, Generae a a random, G, G, Se ( a b subracon G,, G, ), c subracon( G, G, ), add (,, ) new a b c Se G, 1 moe( G,, new ) ; G rearbymuualinfo ( G, D) ;, 1, 1 G rearmaxarens ( G, u) ;, 1, 1 If score( G, 1 ) score( G, ), hen G If score( G, 1) score( G ), hen G end for /* onlooker bee hase*/ for s 1,, do Comue robably G, 1, 1 G 1, 1, end f, end f for each food source end for Se q, 1 Reea f rand hen se q=q+1 for food source X, enerae he canddae food X and end f 1 f s se 1; end f X unl q s /* scou bee hase*/ f max( l )>lm hen Scou bee wll enerae a new food source o subsue end f end for X based on s fness alue X by he way of emloyee bee, hen selec he beer food source beween X 135 Volume 1, Number, March 15

7 4 Exermenal Ealuaon In order o es he behaor of he mehod roosed n hs aer, we resen he exermenal resuls carred ou wh our alorhm on a sandard nework daa ses (Asa Nework) A comarae sudy wh ABC-PSO-BN alorhm and PSO-BN([15]) s also erformed The exermens laform was a ersonal comuer wh Penum 4, 36GHZ CPU, 51 M memory, and Wndows XP The alorhm was mlemened by Malab 7 In our mehod, he unconsraned omzaon roblem was soled by calln he funcon bnro( ) n Malab These exermenal arameers were se as follows: he confdence alue of es s 95%, S=3, lm=1 F True Asa nework F 3 Beer Bayesan nework learned by our mehod For Asa nework, under daase s 8, oulaon sze s 3, maxcycle=3, our mehod can fnd beer nework(see F 3); howeer he mehod [15] needs he daasze s 8, oulaon sze s 3, maxcycle=6 Moreoer, wh a samle sze of 1, maxcycle= and oulaon sze of 15, our mehod fnds he bes nework srucure(he relaon beween er and score s en n F 4); howeer, he mehod [15] needs he daasze s 15, maxcycle=3 and oulaon sze of 15 From he comarson, can be seen ha our mehod s more effcency F 4 Relaon of eraon and score 5 Conclusons We hae deeloed a new bybrd alorhm for learnn Bayesan nework based on ABC and PSO alorhms Our alorhm frs soled he unconsraned omzaon roblem o oban an undreced rah Ths rocedure can effecely resrc he sace of canddae soluons, so ha many unnecessary searches can be aoded The alorhm was esed on a sandard nework srucure The exermenal resuls llusrae ha he new alorhm s sueror boh n erms of qualy of he soluons and comuaonal me References 136 Volume 1, Number, March 15

8 [1] Heckerman, D, Geer, D, e al (1995) Learnn Bayesan neworks: The combnaon of knowlede and sascal daa Machne Learnn,, [] J, J Z, Hu, R B, Zhan, H X, e al (11) A hybrd mehod for learnn Bayesan neworks based on an colony omzaon Aled Sof Comun, 11(4), [3] Pno, P C, Naele, A, Dejon, M, e al (9) Usn a local dscoery an alorhm for Bayesan nework srucure learnn IEEE Transacons on Eoluonary Comuaon, 13(4), [4] J, J Z, Hu, R B, & Lu, C N (9) A Bayesan nework learnn alorhm based on ndeendence es and an colony omzaon Aca Auom Snca, 35(3), [5] Lus, M C, Juan, M F, Jose, A G, e al () An colony omzaon for learnn Bayesan neworks Inernaonal Journal of Aroxmae Reasonn, 31, [6] Chen, J, Grener, R, Kelly, J, e al () Learnn bref neworks from daa: an nformaon heory based aroach Arfcal Inellence, 137, 43-9 [7] De Camos, L M, & Huee, J F () A new aroach for learnn belef neworks usn ndeendence crera Inernaonal Journal of Aroxmae Reasonn, 4(1), [8] De Camos, L M, Fernandez-Luna, J M, Ganez, J A, e al () An colony omzaon for learnn Bayesan neworks Inernaonal Journal of Aroxmae Reasonn, 31(3), [9] Larranaa, P, Poza, M, Yurramend, Y, e al (1996) Srucure learnn of Bayesan nework by enec alorhms: a erformance analyss of conrol arameers IEEE Transacons on Paern Analyss and Machne Inellence, 18(9), [1] Jose, A G, & Jose, M P () Searchn for he bes elmnaon sequence n Bayesan neworks by usn an colony omzaon Paern Reconon Leers, 3(1-3), [11] Man, L W, & Kwon, S L (4) An effcen daa mnn mehod for learnn Bayesan neworks usn an eoluonary alorhm-based hybrd aroach IEEE Transacons on Eoluonary Comuaon, 8(4), [1] Wan, T, & Yan, J (1) A heursc mehod for learnn Bayesan neworks usn dscree arcle swarm omzaon Knowlede and Informaon Sysems, 4, [13] Karaboa, D, & Basurk, B (1) On he erformance of arfcal bee colony (ABC) alorhm Aled Sof Comun, 4, [14] J, J Z, We, H K, & Lu, C N (13) An arfcal bee colony alorhm for learnn Bayesan neworks Sof Comun, 17(6), [15] Wan, C F, & L, J C (13) Hybrd arcle swarm alorhm for learnn Bayesan nework srucure Scence and Technoloy Reew, 31(), 5-55 Wan Chun-Fen was born n 1978 He receed hs MS deree n 6 from Henan Normal Unersy He receed hs PhD deree n 1 n Xdan Unersy Hs research neress nclude omzaon heory and s alcaons, Bayesan nework learnn Lu Ku was born n 1978 n Jaozuo, Henan Pronce, Chna, n 198 He receed hs BA deree n mahemacs and aled mahemacs from Henan Normal Unersy, Xnxan, Chna, n 4, he MS deree n aled mahemacs from Xdan Unersy, X an, Chna, n 8, and he PhD deree n aled mahemacs from Xdan Unersy, X an, Chna Hs research neress nclude wreless sensor neworks omzaon and dynamc neworkn echnoloy 137 Volume 1, Number, March 15