Efficient Bayesian Network Learning for System Optimization in Reliability Engineering

Transcription

1 Qualy Technology & Quanave Managemen Vol. 9, No., pp. 97-, 202 QTQM ICAQM 202 Effcen Bayesan Newok Leanng fo Sysem Opmzaon n Relably Engneeng A. Gube and I. Ben-Gal Depamen of Indusal Engneeng, Faculy of Engneeng, Tel Avv Unvesy, Tel-Avv 69978, Isael (Receved Ocobe 200, acceped Mach 20) Absac: We pesen a new Bayesan newok modelng ha leans he behavo of an unknown sysem fom eal daa and can be used fo elably engneeng and opmzaon pocesses n ndusal sysems. The suggesed appoach eles on quanave cea fo addessng he ade-off beween he complexy of a leaned model and s pedcon accuacy. These cea ae based on measues fom Infomaon Theoy as hey pedeemne boh he accuacy as well as he complexy of he model. We llusae he poposed mehod by a classcal example of sysem elably engneeng. Usng compue expemens, we show how n a ageed Bayesan newok leanng, a emendous educon n he model complexy can be accomplshed, whle mananng mos of he essenal nfomaon fo opmzng he sysem. Keywods: Bayesan newoks, dffeenal complexy, muual nfomaon, elably of complex sysems, esouces opmzaon.. Inoducon ayesan newok (BN) s a pobablsc model epesenng he elaons beween andom B vaables of a cean doman. BNs have been exensvely employed n vaous applcaons n engneeng and decson makng (Peal []). A BN s composed of a deced acyclc gaph (DAG), and a se of paamees. The DAG conans veces (o nodes heenafe) and edges, whee each node epesens a andom vaable and an edge, connecng wo nodes, epesens a pobablsc dependency beween hese andom vaables. Essenally, he BN encodes he on pobably dsbuon P( X ) of he doman s andom vaables ha ae denoed by X. Snce BNs can be pesened gaphcally hey ae faly nuve (Ben-Gal [], Heckeman [8]). A BN can be leaned fom obsevable daa. Gaphcal consans can be appled whle leanng, and can be used effcenly fo lmng he complexy of he esulng model. In hs pape, we pay specal aenon o complexy-educon vesus he accuacy of he model ha can be accomplshed fom he leanng. In ode o effcenly lean a BN, numeous leanng mehods have been suggesed n ecen yeas (e.g., Chckeng [], Heckeman [8], Heckeman e al. [9], Cheng e al. [2], Peal [2]). In pacula, he K2 (Coope and Heskovs [5]) and he PC (Spes e al. []) ae wo examples of pedomnan BN leanng algohms. Mos of hese mehods, addess he model complexy by usfyng ndependences o condonal ndependences beween vaables, hough sascal ess ha ae mposed on he daa n one way o anohe. One of he mehods ha we follow hee s he addng-aows by Wllamson []. The addng-aows s an algohm whch aemps o maxmze he oal nfomaon wegh of he

2 98 Gube and Ben-Gal BN. Wllamson [] showed ha a BN, sasfyng some abay consan, ha bes appoxmaes a on pobably dsbuon, s one fo whch oal muual nfomaon (MI) wegh s maxmzed. He genealzed he agumens pesened eale by Chow and Lu [] egadng spannng ees. Chow and Lu [] poved ha, mnmzng he Kullback-Leble (KL) dvegence beween he ue dsbuon and he dsbuon epesened by a Bayesan ee, s equvalen o maxmzng he sum of MI weghs whn he ee. Noneheless, he undelyng obecve was o bes appoxmae he on pobably dsbuon descbng he doman whou addessng any andom vaable dffeenly fom he ohe vaables. In pacula, Chow and Lu dd no aemp o addess opmzaon cases - as we am dong hee - whee some of he vaables mgh have lage effecs on he age vaable han ohes. Ou undelyng pupose s o lean a BN n an effcen manne such ha can suppo opmzaon pocess of a complex sysem. In pacula, we consde a sysem elably-avalably-mananably (RAM) poblem whee he knowledge s exaced fom daa geneaed by he sysem (o ohe dencal sysems), encoded no he leaned BN, such ha he edundan nfomaon s dmnshed. In ode o mpove he elably of he sysem, one mgh have o esablsh a sysem model ha epesens he sysem s pefomance as a esponse funcon of some npu vaables. Havng done ha, maxmzaon of an obecve funcon could be accomplshed by changng some of he conollable npu vaables. Fo example, when consdeng he avalably of a RAM sysem, as we do n hs pape, he conollable vaables can be he numbe of spae pas of dffeen ypes, he pevenve manenance polcy and he numbe of wokbenches o be used. The sysem model, howeve, s fundamenally dffcul o esablsh due o seveal easons (Zacks [7]) such as: Exponenal complexy n he numbe of componens n he sysem Unknown falue dsbuons as well as dsbuons of ohe elevan pocesses Ineacons beween vaous componens ha affec he sysem saes These obsacles can be ackled by usng numecal mehods and compue expemens (Kenne and Zacks [ 0]), e.g., usng Mone Calo Smulaon fo pedcng he sysem oupu. The sascal popees of man-made sysems ae fundamenally dffcul o eeve. Ye, hey ae ofen modeled va Webull dsbuons fo falues (Zacks [8]) and Log-nomal dsbuons fo epas/eplacemens (Zacks [6]). Wh he gowng numbe of npus, howeve, numecal calculaons should geneae as many pedcons of he obecve funcon as equed fo he opmzaon pocedue - eachng ens of housands and somemes hundeds of housands ealzaons. Ths appoach becomes apdly mpaccal fo opmzaon pupose, as he calculaon pocedues of a sngle pedcon ealzaon mgh las mnues and somemes hous (Dub [6]). The movaon of hs wok s o mpove he pefomance of a complex sysem wh a mnmal use of expe knowledge fo modelng he sysem, avodng he above-lsed obsacles. We clam ha BN leanng fom daa could addess such a ask effcenly, as wll be llusaed n secon 5. In hs pape we pesen a BN leanng mehod oened o suppo opmzaon. To do so, we maxmze he sum of weghs (n ems of muual nfomaon measues) abou he age vaable, and hen we maxmze he sum of he emanng weghs whn he newok, accodng o he pncples of Wllamson []. I s shown ha when he leanng s age-oened, a good ade-off beween he model accuacy and he model complexy can

3 Effcen Bayesan Newok Leanng fo Sysem Opmzaon 99 be acheved f he pedeemned age vaable s aken no accoun a he leanng sage. Moe fomally, he poposed mehod has he followng wo sages: fo a gven age vaable X X, he appoxmaon o he magnal pobably dsbuon P( X ) as a funcon of he ene doman can be effcenly acheved by fs obanng he se of he mos affecng (paen) vaables Z X whee X Z, such ha he sum of he MI weghs beween X and Z s maxmzed. Then, havng obaned Z, by maxmzng he oal MI weghs among he vaables ha ae ncluded n Z. Geneally n hs pape, vecos and ses ae bold, andom vaables ae denoed by capal lees and he ealzaons by small case lees. The es of hs pape s oganzed as follows. Secon 2 povdes he mahemacal fomulaon o BN and BN leanng. I dscusses he unconsaned and he consaned leanng appoaches, boh compaed o Wllamson s [] mehod. Secon gves a schemac example and compaes he addng-aows appoach wh he poposed mehod, n boh unconsaned and consaned leanng confguaons. The algohm s pesened and dealed n Secon. Secon 5 pesens he suggesed appoach hough a eal-lfe example of a RAM poblem. I also suggess quanfed measues fo he ade-off beween model accuacy and model complexy. Secon 6 summazes he pape. 2. Bayesan Newoks Leanng A Bayesan newok BG (, ) can ofen be used o epesen he on pobably dsbuon of a veco of andom vaables X ( X,..., X N ). The sucue G( VE, ) s a deced acyclc gaph (DAG) composed of V, a veco of nodes epesenng he andom veco X, and E, a se of deced edges connecng he nodes. An edge E V V manfess dependence beween he vaables X and X, whle he absence of an edge demonsaes ndependence beween he vaables. A deced edge E connecs a paen node V o s chld node V (Heckeman [8], Yehezkel and Lene [5]). We denoe by L Z {,..., X X } he se of paen vaables of he andom vaable X epesened by he L se of paen nodes D {,..., V V } n G( VE, ) whee fo any leal, he supescp sands fo s ndex n he coespondng se and whee L Z s he sze of he se Z X. The se of paamees holds he local condonal pobables ove X, px ( z ) ha quanfy he edges fo each node sae x and each paens sae z (.e., a conuncon of L saes x... x ) of Z. Wllamson [] ndcaed ha a BN suffces o deemne a pobably dsbuon, snce fo each aomc sae ha s defned by a conuncon of all vaable saes ( x,..., x N ), one obans px (,..., xn ) px ( z ). He showed ha a BN, sasfyng some abay consan, ha bes appoxmaes a on pobably dsbuon, s one fo whch he sum of MI weghs ove he edges s maxmzed. In pacula, he genealzed he agumen pesened by Chow and Lu [] egadng he bes appoxmang spannng ee. Chow and Lu [] poved ha a Bayesan ee (wh encoded pobably dsbuon q ) ha bes appoxmaes a ue dsbuon p, s a maxmum wegh spannng ee. They used he Kullback-Leble (KL) dvegence (also known as he elave enopy) as a dsance measue beween he wo dsbuons: px (,..., x ) d( p q) p( x,..., x )log, () (,..., ) N N x,..., xn X qx xn whee edges beween wo nodes n he ee ae weghed by he MI of he coespondng andom vaables, defned as follows:

4 00 Gube and Ben-Gal px (, x ) I( X, X ) p( x, x )log., p( x ) p( x ) x x (2) Chow and Lu [] also poposed an algohm (we shall efe o hencefoh as he CLA) based on he weghng appoach. Wllamson [] mplemened hs saghfowad genealzaon o a BN, smply by eplacng he sngle paen vaable X wh he se of paen vaables Z as follows: px (, z ) I( X, Z ) p( x, z )log. ( ) ( ) x X, z Z px pz In a smla manne as Chow and Lu [] used (2) as he MI wegh whn a Bayesan ee, Wllamson [] used () as he MI wegh beween a vaable and a se of paen vaables n a BN. Snce () follows he MI chan ule, s moe complex - n he same sense ha a BN s moe complex han a Bayesan ee. 2.. Unconsaned Leanng-A Complee Newok In a complee BN, each node s conneced o all he ohe ( N ) nodes, esulng n a maxmum oal-weghed unconsaned BN. The oal wegh n he newok s a fxed sum of condonal MI elemens, whle he edges decons ae subec o he equemen ha he gaph mus be a DAG. Noe ha hee mgh be N! possble complee BNs ha would povde maxmum oal weghed BN. The oal MI wegh of a complee BN can be calculaed as follows: N I( X ; Z ) I( X ; X,..., X ) I( X ; X,..., X )... I( X ; X ). () N 2 2 N N N The em n () can be hen eoganzed: () N N N I( X ; Z ) I( X ; X X,..., X ), (5) whee ( X,..., X ) f. Snce MI s a symmec measue,.e., I( A; B) I( B; A) fo any andom vaables A and B, he oal wegh n (5) sums up o he same soluon egadless of he sang pon (.e., egadless of he ode of summaon), despe he dffeen elemens compsng he oal wegh. The above undesandng s of sgnfcan mpoance because mples ha he ode of an unconsaned BN leanng s elevan fom he pespecve of obanng he oal of he MI weghs. Fo any se of vaables ABC,, he condonal nfomaon I( A; B C ) s also called he (condonal) nfomaon gan (IG) beween B and A gven C. Noe ha some edges mgh conbue a zeo wegh (whee he IG s zeo), hence, s vable o emove hem fom he BN Consaned Leanng We sa by efeng o wo specfc consans menoned by Wllamson []. The consan C s mposed on K, he maxmum numbe of paens fo each node. The K consan C 2 s mposed on k, he complexy of a BN, whee k ( N K )2 fo bnay nodes (of wo saes). The above expesson mples ha consanng he complexy s

5 Effcen Bayesan Newok Leanng fo Sysem Opmzaon 0 an alenave way fo lmng he numbe of paens. Upon ha, we sugges wo addonal nfomaon-elaed consans C and C elaed o he pecenage elave IG (PRIG) and he pecenage elave nfomaon exploaon (PRIE) measues, especvely. C s mposed on a mnmum equed PRIG (by he use-adused paamee ) and C s mposed on he maxmum PRIE (by he use-adused paamee ). Fo any gven vaable X X, wh paen vaables se Z X, and a se of canddae vaables Z X, C can be expessed as C I( X; Z Z )/ H( X) 00 and C can be expessed as C I( X; Z)/ H( X) 00. The ange of as well as of s [0-00]. 0 mples ha any feasble edge could be dawn fom each poenal paen, excep fo hose conbung zeo weghs, wheeas 00 mples ha he cuen node wll no have any paens. 0 mples ha he cuen node wll no have any paens and 00 mples ha any feasble edge could be dawn fom each poenal paen, excep fo hose conbung zeo weghs. The PRIG can be consdeed as a so of a devave of he PRIE. Boh consans ae essenal fo easonably conollng he complexy as hey epesen s scale and shape alogehe. Noe howeve ha he selecon s odeed by he elave nfomaon gan, and no abaly as done by he convenonal consans C and C 2. Consde he MI beween a andom vaable X X and a se of vaables Z X epesenng s paens. The MI can be obaned usng he chan ule of nfomaon as follows: L I( X ; Z ) I( X ; X X,..., X ). (6) Gven I( X; Z ), f one consdes o add anohe paen V o V n G( VE, ), L epesenng he vaable X L X, hen he MI beween X and Z X should ake he followng fom: L L I( X ; Z X ) I( X ; Z ) I( X ; X Z ). (7) L L The IG beween X and X gven Z, s he magnal nfomaon ganed upon L addng he node X o D, he paens se of he node V. Snce Chow & Lu [] consdeed only Bayesan ees ( K max ( L) ), hey dd no addess he nfomaon chan ule. In he seach pocedue hey focused on fndng he lages MI measue beween each of he emanng nodes and he nodes aleady populaed whn he ee. In a ee, only L he fs componen of he MI n he chan,.e., ( ; I X X ) s consdeed,..., N. Moeove, he geedy seach of he CLA asceans a ee wh maxmal oal wegh, because coves all possble edges. Howeve, such an appoach would no hold n he consdeed ealm of BNs, whee mulple paens ae avalable. Wllamson [] dd consde he IG of each and evey edge whle maxmzng he oal nfomaon wegh whn he BN. To assue a newok wh a maxmal oal wegh, one has o seach all possble ses of paens fo each node, asng he complexy of he poblem up o a level of compuaonal nacably n lage newoks. As a paccal soluon, Wllamson [] suggesed he addng-aows leanng algohm. The addng-aows can be seen as a genealzaon o he CLA by eplacng he scong nfomaon quany I( X, X ) wh he condonal nfomaon quany I( X, X Z ) and ensung a each sage ha he se of consans C { C, C2,...} sll holds and he gaph emans a DAG. As seen below we popose a somewha smla appoach.

6 02 Gube and Ben-Gal. A Small Illusave Example Consde he oy example n Table. Hee, X { X, X2, X, X} and he daase conans welve nsances of X. The addng-aows algohm, when appled n hs example, yelds an unconsaned BN as shown n Fgue. Each node s depced by a ccle, labeled by he coespondng vaable name. The aows depc he edges, each abued alongsde by s IG expesson and wegh. Table. Illusave Example of daa. Case X X 2 X X Fgue. An unconsaned Bayesan newok leaned by he addng-aows algohm appled o he small daase n Table. Accodng o (5), and as long as he gaph emans a DAG, he locaon of each vaable n he newok s unmpoan when he pupose s encodng he on pobably dsbuon. The fac ha X, fo example, depends on X and no he oppose can be smply evesed by usng Bayes ule. Tha s, n he above example, one could evese he edge E X, X (eplacng wh E and updae he pobably ses n accodngly. X, X ) Fo llusaon puposes, suppose now ha vaable X s defned as a age vaable ha should be opmzed. The poposed algohm, whch s pesened n Secon, daws one edge a a me,.e., he one fo whch IG wegh s he lages. The pocess s dealed n Table 2. Havng saed wh X as he age vaable, an edge fom X o X s dawn, ganng 0.89 bs of nfomaon. Nex, snce no nfomaon s ganed f X o X 2 ae added o X condoned on X, no ohe paens ae added snce he soppng condon s

7 Effcen Bayesan Newok Leanng fo Sysem Opmzaon 0 sasfed. The same oune s appled hen o each paen of X n un (only o X n he cuen example). The fs paen added o X s X (havng an IG wegh of b n hs case) and he second paen s X 2. Snce addng an edge fom X 2 o X s condoned on X, he edge s wegh s 0.5 bs (ahe han 0.8 bs when applyng he addng-aows). Fnally, and n he same manne, an edge s dawn fom X 2 o X. The BN sucue ha was leaned by he poposed algohm fom he daa n Table s shown n Fgue 2 whee he node epesenng he age vaable s bold. Table 2. Infomaon gan weghs of each possble edge beween he poenal paens and he cuen leaned node, gven s exsng paens. Poenal Paens Node Exsng Paens X X 2 X X X X 0 0 X 0.8 X X 0.5 X 0. Fgue 2. An unconsaned Bayesan newok leaned by he poposed algohm appled o he small daase n Table. As expeced, he oal nfomaon wegh of he unconsaned BN, leaned by he poposed algohm, whch s I( X, Z) bs s equal o he one obaned by Wllamson s addng-aows algohm, whch s also I( X, Z) bs. These BNs ae only wo ou of! 2 possble complee newoks, each of whch would yeld a maxmum oal wegh (of 2 bs n he cuen example). Noe ha an unconsaned BN mus conan all N( N )/2 possble edges. Noneheless, fo he eade s convenence, he zeo weghed edges ae no dsplayed n he fgue. Le us consde now he convenonal consan C poposed by Wllamson and lm he numbe of paens o one ( K ). Fgue and Fgue show he esuls obaned by he

8 0 Gube and Ben-Gal addng-aows algohm and by he poposed algohm especvely, boh wh K. The oal nfomaon wegh ascbed o he BN leaned by he addng-aows s.8 bs, abou 6% less han ha of he unconsaned BNs n Fgue and Fgue 2. Ths esul can be compaed wh he.5 bs obaned by he poposed algohm unde C, ha s 25% less han he unconsaned BN. Noe ha alhough he oal wegh of he BN shown n Fgue s smalle, he poposed algohm fulflls s obecve - o maxmze he nfomaon on he age vaable. Fgue. A Bayesan newok leaned by he addng-aows algohm appled o he daase n Table wh a consan of a sngle paen pe node, K. Fgue. A Bayesan newok leaned by he poposed algohm appled o he daase n Table wh a consan of a sngle paen pe node, K. Nowhsandng, he ousandng consequence of he BN shown n Fgue s ha he age vaable X s no conneced o any ohe vaable. Snce X has elavely small nfomaon abou he es vaables, s excluded fom he ee. Hee s whee he pofound gap comes n. Whle he esul shown n Fgue would be sasfyng fo Wllamson s [] and Chow s & Lu s [] common obecve, dsplays an undesable oucome fo he poposed obecve. Ou obecve n hs case s o maxmze he nfomaon abou he age vaable X va ohe vaables whch compse he ene on dsbuon. Fgue emphaszes ha he mehod poposed by Wllamson [] does no sasfy such a equemen effcenly.

9 Effcen Bayesan Newok Leanng fo Sysem Opmzaon 05. Poposed Algohm Ou undelyng assumpon n hs sudy s ha a age vaable X X s gven, and ha we am o bes appoxmae he pobably dsbuon px ( ) usng he law of complee pobably wh espec o he ene doman. Namely, we wsh o epesen he expesson shown n (8) wh an appoxmaon shown n (9). C C C C x X px ( ) px ( x ) p( x ), (8) C qx ( ) px ( z ) p( z ), (9) z Z C whee x denoes he aomc saes of X X \ X and z denoes a sae of Z X, he se of vaables epesenng he paens of X n he BN. Amng a mnmzng d( p q ) fo p n (8) and q n (9), we consde wo seps. In he fs sep we am o fnd he se C Z X such ha I( X; Z ) s maxmzed. In he second sep we apply Wllamson s [] mehod and aemp o maxmze he sum of nfomaon weghs only whn Z C X nsead of whn he ene doman X. Fnally, Mn( d( p( X) q( X ))) s appoxmaed by he followng sages. Z ag max ( I( X ; Z )). C Z X 2. Max( X ( ; )). Z I X Z Thus, n he fs sage we look fo he paens subse of he age vaable ha maxmzes he nfomaon abou he age vaable. In he second sage we maxmze he sum of nfomaon weghs whn he paens subse, as shown n (5) by Wllamson fo he ene doman. The above appoxmaon s of sgnfcan mpoance as mples ha lage amoun of daa can be fleed ou whle leanng, when he obecve s he opmzaon of a gven age vaable. We also sugges consanng he newok dffeenally. Le us edefne he consans as C - he -h consan appled o he age vaable and C - he -h consan appled o he es of he vaables (efeed o as abues n classfcaon poblems). Le us also compue he complexy fo he geneal case. Recall ha f vaable X akes one ou of x values, hen he complexy n ems of he numbe of ndependen paamees s expessed, as follows: N K k ( x ) x. (0) C Equaon (0) s fee of he assumpon of bnay nodes, as quanfes he complexy of vaables wh mulple saes. Ou poposed algohm handles wo opons of epesenaons, ha follow fom he above menoned sages. They ae smla wh espec o sage, bu ae dffeen wh espec o sage 2. The fs algohmc opon maxmzes X C I( X ; Z ) and wll be efeed o X as he Exended-Famly epesenaon. The second opon maxmzes X ( ; ) Z I X Z and wll be efeed o as he Nuclea-Famly epesenaon. Ou poposed algohm employs a ecusve pocedue ha can be appled on any gven node a a me, and wh any se of poenal paens. The pocedue adds edges fom canddae nodes o he node o whch he

10 06 Gube and Ben-Gal pocedue s cuenly appled-each me an edge, fo whch IG wegh s he lages. The soppng condon fo hs pocedue s he beakng of some abay consan C C (e.g., eachng he maxmum numbe of paens) o when he se of canddae paens s empy. The pocedue begns wh he node epesenng he age vaable (he age node). Havng deemned he age node s paens, he same pocedue s hen appled o each one of he paens n un, obanng each paen s ancesos and so foh. The canddae paens a each call of he pocedue ae subec o he algohmc opon of epesenaon. In he case of he Exended-Famly opon, all nodes, excludng hose whch mgh beak he DAG, ae canddaes as ancesos and he emanng ancesos ae beng looked fo also ousde he paens of he age vaable. In he case of he Nuclea-Famly opon, only he paens of he age node ae canddaes as ancesos. The Nuclea-Famly ends up wh a BN n whch all nodes ae conneced o he age node and among hemselves up o a level conolled by he consans. A flow cha of he poposed algohm, wh X as he age vaable and X as he vaables doman, s gven n Fgue 5. Fgue 5. A flow cha of he poposed suboune, akng a dffeenal se of consans C { C, C }, whch s a funcon of h old and h new and s egaded ue f s sasfed o false ohewse. The consans may nclude (bu ae no lmed o) he maxmum numbe of paens; he maxmum numbe of chlden; and. Noe ha he npu paamees and ae appled hough C and C. T epesens he se of nodes, epesenng he canddae vaables emanng ehe fom he ene doman (he case of he Exended-Famly mode) o fom he paens of he cuen node ha epesens he vaable as he cuen agumen of he pocedue (he case of he Nuclea- Famly mode).

11 Effcen Bayesan Newok Leanng fo Sysem Opmzaon A Communcaon Sysem Case Sudy In hs secon we pesen an mplemenaon of a ealsc example. The example daa s aken fom a smulaon of an opeang communcaon sysem, suppled by spae pas of dffeen ypes, each of whch has a unaound me upon falues (Dub [6]). A elably block dagam (RBD) of he sysem s shown n Fgue 6 dsplayng edundancy. The fs lne compses he powe supply uns (componens,, 7), he second lne compses ansmes (componens 2, 5, 8) and he hd lne compses modems (componens, 6, 9). The gh-hand module of he RBD s a Lne module epesenng he wng of he sysem (componens 0, ). The falue dsbuons of he componens n hs example ae Exponenal. The mean me o falue (MTTF) and he ecyclng mes of he un ypes ae pesened n Fgue 6. The Lne componens ae dscaded. Namely, upon falue, hey ae no shpped o he epa depo and can be eplaced only wh new spae pas ha wee puchased and allocaed fo n advance (n ohe wods, he cyclng me s nfny). Also, a leas wo of he sees on he lef-hand sde of he sysem mus opeae fo he sysem o funcon popely. Fgue 6. The RBD of a communcaon sysem wh he componens falue and epa daa and he cos of spaes. Upon falue, he faled componen s aken fom he sysem and s shpped fom he feld o a epa depo. Havng been epaed, he componen wll eun o he soage as an avalable spae pa. If an old demand fo a spae pa s due a ha me, wll be sen o he sysem fo a eplacemen of a faled componen. The me snce falue o eplacemen s also efeed o as he unaound me. An llusaon of he above descbed logsc cycle s gven n Fgue 7. Roughly speakng, he opmzaon poblem ases fom a compeon of wo obecves: maxmzng pefomance vesus mnmzng he cos of spae pas o ohe sysem esouces.

12 08 Gube and Ben-Gal Fgue 8 llusaes a compason among vaous opmzaon appoaches of pevous sudes on a smla sysem, opeang n a dffeen logsc envonmen, whch ncluded mulple logsc levels and mulple felds, ha apa fom spae pas, ae nfluenced by ohe esouces, such as he numbe of wokbenches. Ths fgue, based on Gube and Keane [7] pesens unfomly dsbued andom samples of he sysem unavalably as a plafom fo applyng Genec Algohms (GA) fo opmzaon. The movaon was nsped by he ndependency of he GA appoach of he physcal model compaed o ohe mehods, ha eque nfomaon on he physcal sysem. Fgue 8 daws he densy of he sysem unavalably ove he esouces space, scalng he esouces mxue on he cos axs. The cos s sampled unfomly, by whch spae pas and ohe esouces ae allocaed. The coespondng esulng Paeo Fon of he unavalably s compaed wh he unavalably accomplshed by ohe opmzaon appoaches. Each sample shown n Fgue 8 s an oucome of a complee compue expemen (see Kene and Zacks [0]) a Mone Calo Smulaon of a well-defned model ha mees he defnons and descpons of he above communcaon sysem. Fgue 7. Logsc cycle of he communcaon sysem LRUs. Fgue 8. A communcaon sysem unavalably as a funcon of he cos allocaed fo suppong he sysem s pefomance (based on Gube and Keane [7]).

13 Effcen Bayesan Newok Leanng fo Sysem Opmzaon 09 The npu o ou poposed BN leane was a fla daabase conanng he saes of all componens, also known as lne eplaceable uns (LRUs), assemblng he sysem along wh he Sysem sae and he spae pas avalably, held by fou Sock vaables (.e., a sock pe each LRU ype). The Sysem pefomance s defned as he age vaable n hs example, and s measued by he Sysem avalably ha deves fom s elably and esouces suppo. Snce n hs example he LRU vaables ae no consdeed as conollable vaables, he ncluson n he BN s no eally elevan fo he opmzaon. The only conollable vaables ae he Sock vaables, as hey consue he spae pas saegy suppong he communcaon sysem avalably. The obsevaons wee sampled on a daly bass houghou 0 smulaed yeas of opeaon. Fgue 9 and Fgue 0 show he BNs leaned by he poposed algohm n Exended-Famly and n Nuclea-Famly modes, especvely. Boh BNs wee consaned o comply wh complexy levels assocaed wh [ 98%] and [ 25%]. Fgue 9. A Bayesan newok supply chan of he communcaon sysem, as leaned by he poposed algohm n an Exended-famly mode, consans ae se o [ 98% and 25%]. The complexy of he BN shown n Fgue 9, obaned by (0) s k 090. The complexy of he BN shown n Fgue 0 s k 50 (hus, fou odes of magnude off!), dsplayng he emendous dffeence beween he wo algohmc modes. Fgue 9 emphaszes how exhausve compuaon can be wased o no aval, should one aemps o lean a BN fo opmzaon puposes whou consdeng he consans dffeenally, as poposed. Whle he nfomaon wegh assocaed wh he Sysem vaable emans he same, much nfomaon wegh s efleced among he abues, nceasng he complexy: no only would end up wh an exemely complex BN, bu also he leaned BN mgh be nfeo fo opmzaon puposes han he one leaned by he Nuclea-famly mode. In hs sage we sudy he complexy level as a funcon of and, gong fom mnmum PRIG of 0% o some posve small fxed-values. Ths s pefeable fo llusaon puposes, as smoohes he complexy cuve due o s exponenal naue. Hence, we se fxed mnmum

14 0 Gube and Ben-Gal PRIGs of [ 2%]. The esulng complexy of he BN model as a funcon of and s llusaed n Fgue. Fgue 0. A Bayesan newok supply chan of he communcaon sysem, as leaned by he poposed algohm n a Nuclea-famly mode, consans ae se o [ 98% and 25%]. Fgue. The complexy ( k ), as a funcon of and,.e., he MaxPRIE abou he age vaable and MaxPRIE abou he es vaables, whch ae appled hough C.

15 Effcen Bayesan Newok Leanng fo Sysem Opmzaon The dashed dagonal cuve on op of he complexy suface desgnaes he adonal o convenonal complexy ade-off, as a one-dmensonal measue. The ade-off s execued beween and he model complexy (as a scala ha conols he complexy whou dffeenang he age vaable fom he es abues) and ceaes a lmed decson lne. Along hs lne, wo coespondng BNs ae llusaed. The locaons can be egaded an oucome of a saegc decson. In each of hese BNs, he boom node, ha s he Sysem node, epesens he age vaable. One can see ha along he dagonal cuve, he BNs end o be naow, mplyng ha he nfomaon s elavely exploed among all he vaables, whee few nodes ae conneced o he Sysem node. Smlaly, he K2 cuve n Fgue demonsaes such lmaon. Fuhemoe, he K2 algohm does no consde nfomaon measues fo consanng he BN and, heefoe, s complexy cuve does no fom an nfomaon-based decson lne. Though he dffeenal oulook, whee and ae dsplayed ohogonally, he ade-off changes sgnfcanly, enablng a decson plane, ahe han a decson lne whee all he vaables ae eaed equvalenly. I s clea fom he gaph ha he complexy can be educed emakably, whle mananng a consdeable level of he maxmum age PRIE equemen. Fo example, he BN ponng o [ 00%; 0%] connecs fou LRU nodes o he Sysem node, povdng he maxmal possble nfomaon fo opmzaon wh a complexy value of 9. Ths pon s he end pon of he [ 0%] cuve, efeed o as he Bes Tage cuve. In addon, hee moe cuves wh fxed values ae dsplayed: he [ 0%] cuve; he [ 5%] cuve; and he [ 0%] cuve. To llusae he oucomes ha can be leaned fom a BN, Fgue 2 demonsaes some opmzaon consdeaons based on he aned newok, wh s sock conollable vaables se o {,,, 7}. Ths BN was leaned by he Exended-famly algohmc mode enablng o epesen he ndec nfluence of he Sock vaables. To avod a complex BN sucue, such as he one shown n Fgue 9, he mnmum PRIGs wee se o comply wh 2%; and 5%. Fgue 2. Two assgnmens of a Bayesan newok of a communcaon sysem supply chan, leaned by he poposed algohm n an Exended-Famly mode, consaned o [ 2% and 5%]. The lef-hand assgnmen shows he dsbuon of each vaable, gven ha he Sysem vaable s n sae x (an avalable sae of he sysem). The gh-hand assgnmen shows he dsbuon of each vaable, gven ha he Sysem s n sae x 0 (an unavalable sae of he sysem).

16 2 Gube and Ben-Gal Havng lean hs BN, one can fsly noce ha he domnan LRU vaables affecng he age Sysem vaable ae LRU, LRU7, LRU0 and LRU. Also, ha LRU as well as LRU7 depend on Sock, whee LRU0 and LRU depend on Sock. The aveage sysem avalably wh hs seng s abou 99%. Gven ha he sysem s unavalable, he domnan paen LRUs (.e., LRU0 and LRU) ae faled, wh a condonal pobably of 5% and 65% especvely (akng he sae x 0 n Fgue 2). Gven he condonal sae dsbuon of LRU0 and LRU, he dsbuon of Sock changes such ha hee s a shoage of spae pas wh a pobably of 78% compaed wh a condonal pobably of 5% when he sysem s avalable. Ths obsevaon can sugges on nceasng he numbe of spae pas n sock numbe. Indeed, unnng a smulaon sudy wh a sock seng of {,,,8}, nceases he aveage avalably o nealy 99.9%. Noe ha he cos of he added spae pa s he cheapes among ohe pas n hs example. Alhough we do no dscuss hee he opmzaon ade-off beween unavalably coss and sock coss, hs example dsplays he means by whch such a desed opmzaon can be pocessed, as he above BN ncopoaes all he suffcen nfomaon fo cayng ou. 6. Summay Bayesan newoks ae pedomnan modelng mehods when dealng wh lage sysems ha nvolve and can supply lage amouns of daa. The dawback of modelng based on lage daases s ofen he equed modelng complexy. Thee exs seveal mehods ha ackle hs ssue. Ths pape connues a lne of pevous woks, as suggess a paccal appoach, fo bee a complexy conol, by dsngushng beween he age vaable vesus ohe vaables n he sysem. Ths enables he dsncon of he equed complexy fo undesandng wha affecs a age vaable, as opposed o geneal complexy measues ha ae elevan o he age vaable. In hs wok, we pesen a new algohmc appoach, whch effcenly ackles he complexy ssue. Accodngly, he algohm ams a maxmzng he nfomaon beween he age vaable and he elevan doman vaables (abues) and subsequenly maxmzes he nfomaon weghs among hese vaables. Though a ealsc example, was llusaed, how he poposed algohm handles well he ade-off beween nfomaon measues and complexy when leanng a BN fom a lage daase ha was geneaed by compue expemens. In pacula, measung he complexy dffeenally s suggesed, sessng he ably o educe he complexy emakably whle mananng mos of he essenal nfomaon fo pefomng he desed opmzaon. Noe ha he poposed algohm s no lmed o daa bases elaed o sysems engneeng, and can poenally addess vaous ypes of applcaons wh dffeen obecves. Acknowledgemens Ths eseach was suppoed by THE ISRAEL SCIENCE FOUNDATION (gan No. 62/0). Refeences. Ben-Gal, I. (2007). Bayesan Newoks. Encyclopeda of Sascs n Qualy and Relably, F. Rugge, F. Faln and R. Kene (eds.), John Wley & Sons. 2. Cheng, J. and Gene, R. (200). Leanng Bayesan belef newok classfes: algohms and

17 Effcen Bayesan Newok Leanng fo Sysem Opmzaon sysem. Lecue Noes n Compue Scence, -5.. Chckeng, D. M. (2002). Opmal sucue denfcaon wh geedy seach. Jounal of Machne Leanng Reseach,, Chow, C. K. and Lu, C. N. (968). Appoxmang dscee pobably dsbuons wh dependence ees. IEEE Tansacons on Infomaon Theoy, (), Coope, G. F. and Heskovs, E. (992). A Bayesan mehod fo he nducon of pobablsc newoks fom daa. Machne Leanng, 9, Dub, A. (2000). Mone Calo Applcaons n Sysems Engneeng. John Wley & Sons. 7. Gube, A. and Keane, A. J. (2006). Opmsaon of Sysem Resouces n Relably Avalably & Mananably Poblems Usng Genec Algohms, n 6 h Inenaonal Mce Symposum, Mce Akademy, Exee, UK. 8. Heckeman, D. (995). A Tuoal on Leanng wh Bayesan Newoks. In Leanng n Gaphcal Models, M. Jodan (eds.). MIT Pess, Cambdge, MA, Heckeman, D., Gege, D. and Chckeng, D. M. (995). Leanng Bayesan newoks: The combnaon of knowledge and sascal daa. Machne Leanng, 20, Kenne, R. and Zacks, S. (998). Moden Indusal Sascs: Desgn and Conol of Qualy and Relably. Boson: Duxbuy Pess.. Peal, J. (988). Pobablsc Reasonng n Inellgen Sysems: Newoks of Plausble Infeence. Mogan Kaufmann: San Fancsco. 2. Peal, J. (2000). Causaly: Models, Reasonng, and Infeence. Unvesy Pess: Cambdge.. Spes, P., Glymou, C. and Schenes, R. (2000). Causaon, Pedcon and Seach. 2 nd edon, MIT Pess.. Wllamson, J. (2000). Appoxmang Dscee Pobably Dsbuons wh Bayesan Newoks, n Poceedngs of he Inenaonal Confeence on Afcal Inellgence n Scence and Technology, 6-20 Decembe: Hoba Tasmana. 5. Yehezkel, R. and Lene B. (2009). Bayesan newok sucue leanng by ecusve auonomy denfcaon. Jounal of Machne Leanng Reseach, 0, Zacks, S. (966). Sequenal esmaon of he mean of a log-nomal dsbuon havng a pescbed popoonal closeness. The Annals of Mahemacal Sascs, 7, Zacks S. (992). Inoducon o Relably Analyss: Pobably Models and Sascs Mehods. Spnge-Velag, New Yok. 8. Zacks, S. (200). Dsbuons of Falue Tmes Assocaed wh Non-Homogeneous Compound Posson Damage Pocesses, Lecue Noes-Monogaph Sees, 5, A Fesschf fo Heman Rubn, Auhos Bogaphes: Avv Gube s a docoal suden, unde he supevson of Pofesso Iad Ben-Gal, n he Depamen of Indusal Engneeng a Tel Avv Unvesy. Hs eseach combnes applcaons fom Machne Leanng and Infomaon Theoy. The eseach s focused on Bayesan newok leanng, manly fo opmzaon puposes n ndusal and sevce sysems. Avv s a B.Sc. (2000) and an M.Sc. (200) n Nuclea Engneeng fom Ben-Guon Unvesy of he Negev. He has 8 yeas expeence n Modellng & Smulaon of complex sysems as a consulan and a seno uo, and as an algohms develope. Dung hs sudes, Avv s eachng a mandaoy couse fo undegaduae sudens n he Compue Inegaed Manufacung (CIM) Laboaoy. Avv's wok awaded fs pze a he IE & M 200 fo an ousandng eseach wok.

18 Gube and Ben-Gal Iad Ben-Gal s an Assocae Pofesso and he head of he CIM Lab n he Depamen of Indusal Engneeng a Tel Avv Unvesy. Hs eseach neess nclude sascal mehods fo conol and analyss of sochasc pocesses; Applcaons of Infomaon Theoy o ndusal poblems; Machne Leanng and Auomaon and Compue Inegaed Manufacung sysems. He holds a B.Sc. (992) degee fom Tel-Avv Unvesy, M.Sc. (996) and Ph.D. (998) degees fom Boson Unvesy. He s a membe of he Insue fo Opeaons Reseach and Managemen Scences (INFORMS), he Insue of Indusal Engnees (IIE), The Euopean Newok fo Busness and Indusal Sascs (ENBIS) and an eleced membe n he Inenaonal Sascal Insue (ISI). He s a Depamen Edo n he IIE Tansacons on Qualy and Relably and seves n he Edoal Boads of seveal ohe pofessonal ounals. He woe and eded fve books, publshed moe han 70 scenfc papes and eceved seveal bes papes awads. Hs papes have been publshed n IIE Tansacons, Inenaonal Jounal of Poducon Reseach, Technomecs, IEEE Tansacon, Qualy and Relably Engneeng Inenaonal, Jounal of Sascal Plannng and Infeence, as well as Bonfomacs and BMC Bonfomacs. Pof. Ben-Gal supevsed dozens of gaduae sudens and eceved seveal eseach gans, among hem fom Geneal Moos, IEEE, he Isael Mnsy of Scence and he Euopean Communy.