Exact Inference: Introduction. Exact Inference: Introduction. Exact Inference: Introduction. Exact Inference: Introduction.

Size: px

Start display at page:

Download "Exact Inference: Introduction. Exact Inference: Introduction. Exact Inference: Introduction. Exact Inference: Introduction."

Jonas Benson
5 years ago
Views:

1 Exat nferene: ntroduton Exat nferene: ntroduton Usng a ayesan network to ompute probabltes s alled nferene n general nferene nvolves queres of the form: E=e E = The evdene varables = The query varables ssume a sngle varable for now Exat nferene: ntroduton Tastesood asepperon asmushrooms asnhoves ookwashesands n example of a query would be: Tastesood = true asepperon = true asmushrooms = true asnhoves = false Note: Even though ookwashesands s n the ayesan network t s not gven values n the query e. they do not appear ether as query varables or evdene varables They are treated as unobserved varables Exat nferene: ntroduton Reall that: E e E e E e Y y y n and... parents Enumeraton-sk algorthm: n nswer queres by omputng sums of produts of ondtonal probabltes from the network 4

2 Exat nferene: ntroduton Exat nferene: ntroduton Whenever you see a ondtonal lke =true =true use the han Rule: = / Query: =true =true ow do you solve ths? steps:. Express t n terms of the jont probablty dstrbuton. Express the jont probablty dstrbuton n terms of the entres n the Ts of the ayes net true true true true true Exat nferene: ntroduton a Whenever you need to get a subset of the varables eg. from the full jont dstrbuton use margnalzaton: true true Y y y true true true true true a true 7 Exat nferene: ntroduton a a true true a true To express the jont probablty dstrbuton as the entres n the Ts use:... true true true true a true a a N N arents 8

3 Exat nferene: ntroduton Exat nferene: ntroduton Take the probabltes that don t depend on the terms n the summaton and move them outsde the summaton mplfy f possble. true true true true a a true a a true true true true true a a a a 9 true true true true true a a a a true true true true a a a 0 Exat nferene: ntroduton Exat nferene: ntroduton Exat nferene n graphal models s Nhard Exponental tme n worst ase Example # : a a a pproxmate nferene s also N-hard ut ths s n the worst ase. n prate t s muh more effent Note: s not nstantated wth a value. We are omputng the table. f has k values and has k values the number of arthmet operatons requred s Ok f the han has n nodes omputng the jont probablty n s Onk Naïve approah requred Ok n operatons

4 4 Exat nferene: ntroduton Use dynam programmng to work from the nnermost summaton outward. 4 Exat nferene: ntroduton Exat nferene: ntroduton Two key deas to varable elmnaton:. ue to struture of N some subexpressons n the jont only depend on a small number of varables. ynam programmng ahes the ntermedate results to avod reomputng them exponentally many tmes

5 Reall: et be a set of random varables fator s a funton from Val R The set of varables s alled the sope of the fator and denoted ope[] We wll be manpulatng fators 7 et be a set of varables and Y a varable. et Y be a fator. We defne the fator margnalzaton of Y n denoted Y to be a fator over suh that: Y Y Ths operaton s also alled summng out of Y n ummng out = = = = = =0.9 Note: we only sum up entres n the table where the values of math up. 9 n a ayesan network: ummng out all varables results n a fator wth value n a Markov network: ummng out all varables n the unnormalzed dstrbuton ~ defned by the produt of fators n the Markov network results n the partton funton 0

6 Reall: et Y and Z be three dsjont sets of varables and let Y and YZ be two fators. We defne the fator produt x to be a fator : ValYZ R as follows: YZ = Y YZ Example of a fator produt: = = = = = = =0 00.= = = = =0.8 Operatons over fators: ddton s ommutatve: Multplaton s ommutatve: roduts are assoatve: Exhangng summatons and produts: f ope[ ] s not n the terms of Y Y Example: 4

7 The general problem nvolves a sum-produt nferene task: Z Trk: ush n the summatons as far as you an roedure um-rodut-ve // set of fators Z // et of varables to be elmnated < // Orderng on Z. et Z Z k be an orderng of Z suh that. Z < Z j f and only f < j. for = k 4. um-rodut-elmnate-varz. *. Return * roedure um-rodut-elmnate-var // set of fators Z // Varable to be elmnated. {: Z ope[]}. -. ' 4. Z. Return {} et be some set of varables and let be a set of fators suh that for eah ope[]. et Y be a set of query varables and let Z = Y. Then for any orderng < over Z um- rodut-vez< returns a fator *Y suh that * Y Z 7 8 7

8 8 9 Example: ompute Y for ayesan network. et: where Z={Z Z m } = - Y elmnate all nonquery varables n } { arents Note: We an do the exat same thng on a Markov network exept the fnal fator *Y s unnormalzed. 0 We wll ompute usng the elmnaton orderng. Note that: = =. Elmnaton orderng:. Elmnatng :. Elmnatng : Elmnaton orderng:. Elmnatng : 4. Elmnatng : Note: 4 sne. owever n ths elmnaton orderng you do need to generate ths fator for the next step.

9 Elmnaton orderng:. Elmnatng :. Elmnatng : 4 Elmnaton orderng: 7. Elmnatng : Note: You an use any elmnaton orderng eg.. Ths s a bad orderng beause t produes fators wth very large sope see Table 9. pg 0 4 omputng: tep Varable Elmnated Fators Used Varables nvolved New Fator ow do we deal wth evdene? eg. = true = hgh = false? Note that: hgh false hgh false hgh false 9

10 Reall roposton 4.7: et be a ayesan network over and E = e an observaton. et W = E. Then W e s a bbs dstrbuton defned by the fators where arents [ E e] The partton funton for ths bbs dstrbuton s e Ths means we an sum out entres n the redued fator roedure ond-rob-ve K // network over Y // et of query varables E =e // Evdene. Fators parameterzng K. Replae eah by [E=e]. elet an elmnaton orderng < 4. Z Y E. * um-rodut-ve<z. yvaly * y 7. return * Note: * represents Ye so [=hgh=false] then normalze wth =hgh=false. dvde * by to get Ye 7 8 tep Varable Elmnated omputng: =hgh=false Fators Used Varables nvolved New Fator [=hgh] [=hgh] [=false]. [=hgh]

Artificial Intelligence Bayesian Networks

Artificial Intelligence Bayesian Networks Artfcal Intellgence Bayesan Networks Adapted from sldes by Tm Fnn and Mare desjardns. Some materal borrowed from Lse Getoor. 1 Outlne Bayesan networks Network structure Condtonal probablty tables Condtonal