CIS587 - Artificial Intellgence. Bayesian Networks CIS587 - AI. KB for medical diagnosis. Example.

Size: px

Start display at page:

Download "CIS587 - Artificial Intellgence. Bayesian Networks CIS587 - AI. KB for medical diagnosis. Example."

Marilyn Thomas
6 years ago
Views:

1 CIS587 - Artfcal Intellgence Bayesan Networks KB for medcal dagnoss. Example. We want to buld a KB system for the dagnoss of pneumona. Problem descrpton: Dsease: pneumona Patent symptoms (fndngs, lab tests): Fever, Cought, Paleness, WBC (whte blood cells) count, Chest pan, etc. Representaton of a patent case: Statements that hold (are true) for that patent. E.g: Fever =True Cough =False WBCcount=Hgh Dagnostc task: we want to nfer whether the patent suffers from the pneumona or not gven the symptoms 1

2 Uncertanty To make dagnostc nference possble we need to represent rules or axoms that relate symptoms and dagnoss Problem: dsease/symptoms relaton s not determnstc (thngs may vary from patent to patent) t s uncertan Dsease Symptoms uncertanty A patent sufferng from pneumona may not have fever all the tmes, may or may not have a cough, whte blood cell test can be n a normal range. Symptoms Dsease uncertanty Hgh fever s typcal for many dseases (e.g. bacteral dseases) and does not pont specfcally to pneumona Fever, cough, paleness, hgh WBC count combned do not always pont to pneumona Modelng the uncertanty. Relaton between the dsease and symptoms s not determnstc. Key ssues: How to descrbe, represent the relatons n the presence of uncertanty? How to manpulate such knowledge to make nferences? Humans can reason wth uncertanty.? 2

3 Methods for representng uncertanty KB systems based on propostonal and frst-order logc often represent uncertan statements, axoms of the doman n terms of rules wth varous certanty factors Very popular n 70-80s (MYCIN) If Then 1. The stan of the organsm s gram-postve, and 2. The morphology of the organsm s coccus, and 3. The growth conformaton of the organsm s chans wth certanty 0.7 the dentty of the organsm s streptococcus Problems: Channg of multple nference rules (propagaton of uncertanty) Combnatons of rules wth the same conclusons After some number of combnatons results not ntutve. Probablty theory a well-defned coherent theory for representng uncertanty and for reasonng wth t Representaton: Propostonal statements assgnment of values to random varables Pneumona = True WBCcount = hgh Probabltes over statements model the degree of belef n these statements Pneumona = True) = WBCcount = hgh) = Pneumona= True, Fever= True) = Pneumona= False, WBCcount = normal, Cough = False) =

4 Jont probablty dstrbuton Jont probablty dstrbuton (for a set varables) Defnes probabltes for all possble assgnments to values of varables n the set pneumona, WBCcount ) Pneumona True False 2 3 table WBCcount hgh normal low Pneumona) WBCcount) Margnalzaton - summng out varables Condtonal probablty dstrbuton Condtonal probablty dstrbuton: Probablty dstrbuton of A gven (fxed B) A, B) P ( A B) = B) Condtonal probablty s defned n terms of jont probabltes Jont probabltes can be expressed n terms of condtonal probabltes P ( A, B) = A B) B) Condtonal probablty s useful for dagnostc reasonng the effect of a symptoms (fndngs) on the dsease P ( Pneumona= True Fever = True, WBCcount= hgh, Cough= True) 4

5 Modelng uncertanty wth probabltes Full jont dstrbuton: jont dstrbuton over all random varables defnng the doman t s suffcent to represent the complete doman and to do any type of probablstc reasonng Problems: Space complexty. To store full jont dstrbuton requres to remember O(d n ) numbers. n number of random varables, d number of values Inference complexty. To compute some queres requres. O(d n ) steps. Acquston problem. Who s gong to defne all of the probablty entres? Pneumona example. Complextes. Space complexty. Pneumona (2 values: T,F)ever (2: T,F), Cough (2: T,F), WBCcount (3: hgh, normal, low), paleness (2: T,F) Number of assgnments: 2*2*2*3*2=48 We need to defne at least 47 probabltes. Tme complexty. Assume we need to compute the probablty of Pneumona=T from the full jont Pneumona = T ) = = Fever =, Cough = T j T k= h, n, l u T Sum over 2*2*3*2=24 combnatons j, WBCcount = k, Pale = u) 5

6 Modelng uncertanty wth probabltes Knowledge based system era (70s early 80 s) Extensonal non-probablstc models Probablty technques avoded because of space, tme and acquston bottlenecks n defnng full jont dstrbutons Negatve effect on the advancement of KB systems and AI n 80s n general Breakthrough (late 80s, begnnng of 90s) Bayesan belef networks Gve solutons to the space, acquston bottlenecks Partal solutons for tme complextes Bayesan belef networks (BBNs) Bayesan belef networks. Represent the full jont dstrbuton more compactly wth a smaller number of parameters. Take advantage of condtonal and margnal ndependences among components n the dstrbuton A and B are ndependent P ( A, B) = A) B) A and B are condtonally ndependent gven C P ( A, B C ) = A C ) B C ) P ( A C, B) = A C ) 6

7 Alarm system example. Assume your house has an alarm system aganst burglary. You lve n the sesmcally actve area and the alarm system can get occasonally set off by an earthquake. You have two neghbors, Mary and John, who do not know each other. If they hear the alarm they call you, but ths s not guaranteed. We want to represent the probablty dstrbuton of events: Burglary, Earthquake, Alarm, Mary calls and John calls Causal relatons %UJODU\ (DUWKTDNH $ODUP -RKQ&DOOV 0DU\&DOOV Bayesan belef network example. 7

8 Bayesan belef networks (general) Two components: B = ( S, Θ S ) Drected acyclc graph Nodes correspond to random varables (Mssng) lnks encode ndependences Parameters Local condtonal probablty dstrbutons for every varable-parent confguraton P pa( )) Where: pa ( ( ) - stand for parents of B E A J M Jont dstrbuton n Bayesan networks Full jont dstrbuton s defned n terms of local condtonal dstrbutons (va the chan rule): 1, 2,.., n ) = Example: = 1,.. n Probablty for one possble assgnments of values: pa( B = T, E = T, A = T, J = T, M = F) = )) B J A E M? 8

9 Full jont dstrbuton n BBNs Full jont dstrbuton s defned n terms of local condtonal dstrbutons (va the chan rule): 1, 2,.., n ) = Example: = 1,.. n Probablty for one possble assgnments of values: B= T, E = T, T, J = T, M = F) = pa( P ( B= T) T) T B= T, T) J = T T) M = F T) )) B J A E M Independences n BBNs 3 basc ndependence structures %UJODU\ %UJODU\ (DUWKTDNH $ODUP $ODUP $ODUP -RKQ&DOOV 0DU\&DOOV -RKQ&DOOV 1. JohnCalls s ndependent of Burglary gven Alarm 2. Burglary s ndependent of Earthquake (not knowng Alarm) Burglary and Earthquake are not ndependent gven Alarm!! 3. MaryCalls s ndependent of JohnCalls gven Alarm 9

10 Independences n BBNs Other dependences/ndependences n the network %UJODU\ (DUWKTDNH $ODUP 5DGLR5HSRUW -RKQ&DOOV 0DU\&DOOV Earthquake and Burglary are not ndependent gven MaryCalls Burglary and MaryCalls are not ndependent (not knowng Alarm) Burglary and RadoReport are ndependent gven Earthquake Burglary and RadoReport are not ndependent gven MaryCalls Parameters: full jont: BBN: Parameter complexty problem In the BBN the full jont dstrbuton s expressed as a product of condtonals (of smaller) complexty,,.., ) = pa( )) 1 2 n = 1,.. n = + 2( ) + 2(2) = 20 %UJODU\ $ODUP (DUWKTDNH Parameters to be defned: full jont: = 31 -RKQ&DOOV 0DU\&DOOV BBN: (2) + 2(1) = 10 10

11 Model acquston problem The structure of the BBN typcally reflects causal relatons BBNs are also sometme referred to as causal networks Causal structure s very ntutve n many applcatons doman and t s relatvely easy to obtan from the doman expert Probablty parameters of BBN correspond to condtonal dstrbutons relatng a random varable and ts parents only Ther complexty much smaller than the full jont Easer to come up (estmate) the probabltes from expert or automatcally by learnng from data Inference n Bayesan networks BBN models compactly the full jont dstrbuton by takng advantage of exstng ndependences between varables Smplfes the acquston of a probablstc model But we are nterested n solvng varous nference tasks: Dagnoss Predcton Requre to compute a varety of probablstc queres: Burglary JohnCalls = T) JohnCalls Burglary = T) Alarm) Queston: Can we take advantage of ndependences to construct specal algorthms and speedng up the nference? 11

12 Inference n Bayesan network Bad news: Exact nference problem n BBNs s NP-hard (Cooper) Approxmate nference s NP-hard (Dagum, Luby) But very often we can acheve sgnfcant mprovements Assume our Alarm network %UJODU\ (DUWKTDNH $ODUP -RKQ&DOOV 0DU\&DOOV Assume we want to compute: P ( J = T) Inference n Bayesan networks Approach 1. Blnd approach. Sum over the jont dstrbuton for all unnstantated varables, express the jont dstrbuton as a product of condtonals J = T) = B =, E = T j T k T l T. F j, A = k, J = T, M = l) = T j T k T l T J = T k) M = l k) k B=, j) B= ) j) Computatonal cost: Number of addtons: 16 Number of products: 16*5=80 12

13 Inference n Bayesan networks Approach 2. Interleave sums and products Combnes sums and product n a smart way (multplcatons by constants can be taken out of the sum) J = T) = J = T k) M = l k) k B=, j) B= ) j) = T j T k T l T J = = T k) M = l k) B= )[ k B=, j) j)] = T k T. Fl T k T j T J = T k)[ M = l k)][ B = )[ k B=, E = j) E = j)]] l T T j T Computatonal cost: Number of addtons: 2*(4+2)=12 Number of products: 2*8+2*4+3*2= 2*(15)=30 Inference n Bayesan network Exact nference algorthms: Symbolc nference (D Ambroso) Pearl s message passng algorthm (Pearl) Clusterng and Jon tree approach (Laurtzen, Spegelhalter) Arc reversal (Olmsted, Schachter) Approxmate nference algorthms: Monte Carlo methods: Forward samplng, Lkelhood samplng Varatonal methods 13

14 BBNs bult n practce In varous areas: Intellgent user nterfaces (Mcrosoft) Troubleshootng, dagnoss of a techncal devce Medcal dagnoss: Pathfnder (Intellpath) CPSC Munn QMR-DT Collaboratve flterng Mltary applcatons Insurance, credt applcatons (ICU) Alarm network 14

Pttsburgh) 422 nodes and 867 arcs QMR-DT Medcal dagnoss

15 CPCS Computer-based Patent Case Smulaton system (CPCS-PM) developed by Parker and Mller (at Unversty of Pttsburgh) 422 nodes and 867 arcs QMR-DT Medcal dagnoss n nternal medcne Bpartte network of dsease/fndngs relatons 15

Bayesian belief networks

Bayesian belief networks CS 1571 Introducton to I Lecture 24 ayesan belef networks los Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square CS 1571 Intro to I dmnstraton Homework assgnment 10 s out and due next week Fnal exam: December