Bayesian belief networks

Size: px

Start display at page:

Download "Bayesian belief networks"

Annabella Long
6 years ago
Views:

1 CS 1571 Introducton to I Lecture 24 ayesan belef networks los Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square CS 1571 Intro to I dmnstraton Homework assgnment 10 s out and due next week Fnal exam: December :00-1:50pm 5129 Sennott Square CS 1571 Intro to I

2 odelng uncertanty wth probabltes Knowledge based system era 70s early 80 s xtensonal non-probablstc models Solve the space tme and acquston bottlenecks n probablty-based models froze the development and advancement of K systems and contrbuted to the slow-down of I n 80s n general reakthrough late 80s begnnng of 90s ayesan belef networks Gve solutons to the space acquston bottlenecks Partal solutons for tme complextes ayesan belef network CS 1571 Intro to I ayesan belef networks Ns ayesan belef networks. Represent the full jont dstrbuton over the varables more compactly wth a smaller number of parameters. ake advantage of condtonal and margnal ndependences among random varables and are ndependent P and are condtonally ndependent gven C P C C C P C C CS 1571 Intro to I

3 larm system example. ssume your house has an alarm system aganst burglary. You lve n the sesmcally actve area and the alarm system can get occasonally set off by an earthquake. You have two neghbors ary and ohn who do not know each other. If they hear the alarm they call you but ths s not guaranteed. We want to represent the probablty dstrbuton of events: arthquake larm ary calls and ohn calls Causal relatons arthquake larm arycalls CS 1571 Intro to I ayesan belef network. 1. Drected acyclc graph Nodes random varables arthquake larm ary calls and ohn calls Lnks drect causal dependences between varables. he chance of larm s nfluenced by arthquake he chance of ohn callng s affected by the larm arthquake larm arycalls CS 1571 Intro to I

4 ayesan belef network. 2. Local condtonal dstrbutons relate varables and ther parents arthquake larm arycalls CS 1571 Intro to I ayesan belef network. F F larm F F arthquake F F F F F arycalls F F CS 1571 Intro to I

5 ayesan belef networks general wo components: S ΘS Drected acyclc graph Nodes correspond to random varables ssng lnks encode ndependences Parameters Local condtonal probablty dstrbutons for every varable-parent confguraton pa Where: pa - stand for parents of F F F F F CS 1571 Intro to I Full jont dstrbuton n Ns Full jont dstrbuton s defned n terms of local condtonal dstrbutons obtaned va the chan rule: n xample: 1.. n ssume the followng assgnment of values to random varables F pa hen ts probablty s: F P F CS 1571 Intro to I

6 ayesan belef networks Ns ayesan belef networks Represent the full jont dstrbuton over the varables more compactly usng the product of local condtonals. ut how dd we get to local parameterzatons? nswer: Graphcal structure encodes condtonal and margnal ndependences among random varables and are ndependent P and are condtonally ndependent gven C P C C P C C C he graph structure mples the decomposton!!! CS 1571 Intro to I Independences n Ns 3 basc ndependence structures: arthquake larm larm larm arycalls CS 1571 Intro to I

7 Independences n Ns arthquake larm larm larm arycalls 1. s ndependent of gven larm P P CS 1571 Intro to I Independences n Ns arthquake larm larm larm arycalls 2. s ndependent of arthquake not knowng larm and arthquake become dependent gven larm!! P CS 1571 Intro to I

8 Independences n Ns arthquake 3. larm larm larm arycalls 3. arycalls s ndependent of gven larm P P CS 1571 Intro to I Independences n N N dstrbuton models many condtonal ndependence relatons among dstant varables and sets of varables hese are defned n terms of the graphcal crteron called d- separaton D-separaton and ndependence Let Y and Z be three sets of nodes If and Y are d-separated by Z then and Y are condtonally ndependent gven Z D-separaton : s d-separated from gven C f every undrected path between them s blocked wth C Path blockng 3 cases that expand on three basc ndependence structures CS 1571 Intro to I

9 Undrected path blockng s d-separated from gven C f every undrected path between them s blocked 1. Path blockng wth a lnear substructure Z Y n Z n C Y n CS 1571 Intro to I Undrected path blockng s d-separated from gven C f every undrected path between them s blocked 2. Path blockng wth the wedge substructure Z n Z n C Y Y n CS 1571 Intro to I

10 Undrected path blockng s d-separated from gven C f every undrected path between them s blocked 3. Path blockng wth the vee substructure n Z Y n Y Z or any of ts descendants not n C CS 1571 Intro to I Independences n Ns arthquake larm RadoReport arycalls arthquake and are ndependent gven arycalls? CS 1571 Intro to I

11 Independences n Ns arthquake larm RadoReport arycalls arthquake and are ndependent gven arycalls F and arycalls are ndependent not knowng larm? CS 1571 Intro to I Independences n Ns arthquake larm RadoReport arycalls arthquake and are ndependent gven arycalls F and arycalls are ndependent not knowng larm F and RadoReport are ndependent gven arthquake? CS 1571 Intro to I

12 Independences n Ns arthquake larm RadoReport arycalls arthquake and are ndependent gven arycalls F and arycalls are ndependent not knowng larm F and RadoReport are ndependent gven arthquake and RadoReport are ndependent gven arycalls? CS 1571 Intro to I Independences n Ns arthquake larm RadoReport arycalls arthquake and are ndependent gven arycalls F and arycalls are ndependent not knowng larm F and RadoReport are ndependent gven arthquake and RadoReport are ndependent gven arycalls F CS 1571 Intro to I

13 ayesan belef networks Ns ayesan belef networks Represents the full jont dstrbuton over the varables more compactly usng the product of local condtonals. So how dd we get to local parameterzatons? n 1.. n pa he decomposton s mpled by the set of ndependences encoded n the belef network. CS 1571 Intro to I Full jont dstrbuton n Ns Rewrte the full jont probablty usng the product rule: F CS 1571 Intro to I

14 CS 1571 Intro to I Full jont dstrbuton n Ns F F F P F P Rewrte the full jont probablty usng the product rule: CS 1571 Intro to I Full jont dstrbuton n Ns F F F P F P F P F P Rewrte the full jont probablty usng the product rule:

15 CS 1571 Intro to I Full jont dstrbuton n Ns F F F P F P F P F P P Rewrte the full jont probablty usng the product rule: CS 1571 Intro to I Full jont dstrbuton n Ns F F F P F P F P F P P P P Rewrte the full jont probablty usng the product rule:

16 Full jont dstrbuton n Ns Rewrte the full jont probablty usng the product rule: F P F F P F P F P F P P P F CS 1571 Intro to I Parameter complexty problem In the N the full jont dstrbuton s defned as: P n P pa 1.. n What dd we save? larm example: 5 bnary rue False varables # of parameters of the full jont: One parameter s for free: larm arthquake arycalls CS 1571 Intro to I

17 Parameter complexty problem In the N the full jont dstrbuton s defned as: P n P pa 1.. n What dd we save? larm example: 5 bnary rue False varables # of parameters of the full jont: One parameter s for free: # of parameters of the N:? larm arthquake arycalls CS 1571 Intro to I ayesan belef network. In the N the full jont dstrbuton s expressed usng a set of local condtonal dstrbutons F arthquake F F larm F F F F F F arycalls F F CS 1571 Intro to I

18 ayesan belef network. In the N the full jont dstrbuton s expressed usng a set of local condtonal dstrbutons 2 2 F arthquake F F larm F F F F F F arycalls F F CS 1571 Intro to I Parameter complexty problem In the N the full jont dstrbuton s defned as: P n P pa 1.. n What dd we save? larm example: 5 bnary rue False varables # of parameters of the full jont: One parameter s for free: # of parameters of the N: CS 1571 Intro to I One parameter n every condtonal s for free:? larm arthquake arycalls

19 Parameter complexty problem In the N the full jont dstrbuton s defned as: P n P pa 1.. n What dd we save? larm example: 5 bnary rue False varables # of parameters of the full jont: One parameter s for free: # of parameters of the N: CS 1571 Intro to I One parameter n every condtonal s for free: larm arthquake arycalls odel acquston problem he structure of the N typcally reflects causal relatons Ns are also sometme referred to as causal networks Causal structure s ntutve n many applcatons doman and t s relatvely easy to defne to the doman expert Probablty parameters of N are condtonal dstrbutons relatng random varables and ther parents Complexty s much smaller than the full jont It s much easer to obtan such probabltes from the expert or learn them automatcally from data CS 1571 Intro to I

20 Ns bult n practce In varous areas: Intellgent user nterfaces crosoft roubleshootng dagnoss of a techncal devce edcal dagnoss: Pathfnder Intellpath CPSC unn QR-D Collaboratve flterng ltary applcatons usness and fnance Insurance credt applcatons CS 1571 Intro to I Dagnoss of car engne Dagnose the engne start problem CS 1571 Intro to I

21 Car nsurance example Predct clam costs medcal lablty based on applcaton data CS 1571 Intro to I ICU larm network CS 1571 Intro to I

and 867 arcs CS 1571 Intro to I QR-D edcal dagnoss n nternal

22 CPCS Computer-based Patent Case Smulaton system CPCS-P developed by Parker and ller Unversty of Pttsburgh 422 nodes and 867 arcs CS 1571 Intro to I QR-D edcal dagnoss n nternal medcne partte network of dsease/fndngs relatons CS 1571 Intro to I

CIS587 - Artificial Intellgence. Bayesian Networks CIS587 - AI. KB for medical diagnosis. Example.

CIS587 - Artificial Intellgence. Bayesian Networks CIS587 - AI. KB for medical diagnosis. Example. CIS587 - Artfcal Intellgence Bayesan Networks KB for medcal dagnoss. Example. We want to buld a KB system for the dagnoss of pneumona. Problem descrpton: Dsease: pneumona Patent symptoms (fndngs, lab tests):