CS 188: Aificial Inelligence Fall 2007 Lecue 15: Bayes Nes 10/18/2007 Dan Klein UC Bekeley Pobabilisic Models A pobabilisic model is a join disibuion ove a se of vaiables Given a join disibuion, we can eason abou unobseved vaiables given obsevaions (evidence) Geneal fom of a quey: Suff you cae abou Suff you aleady know his kind of poseio disibuion is also called he belief funcion of an agen which uses his model 1
Independence wo vaiables ae independen if: his says ha hei join disibuion facos ino a poduc wo simple disibuions Independence is a modeling assumpion Empiical join disibuions: a bes close o independen Wha could we assume fo {Weahe, affic, Caviy, oohache}? How many paamees in he join model? How many paamees in he independen model? Independence is like somehing fom CSPs: wha? Example: Independence N fai, independen coin flips: H H H 2
Example: Independence? Mos join disibuions ae no independen Mos ae pooly modeled as independen wam cold P S sun ain P 0.6 0.4 S P S P wam wam sun ain 0.4 0.1 wam wam sun ain 0.3 0.2 cold cold sun ain 0.2 0.3 cold cold sun ain 0.3 0.2 Condiional Independence P(oohache,Caviy,Cach)? If I have a caviy, he pobabiliy ha he pobe caches in i doesn' depend on whehe I have a oohache: P(cach oohache, caviy) = P(cach caviy) he same independence holds if I don have a caviy: P(cach oohache, caviy) = P(cach caviy) Cach is condiionally independen of oohache given Caviy: P(Cach oohache, Caviy) = P(Cach Caviy) Equivalen saemens: P(oohache Cach, Caviy) = P(oohache Caviy) P(oohache, Cach Caviy) = P(oohache Caviy) P(Cach Caviy) 3
Condiional Independence Uncondiional (absolue) independence is vey ae (why?) Condiional independence is ou mos basic and obus fom of knowledge abou unceain envionmens: Wha abou his domain: affic Umbella Raining Wha abou fie, smoke, alam? he Chain Rule II Can always faco any join disibuion as an incemenal poduc of condiional disibuions Why? his acually claims nohing Wha ae he sizes of he ables we supply? 4
ivial decomposiion: he Chain Rule III Wih condiional independence: Condiional independence is ou mos basic and obus fom of knowledge abou unceain envionmens Gaphical models help us manage independence Gaphical Models Models ae descipions of how (a poion of) he wold woks Models ae always simplificaions May no accoun fo evey vaiable May no accoun fo all ineacions beween vaiables Wha do we do wih pobabilisic models? We (o ou agens) need o eason abou unknown vaiables, given evidence Example: explanaion (diagnosic easoning) Example: pedicion (causal easoning) Example: value of infomaion 5
Bayes Nes: Big Picue wo poblems wih using full join disibuions fo pobabilisic models: Unless hee ae only a few vaiables, he join is WAY oo big o epesen explicily Had o esimae anyhing empiically abou moe han a few vaiables a a ime Bayes nes (moe popely called gaphical models) ae a echnique fo descibing complex join disibuions (models) using a bunch of simple, local disibuions We descibe how vaiables locally ineac Local ineacions chain ogehe o give global, indiec ineacions Fo abou 10 min, we ll be vey vague abou how hese ineacions ae specified Gaphical Model Noaion Nodes: vaiables (wih domains) Can be assigned (obseved) o unassigned (unobseved) Acs: ineacions Simila o CSP consains Indicae diec influence beween vaiables Fo now: imagine ha aows mean causaion 6
Example: Coin Flips N independen coin flips X 1 X 2 X n No ineacions beween vaiables: absolue independence Example: affic Vaiables: R: I ains : hee is affic R Model 1: independence Model 2: ain causes affic Why is an agen using model 2 bee? 7
Example: affic II Le s build a causal gaphical model Vaiables : affic R: I ains L: Low pessue D: Roof dips B: Ballgame C: Caviy Example: Alam Newok Vaiables B: Buglay A: Alam goes off M: May calls J: John calls E: Eahquake! 8
Bayes Ne Semanics Le s fomalize he semanics of a Bayes ne A se of nodes, one pe vaiable X A dieced, acyclic gaph A condiional disibuion fo each node A collecion of disibuions ove X, one fo each combinaion of paens values A 1 X A n CP: condiional pobabiliy able Descipion of a noisy causal pocess A Bayes ne = opology (gaph) + Local Condiional Pobabiliies Pobabiliies in BNs Bayes nes implicily encode join disibuions As a poduc of local condiional disibuions o see wha pobabiliy a BN gives o a full assignmen, muliply all he elevan condiionals ogehe: Example: his les us econsuc any eny of he full join No evey BN can epesen evey join disibuion he opology enfoces ceain condiional independencies 9
Example: Coin Flips X 1 X 2 X n h h h Only disibuions whose vaiables ae absoluely independen can be epesened by a Bayes ne wih no acs. Example: affic R 1/4 3/4 3/4 1/4 1/2 1/2 10
Example: Alam Newok Example: Naïve Bayes Imagine we have one cause y and seveal effecs x: his is a naïve Bayes model We ll use hese fo classificaion lae 11
Example: affic II Vaiables : affic R: I ains L: Low pessue D: Roof dips B: Ballgame D L R B Size of a Bayes Ne How big is a join disibuion ove N Boolean vaiables? How big is an N-node ne if nodes have k paens? Boh give you he powe o calculae BNs: Huge space savings! Also easie o elici local CPs Also uns ou o be fase o answe queies (nex class) 12
Building he (Enie) Join We can ake a Bayes ne and build he full join disibuion i encodes ypically, hee s no eason o build ALL of i Bu i s impoan o know you could! o emphasize: evey BN ove a domain implicily epesens some join disibuion ove ha domain Basic affic ne Example: affic Le s muliply ou he join R 1/4 3/4 3/16 1/16 6/16 3/4 1/4 6/16 1/2 1/2 13
Example: Revese affic Revese causaliy? R 9/16 7/16 1/3 2/3 3/16 1/16 6/16 6/16 1/7 6/7 Causaliy? When Bayes nes eflec he ue causal paens: Ofen simple (nodes have fewe paens) Ofen easie o hink abou Ofen easie o elici fom expes BNs need no acually be causal Someimes no causal ne exiss ove he domain (especially if vaiables ae missing) E.g. conside he vaiables affic and Dips End up wih aows ha eflec coelaion, no causaion Wha do he aows eally mean? opology may happen o encode causal sucue opology eally encodes condiional independencies 14
Ceaing Bayes Nes So fa, we alked abou how any fixed Bayes ne encodes a join disibuion Nex: how o epesen a fixed disibuion as a Bayes ne Key ingedien: condiional independence he execise we did in causal assembly of BNs was a kind of inuiive use of condiional independence Now we have o fomalize he pocess Afe ha: how o answe queies (infeence) 15