Probabilistic Models. CS 188: Artificial Intelligence Fall Independence. Example: Independence. Example: Independence? Conditional Independence

Transcription

1 C 188: Aificial Inelligence Fall 2007 obabilisic Models A pobabilisic model is a join disibuion ove a se of vaiables Lecue 15: Bayes Nes 10/18/2007 Given a join disibuion, we can eason abou unobseved vaiables given obsevaions (evidence) Geneal fom of a quey: Dan Klein UC Bekeley uff you cae abou uff you aleady know is kind of poseio disibuion is also called e belief funcion of an agen wic uses is model Independence wo vaiables ae independen if: Example: Independence N fai, independen coin flips: is says a ei join disibuion facos ino a poduc wo simple disibuions Independence is a modeling assumpion Empiical join disibuions: a bes close o independen Wa could we assume fo {Weae, affic, Caviy, ooace}? ow many paamees in e join model? ow many paamees in e independen model? Independence is like someing fom Cs: wa? Example: Independence? Condiional Independence Mos join disibuions ae no independen Mos ae pooly modeled as independen wam ain (ooace,caviy,cac)? If I ave a caviy, e pobabiliy a e pobe caces in i doesn' depend on wee I ave a ooace: (cac ooace, caviy) = (cac caviy) e same independence olds if I don ave a caviy: (cac ooace, caviy) = (cac caviy) wam wam ain ain wam wam ain ain Cac is condiionally independen of ooace given Caviy: (Cac ooace, Caviy) = (Cac Caviy) Equivalen saemens: (ooace Cac, Caviy) = (ooace Caviy) (ooace, Cac Caviy) = (ooace Caviy) (Cac Caviy) 1

2 Condiional Independence Uncondiional (absolue) independence is vey ae (wy?) e Cain ule II Can always faco any join disibuion as an incemenal poduc of condiional disibuions Condiional independence is ou mos basic and obus fom of knowledge abou unceain envionmens: Wy? Wa abou is domain: affic Umbella aining Wa abou fie, smoke, alam? is acually claims noing Wa ae e sizes of e ables we supply? ivial decomposiion: e Cain ule III Wi condiional independence: Condiional independence is ou mos basic and obus fom of knowledge abou unceain envionmens Gapical models elp us manage independence Gapical Models Models ae descipions of ow (a poion of) e wold woks Models ae always simplificaions May no accoun fo evey vaiable May no accoun fo all ineacions beween vaiables Wa do we do wi pobabilisic models? We (o ou agens) need o eason abou unknown vaiables, given evidence Example: explanaion (diagnosic easoning) Example: pedicion (causal easoning) Example: value of infomaion Bayes Nes: Big icue wo poblems wi using full join disibuions fo pobabilisic models: Unless ee ae only a few vaiables, e join is WAY oo big o epesen explicily ad o esimae anying empiically abou moe an a few vaiables a a ime Bayes nes (moe popely called gapical models) ae a ecnique fo descibing complex join disibuions (models) using a bunc of simple, local disibuions We descibe ow vaiables locally ineac Local ineacions cain ogee o give global, indiec ineacions Fo abou 10 min, we ll be vey vague abou ow ese ineacions ae specified Gapical Model Noaion Nodes: vaiables (wi domains) Can be assigned (obseved) o unassigned (unobseved) Acs: ineacions imila o C consains Indicae diec influence beween vaiables Fo now: imagine a aows mean causaion 2

3 Example: Coin Flips N independen coin flips X 1 X 2 X n No ineacions beween vaiables: absolue independence Vaiables: : I ains : ee is affic Example: affic Model 1: independence Model 2: ain causes affic Wy is an agen using model 2 bee? Example: affic II Le s build a causal gapical model Vaiables : affic : I ains L: Low pessue D: oof dips B: Ballgame C: Caviy Example: Alam Newok Vaiables B: Buglay A: Alam goes off M: May calls J: Jon calls E: Eaquake! Bayes Ne emanics obabiliies in BNs Le s fomalize e semanics of a Bayes ne A se of nodes, one pe vaiable X A dieced, acyclic gap A condiional disibuion fo eac node A collecion of disibuions ove X, one fo eac combinaion of paens values A 1 X A n Bayes nes implicily encode join disibuions As a poduc of local condiional disibuions o see wa pobabiliy a BN gives o a full assignmen, muliply all e elevan condiionals ogee: Example: C: condiional pobabiliy able Descipion of a noisy causal pocess A Bayes ne = opology (gap) + Local Condiional obabiliies is les us econsuc any eny of e full join No evey BN can epesen evey join disibuion e opology enfoces ceain condiional independencies 3

4 Example: Coin Flips Example: affic X 1 X 2 X n 1/4 3/4 3/4 1/4 1/2 1/2 Only disibuions wose vaiables ae absoluely independen can be epesened by a Bayes ne wi no acs. Example: Alam Newok Example: Naïve Bayes Imagine we ave one cause y and seveal effecs x: is is a naïve Bayes model We ll use ese fo classificaion lae Example: affic II ize of a Bayes Ne Vaiables : affic : I ains L: Low pessue D: oof dips B: Ballgame D L B ow big is a join disibuion ove N Boolean vaiables? ow big is an N-node ne if nodes ave k paens? Bo give you e powe o calculae BNs: uge space savings! Also easie o elici local Cs Also uns ou o be fase o answe queies (nex class) 4

5 Building e (Enie) Join We can ake a Bayes ne and build e full join disibuion i encodes Example: affic Basic affic ne Le s muliply ou e join ypically, ee s no eason o build ALL of i Bu i s impoan o know you could! o empasize: evey BN ove a domain implicily epesens some join disibuion ove a domain 1/4 3/4 3/4 1/4 1/2 1/2 3/16 1/16 Example: evese affic evese causaliy? Causaliy? Wen Bayes nes eflec e ue causal paens: Ofen simple (nodes ave fewe paens) Ofen easie o ink abou Ofen easie o elici fom expes 9/16 7/16 1/3 2/3 1/7 6/7 3/16 1/16 BNs need no acually be causal omeimes no causal ne exiss ove e domain (especially if vaiables ae missing) E.g. conside e vaiables affic and Dips End up wi aows a eflec coelaion, no causaion Wa do e aows eally mean? opology may appen o encode causal sucue opology eally encodes condiional independencies Ceaing Bayes Nes o fa, we alked abou ow any fixed Bayes ne encodes a join disibuion Nex: ow o epesen a fixed disibuion as a Bayes ne Key ingedien: condiional independence e execise we did in causal assembly of BNs was a kind of inuiive use of condiional independence Now we ave o fomalize e pocess Afe a: ow o answe queies (infeence) 5