Spezielle Themen der Künstlichen Intelligenz

Transcription

1 Bayesian neworks Spezielle Themen der Künslichen Inelligenz! A graphical noaion or condiional independ. asserions, and a compac speciicaion o he ull join disribuion! Encodes he assumpion ha each node is condiionally independen o is non-descendans given is parens 9. Termin: Bayesian neworks & decision making! Then, each join disribuion can be acored as ollows P (X1,,Xn) =!i =1 P (Xi X1,,Xi-1) =!i =1P (Xi Parens(Xi)) STdKI/SS07 2 Probabilisic inerences in Bayes Nes Inerence in Bayesian Neworks! Query ypes (given evidence E): " Condiional probabiliy query: wha is he poserior probabiliy disribuion over values o a subse Y? " Mos probable explanaion query: wha is he mos likely assignmen o values o remaining (hidden) variables X-E? " Maximum a poseriori query: wha is he mos likely assignmen o values o subse Y?! Wors case, exac inerence is NP-hard! In pracice, much easier wih variable eliminaion or approximaive mehods Exac! Enumeraion: P(X E=e) =!P(X,E=e) =!"y P(X,E=e, Y=y)! Variable eliminaion: Avoid repeaed compuaions Approximaive! Sampling rom empy nework (no evidence) " draw N samples and compue approximae poserior " generae evens condiioned on CPT enries! Likelihood weighing " generae only samples ha agree wih he evidence! Markov chain simulaion " sochasic process whose saionary disribuion o saes is he rue poserior 4

2 p(smar)=.8 smar prepared pass sudy p(sudy)=.6 p(air)=.9 air P(s pa=rue) =!P(s,pa) =! "pr"sm" P(s,pa,pr,sm,) =! "pr"sm" P(s)P(pa pr,sm,)p(pr sm,s)p(sm)p() =! P(s) "sm" P(pa pr=t,sm,)p(pr=t sm,s)p(sm)p()! P(s) "sm" P(pa pr=f,sm,)p(pr=f sm,s)p(sm)p() p(pass ) =! P(s) " P(pa pr=t,sm=t,)p(pr=t sm=t,s)p(sm=t)p()! P(s) " P(pa pr=t,sm=f,)p(pr=t sm=f,s)p(sm=f)p()! P(s) " P(pa pr=f,sm=t,)p(pr=f sm=t,s)p(sm=t)p()! P(s) " P(pa pr=f,sm=f,)p(pr=f sm=f,s)p(sm=f)p() =... = 1/P(pa) 0.6 ( ) = 0.40 p(pr ) smar smar sudy.9.7 sudy.5.1 smar smar prep prep prep prep air air Query: Wha is he probabiliy ha a suden sudied, given ha hey pass he exam? Answer: The probabiliy ha a suden sudied, given ha she pass he exam, is 40% Using Bayes nes o model change The world changes; we need o rack and predic i Diabees managemen vs vehicle diagnosis Basic idea: copy sae and evidence variables or each ime sep X = se o unobservable sae variables a ime e.g., BloodSugar, SomachConens, ec. E = se o observable evidence variables a ime e.g., MeasuredBloodSugar, P ulseae, F oodeaen This assumes discree ime; sep size depends on problem Noaion: X a:b = X a, X a1,..., X b 1, X b 6 Markov chains / Markov processes Consruc a Bayes ne rom hese variables: parens? Markov assumpion: X depends on bounded subse o X 0: 1 Firs-order Markov process: P(X X 0: 1 ) = P(X X 1 ) Second-order Markov process: P(X X 0: 1 ) = P(X X 2, X 1 ) Firs!order Second!order X!2 X!1 X X 1 X 2 X!2 X!1 X X 1 X 2 Sensor Markov assumpion: P(E X 0:, E 0: 1 ) = P(E X ) Saionary process: ransiion model P(X X 1 ) and sensor model P(E X ) ixed or all ransiion model sensor model (changes ollows ixed laws) Markov processes Need prior probabiliy P(X0) over saes a ime 0 Then we have: P(X0,X1,...,X,E1,...,E)=P(X0)!!"#$$% P(Xi!Xi-1)P(Ei!Xi) Example ain!1 Umbrella!1!1 P( ) ain Umbrella P(U ) ain 1 Umbrella 1 Firs-order Markov assumpion no exacly rue in real world! Possible ixes: 1. Increase order o Markov process 2. Augmen sae, e.g., add T emp, P ressure 7 8

3 Inerence asks Inerence asks DynamicDynamic Bayesian neworks Bayesian neworks Filering: P(X e1:) belie sae inpu o he decision process o a raional agen X, E conain arbirarily many variables in a replicaed Bayes ne BMeer 1 Predicion: P(Xk e1:) or k > 0 evaluaion o possible acion sequences; like ilering wihou he evidence P( 0) ain 0 ain 1 Smoohing: P(Xk e1:) or 0 k < beer esimae o pas saes, essenial or learning Baery 0 Baery 1 X0 X1 XX 0 X1 Umbrella 1 Mos likely explanaion: arg maxx1: P (x1: e1:) speech recogniion, decoding wih a noisy channel given observaions, ind sequence o saes ha is mos likely o have generaed hem (e.g.vierbi algorihm) Z1 Need o give ransiion and senor model (saionary) only or irs slice (see Sec or algorihms) 9 Sensor variables Z, BMeer; sae variables X, Xdo, Baery Hidden Markov Model = DBN wih a single discree sae variable Chaper 15, Secions Chaper 15, Secions Exac inerence in DBNs Approximaive inerence in DBNs DBNs are Bayesian neworks, i.e., we can use our known algorihms Paricle ilering: ensure ha he populaion o samples ( paricles ) racks he high-likelihood regions o he sae-space Naive he nework and run any exac algorihm exac mehod: inerence unroll algorihm (e.g. variable eliminaion) Naive mehod: unroll he nework and run any exac algorihm Paricle ilering Algorihm: Creae N samples rom prior disribuion P(X0), hen cycle Propagae by sampling nex sae x1, given x and using P(X1 x) Basic idea: ensure ha he populaion o samples ( paricles ) 2. Weigh samples by likelihood i assigns o new evidence P(e1 x1) racks he high-likelihood regions o he sae-space 3. esample new N samples rom he curren populaion, probabiliy ha sample is replicaed proporional o isorweigh e eplicae paricles proporional o likelihood Exac inerence in DBNs Exac inerence in DBNs Exac inerence: Unroll nework o accommodae all observaions and run P( 0) ain 0 P( 0) ain 0 P( 0) ain 1 ain 0 P( 0) ain 1 Umbrella 1 Umbrella 1 ain 0 ain 2 ain 1 P(U01 ) ain 1 Umbrella 1 Umbrella 1 ain 3 P(U01 ) ain 2 Umbrella 2 Umbrella 2 ain 4 P(U01 ) ain 3 Umbrella 3 Umbrella 3 ain 5 P(U01 ) ain 4 Umbrella 4 Umbrella 4 ain 6 P(U01 ) ain 5 Umbrella 5 Umbrella 5 ain 7 P(U01 ) ain 6 Umbrella 6 Umbrella 6 ain 7 Umbrella 7 Umbrella 7 Problem: inerence cos or each updae grows wih Problem: inerence cos or each updae grows wih ollup ilering: add slice 1, sum ou slice using variable eliminaion ollup ilering: add n1 slice 1, sum ou n2 slice using variable eliminaion ), always updae exponenial cos O(d in) number o sae variables Larges acor size) is O(d Coss (acor almos 2n n1 n2 (c. HMM updae cos O(d )) cos O(d ) ), updae Larges acor is O(d (c. HMM updae cos O(d2n)) ain ain 1 ain 1 (b) Weigh (c) esample rue alse (a) Propagae SS07 Widely usedstdki or /racking nonlinear 11 ain 1 umbrella observed sysems, esp. in vision Also used or simulaneous localizaion and mapping in mobile robos 105-dimensional sae space Chaper 15, Secions

4 Paricle ilering Widely used or racking nonlinear sysems, especially in vision, sellocalizaion and mapping in mobile robos. Consisen (proo see Sec. 15.5) Approximaion error remains bounded over ime, a leas empirically. In pracice eicien, ye no heoreical guaranees (so ar) Avg absolue error LW(25) LW(100) LW(1000) LW(10000) E/SOF(25) Time sep Paricle ilering Likelihood weighing Summary! Exac inerence inracable in large neworks " Enumeraion, Variable eliminaion! Approximaive inerence " Likelihood Weighing, Mone Carlo Markov chains! Temporal models (DBN, HMM) " sae and sensor variables, replicaed over ime " Assumpions! Markov assumpion! ransiion model P(X X!1)! Sensor (Markov) assumpion! sensor model P(E X)! Saionary assumpion! models ixed! Tasks: ilering, predicion, smoohing, likely explanaion " approx. recursive algorihms exis (consan cos per sep) Making decisions under uncerainy Making decisions Suppose I believe he ollowing: Ge o airpor on ime P(A25 ges me here on ime )! = 0.04 P(A90 ges me here on ime )! = 0 P(A120 ges me here on ime )! = 5 P(A1440 ges me here on ime )! = Which acion o choose depends on likelihood an acion succeeds and my preerences over is possible consequences Uiliy heory is used o represen and iner preerences Decision heory = probabiliy heory uiliy heory STdKI/SS

5 Maximum expeced uiliy A nondeerminisic acion has possible oucome saes esuli(a) Prior o execuing A, he agen deermines he probabiliies P(esuli(A) Do(A),E) The expeced uiliy o A, given evidence E, is deined as EU(A E)=!i P(esuli(A) Do(A),E) U(esuli(A)) Principle o maximum expeced uiliy (MEU) An agen is raional i i chooses he acion ha yields he highes expeced uiliy, averaged over all possible oucomes o he acion. Le s ocus on one-sho decisions over single acions or now. (sequences o acions laer) Basis o uiliy heory - preerences An agen chooses among prizes (A, B, ec.) and loeries, i.e., siuaions wih uncerain prizes Loery L = [p, A; (1 p), B] 1!p L=[pi,Ci] oucome Ci can occur wih probabiliy pi Noaion: A B A B A B A preerred o B indierence beween A and B B no preerred o A Key quesion: How are preerences relaed when making decisions? L p A B aional preerences aional preerences cond. Idea: preerences o a raional agen mus obey consrains. aional preerences behavior describable as maximizaion o expeced uiliy Consrains: Orderabiliy (A B) (B A) (A B) Agen canno avoid deciding Transiiviy (A B) (B C) (A C) Coninuiy A B C p [p, A; 1 p, C] B Indieren beween loery A vs. C, and geing B or sure Subsiuabiliy A B [p, A; 1 p, C] [p, B; 1 p, C] Monooniciy A B (p q [p, A; 1 p, B] [q, A; 1 q, B]) Violaing he consrains leads o sel-eviden irraionaliy For example: an agen wih inransiive preerences can be induced o give away all is money I B C, hen an agen who has C would pay (say) 1 cen o ge B I A B, hen an agen who has B would pay (say) 1 cen o ge A I C A, hen an agen who has A would pay (say) 1 cen o ge C 1c B A 1c C 1c 19 20

6 Uiliies and preerences Preerences are a basic propery o raional agens. The exisence o a uiliy uncion ollows hen rom wo principles: Theorem (amsey, 1931; von Neumann and Morgensern, 1944): Given preerences saisying he consrains here exiss a real-valued uncion U such ha U(A) U(B) A B Uiliy principle U([p 1, S 1 ;... ; p n, S n ]) = Σ i p i U(S i ) Max. Expeced Uiliy principle MEU principle: Choose he acion ha maximizes expeced uiliy Tha is, uiliy uncion can be ormulaed in accord wih agen s preerences. Noe: Can ac raionally wihou expliciely rying o maximize expeced uiliy Uiliy uncions Uiliy uncion maps rom saes o real number. Bu which numbers? Preerences o real agens are usually sysemaic, and here are sysemaic ways o designing uiliy uncions. Monoonic preerences: Agen preers more money o less, all oher hings being equal. Does ha say anyhing abou loeries involving money? Ge $ or sure o lip coin or 50% chance o geing $ ? Expeced moneary value (EMV) = 0.5 $0 0.5 $ = $ EU(Accep) = 0.5 U(Sk0) 0.5 U(Sk ) (Sk=sae o possessing $k) EU(Decline) = U(Sk ) aional decision depends on uiliies assigned o oucome saes! The uiliy o money Uiliy scales Money does no behave as a uiliy uncion Given a loery L wih expeced moneary value EMV (L), usually U(L) < U(SEMV(L)) U(EMV (L)), i.e., people are risk-averse Uiliy curve: or wha probabiliy p am I indieren beween a prize x and a loery [p, $M; (1 p), $0] or large M? Typical empirical daa, exrapolaed wih risk-prone behavior: Uiliy decreases less when debs are already large (geing more risk-seeking) U o o o o o o o o $ o o o o!150,000 o 800,000 o o Uiliy increases less when a lo is already possessed (geing less risk-averse) bes possible prize u Normalized uiliies: u = 1.0, u = 0.0 wors possible caasrophe u Micromors: one-millionh chance o deah Sudies (rom 1980): worh ~$20 useul or ussian roulee, paying o reduce produc risks, ec. QALYs: qualiy-adjused lie years useul or medical decisions involving subsanial risk Noe: behavior is invarian w.r.. ve linear ransormaion U (x) = k 1 U(x) k 2 where k 1 > 0 Wih deerminisic prizes only (no loery choices), only ordinal uiliy can be deermined, i.e., oal order on prizes Any monoonic ransormaion leaves behavior unchanged 23 24

7 Muli-aribue uiliy Sric dominance (no uncerainy) Typically deine aribues such ha U is monoonic in each Oen oucomes are characerized by wo or more aribues. How can we handle uiliy uncions o many variables X 1... X n? E.g., wha is U(Deahs, Noise, Cos)? How can complex uiliy uncions be assessed rom preerence behaviour? Sric dominance: choice B sricly dominaes choice A i i X i (B) X i (A) (and hence U(B) U(A)) X 2 C B This region dominaes A X 2 B C Idea 1: ideniy condiions under which decisions can be made wihou complee ideniicaion o U(x 1,..., x n ) (exploiing he dominance o xi) A D A Idea 2: ideniy various ypes o independence in preerences and derive consequen canonical orms or U(x 1,..., x n ) Deerminisic aribues X 1 Uncerain aribues X 1 Holds seldom in pracice Sochasic dominance (uncerainy) Sochasic dominance cond. Probabiliy S1 S Negaive cos Probabiliy S1 S Negaive cos Disribuion p 1 sochasically dominaes disribuion p 2 i p 1(x)dx p p2(x)dx 2()d (greaer probabiliy o higher values) I U is monoonic in x, hen A 1 wih oucome disribuion p 1 sochasically dominaes A 2 wih oucome disribuion p 2 : p 1(x)U(x)dx p 2(x)U(x)dx EU(A1)! EU(A2) Muliaribue case: sochasic dominance on all aribues opimal Sochasic dominance can oen be deermined wihou exac disribuions using qualiaive reasoning E.g., consrucion cos increases wih disance rom ciy S 1 is closer o he ciy han S 2 S 1 sochasically dominaes S 2 on cos E.g., injury increases wih collision speed Can annoae belie neworks wih sochasic dominance inormaion: X Y (X posiively inluences Y ) means ha For every value z o Y s oher parens Z x 1, x 2 x 1 x 2 P(Y x 1, z) sochasically dominaes P(Y x 2, z) Qualiaive probabilisic neworks (QPNs): insead o giving condiional probabiliies, jus deine qualiaive consrains ha new evidence on X increases () or decreases (-) he belie value o Y

8 Age GoodSuden SocioEcon ExraCar Age GoodSuden SocioEcon ExraCar DrivingHis DrivingHis DrivQualiy Anilock Airbag CarValue HomeBase AniThe DrivQualiy Anilock Airbag CarValue HomeBase AniThe uggedness Acciden The uggedness Acciden The OherCos OwnCos OherCos OwnCos MedicalCos LiabiliyCos ProperyCos MedicalCos LiabiliyCos ProperyCos SocioEcon Age GoodSuden ExraCar DrivingHis Anilock DrivQualiy uggedness Acciden Airbag CarValue HomeBase AniThe The OherCos OwnCos SocioEcon Age GoodSuden ExraCar DrivingHis Anilock DrivQualiy Airbag CarValue HomeBase AniThe! uggedness Acciden The OherCos OwnCos MedicalCos LiabiliyCos ProperyCos MedicalCos LiabiliyCos ProperyCos 31 32

9 SocioEcon Age GoodSuden DrivingHis ExraCar! Anilock DrivQualiy Airbag CarValue HomeBase AniThe! uggedness Acciden The OherCos OwnCos SocioEcon Age GoodSuden DrivingHis ExraCar! Anilock DrivQualiy Airbag CarValue HomeBase AniThe! uggedness Acciden The OherCos OwnCos MedicalCos LiabiliyCos ProperyCos MedicalCos LiabiliyCos ProperyCos Preerences wihou uncerainy X 1 and X 2 preerenially independen o X 3 i preerence beween x 1, x 2, x 3 and x 1, x 2, x 3 does no depend on x 3 E.g., Noise, Cos, Saey : 20,000 suer, $4.6 billion, 0.06 deahs/mpm vs. 70,000 suer, $4.2 billion, 0.06 deahs/mpm Theorem (Leonie, 1947): i every pair o aribues is P.I. o is complemen, hen every subse o aribues is P.I o is complemen: muual P.I.. Theorem (Debreu, 1960): muual P.I. addiive value uncion: V (S) = Σ i V i (X i (S)) Hence assess n single-aribue uncions; oen a good approximaion Preerences wih uncerainy Need o consider preerences over loeries: (X,Y ses o aribues) X is uiliy-independen o Y i preerences over loeries in X do no depend on y (values in Y) Muual U.I.: each subse is U.I o is complemen muliplicaive uiliy uncion: (here, or hree aribues, Ui=Ui(xi)) U = k 1 U 1 k 2 U 2 k 3 U 3 k 1 k 2 U 1 U 2 k 2 k 3 U 2 U 3 k 3 k 1 U 3 U 1 k 1 k 2 k 3 U 1 U 2 U 3 ouine procedures and soware packages or generaing preerence ess o ideniy various canonical amilies o uiliy uncions 35 36

10 Decision neworks Add acion nodes and uiliy nodes o belie neworks o enable raional decision making Sae variable/chance nodes, cond. disribuion over parens (chance nodes, decision nodes) Air Traic Liigaion Airpor Sie Deahs Noise Decision/Acion nodes dieren value or each choice o acion Uiliy nodes deines uiliy uncion, Consrucion Cos parens = all var s ha Algorihm: aec he uiliy 1. Se Forevidence each value variables o acion or node curren sae 2. For each compue possible expeced value value o he o decision uiliy node: given acion, evidence (a) eurn se decision MEU acion node, (b) calc poserior or parens o uiliy nodes, (c) calc resuling uiliy or he acion U Value o inormaion 3. eurn he STdKI acion / SS07wih he highes uiliy (MEU) 37 Idea: compue value o acquiring each possible piece o evidence Can be done direcly rom decision nework Example: buying oil drilling righs Two blocks A and B, exacly one has oil, worh k Prior probabiliies 0.5 each, muually exclusive Curren price o each block is k/2 Consulan oers accurae survey o A. Fair price? Soluion: compue expeced value o inormaion = expeced value o bes acion given he inormaion minus expeced value o bes acion wihou inormaion Survey may say oil in A or no oil in A, prob. 0.5 each (given!) = [0.5 value o buy A given oil in A 0.5 value o buy B given no oil in A ] 0 = (0.5 k/2) (0.5 k/2) 0 = k/2 38 General ormula Curren evidence E, curren bes acion α Possible acion oucomes S i, poenial new evidence E j EU(α E) = max a Σ i U(S i ) P (S i E, a) Suppose we knew E j = e jk, hen we would choose α ejk s.. EU(α ejk E, E j = e jk ) = max a Σ i U(S i ) P (S i E, a, E j = e jk ) E j is a random variable whose value is currenly unknown mus compue expeced gain over all possible values: V P I E (E j ) = ( Σ k P (E j = e jk E)EU(α ejk E, E j = e jk ) ) EU(α E) Properies o VPI Nonnegaive in expecaion, no pos hoc j, E V P I E (E j ) 0 Nonaddiive consider, e.g., obaining E j wice V P I E (E j, E k ) V P I E (E j ) V P I E (E k ) Order-independen V P I E (E j, E k ) = V P I E (E j ) V P I E,Ej (E k ) = V P I E (E k ) V P I E,Ek (E j (VPI = value o perec inormaion) 39 40

11 Qualiaive behaviors Inormaion-gahering agen a) Choice is obvious, inormaion worh lile b) Choice is nonobvious, inormaion worh a lo c) Choice is nonobvious, inormaion worh lile P ( U E j ) P ( U E j ) P ( U E j ) Agen chooses beween sensing acion (EQUEST, which will yield evidence in nex percep) or real acion. U U U U 2 U 1 U 2 U 1 U 2 U 1 (a) (b) (c) Broad uiliies, di. can be high Narrow uiliies, di. will be small Exension: consider all possible sensing acion sequences and all possible oucomes o hose requess. Because values o requess depend on previous requess, need o build condiional plans Summary probabiliy heory: wha he agen should believe uiliy heory: wha he agen wans! decision heory: wha he agen should do Agen wih preerences beween loeries ha are consisen wih simple axioms can be described by uiliy uncion; selecs acions as i maximizing is expeced uiliy Muliaribue uiliies, sochasic dominance as echnique or making decisions wihou precise uiliy values Decision neworks o express and solve decision problems, oen involves gahering urher evidence based on value o inormaion 43