Decision Networks. CS 188: Artificial Intelligence. Decision Networks. Decision Networks. Decision Networks and Value of Information

Transcription

1 CS 188: Artificil Intelligence nd Vlue of Informtion Instructors: Dn Klein nd Pieter Abbeel niversity of Cliforni, Berkeley [These slides were creted by Dn Klein nd Pieter Abbeel for CS188 Intro to AI t C Berkeley. All CS188 mterils re vilble t ME: choose the ction which mximizes the expected utility given the evidence Cn directly opertionlize this with decision networks Byes nets with nodes for utility nd ctions Lets us clculte the expected utility for ech ction New node types: Chnce nodes (just like BNs) Actions (rectngles, cnnot hve prents, ct s observed evidence) tility node (dimond, depends on ction nd chnce nodes)

2 Action selection = leve Instntite ll evidence Set ction node(s) ech possible wy Clculte posterior for ll prents of utility node, given the evidence Clculte expected utility for ech ction Choose mximizing ction = tke Optiml decision = leve W P(W) sun 0.7 rin 0.3 A W (A,W) leve sun 100 leve rin 0 tke sun 20 tke rin 70 Decisions s Outcome Trees Exmple: {} {} {} = leve A W (A,W) leve sun 100 leve rin 0 tke sun 20 tke rin 70 = tke (t,s) (t,r) (l,s) (l,r) W P(W F=bd) sun 0.34 rin 0.66 Almost exctly like expectimx / MDPs Wht s chnged? Optiml decision = tke =bd

3 Decisions s Outcome Trees Ghostbusters Decision Network Demo: Ghostbusters with probbility {b} Bust W {b} W {b} Ghost Loction =bd (t,s) (t,r) (l,s) (l,r) Sensor (1,1) Sensor (2,1) Sensor (1,2) Sensor (1,3) Sensor (1,n) Sensor (m,1) Sensor (m,n) Video of Demo Ghostbusters with Probbility Vlue of Informtion

4 Vlue of Informtion VPI Exmple: Ide: compute vlue of cquiring evidence Cn be done directly from decision network Exmple: buying oil drilling rights Two blocks A nd B, exctly one hs oil, worth k You cn drill in one loction Prior probbilities 0.5 ech, & mutully exclusive Drilling in either A or B hs E = k/2, ME = k/2 Question: wht s the vlue of informtion of O? Vlue of knowing which of A or B hs oil Vlue is expected gin in ME from new info Survey my sy oil in or oil in b, prob 0.5 ech If we know OilLoc, ME is k (either wy) Gin in ME from knowing OilLoc? VPI(OilLoc) = k/2 Fir price of informtion: k/2 O P 1/2 b 1/2 DrillLoc OilLoc D O k b 0 b 0 b b k ME with no evidence ME if forecst is bd ME if forecst is good distribution F P(F) good 0.59 bd 0.41 A W leve sun 100 leve rin 0 tke sun 20 tke rin 70 Vlue of Informtion VPI Properties Assume we hve evidence E=e. Vlue if we ct now: {+e} Nonnegtive Assume we see tht E = e. Vlue if we ct then: P(s +e) BT E is rndom vrible whose vlue is unknown, so we don t know wht e will be Expected vlue if E is reveled nd then we ct: Vlue of informtion: how much ME goes up by reveling E first then cting, over cting now: P(s +e, +e ) {+e, +e } {+e} P(+e +e) {+e, +e } P(-e +e) {+e, -e } Nondditive (think of observing E j twice) Order-independent

5 Quick VPI Questions Vlue of Imperfect Informtion? The soup of the dy is either clm chowder or split pe, but you wouldn t order either one. Wht s the vlue of knowing which it is? There re two kinds of plstic forks t picnic. One kind is slightly sturdier. Wht s the vlue of knowing which? You re plying the lottery. The prize will be $0 or $100. You cn ply ny number between 1 nd 100 (chnce of winning is 1%). Wht is the vlue of knowing the winning number? No such thing (s we formulte it) Informtion corresponds to the observtion of node in the decision network If dt is noisy tht just mens we don t observe the originl vrible, but nother vrible which is noisy version of the originl one VPI Question POMDPs VPI(OilLoc)? VPI(ScoutingReport)? Scout DrillLoc OilLoc VPI(Scout)? VPI(Scout ScoutingReport)? Scouting Report Generlly: If Prents() Z CurrentEvidence Then VPI( Z CurrentEvidence) = 0

6 POMDPs Exmple: Ghostbusters Demo: Ghostbusters with VPI MDPs hve: Sttes S Actions A Trnsition function P(s s,) (or T(s,,s )) Rewrds R(s,,s ) POMDPs dd: Observtions O Observtion function P(o s) (or O(s,o)) POMDPs re MDPs over belief sttes b (distributions over S) We ll be ble to sy more in few lectures s s, s,,s s' b b, o b' In (sttic) Ghostbusters: Belief stte determined by evidence to dte {e} Tree relly over evidence sets Probbilistic resoning needed to predict new evidence given pst evidence Solving POMDPs One wy: use truncted expectimx to compute pproximte vlue of ctions Wht if you only considered busting or one sense followed by bust? You get VPI-bsed gent! bust b b, e b {e} ( bust, {e}) e bust ( bust, {e, e }) {e} e, e {e, e } sense {e}, sense {e, e } Video of Demo Ghostbusters with VPI More Generlly* Generl solutions mp belief functions to ctions Cn divide regions of belief spce (set of belief functions) into policy regions (gets complex quickly) Cn build pproximte policies using discretiztion methods Cn fctor belief functions in vrious wys Overll, POMDPs re very (ctully PSPACE-) hrd Most rel problems re POMDPs, nd we cn rrely solve then in their full generlity

7 Next Time: Dynmic Models