Plans. Memoryless plans Definition. Memoryless plans Example. Images Formal definition. Images. Preimages. Preimages. Definition

Transcription

1 Noetermnstc plannng (May 25, 2005) Plans AND-OR search Dynamc programmng (Albert-Ludwgs-Unverstät Freburg) 1 / 56 Plans Plans 1. map a state/an observaton to an operator. We use ths defnton of plans for fully observable problems only. 2. Cotonal plans generalze memoryless plans. They are needed for problems wthout full observablty. The state of the executon of a cotonal plan depes on observatons on earler executon steps. The state of the executon = a prmtve form of memory. The operator to be executed depes on the state of the executon. (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 Let S be the set of all states. A memoryless plan s a partal functon π : S O. Executon of a memoryless plan 1. Determne the current state s (full observablty!!!). 2. If π(s) s not defned then termnate executon. If the objectve s to reach a goal state, then π(s) s not defned f s s a goal state so that the executon termnates. 3. Execute acton π(s). 4. Goto step 1. (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 Formal defnton Image The mage of a set T of states wth respect to an operator o s the set of those states that can be reached by executng o n a state n T. T mg o (T ) (Image of a state) mg o (s) = {s S sos } (Image of a set of states) mg o (T ) = s T mg o(s) (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 Weak premage The premage of a set T of states wth respect to an operator o s the set of those states from whch a state n T can be reached by executng o. premg o (T ) T Formal defnton (Weak premage of a state) premg o (s ) = {s S sos } (Weak premage of a set of states) premg o (T ) = s T premg o(s). (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56

2 Strong premage The strong premage of a set T of states wth respect to an operator o s the set of those states from whch a state n T s always reached when executng o. Formal defnton spremg o (T ) T (Strong premage of a set of states) spremg o (T ) = {s S s T, sos, mg o (s) T } (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 AND-OR search for fully observable problems AND-OR search 1. Heurstc search (forward) Noetermnstc plannng can be vewed as AND-OR search. OR nodes: Choce between operators AND nodes: Noetermnstcally reached state Heurstc AND-OR search algorthms: AO*, Dynamc programmng (backward) Idea Compute operator/dstance/value for a state based on the operators/dstances/values of ts all successor states actons needed for goal states. 2.2 If states wth actons to goals are known, states wth + 1 actons to goals can be easly dentfed. Automatc reuse of already fou plan suffxes. OR s 1 s 2 s 3 s 4 OR OR OR OR s 5 s 6 s 7 s 8 s 9 s 10 s 11 s 12 s 13 s 14 s 15 s 16 s 17 s 18 s 19 s 20 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 Dynamc programmng Dynamc programmng Dynamc programmng Plannng by dynamc programmng If for all successors of state s wth respect to operator o a plan exsts, assgn operator o to s. Base case = 0: In goal states there s nothng to do. Iuctve case 1: If there s o O such that for all s mg o (s) s s a goal state or π(s ) was assgned on teraton 1, then assgn π(s) = o. dstance to G G Connecton to dstances If s s assgned a value on teraton 1, then the backward dstance of s s. The dynamc programmng algorthm essentally computes the backward dstances of states. (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 of dstance sets Let G be a set of states a O a set of operators. Defne the backward dstance sets D bwd for G, O that consst of those states for whch there s a guarantee of reachng a state n G wth at most operator applcatons. D bwd D bwd 0 = G = D 1 bwd spremg o(d 1 bwd ) for all 1 Let G be as set of states a O a set of operators, a let D0 bwd, D1 bwd,... be the backward dstance sets for G a O. Then the backward dstance from a state s to G s If s D bwd δ bwd G { 0 f s G (s) = f s D bwd for all 0 then δg bwd (s) =. \D 1 bwd (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56

3 Constructon of a plan based on dstances Makng the algorthm a logc-based algorthm Extracton of a plan from dstance sets 1. Let S S be those states havng a fnte backward dstance. 2. Let s be a state wth dstance = δg bwd (s) Assgn to π(s) any operator o O such that mg o (s) D 1 bwd. Hence o decreases the backward dstance by at least one. The plan π solves the plannng problem for S, I, O, G, P ff I S. An algorthm that represents the states explctly s feasble for transton systems wth at most 10 6 or 10 7 states. For plannng wth bgger transton systems structural propertes of the transton system have to be taken advantage of. Representng state sets as propostonal formulae often allow takng advantage of the structural propertes: a formula that represents a set of states or a transton relaton that has certan regulartes may be very small n comparson to the set or relaton. (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 Makng the algorthm a logc-based algorthm for noetermnstc operators We use a formula φ as a data structure for representng the set {s S s = φ}. We show that regresson regr o (φ) for noetermnstc operators s one way of computng strong premages. We present general technques for computng mages, premages a strong premages of sets of states represented as formulae. Many of the algorthms presented later n the lecture can be lfted to use a logc-based representaton, thereby expang ther range of applcablty to much bgger transton systems. We can easly generalze our regresson operaton for determnstc operators to regresson for noetermnstc operators of a restrcted syntactc form. ( for noetermnstc operators) Let φ be a propostonal formula a o = c, e 1 e n an operator where e 1,..., e n are determnstc. Defne regr o (φ) = regr c,e1 (φ) regr c,en (φ). (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 for noetermnstc operators Illustraton for noetermnstc operators Correctness regr c,(e1 e 2) (φ) = regr c,e1 (φ) regr c,e2 (φ) regr c,e2 (φ) regr c,e1 (φ) φ Let φ be a formula over A, o an operator over A, a S the set of all states over A. Then {s S s = regr o (φ)} = spremg o ({s S s = φ}). Let o = c, (e 1 e n ). {s S s = regr o (φ)} = {s S s = regr c,e1 (φ) regr c,en (φ)} = {s S s = regr c,e1 (φ),..., s = regr c,en (φ)} = {s S app c,e1 (s) = φ,..., app c,en (s) = φ} = {s S s = φ for all s mg o (s), there s s = φ wth sos } = spremg o ({s S s = φ}) 3rd = s by propertes of determnstc regresson. 4th = s by mg o (s) = {app c,e1 (s),..., app c,en (s)}. (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 for noetermnstc operators wth formulas Let o = d, (b c). Then regr o (b c) = regr d,b (b c) regr d, c (b c) = (d ( c)) (d (b )) d c b. By usng regresson we can compute formulas that represent backward dstance sets. Let G be a formula a O a set of operators. The backward dstance sets D bwd for G, O are represented by the followng formulae. D bwd D bwd 0 = G = D bwd 1 regr o (D 1 bwd ) for all 1 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56

4 wth formulas General mages a premages wth formulas Let G be a formula a O a set of operators, a let D0 bwd, D1 bwd,... be the formulae representng the backward dstance sets for G a O. Then the backward dstance from a state s to G s If s = D bwd δ bwd G { 0 f s = G (s) = f s = D bwd for all 0 then δg bwd (s) =. D bwd 1 The defnton of regresson covers only a subclass of noetermnstc operators. How to defne strong premages for all operators, a mages a premages? Now we apply a general dea: 1. Vew operators/actons as bnary relatons. 2. Represent these bnary relatons as formulae. 3. Defne relatonal operatons for relatons represented as formulae. (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 General mages a premages wth formulas General mages a premages wth formulas Defne the set of state varables possbly changed by e as changes(a) = {a} changes( a) = {a} changes(c e) = changes(e) changes(e 1 e n ) = changes(e 1 ) changes(e n ) changes(e 1 e n ) = changes(e 1 ) changes(e n ) Assumpton Let e 1 e n occur n the effect of an operator. If e 1,..., e n are not all determnstc then a a a may occur as an atomc effect n at most one of e 1,..., e n. Ths assumpton rules out effects lke (a b) ( a c) that may make a smultaneously true a false. In noetermnstc choces e 1 e n the formula for each e has to express the changes for exactly the same set B of state varables. τ B (e) = τ B(e) when e s determnstc τ B (e 1 e n ) = τ B (e 1) τ B (e n) τ B (e 1 e n ) = τ B\(B2 Bn) (e 1) τ (e B2 2) τ (e Bn n) where B = changes(e ) for {2,..., n} (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 General mages a premages wth formulas General mages a premages wth formulas We translate the effect e = (a (d a)) (c d) nto a propostonal formula. The set of state varables s A = {a, b, c, d}. τ{a,b,c,d} (e) = τ {a,b}(a (d a)) τ (a) τ = (τ {a,b} {a,b} {c,d} (c d) (d a)) (τ {c,d} (c) τ {c,d} (d)) = ((a (b b )) (((a d) a ) (b b ))) ((c (d d )) ((c c ) d )) Let A be a set of state varables. Let o = c, e be an operator over A n normal form. Defne τa (o) = c τ A (e). Lemma Let o be an operator. Then {v v s a valuaton of A A, v = τ } = {s s [A /A] s, s S, s mg o (s)}. (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 Exstental a unversal abstracton The most mportant operatons performed on transton relatons represented as propostonal formulae are based on exstental abstracton a unversal abstracton. Exstental abstracton of a formula φ wth respect to a A: a.φ = φ[ /a] φ[ /a]. Unversal abstracton s defned analogously by usng conjuncton nstead of dsjuncton. Unversal abstracton of a formula φ wth respect to a A: a.φ = φ[ /a] φ[ /a]. -abstracton s b.((a b) (b c)) = ((a ) ( c)) ((a ) ( c)) c a a c ab.(a b) = b.( b) ( b) = (( ) ( )) (( ) ( )) = ( ) ( ) = -abstracton s also known as forgettng: mon tue((mon tue) (mon work) (tue work)) tue((work (tue work)) (tue (tue work))) work (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56

5 a -abstracton n terms of truth-tables Propertes of abstracton operatons a a a correspo to combnng pars of lnes wth the same valuaton for varables other than a. a b c a (b c) c.(a (b c)) a b a b c.(a (b c)) c.(a (b c)) a a b c.(a (b c)) (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 Exstental a unversal abstracton of φ wth respect to a set of atomc propostons B = {b 1,..., b n } are B.φ = b 1.( b 2.(... b n.φ...)) B.φ = b 1.( b 2.(... b n.φ...)). (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 Propertes of abstracted formulas Propertes of a abstracton 1. Let φ be a formula over A. Then A.φ a A.φ are formulae that consst of the constants a a the logcal connectves only. 2. The truth-values of these formulae are epeent of the valuaton of A, that s, ther values are the same for all valuatons. 3. A.φ f a only f φ s satsfable. 4. A.φ f a only f φ s vald. Lemma If φ s a formula over A A a v a valuaton of A then 1. v = A.φ ff v v = φ for some valuaton v of A. 2. v = A.φ ff v v = φ for all valuatons v of A. (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 Sze of abstracted formulae Abstractng one varable takes polynomal tme n the sze of the formula. Abstractng one varable may double the formula sze. Abstractng n varables may ncrease sze by factor 2 n. For makng abstracton practcal the formulae must be smplfed, for example wth equvalences lke φ φ, φ, φ, φ φ,, a. by -abstracton Let A = {a 1,..., a n }, A = {a 1,..., a n}, φ 1 be a formula on A representng a row vector V 1 2 n (equvalently, a set of valuatons of A), a φ 2 a formula on A A representng a matrx M 2 n 2 n (equvalently, a bnary relaton on valuatons of A). The product matrx V M of sze 1 2 n s represented by A.(φ 1 φ 2 ) whch s a formula on A. To obtan a formula over A we have to rename the varables. ( A.(φ 1 φ 2 ))[A/A ] (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 by -abstracton Matrx multplcaton by -abstracton Let A = {a, b} be the state varables ( ) = ( ) represents the mage of {00, 10} wth respect to a relaton. a. b.( b (b b )) b.( b (b b )) ( ( b )) ( ( b )) b The formula b represents {01, 11}. (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 Let A = {a 1,..., a n }, A = {a 1,..., a n}, A = {a 1,..., a n}, φ 1 be a formula on A A representng matrx M 1 a φ 2 a formula on A A representng matrx M 2. The matrces M 1 a M 2 have sze 2 n 2 n. The product matrx M 1 M 2 s represented by whch s a formula on A A. A.(φ 1 φ 2 ) (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56

6 Matrx multplcaton by -abstracton Matrx multplcaton Let φ 1 = a a a φ 2 = a a represent two actons, reversng the truth-value of a a dong nothng. The sequental composton of these actons s a.φ 1 φ 2 = ((a ) ( a )) ((a ) ( a )) ((a ) ( a )) ((a ) ( a )) ( a a ) (a a ) a a. Multply ( a a ) ( b b ) a (a b ) (b a ): Ths s = a. b.( a a ) ( b b ) (a b ) (b a ) ( a b ) ( b a ). (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 a premages by formula manpulaton by formula manpulaton Defne s[a /A] = { a, s(a) a A}. Lemma Let φ be a formula on A a v a valuaton of A. Then v = φ ff v[a /A] = φ[a /A]. Let o be an operator a φ a formula. Defne mg o (φ) = ( A.(φ τa (o)))[a/a ] premg o (φ) = A.(τA (o) φ[a /A]) spremg o (φ) = A.(τA (o) φ[a /A]) A.τA (o). Let T = {s S s = φ}. Then {s S s = mg o (φ)} = {s S s = ( A.(φ τ ))[A/A ]} = mg o (T ). s = ( A.(φ τ ))[A/A ] ff s [A /A] = A.(φ τ ) ff there s valuaton s of A s.t. (s s [A /A]) = φ τ ff there s valuaton s of A s.t. s = φ a (s s [A /A]) = τ ff there s s T s.t. (s s [A /A]) = τ ff there s s T s.t. s mg o (s) ff s mg o (T ). (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 by formula manpulaton by formula manpulaton Let T = {s S s = φ}. Then {s S s = premg o (φ)} = {s S s = A.(τ φ[a /A])} = premg o (T ). s = A.(τ φ[a /A]) ff there s s 0 : A {0, 1} s.t. (s s 0 ) = τ φ[a /A] ff there s s 0 : A {0, 1} s.t. s 0 = φ[a /A] a (s s 0 ) = τ ff there s s : A {0, 1} s.t. s = φ a (s s 0 ) = τ ff there s s T s.t. (s s [A /A]) = τ ff there s s T s.t. s mg o (s) ff there s s T s.t. s premg o (s ) ff s premg o (T ). Above we defne s = s 0 [A/A ] (a hence s 0 = s [A /A].) Let T = {s S s = φ}. Then {s S s = spremg o (φ)} = {s S s = A.(τA (o) φ[a /A]) A.τA (o)} = spremg o(t ). See the lecture notes. (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 vs. regresson Summary Summary of matrx/logc/relatonal operatons Corollary Let o = c, (e 1 e n ) be an operator such that all e are determnstc. The formula spremg o (φ) s logcally equvalent to regr o (φ). {s S s = regr o (φ)} = spremg o ({s S s = φ}) = {s S s = spremg o (φ)}. matrces formulas state sets vector V 1 n formula on A set matrx M n n formula on A A relaton V 1 n + V 1 n φ 1 φ 2 unon φ 1 φ 2 ntersecton V 1 n M n n ( A.(φ τa (o)))[a/a ] mg o (T ) M n n V n 1 A.(τA (o) /A]) premg o (T ) A.(τA (o) φ[a /A]) A.τA (o) o(t ) (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56

7 vs. SAT a premages of sets of operators vs. SAT Image computaton vs. plannng by satsfablty We tested plan exstence by testng satsfablty of The unon of mages of φ wth respect to all operators o O s mg o (φ). Ths can be computed more drectly by usng the dsjuncton τ A(o) of the transton formulae: A.(φ ( τ ))[A/A ]. Same works for premages. ι 0 R 1 (A 0, A 1 ) R 1 (A t 1, A t ) G t where R 1 (A, A ) = τ A(o). -abstractng A 0 A t 1 yelds A t 1.( A 0.(ι 0 R 1 (A 0, A 1 )) R 1 (A t 1, A t ) G t ). Ths s equvalent to conjonng the t-fold mage of ι mg o ( mg o (ι) ) wth G to test goal reachablty n t steps. We can do the same wth premages startng from G. (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 Shannon expanson Bnary decson dagrams 3-place connectve f-then-else s defned by where a s a proposton. te(a, φ 1, φ 2 ) = (a φ 1 ) ( a φ 2 ) Shannon expanson of a formula φ wth respect to a A s φ (a φ[ /a]) ( a φ[ /a]) = te(a, φ[ /a], φ[ /a]) By repeated applcaton of Shannon expanson any propostonal formula can be transformed to an equvalent formula contanng no other connectves than te a propostonal varables only n the frst poston of te. (a b) (b c) te(a, ( b) (b c), ( b) (b c)) te(a, b c, b) te(a, te(b, c, c), te(b,, )) te(a, te(b,, c), te(b,, )) te(a, te(b,, te(c,, )), te(b,, )) (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 Bnary decson dagrams Canoncty Bnary decson dagrams: example a Transformaton to ordered BDDs 1. Fx an orderng a 1,..., a n on all propostonal varables. 2. Apply Shannon expanson to all varables n ths order. 3. Represent the resultng formulae as drected acyclc graphs (DAG) so that shared subformulae occur only once. b c b Let φ 1 a φ 2 be two ordered BDDs obtaned by usng the same varable orderng. Then φ 1 φ 2 f a only f φ 1 a φ 2 are somorphc (the same DAG.) 1 0 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 Satsfablty algorthms vs. BDDs Propertes of CPC normal forms Comparson: formula sze, runtme technque sze of R 1 (P, P ) runtme for plan length n satsfablty not a problem exponental n n BDDs major problem less depeent on n Comparson: resource consumpton technque crtcal resource satsfablty runtme BDDs memory Comparson: applcaton doman technque types of problems satsfablty lots of state varables, short plans BDDs few state varables, long plans Trade-offs between dfferent CPC normal forms Normal forms that allow faster reasonng are more expensve to construct from an arbtrary propostonal formula a may be much bgger. Propertes of dfferent normal forms φ TAUT? φ SAT? φ φ? crcuts poly poly poly co-np-hard NP-hard co-np-hard formulae poly poly poly co-np-hard NP-hard co-np-hard DNF poly exp exp co-np-hard n P co-np-hard CNF exp poly exp n P NP-hard co-np-hard BDD exp exp poly n P n P n P For BDDs one / s polynomal tme/sze (sze s doubled) but repeated / lead to exponental sze. (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56 (Albert-Ludwgs-Unverstät Freburg) AI Plannng May 30, / 56