LPG: A Planner Based on Local Search for Planning Graphs with Action Costs

Transcription

1 LPG: A Plnner Bsed on Locl Serch for Plnning Grphs with Action Costs Alfonso Gerevini nd Ivn Serin Diprtimento di Elettronic per l Automzione, Università degli Studi di Bresci Vi Brnze 38, I Bresci, Itly {gerevini,serin}@ing.unibs.it Abstrct We present LPG, fst plnner using locl serch for solving plnning grphs. LPG cn use vrious heuristics bsed on prmetrized objective function. These prmeters weight different types of inconsistencies in the prtil pln represented by the current serch stte, nd re dynmiclly evluted during serch using Lgrnge multipliers. LPG s bsic heuristic ws inspired by Wlkst, which in Kutz nd Selmn s Blckbox cn be used to solve the SAT-encoding of plnning grph. An dvntge of LPG is tht its heuristics exploit the structure of the plnning grph, while Blckbox relies on generl heuristics for SAT-problems, nd requires the trnsltion of the plnning grph into propositionl cluses. Another mjor difference is tht LPG cn hndle ction costs to produce good qulity plns. This is chieved by n nytime process minimizing n objective function bsed on the number of inconsistencies in the prtil pln nd on its overll cost. The objective function cn lso tke into ccount the number of prllel steps nd the overll pln durtion. Experimentl results illustrte the efficiency of our pproch showing, in prticulr, tht for set of well-known benchmrk domins LPG is significntly fster thn existing Grphpln-style plnners. Introduction Blum nd Furst s plnning grph nlysis (1997) hs become populr pproch to plnning on which vrious plnners re bsed (e.g., (Blum nd Furst 1997; Fox nd Long 1998; Kutz nd Selmn 1999; Koehler et l. 1997; Kmbhmpti, Minh nd Do 2001; Smith nd Weld 1999)). In this pper we present LPG, plnner bsed on locl serch nd plnning grphs, which considerbly extends preliminry techniques introduced in (Gerevini nd Serin 1999). The serch spce of LPG is formed by prticulr subgrphs of plnning grph, clled ction grphs, representing prtil plns. The opertors for moving from one serch stte to the next one re some grph modifictions corresponding to prticulr revisions of the represented prtil pln. LPG uses vrious heuristics bsed on prmetrized evlution function, where the prmeters weight different types of inconsistencies in n ction grph, nd re dynmiclly Copyright c 2002, Americn Assocition for Artificil Intelligence ( All rights reserved. evluted using Lgrnge multipliers. LPG s bsic heuristic, Wlkpln, ws inspired by Wlkst (Selmn, Kutz, nd Cohen 1994), stochstic locl serch procedure which in Kutz nd Selmn s Blckbox (1999) cn be used to solve the SAT-encoding of plnning grph (Kutz, McAllester, nd Selmn 1996). An dvntge of Wlkpln with respect to Blckbox is tht it opertes directly on the plnning grph. The serch neighborhood of Wlkpln is defined nd evluted by exploiting the structure of the grph, which in the SATtrnsltion of the grph is lost, or hidden. Another difference regrds the number of prllel steps in the pln (length of the pln). LPG cn increment the length of the pln during the serch process, while in SAT-bsed plnners this is done off-line. Most existing plnners hve limited notion of pln qulity tht is bsed on the number of ctions, or prllel time steps. LPG hndles more ccurte notion tking into ccount the execution cost of n ction. Execution costs re represented s numbers ssocited with the ctions. The serch uses this informtion to produce plns of good qulity by minimizing n objective function including term for the evlution of the overll execution cost of pln. This is performed through n nytime process generting succession of plns, ech of which improves the previous ones. A similr process cn be used to generte plns of good qulity in terms of their number of prllel steps or their overll durtion. Experimentl results illustrte the efficiency of our pproch for pln genertion indicting, in prticulr, tht LPG is significntly fster thn four other plnners bsed on plnning grphs: IPP4.1 (Koehler et l. 1997), STAN3s (Fox nd Long 1998), GP-CSP (Kmbhmpti, Minh nd Do 2001), nd the recent version 3.9 of Blckbox. Other experimentl results show the effectiveness of our incrementl process for generting good qulity plns. The second section introduces our bsic locl serch techniques for plnning grphs; the third section presents some techniques improving the serch process; the fourth section concerns pln qulity in LPG; the fifth section gives our experimentl results; the finl section gives conclusions nd mentions further work. AIPS

2 Locl Serch in the Spce of Action Grphs In this section we give the necessry bckground on plnning grphs (Blum nd Furst 1997), ction grphs, nd the bsic locl serch techniques introduced in (Gerevini nd Serin 1999). Plnning Grphs nd Action Grphs A plnning grph is directed cyclic led grph with two kinds of nodes nd three kinds of edges. The s lternte between fct, contining fct nodes, nd n ction contining ction nodes. An ction node t t represents n ction (instntited opertor) tht cn be plnned t time step t. Afct node represents proposition corresponding to precondition of one or more ctions t time step t, ortoneffect of one or more ctions t time step t 1. The fct nodes of 0 represent the positive fcts of the initil stte of the plnning problem (every fct tht is not mentioned in the initil stte is ssumed flse). In the following, we indicte with [u] the proposition (ction) represented by the fct node (ction node) u. The edges in plnning grph connect ction nodes nd fct nodes. In prticulr, n ction node of i is connected by: precondition edges from the fct nodes of i representing the preconditions of []; dd-edges to the fct nodes of i +1representing the positive effects of []; delete-edges to the fct nodes of i +1representing the negtive effects of []. Ech fct node f t l is ssocited with ction node t the sme, which represents dummy ction hving [f] s its only precondition nd effect. Two ction nodes nd b re mrked s mutully exclusive in the grph when one of the ctions deletes precondition or dd-effect of the other (interference), or when precondition node of nd precondition node of b re mrked s mutully exclusive (competing needs). Two proposition nodes p nd q in proposition re mrked s exclusive if ll wys of mking proposition [p] true re exclusive with ll wys of mking [q] true (ech ction node hving n dd-edge to p is mrked s exclusive of ech ction node b hving n dd-edge to q). When two fcts or ctions re mrked s mutully exclusive, we sy tht there is mutex reltion between them. Afct node q of i is supported in subgrph G of plnning grph G if either (i) in G there is n ction node t i 1 representing n ction with (positive) effect [q], or (ii) i =0(i.e., [q]isproposition of the initil stte). Given plnning problem Π, the corresponding plnning grph G cn be incrementlly constructed by strting from 0 using polynomil lgorithm. The lst of the grph is propositionl where the gol nodes re present, nd there is no mutex reltion between them. Without loss of generlity, we will ssume tht the gol nodes of the lst represent preconditions of specil ction [ end ], which is the lst ction in ny vlid pln. An ction grph (A-grph) A of G is subgrph of G contining end nd such tht, if is n ction node of G in A, then lso the fct nodes of G corresponding to the preconditions nd positive effects of [] reina, together with the edges connecting them to. A solution grph for G is n ction grph A s of G such tht ll precondition nodes of the ctions in A s re supported, nd there is no mutex reltion between ction nodes of A s. A solution grph for G represents vlid pln for Π. If the serch for solution grph fils, G cn be itertively expnded by dding n extr, nd performing new serch using the resultnt grph. Bsic Serch Neighborhood nd Heuristics Given plnning grph G, the locl serch process strts from n initil A-grph A of G (i.e., prtil pln), nd trnsforms it into solution grph through the itertive ppliction of some grph modifictions improving the current prtil pln. The two bsic modifictions consist of dding new ction node to the current A-grph, or removing n ction node from it (together with the relevnt edges). At ny step of the serch process, which produces new A-grph, the set of ctions tht cn be dded or removed is determined by the constrint violtions tht re present in the current A-grph. A constrint violtion (or inconsistency) is either mutex reltion involving ction nodes in the current A-grph, or n unsupported fct which is precondition of n ction in the current prtil pln. The generl scheme for serching solution grph ( finl stte of the serch) consists of two min steps. The first step is n initiliztion of the serch in which we construct n initil A-grph. The second step is locl serch process in the spce of ll A-grphs, strting from the initil A-grph. We cn generte n initil A-grph in severl wys. Three possibilities tht cn be performed in polynomil time, nd tht we hve implemented in LPG re: (i) rndomly generted A-grph; (ii) n A-grph where ll precondition fcts re supported (but in which there my be some violted mutex reltions); n A-grph obtined from n existing pln given in input to the process. Once we hve computed n initil A-grph, ech bsic serch step rndomly chooses n inconsistency in the current A-grph. If this is n unsupported fct node, then in order to resolve (eliminte) it, we cn either dd n ction node tht supports it, or we cn remove n ction node which is connected to tht fct node by precondition edge. If the chosen inconsistency is mutex reltion, then we cn remove one of the ction nodes of the mutex reltion. When we dd (remove) n ction node to del with n inconsistency, we lso dd (remove) ll edges connecting the ction node with the corresponding precondition nd effect nodes in the plnning grph. Given plnning grph G, nction grph A of G nd constrint violtion s in A, the neighborhood N(s, A) of s in A is the set of ction grphs obtined from A by pplying grph modifiction tht resolves s. Atech step of the locl serch, the elements in the neighborhood re weighted ccording to n evlution function, nd n element with the best evlution is then chosen s the possible next subgrph (serch stte). Given n ction subgrph A of plnning grph G, the generl evlution function F of A is defined s follows: 14 AIPS 2002

3 F (A) = G mutex(, A)+precond(, A) where: { 0 if A mutex(, A) = me(, A) if A { 0 if A precond(, A) = pre(, A) if A nd me(, A) is the number of ction nodes in A which re mutully exclusive with, while pre(, A) is the number of precondition fcts of which re not supported in A. It is esy to see tht, when the serch reches n A-grph for which F is zero, this is solution grph. The simple use of the generl function to guide the locl serch hs the drwbck tht it cn led to locl minim from which the serch cn not escpe. For this reson, insted of using F,weuse prmetrized ction evlution function E which llows to specify different types of heuristics. This function specifies the cost of inserting (E i ) nd of removing (E r )nction [] inthe prtil pln represented by the current ction subgrph A: E([], A) i = α i pre(, A)+β i me(, A)+γ i unsup(, A) E([], A) r = α r pre(, A)+β r me(, A)+γ r sup(, A), where me(, A) nd pre(, A) re defined s in F ; unsup(, A) is the number of unsupported precondition fcts in A tht become supported by dding to A; sup(, A) is the number of supported precondition fcts in A tht become unsupported by removing from A; α, β nd γ re prmeters specilizing the function. By ppropritely setting the vlues of these coefficients we cn implement different heuristic methods imed t mking the serch less influenced by locl minim. Their vlues hve to stisfy the following constrints: α i, β i > 0 nd γ i 0 in E i ; α r, β r 0 nd γ r > 0 in E r. Note tht the positive coefficients of E (α i,β i nd γ r ) determine n increment of E which is relted to n increment of the number of inconsistencies. Anlogously, the non-positive coefficients of E determines decrement of E which is relted to decrement of the number of inconsistencies. In (Gerevini nd Serin 1999) we proposed three bsic heuristics to guide the locl serch: Wlkpln, Tbupln nd T-Wlkpln. Inthis pper we focus on Wlkpln, which is similr to the heuristic used by Wlkst (Selmn, Kutz, nd Cohen 1994). In Wlkpln we hve γ i =0, α r =0, nd β r =0, i.e., the ction evlution function is E([], A) i = α i pre(, A)+β i me(, A) E([], A) r = γ r sup(, A). The best element in the neighborhood is the A-grph introducing the fewest new constrint violtions (Wlkpln does not consider the constrint violtions of the current A-grph tht re resolved). Like Wlkst, Wlkpln uses noise prmeter p: given n A-grph A nd (rndomly chosen) constrint violtion, if there is modifiction tht does not introduce new constrint violtions, then the corresponding A-grph in N(s, A) is chosen s the next A-grph; otherwise, with probbility p one of the grphs in N(s, A) is chosen rndomly, nd with probbility 1 p the next A-grph is chosen ccording to the minimum vlue of the ction evlution function. Neighborhood Refinements In this section we present some improvements of the bsic locl serch technique described in the previous section. Dynmic Heuristics Bsed on Lgrnge Multipliers As shown in (Gerevini nd Serin 2001) the vlues of the coefficients α, β nd γ in E cn significntly ffect the efficiency of the serch. Since their vlue is sttic, the optiml performnce cn be obtined only when their vlues re ppropritely tuned before serch. Moreover, the best prmeter setting cn be different for different plnning domins, or even for different problems in the sme domin. In order to cope with this drwbck, we hve revised the evlution functions by weighting their terms with dynmic coefficients similr to the Lgrnge multipliers for discrete problems tht hve been used for solving SAT problems (Shng nd Wh 1998; Wh nd Wu 1999; Wu nd Wh 2000). The use of these multipliers gives two importnt improvements to our locl serch. First, the revised cost function is more informtive, nd cn discriminte more ccurtely the elements in the neighborhood. Secondly, the new cost function does not depend on ny sttic coefficient (the α, β nd γ coefficients cn be omitted, nd the initil defult vlue of ll new multipliers is scrcely importnt). The generl ide is tht ech constrint violtion is ssocited with dynmic multiplier tht weights it. If the vlue of the multiplier is high, then the corresponding constrint violtion is considered difficult, while if the vlue is low it is considered esy. Using these multipliers, the qulity of n A-grph will be estimted not only in terms of the number of constrint violtions tht re present in it, but lso in terms of n estimtion of the difficulty to solve the prticulr constrint violtions tht re present. Since the number of constrint violtions tht cn rise during the serch cn be very high, insted of mintining distinct multiplier for ech of them, our technique ssigns multiplier to set of constrint violtions ssocited with the sme ction. Specificlly, for ech ction,wehve multi- plier λ m for ll mutex reltions involving, nd multiplier λ p for ll preconditions of. Intuitively, these multipliers estimte the difficulty of stisfying ll preconditions of n ction, nd of voiding to hve tht is mutully exclusive with ny other ction in the A-grph. The multipliers cn be used to refine the cost E([], A) i of inserting the ction node nd the cost E([], A) r of removing. Inprticulr, the evlution function for Wlkpln becomes E λ ([], A) i = λ p pre(, A)+λ m me(, A) E λ ([], A) r =, AIPS

4 where is the sum of the supported preconditions in A tht become unsupported by removing, weighted by λ j p, i.e., = (pre( j, A []) pre( j, A)). j A {} λ j p The multipliers hve ll the sme defult initil vlue, which is dynmiclly updted during the serch. Intuitively, we wnt to increse the multipliers ssocited with ctions introducing constrint violtions tht turn out to be difficult to solve, while we wnt to decrese the multipliers ssocited with ctions whose constrint violtions tend to be esier. These will led to prefer grphs in the neighborhood tht pper to be globlly esier so solve (i.e., tht re closer to solution grph). The updte of the multipliers is performed whenever the locl serch reches locl minimum or plteu (i.e., when ll the elements in the neighborhood hve E λ greter thn zero). The vlues of the multipliers of the ctions tht re responsible of the constrint violtions present in the current ction grph re then incresed by certin smll quntity δ +, while the multipliers of the ctions tht do not determine ny constrint violtion re decresed by δ.for exmple, if is in the current A-grph nd is involved in mutex reltion with nother ction in the grph, then λ m is incresed by predefined positive smll quntity δ + (otherwise it is decresed by δ ). 1 No-op Propgtion This technique is bsed on the observtion tht, whenever n ction is dded to the current pln (ction grph), its effects cn be propgted to the next time steps (s), unless there is n ction interfering with them. Suppose tht P is one of the effects of n ction t time step () t. Pis propgted to the next s by dding the corresponding no-ops, until we rech where the no-op for P is mutully exclusive with some other ction in the current pln. Notice tht in order to gurntee soundness, whenever we dd n ction t some time step fter t which is mutex with propgted no-op, it is necessry to block this no-op t t. Similrly, when n ction tht blocks no-op is removed, this no-op cn be further propgted (unless there is nother ction blocking it). The no-op propgtion cn be efficiently mintined by exploiting the dt structures of the plnning grph. Clerly, the propgtion is useful becuse it cn eliminte further inconsistencies (n ction is dded to support specific precondition, but the effects of this ction could lso 1 In order to mke the increment of λ p more ccurte, in our implementtion δ + is weighted by tking into ccount the proportion k of unsupported preconditions of with respect to the totl number of unsupported preconditions in the ction grph (i.e, λ p is incresed by k δ + ). Similrly, when we increse λ m, δ + is weighted by the proportion of mutex involving with respect to the totl number of mutex in the grph. Moreover, the unbounded increse or decrese of multiplier is prevented by imposing mximum nd minimum vlues to them. The defult vlues for δ + nd δ used in our tests re 10 3 nd respectively. support dditionl preconditions through the propgtion of their no-ops). Moreover, the no-op propgtion cn be exploited to extend the serch neighborhood, nd possibly reduce the number of serch steps. Given n unsupported precondition Q of nd ction t l of n A-grph A, the bsic neighborhood of A for Q consists of the grphs derived from A by either removing or dding n ction t l 1 which supports Q. In principle, mong these ctions we could use the no-op for Q (if vilble), which corresponds to ssuming tht Q is chieved by n erlier ction (possibly the initil stte), nd persists up to the time step when is executed. Insted of using this no-op, we use n ction t n erlier which hs Q s one of its effects, nd such tht Q cn be propgted up to l. When there is more thn one of such ctions, the neighborhood includes ll corresponding ction grphs derived by dding ny one of them. Experimentl results showed tht the use of the no-op propgtion cn significntly reduce both the number of serch steps nd the CPU-time required to find solution grph. Action Ordering nd Incrementl Grph Length In generl, locl serch methods do not exmine the serch spce in n exhustive wy, nd the hence lso the serch techniques described in the previous section cn not determine when it is necessry to increse the length (number of s) of the current plnning grph. Originlly, we overcme this limittion by utomticlly expnding the grph fter certin number of serch steps. However, this hs the drwbck tht, when the grph is extended severl times, significnt mount of CPU-time is wsted for serching the sequence of expnded grphs tht re generted before the lst expnsion. Recently, we hve developed n lterntive method for expnding the grph during serch tht leds to better performnce. In prticulr, we hve extended the serch neighborhood with A-grphs derived by two new types of grph modifictions tht re bsed on ordering mutex ctions nd possibly expnding the grph. Given two ctions tht re mutex t time step i, weexmine if this inconsistency cn be eliminted by postponing to time step i +1one of the ctions, or by nticipting it step i 1. Wesy tht n ction t step i cn be postponed to step i +1of A if the ction grph obtined by moving from i to i +1does not contin new inconsistencies with respect to A. 2 The condition for nticipting to i 1 is nlogous. When one of the ctions involved in mutex reltion cn be postponed or nticipted, the corresponding ction grph is lwys chosen from the neighborhood. If more thn one of these modifictions is possible, we choose rndomly one of the corresponding A-grphs. If none of the two ctions cn be postponed (or nticipted), we try to postpone (nticipte) one of them in combintion with grph extension. 2 In order to simplify the nottion, in this nd the next sections we indicte with (f) both n ction node (fct node) nd the corresponding represented ction (fct). The ctul mening of the nottion will be cler from the context. 16 AIPS 2002

5 () i+1 i (b) i+1 i P1 P1 P1 P6 P6 P7 P3 P8 mutex P6 1 P2 P2 P2 P7 P10 P P3 P8 P11 P11 P4 2 3 P4 (c) i+2 i+1 i P1 4 1 P7 P3 P8 P8 2 3 P1 P6 P6 P2 P2 P10 P11 P4 Figure 1: Exmples of ction ordering nd increment of grph length to del with mutex reltion. Specificlly, first we extend the plnning grph of one ; then we shift forwrd ll ctions from step i +1(i 1); finlly we move one of the two ctions to step i +1(i 1), provided tht this move does not introduce new inconsistencies (e.g., the moved ction blocks no-op tht is used to support precondition t lter step). If none of the two ctions cn be moved even fter grph extension, the ction grphs of the neighborhood re those derived by simply removing one of the two ctions. For exmple, in Figure 1() we hve tht ctions 1 nd 2 re mutex t i. Assume tht 1 hs two supported preconditions P 1 nd P 2, nd one dditive effect P 6 tht is needed by other ctions in the following s of the grph. Fct P 8 is supported by 2 nd is precondition of 4 ;fct P 7 is not used to support ny precondition. Figure 1(b) illustrtes cse in which 1 cn be postponed to i+1. This is possible becuse there is no mutex reltion between (1) 1 nd 4, (2) 1 nd the no-ops for P 5 nd P 9, (3) the no-ops supporting the preconditions of 1 nd ny other ction t i. Figure 1(c) illustrtes cse in which 1 cn not be postponed to i +1(becuse, for exmple, 4 deletes precondition of 1 ), but it cn be postponed in combintion with grph expnsion. This is possible becuse there is no mutex reltion between (1) 1 nd the no-ops of P 5,P 8,P 9, (2) between the no-ops supporting the preconditions of 1 nd ny other ction of i. Heuristic Precondition Costs The heuristic functions E nd E λ previously described evlute the A-grphs in the neighborhood by considering their number of unsupported preconditions nd unstisfied mutex reltions. An experimentl nlysis reveled tht, while computing this functions is quite fst, sometimes more ccurte evlution cn be very useful. In fct, it cn be the cse tht, lthough the insertion of new ction 1 leds to fewer new unsupported preconditions thn those introduced by n lterntive ction 2, the unsupported preconditions of 1 re more difficult to stisfy (support) thn those of 2 (i.e., the inconsistencies tht 1 would introduce require more serch steps thn the inconsistencies of 2 ). For this reson we hve refined E λ by using n heuristic estimtion (H) ofthe difficulty of supporting precondition which exploits informtion stored in the structure of the underlying plnning grph. More precisely, E λ (, A) i becomes E H (, A) i = λ p MAX f pre() H(f,A)+λ m me(, A), where H(f,A) is the heuristic cost of supporting f, nd is recursively defined in the following wy: 0 if f is supported H(f,A) = H(f, A) if f is no-op with precondition f MAX H(f, A)+me( f, A)+1otherw. f pre( f ) where f = ARGMIN { A f } { E(, A) i} nd A f is the set of ction nodes of the plnning grph t the preceding the of f, tht hve f s one their effects nodes. Informlly, the heuristic cost of n unsupported fct f is determined by considering ll ctions t the preceding the of f whose insertion would support f. Among these ctions, we choose the one with the best evlution ( f ) ccording to the originl ction evlution function E. 3 H(f,A) is recursively computed by summing the number of mutex reltions introduced by f to the mximum of the heuristic costs of its unsupported preconditions (H(f, A)). The lst term +1 tkes into ccount the insertion of f to support f. Similrly, we refine the cost of removing n ction from A by considering the mximum heuristic cost of supporting fct tht becomes unsupported when is removed. More precisely, E H (, A) r = MAX f, λj p H(f,A ) j K where K is the set of pirs f, j such tht f is precondition of n ction node j which becomes unsupported by removing from A. In Figure 2 we give n exmple illustrting H. Ech ction node is ssocited with pir of numbers (in brckets) indicting the number of unsupported preconditions nd the number of mutex reltions involving the node. For exmple, 3 t l +1hs two unsupported preconditions nd is involved in 5 mutex reltions. Ech fct node is ssocited with number specifying the heuristic cost of supporting it. Suppose we wnt to determine the heuristic cost of the unsupported precondition P 1 t l +2. The ction with 3 When more thn one ction with minimum cost exists, if the no-op is one of these ctions, then we choose it, otherwise we choose rndomly mong them. AIPS

6 Grph s l+2 l+1 l 1 (3,1) 4 (3,1) [0] P1 [5] 2 (1,1) 3 (2,5) 7 (3,3) P3 [3] P4 [0] 5 (0,2) 6 (2,1) (0, 0) P6 [0] P4 [0] (2,0) P2 [3] 8 (0,2) 9 (2,4) Figure 2: Exmple illustrting the heuristic cost of unsupported preconditions. Dshed boxes correspond to ctions/fcts which do not belong to A minimum cost chieving P 1 is 2. The ction with minimum cost chieving P 3 is 5, which hs ll preconditions supported. Thus we hve H(P 3, A) = MAX f pre( 5) H(f, A)+me( 5, A)+1 = Since H(P 4, A) =0,wehve H(P 1, A) = MAX f pre( 2) H(f, A)+me( 2, A)+1 = Modelling Pln Qulity In this section we extend our techniques to mnge plns where ech ction hs n rbitrry number ssocited with it representing, for exmple, its execution cost. While tking this informtion into ccount is prcticlly very importnt to produce good qulity plns, most of the domin-independent plnners in the literture neglect it, nd use the number of ctions (or prllel steps) s simple mesure of pln qulity. In our frmework execution costs cn be esily modelled by extending the objective function of the locl serch with term evluting the overll execution cost of pln (ction grph), which is nturlly defined s the sum of the costs of its ctions. Similrly, we cn esily model the number of prllel steps in pln, nd use this criterion s mesure of pln qulity. The user cn weight the optimiztion terms of the revised objective function (execution cost nd number of steps) ccording to the reltive importnce. More formlly, the generl evlution function F cn be revised to F + (A) =µ C Cost()+µ S Steps(A)+ A + A(λ m mutex(, A)+λ p precond(, A)) where Cost() is the execution cost of, µ C nd µ S re non-negtive coefficients set by the user to weight the reltive importnce of execution cost nd number of prllel steps (µ C +µ S 1), nd Steps(A) is the number of steps in the A-grph A contining t lest one ction different from no-ops. The evlution function E H defined in the previous section is refined to E + H in the following wy (U denotes the set of preconditions chieved by tht become unsupported by removing ): E + H (, A)i = E + H (, A)r = µ C mx CS (Cost() + MAX f pre() H C(f,A)) + + µ S (Steps(A + ) Steps(A) + mx CS MAX H S(f,A)) + f pre() + 1 mx E E H (, A) i µ C mx CS ( Cost()+MAX f U H C(f,A )) + + µ S (Steps(A ) Steps(A)+ mx CS MAX H S(f,A )) + f U + 1 E H (, A) r mx E where H C (f,a) nd H S (f,a) re n estimtion of the increse of the overll execution cost nd prllel steps, respectively, tht is required to support the fct f in A. H C nd H S re recursively defined in wy similr to the definition of H given in the previous section: if f is supported in A, then H C (f,a) =0nd H S (f,a) =0, otherwise H C (f,a) =Cost( f )+MAX f pre( f ) H C(f, A) H S (f,a) =Steps(A+ f ) Steps(A)+ MAX H S(f, A) f pre( f ) in which f is defined s in the definition of H. For exmple, consider gin Figure 2 nd suppose tht Cost( 2 )=10 nd Cost( 5 )=30.Wehve tht H C (P 1, A) =40. The fctors 1/mx CS, nd 1/mx E re used to normlize ech term of E + H to vlue less thn or equl to 1. Without this normliztion the vlues of the terms corresponding to the execution cost (first term) could be much higher thn the vlue corresponding to the constrint violtions (third term). This would guide the serch towrds good qulity plns without pying sufficient ttention to their vlidity. On the contrry, especilly when the current prtil pln contins mny constrint violtions, we would like tht the serch give more importnce to stisfying these logicl constrints. The vlue of mx CS is defined s µ C mx C + µ S mx S, where mx C (mx S )is the mximum vlue of the first (second) term of E + H over ll A-grphs (prtil plns) in the neighborhood, multiplied by the number κ of constrint violtions in the current ction 18 AIPS 2002

7 grph; mx E is defined s the mximum vlue of E H over ll possible ction insertions/removls tht eliminte the inconsistency under considertion. 4 The revised heuristic function E + H cn be used to produce succession of vlid plns where ech pln is n improvement of the previous ones in terms of execution cost or prllel steps. The first of these plns is the first pln π without constrint violtions tht is generted (i.e., the third term of F + is zero). π is then used to initilize second serch which is terminted when new vlid pln π tht is better thn π is generted. π is then used to initilize third serch, nd so on. This is n nytime process tht incrementlly improves the qulity of the plns, nd tht cn be stopped t ny time to give the best pln computed so fr. Ech time we strt new serch some inconsistencies re forced in the initil pln by rndomly removing some of its ctions. Similrly, some rndom inconsistencies re introduced during serch when vlid pln tht does not improve the pln of the previous serch is reched. This method of modelling pln qulity cn be extended to del with limited form of temporl plnning where ech ction is ssocited with temporl durtion D(). Under the ssumption tht the execution of ll ctions t ech step of the pln termintes before strting executing the ctions t the next step, we cn serch for prllel plns with the shortest possible durtion by including in F + new term tht is proportionl to the sum of the longest ction t ech step of the current prtil pln. Let Dely(, A) be the temporl dely determined by in the A-grph A, i.e., Dely(, A) =MAX(0,D() MAX S D( )), where S is the set of ctions t the sme of in A (except ). The new terms in E + H to hndle pln durtion re proportionl to +Dely(, A) + MAX H D(f,A),inE + H (, A)i f pre() Dely(, A) +MAX f U H D(f,A ),ine + H (, A)r where H D (f,a) is heuristic estimtion of the increse of the overll pln durtion tht is required to support f (the forml definition of H D (f,a) is similr to the definition of H C (f,a)). Despite the previous ssumption could be relxed in the computed pln by considering its cusl structure (dependencies between ction effects nd preconditions), in generl, it is not gurnteed tht minimizing the durtion term of F + lwys leds to the shortest possible prllel plns. On the other hnd, when the domin dmits only liner plns, minimizing the durtion term corresponds to serching for optiml plns. Hence, our simple pproch for durtion optimiztion should be seen s n pproximte technique. 4 The role of κ is to decrese the importnce of the first two optimiztion terms when the current pln contins mny constrint violtions, nd to increse it when the serch pproches vlid pln. For lck of spce we omit further detils on the formliztion of mx C, mx S nd mx E. Experimentl Results Our techniques re implemented in system clled LPG (Locl serch for Plnning Grphs). Using LPG we hve conducted three experiments. The first experiment ws imed t compring the performnce of Wlkpln in LPG nd other plnners bsed on plnning grphs for some well-known benchmrk problems. We considered Blckbox 3.9 with solvers Wlkst nd Chff (Moskewicz et l. 2001); GP-CSP (Kmbhmpti, Minh nd Do 2001); IPP4.1 (Koehler et l. 1997); nd STAN3s (Fox nd Long 1998; 1999). 5 The results of this experiment re in Tble 1. The 2nd nd the 3rd columns of the tble give the totl time required by LPG, nd the serch time of Wlkst for the SAT-encoding of the plnning grph with the minimum number of s tht is sufficient to solve the problem. 6 Wlkpln performed lwys better thn Wlkst, nd ws up to four orders of mgnitude fster. The fourth column of Tble 1 gives the results for Blckbox using the recent solver Chff insted of Wlkst, which currently is the fstest systemtic SAT solver. LPG ws significntly fster thn Blckbox lso in this cse. The remining columns of the tble give the CPUtimes for GP-CSP, IPP nd STAN, which were up to 3 or 4 orders of mgnitude slower thn LPG. (However, it should be noted tht these plnners, s well s Blckbox with Chff, compute plns with miniml number of steps.) In the second experiment we tested the performnce of LPG for some problems of the AIPS-98/00 competition set with respect to the plnner tht won the lst competition, FF (Hoffmnn nd Nebel 2001). The results re in Tble 2. 7 In terms of CPU-time LPG ws comprble to FF. In Logistics LPG performed better thn FF; in Elevtor (m10-problems) FF performed better thn LPG; while in Mprime we hve mixed results. 8 For mprime-05 FF required more thn 600 seconds, while LPG determined tht the problem is not solvble without serch (this ws detected during the construction of the plnning grph). In terms of number of ctions 5 All tests were run on Pentium II 400 MHz with 768 Mbytes. We did not use version 4 of STAN becuse it uses methods lterntive to plnning grphs. 6 Without this informtion the times required by Blckbox using Wlkst were significntly higher thn those in the tble. The prmeters of Wlkst (noise nd cutoff) were set to the combintion of vlues tht gve the best performnce over mny tries. For Blocks-world nd Gripper we used cutoff , vrious vlues of noise nd severl restrts. For log-d nd bw-lrge-/b we used the the option compct -b 8, becuse it mde the problems esier to solve. In this nd in the next experiments the noise fctor for Wlkpln ws lwys 0.1; the cutoff ws initilly set to 500, nd then utomticlly incremented t ech restrt by fctor of However, note tht compring these two systems is not completely fir, becuse FF computes liner plns while LPG computes prllel plns, nd hence they solve (computtionlly) different problems. 8 We considered slight modifiction of Mprime where opertor prmeters with different nmes re bound to different constnts, which is n implicit ssumption in the implementtion of both FF nd LPG. AIPS

8 () 0 Source City_8_1 pckge_4 pckge_3 City_1_3 Airplne1 Arplne2 City_5_9 pckge_2 City_9_8 pckge_1 Globl cost = (b) 0 Source City_8_1 pckge_4 pckge_3 City_1_3 Airplne1 Arplne2 City_5_9 pckge_2 City_9_8 pckge_1 Globl cost = (c) Source pckge_3 City_1_3 City_8_1 pckge_4 Airplne1 Arplne2 City_5_9 pckge_2 City_9_8 pckge_1 Globl cost = 2369 Figure 3: Three plns nd the reltive execution costs computed by FF (1st picture) nd by LPG (2nd nd 3rd pictures) for simple logistics problem with ction costs. Plnning LPG Blckbox GP- IPP STAN problem Wpln Wst Chff CSP rocket rocket-b log log-b log-c log-d bw-lrge bw-lrge-b TSP TSP TSP TSP out 11.9 gripper gripper Tble 1: Averge CPU-seconds over 25 runs required by LPG using Wlkpln (Wpln) with E H nd other plnners bsed on plnning grph (Wst indictes Wlkst). The times for LPG re totl serch times; the times for Wlkst refer only to the plnning grph with the minimum number of s where solution exists; the times for Chff nd GP-CSP re the totl times using the reltive defult settings, except for Gripper nd the hrdest TSP problems, where for Chff we used the settings suggested by Blckbox for hrd problems. mens tht the plnner did not find solution fter 1500 seconds (in these cses the verge ws over 8runs); out tht the system went out of memory. for the problems of Tble 2, FF-plns re better thn LPGplns except in 2 cses. We believe there re severl resons for this: LPG considers only one inconsistency t ech serch step, nd in this sense its serch is more locl thn FF serch, whose heuristic considers ll the problem gols; Wlkpln, like Wlkst, does not distinguish elements in the serch neighborhood tht improve the current A-grph; in this experiment the coefficient of the ction cost term in E H ws set to low vlue, becuse we were interested in the fst Plnning LPG FF problem Time Steps Act. Time Act. log (4.76) log (4.79) log (7.36) log (7.46) log (7.49) m10-s (0.81) m10-s (0.94) m10-s (1.10) m10-s (1.26) m10-s (1.45) mprime (1.92) mprime (8.72) mprime (3.90) mprime (1.56) mprime (2.43) > 600 Tble 2: CPU-seconds, number of steps nd number of ctions for LPG (Wlkpln) nd FF for some problems of the AIPS competitions. The dt for LPG re verges over 25 runs. The CPU-time in brckets is the time required for constructing the grph. This is lrgely dominted by the instntition of the opertors (especilly for Mprime), which in FF s implementtion is very fst nd in LPG cn be drsticlly reduced by using similr implementtion. computtion of the first solution. However, s we hve described in the previous section, LPG cn generte good qulity plns in more generl sense by minimizing the heuristic evlution function E + H. The third experiment tht we hve conducted concerns the use of E + H in LPG to produce plns of good qulity. In prticulr, we hve tested LPG with modifiction of the Logistics domin where ech ction hs n execution cost ssocited with it. Figure 3 shows three plns computed by FF nd LPG for simple problem in this domin (for the ske of clrity we show only the fly ctions). We hve two irplnes, four pckges, nd four cities, ech of which hs n irport, 20 AIPS 2002

9 Pln cost for TSP FF solution Pln cost for Logistics-b Pln cost for Logistics-b with connections Optiml cost CPU-SECONDS (log scle) FF solution Optiml cost CPU-SECONDS (log scle) FF solution Optiml cost CPU-SECONDS (log scle) Pln cost for Logistics Pln durtion for Logistics-16 in mixed optimiztion Pln cost for Logistics-16 in mixed optimiztion CPU-SECONDS (log scle) CPU-SECONDS (log scle) CPU-SECONDS (log scle) Figure 4: Incrementl qulity of the plns computed by LPG nd reltive CPU-time. The first three pictures plots refer to single problem instnces (10 runs of LPG for ech problem). Ech of the other three pictures plots verge mesures (cost or durtion) for 10 vrints of logistics-16. For ech vrint we considered the sequence of the first 11 plns generted. The points connected refer to the verge cost or durtion for the sme vrint. Averge dt re over 25 runs of the incrementl optimiztion process, for totl of 2750 plns in ech picture. port nd truck. Initilly, ll pckges nd irplnes re t the source city; the finl loctions of the pckges is shown in the pictures of the figure. Cities nd irports re plced in grid, nd the cost of flying between two irports depends the reltive Eucliden distnce in the grid (e.g., the distnce between city 9 8 to city 1 3 in Figure 3() is the distnce between points (9,8) nd (1,3) on the grid). The cost of flying by Airplne1 (fly1) is the trvel distnce times 100, while the cost of flying by Airplne2 (fly2) is the trvel distnce times 50 (i.e., fly1 is twice s expensive s fly2); loding nd unloding pckges on/from trucks or irplnes hve cost 1, driving truck between the port nd the irport of city hs cost 10. Figure 3() shows the fly ctions of the irplnes in the pln computed by FF, which does not tke into ccount ction costs, nd produces pln with overll cost Figures 3(b/c) show the fly ctions in the first two plns computed by run of LPG. The first pln hs overll cost 2664, the second On verge the optiml pln for this problem is computed by LPG in 0.85 seconds. Figure 4 gives results for TSP problem with 7 cities nd further logistics problems. The horizontl lines indicte the optiml execution costs, nd the cost of the solution computed by FF. In TSP-7 nd logistics-b cities nd the reltive irports were rndomly plced in grid of loctions, nd the ction costs were determined s in Figure 3. The third picture regrds modifiction of logistics-b in which only 60% of ll possible pirs of cities re connected by direct flights (in the originl version there is direct flight between ech pir of cities). This problem ws utomticlly generted by progrm tht rndomly genertes vrints of Logistics problem by ugmenting it with flight connections, nd ensuring tht ech pir of cities is connected vi some pth of possible flights. The plots of the first three pictures show the incresing pln qulity in terms of execution cost for 10 runs of LPG (using E + H with µ C = 1.0), indicting tht LPG quickly found the optiml solution, or solution close to the optiml one. The fourth picture of Figure 4 plots the verge execution costs for 10 rndomly generted vrints of more difficult problem from the AIPS-00 competition (logistics-16). The vrints differ in the initil position of the irports nd cities on grid of loctions, which ws rndomly determined. These results show tht the incrementl process for generting better qulity plns on verge significntly AIPS

10 improves the first plns produced (the more time is dedicted to LPG, the better the pln is). The lst two pictures of Figure 4 concern the minimiztion of pln durtion nd execution cost, respectively, when both these terms re included in the evlution function E + H. Action durtions were determined in the following wy: lod/unlod ctions hve durtion 1; the durtion of flying Airplne1 (the expensive one) is the sme s the trvel distnce; the durtion of flying Airplne2 is twice the trvel distnce. Improving pln durtion nd execution cost simultneously led to sequences of plns where these criteri do not lwys decrese monotoniclly, if considered independently. This is becuse in some cses less expensive plns might require longer execution. In generl, when we hve multiple optimiztion criteri, there is trdeoff mong the different criteri to consider. In LPG this trdeoff cn be tken into ccount by ppropritely weighting the optimiztion terms of the evlution function (in the tests of lst 2 plots of Figure 4 µ C ws 0.05 nd the weight for the temporl term 0.5). Conclusions We hve presented collection of techniques for locl serch on plnning grphs tht re implemented in LPG. The bsic serch scheme of LPG is similr to the stochstic serch of Wlkst, but hs different serch spce. While Wlkst uses simple generl heuristics for SAT problems, LPG uses severl techniques to define nd evlute the serch neighborhood tht exploit the structure of the underlying grph. These include Lgrnge multipliers, no-op propgtion, ction ordering nd incrementl grph length, nd heuristic precondition costs. In our frmework ech ction cn be ssocited with number representing its cost. The overll pln cost cn be esily modeled by using n pproprite evlution function, which cn be incrementlly minimized through n nytime process. Similrly, LPG cn produce plns of good qulity in terms of the number of prllel steps in the pln, nd the overll durtion of the pln. Experimentl results in severl benchmrk domins give n empiricl demonstrtion of the efficiency of our pproch, showing, in prticulr, tht LPG cn be orders of mgnitude fster thn other existing plnners bsed on plnning grphs. On the other hnd, the first pln computed by LPG cn be of poor qulity. When this is the cse, LPG produces dditionl plns tht incrementlly improve pln qulity in terms of one or more criteri specified in the evlution function. Current work include further experiments to test other locl serch schemes implemented in LPG (Tbupln nd T-Wlkpln), nd the development of new types of grph modifictions nd heuristics to guide the serch. Finlly, very recently n extended version of LPG hndling complex PDDL2.1 domins involving numericl quntities nd ction durtions ( prticipted in the 3rd Interntionl Plnning Competition, nd it ws wrded for distinguished performnce of the first order References Blum, A., nd Furst, M Fst plnning through plnning grph nlysis. Artificil Intelligence 90: Fox, M., nd Long, D Efficient implementtion of the Pln Grph in STAN. Journl of Artificil Intelligence reserch (JAIR), vol. 10, Fox, M., nd Long, D The detection nd exploittion of symmetry in plnning problems. In Proc. of the 16th Int. Joint Conf. on Artificil Intelligence (IJCAI-99). Gerevini, A., nd Serin, I Fst plnning through greedy ction grphs. In Proc. of the 16th Nt. Conf. of the Am. Assoc. for Artificil Intelligence (AAAI-99). Gerevini, A., nd Serin, I Lgrnge multipliers for locl serch on plnning grphs. In Proc. of the ECAI-2000 Workshop on Locl Serch for Plnning nd Scheduling. Springer-Verlg. Hoffmnn, J., nd Nebel, B The FF plnning system: Fst pln genertion through heuristic serch. Journl of Artificil Intelligence reserch (JAIR) 14: Kutz, H., nd Selmn, B Unifying SAT-bsed nd grph-bsed plnning. In Proc. of the 16th Interntionl Joint Conference on Artificil Intelligence (IJCAI-99). Kutz, H.; McAllester, D.; nd Selmn, B Encoding plns in propositionl logic. In Proc. of the 5th Int. Conf. on Principles of Knowledge Representtion nd Resoning (KR 96). Koehler, J.; Nebel, B.; Hoffmnn, J.; nd Dimopoulos, Y Extending plnning grphs to n ADL subset. In Proc. of Fourth Europen Conference on Plnning (ECP 97). Springer Verlg. Moskewicz, M.; Mdign, C.; Zho, Y.; Zhng, L.; nd Mlik, S Chff: Engineering n efficient SAT solver. In Proc. of 39th Design Automtion Conference. Kmbhmpti, R.; Minh, S.; nd Do, B Plnning s Constrint Stisfction: Solving the plnning grph by compiling it into CSP. Artificil Intelligence 132(2): Selmn, B.; Kutz, H.; nd Cohen, B Noise strtegies for improving locl serch. In Proc. of the Twelfth Nt. Conference of the Americn Assocition for Artificil Intelligence (AAAI-94), Shng, Y., nd Wh, B A discrete Lgrngin-bsed globl-serch method for solving stisfibility problems. Journl of Globl Optimiztion 12,1: Smith, D., nd Weld, D Temporl plnning with mutul exclusion resoning. In Proc. of the 16th Interntionl Joint Conference on Artificil Intelligence (IJCAI-99). Wh, B. W., nd Wu, Z. Oct The Theory of Discrete Lgrnge Multipliers for Nonliner Discrete Optimiztion. In Proc. of Principles nd Prctice of Constrint Progrmming. Springer-Verlg. Wu, Z., nd Wh, B An efficient globl-serch strtegy in discrete Lgrngin methods for solving hrd stisfibility problems. In Proc. of the 17th Nt. Conf. of the Americn Assoc. for Artificil Intelligence (AAAI-00). 22 AIPS 2002