An LP-Based Heuristic for Optimal Planning

Transcription

1 An LP-Bsed Heuristic for Optiml Plnning Menkes vn den Briel 1, J. Benton 2, Suro Kmhmpti 2, nd Thoms Vossen 3 Arizon Stte University, Deprtment of Industril Engineering 1, Deprtment of Computer Science nd Engineering 2, Tempe AZ, 85287, USA {menkes,j.enton,ro}@su.edu University of Colordo, Leeds School of Business 3, Boulder CO, 80309, USA vossen@colordo.edu Astrct. One of the most successful pproches in utomted plnning is to use heuristic stte-spce serch. A populr heuristic tht is used y numer of stte-spce plnners is sed on relxing the plnning tsk y ignoring the delete effects of the ctions. In severl plnning domins, however, this relxtion produces rther wek estimtes to guide serch effectively. We present relxtion using (integer) liner progrmming tht respects delete effects ut ignores ction ordering, which in numer of prolems provides etter distnce estimtes. Moreover, our pproch cn e used s n dmissile heuristic for optiml plnning. Key words: Automted plnning, improving dmissile heuristics, optiml relxed plnning 1 Introduction Mny heuristics tht re used to guide heuristic stte-spce serch plnners re sed on constructing relxtion of the originl plnning prolem tht is esier to solve. The ide is to use the solution to the relxed prolem to guide serch for the solution to the originl prolem. A populr relxtion tht hs een implemented y severl plnning systems, including UNPOP [17, 18], HSP [4, 5], nd FF [15], involves using relxed ctions in which the delete effects of the originl ctions re ignored. For exmple, FF estimtes the distnce etween n intermedite stte nd the gols y creting plnning grph [3] using relxed ctions. From this grph, FF extrcts in polynomil time relxed pln whose corresponding pln length is used s n indmissile, ut effective, distnce estimte. One cn trnsform this pproch into n dmissile heuristic y finding the optiml relxed pln, lso referred to s h + [14], ut computing such pln is NP-Complete [8]. In order to extrct the optiml relxed pln one must extend the relxed plnning grph to level off [3] so tht ll rechle ctions cn e considered. Although ignoring delete effects turns out to e quite effective for mny plnning domins, there re some ovious weknesses with FF s relxed pln

2 2 M.H.L. vn den Briel, J. Benton, S. Kmhmpti, T. Vossen heuristic. For exmple, in relxed pln no tom chnges more thn once, if n tom ecomes true it remins true s it is never deleted. However, in pln corresponding to the originl prolem n tom my e dded nd deleted severl times. In order to improve the qulity of the relxed pln we consider relxtion sed on relxed orderings. In prticulr, we view plnning prolem s set of intercting network flow prolems. Given plnning domin where sttes re defined in terms of n oolen or multi-vlued stte vriles (i.e. fluents), we view ech fluent s seprte flow prolem, where nodes correspond to the vlues of the fluent, nd rcs correspond to the trnsitions etween these vlues. While network flow prolems re computtionlly esy, wht mkes this flow prolem hrd is tht the flows re coupled s ctions cn cuse trnsitions in multiple fluents. We set up n IP formultion where the vriles correspond to the numer times ech ction is executed in the solution pln. The ojective is to minimize the numer of ctions, nd the constrints ensure tht ech pre-condition is supported. However, the constrints do not ensure tht pre-conditions re supported in correct ordering. Specificlly, for pre-conditions tht re deleted we setup lnce of flow constrints. Tht is, if there re m ction instnces in the pln tht cuse trnsition from vlue f of fluent c, then there must e m ction instnces in the pln tht cuse trnsition to vlue f of c. Moreover, for pre-conditions tht re not deleted we simply require tht they must e supported. The relxtion tht we pursue in this pper is tht we re not concerned out the specific positions of where the ctions occur in the pln. This cn, to some extent, e thought of s ignoring the ordering constrints tht ensure tht the ctions cn e linerized into fesile pln. An ttrctive spect of our formultion is tht it is not dependent on the length of the pln. Previous integer progrmming-sed formultions for plnning, such s [7, 9, 21], use step-sed encoding. In step-sed encoding the ide is to set up formultion for given pln length nd increment it if no solution cn e found. The prolem with such n encoding is tht it my ecome imprcticlly lrge, even for medium sized plnning tsks. In step-sed encoding, if l steps re needed to solve plnning prolem then l vriles re introduced for ech ction, wheres in our formultion we require only single vrile for ech ction. The estimte on the numer of ctions in the solution pln, s computed y our IP formultion, provides lower ound on the optiml (minimum numer of ctions) pln. However, since solving n IP formultion is known to e computtionlly intrctle, we use the liner progrmming (LP) relxtion which cn e solved in polynomil time. We will see tht this doule relxtion is still competitive with other dmissile heuristics fter we dd non-stndrd nd strong vlid inequlities to the formultion. In prticulr, We show tht the vlue of our LP-relxtion with dded inequlities, gives very good distnce estimtes nd in some prolem instnces even provides the optiml distnce estimte. While the current pper focuses on dmissile heuristics for optiml sequentil plnning, the flow-sed formultion cn e esily extended to del

3 An LP-Bsed Heuristic for Optiml Plnning 3 with more generl plnning prolems, including cost-sed plnning nd oversuscription plnning. In fct, generliztion of this heuristic leds to stteof-the-rt performnce in oversuscription plnning with non-uniform ctions costs, gol utilities s well s dependencies etween gol utilities [2]. This pper is orgnized s follows. In Section 2 we descrie our ction selection formultion nd descrie numer of helpful constrints tht exploit domin structure. Section 3 reports some experimentl results nd relted work is descried in Section 4. In Section 5 we summrize our min conclusions nd descrie some venues for future work. 2 Action Selection Formultion We find it useful to do the development of our relxtion in terms of multi-vlued fluents (of which the oolen fluents re specil cse). As such, we will use the SAS+ formlism [1] rther thn the usul STRIPS/ADL one s the ckground for our development. SAS+ is plnning formlism tht defines ctions y their previl-conditions nd effects. Previl-conditions descrie which vriles must propgte certin vlue during the execution of the ction nd effects descrie the pre- nd post-conditions of the ction. To mke the connection etween plnning nd network flows more strightforwrd, we will restrict our ttention to suclss of SAS+ where ech ction tht hs n post-condition on fluent lso hs pre-condition on tht fluent. We emphsize tht this restriction is mde for ese of exposition nd cn e esily removed; indeed our work in [2] voids mking this restriction. 2.1 Nottion We define SAS+ plnning tsk s tuple Π = C, A, s 0, s, where C = {c 1,..., c n } is finite set of stte vriles, where ech stte vrile c C hs n ssocited domin V c nd n implicitly defined extended domin V c + = V c {u}, where u denotes the undefined vlue. For ech stte vrile c C, s[c] denotes the vlue of c in stte s. The vlue of c is sid to e defined in stte s if nd only if s[c] u. The totl stte spce S = V c1... V cn nd the prtil stte spce S + = V c V c + n re implicitly defined. A is finite set of ctions of the form pre, post, prev, where pre denotes the pre-conditions, post denotes the post-conditions, nd prev denotes the previl-conditions. For ech ction A, pre[c], post[c] nd prev[c] denotes the respective conditions on stte vrile c. The following two restrictions re imposed on ll ctions: (1) Once the vlue of stte vrile is defined, it cn never ecome undefined. Hence, for ll c C, if pre[c] u then pre[c] post[c] u; (2) A previl- nd post-condition of n ction cn never define vlue on the sme stte vrile. Hence, for ll c C, either post[c] = u or prev[c] = u or oth.

4 4 M.H.L. vn den Briel, J. Benton, S. Kmhmpti, T. Vossen s 0 S denotes the initil stte nd s S + denotes the gol stte. We sy tht stte s is stisfied y stte t if nd only if for ll c C we hve s[c] = u or s[c] = t[c]. This implies tht if s [c] = u for stte vrile c, then ny defined vlue f V c stisfies the gol for c. While SAS+ plnning llows the initil stte, the gol stte nd the pre-conditions of n ction to e prtil, we ssume tht s 0 is totl stte nd tht ll preconditions re defined for ll stte vriles on which the ction hs post-conditions (i.e. pre[c] = u if nd only if post[c] = u). The ssumption tht s 0 is totl stte is common prctice in utomted plnning. However, the ssumption tht ll preconditions re defined is quite strong, therefore, we will riefly discuss wy to relx this second ssumption in Section 5. An importnt construct tht we use in our ction selection formultion is the so-clled domin trnsition grph [13]. A domin trnsition grph is grph representtion of stte vrile nd shows the possile wys in which vlues cn chnge. Specificlly, the domin trnsition grph DTG c of stte vrile c is leled directed grph with nodes for ech vlue f V c. DTG c contins leled rc (f 1, f 2 ) if nd only if there exists n ction with pre[c] = f 1 nd post[c] = f 2 or pre[c] = u nd post[c] = f 2. The rc is leled y the set of ctions with corresponding pre- nd post-conditions. For ech rc (f 1, f 2 ) with lel in DTG c we sy tht there is trnsition from f 1 to f 2 nd tht ction hs n effect in c. We use the following nottion. DTG c = (V c, E c ): is directed domin trnsition grph for every c C V c : is the set of possile vlues for ech stte vrile c C E c : is the set of possile trnsitions for ech stte vrile c C Vc V c represents the previl condition of ction in c Ec E c represents the effect of ction in c A E c := { A : Ec > 0} represents the ctions tht hve n effect in c, nd A E c (e) represents the ctions tht hve the effect e in c A V c := { A : Vc > 0} represents the ctions tht hve previl condition in c, nd A V c (f) represents the ctions tht hve the previl condition f in c V c + (f): to denote the in-rcs of node f in the domin trnsition grph G c; Vc (f): to denote the out-rcs of node f in the domin trnsition grph G c ; Moreover, we define the composition of two stte vriles, which is relted to the prllel composition of utomt [10], s follows. Definition 1. (Composition) Given the domin trnsition grph of two stte vriles c 1, c 2, the composition of DTG c1 nd DTG c2 is the domin trnsition grph DTG c1 c 2 = (V c1 c 2, E c1 c 2 ) where V c1 c 2 = V c1 V c2 ((f 1, g 1 ), (f 2, g 2 )) E c1 c 2 if f 1, f 2 V c1, g 1, g 2 V c2 nd there exists n ction A such tht one of the following conditions hold. pre[c 1 ] = f 1, post[c 1 ] = f 2, nd pre[c 2 ] = g 1, post[c 2 ] = g 2 pre[c 1 ] = f 1, post[c 1 ] = f 2, nd prev[c 2 ] = g 1, g 1 = g 2

5 An LP-Bsed Heuristic for Optiml Plnning 5 pre[c 1 ] = f 1, post[c 1 ] = f 2, nd g 1 = g 2 We sy tht DTG c1 c 2 is the composed domin trnsition grph of DTG c1 nd DTG c2. Exmple. Consider the set of ctions A = {,, c, d} nd set of stte vriles C = {c 1, c 2 } whose domin trnsition grphs hve V c1 = {f 1, f 2, f 3 }, V c2 = {g 1, g 2 } s the possile vlues, nd E c1 = {(f 1, f 3 ), (f 3, f 2 ), (f 2, f 1 )}, E c2 = {(g 1, g 2 ), (g 2, g 1 )} s the possile trnsitions s shown in Figure 1. Moreover, A E c 1 = {,, c}, A E c 2 = {, d} re the ctions tht hve n effect in c 1 nd c 2 respectively, nd A V c 1 =, A V c 2 = {} re the ctions tht hve previl condition in c 1 nd c 2 respectively. The effect nd previl condition of ction re represented y Ec 1 = (f 1, f 3 ) nd Vc 2 = g 1 respectively nd the set of in-rcs for node g 1 is given y V c + 2 (g 1 ) = {(g 2, g 1 )}. Note tht, since previl conditions do not chnge the vlue of stte vrile, we do not consider them to e trnsitions. The common ctions in the composed domin trnsition grph, tht is, ctions in A E c 1 A E c 2 cn only e executed simultneously in the two domin trnsition grphs. Hence, in the composition the two domin trnsition grphs re synchronized on the common ctions. The other ctions, those in A E c 1 \A E c 2 A E c 2 \A E c 1, re not suject to such restriction nd cn e executed whenever possile. f 1,,g 1 d f 3,g 2 c f 1,g 2 f 1 f 2 c g 1 d f 3,g 1 c d f 2,g 1 f 3 g 2 f 2,g 2 DTG c1 DTG c2 DTG c1 c2 Fig.1. Two domin trnsition grphs nd their composition. 2.2 Formultion Our ction selection formultion models ech domin trnsition grph in the plnning domin s n ppropritely defined network flow prolem. Interctions etween the stte vriles, which re introduced y the pre-, post-, nd previlconditions of the ctions, re modeled s side constrints on the network flow

6 6 M.H.L. vn den Briel, J. Benton, S. Kmhmpti, T. Vossen prolems. The vriles in our formultion indicte how mny times n ction is executed, nd the constrints ensure tht ll the ction pre-, post-, nd previlconditions must e respected. Becuse we ignore ction ordering, we re solving relxtion on the originl plnning prolem. Our relxtion, however, is quite different from the more populr relxtion tht ignores the delete effects of the ctions. Vriles. We define two types of vriles. We crete one vrile for ech ground ction nd one for ech stte vrile vlue. The ction vriles indicte how mny times ech ction is executed nd the vriles representing stte vrile vlues indicte which vlues re chieved t the end of the solution pln. The vriles re defined s follows. x Z +, for A; x 0 is equl to the numer of times ction is executed. y c,f {0, 1}, for c C, f V c ; y c,f is equl to 1 if the vlue f in stte vrile c is chieved t the end of the solution pln, nd 0 otherwise. Ojective function. The ojective function tht we use minimizes the numer of ctions. Note, however, tht we cn del with ction costs y simply multiplying ech ction vrile with cost prmeter c. Gol utilities cn e delt with y including the summtion c C,f V u c:f=s [c] c,fy c,f, where u c,f denotes the utility prmeter for ech gol. x (1) A Constrints. We define three types of constrints. Gol constrints ensure tht the gols in the plnning tsk re chieved. This is done y fixing the vriles corresponding to gol vlues to one. Effect impliction constrints define the network flow prolems of the stte vriles. These constrints ensure tht the effect of ech ction (i.e. trnsition in the domin trnsition grph) is supported y the effect of some other ction. Tht is, one my execute n ction tht deletes certin vlue if nd only if one executes n ction tht dds tht vlue. These constrints lso ensure tht ll gols re supported. The previl condition impliction constrints ensure tht the previl conditions of n ction must e supported y the effect of some other ction. The M in these constrints denotes lrge constnt nd llows ctions with previl conditions to e executed multiple times s long s their previl condition is supported t lest once. Note tht, the initil stte utomticlly dds the vlues tht re present in the initil stte. Gol constrints for ll c C, f V c : f = s [c] y c,f = 1 (2)

7 An LP-Bsed Heuristic for Optiml Plnning 7 Effect impliction constrints for ll c C, f V c e V + c (f): A E c (e) x + 1{if f = s 0 [c]} = e V c (f): A E c (e) x + y c,f (3) Previl condition impliction constrints for ll c C, f V c : A V c (f) e V + c (f): A E c (e) x + 1{if f = s 0 [c]} x /M (4) One gret dvntge of this IP formultion over other step-sed encodings is its size. The ction selection formultion requires only one vrile per ction, wheres step-sed encoding requires one vrile per ction for ech pln step. In step-sed encoding, if l steps re needed to solve plnning prolem then l vriles re introduced for ech ction. Note tht ny fesile pln stisfies the ove constrints. In fesile pln ll gols re stisfied, which is expressed y the constrints (2). In ddition, in fesile pln n ction is executle if nd only if its pre-conditions nd previl conditions re supported, which is expressed y constrints (3) nd (4). Since ny fesile will stisfy the constrints ove, the formultion provides relxtion to the originl plnning prolem. Hence, n optiml solution to this formultion provides ound (i.e. n dmissile heuristic) on the optiml solution of the originl plnning prolem. 2.3 Adding Constrints y Exploiting Domin Structure We cn sustntilly improve the qulity of LP-relxtion of the ction selection formultion y exploiting domin structure in the plnning prolem. In order to utomticlly detect domin structure in plnning prolem we use the soclled cusl grph [22]. The cusl grph is defined s directed grph with nodes for ech stte vrile nd directed rcs from source vriles to sink vriles if chnges in the sink vrile hve conditions in the source vrile. In other words, there is n rc in the cusl grph if there exists n ction tht hs n effect in the source vrile nd n effect or previl condition in the source vrile. We differentite etween two types of rcs y creting n effect cusl grph nd previl cusl grph s follows ([16] use leled rcs to mke the sme distinction). Definition 2. (Effect cusl grph) Given plnning tsk Π = C, A, s 0, s, the effect cusl grph G effect Π = (V, E effect ) is n undirected grph whose vertices correspond to the stte vriles of the plnning tsk. G effect Π contins n edge (c 1, c 2 ) if nd only if there exists n ction tht hs n effect in c 1 nd n effect in c 2. Definition 3. (Previl cusl grph) Given plnning tsk Π = C, A, s 0, s, the previl cusl grph G previl Π = (V, E previl ) is directed grph whose nodes

8 8 M.H.L. vn den Briel, J. Benton, S. Kmhmpti, T. Vossen correspond to the stte vriles of the plnning tsk. Gprevil Π contins directed rc (c 1, c 2 ) if nd only if there exists n ction tht hs previl condition in c 1 nd n effect in c 2. By nlyzing the effect cusl grph, the previl cusl grph, nd the domin trnsition grphs of the stte vriles, we re le to tighten the constrints of the integer progrmming formultion nd improve the vlue of the corresponding LP relxtion. In prticulr, we dd constrints to our formultion if certin (glol) cusl structure nd (locl) ction sustructures re present in the cusl grphs nd domin trnsition grphs respectively. Exmple. The effect cusl grph nd previl cusl grph corresponding to the exmple shown in Figure 1 is given y Figure 2. Since ction hs n effect in stte vriles c 1 nd c 2 there is n edge (c 1, c 2 ) in G effect Π. Similrly, since ction hs n effect in c 1 nd previl condition in c 2 there is n rc (c 2, c 1 ) in G previl Π. c 1 c 2 c 1 c 2 G effect G previl G Fig.2. The effect cusl grph nd previl cusl grph corresponding to Figure 1. Type 1 Domin Structure Constrints. The first set of domin structure constrints tht we dd to the ction selection formultion dels with cycles in the cusl grph. Cusl cycles re undesirle s they descrie two-wy dependency etween stte vriles. Tht is, chnges in stte vrile c 1 will depend on conditions in stte vrile c 2, nd vice vers. It is possile tht cusl cycles involve more thn two stte vriles, ut we only consider 2- cycles (i.e. cycles of length two). A cusl 2-cycle my pper in the effect cusl grph nd in the previl cusl grph. Note tht, since the effect cusl grph is undirected, ny edge in the effect cusl grph corresponds to 2-cycle. For every 2-cycle involving stte vriles c 1 nd c 2 we crete the composition DTG c1 c 2 if the following conditions hold. For ll A E c 1 we hve (A E c 2 A V c 2 ) For ll A E c 2 we hve (A E c 1 A V c 1 ) In other words, for every ction tht hs n effect in stte vrile c 1 (c 2 ) we hve tht hs n effect or previl condition in stte vrile c 2 (c 1 ). This condition will restrict the composition to provide complete synchroniztion of the two domin trnsition grphs. Now, for ech composed domin trnsition grph tht is creted we define n ppropritely defined network flow prolem.

9 An LP-Bsed Heuristic for Optiml Plnning 9 The corresponding constrints ensure tht the two-wy dependencies etween stte vriles c 1 nd c 2 re respected. Type 1 domin constrints for ll c 1, c 2 C such tht DTG c1 c 2 is defined nd f V c1, g V c2 e V + c 1 c 2 (f,g): A E c 1 c 2 (e) x + 1{if f = s 0 [c 1 ] g = s 0 [c 2 ]} = e V c 1 c 2 (f,g): A E c 1 c 2 (e) x (5) Exmple. In order to provide some intuition s to why these constrints re importnt nd help improve the ction selection formultion, consider the following scenrio. Assume we re given the set of ctions A = {, } nd the set of stte vriles C = {c 1, c 2 }, such tht E c 1 = (f 1, f 2 ), E c 2 = (g 2, g 3 ), E c 1 = (f 2, f 3 ), nd E c 2 = (g 1, g 2 ). The effect impliction constrint (3) llows the effect of ction support the effect of ction in c 1, nd it llows the effect of ction support the effect of ction in c 2. However, in the composed domin trnsition grph, it is cler tht neither ction or cn support ech other. Hence, if ctions nd re selected in the solution pln, then the solution pln must include one or more other ctions in order to stisfy the network flow constrints in the composed domin trnsition grph. f 3,g 3 f 1,,g 1 f 1,g 2 f 1 g 1 f 3,g 2 f 1,g 3 f 2 g 2 f 3,g 1 f 2,g 1 f 3 g 3 f 2,g 3 f 2,g 2 DTG c1 DTG c2 DTG c1 c2 Fig.3. Actions nd cn oth support ech other in either DTG c1 nd DTG c2, ut not in DTG c1 c 2.

10 10 M.H.L. vn den Briel, J. Benton, S. Kmhmpti, T. Vossen Type 2 Domin Structure Constrints. The second set of domin structure constrints tht we dd to the ction selection formultion dels with the structure given in Figure 4. Tht is, we hve n rc (c 1, c 2 ) in the previl cusl grph tht is not in 2-cycle. In ddition, we hve pir of ctions nd tht hve different previl conditions in c 1 nd different effects in c 2, such tht supports in c 2. Since ctions nd re mutex (ction deletes post-condition of ction ) they cnnot e executed in prllel. Therefore, in solution pln we must hve tht either is executed efore, or tht is executed efore. If the solution pln executes efore, then the network flow prolem corresponding to stte vrile c 1 must hve flow out of f 1. On the other hnd, if is executed efore, then the network flow prolem corresponding to stte vrile c 2 must hve flow out of g 3. These flow conditions my seem rther ovious, they re ignored y the ction selection formultion. Therefore, we dd the following constrints to our formultion to ensure tht the flow conditions with respect to the domin structure given in Figure 4 re stisfied. Type 2 domin constrints for ll c 1, c 2 C such tht (c 1, c 2 ) E previl, (c 2, c 1 ) / E previl nd f 1, f 2 V c1, g 1, g 2, g 3 V c2, e 1 V c + 2 (g 2 ), e 2 Vc 2 (g 2 ) : A E c 2 (e 1 ), A V c 2 (e 2 ), g 3 = hed(e 2 ) x + x 1 e V c 1 (f 1): A E c (e) x + e V c 2 (g 3): A E c (e) x (6) g 3 f 2 g 2 c 1 c 2 f 1 g 1 G previl DTGc1 DTGc2 Fig.4. Domin structure for type 2 domin structure constrints. 3 Experimentl Results In this section, we give generl ide of the distnce estimtes tht oth dmissile nd indmissile heuristics provide on set of plnning enchmrks from the interntionl plnning competitions (IPCs). In prticulr, we will compre the results of our ction selection formultion (with nd without the domin structure constrints) with four distnce estimtes: (1) the dmissile distnce

11 An LP-Bsed Heuristic for Optiml Plnning 11 estimte given y step sed formultion tht is very similr to Lpln [9], (2) the dmissile distnce estimte h +, which represents the length of the optiml relxed pln in which the delete effects of the ctions re ignored [14], (3) the indmissile distnce estimte h FF, which represents the relxed pln heuristic of the FF plnner [15], nd (4) the optiml distnce estimte given y Stplnner [19] using the -opt flg. We use Logistics nd Freecell from IPC2, Driverlog nd Zenotrvel from IPC3, nd TPP from IPC5. In ddition, we included few results on well known Blocksworld prolems. We focus on these domins minly ecuse we ssume tht ll pre-conditions re defined. There re severl plnning domins where this ssumption does not hold which limits our experimenttion. In Section 5, however, we riefly discuss how we cn relx this ssumption. All our tests were performed on 2.67GHz Linux mchine with 1GB of memory using 15 minute timeout. The heuristics tht use liner progrmming were solved using ILOG CPLEX 10.1 [11], commercil LP/IP solver. Tle 1 summrizes the results. The results represent the distnce estimte in terms of the numer of ctions from the initil stte to the gols. LP nd LP shows the results of our ction selection formultion with nd without the domin structure constrints respectively. Lpln shows the results of formultion tht is very similr Lpln (the ctul plnner is not pulicly ville). They were otined y running step-sed encoding tht finds minimum length plns. The vlues in this column represent the pln length t which the LP-relxtion returns fesile solution. h + shows the length of the optiml relxed pln nd h FF shows the length of FF s extrcted relxed pln. Finlly, Optiml shows the results of Stplnner using the -opt flg, which returns the minimum length pln. Note tht, Stplnner does not solve relxtion, ut the ctul plnning prolem, so the vlues in this column represent the optiml distnce estimte. When compring the results of LP with Lpln nd h +, we see tht in mny prolem instnces LP provides etter distnce estimtes. However, there re few instnces in the Freecell nd Blocksworld domins in which oth Lpln nd h + provide etter estimtes. The ction selection formultion does clerly outperform oth Lpln nd h + in terms of sclility nd time to solve the LPrelxtion. Lpln, generlly tkes the most time to solve ech prolem instnce s it spends time finding solution on pln length for which no fesile solution exists. Both Lpln nd h + fil to solve severl of the lrge instnces within the 15 minutes timeout. The LP-relxtion of the ction selection formultion typiclly solves ll smll nd some medium sized prolem instnces in less thn one second, ut the lrgest prolem instnces tke severl minutes to solve nd on the lrgest TPP prolem instnces it times out t 15 minutes. When we compre the results of LP with h FF we re compring the difference etween n dmissile nd n indmissile heuristic. The heuristic computtion of FF s relxed pln is very fst s it solves most prolem instnces in frction of second. However, the distnce estimte it provides is not dmissile s it cn overestimte the minimum distnce etween two sttes (see for exmple, driverlog7 nd zenotrvel6). The results tht our ction selection formultion

12 12 M.H.L. vn den Briel, J. Benton, S. Kmhmpti, T. Vossen provides re dmissile nd thus cn e used in n optiml serch lgorithm. Moreover, in some prolem instnces the qulity of our distnce estimte is outstnding. For exmple, in the Logistics, Driverlog, nd Zenotrvel domins, the distnce estimte given y LP equls the optiml distnce in ll prolem instnces for which Stplnner found the optiml solution. Finlly, when compring the results of LP with LP we see tht the domin structure constrints help improve the vlue of the LP-relxtion in mny prolem instnces except in instnces in the Freecell nd Blocksworld domins. Both these domins seem to hve domin structure tht we hve not cptured yet in our constrints. While this my seem like prolem, we rther tke this s chllenge, s we elieve tht more domin structure cn e exploited. 4 Relted Work Admissile heuristics for optiml plnning (such s, minimize the numer of ctions in the solution pln or minimize the cost of the ctions in the solution pln) re very scrce nd often provide poor distnce estimtes. On the other hnd, indmissile heuristics re plentiful nd hve shown to e very effective in solving utomted plnning prolems. HSPr [12] is one of the few pproches tht descries dmissile heuristics for plnning. HSPr cretes n ppropritely defined shortest-pth prolem to estimte the distnce etween two sttes. Our work differs from HSPr s we use liner progrmming to solve relxtion of the originl plnning prolem. The use of liner progrmming s n dmissile heuristic for optiml plnning ws introduced y the Lpln plnning system [9]. However, the ide ws never incorported in other plnning systems due to poor performnce results. Lpln sets up n IP formultion for step-sed plnning. Thus, when the LP-relxtion of the IP hs solution for pln length l, ut not for pln length l 1, then the minimum length pln must e t lest l steps long. The drwcks with the LP-relxtion to step-sed encoding is tht gol chievement cn e ccumulted over different pln steps. In generl, the qulity of the LP-relxtion of n IP formultion depends on how the prolem is formulted ([21] descrie the importnce of developing strong IP formultions in utomted plnning). 5 Conclusions We descried n integer progrmming formultion whose LP-relxtion cn e used s n dmissile heuristic for optiml plnning, including plnning prolems tht involve costs nd utilities. In fct, in ongoing work we hve successfully incorported our LP-sed heuristic in serch lgorithm tht solves oversuscription plnning prolems [2]. Our ction selection formultion nd the heuristic it provides differs in two wys from other formultions tht hve een used in plnning: (1) we do not use step sed encoding, nd so, do not hve to del with ound on the pln length in the IP formultion, nd (2) we ignore ction ordering, which provides

13 An LP-Bsed Heuristic for Optiml Plnning 13 rther different view on relxed plnning thn the more populr pproch tht ignores the delete effects of the ctions. The experimentl results show tht the ction selection formultion oftentimes provides etter distnce estimtes thn step-sed encoding tht is similr to Lpln [9]. It outperforms this step-encoding with respect to sclility nd solution time, mking it vile distnce estimte for heuristic stte-spce plnner. Moreover, in most prolem instnces it outperforms h + [14], which provides the optiml relxed pln length when delete effects re ignored. Hence, the relxtion sed on ignoring ction orderings seems to e stronger thn the relxtion sed on ignoring delete effects. Unlike most dmissile heuristics tht hve een descried in the plnning literture, we cn use the ction selection formultion to provide n dmissile distnce estimte for vrious optimiztion prolems in plnning, including ut not limited to, minimizing the numer of ctions, minimizing the cost of ctions, mximizing the numer of gols, nd mximizing the gol utilities. There re severl interesting directions tht we like to explore in future work. First, we would like to relx the ssumption tht ll preconditions re defined. This would llow us to crete generl ction selection formultion nd tckle much roder rnge of plnning domins. We simply need to replce the current ction vriles with vriles tht represent the ction effects the ction previl conditions. In cse n ction hs one or more undefined pre-conditions we crete one effect vrile for ech possile vlue tht the pre-condition my tke nd introduce n extr previl vrile s well. We hve preliminry implementtion of this generl ction selection formultion [2], ut hve not yet extended it with the domin structure constrints. Second, it would e interesting to nlyze plnning domins more crefully nd see if there re more domin structures tht we cn exploit. We lredy hve encountered two domins, nmely Freecell nd Blocksworld, tht my suggest tht other domin structures could e discovered. Acknowledgements. This reserch is supported in prt y the NSF grnt IIS308139, the ONR grnt N , nd y the Lockheed Mrtin sucontrct TT to Arizon Stte University s prt of the DARPA integrted lerning progrm. References 1. Bäckström, C., Neel, B.: Complexity results for SAS+ plnning. Computtionl Intelligence, Vol 11(4), (1995) Benton, J., vn den Briel, M.H.L., Kmhmpti, S.: A hyrid liner progrmming nd relxed pln heuristic for prtil stisfction plnning prolems. To pper: In Proceedings of the 17th Interntionl Conference on Automted Plnning nd Scheduling (2007) 3. Blum, A., Furst, M.: Fst plnning through plnning grph nlysis. In Proceedings of the 14th Interntionl Joint Conference on Artificil Inteligence (1995)

14 14 M.H.L. vn den Briel, J. Benton, S. Kmhmpti, T. Vossen 4. Bonet, B., Loerincs, G., Geffner, H.: A fst nd roust ction selection mechnism for plnning. In Proceedings of the 14th Ntionl Conference on Artificil Intelligence (1997) Bonet, B., Geffner, H.: Plnning s heuristic serch. Aritificil Intelligence, Vol 129(1). (2001) Bote A., Müller M., Scheffer J.: Fst plnning with itertive mcros. In Proceedings of the 20th Interntionl Joint Conference on Artificil Intelligence (2007) vn den Briel, M.H.L., Kmhmpti, S., Vossen, T.: Reviving integer progrmming Approches for AI plnning: A rnch-nd-cut frmework. In Proceedings of the 15th Interntionl Conference on Automted Plnning nd Scheduling (2005) Bylnder, T.: The computtionl complexity of propositionl STRIPS plnning. Artificil Intelligence, Vol 26(1-2). (1995) Bylnder, T.: A liner progrmming heuristic for optiml plnning. In Proceedings of the 14th Ntionl Conference on Artificil Intelligence (1997) Cssndrs, C.G., Lfortune, S.: Introduction to Discrete Event Systems. Kluwer Acdemic Pulishers (1999) 11. ILOG Inc. ILOG CPLEX 8.0 user s mnul. Mountin View, CA (2002) 12. Hslum, P., Geffner, H.: Admissile heuristics for optiml plnning. In Proceedings of the Interntionl Conference on Artificil Intelligence Plnning nd Scheduling (2000) 13. Helmert, M.: The Fst Downwrd plnning system. Journl of Artificil Intelligence Reserch, Vol 26. (2006) Hoffmnn, J.: Where ignoring delete lists works: Locl serch topology in plnning enchmrks. Journl of Artificil Intelligence Reserch, Vol 24. (2005) Hoffmnn, J. Neel, B.: The FF plnning system: Fst pln genertion through heuristic serch. Journl of Artificil Intelligence Reserch, Vol 14. (2001) Jonsson, P., Bäckström, C.: Trctle pln existence does not imply trctle pln genertion. Annls of Mthemtics nd Artificil Intelligence, Vol 22(3). (1998) McDermott, D.: A heuristic estimtor for mens-ends nlysis in plnning. In Proceedings of the 3rd Interntionl Conference on Artificil Intelligence Plnning Systems. (1996) McDermott, D.: Using regression-mtch grphs to control serch in plnning. Artificil Intelligence, Vol 109(1-2). (1999) Rintnen, J., Heljnko, K., Niemelä, I.: Plnning s stisfiility: prllel plns nd lgorithms for pln serch. Alert-Ludwigs-Universitt Freiurg, Institut fr Informtik, Technicl report 216. (2005) 20. Vidl, V.: A lookhed strtegy for heuristic serch plnning. In Proceedings of the 14th Interntionl Conference on Automted Plnning nd Scheduling (2004) Vossen, T., Bll, B., Lotem, A., Nu, D.S.: On the use of integer progrmming models in AI plnning. In Proceedings of the 18th Interntionl Joint Conference on Artificil Intelligence. (1999) Willims, B.C., Nyk, P.P.: A rective plnner for model-sed executive. In Proceedings of the 15th Interntionl Joint Conference on Artificil Intelligence. (1997)

15 An LP-Bsed Heuristic for Optiml Plnning 15 Prolem LP LP Lpln h + h F F Optiml logistics logistics logistics logistics logistics logistics logistics logistics logistics freecell freecell freecell freecell freecell freecell freecell freecell freecell driverlog driverlog driverlog driverlog driverlog driverlog driverlog driverlog driverlog zenotrvel zenotrvel zenotrvel zenotrvel zenotrvel zenotrvel zenotrvel zenotrvel zenotrvel tpp tpp tpp tpp tpp tpp tpp tpp tpp w-sussmn w-12step w-lrge w-lrge Tle 1. Distnce estimtes from the initil stte to the gol (vlues shown shown in old equl the optiml distnce). A dsh - indictes timeout of 15 minutes, nd str indictes tht the vlue ws rounded to the nerest deciml.