Loosely Coupled Formulations for Automated Planning: An Integer Programming Perspective

Transcription

1 Journal o Artiicial Intellience Researc 3 (008) 7-57 Submitted 09/07; publised 0/08 Loosely Coupled Formulations or Automated Plannin: An Inteer Prorammin Perspective Menkes H.L. van den Briel Department o Industrial Enineerin Arizona State University, Tempe, AZ 858 USA Tomas Vossen Leeds Scool o Business University o Colorado at Boulder, Boulder CO, USA menkes@asu.edu vossen@colorado.edu Subbarao Kambampati Department o Computer Science and Enineerin Arizona State University, Tempe, AZ 858 USA rao@asu.edu Abstract We represent plannin as a set o loosely coupled network low problems, were eac network corresponds to one o te state variables in te plannin domain. Te network nodes correspond to te state variable values and te network arcs correspond to te value transitions. Te plannin problem is to ind a pat (a sequence o actions) in eac network suc tat, wen mered, tey constitute a easible plan. In tis paper we present a number o inteer prorammin ormulations tat model tese loosely coupled networks wit varyin derees o lexibility. Since merin may introduce exponentially many orderin constraints we implement a so-called branc-and-cut aloritm, in wic tese constraints are dynamically enerated and added to te ormulation wen needed. Our results are very promisin, tey improve upon previous plannin as inteer prorammin approaces and lay te oundation or inteer prorammin approaces or cost optimal plannin.. Introduction Wile inteer prorammin approaces or automated plannin ave not been able to scale well aainst oter compilation approaces (i.e. satisiability and constraint satisaction), tey ave been extremely successul in te solution o many real-world lare scale optimization problems. Given tat te inteer prorammin ramework as te potential to incorporate several important aspects o real-world automated plannin problems (or example, numeric quantities and objective unctions involvin costs and utilities), tere is siniicant motivation to investiate more eective inteer prorammin ormulations or classical plannin as tey could lay te roundwork or lare scale optimization (in terms o cost and resources) in automated plannin. In tis paper, we study a novel decomposition based approac or automated plannin tat yields very eective inteer prorammin ormulations.. We use te term inteer prorammin to reer to inteer linear prorammin unless stated oterwise. c 008 AI Access Foundation. All rits reserved.

2 Van den Briel, Vossen & Kambampati Decomposition is a eneral approac to solvin problems more eiciently. It involves breakin a problem up into several smaller subproblems and solvin eac o te subproblems separately. In tis paper we use decomposition to break up a plannin problem into several interactin (i.e. loosely coupled) components. In suc a decomposition, te plannin problem involves bot indin solutions to te individual components and tryin to mere tem into a easible plan. Tis eneral approac, owever, prompts te ollowin questions: () wat are te components, () wat are te component solutions, and (3) ow ard is it to mere te individual component solutions into a easible plan?. Te Components We let te components represent te state variables o te plannin problem. Fiure illustrates tis idea usin a small loistics example, wit one truck and a packae tat needs to be moved rom location to location. Tere are a total o ive components in tis example, one or eac state variable. We represent te components by an appropriately deined network, were te network nodes correspond to te values o te state variable (or atoms tis is T =trueandf = alse), and te network arcs correspond to te value transitions. Te source node in eac network, represented by a small in-arc, corresponds to te initial value o te state variable. Te sink node(s), represented by double circles, correspond to te oal value(s) o te state variable. Note tat te eects o an action can trier value transitions in te state variables. For example, loadin te packae at location makes te atom pack-in-truck true and pack-at-loc alse. In addition, loadin te packae at location requires tat te atom truck-at-loc istrue. Wile te idea o components representin te state variables o te plannin problem can be used wit any state variable representation, it is particularly syneristic wit multivalued state variables. Multi-valued state variables provide a more compact representation o te plannin problem tan teir binary-valued counterparts. Tereore, by makin te conversion to multi-valued state variables we can reduce te number o components and create a better partitionin o te constraints. Fiure illustrates te use o multi-valued state variables on our small loistics example. Tere are two multi-valued state variables in tis problem, one to caracterize te location o te truck and one to caracterize te location o te packae. In our network representation, te nodes correspond to te state variable values ( = at-loc, = at-loc, and t = in-truck), and te arcs correspond to te value transitions.. Te Component Solutions We let te component solutions represent a pat o value transitions in te state variables. In te networks, nodes and arcs appear in layers. Eac layer represents a plan period in wic, dependin on te structure o te network, one or more value transitions can occur. Te networks in Fiures and eac ave tree layers (i.e. plan periods) and teir structure allows values to persist or cane exactly once per period. Te layers are used to solve te plannin problem incrementally. Tat is, we start wit one layer in eac network and try to solve te plannin problem. I no plan is ound, all networks are extended by one extra layer and a new attempt is made to solve te plannin problem. Tis process is repeated until a plan is ound or a time limit is reaced. In Fiures and, a pat (i.e. 8

3 Loosely Coupled Formulations or Automated Plannin loc loc truck-at-loc T Load at loc T Drive loc loc T - T F F F F truck-at-loc T - T Drive loc loc T Unload at loc T F F F F pack-at-loc T Load at loc T - - T T F F F F pack-at-loc T - T - T Unload at loc T F F F F pack-in-truck T Load at loc T - T Unload at loc T F F F F Fiure : Loistics example broken up into ive components (binary-valued state variables) tat are represented by network low problems. truck-location pack-location Load at loc Load at loc Drive loc loc - t t t t Unload at loc Unload at loc Fiure : Loistics example broken up into two components (multi-valued state variables) tat are represented by network low problems. a solution) rom te source node to one o te sink nodes is ilited in eac network. Since te execution o an action triers value transitions in te state variables, eac pat in a network corresponds to a sequence o actions. Consequently, te plannin problem can be tout o as a collection o network low problems were te problem is to ind a pat (i.e. a sequence o actions) in eac o te networks. However, interactions between 9

4 Van den Briel, Vossen & Kambampati te networks impose side constraints on te network low problems, wic complicate te solution process..3 Te Merin Process We solve tese loosely coupled networks usin inteer prorammin ormulations. One desin coice we make is tat we expand all networks (i.e. components) toeter, so te cost o indin solutions or te individual networks as well as merin tem depends on te diiculty o solvin te inteer prorammin ormulation. Tis, in turn, typically depends on te size o te inteer prorammin ormulation, wic is partly determined by te number o layers in eac o te networks. Te simplest idea is to ave te number o layers o te networks equal te lent o te plan, just as in sequential plannin were te plan lent equals te number o actions in te plan. In tis case, tere will be as many transitions in te networks as tere are actions in te plan, wit te only dierence tat a sequence o actions correspondin to a pat in a network could contain no-op actions. An idea to reduce te required number o layers is by allowin multiple actions to be executed in te same plan period. Tis is exactly wat is done in Grapplan (Blum & Furst, 995) and in oter planners tat ave adopted te Grapplan-style deinition o parallelism. Tat is, two actions can be executed in parallel (i.e. in te same plan period) as lon as tey are non-intererin. In our ormulations we adopt more eneral notions o parallelism. In particular, we relax te strict relation between te number o layers in te networks and te lent o te plan by canin te network representation o te state variables. For example, by allowin multiple transitions in eac network per plan period we permit intererin actions to be executed in te same plan period. Tis, owever, raises issues about ow solutions to te individual networks are searced and ow tey are combined. Wen te network representations or te state variables allow multiple transitions in eac network per plan period, and tus become more lexible, it becomes arder to mere te solutions into a easible plan. Tereore, to evaluate te tradeos in allowin suc lexible representations, we look at a variety o inteer prorammin ormulations. We reer to te inteer prorammin ormulation tat uses te network representation sown in Fiures and as te one state cane model, because it allows at most one transition (i.e. state cane) per plan period in eac state variable. Note tat in tis network representation a plan period mimics te Grapplan-style parallelism. Tat is, two actions can be executed in te same plan period i one action does not delete te precondition or add-eect o te oter action. A more lexible representation in wic values can cane at most once and persist beore and ater eac cane we reer to as te eneralized one state cane model. Clearly, we can increase te number o canes tat we allow in eac plan period. Te representations in wic values can cane at most twice or k times, we reer to as te eneralized two state cane and te eneralized k state cane model respectively. One disadvantae wit te eneralized k state cane model is tat it creates one variable or eac way to do k value canes, and tus introduces exponentially many variables per plan period. Tereore, anoter network representation tat we consider allows a pat o value transitions in wic eac value can be visited at most once per plan period. Tis way, we can limit te number o variables, but may introduce cycles in our networks. Te 0

5 Loosely Coupled Formulations or Automated Plannin inteer prorammin ormulation tat uses tis representation is reerred to as te state cane pat model. In eneral, by allowin multiple transitions in eac network per plan period (i.e. layer), te more complex te merin process becomes. In particular, te merin process cecks weter te actions in te solutions o te individual networks can be linearized into a easible plan. In our inteer prorammin ormulations, orderin constraints ensure easible linearizations. Tere may, owever, be exponentially many orderin constraints wen we eneralize te Grapplan-style parallelism. Rater tan insertin all tese constraints in te inteer prorammin ormulation up ront, we add tem as needed usin a branc-and-cut aloritm. A branc-and-cut aloritm is a branc-and-bound aloritm in wic certain constraints are enerated dynamically trouout te branc-and-bound tree. We sow tat te perormance o our inteer prorammin (IP) ormulations sow new potential and are competitive wit SATPLAN04 (Kautz, 004). Tis is a siniicant result because it orms a basis or oter more sopisticated IP-based plannin systems capable o andlin numeric constraints and non-uniorm action costs. In particular, te new potential o our IP ormulations as led to teir successul use in solvin partial satisaction plannin problems (Do, Benton, van den Briel, & Kambampati, 007). Moreover, it as initiated a new line o work in wic inteer and linear prorammin are used in euristic state-space searc or automated plannin (Benton, van den Briel, & Kambampati, 007; van den Briel, Benton, Kambampati, & Vossen, 007). Te remainder o tis paper is oranized as ollows. In Section we provide a brie backround on inteer prorammin and discuss some approaces tat ave used inteer prorammin to solve plannin problems. In Section 3 we present a series o inteer prorammin ormulations tat eac adopt a dierent network representation. We describe ow we set up tese loosely coupled networks, provide te correspondin inteer prorammin ormulation, and discuss te dierent variables and constraints. In Section 4 we describe te branc-and-cut aloritm tat is used or solvin tese ormulations. We provide a eneral backround on te branc-and-cut concept and sow ow we apply it to our ormulations by means o an example. Section 5 provides experimental results to determine wic caracteristics in our approac ave te reatest impact on perormance. Related work is discussed in Section 6 and some conclusions are iven in Section 7.. Backround Since our ormulations are based on inteer prorammin, we briely review tis tecnique and discuss its use in plannin. A mixed inteer proram is represented by a linear objective unction and a set o linear inequalities: min{cx : Ax b, x,..., x p 0andinteer,x p+,..., x n 0}, were A is an (m n) matrix,c is an n-dimensional row vector, b is an m-dimensional column vector, and x an n-dimensional column vector o variables. I all variables are continuous (p =0)weavealinear proram, i all variables are inteer (p = n) weave an inteer proram, andix,..., x p {0, } we ave a mixed 0- proram. Te set S = {x,..., x p 0andinteer,x p+,..., x n 0:Ax b} is called te easible reion, andan n-dimensional column vector x is called a easible solution i x S. Moreover, te unction

6 Van den Briel, Vossen & Kambampati cx is called te objective unction, and te easible solution x is called an optimal solution i te objective unction is as small as possible, tat is, cx =min{cx : x S} Mixed inteer prorammin provides a ric modelin ormalism tat is more eneral tan propositional loic. Any propositional clause can be represented by one linear inequality in 0- variables, but a sinle linear inequality in 0- variables may require exponentially many clauses (Hooker, 988). Te most widely used metod or solvin (mixed) inteer prorams is by applyin a branc-and-bound aloritm to te linear prorammin relaxation, wic is muc easier to solve. Te linear prorammin (LP) relaxation is a linear proram obtained rom te oriinal (mixed) inteer proram by relaxin te interality constraints: min{cx : Ax b, x,..., x n 0} Generally, te LP relaxation is solved at every node in te branc-and-bound tree, until () te LP relaxation ives an inteer solution, () te LP relaxation value is inerior to te current best easible solution, or (3) te LP relaxation is ineasible, wic implies tat te correspondin (mixed) inteer proram is ineasible. An ideal ormulation o an inteer proram is one or wic te solution o te linear prorammin relaxation is interal. Even tou every inteer proram as an ideal ormulation (Wolsey, 998), in practice it is very ard to caracterize te ideal ormulation as it may require an exponential number o inequalities. In problems were te ideal ormulation cannot be determined, it is oten desirable to ind a stron ormulation o te inteer proram. Suppose tat te easible reions P = {x R n : A x b } and P = {x R n : A x b } describe te linear prorammin relaxations o two IP ormulations o a problem. Ten we say tat ormulation or P is stroner tan ormulation or P i P P. Tat is, te easible reion P is subsumed by te easible reion P.Inoter words P improves te quality o te linear relaxation o P by removin ractional extreme points. Tere exist numerous powerul sotware packaes tat solve mixed inteer prorams. In our experiments we make use o te commercial solver CPLEX 0.0 (Inc., 00), wic is currently one o te best LP/IP solvers. Te use o inteer prorammin tecniques to solve artiicial intellience plannin problems as an intuitive appeal, especially iven te success IP as ad in solvin similar types o problems. For example, IP as been used extensively or solvin problems in transportation, loistics, and manuacturin. Examples include crew scedulin, veicle routin, and production plannin problems (Jonson, Nemauser, & Savelsber, 000). One potential advantae is tat IP tecniques can provide a natural way to incorporate several important aspects o real-world plannin problems, suc as numeric constraints and objective unctions involvin costs and utilities. Plannin as inteer prorammin as, neverteless, received only limited attention. One o te irst approaces is described by Bylander (997), wo proposes an LP euristic or partial order plannin aloritms. Wile te LP euristic elps to reduce te number o expanded nodes, te evaluation is rater time-consumin. In eneral, te perormance o. Wile te inteer prorammin problem is NP-complete (Garey & Jonson, 979) te linear prorammin problem is polynomially solvable (Karmarkar, 984).

7 Loosely Coupled Formulations or Automated Plannin IP oten depends on te structure o te problem and on ow te problem is ormulated. Te importance o developin stron IP ormulations is discussed by Vossen et al. (999), wo compare two ormulations or classical plannin: () a straitorward ormulation based on te conversion o te propositional representation by SATPLAN wic yields only mediocre results, and () a less intuitive ormulation based on te representation o state transitions wic leads to considerable perormance improvements. Several ideas tat urter improve ormulation based on te representation o state transitions are described by Dimopoulos (00). Some o tese ideas are implemented in te IP-based planner Optiplan (van den Briel & Kambampati, 005). Approaces tat rely on domain-speciic knowlede are proposed by Bockmayr and Dimopoulos (998, 999). By exploitin te structure o te plannin problem tese IP ormulations oten provide encourain results. Te use o LP and IP as also been explored or non-classical plannin. Dimopoulos and Gerevini (00) describe an IP ormulation or temporal plannin and Wolman and Weld (999) use LP ormulations in combination wit a satisiability-based planner to solve resource plannin problems. Kautz and Walser (999) also solve resource plannin problems, but use domain-speciic IP ormulations. 3. Formulations Tis section describes our IP ormulations tat model te plannin problem as a collection o loosely coupled network low problems. Eac network represents a state variable, in wic te nodes correspond to te state variable values, and te arcs correspond to te value transitions. Te state variables are based on te SAS+ plannin ormalism (Bäckström & Nebel, 995), wic is a plannin ormalism tat uses multi-valued state variables instead o binary-valued atoms. An action in SAS+ is modeled by its pre-, post- and prevailconditions. Te pre- and post-conditions express wic state variables are caned and wat values tey must ave beore and ater te execution o te action, and te prevailconditions speciy wic o te uncaned variables must ave some speciic value beore and durin te execution o an action. A SAS+ plannin problem is described by a tuple Π= C, A, s 0,s were: C = {c,..., c n } is a inite set o state variables, were eac state variable c C as an associated domain V c and an implicitly deined extended domain V c + = V c {u}, were u denotes te undeined value. For eac state variable c C, s[c] denotes te value o c in state s. Tevalueoc is said to be deined in state s i and only i s[c] u. Te total state space S = V c... V cn and te partial state space S + = V c +... V c + n are implicitly deined. A is a inite set o actions o te orm pre, post, prev, werepre denotes te preconditions, post denotes te post-conditions, and prev denotes te prevail-conditions. For eac action a A, pre[c],post[c] and prev[c] denotes te respective conditions on state variable c. Te ollowin two restrictions are imposed on all actions: () Once te value o a state variable is deined, it can never become undeined. Hence, or all c C, ipre[c] u ten pre[c] post[c] u; () A prevail- and post-condition o an action can never deine a value on te same state variable. Hence, or all c C, eiter post[c] =u or prev[c] =u or bot. 3

8 Van den Briel, Vossen & Kambampati s 0 S denotes te initial state and s S + denotes te oal state. Wile SAS+ plannin allows te initial state and oal state to be bot partial states, we assume tat s 0 is a total state and s is a partial state. We say tat state s is satisied by state t i and only i or all c C we ave s[c] =u or s[c] =t[c]. Tis implies tat i s [c] =u or state variable c, ten any deined value V c satisies te oal or c. To obtain a SAS+ description o te plannin problem we use te translator component o te Fast Downward planner (Helmert, 006). Te translator is a stand-alone component tat contains a eneral purpose aloritm wic transorms a propositional description o te plannin problem into a SAS+ description. Te aloritm provides an eicient roundin tat minimizes te state description lent and is based on te preprocessin aloritm o te MIPS planner (Edelkamp & Helmert, 999). In te remainder o tis section we introduce some notation and describe our IP ormulations. Te ormulations are presented in suc a way tat tey proressively eneralize te Grapplan-style parallelism trou te incorporation o more lexible network representations. For eac ormulation we will describe te underlyin network, and deine te variables and constraints. We will not concentrate on te objective unction as muc because te constraints will tolerate only easible plans. 3. Notation For te ormulations tat are described in tis paper we assume tat te ollowin inormation is iven: C: a set o state variables; V c : a set o possible values (i.e. domain) or eac state variable c C; E c : a set o possible value transitions or eac state variable c C; G c =(V c,e c ) : a directed domain transition rap or every c C; State variables can be represented by a domain transition rap, were te nodes correspond to te possible values, and te arcs correspond to te possible value transitions. An example o te domain transition rap o a variable is iven in Fiure 3. Wile te example depicts a complete rap, a domain transition rap does not need to be a complete rap. Furtermore, we assume as iven: E a c E c represents te eect o action a in c; V a c V c represents te prevail condition o action a in c; A E c := {a A : Ec a > 0} represents te actions tat ave an eect in c, anda E c (e) represents te actions tat ave te eect e in c; A V c := {a A : Vc a > 0} represents te actions tat ave a prevail condition in c, and A V c () represents te actions tat ave te prevail condition in c; C a := {c C : a A E c AV c } represents te state variables on wic action a as an eect or a prevail condition. 4

9 Loosely Coupled Formulations or Automated Plannin Fiure 3: An example o a domain transition rap, were V c = {,,} are te possible values (states) o c and E c = {(,), (,), (, ), (, ), (, ), (, )} are te possible value transitions in c. Hence, eac action is deined by its eects (i.e. pre- and post-conditions) and its prevail conditions. In SAS+ plannin, actions can ave at most one eect or prevail condition in eac state variable. In oter words, or eac a A and c C, weavetatec a and V c a are empty or Ec a + V c a. An example o ow te eects and prevail conditions aect one or more domain transition raps is iven in Fiure 4. Fiure 4: An example o ow action eects and prevail conditions are represented in a domain transition rap. Action a as implications on tree state variables C a = {c,c,c 3 }. Te eects o a are represented by E a c = {(,)} and E a c = {(, )}, and te prevail condition o a is represented by V a c 3 = {}. In addition, we use te ollowin notation: V + c (): to denote te in-arcs o node in te domain transition rap G c; V c (): to denote te out-arcs o node in te domain transition rap G c ; P + c,k (): to denote pats o lent k in te domain transition rap G c tat end at node. NotetatP c, + () =V + c (). P c,k (): to denote pats o lent k in te domain transition rap G c tat start at node. NotetatPc, () =V c (). 5

10 Van den Briel, Vossen & Kambampati P c,k (): to denote pats o lent k in te domain transition rap G c tat visit node, but tat do not start or end at. 3. One State Cane (SC) Formulation Our irst IP ormulation incorporates te network representation tat we ave seen in Fiures and. Te name one state cane relates to te number o transitions tat we allow in eac state variable per plan period. Te restriction o allowin only one value transition in eac network also restricts wic actions we can execute in te same plan period. It appens to be te case tat te network representation o te SC ormulation incorporates te standard notion o action parallelism wic is used in Grapplan (Blum & Furst, 995). Te idea is tat actions can be executed in te same plan period as lon as tey do not delete te precondition or add-eect o anoter action. In terms o value transitions in state variables, tis is sayin tat actions can be executed in te same plan period as lon as tey do not cane te same state variable (i.e. tere is only one value cane or value persistence in eac state variable). 3.. State Cane Network Fiure 5 sows a sinle layer (i.e. period) o te network wic underlies te SC ormulation. I we set up te IP ormulation wit T plan periods, ten tere will be T +layers o nodes and T layers o arcs in te network (te zerot layer o nodes is or te initial state and te remainin T layers o nodes and arcs are or te successive plan periods). For eac possible state transition tere is an arc in te state cane network. Te orizontal arcs correspond to te persistence o a value, and te diaonal arcs correspond to te value canes. A solution pat to an individual network ollows te arcs wose transitions are supported by te action eect and prevail conditions tat appear in te solution plan. SC network Period t Fiure 5: One state cane (SC) network. 3.. Variables We ave two types o variables in tis ormulation: action variables to represent te execution o an action, and arc low variables to represent te state transitions in eac network. 6

11 Loosely Coupled Formulations or Automated Plannin We use separate variables or canes in a state variable (te diaonal arcs in te SC network) and or te persistence o a value in a state variable (te orizontal arcs in te SC network). Te variables are deined as ollows: x a t {0, }, ora A, t T ; x a t is equal to i action a is executed at plan period t, and0oterwise. ȳ c,,t {0, }, orc C, V c, t T ;ȳ c,,t is equal to i te value o state variable c persists at period t, and0oterwise. y c,e,t {0, }, orc C, e E c, t T ; y c,e,t is equal to i te transition e E c in state variable c is executed at period t, and0oterwise Constraints Tere are two classes o constraints. We ave constraints or te network lows in eac state variable network and constraints or te action eects tat determine te interactions between tese networks. Te SC inteer prorammin ormulation is: State cane lows or all c C, V c { i = s0 [c] y c,e, +ȳ c,, = () 0 oterwise. e Vc () y c,e,t+ +ȳ c,,t+ = y c,e,t +ȳ c,,t or t T () e V c () e V c + () e V c + () y c,e,t +ȳ c,,t = i = s [c] (3) Action implications or all c C, t T x a t = y c,e,t or e E c (4) a A:e E a c x a t ȳ c,,t or a A, V a c (5) Constraints (), (), and (3) are te network low constraints or state variable c C. Constraint () ensures tat te pat o state transitions beins in te initial state o te state variable and constraint (3) ensures tat, i a oal exists, te pat ends in te oal state o te state variable. Note tat, i te oal value or state variable c is undeined (i.e. s [c] =u) ten te pat o state transitions may end in any o te values V c. Hence, we do not need a oal constraint or te state variables wose oal states s [c] are undeined. Constraint () is te low conservation equation and enorces te continuity o te constructed pat. Actions may introduce interactions between te state variables. For instance, te eects o te load action in our loistics example aect two dierent state variables. Actions link state variables to eac oter and tese interactions are represented by te action implication 7

12 Van den Briel, Vossen & Kambampati constraints. For eac transition e E c, constraints (4) link te action execution variables tat ave e as an eect (i.e. e Ec a ) to te arc low variables. For example, i an action x a t wit eect e Ec a is executed, ten te pat in state variable c must ollow te arc represented by y c,e,t. Likewise, i we coose to ollow te arc represented by y c,e,t,ten exactly one action x a t wit e Ea c must be executed. Te summation on te let and side prevents two or more actions rom intererin wit eac oter, ence only one action may cause te state cane e in state variable c at period t. Prevail conditions o an action link state variables in a similar way as te action eects do. Speciically, constraint (5) states tat i action a is executed at period t (x a t =),ten te prevail condition Vc a is required in state variable c at period t (ȳ c,,t =). 3.3 Generalized One State Cane (GSC) Formulation In our second ormulation we incorporate te same network representation as in te SC ormulation, but adopt a more eneral interpretation o te value transitions, wic leads to an unconventional notion o action parallelism. For te GSC ormulation we relax te condition tat parallel actions can be arraned in any order by requirin a weaker condition. We allow actions to be executed in te same plan period as lon as tere exists some orderin tat is easible. More speciically, witin a plan period a set o actions is easible i () tere exists an orderin o te actions suc tat all preconditions are satisied, and () tere is at most one state cane in eac o te state variables. Tis eneralization o conditions is similar to wat Rintanen, Heljanko and Niemelä (006) reer to as te -step semantics semantics. To illustrate te basic concept, let us aain examine our small loistics example introduced in Fiure. Te solution to tis problem is to load te packae at location, drive te truck rom location to location, and unload te packae at location. Clearly, tis plan would require tree plan periods under Grapplan-style parallelism as tese tree actions interere wit eac oter. I, owever, we allow te load at loc andtedrive loc loc action to be executed in te same plan period, ten tere exists some orderin between tese two actions tat is easible, namely load te packae at te location beore drivin te truck to location. Te key idea beind tis example sould be clear: wile it may not be possible to ind a set o actions tat can be linearized in any order, tere may neverteless exist some orderin o te actions tat is viable. Te question is, o course, ow to incorporate tis idea into an IP ormulation. truck-location pack-location Load at loc Drive loc loc Load at loc Unload at loc Unload at loc t t t Fiure 6: Loistics example represented by network low problems wit eneralized arcs. 8

13 Loosely Coupled Formulations or Automated Plannin Tis example illustrates tat we are lookin or a set o constraints tat allow sets o actions or wic: () all action preconditions are met, () tere exists an orderin o te actions at eac plan period tat is easible, and (3) witin eac state variable, te value is caned at most once. Te incorporation o tese ideas only requires minor modiications to te SC ormulation. Speciically, we need to cane te action implication constraints or te prevail conditions and add a new set o constraints wic we call te orderin implication constraints State Cane Network Te minor modiications are revealed in te GSC network. Wile te network itsel is identical to te SC network, te interpretation o te transition arcs is somewat dierent. To incorporate te new set o conditions, we implicitly allow values to persist (te dased orizontal arcs in te GSC network) at te tail and ead o eac transition arc. Te interpretation o tese implicit arcs is tat in eac plan period a value may be required as a prevail condition, ten te value may cane, and te new value may also be required as a prevail condition as sown in Fiure 7. GSC network Generalized state cane arc Period t Period t Fiure 7: Generalized one state cane (GSC) network Variables Since te GSC network is similar to te SC network te same variables are used, tus, action variables to represent te execution o an action, and arc low variables to represent te low trou eac network. Te dierence in te interpretation o te state cane arcs is dealt wit in te constraints o te GSC ormulation, and tereore does not introduce any new variables. For te variable deinitions, we reer to Section Constraints We now ave tree classes o constraints, tat is, constraints or te network lows in eac state variable network, constraints or linkin te lows wit te action eects and prevail conditions, and orderin constraints to ensure tat te actions in te plan can be linearized into a easible orderin. 9

14 Van den Briel, Vossen & Kambampati Te network low constraints or te GSC ormulation are identical to tose in te SC ormulation iven by ()-(3). Moreover, te constraints tat link te lows wit te action eects are equal to te action eect constraints in te SC ormulation iven by (4). Te GSC ormulation diers rom te SC ormulation in tat it relaxes te condition tat parallel actions can be arraned in any order by requirin a weaker condition. Tis weaker condition aects te constraints tat link te lows wit te action prevail conditions, and introduces a new set o orderin constraints. Tese constraints o te GSC ormulation are iven as ollows: Action implications or all c C, t T x a t ȳ c,,t + y c,e,t + e V c + () e Vc () y c,e,t or a A, V a c (6) Orderin implications a V (Δ) x a t V (Δ) or all cycles Δ G prec (7) Constraint (6) incorporates tis new set o conditions or wic actions can be executed in te same plan period. In particular, we need to ensure tat or eac state variable c, te value V c olds i it is required by te prevail condition o action a at plan period t. Tere are tree possibilities: () Te value olds or c trouout te period. () Te value olds initially or c, but te value is caned to a value oter tan by anoter action. (3) Te value does not old initially or c, but te value is caned to by anoter action. In eiter o te tree cases te value olds at some point in period t so tat te prevail condition or action a can be satisied. In words, te value may prevail implicitly as lon as tere is a state cane tat includes. As beore, te prevail implication constraints link te action prevail conditions to te correspondin network arcs. Te action implication constraints ensure tat te preconditions o te actions in te plan are satisied. Tis, owever, does not uarantee tat te actions can be linearized into a easible order. Fiure 7 indicates tat tere are implied orderins between actions. Actions tat require te value as a prevail condition must be executed beore te action tat canes into. Likewise, an action tat canes into must be executed beore actions tat require te value as a prevail condition. Te state cane low and action implication constraints outlined above indicate tat tere is an orderin between te actions, but tis orderin could be cyclic and tereore ineasible. To make sure tat an orderin is acyclic we start by creatin a directed implied precedence rap G prec =(V prec,e prec ). In tis rap te nodes a V prec correspond to te actions, tat is, V prec = A, andwe create a directed arc (i.e. an orderin) between two nodes (a, b) E prec i action a as to be executed beore action b in time period t, orib as to be executed ater a. Inparticular, we ave E prec = (a, b) (a, b) (a,b) A A,c C, V a c,e E b c: e V c, (a,b) A A,c C, V b c,e E a c : e V + c, 30

15 Loosely Coupled Formulations or Automated Plannin Te implied orderins become immediately clear rom Fiure 8. Te iure on te let depicts te irst set o orderins in te expression o E prec. It says tat te orderin between two actions a and b tat are executed in te same plan period is implied i action a requires a value to prevail tat action b deletes. Similarly, te iure on te rit depicts second set o orderins in te expression o E prec. Tat is, an orderin is implied i action a adds te prevail condition o b. a b a b Fiure 8: Implied orderins or te GSC ormulation. Te orderin implication constraints ensure tat te actions in te inal solution can be linearized. Tey basically involve puttin an n-ary mutex relation between te actions tat are involved in eac cycle. Unortunately, te number o orderin implication constraints rows exponentially in te number o actions. As a result, it will be impossible to solve te resultin ormulation usin standard approaces. We address tis complication by implementin a branc-and-cut approac in wic te orderin implication constraints are added dynamically to te ormulation. Tis approac is discussed in Section Generalized k State Cane (GkSC) Formulation In te GSC ormulation actions can be executed in te same plan period i () tere exists an orderin o te actions suc tat all preconditions are satisied, and () tere occurs at most one value cane in eac o te state variables. One obvious eneralization o tis would be to relax te second condition and allow at most k c value canes in eac state variable c, werek c V c. By allowin multiple value canes in a state variable per plan period we, in act, permit a series o value canes. Speciically, te GkSC model allows series o value canes. Obviously, tere is a tradeo between loosenin te networks versus te amount o work it takes to mere te individual plans. Wile we ave not implemented te GkSC ormulation, we provide some insit in tis tradeo by describin and evaluatin te GkSC ormulation wit k c =orallc C We will reer to tis special case as te eneralized two state cane (GSC) ormulation. One reason we restrict ourselves to tis special case is tat te eneral case o k state canes would introduce exponentially many variables in te ormulation. Tere are IP tecniques, owever, tat deal wit exponentially many variables (Desaulniers, Desrosiers, & Solomon, 005), but we will not discuss tem ere State Cane Network Te network tat underlies te GSC ormulation is equivalent to GSC, but spans an extra layer o nodes and arcs. Tis extra layer allows us to ave a series o two transitions per plan 3

16 Van den Briel, Vossen & Kambampati period. All transitions are eneralized and implicitly allow values to persist just as in te GSC network. Fiure 9 displays te network correspondin to te GSC ormulation. In te GSC network tere are eneralized one and two state cane arcs. For example, tere is a eneralized one state cane arc or te transition (,), and tere is a eneralized two state canes arc or te transitions {(, ), (, )}. Since all arcs are eneralized, eac value tat is visited can also be persisted. We also allow cyclic transitions, suc as, {(,), (, )} i is not te prevail condition o some action. I we were to allow cyclic transitions in wic is a prevail condition o an action, ten te action orderin in a plan period can not be implied anymore (i.e. te prevail condition on would eiter ave to occur beore te value transitions to, or ater it transitions back to ). Tus i tere is no prevail condition on ten we can saely allow te cyclic transition {(,), (, )}. state cane arcs state canes arcs Period t Period t Fiure 9: Generalized two state cane (GSC) network. On te let te subnetwork tat consists o eneralized one state cane arcs and no-op arcs, on te rit te subnetwork tat consists o te eneralized two state cane arcs. Te subnetwork or te two state cane arcs may include cyclic transitions, suc as, {(, ), (, )} as lon as is not te prevail condition o some action Variables As beore we ave variables representin te execution o an action, and variables representin te lows over one state cane (diaonal arcs) or persistence (orizontal arcs). In addition, we ave variables representin pats over two consecutive state canes. Hence, we ave variables or eac pair o state canes (,,) suc tat (,) E c and (, ) E c. We will restrict tese pats to visit unique values only, tat is,,, and, or i is not a prevail condition o any action ten we also allow pats were =. Te variables rom te GSC ormulation are also used in GSC ormulation. Tere is, owever, an additional variable to represent te arcs tat allow or two state canes: y c,e,e,t {0, }, or c C, (e,e ) P c,, t T ; y c,e,e,t isequaltoi tere exists a value V c and transitions e,e E c, suc tat e V c + () and e Vc (), in state variable c are executed at period t, and0oterwise. 3

17 Loosely Coupled Formulations or Automated Plannin Constraints We aain ave our tree classes o constraints, wic are iven as ollows: State cane lows or all c C, V c { i = s0 [c] y c,e,e, + y c,e, +ȳ c,, = (8) 0 oterwise. (e,e ) P c, () e Vc () y c,e,e,t+ + y c,e,t+ +ȳ c,,t+ = (e,e ) P c, () e Vc () y c,e,e,t + y c,e,t +ȳ c,,t or t T (9) (e,e ) P + c, () e V c + () y c,e,e,t + y c,e,t +ȳ c,,t = i { s [c]} (0) (e,e ) P + c, () e V c + () Action implications or all c C, t T x a t = y c,e,t + a A:e E a c (e,e ) P c, :e =e e =e y c,e,e,t or e E c () x a t ȳ c,,t + y c,e,t + y c,e,t + y c,e,e,t + e V c + () e Vc () (e,e ) Pc, () y c,e,e,t + y c,e,e,t or a A, Vc a () (e,e ) P + c, () (e,e ) P c, () Orderin implications a V (Δ) x a t V (Δ) or all cycles Δ G prec (3) Constraints (8), (9), and (0) represent te low constraints or te GSC network. Constraints () and () link te action eects and prevail conditions wit te correspondin lows, and constraint 3 ensures tat te actions can be linearized into some easible orderin. 3.5 State Cane Pat (PatSC) Formulation Tere are several ways to eneralize te network representation o te GSC ormulation and loosen te interaction between te networks. Te GkSC ormulation presented one eneralization tat allows up to k transitions in eac state variable per plan period. Since it uses exponentially many variables anoter way to eneralize te network representation o te GSC ormulation is by requirin tat eac value can be true at most once per plan period. To illustrate tis idea we consider our loistics example aain, but we now use 33

18 Van den Briel, Vossen & Kambampati truck-location Load at loc Drive loc loc Unload at loc pack-location - Load at loc - Unload at loc - t t Fiure 0: Loistics example represented by network low problems tat allow a pat o value transitions per plan period suc tat eac value can be true at most once. a network representation tat allows a pat o transitions per plan period as depicted in Fiure 0. Recall tat te solution to te loistics example consists o tree actions: irst load te packae at location, ten drive te truck rom location to location, and last unload te packae at location. Clearly, tis solution would not be allowed witin a sinle plan period under Grapplan-style parallelism. Moreover, it would also not be allowed witin a sinle period in te GSC ormulation. Te reason or tis is tat te number o value canes in te packae-location state variable is two. First, it canes rom pack-at-loc to pack-in-truck, and ten it canes rom pack-in-truck to pack-at-loc. As beore, owever, tere does exists an orderin o te tree actions tat is easible. Te key idea beind tis example is to sow tat we can allow multiple value canes in a sinle period. I we limit te value canes in a state variable to simple pats, tat is, in one period eac value is visited at most once, ten we can still use implied precedences to determine te orderin restrictions State Cane Network In tis ormulation eac value can be true at most once in eac plan period, ence te number o value transitions or eac plan period is limited to k c were k c = V c oreac c C. In te PatSC network, nodes appear in layers and correspond to te values o te state variable. However, eac layer now consists o twice as many nodes. I we set up an IP encodin wit a maximum number o plan periods T ten tere will be T layers. Arcs witin a layer correspond to transitions or to value persistence, and arcs between layers ensure tat all plan periods are connected to eac oter. Fiure displays a network correspondin to te state variable c wit domain V c = {,,} tat allows multiple transitions per plan period. Te arcs pointin ritwards correspond to te persistence o a value, wile te arcs pointin letwards correspond to te value canes. I more tan one plan period is needed te curved arcs pointin ritwards 34

19 Loosely Coupled Formulations or Automated Plannin link te layers between two consecutive plan periods. Note tat wit unit capacity on te arcs, any pat in te network can visit eac node at most once. PatSC network Period t Fiure : Pat state cane (PatSC) network Variables We now ave action execution variables and arc low variables (as deined in te previous ormulations), and linkin variables tat connect te networks between two consecutive time periods. Tese variables are deined as ollows: z c,,t {0, }, orc C, V c, 0 t T ; z c,,t is equal to i te value o state variable c is te end value at period t, and0oterwise Constraints As in te previous ormulations, we ave state cane low constraints, action implication constraints, and orderin implication constraints. Te main dierence is te underlyin network. Te PatSC inteer prorammin ormulation is iven as ollows: State cane lows or all c C, V c { i = s0 [c] z c,,0 = (4) 0 oterwise. y c,e,t + z c,,t = ȳ c,,t (5) e V c + () ȳ c,,t = e Vc () y c,e,t + z c,,t or t T (6) z c,,t = i s [c] (7) Action implications or all c C, t T x a t = y c,e,t or e E c (8) a A:e E a c x a t ȳ c,,t or V a c (9) 35

20 Van den Briel, Vossen & Kambampati Orderin implications a V (Δ) x a t V (Δ) or all cycles Δ G prec (0) Constraints (4)-(7) are te network low constraints. For eac node, except or te initial and oal state nodes, tey ensure a balance o low (i.e. low-in must equal low-out). Te initial state node as a supply o one unit o low and te oal state node as a demand o one unit o low, wic are iven by constraints (4) and (7) respectively. Te interactions tat actions impose upon dierent state variables are represented by te action implication constraints (8) and (9), wic ave been discussed earlier. Te implied precedence rap or tis ormulation is iven by G prec =(V prec,e prec ). It as an extra set o arcs to incorporate te implied precedences tat are introduced wen two actions imply a state cane in te same class c C. Te nodes a V prec aain correspond to actions, and tere is an arc (a, b) E prec i action a as to be executed beore action b in te same time period, or i b as to be executed ater a. More speciically, we ave E prec = E prec (a,b) A A,c C, V c,e Ec a,e Ec: b e V c + () e Vc () (a, b) As beore, te orderin implication constraints (0) ensure tat te actions in te solution plan can be linearized into a easible orderin. 4. Branc-and-Cut Aloritm IP problems are usually solved wit an LP-based branc-and-bound aloritm. Te basic structure o tis tecnique involves a binary enumeration tree in wic brances are pruned accordin to bounds provided by te LP relaxation. Te root node in te enumeration tree represents te LP relaxation o te oriinal IP problem and eac oter node represents a subproblem tat as te same objective unction and constraints as te root node except or some additional bound constraints. Most IP solvers use an LP-based branc-and-bound aloritm in combination wit various preprocessin and probin tecniques. In te last ew years tere as been siniicant improvement in te perormance o tese solvers (Bixby, 00). In an LP-based branc-and-bound aloritm, te LP relaxation o te oriinal IP problem (te solution to te root node) will rarely be inteer. Wen some inteer variable x as a ractional solution v we branc to create two new subproblems, suc tat te bound constraint x v is added to te let-cild node, and x v is added to te rit-cild node. Tis brancin process is carried out recursively to expand tose subproblems wose solution remains ractional. Eventually, ater enou bounds are placed on te variables, an inteer solution is ound. Te value o te best inteer solution ound so ar, Z, is reerred to as te incumbent and is used or prunin. 36

21 Loosely Coupled Formulations or Automated Plannin In a minimization problem, brances emanatin rom nodes wose solution value Z LP is reater tan te current incumbent, Z, can never ive rise to a better inteer solution as eac cild node as a smaller easible reion tan its parent. Hence, we can saely eliminate suc nodes rom urter consideration and prune tem. Nodes wose easible reion ave been reduced to te empty set, because too many bounds are placed on te variables, can be pruned as well. Wen solvin an IP problem wit an LP-based branc-and-bound aloritm we must consider te ollowin two decisions. I several inteer variables ave a ractional solution, wic variable sould we branc on next, and i te branc we are currently workin on is pruned, wic subproblem sould we solve next? Basic rules include use te most ractional variable rule or brancin variable selection and te best objective value rule or node selection. For our ormulations a standard LP-based branc-and-bound aloritm approac is very ineective due to te lare number (potentially exponentially many) orderin implication constraints in te GSC, GSC, and PatSC ormulations. Wile it is possible to reduce te number o constraints by introducin additional variables (Martin, 99), te resultin ormulations would still be intractable or all but te smallest problem instances. Tereore, we solve te IP ormulations wit a so-called branc-and-cut aloritm, wic considers te orderin implication constraints implicitly. A branc-and-cut aloritm is a branc-andbound aloritm in wic certain constraints are enerated dynamically trouout te branc-and-bound tree. A lowcart o our branc-and-cut aloritm is iven in Fiure. I, ater solvin te LP relaxation, we are unable to prune te node on te basis o te LP solution, te branc-and-cut aloritm tries to ind a violated cut, tat is, a constraint tat is valid but not satisied by te current solution. Tis is also known as te separation problem. I one or more violated cuts are ound, te constraints are added to te ormulation and te LP is solved aain. I none are ound, te aloritm creates a branc in te enumeration tree (i te solution to te current subproblem is ractional) or enerates a easible solution (i te solution to te current subproblem is interal). Te basic idea o branc-and-cut is to leave out constraints rom te LP relaxation o wic tere are too many to andle eiciently, and add tem to te ormulation only wen tey become bindin at te solution to te current LP. Branc-and-cut aloritms ave successully been applied in solvin ard lare-scale optimization problems in a wide variety o applications includin scedulin, routin, rap partitionin, network desin, and acility location problems (Caprara & Fiscetti, 997). In our branc-and-cut aloritm we can stop as soon as we ind te irst easible solution, or we can implicitly enumerate all nodes (trou prunin) and ind te optimal solution or a iven objective unction. Note tat our ormulations can only be used to ind bounded lent optimal plans. Tat is, ind te optimal plan iven a plan period (i.e. a bounded lent). In our experimental results, owever, we ocus on indin easible solutions. 4. Constraint Generation At any point durin runtime tat te cut enerator is called we ave a solution to te current LP problem, wic consists o te LP relaxation o te oriinal IP problem plus any added bound constraints and added cuts. In our implementation o te branc-and-cut 37