CMU-Q Lecture 5: Classical Planning Factored Representations STRIPS. Teacher: Gianni A. Di Caro

Transcription

1 CMU-Q Lecture 5: Classical Planning Factored Representations STRIPS Teacher: Gianni A. Di Caro

2 AUTOMATED PLANNING: FACTORED STATE REPRESENTATIONS 2

3 PLANNING, FOR (MORE) COMPLEX WORLDS Searching for plan of action to achieve one s goal is a critical part of AI (in both open and closed loop) In fact, planning is glorified search But search needs more powerful state representations than those used so far, in order to be effective So far: states are indivisible, they have no internal structure Planning exploiting structured representation of states And let s keep living in deterministic, known, fully observable, single agent worlds This is what is commonly termed Classical Planning 3

4 STATE REPRESENTATIONS So far Now (Structured) B C B C (a) Atomic (b) Factored (b) Structured 4

5 STATE REPRESENTATIONS The vacuum-world example 5

6 THE NEED FOR FACTORED STATES The goal is to reach the banana, but achieving the goal requires achieving, in the correct sequence, a number of sub-goals that overall make a Plan 6

7 PROPOSITIONAL STRIPS PLANNING STRIPS / PDDL language(s) to represent / solve planning problems based on propositional (factored) state representation STRIPS = Stanford Research Institute Problem Solver (1971) o Originally based on first-order logic, later simplified to include only propositional logic (Logic-based) Language expressive enough to describe a wide variety of problems, but restrictive enough to allow efficient algorithms to operate over it PDDL = Planning Domain Definition Language ( ), the standard language for defining planning domains and problems, it includes original STRIPS + more advanced features. Last version is 3.1, from Compact representation of planning. A state is a conjunction of propositions, e.g., at(truck 1,Shadyside) at(truck 2,Oakland) o Proposition: A statement that is either true or false à A fact o Predicate: a proposition that contains variables/parameters, such as at(truck, place) States are transformed via operators (actions) that have the form Preconditions Postconditions (effects) 7

8 RUNNING EXAMPLE: BLOCKS WORLD A C A B B C Start Goal Objects of the world: Block A Block B Block C Table Hand Predicates that can be used to describe the world: ontable(x) ( on(x, [Table Y] ) on(x, Y) clear(x) holding(x) handempty(x) Negation of all the above Actions: 8

9 REPRESENTING STATES AS SET OF FACTS World states are represented as sets of facts: conjunction of propositions (conditions) Fact(ored) representation of states! State 1 = { holding(a), clear(b), on(b,c), ontable(c)} Closed World Assumption (CWA): Facts not listed in a state are assumed to be false. Under CWA the assumption the agent has full observability and only positive facts need to be stated 9

10 STATES State: o Propositional literals: Poor Unknown o Ground first order literals: At(Plane 1, Rome) At(Plane 2, Tokyo) At(x, Rome) At(y, Tokyo) o Function-free: At(Father(Tom), NY) At(Alex, NY) Father(Alex, Tom) The world is represented through a set of features / objects (e.g., planes, people, cities) and each proposition states a fact that attributes values to features Objects State propositions CWA 10

11 REPRESENTING GOALS AS SET OF FACTS Also Goals (being world states) are represented as sets of facts Example: state { on(a,b) } can be set as a goal A goal state is any state that includes all the goal facts State 1 is a not goal state for the goal { on(a,b) } State 2 is a goal state for the goal { on(a,b) } 11

12 GOALS Goals: A conjunction of facts, At(P 1, JFK) At(P 2, SFO), that may also contain variables, such as: At(p, JFK) Plane(p) to have any plane at JFK The aim is to reach a state that entails a goal: OnTable(A) OnTable(B) OnTable(D) On(C, D) Clear(A) Clear(B) Clear(C) satisfies the goal to stack C on D A goal g is a conjunction of sub-goals! g = g 1 g 2 g n We can focus on getting individual sub-goals. Not possible in atomic representations! Goals are reached through sequences of actions (the plan) 12

13 ACTIONS Actions: Operators with Preconditions + Effects (Postconditions) Pre-cond is a conjunction of positive and negative conditions that must be satisfied to apply the operation Post-cond is a conjunction of positive and negative conditions that become true when the operation is applied PutDown(A,B): as PRE, robot hand is holding A + B s top is clear à the action puts A down on top of B A STRIPS action definition specifies: ü A set PRE of preconditions facts ü A set ADD of add effect facts (to state facts) ü A set DEL of delete effect facts (from state facts) In STRIPS only positive preconditions are used 13

14 EXAMPLE: MOVE OPERATOR 14

15 ACTION SCHEMA Action schema: a number of different actions that can be derived by universal quantification of the variables An action schema to fly a plane from one location to another: Action(Fly(p, from, to), PRECOND: At(p, from) Plane(p) Airport(from) Airport(to) EFFECT: At(p, from) At(p, to)) An action is applicable in state s if s entails the preconditions The facts negated by the effect of Action are removed from s, while the positive facts resulting from Action are added to s 15

16 ACTION SCHEMA Action schema: Action(Name(v 1, v 2,., v n ), PRECONDITIONS: P 1 (v) P 2 (v) P m (v) ADD-LIST: {F 1 (v), F 2 (v),., F q (v)} DELETE-LIST: {S i (p), S j (p) S k (p)} RESULT(s, a) = (s DELETE(a)) ADD(a) 16

17 (CLASSICAL, PROPOSITIONAL) PLANNING PROBLEM Planning domain: o Set of Action schemas (actions) o Set of Predicates (conjunction of predicates à states) Planning problem (instance): o Planning domain o Set of Objects (world features) o Initial state (facts/propositions about the objects) o Goal(s) Solution of the planning problem: A sequence of actions that, starting from the initial state, end in a state s that entails the goal 17

18 AUTOMATED PLANNING PROBLEM An action-state model P = < S, s +,-.,, S /0-1, A, T, c, G > S: the set of states (can be atomic, factorial) s start S: the initial state, in S S goal S : the subset of goal states, in S A : the set of possible actions, can be defined as A(s) T: S A S : the Successor / state Transition function c: S A R : the step cost for taking action a in state s G: S {0,1} : criterion to check whether or not at a goal / terminal state Solution plan: Path [s start, s 1, s 2, s goal ] associated to the feasible sequence of actions, [a 1, a 2, a n ] such that cost(path) is minimized 18

19 EXAMPLE: AIR CARGO TRANSPORTATION Air cargo transportation problem (from R&N) Predicates: At, Cargo, Plane, Airport, In Objects: C 1 (cargo container), C 2, P 1 (plane), P 2, SFO, JFK Actions: Load, Unload, Fly 19

20 BLOCKS WORLD C A Start B A B B C A C on(a,b), on(b,c) Goal A B C Plan: MoveToTable (C, A) MoveFromTable (B, C) MoveFromTable (A, B) MoveToTable(X, Y) Pre: clear(x) on(x,y) on(x,table) clear(x) on(x,y) Move (X, From, To): clear(x) on(x, From) clear(to) block(x) block(to) on(x,to) clear(to) on(x,from) MoveFromTable (X, Y) 20

21 COMPLEXITY OF PLANNING PLANSAT is the problem of determining whether a given planning problem is satisfiable In general PLANSAT is PSPACE-complete (~require an amount of space which is exponential in the size of the input) Bounded PlanSAT = decide if plan of given length exists (Bounded) PlanSAT decidable but PSPACE-hard Disallow neg effects: (Bounded) PlanSAT NP-hard Disallow neg preconditions: PlanSAT in P but finding optimal (shortest) plan still NP-hard 21

22 COMPLEXITY RESULTS FOR PLANSAT 22

23 PLANNING AS SEARCH (Forward) Search from initial state to goal Can use search techniques, including heuristic search At(P 1,A) At(P 2,A) Fly(P 1,A,B) Fly(P 2,A,B) At(P 1,B) At(P 2,A) At(P 1,A) At(P 2,B) 23

24 (FORWARD) STATE-SPACE SEARCH In absence of function symbols, the state space of a planning problem is finite Any graph search algorithm that is complete will be a complete planning algorithm Irrelevant action problem: All applicable actions are considered at each state! The resulting branching factor b is typically large and the state space is exponential in b Needs for good heuristics! At home get milk, bananas and a cordless drill return home 24

25 (FORWARD) STATE-SPACE SEARCH Air Cargo Example Initial state: 10 airports, each airport has 5 planes and 20 pieces of cargo Goal: transport all the cargos at airport A to airport B Solution: load the 20 pieces of cargo at A into one of the planes at A and fly it to B Avg Branching factor b: each of the 50 planes can fly to 9 other airports, and each of the 200 packages can be either unloaded (if it is loaded), or loaded into any plane at its airport (if it is unloaded) à ~ 2000 possible actions per state Number of states to explore: O(b d )

26 FIND A HEURISTIC: RELAX THE PROBLEM Define a Relaxed problem: (Potentially) Easy to solve The solution of the relaxed problem gives an admissible heuristic for A* Any ideas about how to perform a general relaxation? Relaxation: Remove all preconditions from actions Every action will always be applicable, and any condition (sub-goal) can be potentially achieved in one step (if there is an action that sets the sub-goal literal to true, otherwise the problem is impossible) h(x) = Cost-to-go(al) of the relaxed problem from state x Equivalent to adding edges to the state graph: including forbidden actions 26

27 DOMAIN-INDEPENDENT HEURISTIC Solving the relaxed problem should be easy enough, we can even think to take a shortcut, by setting the solution to be the same as the number of unsatisfied sub-goals from current state x h(x) =~ number of unsatisfied goals from current state x o It looks like a correct estimate and also admissible or maybe not? (we need admissibility!) Impossible to derive such a heuristic with atomic states! The successor function is a black box, here we exploit the structure of the representation The heuristic is domain-independent! With atomic states, in general only domain-specific heuristics are possible 27

28 HEURISTIC: IGNORE PRECONDITIONS Complications, that could made the heuristic function h(x) neither admissible nor being the solution of the relaxed problem: a. Some operations achieve multiple sub-goals (have multiple post-conditions) b. Some operations undo the effects of others To get an admissible and efficient heuristic ignore preconditions and, in addition ignore (i.e., further relax) 1. Just a 2. Just b 3. Both a and b 28

29 HEURISTIC: IGNORE PRECONDITIONS Complications, that could made the heuristic function h(x) neither admissible nor being the solution of the relaxed problem: a. Some operations achieve multiple sub-goals (have multiple post-conditions) h(x) doesn t correspond to the solution of the relaxed problem + it violates admissibility b. Some operations undo the effects of others h(x) doesn t correspond to the solution of the relaxed problem 29

30 IGNORE PRECONDITIONS + NON-GOAL EFFECTS To avoid actions that can cancel each other effects: remove all the effects of actions, except those that are facts g i, i=1,,n, in the goal g (i.e., sub-goals) Exploit factored structure! g 1 g 2 g 4 h x = from x, the min number of actions such that the union of their effects contains all n sub-goals g i Admissible g 5 g 3 g 6 Computing h(x) = solving a SET COVER problem: NP-hard! Greedy log n approximation: o Admissibility is lost! A2 A3 A4 A5 A6 A7 G1 x X X x G2 x X x G3 x x X G4 x X 30

31 IGNORE (SPECIFIC) PRECONDITIONS Ignore specific preconditions to derive domain-specific heuristics Sliding block puzzle, move(t,s 1,s 2 ) action: On(t, s 1 ) Blank(s 2 ) Adjacent(s 1, s 2 ) On(t, s 2 ) Blank(s 1 ) On(t, s 1 ) Blank(s 2 ) Consider two options for removing specific preconditions from move() a.removing Blank(s B ) Adjacent(s C, s B ) b.removing Blank(s B ) Poll: Match option to heuristic: 1.a Manhattan, b #misplaced tiles 2.a #misplaced tiles, b Manhattan 3.b #misplaced tiles, a is inadmissible 4.b Manhattan, a is inadmissible Example state Goal state 31

32 BACKWARD STATE-SPACE SEARCH Searching from a goal state to the initial state (regression) We only need to consider actions that are relevant to the goal (or current state) Relevant-state search This can makes a strong reduction in branching factor, such that it could be more efficient than forward (progression) search Imagine trying to figure out how to get to some small place with few traffic connections from somewhere with a lot of traffic connections At(P 1, A) At(P 2, A) Fly(P 1, A, B) Fly(P 2, A, B) At(P 1, B) At(P 2, A) At(P 1, A) At(P 2, B) At(P 1, A) At(P 2, B) At(P 1, B) At(P 2, A) Fly(P 1, A, B) Fly(P 2, A, B) At(P 1, B) At(P 2, B) 32

33 BACKWARD STATE-SPACE SEARCH Regression from a (goal) state g over the action a, gives state g o g = (g ADD(a)) Preconditions(a) DEL(a) doesn t appear: we don t know whether the facts negated by DEL(a) were true or not before a, therefore nothing can be said about them Variables can be included, such that a set of states is defined: o Goal At(C 2, SFO) Unload(C 2, p, SFO) g = In(C 2,p) At(p, SFO) Cargo(C 2 ) Plane(p) Airport(SFO) 33

34 BACKWARD STATE-SPACE SEARCH How to select actions? Relevant actions only o Have an effect which is in the set of (current) goal conditions Goal: At(C 1, JFK) At(C 2, SFO) Unload(C 2, p, SFO) is relevant, Fly(p, JFK, SFO) is not relevant Consistent actions only o Have no effect which negates an element of the goal Goal: A B C, action a with effect A B C is not consistent Ø How to define good heuristics? 34

35 CONCLUSIONS, SO FAR Representations using factored states allows to reason about the (factored) structure of the states (in terms of sets of variables) and exploit it A goal states is a conjunction of sub-goals that can be individually satisfied STRIPS / PDDL language to express problem domains and problem instances in a way which expressive enough while allowing for efficient solutions Forward search can in principle applied but the state space and, more importantly, the branching factor b is expected to be very large Uninformed search can t really be used Informed search, A*, is an option but it needs very good heuristics! It may not be obvious to define general, problem-independent, admissible heuristics (in polynomial time) Backward search can potentially be easier since it can partially overcome the problem of irrelevant actions, however, defining heuristics for backward search is even more difficult than forward search We need a better tool, possibly a way for generating tight admissible heuristics for A* in automatic 35

36 PLANNING GRAPHS Graph-based data structure representing a polynomial-size/time approximation of the exponential search tree Can be used to automatically produce good heuristic estimates (e.g., for A*) Can be used to search for a solution using the GRAPHPLAN algorithm 36

37 Planning Graphs Leveled graph: vertices organized into levels/stages, with edges only between levels Two types of vertices on alternating levels: o Conditions o Operations Two types of edges: o Precondition: from condition to operation o Postcondition: from operation to condition 37

38 GENERIC PLANNING GRAPH Condition S 0 contains all the conditions that hold in initial state 38

39 GENERIC PLANNING GRAPH Level S 0 Level O 0 Precondition Condition Add operation to level O i if its preconditions appear in level S i 39

40 GENERIC PLANNING GRAPH Precondition Level S 1 Add condition to level S i if it is the postcondition of an operation (it is in ADD or DELETE lists) in level O IJC Keep a previous condition of no action negates it (persistence, no-op action) Condition No-Op (Persistent action) 40

41 CONDITIONS MONOTONICALLY INCREASE p No-Op O 1 p p No-Op O 1 No-Op p p O 2 q No-Op q No-Op q r No-Op r No-Op r #Conditions (always carried forward by no-ops) 41

42 GENERIC PLANNING GRAPH No-Op (Persistent action) Level S 2 Level O 2 Condition Precondition Operation Postcondition Repeat 42

43 GENERIC PLANNING GRAPH Idea: S i contains all conditions that could hold at stage i based on past actions; O i contains all operations that could have their preconditions satisfied at time i No ordering among the operations is assumed at each stage, they could be executed in parallel The level j at which a condition first appears is a (good) estimate of how difficult is to achieve that condition Can optimistically estimate how many steps it takes to reach a goal g (or sub-goal g i ) from the initial state: admissible heuristic! 43

44 OPERATIONS MONOTONICALLY INCREASE p No-Op O 1 p p No-Op O 1 No-Op p p O 2 q No-Op q No-Op q r No-Op r No-Op r #Operations (as a result of conditions monotonic increase, that keep previous preconditions hold and set new preconditions true) 44

45 MUTUAL EXCLUSION LINKS As it is the graph would be too optimistic! The graph data structure also records conflicts between actions or conditions: two operations or conditions are mutually exclusive (mutex) if no valid plan can contain both at the same time A bit more formally: o Two operations are mutex if their preconditions or postconditions are mutex (inconsistent effects, competing needs, interference) o Two conditions are mutex if one is the negation of the other, or all action pairs that achieve them are mutex (inconsistent support) 45

46 A RUNNING EXAMPLE Have cake and eat cake too problem 46

47 A RUNNING EXAMPLE Only Eat(Cake) is applicable 47

48 A RUNNING EXAMPLE 48

49 MUTEX CASES Inconsistent postconditions (two ops): one operation negates the effect of the other; Eat(Cake) and no-op Have(Cake) Interference (two ops): a postcondition of one operation negates a precondition of other; Eat(Cake) and no-op Have(Cake) (issue in parallel execution, the order should not matter but here it would) B B Inconsistent Postconditions B B Interference 49

50 A RUNNING EXAMPLE Interference Inconsistent postconditions Negation of each other Interference Inconsistent post 50

51 MUTEX CASES Competing needs (two ops): a precondition of one operation is mutex with a precondition of the other because they are the negate of each other, like for Bake(Cake) and Eat(Cake), or because they have inconsistent support B B Competing Needs Inconsistent support (two conditions): each possible pair of operations that achieve the two conditions is mutex. Have(Cake) and Eaten(Cake), are mutex in S 1 but not in S 2 because they can be achieved by Bake(Cake) and Eaten(Cake) Inconsistent Support B C 51

52 A RUNNING EXAMPLE Inconsistent support Competing needs 52

53 SUMMARY STRIPS Language for Automated Planning Factored state representation Actions and action schema Planning problem Complexity of planning Planning as search problem Forward and Backward search Need for domain-independent heuristics Difficulty to define admissible heuristics for A* Planning graph data structure: construction and stored information 53