Chapter 4 Deliberation with Temporal Domain Models

Transcription

1 Last update: April 10, 2018 Chapter 4 Deliberation with Temporal Domain Models Section 4.4: Constraint Management Section 4.5: Acting with Temporal Models Automated Planning and Acting Malik Ghallab, Dana Nau and Paolo Traverso Dana S. Nau University of Maryland Licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License 1

2 Outline Introduction Representation Temporal planning = 4.4 Constraint Management Ø Consistency of object constraints and time constraints Ø Controlling the actions when we don t know how long they ll take = 4.5 Acting with temporal models 2

3 Constraint Management = Each time TemPlan applies a resolver, it modifies (T,C) Ø Some resolvers will make (T,C) inconsistent No solution in this part of the search space Detect inconsistency => prune this part of the search space Don t detect it => waste time looking for a solution = Analogy: PSP checked simple cases of inconsistency Ø E.g., can t create a constraint a b if there s already a constraint b a Ø Ignored more complicated cases = Example: Ø c 1, c 2, c 3 Containers = {c1, c2} Ø Suppose that to resolve 3 threats, PSP chooses these resolvers: c 1 c 2, c 2 c 3, c 1 c 3 Ø No solutions in this part of the search space, but PSP searches it anyway 3

4 Constraint Management in TemPlan = At various points, check consistency of C Ø If C is inconsistent, then (T,C) is inconsistent Can prune this part of the search space Ø If C is consistent then (T,C) may or may not be consistent = Example of a case where C is consistent but (T,C) isn t: T = {[t 1,t 2 ] loc(r1)=loc1, [t 3,t 4 ] loc(r1)=loc2} C = (t 1 < t 3 < t 4 < t 2 ) Ø Gives loc(r1) two values during [t 3,t 4 ] 4

5 Consistency of C = C contains two kinds of constraints Ø Object constraints loc(r) l 2, l {loc3, loc4}, r = r1, o o Ø Temporal constraints t 1 < t 3, a < t, t < t, a t t b = Assume object constraints are independent of temporal constraints and vice versa Ø exclude things like t < f (l,r) = Then two separate subproblems Ø (1) check consistency of object constraints Ø (2) check consistency of temporal constraints Ø C is consistent iff both are consistent 5

6 Object Constraints = Constraint-satisfaction problem (CSP) NP-hard = Can write an algorithm that s complete but runs in exponential time If there s an inconsistency, always finds it Might do a lot of pruning, but spend lots of time at each node = Instead, use a constraint-satisfaction technique that s incomplete but takes polynomial time arc consistency, path consistency = Detects some inconsistencies but not others Ø Runs much faster, but prunes fewer nodes 6

7 To represent time constraints: = Simple Temporal Networks (STNs) Time Constraints Ø Networks of constraints on time points = Synthesize incrementally them starting from ϕ 0 Ø Templan can check time constraints in time O(n 3 ) = Incrementally instantiated at acting time = Kept consistent throughout planning and acting t1 [6, 7] [1, 2] t2 [1, 7] t3 [4, 5] t5 [3, 4] [1, 2] t4 7

8 = Simple Temporal Network (STN): = a pair (V,E), where Time Constraints V = {a set of temporal variables {t 1,, t n } E V 2 is a set of arcs = Each arc (t i,t j ) is labeled with an interval [a,b] Represents constraint a t j t i b Equivalently, b t i t j a = Representing unary constraints: dummy variable t 0 = 0 Ø Arc r 0i = (t 0,t i ) labeled with [a,b] represents a t i 0 b = Solution to an STN: integer value for each t i, all constraints satisfied = Consistent STN: has a solution = Minimal STN: for every arc (t i,t j ) with label [a,b], for every t [a,b] Ø there s at least one solution such that t j t i = t t 1 t 1 t 2 [1,2] [3,4] [2,3] t 2 [1,2] [3,4] [ 3, 2] t 3 t 3 = Shorthand: instead of a t j t i b, write r ij = [a ij,b ij ] 8

9 Operations on STNs = Intersection, r ij t j t i r ij = [a ij, b ij ] t j t i r ij = [a ij, b ij ] t i r ij t j Infer t j t i r ij r ij = [max(a ij,a ij ), min(b ij,b ij )] r ij r ij = Composition, t k t i r ik = [a ik,b ik ] r ik t k r kj t j t k r kj = [a kj,b kj ] t i t j Infer t j t i r ik r kj = [a ik +a kj, b ik +b kj ] r ik r kj Reason: shortest and longest times for the two intervals = Consistency checking r ik t k r kj r ik, r kj, r ij are consistent if r ij (r ik r kj ) t i r ij r ik r kj t j r ij (r ik r kj ) 9

10 Two Examples t 2 [1,2] [3,4] t 2 [1,2] [3,4] t 1 [2,3] = STN (V,E), where Ø V = {t 1, t 2, t 3 } Ø E = {r 12 =[1,2], r 23 =[3,4], r 13 =[2,3]} = Composition: Ø r 13 = r 12 r 23 = [4,6] = Can t satisfy both r 13 and r 13 Ø r 13 r 13 = [2,3] [4,6] = = (V,E) is inconsistent t 3 t 1 [2,5] = STN (V,E), where Ø V = {t 1, t 2, t 3 } Ø E = {r 12 =[1,2], r 23 =[3,4], r 13 =[2,5]} = As before, r 13 = [4,6] = This time, (V,E) is consistent Ø r 13 r 13 = [4,5] = Minimal network: change r 13 to [4,5] t 2 [1,2] [3,4] t 3 t 1 [4,5] t 3 10

11 Operations on STNs PC(V,E): for 1 k n do for 1 i < j n, i k, j k do r ij r ij [r ik r kj ] if r ij = then return inconsistent i t1 [6, 7] [1, 2] k t2 t3 [4, 5] [3, 4] [1, 2] t4 j = PC (Path Consistency) algorithm: Ø Consistency checking on all triples Ø If an arc has no constraint, use [, + ] Ø n constraints => n 3 triples => time O(n 3 ) t5 = Example: k = 2, i = 1, j = 2 r 12 = [1,2] r 24 = [3,4] r 14 = [, ] r 12 r 24 = [1+3, 2+4] = [4,6] r 14 [max(,4), min(,6)] = [4,6] 11

12 Operations on STNs PC(V,E): for 1 k n do for 1 i < j n, i k, j k do r ij r ij [r ik r kj ] if r ij = then return inconsistent i t1 [6, 7] [1, 2] k t2 t3 [4, 5] [3, 4] [1, 2] t4 j = Detects inconsistent networks Ø r ij = [a ij,b ij ] empty => inconsistent = Makes STN minimal Ø Shrinks each r ij to exclude values that aren t in any solution = Can modify it to make it incremental Ø Input: a consistent, minimal STN, and a new constraint r ij Ø Incorporate r ij in time O(n 2 ) t5 = Example: k = 2, i = 1, j = 2 r 12 = [1,2] r 24 = [3,4] r 14 = [, ] r 12 r 24 = [1+3, 2+4] = [4,6] r 14 [max(,4), min(,6)] = [4,6] 12

13 Pruning TemPlan s search space = Take the time constraints in C Ø Write them as an STN Ø Use Path Consistency to check whether STN is consistent Ø If it s inconsistent, TemPlan can backtrack 13

14 Controllability = Section of the book = Suppose TemPlan gives you a chronicle and you want to execute it Ø Constraints on time points Ø Need to reason about these in order to decide when to start each action t1 [30, 50] bring&move t2 [-5, 5] t3 uncover [5, 10] t4 14

15 Controllability = Solid lines: duration constraints Ø Robot will do bring&move, will take 30 to 50 time units Ø Crane will do uncover, will take 5 to 10 time units = Dashed line: synchronization constraint Ø Don t want either the crane or robot to wait long Ø At most 5 seconds between the two ending times = Objective Ø Choose time points that will satisfy all the constraints t1 [30, 50] t2 bring&move uncover t3 [5, 10] [-5, 5] t4 15

16 Controllability = Suppose we run PC = PC returns a minimal and consistent network = There exist time points that satisfy all the constraints = Would work if we could choose all four time points Ø But we can t choose t 2 and t 4 t1 [30, 50] t2 bring&move uncover t3 [5, 10] [-5, 5] t4 = t 1 and t 3 are controllable Ø Actor can control when each action starts = t 2 and t 4 are contingent Ø can t control how long the actions take Ø random variables that are known to satisfy the duration constraints t 2 [t 1 +30, t 1 +50] t 4 [t 3 +5, t 3 +10] t1 [15, 50] bring&move [30, 50] [25,55] t3 t2 [0, 15] [-5, 5] uncover [5, 10] t4 16

17 Controllability = Can t guarantee that all of the constraints will be satisfied = Start bring&move at time t 1 = 0 = Suppose the durations are bring&move 30, uncover 10 Ø t 2 = = 30 Ø t 4 = t Ø t 4 t 2 = t 3 20 = Constraint 5 t 4 t 2 5 à 5 t = Need to start uncover at t 3 25 Ø If t 3 > 25 then t 4 t 2 > 5 t1 [15, 50] bring&move [30, 50] [25,55] t3 = But if we start uncover at t 3 25, neither action has finished yet Ø We don t yet know how long they ll take = Might instead get this: bring&move 50, uncover 5 Ø t 2 = = 50 Ø t 4 = t = 30 Ø t 4 t = 20 t2 [0, 15] [-5, 5] uncover [5, 10] t4 17

18 STNUs = STNU (Simple Temporal Network with Uncertainty): Ø A 4-tuple (V,Ṽ,E,Ẽ ) V ={controllable time points} = {starting times of actions} Ṽ ={contingent time points} = {ending times of actions} E ={controllable constraints}, Ẽ ={contingent constraints} = Controllable and contingent constraints: Ø Synchronization between two starting times: controllable Ø Duration of an action: contingent Ø Synchronization between ending points of two actions: contingent Ø Synchronization between end of one action, start of another: Controllable if the new action starts after the old one ends Contingent if the new action starts before the old one ends = Want a way for the actor to choose time points in V (starting times) that guarantee that the constraints are satisfied 18

19 Three kinds of controllability = (V,Ṽ,E,Ẽ ) is strongly controllable if the actor can choose values for V such that success will occur for all values of Ṽ that satisfy Ẽ Ø Actor can choose the values for V offline Ø The right choice will work regardless of Ṽ = (V,Ṽ,E,Ẽ ) is weakly controllable if the actor can choose values for V such that success will occur for at least one combination of values for Ṽ Ø Actor can choose the values for V only if the actor knows in advance what the values of Ṽ will be = Dynamic controllability: Ø Game-theoretic model: actor vs. environment Two player, zero sum, extensive form, imperfect information Ø A player s strategy: a function σ telling what to do in every situation Choices may differ depending on what has happened so far Ø (V,Ṽ,E,Ẽ ) is dynamically controllable if strategy for actor that will guarantee success regardless of the environment s strategy 19

20 For t = 0, 1, 2, Dynamic Execution 1. Actor chooses an unassigned set of variables V t V that all can be assigned the value t without violating any constraints in E actions the actor chooses to start at time t 2. Simultaneously, environment chooses an unassigned set of variables Ṽ t Ṽ that all can be assigned the value t without violating any constraints in Ẽ actions that finish at time t 3. Each chosen time point v is assigned v t 4. Failure if any of the constraints in E Ẽ are violated There might be violations that neither V t nor Ṽ t caused individually 5. Success if all variables in V Ṽ have values and no constraints are violated = Dynamic execution strategies σ A for actor, σ E for environment Ø σ A (h t 1 ) = {what events in V to assign = t, given h t 1 } Ø σ E (h t 1 ) = {what events in Ṽ to trigger at time t, given h t 1 } h t = h t 1. (σ A (h t 1 ) σ E (h t 1 )) r ij = [l,u] is violated if t i and t j have values and t j t i [l,u] Ø (V,Ṽ,E,Ẽ ) is dynamically controllable if σ A that will guarantee success σ E 20

21 Example = Instead of a single bring&move task, two separate bring and move tasks = Actor s dynamic execution strategy Ø trigger t 1 at whatever time you want Ø wait and observe t Ø trigger t at any time from t to t + 5 Ø trigger t 3 = t + 10 t 1 [15, 25] t [0, 5] t [15, 20] bring move Ø for every t 2 [t + 15, t + 20] and every t 4 [t 3 + 5, t ] t 4 [t + 15, t + 20] so t 4 t 3 [ 5, 5] Ø So all the constraints are satisfied t 3 t 2 uncover [5, 10] [-5, 5] t4 21

22 Dynamic Controllability Checking = For a chronicle ϕ = (A,ST,T,C) Ø Temporal constraints in C correspond to an STNU Ø Put code into TemPlan to keep the STNU dynamically controllable = If we detect cases where it isn t dynamically controllable, then backtrack = If PC(V Ṽ,E Ẽ ) reduces a contingent constraint then (V,Ṽ,E,Ẽ ) isn t dynamically controllable can prune this branch = If it doesn t reduce any contingent contraints, we don t know whether (V,Ṽ,E,Ẽ ) is dynamically controllable = Two options Ø Either continue down this branch and backtrack later if necessary, or Ø Extend PC to detect more cases where (V,Ṽ,E,Ẽ ) isn t dynamically controllable additional constraint propagation rules PC(V,E): for 1 k n do for 1 i < j n, i k, j k do r ij r ij [r ik r kj ] if r ij = then return inconsistent 22

23 Additional Constraint Propagation Rules = Case 1: u 0 Ø t must come before t e = Add a composition constraint [a,b ] = Find [a,b ] such that [a,b ] [u,v] = [a,b] Ø [a +u, b +v] = [a,b] Ø a = a u, b = b v t s [a, b] t e [u, v] t contingent controllable 23

24 Additional Constraint Propagation Rules = Case 2: u < 0 and v 0 Ø t may be either before or after t e = Add a wait constraint Ø Wait until either t e occurs or current time is t s + b v, whichever comes first t s [a, b] t e [u, v] t contingent controllable 24

25 Extended Version of PC = We want a fast algorithm that TemPlan can run at each node, to decide whether to backtrack = There s an extended version of PC that runs in polynomial time, but it has high overhead = Possible compromise: use ordinary PC most of the time Ø Run extended version occasionally, or at end of search before returning plan 25

26 Outline Introduction Representation Temporal planning Consistency and controllability = 4.5 Acting with temporal models Ø Acting with atemporal refinement Ø Dispatching Ø Observation actions 26

27 Atemporal Refinement of Primitive Actions = Templan s action templates may correspond to compound tasks Ø In RAE, refine into commands with refinement methods Ø Action template (descriptive model) Ø Refinement method (operational model) 27

28 Atemporal Refinement of Primitive Actions = Templan s action templates may correspond to compound tasks Ø In RAE, refine into commands with refinement methods Ø Action template (descriptive model) unstack(k,c,p) assertions: constraints: Ø Refinement method (operational model) 28

29 Discussion = Pros Ø Simple online refinement with RAE Ø Avoids breaking down uncertainty of contingent duration Ø Can be augmented with temporal monitoring functions in RAE Ø E.g., watchdogs, methods with duration preferences = Cons Ø Does not handle temporal requirements at the command level, e.g., synchronize two robots that must act concurrently = Can augment RAE to include temporal reasoning Ø Call it erae Ø One essential component: a dispatching function 29

30 Acting With Temporal Models = Dispatching procedure: a dynamic execution strategy Ø Controls when to start each action Ø Given a dynamically controllable plan with executable primitives, triggers corresponding commands from online observations = Example Ø robot r2 needs to leave dock d2 before robot r1 can enter d2 Ø crane k needs to uncover c then put c onto r1 w2 k r2 q cʹ c p d2 t 1 t 2 t 3 leave(r2,d2) t 5 w1 r1 d1 leave(r1,d1) navigate(r1) enter(r1,d2) t 4 unstack(k,cʹ,p) t 6 unstack(k,c) putdown(k,c,r1) leave(r1,d2) t 7 t 8 t 9 stack(k,cʹ,q) 30

31 Dispatching = Let (V,Ṽ,E,Ẽ ) be a controllable STNU that s grounded = Different from a grounded expression in logic Ø At least one time point t is instantiated = This bounds each time point t within an interval [l t,u t ] Controllable time point t in the future: = t is alive if current time now [l t, u t ] = t is enabled if Ø it s alive Dispatch(V,Ṽ,E,Ẽ ) = initialize the network Ø for every precedence constraint t < t, t has occurred = while there are time points in V that haven t been triggered, do Ø update now Ø update the time points in Ṽ that were triggered since the last iteration Ø update enabled Ø trigger every t enabled such that now = u t Ø arbitrarily choose other time points in enabled, and trigger them Ø propagate values of triggered timepoints (change [l t,u t ] for each future timepoint t) Ø for every wait constraint t e, α, t e has occurred or α has expired 31

32 Example = trigger t 1, observe leave finish = enable and trigger t 2, this enables t 3, t 4 = trigger t 3 soon enough to allow enter(r1,d2) at time t 5 = trigger t 4 soon enough to allow stack(k,cʹ) at time t 6 = rest of plan is linear: choose each t i after the previous action ends t 3 leave(r2,d2) Dispatch(V,Ṽ,E,Ẽ ) = initialize the network = while there are time points in V that haven t been triggered, do Ø update now Ø update the time points in Ṽ that were triggered since the last iteration Ø update enabled Ø trigger every t enabled such that now = u t Ø arbitrarily choose other time points in enabled, and trigger them Ø propagate values of triggered timepoints (change [l t,u t ] for each future timepoint t) t 1 t 2 t 5 leave(r1,d1) t 4 navigate(r1) unstack(k,cʹ,p) enter(r1,d2) t 7 t 8 t 9 t 6 unstack(k,c) putdown(k,c,r1) leave(r1,d2) stack(k,cʹ,q) What about a more detailed example? 32

33 t 1 Example from slide 21 Dispatch(V,Ṽ,E,Ẽ ) [15, 25] t [0, 5] bring = trigger t 1 at time 0 t [15, 20] move = wait and observe t; this enables t t 3 t 2 uncover [5, 10] = trigger t at any time from t to t+5 = trigger t 3 at time t + 10 t 2 [t + 15, t + 20] t 4 [t 3 + 5, t ] = [t + 15, t + 20] = so t 4 t 3 [ 5, 5] [-5, 5] t4 = initialize the network = while there are time points in V that haven t been triggered, do Ø update now Ø update the time points in Ṽ that were triggered since the last iteration Ø update enabled Ø trigger every t enabled such that now = u t Ø arbitrarily choose other time points in enabled, and trigger them Ø propagate values of triggered timepoints (change [l t,u t ] for each future timepoint t) 33

34 t 1 Example from slide 21 [15, 25] t [0, 5] t [15, 20] bring move = trigger t 1 at time 0 = wait and observe t; this enables t = trigger t at any time from t to t+5 = trigger t 3 at time t + 10 t 2 [t + 15, t + 20] t 4 [t 3 + 5, t ] = [t + 15, t + 20] = so t 4 t 3 [ 5, 5] t 3 t 2 uncover [5, 10] [-5, 5] t4 Dispatch(V,Ṽ,E,Ẽ ) = initialize the network = while there are time points in V that haven t been triggered, do Ø update now Ø update the time points in Ṽ that were triggered since the last iteration Ø update enabled Ø trigger every t enabled such that now = u t Ø arbitrarily choose other time points in enabled, and trigger them Ø propagate values of triggered timepoints (change [l t,u t ] for each future timepoint t) To use the extended PC algorithm here, I think we would need additional rule(s) Current ones seem inapplicable 34

35 Deadline Failures = Suppose something makes it impossible to start an action on time = Do one of the following: Ø stop the delayed action, and look for new plan Ø let the delayed action finish; try to repair the plan by resolving violated constraints at the STNU propagation level e.g., accommodate a delay in navigate by delaying the whole plan Ø let the delayed action finish; try to repair the plan some other way t 3 leave(r2,d2) t 1 t 2 t 5 leave(r1,d1) t 4 navigate(r1) unstack(k,cʹ,p) enter(r1,d2) t 7 t 8 t 9 t 6 unstack(k,c) putdown(k,c,r1) leave(r1,d2) stack(k,cʹ,q) 35

36 Partial Observability = Tacit assumption: all occurrences of contingent events are observable Ø Observation needed for dynamic controllability Ø In general not all events are observable = POSTNU (Partially Observable STNU) Controllable Timepoints Contingent = Dynamically controllable? Invisible Observable 36

37 Observation Actions t0 [19:00, 19:30] t1 t [1, 2] working tʹ [20, 25] driving t2 [-5, 10] t3 [25, 30] cooking t4 Controllable Contingent Invisible observable

38 Dynamic Controllability = A POSTNU is dynamically controllable if Ø there exists an execution strategy that chooses future controllable points to meet all the constraints, given the observation of past visible points = Observable visible = Observable means it will be known when observed = It can be temporarily hidden Timepoints Controllable Contingent Invisible Observable Visible Hidden 38

39 Outline Introduction Representation Temporal planning Consistency and controllability Acting with executable primitives = Summary 39

40 Summary = Managing constraints in TemPlan: like CSPs Ø Temporal constraints: STNs, PC algorithm (path consistency) = Acting Ø Dynamic controllability Ø STNUs Ø RAE and erae Ø Dispatching 40