7. Shortest Path Problems and Deterministic Finite State Systems

Transcription

1 7. Shortest Path Problems and Deterministic Finite State Systems In the next two lectures we will look at shortest path problems, where the objective is to find the shortest path from a start node to an end node. It will be shown that these problems are equivalent to the standard problem we have looked at in lecture two when the system is deterministic and has a finite state space. 7.1 The Shortest Path (SP Problem Graph We consider a graph which is defined by a finite vertex space V and a weighted edge space C := {(i, j, c i,j V V R { } i, j V}, where c i,j denotes the arc length or cost from vertex (or node i to j. An example can be found in Fig 7.1. If there is no edge from node i to node j, then c i,j =. i 2 c i2,i 4 c i1,i 2 i 4 i 1 c i2,i 3 c i3,i 2 c i1,i 3 i 3 Figure 7.1: Graph with nodes and edges with associated lengths. Paths A path is defined as an ordered list of nodes, that is Q := (i 1, i 2,..., i q, Date compiled: January 11,

2 where each element of Q is in V and q is the number of nodes in path Q. The set of all paths that start at some node S V and end at node T V is denoted by Q S,T. Path Length The path length associated with path Q is the sum of the arc lengths over the path, that is q 1 J Q = c ih,i h+1. Objective h=1 Determine a path Q that has the smallest length from node S V to T V, that is, Q = arg min Q Q S,T J Q. In order for this problem to make sense, the following assumption must be satisfied: Assumption 7.1. For all i V and for all Q Q i,i, J Q 0. Assumption 7.1 ensures that there are no negative cycles in the graph. This implies c i,i 0 for all i V Solving SP Problems With further assumptions on the problem data, the SP problem can be solved directly using efficient label correcting methods as will be discussed next lecture. Another method is to map the SP problem to that of a deterministic, finite state system on which we can apply standard DPA that we are familiar with. This will be shown in Section Deterministic Finite State (DFS Problem Consider the standard problem in Section 1.1 where there are no disturbances w k and the state space S k is a finite set for all k = 0,..., N: Dynamics x k+1 = f k (x k, u k, x k S k, k = 0,..., N, u k U k (x k, k = 0,..., N 1 (7.1 Cost Function N 1 g N (x N + g k (x k, u k (7.2 Objective k=0 Since the problem is deterministic, feedback control results in no advantage in terms of cost reduction. Therefore, given x 0 S 0, we want to construct an optimal control sequence (u 0,..., u N 1 that minimizes the cost function (7.2. Nonetheless, this problem can be solved using the DPA. 2

3 7.3 Equivalence of SP and DFS DFS to SP We construct the graph of the associated problem. Every state x k S k at each stage k is represented by a node in the graph. The given x 0 is designated as the starting node S and an artificial terminal node T is added to handle the final stage, such that the arc lengths to T are simply the terminal costs of the DFS. The arc lengths for the rest of the nodes is the (smallest stage cost between two states and is infinite if there is no control that links them. To be precise, the vertex space is the union of all stage and state pairs, that is, ( N V := V k {T}, where k=0 V 0 := {(0, x 0 } V k := {(k, x k x k S k }, k = 1,..., N, and S := (0, x 0. The weighted edge space is then C := (k, x k V k ( (k, xk, (k + 1, x k+1, c (k + 1, x k+1 V k+1 c = min g k(x k, u {u U k (x k x k+1 =f k (x k,u} k {0,..., N 1} {( (N, xn, T, g N (x N } (N, x N V N. Note that the min operator is subject to the control space constraint and the dynamics (to ensure that a control input can link the respective states. If the constraints cannot be satisfied, the min returns. Furthermore, other unconnected edges have infinite arc length. This conversion can be visualized in Fig

4 (1, 1 (2, 1 (N, 1 g N (1 (1, 2 (2, 2 (N, 2 T (0, x 0 g N (2 (1, n (2, n (N, n g N (n V 0 V 1 V 2 V N Figure 7.2: The graph corresponding to DFS. Without loss of generality, S k = {1,..., n} for k = 1,..., N SP to DFS Consider a graph with a vertex space V and a weighted edge space C. We want to find a shortest path from any node S V to T V. Since we exclude the possibility that a cycle has negative cost, it is clear that an optimal path need not have more than V elements, where V denotes the number of elements (the cardinality of set V. Assume c i,i = 0 for all i V, which allow for degenerate moves. We can formulate this problem as an N := V 1 stage DFS: State space: S k := V\{T} for k = 1,..., N 1, S N := {T} and S 0 := {S}. Control space: U k := V\{T} for k = 0,..., N 2, and U N 1 := {T} The dynamics are x k+1 = u k, u k U k, k = 0,..., N 1 The stage cost functions are g k (x k, u k := c xk,u k, k = 0,..., N 1 g N (T := 0 Note that stage costs can be infinite, which corresponds to the case that there is no edge between the nodes. We can solve the above DFS using DPA, where J k (i is the optimal cost of getting from node i to node T in N k = V 1 k moves: J N (T = g N (T = 0, J k (i = min u U k (g k (i, u + J k+1 (u, i S k, k = N 1,..., 0 J N 1 (i = c i,t, J k (i = J 0 (S = i V\{T} min (c i,j + J k+1 (j, i V\{T}, k = N 2,..., 1 j V\{T} min (c S,j + J 1 (j j V\{T} 4

5 Remarks: J 0 (S = J Q, where the path Q can be inferred from the minimizations Remove the degenerate moves, that is, the steps where we don t move; we assumed that c i,i = 0 and thus it does not hurt or help us to stay at node i. If the problem had c i,i > 0 for some i V, then it only hurts us to stay. We can terminate the algorithm early if J k (i = J k+1 (i for all i V\{T}. Forward DP Algorithm From Fig. 7.1, it can be seen that shortest path problem is symmetric: an optimal path from S to T is also a shortest path from T to S, where all the arcs are flipped, that is, the direction of each arc is reversed and its length remains the same. We can therefore formulate the DPA for the reverse SP problem as follows. First we define an auxiliary SP problem. In particular, let c j,i := c i,j, (i, j, c i,j C be the weighted edges of the auxiliary problem. The objective is to find the optimal path from T to S. We can now apply the DP as in Section 7.3.2: J N 1 (j = c j,s = c S,j, J k (j = J 0 (T = j V\{S} min ( c j,i + J k+1 (i, j V\{S}, k = N 2,..., 1 = min (c i,j + J k+1 (i, j V\{S}, k = N 2,..., 1 min (c i,t + J k+1 (i. Let l := N k and J F l := J N l. Then the above become J F 1 (j = c S,j, J F l (j = J F N(T = min min j V\{S} (c i,j + Jl 1 F (i, j V\{S}, l = 2,..., N 1 ( c i,t + JN 1(i F, where Jl F (j is the optimal cost-to-arrive to node j from node S in l moves (in the original SP problem. 7.4 Hidden Markov Models and the Viterbi Algorithm We now consider an estimation problem and show how it can be converted to an SP problem. Dynamics Consider the finite state, time-invariant 1 system x k+1 = w k, x k S, P ij := p w x (j i, i, j S (7.3 1 Can be easily extended to time-varying distributions as well. 5

6 where S = {1,..., n} is a finite set and the distribution p w x is given. Furthermore, the initial state x 0 is not known but its distribution p x0 is. This system is also known as a Markov chain. Measurement Model When a state transition occurs, the states before and after the transition are unknown (or hidden to us, but we obtain an observation z that relates to that transition, that is M ij (z := p z x,w (z i, j, z Z (7.4 where Z is the measurement space, and the time-invariant distribution p z x,w is given and is known as the likelihood function. Furthermore, Markov chains whose state transitions are imperfectly observed according to (7.4 are called Hidden Markov Models. We assume independent observations, that is, observation z k is conditionally independent with all prior variables x l, l < k 1 and z l, l < k, given x k 1, x k. Objective Let Z i := (z i,..., z N and X i := (x i,..., x N. Given a measurement sequence Z 1 = (z 1,..., z N, we want to find the most likely state trajectory X 0 = (x 0,..., x N. In particular, solve for a maximum a posteriori estimate ˆX 0 := (ˆx 0,..., ˆx N where ˆX 0 = arg max X 0 p(x 0 Z 1. By the conditioning rule, p(x 0, Z 1 = p(x 0 Z 1 p(z 1. Since p(z 1 is fixed and non-negative, maximizing p(x 0 Z 1 is equivalent to maximizing p(x 0, Z 1. Note that p(x 0, Z 1 = p(x 0, X 1, Z 1 1 = p(x 1, Z 1 x 0 2 = p(x 2, Z 2 x 0, x 1, z 1 p(z 1, x 1 x 0 3 = p(x 2, Z 2 x 0, x 1, z 1 p(z 1 x 1, x 0 p(x 1 x 0 4 = p(x 2, Z 2 x 0, x 1, z 1 M x0 x 1 5 = p(x 3, Z 3 x 0, x 1, z 1, x 2, z 2 p(x 2, z 2 z 1, x 1, x 0 M x0 x 1 5 = p(x 3, Z 3 x 0, x 1, z 1, x 2, z 2 p(z 2 x 2, z 1, x 1, x 0 p(x 2 z 1, x 1, x 0 M x0 x 1 6 = p(x 3, Z 3 x 0, x 1, z 1, x 2, z 2 M x1 x 2 (z 2 P x1 x 2 M x0 x 1 Steps:. N = P xk 1 x k (z k k=1 1. Use conditioning rule on x 0 2. Use conditioning rule on x 1, z 1 3. Use conditioning rule on x 1 4. By (7.3 and (7.4 6

7 5. Apply step 2 to step 3 on x 2 and z 2 6. z 2 is conditionally independent on x 2, x 1 with prior states and measurements, (7.3 and (7.4. Since the probabilities in the product are non-negative, maximizing p(x 0, Z 1 is equivalent to minimize X 0 ( c S,(0,x0 + N k=1 c (k 1,xk 1,(k,x k (7.5 where { ln( if > 0 c S,(0,x0 = if = 0 { ln(p xk 1 x c (k 1,xk 1,(k,x k = k (z k if P xk 1 x k (z k > 0 if P xk 1 x k (z k = 0 We can now construct a graph of state-time pairs (similar to Section 7.3.1, with artificial starting node S and terminal node T, and the arc costs inferred from (7.5 (see Fig ln(p 11 M 11 (z 1 (0, 1 (1, 1 (N, 1 ln(p x0 (1 ln(p 12 M 12 (z 1 0 S ln(p x0 (2 (0, 2 ln(p 22 M 22 (z 1 (1, 2 (N, 2 0 T (0, n (1, n (N, n 0 k = 0 k = 1 k = N Figure 7.3: State estimation of a hidden Markov model viewed as a SP problem from S to T. In practice, the shortest path is constructed sequentially by forward DP, as costs to arrive and thus the optimal state estimates can be computed in real-time. 7