INFORMS ANNUAL MEETING WASHINGTON D.C. 2008

Size: px

Start display at page:

Download "INFORMS ANNUAL MEETING WASHINGTON D.C. 2008"

Bridget Strickland
5 years ago
Views:

1 Sensor Information Monotonicity in Disambiguation Protocols Xugang Ye Department of Applied Mathematics and Statistics, The Johns Hopkins University

2 Stochastic ti Ordering Comparing Two Random Numbers: A, B Three measures: By Expectation: A E B if E(A) E(B) ~ E( ) denotes expectation By Quantile: A Q,,p B if Q(A, p) Q(B, p) ~ Q(X, p) = inf {x: F X (x) p} denotes p - quantile By Distribution: A D B if F A (x) F B (x) for all x ~ F(x) = P(X x) denotes distribution function The third measure is usually called stochastic ordering. Some other notations include A STO B and A p B. If strict inequality is also true for some x, then the ordering is also called strict, i.e. A < STO B or A p B.

3 Stochastic ti Ordering Properties: A D B => E(A) E(B) A D B => A Q,p B for any p 1 E[f(A)] E[f(B)] for all non-decreasing function f => A D B A Q,p B for any p 1=>A A D B A C = c D B C = c for any c => A D B A D B and f is non-decreasing function => f(a) D f(b) Remarks: Ordering by expectation is the most stable, but there might exist tiny possibility of infinite expectation. Ordering by quantile or distribution function may overcome the infinite expectation problem, but they are more difficult to implement than the expectation measure since much more samples are needed to estimate the distribution function. Ordering by distribution function is the strongest.

4 Stochastic ti Ordering Ideal Picture vs. Real World System Input: X Prior Information: X Function f Strategy (policy) New Information Output: Y = f(x) Decisions Cost: Y Given X 1 STO X 2, compare Y 1 and Y 2 Given X 1 is stochastically better than X 2, compare Y 1 and Y 2

5 Random Disambiguation Paths The Simplest Example s a: l e: l, ρ, c, X l : cost of going from s to t by deterministic arc a l: cost of going from s to t by nondeterministic arc e if e is traversable ρ : probability that e is not traversable c: cost of disambiguating e X: indicator of e, that is, P(X = 1 ) = ρ and P(X = ) = 1 ρ t There are two options: one is to go from s to t by a (certain way); the other is to disambiguate e, with a cost c, at first, then make further decision according to the disambiguation result. After disambiguation, If X =, then it s wise to choose between a and e the one with smaller traveling cost; If X = 1, then there is only one choice left, that is, to go from s to t by a. Question: What is the best policy to go from s to t?

6 Random Disambiguation Paths Cost Analysis Option 1: the cost is C 1 = l Option 2: the cost is c + l if X = ; C 2 = c + l if X = 1 C 2 is random with mean E(C 2 ) = ρ(c + l ) + (1 ρ)(c + l) Consider E(C 1 ) E(C 2 ) = l ρ(c + l ) (1 ρ)(c + l) =(1 ρ)l c (1 ρ)l <Ponder on this!> = (1 ρ)(l (l + c/(1 ρ))) = (1 ρ)((l + /(1 ρ)) (l + c/(1 ρ))) A discovery: E(C 1 ) E(C 2 ) if and only if l + /(1 ρ) l + c/(1 ρ) A conclusion: choose Option 1 if l + /(1 ρ) l + c/(1 ρ); choose Option 2 otherwise. This gives the optimal strategy under the expectation measure.

7 Random Disambiguation Paths More Complicated Scenario s a: l e 1 : l 1, ρ 1, c 1, X 1... e m : l m, ρ m, c m, X m l : cost of going from s to t by deterministic arc a l i : cost of going from s to t by nondeterministic arc e i if e i is traversable ρ i : probability that e i is not traversable c i : cost of disambiguating e i X i : indicator of e i, that is, P(X i =1)=ρ = ρ i and P(X i = ) = 1 ρ i t Assume independency. There are too many options. There are (m + 1)! distinct policies with each one denoted as a permutation of a, e 1, e 2,, e m (these policies form a class called balk-free class). There are also many policies outside the balkfree class. Question: What is the best policy to go from s to t?

8 Random Disambiguation Paths m = 2 The dynamic programming search tree for finding the optimal policy for traversing the parallel graph in which there is a deterministic arc a and two nondeterministic arcs e 1, e 2. ρ 1 = ρ(e 1 ), ρ 2 = ρ(e 2 ), l = l(a), l 1 = l(e 1 ), l 2 = l(e 2 ), c 1 = c(e 1 ), c 2 = c(e 2 ). The root of the tree is the problem of evaluating E*({e 1, e 2 } a), which is recursively reduced into subproblems via conditioning

9 Random Disambiguation Paths An Optimal Policy: Sort l + /(1 ρ), l 1 + c 1 /(1 ρ 1 ),, l m + c m /(1 ρ m ) in ascending order, the corresponding ordered list a 1, a 2,, a m+1, as a permutation of a, e 1,, e m, defines an optimal policy under the expectation measure. To execute the policy, check a 1, if it is deterministic, then traverse it; otherwise, disambiguate it. If the disambiguation result tells that it is traversable, then traverse it; otherwise, check a 2, continue this process until reaching t. Generally speaking, the policy is to, in a dynamic manner, solve shortest path problems. Proof of the optimality under the expectation measure and the further theoretical development are tricky, but It inspires a motivating heuristic method for more general settings and real-world applications.

10 Random Disambiguation Paths Back to The Simplest Example Suppose ρ is unknown, but s a: l e: l, ρ, c, X, Y l : cost of going from s to t by deterministic arc a l: cost of going from s to t by nondeterministic arc e if e is traversable ρ : probability that e is not traversable c: cost of disambiguating e X: indicator of e, that is, P(X = 1 ) = ρ and P(X = ) = 1 ρ t There is another observable random variable Y (,1) such that P(Y y X = ) = F (y) P(Y y X = 1) = F 1 (y) (usually assume continuous distributions) Suppose l > l + c (consider nontrivial case) Policy: compare l with l + c/(1 Y). If l < l + c/(1 Y), then traverse a; otherwise, disambiguate e. If disambiguation result shows X =, then traverse e; otherwise, traverse a. The cost C of going from s to t is c+ l if l > l+ c/(1 Y), X = C = l if l l + c /(1 Y ) c + l if l > l + c/(1 Y), X = 1

11 Random Disambiguation Paths Let α = (l l c ) / (l l) we can rewrite the cost function as c + l if Y < α, X = C = l if Y α c + l if Y < α, X = 1 We can compute (via conditioning) P(Y < α, X = ) = P(Y < α X = )P(X = ) = (1 ρ)f (α) P(Y < α, X = 1) = P(Y < α X = 1)P(X = 1) = ρf 1 (α) P(Y α) = P(Y α X = )P(X = ) +P(Y α X = 1)P(X = 1) = (1 ρ)[1 F (α)] + ρ[1 F 1 (α)] The cost distribution function is F C if x< c+ l (1 ρ) F ( α) if c + l x < l ( x) = 1 ρ F ( α ) if l x < c + l 1 if c+ l x 1 Ponder on this!

12 Stochastic ti Ordering Prior Information: Y Strategy (policy) Decisions Cost: C Y X = ~ F Y X = 1 ~ F 1 New Informat tion: X P(X = 1) = ρ P(X = ) = 1 ρ Consider Y (1) : Y (1) X = ~ F (1) Y (1) X = 1 ~ F 1 (1) Y (2) : Y (2) X = ~ F (2) Y (2) X = 1 ~ F (2) 1 Suppose F (1) (y) F (2) (y) F (1) 1 (y) F (2) 1 (y) ~ that is, Y (1) is stochastically at least as good as Y (2), or prior information is stochastically ordered. Implication: F (1) ( x ) F (2) ( x ) for C any x >, hence C (1) (2) D C,ie i.e. the random costs are also stochastically ordered. It s also straightforward that E[C (1) ] E[C (2) ] and C (1) Q,p C (2) C Here comes an important concept: sensor monotonicity

13 AR Real lw World lda Application Random Disambiguation Paths in US Navy

14 Background Costal Battlefield Reconnaissance and Analysis (COBRA) Allows naval expeditionary forces to conduct airborne, standoff reconnaissance and automatic detection of minefields in costal area Allows marine corps to successfully conduct quick Ship-to- Objective Maneuver in face of mine threats without personnel casualties and equipment losses

15 Background COBRA system consists of three primary components Airborne Payload Sensor Unit Benefit/Cost Ratio Analysis Tactical lcontrol lsoftware Processing Station Navigation Unit Navigation Algorithm

Problem Description Overall Problem Navigate a

Decision under uncertainty Probabilistic prior

16 Problem Description Overall Problem Navigate a combat unit safely and swiftly through a costal environment with mine threats and to reach a preferable target location Features Decision under uncertainty Probabilistic prior information Disambiguation capability Dynamic Learning

17 Problem Decomposition Terrain Modeling Minefield model Mark information Graph generation Dynamic Shortest Path Problem Search algorithm Replanning

18 Minefield Model Simulated Risk Centers and Disks t Simulate detections d i, i = 1, 2,, m via spatial ilpoint process True detection: X i = 1 False detection: X i = i 6 y 5 4 D i 3 d i 2 1 s x Adi disambiguation i of fdd i at a cost c i > happens when the agent is right outside D i but about to enter D i and d i has not been disambiguated

19 Mark Information False mine Y i X i = ~ F f < Y i < True mine Y i X i = 1 ~ F f

20 Graph Generation Marker of arc a = (u, v): Y I \ I Y + (a) =, v u where I u = {i u is covered by D i } and I v = {i v is covered by D i } Knowledge of true-false status: if X i =; Y i+ = 1 if X i =1; Y i Marker of intersection: if d i has not been disambiguated Y I = 1 - (1 Y + ), where I {1, 2,, m} i I i Extended length function : l(a) if Y + (a) <1; l + (a) = + if Y + (a) =1 Extended disambiguation cost function : ci if < Y + (a) <1; c + i [ I( v)\ I( u)] Id (a)= otherwise, where I d = {i d i has not been disambiguated} CR weight function: + c ( a) W CR,Y (a)=l l + (a) Y ( a)

21 Dynamic Shortest t Path Problem Under the knowledge (Y +, l +, c + ) of fthe terrain, find a shortest t path relative to W CR, Y from its current location to the target location t, let the agent follow the shortest path plan until the agent reaches t or encounters a nondeterministic arc. In the former case, the navigation process successfully completes; in the later case, the agent disambiguates the arc by disambiguating all the newly encountered risk disks. The disambiguation results update the knowledge (Y +, l +, c + ) and a new shortest path from agent s current location to the target location t relative to updated W CR,Y is found for the agent to follow.

22 Search Algorithm A* algorithm One of the greatest achievement in Artificial Intelligence (AI) It employs a best-first search strategy It finds a shortest path as long as there exists one It uses heuristic information to reduce the search tree It can be derived from the primal-dual algorithm for general LP It is practically proved very efficient g(v) v h(v) s Closed list t Open list

23 The A* Algorithm Graph/Network: A directed graph G = (V, A, W, δ, b) V is the set of nodes. A is the set of arcs. W: A R is the weight function. s V is a specified starting node. t V is a specified target node. δ > is a constant such that δ W (a) < + for any a A. b > is a constant integer such that {v (u, v) A or (v, u) A} b for all u V. There exists a heuristic function h: V R such that h(v) for all v V, h(t) ) =, and W(u, v) ) + h(v) ) h(u) )for all ll( (u, v) ) A. ~ Consistent heuristic Denote dist(u, v) as the distance (length of the shortest path) from u to v.

24 The A* Algorithm Notations: h: heuristic O: Open list E: Closed list d: distance label f: node selection key pred: predecessor Steps: Given G, s, t, and h Step 1. Set O = {s}, d(s) =, and E = φ. Step 2. If O = φ and t E, then stop (there is no s-t path); otherwise, continue. Step 3. Find u = argmin v O f(v) = d(v) + h(v). Set O = O \ {u} and E = E {u}. If t E, then stop (a shortest s-t path is found); otherwise, continue. Step 4. For each node v V such that (u, v) A and v E, if v O, then set O = O {v}, { } d(v) ) = d(u) )+W( W(u, v), ) and pred(v) ) = u; otherwise, if d(v) > d(u) + W(u, v), then set d(v) = d(u) + W(u, v) and pred(v) = u. Go to Step 2.

25 Sensor and Sensor Ordering Sensor Notation: ti S = (F, F 1 ) Valid sensor: F (y) F 1 (y) for any y 1 Discerning i sensor: F (.5) >.5 > F 1 (.5) Y i ~ S < = > Y i X i = ~ F Y i X i = 1 ~ F 1 Beta sensor: F = Beta (3.5 - λ, λ) F 1 = Beta (3.5 + λ, λ) Sensor Ordering: Notation: ti S (1) = (F (1), F (1) 1 )i is said idto be at least as good as S (2) = (F (2), F (2) 1 ) if F (1) (y) F (2) (y) and F (1) 1 (y) F (2) 1 (y) for any y 1 f beta (x;a,b) f beta (x;1.5,5.5) Probability Density Function: Beta f beta (x;2.5,4.5) f beta (x;4.5,2.5) f beta (x;5.5,1.5) x

Simulation 1 9 1.9 8 7 2 3.8.7 6 4.6 y 5 y 5.5 4 6.4 3 7.3 2 8.2 1 9.

26 Simulation y 5 y x x A realization of trajectory in a real terrain (left) and in one of its marked map (right) Sensor parameter λ =.5. Total cost: ; traveling cost: ; and disambiguation cost: 27. There are totally 12 disambiguations. Total simulation run time in a PC with Pentium 4 CPU and 1G RAM: seconds.

Simulation 1 9 1.9 8 7 2 3.8.7 6 4.6 y 5 y 5.5 4 6.4 3 7.3 2 8 2.2 1 9.

one of its marked map (right). Sensor parameter λ = 3.. Total cost: 174.77; traveling cost: 168.

27 Simulation y 5 y x x A realization of another trajectory in the same real terrain (left) and in one of its marked map (right). Sensor parameter λ = 3.. Total cost: ; traveling cost: 168.2; and disambiguation cost: There are totally 3 disambiguations. Total simulation run time in a PC with Pentium 4 CPU and 1G RAM: seconds.

28 Statistical Analysis: Conditional Experiments Average Cost vs. Sensor Parameter Empirical CDFs 26 Probability Length λ =.1 22 λ =.5 λ = λ = λ = λ = 2.5 λ = λ = Average Co ost λ Graphic statistical ti ti results of the data from the experiments conditioning i on terrain T 1. For each i = 1, 2,, 8, the sample size under λ i is 4. Left: plot of ECDFs; Right: error bar plot of average cost vs. sensor parameter Kolmogorov-Smirnov tests t for comparing sample distributions ib ti t tests for comparing sample means

29 1.8 Statistical Analysis: Unconditional Experiments Average Cost vs. Sensor Parameter Empirical CDFs y Probability λ =.1 λ =.5 λ = 1. λ = 1.5 λ = 2. λ = 2.5 λ = 3. λ = Length Average Co ost λ Graphic statistical ti ti results of the data from the unconditional experiments. For each i = 1, 2,, 8, the sample size under λ i is 25. Left: plot of ECDFs; Right: error bar plot of average cost vs. sensor parameter Kolmogorov-Smirnov tests t for comparing sample distributions ib ti t tests for comparing sample means

30 COBRA Data y COBRA Terrain x Projection y Projected COBRA Terrain x

COBRA Data y 9 8 7 6 5 4 3 2 1 s C d = 5 λ = -4-2 2

3 1 2 4 6 8 1 x Trajectory simulation under original

There is no disambiguation; Total simulation run

31 COBRA Data y s C d = 5 λ = x t y C d = 5 λ = x Trajectory simulation under original markers (λ = ), the total cost is There is no disambiguation; Total simulation run time in a PC with Pentium 4 CPU and 1G RAM is seconds 8 9 s t

COBRA Data y 9 8 7 6 5 4 3 2 1 s C d = 5 λ =.

4 7 8 9 1 2 4 6 8 1 x Trajectory simulation under improved markers (λ =.4).

32 COBRA Data y s C d = 5 λ = x t y C d = 5 λ = x Trajectory simulation under improved markers (λ =.4). The total cost is with one disambiguation. Total simulation run time in a PC with Pentium 4 CPU and 1G RAM is seconds s t

33 COBRA Data 98 Total Cost vs. Improvement Parameter 97 Marker improvement scheme Y i = λ +(1 λ) Y i if X i =1; (1 λ) Y Y i if X i = Total Cost λ Plot of total cost vs. improvement parameter for COBRA runs. The mesh of values of λ are.1i, i = 1, 2,, 1. Starting location: s = ( 3, 25); target location t = (3, 6); and disambiguation cost per disk is C d =5 5.

34 Deterministic Shortest Path Cost Average Average Cost vs. Sensor Parameter Average costs of nondeterministic traversals Average length of deterministic shortest paths Critical sensor parameter Percen nt Histogram With Fit 3-Parameter Lognormal 3 35 Data 4 45 Variable lambda =.1 lambda =.5 lambda = 1. lambda = 1.5 lamba = 2. lambda = 2.5 lambda = 3. lambda = 3.49 Loc Scale Thresh N λ Histogram Deterministic shortest paths vs. nondeterministic traversals. Unconditional experiments. The critical parameter of the Beta sensor is λ* = Percent c_d = inf

35 Thanks! question?

A Note on the Connection between the Primal-Dual and the A* Algorithm

A Note on the Connection between the Primal-Dual and the A* Algorithm Xugang Ye, Johns Hopkins University, USA Shih-Ping Han, Johns Hopkins University, USA Anhua Lin, Middle Tennessee State University,