Dynamic Vehicle Routing for Translating Demands: Stability Analysis and Receding-Horizon Policies

Transcription

1 1 Dynamic Vehicle Routing for Translating Demands: Stability Analysis and Receding-Horizon Policies Shaunak D. Bopardikar Stephen L. Smith Francesco Bullo João P. Hespanha Abstract We introduce a problem in which demands arrie stochastically on a line segment, and upon arrial, moe with a fixed elocity perpendicular to the segment. We design a receding horizon serice policy for a ehicle with speed greater than that of the demands, based on the translational minimum Hamiltonian path (TMHP). We consider Poisson demand arrials, uniformly distributed along the segment. For a fixed segment width and fixed ehicle speed, the problem is goerned by two parameters; the demand speed and the arrial rate. We establish a necessary condition on the arrial rate in terms of the demand speed for the existence of any stabilizing policy. We derie a sufficient condition on the arrial rate in terms of the demand speed that ensures stability of the TMHP-based policy. When the demand speed tends to the ehicle speed, eery stabilizing policy must serice the demands in the first-come-first-sered (FCFS) order; and the TMHP-based policy becomes equialent to the FCFS policy which minimizes the expected time before a demand is sericed. When the demand speed tends to zero, the sufficient condition on the arrial rate for stability of the TMHP-based policy is within a constant factor of the necessary condition for stability of any policy. Finally, when the arrial rate tends to zero for a fixed demand speed, the TMHP-based policy becomes equialent to the FCFS policy which minimizes the expected time before a demand is sericed. We numerically alidate our analysis and empirically characterize the region in the parameter space for which the TMHP-based policy is stable. Index Terms Dynamic ehicle routing, autonomous ehicles, queueing theory, minimum Hamiltonian path. I. INTRODUCTION Vehicle routing problems are concerned with planning optimal ehicle routes for proiding serice to a gien set of customers. The routes are planned with complete information of the customers, and thus the optimization is static, and typically combinatorial [3]. In contrast, Dynamic Vehicle Routing (DVR) considers scenarios in which not all customer information is known a priori, and thus routes must be re-planned as new customer information becomes Preliminary ersions of this work appeared as [1] and as []. This material is based upon work supported in part by ARO-MURI Award W911NF , ONR Award N and by the Institute for Collaboratie Biotechnologies through the grant DAAD19-03-D from the U.S. Army Research Office. S. D. Bopardikar, F. Bullo and J. P. Hespanha are with the Center for Control, Dynamical Systems and Computation, Uniersity of California at Santa Barbara, Santa Barbara, CA 93106, USA. {shaunak,bullo}@engineering.ucsb.edu, hespanha@ece.ucsb.edu. S. L. Smith is with the Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge MA 0139, USA; slsmith@mit.edu October 4, 009 DRAFT

2 aailable. DVR problems naturally occur in scenarios where autonomous ehicles are deployed in complex and uncertain enironments. Examples include search and reconnaissance missions, and enironmental monitoring. An early DVR problem is the Dynamic Traeling Repairperson Problem (DTRP) [4], in which customers, or demands arrie sequentially in a region and a serice ehicle seeks to sere them by reaching each demand location. In this paper, we introduce a dynamic ehicle routing problem in which the demands moe with a specified elocity upon arrial, and we design a noel receding horizon control policy for a single ehicle to serice them. This problem has applications in areas such as sureillance and perimeter defense, wherein the demands could be isualized as moing targets trying to cross a region under sureillance by an Unmanned Air Vehicle [5], [6]. Another application is in the automation industry where the demands are objects that arrie continuously on a coneyor belt and a robotic arm performs a pick-and-place operation on them [7]. Contributions We introduce a dynamic ehicle routing problem in which demands arrie ia a stochastic process on a line segment of fixed length, and upon arrial, translate with a fixed elocity perpendicular to the segment. A serice ehicle, modeled as a first-order integrator haing speed greater than that of the demands, seeks to sere these mobile demands. The goal is to design stable serice policies for the ehicle, i.e., the expected time spent by a demand in the enironment is finite. We propose a noel receding horizon control policy for the ehicle that serices the translating demands as per a translational minimum Hamiltonian path (TMHP). In this paper, we analyze the problem when the demands are uniformly distributed along the segment and the demand arrial process is Poisson with rate λ. For a fixed length W of the segment and the ehicle speed normalized to unity, the problem is goerned by two parameters; the demand speed and the arrial rate λ. Our results are as follows. First, we derie a necessary condition on λ in terms of for the existence of a stable serice policy. Second, we analyze our noel TMHP-based policy and derie a sufficient condition for λ in terms of that ensures stability of the policy. With respect to stability of the problem, we identify two asymptotic regimes: (a) High speed regime: when the demands moe as fast as the ehicle, i.e., 1 (and therefore for stability, λ 0 + ); and (b) High arrial regime: when the arrial rate tends to infinity, i.e., λ + (and therefore for stability, 0 + ). In the high speed regime, we show that: (i) for existence of a stabilizing policy, λ must conerge to zero as 1/ ln(), (ii) eery stabilizing policy must serice the demands in the first-come-first-sered (FCFS) order, and (iii) the TMHP-based policy becomes equialent to the FCFS policy which we establish is optimal in terms of minimizing the expected time to serice a demand. In the high arrial regime, we show that the sufficient condition on for the stability of the TMHP-based policy is within a constant factor of the necessary condition on for stability of any policy. Third, we identify another asymptotic regime, termed as the low arrial regime, in which the arrial rate λ 0 + for a fixed demand speed. In this low arrial regime, we establish that the TMHP-based policy becomes equialent to the FCFS policy which we establish is optimal in terms of minimizing the expected time to serice a demand. Fourth, for the analysis of the TMHP-based policy, we study the classic FCFS policy in which demands are sered in the order in which they arrie. We determine necessary and sufficient conditions DRAFT October 4, 009

3 3 on λ for the stability of the FCFS policy. Fifth and finally, we alidate our analysis with extensie simulations and proide an empirically accurate characterization of the region in the parameter space of demand speed and arrial rate for which the TMHP-based policy is stable. Our numerical results show that the theoretically established sufficient stability condition for the TMHP-based policy in the high arrial asymptotic regime seres as a ery good approximation to the stability boundary, for nearly the entire range of demand speeds. A plot of the theoretically established necessary and sufficient conditions for stability in the -λ parameter space is shown in Figure 1. The bottom figures are for the asymptotic regimes of λ +, and 1, respectiely. Related work Dynamic ehicle routing (DVR) is an area of research that integrates fundamentals of combinatorics, probability and queueing theories. One of the early ersions of a DVR problem is the Dynamic Traeling Repairperson Problem [4] in which the goal is to minimize the expected time spent by each demand before being sered. In [4], the authors propose a policy that is optimal in the case of low arrial rate, and seeral policies within a constant factor of the optimal in the case of high arrial rate. In [8], they consider multiple ehicles, and ehicles with finite serice capacity. In [9], a single policy is proposed which is optimal for the case of low arrial rate and performs within a constant factor of the best known policy for the case of high arrial rate. Recently, there has been an upswing in ersions of DVR such as in [10] where different classes of demands hae been considered; and in [11] which addresses the case of demand impatience. DVR problems addressing motion constraints on the ehicle hae been presented in [1], while limited sensing range for the ehicle has been considered in [13]. Problems inoling multiple ehicles and with minimal communication hae been considered in [14]. In [15], adaptie and decentralized policies are deeloped for the multiple serice ehicle ersions, and in [16], a dynamic team-forming ariation of the Dynamic Traeling Repairperson problem is addressed. Related dynamic problems include [17], wherein a receding horizon control has been proposed for multiple ehicles to isit target points in uncertain enironments; and [18], wherein pickup and deliery problems hae been considered. Another related area of research is the ersion of the Euclidean Traeling Salesperson Problem with moing points or objects. The static ersion of the Euclidean Traeling Salesperson Problem consists of determining the minimum length tour through a gien set of static points in a region [19]. Vehicle routing with objects moing on straight lines was introduced in [7], in which a fixed number of objects moe in the negatie y direction with fixed speed, and the motion of the serice ehicle is constrained to be parallel to either the x or the y axis. For a ersion of this problem wherein the ehicle has arbitrary motion, termed as the translational Traeling Salesperson Problem, a polynomial-time approximation scheme has been proposed in [0] to catch all objects in minimum time. Another ariation of this problem with object motion on piece-wise straight line paths, and with different but finite object speeds has been addressed in [1]. Other ariants of the Euclidean Traeling Salesperson Problem in which the points are allowed to moe in different directions hae been addressed in []. A third area of research related to this work is a geometric location problem such as [3], and [4], where gien a set of static demands, the goal is to find supply locations that minimize a cost function of the distance from each October 4, 009 DRAFT

4 Necessary condition for stability of any policy Sufficient condition for stability of TMHP-based policy 30 Arrial rate λ Stabilization impossible for any policy 10 5 TMHP policy stable Demand speed Arrial rate λ Demand speed Arrial rate λ Demand speed Fig. 1. A summary of stability regions for the TMHP-based policy and the FCFS policy. Stable serice policies exist only for the region under the solid black cure. In the top figure, the solid black cure is due to part (i) of Theorem III.1 and the dashed blue cure is due to part (i) of Theorem III.. In the asymptotic regime shown in the bottom left, the dashed blue cure is described in part (ii) of Theorem III., and is different than the one in the top figure. In the asymptotic regime shown in the bottom right, the solid black cure is due to part (ii) of Theorem III.1, and is different from the solid black cure in the top figure. demand to its nearest supply location. In our problem, in the asymptotic regime of low arrial, when the arrial rate λ tends to zero for a fixed demand speed, the problem becomes one of proiding optimal coerage. In this regime, the demands are sered in a first-come-first-sered order; such policies are common in classical queuing theory [5]. Another related work is [6], in which the authors study the problem of deploying robots into a region so as to proide optimal coerage. Other releant literature include [7] in which multiple mobile targets are to be kept under sureillance by DRAFT October 4, 009

5 5 multiple mobile sensor agents. The authors in [8] propose mixed-integer-linear-program approach to assigning multiple agents to time-dependent cooperatie tasks such as tracking mobile targets. In an early ersion of this work [1], [], we analyzed a preliminary ersion of the TMHP-based policy and the FCFS policy respectiely. In this paper, we present and analyze a unified policy for the problem. We also present new simulation results to numerically determine the region of stability for the policy proposed in this paper. As in [17] and in [18], the problem addressed in this paper applies seeral control theoretic concepts to a releant robotic application. In particular, we design policies or control laws in order to stabilize a stochastic and dynamic system. The solution that we propose is a ersion of receding horizon control, in which one computes the optimal solution based on all of the present information and then repeatedly re-computes as new information becomes aailable. Our stability analysis relies on writing the closed-loop dynamical system as a recursion in a state of the system, and then ensuring that this state has a bounded eolution. Organization This paper is organized as follows. A short reiew of results on optimal motion and combinatorics is presented in Section II. The problem formulation, the TMHP-based policy, and the main results are presented in Section III. The FCFS policy is presented and analyzed in Section IV. Utilizing the results of Section IV, the main results are proen in Section V. Finally, simulation results are presented in Section VI. II. PRELIMINARY RESULTS In this section, we proide some background results which would be used in the remainder of the paper. A. Constant bearing control In this paper, we will use the following result on catching a moing demand in minimum time. Definition II.1 (Constant bearing control) Gien initial locations p := (X,Y ) R and q := (x,y) R of the serice ehicle and a demand, respectiely, with the demand moing in the positie y-direction with constant speed ]0,1[, the motion of the serice ehicle towards the point (x,y + T), where ( )(X x) T(p,q) := + (Y y) (Y y), (1) with unit speed is defined as the constant bearing control. Constant bearing control is illustrated in Figure and characterized in the following proposition. Proposition II. (Minimum time control, [9]) The constant bearing control is the minimum time control for the serice ehicle to reach the moing demand. October 4, 009 DRAFT

6 6 p = (X,Y ) C = (x, y + T) q = (x, y) (0, 0) W Fig.. reach the demand. Constant bearing control. The ehicle moes towards the point C := (x, y + T), where x, y, and T are as per Definition II.1, to B. Euclidean and Translational minimum Hamiltonian path (EMHP/TMHP) problems Gien a set of points in the plane, a Euclidean Hamiltonian path is a path that isits each point exactly once. A Euclidean minimum Hamiltonian path (EMHP) is a Euclidean Hamiltonian path that has minimum length. In this paper, we also consider the problem of determining a constrained EMHP which starts at a specified start point, isits a set of points and terminates at a specified end point. More specifically, the EMHP problem is as follows. Gien n static points placed in R, determine the length of the shortest path which isits each point exactly once. An upper bound on the length of such a path for points in a unit square was gien by Few [30]. Here we extend Few s bound to the case of points in a rectangular region. For completeness, we hae included the proof in the Appendix. Lemma II.3 (EMHP length) Gien n points in a 1 h rectangle in the plane, where h R >0, there exists a path that starts from a unit length edge of the rectangle, passes through each of the n points exactly once, and terminates on the opposite unit length edge, haing length upper bounded by hn + h + 5/. Gien a set Q of n points in R, the Euclidean Traeling Salesperson Problem (ETSP) is to determine the shortest tour, i.e., a closed path that isits each point exactly once. Let ETSP(Q) denote the length of the ETSP tour through Q. The following is the classic result by Beardwood, Halton, and Hammersly [31]. Theorem II.4 (Asymptotic ETSP length, [31]) If a set Q of n points are distributed independently and uniformly in a compact region of area A, then there exists a constant β TSP such that, with probability one, ETSP(Q) lim = β TSP A. () n + n DRAFT October 4, 009

7 7 The constant β TSP has been estimated numerically as β TSP ± 0.000, [3]. Next, we describe the TMHP problem which was proposed and soled in [0]. This problem is posed as follows. Gien initial coordinates; s of a start point, Q := {q 1,...,q n } of a set of points, and f of a finish point, all moing with the same constant speed and in the same direction, determine a path that starts at time zero from point s, isits all points in the set Q exactly once and ends at the finish point, and the length L T, (s, Q,f) of which is minimum. In what follows, we wish to determine the TMHP through points which translate in the positie y direction. We also assume the speed of the serice ehicle to be normalized to unity, and hence consider the speed of the points ]0,1[. A solution for the TMHP problem is the Conert-to-EMHP method: (i) For ]0,1[, define the conersion map cnrt : R R by ( x cnrt (x,y) =, y ). (ii) Compute the EMHP that starts at cnrt (s), passes through the set of points gien by {cnrt (q 1 ),...,cnrt (q n )} and ends at cnrt (f). (iii) Moe between any two demands using the constant bearing control. For the Conert-to-EMHP method, the following result is established. Lemma II.5 (TMHP length, [0]) Let the initial coordinates s = (x s,y s ) and f = (x f,y f ), and the speed of the points ]0,1[. The length of the TMHP is L T, (s, Q,f) = L E (cnrt (s), {cnrt (q 1 ),...,cnrt (q n )},cnrt (f)) + (y f y s ), where L E (cnrt (s), {cnrt (q 1 ),...,cnrt (q n )},cnrt (f)) denotes the length of the EMHP with starting point cnrt (s), passing through the set of points {cnrt (q 1 ),...,cnrt (q n )}, and ending at cnrt (f). This lemma implies the following result: gien a start point, a set of points and an end point all of whom translate in the positie ertical direction at speed ]0,1[, the order of the points followed by the optimal TMHP solution is the same as the order of the points followed by the optimal EMHP solution through a set of static locations equal to the locations of the moing points at initial time conerted ia the map cnrt. III. PROBLEM FORMULATION AND THE TMHP-BASED POLICY In this section, we pose the dynamic ehicle routing problem with translating demands and present the TMHPbased policy along with the main results. A. Problem Statement We consider a single serice ehicle that seeks to serice mobile demands that arrie ia a spatio-temporal process on a line segment with length W along the x-axis, termed the generator. The ehicle is modeled as a first-order integrator with speed upper bounded by one. The demands arrie uniformly distributed on the generator ia a October 4, 009 DRAFT

8 8 temporal Poisson process with intensity λ > 0, and moe with constant speed ]0,1[ along the positie y-axis, as shown in Figure 3. We assume that once the ehicle reaches a demand, the demand is sered instantaneously. The ehicle is assumed to hae unlimited fuel and demand sericing capacity. (X(t), Y (t)) (0, 0) W Fig. 3. The problem set-up. The thick line segment is the generator of mobile demands. The dark circle denotes a demand and the square denotes the serice ehicle. We define the enironment as E := [0,W] R 0 R, and let p(t) = [X(t),Y (t)] T E denote the position of the serice ehicle at time t. Let Q(t) E denote the set of all demand locations at time t, and n(t) the cardinality of Q(t). Sericing of a demand q i Q and remoing it from the set Q occurs when the serice ehicle reaches the location of the demand. A static feedback control policy for the system is a map P : E F(E) R, where F(E) is the set of finite subsets of E, assigning a commanded elocity to the serice ehicle as a function of the current state of the system: ṗ(t) = P(p(t), Q(t)). Let D i denote the time that the ith demand spends inside the set Q, that is, the delay between the generation of the ith demand and the time it is sericed. The policy P is stable if under its action, lim sup E[D i ] < +, i + that is, the steady state expected delay is finite. Equialently, the policy P is stable if under its action, lim sup E[n(t)] < +, t + that is, if the ehicle is able to serice demands at a rate that is on aerage at least as fast as the rate at which new demands arrie. In what follows, our goal is to design stable control policies for the system. To obtain further intuition into stability of a policy, consider the -λ parameter space. In the asymptotic regime of high speed, where 1, the arrial rate λ must tend to zero for stability, otherwise the serice ehicle would hae to moe successiely further away from the generator in expected alue, thus making the system unstable. Similarly, stability in the asymptotic region of high arrial, where λ + implies that the demand speed must tend to zero. Thus, our primary goals are: (i) to characterize regions in the -λ parameter space in which one can neer design any stable policy, and (ii) design a noel policy and determine its stability region in the -λ parameter space, with additional emphasis in the aboe two asymptotic regimes. In addition, for the asymptotic regime of low DRAFT October 4, 009

9 9 arrial, where for a fixed speed < 1, the arrial rate λ 0 +, stability is intuitie as demands arrie ery rarely. Hence, in this regime, we seek to minimize the steady state expected delay for a demand. B. The TMHP-based policy We now present a noel receding horizon serice policy for the ehicle that is based on the repeated computation of a translational minimum Hamiltonian path through successie groups of outstanding demands. For a gien arrial rate λ and demand speed ]0,1[, let (X,Y ) denote the ehicle location in the enironment that minimizes the expected time to serice a demand once it appears on the generator. The expression for the optimal location (X,Y ) is postponed to Section IV-A. The TMHP-based policy is summarized in Algorithm 1, and an iteration of the policy is illustrated in Figure Algorithm 1: The TMHP-based policy Assumes: The optimal location (X,Y ) E is gien. if no outstanding demands are present in E then else Repeat. Moe to the optimal position (X,Y ). Serice all outstanding demands by following a translational minimum Hamiltonian path starting from the ehicle s current location, and terminating at the demand with the lowest y-coordinate. (X, Y ) q last q last q last Fig. 4. An iteration of a receding horizon serice policy. The ehicle shown as a square seres all outstanding demands shown as black dots as per the TMHP that begins at (X, Y ) and terminates at q last which is the demand with the least y-coordinate. The first figure shows a TMHP at the beginning of an iteration. The second figure shows the ehicle sericing the demands through which the TMHP has been computed while new demands arrie in the enironment. The third figure shows the ehicle repeating the policy for the set of new demands when it has completed serice of the demands present at the preious iteration. C. Main Results The following is a summary of our main results and the locations of their proofs within the paper. We begin with a necessary condition on the problem parameters for stability of any policy, the proof of which is presented October 4, 009 DRAFT

10 10 in Section V-A. The condition is policy independent and especially in the asymptotic regime of high speed, i.e., 1 (and therefore λ 0 + ), we characterize the way in which λ 0 +. Theorem III.1 (Necessary condition for stability) The following are necessary conditions for the existence of a stabilizing policy: (i) For a general ]0,1[, λ 4 W. (ii) For the asymptotic regime of high speed, where 1, eery stabilizing policy must sere the demands in the order in which they arrie. Further, the arrial rate must tend to zero as per λ 3 W ln(). Next, we present a sufficient condition on the problem parameters that ensures stability of the TMHP-based policy, the proof of which is presented in Section V-B. We present a condition for a general alue of the demand speed and especially in the asymptotic regime of high arrial, i.e., λ + (and therefore 0 + ), we characterize the way in which λ + and 0 +, in order to ensure stability of the TMHP-based policy. We first introduce the following notation. Let 3 W λ FCFS (,W) := W 1 +, for suf, 1 )), otherwise, (1 + ) ( C suf ln ( 1 where C suf = π/ ln(0.5 3/ ), and suf is the solution to 1 3 ( )(C suf ln( ) + ln ) = 0, and is approximately equal to /3. Theorem III. (Sufficient condition for stability) The following are sufficient conditions for stability of the TMHPbased policy. (i) For a general ]0,1[, { ( ) 3/ } λ < max W(1 + ),λ FCFS(,W). (ii) In the asymptotic regime of high arrial where λ + (and so 0 + ), the policy is stable if λ < 1 β TSP W, where β TSP A plot of the necessary and sufficient conditions is shown in Fig. 1. In the asymptotic regime of high speed, the sufficient condition from part (i) of Theorem III. simplifies to 6 λ < W ln() =: λ1suf, and the necessary condition established in part (ii) of Theorem III.1 simplifies to 3 λ W ln() =: λ1nec. DRAFT October 4, 009

11 11 In the asymptotic regime of high arrial, the sufficient condition from part (ii) of Theorem III. is λ < 1/(βTSP W) =: λ 0+ suf, and the necessary condition established in part (i) of Theorem III.1 is λ 4/(W) =: λ0+ nec. Theorems III.1 and III. immediately lead to the following corollary. Corollary III.3 (Constant factor sufficient condition) In the asymptotic regime of (i) high speed, which is the limit as 1, the ratio λ 1 nec/λ 1 suf 3. (ii) high arrial, which is the limit as λ +, the ratio λ 0+ nec/λ 0+ suf 4β TSP.07. The third and final result shows optimality of the TMHP-based policy with respect to minimizing the expected delay in the asymptotic regimes of low arrial and high speed. Theorem III.4 (Optimality of TMHP-based policy) In the asymptotic regimes of (i) low arrial, where λ 0 + for a fixed ]0,1[, and (ii) high speed, where 1 (and therefore λ 0 + ), the TMHP-based policy seres the demands in the order in which they arrie, and also minimizes the expected time to serice a demand. In other words, the TMHP-based policy becomes equialent to a first-come-first-sered (FCFS) policy in the aboe two asymptotic regimes. Theorem III.4 is proen for the FCFS policy in Section V-B. By the equialence between FCFS and the TMHP-based policies in the aboe two asymptotic regimes, the result also holds for the TMHP-based policy. To study the stability of the TMHP-based policy, we introduce and analyze the FCFS policy in the next section. IV. THE FIRST-COME-FIRST-SERVED (FCFS) POLICY AND ITS ANALYSIS In this section, we present the FCFS policy and establish some of its properties, and use these properties as tools to analyze the TMHP-based policy. In the FCFS policy, the serice ehicle uses constant bearing control and serices the demands in the order in which they arrie. If the enironment contains no demands, the ehicle moes to the location (X,Y ) which minimizes the expected time to catch the next demand to arrie. This policy is summarized in Algorithm Algorithm : The FCFS policy Assumes: The optimal location (X,Y ) E is gien. if no outstanding demands are present in E then else Moe toward (X,Y ) until the next demand arries. Moe using the constant bearing control to serice the furthest demand from the generator. 5 Repeat. October 4, 009 DRAFT

12 1 Figure 5 illustrates an instance of the FCFS policy. The following lemma establishes the relationship between the FCFS policy and the TMHP-based policy. (X, Y ) q i q i+ q i+1 q i+3 W Fig. 5. The FCFS policy. The ehicle serices the demands in the order of their arrial in the enironment, using the constant bearing control. Lemma IV.1 (Relationship between TMHP-based policy and FCFS policy) Gien an arrial rate λ and a demand speed, if the FCFS policy is stable, then the TMHP-based policy is stable. Proof: Consider an initial ehicle position and a set of outstanding demands, all of which hae lower y- coordinates than the ehicle. Let us compare the amount of time required to serice the outstanding demands using the TMHP-based policy with the amount of time required for the FCFS policy. Both policies generate paths through all outstanding demands, starting at the initial ehicle location, and terminating at the outstanding demand with the lowest y-coordinate. By definition, the TMHP-based policy generates the shortest such path. Thus, the TMHP-based policy will require no more time to serice all outstanding demands than the FCFS policy. Since this holds at eery iteration of the policy, the region of stability of TMHP-based policy contains the region of stability for the FCFS policy. In the following subsections, we analyze the FCFS policy. We then combine these results with the aboe lemma to establish analogous results for the TMHP-based policy. The first question is, how do we compute the optimal position (X,Y )? This will be answered in the following subsection. A. Optimal Vehicle Placement In this subsection, we study the FCFS policy when ]0,1[ is fixed and λ 0 +. In this regime, stability is not an issue as demands arrie ery rarely, and the problem becomes one of optimally placing the serice ehicle (i.e., determining (X,Y ) in the statement of the FCFS policy). We seek to place the ehicle at location that minimizes the expected time to serice a demand once it appears on the generator. Demands appear at uniformly random positions on the generator and the ehicle uses the constant bearing control to reach the demand. Thus, the expected time to reach a demand generated at position q = (x,0) DRAFT October 4, 009

13 13 from ehicle position p = (X,Y ) is gien by E[T(p,q)] = 1 W( ) W 0 ( ( )(X x) + Y Y ) dx. (3) The following lemma characterizes the way in which this expectation aries with the position p. Lemma IV. (Properties of the expected time) The expected time p E[T(p,q)] is conex in p, for all p [0,W] R >0. Additionally, there exists a unique point p := (W/,Y ) R that minimizes p E[T(p,q)]. Proof: Regarding the first statement, it suffices to show that the integrand in equation (3), T(p,(x, 0)) is conex for all x. To do this we compute the Hessian of T((X,Y ),(x,0)) with respect to X and Y. Thus, for Y > 0, T T X X Y 1 Y = ) Y (X x) 3/. (( )(X x) + Y Y (X x) (X x) T Y X T Y The Hessian is positie semi-definite because its determinant is zero and its trace is non-negatie. This implies that T(p,q) is conex in p for each q = (x,0), from which the first statement follows. Regarding the second statement, since demands are uniformly randomly generated on the interal [0, W], the optimal horizontal position is X = W/. Thus, it suffices to show that Y E[T((W/,Y ),q)] is strictly conex. From the T/ Y term of the Hessian we see that T(p,q) is strictly conex for all x W/. But, letting p = (W/,Y ) and q = (x,0) we can write E[T(p,q)] = 1 W( ) x [0,W]\{W/} T(p,q)dx. The integrand is strictly conex for all x [0,W] \ {W/}, implying that E[T(p,q)] is strictly conex on the line X = W/, and that the point (W/,Y ) is the unique minimizer. Lemma IV. tells us that there exists a unique point p := (X,Y ) which minimizes the expected trael time. In addition, we know that X = W/. Obtaining a closed form expression for Y does not appear to be possible. Computing the integral in equation (3), with X = W/, one can obtain ( E[T(p,q)] = Y ( aw a 4Y Y ln 1 + aw ) ) aw aw 4Y 4Y, where a =. For each alue of and W, this conex expression can be easily numerically minimized oer Y, to obtain Y. A plot of Y as a function of for W = 1 is shown in Figure 6. For the optimal position p, the expected delay between a demand s arrial and its serice completion is D := E[T(p,(x,0))]. Thus, a lower bound on the steady-state expected delay of any policy is D. We now characterize the steady-state expected delay of the FCFS policy D FCFS, as λ tends to zero. Lemma IV.3 (FCFS optimality) Fix any ]0,1[. Then in the limit as λ 0 +, the FCFS policy minimizes the expected time to serice a demand, i.e., D FCFS D. October 4, 009 DRAFT

14 Optimal position Y () Demand speed Fig. 6. The optimal position Y of the serice ehicle which minimizes the expected distance to a demand, as a function of. In this plot, the generator has length W = 1. Proof: We hae shown how to compute the position p := (X,Y ) which minimizes equation (3). Thus, if the ehicle is located at p, then the expected time to serice the demand is minimized. But, as λ 0 +, the probability that demand i + 1 arries before the ehicle completes serice of demand i and returns to p tends to zero. Thus, the FCFS policy is optimal as λ 0 +. Remark IV.4 (Minimizing the worst-case time) Another metric that can be used to determine the optimal placement (X,Y ) is the worst-case time to serice a demand. Using an argument identical to that in the proof of Lemma IV.3, it is possible to show that for fixed ]0,1[, and as λ 0 +, the FCFS policy, with (X,Y ) = (W/, W/), minimizes the worst-case time to serice a demand. B. A Necessary Condition for FCFS Stability In the preious subsection, we studied the case of fixed and λ 0 +. In this subsection, we analyze the problem when λ > 0, and determine necessary conditions on the magnitude of λ that ensure the FCFS policy remains stable. To establish these conditions we utilize a standard result in queueing theory (cf. [5]) which states that a necessary condition for the existence of a stabilizing policy is that λe[t] 1, where E[T] is the expected time to serice a demand (i.e., the trael time between demands). We begin with the following result. Proposition IV.5 (Special case of equal speeds) For = 1, there does not exist a stabilizing policy. Proof: When = 1, each demand and the serice ehicle moe at the same speed. If a demand has a higher ertical position than the serice ehicle, then clearly the serice ehicle cannot reach it. The same impossibility DRAFT October 4, 009

15 15 result holds if the demand has the same ertical position and a distinct horizontal position as the serice ehicle. In summary, a demand can be reached only if the serice ehicle is aboe the demand. Next, note that the only policy that ensures that a demand s y-coordinate neer exceeds that of the serice ehicle (i.e., that all demands remain below the serice ehicle at all time) is the FCFS policy. In what follows, we proe the proposition statement by computing the expected time to trael between demands using the FCFS policy. First, consider a ehicle location p := (X,Y ) and a demand location with initial location q := (x,y), the minimum time T in which the ehicle can reach the demand is gien by T(p,q) = (X x) + (Y y), if Y > y, (4) (Y y) and is undefined if Y y. Second, assume there are many outstanding demands below the serice ehicle, and none aboe. Suppose the serice ehicle completed the serice of demand i at time t i and position (x i (t i ),y i (t i )). Let us compute the expected time to reach demand i + 1, with location (x i+1 (t i ),y i+1 (t i )). Since arrials are Poisson, it follows that y i (t i ) > y i+1 (t i ), with probability one. To simplify notation, we define x = x i (t i ) x i+1 (t i ) and y = y i (t i ) y i+1 (t i ). Then, from equation (4) T(q i,q i+1 ) = x + y y Taking expectation and noting that x and y are independent, E[T(q i,q i+1 )] = 1 ( E [ x ] E = 1 ( ) x y + y. [ 1 y ] ) + E[ y]. Now, we note that E[ y] = 1/λ, that E [ x ] is a positie constant independent of λ, and that [ E 1 y ] = Thus E[T(q i,q i+1 )] = +, and for eery λ > 0, + y=0 1 y λe λy dy = +. λe[t(q i,q i+1 )] = +. This means that the necessary condition for stability, i.e., λe[t(q i,q i+1 )] 1, is iolated. Thus, there does not exist a stabilizing policy. Next we look at the FCFS policy and gie a necessary condition for its stability. Lemma IV.6 (Necessary stability condition for FCFS) A necessary condition for the stability of the FCFS policy is 3 W, λ 3 ( W (1 + ) C nec ln ( 1 for nec, )), otherwise, where C nec = ln() γ, where γ is the Euler constant; and nec is the solution to the equation (1 + )(C nec 0.5 ln( ) + ln) = 0, and is approximately equal to 4/5. October 4, 009 DRAFT

16 16 Proof: Suppose the serice ehicle completed the serice of demand i at time t i at position (x i (t i ),y i (t i )), and demand i + 1 is located at (x i+1 (t i ),y i+1 (t i )). Define x := x i (t i ) x i+1 (t i ) and y := y i (t i ) y i+1 (t i ). For ]0,1[, the trael time between demands is gien by T = 1 ( ) ( ) x + y y. (5) Obsere that the function T is conex in x and y. Jensen s inequality leads to E[T] 1 ( ) ( )(E[ x]) + (E[ y]) E[ y]. Substituting the expressions for the expected alues, we obtain E[T] 1 ( From the necessary condition for stability, we must hae By simplifying, we obtain ( ) W 9 + λ λ 1 ( ( )W λe[t] 1 λ 9 ). ) + λ 1. λ λ 3 W. (6) This proides a good necessary condition for low. Next, we obtain a much better necessary condition for large. Since T is conex in x, we apply Jensen s inequality to equation (5) to obtain E[T y] 1 ( ) ( )W /9 + y y, (7) where E[ x] = W/3. Now, the random ariable y is distributed exponentially with parameter λ/ and probability density function Un-conditioning equation (7) on y, we obtain E[T] = + 0 E[T y]f(y)dy f(y) = λ e λy/. λ ( ) + 0 ( ) ( )W + y 9 y e λy/ dy. (8) The right hand side can be ealuated using the software Maple R and equals ( πw 3 λw [H ) ( λw )] 1 Y λ( ), where H 1 : R R is the 1st order Strue function and Y 1 : R R is 1st order Bessel function of the nd kind [33]. Using a Taylor series expansion of the function H 1 (z) Y 1 (z) about z = 0, followed by a subsequent analysis of the higher order terms, one can show that H 1 (z) Y 1 (z) 1 ( ) π z + C necz z ln(z), z 0, DRAFT October 4, 009

17 17 where C nec = 1/ + ln() γ, and γ is the Euler constant. This inequality implies that equation (8) can be written as where we hae used the fact that ( ( E[T] λ(1 + ) + λw λw )) C nec ln, 18 3 λ() λ( ) = λ(1 + ). To obtain a stability condition on λ we wish to remoe λ from the ln term. To do this, note that from equation (6) we hae λw/3 < 1, and thus ( E[T] λ(1 + ) + λw C nec ln Wλ 18 3 ln W ) ( ( )) 3 λ(1 + ) + λw C nec ln. 18 The necessary stability condition is λe[t] 1, from which a necessary condition for stability is ( ( )) λ W C nec ln = Soling for λ when ln( /) < C nec, we obtain that 3 λ ( W (1 + ) C nec ln ( 1 )), (9) The condition C nec > ln( /), implies that the aboe bound holds for all > 1/ 1 + e Cnec. We now hae two bounds; equation (6) which holds for all ]0,1[, and equation (9) which holds for > 1/. The final step is to determine the alues of for which each bound is actie. To do this, we set the right-hand side of equation (6) equal to the right-hand side of equation (9) and denote the solution by nec. Thus, the necessary condition for stability is gien by equation (6) when nec, and by equation (9) when > nec. C. A Sufficient Condition for FCFS stability In Section IV-B, we determined a necessary condition for stability of the FCFS policy. In this subsection, we will derie the following sufficient condition on the arrial rate that ensures stability for the FCFS policy. To establish this condition, we utilize a standard result in queueing theory (cf. [5]) which states that a sufficient condition for the existence of a stabilizing policy is that λe[t] < 1, where E[T] is the expected time to serice a demand (i.e., the trael time between demands). Lemma IV.7 (Sufficient stability condition for FCFS) The FCFS policy is stable if 3 W 1 +, for suf, λ < 1 W (1 + ) ( C suf ln ( )), otherwise, 1 where C suf = π/ ln(0.5 3/ ), and suf is the solution to 1 3 ( )(C suf ln( ) + ln ) = 0, and is approximately equal to /3. October 4, 009 DRAFT

18 18 Proof: We begin with the expression for the trael time between two consecutie demands using the constant bearing control (cf. Definition II.1). ( ) x T = + y y x + y, (10) where we used the inequality a + b a + b. Taking expectation, E[T] W 3 + λ( ), since the demands are distributed uniformly in the x-direction and Poisson in the y-direction. A sufficient condition for stability is λe[t] < 1 λ < 3 W 1 +. (11) The upper bound on T gien by equation (10) is ery conseratie except for the case when is ery small. Alternatiely, taking expected alue of T conditioned on y, and applying Jensen s inequality to the square-root part, we obtain E[T y] 1 ( ) ( )W /6 + y y, since E [ x ] = W /6. Following steps which are similar to those between equation (7) and equation (8), we obtain πw E[T] 6 ( λw [H ) ( λw )] 1 Y λ( ). (1) In [33], polynomial approximations hae been proided for the Strue and Bessel functions in the interals [0, 3] and [3,+ ). We seek an upper bound for the right-hand side of (1) when is sufficiently large, i.e., when the argument of H 1 and Y 1 is small. From [33], we know that H 1 (z) z, Y 1(z) ( J 1 (z)ln z π 1 ), and J 1 (z) z, for 0 z 3, z where z := λw /( 6), and J 1 : R R denotes the Bessel function of the first kind. To obtain a lower bound on Y 1 (z), we obsere that if 0 z, then due to the ln term in the aboe lower bound for Y 1 (z), we can substitute z/ in place of J 1 (z). Thus, we obtain H 1 (z) z, Y 1(z) π Substituting into equation (1), we obtain [ πw E[T] 6 λw ( π λw λw 6 which yields E[T] λw 1 ( π ln λw 3 ln ( z ln z 1 ), for 0 z. (13) z ) 3 ln λw )] 6 λ( ), 1 λ(1 + ). (14) Now, let λ be the least upper bound on λ for which the FCFS policy is unstable, i.e., for eery λ < λ, the FCFS policy is stable. To obtain λ, we need to sole λ E[T] = 1. Using equation (14), we can obtain a lower DRAFT October 4, 009

19 19 bound on λ by simplifying λ W 1 ( π ln λ W 3 ln ) From the condition gien by equation (11), the second term in the parentheses satisfies λ W 3 > 1 +. Thus, we obtain, 1 λ W (1 + ) ( C suf ln ( )), 1 where C suf = π/ ln(0.5 3/ ). Since λ < λ implies stability, a sufficient condition for stability is 1 λ < W (1 + ) ( C suf ln ( )). (15) 1 To determine the alue of the speed suf beyond which this is a less conseratie condition than equation (11), we sole 1 suf ( )) W (1 + suf ) 1 C ln( = 3 suf W suf suf 1 + suf. For > suf, one can erify that the numerical alue of the argument of the Strue and Bessel functions is less than, and so the bounds gien by equation (13) used in this analysis are alid. Thus, a sufficient condition for stability is gien by equation (11) for suf, and by equation (15) for > suf. Remark IV.8 (Tightness in limiting regimes) As 0 +, the sufficient condition for FCFS stability becomes λ < 3/W, which is exactly equal to the necessary condition gien in Lemma IV.6. Thus, the condition for stability is asymptotically tight in this limiting regime. Figure 7 shows a comparison of the necessary and sufficient stability conditions for the FCFS policy. It should be noted that λ conerges to 0 + extremely slowly as tends to 1, and still satisfy the sufficient stability condition in Lemma IV.7. For example, with = , the FCFS policy can stabilize the system for an arrial rate of 3/(5W). V. PROOFS OF THE MAIN RESULTS In this section, we present the proofs of the main results which were presented in Section III-C. A. Proof of Theorem III.1 We first present the proof of part (i). We begin by looking at the distribution of demands in the serice region. October 4, 009 DRAFT

20 0 6 Necessary condition for stability of FCFS policy Sufficient condition for stability of FCFS policy 5 Arrial rate λ Demand speed Fig. 7. The necessary and sufficient conditions for the stability for the FCFS policy. The dashed cure is the necessary condition for stability as established in Lemma IV.6; while the solid cure is the sufficient condition for stability as established in Lemma IV.7. Lemma V.1 (Distribution of outstanding demands) Suppose the generation of demands commences at time 0 and no demands are sericed in the interal [0,t]. Let Q denote the set of all demands in [0,W] [0,t] at time t. Then, gien a Borel measurable set R of area A contained in [0,W] [0,t], e λa ( λa) n P[ R Q = n] =, where n! λ := λ/(w). Proof: We first establish the result for a rectangle. Let R = [l,l+ l] [h,h+ h] be a rectangle contained in [0,W] [0,t] with area A = l h. Let us calculate the probability that at time t, R Q = n (that is, the probability that R contains n points in Q). We hae [ [ h P[ R Q = n] = P i demands arried in, h + h ]] P[n of i are generated in [l,l + l]]. i=n Since the generation process is temporally Poisson and spatially uniform the aboe equation can be rewritten as Now we compute P[ R Q = n] = P[i demands arried in [0, h/]] P[n of i are generated in [0, l]]. (16) i=n P[i demands arried in [0, h/]] = e λ h/ (λ h/) i, i! and P[n of i are in [0, l]] = ( i n )( l W ) n ( 1 l ) i n, W DRAFT October 4, 009

21 1 so that, substituting these expressions and adopting the shorthands L := l/w and H := h/, equation (16) becomes P[ R Q = n] = e λh L n (λh) i ( ) i (1 L) i n. (17) i! n i=n Rewriting (λh) i as (λh) n (λh) i n, and using the definition of binomial ( ) i n = i! n!(i n)!, equation (17) reads λh (λlh)n (λh(1 L)) j λh+λh(1 L) (λlh)n λlh (λlh)n P[ R Q = n] = e = e = e. n! j! n! n! Finally, since LH = A/(W), we obtain j=0 P[ R Q = n] = e λa ( λa) n where λ := λ/(w). Thus, the result is established for a closed rectangle. But notice that the result also holds for the case when a rectangle is open. Further, the result holds for a countable union of disjoint rectangles due to the demand generation process being uniform along the generator and Poisson in time. Now, a Borel measurable set on [0,W] [0,t] can be generated by countable unions and complements of rectangles. But both, the unions and the complements of rectangles can be written as a disjoint union of a countable number of rectangles. Hence, the result holds for any Borel measurable set in [0,W] [0,t]. n!, Remark V. (Uniformly distributed demands) Lemma V.1 shows us that the number of demands in an unsericed region with area A is Poisson distributed with parameter λa/(w), and conditioned on this number, the demands are distributed uniformly. Lemma V.3 (Trael time bound) Consider the set Q of demands that are uniformly distributed in E at time t. Let T d be a random ariable giing the minimum amount of time required to trael to a demand in Q from a ehicle position selected a priori. Then E[T d ] 1 W λ. Proof: Let p = (X,Y ) denote the ehicle location selected a priori. To obtain a lower bound on the minimum trael time, we consider the best-case scenario, when no demands hae been sericed in the time interal [0, t]. Consider a demand in Q with position (x,y) at time t. Using Proposition II., we can write the trael time T from p to q := (x,y) implicitly as T(p,q) = (X x) + ((Y y) T(p,q)). (18) Next, define the set S T as the collection of demands that can be reached from (X,Y ) in T or fewer time units. From equation (18) we see that when < 1, the set S T is a disk of radius T centered at (X,Y T). That is, S T := {(x,y) E (X x) + ((Y T) y) T }, where we hae omitted the dependence of T on p and q. The area of the set S T, denoted area(s T ), is upper bounded by πt, and the area is equal to πt if the S T does not intersect a boundary of E. Now, by Lemma V.1 October 4, 009 DRAFT

22 the demands in an unsericed region are uniformly randomly distributed with density λ = λ/(w). Let us compute the distribution of T d := min q Q T(p,q). For eery ehicle position p chosen before the generation of demands, the probability that T d > T is gien by P[T d > T] = P[ S T Q = 0] = e λ area(s T ) e λπt /(W), where the second equality is by Lemma V.1, and the last inequality comes from the fact that area(s T ) πt (Note that this is equialent to assuming that the entire plane R contains demands with density λ). Hence we hae + + π E[T d ] P[T d > T]dT e λπt /(W) dt = λπ/(w) = 1 W λ. 0 0 We can now proe part (i) of Theorem III.1. Proof of part (i) of Theorem III.1: A necessary condition for the stability of any policy is λe[t] 1, where E[T] is the steady-state expected trael time between demands i and i+1. For eery policy E[T] E[T d ] 1 W λ. Thus, a necessary condition for stability is that λ 1 W λ 1 λ 4 W. Remark V.4 (Constant fraction serice) A necessary condition for the existence of a policy which serices a fraction c ]0,1] of the demands is that λ 4 c W. Thus, for a fixed ]0,1[ no policy can serice a constant fraction of the demands as λ +. This follows because in order to serice a fraction c we require that cλe[t d ] < 1. In order to serice a fraction c of the demands, we consider a subset of the generator haing length cw, with the arrial rate on that subset being equal to cλ. The use of the TMHP-based policy on this subset and with the arrial rate cλ gies a sufficient condition for stability analogous to Theorem III., but with an extra term of c in the denominator. For the proof of part (ii) of Theorem III.1, we first recall from Lemma IV.6 that for stability of the FCFS policy, although λ must go to zero as 1, it can go ery slowly to 0. Specifically, λ goes to zero as 1 ln(). This quantity goes to zero more slowly than any polynomial in (). We are now ready to complete the proof of Theorem III.1. DRAFT October 4, 009

23 3 Proof of part (ii) of Theorem III.1: The central idea of this proof is that in the limit as 1, if the ehicle skips a demand and serices the next one at an instance, then to reach the demand it skipped, the ehicle has to trael infinitely far from the generator which leads to instability. Obsere that the condition on λ in the statement of part (ii) is the expression gien by the necessary condition for FCFS stability in the asymptotic regime as 1, from Lemma IV.6. Therefore, suppose there is a policy P that does not sere demands FCFS, but can stabilize the system with λ = B() p, for some p > 0, and B > 0. Let t i be the first instant at which policy P deiates from FCFS. Then, the demand sered immediately after i is demand i + k for some k > 1. When the ehicle reaches demand i + k at time t i+1, demand i + 1 has moed aboe the ehicle. To ensure stability, demand i + 1 must eentually be sered. The time to trael to demand i + 1 from any demand i + j, where j > 1, is ( ) ( ) x y T(q i+j,q i+1 ) = + + y y + y = y, where x and y are now the minimum of the x and y distances from q i+j to the q i+1. The random ariable y/ is Erlang distributed with shape j 1 1 and rate λ, implying P[ y c] 1 e λc/, for each c > 0, and in particular, for c = () 1/ p. Now, since λ = B() p as 1, y > () 1/ p, with probability one. Thus T(q i+j,q i+1 ) () (p+1/), with probability one, as 1. Thus, the expected number of demands that arrie during T(q i+j,q i+1 ) is λt(q i+j,q i+1 ) B() p () (p+1/) B() 1/ +, as 1. This implies that with probability one, the policy P becomes unstable when it deiates from FCFS and that any deiation must occur with probability zero as 1. Thus, a necessary condition for a policy to be stabilizing with λ = B() p is that, as 1, the policy must sere demands in the order in which they arrie. But this needs to hold for eery p and, by letting p go to infinity, B() p conerges to zero for all (0,1]. Thus, a non-fcfs policy cannot stabilize the system no matter how quickly λ 0 + as 1. Hence, as 1, eery stabilizing policy must sere the demands in the order in which they arrie. Additionally, notice that the definition of the FCFS policy is that it uses the minimum time control (i.e., constant bearing control) to moe between demands, thus the expression in part (ii) of Theorem III.1 is a necessary condition for all stabilizing policies as 1. October 4, 009 DRAFT