Beyond heavy-traffic regimes: Universal bounds and controls for the single-server (M/GI/1+GI) queue

Transcription

1 1 / 33 Beyond heavy-traffic regimes: Universal bounds and controls for the single-server (M/GI/1+GI) queue Itai Gurvich Northwestern University Junfei Huang Chinese University of Hong Kong Stochastic Networks 216

2 2 / 33 The intuitive Derivation of a Brownian Queue Service,, 1 Load The waiting time/workload process in the M/GI/1 queue: A(t) W (t) = W () + s i (t I(t)) i=1 A(t) = W () (1 ρ)t (t I(t)) + s i ρt i=1 (M/GI/1) Ŵ (t) = W () (1 ρ)t (t Î(t)) + E[s 2 ]B(t) (Brownian Queue)

3 3 / 33 Brownian approximations as a model A tractable and useful tool in the modeler s toolbox Pricing in queues (e.g. Kim and Randhawa, 215) Competition between queues (e.g. Allon and Federgruen, 28) Contracting in services (e.g. Akan et. al. 211) Inventory Optimization (e.g. Allon and Van Mieghem 21) Initial Model difficult Brownian approximation? Accurate

4 Example: Dual Sourcing (Inventory) Net Inventory evolution (Mexico and.3 China renewal inputs S c, S M ): I(t) = I() + S c (t) +.25 S M (TM s (t)) D(t), 121 t TM s (t) = 1{I(u) <.2 s}ds. 9 zed via Simulation 5 Scaled cost: Brownian vs. optimal TBS Relative China cost c C /c M Scaled cost Relative China cost c C /c M.35 Optimization by simulation.3 = 1 = 1 = 1 Scaled cost Asymptotic (analytical).25 * C.2 Simulated cost of Brownian prescription = 1 = 1 = 1 Optimization by simulation = 1 = 1 = 1 Asymptotic (analytical) C * Relative China cost c C /c M, the relative error in scaled cost ption and the optimal control was ain implication is that the Browngood and useful approximation of and the cost Ĉ ˆ p ˆ M p when using the s formulae. One can observe that the scaled square Gad Allon, Jan Simulated A. Van cost Mieghem of Brownian (21). prescription Global Dual Sourcing: Tailored Base-Surge Allocation to Near- and Offshore = 1Production. = 1 Management = 1 Science 56(1): / 33

5 Example: Admission Control to a Many-Server Queue Koçağa, Yaşar Levent and Ward, Amy R (21). Admission control for a multi-server queue with abandonment. Queueing Systems, 65(3): ( ) 1 Asymptotic optimality for ρ() = 1 + O as (and N) Punchline: Universally accurate 5 / 33

6 Utilization (regime) assumptions and consequences 11 1 Service exp 1 Load 1.1 Patience exp 1 Embedding Consequence (as, µ ) ρ() = W ( ) Reflected OU (critical load) ρ() 1.1 (W ( ) µ (ρ() 1)) free OU θ (overload) Universal process approx in Ward and Glynn (23), Ward (212) 6 / 33

7 7 / 33 Sensitivity of the limit to patience modeling Patience hazard rate Mandelbaum Avishai and Sergey Zeltyn. (213) Data-stories about (im)patient customers in tele-queues. Queueing Systems 75(2), With ρ() = 1 β (critical load): E[W ] = O(1/ )

8 8 / 33 Sensitivity of the limit to patience modeling Consider the critically loaded M/M/1 + GI: ρ() = 1 β Finite patience drawn from a distribution F a ( ). Limit Theorems differ by model F a F does not scale with and has f a () > : diffusion limit has linear drift; Ward and Glynn (25); F a has hazard rate that scales with : F a (x) = 1 e x h( u)du, limit has non-linear drift; Reed and Ward (28).

9 9 / 33 Sensitivity of scaling to patience modeling Suppose ρ() = 1 (critical loading). F a =exponential (fixed): E[W ] = Θ ( ) 1, as. F a (x) = x 2 for x [, 1] (fixed): ( ) 1 E[W ] = Θ 1/3, as. Different patience dist. different scaling needed for limits. A result that bypasses case-by-case analysis and interpretation..

10 The M/GI/1 + GI queue Service, 1 Load Patience The virtual wait V (t) is the time an infinitely patient customer, arriving at time t, would have to wait. The waiting time is the minimum of the virtual wait and the customer s patience: W (t) = min(ν, V (t)). The first order ( fluid ) proxy for the stationary virtual wait is w that solves µ = F( w). 1 / 33

11 11 / 33 Dynamics of the virtual waiting time V (t) is the work contained in jobs that will not abandon. A(t) V (t) = V () + s i 1 {vi >ω i } (t I(t)). i=1 Satisfies the natural positivity properties, - V (t), t ; - I( ) is nondecreasing with I() = ; - 1 {V (s)>} di(s) =. Process limits for the GI/GI/1+GI queue: Ward and Glynn (23,25), Reed and Ward (28), Jennings and Reed (212)

12 The intuitive Brownian queue A(t) V (t) = V () + s i 1 {vi >ω i } (t I(t)) i=1 t = V () + + I(t) A(t) ρ F a (V (s))ds t + i=1 t s i 1 {vi >ω i } ρ F a (V (s))ds t V (t) = V () + ρ F a ( V (s))ds t + σb(t) + (µ Î(t), σ = )E[s1 2] ( x (dx) = G exp 2 π V ) ρ F a (u) 1 σ 2 du dx, x [, ). No scaling. The recommendation is to use π V as a proxy for π V. 12 / 33

13 A notion of approximation accuracy For the M/M/1 queue V = W and Ŵ (t) is a one dimensional RBM. If ρ < 1, Ŵ := Ŵ ( ) is expo(mean = ρ/(µ(1 ρ))) M/M/1 : E[W k ] = k!ρ (µ(1 ρ)) k, Brownian Q : E[Ŵ k ] = k!ρ k (µ(1 ρ)) k. The approximation gap for the k th moment is E[W k ] E[Ŵ k ] = ρk! (µ(1 ρ)) k (1 ρk 1 ) = k(1 ρk 1 ) ρ k 3 (1 ρ) E[ V k 1 ] k(k 1) ρ k 3 E[Ŵ k 1 ]. For k = 1 the gap is (the P-K formula for the M/GI/1 queue). 13 / 33

14 A notion of approximation accuracy For the M/GI/1 + GI queue, it universally holds E[W k ] E[Ŵ k ] C E[Ŵ k 1 ] Waiting Time Scaled Error Waiting Time Scaled Error Figure: Hyper Exponential patience: F a (x) = 4 7 (1 e 4x ) (1 e x/2 ). M/M/1+GI moments using Zeltyn and Mandelbaum (25). 14 / 33

15 15 / 33 Queue families The M/GI/1+GI queue primitives are p = (Arrival rate, Service time dist. F s, Patience dist. F a ) We will define Q-families parameterized by a constant H. and prove results of the form E[Wp k ] E[Ŵ p k ] sup C H p Q(H) E[Ŵ p k 1 ] Universality = size the family Q(H) Recall: scaling is sensitive to patience dist. and other primitives.

16 Queue family Q(H) = {p = (, F s, F a )} (i) service-time moments: E [ ( )] s 1 exp δ H E[s 1 ] H, and there exists a concentration constant c p (µh) 1 such that (ii) finite load: ρ [H 1, H], ρ 1 H. c p (iii) polynomial growth: F a is differentiable with density f a : (iv) concentration: and f a (y) H c 2 p ( 1 + y w p c p H), ρ F a (y) 1 H 1 1 c p, for all y w p + c p H, ρ F a (y) 1 H 1 1 c p, for all y w p c p H, Notice: c p, w p vary with the primitives. 16 / 33

17 Indeed a large family exp(θ) patience (ρ > 1 : ρe θ w = 1): ( ρ F a w + H ) 1 H 1 1 c p ( ) ρe θ w+ H 1 H 1 1 F a c p H ρ 1 ρ > 1 Infinite 1 (1 ρ) exp(θ) 1 (1 ρ) 1 1 Uniform[, α] 1 (1 ρ) 1 1 HyperExp(θ, θ) 1 (1 ρ) 1 1 Power(α, k) 1 (1 ρ) 1 k+1 k (ρ 1) 1 k 1 Erlang(k, θ) 1 (1 ρ) 1 k+1 k (ρ 1) 1 k 1 Beta(α, β) 1 (1 ρ) 1 α+1 α /ρ (ρ 1) 1 1 α max(θ, 2/θ, ρ, 1/ρ) max(1/α, α/ρ, ρ, 1/ρ) max(θ, 2/θ, ρ, 1/ρ) 1 α 1 ρ 2k (k ρ) (1 α) k 2 k+1 (ρ k)h E Γ(k) max(θ k, 1 ρ 1 θ k ) U 2 α+β (ρ α)γ(α+β) (ρ 1)Γ(α)Γ(β) min(l,1) U Table: c p and H for a family of patience distributions. All these are in one queue family. 17 / 33

18 18 / 33 The accuracy of the Brownian approximation Theorem (Virtual waiting time) Given H > and k N, there exists a constant C 1 H,k > such that E[(V w) k ] E[( V w) k ] = ± C1 H,k E[ V w k 1 ], p Q(H). Corollary (Waiting time) Given H and k N, there exists a constant C 2 H,k > such that E[W k ] E[Ŵ k ] = ± C2 H,k E[Ŵ k 1 ], p Q(H). For k = 1, the error is O(1/).

19 19 / 33 The accuracy of the Brownian approximation Corollary (Queue length) Given H, there exists a constant CH,1 2 > such that E[Q] = E[W ] = E[Ŵ ] ± C2 H,1, p Q(H). The mean-queue approximation gap is a constant. Corollary (Abandonment) Given H, there exists a constant C H > such that Ab = E[F a ( V )] ± C H 2 E[ V w 2 ], p Q(H). For example, if F a = exp(θ), the Ab approximation gap is O(1/).

20 2 / 33 About the tightness of the Q-family conditions Virtual Waiting Time Scaled Error M/D/1 + GI with ρ = 1 ( w = ) and F a = Gamma(.5, 2). F a = Gamma(.5, 2) violates our conditions.

21 21 / 33 c p captures concentration/scaling Lemma (Concentration bounds) There exist constants CH,k V, cv H,k > such that c V H,k E[ V w k ] c p k C V H,k, p Q(H). There exist C H,k, C 1 H,k > such that for all p Q(H), E[(V w) k ] E[( V w) k ] = ± C1 H,k E[ V w k 1 ] = ± C H,k ck 1 p.

22 c p captures concentration/scaling Lemma (Concentration bounds) There exist constants CH,k V, cv H,k > such that c V H,k E[ V w k ] c p k C V H,k, p Q(H). There exist C H,k, C 1 H,k > such that for all p Q(H), E[(V w) k ] E[( V w) k ] = ± C1 H,k E[ V w k 1 ] = ± C H,k ck 1 p. Example: F a = exp(θ) c p = 1 and E[(V w) k ] E[( V w) k ] = ± C H,k ( ) 1. k 1 21 / 33

23 22 / 33 Underlying Math: Generator comparisons (B&D) g k (x) = (x w)k E[( V w) k ] (E[ V w ]) k so that E[g k ( V )] = Solve (for Ψ) (ÂΨ)(x) = g k(x) (Brownian Poisson Eqn) E[(A Ψ)(V ) g k (V )] = E[(A Ψ)(V ) (Â Ψ)(V )] + E[(Â Ψ)(V ) g k (V )] = E[(A Ψ)(V ) (Â Ψ)(V )] If E[A Ψ(V )] = (Glynn and Zeevi (28)), then E[g k (V )] = E[(V w)k E[( V w) k ]] (E[ V w ]) k = E[(A Ψ)(V ) (Â Ψ)(V )]

24 Generator comparison and gradient bounds E[(V w) k ] E[( V w) k ] (E[ V w ]) k E[(A Ψ)(V ) (Â Ψ)(V )] A Ψ(x) = Ψ (1) (x) + F a (x) E [ Ψ(x + s1 ) Ψ(x) ] [ = Ψ (1) (x) + F a (x) E Ψ (1) (x)s ] 2 Ψ (2) (x)s1 2 + ɛ(x, s 1) = Â Ψ(x) + F a (x) E [ɛ(x, s 1 )]. E[ɛ(x, s 1 )] has Ψ s derivative of order m > 2. E[ A Ψ(V ) Â Ψ(V ) ] E[ F a (V ) E [ɛ(v, s 1 )]] show C H E[ V w ]. Where does V on the right-hand side come from? Show = Gradient + Apriori moment bounds (via c p drift cond.) See Braverman and Dai (216) 23 / 33

25 From performance analysis to optimization 24 / 33

26 Two ways in which regimes arise Consider a sequence of queues with ρ() = 1 β Identifying the optimal regime: Minimizing capacity + linear delay cost in the M/M/1 queue µ () := min µ c s µ + c w E[W (µ)] = + so that (1 ρ ()) = 1 + ρ cw 1 () 1 c s c w c s cw c s c w c s, as If c w = 1 4 and c s = 1, then, ρ = 1/2. 25 / 33

27 26 / 33 Dynamic optimization: Service-rate control in the M/G/1 queue Steps: Arrival rate ; Service time distribution F s with E[s 1 ] = 1. Controlled service rate µ(θ) = (1 + θ). Holding cost hx m. p = (, h). [ Jp,m V, 1 t ( = inf lim θ Θ V t t E x h(v (θ, s)) m + (θ(s)) 2) ] ds. An unscaled Brownian Control Problem (BCP) Universality over Q(H) = {(, h) : H 1, h (, H)} We will be agnostic to whether (or not) h scales down with

28 27 / 33 The Brownian control problem t t V (θ, t) =V () θ(s)ds + (1 + θ(s))1{v (θ, s) = }ds A(t) + s i t i=1 t V (θ, t) =V () θ(s)ds + + E[s1 2]B(t). t (1 + θ(s))1{ V (θ, s) = }ds [ J V, 1 t ( p,m = inf lim t t E x h( V (θ, s)) m + (θ(s)) 2) ] ds θ Θ V (BCP)

29 Universal optimality gap Theorem An optimal stationary (Brownian) policy θ p,m(x) exists and, for any p Q(H) := {(, h) : H 1, h (, H]}, J V, p,m J V p,m(θ p,m) B H (, m)j V, p,m 1 B H (, m) as. The gap is if m = / 33

30 Universal optimality gap Theorem An optimal stationary (Brownian) policy θ p,m(x) exists and, for any p Q(H) := {(, h) : H 1, h (, H]}, J V, p,m J V p,m(θ p,m) B H (, m)j V, p,m 1 B H (, m) as. The gap is if m = 2. Recall, we found for the (uncontrolled) M/GI/1 + GI queue: E[(V w) k ] E[( V w) k ] = ± C1 H,k E[ V w k 1 ] 28 / 33

31 For control, too, generator comparisons { γ = min z (Âz Ψ)(x) + (z) 2 + hx m}, (Diffusion HJB) Ψ() = Ψ (1) () = and Ψ (1) (x), for all x, Given ( Ψ, γ) : optimal service rate ẑ (x) = Ψ (1) (x) 2 γ = min z {(Az Ψ)(x) + (z)2 + hx m } (M/GI/1 Bellman) (Doshi (1978)) If for relevant values of z, A z Ψ Âz Ψ: min z {(Az Ψ)(x) + z + hx m } min z (Âz Ψ)(x) + z + hx m } Ψ, γ p,m almost solves the M/G/1 Bellman equation. 29 / 33

32 3 / 33 From BCP to M/GI/1 optimality Lemma Fix (p, m) and let ( Ψ, γ) be the solution the (BCPs) HJB equation. Then, for any admissible control θ for the M/G/1 queue (and any x, t ): E x [ t ( h(v (θ, s)) m + (θ(s)) 2) ] ds Ψ(x) ] E x [ Ψ(V (θ, t)) + γt [ t ( where A x (θ, t) = E x A θ(s) + A x (θ, t), ) ] Ψ(V (θ, s)) Âθ(s) Ψ(V (θ, s)) ds. If θ is the BCP stationary control ẑ, the inequality is replaced with equality. If m = 2, A x (θ, t) for any control θ.

33 31 / 33 From BCP to M/GI/1 optimality Lemma Fix m and let ( Ψ p,m, γ p,m ) be the (family of) solutions to the HJB equation. Then, there exist constants CH,m 1, C2 H,m such that, for any order optimal family of policies {θ p,m, p Q(H)}, lim inf t 1 t Ax p,m(θ p,m, t) C 1 H,m BH (, m)j Y, p,m 1, x, and under the stationary policy θ p,m, lim sup t 1 t Ax p,m(θ p,m, t) C 2 H,m BH (, m)j Y, p,m 1, x.

34 32 / 33 Conclusion In great generality, it is fine to use the intuitive Brownian queue of the M/GI/1+GI. It is universally accurate in regime and patience-scaling. Similar ideas are applied to static and dynamic optimization Underlying math: Avoid scaling through Q-families. From the universal proximity of operators to the universal proximity of equation solutions (Poisson or HJB).

35 33 / 33 Time Dependent Expectations First moment Second moment Scaled gap Time Time Figure: Time-dependent performance for M/M/1 with µ = 1: (LHS) ρ =.9, (RHS) Scaled gap