Designing load balancing and admission control policies: lessons from NDS regime

Size: px

Start display at page:

Download "Designing load balancing and admission control policies: lessons from NDS regime"

Laureen Henry
5 years ago
Views:

1 Designing load balancing and admission control policies: lessons from NDS regime VARUN GUPTA University of Chicago Based on works with : Neil Walton, Jiheng Zhang

2 ρ K θ is a useful regime to study the interplay of service distributions and control policies 1. Load balancing with PS servers 2. Concurrency control in a statedependent PS server 1 Poisson(λ) Load Balancer 2 FCFS buffer μ(n) K Concurrency Control

3 GOAL: Min. mean sojourn time E[T] Poisson(λ) Load Balancer K Processor Sharing servers Knows queue lengths Not job sizes Q: Optimal load balancing policy? A: Exponential service Shortest-Queue General service??

4 10 E[T] Det Exp Bim-1 Weib-1 Weib-2 Bim-2 SERVICE DISTRIBUTIONS (INC. VARIANCE ) RANDOM ROUND-ROBIN LEAST-WORK S. QUEUE OPT-0 Q: Is Shortest Queue insensitive and near-optimal? GOAL: a meaningful asymptotic regime

5 Option 1 Light traffic: K = fixed, λ 0 # Servers: K Arr. Rate: λ Mean service time: E S = 1 Load: ρ λ K Can not infer insensitivity! E T 1 Option 2 Conventional heavy traffic: K = fixed, λ K System collapses to a single PS server. Even Least-Work becomes insensitive.

6 Option 3 Halfin-Whitt: K, K λ~β K # Servers: K Arr. Rate: λ Mean service time: E S = 1 Load: ρ λ K Can not infer insensitivity! In fact, for K E T 1 K λ ~ ω(1) E T 1 K λ ~ o(1) Collapses to PS server

7 Option 4 K, K λ = α # Servers: K Arr. Rate: λ Mean service time: E S = 1 Load: ρ λ K Equivalently, ρ K θ e α Under the above setting, 1 < E[T] E[S] < Non-degenerate slowdown (NDS) regime

8 1. Shortest Queue insensitive under NDS? 2. Shortest Queue near-optimal under NDS?

9 Poisson(λ) K K K λ = α Analysis technique: Process for total jobs in system λ λ λ λ λ λ 0 1 K 3K/2 2K 2K+1 2K+2? = mean departure rate given 3K/2 jobs

10 K/2 N = 3K/2 Poisson(λ) Rate = K/2 K/2 Rate = K Departure rate = K 1 (not K) Key: Find the O(1) fluctuations O(1) idle queues

11 γk N = (1+γ)K (0<γ<1) Poisson(λ) (1- γ)k Departure rate = K (1-γ)/ γ Rate = (1-γ)K Rate = K O(1) idle queues

12 Poisson(λ) K K K λ = α Analysis technique: Process for total jobs in system λ λ λ λ λ λ 0 1 K (1+ )K 2K 2K+1 2K+2 Asymptotically negligible probability mass K- (1- )/ K K

13 More formally: N K Kt N(t) K where N(t) is a Brownian motion with drift: μ n = α + (2 n)+! n 1 variance: σ 2 = 2. COROLLARY Shortest-Queue: E N SQ = g(α) Central-Queue (M/M/K): E N CQ = α sup α E[N SQ ] E[N CQ ] 1.14

14 E[N SQ ] E[N CQ ] 1 K=64 K=16 K=4 Type equation here.

15 More formally: N K Kt N(t) K where N(t) is a Brownian motion with drift: μ n = α + (2 n)+ n 1 variance: σ 2 = 2. COROLLARY Shortest-Queue: E N SQ = g(α)! IQF up to factor 2 worse than Central Queue Central-Queue (M/M/K): E N CQ = α Idle-Queue-First: E N IQF = α

16 1. Shortest Queue insensitive under NDS? 2. Shortest Queue near-optimal under NDS?

17 Proposition: For Phase-type service distributions with SQ routing, lim K Intuitively: N (K) K is insensitive beyond E S. 1. Time-scale separation: N K (t) evolves slowly 2. Closed-system (fluid limit) under SQ is insensitive Proposition: For H 2 service distribution with Expected-Least-Work routing, lim N (K) K is sensitive to distribution. K

18 Conjecture: 1. For any non-anticipative (but possibly size-aware) load balancer, lim K E N Π (K) K E N CQ. 2. For OPT-0 (myopic size-aware) lim K (K) E N OPT 0 K = E N CQ. Conjecture + Propositions SQ is within 14% of the optimal size-aware policy in NDS regime!

19 ρ K θ is a useful regime to study the interplay of service distributions and control policies 1. Load balancing with PS servers 2. Concurrency control in a statedependent PS server 1 Poisson(λ) Load Balancer 2 FCFS buffer μ(n) K Concurrency Control

20 State-dependent Processor Sharing μ(n) μ(n) n jobs at server service rate μ(n)/n per job

21 FCFS buffer Sharing Limit Control (L) State-dependent Processor Sharing μ(n) μ(n) Motivation Congestion based friction e.g., server thread-pool management, traffic flow Service systems with human agents

22 FCFS buffer Sharing Limit Control (L) State-dependent Processor Sharing μ(n) μ(n) L* = 1 Straw-man: Set to maximum efficiency point (L*) High efficiency but FCFS dominates Trade-off between arrival rate and service variability

23 FCFS buffer Sharing Limit Control (L) State-dependent Processor Sharing μ(n) μ(n) L* = 1 GOAL: Diffusion approximation for static L, general service distribution Q: What is a meaningful sequence of system that faithfully approximates the original system?

24 1.3 μ(n) λ L Option 1 Conventional heavy traffic: λ (r) μ L Does not capture original system!

25 1.3 μ(n) λ L Option 2 Scale sharing limit: L (r) = Lr Stretch service rate curve: μ r (rx) μ x where μ x is a continuous extension of μ n In this example, system gets stuck at 0.

26 Proposal Feed the original system M/M/ input Let H n = Prob N n and H a continuous differentiable interpolation 1 H(n) H x Scaling: In the r th system L (r) = Lr Devise μ r rx so that under M/M/ input: H r (rx) H x L

27 Scaling: In the r th system L (r) = Lr Under M/M/ input: H r (rx) H x We guarantee a limit that depends on the entire μ n Reverse engineered service rates: H(n) L H x! lim r λ μ r rx = λ r ρ r θ Entire μ r curve collapses to λ at rate 1/r d log( h x ) dx α(x) μ(n) drift function λ

28 E N n=0 (n L) π(n) n=0 π(n) c 2 s +1 c 2 s +c2 a c 2 s +1 c 2 s +c2 a + c s n=0 (n L) + π(n) n=0 π(n) c 2 s +1 c 2 s +c2 a c 2 s +1 c 2 s +c2 a where π(n) is probability mass function for the original system under M/M/ input 12 E[N] Weibull Pareto (1.1) LogNormal Approx Sharing Limit (L)

29 1. ρ K θ regime useful for: qualitative insights into load balancing policies for finite systems designing service variability based control policies for finite systems 2. Shortest Queue Near-optimal ( 14%) without knowing job sizes Idle-Queue-First is a poor proxy for SQ but going until queue length 1 gains all benefits.

State-dependent Limited Processor Sharing - Approximations and Optimal Control

State-dependent Limited Processor Sharing - Approximations and Optimal Control Varun Gupta University of Chicago Joint work with: Jiheng Zhang (HKUST) State-dependent Limited Processor Sharing Processor