On the Resource/Performance Tradeoff in Large Scale Queueing Systems

Transcription

1 On the Resource/Performance Tradeoff in Large Scale Queueing Systems David Gamarnik MIT Joint work with Patrick Eschenfeldt, John Tsitsiklis and Martin Zubeldia (MIT)

2 High level comments

3 High level comments Many modern queueing systems are large scale

4 High level comments Many modern queueing systems are large scale Operating optimally requires large scale resources

5 High level comments Many modern queueing systems are large scale Operating optimally requires large scale resources It is of interest to understand the best performance under limited resources availability

6 High level comments Many modern queueing systems are large scale Operating optimally requires large scale resources It is of interest to understand the best performance under limited resources availability In this work we study Join-the-Shortest-Queue (JSQ) policy in heavy traffic and compare it with M/M/N design Dispatching policies with limited memory and limited information exchange in many server queueing systems

7 Join the Shortest Queue in heavy traffic n n parallel servers Exp(1) service Pois(nλ n ) arrival. λ n = 1 β/ n. Choose any shortest queue upon arrival Compare with M/M/N - global buffer, join the smallest workload

8 Join the Shortest Queue in heavy traffic n n parallel servers Exp(1) service Pois(nλ n ) arrival. λ n = 1 β/ n. Choose any shortest queue upon arrival Compare with M/M/N - global buffer, join the smallest workload

9 Join the Shortest Queue in heavy traffic n n parallel servers Exp(1) service Pois(nλ n ) arrival. λ n = 1 β/ n. Choose any shortest queue upon arrival Compare with M/M/N - global buffer, join the smallest workload

10 Prior work on JSQ: fixed number of servers Winston 77 : JSQ is the optimal policy if customers are routed to servers immedaitely. Foschini and Salz 78 : diffusion limit for heavy traffic with fixed number of servers. Mukherjee, Borst, van Leeuwaarden and Whiting 15: a combination of JSQ with a Supermarket Model.

11 Notation Q n i (t) is the number of servers with queue length at least i, including those in service n Q n 1 (t) Qn 2 (t) 0 Q i (t) Q i+1 (t) is the number of servers with exactly i customers n Q n 1 (t) is the number of idle servers X n 1 (t) = (Qn 1 (t) n)/ n, X n i (t) = Q n i (t)/ n

12 Main result described qualitatively

13 Main result described qualitatively: queues

14 Main result described qualitatively: queues

15 Main result described qualitatively: queues O ( n ) idle servers and O ( n ) servers with exactly one customer waiting

16 Main result described qualitatively: queues O ( n ) idle servers and O ( n ) servers with exactly one customer waiting Upon rescaling, they form a 2-dimensional reflected Ornstein-Uhlenbeck process

17 Main result described qualitatively: queues O ( n ) idle servers and O ( n ) servers with exactly one customer waiting Upon rescaling, they form a 2-dimensional reflected Ornstein-Uhlenbeck process Longer queues disappear in constant time

18 Main result described qualitatively: waiting times

19 Main result described qualitatively: waiting times Two possibilities for arriving customer:

20 Main result described qualitatively: waiting times Two possibilities for arriving customer: At least one idle server, so zero wait No idle servers, join queue behind one customer, so wait Exp(1)

21 Main result described qualitatively: waiting times Two possibilities for arriving customer: At least one idle server, so zero wait No idle servers, join queue behind one customer, so wait Exp(1) Aggregate waiting time for customers arriving in [0, t] is O ( n )

22 Main result described qualitatively: waiting times Two possibilities for arriving customer: At least one idle server, so zero wait No idle servers, join queue behind one customer, so wait Exp(1) Aggregate waiting time for customers arriving in [0, t] is O ( n ) Order n arrivals in [0, t]

23 Main result described qualitatively: waiting times Two possibilities for arriving customer: At least one idle server, so zero wait No idle servers, join queue behind one customer, so wait Exp(1) Aggregate waiting time for customers arriving in [0, t] is O ( n ) Order n arrivals in [0, t] Fraction of customers who wait: O ( 1/ n )

24 Main result described qualitatively: waiting times Two possibilities for arriving customer: At least one idle server, so zero wait No idle servers, join queue behind one customer, so wait Exp(1) Aggregate waiting time for customers arriving in [0, t] is O ( n ) Order n arrivals in [0, t] Fraction of customers who wait: O ( 1/ n ) Average waiting time O ( 1/ n ) - same as for M/M/N.

25 JSQ as a reflected process Fix k 3, b R k, y D k. Theorem There exists a unique solution x(t) to the integral equation x(t) = b + y(t) + (x(t))dt + U(t) 0 1{x(t) }du(t) = 0, This is a variation on a result of Pang, Talreja, and Whitt 07.

26 An integral equation For k 3, B R +, b R k, y D k, there is a unique solution (x, u) to x 1 (t) = b 1 + y 1 (t) + x 2 (t) = b 2 + y 2 (t) + x i (t) = b i + y i (t) + x k (t) = b k + y k (t) + t 0 t 0 t 0 t 0 ( x 1 (s) + x 2 (s))ds u 1 (t) ( x 2 (s) + x 3 (s))ds + u 1 (t) u 2 (t), ( x i (s) + x i+1 (s))ds, 3 i k 1, x k (s)ds, x 1 (t) 0, 0 x 2 (t) B, x i (t) 0, u 1 (t), u 2 (t) 0, t 0, 0 1{x 1 (t) < 0}du 1 (t) = 0, 0 1{x 2 (t) < B}du 2 (t) = 0.

27 JSQ heavy traffic limit Theorem (Main Result) Suppose X n (0) X(0) with X n k+1 (0) = 0. Then X n X where X 1 0, X i 0, i 2, and nondecreasing U 1 0 such that X 1 (t) = X 1 (0) + t 2W (t) βt + X 2 (t) = X 2 (0) + U 1 (t) + X i (t) = X i (0) + X k (t) = X k (0) + 0 = 0 t 0 t 0 t 0 0 ( X 1 (s) + X 2 (s)) ds U 1 (t), ( X 2 (s) + X 3 (s))ds, ( X i (s) + X i+1 (s))ds, 3 i k 1, X k (s)ds, X i (t) = 0, i k + 1, 1{X 1 (t) < 0}dU 1 (t), where W is a standard Brownian motion.

28 Proof outline Introduce truncated approximation of system. show that the truncated system converges to the Ornstein-Uhlenbeck process. Show the original and truncated systems have same behavior whp.

29 Truncated model n n 1 Initially no queue longer than k. Reject any arrival when ˆQ 2 n (t) = n. ˆQ i n (t), i 3 decreases monotonically in t. n

30 Truncated model n n 1 Initially no queue longer than k. Reject any arrival when ˆQ 2 n (t) = n. ˆQ i n (t), i 3 decreases monotonically in t. n

31 Connecting truncated and untruncated Since Q2 n (0) < n, truncated system and full system are identical until the first time ˆQ 2 n (t) = n. The weak convergence of the truncated system ˆX n X implies ( ) P sup ˆQ 2 n (s) n 0. 0 s t This further implies X n X.

32 Open questions Waiting time distribution for a customer arriving at time t Steady state of the limiting system Convergence of steady state in n-th system to steady state of limiting system (interchange of limits) General service times distribution

33 Dispatching with limited memory and information exchange Resource Constrained Pull Based (RCPB) policy Dispatcher 15,3,28,6,87 15 n parallel servers. Exp(1) service. Pois(λn) arrival. λ < 1. Dispatcher can store up to C IDs of idle servers. Idle servers send reminders at rate µ. Job is assigned to an idle server, if at least one idle server ID is available. Otherwise u.a.r.

34 Some relevant literature Badonnel and Burgess 08: Pull-based load balancing Stolyar 15: Pull-based load distribution in heterogeneous systems Literature on Supermarket Model

35 Main results: positive S N (t) = (Q i (t)/n, i 1), Q i (t) - number of servers with length i. 0 M(t) C - number of tokens at Dispatcher.

36 Main results: positive S N (t) = (Q i (t)/n, i 1), Q i (t) - number of servers with length i. 0 M(t) C - number of tokens at Dispatcher. Theorem

37 Main results: positive S N (t) = (Q i (t)/n, i 1), Q i (t) - number of servers with length i. 0 M(t) C - number of tokens at Dispatcher. Theorem ODE (Fluid model limit) S N (t) N s(t).

38 Main results: positive S N (t) = (Q i (t)/n, i 1), Q i (t) - number of servers with length i. 0 M(t) C - number of tokens at Dispatcher. Theorem ODE (Fluid model limit) S N (t) N s(t). s(t) s.

39 Main results: positive S N (t) = (Q i (t)/n, i 1), Q i (t) - number of servers with length i. 0 M(t) C - number of tokens at Dispatcher. Theorem ODE (Fluid model limit) S N (t) N s(t). s(t) s. Interchange of limits: S N (t) t π N N s.

40 Description of the equilibrium Theorem The equilibrium is given by P0 = 0 k C ( ) µ(1 λ) k λ s i = λ (λp 0 )i 1, i 1 E[Delay] = λp 0 1 λp0. 1,

41 Uniformly bounded delay in λ 1

42 Uniformly bounded delay in λ 1 Note: as λ 1, the effective rate of messages decreases (1 λ)µ.

43 Uniformly bounded delay in λ 1 Note: as λ 1, the effective rate of messages decreases (1 λ)µ. Given a budget ν (1 λ)µ and memory size C, how does the delay scale as λ 1?

44 Uniformly bounded delay in λ 1 Note: as λ 1, the effective rate of messages decreases (1 λ)µ. Given a budget ν (1 λ)µ and memory size C, how does the delay scale as λ 1? Theorem The delay is uniformly bounded in λ: sup E[Delay] λ<1 1 k C ν k 1.

45 Uniformly bounded delay in λ 1 Note: as λ 1, the effective rate of messages decreases (1 λ)µ. Given a budget ν (1 λ)µ and memory size C, how does the delay scale as λ 1? Theorem The delay is uniformly bounded in λ: sup E[Delay] λ<1 1 k C Note: For the supermarket model E[Delay] 1 ( ) 1 log d log. 1 λ ν k 1.

46 Lower bound on delays for general policies queries messages Dispatcher Dispatcher memory capacity C log n. Dispatcher queries some servers upon arrivals. Servers send messages to Dispatcher. Memory state is updated at events.

47 Lower bound on delays for general policies Theorem Every symmetric dispatching policy induces a delay bounded away from zero: for every λ < 1 lim inf E[Delay π π ] > 0.

48 Thank you.