Control of multi-class queueing networks with infinite virtual queues

Transcription

1 Control of multi-class queueing networks with infinite virtual queues Erjen Lefeber (TU/e) Workshop on Optimization, Scheduling and Queues Honoring Gideon Weiss on his Retirement June 8, 2012 Where innovation starts

2 Multi-class queueing network with IVQs 2/22 i = 1 i = 2 i = i = i =

3 Example: Push pull queueing system Kopzon, Weiss (2002); Kopzon, Nazarathy, Weiss (2009); Nazarathy, Weiss (2010) 3/22 Static production planning problem α 1, α 2 nominal input rates u i max u,α w α fraction of time spent on class i

4 Example: Push pull queueing system Kopzon, Weiss (2002); Kopzon, Nazarathy, Weiss (2009); Nazarathy, Weiss (2010) 3/22 Static production planning problem α 1, α 2 nominal input rates u i max u,α w α fraction of time spent on class i

5 Example: Push pull queueing system 4/22 max w 1α 1 + w 2 α 2 u,α λ u 1 α 1 s.t. λ 1 µ u λ 2 0 u 3 = 0 α λ 2 µ 2 u 4 0 [ ] u 1 [ ] u u 3 1 u 4 u, α 0

6 Example: Push pull queueing system 5/22 Three possible solutions (exclusing singular cases λ 1 = µ 1 or λ 2 = µ 2 ): 1. α 1 = min{λ 1, µ 1 }, α 2 = 0, 2. α 1 = 0, α 2 = min{λ 2, µ 2 }, 3. α 1 = λ 1µ 1 (λ 2 µ 2 ) λ 1 λ 2 µ 1 µ 2, α 2 = λ 2µ 2 (λ 1 µ 1 ) λ 1 λ 2 µ 1 µ 2. Interesting solution: solution 3 ρ1 = ρ 2 = 1 (full utilization of servers) ρ1 = λ 2(λ 1 µ 1 ) λ 1 λ 2 µ 1 µ 2 < 1, ρ 2 = λ 1(λ 2 µ 2 ) λ 1 λ 2 µ 1 µ 2 < 1.

7 Example: Push pull queueing system 5/22 Three possible solutions (exclusing singular cases λ 1 = µ 1 or λ 2 = µ 2 ): 1. α 1 = min{λ 1, µ 1 }, α 2 = 0, 2. α 1 = 0, α 2 = min{λ 2, µ 2 }, 3. α 1 = λ 1µ 1 (λ 2 µ 2 ) λ 1 λ 2 µ 1 µ 2, α 2 = λ 2µ 2 (λ 1 µ 1 ) λ 1 λ 2 µ 1 µ 2. Interesting solution: solution 3 ρ1 = ρ 2 = 1 (full utilization of servers) ρ1 = λ 2(λ 1 µ 1 ) λ 1 λ 2 µ 1 µ 2 < 1, ρ 2 = λ 1(λ 2 µ 2 ) λ 1 λ 2 µ 1 µ 2 < 1.

8 Example: Push pull queueing system 6/22 Question Can we stabilize system with ρ i = 1 and ρ i < 1? Two cases inherently stable case: λ 1 < µ 1 and λ 2 < µ 2 inherently unstable case: λ 1 > µ 1 and λ 2 > µ 2

9 Example: Push pull queueing system 6/22 Question Can we stabilize system with ρ i = 1 and ρ i < 1? Two cases inherently stable case: λ 1 < µ 1 and λ 2 < µ 2 inherently unstable case: λ 1 > µ 1 and λ 2 > µ 2

10 Inherently stable case: λ 1 > µ 1, λ 2 > µ 2 7/22 Positive result Pull priority stabilizes network Observation For inherently unstable case: pull priority is not stabilizing.

11 Inherently stable case: λ 1 > µ 1, λ 2 > µ 2 7/22 Positive result Pull priority stabilizes network Observation For inherently unstable case: pull priority is not stabilizing.

12 Inherently unstable case: λ 1 < µ 1, λ 2 < µ 2 Kopzon, Nazaraty, Weiss (2009); Nazarathy, Weiss (2010): Positive result Threshold policy stabilizes network 8/22

13 Problem Guo, Lefeber, Nazarathy, Weiss, Zhang (2011): 9/22 Key research question Can we stabilize a MCQN-IVQ with ρ i < 1 for all servers? Some positive results IVQ re-entrant line (LBFS stable; FBFS not necessarily) Two re-entrant lines on two servers (pull priority) Ring of machines (pull priority) Fluid model framework for verifying stability

14 Problem Guo, Lefeber, Nazarathy, Weiss, Zhang (2011): 9/22 Key research question Can we stabilize a MCQN-IVQ with ρ i < 1 for all servers? Some positive results IVQ re-entrant line (LBFS stable; FBFS not necessarily) Two re-entrant lines on two servers (pull priority) Ring of machines (pull priority) Fluid model framework for verifying stability

15 Problem setting Consider a MCQN-IVQ with s servers. Server i serves 1 IVQ and n i 1 standard queues. Assumption: ρ = 1. Let n = s i=1 n i ; s n constituency matrix C, rank C = s n. IVQ at server i served at rate λ i, Standard queue j served at rate µ j > 0. Let Λ = diag(λ 1,..., λ s ), M = diag(µ 1,..., µ n ) n n Routing matrix P: p jk fraction of class j routed to k. Assumption: P has spectral radius < 1, i.e. (I P ) invertible. s n matrix P IVQ. p IVQ ij fraction of IVQ at server i routed to j Dynamics fluid model (u j (t) fraction of time spent on std. queue j) subject to Q(t) = P IVQΛ[1 Cu(t)] (I P )Mu(t) Q(0) = Q 0 0 Q(t) 0 u(t) Cu(t) 1 10/22

16 Problem setting Consider a MCQN-IVQ with s servers. Server i serves 1 IVQ and n i 1 standard queues. Assumption: ρ = 1. Let n = s i=1 n i ; s n constituency matrix C, rank C = s n. IVQ at server i served at rate λ i, Standard queue j served at rate µ j > 0. Let Λ = diag(λ 1,..., λ s ), M = diag(µ 1,..., µ n ) n n Routing matrix P: p jk fraction of class j routed to k. Assumption: P has spectral radius < 1, i.e. (I P ) invertible. s n matrix P IVQ. p IVQ ij fraction of IVQ at server i routed to j Dynamics fluid model (u j (t) fraction of time spent on std. queue j) subject to Q(t) = P IVQΛ[1 Cu(t)] (I P )Mu(t) Q(0) = Q 0 0 Q(t) 0 u(t) Cu(t) 1 10/22

17 Problem setting Consider a MCQN-IVQ with s servers. Server i serves 1 IVQ and n i 1 standard queues. Assumption: ρ = 1. Let n = s i=1 n i ; s n constituency matrix C, rank C = s n. IVQ at server i served at rate λ i, Standard queue j served at rate µ j > 0. Let Λ = diag(λ 1,..., λ s ), M = diag(µ 1,..., µ n ) n n Routing matrix P: p jk fraction of class j routed to k. Assumption: P has spectral radius < 1, i.e. (I P ) invertible. s n matrix P IVQ. p IVQ ij fraction of IVQ at server i routed to j Dynamics fluid model (u j (t) fraction of time spent on std. queue j) subject to Q(t) = P IVQΛ[1 Cu(t)] (I P )Mu(t) Q(0) = Q 0 0 Q(t) 0 u(t) Cu(t) 1 10/22

18 Problem setting Consider a MCQN-IVQ with s servers. Server i serves 1 IVQ and n i 1 standard queues. Assumption: ρ = 1. Let n = s i=1 n i ; s n constituency matrix C, rank C = s n. IVQ at server i served at rate λ i, Standard queue j served at rate µ j > 0. Let Λ = diag(λ 1,..., λ s ), M = diag(µ 1,..., µ n ) n n Routing matrix P: p jk fraction of class j routed to k. Assumption: P has spectral radius < 1, i.e. (I P ) invertible. s n matrix P IVQ. p IVQ ij fraction of IVQ at server i routed to j Dynamics fluid model (u j (t) fraction of time spent on std. queue j) subject to Q(t) = P IVQΛ[1 Cu(t)] (I P )Mu(t) Q(0) = Q 0 0 Q(t) 0 u(t) Cu(t) 1 10/22

19 Problem setting Consider a MCQN-IVQ with s servers. Server i serves 1 IVQ and n i 1 standard queues. Assumption: ρ = 1. Let n = s i=1 n i ; s n constituency matrix C, rank C = s n. IVQ at server i served at rate λ i, Standard queue j served at rate µ j > 0. Let Λ = diag(λ 1,..., λ s ), M = diag(µ 1,..., µ n ) n n Routing matrix P: p jk fraction of class j routed to k. Assumption: P has spectral radius < 1, i.e. (I P ) invertible. s n matrix P IVQ. p IVQ ij fraction of IVQ at server i routed to j Dynamics fluid model (u j (t) fraction of time spent on std. queue j) subject to Q(t) = P IVQΛ[1 Cu(t)] (I P )Mu(t) Q(0) = Q 0 0 Q(t) 0 u(t) Cu(t) 1 10/22

20 Problem setting Consider a MCQN-IVQ with s servers. Server i serves 1 IVQ and n i 1 standard queues. Assumption: ρ = 1. Let n = s i=1 n i ; s n constituency matrix C, rank C = s n. IVQ at server i served at rate λ i, Standard queue j served at rate µ j > 0. Let Λ = diag(λ 1,..., λ s ), M = diag(µ 1,..., µ n ) n n Routing matrix P: p jk fraction of class j routed to k. Assumption: P has spectral radius < 1, i.e. (I P ) invertible. s n matrix P IVQ. p IVQ ij fraction of IVQ at server i routed to j Dynamics fluid model (u j (t) fraction of time spent on std. queue j) subject to Q(t) = P IVQΛ[1 Cu(t)] (I P )Mu(t) Q(0) = Q 0 0 Q(t) 0 u(t) Cu(t) 1 10/22

21 Problem setting Dynamics fluid model 11/22 Q(t) = P IVQΛ[1 Cu(t)] (I P )Mu(t) Q(0) = Q 0 = P IVQΛ1 [P }{{} IVQΛC + (I P )M] u(t) }{{} α R subject to 0 Q(t) 0 u(t) Cu(t) 1 Additional assumptions Controllable system, i.e. R is invertable. ρ < 1, i.e. CR 1 α < 1. All standard queues are served: u = R 1 α > 0

22 Problem setting Dynamics fluid model 11/22 Q(t) = P IVQΛ[1 Cu(t)] (I P )Mu(t) Q(0) = Q 0 = P IVQΛ1 [P }{{} IVQΛC + (I P )M] u(t) }{{} α R subject to 0 Q(t) 0 u(t) Cu(t) 1 Additional assumptions Controllable system, i.e. R is invertable. ρ < 1, i.e. CR 1 α < 1. All standard queues are served: u = R 1 α > 0

23 Problem setting 12/22 To summarize: Q(t) = α Ru(t) Q(0) = Q 0 subject to 0 Q(t) 0 u(t) Cu(t) 1 Furthermore C full rank 0-1 matrix, i.e., CC is invertible I P and R are invertible (also (I P ) 1 0) 0 < R 1 α = u CR 1 α < 1 Problem: Determine stabilizing u

24 Example 13/22 Dynamics: [ ] Q 1 (t) = Q 2 (t) [ λ1 λ 2 ] [ ] [ ] µ1 λ 1 u1 λ 2 µ 2 u 2 C = [ ] Constraints Assumptions: 0 Q(t) 0 u(t) u(t) 1 R invertible: µ 1 µ 2 λ 1 λ 2 or ρ 1 ρ 2 1 CR 1 α < 1, R 1 1 ρ α = 0: 1 1 ρ 1 ρ 1 ρ 2 > 0, 2 1 ρ 1 ρ 2 > 0.

25 Example 13/22 Dynamics: [ ] Q 1 (t) = Q 2 (t) [ λ1 λ 2 ] [ ] [ ] µ1 λ 1 u1 λ 2 µ 2 u 2 C = [ ] Constraints Assumptions: 0 Q(t) 0 u(t) u(t) 1 R invertible: µ 1 µ 2 λ 1 λ 2 or ρ 1 ρ 2 1 CR 1 α < 1, R 1 1 ρ α = 0: 1 1 ρ 1 ρ 1 ρ 2 > 0, 2 1 ρ 1 ρ 2 > 0.

26 Example Conditions: ρ 1 ρ 2 1, 1 ρ 1 1 ρ 1 ρ 1 ρ 2 > 0, 2 1 ρ 1 ρ 2 > 0 14/22 ρ ρ 1

27 Example: uncontrollable case Some words about case λ 1 = µ 1, λ 2 = µ 2, i.e., R not invertible Uncontrollable dynamics [ ] [ ] [ ] [ ] Q 1 (t) λ1 λ1 λ = 1 u1 (t) Q 2 (t) λ 2 λ 2 λ 2 u 2 (t) 15/22 Define change of coordinates: z 1 (t) = Q 1 (t) + Q 2 (t) z 2 (t) = λ 2 Q 1 (t) λ 1 Q 2 (t) Then we have ] [ż1 (t) ż 2 (t) = [ ] λ1 + λ 2 0 [ λ1 + λ 2 λ 1 + λ ] [ u1 (t) u 2 (t) In particular the variable z 2 (t) evolves independent of the policy chosen. ]

28 Example: uncontrollable case Some words about case λ 1 = µ 1, λ 2 = µ 2, i.e., R not invertible Uncontrollable dynamics [ ] [ ] [ ] [ ] Q 1 (t) λ1 λ1 λ = 1 u1 (t) Q 2 (t) λ 2 λ 2 λ 2 u 2 (t) 15/22 Define change of coordinates: z 1 (t) = Q 1 (t) + Q 2 (t) z 2 (t) = λ 2 Q 1 (t) λ 1 Q 2 (t) Then we have ] [ż1 (t) ż 2 (t) = [ ] λ1 + λ 2 0 [ λ1 + λ 2 λ 1 + λ ] [ u1 (t) u 2 (t) In particular the variable z 2 (t) evolves independent of the policy chosen. ]

29 Example: uncontrollable case Some words about case λ 1 = µ 1, λ 2 = µ 2, i.e., R not invertible Uncontrollable dynamics [ ] [ ] [ ] [ ] Q 1 (t) λ1 λ1 λ = 1 u1 (t) Q 2 (t) λ 2 λ 2 λ 2 u 2 (t) 15/22 Define change of coordinates: z 1 (t) = Q 1 (t) + Q 2 (t) z 2 (t) = λ 2 Q 1 (t) λ 1 Q 2 (t) Then we have ] [ż1 (t) ż 2 (t) = [ ] λ1 + λ 2 0 [ λ1 + λ 2 λ 1 + λ ] [ u1 (t) u 2 (t) In particular the variable z 2 (t) evolves independent of the policy chosen. ]

30 Example: uncontrollable case Some words about case λ 1 = µ 1, λ 2 = µ 2, i.e., R not invertible Uncontrollable dynamics [ ] [ ] [ ] [ ] Q 1 (t) λ1 λ1 λ = 1 u1 (t) Q 2 (t) λ 2 λ 2 λ 2 u 2 (t) 15/22 Define change of coordinates: z 1 (t) = Q 1 (t) + Q 2 (t) z 2 (t) = λ 2 Q 1 (t) λ 1 Q 2 (t) Then we have ] [ż1 (t) ż 2 (t) = [ ] λ1 + λ 2 0 [ λ1 + λ 2 λ 1 + λ ] [ u1 (t) u 2 (t) In particular the variable z 2 (t) evolves independent of the policy chosen. ]

31 Example: controller design 16/22 System Q(t) = α Ru(t) Q(0) = Q 0 0 Q(t) 0 u(t) 1 Basic idea Decouple state from input, i.e. what does u i control? Define change of coordinates z(t) = R 1 Q(t): Transformed system z(t) = R 1 α u(t) = u u(t) z(0) = z 0 = R 1 Q 0 0 Rz(t) 0 u(t) 1

32 Example: controller design 16/22 System Q(t) = α Ru(t) Q(0) = Q 0 0 Q(t) 0 u(t) 1 Basic idea Decouple state from input, i.e. what does u i control? Define change of coordinates z(t) = R 1 Q(t): Transformed system z(t) = R 1 α u(t) = u u(t) z(0) = z 0 = R 1 Q 0 0 Rz(t) 0 u(t) 1

33 Example: controller design 16/22 System Q(t) = α Ru(t) Q(0) = Q 0 0 Q(t) 0 u(t) 1 Basic idea Decouple state from input, i.e. what does u i control? Define change of coordinates z(t) = R 1 Q(t): Transformed system z(t) = R 1 α u(t) = u u(t) z(0) = z 0 = R 1 Q 0 0 Rz(t) 0 u(t) 1

34 Example: controller design 16/22 System Q(t) = α Ru(t) Q(0) = Q 0 0 Q(t) 0 u(t) 1 Basic idea Decouple state from input, i.e. what does u i control? Define change of coordinates z(t) = R 1 Q(t): Transformed system z(t) = R 1 α u(t) = u u(t) z(0) = z 0 = R 1 Q 0 0 Rz(t) 0 u(t) 1

35 Example 17/22 Change of coordinates µ 2 λ 1 z 1 (t) = Q 1 (t) Q 2 (t) µ 1 µ 2 λ 1 λ 2 µ 1 µ 2 λ 1 λ 2 λ 2 µ 1 z 2 (t) = Q 1 (t) + Q 2 (t) µ 1 µ 2 λ 1 λ 2 µ 1 µ 2 λ 1 λ 2 Resulting control problem while making sure that z 1 (t) = u 1 u 1 (t) 0 u 1 (t) 1 z 2 (t) = u 2 u 2 (t) 0 u 2 (t) 1 [ ] [ ] µ1 λ 0 1 z1 (t) λ 2 µ 2 z 2 (t)

36 Example Neglecting the latter constraint, the problem of controlling z 1 (t) = u1 u 1 (t) 0 u 1 (t) 1 z 2 (t) = u2 u 2 (t) 0 u 2 (t) 1 becomes easy: 1 if z 1 (t) > 0 1 if z 2 (t) > 0 u 1 (t) = u1 if z 1 (t) = 0 u 2 (t) = u2 if z 2 (t) = 0 0 if z 1 (t) < 0 0 if z 2 (t) < 0 18/22 Observations Above controller also solves problem with constraint Optimal controller for minimizing 0 z(t) 1dt. Minimal time controller

37 Example Neglecting the latter constraint, the problem of controlling z 1 (t) = u1 u 1 (t) 0 u 1 (t) 1 z 2 (t) = u2 u 2 (t) 0 u 2 (t) 1 becomes easy: 1 if z 1 (t) > 0 1 if z 2 (t) > 0 u 1 (t) = u1 if z 1 (t) = 0 u 2 (t) = u2 if z 2 (t) = 0 0 if z 1 (t) < 0 0 if z 2 (t) < 0 18/22 Observations Above controller also solves problem with constraint Optimal controller for minimizing 0 z(t) 1dt. Minimal time controller

38 Example: Controller Controller for stochastic queueing network 19/22 { 1 if µ 2 λ µ u 1 (t) = 1 µ 2 λ 1 λ 2 Q 1 (t) > 1 µ 1 µ 2 λ 1 λ 2 Q 2 (t) and Q 1 (t) > 0 µ 0 if 2 λ µ 1 µ 2 λ 1 λ 2 Q 1 (t) < 1 µ 1 µ 2 λ 1 λ 2 Q 2 (t) or Q 1 (t) = 0 { 1 if λ 2 µ µ u 2 (t) = 1 µ 2 λ 1 λ 2 Q 1 (t) < 1 µ 1 µ 2 λ 1 λ 2 Q 2 (t) and Q 2 (t) > 0 λ 0 if 2 µ µ 1 µ 2 λ 1 λ 2 Q 1 (t) > 1 µ 1 µ 2 λ 1 λ 2 Q 2 (t) or Q 2 (t) = 0 Lyapunov function: cost-to-go from optimal control problem z1 2 /(1 u 1 ) + z2 2 /(1 u 2 ) if z 1 0 and z 2 0 z1 2 V(z) = /u 1 + z2 2 /(1 u 2 ) if z 1 0 and z 2 0 z1 2 /(1 u 1 ) + z2 2 /u 2 if z 1 0 and z 2 0 z1 2/u 1 + z2 2 /u 2 if z 1 0 and z 2 0

39 Example: Controller Controller for stochastic queueing network 19/22 { 1 if µ 2 λ µ u 1 (t) = 1 µ 2 λ 1 λ 2 Q 1 (t) > 1 µ 1 µ 2 λ 1 λ 2 Q 2 (t) and Q 1 (t) > 0 µ 0 if 2 λ µ 1 µ 2 λ 1 λ 2 Q 1 (t) < 1 µ 1 µ 2 λ 1 λ 2 Q 2 (t) or Q 1 (t) = 0 { 1 if λ 2 µ µ u 2 (t) = 1 µ 2 λ 1 λ 2 Q 1 (t) < 1 µ 1 µ 2 λ 1 λ 2 Q 2 (t) and Q 2 (t) > 0 λ 0 if 2 µ µ 1 µ 2 λ 1 λ 2 Q 1 (t) > 1 µ 1 µ 2 λ 1 λ 2 Q 2 (t) or Q 2 (t) = 0 Lyapunov function: cost-to-go from optimal control problem z1 2 /(1 u 1 ) + z2 2 /(1 u 2 ) if z 1 0 and z 2 0 z1 2 V(z) = /u 1 + z2 2 /(1 u 2 ) if z 1 0 and z 2 0 z1 2 /(1 u 1 ) + z2 2 /u 2 if z 1 0 and z 2 0 z1 2/u 1 + z2 2 /u 2 if z 1 0 and z 2 0

40 Controller design: general case 20/22 System Q(t) = α Ru(t) Q(0) = Q 0 0 Q(t) 0 u(t) Cu(t) 1 Change of coordinates: z(t) = R 1 Q(t) Transformed system z(t) = u u(t) z(0) = z 0 0 Rz(t) 0 u(t) Cu(t) 1 Objective min u(t) 0 z(t) 1 dt

41 Controller design: general case 20/22 System Q(t) = α Ru(t) Q(0) = Q 0 0 Q(t) 0 u(t) Cu(t) 1 Change of coordinates: z(t) = R 1 Q(t) Transformed system z(t) = u u(t) z(0) = z 0 0 Rz(t) 0 u(t) Cu(t) 1 Objective min u(t) 0 z(t) 1 dt

42 Controller design: general case 20/22 System Q(t) = α Ru(t) Q(0) = Q 0 0 Q(t) 0 u(t) Cu(t) 1 Change of coordinates: z(t) = R 1 Q(t) Transformed system z(t) = u u(t) z(0) = z 0 0 Rz(t) 0 u(t) Cu(t) 1 Objective min u(t) 0 z(t) 1 dt

43 Controller design: general case 20/22 System Q(t) = α Ru(t) Q(0) = Q 0 0 Q(t) 0 u(t) Cu(t) 1 Change of coordinates: z(t) = R 1 Q(t) Transformed system z(t) = u u(t) z(0) = z 0 0 Rz(t) 0 u(t) Cu(t) 1 Objective min u(t) 0 z(t) 1 dt

44 mpsclp Multi parametric Separated Continuous Linear Program: 21/22 min u(t) 0 z 1 (t) dt subject to ż(t) = u u(t) z(0) = z 0 0 u(t) Cu(t) 1 0 Rz(t) Multi parametric since we want solution as function of z 0. Conjecture mpsclp can be solved explicitely and solution has nice structure

45 mpsclp Multi parametric Separated Continuous Linear Program: 21/22 min u(t) 0 z 1 (t) dt subject to ż(t) = u u(t) z(0) = z 0 0 u(t) Cu(t) 1 0 Rz(t) Multi parametric since we want solution as function of z 0. Conjecture mpsclp can be solved explicitely and solution has nice structure

46 T mpsclp: structure 22/22 x q t