Stochastic Optimization for Undergraduate Computer Science Students

Transcription

1 Stochastic Optimization for Undergraduate Computer Science Students Professor Joongheon Kim School of Computer Science and Engineering, Chung-Ang University, Seoul, Republic of Korea 1

2 Reference 2

3 Outline Motivation Theory Introduction to Queues (Queue Dynamics) Basic Queueing Theory: A Quick Review Dynamic Server Example Lyapunov Drift and Lyapunov Optimization Applications Introduction Core Scheduling with 3-Core CPU Buffer-Stable Adaptive Per-Module Power Allocation for Energy-Efficient 5G Platforms Quality-Aware Streaming and Scheduling for Device-to-Device Video Delivery 3

4 Optimization with Queueing Matching with Queues Q 1 [t] a 1 p 11 =3 b 1 p 12 =0 p 21 =0 Q 2 [t] a 2 p 22 =1 b 2 p 31 =2 Q 3 [t] a 3 p 32 =0 4

5 Outline Motivation Theory Introduction to Queues (Queue Dynamics) Basic Queueing Theory: A Quick Review Dynamic Server Example Lyapunov Drift and Lyapunov Optimization Applications Introduction Core Scheduling with 3-Core CPU Buffer-Stable Adaptive Per-Module Power Allocation for Energy-Efficient 5G Platforms Quality-Aware Streaming and Scheduling for Device-to-Device Video Delivery 5

6 Queue Dynamics Q i [t] Queue Resources Arrival Process a i [t] Departure (Service) Process b i [t] [Definition (Queue Dynamics)] A single-server discrete-time queueing system: Q t + 1 = max Q t b t, 0 + a t, for all t = 0,1,2, Alternative Form: where Q t + 1 = Q t b t + a t, for all t = 0,1,2, b t = min Q t, b[t] 6

7 Queue Dynamics [Definition (Time-Average Rate)] An arrival process a[t] t=0 and a service process b[t] t=0 have time average rates a and b, respectively, if 1 lim t t t 1 τ=0 a[τ] = a, 1 lim t t t 1 τ=0 b[τ] = b [Definition (Rate Stable)] A queue Q[t] is rate stable if Q[t] lim t t = 0 lim Q t < t [Definition (Mean Rate Stable)] A queue Q[t] is mean rate stable if E Q[t] lim t t = 0 lim E Q t < t 7

8 Outline Motivation Theory Introduction to Queues (Queue Dynamics) Basic Queueing Theory: A Quick Review Dynamic Server Example Lyapunov Drift and Lyapunov Optimization Applications Introduction Core Scheduling with 3-Core CPU Buffer-Stable Adaptive Per-Module Power Allocation for Energy-Efficient 5G Platforms Quality-Aware Streaming and Scheduling for Device-to-Device Video Delivery 8

9 Brief Introduction Q[t] Queue Resources Arrival Rate (λ) Departure/Service Rate (μ) Basic Assumption for Stability: μ > λ Queues B/B/1 Queue: Arrival with Bernoulli Distribution and Departure with Bernoulli Distribution B/G/1 Queue: Arrival with Bernoulli Distribution and Departure with General Distribution G/G/1 Queue: Arrival with General Distribution and Departure with General Distribution B/B/1 Queue Size Q = λ(1 λ) μ λ Queueing Delay (Little s Theorem) W = Q λ = 1 λ μ λ 9

10 Outline Motivation Theory Introduction to Queues (Queue Dynamics) Basic Queueing Theory: A Quick Review Dynamic Server Example Lyapunov Drift and Lyapunov Optimization Applications Introduction Core Scheduling with 3-Core CPU Buffer-Stable Adaptive Per-Module Power Allocation for Energy-Efficient 5G Platforms Quality-Aware Streaming and Scheduling for Device-to-Device Video Delivery 10

11 Dynamic Scheduling Example (3-Queue and 2-Server Problem) a 1 [t] a 2 [t] Q 1 [t] Q 2 [t] [Problem] All packets have fixed length, and a queue that is allocated a server on a given slot can serve exactly one packet on that slot. Every slot we choose which 2 queues to serve. The service is given for i 1,2,3 by: b i t = 1 if a server is connected to queue i on time slot t; and b i t = 0 for the other cases. Assume the arrival processes have well defined time average rates a 1 av, a 2 av, a 3 av, in units of packets/slot. Design a server allocation algorithm to make all queues rate stable when arrival rate is a 1 av, a 2 av, a 3 av = 0.7,0.9,0.4. a 3 [t] Q 3 [t] [Solution] Choose the service vector b 1 t, b 2 t, b 3 t every slot, as follows: (0, 1, 1) with probability p 1, (1, 0, 1) with probability p 2, and (1, 1, 0) with probability p 3. 11

12 Outline Motivation Theory Introduction to Queues (Queue Dynamics) Basic Queueing Theory: A Quick Review Dynamic Server Example Lyapunov Drift and Lyapunov Optimization Applications Introduction Core Scheduling with 3-Core CPU Buffer-Stable Adaptive Per-Module Power Allocation for Energy-Efficient 5G Platforms Quality-Aware Streaming and Scheduling for Device-to-Device Video Delivery 12

13 Lyapunov Drift and Optimization Time-Average Optimization under Queue Stability minimize: y 0 subject to: [C1] y l 0, l 1,, L, [C2] α t A ω t, t 0,1,, [C3] Queue Stability t 1 1 y 0 = lim y t t 0 [t] τ=0 t 1 1 y l = lim y t t l [t], l 1,, L τ=0 ω[t]: random event at t α[t]: action control which is chosen after observing ω[t] at t A ω[t] : an action space associated with ω[t] Lyapunov Drift 13

14 Lyapunov Drift and Optimization [Definition (Quadratic Lyapunov Function) or (Lyapunov Function)] L Q[t] = 1 2 K k=1 Q 2 k [t], where Q t = Q 1 t,, Q K t [Definition (One-Slot Conditional Lyapunov Drift) or (Lyapunov Drift)] Q[t] = E L Q[t + 1] L Q[t] Q[t] means the expected change in the Lyapunov function over one slot t t + 1, given that the current state in slot t is Q[t]. 14

15 Lyapunov Drift and Optimization [Theorem (Lyapunov Drift)] Consider the quadratic Lyapunov function, and assume E L Q[t] <. Suppose that there exist constants B > 0, ε 0, such that the following drift condition holds, τ 0,1, and all possible Q τ : Q τ B ε Then, If ε 0, then all queues Q k τ are mean rate stable. K k=1 Q k τ 15

16 Lyapunov Drift and Optimization [Theorem (Lyapunov Optimization)] Suppose that E L Q[t] < and there exist constants B > 0, V 0, ε 0. In addition, suppose that there exists y such that for all slots τ 0,1, and all possible values of Q τ : Q τ + V E y 0 τ Q τ B + V y ε K k=1 Q k τ [Theorem (Optimization with Lyapunov Drift)] Q τ + V E y 0 τ Q τ B + V E y 0 τ Q τ + K k=1 Q k τ E a k t b k t Q τ 16

17 Lyapunov Drift and Optimization [Theorem (Optimization with Lyapunov Drift)] Q τ + V E y 0 τ Q τ B + V E y 0 τ Q τ + K k=1 Q k τ E a k t b k t Q τ minimize: V y 0 α t, ω t K + k=1 Q k t a k α t, ω t b k α t, ω t Stable? Separable? V? 17

18 Outline Motivation Theory Introduction to Queues (Queue Dynamics) Basic Queueing Theory: A Quick Review Dynamic Server Example Lyapunov Drift and Lyapunov Optimization Applications Introduction Core Scheduling with 3-Core CPU Buffer-Stable Adaptive Per-Module Power Allocation for Energy-Efficient 5G Platforms Quality-Aware Streaming and Scheduling for Device-to-Device Video Delivery 18

19 Introduction (For better understanding ) Basic Form (Separable) minimize: V y 0 α t, ω t K + k=1 Q k t a k α t, ω t b k α t, ω t Objective Function minimize: V y 0 α t + Q t a α t b α t Observation Control 19

20 Outline Motivation Theory Introduction to Queues (Queue Dynamics) Basic Queueing Theory: A Quick Review Dynamic Server Example Lyapunov Drift and Lyapunov Optimization Applications Introduction Core Scheduling with 3-Core CPU Buffer-Stable Adaptive Per-Module Power Allocation for Energy-Efficient 5G Platforms Quality-Aware Streaming and Scheduling for Device-to-Device Video Delivery 20

21 Example #1: Core Scheduling with 3-Core CPU Task Queue, Q[t] Processing with Multiple Cores Arrival Process a[t] Departure Process b[t] Tradeoff More core allocation more power consumption (-); more departure in queue (good for stability) (+) Less core allocation less power consumption (+); less departure in queue (bad for stability) (-) Objective of Optimization with the Tradeoff We want to minimize time-average power consumption subject to queue stability 21

22 Example #1: Core Scheduling with 3-Core CPU Optimization with Lyapunov Drift Minimize: V P α t + Q t a α t b α t V: Tradeoff parameter α t : Core selection action at t (in this three core case, α t 0,1,2,3 ) Q t : Queue backlog size at t a α t : Arrival process with given control action at t: In this case, the arrival is random. So, this will be ignored. b α t : Departure process with given control action at t P α t : Power consumption when our core selection is α t Final Form: Minimize: V P α t Q t b α t Intuition If queue is empty (Q t = 0), we have to minimize V P α t. This means we do not need to allocate cores. If queue is almost infinite (Q t ), we have to minimize b α t (i.e., maximize b α t ). This means we have to allocate all cores. 22

23 Example #1: Core Scheduling with 3-Core CPU Example-based Understanding Minimize: V P α t Q t b α t [denoted by F t ] V = 10; // We want to focus on our objective function ten times more than queue stability α t 0,1,2,3 ; // We can allocation 1, 2, or 3 cores. Or, we can turn off the CPU b α t ; b α t = 0 =0; // If no cores are selected, there is no departure process. b α t = 1 =6; // If 1 core is selected, 6 tasks will be processed. b α t = 2 =11; // If 2 cores are selected, 11 tasks will be processed. b α t = 3 =15; // If 3 cores are selected, 15 tasks will be processed. P α t ; P α t = 0 = 0; // If no cores are selected, there is no power consumption. P α t = 1 =3; // If 1 core is selected, there exits 3 amounts of power consumption. P α t = 2 =5; // If 2 cores are selected, there exits 5 amounts of power consumption. P α t = 3 =6; // If 3 cores are selected,there exits 6 amounts of power consumption. 23

24 Example #1: Core Scheduling with 3-Core CPU Tradeoff Table b α t P α t α t = α t = α t = α t = Actions: F t = V P α t Q t b α t where V = 10 t = 0 t = 1 t = 2 t = 3 Queue-Backlog, Q t Arrival (random), a t α t α t = = = = = 0 α t = = = = = 0 α t = = = = = 5 α t = = = = = 15 24

25 Outline Motivation Theory Introduction to Queues (Queue Dynamics) Basic Queueing Theory: A Quick Review Dynamic Server Example Lyapunov Drift and Lyapunov Optimization Applications Introduction Core Scheduling with 3-Core CPU Buffer-Stable Adaptive Per-Module Power Allocation for Energy-Efficient 5G Platforms Quality-Aware Streaming and Scheduling for Device-to-Device Video Delivery 25

26 Example #2: Buffer-Stable Adaptive Per-Module Power Allocation Tradeoff More MAA power-on more power consumption (-); more departure in queue (good for stability) (+) Less MAA power-on less power consumption (+); less departure in queue (bad for stability) (-) Objective of Optimization with the Tradeoff We want to minimize time-average power consumption subject to queue stability 26

27 Example #2: Buffer-Stable Adaptive Per-Module Power Allocation Optimization with Lyapunov Drift Minimize: V P α t + Q t a α t b α t V: Tradeoff parameter α t : MAA power-on/off action at t (α t 0,1,2,3,4,5,6,7,8 ), i.e., the number of power-on MAAs Q t : Queue backlog size at t a α t : Arrival process with given control action at t: In this case, the arrival is random. So, this will be ignored. b α t : Departure process with given control action at t P α t : Power consumption when our MAA power-on/off decision is α t Final Form: Minimize: V P α t Q t b α t Intuition If queue is empty (Q t = 0), we have to minimize V P α t. This means we do not need to power-on MAAs. If queue is almost infinite (Q t ), we have to minimize b α t (i.e., maximize b α t ). This means we have to power-on all MAAs. 27

28 Example #2: Buffer-Stable Adaptive Per-Module Power Allocation Plotting Result More Energy-Efficiency (V increases) More Queue Stability (V decreases) 28

29 Outline Motivation Theory Introduction to Queues (Queue Dynamics) Basic Queueing Theory: A Quick Review Dynamic Server Example Lyapunov Drift and Lyapunov Optimization Applications Introduction Core Scheduling with 3-Core CPU Buffer-Stable Adaptive Per-Module Power Allocation for Energy-Efficient 5G Platforms Quality-Aware Streaming and Scheduling for Device-to-Device Video Delivery 29

30 Example #3: Quality-Aware Streaming and Scheduling Tradeoff High compression on chunks Low quality on chunks (-); More stabilization on queues (+) Less compression on chunks High quality on chunks (+); Less stabilization on queues (-) Objective of Optimization with the Tradeoff We want to maximize time-average video quality subject to queue stability 30

31 Example #3: Quality-Aware Streaming and Scheduling Optimization with Lyapunov Drift Maximize: V Quality α t Q t a α t b α t V: Tradeoff parameter α t : Compression action at t (α t 1,2,3 ), i.e., three different levels of compression Q t : Queue backlog size at t a α t : Arrival process with given α t at t: This is the size of chunks (due to three different levels of compression) b α t : Departure process at t: In this case, the system transmits packets as much as the network allows. P α t : Power consumption when our MAA power-on/off decision is α t Final Form: Maximize: V Quality α t Q t a α t Intuition If queue is empty (Q t = 0), we have to maximize V Quality α t. This means we are doing less compression for better quality of streaming. If queue is almost infinite (Q t ), we have to minimize a α t. This means we are doing high compression for queue stability. 31