On the Design and Application of Thermal Isolation Servers

On the Design and Application of Thermal Isolation Servers Rehan Ahmed, Pengcheng Huang, Max Millen, Lothar Thiele EMSOFT 2017 October 16, 2017 1/25

2/25 The Temperature Problem 1 1 Extremetech: https://tinyurl.com/y8gmopks 2 CPU DB: Recording Microprocessor History : https://tinyurl.com/y89ysl93 3 TECHPOWERUP: https://tinyurl.com/y74yaoyx

2/25 The Temperature Problem 1 2 1 Extremetech: https://tinyurl.com/y8gmopks 2 CPU DB: Recording Microprocessor History : https://tinyurl.com/y89ysl93 3 TECHPOWERUP: https://tinyurl.com/y74yaoyx

2/25 The Temperature Problem 1 2 3 1 Extremetech: https://tinyurl.com/y8gmopks 2 CPU DB: Recording Microprocessor History : https://tinyurl.com/y89ysl93 3 TECHPOWERUP: https://tinyurl.com/y74yaoyx

2/25 The Temperature Problem 1 2 3 Deadline miss 1 Extremetech: https://tinyurl.com/y8gmopks 2 CPU DB: Recording Microprocessor History : https://tinyurl.com/y89ysl93 3 TECHPOWERUP: https://tinyurl.com/y74yaoyx

2/25 The Temperature Problem 1 2 3 Deadline miss τ 1 0 1 2 3 4 5 τ 1 s deadline 1 Extremetech: https://tinyurl.com/y8gmopks 2 CPU DB: Recording Microprocessor History : https://tinyurl.com/y89ysl93 3 TECHPOWERUP: https://tinyurl.com/y74yaoyx

2/25 The Temperature Problem 1 2 3 Deadline miss τ 2 τ 1 0 1 2 3 4 5 τ 1 s deadline 1 Extremetech: https://tinyurl.com/y8gmopks 2 CPU DB: Recording Microprocessor History : https://tinyurl.com/y89ysl93 3 TECHPOWERUP: https://tinyurl.com/y74yaoyx

2/25 The Temperature Problem 1 2 3 Deadline miss DTM τ 2 τ 1 τ 1 0 1 2 3 4 5 τ 1 s deadline 1 Extremetech: https://tinyurl.com/y8gmopks 2 CPU DB: Recording Microprocessor History : https://tinyurl.com/y89ysl93 3 TECHPOWERUP: https://tinyurl.com/y74yaoyx

3/25 Multicore Makes Things Harder 1. Inter core thermal interactions. Core 0 Core 1 τ 2 DTM Core 2 Core 3 τ 1 0 1 2 3 4 5

3/25 Multicore Makes Things Harder 1. Inter core thermal interactions. Core 0 Core 1 τ 2 DTM Core 2 Core 3 τ 1 0 1 2 3 4 5 2. Rise/fall in temperature may be non-monotonic. Core 0 25 Core 2 25 Core 1 25 Core 3 25 time C0 Temperature

3/25 Multicore Makes Things Harder 1. Inter core thermal interactions. Core 0 Core 1 τ 2 DTM Core 2 Core 3 τ 1 0 1 2 3 4 5 2. Rise/fall in temperature may be non-monotonic. Core 0 25 Core 2 25 Core 1 25 Core 3 25 Core 0 50 Core 2 0 Core 1 0 Core 3 0 time C0 Temperature

4/25 Mixed-Criticality Complicates Problem Further LO DTM HI 0 1 2 3 4 5 Task that heats up the processor may be low criticality. Task that suffers due to deadline violation may be high criticality. Certification demands isolation.

5/25 Related Research Unicore: Wang and Bettati (2006), Chen, Wang, and Thiele (2009), Kumar and Thiele (2011), M. Ahmed et al. (2011). Multicore: Chantem, Dick, and Hu (2011), Fisher et al. (2009), R. Ahmed, Ramanathan, and Saluja (2016). Temperature bounding: Schor, Bacivarov, et al. (2012) and Schor, Yang, et al. (2011)

5/25 Related Research Unicore: Wang and Bettati (2006), Chen, Wang, and Thiele (2009), Kumar and Thiele (2011), M. Ahmed et al. (2011). Multicore: Chantem, Dick, and Hu (2011), Fisher et al. (2009), R. Ahmed, Ramanathan, and Saluja (2016). Temperature bounding: Schor, Bacivarov, et al. (2012) and Schor, Yang, et al. (2011) Common limitations: 1. No schemes proposed for mixed-critical applications. 2. Current schemes solve the temperature minimization problem, NOT the temperature isolation problem.

6/25 Contributions 1. Propose Thermal Isolation Servers Provide thermal isolation by construction. Can bound the temperature increase caused by a set of tasks. Time and space composable. Can schedule tasks based on both dynamic and fixed priority. 2. Validated on a hardware platform. Mixed-critical flight management system.

7/25 Thermal Model Fundamentals to upper layer to upper layer R 12 =R 21 R 23 =R 32 T 1 T 2 T 3 1/K 11 1/K 22 1/K 33 T A C 11 P 1 T A C T A 22 P 2 C 33 P 3 Model T (t) = A T (t) + B(t) Where A = C 1 (G + φ K) and B(t) = C 1 (K T A + ψ(t)) Steady state T (B(0)) = A 1 B(0) Thermal model solution T (t) = e A t (T (0) T (B(0)) + T (B(0))

8/25 Thermal Component Based Analysis Thermal component Θ(t, z): Temperature increase caused by a given execution. z st z e z cr z

8/25 Thermal Component Based Analysis Thermal component Θ(t, z): Temperature increase caused by a given execution. z st z e z cr z Temperature ( C) 16 12 8 4 0 Cooling component 16 12 8 4 0 0.00 0.01 0.02. +.+ 16 12 8 4 0 0.1 Θ(t, z 1) 0.1 + τ 1 0.0 0.00 0.01 0.02 + 0.0 Θ(t, z 2) 16 12 8 4 0 0.00 0.01 0.02 Time (sec) = τ 2 = 16 12 8 4 0 Overall temperature Core 1 16 12 8 Core 2 4 0 0.00 0.01 0.02

9/25 Thermal Isolation Servers A given Thermal Isolation Server (TIS) S i is a statically scheduled periodic resource characterized by the following attributes: P i = 3 U i P i = 2 φ i = 0.5... 0 1 2 3 4 5 6 7 1. P i : Period 2. U i : Utilization 3. φ i : Phase 4. cr i : Core where S i is executed. Also called self core 5. Π i : Taskset assigned to S i 6. Λ i : Thermal budget. Function of Pi, U i and cr i

10/25 TIS Temperature Guarantees Thermal budget increases with Utilization. Thermal budget increases with Period. Fluid server is optimal. Server period is infinitesimally small. Not practical. Λ i 1 ( C) 18 16 14 12 10 8 6 4 2 0 1.0 0.8 0.6 Ui 0.4 0.2 0.0 0.10.20.3 0.4 0.5 Pi (s)

Temperature 10/25 TIS Temperature Guarantees Thermal budget increases with Utilization. Thermal budget increases with Period. Fluid server is optimal. Server period is infinitesimally small. Not practical. Schedule 1 Schedule 2 Fluid Λ i 1 ( C) 18 16 14 12 10 8 6 4 2 0 1.0 0.8 0.6 Ui 0.4 0.2 0.0 0.10.20.3 0.4 0.5 Pi (s) Time Schedule 2 Schedule 1 Time

11/25 TIS Temperature Guarantees Augmented Utilization: U i (ɛ) = max(p i U i ɛ, 0) P i... 0 1 2 3 4 5 6 7

11/25 TIS Temperature Guarantees Augmented Utilization: U i (ɛ) = max(p i U i ɛ, 0) P i... 0 1 2 3 4 5 6 7 Augmented Utilization 0.3 0.2 0.1 0.0 Λ i i = 5 C ɛ = 0 ɛ = 10µs ɛ = 50µs ɛ = 100µs 0.000 0.005 0.010 Period (s) Maximum utilization point is non-zero. Favor high period servers when ɛ is large and vice-versa.

12/25 TIS Timing Guarantees Supply Bound Function of TIS: sbf(s i, l, ɛ) = l/p i P i U i (ɛ) + max {l P i (1 U i (ɛ)) l/p i P i, 0} Demand Bound Function dbf EDF (Π, l) = τ j Π max {( (l D j )/W j + 1) E j, 0} 4 3 sbf(s i, l, ɛ) dbf EDF ({τ j }, l) U i(ɛ)p i = 1.2 2 P i = 2 1 E j = 0.8 D j = 1.8 0 0 1 2 3 4 5 6 W j = 2.0 l

13/25 Composability of Servers Theorem 5.8: Given n TISs, the maximum temperature increase due to their execution is upper bounded by: 1 i n Λ i Temporal composability Spacial composability S 2 S 1 S 2 S 1... 0 1 2 3 4 5 Restrictive due to same period. Possibly better to have one larger server. S 1 S 1 S 2 S 2... 0 1 2 3 4 5 No same period restriction. Servers can be designed independently.

14/25 Design heuristic taskset partition tasks Partitioned tasks Search TISs TIS configurations MILP formulation: Minimize maximum temperature for period = 0. Utilization of each core is 1. For each core, search TIS such that: Conditions for timing feasibility are satisfied. Termal budget is minimized. Schedule

15/25 Setup: Evaluation Tasks: Synthetically generated, harmonic, implicit deadline periodic tasks. Platform: 8 mm Core0 Core1 Core2 Core3 8 mm Scheduling schemes: Parameter ψ active ψ idle φ T A Value 70 W 20 W zero matrix 25 C WF_EDF_x: Tasks partitioned using worst-fit bin packing. EDF scheduling used. x specifies the preemption overhead in µs. Opt_TIS_x: Tasks are scheduled using TISs. x is the server overhead (ɛ) in µs. EDF used within server active time.

16/25 Evaluation 1.00 Schedulability 0.75 0.50 0.25 0.00 Opt TIS 0 Opt TIS 20 Opt TIS 50 WF EDF 0 0.675 0.700 0.725 0.750 0.775 0.800 0.825 Utilization Thermal isolation has low cost. May even improve schedulability!!

17/25 Emulation Platform: Lenovo Thinkpad T440p (Core i7-4700mq quad-core processor). Operating frequency of all cores set to max (3.2 GHz). Fan speed set to max. OS: Ubuntu 16.04 with preempt-rt. Augmented SF3P[Sigrist et al. (2015)] scheduling framework. Thermal constraint = 70 C 4 4 https://www3.lenovo.com/us/en/laptops/thinkpad/t-series/t440p/

18/25 Emulation Application: Flight management system: Purpose CL # P(ms)E (ms) Sensor data acquisition HI 5 200 10 HI 3 200 10 Localization HI 3 1000 50 HI 1 5000 50 Flight-plan HI 4 1000 50 management LO 4 1000 50 HI 2 1000 50 HI 1 5000 750 Flight-plan HI 1 5000 180 computation HI 1 5000 150 HI 1 5000 90 HI 1 5000 75 Guidance HI 1 200 10 Nearest AirportLO 1 1000 50 5 How to design TISs So that there is no thermal violation? How to guarantee this? 5 https://www.anandtech.com/show/4083/the-sandy-bridge-review-intel-corei7-2600k-i5-2500k-core-i3-2100-tested

18/25 Emulation Application: Flight management system: Purpose CL # P(ms)E (ms) Sensor data acquisition HI 5 200 10 HI 3 200 10 Localization HI 3 1000 50 HI 1 5000 50 Flight-plan HI 4 1000 50 management LO 4 1000 50 HI 2 1000 50 HI 1 5000 750 Flight-plan HI 1 5000 180 computation HI 1 5000 150 HI 1 5000 90 HI 1 5000 75 Guidance HI 1 200 10 Nearest AirportLO 1 1000 50 5 C0 SF3P C1 LO EDF C2 HI TIS C3 HI TIS How to design TISs So that there is no thermal violation? How to guarantee this? 5 https://www.anandtech.com/show/4083/the-sandy-bridge-review-intel-corei7-2600k-i5-2500k-core-i3-2100-tested

19/25 Emulation Step 1: Determine the available thermal budget for TISs Perform thermal callibration tests to determine the peak temperature caused by execution of LO tasks on core 1 (T LO ). Available thermal budget: T T LO = [16, 28.88, 27.4]

19/25 Emulation Step 1: Determine the available thermal budget for TISs Perform thermal callibration tests to determine the peak temperature caused by execution of LO tasks on core 1 (T LO ). Available thermal budget: T T LO = [16, 28.88, 27.4] Step 2: Determine the thermal model Steady state: Directly from calibration tests. Transient: Estimating the temperature transfer function.

19/25 Emulation Step 1: Determine the available thermal budget for TISs Perform thermal callibration tests to determine the peak temperature caused by execution of LO tasks on core 1 (T LO ). Available thermal budget: T T LO = [16, 28.88, 27.4] Step 2: Determine the thermal model Steady state: Directly from calibration tests. Transient: Estimating the temperature transfer function. Step 3: Design TISs Partition HI tasks to cores 2 and 3 Search for servers Verify that total budget is less than T T LO.

20/25 Emulation Worst-Fit with EDF Temperature ( C) Core 1 70 65 60 55 50 0 10 20 30 40 50 Core 2 0 10 20 30 40 50 Time (sec) Core 3 0 10 20 30 40 50 Thermal constraint is violated!

21/25 Emulation Temperature ( C) Core 1 70 bound=65.17 65 60 55 50 0 10 20 30 40 50 Our Aproach Core 2 bound=62.63 0 10 20 30 40 50 Time (sec) Core 3 bound=63.53 0 10 20 30 40 50 S 1 : P 1 = 10ms, U 1 = 0.693, cr 1 = 2, Λ 1 = [6.42, 14.62, 5.69] S 2 : P 1 = 10ms, U 1 = 0.546, cr 2 = 3 Λ 2 = [3.75, 5.90, 14.24] Bound = T LO + Λ 1 + Λ 2

22/25 Takeaways... Proposed a TISs which provide thermal isolation on a multicore by construction. Temporal and spacial composability. Static or dynamic priority scheduling. Proposed a heuristic to approach to design TISs. Schedulability cost of providing thermal isolation is small. Emulated the TIS on a hardware platform to validate theory.

Questions?? 23/25

24/25 Backup 1 Following three calibration tests are performed: 1. Test1: Core 1 executing LO using EDF. Cores 2 and 3 idle. 2. Test2: Core 1 executing LO using EDF. Cores 2, 3 active. 3. Test3: Core 1 always idle. Cores 2 and 3 active. T [ P 99.9(T (t, Test1)) 1 P 99.9(T (t, Test2)) 2 P 99.9(T (t, Test3)) 2 + [T (B idle )] 2 P 99.9(T (t, Test2)) 3 P 99.9(T (t, Test3)) 3 + [T (B idle )] 3 ] = [ 16.0 28.88 27.4 ]

25/25 Backup 2 6 6 https://www.anandtech.com/show/4083/the-sandy-bridge-review-intel -core-i7-2600k-i5-2500k-core-i3-2100-tested