1 Energy savings in data centers: A framework for modelling and control of servers cooling Riccardo Lucchese Jesper Olsson Anna-Lena Ljung Winston Garcia-Gabin Damiano Varagnolo ABB Corporate Research, Västerås, Sweden Luleå University of Technology, Sweden
2 Definition: Data center From Wikipedia 1 : [...] a facility used to house computer systems and associated components the main purpose [...] is running the IT systems applications [...] of the organization [...] large data centers are industrial scale operations using as much electricity as a small town 1 wikipedia.org/wiki/data_center
3
4
5 Structures within a data center and their logical connections computing server blades storage networking equipment... CRACs Cooling towers... cooling power UPSs PDUs...
6 Footprint EU-28: (year 2013) Industry market: 18.85 billion e Electrical power consumption: 103.4 GWh ( 3% of electrical power production) 38.6 metric tonnes of CO 2 emissions (347g/kWh)
7 Is there a problem to solve? i) Electronics convert virtually all power into heat ii) Cooling accounts for up to 40% of power budget iii) Computing s power consumption dependent on thermodynamical state iv) DCs rarely operate at peak computing capacity 2 Problem: Efficiency (static, nominal, cooling policies waste power) 2 Often desired since it is paramount to maintain Quality of Service
8 Control-oriented solution Measure & predict the thermodynamical state to reduce overprovision of the cooling resources
9 Potential control levels i) single server level (focus of this paper) ii) rack level iii) room level
Example of atomic unit : air-cooled blade air outlet CPUs DIMMs fans air inlet 10
Modeling air-cooled blades, intuitions 11
11 Modeling air-cooled blades, intuitions u 1 u 2 u m
11 Modeling air-cooled blades, intuitions u 1 u 2 λ d 12 λ d 22 λ d n2 u m
11 Modeling air-cooled blades, intuitions λ r 11 u 1 u 2 λ d 12 λ d 22 λ r 21 λ r 12 λ d n2 u m
12 Modeling air-cooled blades, in math Notation states: x c = [x c 1... x c n ]T = temperatures of IT components x f = [x f 1... x f n ]T = temperatures of air flows through IT components exogenous inputs: p = [p 1... p n ] T = electrical power dissipated by IT components x i = temperature of air inlet manipulable inputs: u = [u 1... u m ] T = air mass flows produced by fans (determining the air flows through the IT components f = [f 1... f n ] T )
13 Ingredients to be modeled 1 2 n i) mass of the air flows f ii) temperature of the air flows x f iii) temperature of the IT components x c u 1 u 2 u m
14 Model of the mass of the air flows 1 2 n f = Λ d [u] + Λ r [f] (1) Λ[u] = ( m+d d ) i=1 a i u α i (2) u 1 u 2 u m Generalizes: i) models without warm recirculation flows (Λ r = 0) ii) models preserving total mass-flows (Λ d and Λ r row-stochastic matrices)
Model of the mass of the air flows 1 2 n f = Λ d [u] + Λ r [f] (1) Λ[u] = ( m+d d ) i=1 a i u α i (2) u 1 u 2 u m Generalizes: i) models without warm recirculation flows (Λ r = 0) ii) models preserving total mass-flows (Λ d and Λ r row-stochastic matrices)
15 Model of the temperature of the air flows Assumptions: i) perfect flow mixing ii) heat energy conservation x f j = Λd (j) [u] xi + n h=1 Λr (j,h) [f] xc h f j (3) corresponds to averaging
Model of the temperature of the air flows Assumptions: i) perfect flow mixing ii) heat energy conservation x f j = Λd (j) [u] xi + n h=1 Λr (j,h) [f] xc h f j (3) corresponds to averaging
16 Model of the temperature of the IT components ẋ c j = h j f j (u) (x c j x f j (u, xc, x i )) convection + [R (j) ρ j ] [ xc x i ] conduction (4) + b j p j electrical power Euler forward discretization: x c j(k + 1) = Ψ j (x c j(k), u, p, x i ) (5)
Model of the temperature of the IT components ẋ c j = h j f j (u) (x c j x f j (u, xc, x i )) convection + [R (j) ρ j ] [ xc x i ] conduction (4) + b j p j electrical power Euler forward discretization: x c j(k + 1) = Ψ j (x c j(k), u, p, x i ) (5)
17 Minimum cost fan control H 1 m min u(0),...,u(h 1) t=0 h=1 (u h (t)) 3 subject to (for 0 t H 1, 1 j n): x c (0) = x c 0 (6) u min u(t) u max x c (t + 1) x c max x c j(t + 1) = Ψ j (xc j(t), u(t), p(t), x i )
18 numerical simulations i) how accurate is the polynomial flows model? ii) how conservative is the control strategy?
19 Validation of the model using CFD Algorithm: 1 set boundary conditions (i.e., p, u, x i ) 2 use CFD model as a virtual plant 3 estimate the parameters using RLS
20 Validation of the model using CFD deg(λ d ) = 2, deg(λ r ) = 1 CPU temp. 80 60 40 2 0 2 error CPU temp. 80 60 40 deg(λ d ) = 2, deg(λ r ) = 2 2 0 100 200 300 400 500 600 700 Trial 2 error
21 Validation of the control strategy Experiment 1: p(t) = p max Experiment 2: p(t) = stochastic process with seasonal trends (medium-high loads)
22 Validation of the control strategy - Experiment 1 80 CPU 1 x c (t) [ C] 60 40 CPU 2 DIMM 1 DIMM 2 DIMM 3 DIMM 4 u(t) [1] 1 0.8 0.6 0.4 0.2 0 1 10 20 30 40 50 60 Time t [s] fan 1 fan 2 fan 3 fan 4 fan 5 fan 6
23 Validation of the control strategy - Experiment 2 80 CPU 1 x c (t) [ C] 60 40 CPU 2 DIMM 1 DIMM 2 DIMM 3 DIMM 4 u(t) [1] 1 0.8 0.6 0.4 0.2 0 60 70 80 90 100 110 120 Time t [s] fan 1 fan 2 fan 3 fan 4 fan 5 fan 6
24 Conclusions and future works i) promising ability at capturing complex convective cooling effects ii) functional structure open to other cooling applications iii) can be identified starting from CFD models i) validate on real plants ii) generalize to datacenter room control iii) generalize to generic thermal networks
24 Conclusions and future works i) promising ability at capturing complex convective cooling effects ii) functional structure open to other cooling applications iii) can be identified starting from CFD models i) validate on real plants ii) generalize to datacenter room control iii) generalize to generic thermal networks
25 RISE SICS North ICE 3000-4000 servers (2MW) 160 m 2 lab biogas back up generators connections with the urban district heating network Generality, Flexibility and Expandability Experiment-as-a-Service
26
RISE SICS North ICE photos by Per Bäckström 27
28 Energy savings in data centers: A framework for modelling and control of servers cooling Riccardo Lucchese Jesper Olsson Anna-Lena Ljung Winston Garcia-Gabin Damiano Varagnolo ABB Corporate Research, Västerås, Sweden Luleå University of Technology, Sweden