A New Perspective on Energy Accounting in Multi-Tenant Data Centers Mohammad A. Islam and Shaolei Ren University of California, Riverside
Energy accounting? Measuring energy consumption Why energy accounting? Increasing energy footprint of data centers Pressure from government and environmental groups Tax benefits and positive social image 2
Multi-tenant data center Tenant Server Tenant Server Tenant Server Power, Cooling, Network Access 3
Multi-tenant data center Hyper-scale (e.g. google): 7.8% Multi-tenant: 37% Enterprise: 53% Percentage of total data center industry electricity usage 4
Clicking Clean Multi-tenant data centers have been included since 2014 5
Energy consumption in multi-tenant data centers 6
Energy consumption Utility UPS PDU Tenants Server Racks PDU Cooling non-it Energy IT Energy 7
Energy consumption Utility Metered at system UPS level PDU PDU Cooling Tenants Server Racks Metered for each tenant non-it Energy IT Energy 8
Energy accounting Energy consumption of non-it unit j, supporting a subset of tenants N j N P sj = F j P Ti = Φ ij i N j i N j P Ti is IT power, Φ ij is non-it energy share F j ( ) is energy function for non-it equipment S 1 S 2 S j S M Non-IT units T 1 T 2 T i T N Tenants 9
Energy accounting problem Tenant i supported by a subset of non-it unit M j M Tenant i s total non-it energy consumption is Φ i = i M j Φ ij Total energy accounted for tenant i P Ti + Φ i How to decide? Already metered 10
UPS Efficiency UPS Loss Accounting non-it energy is not trivial 100% 90% Non-linear energy function F j ( ) Data center operates mostly at non-linear region High PUE (~1.4) in even state-of-the-art facilities 9% 6% Double Conversion Delta Conversion Normal operating zone 80% Double Conversion 70% Delta Conversion 0% 25% 50% 75% 100% UPS Load 3% 0% 0% 25% 50% 75% 100% UPS Load 11
How to fairly distribute non-it energy among tenants? 12
Fairness in accounting policy Distributing shared cost/payoff Classic economics problem Fairness axioms Efficiency Symmetry Null player Additivity 13
Fairness axioms Efficiency: Sum of all tenants non-it energy shares is equal to the data center level total non-it energy i N j Φ ij = P Sj, j M Φ 1j Φ 2j Φ 3j P Sj 14
Fairness axioms Symmetry: If two tenants are interchangeable and indistinguishable in non-it energy, they are accounted for the same amount If F j l X i P Tl = F j l X k P Tl, Φ ij = Φ kj 15
Fairness axioms Null-player: Tenants that have no impact on non-it energy, share zero non-it energy If F j l X i P Tl = F j l X P Tl, Φ ij = 0 0 16
Fairness axioms Additivity: A tenant s non-it energy is equal to sum of that in sub-accounting periods Time 17
Existing policies Policy #1: Average energy usage based Φ 1j = P Sj P T 1 P T 1 +P T2 +P T3 + Policy #2: Instantaneous energy usage based Φ 1j = 1 T t P Sj (t) P T 1 (t) P T 1 t +P T2 t +P T3 Policy #3: Power subscription based Φ 1j = P Sj T 1 T 1 +T 2 +T 3 + Policy #4: Equally allocated to all Φ 1j = 1 N j P Sj t + 18
Non-IT Energy (kwh) Checking the existing policies 1200 800 400 0 Only T#2 Only T#3 T#1 & T#2 T#1 & T#3 We should account equal share of non-it energy to T#2 and T#3, i.e., Φ T#2 = Φ T#3 19
Non-IT Energy (kwh) Checking the existing policies 400 T#2 T#3 350 300 250 Policy #1 Policy #2 Policy #3 20
How to find a fair accounting policy for multi-tenant data centers? 21
The fair policy Shapley Value proposed in 1953 by Lloyd Shapley Uniquely satisfies the fairness axioms Accounts for the marginal contribution 22
Shapley value Φ ij = X N j {i} X! N j X 1! N j! F j P X + P i F j (P X ) 23
Shapley value explained Φ ij = X N j {i} X! N j X 1! N j! F j P X + P i F j (P X ) Marginal Contribution P i KWh P 1 P 2 F j (P 1 + P 2 ) + P i ) 24
Shapley value explained Φ i = X N {i} X! N X 1! N! F P X + P i F(P X ) X = 0 X = 1 X = 2 1 N 25
Shapley value is fair in theory, but difficult to apply in practice 26
Challenges of Shapley value Unobservable states Φ i = X N {i} X! N X 1! N! F P X + P i F(P X ) Requires evaluation of non-it energy for a subset of tenants 27
Challenges of Shapley value Unobservable states Φ i = X N {i} X! N X 1! N! F P X + P i F(P X ) Requires evaluation of non-it energy for a subset of tenants 28
Challenges of Shapley value Exponential complexity Φ i = X N {i} X! N X 1! N! F P X + P i F(P X ) 2 N possible combinations for one tenant 29
Challenges of Shapley value Exponential complexity Φ i = X N {i} X! N X 1! N! F P X + P i F(P X ) 2 N possible combinations for one tenant 30
UPS Loss QSEA Quick Shapley value based Energy Accounting A novel quick and fair energy accounting method Key idea: Quadratic approximation of non-it units energy functions F j x = a j x 2 + b j x + c j 9% 6% Double Conversion Delta Conversion y = 0.03455x 2 + 0.00959x + 0.03234 3% 0% 0% 25% 50% 75% 100% UPS Load y = 0.01081x 2 + 0.0064x + 0.01333 31
QSEA F j x = a j x 2 + b j x + c j Φ ij = X N j {i} X! N j X 1! N j! F j P X + P i F j (P X ) Φ ij = 2a jp Ti n j! X N j {i} r X! n j r x 1! P X + a j P 2 Ti + b j P Ti + c j n j where n j = N j, r X = X, and P X = k X P Tk. Each tenant s power P Ti appears Using this we get, X N j {i} n j 2 u 1 r X! n j r x 1! P X times in the subsets X of size u. = n j! 2 k Nj {i} P Tk 32
QSEA Φ i,j = P Ti a j k N j P Tk + b j + c j N j Unobservable states Model based evaluation Exponential complexity Closed form Shapley value 33
QSEA explained Φ i,j = P Ti a j k N j P Tk + b j + c j N j Divide dynamic energy proportional to IT energy Equally divide the idle energy among active tenants 34
QSEA An efficient Shapley value based fair policy 35
Evaluation of QSEA 36
Setup Configuration 12 tenants with capacity ranging from 25KW to 150 KW 4 PDUs and 2 UPSs running in 2N redundancy Cooling two different cases Chiller cooling Outside air cooling Data center simulator with different components Energy functions F j are learned for all non-it units 37
Non-IT Energy (%) Non-IT Energy (%) Deviation (%) Results 30 20 QSEA differs from current policies QSEA Policy #1 Policy #2 Policy #3 30 20 Close to SEA-Original, deviation <2% QSEA SEA-Original Difference 6 4 10 10 2 0 0 0 Large Medium Small #1 #3 #5 #7 #9 #11 Tenant Tenants QSEA favors large tenants Small deviation from original Shapley value based accounting 38
Time (ms) Results 1E+7 1E+4 SEA-Original QSEA 1E+1 1E-2 Much faster than original Shapley value For 15 tenants, QSEA takes ~0.15ms, whereas SEA- Original takes ~50seconds 3 10 5 times faster! 3 8 13 18 Number of Tenants 39
QSEA A novel energy accounting policy for multi-tenant data centers Fair, Efficient & Accurate 40