Combinatorial Auction-Based Allocation of Virtual Machine Instances in Clouds

Transcription

1 Combinatorial Auction-Based Allocation of Virtual Machine Instances in Clouds Sharrukh Zaman and Daniel Grosu Department of Computer Science Wayne State University Detroit, Michigan 48202, USA CloudCom 2010 December 01, 2010

2 Outline 1 Motivation 2 Virtual Machine Allocation Problem 3 CA-LP: A Linear Programming-Based Mechanism 4 CA-GREEDY: A Greedy-Based Mechanism 5 A Fixed-Price Mechanism 6 Experimental Results 7 Conclusion

3 Outline 1 Motivation 2 Virtual Machine Allocation Problem 3 CA-LP: A Linear Programming-Based Mechanism 4 CA-GREEDY: A Greedy-Based Mechanism 5 A Fixed-Price Mechanism 6 Experimental Results 7 Conclusion

4 Fixed-price Mechanisms (e.g., Windows Azure) Pay fixed price for resource usage Not economically efficient Cannot guarantee that the user who values a bundle of VM instances the most, gets it Cannot guarantee optimal revenue What if some users want to pay more? Or less? Cannot distribute loads evenly over time You pay the same price at 9am and at 2am!

5 Auctions in Clouds Example: Amazon EC2 Spot Instances Employed to sell unused resources All winning bidders pay the same (per unit) price Not a combinatorial auction

6 Combinatorial Auctions in Practice Combinatorial auctions: Auctions where users bid for bundles of items and bundles are considered as units of allocations. Enable bidders to express their valuations in a more meaningful way Proved more efficient where bundles are involved Successful implementations: selling wireless spectrum, transportation procurement

7 Different types of resources (virtual machines) available Users request for combination (bundle) of VM instances Bidding for bundle is more realistic Available combinatorial auction mechanisms do not apply directly to VM instance allocation problem Our goal: Design suitable combinatorial auction-based mechanisms for allocating VM instances in clouds

8 Outline 1 Motivation 2 Virtual Machine Allocation Problem 3 CA-LP: A Linear Programming-Based Mechanism 4 CA-GREEDY: A Greedy-Based Mechanism 5 A Fixed-Price Mechanism 6 Experimental Results 7 Conclusion

9 Model Resources to allocate m virtual machine types, VM 1,...,VM m k instances of each VM type w 1,...,w m : computing power of VMs (relative) Assumption: w 1 = 1, w 1... w m

10 Model (Cont.) Auction participants n users, u 1,...,u n User u j bids B j = (r j 1,...,rj m,v j ) r j i : Number of VM i instances requested by u j v j : user u j s valuation for the bundle (r j 1,...,rj m) Maximum willingness to pay for the bundle Value she gets if the bundle is allocated

11 Bidders Assumption: The bidders are single-minded. Valuation function of a single-minded bidder for any bundle S: v(s) = { vj if S j S 0 otherwise S j = bundle requested v j = valuation of S j

12 Virtual Machine Allocation Problem (VMAP) Determine the set of winning users and the prices they need to pay W: Set of winners u j W means user u j gets her requested bundle p = (p 1,...,p n ): Price each user pays to auctioneer 0 p j v j, if u j W p j = 0, if u j / W

13 Characteristics of a Good Solution Efficient: Maximize social welfare max u j W v j Truthful: Reporting true value maximizes user s utility

14 Characteristics of a Good Solution Efficient: Maximize social welfare max u j W v j Truthful: Reporting true value maximizes user s utility Utility: Net value obtained { vj p U j (W,p j ) = j, if u j W 0, otherwise

15 Challenges Winner determination in CA is NP-hard Not all approximate solutions guarantee truthfulness Existing CA mechanisms cannot solve VMAP directly Need to extend existing mechanisms or design new mechanisms for solving VMAP

16 Outline 1 Motivation 2 Virtual Machine Allocation Problem 3 CA-LP: A Linear Programming-Based Mechanism 4 CA-GREEDY: A Greedy-Based Mechanism 5 A Fixed-Price Mechanism 6 Experimental Results 7 Conclusion

17 CA-LP Mechanism A linear programming-based approximation mechanism CA-LP: generalizes the mechanism proposed in [Archer et al 2005] Archer et al mechanism: bidders can request at most one copy of each item CA-LP: bidders can request multiple copies of each item Truthful in expectation (as the original mechanism) Reference A. Archer, C. Papadimitriou, K. Talwar, and E. Tardos, An approximate truthful mechanism for combinatorial auctions with single parameter agents, Internet Mathematics, vol. 1, no. 2, pp , 2005.

18 CA-LP Mechanism: Step 1 Collect bids B j = (r j 1,...,rj m,v j ) from users j r j 1 r j 2 v j x j y j Winner Price Y Y N N N N 0.00 CA-LP Example Setting: two types of VM instances (m = 2), six users (n = 6), and eight instances of each type (k = 8)

19 CA-LP Mechanism: Step 1 Collect bids B j = (r j 1,...,rj m,v j ) from users j r j 1 r j 2 v j x j y j Winner Price Y Y N N N N 0.00 CA-LP Example Setting: two types of VM instances (m = 2), six users (n = 6), and eight instances of each type (k = 8)

20 CA-LP Mechanism: Step 1 Collect bids B j = (r j 1,...,rj m,v j ) from users j r j 1 r j 2 v j x j y j Winner Price Y Y N N N N 0.00 CA-LP Example Setting: two types of VM instances (m = 2), six users (n = 6), and eight instances of each type (k = 8)

21 CA-LP Mechanism: Step 2 Determine winners Solve linear program subject to n max x j v j j=1 n j=1 x j r j i k i = 1,...m 0 x j 1 j = 1,...n where k = (1 ǫ)k

22 CA-LP Mechanism: Step 2 Determine winners Solve linear program Sort users by descending order of x j j r j 1 r j 2 v j x j y j Winner Price Y Y N N N N 0.00 CA-LP Example Setting: two types of VM instances (m = 2), six users (n = 6), and eight instances of each type (k = 8)

23 CA-LP Mechanism: Step 2 Determine winners Solve linear program Sort users by descending order of x j Generate random numbers, y j j r j 1 r j 2 v j x j y j Winner Price Y Y N N N N 0.00 CA-LP Example Setting: two types of VM instances (m = 2), six users (n = 6), and eight instances of each type (k = 8)

24 CA-LP Mechanism: Step 2 Determine winners Solve linear program Sort users by descending order of x j Generate random numbers, y j u j is a winner if x j y j and resources exist j r j 1 r j 2 v j x j y j Winner Price Y Y N N N N 0.00 CA-LP Example Setting: two types of VM instances (m = 2), six users (n = 6), and eight instances of each type (k = 8)

25 CA-LP Mechanism: Step 2 Determine winners Solve linear program Sort users by descending order of x j Generate random numbers, y j u j is a winner if x j y j and resources exist j r j 1 r j 2 v j x j y j Winner Price Y Y N N N N 0.00 CA-LP Example Setting: two types of VM instances (m = 2), six users (n = 6), and eight instances of each type (k = 8)

26 CA-LP Mechanism: Step 3 Calculate payments Find v j for which x j < y j p j = v j if u j W, 0 otherwise j r j 1 r j 2 v j x j y j Winner Price Y Y N N N N 0.00 CA-LP Example Setting: two types of VM instances (m = 2), six users (n = 6), and eight instances of each type (k = 8)

27 Outline 1 Motivation 2 Virtual Machine Allocation Problem 3 CA-LP: A Linear Programming-Based Mechanism 4 CA-GREEDY: A Greedy-Based Mechanism 5 A Fixed-Price Mechanism 6 Experimental Results 7 Conclusion

28 CA-GREEDY Mechanism A greedy-based approximation mechanism CA-GREEDY: generalizes the mechanism proposed in [Lehman et al 2002] CA-GREEDY: considers the weighted total of number of VM instances when computing the allocation Truthful Computes a M-approximate solution to VMAP, where M = k m i=1 w i Reference D. Lehmann, L. I. OCallaghan, and Y. Shoham, Truth revelation in approximately efficient combinatorial auctions, Journal of the ACM, vol. 49, no. 5, pp , 2002.

29 CA-GREEDY Mechanism: Step 1 Collect bids B j = (r j 1,...,rj m,v j ) from users j r j 1 r j 2 v j sj v j / s j Winner Price N Y Y N N N 0.00 CA-GREEDY Example Setting: two types of VM instances (m = 2) with one and two processors (w = (1,2)), six users (n = 6), and eight instances of each VM type (k = 8)

30 CA-GREEDY Mechanism: Step 2 Determine winners Weighted sum of VM of each bid: s j = m Bid density: v j / s j i=1 rj i w i j r j 1 r j 2 v j sj v j / s j Winner Price N Y Y N N N 0.00 CA-GREEDY Example Setting: two types of VM instances (m = 2) with one and two processors (w = (1,2)), six users (n = 6), and eight instances of each VM type (k = 8)

31 CA-GREEDY Mechanism: Step 2 Determine winners Weighted sum of VM of each bid: s j = m i=1 rj i w i Bid density: v j / s j Sort bids by density, allocate highest to lowest, until resources exhaust j r j 1 r j 2 v j sj v j / s j Winner Price Y Y N N N N 0.00 CA-GREEDY Example Setting: two types of VM instances (m = 2) with one and two processors (w = (1,2)), six users (n = 6), and eight instances of each VM type (k = 8)

32 CA-GREEDY Mechanism: Step 3 Calculate payments Find u j that would win if u j would not participate p j = s j (v j / s j ), if u j W p j = 0 otherwise j r j 1 r j 2 v j sj v j / s j Winner Price Y Y N N N N 0.00 CA-GREEDY Example Setting: two types of VM instances (m = 2) with one and two processors (w = (1,2)), six users (n = 6), and eight instances of each VM type (k = 8)

33 Outline 1 Motivation 2 Virtual Machine Allocation Problem 3 CA-LP: A Linear Programming-Based Mechanism 4 CA-GREEDY: A Greedy-Based Mechanism 5 A Fixed-Price Mechanism 6 Experimental Results 7 Conclusion

34 Fixed-Price Mechanism Collect bids B j = (r j 1,...,rj m,v j ) from users Allocate bundles (f 1,...,f m ): Fixed prices for VM 1,...,VM m Allocate bundle, on a first-come-first-served basis if valuation meets predetermined fixed price v j m i=1 r j i f i Calculate payments User pays the predetermined fixed price if she gets the bundle p j = m i=1 r j i f i

35 Outline 1 Motivation 2 Virtual Machine Allocation Problem 3 CA-LP: A Linear Programming-Based Mechanism 4 CA-GREEDY: A Greedy-Based Mechanism 5 A Fixed-Price Mechanism 6 Experimental Results 7 Conclusion

36 Simulation Parameters Parameter Description / Values Total time Five simulation days Auction frequency Every hour Users Maximum of 100,000 users VM weights w = (1,2,4,8) VM prices f = 0.12,0.24,0.48,0.96 User types Type 1, 2, 3: high, medium, low valuation Bidding A user needs to win t j times to complete job Simulation Parameters

37 Comparison of Overall Performance User and System Parameters (log 10 scale) FIXED-PRICE Served Revenue 5.53% 8.01% 5.60% CA-LP CA-GREEDY 116, , ,098 Utilization 54.63% 94.70% 91.86% Time 1 sec 26 sec 837 sec Overall performance of the mechanisms

38 Types of Users Completing Jobs 12 Users Completing Tasks FIXED-PRICE CA-LP CA-GREEDY 10 8 Percent Served Type 1 Type 2 Type 3 User Type Percentage of served users

39 Varying Valuation Ranges Revenue vs. Valuation Ranges FIXED-PRICE CA-LP CA-GREEDY Revenue Range of Valuations (min-max) Generated revenue vs. ranges of valuation

40 Varying Valuation Ranges - 2 Resource Utilization vs. Valuation Ranges 100 FIXED-PRICE CA-LP CA-GREEDY 80 VM Utilization (%) Range of Valuations (min-max) Effect of valuation on machine utilization

41 Outline 1 Motivation 2 Virtual Machine Allocation Problem 3 CA-LP: A Linear Programming-Based Mechanism 4 CA-GREEDY: A Greedy-Based Mechanism 5 A Fixed-Price Mechanism 6 Experimental Results 7 Conclusion

42 Conclusion Combinatorial auction mechanisms are more desirable than fixed-price mechanisms CA-LP has better performance over CA-GREEDY, but poor running time Use CA-GREEDY as a general solution - good running time, comparable performance

43 Thank You! Questions?