EDIC RESEARCH PROPOSAL 1 Designing Negotiation Protocols for Utility Maxiization in Multiagent Resource Allocation Tri Kurniawan Wijaya LSIR, I&C, EPFL Abstract Resource allocation is one of the ain concerns in econoics and coputer science. We focus on the resource allocation to several agents or consuers, especially on how we can find such allocation such that agent s utility is axiized. We start with the introduction of ultiagent resource allocation. Then, we continue with an approach to deal with two or ore utility functions in a nonlinear way. Finally, we present the application of MARA in the sart grid doain, where each agent optiizes her own electricity consuption scheduling to reduce her bill results in iniizing overall syste cost. Index Ters ultiagent systes, negotiation, auction, sart grid I. INTRODUCTION MULTIAGENT Resource Allocation MARA is the process of distributing a nuber of ites aongst a nuber of agents [1]. Recently, MARA has been applied in any applications. As an exaple, in Section III, we describe an application of MARA in the sart grid, where we discuss about distributing resources electricity load to agents consuers in order to iniize cost. There are several key concepts in MARA: resources: ites that are being distributed, Proposal subitted to coittee: July 5th, 2012; Candidacy exa date: July 12th, 2012; Candidacy exa coittee: Boi Faltings, Karl Aberer, Jean-Yves Le Boudec. This research plan has been approved: Date: Doctoral candidate: nae and signature Thesis director: nae and signature Thesis co-director: if applicable nae and signature Doct. prog. director: R. Urbanke signature EDIC-ru/05.05.2009 agents: entities who receive the resources, allocation: a distribution of resources aong agents, agent preferences/valuations: how uch an agent gives a value to a resources, In addition, there are two other iportant notions, naely social welfare and allocation procedure, that we describe in the following subsections. A. Social Welfare In MARA we ai to achieve an optial allocation, or at least nearly-optial when finding the optial one is considered as coputationally expensive. The notion of optial allocation is easured based on agents utility on the allocation. In this case, social welfare is a odel to aggregate agents utility. Let us denote the set of agents as N. Pareto optiality. Each agent i N has a preference relation i. Let P and Q be allocations. Then, P i Q eans that agent i prefers allocation P no ore than Q, i.e. agent i s utility for allocation P is less than or equal to her utility for allocation Q. Whereas, P i Q eans that P i Q but not Q i P An allocation P is Pareto-doinated by another allocation Q if and only if: i P i Q for all agents i N, and ii P i Q for at least one agent i N. An allocation is pareto optial or pareto efficient if and only if it is not pareto-doinated by any other allocation. Utilitarian social welfare. Let u i be the utility function of agent i for an allocation. Assue that we have two allocations, P and Q, then u i P u i Q eans that agent i likes allocation P no ore than Q. Then, the utilitarian social welfare of the allocation P is defined as the su of all agents utility for P, i.e., sw u P = i N u i P. When we axiize the utilitarian social welfare, it eans that we axiize the total utility of all agents with respect to the allocation. Egalitarian social welfare. Using the definition of agent s utility as in the utilitarian social welfare above, the egalitarian social welfare of the allocation P is defined as: sw e P = in{u i P i N }. When we axiize the egalitarian social welfare, it eans that we ai to axiize the lowest utility of any agent i N with respect to the allocation.
EDIC RESEARCH PROPOSAL 2 B. Allocation Procedures An allocation procedure is a way to find an allocation of resources. Centralized vs. Distributed. In ost of the cases, centralized way is preferred because of its siplicity in the counication protocols and hence, often regarded as sipler echanis. However, soeties it is difficult to find an entity could be a third party, or one of the agents which is able to act as the ediator, because of its liitation in the coputational power since soe centralized allocation algorith could be coputationally hard or lack of trustworthiness. While for distributed approach, the allocation algorith could be ore feasible. Auction. In the auction, agents report their preference/valuation of the resource. This process is called as bidding, while the agents are called as the bidders. The winner of the auction, i.e. the agent who gets the resource is deterined by the auctioneer. Bidding ay be open publicly such as in English or Dutch auction or private such as in Vickrey auction. We refer the reader to [3] for ore extensive text on auctions. Negotiation Protocol Contract-Net [5]. This protocol was priarily designed for task allocation, however, it is also suited for MARA. The protocol consists of two roles anager and bidder and four phases: i Announceent phase: the anager counicates the resource available to the agents the bidders. ii Bidding phase: the bidders subit their proposal to the anager. iii Assignent phase: the anager elects the best bid and assigns the resource. iv Confiration phase: the elected bidder confirs whether she wants to obtain the resource or not. This protocol is siilar to auction, except that in Contract- Net, there is a confiration phase for the elected bidder or the winner in the auction to confir whether she wants to obtain the resource. Instead of confiration, the last phase in the Contract-Net can also be seen as an opportunity for the elected bidder to cancel her bid. Most of the works in the negotiation in ultiagent environents consider linear utility function for each agent. However, soeties nonlinear utility representation is necessary. This challenge is discussed in the next section. Then, in Section III we discuss about an application of MARA in the sart grid doain. Finally, we conclude with suary and research plan. II. NEGOTIATION PROTOCOL WITH NONLINEAR UTILITY SPACES In this section, we consider a ulti-issue negotiation protocol with nonlinear utility spaces [2]. In reality, negotiations often involve interdependent and ultiple issues. Consider an exaple when two designers design a new car, then there are ultiple issues to be discussed, e.g. engine, color, tire, etc. Each designer has her own preferences over these issues, and each issue has her own constraint. For exaple, if the tire is large, and the body style is R.V., then the size of the Fig. 1: A constraint which has value 55 for issue 1 between 3 and 7, and issue 2 between 4 and 6 [2]. engine needs to be larger than 2,500 cc. This interdependency between issue akes linear representation of agent s utility often not expressive enough. The work in [2] addressed this challenge by proposing a negotiation protocol with nonlinear utility spaces. In addition, because a constraint between tire and engine could be different between copany, the work in [2] also enables each agent to express her own constraints for each issue. Notation. Let us denote N to be the set of agents and S = {s 1,..., s } be the set of issues to be negotiated. The nuber of issues, S, represents the diensions of the utility spaces. For each issue s S, we have s [0, X], where [0, X] is an integer doain fro 0 to X. A contract is a vector of issue values, i.e. s = [s 1,... s ]. Let C be a set of constraints. A constraint c k C is a hyper diensional solid aong ultiple issues having value w i c k, s for agent i N if it is satisfied by contract s. Figure 1 shows an exaple of a constraint over issue 1 and 2, which has value 55 if issue 1 is in [3,7] and issue 2 is in [4,6]. A utility for agent i for a contract s is defined as: u i s = w i c k, s c k C, s xc k where xc k is a set of possible contracts solutions of c k. Bid generation. Before a negotiation to reach a deal, each agent akes bids, where each bid contains a contract point: i Sapling: each agent saples her utility space. ii Adjusting: for each point sapled, each agent adjusts her sapling point in order to find local optiu. This can be done using nonlinear optiizer, such as siulated annealing. Figure 2 gives an illustration. iii Bidding: for each adjusted contract point, an agent calculate her utility. If the utility is larger than a threshold δ, then she defines a bid which consists of a set of constraints containing the contract. The bid value is the su of the value of constraints included in the bid. Deal identification. A ediator identifies the final contract by finding cobinations of bids that share at least soe contract
EDIC RESEARCH PROPOSAL 3 described. Second, a distributed algorith executed by each consuer to iniize overall syste cost is presented. A. Syste Modeling Fig. 2: Each agent adjusts each of her saple point in order to find local optiu [2]. Load Modeling. Let us denote N as the set of consuers or the agents, and H = {1,... H} as an hourly tie slot in a day where H = 24. The daily electricity load of consuer n N is denoted by l n = [l 1 n,..., l H n ]. Then, we can define the total load over all consuers at a tie slot h H as L h = n N l h n. The peak load of a day can be coputed as L peak = ax h H L h, and the average load of a day is L avg = h H L h. 1 H Energy cost odel. considered: There are two assuptions that are i the cost functions are increasing, i.e. for each h H C h L h < C h L h, L h < L h ii the cost functions are strictly convex, i.e. for each h H C h θl h + 1 θl h < θc h L h + 1 θc h L h where θ R and 0 < θ < 1. An exaple of an actual energy cost function which satisfies both assuptions is the quadratic cost function: C h L h = a h L h 2 + b h L h + c h [6]. Energy consuption scheduling. Let A n denote a set of electronic appliances owned by consuer n N. For each appliance a A n, we denote its load consuption for each tie slot of the day as: x n,a = [x 1 n,a,... x H n,a] Fig. 3: An exaple of deal identification process. The best contract point which axiizes the utility is chosen [2]. points consistent and then she chooses a contract point which axiizes the total value of the bids axiization. When the ediator cannot find any bids that share contract points, especially when the sapling size is sall or the threshold is high, the negotiation fails to achieve an agreeent. Figure 3 shows an exaple of a deal identification process. III. DISTRIBUTED ELECTRICITY LOAD ALLOCATION IN THE SMART GRID In this section, we discuss about a work in deand side anageent, based on [4]. In this work, we do not ai to reduce the aount of the electricity consued by consuers. Instead, we ai to reduce her and eventually overall energy cost by adjusting her consuption schedule. In this case we regard consuer s cost as her utility. The less the cost she has to pay, the higher her utility is. First, the syste is briefly where x h n,a is the energy consuption of consuer n for appliance a at tie slot h H. Then, we have for each h H: ln h = 1 a A n x h n,a Constraints. In addition, we define the total energy requireent of consuer n N for appliance a A n in a day as E n,a. In order to capture consuer tie preferences to turn on an appliance, we define α n,a as the consuer n preferred starting tie for appliance a and β n,a as the ending tie, where α n,a < β n,a. Furtherore, we define a feasible tie range H na = {α n,a,..., β n,a }. Then, it is required that: = E n,a 2 and h H n,a x h n,a x h n,a = 0, h H \ H n,a 3 For appliance s energy constraint, we define γn,a in and γn,a ax as the iniu standby and axiu energy level of
EDIC RESEARCH PROPOSAL 4 appliance a A n respectively. Hence, it is required that for all h H n,a, γn,a in x h n,a γn,a ax 4 Energy cost iniization. Let us denote x n as the vector of energy consuption x n,a of consuer n N for all appliances a A n. Given the price function, we want to iniize the energy cost: in C h x h n,a x n X n, n N h H n N a A n where X n is the set of all possible energy consuption schedule x n satisfying Eq. 2, 3, and 4. Consuer s bill. For consuer n N we denote her daily electricity bill as b n. We require that: b n C h ln h 5 n N h H n N which eans that the total bill over all consuer should be greater or equal the total energy cost. In order to get the profit ratio ade by the copany we can also rewrite Eq. 5 as: n N κ = b n 1 6 h H C h n N lh n where κ is the profit ratio ade by the energy copany. It is clear that if κ = 1 then it eans that consuer s total bill equal to the total energy cost, and when κ > 1 it eans that the energy copany ade profit. In addition, in order to be fair, we require that the larger electricity load consued, the higher the bill should be, i.e. for consuer n, N : b n b = h H lh In order to analyze the bill ade by a particular consuer with respect to all other consuers, we can also rewrite Eq. 7 as: b = b n b = b n b n = h H lh h H lh h H lh b Let us denote L n as the total load of consuer n N in a day, i.e. L n =. Then we can rewrite Eq. 8 as: L n b n = L b 9 which shows that the bill consuer n has to pay is proportional to her load with respect to the total load and the total bill of all consuers. 7 8 B. Distributed Optiization In order to get the details on how the cost function relate to the consuer bill, we can use Eq. 6, and rewrite Eq. 9 above as: b n = = κ L n L C h h H κ L n L C h ln h + h H l h \n l h 10 Then, for connecting consuer s bill with her detail consuption schedule for each appliance x h n,a, we can use Eq. 1 to rewrite the ter l h n in Eq. 10: κ L n b n = L C h x h n,a + l h h H a A n \n 11 Fro Eq. 11, we can see that in order to iniize her bill b n, consuer n can change her consuption schedule the ter x h n,a in Eq. 11. In other words, consuer n should find consuption scheduling vector x n which iniize b n in Eq. 11. Indeed, this is the case, as has been shown in [4]. In addition, to find a consuption schedule which iniize her bill, the only inforation consuer n needs to know is the total consuption over all consuers for each tie slot h H. This is another advantage fro privacy point of view since a consuer do not need to reveal her detail appliance consuption schedule to the other consuers. Algorith 1 shows the procedure on how the consuer n N interact with the syste and update her consuption schedule in order to iniize her bill. In the beginning, the initial load l n and l n is deterined by rando since it is unknown where l n is the load vector for the set of consuers N \ n. Then, in the beginning tie of the day, the schedule optiization is perfored line 3 and 4. However, since this can also be seen as a day-ahead optiization, the optiization can also be done at any given tie point of the day to deterine toorrow s schedule. When there is an update on the schedule, the consuer update her schedule and announces it to the syste line 5-7. And vice versa, when there are soe external updates fro other consuers an update essage is received and we update l n line 9-10. This procedure continue until there is no update in the syste line 12. It has been shown in [4] that when each consuer does this her own optiization, eventually overall energy cost is iniized. However, this requires consuers to update their consuption schedule not in the sae tie. Otherwise, there will be an oscillation between the cheap or expensive tie slot. In this case, the solution proposed in [4] is that each consuer should take turn to update her schedule. The turn allocation is assued to be controlled by the energy copany. IV. SUMMARY First, we introduced ultiagent resource allocation. In particular we focused on the notion of social welfare and allocation procedure. Second, we presented a negotiation protocol to
EDIC RESEARCH PROPOSAL 5 Algorith 1: Schedule optiization - for consuer n 1 Randoly initialize l n and l n 2 repeat 3 At the beginning of the day, 4 find x n which iniize Eq. 11 5 if x n is different fro the current schedule x n then 6 update the schedule, x n x n 7 broadcast l n to the syste 8 end 9 if update essage is received then 10 update l n accordingly 11 end 12 until no consuer announces new schedule deal with nonlinear utility spaces. We regarded this approach as a unique one, since ost of the approaches in the literature cobined two or ore utility functions in linear fashion. And finally, we discussed an application exaple of ultiagent resource allocation in the sart grid doain. A pricing echanis, which depends on the aount of load consued at a particular hour, is used to encourage consuer to adjust their consuption schedule such that her and overall energy cost is iniized. V. RESEARCH PLAN First, I the author plan to consider the notion of consuer convenience and study how it can play a role in the work by [4]. Then, we will have two utilities, naely consuer energy cost and consuer convenience. In this case, I also would like to consider the approach by [2], especially on their deal identification algorith. However, since the deal identification in [2] is a centralized approach, we ight have to drop the distributed properties of algorith 1. Second, the work in [2] is particularly interesting in how they deal with nonlinear utility spaces. However, their negotiation protocol can be better in ters of finding an agreeent between agents. In [2], for a bid, each agent considers only contract points in their local optia which has value greater than a threshold. Consider a case when an agreeent cannot be achieved, but actually there could be an agreeent if soe agents subit a bid which is not their local optia but still have values greater than the threshold. When we regard an agreeent is better than no agreeent at all, then we know fro the case above that we can iprove the negotiation protocol. The idea is to put all contract points which have values ore than the threshold into the bids. Then, we let the deal identification process possibly with soe odification deterine the agreeent which axiizes agents utility. The side effect of the proposed idea is that the deal identification now has to consider uch ore bids than the current version. Third, the solution proposed in [4] to take turn in order to avoid all consuers update their schedule in the sae tie can take a long tie before it has an effect to the overall energy cost when we have a large nuber of consuers. Hence, I propose to assign a so called participation rate to each consuer, which is a probability whether a consuer should update her schedule or not for that particular day. Another advantage of this idea is that we require no centralized control for the optiization since taking turn echanis is no longer needed. Fourth, in the sart grid, having such a high peak load is undesirable since this high load is a potential cause for a power failure and incur an expensive generation cost. Inline with the work in [4], I would like to analyze and understand consuer s consuption profile in order to give a highly accurate suggestion to consuers on how they can shift their consuption schedule such that the peak load and overall energy cost is reduced. REFERENCES [1] Y. Chevaleyre, P. E. Dunne, U. Endriss, J. Lang, M. Leatre, N. Maudet, J. Padget, S. Phelps, J. A. Rodrguez-aguilar, and P. Sousa, Issues in ultiagent resource allocation, Inforatica, vol. 30, p. 2006, 2006. [2] T. Ito, H. Hattori, and M. Klein, Multi-issue negotiation protocol for agents: exploring nonlinear utility spaces, in Proceedings of the 20th international joint conference on Artifical intelligence, ser. IJCAI 07. San Francisco, CA, USA: Morgan Kaufann Publishers Inc., 2007, pp. 1347 1352. [3] V. Krishna, Auction Theory. Acadeic Press, Mar. 2002. [4] A. Mohsenian-Rad, V. W. S. Wong, J. Jatskevich, R. Schober, and A. Leon-Garcia, Autonoous Deand-Side Manageent Based on Gae-Theoretic Energy Consuption Scheduling for the Future Sart Grid, IEEE Transactions on Sart Grid, vol. 1, no. 3, pp. 320 331, Dec. 2010. [5] R. G. Sith, Distributed artificial intelligence, A. H. Bond and L. Gasser, Eds. San Francisco, CA, USA: Morgan Kaufann Publishers Inc., 1988, ch. The contract net protocol: high-level counication and control in a distributed proble solver, pp. 357 366. [6] A. J. Wood and B. F. Wollenberg, Power Generation, Operation, and Control. Wiley-Interscience, January 1996.