Convergence Rate of Best Response Dynamics in Scheduling Games with Conflicting Congestion Effects

Transcription

1 Convergence Rate of est Response Dynamics in Scheduling Games with Conflicting Congestion Effects Michal Feldman Tami Tamir Abstract We study resource allocation games with conflicting congestion effects. In such games, selfinterested players choose resources with the objective of minimizing their individual cost. A a job s cost consists of two components: its resource s load (which increases with congestion) and its share in the resource s activation cost (which decreases with congestion). In [4], we provide results for a setting of heterogeneous jobs and identical machines with respect to equilibrium existence and the inefficiency introduced due to self-interested behavior. We also showed that for jobs with arbitrary lengths, best-response dynamics (RD) are not guaranteed to converge to a Nash equilibrium, while for unit length jobs, any RD must converge. In this work we study the convergence rate of RD for instances with unit length jobs. RD dynamics are a large family of dynamics, differing from each other in the order in which players apply their best-response moves. We show that the convergence of RD in general might take a super-linear number of moves, and we propose a specific form of RD that ensures convergence within a linear number of moves. 1 Introduction In resource allocation applications, tasks are assigned to resources to be performed. For example, in job scheduling models, jobs are assigned to servers to be processed, and in network routing models, traffic is assigned to network links to be routed. In the last decade, algorithmic game theory has introduced game theoretic considerations to many of these problems [10, 8, 1]. At the heart of the game theoretic view is the assumption that the players have strategic considerations and act to minimize their own cost, rather than optimizing the global objective. In resource allocation settings, this would mean that the jobs choose a resource instead of being assigned to one by a central designer. The literature is divided into two main approaches with respect to the cost function. The first class of models emphasizes the negative congestion effect, and assumes that the cost of a resource is some non-decreasing function of its load. Job scheduling [6, 12] and selfish routing [5, 11] belong to this class of models. The second class assumes that each resource has some activation cost, which should be covered by its users, thus a user wishes to share its resource with additional users in attempt to decrease its share in the activation cost. Roughly speaking, the cost of using a resource in this class is some School of usiness Administration and Center for the Study of Rationality, Hebrew University of Jerusalem. mfeldman@huji.ac.il. School of Computer Science, The Interdisciplinary Center, Herzliya, Israel. tami@idc.ac.il 1

2 decreasing function of its load. Positive congestion effects have been considered in network design games [2, 1]. We claim that in practice both the positive and the negative congestion effect take place. On the one hand, a heavy-loaded resource might be less preferred due to negative congestion effects; on the other hand, resources do have some activation cost, and sharing this cost with other users releases the burden on a single user. Our goal is to combine these two components into a unified cost function. Consequently, the cost function in our model is composed of (i) the load on its resource, and (ii) its share in the activation cost of its chosen resource. An additional assumption we wish to challenge is the existence of an a priori given set of resources. In many practical settings a set of users controlling some jobs have the opportunity to utilize a new resource at their own cost. For example, a user might be able to purchase a dedicated server for his job if he is willing to cover its cost. Consequently, we consider settings in which the number of resources is unlimited a priori. (Obviously, the number of resources will never exceed the number of users.) In our model, each resource is associated with some fixed activation cost, which should be jointly incurred by the set of jobs using it. A crucial question in this setting is how to divide the resource cost among its users. Sharing of joint costs among heterogeneous players is a common problem, and a large number of sharing rules have been proposed for this problem, each associated with different efficiency and fairness properties [9, 7]. Here, our focus is not on the mechanism design point of view. Rather, we analyze two specific sharing rules with respect to equilibrium existence, computation, convergence and quality. The first rule is the uniform sharing rule, under which the resource s cost is shared evenly among its users. The second rule is the proportional sharing rule, under which the resource s cost is shared among its users in proportion to their sizes. Note that under both sharing rules, for a sufficiently small activation cost, the unique NE will be one in which each job is processed by a different resource. In the other extreme, for a sufficiently large activation cost (in a sense that will be formalized below), the unique NE will be one in which all the jobs will be assigned to a single resource. 1.1 Model and Preliminaries We consider a job-scheduling setting with identical machines and unit-length jobs. An instance is given by the number of jobs n, and the activation cost. An assignment method produces an assignment s = (s 1,..., s n ) S of jobs into machines, where s j M denotes the machine that job j is assigned to. The assignment is referred to as a schedule, profile, joint action, or configuration (we use all these terms interchangeably). The load of a machine M i in a schedule s, denoted L i (s), is the number of jobs assigned to it. The makespan of a schedule is the load on the most loaded machine; we denote it L max (s) = max i L i (s). Given a job-scheduling setting and an activation cost, a job-scheduling game is induced, where the set of players corresponds to the set of individual jobs, and the action space S j of each player j is the set of individual machines. The cost function of job j in a given schedule is composed of the load on the job s machine and the job s share in the machine s activation cost; i.e., given a profile s in which s j = M i, the cost of job j is given by: c j (s) = L i (s) + /L j (s), An assignment s S is a pure Nash equilibrium (NE) if no job j N can benefit from unilaterally deviating from its machine to another machine; i.e., for every j N and every s j M it holds that c j (s j, s j ) c j (s). 2

3 2 Convergence of est-response Dynamics est-response Dynamics (RD) is a local search method where in each step some player is chosen and plays its best-response strategy, given the strategies of the others. In systems where agents repeatedly perform improvement steps until they reach a Nash equilibrium, the notion of a pure NE is well justified. In order to better understand the behavior of these dynamics, this section explores the convergence rate of best-response dynamics into a pure NE. We first show that unlike other job-scheduling games, in our model RD do not necessarily converge to a Nash equilibrium. Observation 2.1 Under the proportional sharing rule RD might not converge to a NE. Proof: Consider the instance with four jobs of lengths 10, 10, 10, 20, and = 72. In a machine with two jobs of length 10, each of the jobs incurs a cost of = 56. In a machine with jobs of lengths 10 and 20, the jobs of lengths 10 and 20 cost = 54 and = 78, respectively. In a machine with jobs of lengths 10, 10, 20, each of the jobs of length 10 incurs a cost of = 58 while the long job s cost is = 76. Consider a profile in which two jobs of length 10 are assigned to one machine and jobs of lengths 10, 20 are assigned to a second machine. The long job can reduce its cost by migrating to the other machine. Then, one of the jobs of length 10 assigned to the machine that the long job joined can benefit by migrating to the other machine. That brings the system back to the initial configuration. Thus, RD are not guaranteed to converge. Note that in the last example, RD will not converge to a NE under the given initial state, independent of the order in which the agents respond. In contrast to the general case, in which jobs have arbitrary lengths, if all jobs have the same length, then the induced game is equivalent to a congestion game in which the maximal number of resources is n [?]. W.l.o.g., we assume that p j = 1, i.e., the case of unit-length jobs; the activation cost can be scaled accordingly. In the case of unit-size jobs, one can easily verify that the function Φ(s) = i H l i l2 i, where l i denotes the number of jobs on machine i, H 0 = 0, and H k = 1 + 1/ /k, is a potential function for the game. Convergence to a NE is guaranteed in potential games, but the convergence time might be exponential. RD dynamics are a large family of dynamics, differing from each other in the order in which players apply their best-response moves. In [?] we study the convergence time of RD of uniformlength jobs. We show that the convergence of RD might take Ω(n log n ) moves. We also propose a specific form of RD, in which at every time step, a job that incurs the highest cost is chosen to perform its best-response, and we show that this method ensures convergence within a linear number of moves. 2.1 RD of Uniform-Length Jobs. In contrast to the general case, in which jobs have arbitrary lengths, if all jobs have the same length, then the induced game is equivalent to a congestion game in which the maximal number of resources is n [?]. W.l.o.g., we assume that p j = 1, i.e., the case of unit-length jobs; the activation cost can be scaled accordingly. In the case of unit-size jobs, one can easily verify that the function Φ(s) = i H l i l2 i, where l i denotes the number of jobs on machine i, H 0 = 0, and H k = 1 + 1/ /k, is a potential function for the game. Convergence to a NE is guaranteed in potential games, but the convergence time might be exponential. In this section we study the convergence time of RD of uniform-length jobs. RD dynamics are a large family of dynamics, differing from each other in the order in which players apply their 3

4 best-response moves. We show that the convergence of RD in general might take Ω(n log n ) moves, and propose a specific form of RD that ensures convergence within a linear number of moves. Specifically, at every time step, a job that incurs the highest cost is chosen to perform its best-response (where ties are broken arbitrarily). We refer to this process as Max-Cost-RD (MC-RD hereafter). The analysis of the convergence rate of RD is quite complicated and requires several preparations and terminology. Clearly, with unit-length jobs, all the jobs assigned to a particular machine incur the same cost. We denote the cost of a job that is assigned to a machine with load x by c(x), where c(x) = x+/x. The following simple claim is used throughout the section. Claim 2.1 The function c(x) = x + /x for x 0 gets its minimum at x =, is decreasing for x [0, ) and increasing for x >. Practically, the cost function is applied to integral values only. If is a square, then the load achieving minimal cost is exactly. In general, denote by l = l () {, } the load achieving minimal cost. The cost function gets its minimum at l, is decreasing for x [0, l ], and increasing for x > l. We denote by l t i the load of machine M i at time t, i.e., before the migration of iteration t took place. A machine that has load at least (respectively, smaller than) l is said to be a high (resp., low) machine. The following observation establishes that if at some iteration a job enters a low machine, then in subsequent iterations that machine will attract more migrations up to load at least l. Since c(l + 1) < c(l) for l < l, it follows that if a low machine is a best-response at some iteration, it will continue to be the best-response until it is filled up to load at least l. Formally, Observation 2.2 If at some iteration t there is a migration to a low machine M i such that l t i = l x for some x > 0, then the following x 1 iterations will involve migrations to M i. 1 Properties of MC-RD. y the design of the MC-RD process and as a direct corollary of Claim 2.1, every migration in the MC-RD process is from either the lowest or the highest machine into either the lowest-high or the highest-low machine (see Figure 1). Figure 1: MC-RD process 1 It is possible that the system reaches a NE and the RD process terminates before x 1 iterations are performed. 4

5 Since all jobs on a particular machine share the same cost, the MC-RD process actually selects in iteration t a machine M i for which c(l t i ) is maximal, and moves an arbitrary job from M i to a machine M k for which c(l t k + 1) is minimal. While the MC-RD process does not specify which job is migrating from M i, for simplicity we will assume a LIFO (last in first out) job selection rule. Specifically, the job that entered M i last is the one to migrate. If all jobs on M i were assigned to it in the initial configuration, then an arbitrary job is selected. Since the RD-process can be characterized by the load-vector of the machines in every time step, the number of iterations is independent of the job-selection rule. In particular, our analysis of the convergence rate of MC-RD applied with a LIFO job-selection rule is valid for any MC-RD process. We next observe that if at some iteration a job leaves some low machine, then in the following iterations all the jobs assigned to that machine will leave it one by one until the machine empties out. To see this, note that c(l 1) > c(l) for l < l ; thus if a low machine has the highest cost, it continues to achieve the maximal cost after its load decreases. Observation 2.3 If at some iteration t there is a migration from a low machine M i such that l t i = l x for some x > 0, then the following l x 1 iterations will involve migrations from M i. We are now ready to state the bound on the convergence rate. Theorem 2.4 For every job scheduling game with identical jobs, every MC-RD process converges to a NE within at most n 1 steps. Proof: First note at least one machine that is active in the initial configuration will remain active in the final configuration. Thus, at least one job, located on this machine, does not migrate at all. If every other job migrates at most once then clearly there are at most n 1 iterations. We show that it is possible for some jobs to migrate twice, but not more than twice. We show that a second migration of a job might happen only towards the end of the RD process and that the number of jobs that migrate twice is smaller than the number of jobs that do not migrate at all. Assume that at least one job migrates more than once. Let j be the first job to re-migrate. Assume that j migrates from M 1 to M 2 at time t and from M 2 to M 3 at time t > t. y our LIFO assumption, l 2 (t ) = l 2 (t) + 1. Claim 2.2 At time t, all the machines, except possibly for M 3, have load l 2 (t ) or l 2 (t ) 1. Proof: We first show that there are no low machines at time t. Assume by way of contradiction that M 0 is low at time t. The load on M 0 remains the same between time t and t ; that is, l 0 (t ) = l 0 (t) because M 0 is not the current best response and according to Observation 2.2, if it were the best response at some time point then it would keep attracting jobs until it was not low anymore. At time t, M 2 is the max-cost machine, therefore c(l 0 (t )) c(l 2 (t )). Also c(l 0 (t )) = c(l 0 (t)) > c(l 0 (t) + 1), because M 0 is low. However, at time t, j preferred joining M 2 having cost c(l 2 (t )) to joining M 0 having cost c(l 0 (t) + 1), a contradiction. Given that there are no low machines at time t, we only need to consider machines that are high at time t. As M 2 is the high-cost machine, there are no machines with load larger than l 2 (t ) at time t. We show that no high machine (except maybe for M 3 ) has load less than l 2 (t ) 1 = l 2 (t) at time t. Assume by way of contradiction that there is a machine, M 4 M 3, with load less than l 2 (t ) 1. The load of M 4 did not decrease between t and t because in order to get below l 2 (t ) 1 it must be that the max-cost is achieved by load l 2 (t ) 1, while we know that the load on M 2 is higher during this whole time interval. The same argument can be applied to M 3 (we know that l 3 (t ) < l 2 (t ) 1 as otherwise the second migration of j would not be beneficial). Therefore, the load on both M 3 and M 4 does not decrease between t and t. If M 3 or M 4 were high already at time t, then j should have preferred one of them to M 2. Thus, 5

6 both M 3 and M 4 were low at time t and by the choice of j at time t, we know that c(l 4 (t) + 1) c(l 2 (t)) as well as c(l 3 (t) + 1) c(l 2 (t)). Assume w.l.o.g. that l 3 (t) l 4 (t); that is, M 4 becomes active before M 3. Since at time t both are high, it must be that M 3 became a best-response between t and t. However, the load on M 4 during this time interval was always less than l 2 (t ) 1, and since c(l 3 (t) + 1) c(l 2 (t)) > c(l 4 (t )) for every t between t and t, we get a contradiction to the choice of M 3 as a best-response during this time interval. Denote by N 1 the number of jobs migrating once, and by N 2 the number of jobs migrating more than once. Given that at time t, all machines except for M 3 have loads l 2 (t ), l 2 (t ) 1, the only remaining migrations starting at time t are to M 3. Therefore, jobs migrating more than once can only migrate twice and their second migration is to M 3. Thus, N 2 is in fact the number of jobs migrating twice. Since M 3 had some initial load at least 1, we conclude that N 2 is strictly less than the final load of M 3. Formally, N 2 < l fin 3. Let M be the most loaded machine when the RD process begins. Simple enumeration argument implies that the lower l fin 3 jobs on M do not migrate at all during the RD process. Therefore, the total number of jobs migrating once is N 1 n N 2 l fin 3 and the total number of migrations is N 1 + 2N 2 n + N 2 l fin 3 < n. The above analysis is tight. An instance achieving this bound is one in which n machines are assigned a single job each. For every n, the RD process will end up with a single machine with load n. Each of the jobs except for the one initially located on this machine migrates once. A more interesting instance, in which some jobs migrate twice, is the following: there are m machines and n = (m 1)(/2) + 1 jobs. Initially, one machine is assigned n (m 1) jobs, and the remaining m 1 machines are assigned a single job each. In a possible MC-RD process, each of the low machines, in turn, attracts (/2) 1 jobs from the high machine, till we get load vector /2 + 1, /2,..., /2, 1. At this point, the last low machine becomes active and attracts jobs from all the other machines in a round-robin fashion. The total number of migrations is n m. We demonstrate below such a slow MC-RD process with 5 machines and = 100. Each line gives the load vector on the machines at some time point. The numbers in parentheses show the number of migrations performed since the previously presented time point (49) (49) (49) (1) (39) In contrast to max-cost RD, the convergence time of arbitrary RD, in which the moving job is not necessarily the one having maximal cost, might not be linear in n. Theorem 2.5 There exists a job scheduling game with identical jobs and a RD process such that the convergence time to a NE is Ω(n log n ). Proof: We describe an instance and a possible RD convergence of this instance. Given m, let n = m/2. Consider an initial assignment in which one machine, M 0, is assigned n (m 1) jobs, and m 1 machines, denoted M 1,..., M m 1, are assigned a single job each. A possible RD process can be described as a sequence of m 1 phases. Phase k for k = 1,..., m 1 begins when machine 6

7 M k is the best-response machine for the first time (i.e., its load increases from 1 to 2). Each phase consists of two stages: (i) balancing - after which the loads on machines M 1,..., M k are equal (up to a gap of 1), and (ii) filling - after which the loads on machines M 1,..., M k are all /2. We demonstrate below such a slow RD process with 5 machines and = 100. Each line gives the load vector on the machines at some time point. The numbers in parentheses show the number of migrations performed since the previously presented time point. Phase 1 Phase 2 Phase 3 Phase (49) (1) (24) (48) (1) (24) (16) (48) (1) (24) (16) (12) (48) For simplicity, in the following analysis, we ignore floors and ceilings and the possible gap of 1 between the machines load after the balancing stage. One can easily verify that in a closer analysis, the total number of migrations might be reduced by at most m. In general, for arbitrary m,, the balancing stage in the first phase is empty. The filling stage consists of /2 1 migrations from M 0 to M 1. After this stage, M 2 becomes a best-response machine (an added job will incur a cost of /2 + 2, which is the current cost of a job on M 1 ). Thus, the second phase begins with a migration from M 0 to M 2. In the balancing stage jobs are migrating from M 1 to M 2 until they both have load (/2 + 2)/2. In the filling stage, the best-response alternates between the machines M 1, M 2, until both have loads /2, and M 3 becomes the new best-response machine. In general, the balancing stage of phase k can be described in a recursive manner: first, the balancing stage of phase k 1 (on machines M 1,..., M k 1 ) is repeated on machines M 2,..., M k ; this results in balancing the loads on M 2,..., M k. Next, there are additional migrations from M 1 to M 2,..., M k. Specifically, in the last part of the balancing stage of phase k, we move from load vector (on M 1,..., M k ) /2, k 2 k 1 part includes 2k migrations. 2, k 2 k 1 2,..., k 2 k 1 2 to load vector k 1 k 2,..., k 1 k 2. This last In the filling stage of phase k, the load on the k machines M 1,... M k is increased from k 1 k 2 to 2, which takes 2 migrations (in fact, the exact number of migrations is 2 2, but the overall distortion in the total counting is at most 2(m 1), which is negligible). Denote by b k (f k ) the number of migrations in the balancing (filling) stage of phase k. According to the above description, for every k > 1 it holds that b(k) = b k 1 + 2k and f(k) = 2. The total number of migrations in phase k is b k + f k = b k 1 + 2k + 2. Also, for k = 1, we have b(1) = 0, f(1) = 2. Summing up all m 1 phases, we get that the total number of migrations until 7

8 convergence is m 1 (b k + f k ) = k=1 m 1 (b k 1 + 2k + 2 ) = m 1 2 (m 1) + (b k 1 + 2k ) k=1 = 2 (m 1) + 2 (1 2 (m 2) (m 3) (m (m 1))) 3 m 1 = 2 (m 1 + m m m m 1 1) = (m log(m 1) (m 1)) 2 Since m = 2n, we get that the total number of migrations is 2 (2n log(2n 1) (2n 1)) > n(log(2n 1) 1)) = Ω(n log n ). k=1 While the convergence rate of general RD is super linear, the following theorem establishes an upper bound of n 2. Closing the gap remains an open problem. Theorem 2.6 For every job scheduling game with identical jobs, every RD process converges to a NE within at most n 2 steps. Proof: Let m low be the number of low machines in the initial configuration. Among these low machines there are m 1 machines that will be emptied along the RD process, and m 2 machines that will form the best-response during the process and (by Observation 2.2) end up high. Partition the RD process into phases. Phase 0 (which might be empty) consists of migrations as long as the best response is a high machine, these migrations might be among high machines or from low machines that are emptied into high machines. Index the m 2 most-loaded low machines in non-increasing order of initial load. For i > 0, phase i begins when the i-th low machine form the best-response for the first time. Claim 2.3 In each phase, there are at most n migrations. Proof: For i > 0, phase i begins with migrations to the i-th low machine till it becomes high. This machine will never become low again, because the job at height l will never leave it, thus, jobs migrating to the i-th machine, as long as it is low, will not migrate again (in this, and in later phases). The following migrations (and all the migrations in phase 0), are migrations to high machines among high machines or from low machines that are emptied. During such migrations, the bestresponse value can only increase, therefore, if job j is migrating to load l, the best-response value will always be at least c(l + 1) and by the LIFO property, j will not leave the machine as long as the best-response value is not strictly lower. It follows that the total number of migrations is bounded by n(m 2 + 1). Since m 2 < n, this is at most n 2. It is interesting to compare our results to those established for the standard model that considers only the negative congestion effects (i.e., where a job s cost is simply the load of its chosen machine). It has been shown in [3] that if the order of the jobs performing their best-response moves is determined according to their lengths (i.e., longer job first), then best-response dynamics reaches a pure Nash equilibrium within at most n improvement steps. In contrast, if the jobs move in an arbitrary order, then convergence to a Nash equilibrium might take an exponential number of 8

9 steps. These results imply that for the special case of equal-length jobs, convergence occurs within at most n steps. Our results provide evidence that when there are conflicting congestion effects, it might take longer to reach a Nash equilibrium. Nevertheless, for the special case of max-cost RD, the consideration of positive congestion effects (through activation costs) does not lead to a longer convergence time. References [1] E. Anshelevich, A. Dasgupta, J. M. Kleinberg, É.Tardos, T. Wexler, and T. Roughgarden. The Price of Stability for Network Design with Fair Cost Allocation. In Symposium on the Foundations of Computer Science (FOCS), , [2] A. Epstein, M. Feldman, and Y. Mansour. Strong Equilibrium in Cost Sharing Connection Games. In ACM Conference on Electronic Commerce (ACMEC), [3] E. Even-Dar, A. Kesselman, and Y. Mansour. Convergence Time to Nash Equilibria. In In Proc. 30th International Colloq. on Automata, Languages and Programming, , [4] M. Feldman and T. Tamir. Conflicting Congestion Effects in Resource Allocation Games. Proc. of the 4th International Workshop on Internet and Network Economics (WINE), December [5] D. Fotakis, M. Mavronicolas S. Kontogiannis, and P. Spiraklis. The Structure and Complexity of Nash Equilibria for a Selfish Routing Game. In International Colloquium on Automata, Languages and Programming (ICALP), pages , [6] R.L. Graham. ounds on Multiprocessing Timing Anomalies. SIAM Journal of Applied Mathematics, 17: , [7] S. Herzog, S. Shenker, and D. Estrin. Sharing the Cost of Multicast Trees: An Axiomatic Analysis. IEEE/ACM Transactions on Networking, [8] E. Koutsoupias and C. Papadimitriou. Worst-case Equilibria. Computer Science Review,3(2): 65-69, [9] H. Moulin and S. Shenker. Strategyproof Sharing of Submodular Costs: udget alance Versus Efficiency. Journal of Economic Theory, 18: , [10] N. Nisan and A. Ronen. Algorithmic Mechanism Design. Games and Economic ehavior, 35:166196, [11] T. Roughgarden and E. Tardos. How ad is Selfish Routing? Journal of the ACM, 49(2): , [12]. Vöcking. Selfish Load alancing. In Algorithmic Game Theory edited by N. Nisan, T. Roughgarden, E. Tardos and V. Vazirani. Cambridge University Press,