Non-cooperative scheduling of multiple bag-of-task applications

Transcription

1 INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE Non-cooperative scheduling of multiple bag-of-tas applications Arnaud Legrand Corinne Touati N 5819 January 2006 Thème NUM ISSN ISRN INRIA/RR FR+ENG apport de recherche

2

3 Non-cooperative scheduling of multiple bag-of-tas applications Arnaud Legrand, Corinne Touati Thème NUM Systèmes numériques Projet MESCAL Rapport de recherche n 5819 January pages Abstract: Multiple applications that execute concurrently on heterogeneous platforms compete for CPU and networ resources. In this paper we analyze the behavior of non-cooperative schedulers using the optimal strategy that maximize their efficiency. Meanwhile fairness is ensured at a system level ignoring applications characteristics. We limit our study to simple single-level master-worer platforms and the case where applications consist of a large number of independent tass. The tass of a given application all have the same computation and communication requirements, but these requirements can vary from one application to another. Therefore, each scheduler aims at maximizing its throughput. We give closed-form formula of the equilibrium reached by such a system and study its performances. We characterize the situations where this Nash equilibrium is Pareto-optimal and show that even though no catastrophic situation (Braess-lie paradox) can occur, such an equilibrium can be arbitrarily bad for any classical performance measure. ey-words: heterogeneous processors, master-slave tasing, steady-state, throughput, non-cooperative scheduling, Nash equilibrium, Braess-lie paradox Unité de recherche INRIA Rhône-Alpes 655, avenue de l Europe, Montbonnot Saint Ismier (France) Téléphone : Télécopie

4 Ordonnancement non-coopératif d applications régulières Résumé : Lorsque plusieurs applications s exécutent simultanément sur une plateforme de calcul hétérogène, elles entrent en compétition pour l accès aux ressources de calcul et de communication. Dans ce rapport, nous analysons le comportement de ordonnanceurs ne coopérant pas et utilisant la stratégie qui optimise leur performance. L équité d accès aux ressources est donc assurée au niveau du système sans tenir compte des spécificités des différentes applications. Nous limitons notre étude aux simples platesformes maître-esclave arborescentes à un seul niveau et au cas où chaque application est constituée d un très grand nombre de tâches indépendantes. Les tâches d une même application ont toutes les mêmes besoins en terme de calcul et de communications mais ces besoins peuvent varier d une application à l autre. Dans un tel contexte, il est naturel que chaque application cherche à optimiser son propre débit. Nous donnons des formes closes de l équilibre atteint par un tel système et nous étudions ses performances. Nous caractérisons les situations où cet équilibre de Nash est Pareto-optimal et montre que même si aucune situation catastrophique de type paradoxe de Braess ne peut se produire, cet équilibre est arbitrairement mauvais pour toutes les mesures de performances classiques. Mots-clés : Ressources hétérogènes, maître/esclaves, débit, régime permanent, ordonnancement non-coopératif, équilibre de Nash, paradoxe de Braess

5 Non-cooperative scheduling of multiple bag-of-tas applications 3 1 Introduction The recent evolutions in computer networs technology, as well as their diversification, yield to a tremendous change in the use of these networs: applications and systems can now be designed at a much larger scale than before. Large-scale distributed platforms (Grid computing platforms, enterprise networs, peer-to-peer systems) result from the collaboration of many people. Thus, the scaling evolution we are facing is not only dealing with the amount of data and the number of computers but also with the number of users and the diversity of their behavior. Therefore computation and communication resources have to be fairly shared between users, otherwise users will leave the group and join another one. However, the variety of user profiles requires resource sharing to be ensured at a system level. We claim that even in a perfect system where each application competing on a given resource always receives the same share as the other ones and where no degradation of resource usage (e.g. pacet loss or context switching overhead) occurs when a large number of applications use a given resource, non-cooperative usage of the system leads to important application performance degradation and resource wasting. In this context, we mae the following contributions: We present a simple yet realistic situation where the system-level fairness fails to achieve a relevant application-level fairness. More precisely, we study the situation where multiple applications consisting of large numbers of independent identical tass execute concurrently on heterogeneous platforms and compete for CPU and networ resources. As the tass of a given application all have the same computation and communication requirements (but these requirements can vary for different applications), each scheduler aims at maximizing its throughput. This framewor had previously been studied in a cooperative centralized framewor [BLCJF + 06]. The resource sharing aspect was therefore not present and is extensively described in Section 2. We first characterize in Section 3.1 the optimal selfish strategy for each scheduler (i.e. the scheduling strategy that will maximize its own throughput in all circumstances and adapt to external usage of resources) and then propose in Section 3.2 equivalent representations of such non-cooperative schedulers competition. The particularities of these representations enable us to characterize the structure of the resulting Nash equilibrium as well as closed-form values of the throughput of each application (see Section 3.3 and 3.4). Using these closed-form formulas, we derive in Section 4 necessary and sufficient conditions on the system parameters (in term of bandwidth, CPU speed,... ) for the non-cooperative equilibrium to be Pareto-optimal. We also quicly study the degree of inefficiency of this equilibrium through a simple example. When studying properties of Nash equilibria, it is important to now whether paradoxical situations lie the ones exhibited by Braess in his seminal wor [Bra68] can RR n 5819

6 4 A. Legrand, C. Touati occur. In such situations, the addition of resource (a new machine, more bandwidth or more computing power in our framewor) can result in a simultaneous degradation of the performance of all the users. Such situations only occur when the equilibrium is not Pareto-optimal, which may be the case in this framewor. We investigate in Section 4.4 whether such situations can occur in our considered scenario and conclude with a negative answer to this question. Last, we show in Section 5, that even when the non-cooperative equilibrium is Pareto-optimal, the throughput of each application is far from being monotonous with a resource increase. This enables us to prove that this equilibrium can be arbitrarily bad for any of the classical performance measures (average throughput, maximal throughput, minimum throughput). Section 6 concludes the paper with a discussion of extensions of this wor and future directions of research. 2 Platform and Application Model 2.1 Platform Model P 0 B 1 B n B N P 1 P n P N W 1 W n W N Our master-worer platform is made of N + 1 processors P 0, P 1,..., P N. P 0 denotes the master processor, which doesn t perform any computation. Each processor P n is characterized by its computing power W n (in Mflop.s 1 ) and the capacity of its connection with the master B n (in Mb.s 1 ). Last, we define the communication-to-computation ratio C n of processor P n as B /W. This model leads us to the following definition: Definition 1. We denote by physical-system a triplet (N, B, W ) where N is the number of machines, and B and W the vectors of size N containing the lin capacities and the computational powers of the machines. We assume that the platform performs an ideal fair sharing of resources among the various requests. More precisely, let us denote by N n (B) (t) (resp. N n (W ) (t)) the number of ongoing communication (resp. computation) from P 0 to P n (resp. on P n ) at time t. Resources are shared fairly between the different users. Thus, the platform ensures that the amount of bandwidth received at time t by a communication from P 0 to P n is INRIA

7 Non-cooperative scheduling of multiple bag-of-tas applications 5 exactly B n /N n (B) (t). Liewise, the amount of processor power received at time t by a computation on P n is exactly W n /N n (W ) (t). Therefore, the time T needed to transfer a file of size b from P 0 to P n starting at time t 0 is such that T t=t 0 B n N n (B).dt = b. (t) Liewise, the time T needed to perform a computation of size w on P n starting at time t 0 is such that T W n t=t 0 N n (W ).dt = w. (t) Last, we assume that communications to different processors do not interfere. This amounts to say that the networ card of P 0 has a sufficiently large bandwidth not to be a bottlenec. This hypothesis maes sense when worers are spread over the Internet and are not too numerous. 2.2 Application Model We consider applications, A, 1. Each application is composed of a large set of independent, same-size tass. We can thin of each A as bag of tass, and the tass are files that require some processing. A tas of application A is called a tas of type. SETI@home [SET], factoring large numbers [CDu + 96], the Mersenne prime search [Pri], ClimatePrediction.NET [BOI], Einstein@Home [EIN], processing data of the Large Hadron Collider [LHC] are a few examples of such typical applications. We let w be the amount of computation (in floating point operations) required to process a tas of type. Similarly, b is the size (in Mb) of (the file associated to) a tas of type. We assume that the only communication required is outwards from the master, i.e. that the amount of data returned by the worer is negligible. This is a common hypothesis [BOBLC + 04] as in steady-state, the output-file problem can be reduced to an equivalent problem with bigger input-files. Last, we define the communication-tocomputation ratio of tass of type as b /w. This model leads us to the following definition: Definition 2. We define a user-system a triplet (, b, w) where is the number of applications, and b and w the vectors of size representing the size and the amount of computation associated to the different applications. 2.3 Rules of the game In the following our applications run on our N processors and compete for the networ and CPU access: Definition 3. A system is a sextuplet (, b, w, N, B, W ), with,b,w,n,b,w defined as for a user-system and a physical-system. RR n 5819

8 6 A. Legrand, C. Touati We assume that each application is scheduled by its own scheduler. As each application comprises a very large number of independent tass, trying to optimize the maespan is nown to be vainly tedious [Dut04] especially when resource availability may vary over time. Maximizing the throughput of a single application is however nown to be a much more relevant metric in our context [BLCJF + 06, HP04]. Let us denote by α the average number of tass of type performed per unit of time on the processor P n. In the rest of this document, we will denote by α = n α. The scheduler of each application thus aims at maximizing its own throughput, i.e. α. However, as they are sharing the same set of resources, we have the following general constraints 1 : Computation n 0, N : =1 α.w W n Communication n 1, N : =1 α.b B n 3 A non-cooperative game Is this article, we assume that a process running on P 0 can communicate with as many processors as it wants. This model is called multi-port [BSP + 99, BNGNS00] and is reasonable if the number of worers is relatively small and the process can mae use of threads to handle communications. We first study the situation where only one application is scheduled on the platform. This will enable us to simply define the scheduling strategy that will be used by each players in the more general case where many applications are considered. 3.1 One application When there is only one application, our problem reduces to the following linear program: Maximize n α n,1, under the constraints (1a) n 0, N : α n,1.w 1 W n (1b) n 1, N : α n,1.b 1 B n (1c) n, α n,1 0 We can easily show( that the ) optimal solution to this linear program is obtained by setting α n,1 = min Wn w, Bn b. In a practical setting, this amounts to say that the master process will saturate each of the worers by sending them as much tass as possible. On a stable platform, W n and B n can easily be measured and the α n,1 s can thus easily be computed. On an unstable one this may be more tricy. However a simple acnowledgement mechanism enables the master process to ensure that it is not over-flooding the worers, while always converging to the optimal throughput. In a multiple-applications context, each player (process) strives to optimize its own performance measure, considered here to be its throughput α. Hence, we will assume 1 The notation a, b denotes the set of integers comprised between a and b, i.e. a, b = N [a, b] (1) INRIA

9 Non-cooperative scheduling of multiple bag-of-tas applications 7 that each of them constantly floods the worers while ensuring that all the tass they sends are performed (e.g. using an acnowledgement mechanism). This adaptive strategy automatically cope with other schedulers usage of resource and selfishly maximize the throughput of each application. We suppose a purely non-cooperative game where no scheduler decides to ally to any other. As the players constantly adapt to each others actions, they may (or not) reach some equilibrium, nown in game theory as Nash equilibrium [Nas50, Nas51]. In the remaining of this paper, we will denote by the rates achieved at such stable states. 3.2 Many applications: canonical representations First, we can see that in the multi-port model, communication and computation resources are all independent. Let us consider in this section the use of resources on a arbitrary worer n. We only need to ensure that the number of tass sent to P n is equal to the number of tass processed on P n. In a steady state, actions of the players will interfere on each resource in (a priori) a non predictable order. The resource usage may be arbitrarily complex (see Figure 1(a)). However, for any given subset S of 1,, we can define the fraction of time where all players of S (and only them) use a given resource. This enables us to reorganize the schedule into an equivalent representation (see Figure 1(b)) with at most 2 time intervals (the number of possible choices for the subset S). In this representation, the fractions of time spent using a given resource (which can be a communication lin or a processor) are perfectly equal to the ones in the original schedule. However such a representation is still too complex (2 is a large value). Hence, we now explain how to build two more compact equivalent canonical representation (see Figure 1(c) and 1(d)) Sequential canonical representation The first compact form we define is called sequential canonical representation (see Figure 1(c)). If the schedulers were sending data one after the other on this lin, the th scheduler would have to communicate during exactly µ (seq) = α(nc) b B n of the time to send the same amount of data as in the original schedulers 2. This value is called sequential communication time ratio. Similarly, we can define the sequential computation time ratio ν (seq) relation between µ (seq) and ν (seq) : µ (seq) as α(nc) w W n. We have the following = C n ν (seq) (2) We can therefore obtain a canonical schedule (see Figure 1(c)) with at most + 1 intervals whose respective sets of players are {1}, {2},..., {},. This communication scheme is thus called sequential canonical representation and has the same values. 2 Note that we have µ (seq) n,1 µ (seq) n,2 µ (seq) n,. RR n 5819

10 8 A. Legrand, C. Touati Resource Usage Resource Usage 0 1 (a) Complex arbitrary schedule Resource Usage µ (seq) n,1 µ (seq) n,2 µ (seq) n,3 time 0 1 Resource Usage (b) Sorted schedule µ (par) n,1 µ (par) µ (par) n,3 n,2 time 0 1 time (c) Sequential canonical representation: areas are preserved but using times are minimized 0 1 µ (par) n,1 µ (par) n,2 µ(par) n,3 time (d) Parallel canonical representation: areas are preserved but using times are maximized Figure 1: Various schedule representations. Each application is associated to a color: Application 1 is green, application 2 is yellow and application 3 is blue. The area of each application is preserved throughout all transformations. However, communication and computation times have all been decreased as each scheduler is now using the networ lin and the CPU exclusively. We will see later that this information loss does not matter for multi-port schedulers. Parallel canonical representation The second compact form we define is called parallel canonical representation (see Figure 1(d)). In this scheme, resource usage is as conflicting as possible. Let us denote by µ (par) (resp. ν (par) ) the fraction of time spent by player to communicate with P n (resp. to compute on P n ) in such a configuration. µ (par) is the parallel communication time ratio and ν (par) is the parallel computation time ratio. Let us focus on the canonical representation of resources in the communication channel, the case of processor sharing being similar. To simplify the analysis, let us suppose that µ (par) n,1 µ (par) n,2 µ (par) n,. Thus, communication times verify the INRIA

11 Non-cooperative scheduling of multiple bag-of-tas applications 9 following set of equations (where µ (par) = µ (par) n,1 b 1 = B n µ(par) n,1 µ (par) 1 ): n,2 b 2 = B n µ(par) n,1 + B n 1 µ(par) n,2. n, b = =1. B n + 1 µ(par) We can easily chec that this system has a unique solution, that all the µ (par) are non-negative and that their sum is no larger than 1. From these µ (par) s we can therefore obtain a canonical schedule (see Figure 1(d)) with at most + 1 intervals whose respective player sets are respectively {1,..., }, {2,..., },..., {},. This communication scheme is thus called parallel canonical representation and has the same values. However, communication times have all been increased as each scheduler is now interfering with as much other scheduler as possible. (3) Particularities of multi-port selfish schedulers The same reasonings can be applied to computation resources and therefore, for a given worer, both communication and computation resources can be put in canonical form (see Figure 2(a)). As we have seen in Section 3.1, the scheduling algorithm used by the players consist in constantly flooding worers. Hence it is impossible that both µ (par) and ν (par) are smaller than 1. A player is thus said to be either communication-saturated (µ (par) = 1) or computationsaturated (ν (par) = 1). Proposition 1. If there exists a communication-saturated application then µ(seq) = 1. Similarly, if there exists a computation-saturated application then ν(seq) = 1. As two computation-saturated players 1 and 2 receive the same amount of computation power and compute during the same amount of time, we have 1 w 1 = 2 w 2. Therefore if 1 2, we have 1 b 1 2 b 2. The same reasoning holds for two communication-saturated players as well as for a mixture of both. As a consequence, in a multi-port setting, players should be first sorted according the to build the canonical schedule. The very particular structure of this schedule (see Figure 2(b)) will enable us in the following section to give closed-form formula for the. All these remars can be summarized in the following proposition: Proposition 2. If there is an equilibrium, let us denote by B n the set of communicationsaturated applications and by W n the set of computation-saturated applications. If c 1 c 2 c, then there exists m 0, such that W n = 1, m and B n = m+1,. We have: RR n 5819

12 10 A. Legrand, C. Touati Communication Computation 0 µ (par) n,1 µ (par) µ (par) n,2 n,3 µ (par) n,4 0 ν (par) n,4 ν (par) n,3 (a) Parallel canonical form of an arbitrary schedule ν (par) n,1 ν (par) n,2 Communication Computation 0 µ (par) n,1 µ (par) µ (par) n,2 n,3 µ (par) n,4 0 ν (par) n,4 ν (par) n,3 ν (par) n,1 ν (par) n,2 (b) Parallel canonical schedule for a given processor under the noncooperative multi-port assumptions. Application 3 (blue) and 4 (red) are communication saturated: they receive the same amount of communication resource. Application 1 (green) and 2 (yellow) are computation saturated: they receive the same amount of computation resource. Figure 2: Parallel canonical schedules Parallel representation: Communications: Computations: Sequential representation: Communications: Communications: B n {}}{ µ (par) n,1 µ (par) n,m < µ (par) n,m+1 = = µ(par) n, = 1 and n,1 = = ν n,m (par) (par) > νn,m+1 ν(par) n, }{{} W n 1 = ν (par) B n {}}{ µ (seq) n,1 µ (seq) n,m < µ (seq) n,m+1 = = µ(seq) n, = G n and n,1 = = ν n,m (seq) (seq) > νn,m+1 ν(seq) n, }{{} W n F n = ν (seq) INRIA

13 Non-cooperative scheduling of multiple bag-of-tas applications Closed-form solution of the equations Theorem 1. We assume c 1 c 2 c. Let us denote by W n the set of players that are computation-saturated and by B n the set of players that are communication-saturated. 1. If B n and W n are non-empty, we have: = Bn b = Wn w P cp W n p Wn Cn W P P n B n p Wn cp p Bn P B n Cn p Bn cp W P P n B n p Wn cp p Bn 1 cp 1 cp if B n if W n (4) 2. If W n = : 3. If B n = : = B n.b = W n.w Proof. Let us start by the two easy cases where W n = or B n = : If B n =, then all applications use the CPU of P n at any time. Therefore they all receive the exact same amount of CPU, i.e. B n /, hence = Wn.w. If W n =, the exact same reasoning holds and we get = Bn.b. Let us now focus on the more interesting case where both B n and W n. Using the definition of sequential communication and computation times, we have: { p B n µ (seq) n,p p B n ν n,p (seq) + p W n µ (seq) n,p = 1 + p W n ν (seq) n,p = 1 (5) As we have = b B n µ (seq) following relation: and p, ν (seq) n,p = w W n ν (seq), ν(seq) n,p and µ (seq) n,p are lined by the = C n c p µ (seq) n,p (6) Two applications from B n communicate all the time. Therefore the send the exact same amount of data and thus: p B n, µ (seq) n,p = G n Similarly, we get p W n, ν (seq) n,p = F n Using these relations, equations (5) can be written: { Bn G n + F n p W n c p C n = 1 W n F n + G n p B n C n cp = 1 RR n 5819

14 12 A. Legrand, C. Touati Solving this system, we get: G n = F n = W P cp n p Wn Cn W P P n B n p Wn cp p Bn B P n Cn p Bn cp W n B n P p Wn cp P p Bn 1 cp 1. cp And finally: = Bn b = Wn w P cp W n p Wn Cn W P P n B n p Wn cp p Bn P B n Cn p Bn cp W P P n B n p Wn cp p Bn 1 cp 1 cp if B n if W n 3.4 Conditions on the sets Let us first start with a technical lemma that will prove to be very useful in the following. Lemma 1. Let γ 1 < < γ be positive numbers. We have: 1. If 1/γ then γ > ; 2. If γ then 1/γ > ; 3. If γ > and 1/γ >, then there exists exactly one m 1, such that: m =1 γ m < 1 γ =m < γ m+1 γ Proof. Let us prove each statement one after the other: 1. Let us assume that 1/γ. As (1 γ ) 2 0, we have (2 γ ) 1/γ. Therefore, we have (2 γ ) <, hence γ >. 2. The same proof as before can be used by setting γ = 1/γ. 3. Let us now assume that γ < and 1/γ > and let us define ϱ by P m =1 ϱ(m) = 1 γ P. Let us also define f and g by =m γ f(m) = γ m+1 g(m) = γ m =m+1 =m+1 (1 1γ ) (1 1γ ) From these definitions, we can easily see that: m (1 γ ) =1 m (1 γ ) (!m γ m < ϱ(m) < γ m+1 ) (!m f(m) > 0 and g(m) < 0) (7) =1 INRIA

15 Non-cooperative scheduling of multiple bag-of-tas applications 13 By noticing that we have ( f(m) g(m + 1) = γ m ) + (1 γ m+1 ) = 0, γ m+1 condition (7) can be rewritten as (!m γ m < ϱ(m) < γ m+1 ) (!m f(m) > 0 and f(m 1) < 0) Therefore, to prove our lemma, we should study the variations of f. We have 3 : ( f(0) = γ +1 ) =1 1 γ < 0 (by hypothesis) ( ) f() = =1 γ > 0 (by hypothesis) ) f(m + 1) f(m) = (γ m+2 γ m+1 ) (1 1γ Then, we have =m+2 } {{ } d(m) d(0) = =1 1 γ < 0 (by hypothesis) d( 1) = 0 d(m + 1) d(m) = 1 γ m+2 1 So let us denote by M the first value such that γ > 1 and γ 1 < 1 (we 0 1 M M 2 M γ 1 d() 0 f() 0 m Figure 3: Variation table of f, h and γ. now that such value exist otherwise we would have γ or 1/γ ). Therefore, we now that d is strictly increasing on 0, M 2 and strictly decreasing on M 2, 0 (see Figure 3). Since d(0) < 0, and d( 1) = 0, we now that 3 We can freely set γ +1 = γ + 1 so that f is defined on 0,. RR n 5819

16 14 A. Legrand, C. Touati d(m 2) > 0. Therefore, there exists a M 1, 1 such that d is negative on 0, M 1 and positive on M, 1. Last, since f(0) < 0 and f() > 0, we now there exists exactly one m such that f(m) > 0 and f(m 1) < 0, hence the result. Theorem 2. Assuming c 1... c, B n and W n can be determined using the following formula: 1. If C n then B n =. 2. Else if Cn then W n =. 3. Else we have W n and B n. Let m be the integer such that 4 : c m C n < m m =1 C n m =m+1 Cn < c m+1 C c n Then we have W n = {1,..., m} and B n = {m + 1,..., }. Proof. First of all, we shall remar that the previous cases are exclusives using Lemma 1 with γ = /C n. Therefore, we only have to prove that, depending on the structure of B n and W n, we fall in either one or the other of the previous cases: Suppose that B n =, then, ν (seq) C n. Hence B n = Cn. = 1/ and 1 µ(seq) = C n ν (seq) = Suppose that W n =, then, µ (seq) Cn. Hence W n = Cn. = 1/ and 1 ν(seq) Suppose that both W n and B n. Remember that ν n,p (seq) by the following relation: p, ν (seq) n,p = C n µ (seq) n,p c p = Cn µ (seq) n,p = and µ (seq) n,p Let m such that m B n and m + 1 W n. Recalling proposition 2, we have: µ (seq) n,1 µ (seq) n,m < µ(seq) n,m+1 = = µ(seq) n, From ν (seq) n,m+1 = Cn c m+1 µ (seq) n,m+1 and equation (4) of Theorem 1, we get: are lined and ν(seq) n,1 = = ν n,m (seq) > ν(seq) n,m+1 ν(seq) n, c m+1 C n = µ (seq) n,m+1 ν (seq) n,m+1 > µ(seq) n,m+1 ν n,m (seq) = m m =1 C n m =m+1 Cn 4 We now such an integer exists using Lemma 1 with γ = /C n INRIA

17 Non-cooperative scheduling of multiple bag-of-tas applications 15 Similarly, we have: c m Therefore, we have: = µ(seq) n,m C n ν n,m (seq) < µ(seq) n,m+1 ν n,m (seq) = m m =1 C n m =m+1 Cn c m C n < m m =1 C n m < c m+1 =m+1 Cn C c n that is to say γ m < m m =1 γ m =m+1 1 γ < γ m+1. Remar 1. From these equations, we see that there always exists exactly one noncooperative equilibrium. 4 Inefficiencies and paradoxes In this section, we study the inefficiency of the Nash equilibria, in the Pareto sense, and their consequences. Let us start by recalling the definition of the Pareto optimality. Definition 4. Let G be a game with players. Each of them is defined by a set of possible strategies S and utility functions u defined on S 1 S. A vector of strategy is said to be Pareto optimal if it is impossible to strictly increase the utility of a player without strictly decreasing the one of another. In other words, ( ) (s 1,..., s ) S 1 S is Pareto optimal iff ( (s 1,..., s ) S 1 S, ) i, u i (s 1,..., s ) > u i (s 1,..., s ) j, u j (s 1,..., s ) < u j (s 1,..., s ) We recall that in the considered system, the utility functions are the α, that is to say, the average number of tass of application processed per time-unit, while the strategies are the scheduling algorithms (i.e. which resource to use and when to use them). After expressing the utility set, i.e. the achievable utility vectors (subsection 4.1), we comment of the efficiency of the Nash equilibrium, in the case of a single worer (subsection 4.2), and then of multiple worers (4.3). Additionally, it is nown that when Nash Equilibria are inefficient, some paradoxical phenomenon can occur (see, for instance [Bra68]). We hence study in subsection 4.4, the occurrence of Braess phenomena in this system. RR n 5819

18 16 A. Legrand, C. Touati 4.1 Convex and compact utility set Let consider a system with applications running over N machines. We recall that the set of achievable utilities is: α 1,,..., α, 1, : N n=1 α = α (α ) 1 n 1, N : =1 α.w W n n 1, N : =1 α.b B n n 1, N, 1, : α 0 The utility set is hence convex and compact. 4.2 Single processor In this subsection we show that when the players (here the applications) compete in a non-cooperative way over a single processor, the resulting equilibrium (see Section 3.2) is Pareto optimal. Theorem 3. On a single-processor system, the allocation at the Nash equilibrium is Pareto optimal. Proof. Suppose that the non-cooperative allocation is not Pareto optimal on P n. Then, there exists another allocation α. such that Then,, α, and q 1,, n,q < α n,q. = µ(seq) Bn b eµ(seq) Bn b =1 = α and µ(seq) n,q B n b µ (seq) < =1 < eµ(seq) n,q b B n, which implies µ (seq) (8) Similarly, we get We have two possibilities: =1 ν (seq) < =1 ν (seq) (9) If in the non-cooperative solution, at least one application is communicationsaturated (i.e. B n ), then 1 = µ(seq) (recall proposition 1). However, as µ(seq) 1 by definition, we have =1 µ(seq) =1 µ(seq), which is in contradiction with (8). We get a similar contradiction with (9) when at least one application is computationsaturated (i.e. W n ) in the non-cooperative solution. INRIA

19 Non-cooperative scheduling of multiple bag-of-tas applications Multi-processors and inefficiencies Interestingly, although we have seen that the Nash Equilibria are Pareto optimal on each worer, we show in this section that this Equilibrium is not Pareto optimal for the whole system. We show that the Nash Equilibrium can be inefficient and exhibit such an example on a system consisting of two machines and two applications (subsection 4.3.1). We then give a formal necessary and sufficient condition for the equilibrium to be Pareto optimal (subsection and 4.3.3). We finally comment on different measures of inefficiency (subsection 4.3.4) Example of Pareto inefficiency The Pareto optimality is a global notion. Hence, although at each processor, the allocation is Pareto optimal, the result may not hold for a random number of machines. We will see later (Theorem 4) that the result holds if we have on all the machines W n OR on all the machines B n. Yet, if there are two worers n 1 and n 2 such that W n1 = and B n2 =, then the allocation may be Pareto sub-optimal. We illustrate this on a simple example with a two-applications and two-machines system. Example 1. Consider a system with two computers A and B, with parameters B 1 = 50, W 1 = 100, B 2 = 100, B 2 = 50 and two applications of parameters b 1 = 1, w 1 = 2, b 2 = 2 and w 2 = 1. If the applications were collaborating such that application 1 was processed exclusively to computer A and application 2 in computer B, their respective throughput would be α (coop) 1 = α (coop) 2 = 50. Yet, with the non-cooperative approach they only get a throughput of 1 = 2 = Sufficient condition Theorem 4. If, in a system with a finite number N of machines, we have n 1, N, B n then, the non-cooperative allocation is Pareto optimal. Proof. Suppose that for the N machines, B n. Suppose that the non-cooperative allocation is not Pareto optimal. Then, there exists another allocation α such that, α, and q 1,, q < α q. RR n 5819

20 18 A. Legrand, C. Touati Then, N N n=1 n=1 µ n,q (seq) B n b q µ (seq) Bn b < N n=1 = N n=1 α(nc) eµ (seq) n,q b q B n = α(nc), which implies α = N n=1 α = N n=1 eµ (seq) Bn b and N =1 n=1 n=1 =1 µ (seq) B n < N =1 n=1 µ (seq) B n (10) However, as on each machine, at least one application is communication saturated (i.e. n, B n ) then µ(seq) = 1, which implies that n, µ(seq) µ(seq). Hence finally N N µ (seq) B n which is in contradiction. with (10). n=1 =1 µ (seq) B n, Corollary 1. If, in a system with a finite number N of machines, we have n N, W n then, the non-cooperative allocation is Pareto optimal Necessary and sufficient condition Suppose that the applications are not all identical, that is to say that there exists 1 and 2 such that 1 < 2. We have seen in Section 4.3.2, that two sufficient conditions for the Nash equilibrium to be Pareto optimal is that: n, W n (i.e. n, Cn > ) or n, B n (i.e. n, C n > ). In this section, we show that this condition is actually necessary. Theorem 5. Suppose that the applications are not all identical, that is to say that there exists 1 and 2 such that 1 < 2. Suppose that there exists two worers, namely n 1 and n 2 such that: W n1 = and B n2 =, then the allocation at the Nash Equilibrium is Pareto inefficient. Proof. Let renumber the applications such that c 1 c 2 c and suppose that there exists 1, 1 such that < +1. To show the Pareto inefficiency of the Nash Equilibrium, we construct another allocation α in which: 2, 1, = α 1 < α 1 < α INRIA

21 Non-cooperative scheduling of multiple bag-of-tas applications 19 The idea for this is to eep the allocations of the applications 2,..., 1 unchanged. We note that n 1 is relatively poor in bandwidth while n 2 relatively lacs of computational capability. Also, as c 1 < c, the idea of the construction is to increase the fraction of jobs from application 1 (respectively ) sent to worer n 1 (resp. n 2 ). More precisely, we tae: n / {n 1, n 2 }, 1;, µ (seq) n {n 1, n 2 }, 2; 1, µ (seq) ν (seq) n 2,1 = ν(seq) n 2,1 ε 1, ν (seq) n 2, = ν(seq) n 2, +ε 3, µ (seq) n 1,1 = µ (seq), ν(seq) = ν (seq), = µ (seq), ν(seq) = ν (seq), = µ(seq) n 1,1 +ε 2, and µ (seq) n 1, = µ(seq) n 1, ε 4. ( ) Hence 2, 1, α = and n / {n 1, n 2 }, α n,1 = n,1 and α n, = n,. These definitions leads us to the following equations for ε 1, ε 2, ε 3 and ε 4 : ( ) ( ) ( µ(seq) n 2, = µ(seq) n 2, + c C n2 ε 3 c 1 C n2 ε 1 ) ( ) µ(seq) n 1, = µ(seq) n 1, + (ε 2 ε 4 ) ( ν(seq) n 2, = ν(seq) n 2, + (ε 3 ε 1 ) ) ( ) ν(seq) n 1, = Cn1 ν(seq) n 1, + c 1 ε 2 Cn 1 c ε 4 As B n2 =, we have W n2 hence ν(seq) n 2, = 1. By setting ε 1 = ε 3, we get ν(seq) n 2, = 1. Also, as B n 2 =, we have µ(seq) n 2, < 1, hence for ε 1 sufficiently small µ(seq) n 2, < 1. As W n1 =, we have B n1 hence µ(seq) = 1. By setting ε 3 = ε 4, we get µ(seq) n 1, ν(seq) n 1, n 1, = 1. Also, as W n 1 =, we have ν(seq) n 1, < 1, hence for ε 2 sufficiently small < 1. Therefore, by setting ε 1 and ε 2 sufficiently small, α is a valid allocation. Finally, since W n1 = B n2 =, then n 1, = B n 1.b finally, α 1 > 1 and α > α n1, + α n2, > n 1, + α(nc) n 2, and and n 2, = W n 1..w are equivalent to α n 1,1 + α n2,1 > n 1,1 + α(nc). We have: α n1,1 + α n2,1 n 1,1 + α(nc) n 2,1 = B n 1 ε 2 W n 2 ε 1 b 1 w 1 α n1, + α n2, > n 1, + α(nc) n 2, = W n 2 w ε 1 W n 1 w ε 2 Therefore, the conditions α 1 > 1 and α > are equivalent to: Hence n 2,1 and W n2 B n1 c 1 < ε 2 ε 1 < W n 2 B n1 c (11) Hence, for ε 1 and ε 2 sufficiently small and satisfying (11), the allocation α is strictly Pareto superior to the Nash equilibrium of the system. RR n 5819

22 20 A. Legrand, C. Touati Degree of inefficiency We have seen that the Nash equilibrium of the system can be Pareto inefficient. An natural question then raising is how much inefficient is it? Unfortunately, defining Pareto inefficiency is still an open question. We recall in the following some possible definitions and study their properties. Definition of degree of inefficiency the α. Classical efficiency measure are: Let us denote by f an efficiency measure on Profit f(α 1,..., α ) = =1 α Max-min f(α 1,..., α ) = min =1 α Proportional f(α 1,..., α ) = =1 α Inverse f(α 1,..., α ) = ( =1 1 α ) 1 For a given system S (i.e. platform parameters along with the description of our applications), we denote by (S), the rates achieved on system S by the noncooperative algorithm of Section 3. Let us denote by α (f) (S), the optimal rates on system S for the metric f. One can define the inefficiency of the non-cooperative allocation for a given metric and a given system as: ( ) f α (f) 1 (S),..., α(f) ( (S) ) 1 f ( 1 (S),..., (S) The degree of inefficiency φ f can be defined as the largest inefficiency: φ f = max I ( f f α (f) 1 ) (S),..., α(f) (S) ( 1 (S),..., (S) ) 1 Similarly, Papadimitriou [Pap98] introduced the now popular measure price of anarchy, which corresponds to the inefficiency degree for the profit metrics. Properties parameters: Consider again a system with two applications and two machines with b 1 = 1 w 1 = N b 2 = N w 2 = 1 B A = 100 W A = 100 N B B = 100 N W B = 100 INRIA

23 Non-cooperative scheduling of multiple bag-of-tas applications α Nash Equilibria α 1 Figure 4: Pareto-inefficiency of the Nash Equilibrium on a two-machines/twoapplications system Consider then the non-cooperative approach. We have W A = and B B =. We have: However, n,1 = B A + W ( B = 50 2b 1 2w 1 n,2 = B A 2b 2 + W B 2w 2 = 50 ) N ( ) N α (coop) n,1 = 100 α (coop) n,2 = 100. As the utility set is clearly symmetrical for this system (see Figure 4), and the efficiency measure do not favor a particular application, the optimal point for all the previous measures is α (coop). As α(coop) n,1 n,1 = α(coop) n,2 n,2 N + 2, the efficiency degree of all previous measures is larger than two (except for the Proportional measure that is equal to 4). 4.4 Braess-lie paradoxes When studying properties of Nash equilibria in routing systems, Braess exhibited an example in which, by adding resource to the system (in his example, a route), the performance of all the users were degraded. We investigate in this section whether such situations can occur in our considered scenario. Based on the definition of the Pareto optimality, we define the concept of strict Pareto-superiority. Definition 5. We say that a utility point a is strictly Pareto-superior to a point b is for all players a i < b i. RR n 5819

24 22 A. Legrand, C. Touati Obviously, a Pareto-optimal point is such that there exists no achievable point strictly-pareto superior to it. Let us consider a system (called initial ) and a second one (referred to as the augmented system), derived from the first one by adding some quantity of resource. Intuitively, the Nash equilibrium aug in the augmented system should be Pareto-superior to the one in the initial system ini. We say that a Braess paradox happens when ini is strictly Pareto-superior to point aug. Obviously, every achievable state in the initial system is also achievable in the augmented system. Hence if a is an achievable point in the initial system and if b is a Pareto optimal point is the augmented one, then a cannot be strictly Pareto superior to b. Hence Braess paradoxes are consequences of the Pareto inefficiencies of the Nash equilibria Property We show that, even though the Nash equilibria may not be Pareto optimal, in the considered scenario, Braess paradoxes cannot occur. Theorem 6. In the non-cooperative multi-port scheduling problem, Braess lie paradoxes cannot occur. To prove this proposition, we need to use two lemmas and the following definition. Definition 6 (Equivalent subsystem). Consider a system S = (, b, w, N, B, W). We define the new subsystem S = (, b, w, N, B, W) by: For each worer n, { W Bn c n = if W n =, otherwise. W n and Bn = { B n W n if B n =, otherwise. We shall now precise why S is said to be an equivalent subsystem of S. Lemma 2 (Equivalent subsystem). Consider a system S = (, b, w, N, B, W) and its Nash equilibrium. i) The system S is a subsystem of S, i.e. for all worer n: Bn B n and W n W n. ii) The Nash equilibrium of the subsystem S verifies: n,, = α(nc) iii) The Nash equilibrium of the subsystem S is Pareto-optimal. Proof. Let n 1; N. i) If W n =, then using theorem 2, we now that Cn W n. If W n, then W n = W n, hence the result. Similarly, if B n =, then using theorem 2, we now that B n = W n B n. If B n, then B n = B n, hence the result., hence W n = Bn C n, hence INRIA

25 Non-cooperative scheduling of multiple bag-of-tas applications 23 ii) If W n =, then using theorem 1, we now that B n = B n. Therefore ec n = Bn = 1, hence W n = and = e B n.b.b. As W n =, we have = Bn.b =. If B n =, then using theorem 1, we now that = Wn.w. As B n =, we have W n = W n. Therefore e C n = 1, hence B n = and = f W n.w = Wn.w Last, if neither B n = nor W n =, then (B n, W n ) = ( B n, W n ), and =. = α(nc). iii) Finally, note that n, W n and B n. Hence, from Prop. 4 the Nash equilibrium is Pareto optimal. Lemma 3. Consider two systems S = (, b, w, N, B, W) and S = (, b, w, N, B, W ) and their respective equivalent subsystems S = (, b, w, N, B, W) and S = (, b, w, N, B, W ). Suppose that n, B n B n and W n W n then n, B n B n and W n W n. Proof. Let and be the Nash equilibrium of S and S. If is Pareto optimal then B n = B n and W n = W n. Hence, n, B n B n B n = B n. Similarly n, W n W n hence the result. Suppose that is not Pareto optimal. Let n 1, N. If W s and B s then BB s = B s and W W s = W s. Hence BB s B s B s = BB s, similarly W W s W W s. If B s = then W W s = B s/. But, as B s B s, then B s / W W s. P As W Ws BB s (from the construction of (N, BB, W W )), then W W s W W s and BB s B s B s = BB s. Similarly, if B n =, then BB s = W s P 1/c. As W W then BB s BB s while W W s W s W s = W W s. We can then finally prove Theorem 6: Proof. Consider a user-system S = (, b, w), and two physical-systems: the initial system (N, B, W) and its Nash equilibrium and a second system obtained by adding some quantity of resource to the first one (N, B, W ), and its Nash equilibrium. We want to show that cannot be Pareto superior to. Suppose that the second system is obtained by adding some machines to the system (i.e. N > N). Then, as the non cooperative game at each machine is independent, the equilibrium on the N original machines will not be affected by the new machines. On the other hand, as in the new machine the allocation of throughput of each application will be strictly positive, then necessary is strictly Pareto superior to. Suppose now that the second system consists of N machines with respective bandwidth and computational capacities B and W. By definition of the augmented system we have n 1, N, B n B n and W n W n. RR n 5819

26 24 A. Legrand, C. Touati Consider the equivalent subsystems of the initial and the augmented systems (N, B, W) and (N, B, W ) (defined as in Lemma 2). Then, from Lemma 3, we have n, B n B n and W n W n. As the Nash equilibria α(nc) and are both Pareto optimal in these systems, then cannot be Pareto superior to. 5 Performance measures In this section we show that unexpected behavior of some typical performance can occur even for Pareto optimal situations. To ensure optimality of the Nash equilibrium, we consider applications running on a single processor (Prop. 3). We recall that the Pareto optimality is a global performance measure. Yet, it is possible that, while the resources of the system increase (either by the adding of capacity to a lin or of computational capabilities to a processor), the performance measure of a given application decreases. The aim of this section is to illustrate this phenomenon on some typical performance measures. More precisely, we show through examples the non-monotonicity of the maximal throughput (Subsection 5.2), of the minimal throughput (Subsection 5.3) and of the average throughput (Subsection 5.4). We finally end this section with an example where all these performance measures decrease simultaneously with the increase of the resources. We can note that all of these performance measures are consequences of the nonmonotonicity of the throughput of a given application ( ), which we hence analytically study (Subsection 5.1). In the following, we suppose that only one of the resource of the system increases. By symmetry, we suppose that the computational capacity (W n ) is constant, while the lin capacity B n increases. 5.1 Variation of the throughput of a given application Even on a single processor, the throughput of the application is not a increasing function. Even worse, the degradation can be arbitrarily large. When considering the equations at the Nash equilibrium (1), we can distinguish 3 cases : 2 saturated situations that are : satw n If W n = (i.e. B n W n / 1 ), then = Bn.b, i.e. the throughput of each application is proportional to B n. We will note satw n = ]0, W n / 1 ]. c satb n If B n = (i.e. W n B n), then = Wn.w, i.e. the throughput of each application is constant with respect with B n. We will note satb n = ], + ]. W n 1 continuous situation when W n / 1 < B n < W n. INRIA

27 Non-cooperative scheduling of multiple bag-of-tas applications 25 Obviously, in the saturated situations, the throughput are increasing or constant and the order between the applications is preserved (i.e. if for B n satw n (resp. satb n ), 1 2 then for all B n satw n (resp. satb n ) we have 1 2. To simplify the analysis, we consider the degradation obtained when B n = W n compared to the situation where B n = W n / 1. It is hence a lower bound on the actual maximum achievable degradation. Consider now a system of applications. Suppose that W n is given. For B n = 1, then, for all, = Bn b = P Wn b. When B n is larger than W n, p cp W n / then = Wn w. Hence: before = after p /c p Suppose now that p, c p = and = 1. Then α before α after /2. Hence, when the number of applications grows to infinity, the degradation of the application having the smaller also grows to infinity. Remar 2. The applications with the smaller coefficient is the most penalized by an increase of the communication resource. 5.2 Maximal performance We show in this section that even in a single processor system, the maximal throughput can strictly decrease with the adding of resource, and illustrate it with a numerical example (Fig. 5) α 1 α 2 α 3 α b = {5, 4, 6, 2}, w = {3, 4, 7, 5}, = 4, W n = B n Figure 5: The maximal throughput can decrease while the resource (the bandwidth) increases. Theorem 7. In a system with applications and a single processor, for a given W n, RR n 5819

28 26 A. Legrand, C. Touati P if B n Wn the application having the higher throughput having the smaller value of b. if B W n the application having the higher throughput α(nc) whose w is the smallest. is the one is the one Additionally, for given values of, the degradation of the higher throughput when B n increases can be unbounded. More precisely, a lower bound of the maximal degradation is min proportional to w 1/c. Hence, for appropriate choices of min b and min w min b (for all fixed), the degradation can be chosen arbitrarily large. Proof. Let us consider a system with applications, with given values of and a given value of W n. Then, when B n = W n / 1 then W n = and max = B n. min b = W n (min b ) 1 and when B n = W n we have B n = and max = W n. min w Hence, the application having the higher throughput can be different in satb n and satw n (as illustrated in Fig 5) and the maximum degradation when B grows from satb n to min satw n is hence w. 1/c min b Consider for example the example depicted in Fig. 5. As max w = w 1 and min b = b 4, then the application having the higher throughput is application 1 in satb n and application 4 in satw n, and a lower bound of the degradation is 90/ Minimal performance We show here a similar result as what we obtained in Section 5.2 for the minimal throughput. That is, even in a single processor system, the minimal performance can decrease with an increase of the resource. We illustrate this property with a numerical example (Fig. 6). Theorem 8. In a system with applications and a single processor, for a given W n, if B n W n / 1 the application having the smaller throughput one having the largest value of w. W n if B n the application having the lower throughput α(nc) whose b is the largest. is the is the one Additionally, for given values of, the degradation of the smaller throughput when B increases can be unbounded. More precisely, a lower bound of the maximal degradation is max proportional to w. Hence, for appropriate choices of max b and max w 1/c max b (for all fixed), the degradation can be chosen arbitrarily large. INRIA

29 Non-cooperative scheduling of multiple bag-of-tas applications α 1 α 2 α 3 α b = {5, 7, 10, 4}, w = {4, 7, 14, 6}, = 4, W = B n Figure 6: The minimal throughput can decrease with the resource (here the bandwidth) Proof. Let us consider a system with applications, with given values of and a given value of W n. Then, when B n = W n / and when B n = 1 then W n = and min W n = B n max b = we have B n = and min = W n. max w W n max b. 1/ Hence, the application having the smaller throughput can be different in satb n and satw n (as illustrated in Fig 6) and the maximum degradation when B n grows from max satb n to satw n is hence w. 1/c max b Consider for example the example depicted in Fig. 6. As argmax{w } = argmax{b } = 3, then the application having the lower throughput is application 3 in both satb n and satw n, and a lower bound of the degradation is 56/ Average performance In this section we focus on the monotonicity of the average throughput, and illustrate our result with a numerical example (Fig. 7). Theorem 9. Consider, as a performance measure, the average of the applications throughputs. Then, the degradation of these performance measure when B increase can be arbitrarily large. Proof. Consider a system with applications, with given values of and a given value of W n. Then, when B n = W n / then W n = and α(nc) P 1 = Wn 1/ 1 b RR n 5819

30 28 A. Legrand, C. Touati and when B n = W n we have B n = and α(nc) performance degradation becomes p bef p aft = w b 1 b 1 w. applications are such that b 1 = w 1 = 1 and 1, b = P 2 p bef bound becomes: = P ( 2 ) p aft P P Consider the example depicted in Fig / = Wn 1 w. Hence, the Suppose further that the w = P. Then, the lower A lower bound of the degradation is α 1 α 2 α 3 α 4 α b = { 15,17,2,1}, w = { 14,17,3,2}, =4, W n =10. B n Figure 7: The average throughput can decrease with the resource (here the bandwidth) 5.5 All measures considered simultaneously We end this section with an example in which all the performance measures we considered are simultaneously degraded when the bandwidth B n of the lin connecting the master to the slave is increased. Consider the example represented in Fig. 8. In this example, when the bandwidth B is 9.5 the three measures (namely the higher throughput, the lower throughput and the average throughput) have lower values than when the bandwidth B is only equal to Conclusion We have presented a simple yet realistic situation where the system-level fairness fails to achieve a relevant application-level fairness. Even though the system achieves a perfect sharing of resources between applications, the non-cooperative usage of the system leads to important application performance degradation and resource wasting. We have proved the existence and uniqueness of the Nash equilibrium in our framewor and extensively studied its property. Surprisingly, the equilibrium is Pareto-optimal on each worer independently. However, it may not be globally Pareto-optimal. We have proved that no INRIA