Part II: Integral Splittable Congestion Games. Existence and Computation of Equilibria Integral Polymatroids

Kombinatorische Matroids and Polymatroids Optimierung in der inlogistik Congestion undgames im Verkehr Tobias Harks Augsburg University WINE Tutorial, 8.12.2015

Outline Part I: Congestion Games Existence of Equilibria Computation of Equilibria Matroids Part II: Integral Splittable Congestion Games Existence and Computation of Equilibria Integral Polymatroids Part III: Nonatomic Congestion Games Efficiency of Equilibria The Braess Paradox Matroids are Immune to Braess Paradox

Strategic Games Strategic game G = (N, X, π) N = {1,..., n} set of players X = i N X i set of pure strategies x = (x 1,..., x N ) strategy profile π i (x) : X R, i N private cost/utility

Strategic Games Strategic game G = (N, X, π) N = {1,..., n} set of players X = i N X i set of pure strategies x = (x 1,..., x N ) strategy profile π i (x) : X R, i N private cost/utility Mixed strategy: for each player a probability distribution over pure strategies

Solution Concept Definition Pure Nash equilibrium (PNE): no player has an incentive to unilaterally deviate. Definition Mixed Nash equilibrium (MNE): no player has an incentive to unilaterally change her mixed strategy. Minimax Theorem John von Neumann (1928), Nash (1950)

Motivation: Party Affiliation Games each strategy set X i = {1, 1} Weight w i,j measures relationship between i and j payoff u i (x) = j N x i x j w i j max u 2 (x) = 0 2 1 u 1 (x) = 1 1 3 2 1 u 4 (x) = 0 4 1 5 1 1 2 3 3 u 5 (x) = 5 u 3 (x) = 0

Solution Concept: Pure Nash Equilibrium Definition Pure Nash equilibrium (PNE): no player has an incentive to unilaterally change his pure strategy. Definition Mixed Nash equilibrium (MNE): no player has an incentive to unilaterally change his mixed strategy.

Nash (1951) Existence of Nash Equilibria Theorem Every finite game possesses a mixed Nash equilibrium. Pure Nash Equilibrium need not exist! Example: Assymmetric Party Affiliation Game 1 1 2 1 In the mixed Nash equilibrium, each player chooses each party with probability 1/2.

Part I Congestion Games

Congestion Games Modell N = {1,..., n} set of players R = {r 1,..., r m } set of resources X = i N X i set of strategy profiles with X i 2 R Strategy profile x = (x 1,..., x n ) X Load of a resource x r = {i N : r x i } for x X Cost functions c r : N R nondecreasing (convex) private cost: π i (x) = r x i c r (x r )

Example

Fundamental Questions 1 When do pure Nash equilibria exist? 2 How do players find them? 3 How difficult is it to compute them?

Potential Functions Definition (Exact potential function) P : X 1 X n R If a player changes his action, the change in the potential function value is equal to the change in her payoff. b Potential u i (x i, x i ) u i (y i, x i ) = P(x i, x i ) P(y i, x i ) u i (x i, x i ) u i (y i, x i ) = b i (P(x i, x i ) P(y i, x i )) Monderer and Shapley (1996)

Merits of Potential Games Theorem Every finite exact potential game (with potential P) possesses a PNE every sequence of improving moves is finite (FIP) every local minimum of P is a PNE Monderer and Shapley (1996)

Proof path γ = (x 0, x 1,..., ) sequence of unilateral moves improvement path γ = (x 0, x 1,..., ) sequence of unilateral improving moves γ = x 0, x 1,... improvement path P(x 0 ) > P(x 1 ) > > must be finite.

Congestion Games are Potential Games Theorem (Rosenthal 73) Every congestion game admits an exact potential function possesses a PNE possesses the Finite Improvement Property, that is, every sequence of improving moves is finite.

Proof Rosenthal s exact potential function P : X 1 X n R is defined as P(x) := r R x r k=1 c r (k). (1) Let x X and y i x i be a unilateral deviation of i. u i (x i, y i ) u i (x) = c r (x r + 1) c r (x r ). r y i r x i r / x i r / y i The potential of (x i, y i ) is given by: P(x i, y i ) = r R x r k=1 c r (k) + c r (x r + 1) c r (x r ) r y i r x i r / x i r / y i }{{} =u i (x i,y i ) u i (x) = P(x) + u i (x i, y i ) u i (x).

Complexity of Computing PNE Theorem (Fabrikant et al. 04, Ackermann et al. 08) It is PLS-complete to compute a PNE even for symmetric congestion games with affine costs.

Complexity of Computing PNE Theorem (Fabrikant et al. 04, Ackermann et al. 08) It is PLS-complete to compute a PNE even for symmetric congestion games with affine costs. Theorem (Fabrikant et al. 04) For symmetric network congestion games, there is a polynomial time algorithm to compute a PNE. Subdivide each arc e into n parallel arcs with capacity 1 each and assign costs c ei = c e (i) for i {1,..., n}. Remark (Ackermann et al. 08) There are instances on which every best response dynamic needs exponential convergence time.

Introduction Matroids Definition (Matroid) A matroid is a pair M = (R, I) where R is a set of resources, and I is a family of subsets of S such that: 1 I. 2 If I J and J I, then I I. 3 Let I, J I and I < J, then there exists an x J \ I such that I + x I. A set system R, I that only satisfies (1) and (2) is called an independence system.

Example: Uniform Matroids The independent sets of a k-uniform matroid are the sets that contain at most k elements. Example 4 resources: {1, 2, 3, 4}

Example: Uniform Matroids The independent sets of a k-uniform matroid are the sets that contain at most k elements. Example 4 resources: {1, 2, 3, 4} Independent sets of the 3-uniform matroid: I = {, {1}, {2}, {3}, {4}, {1, 2}, {1, 3}, {1, 4}, {2, 3}, {2, 4}, {3, 4}, {1, 2, 3}, {1, 2, 4}, {1, 3, 4}, {2, 3, 4}}

Graphic Matroid (Cycle Matroid) 3 6 2 4 5 1 Figure : K 4 with two bases B 1 (red), B 2 (blue).

Fundamental Results Theorem Let (R, I) be an independence system. The, the following is equivalent: M1 M = (R, I) is a matroid. M2 I, J I with I = J + 1 there is x I \ J with J + x I. M3 For every I R, every basis of I has the same cardinality (a basis of I E is an inclusion-wise maximal set in I ). Proof. (M1) (M2) is trivial. (M1) (M3) ist trivial. (M3) (M1): Let I, J I with I > J. With (M3) we have that J is no base of I J. Hence, there is x (I J) \ J = I \ J I with J + x I.

Fundamental Results Theorem (Basis exchange theorem) Let (R, I) be a matroid with basis system B. Then, 1 B 2 For every B 1, B 2 B and x B 1 \ B 2 there is y B 2 \ B 1 such that B 1 x + y B. Proof. The bases set of (R, I) satisfies (1) since I. For condition (2) let B 1, B 2 B and x B 1 \ B 2. Since B 1 x I we can use (M2): B 1 x + 1 = B 2 hence there is y B 2 \ (B 1 x) with B 1 x + y I. As all bases have the same cardinality (see (M3)) we get B 1 x + y B.

1 2 5 3 4 1 2 5 1 3 4 Figure : Two bases B (red) and B (blue) of a graphic matroid

1 2 5 3 4 1 2 5 1 3 4 Figure : Two bases B (red) and B (blue) of a graphic matroid 5 2 B \ B 4 3 B \ B Figure : Bipartite graph G(B B ) = (B B, R). (x, y) R B x + y B.

Perfect matching in G(B B ) Theorem (Matching Property) Let M = (R, I) be a matroid. Then, for every pair (B, B ) of bases of M there exists a perfect matching W (B, B ) in G(B B ).

Matroid Congestion Games Matroid Congestion Games N = {1,..., n} set of players R = {r 1,..., r m } set of resources X i 2 R with X i = B i for M i = (R, I i ) private cost for B = (B 1,..., B n ) B : π i (B) = c r (x r ) r B i

Best-Response - Polynomial Time Theorem (Ackermann, Röglin, Vöcking 08) For matroid congestion games, the best-response dynamic converges after at most n 2 m max i N rk i n 2 m 2 steps.

Best-Response - Polynomial Time Theorem (Ackermann, Röglin, Vöcking 08) For matroid congestion games, the best-response dynamic converges after at most n 2 m max i N rk i n 2 m 2 steps. Let L be a list of all cost values c r (i), r R = {r 1,..., r m }, i N = {1,..., n} sorted in non-decreasing order. Define alternative cost function c r : N {1,..., n m}, where i [0, n] is mapped to list position of c r (i)in L (same costs are mapped to same position).

Lemma Let B i be a best-response w.r.t. B B with B i B i. Then B i decreases also the private costs w.r.t. c.

Lemma Let B i be a best-response w.r.t. B B with B i B i. Then B i decreases also the private costs w.r.t. c. Consider G(B i B i) with perfect matching N. Let (u, v) N, i.e., v B i \ B i, u B i \ B i and B i v + u B i. For B = (B i, B i) with load vector x we have c v (x v ) c u (x u + 1) for all (u, v) N, as else B i := Bi v + u would give less costs than Bi. There must be (v, u) N with c v (x v ) < c u (x u + 1), since Bi strictly decreases the private cost for i. Thus, also the private costs under c must decrease.

Proof Consider P w.r.t. c. We get c r (i) n m for all r R and all 1 i n.

Proof Consider P w.r.t. c. We get Hence, c r (i) n m for all r R and all 1 i n. P (B) = r R x r i=1 c r (i) r R x r i=1 n m n 2 m max i N rk i.

Proof Consider P w.r.t. c. We get Hence, c r (i) n m for all r R and all 1 i n. P (B) = r R x r i=1 c r (i) r R x r i=1 Remark (Ackermann, Röglin, Vöcking 08) n m n 2 m max i N rk i. For every non-matroid set system X i, i N, there are isomorphic instances with exponentially long best-response sequences.

Part II Integral Splittable Congestion Games

Integral Splittable Congestion Games Integral Congestion Games N = {1,..., n} set of players R = {r 1,..., r m } set of resources X i 2 R set of allowable subsets Demands d i N Strategy corresponds to integral distribution of d i among the sets in X i cost of a resource c r (x) = c r ( i N x i,r ) private cost π i (x) = r R c r (x)x i,r

A Counter-Example Rosenthal (1973b) One player from A to C with demand 2 ( two taxicaps ) One player from A to B with demand 1

Positive Results Theorem (Tran-Thanh et al. (2011)) If X i consists of singletons, then there is a PNE.

Maximal Structures What are maximal structures of X i such that the existence of PNE is guaranteed?

Maximal Structures What are maximal structures of X i such that the existence of PNE is guaranteed? Theorem (H, Klimm, Peis (2014)) Polymatroids are the maximal structure.

Submodular Functions Definition (integral, submodular, monotone, normalized) f : 2 R N is submodular if f (U) + f (V ) f (U V ) + f (U V ) for all U, V 2 R. f is monotone, if U V f (U) f (V ) f is normalized, if f ( ) = 0. Example rank function rk : 2 R N of a matroid M = (I, R) for F R : rk(f ) := max{ B, B basis of F }

Integral Polymatroids integral polymatroid { P f := x N R } x r f (U) for all U R. r U

Integral Polymatroids integral polymatroid { P f := x N R } x r f (U) for all U R. r U truncated integral polymatroid P f (d) := {x N R r U x r f (U) for all U R, r R x r d}.

Integral Polymatroids integral polymatroid { P f := x N R } x r f (U) for all U R. r U truncated integral polymatroid P f (d) := {x N R r U x r f (U) for all U R, r R x r d}. integral polymatroid base polytope { B f (d) := x N R x r f (U) for all U R, } x r = d. r U r R

Congestion Games on Polymatroids Strategic Game G = (N, X, π) X i = B f (i)(d i ) π i (x) = r R c i,r (x)x i,r B f (i)(d i ) = { x i N R x i,r f (i) (U) for all U R, } x i,r = d i r U r R

Examples Singletons f (i) ({r}) = d i falls r R i und f (i) ({r}) = 0, sonst. Bases of Matroids f (i) is rank function of some matroid M i = (R i, I i ) and d i = rk i (R i ). Spanning Trees R edges of a graph R i R connected subgraph Bases of M i = (R i, I i ) are spanning trees of R i

Existence Proof Induction over d = i N d i d = 1 trivial d d + 1 let x be a PNE for a game with demand d In step d d + 1 exactly for one player i: di d i + 1.

Existence Proof (contd.) Hamming Distance of x, y N R : H(y, x) = r R y r x r Sensitivity Lemma - Increased Demand Let x be given. [ There exists a best response ] y i argmin π i (y i, x i ) s.t. y i B f (i)(d i + 1) with H(x i, y i ) = 1.

Existence Proof (contd.) Sensitivity Lemma - Increased Load Let a r = i j x i,r, r R be given and x j be a best response to a. If a r a r + [ 1 for one r R, then there ] exists a best response y j argmin π j (y j, a) s.t. y j B f (j)(d j ) with H(x j, y j ) {0, 2}.

Existence Proof (contd.) Invariant There is a sequence of best-responses y k with H(y 0 = y, y k ) = r R y r x r = 2.

Existence Proof (contd.) Invariant There is a sequence of best-responses y k with H(y 0 = y, y k ) = r R y r x r = 2. Convergence The sequence y k is finite. Define marginal costs of every unit of every player Sorted vector of marginal costs decreases lexicographically Remark For non-polymatroid X i, there are counterexamples.

Part III Nonatomic Congestion Games - The Braess Paradox

Braess Paradox via Cost Reductions x 1 x 1 s 1 x t s 1 0 t x Left equilibrium: C(x) = ( 1 2 1 2 + 1 2 1) 2 = 3/2 Right equilibrium: C(x) = 1 1 + 1 1 = 2

Braess Paradox via Cost Reductions x 1 x 1 s 1 x t s 1 0 t x Left equilibrium: C(x) = ( 1 2 1 2 + 1 2 1) 2 = 3/2 Right equilibrium: C(x) = 1 1 + 1 1 = 2 G is immune to BP-free iff G is series-parallel [Milchtaich, 06, Chen et al. 15]. What about other strategy spaces: tours in a graph scheduling jobs on machines spanning trees Steiner trees

Nonatomic Congestion Model Model M N = {1,..., n} populations, represented by [0, d i ], i N R = {r 1,..., r m } resources Strategies S i 2 R with S = i N S i Strategy distribution (x S ) S Si with S S i x S = d i Load of overall distribution x: x r = S S:r S x S Cost function c r : R + R + nondecreasing

Nonatomic Congestion Games If player from population i picks S she gets disutility: π i,s (x) = r S c r (x r ). Definition (Wardrop Equilibrium) π i (x) := r S i c r (x r ) r S i for any S i, S i S i with x Si > 0, for all i N. c r (x r )

Braess Paradox via Cost Reductions x 1 x 1 s 1 x t s 1 0 t x Figure : Example of the Braess paradox.

Braess Paradox via Demand Reductions x t 2, s 3 c(x) s 1, s 2 2 c(x) 1 t 1, t 3 M M + 1 x d 1 = 1, d 2 = 2, d 3 = M with cost C(x ) = 1 2 + 2 2 + M 0 = 6 Reducing d 2 0: C(x new ) = M + 2

Weak/Strong Braess Paradox Let x and x be Wardrop equilibria before and after a cost/demand reduction, resp. Definition (S i ) i N admits the 1 weak BP, if there exists a resource r R with c r ( x r ) > c r (x r ). 2 strong BP, if there exists a population i N with π i ( x) > π i (x).

Main Result Q: What is the maximal combinatorial structure of (S i ) i N which is immune to the weak or strong Braess paradox?

Main Result Q: What is the maximal combinatorial structure of (S i ) i N which is immune to the weak or strong Braess paradox? A: Bases of matroids (and only matroids!) (jointly with S. Fujishige, M. Goemans, B. Peis and R. Zenklusen) Theorem Let (S i ) i N be a family of set systems. Then, the following statements are equivalent. (I) For all i N, the clutter cl(s i ) consists of the base sets of a matroid M i = (R, I i ). (II) (S i ) i N is immune to the weak (and strong) Braess paradox.

Compact Representations of Strategies P(M) := x RS 0 x S = d i for all i N. S S i

Compact Representations of Strategies P(M) := x RS 0 x S = d i for all i N. S S i Resource-based polytope P(M) R R 0 P(M) := x S χ S i N S S i x P(M), χ S {0, 1} R for S R is the characteristic vector of S. Theorem (Beckmann et al. 56) x is a Wardrop equilibrium if and only if it solves min x P(M) Φ(x) := x r c r (t) dt. (2) r R We call Φ the Beckmann potential. 0

Polymatroids and Submodular Functions Definition (submodular, monotone, normalized) f : 2 R R is submodular if f (U) + f (V ) f (U V ) + f (U V ) for all U, V 2 R. f is monotone, if U V f (U) f (V ) f is normalized, if f ( ) = 0. P h := { } x R R + x(u) h(u) U R, x(r) = h(r), for U R, x(u) := r U x r. Matroid Base Polytope P di rk i = { } x i R R + x i (U) d i rk i (U) U R, x i (R) = d i rk i (R) P(M) := i N P d i rk i = P i N d i rk i

Sensitivity in Polymatroid Optimization min x P h Φ(x) := r R x r 0 c r (t) dt, (3) where P h is a polymatroid base polytope with rank function h and for all r R, c r : R 0 R 0, are non-decreasing and continuous functions. Optimality Conditions x P h is optimal if and only if c e (x e ) c f (x f ) for all e, f R, x e > 0 with x := x + ɛ(χ f χ e ) P h for some ɛ > 0.

Matroids are Immune to Braess Paradox To show c e ( x e ) c e (x e ) for all e R. Assume c e ( x e ) > c e (x e ) for some e R.

Matroids are Immune to Braess Paradox To show c e ( x e ) c e (x e ) for all e R. Assume c e ( x e ) > c e (x e ) for some e R. Then x e > x e.

Matroids are Immune to Braess Paradox To show c e ( x e ) c e (x e ) for all e R. Assume c e ( x e ) > c e (x e ) for some e R. Then x e > x e. Strong exchange property of polymatroids: f R e with x f < x f und ɛ > 0 such that x ɛχ f + ɛχ e P h and x ɛχ e + ɛχ f P h. Wardrop equilibrium condition: 1 c f (x f ) c e (x e ), as x WE for c,

Summary Part I matroids lead to fast convergence of best-response in congestion games Part II polymatroids allow PNE in integral splittable congestion games Part III matroids are immune to Braess Paradox