LEARNING IN CONCAVE GAMES
|
|
- Theodore Lloyd
- 6 years ago
- Views:
Transcription
1 LEARNING IN CONCAVE GAMES P. Mertikopoulos French National Center for Scientific Research (CNRS) Laboratoire d Informatique de Grenoble GSBE ETBC seminar Maastricht, October 22, 2015
2 Motivation and Preliminaries Learning Perspectives Context and motivation Concave games: finitely many players continuous action spaces individually concave payoff functions
3 Motivation and Preliminaries Learning Perspectives Context and motivation Concave games: finitely many players continuous action spaces individually concave payoff functions Context/applications: Standard in economics & finance (multi-portfolio optimization, auctions, oligopolies, ) Networking (routing, tolling, network economics, ) Electrical engineering (wireless communications, electricity grids, )
4 Motivation and Preliminaries Learning Perspectives Context and motivation Concave games: finitely many players continuous action spaces individually concave payoff functions Context/applications: Standard in economics & finance (multi-portfolio optimization, auctions, oligopolies, ) Networking (routing, tolling, network economics, ) Electrical engineering (wireless communications, electricity grids, ) What this talk is about: Distributed learning algorithms that allow players to converge to an equilibrium state.
5 Basic Definitions A concave game consists of: A finite set of players N = {1,..., N}. A compact, convex set of actions x k X k per player. An individually concave payoff function u k k X k R per player, i.e. u k (x k ; x k ) is concave in x k for all x k l k X l. Each player seeks to maximize his individual payoff. Fine print: Each X k assumed to live in a finite-dimensional ambient space V k R d k. (No infinite dimensionalities in this talk) Each ambient space V k equipped with a norm.
6 Example 1: Finite (Affine) Games A finite game consists of: A finite set of players N = {1,..., N}. A finite set of actions α k A k per player. Each player s payoff function u k k A k R. In the mixed extension of a finite game, players can play mixed strategies x k (A k ). Corresponding (expected) payoff: u k (x) = α1 A 1 αn A N x 1,α1 x N,αN u k (α 1,..., α N ) The mixed strategy space X k = (A k ) is convex and u k (x k ; x k ) is linear in x k mixed extensions of finite games are concave
7 Example 2: Routing Consider the following model of Internet congestion: Origin nodes (v k ) generate traffic that must be routed to intended destination nodes (w k ) v 1 A E w 1 D w 2 B v 2 C
8 Example 2: Routing Consider the following model of Internet congestion: Origin nodes (v k ) generate traffic that must be routed to intended destination nodes (w k ) Set of paths A k joining v k w k. v 1 A E w 1 D w 2 B v 2 C
9 Example 2: Routing Consider the following model of Internet congestion: Origin nodes (v k ) generate traffic that must be routed to intended destination nodes (w k ) Set of paths A k joining v k w k. Actions: traffic distributions over different paths X k = {x k x kα 0 and α Ak x kα = ρ k } v 1 A x 1,A E x 1,BE w 1 D x 2,ED w x 2 2,C C B v 2
10 Example 2: Routing Consider the following model of Internet congestion: Origin nodes (v k ) generate traffic that must be routed to intended destination nodes (w k ) Set of paths A k joining v k w k. Actions: traffic distributions over different paths X k = {x k x kα 0 and α Ak x kα = ρ k } Path latency: l kα (x) r α l r(y r) where y e = k α e x kα is the total load on link e and l e (y e ) is the induced delay. v 1 A x 1,A E x 1,BE B w 1 v 2 D x 2,ED w x 2 2,C C Congested
11 Example 2: Routing Consider the following model of Internet congestion: Origin nodes (v k ) generate traffic that must be routed to intended destination nodes (w k ) Set of paths A k joining v k w k. Actions: traffic distributions over different paths X k = {x k x kα 0 and α Ak x kα = ρ k } Path latency: l kα (x) r α l r(y r) where y e = k α e x kα is the total load on link e and l e (y e ) is the induced delay. Payoff: u k (x) = α Ak x kα l kα (x). v 1 A x 1,A E x 1,BE B w 1 v 2 D x 2,ED w x 2 2,C C Congested
12 Example 2: Routing Consider the following model of Internet congestion: Origin nodes (v k ) generate traffic that must be routed to intended destination nodes (w k ) Set of paths A k joining v k w k. Actions: traffic distributions over different paths X k = {x k x kα 0 and α Ak x kα = ρ k } Path latency: l kα (x) r α l r(y r) where y e = k α e x kα is the total load on link e and l e (y e ) is the induced delay. Payoff: u k (x) = α Ak x kα l kα (x). v 1 A x 1,A E x 1,BE B w 1 v 2 D x 2,ED w x 2 2,C C Congested Under standard assumptions for l e (convex, increasing), u k is concave in x k G(N, X, u) is a concave game
13 Nash Equilibrium and Payoff Gradients A Nash equilibrium is an action profile x X such that u k (x k ; x k) u k (x k ; x k) for every unilateral deviation x k X k, k N There is no direction that could unilaterally increase a player s payoff.
14 Nash Equilibrium and Payoff Gradients A Nash equilibrium is an action profile x X such that u k (x k ; x k) u k (x k ; x k) for every unilateral deviation x k X k, k N There is no direction that could unilaterally increase a player s payoff. Alternative characterization: consider the individual payoff gradient of player k: v k (x) = k u k (x) u k (x k ; x k ), (differentiation taken only w.r.t. x k ; the opponents profile x k is kept fixed) Since u k is concave in x k, x is an equilibrium if and only if v k (x ) z k 0 for every tangent vector z k TC k (x k ), k N. Fine print: x k X k V k treated as primal variables; payoff gradients v k V k treated as duals and assumed Lipschitz.
15 Equilibrium Existence and Uniqueness Every concave game admits a Nash equilibrium (Debreu, 1952; Rosen, 1965). What about uniqueness?
16 Equilibrium Existence and Uniqueness Every concave game admits a Nash equilibrium (Debreu, 1952; Rosen, 1965). What about uniqueness? Theorem (Rosen, 1965) Suppose that the players payoff gradients satisfy the monotonicity property: k λ k v k (x ) v k (x) x k x k < 0 (R) for some λ 0 and for all x x X. Then, the game admits a unique Nash equilibrium.
17 Equilibrium Existence and Uniqueness Every concave game admits a Nash equilibrium (Debreu, 1952; Rosen, 1965). What about uniqueness? Theorem (Rosen, 1965) Suppose that the players payoff gradients satisfy the monotonicity property: k λ k v k (x ) v k (x) x k x k < 0 (R) for some λ 0 and for all x x X. Then, the game admits a unique Nash equilibrium. Rosen (1965) calls this condition diagonal strict concavity. Define the λ-weighted Hessian H(x; λ) = j,k H jk (x; λ) of the game as: H jk (x; λ) = λ j k v j (x) + λ k ( j v k (x)) If H(x; λ) 0 for all x X, the game admits a unique equilibrium.
18 Learning via Payoff Gradient Ascent Pavlov s dog reaction to improving one s payoffs: ascend the payoff gradient x x + γ v(x) where γ is a (possibly variable) step-size parameter. Problem: must respect players action constraints (x X )
19 Learning via Payoff Gradient Ascent Pavlov s dog reaction to improving one s payoffs: ascend the payoff gradient x x + γ v(x) where γ is a (possibly variable) step-size parameter. Problem: must respect players action constraints (x X ) To do that, rewrite the gradient ascent process y y + γ v(x), x y,
20 Learning via Payoff Gradient Ascent Pavlov s dog reaction to improving one s payoffs: ascend the payoff gradient x x + γ v(x) where γ is a (possibly variable) step-size parameter. Problem: must respect players action constraints (x X ) To do that, rewrite the gradient ascent process and project: y y + γ v(x), x arg min y x 2, x X
21 Learning via Payoff Gradient Ascent Pavlov s dog reaction to improving one s payoffs: ascend the payoff gradient x x + γ v(x) where γ is a (possibly variable) step-size parameter. Problem: must respect players action constraints (x X ) To do that, rewrite the gradient ascent process and project: y y + γ v(x), x arg max { y x 1 x 2 x 2 2}, X
22 Learning via Payoff Gradient Ascent Pavlov s dog reaction to improving one s payoffs: ascend the payoff gradient x x + γ v(x) where γ is a (possibly variable) step-size parameter. Problem: must respect players action constraints (x X ) To do that, rewrite the gradient ascent process and regularize: y y + γ v(x), x arg max { y x h(x )}, x X where the penalty function (or regularizer) h X R is smooth and strongly convex: h(tx + (1 t)x ) th(x) + (1 t)h(x ) 1 2 Kt(1 t) x x 2 for some K > 0 and for all t 0, x, x X.
23 Examples 1. The quadratic penalty h(x) = 1 2 α xα 2 = 1 2 x 2 2 gives the Euclidean projection Π(y) = arg max{ y x 1 2 x 2 2 } = arg min y x 2 2 x X x X
24 Examples 1. The quadratic penalty h(x) = 1 2 α xα 2 = 1 2 x 2 2 gives the Euclidean projection Π(y) = arg max{ y x 1 2 x 2 2 } = arg min y x 2 2 x X x X 2. If X = d, the (negative) Gibbs entropy h(x) = α x α log x α gives the logit map G(y) = (exp(y1),..., exp(y d)) d α=1 exp(y α)
25 Examples 1. The quadratic penalty h(x) = 1 2 α xα 2 = 1 2 x 2 2 gives the Euclidean projection Π(y) = arg max{ y x 1 2 x 2 2 } = arg min y x 2 2 x X x X 2. If X = d, the (negative) Gibbs entropy h(x) = α x α log x α gives the logit map G(y) = (exp(y1),..., exp(y d)) d α=1 exp(y α) 3. If X = {X X 0, tr(x) 1}, the von Neumann entropy h(x) = tr[x log X] gives Q(Y) = exp(y) 1 + tr[exp(y)]
26 Examples 1. The quadratic penalty h(x) = 1 2 α xα 2 = 1 2 x 2 2 gives the Euclidean projection Π(y) = arg max{ y x 1 2 x 2 2 } = arg min y x 2 2 x X x X 2. If X = d, the (negative) Gibbs entropy h(x) = α x α log x α gives the logit map G(y) = (exp(y1),..., exp(y d)) d α=1 exp(y α) 3. If X = {X X 0, tr(x) 1}, the von Neumann entropy h(x) = tr[x log X] gives Q(Y) = exp(y) 1 + tr[exp(y)] etc. Important: if dh(x) as x bd(x ), we say that h is steep. Steep penalty functions induce interior point methods: im Q = rel int X.
27 Learning via Mirror Descent Multi-agent mirror descent: y k (n + 1) = y k (n) + γ n v k (x(n)) x k (n + 1) = Q k (y k (n + 1)) (MD) where γ n is a variable step-size and the choice map Q k is defined as: Q k (y k ) = arg max x k X k { y k x k h k (x k )} Long history in optimization (Nemirovski, Yudin, Nesterov, Juditski, Beck, Teboulle, ) and, more recently, in machine learning (Shalev-Shwartz, ) Well-understood for single-agent problems
28 Learning via Mirror Descent Multi-agent mirror descent: y k (n + 1) = y k (n) + γ n v k (x(n)) x k (n + 1) = Q k (y k (n + 1)) (MD) where γ n is a variable step-size and the choice map Q k is defined as: Q k (y k ) = arg max x k X k { y k x k h k (x k )} Long history in optimization (Nemirovski, Yudin, Nesterov, Juditski, Beck, Teboulle, ) and, more recently, in machine learning (Shalev-Shwartz, ) Well-understood for single-agent problems Multi-agent problems (games):???
29 Variational Stability No uncoupled dynamics can always lead to equilibrium (Hart and Mas-Colell, 2003) must refine convergence target Definition We say that x X is variationally stable if k λ k v k (x) x k x k < 0 for some λ 0 and for all x x in a neighborhood U of x. If U = X, x will be called globally variationally stable.
30 Variational Stability No uncoupled dynamics can always lead to equilibrium (Hart and Mas-Colell, 2003) must refine convergence target Definition We say that x X is variationally stable if k λ k v k (x) x k x k < 0 for some λ 0 and for all x x in a neighborhood U of x. If U = X, x will be called globally variationally stable. Contrast with Nash equilibrium (which it refines): k λ k v k (x k ; x k) x k x k < 0 Compare with notion of (Taylor) evolutionary stability in multi-population games: k v k (x) x k x k < 0 for all x x near x Global/local ESSs are globally/locally variationally stable. Rosen s condition implies global variational stability.
31 Strict Nash Equilibrium Recall: a Nash equilibrium is an action profile x X such that v k (x ) z k 0 for every tangent vector z k TC k (x k ), k N. Definition x is called strict if the above inequality is strict for every nonzero z k TC k (x k ), k N.
32 Strict Nash Equilibrium Recall: a Nash equilibrium is an action profile x X such that v k (x ) z k 0 for every tangent vector z k TC k (x k ), k N. Definition x is called strict if the above inequality is strict for every nonzero z k TC k (x k ), k N. Some basics: Generalizes notion of strict equilibrium in finite games (pure, no payoff equalities). If x is a strict equilibrium, then x is variationally stable. If x is a strict equilibrium, then it is a corner of X (i.e. the tangent cone TC(x ) of X at x does not contain any lines).
33 Local convergence Proposition Suppose that (MD) is run with a small enough step-size γ n such that j=1 γ j = and n j=1 γ 2 j / n j=1 γ j 0. If x(n) x, then x is a Nash equilibrium of the game.
34 Local convergence Proposition Suppose that (MD) is run with a small enough step-size γ n such that j=1 γ j = and n j=1 γ 2 j / n j=1 γ j 0. If x(n) x, then x is a Nash equilibrium of the game. Theorem Suppose that x X is variationally stable and (MD) is run with the same conditions as above. Then, x is locally attracting. Corollary Strict equilibria are locally attracting.
35 Local convergence Proposition Suppose that (MD) is run with a small enough step-size γ n such that j=1 γ j = and n j=1 γ 2 j / n j=1 γ j 0. If x(n) x, then x is a Nash equilibrium of the game. Theorem Suppose that x X is variationally stable and (MD) is run with the same conditions as above. Then, x is locally attracting. Corollary Strict equilibria are locally attracting. Proposition Assume that x X is a strict equilibrium with x dom h im Q (i.e. h is not steep at x ). Then, convergence to x occurs after a finite number of iterations.
36 Global convergence Theorem Suppose that x X is globally variationally stable and the algorithm s step-size sequence γ j satisfies j=1 γ j = and n j=1 γ 2 j / n j=1 γ j 0. Then, x(n) x for every initialization of (MD). Corollary If Rosen s condition holds, players converge to equilibrium from any initial condition.
37 Global convergence Theorem Suppose that x X is globally variationally stable and the algorithm s step-size sequence γ j satisfies j=1 γ j = and n j=1 γ 2 j / n j=1 γ j 0. Then, x(n) x for every initialization of (MD). Corollary If Rosen s condition holds, players converge to equilibrium from any initial condition. Proof idea. x(n) is an asymptotic pseudo-trajectory of the continuous-time dynamics ẏ = v(q(y)) A (global) Lyapunov function is given by the (λ-weighted) Fenchel coupling F(y) = k λ k [h k (x k ) + h k (y k ) y k x k ] Standard stochastic approximation results do not suffice for convergence of x(n). Show that x(n) visits a neighborhood of x infinitely often directly. Use Benaïm s theory of attractors on the flow induced by Q on X.
38 Learning with Imperfect Information The above analysis relies on perfect observations of the payoff gradients v k (x). In finite games, it is not too hard to deduce u k (α k ; α k ) for every action α k A k given a fixed action α k A k of one s opponents. However, knowing u k (x k ; x k ) is much more demanding (because of mixing).
39 Learning with Imperfect Information The above analysis relies on perfect observations of the payoff gradients v k (x). In finite games, it is not too hard to deduce u k (α k ; α k ) for every action α k A k given a fixed action α k A k of one s opponents. However, knowing u k (x k ; x k ) is much more demanding (because of mixing). Imperfect feedback: players only have access to noisy estimates of their payoff gradients, i.e. ˆv k (n) = v k (x(n)) + z k (n) Statistical hypotheses for the noise process z k (n): (H1) Unbiasedness: E[z(n + 1) F n ] = 0. (H2) Finite mean squared error: E [ z(n + 1) 2 F n ] <. (H2+) Finite errors: sup n z(n) < (a.s.).
40 Convergence Analysis Run (MD) with steep penalty functions and a small enough step-size γ n such that n=1 γ 2 n < n=1 γ n =.
41 Convergence Analysis Run (MD) with steep penalty functions and a small enough step-size γ n such that n=1 γ 2 n < n=1 γ n =. Theorem Suppose that x is globally variationally stable and (H1), (H2) hold. Then, x(n) converges to x (a.s.) for every initialization of (MD).
42 Convergence Analysis Run (MD) with steep penalty functions and a small enough step-size γ n such that n=1 γ 2 n < n=1 γ n =. Theorem Suppose that x is globally variationally stable and (H1), (H2) hold. Then, x(n) converges to x (a.s.) for every initialization of (MD). Theorem Suppose that x is variationally stable and (H1), (H2) hold. Then, for every ε > 0, there exists a neighborhood U of x such that P(lim n x(n) = x x(0) U) 1 ε, i.e. x attracts all nearby initializations of (MD) with high probability. Under (H2+), the above also holds for ε = 0.
43 Applications to finite games Suppose that players play repeatedly a finite game G G(N, A, u): 1. At stage n + 1, each player selects an action α k (n + 1) A k based on a mixed strategy x k (n) X k. 2. Players estimate (noisily) the payoff of each of their actions: ˆv kα (n + 1) = u k (α; α k (n + 1)) + z kα (n + 1) α A k 3. Players update their mixed strategies using (MD) and the process repeats.
44 Applications to finite games Suppose that players play repeatedly a finite game G G(N, A, u): 1. At stage n + 1, each player selects an action α k (n + 1) A k based on a mixed strategy x k (n) X k. 2. Players estimate (noisily) the payoff of each of their actions: ˆv kα (n + 1) = u k (α; α k (n + 1)) + z kα (n + 1) α A k 3. Players update their mixed strategies using (MD) and the process repeats. Corollary With assumptions as above, strict equilibria are locally attracting with high probability.
45 Convergence in a Potential Game 1.0 n = 1 3, 1 0, , 0 1,
46 Convergence in a Potential Game 1.0 n = 3 3, 1 0, , 0 1,
47 Convergence in a Potential Game 1.0 n = 5 3, 1 0, , 0 1,
48 Convergence in a Potential Game 1.0 n = 8 3, 1 0, , 0 1,
49 Convergence in a Potential Game 1.0 n = 10 3, 1 0, , 0 1,
50 Convergence in a Potential Game 1.0 n = 20 3, 1 0, , 0 1,
51 Convergence in a Potential Game 1.0 n = 50 3, 1 0, , 0 1,
52 Perspectives Rates of convergence? Doable for empirical frequencies of play. Less so for last iterate.
53 Perspectives Rates of convergence? Doable for empirical frequencies of play. Less so for last iterate. Coupled constraints on the players action sets? Possible, but lose distributedness (because of constraint coupling).
54 Perspectives Rates of convergence? Doable for empirical frequencies of play. Less so for last iterate. Coupled constraints on the players action sets? Possible, but lose distributedness (because of constraint coupling). Observations of realized payoffs only (single call to u k instead of v k )? Standard estimators often fail because of infinite variance. Not a problem in online learning maybe in multi-agent case as well? Two-time-scale stochastic approximation can help (at the cost of convergence speed).
55 Perspectives Rates of convergence? Doable for empirical frequencies of play. Less so for last iterate. Coupled constraints on the players action sets? Possible, but lose distributedness (because of constraint coupling). Observations of realized payoffs only (single call to u k instead of v k )? Standard estimators often fail because of infinite variance. Not a problem in online learning maybe in multi-agent case as well? Two-time-scale stochastic approximation can help (at the cost of convergence speed).
Distributed Learning based on Entropy-Driven Game Dynamics
Distributed Learning based on Entropy-Driven Game Dynamics Bruno Gaujal joint work with Pierre Coucheney and Panayotis Mertikopoulos Inria Aug., 2014 Model Shared resource systems (network, processors)
More informationInertial Game Dynamics
... Inertial Game Dynamics R. Laraki P. Mertikopoulos CNRS LAMSADE laboratory CNRS LIG laboratory ADGO'13 Playa Blanca, October 15, 2013 ... Motivation Main Idea: use second order tools to derive efficient
More informationConvergence to Nash equilibrium in continuous games with noisy first-order feedback
Convergence to Nash equilibrium in continuous games with noisy first-order feedback Panayotis Mertikopoulos and Mathias Staudigl Abstract This paper examines the convergence of a broad class of distributed
More informationNear-Potential Games: Geometry and Dynamics
Near-Potential Games: Geometry and Dynamics Ozan Candogan, Asuman Ozdaglar and Pablo A. Parrilo January 29, 2012 Abstract Potential games are a special class of games for which many adaptive user dynamics
More informationMirror Descent Learning in Continuous Games
Mirror Descent Learning in Continuous Games Zhengyuan Zhou, Panayotis Mertikopoulos, Aris L. Moustakas, Nicholas Bambos, and Peter Glynn Abstract Online Mirror Descent (OMD) is an important and widely
More informationNear-Potential Games: Geometry and Dynamics
Near-Potential Games: Geometry and Dynamics Ozan Candogan, Asuman Ozdaglar and Pablo A. Parrilo September 6, 2011 Abstract Potential games are a special class of games for which many adaptive user dynamics
More informationarxiv: v2 [math.oc] 16 Jan 2018
LEARNING IN GAMES WITH CONTINUOUS ACTION SETS AND UNKNOWN PAYOFF FUNCTIONS arxiv:1608.07310v2 [math.oc] 16 Jan 2018 PANAYOTIS MERTIKOPOULOS 1 AND ZHENGYUAN ZHOU 2 Abstract. This paper examines the convergence
More informationPairwise Comparison Dynamics for Games with Continuous Strategy Space
Pairwise Comparison Dynamics for Games with Continuous Strategy Space Man-Wah Cheung https://sites.google.com/site/jennymwcheung University of Wisconsin Madison Department of Economics Nov 5, 2013 Evolutionary
More informationStochastic Mirror Descent in Variationally Coherent Optimization Problems
Stochastic Mirror Descent in Variationally Coherent Optimization Problems Zhengyuan Zhou Stanford University zyzhou@stanford.edu Panayotis Mertikopoulos Univ. Grenoble Alpes, CNRS, Inria, LIG panayotis.mertikopoulos@imag.fr
More informationPopulation Games and Evolutionary Dynamics
Population Games and Evolutionary Dynamics William H. Sandholm The MIT Press Cambridge, Massachusetts London, England in Brief Series Foreword Preface xvii xix 1 Introduction 1 1 Population Games 2 Population
More informationSmooth Calibration, Leaky Forecasts, Finite Recall, and Nash Dynamics
Smooth Calibration, Leaky Forecasts, Finite Recall, and Nash Dynamics Sergiu Hart August 2016 Smooth Calibration, Leaky Forecasts, Finite Recall, and Nash Dynamics Sergiu Hart Center for the Study of Rationality
More informationEvolutionary Game Theory: Overview and Recent Results
Overviews: Evolutionary Game Theory: Overview and Recent Results William H. Sandholm University of Wisconsin nontechnical survey: Evolutionary Game Theory (in Encyclopedia of Complexity and System Science,
More informationSurvival of Dominated Strategies under Evolutionary Dynamics. Josef Hofbauer University of Vienna. William H. Sandholm University of Wisconsin
Survival of Dominated Strategies under Evolutionary Dynamics Josef Hofbauer University of Vienna William H. Sandholm University of Wisconsin a hypnodisk 1 Survival of Dominated Strategies under Evolutionary
More informationStability of Feedback Solutions for Infinite Horizon Noncooperative Differential Games
Stability of Feedback Solutions for Infinite Horizon Noncooperative Differential Games Alberto Bressan ) and Khai T. Nguyen ) *) Department of Mathematics, Penn State University **) Department of Mathematics,
More informationLearning Methods for Online Prediction Problems. Peter Bartlett Statistics and EECS UC Berkeley
Learning Methods for Online Prediction Problems Peter Bartlett Statistics and EECS UC Berkeley Course Synopsis A finite comparison class: A = {1,..., m}. 1. Prediction with expert advice. 2. With perfect
More information6.254 : Game Theory with Engineering Applications Lecture 7: Supermodular Games
6.254 : Game Theory with Engineering Applications Lecture 7: Asu Ozdaglar MIT February 25, 2010 1 Introduction Outline Uniqueness of a Pure Nash Equilibrium for Continuous Games Reading: Rosen J.B., Existence
More informationOnline Convex Optimization
Advanced Course in Machine Learning Spring 2010 Online Convex Optimization Handouts are jointly prepared by Shie Mannor and Shai Shalev-Shwartz A convex repeated game is a two players game that is performed
More informationDETERMINISTIC AND STOCHASTIC SELECTION DYNAMICS
DETERMINISTIC AND STOCHASTIC SELECTION DYNAMICS Jörgen Weibull March 23, 2010 1 The multi-population replicator dynamic Domain of analysis: finite games in normal form, G =(N, S, π), with mixed-strategy
More informationPopulation Games and Evolutionary Dynamics
Population Games and Evolutionary Dynamics (MIT Press, 200x; draft posted on my website) 1. Population games 2. Revision protocols and evolutionary dynamics 3. Potential games and their applications 4.
More informationSpatial Economics and Potential Games
Outline Spatial Economics and Potential Games Daisuke Oyama Graduate School of Economics, Hitotsubashi University Hitotsubashi Game Theory Workshop 2007 Session Potential Games March 4, 2007 Potential
More information6.207/14.15: Networks Lecture 10: Introduction to Game Theory 2
6.207/14.15: Networks Lecture 10: Introduction to Game Theory 2 Daron Acemoglu and Asu Ozdaglar MIT October 14, 2009 1 Introduction Outline Mixed Strategies Existence of Mixed Strategy Nash Equilibrium
More informationA Low Complexity Algorithm with O( T ) Regret and Finite Constraint Violations for Online Convex Optimization with Long Term Constraints
A Low Complexity Algorithm with O( T ) Regret and Finite Constraint Violations for Online Convex Optimization with Long Term Constraints Hao Yu and Michael J. Neely Department of Electrical Engineering
More informationConvex Optimization on Large-Scale Domains Given by Linear Minimization Oracles
Convex Optimization on Large-Scale Domains Given by Linear Minimization Oracles Arkadi Nemirovski H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology Joint research
More informationAdaptive Online Gradient Descent
University of Pennsylvania ScholarlyCommons Statistics Papers Wharton Faculty Research 6-4-2007 Adaptive Online Gradient Descent Peter Bartlett Elad Hazan Alexander Rakhlin University of Pennsylvania Follow
More information1 Directional Derivatives and Differentiability
Wednesday, January 18, 2012 1 Directional Derivatives and Differentiability Let E R N, let f : E R and let x 0 E. Given a direction v R N, let L be the line through x 0 in the direction v, that is, L :=
More information6.207/14.15: Networks Lecture 11: Introduction to Game Theory 3
6.207/14.15: Networks Lecture 11: Introduction to Game Theory 3 Daron Acemoglu and Asu Ozdaglar MIT October 19, 2009 1 Introduction Outline Existence of Nash Equilibrium in Infinite Games Extensive Form
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More information6. Proximal gradient method
L. Vandenberghe EE236C (Spring 2016) 6. Proximal gradient method motivation proximal mapping proximal gradient method with fixed step size proximal gradient method with line search 6-1 Proximal mapping
More information15-850: Advanced Algorithms CMU, Fall 2018 HW #4 (out October 17, 2018) Due: October 28, 2018
15-850: Advanced Algorithms CMU, Fall 2018 HW #4 (out October 17, 2018) Due: October 28, 2018 Usual rules. :) Exercises 1. Lots of Flows. Suppose you wanted to find an approximate solution to the following
More informationExponential Moving Average Based Multiagent Reinforcement Learning Algorithms
Exponential Moving Average Based Multiagent Reinforcement Learning Algorithms Mostafa D. Awheda Department of Systems and Computer Engineering Carleton University Ottawa, Canada KS 5B6 Email: mawheda@sce.carleton.ca
More informationThe FTRL Algorithm with Strongly Convex Regularizers
CSE599s, Spring 202, Online Learning Lecture 8-04/9/202 The FTRL Algorithm with Strongly Convex Regularizers Lecturer: Brandan McMahan Scribe: Tamara Bonaci Introduction In the last lecture, we talked
More informationNoncooperative Games, Couplings Constraints, and Partial Effi ciency
Noncooperative Games, Couplings Constraints, and Partial Effi ciency Sjur Didrik Flåm University of Bergen, Norway Background Customary Nash equilibrium has no coupling constraints. Here: coupling constraints
More information1 Lattices and Tarski s Theorem
MS&E 336 Lecture 8: Supermodular games Ramesh Johari April 30, 2007 In this lecture, we develop the theory of supermodular games; key references are the papers of Topkis [7], Vives [8], and Milgrom and
More informationPopulation Dynamics Approach for Resource Allocation Problems. Ashkan Pashaie
Population Dynamics Approach for Resource Allocation Problems by Ashkan Pashaie A thesis submitted in conformity with the requirements for the degree of Master of Applied Science Graduate Department of
More informationConvex Analysis and Economic Theory AY Elementary properties of convex functions
Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analysis and Economic Theory AY 2018 2019 Topic 6: Convex functions I 6.1 Elementary properties of convex functions We may occasionally
More informationA Generic Bound on Cycles in Two-Player Games
A Generic Bound on Cycles in Two-Player Games David S. Ahn February 006 Abstract We provide a bound on the size of simultaneous best response cycles for generic finite two-player games. The bound shows
More information6.254 : Game Theory with Engineering Applications Lecture 8: Supermodular and Potential Games
6.254 : Game Theory with Engineering Applications Lecture 8: Supermodular and Asu Ozdaglar MIT March 2, 2010 1 Introduction Outline Review of Supermodular Games Reading: Fudenberg and Tirole, Section 12.3.
More informationLogarithmic Regret Algorithms for Strongly Convex Repeated Games
Logarithmic Regret Algorithms for Strongly Convex Repeated Games Shai Shalev-Shwartz 1 and Yoram Singer 1,2 1 School of Computer Sci & Eng, The Hebrew University, Jerusalem 91904, Israel 2 Google Inc 1600
More informationRelative-Continuity for Non-Lipschitz Non-Smooth Convex Optimization using Stochastic (or Deterministic) Mirror Descent
Relative-Continuity for Non-Lipschitz Non-Smooth Convex Optimization using Stochastic (or Deterministic) Mirror Descent Haihao Lu August 3, 08 Abstract The usual approach to developing and analyzing first-order
More informationUnified Convergence Proofs of Continuous-time Fictitious Play
Unified Convergence Proofs of Continuous-time Fictitious Play Jeff S. Shamma and Gurdal Arslan Department of Mechanical and Aerospace Engineering University of California Los Angeles {shamma,garslan}@seas.ucla.edu
More informationOne Mirror Descent Algorithm for Convex Constrained Optimization Problems with Non-Standard Growth Properties
One Mirror Descent Algorithm for Convex Constrained Optimization Problems with Non-Standard Growth Properties Fedor S. Stonyakin 1 and Alexander A. Titov 1 V. I. Vernadsky Crimean Federal University, Simferopol,
More information6. Proximal gradient method
L. Vandenberghe EE236C (Spring 2013-14) 6. Proximal gradient method motivation proximal mapping proximal gradient method with fixed step size proximal gradient method with line search 6-1 Proximal mapping
More informationLocal Stability of Strict Equilibria under Evolutionary Game Dynamics
Local Stability of Strict Equilibria under Evolutionary Game Dynamics William H. Sandholm October 19, 2012 Abstract We consider the stability of strict equilibrium under deterministic evolutionary game
More informationProximal methods. S. Villa. October 7, 2014
Proximal methods S. Villa October 7, 2014 1 Review of the basics Often machine learning problems require the solution of minimization problems. For instance, the ERM algorithm requires to solve a problem
More informationBig Data Analytics: Optimization and Randomization
Big Data Analytics: Optimization and Randomization Tianbao Yang Tutorial@ACML 2015 Hong Kong Department of Computer Science, The University of Iowa, IA, USA Nov. 20, 2015 Yang Tutorial for ACML 15 Nov.
More informationGame Theory and its Applications to Networks - Part I: Strict Competition
Game Theory and its Applications to Networks - Part I: Strict Competition Corinne Touati Master ENS Lyon, Fall 200 What is Game Theory and what is it for? Definition (Roger Myerson, Game Theory, Analysis
More informationEvolution & Learning in Games
1 / 28 Evolution & Learning in Games Econ 243B Jean-Paul Carvalho Lecture 5. Revision Protocols and Evolutionary Dynamics 2 / 28 Population Games played by Boundedly Rational Agents We have introduced
More informationGame Theory and Control
Game Theory and Control Lecture 4: Potential games Saverio Bolognani, Ashish Hota, Maryam Kamgarpour Automatic Control Laboratory ETH Zürich 1 / 40 Course Outline 1 Introduction 22.02 Lecture 1: Introduction
More informationHow hard is this function to optimize?
How hard is this function to optimize? John Duchi Based on joint work with Sabyasachi Chatterjee, John Lafferty, Yuancheng Zhu Stanford University West Coast Optimization Rumble October 2016 Problem minimize
More informationLecture 5: Linear models for classification. Logistic regression. Gradient Descent. Second-order methods.
Lecture 5: Linear models for classification. Logistic regression. Gradient Descent. Second-order methods. Linear models for classification Logistic regression Gradient descent and second-order methods
More informationAccelerating Stochastic Optimization
Accelerating Stochastic Optimization Shai Shalev-Shwartz School of CS and Engineering, The Hebrew University of Jerusalem and Mobileye Master Class at Tel-Aviv, Tel-Aviv University, November 2014 Shalev-Shwartz
More informationBayesian Persuasion Online Appendix
Bayesian Persuasion Online Appendix Emir Kamenica and Matthew Gentzkow University of Chicago June 2010 1 Persuasion mechanisms In this paper we study a particular game where Sender chooses a signal π whose
More informationarxiv: v1 [math.oc] 7 Dec 2018
arxiv:1812.02878v1 [math.oc] 7 Dec 2018 Solving Non-Convex Non-Concave Min-Max Games Under Polyak- Lojasiewicz Condition Maziar Sanjabi, Meisam Razaviyayn, Jason D. Lee University of Southern California
More informationGains in evolutionary dynamics. Dai ZUSAI
Gains in evolutionary dynamics unifying rational framework for dynamic stability Dai ZUSAI Hitotsubashi University (Econ Dept.) Economic Theory Workshop October 27, 2016 1 Introduction Motivation and literature
More informationInformed Principal in Private-Value Environments
Informed Principal in Private-Value Environments Tymofiy Mylovanov Thomas Tröger University of Bonn June 21, 2008 1/28 Motivation 2/28 Motivation In most applications of mechanism design, the proposer
More informationMean-field equilibrium: An approximation approach for large dynamic games
Mean-field equilibrium: An approximation approach for large dynamic games Ramesh Johari Stanford University Sachin Adlakha, Gabriel Y. Weintraub, Andrea Goldsmith Single agent dynamic control Two agents:
More informationProximal and First-Order Methods for Convex Optimization
Proximal and First-Order Methods for Convex Optimization John C Duchi Yoram Singer January, 03 Abstract We describe the proximal method for minimization of convex functions We review classical results,
More informationStable Games and their Dynamics
Stable Games and their Dynamics Josef Hofbauer and William H. Sandholm January 10, 2009 Abstract We study a class of population games called stable games. These games are characterized by self-defeating
More informationBoundary Behavior of Excess Demand Functions without the Strong Monotonicity Assumption
Boundary Behavior of Excess Demand Functions without the Strong Monotonicity Assumption Chiaki Hara April 5, 2004 Abstract We give a theorem on the existence of an equilibrium price vector for an excess
More informationMultimarket Oligopolies with Restricted Market Access
Multimarket Oligopolies with Restricted Market Access Tobias Harks 1 and Max Klimm 2 1 Department of Quantitative Economics, Maastricht University, the Netherlands. t.harks@maastrichtuniversity.nl 2 Department
More informationAn Optimal Affine Invariant Smooth Minimization Algorithm.
An Optimal Affine Invariant Smooth Minimization Algorithm. Alexandre d Aspremont, CNRS & École Polytechnique. Joint work with Martin Jaggi. Support from ERC SIPA. A. d Aspremont IWSL, Moscow, June 2013,
More informationVariational approach to mean field games with density constraints
1 / 18 Variational approach to mean field games with density constraints Alpár Richárd Mészáros LMO, Université Paris-Sud (based on ongoing joint works with F. Santambrogio, P. Cardaliaguet and F. J. Silva)
More informationAn inexact subgradient algorithm for Equilibrium Problems
Volume 30, N. 1, pp. 91 107, 2011 Copyright 2011 SBMAC ISSN 0101-8205 www.scielo.br/cam An inexact subgradient algorithm for Equilibrium Problems PAULO SANTOS 1 and SUSANA SCHEIMBERG 2 1 DM, UFPI, Teresina,
More informationCorrelated Equilibria: Rationality and Dynamics
Correlated Equilibria: Rationality and Dynamics Sergiu Hart June 2010 AUMANN 80 SERGIU HART c 2010 p. 1 CORRELATED EQUILIBRIA: RATIONALITY AND DYNAMICS Sergiu Hart Center for the Study of Rationality Dept
More informationChapter 9. Mixed Extensions. 9.1 Mixed strategies
Chapter 9 Mixed Extensions We now study a special case of infinite strategic games that are obtained in a canonic way from the finite games, by allowing mixed strategies. Below [0, 1] stands for the real
More informationMulti-Agent Learning with Policy Prediction
Multi-Agent Learning with Policy Prediction Chongjie Zhang Computer Science Department University of Massachusetts Amherst, MA 3 USA chongjie@cs.umass.edu Victor Lesser Computer Science Department University
More informationOSNR Optimization in Optical Networks: Extension for Capacity Constraints
5 American Control Conference June 8-5. Portland OR USA ThB3.6 OSNR Optimization in Optical Networks: Extension for Capacity Constraints Yan Pan and Lacra Pavel Abstract This paper builds on the OSNR model
More informationHessian Riemannian Gradient Flows in Convex Programming
Hessian Riemannian Gradient Flows in Convex Programming Felipe Alvarez, Jérôme Bolte, Olivier Brahic INTERNATIONAL CONFERENCE ON MODELING AND OPTIMIZATION MODOPT 2004 Universidad de La Frontera, Temuco,
More informationOptimization and Optimal Control in Banach Spaces
Optimization and Optimal Control in Banach Spaces Bernhard Schmitzer October 19, 2017 1 Convex non-smooth optimization with proximal operators Remark 1.1 (Motivation). Convex optimization: easier to solve,
More informationStochastic Proximal Gradient Algorithm
Stochastic Institut Mines-Télécom / Telecom ParisTech / Laboratoire Traitement et Communication de l Information Joint work with: Y. Atchade, Ann Arbor, USA, G. Fort LTCI/Télécom Paristech and the kind
More informationAccelerated Proximal Gradient Methods for Convex Optimization
Accelerated Proximal Gradient Methods for Convex Optimization Paul Tseng Mathematics, University of Washington Seattle MOPTA, University of Guelph August 18, 2008 ACCELERATED PROXIMAL GRADIENT METHODS
More informationFair and Efficient User-Network Association Algorithm for Multi-Technology Wireless Networks
Fair and Efficient User-Network Association Algorithm for Multi-Technology Wireless Networks Pierre Coucheney, Corinne Touati, Bruno Gaujal INRIA Alcatel-Lucent, LIG Infocom 2009 Pierre Coucheney (INRIA)
More informationOnline Learning and Online Convex Optimization
Online Learning and Online Convex Optimization Nicolò Cesa-Bianchi Università degli Studi di Milano N. Cesa-Bianchi (UNIMI) Online Learning 1 / 49 Summary 1 My beautiful regret 2 A supposedly fun game
More information4. Opponent Forecasting in Repeated Games
4. Opponent Forecasting in Repeated Games Julian and Mohamed / 2 Learning in Games Literature examines limiting behavior of interacting players. One approach is to have players compute forecasts for opponents
More informationRandom Access Game. Medium Access Control Design for Wireless Networks 1. Sandip Chakraborty. Department of Computer Science and Engineering,
Random Access Game Medium Access Control Design for Wireless Networks 1 Sandip Chakraborty Department of Computer Science and Engineering, INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR October 22, 2016 1 Chen
More informationZero sum games Proving the vn theorem. Zero sum games. Roberto Lucchetti. Politecnico di Milano
Politecnico di Milano General form Definition A two player zero sum game in strategic form is the triplet (X, Y, f : X Y R) f (x, y) is what Pl1 gets from Pl2, when they play x, y respectively. Thus g
More informationDivision of the Humanities and Social Sciences. Supergradients. KC Border Fall 2001 v ::15.45
Division of the Humanities and Social Sciences Supergradients KC Border Fall 2001 1 The supergradient of a concave function There is a useful way to characterize the concavity of differentiable functions.
More informationCYCLES IN ADVERSARIAL REGULARIZED LEARNING
CYCLES IN ADVERSARIAL REGULARIZED LEARNING PANAYOTIS MERTIKOPOULOS, CHRISTOS PAPADIMITRIOU, AND GEORGIOS PILIOURAS Abstract. Regularized learning is a fundamental technique in online optimization, machine
More informationIFT Lecture 6 Nesterov s Accelerated Gradient, Stochastic Gradient Descent
IFT 6085 - Lecture 6 Nesterov s Accelerated Gradient, Stochastic Gradient Descent This version of the notes has not yet been thoroughly checked. Please report any bugs to the scribes or instructor. Scribe(s):
More informationA Modular Analysis of Adaptive (Non-)Convex Optimization: Optimism, Composite Objectives, and Variational Bounds
Proceedings of Machine Learning Research 76: 40, 207 Algorithmic Learning Theory 207 A Modular Analysis of Adaptive (Non-)Convex Optimization: Optimism, Composite Objectives, and Variational Bounds Pooria
More informationLeast squares under convex constraint
Stanford University Questions Let Z be an n-dimensional standard Gaussian random vector. Let µ be a point in R n and let Y = Z + µ. We are interested in estimating µ from the data vector Y, under the assumption
More informationIntroduction to Machine Learning (67577) Lecture 7
Introduction to Machine Learning (67577) Lecture 7 Shai Shalev-Shwartz School of CS and Engineering, The Hebrew University of Jerusalem Solving Convex Problems using SGD and RLM Shai Shalev-Shwartz (Hebrew
More informationSubgradient Method. Ryan Tibshirani Convex Optimization
Subgradient Method Ryan Tibshirani Convex Optimization 10-725 Consider the problem Last last time: gradient descent min x f(x) for f convex and differentiable, dom(f) = R n. Gradient descent: choose initial
More informationLecture 7 Monotonicity. September 21, 2008
Lecture 7 Monotonicity September 21, 2008 Outline Introduce several monotonicity properties of vector functions Are satisfied immediately by gradient maps of convex functions In a sense, role of monotonicity
More informationLecture 14: Approachability and regret minimization Ramesh Johari May 23, 2007
MS&E 336 Lecture 4: Approachability and regret minimization Ramesh Johari May 23, 2007 In this lecture we use Blackwell s approachability theorem to formulate both external and internal regret minimizing
More informationThe discrete-time second-best day-to-day dynamic pricing scheme
The discrete-time second-best day-to-day dynamic pricing scheme Linghui Han, David Z.W. Wang & Chengjuan Zhu 25-07-2017 School of Civil & Environmental Engineering Nanyang Technological University, Singapore
More informationLecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem
Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Michael Patriksson 0-0 The Relaxation Theorem 1 Problem: find f := infimum f(x), x subject to x S, (1a) (1b) where f : R n R
More information10-725/36-725: Convex Optimization Prerequisite Topics
10-725/36-725: Convex Optimization Prerequisite Topics February 3, 2015 This is meant to be a brief, informal refresher of some topics that will form building blocks in this course. The content of the
More informationA Multilevel Proximal Algorithm for Large Scale Composite Convex Optimization
A Multilevel Proximal Algorithm for Large Scale Composite Convex Optimization Panos Parpas Department of Computing Imperial College London www.doc.ic.ac.uk/ pp500 p.parpas@imperial.ac.uk jointly with D.V.
More informationGeneralized Mirror Descents with Non-Convex Potential Functions in Atomic Congestion Games
Generalized Mirror Descents with Non-Convex Potential Functions in Atomic Congestion Games Po-An Chen Institute of Information Management National Chiao Tung University, Taiwan poanchen@nctu.edu.tw Abstract.
More informationStochastic Optimization with Variance Reduction for Infinite Datasets with Finite Sum Structure
Stochastic Optimization with Variance Reduction for Infinite Datasets with Finite Sum Structure Alberto Bietti Julien Mairal Inria Grenoble (Thoth) March 21, 2017 Alberto Bietti Stochastic MISO March 21,
More informationAlgorithms for Nonsmooth Optimization
Algorithms for Nonsmooth Optimization Frank E. Curtis, Lehigh University presented at Center for Optimization and Statistical Learning, Northwestern University 2 March 2018 Algorithms for Nonsmooth Optimization
More informationAnisotropic congested transport
Anisotropic congested transport Lorenzo Brasco LATP Aix-Marseille Université lbrasco@gmail.com http://www.latp.univ-mrs.fr/~brasco/ Sankt Peterburg, 04/06/2012 References Part of the results here presented
More informationOptimality Conditions for Nonsmooth Convex Optimization
Optimality Conditions for Nonsmooth Convex Optimization Sangkyun Lee Oct 22, 2014 Let us consider a convex function f : R n R, where R is the extended real field, R := R {, + }, which is proper (f never
More informationA Unified Approach to Proximal Algorithms using Bregman Distance
A Unified Approach to Proximal Algorithms using Bregman Distance Yi Zhou a,, Yingbin Liang a, Lixin Shen b a Department of Electrical Engineering and Computer Science, Syracuse University b Department
More informationApplied Mathematics Letters
Applied Mathematics Letters 25 (2012) 974 979 Contents lists available at SciVerse ScienceDirect Applied Mathematics Letters journal homepage: www.elsevier.com/locate/aml On dual vector equilibrium problems
More informationKey words. saddle-point dynamics, asymptotic convergence, convex-concave functions, proximal calculus, center manifold theory, nonsmooth dynamics
SADDLE-POINT DYNAMICS: CONDITIONS FOR ASYMPTOTIC STABILITY OF SADDLE POINTS ASHISH CHERUKURI, BAHMAN GHARESIFARD, AND JORGE CORTÉS Abstract. This paper considers continuously differentiable functions of
More informationEvolutionary Dynamics and Extensive Form Games by Ross Cressman. Reviewed by William H. Sandholm *
Evolutionary Dynamics and Extensive Form Games by Ross Cressman Reviewed by William H. Sandholm * Noncooperative game theory is one of a handful of fundamental frameworks used for economic modeling. It
More informationStochastic Evolutionary Game Dynamics: Foundations, Deterministic Approximation, and Equilibrium Selection
Stochastic Evolutionary Game Dynamics: Foundations, Deterministic Approximation, and Equilibrium Selection William H. Sandholm October 31, 2010 Abstract We present a general model of stochastic evolution
More informationAn Online Convex Optimization Approach to Blackwell s Approachability
Journal of Machine Learning Research 17 (2016) 1-23 Submitted 7/15; Revised 6/16; Published 8/16 An Online Convex Optimization Approach to Blackwell s Approachability Nahum Shimkin Faculty of Electrical
More information