Mini Problems Daniel P. Palomar Hong Kong University of Science and Technolgy (HKUST) ELEC547 - Convex Optimization Fall 2009-10, HKUST, Hong Kong
Outline of Lecture Introduction Matrix games Bilinear problems Robust LP Convex-concave games Lagrange game Summary Daniel P. Palomar 1
Introduction Games are optimization problems with more than one decision maker (or player, using the terminology of game theory), often with conflicting goals. The general theory of games is complicated, but some classes of games are closely related to convex optimization and have a nice theory. Game theory goes back to the mid-1990s (1947 book by von Neumann and Morgenstein and Nash theorem in 1950). Daniel P. Palomar 2
Foreword by Nash in 2008 JSAC Special Issue Daniel P. Palomar 3
Matrix Game A matrix game is a two-player zero-sum discrete game that can be described by a payoff matrix. Player 1 chooses his strategy (index k) to minimize the payoff P kl. Player 2 chooses his strategy (index l) to imize the payoff P kl. Pure strategies: each user is allowed to choose only one strategy. Mixed strategies: each user is allowed to randomize over a set of strategies. Daniel P. Palomar 4
Pure-Strategy Matrix Game Player 1 can expect that, for a given choice k, player 2 will choose the largest entry in row k. This means that he has to choose the row with the minimum largest entry: min k l P kl. Similarly, player 2 expects that, for a given choice l, player 1 will choose the smallest entry in column l. He will then choose the column with the imum smalles entry: l min k P kl. But, which one is the correct formulation? Are they the same? Daniel P. Palomar 5
Pure-Strategy Matrix Game (II) In general, the two formulations lead to different results. However, we can show that one is smaller than the other. Lemma: l min k P kl min k l P kl. Proof. min ey ex f (x, ỹ) ex min ey f ( x, ỹ) min ey f ( x,y) ex x,y f ( x, ỹ). Daniel P. Palomar 6
Pure-Strategy Matrix Game (II) But, in general, for pure-strategy matrix games l min k P kl min k l P kl. This is related to the concept of Nash equilibrium (NE). If the game admits a NE, then l min k P kl = min k l P kl. Nash famous theorem shows that for games with N players, under some conditions, a NE exists. Daniel P. Palomar 7
Mixed-Strategy Matrix Game Now consider that each player chooses a strategy with some probability (independent of the other user). Suppose that u and v are the mixed strategies for players 1 and 2. The expected payoff is i,j u i v j P ij = u T Pv. Reasoning as before, player 1 will choose u as the solution to min u v u T Pv s.t. u 0, 1 T u = 1 v 0, 1 T v = 1. Daniel P. Palomar 8
Mixed-Strategy Matrix Game (II) Since v u T Pv = i [ P T u ], we can rewrite the problem as i [ min P T u ] u i i s.t. u 0, 1 T u = 1 or, in epigraph form, (P 1 ) minimize t u,t subject to u 0, 1 T u = 1 t1 P T u. Daniel P. Palomar 9
Mixed-Strategy Matrix Game (III) Similarly, for player 2 we have (P 2 ) imize u,µ µ subject to v 0, 1 T v = 1 Pv µ1. Since -min min-, p 2 p 1. We can interpret the difference p 1 p 2 as the advantage conferred on a player by knowing the opponent s strategy. Daniel P. Palomar 10
Mixed-Strategy Matrix Game (IV) In this case, however, we have a stronger characterization: Theorem: For feasible mixed-strategy matrix games: p 1 = p 2. Proof. Let s find the dual of problem P 1. The Lagrangian is L (t,u; λ, σ, µ) = t + λ T ( P T u t1 ) σ T u µ ( 1 T u 1 ) ( ) = t 1 λ T 1 + (Pλ σ µ1) T u + µ, Daniel P. Palomar 11
Mixed-Strategy Matrix Game (V) Proof. (cont d) the dual function is g (λ, σ, µ) = { µ if λ T 1 = 1 and Pλ = σ + µ1 µ1 otherwise, and the dual problem is imize λ,µ µ subject to λ 0, 1 T λ = 1 Pλ µ1 which happens to be problem P 2. The result follows because problems P 1 and P 2 are dual of each other and strong duality holds (since the LPs are feasible). Daniel P. Palomar 12
Bilinear Problems Consider the following bilinear problem (more general than the mixed-strategy matrix game): min x s.t. x T Py y Ax b Cy d. Again, -min min-, but do we have equality? In this case, we cannot proceed as before since [ y P T x ]. i i x T Py Daniel P. Palomar 13
Bilinear Problems (I) Sometimes, depending on the problem structure, the inner imization can be solved in closed form (like in the application of worst-case robust beamforming design). In general, however, the inner imization does not have a closed form as it is an LP and we know that LPs do not have closed-form solutions (except in particular cases). We can resort to duality to deal with the inner imization y x T Py s.t. Cy d. Daniel P. Palomar 14
Bilinear Problems (II) Lemma: The dual problem of the inner imization is min λ s.t. d T λ P T x = c T λ λ 0. Proof. The Lagrangian is L (y;λ) = ( P T x ) T y + λ T (d Cy) = ( P T x C T λ ) T y + d T λ, the dual function is g (λ) = { d T λ if P T x = C T λ otherwise, and the dual problem follows. Daniel P. Palomar 15
Bilinear Problems (III) We are now ready to rewrite the mini problem in the more convenient form of a minimization problem: Lemma: The original mini problem (assuming feasibility of the inner imization) can be rewritten as the following LP: (Mini-LP) minimize x,λ subject to d T λ P T x = C T λ λ 0 Ax b. Proof. Since the original problem has a nonempty feasible set, strong duality holds and the primal value is equal to the dual value. Daniel P. Palomar 16
Bilinear Problems (IV) Let s deal now with the imin formulation (as opposed to the mini): y s.t. min x T Py x Ax b Cy d. Lemma: The dual problem of the inner minimization is ν b T ν s.t. Py + A T ν = 0 ν 0. Daniel P. Palomar 17
Bilinear Problems (V) Lemma: The original imin problem (assuming feasibility of the inner minimization) can be rewritten as the following LP: (Maximin-LP) imize b T ν y,ν subject to Py + A T ν = 0 ν 0 Cy d. So far, we have been able to rewrite the imin and mini problems as LPs. Daniel P. Palomar 18
Bilinear Problems (VI) At this point, however, we don t know yet if imin equals mini, which is answered in the next result: Theorem: For feasible bilinear problems: -min = min-. Proof. It follows by showing that (Mini-LP) and (Maximin-LP) are dual of each other and strong duality for feasible LPs. Let s find the dual of (Mini-LP). The Lagrangian is L (x,λ; ν,σ,µ) = d T λ + ν T (Ax b) σ T λ + µ ( T P T x C T λ ) = (d σ Cµ) T λ + ( A T ν + Pµ ) T x b T ν, Daniel P. Palomar 19
Bilinear Problems (VII) Proof. (cont d) the dual function is g (ν, σ, µ) = { b T ν if d Cµ = σ 0 and A T ν + Pµ = 0 otherwise, and the dual problem is which is identical to (Maximin-LP). imize b T ν ν,µ subject to Pµ + A T ν = 0 ν 0 d Cµ Daniel P. Palomar 20
Robust LP An LP is of the form: minimize c T x x subject to a T i x + b i 0 i = 1,,m. The parameters of the problem c, (a i,b i ) for i = 1,,m are usually assumed to be perfectly known. In real applications, however, there may be uncertainty in these parameters. This leads to robust optimization. We will next consider the case where knowledge of the parameter a i is imperfect: a i A i where the uncertainty set A i is known. Daniel P. Palomar 21
Robust LP (I) The uncertainty set can be modeled in different ways, e.g., as an ellipsoid (as in the application of robust beamforming) or as a polyhedron. We will model A i as an affine transformation of a polyhedron: A i = {a i = ā i + B i u i, D i u i d i }. The robust LP (to imperfect knowledge of a i ) is then minimize x subject to c T x sup (ā i + B i u i ) T x + b i 0 i = 1,,m u i U i where U i = {u i D i u i d i }. Daniel P. Palomar 22
Robust LP (II) We can now rewrite the robust formulation as minimize c T x x subject to f i (x) 0 i = 1,,m where f i (x) is the optimal value of ( ) imize x T B i ui + ( ā T u i x + b ) i i subject to D i u i d i whose dual problem is minimize z i d T i z i + ( ā T i x + b ) i subject to D T i z i = B T i x z i 0. Daniel P. Palomar 23
Robust LP (III) Now, from strong duality, the dual value equals the imum value f i (x). Therefore, we have the following result: Theorem: The robust LP can be rewritten as the following LP: minimize c T x x,{z i } subject to d T i z i + ( ā T i x + b i) 0 i = 1,,m D T i z i = B T i x z i 0. Daniel P. Palomar 24
Convex-Concave Games Consider now a more general game where the payoff function is f (x,y) with player 1 minimizing over x and player 2 imizing over y. We already know that y min f (x,y) min x x y f (x,y). We say that (x,y ) is a solution of the game (Nash equilibrium or saddle point) if f (x,y) f (x,y ) f (x,y ) x,y. In words: at a saddle point, neither player can do better by unilaterally changing his strategy. Daniel P. Palomar 25
Convex-Concave Games (I) Convex-concave problems have been extensively characterized and we now state two of the nicest results: Lemma: If f is convex-concave (and some other conditions), then a saddle-point exists. Lemma: If a saddle-point exists, then -min = min-. Proof. From the existence of a saddle-point, we have f (x,y ) = min x f (x,y ) y min x f (x,y) as well as f (x,y ) = y f (x,y) min x y f (x,y), Daniel P. Palomar 26
Convex-Concave Games (II) Proof. (cont d) which implies y min x f (x,y) min x y f (x,y). This combined with the well-known inequality y min x f (x,y) min x y f (x,y) shows that y min x f (x,y) = min x y f (x,y). Daniel P. Palomar 27
Lagrange Game We will now study a very particular game related to Lagrange duality. Consider the Lagrangian of a general optimization problem: L(x;λ) = f 0 (x) + m λ i f i (x). i=1 Observe that sup λ 0 L(x;λ) = { f0 (x) if f i (x) 0 i = 1,,m + otherwise Daniel P. Palomar 28
Lagrange Game (I) Therefore, we can express the optimal value of the primal problem as p = inf sup L(x;λ). x λ 0 On the other hand, by definition of dual function (as infimum of the Lagrangian), we have d = sup λ 0 g (λ) = sup λ 0 inf x L(x;λ). Thus, from the fact that -min min-, we have i.e., weak duality holds! d p, Daniel P. Palomar 29
Lagrange Game (II) Recall that -min = min- for a convex-concave function. Now, if the primal problem is convex, then the Lagrangian L(x;λ) is convex in x and concave (linear in fact) in λ. Thus (caution: additional assumptions are needed as the dual feasible set is not compact), i.e., strong duality holds. d = p, Daniel P. Palomar 30
Summary We have explored mini problems (two-player zero-sum games). In practice, mini formulations appear in robust optimization. In some cases, a saddle-point or Nash equilibrium exists and then -min = min-. Different ways to deal with mini problems: 1) use specific numerical methods for mini problems, 2) solve in closed-form the inner im., 3) rewrite the inner im. as the dual minimiz., 4) use a technique like the S-lemma to get rid of the inner im. Game theory considers the more general and complicated case of N players. Daniel P. Palomar 31
References on Mini R. T. Rockafellar, Convex Analysis, 2nd ed. Princeton, NJ: Princeton Univ. Press, 1970. D. P. Bertsekas, A. Nedic, and A. E. Ozdaglar, Convex Analysis and Optimization, Athena Scientific, Belmont, MA, 2003. Daniel P. Palomar, John M. Cioffi, and Miguel A. Lagunas, Uniform Power Allocation in MIMO Channels: A Game-Theoretic Approach, IEEE Trans. on Information Theory, vol. 49, no. 7, July 2003. Antonio Pascual-Iserte, Daniel P. Palomar, Ana Pérez-Neira, Miguel A. Lagunas, A Robust Maximim Approach for MIMO Comm. with Imperfect CSI Based on Convex Optimization, IEEE Trans. on Signal Processing, vol. 54, no. 1, Jan. 2006. Jiaheng Wang and Daniel P. Palomar, Worst-Case Robust Transmission in MIMO Channels with Imperfect CSIT, IEEE Trans. on Signal Processing, vol. 57, no. 8, pp. 3086-3100, Aug. 2009. Daniel P. Palomar 32
References on Game Theory J. Osborne and A. Rubinstein, A Course in Game Theory, MIT Press, 1994. Gesualdo Scutari, Daniel P. Palomar, and Sergio Barbarossa, Competitive Design of Multiuser MIMO Systems based on Game Theory: A Unified View, IEEE JSAC: Special Issue on Game Theory, vol. 25, no. 7, Sept. 2008. Gesualdo Scutari, Daniel P. Palomar, and Sergio Barbarossa, Optimal Linear Precoding Strategies for Wideband Noncooperative Systems Based on Game Theory, IEEE Trans. on Signal Processing, vol. 56, no. 3, March 2008. Gesualdo Scutari, Daniel P. Palomar, and Sergio Barbarossa, Asynchronous Iterative Water- Filling for Gaussian Frequency-Selective Interference Channels, IEEE Trans. on Information Theory, vol. 54, no. 7, July 2008. Gesualdo Scutari, Daniel P. Palomar, and Sergio Barbarossa, Cognitive MIMO Radio: A Competitive Optimality Design Based on Subspace Projections, IEEE Signal Processing Magazine, Nov. 2008. Daniel P. Palomar 33