AM 221: Advanced Optimization Spring Prof. Yaron Singer Lecture 6 February 12th, 2014

Transcription

1 AM 221: Advanced Otimization Sring 2014 Prof. Yaron Singer Lecture 6 February 12th, Overview In our revious lecture we exlored the concet of duality which is the cornerstone of Otimization Theory. The goal of today s lecture is to exlore the conseuences of duality in two fields where otimization is ubiuitous: Game Theory and Learning Theory. 2 A rimer in Game Theory We will focus on two-layer games which constitute a nice introduction to the key concets of Game Theory. A two-layer game is defined by: a set of two layers set of strategies M = {1,..., m} and N = {1,..., n} for layer 1 and layer 2 resectively. ayoff matrices A R m n and B R m n for layer 1 and layer 2 resectively. The signification of the ayoff matrices is the following: when layer 1 lays action i {1,... m} and layer 2 lays action j {1,..., n}, A ij is the gain of layer 1 and B ij is the gain of layer 2. Examle 1. A famous examle of two-layer games is the Battle of the sexes. This game models a situation where a coule is trying to agree on a rogram for the evening. The sets of strategies are the same for the husband and the wife: watch a football game or go to the oera. The coule would refer to do the same activity, but the husband would refer to watch the football game while the wife would refer to go to the oera. Possible ayoffs for this situation could be the following: Oera Football Oera (3, 2) (1, 1) Football (0, 0) (2, 3) Figure 1: Battle of the sexes game. The rows reresent the strategies of the wife and the columns reresent the strategies of the husband. In this comact reresentation, we combine both ayoff matrices into one table. The uer left cell of this table should be read like this: when the wife and the husband both agree to go to the oera, the gain of the wife is 3 and the gain of the husband is 2. 1

2 When analyzing a game, one would like to know which combination of strategies are likely to be chosen by the layers. Assuming that the layers are rational, i.e utility imizers, the notion of Nash euilibria follows naturally. Definition 2. A strategy rofile (i, j) M N is called a Nash euilibrium if the strategy layed by each layer is otimal given the strategy chosen by the other layer: A ij A kj for all k M. B ij B ik for all k N. Examle 3. In the battle of sexes game introduced above, there are two nash euilibria: (Oera, Oera) and (Football, Football). (Oera, Football), for examle, is not a Nash euilibrium since the wife could increase her gain from 1 to 2 by choosing Football instead of Oera. Examle 4. Another famous examle of two-layer games is the risoners dilemma. In this roblem, two risoners are in solitary confinement. The olice is convinced that both risoners are guilty but does not have enough evidence to convict them to a long rison sentence. Thus, the olice interrogates the risoners searately. Each risoner can either stay silent (not reveal anything to the olice) or betray the other. If only one of the risoners betrays the other, then he will be set free and the other will be convicted to a long rison sentence. However, if they betray each other, both risoners will serve a long rison sentence. Possible ayoffs modeling this situation could be the following: Silent Betray Silent ( 1, 1) ( 10, 0) Betray (0, 10) ( 8, 8) A close insection of this table shows that there is only one Nash euilibrium: (Betray, Betray). Indeed, in all other cases, a silent risoner can always increase his gain (or reduce his loss) by choosing to betray instead of staying silent. In this case, the Nash euilibrium is not otimal for the risoners: they could be both better off by staying silent. This subotimality is a conseuence of the nature of the game: both risoners choose their action without knowing what the other risoner does. 3 Mixed strategies, zero-sum games and the mini theorem When studying games, a more general notion of strategies can also be defined: mixed strategy. A mixed strategy is an assignment of a robability to each (ure) strategy so that. A mixed strategy allows the layer to choose a ure strategy at random. Formally, a mixed strategy for layer 1 over the set of ure strategies M = {1,..., m} is a robability vector x M = {u [0, 1] m : m i=1 u i = 1}. A mixed strategy for layer 2 is defined in a similar way. Note that a ure strategy can be seen as a degenerate case of mixed strategy where a single robability is set to 1 and all the others are set to 0. 2

3 The notion of gain and Nash euilibrium can be extended to this generalized notion of strategy. If layer 1 and layer 2 have mixed strategies x M and y N resectively, their gains are defined as the exected gain when they both lay selecting ure strategies at random according to x and y. Formally, the gain of layer 1 will be: x i y j A ij = x T Ay (i,j) M N and similarly for layer 2 by relacing A with B. The notion of Nash euilibria is also extended using this exected gain. Definition 5. A rofile of mixed strategies x M and y N is a mixed Nash euilibrium iff: x T Ay u T Ay for all u M. x T By x T Bv for all v N. Examle 6. The enalty kick game models the situation in soccer where a layer is about to kick a enalty: usually the goalie stands at the middle of the goal line and the layer choose to aim either to the left or to the right of the goal. Because the layer stands very close to the goal, the goalie has to decide on a direction in advance and starts diving in this direction even before the layer hits the ball. If the goalie dives in the direction chosen by the layer, he saves the shot, otherwise he fails. Possible ayoffs modeling this situation could be: Left Right Left (+1, 1) ( 1, +1) Right ( 1, +1) (+1, 1) where the rows reresent the dive direction chosen by the goalie and the columns reresent the kick direction chosen by the layer. Note that this game has no ure Nash euilibria: in any situation the layer with gain 1 can increase his gain by changing his strategy. Intuitively, this simly exresses that there is always a better strategy in retrosect for the losing layer. However, this game has one mixed Nash euilibrium: (1/2, 1/2), (1/2, 1/2). When both layers choose a direction uniformly at random, this leads to an exected gain of 0 for both of them. Any other choice of robabilities would lead to an unbalancedness either to the left or to the right which could be exloited by the other layer. The fact that the enalty kick game has a mixed Nash euilibrium is not surrising. It is in fact a conseuence of a general theorem. Remark that in the enalty kick game, the gain of the kicker is exactly the oosite of the gain of the goalie. More generally we have: Definition 7. A two-layer game is a zero-sum game iff B = A with A and B the ayoffs matrices of the layers. Zero-sum game are sometimes called strictly cometitive in that the gain of a layer is exactly the loss of the other(s) layer(s): cooeration is useless in these games. We are now ready to state the main theorem: 3

4 Theorem 8. For every two-layer zero-sum game, there exists a mixed Nash euilibrium. This theorem is a simle conseuence of the following theorem which is itself a simle conseuence of the strong duality theorem. Theorem 9 (von Neumann s mini theorem). We have: x M Proof. First remark that for a given x M : min x T Ay = min y N y N x T Ay x M min x T Ay = min y N j N (xt A) j hence, the left-hand side of the mini theorem can be written as the following LP: t s.t. t (x T A) j, j N x 0 x i = 1 i M Similarly, the right-hand side can be written as the following LP: min s s.t. s (Ay) i, i M y 0 y j = 1 j N and both LPs are dual from each other. The theorem then simly follows from the strong duality theorem. Let us now see how to rove the existence of mixed Nash euilibria in zero-sum two-layer games. Proof. Let us consider a zero-sum two-layer game with A the ay-off matrix for layer A. Let us write v = x M min y N x T Ay = min y N x M x T Ay and consider x arg x M min y N x T Ay and y arg min y N x M x T Ay. Then we claim that (x, y ) is a mixed Nash euilibrium. Indeed, it follows from the definition of x and y that: x T Ay v x T Ay for all x M. x T Ay v x T Ay, i.e. x T By x T By for all y N. which concludes the roof. 4

5 4 Learning and Boosting We now study an alication of duality to learning. More recisely we consider a classification roblem over a sace X for which we are given a set of hyothesis (ossible classifiers) H = {h : X {0, 1}}. The data is samled from an unknown distribution over X. The weak learning assumtion states that the set of hyothesis H is good in the following sense: for any distribution there exists a hyothesis h H which is wrong less than half the time (i.e is better than a uniformly random classifier). Formally: γ,, h H, P x [h(x) c(x)] 1 2 γ 2 where c(x) is by definition the correct class (the true answer) of x X. Surrisingly the weak learning assumtion imlies something much stronger: it is ossible to combine the classifiers in H to construct a classifier which is always right. This is known as strong learning. This is a conseuence of the mini theorem. Theorem 10. Let H be a set of hyothesis satisfying the weak learning assumtion, then there exists a distribution on H such that the weighed majority classifier: { 1 if h H c (x) := hh(x) otherwise is always correct, i.e, c (x) = c(x) for all x X. Proof. All we have to do is to find the robability, that is, weights to assign to each classifier h H. Let us define the matrix M { 1, +1} X H such that M ij is +1 when classifier h j is wrong on data x i and 1 otherwise. Note that weak learning can be written as: X i δ hj (x i ) c(x i ) 1 2 γ 2 i=1 (1) where δ hj (x i ) c(x i ) is +1 when h j (x i ) c(x i ) and 0 otherwise. We have M ij = 2δ hj (x i ) c(x i ) 1, hence (1) can be written more comactly as: T Me j γ, where e j is the j-th basis vector of R H. We know that we can find such a j for all, i.e: min j T Me j = min T M γ where is a distribution on X and is a distribution on H and where the euality follows from the fact that the basis vectors are the basic feasible solutions of the standard simlex. By the mini theorem: min T M = min e T i M γ i i.e, there exists a distribution over H such that for all i, (M) i γ. This imlies that the weighted classifier defined in the statement of the theorem is always correct. 5