A Generalization of Two-Player Stackelberg Games to Three Players

Transcription

1 A Generliztion of Two-Plyer Stckelerg Gmes to Three Plyers Grrett Andersen 1 Introduction Two-plyer Stckelerg gmes nd their pplictions to security re currently very hot topic in the field of Algorithmic Gme Theory [7]. In two-plyer Stckelerg gmes, insted of hving the plyers move simultneously, one plyer is designted s the leder nd one s the follower. The gme then works s follows: the leder chooses strtegy (possily mixed) to commit to which the follower oserves efore choosing his response. A lrge prt of the ppel of two-plyer Stckelerg gmes is the fct tht they cn e solved using liner progrmming in polynomil time in the size of the gme [3]. One ovious question then, is whether or not this trctility cn e extended to Stckelerg gmes with more plyers. This is ddressed somewht in [2], however the model presented relies on the followers oeying signl from the leder, which is very strong ssumption. In this pper, I propose n lterntive model with weker ssumptions nd then nlyze its complexity. 2 Stckelerg Gmes With Three Plyers In Stckelerg gmes with more thn two plyers, one plyer is designted s the leder nd the rest re designted s followers. After the leder chooses strtegy to commit to, the followers oserve this strtegy nd then respond simultneously. Becuse the followers respond simultneously, it would e nturl to require the followers responses to constitute Nsh Equilirium. However, finding Nsh Equilirium is lredy known to e NP-hrd in the vst mjority of scenrios, so in the interest of computility, this requirement is relxed slightly; the followers re only required to ply correlted equilirium etween themselves. This is done ecuse finding correlted 1

2 equilirium of simultneous move gme cn e done in polynomil time in the size of the gme using liner progrmming [6]. In generl though, there is still the question of which correlted equilirium the followers re going to ply. From the perspective of the leder, nturl thing to do would e to optimize his commitment strtegy for the worst cse scenrio, where fter oserving his commitment strtegy, the followers will lwys respond y plying the correlted equilirium tht he prefers the lest. This is the model tht will e used for the rest of the pper, lthough only the cse with one leder nd two followers is considered. Initilly the hope ws solvele using liner progrmming, however in the next section I will provide NP-hrdness reduction from multi-plyer minmx gmes (which implies tht the prolem cn t e descried s liner progrm). 3 Hrdness Reduction In multi-plyer minmx gmes, two plyers simultneously ttempt to minimize third plyer s utility (without regrd for their own utilities). If correltion etween the two punishing plyers is not llowed, then finding the optiml multi-plyer minmx strtegy is known to e NP-hrd [1] [5]. So, given three-plyer norml-form gme with plyers A, B, nd C, suppose tht A nd B re simultneously trying to punish C s much s possile, without eing le to correlte their strtegies. Note tht without loss of generlity, we cn entry-wise replce the utilities of A nd B with the negtive utilities of C ecuse chnging their utilities wont hve ny effect on the punishment of C. Let s P nd s P e optiml multi-plyer minmx strtegy profiles for A nd B, nd let s P c e est response of C to these strtegies. Then, let u P, u P, nd u P c e the resultnt utilities for ech plyer in this outcome. Note tht u P = u P = u P c ecuse of the wy the utilities of plyers A nd B were chnged. Now suppose tht this sme (modified) gme is treted s three-plyer Stckelerg gme with A s the leder nd B nd C s the followers. As in the previous section, fter A chooses strtegy to commit to, B nd C oserve this strtegy nd respond y plying the correlted equilirium tht A would prefer the lest. Note tht fter A commits, B nd C re left plying two-plyer zero-sum gme with some minimx vlue v (with B s the minimizer). And, for ny correlted equilirium in two plyer zerosum gme with minimx vlue v, the minimizer s utility must e v nd the mximizer s utility must e v [4]. So fter A commits, B is gurnteed to 2

3 get utility v in ny correlted equilirium of the resultnt gme, which lso mens tht plyer A will get utility v ecuse they hve the sme utilities. Therefore, once plyer A chooses his commitment strtegy, he will lwys e indifferent over ll correlted equilirium responses of B nd C. The following lgorithm sed on this Stckelerg gme cn e used to clculte optiml multi-plyer minmx strtegies for A nd B. First, for every outcome, replce A s nd B s utilities y the negtive utility of C. Then, in the corresponding three-plyer Stckelerg gme with A s the leder, compute A s optiml strtegy to commit to. Cll this strtegy nd let u S, u S, nd u S c e the plyers utilities when A commits to this strtegy nd B nd C respond y plying correlted equilirium s S (rememer ech plyer s utility is constnt over ll correlted equiliri tht B nd C could ply). Then, fix s S nd run the minimx lgorithm on the resulting two-plyer zero-sum gme to find minimx strtegy s S for plyer B nd mximin strtegy s S c for plyer C. If B nd C ply these strtegies, B will get utility v nd C will get utility v, where v is the minimx vlue of the gme, so their utilities must e the sme s in ny correlted equilirium of the gme. Therefore, if the plyers ply s S, s S, nd s S c s their strtegies, the resultnt utilities for ech plyer must e u S, u S, nd u S c (i.e. the sme s if B nd C responded with correlted equilirium). The rest of this section will e dedicted to showing tht s S nd s S re optiml multi-plyer minmx strtegies for A nd B. The sic proof ide is pretty simple. It just hs to e shown tht C s utility when plying est response ginst n optiml multi-plyer minmx strtegy is oth less thn or equl nd greter thn or equl to his utility when A commits to the optiml commitment strtegy nd B nd C respond y plying correlted equilirium. The following lemms will e useful: Lemm 1: In the simultneous move gme, s S c C when plyers A nd B ply s S nd s S. is est response for plyer Proof: Suppose tht there exists some strtegy s c for C tht is etter thn s S c when A nd B ply s S nd s S. Then in the corresponding Stckelerg gme, if A commits to s S nd B plys s S in the resultnt two-plyer zerosum gme, then C could increse his utility y switching from s S c to s c. This would either contrdict the fct tht s S c is mximin strtegy for C or the fct tht s S is minimx strtegy for B (for the two-plyer zero-sum gme tht occurs fter A commits). 3

4 Therefore, in the simultneous move gme, if A nd B ply s S nd s S nd C plys est response, the resulting utilities for the plyers will e u S, u S, nd u S c. Lemm 2: Suppose A s strtegy is fixed to e s P in the simultneous move gme. Then, the result of this fixing is tht plyers B nd C re left plying two-plyer zero-sum gme with some minimx vlue v (with B s the minimizer). If B nd C ply s P nd s P c in this new gme, it must e tht the resulting utility for C is v nd the resulting utility for B is v. Proof: Suppose tht C s utility is greter thn v in this outcome. Then plyer B cn switch from plying s P to plying minimx strtegy to gurntee tht C cnnot get more thn v. This contrdicts the fct tht s P nd s P re optiml multi-plyer minmx strtegies. Now suppose tht C s utility is less thn v in this outcome. Then C cn switch from plying s P c to plying mximin strtegy to gurntee himself utility of t lest v. This would contrdict the fct tht s P c is est response to s P nd. s P Lemm 3: u S u P Proof: Suppose A commits to s P in the Stckelerg gme. If plyers B nd C respond y plying s P nd s P c, then y Lemm 2, the resulting utility for A must e the sme s if B nd C hd responded with correlted equilirium. Therefore if A commits to s P nd B nd C respond with correlted equilirium, plyer A s utility will e u P. Tht mens tht u S u P ecuse A cn gurntee himself utility of t lest u P y committing to s P. This lso implies tht u S c u P c ecuse C s utility will e equl to the negtive utility of plyer A in every outcome. Theorem: s S nd s S re optiml multi-plyer minmx strtegies. Proof: By Lemm 1, if A nd B ply s S in the simultneous move gme nd C plys est response to these, plyer C s utility will e u S Also y Lemm 3, u S c u P c. nd s S I hve shown how to compute optiml multi-plyer minmx strtegies for ech punishing plyer given n lgorithm to find the optiml strtegy for the leder to commit to in the three-plyer Stckelerg model I introduced. c. 4

5 Therefore, ecuse computing optiml multi-plyer minmx strtegies is n NP-hrd prolem, finding the optiml strtegy to commit to must lso e NP-hrd. 4 Other Possile Implementtions Although the fct tht this prolem is NP-hrd implies tht it cnnot e written s liner progrm, it still my e possile to descrie it s Mixed- Integer Progrm nd solve it using the vriety of techniques ville for those prolems. Unfortuntely, I ws unle to think of formultion even in this relxed setting nd the prolem seems inherently highly non-liner. One thing I did try ws to solve this prolem using solvers which ccept non-liner ojectives nd/or non-liner constrints. I tried two differnet implementtions of the prolem, one with non-liner ojective nd liner constrints, nd one with liner ojective nd non-liner constrints. The implementtion with the non-liner ojective is: mximize u L (p 1, p 2..., p n ) such tht Σ r p r = 1 (r) p r 0 Where u L (p 1, p 2..., p n ) is equl to the leder s utility when he plys strtegy profile p 1, p 2..., p n nd the followers respond y plying the correlted equilirium etween themselves tht the leder prefers the lest. This cn e considered the nive implementtion. Another possile implementtion, with liner ojective, ut non-liner constrints is: mximize v such tht (c, h) v + Σ c y c,c (Σ r p r (u f1 (r, c, h) u f1 (r, c, h))) + Σ h z h,h (Σ r p r (u f2 (r, c, h) u f2 (r, c, h ))) Σ r p r u L (r, c, h) Σ r p r = 1 (r) p r 0 (c, c ) y c,c 0 (h, h ) z h,h 0 5

6 Where the r s, c s, nd h s represent the pure strtegies of the leder, follower 1, nd follower 2, respectively. This formultion is sed on the dul of the liner progrm which solves for the three-plyer correlted equilirium tht minimizes the leder s utility, without individul rtionlity constrints for the leder, seen elow. mximize v such tht (r, c, h) v + Σ c y c,c (u f1 (r, c, h) u f1 (r, c, h)) + Σ h z h,h (u f2 (r, c, h) u f2 (r, c, h )) u L (r, c, h) (c, c ) y c,c 0 (h, h ) z h,h 0 This progrm would find mximum lower ound on the leder s utility in ny three-plyer correlted equilirium without rtionlity constrints for the leder. If this progrm is modified y replcing ech u(r, c, h) y Σ r p r u(r, c, h), it would give mximum lower ound in the cse where the leder commits to plying the distriution given y the p r s nd then the leder s lest preferred correlted equilirium is plyed (exctly the outcome tht is eing solved for). See [2] for more informtion. I tested these two implementtions on three NLP solvers provided y the Mtl(R) Optimiztion Toolox, f mincon, GeneticAlgorithm, nd P tternserch. There re significntly more dvnced commercil solvers ville for solving NLPs, however I ws unle to otin license for them. Nevertheless, the results for these three solvers were not very promising. Repeted execution of solver on the sme prmeters gve significntly different nswers ech time, indicting tht the solvers were getting stuck on locl optim. This ws true for oth implementtions nd ll three solvers. One interesting thing though, ws tht the first implementtion didn t vry nerly s much nd gve consistently etter solutions thn the second. I ttriute this to the fct tht hving more vriles is highly undesirle for non-liner progrms ecuse of how the solvers hve to serch through the vrile spce. It is possile tht the commercil solvers re much more vile, ut I m not sure I will get the chnce to try them out. References [1] Christin Borgs, Jennifer Chyes, Nicole Immorlic, Adm Tumn Kli, Vh Mirrokni, nd Christos Ppdimitriou. The myth of the 6

7 folk theorem. Gmes nd Economic Behvior, 70(1):34 43, [2] Vincent Conitzer nd Dmytro Korzhyk. Commitment to correlted strtegies. In AAAI, [3] Vincent Conitzer nd Tuoms Sndholm. Computing the optiml strtegy to commit to. In Proceedings of the 7th ACM conference on Electronic commerce, pges ACM, [4] Frncoise Forges. Correlted equilirium in two-person zero-sum gmes. Econometric, 58(2):515, Mrch [5] Kristoffer Arnsfelt Hnsen, Thoms Dueholm Hnsen, Peter Bro Miltersen, nd Troels Bjerre Sørensen. Approximility nd prmeterized complexity of minmx vlues. In Internet nd Network Economics, pges Springer, [6] Christos H Ppdimitriou nd Tim Roughgrden. Computing correlted equiliri in multi-plyer gmes. Journl of the ACM (JACM), 55(3):14, [7] Jmes Pit, Mnish Jin, Jnusz Mrecki, Fernndo Ordóñez, Christopher Portwy, Milind Tme, Crig Western, Prveen Pruchuri, nd Srit Krus. Deployed rmor protection: the ppliction of gme theoretic model for security t the los ngeles interntionl irport. In Proceedings of the 7th interntionl joint conference on Autonomous gents nd multigent systems: industril trck, pges Interntionl Foundtion for Autonomous Agents nd Multigent Systems,