A Dynamic Theory of Strategic Decision Making applied to the Prisoner s Dilemma

Transcription

1 Dynaic Theory of Strategic Decision Making applied to the Prisoner s Dilea Gerald H Thoas and Keelan Kane Milwaukee School of Engineering dapt of Illinois Introduction The classic prisoner s dilea (PD) gae has een extensively investigated y gae theorists since the late 95s, and has een scrutinized in oth theoretical and epirical contexts It has een concluded y any that the Nash equiliriu does not apply to this gae Here we reexaine the issue fro the perspective of a dynaic theory of strategic decision aking constructed along the lines of physics (Thoas 26) and show fro this larger perspective that the Nash equiliriu ust e extended to include dynaic possiilities We show that this forulation leads one to recognize that interactions siultaneously involve oth self-interest and the interests of others, even if one starts y adopting the assuption that agents are driven only y self-interest or only y other-interest This result has consequences far eyond the siple exaple of the prisoner s dilea The dynaic view is extraordinarily rich and provides a strategy for exaining general decision aking processes We note that the approach is ased on oserved ehaviors (strategies) and oserved outcoes (utilities) s such it is suject to direct oservation and refineent using the scientific ethod We use the noenclature egoists (self-interest) and altruists (other-interest) as descried for exaple y Eshel et al (998) and coine it with the physics fraework in order to exaine the PD gae This paper egins with a forulation of the Prisoner s Dilea, a description of the physical fraework, followed y a description of the PD gae in the sae ters

2 2 Dynaic Theory of Strategic Decision Making applied to the Prisoner s Dilea 2 Prisoner s Dilea Forulation The prisoner s dilea is a gae etween two players/prisoners specified y a payoff atrix for each player The payoff atrix for player is: G N C N C There is an identical payoff atrix for player 2 Using the standard gae theory analysis we see that if player chooses N to not confess, the worst that can happen is that player 2 confesses C2 and the payoff is - If however player chooses C to confess, the worst that can happen is that player 2 again confesses C2 with a payoff of Since this last choice has a etter payoff, the standard gae theory analysis concludes that player confesses Player 2 sees exactly the sae gae atrix, and so also confesses The identified solution is called the Nash equiliriu It is the gae theory optial solution despite the dilea that if each player were not to confess they would oth e etter off Indeed, the latter also squares etter with what soe of would expect To apply the new dynaic theory of decisions, we first frae the gae each player sees as an equivalent syetric zero-value gae; ecause there are two distinct payoffs the overall gae is a non-zero su gae Each player s view is of a zero-su gae whose expected payoff is - units The equivalent syetric gae is a three player gae: two syetric players whose strategic choices are the union of the original two players strategies, and a third hedge player with a single strategy whose payoffs are such to insure the new gae has a zero payoff if the optial strategies of the original gae are chosen n identical equivalent gae exists for player 2 The payoff atrix when ultiplying the equiliriu strategy yields zero The relative scale for the hedge strategy does not affect the strategic outcoe The syetric gae payoff is the antisyetric atrix: F N C N C H N C N C 2 2 H 2 2 Syetric gaes are equivalent to linear prograing proles, and adit to siple nuerical analysis Syetric gaes can also e solved as coupled differential equations (for exaple, see Luce and Raiffa): dv F V d This differential equation will e the starting point for a dynaic theory This equation descries stationary ehavior if the flow vector V does not change in tie, which

3 3 Dynaic Theory of Strategic Decision Making applied to the Prisoner s Dilea occurs when the right hand side vanishes The dynaic notion of stationary flow replaces the gae theory notion of equiliriu The differential equation also descries dynaic ehavior that is not stationary 3 Egoists and altruists in a dynaic theory 3 Thoas Dynaic Theory of Strategic Decisions Thoas (26) takes the differential equation for of the gae theory seriously as representing the gae, identifying the hedge strategy with tie general for of the differential equation is postulated, taking into account the geoetry of the strategy space through the active geoetry etric eleents g and inactive geoetry etric eleents The etric eleents provide the ethod for specifying distance etween two plays of the sae gae If the etric eleents are independent of a strategy, it is considered inactive; otherwise it is active Strategies are defined as in standard gae theory with the gae in its extensive for, and with an antisyetric decision atrix F for each player In addition to strategies for each player, there is the outcoe or utility for that player: the hypothesis is that the outcoe is an inactive diension of the geoetry Successive plays of the sae gae represent a flow in this geoetry For a active strategies, the flow is represented as V ; for inactive strategies the flow is considered a charge and represented y V The ehavior of a new play of a gae is deterined y those gaes already played y a set of deterinistic causal equations in which pressure p and atter density have een ascried to the gae The full set of equations is the econoic version of Einstein s equations applied to this geoetry The resultant flow equations, the econoic version of Euler s equations, result fro the conservation of energy and oentu: DV p g V 2 V V a ha p These equations replace the differential equation fro the last section with the weighted su, fro each player s charge ties its payoff, identified as the zero-su syetric zero-value gae F The su represents the coposite payoff that deterines the ehavior of the gae, and will in general e quite different fro the separate payoffs for non-zero su gaes unless each player sees the sae zero-su gae There are three iportant consequences of the ove equations: any definition of equiliriu is dynaic and ased on the flows eing stationary; the gae aspect of the equation is represented as the specific coposite su ; and there are nongae aspects that influence the dynaics Moreover, in the coposite su, the coefficients are not aritrary, ut are theselves governed y equations that also derive fro the econoic Einstein equations, and are the analogs of Maxwell s equations:

4 4 Dynaic Theory of Strategic Decision Making applied to the Prisoner s Dilea g g g F p V V ac d a 2 cd g a These equations show that the current, the product of the charge V, flow V, atter density p and coupling constant, deterine the player decision atrix F The for of these Maxwell equations have profound iplications that depend on the nuer of active diensions of the geoetry, and split into two distinct sets of equations The first set, the tie coponent of the equations, has the for of Coulo s Law E j The iportant consequence of this is that this equation iplies that like charges repel and opposite charges attract This equation does not involve second order tie derivatives, and is a constraint equation that ust e satisfied as an initial condition If like charges are neary at the start, the for of the equations iplies that they ove away fro each other as tie increases For this reason we argue that systes coprised of only egoist players (or only altruist players) are expected to fly apart Matter as a rule consists of equal ixtures of charges separated y short distances We identify the positive charges with altruistic ehavior, negative charges with egoist ehavior and equal ixtures with noral ehavior We assert that noral ehavior is likely coposed of equal aounts of altruist and egoist ehavior The second set, the space coponents of the equations, has the for analogous to the Biot-Savart Law B E t j in which currents generate tie changing agnetic and electric fields For a general diension of the geoetry, such equations can e converted to wave equations involving the second order tie derivatives of a vector potential Such waves radiate with a fixed and finite velocity whenever the active space-tie diension D is greater than two In such cases the nuer of polarization states for the radiation is D 2 In physics, this active diension is four, resulting in two possile polarization states; such radiation is called light or photons For the prisoner s dilea exaple, the total diension of the space is seven with two inactive diensions (the utilities or outcoes), so the active diension is five or less For systes that are not static (tie is active) and in which each player has at least one active strategy, then the active diension is three or greater There will then e radiation and this radiation deterines which past events can influence any given event The radiation is an intrinsic property of the strategic decision fields of each player, is a consequence of the theory eing a gauge theory, and its confiration would provide strong support for this physical approach In particular if a player consists of opposite charges separated y soe distance circling each other, then there would e acceleration and hence radiation leading to the charges collapsing onto each other In physics, this is prevented y adopting the quantu wave view of atter We thus have soe striking consequences that extend far eyond the prisoner s dilea gae

5 5 Dynaic Theory of Strategic Decision Making applied to the Prisoner s Dilea 32 Coposite Fields, Charges and Null Behaviors If a gae is played the sae way over and over, the gae creates a current for each player that through the hoologous Maxwell equation ove creates the decision field F test play of the gae will then see a force through the hoologous Euler flow equations Since the equations are highly non-linear, we elieve it is helpful to think of these equations sequentially to descrie how they work In this conceptualization, the force ased on the decision for a given player is deterined y the size of V The size can e judged against the direction of the decision field defined as the null vector defined as satisfying F If the flow is along the null direction there is no force; the null direction provides the direction of the static equiliriu It can e shown that non-equiliriu otion occurs in a helix around this direction, with a direction of rotation and a frequency given y the strength of the charge density There is a decision atrix for each player, and the corresponding null vectors are in general not equal We use these two concepts elow in our analysis of the prisoner s dilea: for each player we identify the null direction and the charge V We ephasize that the null direction is deterined y the sources that generate it, and so is not independent of the charge In a coplete analysis, we would solve the coupled equations siultaneously For this paper, we siply illustrate the possiilities y considering appropriate special cases 4 Prisoner s Dilea nalysis There are two charges altruist (positive) and egoist (negative) In addition, for each prisoner there are two null ehaviors depending on whether the decision atrix derived fro the Maxwell equations results fro an altruist or egoist charge There are sixteen cases in all We create the coposite y taking the su of the player decision atrices weighted y their player s charge assuing that a player is either altruistic or egoistic The sixteen possile cases can e reduced to the following four cases: There is a coposite ehavior constructed fro two egoists (egoistic null ehavior); There is a coposite arket constructed fro two altruists (altruistic null ehavior); There is coposite arket constructed fro an altruist and an egoist where each acts appropriately or each acts oppositely to their null ehavior; nd there is a coposite arket constructed fro an altruist and an egoist where one acts appropriately and the other oppositely 4 Egoistic null ehavior fro two egoists We elieve the coposite arket field for the prisoner s dilea, as usually stated reflects an egoist null ehavior In this and the other exaples, we copute

6 6 Dynaic Theory of Strategic Decision Making applied to the Prisoner s Dilea assuing that the charge for an egoist is, for an altruist is +, and that the egoist null ehavior is that given in the earlier section With this in ind the coposite field is ade up of two identical payoff atrices If oth players are otivated y self-interest, they oth confess If the players utilities were coparle, such as expressile in for exaple in dollars, the su of the payoffs is 8 units, significantly less than the penalty of 2 if they cooperated The coposite atrix is: The dilea is that this does not correspond to our experience We know that prisoners, such as Mafiosi, ay hold a great deal of loyalty to others, and will not in general confess The arguent is even stronger if we consider political prisoners Nevertheless, for this case, the null vector of the coposite is that each player confesses since this is the null vector of each ter 42 ltruistic null ehavior fro two altruists n altruistic player differs fro an egoistic player in that such a player sees a different utility and arket Let s start with a purely egoistic world and appeal to the hoologous Maxwell equations to otain guidance for the for in a purely altruistic world If the sign of the charge changes, and the arket fields (ie the notion of utility) keep the sae signs, then the flows will in general e reversed To keep the flow positive, it suggests we change the sign of the utility This has the intuitive appeal that the altruist akes the sae type of in-ax arguent that the egoist does, ut the altruist player takes their utilities to e opposite in sign The coposite arket field is ade up of what each player sees: The first ter is player Such a player would look at the possiilities as follows: if she does not confess, the worst that can happen is if player 2 does not confess, and she gains units; if she confesses, the worst that can happen is that player 2 does not confess, and she gains unit Of these two cases the est for her is not to confess The optial strategy as she sees it is Player 2 sees the sae possiilities Thus this is the opposite of two egoists

7 7 Dynaic Theory of Strategic Decision Making applied to the Prisoner s Dilea If oth players are altruistic, they oth reain silent The coposite arket field yields the sae result The coposite gae is altruistic, oth players choose to reain silent, and the total payoff if their utilities were copatile is 2 We otain the iportant result that with two altruistic players, gae theory applies using the usual rules for coputing the null ehavior if we allow the utilities to reflect altruistic charge 43 Coposite gae with egoist and altruist where each acts appropriately (or oth oppositely) to their ehavior In the two previous cases, the null ehavior of the coposite gae does not depend on the charges of the players If an egoist played as if she were an altruist (opposite charge), then only the coefficient of the coposite null ehavior would change resulting in the sae equiliriu value If we have an egoist null ehavior for player and altruist null ehavior for player 2, then the coposite null ehavior depends on the relative sign and value of the charges To keep things siple we continue to focus on unit charges, and allow the charges to e either + or Thus in the two previous cases, we were le to deduce the coposite null ehavior y taking the null ehavior of each player individually In the reaining cases, the coposite null ehavior need not e the null ehavior of any one player The null ehavior is the stationary direction along which the coposite arket produces no force, We find the coposite null ehavior y finding those vectors in the null space of the coposite atrix The null vectors can e found using standard techniques and verified y inspection We provide only the answer here, and the reader can easily verify their correctness We start y taking the charge for player to e and the charge for player 2 to e + corresponding to the arket that they see respectively The coposite field is then: We have taken the for for player 2 fro the previous section Neither the pure egoist nor the pure altruist solution is an equiliriu value The coposite decision atrix is: The null ehavior is The altruist (player 2) predoinantly chooses the option to confess (with odds of 9:) and the egoist (player ) predoinantly

8 8 Dynaic Theory of Strategic Decision Making applied to the Prisoner s Dilea chooses the option to not confess (with odds 9:) Each player is ipacted significantly ecause of the presence of the oppositely ehaving player If each player were to act oppositely to their arket, then the charges would e opposite to the calculation ove Though it changes the direction of otion around the null direction, it does not change the direction 44 Coposite gae with egoist and altruist where one acts appropriately and the other oppositely We enuerate the total cases in the following tle, where the rows and coluns lel player and 2 respectively, E and represent egoist and altruist, and M E and M represent the egoist or altruist null ehavior the player sees: / 2 E M E E M M E M E M X X X X E E 4 3 E M X X X X M X X X X 3 4 M X X X X Of these 6 total possiilities there are four distinct cases of which we have considered the egoistic null ehavior with two egoists ( X ), the altruistic null ehavior with two altruists ( X 2 ), and the coposite gae with one egoist null ehavior and one altruistic null ehavior with oth acting the sae (or opposite) to their null ehavior ( X 3 ) There reains one case X 4 that is distinct fro the others We select as representative an egoist null ehavior and an altruistic null ehavior where oth players are egoists ( X 4 ) The coposite arket is: The coposite gae is: The coposite null ehavior is The negative strategy ay lie outside the allowed range, and a ore detailed dynaic analysis is called for However we suggest a possile interpretation assuing strategies are ounded inside a ox characterized y positive strategies In this case, the dynaic equations operate even 9

9 9 Dynaic Theory of Strategic Decision Making applied to the Prisoner s Dilea though it is not possile to ove along that direction The ox provides additional oundary conditions The negative strategy for the egoist reflects the possiility that he can not play the confess strategy indefinitely This is ecause the optial strategy lies outside the ox Siilarly, the altruist can t play the silent strategy indefinitely Both players are influenced y the other s ehaviors 5 Suary We assert that the Nash equiliriu ust e extended to include dynaic possiilities dynaic theory extends the static standard gae theory, and structurally changes the notion and iportance of gae theory equiliriu Dynaic theories are characterized y coplex ehavior and distinguished y notions such as fixed points, stationary or constant flow that can e attractive or repulsive, and periodic or sei-periodic ehavior dynaic theory can display even ore coplex ehavior such as turulence, chaos or ios [Selli 25, and this conference], each with their own characterizations Indeed we have oserved iotic ehavior in the proposed dynaic theory Our analysis is ased on a causal evolution of ehavior fro soe fixed point in tie Because of the nature of the dynaics we are led to the conclusion that gaes are not player only y altruists, or indeed only y altruists, ut y players reflecting an equal ixture of oth attriutes (charges) To the extent that such charges occupy neary spaces, it ay e that soe quantu forulation is necessary for the theory to e consistent The focus of this paper has een on a ehaviorist view of the prisoner s dilea, and on how to articulate the ehavior inside a dynaic theory There are any other interesting questions that relate to the prisoner s dilea, and in the process of studying that prole we have collected a few preliinary though y no eans coplete thoughts on these questions lthough the prisoner s dilea originated fro efforts to anticipate whether suspects arrested for particular cries would confess or reain silent (hence the gae s nae), the prisoner s dilea fraework has een generalized and redesigned to investigate other phenoena that rely on siilar reasoning eg, whether people will pollute the environent or will dispose of waste properly (see, eg, Caerer, 23, Chapter 2) There are epirical investigations of huan participants presented with prisoner dilea gae situations, which have yielded results suggesting that other contexts in which the gae is played affect players decisions Soe of the phenoena oserved in epirical studies have otivated psychological approaches to odeling the gae (eg, Dufwenerg & Kirchsteiger, 998; Rin, 993) The hallark of these psychological fraeworks is that they attept to odel players fairness or kindness,, as well as each player s eliefs out whether his or her own actions will e reciprocated with (un)kind or (un)fair actions s Rin (993) notes in his psychological odel, the notion of altruis can ear on the notion of fairness: the sae people who are altruistic to other altruistic people are also otivated to hurt those who hurt the (p 28, ephasis in original)

10 Dynaic Theory of Strategic Decision Making applied to the Prisoner s Dilea References Caerer, Colin F (23) Behavioral Gae Theory: Experients in Strategic Thinking Russell Sage Foundation: New York, NY Dufwenerg, Martin & Kirchsteiger, Georg (998) theory of sequential reciprocity Tilurg Center for Econoic Research discussion paper 9837 Eshel, Ilan; Sauelson, Larry; & Shaked, vner (998) ltruists, egoists, and hooligans in a local interaction odel The erican Econoic Review, March, Luce, R D, and Raiffa, H (957), Gaes and Decisions (Dover Pulications, NY) Rin, Matthew (993) Incorporating fairness into gae theory and econoics erican Econoic Review, 83, Selli, Hector, Bios, a Study of Creation (World Scientific, 25) Thoas, Gerald H, Geoetry, Language and Strategy (World Scientific 26)