Costless Delay in Negotiations

Transcription

1 Costless Delay in Negotiations P. Jean-Jacques Herings Maastricht University Harold Houba VU University Amsterdam and Tinbergen Institute Preliminary version. Do not quote without the permission of the authors January 31, 2012 Abstract We study strategic negotiation models featuring costless delay, general recognition procedures, endogenous voting orders, and finite sets of alternatives. An example shows non-existence of stationary subgame-perfect equilibria SSPEs) and the inapplicability of the recursive equations for deriving SSPE utilities. These equations apply if and only if perpetual disagreement is excluded. SSPEs under costly delay exist, and also their limit under vanishing costly delay. Limit SSPEs can only be SSPE under costless delay if these exclude perpetual disagreement. Robustness against one-stage-deviations is insuffi cient for SSPE under costless delay, and we provide necessary and suffi cient conditions for SSPEs that exclude perpetual disagreement. JEL Classification: C72 Noncooperative Games, C73 Stochastic and Dynamic Games, C78 Bargaining Theory Keywords: Bargaining, existence, one-stage-deviation principle, dynamic programming, recursive equations, Markov decision process The authors thank Dinard van der Laan, János Flesch and Eilon Solan and several participants of the Conference on Economics Design 2009 and the 11 th conference of the Society for the Advancement of Economic Theory for valuable comments. Department of Economics, Maastricht University, P.O. Box 616, 6200 MD Maastricht, The Netherlands, P. Jean-Jacques Herings would like to thank the Netherlands Organisation for Scientific Research NWO) for financial support. Department of Econometrics, VU University Amsterdam, De Boelelaan 1105, 1081 HV Amsterdam, Netherlands.

2 1 Introduction Strategic negotiation theory has contributed significantly to the understanding of negotiation processes. 1 Many influential contributions analyze costly delay or limits of vanishing costly delay, and only a few consider costless delay. 2 Costly delay is often modeled as a risk of breakdown or discounting. Negotiation models with costly delay have the property of continuity at infinity, a suffi cient condition under which robustness against one-stage deviations characterizes subgame-perfect equilibrium SPE) as well as a system of recursive equations characterizes the stationary subgame-perfect equilibrium SSPE) payoffs. Negotiation models with costless delay, however, lack continuity at infinity, see e.g.?). This raises the obvious question whether robustness or the recursive equations still characterize SSPE in case of costless delay. For a motivating example, we show the necessity to address these issues. This example is the symmetric hedonic game of coalition formation proposed in?) and?) that has no SSPEs in pure strategies under costless delay. We derive the symmetric SSPE in mixed strategies under costly delay, and derive its limit under vanishing costly delay. Even though this SSPE s limit strategy profile is well defined, it fails robustness against one-stage deviations under costless delay and, hence, the limit SSPE obviously fails as a SSPE under costless delay. Moreover, we show nonexistence of symmetric SSPE in mixed strategies under costless delay due to a lack of robustness against one-stage deviations. Technically speaking, we show that the correspondence of symmetric SSPE strategies lacks upper semi-continuity and may be empty valued. A puzzling phenomenon is that the symmetric SSPE converges to a strategy profile that induces Pareto ineffi cient perpetual disagreement, whereas the corresponding SSPE utilities converge to Pareto effi cient utilities. Also puzzling is that the limit SSPE utilities are a solution to the system of recursive equations but fail to represent the correct expected utilities. We provide an explanation for 1?) boosted the literature on strategic negotiations. For surveys we refer to e.g.?),?),?),?) and?). 2 Costless delay is analyzed in e.g.?),?),?),?),?),?),?) and?). 1

3 these phenomena that enables us to derive conditions under which the robustness against one-stage deviations and recursive equations characterize SSPEs in mixed strategies. We address these issues in a general negotiation model in discrete time with an arbitrary number of players, stochastic recognition of the proposing player, public and sequential voting by endogenous orders and discrete sets of feasible alternatives. We explicitly include costly and costless delay to study limits of vanishing costly delay. At any negotiation round, one player is recognized to make a proposal. Proposals specify an alternative, a set of players who have the right to approve this alternative, and an order in which the players in this set sequentially and publicly vote. The first vote against the proposed alternative ends the current round of voting. After that, nature decides whether the negotiations permanently break down in case of costly delay), or who will be next round s recognized player. The negotiations start with nature recognizing the first proposing player. Our model s recognition rules are more general than the institutions analyzed in the literature to allow as special cases: Fixed rotating orders of recognized players including alternating offers procedures; Markov recognition probabilities including stationary random recognition rules; and coalitional negotiation procedures including endogenous protocols in which a rejector becomes the next recognized player in case the negotiations do not break down. 3 The main reasons to assume public and sequential voting in our model is that i) it captures such voting rules of several negotiations models in the literature and ii) that, in SSPE, such voting rules achieve the stage-undominated voting strategies under simultaneous voting rules in?). So, our model implicitly obtains the interesting voting strategies under simultaneous voting. The endogenous voting orders in our model extend upon the exogenous voting rules in the literature. This class of negotiation models belongs to the class of recursive games with perfect information, which is a subclass of stochastic games. Under costly delay, existence of an SSPE is not an issue, since it follows from standard results on equilibrium existence in 3 In terms of institutions, we encompass models analyzed by e.g.?),?),?),?),?),?),?),?),?) and?),?),??,?),?),?) and?). 2

4 stochastic games, see e.g.?),?), and?). For the class of stochastic games,?) show that generically the set of SSPEs is finite, and?) show that generically there is an odd number of SSPEs. For stochastic games under costless delay, non-existence of Nash equilibrium in mixed strategies has been noted by?) and has spurred an extensive literature seeking existence of weaker notions of Nash equilibrium in special classes of stochastic games and conditions under which SPE exist. For ε-nash equilibria, existence has been shown by?) for two-person zero-sum stochastic games and by??,? and?), for general 2-player stochastic games. A general result for stochastic games with three or more players is lacking thus far. For the subclass of recursive games with non-negative utilities,?) have demonstrated the existence of a subgame-perfect ε-equilibrium for every ε > 0. For a class of coalitional bargaining models that belong to the class of negotiation models we consider,?) derive a necessary and suffi cient condition for existence of pure SSPEs under costless delay. General conditions for existence of mixed S)SPE is an open issue. In our analysis, we try to impose just what is needed to establish results. For all our results, which we are about to summarize, strategy profiles that exclude perpetual disagreement are all we need. A suffi cient condition such that all SSPE strategy profiles exclude such disagreement is the following: Every player is able to propose some alternative and some coalition whose members all prefer this alternative to the status quo. This is a mild assumption that relaxes the popular assumption of an essential bargaining problem. It is also easy to check. As mentioned, our results are more general than that. In our summary, we present our results as generally as possible. Our first result states that the expected utilities consistent with stationary strategy profiles solve the recursive equations if and only if such strategy profiles exclude an infinite cycle of possibly stochastically) recognized players, i.e., perpetual disagreement. The explanation is as follows: Stationary strategy profiles induce a Markov process and the corresponding expected utilities can be expressed in terms of this process. These utilities always satisfy the recursive equations, but not vice versa. The Markov process has 3

5 absorbing states that either represent which agreement has been reached or permanent) breakdown. This process might cycle forever on the other states, which represent who is recognized. Such cycling is excluded by the necessary and suffi cient conditions. Of course, under costly delay the positive risk of breakdown excludes perpetual disagreement a priori, and then the recursive equations are applicable. Only under costless delay it may occur that the equilibrium computations return stationary strategy profiles with perpetual disagreement and then the recursive equations are useless because the entire Null-space solves these equations. Indeed, this is the case in one of our examples: the limit symmetric SSPE induces perpetual disagreement, which explains one puzzle for the limit SSPE. Our second result characterizes the behavior associated with stationary strategy profiles that are robust against one-stage deviations. A recognized player s proposal maximizes his expected utility given the stochastic approval decisions of the other players and taking into account the expected continuation payoffs of disagreement after disapproval. A voting player always wields a threshold strategy such that he accepts rejects) for sure any alternative with expected utility of acceptance above below) this threshold. In case of indifference, he might randomize, as our motivating examples illustrate. This result confirms the SSPE behavior known for costly delay and, because this behavior extends to costless delay, this result shows that players behave qualitatively similar under costless delay. Our third result establishes that, under costless delay, robustness against one-stage deviations is no longer a suffi cient condition for SSPE. We show this by means of an example in which each player s security level in a bilateral alternating offers procedure is supported by always making unacceptable proposals and never approving to any agreement, i.e., perpetual disagreement. We derive two sets of stationary strategy profiles that are robust against one-stage deviations, but all profiles from one of these sets yield one of the players less than his security level. Clearly, the latter player fails to select a best response. Our fourth result establishes, under costless delay, novel necessary and suffi cient conditions for the subclass of SSPEs that exclude perpetual disagreement. These conditions 4

6 consist of robustness against one-stage deviations, the recursive equations and a condition that ensures that each player plays a best response. This latter condition is an immediate consequence of the observation that, given stationary strategies by the other players, each player s best response is the optimal solution of a Markov decision process with expected total rewards that are bounded. 4 Under a technical condition on the Markov decision process, one that is always satisfied in our model in case the strategy profile under consideration excludes perpetual disagreement, the set of optimal nonstationary history-dependent strategies always contains an optimal stationary strategy, which is a player s best response. The Markov decision process might also identify nonoptimal solutions, but a simple characterization how to select among solutions discriminates against such nonoptimal solutions. The latter selection is the additional condition in our result. Our fifth result deals with the existence of quasi SSPE, which we define as stationary strategy profiles that are robust against one-stage deviations pretending the recursive equations are always valid. Under costly delay, quasi SSPE is equivalent to SSPE. We show that the correspondence of quasi SSPE is non-empty, compact and upper semi-continuous. So, quasi SSPE always exist, and the limit of the set of quasi SSPEs read SSPE) under vanishing costly delay always exists. Moreover, every limit SSPE that excludes perpetual disagreement is also SSPE under costless delay. So, a suffi cient condition for existence of SSPEs under costless delay is that there exists a limit SSPE that excludes perpetual disagreement. However, as our motivating example shows, the limit SSPE may induce perpetual disagreement and, therefore, fail as SSPE under costless delay. Our focus on SSPE obscures that many of our results are more general than might appear. At the end of our paper, we show that, by enlarging the state space, more general results can be immediately obtained. Our results extend to SPE on the class of nonstationary strategy profiles that can be represented as finite automata, which covers most of the relevant strategy space in many negotiation models, see e.g. the discussion in Section We refer to e.g.?) for a survey of Markov decision processes. 5

7 in?). This class of automata include the automata needed to apply the method proposed in?) to establish lower and upper bounds on the set of SPE payoffs. The enlarged state space can also capture a stochastically fluctuating sets of feasible proposals read utilities), similar to e.g.?). This paper is organized as follows. After the introduction of the negotiation model in Section 2, two motivating examples are discussed in Section 3, where the second one yields the puzzling results that motivated this paper. The necessary and suffi cient conditions such that the expected utilities can be derived from the recursive equations are established in Section 4. Robustness against one-stage-deviations is investigated in Section 5. Vanishing costly delay is investigated in Section 6. Section 7 discusses extensions of our results and Section 8 concludes. 2 The Model Consider n 2 players who negotiate the selection of an alternative from m 2 alternatives in the shadow of a status quo under sequentially and public voting. Players are indexed by i and belong to the finite set N = {1,..., n}. Alternatives are indexed by a and belong to the finite set A = {a 1,..., a m }. The status quo is also the outcome under breakdown. It may or may not be available as the alternative "maintain the status quo". In order to distinguish both, the status quo is denoted by q. A voting order is a permutation with domain a coalition C N, where we allow for C being equal to the empty set. The set O 2 N denotes the collection of feasible voting orders, consisting of a group of players who have the right to approve. Proposals consist of an alternative a A and a voting order o O. A feasible voting order o defines the set Co) of players who have the right to approve the proposal, and if they all approve the alternative is implemented and the negotiations end. The non-empty set X i A O denotes recognized player i s set of feasible proposals. For convenience, the recognized player votes on his own proposed alternative, which ensures player i belongs to C o). In many negotiation models, however, 6

8 the recognized player automatically casts a vote in favor of his proposed alternative. A minor modification captures the latter, but it would exclude the voting protocol in e.g.?). Our formulation allows for the possibility that the set C o) depends on the proposed alternative. Also, the voting order allows that feasible alternatives may depend on sets of players. Negotiations proceed in discrete time, where t N denotes the t-th round of negotiations. At round t, recognized player i t N first proposes x t = a t, o t ) X it, after which all players in Co t ) sequentially and publicly vote according to o t. Given x t = a t, o t ), alternative a t is implemented if all players in Co t ) approve. Otherwise, the first voter in Co t ) against ends the voting in round t and alternative a t is rejected. The identity of the first voter in Co t ) against is denoted by r t. If all players in o t approve, we define r t = 0. If r t N, round t is concluded with a draw by nature that is modeled as a compound lottery: First, nature decides with probability δ [0, 1] whether the negotiations proceed to round t + 1. With complementary probability 1 δ the negotiations break down, leading to the implementation of the status quo q. Second, in case negotiations proceed to round t + 1, nature recognizes player i at round t + 1 with probability ρ i i t, x t, r t ) [0, 1], where i N ρ i i t, x t, r t ) = 1. Note that it is standard to identify costly delay with δ < 1 and costless delay with δ = 1. Prior to the first round, nature recognizes player i with probability ρ i [0, 1], where i N ρ i = 1. The vector of these probabilities is denoted ρ. The negotiation procedure fits the framework of multi-stage games with perfect information, see e.g.?). It has n + 2 stages per round t N under the understanding that players in N\Co t ) choose the trivial action "do nothing" after all players in o t have voted. In terms of?), the multi-stage game is cyclical with cycle length n + 2. Stages are indexed t, k), t N and k = 1,..., n + 2. The recognized player proposes at stage k = 1, all players sequentially vote or do nothing at stages k = 2,..., n + 1 and nature moves at stage k = n + 2. For notational ease, we extend the definition of r t to all stages in round t. For stages 7

9 t, 1),... t, n + 1), we define r t,k Co t ) {0} as follows. Since there is no voting in stage t, 1), we set r t,1 = r t,2 = 0. For k = 3,..., n + 2, we define r t,k = 0 if no rejection has occurred in stages t, 2),..., t, k 1), where "do nothing" and "vote against" have different meaning. Otherwise r t,k is equal to the player in Co t ) who rejected the proposal. Whenever it does not cause confusion, we denote r t, ot +2 by r t, consistent with previous notation. Histories are defined recursively for all t N and k = 1..., n + 2. The history up to stage t, k) is denoted h t,k. The initial history h 1,1 = i 1 ). For t N, define the history at the first stage of round t as h t,1 = h t 1,n+2, i t ), the history at the second stage as h t,2 = h t,1, x t ) and the history at stages k = 3,..., n + 2 as h t,k = h t,k 1, r t,k). The non-empty and finite set of all histories up to stage t, k) is denoted H t,k and the set of all finite histories is H = t,k) N {1,...,n+2} H t,k. Since the negotiation procedure has perfect information, histories define subgames and vice versa. Mixed behavioral strategies and strategy profiles are defined in the usual way: σ i is a function from the set of histories at which player i has to act into a probability distribution over the history-dependent set of feasible actions and Σ i denotes the set of all such strategies. A strategy profile is σ Σ i N Σ i. Sometimes we write σ = σ i, σ i ). Any strategy profile σ Σ induces cumulative probabilities that some agreement is accepted prior to or at round t. For σ Σ, π a, t; σ) [0, 1] denotes the cumulative probability of reaching agreement on a A at round τ t. For all σ Σ, these cumulative probabilities are well defined, non-decreasing in t, and bounded due to a A π a, t; σ) 1 for all t N. Hence, for all a A, π a, t; σ) converges as t goes to infinity and we define π a; σ) as this limit cumulative probability: π a; σ) = lim t π a, t; σ), a A. Note that 1 a A π a; σ) is the probability of perpetual disagreement. Players have expected utility functions. Player i derives utility from agreed upon alternatives and the status quo alternative denoted by the numbers u i a), a A, respectively, u i q). Because expected utility functions are unique up to affi ne transformations, we nor- 8

10 malize such that u i q) = 0 and ū i = max a A {q} u i x) 0. Also, we define u i 0 as u i = min a A {q} u i a). Expected utilities are defined in the usual way, and we omit stating these as functions of strategy profiles. In terms of cumulative probabilities, player i s expected utility in case of σ Σ is given by U i σ) = a A π a; σ) u i a) u i q) ) + u i q). 1) Note that U i σ) [u i, ū i ]. In case of perpetual disagreement with probability one, it holds that π a; σ) = 0 for all a A, and the expected utility U i σ) = u i q) = 0. This completes the description of the negotiation model as a multi-stage game with perfect information. Since we have a multi-stage game with perfect information, the concept of subgameperfect equilibrium SPE) is appropriate. The analysis in Section 4 and 6 deals with strategy profiles which we call stationary strategy profiles. We first define an exogenous partition of the set of all histories, and then define strategy profiles on this partition. 5 The partition for non-trivial rounds is as follows: At stage t, 1) only the identity of the recognized player matters. At voting stages t, 2),..., t, o t + 1), the identity of the recognized player and the proposal x made matter. Formally, for i N, x X i, and k {2,..., n+1}, we define H i) = {h t N H t,1 h = h t 1,n+2, i) for some t N}, H i, x, k) = { h t N H t,k h = h t 1,n+2, i, x i, 0,..., 0) for some t N }. A stationary strategy σ S,i for player i specifies σ S,i h t,k) = σ S,i h t,k ) whenever k = k = 1 and h t,k, h t,k Hi) or k = k {2,..., n + 1} and h t,k, h t,k Hi, x, k) for some x X i. Therefore, player i s stationary strategy reflects that bygones are bygones. When a player i N is chosen as the proposing player, he chooses a history-independent probability distribution over X i. When player i N is chosen as a responder at stage t, k), he conditions his behavior only on the recognized player and the proposal made. We define 5?) define stationary strategy profiles as strategy profiles on an endogenously determined partition, which, depending on the negotiation protocol, may be coarser than the one we study in this paper. 9

11 α i as a probability distribution over X i and α i x) as the probability that player i proposes x X i. Similarly, we define β i j, x, k i o)) as the probability that player i votes in favor of the proposal x = a, o) X j made by player j at stage k i o), where k i o) {2,..., n + 1} denotes the stage where player i votes according to the proposed voting order o. All such probabilities form β i = {β i j, x, k i o))} j N,x X j. A stationary strategy profile is denoted by α, β), where α = α 1,..., α n ) and β = β 1,..., β n ). We write α i, β i ) for the stationary strategies of all players except player i. A SPE in stationary strategies is denoted SSPE. For δ [0, 1), existence of an SSPE is not an issue, since it follows from standard results on equilibrium existence in stochastic games, see e.g.?),?), and?). For the class of stochastic games,?) show that the set of SSPEs is generically finite, and?) show that generically there is an odd number of stationary equilibria and they provide an algorithm to compute the equilibrium that would be selected by a generalization of the tracing procedure. To conclude this section, we illustrate how institutional aspects of several influential bargaining models can be seen as special cases. We first discuss the set of feasible proposals. In bilateral unanimity) bargaining over a discrete set of alternatives A, 6 X i = {a, i, i)) a A} expresses that approval by responding player i i is required and that the voting order is exogenous with player i who votes before player i. We interpret i s vote against his proposed alternative as the alternative "pass". The model in?) is captured by X i = {a, i, i)) a A} and this reflects that the recognized player casts the last vote. Voting against by the recognized player is interpreted as if this player revokes his proposed alternative. Unanimity bargaining imposes the subset O u = {o O C o) = N} of feasible voting orders. Then, unanimity bargaining over A and allowing all endogenous voting orders is captured by X i = A O u, whereas the case of an exogenous voting order, say in ascending order, is captured by X i = {a, 1, 2,..., n)) a A}. Majority approval requires the subset O m = {o O C o) > n/2} of feasible voting orders on subsets of 6 For example, in case of a smallest money unit, see e.g.?) and?). 10

12 players who form a majority. Then, X i A O m. In case player j is a veto-player, then only x = a, o) belong to X i with j o reflecting that the veto-player always casts a vote. Or, in case player i is a dictator, only x = a, i)) belong to X i reflecting that i does not require approval from the other players. These special cases can be easily extended to general collections of decisive coalitions among arbitrary number of players as in e.g.?) and??,?). In principle, it is a minor modification to exclude recognized player i from the proposed voting order o and let this player do nothing during the voting, as in our motivating examples. Next, we consider coalitional negotiation models in which the recognized player proposes an alternative and a coalition, and the voting order among the coalition s members is exogenous, see e.g.?) for such exogenous voting protocols in which recognized players do not vote. In such models, the recognized player at round t propose an alternative a t A and a coalition C t N containing this player. The coalition C t specifies an exogenous sequential voting order o t = õ C t ), where õ C) maps C into a permutation of players in C. Alternatively, we might say that the recognized player chooses an alternative and an exogenous voting order where the order specifies the proposed coalition. Formally, for player i we define the set O i = {o O C N : o = õ C) and i C} consisting of all voting orders containing player i. Then, X i A O i. Recall that restrictions on a may depend on the proposed coalition C o), such as subsets A Co) A. If we additionally impose that the recognized player automatically casts a vote in favor and that there is one exogenous voting order per coalition, then X i A O i captures the bargaining procedure in e.g.?). This procedure returns in our second motivating example. We now turn to popular recognition rules in the literature. These are captured as follows: The bilateral alternating-offers procedure imposes ρ 2 1, x t, r t ) = ρ 1 2, x t, r t ) = 1. The modification in?) is captured by ρ 1 1, x t, 1) = ρ 1 2, x t, 1) = 1 and ρ 2 1, x t, 2) = ρ 2 2, x t, 2) = 1. Fixed rotating orders of recognized players can be modelled similarly, for example the infinitely-repeated order 1,..., n is captured by ρ i+1 i, x t, r t ) = 1 for all i, r t 11

13 N, where we write i + 1 instead of i + 1 mod 3. Markov recognition probabilities ρ i,j, i, j N, impose recognition probabilities ρ j i, x t, r t ) = ρ i,j. These include stationary random recognition rules as a special case in which all ρ i,j are independent of recognized player i. Next, we consider once more coalitional negotiation models in which the recognized player proposes from X i A O i. When we ignore that the recognized player does not vote, the essence of the protocol in e.g.?) is that the first voter in o t against becomes next round s recognized player, i.e., ρ r t i, a t, o t ), r t ) = 1, for all r t O i. 3 Motivating Examples We discuss two important examples in this section, where we restrict attention to symmetric strategy profiles for explanatory reasons. The first example illustrates some technical issues that arise in applying robustness against one-stage deviations and the recursive equations. The order of presentation follows the structure of our analysis. The second example illustrates that the symmetric SSPE under costly delay converges as the costly delay vanishes, but that its limit fails as an SSPE under costless delay. This example does not have any symmetric SSPE under costless delay. This example also exhibits the puzzling results discussed in the introduction. Example 1 Common-interest alternating-offers bargaining Consider bilateral alternating-offers bargaining with two players, so N = {1, 2}, on a single alternative â in the shadow of the status quo q. The utility of accepting â is 1 for each player, and 0 for the status quo. The sets of feasible proposals are given by X 1 = {â, 2)), pass} and X 2 = {â, 1)), pass}. 7 To obtain the alternating-offers bargaining procedure, we specify recognition probabilities ρ 1 2, x, r) = 1 and ρ 2 1, x, r) = 1 for all x X and r N. This example is a special case of?) with pass as a minor extension. 7 For expository simplicity, we interpret â, i)) as â, i, i)) and i as the first voter votes in favor. Similar, pass means â, i, i)) and i as the first voter votes against. 12

14 We consider symmetric stationary strategies: The recognized player proposes with probability α alternative â and passes with probability 1 α. When it comes to a vote, β [0, 1] denotes the probability of approving â. Conditional on being recognized, v p denotes the expected utility for the proposing p) player. And conditional on being the other player, v r denotes the expected utility of the responding r) player. Given the stationary strategy profile α, β), agreement on â in round t is reached with probability αβ [0, 1] per period and, under costless delay, the negotiations proceed to round t + 1 with probability 1 αβ) t. Hence, the conditional expected utilities v p and v r are equal and given by v p = v r = τ=0 1 αβ)τ αβ 1. Clearly, these utilities are discontinuous in the product term αβ, because v p = v r = 1 whenever αβ > 0 and v p = v r = 0 for αβ = 0. Expressed as recursive equations, we would write v p = αβ +1 αβ) v r and v r = αβ + 1 αβ) v p. Note that, for αβ = 0, we obtain the undetermined system v r = v p and any v p R is a solution. Our analysis of the recursive equations is performed in matrix notation. Then, the recursive system [ ] [ ] [ ] 1 αβ 1 v p αβ = αβ 1 1 v r αβ is non-singular if and only if αβ 0. These recursive equations apply if and only if the stationary strategy profile rules out perpetual disagreement. To put it differently, the negotiations conclude with probability one within finite expected time with agreement on â. In this case, v p = v r = 1 is the unique solution. However, when perpetual disagreement occurs due to αβ = 0, then the singular and recursive system yields the entire Null space as its solution. This space contains the expected utilities v r = v p = 0 that are consistent with the stationary strategies in case αβ = 0. Robustness against one-stage deviations together with value functions is a well-known technique that implies the following three conditions in our example, which we state without further detail: 13

15 1. For v r [0, 1], the recognized player solves v p = max α [0,1] α β β) vr ) + 1 α) v r = max α [0,1] 1 vr ) αβ + v r. Thus, 1 v r ) β > 0 α = 1. And 1 v r ) β = 0 α [0, 1]. 2. For v p [0, 1], the responding player solves max β β) β [0,1] vp. Thus, v p < 1 β = 1. And v p = 1 β [0, 1]. 3. Conditional expected utilities given by v p = v r = τ=0 1 αβ)τ αβ. The recursive equation for the recognized player is clearly recognizable under 1, for the responding player we should realize that v r = α max β [0,1] β β) v p) + 1 α) v p reads as the second recursive equation. So, robustness against one-stage deviations implicitly imposes the recursive equations. The third condition imposes expected utilities that are consistent with stationary strategies. As we have seen, the Null space in the singular case contains more than these consistent expected utilities. Straightforward solving of these three conditions rules out that αβ = 0 and v p = v r = 0 in any SSPE, because these value functions would imply the contradiction α = β = 1 by the first two conditions. So, αβ > 0 and, by the third condition, v p = v r = 1. Then, by the first two conditions, all α [0, 1] and β [0, 1] are optimal including all αβ > 0. Note that v p = v r = 1 and αβ = 0 would be inconsistent with the third condition, i.e., v p = v r = 0, and fail as SSPE. Therefore, these three conditions yield the set of symmetric stationary strategies {α, β) [0, 1] 2 αβ > 0} and v p = v r = 1. This set is not closed, which differs from the case of costly delay, i.e., δ < 1. The closure of the above set can be obtained if we solve the first two conditions and the recursive equations replacing the third condition, which we will call quasi SSPE. Robustness against one-stage deviations is a necessary condition for S)SPE, and it is also suffi cient under costly delay. Is robustness 14

16 also suffi cient under costless delay? For this example it is, because every stationary strategy profile such that αβ > 0 yields the maximal attainable utility in this example. For costly delay, we note that for δ < 1 substitution of δv p < 1 for v p and δv r < 1 for v r in the calculations above immediately establishes that α = β = 1 with v p = v r = 1 is the unique SSPE. Trivially, when taking the limit δ goes to 1, this SSPE converges to a limit point in the set SSPE strategies. In general, such limit points are quasi SSPE. Quasi SSPE are SSPE if the conditional expected utilities can be replaced by the recursive equations. The latter is equivalent to negotiations that conclude with agreement within finite expected time for sure. In this example, if αβ > 0. As a corollary, existence of SSPE at δ = 1 can be established by considering limit SSPEs and checking whether the negotiations conclude within finite expected time in the limit SSPE, which is a technique that generalizes those suggested in?) and?). Example 2 The Condorcet paradox Consider negotiations between three players in a game of coalition formation. We have N = {1, 2, 3} and three possible alternatives that are related to one of three possible coalitions that may form: {1, 2}, {2, 3}, and {3, 1}. We have A = {a 12, a 23, a 31 } and assume utilities are given by u a 12) = 2, 1, 0), u a 23) = 0, 2, 1), and u a 31) = 1, 0, 2). Players propose coalitions in which they are contained and decision making takes place by means of majority voting. That is X 1 = {a 12, 2)), a 31, 3))}, X 2 = {a 23, 3)), a 12, 1))}, X 3 = {a 31, 1)), a 23, 2))}. The utilities display a cyclical pattern that resembles the Condorcet paradox in the sense that players 2 and 3 prefer coalition {2, 3} to {1, 2}, players 3 and 1 prefer coalition {3, 1} to {2, 3}, and players 1 and players 1 and 2 prefer coalition {1, 2} to {3, 1}. The formation of a coalition is determined by the negotiation process in e.g.?). Some player, say i N, is selected randomly at the first round. This player proposes to one of the 15

17 other players to form a coalition, either x + = a i,i+1, i + 1)) or x = a i 1,i, i 1)). 8 So, i prefers the coalition with i+1 to the coalition with i 1. If the player who is proposed to, say j N \{i}, approves, the negotiations end with i and j forming a coalition. Otherwise, j is next in turn to make a proposal, unless breakdown occurs. Thus, ρ j i, x, j) = 1 and j proposes next with probability δ. The stationary partition of relevant histories is as follows: i proposes, i is proposed to by i 1, and i is proposed to by i + 1. In this example, we consider symmetric stationary strategies. Such strategies are summarized for each player by three probabilities, α, β, and β +, where α denotes a player s probability of proposing x +, his most preferred coalition. It follows that the player proposes x with probability 1 α. The probability by which a player approves x is given by β and the probability a player approves x + is given by β +. A symmetric SSPE is therefore described by α, β, β + ). Conditional on being recognized, v denotes the expected utility for the recognized player, v + denotes the expected utility of his most preferred partner, and v that of his least preferred partner. Clearly, 0 v+v + +v 3. In particular, perpetual disagreement implies v+v + +v = 0, whereas for δ < 1, v + v + + v = 3 only holds if agreement is immediate. The following result is shown in the Appendix. Proposition 3 1. For all δ 0, 1/2], the unique symmetric SSPE is given by α, β, β + ) = 1, 1, 1) with conditional expected utilities v = 2, v + = 1, and v = For δ 1/2, 1), the unique symmetric SSPE is given by α = 1, β = 1 δ2 ) + 1 δ) 1 + 2δ) δ 2 0, 1), and β + = 1, with conditional expected utilities v = 1/δ 1, 2), v + = 1, and v = 1 β )δ = 1 1 δ) 1 + 2δ). δ 8 We write i + 1 or i 1 instead of i + 1 mod 3, respectively i 1 mod 3. 16

18 3. For δ = 1, there does not exist any symmetric SSPE in mixed strategies. For δ [0, 1), the recognized player always proposes his most preferred coalition, i.e. x +, with probability one. A player always approves his most preferred coalition, i.e. β + = 1. For δ 1, a player also approves his least preferred alternative, i.e. 2 β = 1, and consequently, there is immediate agreement with probability one on the recognized player s most preferred coalition. However, for δ > 1, a player randomizes when voting on his least preferred 2 coalition, i.e. 0 < β < 1. Nevertheless, the recognized player forgoes the immediate agreement for sure on x and strictly prefers the risky proposal x +. It follows that v > 1. In such an SSPE, negotiations end with probability 1 β ) t 1 β > 0 in round t and, before termination we observe the following cycling behavior on the equilibrium path: first proposer i proposes to i + 1, who in turn proposes to i 1, who in turn proposes to i and so on. Agreement is reached with probability one within finite expected time. Since δ < 1 means that delay is costly, the SSPE is Pareto ineffi cient without relying on features like asymmetric information or increasing cake sizes over time as in?). v + v + + v δ When we plot v + v + + v, we obtain an U shape on the domain 1, 1), with minimum 2 around 2.60 at δ When δ goes to 1, v + v + + v goes to 3 with a slope converging 17

19 to +. Although this limit exists, there does not exist a symmetric SSPE at δ = 1. 9 The symmetric SSPE correspondence lacks upper semi-continuity at δ = 1 and this property is new to the bargaining literature. The intuitive reason for non-existence is that the limit SSPE specifies β = 0, so the players end up in perpetual disagreement, which fails as SSPE behavior. Paradoxically, the limit value of the SSPE utilities is given by v = v + = v = 1, which does not capture the situation of perpetual disagreement with v = v + = v = 0 in the limit. These limit results raise the issue what causes this paradox. The answer lies in reexamining the relation between 1) and the standard recursive equations that determine v, v +, and v in the symmetric SSPE for δ = 1. For δ < 1, these equations in our example are given by v = α [ 2β + 1 β )δv ] + 1 α) [ β β + )δv +], v + = α [ β + 1 β )δv ] + 1 α)1 β + )δv, 2) v = α1 β )δv α) [ 2β β + )δv ]. At δ = 1 the system 2) does not have full rank and yields v = v + = v as its solution. Clearly, v = v + = v = 0 is both feasible and consistent with 1), but the limit of SSPE utilities is equal to v = v + = v = 1. Our general results provide the following insights: First, the limit of SSPEs always exists. Second, we derive the following necessary and suffi cient condition for a unique solution of the recursive equations at δ = 1 to be consistent with 1): The stationary strategies α, β, β + ) should be such that with probability one the negotiations end within finite expected time. Third, the limit of SSPEs is an SSPE at δ = 1 if and only if the limit strategy profile satisfies the necessary and suffi cient condition. Otherwise, which is exactly the case in this example, the limit of SSPEs fails to be an SSPE itself. 9 Proposition 3 extends the non-existence results for pure strategies in?) at δ = 1 and?) at δ = 0.99 to all δ 1 2, 1]. Proposition 3 states existence of SSPE in mixed strategies for this range of δ s. 18

20 4 Expected utilities and recursive equations The motivating examples show that the solution of the recursive equations might fail to coincide with expected utilities that are consistent with stationary strategy profiles. In this section, we establish the necessary and suffi cient conditions such that both the recursive equations have a unique solution and that this solution is equivalent to the vector of expected utilities that are consistent. 4.1 Stationary strategy profiles and Markov processes We start with the observation that every stationary strategy profile α, β) induces a Markov process on N A {q}, where states in N indicate the recognized player in the current round, states in A which agreement has been reached before the current round, and the state q that the negotiation process has broken down in the past. The states in N are loosely speaking transient states and the states in A {q} are absorbing states, which is the translation that the negotiations are concluded once agreement is reached or the negotiations break down. Stationary Markov processes are defined by their state space and their stationary transition probabilities. The transition probabilities determine the expected utilities conditional on the state. In this subsection, we derive these probabilities and these expected utilities. Recall that round t N consists of n+2 stages in the multi-stage game representation of the negotiation model. The initial history at round t is h t,1 H t,1, and it specifies the state i t N A {q}. Then, any stationary strategy profile σ S = α, β) leads to a probability distribution on the state i t+1 N A {q} reached in round t + 1, and this induces a stationary Markov process on N A {q} in discrete time t N. We will express the transition probabilities as functions of α, β) and the recognition probabilities ρ i i, x, j). Given state i t = i and proposal x t = x = a, o) X i, the probability of accepting a is defined as π { i, x a; σ S} = j Co) βj i, x, k j o) ). 3) 19

21 This probability states the probability of reaching absorbing state a from the current state i within one round. Similarly, given state i t = i and proposal x t = x = a, o) X i, the probability that player j Co) is the first voter against is given by π { i, x, j; σ S} = 1 β j i, x, k j o) )) j Co):k j o)<k j o) βj i, x, k j o)). 4) This probability determines the following two probabilities. The probability that player i N is recognized in the next round is defined as π { i, x i ; σ S} = δ π { i, x, j; σ S} ρ i i, x, j) 5) j Co) and the probability of breakdown at round t is π { i, x q; σ S} = 1 δ) π { i, x, j; σ S}. 6) j Co) Note that all probabilities π { i t, x i t+1 ; σ S} are continuous in α, β) and δ. The definitions imply the following result, which is stated without further proof. Lemma 4 The stationary strategy profile σ S = α, β) induces the stationary Markov process on N A {q} with transition probabilities Λ σ S, δ ) P N σ S, δ ) P A σ S, δ ) P q σ S, δ ) = 0 I 0, 7) where P N σ S, δ ) assigns transition probabilities from N to N, P A σ S, δ ) from N to A, and P q σ S, δ ) from N to {q}. For i, i N and a A, the associated elements are given by p N ii σ S, δ ) = x X i α i x) π { i, x i ; σ S}, 8) p A ia p q iq σ S, δ ) = { α i x) π } i, x a; σ S, 9) x=a,o) X i σ S, δ ) = { α i x) π } i, x q; σ S. 10) x X i Moreover, all probabilities in Λ σ S, δ ) are continuous in α, β) and δ. 20

22 Next, we derive the expected utilities associated with Markov process Λ σ S, δ ), where the proof is deferred to the Appendix. Lemma 5 For σ S = α, β) and the initial probability distribution ρ, the probability that player i N is recognized at round t N is the i-th element of ρ P N σ S, δ ) t, the probability that alternative a l A is approved in round t is the l-th element of ρ P N σ S, δ ) t 1 P A σ S, δ ) and the probability of breakdown q at round t is ρ P N σ S, δ ) t 1 P q σ S, δ ). Furthermore, the cumulative probability π a l, t; σ S) that alternative a l A is approved on or before round t is the l-th element of ρ [ t τ=1 P N σ S, δ ) τ 1 ] P A σ S, δ ). Moreover, all these probabilities are continuous in α, β) and δ. For finite t N, all probabilities encountered thus far are continuous in the stationary strategy profile σ S and the discount factor δ. This continuity will be lost in the limit as t goes to infinity. Taking this limit is part of our next topic, because it is required in defining conditional expected utilities. In Section 2 we argued that the non-decreasing cumulative probability π a, t; σ S) [0, 1] converges to a well-defined limit π a; σ S) as t goes to. Since the convergence is independent of the initial distribution ρ, it immediately follows that every row of) the matrix [ t τ=1 P N σ S, δ ) τ 1 ] P A σ S, δ ) also converges as t goes to. This matrix is discontinuous in the stationary strategy profile σ S in case δ = 1, as can be verified in Example 1 for all stationary strategy profiles inducing perpetual disagreement. Consequently, the conditional expected utilities that are consistent with such strategy profile are discontinuous as well. This discontinuity arises due to the lack of continuity at infinity that bargaining models with costless delay have. Let v j i; σ S, δ ), i, j N, denote the conditional expected utility for player j that is consistent with Markov process Λ σ S, δ ) when player i is recognized in the current round of the negotiations. By definition, v j i; σ S, δ ) is a convex combination of the utilities associated with the absorbing states u j a), a A, and u j q). So, v j i; σ S, δ ) [u j, ū j ]. 21

23 Summing over t N the expected utility of reaching agreement at round t, we obtain the following result. Lemma 6 For σ S = α, β) and i, j N, v j i; σ S, δ ) is the i-th element of v j σ S, δ ) [ = P N σ S, δ ) ] τ 1 P A σ S, δ ) u j, 11) τ=1 where u j = u j a 1 ),..., u j a m )). Note that ρ v j σ S, δ ) = m l=1 π a l ; σ S) P A l σ S, δ ) u j l = U σ S), where P A l σ S, δ ) denotes the l-th element of P A σ S, δ ), is consistent with 1). 4.2 Necessary and suffi cient conditions Section 4.1 implies that every stationary strategy profile induces a well-defined Markov process. In this subsection, we investigate the necessary and suffi cient conditions on this Markov process for which the conditional expected utilities that are consistent with the stationary strategy profile are equivalent to the unique solution of the recursive equations. Our key results rely on standard results for non-negative matrices with row sums that are at most equal to one, as in e.g.?). The first result establishes that the conditional expected utilities of 11) satisfy the standard recursive equations. It follows from rewriting 11) recursively, which we omit. Proposition 7 Let σ S = α, β). For j N, v j σ S, δ ) in 10) solves the recursive equation v j σ S, δ ) = P N σ S, δ ) v j σ S, δ ) + P A σ S, δ ) u j. The right-hand side is a continuous function in σ S = α, β), δ and v j σ S, δ ). The recursive equations admit v j σ S, δ ) = [ I P N σ S, δ )] 1 P A σ S, δ ) u j as their unique solution if and only if the matrix I P N σ S, δ ) is non-singular. Otherwise, I P N σ S, δ ) is singular. Singularity is equivalent to a determinant equal to zero, and this 22

24 implies that P N σ S, δ ) has at least one real eigenvalue equal to 1. Consequently, a nonsingular matrix I P N σ S, δ ) must be equivalent to the matrix P N σ S, δ ) having real eigenvalues unequal to 1. By the theory of Markov chains, Λ σ S, δ ) admits 1 as an eigenvalue and all the absolute values of eigenvalues are bounded from above by 1. Moreover, because the matrix Λ σ S, δ ) is upper triangular in terms of the matrices that define it, its eigenvalues are either equal to the eigenvalues of P N σ S, δ ) or 1. So, I P N σ S, δ ) is non-singular is equivalent to the statement that all absolute values of eigenvalues of P N σ S, δ ) are less than 1. But then also, lim t P N σ S, δ ) t = 0. The latter convergence result means that within finite expected time the Markov process leaves the states in N for sure and, hence, reaches within finite expected time one of the absorbing states in A {q} for sure. So, the recursive equations have a unique solution if and only if the negotiation process ends within finite expected time for sure, either with an agreement reached or break down. Proposition 7 implies that the recursive equations are a necessary condition that follow from stationary strategy profiles. The singularity of the recursive system implies that a stronger suffi cient condition is needed to establish equivalence. Our discussion thus far hints at that the suffi cient condition should be related to a condition that ensures that all absolute values of eigenvalues of P N σ S, δ ) are less than one. This condition is related to indecomposable matrices and their finest decomposition. An n n matrix M 0 is indecomposable if there does not exist an n n permutation matrix Π such that [ ] ΠMΠ 1 M 11 M 12 =, 0 M 22 where M 11 and M 22 are square, see e.g.?). 10 If a matrix is decomposable, then an appropriate permutation matrix exists from which the upper-triangular block form can be obtained. In many cases it is possible to further decompose M 11 or M 22. The finest decomposition of M consists of a upper-triangular block form whose diagonal blocks are indecomposable. The finest decomposition exists and is unique, see e.g.?). For P N σ S, δ ), 10?) calls indecomposable matrices irreducible. 23

25 we define the finest decomposition into h 2 blocks, 1 h n, as ˆP 11 σ S, δ ) ˆP12 σ S, δ ) ˆP1h σ S, δ ) P N σ S, δ ) 0 ˆP22 σ = S, δ ) ˆP2h σ S, δ ) ˆPhh σ S, δ ), where ˆP ed σ S, δ ) 0, d, e = 1,..., h and d e, denotes the e, d)-th block or matrix in this decomposition and all diagonal blocks ˆP ee are indecomposable square matrices. Denote all states associated with ˆP ee σ S, δ ) as N e σ S, δ ) N. The following result characterizes the suffi cient conditions. Proposition 8 Let σ S = α, β). The expected utilities v j σ S, δ ) that are consistent with σ S = α, β) coincide with the unique solution of the recursive equations if and only if each indecomposable block ˆP ee σ S, δ ), e = 1,..., h, has at least one row sum less than 1. Proof. By definition, P N σ S, δ ) 0 is a non-negative matrix with row sums of at most 1. So, many standard results as in e.g.?) or the references therein) directly apply. Let λ M) denote the largest absolute value of eigenvalues of the square matrix M 0 whose row sums are at most 1. Then, we have 1. λ P N σ S, δ )) [0, 1] is a real eigenvalue of P N σ S, δ ), 2. the sum t=0 P N σ S, δ ) t converges if and only if λ P N σ S, δ )) < 1, 3. if λ P N σ S, δ )) < 1, then the inverse [ I P N σ S, δ )] 1 = t=0 P N σ S, δ ) t exists and is non-negative, For our negotiation model, 3 implies v j σ S, δ ) [ = P N σ S, δ ) ] τ 1 P A σ S, δ ) u j = [ I P N σ S, δ )] 1 P A σ S, δ ) u j, τ=1 and then the following result is immediate: A. If λ P N σ S, δ )) < 1, then the expected utilities v j σ S, δ ) that are consistent with σ S = α, β) coincide with the unique solution of the recursive equations. 24