Two Player Non Zero-sum Stopping Games in Discrete Time

Transcription

1 Two Payer Non Zero-sum Stopping Games in Discrete Time Eran Shmaya and Eion Soan May 15, 2002 Abstract We prove that every two payer non zero-sum stopping game in discrete time admits an ɛ-equiibrium in randomized strategies, for every ɛ > 0. We use a stochastic variation of Ramsey Theorem, which enabes us to reduce the probem to that of studying properties of ɛ-equiibria in a simpe cass of stochastic games with finite state space. Keywords: Stopping games, Dynkin games, stochastic games. MSC 2000 subject cassification: primary - 60G40, secondary - 91A15, 91A05. The Schoo of Mathematica Sciences, Te Aviv University, Te Aviv 69978, Israe. e-mai: gawain@post.tau.ac.i Keogg Schoo of Management, Northwestern University, and the Schoo of Mathematica Sciences, Te Aviv University, Te Aviv 69978, Israe. e-mai: eions@post.tau.ac.i The research of the second author was supported by the Israe Science Foundation (grant No ). 1

2 1 Introduction Consider the foowing optimization probem, that was presented by Dynkin (1969). Two payers observe a reaization of two rea-vaued processes (x n ) and (r n ). Payer 1 can stop whenever x n 0, and payer 2 can stop whenever x n < 0. At the first stage θ in which one of the payers stops, payer 2 pays payer 1 the amount r θ, and the process terminates. If no payer ever stops, payer 2 does not pay anything. A strategy of payer 1 is a stopping time µ that satisfies {µ = n} {x n 0}. A strategy ν of payer 2 is defined anaogousy. The termination stage is simpy θ = min{µ, ν}. For a given pair (µ, ν) of strategies, denote by γ(µ, ν) = E[1 {θ<+ } r θ ] the expected payoff of payer 1. Dynkin (1969) proved that if sup n 0 r n L 1 this probem has a vaue v; that is v = sup inf µ ν γ(µ, ν) = inf ν sup γ(µ, ν). µ He moreover characterized ɛ-optima strategies; that is, strategies µ (resp. ν) that achieve the supremum (resp. the infimum) up to ɛ. Neveu (1975) generaized this probem by aowing both payers to stop at every stage, and by introducing 3 rea vaued processes (r {1},n ),(r {2},n ) and (r {1,2},n ). The payoff payer 2 pays payer 1 is defined by γ(µ, ν) = E[1 {µ<ν} r {1},µ + 1 {µ>ν} r {2},ν + 1 {µ=ν<+ } r {1,2},µ ]. He then proved that this probem has a vaue, provided (a) sup n 0 max{ r {1},n, r {2},n, r {1,2},n } L 1, and (b) r {1},n = r {1,2},n r {2},n. Recenty Rosenberg et a (2001) studied games in Neveu s setup, but they aowed the payers to use randomized stopping times; a strategy is a [0, 1]-vaued process, that dictates the probabiity by which the payer stops at every stage. They prove that the probem has a vaue, assuming ony condition (a). A broad iterature provides sufficient conditions for the existence of the vaue in continuous time (see, e.g., Bismuth (1979), Aario-Nazaret, Lepetier and Marcha (1982), Lepetier and Maingueneau (1984), Touzi and Vieie (2002)). Some authors work in the diffusion case, see e.g. Cvitanić and Karatzas (1996). 2

3 The non zero-sum probem in discrete time was studied, amongst others, by Mamer (1987), Morimoto (1986) and Ohtsubo (1987, 1991). In the non zero-sum case, the processes (r {1},n ),(r {2},n ) and (r {1,2},n ) are R 2 -vaued, and the expected payoff of payer i, i = 1, 2, is γ i (µ, ν) = E µ,ν [1 {µ<ν} r i {1},µ + 1 {µ>ν} r i {2},ν + 1 {µ=ν<+ } r i {1,2},µ]. The goa of each payer is to maximize his own expected payoff. Given ɛ > 0, a pair of stopping times (µ, ν) is an ɛ-equiibrium if for every pair of stopping times (µ, ν ), γ 1 (µ, ν) γ 1 (µ, ν) ɛ, and γ 2 (µ, ν) γ 2 (µ, ν ) ɛ. The above mentioned authors provide various sufficient conditions under which ɛ-equiibria exist. In the present paper we study two payer non zero-sum probems in discrete time with randomized stopping times, and we prove the existence of an ɛ-equiibrium for every ɛ > 0, under condition (a). Our technique is based on a stochastic variation of Ramsey Theorem (Ramsey (1930)), which states that for every cooring of a compete infinite graph by finitey many coors there is a compete infinite monochromatic subgraph. This variation serves as a substitute for a fixed point argument, which is usuay used to prove existence of an equiibrium. It aows us to reduce the probem of existence of ɛ-equiibrium in a genera stopping game to that of studying properties of ɛ-equiibria in a simpe cass of stochastic games with finite state space. The paper is arranged as foows. In section 2 we provide the mode and the main resut. A sketch of the proof appears in section 3. In section 4 we present a stochastic variation of Ramsey Theorem. In section 5 we show that to prove existence of ɛ-equiibria in a genera stopping game, it is sufficient to consider a restricted cass of stopping games. In section 6 we define the notion of games payed on a finite tree, and we study some of their properties. In section 7 we construct an ɛ-equiibrium. We end by discussing extensions to more than two payers in section 8. 2 The Mode and the Main Resut A two-payer non zero-sum stopping game is a 5-tupe Γ = (Ω, A, p, F, R) where: 3

4 (Ω, A, p) is a probabiity space. F = (F n ) n 0 is a fitration over (Ω, A, p). R = (R n ) n 0 is a F-adapted R 6 -vaued process. The coordinates of R n are denoted by RQ,n i, i = 1, 2, φ Q {1, 2}. A (behavior) strategy for payer 1 (resp. payer 2) is a [0, 1]-vaued F- adapted process x = (x n ) n 0 (resp. y = (y n ) n 0 ). The interpretation is that x n (resp. y n ) is the probabiity by which payer 1 (resp. payer 2) stops at stage n. Let θ be the first stage, possiby infinite, in which at east one of the payers stops, and et φ Q {1, 2} be the set of payers that stop at stage θ (provided θ <.) The expected payoff under (x, y) is given by γ i (x, y) = E x,y [R i Q,θ1 {θ< } ], (1) where the expectation E x,y is w.r.t the distribution P x,y over pays induced by (x, y). Definition 2.1. Let Γ = (Ω, A, p, F, R) be a non zero-sum stopping game, and et ɛ > 0. A pair of strategies (x, y ) is an ɛ-equiibrium if γ 1 (x, y ) γ 1 (x, y ) + ɛ and γ 2 (x, y) γ 2 (x, y ) + ɛ, for every x and y. The main resut of the paper is the foowing. Theorem 2.2. Let Γ = (Ω, A, p, F, R) be a two-payer stopping game such that sup n 0 R n L 1 (p). Then for every ɛ > 0, the game admits an ɛ-equiibrium. 3 Sketch of the Proof In the present section we provide the main ideas of the proof. Let Γ be a stopping game. To simpify the presentation, assume that F n is trivia for every n, so that the payoff process is deterministic. Assume aso that payoffs are uniformy bounded by 1. Given ɛ > 0, fix a finite covering M of the space of payoffs [ 1, 1] 2 by sets with diameter smaer than ɛ. For every two non negative integers k < define the periodic game G(k, ) as the game that starts at stage k, and, if not stopped earier, restarts at 4

5 stage. Formay, G(k, ) is a stopping game in which the termina payoff at stage n is equa to the payoff at stage k + (n mod ) in Γ. This periodic game is a simpe stochastic game (see, e.g., Shapey (1953), or Fesch et a (1996)), and is known to admit an ɛ-equiibrium in periodic strategies. Assign to each pair of non negative integers k < an eement m(k, ) M which contains a periodic ɛ-equiibrium payoff of the periodic game G(k, ). Thus, we assigned to each k < a coor m(k, ) M. A consequence of Ramsey Theorem is that there is an increasing sequence of integers 0 k 1 < k 2 < such that m(k 1, k 2 ) = m(k n, k n+1 ) for every n. Assume first that k 1 = 0. A naive candidate for a 4ɛ-equiibrium suggests itsef: between stages k n and k n+1, the payers foow a periodic ɛ-equiibrium in the game G(k n, k n+1 ) with corresponding payoff in the set m(k 1, k 2 ). So that this strategy pair is indeed 4ɛ-equiibrium, one has to study properties of the ɛ-equiibria in periodic games. The compete soution of this case appears in Shmaya et a (2002), where it is observed that in each periodic game G(k, ) there exists a periodic ɛ-equiibrium that satisfies one of the foowing conditions. (i) In this ɛ-equiibrium, no payer ever stops. (ii) Under this ɛ-equiibrium, both payers receive non negative payoff, and termination occurs in each period with probabiity at east ɛ 2. (iii) If some payer receives in this ɛ-equiibrium a negative payoff, then his opponent stops in each period with probabiity at east ɛ 2. The fact that at east one of these conditions hod is sufficient to prove that the concatenation described above is a 4ɛ-equiibrium, with corresponding payoff in the convex hu of m(k 1, k 2 ). If k 1 > 0, choose an arbitrary m m(k 1, k 2 ). Between stages 0 and k 1, the payers foow an equiibrium in the k 1 -stage game with termina payoff m; that is, if no payer ever stops before stage k 1, the payoff is m. From stage k 1 and on, the payers foow the strategy described above. It is easy to verify that this strategy pair forms a 5ɛ-equiibrium. When the payoff process is genera, few difficuties appear. First, a periodic game is defined now by two stopping times µ 1 < µ 2 ; µ 1 indicates the initia stage, and µ 2 indicates when the game restarts. So that we can anayze this periodic game, we have to reduce the probem to the case where the σ-agebras F µ1, F µ1 +1,..., F µ2 are finite. This is done in section 7. Second, we have to generaize Ramsey Theorem to this stochastic setup. This is done in section 4. Third, we have to study properties of ɛ-equiibria in these periodic games, so that a proper concatenation of ɛ-equiibria in the different periodic games 5

6 woud generate a 4ɛ-equiibrium in the origina game. This is done in section 6. 4 A Stochastic Variation of Ramsey Theorem In the present section we provide a stochastic variation of Ramsey Theorem. Let (Ω, A, p) be a probabiity space and F = (F n ) n 0 a fitration. A stopping times that appear in the seque are F-adapted. For every set A Ω, A c = Ω \ A is the compement of A. For every A, B A, A hods on B if and ony if p(a c B) = 0. Definition 4.1. A NT-function is a function that assigns to every integer n 0 and every bounded stopping time τ a F n -measurabe r.v. that is defined over the set {τ > n}. We say that a NT-function f is C-vaued, for some set C, if the r.v. f n,τ is C-vaued, for every n 0 and every τ. Definition 4.2. A NT-function f is F-consistent if for every n 0, every F n -measurabe set F, and every two bounded stopping times τ 1, τ 2, we have τ 1 = τ 2 > n on F impies f n,τ1 = f n,τ2 on F. When f is a NT-function, and σ < τ are two bounded stopping times, we denote f σ,τ (ω) = f σ(ω),τ (ω). Thus f σ,τ is a F σ -measurabe r.v. The main resut of this section is the foowing. Theorem 4.3. For every finite set C of coors, every C-vaued F-consistent NT-function c, and every ɛ > 0, there exists a sequence of bounded stopping times 0 θ 0 < θ 1 < θ 2 <... such that p(c θ0,θ 1 = c θ1,θ 2 = c θ2,θ 3 =...) > 1 ɛ. Comment: The natura stochastic generaization of Ramsey Theorem requires the stronger condition p(c θ0,θ 1 = c θi,θ j 0 i < j) 1 ɛ. We do not know whether this generaization is correct. The foowing exampe shows that a sequence of stopping times θ 0 < θ 1 < θ 2 < θ 3 <... such that p(c θ0,θ 1 = c θ1,θ 2 =...) = 1 need not exist even without the boundedness condition. Exampe 4.4. Let X n be a biased random wak on the integers, X 0 = 0 and p(x n+1 = X n + 1) = 1 p(x n+1 = X n 1) = 3/4. Let F n = σ(x 0, X 1,..., X n ). For every n 0 et R n = 1 k n {X k = 1}; for every 6

7 finite (but not necessariy bounded) stopping time τ define c n,τ = Red on R n {τ > n} and c n,τ = Bue on Rn {τ c > n}. Since p( n 0 R n ) < 1, whereas for every finite stopping time θ and every B F θ p( n 0 R n B) > 0, it foows that for every sequence θ 0 < θ 1 <... of finite stopping times p(c θ0,θ 1 = Bue) > 0 whereas p(c θ0,θ 1 = c θ1,θ 2 =... = Bue c θ0,θ 1 = Bue) < 1. We start by proving a sighty stronger version of Theorem 4.3 when C = 2. Lemma 4.5. Let C = {Bue, Red}, and et c be a C-vaued F-consistent NT-function. For every ɛ > 0 there exist N N, two sets R, B F N, and a sequence N τ 0 < τ 1 < τ 2 <... of bounded stopping times, such that: a) R = B c. b) p(c τ0,τ 1 = c τ1,τ 2 =... = Red R) > 1 ɛ. c) p(c τk,τ = Bue k, B) > 1 ɛ. Proof. We caim first that for every n N one can find two sets R n, B n F n and a bounded stopping time σ n such that: 1. p(r n B n ) > n. 2. {σ n > n} on R n and c n,σn = Red on R n. 3. For every bounded stopping time τ, c n,τ = Bue on B n {τ > n}. To see this, fix n N. Ca a set F F n red if there exists a bounded stopping time σ F such that on F both σ F > n and c n,σf = Red. Observe that since c is F-consistent, if F, G F n are red, then so is F G. Let α = sup F {p(f ), F F n is red}. For every k 1 et F k F n be a red set such that p(f k ) > α 1 k. Let F = k 1 F k. Observe that F F n and p(f ) = α. Moreover, no subset of F c with positive probabiity is red. Let R n = F 2 n, et σ n be a bounded stopping time such that on R n σ n > n and c n,σn = Red, and et B n = F c. This concudes the proof of the caim. Let B = {B n i.o.}, and set R = B c. Since R, B n F n, there is N N and two sets B, R F N such that (i) R = B c, (ii) p(b B) > 1 ɛ, and (iii) p(r R) > 1 ɛ. On R, and therefore aso on R R, both B n and 7

8 (B n R n ) c occur ony finitey many times. By sufficienty increasing N we assume w..o.g. that p( n N R n R R) > 1 ɛ. In particuar, p( n N R n R) > 1 2ɛ. (2) Let N = n 0 < n 1 < n 2 < be a sequence of integers such that, for every k 0, p(t k B B) > 1 ɛ, where T 2 k+1 k = nk n<n i+k B n. Then p( k 0 T k B B) > 1 ɛ, and therefore p( k 0 T k B) > 1 2ɛ. (3) We now define the sequence (τ k ) k 0 inductivey, working separatey on R and B. Consider first the set R. Define τ 0 = N. Given τ k, define τ k+1 = Σ n N σ n 1 {τk =n} R n R on R n ({τ k = n} R n ). Since τ k and (σ n ) n 0 are bounded, τ k+1 can be extended to a bounded stopping time on R. By definition c τ0,τ 1 = c τ1,τ 2 =... = Red on R ( n N R n ), and it foows from (2) that p(c τ0,τ 1 = c τ1,τ 2 =... = Red R) 1 2ɛ. Consider now the set B. Define τ 0 = N. Define τ k+1 (w) = min{n k n < n k+1, w B n } on B T k, and τ k+1 = n k+1 1 on B \ T k. By (c), for every k, N, c τk,τ = Bue on B ( k 0 T k ), and it foows from (3) that p(c τk,τ = Bue k, B) > 1 2ɛ. Proof of Theorem 4.3 We prove the Theorem by induction on C. The case C = 2 foows from Lemma 4.5. Assume we have aready proven the emma whenever C = r and assume C = r + 1. Let Red be a coor in C. By considering a coors different from Red as a singe coor, and appying Lemma 4.5 there exist N N, two sets R, B F N, and a sequence of stopping times N τ 0 < τ 1 <... such that: (i) R = B c, (ii) p(c τ0,τ 1 = c τ1,τ 2 =... = Red R) > 1 ɛ/2, and (iii) p(c τk,τ Red k, B) > 1 ɛ/2. We define θ i separatey on R and B. On R, we et θ i = τ i. We now restrict ourseves to the space ( B, A B, p B) with the fitration G n = F τn B. Let c be the C-vaued NT function over G defined by c n,β = c τn,τβ for every G-adapted stopping time β, where τ β = n τ n1 {β=n} is an F-adapted stopping time. Let c be the cooring that is obtained from c by changing the coor Red with another coor in C, say Green: { c cn,β, if c n,β = n,β Red Green, if c n,β = Red. 8

9 As c is a C \ {Red}-vaued G-consistent NT-function, one can appy the induction hypothesis, and obtain a sequence of G-adapted stopping times 0 β 0 < β 1 < β 2 <... such that p(c β 0,β 1 = c β 1,β 2 =... B) > 1 ɛ/2. (4) By (4) and (iii) it foows that p( c β0,β 1 = c β1,β 2 =... B) > 1 ɛ. We define θ i = τ βi on B. Thus p(c θ0,θ 1 = c θ1,θ 2 =... B) > 1 ɛ. (5) Combining (ii) and (5) we get p(c θ0,θ 1 = c θ1,θ 2 =...) > 1 ɛ, as desired. 5 Restricting the Cass of Games In the present section we show that to prove Theorem 2.2 it is sufficient to consider a restricted cass of stopping games. Definition 5.1. Let Γ = (Ω, A, p, F, R) be a stopping game and B A with p(b) > 0. The game restricted to B is the stopping game Γ B = (B, A B, p B, F B, R), where A B = {A B, B A}, p B (A) = p(a B) for every A A B, and F B,n = {F B, F F n } for every n 0. The foowing emma is standard. Lemma 5.2. Let (Ω, A) be a measurabe space and et B Ω. Let A B = {A B, A A}. Then for every A B -measurabe function x on B there exists a A-measurabe function x on Ω such that x = x on B. Set m = sup n 0 max{ R i Q,n, i = 1, 2, Q {1, 2}}). Since m L1 (p), for every ɛ > 0 there exists N N such that E[m1 {m>n} ] < ɛ. (6) Set B = {m N}. By Lemma 5.2 and (6), any ɛ-equiibrium in Γ B can be extended to a 3ɛ-equiibrium in Γ. In particuar, it is sufficient to consider games in which the payoff process is uniformy bounded. We further assume w..o.g. that the payoff process is uniformy bounded by 1. Lemma 5.2 gives us the foowing. 9

10 Lemma 5.3. Let Γ = (Ω, A, p, F, R) be a stopping game with payoffs bounded by 1, et B A such that p(b) > 1 δ, and et ɛ > 0. Assume that the game Γ B admits an ɛ-equiibrium. Then the game Γ admits an ɛ + 2δ-equiibrium. Definition 5.4. Let Γ = (Ω, A, p, F, R) be a stopping game, and et τ be a bounded F-adapted stopping time. The game that starts at τ is defined by Γ τ = (Ω, A, p, F, R), where F n = F τ+n and R n = R τ+n for every n 0. In particuar for every bounded F-adapted stopping time τ, and every B A, Γ B,τ is the game restricted to B that starts at τ. Lemma 5.5. Let Γ = (Ω, A, p, F, R) be a stopping game, τ a bounded F- adapted stopping time, and ɛ > 0. Let (B 1,..., B k ) be a finite F τ -measurabe partition of Ω. Suppose that the games Γ Bi,τ, 1 i k, admit ɛ-equiibria. Then the game Γ admits an ɛ-equiibrium. Proof. Let (x i, y i ) be an ɛ-equiibrium in Γ Bi,τ, 1 i k. Consider the strategy profie (x, y) for the game Γ τ defined by x = x i and y = y i on B i. Then (x, y) is an ɛ-equiibrium in Γ τ. For i = 1, 2, et γ i = E x,y [RQ,θ i 1 {θ< } F τ ] be the payoff to payer i in Γ τ conditioned on F τ. By standard toos in dynamic programming, the finite stage game which, if no payer stops before stage τ, terminates at stage τ with termina payoff γ = (γ 1, γ 2 ), has an equiibrium ( x, ȳ). Foowing ( x, ȳ) up to stage τ and (x, y) afterwards forms an ɛ-equiibrium in Γ. The main resut of this section is the foowing. Proposition 5.6. Suppose that every stopping game Γ = (Ω, A, p, F, R) that satisfies A.1-A.6 beow admits an ɛ-equiibrium, for every ɛ > 0. Then Theorem 2.2 hods. A.1: There exists K N such that for every n 0, R n {0, ± 1 K, ± 2 K,..., ± K K }6. A.2: R 1 := im sup n R 1 {1},n is constant, and R1 {1},n R1 for every n 0. A.3: R 2 := im sup n R 2 {2},n is constant, and R2 {2},n R2 for every n 0. A.4: R 1 > 0 or R 2 > 0. A.5: R 2 {1},n < R2 whenever R 1 {1},n = R1. A.6: R 1 {2},n < R1 whenever R 2 {2},n = R2. 10

11 Proof. Assume that every stopping game Γ = (Ω, A, p, F, R) that satisfies A.1-A.6 admits an ɛ-equiibrium, for every ɛ > 0. Let Γ = (Ω, A, p, F, R) be a stopping game with payoffs uniformy bounded by 1, and et ɛ > 0. We prove that Γ admits a Cɛ-equiibrium for some C > 0 by graduay reducing Γ to a game that satisfies A.1-A.6 If (x, y) is an ɛ-equiibrium in some stopping game Γ = (Ω, A, p, F, R), it is a 3ɛ-equiibrium in every stopping game Γ = (Ω, A, p, F, R ) such that R n R n < ɛ for every n 0. By choosing K > 1/ɛ, one can therefore assume A.1. Let R 1 = im sup R{1},n 1. R1 is a n 0 F n-measurabe r.v. that by A.1 admits finitey many vaues. Let N be arge enough, and et { B t } t {0,± 1 K,± 2 K,...,± K K } be a F N -measurabe partition of Ω such that p(r 1 = t B t ) > 1 ɛ. 1 By sufficienty increasing N we assume w..o.g that, for each t {0, ± 1, ± 2,..., ± K }, K K K p( n N {R1 {1},n t} {R1 = t} B t ) > 1 ɛ. Then p(b t Bt ) > 1 2ɛ, where B t = B t {R 1 = t} {R{1},n 1 t}. Let B = t B t. Then n N p(b) > 1 2ɛ. (7) Suppose that the games Γ Bt,N admit ɛ-equiibria for t {0, ± 1 K, ± 2 K,..., ± K K }. By Lemma 5.5 the game Γ B admits an ɛ-equiibrium. By (7) and Lemma 5.3 it foows that Γ admits a 5ɛ-equiibrium. Therefore it is sufficient to prove the existence of an ɛ-equiibrium in the games Γ Bt,N, so that one can assume that A.2 (and anaogousy A.3) hods Using simiar arguments we can assume that T 2 = im sup{r 2 {1},n R1 {1},n = R 1 } and T 1 = im sup{r 1 {2},n R2 {2},n = R2 } are constant. One can furthermore assume that R 2 {1},n T 2 whenever R 1 {1},n = R1, and that R 1 {2},n T 1 whenever R 2 {2},n = R2. We now show that if at east one of A.4-A.6 is not satisfied, the game admits an ɛ-equiibrium, for every ɛ > 0. If R 1 0 and R 2 0 then the strategy pair (x, y) that is defined by x n = y n = 0 (aways continue) is an equiibrium. We can thus assume that A.4 is satisfied. Assume w..o.g. that R 1 > 0. If T 2 R 2 then by A.4 the foowing strategy (x, y) is an ɛ-equiibrium: x n = ɛ 1 {R 1 {1},n =R 1 } and y n = 0. If T 1 R 1 and T 2 < R 2 then the 1 By convention p(φ φ) = 1. 11

12 foowing strategy (x, y) is an ɛ-equiibrium: y n = ɛ 1 {R 2 {2},n =R 2 }, x n = { } 0 n N ɛ 1 {R 1 {1},n =R 1 } n > N, where N is arge enough so that P 0,y (θ < N) > 1 ɛ, and 0 is the strategy that never stops. The remaining case is T 2 < R 2 and T 1 < R 1, so that A.5 and A.6 are satisfied. 6 Stopping Games on Finite Trees An important buiding bock in our anaysis are stopping games that are payed on a finite tree. In the present section we define these games and study some of their properties. 6.1 The Mode and the Main Resut Definition 6.1. A stopping game on a finite tree (or simpy a game on a tree) is a tupe T = (S, S 1, r, (C s, p s, R s ) s S\S1 ), where (S, S 1, r, (C s ) s S\S1 ) is a tree; S is a non empty finite set of nodes, S 1 S is a finite set of eaves, r S is the root, and for each s S \S 1, C s S \ {r} is the set of chidren of s. We denote by S 0 = S \ S 1 the set of nodes which are not eaves. For every s S, depth(s) is the depth of s - the ength of the path that connects the root to s. For every s S 0, p s is a probabiity distribution over C s. R s R 6 is the payoff at s. The coordinates of R s are denoted (R i Q,s ) i=1,2,φ Q {1,2}. Throughout this section we consider games on trees whose payoffs (R s ) s S0 satisfy the foowing conditions for every i = 1, 2, every Q {1, 2}, and every s S 0, (B.1) R i Q,s {0, ± 1 K,..., ± K K } for some K N, (B.2) Ri {i},s R i, where R 1, R 2 R, and at east one of them is positive, (B.3) R 2 {1},s < R2 whenever R 1 {1},s = R1, and (B.4) R 1 {2},s < R1 whenever R 2 {2},s = R2. Observe the simiarity between these conditions and conditions A.1-A.6. A stopping game on a finite tree starts at the root and is payed in stages. Given the current node s S 0, and the sequence of nodes that were aready 12

13 visited, both payers decide, simutaneousy and independenty, whether to stop or to continue. Let Q be the set of payers that decide to stop. If Q φ, the pay terminates, and the termina payoff to payer i is RQ,s i. If Q = φ, a new node s in C s is chosen according to p s. The process now repeats itsef, with s being the current node. If s S 1 the new current node is the root r. Thus, payers cannot stop at eaves. The game on the tree is essentiay payed in rounds. The round starts at the root, and ends once it reaches a eaf. The coection (p s ) s S0 of probabiity distributions induces a probabiity distribution over the set of eaves S 1, or, equivaenty, over the set of branches that connect the root to the eaves. For each set D S 0, we denote by p D the probabiity that the chosen branch passes through D. For each s S we denote by F s the event that the chosen branch passes through s. Definition 6.2. Let T = (S, S 1, r, (C s, p s, R s ) s S0 ) and T = (S, S 1, r, (C s, p s, R s) s S 0 ) be two games on trees. T is a subgame of T if (i) S S, (ii) r = r, and (iii) for every s S 0, C s = C s, p s = p s and R s = R s. In words, T is a subgame of T if one removes a the descendents (in the strict sense) of severa nodes from the tree (S, S 1, r, (C s ) s S0 ), and keep a other parameters fixed. Observe that this notion is different than the standard definition of a subgame in game theory. Let T = (S, S 1, r, (C s, p s, R s ) s S0 ) be a game on a tree. For each subset D S 0 we denote by T D the subgame of T generated by trimming T from D downward. Thus, a strict descendents of nodes in D are removed. For every subgame T of T, and every subgame T of T, et p T,T = p S 1 \S 1 be the probabiity that the chosen branch in T passes through a eaf of T stricty before it passes through a eaf of T. 2 Consider the first round of the game. Let t be the stopping stage. If no termination occurred in the first round t =. If t < et s be the node (of depth t) in which termination occurred, and et Q be the set of payers that stop at stage t. The r.v. r i = RQ,s i 1 {t< } is the payoff to payer i in the first round. A stationary strategy of payer 1 (resp. payer 2) is a function x : S 0 [0, 1] (resp. y : S 0 [0, 1]): x(s) is the probabiity that payer 1 stops at s. Denote by P x,y the distribution over pays induced by (x, y), and by E x,y the corresponding expectation operator. 2 Here, S 1 (resp. S 1 ) is the set of eaves of T (resp. T ). 13

14 For every pair of stationary strategies (x, y) we denote by π(x, y) = P x,y (t < ) the probabiity that under (x, y) the game terminates in the first round of the game; that is, the probabiity that the root is visited ony once aong the pay. We denote by ρ i (x, y) = E x,y [r i ], i = 1, 2, the expected payoff of payer i in a singe round. Finay, we set γ i (x, y) = ρ i (x, y)/π(x, y). 3 This is the expected payoff under (x, y). In particuar, π(x, y) γ i (x, y) = ρ i (x, y). (8) When we want to emphasize the dependency of these variabes on the game T, we wi write π(x, y; T ), ρ i (x, y; T ) and γ i (x, y; T ). Observe that for every pair of stationary strategies (x, y) π(x, 0) + π(0, y) π(x, y), (9) where 0 is the strategy that never stop; that is, 0(s) = 0 for every s. Definition 6.3. A pair of stationary strategies (x, y) is an ɛ-equiibrium of the game T if, for each pair of strategies (x, y ), γ 1 (x, y) γ 1 (x, y) + ɛ and γ 2 (x, y ) γ 2 (x, y) + ɛ. Comment: A stopping game on a finite tree T is equivaent to a recursive absorbing game, where each round of the game T corresponds to a singe stage of the recursive absorbing game. A recursive absorbing game is a stochastic game with a singe non absorbing state, in which the payoff in non absorbing states is 0. Fesch et a (1996) proved that every recursive absorbing game admits an ɛ-equiibrium in stationary strategies. This resut aso foows from the anaysis of Vrieze and Thuijsman (1989). However, there is no bound on the per-round probabiity of termination under this ɛ-equiibrium, and we need to bound this quantity. The main resut of this section is the foowing. Proposition 6.4. For every stopping game on a finite tree T, every ɛ > 0 sufficienty sma, and every a 1, a 2, b 1, b 2 that satisfy R i ɛ a i < b i for i {1, 2}, there exist a set D S 0 of nodes and a strategy pair (x, y) in T such that: 3 By convention, 0 0 = 0. 14

15 1. In every subgame T of T D there are no ɛ-equiibria in T with corresponding payoffs in [a 1, b 1 ] [a 2, b 2 ]. 2. Either D = φ (so that T D = T ), or the foowing three conditions hod: (a) a 1 ɛ γ 1 (x, y) and a 2 ɛ γ 2 (x, y). (b) For every pair (x, y ) of strategies, γ 1 (x, y) b 1 +7ɛ and γ 2 (x, y ) b 2 + 7ɛ. (c) π(x, y) ɛ 2 p D. Observe that 2(a) and 2(b) impy that if b i a i ɛ then (x, y) is a 9ɛequiibrium in T with corresponding payoffs in [a 1 ɛ, b 1 +7ɛ] [a 2 ɛ, b 2 +7ɛ]. 6.2 Union of Strategies Given n stationary strategies x 1, x 2,..., x n, we define their union x by x(s) = 1 Π 1 k n (1 x k (s)). The probabiity that the union strategy continues at each node is the probabiity that a of its components continue. We denote x = x 1 +x x n. Given n pairs of stationary strategies α k = (x k, y k ), 1 k n, we denote by α α n the stationary strategy pair (x, y) that is defined by x = x x n, y = y y n. Consider now n copies of the game that are payed simutaneousy, such that the choice of a new node is the same across the copies; that is, a copies that have not terminated at stage t are at the same node. Nevertheess, the otteries made by the payers concerning the decision whether to stop or not are independent. Let α k = (x k, y k ), 1 k n, be the stationary strategy pair used in copy k and et α = α α n. We consider the first round of the game. Let t k be the stopping stage in copy k, et s k be the node in which termination occurred, et Q k be the set of payers that stop at stage t k and et r i k = Ri Q k,s k 1 {tk < }. Then π k = π(x k, y k ) = P(t k < ) and ρ i k = ρi (x k, y k ) = E[r i k ]. Let t,r,ρ,π be the anaog quantities w.r.t. α: t = min{t k, 1 k n}, r i = R i Q,s 1 {t< }, where s = s k for which t k is minima, and Q = k s k =s Q k. Then ρ i = ρ i (x, y) = E[r i ] and π = π(x, y) = P(t < ). Let γ k = γ(x k, y k ) be the expected payoff under α k = (x k, y k ), and γ = γ(x, y) be the corresponding quantity under α. 15

16 The foowing emma foows from the independence of the pays given the branch. Lemma 6.5. Let s S 0 be a node of depth j. Then, for every 1 k, n, k, the event {t k j} and the random variabe t k 1 {tk j} are independent of t given F s. Lemma 6.6. Let N = n k=1 1 {t k < } be the number of copies that terminate in the first round. Then 1. n k=1 π k E[N1 {N 2} ] π n k=1 π k, and 2. n k=1 ρi k E[(N + 1)1 {N 2}] ρ i n k=1 ρi k + E[(N + 1)1 {N 2}] for each payer i {1, 2}. Proof. Observe that N N1 {N 2} = 1 {N=1} 1 {N 1} N = n k=1 1 {tk < }. The first resut foows by taking expectations. For the second resut, note that n rk i (N + 1)1 {N 2} r i k=1 n rk i + (N + 1)1 {N 2}. (10) k=1 Indeed, on {N 1} (10) hods with equaity, and on {N 2} the eft hand side is at most 1, whereas the right hand side is at east +1. The resut foows by taking expectations. 6.3 Heavy and Light Nodes Definition 6.7. Let σ = (x, y) be a pair of stationary strategies and et δ > 0. A node s S 0 is δ-heavy with respect to σ if P σ (t < F s ) δ; that is the probabiity of termination in the first round given that the chosen branch passes through s is at east δ. The node s is δ-ight w.r.t. σ if P σ (t < F s ) < δ. For a fixed δ, we denote by H δ (σ) the set of δ-heavy nodes w.r.t. σ. Two simpe impications of this definition are the foowing. 16

17 Fact 1 H δ (α 1 ) H δ (α 1 +α 2 ). Fact 2 H δ1 (σ) H δ2 (σ) whenever δ 1 δ 2. Lemma 6.8. Let ɛ > 0 be sufficienty sma, and et (x, y) be a stationary ɛ-equiibrium such that R i ɛ γ i (x, y), i = 1, 2. Then H ɛ (x, y) φ. In particuar, by Fact 2, H ɛ 2(x, y) φ. Comment: The proof hinges on the assumption that R{1},s 2 < R2 whenever R{1},s 1 = R1. As a counter exampe when this condition does not hod, take a game in which (a) RQ,s i = 1 for every i, Q and s, and (b) R1 = R 2 = 1. Then any stationary strategy pair which stops with positive probabiity is a 0-equiibrium. Proof. Assume w..o.g that π(x, 0) π(0, y). Let r be the probabiity that, under (x, 0) termination occurs at a node s in which R 1 {1},s < R1. Since payoffs are discrete, ρ 1 (x, 0) π(x, 0) ((1 r)r 1 + r(r 1 1 K )) = π(x, 0) (R1 r ). (11) K Assume to the contrary that H ɛ (x, y) = φ. Then, in particuar, H ɛ (0, y) = φ. It foows that the sequence {(0, y), (x, 0)} is ɛ-orthogona. By Lemma 6.14, (8), (11) and sinc (x, y) is an ɛ-equiibrium, (π(0, y) + π(x, 0)) γ 1 (x, y) It foows that ρ 1 (0, y) + ρ 1 (x, 0) + 6ɛ(π(0, y) + π(x, 0)) π(0, y) γ 1 (0, y) + π(x, 0) (R 1 r ) + 6ɛ(π(0, y) + π(x, 0)) K π(0, y) (γ 1 (x, y) + ɛ) + π(x, 0) (R 1 r ) + 6ɛ(π(0, y) + π(x, 0)). K π(x, 0) γ 1 (x, y) π(x, 0)(R 1 r ) + 7ɛ π(0, y) + 6ɛ π(x, 0). K Since R 1 ɛ γ 1 (x, y), π(x, 0) (R 1 ɛ) π(x, 0)(R 1 r ) + 7ɛ π(0, y) + 6ɛ π(x, 0), K 17

18 which impies π(x, 0)r 7ɛK (π(0, y) + π(x, 0)). (12) Since payoffs are discrete and bounded by 1, since R{1},s 2 < R2 whenever R{1},s 1 = R1, and by (12), ρ 2 (x, 0) π(x, 0) ((1 r)(r 2 1K ) ) + r 1 = π(x, 0) ((R 2 1K ) + r(1 R2 + 1K ) ) (13) π(x, 0)(R 2 1 K + 3r) π(x, 0) (R 2 1 ) + 21ɛK(π(0, y) + π(x, 0)). K By Lemma 6.14, since ρ 2 (0, y) π(0, y) R 2, (13), and since π(x, 0) π(0, y) (π(0, y) + π(x, 0))γ 2 (x, y) ρ 2 (0, y) + ρ 2 (x, 0) + 6ɛ(π(0, y) + π(x, 0)) π(0, y) R 2 + π(x, 0) (R 2 1 ) + (6ɛ + 21ɛK) (π(0, y) + π(x, 0)) K = (π(0, y) + π(x, 0)) (R 2 + 6ɛ + 21ɛK) π(x, 0) 1 K (π(0, y) + π(x, 0)) (R 2 + 6ɛ + 21ɛK 1 2K ). (14) Since ɛ is sufficienty sma, and R i ɛ γ i (x, y), it foows from A.4 that γ i (x, y) > 0 for i = 1 or i = 2. In particuar, it foows that π(x, y) > 0, so that by (9) π(x, 0) + π(0, y) π(x, y) > 0. It foows that, for sufficienty sma ɛ (e.g. ɛ < 1 2K(7+21K) ), a contradiction. γ 2 (x, y) R 2 + 6ɛ + 21ɛK 1 2K R2 ɛ, 6.4 Orthogona Strategies Definition 6.9. Let δ > 0. A sequence (α 1, α 2,..., α n ) of stationary strategy pairs is δ-orthogona if α k+1 (s) = (0, 0) for every 1 k n 1 and every node s H δ (α α k ); that is α k+1 continues on δ-heavy nodes of α α k. 18

19 Lemma Let δ > 0, et (α 1,..., α n ) be a δ-orthogona sequence of stationary strategy pairs, et k {1,..., n}, and et s S be a node of depth j. Then P({j t k < } ( <k {t < }) F s ) δ P(j t k < F s ). (15) Proof. Fix k {1,..., n}. We prove the emma by induction on the nodes of T, starting from the eaves and cimbing up to the root. Let s S 1 be a eaf of T. Since s is a eaf, P(j t k < ) = 0 and (15) is satisfied triviay. Assume now that s S 0. Then: P({j t k < } ( <k {t < }) F s ) = P({t k = j} ( <k {t < }) F s ) + s C s p s [s ] P({j + 1 t k < } ( <k {t < }) F s ). (16) By the induction hypothesis, for every chid s C s, P({j + 1 t k < } ( <k {t < }) F s ) δ P(j + 1 t k < F s ). (17) By Lemma 6.5 {t k = j} and <k {t < } are independent given F s. Therefore P({t k = j} ( <k {t < }) F s ) = P(t k = j F s ) P( <k {t < } F s ). If s is δ-ight w.r.t α α k 1 then P( <k {t < } F s ) < δ whie if s is δ-heavy then P(t k = j F s ) = 0 according to the definition of orthogonaity. In particuar, P({t k = j} ( <k {t < }) F s ) δ P(t k = j F s ). (18) Eqs. (16),(17) and (18) yied as desired. P({j t k < } ( <k {t < })) δ P(t k = j F s ) + δ s C s p s [s ] P(j + 1 t k < F s ) = δ P(j t k < F s ), 19

20 Appying Lemma 6.10 to the root we get: Coroary Let δ > 0, and et (α 1,..., α n ) be a δ-orthogona sequence of stationary strategy pairs. For every k {1,..., n}, P({t k < } ( <k {t < })) δ P({t k < }) = δπ k. Lemma Let δ > 0, et (α 1,..., α n ) be a δ-orthogona sequence of stationary strategy pairs, and et N = n k=1 1 {t k < }. Then E[(N + 1)1 {N 2} ] 3δ(π 1 + π π n ). Proof. Observe that N + 1 3(N 1) on {N 2}, and (N 1)1 {N 2} = n k=1 1 {t k < } ( <k{t < }). Therefore E[(N+1)1 {N 2} ] 3E[(N 1)1 {N 2} ] = 3 The resut foows by Coroary n P({t k < } ( <k {t < })). k=1 From Lemma 6.6 and Lemma 6.12 we get the foowing. Coroary Let δ > 0, and et (α 1,... α n ) be a δ-orthogona sequence of strategy pairs. Denote α = α α n. Then for i = 1, 2 1. (1 3δ) n k=1 π k π n k=1 π k. 2. n k=1 ρi k 3δ n k=1 π k ρ i n k=1 ρi k + 3δ n k=1 π k. Lemma Let δ > 0, and et (α 1,... α n ) be a δ-orthogona sequence of stationary strategy pairs. Denote α = α α n. Then for i = 1, 2 n n n n n ρ i k 6δ π k γ i π k ρ i k + 6δ π k. k=1 k=1 Proof. By Coroary 6.13 and (8) n ρ i k 3δ k=1 n π k ρ i = γ i π k=1 k=1 k=1 k=1 { γ i n k=1 π k, if γ i > 0 γ i (1 3δ) n k=1 π k, if 1 γ i 0. In both case, the right-hand side is bounded by γ i n k=1 π k + 3δ n k=1 π k, so that n n n ρ i k 6δ π k γ i π k. k=1 k=1 k=1 The proof of the right-hand side inequaity is simiar. 20

21 From Lemma 6.14 and (8) we get: Coroary Let δ > 0, and et (α 1,... α n ) be a δ-orthogona sequence of stationary strategy pairs. Denote α = α α n. Let 1 u, v If u γ i k for each k {1,..., n}, then u 6δ γi. 2. If γ i k v for each k {1,..., n}, then γi v + 6δ. 6.5 Strong Orthogonaity In the present section we define a stronger notion of orthogonaity and study its properties. Definition Let δ > 0. A sequence (α 1, α 2,..., α n ) of stationary strategy pairs is δ-strongy orthogona if, for every k {1,..., n 1} and every node s H δ (α α k ), α k+1 (s ) = (0, 0) for s = s and for every descendent s of s; that is α k+1 continues from s onwards. The foowing emma suggests a way to construct ɛ-orthogona sequences of strategy pairs from a singe ɛ 2 -strongy orthogona sequence. Lemma Let ɛ > 0 and et y 1, y 2,..., y n be stationary strategies of payer 2 such that the sequence ((0, y 1 ),..., (0, y n )) is ɛ 2 -strongy orthogona. Let x be any pure stationary strategy of payer 1 that does not stop twice on the same branch; that is, if x(s) = 1 then x(s ) = 0 for every descendant s of s. Define strategies ( x k ) n k=1 of payer 1 in the foowing way: for each s S such that x(s) = 1 et x k (s) = 1, where k n is the greatest index for which s / H ɛ ((0, y 1 ) (0, y k 1 )). Define x k (s) = 0 otherwise. Let ᾱ k = ( x k, y k ). Then the sequence (ᾱ 1,..., ᾱ n ) is ɛ-orthogona. Proof. By the definition of ( x k ) 1 k n {1,..., n 1} and Fact 1, we get, for every If x (s) = 1 then s H ɛ ((0, y 1 ) (0, y )). If x +1 (s) = 1 then s / H ɛ ((0, y 1 ) (0, y )). (19) Let {1,..., n 1}, and et s S be ɛ-heavy with respect to σ = ᾱ ᾱ. We prove that x +1 (s) = y +1 (s) = 0. We first prove that x +1 (s) = 0. Since s is ɛ-heavy w.r.t. σ = ᾱ ᾱ, P σ (t < F s ) ɛ. Assume to the contrary that x +1 (s) = 1. By (19) 21

22 s is ɛ-ight w.r.t. (0, y 1 ) (0, y ), and therefore P (0,y1 ) (0,y )(t < F s ) < ɛ. It foows that P ( x1,0) ( x,0)(t < F s ) > 0, a contradiction to the assumption that x does not stop twice on the same branch. We proceed to prove that y +1 (s) = 0. Assume first that there exists an ancestor s of s such that x 1 (s ) x (s ) = 1. By (19) and Fact 1 s H ɛ ((0, y 1 ) (0, y )). Since ((0, y 1 ),..., (0, y n )) is ɛ-strongy orthogona, y +1 (s) = 0. We assume now that x 1 (s ) x (s ) = 0 for every ancestor s of s. Let D be the (possiby empty) set of s s descendants d that are ɛ-heavy w.r.t. (0, y 1 ) (0, y ), and et D be the set that is obtained by removing from D a nodes that have strict ancestor in D. By the definition of D, P (0,y1 ) (0,y )(t < F d ) ɛ for every d D. Let Y = d D F d. Since this is a mutuay disjoint union, it foows that if Y φ then P (0,y1 ) (0,y )(t < Y ) ɛ ɛ P σ (t < Y ). By (19) and the definition of ( x k ) 1 k n it foows that P (0,y1 ) (0,y )(t < Y c F s ) = P σ (t < Y c F s ) ɛ P σ (t < Y c F s ). Combining the ast two inequaities, and observing that Y F s, we get P (0,y1 ) (0,y )(t < F s ) ɛ P σ (t < F s ) ɛ 2. Thus s is ɛ 2 -heavy with respect to (0, y 1 ) (0, y ) and, as the sequence ((0, y 1 ),..., (0, y n )) is ɛ 2 -orthogona, y +1 (s) = 0. Lemma Let ɛ > 0 be sufficienty sma and et a 1, b 1, a 2, b 2 satisfy a i < b i for i {1, 2}. Let (α 1,..., α n ) be an ɛ 2 -strongy orthogona sequence of stationary strategy pairs such that α k is an ɛ-equiibrium for each k = 1,..., n. Assume that for each k γ k [a 1, b 1 ] [a 2, b 2 ], where γ k is the payoff that corresponds to α k. Let α = α α n = (x, y). Then a) a i ɛ γ i (x, y). b) For each pair (x, y ) of stationary strategies, γ 1 (x, y) b 1 + 7ɛ and γ 2 (x, y ) b 2 + 7ɛ. Proof. Denote α k = (x k, y k ). We prove the resut ony for payer 1. We first prove (a). Since a 1 γ 1 k (x k, y k ) for each 1 k n, it foows from Coroary 6.15, and since ɛ is sufficienty sma, that a 1 ɛ a 1 6ɛ 2 γ 1 (x, y). 22

23 We now prove (b). Let x be a stationary strategy that maximizes payer 1 s payoff against y: γ 1 ( x, y) = max x γ 1 (x, y). Fixing y, the game reduces to a Markov decision process, hence such an x exists. Moreover, there exists such a strategy x that is pure (that is, x(s) {0, 1} for every s) and stops at most once in every branch. Observe that since the sequence (α 1,..., α n ) is ɛ 2 -strongy orthogona, so is the sequence ((0, y 1 ),..., (0, y n )). Let x 1,..., x k be the strategies defined in Lemma 6.17 w.r.t. x and y 1,..., y n. Then x = x x n, and (ᾱ 1,..., ᾱ n ) is ɛ-orthogona, where ᾱ k = ( x k, y k ). For each k, (x k, y k ) is an ɛ-equiibrium, and therefore γ 1 ( x k, y k ) b 1 + ɛ. By Coroary 6.15 and the definition of x, for every x one has γ 1 (x, y) γ 1 ( x, y) b 1 + ɛ + 6ɛ = b 1 + 7ɛ. 6.6 Proof of Proposition 6.4 We now prove Proposition 6.4. Consider the foowing recursive procedure: 1. Initiaization: Start with the game T = T, the strategy pair σ 0 = (0, 0) (aways continue) and k = If there exists a stationary ɛ-equiibrium in a subgame T of T with corresponding payoff in [a 1, b 1 ] [a 2, b 2 ]: (a) Set k = k + 1 and et α k = (x k, y k ) be any such ɛ-equiibrium. Extend x k and y k to strategies on T by setting x k (s) = y k (s) = 0 for every node s S 0 \ T. (b) Set σ k = σ k 1 +α k. (c) Let H k = H ɛ 2(σ k ) be the set of ɛ 2 -heavy nodes of σ k (by Fact 1 H k 1 H k.) Set T = T Hk. (d) Start stage 2 a over. 3. If, for a subgames T of T, there are no ɛ-equiibria in T with corresponding payoff in [a 1, b 1 ] [a 2, b 2 ], set n = k, x = x x n, y = y y n, and D = H n. The idea is to keep adding strongy orthogona ɛ-equiibria as ong as we can. The procedure continues unti there is no ɛ-equiibrium in any subgame of T with payoffs in [a 1, b 1 ] [a 2, b 2 ]. The termination of the procedure foows from Lemma

24 The first part of Proposition 6.4 is an immediate consequence of the termination of the procedure. We now prove that σ n = (x, y) satisfies the requirements of the second part. Since D = H n is the set of ɛ 2 -heavy nodes of (x, y), caim 2c in Proposition 6.4 foows. For every 1 k n, γ i (x k, y k ) R i ɛ, so that (x k, y k ) is an ɛ-equiibrium in T. Thus ((x 1, y 1 ),..., (x n, y n )) is an ɛ 2 -strongy orthogona sequence of stationary ɛ-equiibria. The remaining caims of Proposition 6.4 foow from Lemma Equiibria with Low Payoff In Proposition 6.4 we consider ɛ-equiibria with corresponding payoffs (u 1, u 2 ) such that u i R i ɛ. We now dea with the case in which one of the payers (w..o.g. payer 1) gets ow payoff. Lemma Let ɛ > 0, and et (x, y) be a stationary ɛ -equiibrium in T 2 such that γ 1 (x, y) R 1 ɛ. Then π(0, y) ɛ r 4 1, where r 1 = p( {F s, R{1},s 1 = R 1 }) is the probabiity that, if both payers never stop, the game visits a node s with R{1},s 1 = R1 in the first round. { 1, if R 1 Proof. Consider the foowing strategy z of payer 1: z s = {1},s = R 1 0, otherwise By the definition of z, and since payoffs are bounded by 1, ρ 1 (z, y) = E z,y [R 1 Q,s1 {t< } ] = E z,y [R 1 Q,s1 {t<,q={1}} ] + E z,y [R 1 Q,s1 {t<,2 Q} ] = R 1 P z,y (t <, Q = {1}) + E z,y [R 1 Q,s1 {t<,2 Q} ] R 1 P z,y (t <, Q = {1}) P z,y (t <, 2 Q). Since (x, y) is an ɛ -equiibrium, it foows that 2 (20). γ 1 (z, y) γ 1 (x, y) + ɛ 2 R1 ɛ 2. (21) Since π(z, y) = P z,y (t <, Q = {1}) + P z,y (t <, 2 Q), and by (21), (8) and (20) we get: (P z,y (t <, Q = {1}) + P z,y (t <, 2 Q)) R 1 π(z, y) ɛ 2 = = π(z, y) (R 1 ɛ 2 ) π(z, y) γ1 (z, y) = ρ 1 (z, y) R 1 P z,y (t <, Q = {1}) P z,y (t <, 2 Q). 24

25 In particuar, ɛ 2 π(z, y) (1 + R1 ) P z,y (t <, 2 Q) 2 P z,y (t <, 2 Q). As π(z, 0) π(z, y) and P z,y (t <, 2 Q) π(0, y) one has as desired. π(0, y) P z,y (t <, 2 Q) ɛ 4 π(z, y) ɛ 4 π(z, 0) = ɛ 4 r 1, 7 Constructing an ɛ-equiibrium In the present section we use a the toos we have deveoped so far to construct an ɛ-equiibrium. In section 7.1 we define a procedure that attaches for every finite tree T a coor. In section 7.2 we expain the main ideas of the construction. We then proceed with the forma proof. We fix throughout a stopping game that satisfies conditions A.1-A.6 in Proposition 5.6. In particuar, the constants R 1 and R 2 are fixed. We aso fix ɛ > 0 sufficienty sma. 7.1 Cooring a Finite Tree Definition 7.1. Let a 1 < b 1 and a 2 < b 2. A rectange [a 1, b 1 ] [a 2, b 2 ] is bad if R 1 ɛ a 1 and R 2 ɛ a 2. It is good if b 1 R 1 ɛ or b 2 R 2 ɛ. Let M be a finite covering of [ 1, 1] 2 with (not necessariy disjoint) rectanges [a 1, b 1 ] [a 2, b 2 ] such that b 1 a 1 < ɛ and b 2 a 2 < ɛ, a of which are either good or bad. Thus, for every u [ 1, 1] 2 there is a rectange m M such that u m. We denote by H = {h 1, h 2,..., h J } the set of bad rectanges in M, and by G = {g 1, g 2,..., g V } the set of good rectanges in M. Set C = G {ξ}. This set is composed of the set G of good rectanges together with another symbo ξ. For every game on a tree T consider the foowing procedure which attaches an eement c C to T : Set T (0) = T. 25

26 For 1 j J appy Proposition 6.4 to T (j 1) and h j = [a j,1, b j,1 ] [a j,2, b j,2 ], to obtain a subgame T (j) of T (j 1) and strategies (x (j) T, y(j) T ) in T such that 4 1. No strict subgame of T (j) has an ɛ-equiibrium with corresponding payoffs in h j. 2. Either T (j) = T (j 1) or the foowing three conditions hod. (a) a j,i ɛ γ i (x (j) T, y(j) T ) for i {1, 2}. (b) For every pair (x, y ), γ 1 (x, y (j) T ) b j,1 + 7ɛ and γ 2 (x (j) T, y ) b j,2 + 7ɛ. (c) π(x (j) T section 6.1., y(j) T ) ɛ2 p T (j),t (j 1), where p T (j),t (j 1) is defined in If T (J) is trivia (that is, the ony node is the root,) set c(t ) = ξ. Otherwise choose a stationary ɛ -equiibrium 2 (x(0), y (0) ) of T (J). By construction, the corresponding ɛ -equiibrium payoff ies in a good rectange 2 g G. Set c(t ) = g. 7.2 The Main Idea of the Construction Before formay constructing a Kɛ-equiibrium strategy pair for some fixed K > 0, we expain the basic idea of the construction. Assume for simpicity that a the σ-agebras F n are finite. In this case, every n 0, every ω Ω and every stopping time τ such that τ(ω) > n define naturay a game Γ n,τ (ω) on a tree; the root is the atom F of F n that contains ω, the nodes are a atoms F m n F m that satisfy 5 (i) F F, and (ii) if F F m then τ m on F. A atoms F where there is an equaity in (ii) are eaves. In section 7.1 we attached to each such tripet an eement from a finite set C - a coor. By Theorem 4.3, there is a sequence of bounded stopping times 0 τ 0 τ 1 such that p(c τ0,τ 1 = c τj,τ j+1 j > 0) 1 ɛ. Fix for a moment 0. In section 7.1 we constructed for each one of the finitey many trees T = Γ τ (ω),τ +1 (ω) and each j = 0,..., J a subtree 4 (x (j) T, y(j) T ) as given by Proposition 6.4 are strategies in T (j 1). We extend them to strategies in T by etting them continue from the eaves of T (j 1) downward. 5 In this union, a set F which is an atom of severa F m s is counted severa times. Thus, the union is actuay a union of pairs {(m, F ), F is an atom of F m }. 26

27 T (j) and a pair of stationary strategies (x (j), y (j) ). The eaves of the subtrees define naturay a stopping time τ (j). Thus, we obtain a sequence of stopping times τ τ (J) τ (J 1)... τ (1) τ (0) = τ +1. Let (x (j), y (j) ) J j=0 be the coection of strategy pairs that was generated during the cooring procedure. For 1 j J, et I j = {τ (j) < τ (j 1) i.o.}. Set G = ( j I j ) c. Then, on G, τ (J) < τ +1 ony finitey ( many times. In particuar, ) there is L 0 sufficienty arge such that p τ (J) < τ +1 for some L G < ɛ. Assume w..o.g. that L = 0. Set G v = G {τ (J) = τ +1 } {c τ,τ +1 = g v 0}, for every v = 1,..., V. Moduo punishment strategies, on I j, the Kɛ-equiibrium strategy pair coincides with the concatenation of the strategy pairs (x (j), y (j) ). It yieds payoff in the rectange h j. The condition {τ (j) < τ (j 1) i.o.} ensures that under the concatenation the game wi eventuay terminate with probabiity 1. On G v, the Kɛ-equiibrium strategy pair coincides with the concatenation of the strategy pairs (x (0), y (0) ). When the fitration is genera, one needs to approximate the F n s by finite sub-σ-agebras. This fact introduces some technica difficuties, but do not ater the genera idea. Adding a threat of punishment might be necessary as the foowing exampe shows. Exampe 7.2. Consider a game with deterministic payoffs: R {1},n = ( 1, 2), R {2},n = ( 2, 1), and R {1,2},n = (0, 3). We first argue that a ɛ-equiibrium payoffs are cose to ( 1, 2). Given a strategy x of payer 1, payer 2 can aways wait unti the probabiity of termination under x is exhausted, and then stop. Therefore, in any ɛ-equiibrium, the probabiity of termination is at east 1 ɛ, and the corresponding payoff is cose to the convex hu of ( 1, 2) and ( 2, 1). Since payer 1 can aways guarantee 1 by stopping at the first stage, the caim foows. However, in every ɛ-equiibrium (x, y), we must have P 0,y (θ < ) 1/2, otherwise payer 1 receives more than 1 by never stopping. Thus, an ɛ-equiibrium wi have the foowing structure, for some integer N. Payer 1 stops with probabiity at east 1 ɛ before stage N, and with probabiity at most ɛ after that stage; payer 2 stops with probabiity at most ɛ before stage N, and with probabiity at east 1/2 after that stage. 27

28 The strategy of payer 2 serves as a threat of punishment: if payer 1 does not stop before stage N, he wi be punished in subsequent stages. 7.3 Notations Denote δ n = ɛ 2 /2 n+2 for each n 0. Set n = k n δ k = ɛ 2 /2 n+1, so that n 0 n = ɛ 2. For every i 0 and every n N we choose once and for a a partition Bi n of the n 1-dimensiona simpex (n) = {x R n n j=1 x j = 1, x j 0 j} such that the diameter of each eement in Bi n is ess than δ i in the norm 1. We furthermore choose once and for a for each B Bi n an eement q B B. Definition 7.3. Let F = (F n, F n+1,..., F M ) be a sequence of σ-agebras. A F-strategy x for payer 1 is a coection x = (x i ) M i=n, where for each i, x i is a F i -measurabe [0, 1]-vaued r.v. F-strategies y of payer 2 are defined anaogousy. Given a pair (x, y) of F-strategies and a F-adapted stopping time τ > n, we denote by π(x, y; F, n, τ) the conditiona probabiity under (x, y) that the game that start at stage n ends before τ, and by ρ(x, y; F, n, τ) the corresponding expected payoff. We define γ(x, y; F, n, τ) = ρ(x,y;f,n,τ). These π(x,y;f,n,τ) are F n -measurabe r.v.s. In the seque, the sequence (F n, F n+1,..., F M ) in Definition 7.3 wi either coincide with the fitration of the game, or be a sequence of finite sub-σagebras that, in some sense, approximate the fitration. 7.4 Cose Games Let T be a stopping game on a finite tree with payoffs bounded by 1. Reca that S 0 is the set of nodes which are not eaves, and (p s ) s S0 are the probabiity distributions over chidren. Let T be a game that coincides with T, except for the probabiity distributions over chidren ( p s ) s S0 which satisfy p s p s 1 η depth(s), where (η j ) j 0 is a sequence of positive reas. Observe that the set of strategies of the two payers in T and in T coincide. Under these notations we have the foowing estimates. 28