OPTIMAL STOPPING FOR SHEPP S URN WITH RISK AVERSION

Transcription

1 OPTIMAL STOPPING FOR SHEPP S URN WITH RISK AVERSION ROBERT CHEN 1, ILIE GRIGORESCU 1 AND MIN KANG 2 Abtract. An (m, p) urn contain m ball of value 1 and p ball of value +1. A player tart with fortune k and in each game draw a ball without replacement with the fortune increaing by one unit if the ball i poitive and decreaing by one unit if the ball i negative. having to top when k = 0 (rik averion). Let V (m, p, k) be the expected value of the game. We are tudying the quetion of the minimum k uch that the net gain function of the game V (m, p, k) k i poitive, in both the dicrete and the continuou (Brownian bridge) etting. Monotonicity in variou parameter m, p, k i etablihed. Since the cae m p < 0 i trivial, for p, either m p α 2p, when the the gain function cannot be poitive, or m p < α 2p, when it i ufficient to have k p log p, where α i a contant. We alo determine an approximate optimal trategy with exponentially mall probability of failure in term of p. The problem goe back to Shepp [2], who determined the contant α in the unretricted cae when the net gain doe not depend on k. A new proof of hi reult i given in the continuou etting. 1. Introduction In [2], (alo in [3], [4], and [6]) Shepp tudied a erie of optimal topping problem. One of them i given in part a) of Section 6 of that paper, which contain a ketch of the proof, a follow. An (m, p) urn contain m ball of value 1 and p ball of value +1, and the player i allowed to draw ball randomly without replacement until he want to top (he i alo allowed not to draw at all). We are intereted in finding the value for m and p uch that Date: March 15, Mathematic Subject Claification. Primary: 60G40; Secondary: 91A60, 93E20, 60C05. Key word and phrae. Key word and phae: random urn, Chow and Robbin Sn/n problem, Brownian bridge, optimal trategy, optimal topping. 1 Department of Mathematic, Univerity of Miami, 1365 Memorial Drive, Coral Gable, FL , igrigore@math.miami.edu. 2 Department of Mathematic, North Carolina State Univerity, SAS Hall, 2311 Stinon Dr., Box 8205, Raleigh, North Carolina kang@math.ncu.edu. 1

2 there exit an optimal drawing policy for which the expected return (1.4), a a function of m and p, i poitive. Let C denote the et of all (m, p) urn for which the expected return i poitive. Shepp reaon, by analogy with the main problem in hi paper (the S n /n problem of Chow and Robbin [1]) that there exit a equence β(1), β(2),... of integer for which (1.1) C = {(m, p) : m β(p)}. Of coure β(p) p and for large p we have [β(p) p] (1.2) lim = α, p 2p where α = i the unique olution of the following integral equation explained later on in thi paper in equation (3.22), more exactly α = (1 α 2 ) 0 e αt t2 /2 dt. The initial fortune k i not relevant for the contruction of C in the unretricted model, when the player would be able to play for any k, i.e. with unbounded debt. In thi paper, we conider thi urn cheme problem with rik averion, which mean that the player cannot go below a certain value of hi fortune (cannot borrow money). We introduce ome notation. The proce (k n ) of fortune at time n, indexed by the dicrete time unit n 0 i adapted to a filtration (F n ) n 0. The aumption that the player cannot look into the future i formalized by requiring that any topping trategy τ be a topping time, i.e. {τ > n} F n, n 0. We notice that trivially τ m + p and taking z Z { } we define τ 0 = inf{n 0 k n z} a the hitting time of the level et z, with the uual convention that the infimum over the empty et i +. We hall define W (m, p, k; z) the value function of the game tarting with fortune k 0, m 0 negative ball, p 0 poitive ball, and topping when reaching fortune z (1.3) W (m, p, k; z) = up E[k τ ] τ T where T i the et of topping time not exceeding the minimum of the hitting time of the level et k = z and the final time n = m+p. In thi cae T i the et of admiible trategie of the game. In the dicrete cae, z < m i identical to z =. However, due to caling, in the continuou cae preented in Section 3 thi etup i relevant. Additionally G(m, p, k; z) = W (m, p, k; z) k i the expected return function of the game. 2

3 Inpecting the dynamic programming equation (2.1) we ee that when z = the return function i independent of k. Thi i the cae conidered in [2] and then we may implify the notation by etting (1.4) V (m, p) := G(m, p, k; ). We retain the notation V (m, p) in thi cae for conitency with the literature on the unretricted model [2, 3, 4]. For example [3] adopt k = 0 ince the initial fortune i not relevant, a mentioned. In the rik averion cae, which i the ubject of thi paper, without lo of generality, we et z = 0. To implify notation, ince throughout Section 2 dedicated to the dicrete cae the value z i fixed, we denote (1.5) V (m, p, k) = W (m, p, k; 0) and G(m, p, k) = G(m, p, k; 0). We are intereted in finding the mallet k for which the player make a poitive expected gain for a given (m, p) urn in C, i.e., finding the minimal k o that the expected value V (m, p, k) exceed k, when the player play optimally. The anwer can be ummarized a follow: (i) if m < k then the unretricted cae and the rik-averion cae are not ditinguihable and thu G(m, p, k) > 0 (Theorem 5 and 7); (ii) if m < p, then G(m, p, k) > 0 for all k 1; (iii) if m p, p i large and m p + α 2p, then G(m, p, k) = 0 for all k by (1.2) and Theorem 7; (iv) if m k, p i large and p m < p+α 2p, then a ufficient condition for G(m, p, k) > 0 i given in Theorem 1. Cae (iv) i the mot difficult ince it exhibit critical behavior, pointing naturally to the parabolic caling (3.7) from the invariance principle for random walk. The next reult follow a a conequence of Theorem 12, which i, in it turn, baed on the bound etablihed in Propoition 3. Theorem 1. Let (m, p) C defined in (1.1). If p i large and p m < p + α 2p, then there exit k(m, p) > 0 uch that G(m, p, k) > 0 if k k(m, p) and G(m, p, k) = 0 if 3

4 k < k(m, p). An upper bound for k(m, p) i given by c(p) 2p, with (1.6) c(p) = If, in addition, m p + α 0 2p, for ome α0 c ln p, c 1 = 3 2 ln 2 5 ln α exceeding the value x (α 0 ) defined in Propoition 3. The proof i at the end of the paper, after Theorem 13. [0, α), then c(p) ha finite order, not Remark. It i natural to look at k(m, p) on a cale of order m + p 2p, ince thi correpond to the caling for the diffuive limit from Theorem 9. Aymptotically, only pair (m, p) with m p + α 2p belong to C. The value of c = k(m, p)/ 2p i bounded and depend only on u = (m p)/ m + p (ee Propoition 3) provided that u remain bounded above away from α, ay u α 0 < α. It i only the wort cae cenario u α when c. In other word, given an arbitrary (m, p) C, the et of value allowing a poitive gain in the unretricted etting, (1.6) give a value of k that enure a poitive gain for the rik-averion etting. Numerical data how that c 3 for p from 2 to 1,000, which i conitent with the order of the approximation that predict c 2.22 with the logarithmic bound. We give a brief outline of the paper. Section 2 give monotonicity and comparion reult between the unretricted (z = ) and the rik averion (z = 0) cae in the dicrete etting via the recurrence correponding to the dynamical programming equation for the optimal value (2.1). Section 3 treat the continuou limit of the problem, where under the diffuive caling of Brownian motion (invariance principle) the underlying proce i a Brownian bridge. Theorem 10 prove the relevant monotonicity reult analogue to thoe of Section 2. Theorem 11 give a new proof in the cae z = outlined by Shepp [2], while Theorem 12 give an approximate winning trategy in the rik averion cae. In the dicrete etting, Theorem 1 and 13 provide the aymptotic upper bound for critical value of k for which a poitive return i poible, a well a the approximate trategy to achieve it, via the continuou caling limit. 2. Monotonicity reult in the dicrete cae The function V (m, p, k) atifie the following recurive relation, (2.1) V (m, p, k) = max k, m V (m 1, p, k 1) + p m + p m + p V (m, p 1, k + 1) 4

5 for all m, p, k 1 with the following boundary condition: For any m, p 0, V (m, p, 0) = 0, for any m, k 0, V (m, 0, k) = k, and for any p 0, k 1, V (0, p, k) = p + k. The optimal drawing policy i a follow: The game end if the player doe not have fortune any more or the firt time V (i, j, n) = n for ome n 1. In [3] and [2], Boyce and Shepp proved the following three inequalitie: V (m + 1, p) V (m, p), V (m, p) V (m, p + 1), and V (m, p) V (m + 1, p + 1) for all m, p 0. Below we how that the function V (m, p, k) exhibit imilar monotonicity. Theorem 2. For any fixed m, p 0, V (m, p, k) i increaing in k. Proof. We will prove thi inequality by induction on t = m + p. If either m or p equal 0, it i eay to ee that V (m, p, k) i increaing in k. From now on we will aume m, p 1. Suppoe t = m + p = 2, then m = p = 1. Then we have V (1, 1, 0) = 0, V (1, 1, 1) = 1 and V (1, 1, k) = 1 2 (2k + 1) for all k 2. So V (m, p, k) i increaing in k when t = m + p = 2. Suppoe that V (m, p, k) i increaing in k for all 2 t = m + p 1 for ome 3. Now for t = m + p =, we have V (m, m, k) = max{k, m V (m 1, m, k 1) + (1 m )V (m, m 1, k + 1)} max{k + 1, m V (m 1, m, k) + (1 m )V (m, m 1, k + 2)} = V (m, m, k + 1). Hence Theorem 1 alo hold for t = m + p =. Therefore, by mathematical induction, V (m, p, k) i increaing in k for all m, p 0. Theorem 3. For any fixed m, p, k 1, V (m 1, p, k 1) V (m, p 1, k + 1). Proof. When k = 1, V (m 1, p, k 1) = V (m 1, p, 0) = 0 < 2 V (m, p 1, 2) = V (m, p 1, k + 1). From now on, we will aume that k 2. A in the proof of Theorem 2, we will ue mathematical induction on t = m + p. Firt note that t 2, ince m, p 1. Suppoe that t = 2, then m = 1 and p = 1. V (0, 1, k 1) = k < k + 1 = V (1, 0, k + 1). Let t = 3. If m = 2 and p = 1, then V (1, 1, k 1) = k 1 2 k + 1 = V (2, 0, k + 1) for k 3. For k = 2, we have V (1, 1, k 1) = V (1, 1, 1) = 1 2 = k + 1 = V (2, 0, k + 1). If m = 1 and p = 2, then V (0, 2, k 1) = k + 1 k t = 2, 3. = V (1, 1, k + 1). So Theorem 3 hold for Suppoe that Theorem 3 hold for all m, p 1 uch that t = m + p 1 where 4. Now let t = m + p =. If m = 1, then V (m 1, p, k 1) = V (0, p, k 1) = k + p 1 5

6 k +p 1+ 1 p = V (1, p 1, k +1) = V (m, p 1, k +1), hence Theorem 3 hold. If p = 1, then V (m 1, p, k 1) = V (m 1, 1, k 1) = k 1 k + 1 = V (m, 0, k + 1) = V (m, p 1, k + 1), ince m 3 in thi cae. So Theorem 3 hold. From now on we will aume that m, p 2 and t = m + p = 4. For all 2 m 2, note that V (m 1, m, k 1) = max{k 1, m 1 m V (m 2, m, k 2) + V (m 1, m 1, k)}, 1 1 V (m, m 1, k + 1) = m m 1 max{k + 1, V (m 1, m 1, k) + V (m, m 2, k + 2)}. 1 1 By induction hypothei, we know V (m 2, m, k 2) V (m 1, m 1, k) V (m, m 2, k + 2), hence V (m 1, m, k 1) V (m, m 1, k + 1). Now Theorem 2 alo hold for t = m + p =. Induction complete the proof. Theorem 4. For all m, p, k 0, V (m, p, k) V (m, p + 1, k) and V (m, p, k) V (m 1, p, k). Proof. If one of m, p, k i 0, it i eay to check that Theorem 4 hold. So we will aume that m, p, k 1. We will ue induction argument on t = m + p. Suppoe t = 2, then m = p = 1 ince m, p 1. If k = 1, then V (1, 1, 1) = 1 2 = V (0, 1, 1) and V (1, 1, 1) = = V (1, 2, 1). If k 2, then V (1, 1, k) = k k + 1 = V (0, 1, k) and V (1, 1, k) = k k = V (1, 2, k). Hence Theorem 4 hold for t = 2. Aume that Theorem 4 hold for t and aume that t = m + p = + 1, where 2. Note that V (m, + 1 m, k) = max m + 1 m k, V (m 1, + 1 m, k 1) + V (m, m, k + 1), V (m 1, + 1 m, k) = max k, m 1 V (m 2, + 1 m, k 1) m V (m 1, m, k + 1) and V (m, + 2 m, k) = max m + 2 m k, V (m 1, + 2 m, k 1) + V (m, + 1 m, k + 1)

7 Hence to compare V (m 1, + 1 m, k) with V (m, + 1 m, k), it i enough to look at the following difference, m 1 m + 1 V (m 2, + 1 m, k 1) m V (m 1, + 1 m, k 1) m + 1 V (m 1, m, k + 1) V (m, m, k + 1) = m V (m 2, + 1 m, k 1) V (m 1, + 1 m, k 1) m + 1 V (m 1, m, k + 1) V (m, m, k + 1) m V (m 1, m, k + 1) V (m 2, + 1 m, k 1) 0, ( + 1) The reaon we get thi difference non-negative i the induction hypothei that grant V (m 2, +1 m, k 1) V (m 1, +1 m, k 1) 0 and V (m 1, m, k+1) V (m, m, k+1) 0 and Theorem 3 implying V (m 1, m, k+1) V (m 2, +1 m, k 1) 0. Hence V (m, + 1 m, k) V (m 1, + 1 m, k). The proof for V (m, + 2 m, k) V (m, +1 m, k) i completely analogou. Therefore by mathematical induction, Theorem 4 hold for all m, p, k 0. Boyce (in [3]) and Shepp (in [2]) proved that V (m, p) V (m + 1, p + 1) for any (m, p) urn. However, the tatement that V (m, p, k) V (m + 1, p + 1, k) i not true in general. The equation (2.1) enable actual numerical computation and we ee that, for example, V (8, 7, 5) < V (7, 6, 5) and V (6, 7, 3) < V (5, 6, 3). Theorem 5. For all m, p 0 and k m + 1, V (m, p) = V (m, p, k) k. Proof. It i eay to check that V (m, p) = V (m, p, k) k for all m + p = t = 0, 1, 2 and k m + 1. Aume that Theorem 5 hold for all t = m + p = 0, 1, 2,..., 1. Then by induction hypothei, V (m, m) = max 0, m [V (m 1, m) 1] + m [V (m, m 1) + 1] 7

8 = max 0, m [V (m 1, m, k 1) (k 1) 1]+ m = max 0, m [V (m, m 1, k+1) (k+1)+1] m V (m 1, m, k 1) + V (m, m 1, k + 1) k = V (m, m, k) k. Therefore, Theorem 5 hold for all m + p = t = 0, 1, 2,...,. By mathematical induction, the proof i complete. Recall that G(m, p, k) = V (m, p, k) k denote the expected net profit under an optimal drawing policy. Below we how that the expected net profit, G(m, p, k) i non-decreaing in k. Theorem 6. For all m, p 0, G(m, p, k) i non-decreaing in k. Proof. By Theorem 5, for all m, p 0, G(m, p, k) = V (m, p) for all k m + 1. We will aume 0 k m. It i eay to check that Theorem 6 hold if one of m, p, k equal 0. We will aume that m, p 1 and 1 k m. Then we have (2.2) G(m, p, k) = max 0, m V (m 1, p, k 1) + p m + p V (m, p 1, k + 1) k m + p Let t = m + p be the induction parameter. Firt conider the cae t = 2, then m = p = 1 ince we aume m, p 1. Then G(1, 1, k) = max 0, 1 2 [V (0, 1, k 1) + V (1, 0, k + 1)] k = 0. Hence Theorem 6 hold for t = 2. Now let u aume that Theorem 6 hold for for all m, p 1 and t = m + p = 2, 3,..., 1, where 3. Now for t =, the equation (2.2) give G(m, p, k) = max 0, m = max 0, m V (m 1, m, k 1) + m [V (m 1, m, k 1) (k 1)]+ m. V (m, m 1, k + 1) k [V (m, m 1, k+1) (k+1)]+ 2m (2.3) = max 0, m G(m 1, m, k 1) + m G(m, m 1, k + 1) + 2m. 8

9 On the other hand, G(m, p, k + 1) = max 0, m m V (m 1, m, k) + V (m, m 1, k + 2) (k + 1) = max 0, m [V (m 1, m, k) k] + m [V (m, m 1, k + 2) (k + 2)] + 2m (2.4) = max 0, m G(m 1, m, k) + m G(m, m 1, k + 2) + 2m. From the induction hypothei, we know that G(m 1, m, k 1) G(m 1, m, k) and G(m, m 1, k + 1) G(m, m 1, k + 2). Hence comparing the equation (2.3) and (2.4), we conclude G(m, m, k) G(m, m, k + 1) and Theorem 6 hold for t = m + p =. Theorem 7. For all m, p, k 0, V (m, p) G(m, p, k). Proof. By Theorem 5, V (m, p) G(m, p, k) hold if k m + 1. Now let u aume that k m. It i eay to verify that Theorem 7 hold if one of m, p, k i 0, we will focu on the cae m, p, k 1. Let t = m + p be the induction parameter. Firt notice that Theorem 7 hold for t = 2, in other word, for m = p = 1. Suppoe that V (m, p) G(m, p, k) hold for all m, p uch that 2 t = m + p 1 where 3. Then for t = m + p =, the induction hypothei give V (m, m) = max 0, m m [V (m 1, m) 1] + [V (m, m 1) + 1] max 0, m [V (m 1, m, k 1) (k 1)]+ m [V (m, m 1, k+1) (k+1)]+ 2m = max 0, m V (m 1, m, k 1) + m V (m, m 1, k + 1) k. Hence V (m, m) V (m, m, k) k and the mathematical induction complete the proof. Theorem 8. For all m, p, k 0, G(m, p, k) G(m + 1, p + 1, k + 1). 9

10 Proof. It i eay to check if one of m, p, k i 0, Theorem 8 hold. We will aume that m, p, k 1. For t = m + p which will be ued a the induction parameter, we firt etablih the inequality for all m, p uch that t = m + p 3 and for all k 3. G(1, 1, 1) = 0 < G(2, 2, 2) = 1 3 and G(1, 1, 2) = 1 2 < G(2, 2, 3) = 2 3. G(1, 2, 1) = 2 3 < G(2, 3, 2) = 6 5, G(1, 2, 2) = 4 3 < G(2, 3, 3) = 3 2, G(1, 2, 3) = 4 3 < G(2, 3, 4) = 3 2. G(2, 1, 1) = G(3, 2, 2) = 0, G(2, 1, 2) = G(3, 2, 3) = 0 and G(2, 1, 3) = 0 < G(3, 2, 4) = 1 5. Let u aume that Theorem 8 hold for all m, p uch that t 1 where 4. Now let t = and k, then (2.5) G(m, m, k) = max 0, m G(m 1, m, k 1) + m G(m, m 1, k + 1) + 2m and on the other hand, (2.6) G(m + 1, m + 1, k + 1) = max 0, m + 1 G(m, + 1 m, k) m + 2 G(m + 1, m, k + 2) + 2m. + 2 Firt note that G(m 1, m, k 1) G(m, + 1 m, k) and G(m, m 1, k + 1) G(m + 1, m, k + 2) from the induction hypothei. Now we will how G(m, m, k) G(m+1, m+1, k +1) by dealing with the cae, < 2m, = 2m and > 2m eparately. If < 2m, the difference of the econd term in the equation (2.6) and (2.5) i m m G(m, + 1 m, k) m + 2 G(m + 1, m, k + 2) + 2m + 2 m G(m 1, m, k 1) + G(m, m 1, k + 1) + 2m (2.7) 2m G(m, + 1 m, k) G(m + 1, m, k + 2) 2 0. ( + 2) To ee the inequality in (2.7), uing Theorem 3, note that G(m + 1, m, k + 2) + 2 G(m, + 1 m, k) = V (m + 1, m, k + 2) V (m, + 1 m, k) 0. 10

11 If = 2m, due to the induction aumption, G(m, m + 1, k) G(m 1, m, k 1) and G(m + 1, m, k + 2) G(m, m 1, k + 1). Hence G(m + 1, + 1 m, k + 1) G(m, m, k) = G(m + 1, m + 1, k + 1) G(m, m, k) = 1 2 G(m, m + 1, k) G(m 1, m, k 1) + G(m + 1, m, k + 2) G(m, m 1, k + 1) 0. Finally, if > 2m, then note that G(m + 1, p + 1, k + 1) G(m, p, k) = V (m + 1, p + 1, k + 1) V (m, p, k) 1. Here p > m ince = m + p > 2m. By mathematical induction, we can prove that V (m + 1, p + 1, k + 1) V (m, p, k) 1 0 if m < p. During the induction proce, we will encounter the cae that p = m + 1. In that cae V (m + 1, m + 2, k + 1) V (m, m + 1, k) = m + 1 m + 2 V (m, m + 2, k) + V (m + 1, m + 1, k + 2) 2m + 3 2m + 3 m m + 1 V (m 1, m + 1, k 1) + V (m, m, k + 1) 2m + 1 2m + 1 By the induction hypothei, V (m, m + 2, k) V (m 1, m + 1, k 1) 1. Similarly, ince V (m+1, m+1, k+2) = G(m+1, m+1, k+2)+k+2 and V (m, m, k+1) = G(m, m, k+1)+k+ 1, V (m, m+2, k) V (m 1, m+1, k 1) 1 becaue G(m+1, m+1, k+2) G(m, m, k+1). Therefore, G(m + 1, p + 1, k + 1) G(m, p, k) = V (m + 1, p + 1, k + 1) V (m, p, k) 1 0 if > 2m. Hence the proof i complete by induction. 3. The continuou cae Let M t be the number of ball marked with (minu), P t the number of ball marked with + (plu) and K t the current amount of money (fortune) at time t 0. We aume for the moment that the time i dicrete. A player pick a ball without replacement. If the ball i a minu, he loe one dollar, and if the ball i a plu, he win one dollar. We pecify that in thi variant of the game, the player mut play each time. Thi would not affect the anwer to the quetion on the minimum initial fortune to guarantee a poitive net profit G(m, p, k). However, it implifie the conervation law (3.1). 11

12 The game can only lat a maximum of P 0 + M 0 time unit, but will top a oon a K t = 0. There exit contant A, B equal to the two conerved quantitie in the problem (3.1) K t M t + P t = B, M t + P t + t = A. The probability that K t move up by one unit i equal to q t = P t /(P t + M t ) and down by one unit to 1 q t. Aume the proce (K t ) i defined on a probability pace (Ω, F, P ) and i adapted to the filtration (F t ). Let τ 0 be the hitting time of the level et zero (running out of money). We want to olve the optimal topping problem (3.2) V (M 0, P 0, K 0 ) = up E[K τ ], τ topping time. 0 τ A τ0 A player can control the outcome by chooing the topping time τ when to quit. The only retriction are that he cannot play after running out of money and he cannot look into the future, which i formalized by having {τ t} F t for any t [0, A]. The time of depletion of the plue (repectively minue) τ ± are uch that τ + τ A 1. For τ + τ t A the fortune move determinitically with q t = 0, repectively q t = 1. Thi allow to write the relation (3.3) Zt = Kt B A t = 1 2q t = Mt Pt, if 0 t < τ Pt+Mt + τ τ 0, Z t { 1, +1}, if τ + τ τ 0 t τ 0 where +1 i aumed when τ + < τ and 1 in the oppoite cae (equality cannot happen). It follow that q t = 1 2 (1 Z t), leading to the inhomogeneou Markov chain (3.4) K t+1 = K t ± 1, with probability 1 2 (1 Z t). Propoition 1. The proce (Z t ) i a martingale. Proof. Evidently K t K 0 + t < and A t 1, τ 0 i a finite topping time bounded above by a contant and the conditional expectation of the change in one tep can be verified directly. It follow that Z 0 = M t P t t=0 = M 0 P 0 = K 0 B, P t + M t P 0 + M 0 A 12

13 and ince K t B = (A t)z t, we can write (3.5) K t = K 0 (1 t A ) + B t A + (A t)(z t Z 0 ). It i known [5] (6.24) that the Brownian bridge (X t ) pinned at (a, b), i.e. X a = b - almot urely and tarting at (0, k 0 ) atifie (3.6) X t = k 0 (1 t a ) + b t a + (a t) t 0 dw a. One can ee that Z t i a dicrete verion of the martingale appearing in the tochatic integral. Let N > 0 and (3.7) k N t = N 1 K N 2 t, k 0 = N 1 K 0, b = N 1 B, a = N 2 A. The parabolic caling i natural to the Brownian motion. It can be regarded (conidering N ) a an invariance principle for the original K t with the undertanding that N 2 t macro = t micro, NK macro = K micro. We define (k N t ) t 0 at all lattice point N 1 Z, t = N 2 Z + and by linear interpolation between lattice point. The reulting path are continuou. We then have the following invariance principle. Theorem 9. Under the caling from (3.7), the proce (k N t ) t 0 converge weakly (in ditribution) to the Brownian bridge, i.e. the proce atifying the SDE (3.8) dx t = b X t a t dt + dw t, X 0 = k 0, where (W t ) t 0 i a tandard Brownian motion. Proof. The proof i tandard o we only ketch it. The caling limit i proven by howing that (k N t ) t 0, indexed by N > 0, i tight a a probability meaure on C([0, a δ], R). Here δ i arbitrary, trictly le than one. Any limit point i the law of a proce olving the martingale problem for Lf(t, x) = t f(t, x) + b x a t f (t, x) f (t, x) for all f Cc 1,2 ([0, a δ] R, R) (the ubcript i for compact upport in the pace variable). It i a conequence of Doob maximal inequality that the martingale part i tight. The drift term can be conidered on any time interval away from a, i.e. [0, a δ]. Uniquene of the olution to the martingale problem prove the reult on [0, a] by letting δ 0. 13

14 Denote the optimal function (3.9) w(a, b, k 0 ; z) = up E[X τ ], τ 0 = inf{t > 0 X t z} 0 τ τ0 a for the Brownian bridge (0, k 0 ) (a, b) topped at the level et z. Comparing to (3.1) we ee that necearily M t + P t O(N 2 ), M t P t O(N). The caling carrie over to (3.10) p t = N 2 P t, m t = N 2 M t, m t p t = N 1 (k t b) via the equation m t p t = k t b, m t + p t = a t. The analogue problem to (3.2) correpond to z = 0 and t = 0 and atifie the caling relation (3.11) v(m 0, p 0, k 0 ) := w(m 0 + p 0, k 0 m 0 + p 0, k 0 ; 0) N 1 V (M 0, P 0, K 0 ). Theorem 10. The optimal function v(m, p, k) and the gain function g(m, p, k) = v(m, p, k) k are (i) increaing in k when p and m are fixed; (ii) decreaing in u = m p when k and m + p are fixed. (iii) When z = the gain function w(m + p, k m + p, k; z) k doe not depend on k and i decreaing in m p when m + p i fixed. Proof. The firt obervation i that w(a, b, k; z) i increaing in b, k = k 0 and decreaing in z. In the variable z we imply have a larger et of topping time. In the other variable, we ue formula (3.6). The lat term i independent of any parameter. The expected value are monotone for each topping time τ in the optimization problem (3.9). It follow that the upremum are monotone in the repective parameter. We notice that uing the relation (3.12) w(a, b d, k d; z d) + d = w(a, b, k; z), true for any real number d, for (i) d = k we obtain v(m, p, k) = w(m + p, (m p), 0; k) + k and g(m, p, k) = w(m + p, (m p), 0; k), which how that v and g are increaing in k and (ii) decreaing in u = m p when k and m + p are fixed; (iii) d = k m + p we obtain g(m, p, k) = w(m + p, 0, m p; k + (m p)) (m p) which i increaing in k. When z = the fourth component of the function i and the gain function doe not depend on k and i decreaing in m p. 14

15 The optimal function can be further implified by noticing that (3.13) w(a, b, k 0 ; z) = a 1 2 w(1, a 1 2 b, a 1 2 k0 ; a 1 2 z) and uing (3.12). Putting z = 0 and d = b, x = a 1 2 k0, r = a 1 2 b and u = x r = a 1 2 (m0 p 0 ), we can ee that (3.14) a 1 2 (w(1, 0, u; r) + r) = w(a, b, k0 ; 0), which, in the original variable a = p 0 + m 0 = p + m, b = b 0 = k 0 m 0 + p 0 = k m + p, can be written a (3.15) v(m, p, k) = m + p w 1, k m + p k, ; 0. m + p m + p In imilar fahion, we have the gain function (3.16) g(m, p, k) = v(m, p, k) k = a 1 2 (w(1, 0, u; r) u) with the ame notation. An explicit formula (3.25) exit in [2] for the unretricted reource cae when the player doe not face ruin, i.e. for z =. Let τ c be the hitting time of the curve c a t by the Brownian bridge X t between the point (0, u) (a, 0), a > 0, c 0 and (3.17) v(a, u, c) = E (a,0) (0,u) [X τc1 τc<a]. In the following we drop the initial and terminal point cript when there i no danger of confuion. It i proven in the ame paper via (3.19) that when c > ua 1 2, then τc < 1 almot urely (evidently when c = u we have τ c = 0). The Brownian bridge (3.8) admit a repreentation due to Doob of the form (3.18) X t = b + B + k 0 b 1 + /a, = t 1 t/a, 0 t < a. Baed on the normalization decribed in (3.13), we hall focu on the Brownian bridge X t with a = 1. In term of (3.18) with a 1, k 0 u and b 0, the topping time = τ c /(1 τ c ) will give the equation (3.19) B + u = c

16 Thi permit an exact calculation, with the exact formula (3.20) v(1, u, c) = ce[ 1 τ c ] = where we introduced the notation (3.21) h(a, u) = 0 ch(1, u) h(1, c) = c e u 2 2 Φ(u), e c2 2 Φ(c) e λu λ2 a 2 dλ, < u <, a > 0. Here Φ(z) denote the ditribution function of the tandard normal. integration or uing the Laplace tranform of the complementary error function, h(a, u) = a 1 2 h(1, a 1 2 u) and h(1, u) = 2πe u 2 2 Φ(u). Then, directly by The function in (3.20) ha a maximum in the variable c at α > 0, equal to the olution of h(1, α) αh (1, α) = 0, or equivalently (3.22) α = (1 α 2 ) 0 λ2 λα e 2 dλ. Similarly u (α/h(1, α))h(1, u) u i well defined for all real u and convex. It global minimum i zero and i achieved at u = α. Thi i exactly the gain function in the cae z =, i.e. g(1, 0, u) = w(1, 0, u; ) u from (3.16) (uing the ame notation for implicity). It ha a double zero at u = α (3.23) g(1, 0, α) = 0, g (1, 0, α) = 0, g (1, 0, α) = 2α > 0, where the lat equality i obtained by direct computation. Theorem 11 how that, a long a u α, the optimal function i the value of v at c = α (3.24) w(1, 0, u; ) = v(1, u, α). More preciely, with the equality (3.22) in mind, (1 α 2 ) e λu λ 2 2 (3.25) w(1, 0, u; ) = 0 dλ, u α u u > α. A dicued, the game allow an expected net gain w(1, 0, u; ) > u only when u < α. In view of (3.14) we have (3.26) w(a, b, k; ) = a 1 2 w(1, 0, a 1 2 (k b); ) + b 16

17 and again (3.27) w(a, b, k; ) = α ah(a, k b) + b. h(1, α) The relation (3.28) u w(1, 0, u; ) = how that (3.29) Theorem 11. The proce α h(1, α) a 1 2 h(1, a 1 2 (k b)) + b = t i a (F t ) - martingale. αh(1, u) h(1, α) u, α h(1, α) (a t)h(a t, k t b) + b, u α α ah(a, k b) + b k. h(1, α) 0 t a Due to (3.29), by comparing the value at t = 0 and t = τ, a topping time in the interval [0, a], we obtain the upper bound (3.30) α ah(a, k b) + b w(a, b, k; ). h(1, α) For initial 0 a 1 2 (k b) α, the Brownian bridge Xt = (k t b)/(a t) reache the curve α a t at τ α [0, a] with probability one and the bound i achieved for thi topping time, proving (3.30) hold with equality. For initial a 1 2 (k b) > α, the optimal function (3.26) i contant equal to k and i realized at τ = 0. Proof. We have to how that t (a t)h(a t, k t b) i a martingale. The function i non-negative. Modulo a determinitic linear part in t, a Brownian Bridge ha a mean zero Gauian component repreented by the tochatic integral in (3.6) with covariance t ta 1. For a given t [0, a] the variance i t(a t)a 1. Fubini theorem applied to (3.21) when taking the expected value prove that E[(a t)h(a t, k t b)] i alway finite. Ito formula give the time component h(a t, k t b) (a t) a h(a t, k t b) +(a t) u h(a t, k t b)( k t b a t ) u 2 h(a t, k t b) = [uh(a t, u) (a t) u u h(a t, u)] = 0, u = k t b, 17

18 after noticing that for any a > 0, u R, the function h atifie the equation uh(a, u) a h(a, u) = u λ 2 0 λ eλu 2 a dλ = 1. From here we apply the optional topping theorem. To prove that a oon a a 1 2 (k b) > α we have w(a, b, k; ) = k, we notice that in Theorem 10 (iii) the function u w(1, 0, u; ) u i non-increaing in u. Since at u = α the function vanihe, and the olution i trivially non-negative by adopting τ = 0 in (3.9), we are done. Let X t be the Brownian bridge pinned at zero when t = 1, i.e. cae a = 1 in (3.17). The expected value of the proce at the topping time τ c τ 0 provide a lower bound for the optimal function w(1, 0, u; u k). Formula (3.41) p. 265 in [5] give the ditribution function of τ 0 (3.31) F τ0 () = P u (τ 0 ) = 1 Φ(b t + b + u t ) + e 2(b+u)b Φ(b t b + u t ), t = 1. Here and the next propoition we follow the convention from (3.17) that E u [ ] = E (1,b) (0,u) [ ]. Propoition 2. Baed on the ditribution function F τ0 and F τc we can calculate (3.32) E u [X τc τ01 τc<1] = v(1, u, c) 1 with lower bound l 2 l 1 l 0, in order of implicity, (3.33) l 2 = v(1, u, c) ( v(1 t, b, c) + b)p u (τ c > t)df τ0 (t), (c 1 t + b)p u (τ c > t)df τ0 (t), (3.34) l 1 = v(1, u, c) (c + b) 1 0 P u (τ c > t)df τ0 (t), (3.35) l 0 = v(1, u, c) (c + b)p u (τ 0 < 1), P u (τ 0 < 1) = e 2(b+u)b. Remark. 1) The Laplace tranform of F τc can be calculated in the ame way a (3.20) [2]. 2) Formula (3.20) calculate the expected value under the event that τ c 1 (the event τ c = 1 ha zero probability), which wa hown to have probability one when 0 u c. However, the function v i well defined when u < 0. Thi i needed in the integrand of the econd term of (3.32) where the tarting point b after τ 0 may be negative. 18

19 Proof. Denote τ = τ c τ 0 for implification. In all relevant cae c > 0 and τ c = τ 0 ha zero probability. Then (3.36) E u [X τ 1 τc<1] = E u [X τ 1 τc<1,τc<τ0] + E u [X τ 1 τc<1,τc>τ0] = E u [X τc 1 τc<1,τc<τ0] bp u (τ c < 1, τ c > τ 0 ) (3.37) = v(1, u, c) E u [X τc 1 τc<1,τc>τ0] bp u (τ c < 1, τ c > τ 0 ). Since c > u guarantee τ c < 1 with probability one, the probability in the lat term ubtracted equal (3.38) P u (τ c < 1, τ c > τ 0 ) = 1 0 P u (τ c > t)df τ0 (t) P u (τ 0 < 1) = e 2(b+u)b, where the upper bound wa obtained from (3.31) at t = +. Noting that {τ c > τ } = {τ c > τ 0 }, the other term ubtracted in (3.37) equal E u [X τc 1 τc<11 τc>τ ] = E u[1 τc>τ E u[x τc θ τ +τ 1 τc θ τ +τ <1 F τ ]] = 1 0 v(1 t, b, c)p u (τ c > t)df τ0 (t) which give (3.32) after applying the trong Markov property for τ ince {τ c > τ c τ 0 } i F τ - meaurable. We recall the gain function g(m, p, k) = v(m, p, k) k = w(a, b, k; 0) k = a 1 2 (w(1, 0, u; u x) u) from (3.16). Propoition 3. Let ḡ(u, x) = (w(1, 0, u; u x) u) +. (i) If u α, then ḡ(u, x) = 0 for any x 0. (ii) If 0 u < α, there exit x = x (u) u uch that ḡ(u, x) > 0 for any x > x and ḡ(u, x) = 0 for any x x. In thi cae, for any c u, x u (3.39) ḡ(u, x) v(1, u, c) u (c + x u)e 2x(x u), and for any c (u, α], the lower bound from (3.39) een a a function of x ha a unique zero x c (u, ), and the contant x inf c [u,α] x c. There exit value u for which the infimum over c i obtained at c < α. Moreover, lim u α inf c [u,α] x c = +. 19

20 Remark. 1) We may adopt c = α in (3.39) to obtain an etimate for x. The value obtained by (3.39) i not optimal. 2) The limit of the upper bound of x (u) only ay that our approximation (3.39) blow up a u α. It i a conjecture that, in fact, lim u α x (u) < +. Proof. The topping time τ = τ c τ 0 provide a poible trategy and thu a lower bound for the gain function i E u [X τ ] u. Thi i provided by the exact calculation (3.36), noticing that b = x u. The integral in (3.32) i bounded above by (3.40) cp u (τ 0 < τ c ) cp u (τ 0 < 1) = ce 2(b+u)b = ce 2x(x u). and the other term ubtracted in (3.37) i exact. Since E u [X τc ] = E u [X τc 1 τc<τ0] + E u [X τc 1 τc τ0] and X τc 1 τc<τ0 = X τc τ01 τc<τ0, we have the inequality E u [X τc τ01 τc<τ0] = E u [X τc ] E u [X τc 1 τc τ0] E u [X τc ] cp u (τ 0 < 1) v(1, u, c) cp u (τ 0 1), ince X τc = c 1 τ c c and τ c 1 a.. On the other hand E u [X τc τ0] = E u [X τc τ01 τc<τ0] + E u [X τc τ01 τc τ0] v(1, u, c) (c + (x u) + )P u (τ 0 1). Equation (3.40) p. 240 in [5] how that P u (τ 0 1) = e 2x(x u) a long a x > 0 and x > u. The function x v(1, u, c) u (c + x u)e 2x(x u) in (3.39) i conidered only for x u. Since v(1, u, c) c < 0, it tart with a negative value at x = u. It derivative e 2x(x u) [(4x 2u)(c + x u) 1] cannot have two zero on [u, ), otherwie the vertex of the parabola in bracket (3u 2c)/4 < u < c would alo be greater than u. Becaue the derivative ha at mot one zero, the function tart with a negative value and end a x with a poitive value v(1, u, c) u > 0, it i analytic (the zero are not dene), it follow that there exit exactly one zero in (u, ). Since v(1, u, x) u > 0 for 0 u < α, we are done. It i eay to ee that u < 0 (i.e. m 0 < p 0 ), then any trategy produce a win with probability one ince the terminal point i zero, greater than u. 20

21 Let x be the contant obtained in (3.39) and c the value for which it i achieved. Theorem 12. A winning trategy for the continuum problem (3.9) with initial data (m, p, k) i to calculate u = m p m+p and (i) if u α then top; (ii) if 0 u < α, then play until k t b = m t p t c mt + p t or k t = 0, whichever occur firt, allowing the non-optimal but exponentially mall probability of ruin. Proof. After time caling with a factor of a, (k at x)/ a c 1 t i equivalent to k t b = m t p t c m t + p t for any c and thu for the optimum c a well. It i clear from (3.32) that for any c u (3.41) w(1, 0, u; b) u 0 v(1, 0, u; b) u, where v(1, 0, u; b) = up E u [X τc τ0 1 τc 1]. c u Uing Propoition 3 we optimize over c obtaining c. Finally we can formulate an approximate winning trategy for the dicrete problem (3.2), providing the bet next move baed on the current data. Theorem 13. Let (M, P, K) be the current data and aume M + P i ufficiently large. Calculate u = M P M+P > 0 and (i) if u α then top; (ii) if 0 u < α, then check if K > x (u) M + P ; if true continue, and if not, top. Proof. The dicrete cae i an approximation baed on the continuum cae. Let X C([0, a], R) be the ubet of continuou path ω pinned at zero at time t = a and X 0 Ω be the ubpace of path killed at the exit time t 0 = t 0 (ω) from the open et bounded above by x = c 1 t and below by x = z, i.e. ω(t) ω(t 0 ) for all t t 0. Thi time i finite, t 0 [0, a] for ω Ω ince the upper boundary i itelf in X. The trajectorie of the dicrete time proce can be embedded in the pace X 0 by imple linear interpolation, in the tandard contruction ued to prove Donker invariance principle. Under the caling (3.7), the family of procee {k N t } N>0 i tight (i.e. the law are compact in the ene of Prokhorov theorem). The limiting proce (X t ), the Brownian bridge killed upon exiting the region at t 0, i alo concentrated on the ame path pace X 0. Now conider a functional Φ on X defined by Φ(ω( )) := ω(t 0 ) = max{c 1 t 0, z}. We note that Φ i not continuou on X and X 0 i not a cloed ubpace of X in the upremum 21

22 norm topology. However, convergence till hold. From Portmanteau theorem on page 347 in [7], it follow that for a meaurable functional Φ(ω( )) on Ω 0 we have (3.42) lim N E[Φ(kN )] = E[Φ(X )] a oon a Φ( ) i bounded and continuou at all point in X 0 and the probability law P X of the limiting proce (X ) atifie P X (X 0 ) = 1. It i ufficient to ee that Φ(ω( )) := ω(t 0 ) = max{c 1 t 0, z} i bounded by c z and i continuou on the et of killed path X 0. If two path are within ditance from each other in the uniform norm, baed on the fact that the exit time from the region i actually in the interval [0, a], we can pick the one exiting firt. From there on, the other path mut exit at a point which necearily i within of the exit value of the firt one, ince the firt path remained contant. The functional Φ define the value of the game for both the dicrete (k N t ) and continuou limit (X t). Since Propoition 3 prove a lower bound for the continuou limit, we obtained that the approximate trategy (i)-(ii) i a winning trategy a N = M + P i large. We are ready to complete the proof of Theorem 1 in the dicrete model verion. Proof of Theorem 1. 1) The exitence of a critical k(m, p) i a conequence of Theorem 6 which how that the net gain function i nondecreaing in k. It i ufficient to find a value k with poitive net gain. Part 2) of the proof will prove the exitence of uch a k and the upper bound for it minimal value a p. 2) We denote z = (m p)/ 2p and we have z < α in the hypothei. The ratio u = (m p)/ m + p from Propoition 3 will then atify u < α. Thi give u = u(z) = z(1 + z ) 1 z 2 = z(1 2p 2 2p + 3z2 16p...). Let x(u) be the olution of the equation in x of the lower bound from (3.39) et equal to zero. Any c α in the formula would provide a lower bound, o we may take the extreme cae c = α. More preciely x(u) i the olution of v(1, u, α) u (α + x u)e 2x(x u) = 0, u = z(1 + z 2p ) 1 2. It i then ufficient to take (3.43) c(p) = up x(u(z))(1 + z ) 1 2 z<α 2p 22

23 uch that k c(p) 2p implie G(m, p, k) > 0. For large p, the function maximized in the formula for c(p) i an increaing function in z, which lead to c(p) = x(u(α))(1 + α 2p ) 1 2, which i, a expected, the cae when m = p + α 2p. From (3.23) we know that v(1, u, α) u 1 2 g (1, 0, α)(α u) 2, with g (1, 0, α) = 2α, which give the aymptotic value of x(u), now a function of p only, α 2 ( + x)e 2x(x α+) = 0, = α2 2 2p. The function in the equation i increaing in x, giving a ufficient lower bound x x δ where x δ i the olution of the impler equation α 2 e (2 δ)x2 = 0, for any δ > 0 ufficiently mall. Inverting x δ, letting δ 0 and uing the relation between and p we obtain the ufficient condition c(p) ln α 2 + ln 2 2 α ln p, which implie the lower bound (1.6). 3) To prove the econd bound of the theorem when m p + α 0 2p we imply notice that in that cae c(p) defined in (3.43) ha limit x (α 0 ) a p becaue the upremum i taken over value z α 0 < α. 4. Open problem The ummary of the current reult given above Theorem 1 point out to a a few open quetion. Aume m p and p i large. If m p + α 2p, then V (m, p) = 0 and V (m, p, k) k = 0. We concentrate on the cae p m < p + α 2p correponding to (iv). In thi cae we know that V (m, p) > 0. Since (4.1) 0 V (m, p, k) k V (m, p), a quetion i: What are the larget value of k for which 1) the firt inequality i an equality, and then 2) the econd inequality i trict, repectively? Denote by k 1 (p) and k 2 (p) the two value. If k k 1 (p) the econd equality cannot take place, o k k 2 (p), implying that k 1 (p) k 2 (p). We know that k > m implie k > k 2 (p), o we have k 1 (p) k 2 (p) m < p + α 2p. Theorem 2 prove that k 1 (p) i at mot of order p ln p. 23

24 Suppoe γ (0, α) and m p + γ 2p. Then what are k 1, k 2 een a function of γ? Some numerical computation eem to upport that, in fact, G(m, p, k) < V (m, p) if k m, conjecturing that k 2 (p) = m p + α 2p whenever (m, p) C. However we don t have a proof for the trict inequality, G(m, p, k) < V (m, p) if the (m, p) urn i in C. Acknowledgement. We wih to thank to Burton Roenberg for valuable dicuion related to the computational apect of the problem. Reference [1] Y. S. Chow and H. Robbin, On optimal topping rule for Sn/n, Illinoi J. Math., Vol. 9, pp (1965). [2] L. A. Shepp, Explicit Solution to Some Problem of Optimal Stopping, Ann. Math. Statit., Vol. 40, pp (1969). [3] W. M. Boyce, On a Simple Optimal Stopping Problem, Dicrete Math., Vol. 5, pp (1973). [4] R. W. Chen and F. K. Hwang, On the Value of an (m,p) Urn, Congreu Numerantium, Vol. 41, pp (1984). [5] I. Karatza and S. E. Shreve, Brownian Motion and Stochatic Calculu, Graduate Text in Mathematic, Vol Springer-Verlag, New York (1991). [6] R. W. Chen, A. M. Odlyzko, L. A. Shepp and A. Zame, An Optimal Acceptance Policy for an Urn Scheme, Siam J. Dicrete Math., Vol. 11, pp (1998). [7] P. Mörter and Y. Pere, Brownian Motion, Cambridge Serie in Statitical and Probabilitic Mathematic, Cambridge Univerity Pre, Cambridge (2010). 24