Sequetial Aalysis, 5: 35 363, 006 Copyright Taylor & Fracis Group, LLC ISSN: 0747-4946 prit/53-476 olie DOI: 0.080/07474940600934805 Optimal Two-Choice Stoppig o a Expoetial Sequece Larry Goldstei Departmet of Mathematics, Uiversity of Souther Califoria, Los Ageles, Califoria, USA Ester Samuel-Cah Departmet of Statistics, The Hebrew Uiversity of Jerusalem, Jerusalem, Israel Abstract: Let X X X be idepedet ad idetically distributed with distributio fuctio F. A statisticia may choose two X values from the sequece by meas of two stoppig rules t t, with the goal of maximizig E X t X t. We describe the optimal stoppig rules ad the asymptotic behavior of the optimal expected stoppig values, V, as, whe F is the expoetial distributio. Specifically, we show that lim F V = e, ad cojecture that this same limit obtais for ay F i the (Type I) domai of attractio of exp e x. Keywords: Domais of attractio; Multiple-choice stoppig rules; Prophet value. Subject Classificatios: 60G40.. INTRODUCTION AND SUMMARY Let X X be idepedet ad idetically distributed (i.i.d.) radom variables from a kow distributio F, where is a fixed horizo. We cosider the situatio where the aim of a statisticia (optimal stopper) is to sequetially pick as large a X value as possible, but ulike the classical case, where oly oe choice is permitted, the statisticia here is permitted two choices, ad the secod choice may deped o the first. The value to the statisticia for usig stoppig rules t ad t is the maximum of X t ad X t, leadig to the goal of maximizig E X t X t over all stoppig rules t t satisfyig t t. The statisticia s first choice, X t, ca be thought of as a guarateed fallback value. A situatio as described may arise, Received Jauary 4, 005, Revised March 9, 005, May 3, 005, Accepted May 3, 005 Recommeded by Nitis Mukhopadhyay Address correspodece to Larry Goldstei, Departmet of Mathematics KAP 08, Uiversity of Souther Califoria, Los Ageles, CA 90089-53, USA; Fax: 3-740-44; E-mail: larry@math.usc.edu
35 Goldstei ad Samuel-Cah e.g., if oe is iterested i buyig a house, ad while ispectig houses, ad oly oe house is eeded, oe may put oe other house o hold. Oe iterestig aspect of this problem is that the usual backward iductio does ot apply directly, ad a two-stage backward iductio is eeded. Cosider first a oe-choice situatio, where a fixed value x has already bee promised. Let V x deote the optimal retur for this situatio. The, clearly, with X F, V 0 x = x ad V x = E[ X V x ] for all, (.) ad the optimal stoppig rule i the -horizo problem is = mi { i X i V i x } (.) If we deote g x = E X x the the recursio i (.) ca be writte as V x = g g x = g x (.3) The usual oe-choice value, V, is clearly V, or, if X 0, the also equal to V 0. Let V deote the optimal value attaiable i the two-choice situatio, for horizo. The, similar to the form of (.), we have the followig backward iductio: V = E X X ad V = E[ V V X ] for all > (.4) The iterpretatio of (.4) is as follows: If the curret observatio (which we have deoted as just X) is large eough, take it, ad the cotiue optimally as i the oe-choice situatio, where the value X is guarateed. If X is ot chose, cotiue optimally with the two-choice situatio ad horizo. The optimal strategy is, therefore, for a first choice use = mi{ i V i X i V i} for a secod choice, apply the rule of (.) adapted to the time of the first choice, ad to the value X chose, that is, use = mi{ i <i X i V i X } Our iterest lies i fidig the asymptotic behavior, as, of the sequece V of (.4). It is well kow that there are three types of asymptotic distributios for the maximum (see Leadbetter et al., 983, p. 4), correspodig to three domais of attractio. The asymptotic behavior of V for the oe-choice situatio has bee studied by Keedy ad Kertz (99), who show that the limitig behavior of V depeds upo the domai of attractio to which F belogs. This will therefore clearly also be the case for the two-choice value sequece, V. It is evidet that, because of the much more complicated structure of the value sequece V over V, as give i (.4) ad (.) respectively, the study of the asymptotic behavior for two choices will be more ivolved.
Optimal Two-Choice Stoppig 353 I the preset article, we study the case where F x = e x, i.e., the expoetial distributio. This distributio belogs to the domai of attractio of exp e x. Examples of the asymptotic behavior of V for the two-choice stoppig problem o i.i.d. sequeces with distributios belogig to the two other domais of attractio are studied i Assaf et al. (004, 006). Our mai result here, for the caoical represetative of the distributios i the Type I domai of attractio, is the followig. Theorem.. The Let X X X be i.i.d. expoetially distributed radom variables. lim ( F V ) = e (.5) where V is the optimal two choice value. Let M = max X X. The correspodig asymptotic values for V EM are ad lim F EM = e (.6) where is Euler s costat (see, e.g., Leadbetter et al., 983), ad, as obtaied by Keedy ad Kertz (99), lim ( F ( )) V = (.7) Limit (.6) for the maximum ad limit (.7) for the optimal oe-choice rule hold for ay F belogig to the domai of attractio of exp e x. I additio, the two correspodig limits over the other two domais of attractio behave i this same fashio, as do all kow limits for the optimal two-choice rule (see Assaf et al., 004, 006). Based o this evidece, we cojecture that (.5) holds for all F i the Type I domai of attractio. For the expoetial distributio, we may assume without loss of geerality that =, for which (.5) (.7) are easily see to be equivalet to ad lim ( V log ) = log e = 0 4586 (.8) lim log = = 0 577 ad ( lim V log ) = 0 (.9) respectively.. PRELIMINARIES AND HEURISTICS For F x = e x we have V x = g x = E x X = x e x. To simplify otatio, we write V istead of V. Usig (.3), the recursio i (.4) ca be writte as V = 0 g x V e x dx (.)
354 Goldstei ad Samuel-Cah or, if we let b > 0 deote the uique value such that g b = V (also called the idifferece value), the (.) ca be rewritte as Set The, (.) ca be rewritte as or V = e b V g x e x dx (.) b h x = g x log log a = log / (.3) B = b log ad W = V log = h B (.4) W = ( ) e B W h x e x dx a (.5) B W W = e B W h x e x dx a (.6) B To motivate our result, cosider the followig heuristics. Assume that for some B ad h, The, uder regularity, usig (.4), B B ad h y h y as W = h B h B = W (.7) But ow, should W W coverge to a ozero costat A, W would grow like A k A log, givig a cotradictio. Hece W W must ted to zero, ad takig limits i (.6) yields 0 = We B B h x e x dx (.8) Substitutig h B = W from (.7) ito (.8) gives a equatio for the ukow B, thus yieldig W if h were kow. Here is a heuristic for determiig h: By (.9), [ lim V log ] = 0 Sice V is the value whe othig is guarateed, we have V = g 0, ad thus g 0 log (.9) Suppose that for large eough ad a fixed guarateed value x, there is t such that g x = g t 0 = g g t 0 (.0)
Optimal Two-Choice Stoppig 355 That is, there is some umber of extra observatios t such that the statisticia is idifferet to havig t variables from which to chose, or the guarateed x ad variables. Equatio (.0) implies x = g t 0 log t, yieldig t e x. But o the other had, g t 0 log t log e x g x. Usig (.3), we have This suggests h x log e xlog log = log e x lim h x = h x = log e x = x log e x <x< (.) From (.8), (.7), ad (.), B solves = log ( e B) e B e B log ( e Bu) e u du (.) Lettig s = e u, the itegral i (.) ca be evaluated as log ( e Bu) ) e u du = log ( eb ds = e B(( e B) log e B B ) 0 0 s ad ow substitutio back ito (.) yields B = log e B, the uique solutio of which is B = log e = 0 543 ad ow, from (.7) ad (.), W = log e = 0 45867, which is equivalet to coclusio (.5) of Theorem.. A rigorous proof of the theorem is give i Sectio 4. 0 3. PROPERTIES OF h AND THE LIMITING h Lemma 3.. h x is strictly mootoe icreasig for log x<. Proof. We have that g x, ad hece g x, are strictly mootoe icreasig for x 0, ad ow the result follows by (.3). Lemma 3.. Let h x be give i (.). The, h x > h x for x log, =. Proof. For = the claim is simply that h x = x e x >x log e x = h x for all x 0, which is immediate. Now suppose the claim holds for. We show that it holds for. By the iductio hypothesis, g x = log h x log > log h x log = log log e x log = log e x (3.)
356 Goldstei ad Samuel-Cah Thus, sice g is icreasig, g x = g g x > g log e x = log e x e log ex = log e x (3.) ex Thus, similar to (3.) it suffices to show that the right-had side of (3.) is greater tha log e x. The latter statemet is equivalet to > log ( ) e x e, x which clearly holds. Lemma 3.3. Let x = h x h x. The, x <e x /, for x log. Proof. For = we have x = e x log e x, so clearly the statemet holds for =. Now, usig (.3), h x = g x log log = g x log e g xlog log = h x a a e h xa log I particular, for =, = h x a e h xa a (3.3) h x = h x a e h xa a = x a e xa e xa e xa a We shall show directly that the lemma is true for =, for which For log x 0 we shall show x = e x( e e x ) log e x (3.4) x e x < 0 that is, e x( e ) e x log e x <0 (3.5) Differetiatio shows that the left-had side of (3.5) is icreasig i x for x 0. Thus we shall show that for x = 0 iequality (3.5) holds, that is, that e log < 0, which is equivalet to e log 4 < 0, which clearly holds. Now, for x>0 the iequality log e x >e x e x holds. Substitutig this i (3.4) we have x < e x( e e x e x) (3.6) We shall show that the right-had side of (3.6) is less tha e x /, which is equivalet to e e x e x < (3.7)
Optimal Two-Choice Stoppig 357 Now the left-had side of (3.7) is decreasig i x: thus it suffices to show (3.7) for x = 0, where the iequality simplifies to e <, which clearly holds. Thus the lemma holds for =. Suppose the lemma holds for. We shall show that it holds for. By (3.3), for x log h x a = h x e h x a (3.8) We show that, for x log, x a < e x / = e x by a Taylor expasio of h x a. Note that h x = e x / e x h x = e x / e x > 0 thus, for some 0, Thus, by (3.8) ad (3.9), e x a h x a = h x a e x a > h x a e x (3.9) ex x a < x e h x a a e x < x < x e x e x = x e x < e x e x e x a e x e x ( ) e x where the secod iequality uses h x > h x by Lemma 3., the third iequality uses log y >y y for 0 <y<, ad the last iequality uses the iductio hypothesis. Thus we must show that for x log we have <, e x ad hece it is sufficiet to show <. But 3/ / > ; 8 hece it is sufficiet to show that <, or equivaletly that < 3/ 8 4, which holds for. 4. PROOF OF THEOREM. Lemma 4.. For some costat A q, let q be a cotiuous ad strictly mootoe icreasig fuctio i the iterval A q such that for all y A q the itegral q x e x dx is fiite. Further, defiig y Q y = y q x e x dx q y e y (4.)
358 Goldstei ad Samuel-Cah suppose Q A q >0. The, lim Q y = (4.) y Q y is mootoe decreasig, ad there exists a uique value A q such that Q = 0. Proof. The assumptio that the itegral i (4.) is fiite ad q is icreasig implies that q y e y 0asy ; thus (4.) holds. The fuctio Q is differetiable with dq y /dy = q y e y < 0; thus Q is mootoe decreasig. Sice Q A q >0, Q y is cotiuous, ad egative for all y sufficietly large. Hece the root exists ad is uique i A q. Theorem 4.. Let A q ad q be as i Lemma 4.. The, there exists 0 such that for ay r 0 ad r A q, the sequece for r is well defied by the recursio ( q = q ) e q x e x dx a (4.3) ad satisfies lim = where is the root of (4.) whose existece ad uiqueess i A q is guarateed i Lemma 4.. Proof. First, rewrite (4.3) as q q = Q ( ) a (4.4) Note that for all, ad that 0 < a < (4.5) ( ) Q c a is positive ad decreasig i with limit 0 for all c, ad is decreasig i ad egative for all sufficietly large with limit 0 for c>. We show that for ay ad with A q < < <, for all sufficietly large, is well defied ad < < ; clearly the theorem follows. Let A q < < < be give, ad let 0 be so large that for all 0 ( ) Q A q a <q q (4.6a) ( ) ( ) a <q q (4.6b)
Optimal Two-Choice Stoppig 359 ( ) ( ) Q a < 0 (4.6c) a < q q (4.6d) We first show that if A q < < < for 0, the is well defied ad satisfies < < ; thus the sequece remais i the iterval for all 0. We show this fact by cosiderig the followig cases. Case A is < By (4.4), (4.5), ad (4.6a), ad the fact that q is icreasig ad Q is decreasig, q < q Q < q Q A q ( ) a = q ( ) a < q q q q (4.7) thus exists uiquely by the strict mootoicity of q ad satisfies Case B is < < < < < There are two subcases B, Q ( ) a > 0 which may happe for small, ad B, I Subcase B, by (4.6b), Q ( ) a 0 (4.8) ( q < q < q Q ( ) < q a < q ) a = q ( ( )) q q <q so agai is well defied ad < <. Subcase B ca be combied with Case C.
360 Goldstei ad Samuel-Cah Case C is < I this case, by (4.6a), ad i subcase B by (4.8), if exists, it must be smaller tha ; thus q > q, but also q = q Q ( ) a > q a > q q q > q (4.9) where the iequalities are justified by (4.), (4.6d), ad >, this last of which holds for Case C as well as for Case B, so i particular for subcase B. Thus agai exists ad < <. It remais to show that for ay r 0 ad ay startig value r A q c, is well defied ad will evetually eter the iterval. First, suppose r A q. The the sequece will be well defied ad start out mootoe icreasig, ad (4.7) ad its subsequet iequalities cotiue to hold as log as, ad for all such oe has <. There are two possibilities: Either (a) for some k the iequality < k < holds, i which case we have show that < < for all >k, or (b) the sequece is mootoe icreasig throughout with lim = 0, which ecessarily satisfies 0. We show that (b) leads to a cotradictio. Clearly Q 0 >0. By (4.4), q q > Q 0 thus for arbitrarily large ad m>, m q m q > Q 0 k= ( ) a m k k= ( ) k a k Now, the right-had side teds to ifiity as m ; thus the value q m must also ted to ifiity, cotradictig the fact that m. Now cosider a startig value r for r 0 satisfyig r <. By (4.6c) the sequece will be well defied ad decreasig, as log as /, ad (4.9) cotiues to hold; thus >. Agai there are two possibilities. Either (a) for some we have > >, i which case the theorem holds, or (b) the sequece is mootoe decreasig for all, with, ad thus the limit 0 exists, ad clearly satisfies Q 0 <0. We suppose (b) ad show that this leads to a cotradictio. By (4.4) ad (4.5), q q < Q 0
Optimal Two-Choice Stoppig 36 thus for m arbitrarily large, m q m q < Q 0 k= m m k= (4.0) k Now, the last summad o the right-had side of (4.0) coverges to a fiite limit, while the first term there teds to as m. Thus q m must also ted to, cotradictig the fact that q m q. Let H be give by (4.) with q replaced by h of (.); the chage of variable x = B u i the itegral i this defiitio of H shows that (.) is the equatio H B = 0, ad usig Lemma 4. we coclude that the solutio log e is uique. Let <A be some costat, ad defie h j x = h x e x j for A x< The by Lemmas 3. ad 3.3, for all j > j A = e A we have h x < h j x < h j x for A x< (4.) Also, sice there is some j 0 A j A such that for all j j 0 A, d h j x dx = ex e x e x j > 0 the fuctios h j x j j 0 A are strictly icreasig i A. Lemma 4.. Let H j x be defied as i (4.), with q x replaced by h j x. The, for all j j 0 A there exists a value j A such that H j j = 0, lim j = log e ad j lim h j j = h log e = log e (4.) j Proof. Sice H j x H x uiformly o A, i particular lim j H j A = H A > H log e = 0. Thus for all j>j 0 A the value j exists uiquely i A. Now (4.) follows from the uiform covergece of H j x ad h j x to H x ad h x, respectively, o A. Note (.5) ca be rewritte as W = h B = [ h B h y ] e y dy a (4.3) log whereas (4.3) ca be rewritte, with h istead of q (keepig the otatio), as h = log h h y e y dy a (4.4) Comparig (4.3) ad (4.4), we see that the oly differece betwee the two expressios is that i (4.3) the fuctio i the itegral depeds o, whereas i (4.4) this fuctio is fixed. We ca ow prove our mai result.
36 Goldstei ad Samuel-Cah Proof of Theorem.. We apply Theorem 4. to (4.4) for 0 with startig value 0 = B 0 as i (.4), where 0 is the value give by Theorem 4. for A ad h, after which recursio (4.4) is well defied. For all j>j 0 A let r j = 0 j, ad for r j, defie the sequece j through (4.4) with h replaced by h j, ad iitial value j rj = B rj. The by (4.), (4.3), ad (4.4), <B < j, ad thus the iequality h <h B = W < h j j holds for all >r j, otig that the right-had side of (4.4), say, is made larger by replacig h by a larger fuctio. Thus, as, log e = lim h lim if W lim sup W lim h j j = h j j (4.5) Now, by Lemma 4., if we let j, from (4.5), log e lim if W lim sup W log e from which (.8) follows, to which the theorem is equivalet. Remark 4.. The limitig oe-choice value ca be obtaied i a similar, but simpler way. Let V deote the sequece of oe-choice optimal values ad let W = V log. Sice V = E X V it follows that the W sequece satisfies (4.3) with q x = x ad = W. By Theorem 4., it therefore follows that lim W = W is the solutio of Q = 0, where Q y = y xe x ds ye y i.e., Q y = e y which implies W = 0. This clearly agrees with the more geeral result of Keedy ad Kertz (99); see (.9). Remark 4.. A measure of the limitig effectiveess of havig a secod choice is the value lim V V / EM V. It compares the relative advatage of havig two choices over havig oly oe choice, divided by the similar advatage for the prophet, whose value is EM. For the expoetial distributio we have lim V V EM V = log e = 0 7946 (4.6) where is the Euler costat. For the large subclasses of distributios of Types III ad II, treated i Assaf et al. (004, 006), the correspodig miimal values over all -values is (4.6) ad 0 7880, respectively. Thus the miimal savig i all the kow cases is ear 80%. Remark 4.3. The model cosidered here is where observatios arrive determiistically, oe per time uit. If istead observatios were to arrive accordig to a Poisso process with rate, various quatities that are approximate or asymptotic here become exact. For example, it is easy to see that V, the optimal
Optimal Two-Choice Stoppig 363 value for the oe-stop problem o the iterval 0, satisfies the differetial equatio V = The equatio ca be solved explicitly, givig 0 x V 0 e x dx V = log as compared to the approximate expressio (.9). Subject to solvig the correspodig equatios that give the optimal two-stop value, this approach may be carried out to yield results such as Theorem.; see Keedy ad Kertz (990) for use of the Poisso process settig i the oe-stop problem. Added i Proof: After the preset work was completed, the paper of Kühe ad Rüschedorf (00) came to our attetio. That paper treats the same problem as the oe here usig Poisso-approximatios, basig their results o their earlier detailed paper Kühe ad Rüschedorf (000). Detailed results are give for the domai of attractio e e x cosidered here, ad our cojecture, that (.5) holds for all F i this domai, is prove. ACKNOWLEDGMENTS The authors would like to thak the referees for their careful readig ad helpful commets, ad oe of them i particular for poitig out the coectio to ad simplificatio that occurs i the Poisso process settig (Remark 4.3). The secod author would like to ackowledge support by the Israel Sciece Foudatio, grat umber 467/04. REFERENCES Assaf, D., Goldstei, L., ad Samuel-Cah, E. (004). Optimal Two Choice Stoppig, Advaces i Applied Probability 36: 6 47. Assaf, D., Goldstei, L., ad Samuel-Cah, E. (006). Maximizig Expected Value with Two Stage Stoppig Rules, Radom Walks, i Sequetial Aalysis ad Related Topics, pp. 5, Sigapore: World Scietific Press. Keedy, D. P. ad Kertz, R. P. (990). Limit Theorems for Threshold-Stopped Radom Variables with Applicatios to Optimal Stoppig, Advaces i Applied Probability : 396 4. Keedy, D. P. ad Kertz, R. P. (99). The Asymptotic Behavior of the Reward Sequece i the Optimal Stoppig of I.I.D. Radom Variables, Aals of Probability 9: 39 34. Kühe, R. ad Rüschedorf, L. (000). Approximatio of Optimal Stoppig Problems, Stochastic Processes ad Applicatios 90: 30 35. Kühe, R. ad Rüschedorf, L. (00). O Optimal Two-Stoppig Problems, i Limit Theorems i Probability ad Statistics, Vol., I. Berkes, E. CWM, ad Csörgö, eds., pp. 6 7, Budapest: Jáos Bolyai Mathematical Society. Leadbetter, M. R., Lidgre, G., ad Rootzé, H. (983). Extremes ad Related Properties of Radom Sequeces ad Processes, New York: Spriger-Verlag.