Applied Probability Trust (24 March 2004) Abstract

Transcription

1 Alied Probability Trust (24 March 2004) STOPPING THE MAXIMUM OF A CORRELATED RANDOM WALK, WITH COST FOR OBSERVATION PIETER ALLAART, University of North Texas Abstract Let (S n ) n 0 be a correlated random walk on the integers, let M 0 S 0 be an arbitrary integer, and let M n =max{m 0,S 1,...,S n }. An otimal stoing rule is derived for the seuence M n nc, where c>0 is a fixed cost. The otimal rule is shown to be of threshold tye: sto the first time that M n S n, where is a certain nonnegative integer. An exlicit exression for this otimal threshold is given. Keywords: Correlated random walk; momentum; stoing rule; otimality rincile; linear difference euation AMS 2000 Subject Classification: Primary 60G40; 62L15 Secondary 60G50 1. Introduction Consider a layer owning a commodity whose rice rocess exhibits momentum in the following way: If the rice goes u at stage n, it will take another ste u at stage n + 1 with robability, orastedown with robability 1. Likewise, if the rice takes a ste down at stage n, it will take another ste down at the next stage with robability, or a ste u with robability 1. The layer may sto at any time, and sell the commodity for the highest rice that has occurred since the time of urchase. However, a fixed cost c>0 is incurred for each stage during which the commodity is held. When should the layer sell the commodity in order to maximize his exected return (that is, selling rice minus cost)? The rocess described above is usually referred to in the literature as a correlated random walk. More formally, let S 0 be an arbitrary integer and for n 1, define S n = S 0 + X X n,where{x n } n 0 is a Postal address: Mathematics Deartment, P.O. Box , Denton, TX , USA. 1

2 2 P. Allaart { 1, 1}-valued Markov chain with one-ste transition matrix [ ] 1 1 We can think of X 0 as the initial direction of the walk, since its value determines the robabilities of an u-ste or a down-ste at the first stage. To avoid degenerate cases, it will be assumed that 0 <,<1. Let M 0 be an arbitrary integer such that M 0 S 0,andforn 1define M n =max{m 0,S 1,...,S n }. The goal of this aer is to find a stoing rule τ that maximizes E(M τ τc). Fora random walk without correlation (the case + = 1), this roblem was considered by Ferguson and MacQueen ([4], 4). They showed that there exists a nonnegative integer such that the otimal rule is of the form: sto at the first time n for which M n S n. They also comuted the threshold for the secial case =1/2, obtaining = 1/(2c), where x denotes the greatest integer less than or eual to x. The resent article extends this result to random walks with correlation. Correlated random walks were introduced by Goldstein [6] to model diffusion in turbulent media. Other early work on the subject is due to Gillis [5] and Mohan [10]. Correlated random walks have subseuently been used to model such diverse henomena as a gambler s fortune [11], the behavior of a inball machine [12], and the growth of tree roots [7]. A related class of rocesses called directionally reinforced random walks was introduced by Mauldin, Monticino and von Weizsäcker [9]. In a directionally reinforced random walk, the robabilities of an u- or down ste deend not only on the direction of the most recent ste, but also on the number of successive stes the walk has just taken in that direction. In a recent aer, Allaart and Monticino [2] derived otimal single and multile stoing rules for directionally reinforced random walks in one dimension. This aears to be the first work to consider otimal stoing roblems for a class of rocesses exhibiting momentum. A subseuent article [1] treats the otimal stoing roblem forcorrelated random walks with a discount factor. In that aer, the analysis and the final results are somewhat tedious, due to the incomatibility of the linear motion of the walk and the geometric discount factor. In the current setting, however, the linear cost structure allows for a more elegant answer to the otimal stoing roblem: as in the case considered by Ferguson and MacQueen, it is otimal to sto at the first time n for which M n S n, for

3 Stoing a correlated random walk 3 some nonnegative integer. The calculation of this threshold is fairly straightforward, and follows from the solution of a simle system of linear difference euations. 2. Form of the otimal stoing rule For n 0, let Y n = M n nc, andletf n be the sigma algebra generated by the random trilets (M 0,S 0,X 0 ),...,(M n,s n,x n ). For s Z, m s and x { 1, 1}, define V (m, s, x) =sue[m τ τc M 0 = m, S 0 = s, X 0 = x], (2.1) τ where τ ranges over the set of all stoing rules adated to the filtration {F n } n 0 such that 0 τ< almost surely. Call a stoing rule otimal for the trile (m, s, x) if it attains the suremum in (2.1). The existence of an otimal rule deends on the magnitude of c relative to the drift δ of the walk, where δ = 2 ( + ). It can be shown that, indeendently of the initial values, E S n /n δ as n. (See, for instance, 4 of [2]. The argument given there is easily generalized to arbitrary first-ste robabilities.) Lemma 2.1. Assume c>δ.then (i) E(su n Y n ) <, and (ii) lim n Y n = almost surely. Conseuently, there exists an otimal stoing rule, and it is given by the Princile of Otimality: Sto at the first time n at which Y n = ess su E[Y τ F n ]. τ n Proof. Clearly, the initial conditions do not affect the asymtotic roerties of the walk, so without loss of generality assume that S 0 = M 0 =0 and X 0 = 1. Let T 1,T 2,... be the times at which the walk reverses its direction. That is, let T 0 =0,andfork 1, let T k =inf{n >T k 1 : X n = X Tk 1 }. Then the random variables (T k T k 1 ) k 1 are indeendent, and the distribution of T k T k 1 is geometric with arameter 1 if k is even, and 1 if k is odd.

4 4 P. Allaart Now define Z k = S T2k S T2k 2 c(t 2k T 2k 2 ), k =1, 2,... and let S n = n k=1 Z k, n IN. From the reresentation Z k =(T 2k T 2k 1 ) (T 2k 1 T 2k 2 ) c(t 2k T 2k 2 ) =(1 c)(t 2k T 2k 1 ) (1 + c)(t 2k 1 T 2k 2 ), it follows that the Z k s are i.i.d. Moreover, EZ 1 =(1 c)e(t 2 T 1 ) (1 + c)e(t 1 T 0 ) = 1 c 1 1+c 1 = 2 (δ c) (1 )(1 ) < 0. Since E Zk 2 <, Theorem 5 of Kiefer and Wolfowitz [8] imlies that E(su S n) <. Moreover, S n almost surely by the Strong Law of Large Numbers. Since for n = T 2k we have S n nc = S k, and since the rocess S n nc takes on its local maximum values at the oints n = T 2k 1, it follows that E(su(S n nc)) < and S n nc almost surely. Now define, for k 1, N k := inf{n : S n = k} (= if no such n exists). Assume that for a articular samle ath, S n nc. If N k < for all k 1, then M Nk N k c = S Nk N k c.andifn k = for some k, thenm n is bounded, so Y n since c>0. This establishes art (ii) of the lemma. Part (i) follows since Y n max 1 j n (S j jc), and so E(su Y n )=E(su(S n nc)) <. The remaining statements of the lemma follow from Theorem 4.5 of Chow et al. [3]. Corollary 2.1. If c>δ,therule τ =inf{n 0:M n = V (M n,s n,x n )} is otimal. Proof. The rocess (M n,s n,x n ) is a time-homogeneous Markov chain. Thus, on the set {M n = m, S n = s, X n = x}, ess su τ n E[Y τ F n ]=sue[m τ τc M n = m, S n = s, X n = x] τ n =sue[m τ n τc M 0 = m, S 0 = s, X 0 = x] τ n =sue[m τ τ c M 0 = m, S 0 = s, X 0 = x] nc τ 0 = V (m, s, x) nc. Alying Lemma 2.1 comletes the roof.

5 Stoing a correlated random walk 5 Next, define the short-hand notation Observe that E m,s,x [.] :=E[. M 0 = m, S 0 = s, X 0 = x]. V (m, s, x) m =sue 0,0,x [m (s + M τ ) τc] m τ =sue 0,0,x [{M τ (m s)} + τc], τ and hence the otimal rule (if one exists) deends on m and s only through their difference. We can therefore arbitrarily set m = 0, and consider the uantities W + (k) =V (0, k, 1), W (k) =V (0, k, 1) (k 0). It is clear that the functions W + (k) andw (k) are nonnegative and nonincreasing. Hence if we define + =inf{k 0:W + (k) =0}, and =inf{k 0:W (k) =0} (with the convention that inf( ) = ), then the otimal rule is: Sto the first time n 0 at which either M n S n + and X n =1,orM n S n and X n = 1. Moreover, the otimal values V (m, s, ±1) can be obtained from the functions W + (k) andw (k) through the identities V (m, s, 1) = W + (m s)+m, V (m, s, 1) = W (m s)+m. Lemma (2.2) Proof. It is to be shown that W (k) = 0 whenever k By conditioning on X 1 it can be seen that, for k 1, W (k) =max{0,w (k +1)+(1 )W + (k 1) c}. Now let k and suose that W (k) > 0. Then, using the definition of + and the fact that both W and W + are nonincreasing, W (k) =W (k +1)+(1 )W + (k 1) c = W (k +1) c W (k) c<w (k), a contradiction. Hence W (k) =0.

6 6 P. Allaart In view of Lemma 2.2, the first condition in the otimal rule above cannot occur before the second does, unless it occurs at time 0. As a result, the otimal rule is either τ 0(ifX 0 =1andM 0 S 0 + ), or else τ = σ( ), where for K 0, σ(k) =inf{n 0:M n S n K and X n = 1}. 3. Calculation of the otimal threshold The rest of the aer is devoted to calculating the exected returns from the rules σ(k), and maximizing over K. To this end, define for K 0andi 0, f + i (K) =E 0, i,1 Y σ(k), and f i (K) =E 0, i, 1 Y σ(k). Proosition 3.1. (i) For i K, f i c (K i)+ δ f i (K) = [ 1 (ii) For K {0, 1} and i 0, (K) =0.For0 i<k, 1 ( c ) [( ) K 1 δ 1 ( 3 1 ) ] c (K i) ( 1 f + i (K) =(i+1 c)/(1 ). ( ) ] i,, ) c(k 2 i 2 ), =. (3.1) For K 2, (fi+1 i+2 (K)+c)/(1 ), 0 i K 2 f + i (K) = [ i (K 2) f + K 2 (K)+ c ] c 1 1, i K 1. (Full exressions for f + i (K) are unenlightening, and in any case are not needed for the derivation of the otimal rule. They are therefore omitted.) Remark 3.1. One would exect the exressions for f i (K) ineuation (3.1) to be continuous at =. That this is indeed the case can be

7 Stoing a correlated random walk 7 seen by alying standard limit comutation techniues. The technical details are somewhat tedious, and are omitted here since continuity is not of aramount imortance in what follows. Proof of Proosition 3.1. Fix K 0, and let a i := f + i (K), and b i := f i (K). Then a i and b i satisfy the difference euations a i = a i 1 +(1 )b i+1 c, i 1 (3.2) b i =(1 )a i 1 + b i+1 c, 1 i<k (3.3) as well as the boundary conditions a 0 = (a 0 +1)+(1 )b 1 c, (3.4) b 0 =(1 )(a 0 +1)+b 1 c, (3.5) and Now consider three cases. b i =0, i K. (3.6) Case 1. K = 0. In this case, statement (i) is trivial. To see (ii), note that b i =0foralli 1, and so (3.2) and (3.4) imly that a 0 =( c)/(1 ) and, inductively, that a i =( i+1 c)/(1 ) fori 0. Case 2. K = 1. In this case, statement (ii) follows by the same argument as in Case 1. In articular, a 0 =( c)/(1 ), and substituting this into (3.5) yields the exression for b 0 = f0 (1) given in art (i) of the roosition. Case 3. K 2. We derive a second order difference euation for the b i s as follows. By (3.3), we have (1 )a i = b i+1 b i+2 + c, 0 i K 2. On the other hand, (3.2)-(3.5) imly that It follows that (1 )a i = b i +(1 )(b i+1 c), 0 i K 1. b i+2 ( + )b i+1 + b i =(2 )c, 0 i K 2. (3.7) Suose first that. Using the transformation d i = b i+1 b i + c δ, (3.8)

8 8 P. Allaart (3.7) can be written as d i+1 =(/)d i, 0 i K 2, so that d i =(/) i d 0 for i =0, 1,...,K 1. The value of d 0 is obtained by eliminating a 0 from (3.4) and (3.5). This yields d 0 = b 1 b 0 + c δ = c ( ) 1 δ ( c) (1 c) 1 ( ) 1 ( c ) = 1 δ 1, and hence, ( ) 1 ( c ) ( ) i d i = 1 δ 1, 0 i K 1. Using this exression and (3.8), it follows inductively that i 1 [ 1 ( c ) ( ) ν b i b 0 = δ 1 c ] ν=0 = ( c ) δ 1 1 (/)i 1 / Finally, since b K =0,weobtain b i = c (K i)+ ( ) [ 1 ( c ) ( ) K δ 1 δ 1 δ ( c ) i. δ ( ) ] i, 0 i K. Suose next that =. Letd i = b i+1 b i. Then (3.7) reduces to ( ) 1 d i+1 d i =2 c, 0 i K 2. The initial condition becomes d 0 =2c 1, and thus, ( ) 1 d i =2 ic +2c 1, 0 i K 1, and i 1 [ ( ) ] 1 b i b 0 = 2c ν +2c 1 ν=0 ( ) 1 = i(i 1) c +(2c 1)i.

9 Stoing a correlated random walk 9 Finally, since b K =0, ( ) 1 b i =(1 2c)(K i) c{k(k 1) i(i 1)} [ ( = ) ] ( ) 1 c (K i) (K 2 i 2 )c, 0 i K. This comletes the roof of statement (i) in Case 3. The exressions for f i + (K) in statement (ii) follow from (3.3) if 0 i K 2, and from iteration of (3.2) if i K 1 (recalling that b i+1 = 0 in the latter case). The following immediate conseuence of Proosition 2.1 states that if c δ, thestorulesσ(k) yield arbitrarily large returns if K is chosen large enough. This is intuitively lausible, since the average growth of the walk er time unit exceeds the cost for one observation. Corollary 3.1. If c δ, then V (m, s, x) = for all m, s, and x. Moreover, lim E m,s,x[m σ(k) σ(k)c] =. K Proof. Since c>0, the hyothesis imlies that >. Hence by (3.1), f i (K) as K. The same is then true for f + i (K). Theorem 3.1. Suose c>δ.then (i) V (m, s, x) < for all m, s, andx. (ii) The otimal rule is τ = { 0 if X 0 =1and M 0 S 0 +, σ( ) otherwise. (iii) =max{0, κ }, where ( 1 log / 1 κ = 1 ) c, if c δ ( ) 1 2c 1, if =, and κ denotes the smallest integer greater than or eual to κ.

10 10 P. Allaart (iv) inf{k 0: k+1 c}, if 1 + = inf{k 0: k ( 2) [1 + (1 )W/c] 1 }, if 2, where W := W + ( 2) = (f 1 ( )+c)/(1 ). (v) The otimal exected returns are W + (i) =f i + ( ) if i< +,and =0otherwise; and W (i) =f i ( ),fori 0. Proof. Notice that is the (common) value of K that maximizes each of the functions f i (K) (i 0). This value can be found by looking at the first-order differences of f := f0 (say). For the case, thisgives f(k +1) f(k) = c δ 1 ( c ) ( ) K 1 δ 1. Investigate the cases < and > searately to conclude that f(k + 1) f(k) is decreasing and eventually negative. (If <this follows since δ<0.) Hence f is maximized at the smallest K for which f(k +1) f(k). This ineuality reduces to ( ) K 1 1 c c δ if >,andto ( ) K 1 1 c c δ if <. In both cases, the exression for given in the statement of the theorem follows. On the other hand, if = we have ( f(k +1) f(k) =1 3 1 ) ( ) 1 c (2K +1)c, and the ineuality f(k +1) f(k) simlifies to K ( ) 1 1 2c 1. (3.9) This establishes the exression for in the case =. The exressions for + follow immediately from Proosition 3.1 and the observation that + is the smallest i for which f + i ( ) 0.

11 Stoing a correlated random walk 11 Remark 3.2. As earlier in Proosition 3.1, the exressions for κ in Theorem 3.1 can be shown to be continuous at =. The details are omitted. Corollary 3.2. If c, theruleτ 0 is otimal for any trilet (m, s, x) of initial conditions different from (0, 0, 1). Proof. Observe first that the hyothesis c imlies that c>δ,since δ = (1 )( + ) 2 > 0. Thus the otimal rule is given by Theorem 3.1. It suffices to show that 1, for then + =inf{k : k+1 c} =0. Suose first that >.Since(1 )δ, it follows that 1 1 c c δ 1 1 δ (1 ) (1 ) ( ) =. Hence art (iii) of Theorem 3.1 imlies that 1. The case <can be handled similarly. Finally, for =, the statement follows since ( ) 1 1 2c 1 ( ) = 1 2 2(1 ) 1, so(3.9)holdsforeveryk 1, and hence f(k) is decreasing for K 1. Remark 3.3. For the case of a classical symmetric random walk ( = = 1/2), Theorem 3.1 gives = 1/(2c) 1. Since there is no correlation between stes, it follows immediately that + = 1/(2c) 1 as well, though this seems difficult to verify from the exressions of Theorem 3.1. Ferguson and MacQueen [4] give the otimal threshold = 1/(2c) for this case. Note that if 1/(2c) is non-integer, the values of and coincide. If 1/(2c) is an integer, the thresholds disagree, but the roof of Theorem 3.1 shows that it is immaterial whether σ( )orσ( )isused. For, if K =1/(2c) 1, then (3.9) above holds with euality, and hence f0 (K +1)=f0 (K). Acknowledgement The author is grateful to an anonymous referee for ointing out a critical error in an earlier draft, and for several suggestions that imroved the resentation of this aer.

12 12 P. Allaart References [1] Allaart, P. C. (2004). Otimal stoing rules for correlated random walks with adiscount.j. Al. Prob. 41, [2] Allaart, P. C. and Monticino, M. G. (2001). Otimal stoing rules for directionally reinforced rocesses. Adv. Al. Prob. 33, [3] Chow, Y. S., Robbins, H. and Siegmund, D. (1971). Great exectations: the theory of otimal stoing. Houghton Mifflin, Boston. [4] Ferguson, T. S. and MacQueen, J. B. (1992). Some time-invariant stoing rule roblems. Otimization 23, [5] Gillis, J. (1955). Correlated random walk. Proc. Camb. Phil. Soc. 51, [6] Goldstein, S. (1951). On diffusion by discontinuous movements, and on the telegrah euation. Quart. J. Mech. 4, [7] Henderson, R. and Renshaw, E. (1980). Satial stochastic models and comuter simulation alied to the study of tree root systems. Comstat 80, [8] Kiefer, J. and Wolfowitz, H. (1956). On the characteristics of the general ueueing rocess, with alication to random walk. Ann. Math. Statist. 27, [9] Mauldin, R. D., Monticino, M. G. and Von Weizsäcker, H. (1996). Directionally reinforced random walks. Adv. Math. 117, [10] Mohan, C. (1955). The gambler s ruin roblem with correlation. Biometrika 42, [11] Mohanty, S. G. (1966). On a generalized two-coin tossing roblem. Biometrische Z. 8, [12] Renshaw, E. and Henderson, R. (1981). The correlated random walk. J. Al. Prob. 18,