STOPPING SIMULATED PATHS EARLY

Transcription

1 Proceedings of the 2 Winter Siulation Conference B.A.Peters,J.S.Sith,D.J.Medeiros,andM.W.Rohrer,eds. STOPPING SIMULATED PATHS EARLY Paul Glasseran Graduate School of Business Colubia University New Yor, NY 27, U.S.A. Jerey Stau School of Operations Research and Industrial Engineering Cornell University Ithaca, NY 4853, U.S.A. ABSTRACT We provide results about stopping siulation paths early as a variance reduction technique, adding to our earlier wor on this topic. The proble of pricing a financial instruent with cashflows at ultiple ties, such as a ortgage-baced security, otivates this approach, which is ore broadly applicable to probles in which early steps are ore inforative than later steps of a path. We prove a liit theore that deonstrates that this relative inforativeness of siulation steps, not the nuber of steps, deterines the effectiveness of the ethod. Next we consider an extension of the idea of stopping siulation paths early, showing how early stopping can be rando and depend on the state a path has reached, yet still produce an unbiased estiator. We illustrate the potential effectiveness of such estiators, and describe directions for future research into their design. INTRODUCTION In a standard siulation for a purpose such as pricing a financial derivative, every siulated path reaches each of a fixed nuber of tie steps. We consider siulations that estiate an expected total reward, for concreteness focusing on a su of discounted cashflows that a derivative security pays. When discounted cashflows at early steps are ore iportant than those at later steps, devoting equal coputational resources to siulation at early steps and later steps sees to be a suboptial allocation of resources. An exaple is a ortgage-baced security, whose onthly cashflows are the su of all payents ade on a pool of ortgages. These payents are noinally constant, but the value of discounted cashflows tends to decline because of the tie value of oney and because soe hoeowners prepay their ortgages, thus exiting the pool. In previous wor, we showed how to find a policy of early stopping, with soe paths scheduled to terinate before the final tie step, that iniizes the variance of an estiator with a fixed coputational budget Glasseran 38 and Stau 2b. In Section 2, we review the foralis and ain results of the previous paper, including the use of the ethod for soe probles to which it is seeingly inapplicable, such as those with only a terinal cashflow. Section 3 contains a new theore, which proves that the variance reduction obtained by early stopping depends not on the nuber of tie steps, but on the shape of the covariance atrix of discounted cashflows. An extension of early stopping occupies Section 4: here we introduce a ethod where the decision to stop is randoized and depends on the current state, rather than being planned in advance. We then suggest directions for future research into its effective use. 2 REVIEW OF EARLY STOPPING This section presents without proof a result of Glasseran and Stau 2b. Motivated by the proble of pricing a ortgage-baced security, we consider a siulation to estiate the expectation of a su X X, interpreting the ters X,...,X as discounted cashflows. When soe paths stop early, there are n paths that reach step, producing the observations X,...,X n. The nubers of paths reaching each step ust satisfy a onotonicity constraint: n n +. Let the cost of step be c and the total coputational budget be C. The budget constraint is c n C. The obvious estiator of the ean µ E[X] is now ˆµ n n i X i We want to iniize over all feasible choices of n,...,n the variance of ˆµ, whichisvar[ˆµ] v /n where v Var[X ]+2Cov X, j+ X j.

2 The variance coponents v,...,v on which the solution is based are typically unnown, but they can be estiated fro a set of initial siulated paths, and the optial resource allocation applied to the rest of the siulation. The resource allocation proble is in v n s.t. c n C 2 and n n +,,..., The solution to proble 2 involves the convex hull of a graph deterined by the costs c and the variance coponents v. Define the partial sus and the graph V v j and C j j V {C, V,...,}. Let the function V be the upper convex hull of this graph, and let V denote V C. The solution is based on the slopes u v /c where v is the increent V V. Here we restate Theore of 2b. Theore. The solution to the resource allocation proble 2 is n ν u ν c u C 2 c j The other approach is to study new rando variables X,...,X that satisfy the following two properties. At step of the siulation, X can be coputed. The su X X X. The estiator based on the new rando variables has the sae expectation µ, but different variance coponents v,...,v, which ay render early stopping ore effective. The optial choice of X,...,X would satisfy j X j E[X F ], but of course this conditional expectation is unnown. However, there are any probles for which we have a decent approxiation to the conditional expectation E[X F ], and this enables early stopping to provide ore variance reduction. It is this perspective to which we will return in Section 4 as we develop ethods of rando early stopping. 3 CONTINUOUS LIMIT This section shows that the shape of the covariance atrix of discounted cashflows deterines the solution and effectiveness of the resource allocation proble 2. First define a sequence of resource allocation probles with convergent shape as follows. Let v be a positive, continuous function on [, ],andletv t vs ds. The sequence of probles is indexed by size : intheth { proble, C is the budget and V is the point set C }, V,...,, where C C V V / V t v s ds v t v[t]/ and the ratio of optial variance to standard variance is R i j v i c j u j /u i i j v i c j. We go on to describe two ethods that can enhance the effectiveness of early stopping and ae it ore broadly applicable. The first involves the statistical perspective on issing data. Viewing the siulation as an experient, the data table has any issing observations X i when path i stops before reaching step. The theory of issing data leads to estiators which are superior to ˆµ because they tae advantage of dependence between the variables X,...,X and use inforation about the ore frequently observed variables to learn about those less frequently observed. 39 The th proble has variance coponents v v // and equal costs c C /. Let V be the upper convex hull of V and V be the upper convex hull of V. We will prove in the leas in the Appendix that this sequence of probles has convergent shape, as described by V, and that the upper convex hulls V and their slopes v also converge respectively to V and its derivative v. Then Theore 2 asserts that this sequence of probles has solution and effectiveness that also converge. To ae this precise, we define v n t t v s ds 3

3 and F v v s ds s v s ds. 4 These are the quantities of interest. Lea. The th proble of for 2 has solution given by n n / and its optial variance objective is F. Proof. Because V is continuous and piecewise linear, so is its upper convex hull V,andv is piecewise constant. We have v v // just lie v v //. By Theore, the solution to the th proble is n v νc C v /c j c j v j v / j v j /. Because v is piecewise constant, n is indeed equal to n t as defined in equation 3 for t [ /, /. The variance objective of the th proble is v n v v j j v v j j v j j v v v v and again because v is piecewise constant, this equals F as defined in equation Also define nt and F as in equations 3 and 4, but without the superscript. Now we can state the ain theore, whose proof, involving any leas, is deferred to the Appendix. Theore 2. For t,, li n t nt; also li F F. With no early stopping, the variance objective of the th proble would be V, which is the -step Rieann approxiation to the integral vt dt; these converge because we assued v is continuous on the copact set [, ]. Therefore the convergence of the optial variance objectives F to F is equivalent to convergence of the variance reduction ratios F /V. The next theore states that these liits of solutions to discrete optiization probles are also the solutions to a continuous optiization proble that is a liit of the discrete probles. Its proof is siilar to that of the discrete result Theore of 2b, but sipler and ore elegant. Theore 3. If v is continuously differentiable, then the liits n and F are the solution and objective of the calculus of variations proble vt in nt dt s.t. nt dt and n nondecreasing and continuously differentiable on,. Proof. Because n is differentiable, the onotonicity constraint is n, so the Lagrangian integrand function is vt/nt + νnt + λtn t. The Euler-Lagrange equation is then vt/nt 2 + ν λ t. Copleentary slacness is λtn t. We will show that ν v t dt v nt t ν λt V t V t nt 2 satisfy these conditions as well as the constraints. Monotonicity holds because the slope v of the upper convex 2

4 hull V ust be nonincreasing. Because v is positive, so is v, and as it is continuously differentiable, n ust also be continuously differentiable on,. The budget constraint nt dt is satisfied because the nuerator of nt integrates to ν. As for copleentary slacness, where V t V t, λt. It reains to consider intervals [a, b] where V and V are equal at the endpoints but not on the interior. Here the upper convex hull V ust be linear, i.e. its second derivative v is zero, so n is also zero, and copleentary slacness is satisfied. Turning to the Euler-Lagrange equation,we differentiate λ to get λ t v t vt nt 2 ν vt nt 2 2λtn t nt 2V t V tn t nt 3 By copleentary slacness, the third ter is zero, and the Euler-Lagrange equation is verified. The assuption that v be continuously differentiable is technical: it siplifies the use of the calculus of variations. Liewise the constraint that n be continuouslydifferentiable does not see to be central to the result. 4 STATE-DEPENDENT STOPPING a new a probability easure P by P [S + Q S s] psp[s + Q S s] for any event Q and state s, P [S + S s] ps for any state s, and P [S + S ]. That is, at each step of the siulation there is statedependent probability ps of entering an absorbing ceetery state, but if that does not happen, the transition probabilities are the sae. Define the indicator variable A {S } which has the value when a path is still alive. This is the opposite of the approach of Glasseran and Stau 2a, where we considered conditioning on the event that the state vector not enter a real ceetery state, for instance, that a barrier option not noc out. There we introduced a schee to reduce estiator variance, despite doing ore wor per path, by replacing the indicator variable A with a lielihood ratio The early stopping ethodology of Section 2 succeeds when the unconditional variance coponents v are decreasing. It allocates ore coputational resources to early steps at which their expenditure produces ore variance reduction. Next we investigate how it ight be possible to allocate resources on the basis of conditional variance. This is potentially uch ore effective because the conditional variance of future cashflows often varies greatly across the possible states at one tie. For instance, if a barrier option gets noced out, then it is certain that all future cashflows are zero, and it is obvious that a siulated path should stop at this step. More generally, when a path reaches a state with low conditional variance, it ight be advantageous to do less wor in siulating the rest of the path, which will provide little inforation about the expectation being estiated. But how can a lesser but positive aount of wor be done conditional on the current siulated state, when the only choice can be to siulate the next step of this path or not? One possibility is to randoize the decision whether or not to continue siulating this path, with the probability of continuation positively related to the benefit of continuation. We foralize such a schee by introducing a new ceetery state to the state space. Fro the probability easure P for the state vector path S,...,S,wedefine 32 L j ps j which expresses the probabilitity of surviving up to step given the path S,...,S. Here the goal is to achieve variance reduction, despite increased variance per path, by doing less wor per path and thus running ore paths with a fixed coputational budget. Let F be the siga-algebra generated by S,...,S and let A be the event {A }. Lea 2. The Radon-Niody derivative of the probability easure P with respect to the restriction of P to A which is erely a easure relative to F is /L,i.e. E [XA /L ]E[X] for any F -easurable rando variable X such that E[X] exists and is finite. This lea strongly resebles well-nown results of iportance sapling, and the technicalities of its proof are analogous to those of Lea of 2a. The state-dependent probability of continuation ps controls how uch expected wor will be done on the rest

5 of a path which has reached state S. We can construct an unbiased estiator, using the lielihood ratio L,even though the siulation is less liely to reach soe states than others. States which are unliely tend to be reached with a low value of L, and receive a correspondingly large weight. Let τ ax{ A } be the rando step at which the siulated path stops. If it is less than, the nuber of steps, there is early stopping, and we have not generated the total su of discounted cashflows X X,but we have learned soething about it. Let Y τ represent a guess at the expected value of X given this inforation; this guess could be a sophisticated approxiation based on nowledge about the siulation proble, or as siple as τ X, or even just zero. That is, for each value of, Y Y S,...,S is a deterinistic function of the path to date, which is generally not equal to the unnown conditional expectation E[X F ]. However, we require Y X. An estiator of E[X] under P is X Y τ L τ τ j Y j. ps j L j The intuition is that Y τ is our best guess at what X would be if we were to continue the siulation rather than stopping at step τ. This guess deserves a weight /L τ to ae up for the hazard of early stopping before reaching this step. To avoid bias, we ust subtract soething to ae up for the guesses that we would have ade had we stopped at earlier steps. Although this way of writing the estiator is ost intuitive, another expression is ore useful for theory. Let Y denote. Lea 3. The estiator X also equals j A j L j Y j Y j. Proof. The indicator function for the event that the path stops at step j is A j A j+.thatis,a τ A τ+ while j τ iplies A j A j+. Therefore X Y τ L τ τ j Y j ps j L j Y τ τ + Y j L τ L j L j+ j j A j L j Y j Y j. 322 Using this, we can prove the ey theore. Theore 4. The estiator X is unbiased under the P siulation schee, i.e. E [X ]E[X]. Proof. Using the forulation of Lea 3, E [X ] E A j Y j Y j L j j E [Y j A j /L j Y j A j /L j ] j E[Y j ] E[Y j ] j E[Y ] E[Y ]E[X]. The third equality is justified by Lea 2 since Y j and Y j are both F j -easurable. This approach is closely related to the ethodology of introducing new variables X,...,X, described at the end of Section 2. Just as there, the optial choice of Y j is the unnown conditional expectation E[X F j ], and in practice one ust attept to use nowledge about the siulation proble to construct a good approxiation. Here we also face two ore challenges: finding a good randoized stopping policy P, and designing a coputationally efficient algorith. The proble of finding the variance-iniizing P given a policy of guesses Y,...,Y appears difficult. However, preliinary investigations suggest that the optial continuation probabilities ps are large when proceeding to step + fro state S yields a large reduction in the su of the conditional variance of reaining discounted payoffs and the squared bias of the guess for the value of reaining discounted payoffs. As before, these quantities are unnown, but a decent guess at the produces an unbiased estiator with reduced variance that is erely suboptial in the sense of not providing the axiu possible variance reduction. 5 CONCLUSIONS In this paper we provided further analysis and an extension of the idea of stopping siulated paths early as a variance reduction technique. We proved that when paths stop early independent of the state reached, the extent of variance reduction depends on the shape of the covariance atrix of discounted cashflows, not on its size. The ethod is appropriate for probles where siulation produces ore valuable inforation at early tie steps than later ones. We also showed how to construct an unbiased estiator when the decision to stop paths early is rando and depends on

6 the state reached. This opens up the possibility of greater variance reduction, if one can avoid doing wor siulating the tails of paths when they are unliely to produce uch valuable inforation. These advances increase our ability to use our nowledge about the structure of siulation probles to apply coputational resources efficiently. ACKNOWLEDGMENTS This research was supported in part by NSF grant DMS74637, an IBM University Faculty Award, and an NSF graduate research fellowship. APPENDIX: PROOF OF THEOREM 2 Lea 4. The sequence of functions V converges uniforly to V. Proof. Each V t is a Rieann approxiation to the integral V t. Since v is continuous on the copact set [, ], it is uniforly continuous there, and it is Rieannintegrable. Unifor continuity says ɛ >, δ s, t [, ], t s <δ vt vs <ɛ Therefore by choosing s [t]/, it follows that t s < /, and ɛ >, δ t [, ], > /δ v t vt v[t]/ vt <ɛ This is unifor convergence of v to v. Next, ɛ >, δ t [, ], > /δ reasoning, switching the role of V and V,showsthat V t V t + ɛ. Cobining these, sup V t V t ɛ t [,] and thus V converges uniforly to V. Lea 6. There exist nonincreasing functions v and v such that V t v s ds and V t v s ds. Moreover, for t,, li v t v t. Proof. As concave functions, V and V have both left and right derivatives at every point in,. Ifs < t < u, focusing on V, V x V s li x s+ x s V t V x li x t t x V x V t li x t+ li x u x t V u V x u x The left and right derivatives are unequal only on a set of easure zero. For these assertions, see, for instance, Sundara 996, It follows that any function which at each point equals either one-sided derivative is nonincreasing and integrates to V. The sae holds for V, which indeed has at ost points where the onesided derivatives are unequal. Next, for the convergence of v to v. By Lea 5, we now that for any open interval a, b,, b v t v t dt A- a V b V b V a V a V t V t v s vs ds < tɛ ɛ That is, V converges uniforly to V. Lea 5. The sequence of functions V converges uniforly to V. Proof. To prove the convergence of V to V,define ɛ sup t [,] V t V t. By unifor convergence of V to V, ɛ converges to zero. Because V t is concave, V t + ɛ is concave. Further, using V t V t, V t + ɛ V t V t + V t V t Therefore, by definition of V t as the least concave curve above V t, V t V t + ɛ. The exact sae 323 The strategy is to show that { t, li inf v t <v t } or li sup v t >v t A-2 iplies the contrary of A-, and thus A-2 is false, i.e. for every t,, li inf v t v t li sup v t and thus li v t exists and equals v t. Suppose then that there is an a, such that li inf v a < v a. Define ɛ v a li inf v a >. By definition of li inf, there is a sequence { } such that for all, v a <v a ɛ/2. Because v is continuous and nonincreasing and a <,

7 there is a b a, such that <v a v b <ɛ/4. Both v and v are nonincreasing, so for t a, b, v t v t v b v a v b v a + v a v a > ɛ/4 + ɛ/2 ɛ/4 > Therefore b v t v t dt >b aɛ/4 > a for all, thus li inf b a v t v t dt >, contradicting A-. On the other hand, suppose there is a b, such that li sup v b >v b. Nowletɛ be li sup v b v b >, and there is a sequence { } such that for all, v b >v b + ɛ/2. This tie there is an a, b such that <v a v b <ɛ/4, and we get for t a, b v t v t v a v b v a v b + v b v b < ɛ/4 ɛ/2 ɛ/4 < bounded above on the whole interval [, ]. Either way, v is bounded above, and so is the function v. The sae reasoning applies to boundedness away fro zero. If there is a neighborhood of t onwhichv V, then v v, and if not, there exists t < such that V V t V V t tv. If there is a neighborhood of t onwhichv V,then v v v, and if not, for soe i we have v ji v j// i. Proof of Theore 2. This proof frequently relies on Lea 7. The functions v and v are bounded away fro zero, so we can tae square roots, which are theselves bounded away fro zero. Where v v,trivially v v. Since these are also positive and bounded above, by the Bounded Convergence Theore, v s ds v s ds. Consequently, n converges to n on,. Alsov / v v/ v because the denoinators are bounded away fro zero, and the nuerators and denoinators converge separately. With nuerators positive and bounded above, and denoinators bounded away fro zero, these fractions are positive and bounded above. So again we can apply the Bounded Convergence Theore, and F converges to F. for all, thus li sup b a v t v t dt <, contradicting A-. Lea 7. The functions vt, v t, v t, and v t are bounded above and bounded away fro zero in and t. Proof. Because v is positive and continuous, it is both bounded above and bounded away fro zero on [, ]. By the construction of v, its values are values of v, soit too is both bounded above and bounded away fro zero. Since v and v are nonincreasing and positive, we can focus on t for proving their boundedness above and on t for proving their boundedness away fro zero. If there is a one-sided neighborhood of t on which V V,thenv v. If not, then there exists a point t > suchthatv t V t tv. Either way, v is finite and the function v is bounded above. Liewise, if there is a neighborhood of t onwhich V V,thenv v v. If not, there exists a point i/ > such that i/v V i/ V i/ i/ v s ds i v j/ j So v is a Rieann average of v on the interval [, i/, and this is bounded above in because v is 324 REFERENCES Glasseran, P., and J. Stau. 2a. Conditioning on onestep survival for barrier option siulations. Operations Research, forthcoing. Glasseran, P., and J. Stau. 2b. Resource allocation aong siulation tie steps. Woring paper, Graduate School of Business, Colubia University, New Yor, New Yor. Sundara, R. K A First Course in Optiization Theory. New Yor: Cabridge University Press. AUTHOR BIOGRAPHIES PAUL GLASSERMAN is the Jac R. Anderson Professor and chairan in the division of Decision, Ris, and Operations at Colubia Business School, and holds a secondary appointent in the Departent of Industrial Engineering and Operations Research. He currently serves as departental editor of Manageent Science for Stochastic Models and Siulation. His e-ail address is <pg2@colubia.edu> and web address is < JEREMY STAUM is a Visiting Assistant Professor in the School of Operations Research and Industrial Engineering at Cornell University. He received his Ph.D. fro Colubia University in 2. His research interests include variance reduction techniques and financial engineering.