MIN-MAX REPRESENTATIONS OF VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS AND APPLICATIONS IN RARE-EVENT SIMULATION

Transcription

1 MIN-MAX REPRESENTATIONS OF VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS AND APPLICATIONS IN RARE-EVENT SIMULATION BOUALEM DJEHICHE, HENRIK HULT, AND PIERRE NYQUIST Abstract. In this paper a duality relation between the Mañé potential and the action functional is derived in the context of convex and state-dependent Hamiltonians. The duality relation is used to obtain min-max representations of viscosity solutions of first order evolutionary Hamilton-Jacobi equations. These min-max representations naturally suggest classes of subsolutions of Hamilton-Jacobi equations that arise in the theory of large deviations. The subsolutions, in turn, are good candidates for designing efficient rare-event simulation algorithms. 1. Introduction The motivation for this paper comes from the challenging problem to efficiently compute probabilities of rare events by stochastic simulation. Examples of such events include the probability that a diffusion process leaves a stable domain, voltage collapse in power systems, the probability of a large loss in a financial portfolio, the probability of buffer overflow in a queueing system, etc. See, e.g., [1, 29, 33] and references therein for numerous examples. For rare events the standard Monte Carlo technique fails because few particles will contribute to the computational task, due to the rare event not occuring, leading to a large relative error. To reduce the variance a control mechanism that forces particles towards the region associated with the rare event must be introduced. To obtain an unbiased estimator a weight is attached to each particle and the estimator is the sum of the weights of all the particles that end up in the specified region of the state space. The design of the controlled simulation algorithm must not only force particles towards the region in question, but also keep the associated weights under control. Examples of such techniques include importance sampling and multi-level splitting, see [1, 33], as well as genealogical particle methods, see [11]. Traditionally the design of rare event simulation algorithms are based on mimicking the large deviation behavior. More precisely, whenever a large deviation result is available that gives the exponential decay rate of probabilities of rare events and the most likely path to the rare event, the idea is to construct the control mechanism so that the system tends to follow this the most likely path (to the rare event). Although this approach has MSC21 subject classifications. Primary 35D4, 35F21; secondary 65C5, 49L25. Key words and phrases. Hamilton-Jacobi equations, duality, rare-event simulation, importance sampling. Corresponding author (hult@kth.se) H. Hult s research is supported by the Swedish Research Council 1

2 2 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS turned out to be reasonably successful, there are examples where this simple heuristic fail, see [17, 3], and the design issue is delicate. More recently, it has been demonstrated that, in many models in applied probability, the construction of efficient rare-event simulation algorithms is intimately connected with solutions to partial differential equations of Hamilton-Jacobi type that arise in large deviation theory. Suppose that the rate function associated with the large deviations of a sequence of stochastic processes {X n (t); t [, T ] is of the form T t L(ψ(s), ψ(s))ds, where ψ is an absolutely continuous function and L is the local rate function, such that v L(x, v) is convex for all x R n. Then, the large deviations rate of the probability P t,x (X n (T ) / Ω), t < T, x Ω, where Ω is an open subset of R n, is given by Ū(t, x) = inf ψ { T t L(ψ(s), ψ(s))ds, ψ(t) = x, ψ(t ) / Ω where the infimum is taken over all absolutely continuous functions. Since Ū is the value function of a variational problem it satisfies, in the sense of a viscosity solution, a Hamilton-Jacobi terminal value problem of the form {Ūt (t, x) H(x, DŪ(t, x)) =, (t, x) [, T ) Ω, (1.1) Ū(T, x) =, x Ω, where H is the Fenchel-Legendre transform of L, see e.g. [26]. In the context of importance sampling the connection between efficient simulation algorithms and certain subsolutions of the Hamilton-Jacobi equation is established in [18, 19, 16, 15]. In addition, [36] studies the use of solutions (assumed smooth enough) of Hamilton-Jacobi equations for designing efficient simulation algorithms. See also [13, 14] for multi-level splitting and [12] for genealogical particle methods. The essence of the developed theory is, roughly speaking, that the design of efficient stochastic simulation algorithms for computing probabilities of rare events is equivalent to finding subsolutions of the associated Hamilton-Jacobi equation whose value at the initial point agree with the value of the viscosity solution. In this paper we develop a systematic approach to the construction of viscosity subsolutions, useful in rare-event simulation, that are based on a novel min-max representation of viscosity solutions to the associated Hamilton-Jacobi equation. We consider Lagrangians (x, v) L(x, p) that are convex in v. The main result, Theorem 2.1, proves a duality between Mañé s potential and the action functional and is briefly described in what follows. The Mañé potential at level c is given by the value of the variational problem S c (x, y) = inf ψ,t { t c + L(ψ(s), ψ(s))ds, ψ() = x, ψ(t) = y, x, y R n, where the infimum is taken over all absolutely continuous functions ψ : [, ) R n and t >, see [31]. Whenever it is continuous, y S c (x, y) is a viscosity subsolution of the

3 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS 3 stationary Hamilton-Jacobi equation H(y, DS(y)) = c, y R n, where H denotes the Fenchel-Legendre transform of L and D denotes the gradient. An object similar to the Mañé potential is the action functional given by { t M(t, y; x) = inf L(ψ(s), ψ(s))ds, ψ() = x, ψ(t) = y, t > x, y R n, ψ where the infimum is taken over all absolutely continuous functions ψ : [, t] R n. This action functional is a well-studied object in large deviations theory and control-theory, see for example [28, 27] and the references therein. In the weak KAM and dynamical systems literature it is often referred to as Mather s action functional - see the overview paper [32] and references therein - even though the functional was known well before the papers by Mather. From the definition of the Mañé potential it is elementary to show that S c (x, y) = inf{m(t, y; x) + ct. t> The main result of this paper, Theorem 2.1, shows that, in the one-dimensional setting, n = 1, and for all t < t L, the dual relation also holds: M(t, y; x) = sup c>c L {S c (x, y) ct, where t L is a time that depends on the Lagrangian and (x, y) and c L denotes the smallest c such that S c >. In addition, an example is given that illustrates why the duality may fail for t > t L. The duality result is used to derive min-max representations of viscosity solutions of various time-dependent problems. For the initial value problem { V t (t, y) + H(y, DV (t, y)) =, (t, y) (, ) R, V (, y) = g(y), y R n, the duality leads to a min-max representation of the form V (t, y) = inf x sup {g(x) + S c (x, y) ct, (t, y) [, t L ) R. c>c L The min-max representation may be viewed as a generalization, to state-dependent Hamiltonians, of the classical Hopf-Lax-Oleinik formula, which states that if H(x, p) = H(p), then the solution to the initial value problem is given by V (t, y) = inf x { g(x) + tl ( y x t See [6, 3] for further details and generalizations of Hopf-Lax representation formulas to some state-dependent Hamiltonians. Similar min-max representations are stated for terminal value problems, problems on domains, and exit problems. represented as Ū(t, x) = inf ). For instance, the viscosity solution sup y Ω c>c L { S c (x, y) c(t t), Ū to (1.1) can be

4 4 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS where S c is the Mañé potential associated with the Lagrangian L. The min-max representations naturally suggest families of viscosity subsolutions useful for the design of rare-event simulation algorithms for time-dependent problems with state-dependent Hamiltonians. Indeed, for any c > c L, y Ω and K sufficiently large, the function (t, x) S c (x, y) c(t t) K is the type of subsolution to (1.1) that can be used to design efficient algorithms. We illustrate the applications in rareevent simulation in detail for exit problems of small-noise diffusions and birth-and-death processes. The paper is organized as follows. In Section 2 the Mañé potential and the relevant action functional are introduced and the duality result is given. An example that illustrates the duality, and when it may fail, is provided. Background material on viscosity solutions of stationary first order Hamilton-Jacobi equations is given in Section 3. In Section 4 evolutionary Hamilton-Jacobi equations are introduced and a min-max representations for the initial value problem is obtained form the duality result in Section 2. Similar representations for terminal value problems, problems on domains and exit problems are also presented. In Section 5 a direct relation between the min-max representation and the Hopf-Lax-Oleinik formula is presented for state-independent convex Hamiltonians. In Section 6 it is shown how the min-max representation naturally suggests families of subsolutions appropriate for the design of efficient rare event simulation algorithms. Examples related to small-noise diffusions and birth-and-death processes are also provided. 2. Convex duality In this section the Mañé potential is introduced and its properties established for the specific setting under consideration. Next, the action functional, arising e.g. in large deviations and control theory, is presented and a duality relation between the action functional and the Mañé potential is derived. Throughout the paper we make the following assumption. Let the Langrangian L : R n R n R be a locally bounded measurable function that is convex in the second coordinate and let the Hamiltonian H be the Fenchel-Legendre transform of L. That is, By convex duality it follows that H(x, p) = sup{ p, v L(x, v). (2.1) v L(x, v) = sup{ p, v H(x, p). p 2.1. The Mañé potential. For c R, the Mañé potential at level c, originally introduced by Mañé in [31], is the function S c : R n R n R defined by { t S c (x, y) = inf c + L(ψ(s), ψ(s))ds, ψ() = x, ψ(t) = y, x, y R n, (2.2) ψ,t where the infimum is taken over all t > and absolutely continuous ψ : [, ) R n. Since L is locally bounded it follows that S c (x, y) <, for all x, y R n and c <. It is possible that S c is identically for small c. Indeed, if L(x, v) = 1 2 v 2 and c <,

5 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS 5 then it follows from the definition (2.2) that S c (x, y) = for all x, y R n. Next, some elementary properties of S c are established. Proposition 2.1. The following properties hold. (i) For each x, y R n, the function c S c (x, y) is nondecreasing. (ii) For each c R, the function (x, y) S c (x, y) satisfies the triangle inequality: S c (x, z) S c (x, y) + S c (y, z), x, y, z R n. (2.3) (iii) If S c (x, y ) = for some x, y R n and c R, then S c (x, y) = for all x, y R n. (iv) If S c >, then S c (x, x) =, for each x R n. Proof. (i) follows immediately from the definition of the Mañé potential. (ii) To prove the triangle inequality, note first that if S c (x, z) =, then there is nothing to prove. Suppose S c (x, z) >. Then S c (x, y) > and S c (y, z) > as well, for otherwise, if S c (x, y) =, then there exists, for each N >, a t N > and an absolutely continuous path ψ N with ψ N () = x and ψ N (t N ) = y such that S c (x, y) tn c + L(ψ N (s), ψ N (s))ds N. Let τ > and ϕ be any absolutely continuous path with ϕ() = y and ϕ(τ) = z and τ c + L(ϕ(s), ϕ(s))ds =: C <. Then, by concatenating ψ N and ϕ as it follows that S c (x, z) ψ N (s)i{ s t N + ϕ(s t N )I{t N < s t N + τ tn c + L(ψ N (s), ψ N (s))ds + τ c + L(ϕ(s), ϕ(s))ds N + C. By sending N it follows that S c (x, z) =, which is a contradiction. Consequently, S c (x, y) >. A similar argument shows that S c (y, z) >. To proceed with the proof of the triangle inequality, take an arbitrary ɛ >, and select t 1, t 2 > and absolutely continuous paths ψ 1, ψ 2 with ψ 1 () = x, ψ 1 (t 1 ) = y, ψ 2 () = y and ψ 2 (t 2 ) = z such that S c (x, y) S c (y, z) Concatenate the two trajectories by t1 t2 c + L(ψ 1 (s), ψ 1 (s))ds ɛ 2, c + L(ψ 2 (s), ψ 2 (s))ds ɛ 2. ψ(s) = ψ 1 (s)i{ s t 1 + ψ 2 (s t 1 )I{t 1 < s t 1 + t 2.

6 6 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS It follows that S c (x, y) + S c (y, z) = t1 + t2 t1 +t 2 c + L(ψ 1 (s), ψ 1 (s))ds S c (x, z) ɛ. c + L(ψ 2 (s), ψ 2 (s))ds ɛ c + L(ψ(s), ψ(s))ds ɛ Since ɛ > is arbitrary the triangle inequality follows. (iii) follows from the triangle inequality. To prove (iv), take x R n and let ɛ >, h > be such that h(c + L(x, )) < ɛ and ψ(s) = x for each s h. By definition of the Mañé potential S c (x, x) h(c + L(x, )) < ɛ. Since ɛ > is arbitrary it follows that S c (x, x). The reverse inequality, S c (x, x), follows from the triangle inequality The action functional. For any x R n and (t, y) (, ) R n, let M be the action functional { t M(t, y; x) = inf L(ψ(s), ψ(s))ds, ψ() = x, ψ(t) = y, (2.4) ψ where the infimum is taken over all absolutely continuous ψ : [, ) R n. M is the action functional of Mather, see [32], viewed as a function of (t, y) The duality theorem. From the definition (2.2) of the Mañé potential it follows immediately that S c (x, y) = inf{m(t, y; x) + ct. t> In the one-dimensional case, n = 1, the dual relationship holds for all t not too large. Let c L denote the infimum over all c such that S c >. Theorem 2.1. Assume (2.1). For each x, y R n and c R, For n = 1, x, y R and t < t L = lim c cl c+ S c (x, y), S c (x, y) = inf{m(t, y; x) + ct, (2.5) t> M(t, y; x) = sup c>c L {S c (x, y) ct. (2.6) Moreover, if S c L >, then (2.6) holds for t = t L and, in addition, if either x A := {x : L(x, ) = c L or y A, then (2.6) holds for all t > and for t t L. M(t, y; x) = S c L (x, y) c L t,

7 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS 7 Proof. As mentioned above (2.5) follows from the definition (2.2) of the Mañé potential. Let us prove (2.6). By (2.5) it follows that sup{s c (x, y) ct = sup c>c L inf c>c L s> {M(s, y; x) c(t s) M(t, y; x), by taking s = t, so it is sufficient to prove M(t, y; x) sup c>cl {S c (x, y) ct. Take n = 1, x, y R and suppose that y > x and t L >. The argument for y < x is similar. The proof relies on the construction of a convex upper bound M(, y; x) of M(, y; x). Let M(t, y; x) = {ψ : [, t] R, abs. cont., ψ() = x, ψ(t) = y, ψ strictly increasing, t >, M = t> M(t, y; x), M 1 = {ψ 1 : ψ M, M 1 1 = {ξ : ξ(z) = d dz ψ 1 (z), ψ 1 M 1. Since each ψ M is absolutely continuous and strictly increasing, so is ψ 1. Moreover, ψ 1 (x) = and ψ 1 (t) = y for some t >. Consequently, each ξ M 1 1 is strictly positive a.e. and y ξ(z)dz = t for some t >. By a change of variables it follows that, x for all t >, { t M(t, y; x) inf L(ψ(s), ψ(s)) ds, ψ(t) = y ψ M = inf ξ M 1 1 { y x ( L z, 1 ) ξ(z)dz, ξ(z) y x ξ(z)dz = t =: M(t, y; x). By the convexity of v L(x, v) it follows that F : ξ y L(z, 1 )ξ(z)dz is convex and, x ξ(z) consequently, M(t, y; x) is the value of the convex optimization problem of minimizing the convex functional F over the convex set M 1 1, subject to the linear constraint G(ξ) := y ξ(z)dz = t. x For c R, let { t S c (x, y) = inf ψ M The proof proceeds by showing the relation To prove this claim, let c + L(ψ(s), ψ(s))ds, t > = inf ξ M 1 1 {F (ξ) + cg(ξ). M(t, y; x) = sup{ (x, y) ct, for all t >. (2.7) c R A = {(r, s) (, ) (, ) : r F (ξ), s = G(ξ), some ξ M 1 1, and note that A is convex. For any c R, the following representation holds: S c (x, y) = inf ξ M 1 1 {F (ξ) + cg(ξ) = inf (r,s) A { (1, c), (r, s). Take t >, let µ t = M(t, y; x) = inf ξ M 1 {F (ξ), G(ξ) = t and (1, c t) be the normal 1 vector to the tangent plane of A at (µ t, t). If A has a corner at (µ t, t) so that c t is not

8 8 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS unique, take the largest c t. By the choice of c t (1, c t ), (r, s) (µ t, t), (r, s) A. The inequality in the last display can be rewritten as µ t r + c t (s t) and consequently, µ t inf (r,s) A {r + c t(s t) inf ξ M 1 1 {F (ξ) + c t (G(ξ) t) inf ξ M 1 1 {F (ξ), G(ξ) = t = µ t. It follows that all inequalities in the last display are actually equalities and, in particular, S ct (x, y) c t t = inf { (1, c t), (r, s) c t t = inf { (1, c t), (r, s t) = M(t, y; x). (r,s) A (r,s) A This completes the proof of (2.7) and we conclude that We proceed by showing that M(t, y; x) M(t, y; x) = S ct (x, y) c t t. S c (x, y) = S c (x, y), c > c L. (2.8) To prove (2.8) take c > c L. The inequality S c (x, y) S c (x, y) is trivial, so it is sufficient to show S c (x, y) S c (x, y). Suppose, on the contrary, that there exist ɛ >, T > and an absolutely continuous path ψ with ψ() = x and ψ(t ) = y such that Let T c + L(ψ(s), ψ(s))ds S c (x, y) ɛ. s ψ (s) = sup ψ(u) = u s ψ(u) ds, and let B ψ = {s > : ψ (s) = ψ(s) and ψ(s) > be the points of increase of ψ. Then T c + L(ψ(s), ψ(s))ds = c + L(ψ(s), ψ(s))ds + c + L(ψ(s), ψ(s))ds B ψ Bψ c = c + L(ψ (s), ψ (s))ds + c + L(ψ(s), ψ(s))ds B ψ Bψ c S c (x, y) + c + L(ψ(s), ψ(s))ds. Consequently, B c ψ B c ψ c + L(ψ(s), ψ(s))ds ɛ. It follows that ψ has some excursion with negative cost. Repeating this excursion N times implies that S c (x, y) S c (x, y) + N c + L(ψ(s), ψ(s))ds S c (x, y) Nɛ. S c ψ Letting N implies S c (x, y) =, which contradicts c > c L. We conclude that S c (x, y) = S c (x, y) for c > c L. This proves (2.8).

9 Finally we show that VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS 9 To see this, suppose t < t L. By (2.8) it follows that t < t L implies c t > c L. (2.9) lim c c L c+ Sc (x, y) = lim c cl c+ S c (x, y) = t L > t. Since c S c (x, y) ct is concave with with supremum at c t, it follows that and, furthermore, that c Sc t (x, y) t c+ Sc t (x, y), t L > t c+ Sc t (x, y). Concavity of c S c (x, y) implies that c c+ Sc t (x, y) is non-increasing and we conclude that c t > c L. This proves (2.9). The proof of (2.6) is completed by combining (2.7), (2.8) and (2.9). Indeed, with x < y and t < t L, by (2.7) M(t, y; x) S ct (x, y) c t t. By (2.9) it follows that c t > c L and finally (2.8) shows that M(t, y; x) S ct (x, y) c t t S ct (x, y) c t t. This completes the proof of (2.6). Suppose S c L >. Then, by similar arguments, (2.8) holds for c c L and (2.9) can be restated as t t L implies c t c L, which implies that (2.6) holds for all t t L. To prove the final statement, take x A, y R and t t L. Since x A it follows that L(x, ) = c L and therefore, M(t, y; x) M(t t L, x; x) + M(t L, y; x) The proof is completed by showing Since S c L > it follows that t tl L(x, )ds + M(t L, y; x) = c L (t t L ) + M(t L, y; x). M(t L, y; x) = S c L (x, y) c L t L. M(t L, y; x) = sup c c L {S c (x, y) ct L. As c c+ S c (x, y) is non-decreasing and, by definition of t L, c+ S c (x, y) t L, c > c L it follows that the concave function c S c (x, y) ct L achieves its maximum over [c L, ) at c L. Consequently M(t L, y; x) = S c L (x, y) c L t L.

10 1 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS If instead y A and x R, then, similarly This completes the proof. M(t, y; x) M(t L, y; x) + M(t t L, y; y) M(t L, y; x) + t tl = M(t L, y; x) c L (t t L ) = S c L (x, y) c L t. L(x, )ds Remark 2.2. (i) The dual relation (2.6) is proved in the one-dimensional case. We believe the relation is true in the general case, n 2, but the arguments used in the proof are not easily extended to the multidimensional setting. (ii) Note that the set A = {x : L(x, ) = c L = {x : inf p H(x, p) = c L. In many of the examples considered in this paper the set A is identical to the projected Aubry set, see for example [23, 22]. The duality between S c (x, y) and M(t, y; x) can be given the following intuitive physical interpretation. The optimal t in the representation (2.5) is the optimal time it takes to move from x to y in a system with energy level c. Similarly, the optimal c in the representation (2.6) is the energy level at which it takes precisely time t to move from x to y along the most cost efficient path. The duality may fail if t is sufficiently large that it exceeds the optimal time in the definition of S c L (x, y). In convex analysis terms: c S c (x, y) is always concave but t M(t, y; x) is convex only for t < t L. The following example serves as an illustration. Example 2.3. Consider the Lagrangian with convex conjugate L(x, v) = v2 2 + x2 2, H(x, p) = p2 2 x2 2. (2.1) Suppose that x > and y > x. The claim is that for t sufficiently large, M(t, y; x) > U(t, y; x). The Euler-Lagrange equation associated with M is ψ(s) d ds ψ(s) =, with boundary conditions ψ() = x, ψ(t) = y. The solution to this ODE is given by and the associated time derivative is ψ(s) = y xe t 2 sinh(t) es + xet y 2 sinh(t) e s, ψ(s) = y xe t 2 sinh(t) es xet y 2 sinh(t) e s.

11 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS 11 It follows that the cost associated with this (optimal) trajectory is M(t, y; x) = 1 t (ψ(s) ψ(s) ) 2 ds = x2 + y 2 cosh(t) 2 sinh(t) xy sinh(t). The Mañé potential is given by y S c (x, y) = sign(z x) z 2 + 2cdz x ( = 1 y 2c + y 2 2 x ( )) 2c + y2 + y 2c + x 2 + 2c log. 2c + x2 + x To see this, Proposition 3.1 in the following section will be used. The stationary Hamilton- Jacobi equation takes the form DS c (x, y) 2 y2 2 2 = c, where D denotes gradient with respect to y. The expression for S c (x, y) is obtained by solving for DS c (x, y), integrating from x to y, and using that S c is the maximal subsolution, see Proposition 3.1(ii). From (3.3) it follows that c L and it is easy to check that equality holds in this case, that is, c L =. To illustrate that the duality of Theorem 2.1 does not necessarily hold for t > t L (x, y), we compare the derived expressions for M(t, y; x) and U(t, y; x) = sup c>cl {S c (x, y) ct for specific choices of x, y, and t; the optimization over c in U(t, y; x) is solved numerically. Figure 1 shows U and M as functions of t for x =.5, y = 1. Note that this is an arbitrary choice of x and y and it is easily checked that similar characteristics appear for other choices. A closer look at U and M for this particular choice of x and y reveals that the 1.2 U(t,y;x) M(t,y;x) Time t Figure 1. U(t, y; x) and M(t, y; x) for x =.5, y = 1.

12 12 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS duality ceases to hold at (roughly) t For the specific choice (2.1) of Hamiltonian H, S c (x, y) is clearly continuous in c and it is elementary to compute c+ S c (x, y) = ( / c)s c (x, y) as well as the (right-hand) limit as c c L : t L (x, y) = lim c+ S c (x, y) c cl ( y = 2 2c L + y x 2 2 2c L + x + log y + ) 2c L + y 2 2 x + 2c L + x 2 ( ) 1 + c L 2c L + y 2 + y 2c L + y 1 2 2c L + x 2 + x. 2c L + x 2 For the choice x =.5, y = 1 the limit is t L.6931, and we have already seen that M(t, y; x) > U(t, y; x) for t > This illustrates that the duality ceases to hold for t > t L (x, y). Next, consider the choice x = A. Then and M(t, y; ) = y2 2 cosh(t) sinh(t), ( S c (, y) = y ) y y 2 2c + y2 + c log + 2 2c 2c + 1. It is easily checked that in this case M(t, y; ) and U(t, y; ) agree for all choices of y and t >. In fact, differentiating S c (, y) with respect to c reveals that t L = in this case. 3. The stationary Hamilton-Jacobi equation In this section properties of the Mañé potential and its connection to stationary Hamilton-Jacobi equations are presented. Given a Hamiltonian H and c R, the stationary Hamilton-Jacobi equation is H(y, DS(y)) = c, y R n. (3.1) A continuous function S : R n R is a viscosity subsolution (supersolution) of the stationary Hamilton-Jacobi equation (3.1) if, for every function v C (R n ), if S v has a local maximum (minimum) at y R n, (3.2) then H(y, Dv(y )) c ( c). It is a viscosity solution if it is both a viscosity subsolution and a viscosity supersolution. The Mañé critical value is the infimum over c for which (3.1) admits a viscosity subsolution. With some abuse of notation it will be denoted by c H. It may be observed that c H sup inf H(y, p). (3.3) y p Indeed, if (3.1) admits a viscosity subsolution U c at level c, then for almost every y there is a v C (R n ) such that U c v has a local maximum at y and inf p H(y, p)

13 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS 13 H(y, Dv(y)) c. The claim follows by taking supremum over y. Examples where c H = sup y inf p H(y, p) are provided below. The Mañé potential (2.2) is well studied within weak KAM theory where it is commonly assumed that the Hamiltonian is uniformly superlinear; for each K there exists C(K) R such that H(y, p) K p C(K) for each y, p. Under such an assumption there exist critical viscosity subsolutions, that is, there exists a global viscosity subsolution to (3.1) for c = c H, see [24, 23]. In this paper it is assumed that the Hamiltonian is given by the Fenchel-Legendre transform of a Lagrangian L, as in (2.1), and consequently p H(y, p) is convex in p, for every y R n. For instance, the Hamiltonian associated with the unit rate Poisson process, which is of the form H(p) = e p 1, p R, is covered by our assumptions. For this choice of H the Mañé critical value is c H = 1, but there can be no critical subsolution S as it would have to satisfy DS(y) = almost eveywhere, see Example 3.3 below. The following properties of the Mañé potential are well known and similar statements appear in [23, 24, 25], see also the lecture notes [22, 5]. Because our assumptions on the Hamiltonian are different a proof is included for completeness. Proposition 3.1. Assume (2.1) and take c R. (i) Suppose that S c >. Take x R n and suppose that the function y S c (x, y) is continuous. Then y S c (x, y) is a viscosity subsolution to H(y, DS(y)) = c on R n and a viscosity solution on R n \ {x. (ii) Take x R n and suppose that the function y S c (x, y) is continuous. Then, S c (x, y) = sup S S c x S(y), for each y R n, where Sx c is the collection of all continuous viscosity subsolutions to H(y, DS(y)) = c that vanish at x. Take x R n and suppose that, for each c > c L, the function y S c (x, y) is continuous. For c > c H there exist viscosity subsolutions to (3.1) and by Proposition 3.1(ii) it follows that S c >. Consequently, c H c L. Similarly, for c < c H there are no subsolutions and by Proposition 3.1(i) S c =, which implies c H c L. This proves the following. Corollary 3.1. Take x R n and suppose that, for each c > c L, the function y S c (x, y) is continuous. Then c H = c L. Before proceeding to the proof of Proposition 3.1 we state an important lemma that can be interpreted as a dynamic programming property of the Mañé potential. Lemma 3.1. Suppose that S c >. For any x, y R n with y x and ɛ > there exist < δ < x y, y with y y < δ, h > and an absolutely continuous path ψ with ψ() = y, ψ(h) = y, and ψ(s) y < δ for all s [, h], such that S c (x, y ) S c (x, y) + h c + L(ψ(s), ψ(s))ds ɛ. Proof. Given x, y R n with x y and ɛ >, take t > and an absolutely continuous function ϕ with ϕ() = x, ϕ(t) = y such that S c (x, y ) t c + L(ϕ(s), ϕ(s))ds ɛ.

14 14 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS Let < δ < x y and take h > such that ϕ(s) y < δ for each s [t h, t]. With y = ϕ(t h) and ψ(s) = ϕ(s + t h), s [, h], it follows that S c (x, y ) = t t h This completes the proof. c + L(ϕ(s), ϕ(s))ds ɛ S c (x, y) + c + L(ϕ(s), ϕ(s))ds + h t t h c + L(ψ(s), ψ(s))ds ɛ. c + L(ϕ(s), ϕ(s))ds ɛ Proof of Proposition 3.1. Proof of (i). Suppose that S c >, take x R n and suppose that y S c (x, y) is continuous. First we prove the viscosity subsolution property. For v C (R n ), suppose that S c (x, ) v has a local maximum at y and, contrary to what we want to show, that H(y, Dv(y)) c θ > for y y δ, for some δ >. We may assume that δ is sufficiently small that S c (x, y) v(y) S c (x, y ) v(y ), for y y δ. Take any y with y y δ and consider any absolutely continuous path ψ such that ψ() = y, ψ(h) = y and ψ(s) y δ for all s [, h]. By the triangle inequality (2.3) and the last inequality S c (x, y ) S c (x, y) v(y ) v(y) = = h h h h c + L(ψ(s), ψ(s))ds c + L(ψ(s), ψ(s))ds d ds v(ψ(s)) L(ψ(s), ψ(s)) c ds Dv(ψ(s)), ψ(s) L(ψ(s), ψ(s)) c ds. We may assume that ψ is chosen such that, using the conjugacy between H and L, for all s [, h]. Then H(ψ(s), Dv(ψ(s))) Dv(ψ(s)), ψ(s) L(ψ(s), ψ(s)) + θ 2, θh 2 h H(ψ(s), Dv(ψ(s))) c ds θh, which is a contradiction. Thus, it must indeed hold that H(y, Dv(y )) c. Next, we prove the supersolution property on R n \{x. Take v C (R n ) and suppose S c (x, ) v has a local minimum at y x and, contrary to what we want to show, that H(y, Dv(y)) c θ < for y y δ, for some δ >. We may assume that δ is sufficiently small that x y > δ and S c (x, y) v(y) S c (x, y ) v(y ), for y y δ.

15 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS 15 By Lemma 3.1 we may select y with y y δ and an absolutely continuous path ψ such that ψ() = y, ψ(h) = y and ψ(s) y δ for all s [, h], with the property that S c (x, y ) S c (x, y) + The last inequality implies that We conclude that h θh 2 Sc (x, y) S c (x, y ) + v(y) v(y ) + = = h h h h c + L(ψ(s), ψ(s))ds θh 2. h c + L(ψ(s), ψ(s))ds c + L(ψ(s), ψ(s))ds d ds v(ψ(s)) + L(ψ(s), ψ(s)) + c ds Dv(ψ(s)), ψ(s) + L(ψ(s), ψ(s)) + c ds ( ) H(ψ(s), Dv(ψ(s))) c ds. θh h 2 H(ψ(s), Dv(ψ(s))) c ds θh, which is a contradiction. Thus, it must indeed hold that H(y, Dv(y )) c. This completes the proof of (i). Proof of (ii). Let c R. If there are no viscosity subsolutions at level c, then by (i) S c = and Sx c =, which implies that sup S S c x S(y) = as well. If there exist continuous viscosity subsolutions at level c, take x R n and let S be a continuous viscosity subsolution of H(y, DS(y)) = c on R n. It is sufficient to show that for any y R n, t > and absolutely continuous function ψ with ψ() = x and ψ(t) = y, S(y) S(x) t c + L(ψ(s), ψ(s))ds. (3.4) To show (3.4), fix t >, y R n, an absolutely continuous path ψ with ψ() = x and ψ(t) = y and take an arbitrary ɛ >. For every s [, t], let v s C (R n ) be such that S v s has a local maximum at ψ(s). Then, there exists δ s > such that and consequently that S(z) v s (z) S(ψ(s)) v s (ψ(s)), for z ψ(s) < δ s, S(z) S(ψ(s)) v s (z) v s (ψ(s)), for z ψ(s) < δ s. (3.5) By continuity of H and Dv s we may, in addition, assume that δ s is sufficiently small that H(z, Dv s (z)) c + ɛ t, for z ψ(s) < δ s.

16 16 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS For every s [, t], let h s > be such that ψ(u) ψ(s) < δ s for every u with u s < h s. This is possible due to the continuity of ψ. The union [, h ) (s, s + h s ), s (,t] is an open cover of [, t]. Since [, t] is compact there is a finite subcover, which we may assume is of the form n 1 [, h ) (s k, s k + h sk ), k=1 where = s < s 1 < < s n 1 < s n = t. Since the finite union is a subcover, it must hold that s k 1 < s k < s k 1 + h sk 1 for each k = 1,..., n. It follows that, using (3.5) and the conjugacy between H and L, n S(y) S(x) = S(ψ(s k )) S(ψ(s k 1 )) = k=1 n v sk 1 (ψ(s k )) v sk 1 (ψ(s k 1 )) k=1 sk n k=1 sk n k=1 sk n k=1 t = ɛ + Since ɛ > was arbitrary the claim follows. s k 1 Dv sk 1 (ψ(s)), ψ(s) ds s k 1 H(ψ(s), Dv sk 1 (ψ(s))) + L(ψ(s), ψ(s)) ds s k 1 c + ɛ t + L(ψ(s), ψ(s)) ds c + L(ψ(s), ψ(s)) ds. We proceed by computing Mañé s critical value, c H, for some Hamiltonians arising in the theory of large deviations of stochastic processes. Example 3.1 (Critical diffusion process). Let U : R n R be a potential function and b(y) = DU(y). Consider the Hamiltonian H(y, p) = b(y), p p 2. Then c H = sup y inf p H(y, p) = 1 inf 2 y b(y) 2. Indeed, from (3.3), c H 1 inf 2 x b(y) 2 and U is a subsolution to H(y, DS(y)) = 1 inf 2 y b(y) 2, which implies c H 1 inf 2 y b(y) 2. In particular, if DU(y) = for some y, then c H =. The Mañé potential can be viewed as a generalization of Freidlin-Wentzell s quasi-potential described in [28, Ch. 4]. Example 3.2 (Birth-and-death process). Consider an interval (a, b) R and functions µ : (a, b) [, ) and λ : (a, b) [, ) satisfying b log( µ(y)/λ(y))dy <. a Consider the Hamiltonian H(y, p) = λ(y)(e p 1) + µ(y)(e p 1).

17 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS 17 In this case c H = sup y inf p H(y, p) = inf y ( µ(y) λ(y)) 2. To see this, recall from (3.3) that c H inf y ( µ(y) λ(y)) 2. A subsolution of is given by Indeed, H(y, DS(y)) = inf y ( µ(y) λ(y)) 2, U(y) = y a log( µ(z)/λ(z))dz. H(y, DU(y)) = ( µ(y) λ(y)) 2 inf y ( µ(y) λ(y)) 2. Example 3.3 (Pure birth process). Let λ : [, ) n [, ) n and put n H(y, p) = λ j (y)(e p j 1). j=1 In this case c H = sup y inf p H(y, p) = inf n y j=1 λ j(y) =: λ. Indeed, from (3.3) it follows that c H λ and for any c ( λ, ) and α log(1 + c/λ ), the function α 1, y is a subsolution to H(y, DS(y)) = c, which implies c H λ. This section concludes by providing a sufficient condition for the continuity of y S c (x, y). Proposition 3.2. Suppose that the Lagrangian L is continuous at (y, ) for each y R n. Then, for each x R n and c > c L the function y S c (x, y) is continuous. Proof. Take y R n and ɛ >. To prove continuity at y we show that there exists a δ > such that y y < δ implies S c (x, y ) S c (x, y) + ɛ, (3.6) S c (x, y) S c (x, y ) + ɛ. (3.7) We begin to prove (3.6). Since L is continuous at (y, ) we may select δ such that L(y + z, v) L(y, ) + 1 for all z < δ and v < δ. Pick h > such that h(c + L(y, ) + 1) < ɛ/2 and let δ = hδ. For y y < δ, take t > h and an absolutely continuous path ψ with ψ() = x, ψ(t h) = y such that S c (x, y) t h and ψ(s) = h 1 (y y) for t h s t. Then, S c (x, y ) t h c + L(ψ(s), ψ(s)) ds ɛ 2, c + L(ψ(s), ψ(s)) ds + t t h S c (x, y) + ɛ 2 + h(c + L(y, ) + 1) S c (x, y) + ɛ, by the choice of h. The proof of (3.7) is similar. c + L(ψ(s), ψ(s)) ds

18 18 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS 4. Minmax representation of viscosity solutions In this section it will be demonstrated how the duality in Theorem 2.1 leads to minmax representations of viscosity solutions of evolutionary Hamilton-Jacobi equations. For an introduction to Hamilton-Jacobi equations the reader is referred to [2, 4, 21, 23, 9, 8]. Consider a Hamiltonian H : R n R n R as in (2.1). The evolutionary Hamilton- Jacobi equation is V t (t, y) + H(y, DV (t, y)) =, (t, y) (, ) R n, (4.1) where V t = V/ t and DV = ( V/ y 1,..., V/ y n ). A continuous function V : (, ) R n R is a viscosity subsolution (supersolution) of (4.1) if, for every v C ((, ) R n ), if V v has a local maximum (minimum) at (t, y ) (, ) R n, then v t (t, y ) + H(x, Dv(t, y )) ( ). V is a viscosity solution if it is both a subsolution and a supersolution of (4.1). Recall the action functional M in (2.4). The action functional plays a similar role for the evolutionary Hamilton-Jacobi equation as the Mañé potential does for the stationary Hamilton-Jacobi equation. Proposition 4.1. Take x R n and assume that (t, y) M(t, y; x) is continuous. (i) M( ; x) is a viscosity subsolution to (4.1) on (, ) R n and a viscosity solution on (, ) R n. (ii) M(t, y; x) = sup V S,x V (t, y), where S,x is the collection of all continuous viscosity subsolutions to (4.1) vanishing at (, x). The proof of Proposition 4.1 is almost identical to that of Proposition 3.1 and is therefore omitted Constructing viscosity solutions from the Mañé potential. In this section the Mañé potential will be used to construct viscosity solutions to the evolutionary Hamilton-Jacobi equation. Suppose that L and H satisfy (2.1) and y S c (x, y) is continuous for x R n and c > c L. By Proposition 3.1(i), y S c (x, y) is a viscosity subsolution to H(y, DS(y)) = c for each x R n and c > c L. It follows immediately that the function (t, y) S c (x, y) ct is a viscosity subsolution of the evolutionary Hamilton-Jacobi equation (4.1). Perron s method, see [2, Theorem V.2.14], implies that the function U( ; x) given by U(t, y; x) = sup c>c L {S c (x, y) ct, (t, y) [, ) R n, (4.2) is also a viscosity subsolution to (4.1). Moreover, (t, y) S c (x, y) ct is a viscosity solution to (4.1) on R n \ {x. This property also transfers to U(t, y; x) as the following proposition shows. Proposition 4.2. For x R n, the function U in (4.2) is a viscosity solution to (4.1) on (, ) R n \ {x.

19 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS 19 Proof. Since y S c (x, y) is a viscosity subsolution to H(y, DS(y)) = c, for any c > c L, it follows by Perron s method that U(t, y; x) is a viscosity subsolution to (4.1). It remains to show the supersolution property. Fix x R n and take v C ((, ) R n \ {x). Suppose that U( ; x) v has a local minimum at (t, y ) where t > and y x. We must show that v t (t, y ) + H(y, Dv(t, y )). Suppose, on the contrary, that there exist θ > and δ > such that v t (t, y) + H(y, Dv(t, y)) θ, for all (t, y) with t t + y y < δ. We will arrive at a contradiction by showing that there is a c > c L such that the viscosity supersolution property is violated for the function (t, y) S c (x, y) ct at some point (t, y), t >, y x. We may assume that the δ above is sufficiently small that x y > δ and U(t, y; x) v(t, y) U(t, y ; x) v(t, y ), for all (t, y) with t t + y y < δ. Then, the subsolution property is strict at (t, y ) in the sense that there is a θ 1 > such that, for all w C ((, ) R n \ {x) such that U( ; x) w has a local maximum at (t, y ) w t (t, y) + H(y, Dw(t, y)) θ 1, (4.3) for all t t + y y < δ 1, some δ 1 >. To prove (4.3), observe first that since U( ; x) w has a local maximum at (t, y ) and U( ; x) v has a local minimum at (t, y ) we may select δ 1 such that U(t, y; x) w(t, y) U(t, y ; x) w(t, y ), U(t, y; x) v(t, y) U(t, y ; x) v(t, y ), for all (t, y) with t t + y y < δ 1. Consequently, w v satisfies w(t, y) v(t, y) w(t, y ) v(t, y ), for all (t, y) with t t + y y < δ 1 so w v has a local minimum at (t, y ). Since both v and w are in C ((, ) R n \ {x) it follows that w t (t, y ) = v t (t, y ) and Dw(t, y ) = Dv(t, y ). We conclude that w t (t, y ) + H(y, Dw(t, y )) = v t (t, y ) + H(y, Dv(t, y )) θ. Now (4.3) follows by taking θ 1 (, θ) and using continuity of H, w t and Dw. We may, without loss of generality, assume that w in (4.3) is such that w(t, y ) = U(t, y ; x) and w(t, y) > U(t, y; x), t t + y y < δ 1. Take < ɛ < δ 1 so that (4.3) holds on N ɛ = {(t, y) : t t + y y ɛ. Since w(t, y) > U(t, y; x) on N ɛ there is an η > such that Moreover, we may select c > c L such that w(t, y) η U(t, y; x), (t, y) N ɛ. S c (x, y ) ct > U(t, y ; x) η = w(t, y ) η.

20 2 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS Rewriting the last two displays we find that S c (x, y ) ct w(t, y ) > η, S c (x, y) ct w(t, y) η, (t, y) N ɛ. It follows that the maximum of the continuous function (t, y) S c (x, y) ct w(t, y) over the compact set N ɛ is attained at some (t ɛ, y ɛ ) in the open neighborhood N ɛ and by (4.3) w t (t ɛ, y ɛ ) + H(y ɛ, Dw(t ɛ, y ɛ )) θ 1. Let v ɛ C ((, ) R n \ {x) be such that the function (t, y) S c (x, y) ct v ɛ (t, y) has a local minimum at (t ɛ, y ɛ ). Then, there is a δ 2 > such that S c (x, y) ct w(t, y) S c (x, y ɛ ) ct ɛ w(t ɛ, y ɛ ), S c (x, y) ct v ɛ (t, y) S c (x, y ɛ ) ct ɛ v ɛ (t ɛ, y ɛ ), for all (t, y) with t t ɛ + y y ɛ < δ 2. Consequently, w v ɛ has a local minimum at (t ɛ, y ɛ ) and w t (t ɛ, y ɛ ) = v ɛ t(t ɛ, y ɛ ) and Dw(t ɛ, y ɛ ) = Dv ɛ (t ɛ, y ɛ ). We conclude that v ɛ t(t ɛ, y ɛ ) + H(y ɛ, Dv ɛ (t ɛ, y ɛ )) = w t (t ɛ, y ɛ ) + H(y ɛ, Dw(t ɛ, y ɛ )) θ 1 <. The last display contradicts the viscosity supersolution property of (t, y) S c (x, y) ct at (t ɛ, y ɛ ). We conclude that the supersolution property holds for U( ; x) on (, ) R n \ {x. Take x R n and assume the required continuity. By Proposition 4.1 and Proposition 4.2 both (t, y) M(t, y; x) and (t, y) U(t, y; x) are viscosity solutions to (4.1) on (, ) R n \ {x. At t =, M(, y; x) = U(, y; x) = if y x and = if y = x. However, in the present setting there is no valid comparison principle so equality between M and U need not hold for t >. Indeed, by Theorem 2.1, when n = 1 M(t, y; x) = U(t, y; x) for t t L, but it may happen that M(t, y; x) > U(t, y; x) for t > t L as illustrated in Example Minmax representation for initial value problems. Given an initial function g : R n R, the initial value problem for the Hamilton-Jacobi equation is to find V : [, ) R n R such that V satisfies (4.1) and V (, y) = g(y), y R n. (4.4) A continuous function V : (, ) R n R is a viscosity subsolution (supersolution) if it is a viscosity subsolution (supersolution) to (4.1) and V (, y) g(y) ( g(y)). If V is uniformly continuous and H satisfies Condition 4.1 below, then the comparison principle holds and the solution of the initial value problem is unique, see e.g. Theorem 3.7 and Remark 3.8 in Chapter II of [2]. Condition 4.1. H is uniformly continuous on R n B (R) for each R > and H(x, p) H(y, p) ω( x y (1 + p )), for x, y, p R n, where B (R) = {p R n : p < R and ω : [, ) [, ) is a continuous nondecreasing function with ω() =.

21 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS 21 Moreover, viscosity solutions can be given a variational representation. Given an initial function g : R n R, let V be the value function of the variational problem { t V (t, y) = inf g(ψ()) + L(ψ(s), ψ(s))ds, ψ(t) = y, (4.5) ψ where (t, y) [, ) R n and the infimum is taken over all absolutely continuous functions ψ : [, ) R n. It is well known that if V is continuous then it is a continuous viscosity solution to (4.4), see e.g. [2, Ch. III, Sec. 3]. From Theorem 2.1 the following min-max representation of V is obtained. Proposition 4.3. Suppose that n = 1, L and H are as in (2.1) and V is given by (4.5). If either y A and t > or y R n and t < t L, then V (t, y) = inf x sup c>c L {g(x) + S c (x, y) ct. (4.6) Moreover, if V is continuous, then it is a viscosity solution to (4.4) on [, t L ) R. Proof. It follows from (2.6) that V (t, y) = inf{g(x) + M(t, y; x) = inf sup {g(x) + S c (x, y) ct. x x c>c L 4.3. Minmax representation for terminal value problems. Let the following be given: A time T >, a Lagrangian L and Hamiltonian H as in Section 2.3, and a terminal cost function g. Consider a terminal value problem with value function V (t, x) = inf ψ { T t L(ψ(s), ψ(s))ds + g(ψ(t )), ψ(t) = x, where the infimum is taken over all absolutely continuous functions ψ on [, T ], with ψ(t) = x. By changing the direction of the paths it follows that V (t, x) is equal to { T t inf g(ψ()) + L(ψ(s), ψ(s))ds, ψ(t t) = x = V (T t, x), where V is the value function of the forward problem (4.5) with L(x, v) = L(x, v). For c > c L, let S c (x, y) denote the Mañé potential associated with L. Then, it holds that S c (x, y) = S c (y, x) and, for n = 1, the min-max representation of Proposition 4.3 can be expressed as V (t, x) = V (T t, x) = inf y = inf y sup {g(y) + S c (y, x) c(t t) c>c L sup {g(y) + S c (x, y) c(t t), c>c L if either x A and t > or x R n and T t < t L. The Hamiltonian of the corresponding forward problem is H(x, p) = sup v { p, v L(x, v) = sup{ p, v L(x, v) = H(x, p). v

22 22 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS If V is continuous, then so is V and since V is a continuous viscosity solution to (4.4) on [, t L ) it follows that V is a continuous viscosity solution to { Vt (t, x) H(x, D V (t, x)) =, (t, x) [, T ) R, V (T, x) = g(x), x R, if T t L. In general, it is not possible to interchange the inf and sup in the min-max representation as the following example shows. In Section 6 this particular example is discussed further in the context of rare-event simulation. More generally, results such as Sion s general minimax theorem [35] can be applied on a case-by-case basis to check whether or not interchanging inf and sup is allowed. Example 4.1. Consider a one-dimensional terminal value problem, with Hamiltonian H(x, p) = H(p) = p p2 and g(x) = on (a, b) and g(x) = on (a, b), where a < 1 < b and b 1 < 1 a. Since H does not depend on x we have A = R. The Mañé critical value is c H = c L = 1/2 and the Mañé potential is given by S c (x, y) = { (y x)( c), y x, (x y)( c), y < x. By performing the optimization it follows that and, for x < a, we have sup{ S c (x, y) c(t t) = c>c L { T t ( y x 2 T t ( x y 2 V (t, x) = inf sup { S c (x, y) c(t t) = inf y {a,b c>c L In particular, with T = 1, we have V (, ) = T t 1)2, y x, T t 1)2, y < x. y {a,b 1 inf y {a,b 2 (y 1)2 = 1 2 (b 1)2. T t ( y x ) 2. 2 T t 1 Consider interchanging the order of the inf and sup. For any c > c L the infimum over the boundary is { inf { S a( c) c, for c, c (, y) c = y {a,b b( c) c, for c <. An elementary calculation shows that sup c>cl inf y {a,b { S c (, y) c is equal to ( sup{a( 1 + ) ( 1 + 2c) c sup{b( 1 + ) 1 + 2c) c =. c c< We conclude that V (, ) = inf sup { S c (, y) c > sup inf { S c (, y) c. y {a,b y {a,b c>c L c>c L

23 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS Minmax representation for problems on domains. Let Ω := (a, b) R be an open interval, Ω := {a, b, g : Ω R a function representing the boundary condition and, for (t, y) (, ) (a, b), let V (t, y) = inf ψ { g(ψ()) + t L(ψ(s), ψ(s))ds, ψ() Ω, ψ(t) = y, where the infimum is taken over all absolutely continuous functions ψ : [, ) Ω, with ψ() Ω and ψ(t) Ω, t >. If either y A and t > or y R and t < t L, then the min-max representation is given by V (t, y) = inf x {a,b sup{g(x) + S c (x, y) ct. (4.7) c>c L If V is continuous, then it is a continuous viscosity solution to { V t (t, y) + H(y, DV (t, y)) =, (t, y) (, ) Ω, V (, y) = g(y), y Ω. Similarly, the terminal value problem on Ω is { T V (t, x) = inf L(ψ(s), ψ(s))ds + g(ψ(t )), ψ(t) = x, ψ(t ) Ω, t where (t, x) [, T ) Ω. If either x A and t > or x R and t < t L, then the min-max representation is given by V (t, x) = inf y Ω sup c>c L {g(y) + S c (x, y) c(t t) (4.8) If V is continuous, then it is a continuous viscosity solution to { Vt (t, x) H(x, D V (t, x)) =, (t, x) [, T ) Ω, V (T, x) = g(x), x Ω. (4.9) 4.5. Minmax representation for exit problems. Let Ω : (a, b) R be an open interval, Ω := {a, b, g : Ω R be the boundary condition and take T >. Consider the minimal cost W of leaving the interval before time T, when starting from (t, x) [, T ) Ω. The function W is given by W (t, x) = inf ψ,σ { σ L(ψ(s), ψ(s))ds + g(ψ(σ)), ψ(t) = x, ψ(σ) Ω, t where t σ T. By the change of variables, τ = T σ + t, and, for t s T, ϕ(s) = ψ(t + s τ). { T W (t, x) = L(ϕ(s), ϕ(s))ds + g(ϕ(t )), ϕ(τ) = x, ϕ(t ) {a, b inf ψ,t τ T = inf t τ T with V as in (4.9). τ V (τ, x), (t, x) [, T ) (a, b),

24 24 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS If either x A or T < t L, then W can be represented as W (t, x) = inf t τ T inf sup {g(y) + S c (x, y) c(t τ) (4.1) y Ω c>c L Obviously W (t, x) V (t, x). If c L, then it follows that W (t, x) = inf t τ T inf y Ω sup c>c L sup y Ω c>c L inf We have proved the following. = V (t, x). inf sup {g(y) + S c (x, y) c(t τ) y Ω c>c L inf {g(y) + S c (x, y) c(t τ) t τ T {g(y) + S c (x, y) c(t t) Proposition 4.4. If c L and either x A or T < t L, then W (t, x) = V (t, x), t T. Note also that if W is continuous, then it is a continuous viscosity solution to { Wt (t, x) H(x, D W (t, x)) =, (t, x) [, T ) Ω, W (t, x) = g(x), (t, x) [, T ] Ω. 5. The Hopf-Lax-Oleinik representation (4.11) Suppose the Hamiltonian H is state-independent, that is, H(x, p) = H(p) and convex. Then A = R n. If g is uniformly continuous, then the Hopf-Lax-Oleinik representation, see [21, Ch. X], states that the function V (t, y) = inf x { g(x) + tl ( y x is the unique continuous viscosity solution to { V t (t, y) + H(DV (t, y)) =, (t, y) (, ) R n, V (, y) = g(y), y R n. t ), (5.1) We will demonstrate a direct relation between the Hopf-Lax-Oleinik representation and the min-max representation (4.6). Proposition 5.1. If H is convex and state-independent, then, for all y R n, ( y x ) sup{s c (x, y) ct = tl. c>c L t Moreover, if the initial function g is uniformly continuous, then V (t, y) = inf x sup {g(x) + S c (x, y) ct = inf c>c L x is the unique continuous viscosity solution to (4.4). { g(x) + tl ( y x t ).

25 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS 25 Proof. We begin by proving the inequality: for each x, ( y x ) sup{s c (x, y) ct tl. c>c L t Take x R n, c > c L and observe that for p such that H(p) = c { t S c (x, y) = inf H(p) + L( ψ(s))ds, ψ() = x, ψ(t) = y ψ,t { t inf p, ψ(s) ds, ψ() = x, ψ(t) = y ψ,t = p, y x, where the inequality holds due to the convex conjugacy between L and H. It follows that S c (x, y) ct sup { p, y x th(p) p:h(p)=c { = t sup p:h(p)=c p, y x t H(p). By Proposition 3.1, S c (x, y) = for c < c L, which implies that the supremum over c > c L can be extended to the whole of R. That is, The reverse inequality sup{s c (x, y) ct = sup{s c (x, y) ct c>c L c R { t sup sup c R p:h(p)=c ( y x = tl t ). sup{s c (x, y) ct tl c>c L p, y x t ( y x t ), H(p) follows immediately by taking ψ(s) = (y x)/t and observing that t S c (x, y) c + L( ψ(s))ds [ = c + L( y x )] t. t 6. Applications in rare-event simulation The simulation of rare events in stochastic models and the computation of their probabilities is a challenging problem with numerous applications in, for instance, biology, chemistry, engineering, finance, operations research, etc. In the rare-event setting the standard Monte Carlo algorithm fails because few particles will hit the relevant part of the state space, leading to a large relative error. There are several variance reduction techniques to improve computational efficiency that try to control the simulated particles in such a way that they reach the relevant part of the space. Such techniques can, if designed well, reduce the computational cost by several orders of magnitude. Examples of

26 26 VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS such techniques are importance sampling, multi-level splitting, and genealogical particle methods. The common feature of all algorithms designed for the rare-event setting is that the control mechanism must be carefully chosen to control the relative error. Roughly speaking the large deviations of the stochastic model must be taken into account and guide the design of the algorithm. In a series of papers [18, 19, 16, 15] the authors have established the connection between efficient importance sampling algorithms and subsolutions to associated partial differential equations of Hamilton-Jacobi type that arise in large deviation theory; in [36] solutions to the relevant equations are used. The results can be briefly summarized as follows. To compute an expectation of the form E[exp{ ng(x n (T ))I{X n (T ) / Ω] the choice of sampling dynamics is associated with a control problem whose value function, in the rare-event limit, is given as the solution V to a Hamilton-Jacobi equation of the form (4.9). By constructing a (piecewise) classical subsolution Ū to (4.9), that is a piecewise C1 ( Ω) (i.e., in the state variable) function Ū satisfying {Ūt (t, x) H(x, DŪ(t, x)), (t, x) [, T ) Ω, (6.1) Ū(T, x) g(x), x Ω, the change of measure can be based on DŪ and the performance of the resulting algorithm is determined by the initial value Ū(, x ) of the subsolution. Asymptotically optimal performance is obtained if the value of the subsolution at the initial point (, x ) coincides with that of the solution, Ū(, x ) = V (, x ). In multi-level splitting the situation is similar. In the most simple version of multi-level splitting the state space is partitioned into an increasing sequence of sets C C 1... given as the level sets of an importance function U. A particle is simulated from an initial point x and as it crosses over from, say, C k+1 to C k for the first time, the particle generates a number of offsprings that are simulated independently of each other. Particles are killed if they reach a termination set. Each particle carries a weight that is updated at every split. By this procedure a random tree is produced, where each leaf is a particle that has either hit the set of interest or been killed. The sum of the weights of the particles that reach the target set is the estimate of the rare-event probability. The design of an efficient multi-level splitting algorithm relies on that the associated importance function is a certain multiple of a viscosity subsolution of the Hamilton-Jacobi equation associated with the large deviations of the system, see [13, 14]. In what follows the emphasis will be on the construction of families of viscosity subsolutions associated with the min-max representation. To be precise, in this section the term viscosity subsolution refers to a function that satisfies the inequalities (6.1) in the viscosity sense. For brevity the discussion is focused on terminal value problems, exit problems can be treated completely analogously. Moreover, in the examples considered in Sections 6.2 and 6.3 the derived subsolutions are indeed piecewise continuous on the relevant domains and the results on efficiency of the corresponding simulation algorithms are applicable Construction of subsolutions. The min-max representation (4.8) provides at least two convenient ways to construct families of viscosity subsolutions, suitable for the