Singular Perturbations of Stochastic Control Problems with Unbounded Fast Variables

Singular Perturbations of Stochastic Control Problems with Unbounded Fast Variables Joao Meireles joint work with Martino Bardi and Guy Barles University of Padua, Italy Workshop "New Perspectives in Optimal Control and Games" Nov. 10-12, 2014, Rome SADCO Meeting Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 1 / 30

Plan The Two-scale system The Optimal Control Problem The HJB equation The PDE approach to the singular limit ɛ 0 The effective Hamiltonian H The Ergodic Problem (EP) Convergence Result Tools Approximation of (EP) by truncation or state constraints Sketch of the proof Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 2 / 30

Two-scale systems (Ω, F, F t, P) complete filtered probability space. We consider stochastic control systems with small parameter ɛ > 0 of the form: { dxs = F(X s, Y s, u s )ds + 2σ(X s, Y s, u s )dw s, X s0 = x R n dy s = 1 ɛ ξ 1 sds + ɛ τ(y s)dŵs, Y s0 = y R m. Basic assumptions F and σ Lipschitz functions in (x, y) uniformly w.r.t. u, + conditions (later) ττ T = 1 F(x, y, u) + σ(x, y, u) C(1 + x ) u U (compact), ξ takes values in R m Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 3 / 30

The Optimal Control Problem For θ > 1, we consider Payoff Functionals for t [0, T ] of the form α > 1 J ɛ (t, x, y, u, ξ) = E x,y [ T t (l(x s, Y s, u s ) + 1 θ ξ s θ )ds + g(x T )]. g continuous, g(x) C(1 + x α ) and g bounded below l contiuous and + conditions (later) Value Function is: l 0 y α l 1 0 l(x, y, u) l 1 0 (1 + y α ) V ɛ (t, x, y) = inf u,ξ Jɛ (t, x, y, u, ξ), 0 t T Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 4 / 30

Admissible control ξ Definition For T > 0, we say that ξ is admissible if E y[ T ξ s θ ds ] < +. It is possible to see that E x[ T 0 X s α ds ] < + (standard) 0 E y[ T 0 Y s α ds ] < + (less standard, uses the admissibility of ξ and α θ ) V ɛ is bounded by below V ɛ (t, x, y) C(1 + x α + y α ) (equi boundedness) Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 5 / 30

The HJB equation The HJB equation associated via Dynamic Programming to the value function V ɛ is with Vt ɛ + H(x, y, D x V ɛ, DxxV 2 ɛ, D2 xyv ɛ ) 1 ɛ 2ɛ yv ɛ + 1 θ D yv ɛ θ = 0 ɛ H(x, y, p, M, Z ) := sup{ trace(σσ T M) F p 2trace(στ T Z T ) l} u in (0, T ) R n R m complemented with the terminal condition V ɛ (T, x, y) = g(x, y) This is a fully nonlinear degenerate parabolic equation. Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 6 / 30

Well Posedness Theorem Suppose θ α where α is the conjugate number of α, that is, α = α α 1. For any ɛ > 0, the function V ɛ is the unique continuous viscosity solution to the Cauchy problem with at most α-growth in x and y. Moreover the functions V ɛ are locally equibounded. Uses Comparison principle between sub and super solution to parabolic problems super linear growth conditions (see [Da Lio - Ley 2011]), Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 7 / 30

PDE approach to the singular limit ɛ 0 Search effective Hamiltonian H s. t. V ε (t, x, y) V (t, x) as ε 0, V solution of { V (CP) t + H ( x, D x V, DxxV 2 ) = 0 in (0, T ) R n, V (T, x) = g(x) in R n Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 8 / 30

The effective Hamiltonian H Finding the candidate limit Cauchy problem of the singularly perturbed problem as ɛ 0... Ansatz: V ɛ (t, x, y) = V (t, x) + ɛχ(y), with χ(y) C 2 (R m ). We get V t + H(x, y, D x V, D 2 xxv, 0) 1 2 yχ + 1 θ D yχ θ = 0 with H(x, y, p, M, 0) = sup u { trace(σσ T M) F p l}. We wish that H(x, D x V, D 2 xxv ) = H(x, y, D x V, D 2 xxv, 0) 1 2 yχ + 1 θ D yχ θ. Idea is to frozen ( t, x), set p := D x V ( t, x) and M := D 2 xxv ( t, x) and let only y varie. Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 9 / 30

The effective Hamiltonian H H( x, p, M) is a constant that we will denote by λ. Thus λ = H( x, y, p, M, 0) 1 2 yχ + 1 θ D yχ θ λ 1 2 yχ + 1 θ D yχ θ = H( x, y, p, M, 0). If we call f (y) := H( x, y, p, M, 0) and impose χ(0) = 0, to avoid the ambiguity of additive constant, we are lead to the following ergodic problem: (EP) { λ 1 2 yχ + 1 θ D yχ θ = f (y) in R m, χ(0) = 0, where unknown is the pair (λ, χ) R C 2 (R m ). Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 10 / 30

The (EP) Ergodic Problem { λ 1 2 yχ + 1 θ D yχ θ = f (y) χ(0) = 0. in R m, Unknown is the pair (λ, χ) R C 2 (R m ). Such type of ergodic problems were studied by Naoyuki Ichihara in [Ichihara 2012]. Assumptions in [Ichihara 2012] f C 2 (R m ) f 0 > 0 s.t. f 0 y α f 1 0 f (y) f 1 0 (1 + y α ) (comes from my l) Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 11 / 30

The (EP) - Results from Ichihara (local gradient bounds) For any r > 0, there exists a constant C > 0 depending only on r, m and θ such that for any solution (λ, χ) of (EP), sup Dχ C(1 + sup f λ 1 θ + sup Df B r B r+1 B r+1 χ(y) C(1 + y γ ) (γ = α θ + 1) 1 2θ 1 ) (Uniqueness) There exists a unique solution (λ, χ) of (EP) such that χ belongs to Φ γ := {v C 2 (R m ) C p (R m v(y) ) lim inf y y γ > 0}. H = λ is the minimum of the constants for which (EP) has a solution φ C 2 (R m ) Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 12 / 30

The (EP)- From Ichihara Theorem Let (λ, χ) be a solution of (EP) such that χ is bounded by below. Then, T ɛχ(y) + λ(t t) = inf Ey[ ( 1 ξ A t θ ξ s θ + f (Ys ξ ) ) ds + ɛχ(y ξ T )], T > t. equality is reach for the optimal feedback control ξ. From this result, you can deduce FORMULA (F) T ɛ(χ 1 χ 2 )(y)+(λ 1 λ 2 )(T t) E y [ t + ɛe y [(χ 1 χ 2 )(Y ξ T )] ( f1 f 2 )(Ys ξ ) ) ds] (λ i, χ i ) solution of (EP) with χ i bounded by below and f = f i (i = 1, 2). ξ optimal feedback for (EP) with f = f 2. Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 13 / 30

The Convergence Theorem Theorem lim ɛ 0 V ɛ (t, x, y) = V (t, x) locally uniformly, V being the unique solution of { V t + H ( x, D x V, DxxV 2 ) = 0 in (0, T ) R n, V (T, x) = g(x) in R n satisfying V (t, x) C(1 + x α ). Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 14 / 30

Tools V ɛ equibounded uniformly in ɛ < u ɛ V ɛ Exists ρ (0, 1) s.t satisfies V ɛ (t, x, y) C(1 + x α + y α ) u ɛ (t, x, y) = (T t)[ɛρ(1 + y 2 ) γ 2 Then the relaxed semilimits are finite! V (t, x) = V R (t, x) = < u ɛ V ɛ lim inf ɛ 0,(t,x ) (t,x) lim sup ɛ 0,(t,x ) (t,x) y B R (0) 2f 1 0 ] + inf g inf V ɛ (t, x, y), y R m sup V ɛ (t, x, y). Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 15 / 30

V is a super solution of the limit PDE Tools Perturbed test function method, evolving from Evans (periodic homogenisation) and Alvarez-M.Bardi (singular perturbations with bounded fast variables). approximation of (EP) by truncation Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 16 / 30

Approximation of (EP) by truncation (EP) R { By Ichihara results, λ R 1 2 yχ R + 1 θ D yχ R θ = f (y) (f 0 y α 1 R + R) in R m, χ R (0) = 0. it has a unique pair of solutions (λ R, χ R ) R C 2 (R m ) such that χ R Φ γ 1 Rθ χ R (y) C(1 + y γ 1 Rθ ) Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 17 / 30

Approximation of (EP) by truncation Theorem There exists a sequence R j as j s.t. the pair of solutions (λ R, χ R ) of (EP) R with χ R Φ γ 1 converges to the unique solution Rθ (λχ) of (EP) such that χ Φ γ Sketch of the proof 0 < R < R C not depending on R sup Dχ R C B R Classical theory for quasilinear elliptic equations + Schauder s Theory = χ R 2+Γ,B R is bounded by a constant not depending on R > R. Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 18 / 30

In particular, {χ R } R>R is pre-compact. Namely, R j as j and v C 2 (R m ) s.t. χ Rj v, Dχ Rj Dv, D 2 χ Rj D 2 v in C 2 (R m ) as j Formula (F) gives λ λ Rj λ Rj λ Rj+1 IMPLIES there exists a convergence subsequence, λ Rj c R Conclusion: (λ Rj, χ Rj ) (c, v). BUT χ Rj Φ γ 1 and χ Rj (0) = 0, Rθ so lim χ Rj = v Φ γ and v(0) = 0. We are in the right class for which there is uniqueness for (EP), v = χ, c = λ. Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 19 / 30

Perturbed test function method V is a super solution of the limit PDE () in (0, T ) R n? Fix an arbitrary ( t, x) (0, T ) R n. This means, if ψ is a smooth function such that ψ( t, x) = V ( t, x) and V ψ has a strict minimum at ( t, x) then ψ t ( t, x) + H( x, D x ψ( t, x), D 2 xxψ( t, x)) 0. WE ARGUE BY CONTRADICTION. Assume that there exists η > 0 such that ψ t ( t, x) + H( x, D x ψ( t, x), D 2 xxψ( t, x)) < 2η < 0. Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 20 / 30

Perturbed test function method Perturbed test function: ψ ɛ (t, x, y) = ψ(t, x) + ɛχ R (y). χ R, with R = R j, in the conditions of last Theorem with f substituted by I(y) = inf x x, t t < 1 f t,x (y) R (f t,x (y) = H(x, y, D x ψ(t, x), Dxxψ(t, 2 x), 0)) Then ψ ɛ satisfies in ψt ɛ + H(x, y, D x ψ ɛ, Dxxψ 2 ɛ, 0) 1 2ɛ yψ ɛ + 1 D yψ ɛ θ < 0 θ ɛ Q R =] t 1 R, t + 1 R [ B 1 R ( x) R m. Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 21 / 30

Perturbed test function method V ɛ (t, x, y) ψ ɛ (t, x, y) u ɛ (t, x, y) ψ(t, x) ɛc ( 1 + y γ 1 θr ) >. Hence it exists lim inf ɛ 0,(t,x ) (t,x) inf y R m(v ɛ ψ ɛ )(t, x, y) >. Since ψ ɛ is bounded by below, we can conclude that lim inf ɛ 0,(t,x ) (t,x) inf y R m(v ɛ ψ ɛ )(t, x, y) = (V ψ)(t, x). But ( t, x) is a strict minimum point of V ψ so the above relaxed lower limit is > 0 on Q R. Hence we can find ζ > 0 : V ɛ ζ ψ ɛ on Q R for ɛ small. Claim V ɛ ζ ψ ɛ in Q R and Contradiction on the black board. Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 22 / 30

V R is a sub solution of a perturbed PDE { V t + H ( x, D x V, DxxV 2 ) 1 R V (T, x) = g(x) in R n = 0 in (0, T ) Rn Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 23 / 30

V R is a sub solution of a perturbed PDE Tools Perturbed test function method approximation of (EP) by state constraints S(y) = sup x x, t t < 1 f t,x (y) R sub quadratic case (1 < θ 2) λ R 1 2 yχ R + 1 θ D yχ R θ = S(y) in B R (0), χ R + as y B R (0), χ R (0) = 0. super quadratic (θ > 2) λ R 1 2 yχ R + 1 θ D yχ R θ = S(y) in B R (0), λ R 1 2 yχ R + 1 θ D yχ R θ S(y) χ R (0) = 0. on B R (0), Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 24 / 30

V R is a sub solution of a perturbed PDE Comparison Results for such problems (T. Tchamba s Phd thesis [super quadratic case] + [Barles-Da Lio, 2006] [sub quadratic case] H is the minimum of the constants for which (EP) has a solution φ C 2 (R m ) Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 25 / 30

sup R VR is a sub solution of the limit PDE V R is a sub solution of { V t + H ( x, D x V, DxxV 2 ) 1 R = 0 in (0, T ) Rn V (T, x) = g(x) in R n for all R large enough Since the supremum of sub solutions is a sub solution, we see that sup R VR is a sub solution of the limit PDE { V t + H ( x, D x V, DxxV 2 ) = 0 in (0, T ) R n V (T, x) = g(x) in R n Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 26 / 30

Comparison Principle and Uniform Convergence Comparison principle between sub and super solution to parabolic problems satisfying (see [Da Lio - Ley 2011]), gives V (t, x) K (1 + x α ) uniqueness of solution V of CP, V (t, x) sup R VR (t, x), then V = sup R VR = V and, as ɛ 0, V ɛ (t, x, y) V (t, x) locally uniformly. Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 27 / 30

Work in progress and close perspectives associate to the limit PDE a "limit control problem"; list examples; re-obtain and extend Naoyuki Ichihara results using only PDE methods (partially done); simpler formula "(F)"? Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 28 / 30

Grazie! Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 29 / 30

"Sadko in the Underwater Kingdom" by Ilya Repin Joao Meireles (Università di Padova) SP of SCP with Unbounded FV Rome, November 2014 30 / 30