Numerical schemes of resolution of stochastic optimal control HJB equation
|
|
- Randolph Terry
- 5 years ago
- Views:
Transcription
1 Numerical schemes of resolution of stochastic optimal control HJB equation Elisabeth Ottenwaelter Journée des doctorants 7 mars 2007 Équipe COMMANDS Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
2 Outline 1 Previous results Model problem Generalized Finite Differences (GFD) 2 A fast algorithm for the 2D HJB equation of stochastic control Structure of 2D diffusion matrices Stern-Brocot tree Decomposition of the scaled diffusion matrix Projection error Numerical results 3 Other schemes : implicit and splitting Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
3 Outline 1 Previous results Model problem Generalized Finite Differences (GFD) 2 A fast algorithm for the 2D HJB equation of stochastic control Structure of 2D diffusion matrices Stern-Brocot tree Decomposition of the scaled diffusion matrix Projection error Numerical results 3 Other schemes : implicit and splitting Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
4 Model problem T Min IE l(t, y(t), u(t))dt + l F (y(t )); τ { (P τ,x ) dy(t) = f (t, y(t), u(t))dt + σ(t, y(t), u(t))dw(t), y(τ) = x, u(t) U, τ [0, T ], t [τ, T ]. y(t) IR n, u(t) IR m state and control variable, l and l F, real functions, running and final cost, f : IR IR n IR m IR n the drift, σ mapping into the space of n r matrices, w standard r dimensional Brownian motion, l, l F, f and σ Lipschitz and bounded. Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
5 HJB Equation V value function of problem (P τ,x ), unique viscosity solution of v t (t, x) = inf {l(t, x, u) + f (t, x, u) v x(t, x) + a(t, x, u) v xx (t, x)}, u U for all t, x [0, T ] IR n. v(t, x) = l F (x), for all x IR n. (HJB) a(t, x, u) := 1 2 σ(t, x, u)σ(t, x, u) covariance matrix, A B := n i,j=1 A ijb ij scalar product associated with the Frobenius norm, Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
6 Previous results Lions and Mercier (1980), Menaldi (1989) : Classical finite difference methods, Kushner (1977), Kushner and Dupuis (1992) : Markov chain approximation, Camilli and Falcone (1995) : methods based on a priori time discretization (and the related dynamic programming principle for discrete time problems), Krylov (2000) : an error estimate of a large class of discretization schemes. Barles and Jakobsen (2002, 2003) : improvements of the error estimates. Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
7 Outline 1 Previous results Model problem Generalized Finite Differences (GFD) 2 A fast algorithm for the 2D HJB equation of stochastic control Structure of 2D diffusion matrices Stern-Brocot tree Decomposition of the scaled diffusion matrix Projection error Numerical results 3 Other schemes : implicit and splitting Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
8 Discretization Space and time discretization h 1,..., h n space discretization steps, x k := n i=1 k ie i point of the state space, k Z n, Q N number of time steps, h 0 := T /Q time step, t q := qh 0, q = 0,..., Q. v q k approximation of the value function V at (t, x) = (t q, x k ). Upwind finite difference operator ( D ± ϕ k )i = ϕ k+e i ϕ k h i if f (t q, x k, u) i 0, ϕ k ϕ k ei h i if not. ϕ = {ϕ k } real valued function over Z n. Second order finite difference operator associated with ξ Z n ξ ϕ k := ϕ k+ξ 2ϕ k + ϕ k ξ Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
9 Discretization Space and time discretization h 1,..., h n space discretization steps, x k := n i=1 k ie i point of the state space, k Z n, Q N number of time steps, h 0 := T /Q time step, t q := qh 0, q = 0,..., Q. v q k approximation of the value function V at (t, x) = (t q, x k ). Upwind finite difference operator ( D ± ϕ k )i = ϕ k+e i ϕ k h i if f (t q, x k, u) i 0, ϕ k ϕ k ei h i if not. ϕ = {ϕ k } real valued function over Z n. Second order finite difference operator associated with ξ Z n ξ ϕ k := ϕ k+ξ 2ϕ k + ϕ k ξ Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
10 Discretization Space and time discretization h 1,..., h n space discretization steps, x k := n i=1 k ie i point of the state space, k Z n, Q N number of time steps, h 0 := T /Q time step, t q := qh 0, q = 0,..., Q. v q k approximation of the value function V at (t, x) = (t q, x k ). Upwind finite difference operator ( D ± ϕ k )i = ϕ k+e i ϕ k h i if f (t q, x k, u) i 0, ϕ k ϕ k ei h i if not. ϕ = {ϕ k } real valued function over Z n. Second order finite difference operator associated with ξ Z n ξ ϕ k := ϕ k+ξ 2ϕ k + ϕ k ξ Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
11 Strong consistency Stencil S : finite subset of Z n \ {0}, a h := {a ij /h i h j } : scaled covariance matrix, Approximation of the second-order term in the HJB equation αq,k,ξ u ξv q k, are to be set ξ S where αu q,k,ξ Strongly consistent (1) approximation of a(t, x, u) Dxx 2 if αq,k,ξ u ξξ = a h (t q, x k, u), for all k Z n. ξ S (1) Consistency of generalized finite difference schemes for the stochastic HJB equation. Bonnans, Zidani (2003) Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
12 Explicit backward scheme v q 1 k v q k = inf h 0 u U v Q k = l F l(t q, x k, u) + f (t q, x k, u) Dq,k u v q k + αq,k,ξ u ξv q k ξ S If the coefficients αq,k,ξ u are nonnegative and n i=1 the scheme is monotone. f i h i + 2 trace a h 1 h 0. When min i h i 0, we may take h 0 = C min i ( h 2 i ), for C > 0 small enough (depending on f and a). Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
13 Scheme properties Strong consistency and monotonicity GFD are a particular case of consistent Markov chain approximations convergent in view of Kushner and Dupuis (1992), monotonicity, stability and consistence convergence (2), Krylov, Barles and Jacobsen hypotheses satisfied, the error estimates apply. (2) Barles and Souganidis (1991) Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
14 Computation of coefficients α u q,k,ξ To be done at each point of the grid, for each time step, possibly for each control, Solution of a linear program, Expensive if the stencil is large. size(s) := max{ ξ ; ξ S} Main result (3) For 2D problems : an algorithm for computing the coefficients in O(size(S)) operations, For nonconsistent problems : computation of the closest consistent matrix in O(size(S)) operations. The closest consistent matrix for stencil of size p max is computed in O(1) operations after having obtained the closest consistent matrix for stencil of size p max 1. (3) A fast algorithm for the two dimensional HJB equation of stochastic control, Bonnans, Ottenwaelter, Zidani (2004) Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
15 Outline 1 Previous results Model problem Generalized Finite Differences (GFD) 2 A fast algorithm for the 2D HJB equation of stochastic control Structure of 2D diffusion matrices Stern-Brocot tree Decomposition of the scaled diffusion matrix Projection error Numerical results 3 Other schemes : implicit and splitting Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
16 Cone of positive semidefinite matrices z 3 Q IV D H 13 Q III 1 Ω H 2 Q II C ( ) a11 a 12 a 12 a 22 a 11 2a12 a 22 O 1 Q I z 1 z 2 Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
17 Cone of positive semidefinite matrices ξ 2 z 2 0 q III q II Q II Q I z 1 < z 3 Ω z 1 z 3 q IV q I Q III Q IV O ξ 1 z 2 < 0 Figure: Correspondence of regions ξ D IR 2 ξξ C Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
18 Stencil O Figure: Family relations in regular grid Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
19 Outline 1 Previous results Model problem Generalized Finite Differences (GFD) 2 A fast algorithm for the 2D HJB equation of stochastic control Structure of 2D diffusion matrices Stern-Brocot tree Decomposition of the scaled diffusion matrix Projection error Numerical results 3 Other schemes : implicit and splitting Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
20 Stern-Brocot tree Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
21 Outline 1 Previous results Model problem Generalized Finite Differences (GFD) 2 A fast algorithm for the 2D HJB equation of stochastic control Structure of 2D diffusion matrices Stern-Brocot tree Decomposition of the scaled diffusion matrix Projection error Numerical results 3 Other schemes : implicit and splitting Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
22 Algorithm DECOMP Ω 1 0 Figure: Correspondence of directions p max S pmax ( Polyhedral ) cone C(S pmax ) Data : p max N, ε= dist a a, C(S p max ) relative error. Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
23 Main result Theorem Algorithm DECOMP provides a decomposition of a h with a relative error lower than ε, and stops after at most p max iterations. The cost of each iteration is O(1) operations, and hence, its total cost is no more than O(p max ). Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
24 Outline 1 Previous results Model problem Generalized Finite Differences (GFD) 2 A fast algorithm for the 2D HJB equation of stochastic control Structure of 2D diffusion matrices Stern-Brocot tree Decomposition of the scaled diffusion matrix Projection error Numerical results 3 Other schemes : implicit and splitting Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
25 Projection error Lemma The distance from a PSD matrix a to C(S pmax ) is at most ε pmax a, and ε pmax = p 2 max + 1 p max 2 2 p 2 max p 2 max. Conversely, given ε > 0, the distance from a to C(S pmax ) is at most ε when p max p ε, with 1 ε p ε := 2 ε 2 ε. 1 ε 2 Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
26 Solution error Remark If consistency does not hold, and ε = 0, then algorithm DECOMP computes the decomposition of the projection of a(t, x, u) onto C(S pmax ). In that case, the numerical scheme can be interpreted as a consistent approximation for the perturbed HJB equation. v, v : resp. the solution of the HJB equation and the (well-defined) solution of the perturbed HJB equation, When the step size vanishes, the limit of error between the two solutions is v v, Combining Jakobsen and Karlsen estimates of this error (2002) with the previous Lemma, we obtain v v C a a 1/2 C ε 1/2 p max where C and C do not depend on p max. C /p max Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
27 Outline 1 Previous results Model problem Generalized Finite Differences (GFD) 2 A fast algorithm for the 2D HJB equation of stochastic control Structure of 2D diffusion matrices Stern-Brocot tree Decomposition of the scaled diffusion matrix Projection error Numerical results 3 Other schemes : implicit and splitting Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
28 An uncontrolled problem Test function 1 { W (t, x1, x 2 ) = (1 + t) sin x 1 sin x 2 0 x 1 π; 0 x 2 π; 0 t 1. ( sin(x1 + x where σ(t, x 1, x 2 ) = 2 ) β 0 cos(x 1 + x 2 ) 0 β f (t, x 1, x 2 ) = 0 ) Ex 1 : p max = 5, β 2 = 0.1 a C Ex 2 : p max = 5, β 2 = 0 a C Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
29 Numerical results log e Ex 2 5 Ex log x Figure: Error vs discretization step, p max = 5 Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
30 Optimal control Test function 2 { W (t, x1, x 2 ) = (1 + t) sin x 1 sin x 2 1 x 1 1; 1 x 2 1; 0 t 0.5 where σ(t, x 1, x 2 ) = ( 2 sin(x1 + x 2 ) 2 cos(x1 + x 2 ) ) does not depend on the control, f (t, x, u) = u, u u2 2 1 is the control. Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
31 Numerical results log10 e 1.6 p = p = p = log10 x p = Figure: Error vs discretization step, optimal control, p max = 1, 2, 4, 10 Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
32 Outline 1 Previous results Model problem Generalized Finite Differences (GFD) 2 A fast algorithm for the 2D HJB equation of stochastic control Structure of 2D diffusion matrices Stern-Brocot tree Decomposition of the scaled diffusion matrix Projection error Numerical results 3 Other schemes : implicit and splitting Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
33 Implicit and splitting schemes Explicit Euler scheme needs t = O ( ( x) 2). Implicit Euler scheme allows large time steps, but large scale linear systems to solve at each time step. Since our scheme expresses the evolution operator as a sum of rank-one diffusion operators, whose directions are given by the stencil, a natural alternative is to use a splitting decomposition method, for which up to S (or 2 S for second-order schemes) tridiagonal linear systems have to be solved at each time step. Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
34 Cost For 2D systems : on a square grid, in the case h 1 = h 2, N h = O(1/h 1 ) unknowns per column, the bandwidth is as much as p max N h, in the case of stencil S pmax. The cost of factorization is, when p max N h, of the order of p 2 max N 4 h For the Laplace operator we have p max = 1 and solving the implicit scheme is already expensive. Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
35 Implicit scheme where and V (t + h 0, x) V (t, x) h 0 = inf u U {F(t + h 0, x, u, V )}, t h 0 {1,..., N t }, x IR N, V (0, x) = Φ(x), x IR N, F (t, x, u, V ) = l(t, x, u) + F 0 (t, x, u, V ) := F i (t, x, u, V ) := F i (t, x, u, V ) := 1 i S F i (t, x, u, V ) l(t, x, u), UW i (f, V )(t, x, u) + α ξ i (t, x, u) ξ i V (t, x), 1 i N α ξ i (t, x, u) ξ i V (t, x), N < i S UW i (f, V ): upwind finite difference first order operator associated to f. Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
36 Implicit splitting scheme Minimization w.r.t. the control u : Howard algorithm. Loop on the stencil directions inside a time step. V (0, x) = Φ(x), x IR N, and t h 0 {1,..., N t } : û(0, x) argmin F (0, x, u, V ) u U V 0 (x) = V (t, x) V i+1 (x) = V i (x) + h 0 F i (t + h 0, x, û(t, x), V i+1 ), V (t + h 0, x) = V S +1 (x) û(t + h 0, x) argmin {F (t + h 0, x, u, V )}. u U i = 0,..., S, Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
37 Scheme properties Since monotonicity and consistency convergence (Barles and Souganidis, 1991), we study the conditions of monotonicity and consistency for the splitting scheme inside a time step, for a loop on the directions. Monotonicity For each direction the implicit scheme is monotone, The splitting scheme is monotone as a composition of monotone schemes. Consistency : open problem. Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
38 Numerical test Test function : 2D Gaussian function W (t, x 1, x 2 ) = 1 1 2π(2t + 1) e 2(2t + 1) ( a22 x 2 1 2a 12x 1 x 2 + a 11 x 2 2 ) (a 11 a 22 a 2 12 ) Solution of the PDE W t (t, x 1, x 2 ) = a W xx (t, x 1, x 2 ) where a = ξξ ξ 2 + ξ ξ ξ 2, t 0 = 0.5. Tests with ξ = ( 1 0 ) ( 2, ξ = 1 ). Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
39 Numerical results : simple splitting Simple splitting N x Error Order Time N t sec sec sec sec sec sec 642 Remark t = O( x) Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
40 Numerical results : Strang splitting N x Strang splitting Error Order Time N t sec sec sec sec sec sec 642 Remark Strang splitting divides the error by 2. Elisabeth Ottenwaelter (INRIA et CMAP) Numerical schemes for HJB 7 mars / 38
A FAST ALGORITHM FOR THE TWO DIMENSIONAL HJB EQUATION OF STOCHASTIC CONTROL
ESAIM: MAN Vol. 8, N o 4, 004, pp. 7 75 DOI: 0.05/man:00404 ESAIM: Mathematical Modelling and Numerical Analysis A FAST ALGORITHM FOR THE TWO DIMENSIONAL HJB EQUATION OF STOCHASTIC CONTROL J. Frédéric
More informationControlled Diffusions and Hamilton-Jacobi Bellman Equations
Controlled Diffusions and Hamilton-Jacobi Bellman Equations Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Winter 2014 Emo Todorov (UW) AMATH/CSE 579, Winter
More informationA Narrow-Stencil Finite Difference Method for Hamilton-Jacobi-Bellman Equations
A Narrow-Stencil Finite Difference Method for Hamilton-Jacobi-Bellman Equations Xiaobing Feng Department of Mathematics The University of Tennessee, Knoxville, U.S.A. Linz, November 23, 2016 Collaborators
More informationA Concise Course on Stochastic Partial Differential Equations
A Concise Course on Stochastic Partial Differential Equations Michael Röckner Reference: C. Prevot, M. Röckner: Springer LN in Math. 1905, Berlin (2007) And see the references therein for the original
More informationHigh-order filtered schemes for time-dependent second order HJB equations
High-order filtered schemes for time-dependent second order HJB equations Olivier Bokanowski, Athena Picarelli, and Christoph Reisinger November 15, 2016 Abstract In this paper, we present and analyse
More informationSuboptimal feedback control of PDEs by solving Hamilton-Jacobi Bellman equations on sparse grids
Suboptimal feedback control of PDEs by solving Hamilton-Jacobi Bellman equations on sparse grids Jochen Garcke joint work with Axel Kröner, INRIA Saclay and CMAP, Ecole Polytechnique Ilja Kalmykov, Universität
More informationIntertwining of Markov processes
January 4, 2011 Outline Outline First passage times of birth and death processes Outline First passage times of birth and death processes The contact process on the hierarchical group 1 0.5 0-0.5-1 0 0.2
More informationPreliminary Examination, Numerical Analysis, August 2016
Preliminary Examination, Numerical Analysis, August 2016 Instructions: This exam is closed books and notes. The time allowed is three hours and you need to work on any three out of questions 1-4 and any
More informationFast Marching Methods for Front Propagation Lecture 1/3
Outline for Front Propagation Lecture 1/3 M. Falcone Dipartimento di Matematica SAPIENZA, Università di Roma Autumn School Introduction to Numerical Methods for Moving Boundaries ENSTA, Paris, November,
More informationSolution of Stochastic Optimal Control Problems and Financial Applications
Journal of Mathematical Extension Vol. 11, No. 4, (2017), 27-44 ISSN: 1735-8299 URL: http://www.ijmex.com Solution of Stochastic Optimal Control Problems and Financial Applications 2 Mat B. Kafash 1 Faculty
More informationOn continuous time contract theory
Ecole Polytechnique, France Journée de rentrée du CMAP, 3 octobre, 218 Outline 1 2 Semimartingale measures on the canonical space Random horizon 2nd order backward SDEs (Static) Principal-Agent Problem
More informationAsymptotic Perron Method for Stochastic Games and Control
Asymptotic Perron Method for Stochastic Games and Control Mihai Sîrbu, The University of Texas at Austin Methods of Mathematical Finance a conference in honor of Steve Shreve s 65th birthday Carnegie Mellon
More informationMean field games and related models
Mean field games and related models Fabio Camilli SBAI-Dipartimento di Scienze di Base e Applicate per l Ingegneria Facoltà di Ingegneria Civile ed Industriale Email: Camilli@dmmm.uniroma1.it Web page:
More informationu(0) = u 0, u(1) = u 1. To prove what we want we introduce a new function, where c = sup x [0,1] a(x) and ɛ 0:
6. Maximum Principles Goal: gives properties of a solution of a PDE without solving it. For the two-point boundary problem we shall show that the extreme values of the solution are attained on the boundary.
More informationThe HJB-POD approach for infinite dimensional control problems
The HJB-POD approach for infinite dimensional control problems M. Falcone works in collaboration with A. Alla, D. Kalise and S. Volkwein Università di Roma La Sapienza OCERTO Workshop Cortona, June 22,
More informationMatrix stabilization using differential equations.
Matrix stabilization using differential equations. Nicola Guglielmi Universitá dell Aquila and Gran Sasso Science Institute, Italia NUMOC-2017 Roma, 19 23 June, 2017 Inspired by a joint work with Christian
More informationNonlinear representation, backward SDEs, and application to the Principal-Agent problem
Nonlinear representation, backward SDEs, and application to the Principal-Agent problem Ecole Polytechnique, France April 4, 218 Outline The Principal-Agent problem Formulation 1 The Principal-Agent problem
More informationSébastien Chaumont a a Institut Élie Cartan, Université Henri Poincaré Nancy I, B. P. 239, Vandoeuvre-lès-Nancy Cedex, France. 1.
A strong comparison result for viscosity solutions to Hamilton-Jacobi-Bellman equations with Dirichlet condition on a non-smooth boundary and application to parabolic problems Sébastien Chaumont a a Institut
More informationSome Properties of the Augmented Lagrangian in Cone Constrained Optimization
MATHEMATICS OF OPERATIONS RESEARCH Vol. 29, No. 3, August 2004, pp. 479 491 issn 0364-765X eissn 1526-5471 04 2903 0479 informs doi 10.1287/moor.1040.0103 2004 INFORMS Some Properties of the Augmented
More informationOn the Multi-Dimensional Controller and Stopper Games
On the Multi-Dimensional Controller and Stopper Games Joint work with Yu-Jui Huang University of Michigan, Ann Arbor June 7, 2012 Outline Introduction 1 Introduction 2 3 4 5 Consider a zero-sum controller-and-stopper
More informationAdaptive estimation of the copula correlation matrix for semiparametric elliptical copulas
Adaptive estimation of the copula correlation matrix for semiparametric elliptical copulas Department of Mathematics Department of Statistical Science Cornell University London, January 7, 2016 Joint work
More informationIMPROVING POLICIES FOR HAMILTON-JACOBI-BELLMAN EQUATIONS
IMPROVING POLICIES FOR HAMILTON-JACOBI-BELLMAN EQUATIONS BY POSTPROCESSING Jan Blechschmidt Roland Herzog January 8, 2018 A postprocessing strategy for optimal controls obtained via numerical solution
More informationFiltered scheme and error estimate for first order Hamilton-Jacobi equations
and error estimate for first order Hamilton-Jacobi equations Olivier Bokanowski 1 Maurizio Falcone 2 2 1 Laboratoire Jacques-Louis Lions, Université Paris-Diderot (Paris 7) 2 SAPIENZA - Università di Roma
More informationp 1 ( Y p dp) 1/p ( X p dp) 1 1 p
Doob s inequality Let X(t) be a right continuous submartingale with respect to F(t), t 1 P(sup s t X(s) λ) 1 λ {sup s t X(s) λ} X + (t)dp 2 For 1 < p
More informationImplicit explicit numerical schemes for Parabolic Integro-Differential Equations. Maya Briani. LUISS Guido Carli
Implicit explicit numerical schemes for Parabolic Integro-Differential Equations Maya Briani LUISS Guido Carli & Istituto per le Applicazioni del Calcolo - CNR Joint work with R. Natalini (Istituto per
More informationLecture 5. If we interpret the index n 0 as time, then a Markov chain simply requires that the future depends only on the present and not on the past.
1 Markov chain: definition Lecture 5 Definition 1.1 Markov chain] A sequence of random variables (X n ) n 0 taking values in a measurable state space (S, S) is called a (discrete time) Markov chain, if
More informationOn Stopping Times and Impulse Control with Constraint
On Stopping Times and Impulse Control with Constraint Jose Luis Menaldi Based on joint papers with M. Robin (216, 217) Department of Mathematics Wayne State University Detroit, Michigan 4822, USA (e-mail:
More informationHeterogeneous Agent Models in Continuous Time Part I
Heterogeneous Agent Models in Continuous Time Part I Benjamin Moll Princeton University of Chicago, 28 September 2015 What this mini-course is about Many interesting questions require thinking about distributions
More informationGeneralized transition waves and their properties
Generalized transition waves and their properties Henri Berestycki a and François Hamel b a EHESS, CAMS, 54 Boulevard Raspail, F-75006 Paris, France & University of Chicago, Department of Mathematics 5734
More informationPDEs in Image Processing, Tutorials
PDEs in Image Processing, Tutorials Markus Grasmair Vienna, Winter Term 2010 2011 Direct Methods Let X be a topological space and R: X R {+ } some functional. following definitions: The mapping R is lower
More informationOrdinary Differential Equation Theory
Part I Ordinary Differential Equation Theory 1 Introductory Theory An n th order ODE for y = y(t) has the form Usually it can be written F (t, y, y,.., y (n) ) = y (n) = f(t, y, y,.., y (n 1) ) (Implicit
More informationUniformly Uniformly-ergodic Markov chains and BSDEs
Uniformly Uniformly-ergodic Markov chains and BSDEs Samuel N. Cohen Mathematical Institute, University of Oxford (Based on joint work with Ying Hu, Robert Elliott, Lukas Szpruch) Centre Henri Lebesgue,
More informationProf. Erhan Bayraktar (University of Michigan)
September 17, 2012 KAP 414 2:15 PM- 3:15 PM Prof. (University of Michigan) Abstract: We consider a zero-sum stochastic differential controller-and-stopper game in which the state process is a controlled
More informationProper Orthogonal Decomposition in PDE-Constrained Optimization
Proper Orthogonal Decomposition in PDE-Constrained Optimization K. Kunisch Department of Mathematics and Computational Science University of Graz, Austria jointly with S. Volkwein Dynamic Programming Principle
More informationMathematical Methods for Neurosciences. ENS - Master MVA Paris 6 - Master Maths-Bio ( )
Mathematical Methods for Neurosciences. ENS - Master MVA Paris 6 - Master Maths-Bio (2014-2015) Etienne Tanré - Olivier Faugeras INRIA - Team Tosca November 26th, 2014 E. Tanré (INRIA - Team Tosca) Mathematical
More informationIndex. higher order methods, 52 nonlinear, 36 with variable coefficients, 34 Burgers equation, 234 BVP, see boundary value problems
Index A-conjugate directions, 83 A-stability, 171 A( )-stability, 171 absolute error, 243 absolute stability, 149 for systems of equations, 154 absorbing boundary conditions, 228 Adams Bashforth methods,
More informationNumerical Optimal Control Overview. Moritz Diehl
Numerical Optimal Control Overview Moritz Diehl Simplified Optimal Control Problem in ODE path constraints h(x, u) 0 initial value x0 states x(t) terminal constraint r(x(t )) 0 controls u(t) 0 t T minimize
More informationHeterogeneous Agent Models in Continuous Time Part I
Heterogeneous Agent Models in Continuous Time Part I Benjamin Moll Princeton Rochester, 1 March 2017 What this lecture is about Many interesting questions require thinking about distributions Why are income
More informationReflected Brownian Motion
Chapter 6 Reflected Brownian Motion Often we encounter Diffusions in regions with boundary. If the process can reach the boundary from the interior in finite time with positive probability we need to decide
More informationDeterministic Dynamic Programming
Deterministic Dynamic Programming 1 Value Function Consider the following optimal control problem in Mayer s form: V (t 0, x 0 ) = inf u U J(t 1, x(t 1 )) (1) subject to ẋ(t) = f(t, x(t), u(t)), x(t 0
More informationMONOTONE SCHEMES FOR DEGENERATE AND NON-DEGENERATE PARABOLIC PROBLEMS
MONOTONE SCHEMES FOR DEGENERATE AND NON-DEGENERATE PARABOLIC PROBLEMS by MIRNA LIMIC M.B.A. Rochester Institute of Technology, 2 B.Sc., Economics and Business, 999 A THESIS SUBMITTED IN PARTIAL FULFILLMENT
More informationA review of stability and dynamical behaviors of differential equations:
A review of stability and dynamical behaviors of differential equations: scalar ODE: u t = f(u), system of ODEs: u t = f(u, v), v t = g(u, v), reaction-diffusion equation: u t = D u + f(u), x Ω, with boundary
More informationBrownian motion. Samy Tindel. Purdue University. Probability Theory 2 - MA 539
Brownian motion Samy Tindel Purdue University Probability Theory 2 - MA 539 Mostly taken from Brownian Motion and Stochastic Calculus by I. Karatzas and S. Shreve Samy T. Brownian motion Probability Theory
More informationprocess on the hierarchical group
Intertwining of Markov processes and the contact process on the hierarchical group April 27, 2010 Outline Intertwining of Markov processes Outline Intertwining of Markov processes First passage times of
More informationDynamical systems with Gaussian and Levy noise: analytical and stochastic approaches
Dynamical systems with Gaussian and Levy noise: analytical and stochastic approaches Noise is often considered as some disturbing component of the system. In particular physical situations, noise becomes
More informationSchrodinger Bridges classical and quantum evolution
.. Schrodinger Bridges classical and quantum evolution Tryphon Georgiou University of Minnesota! Joint work with Michele Pavon and Yongxin Chen! LCCC Workshop on Dynamic and Control in Networks Lund, October
More informationChapter Two: Numerical Methods for Elliptic PDEs. 1 Finite Difference Methods for Elliptic PDEs
Chapter Two: Numerical Methods for Elliptic PDEs Finite Difference Methods for Elliptic PDEs.. Finite difference scheme. We consider a simple example u := subject to Dirichlet boundary conditions ( ) u
More informationAn iterative procedure for constructing subsolutions of discrete-time optimal control problems
An iterative procedure for constructing subsolutions of discrete-time optimal control problems Markus Fischer version of November, 2011 Abstract An iterative procedure for constructing subsolutions of
More informationDefect-based a-posteriori error estimation for implicit ODEs and DAEs
1 / 24 Defect-based a-posteriori error estimation for implicit ODEs and DAEs W. Auzinger Institute for Analysis and Scientific Computing Vienna University of Technology Workshop on Innovative Integrators
More informationBackground. Background. C. T. Kelley NC State University tim C. T. Kelley Background NCSU, Spring / 58
Background C. T. Kelley NC State University tim kelley@ncsu.edu C. T. Kelley Background NCSU, Spring 2012 1 / 58 Notation vectors, matrices, norms l 1 : max col sum... spectral radius scaled integral norms
More informationu n 2 4 u n 36 u n 1, n 1.
Exercise 1 Let (u n ) be the sequence defined by Set v n = u n 1 x+ u n and f (x) = 4 x. 1. Solve the equations f (x) = 1 and f (x) =. u 0 = 0, n Z +, u n+1 = u n + 4 u n.. Prove that if u n < 1, then
More informationGradient Descent and Implementation Solving the Euler-Lagrange Equations in Practice
1 Lecture Notes, HCI, 4.1.211 Chapter 2 Gradient Descent and Implementation Solving the Euler-Lagrange Equations in Practice Bastian Goldlücke Computer Vision Group Technical University of Munich 2 Bastian
More informationTowards stability and optimality in stochastic gradient descent
Towards stability and optimality in stochastic gradient descent Panos Toulis, Dustin Tran and Edoardo M. Airoldi August 26, 2016 Discussion by Ikenna Odinaka Duke University Outline Introduction 1 Introduction
More informationMathematical Methods - Lecture 9
Mathematical Methods - Lecture 9 Yuliya Tarabalka Inria Sophia-Antipolis Méditerranée, Titane team, http://www-sop.inria.fr/members/yuliya.tarabalka/ Tel.: +33 (0)4 92 38 77 09 email: yuliya.tarabalka@inria.fr
More informationKernel Method: Data Analysis with Positive Definite Kernels
Kernel Method: Data Analysis with Positive Definite Kernels 2. Positive Definite Kernel and Reproducing Kernel Hilbert Space Kenji Fukumizu The Institute of Statistical Mathematics. Graduate University
More informationFast convergent finite difference solvers for the elliptic Monge-Ampère equation
Fast convergent finite difference solvers for the elliptic Monge-Ampère equation Adam Oberman Simon Fraser University BIRS February 17, 211 Joint work [O.] 28. Convergent scheme in two dim. Explicit solver.
More informationMatrix concentration inequalities
ELE 538B: Mathematics of High-Dimensional Data Matrix concentration inequalities Yuxin Chen Princeton University, Fall 2018 Recap: matrix Bernstein inequality Consider a sequence of independent random
More informationChap. 1. Some Differential Geometric Tools
Chap. 1. Some Differential Geometric Tools 1. Manifold, Diffeomorphism 1.1. The Implicit Function Theorem ϕ : U R n R n p (0 p < n), of class C k (k 1) x 0 U such that ϕ(x 0 ) = 0 rank Dϕ(x) = n p x U
More informationBrownian Motion. 1 Definition Brownian Motion Wiener measure... 3
Brownian Motion Contents 1 Definition 2 1.1 Brownian Motion................................. 2 1.2 Wiener measure.................................. 3 2 Construction 4 2.1 Gaussian process.................................
More informationChapter 3 Second Order Linear Equations
Partial Differential Equations (Math 3303) A Ë@ Õæ Aë áöß @. X. @ 2015-2014 ú GA JË@ É Ë@ Chapter 3 Second Order Linear Equations Second-order partial differential equations for an known function u(x,
More informationON THE POLICY IMPROVEMENT ALGORITHM IN CONTINUOUS TIME
ON THE POLICY IMPROVEMENT ALGORITHM IN CONTINUOUS TIME SAUL D. JACKA AND ALEKSANDAR MIJATOVIĆ Abstract. We develop a general approach to the Policy Improvement Algorithm (PIA) for stochastic control problems
More informationStochastic optimal control with rough paths
Stochastic optimal control with rough paths Paul Gassiat TU Berlin Stochastic processes and their statistics in Finance, Okinawa, October 28, 2013 Joint work with Joscha Diehl and Peter Friz Introduction
More informationRough Burgers-like equations with multiplicative noise
Rough Burgers-like equations with multiplicative noise Martin Hairer Hendrik Weber Mathematics Institute University of Warwick Bielefeld, 3.11.21 Burgers-like equation Aim: Existence/Uniqueness for du
More information{σ x >t}p x. (σ x >t)=e at.
3.11. EXERCISES 121 3.11 Exercises Exercise 3.1 Consider the Ornstein Uhlenbeck process in example 3.1.7(B). Show that the defined process is a Markov process which converges in distribution to an N(0,σ
More informationSome SDEs with distributional drift Part I : General calculus. Flandoli, Franco; Russo, Francesco; Wolf, Jochen
Title Author(s) Some SDEs with distributional drift Part I : General calculus Flandoli, Franco; Russo, Francesco; Wolf, Jochen Citation Osaka Journal of Mathematics. 4() P.493-P.54 Issue Date 3-6 Text
More informationConcentration inequalities for Feynman-Kac particle models. P. Del Moral. INRIA Bordeaux & IMB & CMAP X. Journées MAS 2012, SMAI Clermond-Ferrand
Concentration inequalities for Feynman-Kac particle models P. Del Moral INRIA Bordeaux & IMB & CMAP X Journées MAS 2012, SMAI Clermond-Ferrand Some hyper-refs Feynman-Kac formulae, Genealogical & Interacting
More informationStability of optimization problems with stochastic dominance constraints
Stability of optimization problems with stochastic dominance constraints D. Dentcheva and W. Römisch Stevens Institute of Technology, Hoboken Humboldt-University Berlin www.math.hu-berlin.de/~romisch SIAM
More informationBridging the Gap between Center and Tail for Multiscale Processes
Bridging the Gap between Center and Tail for Multiscale Processes Matthew R. Morse Department of Mathematics and Statistics Boston University BU-Keio 2016, August 16 Matthew R. Morse (BU) Moderate Deviations
More informationFinite difference method for elliptic problems: I
Finite difference method for elliptic problems: I Praveen. C praveen@math.tifrbng.res.in Tata Institute of Fundamental Research Center for Applicable Mathematics Bangalore 560065 http://math.tifrbng.res.in/~praveen
More informationWeak convergence and large deviation theory
First Prev Next Go To Go Back Full Screen Close Quit 1 Weak convergence and large deviation theory Large deviation principle Convergence in distribution The Bryc-Varadhan theorem Tightness and Prohorov
More informationWeak Convergence of Numerical Methods for Dynamical Systems and Optimal Control, and a relation with Large Deviations for Stochastic Equations
Weak Convergence of Numerical Methods for Dynamical Systems and, and a relation with Large Deviations for Stochastic Equations Mattias Sandberg KTH CSC 2010-10-21 Outline The error representation for weak
More informationProximal-like contraction methods for monotone variational inequalities in a unified framework
Proximal-like contraction methods for monotone variational inequalities in a unified framework Bingsheng He 1 Li-Zhi Liao 2 Xiang Wang Department of Mathematics, Nanjing University, Nanjing, 210093, China
More informationLinear-Quadratic-Gaussian (LQG) Controllers and Kalman Filters
Linear-Quadratic-Gaussian (LQG) Controllers and Kalman Filters Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Winter 204 Emo Todorov (UW) AMATH/CSE 579, Winter
More informationRegularity of the density for the stochastic heat equation
Regularity of the density for the stochastic heat equation Carl Mueller 1 Department of Mathematics University of Rochester Rochester, NY 15627 USA email: cmlr@math.rochester.edu David Nualart 2 Department
More informationLecture on Parameter Estimation for Stochastic Differential Equations. Erik Lindström
Lecture on Parameter Estimation for Stochastic Differential Equations Erik Lindström Recap We are interested in the parameters θ in the Stochastic Integral Equations X(t) = X(0) + t 0 µ θ (s, X(s))ds +
More informationSolvability of G-SDE with Integral-Lipschitz Coefficients
Solvability of G-SDE with Integral-Lipschitz Coefficients Yiqing LIN joint work with Xuepeng Bai IRMAR, Université de Rennes 1, FRANCE http://perso.univ-rennes1.fr/yiqing.lin/ ITN Marie Curie Workshop
More informationOn the Bellman equation for control problems with exit times and unbounded cost functionals 1
On the Bellman equation for control problems with exit times and unbounded cost functionals 1 Michael Malisoff Department of Mathematics, Hill Center-Busch Campus Rutgers University, 11 Frelinghuysen Road
More informationEL2520 Control Theory and Practice
EL2520 Control Theory and Practice Lecture 8: Linear quadratic control Mikael Johansson School of Electrical Engineering KTH, Stockholm, Sweden Linear quadratic control Allows to compute the controller
More informationIntroduction to numerical schemes
236861 Numerical Geometry of Images Tutorial 2 Introduction to numerical schemes Heat equation The simple parabolic PDE with the initial values u t = K 2 u 2 x u(0, x) = u 0 (x) and some boundary conditions
More informationLecture 1: Introduction to low-rank tensor representation/approximation. Center for Uncertainty Quantification. Alexander Litvinenko
tifica Lecture 1: Introduction to low-rank tensor representation/approximation Alexander Litvinenko http://sri-uq.kaust.edu.sa/ KAUST Figure : KAUST campus, 5 years old, approx. 7000 people (include 1400
More informationA Semi-Lagrangian discretization of non linear fokker Planck equations
A Semi-Lagrangian discretization of non linear fokker Planck equations E. Carlini Università Sapienza di Roma joint works with F.J. Silva RICAM, Linz November 2-25, 206 / 3 Outline A Semi-Lagrangian scheme
More informationCITY UNIVERSITY LONDON
No: CITY UNIVERSITY LONDON BEng (Hons)/MEng (Hons) Degree in Civil Engineering BEng (Hons)/MEng (Hons) Degree in Civil Engineering with Surveying BEng (Hons)/MEng (Hons) Degree in Civil Engineering with
More informationWeak solutions to Fokker-Planck equations and Mean Field Games
Weak solutions to Fokker-Planck equations and Mean Field Games Alessio Porretta Universita di Roma Tor Vergata LJLL, Paris VI March 6, 2015 Outlines of the talk Brief description of the Mean Field Games
More informationSupplemental Material for KERNEL-BASED INFERENCE IN TIME-VARYING COEFFICIENT COINTEGRATING REGRESSION. September 2017
Supplemental Material for KERNEL-BASED INFERENCE IN TIME-VARYING COEFFICIENT COINTEGRATING REGRESSION By Degui Li, Peter C. B. Phillips, and Jiti Gao September 017 COWLES FOUNDATION DISCUSSION PAPER NO.
More informationMulti-Factor Finite Differences
February 17, 2017 Aims and outline Finite differences for more than one direction The θ-method, explicit, implicit, Crank-Nicolson Iterative solution of discretised equations Alternating directions implicit
More informationSchrodinger Bridges classical and quantum evolution
Schrodinger Bridges classical and quantum evolution Tryphon Georgiou University of Minnesota! Joint work with Michele Pavon and Yongxin Chen! Newton Institute Cambridge, August 11, 2014 http://arxiv.org/abs/1405.6650
More informationNumerical approximation for optimal control problems via MPC and HJB. Giulia Fabrini
Numerical approximation for optimal control problems via MPC and HJB Giulia Fabrini Konstanz Women In Mathematics 15 May, 2018 G. Fabrini (University of Konstanz) Numerical approximation for OCP 1 / 33
More informationEntropy stable schemes for degenerate convection-diffusion equations
Entropy stable schemes for degenerate convection-diffusion equations Silvia Jerez 1 Carlos Parés 2 ModCompShock, Paris 6-8 Decmber 216 1 CIMAT, Guanajuato, Mexico. 2 University of Malaga, Málaga, Spain.
More informationBuilding Solutions to Nonlinear Elliptic and Parabolic Partial Differential Equations
Building Solutions to Nonlinear Elliptic and Parabolic Partial Differential Equations Adam Oberman University of Texas, Austin http://www.math.utexas.edu/~oberman Fields Institute Colloquium January 21,
More informationConvex Optimization on Large-Scale Domains Given by Linear Minimization Oracles
Convex Optimization on Large-Scale Domains Given by Linear Minimization Oracles Arkadi Nemirovski H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology Joint research
More informationBSDEs and PDEs with discontinuous coecients Applications to homogenization K. Bahlali, A. Elouain, E. Pardoux. Jena, March 2009
BSDEs and PDEs with discontinuous coecients Applications to homogenization K. Bahlali, A. Elouain, E. Pardoux. Jena, 16-20 March 2009 1 1) L p viscosity solution to 2nd order semilinear parabolic PDEs
More informationSplitting methods with boundary corrections
Splitting methods with boundary corrections Alexander Ostermann University of Innsbruck, Austria Joint work with Lukas Einkemmer Verona, April/May 2017 Strang s paper, SIAM J. Numer. Anal., 1968 S (5)
More informationNecessary optimality conditions for optimal control problems with nonsmooth mixed state and control constraints
Necessary optimality conditions for optimal control problems with nonsmooth mixed state and control constraints An Li and Jane J. Ye Abstract. In this paper we study an optimal control problem with nonsmooth
More informationUnconditionally stable scheme for Riccati equation
ESAIM: Proceedings, Vol. 8, 2, 39-52 Contrôle des systèmes gouvernés par des équations aux dérivées partielles http://www.emath.fr/proc/vol.8/ Unconditionally stable scheme for Riccati equation François
More informationPartial Differential Equations II
Partial Differential Equations II CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Justin Solomon CS 205A: Mathematical Methods Partial Differential Equations II 1 / 28 Almost Done! Homework
More informationPARAREAL TIME DISCRETIZATION FOR PARABOLIC CONTROL PROBLEM
PARAREAL TIME DISCRETIZATION FOR PARABOLIC CONTROL PROBLEM Daoud S Daoud Dept. of Mathematics Eastern Mediterranean University Famagusta-North Cyprus via Mersin 10-Turkey. e mail: daoud.daoud@emu.edu.tr
More informationOn nonlocal Hamilton-Jacobi equations related to jump processes, some recent results
On nonlocal Hamilton-Jacobi equations related to jump processes, some recent results Daria Ghilli Institute of mathematics and scientic computing, Graz, Austria 25/11/2016 Workshop "Numerical methods for
More informationImplicitly Defined High-Order Operator Splittings for Parabolic and Hyperbolic Variable-Coefficient PDE Using Modified Moments
Implicitly Defined High-Order Operator Splittings for Parabolic and Hyperbolic Variable-Coefficient PDE Using Modified Moments James V. Lambers September 24, 2008 Abstract This paper presents a reformulation
More informationOptimal Trade Execution with Instantaneous Price Impact and Stochastic Resilience
Optimal Trade Execution with Instantaneous Price Impact and Stochastic Resilience Ulrich Horst 1 Humboldt-Universität zu Berlin Department of Mathematics and School of Business and Economics Vienna, Nov.
More informationComputation Fluid Dynamics
Computation Fluid Dynamics CFD I Jitesh Gajjar Maths Dept Manchester University Computation Fluid Dynamics p.1/189 Garbage In, Garbage Out We will begin with a discussion of errors. Useful to understand
More information