An adaptive RBF-based Semi-Lagrangian scheme for HJB equations

Transcription

1 An adaptive RBF-based Semi-Lagrangian scheme for HJB equations Roberto Ferretti Università degli Studi Roma Tre Dipartimento di Matematica e Fisica Numerical methods for Hamilton Jacobi equations in optimal control and related fields, Linz, 22 Nov 2016 joint work with G. Ferretti, O. Junge, A. Schrieber UNIVERSITÀ DEGLI STUDI ROMA TRE Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, UNIVERSITÀ 22 Nov 2016 DEGLI 1 / STUDI 41

2 Numerical Methods in Optimal Control: algorithms, analysis and applications Rome (INdAM), June 19 23, Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

3 Outline 1 Optimal Control via Dynamic Programming 2 Semi-Lagrangian approximation Construction of the SL scheme Convergence of the SL scheme Lagrange vs Least squares space reconstruction operators 3 Shepard/SL scheme The single-level case Full-grid multilevel case Adaptive multilevel case 4 Numerical examples 1D examples 2D example 5 Implementation and efficiency issues 6 Conclusions Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

4 Optimal Control and dynamic programming Controlled dynamical system { ẏ(s) = f (y(s), u(s)), y(t 0 ) = x, the objective of Optimal Control theory is to find and optimal pair (y, u ) minimizing the Cost functional (e.g., of the infinite horizon form) J x (u) = 0 g(y x (s; u), u(s))e λs ds. In Dynamic Prograqmming (DP) techniques, the starting point is to introduce the value function: v(x) = inf u U J x(u) which represents the best possible performance of the system when starting from the state x. Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

5 Optimal Control and dynamic programming Controlled dynamical system { ẏ(s) = f (y(s), u(s)), y(t 0 ) = x, the objective of Optimal Control theory is to find and optimal pair (y, u ) minimizing the Cost functional (e.g., of the infinite horizon form) J x (u) = 0 g(y x (s; u), u(s))e λs ds. In Dynamic Prograqmming (DP) techniques, the starting point is to introduce the value function: v(x) = inf u U J x(u) which represents the best possible performance of the system when starting from the state x. Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

6 From Dynamic Programming to Hamilton Jacobi Bellman equation DP Principle gives a characterization of value function via the functional equation (which holds for any τ > 0): { τ } v(x) = inf g(y x (s; u), u(s))e λs ds + e λτ v(y x (τ; u)) u U 0 Assuming v differentiable and passing to the limit for τ 0, we obtain the Hamilton Jacobi Bellman (HJB) equation: λv(x) + sup { f (x, u) Dv(x) g(x, u)} = 0. u U In general, in control probelms, v / C 1, and therefore a suitable weak concept of solution (usually, in the viscosity sense) must be used. Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

7 Construction of a Semi-Lagrangian scheme for the HJB equation (1) Time discretization Once fixed a time step t, the system evolution is approximated by a one-step scheme, and the integral by a numerical quadrature, e.g., Euler scheme and rectangle quadrature: { } t v(x) = inf u U 0 g(y x (s; u), u(s))e λs ds + e λ t v(y x ( t; u)) v t { (x) = min t g(x, u) + e λ t v t (x + t f (x, u)) }. u U Clearly, more accurate choices are possible. Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

8 Construction of a Semi-Lagrangian scheme for the HJB equation (1) Time discretization Once fixed a time step t, the system evolution is approximated by a one-step scheme, and the integral by a numerical quadrature, e.g., Euler scheme and rectangle quadrature: { } t v(x) = inf u U 0 g(y x (s; u), u(s))e λs ds + e λ t v(y x ( t; u)) v t { (x) = min t g(x, u) + e λ t v t (x + t f (x, u)) }. u U Clearly, more accurate choices are possible. Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

9 Construction of a Semi-Lagrangian scheme for the HJB equation (2) Space discretization A space grid of nodes {x 1,..., x N } with space step x is set, the semi-discrete equation is collocated at the nodes, and the computation of v is replaced by an interpolation I : v t (x) = min u U { t g(x, u) + e λ t v t (x + t f (x, u)) }. { v j = min t g(xj, u) + e λ t I [V ](x j + t f (x j, u)) }. u U In a similar way, time-dependent HJB equations can also be handled Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

10 Construction of a Semi-Lagrangian scheme for the HJB equation (2) Space discretization A space grid of nodes {x 1,..., x N } with space step x is set, the semi-discrete equation is collocated at the nodes, and the computation of v is replaced by an interpolation I : v t (x) = min u U { t g(x, u) + e λ t v t (x + t f (x, u)) }. { v j = min t g(xj, u) + e λ t I [V ](x j + t f (x j, u)) }. u U In a similar way, time-dependent HJB equations can also be handled Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

11 Accuracy, stability and convergence issues Accuracy: consistency rate follow from the rate of convergence of the elementary building blocks (one-step scheme, quadrature formula, space reconstruction) Monotonicity: this is the most usual stability requirement, sometimes relaxed as quasi-monotonicity: a monotonicity defect of o( t) is allowed Convergence: so far, the most general convergence theory (Barles Souganidis theorem) requires consistency, L stability and monotonicity (or at least quasi-monotonicity) Convergence estimates: for SL schemes and Lipschitz solutions, the typical convergence estimate reads v j v(x j ) C 1 t p + C 2 x t Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

12 Accuracy, stability and convergence issues Accuracy: consistency rate follow from the rate of convergence of the elementary building blocks (one-step scheme, quadrature formula, space reconstruction) Monotonicity: this is the most usual stability requirement, sometimes relaxed as quasi-monotonicity: a monotonicity defect of o( t) is allowed Convergence: so far, the most general convergence theory (Barles Souganidis theorem) requires consistency, L stability and monotonicity (or at least quasi-monotonicity) Convergence estimates: for SL schemes and Lipschitz solutions, the typical convergence estimate reads v j v(x j ) C 1 t p + C 2 x t Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

13 Accuracy, stability and convergence issues Accuracy: consistency rate follow from the rate of convergence of the elementary building blocks (one-step scheme, quadrature formula, space reconstruction) Monotonicity: this is the most usual stability requirement, sometimes relaxed as quasi-monotonicity: a monotonicity defect of o( t) is allowed Convergence: so far, the most general convergence theory (Barles Souganidis theorem) requires consistency, L stability and monotonicity (or at least quasi-monotonicity) Convergence estimates: for SL schemes and Lipschitz solutions, the typical convergence estimate reads v j v(x j ) C 1 t p + C 2 x t Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

14 Lagrange interpolation The basic choice for the space reconstruction operator is the P 1 /Q 1 (piecewise linear/multilinear) Lagrange interpolation I 1 [ ]. In a single space dimension and on a generic interval I j = [x j, x j+1 ], it reads: I 1 [V ](x) = x x j+1 v j + x x j v j+1 x j x j+1 x j+1 x j = x j+1 x v j + x x j x x v j+1 = ψ j (x)v j + ψ j+1 (x)v j+1 For x I j, this is a convex combination of v j and v j+1. This ensures monotonicity and L stability. Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

15 Shepard s method A second space reconstruction will be used here, namely Shepard s method. Rather than an interpolation, this is a weighted least squares technique, based on compactly supported, positive Radial Basis Functions (RBFs). Let X = {x 1,, x N } Ω R d be the set of grid nodes Let w(x, y) : Ω Ω R suitable weight functions, such that w(x, y) = 0 if x y 2 is large, that is, w(x, y) = Φ ρ (x y) with Φ ρ : R d R ρ plays the role of a scale parameter for w (e.g., the radius of its support) Shepard s approximation for f I [f ](x) = N i=1 w(x i, x) N j=1 w(x j, x) f (x i) Radial Basis Functions are usually considered as more suitable for local refinements this feature can be exploited for adaptive methods Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

16 Shepard s method A second space reconstruction will be used here, namely Shepard s method. Rather than an interpolation, this is a weighted least squares technique, based on compactly supported, positive Radial Basis Functions (RBFs). Let X = {x 1,, x N } Ω R d be the set of grid nodes Let w(x, y) : Ω Ω R suitable weight functions, such that w(x, y) = 0 if x y 2 is large, that is, w(x, y) = Φ ρ (x y) with Φ ρ : R d R ρ plays the role of a scale parameter for w (e.g., the radius of its support) Shepard s approximation for f I [f ](x) = N i=1 w(x i, x) N j=1 w(x j, x) f (x i) Radial Basis Functions are usually considered as more suitable for local refinements this feature can be exploited for adaptive methods Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

17 Shepard s method A second space reconstruction will be used here, namely Shepard s method. Rather than an interpolation, this is a weighted least squares technique, based on compactly supported, positive Radial Basis Functions (RBFs). Let X = {x 1,, x N } Ω R d be the set of grid nodes Let w(x, y) : Ω Ω R suitable weight functions, such that w(x, y) = 0 if x y 2 is large, that is, w(x, y) = Φ ρ (x y) with Φ ρ : R d R ρ plays the role of a scale parameter for w (e.g., the radius of its support) Shepard s approximation for f I [f ](x) = N i=1 w(x i, x) N j=1 w(x j, x) f (x i) Radial Basis Functions are usually considered as more suitable for local refinements this feature can be exploited for adaptive methods Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

18 Shepard s method A second space reconstruction will be used here, namely Shepard s method. Rather than an interpolation, this is a weighted least squares technique, based on compactly supported, positive Radial Basis Functions (RBFs). Let X = {x 1,, x N } Ω R d be the set of grid nodes Let w(x, y) : Ω Ω R suitable weight functions, such that w(x, y) = 0 if x y 2 is large, that is, w(x, y) = Φ ρ (x y) with Φ ρ : R d R ρ plays the role of a scale parameter for w (e.g., the radius of its support) Shepard s approximation for f I [f ](x) = N i=1 w(x i, x) N j=1 w(x j, x) f (x i) Radial Basis Functions are usually considered as more suitable for local refinements this feature can be exploited for adaptive methods Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

19 Single-level Shepard/SL scheme Shepard s approximation for f I [f ](x) = N i=1 w(x i, x) N j=1 w(x j, x) f (x i) = N ψ i (x)f (x i ) Basis functions ψ( ) are positive and with unitary sum, therefore the reconstruction is monotone By Barles Souganidis theorem, the Shepard implementation of the SL scheme is therefore monotone and convergent The reconstruction error is again O( x) on Lipschitz functions, and as a consequence we obtain again i=1 v j v(x j ) C 1 t p + C 2 x t Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

20 Some remarks on least squares RBF reconstructions (1) In presence of singularities, Shepard s method cuts the smaller scales, thus causing a relevant error with respect to the true value of the function Shepard s reconstruction is exact on constants, but is not in general an interpolation, and may have nonzero residuals on the nodes themselves The residual is concentrated in the neighbourhood of singularities in the first derivatives, and this can indicate the need for refinement f (x, y) = 9 (x 5) 2 Function f and reconstruction Residual at nodes Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

21 Some remarks on least squares RBF reconstructions (1) In presence of singularities, Shepard s method cuts the smaller scales, thus causing a relevant error with respect to the true value of the function Shepard s reconstruction is exact on constants, but is not in general an interpolation, and may have nonzero residuals on the nodes themselves The residual is concentrated in the neighbourhood of singularities in the first derivatives, and this can indicate the need for refinement f (x, y) = 9 (x 5) 2 Function f and reconstruction Residual at nodes Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

22 Some remarks on least squares RBF reconstructions (1) In presence of singularities, Shepard s method cuts the smaller scales, thus causing a relevant error with respect to the true value of the function Shepard s reconstruction is exact on constants, but is not in general an interpolation, and may have nonzero residuals on the nodes themselves The residual is concentrated in the neighbourhood of singularities in the first derivatives, and this can indicate the need for refinement f (x, y) = 9 (x 5) 2 Function f and reconstruction Residual at nodes Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

23 Some remarks on least squares RBF reconstructions (2) Constants: Shepard s method has no numerical error, both at the nodes and between nodes Linear functions: On structured uniform grids, Shepard s method is exact at the nodes, and has a local error expressed as f (x) I [f ](x) C 1 (ρ)f (x) x with C 1 (ρ) decreasing with the scale parameter ρ Quadratic functions: There is a nonzero residual at the nodes, and this introduces a further term in the error: f (x) I [f ](x) C 1 (ρ)f (x) x + C 2 (ρ)f (x) x 2 with C 2 (ρ) increasing with the scale parameter ρ Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

24 Some remarks on least squares RBF reconstructions (2) Constants: Shepard s method has no numerical error, both at the nodes and between nodes Linear functions: On structured uniform grids, Shepard s method is exact at the nodes, and has a local error expressed as f (x) I [f ](x) C 1 (ρ)f (x) x with C 1 (ρ) decreasing with the scale parameter ρ Quadratic functions: There is a nonzero residual at the nodes, and this introduces a further term in the error: f (x) I [f ](x) C 1 (ρ)f (x) x + C 2 (ρ)f (x) x 2 with C 2 (ρ) increasing with the scale parameter ρ Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

25 Some remarks on least squares RBF reconstructions (2) Constants: Shepard s method has no numerical error, both at the nodes and between nodes Linear functions: On structured uniform grids, Shepard s method is exact at the nodes, and has a local error expressed as f (x) I [f ](x) C 1 (ρ)f (x) x with C 1 (ρ) decreasing with the scale parameter ρ Quadratic functions: There is a nonzero residual at the nodes, and this introduces a further term in the error: f (x) I [f ](x) C 1 (ρ)f (x) x + C 2 (ρ)f (x) x 2 with C 2 (ρ) increasing with the scale parameter ρ Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

26 Shepard/SL scheme on the eikonal equation We apply the Shepard/SL scheme to the problem vx 2 {v t + 2 = 0 v(x, 0) = max(1 (x 2) 2, 0) Shepard s reconstruction smoothes out the singularity Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

27 Remarks Efficiency issues the Shepard/SL scheme is slower than the P 1 /SL scheme At nodes close to the boundary, numerical solution is less accurate Numerical solution is smoothed out at switching points/curves Why choosing adaptivity? A better resolution of singularities may be expected Refinements are avoided in smooth regions (where the scheme is already accurate enough) Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

28 Remarks Efficiency issues the Shepard/SL scheme is slower than the P 1 /SL scheme At nodes close to the boundary, numerical solution is less accurate Numerical solution is smoothed out at switching points/curves Why choosing adaptivity? A better resolution of singularities may be expected Refinements are avoided in smooth regions (where the scheme is already accurate enough) Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

29 Remarks Efficiency issues the Shepard/SL scheme is slower than the P 1 /SL scheme At nodes close to the boundary, numerical solution is less accurate Numerical solution is smoothed out at switching points/curves Why choosing adaptivity? A better resolution of singularities may be expected Refinements are avoided in smooth regions (where the scheme is already accurate enough) Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

30 Remarks Efficiency issues the Shepard/SL scheme is slower than the P 1 /SL scheme At nodes close to the boundary, numerical solution is less accurate Numerical solution is smoothed out at switching points/curves Why choosing adaptivity? A better resolution of singularities may be expected Refinements are avoided in smooth regions (where the scheme is already accurate enough) Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

31 Remarks Efficiency issues the Shepard/SL scheme is slower than the P 1 /SL scheme At nodes close to the boundary, numerical solution is less accurate Numerical solution is smoothed out at switching points/curves Why choosing adaptivity? A better resolution of singularities may be expected Refinements are avoided in smooth regions (where the scheme is already accurate enough) Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

32 Remarks Efficiency issues the Shepard/SL scheme is slower than the P 1 /SL scheme At nodes close to the boundary, numerical solution is less accurate Numerical solution is smoothed out at switching points/curves Why choosing adaptivity? A better resolution of singularities may be expected Refinements are avoided in smooth regions (where the scheme is already accurate enough) Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

33 Full-grid multilevel approximation Let X 1, X 2,..., X n be a sequence of space grids for the levels 1, 2,..., n, and let w j (x, y) = δ d j Φ((x y)/δ j ) be weight functions with supports of radius δ j (in our case, δ j 2). At each level, the multilevel approximation is constructed working on the residual of the previous levels, each time on a finer grid. Let e 0 = f, and let s j := s ej 1,X j be the approximation of the residual e j 1 at the level j and e j its residual. Then, the multilevel reconstruction reads: e 0 = f f 1 = s 1, e 1 = f s 1, f 2 = s 1 + s 2,. e j 1 = f (s 1 + s s j 1 ), f j = s 1 + s s j. Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

34 Full-grid multilevel approximation Let X 1, X 2,..., X n be a sequence of space grids for the levels 1, 2,..., n, and let w j (x, y) = δ d j Φ((x y)/δ j ) be weight functions with supports of radius δ j (in our case, δ j 2). At each level, the multilevel approximation is constructed working on the residual of the previous levels, each time on a finer grid. Let e 0 = f, and let s j := s ej 1,X j be the approximation of the residual e j 1 at the level j and e j its residual. Then, the multilevel reconstruction reads: e 0 = f f 1 = s 1, e 1 = f s 1, f 2 = s 1 + s 2,. e j 1 = f (s 1 + s s j 1 ), f j = s 1 + s s j. Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

35 Full-grid multilevel approximation Let X 1, X 2,..., X n be a sequence of space grids for the levels 1, 2,..., n, and let w j (x, y) = δ d j Φ((x y)/δ j ) be weight functions with supports of radius δ j (in our case, δ j 2). At each level, the multilevel approximation is constructed working on the residual of the previous levels, each time on a finer grid. Let e 0 = f, and let s j := s ej 1,X j be the approximation of the residual e j 1 at the level j and e j its residual. Then, the multilevel reconstruction reads: e 0 = f f 1 = s 1, e 1 = f s 1, f 2 = s 1 + s 2,. e j 1 = f (s 1 + s s j 1 ), f j = s 1 + s s j. Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

36 Nested multilevel grids Multilevel 1D grid binary tree Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

37 Adaptive multilevel approximation Space reconstruction is no longer monotone if implemented in multilevel form A convergence result can be proved for full-grid, N-levels approximation, without exploiting Barles Souganidis theorem. The error estimate is of the form v j v(x j ) C 1 t p + C 2 x 2 N 1 t In the adaptive code: At a certain node x j, a new level is created only if e j > τ (with τ a suitable refinement threshold) The threshold τ must discriminate between smooth and nonsmooth regions. For example, defining τ = x 3/2 sets the threshold at an intermediate order of magnitude (Convergence analysis for the adaptive case in progress) Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

38 Adaptive multilevel approximation Space reconstruction is no longer monotone if implemented in multilevel form A convergence result can be proved for full-grid, N-levels approximation, without exploiting Barles Souganidis theorem. The error estimate is of the form v j v(x j ) C 1 t p + C 2 x 2 N 1 t In the adaptive code: At a certain node x j, a new level is created only if e j > τ (with τ a suitable refinement threshold) The threshold τ must discriminate between smooth and nonsmooth regions. For example, defining τ = x 3/2 sets the threshold at an intermediate order of magnitude (Convergence analysis for the adaptive case in progress) Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

39 Adaptive multilevel approximation Space reconstruction is no longer monotone if implemented in multilevel form A convergence result can be proved for full-grid, N-levels approximation, without exploiting Barles Souganidis theorem. The error estimate is of the form v j v(x j ) C 1 t p + C 2 x 2 N 1 t In the adaptive code: At a certain node x j, a new level is created only if e j > τ (with τ a suitable refinement threshold) The threshold τ must discriminate between smooth and nonsmooth regions. For example, defining τ = x 3/2 sets the threshold at an intermediate order of magnitude (Convergence analysis for the adaptive case in progress) Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

40 Adaptive multilevel approximation Space reconstruction is no longer monotone if implemented in multilevel form A convergence result can be proved for full-grid, N-levels approximation, without exploiting Barles Souganidis theorem. The error estimate is of the form v j v(x j ) C 1 t p + C 2 x 2 N 1 t In the adaptive code: At a certain node x j, a new level is created only if e j > τ (with τ a suitable refinement threshold) The threshold τ must discriminate between smooth and nonsmooth regions. For example, defining τ = x 3/2 sets the threshold at an intermediate order of magnitude (Convergence analysis for the adaptive case in progress) Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

41 Pseudo-code (1D case) Require: nodes, τ, n_steps, maximum number of levels N_lev. Ensure: Value function, value, control, feedback. 1: for n = 1,..., n_steps do 2: for each node x do 3: build the first level approximation of value and feedback 4: end for 5: for i = 1,..., N_lev 1 do 6: for each node x at level i do 7: if value(x) SHEPARD(x) > τ then 8: generate 2 nodes at the level i + 1 9: end if 10: end for 11: for each new node x do 12: compute the new approximation of value and feedback 13: compute the space reconstruction of value via the function SHEPARD 14: save in value the new residual 15: end for 16: end for 17: end for Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

42 A time-dependent example 1D vx 2 {v t + 2 = 0 v(x, 0) = max(1 (x 2) 2, 0) Single level Multilevel Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

43 A time-dependent example 1D vx 2 {v t + 2 = 0 v(x, 0) = max(1 (x 2) 2, 0) Single level Multilevel Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

44 A time-dependent example 1D vx 2 {v t + 2 = 0 v(x, 0) = max(1 (x 2) 2, 0) Single level Multilevel Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

45 A time-dependent example 1D vx 2 {v t + 2 = 0 v(x, 0) = max(1 (x 2) 2, 0) Single level Multilevel Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

46 A time-dependent example 1D vx 2 {v t + 2 = 0 v(x, 0) = max(1 (x 2) 2, 0) Single level Multilevel Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

47 A time-dependent example 1D vx 2 {v t + 2 = 0 v(x, 0) = max(1 (x 2) 2, 0) Single level Multilevel Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

48 A time-dependent example 1D vx 2 {v t + 2 = 0 v(x, 0) = max(1 (x 2) 2, 0) Single level Multilevel Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

49 A time-dependent example 1D vx 2 {v t + 2 = 0 v(x, 0) = max(1 (x 2) 2, 0) Single level Multilevel Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

50 A time-dependent example 1D vx 2 {v t + 2 = 0 v(x, 0) = max(1 (x 2) 2, 0) Single level Multilevel Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

51 A time-dependent example 1D vx 2 {v t + 2 = 0 v(x, 0) = max(1 (x 2) 2, 0) Single level Multilevel Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

52 A time-dependent example 1D vx 2 {v t + 2 = 0 v(x, 0) = max(1 (x 2) 2, 0) Single level Multilevel Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

53 A time-dependent example 1D vx 2 {v t + 2 = 0 v(x, 0) = max(1 (x 2) 2, 0) Single level Multilevel Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

54 A stationary example 1D Maximization problem: f (x, u) = u µx + mx ρ n ρ + x ρ g(x, u) = au σ kβx 2 with U = [0, 0.4], λ = 0.1, Ω = [0, 4], and constants a = 2, σ = β = k = 1/2, m = n = 1, ρ = 2, µ = Single level Multilevel Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

55 A stationary example 2D (1) This test is a reformulation in terms of an infinite horizon problem of a chemotherapy control problem. The dynamics is given by the compartment model: { ẋ 1 (t) = a 1 x 1 (t) + 2(1 u(t))a 2 x 2 (t) ẋ 2 (t) = a 1 x 1 (t) a 2 x 2 (t), in which the two state compartments denote two successive phases of cellular growth, with the drug u acting only on the second phase. The running cost is g(x, u) = u + r 1 ẋ 1 + r 2 ẋ 2 = u + r 1 ( a 1 x 1 + 2(1 u)a 2 x 2 ) + r 2 (a 1 x 1 a 2 x 2 ) thus aiming at a reduction of the tumour mass, but avoiding an excessive dose of drug. Here, the parameters have been set as U = [0, 1], λ = 0.3, Ω = [0, 1] 2, and, following the literature, a 1 = 0.197, a 2 = 0.356, r 1 = 6.94 and r 2 = Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

56 A stationary example 2D (2) Single level solution Single level solution Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

57 A stationary example 2D (3) Adaptive 3-level solution Adaptive 3-level grid Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

58 Data structure for the multilevel grid (1) Sparse memorization of the multilevel grid: each row of the matrix contains a list of the nodes at the corresponding level Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

59 Data structure for the multilevel grid (2) Full memorization of the multilevel grid: each row of the matrix contains the full grid at the corresponding level Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

60 Efficiency issues For the implementation in form of a multiple (level-by-level) list: Memorization is very compact, but execution times are high Difficult to keep memory of the nonzero basis functions redundant computation For a full memorization of the multilevel grid: Execution times are faster, but large memory occupation Possible intermediate solutions: Node list, with implementation of a dependency tree of the multilevel grid (nonzero basis functions are found via a tree-based splitting of the computational domain) Full grid with a sparse matrix memorization Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

61 Efficiency issues For the implementation in form of a multiple (level-by-level) list: Memorization is very compact, but execution times are high Difficult to keep memory of the nonzero basis functions redundant computation For a full memorization of the multilevel grid: Execution times are faster, but large memory occupation Possible intermediate solutions: Node list, with implementation of a dependency tree of the multilevel grid (nonzero basis functions are found via a tree-based splitting of the computational domain) Full grid with a sparse matrix memorization Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

62 Efficiency issues For the implementation in form of a multiple (level-by-level) list: Memorization is very compact, but execution times are high Difficult to keep memory of the nonzero basis functions redundant computation For a full memorization of the multilevel grid: Execution times are faster, but large memory occupation Possible intermediate solutions: Node list, with implementation of a dependency tree of the multilevel grid (nonzero basis functions are found via a tree-based splitting of the computational domain) Full grid with a sparse matrix memorization Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

63 Conclusions Pros: Cons: Better approximation in nonsmooth regions Good robustness of the refinement indicator At the moment, the adaptive code is less efficient than the single-level, Lagrange-based scheme Complicate dependence on the choice of the basis and of the refinement threshold Future tasks: Improve the sparse memorization to speed up the reconstruction procedure Fully unstructured refinement Convergence analysis for the adaptive case Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

64 Conclusions Pros: Cons: Better approximation in nonsmooth regions Good robustness of the refinement indicator At the moment, the adaptive code is less efficient than the single-level, Lagrange-based scheme Complicate dependence on the choice of the basis and of the refinement threshold Future tasks: Improve the sparse memorization to speed up the reconstruction procedure Fully unstructured refinement Convergence analysis for the adaptive case Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

65 Conclusions Pros: Cons: Better approximation in nonsmooth regions Good robustness of the refinement indicator At the moment, the adaptive code is less efficient than the single-level, Lagrange-based scheme Complicate dependence on the choice of the basis and of the refinement threshold Future tasks: Improve the sparse memorization to speed up the reconstruction procedure Fully unstructured refinement Convergence analysis for the adaptive case Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

66 References M. Falcone, R. Ferretti, Semi-Lagrangian Approximation Schemes for Linear and Hamilton-Jacobi Equations, SIAM (2014). G. Ferretti, R. Ferretti, O. Junge, A. Schreiber, A multilevel radial basis function scheme for the HJB equation, submitted for IFAC O. Junge, A. Schreiber, Dynamic Programming using Radial Basis Functions, Discrete and Continuous Dynamical Systems, A, 35 (2015). U. Ledzewicz H. Schättler, Optimal bang-bang controls for a two-compartment model in cancer chemotherapy, Journal of Optimization Theory and Applications, 114 (2002), H. Wendland, Local polynomial reproduction and moving least squares approximation, IMA J. Num. Anal., 21 (2001), H. Wendland, Multiscale analysis in Sobolev spaces on bounded domains, Numerische Mathematik, 116 (2010), H. Wendland, Scattered data approximation, Cambridge University Press, Cambridge UK (2005). Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41

67 Thank you for your attention! Roberto Ferretti (Roma Tre) Adaptive multilevel Shepard for HJB SSCMSE Linz, 22 Nov / 41