Multigrid methods for pde optimization

Transcription

1 Multigrid methods for pde optimization Alfio Borzì Università degli Studi del Sannio Dipartimento e Facoltà di Ingegneria, Benevento, Italia Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 1 / 64

2 Introduction Multigrid methods and optimization with partial differential equations are two modern fields of research in applied mathematics and engineering, both starting in the seventies with the works of A. Brandt and W. Hackbusch, and J.L. Lions. PDE-based optimization problems arise in application to medicine biology fluid dynamics mechanics chemical processes...inverse problems, control problems, design problems are large-sized problems many have nonlinear nondifferentiable structure pose new theoretical & scientific computing challenges Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 2 / 64

3 Physiology: Control of cardiac arrhythmia y 1 y y 1 t y 2 t = ky 1 (y 1 a)(y 1 1) y 1 y 2 + σ y 1 + u = [ɛ 0 + µ 1y 2 µ 2 + y 1 ][ y 2 ky 1 (y 1 b 1)] y 1 - transmembrane potential and y 2 - conduttance Austrian Science Fund SFB MOBIS Fast Multigrid Methods for Inverse Problems Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 3 / 64

4 Quantum computing: Control of Bose Einstein condensates The BEC is described by the Gross-Pitaevskii equation i ψ(x, t) = ( 12 ) 2 + V (x, u(t)) + g ψ(x, t) 2 ψ(x, t) Austrian Science Fund FWF Project Quantum optimal control of semiconductor nanostructures Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 4 / 64

5 CFD Shape design optimization: From pipes to engines Navier-Stokes equations, RSM turbulence models, multi-phase, spray and combustion Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 5 / 64

6 Outline of the talk A.B. and V. Schulz, Multigrid methods for PDE optimization, SIAM Review. Optimization with PDE constraints SQP multigrid Schur-complement-smoothing multigrid Collective-smoothing multigrid Multigrid for optimization Convergence theory Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 6 / 64

7 Optimization with PDE constraints Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 7 / 64

8 The formulation of PDE-based optimization problems A model of the governing system A description of the optimization mechanism A criterion that models the purpose of the optimization and its cost We have an infinite-dimensional constrained minimization problem minimize J(y, u) under the constraint c(y, u) = 0 L(y, u, p) = J(y, u)+(c(y, u), p) Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 8 / 64

9 Characterization of the optimizing solution min J(y, u) := h(y) + ν g(u) J : Y U R u U ad s.t. c(y, u) = 0 The existence of cy 1 enables a distinction between y, the state variable, and u U ad U, the optimization variable in the admissible set. So we have the mapping u J(y(u), u) in the form u IFT y(u) J(y(u), u) =: Ĵ(u) reduced objective The solution of min u Uad Ĵ(u) is characterized by the following optimality system c(y, u) = 0 c y (y, u) p = h (y) (ν g (u) + cu p, v u) 0 for all v U ad We have Ĵ(u) = ν g (u) + c u p(u), the reduced gradient. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 9 / 64

10 Elliptic optimization problems Consider the simplest objective where z L 2 is a target function. Linear elliptic optimal control problem J(y, u) = 1 2 y z 2 L 2 + ν 2 u 2 L 2 c(y, u) = y u f Bilinear elliptic parameter identification problem c(y, u) = y u y f we assume that c(y, u) = 0 provides the mapping u y(u). A. B. and M. Vallejos, Multigrid optimization methods for linear and bilinear elliptic optimal control problems, Computing, 82 (2008), pp Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 10 / 64

11 Elliptic optimality systems Linear Bilinear y u = f in Ω p + y = z in Ω νu p = 0 in Ω y = 0 and p = 0 on Ω Ĵ(u) = νu p 2 Ĵ(u) = νi + 2 y uy = f in Ω p + y up = z in Ω νu yp = 0 in Ω y = 0 and p = 0 on Ω Ĵ(u) = νu yp 2 Ĵ(u) = νi + y( + u) 2 y + p( + u) 1 y + y( + u) 1 p Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 11 / 64

12 Quantum control problems Control of Bose Einstein condensates in magnetic microtraps min J(ψ, u) := 1 ( u U 2 1 ψ d ψ(t ) 2 ) + γ T 2 0 ( u(t))2 dt i ψ t (x, t) = ( V (x, u(t)) + g ψ(x, t) 2) ψ(x, t) A. B. and U. Hohenester, Multigrid optimization schemes for solving Bose Einstein condensates control problems, SIAM J. Sci. Comp., 30 (2008), pp Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 12 / 64

13 Optimality system for BEC control The optimal solution is characterized by the optimality system i ψ ( t = 12 ) 2 + V u + g ψ 2 ψ i p ( t = 12 ) 2 + V u + 2g ψ 2 p + g ψ 2 p γü = Re ψ V u u p, with the initial and terminal conditions ψ(0) = ψ 0 and ip(t ) = ψ d ψ(t ) ψ d u(0) = 0, u(t ) = 1. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 13 / 64

14 Optical flow The optical flow (u, v) determines the transformation from an image frame to the next. Find w and I such that { It + w I = 0, in Q = Ω (0, T ), and minimize I (, 0) = Y 1, J(I, w) = 1 N I (x, y, t k ) Y k 2 dω + α Φ( w 2 k=1 Ω 2 Q t 2 )dq + β Ψ( u v 2 )dq + γ Q 2 w 2 dq, Q where α, β, and γ are given optimization weights. A. B., K. Ito, and K. Kunisch, Optimal control formulation for determining optical flow, SIAM J. Sci. Comput., 24(3) (2002), pp Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 14 / 64

15 Optimality system The state equation evolving forward I t + w I = 0. The adjoint equation evolving backwards p t + ( wp) = 0, on t (t k 1, t k ), p(, t + k ) p(, t k ) = I (, t k) Y k, t = t k, for k = 2,... N 1. Two (space-time) elliptic control equations α 2 u t 2 + β [Ψ ( w 2 ) u] + γ ( w) x = I p x, α 2 v t 2 + β [Ψ ( w 2 ) v] + γ ( w) y = I p y, where w 2 = u 2 + v 2. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 15 / 64

16 Multigrid schemes in the SQP approach Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 16 / 64

17 Multigrid SQP schemes These schemes are recommended for u U = R nu (and no further multigrid structure within U). An SQP method iterates over the following steps (0) initialize l = 0, y 0, u 0 (1) solve the adjoint problem cy (y l, u l )p l = J y (y l, u l ) and build the reduced gradient Ĵ(u) l = Ju + cu (y l, u l )p l (2) build some approximation B l H(y l, u l, p l ) of the reduced Hessian, e.g., by Quasi-Newton update formulae 1 (3) solve u = arg min u LU(u l ) 2 u B l u + Ĵ(u) l u, where LU(u l) denotes the linearization of U in u l (4) solve the linear problem c y (y l, u l ) y = (c u (y l, u l ) u + c(y l, u l )) (5) update (y l+1, u l+1 ) = (y l, u l ) + τ ( y, u), where τ is some line-search updating factor in the early iterations. A multigrid scheme is applied in steps (1) and (4). Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 17 / 64

18 Consistency condition for multigrid SQP schemes SQP convergence theory requires that the reduced gradient can be interpreted as a derivative, i.e. we need the consistency condition Ĵ(u) l = u J(y l Ac u (y l, u l )u, u l + u) where A c y (y l, u l ) 1 is the approximation to c y (y l, u l ) 1 defined by the multigrid algorithm for the forward problem. This fact results in the following requirements for the construction of the multigrid components ( ( AI k 1 k = FIk 1) k, AS = ( F S), AI k k 1 = FI k 1 k where A and F label the operators for the adjoint and forward problems. V. Schulz, Solving discretized optimization problems by partially reduced SQP methods, Comput. Vis. Sci. 1 (1998), pp ) Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 18 / 64

19 Schur-complement multigrid approach Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 19 / 64

20 The KKT-matrix Many multigrid optimization approaches use a smoothing concept based on a Schur-complement splitting of the KKT-matrix. Let w = ( y, u, p) be the solution to the equation Aw = y L(y, u, p) u L(y, u, p) c(y, u) =: f. (3) where L(y, u, p) is the Lagrangian of the optimization problem and the operator matrix L yy L yu cy A = L uy L uu cu (4) c y c u 0 is the Karush-Kuhn-Tucker matrix, i.e. matrix of second order derivatives of the Lagrangian of the optimization problem. All variants of SQP methods consider approximations of the matrix A above. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 20 / 64

21 Schur-complement-based smoothing Use the iterative scheme w l = w l 1 + R(f A w l 1 ) For a linear elliptic optimal control problem A = 0 ν I I I 0 I 0 where we can use a multigrid Poisson-solver for inverting. This is the starting point of the early multigrid optimization methods. W. Hackbusch, Fast solution of elliptic control problems, J. Optim. Theory and Appl., 31(1980), pp Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 21 / 64

22 Schur decomposition and Schur complement Consider [ ] A B A = B D with symmetric blocks A and D, and A invertible, is an explicit reformulation of a block-gauss-decomposition, i.e. [ ] [ ] I A A 1 B A 0 = 0 I B S where S = D BA 1 B is the Schur-complement. Iterative Schur-complement solvers are based on the scheme [ w l = w l 1 I à + 1 B 0 I ] [ à 0 B S ] 1 (f A w l 1 ). (5) where à and S are approximations to A and S. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 22 / 64

23 Choice of the blocks, different strategies First tentative choice (range space factorization) [ ] Lyy L A = yu L uy The A-block may not be invertible. This arrangement is not well suited for PDE constrained optimization problems, unlike to variational problems like Stokes or Navier-Stokes [Braess-Sarazin 97, Wittum 88] Interchanging the 2nd and 3rd row and column in the matrix A and identifying [ ] Lyy cy A =, B = [ ] L c y 0 uy cu, D = Luu leads to a nullspace decomposition. L uu Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 23 / 64

24 Choice of the blocks, different strategies With nullspace decomposition the Schur-complement reads S = L uu L uy cy 1 c u cu cy 1 L yu + cu cy 1 L yy cy 1 c u that is the reduced Hessian that characterizes the optimization problem. Coercivity of the reduced Hessian guarantees well-posedness of the overall optimization problem. In the case of a linear control problem one obtains S = ν I 0 1 I I I 1 I 1 I = ν I + ( 1 ) 2 With the choice à = [ Lyy c y c y 0 ] and S = ν I one obtains a transforming smoothing iteration. Th. Dreyer, B. Maar, V. Schulz, Multigrid optimization in applications, J. Comput. Appl. Math., 120 (2000), pp Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 24 / 64

25 Collective smoothing multigrid approach Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 25 / 64

26 CSMG scheme A collective smoothing multigrid (CSMG) approach means solving the optimality system for the state, the adjoint, and the control variables simultaneously in the multigrid process by using collective smoothers for the optimizations variables. A CSMG based scheme aims at realizing the tight coupling in the optimality system along the hierarchy of grids in space and time. The smoothing process: The basic idea is to solve the optimality system at grid or block-grid point level, within the admissible set and consistently with the differential structure of the problem. The CSMG multigrid schemes result robust with respect to opimization parameters and provide mesh-independent convergence. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 26 / 64

27 CSMG smoothing for a linear elliptic control problem y u = f p + y = z (νu p, v u) 0 for all v U ad where U ad := {u L 2 (Ω) u L (x) u(x) u H (x) a.e. in Ω} Let A = y i 1,j y i+1,j y i,j 1 y i,j+1 h 2 f i,j, B = p i 1,j p i+1,j p i,j 1 p i,j+1 h 2 z i,j. Then A + 4y i,j h 2 u i,j = 0, B + 4p i,j + h 2 y i,j = 0, νu i,j p i,j = 0. y i,j (u i,j ) = 1 4 (h2 u i,j A) p i,j (u i,j ) = 1 16 ( h2 u i,j + h 2 A 4B) Setting Ĵ(u) = νu p(u) = 0, we obtain the auxiliary variable ũ i,j = 1 16ν + h 4 (h2 A 4B). Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 27 / 64

28 Collective Smoothing: Projection The new value for u i,j resulting from the smoothing step is given by u H i,j if ũ i,j u H i,j u i,j = ũ i,j if u Li,j < ũ i,j < u H i,j. u Li,j if ũ i,j u Li,j This collective Gauss-Seidel step satisfies the inequality constraint. Example: ũ > u H ; then u = u H. Therefore (v u) 0 for any v U ad, and νu p = νu (4 B h 2 A h 4 u ij )/16 = [(16ν + h 4 )u (4 B h 2 A)]/16 < [(16ν + h 4 )ũ (4 B h 2 A)]/16 = 0. Therefore (νu p) (v u) 0 for all v U ad. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 28 / 64

29 A discovery: Bang-bang control phenomenon Consider ul = 30 and uh = 30, ν = 0, z(x, y ) = sin(4πx) sin(2πy ). mesh ρ(y ), 0.12, 0.12, 0.12, y z ρ(p) r (y ) 0, r (p) , , , The state (left) and the control (right). A. B. and K. Kunisch, A multigrid scheme for elliptic constrained optimal control problems, Comput. Optim. Appl., 31 (2005), pp Alfio Borzı (Universita degli Studi del Sannio, Multigrid Italy) methods for pde optimization 29 / 64

30 Multigrid collective smoothing with FEM Optimality system Define Q Y M U = F, Q P + M Y = Z, (νu P, Φ U) 0, Φ U. n k n k n k n k C 1 = q i,j Y j, C 2 = q i,j P j, C 3 = m i,j Y j, and C 4 = m i,j U j. j=1 j i j=1 j i Y i (U i ) = 1 q i,i (F i C 1 + C 4 + m i,i U i ), P i (U i ) = 1 [q qi,i 2 i,i (Z i C 2 C 3 ) m i,i (F i C 1 + C 4 + m i,i U i )]. Define the auxiliary variable as j=1 j i 1 Ũ i = νqi,i 2 + [q i,i (Z i C 2 C 3 ) m i,i (F i C 1 + C 4 )]. m2 i,i j=1 j i Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 30 / 64

31 Numerical results - CSMG-FEM Control-constrained linear problem Figure: Numerical solutions y (left) and u (right) for the control- constrained linear elliptic optimal control problem with ν = Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 31 / 64

32 CSMG smoothing for bilinear optimization problems Let A = y i 1,j y i+1,j y i,j 1 y i,j+1 h 2 f i,j, B = p i 1,j p i+1,j p i,j 1 p i,j+1 h 2 z i,j. Then A + 4y i,j h 2 u i,j y i,j = 0, B + 4p i,j + h 2 y i,j h 2 u i,j p i,j = 0, νu i,j y i,j p i,j = 0. y i,j (u i,j ) = 1 4 h 2 u i,j A p i,j (u i,j ) = 1 (4 h 2 u i,j ) (h 2 A + h 2 u 2 i,j 4B) Assume (4 h 2 u i,j ) 0 at any ij. Setting Ĵ(u) = νu y(u)p(u) = 0, we obtain u i,j as solution of the quartic polynomial equation νh 6 ui,j 4 12νh4 ui,j νh2 ui,j 2 (64ν + h2 AB)u i,j (h 2 A 2 4AB) = 0. Let the auxiliary variable ũ i,j be the solution of the quartic polynomial νh 6 ui,j 4 12νh4 ui,j νh2 ui,j 2 (64ν + h2 AB)u i,j (h 2 A 2 4AB) = 0. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 32 / 64

33 Bilinear control problem / parameter identification problem min u U J(y, u) := 1 2 y z 2 L 2 + ν 2 u 2 L 2 y u y = f in Ω y = 0 on Ω The (non attainable) target function z is given by { 2, on (0.25, 0.75) (0.25, 0.75) z(x 1, x 2 ) = 1, otherwise. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 33 / 64

34 Numerical results Bilinear control problem / parameter identification problem Figure: Numerical solutions y (left) and u (right) for the control- unconstrained bilinear elliptic optimal control problem with ν = (CSMG, FDM) ν = 10 2 ν = 10 4 mesh CSMG NCG CSMG NCG Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 34 / 64

35 Other developments for elliptic optimization problems CSMG scheme for state-constrained elliptic control problems. Collective smoothing AMG for convection-diffusion optimality systems with jumping coefficients. CSMG schemes for optimality systems and higher-order discretization. Globalization techniques within CSMG schemes applied to non convex optimization problems. CSMG methods for singular control problems. CSMG methods for boundary control problems. CSMG convergence theory in the LFA and BPX frameworks. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 35 / 64

36 CSMG schemes for time-dependent problems Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 36 / 64

37 Reaction diffusion process control through source terms Consider a reaction-diffusion model min u L 2 (Q) J(y, u) t y + G(y) + σ y = u in Q = Ω (0, T ) y = y 0 in Ω {t = 0} y = 0 on Σ = Ω (0, T ) Control required to track a desired trajectory y d (x, t) reach a desired terminal state y T (x) For this purpose, the following objective can be considered J(y, u) = α 2 y y d 2 L 2 (Q) + β 2 y(, T ) y T 2 L 2 (Ω) + ν 2 u 2 L 2 (Q) Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 37 / 64

38 Reaction diffusion process control through boundary terms min u L 2 (Σ) J(y, u) t y + G(y) + σ y = 0 in Q = Ω (0, T ) y = y 0 in Ω {t = 0} y n = u on Σ = Ω (0, T ) Control required to track a desired trajectory y d (x, t) reach a desired terminal state y T (x) For this purpose, the following objective can be considered J(y, u) = α 2 y y d 2 L 2 (Q) + β 2 y(, T ) y T 2 L 2 (Ω) + ν 2 u 2 L 2 (Σ) Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 38 / 64

39 Optimality systems Distributed source control: The solution is characterized by t y + G(y) + σ y = u in Q t p + G (y)p + σ p + α(y y d ) = 0 in Q νu p = 0 in Q y = 0, p = 0 on Σ Neumann boundary control: The optimal solution satisfies t y + G(y) + σ y = 0 in Q t p + G (y)p + σ p + α(y y d ) = 0 in Q νu p = 0 on Σ y = u, p n n = 0 on Σ With initial condition y(x, 0) = y 0 (x) for the state variable (evolving forward in time). And the terminal condition for the adjoint variable (evolving backward in time) p(x, T ) = β(y(x, T ) y T (x)). Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 39 / 64

40 Discretization The optimality systems are discretized by, e.g., finite differences and backward Euler scheme. Ω h defines the set of interior mesh-points, (x i, y j ), 2 i, j N x. The space-time grid is defined by Q h,δt = {(x, t m ) : x Ω h, t m = (m 1)δt, 1 m N t + 1, δt = T /N t } Time difference operators t yh m = y h m y m 1 h δt Example distributed control: and t + ph m = pm+1 h ph m δt t yh m + G(yh m ) + σ h yh m = uh m t + ph m + G (yh m )ph m + σ h ph m + α(yh m ydh) m = 0 νu m h p m h = 0 Boundary control: Consider the optimality system on the boundary and discretize the boundary derivative using a second-order centered scheme. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 40 / 64

41 Coarsening strategies in space-time domains Coarsening strategy Consider L levels, k = 1,..., L. Coarsening in the space directions: h = h k = h 1 /2 k 1. Coarsening in time direction: δt = δt k = δt 1 /s k 1, s = 1 semicoarsening in space; s = 2 standard time coarsening; s = 4 double time coarsening. Transfer operators I k 1 k denotes the injection or FW operator for restriction. Ik 1 k denotes bilinear interpolation in space and If s = 1 no interpolation in time is needed, if s {2, 4} then Ik 1 k corresponds to bilinear interpolation in space and in time. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 41 / 64

42 Collective smoothing: Two essential requirements Coupling between state and control variables. Preserve opposite orientation of state and adjoint equations. Backward Euler discretization t = 0 y 0 y t = T p p 0 Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 42 / 64

43 Time-Splitted Collective Gauss-Seidel Iteration (TS-CGS) y p «(1) i j m y = p «(0)» (1 + 4σγ) + δt G δt/ν + i j m δt(α + G p) (1 + 4σγ) + δt G 1 ry rp «i j m 1 Set the starting approximation. 2 For m = 2,..., N t do 3 For ij in, e.g., lexicographic order do y (1) i j m = y (0) i j m + [ (1 + 4σγ) + δtg ] r y (w) + δt ν r p(w) [ (1 + 4σγ) + δtg ] 2 + δt2 ν (α + G p) (0) i j m p (1) i j N t m+2 = p (0) i j N t m+2 4 end. + [ (1 + 4σγ) + δtg ] r p (w) δt(α + G p) r y (w) [ (1 + 4σγ) + δtg ] 2 + δt2 ν (α + G (0) i j N p) t m+2 Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 43 / 64

44 Time-Line Collective Gauss-Seidel Iteration (TL-CGS) Consider the discrete optimality system at i, j and for all time steps and solve the resulting block-tridiagonal system. ( y p ) (1) i j ( y = p ) (0) i j + M 1 ( ry r p ) The block-tridiagonal system has the following form M = A 2 C 2 B 3 A 3 C 3 B 4 A 4 C 4 C N t B N t +1 A N t +1 B m =» ,A m = [ and C m = i j (1 + 4σγ) + δtg δt ν δt(α + G p) (1 + 4σγ) + δtg» Centered at t m, the entries B m, A m, C m refer to the variables (y, p) at t m 1, t m, and t m+1, respectively. ] Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 44 / 64

45 Smoothing factors by local Fourier analysis -2-4 Log nu gamma Log nu gamma Convergence factors of TL-CGS scheme (left) and TS-CGS scheme η TL-CGS TS-CGS γ \ ν The estimated convergence factors η for TL-CGS and TS-CGS multigrid schemes Alfio(ν Borzì 1 = (Università ν 2 = 1); degli δt Studi = 1/64, del Sannio, α = Multigrid Italy) 1. methods for pde optimization 45 / 64

46 u u Long time intervals: CSMG - Receding horizon techniques Consider the optimal control problem of tracking y d for t 0. Define time windows of size t. In each time window, an optimal control problem with tracking (α = 1) and terminal observation (β = 1) is solved. Multigrid Receding Horizon Scheme (CSMG-RH) Applied to the control of a solid fuel ignition model t y + σ y + δe y = f y y d y y d Time evolution of the state y and the desired trajectory y d (left) and the optimal control u (right) at (x 1, x 2 ) = (0.5, 0.5). Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 46 / 64

47 Other developments for parabolic optimization problems A.B., Multigrid methods for parabolic distributed optimal control problems, J. Comp. Appl. Math, 157 (2003), A.B. and R. Griesse, Distributed optimal control of lambda-omega systems, Journal of Numerical Mathematics, 14 (2006), A.B. and R. Griesse, Experiences with a space-time multigrid method for the optimal control of a chemical turbulence model, Int. J. Numer. Meth. Fluids., 47 (2005), A.B., Space-time multigrid methods for solving unsteady optimal control problems, (Chapter 5) in L.T. Biegler, O. Ghattas, M. Heinkenschloss, D. Keyes and B. van Bloemen Waanders (Eds.), Real-Time PDE-Constrained Optimization, Computational Science and Engineering, Vol. 3, SIAM, Philadelphia, A.B., Solution of lambda-omega systems: Theta-schemes and multigrid methods, Numerische Mathematik, 98(4) (2004), Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 47 / 64

48 MGOPT multigrid for optimization Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 48 / 64

49 MGOPT framework The MGOPT solution to the optimization problem min u Ĵ(u) requires to define a hierarchy of minimization problems min u k Ĵ k (u k ) k = 1, 2,..., L where u k V k and Ĵk( ) is the reduced cost functional. Among spaces V k, restriction operators I k 1 k : V k V k 1 and prolongation operators Ik 1 k : V k 1 V k are defined. Require that (I k 1 k u, v) k 1 = (u, Ik 1 k v) k for all u V k and v V k 1. We also choose an optimization scheme as smoother That provides sufficient reduction for some η (0, 1). Ĵ k (O k (u (l) k u (l) k = O k (u (l 1) k ) )) < Ĵ k (u (l) ) η Ĵ k (u (l) ) 2 Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 49 / 64 k k

50 FAS MGOPT The MGOPT scheme is a FAS scheme applied to the gradient problem Ĵ k (u k ) = 0 where smoothing is replaced by any optimization scheme and a linesearch (damping) is applied to the coarse-grid correction step. FAS Coarse Grid Problem A k (u k ) = f k A k 1 (u k 1 ) = I k 1 k f k + A k 1 (u k 1 ) I k 1 k A k (u k ) }{{} τ MGOPT Coarse Grid Problem Ĵ k (u k ) = 0 Ĵ k 1 (u k 1 ) = Ĵ k 1 (u k 1 ) I k 1 k Ĵ k (u k ) }{{} τ R.M. Lewis and S.G. Nash, Model problems for the multigrid optimization of systems governed by differential equations, SIAM J. Sci. Comp., 26 (2005), pp Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 50 / 64

51 MGOPT Smoothing In the context of MGOPT, the smoother is any standard one-grid optimization scheme. MGOPT Steepest descent method Nonlinear conjugate gradient method Newton-type schemes u k+1 = u k + α k p k where p k = B 1 k Ĵ k can be applied to any optimization problem given the reduced gradient independent of discretization independent of structure of the problem Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 51 / 64

52 MGOPT scheme Initialize u 0 k. If k = 1, solve min Ĵ k (u k ) (f k, u k ) k and return. Else, u k 1 Pre-optimization: u l k = O k(u l k, f k), l = 1, 2,..., γ 1 2 Coarse grid problem Restrict the solution of (1): u γ1 k 1 = Ik 1 k u γ1 k Fine-to-coarse correction: τ k 1 = Ĵ k 1 (u γ1 k 1 ) Ik 1 k Ĵ k (u γ1 f k 1 = I k 1 k f k + τ k 1 Apply MGOPT to the Coarse grid problem: min u k 1 Ĵ k 1 (u k 1 ) (f k 1, u k 1 ) k 1 3 Coarse grid correction Prolongate the error: d = I k k 1 (u k 1 u γ1 k 1 ) Perform a line search in the direction d to obtain a step length α k. Coarse grid correction: u γ1+1 k = u γ1 k + α kd 4 Post-optimization: u l k = O k(u l k, f k), l = γ 1 + 2,..., γ 1 + γ k ) Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 52 / 64

53 Control of Bose Einstein condensates in magnetic microtraps We consider transport of Bose-Einstein condensates in magnetic microtraps, controllable by external parameters such as wire currents or radio-frequency fields. The mean-field dynamics of the condensate is described by the Gross-Pitaevskii equation (GPE) i ψ(x, t) = ( 12 ) 2 + V (x, u(t)) + g ψ(x, t) 2 ψ(x, t) V (x, u(t)) is a three-dimensional potential produced by a magnetic microtrap. u(t) is a control parameter that describes the variation of the confining potential. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 53 / 64

54 Computational performance of CNCG and MGOPT Transport of Bose Einstein condensates in magnetic microtraps CNCG MGOPT ( 1 g 2 1 ψd, ψ(t ) 2 ) ( CPU ψd, ψ(t ) 2 ) CPU Table: Computational performance of the CNCG and MGOPT schemes for different values of g; T = 7.5, γ = 10 4, mesh Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 54 / 64

55 Multigrid convergence theory Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 55 / 64

56 CSMG convergence theory step-by-step: Scalar problem 1. Multigrid convergence theory for scalar elliptic equation y = f in Ω, and y = 0 on Ω. discrete matrix form A k y k = f k Theorem 1: Let M k := I k B k A k. Then there exists a positive constant δ < 1 such that (A k M k y, y) k δ (A k y, y) k, for all y V k. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 56 / 64

57 CSMG convergence theory step-by-step: Vector problem 2. Multigrid convergence theory for decoupled symmetric system ν y = νf in Ω, and y = 0 on Ω, p = z in Ω, and p = 0 on Ω. ( ) νak 0 discrete matrix form Ā k w k = g k, Ā k = 0 A k This system is exactly two copies of Poisson problem. Hence the multigrid convergence theory for this system inherits the properties of the scalar case. Theorem 2: Let M k := Īk B k Ā k. Then there exists a positive constant δ < 1 such that where δ = δ. (Ā k M k w, w) k δ(ā k w, w) k, for all w = (y, p) V k V k, Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 57 / 64

58 CSMG convergence theory step-by-step: Coupling 3. Multigrid convergence theory for optimality systems ν y p = νf in Ω, and y = 0 on Ω, p + y = z in Ω, and p = 0 on Ω. Consider A k = Ā k + D k, D k = ( 0 Ik I k 0 ). Theorem 3: Let M k := I k B k A k. Then there exist positive constants h 0 and ˆδ < 1 such that h 1 < h 0 (Ā k M k w, w) k ˆδ(Ā k w, w) k, for all w = (y, p) V k V k, where ˆδ = δ + Ch 1. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 58 / 64

59 Theoretical analysis of MGOPT We assume that for each k, Ĵ k is twice Fréchet differentiable and 2 Ĵ k is (locally) positive definite and satisfies the conditions ( 2 Ĵ k (u)v, v) k β v 2 k and 2 Ĵ k (u) 2 Ĵ k (v) λ u v k uniformly for some positive constants β and λ. We remark that ) (Ĵk 1 (u k 1 ) (f k 1, u k 1 ) k 1 = I uk 1=I k 1 k 1 k k u k We use the expansion ( Ĵ k (u k ) f k ), Ĵ k (u + z) = Ĵk(u) + ( Ĵk(u), z) k ( 2 Ĵ k (u + tz)z, z) k dt. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 59 / 64

60 Lemma 1: For u, v V k assume ( Ĵk(u) f k, v) k 0 and let γ be such that [ 1 0 γ 2δ( Ĵk(u) f k, v) k ( 2 Ĵ k (u + tγv)v, v) k dt 0 ] 1 δ [0, 1]. Then (1 δ)γ( Ĵk(u) f k, v) k Ĵk(u) Ĵk(u + γv) + γ(f k, v) k γ( Ĵk(u) f k, v) k. We can find an explicit estimate for α such that a sufficient decrease is satisfied. Lemma 2: For u, v V k assume ( Ĵk(u) f k, v) k 0 and let { } ( Ĵ k (u) f k, v) k α(u, v) = min 2, λ v 3 k + ( 2Ĵ, λ > 0.Then k(u)v, v) k Ĵ k (u + α(u, v)v) Ĵk(u) + α(u, v)(f k, v) α(u, v)( Ĵk(u) f k, v) k. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 60 / 64

61 The coarse-grid correction provides a descending direction The following lemma states that the coarse-to-fine minimization step with step-length α is a minimizing step. Lemma 3: Let u V k and define ũ = I k 1 k u. Let ṽ V k 1 and define v = I k k 1 (ṽ ũ). Assume Ĵ k 1 (ṽ) (f k 1, ṽ) k 1 Ĵk 1(ũ) (f k 1, ũ) k 1, where Then ( (f k 1, ṽ) k 1 = I k 1 k ( (f k 1, ũ) k 1 = ) f k + Ĵk 1(ṽ) I k 1 k Ĵk(v), ṽ, and k 1 ) I k 1 k f k + Ĵk 1(ũ) I k 1 Ĵk(u), ũ k k 1. Ĵ k (u + α(u, v)v) Ĵk(u) α(u, v)(f k, v) k 1 2 α(u, v)( Ĵk(u) f k, v) k. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 61 / 64

62 Convergence of MGOPT Theorem 4: Let Ĵ k (u k ) (f k, u k ) k be twice Fréchet differentiable and let 2 Ĵ k be locally Lipschitz continuous and satisfies the Hessian conditions in a neighborhood Vk ɛ of u k = argmin uk (Ĵ k (u k ) (f k, u k ) k ). Then the MGOPT method provides a minimizing iteration. Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 62 / 64

63 References on CSMG and MGOPT convergence theory A.B., K. Kunisch, and Do Y. Kwak, Accuracy and Convergence Properties of the Finite Difference Multigrid Solution of an Optimal Control Optimality System, SIAM J. Control Opt., 41(5) (2003), A.B. and M. Vallejos, Multigrid optimization methods for linear and bilinear elliptic optimal control problems, Computing, 82 (2008), pp W. Hackbusch and A. Reusken, Analysis of a damped nonlinear multilevel method, Numer. Math., 55 (1989), pp S.G. Nash, A multigrid approach to discretized optimization problems, Optimization Methods and Software, 14 (2000), pp J. Schoeberl and W. Zulehner, Symmetric Indefinite Preconditioners for Saddle Point Problems with Applications to PDE-Constrained Optimization Problems, SIAM J. Matrix Analysis and Applications, 29(2007), pp Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 63 / 64

64 Best regards and many thanks for your interest!! See you in Benevento Alfio Borzì (Università degli Studi del Sannio, Multigrid Italy) methods for pde optimization 64 / 64