An optimal control problem for a parabolic PDE with control constraints

An optimal control problem for a parabolic PDE with control constraints PhD Summer School on Reduced Basis Methods, Ulm Martin Gubisch University of Konstanz October 7 Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 / 9

The optimal control problem Optimization problem formulation The formal Lagrange principle Existence and uniqueness Numerical solution algorithms Method of steepest descent Primal-dual active set strategy Numerical experiments Optimality System Proper Orthogonal Decomposition The Reduced Order Model The POD optimality system Lagrange functional & necessary first-order conditions Operator splitting ansatz The next steps 4 References Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 / 9

The optimal control problem Optimization problem formulation The general optimal control problem We consider the following convex optimization problem in Hilbert space: min (y,u) Y U J(y, u) subject to E(y, u) =, y Y ad, u U ad Y state space, U control space (both Hilbert spaces) J convex cost functional E state equation (parabolic pde) Y ad Y and U ad U both nonempty, closed and convex, U ad bounded. Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 / 9

The optimal control problem Optimization problem formulation The cost functional J(y, u) := y y Q H Q + y(t) y Ω H Ω + u u Θ H Θ Ω R d open and bounded domain, d {,, } Θ = (, T) time interval, T > Q = Θ Ω time-space cylinder H Q := L (Q), H Ω := L (Ω), H Θ := L (Θ) m, HQ = σ Q, L (Q),, HΩ = σ Ω, L (Ω),,, HΘ = κ i, L (Θ) σ Q L (Q), σ Ω L (Ω), κ,..., κ m R nonnegative weights y Q L (Q), y Ω L (Ω), u Θ L (Θ) m (given) desired states and controls Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 4 / 9

The state equation The optimal control problem Optimization problem formulation d m dt y(t, x) σ y(t, x) = f (t, x) + u i (t)χ i (x) i= f.a.a. (t, x) Q y(t, x) = f.a.a. (t, x) Θ Ω y(, x) = y (x) f.a.a. x Ω f L (Q), y L (Ω), σ >, u i L (Θ) (V, H, V ) is the Gelfand triple H (Ω) L (Ω) H (Ω) y Y := {ϕ L (Θ, V) ϕ t L (Θ, V )} C (Θ, L (Ω)) Y ad := {ϕ Y y() = y } is the space of admissible states χ,..., χ m L (Ω) denote the control-shape functions Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 5 / 9

The control constraints The optimal control problem Optimization problem formulation For the control functions u,..., u m we claim the box constraints u a,i (t) u i (t) u b,i (t) f.a.a. t Θ, i.e., the set of admissible control components is U ad := {u L (Θ) m u [u a, u b ] a.e. in Θ}. Notice that the set of admissible control-state pairs is nonempty, closed and convex. X ad = Y ad U ad Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 6 / 9

The Lagrange function The optimal control problem The formal Lagrange principle The corresponding Lagrange functional L : Y U L (Θ, V) R, (y, u, p) L (y, u, p) is defined as L (y, u, p) = J(y, u) + E(y, u)p = Q + σ Q y y Q d(x, t) + m κ i u i u Θ,i dt i= Θ ( + t σ ) Ω y, p V,V Q σ Ω y(t) y Ω dx ( f + m u i χ i )p d(x, t) i= Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 7 / 9

The optimal control problem The formal Lagrange principle Necessary first-order optimality conditions The variational inequalities y L (y, u, p )(y y ) for all y Y ad, L (y, u, p )(u i u i ) u i for all u i U ad. hold in an optimal point (y, u, p ) X ad L (Q). Additionally, the side-condition holds in (y, u, p ): p L (y, u, p ) = E(y, u ) =. Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 8 / 9

The adjoint equation The optimal control problem The formal Lagrange principle The first variational inequality is equivalent to y L (y, u, p )ỹ = for all ỹ Y := {ϕ Y ϕ() = }. By calculus of variation, we derive that p itself is a solution to a parabolic PDE: d dt p(t, x) σ p(t, x) = σ Q(t, x)(y (t, x) y Q (t, x)) a.e. in Q p(t, x) = a.e. in Θ Ω p(t, x) = σ Ω (x)(y (T, x) y Ω (x)) a.e. in Ω Since the solution operator to this backwards heat equation is the adjoint solution operator to the state equation, the Lagrange multiplier p is called the adjoint state. Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 9 / 9

The optimal control problem The projection condition The formal Lagrange principle The second variational inequality is equivalent to m m χ i p ũ i d(x, t) + κ i (u i u Θ,i )ũ i dt i= Q i= Θ ũi=u i u i. Hence, the components of u fulfill the projection conditions ( ) u i (t) = P [ua,i(t),u b,i(t)] χ i (x)p (t, x) dx + u Θ,i (t). κ i Ω Since p = S y and y = Su hold, u can be determined as the single fixpoint of ( ) u i P [ua,i,u b,i] χ i (x)(s Su)(x) dx + u Θ,i κ i Ω Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 / 9

The optimal control problem Existence and uniqueness Existence and uniqueness Let S : u y(u) the solution operator for the state equation, i.e. E(y, u) = y = Su. The optimization problem then can be replaced by the reduced problem min u U ad F (u) := Su y Q H Q + (Su)(T) y Ω H Ω + u u Θ H Θ Since S is affine linear and continuous and U ad is nonempty, closed and convex, this quadratic optimization problem has a solution. Under the assumption that the regularization parameters κ,..., κ m are strictly positive, this solution is unique. Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 / 9

Numerical solution algorithms Projected gradient method (PGM) Method of steepest descent ALGORITHM: Let u n U ad any non-optimal admissible control. Task: Find a direction d n L (Θ) m, d n L (Θ) m =, and a stepsize s n > such that u n+ := P [ua,u b](u n + s n d n ) u n. In this context, means that (u n ) n N converges at least linearly towards u. We choose d n as the negative gradient d n := F (u n ) (Taylor & Cauchy-Schwarz). An efficient stepsize is s n (N ) := N where N is the minimal N N such that the ARMIJO condition holds: F (P [ua,u b](u n +s(n)d n )) F (u n ) s(n) P [u a,u b](u n +s(n)d n ) u n L (Θ) m. Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 / 9

Numerical solution algorithms Primal-dual active set strategy Primal-dual active set strategy The projection condition ( ) u i = P [ua,i,u b,i] χ i (x)p (x) dx + u Θ,i κ i can be written as Ω ( ) u i = ( χ Ai χ Bi ) χ i (x)p (x) dx + u Θ,i + χ Ai u a,i + χ Bi u b,i κ i Ω where χ M denotes the indicator function and the active sets A i, B i are { A i := t Θ κ i Ω { B i := t Θ κ i Ω χ M (x) := { x M x / M } χ i (x)p (t, x) dx + u Θ,i (t) < u a,i (t) ; } χ i (x)p (t, x) dx + u Θ,i (t) > u b,i (t). Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 / 9

Numerical solution algorithms Primal-dual active set strategy Primal-dual active set strategy The optimality system then is given as the semilinear coupled parabolic system ( d σ ) y m χ dt iu i = f i= ( d σ ) p +σ dt Qy = σ Qy Q ( ) ( χ Ai χ Bi ) κ i χ ip dx +u i = ( χ Ai χ Bi )u Θ,i Ω +χ Ai u a,i + χ Bi u b,i ALGORITHM: Let (p n, y n, u n ) given. { } A n+ i := t Θ κ i χ i (x)p n (t, x) dx + u Θ,i (t) < u a,i (t) ; { Ω } B n+ i := t Θ κ i χ i (x)p n (t, x) dx + u Θ,i (t) > u b,i (t) ; Ω if A n+ i = A n i and B n+ i = B n i finish else (p n+, y n+, u n+ ) := the solution of the pde system n := n + Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 4 / 9

Numerical solution algorithms Numerical experiments Shape functions, initial control and control constraints.9.9.9.8.8.8.7.7.7 shape function χ (x).6.5.4 shape function χ (x).6.5.4 shape function χ (x).6.5.4...........4.6.8..4.6.8 space variable x..4.6.8..4.6.8 space variable x..4.6.8..4.6.8 space variable x.5 lower bound u a initial control u.5 upper bound u b.5 lower bound u a initial control u.5 upper bound u b.5 lower bound u a initial control u.5 upper bound u b u(t).5 u(t).5 u(t).5.5.5.5.5.5.5.5 t.5.5.5.5 t.5.5.5.5 t Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 5 / 9

Numerical solution algorithms Numerical experiments Optimal control components with and without restrictions.5 lower bound u a optimal control u *.5 upper bound u b.5 lower bound u a optimal control u *.5 upper bound u b.5 lower bound u a optimal control u *.5 upper bound u b.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5 lower bound u a.5 unrestr. control optimal control u * lower bound u a.5 unrestr. control optimal control u * lower bound u a unrestr. control optimal control u *.5 upper bound u b.5 upper bound u b.5 upper bound u b.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5.5 Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 6 / 9

Numerical solution algorithms Numerical experiments Optimal and desired control and state.5 control function u = Σ u i (t) χ i (x).5.5.5 state function y(t,x).5 decreasing of objective function 4 6 8.5.5 space x.5 time t space x.5 time t 4 5 5 5 5 iteration steps reference control u Θ (t,x).8.6.4. reference state y Q (t,x).5.5.5 decreasing of gradient 4 6 8.5.5 space x.5 time t space x.5 time t 4 5 5 5 5 iteration steps Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 7 / 9

Optimality System Proper Orthogonal Decomposition The reduced order model The Reduced Order Model In the following, we consider the non-discretized, unconstraint optimization problem min J(y, u) := σ Q y,u Q y y Q d(x, t) + m κ i u i (t) dt i= Θ subject to { ẏ(t) + Ay(t) = f (t) + Bu(t) y() = y (P) with A = σ and (Bu)(t) := m χ i (x)u i (t). i= A POD basis ψ = (ψ,..., ψ l ) for y is given by the eigenvectors belonging to the l largest eigenvalues of YY : H H, (YY )ψ = y(t), ψ H y(t) dt. Θ Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 8 / 9

Optimality System Proper Orthogonal Decomposition The reduced order model The Reduced Order Model The reduced model reads as min J(ψ)(y, u) = σ Q y(t), M(ψ)y(t) y(t), y y,u Q (t) + y Q (t) dt + Θ m κ i u i (t) dt i= Θ subject to { M(ψ)ẏ(t) + A(ψ)y(t) = f(ψ)(t) + B(ψ)u(t) M(ψ)y() = y (ψ) where (P l ) M(ψ) = ( ψ i, ψ j H ) R l l, A(ψ) = ( ψ i, Aψ j H ) R l l, f(ψ) = ( f, ψ i H ) L (Θ, R l ), B(ψ) = ( χ i, ψ j H ) R l m, y Q (ψ) = ( y Ω, ψ i H ) L (Θ, R l l ), y (ψ) = ( y, ψ i H ) R l. Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 9 / 9

Optimality System Proper Orthogonal Decomposition The Reduced Order Model Implementation for (P l ) ALGORITHM: Let u n an admissible control. Solve the full PDE E(y, u n ) = to get the corresponding state y n. Compute a POD basis (ψ l n) for y n. Solve (P l ) with a globalized SQP method (Sequential Quadratic Programming, a Newton method) to get a better control u n+. 4 n := n + Since the expensive optimization is done for the small system, this approach needs less effort than solving (P). Disadvantage: If the dynamics of y n differ much from those of the optimal state y, the POD basis (ψ l n) is not very efficient. This weakness can be eliminated by choosing (ψl n ) in a way such that the goal of minimizing J is respected. Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 / 9

Optimality System Proper Orthogonal Decomposition The POD optimality system The POD optimality system min J(y, u, ψ) := J(ψ)(y, u) y,u,ψ subject to E(ψ)(y, u) = M(ψ)ẏ(t) + A(ψ)y(t) f(ψ)(t) B(ψ)u(t) = M(ψ)y() = y (ψ) E(y, u) = ẏ(t) + Ay(t) f (t) Bu(t) = y() = y YY ψ i = λ i ψ i ψ i, ψ j H = δ ij (P l OS-POD ) To derive necessary first-order conditions, we use the Lagrange framework. Let z := (y, y, ψ, λ, u) Z := H (Θ, R l ) W(Θ) Ψ l R l L (Θ, R m ) ξ := (q, p, µ, η) Ξ := L (Θ, R l ) L (Θ, V) Ψ l R l. Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 / 9

Optimality System Proper Orthogonal Decomposition Lagrange functional & necessary first-order conditions Lagrange functional & necessary first-order conditions Define the Lagrange functional L : Z Ξ R as L (z, ξ) = J(y, u, ψ) + E(ψ)(y, u), q L (Θ,R m ) + E(y, u), p L (Θ,V ),L (Θ,V) l l + (YY λ i Id)ψ i, µ i Ψ l + ψ i Ψ, η i l R l. i= i= The necessary first-order conditions for an optimal point (z, ξ) are: L y (z, ξ) = which implies M(ψ) q(t) + A(ψ)q(t) = σ Q (E(ψ)y(t) y Q (ψ)(t)) + final condition; L y (z, ξ) = which implies ṗ(t) + Ap(t) = l y(t), µ i H ψ i + y(t), ψ i H µ i + final condition; i= Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 / 9

Optimality System Proper Orthogonal Decomposition Lagrange functional & necessary first-order conditions Lagrange functional & necessary first-order conditions L λ (z, ξ) = which implies the complementary slackness condition L ψ (z, ξ) = which implies = l (YY λ i Id), µ i H + i= ψ i, µ i H = ; l, ψ i H η i i= + σ Q y(t), M(ψ, )y(t) y Q (t, ) R m dt + Θ Θ q(t), (M(ψ, ) + M(, ψ))ẏ(t) + (A(ψ, ) + A(, ψ))y(t) B( )u(t) R m dt =: l (YY λ i Id), µ i H +, ψ i H η i + g i (y, ψ, u, q), H,H. i= Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 / 9

Optimality System Proper Orthogonal Decomposition Lagrange functional & necessary first-order conditions Lagrange functional & necessary first-order conditions Hereby, we set M(ψ, φ) := ( ψ i, φ j H ), A(ψ, φ) = ( ψ i, Aφ j V,V ) and B(φ) = ( χ i, φ i ) H. Together with the complementary slackness, this implies η i = g i(y, ψ, u, q), ψ H,H; (YY λ i Id)µ i = η i ψ i g i (y, ψ, u, q). Finally, L u (z, ξ) = which implies m κ i u i (t) = B t (ψ)q(t) + B p(t); i= in case of constraint optimization, here the fun begins. Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 4 / 9

Optimality system Optimality System Proper Orthogonal Decomposition Operator splitting ansatz We do not solve the single equations in the resulting optimality system M(ψ) q(t) + A(ψ)q(t) = σ Q (M(ψ)y(t) y Q (ψ)(t)) + final cond. ṗ(t) + Ap(t) = y(t), µ i H ψ i + y(t), ψ i H µ i + final cond. η i = (g i(y, ψ, u, q), q) H,H µ i = (YY λ i Id) (η i ψ i + g i (y, ψ, u, q)) κi u i (t) = B t (ψ)q(t) + B p(t), simultaniously, but use the operator splitting ansatz z = z (y, ψ, u) := (y l, u) and z = (y, ψ, λ). The optimization subproblem (P l ) within (P l OS-POD ) determines z = z (z ) for fixed z with SQP. The remaining minimization with respect to z is done with the Gradient Method; this requires the expensive solutions of the full system. Hereby, the Gradient Method requires one forward and one backward solve of E(y, u) = and the update of the POD basis requires one forward solve. In case of constraints, this step becomes very problematic as we have seen before. Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 5 / 9

Optimality System Proper Orthogonal Decomposition Splitting optimization algorithm Operator splitting ansatz M(ψ) q(t) + A(ψ)q(t) = σ Q (M(ψ)y(t) y Q (ψ)(t)) + final cond. ṗ(t) + Ap(t) = y(t), µ i H ψ i + y(t), ψ i H µ i + final cond. η i = (g i(y, ψ, u, q), q) H,H µ i = (YY λ i Id) (η i ψ i + g i (y, ψ, u, q)) κi u i (t) = B t (ψ)q(t) + B p(t), ALGORITHM: Let a POD basis ψ n given. Solve (P l ): min J(ψ n )(y n+, ũ n+ ) s.t. E(ψ)(y n+, ũ n+ ) =, y n+ () = y (ψ n ) by SQP. Hereby, q n+ is determined, too. Solve E(y n+, ũ n+ ) =, y() = y, and calculate ηi n+, µ n+ i to determine p n+. Choose direction F (u n ) = ( κ i B t (ψ n i )qn+ (t) + B p n+ (t)) in a gradient step to get a new control u n+. 4 Determine a new POD basis ψ n+ by solving E(y n+, u n+ ) =, y() = y and set n := n +. Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 6 / 9

Optimality System Proper Orthogonal Decomposition The next steps The next steps We postulate additional mixed control-state box constraints a l m y i (t)ψ i + ɛ u i (t)χ i b i= i= To-Do-List: Existence of optimal solutions to (P l OS-POD )? Existence of Lagrange multipliers? A-posteriori error estimates? Behavior for ɛ (pure state constraints)? Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 7 / 9

References I References Hintermüller, M.: Mesh-independence and fast local convergence of a primal-dual active-set method for mixed control-state constrained elliptic control problems. ANZIAM J. 49, No., pp. 8, 7. Hintermüller, M., Ito, K. & Kunisch, K.: The primal-dual active set strategy as a semismooth Newton method. SIAM J. Optim., No., pp. 865 888,. Hintermüller, M., Kopacka, I. & Volkwein, S.: Mesh-independence and preconditioning for solving parabolic control problems with mixed control-state constraints. ESAIM COCV 5, pp. 66 65, 9. Hinze, M. & Volkwein, S.: Error estimates for abstract linear-quadratic optimal control problems using proper orthogonal decomposition. Comput. Optim. Appl. 9, pp. 9 45, 8. Kunisch, K. & Volkwein, S.: Control of Burgers equation by a reduced order approach using proper orthogonal decomposition. J. Optim. Theor. Appl., pp. 45 7, 999. Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 8 / 9

References II References Kunisch, K. & Volkwein, S.: Galerkin proper orthogonal decomposition methods for parabolic problems. Numer. Math. 9, pp. 7 48,. Kunisch, K. & Volkwein, S.: Proper Orthogonal Decomposition for Optimality Systems. ESAIM: MAN, Vol. 4, No., pp., 8. Meyer, C., Prüfert, U. & Tröltzsch, F.: On two numerical methods for state-constrained elliptic control problems. Optim. Meth. Software, No. 6, pp. 87 899, 7. Tröltzsch, F.: Regular Lagrange multipliers for control problems with mixed pointwise control-state constraints. SIAM J. Optim. 5, No., pp. 66 64, 5. Tröltzsch, F.: Optimale Steuerung partialler Differentialgleichungen. Vieweg+Teubner, nd ed., 9. Tröltzsch, F. & Volkwein, S.: POD a-posteriori error estimates for linear-quadratic optimal control problems. Comput. Optim. Appl. 9, pp. 9 45, 8. Martin Gubisch (University of Konstanz) PhD Summer School RBM, Ulm October 7 9 / 9