Model-based Control and Optimization with Imperfect Models Weihua Gao, Simon Wenzel, Sergio Lucia, Sankaranarayanan Subramanian, Sebastian Engell

Transcription

1 Model-based Control and Optimization with Imperfect Models Weihua Gao, Simon Wenzel, Sergio Lucia, Sankaranarayanan Subramanian, Sebastian Engell Process ynamics Group ept. of Biochemical and Chemical Engineering, TU ortmund, Germany TU Lyngby,.5.06

2 about 5 Ph candidates from many countries, having degrees in (Bio-)Chemical Engineering, Computer Science, Electrical Engineering, Automation and Robotics 5 Postocs: Weihua Gao, Radoslav Paulen, Maren Urselmann, Jian Cui, Elrashid Idris part-time secretaries, technician More than 65 finished Ph theses since 990

3 Outline Motivation Iterative optimization using modifier adaptation Multi-stage optimizing control Idea and problem formulation Results for a case study Output feedback multi-stage optimizing control Summary Lyngby, May, 06 Process ynamics

4 Motivation for optimizing control Operational excellence Optimal utilization of equipment and resources Minimization of unplanned shut-downs Meeting of quality standards without re-work Energy efficiency Resource efficiency Operations in the process industries are subject to significant uncertainties Changing process behaviours (e.g. catalyst efficiency) Changing equipment Changing feeds External influences as e.g. outside temperature Efficient, safe and reliable operation requires reactive measurement-based control and optimization Lyngby, May, 06 4 Process ynamics

5 Available technologies Feedback control to track given set-points and constraints Requires a margin around the constraints Set-points must be chosen well Optimization of the set-points and feedback control (Real-time Optimization (RTO) plus classical control or MPC) Model-based (directly) optimizing control: The target of the controller is an economic optimization under constraints: Safety limits Product quality constraints Equipment limitations S. Engell: Feedback Control for Optimal Process Operation. J. Process Control 7, 007, 0-9 RTO and optimizing control depend critically on the accuracy of the model that is employed. Lyngby, May, 06 5 Process ynamics

6 Focus of this talk In this talk the focus is on optimization and control using inaccurate or simplified models. ew strategies for robust control and optimization will be presented: MAwQA Modifier Adaptation with Quadratic Approximation Robust iterative data and model based optimization MSMPC Multi-stage onlinear Model Predictive Control A new efficient robust MPC strategy Lyngby, May, 06 6 Process ynamics

7 epartment of Biochemical and Chemical Engineering Process ynamics Group () Iterative Optimizing Control by Modifier Adaptation with Quadratic Approximation Weihua Gao, Simon Wenzel, Sebastian Engell The research leading to these results was funded by the ERC Advanced Investigator Grant MOBOCO under the grant agreement o Process ynamics

8 Real-time optimization Planning and Scheduling C Steady-state optimization Validation RTO Control Layer Model update Reconciliation Cn Model-based upper-level optimization system Quasi-stationary optimization of the set-points of the plant Targeting economic optimality Plant Lyngby, May, 06 8 Process ynamics

9 Batch chromatography Goal: Lyngby, May, 06 9 Process ynamics

10 Lyngby, May, 06 Process ynamics Model of batch chromatography 0 General Rate Model Mobile phase )) ( ( ) (,, p i p i p i l i i ax i r c c r k x c u x c t c Stationary phase r c r r r t c t q i p i p p i p p i p,,, ) ( Adsorption Isotherm ),, (,,, n p p p i c c c f q Liquid Solid particle i p c, q i c i q i i p c,

11 Effect of uncertainty in the adsorption isotherm Elution profiles: real plant Challenge: Optimization in the presence of model uncertainty Lyngby, May, 06 Process ynamics

12 Measurement-based RTO strategies Model-based Measurements Two-step approach: parameter estimation and subsequent optimization ecessary conditions of optimality tracking Model-free Structural mismatch leads to wrong results Sufficient excitation required Limited number of inputs Sequence of arcs must be fixed Lyngby, May, 06 Process ynamics

13 The principle of Modifier Adaptation Using the collected data, the bias (offset) between the plant and the model and the empirical gradients are estimated and used to modify the optimization problem: Instead of the optimizer solves bias modifiers gradient modifiers If the bias and gradients are correct, this converges to the true optimum! Lyngby, May, 06 Process ynamics

14 Gradient estimation Finite difference approximation from the measurements at the latest n u + setpoints: S can become ill-conditioned gradient becomes unreliable Monitoring of the condition number Additional moves to improve the condition number W. Gao and S. Engell: Iterative Set-Point Optimization of Batch Chromatography, Computers and Chemical Engineering 9, 005, Setpoints close to each other: Approximation good but sensitive to noise Setpoints far apart: Error in the gradient but robust Lyngby, May, 06 4 Process ynamics

15 Influence of noise on the gradient error (-,-) (-,-.8). (-.8,-) (-,-). (-,-) (-,0). (0,-) u u u Lyngby, May, 06 5 Process ynamics

16 Quadratic approximation in erivative-free Optimization Optimization based upon probing of the target function Repeatedly constructing local quadratic functions Mathematically well founded Idea: Estimate the gradients via quadratic approximation Lyngby, May, 06 6 (Audet, 008) Process ynamics

17 MAWQA-Algorithm Modifier Adaptation with Quadratic Approximation Iterative optimization combined with estimation of the gradients by quadratic approximation Theory available how to choose the points which are interpolated to get a good approximation of the curvature earby points for accuracy istant points for robustness Trust region estimation prevents too large steps Monitoring of model quality and switching between MA and pure FO Weihua Gao, Simon Wenzel and Sebastian Engell IFAC ACHEM 05 European Control Conference 05 Computers and Chemical Engineering, online 06 Lyngby, May, 06 7 Process ynamics

18 Gradient computed by quadratic approximation (-,-) (-,-.8) (-.8,-) (-,-). (-,-) (-,0) (0,-) u u Lyngby, May, 06 8 Process ynamics

19 Gradient via quadratic approximation Capture the curvature information from more distant points to decrease the approximation error Explore the inherent smoothness of the mapping to decrease the influence of the noise Screen points for a well-distributed regression set Lyngby, May, 06 9 Process ynamics

20 Illustration of regression set screening Sufficiently distant and well-distributed points are indispensable for capturing the curvature reliably from noisy data The use of many points in a neighborhood can improve the accuracy of the gradient estimation Lyngby, May, 06 0 Process ynamics

21 Trust region Allow large moves along a direction in which more data has been collected Bound aggressive moves along a direction in which the plant still needs to be probed Lyngby, May, 06 Process ynamics

22 Application to the batch chromatography example Significantly improved robustness to noise and speed of convergence compared to our previous work (and to that of others). Lyngby, May, 06 Process ynamics

23 Ten-variable synthetic example The objective and one constraint are given by noisy implicit blackbox functions Caballero and Grossmann (008): Algorithm based on fitting response surface takes 800 sampled points to reach the optimum. Lyngby, May, 06 Process ynamics

24 Optimization results for the synthetic example Evolution of the objective Evolution of the constraint Lyngby, May, 06 6 sampled points used 4 Process ynamics

25 Generation of data-collecting moves The regression set is not well-poised In the probed operating range, the function is not quadratic ual-control mechanism with set-point moves Lyngby, May, 06 5 Process ynamics

26 epartment of Biochemical and Chemical Engineering Process ynamics Group () Multi-stage onlinear Model-predictive Control Sergio Lucia, Sankaranarayanan Subramanian, Sebastian Engell The research leading to these results was supported by the German Research Council FG The research leading to these results was supported by the ERC Advanced Investigator Grant MOBOCO under the grant agreement o Process ynamics

27 Model predictive control State Constraint x(t) Control u(t) Time MPC Solve optimization problem Mathematical Model Cost function Constraints Apply first control input Take new measurements Optimize again Apply first control input Take new measurements and optimize again past future Time Economic cost function can be used in the optimization optimizing control Lyngby, May, 06 7 Process ynamics

28 Model predictive control: Wrong model State Constraint x(t) Control u(t) past future Time Time MPC Solve optimization problem Mathematical Model Cost function Constraints Apply first control input Take new measurements What happens if the prediction is not exact? Violation of constraints ecreased performance Instability Lyngby, May, 06 8 Process ynamics

29 Multi-stage MPC: Formulation x 0 d 0 u 0 u 0 d 0 x x u d u d u d 4 u d 5 u d 6 u d State Control Time u 0 d 0 x 7 u d 8 u d u d Prediction horizon Uncertainty is modelled by a scenario tree 9 Constraints must be met for all values of the uncertainty Controller can react to the information gained at the next stage This is taken into account in the optimization of the next decisions Lyngby, May, Time Process ynamics

30 Robust Multi-Stage MPC Closed-loop formulation by means of an open loop optimization problem Applied to scheduling problems [Sand and Engell, Comp. Chem. Engg. 005] Early work on linear MPC [de la Peña et al., 005], [Bernardini et al., 009] Proposed for onlinear MPC in [adhe & Engell, 008] Many publications by [Lucia et al.] since 0 with promising results, [Lucia, Finkler, Engell, ACHEM 0, J. Proc. Control 0, ] Lyngby, May, 06 0 Process ynamics

31 Multi-stage MPC: Robust horizon Avoid the exponential growth by branching the tree only up to the robust horizon x 0 d 0 u 0 u 0 d 0 x x u d u d 4 u u d d 5 u d 6 u d u d u d u d d 4 u 5 ud 6 ud x x x 4 x 5 x 6 x d u u d u d d 4 u 5 ud 6 ud x 4 x 4 x 4 4 x 4 5 x 4 6 x 4 u 0 d 0 x 7 u d 8 u d 9 u d d 7 u 8 ud 9 ud 7 x 8 x 9 x d 7 u 8 ud 9 ud 7 x 4 8 x 4 9 x 4 Prediction Horizon = 4 Robust Horizon = Lyngby, May, 06 Process ynamics

32 An industrial batch polymerization reactor control problem 8 differential states control inputs uncertain parameters Model provided by BASF SE B S. Lucia, J. Andersson, H. Brandt, M. iehl, S. Engell: Handling Uncertainty in Economic onlinear Model Predictive Control, J. Process Control 4 (04), Lyngby, May, 06 Process ynamics

33 m W = m W,F Industrial batch polymerization reactor model m A = m A,F k R m A,R p k R m AWT m A m ges m P = k R m A,R + p k R m AWT m A T R = m ges 8 differential states control inputs uncertain parameters c p,r m ges m F c p,f T F T R + ΔH R k R m A,R k K A T R T S m AWT c p,r T R T EK TS = /(c p,s m S ) k K A T R T S k K A T S T M I T M = m c p,w m M,KW c p,w T M T M + k K A T S T M M,KW T EK = m c p,r m AWT c p,w T R T EK α T EK T AWT + p k R m A m AWT ΔH R AWT m ges TAWT = m c p,w m AWT,KW c p,w T I AWT T AWT α T AWT T EK AWT,KW k R = k 0 e E a RT R k U U + k U U E a k R = k 0 e RT EK (k U U + k U U) Lyngby, May, 06 Process ynamics

34 Formulation of the MPC problem Control task: Minimize the batch time while satisfying temperature constraints for all values of two uncertain (±0 %) parameters Standard MPC with tracking cost: J track = p k=0 j m P,k j + q T R,k T set + r Δuk j Multi-stage MPC with economic cost function: p J eco = ω i m P,k j + r Δu k j i= k=0 Lyngby, May, 06 4 Process ynamics

35 Standard MPC: o uncertainties Comparison of tracking and economic MPC Reactor temperature Adiabatic temperature Monomer feed rate Jacket inlet temperature Lyngby, May, 06 5 Process ynamics

36 Simulation results for different scenarios Standard MPC Standard MPC with conservative choice of parameters Simulations for different values of k and ΔH (±0%) Lyngby, May, 06 6 Process ynamics

37 Multi-stage MPC Simple scenario tree Simulation results for different scenarios extreme values of the uncertainties Tree branches only at the fist stage Simulations for different values of k and ΔH (±0%) x 0 d 0 u 0 d 0 u 0 d u u d u 0 d u 0 d 0 7 u 7 0d 0 8 u 0 8 d 0 9 u 0 9 d 0 x x x 4 x 5 x 6 x 7 x 8 x 9 x u d u d u d 4 4 u d 5 5 u d 6 6 u d 7 7 u d 8 8 u d 9 9 u d u d u d u d 4 4 u d 5 5 u d 6 6 u d 7 7 u d 8 8 u d 9 9 u d x x x 4 x 5 x 6 x 7 x 8 x 9 x u d u d u d 4 4 u d 5 5 u d 6 6 u d 7 7 u d 8 8 u d 9 9 u d x 4 x 4 x 4 4 x 4 5 x 4 6 x 4 7 x 4 8 x 4 9 x u 9 d 9 u 9 d 9 u 9 d u 9 d u 9 d u 9 d u 9 d u 9 d u 9 d 9 x 0 x 0 x 0 4 x 0 5 x 0 6 x 0 7 x 0 8 x 0 9 x 0 Lyngby, May, 06 7 Process ynamics

38 Comparison with standard MPC Scenario Batch time in hours ΔH R k 0 Standard MPC Standard (cons.) Multi-stage +0% +0% infeasible % 0% infeasible % -0% infeasible % +0% % 0% % -0% % +0% % 0% % -0% Av. batch time [h] infeasible.5.0 Batch time reduction of 60% w.r.t. standard (cons.) MPC Comp time [s] Standard MPC Standard (cons.) Multi-stage Average Maximum Lyngby, May, 06 8 Process ynamics

39 Comparison with open-loop robust MPC Multi-stage MPC Open-loop robust MPC ~ 5% longer batch! Lyngby, May, 06 9 Process ynamics

40 Output feedback MPC - Motivation State Constraint x(t) x(t) Control past future u(t) Time Time The values of the states are not known generally. The states need to be estimated based on measurements. Lyngby, May, 06 Process ynamics

41 Multi-stage output feedback MPC Instead of predicting the future state, predict the estimates x k+ = f est (x k, u k, d k ) If the estimates can be predicted, the states can be assumed to be bounded by x k x k Σ k Since we know the estimates at every time, the feedback policy can be obtained based on the predicted estimates EKF or UKF equations can be used to get x k+ = f x k, u k, d k + K k ν k Lyngby, May, 06 Process ynamics

42 Multi-stage output feedback MPC using the EKF Start with current estimate x 0 and covariance information P 0 from the estimator Propagate the state estimate and the covariance information Use the EKF equations to estimate future states for different values of the innovations x k+ = f x k, u k, p k + K k ν k The covariance of the innovations C k P k C k T + R k is used to get the samples of the innovations ˆx 0 P 0 v 0 u 0 u 0 v 0 u 0 v 0 ˆx P ˆx P ˆx P u 4 u v 5 u v 6 u v 7 u v u v u v v 8 u v 9 u v ˆ P ˆ P ˆ P 4 ˆ 4 P 5 ˆ 5 P 6 ˆ 6 P 7 ˆ 7 P8 ˆ 8 P 9 ˆ 9 P Lyngby, May, 06 Process ynamics

43 Output feedback MPC Problem formulation Mathematical formulation: subject to: j x k+ K k j = Φ j P k+ j x k J i = p k=0 = f x k p j, u k j, d k r j x k p j, u k j, d k r j, P k p j = Ψ x k p j, u k j, d k r j, P k p j + K k j ν k r(j) j, k j, k {σ k j } X, u k j U j, k I u k j = u k l if x k p(j) = xk p(l) min u k j i= ω i J i j L(x k+, u j k ) j, x k+, u j k S Kalman update i with the sampled innovations j, k + I The EKF/ UKF equations j, k, l, k I I I Lyngby, May, 06 Process ynamics

44 Industrial batch polymerization reactor m W = m W,F m A = m A,F k R m A,R p k R m AWT m A m ges m P = k R m A,R + p k R m AWT m A T R = m ges 8 differential states control inputs uncertain parameter c p,r m ges m F c p,f T F T R + ΔH R k R m A,R k K A T R T S m AWT c p,r T R T EK TS = /(c p,s m S ) k K A T R T S k K A T S T M I T M = m c p,w m M,KW c p,w T M T M + k K A T S T M M,KW T EK = m c p,r m AWT c p,w T R T EK α T EK T AWT + p k R m A m AWT ΔH R AWT m ges TAWT = m c p,w m AWT,KW c p,w T I AWT T AWT α T AWT T EK AWT,KW k R = k 0 e E a RT R k U U + k U U E a k R = k 0 e RT EK (k U U + k U U) Lyngby, May, 06 Process ynamics

45 Control task: MPC problem formulation Minimize the batch time while satisfying temperature constraints for all the values of the uncertainty (±0 %) Constraint on the temperature of the reactor (90±) C Economic cost function J eco = ω i m P,k i= K k=0 j j + r Δ m F,k + r ΔT I,j I,j M,k + r ΔT AWT,k Only the measurements of states T R and T M are available with measurement noise, standard deviation σ = 0. K EKF with parameter estimation is used Prediction horizon = 0 samples Robust horizon = sample Sampling time = 90 s Lyngby, May, 06 Process ynamics

46 Standard vs output feedback scheme Standard Multi-stage MPC Robust Output Feedback MPC k 0 true =. k 0 nom Lyngby, May, 06 Process ynamics

47 Multi-stage optimizing control - Conclusions The improvement of the robustness is very convincing. irect solution without a need for specific engineering Improvement over nominal MPC even when parameter updates are available online S. Lucia, T. Finkler, S. Engell: Multi-Stage nonlinear model predictive control applied to a semi-batch polymerization reactor under uncertainty, J. Process Control, 0, 06-9 umerically tractable O MPC based upon CasAi There is more: Multiple model state estimation to update the probabilities in the scenario tree ual control improving the model accuracy to improve the control performance Guaranteed constraint satisfaction by reachability analsis also for values that are not in the scenario tree Lyngby, May, Process ynamics

48 Summary Advanced control offers a significant potential for improved operations! One can control well with inaccurate models! Two solutions presented: Iterative model and data based optimization Multi-stage robust model-based optimization Modeling effort is the bottleneck in the solution of practical problems. Robust optimization and control reduce the modeling effort! ifficult problem: Satisfaction of complex product property constraints Soft sensors and additional online measurements - not necessarily selective, e.g. ultrasound, conductivity, ph, turbidity Online measurements, e.g. IR or Raman spectroscopy Open-source software for efficient implementation of MPC and MS-MPC: O-MPC Lyngby, May, Process ynamics

49 epartment of Biochemical and Chemical Engineering Process ynamics Group () Thank you very much for your attention! Sponsored by: The research leading to these results was supported by the German Research Council FG The research leading to these results was supported by the ERC Advanced Investigator Grant MOBOCO under the grant agreement o Process ynamics