Introduction to Optimization

Size: px

Start display at page:

Download "Introduction to Optimization"

Cuthbert Marsh
6 years ago
Views:

1 Introduction to Optimization The maximizing or minimizing of a given function subject to some type of constraints. Make your processes as effective as possible. In a typical chemical plant, there are many control variables for controlling the process, such as maintaining a temperature, level, or flow.

Part 1: Process Control and Control has to do with adjusting flow rates to maintain the controlled variables of the process at specified set-points.

2 Part 1: Process Control and Control has to do with adjusting flow rates to maintain the controlled variables of the process at specified set-points. Optimization chooses the values for key set-points such that the process operates at the best economic conditions. Optimization What is optimal operation temperature?

3 Economic Objective Function V B > V C, V A, or V AF V is the chemical values. At low T, little formation of B; At high T, too much of B reacts to form C; Therefore, the exits an optimum reactor temperature, T* QCAVA QCB VB QCC VC QCA0 V AF Graphical Solution of Optimum Reactor Temperature, T* k1(t) k2(t) A B C

4 Use of fmincon in MATLAB for process optimization ceq, and c describes the nonlinear equalities/inequalities among parameters A and b describes the linear inequalities among parameters Aeq and beq describes the linear equalities among parameters X = fmincon(fun, X0, A, B, Aeq, Beq, lb, ub) defines a set of lower and upper bounds on the design variables, X, so that a solution is found in the range LB <= X <= UB. Use empty matrices for LB and UB if no bounds exist. [x fval] = fmincon(fun,x0,a,b,aeq,beq,lb,ub,nonlcon,options) Estimated parameters Errors of predicted and observed results Function that you want to minimize, i.e. the error Initial guess of the parameters Boundaries of the parameters Choose the right solver The function NONLCON accepts X and returns the vectors C and Ceq, representing the nonlinear inequalities and equalities respectively.

5 Example 1 Find values of x that minimize starting at the point x = [10; 10; 10] and subject to the constraints X = fmincon(fun, X0, A, B, Aeq, Beq, lb, ub) minimizes FUN subject to the linear equalities Aeq*X = Beq as well as A*X <= B. (Set A=[] and B=[] if no inequalities exist.)

6 Use fmincon for optimization %file name example1 clc;close all;clear all; int_guess=[10;10;10];% INITIAL GUESS FOR U'S A=[ ;1 2 2]; % linear inequalities constraints B=[0 72]; [u]=fmincon(@eg_1,int_guess,a,b) % CALL FOR OPTIMIZER %file name eg_1.m function [err]=eg_1(u) x1=u(1); x2=u(2); x3=u(3); err=-x1*x2*x3;

7 Example 2 (dynamic optimization) Consider the batch reactor with following reaction A B C Find the temperature, at which the product B is maximum Mathematical Representation of the system is as :

8 clc;close all;clear all; warning off int_guess=300;% INITIAL GUESS FOR U'S LB=298; % LOWER BOUND OF U UB=398;% UPPER BOUND ON U [u,fval]=fmincon(@reactor_problem,int_guess,[],[],[],[],lb,ub,[]); % CALL FOR OPTIMIZER PROBABLY NOT TO CHANGE BE USER [err]=reactor_problem(u); load data_for_system plot(t,y(:,1),'k') hold on plot(t,y(:,2),'m') legend('opt c_a','opt c_b') u opt c A opt c B 0.2 T=

9 Call ODE to solve the model ode45 slot, init, [opt], [par1, par2,...]) Maximize B function [err]=reactor_problem(u) A=[1 0]; %initial condition [t,y]=ode15s(@reactor_ode,[0 1],A,[],u); %tspan=1 err=-y(end,2); %make the maximum B save data_for_system t Y %just for plotting function [YPRIME]=reactor_ODE(t,x,u) YPRIME=zeros(2,1); T=u; k1=4000*exp(-2500/t); k2=620000*exp(-5000/t); YPRIME(1)=-k1*(x(1))^2; YPRIME(2)=(k1*x(1)^2)-k2*x(2); ODE calculation

10 Part 2: Inverse problem (from experimental data to model construction) Parameter fitting for dynamic model

11 An example for parameter fitting of dynamic bioprocess model

12 A) Nonlinear Parameter Estimation via fmincon in MATLAB References (e.g. literature) Mathematic Model References (e.g. literature) Mathematic Model v.s. Known Parameters Unknown Parameters Simulated Results Parameter estimation Simulated Results v.s. ε Forward problem Experimental Results Inverse problem Experimental Results

13 Nonlinear Parameter Estimation 1. Parameter estimation for linear systems y = A x + b y, x are the vector of dependent and independent variables, A, b are parameters to be estimated regress command in MATLAB (for linear model) 2. Parameter estimation for nonlinear systems y = f (parameters, x) f is the nonlinear function of the estimated parameters For example, y=β 1 +β 2 sin(β 3 t) cftool box for more complicated model equations 3. Parameter estimation for dynamic ode models ( fmincon, nlinfit, etc. in MATLAB)

14 Example 1: parameter estimation in bioengineering Bioreactor model dx dt dp dt ds dt X Y K P / X 1 Y max S X / S S S X X X Five parameters to be estimated: Y P/X, Y X/S, µ max, K s, and S(0) Initial conditions: X(0) = 0.05 g/l; P(0) = 0 g/l; Experimental data t X P S We will use fmincon to find the parameters

15 function [error,ypred] = reactor_model(beta) The Error Function global yobs global t %reactor model S0=beta(5); y0=[ S0]; tspan=t; %we want y at every t [t,y]=ode45(@ff,tspan,y0); function dy = ff(t,y) %function that computes the dydt umax=beta(1); Ks=beta(2); Ypx=beta(3); Yxs=beta(4); X=y(1); P=y(2); S=y(3); u=umax.*s./(ks+s); dy(1)=u.*x; dy(2)=ypx.*u.*x; dy(3)=-1/yxs.*u.*x-u.*x; dy=dy'; end y1=y(:,1); y2=y(:,2); y3=y(:,3); ypred=[y1; y2; y3]; error=sum((ypred-yobs).^2); end Input the observed data Settings to run ODEs The ODEs that describe bioreactor behaviors Calculate the error

16 The fmincon Function %Use fmincon to find the parameters in reactor model clear clc data =xlsread('reactor_model_data.xlsx'); t=data(:,1); X=data(:,2); P=data(:,3); S=data(:,4); x=t; Read data from Excel global yobs global t yobs=[x;p;s]; umax=0.1; Ks=10; Ypx=0.11; Yxs=0.45; S0=14.86; beta0(1)=umax; %initial guess beta0(2)=ks; %initial guess beta0(3)=ypx; %initial guess beta0(4)=yxs; %initial guess beta0(5)=s0; %initial guess Set up the initial guess of parameters

17 %fmincon A=[]; b=[]; Aeq=[]; beq=[]; lb=zeros(5,1); ub=lb+20; nonlcon=[]; x0=beta0;%lb+rand.*(ub-lb); options=optimset('algorithm','interior-point'); The fmincon Function (continued) Q: How to find the global solution? [param, fval] = fmincon(fun,x0,a,b,aeq,beq,lb,ub,nonlcon,options) Settings to run fmincon %get the predicted resutls [error,ypred]=reactor_model(param); n=length(t); Xpred=ypred(1:n); Ppred=ypred(n+1:2*n); Spred=ypred(2*n+1:3*n); A: Randomly perturb the initial guess Run the model again to get the predicted results figure(1) set(gca, 'fontsize',14,'fontweight','bold'); hold on plot(x,xpred,'-r','linewidth',2.5); plot(x,ppred,'-g','linewidth',2.5) plot(x,spred,'-b','linewidth',2.5) plot(x,x,'r+','markersize',10); plot(x,p,'go','markersize',10) plot(x,s,'bx','markersize',10) xlabel('time') ylabel('y') legend('x g/l','p g/l','s g/l') Plot the predicted v.s. observed results

18 Parameters Estimated via fmincon param = fval = µmax = (1/h) Ks = (g/l) Ypx = (g/g) Yxs = (g/g) S0 = (g/l)

19 Estimating confidence intervals of parameters via fmincon Which one makes sense? Numbers in the brackets are the 95% confidence interval of µmax µmax = [ ] (1/h) Or µmax = [ ] (1/h) Estimation of µmax is reliable µmax cannot be accurately estimated How to estimate the confidence intervals of parameters? bootstrap method: randomly resample the experimental observations; and for each new set of sampled experimental observations, use fmincon to find a new set of parameters. Monte Carlo method: randomly perturb the experimental observations within the measurement errors (i.e. standard derivation); and for each new set of perturbed experimental observations, use fmincon to find a new set of parameters.

20 B: Parameter fitting using nlinfit Unlike forward problems, inverse problems require experimental data, and an iterative solution. Because inverse problems require solving the forward problem numerous time, the ode45 solver will be nested within a nonlinear regression routine called nlinfit. The syntax is: [param, r, J, COVB, mse] = nlinfit(x, y, fun, beta0); returns the fitted coefficients param, the residuals r, the Jacobian J of function fun, the estimated covariance matrix COVB for the fitted coefficients, and an estimate MSE of the variance of the error term. X is a matrix of n rows of the independent variable y is n-by-1 vector of the observed data fun is a function handle to a separate m-file to a function of this form: yhat = fun(b,x) where yhat is an n-by-1 vector of the predicted responses, and b is a vector of the parameter values. beta0 is the initial guesses of the parameters.

21 y Example 1: Model fitting ODE equation dy k y dt data =xlsread('exp_data.xls'); %read data from excel yo=100; k=0.6; beta0(1)=yo; beta0(2)=k; ypred yobs Based on experimental data, find y0 and k time (min) x=data(:,1); yobs=data(:,2); [param,resids,j,covb,mse] = nlinfit(x,yobs,'forderinv',beta0); rmse=sqrt(mse); %root mean square error = SS/(n-p) %R is the correlation matrix for the parameters, sigma is the standard error vector [R,sigma]=corrcov(COVB); %confidence intervals for parameters ci=nlparci(param,resids,j); %computed Cpredicted by solving ode45 once with the estimated parameters ypred=forderinv(param,x); %mean of the residuals meanr=mean(resids);

22 Redisuals: the residual of an observed value is the difference between the observed value and the estimated function value. The Jacobian determinant carries important information about the local behavior of F. The root-mean-square error (RMSE) is a measure of the differences between value (Sample and population values) predicted by a model or an estimator and the values actually observed. The covariance matrix generalizes the notion of variance to multiple dimensions. R = corrcov(c) computes the correlation matrix R that corresponds to the covariance matrix C. [R,sigma]=corrcov(COVB); nlparci : The confidence interval calculation is valid when the length of RESID exceeds the length of BETA, and J has full column rank. When J is ill-conditioned, confidence intervals may be inaccurate.

23 figure hold on h1(1)=plot(x,ypred,'-','linewidth',3); %predicted y values h1(2)=plot(x,yobs,'square', 'Markerfacecolor', 'r'); legend(h1,'ypred','yobs') xlabel('time (min)') ylabel('y') %residual scatter plot figure hold on plot(x, resids, 'square','markerfacecolor', 'b'); YLine = [0 0]; XLine = [0 max(x)]; plot (XLine, YLine,'R'); %plot a straight red line at zero ylabel('observed y - Predicted y') xlabel('time (min) ) Function with ode45 function y = forderinv(param,t) %first-order reaction equation tspan=t; %we want y at every t [t,y]=ode45(@ff, tspan, param(1)); %param(1) is y(0) function dy = ff(t, y) %function that computes the dydt dy(1)= -param(2)*y(1); end end

Observed y - Predicted y y 120 100 80 60 40 20 0-20 0 1 2 3 4 5 6 7 8 9 10 time (min) 15 10 5 0-5 -10 ypred yobs One ODE -15 0 1 2 3 4 5 6 7 8 9 10 time (min) Time y 0 107.2637 0.10101 102.9394 0.

24 Observed y - Predicted y y time (min) ypred yobs One ODE time (min) Time y Raw data File name: exp_data.xls Time y

25 y Example 2: Three ODES for batch fermentation 15 S 10 /s 5 X P Yp/s=Yx/s*Yp/x time (min)

26 Three ODEs parameter fitting File name: HW217.xls t X P S

27 Fitting four parameters Script clear all %clear all variables global y0 data =xlsread('hw217.xls'); %initial conditions y0=[ ]; %initial guess Umax=0.1; Ks=10; Ypx=0.11; Yxs=0.45; beta0(1)=umax; beta0(2)=ks; beta0(3)=ypx; beta0(4)=yxs; %Measured data x=data(:,1); yobsx=data(:,2); yobsp=data(:,3); yobss=data(:,4); yobs=[yobsx; yobsp; yobss]; %nlinfit returns parameters, residuals, Jacobian (sensitivity coefficient matrix), %covariance matrix, and mean square error. ode45 is solved many times %iteratively [param,resids,j,covb,mse] = nlinfit(x, yobs,'forderinv2', beta0);

28 rmse=sqrt(mse); %root mean square error = SS/(n-p) n=size(x); nn=n(1); %confidence intervals for parameters ci=nlparci(param,resids,j); %computed Cpredicted by solving ode45 once with the estimated parameters ypred=forderinv2(param,x); ypredx=ypred(1:nn); ypredp=ypred(nn+1:2*nn); ypreds=ypred(2*nn+1:3*nn); %mean of the residuals meanr=mean(resids); Script

29 Script figure hold on plot(x, ypredx, x, ypredp, x, ypreds); %predicted y values plot(x, yobsx, 'r+', x, yobsp, 'ro', x, yobss, 'rx'); xlabel('time (min)') ylabel('y') %residual scatter plot x3=[x; x; x]; figure hold on plot(x3, resids, 'square', 'Markerfacecolor', 'b'); YLine = [0 0]; XLine = [0 max(x)]; plot (XLine, YLine,'R'); %plot a straight red line at zero ylabel('observed y - Predicted y') xlabel('time (min) )

30 function y = forderinv2(param,t) %first-order reaction equation global y0; tspan=t; %we want y at every t [t,y]=ode45(@ff,tspan,y0); %param(1) is y(0) function dy = ff(t,y) %function that computes the dydt dy(1)= param(1)*y(3)/(param(2)+y(3))*y(1); %biomass dy(2)= param(1)*param(3)*y(3)/(param(2)+y(3))*y(1); %product dy(3)= -1/param(4)*dy(1)-1/(param(3)*param(4))*dy(2); %substrate dy=dy'; end % after the ode45, rearrange the n-by-3 y matrix into a 3n-by-1 matrix % and send that back to nlinfit. y1=y(:,1); y2=y(:,2); y3=y(:,3); y=[y1; y2; y3]; end Function and sub-function

Data Assimilation. Matylda Jab lońska. University of Dar es Salaam, June Laboratory of Applied Mathematics Lappeenranta University of Technology

Data Assimilation. Matylda Jab lońska. University of Dar es Salaam, June Laboratory of Applied Mathematics Lappeenranta University of Technology Laboratory of Applied Mathematics Lappeenranta University of Technology University of Dar es Salaam, June 2013 Overview 1 Empirical modelling 2 Overview Empirical modelling Experimental plans for various