Multiplicative Algorithm for computing Optimum Designs

Transcription

1 for computing s DAE Conference 2012 Roberto Dorta-Guerra, Raúl Martín Martín, Mercedes Fernández-Guerrero and Licesio J. Rodríguez-Aragón Optimum Experimental Design group October 20, October 20th / 19

2 Overview.. : Enzyme Inhibition. pv T Measurements. between models. Adsorption Isoterms. Results Discussion. October 20th / 19

3 Objectives Determination of optimum designs is not a straightforward process, even for moderate examples. New schemes for developing iterative algorithms, based on the multiplicative algorithm (Torsney and Martín-Martín, 2009), devoted to the numerical construction of optimum designs are being sought. Provide experimenters with designs that would minimize the experimental cost of measurements while obtaining the most accurate characterization of the mechanisms. October 20th / 19

4 Background October 20th / 19

5 Background for nonlinear models Background Consider the general non-linear regression model y = η(x; θ) + ε, x X, where the random variables ε are independent and normally distributed with zero mean and constant variance σ 2 and θ is the unknown parameter vector. X, called design space. Let Ξ be the set of probability distributions on the Borel sets of X, then any ξ Ξ satisfying ξ(dx) = 1, ξ(x i ) 0, x X, X is called a design measure, or an approximate design. October 20th / 19

6 Background for nonlinear models Background The most common method for analyzing data from a non-linear model is based on the use of the linear Taylor series approximation of the response surface, η(x; θ) η(x; θ 0 ) + [ η(x; θ) θ0 ] t (θ θ 0 ), where denotes the gradient with respect to θ and being θ 0 a prior value of θ. The variance-covariance matrix of the least square estimator of θ is asymptotically approximated by the inverse of the information matrix induced by ξ, M(ξ, θ) = η(x; θ)[ η(x; θ)] t dξ(x). χ October 20th / 19

7 Background for nonlinear models Background Let φ be a real-valued function defined on the k k symmetric matrices and bounded above on {M(ξ, θ) : ξ Ξ}. The optimum design problem is concerned with finding ξ such that φ(m(ξ, θ)) = min ξ Ξ φ(m(ξ, θ)) which is called a φ-optimum design. Caratheodory s theorem implies that for a model with k parameters, a locally φ-optimum design supported at no more than k(k + 1)/2 + 1 points exists. For convenience, designs will be described using a two row matrix, { } x1 x ξ = 2... x N, ξ 1 ξ 2... ξ N being ξ 1,...,ξ N non negative real numbers which sum up to one. October 20th / 19

8 Background for nonlinear models Background The D optimum criterion minimizes the volume of the confidence ellipsoid of the parameters and is given by φ D [M(ξ, θ)] = det{m(ξ, θ)} 1/k, where k is the number of parameters in the model. It is known that this criterion is a convex and non-increasing function of the designs and so, designs with a small criterion value are desirable. A design that minimizes φ D over all the designs on X is called a D optimum design. October 20th / 19

9 Background for nonlinear models Background Since the criterion is convex, standard convex analysis arguments using directional derivatives, when φ is differentiable, show that a design ξ is φ optimum if and only if F φ [ M(ξ, θ), M(ξ xj, θ) ] F φ (ξ, e j ) = d j N ξjd j 0, j=1 with equality at the support points of ξ. M(ξ xj, θ) is the Information Matrix for a single observation at the point x j, e j denotes the j-th unit vector in R N and d j = φ/ ξ xj. The General Equivalence Theorem states that the variance function evaluated using a φ-optimum design achieves its maximum value at the support points of this design. October 20th / 19

10 October 20th / 19

11 Numerical Aproximation of Optimum Design The purpose of experimental design is to select the design points x 1,...,x N and the corresponding weights ξ 1,...,ξ N so that the design ξ is optimum for some criterion, φ. A simple case is the determination of the best N-point φ-optimum design. Let x 1, x 2,...,x N be its support points, x i X = [a, b]. Let: W h = x h x h 1 b a, h = 1,...,N + 1, where x 0 = a and x N+1 = b. The variables W h satisfy W h 0, N+1 h=1 W h = 1. Transformation was proposed by Torsney and Martín-Martín (2009). October 20th / 19

12 Numerical Aproximation of Optimum Design Thus, we have an example of the following type of optimization problem: Minimize a criterion φ(w, ξ) over P = { W = (W 1,...,W N+1 ), ξ = (ξ 1,...,ξ N ) : N+1 N W h 0, W h = 1 and ξ j 0, ξ j = 1 }. h=1 j=1 Then the following simultaneous approaches are used W (r+1) h = W (r) h N+1 t=1 W (r) t g(f (r) h, δ 1) j = g(f (r) t, δ 1 ), ξ(r+1) ξ (r) j g(f (r) j, δ 2 ) N i=1 ξ(r) i g(f (r) i, δ 2 ) where r = 0, 1,... is the iteration number, g(f h, δ 1 ) and g(f j, δ 2 ), are positive increasing, and if δ = 0, g(f, δ) are constants. October 20th / 19

13 Numerical Aproximation of Optimum Design Being F j and F h the vertex directional derivatives: F h = F φ (W, e h ) = d h W h d h, F j = F φ (ξ, e j ) = d j ξ j d j, d h = φ W h, d j = φ ξ j. Therefore if φ is a convex and differentiable function a design ξ will be optimum if and only if, F h = F φ (W, e h ) = F j = F φ (ξ, e j ) = { = 0, for W h > 0 0 for Wh { = 0, = 0, for ξ (x j ) > 0 0 for ξ (x j ) = 0. October 20th / 19

14 Enzyme Inhibition pv T Measurements October 20th / 19

15 Enzyme Inhibition pv T Measurements Searching for optimally designed experiments with more than one independent factor is more complicated than for models with a single factor. Enzyme Inhibition. Pressure, Volume and Temperature measurements. October 20th / 19

16 Enzyme Inhibition Enzyme Inhibition pv T Measurements Enzyme kinetics is the study of the chemical reactions that are catalyzed by enzymes, with a focus on their reaction rates. The binding of an inhibitor can stop a substrate from entering the enzyme s active site and/or hinder the enzyme from catalyzing its reaction. Inhibitor binding is either reversible or irreversible. Enzyme assays are laboratory procedures that measure the rate of enzyme reactions. October 20th / 19

17 Enzyme Inhibition Enzyme Inhibition pv T Measurements This kinetic model is relevant to situations where very simple kinetics can be assumed, (i.e. there is no intermediate or product inhibition). V = V max [S] ( ) ( ) K M 1 + [I] K i + [S] 1 + [I] K i Competitive inhibitors can bind to E, but not to ES. Non-competitive inhibitors have identical affinities for E and ES (K i = K i ). Mixed-type inhibitors bind to both E and ES, but their affinities for these two forms of the enzyme are different (K i K i ). Unknown parameters θ = (V max, K M, K i, K i ). October 20th / 19

18 Enzyme Inhibition Enzyme Inhibition pv T Measurements Example: Considering Polifenol Oxidasa as enzyme, 4-Metylcatecol as substrate and Cianimic acid as a competitive inhibitor, S I = [5, 50] [0, 1.2]µM and θ 0 = (V, K m, K i ) = (131.4, 12.5, 0.42). ξ D = { (8.33, 0) (48, 26, 1.2) (50, 0) }. October 20th / 19

19 Enzyme Inhibition Enzyme Inhibition pv T Measurements D-optima designs for these type of models have been analytically computed by Bogacka et al. (2011). Comparison of convergence rate of the (red) with Wynn-Fedorov (blue) have been obtained: October 20th / 19

20 pv T Measurements Enzyme Inhibition pv T Measurements The characterization of volume or density as a function of temperature and pressure is particularly important for the design of industrial plants, pipelines and pumps. In order to correlate the density values over the temperature and pressure intervals, the following Tait-like equation is used, E(ρ) = η(p, T; θ) = ρ 0 (T) 1 C(T) log B(T)+p, var(ρ) = σ 2. B(T)+p 0 ρ 0 (T) is either a linear function of the required degree or a non-linear function, known as Rackett equation.b(t) and C(T) are linear functions. X = P T, where P and T are permissible ranges of values for p and T. Being θ t = (A 0,...,B 0,...,C 0,...) the set of unknown parameters. October 20th / 19

21 pv T Measurements Enzyme Inhibition pv T Measurements Example: In order to characterize changes of density of 1-phenylundecane. X = P T = [0.1, 65]MPa [293.15, 353, 15]K and the set of best guesses of the parameters are obtained from Milhet et al. (2005), θ 0 = (A 0, A 1, A 2, A 3, B 0, B 1, B 2, C) t Pressure Temperature October 20th / 19

22 pv T Measurements Enzyme Inhibition pv T Measurements The obtained design, ξd is supported at 11 points with different proportions of observations for each point. In this case, 8 of the support points lie at the boundaries of the design space X, while two of them belong to its interior. ξ D = (0.1, ) 0.12 (0.1, ) 0.07 (0.1, ) 0.10 (0.1, ) 0.12 (28.58,308.91) 0.05 (28.58,339.50) 0.07 (28.58, ) 0.07 (65, ) 0.12 (65, ) 0.07 (65, ) 0.09 (65, ) 0.12 t. October 20th / 19

23 pv T Measurements Enzyme Inhibition pv T Measurements Example:In the work of Outcalt and Laesecke (2010), measurements are taken over JP-10. Because of its high thermal stability, high energy density, low cost, and widespread availability, JP-10 is being investigated as a fuel to be used in pulse-detonation engines. In this case the dependence of ρ 0 is characterized by the Racket equation: ρ 0 (T) = A R /B [1+(1 (T/C R)) D R] R. Measurements are taken over X = P T = [0.083, 30]MPa [270, 470]K and the set of best guesses are obtained from Outcalt and Laesecke (2010), θ 0 = (A R, B R, C R, D R, B 0, B 1, B 2, C) t. October 20th / 19

24 pv T Measurements Enzyme Inhibition pv T Measurements The obtained design, ξd is supported at 10 points with different proportions of observations for each point. 9 of the support points lie in the boundary while only one is in the interior of X. ξ D = (0.083, 270) 0.11 (0.083, ) 0.08 (0.083, ) 0.12 (0.083, 470) 0.12 (12.51,439.1) 0.02 (12.60, 470) 0.12 (30, 270) 0.11 (30, ) 0.09 (30, ) 0.11 (30, 470) 0.12 t. October 20th / 19

25 pv T Measurements Enzyme Inhibition pv T Measurements The Equivalence Theorem proves D optimality of the design. Figure shows the attainment of equality to k = 8 at the support points Pressure Temperature October 20th / 19

26 pv T Measurements Enzyme Inhibition pv T Measurements For the extension of the multiplicative algorithm to a two-factor non-linear model, the transformation of pressure values, p i, i = 1,...,N, into a marginal distribution or set of weights W ph, h = 1,...,N + 1, is proposed. Then, the corresponding temperatures T paired with each value p i, T s pi, s = 1,...,q, are transformed into a conditional distribution W Ts p i, s = 1,...,q. On the other hand, design weights need to be simultaneously determined. These conditional and marginal distributions must be optimally chosen. According to Caratheodory s theorem we initially assume k(k + 1)/2 + 1 support points. The optimization problem can be stated as a problem with respect to N + 2 distributions. October 20th / 19

27 pv T Measurements T b W Tq 1 p i p i,t q pi Enzyme Inhibition pv T Measurements Ta,Tb... p i,t 2 pi W T2 p i p i,t 1 pi W p1 W p2 W p3 W T1 p i W pn 1 T a p a p 1 p 2 p 3... p i... p N p b p a,p b October 20th / 19

28 pv T Measurements This leads to the following simultaneous multiplicative iterations, Enzyme Inhibition pv T Measurements W (r+1) p h = W (r+1) T s p i = ξ (r+1) t = W p (r) h g(f (r) p h, δ 1 ) N+1 j=1 W p (r) j g(f p (r) j, δ 1 ) W (r) T s p i g(f (r) T s p i, δ 2 ) q+1 l=1 W (r) T l p i g(f (r) T l p i, δ 2 ) ξ (r) t qn i=1 ξ(r) i g(f (r) t, δ 3 ) g(f (r) i, δ 3 ), h = 1,..., N + 1,, s = 1,...,q + 1, i = 1,...,N,, t = 1,...,qN, where g(f, δ) = Φ(δF), and F p, F T p, F t are the vertex directional derivatives. October 20th / 19

29 between models Atkinson-Fedorov Algorith Adsorption Isoterms Results Discussion October 20th / 19

30 between models between models Atkinson-Fedorov Algorith Adsorption Isoterms Results Discussion Atkinson and Fedorov (1975a,b) introduced the so called T-optimality criterion which has an interesting statistical interpretation as the power of a test for the fit of a second model when the other is considered as the true model. Usually there is no closed form for the T-optimum design and it must be computed through an iterative procedure. October 20th / 19

31 between models between models Atkinson-Fedorov Algorith Adsorption Isoterms Results Discussion Let us assume that the function η(x i, θ) coincides with either η 1 (x, θ 1 ) or η 2 (x, θ 2 ) partially known functions where θ 1 Ω 1 R m 1 and θ 2 Ω 2 R m 2 are the unknown parameter vectors. Let us assume that η(x, θ) = η 1 (x, θ 1 ) is the true model of the process with parameters θ 1 known and η 2 is the rival model. Atkinson and Fedorov (1975a,b) introduced the notion of T -optimality. T 21 (ξ) = min [η(x i, θ) η 2 (x i, θ 2 )] 2 ξ(x i ). θ 2 Ω 2 x i χ The design ξ which maximizes T 21 (ξ) is called the T -optimal design. October 20th / 19

32 between models between models Atkinson-Fedorov Algorith Adsorption Isoterms Results Discussion A design for which the optimization problem θ 2 arg min θ 2 Ω 2 k [ η(xi ) η 2 (x i, θ 2 ) ] 2 ξi i=1 has no unique solution is a singular design, otherwise is called regular. For regular designs the Equivalence Theorem is applicable with the implication: a design ξ is T -optimal if and only if, F j (ξ) = [ η(x j, θ) η 2 (x j, θ 2 ) ] k 2 [ η(xi, θ) η 2 (x i, θ 2 ) ] 2 ξi 0, i=1 with equality at the support points. October 20th / 19

33 Atkinson-Fedorov Algorith between models Atkinson-Fedorov Algorith Adsorption Isoterms Results Discussion Based on the Equivalence Theorem Atkinson and Fedorov (1975a,b) provided the following algorithm: { } For a given initial design ξ (0) x1... x k = k ξ 1... ξ k determine Find the point θ 2 = arg min θ 2 Ω 2 k [ η(xi ) η 2 (x i, θ 2 ) ] 2 ξi i=1 x k+1 = argmax x χ [ η(x) η2 (x, θ 2 ) ] 2 October 20th / 19

34 Atkinson-Fedorov Algorith between models Atkinson-Fedorov Algorith Adsorption Isoterms Results Discussion Let ξ xk+1 be a design with measure concentrated at the single point x k+1, { } xk+1 ξ xk+1 = 1 A new design is constructed in the following way: ξ k+1 = (1 α k+1 )ξ k + α k+1 ξ xk+1 where typical conditions for the sequence {α k } are lim k α k = 0, k=0 α k =, k=0 α2 k <. In our work α s = 1/(s + 1) has been used. October 20th / 19

35 between models Atkinson-Fedorov Algorith Adsorption Isoterms Results Discussion Assuming that each ξ is regular, i.e. Ω 2 (ξ) = θ 2, this first optimization problem was solved using a quasi-newton algorithm through the FORTRAN IMLS routine DBCONF. Then we wish to choose a design ξ optimally, that is, we wish to determine both the support points and the design weights optimally ξ = arg max (x,ξ(x)) [η(x i, θ) η 2 (x i, θ 2 )] 2 ξ(x i ) = arg max x i χ (x,ξ(x)) T 21 (ξ). In this case, the equivalence theorem says nothing about the number of support points of an optimal design. We consider designs with L support points, x 1,...,x L, where L is an appropriate number for each problem. October 20th / 19

36 between models Atkinson-Fedorov Algorith Adsorption Isoterms Results Discussion Let W t = x t x t 1 b a t = 1,...,L + 1 where x 0 = a and x L+1 = b. We have transformed from L variables to L + 1 variables, but these must satisfy W t 0 and t W t = 1. As in Torsney and Martín-Martín (2009) we have an optimization problem with respect to two distributions one defined by the design points and one defined by the design weights. October 20th / 19

37 between models Atkinson-Fedorov Algorith Adsorption Isoterms Results Discussion The multiplicative algorithm extends naturally to the two simultaneous multiplicative iterations: W (r+1) t = ξ (r+1) j = W (r) t L+1 h=1 W (r) h ξ (r) j g(f (r) t, δ 1 ) g(f (r) h, δ 1) g(f (r) j, δ 2 ) L i=1 ξ(r) i g(f (r) i, δ 2 ) Being F j and F h the vertex directional derivatives of T 21 at W and ξ : F h = F T21 (W, e h ) = d h W h d h F j = F T21 (ξ, e j ) = d j ξ j d j Where d h = T 21 W h and d j = T 21 ξ j. October 20th / 19

38 The same first order conditions for a local maximum are used: between models Atkinson-Fedorov Algorith Adsorption Isoterms Results Discussion F h = F T 21 (W h, e h) = F j = F T21 (ξ, e j ) = { = 0, for W h > 0 0, for Wh { = 0 = 0, for ξ j > 0 0, for ξj = 0 October 20th / 19

39 Adsorption Isoterms between models Atkinson-Fedorov Algorith Adsorption Isoterms Results Discussion The modeling of the adsorption phenomena in many chemical and industrial processes has proved to be of great interest. The two models used in the literature to describe the relationship between the amount of gas or water adsorbed, w e, in terms of water activity a w for multilayer adsorption phenomena are the Brunauer-Emmett-Teller (BET) model and the extension known as Guggenheim-Anderson-de Boer (GAB) model. BET model: E[w e ] = w mb c B a w (1 a w )(1 + (c B 1)a w ), unknown parameters: θb t = (w mb, c B ). w mg c G ka w GAB model: E[w e ] = (1 ka w )(1 + (c G 1)ka w ) unknown parameters: θ t G = (w mg, c G, k). October 20th / 19

40 Adsorption Isoterms between models Atkinson-Fedorov Algorith Adsorption Isoterms Results Discussion Cepeda et al. (1999) studied water sorption behavior of coffee for predicting hygroscopic properties as well as designing units for its optimum preservation, storage, etc. The results at 25 C 0 for the GAB adsorption isotherm were w mg = g of H 2 O adsorbed/g of coffee, c G = and k = October 20th / 19

41 Adsorption Isoterms between models Atkinson-Fedorov Algorith Adsorption Isoterms Results Discussion Rodríguez-Aragón and López-Fidalgo (2007) provided the T-optimal design considering the GAB model as the true model using the Atkinson-Fedorov algorithm. After 182 iterations of the algorithm, a design supported at three experimental points was obtained, with a lower efficiency bound of { } ξ = For the same problem the new approach was applied. After 129 iterations of this algorithm, the T-optimum design was obtained, with a lower efficiency bound of October 20th / 19

42 Results Discussion between models Atkinson-Fedorov Algorith Adsorption Isoterms Results Discussion Advantages of computing optima designs with the : Effectiveness to compute D optimum designs for multifactor modesl. T optimization with simpler computations. Locally optimum designs: designs depend on the initial best guesses of the parameters. Open issues: Choice of function g and constants δ, Torsney and Mandal (2006). Possibility of computing ˆθ 2 using the multiplicative algorithm. October 20th / 19

43 October 20th / 19

44 B. Torsney and R. Martín-Martín. algorithms for computing optimum designs. Journal of Statistical Planning and Inference, 139(12): , 2009 B. Bogacka, M. Patan, P. Johnson, K. Youdim, and A. C. Atkinson. Optimum designs for enzyme inhibition kinetic models. Journal of Biopharmaceutical Statistics, 21: , 2011 M. Milhet, A. Baylaucq, and C. Boned. Volumetric properties of 1-phenyldecane and 1-phenylundecane at pressures to 65 MPa and temperature between and K. J. Chem. Eng. Data, 50(4): , 2005 S. L. Outcalt and A. Laesecke. Measurement of density and speed of sound of JP-10 and comparsion to rocket propellants and jet fuels. Energy Fuels, 25: , 2010 A. C. Atkinson and V. V. Fedorov. The design of experiments for discriminating between two rival models. Biometrika, 62:57 70, 1975a A. C. Atkinson and V. V. Fedorov. Optimal design: Experiments for discriminating between several models. Biometrika, 62: , 1975b E. Cepeda, R. Ortíz de Latierro, M. J. San José, and M. Olazar. Water sorption isotherms of roasted coffee and coffee roasted with sugar. Int. J. Food Sci. Technol., 34, 1999 L. J. Rodríguez-Aragón and J. López-Fidalgo. T, D and c optimum designs for BET and GAB adsorption isotherms. Chemom. Intell. Lab. Syst., 89:36 44, 2007 B. Torsney and S. Mandal. Two classes of multiplicative algorithms for construction optimizing distributions. Computational Statistics and Data Analysis, 51: , October 20th / 19