SENSITIVITY ANALYSIS FOR ATMOSPHERIC CHEMISTRY MODELS VIA AUTOMATIC DIFFERENTIATION

Transcription

1 REPORT ON COMPUTATIONAL MATHEMATICS 7/99, DEPARTMENT OF MATHEMATICS, THE UNI- VERSITY OF IOWA SENSITIVITY ANALYSIS FOR ATMOSPHERIC CHEMISTRY MODELS VIA AUTOMATIC DIFFERENTIATION A. SANDU, G. R. CARMICHAEL, AND F. A. POTRA Abstract. Automatic differentiation techniques are used in the sensitivity analysis of a comprehensive atmospheric chemical mechanism. Specifically, ADIFOR software is used to calculate the sensitivity of ozone with respect to all initial concentrations (of 8 species) and all reaction rate constants (78 chemical reactions) for six different chemical regions. Numerical aspects of the application of ADIFOR are also presented. Automatic differentiation is shown to be a powerful tool for sensitivity analysis. Key words. Atmospheric chemistry, Automatic Differentiation.. Introduction. Comprehensive sensitivity analysis of air pollution models remains the exception rather than the rule. Most sensitivity analysis applied to air pollution modeling studies has focused on the calculation of the local sensitivities (first order derivatives of output variables with respect to model parameters) in box model studies of gas phase chemistry (see [8]). The most common form of sensitivity studies with comprehensive atmospheric chemistry/transport models has been done using the so-called brute force method, i.e., a number of input parameters are selected to be varied and the simulation results are then compared. This method becomes less viable as the model becomes more comprehensive. A variety of alternative techniques are available including Green s function analysis (see [8]), adjoint models and several variations of the direct decoupled methods ( see [9]). A recently developed technique for sensitivity study is automatic differentiation technology. Automatic differentiation is implemented by pre compilers that analyze the code written for evaluating a function of several variables. These pre compilers automatically add instructions needed to compute the required derivatives by properly handling quantities that are common to the function and its derivatives and by efficient use of available derivatives in a library. The resulting expanded code is then compiled with a standard compiler into an object code that can simultaneously evaluate derivatives and function values. This approach is superior to finite difference approximation of the derivatives because the numerical values of the computed This work was supported by the grant no. DE FG0 9ER8 from the Department of Energy and by the Center for Global and Regional Environmental Research. Program in Applied Mathematics and Computational Sciences, The University of Iowa, Iowa City, IA (sandu@cgrer.uiowa.edu). Center for Global and Regional Environmental Research and the Department of Chemical and Biochemical Engineering The University of Iowa, Iowa City, IA (gcarmich@icaen.uiowa.edu). Departments of Mathematics and Computer Science, The University of Iowa, Iowa City, IA (potra@math.uiowa.edu).

2 A. SANDU,G. R. CARMICHAEL, AND F. A. POTRA derivatives are much more accurate and the computational effort is significantly lower (see [], []). A promising new implementation developed at Argonne National Laboratory and Rice University over the last couple of years and which has recently been awarded the 99 Wilkinson Prize for Numerical Software is the package ADIFOR (Automatic Differentiation in FORTRAN, see []). It adopts a hybrid approach to computing derivatives that is generally based on the forward mode, but uses the reverse mode to compute the derivatives of assignment statements containing complicated expressions. The forward mode acts similarly to the usual application of the chain rule in calculus. At each computational stage derivatives of the intermediate variables with respect to input variables are computed and are propagated forward through the computational stages. On the other hand the reverse mode is based on adjoint quantities representing derivatives of the output with respect to intermediate variables. The adjoints are computed at each node of the computational graph and they are propagated by reversing the flow of the program and by recomputing intermediate values that have a nonlinear impact on the output. It turns out that this approach is related to the adjoint sensitivity analysis used in such various fields as nuclear engineering (see []), weather forecast (see []) and neural networks (see [0]). Because assignment statements compute generally one dependent variable in terms of several dependent variables, the reverse mode is very efficient and can be implemented as in-line code. ADIFOR requires the user to supply the FORTRAN source code for the function value and for all lower level subroutines as well as a list of the independent and dependent variables in the form of parameter lists or common blocks. ADIFOR then determines which other variables throughout the programs are to be differentiated, and augments the original code with derivative statements. The augmented code is then optimized by eliminating unnecessary operations and temporary variables. The FORTRAN code generated by ADIFOR requires no run-time support and therefore can be ported between different computing environments. In this paper we demonstrate the use of ADIFOR in the sensitivity analysis of a comprehensive gas phase chemical box model. We first present a general description of automatic differentiation and its application along with chemical integrators. Automatic differentiation is then used to calculate first order sensitivities with respect to initial conditions and chemical reaction rate constants for a wide range of chemical conditions corresponding to the IPCC Chemistry Intercomparison study (see [8]).. Preliminaries. If c i (t) is the concentration of the i th species, the kinetics of a chemical system is described as an initial value problem: () dc i (t) = f(t, c,..., c n, β,..., β m ), dt c i (t 0 ) = c 0 i, i = n Intergovernmental Panel on Climate Change.

3 SENSITIVITY ANALYSIS FOR ATMOSPHERIC CHEMISTRY VIA AD where β j, j =,..., m are the parameters of the system (for example, reaction rate constants, etc). The system () can be rewritten in matrix production - destruction form as: () dc(t) dt = P (c(t)) D(c(t)) c(t) where P R n, D R n n, D = diag(d i ) are the production and the destruction terms, respectively. The local sensitivity coefficients are defined as: () s i,j (t) = c i α j where α j represent either the initial values c 0 j or the parameter β j. The term local refers to the fact that these sensitivities describe the system around a given set of values for the parameters α. The system being considered to respond linearly for small perturbations, s i,j measures the ratio between the effect (absolute variation of the output c i ) and the cause (absolute variation of the input α j ). It is sometimes useful to consider the ratio between the relative variation of the output and the relative variation of the input: s i,j = α j c i s i,j = ln(c i) ln(α j ) c i c i ( αj where s i,j is the normalized sensitivity coefficient. The normalized coefficients have the advantage of being adimensional, hence useful when comparing sensitivities with respect to parameters whose absolute values are orders of magnitude apart. If α j = α j then α j = α j + α j = α j and we have the following interpretation: s i,j is the relative variation in c i when the parameter α j doubles its value, if the system responded linearly. The disadvantage of the normalized coefficients is that they are not defined for c i = 0. To overcome this drawback, one may consider the semi-normalised sensitivities s + i,j, representing the absolute variation in c i when α j doubles its value (if the system responded linearly): s + i,j = α j s i,j = c i ln(α j ) c i α j ( αj. Overview of some computational methods for sensitivity analysis. There are many ways to compute sensitivities. In this paper we employ three different methods to compute the local sensitivity coefficients. Conceptually, all three are Other techniques for sensitivity analysis, besides those described here, are available as well; one could mention Green s function method and adjoint models. Their description is beyond the scope of this paper. α j ) )

4 A. SANDU,G. R. CARMICHAEL, AND F. A. POTRA equivalent, in the sense that a small perturbation of a certain input is propagated forward through the system, and the corresponding deviation of all outputs is estimated. Thus, all the methods described below may be called forward propagation methods. They are effective when the sensitivity of all (or many) outputs with respect to one (or few) entries are desired... Brute-Force Approach. Equation () is solved for two sets of parameters α,..., α j,..., α m and α,..., ᾱ j,..., α m, and the obtained outputs are c(t) and c(t), respectively. One-sided difference approach: If ᾱ j = α j and α j = α j + ɛ the following is an approximation of the local sensitivity at the point α,..., α j,..., α m : c i (t) c i(t) c i (t) α j ɛ If the dependence c(α) is smooth enough, then this is a first order approximation of the sensitivity coefficient. Central difference approach: If α j = α j + ɛ and ᾱ j = α j ɛ the following is an approximation of the local sensitivity at the point α,..., α j,..., α m : c i (t) c i(t) c i (t) α j ɛ If the dependence c(α) is sufficiently smooth, then this is a second order approximation of the sensitivity coefficient. The Brute-Force approach requires only one (for one-sided differences) or two (for central differences) extra function evaluations for each independent variable with respect to which sensitivities are desired. The main drawback is that the accuracy of the method is hard to analyze. The smaller the perturbation ɛ, the lower the truncation error resulting from the omission of higher order terms (see the expansion of finite difference formulas in Taylor series), but the higher the loss-of-significance errors, resulting from subtracting two almost equal numbers. At the very best, the brute force approach results in a sensitivity approximation that has half the significant digits of f... Variational Equations. By differentiating () with respect to the vector of parameters we obtain the variational equations: () d dt c i(t). = P i (c) D i (c) c i D i c i, i =,..., n The fact that the sensitivities satisfy () can be proved rigorously, see for example []. The notation A stands for the sensitivity coefficients vector A α, and. = represents a vector (element-by-element) assignment. To obtain c i (t), one has to numerically integrate the large system obtained by appending together () and (). This method is usually referred as the direct approach.

5 SENSITIVITY ANALYSIS FOR ATMOSPHERIC CHEMISTRY VIA AD The initial values c i (0) must be set properly. There are two main drawbacks of this approach: The generation of the variational equations requires significant extra effort; The integration of the large appended system may be very time consuming... Automatic Differentiation. Automatic differentiation techniques are based on the fact that any function ( regardless of its complexity ) is executed on a computer as a well-determined sequence of elementary operations like additions, multiplications and calls to elementary ( intrinsic ) functions such as sin, cos, etc. By repeatidly applying the chain rule: ( ) ( ) f(s) g(t) t f(g(t)) t=t 0 = s s=g(t 0) t=t0 t to the composition of these elementary operations one can compute, completely automatically, derivatives of f that are correct up to machine precision. According to how the chain rule is used to propagate derivatives through the computation, one can distinguish two approaches to AD: the forward and the reverse modes (see [], [] for a detailed discussion). The Forward Mode. Is similar to the way in which the chain rule of differential calculus is usually taught. At each computational stage derivatives of the intermediate variables with respect to input variables are computed. These derivatives are propagated forward through the computational stages. From now on we will refer to forward automatic differentiation as FAD. The Reverse Mode. Adjoint quantities - the derivatives of the final result with respect to intermediate variables - are computed at each step. To propagate adjoints, one has to be able to reverse the flow of a program, and remember or recompute any intermediate value that nonlinearly impacts the final result.. Applying automatic differentiation on ODE integrators. The numerical simulation of the chemical transformations consists in integrating (advancing in time) a system of coupled, stiff ordinary differential equations that models the chemical rate equations. It is of interest to ask how automatic differentiation can be used to obtain sensitivities of concentrations with respect to different parameters, given the rate equations and a numerical routine that solves them. The simplest approach would be to apply AD directly on the numerical integrator; we will call this the black box approach. A more refined technique, which necessitates the expert (user) intervention, would be to generate the sensitivity (variational) equations via automatic differentiation, then to integrate them using a user-defined technique, for example If x = x c i then c i (0) =, otherwise c i (0) = 0. Reffered throughout the paper as AD.

6 A. SANDU,G. R. CARMICHAEL, AND F. A. POTRA Concentration Solver Concentrations Rate Eqn Numerical Integrator ADIFOR AD Generated Code Sensitivities Rate Eqn Concentrations ADIFOR Sensitivity Eqn Numerical Integrator (DDM) Sensitivities Fig.. Alternative ways of using automatic differentiation for calculating chemical sensitivities. The black box approach (upper figure) is completely automatical, while the variational equation approach (lower figure) requires user intervention. employing the direct decoupled method. Figure gives a visual description of these two possibilities. In short, black box FAD computes the sensitivities of the numerical solution, while the variational approach gives a numerical solution to the sensitivity equations. The black box approach is simpler, while the variational equations approach is more flexible. In the next sections we will investigate both approaches... Black-box approach. Given the initial value problem () c = f(t, c, α), c(t 0 ) = c 0 one would like to numerically compute the solution in time c(t) and also to find the sensitivity of this solution with respect to the parameters α R m. Suppose for the sake of discussion that we solve () numerically with a one-step, first order consistent method, expressed in Henrici notation as: () c n+ = c n + h n Φ(t n, c n, c n+, α, h n ) where c k = c(t k ) = c(t 0 + i h i) and Φ is supposed to be smooth in all its arguments. The method is applied with a predefined selection of the stepsize h i (either using a

7 SENSITIVITY ANALYSIS FOR ATMOSPHERIC CHEMISTRY VIA AD 7 constant step or a prescribed step for example, take smaller steps during sunrise and sundown intervals; it is essential that h does not depend on c, as is the case with inline step controllers, typical for all popular variable step-size codes). The consistency of the method implies that Φ(t, c, c, α, 0) = f(t, c, α) and hence, since Φ is supposed to be smooth in all its arguments we have: (7) lim Φ(t, c n, c n+, α, h) h 0. = lim Φ(t, c n, c n+, α, h) h 0. = Φ(t, c n, c n, α, 0). = f c (t, c n, α) c n + f α (t, c n, α) The sensitivities c = c/ α obey the variational equations: d dt c. = fc (t, c, α) c + f α (t, c, α). = Φ(t, c, c, α, 0) Forward automatic differentiation applied on () will generate: (8) c n+. = cn + h n Φ cn (t n, c n, c n+, α, h n ) c n +h n Φ cn+ (t n, c n, c n+, α, h n ) c n+ +h n Φ α (t n, c n, c n+, α, h n ) so c n+. = cn + h n (Φ cn + Φ cn+ )(t n, c n, c n, α, 0) c n + h n Φ α (t n, c n, c n, α, 0) + O(h n). = c n + h n d dt c n + O(h n) Hence by forward automatic differentiating () we obtain (9), a first order consistent method for solving the variational equation. The above remark leads us to the following conclusion: Black box automatic differentiation implicitly generates the sensitivity equations. More exactly, suppose we start with a code that solves () using a numerical method A. The time discretization A of the initial equation () is transformed by black box FAD into a time discretization of the variational equation (8). This represents a new integration method B. We say that B is the derivative of the method A. If the method B is convergent, then the resulting code will be able to consistently calculate sensitivities. Thus, the black box FAD is equivalent to the direct method, since the resulting code applies a numerical integrator (B) to the variational equations (8). The QSSA method belongs to the class studied here, namely order one, explicit, prescribed step methods. The above analysis can be applied to other methods widely used in atmospheric chemistry, e.g. the hybrid algorithm of Young and Boris (used with fixed step-size).

8 8 A. SANDU,G. R. CARMICHAEL, AND F. A. POTRA... Variable step size algorithms. If the algorithm uses an error controller that adjusts the stepsize based on local error estimates, then h n depends on c n and (9) becomes: c n+ = c n + h n Φ c (t n, c n, α, h n ) c n + h n Φ α (t n, c n, α, h n ) + (h n Φ h (t n, c n, α, h n ) + Φ(t n, c n, α, h n )) h n The last term, and the eventual stepsize rejections may create problems when the algorithm is automatically differentiated. In order to have the last term equal to zero one would need hφ h = Φ (which implies Φ = Ψ/h, contradicting consistency) or h = 0. This last condition is not satisfied in general by the variable step-size algorithms; consider for example the asymptotic step controller: h n = ( c ĉ h n tol h n = ( c ĉ h n tol ) p+ ) p+ +Const h n tol /p+ (c ĉ) ( c ĉ) c ĉ +Const h n c ĉ tol tol (p+)/(p+)... Implicit methods. If an implicit method is used for solving () then at each time step a nonlinear system of algebraic equations has to be solved. This is usually done by Picard or by simplified Newton iterations. Black box FAD will transform the nonlinear system in concentrations into a nonlinear system in concentrations and sensitivities, and will add to the iterative solution for concentrations an iterative solution for sensitivities. For example consider the fixed step-size method (9a) (9b) c n+ = c n + hφ(t, c n k,..., c n+, α, h) c n+. = cn + h i=k i= Φ cn i (t, c n k,..., c n+, α, h) c n i c n + hφ α (t, c n k,..., c n+, α, h) Both formulas (9a), (9b) are implicit since they involve Φ(..., c n+ ) and Φ c (c n+ ), respectively. The code generated by black box FAD will consist of an iterative solution of (9a) and (9a) coupled together. If (9a) is solved first, and the obtained value for c n+ is then used in (9b), this becomes a linear system in c n+ ; hence the sensitivities can be obtained with only one matrix decomposition (of I hφ cn+ for all α i ) and without any iterations (the Direct Decoupled Method from [9]). Thus, we may conclude that black box FAD is equivalent to the direct (coupled) method, with which it shares the disadvantage of extra computational effort.

9 SENSITIVITY ANALYSIS FOR ATMOSPHERIC CHEMISTRY VIA AD 9 The convergence of the sensitivity iterations adds to the problems mentioned above. The aspects of derivative convergence are thoroughly examined in [0]; it is shown that, if the concentration iterations are convergent, under natural hypothesis, the new sensitivity iterations are also convergent.... One step methods. Let A be a one-step method consistent of order p; its local error has an asymptotic expansion of the form: c(t + h) c(t) h Φ(t, c(t), h) = By differentiating the above relation: c(t + h) c(t) h Φ(t, c(t), h) = N+ i=p+ N+ i=p+ d i (t, α) h i + O ( h N+) d i (t, α) h i + O ( h N+) which shows that, if the d i are smooth in α, the differentiated method B has the same order of consistency. Also, if the global error of the method satisfies then c(t 0 + nh) c n = N+ i=p+ c(t 0 + nh) c n Φ(t, c(t), h) = (b i (t, α) + β i ) h i + O ( h N+) N+ i=p+ (b i (t, α) + β i ) h i + O ( h N+) Thus if the initial method is convergent of order p and if the expansion coefficients b i and the perturbations β i are smooth in α, then the differentiated method converges with the same order. The above considerations show that although automatic differentiation produces derivatives that are exact up to the machine precision, the accuracy of the sensitivities computed by the black box FAD depends on the accuracy of the underlying numerical integration method (in particular depends on its order).... Linear Multistep methods. By differentiating the linear multistep method applied on the rate equation () one obtains the same method, applied on sensitivity equations (8): (0a) (0b) k α i c n+i = h i=0 k α i c n+i = h i=0 k β i f n+i i=0 k β i ((f c ) n+i c n+i + (f α ) n+i ) i=0 Hence, the following diagram commutes for multistep methods (provided that sensitivity iterations converge if the multistep method is implicit).

10 0 A. SANDU,G. R. CARMICHAEL, AND F. A. POTRA Kinetic Eqn MS Concentrations ADIFOR ADIFOR Variational Eqn MS Sensitivities.. Black box forward automatic differentiation of the QSSA method. As an illustration, we consider in detail the integration of () using the Quasi- Steady- State- Approximation (QSSA) technique (see []). This method solves () with the first order consistent formula: () c(t 0 + h) = e D0 h (c 0 D 0 P 0 ) + D 0 P 0 where () is applied on the typical integration interval [t 0, t 0 + h], c 0 = c(t 0 ), P 0 = P (c 0 ) and D 0 = D(c 0 ). Because of the diagonal form of D, () can be decomposed componentwise, and the implementation of the QSSA step looks like this: () call compute (in : t 0, α, c 0 ; out : P 0, D 0 ) do i =, n c i (t 0 + h) = e Di 0 h end do ( ) c 0,i P i 0 + P i D0 i 0 D0 i The subroutine compute calculates the production and destruction terms at t, c. From (), the FAD will generate the algorithm: call g compute (in : t 0, α, c 0, c 0 ; out : P 0, D 0, P 0, D 0 ) (a) (b) do i =, n c i (t 0 + h) = e Di 0 h ( c 0,i P i 0 D i 0 ) + P i 0 D i 0 c i (t 0 + h) =. ( h D0 i e Di 0 h c 0,i P 0 i ) D0 i ( +e Di 0 h c 0,i P 0 i D0 i + P 0 i D0 i ) (D0 i ) end do + P i 0 D i 0 P i 0 D i 0 (D i 0 )

11 SENSITIVITY ANALYSIS FOR ATMOSPHERIC CHEMISTRY VIA AD Equation (b) is a numerical method applied to (). From (b) it is clear that: c(h) h=0 = c 0 d dh c i(t 0 + h) h=0 = D0 i D i 0 ( c 0,i P 0 i ) D0 i ( c 0 P i 0 D i 0 ) + P i 0 D i 0 (D i 0 ) = D i 0 c 0,i D i 0 c 0,i + P i 0 and by comparing this with () we conclude that (b) is an order consistent method applied to the variational equations.... The steady-state equations. Many codes use the steady-state approximation for the radical differential equations. More exactly, let c = [l, r] with l denoting the long-lived species and r the radical species and let Π(l, r), (l, r) be the production and destruction terms for radicals: dr(t) dt dr(t) dt = Π(l(t), r(t)) (l(t), r(t)) r Consider the radicals to vary so rapidly, that they are at steady state. 0 and Then () r(t) Π(l(t), r(t)) (l(t), r(t)) This algebraic equation is commonly solved by fixed-point (functional) iterations, keeping l(t) = l 0 = constant: do k =, No of iterations call radical (in : α, β, r k, l 0 ; out : Π k, k ) do i =, No of radicals end do i end do k r k+ i = Πk i k i where the superscript k refers to the iteration number. By forward automatic differentiation, this becomes: do k =, No of iterations call g radical (in : α, β, r k, l 0, β, r k, l 0 ; out : Π k, k, Π k, k ) do i =, No of radicals

12 A. SANDU,G. R. CARMICHAEL, AND F. A. POTRA (a) (b) r k+ i = r k+ i end do i end do k Π k i k i. = Πk i k i Πk i k i ( k i ) It is interesting to look at (b) in relation with (). Since the radicals are supposed to be at steady-state, the sensitivities r are constant in time and hence () becomes: () r i. = Π i i r i i i. = Π i i Π i i, i =,..., n i This reveals that the FAD generated (b) is a fixed point iteration to solve the nonlinear system (). For the conditions under which these iterations converge we refer again to [0].... Conclusions for automatically differentiated QSSA. Black box application of forward automatic differentiation on a fixed step-size QSSA algorithm () is equivalent to solving the variational equations () with the first order method given by the formula (b). If the steady state approximation is used for radicals, and the resulting nonlinear equations are solved by fixed point iterations, then forward automatic differentiation will result in an algorithm that uses the steady state assumption on the variational equation and solves the nonlinear equations for r by a fixed point scheme. We point out again here that the errors of the numerical scheme (b) will dominate those induced by the automatic differentiation process. Hence, it is reasonable to expect that the computed sensitivity coefficients are at most as accurate as the computed concentrations.. Sensitivity calculation via FAD-generated variational equations... General setting. As shown by our previous considerations, automatic differentiation implicitly generates the linearized equations (). To be more precise, let the rate equations be described by the subroutine: subroutine compute (in : t, α, c; out : cdot) where α are the parameters, c is the vector of concentrations and cdot = dc/dt at given input arguments. Forward automatic differentiation will generate: subroutine g compute (in : t, α, c, c; out : cdot, cdot) Under the assumptions that sensitivities obey () and that dc/dt is a smooth function of t and α the subroutine g compute completely describes () because: cdot = dc dt = d dt c.

13 SENSITIVITY ANALYSIS FOR ATMOSPHERIC CHEMISTRY VIA AD It is therefore possible to combine the advantages of FAD with efficient numerical schemes if we firstly generate () via automatic differentiation, then solve the variational system using an integrator of choice. This approach should work: better than the black box automatic differentiation, which is in principle equivalent to the direct approach, with a fixed method for integrating the system; the hybrid approach allows the use of variable step-size algorithms and enables the decoupled integration of sensitivity equations ([9]); easier than the usual implemention of the direct (decoupled) method where the variational equations are derived either by hand or by using symbolic manipulation... Direct decoupled method with dedicated algorithms. Dedicated algorithms are explicit methods for the numerical solution of the rate equations () which make use of their special (production-destruction) form; examples of dedicated methods are QSSA, Young and Boris and Two-Step (see [9]). Looking at the variational equation () we remark that it is also formulated in production - destruction form: d dt c i(t) P i (c). = P i (c) D i c i, i =,..., n, where:. = P i (c) D i (c) c i, i =,..., n, D i (c) = D i (c), i =,..., n. Since dedicated integrators perform the computations in a componentwise manner, their use leads to a natural decoupling between the concentration and the sensitivity equations. Thus, we can extend the direct decoupled method of [9] to dedicated integrators. In the case of using QSSA-type methods, since their main computational load (besides function evaluation) consists of calculating the exponentials exp( D i t) and since D i (c) = D i (c), i =,..., n, the algorithm can be optimized, in the sense that the exponentials have to be evaluated only once per component per time step (and not once per sensitivity coefficient per time step).. Accuracy of different methods: numerical comparisons. In this subsection we display some numerical results obtained by integrating variational equations with Plain QSSA, Extrapolated QSSA and Symmetric QSSA methods (as described in []). We have employed VODE (see []) to accurately solve the system of ordinary differential equations, as well as the corresponding variational system. We consider the exact solution to be that obtained by integrating the variational system with VODE, rtol=0, atol=0 8. The experiments correspond to scenario, the free case.

14 A. SANDU,G. R. CARMICHAEL, AND F. A. POTRA relative error relative error time [hours] time [hours] Fig.. Relative errors in computed concentration C (dots) and in brute force C/ [NO] 0 (solid) for C = O (left) and C = OH (right). This illustrates the fact that the number of significant digits in the brute force sensitivity coefficient is at most half of the number of significant digits in the computed concentration Figure compares the relative errors in the computed concentration and the corresponding brute-force sensitivity coefficient / [NO] 0. The integration was done in double precision with VODE (rtol=0 ). Figure illustrates a point that we made earlier, namely that, for the brute force approach, the number of significant digits in the computed sensitivity coefficient is at most half of the number of significant digits in the value of the function (here, the value of the concentration c). 0 0 relative error relative error time [hours] time [hours] Fig.. Relative errors in [O ]/ [NO] 0 (left) and [NO]/ [NO] 0 (right) with brute - force approach (dots), directly differentiated QSSA algorithm (solid), and QSSA integrated variational equations (dash-dashed). This illustrates the accuracy of different methods. Figure compares the relative errors in the sensitivity coefficients / [NO] 0 computed by directly differentiating the QSSA code (solid lines), by integrating the variational equations with QSSA (step-size 0 seconds, dash-dashed line) and by bruteforce estimating them from the outputs of the QSSA code (step-size 0 seconds). This comparison illustrates the fact that the number of significant digits in the computed

15 SENSITIVITY ANALYSIS FOR ATMOSPHERIC CHEMISTRY VIA AD sensitivity coefficients is given by the precision of the integration scheme (here QSSA) and not by the precision of automatic differentiation process (which is of the same order as machine precision). As expected, the brute force approach is the least accurate, while the integration of variational equations seems to be the right choice. The CPU timings corresponding to these numerical experiments are shown in Table. The calculation of the sensitivities of all species with respect to one parameter implies the calculation of 8 coefficients (since the model has 8 variable species); the calculation of the sensitivities of all species with respect to all initial values implies the calculation of 8*8 coefficients. Method No of sensitivity Normalized coefficients time QSSA concentrations only Direct FAD on QSSA Variational eqns with QSSA Variational eqns with Extrapolated QSSA Variational eqns with Symmetric QSSA Table Timings for different methods 7. Application of FAD to a comprehensive atmospheric chemical mechanism. The chemical mechanism used in this study is that presently used in the STEM-II regional scale transport/ chemistry/ removal model (see [7]). This mechanism consists of 8 chemical species and 78 gas phase reactions. The mechanism, based on the work of Lurmann et al. ([]) and Atkinson et al. ([]) is representative of those presently being used in the study of chemically perturbed environments. The mechanism represents the major features of the photochemical oxidant cycle in the troposphere and can be used to study the chemistry of both highly polluted (e.g., near urban centers) and remote environments. The photochemical oxydant cycle is driven by solar energy and involves nitrogen oxides, reactive hydrocarbons, sulphur oxides and water vapour. The chemistry also involves naturally occuring species as well as those produced by anthropogenic activities. Many of the chemical reaction rate coefficients vary with the intensity of solar radiation (photolysis rates), and thus follow a strong diurnal cycle. Others vary with temperature. To test the robustness of the above numerical algorithms, we have employed six different scenarios 7 (summarised in Table ). These conditions represent various chemical environments ranging from: low NO x oceanic boundary layer regions (Ma- The timings may depend on the size of the code and the amount of memory available. Results presented here were obtained on a HP-UX A 9000/7 workstation with 8 M RAM, using level of optimization. 7 These scenarios follow the IPCC (Intergovernmental Panel on Climate Change) photochemistry intercomparison.

16 A. SANDU,G. R. CARMICHAEL, AND F. A. POTRA Scenario Elements Units Marine Land Bio Free Plume Plume Altitude [Km] Temp [K] Pressure [mbar] Nitrogen [0 9 cm ] H O [ppb] H [ppb] H O [ppb] O [ppb] NO x [ppb] HNO [ppb] CO [ppb] CH [ppb] NMHC [ppb] 0 0 (ISOP) Table Initial Conditions for each of the six IPCC scenarios simulated. O conc [ ppb ] time [ hours ] Fig.. description) The variations of ozone under different IPCC scenarios (see Table for a detailed rine); high NO x continental boundary layer regions without (Land) and with isoprene (Bio); dry upper tropospheric regions (Free); biomass burning plumes without (Plume ) and with (Plume ) reactive hydrocarbon species. Further details are presented in Chapter 7 of the current WMO Ozone Assesment (see [8]). ADIFOR was used to calculate sensitivities of ozone with respect to initial conditions and reaction rate pa-

17 SENSITIVITY ANALYSIS FOR ATMOSPHERIC CHEMISTRY VIA AD 7 HCHO HO conc [ ppb ] 0 0 conc [ ppb ] time [ hours ] time [ hours ] NO NO conc [ ppb ] 0 conc [ ppb ] time [ hours ] time [ hours ] HNO 0 0 HNO 0 0 conc [ ppb ] 0 0 conc [ ppb ] time [ hours ] time [ hours ] OH HO conc [ ppb ] conc [ ppb ] time [ hours ] time [ hours ] Fig.. Variation of the most important species under IPCC scenarios

18 8 A. SANDU,G. R. CARMICHAEL, AND F. A. POTRA Lumped sensitivity. Normalized d [O] / d [C]o Species number LUMPED Species number OZONE Species Sensitivity Species Sensit > 0 Species Sensit < 0 HNO.09E+00 O 8.8E-0 H O -.9E-0 O.98E+00 CH.0E-0 CO -.07E-0 CH.00E+00 HNO.89E-0 P RN.78E-0 RAO.88E+00 NO.9E-0 MRO -.08E-07 R O.0E+00 NO.7E-0 V RO -.8E-07 P RN.E+00 M AO.E-07 RAO -.807E-07 CRO.98E+00 KO.0E-07 M V KO -8.79E-08 M CRG.7E+00 M CO.008E-07 OH -7.00E-08 Fig.. Marine case, lumped sensitivities w.r.t. initial values (left) and sensitivities of ozone w.r.t. initial values (right). rameters. In the simulations of these cases the QSSA method with a fixed stepsize of 0 seconds was used. This algorithm is suited for direct automatic differentiation, and is easy to implement when solving (). Its use leads to results having - significant digits. The calculated ozone concentrations for the five cases are presented in Figure, and all other important species in Figure. In the marine boundary layer case, ozone is continuously destroyed throughout the day period. The land and bio conditions show initially ozone production, followed by a net slight destruction of ozone over the simulation period. In the dry free troposphere (Free) ozone values decrease very slowly. Both plume cases show a large net ozone production. The case (Plume-) without non-methane hydrocarbons (NMHC) shows a much slower net ozone production rate, and a distinct diurnal behavior. The Plume- case (with NMHC) shows a very rapid increase in ozone, followed by a period of slow ozone destruction. The calculated local sensitivities of ozone with respect to the initial conditions of each species for the Marine case are shown in Figure. Plotted are the normalized sensitivities at 0 hours of simulation. Also shown are the 8 largest (+) sensitivities (indicating ozone production) and (-) sensitivities 8. Under these conditions ozone 8 The species number list is presented in the appendix

19 SENSITIVITY ANALYSIS FOR ATMOSPHERIC CHEMISTRY VIA AD 9 concentrations are most sensitive to the initial concentration of ozone ( as expected since this case has a net destruction of ozone). Ozone levels increase with increases in CH, NO x = NO + NO and HNO, species which both lead to the production of ozone and also help to modulate the HO concentrations which is the principal lose mechanism for ozone under these conditions. Note also that HNO is the principal source of NO x in this case since its initial condition is an order of magnitude higher than NO x. The largest negative sensitivity is that with respect to H O, which is the dominant source of HO radicals. Also shown are the lumped sensitivities. Lumped sensitivities can help describe the overall effect of a given perturbation. The lumped sensitivity of the system with respect to parameter α j is defined as: (7) L(α j ) = N ( ) ci (t) (α j ) i= where c i, i =,..., N are the concentrations of the component species. Since L(α j ), j =,..., m are functions of time, and since we are interested in the global effect of α j over the system, we employ the mean values of the lumped sensitivity coefficients over the selected time horizon: (8) L(α j ) = t L(α j ) dt t t t If the mean sensitivities: S j = L(α j ) N are far less then one, the system is considered stable with respect to the initial conditions (see [8]). Numerical experiments with the test problem showed that lumped sensitivities are dominated by ozone sensitivity. This may happen because O concentration is larger than other variable species. To obtain a more accurate description of the global response of the system with respect to perturbations of initial conditions, we employ the lumped normalized sensitivities: ˆL(α j ) = N ( αj c i (t) c ) i(t) (9) (α j ) where c i, i =,..., N are the concentrations of the component species. i= The lumped coefficients are thought of as measures of the global influence of one parameter over the whole system. These quantities are a bit more difficult to interpret. A large value can arise from a few very large individual sensitivities or from many

20 0 A. SANDU,G. R. CARMICHAEL, AND F. A. POTRA species being sensitive to changes in one individual parameter (as is the case for the radical species). One drawback is that, when a parameter has a large influence over a specific or small number of species, the lumped coefficient may be large, although, from a chemical standpoint, the global influence is negligible. As an example, look at Table. The large lumped sensitivity associated with DMS can be explained by a strong influence of DMS initial concentration over itself, although the effect of this parameter on important species is negligible. Results for all the cases are summarized in Tables and. The land, bio and free cases are quite similar. One notable difference is the bio case where isoprene is shown to have a large negative sensitivity. This highly reactive species is an important source of peroxyl radicals. The Plume- and Plume- cases both show large net ozone production and thus behave much differently than the previous cases. In the Plume- case, ozone increases with increases in O, CO, CH, and H O. Under high NO x conditions these species are involved in reactions which increase the peoroxyl radical pool, which in turn leads to net ozone production. Ozone decreases with increases in NO x, because in the absence of NMHC these species scavenge the free radicals and shorten the chain propagation reactions. When sufficient N M HC are present then ozone production is larger, and increases in NO x lead to increases in ozone. The importance of reactive hydrocarbons appears both in the production and destruction of ozone. At the time shown in Table, ozone is in a period of net ozone destruction. Here ozone is being destroyed by reaction with hyroperoxyl radicals. As shown in Figure, ozone production is very rapid during the first day of the simulation. In this period ozone production is driven by N M HC reactions involving alkenes, aldehydes, isoporene and aromatics. After this initial period, those less reactive species which supply peroxyl radicals contribute to ozone destruction (e.g., aldehyde, and aromatics and alkanes). NO x, which both produces ozone and scavenges peroxyl radicals, shows positive sensitivities. The lumped sensitivities are much higher indicating the higher overall reactivity of the system, and the importance of the NMHC s. The amount of ozone produced in an air mass is a complex function of the absolute amounts of NO x, the relative amounts (the value of NMHC/NO x ) and the meteorological conditions (solar actinic flux, temperature, water vapor, etc.). A net ozone production efficiency e N has been defined by Lin et. al. (see []): e N = O NO y where e N represents the net number of ozone molecules produced per molecule of NO y consumed. When the above equation is evaluated using observations or model derived estimates of ozone, it represents a lower limit since ozone is deposited at the earth s surface. This metric can be used to help characterize the modeled ozone production efficiencies. The relationships between model calculated O and NO z are presented in Figure. The quantity NO z is defined as: NO z = NO y NO x

21 SENSITIVITY ANALYSIS FOR ATMOSPHERIC CHEMISTRY VIA AD = P RN + P RN + P RP N + HNO + HONO + P AN + T P AN + R N + RAN + RAN + N O + HNO + NO + MP AN + IP AN + INO + MAN + MV N and is a metric which reduces variations arising from differences in the age of the air masses being compared ( see [7], []). This ozone production efficiency is simply a local sensitivity. This efficiency was calculated using ADIFOR for each of the studied cases. The results are shown in Figure 7. The highest ozone production efficiencies occur for the marine case followed by the land, bio and free conditions. For these cases e N ranges from to 0. The plume with NMHC shows a value of, while the plume without NMHC has a very small ozone production efficiency. These values can be compared to those measured in the eastern United States during the summer. Olsyzyna et al. ([]) report a value of (0 for NO y ) at Harvard Forrest, MA, while Trainer et al. ([7]) report an average value of 8. at several rural sites. ADIFOR was also used to calculate the sensitivity of ozone to variations in the reaction rate constants. The results for the individual and lumped sensitivities are shown in Tables and respectively. For the marine case the largest (+) sensitivities occur for the photolysis reaction of NO and HNO and consumption reactions of HO (i.e., the HO + HO reaction and the NO + HO type reactions). These reactions consume the peroxyl radical which by reaction with O destroys ozone. The largest (-) sensitivities occur for the major ozone destruction reactions (O + hν and O + HO ). Again the land, bio and free cases show similar behavior. The bio case shows a much higher reactivity as measured by the magnitudes of the lumped sensitivities. The plume cases show a different behavior. The case without N M HC (Plume-) shows that ozone is most positively sensitive to the reaction rate constants for the NO photolysis rate, the NO reaction with HO, CO oxidation by OH and ozone photolysis. The largest negative sensitivities occur for the NO + OH and O + NO reactions. The Plume- case shows the importance of NMHC reaction rate constants. For example, ozone has a positive sensitivity with respect to the reaction rate constant for the ethene and alkene oxidation by OH, aldehyde photolysis, and reactions involving P AN (i.e., P AN MCO + NO ). Ozone decreases with increases in reaction rate constants for the MCO +NO reaction, and other reactions similar to the Plume- case. 8. Conclusions. ADIFOR.0 has been succesfully used in the sensitivity analysis of a comprehensive tropospheric chemistry model. Automatic differentiation appears to be a valuable tool for sensitivity analysis of atmospheric chemistry models. In this paper we focused solely on the chemical equations. However the method applies to to the coupled transport/ chemistry problems as well. We are presently applying this technique to the STEM-II model ([7]). A valuable aspect of employing automatic differentiation for sensitivity analysis

22 A. SANDU,G. R. CARMICHAEL, AND F. A. POTRA absolute d ozone / d NOz time [hours] Fig. 7. Absolute sensitivities of ozone with respect to initial NO z concentration. The variation in time is shown for different IPCC scenarios. studies is that the atmospheric chemistry/ transport/ removal models are permanently subject to modifications and improvements. For routinely performing sensitivity analysis this means that, whenever a modification is performed in the model, the corresponding adjustment be made in the variational equations. Since the slightest mistake in the generation of variational equations could lead to useless results, one needs to thoroughly check for their correctness. Both the issues of easily generating the sensitivity equations and making sure they are error free can be directly and succesfully addressed by the use of automatic differentiation.

23 SENSITIVITY ANALYSIS FOR ATMOSPHERIC CHEMISTRY VIA AD O Marine Land Bio Free Plume Plume O 8.8E-0.E-0.0E-0 9.E-0.9E-0.E-0 CH.E-0.E-0 8.E-0 9.7E E E-0 CO -.0E-0.E-0.0E-0 -.0E-0 0.E+00.8E-0 HNO.8E-0.8E-0.00E-0.7E-0 9.0E-0.E-0 NO.E E-0.0E-0.E-0-7.8E-0.E-0 NO.E-0.97E-0.09E-0.E-0 -.7E-0.0E-0 HO -.E-0-7.E E-0-7.E-0 9.0E-0 -.0E-0 ISOP E E-0 ETHE E-0 ALD E-0 ALKE E-0 AROM E-0 ALKA E-0 C H E-0 Table Normalized sensitivities of O with respect to initial values (at the end of the fifth day); displayed are the values of modulus greater than E-0. REFERENCES [] H. Amann. Ordinary Differential Equations: An Introduction to Nonlinear Analysis. Walter de Gruyter, 990. [] R.D. Atkinson, D.L. Baulch, R.A. Cox, R.F.JR. Hampson, J.A. Kerr, and J. Troe. Evaluated kinetic and photochemical data for atmospheric chemistry. International Journal of Chemical Kinetics, : 90, 989. [] Ch. Bischof, A. Carle, G. Corliss, A. Griewank, and P. Hovland. ADIFOR generating derivative codes from FORTRAN programs. Technical report, Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, Illinois, 99. [] Ch. Bischof, A. Carle, P. Khademi, and A. Mauer. The ADIFOR.0 system for the automatic differentiation of FORTRAN77 programs. Technical report, Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, Illinois, 99. [] P.N. Brown, G.D. Byrne, and A.C. Hindmarsh. VODE: A Variable Step ODE Solver. SIAM J. Sci. Stat. Comput., 0:08 0, 989. [] D. G. Cacuci. Sensitivity theory for nonlinear systems. I. Nonlinear functional analysis approach. J. Math. Phys., :79 80, 98. [7] G.R. Carmichael, L.K. Peters, and T. Kitada. A second generation model for regional-scale transport/ chemistry/ deposition. Atmospheric environment, 0:7 88, 98. [8] Y. S. Cho. Ph.d. thesis. The University of Iowa, 98. [9] A. M. Dunker. The decoupled direct method for calculating sensitivity coefficients in chemical kinetics. Journal of Chemical Physics, 8:8, 98. [0] A. Griewank, C. Bischof, G. Corliss, A. Carle, and K. Williamson. Derivative convergence for iterative equation solvers. Optimization methods and software, :, 99. [] E. Hesstvedt, O. Hov, and I. Isaacsen. Quasi-steady-state-approximation in air pollution modelling: comparison of two numerical schemes for oxidant prediction. Int. J. Chem. Kinet., 0:97 99, 978. [] L.O. Jay, A. Sandu, F.A. Potra, and G.R. Carmichael. Improved QSSA methods for atmospheric chemistry integration. SIAM Journal on Scientific Computing, 8:8 0, 997. [] X. Lin, M. Trainer, and S. Liu. On the nonlinearity of tropospheric ozone production. Journal of Geophysical Research, 9:,879,888, 988. [] F.W. Lurmann, A.C. Loyd, and R. Atkinson. A chemical mechanism for use in long-range

24 A. SANDU,G. R. CARMICHAEL, AND F. A. POTRA O Marine Land Bio Free Plume Plume HNO.0E E O.9E+00.77E+00.0E+0.E+00.E+00 - CH.0E+00.E E+00.E+00.9E+00 - CO.9E+00.8E+00.08E+0.7E+00.79E+00.00E+0 RAO.8E+00.70E+00.E+00.E R O.0E+00.9E+00 -.E PRN.E+00.E+00 -.E CRO.E MCRG.E NO -.E+00.E+00.E+00.9E+00.9E+0 NO -.E+00.9E+00.E+00.0E+0.0E+0 C H -.9E E+00 - ISOP E E+00 H O - -.E+00 -.E+00 - DMS E+00.E+00 CHO E+00 - ALKE E+00 ALD E+00 AROM E+00 C H E+00 ALKA E+00 Table Lumped sensitivities with respect to initial values (at the end of the fifth day); displayed are the values of modulus greater than E-0. transport/acid deposition computer modeling. Journal of Geophysical Research, 9:0,90 0,9, 98. [] I.M. Navon and U. Muller. FESW - A finite element FORTRAN IV program for solving the shallow water equations. Advances in engineering software, :77 8, 970. [] K. Olszyna, E. Bailey, R. Simonaites, and J. Meagher. O and NO y relationships at a rural site. Journal of Geophysical Research, 99:,7,, 99. [7] D. Parrish, J. Holloway, M. Trainer, P. Murphy, G. Forbes, and F. Fehsenfeld. Export of north american ozone pollution to the north atlantic ocean. Science, 9: 9, 99. [8] M. Prather. Intercomparison of tropospheric chemistry/ transport models. Scientific assesment of ozone depletion, World meteorological organization, 99. [9] J. Verwer and D. Simpson. Explicit Methods for Stiff Odes from Atmospheric Chemistry. Applied Numerical Mathematics, 8: 0, 99. [0] P. Werbos. Applications of advances in nonlinear sensitivity analysis. System modelling and optimization, Springer-Verlag:7 777, 98.