Differential-algebraic equations, the dummy derivatives method, and AD

Transcription

1 Differential-algebraic equations, the dummy derivatives method, and AD John D Pryce Nedialko S Nedialkov 7th International Conference on Algorithmic Differentiation Oxford, UK, September 2016 September 14, /21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

2 1 DAEs & why they aren t ODEs intro 2 Structural analysis of DAEs geeky 3 The dummy derivatives (DDs) method for DAEs geekier 4 How AD helps apply the DDs method geekiest 5 Broader view: the MANDAE project strategic 2/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

3 DAEs and how to represent them A DAE for present purposes is a set of n equations relating n variables x j = x j (t) and some of their time-derivatives. I assume we know DAEs are important (Modelica, etc). As with ODEs, one can always reduce to equivalent first order DAE F(t, x, ẋ) = 0 the form accepted by many codes for the DAE initial value problem, such as DASSL, SUNDIALS,... But we prefer a form with arbitrary higher derivatives: f i ( t, the x j and derivatives of them ) = 0, i = 1,..., n. This lets one formulate many problems more concisely, hence solve by the Nedialkov-Pryce IVP code DAETS more conveniently. 3/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

4 Why DAEs aren t ODEs This system is a DAE: x 1 u(t) = 0, x 1 ẋ 2 = 0 where u(t) is a given driving function. Solving it requires integrating the driving function: x 2 = u(t)dt + C where C is a constant. It is an ODE in disguise. But also a DAE is: x 1 u(t) = 0, ẋ 1 x 2 = 0, whose unique solution has x 2 = u(t). Solving requires differentiating the driving function. It is not like an ODE at all. 4/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

5 Initial values are not obvious for DAEs Simple pendulum: state variables x(t), y(t), λ(t); constants G, L. 0 = A = ẍ + xλ 0 = B = ÿ + yλ G 0 = C = x 2 + y 2 L 2, Initial (x, y) must satisfy overt constraint C = 0 of lying on circle. Initial (ẋ, ẏ) must satisfy hidden constraint C = 0, that is 2(xẋ + yẏ) = 0: motion must be tangential to circle. L (x',y') One can describe configuration at time t by a point in 4D (x, y, ẋ, ẏ)-space that must obey the 2 equations C = 0, C = 0 defining the manifold M of consistent points at a given t. M s dimension, here 4 2 = 2, equals dof, the number of degrees of freedom how many independent IVs one can prescribe. 5/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016 (x,y)

6 Index Integer index (various definitions exist) attempts to measure how far DAE is from being an ODE. Probably most used is differentiation index ν d = the number of times it is necessary to differentiate one or more of the f i = 0 (w.r.t. t) such that from the result one can derive (*) an ODE in all the components. For the examples above x 1 u(t) = 0, x 1 ẋ 2 = 0 has ν d = 1; ODE could be ẋ 1 = u(t), ẋ 2 = u(t). x 1 u(t) = 0, ẋ 1 x 2 = 0 has ν d = 2; ODE could be ẋ 1 = u(t), ẋ 2 = ü(t). Defect: (*) makes you differentiate too much! To solve these, one only need differentiate 0 and 1 times respectively, not 1 and 2. 6/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

7 Index reduction, structural analysis Many numerical methods for higher-index DAEs start with index reduction: augment DAE by derivatives of some of its equations to produce a DAE of smaller index, but larger size. Differentiation index ν d is of limited use for finding how to do this. To compute ν d from its formal definition entails heavy linear algebra. So index reduction is usually done by discrete structural analysis (SA) algorithms based on DAE s sparsity structure. The classical SA algorithm is in the 1988 Pantelides paper The consistent initialisation of differential-algebraic systems. 7/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

8 Offsets Essential idea: find a number c i of times to differentiate equation f i = 0, that gives structurally nonsingular (SNS) system for resulting highest, d j th, derivatives of the x j. That is, one can renumber equations (or variables) so that x (d i ) i occurs in f (c i ) i for i = 1 : n. Equivalently the n n signature matrix Σ = (σ ij ) can be permuted to put finite values on the diagonal, where { highest order of derivative of xj in f i ; or σ ij = if x j does not occur in f i. Such offset vectors c = (c 1,..., c n ), d = (d 1,..., d n ) always exist unless DAE is structurally ill-posed, usually due to modeling error. Unique elementwise smallest non-negative c, d exist, the canonical offsets, which we assume chosen henceforth. 8/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

9 Jacobian An SA-friendly DAE is one whose n n system Jacobian ( ) J = f (c i ) i / x (d j ) j i,j=1 : n is nonsingular at some point on consistent manifold M (above). At such a point, if f i suitably smooth, solutions exist locally through this consistent point and any nearby consistent points. Pantelides method succeeds in consistent initialisation, e.g. for pendulum says initialise (x, y, ẋ, ẏ) DAE is SA-friendly dummy derivative method can be used Daets can solve the DAE. } In practice most DAEs are SA-friendly, don t ask me why. some aren t (Tischendorf, Tan), 9/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

10 Dummy derivatives, DDs Some index reduction methods convert DAE to ODE with more degrees of freedom than the DAE. Then the solution paths of the DAE are a proper subset of those of the ODE. Problem: numerical drift from manifold M, often exponential. DDs by Mattsson & Söderlind 1993 (M&S) are a systematic way to form equivalent ODE with exactly as many DOF as the DAE. View DAE as a flow on M. Then DDs describes this flow in a local coordinate system on M formed of some x j and derivatives. Numerical drift can only be within M, where it is less harmful. If solution path leaves patch of M where coordinates are nonsingular, one must choose new coordinates. This need for DD pivoting or switching complicates a numerical algorithm. 10/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

11 My take on DDs This description is equivalent to M&S s, but supports an implementation that makes DD switching almost trivial. (As M&S) Form derivatives of each f i up to the c i th, giving system f (l) i = 0, l = 0 : c i, i = 1 : n. Its unknowns are derivatives of the x j up to the d j th. View these as unrelated algebraic items x jl : x jl renames x (l) j, l = 0 : d j, j = 1 : n. It is under-determined in general: (no. of items) (no. of equations) = dof where dof is the number of degrees of freedom. 11/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

12 A way to formulate DDs, cont. DDs squares up the system by adding dof equations ẋ jl = x j,l+1, (j, l) in suitable set S with dof elements stating a genuine differential (not algebraic) relation for the selected x jl, which are state items, grouped into the current state vector x S. x S comprises the current local coordinate system of M. Result is an implicit ODE, solvable to give ODE of size dof, locally equivalent to the original DAE: ẋ S = F(t, x S ), I omit how state items are chosen linear algebra on columns of J. Key idea: any spanning set a basis (M&S) or dually: any lin indep set a basis (JDP). 12/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

13 Example: pendulum For pendulum the process produces 5 equations in 7 unknowns. The notation, here, is to rename x as x 0, ẋ as x 1, etc. etc. Augmented system 0 = A = ẍ + xλ 0 = B = ÿ + yλ G 0 = C = x 2 + y 2 L 2 0 = C = 2(xẋ + yẏ) 0 = C = 2(xẍ+ẋ 2 +yẏ+ẏ 2 ) After renaming 0 = A 0 = x 2 + x 0 λ 0 0 = B 0 = y 2 + y 0 λ 0 G 0 = C 0 = x y 2 0 L2 0 = C 1 = 2(x 0 x 1 + y 0 y 1 ) 0 = C 2 = 2(x 0 x 2 +x 2 1 +y 0y 1 +y 2 1 ) Here are the 7 unknowns showing possible state vectors. You can t avoid involving all 7 in solution process; this is general. Producible by DDs; sensible: 1st one says d dt [ ẋ x ] = F([ ẋ x ]) x ẋ ẍ y ẏ ÿ λ (compatible with AD) x ẋ ẍ y ẏ ÿ λ Not producible by DDs; silly: x ẋ ẍ y ẏ ÿ λ x ẋ ẍ y ẏ ÿ λ (poorly compatible with AD) 13/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

14 Strategic Q: how should AD tools help to automate numerical solution of SA-friendly DAEs? First, tool should support d/dt as a first-class operator also highly desirable for representing Modelica. At present widely used tools such as ADOL-C and dcc/dco do not have this ability. Daets uses Ole Stauning s C++ AD package Fadbad++. Stauning included Diff(, q) meaning d q /dt q, at our request. 14/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

15 Pendulum code E.g. code for pendulum, as in Daets user guide: 1 template <typename T> 2 void f c n (T t, const T z, T f, void param ) { 3 // z[0], z[1], z[2] are x, y, λ. 4 const double G = , L = ; 5 f [ 0 ] = D i f f ( z [ 0 ], 2 ) + z [ 0 ] z [ 2 ] ; //A = ẍ + xλ 6 f [ 1 ] = D i f f ( z [ 1 ], 2 ) + z [ 1 ] z [2] G ; //B = ÿ + yλ G 7 f [ 2 ] = s q r ( z [ 0 ] ) + s q r ( z [1]) s q r ( L ) ; //C = x 2 + y 2 L 2 8 } Daets instantiates T with 2 classes for structural analysis and 2 for numerical solution. 15/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

16 AD helping DAE solution (2) Second, tool must be able to differentiate f i selectively. E.g. for pendulum, leave A, B alone, differentiate C twice. Requires tool based on source code transformation? Actually no, as Ned observed. Don t treat different derivatives of one variable in isolation, but together as a truncated power series. Many tools do this naturally. E.g. pendulum really uses 3 objects x = (x 0, x 1, x 2 ) order 2 power series, y = (y 0, y 1, y 2 ) order 2 power series, λ = (λ 0 ) order 0 power series. Then normal Taylor series AD gives the necessary derivatives. 16/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

17 How this AD works (1): evaluate residuals Example: doing C = x 2 + y 2 L 2 proceeds via these steps: Input x = (x 0, x 1, x 2 ), also Lsq constant = (L 2, 0, 0) y = (y 0, y 1, y 2 ) Compute v 1 = x 2 = (x0 2, 2x 0x 1, 2x 0 x 2 + x1 2) v 2 = y 2 = (y0 2, 2y 0y 1, 2y 0 y 2 + y1 2) v 3 = v 1 + v 2 = (x0 2 + y 0 2, 2(x 0x 1 + y 0 y 1 ), 2(x 0 x 2 + y 0 y 2 ) + x1 2 + y 1 2) C = v 3 Lsq = (x0 2 + y 0 2 L2, 2(x 0 x 1 + y 0 y 1 ), 2(x 0 x 2 + y 0 y 2 ) + x1 2 + y 1 2) Returns order 2 power series C holding (C, C, C) = (C 0, C 1, C 2 ). Similarly A, B are returned as order 0 series A = (A 0 ) = (2x 2 + x 0 λ 0 ), B = (B 0 ) = (2y 2 + y 0 λ 0 ). (Second-order terms disagree by factor 2 with earlier since I ve gone from derivatives to Taylor coefficients.) 17/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

18 How this AD works (2): root-find In the context of reducing the DAE to an explicit ODE, suppose DDs has chosen state items x, ẋ, renamed as x 0, x 1. We implement the function F such that ẋ S = F(t, x S ), thus: 1. Input: values of t and x S = (x 0, x 1 ) items x F = (x 2, y 0, y 1, y 2, λ 0 ) are trial values that produce 5 residual values r = r(x F ) = (A 0, B 0, C 0, C 1, C 2 ). 3. Root-find using suitable Jacobians (more AD!) to get x F that makes r = 0; the F means found. 4. This finds x F as a function of t and x S (Implicit Function Theorem). 5. Extract x 2 from x F. Output: (x 1, x 2 ), which = (ẋ 0, ẋ 1 ) = ẋ S = F(t, x S ). A general way to prepare (nonstiff) DAE to solve by e.g. RK method. To DD-switch, change the index set S that defines x S, plus associated index sets and Jacobians. Ned confirms this is a viable way to implement DDs, using methods already provided by the Daets classes. 18/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

19 Broader view: the MANDAE project I m part of MANDAE bid to run EU Innovative Training Network (ITN). 14 ESRs (PhD students) at 7 partner sites across Europe, + McMaster. Technical mission: unify and extend tools for solving DAEs. Agreed: Use an Intermediate Representation IR an abstract object, mapped via an API to a program data structure that one can manipulate. IR = (IR-dae) + (IR-sa) = (DAE as syntax tree or similar) + (results of structural analysis). This can support both overloading and source-transformation tools. Needn t store differentiated equations explicitly as shown above. The IR has a meta aspect that complicates things: Daets uses a C++ form of the DAE, so it uses the C++ preprocessor, class structure, compiler & run-time system to get its effects. Our other code Daesa does similar in Matlab. The IR is passive, it cannot use these things the tools we build must deploy them on the IR s behalf. 19/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

20 Implications: Ned s basic architecture diagram Hence, to maximise use of existing code we propose this structure: Modelica function 1. M-parse IR-dae 2. CPPparse 3. unparse C++ function IR-dae + IR-sa IR-sa 5. generate F 6. generate f F(t,x,x') = 0 x' = f(t,x) 4. do SA 8. build F 7. build f code data structure information flow process DAETS "generate": implement by source-code generation "build": implement by operator overloading 20/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016

21 Implications for MANDAE project Start project by specifying IR and its operations abstractly in principle but as a C++ class (IR-dae is recursive). Implement a C++ API of functions to create, write, read, manipulate IR objects. Done by key work-package WP1. Its ESR (= PhD student) will visit all project sites to gather requirements e.g. ensure API talks to AD tools smoothly. When diagram step 3 unparse done, existing Daets tools do step 4 do SA, achieving IR-dae IR-sa. Meantime other WPs work on step 1, Modelica IR-dae, and steps 4 8 which do IR simulation-code with various flavours of AD, linear algebra, exploiting DAE structure, etc. In particular a basic DDs implementation drops out, from Ned s previous work with Daets. WP1 is crucial, but when it s done coupling is low between other parts of project, reducing overall project risk. 21/21 Introduction Structural analysis Dummy derivatives AD and DDs Broader view: MANDAE AD Sep 2016