Numerical Algorithms for ODEs/DAEs (Transient Analysis) Slide 1
Solving Differential Equation Systems d q ( x(t)) + f (x(t)) + b(t) = 0 dt DAEs: many types of solutions useful DC steady state: state no time variations transient: ckt. waveforms changing with time transient periodic steady state: changes periodic w time linear(ized): all sinusoidal waveforms: AC analysis nonlinear steady state: shooting, shooting harmonic balance noise analysis: analysis random/stochastic waveforms sensitivity analysis: analysis effects of changes in circuit parameters Slide 2
Transient Analysis d q ( x(t)) + f (x(t)) + b(t) = 0 dt What inputs b(t) changing with time Why find waveforms of x(t) as they change with time most general analysis typically needed sine wave, pulse, etc. inputs typical in many applications How solve DAE using numerical methods discretize time : replace d/dt term convert DAE to nonlinear algebraic equation at each discrete time point solve this using NR Slide 3
Solving DAEs: Preliminaries Given a DAE: does it have a solution? Depends. Various conditions need to hold. Easier to analyze if DAEs are really ODEs Ordinary Differential Equations: i.e., q ( x) x d d x(t) + f ( x(t)) + b(t) = 0 ) x(t) = g ( x; t) dt dt Existence/Uniqueness conditions well known for ODEs f(x) needs to be Lipschitz device models must be smooth / bounded / physically reasonable watch those if conditions, 1/x, log(x), sqrt(x), etc. terms! DAEs? Much more involved in practice: modifications of ODE methods + heuristics Slide 4
Analytical Exemplar (Model Problem) k1 A B k2 de e C = I(t) dt R d[a] = (k1 + k2 )[A] + k2 dt x_ = x + b(t) Useful because has analytical solution vector linear systems reducible to this form locally approximates nonlinear systems Prototype for stability analysis of ODE solution methods Slide 5
Existence and Uniqueness Does a solution exist? Examples: x_ = x + b(t) yes: x(t) = x(t0 )e (t t0 ) + Z t e (t ) b( ) d 0 1 x_ = ; 2x q Solution is unique, given an initial condition x(t) = 3 13 x_ = x 2 ( x(0) = x0 x20 t : no solution for t > x20 0; 3 (t k) 2 ; 0 <= t <= k, for ANY k > 0! t>k x(t) = Infinite number of solutions! Slide 6
Existence/Uniqueness Theorem Picard-Lindelöf Theorem (roughly) If g (x; t) is defined over all t, x Lipschitz for all x then x_ (t) = g (x; t) ; global solution x(t0 ) = x0 has a unique g (x; t) is Lipschitz if there exists some finite L such that: kg ( x1 ; t) g ( x2 ; t) k < Lk x1 x2 k; 8 x1 ; x2 Slide 7
The Lipschitz Condition Linear systems: Lipschitz p 3 x, 1 : not Lipschitz x x0 Slide 8
Is this globally Lipschitz? e1 (t) e2 (t) ie (t) n1 KCL: ie + d(e1 e2 ) = 0 e2 d n2 KCL: d(e1 e2 ) + + (Ce2 ) = 0 R dt E BCR: e1 E = 0 d e2 d(e(t) e2 ) n2 KCL: e2 + =0 dt RC C Slide 9
Solving ODEs: Overview of Strategy Discretize the time axis (starting from, e.g., t=0) Approximate d/dt term by finite difference t0 = 0; t1 ; t2 ; ; tn d x xi xi 1 ' e.g., dt ti ti 1 ODE becomes algebraic nonlinear equation xi xi 1 + f ( xi ) + b(ti ) = 0 ti ti 1 {z } hi ( xi ) Solve using Newton-Raphson repeat for i=1,2,3,...,n using previous solution (or initial condition) at ti-1 Key issues: how to discretize? errors/accuracy? computation? coding? Slide 10
Piecewise Polynomial Approximation Using locally polynomial bases assume: ODE solution is locally polynomial characterize polynomial with a few numbers e.g., samples at different points find those numbers so that the local polynomial satisfies the ODE x(t) ' x1 + c1 (t t1 ) + c2 (t t1 )2 ; x(t) ' x1 + x2 x1 (t t1 ); t2 t1 t 2 [t1 ; t2 ] t 2 [t1 ; t2 ] Slide 11
Linear Polynomials: FE and BE Use of locally linear approximations for x(t) x2 x1 x(t) ' x1 + (t t1 ); t2 t1 Knowns: t1, t2, x1 (x1 known from, e.g., initial condition) Unknown: x2 x2 x1 Use linear approx. to express x(t) : x(t) ' t2 t1 Enforce ODE x_ (t) = g (x; t) at t : Forward Euler (FE) 1 t 2 [t1 ; t2 ] x2 x1 _ x(t1 ) = g ( x1 ; t1 ) : = g ( x1 ; t1 ) t2 t1 Enforce ODE x_ (t) = g (x; t) at t2: Backward Euler (BE) x2 x1 _ x(t1 ) = g (x2 ; t2 ) : = g ( x2 ; t2 ) t2 t1 Slide 12
The Forward Euler Method xi xi 1 x_ (ti 1 ) = g ( xi 1 ; ti 1 ) : = g ( xi 1 ; ti 1 ) ti ti 1 ) xi = xi 1 + (ti ti 1 )g (xi 1 ; ti 1 ) Explicit integration method x2 available explicitly in terms of knowns nonlinear solve not needed (single eval of g(.,.) suffices) (we'll see later) not numerically stable (we'll see later) does not work for DAEs Time stepping ODE solution process 1) start with initial condition: x0 = x(t0), i=1 2) find xi using FE equation, above 3) increment i; stop if ti>stoptime, else goto 2 Slide 13
The Backward Euler Method xi xi 1 xi xi 1 g ( xi ; ti ) = 0 x_ (ti ) = g ( xi ; ti ) : = g ( x i ; ti ) ) ti ti 1 ti ti 1 Implicit integration method Finding x2 requires nonlinear solution (N-R) (we'll see later) numerically (over)stable (we'll see later) does work for DAEs Time stepping ODE solution process 1) start with initial condition: x0 = x(t0), i=1 2) find xi using BE equation, above solve it using Newton-Raphson 3) increment i; stop if ti>stoptime, else goto 2 Slide 14
The Trapezoidal Method Average FE and BE: + x_ (t1 ) = g ( x1 ; t1 ) x_ (t2 ) = g ( x2 ; t2 ) x_ (t1 ) + x_ (t2 ) g ( x1 ; t1 ) + g (x2 ; t2 ) = 2 2 PWL assumption x2 x1 ) x(t1 ) = x(t2 ) = t2 t1 Trapezoidal Method: Method at ith timestep: x2 x1 g ( x1 ; t1 ) + g ( x2 ; t2 ) = t2 t1 2 Implicit (N-R needed) xi xi 1 g (xi 1 ; ti 1 ) + g ( xi ; ti ) = ti ti 1 2 Perfectly stable numerically (we'll see later) Slide 15
FE, BE, TRAP: Pictorial Summary Difference: where/how ODE is enforced Slide 16
Linear Multi Step (LMS) Methods FE: xi xi 1 = hi g (xi 1 ; ti 1 ) ; BE: xi xi 1 = hi g ( xi ; ti ) hi ti ti 1 : g ( xi ; ti ) + g (xi 1 ; ti 1 ) TRAP: xi xi 1 = hi 2 Generic: 0 xi + 1 xi 1 = 0g (xi ; ti ) + 1g (xi 1 ; ti 1 ) FE: 0 = 1; 1 = 1; 0 = 0; 1 = hi BE: 0 = 1; 1 = 1; 0 = hi ; 1 = 0 hi hi TRAP: 0 = 1; 1 = 1; 0 = ; 1 = 2 2 pth-order Linear Multi Step integration formula p X i xn i = i=0 p X i g ( xn i ; tn i ) i=0 Slide 17
Use of ODE LMS Methods for DAEs d d q ( x(t)) + f (x(t)) + b(t) = 0 ) q ( x(t)) = g( x; t) dt dt The (simple) idea: re-do all derivations with q (t) q ( x(t)) on LHS, instead of x(t). Generic LMS for p=1 (1-step LMS): 0 q (xi ) + 1 q ( xi 1 ) = 0g (xi ; ti ) + 1g ( xi 1 ; ti 1 ) q ( xi ) q (xi 1 ) = hi g ( xi ; ti ) BE: Implicit (N-R needed) g ( xi ; ti ) + g ( xi 1 ; ti 1 ) q ( xi ) q ( xi 1 ) = hi TRAP: 2 q ( xi ) q ( xi 1 ) = hi g ( xi 1 ; ti 1 ) : FE: Implicit? Slide 18
Stability of LMS Methods Test ODE: x(0) = x0 : exact solution: x(t) = x0 e t : x(t) _ = x(t); solution decays if < 0: What do FE/BE/TRAP produce? FE is qualitatitively wrong Slide 19
Why FE Explodes FE (with constant timestep h) xn = (1 + h )xn 1 ) xn = x0 (1 + h )n : if j1 + h j > 1, solution blows up w.r.t n. solution is qualitatively wrong for < 0: Basic requirement for FE h should be small enough s.t. j1 + h j < 1: Example: = 109 ) h < 2 10 9 : FE limited to small timesteps for even qualitatitive accuracy FE said to be (numerically) unstable if j1 + h j > 1 Slide 20
FE: Stability Picture for Complex In general: eigenvalues can be complex stability condition j1 + h j < 1 : circle in h plane Region of stability Slide 21
Stability of BE BE with constant timestep h xn (1 h ) = xn 1 solution will die down if j1 h j > 1: i.e., all of left half plane. but also much of right half plane BE is overstable 1 ) xn = x0 : n (1 h ) OK only within circle Applications more important much better than FE Slide 22
Stability of TRAP TRAP with constant timestep h µ µ µ n h h 2 + h xn 1 = xn 1 1 + ) xn = x0 : 2 2 2 h solution will die down if Re( ) < 0 solution will blow up if Re( ) > 0 Also very accurate i.e, perfect stability (we'll see later) Only concern DAE initial consistency (we'll see later) (easy practical workaround) Slide 23
Accuracy and Truncation Error Even if stable: is the method accurate? Test problem exact solution: x(nh) = x0 e nh : How do numerical method solutions compare? FE: x(nh) ' x0 (1 + h )n : 1 : n (1 h ) µ n 2 + h TRAP: x(nh) ' x0 2 h : BE: x(nh) ' x0 None are identical, but how different? Slide 24
Taylor Expansion Error Exact: x(nh) = x0 e nh µ 2 2 2 n h = x0 1 + nh + + 2! µ n(n 1) 2 2 FE: x(nh) ' x h + 0 1 + nh + 2 first-order accurate µ n(n + 1) 2 2 BE: x(nh) ' x h + 0 1 + nh + 2 also first-order accurate µ 2 n 2 2 TRAP: x(nh) ' x h + 0 1 + nh + 2 second-order accurate Slide 25
Transient: Timestep Control Choosing the next timestep dynamically LTE based control apply LTE formulae to estimate error error specification: like reltol-abstol (reltol=percentage) make sure these are looser than NR tolerances! element-by-element vs vector norm based change timestep to meet some specified error 2-norm vs max norm; DAE issues (change integration method based on timestep) NR convergence based control cut timestep if NR does not converge also: increase maxiter increase timestep if NR converges too easily also: decrease maxiter Slide 26
ODE/DAE Packages Out There MATLAB has various ODE/DAE integrators DASSL/DASPK: general purpose DAE packages ode23, ode45,...; ode23t, ode15s,... Linda Petzold, UCSB Fortran some tweaking helpful for circuit applications Easy (and often worthwhile) to roll your own tweaking, special heuristics, debugging,... Slide 27
Transient: Other Important Issues What integration method to use? stability? nonlinear solution required? (implicit vs explicit) accuracy loss due to discretization? Vast body of work on ODE integration linear multi-step methods (LMS), Runge-Kutta, symmetric, symplectic, energy-conserving, etc. Stiff differential equations different variables have very different time constants stiffly stable methods allow you to take larger time steps DAE issues Local Truncation Error (LTE) higher order methods (more than 1 previous timepoint) initial condition consistency; stability; index; Dynamic timestep control, NR heuristics,... Slide 28
Solving the Circuit's Equations d q ( x(t)) + f (x(t)) + b(t) = 0 dt Ckt. DAEs: many types of solutions useful DC steady state: state no time variations transient: transient ckt. waveforms changing with time periodic steady state: changes periodic w time linear(ized): all sinusoidal waveforms: AC analysis nonlinear steady state: shooting, shooting harmonic balance noise analysis: analysis random/stochastic waveforms sensitivity analysis: analysis effects of changes in circuit parameters Slide 29
The Periodic Steady State Problem d q ( x(t)) + f (x(t)) + b(t) = 0 dt What inputs b(t) are periodic e.g., sinusoidal suppose outputs x(t) also become periodic (happens for stable circuits and systems... eventually can take a long time) want to find this periodic steady state directly without using general/expensive transient analysis Why audio amps, RF amps, mixers, oscillators, clocks,... linear circuits: frequency-domain analysis important ckt nonlinear => sinusoids will in general be distorted much easier and more insightful than transient How (for linear circuits): AC analysis Slide 30