Finite Difference and Finite Element Methods Georgy Gimel farb COMPSCI 369 Computational Science 1 / 39
1 Finite Differences Difference Equations 3 Finite Difference Methods: Euler FDMs 4 Finite Element Methods (FEM) [optional] Learning outcomes: Be familiar with the finite difference models and methods (Euler FDMs) Optional: Runge-Kutta FDMs, more accurate FEMs Recommended reading: M. T. Heath, Scientific Computing: An Introductory Survey. McGraw-Hill, 00: Chapters 5, 8 11 M. Shäfer, Computational Engineering - Introduction to Numerical Methods. Springer, 006: Chapters 3, 5 G. Strang, Computational Science and Engineering. Wellesley-Cambridge Press, 007: Chapters 1., 3, 6 / 39
Let s Recall Differential Equations Modern science and engineering assume our world is continuous and described by differential equations and integral equations Example I: 1 st -order differential equation du(x) = ; u(0) = 0 General solution: u(x) = x + A Boundary condition: u(0) = 0 gives A = 0 Final solution: u(x) = x Unknown u(x) is specified by a known instant speed of changes: du(x) = lim u(x + x) u(x) x 0 x 3 / 39
Let s Recall Differential Equations, continued Example II: nd -order DE d u(x) = ; u(0) = u(1) = 0 General solution: u(x) = x + Ax + B Boundary conditions: u(0) = 0; u(1) = 0 give B = 0; A = 1 Final solution: u(x) = x x Unknown u(x) is specified by a known instant acceleration of changes: ( ) d u(x) 1 = lim du(x+ x) x 0 x du(x) = lim x 0 ( 1 u(x+ x) u(x+ x) x x u(x+ x) u(x+ x)+u(x) = lim x 0 x ) u(x+ x) u(x) x 4 / 39
Physical Oscillating Second-order Systems Non-damped mass on a spring: m d x(t) = kx dt x(t) a displacement k a spring constant (the Hooke s law) m a mass Displacement x Mass m x Damped mass on a spring (γ the friction constant): m d x(t) dt = γ (t) dt + kx Pendulum: mr d θ dt d θ = g dt R sin θ = mgr sin θ θ R m mg 5 / 39
Differences Vs. Derivatives Finite differences: natural approximations to derivatives du(x) = lim u(x + x) u(x) x 0 x u(x) x u(x + x) u(x) x u u x x Discrete step x can be small but it does not tend to zero! x 6 / 39
Differences Vs. Derivatives Three possibilities for the finite difference: Forward difference u(x+h) u(x) h Test for u(x) = x (x+h) x h = x + h Backward difference u(x) u(x h) h x (x h) h = x h Centred difference u(x+h) u(x h) h (x+h) (x h) h = x Results of testing: The centred difference gives the exact derivative du(x) The forward and the backward differences differ by h = x 7 / 39
Taylor Series Approximation Let u(x) be an arbitrary infinitely differentiable function In plain words, let all its derivatives exist: d n u(x) n ; 1 n Given u(x 0 ) and dn u(x) x=x0 ; n = 1,,..., at x = x n 0, the value u(x 0 + h) at x = x 0 + h is represented by the Taylor s series of the values at x = x 0 : u(x 0 + h) = u(x 0 ) + n=1 u(x 0 ) + n=1 h n n! d n (u(x) n }{{ x=x0 } u [n] (x 0 ) h n n! u[n] (x 0 ) 8 / 39
A Few Famous Taylor Series Exponential function u(x) = e x e 0 = 1; dex x=0 = e x x=0 = 1: u(0 + h) e h = 1 + h + h + h3 6 + h4 4 +... Polynomial ratio u(x) = 1 1+x u(0) = 1; du(x) = x x=0 (1+x ) = 0; ( x=0 d u(x) x=0 = (1+x ) + 8x x=0 (1+x ) = : 3 u(0+h) 1 1 + h = 1 h +h 4... u(1) = 1 = 1 1 + 1 1 +... Trivial for any polynomial u(x) = x 3 u(1) = 1; du(x) = 3x ; d u(x) = 6x; d3 u(x) 3 = 6: u(1 + h) = 1 + 3h + 6 h + 6h3 6 = 1 + 3h + 3h + h 3 9 / 39
Accuracy of Finite Differences Taylor series approximations: u(x + h) = u(x) + hu (x) + 1 h u (x) + 1 6 h3 u [3] (x) +... u(x h) = u(x) hu (x) + 1 h u (x) 1 6 h3 u [3] (x) +... The first-order accuracy O(h) of the one-sided differences: u(x+h) u(x) h = u (x) + 1 hu (x) +... u(x) u(x h) h = u (x) + 1 hu (x) +... The second-order accuracy O(h ) of the centred differences: u(x + h) u(x h) h = u (x) + 1 6 h u [3] (x) +... 10 / 39
Second Difference The second difference approximation of the second derivative: d u u u(x + x) u(x) + u(x x) = x x The second order accuracy O(h ) due to the centred u: u(x + x) u(x) + u(x x) x = u (x)+ 1 1 h u [4] (x)+... Exact second derivatives (x+h) x +(x h) h = for u(x) = x and (x+h)3 x 3 +(x h) 3 h = 6x for u(x) = x 3 O(h ) accuracy (x+h)4 x 4 +(x h) 4 h = 1x + h for u(x) = x 4 with the true second derivative u (x) = 1x 11 / 39
Finite Difference Equations The nd -order differential equation d u(x) Known source function f(x) = f(x) Known boundary conditions, e.g. u(0) = 0 and u(1) = 0 Its discrete form the nd -order finite difference equation: Divide the interval [0, 1] into equal pieces of length h = 1 n+1 that meet at the points x = h, x = h,..., x = nh: Approximate the goal values u(h),..., u(nh) at n discrete points [h, h,..., nh] inside the interval [0, 1] with the values u 1,..., u n, respectively, using the finite difference u x : u 0 u 1 + u = f 1 f(h); u 1 u + u 3 = f f(h);... u n 1 u n + u n+1 = f n f(nh) 1 / 39
Finite Difference Equations: Matrix-Vector Form For the n unknowns u 1, u,..., u n and the boundary conditions u 0 = u n+1 = 0, the differential operator d is replaced with the n (n + ) matrix: 1 1 u 0 = 0 f 1 1 1 u 1 f......... =. 1 h 1 1 1 1. u n u n+1 = 0 f n 1 f n where the source values are specified as f i = f(ih) For the special case d u = 1 with the above boundary conditions, the finite differences give an exact match between u(ih) and u i 13 / 39
Example with the Constant Source f(x) = 1 Differential equation: d u = 1 with u(0) = u(1) = 0 Complete solution u complete = u part + u null : a particular one for u = 1 plus the nullspace one for u = 0 Particular solution: d u Nullspace solution: d u u(x) = x = 1 is solved by u part = x = 0 is solved by u null = Cx + D + Cx + D From the boundary conditions: u(0) = D = 0 u(1) = 1 + C + D = 0 C = 1 u(x) = x x Difference equation: u i 1+u i u i+1 h = 1 with u 0 = u n+1 = 0 (!) Just the same parabolic solution: u i = ih i h However such a perfect agreement between the discrete u i and the exact continuous u(ih) is very unusual 14 / 39
Another Example: Free End Boundary Condition (optional) Free end condition: zero slope u (0) = 0 at x = 0 and u(1) = 0 Differential equation: d u = 1 with du (0) = u(1) = 0 The complete solution: u(x) = x 1 x + Cx + D = Boundary conditions: u (0) = C = 0; u(1) = 0.5 + C + D = 0 D = 0.5 Difference equation: u i 1+u i u i+1 = 1 with u 1 u 0 h h = 0 and u n+1 = 0 1 1 u 1 1 1 1 u 1.......... = 1 h 1 1 } {{ 1 } T n u n 1 u n. 1 1 15 / 39
Free End Boundary Condition, continued u 1 u. u n 1 u n = h n n 1 n... 1 n 1 n 1 n... 1 n n n... 1...... 1 } 1 1 {{ 1... 1 } T 1 n 1 1. 1 1 Discrete solution: u i = h (n+i)(n+1 i) Error w.r.t. the exact solution: u(ih) u i u ( i n+1 ) u i = 1 = 1 n+1 i = 1 (n+1) ( ) 1 i 1 (n+i)(n+1 i) (n+1) (n+1) ( 1 n+1 i (n+1) ) O(h) 16 / 39
Free End Boundary Condition, continued Therefore, the one-sided boundary conditions change the matrix and may result in O(h) error A more accurate difference equation can be constructed by centring the boundary conditions Centred difference equation will have nd -order errors O(h ) Previous and similar differential and difference equations can be met in many practical problems Simple physical examples of oscillators (see Slide 5): a non-damped mass on a spring a damped mass on a spring a pendulum 17 / 39
Stiff Differential Equations [ d v Example: dt w A ] [{}}{ 50 49 = 49 50 ] [ v w ] [ v(0) ; w(0) Solution: v(t) = e t + e 99t and w(t) = e t e 99t ] = [ 0 Time scales differ by a factor of 99 (the condition number of A) Solution decays at the slow time scale of e t, but computing e 99t may require a very small step t for stability Thus, t is controlled by the fast decaying component, and this is really counterproductive! Stiffness comes with any problem involving very different time scales (chemical kinetics, control theory, circuit simulations, etc.) ] 18 / 39
Forward and Backward Euler Methods (optional) Differential equation du(t) dt = f(u, t); an initial value u(0) The rate of change u is determined by the current state u at any moment t Forward Euler: u n+1 u n t Backward Euler: = f(u n, t n ) u n+1 = u n + t f n u n+1 u n t = f(u n+1, t n+1 ) u n+1 t f n+1 = u n Implicit method: if f is linear in u, a linear system is solved at each step 19 / 39
Forward and Backward Euler Methods, continued Example: Linear differential equation u = au (i.e. du(t) dt = au); u(0) = u 0 > 0, with the exact solution: u(t) = e at u(0) u 0 e at Generally, a is a complex number: a = (Re a) + (Im a) 1 Forward Euler: one step u n+1 = (1 + a t)u n u n = (1 + a t) n u 0 Convergence: (1 + a t) T/ t e at as t 0 Sharp instability border 1 + a t = 1, i.e. for a = / t Backward Euler: one step (1 a t)u n+1 = u n u n = (1 a t) n u 0 Convergence: (1 a t) 1 = 1 + a t+ higher order terms, so that (1 a t) T/ t e at as t 0 Stability whenever u = au is stable (i.e. for a having the negative real part: Re a < 0) 0 / 39
Forward and Backward Euler Methods, continued Since forward and backward differences are first order accurate, the errors from both methods are O( t) Second-order methods: Formal notation: f n = f(u n, t n ) and f n+1 = f(u n+1, t n+1 ) Crank-Nicolson: u n+1 u n t = f n+1 + f n u n+1 = 1 + 1 a t 1 1 a tu n Stable even for stiff equations, when a 0 1 / 39
Forward and Backward Euler Methods, continued Second-order methods: Explicit forward Euler: u n+1 u n t = 3f n f n 1 Stable if t is small enough: a t 1 if a is real But explicit systems always impose a limit on t Implicit backward difference: 3u n+1 4u n + u n 1 t = f n+1 More stable and accurate than the Crank-Nicolson method (trapezoidal rule) / 39
Explicit and Implicit Multistep Methods (optional) With p earlier values of u n, the accuracy is increased to order p u = u(t) u(t t) u = u(t) u(t t) + u(t t) 3 u = u(t) 3u(t t) + 3u(t t) u(t 3 t)... p u = u(t) pt(y t) +... + ( 1) p u(t p t) Backward differences: ( + 1 +... + 1 p p ) u n+1 = t f(u n+1, t n+1 ) Backward Euler method: p = 1 Implicit backward difference method: p = 3 / 39
Explicit and Implicit Multistep Methods, continued An alternative: using older values of f(u n, t n ) instead of u n Explicit forward Euler methods: u n+1 u n = t (b 1 f n +... + b p f n p+1 ) order of b 1 b b 3 b 4 limit on a t accuracy for stability p = 1 1 p = 3/ -1/ 1 p = 3 3/1-16/1 5/1 6/11 p = 4 55/4-59/4 37/4-9/4 3/10 The like implicit methods are even more stable and accurate 4 / 39
Runge-Kutta Methods (optional) Highly competitive and self-starting if evaluations of f(u, t) are not too expensive Compound 1-step methods, using Euler s u n + tf n inside the function f Simplified RKM: u n+1 u n t = 1 f n + f(u n + tf n, t n+1 ) }{{} compounding of f 5 / 39
Simplified Runge-Kutta Method (optional) nd -order accuracy An example for u = au: u n+1 = u n + 1 t[au n + a(u n + tau n )] = (1 + a t + 1 ) a t u n }{{} Growth factor G This growth factor G reproduces the exact e a t through the third term 1 a ( t) of the Taylor s series 6 / 39
Higher-order Runge-Kutta Method (optional) Fourth-order RKM: u n+1 u n t = 1 3 (k 1 + k + k 3 + k 4 ) where k 1 = 1 f(u n, t n ) k 3 = 1 f ( u n + tk, t n+1/ ) k = 1 f ( u n + tk 1, t n+1/ ) k 4 = 1 f (u n + tk 3, t n+1 ) This 1-step method needs no special starting instructions It is simple to change t during computations The growth factor reproduces e a t through the fifth term 1 4 a4 ( t) 4 of the Taylor s series Among highly accurate methods, RKM is especially easy to code and run probably the easiest there is! The stability threshold is a t <.78 7 / 39
Finite Element Methods (FEM) [optional] Finite difference methods (FDM) approximate the differential equation (+) Very easy implementation ( ) Low accuracy between grid points Finite element methods (FEM) approximate directly the solution of the differential equation (+) Easy handling of complex geometry and boundaries of a problem domain (±) Usually, more accurate than FDM but this depends on a problem Unknown solution u(x) as a combination of n basis functions φ(x): u(x) = u 1 φ 1 (x) +... + u n φ n (x) FDM are special cases of the FEM with specific basis functions 8 / 39
Finite Element Methods: Weak Form Weak (or variational) form of the differential equation u (x) = f(x) in [0, 1] with u(0) = u(1) = 0: For any smooth function ν(x) satisfying the boundary conditions ν(0) = ν(1) = 0: 1 0 f(x)ν(x) = 1 0 u (x)ν(x) = 1 0 u (x)ν (x) Recall the calculus: Integration by parts b a y(s)z (s)ds = y(s)z(s) b a a b y (s)z(s)ds 9 / 39
Galerkin s FEM: u = f in the discrete weak form Discretisation of ν(x) with n test functions ν 1 (x),..., ν n (x) The weak form gives one equation for each ν j (x) involving numerical coefficients u 1,..., u n : 1 ( n ) 1 dν j (x) = f(x)ν j (x); j = 1,..., n 0 dφ u i (x) i i=1 Ku = f, that is, K ij = n K ij u i = f j ; j = 1,..., n, where i=1 1 0 ( dφi (x) 0 ) dν j (x) 1 and f j = f(x)ν j (x) 0 30 / 39
Galerkin s FEM: u = f (cont.) Mostly, the test functions ν i s are the same as the basis functions φ i s for u(x) Then the stiffness matrix K is symmetric and positive definite Integrals of products of φ i (x) and ν i (x) giving the matrix components K ij should be finite ( < K ij < ) So step functions are not allowed as the basis and test functions! Derivative of a step-function is the delta-function δ, and the squared delta-function δ has an infinite integral Linear hat functions (having step functions as their derivatives) can be used as the basis and test functions 31 / 39
Linear Finite Elements: Hat Functions h = 1 3 ; u(x) = u 1φ 1 (x) + u φ (x) + u 3 φ 3 (x) Basis and test functions φ i (x) = ν i (x); i = 1,, 3: 1 φ 1 (x) φ (x) φ 3 (x) 0 h h 1 φ 1 (x) φ (x) φ 3 (x) 0 x 1 3 3x 0 0 1 3 x 3 3x 3x 1 0 3 x 1 0 3 3x 3x 3 / 39
Linear Finite Elements: An Example with Hat Functions 1 φ 1 (x) φ (x) φ 3 (x) 0.5 u(x) 0 h h 1 u 1 u u 3 0 h h 1 Piecewise-linear function u(x) = u 1 φ 1 (x) + u φ (x) + u 3 φ 3 (x) Differential equation: u = 1 (i.e. f(x) = 1) Free-end border conditions u(0) = 0; u (0) = 1 Complete solution of this differential eqution: u(x) = A + Bx x (nullspace + a particular solution); A = 0 and B = 1 (from the border conditions) u(x) = x x Test functions: ν i : hats φ 1 and φ and one half-hat φ 3 ; h = 1 3 33 / 39
FEM Example with Hat Functions (cont.) Stiffness matrix: E.g. K = K ij = K 11 = K 1 = 1 0 dφ i (x) dφ j (x) 3 dφ 1 (x) i,j=1 dφ (x) = 6 3 0 3 6 3 0 3 3 dφ 3 (x) 0 x 1 3 3 0 0 1 3 x 3 3 3 0 3 x 1 0 3 3 1 0 1 0 ( ) dφ1(x) = 9 1 3 + 9 1 3 + 0 1 3 = 6 dφ 1(x) dφ (x) = 0 1 3 9 1 3 + 0 1 3 = 3 34 / 39
FEM Example with Hat Functions (cont.) Vector f = [f 1, f, f 3 ] T = [ 1 3, 1 3, 1 6] T: f 1 = f = f 3 = 1 φ 1 (x) = 0 1 φ (x) = 0 1/3 0 = 3x = 1 φ 3 (x) = 0 = /3 1/3 3x + 1/3 0 /3 1/3 ( + (3x 1) + ( 3x) x 3x 1 /3 ) /3 1/3 (3 3x) ( ) 3x x /3 ( ) 1 + 3x 3x 1/3 1 (3x ) /3 ( 3x x ) 1 /3 /3 = 1 6 + 1 6 = 1 3 = 1 6 + 1 6 = 1 3 = 1 6 35 / 39
FEM Example with Hat Functions (cont.) Solving the finite element equation Ku = f for the mesh values in u: 6 3 0 u 1 1/3 3 6 3 u = 1/3 0 3 3 1/6 u 3 } {{ }}{{} K u u 1 5/18 u = 4/9 u 3 1/ } {{ } K 1 f } {{ } f u(x) = x x = 5/18 x = 1/3 4/9 x = /3 1/ x = 1 All three values u 1, u, u 3 agree exactly with the true solution of the differential equation u(x) = x x at the mesh points Values u i by the finite differences were not exact for this equation 36 / 39
Finite Element Methods Vs. Finite Differences FEM: two- and three-dimensional meshes of arbitrary geometry, compared to the inflexibility of a finite difference grid Using numerical integration for K and f, FEM allows for any functions f(x) and c(x) ( in solving ) the differential equation = f(x) d c(x) du(x) Linear finite elements: nd -order accuracy O(h ) in u(x) and 1 st -order accuracy O(h) in du(x) Better accuracy the higher-degree finite elements E.g. piecewise cubic polynomials with continuous slopes Adding to the hat functions the bubble functions: quadratic parabolas staying inside each mesh interval 37 / 39
More Accurate Finite Elements (optional) Hat + bubble functions piecewise linear slope u (x): u(x) = u 1 φ 1 (x)+u φ (x)+u 3 φ 3 (x)+u 4 φ 4 (x)+u 5 φ 5 (x)+u 6 φ 6 (x) 1 φ 4 φ 1 φ 5 φ φ 6 φ 3 0 h h 1 u(x) u(0.5) = u 5 + (u 1 + u )/ u = u u 1 h u 1 u u 3 0 h 0.5 h 1 Expected accuracy: 3 rd -order O(h 3 ) in u(x) and nd -order O(h ) in u (x) This higher accuracy without a very fine mesh required by only linear elements! 38 / 39
Numerical Linear Algebra Applied Mathematics (AM) Vs. Computational Science (CS): (AM) Stating a problem and building an equation to describe it (CS) Solving that equation using mostly numerical methods Numerical linear algebra represents this build up, break down process in its clearest form, with matrix models like Ku = f Crucial properties of K: symmetric or not sparse or not banded or not well conditioned or not Often the computations break K into simpler pieces The algorithm becomes clearest when it is seen as a factorisation into: triangular matrices or orthogonal matrices or very sparse matrices 39 / 39