PART IV Spectral Methods Additional References: R. Peyret, Spectral methods for incompressible viscous flow, Springer (2002), B. Mercier, An introduction to the numerical analysis of spectral methods, Springer (1989), C. Canuto, M. Y. Hussaini, A. Quarteroni and T. A. Zang, Spectral Methods Fundamentals in Single Domains, Springer (2006). J. Shen, T. Tang and L.-L. Wang, Spectral Methods: Algorithms, Analysis and Applications, Springer (2011).
Agenda General Formulation Method of Weighted Residuals Approximation of Functions Approximation of PDEs Spectral Theorem Polynomial Approximation Orthogonal Polynomials Convergence Results Spectral Differentiation Numerical Quadratures
Method of Weighted Residuals Approximation of Functions Approximation of PDEs Spectral Methods belong to the broader category of Weighted Residual Methods, for which approximations are defined in terms of series expansions, such that a measure of the error knows as the residual is set to be zero in some approximate sense In general, an approximation u N (x) to u(x) is constructed using a set of basis functions ϕ k (x), k = 0,..., N (note that ϕ k (x) need not be orthogonal ) u N (x) k I N û k ϕ k (x), a x b, I N = {1,..., N}
Method of Weighted Residuals Approximation of Functions Approximation of PDEs Residual for two main problems: Approximation of a function u: R N (x) = u u N Approximate solution of a (differential) equation Lu f = 0: R N (x) = Lu N f In general, the residual R N is cancelled in the following sense: (R N, ψ i ) w = b a w R N ψi dx = 0, i I N, where ψ i (x), i I N are the trial (test) functions and w : [a, b] R + are the weights
Method of Weighted Residuals Approximation of Functions Approximation of PDEs Spectral Method is obtained by: selecting the basis functions ϕ k to form an orthogonal system under the weight w: (ϕ i, ϕ k ) w = δ ik, i, k I N and selecting the trial functions to coincide with the basis functions: ψ k = ϕ k, k I N with the weights w = w ( spectral Galerkin approach ), or selecting the trial functions as ψ k = δ(x x k ), x k (a, b), where x k are chosen in a non-arbitrary manner, and the weights are w = 1 ( Collocation, pseudo-spectral approach ) Note that the residual R N vanishes in the mean sense specified by the weight w in the Galerkin approach pointwise at the points xk in the collocation approach
Method of Weighted Residuals Approximation of Functions Approximation of PDEs Galerkin Method Assume that the basis functions {ϕ k } N k=1 form an orthogonal set Define the residual: R N (x) = u u N = u N k=0 ûkϕ k Cancellation of the residual in the mean sense (with the weight w) (R N, ϕ i ) w = b a ( u ) N û k ϕ k ϕ i w dx = 0, k=0 i = 0,..., N ( ) denotes complex conjugation (cf. definition of the inner product) Orthogonality of the basis / trial functions thus allows us to determine the coefficients û k by evaluating the expressions û k = b a u ϕ k w dx, k = 0,..., N Note that, for this problem, the Galerkin approach is equivalent to the Least Squares Method.
Method of Weighted Residuals Approximation of Functions Approximation of PDEs Collocation Method Define the residual: R N (x) = u u N = u N k=0 ûkϕ k Pointwise cancellation of the residual N û k ϕ k (x i ) = u(x i ), k=0 i = 0,..., N Determination of the coefficients û k thus requires solution of an algebraic system. Existence and uniqueness of solutions requires that det{ϕ k (x i )} 0 (condition on the choice of the collocation points x j and the basis functions ϕ k ) For certain pairs of basis functions ϕ k and collocation points x j the above system can be easily inverted and therefore determination of û k may be reduced to evaluation of simple expressions For this problem, the collocation method thus coincides with an interpolation technique based on the set of points {x j }
Method of Weighted Residuals Approximation of Functions Approximation of PDEs Galerkin Method (I) Consider a generic PDE problem Lu f = 0 a < x < b B u = g x = a B + u = g + x = b, where L is a linear, second-order differential operator, and B and B + represent appropriate boundary conditions (Dirichlet, Neumann, or Robin)
Method of Weighted Residuals Approximation of Functions Approximation of PDEs Galerkin Method (II) Reduce the problem to an equivalent homogeneous formulation via a lifting technique, i.e., substitute u = ũ + v, where ũ is an arbitrary function satisfying the boundary conditions above and the new (homogeneous) problem for v is Lv h = 0 a < x < b B v = 0 x = a B + v = 0 x = b, where h = f Lũ The reason for this transformation is that the basis functions ϕ k (usually) satisfy homogeneous boundary conditions.
Method of Weighted Residuals Approximation of Functions Approximation of PDEs Galerkin Method (III) The residual R N (x) = Lv N h, where v N = N k=0 ˆv kϕ k (x) satisfies ( by construction ) the boundary conditions Cancellation of the residual in the mean (cf. the Weak Formulation ) Thus (R N, ϕ i ) w = (Lv N h, ϕ i ) w, i = 0,..., N N k=0 ˆv k (Lϕ k, ϕ i ) w = (h, ϕ i ) w, i = 0,..., N, where the scalar product (Lϕ k, ϕ i ) w can be accurately evaluated using properties of the basis functions ϕ i and (h, ϕ i ) w = ĥ i An (N + 1) (N + 1) algebraic system is obtained with the matrix determined by the properties of the basis functions {ϕ k } N k=1 the properties of the operator L
Method of Weighted Residuals Approximation of Functions Approximation of PDEs Collocation Method (I) The residual (corresponding to the original inhomogeneous problem) N R N (x) = Lu N f, where u N = û k ϕ k (x) k=0 Pointwise cancellation of the residual, including the boundary nodes: Lu N (x i ) = f (x i ) i = 1,..., N 1 B u N (x 0 ) = g B + u N (x N ) = g +, This results in an (N + 1) (N + 1) algebraic system. Note that depending on the properties of the basis {ϕ 0,..., ϕ N }, this system may be singular.
Method of Weighted Residuals Approximation of Functions Approximation of PDEs Collocation Method (II) Sometimes an alternative formulation is useful, where the nodal values u N (x j ) j = 0,..., N, rather than the expansion coefficients û k, k = 0,..., N are unknown. The advantage is a convenient form of the expression for the derivative u (p) N (x i) = N j=0 d (p) ij u N (x j ), where d (p) is a p-th order differentiation matrix.
Spectral Theorem Polynomial Approximation Orthogonal Polynomials Theorem Let H be a separable Hilbert space and T a compact Hermitian operator. Then, there exists a sequence {λ n } n N and {W n } n N such that 1. λ n R, 2. the family {W n } n N forms a complete basis in H 3. T W n = λ n W n for all n N Systems of orthogonal functions are therefore related to spectra of certain operators, hence the name spectral methods
Spectral Theorem Polynomial Approximation Orthogonal Polynomials Example I Let T : L 2 (0, π) L 2 (0, π) be defined for all f L 2 (0, π) by T f = u, where u is the solution of the Dirichlet problem { u = f u(0) = u(π) = 0 Compactness of T follows from the Lax-Milgram lemma and compact embedding of H 1 (0, π) in L 2 (0, π). Eigenvalues and eigenvectors λ k = 1 k 2 and W k = 2 sin(kx) for k 1
Spectral Theorem Polynomial Approximation Orthogonal Polynomials Example I Thus, each function u L 2 (0, π) can be represented as u(x) = 2 k 1 û k W k (x), where û k = (u, W k ) L2 = 2 π π 0 u(x) sin(kx) dx. Uniform (pointwise) convergence is not guaranteed (only in L 2 sense)!
Spectral Theorem Polynomial Approximation Orthogonal Polynomials Example II Let T : L 2 (0, π) L 2 (0, π) be defined for all f L 2 (0, π) by T f = u, where u is the solution of the Neumann problem { u + u = f u (0) = u (π) = 0 Compactness of T follows from the Lax-Milgram lemma and compact embeddedness of H 1 (0, π) in L 2 (0, π). Eigenvalues and eigenvectors λ k = 1 1 + k 2 and W 0 (x) = 1, W k = 2 cos(kx) for k > 1
Spectral Theorem Polynomial Approximation Orthogonal Polynomials Example II Thus, each function u L 2 (0, π) can be represented as u(x) = 2 k 0 û k W k (x), where û k = (u, W k ) L2 = 2 π π 0 u(x) cos(kx) dx. Uniform (pointwise) convergence is not guaranteed (only in L 2 sense)!
Spectral Theorem Polynomial Approximation Orthogonal Polynomials Example III Expansion in sine series for functions vanishing on the boundaries Expansion in cosine series for functions with first derivatives vanishing on the boundaries Combining sine and cosine expansions we obtain the Fourier series expansion with the basis functions (in L 2 ( π, π)) W k (x) = e ikx, for k 0 W k form a Hilbert basis more flexible then sine or cosine series alone.
Spectral Theorem Polynomial Approximation Orthogonal Polynomials Example III Fourier series vs. Fourier transform Fourier Transform : F 1 : L 2 (R) L 2 (R), F 1 [u](k) = e ikx u(x) dx, k R : F 2 : L 2 (0, 2π) l 2, (i.e., bounded to discrete) û k = F 2 [u](k) = 2π 0 e ikx u(x) dx, k = 0, 1, 2,...
Spectral Theorem Polynomial Approximation Orthogonal Polynomials Theorem (Weierstrass Approximation Theorem) To any function f (x) that is continuous in [a, b] and to any real number ɛ > 0 there corresponds a polynomial P(x) such that P(x) f (x) C(a,b) < ɛ, i.e. the set of polynomials is dense in the Banach space C(a, b) (C(a, b) is the Banach space with the norm f C(a,b) = max x [a,b] f (x) ) Thus the power functions x k, k = 0, 1,... represent a natural basis in C(a, b) Question Is this set of basis functions useful? No! see below
Spectral Theorem Polynomial Approximation Orthogonal Polynomials Find the polynomial P N (of order N) that best approximates a function f L 2 (a, b) [note that we will need the structure of a Hilbert space, hence we go to L 2 (a, b), but C(a, b) L 2 (a, b)], i.e. b a [f (x) P N (x)] 2 dx b where P N (x) = ā 0 + ā 1 x + ā 2 x 2 + + ā N x N a [f (x) P N (x)] 2 dx Using the formula N j=0 āj(e j, e k ) = (f, e k ), j = 0,..., N, where e k = x k N b b ā k x k+j dx = x j f (x) dx N k=0 k=0 a ā k b k+j+1 a k+j+1 k + j + 1 = a b a x j f (x) dx The resulting algebraic problem is extremely ill-conditioned, e.g. for a = 0 and b = 1 1 [A] kj = k + j + 1
Spectral Theorem Polynomial Approximation Orthogonal Polynomials Much better behaved approximation problems are obtained with the use of orthogonal basis functions Such systems of orthogonal basis functions are derived by applying the Schmidt orthogonalization procedure to the system {1, x,..., x N } Various families of orthogonal polynomials are obtained depending on the choice of: the domain [a, b] over which the polynomials are defined, and the weight w characterizing the inner product (, ) w used for orthogonalization
Polynomials defined on the interval [ 1, 1] Legendre polynomials (w = 1) Spectral Theorem Polynomial Approximation Orthogonal Polynomials 2k + 1 1 d k P k (x) = 2 2 k k! dx k (x 2 1) k, k = 0, 1, 2,... Jacobi polynomials (w = (1 x) α (1 + x) β ) J (α,β) k (x) = C k (1 x) α (1+x) β d k dx k [(1 x)α+k (1+x) β+k ] k = 0, 1, 2,..., where C k is a very complicated constant Chebyshev polynomials (w = 1 1 x 2 ) T n (x) = cos(k arccos(x)), k = 0, 1, 2,..., Note that Chebyshev polynomials are obtained from Jacobi polynomials for α = β = 1/2
Spectral Theorem Polynomial Approximation Orthogonal Polynomials Polynomials defined on the periodic interval [ π, π] Trigonometric polynomials (w = 1) S k (x) = e ikx k = 0, 1, 2,... Polynomials defined on the interval [0, + ] Laguerre polynomials (w = e x ) L k (x) = 1 k! ex d k dx k (e x x k ), k = 0, 1, 2,... Polynomials defined on the interval [, + ] Hermite polynomials (w = 1) H k (x) = ( 1) k (2 k k! d k ex2 π) 1/2 dx k e x2, k = 0, 1, 2,...
Spectral Theorem Polynomial Approximation Orthogonal Polynomials What is the relationship between orthogonal polynomials and eigenfunctions of a compact Hermitian operator (cf. Spectral Theorem) Each of the aforementioned families of orthogonal polynomials forms the set of eigenvectors for the following Sturm-Liouville problem [ d p(x) dy ] + [q(x) + λr(x)] y = 0 dx dx a 1 y(a) + a 2 y (a) = 0 b 1 y(b) + b 2 y (b) = 0 for appropriately selected domain [a, b] and coefficients p, q, r, a 1, a 2, b 1, b 2.
Convergence Results Spectral Differentiation Numerical Quadratures Truncated Fourier series: u N (x) = N k= N ûkeikx The series involved 2N + 1 complex coefficients (weight w 1): û k = 1 π ue ikx dx, 2π π k = N,..., N The expansion is redundant for real-valued u the property of conjugate symmetry û k = û k, which reduces the number of complex coefficients to N + 1; furthermore, I(û 0 ) 0 for real u, thus one has 2N + 1 real coefficients; in the real case one can work with positive frequencies only! Equivalent real representation: u N (x) = a 0 + N [a k cos(kx) + b k sin(kx)], k=1 where a 0 = û 0, a k = 2R(û k ) and b k = 2I(û k ).
Convergence Results Spectral Differentiation Numerical Quadratures Uniform Convergence (I) Consider a function u that is smooth and periodic (with the period 2π); note the following two facts: The Fourier coefficients are always less than the average of u û k = 1 π u(x)e ikx dx 1 π 2π M(u) u(x) dx 2π π π If v = d α u dx α = u (α), then û k = ˆv k (ik) α
Convergence Results Spectral Differentiation Numerical Quadratures Uniform Convergence (II) Then, using integration by parts, we have û k = 1 π u(x)e ikx dx = 1 2π π 2π ] π [u(x) e ikx 1 π u (x) e ikx ik π 2π π ik dx Repeating integration by parts p times û k = ( 1) p 1 π u (p) (x) e ikx 2π π ( ik) p dx = û k M(u(p) ) k p Therefore, the more regular is the function u, the more rapidly its Fourier coefficients tend to zero as n
Convergence Results Spectral Differentiation Numerical Quadratures Uniform Convergence (III) We have û k M(u ) = k 2 k Z û ke ikx û 0 + M(u ) n 0 n 2 The latter series converges absolutely Thus, if u is twice continuously differentiable and its first derivative is continuous and periodic with period 2π, then its Fourier series u N = P N u converges uniformly to u for N Spectral convergence if φ Cp ( π, π), then for all α > 0 there exists a positive constant C α such that ˆφ k Cα n, i.e., for a α function with an infinite number of smooth derivatives, the Fourier coefficients vanish faster than algebraically Rate of decay of Fourier transform of a function f : R R is determined by its smoothness ; functions defined on a bounded (periodic) domain are a special case
Convergence Results Spectral Differentiation Numerical Quadratures Theorem (a collection of several related results, see also Trefethen (2000)) Let u L 2 (R) have Fourier transform û. If u has p 1 continuous derivatives in L 2 (R) for some p 0 and a p-th derivative of bounded variation, then û(k) = O( k p 1 ) as k, If u has infinitely many continuous derivatives in L 2 (R), then û(k) = O( k m ) as k for every m 0 (the converse also holds) If there exist a, c > 0 such that u can be extended to an analytic function in the complex strip I(z) < a with u( + iy) c uniformly for all y ( a, a), where u( + iy) is the L 2 norm along the horizontal line I(z) = y, then u a L 2 (R), where u a (k) = e a k û(k) (the converse also holds) If u can be extended to an entire function (i.e., analytic throughout the complex plane) and there exists a > 0 such that u(z) = o(e a z ) as z for all complex values z C, the û has compact support contained in [ a, a]; that is û(k) = 0 for all k > a (the converse also holds)
Convergence Results Spectral Differentiation Numerical Quadratures Radii of Convergence Darboux s Principle [see Boyd (2001)] for all types of spectral expansions (and for ordinary power series), both the domain of convergence in the complex plane and the rate of convergence are controlled by the location and strength of the gravest singularity in the complex plane ( singularities in this context denote poles, fractional powers, logarithms and discontinuities of f (z) or its derivatives) Thus, given a function f : [0, 2π] R, the rate of convergence of its Fourier series is determined by the properties of its complex extension F : C C!!! Shapes of regions of convergence: Taylor series circular disk extending up to the nearest singularity Fourier (and Hermite) series horizontal strip extending vertically up to the nearest singularity Chebyshev series ellipse with foci at x = ±1 and extending up to the nearest singularity
Convergence Results Spectral Differentiation Numerical Quadratures Periodic Sobolev Spaces Let H r p(i ) be a periodic Sobolev space, i.e., H r p(i ) = {u : u (α) L 2 (I ), α = 0,..., r}, where I = ( π, π) is a periodic interval. The space C p (I ) is dense in H r p(i ) The following two norms can be shown to be equivalent in Hp: r [ ] 1/2 [ r ] 1/2 u r = (1 + k 2 ) r û k 2, u r = Cr α u (α) 2 k Z Note that the first definition is naturally generalized for the case when r is non-integer! The projection operator P N commutes with the derivative in the distribution sense: (P N u) (α) = (ik) α û k W k = P N u (α) k N α=0
Convergence Results Spectral Differentiation Numerical Quadratures Estimates of Approximation Error in H s p(i ) Theorem Let r, s R with 0 s r; then we have: u P N u s (1 + N 2 ) s r 2 u r, for u H r p(i ) Proof. u P N u 2 s = (1 + k 2 ) s r+r û k 2 (1 + N 2 ) s r (1 + k 2 ) r û k 2 k >N k >N (1 + N 2 ) s r u 2 r Thus, accuracy of the approximation P N u is better when u is smoother ; more precisely, for u H r p(i ), the L 2 leading order error is O(N r ) which improves when r increases.
Convergence Results Spectral Differentiation Numerical Quadratures Estimates of Approximation Error in L (I ) Lemma (Sobolev Inequality) let u H 1 p(i ), then there exists a constant C such that Proof. Suppose u C p û 0 is the average of u u 2 L (I ) C u 0 u 1 (I ); note the following facts From the mean value theorem: x 0 I such that û 0 = u(x 0 ) Let v(x) = u(x) û 0, then 1 x ( x 2 v(x) 2 = v(y)v (y) dy x 0 x 0 ) 1/2 ( x ) 1/2 v(y) 2 dy v (y) 2 dy 2π v v x 0 u(x) û 0 + v(x) û 0 + 2π 1/2 v 1/2 v 1/2 C u 1/2 0 u 1/2 1, since v = u, v u and û 0 u. As C p (I ) is dense in H 1 p(i ), the inequality also holds for any u H 1 p(i ).
Convergence Results Spectral Differentiation Numerical Quadratures Estimates of Approximation Error in L (I ) An estimate in the norm L (I ) follows immediately from the previous lemma and estimates in the H s p(i ) norm where u H r p(i ) u P N u 2 L (I ) C(1 + N2 ) r 2 (1 + N 2 ) 1 r 2 u r, Thus for r 1 u P N u 2 L (I ) = O(N 1 2 r ) Uniform convergence for all u H 1 p(i ) (Note that u need only to be continuous, therefore this result is stronger than the one given earlier)
Convergence Results Spectral Differentiation Numerical Quadratures Assume we have a truncated Fourier series of u(x) N u N (x) = P N u(x) = û k e ikx k= N The Fourier series of the p th derivative of u(x) is N N u (p) N (x) = P Nu (p) = (ik) p û k e ikx = k= N k= N û (p) k eikx Thus, using the vectors Û = [û N,..., û N ] T and Û (p) = [û (p) N,..., û(p) N ]T, one can introduce the Spectral Differentiation Matrix D (p) defined in Fourier space as Û (p) = ˆD (p) Û, where N p... ˆD (p) = i p 0... N p
Convergence Results Spectral Differentiation Numerical Quadratures Properties of the spectral differentiation matrix in Fourier representation D (p) is diagonal D (p) is singular (diagonal matrix with a zero eigenvalue) after desingularization the 2 norm condition number of D (p) grows in proportion to N p (since the matrix is diagonal, this is not an issue) Question how to derive the corresponding spectral differentiation matrix in real representation? Will see shortly...
Convergence Results Spectral Differentiation Numerical Quadratures Let s return to the Spectral Galerkin Method We need to evaluate the expansion (Fourier) coefficients û k = (u, φ k ) w = b a w(x)u(x)φ k (x) dx, k = 0,..., N Quadrature is a method to evaluate such integrals approximately. Gaussian Quadrature seeks to obtain the best numerical estimate of an integral b a w(x) f (x) dx by picking optimal points x i, i = 1,..., N at which to evaluate the function f (x).
Convergence Results Spectral Differentiation Numerical Quadratures Theorem (Gauß Jacobi Integration Theorem) If (N + 1) interpolation points {x i } N i=0 are chosen to be the zeros of P N+1 (x), where P N+1 (x) is the polynomial of degree (N + 1) of the set of polynomials which are orthogonal on [a, b] with respect to the weight function w(x), then the quadrature formula b a w(x)f (x) dx = N w i f (x i ) is exact for all f (x) which are polynomials of at most degree (2N + 1) i=0
Convergence Results Spectral Differentiation Numerical Quadratures Definition Let K be a non-empty, Lipschitz, compact subset of R d. Let l q 1 be an integer. A quadrature on K with l q points consists of: A set of l q real numbers {ω 1,..., ω lq } called quadrature weights A set of l q points {ξ 1,..., ξ lq } in K called Gauß points or quadrature nodes The largest integer k such that p P k, K p(x) dx = l q l=1 ω lp(ξ l ) is called the quadrature order and is denoted by k q Remark As regards 1D bounded intervals, the most frequently used quadratures are based on Legendre polynomials which are defined on the interval (0, 1) as E k (t) = 1 d k k! (t 2 t) k, k 0. Note dt k that they are orthogonal on (0, 1) with the weight w = 1.
Convergence Results Spectral Differentiation Numerical Quadratures Theorem Let l q 1, denote by ξ 1,..., ξ lq the l q roots of the Legendre polynomial L lq (x) and set ω l = 1 lq t ξ j 0 ξ l ξ j dt. Then {ξ 1,..., ξ lq, ω 1,..., ω lq } is a quadrature of j=1 j l order k q = 2l q 1 on [0, 1]. Proof. Let {L 1,..., L lq } be the set of Lagrange polynomials associated with the Gauß points {ξ 1,..., ξ lq }. Then ω l = 1 0 L l(t) dt, 1 l l q when p(x) is a polynomial of degree less than l q, we integrate both sides of the identity p(t) = l q l=1 p(ξ l)l l (t) dx, t [0, 1] and deduce that the quadrature is exact for p(x) when the polynomial p(x) has degree less than 2l q we write it in the form p(x) = q(x)l lq (x) + r(x), where both q(x) and r(x) are polynomials of degree less than l q ; owing to orthogonality of the Legendre polynomials, we conclude 1 1 l q l q p(t) dt = r(t) dt = ω l r(ξ l ) = ω l p(ξ l ), 0 since the points ξ l are also roots of L lq 0 l=1 l=1
Convergence Results Spectral Differentiation Numerical Quadratures Periodic Gaussian Quadrature If the interval [a, b] = [0, 2π] is periodic, the weight w(x) 1 and P N (x) is the trigonometric polynomial of degree N, the Gaussian quadrature is equivalent to the Trapezoidal Rule (i.e., the quadrature with unit weights and equispaced nodes) Evaluation of the spectral coefficients: Assume {φ} N k=1 is a set of basis functions orthogonal under the weight w û k = b a w(x)u(x)φ k (x) dx = N w i u(x i )φ k (x i ), k = 0,..., N, i=0 where x i are chosen so that φ N+1 (x i ) = 0, i = 0,..., N Denoting Û = [û 0,..., û N ] T and U = [u(x 0 ),..., u(x N )] T we can write the above as Û = TU, where T is a Transformation Matrix