NUMERICAL ANALYSIS STUDY GUIDE. Topic # Rootfinding

Transcription

1 NUMERICAL ANALYSIS STUDY GUIDE BRYON ARAGAM These are notes compiled during the Summer of 2010 while studying for the UCLA Numerical Analysis qualifying exam. Not all topics listed below are covered, which is a reflection of 1) Time constraints or 2) Relevance to the exam based on past exams. I feel as though most of the important topics have been covered, although there are certainly many omissions. There are surely errors and typos in this document, so please use these notes carefully. If you do happen to find an error or a typo, feel free to me. Table 1. General Exam Rubric Topic # Rootfinding 1 Interpolation/Approximation 1 Numerical Integration 1 Linear Algebra 1 Numerical ODEs 1 Finite Differences 2 Finite Elements 1 Other topics that have come up, often mixed in with one of the above topics: Error analysis and asymptotics Analysis of derivative approximations (e.g. second-order finite difference, etc.) Splines Conditioning of numerical algorithms General Topics to cover. 1. Topics Error analysis Asymptotics and error expansions Richardson extrapolation Local truncation error (LTE) analysis and global error analysis for iterative schemes Undergraduate Topics to cover. Rootfinding Bisection Newton s method, secant method Fixed point iteration Systems of nonlinear equations Polynomial interpolation Lagrange interpolation Newton divided differences Splines Approximation of continuous functions Numerical Integration Trapezoidal rule, Simpson s rule Newton-Cotes integration Gaussian quadrature Linear Algebra Some useful theorems Gaussian elimination and LU decomposition 1

2 2 BRYON ARAGAM Iterative methods General framework for iterative methods ODE Topics to cover. LTE analysis for ODE schemes Stability theory Stability of scalar ODEs Stability of autonomous ODE systems Stiff ODEs Euler s method, other basic methods Forward and backward Euler Trapezoidal and Heun s method θ-methods Runge-Kutta methods Linear multistep methods FDM Topics to cover. Convergence, consistency, stability, and Lax-Richtmeyer CFL condition Stability analysis Von Neumann stability analysis Matrix norm analysis Convergence and order of accuracy Well-posedness of equations Basic finite difference schemes Lax-Wendroff idea Crank-Nicolson Examples Methods for systems of PDEs Method of lines ADI methods FEM Topics to cover. Weak variational formulations Basic Sobolev spaces: H k, H k 0, Poincaré lemma Lax-Milgram lemma Examples FEM approximations Galerkin schemes Linear systems for FEM Finite element spaces Error estimates References Error analysis. 2. General Topics Definition. A sequence of numbers x n α is said to converge to α with order p if α x n+1 c α x n p Richardson extrapolation. Suppose we have a quantity y which is approximated by some numerical procedure using a parameter h (often the step size for an ODE/PDE solver). Let us denote this approximation by y(h), and suppose that it is known that y(h) has order of accuracy p. Suppose we are interested in improving our approximation by a linear combination of the form αy(h 1 ) + βy(h 2 ). How can we go about determining α, β and the h i? This is the idea behind Richardson extrapolation.

3 NUMERICAL ANALYSIS STUDY GUIDE 3 For simplicity we choose h 1 = h, a predetermined value, and h 2 = h/2. We can write as well as y(h) y = C p h p + C p+1 h p+1 + y( h 2 ) y = C p ( ) p h + C p+1 2 ( ) p+1 h + 2 Using these two expressions, we can eliminate the pth-order terms, 1 2 p y(h) 1 ( ) p+1 h 2 p y y(h 2 ) + y = C p+1 + = O(h p+1 ). 2 Solving for y, we get a new approximation that is an order better than our earlier approximation: (2.1) y = 2p y( h 2 ) y(h) 2 p 1 + O(h p+1 ). Clearly, the error is now order p + 1, whereas the original estimates were assumed to be of order p. 3. Undergraduate Topics 3.1. Rootfinding. Here we consider the problem of approximating solutions to the equation f(x) = Bisection. The idea of bisection is use the intermediate value theorem to zero in on a root of f(x) in the interval [a, b]. In particular, if f(a) f(b) is negative, then we know that f must have a root in (a, b). Algorithm (Bisection). c = (a+b)/2; if a-b < tol then exit; else if sgn(f(b))sgn(f(c)) < 0 c = a; else b = c; Notice that this algorithm operates under the assumption that there is at least one root in the interval [a, b]. To examine the accuracy of the bisection algorithm, observe that at each step α c b c = 1 (b a), 2 so that inductively we have for the sequence of iterates c n α c n 2 n (b a) Newton s method. Newton s method is a more sophisticated rootfinding algorithm than the bisection algorithm. It s drawback, however, is the necessity of f to be differentiable a theoretical and computational difficulty. Algorithm (Newton s method). Given an initial guess x 0 of a root to f(x) = 0, x n+1 = x n f(x n) f (x n ). To obtain an error estimate for the iterates x n, suppose f(α) = 0 and expand f as a Taylor series around x n : f(α) = 0 = f(x n ) + f (x n )(α x n ) + f (ξ n ) (α x n ) 2 2 = α = x n f(x n) f (x n ) 1 f (ξ n ) 2 f (x n ) (α x n) 2 = α x n+1 = 1 f (ξ n ) 2 f (x n ) α x n 2,

4 4 BRYON ARAGAM which shows that the iterates converge quadratically to the root α. To obtain a more quantitative result, suppose f (α) 0 and let I be an interval on which f (x) 0. Define M := max f (x) x I 2f (x). Then if x 0 I is chosen so that M α x 0 < 1, we can deduce that α x n+1 M α x n 2 M(M α x 0 ) 2n Secant method. The secant method is a modification of Newton s method, and can be seen as a way to avoid having to explicitly calculate the derivative of f. Algorithm (Secant method). Given initial guesses x 0, x 1 of a root to f(x) = 0, x n x n 1 x n+1 = x n f(x n ) f(x n ) f(x n 1 ). We omit the discussion of convergence, however it can be shown that when the secant method converges, it does so at the odd rate of 1 2 (1 + 5) Fixed point iteration. A continuous function f : I I has at least one fixed point in I := [a, b] (this is a consequence of the intermediate value theorem). We say can more if f is contractive: Theorem. Suppose f : I I is continuous and f(x) f(y) λ x y for all x, y I, 0 < λ < 1. Then f has a unique fixed point α I and the sequence of iterates x n+1 = f(x n ) converges to α for any choice of initial value x 0 I. Proof. Suppose x, y are both fixed points of f. Then f(x) f(y) = x y λ x y, which implies λ 1, which contradicts the hypotheses. Now choose x 0 I and define a sequence recursively by x n+1 = f(x n ). We then have which implies α x n = f(α) f(x n 1 ) λ α x n 1 (3.1) α x n λ n α x 0 0. When f is continuously differentiable, it suffices to check that λ := max x I f (x) < 1. Algorithm (Fixed point iteration). Given any x 0 I, define iterates by x n+1 = f(x n ). The fixed point algorithm can be used to find roots of f by considering the fixed point equation f(x)+x = x. To obtain a convergence estimate, note that Applying the estimate (3.1), we now see that α x 0 α x 1 + x 1 x 0 λ α x 0 + x 1 x 0 = α x λ x 1 x 0 α x n λn 1 λ x 1 x 0.

5 NUMERICAL ANALYSIS STUDY GUIDE Systems of nonlinear equations. In this section we consider solving f(x) = 0 where f : D D, D R m. The easiest method to discuss is fixed point iteration, which is a direct generalization of the method for real-valued functions: choose some x 0 D and compute the sequence x n+1 = f(x n ). As long as D is convex and compact, this will work: Theorem. Let D be a compact, convex subset of R m and f : D D be a C 1 function. If Df < 1, then f(x) = x has a unique solution α D and the sequence of iterates x n+1 = f(x n ) converges to α for any choice of x 0 D. The proof is a straightforward generalization of the one-dimensional case. As before, to solve f(x) = 0, consider the fixed point problem f(x) + x = x. We can also generalize Newton s method to m-dimensional nonlinear systems: Algorithm (Newton s method for nonlinear systems). Given an initial guess x 0 of a root to f(x) = 0, x n+1 = x n Df(x n ) 1 f(x n ). Newton s method for nonlinear systems can be shown to converge to a solution quadratically Polynomial Interpolation. Given distinct real numbers x 0, x 1,..., x n and associated desired function values y 0, y 1,..., y n, we are interested in finding a polynomial p(x) which interpolates the data in the sense that p(x i ) = y i for i = 0, 1,..., n. Theorem. Given data (x i, y i ), i = 0, 1,..., n, there is a unique interpolating polynomial p with deg(p) n so that p(x i ) = y i for all i. The standard way of proving this is to show that the associated linear system has a unique solution (e.g. using Vandermonde matrices). A slick way of doing so uses the fundamental theorem of algebra: Proof. Suppose that p interpolates the data (x i, 0), i = 0, 1,..., n and deg(p) n. Then p has n + 1 zeroes, which is impossible unless p 0. Thus the associated homogeneous system of equations has only the trivial solution, whence the inhomogeneous system is uniquely solvable for all choices of y i Lagrange interpolation. We now explicitly construct p using Lagrange interpolation. Consider the polynomial l i (x) := j i x x j x i x j, which has the property that l i (x j ) = δ ij. Thus we can construct an interpolating polynomial by considering p(x) = n y i l i (x). i=0 Given a function f, we can thus approximate f using the n + 1 points x 0, x 1,..., x n by forming the above interpolating polynomial with y i = f(x i ). The standard error estimate for this approximation is the following: Theorem. Suppose f C n+1 (I) where I is the smallest interval containing {x 0,..., x n, x}, x R. Then there exists ξ I so that (3.2) f(x) p n (x) = (x x 0) (x x n ) f (n+1) (ξ). (n + 1)! Here, p n (x) denotes the nth-degree interpolating polynomial for f sampled at x 0, x 1,..., x n. Proof....

6 6 BRYON ARAGAM Newton divided differences. Suppose we are given data (x i, y i ), i = 0, 1,..., n and an interpolating polynomial p n of degree n. Given an additional data point (x n+1, y n+1 ), how can we find p n+1 using what we know about p n? More explicitly, we wish to find a function C(x) so that p n+1 (x) = p n (x) + C(x). Writing C(x) = p n+1 (x) p n (x), we see that C is at most a degree n + 1 polynomial. Furthermore, since C(x i ) = p n+1 (x i ) p n (x i ) = 0 for i = 0, 1,..., n, we see that C(x) = c(x x 0 ) (x x n ). Using p n+1 (x n+1 ) = y n+1, we can now solve for the coefficient c: c = y n+1 p n (x n+1 ) (x n+1 x 0 ) (x n+1 x n ) When we are approximating a continuous function f using y i = f(x i ), this coefficient is usually denoted f[x 0, x 1,..., x n+1 ]. The quantity f[x 0, x 1,..., x n ] is called the nth-order Newton divided difference of f. The following formula is useful in computing f[x 0, x 1,..., x n ]: (3.3) f[x 0, x 1,..., x n ] = f[x 1,..., x n ] f[x 0,..., x n 1 ] x n x 0. The divided differences also give an easy way to remember the formula for the nth-degree polynomial interpolating f: (3.4) p n (x) = f(x 0 ) + (x x 0 )f[x 0, x 1 ] + (x x 0 )(x x 1 )f[x 0, x 1, x 2 ] + + (x x 0 ) (x x n 1 )f[x 0,..., x n ]. Note that since interpolating polynomials are unique, the same error estimate (3.2) holds for formula (3.4) above. A useful interpolation error formula is the following: (3.5) f(x) p n (x) = (x x 0 ) (x x n )f[x 0,..., x n, x] Comparing (3.5) with (3.2), we can write down another useful expression for f[x 0,..., x n, x]: (3.6) f[x 0,..., x n, x] = f (n+1) (ξ) (n + 1)! One last useful expression (see Atkinson, 3.2): (3.7) d dx f[x 0,..., x n, x] = f[x 0,..., x n, x, x] 3.3. Splines. We continue to consider the interpolation problem given data (x i, y i ). We will assume that we are approximating the continuous function f on the interval [a, b], so that y i = f(x i ). An mth degree spline function is a function s(x) satisfying 1 1) s(x) is a polynomial of degree m on each subinterval [x i, x i+1 ] 2) s (k) (x) is continuous on [a, b] for 0 k m 1. The most commonly used splines are linear splines (m = 1, often called linear interpolants) and cubic splines (m = 3). We skip the details of constructing spline functions here, but pause to record two important estimates on spline interpolation error: Theorem (Error Estimates for Spline Interpolation). Suppose f C 2 (resp. f C 4 ) is approximated by a linear interpolant (resp. cubic spline) on a grid a = x 0 < x 1 < < x n = b. Let L i (x) be the interpolating polynomial on the interval [x i 1, x i ], i = 1,..., n, and suppose further that each L i is 1 For convenience we have deviated from the definition found in Atkinson, so that a linear interpolant is first degree and cubic third degree, as expected. By our definition, an mth degree spline function is Atkinson s spline function of order m + 1.

7 NUMERICAL ANALYSIS STUDY GUIDE 7 interpolated by a uniform partition on [x i 1, x i ]. Define δ i = x i x i 1 (resp. δ i = (x i x i 1 )/3). Then the following estimates hold on [x i 1, x i ]: (3.8) (3.9) f(x) L i (x) δ2 i 8 max f (t) (Linear interpolant) [x i 1,x i] f(x) L i (x) δ4 i 24 max [x i 1,x i] f (4) (t) (Cubic spline) These estimates follow immediately from (3.2) and an analysis of the upper bounds of the polynomials (x x 0 )(x x 1 ) h2 4, (x x 0 )(x x 1 )(x x 2 )(x x 3 ) 3h Approximation of continuous functions Numerical Integration. In this section we consider algorithms for approximating integrals of the form I[f] := b a f(x) dx Trapezoidal rule. The idea behind the trapezoidal rule is to approximate f by trapezoids on small subintervals of [a, b], compute the area of each trapezoid, and sum the contributions to obtain an approximation to I[f]. Equivalently, approximate f by a linear interpolant and compute the area under this approximation. Suppose the interval [a, b] is subdivided by equispaced nodes a = x 0 < x 1 < < x n = b, x i = x i 1 +h with h := (b a)/n > 0. Then b a f(x) dx n i=1 This is the Trapezoidal rule: (3.10) I[f] h h 2 [f(x i 1) + f(x i )] = h ( 1 2 f(x 0) + f(x 1 ) + + f(x n 1 ) + 1 ) 2 f(x n), ( 1 2 f 0 + f f n ) 2 f n, (Trapezoidal rule) where we have simplified the notation by letting f i := f(x i ). If we assume that f C 2 [a, b], then we can obtain a reasonable error approximation for this approximation. Let E n (f) := I[f] I n [f], where I n [f] is the approximation in (3.10). First we consider the error incurred on the interval [x i 1, x i ]: xi f(x) dx h xi x i 1 2 [f(x i 1) + f(x i )] = f[x i 1, x i, x] (x x i 1 )(x x i ) dx x i 1 ( ) 1 = 2 f (ξ i ) ( 16 ) (x i x i 1 ) 3, x i 1 ξ i x i = h3 12 f (ξ i ). where we have used the Newton interpolation error estimate (3.5) for the first equality, and formula (3.6) for the second. Thus, summing over all subintervals, n { } xi E n (f) = f(x) dx h x i 1 2 [f(x i 1) + f(x i )] = i=1 n i=1 = h3 12 f (ξ i ) (b a)h2 f (ξ), a ξ b. 12

8 8 BRYON ARAGAM The last step is a clever application of the intermediate value theorem, see Atkinson 5.1 for details. In particular, the magnitude of the error can be bounded by (3.11) E n (f) (b a)h2 12 max [a,b] f (ξ), (Error estimate for trapezoidal rule) Simpson s rule. Simpson s rule improves upon the trapezoidal rule by approximating f by a quadratic spline, rather than a linear spline. For simplicity of notation we derive formulas over [a, b] and then iterate this procedure over a partition of [a, b]. Let c := (a + b)/2 and approximate f with a quadratic interpolant p 2 (x; f) using the nodes a, b, c. The resulting approximation is (3.12) (3.13) (3.14) I[f] = b a b a = b a 6 p 2 (x; f) dx f[a] + (x a)f[a, b] + (x a)(x b)f[a, b, c] dx (f(a) + 4f(c) + f(b)). Now suppose n is even and let a = x 0 < x 1 < < x n = b, x i = x i 1 + h with h := (b a)/n > 0 be a partition of [a, b]. We then have b a f(x) dx = n/2 x2i i=1 n/2 i=1 x 2i 2 f(x) dx (Use (3.14)) h 3 [f(x 2i 2) + 4f(x 2i 1 ) + f(x 2i )] = h 3 (f 0 + 4f 1 + 2f 2 + 4f f n 2 + 4f n 1 + f n ). Thus we have the following expression for Simpson s rule: (3.15) I[f] h 3 (f 0 + 4f 1 + 2f 2 + 4f f n 2 + 4f n 1 + f n ) (Simpson s rule) The error incurred by this approximation is given by (3.16) E n (f) (b a)h4 180 max f (4) (ξ), [a,b] (Error estimate for Simpson s rule) The derivation of this bound is similar to the derivation for the trapezoidal method, and is carried out in detail in Atkinson, Newton-Cotes integration. The Newton-Cotes idea simply continues the trend we have seen with both the trapezoidal and Simpson s rule: approximate f by an interpolating polynomial p n on the interval [a, b] and compute I n [f] := I[p n ] I[f]. For better results, perform the interpolation on a partition of [a, b] using a spline (sometimes called a composite rule). The important fact to know about this procedure is that, rather counterintuitively: I n [f] does not converge to I[f] as n! This is because the integration weights become arbitrarily large as n increases (see Atkinson, 5.2 for more details).

9 NUMERICAL ANALYSIS STUDY GUIDE Gaussian quadrature. In this section we generalize our integrals to include a positive weight function w(x), so I[f] = b a f(x)w(x) dx. Gaussian quadrature approximations take the form of n I[f] I n [f] := w i f(x i ), where the weights w i and nodes x i are chosen so that I n [p] = I[p] exactly for polynomials p up to some degree. If f is well approximated by polynomials on the interval [a, b], then I n [f] will represent a good approximation to I[f]. In practice, when w(x) 1 we consider the integration problem 1 f(x) dx and use the transformation rule b a f(x) dx = b a f i=1 1 ( b a 2 x + a + b ) dx 2 to evaluate I[f]. The important thing to know about Gaussian quadrature is how to derive the weights and nodes for a given n. We do this by example for n = 2. Example. We wish to derive a quadrature formula I 2 [f] = w 1 f(x 1 ) + w 2 f(x 2 ), which has four unknowns w 1, x 1, w 2, x 2. To get four constraints, we will require that E n (x k ) = 0 for k = 0, 1, 2, 3: E n (x k ) = 1 1 x k dx (w 1 x k 1 + w 2 x k 2) = 0, k = 0, 1, 2, 3. Performing the integrations and simplifying, we obtain a system of four nonlinear equations in four unknowns: w 1 + w 2 = 2 w 1 x 1 + w 2 x 2 = 0 w 1 x w 2 x 2 2 = 2 3 w 1 x w 2 x 3 2 = 0. This is a (slightly) annoying system to solve, but it can be done and it s solution is w 1 = w 2 = 1, x 1 = 1/ 3, x 2 = 1/ 3. This yields the quadrature formula ( ) ( 1 I 2 [f] = f 3 + f 1 ) Linear Algebra. This section is split into three parts: The first section outlines some general topics and theorems from linear algebra that come up frequently and are especially useful in studying numerical analysis. The last two sections are devoted to two numerical linear algebra topics that come up frequently, the LU decomposition and iterative schemes Some useful theorems. Here we collect some definitions and theorems from linear algebra that are useful in the study of numerical schemes. We offer most of these without further discussion. We will always assume that A = (a ij ) C n n and µ, λ C unless otherwise mentioned. Definition. A matrix A is called diagonally dominant if a ii > j i a ij for all i = 1,..., n. Definition. The spectrum of a matrix A is defined as it s set of eigenvalues and is denoted by σ(a). The spectral radius of A is defined as ρ(a) := sup{ λ : λ σ(a)} = max 1 i n { λ i }, where the λ i represent the eigenvalues of A. Theorem. A n 0 as n if and only if ρ(a) < 1. Moreover, if ρ(a) < 1 then I A is invertible and (I A) 1 = I + A + A 2 + A 3 +.

10 10 BRYON ARAGAM Theorem (Gershgorin s circle theorem). Define R i = j i a ij, the sum of the off-diagonal entries in the ith row. Then every eigenvalue of A lies in one of the discs B(a ii, R i ). Symbollically, σ(a) B(a ii, R i ). i n Iterative methods. In this section we consider solving Ax = b using an iterative scheme to obtain a sequence of approximations x n to the true solution x. The simplest is to rewrite the system Ax = b as (3.17) x i = 1 b i a ij x j, a ii j i and use this as a basis for our iteration. Suppose a ii 0 for all i. Given an initial guess x (0) R n to x, update our guess by the rule (3.18) x (m+1) i = 1 b i a ij x (m) j (Gauss-Jacobi) a ii j i This is called the Gauss-Jacobi method. If we modify (3.18) to use the updated values of the x (m) i we compute, we obtain the Gauss-Seidel method: (3.19) x (m+1) i = 1 a ii i 1 b i j=1 a ij x (m+1) j n j=i+1 a ij x (m) j (Gauss-Seidel) We will analyze the error for Gauss-Jacobi; the error analysis for Gauss-Seidel is similar (and slightly more complicated). See Atkinson 8.6 for complete details. If we let e (m) := x x (m), we can subtract (3.18) from (3.17) to obtain e (m+1) i a ij e (m) j a ii = j i, Letting µ = max i j i aij a ii we obtain the estimate = e (m+1) i e (m) e (m+1) µ e (m) µ m e (0), which implies that x (m) x if and only if µ < 1. This is true when A is diagonally dominant. j i a ij a ii. as General framework for iterative methods. In this section we briefly discuss the general theory behind iterative schemes such as Gauss-Jacobi and Gauss-Seidel. The main idea is to split the matrix A as A = N M, so that we can define an iterative scheme by the recursion (3.20) Nx (m+1) = b + Mx (m). As long the system Nx = c is easily solvable for instance, N is diagonal, banded, or triangular the iteration (3.20) can be computed quickly and efficiently. For example, write A = D + L + U, where D is the diagonal of A, and L and U are the strictly lower and upper triangular components of A, respectively. Then for Gauss-Jacobi, we have N = D and M = L U, and for Gauss-Seidel we have N = D + L and M = U. The error analysis for such schemes is conceptually simple. We have (3.21) (3.22) Nx = b + Mx Nx (m+1) = b + Mx (m) and subtracting these two equations we obtain (3.23) (3.24) Ne (m+1) = Me (m) = e (m+1) = N 1 Me (m) = (N 1 M) m e (0). Thus in order for e (m) 0, it is necessary and sufficient that (N 1 M) m 0, or equivalently, ρ(n 1 M) < 1.

11 NUMERICAL ANALYSIS STUDY GUIDE ODE Topics Throughout this section, we will be interested in the solution to following initial value problem by numerical means: { y = f(t, y), t 0 t T (4.1) y(t 0 ) = y 0. Some standard notation: h will always denote the time step, so that t n = t n 1 + h and y n = y(t n ). For example, Euler s method is given by (4.2) y n+1 = y n + hf(t n, y n ) LTE analysis for ODE schemes. To introduce the concept of local truncation error and the techniques involved in analyzing the error of an ODE scheme, we will give a complete analysis of Euler s method. Assume that f, f t, f y are continuous and bounded on their domain of definition. This implies, in particular, that f is globally Lipschitz with Lipschitz constant K := sup f y. Proposition. Under the above assumptions, Euler s method is order one. That is, there is a constant C so that y(t i ) y i Ch for all i. To prove this, we will do the following: 1) Estimate the local truncation error (LTE) 2) Find a relation between the global error and the LTE 3) Solve the relation from (2) and bound it Definition. The local truncation error (LTE) of a scheme is the amount by which the exact solution fails to satisfy the difference equation. For example, in Euler s method, we can substitute the exact solution at time t i into the difference equation to get y(t i ) = y(t i 1 ) + hf(y(t i 1 )) + error. The error term here is the LTE, which is often denoted τ i. 1) To estimate τ i, expand y(t i ) as a Taylor series about t i 1 : (4.3) (4.4) So we have an estimate on the LTE: y(t i ) = y(t i 1 ) + hy (t i 1 ) + h2 2 y (t i 1 ) + O(h 3 ) = y(t i 1 ) + hf(t i 1, y(t i 1 )) + h2 2 y (t i 1 ) + O(h 3 ). (4.5) (4.6) τ i = y(t i ) y(t i 1 ) hf(t i 1, y(t i 1 )) = h2 2 y (t i 1 ) + O(h 3 ) Mh 2 + O(h 3 ) where the inequality in (4.6) comes from assuming the derivatives of f are bounded. Thus we have that τ i O(h 2 ) for Euler s method. 2) We have the two equations: (4.7) (4.8) y i = y i 1 + hf(t i 1, y i 1 ) y(t i ) = y(t i 1 ) + hf(t i 1, y(t i 1 )) + τ i Subtracting (4.7) from (4.8) we can solve for the error ε i : ε i = ε i 1 + h(f(t i 1, y(t i 1 )) f(t i 1, y i 1 )) + τ i = ε i ε i 1 + h f(t i 1, y(t i 1 )) f(t i 1, y i 1 ) + τ i ε i 1 + hk y(t i 1 ) y i 1 + τ i So we have the following recursion for the absolute error ε i : (4.9) ε i (1 + hk) ε i 1 + τ i

12 12 BRYON ARAGAM 3) We can now solve the recursion (4.9). Inductively we have the following inequality for the error, n 1 ε n (1 + hk) n ε 0 + (1 + hk) p τ n p. The first term is easily done away with. Since (1 + hk) n e hkn e hkn = e K(T t0) (N = number of timesteps in procedure, so h = (T t 0 )/N), we have (4.10) p=0 (1 + hk) n ε 0 e K(T t0) ε 0. The second term is more involved. We bound the sum as follows: (4.11) n 1 n 1 (1 + hk) p τ n p Mh 2 (1 + hk) p p=0 p=0 = Mh 2 (1 + hk)n 1 (1 + hk) 1 Mh 2 ek(t t0) 1 hk ( = M ek(t t0) ) 1 h K Combining (4.10) and (4.11) we obtain ε i ε 0 e K(T t0) +Ch, where C is the constant in parentheses in (4.11). Thus, if ε 0 = 0 or ε 0 O(h), then we have that the global error for Euler s method is order one. Notice that if we replace Euler s method with the more general explicit scheme y i = y i 1 +hψ(y i 1, h) and assume that Ψ is Lipschitz in the first slot, then the previous argument can be used to show that if τ i O(h p+1 ), then the global error is O(h p ) Stability theory. In this section we consider the problem of determining a time step selection h to ensure that our approximate solution models the expected qualitative behaviour of the exact solution. To do this, we must first decide precisely what qualitative behaviour it is we are trying to mimic. If we consider the simple problem y = λy, we note that for λ < 0, all solutions tend to zero as t, and for λ > 0, all solutions blow up as t. Interpreted differently, we can say that for λ < 0, nearby solution curves contract and for λ > 0, nearby solution curves expand away from each other. How do we interpret this in terms of our approximations y i? Consider applying Euler s method to our test problem: (4.12) (4.13) (4.14) y i = y i 1 + hf(y i 1 ) = y i 1 + hλy i 1 = (1 + hλ)y i 1 This gives y i = 1 + hλ y i 1, which solves our problem. For λ < 0, we require that y i 0 as i increases, and by (4.14) we see that this happens if and only if 1 + hλ < 1. Similarly, if λ > 0, the requirement that y i implies that 1 + hλ > 1. Thus, depending on the sign of λ, we must choose h so that 1 + hλ is the right size Stability of scalar ODEs. In this section, we formalize the discussion above. We are interested in the behaviour of solutions of the ODE (4.15) y = f(t, y) In the sequel, we will refer to our test ODE y = λy as the model problem. First, a definition. Definition. The region of absolute stability of a numerical method is the set S C of hλ such that when the numerical method is applied to the model problem, the iterates obtained satisfy y k 0 as k for all initial conditions. For example, in Euler s method we previously saw that we need 1 + hλ < 1 to ensure the iterates tend to zero. Thus we have S = {hλ C 1 + hλ < 1} = B( 1, 1). In practical situations it is sometimes enough to determine the interval of absolute stability, which is just S R those hλ that lie on the real line. Thus, for example, the interval of absolute stability for

13 NUMERICAL ANALYSIS STUDY GUIDE 13 Euler s method is just ( 2, 0). In a similar fashion, we can derive the intervals of absolute stability for many other methods. Method S Euler s method ( 2, 0) Huen s method ( 2, 0) Trapezoidal method (, 0) Backward Euler (, 0) (2, ) Second-Order Runge-Kutta ( 2, 0) Fourth-Order Runge-Kutta ( 2.78, 0) With this definition in place, we can now state and prove the two major results of this section. Theorem. Consider the ODE y = f(t, y) and it s solutions near a point (t, y ). Then f y (t, y ) < 0 = nearby solutions contract, f y (t, y ) > 0 = nearby solutions expand. Proof. Assuming that f is sufficiently differentiable, we can write (4.16) f(t, y) = f(t, y ) + f y (t, y )(y y ) + f t (t, y )(t t ) +. Consider two solutions u(t) > v(t) satisfying u(t) y < ε and v(t) y < ε. Then by (4.16), (4.17) u (t) = f(t, y ) + f y (t, y )(u(t) y ) + f t (t, y )(t t ) + v (t) = f(t, y ) + f y (t, y )(v(t) y ) + f t (t, y )(t t ) + = u (t) v (t) = f y (t, y )(u(t) v(t)) + So u(t) v(t) satisfies the linear, constant coefficient equation (4.17), and the result follows. We now apply this theorem to describe how the region of absolute stability determines the behaviour of nearby solution curves to (4.15). Theorem. Consider solving the ODE y = f(t, y) near a point (t, y ) with a numerical method with region of absolute stability S. If the timestep h is chosen so that i) h f y (t, y ) S, then nearby solutions contract. ii) h f y (t, y ) / S, then nearby solutions expand. Proof. We will argue for a general Runge-Kutta method. The argument is similar for other numerical schemes. If we assume the function f is sufficiently smooth, we can write f as a Taylor series centred at (t, y ) : y (t) = λy(t) + Bt + C + where λ = f y (t, y ) and B, C are constants. If we plug this expression into the recurrence for a Runge- Kutta method, we obtain polynomials P and Q so that (4.18) y k = P (λ, h)y k 1 + Q(λ, B, C, h) + Now let u k, v k be two nearby approximate solutions. Then using (4.18) we have u k v k = P (u k 1 v k 1 ). Setting D k := u k v k, this equation becomes D k = P D k 1. This implies that D k is a numerical solution to the model problem (?). Thus hλ S implies D k 0, that is, nearby solutions contract. If hλ / S, then D k, and nearby solutions expand.

14 14 BRYON ARAGAM Stability of autonomous ODE systems. Consider the autonomous system of ODEs given by (4.19) y = F (y), where F : R n R n. Then we can extend the previous theorem in a straightforward manner: Theorem. Consider solving the system of ODEs y = F (y) near a point y with a numerical method with region of absolute stability S. Let the eigenvalues of the Jacobian matrix D y F (y ) be given by λ i, i = 1,..., n. If the timestep h is chosen so that for all i i) hλ i S, then nearby solutions contract. ii) hλ i / S, then nearby solutions expand. The proof is a straightforward extension of the proof in the scalar case, and we omit it here. Note that we do not say anything about nonautonomous systems here Stiff ODEs Euler s method, other basic methods Forward and backward Euler Trapezoidal and Heun s method θ-methods Runge-Kutta methods Linear multistep methods. The general form of a linear multistep method (LMM) is (4.20) y n+1 = p a j y n j + h j=0 p b j f(t n j, y n j ) Schemes with b 1 0 are implicit and thus require the use of nonlinear equation solvers to advance the solution. An LMM is called consistent if the LTE of the difference approximation tends to zero when applied to an arbitrary smooth function. The following theorem gives a convenient set of equivalent conditions for an LMM to be consistent. Theorem (Consistency conditions for LMM). An LMM is consistent if and only if the following two conditions hold: p p p (4.21) a j = 1, ja j + b j = 1. j=0 j=0 j= 1 j= 1 To each LMM is associated a characteristic polynomial, which is given by p (4.22) ρ(r) = r p+1 a j r p j According to the consistency conditions, ρ(1) = 0 for a consistent method. Let r 0,..., r p denote the p + 1 roots of the characteristic polynomial. We say that a method satisfies the root condition if 1) r i 1 for all i 2) r i = 1 = ρ (r i ) 0 A consistent LMM is convergent and stable if and only if it satisfies the root condition. j=0

15 NUMERICAL ANALYSIS STUDY GUIDE 15 An important diagram to keep in mind when working with linear multistep methods is Convergent Consistency + Root Condition Stable To find the region of absolute stability for an LMM, replace y k by r k in (4.20) and consider the model problem f(t, y) = λy: p p r n+1 = a j r n j + hλ b j r n j j=0 The region of absolute stability is defined as the set of hλ C so that all the roots r i of the polynomial above satisfy r i 1. For simple methods, we can usually obtain a closed formula for the r i in terms of hλ and deduce the region of absolute stability from this. j= 1 5. FDM Topics 5.1. Convergence, consistency, stability, and Lax-Richtmeyer. Fundamental to the discussion of finite difference methods (FDM for short) are the concepts of stability, convergence, and consistency. Before defining these important terms, we first establish notation. We will mostly consider one-dimensional time-dependent PDE. Let k, h > 0 be given real numbers that define a mesh on R 2 = (t, x) of the form (nk, mh) for n, m Z. Unless otherwise noted, we use the convention that t n = nk and x m = mh. Given a continuously defined function u(t, x), let u n m denote the value u(t n, x m ) = u(nk, mh). Finally, let λ = k/h, an important ratio in the study of hyperbolic FDM. In what follows, we consider the following abstract PDE: { P u = f (5.1) u(0, x) = u 0 (x) In general P will be assumed to be linear (but not necessarily constant coefficient). We denote a generic finite difference scheme by P k,h. Definition. A finite difference scheme approximating the PDE (5.1) is called convergent if for any solution to (5.1) u, the computed approximations u n m satisfy u 0 m u 0 (x) as x m x and u n m u(t, x) as (t n, x m ) (t, x). Definition. Given a PDE P u = f and an associated FDM P k,h, we say that the FDM is consistent with the PDE if for any ϕ C, P k,h ϕ P ϕ pointwise as k, h 0. In order to introduce the concept of stability, we must first define the L 2 -norm of a grid function. Given a grid on R 2 with k, h > 0 and function w defined on R (or a discrete-valued function w m ), define a norm ( 1/2 (5.2) w h := h w m ). m= This is often used to measure the size of an approximated solution at a given time level n: ( 1/2 u n h = h um ) n. m= Definition. An FDM is called stable in a stability region R [0, ) [0, ) if there is an integer J such that for any time T > 0, there is a constant C T such that for any approximation P k,h v n m, (5.3) v n 2 h C T J v j 2 h. How are these concepts related? The answer lies in the following theorem: Theorem (Lax-Richtmeyer Equivalence). Suppose that the PDE (5.1) is well-posed. Then a consistent FDM for (5.1) is convergent if and only if it is stable. j=0

16 16 BRYON ARAGAM This result is fundamental, and will be used many times to establish convergence of a given scheme Stability analysis. According to the Lax-Richtmeyer theorem, in order to check convergence we simply need to verify that our scheme is consistent and stable. Consistency should be an easy matter to verify (usually by Taylor expansions). If a scheme is not consistent, it is probably not worth looking at. Stability, on the other hand, can be trickier to establish. This section is devoted to two standard ways of checking stability of a scheme Von Neumann stability analysis. In this section will discuss how to go about performing a Von Neumann stability analysis. The relevant theory is completely omitted; we will only discuss how and what to do. An excellent introduction to the theory can be found in Strikwerda, 2. Let v n m be an approximation to the PDE P u = f according to some FDM P k,h. We can take the Fourier transform of the grid function v n m to get a real function v(ξ) in Fourier space. The basic idea behind Von Neumann analysis is to find a function g(t) so that (5.4) v n (ξ) = g(hξ) v n 1 (ξ). The function g is called the amplification factor for the scheme P k,h. By studying the scheme using Fourier analysis, we discover that the function g can be obtained by replacing all occurrences of v n m with g n e imt, and solving the resulting equation for the function g(t). Example. Consider solving the transport equation u t + au x = 0 using the forward-time backward-space scheme Subsituting u n m g n e imt, we see that and solving for g, u n+1 m = u n+1 m u n m + a un m u n m 1 = 0, k h = u n m aλ(u n m u n m 1) g n+1 e imt = g n e imt aλ(g n e imt g n e i(m 1)t ), g(t) = 1 aλ(1 e it ) The value of the amplification factor lies in the fact that if g(t) 1, we can see from (5.4) that the scheme must be stable. In fact, more than this is true: Theorem. An FDM for a PDE with constant coefficients is stable if and only if there is a constant K, independent of k, h, t, so that (5.5) g(t, k, h) 1 + Kk. If g is independent of k and h, then the condition g(t) 1 is necessary and sufficient. Note that the above theorem is only true for linear PDE with constant coefficients. This is because of the use of Fourier analysis in it s proof. Establishing the condition (5.5) can be tricky, and is key in proving stability using this technique. Often it will rely on clever trigonometric identities and estimation. If asked to prove stability for a problem with variable coefficients, we can appeal to frozen coefficient analysis. For example, consider the problem u t + a(t, x)u x = 0. By considering this equation for a fixed (t, x ), we obtain a constant coefficient problem on which we can perform Von Neumann stability analysis as above. The result will be a condition on the size of a(t, x ) (e.g. a(t, x )λ 1). The idea of frozen coefficient analysis is to note that if this condition holds for all (t, x), then the variable coefficient problem should also be stable. That is, if a λ 1, for example Matrix norm analysis.

17 NUMERICAL ANALYSIS STUDY GUIDE Convergence and order of accuracy. In considering FDM for the problem (5.1), it will be useful to write our schemes in the form (5.6) P k,h u = R k,h f, where P k,h and R k,h have the obvious interpretation. This will allow us to precisely define the order of accuracy of a given scheme below. Definition. A consistent FDM P k,h u = R k,h f is called accurate of order (p, q) if for any ϕ C, (5.7) P k,h ϕ R k,h P ϕ = O(k p ) + O(h q ). For simple schemes, it is possible to determine the order of accuracy by Taylor expansion. For complicated, higher-order schemes, however, this may be difficult. The standard way of analyzing the order of more complicated schemes is to use the symbol of a difference scheme. Given a difference operator P k,h, the associated symbol p k,h (s, ξ) is defined implicitly by P k,h (e skn e imhξ ) = p k,h (s, ξ)e skn e imhξ. Similarly we define the symbol of a differential operator P : P (e st e iξx ) = p(s, ξ)e st e iξx. When P is a linear operator, it is especially simple to compute the symbol: simply make the following replacements Example. n t s n n x (iξ) n n y (iη) n. i) P u = u tt u xx u yy + u p(s, ξ, η) = s 2 + ξ 2 + η ii) P u = u t au xx 2bu xy cu yy + ac(u x + u y ) p(s, ξ, η) = s + aξ 2 + 2bξη + cη 2 + iac(ξ + η) iii) P u = u t u xx p(s, ξ) = s + ξ 2 The value of symbols lies in the following theorem. Theorem. A consistent FDM P k,h u = R k,h f is accurate of order (p, q) if and only if (5.8) p k,h (s, ξ) r k,h (s, ξ)p(s, ξ) = O(k p ) + O(h q ) Well-posedness of equations. A PDE is called well-posed if it admits a unique solution and depends continuously on the initial/boundary data. There are various ways of making this notion precise; we do not concern ourselves with this here. The relevant fact is the following theorem, which serves as useful test for well-posedness. Theorem. Suppose that P u = f is a PDE which is at most second order in the time derivative. Then the PDE is well-posed if and only if the real parts of each of the s-roots of the symbol p(s, ξ) are uniformly bounded. In other words, let p(s, ξ) be the symbol of an initial-value problem, and let it s roots be denoted by s i (ξ), i = 1,..., n. If there is a constant C so that then the problem is well-posed (and conversely). Re s i (ξ) C for all ξ and each i,

18 18 BRYON ARAGAM 5.5. Basic finite difference schemes. Given a function u(t, x), there are several standard finite difference approximations that are useful. u x D + x u = un m+1 u n m h u x D x u = un m u n m 1 h u x D 0 xu = un m+1 u n m 1 2h + O(h) (Forward or upwind difference) + O(h) (Backward or downwind difference) + O(h 2 ) (Centred first difference) u xx D 2 xu = D + x D x u = un m+1 2u n m + u n m 1 h 2 + O(h 2 ) (Second-order second difference) u xx Dx 2,4 u = un m u n m+1 30u n m + 16u n m 1 u n m 2 12h 2 + O(h 4 ) (Fourth-order second difference) u xxx D 3 xu = D 0 xd 2 xu = un m+2 2u n m+1 + 2u n m 1 u n m 2 2h 3 + O(h 2 ) (Second-order third difference) u xxxx Dxu 4 = DxD 2 xu 2 = un m+2 4u n m+1 + 6u n m 4u n m 1 + u n m 2 h 4 + O(h 2 ) (Second-order fourth difference) Similar expressions clearly hold for time derivatives and derivatives in other spatial variables. The forward and backward differences are accurate to first-order; the centred first and second differences are second-order accurate. A special finite difference approximation that is useful is the discretization of a variable-coefficient diffusion term (a(t, x)u x ) x : (a(t, x)u x ) x = an m+1/2 (un m+1 u n m) a n m 1/2 (un m u n m 1) h 2 + O(h 2 ). For problems with more than one spatial variable, the finite differences above work just as well for non-mixed derivatives (u x, u y, u xx, u yy, etc.). Frequently we need a finite difference for a second-order mixed derivative term u xy : u xy D 0 xd 0 yu = un l+1,m+1 un l 1,m+1 un l+1,m 1 + un l 1,m 1 4h 2. As a composition of second-order centred differences, D 0 xd 0 yu is second-order in h Lax-Wendroff idea. The Lax-Wendroff idea is simple: 1) Expand u(t + k, x) as a Taylor series about the point (t, x) 2) Using the PDE, express each t-derivative as an expression in x-derivatives in u and (possibly) lower-order derivatives of the forcing term f 3) Substitute these expressions back into the Taylor series derived in (1) 4) Replace the x-derivatives with second-order approximations and t-derivatives with first-order approximations Example. Consider the one-dimensional heat equation u t = bu xx. We have (5.9) and hence u tt = b 2 u xxxx, u(t + k, x) = u(t, x) + ku t (t, x) + k2 2 u tt(t, x) + O(k 3 ) = u(t, x) + kbu xx (t, x) + k2 b 2 2 u xxxx(t, x) + O(k 3 ). We now replace u xx with the second-order second difference Dxu 2 and u xxxx with a second-order fourth difference Dxu: 4 u n+1 m = u n m + kb un m+1 2u n m + u n m 1 h 2 + k2 b 2 u n m+2 4u n m+1 + 6u n m 4u n m 1 + u n m 2 (5.10) 2 h 4 = (1 2bµ + 3b 2 µ 2 )u n m + (bµ 2b 2 µ 2 )(u n m+1 + u n m 1) + b2 µ 2 (5.11) 2 (un m+2 + u n m 2). Estimating the expression (5.11) using the norm, one can show that this scheme is stable for µ 1/2b.

19 NUMERICAL ANALYSIS STUDY GUIDE Crank-Nicolson. The Crank-Nicolson scheme arises from considering θ-weighted schemes for equations of the form (5.12) u t = au + bu x + cu xx. Consider approximating (5.12) by an FDM of the form (5.13) D + t u = θ ( au n+1 + bd 0 xu n+1 + cd 2 xu n+1) + (1 θ) ( au n + bd 0 xu n + cd 2 xu n), for some 0 θ 1. The superscript on u denotes at which time level the spatial derivative is acting. The Crank-Nicolson scheme is an implicit, second-order, unconditionally stable scheme which arises from taking θ = 1/2 in the approximation above. The scheme is accurate of order (2,2) second order in time and space. Example. For the heat equation u t = bu xx, the Crank-Nicolson scheme is given by u n+1 m = u n m + bk 2 u n+1 m+1 2un+1 m + u n+1 m 1 h 2 + bk 2 u n m+1 2u n m + u n m 1 h 2 If asked for a stable, convergent, second-order accurate scheme for a linear PDE that is first-order in time, think Crank-Nicolson. Sometimes you are asked to find explicit schemes with these properties, which is one of the few times when Crank-Nicolson cannot save you Examples Methods for systems of PDEs. Fortunately, the rules don t change much when jumping from scalar PDEs to systems of PDEs. The definitions of consistency, stability, and convergence remain the same. Furthermore, the Lax-Richtmeyer equivalence theorem carries over to systems, so in proving convergence we need only focus on establishing consistency and stability. Creating schemes for systems is much like creating schemes for scalar equations. One significant computational modification is that the amplification factor becomes a matrix amplification factor G, as opposed to the scalar function encountered in the scalar case. Consider writing out a scheme for the first-order system A general (linear) scheme will take the form U t + AU x = 0, A R d d, U R d. U n+1 m = A 1 U n m 1 + A 2 U n m + A 3 U n m+1 where A i R d d, i = 1, 2, 3 and U n m = (u 1n m, u 2n m,..., u dn m ). The matrix amplification factor is then found by substituting U n m G n e imt and solving for the matrix G. The condition for stability becomes (5.14) G n C T for all n, 0 nk T ADI methods. Consider a two-dimensional PDE of the form (5.15) u t = P 1 u + P 2 u, where the operators P i are each one-dimensional. For example, the heat equation (5.16) u t = u xx + u yy is of this form. If there are easy ways of solving the one-dimensional problems u t = P 1 u, u t = P 2 u, then the ADI method gives a way of solving (5.15) using these simpler methods. For example, Crank-Nicolson is a convenient second-order scheme for solving the one-dimensional heat equation. The ADI method allows us to take advantage of the efficiency and accuracy of the one-dimensional Crank-Nicolson scheme to solve the two-dimensional problem. Drawing inspiration from the Crank-Nicolson idea, let us approximate (5.15) by writing (5.17) D + t u = 1 2 ( P1 u n+1 + P 1 u n) ( P2 u n+1 + P 2 u n) + O(k 2 ),

20 20 BRYON ARAGAM which can be rewritten as ( I k 2 P 1 k ) ( 2 P 2 u n+1 = I + k 2 P 1 + k ) 2 P 2 u n + O(k 3 ) = (I k2 P 1 k2 ) P 2 + k2 4 P 1P 2 u n+1 = (I + k2 P 1 + k2 ) P 2 + k2 4 P 1P 2 u n + k2 4 P 1P 2 (u n+1 u n ) + O(k 3 ) ( = I k ) ( 2 P 1 I k ) ( 2 P 2 u n+1 = I + k ) ( 2 P 1 I + k ) 2 P 2 u n + k2 4 P 1P 2 (u n+1 u n ) + O(k 3 ). If we note that u n+1 u n O(k), then we can drop the last term to obtain ( I k ) ( 2 P 1 I k ) ( 2 P 2 u n+1 = I + k ) ( 2 P 1 I + k ) (5.18) 2 P 2 u n + O(k 3 ). By discretizing the operators P i, we obtain the ADI method ( I k ) ( 2 P 1h I k ) ( 2 P 2h u n+1 = I + k ) ( 2 P 1h I + k ) (5.19) 2 P 2h If the operators P ih, i = 1, 2 are each pth-order in space, then the ADI method is O(k 3 ) + O(kh p ). Why is the ADI method advantageous? Notice that the method (5.19) only requires the inversion of one one-dimensional operator at a time. Using Crank-Nicolson, for example, we need only employ a tridiagonal solver at the each step to invert our operators. If asked for a method that involves the inversion of one-dimensional operators only, use ADI! The ADI method works for any PDE that separates spatial dimensions (i.e. no mixed derivatives), such as the heat equation. u n 6. FEM Topics 6.1. Weak variational formulations. We assume the reader is familiar with the construction of the weak variational formulation of an elliptic problem. Our focus here will be on techniques for using the Lax-Milgram lemma, handling boundary conditions, and the numerical recipes for implementing FEM models. Throughout, we will consider solving the following model problem on a bounded domain : (6.1) u + a(x)u = f. In particular, we consider the above problem with the following boundary conditions (6.2) u = g on (Dirichlet BCs) (6.3) u = g on ν (Neumann BCs) (6.4) u + γu = g on ν (Robin BCs) We will always assume that (at least) f, g L 2 unless otherwise noted Basic Sobolev spaces: H k, H k 0, Poincaré lemma. We will use the notation D α u for the αth weak derivative of u. Definition. The Sobolev space H k = H k () consists of all u L 2 () so that D α u exists (weakly) for all α k and belongs to L 2 (). We denote by H k 0 the closure of C 0 () in H k (intuitively, H k 0 = { u H k u = 0 on } ). One can define Sobolev spaces for other L p spaces, but we will mainly concern ourselves with the case p = 2, in which case H k becomes a Hilbert space with the inner product (u, v) = (u, v) H k := D α ud α v dx, α k

21 NUMERICAL ANALYSIS STUDY GUIDE 21 and induced norm u = u H k = D α u 2 2 α k We now collect some important results concerning Sobolev spaces that will be useful in the development of the theory below. Theorem (Alternate characterization of H s ). f H s (R n ) if and only if (6.5) (1 + ξ 2 ) s f(ξ) 2 dξ <. R n Note that this allows us to define H s even when s is non-integral. Theorem (Poincaré inequality). There is a constant C = C() such that for all u H 1 0, we have (6.6) u 2 C Du 2. Theorem (Generalized Poincaré inequality). There is a constant C = C() such that for all u H 1, we have (6.7) where u = ffl u dx. u u 2 C Du 2, As a corollary of the Poincaré inequalities, note that Du 2 is a norm on H0 1 that is equivalent to u H 1 0. Proposition. Let be a bounded domain in R n with Lipschitz boundary. Then there exists C > 0 so that (6.8) u L2 ( ) C u H1 (). Proposition. Let be a bounded, connected domain in R n with C boundary. There exists a constant C > 0 so that (6.9) u 2 H 1 C( Du 2 + u 2 ) If we define a new norm ( u H 1, := Du 2 dx + the previous inequality becomes u H 1 C u H1,. 1/2 u 2 ds) 1/2, Inequalities (6.6)-(6.9) will prove especially useful in the study of weak variational problems Lax-Milgram lemma. The standard tool used in establishing existence and uniqueness for elliptic PDE is the Lax-Milgram lemma. Given an elliptic PDE there is an associated bilinear form a(u, v) and a linear form Lv, usually acting the space H k or some appropriate subspace thereof. In our presentation below, H denotes an arbitrary Hilbert space. Remark. Recall that a bilinear form is continuous if and only if it is bounded, that is, there exists a constant Λ > 0 so that a(u, v) Λ u v. A bilinear form is called coercive if there exists a constant α > 0 so that a(v, v) α v 2 > 0 for all v 0. Theorem (Lax-Milgram). Suppose a : H H R is a continuous, coercive bilinear form and L : H R is a continuous linear form. Then there exists a unique u H so that a(u, v) = Lv for all v H.