Advanced Computational Fluid Dynamics AA215A Lecture 2 Approximation Theory. Antony Jameson
|
|
- Alexandrina Nicholson
- 5 years ago
- Views:
Transcription
1 Advanced Computational Fluid Dynamics AA5A Lecture Approximation Theory Antony Jameson Winter Quarter, 6, Stanford, CA Last revised on January 7, 6
2 Contents Approximation Theory. Least Squares Approximation and Projection Alternative Proof of Least Squares Solution Bessel s Inequality Orthonormal Systems Construction of Orthogonal Functions Orthogonal Polynomials Zeros of Orthogonal Polynomials Legendre Polynomials Chebyshev Polynomials Best Approximation Polynomial Best Interpolating Polynomial Jacobi Polynomials Fourier Series Error Estimate for Fourier Series Orthogonality Relations for Discrete Fourier Series Trigonometric Interpolation Error Estimate for Trigonometric Interpolation Fourier Cosine Series Cosine Interpolation Chebyshev Expansions Chebyshev Interpolation Portraits of Fourier, Legendre and Chebyshev
3 Lecture Approximation Theory. Least Squares Approximation and Projection Suppose that we wish to approximate a given function f by a linear combination of independent basis functions φ j, j =,.., n. In general we wish to choose coefficients c j to minimize the error f n c j φ j (.) in some norm. Ideally we might use the infinity norm, but it turns out it is computationally very expensive to do this, requiring an iterative process. The best known method for finding the best approximation in the infinity norm is the exchange algorithm (M.J.D.Powell) On the other hand it is very easy to calculate the best approximation in the Euclidean norm as follows. Choose c j to minimize the least squares integral J(c, c, c,...c n ) = n c j φ j f dx (.) equivalent to minimizing the Euclidean norm. Then we require = J n = c j φ j f φ i dx (.3) c i Thus the best approximation is Where or f = n c jφ j (.4) n c jφ j f, φ i = (f f, φ i ) =, for i =,,,..., n (.5) n a ij c j = b i (.6)
4 LECTURE. APPROXIMATION THEORY 3 where a ij = (φ i, φ j ), b i = (f, φ i ) (.7) Notice that since f is a linear combination of the φ j these equations state that (f f, f ) = (.8) Thus the least squares approximation f is orthogonal to the error f f. This means that f is the projection of f onto the space spanned by the basis functions φ j, in the same way that a three dimensional vector might be projected onto a plane. If the φ j form an orthonormal set, a ij = δ ij (.9) where δ ij = if i = j and if i j, and we have directly c i = (f, φ i ) (.). Alternative Proof of Least Squares Solution Let f = c jφ j (.) be the best approximation and consider another approximation cj φ j (.) Then if f f is orthogonal to all φ j c j φ j f = (c j c j)φ j (f f ) (.3) = (cj c j)φ j + (f f ) (.4) Recall that in the Lecture Notes = = (, ) /.3 Bessel s Inequality Since f is orthogonal to f f the Pythagorean theorem gives Also if the φ j are orthonormal, so we have Bessel s inequality (f f ) (.5) f = f + f f = f + f f (.6) f = c j (.7) c j f (.8) Thus the series c j is convergent and if f f as n we have Parseval s formula c j = f (.9) This is the case for orthogonal polynomials because of the Weiesstrass theorem.
5 LECTURE. APPROXIMATION THEORY 4.4 Orthonormal Systems The functions φ i form an orthonormal set if they are orthogonal and φ i =. We can write where δ ij = if i = j and if i j. An example of an orthogonal set is on the inerval [, π]. Then if j k (φ i, φ j ) = δ ij (.) φ j = cos(jx), for j =,,..., n (.) Also (φ j, φ k ) = π cos(jx) cos(kx) dx = (φ j, φ j ) = π π cos (jx)dx = (cos(j k)x + cos(j + k)x) dx = (.) π ( + cos(jx)) dx = π (.3).5 Construction of Orthogonal Functions This can be done by Gram Schmidt orthogonalization. Define the inner product (f, g) = Functions g i are called independent if b a f(x)g(x)dx (.4) n α i g i = implies α i = for i =,..., n (.5) i= Let g i (x), i =,,..., n, be any set of independent functions. Then set f (x) = d g (x) (.6) f (x) = d (g (x) c f (x)) (.7). f n (x) = d n (g n (x) c n f (x)... c n,n f n (x)) (.8) We require This gives (f i, f j ) = δ ij (.9) d = (g, g ) (.3) Then = (f, f ) = d ((f, g ) c ) and in general c jk = (f j, g k ) (.3) while the d k are easily obtained from (f k, f k ) = (.3)
6 LECTURE. APPROXIMATION THEORY 5.6 Orthogonal Polynomials A similar approach can be used to construct orthogonal polynomials. Given orthogonal polynomials, φ j (x), j =,,..., n, we can generate a polynomial of degree n + by multiplying φ n (x) by x, and we can make it orthogonal to the previous φ j (x) by subtracting a linear combination of these polynomials. Thus, we set n φ n+ (x) = α n xφ n (x) c ni φ i (x) (.33) Now, But (φ i, φ j ) = for i j so α n (xφ n, φ j ) i= n c ni (φ i, φ j ) = (.34) i= c nj φ j = α n (xφ n, φ j ) = α n (φ n, xφ j ) (.35) Since xφ j is a polynomial of order j + this vanishes except for j = n, n. Thus φ n+ (x) = αxφ n (x) c nn φ n (x) c n,n φ n (x) (.36) where and α is arbitrary. c nn = α n(φ n, xφ n ) φ n, c n,n = α n(φ n, xφ n ) φ n (.37).7 Zeros of Orthogonal Polynomials An orthogonal polynomial P n (x) for the interval [a, b] has n distinct zeros in [a, b]. Suppose it had only k < n zeros x, x,..., x k at which P n (x) changes sign. Then consider Q k (x) = (x x )... (x x k ) (.38) where for k = we define Q (x) =. Then Q k (x) changes sign at the same points as P n (x), and b a P n (x)q k (x)w(x)dx (.39) But this is impossible since P n (x) is orthogonal to all polynomials of degree < n..8 Legendre Polynomials These are orthogonal in the interval [, ] with a constant weight function w(x) =. They can be generated from Rodrigues formula L (x) = (.4) L n (x) = d n [ (x n n! dx n ) n], for n =,,... (.4)
7 LECTURE. APPROXIMATION THEORY 6 The first few are L = L (x) = x L (x) = (3x ) L 3 (x) = (5x3 3x) (.4) L 4 (x) = 8 (35x4 3x + 3) L 5 (x) = 8 (63x5 7x 3 + 5x) where they can be successively generated for n by the recurrence formula L n+ = n + n + xl n(x) n n + L n (x) (.43) They are alternately odd and even and are normalized so that while for x < Also (L n, L j ) = L n () =, L n ( ) = ( ) n (.44) Writing the recurrence relation as L n (x) < (.45) { if n j L n (x)l j (x)dx = n+ if n = j (.46) L n (x) = n xl n (x) n n n L n (x) (.47) it can be seen that the leading coefficient of L n (x) is multiplied by (n )/n when L n (x) is generated. Hence the leading coefficient is a n = (n ) 3... n = n n! (n!) (.48) L n (x) also satisfies the differential equation d dx [ ( x ) d dx L n(x).9 Chebyshev Polynomials ] + n(n + )L n (x) = (.49) Let and define x = cos φ, φ π (.5) T n (x) = cos(nφ) (.5)
8 LECTURE. APPROXIMATION THEORY 7 in the interval [-,]. Since we have These are the Chebyshev polynomials. Property : Leading coefficient is n for n Property : Property 3: T n (x) has n zeros at cos(n + )φ + cos(n )φ = cos φ cos nφ, n (.5) T (x) = (.53) T (x) = x (.54). T n+ (x) = xt n (x) T n (x) (.55) T n ( x) = ( ) n T n (x) (.56) ( ) k π x k = cos n and n + extrema in [, ] with the values ( ) k, at for k =,..., n (.57) x k = cos kπ n for k =,..., n (.58) Property 4 (Continuous orthogonality): They are orthogonal in the inner product since (T r, T s ) = (f, g) = π Property 5 (Discrete orthogonality): They are orthogonal in the discrete inner product dx f(x)g(x) x if r s π cos rθ cos sθ dθ = if r = s π if r = s = (.59) (.6) where x j are the zeros of T n+ (x) and (f, g) = n f(x j )g(x j ) (.6) arccos x j = θ j = j + π, j =,,..., n (.6) n +
9 LECTURE. APPROXIMATION THEORY 8 Then (T r, T s ) = n if r s n+ cos rθ j cos sθ j = if r = s n + if r = s = (.63) Property ( 6 (Minimax): / n ) T n (x) has smallest maximum norm in [, ] of all polynomials with leading coefficient unity. Proof: Suppose p n (x) were smaller. Now at the n + extrema x,..., x n of T n (x), which all have the same magnitude of unity, p n (x n ) < n T n(x n ) (.64) p n (x n ) > n T n(x n ) (.65). Thus p n (x) ( / n ) T n (x) changes sign n times in [, ], but this is impossible since it is of degree n and has n roots.. Best Approximation Polynomial Let f(x) P n (x) have maximum deviations, +e, e, alternately at n + points x,..., x n+ in [a, b]. Then P n (x) minimizes f(x) P n (x) Proof: Suppose f(x) Q n (x) < e, then we have at x,..., x n+ Q n (x) P n (x) = f(x) P n (x) (f(x) Q n (x)) (.66) has same sign as f(x) P n (x). Thus it has opposite sign at n + points, giving n + sign changes. But this is impossible since it has only n roots.. Best Interpolating Polynomial The remainder for the Lagrange interpolation polynomial is f(x) P n (x) = w n (x) f n+ (ξ) (n + )! (.67) where w n (x) = (x x )(x x )... (x x n ) (.68) Thus we can minimize w n (x) in [, ] by making x k the zeros of T n+ (x). Then ( ) n w n (x) = T n+(x) (.69) ( ) n w n (x) = (.7)
10 LECTURE. APPROXIMATION THEORY 9. Jacobi Polynomials Jacobi polynomials P n (α,β) (x) are orthogonal in the interval [, ] for the weight function w(x) = ( x) α ( + x) β (.7) Thus the family of Jacobi polynomials includes both the Legendre polynomials L n (x) = P (,) n (x) (.7) and the Chebyshev polynomials T n (x) = P (, ) n (x) (.73).3 Fourier Series Let f(θ) be periodic with period π and square integrable on [, π]. Consider the approximation of f(θ) S n (θ) = n a + (a r cos rθ + b r sin rθ) (.74) Let us minimize Let r= { π f S n = ( f(θ) Sn (θ) ) dθ } (.75) J(a, a,..., a n, b,..., b n ) = f S n (.76) The trigonometric functions satisfy the orthogonality relations π π cos(rθ) cos(sθ)dθ = sin(rθ) sin(sθ)dθ = { if r s π if r = s { if r s π if r = s (.77) (.78) At the minimum allowing for these π sin(rθ) cos(sθ)dθ = (.79) = J π π = f(θ) cos(rθ)dθ + a r cos (rθ)dθ (.8) a r whence and similarly a r = π b r = π π π f(θ) cos(rθ)dθ (.8) f(θ) sin(rθ)dθ (.8)
11 LECTURE. APPROXIMATION THEORY Also since f S n we obtain Bessel s inequality a + n r= ( a r + b π r) f (θ)dθ (.83) π and since the right hand side is independent of n the sum converges and Also lim a r =, r lim b r =, (.84) r lim f S n = (.85) n If not, it would violate Weierstrass s theorem translated to trigonometric functions (Isaacson and Keller, p 3, p 98)..4 Error Estimate for Fourier Series Let f(θ) have K continuous derivatives. Then the Fourier coefficients decay as ( ) ( ) a r = O r K, b r = O r K and Integrating repeatedly by parts ( ) f(θ) S n (θ) = O n K a r = π π = rπ. = r π π π f(θ) cos(rθ)dθ f (θ) sin(rθ)dθ f (θ) cos(rθ)dθ (.86) (.87) (.88) Let M = sup f (K) (θ). Then since cos(rθ) and sin(rθ) and similarly a r M r K (.89) b r M r K (.9)
12 LECTURE. APPROXIMATION THEORY Also S m (θ) S n (θ) m r=n+ m r K r=n+ 4M n = 4M K (a r cos rθ + b r sin rθ) 4M dξ ξ K n K where the integral contains the sum of the rectangles shown in the sketch. (.9) Thus the partial sums S n (θ) form a Cauchy sequence and converge uniformly to f(θ), and f(θ) S n (θ) 4M K n K (.9).5 Orthogonality Relations for Discrete Fourier Series Consider where Then S = N e irθ j = N ω rj (.93) ω = e i π N, ω N = (.94) S = { ωnr if ωr ω r = N if ω r = (.95)
13 LECTURE. APPROXIMATION THEORY Therefore Taking the real part so N cos rθ j cos sθ j = N { e irθ j e isθ j if r s =, N, 4N = N if r s =, N, 4N N N N (.96) cos(r s)θ j = if r s =, N, 4N (.97) cos(r + s)θ j = if r + s =, N, 4N (.98) ( cos(r s)θj + cos(r + s)θ j ) = if r s N if r = s, N N if r = s =, N (.99) Note that N w rj is the sum of N equally spaced unit vectors as sketched, except in the case when ω r =..6 Trigonometric Interpolation Instead of finding a least square fit we can calculate the coefficients of the sum so that the sum exactly fits the function at equal intervals around the circle. Let A r and B r be chosen so that at the n + points It is easy to verify that U n (θ) = n A + (A r cos rθ + B r sin rθ) + A n cos nθ = f(θ) (.) n n r= cos rθ j cos sθ j = sin rθ j sin sθ j = θ j = j π (.) n if r s n if r = s, n n if r = s =, n if r s n if r = s, n if r = s =, n (.) (.3)
14 LECTURE. APPROXIMATION THEORY 3 n sin rθ j cos sθ j = (.4) It follows on multiplying through by cos rθ j or sin rθ j and summing over θ j that A r = n n f(θ j ) cos rθ j (.5) for r =,..., n. Note that writing B r = n n f(θ j ) sin rθ j (.6) c = A (.7) c r = (A r ib r ) (.8) c r = (A r + ib r ) (.9) the sum can be expressed as where and is defined as U n (θ) = n r= n c r e irθ = n cr e irθ (.) r= n c r = n f(θ j )e irθ j (.) n n ar = r= n n r= n a r (a n + a n ) (.).7 Error Estimate for Trigonometric Interpolation In order to estimate the error of the interpolation we compare the interpolation coefficients A r and B r with the Fourier coefficients a r and b r. The total error can then be estimated as the sum of the error in the truncated Fourier terms and the additional error introduced by replacing the Fourier coefficients by the interpolation coefficients. The interpolation coefficients can be expressed in terms of the Fourier coefficients by substituting the infinite Fourier series for f(θ) into the formula for the interpolation coefficients. For this purpose it is convenient to use the complex form. Thus C r = n f(θ j )e irθ j (.3) n = n n = n n e irθ j k= k= c k e ikθ j (.4) c k e i(r k)θ j (.5)
15 LECTURE. APPROXIMATION THEORY 4 Here Hence and similarly Accordingly and similarly e i(r k)θ j = { n if r k = qn where q =,,... if r k qn C r = c r + C r = c r + (.6) (c kn+r + c kn r ) (.7) k= (c kn+r + c kn r ) (.8) k= A r = C r + C r = a r + B r = i(c r C r ) = b r + (a kn+r + a knr r ) (.9) k= (b kn+r b kn r ) (.) Thus the difference between the interpolation and Fourier coefficients can be seen to be an aliasing error in which all the higher harmonics are lumped into the base modes. Now we can estimate A r a r, B r b r using a r M r K, k= b r M r K. (.) We get A r a r M (kn r) K + (kn + r) K k= = M (.) (n) K k K ( ) k= r K + ( ) kn + r K kn Since r n this is bounded by M (n) K k= k K (k + ) (K + )M K n K Using a similar estimate for B r we get ( + A r a r < 5M n K, ) ( dξ ξ K (K + )M K n K + ) K (.3) B r b r < 5M n K (.4) for K. Finally we can estimate the error of the interpolation sum u n (θ) as f(θ) u n (θ) f(θ) S n (θ) + S n (θ) u n (θ) (.5) < 4M n { } K n K + A r a r + B r b r (.6) r= < 4M M(n + ) + K nk n K (.7)
16 LECTURE. APPROXIMATION THEORY 5.8 Fourier Cosine Series For a function defined for θ π the Fourier cosine series is where a r = π S c (θ) = a = π π a r cos rθ (.8) r= π f(θ)dθ (.9) f(θ) cos(rθ)dθ, r > (.3) Hence since d dθ cos(rθ) = at θ =, π the function will not be well approximated at, π unless f (θ) = f (π) =. The cosine series in fact implies an even continuation at θ =, π. Now if all derivatives of f(θ) of order,,..., K, are continuous, and f (p) () = f (p) (π) = for all odd p < K and f (K) (θ) is integrable, then integration by parts from the coefficients of f (K) (θ) gives.9 Cosine Interpolation Let f(θ) be approximated by and let where Then U n (θ) = a r M r k (.3) n a r cos rθ (.3) U n (θ j ) = f(θ j ) for j =,,...n (.33) n cos rθ j cos sθ j = θ j = j + π n + if r s n n+ if r = s n n + if = r = s Now if we multiply the first equation by cos sθ j and sum, we find that a r = n + (.34) (.35) n f(θ j ) cos rθ j for < r < n (.36) a = n + n f(θ j ) (.37)
17 LECTURE. APPROXIMATION THEORY 6. Chebyshev Expansions Let g(x) = a r T r (x) (.38) r= approximate f(x) in the interval [, ]. Then G(θ) = g(cos θ) is the Fourier cosine series for F (θ) = f(cos θ) for θ π, since T r (cos θ) = cos rθ (.39) G(θ) = g(cos θ) = a r cos rθ (.4) Thus and a r = π a = π π π f(cos θ)dθ = π f(cos θ) cos rθdθ = π r= dx f(x) x dx f(x)t r (x) x (.4) (.4) Now from the theory of Fourier cosine series if f (p) (x) is continuous for x and p =,,..., K, and f (K) (x) is integrable a r M r K (.43) Since T r (x) for x the remainder after n terms is O ( n K ). Now F (θ) = f (cos θ) sin θ (.44) F (θ) = f (cos θ) sin θ f (cos θ) cos θ (.45) F (θ) = f (cos θ) sin 3 θ + 3f (cos θ) sin θ cos θ + f (cos θ) sin θ (.46) and in general, if F (p) is bounded, F (p) () = F (p) (π) = (.47) when p is odd, since then F (p) contains the term sin θ. As a result, the favorable error estimate applies, provided that derivatives of f exist up to the required order.. Chebyshev Interpolation We can transform cosine interpolation to interpolation with Chebyshev polynomials by setting and choosing the coefficients so that G n (x) = n a r T r (x) (.48) r= G n (x j ) = f(x j ) for j =,,...n (.49)
18 LECTURE. APPROXIMATION THEORY 7 where x j = cos θ j = cos ( j + n + ) π (.5) These are the zeros of T n+ (x). Now for any k < n we can find the discrete least squares approximation. When k = n, because of the above equation, the error is zero, so the least squares approximation is the interpolation polynomial. Moreover, since G n (x) is a polynomial of degree n, this is the Lagrange interpolation polynomial at the zeros of T n+ (x).. Portraits of Fourier, Legendre and Chebyshev Figure.: Joseph Fourier (768-83)
19 LECTURE. APPROXIMATION THEORY 8 Figure.: Adrien-Marie Legendre (75-833) Figure.3: Pafnuty Lvovich Chebyshev (8-894)
Recall that any inner product space V has an associated norm defined by
Hilbert Spaces Recall that any inner product space V has an associated norm defined by v = v v. Thus an inner product space can be viewed as a special kind of normed vector space. In particular every inner
More informationApproximation theory
Approximation theory Xiaojing Ye, Math & Stat, Georgia State University Spring 2019 Numerical Analysis II Xiaojing Ye, Math & Stat, Georgia State University 1 1 1.3 6 8.8 2 3.5 7 10.1 Least 3squares 4.2
More informationVectors in Function Spaces
Jim Lambers MAT 66 Spring Semester 15-16 Lecture 18 Notes These notes correspond to Section 6.3 in the text. Vectors in Function Spaces We begin with some necessary terminology. A vector space V, also
More informationAPPENDIX B GRAM-SCHMIDT PROCEDURE OF ORTHOGONALIZATION. Let V be a finite dimensional inner product space spanned by basis vector functions
301 APPENDIX B GRAM-SCHMIDT PROCEDURE OF ORTHOGONALIZATION Let V be a finite dimensional inner product space spanned by basis vector functions {w 1, w 2,, w n }. According to the Gram-Schmidt Process an
More informationMath 489AB A Very Brief Intro to Fourier Series Fall 2008
Math 489AB A Very Brief Intro to Fourier Series Fall 8 Contents Fourier Series. The coefficients........................................ Convergence......................................... 4.3 Convergence
More informationInner products. Theorem (basic properties): Given vectors u, v, w in an inner product space V, and a scalar k, the following properties hold:
Inner products Definition: An inner product on a real vector space V is an operation (function) that assigns to each pair of vectors ( u, v) in V a scalar u, v satisfying the following axioms: 1. u, v
More informationApproximation Theory
Approximation Theory Function approximation is the task of constructing, for a given function, a simpler function so that the difference between the two functions is small and to then provide a quantifiable
More informationVector Spaces. Vector space, ν, over the field of complex numbers, C, is a set of elements a, b,..., satisfying the following axioms.
Vector Spaces Vector space, ν, over the field of complex numbers, C, is a set of elements a, b,..., satisfying the following axioms. For each two vectors a, b ν there exists a summation procedure: a +
More informationHilbert Spaces. Hilbert space is a vector space with some extra structure. We start with formal (axiomatic) definition of a vector space.
Hilbert Spaces Hilbert space is a vector space with some extra structure. We start with formal (axiomatic) definition of a vector space. Vector Space. Vector space, ν, over the field of complex numbers,
More informationFinite-dimensional spaces. C n is the space of n-tuples x = (x 1,..., x n ) of complex numbers. It is a Hilbert space with the inner product
Chapter 4 Hilbert Spaces 4.1 Inner Product Spaces Inner Product Space. A complex vector space E is called an inner product space (or a pre-hilbert space, or a unitary space) if there is a mapping (, )
More information256 Summary. D n f(x j ) = f j+n f j n 2n x. j n=1. α m n = 2( 1) n (m!) 2 (m n)!(m + n)!. PPW = 2π k x 2 N + 1. i=0?d i,j. N/2} N + 1-dim.
56 Summary High order FD Finite-order finite differences: Points per Wavelength: Number of passes: D n f(x j ) = f j+n f j n n x df xj = m α m dx n D n f j j n= α m n = ( ) n (m!) (m n)!(m + n)!. PPW =
More informationMATH 220: INNER PRODUCT SPACES, SYMMETRIC OPERATORS, ORTHOGONALITY
MATH 22: INNER PRODUCT SPACES, SYMMETRIC OPERATORS, ORTHOGONALITY When discussing separation of variables, we noted that at the last step we need to express the inhomogeneous initial or boundary data as
More informationLecture 16: Bessel s Inequality, Parseval s Theorem, Energy convergence
Introductory lecture notes on Partial Differential Equations - c Anthony Peirce. ot to be copied, used, or revised without explicit written permission from the copyright owner. ecture 6: Bessel s Inequality,
More informationFunctional Analysis Exercise Class
Functional Analysis Exercise Class Week: December 4 8 Deadline to hand in the homework: your exercise class on week January 5. Exercises with solutions ) Let H, K be Hilbert spaces, and A : H K be a linear
More informationThis ODE arises in many physical systems that we shall investigate. + ( + 1)u = 0. (λ + s)x λ + s + ( + 1) a λ. (s + 1)(s + 2) a 0
Legendre equation This ODE arises in many physical systems that we shall investigate We choose We then have Substitution gives ( x 2 ) d 2 u du 2x 2 dx dx + ( + )u u x s a λ x λ a du dx λ a λ (λ + s)x
More informationBackground and Definitions...2. Legendre s Equation, Functions and Polynomials...4 Legendre s Associated Equation and Functions...
Legendre Polynomials and Functions Reading Problems Outline Background and Definitions...2 Definitions...3 Theory...4 Legendre s Equation, Functions and Polynomials...4 Legendre s Associated Equation and
More informationwhich arises when we compute the orthogonal projection of a vector y in a subspace with an orthogonal basis. Hence assume that P y = A ij = x j, x i
MODULE 6 Topics: Gram-Schmidt orthogonalization process We begin by observing that if the vectors {x j } N are mutually orthogonal in an inner product space V then they are necessarily linearly independent.
More informationLinear Algebra Massoud Malek
CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product
More informationExercise 11. Isao Sasano
Exercise Isao Sasano Exercise Calculate the value of the following series by using the Parseval s equality for the Fourier series of f(x) x on the range [, π] following the steps ()-(5). () Calculate the
More informationPolynomial approximation via de la Vallée Poussin means
Summer School on Applied Analysis 2 TU Chemnitz, September 26-3, 2 Polynomial approximation via de la Vallée Poussin means Lecture 2: Discrete operators Woula Themistoclakis CNR - National Research Council
More informationNumerical Analysis An Advanced Course. Gheorghe Coman, Ioana Chiorean, Teodora Cătinaş
Numerical Analysis An Advanced Course Gheorghe Coman, Ioana Chiorean, Teodora Cătinaş 2 Contents 1 Preliminaries 9 1.1 Linear spaces........................... 9 1.2 Examples of functions spaces...................
More informationswapneel/207
Partial differential equations Swapneel Mahajan www.math.iitb.ac.in/ swapneel/207 1 1 Power series For a real number x 0 and a sequence (a n ) of real numbers, consider the expression a n (x x 0 ) n =
More informationSection 7.5 Inner Product Spaces
Section 7.5 Inner Product Spaces With the dot product defined in Chapter 6, we were able to study the following properties of vectors in R n. ) Length or norm of a vector u. ( u = p u u ) 2) Distance of
More informationi x i y i
Department of Mathematics MTL107: Numerical Methods and Computations Exercise Set 8: Approximation-Linear Least Squares Polynomial approximation, Chebyshev Polynomial approximation. 1. Compute the linear
More informationQualification Exam: Mathematical Methods
Qualification Exam: Mathematical Methods Name:, QEID#41534189: August, 218 Qualification Exam QEID#41534189 2 1 Mathematical Methods I Problem 1. ID:MM-1-2 Solve the differential equation dy + y = sin
More informationBest approximation in the 2-norm
Best approximation in the 2-norm Department of Mathematical Sciences, NTNU september 26th 2012 Vector space A real vector space V is a set with a 0 element and three operations: Addition: x, y V then x
More informationGeneral Inner Product and The Fourier Series
A Linear Algebra Approach Department of Mathematics University of Puget Sound 4-20-14 / Spring Semester Outline 1 2 Inner Product The inner product is an algebraic operation that takes two vectors and
More information8.2 Discrete Least Squares Approximation
Chapter 8 Approximation Theory 8.1 Introduction Approximation theory involves two types of problems. One arises when a function is given explicitly, but we wish to find a simpler type of function, such
More informationCONVERGENCE OF THE FOURIER SERIES
CONVERGENCE OF THE FOURIER SERIES SHAW HAGIWARA Abstract. The Fourier series is a expression of a periodic, integrable function as a sum of a basis of trigonometric polynomials. In the following, we first
More informationReal Variables # 10 : Hilbert Spaces II
randon ehring Real Variables # 0 : Hilbert Spaces II Exercise 20 For any sequence {f n } in H with f n = for all n, there exists f H and a subsequence {f nk } such that for all g H, one has lim (f n k,
More informationSolutions: Problem Set 3 Math 201B, Winter 2007
Solutions: Problem Set 3 Math 201B, Winter 2007 Problem 1. Prove that an infinite-dimensional Hilbert space is a separable metric space if and only if it has a countable orthonormal basis. Solution. If
More informationNumerical Methods I Orthogonal Polynomials
Numerical Methods I Orthogonal Polynomials Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course G63.2010.001 / G22.2420-001, Fall 2010 Nov. 4th and 11th, 2010 A. Donev (Courant Institute)
More informationEXAMINATION: MATHEMATICAL TECHNIQUES FOR IMAGE ANALYSIS
EXAMINATION: MATHEMATICAL TECHNIQUES FOR IMAGE ANALYSIS Course code: 8D Date: Thursday April 8 th, Time: 4h 7h Place: AUD 3 Read this first! Write your name and student identification number on each paper
More informationLEGENDRE POLYNOMIALS AND APPLICATIONS. We construct Legendre polynomials and apply them to solve Dirichlet problems in spherical coordinates.
LEGENDRE POLYNOMIALS AND APPLICATIONS We construct Legendre polynomials and apply them to solve Dirichlet problems in spherical coordinates.. Legendre equation: series solutions The Legendre equation is
More informationTHE PROBLEMS FOR THE SECOND TEST FOR BRIEF SOLUTIONS
THE PROBLEMS FOR THE SECOND TEST FOR 18.102 BRIEF SOLUTIONS RICHARD MELROSE Question.1 Show that a subset of a separable Hilbert space is compact if and only if it is closed and bounded and has the property
More informationInner Product Spaces An inner product on a complex linear space X is a function x y from X X C such that. (1) (2) (3) x x > 0 for x 0.
Inner Product Spaces An inner product on a complex linear space X is a function x y from X X C such that (1) () () (4) x 1 + x y = x 1 y + x y y x = x y x αy = α x y x x > 0 for x 0 Consequently, (5) (6)
More informationScientific Computing
2301678 Scientific Computing Chapter 2 Interpolation and Approximation Paisan Nakmahachalasint Paisan.N@chula.ac.th Chapter 2 Interpolation and Approximation p. 1/66 Contents 1. Polynomial interpolation
More informationTopics in Fourier analysis - Lecture 2.
Topics in Fourier analysis - Lecture 2. Akos Magyar 1 Infinite Fourier series. In this section we develop the basic theory of Fourier series of periodic functions of one variable, but only to the extent
More informationLectures notes. Rheology and Fluid Dynamics
ÉC O L E P O L Y T E C H N IQ U E FÉ DÉR A L E D E L A U S A N N E Christophe Ancey Laboratoire hydraulique environnementale (LHE) École Polytechnique Fédérale de Lausanne Écublens CH-05 Lausanne Lectures
More informationFourier Series. 1. Review of Linear Algebra
Fourier Series In this section we give a short introduction to Fourier Analysis. If you are interested in Fourier analysis and would like to know more detail, I highly recommend the following book: Fourier
More informationbe the set of complex valued 2π-periodic functions f on R such that
. Fourier series. Definition.. Given a real number P, we say a complex valued function f on R is P -periodic if f(x + P ) f(x) for all x R. We let be the set of complex valued -periodic functions f on
More informationPhysics 342 Lecture 2. Linear Algebra I. Lecture 2. Physics 342 Quantum Mechanics I
Physics 342 Lecture 2 Linear Algebra I Lecture 2 Physics 342 Quantum Mechanics I Wednesday, January 27th, 21 From separation of variables, we move to linear algebra Roughly speaking, this is the study
More informationChebyshev approximation
faculty of science and engineering Chebyshev approximation Bètawetenschappelijk Onderzoek: Wiskunde July 017 Student: M.H. Mudde First supervisor: Dr. A. E. Sterk Second supervisor: Prof. dr. A. J. van
More information1. Subspaces A subset M of Hilbert space H is a subspace of it is closed under the operation of forming linear combinations;i.e.,
Abstract Hilbert Space Results We have learned a little about the Hilbert spaces L U and and we have at least defined H 1 U and the scale of Hilbert spaces H p U. Now we are going to develop additional
More informationON A WEIGHTED INTERPOLATION OF FUNCTIONS WITH CIRCULAR MAJORANT
ON A WEIGHTED INTERPOLATION OF FUNCTIONS WITH CIRCULAR MAJORANT Received: 31 July, 2008 Accepted: 06 February, 2009 Communicated by: SIMON J SMITH Department of Mathematics and Statistics La Trobe University,
More informationMATH 5640: Fourier Series
MATH 564: Fourier Series Hung Phan, UMass Lowell September, 8 Power Series A power series in the variable x is a series of the form a + a x + a x + = where the coefficients a, a,... are real or complex
More informationMathematical Methods for Computer Science
Mathematical Methods for Computer Science Computer Laboratory Computer Science Tripos, Part IB Michaelmas Term 2016/17 Professor J. Daugman Exercise problems Fourier and related methods 15 JJ Thomson Avenue
More informationIntroduction to Orthogonal Polynomials: Definition and basic properties
Introduction to Orthogonal Polynomials: Definition and basic properties Prof. Dr. Mama Foupouagnigni African Institute for Mathematical Sciences, Limbe, Cameroon and Department of Mathematics, Higher Teachers
More informationMath 115 ( ) Yum-Tong Siu 1. Derivation of the Poisson Kernel by Fourier Series and Convolution
Math 5 (006-007 Yum-Tong Siu. Derivation of the Poisson Kernel by Fourier Series and Convolution We are going to give a second derivation of the Poisson kernel by using Fourier series and convolution.
More informationLinear Algebra. Paul Yiu. Department of Mathematics Florida Atlantic University. Fall A: Inner products
Linear Algebra Paul Yiu Department of Mathematics Florida Atlantic University Fall 2011 6A: Inner products In this chapter, the field F = R or C. We regard F equipped with a conjugation χ : F F. If F =
More informationTAYLOR AND MACLAURIN SERIES
TAYLOR AND MACLAURIN SERIES. Introduction Last time, we were able to represent a certain restricted class of functions as power series. This leads us to the question: can we represent more general functions
More informationPolynomials. p n (x) = a n x n + a n 1 x n 1 + a 1 x + a 0, where
Polynomials Polynomials Evaluation of polynomials involve only arithmetic operations, which can be done on today s digital computers. We consider polynomials with real coefficients and real variable. p
More informationFurther Mathematical Methods (Linear Algebra) 2002
Further Mathematical Methods (Linear Algebra) Solutions For Problem Sheet 9 In this problem sheet, we derived a new result about orthogonal projections and used them to find least squares approximations
More informationMTH4101 CALCULUS II REVISION NOTES. 1. COMPLEX NUMBERS (Thomas Appendix 7 + lecture notes) ax 2 + bx + c = 0. x = b ± b 2 4ac 2a. i = 1.
MTH4101 CALCULUS II REVISION NOTES 1. COMPLEX NUMBERS (Thomas Appendix 7 + lecture notes) 1.1 Introduction Types of numbers (natural, integers, rationals, reals) The need to solve quadratic equations:
More informationInterpolation. Chapter Interpolation. 7.2 Existence, Uniqueness and conditioning
76 Chapter 7 Interpolation 7.1 Interpolation Definition 7.1.1. Interpolation of a given function f defined on an interval [a,b] by a polynomial p: Given a set of specified points {(t i,y i } n with {t
More informationExamples of the Fourier Theorem (Sect. 10.3). The Fourier Theorem: Continuous case.
s of the Fourier Theorem (Sect. 1.3. The Fourier Theorem: Continuous case. : Using the Fourier Theorem. The Fourier Theorem: Piecewise continuous case. : Using the Fourier Theorem. The Fourier Theorem:
More informationA Concise Introduction to Numerical Analysis. Douglas N. Arnold
A Concise Introduction to Numerical Analysis Douglas N. Arnold School of Mathematics, University of Minnesota, Minneapolis, MN 55455 E-mail address: arnold@umn.edu URL: http://umn.edu/~arnold/ 1991 Mathematics
More informationYou may not start to read the questions printed on the subsequent pages until instructed to do so by the Invigilator.
MATHEMATICAL TRIPOS Part IB Thursday 7 June 2007 9 to 12 PAPER 3 Before you begin read these instructions carefully. Each question in Section II carries twice the number of marks of each question in Section
More informationExercise Solutions to Functional Analysis
Exercise Solutions to Functional Analysis Note: References refer to M. Schechter, Principles of Functional Analysis Exersize that. Let φ,..., φ n be an orthonormal set in a Hilbert space H. Show n f n
More informationMath 61CM - Solutions to homework 6
Math 61CM - Solutions to homework 6 Cédric De Groote November 5 th, 2018 Problem 1: (i) Give an example of a metric space X such that not all Cauchy sequences in X are convergent. (ii) Let X be a metric
More informationLecture # 3 Orthogonal Matrices and Matrix Norms. We repeat the definition an orthogonal set and orthornormal set.
Lecture # 3 Orthogonal Matrices and Matrix Norms We repeat the definition an orthogonal set and orthornormal set. Definition A set of k vectors {u, u 2,..., u k }, where each u i R n, is said to be an
More informationETNA Kent State University
Electronic Transactions on Numerical Analysis. Volume 35, pp. 148-163, 2009. Copyright 2009,. ISSN 1068-9613. ETNA SPHERICAL QUADRATURE FORMULAS WITH EQUALLY SPACED NODES ON LATITUDINAL CIRCLES DANIELA
More informationwhere the bar indicates complex conjugation. Note that this implies that, from Property 2, x,αy = α x,y, x,y X, α C.
Lecture 4 Inner product spaces Of course, you are familiar with the idea of inner product spaces at least finite-dimensional ones. Let X be an abstract vector space with an inner product, denoted as,,
More information(0, 0), (1, ), (2, ), (3, ), (4, ), (5, ), (6, ).
1 Interpolation: The method of constructing new data points within the range of a finite set of known data points That is if (x i, y i ), i = 1, N are known, with y i the dependent variable and x i [x
More informationHilbert Spaces. Contents
Hilbert Spaces Contents 1 Introducing Hilbert Spaces 1 1.1 Basic definitions........................... 1 1.2 Results about norms and inner products.............. 3 1.3 Banach and Hilbert spaces......................
More informationNumerical Analysis Preliminary Exam 10 am to 1 pm, August 20, 2018
Numerical Analysis Preliminary Exam 1 am to 1 pm, August 2, 218 Instructions. You have three hours to complete this exam. Submit solutions to four (and no more) of the following six problems. Please start
More informationMa 221 Eigenvalues and Fourier Series
Ma Eigenvalues and Fourier Series Eigenvalue and Eigenfunction Examples Example Find the eigenvalues and eigenfunctions for y y 47 y y y5 Solution: The characteristic equation is r r 47 so r 44 447 6 Thus
More informationPhysics 342 Lecture 2. Linear Algebra I. Lecture 2. Physics 342 Quantum Mechanics I
Physics 342 Lecture 2 Linear Algebra I Lecture 2 Physics 342 Quantum Mechanics I Wednesday, January 3th, 28 From separation of variables, we move to linear algebra Roughly speaking, this is the study of
More information12.0 Properties of orthogonal polynomials
12.0 Properties of orthogonal polynomials In this section we study orthogonal polynomials to use them for the construction of quadrature formulas investigate projections on polynomial spaces and their
More informationTwo special equations: Bessel s and Legendre s equations. p Fourier-Bessel and Fourier-Legendre series. p
LECTURE 1 Table of Contents Two special equations: Bessel s and Legendre s equations. p. 259-268. Fourier-Bessel and Fourier-Legendre series. p. 453-460. Boundary value problems in other coordinate system.
More informationPreliminary Exam 2016 Solutions to Morning Exam
Preliminary Exam 16 Solutions to Morning Exam Part I. Solve four of the following five problems. Problem 1. Find the volume of the ice cream cone defined by the inequalities x + y + z 1 and x + y z /3
More informationIntroduction to Signal Spaces
Introduction to Signal Spaces Selin Aviyente Department of Electrical and Computer Engineering Michigan State University January 12, 2010 Motivation Outline 1 Motivation 2 Vector Space 3 Inner Product
More informationDifferential Equations
Differential Equations Problem Sheet 1 3 rd November 2011 First-Order Ordinary Differential Equations 1. Find the general solutions of the following separable differential equations. Which equations are
More informationMethods of Mathematical Physics X1 Homework 3 Solutions
Methods of Mathematical Physics - 556 X Homework 3 Solutions. (Problem 2.. from Keener.) Verify that l 2 is an inner product space. Specifically, show that if x, y l 2, then x, y x k y k is defined and
More informationd 1 µ 2 Θ = 0. (4.1) consider first the case of m = 0 where there is no azimuthal dependence on the angle φ.
4 Legendre Functions In order to investigate the solutions of Legendre s differential equation d ( µ ) dθ ] ] + l(l + ) m dµ dµ µ Θ = 0. (4.) consider first the case of m = 0 where there is no azimuthal
More informationAP Calculus Chapter 9: Infinite Series
AP Calculus Chapter 9: Infinite Series 9. Sequences a, a 2, a 3, a 4, a 5,... Sequence: A function whose domain is the set of positive integers n = 2 3 4 a n = a a 2 a 3 a 4 terms of the sequence Begin
More informationRecall: Dot product on R 2 : u v = (u 1, u 2 ) (v 1, v 2 ) = u 1 v 1 + u 2 v 2, u u = u u 2 2 = u 2. Geometric Meaning:
Recall: Dot product on R 2 : u v = (u 1, u 2 ) (v 1, v 2 ) = u 1 v 1 + u 2 v 2, u u = u 2 1 + u 2 2 = u 2. Geometric Meaning: u v = u v cos θ. u θ v 1 Reason: The opposite side is given by u v. u v 2 =
More informationSPECTRAL METHODS: ORTHOGONAL POLYNOMIALS
SPECTRAL METHODS: ORTHOGONAL POLYNOMIALS 31 October, 2007 1 INTRODUCTION 2 ORTHOGONAL POLYNOMIALS Properties of Orthogonal Polynomials 3 GAUSS INTEGRATION Gauss- Radau Integration Gauss -Lobatto Integration
More information4 Hilbert spaces. The proof of the Hilbert basis theorem is not mathematics, it is theology. Camille Jordan
The proof of the Hilbert basis theorem is not mathematics, it is theology. Camille Jordan Wir müssen wissen, wir werden wissen. David Hilbert We now continue to study a special class of Banach spaces,
More informationLeast Squares Curve Fitting A.F. Emery 1
Least Squares Curve Fitting AF Emery 1 Introduction Suppose that a system has a constant response, α, which we attempt to determine by measuring the response n times Let these responses be x i and let
More informationChapter 1. Preliminaries. The purpose of this chapter is to provide some basic background information. Linear Space. Hilbert Space.
Chapter 1 Preliminaries The purpose of this chapter is to provide some basic background information. Linear Space Hilbert Space Basic Principles 1 2 Preliminaries Linear Space The notion of linear space
More informationLecture 1. 1, 0 x < 1 2 1, 2. x < 1, 0, elsewhere. 1
0 - - -3 Lecture Introductory mathematical ideas The term wavelet, meaning literally little wave, originated in the early 980s in its French version ondelette in the work of Morlet and some French seismologists.
More informationMathematics Department Stanford University Math 61CM/DM Inner products
Mathematics Department Stanford University Math 61CM/DM Inner products Recall the definition of an inner product space; see Appendix A.8 of the textbook. Definition 1 An inner product space V is a vector
More informationPower Series Solutions to the Legendre Equation
Power Series Solutions to the Legendre Equation Department of Mathematics IIT Guwahati The Legendre equation The equation (1 x 2 )y 2xy + α(α + 1)y = 0, (1) where α is any real constant, is called Legendre
More information7: FOURIER SERIES STEVEN HEILMAN
7: FOURIER SERIES STEVE HEILMA Contents 1. Review 1 2. Introduction 1 3. Periodic Functions 2 4. Inner Products on Periodic Functions 3 5. Trigonometric Polynomials 5 6. Periodic Convolutions 7 7. Fourier
More informationBessel s and legendre s equations
Chapter 12 Bessel s and legendre s equations 12.1 Introduction Many linear differential equations having variable coefficients cannot be solved by usual methods and we need to employ series solution method
More informationCHAPTER VIII HILBERT SPACES
CHAPTER VIII HILBERT SPACES DEFINITION Let X and Y be two complex vector spaces. A map T : X Y is called a conjugate-linear transformation if it is a reallinear transformation from X into Y, and if T (λx)
More informationClass notes: Approximation
Class notes: Approximation Introduction Vector spaces, linear independence, subspace The goal of Numerical Analysis is to compute approximations We want to approximate eg numbers in R or C vectors in R
More informationDavid Hilbert was old and partly deaf in the nineteen thirties. Yet being a diligent
Chapter 5 ddddd dddddd dddddddd ddddddd dddddddd ddddddd Hilbert Space The Euclidean norm is special among all norms defined in R n for being induced by the Euclidean inner product (the dot product). A
More informationAn idea how to solve some of the problems. diverges the same must hold for the original series. T 1 p T 1 p + 1 p 1 = 1. dt = lim
An idea how to solve some of the problems 5.2-2. (a) Does not converge: By multiplying across we get Hence 2k 2k 2 /2 k 2k2 k 2 /2 k 2 /2 2k 2k 2 /2 k. As the series diverges the same must hold for the
More informationOverview of Fourier Series (Sect. 6.2). Origins of the Fourier Series.
Overview of Fourier Series (Sect. 6.2. Origins of the Fourier Series. Periodic functions. Orthogonality of Sines and Cosines. Main result on Fourier Series. Origins of the Fourier Series. Summary: Daniel
More information14 Fourier analysis. Read: Boas Ch. 7.
14 Fourier analysis Read: Boas Ch. 7. 14.1 Function spaces A function can be thought of as an element of a kind of vector space. After all, a function f(x) is merely a set of numbers, one for each point
More informationPower Series Solutions to the Legendre Equation
Department of Mathematics IIT Guwahati The Legendre equation The equation (1 x 2 )y 2xy + α(α + 1)y = 0, (1) where α is any real constant, is called Legendre s equation. When α Z +, the equation has polynomial
More informationWaves on 2 and 3 dimensional domains
Chapter 14 Waves on 2 and 3 dimensional domains We now turn to the studying the initial boundary value problem for the wave equation in two and three dimensions. In this chapter we focus on the situation
More informationReconstruction of sparse Legendre and Gegenbauer expansions
Reconstruction of sparse Legendre and Gegenbauer expansions Daniel Potts Manfred Tasche We present a new deterministic algorithm for the reconstruction of sparse Legendre expansions from a small number
More informationCurve Fitting and Approximation
Department of Physics Cotton College State University, Panbazar Guwahati 781001, Assam December 6, 2016 Outline Curve Fitting 1 Curve Fitting Outline Curve Fitting 1 Curve Fitting Topics in the Syllabus
More informationLECTURE 16 GAUSS QUADRATURE In general for Newton-Cotes (equispaced interpolation points/ data points/ integration points/ nodes).
CE 025 - Lecture 6 LECTURE 6 GAUSS QUADRATURE In general for ewton-cotes (equispaced interpolation points/ data points/ integration points/ nodes). x E x S fx dx hw' o f o + w' f + + w' f + E 84 f 0 f
More informationChapter 4. Series Solutions. 4.1 Introduction to Power Series
Series Solutions Chapter 4 In most sciences one generation tears down what another has built and what one has established another undoes. In mathematics alone each generation adds a new story to the old
More informationReal Analysis Problems
Real Analysis Problems Cristian E. Gutiérrez September 14, 29 1 1 CONTINUITY 1 Continuity Problem 1.1 Let r n be the sequence of rational numbers and Prove that f(x) = 1. f is continuous on the irrationals.
More informationUniversity of Houston, Department of Mathematics Numerical Analysis, Fall 2005
4 Interpolation 4.1 Polynomial interpolation Problem: LetP n (I), n ln, I := [a,b] lr, be the linear space of polynomials of degree n on I, P n (I) := { p n : I lr p n (x) = n i=0 a i x i, a i lr, 0 i
More information