Chapter 7: Bounded Operators in Hilbert Spaces

Chapter 7: Bounded Operators in Hilbert Spaces I-Liang Chern Department of Applied Mathematics National Chiao Tung University and Department of Mathematics National Taiwan University Fall, 2013 1 / 84

Outline 1 Examples of Bounded Operators 2 Preliminaries 3 Solving Ax = b 4 Unitary operators 5 Compact operators 6 Fredholm operators 2 / 84

Examples of Bounded Operators We have seen many examples of bounded and unbounded operators in Chapter 2. We will review some and provide more examples. 1 The differential operator: D : u u is a unbounded operator, while its inverse: Ku(x) = x u(y) dy = 1 0 0 g(x, y)u(y) dy is a bounded operator from C[0, 1] to C[0, 1]. Here, g(x, y) = 1 if 0 y < x and 0 otherwise. You can see that it is also a bounded map from L 2 (0, 1) to itself. 3 / 84

2 Consider D 2 u = f on (0, 1) with the boundary condition: u(0) = u(1) = 0. Its inversion can be represented as { 1 u(x) = g(x, y)f(y) dy, g(x, y) = 0 x(1 y) for 0 < x < y < 1 y(1 x) for 0 < y < x < 1 The mapping f u is a bounded linear operator from L 2 (0, 1) to itself (Check by yourself). 3 Consider u = f in R 3 with u(x) 0 as x. Then u is given by u(x) = 1 1 f(y) dy 4π x y The mapping f u is a bounded linear map from L 2 (R 3 ) to L 2 (R 3 ). 4 / 84

4 Blur operator: let Kf(x) = K(x, y)f(y) dy. where K(x, y) is called a kernel. In many cases, K is translational invariant, i.e. K(x, y) = K(x y). In this case, we can represent it as a multiplier in the Fourier space: Kf(ξ) = K(ξ) ˆf(ξ). 5 / 84

5 Radon transform. Given a smooth, compact supported function f in R 2 with support in B 1 (0), given any θ S 2 and s R, define (Rf)(θ, s) = f(sθ + y) dy θ where θ = {y (θ, y) = 0}. The transform R is called the Radon transform. We will see that it is a bounded operator from L 2 (B 1 ) to L 2 (S 1 [ 1, 1]). 6 / 84

6 Hilbert transform: (Hf)(x) := P.V. 1 π R f(y) x y dy. The Fourier representation of Hilbert transform is Ĥf(ξ) = sign(ξ) ˆf(ξ). Let u be the the harmonic function on the upper half plane with Dirichlet data f on y = 0. Let v be the complex conjugate of u. Then v(x, 0+) is the Hilbert transform of f. 7 / 84

7 Potential Theory: Consider the potential equation u = 0 in Ω R 2, Let K(x y) = 1 2π ln x y be the fundamental solution of in R 2. That is, y K(x y) = δ(x y). Now, we multiply y u = 0 by K, integrate it over a domain Ω x,ɛ in y, and perform integration by part, then take ɛ 0. Here, Ω x,ɛ is Ω \ B ɛ (x) and ɛ << 1 so that B ɛ (x) Ω if x Ω. 8 / 84

Then we will get Ω ( K(x y)u n (y) ) K(x y) u(y) dy = u(x). n y (1.1) When x Ω, there is only half of the ball B ɛ (x) lies inside Ω and we get Ω ( K(x y)u n (y) ) K(x y) u(y) dy = u(x) n y 2. (1.2) Here, the data u(x) and u n (x) for x Ω are the limits from inside of Ω. The formula (1.1) provides us a representation of u(x), x Ω o, in terms of the boundary data u(x) and u n (x). On the other hand, (1.2) gives a relation between the Dirichlet data u(x) and Neumann data u n (x) on Ω. 9 / 84

For instance, suppose we are given a Dirichlet data u(x) = f(x) for x Ω. Then we can solve u n from K(x y) K(x y)u n (y) dy = f(y) dy + f(x) n y 2. Ω Ω This is called the Fredholm integral equation of first kind. On the other hand, if we are given Neumann data u n = g on the boundary, then we can find u on the boundary from u(x) 2 + Ω K(x y) u(y) dy = K(x y)g(y) dy. n y Ω This is called the Fredholm integral equation of the second kind. 10 / 84

The mapping: u u n on the boundary is a linear map, called the Dirichlet-to-Neumann map. Similarly, its inverse map is a linear map and will be shown that it is a bounded linear map. We will see that this linear map is a bounded operator from L 2 ( Ω) to itself. 11 / 84

What are our concerns The issues we are concerned are: Soving Au = f for existence, uniqueness and stability. Solving the least square problem: Find min Au f 2. Eigen-expansion of A. 12 / 84

Outline 1 Examples of Bounded Operators 2 Preliminaries 3 Solving Ax = b 4 Unitary operators 5 Compact operators 6 Fredholm operators 13 / 84

Operator norm Let H and K are Hilbert spaces. Let A : H K be a linear operator. We recall that a linear operator A is called bounded if there exists an M such that Ax M x for all x H. It is easy to see that a linear operator A is bounded if and only if it is continuous. For a bounded operator A, we define its operator norm A by Ax A := sup x =0 x. 14 / 84

Kernel and Range We denote the kernel of A by N(A) and range by R(A). From the boundedness of A, we get N(A) is closed. The range of a bounded linear map may not be closed. For example, the mapping Kf = x 0 f(y) dy maps L2 [0, 1] to L 2 [0, 1]. But the range is H 1 [0, 1] with u(0) = 0. Its closure is L 2 [0, 1]. Proposition Let A be a bounded linear map from H to K. The following two statements are equivalent: (i) there exists a constant c > 0 such that Ax c x for all x H; (ii) R(A) is closed, and N(A) = {0}. 15 / 84

Proof (i) (ii). If {Ax n } is Cauchy in R(A), then from assumption Ax c x, we get that {x n } is also Cauchy and converges to x H. The continuity of A implies Ax n Ax. Thus, R(A) is closed. Also, from Ax c x, we get that Ax = 0 implies x = 0. Thus, N(A) = {0}. (ii) (i). Since R(A) is closed, R(A) is a Hilbert space. From A : H R(A) being 1-1 and onto, we get A 1 exists from R(A) onto H. The open mapping theorem states that a bounded linear map from a Banach space onto another Banach space maps open sets to open sets. Thus, we get A 1 is also bounded. That is, there exists a constant c 1 such that any y R(A), A 1 y c 1 y. Or, c 1 Ax x x H. 16 / 84

Example Fredholm operators (A = I + K, K is compact) are typical example of bounded operators that have closed range. For instance, in the space C[0, 1], consider Ku = x u(y) dy. Then Au = f implies 0 u + u = f. By using integration factor, we get u(x) = e x (u(0) f(0)) + f(x) x 0 e x+y f(y) dy. In this expression, we don t need to require f exists. Thus, R(A) = C[0, 1]. 17 / 84

The set of bounded operators The set B(H, K) = {A : H K is a bounded linear operator} forms a normed linear space with the above operator norm. It is indeed a Banach space from the completeness of the space K. For, if {A n } is Cauchy in operator norm, then for any x H, it holds that A n x is also a Cauchy in K and hence converges to a unique point y in K. Thus, we can define Ax = y. We leave the rest of the proof to the readers. 18 / 84

Adjoint operators Let A : H K be a bounded operator. We can define the adjoint operator A : K H by (A y )(x) = y (Ax) for all y K, x H. By the Riesz representation theorem, we can identify each y K as a point y K by y (z) = (y, z) for all z K. In terms of the inner products in H and K, the adjoint operator A : K H is defined as (A y, x) = (y, Ax) for all x H, y K. 19 / 84

The adjoint operator A is well-defined: The linear map l(x) = (y, Ax) is bounded. By Riesz representation theorem, there exists a unique z H such that (y, Ax) = (z, x) for all x H. We then define A y = z. (A ) = A. ((A ) x 1, x 2 ) = (x 1, A x 2 ) = (A x 2, x 1 ) = (x 2, Ax 1 ) = (Ax 1, x 2 ) (AB) = B A. ((AB) x, y) = (x, ABy) = (A x, By) = (B A x, y) If A : H K is a bounded operator, then A : K H is also a bounded operator and A = A : sup A y = y =1 sup sup (A y, x) = sup sup (y, Ax) = sup Ax = A. y =1 x =1 x =1 y =1 x =1 20 / 84

Examples. 1 Let A : C n C m be a linear map with matrix representation A = (a ij ) m n. Then its dual operator A has the matrix representation A = (ā ji ) n m. 2 Let K : L 2 [0, 1] L 2 [0, 1] be Kf(x) = Then K f(x) = 1 0 1 0 k(x, y)f(y) dy k(y, x)f(y) dy If A : H H satisfies (Ax, y) = (x, Ay), i,e, A = A, we call it self-adjoint. 21 / 84

Outline 1 Examples of Bounded Operators 2 Preliminaries 3 Solving Ax = b 4 Unitary operators 5 Compact operators 6 Fredholm operators 22 / 84

Solving Ax = b Theorem Let A : H K be a bounded linear map. It holds (i) N(A ) = R(A), R(A) = N(A ) ; (ii) N(A) = R(A ), R(A ) = N(A) ; (iii) A : H = N(A) R(A ) N(A ) R(A), with A : R(A ) R(A) being one-to-one and onto. 23 / 84

Proof (i) First, we show N(A ) R(A). Suppose y N(A ). For any x H, we have (Ax, y) = (x, A y) = 0. Thus, y R(A), or equivalently, y R(A). Next, we show R(A) N(A ). If z R(A), it means that (z, Ax) = 0 for all x H. This implies (A z, x) = 0 for all x H. Thus, A z = 0. Hence, R(A) N(A ). By taking orthogonal complement, we get N(A ) R(A) = R(A). This completes the proof of (i). 24 / 84

(ii) We apply (i) to A to get N(A ) = R(A ). Taking orthogonal complement and using A = A, we get N(A) = R(A ) = R(A ). (iii) Since A is bounded, we get that N(A) is a closed subspace of H. By the orthogonal projection theorem, H = N(A) N(A). Similarly, A is also bounded. Thus, K = N(A ) N(A ). From (i) and (ii), we have N(A) = R(A ) and N(A ) = R(A). If x R(A ) with Ax = 0, then x N(A) N(A) = {0}. Hence A is 1-1 on R(A ). The onto part for A : R(A ) R(A) is trivial. 25 / 84

Remarks. The necessary condition for the solvability of Ax = b is b N(A ). If R(A) is also closed, then this is also a sufficient condition. In finite dimensions, R(A) is always closed. So in finite dimension case, solvability of Ax = b if and only if b N(A ). Another application example is A = I K, where K is a compact operator. In this case, R(A) is also closed, which will be proved later. So, b N(A ) is a necessary and sufficient condition for the solvability of Ax = b. 26 / 84

In the case of R(A) being closed, in particular, N(A ) = {0} if and only if Ax = b is solvable. Thus, the existence of Ax = b is equivalent to the uniqueness of A y = 0. In general, R(A) may not be closed in the infinite dimensional case. In this case, b N(A ) does not imply the solvability of Ax = b. 27 / 84

Example 1 We consider the multiplication operator M : L 2 [0, 1] L 2 [0, 1] defined by Mf(x) = xf(x). Then M = M. For any f L 2 [0, 1], if Mf = 0, then xf(x) = 0. This implies f(x) = 0 almost everywhere. Hence f = 0 in L 2 [0, 1]. This shows N(M) = N(M ) = {0}. Let g 1 on [0, 1]. g L 2 [0, 1]. But the only function f satisfying xf 1 is 1/x which is not in L 2 [0, 1]. We can conclude that R(A) is not closed. 28 / 84

Example 2. Let Af(x) = x 0 f(y) dy. Then A : L2 [0, 1] L 2 [0, 1]. The range of A are those differentiable function with f(0) = 0. It is clear that R(A) L 2 [0, 1]. The step function H(x 1/2) is not in the range of A. We shall come back to the applications of this theorem after we learn compact operator and singular value decomposition. 29 / 84

Outline 1 Examples of Bounded Operators 2 Preliminaries 3 Solving Ax = b 4 Unitary operators 5 Compact operators 6 Fredholm operators 30 / 84

Unitary operators Definition A linear map U : H K is called unitary (or orthogonal) if it is invertible and (Ux, Uy) K = (x, y) H for all x, y H. In other words, a unitary map is 1-1, onto and preserves inner product. Thus, we have Ux = x. This implies U = 1. Two spaces H and K are called isometric if there is a unitary map between them. We then identify K as H. Proposition (a) A linear map U : H H is unitary if and only if (b) U U = UU = I. 31 / 84

Proof (b) (a). If Ux = 0, then x = U Ux = U 0 = 0. Thus, N(U) = {0}. Similarly, N(U ) = {0}. This shows U and U are 1-1. For any x H, U(U x) = x. This shows U is onto. Finally, (Ux, Uy) = (x, U Uy) = (x, y). This shows U is inner product preserving. This shows (b) (a). (a) (b). From (Ux, Uy) = (x, y), we get (U Ux, y) = (x, y) for all x, y H. This implies U Ux = x for all x. Thus, we have U U = I. This together with the 1-1 and onto property, we get U 1 = U. This leads to UU = I. 32 / 84

Examples of unitary operators Given a self-adjoint matrix A in C n (i.e. A = A ), define U = e ia (ia) n =. n! n=0 Then U is a unitary map in C n. Using (A n ) = (A ) n, we get U ( ia ) n = = e ia. n! n=0 Because A = A, we get AA = A 2 = A A. Thus, A commutes with A. From this, we get UU = e ia e ia = e ia ia = I. 33 / 84

In the above, I have used 1 When A is a matrix, then A is finite in B(C n, C n ). Thus, the series A n n=0 converges absolutely and n! uniformly in B(C n, C n ). 2 If AB = BA, then e A+B = e A e B = e B e A. 34 / 84

Examples of unitary operators Suppose H is a Hilbert space. Let {u n } n N and {v n } n N be two orthonormal bases of H. We define a linear map U on the basis by Uu n = λ n v n, for all n N where λ n C and λ n = 1. Then U is a unitary map. For, if u = n α nu n, w = n β nu n, then ( (Uu, Uw) = n αλ n v n, m β m λ m v m ) = (u, w). 35 / 84

Example Consider the harmonic oscillator is quantum system i t ψ = Hψ, ψ(0) = ψ 0. The Hamiltonian H = d 2 /dx 2 + x 2. Its eigenvalues are λ n = n 2, eigenstates are the Hermite polynomials: u n = H n (x)e x2 /2. They constitute an orthonormal basis in L 2 (R). Its solution ψ(t) := U(t)ψ 0 is U(t)ψ 0 := e iλnt α n u n (x), where ψ 0 = α n u n (x). n=0 n=0 The operator U(t) is a unitary operator. 36 / 84

Example Let H = 1 2 2 + V (x) be the Schrödinger operator. It is a self-adjoint unbounded self-adjoint operator. Formally, the solution of the Schr dingier equation can be expressed as i t ψ = Hψ, ψ(0) = ψ 0 U(t)ψ 0 = e ith ψ 0 Formally, the operator U(t) is a unitary operator in L 2 (R). 37 / 84

Example The translation operator T a : L 2 (T) L 2 (T) defined by (T a f)(x) = f(x a) is unitary. (T a f, T a g) = f(x a)g(x a) dx = f(x)g(x) dx = (f, g). T T The invariant T a f = f for a 0 if and only f is a constant function. The set {T a a T} is a unitary representation of the additive group T. 38 / 84

Example The Fourier transform F : L 2 (R) L 2 (R) is a unitary map. This is because the isometry property of F. The invariant F f = f if and only if f(x) = 1 2π e x2 /2. 39 / 84

Example The Hilbert transform H : L 2 (R) L 2 (R) is defined by (Hf)(ξ) = i ξ ξ ˆf(ξ) is a unitary map. This is because (Hf, Hg) = R ( i ξ ) ( ξ ˆf(ξ) i ξ ) ξ ĝ(ξ) dξ = (f, g). 40 / 84

Example Periodic Hilbert transform: H : L 2 (T) L 2 (T) is defined by (Hf) n = i sign n ˆf n. Or equivalently, He inx = i(sign n)e inx. We see that the weight { 1 when n 0 i sign n = 0 when n = 0. Thus, H(1) = 0 and its kernel is the space spanned by 1, i.e. N(H) = 1. Its orthogonal complement is H = {f L 2 (T) 2π Then, H is a unitary map in H. 0 f(x) dx = 0}. 41 / 84

The mean ergodic theorem. The invariant set of a unitary map: M := {x Ux = x}, which is a closed subspace. The invariant space of the shift map T a is 1. The invariant space of e ith (H = 2 x + x 2 ) is 1. Theorem (von Neumann) Let U be a unitary operator on a Hilbert space H and M be its invariant space. Let P be the orthogonal projection onto M. Then, for all x H, we have lim N 1 N + 1 N U n x = P x. (4.3) n=0 42 / 84

Proof 1 From the definition, we have N(I U) = R(P ) = M. We can decompose H = N(P ) R(P ). 2 If x R(P ), then 1 N + 1 N n=0 U n x = 1 N + 1 Thus, (4.3) holds for x R(P ). N x = x = P x. n=0 3 We notice that Since U is unitary, Ux = x if and only if x = U Ux = U x, i.e. N(I U) = N(I U ) = M. From Theorem 3.1, N(P ) = N(I U) = N(I U ) = R(I U). 43 / 84

4 If x R(I U), then x = (I U)y for some y H. 1 N + 1 N n=0 U n x = 1 N + 1 N (U n U n+1 )y n=0 = 1 N + 1 (y U N+1 y) 0, as N. 5 For any x N(P ), we can find x k R(I U) to approximate x. Hence, 1 N U n x N + 1 1 N U n (x x k ) N + 1 + 1 N U n x k N + 1 n=0 n=0 n=0 1 N x x k + U n x k. N + 1 n=0 By taking N, then k, it follows that (4.3) holds for x N(P ). 44 / 84

Deterministic Dynamical System Consider a discrete dynamical system on a set Ω, described by x n+1 = T x n. We associate a probability measure P on Ω. That is, (Ω, P) is a probability space. The mapping T : Ω Ω is measure preserving: i.e. P(T 1 A) = P(A) for all measurable set A Ω. We shall study statistical behavior of the dynamical system: x n+1 = T x n. 45 / 84

An important quantity is the average, which is measured by 1 N + 1 N f(x n ) n=0 for any continuous function f. However, we may not have topology on Ω. Therefore, alternative way is 1 N + 1 N f T n. n=0 The mapping T then induces an operator U on L 2 (Ω, P) by f f T. 46 / 84

Since T is measure preserving, we have f T (x) g T (x) dp(x) = f(x) g(x) dp(x). Ω That is, Uf, Ug = f, g. Thus, U is unitary. Definition A 1-1, onto, measure preserving map T on (Ω, P) is ergodic if the only function f L 2 (Ω, P) such that f = f T are the constant functions. Example: The shift operator T a f(x) = f(x + a) is ergodic. Ω 47 / 84

von Neumann ergodic theorem Theorem Let T : Ω Ω be a 1-1, onto, measure preserving on a probability space (Ω, P). Assume T is ergodic on Ω. Then for any f L 2 (Ω, P), in L 2 (Ω, P). 1 N + 1 N f T n n=0 Ω f(x) dp(x) 48 / 84

Proof 1 The definition states that T is ergodic if the invariant of U is the constant functions, which is spanned by the constant function f 1. 2 Then the von Neumann ergodic theorem implies that: 1 N + 1 N f T n f, 1 1, as N, n=0 where f, 1 = f(x) dp(x). Ω 49 / 84

Remarks Since L 2 convergence implies convergence almost everywhere, we obtain that 1 ( f(x 0 ) + f(x 1 ) + + f(x N ) ) N + 1 for almost all x 0 Ω with x n+1 := T x n. Weyl s ergodic theorem states that: If the shift γ is irrational, then f t (x 0 ) = f s for all f C(T) and for all x 0 T. Ω f(x) dp(x) 50 / 84

Remarks The general ergodic theorem holds for f L 1 (Ω, P), due to Birkhoff. If T is only measure preserving, then the limit lim N 1 N + 1 N f(t n x 0 ) n=0 still exists for almost all x 0 Ω. For general theory, you can visit Wiki: ergodic theory. 51 / 84

Outline 1 Examples of Bounded Operators 2 Preliminaries 3 Solving Ax = b 4 Unitary operators 5 Compact operators 6 Fredholm operators 52 / 84

Compact Operators Definition A linear map A : H K is compact if it maps bounded set into a precompact set in K. A Typical Example: 1 Inverse Laplacian: Consider the Poisson equation u = f in a bounded domain Ω R n with Dirichlet boundary condition. The solution exists uniquely by the Riesz representation theorem and the mapping K = ( ) 1 maps f to u is a bounded operator from L 2 (Ω) to H0 1(Ω). Since H0 1(Ω) is compactly embeded into L2 (Ω), we get K is a compact operator from L 2 (Ω) to L 2 (Ω). 53 / 84

Examples The inverse Laplacian Kf has an integral representation: u(x) = G(x, y)f(y) dy G is called the Green function. G satisfies (a) G(x, y) = G(y, x) (b) G(, y) = δ( y), (c) G(x, y) = 0 for x Ω. Ω The Green function has the following unique representation G(x, y) = 1 1 + h(x, y) 4π x y where h(x, y) = h(y, x), h(, y) is harmonic in Ω and h(x, y) = 1 1 for x Ω. 4π x y 54 / 84

The Newtonian potential N f(x) := Ω 1 1 f(y) dy 4π x y is a compact operator in L 2 (Ω). For 1 x N(x y) = O(1) x y. 2 and In R 3, x N f 2 Ω N x (x) dx f 2 N x (x) dx <. Ω 55 / 84

Examples 2 Integral operators: Let Ω be a bounded domain. Let Ku(x) := k(x, y)u(y) dy. If Ω Ω k x(x, y) 2 dy dx <, then K : L 2 (Ω) H 1 (Ω). Ω Since H ( Ω) L 2 (Ω), we obtain that K is a compact operator in L 2 (Ω). However, we shall show later that we only need k(x, y) 2 dy dx < to ensure K being compact. Ω Ω 56 / 84

Examples 3 A bounded operator with finite dimensional range is compact. Let {φ n } be an orthonormal basis in L 2 (Ω). Suppose λ n be a sequence which converges to 0. Define K N f(x) = where N λ n (f, φ n )φ n (x) = n=1 K N (x, y) = Ω N λ n φ n (y) φ n (x). n=1 K N is a compact operator. K N (x, y)f(y) dy. Remark: Indeed, K N converge uniformly to a compact operator K. 57 / 84

4 The blur operator Ku = k(x y)u(y) dy is a compact operator. 5 The Radon transform is a compact operator. Ω 58 / 84

6 A diagonal operator A : l 2 (N) l 2 (N) defined by K(x 1, x 2, x 3, ) = (λ 1 x 1, λ 2 x 2, λ 3 x 3, ) is compact if and only if λ n 0 as n. Proof. We only need to show that KB 1 (0) is pre compact in H. For any ɛ > 0, we can find an N such that λ n < ɛ for all n N. For any x B 1 (0), we have (Kx) n 2 ɛ 2 x 2 ɛ. n=n Thus, K(B 1 (0)) is precompact. 59 / 84

Proposition (a) Let A : H 1 H 2 and B : H 2 H 3 be bounded linear operators. If one of them is compact, then AB is also compact. (b) Let A n : H H be compact operators and A n A uniformly. Then A is also compact. 60 / 84

Proof (a) The key is that a bounded operator (which is continuous) maps a compact set into a compact set. (b) 1 We use diagonal process to prove (b). 2 Let (x n ) be a bounded sequence in H. We claim that we can find a convergent subsequence of (Ax n ). 3 Since A m are all compact. We start from m = 1, the sequence (A 1 x i ) is bounded and hence has a convergent subsequence (A 1 x 1,n ). 4 The sequence A 2 x 1,n is again bounded, thus we can find subsequence (x 2,n ) of (x 1,n ) such that A 2 x 2,n converges. 61 / 84

5 We continuous this process to get a subsequence (x m,n ) of (x m 1,n ) with (A m x m,n ) n N converges. 6 Then we take the subsequence (x m,m ) which has the property: for every k N, the sequence (A k x m,m ) m N is a Cauchy sequence. 7 We claim that the sequence (Ax m,m ) is also a Cauchy sequence. For any ɛ > 0, we can find k large enough such that A A k < ɛ.with this k, we can find N such that for n, m N, A k x m,m A k x n,n < ɛ. Combining these two, we get Ax m,m Ax n,n < 3ɛ. This shows Ax m,m is a Cauchy sequence, hence Ax n has convergent subsequence. 62 / 84

The Adjoint of a Compact Operator Theorem (Schauder) If K : H K is compact, then so is its dual K. Proof: Suppose (y n ) is a sequence in K with y n 1. We want to show (K y n ) has convergent subsequence. 63 / 84

1 Let B be the unit ball in H and D = KB. From compactness of K, we get that D is compact. 2 Now, (y n ) is not only continuous on D, in fact, they are equi-continuous on D because their norms are bounded by 1. More precisely, we consider F = {y n } C(D). For any y n F, y n (z 1 ) y n (z 2 ) = (y n, z 1 z 2 ) z 1 z 2 because y n 1. 3 By Arzela-Ascoli theorem, (y n ) has a subsequence, still denoted by (y n ), which converges on D. That is, for any small ɛ > 0, we have sup (y n y m, z) < ɛ, if m, n are large enough. z D 64 / 84

4 But this is the same as sup (y n y m, Kx) = x 1 sup (K (y n y m), x) = K y n K y m < ɛ x 1 Hence, (K y n ) is a Cauchy sequence in H. This shows (K y n ) has Cauchy subsequence. 65 / 84

Hilbert-Schmidt operators Typical examples of compact operators are the inversions of elliptic operators. They can be expressed as an integral operator. More general classes are, for example Hilbert-Schmidt Operators Trace class. Here, we discuss the Hilbert-Schmidt class. 66 / 84

Definition of Hilbert-Schmidt Operators Let Ω be a measurable set in R d. Define the integral operator in L 2 (Ω): Ku(x) = with Ω Ω Ω k(x, y)u(y) dy. k(x, y) 2 dy dx <. Then, by Cauchy-Schwarz, K : L 2 (Ω) L 2 (Ω) a bounded operator. ( Ku(x) 2 dx Thus, K 2 Ω Ω Ω ) ( ) k(x, y) 2 dy dx u(y) 2 dy Ω Ω k(x, y) 2 dy dx 67 / 84

Hilbert-Schmidt operators are compact Such operator is called a Hilbert-Schmidt operator. We define the Hilbert-Schmidt norm of K by K 2 HS = k(x, y) 2 dy dx. Ω Ω We claim that Hilbert-Schmidt operators are compact. 68 / 84

Proof 1 Let {e n (y)} be an orthonormal basis in L 2 (Ω). Since k L 2 (Ω Ω), we have for almost x Ω, k(x, ) L 2 (Ω) and we can expand k(x, y) as k(x, y) = a n (x)e n (y) n 2 By Parseval equality k(x, ) 2 = n a n (x) 2 We integrate in x to get k(x, y) 2 dy dx = a n (x) 2 dx Ω Ω n Ω Here, dominant convergence theorem is used. 69 / 84

3 Now, we approximate the operator K by K N u(x) = Ω k N(x, y)u(y) dy with k N (x, y) = n N a n (x)e n (y). Clearly, K N are of finite-dimensional range, thus compact. 4 We claim that K N converges to K uniformly. For K K N 2 k(x, y) k N (x, y) 2 dy dx Ω Ω = a n (x) 2 dx 0. n>n Ω 5 Since all K N are compact operators, we get that K is also a compact operator. 70 / 84

Outline 1 Examples of Bounded Operators 2 Preliminaries 3 Solving Ax = b 4 Unitary operators 5 Compact operators 6 Fredholm operators 71 / 84

Applications: Fredholm Operators In applications, we encounter the operator T := I K, where K is a compact operator, frequently. Indeed, K is treated as a perturbation relative to the identity operator. Below, we analyze its kernel and range. 72 / 84

Example: Helmholtz equation Consider the wave equation u tt = c 2 (x)u xx with, say Dirichlet boundary condition We consider the time-harmonic solution u(x, t) = v(x)e iωt. Then v satisfies v xx + ω2 c 2 (x) v = 0. This is called Helmholtz equation. If there is a time-harmonic source, we are leading to v xx + n(x) 2 v = f 73 / 84

We can invert xx with proper boundary condition to get or v(x) + 1 0 v(x) + G(x, y)n(y) 2 v(y) dy = g(x). 1 0 k(x, y)v(y) dy = g(x). 74 / 84

Fredholm Alternative Theorem Let K be a compact operator in a Hilbert space H. Let T = I K. The following statements hold. (i) N(T ) is finite dimensional. (ii) There is an integer m such that N(T k ) = N(T m ) for all k m. (iii) R(T ) is closed. 75 / 84

Proof of (i) 1 If N(T ) is not finite dimensional, then we can construct an orthonormal set {e n } n N in N(T ). 2 Since T = I K and T e n = 0, we have e n = Ke n. 3 From K being compact, (Ke n ) n N, hence (e n ) n N, is precompact. But this is impossible because (e n, e m ) = 0 for all m n. 76 / 84

Proof of (ii) 1 First, we show that: if N(T m+1 ) = N(T m ), then N(T m+2 ) = N(T m+1 ). It is obvious that N(T m+1 ) N(T m+2 ). Conversely, if x N(T m+2 ), then T x N(T m+1 ). By our assumption N(T m+1 ) = N(T m ). Hence T m (T x) = 0. Thus, we get x N(T m+1 ). 2 Next, we show that there exists an m such that N(T m+1 ) = N(T m ). 77 / 84

If for every m N, N(T m ) is a proper subspace of N(T m+1 ), then for all m N, we can find unit vector e m+1 N(T m ) and e m+1 N(T m+1 ). Then (e m ) m N is an orthonormal set. Take m < n, from T = I K, we have Ke n Ke m = e n T e n e m + T e m. Claim: T e n, e m, T e m N(T n 1 ). (a) e m N(T m ) N(T n 1 ); (b) e n N(T n ) implies T e n N(T n 1 ); and (c) T e m N(T m 1 ) N(T n 1 ). Hence, Ke n Ke m = e n (T e n + e m T e m ) 1 for all m < n. This contradicts to the compactness of K. 78 / 84

Proof of (iii): R(I K) is closed. 1 Suppose (y n = (I K)x n ) is a convergent sequence in R(I K). We want to show that there is an x and a subsequence of (x n ) which converges to x. 2 Let us first assume N(T ) = {0}. Suppose x n is unbounded. Pick up the subsequence, still call it (x n ), whose norm tends to. Consider z n = x n / x n. We have y n x n = z n Kz n. Since y n is a convergent sequence, y n / x n 0. (z n ) is a bounded sequence now, hence Kz n has a convergent subsequence, still denote it by (z n ), which converges to z. 79 / 84

3 Then we have 0 = z Kz. Hence z N(T ). By our assumption, z = 0. But z is the limit of z n and z n = 1 for all n. This is impossible. Hence the assumption x n is unbounded is impossible. 4 Now (x n ) is a bounded sequence. Hence (Kx n ) has convergent subsequence, still denote by Kx n. From y n = x n Kx n, we see both (y n ) and (Kx n ) converge. Thus, (x n ) also converges. 80 / 84

5 Finally, we do not make the assumption N(T ) {0}. Let z k be the projection of x k on N(T ). Consider w k = (x k z k ) N(T ). Then y k = T x k = T (x k z k ) = T w k. We now replace H by N(T ). By the previous argument, we get (w k ) has a subsequence converges to w N(T ). Hence y = lim y k = lim T w k = T w. This shows that R(T ) is closed. 81 / 84

Application: Fredholm Alternative From Schauder s theorem, K is compact if and only if K is compact. From duality principle and (iii), we get N(T ) = R(T ) and N(T ) = R(T ). Theorem (Fredholm Alternative) Let T = I K. Then one of the following is true: (a) either T u = f has a solution for every f H, (b) or T v = 0 has a nontrivial solution. The statement (a) is R(T ) = H, which is equivalent to N(T ) = {0}. The statement (b) is R(T ) {0}. In applications, if N(T ) {0}, then the solvability for T u = f is f N(T ). 82 / 84

Applications: Least-Squares Solutions Let K be a compact operator in H and let T = I K. Consider the equation T u = f. Suppose f N(T ), we look for a solution which minimizes T u f 2. Such a solution is called the least-squares solution. We can decompose H in the domain and range as the follows: T : N(T ) R(T ) N(T ) R(T ). This is because both R(T ) and R(T ) are closed. T is bounded bijection R(T ) R(T ). 83 / 84

Given f H, we decompose f = f + f, with f R(T ) and f N(T ). With f, there exists a unique u R(T ) such that T u = f. For any u H, we claim that T u f 2 T u f 2. We can decompose u = v + u, with v R(T ) and u N(T ). Then T u = T v. We have T u f 2 = T v f 2 + f 2 f 2 = f 2 + T u f 2 = T u f 2. This shows that u is a least-squares solution. Notice that the solution set is u + N(T ). 84 / 84