ME(EE) 550 Foundations of Engineering Systems Analysis Chapter 05: Linear Transformations and Functionals

Transcription

1 ME(EE) 550 Foundations of Engineering Systems Analysis Chapter 05: Linear Transformations and Functionals The concepts of normed vector spaces and inner product spaces, presented in Chapter 3 and Chapter 4, respectively, synergistically combine the topological structure of metric spaces and the algebraic structure of vector spaces. Now we present linear transformations between such spaces, where these linear transformations form a vector space in their own right. We also introduce the concept of a norm in the space of linear transformations. This chapter should be read along with Chapter 4 and Chapter 5 of Naylor & Sell. Specifically, some of the solved examples and exercises in Naylor & Sell would be very useful very useful. 1 Basic concepts Definition 1.1. (Transformations, Operators, and Functionals) Let V and W be two vector spaces (not necessarily of the same dimension) defined over the same field F (which is either R or C). Then, (i) A mapping f : V W is called a transformation from V into W. (ii) A mapping f : V V is called an operator from V into itself. Hence, an operator belongs to a specific class of transformations. (iii) A mapping f : V F is called a function from V into its field. Hence, a functional belongs to a specific class of transformations. Furthermore, ifthe mappingf is linear, i.e., iff(α x V y) = α f(x) W f(y) x,y V and α F, then these mappings are respectively called a linear transformation, a linear operator, or a linear functional. The collection of all linear transformations from V into W forms a vector space, denoted as L(V,W), over the field F. Example 1.1. let V = F n and W = F m for some n,m N. Then, the linear transformation A : V W is an (m n) matrix, i.e., A F m n. Example 1.2. let V = P(F), where P(F) denotes the space of polynomials of any degree with coefficients in F. Then, the linear mapping A : V V is an infinite-dimensional operator. 1

2 Example 1.3. let V = P(R). Then, the mapping f : V R is a functional. For example, the norm in a normed vector space is a functional; however, the norm is not a linear functional. Definition 1.2. (Injectivity, surjectivity, and bijectivity of transformations) Let V and W be two vector spaces (not necessarily of the same dimension) defined over the same field F. Let T : V W be a transformation. Then, (i) T is called one-to-one or injective if ( T(x) = T(y) ) ( x = y ) x,y V. If T : V W is injective, then its left inverse is S : W V such that TS = I V. (ii) T is called onto or surjective if the range space of T is equal to W, i.e., if z W x V such that T(x) = z. If T : V W is surjective, then its right inverse is S : W V such that TS = I W. (iii) T is called bijective if T is both injective and surjective. In that case, there exists a unique inverse of T, denoted as T 1 : W V that is also bijective, and T 1 T = TT 1 = I. Definition 1.3. (Null space) Let L : V W be a linear transformation, then the null space of L is defined as } N(L) {x V : Lx = 0 W Lemma 1.1. (Injectivity of null spaces) Let L : V W be a linear transformation and let A L(V,W)). A is injective if and only if N(A) = {0 V }. Proof. To showthe ifpart, let N(A) = {0 V }. Then, A(x y) = 0 W (x y) = 0 V or x = y. So, Ax = Ay x = y, which implies A is injective. Next, to show the only if part, let N(A) {0 V }. Then, z 0 V such that Az = 0 W. Now, z = (x y) A(x y) = 0 W Ax = Ay with x y, which implies A is not injective. Definition 1.4. (Boundedness of transformations) Let T : V W be a transformation (not necessarily linear), where V and W be normed vector spaces (with possibly different norms) over the same field F. Then T is defined to be bounded if M (0, ) such that T(x) W M x V x V and M is called a bound of the transformation T. Remark 1.1. Let T : V W be a transformation (not necessarily linear), where V and W are normed vector spaces (with possibly different norms) over the same field F. Then T is unbounded if x V such that T(x) W > M x V M (0, ) 2

3 Remark 1.2. Let V and W be two vector spaces (not necessarily of the same dimension) defined over the same field F. Then, the collection of all bounded linear transformations L : V W forms a vector space, denoted as BL(V,W), over the field F that is a subspace of L(V,W). Definition 1.5. (Induced norm of bounded linear transformations) Let A BL(V,W). Then, the induced norm of A is defined as: { } A ind inf M (0, ) : Ax W M x V x V Proposition 1.1. (Equivalence of induced norms) Let A BL(V, W). Then, the following statements are equivalent. (i) If dimv > 0, then A ind = sup x 0V Ax W x V. (ii) A ind = sup x V =1 Ax W. (iii) A ind = sup x V 1 Ax W. Proof. (i) For x 0 V, it follows that Ax W x V M sup x V =1 Ax W = inf x V { M (0, ) : Ax W M x V }. (ii) Let y 0 V. Define x y y x = 1 y 0 V, which implies that A ind = sup x V =1 Ax W. (iii) Let z = αx, where x V = 1, and α 1 for anarbitraryα F. Then, z V = α x V 1,whichimpliesthatsup z V Az W = sup ( x V =1, α 1)( α Ax W ) = sup x V =1 Ax W = A ind. Theorem 1.1. (Equivalence of continuity and boundedness for linear transformations) Let A L(V,W). Then A is continuous if and only if A is bounded. Proof. To show the if part, let A be bounded, i.e., M (0, ) such that Az W M z V z V. Let x,y V and z = x y. Then, since Ais linear, Az = Ax Ay. For continuity of A, we must show that x V ε > 0 δ(ε,x) > 0 such that ( x y V < δ ) ( Ax Ay W < ε ). This achieved by choosing δ = ε M. Next, to show the only if part, let A be continuous. Let us consider a sequence {x k } of non-zero vectors in V converging to 0 V. Let us define z k xk k x ; since A is linear and continuous, {Az k } must converge to 0 V as k. Now, if A is unbounded, then k N x k V such that Ax k W > k x k V, which implies that Az k W = Axk k x k V = Ax k W k x k V > 1 k N. This is a contradiction because {Az k } must converge to 0 V as k. Therefore, A must be bounded. Corollary 1.1. (Boundedness of finite-dimensional linear transformation) If V is finite-dimensional, then every A L(V, W) is continuous. 3

4 Proof. Let dimv = n for some n N and let {e k : k = 1,,n} be a basis of V, where the vector e j consists of all zero elements except 1 being in the j th position. Since A is linear, it follows that Ax W = A n α k e k W = n ( n α k Ae k W ) α k max Ae k W k Since the vectors e k s are linearly independent, it follows from Lemma 1.2 in Chapter 3 (on linear combination in normed spaces) (see also Kreyszig pp ) that there exists c (0, ) such that n α k e k V n c α k for every choice of α k s Therefore, Ax W 1 c max k Ae k W x V A is bounded. Then, by Theorem 1.1, A is continuous. Corollary 1.2. (Continuity of a linear transformation at a point) Let V and W be normed vector spaces over the same field and let a linear transformation A L(V,W) be continuous at a point y V. Then, A is bounded on V. Proof. Let {x k } be a convergent sequence in V such that x k x V. Then, Ax k Ax W = A(x k x) W = A(x k x + y) Ay W. Then, as k, it follows that (x k x+y) y. Since A is continuous at y V, it follows that A(x k x+y) Ay as k. Therefore, Ax k Ax. Hence, A is continuous on V implying that A is bounded on V. Corollary 1.3. Let A L(V,W) bounded. Let {x k } be a sequence in V that converges to x V. Then (i) The sequence {Ax k } in W converges to Ax W. (ii) The null space N(A) is closed in V. Proof. Part (i) Since A is bounded, A is continuous by Theorem 1.1, the image of a convergent sequence under a continuous mapping is also a convergent sequence by Theorem in Naylor & Sell (see p.74). This is also seen from the following. Ax k Ax W = A ( x k x ) W A ind x k x V Part (ii) Let {x k } be a Cauchy sequence in N(A) that converges to x N(A). Then, it follows from Part (i) that Ax k Ax. Since A is continuous and Ax k = 0 W k, we conclude that Ax = 0 W x N(A). Therefore, N(A) is closed in V. 4

5 Remark 1.3. Let V and W be two vector spaces (not necessarily of the same dimension) defined over the same field F. Then, the vector space L(V,W) of linear transformations from V into W must be bounded if V is finite-dimensional, regardless of whether W is finite-dimensional or not. However, if V is infinitedimensional, then L(V,W) may or may not be bounded, regardless of whether W is finite-dimensional or not. Example 1.4. (Unbounded transformation) Let P [0,1] be the space of all real polynomials on [0,1] with the L -norm as the metric. Let D d dt be a transformation D : P [0,1] P [0,1]. It is concluded that D is a linear transformation based on the fact that D(p 1 +αp 2 ) = d( p 1 (t)+αp 2 (t) ) dt = dp 1(t) dt = Dp 1 +αdp 2 p 1,p 2 P [0,1] α R +α dp 2(t) dt Now we show that D is an unbounded transformation. let x k (t) = t k k N. Then, Dx k = d(tk ) = kt k 1 = kx k 1 dt Therefore, Dx k L = kx k 1 L = k x k 1 L. Since x k L = 1 k N, it follows that Dx k L = k x k L and there is no upper bound on k N. Therefore, D is unbounded. It is concluded from Theorem 1.1 that D is a discontinuous transformation. Discontinuity of the derivative operator has been demonstrated earlier from the ε δ perspective. Proposition 1.2. (Bounded inverse) Let A BL(V,W) be a bounded linear transformation. Then, A is invertible on R(A) and A 1 L(R(A),V) is bounded if and only if there exists a lower bound M (0, ) such that Ax W M x V x V. Proof. To prove the if part, let there exist a lower bound M (0, ) such that Ax W M x V x V. Now, to show that A 1 exists on R(A), it suffices to demonstrate that A is injective, i.e., N(A) = {0 V }. Let x N(A), i.e., Ax = 0 W. Since A is given to be bounded, it follows that 0 = Ax W M x V. Since M > 0, it follows that x V x = 0 V. Therefore, N(A) = {0 V }. Next, we show that A 1 L(R(A),V) is bounded. Let u,v A and let u = Ax and v = Ay. Then, linearity of A 1 follows from the fact: α,β F A 1 (αu+βv) = A 1 (αax+βay) = αa 1 Ax+βA 1 Ay = αx+βy = αa 1 u+βa 1 v We proceed as follows to show that A 1 is bounded, u W = Ax W Ax W M x V = M A 1 u V A 1 u V 1 M u W 5

6 To show the only if part, let A 1 exist on R(A), and let A 1 BL(R(A),V). Then there exists m (0, ) such that x V x V = I V x V = A 1 Ax V m Ax W By setting β = 1 m, it follows that Ax W M x V x V. 2 Linear Bounded Functionals and Dual Spaces Let us recall that if V is a vector space over the field F, then f : V F is called a functional. In general, a functional belongs to the class of transformations. This section focuses on the space of bounded linear functionals on a vector space V, which is called the dual space V. The concept of dual spaces is very important for understanding even the rudimentary aspects of the optimization theory. We present a few examples of linear functionals. Example 2.1. On R n, a linear functional is expressed as: f(x) = n α kξ k, where α k R and x = [ξ 1 ξ n ] T f(x) = a T x. Example 2.2. On L 2 [0,1]on the complex field C, alinearfunctional f is expressed as: f(x) = 1 0 dt y(t)x(t), where y L 2[0,1] is given. The notions of boundedness and norm of a functional follow those of a transformation. Definition 2.1. (Norm of a bounded functional) Let V be a vector space over the field F. Then, a functional f : V F is defined to be bounded if M (0, ) such that f(x) M x V x V. The norm of a bounded functional f : V F is defined as: f sup x V =1 f(x). Definition 2.2. (Dual space) Let V be a vector space over the field F. Then, the dual space of V, denoted as V, is the vector space of all linear bounded functionals on V, i.e., V {f L(V,F) : M (0, ) such that f(x) M x V x V}. Remark 2.1. Every bounded linear functional on a normed space (V, ) is continuous by Theorem 1.1 based on the fact that every functional is a transformation. However, note that all linear functionals are not bounded as seen below. Example 2.3. (An example of a linear unbounded functional) Let us consider a subspace U of the space l overthe real field R, in which each sequence has finitely many non-zero elements. Let us define a linear functional f : U R such that f(x) = N n kξ nk, where N N and the (finitely many) non-zero elements of the sequence x U are ξ n1,ξ n2,,ξ nn. Although N N, there is no upper bound on N and hence the linear functional f is unbounded. 6

7 Theorem 2.1. (Completion of dual spaces) Let V be a normed space over a (complete) field F. Then, its dual space V is a Banach space. Proof. Let {z k } be a Cauchy sequence in V. For any x V, {z k (x)} is a Cauchy sequence of scalars because z k (x) z l (x) z k z l x V and z k z l 0 as k,l. Since the field F is complete, the Cauchy sequence {z k (x)} of scalars converges to a scalar z(x) F. that is, z k (x) z(x) x V. We need to show that the functional z is linear and bounded. Linearity of z is established as follows. ( ) z(αx+βy) = lim k zk (αx+βy) = lim αz k (x)+βz k (y) = αz(x)+βz(y) k Since z k is continuous (because it is bounded), the sequence {z k (x)} converges to a continuous functional z. So, z V. Hence, V is a Banach space. Theorem 2.2. (Dual space of R n ) The dual space ( R n) is isometrically isomorphic to R n with Euclidean norm. ( n ) 1 Proof. Let x = [ξ 1 ξ n ] T R n, where n N and let x ξ k 2 2. Let f (R n) n be expressed as f(x) = η kξ k, where η k R, which is a linear combination of ξ k s. Therefore, f is linear. Furthermore, f is bounded because f(x) = n ( n η k ξ k η k 2) 1( n ( n 2 ξ k 2)1 2 = η k 2 x 2)1 < If we choose x = [η 1 η n ] T, then f(x) = n η k 2 by equality in the Cauchy- ( n ) 1 Schwarz sense. That is, f = sup x =1 f(x) = η k 2 2 = x. Theorem 2.3. (Dual Space of l p ) Let p (1, ) and q be its conjugate, i.e., 1 p + 1 q = 1. Then, dual space l p is isometrically isomorphic to l q. Proof. Let {e k } be a Schauder basis for l p, where {e k } δ kj. Then, every x l p has a unique representation x = ξ ke k, where x {ξ 1 ξ 2 ξ 3 }. Let f l p. Since f is linear and bounded, it follows that ( f(x) = f ξ k e k) = ξ k f(e k ) = ξ k η k by defining η k f(e k ). Let us denote y {η 1 η 2 η 3 }. Let a sequence {x n } in l p be defined as x n {ξ n k }, i.e., ξn k p <, such that { q ηk ξk n = η k if k n and η k > 0 0 if k > n or η k = 0 7

8 It follows from the constraint 1 p + 1 q = 1 that (q 1)p = q. By substituting the expression for ξk n in f(x), it follows that f(x n ) = n η k q and, from the property of the induced norm f, f(x n ) f x n ( ) 1 = f ξn k p p ( n ) 1 = f η k (q 1)p p ( n ) 1 = f η k q p Combining the above equations, it follows that n η k q f ( n η k q) 1 p Dividing both sides of the above equation by n η k q and making use of the identity 1 1 p = 1 q, it follows that ( n η k q) 1 1 p ( n = η k q) 1 q f Since the positive integer n in the above equation can be arbitrarily large, by letting n it follows that ( η k q) 1 q f y = {η k } l q and y lq f. ( )1 To establish the equality y lq = η k q q = f, it is necessary to ( )1 show that η k q q f. By Hölder inequality, it follows that f(x) = j=1 ξ j f(e j ) = j=1 ξ j η j x lp y lq f(x) Therefore, x 0, x lp y lq f y lq. Hence, by combining the inequalities, it follows that f = y lq. The mapping l q l p, defined by y f is linear and surjective, and the linear span of the vectors in the Schauder basis {e k } is dense in l p ; furthermore, this mapping is norm-preserving. Therefore, the dual space l p is isometrically isomorphic to l q. 8

9 Theorem 2.4. (Dual Space of l 1 ) The dual space l 1 is isometrically isomorphic to l. Proof. Let {e k } be a Schauder basis for l p, where {e k } δ kj. Then, every x l 1 has a unique representation x = ξ ke k, where x {ξ 1 ξ 2 ξ 3 }. Let f l 1. Since f is linear and bounded, it follows that ( f(x) = f ξ k e k) = ξ k f(e k ) = ξ k η k by defining η j f(e j ), which are uniquely determined by f. Let us denote y {η 1 η 2 η 3 }. Since e k l1 = 1, it follows that sup k η k = f(e k ) f η k f and y l To establish the equality y l = sup k η k = f, it is necessary to show that sup k η k f. Therefore, f(x) = ξ k η k x l1 sup k η k f(x) Therefore, x 0, x l1 sup k η k = y l, which implies f y l. Hence, by combining the inequalities, it follows that f = y l. he mapping l l 1, defined by y f is linear and surjective, and the linear span of the vectors in the Schauder basis {e k } is dense in l 1 ; furthermore, this mapping is norm-preserving. Therefore, the dual space l 1 is isometrically isomorphic to l. Theorem 2.5. (Dual Space of c o ) The dual space c o to l 1. is isometrically isomorphic Proof. It is known that c o is a closed subspace of the complete space l and c o is complete relative to the metric induced by the norm l. Let f c o. Since f is linear and bounded, it follows that ( f(x) = f ξ k e k) = ξ k f(e k ) = ξ k η k by defining η j f(e j ), which are uniquely determined by f. Let us denote y {η 1 η 2 η 3 }. Then, due to boundedness of the linear functional f, it follows that f sup x =1 η k ξ k < η k < 9

10 by choosing x = {ξ k } with ξ k = 1 k. Hence, y = {η k } l 1 and f y l1. Next we establish the equality that is trivial if y = 0 l1 implying that f = 0 c 0. So, we assume that y 0 l1. Given ǫ > 0 n N such that y ǫ n 2 < η k = f(z) f wherethevectorz c o hasallzerocoordinatesafterthen th and,forj = 1,2,,n, z j = yj y j if y j 0 and z j = 0 y j = 0. As ǫ 0, n, and hence the equality f = y l1 is established. Bijectivity between l 1 and c o is established in the same way as between l q and l p in Theorem 2.3. Remark 2.2. The dual space l of l, which has a rather abstract concept, is not commonly encountered in the engineering discipline; it may occasionally come up in analytic number theory. Note that l l 1. Theorem 2.6. (Riesz Representation Theorem for Linear Fuctionals in a Hilbert Space) Let H be a Hilbert space over a (complete) field F and H be its dual space. Then, every vector in H uniquely identifies a vector in H, i.e., f H a unique y H such that f(x) = x,y H x H,and f ind = y H (See Naylor & Sell, p. 345.) Proof. If f = 0 H, i.e., if f(x) = x,y H = 0 x H, then y = 0 H. Therefore, we assume f 0 H, i.e., x H such that f(x) = x,y H 0 for some y 0 H. Since the null space N(f) {x H : f(x) = 0} is a closed subspace of H, it follows that N(f) N (f) = H and dim ( N (f) ) = 1 because f : H F. Now, the orthogonal projection of x H onto the one-dimensional space N (f) is f(x)z for some z N (f) where z 0 H, which implies that ( x f(x)z ) N(f), i.e., ( x f(x)z ),z H = 0 f(x) = x,z H z 2 In other words, the projection of x onto N (f) is f(x)z = x,u u where u z z is the unique unit vector in the one-dimensional space N (f). Notice that the vector u that spans the space N (f) is independent of the choice of x; however, u is dependent on the choice off. By setting y z z 2, we havef(x) = x,y H x H. Thus, existence of y H such that f(x) = x,y H x H is established. To showuniquenessofy, let thereexist ỹ H such that f(x) = x,ỹ H x H. Then, x,y H x,ỹ H = f(x) f(x) = 0 x,(y ỹ) H = 0 x H y = ỹ. Thus, uniqueness of y H is established. Finally, to show that f ind = y H, we proceed as follows. y 2 H = y,y H = f(y) = f(y) f ind y H f ind y H 10

11 where the trivial case of y = 0 H is excluded for which f ind = 0 and y H = 0. By Cauchy-Schwarz inequality it follows that f(x) x H, f(x) = x,y H x H y H sup y H f ind y H x H 0 x H Hence, f ind = y H. 3 Hahn-Banach Theorem The Hahn-Banach theorem is an extension of linear functionals and has many applications. It allows manipulation of normed spaces and associated bounded linear functionals, and also provides an adequate theory of dual spaces. Specifically, it states that a bounded linear functional on a subspace of a normed vector space can be extended to a bounded linear functional on the entire space with the same norm. 3.1 Zorn s Lemma Zorn s Lemma is necessary for proving the Hahn-Banach Theorem and it has also other applications. We introduce the concept of Zorn s Lemma. Definition 3.1. A partially orderedset, abbreviated as poset, is a set S on which a binary relation, known as partial ordering and denoted as, satisfies the following conditions for every α,β,γ S: α α (Reflexivity) If α β and β α, then α = β (Antisymmetry) If α β and β γ, then α γ (Transitivity) Definition 3.2. Two elements α and β of a partially ordered set are called comparable if they satisfy the condition α β or β α or both; two elements are called incomparable for which neither α β holds nor does β α. Definition 3.3. A totally ordered (also called linearly ordered) set or a chain is a partially ordered set such that every pair of elements in the set are comparable. In other words, a chain is a a partially ordered set having no incomparable elements. Definition 3.4. Let ( P, ) be a partially ordered set. Then, Q is a maximally totally ordered subset of P if (i) Q P, (ii) ( Q, ) is totally ordered, and (iii) if any member of P not in Q is adjoined to Q, then the resulting collection of sets is no longer totally ordered by. 11

12 Remark 3.1. Every subset of a nonempty set, which consists of a single element, is totally ordered. Definition 3.5. Let S be a partially ordered set. An upper bound of W S is an element α S such that θ α θ W (1) A lower bound of W S is an element β S such that β θ θ W (2) Depending on S and W, an upper bound or a lower bound of W may or may not exist. Definition 3.6. Let (S, ) be a partially ordered set. An element α S is called a maximal element of S if θ α for every θ S which is comparable to α. In other words, If θ S, then (α θ) (α = θ) (3) Similarly, a minimal element of S is an element β S such that If θ S, then (θ β) (β = θ) (4) A partially ordered set S may or may not have a maximal element or a minimal element. Furthermore, a maximal element need not be an upper bound. Similarly, a minimal element need not be a lower bound. Example 3.1. Let S = (0,1) R; then, ( S, ) is a totally ordered set that has no maximal element and no minimal element. However, 1 R is an upper bound of S; similarly, 0 R is a lower bound of S. As a matter of fact, 1 is the least upper bound of S and 0 is the greatest lower bound of S. Example 3.2. Let S be the set of all points (x,y) in the plane R 2 with y 0. Let us define an ordering on S as ( ) (x,y) ( x,ỹ) ( (x = x) ) (y ỹ) Then, the partially ordered set (S, ) has infinitely many maximal elements. Zorn s lemma: Let S be a partially ordered set such that every chain T S has an upper bound. Then, S has at least one maximal element. Hausdorff Maximality Theorem: Every (nonempty) partially ordered set contains a maximal totally ordered subset. In other words, if S is a maximal totally ordered subset ofa(nonempty) partially orderedset X and if T is a totally ordered subset of X, then ( S T X ) ( S = T ). 12

13 Axiom of Choice: Let S be a set and I be an index set. Then, there exists a mapping, called the choice function, f : I S such that f(α) S α S and S α. That is, for every nonempty set, there exists a choice function. The axiom of choice can also be stated as: The product of a family of nonempty sets indexed by a nonempty set is nonempty. Remark 3.2. Zorn s Lemma and Hausdorff Maximality Theorem are equivalent and they are also equivalent to Axiom of Choice. For details, see Appendix, pp , on Hausdorff Maximality Theorem in Real and Complex Analysis by Rudin and p. 13 in Algebra by Thomas Hungerford. Let us illustrate a simple application of Zorn s lemma. We first make the following assertions: V is a vector space and A is a set of linearly independent vectors belonging to V. X is the collection of all linearly independent sets of vectors in V such that A is a subset of each member in X. is a partial ordering on X. H is a Hamel basis of V such that A H. I is a non-empty index set and Y = {B i : i I} is a chain of X. B = i I B i It follows that the sets in the chain Y can be ordered as: B i1 B i2 B in and Y has an upper bound B. Since Y can be arbitrarily chosen, X has a maximal element H by Zorn s lemma. 3.2 Extension of Linear Functionals In Hahn-Banach theorem, the objective is to extend a linear functional f, defined on a subspace U of a vector space V, which has a certain boundedness property. Definition 3.7. (Extension of a linear functional) Let f be a linear functional on a proper subspace U of a vector space V over the real field R. A linear functional f ext, on another subspace W V, is called an extension of f from U to W if U is a proper subspace of W, i.e., U W. f ext (x) = f(x) x U Definition 3.8. (Sublinear functional) Let V be a vector space over the real field R, and let p : V R. Then, p is called a sublinear functional on V if it has the following two properties: 13

14 Subadditive: p(x+y) p(x)+p(y) x,y V Positive homogeneous: p(αx) = αp(x) α [0, ) x V Remark 3.3. A norm on a vector space is a sublinear functional. Theorem 3.1. (Hahn-Banach Theorem: Extension of Linear Functionals) Let V be a vector space over the real field R, and let p be a sublinear functional on V. Let f be a linear functional on a subspace U V. If f(x) p(x) x U, then there exists an extension f ext of f from U to V such that f ext (x) p(x) x V. Proof. The theorem is proved in the following three steps: Step 1: Let us construct the set E consisting of the linear functional f and all linear extensions g of f, which satisfy the relation: g(x) p(x) on the domain D(g). The set E is partially ordered and Zorn s lemma yields a maximal element f ext E. Step 2: The linear functional f ext is defined on the entire space V. Step 3: The relation f ext (x) p(x) x V is established. Step 1: It is obvious that E is nonempty because f E. Let us define a partial ordering on E as: g h h is an extension of g. That is, D(g) D(h) and h(x) = g(x) x D(g). For any chain H E, let us define a linear functional g E as: D( g) = D(g) and g(x) = g(x) if x D(g) (5) g H Note that, for an x D(g 1 ) D(g 2 ) with g 1,g 2 H, we have g 1 (x) = g 2 (x) because H is a chain so that g 1 g 2 or g 2 g 1. Then, g g for all g H. Hence, H has an upper bound. Since selection of H E is arbitrary, Zorn s lemma implies that E has a maximal element; let us call this maximal element as f ext. By definition, f ext is a linear extension of f that satisfies the condition: f ext (x) p(x) x D(f ext ) (6) Step 2: Now we prove, by contradiction, that D(f ext ) spans the entire vector space V. Let us assume that the assertion is false, i.e., D(f ext ) is a proper subset of V. Then, there exists z ( V \ D(f ext ) ) and z 0 because 0 D(f ext ). Let the subspace W be spanned by D(f ext ) and the vector z. Thus, any x W can be expressed as: x = y +αz where y D(f ext ) and α is a scalar (7) 14

15 The above representation is unique because y D(f ext ) and z ( V D(f ext ). A linear functional g on W is defined by g(y +αz) = f ext (y)+αc where g(z) = c R (8) Note that g is a properextension off ext, i.e., D(f ext ) is a proper subset of D(g), because if α = 0, then g(y) = f ext (y) y D(f ext ). Consequently, if it is proven that g E by showing that g(x) p(x) x D(g), then this will contradict maximality of f ext so that the assertion D(f ext ) V is false, i.e., the truth of the statement D(f ext ) = V is established. Step 3: We will show that g with a real constant value of c in Eq. (8) satisfies the condition g(x) p(x) x D(g). Let y,z D(f ext ) and let w D(f ext ) be fixed. Since p is a subadditive functional and the linear functional f ext p, f ext (y) f ext (z) = f ext (y z) p(y z) = p(y +w w z) p(y +w)+p( w z) (9) Taking the last term to the left and the term f ext (y) to the right in Eq. (9), we have p( w z) f ext (z) p(y +w) f ext (y) (10) Since y does not appear on the left and z does not appear on the right, the inequality in Eq. (10) continues to hold if the supremum, m, is taken over z D(f ext ) on the left and the infimum, M, over y D(f ext ) on the right. Therefore, with the constant c in Eq. (8) being in the closed interval [m,m], it follows from Eq. (10) that p( w z) f ext (z) c z D(f ext ) (11) c p(y +w) f ext (y) y D(f ext ) (12) For α = 0, we already have x D(f ext ). Let us first prove g(x) p(x) x D(g) for α < 0 in Eq. (8). Replacing z in Eq. (11) by α 1 y and multiplying both sides by the positive quantity α yields: αp( w α 1 y)+f ext (y) αc (13) From Eqs. (8) and (11), using x = y +αw yields: g(x) = f ext (y)+αc αp( w α 1 y) = p(αw+y) = p(x) (14) For α > 0, let us replace y in Eq. (12) by α 1 y to obtain: 15

16 c p(α 1 y +w) f ext (α 1 y) (15) Multiplication of Eq. (15) by α yields αc αp(α 1 y +w) αf ext (α 1 y) = p(x) f ext (y) (16) A combination of Eq. (16) with Eq. (8) yields: g(x) = f ext (y)+αc p(x) (17) Remark 3.4. The above derivation of Hahn-Banach theorem does not require continuity (i.e., boundedness) of the linear functionals nor density of the vector space. In some restricted cases (e.g., finite-dimensional and separable Hilbert spaces), it is possible to prove Hahn-Banach Theorem without using Zorn s Lemma, as seen in Chapter 5, p. 111 of Optimization by Vector Space Methods by Luenberger. Niether of these two restrictions, namely, boundedness of linear functionals and density of the vector space, are critical for many engineering applications; for example, Sobolev spaces that form the backbone of finite-element analysis are separable Hilbert spaces. Therefore, in many engineering applications, it is possible to apply the Hahn-Banach theorem to separable normed spaces (e.g., l 1 ) although their dual spaces (e.g., l ) may be nonseparable. The steps of such analysis are briefly outlined below. Let {x 1, x 2,, x n, } be a countable dense set in V; let the sublinear functional pbe bounded andlet f be alinearboundedfunctional onasubspaceu of V suchthatf(x) p(x) x U. Fromthissetofvectorsletusselect, oneatatime, a subset of linearly independent vectors {y 1, y 2,, y n, } which is linearly independent of the subspace U V. The set of vectors {y 1, y 2,, y n, } together with the subspace U generates a dense subspace S, i.e., S = V. Now, the functional f on U can be extended to a functional g on the subspace S by extending f from U to [U +y 1 ] to [[U +y 1 ]+y 2 ] and so on. Finally, the resulting functional g (which is bounded because p is) can be extended from the dense subspace S to the space V. To see the above extension, let z V and let there exist a sequence {s k } of vectors in S converging to z. Let F(z) lim n g(s n ). Notice that F is linear and F g(s n ) p(s n ) p(z) and thus F(z) p(z) on V. Theorem 3.2. (Hahn-Banach Theorem: Generalization) Let V be a vector space over the real field R or the complex field C, and let p be a real-valued functional on V, having the following two properties: 16

17 Subadditive: p(x+y) p(x)+p(y) x,y V Absolute homogeneous: p(αx) = α p α C x V Let f be a linear functional on a subspace U V. If f(x) p(x) x U, then there exists an extension f ext of f from U to V such that f ext (x) p(x) x V. Proof. If V is a vector space over R, the proof is identical to that of Theorem 3.1. If V is a vector space over C, then the functional f is also complex-valued and is split into real and complex parts as: f(x) = f real (x)+if imag (x) (18) where both f real and f imag are real-valued. We note that f real (x) f(x) x U It follows from Theorem 3.1 that a linear extension of f real ext of f real from U to V satisfies the following condition: f real ext (x) p(x) x V Equating the real and imaginary parts of the following equation: we have i[f real (x)+if imag (x)] = if(x) = f(ix) = f real (ix)+if imag (ix) x U f imag (x) = f real (ix) x U (19) f ext (x) = fext real (x) ifreal ext (ix) x V (20) It follows from Eqs. (19) and (20) that f ext (x) = f(x) on U, i.e., f ext is an extension of f from U to V. It remains to prove the following tasks: 1. f ext is a linear functional on the complex vector space 2. f ext (x) p(x) x V Task 1 holds from the fact that, for any complex scalar a + ib, the following relation holds based on Eq. (20): f ext ((a+ib)x) = f real ext (ax+ibx) ifreal ext (iax bx) = af real ext (x)+bf real ext (ix) i[af real ext (ix) bf real ext (x)] = (a+ib)[f real ext (x) ifreal ext (ix)] = (a+ib)f real ext (x) 17

18 Now we prove Task 2. Let f ext (0) = 0 which holds because p(x) 0 x V. Let x 0 be such that f ext (0) 0. Using the polar notation, f ext (x) = f ext (x) exp(iθ) f ext (x) = exp( iθ)f ext (x). Since f ext (x) is real, the absolute homogeneity property of the sublinear functional p yields The proof is thus complete. f ext (x) = fext real ( ) ( ) exp( iθ)x p exp( iθ)x = exp( iθ(x) p(x) = p(x) Further details are available in Real and Complex Analysis by Rudin (see Chapter 5, p. 105). 4 Applications of Hahn-Banach Theorem to Bounded Linear Functionals Theorem 4.1. (Hahn-Banach Theorem: Normed Spaces) Let f be a bounded linear functional on a subspace U of a vector space V, defined on the real field R or the complex field C. Then, there exists a bounded linear functional f ext on V, which is an extension of f to V having the same norm, f ext = f (21) Proof. If U = {0}, then f = 0 and consequently f ext = 0. Let f 0. Since we will use Theorem 3.2 to prove this theorem, we must first find an appropriate sublinear functional p. We have f(x) f U x x U where we select p(x) = f U x (see Remark 3.3). Using Theorem 3.2, it follows that there exists a linear functional f ext, which is an extension of f, satisfies the condition: f ext (x) p(x) = f U x x V Taking supremum over all unity norm x V, we obtain the inequality: f ext V = sup x =1 f ext (x) f U (22) Since a norm cannot decrease under extension, we claim that A combination of Eqs. (22) and (23) proves the theorem. f ext V f U (23) 18

19 Corollary 4.1. Let V be a normed space and let x 0 0 be an arbitrary vector in V. Then, there exists a bounded linear functional g on V such that g = 1 and g(x 0 ) = x 0 V. Proof. Let U be the subspace spanned by the vector x 0. Let us define a linear functional f on U as f(αx 0 ) = αf(x 0 ) = α x 0, where α is a scalar. Then, f is bounded and f = 1 because if x = αx 0, then f(x) = f(αx 0 ) = α x 0 = αx 0 = x Then, Theorem 4.1 implies that f has a linear extension from U to V of norm f ext = f = 1 because f ext (x 0 ) = f(x 0 ) = x 0. Corollary 4.2. Let V be a normed vector space and f V. Then, every x V has the following property: x V = sup f =1 f(x) (24) and if x 0 V is such that f(x 0 ) = 0 f V for all f V, then x 0 = 0. Proof. By replacing x 0 by x in Corollary 4.1, it follows that f(x) sup x V \{0 V } f ext(x) = x f f ext and the proof follows from the fact that f(x f x. Lemma 4.1. Let U be a proper closed subspace of a normed vector space V. Let x 0 V U be arbitrary and the distance from x 0 to U is defined as: Then, there exists f ext V such that δ = inf y U y x 0 > 0 (25) f ext = 1; f ext (y) = 0 y U; and f ext (x 0 ) = δ (26) Proof. Let the subspace W be spanned by U and x 0. Let a bounded linear functional f be defined on W as: f(z) = f(y +αx 0 ) = αδ (27) We will first show that f satisfies Eq. (26) and then extend f to f ext on V by Theorem

20 Linearity of f is readily seen. Since U is closed and δ > 0, it follows that f 0. It follows from Eq. (26) that f(y) = 0 and f(x 0 ) = δ by setting α to 0 and 1, respectively. For α = 0, f(z) = 0. For α 0, it follows from Eq. (25) that f(z) = α δ = α inf y U y x 0 α α 1 y x 0 = y +αx 0 Therefore, f(z) z z W. Hence, f is bounded and f 1. Next we show that f 1. By definition, U contains a sequence {y k } such that y k x 0 δ as k. Let z k y k x 0. Then, f(z k ) = δ by setting α = 1 in Eq. (27). Furthermore, f = sup z W {0} f(z) z f(zk ) z k = δ z k 1 k Hence, f 1 which implies that f = 1. By Theorem 4.1, f is extended to V without increasing the norm. 4.1 Dual Spaces and Separability Theorem 4.2. (Separability) For a normed vector space V, if the dual space V is separable, then V itself is separable. Proof. Given that V isseparable, the unit ballu {f V : f = 1} V has a countable dense subset, say, {f k : k N}, where f k = sup x =1 f k (x) = 1. Therefore,thereexistunitvectorsx k V suchthatf k (x k ) [0,1]. Letf k (x k ) 1 2. Let W be the closure of the space spanned by {x k }. Then, W is separable because W has a countable dense subset, namely, the set of all linear combinations of the vectors x k with rational coefficients. To show that W = V by contradiction, let us assume that W V. Since W is closed, it follows from Lemma 4.1 that there exists f ext V with f ext = 1 and f ext (y) = 0 y W. Since x k W, we have f ext (x k ) = 0 k, which implies 0.5 f k (x k ) = f k (x k ) f ext (x k ) = (f k f ext )(x k ) (f k f ext ) (x k ) = (f k f ext ) The assertion (f k f ext ) 0.5 is a contradiction because {f k } is dense in U ; in fact, (f ext ) = 1. Corollary 4.3. l is not isometrically isomorphic to l 1. 20

21 Proof. Let us assume that l is isometrically isomorphic to l 1. Since l 1 is separable, so is l. By Theorem 4.2, l must be separable. This is a contradiction. Remark 4.1. L is not isometrically isomorphic to L 1 by the same argument as in Corollary 4.3. Remark 4.2. It follows from Corollary 4.3 that the converse of Theorem 4.2 is false. Remark 4.3. Thespacec 0 ofallsequencesofscalarsconvergingtozeroisseparable and c 0 is isometrically isomorphic to l Bounded Linear Functionals on C[a, b] This section presents a general representation formula for bounded linear functionals on C[a,b], where C[a,b] is the space of continuous functions on a fixed compact interval [a,b] with the metric defined as: d(x,y) = max t [a,b] x(t) y(t) (28) Definition 4.1. A function w on [a,b] is defined to be of bounded variation on [a,b] if its total variation is finite, i.e., Var(w) = sup n w(t j ) w(t j 1 ) < (29) j=1 where the supremum is taken over all partitions P n {a = t 0 < t 1 < < t n = b} for some n N (30) Theorem 4.3. (Reisz Theorem on Functionals) Every bounded linear functional f on C[a,b] can be represented by a Riemann- Stieltjes integral f(x) = b a x(t)dw(t) (31) where w is of bounded variation on [a,b] and has the total variation Var(w) = f (32) 21

22 Proof. It follows from Theorem 4.1 that f has an extension f ext from C[a,b] to the space of all bounded functions on [a,b] with the norm defined as: x = max t [a,b] x(t) (33) and the bounded functional f ext has the same norm as f, i.e., f ext = f. If the function w in Eq. (31) is real-valued, then it is defined as: w(t) = f ext (χ [a,t] ) where t [a,b] and the characteristic function χ [a,t] = 1 on the support [a,t]. In general, for a complex-valued w, we use the polar notation to express w(t) = w(t) exp(i θ), where θ = arg(w(t)). For any partition (see Eq. (30)), we have: = f ext (x 1 )+ n j=2 n w(t j ) w(t j 1 ) j=1 f ext (x j ) f ext (x j 1 ) = ε 1 f ext (x 1 )+ n j=2 = f ext ( ε1 x 1 + ε j (f ext (x j ) f ext (x j 1 )) n j=2 f ext ( ε 1 x 1 + ε j (f ext (x j ) x j 1 ) ) n j=2 ε j (x j x j 1 ) ) On the right hand side, f ext = f and the other factor equals to 1 because ε j = 1 and only one of the terms x 1, x 2 x 1, is nonzero and its norm is equal to 1. This is true because of the choice of the structure of x t as the characteristic function χ [a,t]. On the left we take supremum over all partitions of [a,b]. Then, it follows that Var(w) f (34) Hence, w is of bounded variation on [a,b]. Next we prove Eq. (31). For every partition P n of the form similar to Eq. (30) on C[a,b], we define a function as: z Pn = x 0 x 1 + n x j 1 (x j x j 1 ) j=2 22

23 implying that f ext is bounded on [a,b]. By definition of w, = x 0 f ext (x 1 )+ f ext (z Pn ) n x j 1 (f ext (x j ) f ext (x j 1 ) j=2 = x 0 w(t 1 )+ = n t j 1 (w(t j ) w(t j 1 )) j=2 n t j 1 (w(t j ) w(t j 1 )) (35) j=1 where the last equality follows from w(t 0 ) = w(a) = 0. By making the partition P n finer and taking n, the sum on the right hand side of Eq. 35 approaches the integral in Eq. (31). Since f ext (z Pn ) f ext (x), the integral in Eq. (31) becomes equal to f(x) because x C[a,b]. Since b a x(t) dw(t) maxt [a,b] x(t) Var(w), it follows from the supremum over all x C[a,b] that Var(w) f (36) The combination of Eqs. (34) and (36) yields the equality in Eq. (32). The proof is complete. 5 Basic Concepts of Bounded Linear Transformations in Banach Spaces This section first presents background materials on bounded linear transformations in Banach spaces beyond what have been addressed in Chapter 5. In particular, the following four theorems, which are useful for analysis of a variety of advanced engineering problems, are presented below. 1. Baire Category Theorem 2. Banach-Steinhaus Uniform Boundedness Theorem 3. Open Mapping Theorem 4. Closed Graph Theorem 5.1 Background for Baire Category Theorem Let (X,d) be a metric space. Among all subsets of X let us define a class of large sets and a class of small sets with the following properties: (i) A large set is nonempty. (ii) A set S X is large if and only if its complement X \S is a small set. 23

24 (iii) A countable intersection of large sets is a large set. For example, let a probability measure P be defined on a measurable space (X,E). Then, a measurable set L X with probability P(L) = 1 is a large set and a measurable S with P(S) = 0 is a small set. Theorem 5.1. (Baire Category Theorem) Let (X,d) be a complete metric space and let {V k } be a sequence of (nonempty) open, dense subsets of X. Then, V V k is a (nonempty) dense subset of X. Proof. Let U X be nonempty and open; and the choice of U is arbitrary. It suffices to show that ( V k) U is nonempty. Let x 0 U and r 0 (0,1) such that B 3r0 (x 0 ) U. Then, by induction, for every k 1, choose x k U and r k > 0 such that B 3rk (x k ) ( V k B rk 1 (x k 1 ) ) ; this is possible because V k is open and dense. For each k 1, it implies that r k+1 r k 3 and d(x k+1,x k ) r k < 1 3 k {x k} is a Cauchy sequence Since the metric space (X,d) is complete, the Cauchy sequence {x k } converges to some x X. Then, it follows, by triangular inequality, that d(x k,x ) d(x j,x j+1 ) r j 3 k j r k = 3 2 r k j=k Therefore, for every k 1, x B 3rk (x k ) V k. For k = 0, the same argument yields B 3r0 (x 0 ) U. Hence, x ( V k) U. j=k j=k 5.2 Background for Uniform Boundedness Theorem Let (V, V ) and (W, W ) be Banach spaces over the same field. Let Λ : V W be a linear bounded transformation (i.e., Λ BL(V,W)), where the induced norm of Λ is defined as: Λ sup x V =1 Λx W. Based on some results of Baire category theorem, Banach-Steinhaus uniform boundedness theorem is stated and proved below. Theorem 5.2. (Banach-Steinhaus Uniform Boundedness Theorem) Let L BL(V,W) ba a family of bounded linear transformations and let Λ L. Then, either L is uniformly bounded, i.e., sup Λ L Λ < (37) or else there exists a dense set S V such that sup Λ L Λx = x S (38) 24

25 Proof. For every integer n N, let us consider an open set S n {x V : Λx > n for some Λ L} (39) If one of these sets, say S k, is not dense in V, then there exists x 0 V and radius r > 0 such that the closed ball Br (x 0 ) does not intersect S k, i.e., Λx k for some x B r (x 0 ) and every Λ L. Now if x r, then Λ L, Λx = Λ(x 0 +x) Λx 0 Λ(x 0 +x) + Λx 0 2k Therefore, Λ L, Λ sup Λx = 1 x =1 r sup x =r Λx 2k r In this case, the family of linear transformations L is uniformly bounded. The other possibility is that the open sets S n in Eq. (39) are all dense in V. Then, by Baire category theorem, the intersection S n 1 S n is dense in V. By construction, each x S and n 1, there exists a linear transformation Λ L such that Λx < n. Hence, Eq. (37) holds. Remark 5.1. It follows from Banach-Steinhaus Uniform Boundedness Theorem (Theorem 5.2) that if a family of linear transformation Λ BL(V,W) is bounded on the unit ball, then it is uniformly bounded, i.e., the following condition holds. supλx < x with x 1 Λ L This justifies the name uniform boundedness principle. Corollary 5.1. (Continuity of the Pointwise Limit) Let (V, V ) and (W, W ) be Banach spaces over the same field, and let {Λ k : k N} be a sequence of linear transformations with the pointwise limit Λx = lim n Λ nx x V (40) Then, the map Λ in Eq. (40) is a bounded linear transformation. Proof. The sequence {Λ k : k N} is bounded for every x V. Hence, by Theorem 5.2, the sequence {Λ k } is uniformly bounded, which implies that ( ) Λ sup Λx = sup lim Λ nx sup Λ n < x 1 x 1 n n 1 Hence, the linear transformation Λ is bounded. 25

26 5.3 Background for Open Mapping Theorem First let us introduce the notion of open mapping from one metric space to another. Definition 5.1. (Open Mapping) Let (X,d X ) and (Y,d Y ) be metric spaces. Then, a map f : X Y is said to be (d X -d Y ) open if, for every d X -open set A X, the image f(a) is d Y -open. The implication of Definition 5.1 is that, for every x X and r (0, ), the image f ( B r (x) ) of the d X -open ball B r (x) at center x and radius r is a d Y -open set that contains a d Y -open ball centered at f(x). Theorem 5.3. (Open Mapping Theorem) Let (V, V ) and (W, W ) be Banach spaces over the same field. If Λ : U V is a linear, surjective and bounded transformation, then Λ is open, i.e., for every V -open X U, the image Λ(X) is W -open. Proof. The proof of open mapping theorem is presented in three steps. Step 1 By linearity, the image of an open ball B r (x), where the ball radius r > 0 and x V, is expressed as: ( ) ( ) ( ) Λ B r (x) = Λ(x)+Λ B r (0 V ) = Λ(x)+rΛ B 1 (0 V ) ( ) To prove the theorem, it suffices to show that the image Λ B 1 (0 V ) contains an open ball around the origin 0 W in (W, W ). Step 2 Since Λissurjective, W = ) n=1 (B Λ n (0 V ). Recallingthat(W, W )is a Banach space, it follows from Baire Category Theorem (Theorem 5.1) that ( ) at least one of the closures Λ B n (0 V ) has nonempty interior. By rescaling, it follows that ( ) Λ B 1 (0 V ) = 1 ( ) n Λ B n (0 V ) will also have non-empty interior, i.e., there exists w W and r > 0 such ( ) that the ball B r (w) Λ B 1 (0 V ). Furthermore, since the unit ball B 1 (0 V ) ( ) is convex and symmetric, the same is true of its image Λ B n (0 V ) and of its ( ) ( ) closure Λ B n (0 V ). By symmetry B r ( w) Λ B 1 (0 V ), while convexity implies that the ball B r (0 W ) W satisfies ( 1 B r (0 W ) = 2 B r(w)+ 1 ) ( ) 2 B r( w) Λ B 1 (0 V ) Using the linearity of Λ and by rescaling, it follows that, for all n 1, ( ) B 2 n r(0 W ) Λ B n (0 V ) 26

27 ( ) Step 3 The proof is concluded by showing that B r/2 (0 W ) Λ B 1 (0 V ) for all n 1. Letting w B r/2 (0 W ), w.e proceed by induction as follows. By density, we find v 1 B 2 1(0 V ) such that w Λv 1 < 2 2 r. Next, we find v 2 B 2 2(0 V ) such that (w Λv 1 ) Λv 2 < 2 3 r. Continuing by induction, for each n N, we have n 1 w j=1 ( ) Λv j B 2 n(0 V ) Λ B n (0 V ) Therefore, we can select a point v n B 2 n(0 V ) such that ( n 1 w Λv j ) Λv n < 2 n 1 r j=1 Since V is a Banach space and n v n <, the series n v n converges, say n=1 v n = v. Then, we observe that v v n < 2 n = 1 and Λv = lim n=1 n=1 n j=1 n Λv j = w Hence, the image Λ(B 1 (0 V ) contains all points w W with w < r 2. Remark 5.2. If (V, V and (W, W are Banach spaces over the same field, and if Λ : U V is a linear, continuous bijection, then the inverse Λ 1 : W V is continuous as well. This is so because of the following fact since Λ is a bijection, Λ is open if and only if Λ 1 is continuous. 5.4 Background for Closed Graph Theorem Let (V, V ) and (W, W ) be Banach spaces over the same field. The product space V W is the set of all ordered couples (v,w) with v V and w W. The product space V W is a Banach space with the following norm: (v,w) V W v V + w W. Definition 5.2. (Closed Graph) Let Λ be a (possibly unbounded) linear transformation with domain Dom(Λ) V and values in W. Then, the linear transformation Λ is said to be closed if its graph { } Graph(Λ) (v,w) : v Dom(Λ),w = Λv is a closed subset of the product space V W. 27

28 In other words, Definition 5.2 states that the linear transformation Λ : V W is closed provided that the following condition holds: Given two sequences of points v k Dom(Λ) and w k = Λv k W, if v k v and w k w, then v Dom(Λ) and Λv = w. It is noted that every continuous linear transformation Λ : V W is closed. The next result shows that the converse is also true provided that the domain Dom(Λ) is the entire space V. Theorem 5.4. (Closed Graph Theorem) Let (V, V ) and (W, W ) be Banach spaces over the same field. If Λ : U V is a closed linear transformation defined on the entire space V, i.e., Dom(Λ) = V. Then, Λ is continuous. Proof. Let us denote Γ Graph(Λ). By Definition 5.2, Γ is a closed subspace of the product space V W; hence, Γ is a Banach space as well. Now, let us consider two projections, π 1 : Γ V and π 2 : Γ W, respectively defined as: x π 1 (x,λx), and Λx π 2 (x,λx). Since the projection map π 1 is a (linear) bijection between Γ and V, the inverse π 1 1 is continuous. Hence, the composition of two continuous functions Λ = π 2 π 1 1 is continuous. Example 5.1. Let the space V = C 0 (R), which is the space of all bounded continuous functions f : R R, with norm f = sup x =1 f(x). Let Λ be the differentiation operator defined as: Λf = f. The domain of Λ is the subspace of V defined as: Dom(Λ) { f C 0 (R) : f C 0 (R) } = C 1 (R) consisting of all continuously differentiable functions with bounded derivatives. Next we observe that the linear transformation Λ is not bounded and hence not continuous. For example, the functions f k (x) = sinkx are uniformly bounded, i.e., f n C 0= 1 for all n 1. However, the sequence of derivatives Λf k = f k is unbounded, because f k (x) = kcoskx and hence f k C0= k. On the other hand, the linear transformation Λ : f f has closed graph. To see this, let us consider a sequence f k Dom(Λ) such that, for some functions f,g C 0 (R), one may have f k f 0 and f k g 0. If the above two conditions hold, then f is continuously differentiable and f = g. Hence, the point (f,g) V V lies in the graph of Λ. Notice that this example does not contradict the closed theorem (Theorem 5.4), because Λ is not defined on the entire space V = C 0 (R), i.e., Dom(Λ) is a proper subspace of V. 28