Normed and Banach spaces

Normed and Banach spaces László Erdős Nov 11, 2006 1 Norms We recall that the norm is a function on a vectorspace V, : V R +, satisfying the following properties x + y x + y cx = c x x = 0 x = 0 We always assume that the field of V is R or C, i.e. there is a natural absolute value on these fields. Note that otherwise one has to start with putting a norm on the field itself the absolute value on R and C is a norm. A vectorspace equipped with a norm is called normed space (normierter Raum). A norm defines a metric on V with the formula d(x, y) := x y i.e. normed spaces are in particular metric spaces as well. Of course there are many norms on a vectorspace and there is typically not a single optimal one. We have already seen the different metrics d p on C[0, 1] which actually come from the following norms: ( 1 1/p f p := f(x) dx) p 1 p < 0 f := sup x f(x) = max f(x) x They are called L p and L (or supremum- or maximum-) norms, respectively. The names come from the appropriate Lebesgue spaces, since the same formulas define norms on bigger spaces of various integrable functions as well. 1

In most cases, when we talk about normed spaces, it is important to indicate not just the vectorspace, but also the norm. E.g. we will denote the vectorspace of continuous functions by C[0, 1], but if we want to think of it as a normed space, we will specify the norm and use the notation (C[0, 1], p ). Later we will see that the supremum norm, is in some sense the most natural one on C[0, 1], and thus sometimes in some books take it for granted that C[0, 1] is equipped with this norm without explicitly saying it. Homework 1.1 Prove that for any f C[0, 1] lim p f p = f Even the Euclidean space R n has many norms, but in finite dimensions it does not really make a big difference (at least not qualitatively, e.g. as far as convergence is concerned) since we have Lemma 1.2 On a finite dimensional vectorspace V any two norms are equivalent, i.e. for any two norms 1 and 2 there exists two positive constants, c, C, such that c x 1 x 2 C x 2, x V (1.1) Proof. Fix a basis, {e 1, e 2,..., e n } and represent any vector x V with a vector in C n consisting of its coordinates x = n x j e j (x 1, x 2,...,x n ) C n j=1 This representation is clearly one to one. We will show that any norm on V is equivalent with the norm x := max x j j i.e. with the l -norm on the coordinates (CHECK that this is a norm!). In particular this will show that any two norms are equivalent. Recall that in C n any closed bounded set is compact, so consider the unit sphere in the l S := {x V : max x j = 1} j 2

this is a compact set in the topology generated by the -norm (Heine-Borel). If you learned Heine-Borel for the Euclidean norm, then you have to recall that the euclidean norm and the maximumnorm are equivalent, i.e. max x j j ( n j=1 ) 1/2 x j 2 n max x j, j hence they generate the same topology, hence their compact sets are the same. Consider now an arbitrary norm,, as a function on V : f(x) := x Note that this function is continuous with respect to the norm, since x + h x h (triangle inequality) and h = n j=1 h j e j max h j j n e j C h where C is a constant independent of h (it depends only on the basis that we fixed). Therefore we have a continuous function f that we restrict onto the compact set S, i.e. we have a continuous function on a compact set. By a general theorem in analysis we know that the minimum and maximum of a continuous function on a compact set is achieved, in particular there exists two numbers m, M such that j=1 m := min f(x) f(x) M := max f(x) x S x S x S Note that M < and m > 0. This latter holds because if m were zero, then there would be an element y S such that f(y) = 0. But recall that f(y) = y is a norm, so this would imply y = 0, in contradiction with y S. For an arbitrary x V, x 0, we set v := x/ x, then v S and by the homogeneity of the -norm can write x = x v = x f(v) Therefore 0 < m x x M < that proves (1.1). Since equivalent norms generate the same topology and the unit ball is compact in the l norm, thus we proved that 3

Lemma 1.3 In any finite dimensional vectorspace equipped with any norm the unit ball is compact. This statetment is false in infinite dimensional vectorspaces. Consider for example the unit ball in l : B := {(x 1, x 2,...) : sup x j 1} j Take the sequence e 1 := (1, 0, 0, 0,...) e 2 := (0, 1, 0, 0,...) etc. This clearly lies in B, but it has no convergent subsequence. Actually this is not just an example, but the following hold: Proposition 1.4 Let V be an infinite dimensional vectorspace equipped with any norm. Then the closed unit ball B := {x V : x 1} is not compact. Proof. The proof relies on the following version of Riesz Lemma (there will be a very similar Lemma later in Hilbert spaces, that will also be called Riesz Lemma). Lemma 1.5 (Riesz) Let U X be closed proper subspace of a normed vectorspace (X, ) (proper means U X.) Given any number 0 < δ < 1, there is a unit vector x = x δ X (depending on δ) such that x δ u 1 δ, u U The lemma is interesting when δ is very small. To see this, define dist(x, U) := inf{ x u : u U} to be distance of x from the subspace U. Since 0 U, the distance of any unit vector cannot be bigger than 1. The Lemma says that there is an almost optimal unit vector, i.e. a unit vector that is 1 δ distance from U. Proof of the Lemma. Choose an element x U and let d = dist(x, U). By the infimum in the definition of d, there exists a vector u δ U such that x u δ 4 d 1 δ

since d 1 δ > d. Define x δ := x u δ x u δ This is clearly a unit vector and this will be our choice. We need to check its distance from any other elements of U. Let u U be arbitrary and write x x δ u = x u δ u δ x u δ u 1 = x (uδ + x u δ u) x u δ d x u δ > 1 δ and this completes the proof. Question: where did we use that U is closed? Apparently nowhere, but implicitly we did use it in the inequality d > d, that holds only if d > 0 (note that from this it also 1 δ follows that x u δ > 0 which was also tacitly used above). To ensure d = dist(x, U) > 0, the information x U would not be enough without the closedness of U (the distance of a boundary point of a set to the set is zero, even though the boundary point may not lie in the set itself). You may find it weird a subspace may not be closed. Indeed, this is often a property of infinite dimensional subspaces. For example consider C 1 [0, 1] := {f : [0, 1] C : f, f C[0, 1]} the set of continuously differentiable functions on [0, 1] (since the [0, 1] is closed, we also require the existence and continuity of the appropriate one-sided derivatives at the endpoints). Now it is clear that C 1 [0, 1] is an infinite dimensional subspace of C[0, 1], but is not closed: it is easy to find a sequence of differentiable functions that converge in supremum norm to a non-differentiable (but continuous) function (FIND AN EXAMPLE). However, finite dimensional subspaces are always closed: Homework 1.6 Let U X be a finite dimensional subspace of a normed space (X, ), then U is closed. (Hint: let u n u, u n U. Choose a basis in U and express u n in terms of this basis. Using that any two norms on U are equivalent (Lemma 1.2), show that coefficients of u n in the fixed basis are Cauchy sequences, hence they have a limit, thus you can represent u as a linear combination of basis vectors in U.) Finally, we can return to the proof of Proposition 1.4. Start with a unit vector x 1 and define U 1 = span(x 1 ) be the one-dimensional (hence closed) subspace spanned by x 1. Fix 5

δ = 1/2 and apply Riesz Lemma. Then there exists a unit vector x 2 such that x 1 x 2 1. 2 Define U 2 := Span(x 1, x 2 ) and apply again Riesz Lemma to find a unit vector x 3 so that x 3 x 1 1/2 and x 3 x 2 1/2. Etc. The sequence x 1, x 2,... constructed in this way is in the unit ball but it clearly does not have any convergent subsequence. 2 Linear maps We know what a linear map T : V W between two vectorspaces means: this is a vectorspace homeomorphismus, i.e. a map that preserves the basic structure of the vectorspace (addition and scalar multiplication). The words linear maps (lineare Abbildungen) and linear operators (lineare Operatoren) are synonyms and used in parallel. Definition 2.1 Let (V, V ) and (W, W ) be two normed spaces. A linear map T : V W is called bounded (beschränkt) if there exists a finite constant C such that Tv W C v V, v V (2.2) Note that we neglect the parenthesis: the absolutely honest notation would be T(v) instead of Tv. This is a very common abbreviated notation and it is used only for linear maps. The same convention is used for composition maps, e.g. STv = S(T(v)) Homework 2.2 Suppose V is finite dimensional. Then any linear map T : V W is bounded. (Hint: choose a basis in V, express Tv in terms of the T-images of this basis and use that V is equivalent with the maximum-norm of the coefficients in the basis decomposition.) Here are examples of a bounded and an unbounded map: The antiderivative map on the space (C[0, 1], ): A : f (Af)(x) := x 0 f(s)ds is bounded. Actually A maps C[0, 1] into the subspace C 1 [0, 1] C[0, 1]. The inverse map, i.e. the differentiation map D : f (Df)(x) := f (x) 6

from (C 1 [0, 1], ) into (C[0, 1]. ) is unbounded (it is easy to construct a sequence f n such that f n = 1 but f n, thus f n C f n would never hold for any fixed constant C). Understandably the derivative map is probably the most important linear map in functional analysis. It is an unfortunate (?) fact, that it is unbounded: we will see that the theory of bounded maps is much easier. Actually in this course we will almost exclusively work with bounded maps. Comparing derivative and antiderivative (=integration) as linear maps, it is clear that integration behaves much better. This is a general fact: integration is usually a stable operation (e.g. numerically easier and safer to compute, it is less sensitive to imprecisions), while the derivative is unstable. For pedagogical reasons one usually first learns derivative (because it is conceptually simpler) and integration only later. From mathematical point of view integration should be the starting point. We go back to bounded linear maps. If T : (V, V ) (W, W ) is a map between two normed spaces, it is meaningful to ask if the map is continuous (the normed spaces are of course topological spaces as well). The very good news is that within the linear maps, bounded and continuous maps are the same. The proof is easy, but the statement is actually quite surprising nothing like that is expected to hold for general (non-linear) maps! It is a special and strong property of the linearity. Theorem 2.3 Let T : (V, V ) (W, W ) be a linear map. Then the following three properties are equivalent (i) T is bounded; (ii) T is continuous (at any point of V ); (iii) T is continouos at one point. In the following proof we will intentionally neglect the subscripts V and W from the norms, and use only, but think it over at every step, whether the actual norm is in V or in W. This is a very common lousiness of math writing the norm is indexed only if there is a danger of confusion. When it is clear from the context which norm we talk about, we use only. The confusion usually comes when the same space is equipped with two different norms. When we have two spaces and each has its own norm, they cannot be confused, since the argument of the norm indicates which space we consider: if we know that a vector x is in space V, then x must necessarily mean x V. Proof. (i) (ii): Since any normed space is metric, it is sufficient to check continuity along sequences, i.e. we need to show that if x n x, then Tx n Tx. But this is clear from Tx n Tx = T(x n x) C x n x 0 7

(ii) (iii) is trivial. (iii) (i): Suppose T is continuous at x, i.e. we have Tx n Tx whenever x n x. Introducing y n := x n x, we see that then T is continuous at the origin, i.e. for any y n 0 we have Ty n 0. Suppose that T were not bounded. Then there existed a sequence v n such that Tv n n v n, n N But then, by linearity and homogeneity (of the norm): ( vn ) T = 1 n v n while v n = 1 n v n n 0 i.e. we have found a sequence converging to zero (namely v n /n v n ) whose T-image would not converge to zero. Similarly to the matrix norm generated by a vectornorm, we can define the norm of a bounded linear map: Definition 2.4 Let T : (V, V ) (W, W ) be a bounded linear map. The norm of T is defined to be the smallest constant C so that (2.2) holds, i.e. { Tv W } T := sup : v V, v 0 (2.3) v V By linearity, it is the same as T := sup{ Tv W : v V, v V = 1} Note that the norm of T depends on the two norms fixed on V and W, so it would be more honest to indicate this fact but it usually follows from the context and we do not do it. The notation and the name already indicates that the number defined in (2.3) is indeed a norm. To make it absolutely precise, one has to introduce a vector space structure on the set of bounded linear maps between (V, V ) and (W, W ). This is done in the most natural way, if T and S are two such maps, then their sum is defined as (T + S)v := Tv + Sv and similarly for the multiplication by a scalar. Notice that the two plus signs are not the same: the one on the left indicates the addition in the space of maps (just being defined), the one on the right is the addition in W. 8

Homework 2.5 Check that (2.3) indeed defines a norm on the vectorspace of bounded linear operators. Finally, we remark that the norm on operators enjoys two additional properties. First, it is obvious that the identity map, I : (V, V ) (V, V ) is bounded and has norm 1: I = 1 A delicate remark: This relation holds only if V is considered to be equipped with the same norm both as a domain and as a target. For example, if we consider the identity map as I : (C[0, 2], 1 ) (C[0, 2], ) then it is unbounded, while the identity map as I : (C[0, 2], ) (C[0, 2], 1 ) is bounded and has norm 2. The second property is submultiplicativity; the matrix analogue being the bound AB A B (2.4) We formulate it as a lemma, whose proof is a HOMEWORK (it goes exactly as the proof of (2.4) from linear algebra). Lemma 2.6 Let two maps be given: T : (V, V ) (W, W ), S : (W, W ) (U, U ) and suppose that both maps are bounded. Then the composition map ST : (V, V ) (U, U ) is also bounded and ST S T A small remark: Not every matrix norm satisfies (2.4) (e.g. the so called Frobenius norm, A := [Tr(AA )] 1/2, does not). But those matrix norms that are generated by a vector norm (according to Definition 2.4) do satisfy this property. In principle one could similarly define norms on the vectorspace of linear operators between two spaces that have nothing to with any norm on the spaces themselves. For those norms, the submultiplicativity does not hold. We will not encounter such norms, they are unimportant. 9

3 Banach spaces Recall the definition Definition 3.1 A normed vectorspace (V, ) is called Banach space (Banachraum) if it is complete (with respect to the metric generated by ). Obviously, every finite dimensional normed vectorspace is Banach. Moreover, we have also proved that (C(X), ), the space of continuous functions on a metric space, equipped with the supremum norm (metric) is complete. In contrast, the space (C[0, 1], 1 ) is not complete (HOMEWORK!). It will be a fundamental question how to complete it, this will lead us to the Lebesgue integral. The following theorem is very simple, almost tautological, but the idea behind it is fundamental. Theorem 3.2 (Bounded extension principle) Let T : (V, V ) (W, W ) be a bounded map and assume that W is a Banach space. Then T can be extended to the completion of V in a unique way with keeping the norm. Proof. Let Ṽ denote the extension and let x Ṽ. Since Ṽ is dense in V, there is a sequence x n V such that x n x. Clearly Tx n is Cauchy, since Tx n Tx m T x n x m Since W is complete, Tx n converges, say Tx n y. We want to define Tx := y to be the extension of T, but before we can do it, we have to prove that y is independent of the choice of the sequence x n and depends only on x. But this is easy, if x n x were another sequence and Tx n y, then we can merge these two sequences into one sequence by considering x 1, x 1, x 2, x 2, x 3, x 3... This new sequence is also Cauchy, so its T-image is convergent, but then y = y (here we implicitly use the trivial fact that the limit in a normed actually metric space is unique). Now we can really define the extension by Tx := y It is immediate to check that T is linear and T is the restriction of T to V from Ṽ (CHECK). 10

To compute the norm, we have Tx = lim n Tx n lim sup n Tx n lim sup T x n T x n for any x Ṽ, thus T T, and since T is an extension of T, its norm is at least T, so we have T = T. The uniqueness of T is trivial (CHECK) The main application of this principle is that it is enough to specify a bounded linear map on a dense subspace if the target space is Banach. This has enormous advantages. A very naive, almost stupid example is the following. Consider Q R equipped with the usual absolute value as norm. Suppose we want to define the multiplication by two linear operator on R, i.e. T : R R, Tx = 2x Since Q is dense, so the above theorem says that it is sufficient to know the map T : Q R, Tx = 2x i.e. to know how to multiply rational numbers by two, from this information (and the boundedness and linearity) it follows how to multiply any real number by two. Here is a less trivial example that actually you have seen and heavily used but you did not view it in this way. Example: [Definition of the Riemann integral] Let PC[a, b] be the set of piecewise continuous functions on [a, b] (for definiteness, we prescribe that the functions be right continuous at the jumps), i.e. it is the set of all functions f : [a, b] C so that there exists finitely many points, x 1,...x n [a, b], such that f is continuous on (x j, x j+1 ), for any j = 0, 1,...n (where x 0 = a, x n+1 = b), and f is right continuous at all x j : lim x x j +0 f(x) = f(x j), j = 0, 1,...n We equip PC[a, b] with the supremum norm. Let us denote by S[a, b] the space of stepfunctions, i.e. this is a subset of PC[a, b] of the form n 1 f = s j χ [xj,x j+1 ) + s n χ [xn,b] (3.5) j=0 where s j C and χ A denotes the characteristic function of a set A. It is easy to check (HOMEWORK) that S[a, b] is dense in PC[a, b] (with the supremum norm). 11

We define a linear map from S[a, b] to C as follows I : f n s j (x j+1 x j ) j=0 if f is of the form. One has to check that this is well defined, i.e. I(f) does not depend on the representation (3.5) (such representation of a stepfunction is not unique, WHY?) It is easy to see that I : S[a, b] C is a bounded map (with norm b a ), and since C is Banach, thus by the bounded extension principle, we have extended I to PC[a, b]. The map Ĩ : PC[a, b] C is the Riemann integral. If you read the proof of the bounded extension principle, you see that the abstract procedure there exactly corresponds to the procedure how the Riemann integral was defined from stepfunctions to continuous (or piecewise continuous) functions. With a similar argument one can extend I to the set of all Riemann integrable functions. You may wonder if there is some cheating going on here. Eventually what happened so far is quite soft, and Riemann integral just popped up almost as a triviality, while in Analysis I you spend several weeks with it. The point is the same that I emphasized in the completion theorem. Proving that some object exists via some general principle (like completion viewed as a factorspace of Cauchy sequences) is usually not hard or deep. The key is always how to identify the object in a manageable, computationally feasible way. The main point behind the Riemann integral is not so much that one could take the limit of stepfunctions and the area under the graph is obtained as the limit of areas of approximating rectangles. All these may be notationally a bit painful to bookkeep (and this is done quite elegantly by the above abstract language), but nothing deep happens here. The deep and surprising fact is the Newton-Leibniz Theorem that connects the Riemann integral with something else (derivative) and this enables us to compute Riemann integrals without actually going through the stepfunction-limit procedure. 12