The Finite Dimensional Normed Linear Space Theorem Richard DiSalvo Dr. Elmer Mathematical Foundations of Economics Fall/Spring, 20-202 The claim that follows, which I have called the nite-dimensional normed linear space theorem, essentially says that all such spaces are topologically R n with the Euclidean norm. This means that in many cases the intuition we obtain in R, R 2, and R 3 by imagining intervals, circles, and spheres, respectively, will carry over into not only higher dimension R n but also any vector space that has nite dimension. Throughout this discussion I assume that all scalars for forming linear combinations in this nite dimensional linear space are real numbers (I have heard of complex vector spaces, but in this mathematical foundations for economics independent study we are solely concerned with real vector spaces). By nite-dimensional normed linear space I mean a set X of vectors along with addition of vectors and scalar multiplication satisfying the usual axioms for a vector space ( e.g., X is closed under arbitrary linear combinations of its elements), and a norm function : X R satisfying the usual axioms of a norm (e.g., the triangle inequality), with the induced distance function on X dened in the usual way: ρ X (x, y) = x y. Finally, by nite dimensional, I mean that there exists a nite set of vectors {x, x 2,..., } X that are linearly independent and span X (every vector in X can be written as a linear combination of vectors in {x, x 2,..., }, and none of the x i are redundant). A standard example of such a vector space is X = R 3 with Euclidean norm and basis {e, e 2, e 3 } = {(, 0, 0), (0,, 0), (0, 0, )}. These assumptions force X to be highly structured. In particular, it can be shown that every open ball in X is simply a translation and rescaling of the unit open ball centered around the origin. So the metric on X is, in this context, determined entirely by the shape of its unit ball. For example, the Euclidean norm has this unit ball be the normal sphere; the taxicab norm leads to a diamond-shape; the sup norm has its unit ball a cube. Within this context, we have the following. Claim (The Finite-Dimensional Normed Linear Space Theorem ):. Any nite-dimensional normed linear space is complete (Cauchy convergent for sequences in that space). 2. In any nite-dimensional normed linear space, a set is compact if and only if it is closed and bounded (Compact Closed & Bounded. Interestingly, in some applied contexts (when e.g. the linear space will always be R n ) this is used as the denition for compact set.). 3. All norms on a nite dimensional normed linear space induce the same topology on that space; that is, all norms dene (through their induced metric) the same open sets on that space. ( I.e., it does not matter what norms are used for purposes of answering the question what are the open sets, and consequently answering questions of what are the closed sets, bounded sets, what sequences convergent and to what limiting vectors, and all other questions that are dened in terms of open sets.) I followed the exercises on pages 22 through 25 of the following in order to develop these arguments: Carter, Michael. Foundations of Mathematical Economics. Cambridge: MIT Press, 200.
Lemma (the ratio of the total representation magnitude and the magnitude of any vector in a normed linear space is bounded): 2 Let S = {x, x 2,..., } be a linearly independent set of vectors in a normed linear space. Then there is a real number c > 0 such that for every x lins x c ( α + α 2 + + α n ) where x is expressed as x = α x + α 2 x 2 + + α n. Proof: Suppose for the sake of contradiction that c > 0, w lin{x, x 2,..., } such that w < c ( α + α 2 + + α n ), where w = α x + α 2 x 2 + + α n. Step. Since the above is true for all positive c, it is true for all c in the descending set { m m N}. So let us consider the sequence { m } m=. For every term in this aforementioned sequence we are guaranteed (by the supposition) a w m lins such that w m < αm + αm 2 + + αm n m where the αi m s are taken from the basis representation for w m. Choose any particular list of these w m to dene the sequence {w m }. 3 I seek to push the sequence of norms { w m } to tend to 0, but it is not clear that I can do that with the sum α m + α2 m + + αn m in the numerator of each norm. But I can dene another sequence {y m }, to be: given the chosen {w m } sequence, let y m = α m w + αm 2 + + αm m ( m), where the α i s are n taken from the basis representation for w m. Then the norm of each term in the sequence {y m } is y m = α + α 2 + + α n w m < α α + α2 + + αn + α 2 + + α n m = m, again where the am i s are from the basis representation for w m. Consequently, {y n } denes a sequence of vectors with corresponding sequence of norms y m < m 0 (as m ). Step 2. My expression for {y m } can be written as y m = β m x + β2 m x 2 + + βn m where the sum of the absolute values of the betas satises n i= βm i =.4 For each m, for each i =, 2, 3,..., n, we must then have βi m (simply because the sum of these numbers is and they are all positive, so none of 2 Comment: My statement here is an expression of a rewriting the claim as: M > 0 (where M = in the statement of c the claim) such that for any x 0, M α + α 2 + + α n x Of course another way to express the thing is that there is a real number c > 0 small enough so that for any x 0 c x α + α 2 + + α n it is clear that there must be a small enough choice of c (or large enough choice of M) that works globally in a particular given linear space using a particular basis. - Comment: To build my intuition, I tried nding such a c in R 2. Take, for example, the basis {(, 0), (0, )} for R 2. Then (x, y) R 2 is expressed as (x, y) = x(, 0)+y(0, ) by this basis. I used the Euclidean norm of (x, y), that is (x, y) = x 2 + y 2. Given any xed k = (x, y), then it is intuitively clear that to maximize x + y we must choose (x, y) to be on a 45 degree ray emanating from the origin. Since the same situation occurs in all four directions, we might as well suppose that x and y are on the positive 45 degree ray, and so x = y 0. In such a case then (x, y) = x 2 and x + y = 2x. We want to choose c so as to ensure that (x, y) c( x + y ), and this argument shows that in the worst case this expression will look like (for a given (x, y)) (x, y) = x 2 c(2x) = 2 2c (the inequality is denitely true if x = 0), and so it would suce to take c to be 2 2 (or 2 ). In three space (but same context), my intuition leads me to believe that a worst case would be again all coordinates being the same, resting upon the 45 degree ray in the positive octant. In such a case the expression boils down to x 3 3cx, which would be satised if 3 3c, that is, we can take c = 3 3 = 3. The pattern is interesting enough to make a conjecturethat taking c = n will work in the statement of the lemma in R n normed with Euclidean measurement using the standard basis vectors as our linearly independent spanning set. The lemma, of course, is much more general, since it deals with arbitrary normed linear spaces and arbitrary spanning sets. 3 That is, {x m} is a sequence that corresponds to the sequence { } in that each term xm in the {xm} sequence lists vectors m that are assumed to contradict the lemma we're trying to show when c is taken to be m. 4 They satisfy this sum because I dened each y m from the term w m by dividing w m by the exact necessary scalar for this to happen. To be more precise, let w m = α m x + α m 2 x 2 + + α m n xn be the basis representation for the mth term in the w sequence. 2
them can be bigger than ). The weights on the basis vectors used to write each term in the sequence {y m } individually dene sequences of real numbers, which we may denote {βi k} k=, i =, 2, 3,..., n. Each of these coordinate sequences is bounded (by in particular). We proceed as follows. Since the rst coordinate sequence, {β k }, is bounded, the Bolzano-Weierstrass theorem guarantees that it has a convergence subsequence. Then let p(m) be the strictly increasing sequence of natural numbers such that the subsequence {β p(m) } of the rst coordinate sequence converges. Denote the limit of this sequence by β. Apply p( ) to form sequence: y p(m) = β p(m) x + β p(m) 2 x 2 + + βn p(m) we know that the rst coordinate sequence of this new sequence converges to β, but we know little about the others so far. Yet, they are subsequences of their original coordinate sequences, and so they are bounded sequences of real numbers! So we are now guaranteed, again by the Bolzano-Weierstrass theorem, that the sequence of real numbers β p(m) 2 has a convergent subsequence. Let q be the function from the range of p(m) into the positive integers such that β q(p(m)) 2 converges to β 2. Now, apply this this function q to each of the coordinate sequences of y p(m) 's basis representation, obtaining a new sequence y q(p(m)) dened by: y q(p(m)) = β q(p(m)) x + β q(p(m)) 2 x 2 + + βn q(p(m)) Note that now β q(p(m)), being a subsequence of the convergent sequence β p(m), must still converge to β. Also, now have have pinned down the second coordinate, so that it converges to β 2. Continue this process. That is, the third coordinate sequence {β q(p(m)) 3 } in the above sequence of vectors is a bounded sequence of real numbers and therefore has a convergent subsequence. Use the mapping which gives this convergent subsequence, which is from the range of q(p(m)) into N, in order to restrict every coordinate sequence so that the third coordinate sequence converges. Note that rst and second coordinate sequences still converge. We can repeat this argument, n times in total, until every coordinate sequence converges. Let ym = β sm x + β sm 2 x 2 + + βn sm be the sequence of vectors thus obtained. Step 3. Now ym is a subsequence of y m. Since every term in the sequence y m had the property that the sum of the absolute values of the weights in the basis representation was, then every term in the sequence ym has this property as well. The sequence { β sm + βsm 2 + + βsm n } = {} converges to. We know that if the sequence {β sm } converges to β, then the sequence { β sm } converges to β. We also know that if we have any set of sequences all of which converge individually, then their sum converges to the sum of the individual limits. So then the sequence { β sm + βsm 2 + + βsm n } must converge to β + β2 + + βn. Since the limit of a sequence is unique, then in fact β + β2 + + βn =. So the sequence ym converges to the vector y = β x + β 2 x 2 + + β n where not all of the β i can be zero, since the sum of the absolute values of these coecients is. Now we have it. We know y m 0. Therefore, ym 0. Hence ym 0. Yet, {ym} converges to a vector which is a weighing of the basis vectors such that at least one weight must be nonzero. Therefore, {ym} actually does not converge to the zero vector. So there must always exist such a constant c, as claimed. Then we dened y m by y m = α m + αm 2 + + αm n wm = α m + αm 2 + + αm n (αm x + α m 2 x 2 + + α m n ) In the n = 2 case, saying that the absolute values of the betas sum to is the same as saying in the expression for y m above: α α + α 2 + α 2 α + α 2 = α + α 2 α + α 2 = It's clear that this argument can be generalized for arbitrary n N. 3
Claim (): 5 Every nite-dimensional normed linear space is complete. Proof: Let X be a nite dimensional normed linear space of dimension n. We must show that any arbitrary Cauchy sequence in X converges in X. Let {x m } be an arbitrary Cauchy sequence in X. Let {x, x 2,..., } be a basis for X. Then each term x m has a unique representation: x m = α m x + α m 2 x 2 + + α m n Let c be the constant guaranteed by the previous lemma. That is, let c be that constant so that for any x X, the product of c times the sum of the absolute value of the weights in the basis representation of x (by the basis I specied in the rst paragraph above) is no greater than the norm of x. Since {x m } is a Cauchy sequence in X, for any cɛ > 0 there exists an N N such that, for all integers r, m N, x m x r < cɛ Substituting the representations for x m and x r by the basis vectors, we obtain: (α m x + α m 2 x 2 + + α m n ) (α r x + α r 2x 2 + + α r n ) < cɛ some manipulation, and we obtain: [α m α r ] x + [α m 2 α r 2] x 2 + + [α m n α r n] < cɛ By the lemma we know that the constant c satises: c ( α m α r + α m 2 α r 2 + + α m n α r n ) < [α m α r ] x + [α m 2 α r 2] x 2 + + [α m n α r n] < cɛ so that α m α r + α m 2 α r 2 + + α m n α r n < cɛ/c = ɛ Now consider αi m αi r for any i =, 2,..., n. It is clear that this, being a term in the sum of all positive terms of the left hand side, is at least as small as the right hand side of the above inequality. Therefore, we can say for any i, αi m αi r α m α r + α2 m α2 r + + αn m αn r < ɛ This shows that given any ɛ > 0 we can nd a positive integer N so that for all m, r N the distance between the ith weight in the basis representation for x m and the ith weight in the basis representation for x r is strictly smaller than ɛ. That is, for each i, for any ɛ > 0, N N such that, for all m, r N, α m i α r i < ɛ Therefore each sequences of weights {α m i } m=, i =, 2, 3,..., n, (which we called the coordinate sequences in the lemma) are themselves Cauchy sequences of real numbers. Since R is complete, for each i, each such sequence {α m i } converges in R to some α i R. Dene x = α x + α 2 x 2 + + α n. Then x X. It is clear that x m x, since each of the coordinates in the basis representation for x m converges to the corresponding coordinate of x. This shows that every Cauchy sequence in X converges. Therefore, X is complete. 5 Comment: This argument rests on the fact that R is complete. In order to get this into the game, we heavily rely on that we've been taking our scalars from R. Given that we can target any point in our linear space by weighting a nite basis for that space, we can in fact think of our points as, essentially, these weights. Which is to say, our points are eectively in R n. The argument just expresses these thoughts more formally. 4
Claim (2): In a nite dimensional normed linear space, a set is compact if and only if it is closed and bounded. Proof: The proof that any compact set is both closed and bounded is on page 6 of Carter; I do not repeat the argument. That the converse holds in any nite dimensional normed linear space is the result I seek to show here. Let S be a closed and bounded subset of a nite-dimensional normed linear space X. Let X have the basis {x, x 2,..., }. Let {x m } be a sequence in S. We want to show that {x m } has a convergent subsequence (sequential compactness). Let x m = α m x + α m 2 x 2 + + α m n denote the basis representation of x m. Since S is bounded, then it is contained in an open ball. But in a normed linear space, every open ball is merely a translation and rescaling of the unit ball. So say S is contained in the open ball x 0 + rb, where r > 0 is a real number representing the scaling of the unit ball B, and x 0 is the translation of the unit ball. Then the magnitude of any vector in S can be no greater than M = x 0 + r. Therefore any term x m S has x m M, and so by the lemma there is a constant c such that c α m i c ( α m + α m 2 + + α m n ) < x m M for any i =, 2, 3,..., n; that is, αi m < M/c. Since M/c is a constant, this shows that each coordinate sequence {αi m} of the sequence {xm } is bounded. We are going to apply the same argument as in the lemma. Since the coordinate sequence {αi m} is a bounded sequence of real numbers it has a convergent subsequence {α p(m) i } that converges to α. Construct the subsequence formed by selecting coordinates: x p(m) = α p(m) x + α p(m) 2 x 2 + + αn p(m) such that α p(m) α, and who knows what the other coordinate sequences doexcept that they are bounded sequences of real numbers and therefore have convergent subsequences. So repeat the argument for the sequence {α p(m) 2 }, so that for some function q( ), α q(p(m)) 2 α 2. Continuing in this way we obtain a subsequence of {x m }, call it {x m } whose coordinate scalars converge to the vector (α,..., α n ). Let x = α x + α 2 x 2 + + α n. Then x m x because every coordinate in the former converges to the corresponding coordinate of the latter. Since S is closed, every convergent sequence whose terms all come from S converges to an element of S, so x S. Therefore the arbitrary sequence {x m } in S has a subsequence which converges in S. Hence, S is compact. 6 6 Speaking of convergence, the following conjecture arose when I was experimenting with some norms on R 2. I have not proven it. Conjecture: the sequence of norms { x 2n } n= converges to x, where x p = p x p + xp 2 + + xp n x = max{ x, x 2,..., } Comment: Looking at this conjecture a half of a year later, it seems even better to conjecture that p lim x p + x 2 p + + x p n p = max{ x, x 2,..., } Well, the CES production function looks like: p α x p + α 2x ρ 2 + + αnxρ n where the x i 0 and the α i > 0. As p, this function converges to min{x, x 2,..., } (this was proven by Dr. Elmer in February for the case n = 2). But the p-norm is just this production function with p >. 5
Lemma (exercise.22): 7 Let a and b be two norms on a given linear space X. Assume there are positive constants A and B so that for any x X, x a A x b x b B x a (I'm thinking of A as a number that will always stretch the vector x to greater magnitude when measured in norm b than it is when measured by norm A; and similarly for the number B.) Then the two norms a and b are topologically equivalent, by which I mean (not having studied any formal topology) that any open ball around a vector x that we construct by using norm a contains within it an open ball around x constructed using norm b, and the converse. Consequently every open set in X under a is open when X is analyzed under b, and the converse. Hence, all things dened in terms of the open sets on X, will result in the same notions, regardless of which norm we use. Proof: Let B a r (x) be an arbitrary open ball of radius r around the point x X constructed using norm a. Since every open ball in a normed linear space is the same as a rescaling and translation of the unit open ball, we may write B a r (x) = x + rb a for unit ball B a = {x x a < }. Consider A Bb = {x x b < A } = {x A x b < }. Now whenever x A Bb, then A x b <, and we know by the given equations that x a A x b <, hence x B a. Therefore A Bb B a. Then x + r A Bb x + rb a. Therefore any open ball under norm a has within it an open ball under norm b. On the other hand, if B b r(x) is an arbitrary open ball around x using norm b, then B Ba = {x x a < B } = {x B x a < } B b, and so x + r B Ba works as a suitable open ball contained within B b r(x), as desired. 7 Comment: I recall once seeing the hypotheses of this lemma taken as the denition of two norms being equivalent. It is sometimes phrased as: C, D, positive constants, so that x X: C x a x b D x a but dividing over and breaking these inequality into two parts leads to the assumption I have made. Intuitively, the phrasing above leads to this characterization: given any x, if you measure the magnitude of x using b, then, there is a constant quantity D you can stretch x by to make it at least as large as its magnitude under b, under norm a; and there is a constant quantity C you can squash x by in order to make its magnitude smaller under norm a than it is under norm b. Because C and D are constants, we can do this globally in the linear space; and so, any open ball under b will both be contained in, and contain within it, and open ball under norm a. This is enough for us to conclude that the norms are topologically equivalent, since then all the open sets under norm a will be the same under norm b, and that is all we need to dene the topological properties of X. [Note that these intuitive characterizations rest on the norm axiom: cx = c x = c x for c > 0, and cx is a rescaling of x.] 6
Claim (exercise.23): In a nite dimensional linear space, any two norms are topologically equivalent. Proof: Let a and b be two arbitrary norms in a nite dimensional linear space X with dim X = n. I will show that the previous lemma holds between these two norms using the lemma proven at the beginning of this paper, thus showing that these two norms are in fact topologically equivalent (as discussed). Let x X be arbitrary. Let {x, x 2, x 3,..., } be a basis for X. Then x = α x + α 2 x 2 + + α n for unique α i 's. 8 We know by our initial lemma that there are constants c a and c b such that x a c a [ α + α 2 + + α n ] on the other hand by the triangle inequality x b c b [ α + α 2 + + α n ] x a = α x + α 2 x 2 + + α n a α x a + α 2 x 2 a + + α n a max i n { x i a } n α i i= x b = α x + α 2 x 2 + + α n b α x b + α 2 x 2 b + + α n b max i n { x i b } n α i Let k a = max i n { x i a } and k b = max i n { x i b }. 9 For simplicity of notation, let n i= α i = M. Then the above four statements say: k a M x a c a M k b M x b c b M By using the diagonals of these inequalities we obtain the result we need. Since the above implies: k a M x a i= then rearranging we obtain: through similar logic we may obtain: Let ka c b = A and k b c a = B. Then we have: x b c b M k a c b x b x a k b c a x a x b x a A x b these A, B are the constants we sought. x b B x a 8 Comment: A hurdle (maybe) for understanding this argument is to realize that, given any x and basis set for X, the representation-by-basis for x is completely unrelated to the norm on X. The point (, 2) = (, 0) + 2(0, ) in R 2, regardless of whether our norm is Euclidean, taxicab, sup, etc. This is invariant to norm, but the constants c a and c b are clearly not (we xed the norm in the lemma where we obtained these constants, and we showed that they must exist for any norm, but they are denitely functions of the norm we choose). 9 Comment: The maximum norms in the basis set plays a pivotal role in this proof. They exist because we are in nite dimensional space. 7