MA3051: Mathematical Analysis II

MA3051: Mathematical Analysis II Course Notes Stephen Wills 2014/15 Contents 0 Assumed Knowledge 2 Sets and maps................................ 2 Sequences and series............................. 2 Metric spaces................................. 2 Complex analysis............................... 4 Linear algebra................................ 5 1 Motivation 6 2 Inner Product Spaces 8 3 Normed Vector Spaces 13 4 Completeness and Convexity 24 5 Orthogonality 31 6 Continuous/Bounded Linear Maps 39 7 Fourier Series 49

0 Assumed Knowledge The following is material that I believe that you have encountered already; consequently I won t be proving it again, nor (explicitly) asking you to reprove this stuff. The symbol K denotes the real (R) or complex (C) numbers. Sets and maps 1. A map f : X Y between sets X and Y is: (a) injective/one-to-one if f(x 1 ) = f(x 2 ) x 1 = x 2 ; (b) it is surjective/onto if y Y x X such that f(x) = y; (c) it is bijective if it is injective and surjective, equivalently if it is invertible, i.e. g : Y X such that (g f)(x) = x and (f g)(y) = y x X, y Y. Here g f denotes the composition of g and f: (g f)(x) = g(f(x)). 2. If f : X Y is a map then for A X and B Y, f(a) := {f(x) : x A} (the image of A) f 1 (B) := {x X : f(x) B} (the preimage of B). Preimages behave better than images; for any family of subsets (B i ) i I of Y f 1( i B ) i = i f 1 (B i ) and f 1( i B ) i = i f 1 (B i ). Also f 1 (B c ) = (f 1 (B)) c, where B c denotes the complement of B. On the other hand, for sets A i X, f ( i A ) i = i f(a i) and f ( i A ) i i f(a i) with the inclusion above not necessarily being an equality (here I use to denote any subset, denoting proper subsets with, rather than the / convention). 3. An infinite set X is countable if there is a bijection f : X N = {1, 2,...}, otherwise it is uncountable. If X 1, X 2,... are countable, so are X i and X 1 X N for each N N. Sequences and series 1. If (a n ) and (b n) are convergent sequences of numbers in K with limits a and b respectively then a n + b n a + b, a n b n ab, and, if b 0, a n /b n a/b. 2. If a n is a convergent series of numbers in K then a n 0; if b n is a convergent series then so is b n (the converse is not true). Metric spaces 1. A metric space is a set X together with a metric or distance function d : X X R such that (i) d(x, y) 0 for all x, y X, with equality if and only if x = y; 2

(ii) d(x, y) = d(y, x); (iii) d(x, y) d(x, z) + d(z, y) for all x, y, z X. This gives rise to a family of subsets called the open subsets of X, which are those subsets U X such that for each x U there is some r x > 0 such that B(x, r x ) := {y X : d(x, y) < r x } U. Here B(x, r x ) is the open ball of radius r x, centre x, and the choice of r x is allowed to depend on x. The closed subsets are the complements of the open sets. Unions of open sets are open, hence intersections of closed sets are closed. Intersections of finite numbers of open sets are open, with a corresponding statement for finite unions of closed sets. 2. A sequence (x n ) in a metric space converges to x if and only if one of the following equivalent conditions holds: (i) ε > 0 N 1 such that d(x n, x) < ε n N (ii) d(x n, x) 0 (in R) (iii) open sets U that contain x N 1 such that x n U n N. 3. Let A and B be subsets of a metric space X. The following conditions on B are equivalent: (i) B is the smallest closed set containing A (ii) x B, if U X is open and x U then U A (iii) x B a sequence (x n ) A such that x n x The set B that satisfies these conditions is the closure of A, denoted A (it always exists). Consequence: A is closed if and only if A = A, from this it follows that A being closed is equivalent to the condition that whenever (x n ) A is convergent to something in X then in fact we have lim x n A. n 4. A subset A of a metric space X is dense if A = X. X is separable if it contains a countable dense subset (finite or infinite). 5. A Cauchy sequence in a metric space is a sequence (x n ) X that satisfies d(x m, x n ) 0 (formally: ε > 0 N 1 such that d(x m, x n ) < ε m, n N). A metric space is complete if every Cauchy sequence is convergent. 6. A subset of a complete metric space is complete (in itself) if and only if it is closed. 7. If (X, d X ) and (Y, d Y ) are metric spaces, then the following are all metrics on X Y : d 1 ( (x1, y 1 ), (x 2, y 2 ) ) = d X (x 1, x 2 ) + d Y (y 1, y 2 ), d 2 ( (x1, y 1 ), (x 2, y 2 ) ) = (d X (x 1, x 2 ) 2 + d Y (y 1, y 2 ) 2 ) 1/2, d ( (x1, y 1 ), (x 2, y 2 ) ) = max{d X (x 1, x 2 ), d Y (y 1, y 2 )} 3

However, these all induce the same topology on X Y, the product topology, i.e. if U X Y is open with respect to d 1 then it is open with respect to d 2 and d, and so on. 8. (a) A map f : X Y between metric spaces is continuous at the point x X if any of the following equivalent conditions hold: (i) ε > 0 δ > 0 such that d X (x, x ) < δ d Y ( f(x), f(x ) ) < ε; (ii) For every sequence (x n ) that converges to x we have f(x n) f(x). (iii) For each open set V that contains f(x) there is an open set U containing x such that f(u) V ; (b) A map f : X Y between metric spaces is continuous if any of the following equivalent hold: (i) f is continuous at every x X; (ii) open V Y, f 1 (V ) is open in X; (iii) closed F Y, f 1 (F ) is closed in X. 9. The composition of continuous maps is continuous. [Note: (g f) 1 (A) = f 1( g 1 (A) ).] 10. The distance from a point x to a (nonempty) subset A X of a metric space is dist(x, A) = inf{d(x, a) : a A}. The map x dist(x, A) is continuous. 11. A subset C of X is compact if every open cover has a finite subcover. Equivalently, C is compact if every sequence (x n ) has a subsequence that converges to a point in C. The image of a compact subset under a continuous map is compact. A compact subset C of a metric space is closed and bounded (for each x X K > 0 such that C B(x, K)); the converse is true for subsets of K n (the Heine-Borel Theorem) but not in general. Complex analysis 1. A map f : A C C is differentiable at z A if lim h 0 h 1( f(z + h) f(z) ) exists, in which case it is denoted f (z). It is analytic/holomorphic at z if there is some open ball or open set U containing z such that f is differentiable at each w U. If f is analytic on and inside a contour C (a closed curve in the complex plane without too many corners) then Cauchy s Integral Formula states that f(a) = 1 2πi C f(z) z a dz for any a from the interior of the contour, where the integral is carried out anticlockwise around the contour. 4

Linear algebra 1. A subset U of a vector space V is a subspace if either of the following equivalent conditions hold: (i) x + y U x, y U and λx U λ K, x U; (ii) λx + µy U λ, µ K, x, y U. 2. If U and W are subspaces of V then so are U W and U + W := {u + w : u U, w W }. [U W is the largest subspace in both U and W ; U +W is the smallest subspace containing U and W.] Other notation: for A V, x V and λ K, A+x := {a+x : a A} (translation of A by x) and λa := {λa : a A} (scaling of A by λ). 3. If S V, its linear span is the set Lin S := { n λ ix i : n N, λ i K, x i S}, the set of all linear combinations of (finite subsets of) S. It is the smallest subspace that contains S. 4. A map T : V W between vector spaces is linear if any of the following equivalent conditions holds: (i) T (x + y) = T x + T y x, y V and T (λx) = λt x λ K, x V ; (ii) T (λx + µy) = λt x + µt y λ, µ K, x, y V ; (iii) T (x + λy) = T x + λt y λ K, x, y V. The composition of linear maps is linear. The range of T, Ran T := T (V ), is a subspace of W ; the kernel of T, Ker T := {x V : T x = 0}, is a subspace of V. 5. If K = C then a map S : V W is conjugate linear if S(x + y) = Sx + Sy and S(λx) = λsx. 6. A subset S of V is linearly independent if whenever n λ ix i = 0 for n N, λ i K and distinct elements x i of S, then λ 1 = = λ n = 0. It is linearly dependent if it is not linearly independent. 7. If V = Lin S for a finite subset S V then V is finite dimensional, with dim V equal to the size of any linearly independent set S such that Lin S = V ; such sets all have the same size, which is the size of the largest linearly independent set contained in V ; these sets are bases of V. V is infinite dimensional if no such finite set exists, equivalently if it contains an infinite linearly independent set. If dim V = n < then there is a linear bijection (more usually called a linear isomorphism) T : V K n. An example of such a T is obtained by picking a basis {v 1,..., v n } and setting T ( n λ iv i ) = (λ 1,..., λ n ). 8. Rank-nullity formula: if V is finite dimensional and T : V W is linear then dim V = dim Ran T + dim Ker T. 9. If V and W are vector spaces, then V W is a vector space if we define (v 1, w 1 ) + (v 2, w 2 ) = (v 1 + v 2, w 1 + w 2 ), λ(v, w) = (λv, λw). 5

1 Motivation Consider the following problem of an applied mathematical nature: a chain of length L is suspended vertically from a hook, and then subjected to a small perturbation and released from rest. Find the subsequent displacement of the chain. Applying the laws of physics and standard techniques of applied mathematics one obtains the following equations of motion: 2 u t 2 = x ( x u x where u(x, t) is the horizontal displacement of the chain a distance x from the bottom at time t. We have assumed that the chain has uniform density. We also have boundary/initial conditions: u(l, t) = 0, 0 t <, u(x, 0) = u 0 (x), 0 x L, u (x, 0) = 0, t 0 x L sup u(x, t) <, 0 x L, 0 t < A standard technique to solve this now is to apply the technique of separation of variables, i.e. to look for a solution of the form One then obtains and so ) u(x, t) = f(x)g(t). f(x)g (t) = xf (x)g(t) + f (x)g(t) g (t) g(t) = xf (x) f(x) + f (x) f(x). Both sides must equal a constant, λ say, since the left-hand side is a function of t only whereas the right-hand side is a function of x only. Thus we have to solve g (t) + λg(t) = 0 and d ( x df ) + λf(x) = 0. dx dx The first equation has general solution g(t) = A cos λt + B sin λt, and the second can be solved by use of Bessel functions. However, when we start to fit the boundary conditions we find that only certain values λ 1 < λ 2 < are allowable, depending on L, giving an associated sequence u j (x, t) = f j (x)g j (t), where f j and g j are the solutions for this λ j. But now we want to write the general solution of the PDE as a sum of the normal modes u j, that is, write u(x, t) = α j u j (x, t) for constants α j chosen suitably so that we can fit the boundary conditions. Several questions should be apparent: j=1 6

1. In what sense does the series converge? 2. Are there any restrictions on the initial displacement? For example, what guarantees that the solution can be expressed as a sum of normal modes? In particular are there enough? What does enough mean in this context? 3. Most fundamentally why can we add normal modes together to get another solution of the PDE? The answer to the last one is because both the PDE and the related ODEs are linear, i.e. the map u u tt is linear, etc. Both of the ODEs are examples of Sturm Liouville problems, and were solved in the 1830 s long before the advent of Hilbert space theory. However, as soon as we want to generalise the above problem then the methods of Hilbert space theory, and functional analysis more generally, gives a framework that makes this reasonably straightforward. Problems of this nature were the inspiration for the creation of functional analysis which combines the methods of analysis and linear algebra, in particular when dealing with infinite-dimensional vector spaces of functions. Note that the maps S : g g and T : f d ( x df ) dx dx are both linear maps, i.e. T (f 1 + µf 2 ) = T f 1 + µt f 2, and to say that f satisfies the ODE above amounts to saying T f + λf = 0 T f = λf, that is, λ is an eigenvalue of T with f an associated eigenvector. More generally, the set of solutions of any homogeneous, linear ODE (e.g. the types of equation considered in MA2054) form a subspace of the vector space of suitably differentiable functions. For example, solutions to y + 6y 7y = 0 form a subspace X of C 2 [0, 1]. If we turn this into a nonhomogeneous problem by changing the right-hand side as follows: y + 6y 7y = f, then the set of solutions becomes g + X, where g is any solution to the nonhomogeneous problem. Subsets of vector spaces of this form, vector + subspace are called affine-linear spaces. Other examples where we have a need to consider a combination of both metric and linear algebraic ideas is when we consider the collection of all probability measures/distributions on a set Ω. The set of such measures is not a subspace, nor an affine-linear space, but is convex. See Example 4.11 for more details. 7

2 Inner Product Spaces Recall the scalar product of two vectors in R 3 : We have (a 1, a 2, a 3 ).(b 1, b 2, b 3 ) = a 1 b 1 + a 2 b 2 + a 3 b 3. a.b = a b cos θ, where θ is the angle between the vectors a and b, and a = (a 2 1 + a2 2 + a2 3 )1/2 is the length of a. In particular we get with a b a.b a b a.b a b a.b = 0 a, b are orthogonal/perpendicular. We want to generalise this structure to other vector spaces. All our vector spaces will be real or complex, and we will use K to denote R or C. Definition 2.1. An inner product on a vector space X over K is a map, : X X K that satisfies: IP1. x, x 0 for all x X, with equality if and only if x = 0; IP2. x, y + λz = x, y + λ x, z for all x, y, z X and λ K; IP3. x, y = y, x for all x, y X. The space X together with, is called an inner product space, or a pre-hilbert space. Remarks. (i) The three properties are usually known as positivity, linearity and symmetry respectively. (ii) When K = R, (iii) just means x, y = y, x, since these are real numbers. Lemma 2.2. If X is a complex inner product space then for any x, y, z X and λ C we have x + λy, z = x, z + λ y, z. Proof. We have as required. x + λy, z = z, x + λy = z, x + λ z, y = z, x + λ z, y = x, z + λ y, z Exercise 2.3. Show that for a real inner product space we have x + λy, z = x, z + λ y, z. 8

Property IP2 of an inner product space just says that the map y x, y is linear for each fixed choice of x. Lemma 2.2 then says that for a complex inner product space the map x x, y is conjugate linear, but it is linear in the case of a real inner product space. Health warning: Some authors use the opposite convention regarding which variable is linear and which is conjugate linear for complex inner product spaces. We can summarise this by saying that the map, : X X K is bilinear (linear in both arguments) if K = R, and sesquilinear (1 1 2-times linear) if K = C. One immediate consequence is that for any x X we have x, 0 = 0, x = 0. To see this put z = y and λ = 1 in property IP2 of the definition to get and similarly for 0, x. x, 0 = x, y y = x, y x, y = 0, Lemma 2.4. Let X be an inner product space. If x, z = y, z for all z X then x = y. Proof. We have 0 = x, z y, z = x y, z for all z X. In particular this is true when z = x y, and so we have x y, x y = 0 x y = 0 x = y. Example 2.5. 1. K n equipped with the usual inner product: (λ 1,..., λ n ), (µ 1,..., µ n ) = n λ i µ i. 2. C ( [a, b]; K ) = {f : [a, b] K f continuous} with the vector space operations (f + g)(x) = f(x) + g(x), (λf)(x) = λf(x), and equipped with inner product f, g = b a f(t)g(t) dt. More generally, if w C[a, b] with w(t) > 0 for all t [a, b], then f, g w = b a f(t)g(t)w(t) dt is an inner product, involving the weight function w. In the original example we have w(t) = 1 for all t [a, b]. Most of the inner product definitions are easy to check, but you should convince yourself of the following fact: if f : [a, b] K is continuous with b a f(t) dt = 0 then f(t) = 0 for all t [a, b]. 9

3. P n = {polynomials of degree n}, z 0,..., z n C distinct points, and p, q := n p(z i )q(z i ). i=0 Again we need an argument to show why p, p = 0 only happens if p 0. This time use the Fundamental Theorem of Algebra to factorise p into linear factors, of which there can be at most n. 4. On the space M m,n (K) of m n matrices with entries from K we can define A, B = tr(a B) where A denotes the conjugate transpose of A, and tr C = n c ii for any n n matrix C, the trace of C. This example can be viewed as a special case of the first one, by forming a vector of length mn out of the rows of each matrix in M m,n (K). 5. An important infinite-dimensional example is { } l 2 = x = (x n ) : x n K, x n 2 <, the set of square-summable sequences, with inner product x, y = x n y n. To ensure that this one works we first need to check that l 2 is a vector space under the operations: x + y = (x n + y n ), λx = (λx n ). This is postponed for now. 6. Another pair of infinite-dimensional examples that appear in control problems in electrical engineering are RL 2 := {f(z) : f is rational and analytic on the unit circle z = 1}, and its subspace RH 2 := {f(z) : f is rational and analytic on the closed unit disc z 1}, This time we can take as inner product f, g = 1 2πi z =1 f(z)g(z) dz z noting that the integrand is continuous on the circle, so the integral makes sense, and the inner product space properties can be verified just as in part 2. This example has an advantage over part 2 in that the integrals/inner products can be computed easily using Cauchy s Integral Formula. 10

Given an inner product space X, we define x := x, x [0, ) for each x X. For our example of the usual scalar product on R 3 we have a = a, the usual Euclidean length. Proposition 2.6. Let X be an inner product space and x, X, λ K. Then: (i) x ± y 2 = x 2 ± 2 Re x, y + y 2 for all x, y X. superfluous when K = R.) (Note that Re is (ii) x 0, with x = 0 if and only if x = 0. (iii) λx = λ x. Proof. Immediate from the definition of inner products. For example x ± y 2 = x ± y, x ± y = x, x ± y ± y, x ± y = x, x ± x, y ± y, x + (±1) 2 y, y = x 2 ± 2 Re x, y + y 2. Similarly λx = λx, λx = λλ x, x = λ 2 x, x = λ x. We next prove that the inequality satisfied by the usual scalar product on R 3 carries over to general inner product spaces. Theorem 2.7 (Cauchy Schwarz Inequality). Let X be an inner product space. Then for all x, y X we have x, y x y, with equality if and only if x, y are linearly dependent. Proof. Since x, y = 0 if x = 0 or y = 0, we may assume both vectors are nonzero, so that x, y > 0. Pick θ [0, 2π) such that e iθ x, y = x, y. Then for all r R we have 0 re iθ x y 2 = re iθ x 2 + 2 Re re iθ x, y + y 2 = re iθ 2 x 2 2r Re e iθ x, y + y 2 = r 2 x 2 2r x, y + y 2. Thus the quadratic in r has at most one root. It discriminant is ( 2 x, y ) 2 4 x 2 y 2 = 4 ( x, y 2 ( x y ) 2) and so we have Moreover, discriminant 0 x, y x y. x, y = x y discriminant = 0 repeated root re iθ x y = 0 for some r x, y linearly dependent. 11

Proposition 2.8. Let X be an inner product space, and x, y X. Then x + y x + y. Proof. Using the Cauchy Schwarz inequality we get and the result follows. x + y 2 = x 2 + 2 Re x, y + y 2 x 2 + 2 x, y + y 2 x 2 + 2 x y + y 2 = ( x + y ) 2, This now allows us to give the details for one of our examples. Example 2.9. The space l 2 of square-summable sequences is a vector space, and the inner product is well-defined. To see this let x, y l 2, so that x = (x n ), y = (y n) with x n 2 < and y n 2 <. We must show x + y = (x n + y n ) l2, so choose some N N and consider x (N) = (x 1,..., x N ), y (N) = (y 1,..., y N ) K N, the truncations of x and y to N-dimensional vectors. We can apply Proposition 2.8 to x (N) + y (N) to get i.e. x (N) + y (N) 2 x (N) 2 + y (N) 2 N N N x n + y n 2 x n 2 + y n 2. Now let N on the right-hand side to get N x n + y n 2 x n 2 + y n 2 which is valid for all N N. It follows that x n +y n 2 <, and so x+y l 2 as required. We can also show easily that if x l 2 then λx = (λx 1, λx 2,...) l 2. Finally, if x, y l 2 then ( N N ) 1/2 ( N ) 1/2 ( ) 1/2 ( ) 1/2 x n y n x n 2 y n 2 x n 2 y n 2 < for all N N, by applying the Cauchy Schwarz in equality in K N to the truncations. Consequently the series x ny n is absolutely convergent in K, hence convergent. 12

3 Normed Vector Spaces The properties satisfied by the quantity x defined at the end of the last section, which we think of as the length of x, are the basis of a theory more general than just inner product spaces. Definition 3.1. A normed vector space (NVS) over K is a vector space X over K together with a map : X R satisfying the following: N1. x 0 for all x X, with x = 0 if and only if x = 0. N2. λx = λ x for all x X, λ K. N3. x + y x + y for all x, y X. So we can summarise our work above as follows: Proposition 3.2. If X is an inner product space then it is a normed vector space when we define x = x, x. Of course, we should give some examples of normed vector spaces, and determine if we have anything new from this definition. Example 3.3. 1. Let X = K n = {x = (x 1,..., x n ) : x i K}, vector space of n-tuples (which is n-dimensional). Three possible norms on X are x 1 = n ( n x i, x 2 = x i 2) 1/2, x = max x i. 1 i n That these are norms is easy to show; that 2 defines a norm follows from Propositions 2.6 and 2.8. 2. For any a < b in R let X = C([a, b]; K) = {f : [a, b] K f is continuous}, as in part 2 of Example 2.5. Possible norms on X are f 1 = b a ( b 2 f(x) dx, f 2 = f(x) dx), f = sup{ f(x) : a x b}. That these are well-defined follows since [a, b] is compact in R, so the integrals make sense. 3. X = C(R; K) = {f : R K f is continuous} is a vector space in the same way as part 2., but defining a norm is not easy. It is better to consider subspaces such as C c (R; K) = {f X : M > 0 such that f(x) = 0 x > M}, C 0 (R; K) = {f X : f(x) 0 as x }, C b (R; K) = {f X : C > 0 such that f(x) C x R}, a 13

i.e. the continuous functions of compact support, the continuous functions that vanish at infinity, and the bounded continuous functions, respectively. Note that C c (R; K) C 0 (R; K) C b (R; K) C(R; K), and each is a proper subspace of the next. Moreover, we can define a norm on the first three by setting f := sup x R f(x), but this is not well-defined on all of X. In particular some of the examples above are those coming from inner product spaces, since in part 1 we have x 2 = x, x and in part 2 we have f 2 = f, f for the inner products on K n and C[a, b] given in Example 2.5. However, some of the above are not inner product spaces, which can be shown by adapting an elementary result of Euclidean geometry (based on Pythagoras Theorem) which asserts that for vectors x, y in the plane we have x + y 2 + x y 2 = 2 ( x 2 + y 2). (3.1) Definition 3.4. A normed vector space X is said to satisfy the parallelogram law if (3.1) holds for all x, y X. Theorem 3.5. Let X be a normed vector space. The norm on X arises from an inner product if and only if X satisfies the parallelogram law. The inner product is related to the norm through the polarisation identities: x, y = 1 4( x + y 2 x y 2) if K = R, x, y = 1 4 3 n=0 1 i n x + in y 2 = 1 4( x + y 2 i x + iy 2 x y 2 + i x iy 2). if K = C Proof. Suppose X is an inner product space, then x + y 2 + x y 2 = ( x 2 + 2 Re x, y + y 2) + ( x 2 + 2 Re x, y + y 2) = ( x 2 + 2 Re x, y + y 2) + ( x 2 2 Re x, y + y 2) = 2 ( x 2 + y ) 2. Verifying that the inner product can be written in terms of the norm through the polarisation identities is shown similarly. Conversely, suppose X is a normed vector space satisfying the parallelogram law, that K = R, and define, : X X R by x, y = 1 4( x + y 2 x y 2). 14

Then x, x = 1/4 2x 2 = x 2 0 with equality if and only if x = 0. Also, x, y = y, x is easy to see. For any x, y, z X we have x, y + x, z = 1 ( x + y 2 + x + z 2 ( x y 2 + x z 2 ) ) 4 = 1 1 4( 2 { 2x + y + z 2 + y z 2 } 1 ) 2 { 2x y z 2 + y z 2 } = 1 ( 2 2 x + y + z + 22 x y + z ) 4 2 2 2 2 = 2 x, y + z. 2 However, for any w X we have x, 2w = 1 ( x + 2w 2 x 2w 2) = 1 (x + w) + w 4 4( 2 (x w) w 2) = 1 ( {2( x + w 2 + w 2 ) x 2 } {2( x w 2 + w 2 ) x 2 } ) 4 = 2 x, w. Hence we have x, y + x, z = x, y + z, i.e. the map is additive in the second argument. Now for all m, n N we get m x, y = x, my = x, n m n y = n x, m n y, so that q x, y = x, qy for all q Q, q > 0. But it follows readily from the definition that x, 0 = 0, and that x, y = x, y, and so we now get q x, y = x, qy q Q, x, y X. Finally, using density of Q in R, and continuity of norms, we can replace λ Q by any λ R, giving linearity in the second argument. A similar argument works if K = C. Example 3.6. The normed vector spaces (K n, 1 ), (K n, ), (C[a, b], 1 ) and (C[a, b], ) are not inner product spaces. For example if we take x = (1, 0, 0,..., 0) and y = (0, 1, 0,..., 0) in K n then x + y 2 1 + x y 2 1 = (1, 1, 0,..., 0) 2 1 + (1, 1, 0,..., 0) 2 1 = 2 (1 + 1) 2 = 8, whereas 2 ( x 2 1 + y 2 1) = 2(1 2 + 1 2 ) = 4. Similar calculations work for the other cases. Theorem 3.7. If (X, ) is a normed vector space then d(x, y) := x y defines a metric on X that satisfies (i) d(x + z, y + z) = d(x, y) x, y, z X; (ii) d(λx, λy) = λ d(x, y) x, y X, λ K. Conversely, if X a vector space over K and d a metric on X that satisfies (i) and (ii) then defining x := d(x, 0) turns X into a normed vector space. 15

Proof. If X is a normed vector space then d(x, y) = x y 0, with equality iff x y = 0 x = y; d(x, y) = x y = ( 1)(y x) = 1 y x = y x = d(y, x); and d(x, y) = x y = (x z) + (z y) x z + z y = d(x, z) + d(z, y), so that d really is a metric on X. Moreover, d(x + z, y + z) = (x + z) (y + z) = x y = d(x, y); and d(λx, λy) = λx λy = λ(x y) = λ x y = λ d(x, y). The converse, starting from a vector space equipped with a metric that satisfies (i) and (ii), is proved similarly. Since a norm induces a metric, we can talk about open and closed balls: if (X, ) is a normed vector space then we will write B(x, r) := {y X : d(x, y) < r} = {y X : x y < r} and B(x, r) := {y X : x y r}, the open (respectively closed) ball of radius r and with centre x. In particular B(0, 1) is the open unit ball, and B(0, 1) is the closed unit ball. Note that y B(x, r) y x < r y x < 1 r y x r B(0, 1) y x + rb(0, 1), so that the ball B(x, r) is obtained from the open unit ball by scaling and translation. Here we have used some standard notation for operations defined on a subset A of a vector space V : given v V and λ K we set v + A := {v + w : w A}, λa := {λw : w A}, i.e. the subsets obtained from A by translating all vectors in A by v, or, respectively, scaling all vectors in A by λ. Definition 3.8. Let X be a normed vector space. A subset A X is bounded if there is some K 0 such that x K for all x A, i.e. A B(0, K). Proposition 3.9. Let X be a normed vector space. (i) Let (x n ) be a sequence in X, x X. Then x n x x n x 0 (in X) x n x 0 (in R). (ii) The map x x from X to R is continuous. (iii) If (x n ), (y n ) X are sequences with x n x and y n y then x n +y n x+y. If (λ n ) K with λ n λ then λ n x n λx. 16

Proof. (i) By definition x n x in X d(x n, x) 0 in R d(x n x, 0) = x n x 0 in R x n x 0 in X (by translation invariance) (ii) Follows from the reverse triangle inequality: x y x y. (iii) Fix ε > 0. There is some N such that d(x n, x) < ε/2 and d(y n, y) < ε/2 for all n N. But then d(x n + y n, x + y) = (x n + y n ) (x + y) = (x n x) + (y n y) x n x + y n y < ε/2 + ε/2 = ε for all n N, as required. Scalar multiplication follows similarly. Part (iii) shows that the maps X X (x, y) x + y X and K X (λ, x) λx X are continuous (when X X and K X are given their product topology, generated by many different natural metrics on these cartesian products), i.e. the metric and (linear) algebraic aspects of X are in sync. More is true when we specialise to inner product spaces. Corollary 3.10. If X is an inner product space then the map, : X X K is continuous. Proof. Suppose (x n, y n ) (x, y) in X X, i.e. x n x and y n y. Then, by the triangle inequality (in K) and the Caucy-Schwarz inequality, x n, y n x, y = x n x, y n y + x, y n y + x n x, y as n. x n x y n y + x y n y + x n x y 0 We will need to spend some time dealing with subspaces of normed vector spaces in various guises. Recall the following: Definition 3.11. (a) A subset Y of a vector space X is a subspace if x + λy Y for all x, y Y and λ K. Equivalently, Y is closed under addition and scaling. (b) A subset Y of a metric space X is closed if X \ Y is open, equivalently Y is closed if whenever (x n ) Y is a convergent sequence then the limit, x say, must belong to Y. (c) If Y is a subset of a metric space X then its closure, denoted Y, is the smallest closed subset of X that contains Y. Note that y Y if and only if there is some sequence (y n ) every ε > 0 there is some y Y such that d(y, y ) < ε. (d) A subset Y of a metric space X is dense if X = Y. Y such that y = lim n y n, equivalently if and only if for Since a normed vector space is both a vector space and a metric space, it makes sense to talk about closed subspaces of X. We will show shortly that every finitedimensional subspace of a normed vector space is always closed, but the same need not be true of infinite-dimensional subspaces. 17

Example 3.12. Let c 00 denote the set of all eventually zero sequences. That is x = (x n ) c 00 N = N x N such that x n = 0 n > N. This is a subset of l 2 since x c 00 x = (x 1,..., x N, 0, 0,...) for some N N x n 2 = x n 2 <. Moreover it is a subspace: if x, y c 00 then we can choose N x, N y such that x n = 0 for all n > N x and y n = 0 for all n > N y. If we set N = max{n x, N y } then for all n > N we have (x + λy) n = x n + λy n = 0 + λ 0 = 0 so that x + λy c 00. However, it is not closed as a subspace of l 2. Indeed, consider the sequence x = (1, 1/2, 1/3,...) = (1/n) which is in l 2 since n 2 <, but is clearly not in c 00. However, for each N N let x (N) = (1, 1/2,..., 1/N, 0, 0,...) c 00. Then (x (N) ) N=1 c 00 is a sequence (of sequences!) and, moreover, x x (N) 2 = (0, 0,..., 0, 1/(N + 1), 1/(N + 2),...) 2 = ( n=n+1 ) 1/2 1 n 2 0 as N, since we have the tail of a convergent series. Thus the sequence in c 00 converges to a point outside, and so c 00 is not closed. It is not hard to show, however, that c 00 is a dense subspace of l 2, i.e. c 00 = l 2. Proposition 3.13. Let X be a normed vector space and A X a subset of X. (a) There is precisely one subspace of X, called the linear span of A (denoted Lin A) that satisfies any of the following equivalent properties: (i) Lin A is the smallest subspace that contains A; (ii) Lin A is the intersection of all subspaces that contain A; (iii) x Lin A if and only if x = k α is i for some k N, α i K and s i A. (b) There is precisely one subspace of X, called the closed linear span of A (denoted LinA) that satisfies any of the following equivalent properties: (i) LinA is the smallest closed subspace that contains A; (ii) LinA is the intersection of all closed subspaces that contain A; (iii) LinA is the closure Lin A of Lin A, i.e. x Lin A if and only if x = lim for some sequence (x n ) Lin A. n x n 18

Proof. Exercise. Remarks. (i) Note that X is always a subspace of itself, and always closed, so X is always a closed subspace that contains S, no matter what we choose for S. Hence there is always at least one subspace in these intersections. (ii) Each vector x n in (b) (iii) can be written as x n = k n α(n) i s(n) i, i.e. as a finite linear combination of vectors from S, but the number of s(n) i is allowed to vary between the different x n. (iii) For Example 3.12 if we write δ (n) := (0, 0,..., 0, 1, 0,...) where the 1 is in the nth position, and D = {δ (n) : n N}, then it follows that Lin D = c 00, and so Lin D = l 2. That is, the linear span of the sequence δ (n) give a dense subspace of l 2. Example 3.14. Consider the space C[0, 1] of continuous real-valued functions on [0, 1]. A subspace of C[0, 1] is the subspace P of all polynomials. One function that does not belong to P is f(x) = e x. However, this function has a power series representation: x n f(x) = n! and this series converges uniformly on any bounded interval of R since the radius of convergence is infinite. What does this say in terms of norms? Suppose we give C[0, 1] the supremum norm, and write f k (x) = k n=0 xn /n!, then f f k = sup n=0 x [0,1] n=k+1 x n n! 0 as k, i.e. f P, the closure of P, with respect to this norm. Example 3.15. In the previous example we saw that the exponential function belongs to the closure of the subspace P of polynomials, with respect to the supremum norm. In fact more is true courtesy of the Weierstrass Approximation Theorem (Theorem 7.8): P is in fact a dense subspace of (C[a, b], ) for any a < b. From this it is not hard to show that P is also dense in (C[a, b], 1 ) and (C[a, b], 2 ). The definition of linear independence for a not necessarily finite-dimensional vector space is based on the finite-dimensional version. Definition 3.16. Let V be a vector space over K. A subset A V is linearly independent if for every finite subset F = {v 1,..., v n } A we have n α i v i = 0 α 1 = = α n = 0. That is, every finite subset of A is linearly independent in the MA2055 sense. 19

Example 3.17. The subset {δ (n) } of l 2 and the subset {f n C[a, b] : f n (t) = t n } are both easily seen to be linearly independent, but their spans are not all of l 2 and C[a, b] respectively, so these are not bases in the usual sense. However, they do have dense linear spans, which is nearly what the usual definition requires. Definition 3.18. Let X and Y be normed vector spaces. They are isometrically isomorphic if there is a linear map U : X Y that is onto, and such that Ux = x (i.e. U is an isometry, since it preserves lengths). Proposition 3.19. Suppose that U : X Y is a linear isometry. injective. The U is Proof. If x, x X such that Ux = Ux then and so x = x, i.e. U is injective. 0 = Ux Ux = U(x x ) = x x Thus any surjective linear isometry U is bijective, and so invertible. Moreover, U 1 is then automatically linear and isometric. If X is a finite-dimensional vector space, and T : X X is a linear map then the Rank-Nullity Formula states: It follows that dim X = dim Ran T + dim Ker T. T onto Ran T = X dim Ran T = dim X dim Ker T = 0 Ker T = {0} T one-to-one, in which case T is bijective. The same is not true in infinite-dimensions; for example let X = l 2 and consider the map T : l 2 l 2 given by T (x 1, x 2, x 3,...) = (0, x 1, x 2, x 3,...). The map T is known as the right-shift operator. It is easy to see that T is linear, and that T is isometric ( T x = x for every x), hence T is injective, but it is not surjective (e.g. (1, 0, 0,...) / Ran T ), hence not invertible. Proposition 3.20. Let X and Y be inner product spaces, and U : X Y a linear map. Then U is isometric if and only if it preserves inner products, that is Ux, Uy = x, y x, y X. Proof. Exercise: one way is very easy, the other uses polarisation. Example 3.21. The spaces (R 2, 1 ) and (R 2, ) from Example 3.3 are isometrically isomorphic. One way to see this is to draw their closed unit balls. These are both squares, so there is an invertible linear map that takes one square onto the other; this map will define an isometry (check!). 20

Definition 3.22. Let X be a vector space, and let 1 and 2 be norms on X. The norms are called equivalent if there are constants 0 < a b such that for all x X. a x 1 x 2 b x 1 Example 3.23. The norms 1, 2 and on K n are all equivalent. Indeed, it can be shown that for all x K n. x x 2 x 1 n x 2 n x Proposition 3.24. The relation of equivalence of norms defines an equivalence relation on the set of all norms on a vector space X. Proof. Exercise. Proposition 3.25. Let 1 and 2 be equivalent norms on a vector space X, and let (x n ) be a sequence in X. The sequence converges with respect to 1 if and only if it converges with respect to 2, in which case the limit is the same in both cases. Proof. Suppose that (x n ) converges to x with respect to 1, then 0 x x n 2 b x x n 1 b 0 = 0 as n 0, and so (x n ) converges to x with respect to 2. Conversely, if (x n ) converges to x with respect to 2, then 0 x x n 1 a 1 x x n 2 a 1 0 = 0 as n 0, and so (x n ) converges to x with respect to 1. Example 3.26. Returning to C[0, 1], note that for any g C[0, 1] we have for all t [0, 1]. It follows that g 1 = g(t) sup g(x) = g x [0,1] 1 0 g(t) dt 1 0 g dt = g. Consequently f(x) = e x is in the closure of P, the subspace of polynomials, with respect to 1 since f f k 1 f f k 0 as k where f k (x) = k xn /n! is the truncation of the Maclaurin series. However, it can be shown that there is no constant a > 0 such that a g g 1 g C[0, 1], and so the norms 1 and are not equivalent. A similar result is true if we try to compare the norms and 2. 21

Corollary 3.27. If 1 and 2 are equivalent norms on a vector space X then any subset A X is open with respect to 1 if and only if it is open with respect to 2. Proof. We have the following equivalences: A open w.r.t. 1 X \ A closed w.r.t. 1 if (x n ) X \ A converges to x w.r.t. 1 then the limit x X \ A if (x n ) X \ A converges to x w.r.t. 2 then the limit x X \ A X \ A closed w.r.t. 2 A open w.r.t. 2 Exercise 3.28. Construct another proof based on nesting of open balls defined with respect to the two norms, where the radii depend on the constants a and b. Corollary 3.29. If 1 and 2 are equivalent norms on a vector space X then any subset A X is bounded with respect to 1 if and only if it is bounded with respect to 2. Proof. Exercise. Corollary 3.30. If 1 and 2 are equivalent norms on a vector space X and f : X Y is a map into another metric space Y, then f is continuous with respect to 1 if and only if it is continuous with respect to 2. Proof. Exercise. Next, an important result that says that from a topological point of view it is immaterial which norm we put on a finite-dimensional vector space. Theorem 3.31. All norms on a finite dimensional normed vector space are equivalent. Proof. Let X be vector space with dim X = n, and pick a basis e = {e 1,..., e n } of X. Consider the invertible linear map T : K n X given by T α := n α ie i, where α = (α 1,..., α n ). This allows us to define a norm on X by ( n x e := T 1 x = α i 2) 1/2 n if x = α i e i. Now take any other norm X on X. The maps f i : α α i e i are all continuous (K n, 2 ) (X, X ), being compositions of the map K n α α i K and the map K λ λe i., Hence so is the map F : K n n X [0, ), F (α) = α i e i = T α X, since it is the sum of f 1,..., f n, composed with the (new) norm. The unit sphere S := {α K n : α 2 = 1} K n is closed and bounded, hence compact (by the 22

Heine Borel Theorem), so its image under F is compact, thus a := inf α S F (α) and b := sup α S F (α) exist and are attained. In particular, since α 0 for all α S, F (α) > 0 for all α S, and so 0 < a b. But from this we get a T α X b α S a β 2 = a T β e T β X b β 2 = b T β e β K n. The latter equivalence follows by writing α = β/ β 2 to get α S, when β 0, and noting that when β = 0 we have equality throughout the last line. Thus every norm on X is equivalent to e, and the result now follows since equivalence of norms is an equivalence relation. Recall that a subset A of a metric space X is compact if every sequence (x n ) A has a subsequence that is convergent to some point in A. It follows that every compact set must be closed and bounded, but the converse is not always true. Example 3.32. Consider the closed unit ball B(0, 1) of l 2. This closed and bounded subset is not compact. To see this consider the sequence (δ (n) ) B(0, 1). (The inclusion follows since δ (n) 2 = 1 for all n.) We have for any m < n δ (m) δ (n) 2 = (0,..., 0, 1, 0,..., 0, 1, 0...) 2 = 1 2 + ( 1) 2 = 2. Consequently there cannot be any subsequence that is convergent, since no subsequence can be Cauchy, and so the set is not compact. Corollary 3.33. If X is a finite-dimensional normed vector space then a subset A is compact if and only if it is closed and bounded. Proof. This follows since the given norm X is equivalent to the norm e from the proof of Theorem 3.31 and the following equivalences: (i) A is closed and bounded in (X, X ) if and only if it is closed and bounded in (X, e ) by Corollaries 3.27 and 3.29; (ii) Since (X, e ) is isometrically isomorphic to (K n, 2 ) by construction, A is compact with respect to e if and only if it is closed and bounded (by the Heine Borel Theorem); (iii) A is compact in (X, X ) if and only if it is compact in (X, e ) by Proposition 3.25. 23

4 Completeness and Convexity Recall the following: Definition 4.1. (a) A sequence (x n ) in a metric space X is a Cauchy sequence if for every ε > 0 we can find some N N such that d(x m, x n ) < ε for all m, n > N. (b) A metric space is complete if every Cauchy sequence is convergent. Recall that R and hence C are complete (for C consider the sequences of real and imaginary parts to reduce it to the case of R), but Q is not complete, for example there is a sequence (x n ) Q with x n 2 / Q, hence the sequence (x n ) is Cauchy, but not convergent within Q. Definition 4.2. A Banach space is a complete normed vector space. A Hilbert space is a complete inner product space. In particular since every inner product space is a normed vector space, it follows that every Hilbert space is a Banach space, but the converse is not true. Example 4.3. The space K n is a Banach space with respect to any norm on it. To prove this all one needs to do is prove it is complete with respect to one norm, then use Theorem 3.31 and Proposition 3.25. The main part of the proof is similar to the next example, which is more complicated. Example 4.4. The space l 2 is a Hilbert space. We already know from Example 2.9 that l 2 is an inner product space, so it only remains to show that it is complete. So let (x(k)) k=1 l2 be a Cauchy sequence, i.e. x(k) x(j) 2 0 as k, j. That is, we have a sequence of sequences: Now for any j, k, n N we have x(k) n x(j) n 2 x(1) = (x(1) 1, x(1) 2, x(1) 3,...) x(2) = (x(2) 1, x(2) 2, x(2) 3,...) x(3) = (x(3) 1, x(3) 2, x(3) 3,...) x(k) n x(j) n 2 = x(k) x(j) 2 2 0 as k, j 0. It follows that for each n the sequence (x(k) n ) k=1 of components is a Cauchy sequence in the complete space K, and hence convergent to some x n K. Set x = (x 1, x 2,...) where x n = lim x(k) n. We want to show x l 2 and that k x(k) x with respect to 2. Fix ε > 0. There is some K N such that x(k) x(j) 2 < ε for all k, j > K. Then for any N N and k, j > K N x(k) n x(j) n 2 x(k) x(j) 2 2 ε 2. Taking the limit j gives N x(k) n x n 2 ε 2 k > K, 24

since we only have finitely many sequences on the left hand side. But since the above is true for all N N we get x(k) n x n 2 ε 2, that is, x(k) x l 2, hence x = x(k) ( x(k) x ) l 2. Moreover x(k) x 2 ε for each k > K, so that x(k) x with respect to 2. Exercise 4.5. Consider the following two sets: l 1 := {(x n ) : x n < }, l := {(x n ) : sup x n < }. n 1 That is, l 1 is the set of all summable sequences, and l is the set of all bounded sequences. Show that both are vector spaces with respect to the same operations we defined on l 2, and that both are Banach spaces with respect to the following norms: x 1 := x n, for x l 1, x := x n, for x l. n 1 Furthermore show that neither is a Hilbert space. Show that the following is a closed subspace of l : c 0 := {(x n ) : lim n x n = 0}, the set of sequences that converge to 0. Thus show that it is a Banach space, but that it is also not a Hilbert space. Finally, show that c 00 is dense subspace of c 0, l 1 and l 2, but not a dense subspace of l. Hence show that c 00 with any of the norms 1, 2 or is not complete. Example 4.6. The space C[a, b] is a Banach space when given the norm. To see this let (f n ) be a Cauchy sequence in C[a, b], then for each t [a, b] we have f n (t) f m (t) max s [a,b] f n(s) f m (s) = f n f m 0 as m, n, so that f(t) = lim f n(t) exists, being the limit of the Cauchy sequence ( f n (t) ) n. This defines a function f : [a, b] K such that f n f pointwise. In fact imitating the technique of the previous example it is not hard to show that the convergence is in fact uniform, which implies in particular that the limit function f is continuous, so belongs to C[a, b]. The above shows that C[a, b] is complete with respect to the supremum norm, however it is also not hard to show that it is not an inner product space, so this is an example of a Banach space that is not Hilbert space. 25

Example 4.7. The space C[a, b] is not complete when given the norm 2, a norm arising from an inner product. To prove this formally is actually quite fiddly, but consider the following illustrative example: let (f n ) C[ 1, 1] defined by 0 if x [ 1, 0), f n (x) = nx if x [0, 1/n), 1 if x [1/n, 1]. Sketching these graphs it is clear that pointwise the sequence is convergent to the discontinuous function f(x) = 1 (0,1], the indicator function of the subset (0, 1], where { 0 if x [ 1, 0], f(x) = 1 (0,1] (x) = 1 if x (0, 1]. Clearly f / C[ 1, 1], but this in itself is not an immediate problem; what we need to show is that there is no g C[ 1, 1] such that f n g 2 0 as n. This can be done. Now, it appears that one possible solution is to enlarge C[ 1, 1] and instead look at R[ 1, 1], the vector space of Riemann-integrable functions [ 1, 1] K, to which f belongs, since the inner product makes sense on this space, and also f n f 2 0. However, this space is still not complete. The solution to this problem is to instead use the space L 2 [a, b] of Lebesguemeasurable functions f : [a, b] K such that b a f(t) 2 dt <, with inner product f, g = b a f(t)g(t) dt. This contains R[a, b], and hence C[a, b], plus additional functions. However, there is in fact one additional complication, which again can be illustrated using the sequence (f n ) above. We noted that f = 1 (0,1] is the pointwise limit of the f n, but consider the function h = 1 [0,1] where we have added the left endpoint. Although f n (0) h(0), it is the case that But f h 2 2 = 1 1 f(t) h(t) = f(t) h(t) 2 dt. { 1 if t = 0, 0 if t 0 which is a Riemann-integrable function that has integral 0, i.e. f h 0 yet f h 2 = 0. The solution to this is to define an equivalence relation on L 2 [a, b] by saying that f g f g 2 = 0 b a f(t) g(t) 2 dt = 0, which turns out to be equivalent to saying that f(t) and g(t) agree with one another except perhaps for some values t belonging to a subset of [a, b] of Lebesgue measure 26

zero. The space L 2 [a, b] is then, officially, the vector space of equivalence classes, where we define [f] + [g] = [f + g], λ[f] = [λf] if [f] denotes the equivalence class of f. In practice this distinction can be safely ignored in most circumstances. A more abstract answer to the incompleteness problem of (C[a, b], 2 ) is given by the following: Theorem 4.8. Let X be an incomplete normed vector space. Then there is a Banach space Y and a linear isometry T : X Y (i.e. T x = x for all x X) such that Ran T = {T x : x X} is a dense subspace of Y. The space Y is called the completion of X; it is not unique, but if Y is any other Banach space that satisfies the conditions of Theorem 4.8 for some linear map T : X Y then there is a linear, length preserving, invertible map S : Y Y such that ST = T. In particular Y and Y are isometrically isomorphic. Remark. Although Theorem 4.8 says that any incomplete normed vector space can be viewed as a dense subspace of a Banach space, and this can in particular be applied to the incomplete space (C[a, b], 2 ), it is usally advantageous to use the concrete realisation of this completion given by the space L 2 [a, b]. An alternative viewpoint on completeness comes from looking at series. If X is a normed vector space and x i X, i N, then the series x i is convergent if the sequence ( s N := N x i) of partial sums is convergent; it is absolutely N=1 convergent if the series of nonnegative numbers x i <. Proposition 4.9. Let X be a normed vector space. It is a Banach space if and only if every absolutely convergent series is convergent. Proof. Suppose X is a Banach space, and let x i be an absolutely convergent series. For N > M we have, using the triangle inequality, N M N N s N s M = x i x i = x i x i 0 i=m+1 i=m+1 as M, N, since x i <. Thus (s N ) N=1 is Cauchy, hence convergent, as required. Suppose, instead, that every absolutely convergent series is convergent, and let (x n ) be a Cauchy sequence. Then we can choose N 1 < N 2 < such that x n x m < 2 r for all n, m N r. Now set y r = x Nr+1 x Nr, so in particular y r 2 r, and hence r=1 y r 1 <, by the Comparison Test, and basic facts about geometric series. So there is some y X such that But then lim K y r = r=1 K x N K K (x Nr+1 x Nr ) = x NK+1 x N1 y. r=1 = y + x N1, i.e. the Cauchy sequence (x n ) has a convergent subsequence, and so must itself be convergent (also to y + x N1 ). 27

Completeness is a crucial ingredient in the next result. Definition 4.10. A subset C of a vector space V over K is convex if whenever x, y C and t [0, 1] then tx+(1 t)y C. Note that tx+(1 t)y = y +t(x y), so this just says that the line segment from y to x is contained in C for any pair x, y C. Example 4.11. Let Ω = {1,..., n}. A probability measure/distribution on Ω is a collection of n numbers p 1,..., p n with the properties p i 0 i, and n p i = 1. The interpretation is that p i is the probability of outcome i occurring. Note that we can think of this as a vector in R n by writing p = (p 1,..., p n ). If P is then the set of all probability measures on Ω then it is a convex subset (exercise), but not a subspace. In this case a general vector x R n corresponds to a general (signed) measure on Ω. Theorem 4.12. Let X be an inner product space, and C a complete, convex subset. For each x X there is a unique point P x C such that x P x = dist(x, C) := inf{ x z : z C}. Proof. Put d = dist(x, C), then by definition there is a sequence (z n ) C such that x z n d. We can apply the parallelogram law to get 0 z m z m 2 = (x z n ) (x z m ) 2 since 1 2 (z n + z m ) C by convexity. But now = 2 ( x z n 2 + x z m 2) 2x z n z m 2 = 2 ( x z n 2 + x z m 2) 2 2 x z n + z m 2 2 ( x z n 2 + x z m 2) 4d 2 2 ( x z n 2 + x z m 2) 4d 2 2(d 2 + d 2 ) 4d 2 = 0 z m z n 2 0 as m, n, so that the sequence (z n ) is a Cauchy sequence in the complete space C, hence is convergent to some limit z C. Moreover 2 x z = lim n x z n = d by continuity of norms. If y C such that x y = d then z y 2 = 2 ( x y 2 + x z 2) (x y) + (x z) 2 = 2(d 2 + d 2) 2 2 x y + z 2 4d 2 4d 2 = 0, 2 and so z = y, i.e. the nearest point is unique. 28