Basic Properties of Metric and Normed Spaces

Basic Properties of Metric and Normed Spaces Computational and Metric Geometry Instructor: Yury Makarychev The second part of this course is about metric geometry. We will study metric spaces, low distortion metric embeddings, dimension reduction transforms, and other topics. We will discuss numerous applications of metric techniques in computer science. 1 Definitions and Examples 1.1 Metric and Normed Spaces Definition 1.1. A metric space is a pair (X, d), where X is a set and d is a function from X X to R such that the following conditions hold for every x, y, z X. 1. Non-negativity: d(x, y) 0.. Symmetry: d(x, y) = d(y, x). 3. Triangle inequality: d(x, y) + d(y, z) d(x, y). 4. d(x, y) = 0 if and only if x = y. Elements of X are called points of the metric space, and d is called a metric or distance function on X. Exercise 1. Prove that condition 1 follows from conditions -4. Occasionally, spaces that we consider will not satisfy condition 4. We will call such spaces semi-metric spaces. Definition 1.. A space (X, d) is a semi-metric space if it satisfies conditions 1-3 and 4 : 4. if x = y then d(x, y) = 0. Examples. Here are several examples of metric spaces. 1. Euclidean Space. Space R d equipped with the Euclidean distance d(x, y) = x y. 1

. Uniform Metric. Let X be an arbitrary non-empty set. Define a distance function d(x, y) on X by d(x, y) = 1 if x y and d(x, x) = 0. The space (X, d) is called a uniform or discrete metric metric space. 3. Shortest Path Metric on Graphs. Let G = (V, E, l) be a graph with positive edge lengths l(e). Let d(u, v) be the length of the shortest path between u and v. Then (V, d) is the shortest path metric on G. 4. Tree Metrics. A very important family of graph metrics is the family of tree metrics. A tree metric is the shortest path metric on a tree T. 5. Cut Semi-metric. Let V be a set of vertices and S V be a proper subset of V. Cut semi-metric δ S is defined by δ S (x, y) = 1 if x S and y / S, or x / S and y S; and δ S (x, y) = 0, otherwise. In general, the space (X, d) is not a metric since d(x, y) = 0 for some x y. Nevertheless, δ S (x, y) is often called a cut metric. Definition 1.3. A normed space is a pair (V, ), where V is a linear space (vector space) and : V R is a norm on V such that the following conditions hold for every x, y V. 1. x > 0 if x 0.. x = 0 if and only if x = 0. 3. αx = α x for every α R. 4. x + y x + y (convexity). Every normed space (V, ) is a metric space with metric d(x, y) = x y on V. 1. Lebesgue Spaces L p (X, µ) In this section, we define Lebesgue spaces, a very important class of normed spaces. Let (X, µ) be a measure space. We consider the set of measurable real valued functions on X. For p 1, we define the the p norm of a function f by ( 1/p f p = f(x) dµ(x)) p. X If the integral above is infinite (diverges), we write f p =. Similarly, we define f = sup f(x). Now we define the Lebesgue space L p (X, µ) (for 1 p ): L p (X, µ) = {f : f is measurable w.r.t. measure µ; f p < }. Caveat: The norm f p can be equal to 0 for a function f L p (X, µ), which is not identically equal to 0. So formally L p (X, µ) (as defined above) is not a normed space. The

standard way to resolve this problem is to identify functions that differ only on a set of measure 0. The norm is usually defined as f = ess sup x X f(x) = inf{sup f(x) : f(x) = f(x) almost everywhere}. x X Examples. Consider several examples of L p spaces. 1. Space l p. Let X = N, and µ be the counting measure; i.e. µ(s) = S for S N. The elements of l p are infinite sequences of real numbers a = (a 1, a,... ) (which we identify with maps from N to R) s.t. a p <. The p norm of a sequence a = (a 1, a,... ) equals ( ) 1/p a p = a p.. Space l d p. Let X = {1,..., d}, and µ be again the counting measure; i.e. µ(s) = S for S N. The elements of l d p are d-tuples of real numbers a = (a 1, a,..., a d ) R d. The p norm of a vector a = (a 1, a,..., a d ) equals ( d ) 1/p a p = a p. 3. Space L p [a, b]. Let X = [a, b], and µ be the standard measure on R. The elements of L p [a, b] are measurable functions f : [a, b] R with f p <. The p norm of a function f equals ( b 1/p f p = f(x) dx) p. Lemma 1.4. For every 1 p < q, we have l p l q and L q [0, 1] L p [0, 1]. Both inclusions are proper. a Proof. We consider the case when q <. Let a l p. Let I = {i : a i 1}. Note that I is a finite set, as otherwise we would have that a p p i I a i p =. For every i / I, we have a i q < a i p. Therefore, a q q = i I a i q + i/ I a i q i I a i q + i/ I a i p i I a i q + a p p <. We conclude that a l q. Now let f L q [0, 1]. Let I = {x : f(x) 1}. Note that f p < f q when x / I, and I f(x) q dx 1 dx 1. Therefore, I 1 f p p = f(x) p dx = f(x) p dx+ f(x) p dx 1+ f(x) q dx 1+ f q q <. 0 I [0,1]\I [0,1]\I We get that f L p [0, 1]. 3

Exercise. Prove the statement of Lemma 1.4 for q =. Exercise 3. Let (X, µ) be a measure space with µ(x) <, and 1 p < q. Prove that L q (X, µ) L p (X, µ). Show that on the other hand L q (R) L p (R). 1.3 Metric Embeddings Consider two metric spaces (X, d X ), (Y, d Y ) and a map f : X Y. We say that f : X Y is a Lipschitz map if there is a number C such that d Y (f(x 1 ), f(x )) Cd X (x 1, x ) for all x 1, x, X. The Lipschitz constant f Lip of f is the minimum C such that this inequality holds. We say that a bijective map ϕ : X Y is an isometry if for every x 1, x X, d Y (ϕ(x 1 ), ϕ(x )) = d X (x 1, x ). We say that an injective map ϕ : X Y is an isometric embedding if ϕ is an isometry between X and ϕ(x) (the image of X under ϕ). The distortion of a map f : X Y equals f Lip f 1 Lip where f 1 is the inverse map from f(x) to X. Exercise 4. Prove that f has distortion at most D if and only if there is a number c > 0 such that c d X (x 1, x ) d Y (x 1, x ) c D d X (f(x 1 ), f(x )) for every x 1, x X. Exercise 5. Prove the following statements. 1. An isometric embedding has distortion 1.. Let f be a Lipschitz map from X to Y and g be a Lipschitz map from Y to Z then h = g f is a Lipschitz map from X to Z and h Lip f Lip g Lip. 3. Let f be an embedding of X into Y and g be an embedding of Y into Z then the distortion of h = g f is at most the product of distortions of f and g. Embeddings into L p spaces Theorem.1. Every finite metric subspace (X, d) embeds isometrically into l n for n = X. Proof. Denote the elements of X by x 1, x,..., x n. Now define the embedding ϕ : X l n as follows ϕ : x (d(x 1, x), d(x, x),..., d(x n, x)). We claim that ϕ is an isometric embedding. That is, ϕ(x i ) ϕ(x j ) = d(x i, x j ). 4

First, we prove that ϕ(x i ) ϕ(x j ) d(x i, x j ). We need to show that all coordinates of the vector ϕ(x i ) ϕ(x j ) are bounded by d(x i, x j ) in the absolute value. Indeed, the k-th coordinate of ϕ(x i ) ϕ(x j ) equals d(x k, x i ) d(x k, x j ). From the triangle inequalities for x i, x j and x k, it follows that d(x k, x i ) d(x k, x j ) d(x i, x j ). Now, we verify that ϕ(x i ) ϕ(x j ) d(x i, x j ). Note that ϕ(x i ) ϕ(x j ) d(x k, x i ) d(x k, x j ) (the absolute value of the k-th coordiante) for every k. In particular, this inequality holds for k = i. That is, ϕ(x i ) ϕ(x j ) d(x i, x i ) d(x i, x j ) = d(x i, x j ). Theorem.. Let p [1, ). Metric space l d (Euclidean d-dimensional space) embeds isometrically into L p (X, µ) for some space X. Proof. Let X = l d and µ = γ be the / Gaussian measure on X = l d (µ is the probability measure on X with density e x / (π) d/ ). Recall that the elements of L p (l d, γ) are ( 1/p. functions on l d. Let M = x l d 1 dγ(x)) p We construct an embedding ϕ that maps every v l d to a function f v defined as follows: f v (x) = v, x M. We prove that the embedding is an isometry. Consider two vectors u and v. Let w = u v, and e = w/ w. We have, p ϕ(u) ϕ(v) p p = u, x v, x l d M M dγ(x) = 1 u v, x p dγ(x) M p l d = 1 w e, x p dγ(x) = 1 M p M p w p e, x p dγ(x) l d Consider a coordinate frame in which the x 1 -axis is parallel to the vector e (i.e. vector e has coordinates (1, 0,..., 0)). Then e, x = x 1. We get ( ϕ(u) ϕ(v) p = w 1/p x 1 dµ(x)) p = w = u v. M We proved that the map ϕ is an isometry. l d We showed that every finite subset S of l d embeds isometrically into space L p (X, µ). Can we embed S into a simpler space l N p (for some N)? We will see that all spaces L p (X, µ) (of sufficiently large dimension) have essentially the same finite metric subspaces. Therefore, if a metric space embeds into some L p (X, µ), then it also embeds into l N p for some N. Theorem.3. Let S be a finite subset of L p (Z, µ), n = S, and N = ( n + 1. Then S isometrically embeds into l N p. 5 l d

Proof. Consider the linear space M of all symmetric n n matrices with zeros on the diagonal. The dimension of M is ( n. Consider a map f : R n M defined as follows. The map f sends a vector u R n to the matrix A = (a ij ) with entries a ij = u i u j p. Clearly, f(v) M for every v R n. Let B = f(r n ) {f(u) : u R n }. Let C be the convex closed cone generated by B; that is, C is the closure of the following set { m } { m } λ (i) b (i) : b (i) B, λ (i) 0, m N = λ i f(x (i) ) : x (i) R n, λ (i) 0, m N. For every metric space (S, d) on a set S = {s 1, s,..., s n }, we define a matrix F S by ij = d(s i, s j ) p. The theorem follows from the following two lemmas. F S Lemma.4. Suppose that S = {s 1,..., s n } L p (Z, µ) then F S C. Proof. Recall that each element s i is a function from Z to R. Let σ(z) = (s 1 (z),..., s n (z)) for every z Z. We have, Fij S = d(s i, s j ) p = s i (z) s j (z) p dµ(z) = f ij (σ(z))dµ(z). Z Therefore, F S = f(σ(z))dµ(z). Since f(σ(z)) B C for every z Z, we conclude that Z F S C. Lemma.5. Consider a finite metric space S = {s 1,..., s n }. Suppose that F S C. Then S embeds into l N p, where N = ( n + 1. Proof. By the Carathéodory theorem, every point in the cone C can be expressed as a sum of at most dim M + 1 = N points in B. In particular, we can write Z F S = b (k) = f(x (k) ), for some b (1),..., b (N) B and some x (k) f 1 (b (k) ) (x i is a preimage of b (k) ). By the definition of F S, we have d(s i, s j ) p = F S ij = f ij (x (k) ) = x (k) i x (k) j p. (1) We define the embedding ϕ of S to l N p : ϕ(s i ) = (x (1) i, x ( i,..., x (N) i ). Note that equation (1) says that d(s i, s j ) p = ϕ(s i ) ϕ(s j ) p p, and therefore d(s i, s j ) = ϕ(s i ) ϕ(s j ) p. We conclude that ϕ is an isometric embedding. 6

Corollary.6. Suppose that S is a subset of l d. Then S isometrically embeds into l N p, where N = ( ) S + 1. Exercise 6. In our proof, we used the Carathéodory theorem for arbitrary convex sets: every point in the convex hull of S R d is a convex combination of at most d + 1 points from S. Show that if the convex hull CH(S) of S is a cone, then every point CH(S) is a linear combination, with positive coefficients, of at most d points in S. Conclude that in the statement of Theorem.3 we can replace N = ( n + 1 with N = ( n. Definition.7. Let c p (X) be the least distortion 1 with which a finite metric space (X, d) embeds into l p. Theorem.8. For every finite metric space X and every p [1, ], we have 1 = c (X) c p (X) c (X). Proof. The inequality 1 = c (X) c p (X) follows from Theorem.1. Let f be an embedding of X into l (X) with distortion c (X). By Corollary.6, there is an isometric embedding g of f(x) l into l p. Then map g f is an embedding of X into l p with distortion at most c (X). We conclude that c p (X) c (X). 1 A simple compactness argument shows that there is an embedding with the least possible distortion. 7