Chapter 1 Preliminaries

Chapter 1 Preliminaries 1.1 Conventions and Notations Throughout the book we use the following notations for standard sets of numbers: N the set {1, 2,...} of natural numbers Z the set of integers Q the set of rational numbers R the set of real numbers C the set of complex numbers The symbol F stands for either R or C. We use the symbol for the set inclusion relation and do not use the symbol at all. 1.2 Convex Functions In this section, I is an open interval in the real line R that may be bounded or unbounded. A function f : I R is said to be convex if f[λx + 1 λ)y] λfx) + 1 λ)fy), 1.1) for all x, y I and λ [, 1]. It is called concave if the opposite inequality f[λx + 1 λ)y] λfx) + 1 λ)fy) 1.2) holds for all x, y I and λ [, 1]. Note that numbers λx + 1 λ)y in 1.1) and 1.2) belong to the domain I of the function f cf. Exercise 1.1). Geometrically cf. Fig. 1.1), inequality 1.1) means that each point R on the graph of the function f that lies between two other points P and Q on 1

2 1 Preliminaries the graph lies on or below the chord PQ cf. Exercise 1.2). In the case of inequality 1.2), R lies on or above the chord PQ. It is clear that a function f is concave if and only if the function f is convex. y y P convex function R Q x P R concave function Q x Fig. 1.1 Convex and concave functions. The following theorem is instrumental in proving many inequalities that appear later in this chapter. Theorem 1.1. A function f : I R is convex if and only if n ) f λ k x k n λ k fx k ), 1.3) for all x 1,..., x n I and λ k 1 1 k n) such that n λ k = 1. Note that the number n λ kx k belongs to I cf. Exercise 1.1), so the left side in 1.3) is well-defined. Proof. Necessity.) We prove by induction that 1.3) holds for a convex function f. The base case, n = 1, holds trivially. Suppose that 1.3) holds for some m 1 and let x 1,..., x m, x m+1 I, λ k 1 1 k m + 1), m+1 λ k = 1. We may assume that λ m+1 1. Otherwise, the induction step is trivial.) For λ = m λ k, we have λ m+1 = 1 λ, and then, by 1.1) and the induction hypothesis,

1.2 Convex Functions 3 f m+1 ) λ k x k = f λ m m λf λ = m m+1 λ k λ k λ x k ) λ x k + 1 λ)x m+1 ) + λ m+1 fx m+1 ) λ k λ fx k) + λ m+1 fx m+1 ) λ k fx k ). The result follows by induction. Sufficiency.) This follows immediately from 1.3) for n = 2. Similarly, a function f : I R is concave if and only if n ) f λ k x k n λ k fx k ), 1.4) for all x 1,..., x n I and λ k 1 1 k n) such that n λ k = 1. The next theorem is a useful characterization of smooth convex functions. The proof is left to the reader cf. Exercise 1.3). Theorem 1.2. If f is differentiable function on I, then it is convex if and only if f is increasing on I. Similarly, f is concave if and only if f is decreasing on I. Example 1.1. Cf. Fig. 1.2.) Let fx) = x p, p 1, be a power function on, ). Because f x) = px p 1 is increasing on, ), f is a convex function. On the other hand, the natural logarithm function fx) = ln x on, ) is concave because f x) = 1/x is decreasing on, ). y y y =x p y =lnx x Fig. 1.2 The power and logarithmic functions. x

4 1 Preliminaries 1.3 Inequalities In this section, unless otherwise specified, all variables assume values in the field of complex numbers C. Recall that z stands for the absolute value modulus) of a complex number z, z stands for the complex conjugate of z, Rez) and Imz) denote the real and imaginary parts of z, respectively, and z 2 = z z, 2 Rez) = z + z, 2i Imz) = z z, for any z C. We begin by proving the triangle inequality for complex numbers: Indeed, x + y x + y, for all x, y C. 1.5) x + y 2 = x + y)x + y) = x + y) x + ȳ) = x x + xȳ + y x + yȳ = x 2 + xȳ + xȳ + y 2 = x 2 + 2Rexȳ) + y 2 x 2 + 2 x y + y 2 = x + y ) 2, because Rez) z for any complex number z. From this, 1.5) follows immediately. Of course, the triangle inequality also holds for real numbers. For a real number p > 1 we define q by the equation 1 p + 1 q = 1. Clearly, there is a unique real number q > 1 satisfying this equation. Numbers p and q are called conjugate exponents. Theorem 1.3. Young s Inequality.) If p and q are conjugate exponents, then for all complex numbers x and y, xy x p p + y q q. 1.6) Proof. If xy =, the inequality is trivial. Otherwise, x >, y > and, by the concavity of the logarithm function, we have ) x p ln p + y q 1 q p ln x p ) + 1 q ln y q ) = ln x + ln y = ln xy. Because the logarithm is a strictly increasing function, we obtained the desired result. Theorem 1.4. Hölder s Inequality.) If p and q are conjugate exponents, then

1.3 Inequalities 5 n n ) 1/p n ) 1/q x k y k x k p y k q 1.7) for arbitrary complex numbers x 1,..., x n and y 1,..., y n. Proof. We may assume that at least one of the x k s and one of the y k s are positive. Otherwise, both sides in 1.7) are zero and we are done.) Then the real numbers n ) 1/p n ) 1/q u = x k p and v = y k q are positive. We apply Young s inequality 1.6) to x k /u and y k /v to obtain x k u y k v 1 p xk u ) p + 1 q yk ) q, 1 k n, u or, after summing up from k = 1 to k = n, n x ky k 1 n uv p x k p + 1 n u p q yq k = 1 v q p + 1 q = 1, which is equivalent to 1.7). Theorem 1.5. Cauchy-Schwarz Inequality.) For arbitrary complex numbers x 1,..., x n and y 1,..., y n, n x k y k n x k 2 n y k 2. 1.8) Inequality 1.8) follows from Hölder s Inequality by setting p = q = 2. We present an elementary proof which is not build upon the theory of convex functions. Proof. We have n n n tx k + y k 2 = t 2 x k 2 + 2t x k y k + n y k 2, for all real numbers t. Hence the discriminant of the quadratic trinomial on the right hand side must be nonpositive: n 2 n ) n ) 4 x n y n ) 4 x k 2 y k 2, which is equivalent to 1.8).

6 1 Preliminaries The next theorem establishes an inequality which plays a distinguished role in analysis. Theorem 1.6. Minkowski s Inequality.) For arbitrary complex numbers x 1,..., x n, y 1,..., y n and a real number p 1, [ n ] 1/p [ n ] 1/p [ n ] 1/p x k + y k p x k p + y k p. 1.9) Proof. We may assume that both real numbers n ) 1/p n ) 1/p u = x k p and v = y k p are positive. By the triangle inequality, we have Because x k + y k p x k + y k ) p = = [ u x k u + v) u + v u + u = u + v) p x k u + v u + u u + v + v u + v = 1 u x k u + v y k v v u + v v u + v y k v y k v ) p )] p ) p. and the power function x p is convex for p 1, we have u x k u + v u + v ) p y k u x k p + v y k p u + v v u + v u p u + v v. p Hence, u x k + y k p u + v) p x k p + v u + v u p u + v y k p ). v p By summing up both sides of the above inequality, we obtain n n u x k + y k p u + v) p x k p u + v u p + v n y k p ) u + v v p u = u + v) p u + v + v ) u + v [ n ) 1/p n ) 1/p ] p = u + v) p = x k p + y k p, which is equivalent to 1.9).

1.3 Inequalities 7 Hölder s, the Cauchy-Schwartz, and Minkowski s inequalities also hold for infinite sequences of complex numbers. Theorem 1.7. Hölder s Inequality.) Let p and q be conjugate exponents. If the series x k p and y k q are convergent, then the series x ky k converges and ) 1/p ) 1/q x k y k x k p x k q. 1.1) Theorem 1.8. Cauchy-Schwarz Inequality.) If the series x k 2 and y k 2 are convergent, then the series x ky k converges and x k y k x k 2 y k 2. 1.11) Theorem 1.9. Minkowski s Inequality.) Suppose that the series x k p and y k p converge for a real number p 1. Then the series converges and x k + y k p [ ] 1/p [ ] 1/p [ ] 1/p x k + y k p x k p + y k p. 1.12) Proofs of these theorems are straightforward and left as an exercise cf. Exercise 1.6). If < p < 1, then we have to invert the inequality sign in Minkowski s inequality cf. Exercise 1.8). However, in this case, we have the following analog of the inequality in 1.12). Theorem 1.1. Suppose that the series x k p and y k p converge for a real number < p < 1. Then the series x k + y k p converges and x k + y k p x k p + y k p. 1.13)

8 1 Preliminaries The claim of the theorem follows immediately from the following lemma. Lemma 1.1. Let p be a real number such that < p < 1. Then for any complex numbers x and y, x + y p x p + y p. Proof. Let f be a real function of a real variable t defined by ft) = t p t + 1) p + 1, for t. Because f) = and ) 1 f t) = pt p 1 pt + 1) p 1 = p t 1 >, for t >, 1 p t + 1) 1 p we have ft) for t, so t + 1) p t p + 1, for t. Clearly, we may assume that y in the lemma. Then ) x + y p x + y ) p = y p x p p ) y + 1 y p x y + 1 = x p + y p, as desired. We conclude this section by establishing another useful inequality. Theorem 1.11. For any complex numbers x and y, Proof. The function of a real variable t is increasing over [, ) because x + y 1 + x + y x 1 + x + y 1 + y. 1.14) f t) = ft) = t 1 + t t, for t. 1 + t) 2 Then, because x + y x + y cf. 1.5)), we have x + y x + y 1 + x + y 1 + x + y = x 1 + x + y + y 1 + x + y x 1 + x + y 1 + y, and the result follows.

1.4 Integral Inequalities 9 1.4 Integral Inequalities In this section, the functions x and y are Lebesgue measurable functions on the unit interval. Theorem 1.12. Hölder s Inequality.) If p and q are conjugate exponents and 1 x p <, 1 y q <, then 1 xy < and 1 1 ) 1/p 1 ) 1/q xy x p y q. 1.15) Proof. Suppose that one of the factors on the right hand side of 1.15), say, the first one is zero. Then x = almost everywhere on [, 1], hence the left hand side of the inequality 1.15) is zero. Therefore we may assume that the real numbers 1 ) 1/p 1 ) 1/q u = x p and v = y q are positive. By Young s inequality, xt) yt) 1 ) p xt) + 1 ) q yt), t [, 1]. u v p u q u Inasmuch as 1 x p < and 1 y q <, it follows that 1 uv 1 xy 1 p 1 x p u p + 1 q 1 y q v q = 1 p + 1 q = 1, which is equivalent to 1.15). By setting p = q = 2 in Hölder s integral inequality 1.15), we obtain the integral version of the Cauchy-Schwarz inequality 1.11). Theorem 1.13. Cauchy-Schwarz Inequality.) Suppose that 1 x 2 < and 1 y 2 <. Then 1 xy < and 1 1 1 xy x 2 y 2. 1.16) Theorem 1.14. Minkowski s Inequality.) If p 1 and 1 x p <, 1 y p <, then 1 ) 1/p x + y p ) 1/p 1 ) 1/p x p + y p. 1.17)

1 1 Preliminaries Proof. We may assume that quantities u = x p and v = y p are positive numbers. If, say, x p =, then x is zero almost everywhere and we have equality in 1.17).) By the triangle inequality, we have xt) + yt) p xt) + yt) ) p = for t [, 1]. Because = u xt) + v yt) u v [ u xt) u + v) + v u + v u u + v u = u + v) p xt) + v u + v u u + v u u + v + v u + v = 1 yt) v yt) v ) p )] p and the power function α p is convex for p 1, we have u xt) + v ) p yt) u xt) p + v yt) p, u + v u u + v v u + v u p u + v v p for t [, 1]. Hence, u xt) + yt) p u + v) p xt) p u + v u p + v u + v for t [, 1]. It follows that 1 ) p, yt) p 1 u x + y p u + v) p x p u + v u p + v 1 ) y p u + v v p u = u + v) p u + v + v ) u + v [ 1 ) 1/p 1 ) ] 1/p p = u + v) p = x p + y p, v p ), which is equivalent to 1.17). The result of the following theorem follows immediately from Lemma 1.1. Theorem 1.15. Suppose that 1 x p < and 1 y p < for a real number < p < 1. Then 1 x + y p < and

1.5 Equivalence Relations 11 1 1 1 x + y p x p + y p. 1.18) 1.5 Equivalence Relations By definition, a binary relation on a set X is a subset of the Cartesian product X X. For a binary relation R on X we often write xry instead of x, y) R. Among many kinds of binary relations, the equivalence relations and partial orders arguably are most common. We introduce basics of equivalence relations below and cover partial orders in the next section. It is customary to use one symbol,, for various equivalence relations. We follow this tradition here. An equivalence relation on a set X is a reflexive, symmetric, and transitive binary relation on X, that is, a relation on X satisfying the following conditions for all x, y, z X: x x reflexivity, x y implies y x symmetry, x y and y z implies x z transitivity. If is an equivalence relation on X and x X, then the set [x] = {y X : y x} is called the equivalence class of x with respect to. The set of equivalence classes in X with respect to is called the quotient set of X with respect to and denoted by X/. The mapping x [x] which assigns to each x X its equivalence class [x] in X, is called the canonical map of X onto X/. Here are two extreme examples of equivalence relations on a setx. Example 1.2. 1) The complete relation X X. Here, x y for all x, y X, and it is easy to verify that is an equivalence relation. It is clear that there is only one equivalence class in X, namely, the set X itself. Hence the quotient set is a singleton. 2) The equality relation = on X is evidently an equivalence relation. The equivalence class of x X is the singleton {x}. A nontrivial instance of an equivalence relation appears in a formal definition of rational numbers. Example 1.3. A fraction m/n can be viewed as an ordered pair of integers m, n) with n. Two fractions, m, n) and p, q), are said to be equivalent if mq = np:

12 1 Preliminaries m, n) p, q) if and only if mq = np. It is not difficult to verify that is indeed an equivalence relation on the set Z Z \ {}). Equivalence classes of are called rational numbers. Theorem 1.16. Let X be a set and be an equivalence relation on X. Equivalence classes with respect to partition the set X, that is, every element of X belongs to one and only one equivalence class. Proof. By the reflexivity property, x [x]. It remains to show that any two equivalence classes either identical or disjoint. Suppose that [x] [y] and let z [x] [y]. Then z x and z y. By symmetry, x z. For every a [x], a x. By transitivity, a x and x z implies a z. Again by transitivity, a z and z y implies a y. Hence, a [y]. It follows that [x] [y]. By reversing the roles of x and y, we obtain [y] [x]. Therefore, [x] = [y] if [x] [y], and the result follows. An important example of an equivalence relation is found in linear algebra cf. Axler, 215, Section 3.E). Let V be a vector space over the field of scalars F, and U be a subspace of V. We define a binary relation on V by u v if and only if u v U. If u v, then vectors u and v are said to be congruent modulo U. It is readily verified that is an equivalence relation on V. By Theorem 1.16, equivalence classes of form a partition of the vector space V. These classes are called affine subspaces of V. It can be verified that [v] = U + v, that is, an affine subspace [v] is the translation of the subspace U by v. The quotient set V/ is usually denoted by V/U and called the quotient space of V modulo U. The set V/U is a vector space over the field F under operations of addition and multiplication by a scalar defined by [u] + [v] = [u] + [v] and k[u] = [ku], for u, v V and k K. The reader is invited to verify that these operations are well-defined.) 1.6 Zorn s Lemma It is customary to use one symbol for various order relations a symbol that resembles the familiar inequality sign. In what follows, we use the symbol for these relations. A partial order on a set X is a reflexive, antisymmetric, and transitive binary relation on X, that is, it satisfies the following conditions for all x, y, z X:

1.6 Zorn s Lemma 13 x x if x y and y x, then x = y if x y and y z, then x z reflexivity, antisymmetry, transitivity. If for every x and y in X either x y or y x, then the partial order is called a linear order also known as a simple or total order). A prototypical example of a linear order is the usual order on the set N of natural numbers and on any nonempty set of real numbers). Example 1.4. Let X be a set. The set inclusion relation is a partial and not always linear; cf. Exercise 1.11) order on the power set PX). It is a linear order if and only if X = or X is a singleton. Example 1.5. Let us define a binary relation on the set C of complex numbers by u v if and only if Reu) Rev) and Imu) Imv). It is not difficult to verify cf. Exercise 1.12) that this relation is indeed a partial but not linear) order on C. For instance, complex numbers 1 and i are incomparable with respect to the relation that is, 1 i and i 1). Example 1.6. Cf. Example 1.5). Let X = R 2 the 2-dimensional real vector space) and define on X by x 1, y 1 ) x 2, y 2 ) if and only if x 1 < x 2, or else x 1 = x 2 and y 1 y 2. It can be seen that is a linear order on X. Because of its resemblance to the way words are arranged in dictionary, this is called the lexicographical order on R 2. Example 1.7. Let X and Y be sets and F be the set of all functions whose domain is a subset of X and range is a subset of Y. For f F we denote by Df) the domain of the function f. We define a relation on F by f g if and only if Df) Dg) and fx) = gx) for all x Df). In words: f g means that f is a restriction of g, or, equivalently, that g is an extension of f. Because functions in F are subsets of the Cartesian product X Y, we observe that on F is an instance of the set inclusion relation. A partially ordered set a.k.a. poset) is a nonempty set together with a partial order on it. More formally, a partial ordered set is an ordered pair X, ), where X is a set and is a partial order on X. The pair X, ) is an example of an algebraic structure the concept which is common in mathematics. It is customary to call the set X itself a partially ordered set. If the relation is a linear order on X, then X is often called a chain.

14 1 Preliminaries Every nonempty subset Y of a partially ordered set X, ) is a partially ordered set itself with respect to the restriction of the relation to Y. Let X be a partially ordered set. If there is an element a X such that a x for every x in X, then a is called a least element of X. By the antisymmetry property of order, if X has a least element, then it is unique. Similarly, an element a X is called a greatest element of X provided that x a for every x in X; it also is unique if it exists at all. Example 1.8. 1) The set of natural numbers N = {1, 2,...} ordered by the usual relation has the least element but does not have the greatest element. 2) The power set PX) ordered by the inclusion relation has the least element and the greatest element X. These elements are equal only if X =. 3) Let {x 1, x 2,..., x n } be a set of distinct elements of some set X. The sets {x 1 }, {x 1, x 2 },..., {x 1, x 2,..., x n } form a chain in the power set PX). {x 1 } {x 1, x 2 } {x 1, x 2,..., x n } If Y is a subset of a partially ordered set X and a an element of X such that y a for all y Y, then a is said to be an upper bound of Y. A lower bound of a subset of X is defined similarly. An element a of a partially ordered set X is said to be a minimal element of X if x a implies x = a for every x X. Similarly, an element a is called a maximal element of X if a x implies x = a for every x X. Example 1.9. Let X be a set containing more that one element. If the set C = PX) \ of all nonempty subsets of X is ordered by the inclusion relation, then each singleton is a minimal element of C. However, C has no least element. The empty set is a lower bound of C. The claim of the following theorem is a fundamental result of the set theory. The proof uses the Axiom of Choice and is omitted cf. Halmos, 196, Chapter 16). Theorem 1.17. Zorn s lemma.) If X is a partially ordered set such that every chain in X has an upper bound, then X contains a maximal element. To exemplify applications of Zorn s lemma, we show that every nontrivial vector space has a basis. First, we recall some basic concepts of linear algebra. Let V {} be a vector space over a field F. Here, stands for the zero element of V. For a finite sequence of vectors v 1,..., v n in V and a finite sequence of scalars λ 1,, λ n in F, the expression of the form λ 1 v 1 + + λ n v n

1.6 Zorn s Lemma 15 is said to be a linear combination of vectors v 1,..., v n. Let M be a nonempty subset of V. The span of M, denoted by spanm), is the set of all linear combinations of vectors in M. A nonempty set M L is said to be linearly independent if, for every sequence v 1,..., v n of vectors in M, the relation λ 1 x 1 + + λ n x n = implies λ k =, for all k = 1,..., n. A subset B of vectors in V is called a Hamel basis for V if B is linearly independent and spans V. Theorem 1.18. Every vector space V {} has a Hamel basis. Proof. Let M be the set of all linearly independent subsets of V. This set is not empty because any set consisting of a single nonzero vector is linearly independent. The set inclusion relation defines a partial order on M. Every chain C in M has an upper bound, namely, the union C. By Zorn s lemma, M has a maximal element B. Suppose that spanb) V and let v be a vector in V such that v / spanb). Let us show that B {v} is a linearly independent set. In the relation λ v + λ 1 v 1 + + λ n v n =, where v k B, k = 1,, n, we must have λ = because v / spanb). Inasmuch as v k B, k = 1,..., n, and B is a linearly independent set, we have λ k =, k = 1,..., n. It follows that B is a proper subset of the linearly independent set B {v}, which contradicts the definition of B. Notes In Section 1.2, we present only properties of convex functions that are used later in this chapter. More properties are found in Exercise 1.3. The reader can find a comprehensive coverage of this important subject in many textbooks on real analysis see, for instance, Wade 2) and Royden and Fitzpatrick 21)). If f is a convex function on R, ϕ an integrable function over [, 1], and f ϕ also integrable over [, 1], then the following inequality holds: 1 ) f ϕx) dx 1 f ϕ)x)dx. This inequality is known as Jensen s Inequality. The inequality in 1.3) is known as a finite form of Jensen s Inequality.

16 1 Preliminaries Exercises Section 1.2 In Exercises 1.1 1.4, I stands for an interval in R. 1.1. For x 1,..., x n I, the number x = λ 1 x 1 + + λ n x n is said to be a convex combination of the numbers x 1,..., x n, provided that λ k 1 for 1 k n and λ 1 + + λ n = 1. Show that Accordingly, x I. min{x 1,..., x n } x max{x 1,..., x n }. 1.2. Prove that a function f : I R is convex if and only if for any three consecutive points P, R, Q on the graph of the function, the point R lies below or on the chord PQ cf. Fig. 1.1). 1.3. a) Let f be a real-valued function on I. Prove that f is convex if and only if for any numbers a, b, and x in I such that a < x < b, the following inequalities hold: cf. Fig. 1.3). b) Show that fx) fa) x a fb) fa) b a fb) fx) b x 1) a convex function on I has left-hand and right-hand derivatives, f x ) and f x + ), at each point x I, 2) these derivatives are increasing functions on I, and 3) f x ) f x + ) on I. c) Prove Theorem 1.2. 1.4. A function on I in the form fx) = px+ q, p, q R, is said to be affine. Show that a) f is affine on R if and only if fλx + 1 λ)y) = λfx) + 1 λ)fy) for all real numbers x y, and λ. b) f is affine on I if and only if f and f are convex on I. c) If f is convex on I = [a, b] and there is λ, 1) for which fλa + 1 λ)b) = λfa) + 1 λ)fb),

1.6 Zorn s Lemma 17 y Q P R a x b x Fig. 1.3 slopep R) slopep Q) sloperq). then f is affine on [a, b]. 1.5. Show that for all u, v, w C, a) u w u v + v w. b) u v u v. c) u + v 2 + u v 2 = 2 u 2 + v 2 ). Section 1.3 1.6. Prove inequalities 1.1) and 1.12). 1.7. Hölder s Inequality for < p < 1. Let p and q be real numbers such that 1 < p < 1 and p + 1 q = 1 so q < ). Assuming that y k for 1 k n, prove that n n ) 1/p n ) 1/q x k y k x k p y k q. Hint: Note that 1 p > 1, then use 1.7) replacing p with 1 p.) 1.8. Minkowski s Inequality for < p < 1. Assume that < p < 1 and show that [ n ] 1/p [ n ] 1/p [ n ] 1/p x k + y k p x k p + y k p. 1.9. Let x 1,..., x n be positive real numbers. The classical Pythagorean means are the arithmetic mean A), the geometric mean G), and the harmonic mean H). They are defined by

18 1 Preliminaries respectively. Prove that A = Ax 1,..., x n ) = 1 n x 1 + + x n ), G = Gx 1,..., x n ) = n x 1 x 2 x n, n H = Hx 1,..., x n ) = 1 x 1 + + 1, x n min{x 1,..., x n } H G A max{x 1,..., x n }. 1.1. Let a, b, x, and y be positive real numbers. Show that x ln x a + y ln y b x + y)ln x + y a + b. Section 1.6 1.11. Let X be a set with more than one element. Give examples of subsets A and B of X such that A B and B A. 1.12. Show that the relation in Example 1.5 is a partial order on C. 1.13. Consider the set X of all closed disks in the plane that are subsets of the square S = [, 1] [, 1]. a) Describe the maximal elements of the poset X, ). b) Show that X, ) does not have the greatest element. c) Describe the minimal elements of X, ). d) Show that X, ) does not have the least element.