1.4 Cauchy Sequence in R

Transcription

1 Definition. (1.4.1) 1.4 Cauchy Sequence in R A sequence x n R is said to converge to a limit x if ɛ > 0, N s.t. n > N x n x < ɛ. A sequence x n R is called Cauchy sequence if ɛ, N s.t. n > N & m > N x n x m < ɛ. Proposition. (1.4.2) Every convergent sequence is a Cauchy sequence. Proof. Assume x k x. Let ɛ > 0 be given. N s.t. n > N x n x < ɛ 2. n, m N x n x m x x n + x x m < ɛ 2 + ɛ 2 = ɛ.

2 Theorem. (1.4.3; Bolzano-Weierstrass Property) Every bounded sequence in R has a subsequence that converges to some point in R. Proof. Suppose x n is a bounded sequence in R. M such that M x n M, n = 1, 2,. Select x n0 = x 1. Bisect I 0 := [ M, M] into [ M, 0] and [0, M]. At least one of these (either [ M, 0] or [0, M]) must contain x n for infinitely many indices n. Call it I 1 and select n 1 > n 0 with x n1 I 0. Continue in this way to get a subsequence x nk such that I 0 I 1 I 2 I 3 I k = [a k, b k ] with I k = 2 k M. Choose n 0 < n 1 < n 2 < with x nk I k. Since a k a k+1 M (monotone and bounded), a k x. Since x nk I k and I k = 2 k M, we have x nk x < x nk a k + a k x 2 k 1 M+ a k x 0 as k.

3 Corollary. (1.4.5; Compactness) Every sequence in the closed interval [a, b] has a subsequence in R that converges to some point in R. Proof. Assume a x n b for n = 1, 2,. By Theorem 1.4.3, a subsequence x nk and a x b such that x nk x. Lemma. (1.4.6; Boundedness of Cauchy sequence) If x n is a Cauchy sequence, x n is bounded. Proof. N s.t. n N x n x < 1. Then sup n x n 1 + max{ x 1,, x N } (Why?) Theorem. (1.4.3; Completeness) Every Cauchy sequence in R converges to an element in [a, b]. Proof. Cauchy seq. bounded seq. convergent subseq.

4 1.5. Cluster Points of the sequence x n Definition. (1.5.1; cluster points) A point x is called a cluster point of the sequence x n if ɛ > 0, infinitely many values of n with x n x < ɛ In other words, a point x is a cluster point of the sequence x n iff ɛ > 0 & N, n > N s.t. x n x < ɛ Example Both 1 and 1 are cluster points of the sequence 1, 1, 1, 1,. The sequence x n = 1 n has the only cluster point 0. The sequence x n = n does not have any cluster point.

5 Proposition. 1. x is a cluster point of the sequence x n iff a subsequence x nk s.t. x nk x. 2. x n x iff every subsequence of x n converges to x 3. x n x iff the sequence {x n } is bounded and x is its only cluster points. Proof. 1. ( ) Assume x is a cluster point. Then, we can choose n 1 < n 2 < n 3 s.t. x nk x < 1 k. (Why?) This gives a subsequence x nk x. 2. Trivial 3. ( )If not, ɛ and a subseq x nk so that x nk x > ɛ. Since x nk is bounded, a convergent subseq. The limit of that subseq would be a cluster pt of the seq x n different from x, but there are no such pt. Contradiction.

6 Definition. (1.5.3; limit superior & limit inferior of seq x n ) Define the limit superior limx n in the following way: If x n is bounded above, then lim sup n x n = limx n = the largest cluster point limx n = if the set cluster point is empty If x n is NOT bounded above, then limx n = Similarly, we can define the limit inferior limx n. Examples For the seq 1, 0, 1, 1, 0, 1,, limx n = 1 and limx n = 1. If x n = n, then limx n = = limx n Let x n = ( 1) n 1+n n. Then limx n = 1 and limx n = 1.

7 Definition. (1.6.2; Vector space) A real vector space V is a set of elements called vectors, with given operations of vector addition + : V V V and scalar multiplication : R V V such that the followings hold for all v, u, w V and all λ, µ R: 1. v + w = w + v, (v + u) + w = v + (u + w), λ(v + w) = λv + λw, λ(µv) = (λµ)v, (λ + µ)v = λv + µv, 1v = v V s.t. v + 0 = v. v V s.t. v v = 0. A subset of V is called a subspace if it is itself a vector space with the same operations. W is a vector subspace of V iff λv + µu W whenever u, v W and λ, µ R. The straight line W = {(x 1, x 2 ) : x 1 = 2x 2 } is a subspace of R 2.

8 Euclidean space R n & Definitions & Properties The Euclidean n-space R n with the operations (x 1,, x n ) + (y 1,, y n ) = (x 1 + y 1,, x n + y n ) & λ(x 1,, x n ) = (λx 1,, λx n ) is a vector space of dimension n. The standard basis of R n ; e 1 = (1, 0,, 0),, e n = (0,, 0, 1). Unique representation: x = (x 1,, x n ) R n can be expressed uniquely as x = x 1 e x n e n. Inner product of x and y: x, y = n i=1 x iy i Norm of x: x = x, y. Distance between x and y: dist(x, y) = x y Triangle inequality: x + y x + y. Cauchy-Schwartz inequality: x, y x y Pythagorean theorem: If x, y = 0, then x + y 2 = x 2 + y 2.

9 Definition. (1.7.1; Metric Space (M, d) equipped with d =distance) A metric space (M, d) is a set M and a function d : M M R such that 1. d(x, y) 0 for all x, y M. 2. d(x, y) = 0 iff x = y. 3. d(x, y) = d(y, x) for all x, y M. 4. d(x, y) d(x, z) + d(z, y) for all x, y M. Example [Fingerprint Recognition] Let M be a data set of fingerprints in Seoul city police department. Motivation: Design an efficient access system to find a target. We need to define a dissimilarity function stating the distance between the data. The distance d(x, y) between two data x and y must satisfy the above four rules. Similarity queries. For a given target x M and ɛ > 0, arrest all having finger print y M such that d(y, x ) < ɛ.

10 Definition. ( Normed Space (V, )) A normed space (V, ) is a vector space V and a function : V R called a norm such that 1. v 0, v V 2. v = 0 iff v = λv = λ v, v V and every scaler λ. 4. v + w v + w, v, w V Examples V = R and x = x for all x R. V = R 2 and v = v1 2 + v 2 2 for all v = (v 1, v 2 ) R 2. Let V = C([0, 1])=all continuous functions on the interval [a, b]. Define f = sup{ f (x) : x [0, 1]} (called supremum norm).

11 Proposition. If (V, ) is a normed vector space and, then d is a metric in V. Proof. EASY. Examples d(v, w) = v w For V = C([0, 1]), the metric is d(f, g) = f g = sup{ f (x) g(x) : x [0, 1]}. The sup distance between functions is the largest vertical distance between their graphs.

12 Definition. A vector space V with a function, : V V R is called an inner product space if 1. v, v 0 for all v V. 2. v, v = 0 iff v = λv, w = λ v, w, v V and every scaler λ. 4. v + w, h = v, h + w, h. 5. v, w = w, v Examples 1. V = R 2 and v, w = v 1 w 1 + v 2 w 2. Two vectors v and w are orthogonal if v, w = V = C[0, 1] and f, g = 1 0 f (x)g(x)dx 3. v = v, v is a norm on V.

13 Theorem. (Cauchy-Schwarz inequality ) If, is an inner product in a real vector space V, then f, g f g Proof: Suppose g 0. Let h = g g. It suffices to prove that f, h f. (Why? f, g f g iff f, h f.) Denote α = f, h. Then 0 f αh 2 = f αh, f αh = f 2 α h, f α f, h + α 2 = f 2 α 2 Hence, α = f, h f. This completes the proof.

14 Chapter 2: Topology of M = R n Throughout this chapter, assume M = R n ( the Euclidean space ) with the metric d(x, y) = n i=1 x i y i 2 = x y Definition. (D(x, ɛ), open, neighborhood) D(x, ɛ) := {y M : d(y, x) < ɛ} is called ɛ-ball (or ɛ-disk) about x. A M is open if x M, ɛ > 0 s.t. D(x, ɛ) A. A neighborhood of x is an open set A containing x. Open sets: (a, b), D(x, ɛ), {(x, y) R 2 : 0 < x < 1}. The union of an arbitrary collection of open subsets of M is open. (Why?) The intersection of a finite number of open subsets of M is open. (Note that n=1 ( 1/n, 1/n) = {0} is closed. )

15 2.2 Interior of a set A: int(a) Definition. (2.2.1; Interior point & interior of A) Let (M, d) is a metric space and A M. x is called an interior point of A if D(x, ɛ) s.t. D(x, ɛ) A. Denote int(a) := the collection of all interior points of A. Examples. Proofs are very easy. If A = [0, 1], then int(a) = (0, 1). int{(x, y) R 2 : 0 < x 1} = {(x, y) R 2 : 0 < x<1}. If A is open, then int(a) = A. Let (M, d) be a metric space and x 0 M. int{y M : d(y, x 0 ) 1} = {y M : d(y, x 0 )<1}

16 Definition. (2.3-4: Closed sets & Accumulation Points ) A set B in a metric space M is said to be closed if M \ B is open. x M is accumulation point (or cluster point ) of a set A M if ɛ > 0, D(x, ɛ) contains y A with y x. Prove the followings: Closed sets: [a, b], {y R 2 : d(y, x 0 ) 1}. The union of an a finite number of closed subsets of M is closed. (Note that n=1 [1/n, 2 1/n] = (0, 2) is open. ) The intersection of an arbitrary family of closed subsets of M is closed. Why? Every finite set in R n is closed. A set A M is closed iff the accumulation points of A belongs to A. A = {1, 1 2, 1 3, 1 4, } {0} is closed.

17 Definition. ( Closure of A & Boundary of A) Let (M, d) is a metric space and A M. cl(a) :=the intersection of all closed set containing A. A = bd(a) = cl(a) cl(m \ A) is called the boundary of A Examples Closure: cl((0, 1)) = [0, 1], cl{(x, y) R 2 : x > y} = {(x, y) R 2 : x y}. Boundary bd((0, 1)) = {0, 1}, bd{(x, y) R 2 : x > y} = {(x, y) R 2 : x = y}. Let (M, d) is a metric space and A M. Prove that cl(a) = A { accumnulation points of A}. x cl(a) iff inf{d(x, y) : y A} = 0. x bd(a) iff ɛ > 0, D(x, ɛ) A & D(x, ɛ) (M \ A).

18 Definition. (Sequences & Completes) Let (M, d) is a metric space and x k a sequence of points in M. lim k x k = x iff ɛ > 0, N s.t. k N d(x, x k ) < ɛ. x k is Cauchy seq. iff ɛ > 0, N s.t. k, l N d(x k, x l ) < ɛ. x k is bounded iff B > 0 & x 0 M s.t. d(x k, x 0 ) < B for all k. x is a cluster point of the seq. x k iff ɛ, infinitely many k with d(x k, x) < ɛ. The space M is called complete if every Cauchy seq. in M converges to a point in M. In a metric space, it is easy to prove the followings: Every convergent seq. is a Cauchy seq. A Cauchy seq. is bounded. If a subseq. of a Cauchy seq. converges to x, then the sequence itself converges to x.

19 Chapter 3: Compact & Connected sets Throughout this chapter, we assume that (M, d) is a metric space. Definition. (3.1.1: Sequentially compact & Compact) Let A M. A is called sequentially compact if EVERY sequence in A has a subsequence that converges to a point in A. A is compact if EVERY open cover of A has FINITE subcover. An open cover of A is a collection {U i } of open sets such that A i U i. An open cover {U i } of A is said to have finite subcover if a finite subcollection of {U i } covers A. In chapter 1, we proved that every sequence x n in the closed interval [a, b] has a subsequence that converges to a point in [a, b]. Hence, [a, b] is sequentially compact.

20 Examples of compact set 1. Prove that the entire line R is NOT compact. Proof. Clearly, {D(n, 1) : n = 0, ±1, ±2, } is open cover of R but does not have a finite subcover (why?). 2. Prove that A = (0, 1] is not compact. Proof. Clearly, (0, 1] = n=1 (1/n, 2). Hence, {(1/n, 2) : n = 1, 2, } is an open cover of (0, 1] but does not have a finite subcover. 3. Heine-Borel thm. Let A M = R n. A is compact iff A is closed and bounded. Proof. later. 4. Give an example of a bounded and closed set that is not compact. Sol n. Let M = {e n : n = 1, 2, } where e 1 = (1, 0, 0, ), e 2 = (0, 1, 0, ),. Let d(e i, e j ) = 2 if i j. Then (M, d) is a metric space. The entire metric space M is closed and bounded (why?). {D(e n, 1) ; n = 1, 2 } is open cover of M but but does not have a finite subcover (why?). Hence, M is not compact.

21 Theorem. (3.1.3; Bolzano-Weirstrass theorem) A M is compact iff A is sequentially compact. Lemma 1: Let A M. If A is compact, then A is closed. Proof. We will show M \ A is open. Let x M \ A. 1. A n=1 U n where U n = M \ D(x, 1/n) open set. 2. Since A is compact and {U n } covers A, a finite subcover, that is, N s.t. A N n=1 U n. = U N 3. Hence, D(x, 1/N) U c N Ac = M \ A and therefore M \ A is open. Lemma 2: Let A B M. If B is compact and A is closed, then A is compact. Proof. Let U i be an open covering of A. 1. Set V = M \ A. Note that V is open. 2. Thus {U i, V } is an open cover of B. 3. Since B is compact, B has a finite cover, say, {U 1,, U N, V }. Hence, A U 1 U N.

22 Lemma 4: If A is sequentially compact, then A is totally bounded. 1. Definition of totally bounded: A M is totally bounded if ɛ, finite set {x 1,, x N } M s.t. A N i=1 D(x i, ɛ). 2. Proof. If not, then for some ɛ > 0 we cannot cover A with finitely many disks. (i) Choose x 1 A and x 2 A \ D(x 1, ɛ). (ii) By assumption, we can repeat; choose x n A \ n 1 i=1 D(x i, ɛ) for n = 1, 2,. (iii) This seq {x n} satisfies d(x n, x m) > ɛ for all n m. (iv) Hence, x n has no convergent subseq., a contradiction. Summery. Let A M. A is compact A is closed A is a closed subset of a compact set A is compact. A is sequentially compact A is totally bounded.

23 Proof of B-W thm ( ): If A is compact, then A is sequentially compact. Let A be compact. Let {x n } be a seq. in A. 1. To derive a contradiction, assume that {x n } has no convergent subseq. 2. Then, {x n } has infinitely many distinct points {y k } which has no accumulation points. (Why? If not, convergent subseq. ) 3. Hence, some neighborhood U k of y k containing no other y i. 4. {y n } is closed because it has no accumulation points. Hence, {y n } is compact by Lemma 2. Lemma2: Any closed subset of the compact set A is compact. 5. But {U k } is an open cover that has no finite subcover, a contradiction. 6. Hence, x n has a convergence subsequence. The limit lies in A, since A is closed by Lemma 1. Hence, x n has a subsequence that converges to a point in A.

24 Proof of B-W thm ( ): If A is sequentially compact, than A is compact. Suppose {U i } is an open cover of A. We need to prove that {U i } has finite subcover. r > 0 s.t. y A, D(y, r) U i for some U i. Why? 1. If not, y n A s.t. D(y n, 1/n) is not contained in any U i. 2. By assumption, {y n } has a subseq., say, y nk z A. Since z A i U i, z U i0 for some U i0. 3. Since U i0 is open, ɛ > 0 s.t. D(z, ɛ) U i0. 4. Since y nk z, N = n k0 2/ɛ s.t. y N D(z, ɛ/2). 5. But D(y N, 1/N) D(z, ɛ) U i0 (why?), a contradiction. Since A is totally bounded (see Lemma 4), we can write A D(y 1, r) D(y n, r) for finitely many y i. Since D(y k, r) U ik for some U ik, A U i1 U in, finite subcover. Hence, A is compact.

25 Theorem. (3.15; Compact Closed and Totally Bounded) Let A M. A is compact iff A is complete and totally bounded. (Proof of ) Assume A is compact. 1. A is compact totally bounded & sequentially compact. 2. A is sequentially compact A is complete. (Proof of ) Assume A is is complete and totally bounded. It suffices to prove that A is sequentially compact. Assume that {y n } is a sequence in A. 1. We may assume that the y k are all distinct. (Why? If not,...) 2. Since A is totally bounded, for each k = 1, 2, x k1,, x klk M s.t. A D(x k1, 1/k) D(x klk, 1/k) 3. Nest page...

26 Theorem. (Continue...) Let A M. A is compact iff A is complete and totally bounded. (Proof of ) Assume A is is complete and totally bounded. It suffices to prove that A is sequentially compact. Assume that {y n } is a sequence in A. 1. We may assume that the y k are all distinct. (Why? If not,...) 2. Since A is totally bounded, for each k = 1, 2, x k1,, x klk M s.t. A D(x k1, 1/k) D(x klk, 1/k) 3. For k = 1, an infinitely many y n lie in one of these disks D(x 1,j, 1). Hence, we can select a subseq. {y 11, y 12, } lying entirely in one of these disks. 4. Repeat the previous step for k = 2 and obtain the subseq. {y 21, y 22, } of {y 11, y 12, } lying entirely in one of these disks D(x 2j, 1/2). 5. Now choose the diagonal subsequence y 11, y 22, y 33,. This sequence is Cauchy seq. because d(y ii, y jj ) max{1/i, 1/j}. 6. Since A is complete, y ii converges to a point in A.

27 Theorem. (3.2.1, Heine-Borel thm.) Let A M = R n. A is compact iff A is closed and bounded. Proof. Recall Thm 3.1.5: A is compact iff A is closed and totally bounded. Since M = R n is Euclidean space, A is bounded A is totally bounded Caution: If M is not Euclidean space, the above statement is not true. See Example where there is an example that A is bounded but not totally bounded.

28 Theorem. (3.3.1: Nested Set Property) Let F k be a sequence of compact non-empty set in a metric space M such that F 1 F 2 F 3. Then, k=1 F k. 1. For each n, choose x n F n. 2. Since {x n } F 1 and F 1 is compact, a subseq {x nk } that converges to some point z in F 1, that is, x nk z F 1 3. With a rearrangement, we may assume that x n z. (why?) 4. n > N = x n F n F N = x n F N 5. Since lim j x N+j = z & x N+j F N & F N is compact, it must be z F N, N = 1, 2, 3, This completes the proof.

29 Definition. (Path-Connected Sets) φ : [a, b] M is said to be continuous if t k [a, b] t = φ(t k ) φ(t) A continuous path joining x, y M is a continuous mapping φ : [a, b] M such that φ(a) = x, φ(b) = y. A M is said to be path-connected if for any x, y A, there exists a continuous path φ : [a, b] M joining x and y such that φ([a, b]) A.

30 Definition. (3.5.1: Separate, Connected Sets) Let A be a subset of a metric space M. Two open set U, V are said to be separate A if 1. U V A = 2. U A & V A 3. A U V. A is disconnected if such sets U, V exist. A is connected if such sets U, V do not exist.

31 Theorem. (3.3.1) Path-connected sets are connected. 1. Clearly, [a, b] is connected. 2. To derive a contradiction, suppose A is path-connected but not connected. Then open sets U, V such that (i) U V A = & A U V (ii) x U A & y V A 3. Since A is path-connected, a continuous path φ : [a, b] M s.t. φ(a) = x, φ(b) = y, φ([a, b]) A. 4. From Theorem which we will learn soon, φ([a, b]) is connected. This is a contradiction since U, V separate φ([a, b]).

32 Example 3.1 Show that A := {x R n : x 1} is compact and connected. Proof. 1. Since A is closed and bounded, A is compact by Heine-Borel thm. 2. To prove connectedness, let x, y A. 3. Define φ : [0, 1] R n by φ(t) = tx + (1 t)y. Clearly, φ is continuous path joining φ(0) = x and φ(1) = y. 4. φ(t) t x + (1 t) y t + (1 t) = 1 for t [0, 1]. Hence, φ([0, 1]) A. 5. Hence, A is path-connected.

33 Example 3.2 Let A R n, x A and y R n \ A. Let φ : [0, 1] R n be a continuous path joining x and y. Show that t 0 s.t. φ(t 0 ) bd(a). 1. Let t 0 = sup{t : φ([0, t]) A}. This is well-defined because φ(0) = x A. 2. If t 0 = 1, clearly y = φ(t 0 ) bd(a). 3. Assume 0 t 0 < 1. From the definition of t 0, for n = 1, 2,, t n s.t. t 0 t k t n & φ(t n ) A c 4. Since φ(t n ) A c φ(t 0 ), φ(t 0 ) bd(a).

34 Chapter 4. Continuous Mappings Throughout this chapter, we assume that M = R n and N = R m are Euclidean space with the standard metric n d(x, y) = x y = (x i y j ) 2, x, y M j=1 m ρ(v, w) = v w = (v i w j ) 2, v, w N Please note that the same symbol may have different norm depending on its context. Throughout this chapter, we assume that A M = R n and j=1 f : A N = R m is a mapping.

35 Definition. (4.1.1: Continuity of f : A N) Suppose that x 0 {accumulation points of A}. lim x x0 f (x) = b if ɛ > 0, δ > 0 s.t. We write 0 < x x 0 < δ & x A f (x) b < ɛ Let x 0 A. We say that f is continuous at x 0 if either x 0 {accumulation points of A} or lim x x0 f (x) = f (x 0 ). Let B A. f is called continuous on B if f is continuous at each point on B. If A = B, we just say that f is continuous.

36 Theorem. (4.1.4: Continuity of f : A N) The following assertions are equivalent. 1. f is continuous on A 2. For every convergent seq x k x 0 in A, we have f (x k ) f (x 0 ). 3. For each open set U in N, f 1 (U) is open relative to A; that is, f 1 (U) = A V for some open V 4. For each closed set F in N, f 1 (F ) is closed relative to A; that is, f 1 (F ) = A G for some close G Proof. 1 easy = 2? = 4 easy = 3? = 1

37 Proof of (2 = 4) Let F N be closed. We want to prove that f 1 (F ) is closed relative to A. We begin with reviewing the definition of closed. 1. B is closed iff B = B {accumulation points of B}. 2. B is closed iff for every sequence {x k } B that x k x 0, we necessary have x 0 B. 3. B A is closed relative to A iff B = (B {accumulation points of B}) A 4. B A is closed relative to A iff for every sequence {x k } B that x k x 0 A, we necessary have x 0 B. 5. Proof of (2 = 4). Let x k f 1 (F ) and let x k x 0 A. By 2, f (x k ) f (x 0 ). Since F is closed, f (x 0 ) F. x 0 f 1 (F ). f 1 (F ) is closed relative to A.

38 Proof of (3 = 1) For given x 0 A and ɛ > 0, we must find δ > 0 such that x x 0 < δ & x A }{{} x D(x 0,δ) A f (x) f (x 0 ) < ɛ }{{} f (x) D(f (x 0 ),ɛ) 1. Since D(f (x 0 ), ɛ) is open, by 3 f 1 (D(f (x 0 ), ɛ)) is open relative to A. f 1 (D(f (x 0 ), ɛ)) = A V for some open set V. 2. Since x 0 V and V is open, δ > 0 s.t. D(x 0, δ) V. 3. Hence, D(x 0, δ) A f 1 (D(f (x 0 ), ɛ)) and this completes the proof.

39 Theorem. (4.2.1: f (connected) is connected if f C(M)) Suppose that f : M N is continuous and let K M. (i) If K is connected, so is f (K). (ii) If K is path-connected, so is f (K). Proof of (i). Suppose f (K) is not connected. 1. From the definition of disconnectedness, open U, V s.t. f (K) U V, U V f (K) =, U f (K), V f (K) 2. Since f is continuous, f 1 (U) and f 1 (V ) are open. Moreover, K f 1 (U) f 1 (V ), f 1 (U) f 1 (V ) K =, f 1 (U) K, f 1 (V ) K. 3. Hence, K is disconnected, a contradiction.

40 Proof of (ii). If K is path-connected, so is f (K). 1. Let v, w f (K) and let x, y K s.t. f (x) = v, f (y) = w. 2. Since K is path-connected, a continuous curve c : [0, 1] M s.t. c(t) K (0 t 1), c(0) = x, c(1) = y 3. Since f is continuous, it is easy to show that c(t) = f (c(t)) f (K) for 0 t 1 and and c : [0, 1] N is continuous path joining v and w. 4. Hence, f (K) is path-connected

41 Theorem. (4.2.2: f (compact) is compact if f C(M)) Suppose that f : M N is continuous and K M is compact. Then f (K) is compact. It suffices to prove that f (K) is sequentially compact. 1. Let v n f (K). Let x n K s.t. f (x n ) = v n. 2. Since K is compact, a convergent subsequence, say, x nk x 0 K. 3. Since f is continuous, v nk = f (x nk ) f (x 0 ) f (K). This proves that f (K) is sequentially compact.

42 Examples Let f : R 2 R be a continuous map. Denote x = (x 1, x 2 ). Let f (x) = x 1 for x R 2. If K R 2 be compact, so is f (K) = {x 1 : x = (x 1, x 2 ) K}. (Why? Since f is continuous and K is compact, f (K) is compact.) Let f (x) = 7 for x R 2. The set {7} is compact, while R 2 = f 1 ({7}) is not compact. The set A = {f (x) : x = 1} is a closed interval. (Why? K = {x R 2 : x = 1} is compact and connected. Hence, A = f (K) is compact and connected. )

43 Theorem. (1) Let f : A N M and g : A N M be continuous at x 0. Then f ± αg is continuous at x 0 for any α R. fg is continuous at x 0 f /g is continuous at x 0 if g(x 0 ) 0. (2) Suppose f : A N M and h : B N R p are continuous and f (A) B. Then h f : A N R p is also continuous. Proof. EASY

44 Theorem. (4.4.1: Maximum-Minimum Principle) Let f : A M R be continuous and let K be a compact subset in A. Then, f (K) is bounded. x 0, y 0 K such that f (x 0 ) = inf f (K) = inf f (x) & f (y 0) = sup f (K) = sup f (x). x K x K Proof. Since K is compact and f is continuous on K A, f (K) is compact. Hence, f (K) is closed and bounded in R by Heine-Borel thm. This completes the proof.

45 Theorem. (4.5.1: Intermediate Value Theorem) Let f : A M R be continuous. Assume K is a connected subset in A and x, y K and f (x) < f (y). Then, For every number c R such that f (x) < c < f (y), z K s.t. f (z) = c Proof. Since K is connected and f is continuous on K A, f (K) is connected. Hence, [f (x), f (y)] f (K). z K s.t. f (z) = c. This completes the proof.

46 4.6 Uniform Continuity Throughout this section, we assume that f : A R n R m is continuous. Definition. Let B A. f is uniformly continuous on B if for every ɛ > 0, there is δ > 0 s.t. x y < δ & x, y B f (x) f (y) < ɛ. Example. Consider f : R R, f (x) = x 2. Then f is continuous on R, but it is not uniformly continuous. Why? Let x n = n + 1/n and y n = n. Then x n y n = 1/n 0, while f (x n ) f (y n ) 1. Example. Consider f : (0, 1) R, f (x) = 1/x. Then f is continuous on (0, 1), but it is not uniformly continuous. Why? Let x n = 1/n. Then x n+1 x n < 1/n 0, while f (x n+1 ) f (x n ) = 1.

47 Theorem. (Uniform Continuity Theorem) Let f : A R n R m be continuous and let K A be compact. Then f uniformly continuous on K. 1. Let ɛ be given. Since f is continuous on K, for each x K, δ x > 0 s.t. f (D(x, δ x ) K) D ( f (x), ɛ ) 2 2. Since K x D(x, δ x /2) and K is compact, {x 1,, x N } K s.t. K N j=1 D(x j, δ) where δ = 1 2 min{δ x 1,, δ xn }. 3. If x y < δ, x, y K, then x j s.t. x x j < δ. Since y x j y x + x x j < 2δ δ xj, f (x) f (y) f (x) f (x j ) + f (x j ) f (y) ɛ 2 + ɛ 2 = ɛ.

48 Chapter 5. Uniform Convergence This chapter deals with very important results in physical science: a basic iteration technique called the contraction mapping principle (5.7.1) some applications to differential and integral equations and some problems in control theory. (5.7.2, 5.7.3, ) To study such results, we need compactness in a complete metric space (5.5.3) uniform convergence, equi-continuity (5.6.2)

49 Definition. (Pointwise convergence & Uniform Convergence) Let N be a metric space with the metric ρ, A a set, and f k : A N, k = 1, 2, f k f pointwise if for each x A, lim k f k (x) = f (x), i.e. x A, lim ρ(f k(x), f (x)) = 0 k f k f uniformly if lim k sup x A ρ(f k (x), f (x)) = 0, i.e. ɛ > 0, N s.t. k > N sup ρ(f k (x), f (x)) < ɛ x A Examples: f k (x) = x k 0 pointwise in (0, 1). (Why?) f k (x) = x k does NOT converge to 0 uniformly in (0, 1). Show that f n (x) = xn 1+x converges pointwise on [0, 2] but that n the convergence is not uniform.

50 Definition. (5.1.3: Does k g k makes sense?) Denote f n (x) = n k=1 g k(x). k g k = f (pointwise) if f n f pointwise. k g k = f uniformly if f n f uniformly. Examples. ( 1) k x 2k+1 k=0 (2k+1)! = sin x uniformly in the interval [ 100, 100]. k x k = 1 1 x converges uniformly in [ 0.9, 0.9] k x k = 1 1 x converges pointwise (NOT uniformly) in ( 1, 1) k x k does not converge in R \ ( 1, 1)

51 The Weierstrass M-test Theorem. (5.2.1: Cauchy Criterion) Let V be a complete normed vector space with norm, and let A be a set. Let f k : A V is a sequence of functions. Then f k converges uniformly on A iff ɛ > 0, N s.t. l, k > N sup f k (x) f l (x) < ɛ x A Proof of. 1. Assume f k f uniformly. Let ɛ > 0 be given. 2. Then N s.t. k N f k f = sup x A f k (x) f (x) < ɛ/2. 3. Hence, l, k N f k f l f k f + f l f < ɛ 2 + ɛ 2 = ɛ.

52 Theorem. (5.2.1: Continue...) Then f k converges uniformly on A iff ɛ > 0, N s.t. l, k > N sup x A f k (x) f l (x) < ɛ Proof of. 1. From the assumption, f k (x) is Cauchy sequence for all x A. 2. Hence, for all x A, lim k f k (x) and we can define f (x) = lim k f k (x). 3. Let ɛ > 0 be given. From the assumption, N s.t. l, k > N sup x A f l (x) f l (x) < ɛ/2. 4. From 2, x A, N x s.t. l > N x f (x) f l (x) < ɛ/2. 5. From 3 and 4, if k N and x A, then f k (x) f (x) f k (x) f l (x) + f l (x) f (x) < ɛ/2+ɛ/2 for any l N x. 6. From 5, k N sup x A f k (x) f (x) < ɛ.

53 Theorem. (5.2.2: Weierstrass M test) Let V be a complete normed vector space with norm, and let A be a set. Suppose that g k : A V are functions such that sup x A g k (x) < M k and k=1 M k <. Then k=1 g k converges uniformly. Proof. 1. Denote f n (x) = n k=1 g k(x). 2. Then f n (x) f n+l (x) = n+l k=n g k(x) n+l k=n M k. 3. Since lim n k=n M k = 0, it follows from 2 and Theorem that f n converges uniformly.

54 5.5 The space of continuous functions Throughout this section, we assume M = R n, A M, and N = R n. (N,M: complete normed space) Denote C(A, N) = {f a vector space. : f : A N is continuous }. Then C is For f C(A, N), f is said to be bounded if there is a constant C such that f (x) < C for all x A. Denote C b (A, N) = {f C : f is bounded }. Define f = sup f (x) x A f is a measure of the size of f and is called the norm of f.

55 Theorem. ( : C b (A, N) is a complete normed space) Let A M = R m, N = R n. The set C b (A, N) is a complete normed space equipped with the norm f = sup x A f (x) ; that is, 1. C b (A, N) is a normed space. f 0 and f = 0 iff f = 0. αf = α f for α R, f C b. f + g f + g. 2. Completeness: Every Cauchy sequence {f k } in C b (A, N) converges to a function f C b (A, N), that is, lim f k f = lim k sup k x A f k (x) f (x) = 0. Clearly, C b (A, N) is a normed space. (EASY!) From the definition, f k f uniformly iff f k f in C b. From Cauchy criterion (Theorem 5.2.1), C b (A, N) is complete.

56 Examples Let B = {f C([0, 1], R) : f (x) > 0 for all x [0, 1]}. Show that B is open in C([0, 1], R). Proof. 1. In order to prove that B is open, we must show that f B, ɛ > 0 s.t. D(f, ɛ) B. 2. Let f B. Since [0, 1] is compact, f has a minimum value-say, m- at some point in [0, 1]. Hence, inf x [0,1] f (x) = m. 3. Let ɛ = m 2. We will show D(f, ɛ) B. Proof. If g D(f, ɛ), then g f < ɛ, and g(x) f (x) g(x) f (x) f (x) f g m ɛ = m 2 for all x [0, 1]. Hence, g B. D(f, ɛ) B Prove that B is D = {f C b : inf x [0,1] f (x) 0}. Proof. 1. D = D because if f n D f uniformly, then f n (x) f (x) pointwise and inf x [0,1] f (x) If f D, then f n (x) := f (x) + 1 n B and f n f = 1 n 0. B D B.

57 Examples Consider a sequence f n C b such that f n+1 f n r n, where rn is convergent. Prove that f n converges. Proof. 1. Let ɛ > 0 be given. 2. Since r n is convergent, N s.t. n > N r k < ɛ k=n 3. Hence, if n N, then n+k 1 f n+k f n = (f j+1 f j ) j=n n+k 1 j=n f j+1 f j 4. From 3, f n is a Cauchy sequence, so it converges. r j < ɛ j=n

58 Arzela-Ascoli Theorem Throughout this section, we assume that M = R m, A M, N = R n (N,M: complete normed space). Definition. (5.6.1: Equi-continuous) Assume B C(A, N). We say that B is equi-continuous if ɛ > 0, δ > 0 s.t. x y < δ & x, y A sup f B f (x) f (y) < ɛ We say B is pointwise compact iff B x = {f (x) : f B} is compact in N for each x A.

59 Example (Compact sequence) Let f n C b ([0, 1], R) and be such that f n exist and ( ) sup f n C & sup n n sup f n(x) x (0,1) C for a positive constant C. Prove that B := {f n } is equicontinuous. Proof. By the mean value theorem, f n (x) f n (y) sup f n(z) x y C x y, z (0,1) for all n Hence, for given ɛ > 0, we can choose δ = ɛ C and x y < δ & x, y [0, 1] sup f n (x) f n (y) < C x y < ɛ. n Hence, B := {f n } is equi-continuous. (So, {f n } has a convergent subsequence. Why? See Arzela-Ascoli theorem.)

60 Theorem. (5.6.2:Arzela-Ascoli theorem) Let A be compact and B C(A, N). If B is closed, equi-continuous, and pointwise compact, then B is compact, that is, any sequence f n in B has a uniformly convergent subsequence. The proof strategy is based on Bolzano-Wierstrass properties. Theorem. (Special case of Arzela-Ascoli theorem) Let B C([0, 1], R). If B is closed, equi-continuous, and bounded, then B is compact. Proof. 1. Assume f n is a sequence in B. 2. Denote C 1/n = { 1 n, 2 n,, n 1 n, 1}. Let C = nc 1/n. 3. Since C is countable, we can write C = {x 1, x 2, }.

61 Theorem. (Special case of Arzela-Ascoli theorem) Let B C([0, 1], R). If B is closed, equi-continuous, and bounded, then B is compact. Proof. 1. Assume f n is a sequence in B. 2. Denote C 1/n = { 1 n, 2 n,, n 1 n, 1}. Let C = nc 1/n. 3. Since C is countable, we can write C = {x 1, x 2, }. 4. Since B x1 is compact, a convergent subseq of f n (x 1 ). Let us denote this subsequence by f 11 (x 1 ), f 12 (x 1 ),, f 1k (x 1 ), 5. Similarly, the sequence f 1k (x 2 ) has a subsequence f 21 (x 2 ), f 22 (x 2 ),, f 2k (x 2 ), which is converegnt. 6. We proceed in this way and then set g n = f nn.

62 Proof of Arzela-Ascoli theorem 7. g n = f nn is obtained by picking out the diagonal f 11 f 12 f 13 f 1n (1st subseq.) f 21 f 22 f 23 f 11 (2nd sub seq.).... f n1 f n2 f n3 f nn (n-th subseq.) 8. From the construction from the diagonal process, lim g n(x i ) exists for all x i C. n 9. Now, we are ready to prove g n g m = sup g n (x) g m (x) 0 as m, n. x [0,1]

63 Continue Proof of lim n,m sup x A g n (x) g m (x) = 0. a. Let ɛ > 0 be given. b. From equi-continuity of {g n } B, we can choose δ s.t. x y < δ & x, y A = [0, 1] sup g n (x) g n (y) < ɛ/3 n c. Choose L 1 δ. From 8, N s.t. n, m > N sup x i C 1/L g n (x i ) g m (x i ) < ɛ 3. d. For each x A, there exist y j C 1/L s.t. x y j < δ. Therefore, if n, m > N, then g n (x) g m (x) g n (x) g n (y j ) + g n (y j ) g m (y j ) + g m (x) g m (y j ) ɛ 3 + ɛ 3 + ɛ 3 This proves lim n,m sup x A g n (x) g m (x) = 0.

64 Continue From 9, g n is a Cauchy sequence in C([0, 1], N). 11. Since C([0, 1], N) is the complete normed space, g n converges to some g C([0, 1], N). 12. Since B is closed, it must be g B. 13. From 1, 11, and 12, B is sequentially compact, so it is compact. The proof of Arzela-Ascoli theorem is exactly the same as the special case discussed above except the step 2. For the replacement of the step 2, we use the fact that the compact set A is totally bounded. The compactness of A provides that, for each δ > 0, there exist a finite set C δ = {y 1,, y k } such that A k j=1 D(y j, δ).

65 5.7 The contraction mapping principle Theorem. (5.7.1: Contraction mapping principle) Let M be a complete normed space and Φ : M M a given mapping. Assume k [0, 1) s.t. Φ(f ) Φ(g) k f g for all f, g M Then there exists a unique fixed point f M s.t. Φ(f ) = f. In fact, if f 0 M and f n+1 = Φ(f n ), n = 0, 1, 2,, then lim f n f = 0 n Key idea: Φ is shrinking distances: f n+1 f n = Φ(f n ) Φ(f n ) k f n f n 1 k n f 1 f 0

66 The proof of contraction mapping principle: f M s.t. Φ(f ) = f 1. Let f 0 M and f n+1 = Φ(f n ), n = 0, 1, 2,. 2. f 2 f 1 = Φ(f 1 ) Φ(f 0 ) k f 1 f f 3 f 2 = Φ(f 2 ) Φ(f 1 ) k f 2 f 1 k 2 f 1 f Inductively, f n+1 f n k n f 1 f Hence, n=0 f n+1 f n f 1 f 0 n=0 kn = f 1 f k <. 6. From the proof in Example and 4, f n converges. 7. Since M is complete, lim n f n = f for some f M. 8. Φ is uniformly continuous because Φ(f ) Φ(g) k f g. 9. From 8, lim n Φ(f n ) = Φ(f ). 10. Hence, f = lim n f n+1 = lim n Φ(f n ) = Φ(f ).

67 The proof of contraction mapping principle: Uniqueness of the fixed point f 11. To prove the uniqueness, assume g is another fixed point, i.e., Φ(g ) = g 12. Then f g = Φ(f ) Φ(g ) and f g = Φ(f ) Φ(g ) k f g Hence, (1 k) f g Since 1 k < 1, it must be f g = 0 Hence, f = g

68 Theorem. (5.7.2: Existence of sol n of Differential equations) Let A R 2 be an open neighborhood of (t 0, x 0 ). Assume f : A R is continuous function satisfying the following Lipschitz condition: f (t, x 1 ) f (t, x 2 ) K x 1 x 2 for all (t, x 1 ), (t, x 2 ) A. Then, there is a δ > 0 s.t. the equation dx(t) dt = f (t, x), x(t 0 ) = x 0 has a unique C 1 -solution x = φ(t) with φ(t 0 ) = x 0, for t (t 0 δ, t 0 + δ), i.e., φ (t) = f (t, φ(t)) for all t (t 0 δ, t 0 + δ) & φ(t 0 ) = x 0 C 1 -solution = continuously differentiable solution

69 Get insight: Proof of Theorem Before the proof, let us get some insight. Imagine that φ is the solution of dx(t) dt = f (t, x), x(t 0 ) = x 0. Since φ (t) = f (t, φ(t)) with φ(t 0 ) = x 0, φ(t) = φ(t 0 ) + t t 0 φ (s)ds = x 0 + t t 0 f (s, φ(s))ds Hence, φ is a fixed point for the map Φ : M M defined by Φ(φ) = x 0 + t t 0 f (s, φ(s))ds In order to apply the contraction mapping principle, we need to choose a suitable space M. In practice, the solution φ can be achieved from the following iterative method: φ n+1 (t) = Φ(φ n ) = x 0 + t t 0 f (s, φ n (s))ds & φ 0 = x 0

70 Proof of Theorem Let L = sup (x,t) à f (x, t) where à is a closed subset of A. Since f is continuous in A, L <. 2. Choose δ such that Kδ < 1 and {(t, x) : t t 0 < δ, x x 0 < Lδ} à 3. Denote C = C([t 0 + δ, t 0 + δ], R). From theorem 5.5.3, C is a complete normed space (or Banach space) with norm 4. Let φ = sup φ(t) t [t 0 +δ,t 0 +δ] M = {φ C : φ(t 0 ) = x 0 & φ(t) x 0 Lδ} 5. Then, M is also a complete normed space. (Why? M is closed subset of C w.r.t. the norm.)

71 Proof of Theorem Define Φ : M C by (Please find its motivation from the previous slide) Φ(φ) = x 0 + t 6. Claim: φ M Φ(φ) M. Proof. Let φ M and ψ = Φ(φ). ψ(t 0 ) = x 0 and ψ C because lim ψ(t + h) ψ(t) = lim h 0 h 0 t 0 f (s, φ(s))ds t+h t f (s, φ(s))ds lim h 0 Lh = 0 From 1, t t 0 δ ψ(t) x 0 = t t 0 f (s, φ(s))ds L t t 0 Lδ Hence, ψ M. 7. From 6, Φ maps M to M. See the condition of Theorem

72 Proof of Theorem Using the Lipschitz condition, Φ(φ 1 ) Φ(φ 2 ) = sup t [t 0 +δ,t 0 +δ] sup t [t 0 +δ,t 0 +δ] f (s, φ 1 (s)) f (s, φ 2 (s))ds K φ 1 (s) φ 2 (s) ds t t 0 t δk φ 1 φ 2 8. Since δk < 1, Φ(φ 1 ) Φ(φ 2 ) k φ 1 φ 2, k = δk [0, 1) 9. From 5.7.1, φ M s.t. Φ(φ ) = φ. t 0

73 Theorem. (5.7.3: Fredholm equation) Assume that K(x, y) is continuous on [a, b] [a, b] and M = sup K(x, y) x,y [a,b] If λ M b a < 1, then the following Fredholm equation has a unique solution in C([a, b], R): b f (x) = λ K(x, y) f (y) dy + φ(x), x [a, b] a where λ R, φ C([a, b], R). Proof. For f C([a, b], R), we define b (Φ(f ))(x) = λ K(x, y) f (y) dy + φ(x) a

74 Proof of Claim: Φ maps from C([a, b], R) to C([a, b], R). Proof. Let f C([a, b], R). We need to show that Φ(f ) is continuous. Let ɛ > 0 be given. Since [a, b] [a, b] is compact, K(x, y) is uniformly continuous. Hence, δ s.t. (x 1, y) (x 2, y) < δ & (x 1, y), (x 2, y) [a, b] [a, b] ɛ imply K(x 1, y) K(x 2, y) < f b a +1. If x 1 x 2 < δ and x 1, x 2 [a, b], then (Φ(f ))(x 1 ) (Φ(f ))(x 2 ) = b a K(x 1, y) K(x 2, y) f (y) dy δ f b a < ɛ. 2. Set k = λ M b a. Then k < 1 and Φ(f ) Φ(g) = sup x [a,b] b a K(x, y)(f (y) g(y) dy k f g 3. From 5.7.1, unique f C([a, b], R) s.t. Φ(f ) = f.

75 Theorem. (5.7.4: Volterra integral equation) Assuming K(x, y) is continuous on [a, b] [a, b], the Volterra integral equation f (x) = λ x a K(x, y) f (y) dy + φ(x) has a unique solution f (x) for any λ. Proof. For f C([a, b], R), we define (Φ(f ))(x) = λ x a K(x, y) f (y) dy + φ(x) 1. As in 5.7.4, Φ maps from C([a, b], R) to C([a, b], R). 2. Let M = sup x,y [a,b] K(x, y). Then, Φ(f )(x) Φ(g)(x) = λ x a K(x, y)(f (y) g(y))dy λ x a M f g

76 Proof of From 2, Φ 2 (f )(x) Φ 2 (g)(x) = λ λ 4. Inductively, we have x a x a λ 2 M K(x, y)(φ(f )(y) Φ(g)(y))dy M y a λ M f g dy 2 b a 2 f g 2! Φ n (f ) Φ n (g) λ n M n b a n f g n! 5. By the ratio test, λ n M n b a n n! converges. 6. Hence, we can choose N so that λ N M N b a N N! < 1. Φ N is a contraction!

77 Proof of From 6, unique f C([a, b], R) s.t. Φ N (f ) = f. 8. From 7, Φ N+1 (f ) = Φ(f ). 9. From 8, Φ(f ) is a fixed point of Φ N. 10. From 7, 9, and the uniqueness of the fixed point, it must be f = Φ(f ). What a CUTE IDEA is!

78 Examples Example Let Φ : R R be defined by Φ(x) = x + 1. Φ(x) Φ(y) = x y k x y for any k [0, 1), and Φ does not have a unique fixed point. Example Solve x (t) = x(t), x(0) = 1. Solution. Let Φ(φ)(t) = 1 + t 0 φ(s)ds. Let φ 0 = 1 and φ n+1 = Φ(φ n ), n = 0, 1,. Then φ n (t) = n k=0 1 k! tk. Hence, φ n (t) e t. Example Solve x (t) = t x(t) for t near 0 and x(0) = 3. Solution. Let Φ(φ)(t) = 3 + t 0 φ(s)ds. Let φ 0 = 3 and φ n+1 = Φ(φ n ), n = 0, 1,. Then φ n (t) = 3 n k=0 1 k! Hence, φ n (t) 3e t2 /2. ( t 2 2 ) k.

79 Examples Example Consider the integral equation f (x) = a + x 0 xe xy f (y) dy Check directly on which intervals [0, r] we get a contraction. Solution. Let K(x, y) = xe xy and let Φ(f )(x) = a + x 0 xe xy f (y) dy. Then x Φ(f ) Φ(g) = sup K(x, y)(f (y) g(y) dy x [0,r] 0 x sup K(x, y) dy f g x [0,r] 0 = sup 1 e x2 f g x [0,r] Since 0 < 1 e r 2 < 1 for any r, Φ is a contraction for any r.

80 5.8 The Stone-Weierstrass Theorem Aim of Weierstrass Theorem is to show that any continuous function can be uniformly approximated by a function that has more easily managed properties, such as a polynomial. Theorem. (5.8.1: Weierstrass-Bernstein ) Let f C([0, 1], R). There exist a sequence of polynomial p n such that lim n p n f = 0. In fact, p n (x) = n k=0 n! k!(n k)! x k (1 x) n k f (k/n) f unformly Meaning of r k (x) := n! k!(n k)! x k (1 x) n k : Imagine a coin with probability x of getting heads and, consequently, with probability 1 x of getting tails. In n tosses, the probability of getting exactly k heads is that quantity.

81 Rough proof: Weierstrass-Bernstein n k=0 r k(x) = 1 and n k=0 (k/n x)2 r k (x) = x(1 x). lim r k (x) = 0, for any δ > 0 n k n x >δ and lim n k n x <δ r k (x) = 1, for any δ > 0 Suppose that in gambling game called n-tosses, f (k/n) dollars is paid out when exactly k heads turn up when n tosses are made.m The average amount (after a lo ong evening of playing n-tosses) paid out when n tosses are made is p n (x) = n r k (x) f (k/n) f (x) k=0

82 The Weierstrass-Bernstein theorem can be applied to C([a, b], R) because g C([a, b], R) f (x) = g(x(b a) + a) C([a, b], R). Theorem. (5.8.2: Stone-Wierstrass) Let M be a metric space, A M a compact set, and B C(A, R) satisfies the following: 1. B is algebra: f, g B &α R f + g, fg, αg B 2. 1 B 3. x, y A, x y, f B s.t. f (x) f (y). Then B is dense in C(A, R), that is, B = C(A, R). The proof is easy (just technical). I just provide a rough insight. 1. Since B is algebra, f B p n (f ) B. 2. Assume that A is a finite set. Then the proof is trivial. 3. Use the concept of finite δ net for the compact set A.

83 Differentiable Mappings Definition: Let A be an open set in R n. A mapping f : A R n R m is said to be differentiable at x 0 A if a linear function (m n matrix) Df(x 0 ) : R n R m such that f(x) f(x 0 ) Df(x 0 )(x x 0 ) lim x x 0 x x 0 Theorem If f : A R n R m is differentiable, then f j x i exist, and f 1 f 1 f x 1 x 2 1 x n f 2 f Df(x) = x 1 2 x n (called Jacobian matrix) f m x 1 f m x n 1-Dimension. If f : (a, b) R is differentiable at x 0, then a number m = f (x 0 ) such that f(x) f(x 0 ) m(x x 0 ) lim x x 0 x x 0 = 0 = 0 or lim x x0 f(x) f(x 0 ) x x 0 = m 1

84 Thm If f : A R n R m is differentiable at a, then f is continuous at a and Df(a) is uniquely determined. Proof of uniqueness. satisfying f(x) f(a) L 1 (x a) lim x a x a Let L 1 and L 2 be two m n matrix (or linear mappings) = 0 = lim x a f(x) f(a) L 2 (x a) x a It suffices to prove that L 1 e j L 2 e j = 0 for j = 1,, n. L 1 e j L 2 e j = 1 h L 1(he j ) L 2 (he j ) = L 1(he j ) L 2 (he j ) he j = f(a+he j) f(a) L 1 (he j ) [f(a+he j ) f(a) L 2 (he j )] he j f(a+he j) f(a) L 1 (he j ) he j + f(a+he j) f(a) L 2 (he j ) he j 0 as h 0 Proof of continuity: Since lim y a f(y) f(a) Df(a)(y a) = 0, lim y a f(y) f(a) = 0. 2

85 Thm Assume f : A R n R m is differentiable at x and Df(x) = [a ij ]. Then, f j x exist and a i ij = f j x. i Proof. have Denote e 1 = (1, 0,, 0), e 2 = (0, 1, 0,, 0), e n = (0,, 0, 1). We lim y x f(y) f(x) Df(x)(y x) y x = 0 lim h 0 f(x+he i ) f(x) Df(x)(he i ) h = 0, ı = 1, 2,, n lim h 0 m j=1 f j(x+he i ) f j (x) a ij (he i ) 2 h = 0, j = 1, 2,, n f j x i exists and a ij = f j x i 3

86 Thm Let f : A R n R m. If each f j x i exist and continuous on A, then f is differentiable on A. [ ] [Proof for the case n = 2, m = 1.] Let Df(x) = f x 1 (x), f x 2 (x), x A. From the mean value theorem, f(y) f(x) = f(y 1, y 2 ) f(x 1, y 2 ) + f(x 1, y 2 ) f(x 1, x 2 ) = f (u 1, y 1 ) (y 1 x 1 ) + f (x 1, u 2 ) (y 2 x 2 ) x 1 x 2 for some u i between x i and y i. Hence, f(y) f(x) Df(x)(y x) = α (y 1 x 1 ) + β (y 2 x 2 ) [ ] [ ] where α := f x 1 (u 1, y 2 ) f x 1 (x) and β := f x 2 (x 1, u 2 ) f x 2 (x). Due to continuity of f x 1 f(y) f(x) Df(x)(y x) y x and f x 2, α 0 & β 0 as y x and α (y 1 x 1 ) + β (y 2 x 2 ) = α (y1 x 1 ) 2 + (y 2 x 2 ) 2 + β as y x. This proves that lim y x f(y) f(x) Df(x)(y x) y x = 0. 4

87 Remark. About a differentiable map f : A R n R m. The proof of Thm for the general case f : A R n R m is almost same as the special case f : A R 2 R. Intuitively, x f(x 0 ) + Df(x 0 )(x x 0 ) is supposed to be the best affine approximation to f near x 0 It should be noticed that the existence of f j x i derivative Df exist. does not imply that the

88 Directional Derivatives.Let f : A R n R be real-valued function. Let e R n d be a unit vector. dt f(x + te) f(x+te) f(x) t=0 = lim t 0 t the directional derivative of f at x in the direction e. is called If f is differentiable, then lim t 0 f(x+te) f(x) t = Df(x) e. Note that the existence of all directional derivatives at a point need not imply differentiability. Example. Let f(x, y) = xy x 2 +y for x2 y and f(x, y) = 0 if x 2 = y. Note that f is not continuous at (0, 0), since lim t 0 f(t, t 3 t 2 ) = t(t lim 3 t 2 ) t 0 = 1 0 = f(0, 0). But all directional derivative of f at t 2 +t 3 t 2 (0, 0) exist: f(ta, tb) lim t 0 t for any unit vector e = (a, b). = 1 t t 2 ab t 2 a 2 + tb = a 5

89 Chain Rule 6.5.1: Let A R n be open and let f : A R m be differentiable. Let B R n be open, f(a) B, and g : B R p be differentiable. Then h = g f is differentiable on A and Dh(x) = Dg(f(x))Df(x): g 1 g y 1 1 f 1 f y m x 1 1 x n D(g f)(x) = g p f y 1 m x 1 g p y m Proof. From the assumption, it is easy to see that := {}}{ (g f)(x) (g f)(x 0 ) Dg(f(x 0 ))(f(x) f(x 0 )) lim x x 0 x x 0 := {}}{ f(x) f(x 0 ) Df(x 0 )(x x 0 ) lim x x 0 x x 0 Since (g f)(x) (g f)(x 0 ) Dg(f(x 0 ))Df(x 0 )(x x 0 ) = + Dg(f(x 0 )), it follows from the above identities that = 0 = + Dg(f(x 0 )) {}}{ (g f)(x) (g f)(x 0 ) Dg(f(x 0 ))Df(x 0 )(x x 0 ) lim x x 0 x x 0 f m x n = 0 = 0. 6

90

91 Directional derivatives and examples 1. If h(r, θ) = f(r cos θ, r sin θ), then ( h r h θ ) = ( f x f y ) ( cos θ r sin θ sin θ r cos θ ) 2. Consider a surface S defined by f(x) =constant. Then f(x) is orthogonal to this surface. Proof. Let c : [0, 1] R n be a curve lying on S and c(0) = x 0. 0 = d dt f(c(t)) = f(c(t)) c (t). This means that f(c(t)) is orthogonal to its tangent vector c (t). Since this is true for arbitrary curve on S passing x 0, f(x 0 ) is orthogonal to S at x The direction of greatest rate of increase of f(x) is f(x). 7

92 Mean Value Theorem. Suppose f : A R n R is differentiable on an open set A. For any x, y A such that the line segment joining x and y lies in A, c on that segment such that Proof. f(y) f(x) = Df(c) (y x) Define h(t) = f((1 t)x + ty). Then and therefore t 0 (0, 1) such that h(1) h(0) = h (t 0 ) f(y) f(x) = h(1) h(0) = h (t 0 ) = Df((1 t 0 )x + t }{{ 0 y) } (y x) =c 8

93 Definition. A bilinear map B : R m R n R is n m matrix such that B(x, y) = a ij x i y j = (x 1,, x n ) a 11 a 1m..... y 1. ij a n1 a nm v m Definition For positive integer r, f is said to be of class C r if all partial derivatives up to order r exist and continuous. Let f : A R n R is of class C 2. Then 2 f D 2 x 1 x 1 f(x) = f x n x 1 If D 2 f us continuous, D 2 f is symmetric. 2 f x 1 x n. 2 f x n x n 9

94 Taylor s Theorem [Case:f C 3 ]. Let f : A R n R is of class C 3. Suppose x A and x + th A for 0 t 1. Then c = x + t 0 h, 0 < t 0 < 1, such that Proof. where f(x + h) f(x) = f(x + h) f(x) = 1 = n = n i=1 = n i=1 1 f i=1 [ 0 f 0 n i= ! f (x)h i + 1 x i 2! n ( i,j,k=1 n i,j=1 2 f x i x j (x)h i h j 3 f x i x j x k (x + t 0 h)h i h j h k d dt f(x + th)dt = 1 n 0 i=1 x i (x + th)h i d(t 1) x i (x)h i 1 d 0 dt f x i (x)h i + R 1 (h, x) R 1 (h, x) = n i,j=1 1 0 ( f x i (x + th)h i ) (t 1) dt (1 t) dt dt (Why? d(t 1) ( ) 2 f (x + th)h i h j x i x j ) f x i (x + th)h i dt = 1) dt ] dt 10

95 Using d dt where ( ) = (1 t) and integration by part, R 1 (h, x) = ( n (t 1)2 2! = 1 2! n i,j=1 R 2 (h, x) := n i,j,k=1 i,j=1 1 0 d dt (t 1)2 2! 2 f x i x j (x)h i h j + R 2 (h, x) 1 0 (t 1) 2 Recall the second mean value theorem for integral 1 0 f(t)g(t)dt = g(t 0 ) 2! 1 0 ) ( 2 f x i x j (x + th)h i h j ) dt ( ) 3 f (x + th)h i h j h k x i x j x k f(t)dt for some 0 < t 0 < 1. Hence, t 0, 0 < t 0 < 1 such that n ( ) 3 f 1 (t 1) 2 R 2 (h, x) = (x + t 0 h)h i h j h k dt. x i x j x i,j,k=1 k 0 2! }{{} One can proceed by using induction using the same method to get the general Taylor s theorem. 1 3! dt

96 Taylor s Theorem [General Case:f C r ]. Let f : A R n R is of class C r. Suppose x A and x + th A for 0 t 1. Then f(x + h) = f(x) + Df(x) h r! Dr 1 f(x) (h,, h) + R r 1 (x, h) where R r 1 (x, h) is the remainder. Furthermore, R r 1 (x, h) h r 1 0 as h 0 Another proof of Taylor s formula. Let g(t) = f(x+th) for t [0, 1]. Applying one-dimensional Taylor s formula, there exists t (0, 1) such that g(1) = g(0) + r 1 k=1 1 k! g(k) (0) + 1 (r 1)! g(k 1) ( t) Note that R r 1 (x, h) = + 1 r! g(k 1) ( t), g(1) = f(x + h), g(0) = f(x), g (0) = Df(x) h = n i=1 f x i (x)h i g (0) = D 2 f(x) (h, h) = n i,j=1 g (0) = D 3 f(x)(h, h, h) = n i,j,k=1 2 f x i x j (x)h i h ( j ) 3 f x i x j x k (x)h i h j h k 11

97 Theorem If f : A R n R is differentiable and x 0 A is an extreme point for f, then Df(x 0 ) = 0. Proof. Assume Df(x 0 ) 0. We try to prove that f(x 0 ) is not a local extreme value. Let h = Df(x 0) Df(x 0 ). Since f is differentiable at x 0, lim λ 0 1 λ f(x 0 + λh) f(x 0 ) Df(x 0 ) (λh) = 0. Hence, (for given ɛ = Df(x 0) 2 ) there exist δ > 0 such that 0 < λ < δ f(x 0 + λh) f(x 0 ) Df(x 0 ) (λh) < Df(x 0) λ 2 Since Df(x 0 ) h = Df(x 0 ), we have Df(x 0) λ < f(x 0 + λh) f(x 0 ) Df(x 0 ) λ < Df(x 0) λ 2 2 This leads to the followings: 12

98 for 0 < λ < δ, Df(x 0) 2 λ < f(x 0 + λh) f(x 0 ). Hence, f(x 0 ) is not local maximum. for δ < λ < 0, f(x 0 + λh) f(x 0 ) < Df(x 0) 2 λ. Hence, f(x 0 ) is not local minimum.

99 Theorem Suppose f : A R n R is a C 3 function and x 0 is a critical point. If f has a local maximum at x 0, then H x0 (f) is negative semi-definite. If H x0 (f) is negative ( positive ) definite, then f has a local maximum (minimum) at x 0 Indeed, this theorem holds true for f C 2. Proof. Since Df(x 0 ) = 0, Taylor s theorem gives where lim h 0 R 2 (x 0,h) h = 0. f(x 0 + h) f(x) = 1 2 D2 f(x 0 )(h, h) + R 2 (x 0, h) If D 2 f(x 0 ) is negative definite, then 1 2 D2 f(x 0 )(h, h) + R 2 (x 0, h) < 0 for sufficiently small h and therefore f(x 0 + h) f(x 0 ) < 0 for sufficiently small h. Hence, f(x 0 ) has a local maximum at x 0. 13

100 ( ) a b Example The matrix A = is positive definite if b d ( ) ( ) a b x (x, y) > 0 if (x, y) (0, 0). b d y Hence, A is positive definite iff ax 2 + 2bxy + dy 2 > 0 for all all x, y. Therefore, A is positive definite iff a > 0 and ad b 2 > 0. Example ( Let f(x, ) y) = x 2 xy + y 2. Then Df(0, 0) = (0, 0) and 2 1 D 2 f(0, 0) =. Hence, the Hessian is positive definite. Thus 1 2 f has a local minimum at (0, 0). 14

101 Chapter 8. Integration Definition. function. Let A R 2 be a bounded set and let f : A R be a bounded We enclose A in some rectangle B = [a 1, b 1 ] [a 2, b 2 ] and extend f to the whole rectangle by defining it to be zero outside of A. Let P be a partition of B obtained by dividing a 1 = x 0 < x 1 < < x n = b 1 and a 2 = y 0 < y 1 < < y m = b 2 : P = {[x i, x i+1 ] [y j, y j+1 ] }{{} =subrectangle R Define the upper sum of f: : i = 0, 1,, n 1, j = 0, 1,, m 1}. U(f, P) := R P sup{f(x, y) (x, y) R} (volume of R) Define the lower sum of f: L(f, P) := R P inf{f(x, y) (x, y) R} (volume of R) 1