ANALYSIS Lecture Notes

Transcription

1 MA2730 ANALYSIS Lecture Notes Martins Bruveris 206

2

3 Contents Sequences 5. Sequences and convergence 5.2 Bounded and unbounded sequences 8.3 Properties of convergent sequences 0.4 Sequences and functions 3.5 Some important limits 4.6 Bounded sets 5.7 Monotone sequences 7.8 Tail of a sequence 9.9 Subsequences 20.0 Theroem of Bolzano Weierstrass 2. Cauchy sequences 22 2 Series Definition Necessary condition for convergence Comparison test D Alembert test Integral test Alternating series test Absolute convergence 43 3 Power Series Definition and convergence Continuity and differentiability Taylor series Exponential and trigonometric functions Abel s theroem 60 A A bit of history 65 A. Zeno s paradoxes 65 A.2 Augustin-Louis Cauchy 66 Bibliography 67 3

4 4 CONTENTS

5 Chapter Sequences Because I longed to comprehend the infinite I drew a line between the known and unknown. Elizabeth Bartlett. Sequences and convergence A sequence is an infinite list of numbers. Examples of sequences are, 2, 3, 4, 5, 6,...,,,,,,... 4, 2,, 2, 4,, , 3, 5, 7,, 3,... 3, 0, 5, 6, 8, 4, 2,... 3, 3., 3.4, 3.4, 3.45, 3.459,... The dots indicate that the sequence does not stop after 6 elements, but continues indefinitely. It is the finiteness of paper that forces us to stop at some point. In this regard the mathematical usage of the term sequence differs from the colloqial one: in mathematics a finite, ordered list of numbers is a tuple. We can speak of a 3-tuple (7, 2, 5) or an n-tuple (, 2,..., n). A sequence however is an infinite list of numbers. Formally we have the following definition. Definition... A sequence is a map 2 x :. We will write x n := x(n) for the nth element of the sequence and denote the whole sequence by {x n } or {x n}. How do we capture infinitely many elements on a finite sheet of paper? We do this by providing the general rule that allows us to calculate any element of the sequence. For In Carl C. Gaither and Alma E. Cavazos-Gaither, Gaither s Dictionary of Scientific Quotations (p. 047). 2nd Edition. Springer, For the purpose of these notes, = {, 2, 3, 4,... }. 5

6 6 CHAPTER. SEQUENCES example, the first four sequences shown above are defined by the following rules: x n = n x n = ( ) n x n = 2 2 n x n is the n-th prime number. Defining a sequence using dots as in 2, 4, 6, 8, 0,... places the burder upon the reader to divine the general rule underlying the sequence. Therefore this method should be used only when there is no doubt what the general rule is. Remark..2. The notation {x n } and especially the short form {x n} can easily be confused with the notation of a set: a sequence is not a set. There are two key differences, between sequences and sets. First, a sequence keeps track of the order of elements. The two sequences are different, while the two sets, 2, 3, 4, 5, 6,... 2,, 4, 3, 6, 5,... {, 2, 3, 4, 5, 6,... } {2,, 4, 3, 6, 5,... } are the same. Second, elements in a sequence can be repeated. We have for example to constant sequence,,,,..., but as sets {,,,,... } = {}. Similarly, the two sequences, 2, 2, 3, 3, 3,..., 2, 3, 4, 5, 6,... are different, because they differ in their third element as well as all the following ones. It is possible to associate a set to a sequence {x n }. The set {x n : n } is the set containing all elements of the sequence. {x n : n } = {, }. For example, if x n = ( ) n then The key concept in analysis is convergence. This is not something entirely new. In calculus you have come across limits of functions, such as lim x x = 0. Similarly, the derivative of a function is defined as a limit, f f (x + h) f (x) (x) = lim, h 0 h and so is the definite integral b f (x) dx. Convergence is the process that gives rise to limits a and the limit is the number that is the result of this process. Convergence and limits are two sides of the same coin.

7 .. SEQUENCES AND CONVERGENCE 7 Convergence of sequences is were the concepts of convergence and limits are distilled down to the essentials. Consider the sequence 0., 0.0, 0.00, 0.000,..., given by the rule x n = 0 n. We can observe two things: first, the elements of this sequence are strictly greater than 0 and, second, they get closer and closer to 0 as we follow the sequence. The notion of convergence is a way to make precise, what it means for a sequence to approach a number. Informally we say that the sequence {x n } converges to 0, because eventually elements of the sequence become arbitrary small. The following definition is a rigorous version of this statement. Definition..3. A sequence {x n } converges to x if for every ɛ > 0, there exists an N, sucht that for all n N, x n x < ɛ. If {x n } converges to x, we say {x n } is convergent and x is the limit of {x n }. We write lim x n = x, n or equivalently x n x as n. Sometimes we omit the indexing variable n and write simply lim x n = x or x n x. What is the purpose of such a complicated-looking definition? It is hard to improve the following description of our motivation taken from [, p.44]. The real purpose of creating a rigorous definition for convergence is not to have 2n a tool to verify computational statements such as lim n n+2 = 2. Historically, a definition of the limit like Definition..3 came 50 years after the founders of calculus began working with intuitive notions of convergence. The point of having such a logically tight description of convergence is so that we can confidently state and prove statements about convergence of sequences in general. We are ultimately trying to resolve arguments about what is and what is not true regarding the behaviour of limits with respect to the mathematical manipulations we intend to inflict upon them. Lemma..4. The limit of a convergent sequence is unique. Proof. Let {x n } be a convergent sequence and assume it converges to both x and y. Let ɛ > 0 be given. Then there exists N, such that for all n N, we have the two inequalities x n x < ɛ 2 x n y < ɛ 2. Then we can estimate using the triangle inequality, x y = (x x n ) + (x n y) x n x + x n y < ɛ 2 + ɛ 2 = ɛ. We have shown that x y < ɛ for all ɛ > 0. It follows by Proposition..5 that x = y. This shows that the limit of a convergent sequence is unique.

8 8 CHAPTER. SEQUENCES This lemma allows us to speak of the limit of a sequence and justifies the notation lim n x n. The limit might not exist, but if it does, there is only one. In the proof we used the following property of real numbers. Lemma..5. Let x, y be two real numbers. If for all ɛ > 0 the inequality holds, then x = y. x y < ɛ Proof. We proceed by contradiction. Assume x y. Then x y > 0 and we can define ɛ := 2 x y. Clearly ɛ > 0 and by the assumption of the lemma, which, after division by x y, becomes x y < ɛ = x y, 2 < 2. This cannot be and therefore we must have x = y. This lemma may seem obvious and indeed it is. Its importance stems from the fact that this is how equalities are often proved in analysis. It can be difficult or even impossible to show x = y directly because, e.g., x or y are defined as limits. Instead one shows as in Lemma..4 that x y < ɛ for all possible ɛ > 0. This allows one to conclude that x = y..2 Bounded and unbounded sequences Definition.2.. Let {x n } be a sequence and b. () If x n b for all n, we say {x n } is bounded above and b is an upper bound for {x n }. (2) If x n b for all n, then we say {x n } is bounded below and b is a lower bound for {x n }. (3) If {x n } is bounded above and below, we say that {x n } is bounded; otherwise we call {x n } unbounded. If x n b for all n, then b is an upper bound and b a lower bound for {x n } and we say {x n } is bounded by b. Remark.2.2. Let us collect some simple consequences of this definition. () There exist sequences that are bounded above, but not below, and vice versa. For example, the sequence { n} is bounded above, but not below, while the sequence 2, 4, 6, 8,... is bounded below, but not above. (2) Upper and lower bounds of sequences are never unique. For example, if b is an upper bound for the sequence {x n } then so are also b +, b + 2, etc. In fact, every number c > b is also an upper bound.

9 .2. BOUNDED AND UNBOUNDED SEQUENCES 9 (3) An unbounded sequence can be bounded above or below or neither but not both. Examples of unbounded sequences are {( ) n n}, {e n } and { n 2 }. Examples of bounded sequences are {( ) n }, {sin n}, {2 n } and 3 n. The next result shows the relation between convergent and bounded sequences. Proposition.2.3. Every convergent sequence is bounded. Proof. Let {x n } be a convergent sequence and x its limit. We apply the definition of convergence with ɛ :=. Accodring to Definition..3 there exists an N such that for all n, x n x <. Using the triangle inequality we obtain the following bound for n, Now define x n = (x n x) + x x n x + x < + x. b := max { x,..., x N, + x }. Then we have the following inequality for all n, showing that {x n } is bounded. x n b, The definition of convergence allows us to verify whether a given sequence converges to a certain limit, but it does not make it easy to show that a given sequence does not converge. Proposition.2.3 helps with this. Example.2.4. The sequence { n} is divergent. The sequence is not bounded, because given any b we can find an n such that n > b. We can take for example n := b 2 +. Therefore by Proposition.2.3 the sequence { n} must be divergent. Among divergent sequences there are some, such as the sequence, 0, 00, 000, 0000,..., that intuitively seem to converge to +. This sequence is not bounded and therefore by Proposition.2.3 cannot be convergent. However, we can formalize the idea that the sequence grows arbitrary large. Definition.2.5. A sequence {x n } is said to diverge to + if for all b there exists N such that for all n N, x n b. In this case we write lim n x n = + or x n + as n. Similarly we say that {x n } diverges to if for all b there exists N such that for all n N, x n b. In this case we write lim n x n = or x n as n.

10 0 CHAPTER. SEQUENCES Remark.2.6. A sequence {x n } that diverges to ± does not converge. Such a sequence is still divergent in the sense of Definition..3. There are formal similarities between convergent sequences and those that diverge to ±, but there are important differences as well. Sometimes, when we want to capture both cases we will write, for example, assume lim n x n exists, either as a finite number of +, meaning that either {x n } is convergent or it diverges to +. Example.2.7. The sequence {n} =, 2, 3, 4, 5, 6,... diverges to + and the sequence { n 2 + n} = 0, 2, 6, 2, 20, 30,... diverges to. There are divergent sequences that diverge to neither + nor. An example is given by the sequence {( ) n n} =, 2, 3, 4, 5, 6, Properties of convergent sequences The next result allows us to calculate limits of sequences more easily by allowing us to decompose complicated sequences into simpler ones. Proposition.3.. Let {x n }, { y n } be two convergent sequences.. () The sequence {x n + y n } is convergent and (2) The sequence {x n y n } is convergent and lim x n + y n = lim x n + lim y n. n n n lim x n y n = lim x n lim y n. n n n (3) If lim n y n 0, the sequence xn y n is convergent and x n lim = lim n x n. n y n lim n y n Proof. Denote the limits of {x n } and {y n } by x and y respectively. We leave the proof of as an exercise. To show 2 we start with the estimate x n y n x y = x n (y n y) + (x n x)y x n y n y + x n x y. Because {x n } is convergent it is also bounded and we can choose a bound b > 0, meaning that x n b for all n. Then x n y n x y b y n y + x n x y.

11 .3. PROPERTIES OF CONVERGENT SEQUENCES Given ɛ > 0 we choose N such that for n N, Then y n y < ɛ 2b x n y n x y < b and x n x < ɛ 2( y + ). ɛ 2b + ɛ y 2( y + ) ɛ, which shows that lim n x n y n = x y. For 3 we will show that the sequence y n converges to y. The result then follows via 2. We begin with the identity y n y = y n y y n y. Given ɛ > 0 we choose N N such that for n N, Then we have the following estimate for y n, Therefore y n y < y 2 2 ɛ and y n y < 2 y. y n = y + y n y y y n y 2 y. y n y showing that lim n y n = y. = y n y y n y 2 y y 2 n y < ɛ, Corollary.3.2. Let {x n } be a convergent sequence and c. Then {cx n } is convergent and lim cx n = c lim x n. n n Proof. The constant sequence {c} is convergent and we can apply Proposition.3., part 2. Remark.3.3. The assumption in Proposition.3. that both sequences are convergent is essential. For example, the sequences {n} and { n} are both divergent, but their sum {n n} = {0} is convergent. However, we must not write lim 0 = lim n + lim ( n), n n n since the right hand side is an expression without meaning. Proposition.3. shows that convergent + convergent = convergent. What about the sum of a convergent and a divergent sequence. Let {x n } be convergent, {y n } divergent and let z n = x n + y n. Then we can write y n = z n x n, which shows that {z n } must be divergent as well, because otherwise Proposition.3. would imply that {y n } is convergent. Therefore we see that convergent + divergent = divergent.

12 2 CHAPTER. SEQUENCES Remark.3.4. Multiplication of a sequence with a divergent sequence can lead to kinds of results. Consider the three sequences x n = n, y n = n and z n = n 2. The first sequence is convergent while the latter two are divergent. We have x n y n =, which is convergent, but x n z n = n 2, which is divergent. Therefore, Similarly consider the two sequences convergent divergent = undefined. {u n } =, 0, 3, 0, 5, 0,... {v n } = 0, 2, 0, 4, 0, 6,.... Both sequences are divergent, but their product u n v n = 0 is convergent. On the other hand y n z n = n 3, which is divergent. Therefore divergent divergent = undefined. There is one special case when we can multiply a convergent sequence with a divergent one and obtain something convergent. Proposition.3.5. Let {x n }, {y n } be two sequences. If lim n x n = 0 and {y n } is bounded, then {x n y n } is convergent and lim n x n y n = 0. Proof. Assume that { y n } is bounded by b and b > 0. Let ɛ > 0 be given. Because {x n } converges to 0, there exists N such that for n N, Then we have the estimate, Therefore {x n y n } converges to 0. x n < ɛ b. x n y n = x n y n < ɛ b b = ɛ. The following lemma, sometimes also called Sandwich lemma, is often useful to find the limit of a sequence indirectly. Lemma.3.6 (Squeeze lemma). Let {x n }, {u n } and {v n } be three sequences such that for all n, u n x n v n. If {u n } and {v n } are convergent and lim u n = lim v n, n n then x n is also convergent and lim n x n = lim n u n.

13 .4. SEQUENCES AND FUNCTIONS 3 Proof. Denote the common limit of {u n } and {v n } by x. To show that {x n } converges to x, we need to estimate x n x. We can obtain the estimate by combining the two inequalities to yield x n x v n x v n x (x n x) (u n x) u n x, x n x max ( v n x, u n x ). Now let ɛ > 0 be given. Then there exists N such that for all n N v n x < ɛ and u n x < ɛ. Then for n N we also have showing that {x n } converges to x. x n x < ɛ, Proposition.3.7. Let {x n } be a convergent sequence and [a, b] an interval. If for all n, then lim n x n [a, b]. x n [a, b], Proof. Denote the limit of {x n } by x. We will first show the following simpler statement: if x n 0 for all n, then also x 0. We argue by contradiction: Assume x < 0 and let ɛ := x 2. Then there exists N such that for n N, Now we estimate x n from above via x n x < x 2. x n = x + x n x x + x n x < x + x 2 = x x 2 = x 2 < 0. However, this contradicts our assumption that x n 0. To show the general statement, note that x n [a, b] is equivalent to a x n b. Consider the sequence y n := x n a. Then {y n } is convergent and its limit y := lim n y n = x a by the above argument satisfies y 0, which is equivalent to x a. To show that x b, consider similarly the sequence z n := b x n..4 Sequences and functions Definition.4.. Let f : [0, ) be a function. We say that f converges to L as x, if for every ɛ > 0 there exists M such that for x M, f (x) L < ɛ. In this case we write lim f (x) := L. x

14 4 CHAPTER. SEQUENCES Proposition.4.2. Let {x n } be a sequence and f : [0, ) a function such that for all n we have x n = f (n). If the limit lim x f (x) exists then the sequence {x n } is convergent and lim x n = lim f (x). n x Proof. Let L := lim x f (x). To show that {x n } converges to L, let ɛ > 0 be given. Then there exists M such that for x M, f (x) L < ɛ. Choose N satisfying N M. Then for n N we have showing that {x n } converges to L. x n L = f (n) L < ɛ,.5 Some important limits Lemma.5.. Let p > 0. Then ln n lim n n = 0. p Proof. We use L Hôpital s rule to compute this limit, ln n lim n n = lim p n n = lim pnp n pn p = 0. Lemma.5.2. Let a > 0 and p. Then lim n n p e an = 0. Proof. If p < 0, then already lim n n p = 0 and because lim n e an = +, it follows that n lim p n e = 0. The case p = 0 is also simple. an Assume now p > 0. We use the following inequality e x + x, wich is valid for all x. Then we set x = an 2p and obtain e an + an 2p. 2p Hence and lim n n p e an = 0 by Lemma np e n p an ( + an/2p), 2p

15 .6. BOUNDED SETS 5 Lemma.5.3. Let a. Then e an lim n n! = 0. Proof. Define b = e a and choose N such that N 2b. When n N we can estimate b n n! = bn N! b N n (N + )... n bn N! bn n N N! N n bn because N + N, N + 2 N and so forth and b N 2. Therefore e and lim an n n! = 0 by Lemma.3.6. Lemma.5.4. We have Proof. We write 0 bn n! bn N! 2, N n n! lim n n = 0. n n! n n = n n n n... n 2 n n = n 2, because 2 n, 3 n and so on until n n. Therefore and lim n n! n n = 0 by Lemma n! n n 2 n 2, 2 N n,.6 Bounded sets Definition.6.. Let A be a set of numbers. () If there exists b such that x b for all x A, then we say A is bounded above and b is an upper bound of A. (2) If there exists b such that x b for all x A, then we say A is bounded below and b is a lower bound of A. (3) If A is both bounded above and bounded below, we say that A is bounded. (4) If there exists an upper bound b 0 of A such that whenever b is any upper bound for A we have b 0 b, then b 0 is called the least upper bound or the supremum of A. We write sup A := b 0. (5) Similarly, if there exists a lower bound b 0 of A such that whenever b is any lower bound for A we have b 0 b, then b 0 is called the greatest lower bound or the infimum of A. We write inf A := b 0.

16 6 CHAPTER. SEQUENCES To become familiar with these notions, let us look at some examples. Example.6.2. What is an example of a bounded set? The set A := {, 2, 3} is bounded. An upper bound for A is 5 and a lower bound is 2. In the same way one can see that every set with finitely many elements is bounded. How about an unbounded set? Consider the set of all real numbers. This set is not bounded. Why not? If b is a real number, then so is b + and b + b. Therefore b cannot be an upper bound for. Because this is true for all real numbers b, there exists no upper bound for and hence is unbounded. Example.6.3. There exist sets that are bounded below, but not bounded above. An example is the set of natural numbers. Clearly, is a lower bound for and therefore is bounded below. On the other hand, given any b, we can find n with n > b. Therefore b cannot be an upper bound and so no upper bound for exists. In other words, is not bounded above. In a similar way one can construct a set that is bounded above, but not bounded below. Example.6.4. What about the empty set? When we look carefully at the definition we see that is bounded both above and below, because every number b is both an upper as well as a lower bound. However, because every real number is an upper bound, there does not exist a least upper bound. In other words, the empty set has no supremum. By the same argument it has also no infimum. Lower and upper bounds of sets are never unique. This is easy to see: if b is an upper bound for the set A, then b + is also an upper bound for A. Similarly, if b is a lower bound, then b is also a lower bound. However, the next lemma shows that the supremum and infimum are unique. Lemma.6.5. Let A be a set. If the supremum (infimum) of A exists, then it is unique. Proof. Let b 0 and b both be suprema of A. Then b 0 b, because b is an upper bound and b 0 is by definition the least upper bound. On the other hand, b is also a least upper bound and hence b b 0. Taken together we obtain b 0 = b, which shows that the supremum is unique. The same argument can be used to show that the infimum is unique. Example.6.6. A supremum or infimum of A (even if it exists) does not need to be an element of A. For example, the set A := {x : x < } is bounded above and has as a least upper, but itself is not an element of A. On the other hand, if we take B := {x : x }, then the least upper bound of B is also, and in this case B. We have seen in Example.6.4 that the empty set is bounded, but has no least upper bound. Can this happen for other sets as well? It is an important and highly nontrivial property of the real numbers, that every nonempty set as a least upper bound. Theorem.6.7. Every bounded, nonempty subset A has a least upper bound and a least lower bound, i.e., sup A and inf A exist as elements of. The property, that bounded, nonempty subsets have least upper and lower bounds is called the least-upper-bound property or completeness property. The completeness property is the major difference between the real numbers and the rational numbers.

17 .7. MONOTONE SEQUENCES 7 Remark.6.8. Consider the set A := {x : x 2 < 2}. The set A is a subset of and it is bounded: an upper bound is 3 and a lower bound is 3. The least upper bound of A is 2. However, 2 is not a rational number. This shows that subsets of the rational numbers don t always have suprema, that themselves are rational numbers. We will make repeated use of Theorem.6.7 and its consequences. One of the reasons, why this theorem is important is that it shows existence of numbers, that are desribed by limit processes..7 Monotone sequences A special kind of sequences are those whose elements are either increasing or decreasing. They are important, because we can determine whether a monotone sequence converges without having to know what the limit is in advance. We will see in Theorem.7.3 that a monotone sequence is convergent if and only if it is bounded. Definition.7.. A sequence {x n } is monotone increasing if x n x n+ for all n. A sequence {x n } is monotone decreasing if x n x n+ for all n. If a sequence is either monotone increasing or monotone decreasing, we say the sequence is monotone. Sometimes such sequences are called monotonic. Example.7.2. Let us look at some examples of monotone sequences. () The sequence {/n} is monotone decreasing. Similarly, the sequence {n} is monotone increasing. We see that { n } is bounded above and below, while {n} is bounded below, but not bounded above. (2) What about the constant sequence {} or more generally {c} with c? This sequence is both monotone increasing and monotone decreasing, because c c holds. (3) The sequence {( ) n } is not monotone. The first few terms of a sample monotone increasing sequence are shown below We have seen that every convergent sequence is bounded, but not every bounded sequence is convergent. For monotone sequences these two notions coincide: a monotone sequence is bounded if and only if it is convergent. This is the content of the following theorem. Theorem.7.3. A monotone sequence {x n } is bounded if and only if it is convergent.

18 8 CHAPTER. SEQUENCES Furthermore, if {x n } is monotone increasing and bounded, then lim x n = sup {x n : n }. n If {x n } is monotone decreasing and bounded, then lim x n = inf {x n : n }. n Proof. Let us suppose the sequence {x n } is monotone increasing. Assume that the sequence is bounded. Then there exists b such that x n b for all n. In other words, the set {x n : n } is bounded above. Let x := sup {x n : n }, be the supremum. We will now show that lim n x n = x. Let ε > 0 be arbitrary. As x is the supremum, then there must exist N such that x N > x ε. Because {x n } is monotone increasing, it is easy to see (by induction) that x n x N for all n N. Hence x n x = x x n x x N < ε, for all n N. Therefore the sequence converges to x. We already know that a convergent sequence is bounded, which completes the other direction of the implication. The proof for monotone decreasing sequences follows along the same lines. Example.7.4. Consider the sequence n. First n > 0 for all n, and hence the sequence is bounded below. Let us show that it is monotone decreasing. Starting with n + n (why is that true?), we obtain n + n. So the sequence is monotone decreasing and bounded below (hence bounded). We apply Theorem.7.3 to note that the sequence is convergent and in fact lim n = inf n : n. n We already know that the infimum is greater than or equal to 0, as 0 is a lower bound. Take a number b 0 such that b n for all n. We square both sides to obtain b 2 n for all n. This implies b 2 0. As we also have b 2 0, then b 2 = 0 and so b = 0. Hence b = 0 is the greatest lower bound, and so lim n n = 0.

19 .8. TAIL OF A SEQUENCE 9 Example.7.5. A word of caution: we must show that a monotone sequence is bounded in order to use Theorem.7.3. For example, the sequence n is a monotone increasing sequence that grows very slowly. We will see, when we discuss series, in Example that this sequence has no upper bound and so does not converge. It is not at all obvious that this sequence is unbounded. A common example of where monotone sequences arise is the following proposition. The proof is left as an exercise. Proposition.7.6. Let A be a nonempty, bounded set. Then there exist monotone sequences {x n } and { y n } such that x n, y n A and.8 Tail of a sequence sup A = lim n x n and inf A = lim n y n. When we talk about convergence, we are interested in the behaviour of a sequence {x n } for large n. Intuitively it does not matter what the sequence does in the beginning. For example, convergence properties of a sequence do not change, if we change the first 3, 4, 5 or even 00, 000 elements of a sequence. Similarly, we should be able to throw away a finite number of elements of a sequence or add a finite number of elements without changing convergence properties. The notion of the K-tail makes this intuition rigorous. Definition.8.. For a sequence {x n }, the K-tail (where K ) or just the tail of the sequence is the sequence starting at K +, usually written as {x n+k } or {x n } n=k+. The main result about the tail of a sequence is the following proposition. Proposition.8.2. Let {x n } be a sequence. Then the following statements are equivalent: () The sequence {x n } converges. (2) The K-tail {x n+k } converges for all K. (3) The K-tail {x n+k } converges for some K. Furthermore, if any (and hence all) of the limits exist, then they are all equal, i.e., for any K lim x n = lim x n+k. n n Proof. It is clear that 2 implies 3. We will therefore show first that implies 2, and then we will show that 3 implies. In the process we will also show that the limits are equal. Let us start with implies 2. Suppose {x n } converges to some x. Let K be arbitrary and define y n := x n+k. We wish to show that {y n } converges and that the limit is x. Given ɛ > 0, there exists an N such that x x n < ɛ for all n N. Note that n N implies n + K N. Therefore, it is true that for all n N we have that x y n = x x n+k < ɛ.

20 20 CHAPTER. SEQUENCES Therefore {y n } also converges to x. Let us now show that 3 implies. Let K be given, define y n := x n+k and suppose that {y n } converges x. That is, given ɛ > 0, there exists an N such that x y n < ɛ for all n N. Let N := N + K. Then n N implies n K N. Thus, whenever n N we have x x n = x y n K < ɛ. Therefore {x n } converges to x. In other words, the limit does not depend on the beginning of a sequence, it only depends on the tail of a sequence. In other words, the beginning of the sequence may be arbitrary. Example.8.3. How can we use this result? For example, the sequence defined by x n := is decreasing, only if we start at n = 4 (before that it is increasing). That is, and {x n } = 7, 0, 3 25, 8, 5 4, 3 26, 7 65, 0, 9 97, 5 58,... 7 < 0 < 3 25 < 8 > 5 4 > 3 26 > 7 65 > 0 > 9 97 > 5 58 >.... n n 2 +6 This means, that if we throw away the first 3 terms and look at the 3-tail, the sequence is decreasing. The proof is left as an exercise. Since the 3-tail is monotone and bounded below by zero, it is convergent, and therefore the sequence itself is convergent..9 Subsequences A useful concept related to sequences is that of a subsequence. A subsequence of {x n } is a sequence, that contains only some of the elements of {x n } in the same order. Definition.9.. Let {x n } be a sequence and let {n i } be a strictly increasing sequence of natural numbers (that is n < n 2 < n 3 < ). The sequence is called a subsequence of {x n }. {x ni } i= Example.9.2. Here are some examples and nonexamples of subsequences. () Take the sequence n. The sequence 3n is a subsequence. To see how these two sequences fit in the definition, we take n i := 3i. (2) The elements of the subsequence must come from the original sequence, so the sequence, 0, 3, 0, 5,... is not a subsequence of n. Also, the order of elements must be preserved, so the sequence, 3, 2, 5,... is not a subsequence of n. (3) A special kind of subsequence is the tail of a sequence. The K-tail of {x n } is a subsequence. In this case we can take n i := i + K.

21 .0. THEROEM OF BOLZANO WEIERSTRASS 2 Subsequences inherit many properties from the parent sequence. For example, if {x n } is a bounded sequence then any subsequence of {x n } is also bounded. Similarly, if {x n } is monotone increasing then any subsequence is also monotone increasing. This is also true for convergence. Proposition.9.3. If {x n } is a convergent sequence, then any subsequence {x ni } is also convergent and lim x n = lim x ni. n i Proof. Suppose lim n x n = x. That means that for every ɛ > 0 we can find an N such that for all n N x n x < ɛ. It is not difficult to prove by induction that n i i. Hence i N implies n i N. Thus, for all i N we have x ni x < ɛ, and we are done. Example.9.4. This result is very useful to show that some sequences are not convergent. Consider the sequence {( ) n }. This sequence has two constant subsequences: the sequence {} and { }. The first subsequence converges to, but the second subsequence converges to. As shown in Proposition.9.3, if {( ) n } were convergent, then all subsequences would have to have the same limit. This is not the case here and thus the sequence cannot be convergent. Example.9.5. Existence of a convergent subsequence does not imply convergence of the sequence itself. Take the sequence 0,, 0,, 0,,.... That is, x n = 0 if n is odd, and x n = if n is even. The sequence {x n } is divergent, however, the subsequence {x 2n } converges to and the subsequence {x 2n+ } converges to 0..0 Theroem of Bolzano Weierstrass While it is not true that every bounded sequence is convergent, the theorem of Bolzano Weierstrass tells us that we can at least find a convergent subsequence. Theorem.0. (Bolzano Weierstrass). Every bounded sequence in has a convergent subsequence. Proof. Let {x n } be a bounded sequence. By the definition of a bounded sequence, there exist two numbers a < b, such that a x n b for all n. In other words, the sequence is contained in the interval [a, b ]. We will define the required subsequence inductively. Start with n :=, that is x n = x. Our construction proceeds as follows: let y := (a + b )/2 be the midpoint of the interval. Then at least one of the intervals [a, y] and [ y, b ] has to contain infinitely many elements of the sequence {x n }. If it is the first interval, we define a 2 := a and b 2 := y. If it is the

22 22 CHAPTER. SEQUENCES second interval, we define a 2 := y and b 2 := b. In both cases the following properties are satisfied: a a 2 b 2 b b 2 a 2 = 2 (b a ), and the interval [a 2, b 2 ] contains infinitely many elements of the sequence {x n }. Now choose n 2, such that n 2 > n and x n2 [a 2, b 2 ]. This is possible, because the interval contains infinitely many elements of the sequence. We iterate this construction. Having chosen a,..., a k and b,..., b k we let y := (a k + b k )/2 be the midpoint of the interval [a k, b k ] and we define either [a k+, b k+ ] := [a k, y] or [a k+, b k+ ] := [y, b k ], such that [a k+, b k+ ] contains infinitely many elements of the sequence {x n }. Then we choose n k+ such that n k+ > n k and x nk+ [a k+, b k+ ]. Now that we have defined the subsequence {x nk } we have to show that it is convergent. We do this using the squeeze lemma. The property x nk [a k, b k ], which holds for all k by construction, can be written as a k x k b k. If we can show that {a k } and {b k } are both convergent and have the same limit, then we are done. Note that by construction {a k } is monotone increasing and {b k } is monotone decreasing. They are also bounded, because a a 2 a k b k b 2 b. Thus a is a lower bound for both sequences and b is an upper bound. By Theorem.7.3 it follows that both sequences are convergent. Let a := lim k a k and b := lim k b k be their limits. To show that a b we consider the lengths of the intervals [a k, b k ]. The following identity can be shown by induction, (b k a k ) = 2 (b k a k ) = 2 2 (b k 2 a k 2 ) = = 2 k (b a ). When we pass to the limit we obtain lim b k a k = lim k k 2 (b k a ) = 0, and therefore b a = 0. This concludes the proof.. Cauchy sequences A convergent sequence has a limit. The notions of convergence and limit are intimately connected and this is made clear in the definition of convergence: to show that a sequence converges using Definition..3 we have to know in advance what the limit is. This is not always possible. We want to use sequences and limits to define quantities; however, in order to do so we have to solve the chicken-and-egg problem of what come first, a convergent sequence or its limit? We have seen that monotone sequences can help: for a monotone sequence convergence coincides with boundedness and boundedness can be tested without knowing the limit. But monotone sequences are often too restrictive, creating the need for a more general tool.

23 .. CAUCHY SEQUENCES 23 Example... Consider the recursively defined sequence x n+ = x n + x n 2, n 2, with initial conditions x = 0, x =. The first elements in the sequence are The sequence is not monotone, because 0,, 2, 3 4, 5 8, 6, 2 32, Never the less, each following element of the sequence is the midpoint of the interval spanned by the two previous ones and therefore the sequence is bounded: 0 x n for all n. We expect the sequence to converge, but lacking a candidate for the limit it seems difficult to prove it. The notion of a Cauchy sequence will provide a tool to test convergence without having to specify the limit in advance. Definition..2. A sequence {x n } is a Cauchy sequence if for every ɛ > 0 there exists an M such that for all n M and all k M we have x n x k < ɛ. Intuitively this means that the terms of the sequence are eventually arbitrarily close to each other. This is similar to, but different from the definition of convergence, which requires that elements of the sequence are eventually arbitrary close to the limit. The important theorem is that the notions of being Cauchy and being convergent coincide. This will be the content of Theorem..6. First, let us look at some examples. Example..3. The sequence { n } is a Cauchy sequence. Proof: Given ɛ > 0, choose M such that M > 2 ɛ. Then for n, k M we have that n < ɛ 2 and k < ɛ 2. Therefore for n, k M we have f racn k n + k < ɛ 2 + ɛ 2 = ɛ. Example..4. Let us return to the sequence defined in Example... To show that this is a Cauchy sequence we observe that the definition of the sequence can be rewritten as follows Therefore x n+ = x n + x n 2 x n+ x n = 2 (x n x n ). x n+ x n = 2 x n x n = 4 x n x n 2 = 2 n x x 0 = 2 n,

24 24 CHAPTER. SEQUENCES which implies x n+k x n x n+k x n+k + x n+k x n+k x n+ x n 2 + n+k n+k 2 2 = 2 n 2 n k 2. n Therefore {x n } is a Cauchy sequence. Proposition..5. A Cauchy sequence is bounded. Proof. Suppose {x n } is Cauchy. Pick M such that for all n, k M we have x n x k <. In particular, we have that for all n M Hence, using the triangle inequality Let Then x n B for all n. x n x M <. x n x n x M + x M < + x M. B := max { x, x 2,..., x M, + x M }. Theorem..6. A sequence of real numbers is Cauchy if and only if it converges. Proof. We start by showing that every convergent sequence is Cauchy. Let ɛ > 0 be given and suppose {x n } converges to x. Then there exists an M such that for n M we have Hence for n M and k M we have x n x < ɛ 2. x n x k = x n x + x x k x n x + x x k < ɛ 2 + ɛ 2 = ɛ. Alright, that direction was easy. Now suppose {x n } is Cauchy. The difficult part in showing that {x n } converges is that we have to find a candidate for the limit. We have shown in Proposition..5 that {x n } is bounded. Therefore by the theorem of Bolzano Weierstrass, it follows that {x n } has a convergent subsequence {x nk }. Let x be the limit of this convergent subsequence. We will now show that the sequence itself converges to x. Let ɛ > 0 be given. Because {x n } is Cauchy, we can choose M 0, such that for n, k M 0 we have x n x k < ɛ 2. Because {x nk } converges to x, there exists M, such that for k M we have x nk x < ɛ 2. Now let M = max{m 0, M } and k M. As in the proof of Theorem.0. we have n k k and therefore, if k M, then n k M as well. Therefore, using the triangle inequality, This show that {x n } converges to x. x k x x k x nk + x nk x < ɛ 2 + ɛ 2 = ɛ.

25 .. CAUCHY SEQUENCES 25 It should be noted that the Cauchy criterion is stronger than the condition that x n+ x n or x n+j x n for a fixed j converges to 0 as n. In fact, when we will study the harmonic series in Example 2.2.3, we will find a sequence such that x n+ x n = n, yet {x n} is divergent. The key point in the definition of a Cauchy sequence is that n and k are allowed to vary independently from each other and can be arbitrarily far apart. The Cauchy criterion for convergence will be especially useful for series, which we discuss in the next section.

26 26 CHAPTER. SEQUENCES

27 Chapter 2 Series Even as the finite encloses an infinite series And in the unlimited limits appear, So the soul of immensity dwells in minutia And in narrowest limits inhere. What joy to discern the minute in infinity! The vast to perceive in the small, what divinity! Jacob Bernoulli 2. Definition A particularly rich and interesting type of sequences is obtained in the following way: starting with a sequence {x n } we define a new sequence {s n } by summing up more and more elements of {x n }. So, the first elements of {s n } are s := x s 2 := x + x 2 s 3 := x + x 2 + x 3, and continuing in this way we arrive at the general term s n = x + x x n, Ut non-finitam Seriem finita cöercet, Summula, & in nullo limite limes adest: Sic modico immensi vestigia Numinis haerent Corpore, & angusto limite limes abest. Cernere in immenso parvum, dic, quanta voluptas! In parvo immensum cernere, quanta, Deum! From Jacob Bernoulli, Tractatus de Seriebus infinitis Earumque Summa Finita et Usu in Quadraturis Spatiorum & Rectificationibus Curvarum, in Ars Conjectandi (Translation by Helen M. Walker, from A Source Book in Mathematics by D. E. Smith, 929). 27

28 28 CHAPTER 2. SERIES where the dots indicate that we sum over all the elements x k with k between and n. Using the -notation for sums we can write this as s n := n x k. We say that the sequence {s n } is the series associated to the sequence {x n }. Example 2... Consider the sequence. The first terms of the associated series are n k= s = s 2 = 3 2 s 3 = = 6. Because {s n } is itself a sequence, we can write This series, which we can also write as {s n } =, 3 2, 6, 25 2, 37 60, 29 20, ,... s n = is called the harmonic series and we will encounter it several times. n k= Example Another important series is the geometric series. Let r be a real number. The geometric series is the series associated to the sequence {r n }. Its elements are k, s n = + r + r r n = n r k. For the geometric series we have an explicit formula for the finite sums, s n = r n+ r which allows us to easily study convergence of {s n }. Later we will be able to establish convergence and divergence of series for which we do not have explicit formulas, by comparing them to the geometric series. When discussing sequences we were not interested in the behaviour of the first 0, 00 or 000 elements, but rather in the infinite tail of the sequence; in its behaviour as n becomes The harmonic series derives its name from its connection to music. In music a harmonic series is a sequence of sounds whose frequencies are integral multiples of a base frequency. If the base frequency is taken to be, then the frequencies in the series are, 2, 3,.... The wavelengths, which are the reciprocals of the frequencies, are, 2, 3,.... These are the terms in the mathematical harmonic series. A video explaining the musical harmonic series is k=0

29 2.. DEFINITION 29 arbitrary large. The same is true for series. Finite sums interest us less than what happens when we continue adding numbers indefinitely. We will denote this infinite series by x n = x + x 2 + x , where the dots indicate that we keep adding more and more terms. For now this is just a symbol as we do not know whether this infinite sum makes sense. However, if the sequence s n = x + + x n of partial sums converges to a limit s, it would make sense to say that the series itself converges and to define x n := lim n n k= We will make this precise in the following definition. Definition Let {x n } be a sequence and let s n = x + x x n = x k = lim n s n = s. n x k. We call {s n } the infinite series associated to {x n } and denote it by x n. The quantity x n is called the nth term of the series and s n the nth partial sum. The infinite series x n is said to converge, if the sequence {s n } of partial sum converges to a finite limit. The value of the finite limit is called the sum of the series and we write x n = lim n s n. k= An infinite series that does not converge is said to diverge. We are using x n to denote two different things. First, it denotes the infinite series itself, that is the sequence {s n } of partial sums and formally we write x n = x + x Second, for a convergent series it also denotes the sum x n = lim n s n, in which case x n is a well-defined real number. It should be clear from the context which interpretation to use. To shorten notation we will sometimes write n x n or just x n for the infinite series x n.

30 30 CHAPTER 2. SERIES Remark Series do not always start with index. It may be convenient to consider the infinite series x n = x 3 + x , in which case the partial sums are n=3 s 3 = x 3, s 4 = x 3 + x 4, s 5 = x 3 + x 4 + x 5,.... Similarly, any other finite starting index is possible. How does this affect the infinite series? Starting from the following identity for finite sums, n n x k = x + x 2 + x k, k= we see that the series x n on the left converges if and only if the series n=3 x n on the right converges. However, their sums are not equal. Instead they are related via k=3 x n = x + x 2 + x n. This means that the notation x n is unambiguous, if we only want to know whether the series converges, but we have to be careful about the starting index, when we want to talk about the sum. Let us look at some examples of series. Example Consider the geometric series converge? When r, we have the formula n=3 r n. For which values of r does the series n r k = + r + r r n = r n+, r k=0 and hence the nth partial sum is s n = r n+ r The sequence {s n } converges if and only if r <, in which case lim n s n = r. What about r =? If r =, the partial sums are s n = n k = n +, and the sequence {s n } clearly diverges. This means that r n r r < = divergent r. k=0.

31 2.. DEFINITION 3 It is important to recognise geometric series when we encounter them in the wild. For example, the following are examples of geometric series: 2 5 n, n=2 3 2n, ( ) n 2 2n. Example Consider the series. Does the series converge? Note that because n(n + ) n(n + ) = n n +, the partial sums s n = n k= k(k+) form a telescoping sum, s n = (n )n + n(n + ) = n n = n +. + n n + The sequence of partial sums converges to and therefore the series is convergent, n(n + ) = lim n n + =. Remark We have seen that to every sequence {x n } we can associate a series {s n }. Since the series {s n } is itself a sequence we should ask ourselves if series are a special kind of sequences? In other words, which sequences are series and which are not? It turns out that every sequence is a series. To see this, start with a sequence {x n }. For it to be a series we need to find a second sequence {w n }, such that each x n is a sum x n = w + w w n. How do we find such a sequence? Start with the first three elements of {x n } and their relation to {w n }: So, if we define x = w, x 2 = w + w 2, x 3 = w + w 2 + w 3. w := x, w 2 := x 2 w w 3 := x 3 (w + w 2 ) = x 2 x, = x 3 x 2,

32 32 CHAPTER 2. SERIES then the required relationship between x, x 2, x 3 and w, w 2, w 3 is satisfied. More generally, we define w := x, w n := x n x n, n 2. Looking at w + + w n we find a telescopic sum which simplifies to w + + w n = x + (x 2 x ) + (x 3 x 2 ) + + (x n x n ) = x n. Therefore the series associated to {w n } is {x n }. Because we started with an arbitrary sequence {x n } this shows that every sequence is a series. If, as we have seen, every sequence is a series, is the distinction between sequences and series meaningful? The answer, for practical reasons, is, yes. Consider the two series 2 n and n. The first is a geometric series and we have an explicit formula for the partial sums, n = 2 k=0 2 k 2. So, when dealing with a geometric series, we can forget that the n partial sums were obtained by adding up elements of some sequence and work directly with the formula 2 2. This allows us to apply the methods of the previous chapter to study n convergence, boundedness, etc. Not much new is happening here. The situation is different for the second series, the harmonic series. Here we do not have an explicit formula for the partial sums. The best we can do is write them as the sum s n = n k= k = n. We still want to know whether the sequence {s n } is convergent or bounded. But now our only way to access it {s n } is via the sequence n. Our aim in studying series is to gain insight into the behaviour of the sum x n by studying properties of the sequence {x n }. Before we proceed with tests for convergence and divergence of series, a result that tells us that term-by-term operations with convergent series are allowed. Lemma Let x n and y n be two convergent series and c a constant. () The series x n + y n is convergent and x n + y n = x n + y n ;

33 2.2. NECESSARY CONDITION FOR CONVERGENCE 33 (2) The series c x n is convergent and cx n = c x n. Proof. We will show the first statement; the proof for the second one is similar. Let s n = x + + x n t n = y + + y n. Then we have the following identity for finite sums, and by passing to the limit we obtain x n + y n = lim n (x + y ) + (x 2 + y 2 ) + + (x n + y n ) = s n + t n, n k= x k + y k = lim n s n + t n = lim n s n + lim n t n = x n + y n. Note that in the proof all manipulations were done with finite sums before passing to the limit to obtain a statement about infinite sums. This is important, because at the moment we have not yet established rules that would permit us to manipulate infinite sums. 2.2 Necessary condition for convergence We start with a necessary criterion for convergence. Its main use will be to show that certain series are divergent. Proposition If x n converges, then lim n x n = 0. Proof. Let s n = n k= x k be the sequence of partial sums. According to Definition 2..3 the infinite series converges if and only if the sequence {s n } converges. Let s := lim n s n be the limit. We write the nth term of the series as the difference x n = s n s n. The two sequences on the right hand side converge and so Therefore lim n x n = 0. lim x n = lim s n lim s n = s s = 0. n n n We can restate Proposition 2.2. as follows.

34 34 CHAPTER 2. SERIES Corollary If {x n } does not converge to 0, then the series x n is divergent. We should be very careful about what this proposition does and does not say. It states that the terms of a convergent series converge to 0. However, convergence lim n x n = 0 by itself does not tell us anything about the convergence of the series. The harmonic series is the go-to example for this phenomenon. Example The harmonic series n is divergent, even though lim n this, look at the partial sum s 2 n and bracket the terms as follows, s 2 n = n n n = 0. To see The first bracket has 2 terms, the second 4 and the last one 2 n. Together we have n brackets. We can bound s 2 n from below by s 2 n n 2 n = = + n 2. This shows that the sequence {s 2 n}, which is a subsequence of {s n }, is unbounded and therefore cannot be convergent. In particular the series n diverges. 2n 2 Example Does the series converge? We look at the limit 3n 2 + n 2n 2 lim n 3n 2 + n = lim 2 n 2 n 3 + n = 2 3. Because the limit is not 0, Corollary shows that the series is divergent. A next lemma is in some way a restatement of the fact that the infinite series is the limit of partial sums. We will use it later, when dealing with power series. Lemma If x n converges, then lim N n=n = 0. Proof. Let N be fixed. In Remark 2..4 we have seen that the series x n=n n also converges and x n = x + + x N + x n. The divergence of the harmonic series was first proven by Nicole Oresme (c ) using the same method as presented here [4, p. 92]. However, many other proofs exist. A nice collection of over 40 proofs can be found in [6, 8]. The notes [7] contain a collection of interesting facts about the harmonic series. n=n

35 2.3. COMPARISON TEST 35 Let s N = N x n be the nth partial sum. By taking the limit in the identity x n = x n s N, we obtain lim N n=n n=n x n = x n lim N s N = Comparison test One way to establish the convergence or divergence of a series is to compare it to a second series, whose behavior is known. Theorem 2.3. (Comparison test). Let x n and y n be two infinite series with nonnegative coefficients, i.e., x n 0 and y n 0 for all n. Assume there exists a constant K > 0, such that for all n, x n K y n. () If y n converges, then x n converges. (2) If y n diverges, then x n diverges. Proof. Let us assume that the series y n converges. Denote by s n = n x k= k the partial sums of the series x n. Because the terms x n are nonnegative, the sequence {s n } of partial sums is monotone increasing: indeed, we have s n+ = s n + x n+ s n. Next we show that the sequence {s n } is bounded. Let t n = n y k= k be the nth partial sum of the series y n. Then the inequality x n K y n implies s n = x + + x n K y + + K y n = K t n. We assumed that the sequence y n converges, which means that the sequence {t n } converges. In particular {t n } is bounded and hence {s n } is bounded as well. Every monotone and bounded sequence is convergent, therefore the series x n converges. Example Does the series n converge? We have seen in Example 2..6 that the 2 series n(n+) converges and we have the inequality (n + ) 2 n(n + ) (n + ) 2 n(n + ). Therefore the convergence of n(n+) implies the convergence of n 2 = + n=2 n 2 = + (n + ) 2.

36 36 CHAPTER 2. SERIES Remark The assumption that the terms of the series are nonnegative is important. Consider for example the two series ( n) and Clearly n 2 and yet the convergence fo the second series has no influence on the n behaviour of the first one. 2 n. Sometimes it is more practical to apply the comparison test in the following form. Theorem (Limit form of the comparison test). Let x n and y n be two infinite series with nonnegative terms, i.e., x n 0 and y n 0 for all n. Assume the limit exists, either as a real number or L = +. x n L := lim n y n () If 0 < L <, then x n converges if and only if y n converges. (2) If L = 0 and y n converges, then x n converges. (3) If L = + and y n diverges, then x n diverges. Proof. First, let us assume that the limit L is finite, i.e., L +. Because the sequence xn converges, it is bounded, i.e., there exists K > 0, such that for all n, y n x n y n K x n K y n. Now we can apply Theorem 2.3.: if the series y n converges, then x n must converge as well. This proves the first half of the theorem. Now assume L > 0 or L = +. We will show that if x n converges, tehn y n converges. The sequence y n xn x n = y n is convergent: if L is finite and L > 0, then y lim n n x n = L and if L = +, then lim n = 0. In any case we can argue as before with the roles of x n and y n interchanged, thus showing that the convergence of x n implies the convergence of y n or equivalently, the divergence of y n implies the divergence of x n. This proves the other half of the theorem. The comparison test allows us to settle the convergence of the series n for α > 2 α and α <. Example () Consider limit is n α y n x n with α > 2. We compare it to the series n. The 2 lim n n α n 2 because α 2 > 0. Because the series n α converges as well. = lim n n = 0, α 2 n 2 converges, it follows that the series

37 2.4. D ALEMBERT TEST 37 (2) Now consider α <. We compare with the harmonic series n. The limit is lim n n α n Because n diverges, it follows that = lim n α = +. It remains to determine happens with the series this we need finer convergence tests. n α diverges as well. n α when < α < 2. To answer 2.4 D Alembert test One particular application of the comparison test deserves special mention: it is the comparison with the geometric series and is called the d Alembert or ratio test. Theorem 2.4. (D Alembert or ratio test). Let x n be a series of positive numbers such that x n+ L = lim n x n exists, either as a real number or L = +. () If L <, the series x n converges. (2) If L >, the series x n diverges. xn+ Proof. We consider first the case L <. Define r := ( + L)/2. Because the sequence x n converges to L and L <, there exists N such that for all n N, We keep N fixed. Then x n+ x n r x n+ r x n. x N+ r x N x N+ r 2 x N x N+n r n x N. Now we apply the comparison test to the two series x N+n and r n. Because r < the latter series converges and Theorem 2.3. implies that x N+n converges as well. xn+ Now assume L > or L = +. The sequence x n either converges to L with L > or it diverges to +. In either case there exists N, sucht that for n N, x n+ x n x n+ x n. This means that the sequence {x n } is monotone increasing. However, a monotone n=n increasing sequence of positive numbers note that x n > 0 with strict inequality cannot converge to 0. Therefore Corollary shows that the series x n is divergent.

38 38 CHAPTER 2. SERIES The usefulness of the ratio test resides in the fact that one has to consider only two consecutive terms of the series. There is no need to look at a long sum or to somehow find a second series for comparison. The ratio test allows us to determine the convergence of a series by calculating the limit of a sequence; in general limits of sequences are easier to find than sums of series. Remark The ratio test is conspicuously quiet about the case L =. There is a good reason for this: when L =, anything can happen. Consider the two series n and n 2. In both cases we have lim n n+ n = lim n n n + = and lim n (n+) 2 n 2 n 2 = lim n (n + ) =. 2 And yet, the first series diverges while the second one converges. This means that in the case L = other tools are necessary to determine whether a given series converges or diverges. (n!) 2 Example Consider the series. The terms of the series are clearly positive and (2n)! the ratio of consecutive terms is Therefore x n+ [(n + )!] = x n (2n + 2)! (2n)! (n!) = (n + ) 2 (n!) 2 2 (2n + 2)(2n + )(2n)! (2n)! (n!) = (n + ) 2 2 (2n + 2)(2n + ) and the series converges. x n+ lim = lim n x n n n n n n = 4. Example Consider the series n2 e n. The terms of the series are positive and Therefore and the series converges. x n+ x n = (n + )2 e n = + 2 n 2 e n n e. x n+ lim = lim + 2 n x n n n e = e <, Remark While often useful, the ratio test is not all-powerful: we have already seen that it failed to detect the divergence of the harmonic series. Because it is based on the comparison with the geometric series, it is useful for series, whose terms converge to 0 at least as fast as the sequence {r n } for some r <. Never the less, its ease of use means that in practice it is one of the first tests one should try on a series.

39 2.5. INTEGRAL TEST Integral test Trying to use the comparison test often leads to a chicken-and-egg problem: we want to determine the convergence fo a series n x n by comparing it to a series n y n; however we first have to establish convergence of n y n. For the series n n we were able to break 2 the circle by comparing it to the series n n(n+), whose partial sums were telescopic sums leading to an easily computable limit. We cannot expect to always be so lucky. For sequences a powerful tool to determine convergence and calculate limits is L Hôpital s rule. If, given a sequence {x n }, we can find a function f, such that x n = f (n), then we can calculate the limit of {x n } using the limit of f (x), i.e., lim x n = lim f (x). n x The contribution of L Hôpital s rule is that it allowed us to harness the power of calculus to compute limits of sequences. Can something similar be done for series? Theorem 2.5. (Integral test). Let f : [, ) be a continuous, positive and monotone decreasing function. Consider the series f (n). () If the integral (2) If the integral f (x) dx converges, the series f (n) converges. f (x) dx diverges, the series f (n) diverges. Proof. The idea is to approximate the integral f (x) dx by integrals over step functions. Because f is monotone decreasing we have the inequality f (k + ) f (x) f (k), for x [k, k + ]. The integral preserves inequalities and therefore f (k + ) Next we take the sum over k to obtain n k= f (k + ) k+ k n f (x) dx f (k). f (x) dx n k= f (k). Now we argue as in the proof of the comparison test: If the integral f (x) dx converges, then the sequence with elements n f (x) dx is bounded and therefore the sequence n k= f (k + ) of partial sums is bounded. The function f is positive, therefore the sequence of partial sums is monotone increasing, which implies that f (n + ) is convergent. On the other hand, if the integral f (x) dx is divergent, it has to diverge to +, because f is positive. Therefore the sequence of integrals n f (x) dx is unbounded and hence so is the sequence n f (k) of partial sums. This shows that f (n) is divergent. k=

40 40 CHAPTER 2. SERIES Example Consider the series function f (x) = x α. The integral is n Now, because α >, we have meaning with < α < 2. Apply the integral test with the nα x dx = α α + x α+ lim n α n x= =. α n α = n α α, x α dx = α, and therefore n converges for < α < 2. α By looking more closely at the proof of the integral test we can gain some insight into how fast a series is either converging or diverging. Theorem Let f : [, ) be a continuous, positive and monotone decreasing function. Consider the sequence c n := Then {c n } is a convergent sequence. n f (k) k= n f (x) dx. Proof. We will show that the sequence {c n } is monotone decreasing and bounded below. Both properties follow from the inequalities Monotone decreasing means f (k + ) k+ k f (x) dx f (k). c n c n+ 0 n+ n f (x) dx f (n + ) 0, and a lower bound is given by 0, because c n 0 Therefore {c n } is convergent. n f (k) n k= f (x) dx.

41 2.6. ALTERNATING SERIES TEST 4 Example Let us return to the harmonic series n dx = ln x x n x= Therefore Theorem shows that the difference n c n := k ln n k= n. With f (x) = x = ln n. we have converges to a finite limit γ := lim n c n. This constant γ is called the Euler Mascheroni constant and it apears several times in mathematics. Its numerical value is γ = Remarkably little is known about its properties; it is, for example, not known if γ is a rational number. 2.6 Alternating series test Let us look back at Proposition 2.2., which states that if a series x n converges, then lim n x n = 0. The harmonic series in Example showed that, while necessary, the condition lim n x n = 0 is in general not sufficient for convergence: lim n n = 0, but the series n is divergent. There exist, however, circumstances when the condition is sufficient. Consider a monotone decreasing sequence {x n } and sum it up while changing the sign at every step, ( ) n x n = x + x 2 x 3 + x 4 x 5 + x The resulting series ( )n x n is convergent if and only if lim n x n = 0. This is known as the alternating series test or Leibnitz test for convergence. To prove it we will use the following lemma, which states that combining two convergent sequences with the same limit again yields a convergent sequence. Lemma Let {x n } be a sequence, such that the two subsequences {x 2n } and {x 2n+ } converge to the same limit x. Then {x n } is convergent and lim n x n = x. Proof. We will use the defition of convergence. Let ɛ > 0 be given. Then there exists N, such that for all n N, x 2n+ x < ɛ and x 2n x < ɛ. This means that if n 2N, then Therefore {x n } converges to x. x n x < ɛ.

42 42 CHAPTER 2. SERIES Theorem (Alternating series test). Let {x n } be a monotone decreasing sequence and lim n x n = 0. Then the series ( ) n x n converges. Proof. Let {s n } be the sequence of partial sums, i.e., s n = x + x 2 x 3 + x ( ) n x n, and consider the subsequence {s 2n }. Because {x n } is monotone decreasing, we have x 2n+2 x 2n+ 0. For partial sums this implies s 2n+2 = s 2n x 2n+ + x 2n+2 s 2n. Thus the sequence {s 2n } is monotone decreasing. We also have the estimate s 2n = x + x 2 x 3 + x 4 + x 2n+ + x 2n = (x 2 x ) + (x 4 x 3 ) + + (x 2n x 2n+ ) 0, which shows that {s 2n } is bounded below. Therefore the subsequence {s 2n } is convergent. Let s := lim n s 2n. Next we show that the whole sequence {s n } converges to s. To achieve this consider the other subsequence {s 2n+ }. Using the identity s 2n+ = s 2n x 2n+ we obtain that lim s 2n+ = lim s 2n lim x 2n+ = s 0 = s. n n n Now we can conclude using Lemma 2.6. that {s n } converges as well. Example The series ( ) n and lim n n = 0, therefore n is convergent: the sequence n is monotone decreasing ( ) n n is convergent by the alternating series test. Example The assumption that the sequence {x n } is monotone decreasing is important. Otherwise we could consider the sequence {x n } = 0,, 0, 2, 0, 3, 0, 4,.... It is clear that lim n x n = 0, but the corresponding alternating series, ( ) n x n = , diverges just like the harmonic series diverges. Remark The situation here is similar to that of monontone convergent sequences. We saw in Proposition.2.3 that every convergent sequence is bounded, while not every bounded sequence is convergent. However, if we additionally assume that the sequence is monotone, then Theorem.7.3 shows that a monotone bounded sequence in convergent.

43 2.7. ABSOLUTE CONVERGENCE Absolute convergence The realm of the infinite can, if we are not careful, play tricks with us. Example Consider the series ( ) n+ n = The alternating series test shows that the series converges. Let S be the sum. One can show that S = ln 2, but all we need here is that S 0. Now consider S + 2S, written as follows. S = S = S = On the left hand side we have 3 2S as expected. Now let us look at the right hand side. We find the same terms as in the original series, arranged in a different order: instead of alternating one positive and one negative term, the new series adds two positive terms followed by one negative term. Let us write the two sums underneath each other. S = S = Changing the order in which we sum up the terms changes the value of the sum. In order to investigate when and why this happens we will introduce the notions of absolute convergence and conditional convergence. But first we return to Cauchy sequences and rewrite the definition of a Cauchy sequence for series. Proposition (Cauchy s criterion). A series x n is convergent if and only if for every ɛ > 0 there exists N such that for all n N and all k, x n + x n+ + + x n+k < ɛ. Proof. This proposition is a restatement of the fact that a series converges if and only if the sequence of partial sums is a Cauchy sequence. We will in the details. Denote by s n := x + + x n = n k= x k the nth partial sum. Then x n + + x n+k = s n+k s n,

44 44 CHAPTER 2. SERIES and the proposition reads as follows: the series x n is convergent if and only if for every ɛ > 0 there exists N such that for all n N and all k, This is true because of Theorem..6. s n+k s n < ɛ. Most of the convergence tests we encountered in the previous sections the comparison test, d Alembert s test, the integral test can only be applied to series with nonnegative terms. Given a series x n with both positive and negative terms, we can consider the related series x n, obtained from x n by forgetting about the sign of x n. We will see that x n can tell us a lot about the behaviour of our original series x n. Definition A series x n is called abosolutely convergent if and only if the series x n is convergent. The name absolute convergence seems to imply something more than just ordinary convergence. This is, however, not obvious from the definition that an absolutely convergent series is convergent. Note that according to the definition a series x n is absolutely convergent if the related series x n converges. Nothing is said about x n itself. This is resolved in the following lemma. Lemma Every absolutely convergent series is convergent. Proof. We will show that every absolutely convergent series satisfies Cauchy s criterion. Assume x n is an absolutely convergent series. Let ɛ > 0 be given. Because the series x n is convergent, Proposition tells us that there exists N such that for all n N and all k, x n + x n+ + + x n+k < ɛ. Now consider the series x n. Using the triangle inequality we obtain x n + x n+ + + x n+k x n + x n+ + + x n+k < ɛ. Therefore Proposition shows that the series x n is convergent. A very useful inequality is the following. Lemma If x n is absolutely convergent, then x n x n. Proof. For fixed n we can use triangle inequality to obtain n n x k x k. k= k=

45 2.7. ABSOLUTE CONVERGENCE 45 Because x k 0, the sequence n x k= k of partial sums is monotone increasing and therefore via Theorem.7.3, n x k x n. k= Let s n := n k= x k denote the nth partial sum. Then we have shown that for all n, s n x k. Finally we apply Proposition.3.7 to conclude that x n = lim s n x n. n k= Note that if all terms in a series x n are nonnegative, i.e., x n 0, then x n = x n. Therefore, if x n is convergent, it is also absolutely convergent. This means that we can reuse the convergence tests from previous sections by applying them to the series x n. Let us in particular restate d Alamberts test from Theorem Theorem (D Alembert or ratio test). Let x n be a series such that L = lim x n+ exists, either as a real number or L = +. n () If L <, the series x n converges absolutely. (2) If L >, the series x n diverges. Proof. The case L < is Theorem 2.4. applied to the series x n. For the other case the proof of d Alembert s test shows that L > implies that the sequence { x n } does not converge to 0. But if { x n } does not converge to 0, then {x n } also cannot converge to 0. Therefore x n is divergent. Example Consider the series n 2 x n sin n. The terms of this series are x n = sin n n 2 and they take both positive and negative values. To determine convergence we look at the series x n = sin n n. The latter series is convergent: using the inequality 2 sin n n 2 n 2 we can compare it with the series n. Therefore sin n 2 n is absolutely convergent. 2

46 46 CHAPTER 2. SERIES Lemma showed that every absolutely convergent series is convergent. What about the reverse implication? Is every convergent series absolutely convergent? ( ) n Example The series is convergent, but not absolutely convergent. The n convergence was shown using the alternating series test in Example and the divergence of the harmonic series was shown in Example This motivates the following definition. Definition A series that is convergent, but not absolutely convergent, is called conditionally convergent. The next two theorems show why this distinction between conditionally and absolutely convergent series is important. Informally stated, a conditionally convergent series converges, but barely so. A slight change to the series can change the sum or even make the series divergent. An absolutely convergent series on the other hand is difficult to perturb. The change that we have in mind is the rearrangement or reordering of a series. Definition A permutation σ is a bijective function σ :. If σ is a permutation, then the sequence {x σ(n) } is a rearrangement of the sequence {x n}. Example Consider the sequence A rearrangement is given by the sequence, 2, 3, 4, 5, 6, 7, 8, 9, 0,, 2, 3,...., 3, 2, 5, 7, 4, 9,, 6, 3, 5, 8, 7,.... The sequence is obtained by alternating two odd numbers with one even number. Note the difference between a subsequence and a rearrangement. A subsequence uses only some elements of the original sequence, but has to preserve the order. A rearrangement can change the order, but has to use all elements of the sequence. The first theorem states that we can rearrange an absolutely convergent series without changing the sum. In contrast to that, the second theorem shows that we can rearrange a conditionally convergent series give us any sum we want. This also explains the phenomenon we observed in Example Theorem Let x n be an absolutely convergent series and σ a permutation. Then the series x σ(n) is also absolutely convergent and x σ(n) = x n. Theorem (Riemann series theorem). Let x n be a conditionally convergent series and L a real number. Then there exists a permutation σ such that x σ(n) = L.

47 2.7. ABSOLUTE CONVERGENCE 47 Let us close with the following quote by Bernhard Riemann The recognition of the fact that infinite series fall into two classes (according to whether the limit is independent of the ordering of the terms or not) constitutes a turning-point in the conceptualization of the infinite in mathematics. The two classes being the absolutely convergent and the conditionally convergent series.

48 48 CHAPTER 2. SERIES

49 Chapter 3 Power Series 3. Definition and convergence An infinite series u n, being the sum of infininitly many numbers u n, is itself a number. What happens, if we let these numbers depend on a parameter x, as in a n = u n (x)? Then the sum of the series also depends on the parameter. To make this explicit we write the sum as f (x) = u n (x). Now f is a function of x and we can investigate what properties f possesses: if f continuous? Differentiable? We will not allow arbitrary parameter dependance. A rich theory that is both relevant for applications and not too difficult to develop can be obtained by considering monomials, i.e., u n (x) = a n x n, where {a n } is some sequence of numbers. The resulting series f (r) = a n x n is called a power series and x n is the nth coefficient. Given a function f that is n-times differentiable we defined its Taylor polynomial of degree n around 0 to be T n f (x) = f (0) + f (0)x + 2 f (0)x n! f (n) (0)x n = n k=0 k! f (k) (0)x k. If the function has infinitely many derivatives we can can calculate Taylor polynomials of higher and higher degrees and by formally continuing this process we arrive at the Taylor series T f (x) = n! f (n) (0)x n. 49

50 50 CHAPTER 3. POWER SERIES The Taylor series is an example of a power series with coefficients a n = n! f (n) (0). Several questions can be formulated that we will strive to answer in this chapter. () When does the Taylor series T f (x) converge? (2) If the Taylor series converges, does it converge to the original function, i.e., is T f (x) = f (x)? (3) What is the Taylor series of the function T f (x)? (4) Is every power series the Taylor series of some function? Before we can answer any of these questions, let us start with the formal definition. Definition 3... Let {a n } be a sequence. The series is called a power series with coefficients a n. a n x n When talking about power series we will always start the sum at n = 0. In case we start at some other index, it is implicitely assumed that the first coefficients are 0. Note that we call a n the coefficients of the power series, but for fixed x, the terms of the infinite series are the numbers a n x n. Let us determine, where a power series converges. Proposition Let a nx n be a power series. Then there exists R, either a finite number or +, such that () a nx n converges absolutely for x < R (2) a nx n diverges for x > R Proof. We begin the proof by showing the following statement: if the series a nr n converges for some r 0, then it converges absolutely for all x < r. Write a n x n = a n r n x n r. n Because the series a nr n converges, the sequece {a n r n } converges to 0, in particular {a n r n } is bounded. Let b be such that a n r n b for all n 0. Then a n x n = a n r n x n b x n, r r and because x/r <, the series on the right is a convergent geometric series. Therefore a nx n is absolutely convergent by the comparison test.

51 3.. DEFINITION AND CONVERGENCE 5 Next we define R := sup r 0 : a n r n converges. By the above argument it is clear that a nx n diverges for all x > R. If x < R, then there exists r such that x < r < R and a nr n converges. Then it follows that a nx n converges absolutely. This proposition shows that all power series behave fundamentally the same. Excluding the degenarate cases R = 0 and R = +, a power series converges absolutely on the open interval ( R, R). It diverges for x > R, that is on the set \ [ R, R] and it can exhibit any behavior at the two points x = R and x = R. If R = +, the series a nx n converges absolutely for all x R and if R = 0, then the series converges only for x = 0. Definition Let a nx n be a power series. The number R defined in Proposition 3..2 is called the radius of convergence. The question remains, how we can compute the radius of convergence. In many cases the following result is very useful Proposition Let a nx n be a power series. Provided the limit exists either as a finite number or as +, we can calculate the radius of convergence via R = lim a n. n a Proof. We assume the limit exists and define R = lim n. n a n+ We apply d Alembert s test to the series a nx n and consider the quotient lim a n+ x n+ n a n x n = lim r a n+ n = x R. Thus we see that the power series converges absolutely for x < R and diverges for x > R. There are the two degenerate cases to consider. If R = 0, then the sequence a n+ a n = diverges to + and d Alembert s test shows that the power series diverges for all a n+ a n a n+ a x 0. If R = +, then lim n+ n a n x. Example () n 3 x n. Its radius of convergence is (2) x n. Its radius of convergence is n! a n = 0 and the power series converges absolutely for all n 3 R = lim n (n + ) = lim + 3 =. 3 n n (n + )! R = lim = lim n + = +. n n! n

52 52 CHAPTER 3. POWER SERIES 3.2 Continuity and differentiability Proposition 3..2 shows that a power series a nx n with a positive radius of convergence defines a function f (x) = a n x n. on the open interval ( R, R) if R is finite and on if R = +. Our aim in this section is to investigate properties of function defined by power series. We will assume that all power series have a positive radius of convergence. Nothing much of interest can be said of power series that diverge everywhere except at x = 0. First we restate the definition of continuity for a function. Definition Let f : (a, b) be a function and x 0 (a, b). If for every ɛ > 0 there exists δ > 0 such that for all x x x 0 < δ implies f (x) f (x 0 ) < ɛ, we say that f is continuous at x 0. If f is continuous at x 0 for all x 0 in (a, b), we say that f is continuous on (a, b). Proposition Let f (x) = a nx n be a function defined by a power series whose radius of convergence is R. Then f is continuous on the interval ( R, R). Proof. Let f (x) = a nx n and R be the radius of convergence of the power series. Take x 0 ( R, R). We will show that f is continuous at x 0. We define r := (R + x 0 )/2, so that x 0 < r < R. The series a nr n is absolutely convergent and therefore via Lemma there exists N such that a n r n < ɛ 3. It follows via Lemma that for all x satisfying x r also a n x n a n x n a n r n ɛ 3. n=n n=n The idea of the proof is to split the infinite series a nx n into The first part is, which we denote by n=n n=n a n x n = a 0 + a x + + a N x N + a n x n. n=n p(x) = a 0 + a x + + a N x N, is a polynomial and therefore continuous. Thus there exists δ > 0 such that for all x with x x 0 < δ we have p(x) p(x 0 ) < ɛ 3.

53 3.2. CONTINUITY AND DIFFERENTIABILITY 53 Define δ := min(δ, r x 0 ) and let x x 0 < δ. Putting everything together we estimate f (x) f (x 0 ) = a n x n a n x n 0 a n x n + a n x n a n x n 0 + a n x n n=n ɛ 2 + p(x) p(x 0) + ɛ 3 < ɛ. Therefore f is continuous at x 0 and because x 0 was arbitrary it is continuous on ( R, R). This proposition allows us to answer the following question: is it possible for different two power series to represent the same function? Later we will derive an explicit formula for the coefficients in terms of the function, but now we can use Proposition to show that the answer to the above question is no. Proposition Let f (x) = a nx n and g(x) = b nx n be two functions defined by power series. If f (x) = g(x) for all x in some open interval around 0, then a n = b n for all n 0. Proof. By considering the difference h(x) = f (x) g(x), which is also a real analytic function, h(x) = (a n b n )x n, it is enough to prove the following statement: if f (x) = a nx n is a real analytic function defined by a power series with a positive radius of convergence and f (x) = 0 for all x in some open interval around 0, then a n = 0 for all n 0. Let such an f be given and assume that f (x) = 0 for x < R. Because f (0) = a 0, the assumption f (0) = 0 implies a 0 = 0. We proceed by induction. Assume it has been shown that a 0 = = a n = 0. Then f (x) = n=n n=n a k x k = x n a k x k n = x n a k+n x k. k=n k=n Define the real analytic function h(x) := a k=0 k+nx k. For x < R and x = 0 we have h(x) = x n f (x) which implies that the power series a k=0 k x k and a k=0 k+nx k have the same radii of convergence. Because f (x) = 0, also h(x) = 0 for all x 0. Proposition shows that the function h is continuous at 0 and therefore h(0) = lim x 0 h(x) = 0. In particular this implies a n = 0 and by induction we conclude that a n = 0 for all n 0. After showing that functions defined by power series are continuous, the next step is to show that they are differentiable. Lemma The three power series a nx n, na nx n and a n n+ x n+ have the same radius of convergence (either as a finite number or + ). With this lemma, whose proof we skip, we can show that power series can be differentiated term-by-term. k=0 n=n

54 54 CHAPTER 3. POWER SERIES Proposition Let f (x) = a nx n be a function defined by a power series whose radius of convergence is R. Then f is differentiable on ( R, R) and the derivative is given by the power series f (x) = na n x n, with the same radius of convergence. Proof. Let x ( R, R). To show f is differentiable at x we need to look closely at the behaviour of the difference quotient f (y) f (x) y x as y approaches x. We start with the nth term in the power series. Using Taylor s theorem with Peano s remainder, Proposition 3.3., we have y n = x n + nx n (y x) + n(n )ξ n 2 (y x) 2, with ξ between y and x. This allows us to write a n y n = a n x n + (y x) na n x n + 2 (y x)2 n(n )a n ξ n 2, and therefore f (y) f (x) na n x n = y x 2 (y x) n(n )a n ξ n 2. We are interested in the limit y x. Choose r such that x < r < R. Therefore, when y x < r x, it follows that ξ < r and hence n(n )a n ξ n 2 n(n )a n r n 2 =: M. Here we used Lemma to infer that the series n(n )a nr n 2 converges. This allows us to estimate f (y) f (x) na n x n = y x 2 (y x) n(n )a n ξ n 2 y x M, 2 and therefore f f (y) f (x) (x) = lim = y x y x na n x n.

55 3.3. TAYLOR SERIES 55 Corollary Let f (x) = a nx n be a function defined by a power series. Then f is infinitely differentiable and for all n 0, a n = n! f (n) (0). Proof. By iterating Proposition we see that f is infinitely differentiable and f (n) (x) = n(n )... (n k + ) a n x n k. k=n When we evaluate f (n) at x = 0, only the coefficient in front of x 0 remains and we get f (n) (0) = n! a n. 3.3 Taylor series We have seen that an n-times differentiable function f : (a, b), defined on an interval (a, b) containing 0, can be approximated by the Taylor polynomial T n f (x) = f (0) + f (0) + f (0)x + 2 f (0)x n! f (n) (0)x n. If f has n + derivatives we can derive an expression for the remainder term R n f, defined as f (x) = T n f (x) + R n f (x), which measures the error we are making if we replace f by its Taylor polynomial. Several expressions for the remainder term exist and for our purposes we will use Peano s form of the remainder. Proposition Let f : (a, b) be n + -times differentiable and 0 (a, b). Then for every x (a, b) there exists ξ n between 0 and x, such that R n f (x) = f (n+) (ξ n ) (n + )! x n+. When f is infinitely differentiable we call the formal power series T f (x) = n! f (n) (0)x n. the Taylor series of f around 0. We start by considering a function f, defined using a power series, and calculate its Taylor series. A series is called formal, if we are not interested in questions of convergence. A formal series is a sequence of numbers, but it does not necesserily have a finite sum.

56 56 CHAPTER 3. POWER SERIES Proposition Let f (x) = a nx n be a function defined by a power series. Then the Taylor series of f is f itself, T f (x) = a n x n. Proof. This follows from the definition of the Taylor series, T f (x) = n! f (n) (0)x n. and the formula a n = n! f (n) (0), which was shown in Corollary This was the easy direction. Now we start with a function f and consider its Taylor series T f (x) = n! f (n) (0)x n. Under what conditions does the Taylor series have a positive radius of convergence? And if it does have a positive radius of convergence, does it converge to the function we started with? To answer these questions we need to introduce some notation. Definition Let f : (a, b) be a continuous function. We define f (a,b) := sup { f (x) : x (a, b)}, to be the supremum norm of f on (a, b), provided the set is bounded above, and we set f (a,b) = + otherwise. The supremum norm measures, how large values f takes on the interval (a, b). Example () Let f (x) = x 2 and a > 0. Then f ( a,a) = a 2, because f (x) = x 2 assumes its largest values at the endpoints a and a. (2) Let f (x) = sin x. Then f (, ) =, because sin x holds for all x and sin π 2 =. (3) Let f (x) = x. Then f (0,) = +, because the set {x : x (0, )} is not bounded above. Now we can state the result about convergence of Taylor series.

57 3.3. TAYLOR SERIES 57 Proposition Let f : ( R, R) be an infinitely differentiable function. Suppose that R n lim n n! f (n) ( R,R) = 0. Then the radius of convergence of the Taylor series T f (x) is at least R and for all x < R, T f (x) = f (x). Proof. The Taylor polynomials T n f (x) are the partial sums of the Taylor series T n f (x). If we can show that lim n T n f (x) exists for all x < R, then Proposition 3..2 shows that the radius of convergence of the power series T f (x) is at least R. Let x < R. Then which we rewrite as f (x) = T n f (x) + R n f (x), T n f (x) f (x) = R n f (x). This means that it is enough to show that lim n R n f (x) = 0. To estimate R n f (x) we combine the formula for the remainder term from Proposition 3.3., R n f (x) = f (n+) (ξ n ) (n + )! x n+ Rn+ (n + )! f (n+) ( R,R), with the assumption of the proposition. Therefore lim T n f (x) f (x) = lim R n f (x) = 0, n n which shows that T f (x) = lim n T n f (x) = f (x). We cannot omit in Proposition the assumption on the growth of the n-th derivative. The following example shows that there exist functions, whose Taylor series converges, but the limit is not the function we started with. Example Consider the function f (x) = e /x 2, x 0 0, x = 0. We claim that f is infinitely differentiable and that for all n 0, This implies that T f (x) = f (n) (0) = 0. n! f (n) (0)x n = 0. The Taylor series converges, but the limit is the 0-function. In particular T f (x) f (x) for all x 0.

58 58 CHAPTER 3. POWER SERIES To show that f is infinitely differentiable we first derive an expression for f (n) (x) that is valid for x 0. Starting from f (x) = 2 4 x 3 e /x2 f (x) = x 6 e /x2, 6 x 4 we see that the nth derivative is of the form f (n) (x) = p n x e /x2, with a polynomial p n. This can be proven by induction. Then we can take the limit lim f (n) (x) = lim p n e /x2 = 0, x 0 x 0 x which shows that f (n) (0) = 0. Thus f is infinitely differentiable everywhere and the statement about its Taylor series follows. The last question about the relation between power series and Taylor expansion is whether every power series is the Taylor series of some function. In other words, given a power series a nx n, does there exist an infinitely differentiable function f : ( R, R), defined on some interval ( R, R), such that T f (x) = a n x n. If the given power series has a positive radius of convergence, this question was answered by Proposition In this case the power series defines a function whose Taylor series is the power series itself. The interesting case is, when the power series a nx n diverges for all x 0. We can reformulate the question as follows. Given a sequence {b n }, does there exist a function f, satisfying f (n) (0) = b n, for all n 0? The answer to this question is given by Borel s lemma, which we present without proof. Theorem (Borel s lemma). Given a sequence {b n }, there exists an infinitely differentiable function f : such that for all n 0, f (n) (0) = b n. 3.4 Exponential and trigonometric functions Power series are an important tool to study well-known functions such as e x, sin x and cos x. Using only our knowledge of the derivatives, d d x ex = e x d d sin x = cos x d x cos x = sin x, d x

59 3.4. EXPONENTIAL AND TRIGONOMETRIC FUNCTIONS 59 together with the values e 0 = sin 0 = 0 cos 0 =, we will develop them into power series around 0. Proposition For all x, () e x x n = n! = + x + x x (2) sin x = (3) cos x = ( ) n x 2n+ (2n + )! = x x x ( ) n x 2n (2n)! = x x Proof. For f (x) = e x we have for all n, f (n) (x) = e x. Fix R > 0. Then f (n) ( R,R) = sup {e x : x < R} = e R. To apply Proposition we consider the limit This allows us to conclude that for x < R, R n lim n n! f (n) R n ( R,R) = lim n n! er = 0. e x = n! f (n) (0)x n = x n n!. Since R was arbitrary, this holds for all x. We proceed in the same way for sin x. Let f (x) = sin x. Then and so f (2n) (x) = ( ) n sin x f (2n+) (x) = ( ) n cos x. f (2n) (0) = 0 f (2n+) (0) =. Because f (n) (, ) =, we can apply Proposition to conclude that sin x = n! f (n) (0)x n = The proof for cos x proceeds along the same lines. ( ) n x 2n+ (2n + )!. We will use a different method to derive the power series expansion of ln( + x).

60 60 CHAPTER 3. POWER SERIES Proposition Let x <. Then ln( + x) = ( ) n+ x n n = x x x 3 3 x 4 Proof. We start with the geometric series, whose sum is known, + x = x + x 2 x 3 + = The logarithm is the antiderivative of /( + x), d d x ln( + x) = + x, ( ) n x n. and we have seen in Proposition that we can differentiate power series term-by-term. To reverse the process, we consider the power series f (x) = ( ) n+ x n n. Its radius of convergence, which can be computed via Proposition 3..2, is R =. Then Proposition tells us that Therefore and the constant C is determined by f (x) = ( ) n+ x n = ( ) n x n = + x. f (x) = ln( + x) + C, C = f (0) ln( + 0) = Abel s theroem Finding an explicit expression the function defined by a power series is a useful method to evaluate sums. For example, knowing that for x <, allows us to evaluate the sum ln( + x) = ( ) n+ x n n, = 4 n2 = ln = ln 2. n 2

61 3.5. ABEL S THEROEM 6 This method only works if we want to evaluate a power series a nx n at a point x with x < R, where R is the radius of convergence. Consider the series = ( ) n+ n. We have seen in Example that the series converges, but we don t know what the sum is. The series is the power series for ln( + x) evaluated at x =. However the radius of convergence is also R = and so Proposition does not allow us to claim that = ln 2. To justify this we need to closer at the behaviour of the power series as it approaches the radius of convergence. Theorem 3.5. (Abel s theorem). Let f (x) = a nx n be a function defined by a power series with finite radius of convergence R. () If the series a nr n converges, then (2) If the series ( )n a n R n converges, then a n R n = lim x R f (x). ( ) n a n R n = lim f (x). x R + Proof. We consider the case R = first. Let g(y) = b n x n be a power series with radius of convergence R =, and s := b n. By assumption the series b n converges and thus s is finite. We will show that lim y g(y) = s. Let y <. We start by proving the identity g(y) = ( y) s n y n.

62 62 CHAPTER 3. POWER SERIES Let s n = b b n be the nth partial sum. Then ( y) n s k y k = k=0 = n s k y k k=0 n s k y k+ = k=0 n+ k=0 n+ n s k y k n (s k+ b k+ ) y k+ n n+ s k y k s k y k + b k y k = b 0 y 0 + b k y k s n+ y n+ k=0 By taking the limit we obtain ( y) k= n+ = s n+ y n+ + b k y k. s n y n = lim n ( y) k=0 n k=0 k= because y < implies lim n s n+ y n+ = 0. Next we take the geometric series with y <, y = Multiplying both sides with s we obtain k=0 s k y k = lim n s n+ y n+ + k= y n = ( y) y n. s = ( y) s y n. Combining this identity with that for g(y) yields for y <, g(y) s = ( y) (s n s) y n. b n y n = g(y), To show that lim y g(y) = s, we need to estimate g(y) s. Let ɛ > 0 be given. Then there exists N such that for n N, s n s < ɛ 2. Since we are interested in the behavior of g(y) near y =, we can assume that 0 < y <. We split the sum N g(y) s ( y) s n s y n + ( y) s n s y n n=n N ( y) s n s + ɛ 2 ( y)y N N ( y) s n s + ɛ 2. y n

63 3.5. ABEL S THEROEM 63 If we choose y, such that then we obtain N ( y) s n s < ɛ 2, g(y) s < ɛ, showing that lim y g(y) = s. This concludes the proof for the case R =. Now let f (x) = a nx n be given with an arbitrary radius of convergence R. For the first statment, define b n = a n R n and g(y) = f (Ry) and for the second statement let b n = ( ) n a n R n and g(y) = f ( Ry). Remark The converse of Abel s theorem is not true. Existence of the limit lim x R f (x) does not imply convergence of the series a nr n. Consider the geometric series f (x) = + x = ( ) n x n. Then lim x + x = 2, but the series diverges at x =. This means that we need to establish convergence of the series by hand before applying Abel s theorem, which will help us finding the sum of the series. Example We have seen the power series for ln( + x), which is ln( + x) = ( ) n+ x n n. Its radius of convergence is R =. For x =, we obtain the series ( ) n+ n = We showed using the alternating series test in Example that the series converges and therefore Abel s theorem shows that = lim ln( + x) = ln 2. x

64 64 CHAPTER 3. POWER SERIES

65 Appendix A A bit of history Let us not let go the guiding hand of history. History has made all; history can alter all. Ernst Mach A. Zeno s paradoxes In a race, the quickest runner can never overtake the slowest, since the pursuer must reach the point whence the pursued started, so that the slower must always hold a lead. Aristotle, Physics VI:9, 239b5 2 Consider a footrace between Achilles, the Greek hero and great warrior, and a tortoise. To even the odds we give the tortoise a head start of 00 meters. Both competitors start running and after some time Achilles has covered the 00 meters and reached the point, where the tortoise was at the start of the race. But the tortoise, slow as it is, has not been idle, and has covered in this time 0 meters on its own. Achilles keeps running and after covering the 0 meters finds the tortoise still ahead of him, now by meter. And so they continue: whenever Achilles thinks of overtaking the tortoise, he first has to arrive at the point, where the tortoise was a moment ago. But the tortoise won t be there any more and will maintain a lead, no matter how small. Thus Achilles goal of overtaking his opponent is foiled, because the number of distances he has to cover is infinite and how can one complete an infinite amount of tasks in a finite amount of time? Animated versions of the race can be found online 3. This paradox an others like it, concerning the nature of motion and the divisibility of time and space are asrcibed to Zeno of Elea and known as Zeno s paradoxes. In Carl C. Gaither and Alma E. Cavazos-Gaither, Gaither s Dictionary of Scientific Quotations (p. 949). 2nd Edition. Springer, In Aristotle, Physics (Translated by R. P. Hardie and R. K. Gaye). See 6.vi.html# See The Open University, Achilles and the tortoise 60-second adventures in thought, watch?v=skm37pczmwe. 65

66 66 APPENDIX A. A BIT OF HISTORY Zeno of Elea (c.490 c.430 BC) was a pre-socratic Greek philosopher. Little is known about his life, the earliest source being Plato s dialogue Parmenides, written nearly a century after Zeno s death. It is said that he visited Athens around the age of 40, where he met the young Socrates. He was highly regarded by other Greek philosophers; the following verses are ascribed to the philosopher Timon of Phlius : Great is the strength, invincible the might Of Zeno, skilled to argue on both sides Of any question, th universal critic The paradox of Achilles and the tortoise presented profound difficulties for Greek mathematicians and philosophers, who did not have the concept of convergence and infinite series at their disposal. Now that we do have them, we can look at the problem mathematically. Say, Achilles runs 0 meters per second. Then the time it takes him to reach the tortoise can be expressed as the sum This is a convergent geometric series and its sum is =. = 9, meaning that Achilles should speed past the tortoise after 9 seconds. Is this a satisfactory answer? Certainly, this is the answer that is supported by obersvations and the weight of everyday experience. But have we truly explained how Achilles managed to traverse these infinitely many segments in a finite amount of time? The answer to this lies outside the scope of mathematics and leads one into the realm of philosophy. The interested reader can find more information in [5]. Sources: [3, p. 0f], [2]. A.2 Augustin-Louis Cauchy Augustin-Louis Cauchy ( ) was a French mathematician, known for his pioneering work in analysis. After graduating from the engineering school École des Ponts et Chaussés (School for Bridges and Roads) in 80 he moved to Cherbourg in the northwest of France to work as an engineer on port facilities for Napoleon s planned invasion of England. Working on mathematics in his spare time and encouraged by Legendre, he published his first mathematical paper on ployhedra in 8 at the age of 22. Losing interest in engineering he moved in 82 back to Paris hoping to find a position as a mathematician. Diogenes Laertius, The Lives and Opinions of Eminent Philosophers (Translated by C. D. Yonge). See http: //classicpersuasion.org/pw/diogenes/dlzeno-eleatic.htm

67 A.2. AUGUSTIN-LOUIS CAUCHY 67 After several unsuccessful applications he finally obtained a position as assistant professor at the École Polytechnique in 85. It was not a complete coincidence that this happened shortly after Napoleon s defeat. Cauchy s family were staunch royalists supporters of the executed Bourbon king Louis XVI and his brother Louis XVIII, who regained the throne in 84. In 82 Cauchy published a textbook, the Cours d analyse, which for the first time provided precise definitions for notions such as limits, convergence and continuity. This book remains one of the most influential mathematics books ever written. Cauchy was a brilliant mathematician, but not an easy colleague. Niels Abel wrote in 826: Cauchy is fou [mad] and there is Nothing to come out of him, although he is the mathematician for the present Time who knows how Mathematics ought to be treated. [0, p.409] Political circumstances changed again in 830 when the July revolution deposed the reigning king Charles X and replaced him by the non-bourbon king Louis-Philippe. Cauchy refused to swear the oath of allegiance to the new king as was required of all professors and was consequently relieved of his position. He moved abroad, living and teaching in Turin (83 33) and Prague (833 38) before returning to Paris in 838. There he was still unable to obtain an academic appointment because of his continued refusal to swear the oath. In 848 revolutions broke out all over Europe. In France king Louis-Philippe was forced to abdicate and France became a republic. Cauchy regained his university position, which he held until his death in 857 at the age of 67. Cauchy was a very prolific mathematician, publishing in his lifetime 789 mathematical papers, across all the then-known areas of mathematics. His collected works span 27 volumes and took almost a century to collect. His name is one of 72 inscribed in the Eiffel Tower. Sources: [2, 9, ]