Series: Elementary, then Taylor & MacLaurin

Lecture 3 Series: Elementry, then Tylor & McLurin This lecture on series strts in strnge plce, by revising rithmetic nd geometric series, nd then proceeding to consider other elementry series. The relevnce of these to course on clculus is perhps hrd to see, until one worries bout series convergence properties nd integrtion. Even then it remins bit obscure. Much much more useful will be Tylor s nd McLurin s series. Using the derivtives t point, these llow you to pproximte functions in the neighbourhood of tht point, s polynomil or power series in the smll displcement. Every engineer on the plnet uses these. 3.1 Arithmetic progression An rithmetic series or progression is sequence of n numbers successive members of which differ by the sme mount d. Tht is, { 1, ( 1 +d), ( 1 +2d),..., ( 1 +(n 1)d)}. The i-th term is i = 1 + (i 1)d. Suppose we wnted the sum to the n-th term, S n = n i=1 i. Write the sequence in the forwrd nd reverse directions, then dd: ( 1 + 1)d) ( 1 ( 1 S n = 1 + ( 1 + d) + ( 1 + 2d) +... + ( 1 + (n 1)d) = (n + + (n 2)d) + + (n 3)d) +... + 1 (3.1) The sum of corresponding terms is lwys 2 1 + (n 1)d nd there re n of them so the sum is S n = 1 2 n(2 1 + (n 1)d) = 1 2 n( 1 + n ). (3.2) Convergence. The sum of this series lwys diverges s more terms re dded. The only series tht remins finite is string of zeros! 1

3/2 LECTURE 3. SERIES: ELEMENTARY, THEN TAYLOR & MACLAURIN 3.2 Geometric progression A geometric series or progression is sequence of n numbers successive members of which re multiplied by common fctor r. Tht is, { 1, r 1, r 2 1,..., r n 1 1 )}. The i-th term is i = r i 1 1. The sum to n terms is So S n = 1 + r 1 + r 2 1 +... + r n 1 1 ) rs n = r 1 + r 2 1 +... + r n 1 1 + r n 1 ) S n (1 r) = 1 (1 r n ) S n = 1(1 r n ) 1 r (3.3). (3.4) Convergence. The sum of n infinite number of terms will diverge to ± unless r < 1. If this condition is stisfied, r n nd S = 1 1 r. 3.3 Convergence vs Divergence (3.5) The set S 1, S 2,...S n... is known s the set of prtil sums. The definition of convergence is Definition of convergence S n Finite vlue (3.6) n If series does not converge, it diverges. Series cn diverge in different wys. Some hve prtils sums tht tend either to + or, in others successive prtil sums oscillte between ±. 3.4 Absolute vs Conditionl Convergence Suppose series infinite sum S = 1 + 2 + 3 +... (3.7) is convergent. But suppose tht the series contins positive nd negtive terms. Clerly the positive nd negtive terms will cncel somewht, helping to constrin the growth of the sum. We could sk tougher question: does the following converge? S = 1 + 2 + 3 +... (3.8) If it does, the originl series is bsolutely convergent. Furthermore, if we do not know whether S converges, but cn prove S converges then this is sufficient to prove tht S does converge.

3.5. TESTS FOR CONVERGENCE OR NOT 3/3 Becuse the question sked by the S series is tougher, it could be tht S diverges, but S still converges. S is clled conditionlly convergent. Of course, if you do not know whether S converges, simply finding out tht S diverges does not tell you nything bout S. 3.5 Tests for convergence or not It would be helpful if there ws one just one test tht would indicte whether series ws convergent or not. Unfortuntely, we re forced to pply set of rther complicted tests, ny one of which my indicte convergence or divergence. Below, we will refer to series sums s n nd b n s required. Test 1: Simple test (for ll series) If n the series DIVERGES. n Explntion: Limits re its, nd must not chnge if is incresed by 1. Test 2: Comprison Test (for series of ll positive terms) If b n converges, nd ll n b n for sufficiently lrge n, then VERGES. If b n diverges, nd ll n b n for sufficiently lrge n, then VERGES. Comment: This test is obvious enough s the terms re ll positive. But wht does the sufficiently lrge r men? This cvet llows finite number of terms t the strt of the series to disobey the condition. A finite number of terms cn only contribute finite sum, so this cn discounted. (For exmple, suppose you found convergent series tht stisfied n b n for ll terms. Now just stick 47 s the first term of Series A nd s the first term of series B. We now hve 1 > b 1, but Series A must still converge.) For convenience we could just chop of the finite number of misbehving terms, nd del only with the rest of the series. n n CON- DI-

3/4 LECTURE 3. SERIES: ELEMENTARY, THEN TAYLOR & MACLAURIN Test 3: Rtio Comprison Test (for series of ll positive terms) n+1 If b n converges, nd ll b n+1 for sufficiently lrge n, then n b n CONVERGES. n+1 If b n diverges, nd ll b n+1 for sufficiently lrge n, then n b n DIVERGES. Explntion: Test 2 involved comprisons between series, which cn be wkwrd. This test however involves rtios within ech series, nd hence removes the chrcter of the two individul series when mking the comprison. Write n = ( n n 1 ) ( n 1 n 2 )... ( 2 1 ) 1 which must be Note the 1 t the end of the RHS. So, dividing out n b n b 1 1 n 1 b n b 1. ( bn b n 1 ) ( bn 1 b n 2 )... ( b2 b 1 n n ) 1. (3.9) (3.1) Now this is true for ny r. It sys tht the n re decresing more rpidly reltive to the 1 thn the b n re decresing reltive to b 1. But if Series B converges, so must Series A. Test 4: D Alembert s Rtio Test (for series of ll positive terms) If, for n, L = n ( n+1 n ) < 1 the series sum CONVERGES > 1 the series sum DIVERGES = 1 the series sum could do EITHER!. (3.11) Explntion: Note tht this test does not involve comprison. To prove the result, consider vlue L of rtio some wy into the series, where L < L < 1. We cn then choose rtios lter in the series nd be confident in writing t+1 t < L ; t+2 t+1 < L ; t+3 t+2 < L ; As ll numbers re positive, we cn therefore write t+4 t+3 < L ;.... (3.12) t+1 < L t ; t+2 < L 2 t ; t+3 < L 3 t ; t+4 < L 3 t ;... (3.13)

3.5. TESTS FOR CONVERGENCE OR NOT 3/5 nd dding up we hve t+1 + t+2 + t+3 +... < (L + L 2 + L 3 +...) t = L 1 L t, (3.14) where the equlity is from the sum of gp. So we see tht the infinite sum omitting finite number of initil terms converge, nd hence the entire series converges. (The divergence outcome is obvious, nd you should red textbook to find out why L = 1 is uncertin.) Test 5: Cuchy Integrl Test (for series of positive terms) If n consists of positive monotoniclly decresing terms, nd there exists monotic decresing function f (x) such tht f (1) = 1, f (2) = 2, nd so on, then if f (x)dx converges so does n, nd if the integrl diverges, so does 1 n. Explntion: The key to understnding this result is Figure 3.1. f(x) f(x) 1 2 3 4 1 2 3 4 x 1 etc 1 2 3 4 x Figure 3.1: Bckground to the Cuchy Integrl Test. The verticl brs t x = 1, 2, 3,... represent the vlues n t n = 1, 2, 3,..., nd the curve is f (x). Becuse the brs re seprted by x = 1, n is the re given by the sum of the rectngles. It is evident tht N N+1 n > f (x)dx, r=1 1 (3.15) which seems to offer no informtion bout convergence. However, if you copy ech excess re over into the 1 rectngle, you see immeditely tht the excess will be less

3/6 LECTURE 3. SERIES: ELEMENTARY, THEN TAYLOR & MACLAURIN tht 1 1 = f (1). Hence < n 1 f (x)dx < f (1). (3.16) Thus, if the integrl converges, so must n, nd similirly for divergence. Obvious observtion: (for series of ll negtive terms) A series of ll negtive terms cn be turned into one of ll positive terms by multiplying, er, by ( 1). Go bck to Test 2... Test 6: Leibnitz test (for series of lternting terms) If n consists of terms tht lternte in sign nd whose mgnitude decreses monotoniclly nd n n =, then the series CONVERGES. 3.6 Exmple pplictions There re mny worked exmples in the stndrd texts. Here we look t just one. Exmple: Q: Discuss the convergence properties of 1 + x + x 2 2! + x 3 3! + x 4 4! +... (3.17) A: One could esily imgine tht this series would converge for rnge of x vlues, but diverge for lrge x. However... x could be negtive, so consider the modulus of ll the terms, nd pply D Alembert s rtio test. x n+1 n! n (n + 1)! x n = x n + 1. (3.18) This tends to zero s n, so tht the series is bsolutely convergent for ny x. 3.7 Tylor s series expnsion While the series just discussed tend to occur rrely in engineering nlysis, the series expnsions we re bout to discuss re frequently useful cross the entire rnge of engineering specilities. They re very importnt!

3.7. TAYLOR S SERIES EXPANSION 3/7 Suppose we know the vlue f () of function f (x) t x =, nd suppose we know t lest some of its derivtives f (), f (), etc, ll t x =. Cn we estimte the vlue of the function t x = + h, where h is some smll offset? The nswer is yes, from Tylor s expnsion to O(h n ) f (+h) = f ()+hf ()+ h2 2! f ()+ h3 3! f ()+...+ hn n! f (n) ()+R n+1. (3.19) where the reminder R n+1 = hn+1 (n + 1)! f (n+1) (ζ), ζ + h. (3.2) You might sk, why not simply put + h into the function nd compute the result. Two resons: first you might not ctully know wht f (x) is. Second, the expnsion provides n pproximtion to function tht is polynomil in the smll quntity h. This is very useful indeed we hve lredy used the expnsion severl times in Lecture 1. Let s first think bout the expnsion, nd then worry bout the reminder term. 3.7.1 [*For musement*] Understnding the expnsion The zeroth order pproximtion would be to rgue tht ( + h) is not fr from, so f ( + h) f () (3.21) s illustrted in Figure 3.2(). But if we know the first derivtive t x =, we cn mke 1st order correction of C 1 = hf (), so tht, s sketched in Figure 3.2(b), f ( + h) f () + hf (). (3.22) But suppose we lso know f (). This should llow us to estimte the extr correction rising from the extr grdient between nd + h. The extr grdient t x = is zero, of course. Wht is the extr grdient ˆf t n intermedite point p, where p vries from to h? It is ˆf (p) = p f (). (3.23) So, between p nd p + dp this will introduce n infinitesiml extr correction to the function dˆf = ˆf (p)dp = p f () dp. (3.24)

3/8 LECTURE 3. SERIES: ELEMENTARY, THEN TAYLOR & MACLAURIN f() f() 1st order correction +h x +h x Figure 3.2: Integrting between nd h, the totl extr correction rising from the rte of chnge of grdient is C 2 = dˆf = To summrize so fr: pf () dp = f () pdp = h2 2 f (). (3.25) C 1 = C 2 = f ()dp = f () ˆf (p)dp = dp = f ()h (3.26) pf ()dp = f () pdp = f () h2 2! (3.27) (3.28) The 3rd order correction requires us to integrte the extr extr grdient due to f (). C 3 = ˆf (p)dp. (3.29) To find ˆf (p) t p we hve to integrte ˆf from to p. Tht is, ˆf (p) = p ˆf (q)dq But the extr in f t q is ˆf (q) = f ()q (3.3) (3.31)

3.8. EXAMPLES 3/9 Sticking this ll together [ p ] C 3 = f ()qdq dp = f () [ p 2 2 ] dp = f () h3 3! (3.32) We re on roll now... [ p [ q C 4 = [ p = f (4) 3 () 3! ] f (4) ()rdr ] dp = f (4) () h4 4! ] dq dp = f (4) () [ p [ q 2 nd so on nd on! Adding ll the correction to f () gives us the result. 2 ] ] dq dp (3.33) (3.34) f() p +h p+dp h x p Figure 3.3: 3.8 Exmples Fortuntely, Tylor s expnsion is esier to use thn to prove. A reminder (leving out the reminder for now) f ( + h) = f () + hf () + h2 2! f () + h3 3! f () +... (3.35) Exmple: Q: Find the series expnsion of e x bout x = 1. A: This men = 1 nd our smll quntity cn be h. Build tble of derivtives...

3/1 LECTURE 3. SERIES: ELEMENTARY, THEN TAYLOR & MACLAURIN So the series is ] e (1+h) = e [1 + h + h2 2! + h3 3! +... f (x) e x f (1) = e f (x) e x f (1) = e f (x) e x f (1) = e f (x) e x f (1) = e Suppose h =.8 nd we use 5 terms The series sys (3.36) e 1.8 e [1 +.8 +.32 +.853 +.171...] = 6.411 (3.37) nd the exct vlue is e 1.8 = 6.496 n error of.14%. If we include more terms, the error reduces further. Exmple: Q: Find the series expnsion of 1/(1 + x) bout x =. A: The fixed point is nd the smll thing is h. The tble of derivtives is... f (x) (1 + x) 1 f (x) 1(1 + x) 2 f (x) 1. 2(1 + x) 3 f (x) 1. 2. 3(1 + x) 4 In generl, we cn write (3.38) f (n) (x) = ( 1) n 1 n! (1 + x) n+1 (3.39) Now multiply ech by h n /n!, nd the series t x = becomes 1 f ( + h) = 1 + + h = 1 (1 + ) h (1 + ) 2 + h 2 (1 + ) 3 h 3 (1 + ) 4 +... (3.4) [ ( ) ( ) 2 ( ) 3 1 h h h = 1 + +...]. (1 + ) 1 + 1 + 1 + To test with some numbers, let =.5 nd h =.2. h/(1 + ) =.1333 The series is [ f (.7) = 2 ( ) ( ) 2 ( ) 3 ( ) 4.2.2.2.2 1 + +...] =.5883, (3.41) 3 1.5 1.5 1.5 1.5 while the exct vlue is 1/1.7 =.5882.

3.9. A SOURCE OF CONFUSION, CLEARED UP 3/11 3.9 A source of confusion, clered up Different books pper to express Tylor s expnsion differently. Sometimes you ll see f ( + h), other times f (x + h), nd elsewhere it will be f ( + x). This is merely different choice of symbols for the fixed point nd smll offset. The second one is choosing x s the fixed point nd h s the smll thing, so tht f (x + h) = f (x) + hf (x) + h2 2! f (x) +... (3.42) wheres the third is choosing s the fixed point nd x s the smll offset, so tht f ( + x) = f () + xf () + x 2 3.1 The reminder term explined 2! f () +... (3.43) You might cre to red how the reminder term is rrived t. Here we only need to explin wht it mens. First, the reminder is not the next term (tht would be described s the leding error term), but it is n estimte of the sum of ll the neglected terms. Second, notice tht it is esy to remember. It looks lmost exctly like the next term, but f (n+1) is not evluted t x = but t x = ζ, where ζ ( + h). But wht vlue does ζ hve exctly? The wy to work it out is ctully very strightforwrd. As h is specified, R n+1 (ζ) = hn+1 (n + 1)! f (n+1) (ζ) (3.44) is function of ζ lone. So you cn plot its vlue s ζ vries between nd + h, nd use the lrgest mgnitude vlue s the most pessimistic estimte of the reminder.? ζ is somewhere between nd +h +h x Figure 3.4: Exmple:

3/12 LECTURE 3. SERIES: ELEMENTARY, THEN TAYLOR & MACLAURIN Plot f (n+1) ( ζ) between ζ= nd ζ=(+h) +h ζ +h ζ Figure 3.5: Consider the previous exmple, where =.5 nd h =.2. We use terms up to n = 4, so tht R n+1 = ( 1) n+1 h n+1 1 (1 + ζ) n+2 R 5 = ( 1) 5.2 5 1 (1 + ζ) 6 (3.45) Now, by checking dr 5 /dζ we cn see tht there is no turning point. So the lrgest vlue must be when ζ tkes its smllest vlue in this cse: tht is, t ζ = =.5. mx R 5 = ( 1) 5 (.2) 5 1 (1 +.5) 6 (3.46) Figure 3.6 plots the bsolute vlue of the Reminder term nd the bsolute vlue of the Error s function of the number of terms used in the series. Of course, there re only discrete vlues t n = 1, 2, 3,..., but there ppers to be stright line dependence on the log-liner xes. The importnt thing is tht the ctul Error is lwys smller thn our estimte of the reminder. 3.11 McLurin s Series There is no further thought required to write down McLurin s Series. It is simply Tylor Series bout fixed point =.

3.11. MACLAURIN S SERIES 3/13 1 1 bs(reminder) bs(error) 1 2 1 bs(reminder) bs(error) 1 2 1 4 1 3 1 6 1 8 1 4 1 1.5 2 2.5 3 1 1 2 4 6 8 1 Figure 3.6: Becuse the fixed point is zero, this series is most often written using x s the smll quntity, rther thn h. McLurin s Series to O(x n ) f (x) = f ()+xf ()+ x 2 where the reminder 2! f ()+ x 3 3! f ()+...+ x n n! f (n) ()+R n+1. (3.47) R n+1 = x n+1 (n + 1)! f (n+1) (ζ), ζ x. (3.48) Exmple: Q: Determine the first four non-zero terms in the McLurin s series for ln(1 + x), nd write down the generl term. A: The tble of derivtives is: n f (n) (x) f (n) () ln(1 + x) 1 1/(1 + x) 1 2 1/(1 + x) 2-1 3 2/(1 + x) 3 2 4 (3)!/(1 + x) 4-6 n ( 1) n+1 (n 1)!/(1 + x) n ( 1) n+1 (n 1)!

3/14 LECTURE 3. SERIES: ELEMENTARY, THEN TAYLOR & MACLAURIN Hence: ln(1 + x) = x 1 2! x 2 + 2 3! x 3 6 4! x 4 +... + ( 1) n+1(n 1)! x n +... (3.49) n! = x 1 2 x 2 + 1 3 x 3 1 4 x 4 +... + ( 1) n+1 1 n x n +... (3.5) (3.51) 3.12 De l Hopitl s Rule for its You will recll tht sin(x) x x = 1 but tht this ws witing proof. (3.52) The underlying problem is tht both numertor nd denomintor tend to zero, leving nd pprently indeterminte rtio. Let s consider the more generl problem of finding f (x) x g(x) (3.53) where f () = nd g() =. We hve exctly the tool to hndle this in Tylor s expnsion f (x) x g(x) = f () + hf () + (h 2 /2!)f () +... h g() + hg () + (h 2 /2!)g () +... (3.54) But both f () nd g() re zero. Hence f (x) x g(x) = f () + (h/2!)f () +... h g () + (h/2!)g () +... = f () g () (3.55) This is De l Hopitl s rule If f () = g() =, f (x) x g(x) = f () g () (3.56) Notice tht the two derivtives re tken seprtely you do NOT use the quotient rule!

3.12. DE L HOPITAL S RULE FOR LIMITS 3/15 Exmple: Q: Find sin(x) x x (3.57) A: Both numertor nd denomintor re zero, hence using De l Hopitl s rule sin(x) x x = f () g () = cos() 1 = 1. (3.58) Exmple: Q: Find sin(x 2 ) x x 2 (3.59) A #1: If you re wke... Substitute u = x 2 nd tke the it s u. Obviously the nswer is the sme s before, unity. A #2: However, suppose you hdn t spotted tht nd just used De l Hopitl s rule sin(x 2 ) x x 2 = f () g () 2x cos(x) = = cos(). (3.6) 2x t x= Now you re llowed to divide through by 2x nd hence the nswer is cos() = 1. A #3: However however, suppose you hd not spotted even tht, nd just sid tht it is. Going bck to the series expnsion f (x) x g(x) = f () + hf () + (h 2 /2!)f () +... h g() + hg () + (h 2 /2!)g () +... (3.61) you ll tht if f () = g() = nd f () = g () = then f (x) x g(x) = f () g () This llows us to write superchrged version... De l Hopitl s rule (fuller sttement) If f () = g() = nd ll f (n) = g (n) = up to some number n = j, then (3.62) f (x) x g(x) = f (j+1) () g (j+1) () (3.63) So in our exmple we cn keep going! sin(x 2 ) x x 2 = f () g () = 2 cos(x 2 ) 4x 2 sin(x 2 ) = 1. (3.64) 2 t x=

3/16 LECTURE 3. SERIES: ELEMENTARY, THEN TAYLOR & MACLAURIN Notice tht the it you eventully rech does not hve to finite. Exmple: Q: Find ln(x + 1) x sin(x) (3.65) A: Apply De l Hopitl s rule ln(1 + x) x x 2 = 1/(x + 1) (3.66) 2x t x= De L Hopitl s rule relly just involves series expnsions, which mens tht n equivlent method of solving such problems is simple to write down the series expnsions. As the it in the lst exmple is tken bout x =, we could use the McLurin expnsion for ln(1 + x) where x is smll. We re interested in ln(1 + x) x 2 = x 1 2 x 2 + 1 3 x 3... x 2 = 1 x 1 2 + 1 3 x 1 4 x 2... (3.67) which obviously shoots off to infinity. 3.13 Using Tylor s to estimte derivtives numericlly Another very useful ppliction of Tylor s expnsion is in the estimtion of the derivtives of unknown function for which one only hs informtion t discrete vlues of the independent vrible, s shown in Figure 3.7. This is very useful computtionlly, s mesurements re often obtined t discrete points or t discrete times. f(x) f() h +h +2h x Figure 3.7: The function vlue is known only t discrete vlues of x. We know f ( h), f (), f ( + h), nd so on, but we wnt to estimte the derivtives. Using Tylor s expnsion f ( + h) = f () + hf () + h2 2! f () + h3 3! f () +... (3.68)

3.14. SUMMARY 3/17 So we could estimte or f ( + h) f () hf () + h2 2! f () (3.69) [ ] f ( + h) f () f () h (3.7) with leding error term h 2! f () tht is O(h) (order h, or proportionl to h). This mens tht if you mke h smller by fctor of 1, the error in the estimte is expected to reduce by fctor of 1. But we cn do better. Agin using Tylor s expnsion f ( + h) = f () + hf () + h2 2! f () + h3 3! f () +... (3.71) f ( h) = f () hf () + h2 2! f () h3 3! f () +... (3.72) Now subtrct... Blnk becuse it is tutoril exercise 3.14 Summry In this lecture we hve consider how to test the convergence properties if infinite series. We then looked sw how Tylor s expnsion uses the known derivtives of function f (x) t point to estimte vlues of the function t some smll displcement from the point. We sw tht McLurin s series is just Tylor s expnsion pplied t x =. Finlly we sw how Tylor s expnsion cn explin De L Hopitl s theorem for discovering the ctul it of quotient function which nively hs the vlue /.