A tutorial on the Aitken convergence accelerator

Transcription

1 A tutoral o the Atke covergece accelerator By Cedrck Collomb Abstract. Ths tutoral troduces the cocept ad usefuless of covergece accelerators hopefully a smple maer tellgble to ay reader wth mmal mathematcal ad egeerg sklls. The Atke covergece accelerator s derved as a eample wth both tutve eplaatos (every step s purposely made uecessarly detaled for ease of uderstadg ad a smple demostrato. Source code s provded as a way to skp the potetal formula obfuscato order to help the reader mplemet ad use the Atke covergece accelerator mmedately after readg ths documet.. Itroducto There ests Mathematcs ad Computer Scece a large umber of teratve algorthms whose goal s usually to reach a soluto to a problem wth a certa tolerace wth a gve umber of teratos. Iteratg meas gog over a patter of steps ad procedures that ca sometmes be comple ad sometmes take a substatal amout of tme eve for fast moder computers. Eamples of commo teratve algorthms are root solvers, matr verso, lear equato solvers, ad tegrators. Each of these have deep mplcatos domas such as Egeerg, geeral; Computer Graphcs; Vdeo Games where rutme performace s crtcal; ad also surprsgly Iteret search eges for features such as page rak [4]. What f there ested methods to reach the same result faster wth less terato? Ths s the purpose of covergece accelerators. The rest of the tutoral s orgazed as follow: Secto troduces the Atke formula graphcally. Secto 3 troduces the Atke formulas aalytcally. Secto 4 gves a eample use of the Atke formulas. Secto 5 demostrates that the Atke formula accelerates covergece uder gve codtos. Secto 6 troduces teratve Atke formulas ad Meyers terated Atke formulas. Secto 7 eteds Meyers formulas. Secto 8 cocludes ths tutoral.. A graphcal eplaato of the Atke δ accelerator formula N Image you have a coverget sequece ( ad that ths sequece coverges to wth ( Gve a tal start pot shows how to proceed. Place f. teratvely defed by f (,, you ca costruct the sequece ( graphcally. Fgure N o the as, fd the pot of the curve y f ( ccollomb@yahoo.com / The Atke accelerator beg called δ process s due to the fact that (3, the umerator s a dfferece squared.e. ( ad the deomator s a dfferece of a dfferece.e. as (.

2 that s vertcally alged wth (, the y value of ths pot s ( f. Sce by defto f we ow have foud the value of. I order to place graphcally o the as, fd the pot of the curve y that s horzotally alged wth f ( ad project ths pot vertcally o the as. Repeatg the same procedure wll gve you, the as eeded., ad you ca cotue to get as may terms of the sequece ( 3 N Fgure y ( ( y f ( ( f f f ( P y f ( f ( P Now let s focus o the frst two pots P f ( ad P f (, ad the le gog through those two pots as show Fgure. The equato of ths le s y f ( ( ad f ( by gves y ( ( f f (. Replacg f ( by ( The dea behd Atke s to appromate by the tersecto of the le gog through P ad P, wth the le y. Fdg the tersecto s relatvely smple ad just requre solvg (, where the oly ukow s.

3 Regroupg the terms o the left sde of the equal sg gves ad after multplyg both sdes of the equato by therefore we have ( ( ( we fd ( ( ( ( ( ( ( ( Subtractg ad addg ( ( ( ( ( (( ( umerator ca be factored by ( ( ( ( factored by whch gves ( ( ( Smplfyg by ( ( the mddle of the umerator of ( gves whch frst two terms of the ad whch last two terms of the umerator ca be ( ( ( ( ( gves. Therefore (. ( ( ( ( (3 Ths s the Atke formula to fd a appromate of gve three cosecutve terms of a sequece. 3. A aalytcal eplaato of the Atke formulas As for the prevous secto you have a coverget sequece ( by f (, ad that ths sequece coverges to wth ( that f ( has a Taylor developmet at of order. We defe h, therefore h f by, ad f ( (( gves, ( O ( ( ( ( N teratvely defed f. Now let s admt f h f hf O h (4. Let α f (. Replacg by h, h by by α (4, as well as eglectg the order two error ( α ( f (5 3

4 Note that we do ot kow ad also geerally do ot kow α (5. Applyg the prevous formula to two cotuous elemets of the sequece ad, gves two equatos wth two ukows ad α. ( α ( ( α α f (6 ( α ( ( α α f (7 Subtractg (6 from (7 gves α (. Therefore gves us α. α α We use (8 to calculate ad that wll come hady very soo. α α ( ( α ad α ( ( α Combg (8 ad (9 gves α ( ( (8 (9 ( We use (6 ad (9 to fd a frst formula for. α Usg (6 ad solvg for gves α α Subtractg ad addg gves α ( Factorg the rght sde of the umerator gves α α Smplfyg gves α Therefore ad usg (9 gves α ( ( ( ( ( 4

5 Usg ( a dfferet way leads to the secod epresso for. α α α Subtractg ad addg α ( gves α ( α α ( Factorg the umerator gves α α ( Smplfyg gves, ad usg ( gves α ( ( ( ( (3 The thrd epresso for comes from (7. α Usg (7 ad solvg for gves α α α α Subtractg ad addg α gves α ( α α ( Factorg the umerator gves α α ( Smplfyg gves α Usg ( gves the most commo formulato of the Atke accelerator. ( ( ( (4 To resume there are three equvalet ways to epress, equatos (3 ad (4 could have bee drectly epressed from ( but t was worth dervg from the orgal equatos for pedagogc reasos. The hgh level tem that s very mportat to uderstad s that the Atke accelerator gves a appromate of the soluto of your problem ( based o three cosecutve terms of a coverget sequece. ( (3 (4 ( ( ( ( ( ( ( ( ( ( 5

6 4. Eample of use of the Atke accelerator The Newto method s a well kow method to fd roots of the equato g (. The g ( teratve procedure s gve by f ( g (. A eteso of the Newto method descrbed [8] s the Steffese method that uses for every three teratos two teratos of the Newto method f ad the Atke formula (4 o the last three elemets of the sequece for the last terato. 5. A smple demostrato Defto: ( T N s a acceleratg sequece 3 of ( S N true: ( T N coverges to the same lmt S as ( S, ( T N ( S N T wth the followg crtero lm. S Theorem: Let ( λ ; f the followg propertes are N coverges faster tha S be a covergg sequece to S, verfyg S lm N λ wth S. The sequece ( T acceleratg sequece of ( Demostrato: S N. N defed by a. Let s demostrate that lmt S T S ( S ( S ( S Addg S all parethess ad dvdg umerator ad deomator by S gves T S ( S S S S S S S S gves λ lmt S lm ( S S λ λ zero therefore ( T N coverges ad lmt S., s a smplfyg ad takg the lmt whe sce λ, the deomator s ot S 3 Usg the same otatos ad deftos as []. 6

7 b. Let s demostrate that ( T N coverges faster tha ( S. N ( S ( S S ( S ( S ( S ( S T S S S Addg S all parethess ad dvdg umerator ad deomator by S S S T S S gves therefore S S S S S ( λ T ( λ λ lm S λ λ λ λ S. N Therefore ( T N coverges faster tha ( S 6. Iteratve Atke Accelerator Oe mght woder why should we stop there? The Atke accelerator gves a estmate of the lmt of a coverget sequece gve three cosecutve terms, what f we created a sequece of the Atke estmatos ad appled the Atke formula o that sequece? What f we created a coverget sequece that used the Atke accelerator to coverge faster to the lmt? Those are great deas ad t s deed possble to use the Atke terator teratvely to accelerate the covergece further. a. Smple teratve method. ( Let s defe the sequece ( by N result of creatg ( N (. Followg the assumpto that ( N that ( N ( ( whch s the by applyg equato (4 to three cosecutve terms of N coverges, t s reasoable to beleve wll also coverge ad wll coverge faster. Ths s reasoable but although t works some staces t s very mportat to realze t s ot always true. So ths s a very smple way to teratvely apply the Atke accelerator. The advatage of ths method s that you ca get closer to the lmt wth few more arthmetc operatos. The other advatage s that you ca apply ths process wthout the eed of usg f or a 7

8 N sequece ( teratvely defed by f ( to store a certa umber of values of ( (. N. The dsadvatage s that you eed N that wll be used to defe the sequece Table shows the result of the teratve Atke accelerator based o 5 steps of the 9 e sequeces defed by f ( whch lmt s LambertW ( 4. Reachg the same accuracy wthout the terated Atke accelerator would requre calculatg 33. Table ( N ( N ( N Error Table shows the result of the teratve Atke accelerator based o 5 values of the sere defed by j ( j π that coverges to. Reachg the same accuracy wthout the ( j 4 terated Atke accelerator would requre calculatg 95. Table ( N ( N ( N Error b. Meyers teratve method Frst let us rewrte (3 by replacg shows the Atke formula aother form. f, ad by replacg by ( f. It by ( 4 See [9] for more detals o the Lambert W fucto. 8

9 ( f ( ( ( ( f ( ( f ( f (5 Let us ow costruct a sequece ( y N. y s the terato guess start value, chose sometmes radomly or wth a rough estmato depedg the problem you are tryg to solve. y f y. Let s defe y s the frst terato ad defed for the orgal sequece by ( a the y f ( y y f ( y y y a f ( y y. ( Let s apply (5 to y ad y to determe y. ( f ( y y ( ( f ( y y ( f ( y y ( ( f ( y y a ( f ( y y ( f ( y y ( ( y y f y y If we defe ad use the fact that a the ( ( y y a a f y y y y a a f y y (6 We wat to determe y3 by usg (3 to y, yad aother value y. A terestg value for y s gve by ( ( ( y y a a f y y ( y y a a f y y (7 The dea s to terpolate or etrapolate the drecto f ( y aa used for y. Usg (3 to y, y ad Usg (6 ad (7 gves ( y y ( y y y gves y y y y 3 3 Smplfyg umerator ad deomator by aa gves ( y y ( y y ( aa ( f ( y y ( aa ( f ( y y ( aa ( f ( y y ( aa ( f ( y y y by the same dstace ( f ( y y ( ( ( f ( y y ( f ( y y ( f ( y y a the y3 y aaa ( f ( y y ( f ( y y ( f ( y y y y a a f y y 3 If we defe 9

10 y y a ( f y y ad applyg (3 Repeatg the eact same steps to create ( to y, y ad y gves the Meyers teratve Atke formulas. Wth a y y a f y y ( f ( y y for ad a. f y y f y y ( ( ( ( ( (8 ( It s very mportat to realze here that t has ot bee demostrated that ( y s coverget, or that t respects the ecessary codtos to be accelerated by the Atke formula. Although practce the Meyers formula works really well may cases ad ts covergece s may case very mpressve, t mght be ecessary to verfy that t apples to the problem at had. Nevertheless the advatage of ths method s that t does ot requre storg all prevous values of the sequece ad t does ot requre more use of the fucto f. N Table 3 compares the results of the ormal sequece, the teratve Atke sequece ad f cos that the Meyers method, based o 5 values of the sequece defed by ( ( coverges to appromately Reachg the same accuracy wthout Meyers 6 method, requres computg 43 wth basc teratos ad wth the teratve Atke. Table 3 N ( N ( ( N y N Error c. Other teratve deas that do ot work as well There are several creatve ways you ca try to terate wth the Atke accelerator. Followg are some eamples of deas that do ot work really well practce. Reusg the method preseted 6.a by usg (5 stead of (4. It accelerates the covergece however most case the method preseted 6.a coverges faster.

11 Moreover ths method requres callg the fucto g durg the teratos, ad t wll requre teratos to reach the fal value. Aother method s based o usg (5 o two precedg values of the sequece. Ths method works well practce ad has the same advatages as the Meyers method, o storage s requred ad o addtoal calls to f are requred. However most cases the Meyers method coverges much faster. A last method that mght soud reasoable at frst but s flawed s to use (4 o three precedg values of the sequece. Although ths souds good at frst, t does ot use the fucto g ad therefore s ot lkely to coverge o the rght value. Image that two sequeces have the same tal three values, wth ths method they would ed-up havg the same calculated wrog lmt. 7. Usg Meyers method to solve g ( Creatg a fucto f that has for fed pot f ( the root of g s relatvely easy f we defe f by the followg f ( w( g (, w( eeds to be pcked so that the sequece coverges aroud. The Meyers formulas (8 appled to f followg ths defto gve: Wth a y y a ( w( y g ( y (9 ( w( y g ( y ( w( y g ( y ( w( y g ( y for ad a. Most of the tme w( wll be good eough to solve g. 8. Cocluso A well kow result the doma of data compresso s that there s o sgle compresso algorthm that ca compress all data. I other words for a gve compresso algorthm there est some data that ca ot be compressed by the algorthm. Delahaye ad Germa-Boe [5] have show a very smlar result the doma of covergece accelerato by provg that several famles of sequeces have o algorthm acceleratg the covergece of every sequece of the famly. Therefore, as wth the multtude of compresso methods there are also several other covergece acceleratos methods such as Euler s trasformato, Wy s ε algorthm, Brezsk s θ algorthm, Lev s trasforms, ad etrapolato methods.

12 The postve sde of ths complete soluto s that there s always a terest to dscover ew accelerato formulas for sequeces, ad the reader of ths tutoral ca take part ths research effort. 9. Refereces. C. Brezsk, Algorthmes d'accélérato de la Covergece, Étude Numérque, Edtos Techp (978. C. Brezsk, M.R. Zagla, Etrapolato Methods, Studes Computatoal Mathematcs, North Hollad (99 3. A. Meyers, Acceleratg Atke s covergece accelerator, World Wde Web ste at the address 4. S.D. Kamvar, et al., Etrapolato Methods for Acceleratg PageRak Computatos, World Wde Web ste at the address (3 5. J. P. Delahaye ad B. Germa-Boe, Resultats egatfs e accelerato de la covergece, Numer. Math, Vol. 35 (98, pp , 6. X. Gourdo ad P. Sebah, Covergece accelerato of seres, World Wde Web ste at the address ( 7. Davd A. Smth; Wllam F. Ford, Numercal Comparsos of Nolear Covergece Accelerators, Mathematcs of Computato, Vol. 38, No. 58. (Apr., 98, pp J. Mathews, K. Fk, Numercal Methods Usg MATLAB 3 rd edto, Pretce Hall, ( Robert M. Corless, et al., O the Lambert W Fucto, Advaces Computatoal Mathematcs, volume 5 (996, pp Apped. No optmzed C code. // Returs a estmate of the lmt gve three cosecutve terms of a sequece float AtkeEtrapolato( cost float, cost float, cost float float d - ; float d - ; retur d * d / ( d - d ; // Produce aother sequece of - terms usg the Atke etrapolato vod IteratedAtke( float* cost dst, cost float* cost src, cost log for ( log ; < - ; dst[ ] AtkeEtrapolato( src[ ], src[ ], src[ ] ; // Typedef used for Meyers fuctos typedef float (*psmplefucto( float ;

13 // Fed pot terato usg Meyers method. To accelerate Newto method use f-u(/u'( float MeyersFedPot( psmplefucto f, cost float startpot, cost log maiterato, cost float epslo log ter maiterato; float res f( startpot ; // Frst terato float last res - startpot; float prod.f; float step FLT_MAX; whle ( ( < ter && ( epslo < fabsf( step // Oe terato has already bee doe float delta f( res - res; float dff last - delta; prod * last / dff; step prod * delta; res step; last delta; ter--; retur res; // Root fdg usg Meyers method. To accelerate Newto method use gu(/u'( float MeyersRoot( psmplefucto g, cost float startpot, cost log maiterato, cost float epslo log ter maiterato; float last g( startpot ; float res startpot last; // Frst terato float prod.f; float step FLT_MAX; whle ( ( < ter && ( epslo < fabsf( step // Oe terato has already bee doe float delta g( res ; float dff last - delta; prod * last / dff; step prod * delta; res step; last delta; ter--; retur res; 3