Asymptotic Optimality of the. Bisection Method. K. Sikorski. Department of Computer Science Columbia University New York, N.Y G.M.

Transcription

1 Asymptotic Optimality of the Bisectio Method CUCS K. Sikoski Depatmet of Compute Sciece Columbia Uivesity New Yok, N.Y G.M. Toja Depatmet of Physics The Uivesity of weste Otaio Caada N6A3K7 Jauay 1984 AMS(MOS) Subject classificatios (1970): 6SH10, CRS.1S. This eseach was suppoted i pat by the Natioal Sciece Foudatio ude Gat MCS-~

2 ~b.tact The bi8ectio method is show to posses the asymptotically best ate of covegece fo ifiitely diffeetiable fuctios havig zeos of abitay multiplicity. If the multiplicity of zeos is bouded methods ae kow which have asymptotically at least quadatic ate of coveg~ce.

3 Summay. We seek a appoximatio to a zeo of a ifiitely diffeetiable fuctio f: ~ such that f(o) ~ 0 ad f(l} 2 O. It is kow that the eo of the bisectio method usig fuctio evaluatios is 2-(+l}. If the ifomatio used ae fuctio values, the it is kow that bisectio ifomatio ad the bisectio algoithm ae optimal. Taub ad wo~iakowski [6] cojectued that bisectio ifomatio ad the bisectio algoithm ae optimal eve if fa moe geeal ifomatio is pemitted. They pemit adaptive evaluatios of abitay liea fuctioals as ifomatio ad abitay tas fomatios of this ifomatio as algoithms. This cojectue was established i [4]. That is, fo fixed, bisectio ifomatio ad bisectio algoithm ae optimal i the wost case. Thus othig is lost by estictig oeself to fuctio values. Oe may the ask whethe bisectio is optimal i the asymptotic wost case sese,i.e., possesses asymptotically the best ate of covegece. (Asymptotic methods ae, of couse, widely used i pactice.) We pove that the aswe to this questio is positive fo the class F of fuctios havig zeos with abitay multiplicity ad cotiuous

4 fuctioal.. Assumig that evey f i F has Zeos with bouded multiplicity thee ae kow hybid methods which have at least quadatic ate of covegece as teds to ifiity, see, e.g.,bet (11, Taub [51, ad Sectio 5.

5 1 1. Fomulatio of the poblem. Let G = Coo(E) be the F~chet space of ifiitely diffeetiable fuctios o R with the metic p give by p (f,g) Vf,g e G, whee II IIi is the i-th semi-om, i = 0,1,..., "h II i = max ( I h (j) (x) I, x e [- i, i], j = 0, 1,., i }, see, e.g., Schaefe [3]. Obseve that the semi-oms ae mootoic, i.e., IIh!l. 1 2 II h./i, \j i, 'T/h e G. l.+ l. We seek a appoximatio to a zeo of a fuctio which belogs to the class FO' 00 FO = (f e C [0,1],f(O) ~ O,f(l) 2 o,~o.: f(o.) =: 0). Obviously each fuctio f i FO ca be exteded to the fuctio 1 e G. Theefoe without loss of geeality we coside the class F: (1. 1) F = (f e G: f(o) ~ O,f(l) 2 O,~o.,f(o.) = OJ. Defie the solutio opeato s: F ~ [0,1] by (1. 2)

6 2 Ou poblem i. to fid a appoximatio to S(f). To.olve this poblem we use a adaptive ifomatio opeato CD (biefly ifomatio) N: G -+ a defied as follows. Let f G ad (1. 3) N(f) = [L l {f),l 2,f(f),...,L,f(f),... ] whee L. f{') = L'('iYl,,Y 1): G -+ a ~, ~ ~- is a abitay liea fuctioal ad y. = L. (f:yl"",y. 1)' i = 2,3,... ~ ~ ~- By N (f) we deote (1. 4) Note that the vecto N l(f) cotais all compoets of + N (f), N l(f) = [N (f),l 1 f(f»). That is, iceasig + +, we use peviously computed ifomatio. It is coveiet to use the otatio N = (N). We may assume without loss of geeality that fo some fuctio f i F the fuctioals L. f{')' i = 1,2,...,, ae liealy idepedet ~, ad theefoe the fuctioal Ll is ot equal to the zeo

7 3 fuctioal. Let us also deote by ~ the class of all ifomatio opeatos of the fom (1.3). Th e bi sect~o ' ~ 'f oma t' ~o N bis, ~s d e f' ~e d b y (1. 5) L b, is (f) = f ( x, ), ~, f ~ i=1,2,..., whee x, = (a, 1 + b, 1)/2 ~ ~- ~- " with a O = 0, b O = 1 ad a, = ~ { ai-l x, ~ if f(x,) > 0 ~ if f(x,) ~ 0 ~ b, = [bi ~ _l x, ~ if f(x,) < 0, ~ if f(x,)2 ~ Kowig N (f) we appoximate S(f) by a algoithm. By the algoithm e::l = (~ } we mea a sequece of abitay tas fomatios, e::l : N (G) ~ 2, = 1,2,... Let ~(N) deote the class of all algoithms usig the ifomatio N. -th eo of fo a elemet f is defied by The (1.6) e (N'CD,f) = IS(f) - e::l (N (f» I. I the asymptotic settig we wish to fid ~* ad N* such that fo ay f i F the eo e (N*,~*,f) goes to zeo as fast as possible as teds to ifiity, The ifomatio N* ad algoithm ~* ae called optimal iff

8 4 'V N C,"" 'V ~ E I (N), ~ f* E F such that (1. 7) e (N,~, f*) lim > 0 ~up e(n*,~*,f), "If E F. bis _ (,.,.bis) The bisectio algoithm ~. is defied by bis(nbis(f» a (a + b )/2. ~ It is kow that fo evey f i F, ad (N bis bis f*) = e,~, { 2- (+l) /3 2- (+l) /6 -eve, -odd, fo f*(x) = x - 1/6. It was show i [4] that fo a fixed : sup\s(f) - ~ (N (f»\2 sup\s(f) _ <;)bis(nbis(f» I fef fef = 2 - (+l), fo evey N e ~ ad <;) E ten), i.e., that bisectio ifomatio ad algoithm ae optimal fo the wost case model with a fixed umbe of fuctioal evaluatios. Hee we show that bisectio ifomatio ad algoithm ae ealy optimal fo the asymptotic wost case model.

9 5 Moe pecisely, we show that fo evey cotiuous N, i.e., Li,f i (1.3) ae cotiuous, Li,f(gk) k~oo) Li,f(g) wheeve o (gk,g) -k.+a:/ 0, evey algoithm ~ E t(n) ad evey sequece 00 (~} 1,6 ~O, thee exists a fuctio f* i F such that = the uppe limit e (N,~, f*) lim sup...::.: ~ > 0, ad obviously fo evey f i F e (N,~, f*) lim sup ---:;~ ( bis bis f) ~!e N,~, e (N,~,f*).2 lim sup --=.;; , - ~ 5 2 sice - bis bis a e (N,~,f). covege to zeo abitaily slowly. The sequece (5 J may Theefoe we say that the bisectio ifomatio ad algoithm ae ealy optimal fo the asymptotic wost case model; compae- also to [6, p. 199] ad [8]. We fomulate this esult i Theoem 1.1: Fo evey cotiuous ifomatio N E, evey algoithm ~ E ten) ad evey sequece (6 J =1' 6~O, co thee exists a fuctio f* i F such that (1. 8) - 1 im sup e (N, ~, f* ) / (a 2 ) > o. -+<XI []

10 6 2. Sketch of the poof. Fist we give a sketch of the poof of Theoem 1.1. The poof is by cotadictio. Suppose that thee exists a opeato N* - (N~) ad a algoithm ~* = (~~J such that fo evey fuctio f i F (2.1) co. We costuct a cauchy sequece of fuctios (g) 1 ~ = cc c (K) such that g = lim g is i F ad does ot satisfy (2.1). cc Let (6') 1 be ay sequece of positive umbes, = 1 o '\&0 ' 6i < 2' such that 6 = o{!'). The we costuct a sequece of fuct~os (f) l' f e G such that f (x) = 0 = ' 1-1 fo x e I = [a,ej ad 6~2 ~ diam(i ) ~ 2,f(X) < 0 11' (esp. > 0) fo x e [-co,a ). Cespo x e (e,~]) ad N* (f1) = N*(f 1 }, q = 0,1,... Moeove ( f 1} is a ca~chy +q 1 1 sequece ad I 1 c:: I, 'i. We pove that the limit fuctio + fl = lim f1 is i F. -+eo such that By the cotiuity_of N* we get 'V. co Let (m.). 1 be a iceasig sequece of iteges J J=

11 7 (2.2) ' < 6- j ~ I fo "I > m.. J Equatio (2. 1) implies that thee exist 1 > m 1 such that (2.3) I~* (N* (f1) ) - S (f1) I < 6-1 '2- 'i ' 1 The we defie gl = f ad costuct the ext Cauchy 1 2 ~ 2 sequece of fuctios (f} 1 by settig f = * a d. f2. costuct~g ' ~ such a way l- f(x) = 0 fo x e I = [a,el ad 6!~2 ~ 1 f fo ~ 2 that f e G, diam(i 2 ) ~ , > 1,f~(X) < 0 (esp. > 0) fo x e [-~,a~) (esp. x e (e 2, ~ ] ), N* ( f2 ) = N* ( f2), q = 0, 1,..., +q 2 I2 'f ad I+1 C ' We pove that the limit fuctio f2 = lim f2 is i ad by the cotiuity of N* get ~ F "I. defie g2 we obtai such that = f2 ad epeat ou costuctio the sequece ( )=l' ad (f )=l' I this way = ~,2,...,

12 8 (2.4) CD (t) 1 is a cauchy sequece fo evey, - < 0-0 > 0 > max ( l' m ), - = 0 o ad (2. 5) whee belogs to F, (2.6 ) V (2. 7) ad N* (f ) = N*(f+q ) +q ' q=o,l,.., (2.8 ). We defie g = f, show that [g }~ 1 is a Cauchy sequece = ad that the limit fuctio 9 = lim g belogs to F. By the -

13 9 cotiuity of N* ad (2.7) we get (2.9) ~(g) = N*(g ) 'V,,S; costuctio of g iteval +l, Y. +l implies that S(g) belogs to the Theefoe (2.4), (2.5) ad (2.6) yield which cotadicts (2.1) ad completes the poof.

14 10 3. Auxiliay lemmas. I ode to costuct the fuctios i Sectio 2 'We eed a few auxiliay lemmas. Let Ul"",U m be liealy idepedet cotiuous liea fuctioals o of R. Deote A m = u 1 E. ad m J= J G, ad El,...,E m closed subitevals G: supp ( f) c A ). m Lemma 3.1: Fo evey positive c ad evey family of odegeeated itevals E.,i = 1,2,...,m-l, such that 1 Ul"",U m _ l ae liealy idepedet o C(A m _ l ), thee exists a iteval E c a, 'With diam(e ) = e, such that m m Ul,o",U m ae liealy idepedet o Am' o Poof: This is the same as the poof of Lemma 2.1 of [4] = = 'With C [a,b] eplaced by C (a), ad theefoe the poof is omitted. D popositio 3.1: Fo evey, e, 'f: > 0, ad cotiuous ifomatio N E ~ thee exist a fuctio f E G, iteval - I = [a'~]' diam(i ) E (0,2 ] ad itevals E j, j = 1,2,.o.,k, whee k is the maximal umbe of liealy idepedet fuctioals o G amog Ll f,.. 0,Lo f, '

15 11 (deote them by L*l,...,h* ) such that:, -k, (i) L*l,..,Lk* ae liealy idepedet o,, k C (U. 1 E.), J= J < 0 X E (-00, Cl ), (ii) f (x) = = > 0 X E 0 x E [Cl,t3 ], (t3,+<o), ad dis t (E :, I ) J 21 dia{e j ), \jj. II poof: The poof is simila to the poof of Lemma 2.2 of [4]. We pove below a moe geeal Lemma 3.3 which combied with Lemma 3.2 yields the popositio by iductio. The fomulatio of Lemma 3.3 eables easy vayig the iteval I which is eeded i the poof of Theoem Lemma 3.2: The popositio 3.1 holds fo 2 1 with fl such that diam{i ) l 2 ~i/2, whee 5i is the fist elemet of the sequece fom Sectio 2. o Poof: Sice Ll F 0 o G the as i the poof of Lemma 2.1of [3] we coclude that thee exists a iteval E l, diam (E l ) ~ 1/4, such that Ll F 0 o C (E l ). Let 2 -exp(-x ) x E (-00,.0), v 1 (x) = 0 X E [0,1/4], -2 exp(-(x-l/4) ) x (1/4, +00),

16 12 ad Note that vi E G. Defie f1 by if E1 c [3/8,+=), othewise. The diam(i 1 ) = 1/4, so 'i/2 ~ diam(i 1 ) ~ 1/2 ad dist(i 1,E 1 ) 2 1/8 = 1/2 diam(e 1 ) which shows that f1 satisfies Lemma 3.2. o Lemma 3.3: Suppose that popositio 3.1 holds fo, ad let Z be a abitay iteval Z c I. The po positio 3.1 holds fo + 1 with I 1 c Z such that + whee (a'} is as i Sectio 2, k / k / k + 1 ad o ~ +1 ~ L* = L~, i = 1,2,...,k. i,+l 1, c poof: Let Z = mi(2-,mi diam(e.)/2) j=l,...,k J = dia(z )(1 Defie the fuctios, see Fig. 3. 1, 6' - ~;1)/2 ad M = (e 1 +e 2 )/2.. H lex) =, exp(- (x-a +d) (X-M-b) ) x E a -d,m+b J, -2-2 exp(- (x-e) (x-e -d) ) 2 o X E [e,a +d ] 2 othewise

17 13 H 2(x) =, -2-2 exp(-(x-m+b ) (x-e-d) ) x e [M-b,e +d J, exp(-(x-a +d) (x-e) ) 1 o x e [a -d,ell othewise. k coside the fuctios G,l' G,2 e C(Ui=l E i ) such that L! (H. +G.) = 0, lo,,j,j i = 1,2,...,k, j = 1,2. Such fuctios k o C (U. 1 E.). J= J exist sice L! lo, Let ae liealy idepedet (3. 1) h.=h.+g.., J, J, J Choose a positive costat c so small that mi I f (x) 1/4 (3.2) c max\h. (x) 1 ~ mi xea,j xe(-oo,a -d JU[e +d,+00) max max j=1,2 xe~ I h 1. (x) I. C 1/2 -, J - ad (3.3) C Ilh,J./1 ~ 2 - fo j = 1,2 whee ho,j(x) - 1 ad Co = 1.

18 14. CI -d! CI 8,:z /-... "... M-b / 8 ',-...,! /, ~ / ' '...':-.. "" "... / e M" t e 2 \\. I 1 " M+b \'_... I, 8 ',2 I \ '8 " ",,/,l... ~-.", Fig 3.1 Let f. = f + c h., j = 1,2. The f. E G ad, J, J, J (3.4) N(f.)=N(f),,J s ic e L ~ (h.) = 0, i = 1, 2,..., k ad L. f (h.) = 0, ~,,J ~,,J i = 1,2,...,. The ifomatio opeato N+l yields a fuctioal L 1 +, f. If L*l,...,L*,L 1, f ae liealy k, +, depedet o G the Lemma 3.3 holds fo f 1 = f - +,l (also fo f +l = f 2) ad k 1 = k., + cases Of couse i both dist(e.,i 1) 2 diam(e.)/2, i = 1,2,...,k 1. + ~ ad diam(i 1) = diam(z )/2 - b = diam(z )6' 1/(26'). + + If L*l,,...,Lk*,,L +1, f ae liealy idepedet o G the Lemma 2.i yields that thee exists a iteval

19 15 ~ = [e l,e 1 +b J, k 1 = k + 1, such that they ae +l + k liealy idepedet o C{u. E.), The defie f 1 by J= l J + (3.5) = {ff, 1 f+1,2 if ~ +1 c= ( -00, M+b /2], othewise. The if ~ c (-oo,m+b /2], K. 1 + othewise. Obviously dist(e.,i 1) 2 diam(e. }/2, i = 1,2,...,k 1. ~ + ~ + ad diam(i l} = diam(z }/2 - b = diam(z ).&' 1/(26'}. + + Thus the fuctio f 1 satisfies Lemma IJ Lemma 3.4: Let (f } be a sequece of fuctios costucted by applyig Lemma 3.3 to the fuctio f1 fom Lemma 3.2. The (f } is coveget i G ad f = lim f belogs to F. 0 Poof: of the fom ~ Obseve that each of the fuctios f, 2 2 is (3.6 ) f f -1 h = 1 + ~i=l i whee h. = c.h.. fo j = 1 o j = 2, see (3.l), (3.2) ad ~ ~ ~,J (3.3), h. E G ad ~

20 . 16 (3.7) ad max\h. (x) l ~ mi XEa 1. mi 1 fi (x) 1/4, x ( -00, (l - d. ] u [t3. +d., +00 ), max \h i _ l (x) 1/2, (3.8) -i IIh.lI. ~ We fist pove that (f } is a Cauchy sequece, which combied with completeess of G implies covegece. ~ssume without loss of geeality that > m. The. (3.6), (3.. 8) ad mootoicity of semi-oms imply that m-l -i 00 -i () "( f, f ) m ~ l:i=l 2 Ilf-fmlli + I:i=m 2 m-l -i -l - (m-l) = I:. 1 2 lit. h li = J=m J 1. m-l -i -l Ilh ll - (m-l) ~ I:. 1 2 I: = J=m J J ~ - (m-l) (m-l) 4.2- m, = which yields covegece. Let f = lim f. that f ~ Now we pove is i F, i.e., that it has exactly oe zeo. - Recall that I +l C I ad diam(i ) ~ 2. f(a) = 0, a Theefoe 00 = =l I ad f (j) (a) ' = 0, j = 1,2,.. " sice covegece i metic p implies uifom covegece with all deivatives o evey closed iteval i ~. Now we show that a is the oly zeo of f. Namely take abitay

21 17 x F a ad assume without loss of geeality that x > ~. The we show that f (x) > o. Sice I c I ad diam(i ) / 2- +l ~ the thee exists a idex j* such that '9.2 j*. -j* Usig (3.6), (3.7), the fact that x (~,*+2,~) ad J - d ~ 2 we get f(x) j*-l ~ ~ = flex) + t, 1 h, (x) + t. '* h, (x) = f,*(x) + t,.*h, (x) J= J J=J J J J=J J ~ ~ j*-j.2 f,*(x) - t J '=J,*\h J,(X)\.2 f,*(x)-max\h,*(t)lt,_,* 2 J J t 1 J J-J.2 f.*(x) - 2 max\h,* (t) I J t 1. J which completes the poof. o

22 18 4. Coatuctios eeded i the poof of Theoem 1.1 I ode to complete the poof of Theoem 1.1 we co costuct the sequeces (f)_l' = 1,2,..., by use of Lemmas 3.3 ad 3.2 fo the ifomatio N*. 1 1 co Namely let fl z fl fom Lemma 3.2 ad let (f )=2 be the sequece of fuctios fom Lemma 3.3 with the itevale Z equal to I fo evey. Lemma 3.4 yields that fl = lim fl exists ad belogs to F. Moeove (3.4) ll-+cio implies that N (fl) = N (fl), '". Costuctios i the poof of Lemma 3.3 imply that fl has all the popeties fom Sectio 2. co Now suppose we have costucted the sequece (f)=l' 2 1, by applyig Lemma 3.3 to the fuctio f1 fom Lemma 3.2, such that (2.4), (2.5) ad (2.6) ae satisfied, whee f = lim f exists ad belogs to F by Lemma 3.4. ll-+cio We set g = f ad defie th e ext sequece (f +l)~ =l as follows: Set f + l = f fo ~ ad let f + l be the fuctio fom + l Lemma 3.3 applied to the fuctio f~, with the iteval Z give by

23 19 Z : ( Ca,a + diam(i )/6] [!3 diam(i )/6,l3 ] if 8(f) 2 (a~ +6~ )/2, othewise. The f + l ae costucted by Lemma 3.3 with Z +l ' evey Lemma 3.4 implies that f + l = exists ad belogs to F. +l = I fo. lim f + l -+oo Moeove the above defiitio of ~ combied with (2.5) yields (2.8), sice diam (I ) & I 2 Lemma 3.3 yields also that (2.4) ad (2.6) hold fo the sequece (f + 1 ). The costuctio of f + l implies that N* ~f ) = N* (f + l ) V ~ sice 1 >. + l + Theefoe by iductio (2.7) holds. 00 Let (P}=l be the sequece of fuctios p = f fo, = 1,2,..., The Lemma 3.4 yields that 9 = lim p to F. ~ exists ad belogs.' Moeove 9 = lim 9,sice (g ) is a subsequece of +l Obseve that 8(g) = :=l I~ sice I c +l - ad diam(i ) ~ 6-2 Theefoe 8(g) I + l +1 completes ou costuctios. +l I +1 c I which fially Remak 4.1. Obseve that g(j) (x) = 0, j = 0,1,..., x E I. Theefoe, as i the poof of Lemma 3.4 we coclude that

24 20 9(j) (5(9» - 0, j 0,1,...,i.e., that 9 has a zeo with ifiite multiplicity. o

25 21 5. Fial emaks. (i) Remak 4.1 idicates that bisectio is ealy optimal i the subclass of F cosistig of fuctios havig zeos with abitay multiplicity. (ii) The idea of the poof is based o the "emaace" popety itoduced by Delahaye ad Gemai-Boe i [2]. Fo othe applicatios, see also Toja [7]. (iii) If the multiplicity of zeos of fuctios i F is bouded it is possible to costuct ifomatio N ad algoithms ~ which guaatee asymptotically quadatic covegece. If the multiplicity of a zeo is kow, say m, it is eough to use a combiatio of bisectio ad modified Newto's method: x. 1 = x. - mf(x.)/f' (x.), which ~+ ~ ~ ~ coveges quadatically fo i ~ 00, see [5, p. 127]. If the multiplicity m of a zeo is ukow we ca calculate it by usig a combiatio of bisectio ad Newto's method 2 ad applyig Aitke's & fomula, see [5, p. 129, Appedix D]. The kowig m we poceed as above.

26 22 ~ckowledgemeta. We ae geatly idebted to K. Rygielaki, J.F. Taub, G.W. Wa.ilkowaki, ~. emaks o the mauscipt. Weachulz ad H. Woziakowaki fo the, Refeeces. [1] Bet, R.P., ~lgoithms fo Miimizatio without Deivatives, petice Hall, Eglewood Cliffs, New Jesey, [2] Delahaye, J.P., Gemai-Boe, B., The Set of Logaithmically Coveget Sequeces Caot be Acceleated, SIAM. J. Num. Aal., ~ugust 1982, v. 19, No.4, pp [3] Schaefe, H.H., Topological Vecto Spaces, the Mac Milla Compay, New Yok, [4] Sikoski, K., Bisectio is Optimal, Num. Math., 40, 1982, pp [5] Taub, J.F., Iteative Methods fo the Solutio of Equatios, petice-hall, Eglewood Cliffs, 1.~64. Reissued by Chelsea, New Yok, [6] Taub, J.F., Woziakowski, H., A Geeal Theoy of Optimal Algoithms, Acad. Pess, New Yok, [7] TOja, G.M., A Uppe Boud o the Acceleatio of Covegece, submitted fo publicatio. [8] Toja, G.M., Asymptotic Model fo Liea poblems, i pogess.