On relative errors of floating-point operations: optimal bounds and applications

Transcription

1 On relative errors of floating-point operations: optimal bonds and applications Clade-Pierre Jeannerod, Siegfried M. Rmp To cite this version: Clade-Pierre Jeannerod, Siegfried M. Rmp. On relative errors of floating-point operations: optimal bonds and applications <hal v2> HAL Id: hal Sbmitted on 21 Dec 2015 (v2), last revised 3 Nov 2016 (v4) HAL is a mlti-disciplinary open access archive for the deposit and dissemination of scientific research docments, whether they are pblished or not. The docments may come from teaching and research instittions in France or abroad, or from pblic or private research centers. L archive overte plridisciplinaire HAL, est destinée a dépôt et à la diffsion de docments scientifiqes de nivea recherche, pbliés o non, émanant des établissements d enseignement et de recherche français o étrangers, des laboratoires pblics o privés.

2 ON RELATIVE ERRORS OF FLOATING-POINT OPERATIONS: OPTIMAL BOUNDS AND APPLICATIONS CLAUDE-PIERRE JEANNEROD AND SIEGFRIED M. RUMP Abstract. Ronding error analyses of nmerical algorithms are most often carried ot via repeated applications of the so-called standard models of floating-point arithmetic. Given a rondto-nearest fnction fl and barring nderflow and overflow, sch models bond the relative errors E 1 (t) = t fl(t) / t and E 2 (t) = t fl(t) / fl(t) by the nit rondoff. This paper investigates the possibility of refining these bonds, both in the case of an arbitrary real t and in the case where t is the exact reslt of an arithmetic operation on some floating-point nmbers. We provide explicit and attainable bonds on E 1 (t), which are all less than or eqal to /(1 + ) and, therefore, smaller than. For E 2 (t) the bond is attainable whenever t = x ± y or t = xy or, in base β > 2, t = x/y with x, y two floating-point nmbers. However, for division in base 2 as well as for sqare root, smaller bonds are derived, which are also shown to be attainable. This set of sharp bonds is then applied to the ronding error analysis of varios nmerical algorithms: in all cases, we obtain either mch shorter proofs of the best-known error bonds for sch algorithms, or improvements on these bonds themselves. Key words. floating-point arithmetic, ronding to nearest, relative error, nit in the first place AMS sbject classifications. 65G50 1. Introdction. Let F be a standard set of finite floating-point nmbers defined by a radix β, a precision p, and two extremal exponents e min and e max. Let also fl : R F {± } denote any rond-to-nearest fnction, sch that for all t R and if no overflow occrs, t fl(t) = min t f. (1.1) f F In particlar, no specific tie-breaking strategy is assmed for the fnction fl. Or first goal is to provide optimal bonds on the relative errors prodced when applying this ronding fnction, both in the case of an arbitrary real t and in the case where t is the exact reslt of an arithmetic operation on some floating-point nmbers. Here and hereafter, optimal means that each of or bonds is attained for at least one vale of t, which we shall give explicitly. Or second goal is to illstrate the interest of sch bonds for the ronding error analysis of nmerical algorithms: we provide several application examples for which these bonds yield analyses that are shorter or sharper. Classically, two relative errors can be defined, depending on whether the exact vale or the ronded vale is sed to divide the absolte error in (1.1): the error relative to t is E 1 (t) = while the error relative to fl(t) is E 2 (t) = t fl(t) t t fl(t) fl(t) if t 0, if fl(t) 0. INRIA, Laboratoire LIP (CNRS, ENS de Lyon, INRIA, UCBL), Université de Lyon, 46 allée d Italie Lyon cedex 07, France (clade-pierre.jeannerod@ens-lyon.fr). Institte for Reliable Compting, Hambrg University of Technology, Schwarzenbergstraße 95, Hambrg 21071, Germany, and Faclty of Science and Engineering, Waseda University, Okbo, Shinjk-k, Tokyo , Japan (rmp@t-harbrg.de). 1

3 2 CLAUDE-PIERRE JEANNEROD AND SIEGFRIED M. RUMP In each case the relative error is ndefined if the denominator is zero. Or bonds on E 1 (t) and E 2 (t) are smmarized in Table 1 below, where x and y denote elements of F, and where = 1 2 β1 p is the nit rondoff associated with fl and F. All these bonds are attained for specific inpt vales, which we shall describe later on in the paper. Table 1.1 Optimal relative error bonds for varios kinds of inpt t. t bond on E 1 (t) bond on E 2 (t) real nmber x ± y xy x/y { 2 2 for β = 2 and p 3, { for β = 2 and p 3, for even β 2 for even β 2 1+ x It is easily verified that the bonds appearing Table 1 satisfy the ordering < < 1 + < <. The only differences between all these expressions are in the constants of their O( 2 ) terms, as can be seen from the following expansions at = 0: = 32 + O( 3 ), = O( 3 ), 1 + = 2 + O( 3 ), = O( 3 ). In particlar, the first row in Table 1 shows that an optimal bond for E 1 (t) is generally /(1 + ), ths refining the so-called first standard model of floating-point arithmetic. In contrast, it trns ot that the second model, giving E 2 (t), cannot be improved withot frther assmptions on t. These reslts abot the standard models will be explored in Section 2. In Section 3 we establish the other bonds in Table 1 and provide in each case a proof of optimality. Finally, Section 4 illstrates with some examples the two main benefits of sch refined bonds: in some cases, more concise and slightly sharper bonds can be obtained and, in other ones, shorter proofs of existing bonds can be given. Notation and assmptions. Throghot this paper, t R and x, y F, and we assme t belongs to the normal range of F, that is, β emin t (β β 1 p )β emax.

4 OPTIMAL RELATIVE ERROR BOUNDS AND APPLICATIONS 3 Also, all or reslts hold nder the cstomary assmptions that β is even and p 2, and that nderflow and overflow do not occr when applying the ronding fnction fl. Finally, the common tool sed to establish all the error bonds in Table 1 is the fnction fp : R Z from [10], called nit in the first place and defined as follows: fp(0) = 0 and, if r R\{0}, fp(r) is the largest integer power of β sch that fp(r) r. Hence, in particlar, fp(t) t < βfp(t) and fp(t) fl(t) βfp(t). (1.2) 2. Preliminaries: floating-point arithmetic models The two standard models. In nmerical analysis, the most common way of modeling IEEE floating-point arithmetic is to bond both relative errors E 1 and E 2 by the nit rondoff. This is sally expressed by means of the following two relations, called the standard models of floating-point arithmetic [4, p. 40]: fl(t) = t (1 + δ 1 ), δ 1 (first standard model), = t/(1 + δ 2 ), δ 2 (second standard model). Using the fp-fnction, these two models are easily derived as follows. First, recall from (1.2) that t belongs to the interval [fp(t), β fp(t)), which contains (β 1)β p 1 eqally-spaced elements of F. The distance between two sch consective elements is ths β fp(t) fp(t) (β 1)β = 2 fp(t), and ronding to nearest implies that the absolte p 1 error is bonded as t fl(t) fp(t). Then, dividing by either t or fl(t) gives immediately E 1 (t) fp(t) t and E 2 (t) fp(t) fl(t). (2.1) Finally, the lower bonds in (1.2) imply that the two ratios in (2.1) are at most 1. Hence E 1 (t) and E 2 (t), and the two standard models follow. From (2.1) we can also dedce the classical phenomenon of wobbling precision [4, p. 39]: when t and fl(t) come close to βfp(t), then the relative errors E 1 (t) and E 2 (t) can be almost as small as /β. The bond given by the second standard model is best possible. Indeed, if the fnction fl is sch that ties are ronded to even, then this bond is attained for t = 1 +, since in this case fl(1 + ) = 1. If ties are ronded to away, then it is easily checked that the strict ineqality δ 2 < holds; bt the vale is best possible in the sense that for every ɛ R sch that 0 < ɛ, setting t = 1 + ɛ implies fl(t) = 1 and δ 2 = ɛ. In contrast, the bond given by the first standard model is never attained: no matter what the tie-breaking strategy, we have in fact the strict ineqality δ 1 <. This was already remarked in [4, p. 38] and, to see this, it sffices to note that t fp(t) implies that the first ineqality in (2.1) becomes strict, and that otherwise fl(t) = t and ths δ 1 = 0. An attainable bond for δ 1 is described in the next paragraph, which refines [4, Theorem 2.2].

5 4 CLAUDE-PIERRE JEANNEROD AND SIEGFRIED M. RUMP 2.2. A refinement of the first standard model. The following reslt refines the first standard model by giving a bond that can be attained no matter what the tie-breaking strategy. This bond has most probably been known since a long time, as it already appears in Dekker s 1971 paper [3, pp ] in the special case of floating-point addition and mltiplication in radix 2. Bt a general version and its optimality featre do not seem to have been reported elsewhere. Theorem 2.1. If t is a real nmber in the normal range of F then fl(t) = t (1 + δ 1 ), δ Frthermore, this bond is attained if and only if 0 t = (1 + )fp(t). Proof. If t = (1 + )fp(t) then fl( t ) is either fp(t) or (1 + 2)fp(t). Recalling (for example from [8, Lemma 2.1]) that t fl(t) = t fl( t ), we dedce that in both cases t fl(t) = fp(t), from which the eqality E 1 (t) = /(1 + ) follows. If t < (1 + )fp(t), then fl(t) = fp(t) t and ths t fl(t) = t fp(t). Conseqently, E 1 (t) = 1 fp(t)/ t, which is strictly less than 1 1/(1+) = /(1+). If t > (1 + )fp(t), the strict ineqality E 1 (t) < /(1 + ) follows from the first bond in (2.1). In the same way as in the proof above, we can check that when ronding ties to even, the bond in the second standard model is attained if and only if the nonzero real t has the form t = (1 + )fp(t). To smmarize, we have seen how to derive the bonds appearing in the first row of Table 1, that is, E 1 (t) and E 2 (t), (2.2) 1 + and characterized their optimality when t can be any real nmber in the normal range of F. In the next section we examine whether these bonds can or cannot be refined frther when t is the exact reslt of a basic arithmetic operation on floating-point nmbers. 3. Optimal error bonds for floating-point operations. We establish here optimal bonds on both E 1 and E 2 for the operations of addition, sbtraction, mltiplication, fsed mltiply-add, division, and sqare root. In some of or proofs, it will be convenient to se the following straightforward property abot the invariance of relative errors nder scaling by an integer power of the radix. Property 3.1. Let t R and e Z be sch that both t and tβ e lie in the normal range of F. Then E 1 (t) = E 1 (tβ e ) and, if t is not a midpoint, E 2 (t) = E 2 (tβ e ). Proof. If t is not a midpoint then its scaled conterpart tβ e cannot be a midpoint either, so fl(tβ e ) = fl(t)β e independent of the way the ronding fnction fl breaks ties. Hence the two claimed eqalities. Let s now show that the first eqality still holds if t is a midpoint. In this case, t fl(t) = fp(t) and, tβ e being also halfway between two consective floating-point nmbers, tβ e fl(tβ e ) = fp(tβ e ). Now, fp(tβ e ) = fp(t)β e, and we conclde that E 1 (tβ e ) = fp(t)/ t = E 1 (t). In case of midpoints, scaling invariance is lost for E 2 when the ronding fnction fl breaks ties according to the fp of sch midpoints. Fortnately, this does not occr for any of the IEEE rondings: their tie-breaking strategies being independent of fp(t), we always have fl(tβ e ) = fl(t)β e.

6 OPTIMAL RELATIVE ERROR BOUNDS AND APPLICATIONS Addition, sbtraction, and fsed mltiply-add. When t = x + y for some x, y in F, the bonds in (2.2) remain optimal, as they can be attained for (x, y) = (1, ) F 2. The same holds for sbtraction as well as for higher-level operations encompassing addition, like the fsed mltiply-add operation Mltiplication. When t = xy, the theorem below shows that the bonds in (2.2) remain optimal nless the base β and precision p are sch that β = 2 and 2 p + 1 is a Fermat prime. Theorem 3.2. Let β 2 be even, p 2, and x, y F be sch that xy lies in the normal range of F. We have the following, depending on the vale of the base β: If β = 2 then the bonds in (2.2) are optimal if and only if 2 p + 1 is not prime; frthermore, if D is a non-trivial divisor of 2 p + 1, then these bonds are attained for t = xy with ( ) (2 + 2)fp(D) D (x, y) =,. D fp(d) If β > 2 then the bonds in (2.2) are optimal, and attained for t = xy with (x, y) = (2 + 2, 2 1 ). Proof. Recalling Theorem 2.1, we see that for t = xy the optimality of the bonds in (2.2) is eqivalent to the existence of x, y F sch that xy = (1 + )fp(xy). When β > 2, taking x = and y = 2 1 gives xy = 1 + and fp(xy) = 1, from which xy = (1 + )fp(xy) follows. Frthermore, it is easily checked that sch vales of x and y are in F. (Note however that F when β = 2.) Let s now consider the case β = 2. With no loss of generality, we can assme 1 x, y < 2. This implies fp(xy) {1, 2} and, since x, y {1, 1 + 2, 1 + 4,...} and > 0, the prodct xy cannot be eqal to 1 +. Hence, optimality is eqivalent to the existence of x, y F [1, 2) sch that xy = 2 + 2, that is, eqivalent to the existence of integers X, Y sch that XY = (2 p + 1) 2 p 1 and 2 p 1 X, Y < 2 p. (3.1) If 2 p + 1 is prime, then either X or Y mst be larger than 2 p, so (3.1) has no soltion. If 2 p +1 is composite, one can constrct a soltion (X 0, Y 0 ) to (3.1) as follows. Let D denote a non-trivial divisor of 2 p + 1, and let X 0 = 2p +1 D fp(d) and Y 0 = D 2p 1 fp(d). Clearly, X 0 is an integer and the prodct X 0 Y 0 has the desired shape. Ths, it remains to check that Y 0 Z and that both X 0 and Y 0 are in the range [2 p 1, 2 p ). Since 2 p +1 is odd, D mst be odd too, which implies Hence fp(d) + 1 D < 2fp(D) and D < 2 p. (3.2) Conseqently, fp(d) 2 p 1, so that fp(d) divides 2 p 1 and Y 0 is an integer. Frthermore, (3.2) leads to 2 p 1 < 2p +1 2 < X 0 (2 p + 1)(1 1 D ) < 2p and 2 p 1 < Y 0 < 2 p, so that X 0 and Y 0 satisfy the range constraint in (3.1). Finally, mltiplying X 0 and Y 0 by 2 = 2 1 p gives x 0 = (2 + 2)fp(D)/D and y 0 = D/fp(D) in F [1, 2) and sch that x 0 y 0 = = (1 + )fp(x 0 y 0 ). For the so-called basic binary formats of the IEEE standard [5], we have p {24, 53, 113}. These are special cases of the following two sitations, where an explicit vale of a non-trivial divisor of 2 p + 1 can be obtained:

7 6 CLAUDE-PIERRE JEANNEROD AND SIEGFRIED M. RUMP If β = 2 and p is odd (for example, p = 53 or p = 113), then 3 divides 2 p + 1 and ths (x, y) = ( 4+4 3, 3 2 ) is in F 2 and sch that xy = If β = 2 and p 0 mod 3 (for example, p = 24), then 2 p + 1 can be factored as (2 p/3 + 1)(2 2p/3 2 p/3 + 1), so that (x, y) = (2 2 1/ /3, 1 + 1/3 ) is in F 2 and satisfies xy = Crrently, the only known Fermat primes are 2 2l + 1 with l {0, 1, 2, 3, 4}, so that besides p {2, 4, 8, 16}, no precision is known for which the bonds are not sharp. However, for IEEE floating-point arithmetic, we can go beyond Theorem 3.2 by proving that optimality is garanteed for any precision p. Indeed, for all the binary formats other than the basic ones seen above, the IEEE standard assmes p has a special form, and the lemma below shows that in this case 2 p + 1 cannot be a Fermat prime. Lemma 3.3. Let j and k be two integers sch that j 4 and k = 32j, and let p = k d + 13 with d an integer nearest to 4 log 2 k. Then 2 p + 1 is not prime. Proof. If 2 p +1 is prime then p mst be an integer power of two, so that it sffices to check that the latter never occrs for or particlar vales of p. If j {4, 5, 6, 7} then p {113, 144, 175, 206}. Hence in this case p is not an integer power of two. Assme now that j 8. Writing d = 4m + i for integers m, i with 0 i 3, we have 4m + i log 2 k 4m + i If i 0, this implies 2 m+1/8 k 2 m+7/8 and then 2 m+1/8 4m + 10 p 2 m+7/8 4m Since the assmption j 8 implies k 2 8 and ths m 8, it follows that 2 m < p < 2 m+1. Conseqently, p cannot be an integer power of two when i 0. On the other hand, when i = 0, we see that p = 32j 4m + 13 mst be odd, and ths cannot be an integer power of two neither. Corollary 3.4. When t = xy, the bonds in (2.2) are optimal for all the floating-point formats of the IEEE standard Division. This section focses on the largest possible relative errors committed when ronding x/y with x, y in F. As the two theorems below show, the bonds in (2.2) can be refined frther in base 2, bt remain optimal for other bases. Theorem 3.5. Let β 2 be even, p 3, and x, y F be sch that x/y lies in the normal range of F. Then E 1 (x/y) 2 2 if β = 2,

8 and OPTIMAL RELATIVE ERROR BOUNDS AND APPLICATIONS 7 E 1 (x/y) 1 + if β > 2. The bond for β = 2 is attained at (x, y) = (1, 1 ) and, assming ties are ronded to even, the bond for β > 2 is attained at (x, y) = (2 + 2, 2). Proof. Consider first the case where β = 2. Writing t = x/y, we can assme that t > 0 with x, y > 0; applying Property 3.1, we can assme frther that 1 t < 2, and since E 1 (t) is zero for t = 1, we are left with handling t sch that 1 < t < 2. The lower bond on t implies x > y, which for x and y in F is eqivalent to x y + 2 fp(y). Hence, sing y (2 2)fp(y), t fp(y) y = 1 1. (3.3) Since 1/(1 ) is strictly larger than the midpoint 1 +, we dedce that fl(t) {1 + 2, 1 + 4,...}. (3.4) If fl(t) then t and, applying (2.1) together with fp(t) = 1, we obtain E 1 (t) 1+3 ; this bond is at most 22 as soon as 1 6, which holds for β = 2 and p 3. If fl(t) = then either fl(t) t 1 + 3, in which case E 1 (t) = t 1+3, or fl(t) > t, in which case (3.3) gives E 1(t) = 1+2 t This concldes the proof of the bond on E 1 (t) when β = 2. This bond is attained at x = 1 and y = 1, since we then have x, y F as well as x/y = 1/(1 ) and fl(x/y) = Consider now β > 2. In this case, the pper bond on E 1 (t) is the one already established for any real t in the normal range of F. To prove that this bond is attained when dividing x = by y = 2, it sffices to check that x, y F and that x/y = 1 +. Theorem 3.6. Let β 2 be even, p 3, and x, y F be sch that x/y lies in the normal range of F. Then and E 2 (x/y) if β = 2, E 2 (x/y) if β > 2. The bond for β = 2 is attained at (x, y) = (1, 1 ) and, assming ties are ronded to even, the bond for β > 2 is attained at (x, y) = (2 + 2, 2). Proof. The attainability of the two bonds can be shown in exactly the same way as for Theorem 3.5. Frthermore, when β > 2 the pper bond on E 2 is the general one already established earlier. The rest of the proof ths establishes the bond when β = 2. 2 We can assme x, y > 0 and let t = x/y. Since t is not a midpoint [7], we can apply Property 3.1 and proceed as in the proof of Theorem 3.5 to restrict to the sitation where 1 < t < 2, for which (3.3) and (3.4) are tre.

9 8 CLAUDE-PIERRE JEANNEROD AND SIEGFRIED M. RUMP If fl(t) then, applying (2.1) together with fp(t) = 1, we obtain E 2 (t) 1+4. This bond at most as soon as 1 2 6, which holds for β = 2 and p 3. Assme now that fl(t) = If t fl(t) then E 2 (t) = 1 t, and we dedce from (3.3) that E 2 (t) 1 (1 )(1+2) = , as wanted. If t > fl(t) then 2 E 2 (t) = t (3.5) with < t < This pper bond on t, which we have sed to bond E 1 (t) in Theorem 3.5 is now not enogh to bond E 2 (t) as wanted. We can improve it as follows. The range of t implies y + 2 fp(y) < x < y + 6 fp(y) and, since x is in F, we dedce that x = y + 4 fp(y). On the other hand, the floating-point nmber y can be written y = (1 + 2k)fp(y), k {0, 1,..., 2 p 1 1} k 2p 1 Conseqently, t = 1 + and the condition t < is eqivalent to k > 3. Since k is an integer, the latter ineqality is eqivalent to k 2p Hence 2k and we arrive at the pper bond t = O( 3 ). (3.6) 2(1 ) (2+)(1+2) From (3.5) and (3.6) it follows that E 2 (t), which is less than 22 for β = 2 and p 3. This concldes the proof of the bond on E 2 (t) when β = Sqare root. Finally, we show how to refine frther the bonds (2.2) on E 1 (t) and E 2 (t) in the special case where t = x for some positive floating-point nmber x, thereby establishing the bonds in the last row of Table 1. Theorem 3.7. Let β 2 be even, p 2, and x F >0. Then E 1 ( x) , and this bond is attained for x = Proof. Defining t = x, we distingish between the following two cases: If t (1 + )fp(t) then t 2 = x (1 + 4)fp(t), so that t fp(t). This implies E 1 (t) ϕ with ϕ = 1+4, and it can be checked that ϕ when 1/2. If t < (1 + )fp(t) then fl(t) = fp(t) and de to the fact that t 2 is a floating-point nmber, t fp(t). Therefore, E 1 (t) = 1 fp(t)/t Frthermore, x = implies fl( x) = 1, and ths the bond is attained for this vale of x. Theorem 3.8. Let β 2 be even, p 2, and x F >0. Then E 2 ( x) , and this bond is attained for x = Proof. Writing t = x we have t > 0 and fl(t) fp(t) > 0 and we consider the following two sbcases:

10 OPTIMAL RELATIVE ERROR BOUNDS AND APPLICATIONS 9 If fl(t) = fp(t) then t is at most (1+)fp(t), which implies t fp(t) since t 2 = x is in F. Hence E 2 (t) = t/fl(t) If fl(t) > fp(t) then fl(t) (1 + 2)fp(t). On the other hand, t fl(t) fp(t), so that E 2 (t) 1+2 < for positive. Frthermore, x = implies fl( x) = 1, and ths the bond is attained for this vale of x. 4. Application examples. We now illstrate with a few examples the two main benefits of the refinements presented in the previos section: in some cases, more concise and slightly sharper bonds can be obtained and, in other ones, mch shorter proofs of existing bonds can be given. In both cases, the derivations are (relatively) straightforward and do not involve fp-based discssions Example 1: Small arithmetic expressions. Ronding error analyses as those done in [4] typically involve bonds on θ n, where θ n is an expression of the form θ n = n i=1 (1 + δ i) 1 with δ i a relative error term associated with a single floating-point operation. Using the classical bond δ i given by the first standard model, it is easily checked that for all n, n θ n (1 + ) n 1. (4.1) Note that only the pper bond has the form n + O( 2 ), that is, contains terms nonlinear in. From (4.1) it follows that θ n (1 + ) n 1 and, assming n < 1, this bond itself is sally bonded above by the classical fraction γ n = n/(1 n). By applying Theorem 2.1, we can replace δ i by δ i /(1 + ), which immediately leads to the refined enclosre n 1 + θ n ( ) n 1. (4.2) Althogh the pper bond in (4.2) still has O( 2 ) terms in general, it is bonded by n as long as n 3. Conseqently, by jst applying the refinement of the first standard model, we can replace γ n = n+o( 2 ) by n in every error analysis where θ n appears with n 3. This is the case when evalating small arithmetic expressions like the prodct x 1 x 2 x 3 x 4 (sing any parenthesization) or the sms ((x 1 + x 2 ) + x 3 ) + x 4 and ((x 1 +x 2 )+(x 3 +x 4 ))+((x 5 +x 6 )+(x 7 +x 8 )) (sing these specific parenthesizations); in each case the forward error bond classically involves γ 3, which we now replace by Example 2: Leading principal minors of a tridiagonal matrix. Consider the tridiagonal matrix d 1 e 1 c 2 d 2 e 2 A = F n n, en 1 and let µ 1, µ 2,..., µ n be the seqence of its n leading principal minors. Writing µ 1 = 0 and µ 0 = 1, those minors are ths defined by the linear recrrence c n d n µ k = d k µ k 1 c k e k 1 µ k 2, 1 k n.

11 10 CLAUDE-PIERRE JEANNEROD AND SIEGFRIED M. RUMP Assming the (first) standard model of floating-point arithmetic and barring nderflow and overflow, Wilkinson shows in [11, 3] that the evalation of this recrrence prodces floating-point nmbers µ 1,..., µ n sch that µ k = d k (1 + ɛ k ) µ k 1 c k (1 + ɛ k)e k 1 (1 + ɛ k) µ k 2, where (1 ) 2 1 ɛ k (1 + ) 2 1 and (1 ) 3/2 1 ɛ k, ɛ k (1 + )3/2 1. In other words, the compted µ k are the leading principal minors of a nearby tridiagonal matrix A + A = [a ij (1 + δ ij )] that satisfies 2 < δ ii and 3 2 < δ ij O(2 ) if i j. (4.3) Notice that the terms 2 and O( 2 ) come exclsively from the pper bonds on ɛ k, ɛ k, ɛ k. By sing Theorem 2.1, which says δ /(1 + ) instead of jst δ, these pper bonds are straightforwardly improved to ɛ k ( )2 1 < 2 and ɛ k, ɛ k ( )3/2 1 < 3 2. Conseqently, Wilkinson s bonds in (4.3) can be replaced by the following more concise and slightly sharper ones: δ ii < 2 and δ ij < 3 2 if i j Example 3: Eclidean norm of an n-dimensional vector. Given x 1,..., x n in F, let the norm r = x x2 n be evalated in floating-point in the sal way: form the sqares fl(x 2 i ), sm them p in any order into ŝ, and retrn r = fl ( ŝ ). By applying the first standard model, all we can say is ŝ = ( n i=1 x2 i )(1 + θ) with (1 ) n 1 θ (1 + ) n 1, and r = ŝ (1 + δ) with δ. Conseqently, r = r(1 + ɛ), where ɛ = 1 + θ (1 + δ) 1 satisfies (1 ) n/2+1 1 ɛ (1 + ) n/ Althogh the lower bond has absolte vale at most (n/2 + 1) (see Lemma A.1), the pper bond is strictly larger than this, so the standard model gives only (n/2 + 1) ɛ (n/2 + 1) + O( 2 ). (4.4) To avoid the O( 2 ) term above, we can simply apply Lemma 2.1, which says δ /(1 + ), together with the improved bond for inner prodcts from [8], which says θ n. Indeed, from these two bonds we dedce that ɛ is pper bonded by 1 + n (1 + /(1 + )) 1, and the latter qantity is easily checked to be at most (n/2 + 1). Ths, recalling the lower bond in (4.4), we conclde that ɛ (n/2 + 1). (4.5) In particlar, evalating the hypotense x x2 2 in floating-point prodces a relative error of at most 2. Of corse, the bond in (4.5) also applies when scaling by integer powers of the base is introdced to avoid nderflow and overflow.

12 OPTIMAL RELATIVE ERROR BOUNDS AND APPLICATIONS Example 4: Cholesky factorization. We consider A F n n symmetric and its trianglarization in floating-point arithmetic sing the classical Cholesky algorithm. If the algorithm rns to completion, then by sing the two standard models the traditional ronding error analysis concldes that the compted factor R satisfies R T R = A + A with A γ n+1 R T R ; see for example [4, Theorem 10.3]. Here γ n+1 = (n+1) 1 (n+1) has the form (n+1)+o(2 ) and reqires n + 1 < 1. It was shown in [9] that both the qadratic term in and the restriction on n can be removed, reslting in the improved backward error bond A (n + 1) R T R. In the proof of [9, Theorem 4.4], one of the ingredients sed to sppress the O( 2 ) term is the following property: ( a F 0 and b = fl ( a )) b 2 a 2b 2. (4.6) In [9] it is shown that this property may not hold if only the (second) standard model is assmed, and that in this case all we can say is (2 + 2 )b 2 b 2 a 2b 2. Frthermore, a proof of (4.6) is given, which is abot 10 lines long and based on a fp-based case analysis. Instead, or refinement of the second standard model for sqare root provides a mch shorter argment: sing Theorem 3.8, we see that b(1 + δ) = a with δ ; hence b 2 a = (2 + δ)δ b 2 and the range of δ leads to (2 + δ)δ [ , 2 ] [ 2, 2], from which (4.6) follows immediately Example 5: Complex mltiplication with an FMA. Given a, b, c, d F, consider the complex prodct z = (a + ib)(c + id). Varios approximations ẑ = R + i Î to z can be obtained, depending on how R = ac bd and I = ad + bc are evalated in floating-point. It was shown in [1] that the conventional way, which ses 4 mltiplications and 2 additions, gives ẑ = z(1 + ɛ) with ɛ C sch that ɛ < 5, and that the constant 5 is, at least in base 2, best possible. Assme now that an FMA is available, so that we compte, say, R = fl(ac fl(bd)) and Î = fl(ad + fl(bc)). For this algorithm and its variants 1 it was shown in [6] that the bond 5 can be redced frther to 2, and that the latter is essentially optimal. The fact that 2 is an pper bond is established in [6, Theorem 3.1], whose proof is rather long. As we shall see, a mch more concise proof can be obtained by applying directly the refined version of the first standard model. 1 There are three other ways to insert the innermost ronding fl, all giving the same error as the one here.

13 12 CLAUDE-PIERRE JEANNEROD AND SIEGFRIED M. RUMP Denoting by δ 1,..., δ 4 the for ronding errors involved, we have R = (ac bd(1 + δ 1 ))(1 + δ 2 ) = R + Rδ 2 bdδ 1 (1 + δ 2 ) and, similarly, Î = I + Iδ 4 + bcδ 3 (1 + δ 4 ). Now let λ, µ R be sch that δ 2, δ 4 λ and δ 1 (1 + δ 2 ), δ 3 (1 + δ 4 ) µ. This implies R R λ R + µ bd and I Î λ I + µ bc, from which we dedce z ẑ 2 = (R R) 2 + (I Î)2 λ 2 z 2 + 2λµA + µ 2 B, (4.7) where A = R bd + I bc and B = (bd) 2 + (bc) 2. It trns ot that A, B z 2. (4.8) For B, this bond simply follows from the eqality z 2 = (ac) 2 +(bd) 2 +(ad) 2 +(bc) 2. For A, define π = abcd and notice that A = π (bd) 2 + π + (bc) 2 is eqal to either B or ±(2π + (bc) 2 (bd) 2 ); frthermore, in the latter case we have A 2 π + (bc) 2 (bd) 2 min { (ac) 2 + (bd) 2, (ad) 2 + (bc) 2} + max { (bc) 2, (bd) 2} z 2. Ths, combining (4.7) and (4.8), z ẑ (λ + µ) z. Since or refined model gives δ i /(1 + ) for all i, we can take λ = /(1 + ) and µ = /(1 + ) (1 + /(1 + )), which are both less than. Hence, barring nderflow and overflow and since z = 0 implies ẑ = 0, we conclde that ẑ = z(1 + ɛ), ɛ 2+32 (1+) 2 < 2. Note that 2+32 (1+) has the form O( 4 ) as 0. Ths, or approach not 2 only yields a mch shorter proof of the bond 2 of [6], bt it also improves slightly on that bond. Appendix A. A sefl ineqality. We give below a slight variant of the generalized Bernolli ineqality [2, p. 10, Exercise 18(d)]. A detailed proof is given for the sake of completeness. Lemma A.1. Let, x R be sch that 0 < 1 and x 1. Then (1 ) x 1 x. Proof. We show that for each, the difference f (x) := (1 ) x 1 + x is nonnegative over [1, + ). Since f (1) = 0, it sffices to check that f (x) 0 for all x 1. We have f (x) = ln(1 ) (1 ) x +, where ln(1 ) 0 and x (1 ) x is decreasing over [1, + ). Therefore, f is increasing over [1, + ), and all we need is f (1) 0 for all [0, 1). This is tre becase g() := f (1) = ln(1 ) (1 ) + satisfies g(0) = 0 and g () = ln(1 ) 0.

14 OPTIMAL RELATIVE ERROR BOUNDS AND APPLICATIONS 13 REFERENCES [1] R. P. Brent, C. Percival, and P. Zimmermann, Error bonds on complex floating-point mltiplication, Math. Comp., 76 (2007), pp [2] N. L. Carothers, Real analysis, Cambridge University Press, [3] T. J. Dekker, A floating-point techniqe for extending the available precision, Nmer. Math., 18 (1971), pp [4] N. J. Higham, Accracy and Stability of Nmerical Algorithms, SIAM, Philadelphia, second ed., [5] IEEE Compter Society, IEEE Standard for Floating-Point Arithmetic, IEEE Standard , Ag available at [6] C.-P. Jeannerod, P. Kornerp, N. Lovet, and J.-M. Mller, Error bonds on complex floating-point mltiplication with an FMA, Sept Sbmitted to Math. Comp. [7] C.-P. Jeannerod, N. Lovet, J.-M. Mller, and A. Panhalex, Midpoints and exact points of some algebraic fnctions in floating-point arithmetic, IEEE Trans. Compt., 60 (2011), pp [8] C.-P. Jeannerod and S. M. Rmp, Improved error bonds for inner prodcts in floating-point arithmetic, SIAM J. Matrix Anal. Appl., 34 (2013), pp [9] S. M. Rmp and C.-P. Jeannerod, Improved backward error bonds for LU and Cholesky factorizations, Jly Sbmitted to SIAM J. Matrix Anal. Appl. [10] S. M. Rmp, T. Ogita, and S. Oishi, Accrate floating-point smmation, Part I: Faithfl ronding, SIAM J. Sci. Compt., 31 (2008), pp [11] J. H. Wilkinson, Error analysis of floating-point comptation, Nmer. Math., 2 (1960), pp