Faithful Horner Algorithm

Transcription

1 ICIAM 07, Zurich, Switzerland, July 2007 Faithful Horner Algorithm Philippe Langlois, Nicolas Louvet Université de Perpignan, France Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

2 General motivation Evaluation of univariate polynomials with floating point coefficients: the evaluation of a polynomial suffers from rounding errors example : in the neighborhood of a multiple root 1.5e-10 1e-10 5e e-11-1e-10 Horner algorithm Exact value Example by J. Demmel : p(x) = (x 2) 9 in expanded form, evaluated with the Horner algorithm in IEEE double precision, near the multiple root x = 2-1.5e Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

3 General motivation Evaluation of univariate polynomials with floating point coefficients: the evaluation of a polynomial suffers from rounding errors example : in the neighborhood of a multiple root How to improve the accuracy of the Horner algorithm? Here we present : a compensated Horner algorithm that improves the accuracy. two results to ensure faithful polynomial evaluation with this algorithm. IEEE-754 fp arithmetic, rounding to the nearest, no over/underflow. Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

4 Outline 1 Accuracy of the Horner algorithm 2 Compensated Horner algorithm 3 Is the result faithfully rounded for small condition numbers? 4 Practical efficiency 5 Conclusion Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

5 Accuracy of the Horner algorithm We consider the polynomial n p(x) = a i x i, i=0 with a i F, x F Algorithm (Horner algorithm) function r 0 = Horner (p, x) r n = a n for i = n 1 : 1 : 0 r i = r i+1 x a i end Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

6 Accuracy of the Horner algorithm We consider the polynomial n p(x) = a i x i, i=0 with a i F, x F Algorithm (Horner algorithm) function r 0 = Horner (p, x) r n = a n for i = n 1 : 1 : 0 r i = r i+1 x a i end Relative accuracy of the evaluation with the Horner algorithm: p(x) p(x) p(x) 2nu cond(p, x) + O(u 2 ) u is the computing precision = the unit roundoff : IEEE-754 double, 53-bits mantissa, rounding to the nearest u = cond(p, x) denotes the condition number of the evaluation: ai x i cond(p, x) = 1. p(x) Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

7 Accuracy condition number of the problem u Accuracy of polynomial evaluation with the Horner scheme [n=50] 1 relative forward error Horner 2nu cond(p, x) + O(u 2 ) u condition number 1/u Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

8 Accuracy condition number of the problem u Accuracy of polynomial evaluation with the Horner scheme [n=50] 1 relative forward error Horner 2nu cond(p, x) + O(u 2 ) u condition number 1/u How can we obtain more accuracy for polynomial evaluation? Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

9 Compensated algorithms & error-free transformations Compensated algorithms: Algorithms that correct the generated rounding errors. Many examples: Kahan s compensated summation (65), Priest s doubly compensated summation (92), Ogita-Rump-Oishi (SISC 05)... The rounding errors are computed thanks to error-free transformations. Error-Free Transformations (EFT) are algorithms to compute the rounding errors at the current working precision. (x, y) = 2Prod (a, b) 17 flop Dekker (71) such that a b = x + y and x = a b + (x, y) = 2Sum (a, b) 6 flop Knuth (74) such that a + b = x + y and x = a b with a, b, x, y F. Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

10 EFT for the Horner algorithm Consider p(x) = n i=0 a ix i of degree n, a i, x F. Algorithm (Horner) function r 0 = Horner (p, x) r n = a n for i = n 1 : 1 : 0 p i = r i+1 x % error π i F r i = p i a i % error σ i F end Let us define two polynomials p π and p σ such that: n 1 n 1 p π (x) = π i x i and p σ (x) = σ i x i i=0 i=0 Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

11 EFT for the Horner algorithm Consider p(x) = n i=0 a ix i of degree n, a i, x F. Algorithm (Horner) function r 0 = Horner (p, x) r n = a n for i = n 1 : 1 : 0 p i = r i+1 x % error π i F r i = p i a i % error σ i F end Let us define two polynomials p π and p σ such that: n 1 n 1 p π (x) = π i x i and p σ (x) = σ i x i i=0 i=0 Theorem (EFT for Horner algorithm) p(x) = Horner (p, x) + (p π + p σ )(x). }{{}}{{}}{{} exact value F forward error Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

12 EFT for the Horner algorithm Consider p(x) = n i=0 a ix i of degree n, a i, x F. Algorithm (Horner) function r 0 = Horner (p, x) r n = a n for i = n 1 : 1 : 0 p i = r i+1 x % error π i F r i = p i a i % error σ i F end Theorem (EFT for Horner algorithm) Algorithm (EFT for Horner) function [r 0, p π, p σ ] = EFTHorner (p, x) r n = a n for i = n 1 : 1 : 0 [p i, π i ] = 2Prod (r i+1, x) [r i, σ i ] = 2Sum (p i, a i ) p π [i] = π i p σ [i] = σ i end p(x) = Horner (p, x) + (p π + p σ )(x). }{{}}{{}}{{} exact value F forward error Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

13 Compensated Horner algorithm (p π + p σ )(x) is exactly the forward error affecting Horner (p, x). we compute an approximate of (p π + p σ )(x) as a correcting term. Algorithm (Compensated Horner algorithm) function r = CompHorner (p, x) [ r, p π, p σ ] = EFTHorner (p, x) % r = Horner (p, x) ĉ = Horner (p π p σ, x) r = r ĉ Theorem Given p a polynomial with floating point coefficients, and x F, CompHorner (p, x) p(x) p(x) u + (2nu) 2 cond(p, x) + O(u 3 ). Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

14 Accuracy of the result u + condition number u 2. 1 Accuracy of polynomial evaluation with the compensated Horner scheme [n=50] 10-2 relative forward error nu cond(p, x) + O(u 2 ) u + (2nu) 2 cond(p, x) + O(u 3 ) u /u 1/u condition number Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

15 Accuracy of the result u + condition number u 2. 1 Accuracy of polynomial evaluation with the compensated Horner scheme [n=50] 10-2 relative forward error nu cond(p, x) + O(u 2 ) u + (2nu) 2 cond(p, x) + O(u 3 ) u /u 1/u condition number Is the result faithfully rounded for small condition numbers? Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

16 Faithful rounding Definition A floating point number x is said to be a faithful rounding of a real number x if either x = x, or x is one of the two floating point neighbours of x. x x Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

17 Faithful rounding Definition A floating point number x is said to be a faithful rounding of a real number x if either x = x, or x is one of the two floating point neighbours of x. x x The worst case accuracy bound for CompHorner, CompHorner (p, x) p(x) p(x) is too large for reasoning about faithful rounding. u + (2nu) 2 cond(p, x) + O(u 3 ) }{{} >u Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

18 A sufficient condition for faithful rounding We recall: Lemma ĉ c is the error in the computed correcting term ĉ F r = CompHorner (p, x) is the compensated result. ĉ c < u r r is a faithful rounding of p(x). 2 (see Lemma 2.5 in Accurate floating point summation, Rump, Ogita and Oishi, 2005) Using this lemma, we present two results: an a priori upper bound on cond(p, x) to ensure faithful rounding, an a posteriori (running time) test for faithful rounding. Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

19 A priori upper bound on cond(p, x) Theorem cond(p, x) < 1 u u 2 + u γ 2 2n ĉ c < u 2 r CompHorner (p, x) is a faithful rounding of p(x). An upper bound easier to interpret: cond(p, x) 1 16n 2 1 u. We observed roughly cond(p, x) 1/u faithful rounding the upper bound on cond(p, x) may be a bit pessimistic. Example: if we consider a polynomial p of degre n = 50, faithful rounding is ensured as long as cond(p, x) Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

20 An a posteriori test for faithful rounding A bound on the error ĉ c in the computed correcting term ĉ: ( ) γ2n 1 Horner ( p π p σ, x ) c ĉ fl =: β 1 2(n + 1)u Bound satisfied when computed at running time in fp arithmetic. Then, β < u 2 r c ĉ < u 2 r r is a faithful rounding of p(x). This is again a sufficient condition : if this test is satifised, this ensure faithful rounding, else, the compensated may be faithfully rounded or not. Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

21 A priori ( ) and a posteriori ( ) conditions 1 Accuracy of polynomial evaluation with the compensated Horner scheme [n=50] 10-2 relative forward error A priori bound on cond(p, x) u + (2nu) 2 cond(p, x) + O(u 3 ) u /u 1/u condition number Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

22 A priori ( ) and a posteriori ( ) conditions 1 Accuracy of polynomial evaluation with the compensated Horner scheme [n=50] 10-2 relative forward error A priori bound on cond(p, x) u + (2nu) 2 cond(p, x) + O(u 3 ) u /u 1/u condition number Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

23 Overhead to obtain more accuracy Practical overheads compared to the classic Horner algorithm 1 : CompHorner Horner DDHorner Horner CompHornerIsFaith Horner Pentium 4, 3.00 GHz GCC (sse fp unit) ICC Athlon 64, 2.00 GHz GCC Itanium 2, 1.4 GHz GCC ICC CompHorner = Compensated Horner algorithm DDHorner = Horner algorithm + double-double (Bailey s library) CompHornerIsFaith = CompHorner + test for faithful rounding. CompHorner runs a least two times faster than DDHorner. 1 Average ratios for polynomials of degree 5 to 200. Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

24 Conclusion Compensated Horner algorithm: as accurate as the Horner algorithm performed in doubled working precision, very efficient compared to the double-double alternative. Faithful polynomial evaluation with the compensated Horner algorithm: an a priori upper bound on the condition number to ensure faithful rounding, an a posteriori test to check if the computed result is faithfully rounded. Future works: cases of subnormal results, to provide an adaptative evaluation algorithm to ensure faithful rounding for polynomials with arbitrary condition number. Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

25 Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

26 Overhead to obtain more accuracy (Previous measures) Some practical ratios (running times 2 ): CompHorner Horner DDHorner Horner CompHornerIsFaith Horner Pentium 4, 3.00 GHz GCC ICC Athlon 64, 2.00 GHz GCC Itanium 2, 1.4 GHz GCC ICC CompHorner = Compensated Horner algorithm DDHorner = Horner algorithm + double-double (Bailey s library) CompHornerIsFaith = CompHorner + this test for faithful rounding. 2 Average ratios for polynomials of degree 5 to 200. Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

27 Practical efficiency How does the more accurate algorithms compare to each other? CompHornerIsFaith CompHorner DDHorner CompHorner DDHorner CompHornerIsFaith Pentium 4, 3.00 GHz GCC ICC Athlon 64, 2.00 GHz GCC Itanium 2, 1.4 GHz GCC ICC Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

28 Faithful rounding Definition A floating point number x is said to be a faithful rounding of a real number x if x < x < x +. x x x x + The error bound CompHorner (p, x) p(x) p(x) is too large for reasoning about faithful rounding. u + γ 2 2n }{{} 2n 2 u 2 cond(p, x) Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

29 A sufficient condition for faithful rounding We know that: the compensated result is r = r ĉ, the exact result is p(x) = r + c = ( r + ĉ) + (c ĉ). What condition on c ĉ to ensure that r < p(x) < r +. r r r + r + ĉ p(x) = ( r + ĉ) + (c ĉ) c ĉ c ĉ Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

30 A sufficient condition for faithful rounding r is not a power of 2: If r = r ĉ and ĉ c < u 2 r then r is a faithful rounding of ( r + ĉ) + (c ĉ) = p(x). u r u r 2 r r r + r + ĉ p(x) = ( r + ĉ) + (c ĉ) Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16

31 A sufficient condition for faithful rounding r is a power of 2: If r = r ĉ and ĉ c < u 2 r then r is a faithful rounding of ( r + ĉ) + (c ĉ) = p(x). u r u u r r 2 2 r r = 2 k r + r + ĉ p(x) = ( r + ĉ) + (c ĉ) Ph. Langlois (Université de Perpignan) Faithful Horner Algorithm 18 July / 16