Computation of dot products in finite fields with floating-point arithmetic

Size: px

Start display at page:

Download "Computation of dot products in finite fields with floating-point arithmetic"

Virginia Ray
5 years ago
Views:

Computation of dot products in finite fields with floating-point arithmetic Stef Graillat Joint work with Jérémy Jean LIP6/PEQUAN - Université Pierre et Marie Curie (Paris 6) -

1 Computation of dot products in finite fields with floating-point arithmetic Stef Graillat Joint work with Jérémy Jean LIP6/PEQUAN - Université Pierre et Marie Curie (Paris 6) - CNRS Computer-assisted proofs - tools, methods and applications Dagstuhl Seminar Germany, November 15-20, 2009 S. Graillat (Univ. Paris 6) Dot products in finite fields 1 / 36

2 Outline of the talk Motivations Basic Floating-point arithmetic Finite fields Computation of dot products First method Second method Comparison Conclusion and future work S. Graillat (Univ. Paris 6) Dot products in finite fields 2 / 36

3 Motivations Dot products: key tool in numerical linear algebra Fast algorithms in scientific computing Cryptology Error-correcting codes Computer algebra S. Graillat (Univ. Paris 6) Dot products in finite fields 3 / 36

4 Floating-point numbers Normalized floating-point numbers F R: x = ± x 0.x 1... x }{{ M 1 b e, } 0 x i b 1, x 0 0 mantissa b : basis, M : precision, e : exponent such that e min e e max Approximation of R by F with rounding fl : R F. Let x R then fl(x) = x(1 + δ), δ u Unit rounding u = b 1 M for rounding toward zero S. Graillat (Univ. Paris 6) Dot products in finite fields 4 / 36

5 Standard model of floating-point arithmetic Let x, y F and {+,,, /}. The result x y is not in general a floating-point number fl(x y) = (x y)(1 + δ), δ u IEEE 754 standard (1985) Type Size Mantissa Exponent Unit rounding Interval Single 32 bits 23+1 bits 8 bits u = , ±38 Double 64 bits 52+1 bits 11 bits u = , ±308 S. Graillat (Univ. Paris 6) Dot products in finite fields 5 / 36

6 Finite field F p (p prime) F p = Z/pZ = GF (p) = {0, 1,..., p 1} is a finite field with characteristic p S. Graillat (Univ. Paris 6) Dot products in finite fields 6 / 36

7 Finite field F p (p prime) F p = Z/pZ = GF (p) = {0, 1,..., p 1} is a finite field with characteristic p Operations in the field, for a, b Z/pZ: Addition: a + b {0,..., 2(p 1)} a + b (mod p) Z/pZ Multiplication: ab {0,..., (p 1) 2 } ab (mod p) Z/pZ S. Graillat (Univ. Paris 6) Dot products in finite fields 6 / 36

8 Finite field F p (p prime) F p = Z/pZ = GF (p) = {0, 1,..., p 1} is a finite field with characteristic p Operations in the field, for a, b Z/pZ: Addition: a + b {0,..., 2(p 1)} a + b (mod p) Z/pZ Multiplication: ab {0,..., (p 1) 2 } ab (mod p) Z/pZ Reduction modulo p for a Z/pZ: a a (mod p) = a p = a a.invp p p S. Graillat (Univ. Paris 6) Dot products in finite fields 6 / 36

9 Aim Let p 3 a prime number and (a i ) i, (b i ) i two vectors of N scalars in Z/pZ. We want to compute the dot product of a and b in Z/pZ: a b = N a i b i (mod p) i=1 S. Graillat (Univ. Paris 6) Dot products in finite fields 7 / 36

10 Aim Let p 3 a prime number and (a i ) i, (b i ) i two vectors of N scalars in Z/pZ. We want to compute the dot product of a and b in Z/pZ: a b = N a i b i (mod p) i=1 Assumptions: The integers are stored as floating-point numbers F N The prime p satisfies p 1 < 2 M 1 The numbers are assumed to be nonnegative The rounding mode is rounding toward zero S. Graillat (Univ. Paris 6) Dot products in finite fields 7 / 36

11 Rounding toward zero in R + Let x R + fl(x) be the rounding toward zero of x in F Equivalent to a truncation fl(x) F x F S. Graillat (Univ. Paris 6) Dot products in finite fields 8 / 36

12 Rounding toward zero in R + Let x R + fl(x) be the rounding toward zero of x in F Equivalent to a truncation The rounding is less or equal to the exact number: x R +, fl(x) x fl(x) F x F S. Graillat (Univ. Paris 6) Dot products in finite fields 8 / 36

13 Rounding toward zero in R + Let x R + fl(x) be the rounding toward zero of x in F Equivalent to a truncation The rounding is less or equal to the exact number: The rounding error is nonnegative: x R +, fl(x) x x R +, x fl(x) 0 fl(x) F x F S. Graillat (Univ. Paris 6) Dot products in finite fields 8 / 36

14 Error-free Transformations (EFT) Problem : the result of a floating-point operation is generally not representable by a floating-point numbers. Solution: Error-free transformations non-evaluated sum of two floating-point numbers the floating-point result of the operation the rounding error (which is representable in F in our cases) For a, b F N and {+, }, which is mathematically true. a b = fl(a b) + e, with e F, S. Graillat (Univ. Paris 6) Dot products in finite fields 9 / 36

15 Error-free Transformations for the product (1/2) For a, b, c F, FMA(a, b, c) is the rounding of a b + c Algorithm 1 (EFT for the product of two floating-point numbers) function [x, y] = TwoProductFMA(a, b) x = fl(a b) y = FMA(a, b, x) The FMA is now included in the IEEE standard S. Graillat (Univ. Paris 6) Dot products in finite fields 10 / 36

16 Error-free Transformations for the product (2/2) Theorem 1 Let a, b F N and x, y F such that Then [x, y] TwoProductFMA(a, b) ab = x + y, x = fl(ab), 0 y < u.ufp(x), 0 x ab Algorithm TwoProductFMA requires 2 flops. ab x y 2M M 0 2l l = log 2(p) S. Graillat (Univ. Paris 6) Dot products in finite fields 11 / 36

17 Binary euclidean division (1/2) For a, d F N, d 0, the euclidean division of a by d is a = qd + r, 0 r < d For a F N and σ = 2 k, σ a, one defines Algorithm 2 (Split of a floating-point numbers) function [x, y] = ExtractScalar(σ, a) q = fl(σ + a) x = fl(q σ) y = fl(x a) fl is rounding toward zero Algorithm first proposed by Rump, Ogita and Oishi in rounding to the nearest S. Graillat (Univ. Paris 6) Dot products in finite fields 12 / 36

18 Binary euclidean division (2/2) Theorem 2 Let a F N, σ = 2 k, σ a and x, y F such that [x, y] ExtractScalar(σ, a) Then a = x + y, 0 y < u σ, 0 x a, x uσn Algorithm ExtractScalar requires 3 flops. Remark: a = x + y = x uσ + r, x N, 0 r < uσ q x a y σ = 2 k uσ 0 S. Graillat (Univ. Paris 6) Dot products in finite fields 13 / 36

19 Computation of dot products Two different approaches a b = N a i b i (mod p) i=1 S. Graillat (Univ. Paris 6) Dot products in finite fields 14 / 36

20 Computation of dot products Two different approaches First method: a b = N a i b i (mod p) i=1 λ(p 1) < 2 M 1 with λ N S. Graillat (Univ. Paris 6) Dot products in finite fields 14 / 36

21 Computation of dot products Two different approaches First method: a b = N a i b i (mod p) i=1 λ(p 1) < 2 M 1 with λ N Second method: p 1 < 2 M 1 but N < 2 M/2 In double, the maximal vector size are 2 53/ S. Graillat (Univ. Paris 6) Dot products in finite fields 14 / 36

22 Computation of dot products First method S. Graillat (Univ. Paris 6) Dot products in finite fields 15 / 36

23 Computation of dot products: first method Assumption : λ(p 1) < 2 M 1 Consequences : The sum of λ elements of the field can still be stored in the mantissa We can delay the reduction modulo p up to λ summations S. Graillat (Univ. Paris 6) Dot products in finite fields 16 / 36

24 Computation of dot products: first method Assumption : λ(p 1) < 2 M 1 Consequences : The sum of λ elements of the field can still be stored in the mantissa We can delay the reduction modulo p up to λ summations Jean-Guillaume Dumas: λ(p 1) 2 < 2 M S. Graillat (Univ. Paris 6) Dot products in finite fields 16 / 36

25 First method: principle a i b i a i < 2 l+1 b i < 2 l+1 a i b i 2M M 0 2l l = log 2(p) Let l = log 2 (p) S. Graillat (Univ. Paris 6) Dot products in finite fields 17 / 36

26 First method: principle a i b i a i < 2 l+1 b i < 2 l+1 a i b i 2M M 0 2l l = log 2(p) Let l = log 2 (p) = ufp(p) := 2 l p (ufp(p) =most significant bit of p) S. Graillat (Univ. Paris 6) Dot products in finite fields 17 / 36

27 First method: principle a i b i a i < 2 l+1 b i < 2 l+1 a i b i 2M M 0 2l l = log 2(p) Let l = log 2 (p) = ufp(p) := 2 l p (ufp(p) =most significant bit of p) p 3 prime so: ufp(p) < p < 2.ufp(p) i.e. 2 l < p < 2 l+1 Remarks: x [0, 2 l+1 1] F, { 0 x 2 l = x Z/pZ 2 l < x < 2 l+1 = x 2 l Z/pZ 0 x Z/pZ 2 l x 2 l Z/pZ 2 l+1 0 x 2 l 2 l x < 2 l+1 S. Graillat (Univ. Paris 6) Dot products in finite fields 17 / 36

28 First method: principle TwoProductFMA = a i b i = h + r a i b i 2M M 0 2l + 2 l + 1 = log 2 (p) + 1 S. Graillat (Univ. Paris 6) Dot products in finite fields 18 / 36

29 First method: principle TwoProductFMA = a i b i = h + r h a i b i r 2M M 0 2l + 2 l + 1 = log 2 (p) + 1 S. Graillat (Univ. Paris 6) Dot products in finite fields 18 / 36

30 First method: principle TwoProductFMA = a i b i = h + r α h a i b i β r 2M M 0 2l + 2 l + 1 = log 2 (p) + 1 S. Graillat (Univ. Paris 6) Dot products in finite fields 18 / 36

31 First method: principle TwoProductFMA = a i b i = h + r a i b i α h β r 2M M 0 2l + 2 l + 1 = log 2 (p) + 1 After splitting with ExtractScalar: h = α + β with 0 α/2 l+1, β < 2 l+1 S. Graillat (Univ. Paris 6) Dot products in finite fields 18 / 36

32 First method: principle TwoProductFMA = a i b i = h + r a i b i α h β r 2M M 0 2l + 2 l + 1 = log 2 (p) + 1 After splitting with ExtractScalar: h = α + β with 0 α/2 l+1, β < 2 l+1 We accumulate α/2 l+1 Z/pZ or α/2 l+1 2 l Z/pZ We remember the number n α of added 2 l S. Graillat (Univ. Paris 6) Dot products in finite fields 18 / 36

33 First method: principle TwoProductFMA = a i b i = h + r a i b i α h β r 2M M 0 2l + 2 l + 1 = log 2 (p) + 1 After splitting with ExtractScalar: h = α + β with 0 α/2 l+1, β < 2 l+1 We accumulate α/2 l+1 Z/pZ or α/2 l+1 2 l Z/pZ We remember the number n α of added 2 l Similar for β: β Z/pZ or β 2 l Z/pZ n β := number of correction of 2 l for β S. Graillat (Univ. Paris 6) Dot products in finite fields 18 / 36

34 First method: final computation a b = = N a i b i i=1 N α i + i=1 N β i + i=1 N i=1 r i = (α i /2 l+1 2 l ) + α i /2 l+1 + (β i 2 l ) + β i + n α N n α n β N n β N + (n α + n β ) 2 l r i S. Graillat (Univ. Paris 6) Dot products in finite fields 19 / 36

35 First method: final computation a b = = N a i b i i=1 N α i + i=1 N β i + i=1 N i=1 r i = (α i /2 l+1 2 l ) + α i /2 l+1 + (β i 2 l ) + β i + n α N n α n β N n β N + (n α + n β ) 2 l r i λ(p 1) < 2 M 1 = summation by bundle of λ numbers and then reduction mod p S. Graillat (Univ. Paris 6) Dot products in finite fields 19 / 36

36 Performances On Itanium2 With FMA In double precision (p 1 < ) Comparison with GMP S. Graillat (Univ. Paris 6) Dot products in finite fields 20 / 36

37 First method: Performances on Itanium2 (1/4) Figure: Comparison with GMP: time=f ( p [2 23, 2 52 ] ), for N = 10 5 S. Graillat (Univ. Paris 6) Dot products in finite fields 21 / 36

38 First method: Performances on Itanium2 (2/4) Figure: Comparison with GMP: time=f (N, log 2 (p)) GMP on the top S. Graillat (Univ. Paris 6) Dot products in finite fields 22 / 36

39 First method: Performances on Itanium2 (3/4) Figure: Surface: ratio=time(gmp)/time(algo) = f (N, log 2 (p)) S. Graillat (Univ. Paris 6) Dot products in finite fields 23 / 36

40 First method: Performances on Itanium2 (4/4) Figure: ratio=time(gmp)/time(algo) = f (N, log 2 (p)) S. Graillat (Univ. Paris 6) Dot products in finite fields 24 / 36

41 Computation of dot products Second method S. Graillat (Univ. Paris 6) Dot products in finite fields 25 / 36

42 Computation of dot products: second method Assumption : p 1 < 2 M 1 and N < 2 M/2 S. Graillat (Univ. Paris 6) Dot products in finite fields 26 / 36

43 Computation of dot products: second method Assumption : p 1 < 2 M 1 and N < 2 M/2 Idea : Split the number with a representation with only half the mantissa Sum them without error Reduction modulo p only at the end S. Graillat (Univ. Paris 6) Dot products in finite fields 26 / 36

44 Computation of dot products: second method Assumption : p 1 < 2 M 1 and N < 2 M/2 Idea : Split the number with a representation with only half the mantissa Sum them without error Reduction modulo p only at the end Use ExtractScalar to get: s 1 = M 2 i [1, N], a i b i = α i + β i + γ i + δ i = A i 2 M+s 1 + B i 2 M + C i 2 s 1 + D i N a b = 2 M+s 1 A i + 2 M i=1 N B i + 2 s 1 i=1 i=1 i=1 N N C i + D i (mod p) S. Graillat (Univ. Paris 6) Dot products in finite fields 26 / 36

45 Second method: principle of the splitting of a i b i (1/2) 2M 3M/2 M M/2 0 S. Graillat (Univ. Paris 6) Dot products in finite fields 27 / 36

46 Second method: principle of the splitting of a i b i (1/2) a i b i 2M 3M/2 M M/2 0 S. Graillat (Univ. Paris 6) Dot products in finite fields 27 / 36

47 Second method: principle of the splitting of a i b i (1/2) x a i b i y 2M 3M/2 M M/2 0 S. Graillat (Univ. Paris 6) Dot products in finite fields 27 / 36

48 Second method: principle of the splitting of a i b i (1/2) α x a i b i β y 2M 3M/2 M M/2 0 S. Graillat (Univ. Paris 6) Dot products in finite fields 27 / 36

49 Second method: principle of the splitting of a i b i (1/2) a i b i α x β y β ɛ 2M 3M/2 M M/2 0 S. Graillat (Univ. Paris 6) Dot products in finite fields 27 / 36

50 Second method: principle of the splitting of a i b i (1/2) a i b i α x β y β ɛ γ δ 2M 3M/2 M M/2 0 S. Graillat (Univ. Paris 6) Dot products in finite fields 27 / 36

51 Second method: principle of the splitting of a i b i (1/2) a i b i α x β y β ɛ γ δ 2M α 3M/2 β M γ M/2 δ 0 S. Graillat (Univ. Paris 6) Dot products in finite fields 27 / 36

52 Second method: principle of the splitting of a i b i (1/2) a i b i α x β y β ɛ γ δ 2M α 3M/2 β M γ M/2 δ 0 a i b i = α + β + γ + δ S. Graillat (Univ. Paris 6) Dot products in finite fields 27 / 36

53 Second method: principle of the splitting of a i b i (2/2) Split 4 vectors of N < 2 M/2 elements with at most M/2 bits 2M 3M/2 M M/2 0 S. Graillat (Univ. Paris 6) Dot products in finite fields 28 / 36

54 Second method: principle of the splitting of a i b i (2/2) Split 4 vectors of N < 2 M/2 elements with at most M/2 bits α β γ δ 2M 3M/2 M M/2 0 S. Graillat (Univ. Paris 6) Dot products in finite fields 28 / 36

55 Second method: principle of the splitting of a i b i (2/2) Split 4 vectors of N < 2 M/2 elements with at most M/2 bits α β γ δ α β γ δ 2M 3M/2 M M/2 0 S. Graillat (Univ. Paris 6) Dot products in finite fields 28 / 36

56 Second method: Results Final results: a b = N N N α i + β i + γ i + i=1 i=1 i=1 i=1 N δ i (mod p) Total cost: 16N + O(1) flops S. Graillat (Univ. Paris 6) Dot products in finite fields 29 / 36

57 Second method: Performances on Itanium2 (1/3) Figure: Comparison with GMP: time=f (N, log 2 (p)) GMP on the top S. Graillat (Univ. Paris 6) Dot products in finite fields 30 / 36

58 Second method: Performances on Itanium2 (2/3) Figure: Surface: ratio=time(gmp)/time(algo) = f (N, log 2 (p)) S. Graillat (Univ. Paris 6) Dot products in finite fields 31 / 36

59 Second method: Performances on Itanium2 (3/3) Figure: ratio=time(gmp)/time(algo) = f (N, log 2 (p)) S. Graillat (Univ. Paris 6) Dot products in finite fields 32 / 36

60 Comparison of the two methods S. Graillat (Univ. Paris 6) Dot products in finite fields 33 / 36

61 Comparison of the two methods Figure: time(method 2 ) time(method 1 ) = f (N, log 2 (p)) S. Graillat (Univ. Paris 6) Dot products in finite fields 34 / 36

62 Conclusion and future work Conclusion: Two efficient algorithms for computing dot product Efficient algorithms compared to GMP Use of error-free transformations in rounding toward zero Future work: Second method with a splitting in 3 parts (with N < 2 M/3 ) Extension to Galois fields GF (2 n ) Use of longlong library Use of RNS techniques Parallelisation of the algorithms for GPU S. Graillat (Univ. Paris 6) Dot products in finite fields 35 / 36

63 The end Thank you for your attention S. Graillat (Univ. Paris 6) Dot products in finite fields 36 / 36

Fast dot product over finite field

Fast dot product over finite field Jérémy JEAN Jean.Jeremy.Jean@gmail.com February 2010 Master thesis report 1 LIP6 Paris, France Stef Graillat Kungliga Tekniska Högskolan (KTH) Stockholm, Sweden Johan