Theory and practice for hash functions

Transcription

1 Theory and practice for hash functions Bart Preneel eu Title of Presentation Katholieke Universiteit it it Leuven - COSIC firstname.lastname@esat.kuleuven.be Cambridge, 1 February 2012 Insert presenter logo here on slide master

2 Hash functions X.509 Annex D RIPEMD-160 MDC-2 SHA-256 MD2, MD4, MD5 SHA-512 SHA-1 SHA-3 This is an input to a cryptographic hash function. The input is a very long string, that is reduced by the hash function to a string of fixed length. There are additional security conditions: it should be very hard to find an input hashing to a given value (a preimage) or to find two colliding inputs (a collision). h 1A3FD4128A198FB3CA

3 Applications short unique identifier to a string digital signatures data authentication one-way function of a string protection of passwords micro-payments confirmation of knowledge/commitment t pseudo-random string generation/key derivation entropy extraction construction of MAC algorithms, stream ciphers, block ciphers, 2005: 800 uses of MD5 in Microsoft Windows 3

4 Bridging theory and practice? 4

5 Agenda Definitions Iterations (modes) Compression functions Hash functions SHA-3 5

6 Security requirements (n-bit result) preimage 2 nd preimage collision? x??? h h h h h h(x) () h(x) () = h(x ) h(x) () = h(x ) 2 n 2 n 2 n/2 6

7 Length extension length extension: if one knows h(x), easy to compute h(x y) without knowing x or IV IV f H 1 f H 2 f H 3 = h(x) x 1 x 2 x 3 IV H 1 H 2 H 3 H 4 = h(x y) f x 1 x 2 x 3 y solution: output transformation f f f IV f H 1 f H 2 f H 3 f g x 1 x 2 x 3 x 4 7

8 Informal definitions: NIST requirements for SHA-3 One Way Hash Function (OWHF) preimage resistance (2 n ) 2 nd preimage resistance (2 n-l for 2 L -bit message) Collision Resistant Hash Function (CRHF): collision resistant Resistance to length extension Properties scale well with truncationti HMAC-PRF indistinguishable up to 2 n/2 queries Secure in randomized hashing mode 8

9 Brute force (2 nd ) preimage multiple target second preimage (1 out of M): if one can attack 2 t simultaneous targets, the effort to find a single preimage is 2 n-t multiple target second preimage (M out of M): time-memory memory trade-off with Θ(2 n ) precomputation and storage Θ(2 2n/3 ) time per (2 nd ) preimage: Θ(2 2n/3 ) [Hellman 80] full cost per (2 nd ) preimage from Θ(2 n ) to Θ(2 2n/5 ) [Wiener 02] (if Θ(2 3n/5 )t targets t are attacked) answer: randomize hash function with a parameter S (salt, key, spice, ) 9

10 Formal definition: (2 nd ) preimage resistance Notation: Σ = {0,1}, l(n)>n A one-way hash function (OWFH) h is a function with domain D=Σ l(n) and range R=Σ n that satisfies the following conditions: Pre: let x be selected uniformly in D and let M be an adversary that on input h(x) uses time t and outputs M(h(x)) D. For each adversary M, Pr x D { h(m(h(x)))=h(x) } < ε. Here the prob. is also taken over the random choices of M. Sec: letxx be selected uniformly in D=ΣΣ l(n) and let M' be an adversary who on input x uses time t and outputs x' D with x' x. For each adversary M', Pr x D {h(m'(x))=h(x) (x)) h(x) } < ε. Here the prob. is taken over the random choices of M'. 10

11 Formal definitions: collision resistance A collision-resistant hash function (col) h is a function with domain D=Σ l(n) and range R=Σ n that that satisfies the following conditions: for each collision string finder F that uses time t and outputs either? or a pair x, x' Σ l(n) with x' x such that h(x )=h(x) we have that Pr{ F(H)? } < ε. Here the prob. is taken over the random choices of F. Is this correct? No! For every h there are many (short) colliding strings, and so there exist many very simple collision string finders F that output a collision with probability 1 11

12 Formal definitions: collision resistance A collision-resistant hash function family (CRHF) H is a function family {h S } with domain D=Σ l(n) and range R=Σ n and indexed by a key S that that satisfies the following conditions: let F be a collision string finder that on input S Σ s uses time t and outputs either? or a pair x, x' Σ l(n) with x' x such that h S (x )=h S (x). For each F, Pr S { F(H)? } < ε. Here the prob. is also taken over the random choices of F. 12

13 Formal definitions - continued for collision resistance: formalization requires a family, but see [Stinson 06], [Rogaway 06]: formalizing i human ignorance for (2nd) preimage resistance, one can choose the challenge (x) and/or the key (S) that selects the function, resulting in 3 flavors [Rogaway-Shrimpton 04]: 1. random challenge, random key (Pre and Sec) 2. random key, fixed challenge (epre and esec - everywhere) (esec=uowhf) 3. fixed key, random challenge (apre and asec - always) complex relationship (see figure on next slide) 13

14 Relations between security properties [Rogaway-Shrimpton 04] [Andreeva-Stam 10] Even if Coll esec/pre: bound always 2 n/2 << 2 n Q Easy to prove d Relevant in practice rpre How does epre (theoretically easy to prove) relate to dpre (practically relevant)? For sufficiently Col secure H, epre implies dpre (if the message space has sufficient Rényi 2 entropy > Y ) 14

15 Collision resistance more complex to formalize hard to achieve in theory [Simon 98] one cannot derive collision resistance from general preimage resistance (there exists no black box reduction) hard to achieve in practice requires double output length 2 n/2 versus 2 n many attacks Good news: not always necessary Even better news: rarely necessary 15

16 How to solve collision problem in digital signatures? SK A Message m UOWHF TCR $ Random r r {0,1} k esec Signature Sig SK A ( r h r (m)) Hash function is only chosen after the message has been committed to, so collisions for a particular hash function are useless If two messages m, m are found with the same signature, the signer must have cheated by first choosing r and then finding a collision Fine in theory, but will this work in practice? 16

17 Courts and second preimage resistance 17

18 Pseudo-random function computationally indistinguishable from a random function $ $ Adv prf h = Pr [ K K: A h K (.) 1] - Pr [ f RAND(m,n): A f 1] RAND(m,n): set of all functions from m-bit to n-bit strings K h f?or?? D This concept makes only sense for a function with a secret key 18

19 Indifferentiability from a random oracle or PRO property [Maurer+04] variant of indistinguishability appropriate when distinguisher has access to inner component (e.g. building block of a hash function) Simulator S, distinguisher DAdv D, PRO (H,S) is small H (hash function) FIL RO VIL RO S? or? D!! But [Ristenpart-Shacham- Shrimpton 11]: not always sufficient for ROM security 19

20 Properties in practice other properties are needed: near-collision i resistance partial preimage resistance (most of input known) multiplication freeness MAC property Which properties are relevant? How to formalize these requirements and the relation between all these properties? Keyed or unkeyed? Who chooses the key? What about the salt? 20

21 21

22 Iterationti (mode of compression function) 22 22

23 Hash function: iterated structure IV H 1 H 2 H f f f H 3 f g x 1 x 2 x 3 x 4 Fix IV Unambiguous padding Suffix-free encoding [Merkle 89-Damgård 89] f is collision resistant h is collision resistant Are the roles of the 2 inputs of the compression function really equivalent? 23

24 Attacks on MD-type iterations multi-collision attack and impact on concatenation [Joux 04] ] long message 2 nd preimage attack [Dean-Felten-Hu'99], [Kelsey-Schneier 05] Sec security degrades lineary with number 2 t of message blocks hashed: 2 n-t+1 +t2 n/2+1 appending the length does not help! herding attack [Kelsey-Kohno 06] reduces security of commitment using a hash function from 2 n on-line 2 n-t + precomputation ti (n+t)/2 + storage 2 t 24

25 Improving MD iteration salt + output transformation + counter + wide pipe IV salt salt salt salt salt H 1 H 2 H 3 f 2n f f f 2n 2n 2n 2n n g x 1 x 2 x 3 x x 25

26 Improving MD iteration degradation with use: salting (family of functions, randomization) or should a salt be part of the input? PRO/Indif: strong output transformation g also solves length extension long message 2 nd preimage: preclude fix points counter f f i [Biham-Dunkelman 07] multi-collisions, herding: avoid breakdown at 2 n/2 with larger internal memory: known as wide pipe e.g., extended MD4, RIPEMD, [Lucks 05] 26

27 Some Domain Extender Security Results [Andreeva-Neven-P-Shrimpton07], [Andreeva-Mennink-P11] Col Sec asec esec Pre apre epre Indif 1. SMD [M90, D90] Linear hash [BR97] XOR Linear hash [BR97] Shoup's hash [Sho00] BCM [AP08] ? 6. Prefix-free MD [CDMP05] Randomized hash [HK06] HAIFA [BD06] Enveloped MD [BR06] Str. Merkle Tree [Mer80] ? 11. Linear Tree [BR97] ? 12. ROX [ANPS07] ? 27

28 Preservation which constructions preserve which properties? or is it ok if we can prove security for the iteration? assumptions on compression function f for PRO/Indif f is preimage aware (MD) f is FIL-PRO (variant of MD) Impact of results in ideal model? Can tight reductions be made? Is it meaningful to preserve Pre? 28

29 Compression function 29 29

30 Block cipher (E K ) based: single block length Davies-Meyer Miyaguchi-Preneel H H i-1 E i x i E H i x i H i-1 output length = block length m; rate 1; 1 key schedule per encryption 12 secure compression functions (in ideal cipher model) lower bounds: collision 2 m/2, (2nd) preimage 2 m [Preneel+ 93], [Black-Rogaway-Shrimpton 02], [Duo-Li 06], [Stam 09], 30

31 Blake Davies-Meyer + double pipe internally H i-1 E H i x i 31

32 Permutation (π) based: sponge x 1 x 2 x 3 x 4 h1 h2 H1 0 π π π π π π π π H2 0 absorb buffer squeeze Examples: Panama, RadioGatun Keccak (no buffer) Generalization called Parazoa JH, Cubehash, Fuge, Grindahl, Hamsi, Luffa 32

33 Permutation (π) based parazoa JH small permutation Grøstl x i H1 i-1 H1 i x i π 1 H2 i-1 π i H2 i H i-1 π 2 H i 33

34 MDC-2 and MDC-4 [Brachtl+ 89] E 1,E 2 E 1, E 2 n-bit block cipher with n-bit key

35 Some security results [Steinberger 07,Mennink+ 11]

36 2n-to-n bit compression function with 3 permutations Earlier work: [RogawaySteinberger 08, Stam 08, Steinberger 10] Slide credit: Bart Mennink

37 Slide credit: Bart Mennink 2n-to-n bit compression function with 3 permutations and XORs [Mennink+ 11]

38 Iteration modes and compression functions security of simple modes well understood powerful tools available analysis of slightly more complex schemes very difficult need strong assumptions imho not justified is there a point to build larger hash functions from small building blocks? why not construct large permutations (e.g. Keccack = 1600 bits)? what if the building block is not ideal (the larger they are the more risky)? MD versus sponge? 38

39 Hash function constructions 39 39

40 Hash function history 101 DES RSA HA ARDWAR RE SOFTWA ARE AES single block length double block length permu- tations ad hoc schemes security reduction for factoring, DLOG, lattices MD2 MD4 MD5 Dedicated SHA-1 SNEFRU RIPEMD-160 SHA-2 Whirlpool 2010 SHA-3 40

41 Performance of hash functions [Bernstein-Lange] (cycles/byte) AMD Intel Pentium D 2992 MHz (f64) MD4 MD5 SHA-1 RMD- DES SHA- SHA- 160 (estimated) Whirlpool AES AES- hash 41

42 The complexity of collision attacks brute force: 1 million PCs (1 year) or US$ 100,000 hardware (4 days) MD4 MD5 SHA-0 SHA-1 Brute force

43 MD5 advice (RIPE since 92, RSA since 96): stop using MD5 largely ignored by industry until 2009 (click on a cert...) 43

44 Rogue CA attack [Sotirov-Stevens-Appelbaum-Lenstra-Molnar-Osvik-de Weger 08] request user cert; by special Self-signed collision this results in a fake CA root key cert (need to predict serial number + validity period) CA1 CA2 Rogue CA impact: rogue CA that can issue certs that are trusted by all browsers User1 User2 User x 6 CAs have issued certificates signed with MD5 in 2008: Rapid SSL, Free SSL (free trial certificates offered by RapidSSL), TC TrustCenter AG, RSA Data Security, Verisign.co.jp 44

45 log 2 complexity [Wang+ 04] [Wang+ 05] [Sugita+ 06] 20 Most attacks unpublished/withdrawn 10 0 SHA-1 [Mendel+ 08] [Manuel+ 09] [McDonald+ 09] SHA-1 prediction: collision for SHA-1 in the next months 45

46 46

47 SHA

48 NIST AHS competition (SHA-3) SHA-3 must support 224, 256, 384, and 512-bit message digests, and must support a maximum message length of at least 2 64 bits Call: 02/11/07 Deadline (64): 31/10/08 Round 1 (51): 9/12/08 Round 2 (14): 24/7/09 60 Final (5): 9/12/10 0 Standard: Q Q4/08 Q3/09 Q4/10 Q3/12 round d1 round 2 final 48

49 The candidates Slide credit: Christophe De Cannière 49

50 Preliminary cryptanalysis Slide credit: Christophe De Cannière 50

51 End of Round 1 candidates a Slide credit: Christophe De Cannière 51

52 Round 2 candidates a Slide credit: Christophe De Cannière 52

53 Properties: bits and bytes [Watanabe 10] 53

54 Security Analysis of the 2 nd Round SHA-3 Candidates [Andreeva-Mennink-P 10] 54

55 Security results for 256-bit versions l-state size m-bl. size Pre Sec Col Indif BLAKE Grøstl L JH Keccak Skein NIST requirements L L: # msg blocks E. Andreeva, A. Luykx and B. Mennink, "Provable Security of BLAKE with Non- Ideal Compression Function", eprint Archive, Report 2011/620. D. Chang, M. Nandi and M. Yung: Indifferentiability of the hash algorithm BLAKE, eprint Archive, Report 2011/623. E. Andreeva, B. Mennink and B. Preneel, "Security Analysis and Comparison of the SHA-3 Finalists BLAKE, Grøstl, JH, Keccak, and Skein ", Slide credit: Elena Andreeva 55

56 Security Results for 512-bit Versions l-state size m-bl. size Pre Sec Col Indif BLAKE Grøstl L JH Keccak Skein NIST requirements L L: # msg blocks E. Andreeva, A. Luykx and B. Mennink, "Provable Security with Non-Ideal Compression Function", eprint Archive, Report 2011/620. D. Chang, M. Nandi and M. Yung: Indifferentiability of the hash algorithm BLAKE, eprint Archive, Report 2011/623. E. Andreeva, B. Mennink and B. Preneel, "Security Analysis and Comparison of the SHA-3 Finalists BLAKE, Grøstl, JH, Keccak, and Skein ", Slide credit: Elena Andreeva

57 Open questions standard model security proofs (other than Col preservation) improved Indif bounds JH-512 optimality for Sec and Pre security in the ideal model Slide credit: Elena Andreeva 57 57

58 SWIFFTX [Arbitman-Dogon-Lyubashevsky-Micciancio- Peikert-Rosen 08] compression function: SWIFFT: FFT-like operation from (Z 32 2 ) 64 to Z sandwich: 3xSWIFFT - S-boxes - 1xSWIFFT asymptotic proof of security: it can be formally proved that finding a collision in a randomly- chosen compression function from the SWIFFTX family is at least as hard as finding short vectors in cyclic/ideal lattices over the ring Z[α]/(α n +1) is in the worst case. note: SWIFFT mapping is linear and some heuristics are needed to kill the linearity speed: 57 cpb 58

59 FSB [Augot-Finiasz-Gaborit-Manuel-Sendrier 08] compression function: multiplication of vector of Hamming weight w with a truncated quasi-cyclic binary matrix can be interpreted as a syndrome computation of error pattern with weight w MD iteration with Whirlpool as output transformation security can be reduced to: (Computational Syndrome Decoding) Given a binary r x n matrix H, a word s {0,1} r and an integer w > 0, find a word e {0,1} n of Hamming weight w such that eh T = s. (Codeword Finding) Given a binary r x n matrix H and an integer w > 0, and a non-zero word e {0,1} n of Hamming weight w with an all zero H- syndrome. speed 324 cycles/byte RFSB: cycles/byte on Core 2 Q9550 [Bernstein-Lange 11] 59

60 Hash functions: conclusions attacks: cryptographic meltdown but not dramatic for most applications half-life of a hash function is < 1 year solid progress in theory, but still quite a large gap between theory and practice nirwana: efficient i hash h functions with security reductions 60