New Techniques for Cryptanalysis of Cryptographic Hash Functions. Rafael Chen

Transcription

1 New Techniques for Cryptanalysis of Cryptographic Hash Functions Rafael Chen

2

3 New Techniques for Cryptanalysis of Cryptographic Hash Functions Research Thesis Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Rafael Chen Submitted to the Senate of the Technion Israel Institute of Technology Nissan 5771 Haifa April 2011

4

5 The research thesis was done under the supervision of Prof. Eli Biham in the Computer Science Department. I dearly thank my advisor Eli Biham for his guidance, support and invaluable discussions we had during my Ph.D. studies. It is a pleasure to thank Adi Shamir for many discussions and advices which enhanced my knowledge in different areas. Many thanks to my dear friend Dag Arne Osvik for his invaluable assistance whenever I needed. I thank Orr Dunkelman for his advices and support and especially for his willingness to help, and also Yaniv Carmeli for all his help. Last but not least, thanks to Mark Silberstein for providing me the required computing resources for this research. Special thanks are due to my wife Dorit and my children Inbal, Ofir, Nitzan and Rotem for their love, encouragement, patience and support during all these years. The generous financial support of the Technion is gratefully acknowledged. This research was also partially supported by the Israel MOD research and technology unit.

6

7 Contents Abstract 1 Abbreviations and Notations 4 1 Introduction Historical Overview Definitions and Required Properties of Hash Functions Applications of Hash Functions Constructions of Hash Functions Tree Constructions The Merkle-Damgård Construction The Compression Function of Tree and Merkle-Damgård Constructions Attacks on Hash Functions Birthday Attacks Meet-in-the-Middle Preimage Attacks Attacks on the Merkle-Damgård Construction Our Contributions Results Outline of this Thesis Descriptions of MD4, SHA-0 and SHA Merkle-Damgård Construction and Davies-Meyer Construction The Compression Function of MD The Compression Functions of SHA-0 and SHA A Further Notation of a Round Function C[ ] i

8 3 Our Representation of Differential Cryptanalysis of Hash Functions The General Idea Duo Bits and Duo Numbers Active Bits, Inactive Bits, Duo Bits and Duo Numbers Operations on Duo Bits and Duo Numbers Differential Properties of the Operations of the Round Function Differential Properties of Addition Modulo Differential Properties of the Rotation Operation Differential Properties of the F r Function The Usage of Duo Numbers in Our Attacks An Efficient Verification of a Prediction: Using Only a Single Message in the Analysis A Representation of Characteristics by a Set of Equations A Conversion of Predictions to a Set of Equations Advantages of Using a Set of Equations in a Search for a Right Pair Introduction to Differential Cryptanalysis of MD4 and SHA Differential Cryptanalysis of MD Dobbertin s Attack on MD An Analysis of Dobbertin s Attack by Duo Numbers Wang et al s Attack Differential Cryptanalysis of SHA Local Collision Sequences Characteristics and Disturbance Vectors Selecting a Disturbance Vector for the Attack The Chaining Differences Transition Graph A Characteristic of a Single-Block Attack of SHA Constructing Pairs for the Attack Complexity Evaluation The Neutral Bits Technique A Collision Attack Using a 2-Neutral Set Finding a 2-Neutral Set Finding Neutral Bits and Optimizing a Pair ii

9 5.2.2 Finding Neutral Pairs and 2-Neutral Sets Other Numbers of Rounds 91 7 The Multi-Block Technique Solving Initial State Incompatibility by an Additional Block Two-block Attacks A Search Procedure of Disturbance Vectors For a Twoblock Attack Characteristics For a Two-Block Attack A Two-Block Collision of SHA-0 Reduced to 50 Rounds Complexity Evaluation Collisions with More than Two Blocks Revisiting Characteristics and Disturbance Vectors Joux s Four-block Collision of SHA Complexity Evaluation Attacks on Reduced Versions of SHA Selecting a Disturbance Vector for the Attack A Collision of 34-Round SHA Near-Collisions of More Rounds A Collision of 36-Round SHA Techniques to Resolve the Consecutive Disturbances Problem in the IF Rounds A Generalized Test for Conformance Early Test for Conformance Changing the Initial Values A Collision of 36-Round SHA Selecting a Path in the Transition Graph of SHA Constructing a Two-Edge Path with the Same Disturbance Vector A Collision of SHA-1 Reduced to 40 Rounds Strength of Reduced Versions of SHA-1 with More Rounds Revisiting Correction of Consecutive Disturbances iii

10 9 The Second-Order Differential Technique The Characteristics of the Attack The Second-Order Differential Technique Conversion of the Characteristic Ω to a Set of Equations G A Measure of Conformance G(M, A), and a Second- Order Characteristic ω The Collision Search Applications of Second-Order Characteristics First-Order Characteristics Revisited Examples of Applications of Second-Order Characteristics A Representation of the Predicted Difference of States of Conformance ω G as a Set of Equations Non-Sequential Collision-Search Generating Second-Order Characteristics with High Probability Advantages of Using a Set of Equations in a Collision-Search The Complexity of the Attack Confirmation of the Technique A Attack on SHA-1: The Characteristic Ω and the Set of Equations G 171 Abstract in Hebrew iv

11 List of Tables 1.1 Our Main Results and Prior and Independent Results Functions and Constants of MD MD4 Rotations Functions and Constants of SHA-0 and SHA Duo Bits and Their Standard Representation and Difference The Operations (a) AND, (b) OR, (c) XOR and (d) NOT, on 1-bit Duo Numbers The Outputs of IF(a,b,c) with all Possible Values of a,b and c IF(ä, b, c), MAJ(ä, b, c), XOR(ä, b, c) on Duo Numbers Addition of Duo Numbers The Rotation Operation ä l on Duo Numbers Duo Numbers ä r+1 in which a r+1 = Duo Numbers ä r+1 in which a r+1 = Three Examples for Differential Properties of ä 5 in which a = The Subtraction Differences Ω W of Dobbertin s Attack The Differences s 12 and s 20 of the Inner Part The Characteristic of Dobbertin s Attack Rounds A Representation of Dobbertin s Characteristic by Duo Numbers The Characteristic of Wang et al s Attack on MD A Representation of Wang et al s Characteristic by Duo Numbers A Pattern of Differences that Creates a Local Collision v

12 4.8 Characteristics of Local Collision Sequences with a Disturbance in bit 1 and F r = IF, MAJORITY and XOR Message and State Equations of the Characteristics of Table The Disturbance Vector A Characteristic of a Single-Block Attack on SHA Duo Numbers Characteristic of Table 4.11 (Rounds 0,...,39) Duo Numbers Characteristic of Table 4.11 (Rounds 40,...,79) The Set of Equations that Represents the Characteristic of Table The Set of Equations that Represents the Characteristic of Table A Disturbance Vector to Attack SHA-0 Extended to 82 Rounds The Pair M 1, M1 of Example The 2-Neutral-Set of Example Complexities of Attacks on Reduced and Extended Versions of SHA The Disturbance Vectors of 50-Round SHA A Characteristic of 50-Round SHA-0 (First Block) A Characteristic of 50-Round SHA-0 (Second Block) A Two-block Collision of 50-Round SHA The Intermediate Chaining Variables and Differences of the Two-block Collision of 50-Round SHA Probabilities and Complexities of the Two-Block Attack on 50-round SHA Chaining Differences and Their Compatibility with Disturbance Vectors The Disturbance Vectors of the Four-block Collision A Characteristic of a Four-Block Attack on SHA-0(First Block) A Characteristic of a Four-Block Attack on SHA-0 (Second Block) A Characteristic of a Four-Block Attack on SHA-0 (Third Block) A Characteristic of a Four-Block Attack on SHA-0 (Fourth Block) vi

13 7.13 A Four-block Collision of SHA The Intermediate Chaining Variables and Differences of the Four-block Collision of SHA Probabilities and Complexities of the Four-block Attack on SHA Local Collision Conditions for Rounds r,...,r + 5 with the XOR operation at rounds r +2, r +3, r Local Collision Conditions for Rounds r,...,r + 5 with the IF operation at rounds r +2, r +3, r Local Collision Conditions for Rounds r,...,r + 5 with the MAJ operation at rounds r +2, r +3, r A Characteristic of 34-Round Reduced SHA Collision of SHA-1 Reduced to 34 Round A Collision of Two Messages Written in ASCII Letters and Hashed by 34-Round SHA Two Examples of Partially Meaningful Messages that Collide Under 34-Round SHA A Characteristic of 36-Round Reduced SHA A Local Collision with Two Adjacent Disturbances A Two-block Collision of SHA-1 Reduced to 36 Rounds A Characteristic of 40-Round Reduced SHA-1 First Block A Characteristic of 40-Round Reduced SHA-1 Second Block Two-Block Collision of 40-Round SHA Probabilities and Complexities of the 40-Round SHA-1 Attack The Hamming Weights of Disturbance Vectors The Disturbance Vector of 53-round SHA The Disturbance Vector of 58-round SHA A Characteristic of 53-Round Reduced SHA A Characteristic of 58-Round Reduced SHA-1 First Block A Characteristic of 58-Round Reduced SHA-1 Second Block Probabilities and Complexities of the 58-Round SHA-1 Attack The First 20 Rounds of Ω The Second-Order Characteristic ω = (ω M,ω G,N) of Example vii

14 9.3 A Pair that Conforms to 63 Rounds (I(M 2 ) = 63) in which d(m 2 ) = A.1 The Chaining Equations G 1 h of Ω A.2 The Characteristic of the First Block Ω A.3 The Set of Equations G viii

15 List of Figures 1.1 Hash Functions Based on a Tree Construction Hash Functions based on Merkle-Damgård Construction Compression Functions Based on Block Ciphers A Birthday Attack that Creates Two Messages with Different Meanings and Same Hash Result Multi-Collision on Merkle-Damgård Construction Constructing Expandable Message by Using a Fix-point The Construction of Expandable Message in Kelsey Schneier s Attack The Compression Function of MD The Compression Function of SHA-0 and SHA The Idea of Differential Cryptanalysis of Hash Functions An Exploration of the Computed Data of Round r of SHA in Each of the Two Runs The Analyzed Differences of the Round Function of SHA The Analyzed Differences of the Round Function of MD An Adder of Duo Numbers Dobbertin s Attack Two Consecutive Disturbances in the IF Rounds The Chaining Differences Transition Graph of a Single-block Attack on SHA Using Intermediate Near-Collisions to Find Collisions with Two Blocks ix

16 7.2 The Search procedure of Disturbance Vectors for a Two-block Attack The Chaining Difference Path of a Two-block Attack of SHA-0 Reduced to 50 Rounds The Multi-Block Technique Using Intermediate Near-Collisions to Find Collisions The Modified Chaining Transition Graph Matching Disturbance Vectors to a Chaining Difference which Seems Incompatible TheComputationsandConditionsforMatchingh i 1 = to Disturbance Vectors of the Form 01000xxxxxxxxxxx The Four-block Chaining Differences Path of SHA The Transition Graph of 36-Round SHA The 3-block Attack Application of a Second-Order Characteristic to (a) a Pair (M,M ). (b) A message M The Collision-Search Algorithm First and Second Order Characteristics x

17 Abstract A cryptographic hash function H takes a message of an arbitrary length and produces an easy-to-compute message digest H(M) which has fixed, relatively short size. The message digest is used as the digital fingerprint of the message and it has to satisfy several properties. The most important property is collision-freeness, i.e., it should be difficult to find any two messages that have the same message digest. Two other important properties are: Preimage resistance, i.e., given a string s that has the length of a message digest it should be difficult to find a message M such that H(M) = s, and second-preimage resistance, i.e., given a message M 1 it should be difficult to find M 2 such that H(M 1 ) = H(M 2 ). An example for the importance of the collision-freeness property is demonstrated by digital signature schemes. In such a scheme the signer hashes a message, signs the message digest, and sends the message and signature to the receiver. The receiver hashes the message and verifies that the signature on the message digest is authentic. If the signature verification succeeds, the receiver concludes that the message is authentic, and if it fails he clearly concludes that the message is forged or modified. If the hash function is not collision-free, the sender may find two different messages with the same hash value. He can then sign and send one message along with the signature, andclaim later that hesent andsigned theother messagewiththe same signature. The receiver cannot disprove the claim since the signatures on both messages are valid and identical. Similarly, if the receiver holds such a pair of messages, say an innocently looking message and an offending message, and he succeeds to receive a signature on the former, he can claim that he received the signature of the latter, and the signer cannot disprove it. In cryptanalysis of cryptographic hash functions the required properties 1

18 of hash functions are studied. In case a property is shown wrong using some algorithm, the algorithm is called an attack and the function is considered broken. A widely known attack technique is differential cryptanalysis, which uses differences to attack a function. In this technique an attacker searches for a well chosen difference between the ciphertexts of two messages. If pairs of messages are selected with a particular difference, then the probability to find a pair with the well chosen ciphertexts difference is higher than in a random choice of pairs. Such predictions of the evaluation of differences from the plaintext through the intermediate data into the ciphertext are called characteristics. A characteristic defines the initial, intermediate and final differences, and the probabilities to receive these differences when a pair of messages (with the initial difference) is hashed. Differential cryptanalysis of hash functions aims at finding a characteristic with high probability, and at constructing an efficient algorithm that selects messages from which at least one follows the differences of the characteristic. Our contributions are in both aims. The multi-block technique is related to the first aim. It is based on our observation that characteristics of the compression function that predict near-collisions and pseudo-collisions may have higher probabilities than characteristics that predict collisions. Moreover, we identified a weakness of Merkle-Damgård construction that enables using these near and pseudocollisions. The technique suggests to build the collision from a path of several blocks, each leads to a new (non-zero) difference, where the last block leads to a collision. In many cases such a technique is more efficient than the formerly used attacks of a single block. Moreover, we show that using a two-block collision in which the first and second message blocks have the same difference, is typically the most efficient attack. We demonstrated this idea by finding a collision of SHA-1 reduced to 40 rounds. All published attacks on SHA-1 that we are aware of are based on these ideas. In order to find high-probability characteristics of the compression function, we analyze the differential properties of each operation that the round function uses. Along with heuristics assumptions we construct a first-order characteristic of the compression function, that predicts collision. For an efficient selection of messages that follow the characteristic, we developed two novel techniques: The neutral bits technique and the secondorder differential technique. The neutral bits technique eliminates the prob- 2

19 abilistic behavior of the characteristic up to some high round of the compression function (typically Round 30 in SHA-1). Hence, the complexity of the attack is affected by the probabilistic behavior after this round. The secondorder differential technique is based on our observation that differences with specific patterns may be used for a more efficient selection of messages that follows the differences of the characteristic. We call these patterns secondorder characteristics. The technique defines guidelines for the construction of a first-order characteristic such that second-order characteristics with high probability may be found. It also provides an efficient algorithm for selection of messages that uses the second-order characteristics. This algorithm uses the neutral bits technique along with other auxiliary techniques. Using these techniques we constructed a collision attack on SHA-1 with a complexity of 2 58 SHA computations which is the most efficient published attack on this function. These techniques were also crucial in finding the first collision of SHA-0, whose complexity was This collision consisted of four blocks, and used the neutral bits technique. 3

20 Abbreviations and Notations FIPS Federal Information Processing Standards MAC Message Authentication Code NIST National Institute of Standards and Technology NSA National Security Agency SHA Secure Hash Algorithm l A right shift of a 32-bit word by l positions l A left shift of a 32-bit word by l positions l A right rotation of a 32-bit word by l positions l A left rotation of a 32-bit word by l positions a,a A pair of words that corresponds to the data of two runs a The subtraction difference a a a The XOR difference a a ä A duo number A j r Bit j of the reduced state vector A at Round r C(h k 1,M k ) The compression function of H C[ ] The round function of the compression function C(h k 1,M k ) C[i,r](s i,w i,...,w r ) The Computation of s r+1 from s i and W i,...,w r F j r Bit j of the output of the bitwise function F at Round r h k The k th chaining value of the hash function H H(M) The message digest of M computed by the hash function H M A message M k The k th message block of M r An intermediate round of the compression function R The number of rounds in a compression function s r The state of the compression function at Round r Bit j of the expanded message word W at Round r W j r 4

21 Chapter 1 Introduction A cryptographic hash function is used to construct a short fingerprint of arbitrary data. If the data is altered after it is hashed then the fingerprint is no longer valid and the alteration can be detected. Cryptographic hash functions are used to assure the integrity of data, and as a tool for authentication. This research is concerned with the security of cryptographic hash functions. 1.1 Historical Overview Up to the 1970 s, handwritten signatures were the most common means to commit parties to contracts and agreements. When digital data started to replace handwritten and typewritten data, a need for equivalent means to handwritten signature arose. The alternative of a handwritten signature is a digital signature. The idea of a digital signature is that a signer binds a string s to a message M, that assures the source of the message, its integrity and has a property of non-repudiation. Once a signer provides a legitimate signature on M, everyone can verify it and the signer cannot claim later that it is forged. Digital signature schemes are constructed of two algorithms, one for signing and the other for verifying. The signer uses the signature algorithm with a secret key known only to himself, and produces his signature. The verifier uses a public verifying algorithm with a public key, that returns a value of true or false telling if the signature is legitimate or not. Digital signatures 5

22 in practice are applied to a relatively short and fixed-length string (a hash value which is generated by a cryptographic hash function) that uniquely represents a message. There are several reasons for signing the hash value and not the message itself: 1. In most cases the hash value is shorter than the message, thus the efficiency of signing is much higher. 2. Practical signature schemes accept fixed-length inputs. Therefore, if a long message is broken into fixed-length blocks and each block is signed, an attacker can remove or shuffle blocks with their corresponding signatures, and the verifier is not able to identify that the signed message is altered. 3. A cryptographic hash function is one way, thus a signature applied to the hash value is protected against chosen plaintext attack. I.e., an attacker cannot use educated guesses of hash values to sign, since he is not able to reconstruct messages from the hash values. The original motivation for the development of cryptographic hash functions was digital signatures. Later, the properties of cryptographic hash functions were found to be useful for many other applications, and they became building blocks of many other cryptographic protocols. The first hash function was proposed in 1978 by Rabin [42] as part of his proposal of a digital signature scheme that uses a hash function. Rabin s hashfunctionh isdefinedas partofthesignatureschemeinaxiomatic form, to be easy to compute, and to have the property of collision resistance (a formal definition of collision resistance is given in Section 1.2). H is based on a block cipher E, and it is described as follows: A message M is divided into blocks M = M 1,...,M n of a fixed-length k. An initial value h 0 is selected, and the hash function is H = E Mn (E Mn 1 (,...,E M2 (E M1 (h 0 )),...,)). This hash function was later shown to be vulnerable to meet-in-the-middle preimage attack [58]. However, the basic concept of iterative application of a block cipher with a message block used as a key, is used in most practical hash functions from that time till today. A formal definition of a cryptographic hash function was first given by Merklein1979 aspartofhisph.d.thesisandhisworkonthedevelopment of certified digital signatures [36, 37]. Merkle s definitions are close to Rabin s 6

23 with the following changes: He adds the property of preimage resistance to the definition of H, and the assumptions he uses are all concerned with the block cipher E. Merkle assumes E is a random cipher, thus it has the following properties (in the following, P, C, K, denote the plaintext, ciphertext and key respectively and p,c,k denote their length): 1. Given P, the average computational effort to find any K K such that E K (P) = E K (P), is about 2 c Given P and K, the average computational effort to find K such that E K (P) = E K (P), is about 2 c 1. Merkle shows that if a block cipher E has these properties, it can be transformed into a secure cryptographic hash function. Merkle s ideas of 1979 are applicable for all hash functions that are practically in use today. In 1987 Damgård presented a construction for hash functions [16] which is provably secure under the assumption that claw-free sets of permutations exist. A claw free set of permutation is a set S = {f 1,...,f n } where all the members in the set have the same domain and it is hard to find x,y in the domain of f such that f i (x) = f j (y). This construction has a theoretical importance since it presents a provably secure construction. However, it had no practical use, mainly because of very low performances when compared to hash functions based on block ciphers. In 1989 Merkle [36] and Damgård [15] generalized previous constructions and introduce a construction that is based on dividing a message into blocks of fixed-length, and iterating a compression function C on each block. They proved that if the compression function is collision resistant, then the hash function is collision resistant as well. They also proposed several candidates for collision resistant compression functions. Merkle-Damgård construction is similar to Merkle s ideas of 1979 with the following modifications: 1. The compression function receives a fixed-length input of the message blockandchainingvalue( M k + h k 1 ), andcompressesittoareduced length (of the chaining value h k ). The meaning of a collision is that two different inputs of length M k + h k 1 bits have the same output, i.e., C(h k 1,M k ) = C(h k 1,M k ). The original definition of Merkle for 7

24 collision is more restrictive and it allows differences only on the message. That is, the length of the input in which a difference is allowed is M k and C(h k 1,M k ) = C(h k 1,M k ) (or E M(h k 1 ) = E M (h k 1 )) is defined as a collision. 2. E is defined to have a second preimage resistance property, while C is defined to have the property of preimage resistance. 3. The message length is padded to the end of the last block of the message before it is hashed, to strengthen the collision and second preimage resistance of h. The Merkle-Damgård construction (which is described in Section 1.4.2) is the most commonly used construction for hash functions. Up to the early 1990 s, most hash functions were based on existing block ciphers. The motivation was to minimize design effort and the belief was that a hash function based on a secure block cipher is also secure. Such designs are typically slow, e.g., Merkle fastest design based on DES hashes only 18 bit per one application of DES. Moreover, hash functions that are based on block ciphers are subjected to meet-in-the-middle attacks, since a block cipher can be computed both ways (i.e., to encrypt plaintexts and decrypt ciphertexts). Different proposals that aimed at eliminating meetin-the-middle attacks where suggested. The common solution is the feedforward mixing of the input to the output, that forbids the attacker from computing backwards. Several constructions with feed-forward operations were proposed, such as Matyas-Meyer-Oseas, Davies-Meyer and Miaguchi- Preneel. These feed-forward modes solved the meet-in-the-middle weakness. Proposals of dedicated hash functions (that are not based on existing block ciphers) started to appear in Ron Rivest designed a new hash function, called MD4 [44], with the intention of combining security with simplicity and speed. MD4 hashes messages of any length up to bits and produces a 128-bit hash value. It uses computer friendly operations, like addition modulo 2 32 and bitwise operations, unlike most block ciphers designs that used S-boxes at that time. The design principles of MD4 were later used to construct many other hash functions. Among these functions, the de-facto standard MD5 (designed by Rivest) [45], and the SHA family (NIST standard designed by NSA) [41]. In parallel, Merkle proposed Snefru [35], which uses eight-bit to 32-bit S-boxes, and can be applied to 8

25 messages of arbitrary length (in typical cases up to bits). A year later Miyaguchi, Ohta, and Iwata proposed N-Hash [38] which is based on the design of the FEAL block cipher, and uses S-boxes that apply addition operations and byte rotations. Differential cryptanalysis [8] was introduced by Biham and Shamir in It was first used for cryptanalysis of block ciphers, but soon after it was found useful for the cryptanalysis of hash functions. The basic idea of differential cryptanalysis of hash functions is to find a difference between a pair of messages such that the differences of the intermediate data of the pair and its hash results are predictable with relatively high probability. The input differences through the predicted intermediate differences up to the predicted output differences are called a characteristic. If a characteristic predicts a zero output difference with some high probability p, then it is possible to find a collision with a complexity p 1 by randomly choosing pairs that have the input difference of the characteristic. If furthermore p 1 is smaller than the complexity of a generic attack, then the hash function is considered broken. Snefru was broken by differential cryptanalysis immediately after its introduction. In later years also MD4, MD5, SHA, versions of N-Hash and other hash functions were broken by differential cryptanalytic techniques. 1.2 Definitions and Required Properties of Hash Functions The basic idea of cryptographic hash functions is to associate a short easyto-compute and practically-unique value to each arbitrary-length message. A cryptographic hash function definition that formalizes this basic idea is given in [33]: Definition 1.1 ([33]) A cryptographic hash function is a function H that has the following properties: 1. Compression H maps an input M of an arbitrary bit-length (up to a predefined very long maximum length) to an output of a fixed bit-length m. H : {0,1} {0,1} m. 2. Easy to compute M {0,1}, H(M) is easy to compute. 9

26 3. Collision resistance it is computationally infeasible to find any two different messages M,M such that H(M) = H(M ). 4. Preimage-resistance Given y {0,1} m, it is computationally infeasible to find a message M such that H(M) = y. 5. Second preimage resistance given a message M, it is computationally infeasible to find a second message M M such that H(M) = H(M ). The security goal for preimage and second preimage resistance is an average effort of 2 m 1 hash computations, and for collision resistance it is 2 m 2 hash computations. The compression property of H implies that collisions exist, e.g., if we restrictourselves toinputmessages oflengthlandconsiderthatall of the2 m possible outputs of H have the same probability, then about 2 l m messages are hashed to each value. Hence, the requirement for collision resistance is related to the randomness of H in the sense that we require the 2 l m messages be an unpredictable subset of all possible messages. The requirement for a preimage resistance is related to the one-wayness of H. Thus, we can say that the output of H is pseudo-random and H has one-way properties. Other properties which are useful to assess the security of a hash function, and are sometimes called certificational properties [33] are given in the following: 1. Near-collision resistance it is computationally difficult to find any two different messages M,M that have a small Hamming distance between their hash values, i.e., H(M) differs from H(M ) by a small number of bits. 2. partial preimage-resistance given a hash result, it is computationally difficult to recover any part of the message. 3. non-correlation the input bits of a message M should not be correlated to the output bits of H(M). 10

27 1.3 Applications of Hash Functions Most hash functions were originally designed to be used for digital signature schemes, data integrity, and message authentication codes (MAC). The oneway and random behavior properties of hash functions were found to be adequate to many other applications. In the following we give a short list of applications, and the required properties that each of them impose on H. 1. Digital Signature Digital signature schemes were the first motivation for the development of hash functions. They assure the source of the data, its integrity and they also have a non-repudiation property. In general, due to efficiency and to avoid signature forgery, a signature to a message M is made on H(M) and not on M. The hash function should have the properties of collision resistance and second preimageresistance. In some specific signature schemes a preimage-resistance is also needed. Collision resistance and second preimage-resistance are needed to keep the non-repudiation property, e.g., in case the signer finds two colliding messages M,M, he can sign H(M) and claim later that he signed H(M ). Collision resistance also protects from signature forgery. If a collision can be found, an attacker who find M that collide with M, can claim that the original signature is on M. 2. MAC MACapplications areusedtoassurethesourceandintegrity of data. TheinputofMAC consistsofasecret key k anddatam, andthe output is MAC k (M). A receiver of a message and its MAC who knows the secret key, computes the MAC of the message he received, and accepts if the MAC he computed equals what he received. For MAC, given pairs (M i,mac k (M i )), it should be computationally difficult to find a legitimate MAC pair (M,MAC k (M). Notice that MAC is not required to be collision resistant for the parties who know the secret key. 3. Data integrity In order to verify the integrity of data M, H(M) is computed and securely stored. When the data M is retrieved, H(M ) is computed and compared with the securely stored value. If H(M) = H(M ) it is concluded that the data was not changed. Collision resistance is required here to assure that an attacker will not be able to find M with H(M) = H(M ), and replaces M with M. 11

28 4. Challenge-Response protocols in this applications one side in the protocol sends a challenge. The response received from the other side should assure that he knows some secret data. The hash function used for these applications are of the type used with MAC, and the required property is the same as in item Password protection In some identification schemes the hash value of the users passwords are stored in the system to protect against an attacker who can access the password file. For this applications a preimage-resistance is required. 6. Pseudo random source In this case the input to H is the seed and the output is used as a pseudo-random string. 1.4 Constructions of Hash Functions The compression property of a hash function, implies that collisions exist. Although it seems that compression and collision resistance are two contradicting requirements, there are proven constructions that simultaneously satisfy these two requirements under certain assumptions. In the following sections we describe two constructions of collision free hash functions under an assumption that collision resistant compression functions exist, i.e., C : {0,1} n {0,1} m is collision resistant and n > m Tree Constructions The tree construction is due to Damgård [15], and it is based on the existence of collision resistant compression functions of the form C : {0,1} 2n {0,1} n. A description of the construction is given in Figure 1.1. Tree construction allows parallel processing of the message. In practice it is not in use. Construction 1.1 Let C : {0,1} 2mc {0,1} mc be a collision resistant compression function and M {0,1} an arbitrary message. Let d = log 2 ( M /m c ) and t = 2 d. 1. Pad M with zeros and the binary representation of the message length M to obtain PAD(M), where PAD(M) = t m c. 12

29 M d= log 2 ( M /m c ) t=2 d Padding M 1 M 2 M 3 M t 2 M t 1 M t y d,1 y d,2 y d,3 y d,4 y d,t 3 y d,t 2 y d,t 1 y d,t C C C C y d 1,1 y d 1,2 y d 1,t/2 1 y d 1,t/2 C C C y 0,1 Hash Result Figure 1.1: Hash Functions Based on a Tree Construction. 2. Divide PAD(M) into t blocks of size b, i.e., M 1,...,M t. 3. For i = 1,...,t y d,i = x i. 4. For j = d 1,...,0 and i = 1,...,2 j,y j,i = C(y j+1,2i 1 y j+1,2i ). 5. H(M) = y 0,1 Theorem 1.1 H(M) as defined in Construction 1.1 is collision resistant. The proof of Theorem 1.1 is similar to the proof of Theorem 1.2 and it is not given here The Merkle-Damgård Construction Merkle-Damgård construction was introduced in two independent works of Merkle and Damgård in 1989 [15,34]. It is based on the existence of a collision resistant compression function of the form C : {0,1} n {0,1} m for any n > m. Oneof its advantages is that it allows serial processingof the message, therefore a stream of bits can be hashed on-line. This construction 13

30 M padding with 1, 0 s, and message length M 1 M 2 M n 1 M n b b b b IV m c C m c m c m c m c m c C C C Hash Result h 0 h 1 h 2 h n 1 h n Figure 1.2: Hash Functions based on Merkle-Damgård Construction. is the most commonly used, and our research is focused on hash functions based on this type of construction. The construction is depicted in Figure 1.2 Construction 1.2 Let C : {0,1} mc+b {0,1} mc be a collision resistant compression function and M {0,1} an arbitrary message. 1. Pad M with zeros and the binary representation of the message length M to obtain PAD(M), where PAD(M) is a multiple of b. 2. DividePAD(M)inton = PAD(M) /bblocksofsizeb,i.e., M 1,...,M n. 3. Define an initial value C 0 = IV {0,1} mc. 4. For i = 1 to n, C i = C(C i 1 M i ). 5. H(M) = C n Hereafter the C i s refer to the chaining values h i. Theorem 1.2 H(M) as defined in Construction 1.2 is collision resistant. proof: Assumeto the contradiction that it is easy to findm M such that H(M) = H(M ). Given M, M the following algorithm returns a colliding pair. 1. DivideM, M asdescribedinconstruction1.2, i.e., M = M 1,...,M n, M = M 1,...,M n, andcomputethec i saccordingly, i.e.,c 1,...,C n, C 1,...,C n. 14

31 2. If M M return (C n 1 M n, C n 1 M n ). 3. else, fori = 1ton 1ifC i C i andc i+1 = C i+1 return(c i M i+1, C i M i+1 ). 4. else, for i = 1 to n if M i M i return (C i 1 M i, C i 1 M i ) analysis: 1. Step 2 if M M then H(M) = C(C n 1 M n ) = C(C n 1 M n) = H(M ), thus (C n 1 M n, C n 1 M n ) is a pair of different messages that creates a collision under the application of C. 2. Step 3 The algorithm gets to Step 3 if M = M. It checks for a collision of C where the chaining values are differed (C i Ci ) but the compression results equal, i.e., C i+1 = C(C i M i+1 ) = C(Ci M i+1 ) = Ci+1 C i M i+1 and Ci M i+1 is a pair of different messages that creates a collision under the application of C. 3. Step 4-Thealgorithm gets to Step4whenn=n and i {1,...,n}, C i = Ci, thus since M M there must be an index i such that M i Mi. For each such i we have C i+1 = C(C i M i ) = C(Ci M i ) = C i+1, thus the pair (C i 1 M i, Ci 1 M i ) collides under C. 4. Under the assumption that H(M) = H(M ), the algorithm must return in one of the steps since each of the possible paths for a collision of H is covered The Compression Function of Tree and Merkle-Damgård Constructions Even though compression functions that are proven to be collision resistant (under some cryptographic assumptions) exist, they are not used in practice. Instead, much more efficient compression functions that were designed for hashing, and which are believed to be collision resistant are in use. In the following we give two constructions of compression functions. The first is provably secure under the assumption that extracting square roots modulo large numbers with two prime factors is hard, the second is based on random block ciphers. 15

32 C i 1 m c M i b E Σ m c C i = E Mi ( C i 1 ) + C i 1 Figure 1.3: Compression Functions Based on Block Ciphers (Davies-Meyer Construction). Construction 1.3 A compression function based on modular squaring [15] is constructed as follows: Let n = pq, s = n, and p,q two large primes. Let I beapropersubsetof1,...,sandt = I. Foranys-bitstringy = y 1,...,y s define C I (x) to be the concatenation of the y j bits where j I. The compression function is: C(x) = C I ((3F x) 2 modn) : {0,1} m {0,1} t, where 3F x denotes the concatenation of the byte 3F with x. The concatenation assure the modular reduction for every choice x and protects against collisions of the form x 2 and ( x) 2. We note that the number of elements in I should be large enough to protect against birthday attack. Construction 1.4 The construction is illustrated in Figure 1.3 and referred to Davies-Meyer [17]. The compression function uses a block-cipher-like function E in which the message is used as the key and the chaining value as the plaintext text. The compression function C : {0,1} mc+b {0,1} mc is: C(C i 1,M i ) = E Mi (C i 1 )+C i 1. 16

33 transfer NIS : M : Please the amount of 100 to account... deposit new Israeli Shekel number draw US $ : M * : Please the amount of 1,000,000 from account... draw of US Dollar number Figure 1.4: A Birthday Attack that Creates Two Messages with Different Meanings and Same Hash Result. 1.5 Attacks on Hash Functions Birthday Attacks A birthday attack on a hash function H considers H a black-box whose output can be approximated by a random variable. The attack relies on the birthday paradox: when elements are drawn at random with replacement fromaset of N different elements, it isexpected that oneoftheelements will be drawn twice after about N (or more accurately 1.17 N) selections with probability 1 2. Let H be a cryptographic hash function with an output bit-length of n bits. The elements are the hash results, and the number of elements in the set is N = 2 n. Therefore, after hashing about 2 n 2 randomly selected messages, with high probability two hash results will have the same value. The birthday attack may have significant impact when used to create two different messages with different meanings and same hash result. Such an attack is depicted in Figure 1.4, and described as follows: 1. An attacker V constructs two messages with different meanings. 2. Vselects n 2 wordsinm andinm andassignsacandidatereplacement to each of these words. 3. V creates 2 n 2 messages from M, hashes them, and stores the results. Then, he creates messages from M, hash each message and compare it with the stored hash results. 17

34 4. According to the birthday paradox a collision is expected with high probability after hashing about 2 n 2 messages Meet-in-the-Middle Preimage Attacks The property of preimage resistant is vulnerable in cases a meet-in-themiddle attack is feasible [58]. A meet-in-the-middle attack seeks collisions of intermediate data. The attack is practical in cases the hash function is invertible, e.g., Rabin s hash function. An attacker selects a message M of k blocks and hashes it to generate its target hash result h k. He chooses an intermediate chaining variable, e.g., h k 1, creates 2 n 2 different messages of (k 1) blocks each, hashes them and store their hash results. Then, the attacker inverts h k with different message block M k, and compares the results with the stored hash results. By the birthday paradox, a collision is expected after about 2 n 2 inversions of h k with different M k s Attacks on the Merkle-Damgård Construction The security goals of hash functions with output size of n bits are: 2 n 2 executions of the compression function for the collision resistance property, and 2 n for the preimage and second-preimage properties. In this section we describe three attacks. The first shows an unexpected result on the complexity of finding multi-collisions. The other two show that the security provided by hash functions that follow Merkle-Damgård construction does not meet the security goal for second-preimage. The multi-collision attack of Joux [29] shows that finding multi-collisions of a hash function is not much harder than finding a single collision. Dean s attack [18] shows that the security goal for second-preimage resistance cannot be obtained under the assumption that it is easy to find fix-points of the compression function. Kelsey and Schneier s second-preimage attack [32] generalizes Dean s attack and demonstrates that even without this assumption the security goal for second-preimage cannot be met Joux s Multi-Collision Attack An open problem of whether the concatenation of two hash values that are generated by two different hash functions, is more secure than a single hash 18

35 M 1 M 2 M r 1 M r h 0 h1 h 2 h r 1 h r h r 2 M 1 * M 2 * M * r 1 M * r Figure 1.5: Multi-Collision on Merkle-Damgård Construction. value, is solved in [29]. To solve this issue and prove that the concatenation is not more secure, the complexity of finding multi-collision is first discussed: A multi-collision consists of r messages M 1,...,M r, in which H(M 1 ) = H(M 2 ) =,...,= H(M r ). A multi-collision is expected to be found with a complexity of 2 n (r 1) r. Using the iterative structure of the hash function it is shown that the complexity of finding it, is about r 2 n 2 executions of the compression function. The idea is depicted in Figure 1.5. The figure illustrates concatenation of r pairs of colliding messages, in which each pair starts with the chaining value of the previous pair. The complexity of finding such a pair is 2 n 2, thus the complexity of finding the whole chain is about r 2 n 2. From these r pairs, 2 r pairs that produce the same hash result h r can be constructed, thus the complexity of finding the 2 r collision is much lower than expected Dean s Second-Preimage Attack Using Fix-Points Dean [18] shows that hash functions whose compression functions allow an easy finding of fix-points, cannot meet the security goals of second-preimage and collision resistance. A fix-point of a compression function is a pair (h i 1,M i ) that satisfies h i = h i 1 = C(h i 1,M i ). Finding a fix-point of a compression function that is constructed in accordance with Davies-Meyer construction, is easy. A decryption of the value 0 with an arbitrary message M i results with a fixpoint, i.e., h i 1 = D Mi (0), with no control over the value of h i 1. In order to make such a fix-point suitable for an attack, h i 1 should be a chaining value that is received by hashing message blocks with the standard initial 19

36 Fixed Point Fixed Point M1 M 2 M 2 C C C h 0 h 1 h 1 h 1 h 1 Figure 1.6: Constructing Expandable Message by Using a Fix-point. values. In order to find such a chaining value, an attacker executes a meetin-the-middle attack that aims at finding a chaining value that equals the fix-point. The attack proceeds by finding 2 n 2 fix-points and storing their values. Then, message blocks are hashed with the standard initial value h 0 and a match with the stored fix-points is searched. A match is expected after hashing about 2 n 2 different message blocks. Thus, finding a fix-point requires about 2 n 2 +1 executions of the compression function and about 2 n 2 +1 memory. With this match, messages of arbitrary length and with the hash value of the fix-point may be constructed. These messages are formed by the first block just found, and a concatenation of as many fix-points the attacker likes (up to the maximum message length the hash function allows). Such a message is depicted in Figure 1.6. Finding second-preimages without the Merkle-Damgård strengthening of adding the length of the message to the last block, is described as follows: Let M be a very long message of say 2 R blocks. An attacker hashes about R candidates for the block M 1, and compares each of the 2160 R resultingchainingvalueswiththe2 R intermediatechainingvaluesofh(m). Acollision is expected, since each of the former values has a probability 2 (160 R) to collide with the latter set. Once such a collision is found, i.e., h 1 = h i, the second-preimage M is constructed by M = M 1 M i+1,...,m 2 R. The Merkle-Damgård strengthening foils this attack since by adding the length of the message to the last block, the hash results of the two messages equal only before the last block. However, a fix-point may be used to apply the attack even in this case: The attacker starts with finding a fix-point. Consequently, a match with one of the chaining values of the message M 20