The Coincidence Test: a Bayesian Statistical Test for Block Ciphers and MACs

Transcription

1 The Coincidence Test: a Bayesian Statistical Test for Block Ciphers and MACs Alan Kaminsky Rochester Institute of Technology, Rochester, NY, USA September 3, 2013 Abstract. A block cipher or message authentication code(mac) should behave as a random mapping. The mapping s randomness can be evaluated with statistical tests. Statistical test suites typically used for this purpose, such as the NIST test suite, are not well-suited for block ciphers and MACs; also, these test suites employ a frequentist approach, making it difficult to obtain an overall evaluation of the mapping s randomness. This paper describes a statistical test, the coincidence test, that overcomes the aforementioned deficiencies. The coincidence test is well-suited for block ciphers and MACs, and it employs a Bayesian approach that quite naturally yields an overall evaluation of the mapping s randomness. Results of applying the coincidence test to reduced-round and full-round versions of the PRESENT and IDEA block ciphers and the SipHash and SQUASH MACs are reported; the results are analyzed to determine the algorithms randomness margins. Keywords: Statistical tests, Bayesian model selection, PRESENT block cipher, IDEA block cipher, SipHash MAC, SQUASH MAC 1 Introduction A block cipher or message authentication code (MAC), loosely speaking, is supposed to produce random-looking outputs. The outputs randomness can be evaluated using a pseudorandom number generator(prng) statistical test suite, which applies a series of statistical tests to the outputs. Widely used statistical test suites include Diehard [14], Dieharder [3], ENT [22], TestU01 [13], and the NIST test suite [17, 19]. Although its title asserts that the NIST test suite is for cryptographic applications, the NIST test suite is in fact a general-purpose statistical test suite for evaluating the randomness of binary sequences in contrast to statistical tests on sequences of uniformly distributed integers or real numbers, as are typically found in other test suites. The NIST test suite does not care where the binary sequences come from, whether from a cryptographic function or some other source. The official document [17] just describes the 15 statistical tests in the test suite and discusses how to interpret the test results. In practice, two problems arise when using the NIST test suite (or any of the popular statistical test suites) to evaluate a block cipher or MAC. For brevity, henceforth block cipher will refer to a MAC as well.

2 The first problem stems from the fact that a block cipher is not itself a PRNG; it does not generate an arbitrarily-long binary sequence. Rather, the block cipher takes a fixed-length plaintext plus a fixed-length key and maps those to a fixed-length ciphertext. The mapping is supposed to be random, in two senses: a given key should map different plaintexts to ciphertexts that look as though chosen at random, and different keys should map a given plaintext also to ciphertexts that look as though chosen at random. To apply the NIST test suite to a block cipher, the block cipher must be made to behave like a PRNG and generate (typically long) binary sequences. The official document [17] does not specify how to do this. NIST did publish a document [20] describing how NIST made the AES finalist candidate block ciphers generate long binary sequences. One technique was to encrypt a long string of 0s using the block cipher in cipher block chaining (CBC) mode with various randomly chosen keys, and to apply the test suite to the resulting ciphertext strings. Another technique was similar, except a long string of mostly 0s with a few 1s was encrypted using the block cipher in electronic codebook (ECB) mode. Several other techniques were used as well; see [20]. However, when the block cipher is turned into a PRNG and the NIST (or other) test suite is applied, it is the PRNG s randomness that is being tested, not the randomness of the block cipher mapping itself. If a nonrandom bias is detected in the output, it is not clear whether the bias is inherent in the block cipher or is an artifact of the mode in which the block cipher is employed. For example, a block cipher operated in output feedback (OFB) mode is likely to experience a keystream block collision and start repeating the keystream after encrypting about 2 n/2 blocks, where n is the block size (due to the birthday paradox). This does not happen when the block cipher is operated in, say, counter mode. Thus, the slight bias in the OFB keystream arises from the use of OFB mode, not from the block cipher itself; and this bias would be present even if the block cipher mapping was perfectly random. Note that encrypting a string of 0s with a block cipher in CBC mode is identical to generating a keystream with a block cipher in OFB mode; thus, at least one of NIST s statistical tests on the AES finalist candidates had a slight inherent bias due to the block cipher mode. As another example, El-Fotouh and Diepold [6] found that AES would fail some of the tests in the NIST test suite when operated in certain modes, but would pass all the tests when operated in other modes. Again, this implies that the bias is due to the mode of operation, not the block cipher mapping. To summarize, the NIST test suite is not well suited for evaluating block ciphers (or MACs). It would be better to use statistical tests that evaluated the randomness of the block cipher mapping directly, without needing to turn the block cipher into a PRNG. The second problem that arises in practice stems from the fact that the NIST test suite, like most test suites, takes a frequentist approach to statistical testing. That is, from the binary sequence being analyzed, each test calculates a statistic and a p-value. The p-value is the probability that the observed value of the statistic would occur by chance, even if the binary sequence is actually random.

3 If the p-value falls below a significance threshold, the test fails (and indicates nonrandomness), otherwise the test passes. The NIST test suite consists of 15 statistical tests, some of which have multiple variations. Each test and variation is applied to multiple binary sequences produced by the PRNG (block cipher) under test, yielding perhaps several hundred p-values. Usually, some of these tests fail. Does this mean the PRNG is nonrandom? Not necessarily. If the PRNG is random, the p-values should be uniformly distributed between 0 and 1, and we would expect a certain fraction of them to fall below the significance threshold. So the NIST test suite applies a second-level statistical test to the p-values for each first-level test to determine whether the test s p-values are uniformly distributed. Each second-level test yields a second-level p-value, which may pass or fail. From this morass of firstand second-level p-values, the analyst must decide whether the binary sequences, and the block cipher that produced them, are random or nonrandom. The frequentist approach does not specify a procedure for combining multiple p-values into a single number that yields an overall random/nonrandom decision. As an alternative, a Bayesian statistical testing methodology does specify how to combine multiple test results into a single number, the posterior odds ratio, which in turn determines whether the block cipher mapping is random or nonrandom. However, none of the widely used statistical test suites take a Bayesian approach. This paper s novel contributions are twofold: A new statistical test, the coincidence test, is described. The coincidence test evaluates a block cipher s (plaintext, key)-to-ciphertext mapping s randomness directly, without needing to wrap the block cipher inside a mode of operation. The coincidence test uses a Bayesian approach, yielding a clear random/nonrandom decision. To illustrate the coincidence test s usefulness, the results of applying the coincidence test to reduced-round and full-round versions of the PRESENT and IDEA block ciphers and the SipHash and SQUASH MACs are reported. The results reveal each algorithm s randomness margin the number of rounds for which the algorithm produces random outputs, compared to the full number of rounds. The algorithms differing designs influence their randomness margins in interesting ways. The rest of the paper is organized as follows. Section 2 reviews the mathematics of Bayesian inference as used by the coincidence test. Section 3 describes the coincidence test and the parallel program that speeds up the coincidence test s calculations. Section 4 lists and analyzes the coincidence test results for PRESENT, IDEA, SipHash, and SQUASH. Section 5 summarizes related work. Section 6 discusses future work. 2 Bayesian Model Selection The coincidence test uses the methodology of Bayesian model selection applied to binomial distributions. For more information about Bayesian model selection, see [10].

4 2.1 Bayes Factors and Odds Ratios Let H denote a hypothesis, or model, describing some process. Let D denote an experimental data sample, or just sample, observed by running the process. Let pr(h) be the probability of the model. Let pr(d H) be the conditional probability of the sample given the model. Let pr(d) be the probability of the sample, apart from any particular model. Bayes s Theorem states that pr(h D), the conditional probability of the model given the sample, is pr(h D) = pr(d H) pr(h) pr(d). (1) Suppose there are two alternative models H 1 and H 2 that could describe a process. After observing sample D, the posterior odds ratio of the two models, pr(h 1 D)/pr(H 2 D), is calculated from Equation (1) as pr(h 1 D) pr(h 2 D) = pr(d H 1) pr(d H 2 ) pr(h 1) pr(h 2 ), (2) where the term pr(h 1 )/pr(h 2 ) is the prior odds ratio of the two models, and the term pr(d H 1 )/pr(d H 2 ) is the Bayes factor. The odds ratio represents one s belief about the relative probabilities of the two models. Given one s initial belief before observing any samples (the prior odds ratio), the Bayes factor is used to update one s belief after performing an experiment and observing a sample (the posterior odds ratio). Stated simply, posterior odds ratio = Bayes factor prior odds ratio. Suppose two experiments are performed and two samples, D 1 and D 2, are observed. Assuming the samples are independent, it is straightforward to calculate that the posterior odds ratio based on both samples is pr(h 1 D 2,D 1 ) pr(h 2 D 2,D 1 ) = pr(d 2 H 1 ) pr(d 2 H 2 ) pr(h 1 D 1 ) pr(h 2 D 1 ) = pr(d 2 H 1 ) pr(d 2 H 2 ) pr(d 1 H 1 ) pr(d 1 H 2 ) pr(h 1) pr(h 2 ). In other words, the posterior odds ratio for the first experiment becomes the prior odds ratio for the second experiment. Equation (3) can be extended to any number of independent samples D i ; the final posterior odds ratio is just the initial prior odds ratio multiplied by all the samples Bayes factors. Model selection is the problem of deciding which model, H 1 or H 2, is better supported by a series of one or more samples D i. In the Bayesian framework, this is determined by the posterior odds ratio (3). Henceforth, odds ratio will mean the posterior odds ratio. If the odds ratio is greater than 1, then H 1 s probability is greater than H 2 s probability, given the data; that is, the data supports H 1 better than it supports H 2. The larger the odds ratio, the higher the degree of support. An odds ratio of 100 or more is generally considered to indicate decisive support for H 1 [10]. If on the other hand the odds ratio is less than 1, then the data supports H 2 rather than H 1, and an odds ratio of 0.01 or less indicates decisive support for H 2. (3)

5 2.2 Models with Parameters In the preceding formulas, the models had no free parameters. Now suppose that model H 1 has a parameter θ 1 and model H 2 has a parameter θ 2. Then the conditional probabilities of the samples given each of the models are obtained by integrating over the possible parameter values [10]: pr(d H 1 ) = pr(d θ 1,H 1 )π(θ 1 H 1 )dθ 1 (4) pr(d H 2 ) = pr(d θ 2,H 2 )π(θ 2 H 2 )dθ 2 (5) where pr(d θ 1,H 1 ) is the probability of observing the sample under model H 1 with the parameter value θ 1, π(θ 1 H 1 ) is the prior probability density of θ 1 under model H 1, and likewise for H 2 and θ 2. The Bayes factor is then the ratio of these two integrals. 2.3 Odds Ratio for Binomial Models As will be described in Section 3, the coincidence test performs n Bernoulli trials, where the probability of success is θ, and counts the number of successes k, which obeys a binomial distribution. The values n and k constitute the sample D. With this as the model H, the probability of D given H with parameter θ is pr(d θ,h) = ( n k ) θ k (1 θ) n k = n! k!(n k)! θk (1 θ) n k. (6) The coincidence test computes the odds ratio for two binomial models, H 1 and H 2. H 1 is that the Bernoulli success probability θ 1 is a certain value p, the value that the success probability is supposed to have. Then the prior probability density of θ 1 is a delta function, π(θ 1 H 1 ) = δ(θ 1 p), and the Bayes factor numerator (4) becomes pr(d H 1 ) = n! k!(n k)! pk (1 p) n k. (7) H 2 is that the Bernoulli success probability θ 2 is some unknown value between 0 and 1, not necessarily the value it is supposed to have. The prior probability density of θ 2 is taken to be a uniform distribution: π(θ 2 H 2 ) = 1 for 0 θ 2 1 and π(θ 2 H 2 ) = 0 otherwise. The Bayes factor denominator (5) becomes pr(d H 2 ) = 1 0 n! k!(n k)! θk 2(1 θ 2 ) n k dθ 2 = 1 n+1 (8) (see Appendix A for a derivation). Putting everything together, the Bayes factor is pr(d H 1 ) pr(d H 2 ) = n! k!(n k)! pk (1 p) n k (n+1). (9)

6 Substituting the gamma function for the factorial, n! = Γ(n+1) [25], gives pr(d H 1 ) pr(d H 2 ) = Γ(n+1) Γ(k +1)Γ(n k +1) pk (1 p) n k (n+1). (10) Because the gamma function s value typically overflows the range of floating point values in a computer program, we compute the logarithm of the Bayes factor instead of the Bayes factor itself: log pr(d H 1) = logγ(n+1) logγ(k +1) logγ(n k +1) (11) pr(d H 2 ) +klogp + (n k)log(1 p) + log(n+1). The log-gamma function can be computed efficiently (see [16] page 256). When multiple independent runs of the coincidence test are performed, the log (posterior) odds ratio is the log prior odds ratio plus the sum of all the log Bayes factors. Assuming that H 1 and H 2 are equally probable at the start, the log odds ratio is just the sum of all the log Bayes factors. If the log odds ratio is greater than 0, then the data supports H 1, and the test confirms that the Bernoulli success probability is p. Otherwise, the data supports H 2, and the test indicates that the Bernoulli success probability is not equal to p. 3 The Coincidence Test 3.1 Coincidence Test Definition A block cipher coincidence is defined as follows (Figure 1). Let V be an output value for a block cipher, let P be a plaintext input value, let K be an encryption key input value, and let C be the ciphertext output when plaintext P is encrypted with key K. Consider a certain group of output bits G, where g = G is the number of bits in the group. If each bit of C in bit group G is equal to the corresponding bit of V in bit group G, then we say a coincidence has occurred with respect to V, P, K, and G. (G can consist of any group of bits, not necessarily adjacent bits as in the example in Figure 1.) The coincidence test performs n Bernoulli trials with one arbitrarily chosen V, one arbitrarily chosen P, and n different arbitrarily chosen K values, and counts the number of coincidences k; this is one data sample. If the block cipher s (plaintext, key)-to-ciphertext mapping is random, then the probability of a coincidence on each trial is 2 g. The coincidence test decides between two binomial models: H 1 where the success probability is 2 g, and H 2 where the success probability is other than 2 g. The log Bayes factor is given by Equation (11) with p = 2 g : log pr(d H 1) = logγ(n+1) logγ(k +1) logγ(n k +1) (12) pr(d H 2 ) +klog2 g + (n k)log(1 2 g ) + log(n+1).

7 Fig. 1. A block cipher coincidence As presently implemented, the coincidence test operates on a block cipher with a block size of 64 bits and a key size of up to 128 bits. To generate a data sample, the test picks output V, plaintext P, and base key K 0 at random. The testalsopicksmbitpositionsinthekeyatrandom;thesearecalled flipbits. To avoid the possibility of bias in these choices tainting the test results, the random numbers are derived from a file of random bytes archived at the random.org web site. (random.org gathers random bytes by sampling atmospheric radio frequency noise.) The test performs n = 2 m trials, where the keys K used in the trials consist of K 0 with every possible combination of the flip bits inverted or not inverted; this guarantees that the trials keys are all different. The results reported herein used m = 32. For each trial in the sample, the coincidence test encrypts P using K and examines multiple bit groups in the output. At present, the bit groups are 1 One-bit groups: bit 0, 1, 2,..., 63. Two-bit groups: bits 0 1, 2 3, 4 5,..., Four-bit groups: bits 0 3, 4 7, 8 11,..., Eight-bit groups: bits 0 7, 8 15, 16 23,..., Sixteen-bit groups: bits 0 15, 16 31, 32 47, The coincidence probabilities are 2 1, 2 2, 2 4, 2 8, and 2 16, respectively. For each bit group, the test counts the number of coincidences k over all the n trials and computes the log Bayes factor (12). The coincidence test repeats the above for versions of the block cipher algorithm reduced to different numbers of rounds, ranging from the minimum possible number of rounds up through the full number of rounds in the block cipher specification. 1 Increasing the block size, increasing the key size, increasing the number of trials, and examining larger bit groups are future work; see Section 6.

8 The coincidence test repeats all of the above for multiple samples, with each sample using an independent random choice of V, P, and K 0. The test calculates the log Bayes factor for each bit group, each number of block cipher rounds, and each sample. The test computes the log odds ratio for each bit group and each number of block cipher rounds by summing the log Bayes factors over all the samples. For each number of block cipher rounds, the test examines the worstcase (i.e., smallest) log odds ratio over all the bit groups. If the smallest log odds ratio is less than 0, then the test detects that the coincidence probability is other than 2 g, so the test concludes that the block cipher mapping is nonrandom for that number of rounds. Otherwise, the test indicates that the coincidence probability is 2 g as it should be, so the test concludes that the block cipher mapping is random for that number of rounds. It is important to compute the log Bayes factor for each sample separately and then to sum them together to compute the odds ratio, rather than combining the samples first and then computing the log Bayes factor. A simple thought experiment demonstrates why. Consider a one-bit group with coincidence probability 1/2. Suppose one sample with 1000 trials has 0 coincidences, and suppose another sample with 1000 trials has 1000 coincidences clearly nonrandom behavior. (Such results are not uncommon, especially when the block cipher is reduced to one round.) Then each sample s log Bayes factor is and the log odds ratio is , which in fact detects the nonrandom behavior. If the samples are combined first, then the 2000 trials have 1000 coincidences and the log odds ratio is 3.58, which incorrectly indicates random behavior. Let r be the largest number of rounds for which the coincidence test detects nonrandomness, and let R be the full number of rounds. The block cipher s randomness margin is defined to be 1 r/r; it is the number of rounds for which the block cipher produces random outputs as a fraction of the full number of rounds. A larger randomness margin is preferable to a smaller randomness margin. For a MAC, the coincidence test operates in the fashion described above, except P is a message, K is an authentication key, and C is the tag output when message P is authenticated with key K. 3.2 Coincidence Test Program The coincidence test is well-suited for parallel computation. Within each sample, the n trials can all be computed in parallel (with a different combination of flipped key bits in each trial), after which the separate trials coincidence counts canbeaddedtogethertoyieldthesample(n,k).thesamplesthemselvescanalso be computed in parallel (with a different choice of V, P, and K 0 in each sample), after which the separate samples log Bayes factors can be added together to yield the log odds ratios. A coincidence test program running on a parallel computer allows more trials per sample and/or more samples to be computed in the same amount of time as the program running on a single computer. A program to perform the coincidence test was written in Java. The particular block cipher or MAC under test is implemented as a subclass of a CryptoFunction

9 base class, and the program creates and uses an instance of such a subclass specified at run time. The program computes one sample of the coincidence test for a specified number of trials, for all the one- through sixteen-bit groups listed previously, for the minimum number of rounds through the full number of rounds in the cryptographic function. Multiple samples are computed by running the program repeatedly. The program was executed on a cluster parallel computer with ten quadcore nodes. For each of the cryptographic functions described in Section 4, 120 samples were computed, with 40 samples being computed in parallel at a time each on a separate core, with 2 32 trials per sample. 4 Coincidence Test Results The coincidence test was performed on the PRESENT and IDEA block ciphers and the SipHash and SQUASH MACs. These cryptographic functions are briefly described below. PRESENT. The PRESENT block cipher [2] has a 64-bit block and an 80- bit key. An initial subkey addition is followed by 31 rounds. Each round is a substitution-permutation network consisting of parallel application of the same 4 4 S-box to each nibble of the state, a bit permutation, and a subkey addition. The coincidence test examined PRESENT reduced to one round through the full 31 rounds. IDEA. The IDEA block cipher [12] has a 64-bit block and a 128-bit key. It consists of eight rounds. Each round combines four 16-bit state words with six 16- bitsubkeysusingseveralexclusive-ors,additionsmodulo2 16,andmultiplications modulo The eight rounds are followed by an output transformation that incorporates four additional 16-bit subkeys. The coincidence test examined IDEA reduced to one round through the full nine rounds (rounds 1 8 are the actual rounds, round 9 is the output transformation). SipHash.TheSipHashMAC[1]hasa128-bitkeyandgeneratesa64-bittag. SipHash is parameterized by two numbers c and d, yielding variations designated as SipHash-c-d. The SipHash state consists of four 64-bit words initialized to the key words exclusive-ored with fixed values. The message bytes are grouped into 64-bit blocks; the final message block is padded with zero or more null bytes plus a byte containing the message length modulo 256. In the compression phase, as each message block is absorbed, c rounds are applied to the state. In the finalization phase, after all the message blocks have been absorbed, d rounds are applied to the state and the tag is derived from the final state. Each round consists of a series of additions modulo 2 64, exclusive-ors, and rotations. The authors propose SipHash-2-4 as a fast version and SipHash-4-8 as a conservative, higher-security version. The coincidence test examined SipHash-c-d for c = 1 through 4 and d = 0 through 8. The coincidence test used a single-block message (56 bits of data plus the length byte). SQUASH. The SQUASH MAC [18] is designed for challenge-response authentication in tiny devices like RFID chips. It takes a 64-bit challenge (message)

10 Fig.2. Largest number of rounds exhibiting nonrandom behavior and randomness margin for each cryptographic function analyzed P and a 64-bit key K and produces a 32-bit response (tag). A 128-bit nonlinear feedback shift register (NLFSR) is initialized to (K (K P)); the NLFSR is clocked 512 times yielding a state m; the value m 2 (mod ) is computed; and the tag is the middle 32 bits of the result. The coincidence test instead used the middle 64 bits of the result as the tag. The coincidence test examined SQUASH reduced to 0 through 16 initialization rounds (clocks of the NLFSR) as well as 32, 48, 64,..., 512 initialization rounds. For each of the above cryptographic functions, Figure 2 shows the largest number of rounds for which the coincidence test detected nonrandom behavior. Figure 2 also shows each function s randomness margin. For SipHash, the plot shows the number of nonrandom compression plus finalization rounds, and the randomness margin is calculated with respect to the total number of compression plus finalization rounds. Appendix B lists the observed log odds ratios versus rounds for each function, as well as the calculation of the randomness margins. All the functions have substantial randomness margins, except for SipHash-1-2 which does not perform enough rounds to have a large randomness margin. It is interesting to note that IDEA, SipHash, and SQUASH exhibited nonrandom behavior for only four rounds at most, whereas PRESENT exhibited non-

11 random behavior for nine rounds. This is likely because the simple substitutionpermutation design of PRESENT s round function does not randomize the state much in each round. IDEA, SipHash, and SQUASH employ more complicated operations (additions, rotations, multiplications, squarings) than does PRESENT, and these operations appear to randomize the state to a greater degree in each round. 5 Related Work Several authors have studied the randomness of block ciphers by turning the block cipher into a PRNG and applying standard statistical tests to the PRNG. As already mentioned, NIST turned the AES finalist candidate block ciphers into PRNGs and applied the NIST test suite [20]. Hellekalek and Wegenkittl [8] turned AES into a PRNG and applied several tests from the NIST test suite as well as a new test, the gambling test, to AES. The designers of the HIGHT [9] and TWIS [15] lightweight block ciphers turned their ciphers into PRNGs and applied the NIST test suite. Sulak, Doğanaksoy, Ege, and Koçak [21] treated a block cipher as a PRNG that generates short binary sequences ( bits); they modified seven of the 15 statistical tests in the NIST test suite to operate on such short binary sequences, and they applied the modified tests to the AES finalist candidate block ciphers. The technique has also been used on cryptographic functions other than block ciphers. Wang and Zhang turned the SHA-3 candidate BLAKE hash function into a PRNG and applied the NIST test suite [23]. Some authors (such as [9] and [15]) merely make a vague statement to the effect that We applied the NIST test suite, and our cipher passed, leaving the impression that the NIST test suite yields an unequivocal yes/no, random/nonrandom decision about the block cipher under test. In fact, the NIST test suite does not yield such a decision (nor do similar test suites). The authors have not reported the detailed test results and the nuanced analysis of those results that led them to conclude that their ciphers passed. The present work analyzes the randomness of a block cipher s mapping directly, without treating the block cipher as a PRNG. Other authors have used this approach as well. Filiol [7] defined a statistical test based on comparing a cryptographic function s algebraic normal form to that of a random Boolean function, and applied the test to the DES and AES block ciphers as well as several stream ciphers and hash functions. Katos [11] defined a statistical test to measure the diffusion of the block cipher s mapping, but did not apply the test to any actual block ciphers. Doğanaksoy, Ege, Koçak, and Sulak [4] defined four statistical tests based on the block cipher mapping strict avalanche criterion test, linear span test, collision test, and coverage test and applied these tests to the AES finalist candidate block ciphers. The same authors also applied the methodologies of [21] and [4] to the compression functions of the SHA-3 second-round candidate hash functions [5].

12 All of the above works used a frequentist methodology (based on p-values) to analyze the statistical test results. To our knowledge, the present work is the first to use a Bayesian methodology (based on odds ratios). The Bayesian methodology quite naturally combines the results of individual tests into an overall odds ratio, yielding an unequivocal decision about whether the reducedround or full-round block cipher is random (log odds ratio > 0) or nonrandom (log odds ratio < 0). 6 Future Work Future work in the line of research described in this paper includes: Expanding the coincidence test program to support block sizes larger than 64 bits, key sizes larger than 128 bits, and bit group sizes larger than 16 bits. Retargeting the coincidence test program to run on graphics processing unit (GPU) accelerators and supercomputers, enabling samples with larger numbers of trials to be computed. Developing additional Bayesian statistical tests and adding them to the coincidence test to form a new cryptographic statistical test suite. Analyzing additional block ciphers and MACs, as well as hash functions, using the new test suite. Gaining insight into how a cryptographic function s design such as the use of S-boxes versus the use of add-rotate-xor networks affects the number of rounds needed to achieve random behavior. Gaining insight into how a cryptographic function s randomness margin (as determined by statistical testing) relates to the function s security margin (as determined by cryptanalysis). References 1. Aumasson, J., Bernstein, D.: SipHash: a fast short-input PRF. Cryptology eprint Archive, Report 2012/351 (2012) 2. Bogdanov, A., Knudsen, L., Leander, G., Paar, C., Poschmann, A., Robshaw, M., Seurin, Y., Vikkelsoe, C.: PRESENT: an ultra-lightweight block cipher. In: Paillier, P., Verbauwhede, I. (eds.) 9th International Workshop on Cryptographic Hardware and Embedded Systems (CHES 2007). LNCS, vol. 4727, pp Springer, Heidelberg (2007) 3. Brown, R., Eddelbuettel, D., Bauer, D.: Dieharder: a random number test suite (2013). retrieved 10- Apr Doğanaksoy, A., Ege, B., Koçak, O., Sulak, F.: Cryptographic randomness testing of block ciphers and hash functions. Cryptology eprint Archive, Report 2010/564 (2010) 5. Doğanaksoy, A., Ege, B., Koçak, O., Sulak, F.: Statistical analysis of reduced round compression functions of SHA-3 second round candidates. Cryptology eprint Archive, Report 2010/611 (2010)

13 6. El-Fotouh, M., Diepold, K.: Statistical testing for disk encryption modes of operations. Cryptology eprint Archive, Report 2007/362 (2007) 7. Filiol, E.: A new statistical testing for symmetric ciphers and hash functions. In: Deng, R., Bao, F., Zhou, J., Qing, S. (eds.) 4th International Conference on Information and Communications Security (ICICS 2002). LNCS, vol. 2513, pp Springer, Heidelberg (2002) 8. Hellekalek, P., Wegenkittl, S.: Empirical evidence concerning AES. ACM Transaction on Modeling and Computer Simulation 13, (2003) 9. Hong, D., Sung, J., Hong, S., Lim, J., Lee, S., Koo, B., Lee, C., Chang, D., Lee, J., Jeong, K., Kim, H., Kim, J., Chee, S.: HIGHT: a new block cipher suitable for lowresource device. In Goubin, L., Matsui, M. (eds.) 8th International Workshop on Cryptographic Hardware and Embedded Systems (CHES 2006). LNCS, vol. 4249, pp Springer, Heidelberg (2006) 10. Kass, R., Raftery, A.: Bayes factors. Journal of the American Statistical Association 90, (1995) 11. Katos, V.: A randomness test for block ciphers. Applied Mathematics and Computation 162, (2005) 12. Lai, X., Massey, J., Murphy, S.: Markov ciphers and differential cryptanalysis. In Davies, D. (ed.) Advances in Cryptology EUROCRYPT 91. LNCS, vol. 547, pp Springer, Heidelberg (1991) 13. L Ecuyer, P., Simard, R.: TestU01: a C library for empirical testing of random number generators. ACM Transactions on Mathematical Software 33, 22 (2007) 14. Marsaglia, G.: Diehard battery of tests of randomness. ~diehard/cdrom/, retrieved 03-Apr Ojha, S., Kumar, N., Jain, K., Sangeeta: TWIS a lightweight block cipher. In Prakash, A., Sen Gupta, I. (eds.) 5th International Conference on Information Systems Security (ICISS 2009). LNCS, vol. 5905, pp Springer, Heidelberg (2009) 16. Press, W., Teukolsky, S., Vetterling, W., Flannery, B.: Numerical Recipes: The Art of Scientific Computing, Third Edition. Cambridge University Press, Cambridge (2007) 17. Rukhin, A., Soto, J., Nechvatal, J., Smid, M., Barker, E., Leigh, S., Levenson, M., Vangel, M., Banks, D., Heckert, A., Dray, J., Vo, S., Bassham, L.: A statistical test suite for random and pseudorandom number generators for cryptographic applications. NIST Special Publication Revision 1a (2010). nist.gov/groups/st/toolkit/rng/documents/sp800-22rev1a.pdf, retrieved 03- Apr Shamir, A.: SQUASH a new MAC with provable security properties for highly constrained devices such as RFID tags. In Nyberg, K. (ed.) 15th International Workshop on Fast Software Encryption (FSE 2008). LNCS, vol. 5086, pp Springer, Heidelberg (2008) 19. Soto, J.: Statistical testing of random number generators. 22nd National Information Systems Security Conference (1999). toolkit/rng/documents/nissc-paper.pdf, retrieved 03-Apr Soto, J., Bassham, L.: Randomness testing of the Advanced Encryption Standard finalist candidates. NIST IR 6483 (2000). ST/toolkit/rng/documents/aes-report-final.doc, retrieved 03-Apr Sulak, F., Doğanaksoy, A., Ege, B., Koçak, O.: Evaluation of randomness test results for short sequences. In: Carlet, C., Pott, A. (eds.) Sequences and Their Applications SETA LNCS, vol. 6338, pp Springer, Heidelberg (2010)

14 22. Walker, J.: ENT: a pseudorandom number sequence test program (2008). http: // retrieved 10-Apr Wang, H., Zhang, H.: A fast pseudorandom number generator with BLAKE hash function. Wuhan University Journal of Natural Sciences 15, (2010) 24. Weisstein, E.: Beta function. From Mathworld A Wolfram Web Resource. http: //mathworld.wolfram.com/betafunction.html, retrieved 07-May Weisstein, E.: Gamma function. From Mathworld A Wolfram Web Resource. retrieved 07-May-2013 A Derivation of the Bayes Factor Denominator The beta function is defined by this integral [24]: B(a,b) = 1 0 u a 1 (1 u) b 1 du. (13) The beta function is also defined in terms of the gamma function [24]: B(a,b) = Γ(a)Γ(b) Γ(a+b). (14) For an integer argument, Γ(a) = (a 1)! [25]. Therefore, for integer arguments, the beta function is B(a,b) = (a 1)!(b 1)! (a+b 1)!. (15) Equating (13) and (15), substituting k + 1 for a, substituting n k + 1 for b, and substituting θ 2 for u yields 1 0 θ k 2 (1 θ 2 ) n k dθ 2 = k!(n k)! (n+1)!. (16) Finally, substituting (16) into the Bayes factor denominator (8) yields 1 n! pr(d H 2 ) = 0 k!(n k)! θk 2(1 θ 2 ) n k dθ 2 n! = k!(n k)! k!(n k)! (n+1)! = 1 n+1.

15 B Observed Log Odds Ratios Versus Rounds B.1 PRESENT Rounds log O.R. Rounds log O.R. Rounds log O.R B.2 IDEA Rounds log O.R B.3 SipHash Rounds log O.R. log O.R. log O.R. log O.R. d SipHash-1-d SipHash-2-d SipHash-3-d SipHash-4-d

16 B.4 SQUASH Rounds log O.R. Rounds log O.R. Rounds log O.R B.5 Randomness Margins r = Largest number of rounds for which nonrandomness was detected R = Full number of rounds Randomness margin = 1 r/r Rand. Function r R Marg. PRESENT IDEA SipHash SipHash SipHash SipHash SQUASH