Brute force searching, the typical set and Guesswork

Transcription

1 Brute force searching, the typical set and Guesswor Mar M Christiansen and Ken R Duffy Hamilton Institute National University of Ireland, Maynooth marchristiansen, enduffy@nuimie Flávio du Pin Calmon and Muriel Médard Research Laboratory of Electronics Massachusetts Institute of Technology flavio, medard@mitedu Abstract Consider the situation where a word is chosen probabilistically from a finite list If an attacer nows the list and can inquire about each word in turn, then selecting the word via the uniform distribution maximizes the attacer s difficulty, its Guesswor, in identifying the chosen word It is tempting to use this property in cryptanalysis of computationally secure ciphers by assuming coded words are drawn from a source s typical set and so, for all intents and purposes, uniformly distributed within it By applying recent results on Guesswor, for iid sources it is this equipartition ansatz that we investigate here In particular, we demonstrate that the expected Guesswor for a source conditioned to create words in the typical set grows, with word length, at a lower exponential rate than that of the uniform approximation, suggesting use of the approximation is ill-advised I INTRODUCTION Consider the problem of identifying the value of a discrete random variable by only asing questions of the sort: is its value X? That this is a time-consuming tas is a cornerstone of computationally secure ciphers [] It is tempting to appeal to the Asymptotic Equipartition Property AEP) [2], and the resulting assignment of code words only to elements of the typical set of the source, to justify restriction to consideration of a uniform source, eg [3], [4], [5] This assumed uniformity has many desirable properties, including maximum obfustication and difficulty for the inquisitor, eg [6] In typical set coding it is necessary to generate codes for words whose logarithmic probability is within a small distance of the word length times the specific Shannon entropy As a result, while all these words have near-equal lielihood, the distribution is not precisely uniform It is the consequence of this lac of perfect uniformity that we investigate here by proving that results on Guesswor [7], [8], [9], [0], [] extend to this setting We establish that for source words originally constructed from an iid sequence of letters, as a function of word length it is exponentially easier to guess a word conditioned to be in the source s typical set in comparison to the corresponding equipartition approximation This raises questions about the wisdom of appealing to the AEP to justify sole consideration of the uniform distributions for cryptanalysis and provides alternate results in their place II THE TYPICAL SET AND GUESSWORK Let A 0,, m be a finite alphabet and consider a stochastic sequence of words, W, where W is a word of length taing values in A The process W has specific Shannon entropy H W : P W w) log P W w), w A and we shall tae all logs to base e For > 0, the typical set of words of length is T : w A : e H W +) P W w) e H W ) For most reasonable sources [2], P W T ) > 0 for all sufficiently large and typical set encoding results in a new source of words of length, W, with statistics P W w) P W w) P W T ) if w T, ) 0 if w / T Appealing to the AEP, these distributions are often substituted for their more readily manipulated uniformly random counterpart, U, P U w) : T if w T, 2) 0 if w / T, where T is the number of elements in T While the distribution of W is near-uniform for large, it is not perfectly uniform unless the original W was uniformly distributed on a subset of A Is a word selected using the distribution of W easier to guess than if it was selected uniformly, U? Given nowledge of A, the source statistics of words, say those of W, and an oracle against which a word can be tested one at a time, an attacer s optimal strategy is to generate a partial-order of the words from most liely to least liely and guess them in turn [2], [7] That is, the attacer generates a function G : A,, m such that Gw ) < Gw) if P W w ) > P W w) The integer Gw) is the number of guesses until word w is guessed, its Guesswor For fixed it is shown in [2] that the Shannon entropy of the underlying distribution bears little relation to the expected Guesswor, EGW )), the average number of guesses required to guess a word chosen with distribution W using the optimal strategy In a series of subsequent papers [7], [8], [9], [0], under ever less restrictive stochastic assumptions from words made up of iid letters to Marovian letters to sofic shifts, an asymptotic relationship as word length grows

2 between scaled moments of the Guesswor and specific Rényi entropy was identified: ) log EGW ) α ) αr W, 3) + α for α >, where R W β) is the specific Rényi entropy for the process W with parameter β > 0, R W β) : β log P W w) β w A These results have recently [] been built on to prove that log GW ) satisfies a Large Deviation Principle LDP), eg [3] Define the scaled Cumulant Generating Function scgf) of log GW ) by ) Λ W α) : log E e α log GW ) for α R and mae the following two assumptions Assumption : For α >, the scgf Λ W α) exists, is equal to αr W / + α)) and has a continuous derivative in that range Assumption 2: The it g W : log P GW ) ) 4) exists in, 0] Should assumptions and 2 hold, Theorem 3 of [] establishes that Λ W α) g W for all α and that the sequence log GW ) satisfies a LDP with a rate function given by the Legendre Fenchel transform of the scgf, Λ W x) : sup α Rxα Λ W α) Assumption is motivated by equation 3), while the Assumption 2 is a regularity condition on the probability of the most liely word With d γ W : α dα Λ W α), 5) where the order of the size of the set of maximum probability words of W is expγ W ) [], Λ W x) can be identified as x g W if x [0, γ W ] sup α R xα Λ W α) if x γ W, logm)], 6) + if x / [0, logm)] Corollary 5 of [] uses this LDP to prove a result suggested in [4], [5], that ElogGW ))) H W, 7) maing clear that the specific Shannon entropy determines the expectation of the logarithm of the number of guesses to guess the word W The growth rate of the expected Guesswor is a distinct quantity whose scaling rules can be determined directly from the scgf in equation 3), log EGW )) Λ W ) From these expressions and Jensen s inequality, it is clear that the growth rate of the expected Guesswor is less than H W Finally, as a corollary to the LDP, [] provides the following approximation to the Guesswor distribution for large : P GW ) n) n exp Λ W log n) ) 8) for n,, m Thus to approximate the Guesswor distribution, it is sufficient to now the specific Rényi entropy of the source and the decay-rate of the lielihood of the sequence of most liely words Here we show that if W is constructed from iid letters, then both of the processes U and W also satisfy Assumptions and 2 so that, with the appropriate rate functions, the approximation in equation 8) can be used with U or W in lieu of W This enables us to compare the Guesswor distribution for typical set encoded words with their assumed uniform counterpart Even in the simple binary alphabet case we establish that, apart from edge cases, a word chosen via W is exponential easier in to guess on average than one chosen via U III STATEMENT OF MAIN RESULTS Assume that the words W are made of iid letters, defining p p 0,, p m ) by p a P W a) We shall employ the following short-hand: hl) : a l a log l a a l a, for l l 0,, l m ) [0, ] m, l a 0, so that H W hp), and Dl p) : a l a logp a /l a ) Furthermore, define l [0, ] m and l + [0, ] m l arg maxhl) : hl) + Dl p) hp), l 9) l + arg maxhl) : hl) + Dl p) + hp), l 0) should they exist For α >, also define l W α) and ηα) by l W a α) : ηα) : a p /+α)) a b A p/+α)) b l W a log p a for all a A and ) p/+α) a b A p/+α) b log p a 2) Assume that hp)+ logm) If this is not the case, logm) should be substituted in place of hl ) for the U results Proofs of the following are deferred to the Appendix Lemma : Assumption holds for U and W with and Λ U α) : αhl ) Λ W α) αhl α)) Dl α) p), where l + if ηα) hp), l α) l W α) if ηα) hp), hp) + ), l if ηα) hp) + 3)

3 Lemma 2: Assumption 2 holds for U and W with g U g W hl ) and ) min hp) +, log max p a Thus by direct evaluation of the scgfs at α, log EGU )) hl ) and log EGW )) Λ W ) As the conditions of Theorem 3 [] are satisfied ElogGU )) Λ U 0) hl ) and ElogGW )) Λ W 0) hp), and we have the approximations P GU ) n) n exp Λ U log n) ) and P GW ) n) n exp Λ W log n) ) IV EXAMPLE Consider a binary alphabet A 0, and words W constructed of iid letters with P W 0) p 0 > /2 In this case there are unique l and l + satisfying equations 9) and 0) determined by: l0 p 0 logp 0 ) log p 0 ), l 0 + p 0 + logp 0 ) log p 0 ) Selecting 0 < < logp 0 ) log p 0 )) minp 0 /2, p 0 ) ensures that the typical set is growing more slowly than 2 and that /2 < l0 < p 0 < l 0 + < With l W α) defined in equation ), from equations 3) and 4) we have that logp 0 ) if α <, Λ W α) αhl W α)) Dl W α) p), if α logp 0 ) if α <, + α) log p +α 0 + p 0 ) +α From Lemmas and 2 we obtain hl Λ U α) ) if α <, αhl ) if α, and Λ W α) ) if α, hp) + if α, αhl α)) Dl α) p) if α, where l α) is deinfed in equation 3) and ηα) defined in equation 2) With γ defined in equation 5), we have γ W 0, γ U hl ) and γ W hl + ) so that, as hl ) > hl + ), the ordering of the growth rates with word length of the set of most liely words from smallest to largest is: unconditioned source, conditioned source and uniform approximation From these scgf equations, we can determine the average growth rates and estimates on the Guesswor distribution In particular, we have that ElogGW ))) Λ W 0) hp), ElogGW ))) Λ W 0) hp), ElogGU ))) Λ U 0) hl ) As hx, x)) is monotonically decreasing for x > /2 and /2 < l0 < p 0, the expectation of the logarithm of the Guesswor is growing faster for the uniform approximation than for either the unconditioned or conditioned word source The growth rate of the expected Guesswor reveals more features In particular, with A η) hp) + ), log EGW )) 2 logp p 0) 2 ), log EGW )) log EGU )) hl ) 2 logp p 0 ) 2 ), A 0 hl ) Dl p), A > 0 For the growth rate of the expected Guesswor, from these it can be shown that there is no strict order between the unconditioned and uniform source, but there is a strict ordering between the the uniform approximation and the true conditioned distribution, with the former being strictly larger With /0 and for a range of p 0, these formulae are illustrated in Figure The top line plots ElogGU )) loggw ))) ElogGU )) loggw))) hl ) hp), showing that the expected growth rate in the logarithm of the Guesswor is always higher for the uniform approximation than both the conditioned and unconditioned sources The second highest line plots the difference in growth rates of the expected Guesswor of the uniform approximation and the true conditioned source log EGU )) EGW )) hl ) 2 logp p 0 ) 2 ) if η) hp) + Dl p) if η) > hp) + That this difference is always positive, which can be established readily analytically, shows that the expected Guesswor of the true conditioned source is growing at a slower exponential rate than the uniform approximation The second line and the lowest line, the growth rates of the uniform and

4 0 02 x W * x) Difference in expected growth rate U 0) W 0) U ) W ) U ) W ) * x W x) x * U x) p x Fig Bernoullip 0, p 0 ) source Difference in exponential growth rates of Guesswor between uniform approximation, unconditioned and conditioned distribution with 0 Top curve is the difference in expected logarithms between the uniform approximation and both the conditioned and unconditioned word sources Bottom curve is the log-ratio of the expected Guesswor of the uniform and unconditioned word sources, with the latter harder to guess for large p 0 Middle curve is the log-ratio of the uniform and conditioned word sources, which initially follows the lower line, before separating and staying positive, showing that the conditioned source is always easier to guess than the typically used uniform approximation unconditioned expected Guesswor log EGU )) EGW )) hl ) 2 logp p 0) 2 ), initially agree It can, depending on p 0 and, be either positive or negative It is negative if the typical set is particularly small in comparison to the number of unconditioned words For p 0 8/0, the typical set is growing sufficiently quicly that a word selected from the uniform approximation is easier to guess than for unconditioned source For this value, we illustrate the difference in Guesswor distributions between the unconditioned W, conditioned W and uniform U word sources If we used the approximation in 8) directly, the graph would not be informative as the range of the unconditioned source is growing exponentially faster than the other two Instead Figure 2 plots x Λ x) for each of the three processes That is, using equation 8) and its equivalents for the other two processes, it plots log Gw), where Gw),, 2, against the large deviation approximations to log P W w), log P W w) and log P U w), as the resulting plot is unchanging in The source of the discrepancy in expected Guesswor is apparent, with the unconditioned source having substantially more words to cover due to the log x-scale) Both it and the true conditioned sources having higher probability words that sew their Guesswor Fig 2 Bernoulli8/0, 2/0) source, 0 Guesswor distribution approximations For large, x-axis is x / log Gw) for Gw),, 2 and the y-axis is the large deviation approximation / log P X w) x Λ X x) for X W, W and X U The first plateau for the conditioned and uniform distributions correspond to those words with maximum highest probability slowest exponential decay-rate) V CONCLUSION By establishing that the expected Guesswor of a source conditioned on the typical set is growing with a smaller exponent than its usual uniform approximation, we have demonstrated that appealing to the AEP for the latter is erroneous in cryptanalysis and instead provide a correct methodology for identifying the Guesswor growth rate T APPENDIX Note that by the definition of T as a typical set, P W ) > for all sufficiently large and thus log P W T) 0, which we will use in the proofs of both lemmas The proportion of the letter a A in a word w w,, w ) A is given by n w, a) : i : w i a The number of words in a type l, where l [0, ] for all a A and l a, is given by N l) : w A such that n w, a) l a a A The set of all types, those just in the typical set and smooth

5 approximations to those in the typical set are denoted L : l : w A such that n w, a) l a a A, L : l : w T, such that n w, a) l a a A, L : l : a l a log p a [ hp), hp) + ] where it can readily seen that L L for all For U we need the following Lemma Lemma 3: The exponential growth rate of the size of the typical set is log T log m if log m hp) + hl ) otherwise where l is defined in equation 9) PROOF: For fixed, by the union bound max l L! l a)! T + ) m max l L! l a)! For the logarithmic it, these two bounds coincide so consider the concave optimization problem max l L! l a)! We can upper bound this optimization by replacing L with the smoother version, its superset L Using Stirling s bound we have that sup log sup sup l L hl)! l L l a)! logm) hl ) if hp) + logm) if hp) + < logm) For the lower bound, we need to construct a sequence l ) such that l ) L for all sufficiently large and hl) ) converges to either logm) or hl ), as appropriate Let l /m,, /m) or l respectively, letting c arg max p a and define l ) a la + l b if a c, b A la if a c Then l ) L for all > m logp c)/2) and hl ) ) hl ), as required PROOF: Proof of Lemma Considering U ) first, αr U α + α log T αhl ), by Lemma 3 To evaluate Λ U α), as for any n N and α > 0 n n i α x α dx, i 0, again using Lemma 3 we have αhl ) log + α T α log GU log Eeα ) ) log T T i α i log T α αhl ), where we have used Lemma 3 The reverse of these bounds holds for α, 0], giving the result We brea the argument for W into three steps Step is to show the equivalence of the existence of Λ W α) and αr W / + α)) for α > with the existence of the following it log max N l) +α p la l L a 4) Step 2 then establishes this it and identifies it Step 3 shows that Λ W α) is continuous for α > To achieve steps and 2, we adopt and adapt the method of types argument employed in the elongated web-version of [8] Step Two changes from the bounds of [8] Lemma 55 are necessary: the consideration of non-iid sources by restriction to T ; and the extension of the α range to include α, 0] from that for α 0 given in that document Adjusted for conditioning on the typical set we get N + α max l) +α pla a l L w T P W w) Ee α log GW ) ) 5) N l) +α + ) m+α) max l L pla a w T P W w) The necessary modification of these inequalities for α, 0] gives N l) +α max l L pla a w T P W w) Ee α log GW ) ) 6) + ) N m + α max l) +α pla a l L w T P W w) To show the lower bound holds if α, 0] let l arg max N l) +α pla a l L w T P W w) Taing inf log and sup log of equations 5) and 6) establishes that if the it 4) exists, Λ W α) exists and equals it Similar inequalities provide the same result for αr W / + α))

6 Step 2 The problem has been reduced to establishing the existence of log max N l) +α p la l L a and identifying it The method of proof is similar to that employed in Lemma : we provide an upper bound for the sup and then establish a corresponding lower bound If l ) l with l ) L, then using Stirling s bounds we have that log N l ) ) hl) This convergence occurs uniformly in l and so, as L L for all, sup sup l L log max N l) +α p la l L a + α)hl) + ) l a log p a a sup l L αhl) Dl p)) 7) This is a concave optimization problem in l with convex constraints Not requiring l L, the unconstrained optimizer over all l is attained at l W α) defined in equation ), which determines ηα) in equation 2) Thus the optimizer of the constrained problem 7) can be identified as that given in equation 3) Thus we have that sup log max l L N l) +α p la a αhl α)) + Dl α) p), where l α) is defined in equation 3) We complete the proof by generating a matching lower bound To do so, for given l α) we need only create a sequence such that l ) l α) and l ) L for all If l α) l, then the sequence used in the proof of Lemma 3 suffices For l α) l +, we use the same sequence but with floors in lieu of ceilings and the surplus probability distributed to a least liely letter instead of a most liely letter For l α) l W α), either of these sequences can be used Step 3 As Λ W α) αhl α)) Dl α) p), with l α) defined in equation 3), d dα Λ W α) hl α)) + Λ W α) d dα l α) Thus to establish continuity it suffices to establish continuity of l α) and its derivative, which can be done readily by calculus PROOF: Proof of Lemma 2 First consider g U max log P U w T w) log T hl ), using Lemma 3 For W, if g W < hp) + the result follows simply, so assume that this is not the case By the property mentioned at the beginning of this section, the normalisation doesn t play a rôle in the it, ie sup sup log max P W w T w) log max P W w) w T with an analogous equality for the lower bound As P W w) exp hp) )) for all w T, the upper bound follows immediately and we need the corresponding lower bound on inf log max P W w) w T If g W > hp) +, there exists K N such that for all > K, log max w A P W w) > hp) + Then taing > K, max P W w) e hp)+) min p a w T max b A p b To prove this, we use proof by contradiction Assume max P W w) < e hp) ) min p a w T max b A p b Tae a word w arg max w T P W w), there exists at least one letter in w, b A, such that p b < max p a as P W w ) < max w A P W w) We then replace one occurrence of b in w with an element of arg max p a to mae the letter word w Then log P W w ) < log P W w ) As for each we have only changed one letter and by assumption, log P W w ) log P W w ) max ) b A p b min p a < hp) + This implies w T and contravenes our choice of w So inf log max P W w) hp) + w T if g W > hp) + Lastly if g W hp) +, max P W w) w T max w A P W w) if max w A P W w) < e hp) ) e hp) ) min p a otherwise max b A p b The result follows as hp) + + inf inf log min ) p a max b A p b log max P W w) hp) + w A

7 ACKNOWLEDGMENT MC and KD supported by the Science Foundation Ireland Grant No /PI/77 and the Irish Higher Educational Authority HEA) PRTLI Networ Mathematics Grant FdPC and MM sponsored by the Department of Defense under Air Force Contract FA C-0002 Opinions, interpretations, recommendations, and conclusions are those of the authors and are not necessarily endorsed by the United States Government Specifically, this wor was supported by Information Systems of ASDR&E) REFERENCES [] A Menezes, S Vanstone, and P V Oorschot, Handboo of Applied Cryptography CRC Press, Inc, 996 [2] T M Cover and J A Thomas, Elements of Information Theory John Wiley & Sons, 99 [3] J Pliam, On the incomparability of entropy and marginal guesswor in brute-force attacs, in INDOCRYPT, 2000, pp [4] S Draper, A Khisti, E Martinian, A Vetro, and J Yedidia, Secure storage of fingerprint biometrics using Slepian-Wolf codes, in ITA Worshop, 2007 [5] Y Sutcu, S Rane, J Yedidia, S Draper, and A Vetro, Feature extraction for a Slepian-Wolf biometric system using LDPC codes, in ISIT, 2008 [6] F du Pin Calmon, M Médard, L Zegler, J Barros, M Christiansen, and K Duffy, Lists that are smaller than their parts: A coding approach to tunable secrecy, in Proc 50 th Allerton Conference, 202 [7] E Arian, An inequality on guessing and its application to sequential decoding, IEEE Trans, Inf Theory, vol 42, no, pp 99 05, 996 [8] D Malone and W Sullivan, Guesswor and entropy, IEEE Trans Inf Theory, vol 50, no 4, pp , 2004, dwmalone/p/guess02pdf [9] C-E Pfister and W Sullivan, Rényi entropy, guesswor moments and large deviations, IEEE Trans Inf Theory, no, pp , 2004 [0] M K Hanawal and R Sundaresan, Guessing revisited: A large deviations approach, IEEE Trans Inf Theory, vol 57, no, pp 70 78, 20 [] M M Christiansen and K R Duffy, Guesswor, large deviations and Shannon entropy, IEEE Trans Inf Theory, vol 59, no 2, pp , 203 [2] J L Massey, Guessing and entropy, IEEE Int Symo Inf Theory, pp , 994 [3] A Dembo and O Zeitouni, Large Deviations Techniques and Applications Springer-Verlag, 998 [4] E Arian and N Merhav, Guessing subject to distortion, IEEE Trans Inf Theory, vol 44, pp , 998 [5] R Sundaresan, Guessing based on length functions, in Proc 2007 International Symp on Inf Th, 2007