Probabilistic Algorithms

Transcription

1 Probabilistic Algorithms Klaus Sutner Carnegie Mellon University Fall Some Probabilistic Algorithms Probabilistic Primality Testing RP and BPP Where Are We? 3 Examle 1: Order Statistics 4 We have an aarently correct abstract definition of randomness, regrettably but necessarily one that excludes comutability. On the other hand, there are methods to get reasonably random bits by iteration (Mersenne twister) or via uantum hysics (Quantis card). We can use these seudo-random bits to seed u algorithms. It remains to identify comlexity classes that corresond to these randomized algorithms, and check how they relate to traditional classes. We are given a list A a 1,..., a n and some index k, 1 k n. The roblem is to find the k-smallest element in the list: ord(k, A) a k where a 1, a 2,..., a n is the sorted version of the list. For simlicity, assume all elements are distinct, so there is a uniue solution. Secial cases: k 1: min, k n: max, k n/2: median. Verifying in Linear Time 5 Remember: QuickSort 6 The obvious answer is: sort the damn seuence. Alas, this is overkill and does more than the roblem secifies. For small values of k we could also use a buffer. Here is a different idea: if someone gave us an alleged solution b, we could easily verify that b is the right answer: count the elements less than b and check that there are exactly k 1 of them. b ord(k, A) { i a i < b } k 1. Of course, verification is not enough, we need get our hands on b. So we need to ick the element b that is larger than exactly k 1 others. Insight: This looks vaguely (OK, very vaguely) like the artitioning techniue from uick sort. < > Suose we ick some ivot at random and artition. If in fact ord(k, A) then the left block will have size k 1.

2 Partitioning 7 A Beautiful Theorem 8 Of course, in general ord(m, A) for some m k and the left block will have size m 1. Let s write (B,, C) for the result of artitioning with ivot and let m be the osition of after artitioning. If m k: return. If m > k reeat with (B, k). If m < k reeat with (C, k m). If we are out of luck, this will lead to bad slits and linear recursion deth. But if we ick the ivot at random this method is very fast on average. For uite some time it was believed that order statistics was as difficult as sorting and could not be done in less than log-lin time for the worst case (assuming that that only full comarisons between one item and another are ossible). In fact, eole tried to show that sorting could be reduced to order statistics. It was a huge surrise when finally divide-and-conuer algorithm was found that runs in a linear time. Unfortunately, the constants are so bad that the algorithm is of no ractical imortance. Partitioning solves the order statistics roblem in exected linear time. Theorem (Blum, Floyd, Pratt, Rivest, Tarjan 1973) Order statistics can be handled in linear time. Examle 2: Polynomial Identities 9 Counting Roots 10 Problem: Polynomial Zero Testing (PZT) Instance: A olynomial P (x 1,..., x n) with integer coefficients. Question: Is P identically 0? Wait, this is totally trivial... Yes, but there is a glitch: P need not be given in exlicit form as a cofficient vector. For examle, we could have P P 1 P 2... P r where the P i are olynomials. Of course, we can obtain the exlicit form by multilying out, but that is a otentially exonential oeration. In fact, we should think of the olynomial as given in form of a straight-line rogram or an arithmetic circuit using oerations {+,, }. Let F be a field (such as the rationals, reals, comlexes). (Schwartz-Ziel 1980) Let P F[x 1,..., x n] be of degree d and S F a set of cardinality s. If P is not identically zero, then P has at most d s n 1 roots in S. Proof. The roof is by induction on n, the number of variables. The case n 1 is clear. For n > 1, define d 1 and P 1(x 2,..., x n) to be the degree of x 1 in P and the coefficient, resectively. Hence P (x 1,..., x n) x d 1 P 1(x 2,..., x n) + stuff. Suose a 2,..., a n S is a root of P 1, by induction we know there are at most s n 1 (d d ) such roots. Then a, a 2,..., a n could be a root for P for all a S. Otherwise, there are at most d such roots. Adding, we get the claim. Schwartz s Method, II 11 Examle 3: Perfect Matchings 12 The set S F here could be anything. For examle, over Q we might choose S {0, 1, 2,..., s 1}. The main alication of the lemma is to give a robabilistic algorithm to check whether a olynomial is identically zero. Suose P is not identically zero and has degree d. Choose a oint a S n uniformly at random and evaluate P (a). Then Pr[P (a) 0] d s So by selecting S of cardinality 2d the error robability is 1/2. Note that the number of variables lays no role in the error bound. To lower the error bound, we can reeat the basic ste (relying on indeendence), or we can increase s. Some combinatorial roblems can be translated relatively easily into PZT. For examle, suose G {1, 2,..., n}, E is a undirected grah. Define its Tutte matrix by x ij if (i, j) E and i < j, A(i, j) x ij if (i, j) E and i > j, 0 otherwise. The determinant of this matrix is a olynomial in u to n 2 variables x ij. Theorem (Tutte 1947) G has a erfect matching iff its Tutte matrix has non-zero determinant.

3 Comuting the Determinant 13 Unimressed? 14 The idea behind the roof is that in the standard reresentation of the determinant M sign(π) M i,π(i). π i This result holds for general grahs, not just biartite ones. The roof is significantly harder in the general case. In the end, one obtains an O(n 2 e) algorithm, but it s tricky. erfect matchings π contribute a term, but nothing else does. Hence, in exlicit form, M can be exonentially large. But we can use Schwartz s lemma to determine whether it is zero in olynomial time, with small error robability. J. Edmonds Paths, Trees, and Flowers Canad. J. Math. 17(1965) Background: Quadratic Residues 16 Some Probabilistic Algorithms The multilicative subgrou of Z m is Z m { x Z m gcd(x, m) 1 } 2 Probabilistic Primality Testing Definition (Quadratic Residues) a Z m is a uadratic residue modulo m if x 2 a (mod m) has a solution over Z m, and a uadratic non-residue otherwise. RP and BPP Thus Z m can be artitioned into QR m and QNR m. For examle QR 7 {1, 2, 4} QNR 7 {3, 5, 6} Another Examle 17 Legendre Symbol 18 Here are the 2 uadratic residues a for m 20, a comosite modulus, together with the solutions to x 2 a (mod m). a : x 1 : 1, 9, 11, 19 9 : 3, 7, 13, 17 The non-residues are 3, 7, 11, 13, 17, 19. This was done by brute-force comutation (comute suares mod m). Is there a way to check efficiently whether a number is a uadratic residue? Suose is an odd rime and z Z. The Legendre symbol LS (z, ) is defined by: { +1 if z is a uadratic residue, LS (z, ) 1 otherwise. Proosition LS (z, ) z ( 1)/2 (mod ). Write M(k) for the number of stes needed in multilying two k-bit numbers. Then the roosition rovides an O(lg M(lg )) algorithm to comute LS (z, ): use a fast exonentiation algorithm modulo. Unfortunately, we need to be an odd rime here.

4 Jacobi Symbol 19 Key Proerties 20 Suose 1... r where the i are ( odd) rimes, not necessarily distinct, z relatively rime to. The Jacobi symbol is defined by: ( ) z LS (z, 1) LS (z, 2)... LS (z, r) { 1, +1}. If we knew the i then we could easily comute the Jacobi symbol according to its definition. But what if all we know is? The following roerties of the Jacobi symbol will hel in finding an algorithm. The roof is uite elementary. z ( ) z ( ) z1 z 2 ( ) 1 1 ( ) z mod ( ) z1 ( ) z2 ( ) { 1 1 if 3 (mod 4), +1 otherwise. ( ) { 2 1 if 3, 5 (mod 8), +1 otherwise. Law of Quadratic Recirocity 21 Jacobi Symbol Algorithm 22 We need one more ingredient for the algorithm. This is one of Gauss s favorite results, he gave several roofs for it. Theorem (C. F. Gauss 1798) Let, be two odd rimes. Then we have ( ) ( ) if 3 (mod 4), ) + otherwise. ( The theorem together with the roosition makes it ossible to aly mods to cut down the size of the numbers (much like in the comutation of the GCD). We may safely assume 0 < <. sign 1; while( > 0 ) { while( even ) { // eliminating even art / 2; if( mod 8 3, 5 ) sign -sign; } swa and ; // uadratic recirocity if(, 3 mod 4 ) sign -sign; mod ; } if( 1 ) return sign else return 0; // corimality broken Analysis 23 Jacobi Examle 24 First note that the algorithm maintains the invariant: 0 < <, both odd. If both numbers are corime, they will stay so during the execution of the loo (we return 0 if it turns out that corimality is violated). Correctness now follows from the mentioned roerties of the Jacobi symbol. For running time, note that all the numbers have at most k bits where k is the length of and. Hence, all the arithmetic oerations are olynomial time. The same argument that shows that Euclidean algorithm is olynomial time can be used to show that the loo executes no more than 2k times. With a little more effort one can show that the algorithm runs in time O(log log ). The following table shows the comutation for 117 and 271. Result: 1. sign

5 Who Cares? 25 Solovay-Strassen 26 The Jacobi symbol is an extension ) of the Legendre symbol in the sense that whenever is rime we have LS (, ). Note, however, that ( ( ) +1 no longer imlies that is a QR modulo. For examle, ( 2 9 ) ( 2 3 ) 2 1 but 2 QNR 9. Hence we now have two different ways to comute the Legendre symbol for rime. This is the basic ( idea ) behind the Solovay-Strassen algorithm: ick z at random and comute in two ways, say, with results LS 1 and LS 2. z If LS 1 LS 2 then is not rime; if LS 1 LS 2 then is rime with a certain high robability. Solovay-Strassen Primality Testing Algorithm Inut: n ick z at random, 1 < z < n if gcd(z, n) > 1 then return NO LS 1 z (n 1)/2 (mod n) LS 2 ( ) z n if LS 1 LS 2 then return NO else return YES? Correctness 27 It s a Grou 28 Theorem (Solovay-Strassen) If n is rime, the Solovay-Strassen test returns YES?; otherwise, the test returns NO with robability at least 1/2. The running time is O(log n M(log n)). Proof. Suose n is comosite and consider the set S of bad choices for our algorithm where we get a false ositive: { ( S z Zn z (n 1)/2 z ) n } (mod n). Claim: S Z n /2. First note the S is a subgrou because of the multilicativity roerty of the Jacobi symbol. It remains to show that S is roer. Assume otherwise, so that in articular z n 1 1 (mod n) for all z Z n. Consider the rime factor e of n with maximal exonent e and set m n/ e. Pick a generator g for the multilicative subgrou of Z e (which is known to exist). By the CRT, there is an element a Zn such that a g (mod e ) and a 1 (mod m). From our assumtions, a n 1 1 (mod n), and therefore g n 1 1 (mod e ), so that ord(g; Z e) n 1. Proof, Cont d 29 If e > 1 then we have ord(g; Z e) e 1 ( 1) n 1, a contradiction. If e 1( then ) n must be suare-free and contain ( at ) least two rime factors. Clearly 1 since g is a generator, and 1 for all the other rime a factors of n. Hence a (n 1)/2 ( a n ) 1 (mod n). But that contradicts a 1 (mod m). It follows that n is comosite then the robability of the Solovay-Strassen algorithm returning NO is at last 1/2. a Some Probabilistic Algorithms Probabilistic Primality Testing 3 RP and BPP Note that Solovay-Strassen shows that Comosite is in NP (also shown by V. Pratt in 1975 using a different method).

6 Probabilistic Classes 31 Probabilistic Turing Machines 32 These examles and many more make it temting to exand our comlexity zoo a bit: there should be comlexity classes corresonding to robabilistic algorithms. For anyone familiar with nondeterministic classes like NP this is not much of a stretch: we already have comutation trees Technically, it is convenient to define a robabilistic Turing machine (PTM) M to be a Turing machine accetor with two transition functions δ 0 and δ 1. At each ste in the comutation, M chooses δ 0/1 with robability 1/2 (and, of course, indeendently of all other choices). Write M(x) 2 for the random variable that indicates accetance/rejection. We say that M runs in time t(n) if it halts in at most t( x ) stes regardless of the random choices made. So there may be as many as 2 t(n) branches in the comutation tree of M on an inut x of length n. All we need to do is imose additional conditions on acceting branches. Bounding Error 33 BPP 34 The critical art of any argument here will be a bound on errors. There are two tyes of errors that are a riori indeendent. We would like M(x) L(x) for some language L 2. Alas... False Positives We may have x / L but M(x) 1. Remedy: Pr[ M(x) 1 x / L ] small. False Negatives We may have x L but M(x) 0. Remedy: Pr[ M(x) 0 x L ] small. We get different robabilistic classes by imosing different constraints on false ositives/negatives. Let t : N N be a reasonable running time and L 2 a language. Definition (Bounded Error Probabilistic Polynomial Time) A PTM M decides L in time t(n) if for all x 2, M on x halts in at most t( x ) stes (regardless of random choices) and Pr[M(x) L(x)] 2/3 BPP is the class of all languages decided by a PTM in time O(oly). BPP seems to cature the intuitive notion of a roblem efficiently solvable by a feasible (ossibly robabilistic) algorithm very well. BPP, Again 35 The Old Classic 36 To be clear: by Pr[M(x) L(x)] 2/3 we mean x L Pr[M(x) 1] 2/3 x / L Pr[M(x) 0] 2/3 So the machine can make errors in both directions, but the robability of false ositives/negatives is at most 1/3. This is uite different from classes like NP where there are no errors when x / L. Prime, the roblem of determining whether a number is rime, is obviously in co-np. V. Pratt showed that Prime is in NP by constructing short witnesses to rimality, a clever use of basic number theory. Alas, his method does not yield a BPP algorithm. Probabilistic rimality testing algorithm showed Prime to be in BPP, but it was not known to be in P. Then in 2002 Agrawal, Kayena and Saxena ruined everything by showing that Prime is in P (using no more than high school arithmetic in the rocess).

7 Projections 37 Truth Amlification (aka Error Reduction) 38 The magic constant 2/3 in these definitions is by no means sacred. As with NP, one can avoid the funky robabilistic Turing machines by invoking witnesses. Theorem L is in BPP iff there is a olynomial time deterministic Turing machine M and a olynomial such that for all x 2 : Pr[M(x, w) L(x) : w 2 ( x ) ] 2/3 Again, this is different from NP in that we reuire not just one witness but lots of them. This shows that BPP EXP: we can simly enumerate the otential witnesses and count the good ones. Define a new class BPP to contain all languages L such that there exists a oly time PTM M such that for some constant c: Theorem BPP BPP Sketch of roof. Pr[M(x) L(x)] 1/2 + x c For any constant d, we can design a new PTM M such that Pr[M (x) L(x)] 1 2 x d M simly runs M 8 x 2c+d times and takes a majority vote. The correctness roof uses a Chernoff bound. Closure 39 Proof 40 BPP is closed under union, intersection and comlement. Proof. Closure under comlementation follows directly from the definitions. To show closure under intersection, suose M i decides L i in BPP, i 1, 2. Build a new machine M that does the following: Comute b i M i(x). Return min(b 1, b 2). We need to show that M is a BPP machine. Case 1: x L 1 L 2 Then Pr[M(x) 1] 2/3 2/3 4/9, which can be fixed by amlifying M 1 and M 2. Case 2: x / L 1 L 2 Pr[M(x) 0] Pr[M 1(x) 0 M 2(x) 0] Pr[M 1(x) 0] + Pr[M 2(x) 0] Pr[M 1(x) 0 M 2(x) 0] 2/3 For the last ste, consider the three cases x / L 1 L 2, x L 1 L 2 and x L 2 L 1. One-Sided Error 41 The Other Side 42 A BPP machine can make errors in both directions. Here is a more restricted version where in the case of a NO-instance the answer is always NO. Definition x L Pr[M(x) 1] 2/3 x / L Pr[M(x) 0] 1 RP is the class of all languages decided by a one-sided error PTM in olynomial time. Similarly one defines co RP, the comlements of all languages in RP. So this means no false negatives, but otentially false ositives. PZT via Schwartz-Ziel and Solovay-Strassen are both in co RP (or in RP if you fli the uestion). There is a similar truth amlification result as for BPP. So RP roduces no false ositives, but may roduce false negatives, with low robability. It follows immediately that RP NP; alas, closure under comlementation vanishes. RP is closed under union and intersection.

8 Zero-Sided Error 43 Clocking a ZPP Algorithm 44 Here is a wild idea: how about a PTM that never makes a mistake? x L M(x) 1 x / L M(x) 0 Here is the glitch: for some comutations the running time may not be olynomial; the machine is fast only on average. ZPP is the class of all such PTM with exected olynomial running time. These are also called Las Vegas algorithms, as oosed to the more civilized Monte Carlo algorithms. The randomized version of uicksort can be construed as a Las Vegas tye algorithm (though withing oly time): with small robability it will have uadratic running time, on average the running time is log-linear. In reality, we would use a clock to halt the algorithm if it has not returned any (necessarily correct) answer after a olynomial amount of time. In this case we can think of the outut as don t know. A roblem is in ZPP iff it has an always-correct algorithm that has olynomial average-case running time. If we get a don t know we just run the the algorithm again. PP 45 Some Inclusions 46 The most ermissive class of randomized olynomial time comutation is obtained by considering PTM with oly running time and error bounds x L Pr[M(x) 1] > 1/2 x / L Pr[M(x) 1] 1/2 So in this case the error might get arbitrarily close to 1/2 deending on the inut (say, for inut of size n it s 1/2 + 2 n ). This wreaks havoc with arguments based on reeating the algorithm to amlify truth. PP is closed under comlement, union and intersection, but the roofs reuire uite a bit of effort. P ZPP RP co RP RP, co RP BPP PP Some would argue that BPP is a better formalization of the elusive notion of efficiently solvable roblem than P. However, it is not unreasonable to conjecture that P BPP, so there may be no class between the two descritions. One can show that PP PSPACE by counting acceting comutations. And NP? 47 Alas, the relationshi between BPP and NP is currently oen. It is known, though, that BPP Σ 2 Π 2 near the bottom of the olynomial time hierarchy. It follows that P NP imlies P BPP.