The Hidden Subgroup Problem and Quantum Computation Using Group Representations

Transcription

1 The Hidden Subgroup Problem and Quantum Computation Using Group Representations Sean Hallgren Computer Science Department Caltech, MC Pasadena, CA 925 Alexander Russell Department of Computer Science and Engineering University of Connecticut Storrs, CT Amnon Ta-Shma Computer Science Division Tel-Aviv University Israel August 28, 200 Abstract The Hidden Subgroup Problem is the foundation of many quantum algorithms. An efficient solution is known for the problem over Abelian groups, employed by both Simon s algorithm and Shor s factoring and discrete log algorithms. The non-abelian case is open; an efficient solution would give rise to an efficient quantum algorithm for Graph Isomorphism. We fully analyze a natural generalization of the Abelian case algorithm to the non-abelian case. We show that the algorithm finds the normal core of the hidden subgroup, and that, in particular, normal subgroups can be found. We show, however, that this immediate generalization of the Abelian algorithm does not efficiently solve Graph Isomorphism. Introduction Peter Shor s seminal article [23] presented efficient quantum algorithms for computing integer factorizations and discrete logarithms, problems thought to be intractable for classical computation models. A primary ingredient in these algorithms is an efficient solution to the hidden subgroup problem for certain Abelian groups; indeed computing discrete logarithms directly reduces to the hidden subgroup problem. Formally, the hidden subgroup problem is the following: Definition.. Hidden Subgroup Problem (HSP). Given an efficiently computable function f : G S, from a finite group G to a set S, that is constant on (left) cosets of some subgroup H and takes distinct values on distinct cosets, find a set of generators for H. Preliminary version appeared in Proceedings of the Thirty-Second Annual ACM Symposium on Theory of Computing, pages , Portland, Oregon, 2 23 May Supported in part by an NSF Mathematical Sciences Postdoctoral Fellowship. This work was done while the author was at the University of California, Berkeley, with support from an NDSEG Fellowship, a GAANN Fellowship, and NSF Grant CCR Supported by NSF NYI Grant CCR and a David and Lucile Packard Fellowship for Science Most of this work was done while the author was at the University of California, Berkeley.

2 The general paradigm, which gives rise to efficient quantum algorithms for this problem over Abelian groups, is the following: Algorithm.. (Algorithm for the Abelian HSP).. Compute g G g, f(g) and measure the second register f(g). The resulting superposition is h H ch f(ch) for some coset ch of H. Furthermore, the distribution of c over G is uniform. 2. Compute the Fourier transform of the coset state, resulting in H ρ(ch) ρ, h H where Ĝ denotes the set of homomorphisms {ρ : G C}. 3. Measure the first register and observe a homomorphism ρ. A key fact about this procedure is that the resulting distribution over ρ does not depend on which coset ch arises after the first stage. Thus, we can repeat the same experiment many times, each time inducing the same distribution over Ĝ. It is well known that an efficient solution to the HSP for the symmetric group S n gives, in particular, an efficient quantum algorithm for Graph Isomorphism. It is also known how to efficiently compute the Fourier transform over many non-abelian groups, most notably over S n [2]. Nevertheless, until this work, there was no general understanding of the HSP over non-abelian groups. In this paper we study a natural generalization of Algorithm. to non-abelian groups. Namely, we study the following algorithm: Algorithm.2. (Potential Algorithm for the General HSP).. Compute g G g, f(g) and measure the second register f(g). The resulting superposition is h H ch f(ch) for some coset ch of H. Furthermore, c is uniformly distributed over G. 2. Let Ĝ denote the set of irreducible representations of G and, for each ρ Ĝ, fix a basis for the space on which ρ acts. Let d ρ denote the dimension of ρ. Compute the Fourier transform of the coset state, which is the superposition d ρ d ρ i j d ρ H 3. Measure the first register and observe a representation ρ. ( ) ρ(ch) ρ,i, j. h H i, j For more details about the Fourier transform over non-abelian groups, see Section 2. As before, we wish the resulting distribution to be independent of the actual coset ch (and so depend only on the subgroup H). This is guaranteed by measuring only the name of the representation ρ and leaving the matrix indices (the values i and j) unobserved. The question we study is whether this procedure retains enough information to determine H, or, more precisely, whether O(log()) samples of this distribution are enough to determine H with high probability. Our analysis of Algorithm.2 depends on the following theorem, which we believe is interesting on its own and is one of the main technical contributions of the paper: 2

3 Theorem. The probability of measuring the representation ρ in Algorithm.2 is the product of the dimension of ρ, H, and the number of occurrences of the trivial representation of H in ρ. In other words, it is the product of the dimension of ρ, H, and the number of times ρ appears in the induced representation Ind G H H, where H denotes the trivial representation on H. We first apply this to obtain a positive result: Theorem 2. Let H be an arbitrary subgroup of G, and let H G be the largest subgroup of H that is normal in G. With high probability, H G is uniquely determined by observing m = O(log) independent trials of Algorithm.2 when H is the hidden subgroup. When H is normal, H = H G, and this determines H. Our reconstruction result is information theoretic, and applies to any normal subgroup H of any group G without reference to the specific way that the representations ρ are expressed. We proceed at this level of abstraction because there is no known canonical presentation for the representations of a finite group G. In the same vein, there is no general method for computing the Fourier transform over an arbitrary group. A corollary of Theorem is that conjugate subgroups H and H 2 (where H 2 = gh g for some g G) produce exactly the same distribution over ρ and hence cannot be distinguished by this process. In particular, the hidden subgroup problem cannot be solved by Algorithm.2 for a group G with two distinct conjugate subgroups H,H 2 ; the symmetric group S n is such a group. In light of this, one may ask whether Algorithm.2 can distinguish between a coset ch of a non-trivial subgroup H and a coset ch e = {c} of the trivial subgroup H e = {e}, as even this would be enough for solving Graph Isomorphism. However, even for this weaker problem we show: Theorem 3. For the symmetric group S n, Algorithm.2 does not distinguish (even information theoretically) the case that the hidden subgroup is the trivial subgroup from the case that the hidden subgroup is non-trivial. (Specifically, the distributions induced on ρ in these two cases have exponentially small total variation distance.). Related Work. Simon s algorithm [24] implicitly involves distinguishing the trivial subgroup from an order 2 subgroup over the group Z n 2. He shows that a classical probabilistic oracle machine would require exponentially many oracle queries to successfully distinguish the two cases with probability greater than /2. Shor [23] generalizes Simon s algorithm to solve integer factorization and the discrete log problem. In addition to solving a special case of the HSP, he also solves specific cases when the underlying group is not even known. Boneh and Lipton [3] handle a case when a periodic function is not fixed on a coset. Hales and Hallgren [, 2] generalize the results for the case when the underlying Abelian group is unknown, but an estimate is known for the cardinality of its cyclic factors. Kitaev [5] gave an algorithm using eigenvalue estimation for the Abelian stabilizer problem, a problem closely related to the HSP. Mosca and Ekert used eigenvalue estimation to solve the Abelian HSP. The efficient algorithm for the Abelian HSP using the Fourier transform is folklore. As for computing the Fourier transform efficiently, Kitaev also shows how to efficiently compute the Fourier transform over any Abelian group. The fastest known (quantum) algorithm for computing the Fourier transform over Abelian groups is given by Hales and Hallgren [2]. Beals [2] showed how to efficiently compute the Fourier transform over the symmetric group S n. Ettinger, Høyer and Knill [8] show that the HSP has polynomial query complexity. Ettinger and Høyer [5] give a solution for the HSP over the dihedral group D n with polynomially many queries and exponential (classical sequential) time. In [6] and [7] they address whether any measurement will distinguish subgroup states. Roetteler and Beth [9] give a solution to the HSP for a specific non-abelian group. 3

4 Grigni, Schulman, Vazirani and Vazirani [0] independently showed that measuring the representation is not enough for graph isomorphism, and give stronger negative results. They establish the same bounds even when the row of the representation (i.e., i in Algorithm.2 above) is measured, and similar bounds if the column ( j) is measured, under the assumption that random bases are selected for each representation. They also show that the problem can be solved when the intersection of the normalizers of all subgroups of G is large. 2 Representation Theory Background The main tool used by polynomial time quantum algorithms is the Fourier transform. To define the Fourier transform (over a general group) we require the basic elements of representation theory, defined briefly below. For complete accounts, see [22] or [4]. Representation. A representation ρ of a finite group G is a homomorphism ρ : G GL(V), where V is a (finite-dimensional) vector space over C. Fixing a basis for V, each ρ(g) may be realized as a d d matrix over C, where d is the dimension of V. As ρ is a homomorphism, for any g,h G, ρ(gh) = ρ(g)ρ(h) (this second product being matrix multiplication). The dimension d ρ of the representation ρ is d, the dimension of V. A representation provides a means for investigating a group by homomorphically mapping it into a family of matrices. With this realization, the group operation is matrix multiplication and tools from linear algebra can be applied to study the group. We shall be concerned with complex-valued functions on a group G; the representations of the group are relevant to this study, as they give rise to the natural Fourier transform in this non-abelian setting. We say that two representations ρ : G GL(V) and ρ 2 : G GL(W) of a group G are isomorphic when there is a linear isomorphism of the two vector spaces φ : V W for which g G,φρ (g) = ρ 2 (g)φ. In this case, we write ρ = ρ2. Irreducibility. We say that a subspace W is an invariant subspace of a representation ρ if ρ(g)w W for all g G. We assume, without loss of generality, that every ρ(g) is unitary, and, in particular, diagonalizable. Hence there are many subspaces fixed by an individual matrix ρ(g). In order for W to be an invariant subspace for ρ, it must be simultaneously fixed under all ρ(g). The zero subspace and the subspace V are always invariant. invariant, the representation is said to be irreducible. If no nonzero proper subspaces are Decomposition. When a representation does have a nonzero proper invariant subspace V V, it is always possible to find a complementary subspace V 2 (so that V = V V 2 ) which is also invariant. Since ρ(g) fixes V, we may let ρ (g) be the linear map on V given by ρ(g). It is not hard to see that ρ : G GL(V ) is in fact a representation. Similarly, define ρ 2 (g) to be ρ(g) restricted to V 2. Since V = V V 2, the linear map ρ(g) is completely determined by ρ (g) and ρ 2 (g), and in this case we write ρ = ρ ρ 2. There is then a basis for V in which each matrix ρ(g) is block diagonal (with two blocks). Complete reducibility. Repeating the process described above, any representation ρ may be written ρ = ρ ρ 2... ρ k, where each ρ i is irreducible. In particular, there is a basis in which every matrix ρ(g) is block diagonal, the ith block corresponding to the ith representation in the decomposition. While this decomposition is not in general unique, the number of times a given irreducible representation appears in this decomposition (upto isomorphism) depends only on the original representation ρ. 4

5 Characters. The character χ ρ : G C of a representation ρ is defined by χ ρ (g) = tr (ρ(g)), where tr () denotes the trace. It is basis independent, and, as it turns out, completely determines the representation ρ. Elementary properties of trace show that that these characters are in fact class functions, i.e., they are fixed on conjugacy classes: for every g and h we have χ ρ (hgh ) = χ ρ (g). Orthogonality of characters. For two functions f and f 2 on a group, there is a natural inner product: f, f 2 G given by g f (g) f 2 (g). The fact of principal concern to us is the following: given the character χ ρ of any representation ρ and the character χ i of any irreducible representation ρ i, the inner product χ ρ χ i is precisely the number of times the representation ρi appears in the decomposition of ρ. Since each ρ is unitary, the inner product of two characters simplifies slightly: χρ χ i G = χ ρ (g)χ i (g ). g G Orthogonality of the second kind. Let C be a conjugacy class of G. As mentioned before, any character ρ is a class function and thus is fixed on any conjugacy class C; we denote this value by ρ(c). It holds that ρ(c) 2 = C () This is a special case of a more general principle, and we refer the interested reader to Sagan s excellent book ([2], Theorem.0.3). Restriction. A representation ρ of a group G is also automatically a representation of any subgroup H. We refer to this restricted representation on H as Res H ρ. Note that even representations which are irreducible over G may be reducible when restricted to H. Induction. There is a dual notion, that of induction, whereby a representation of a subgroup H < G may be induced to a representation of the whole group G. We delay discussion of this to Section 3.. Up to isomorphism, a finite group has a finite number of irreducible representations; we let Ĝ denote this collection (of representations). As mentioned above, any representation ρ of G may be decomposed into a direct sum of the irreducible representations in Ĝ. In fact, reiterating the comments above, if ρ,...,ρ k are the irreducible representations of G and χ ρ is the character of ρ, the value n i = χ ρ,χ i is precisely the number of times the irreducible representation ρ i appears in the decomposition of the representation ρ into irreducible representations. That is, after a unitary change of basis, the diagonal of the matrix ρ(g) consists of n copies of ρ (g), followed by n 2 copies of ρ 2 (g), etc. We denote this state of affairs by ρ = n ρ n k ρ k. There are two representations which shall play a central role in our discussion: The Trivial Representation. The trivial representation G maps every group element g G to the by matrix (). One feature of the trivial representation is that g G (g) is the matrix (); this sum is the zero matrix for any other irreducible representation. 5

6 The Regular Representation. Fix a vector space V with a basis vector e g, one for every element g G. The regular representation reg G : G GL(V) is defined by reg G (g) : e x e gx, for any x G. It has dimension and, with the basis above, reg G (g) is a permutation matrix for any g G. An important fact about the regular representation is that it contains every irreducible representation of G. In particular, if ρ,...,ρ k are the irreducible representations of G with dimensions d ρ,...,d ρk, then ρ reg = d ρ ρ d ρk ρ k, that is, the regular representation contains each irreducible representation ρ exactly d ρ times. Counting dimensions yields an important relation between the dimensions d ρ and the order of the group: = dρ 2. (2) The main tool in quantum polynomial time algorithms is the Fourier transform. When G is non-abelian, this takes the form described below. Definition 2.. Let f : G C. The Fourier transform of f at the irreducible representation ρ, denoted ˆf(ρ), is the d ρ d ρ matrix ˆf(ρ) = d ρ f(g)ρ(g). g G, i, j dρ We refer to the collection of matrices ˆf(ρ) as the Fourier transform of f. Thus f is mapped into Ĝ matrices of varying dimensions. The total number of entries in these matrices is dρ 2 =, by equation (2) above. The Fourier transform is linear in f ; with the constants used above (i.e. d ρ /) it is in fact unitary, taking the complex numbers f(g) g G to complex numbers organized into matrices. A familiar case in computer science is when the group is cyclic of order n. Then the linear transformation (i.e., the Fourier transform) is a Vandermonde matrix with n-th roots of unity and the matrices are -by-. In the quantum setting we identify the superposition g G f g g with the function f : G C defined by f(g) = f g. In this notation, g G f(g) g is mapped under the Fourier transform to ˆf(ρ) i, j ρ,i, j. We remind the reader that ˆf(ρ) i, j is a complex number. When the first portion of this triple is measured, we observe ρ Ĝ with probability i, j d ρ ˆf(ρ) i, j 2 = ˆf(ρ) 2 where A is the natural norm given by A 2 = tr A A. Let f be the indicator function of a left coset of H in G, i.e. for some c G, if g ch, and f(g) = H 0 otherwise. Our goal is to understand the Fourier transform of f, as this determines the probability of observing ρ, which is ˆf(ρ) 2 = i, j ( ˆf(ρ)) i, j 2. Our choice to measure only the representation ρ (and not the matrix indices) depends on the following key fact about the Fourier transform, also relevant to the Abelian solution: Claim 2.. The probability of observing ρ is independent of the coset. 6

7 dρ Proof. ˆf(ρ) = H h H ρ(ch) = ˆf(ρ) 2 = d ρ dρ H ρ(c) h H ρ(h) and, since ρ(c) is a unitary matrix, H ρ(c) ρ(h) h H 2 = d ρ H 2 ρ(h). h H Given this we may assume, without loss of generality, that our function f is constant and positive on the subgroup H itself, and zero elsewhere. 3 The Probability of Measuring ρ The primary question is that of the probability of observing a given ρ in the experment described in Algorithm.2. We have seen that this is determined by h H ρ(h) which, being a sum of the linear transformations ρ(h), is a linear transformation. We begin by showing that it is a projection: Lemma 3.. Let ρ be an irreducible representation of G. For every subgroup H G, H h H ρ(h) is a projection operator. With the right basis, then, H h H ρ(h) will be diagonal, each diagonal entry being either one or zero. The probability of observing a particular representation ρ then corresponds to the number of ones appearing on the diagonal, i.e., on the rank of ˆf(ρ). Proof of Lemma 3.. Given an irreducible representation ρ of G, we are interested in the sum of the matrices ρ(h) over all h H. Since we only evaluate ρ on H, we may instead consider Res H ρ without changing anything. As mentioned before, though ρ is irreducible (over G), Res H ρ may not be irreducible on H. We may, however, decompose Res H ρ into irreducible representations over H. Then the h H ρ(h) is comprised of blocks, each corresponding to a representation in the decomposition of Res H ρ. In particular, the matrix h H ρ(h) is: h H σ (h) h H σ 2 (h) 0 U U (3) 0 0 h H σ t (h) for some unitary transformation U and some irreducible representations σ i of H (with possible repetitions). (Here denotes conjugate transpose.) We know that the sum is nonzero only when the irreducible representation is trivial, in which case it is H. As in the previous section, we let f : G C be the function f(g) =, for g H, and 0 otherwise. H Then the probability of observing ρ in Algorithm.2 is ( ˆf(ρ)) 2 = d ρ H ρ(h) h H We record this result in the following theorem. 2 = d ρ H H 2 χ ρ,χ H H = H d ρ χ ρ,χ H H. 7

8 Theorem 4. For every subgroup H G, the probability of measuring ρ in Algorithm.2, with hidden subgroup H, is ˆf(ρ) 2 = H d ρ χ ρ,χ H H. Observe that one consequence of the theorem is that the probability of observing a representation ρ depends only on the character of ρ. As the characters are class functions, conjugate subgroups (two subgroups H and H 2 are conjugate if H = gh 2 g for some g G) produce exactly the same distribution over ρ; this rules out using the paradigm of Algorithm.2 with representations names alone to solve the HSP for any group containing a non-normal subgroup. 3. Induced Representations We have discussed the restriction of a representation ρ of G to a subgroup H of G. There is a dual operation, called induction. This extends to all of G a representation ρ defined on a subgroup H. We will only need to work with the representation induced from the trivial representation on H. Let G/H def = {α,...,α t } be a canonical collection of representatives for the left cosets of H in G, so that G = α H α t H, this union being disjoint. Then the induced representation Ind G H H : G GL(W) is defined over the vector space W that has one basis vector e [αi ] for each coset α i H. It is defined by linearly extending the rule Ind G H H(g) : e [αi ] e [gαi ] = e [α j ] where gα i belongs to the coset α j H. Observe that this representation is a permutation representation. As suggested by the notation, the representation is independent of the choice of α i. Example 3.. Ind G {id} {id} = reg G. We now invoke a standard representation-theoretic result to obtain Theorem. Lemma 3.2. (A special case of Frobenius reciprocity) [4, 3.20]): Let H < G and let ρ : G GL(V) be an irreducible representation of G. Then χh χ ρ H = χ Ind GH H χ ρ G, the first inner product being computed over H and the second over G. Combining this with Theorem 4 establishes Theorem. Proof of Theorem. Theorem 4 asserts that the probability of measuring the representation ρ in Algorithm.2 is H d ρ χ χ ρ. By reciprocity, the number of times that the trivial representation of H appears H in Res H ρ is the same as the number of times that ρ appears in Ind G H H, that is χ χ ρ H = χ Ind G H H χ ρ G. Theorem follows. 4 A Positive Result: Normal Subgroups and the Core of H In this section we show that O(log ) queries suffice to reconstruct any normal subgroup. In general, we show that for any subgroup H of G, the algorithm outputs H G, the core of H, which is the largest subgroup of H that is normal in G. As the product H H 2 of two normal subgroups is again a normal subgroup, the core is well-defined. (In fact, the core is precisely g G ghg.) The algorithm we study is the following: Algorithm 4.. 8

9 Setting: H is an arbitrary unknown subgroup of G, and, We are given an efficiently computable function f : G S, that is constant on (left) cosets of H and takes distinct values on distinct cosets. Algorithm: For i =,...,s = 4log 2, Output: N = N s. Run Algorithm.2 and measure an irreducible representation, σ i. Let N i = i j= ker σ j, where ker(σ) = {g G σ(g) = V } is the kernel of σ. Observe that each ker σ i is a normal subgroup of G, so that the resulting subgroup N = N s is normal. We will show that Algorithm 4. converges quickly to H G with high probability in Theorem 5. We reduce the proof of this theorem to two lemmas, described in the next section. Two different sets of proofs of these lemmas are then presented in Sections 4.2 and 4.3, one from the perspective of restricted representations, and one from the perspective of induced representations. 4. The General Structure As discussed above, Theorem 5 is a consequence of the following two lemmas. Lemma 4.. If the irreducible representation σ can be sampled by Algorithm.2 then H G ker(σ). This shows, in particular, that H G N s... N. Lemma 4.2. For any subgroup H G, if N i H then Pr[N i+ = N i ] 2. Before discussing the proofs of these lemmas, we show that together they imply Theorem 5. Theorem 5. Algorithm 4. returns H G with probability 2e log 2 /8. Proof of Theorem 5. LetD H denote the probability distribution over irreducible representations induced by Algorithm.2. We now apply a standard Martingale bound (see [8]) to prove the theorem (based on Lemmas 4.2 and 4.). Let σ,...,σ k be independent random variables distributed according tod H with k = 4log 2. Our goal is to show that Pr[N s H G ] 2e log 2 /8. For each i {,...,k}, let X i be the indicator random variable taking value if N i H or N i+ N i, and zero otherwise. The random variables X,...,X k are not necessarily independent, but by the previous claim, Pr[X i = 0 X,...,X i ] 2, and we can use a martingale bound. As the variables take values in the set {0,}, the sum i X i satisfies the Lipschitz condition (with constant ), and we can apply Azuma s Inequality to conclude that i X i is unlikely to deviate far from its expected value, which k is [ at least 2. In particular, ] we have Pr[ i X i k 2 λ] 2e λ2 /2k, so with λ = log 2 () we have Pr k i=0 X i log 2 () 2e log 2 ()/8. Therefore, except with probability 2e log 2 ()/8 we have N s H. Now, as N s is normal in G, it must be the case that N s H G. From Lemma 4., we know that H G N s. Hence N s = H G, and the algorithm converges to the correct subgroup. 9

10 4.2 Restricted Representations We begin by proving Lemmas 4. and 4.2 from the perspective of restricted representations. By Claim 2., we may assume f is distributed over H itself without loss of generality. In this case, Lemma 3. implies that, up to a scalar multiple, each ˆf(σ) is a projection. We will now show that when the subgroup H is normal, ˆf(σ) is in fact a multiple of the identity, and is nonzero when H is in the kernel of σ: Lemma 4.3. Let H G and ρ : G GL(V) be a irreducible representation of G. Then { c I if H kerρ, where c = H ˆf(ρ) = d ρ, and 0 matrix otherwise. Proof. The lemma follows from an application of Schur s lemma (see, e.g., [25]). We give a complete proof. Two alternative proofs are in [3]. If H ker ρ then the lemma (and the value of c) follows from the discussion in the previous section. Suppose H ker ρ. We will show that ˆf(ρ) must be the zero map. Let ρ g = ρ(g). By Lemma 3. we can decompose V as W f Wa, where H h H ρ h fixes W f and annihilates W a. Our goal is to see that W f = {0}. Observe that since H kerρ, there is some h 0 H for which ρ(h 0 ) is not the identity operator and, considering that each ρ(h) is unitary, their average cannot be the identity operator. (Specifically, consider a unit vector v for which ρ(h 0 )v v; note now that the average over h of ρ(h)v can have unit length only if h,h 2,ρ(h )v = ρ(h 2 )v v.) Hence W f V. Assume that W f {0}. Since ρ is irreducible over G and W f V, there is a vector w f W f and g G such that ρ g w f W f ; we may write ρ g w f = w f + w a, with w a W a, w f W f, and w a 0. As H h H ρ h fixes W f we have w f + w a = ρ g w f = ρ g H ρ h w f h H = H ρ h ρ g w f = h H H ρ h (w f + w a ) = w a, h H where equality = follows since H is normal in G. This is a contradiction; hence W f = {0}, as desired. We will now prove Lemma 4., which states that if the irreducible representation σ can be sampled by Algorithm.2 then H G ker(σ). Proof of Lemma 4.. Let C be a set of coset representatives for H G in H. We have h H ρ(h) = ( c C ρ(c))( h H G ρ(h)), so by Lemma 4.3 we only observe ρ if h H G ρ(h) is a multiple of the identity, i.e. only if H G kerρ. Before proving Lemma 4.2, which is for general subgroups, we will show how the statement can be proved for normal subgroups. Lemma 4.4. If H G and N i H then Pr[N i+ = N i ] 2. Proof. By Lemma 4.3, Theorem 4 and Equation 2 we have: Pr[N i kerρ i+ ] = N i kerρ Pr[Observe ρ] = N i kerρ H d2 ρ = H /N i dρ 2 = H N i 2, where changing the sum follows from the fact that representations of G that map N i to the identity can be identified with representations of G/N i. 0

11 We will now prove Lemma 4.2, which states that for any H G, if N i H then Pr[N i+ = N i ] 2. Proof of Lemma 4.2. This proof is due to Vazirani [26]. Let N be the intersection of the kernels so far. For an irreducible representation ρ, let r ρ be the rank of ˆf(ρ), i.e., the number of times the trivial representation of H appears in ρ. When N H, we will show that the probability of N being contained in the kernel of the H next representation we measure is at most /2, by showing that ρ:n ker ρ d ρr ρ N H N, which is at most HN /2 when N H. Now, if the hidden subgroup had been HN, Theorem 4 would imply ρ: N d ρr ρ =, where r ρ is the number of times the trivial representation of HN appears in ρ. Note that r ρ = r ρ when N kerρ, since H N l HN ρ(l) = ( h H ρ(h))( n N ρ(n)), and ρ(n) is the identity. Since HN H H N = H N, we have that ρ:n ker ρ d ρr ρ H d ρr ρ H N N, as desired. 4.3 Induced Representations We now reprove these two lemmas from the perspective of induced representations. We begin by computing the kernel of the representation Ind G H H. Lemma 4.5. ker(ind G H H) = H G. Proof. ker(ind G H H ) H G. Indeed, if x ker(ind G H H ) then Ind G H H (x) is the identity mapping, i.e., for every g G, Ind G H H(x) : [gh] [gh], or equivalently, [xgh] = [gh]. In particular, for g = e we get xh = H, and therefore x H. Now, as ker(ind G H H) is normal and is contained in H we must have ker(ind G H H) H G. H G ker(ind G H H ). Suppose x H G. Then for any g G, there is some x H G H such that xg = gx. Therefore, Ind G H H (x)[gh] = [xgh] = [gx H] = [gh], and we see that Ind G H H (x) is the identity mapping. Hence, x ker(ind G H H ). Now, by Theorem, any σ that can be sampled by Algorithm.2 appears in Ind G H H we therefore conclude that H G ker(ind G H H) ker(σ), and Lemma 4. follows. This also gives a simple decomposition of Ind G H H when H is normal: Lemma 4.6. Let N G. Then Ind G N N =,N ker(n) d ρρ. Proof. Suppose Ind G N N = n ρρ. We have: n ρ = χ Ind G N N,χ ρ G = χ,χ ρ N = N χ ρ (x) = d ρ, x N where the second equality is by Frobenius reciprocity, and the last one is because N kerρ. We now prove Lemma 4.2: Proof of Lemma 4.2. Denote N = N i. For ρ Ĝ let m ρ = χ ρ,χ H H. We know that Pr [N i ker σ] = σ D H H N i ker(ρ) Ind G H H = m ρ ρ, and m ρ d ρ, Ind G N N = N ker(ρ) d ρ ρ,

12 where the first equation is by Theorem 4, the second because m ρ = χ ρ,χ H H = χ Ind G H H,χ ρ G by Frobenius reciprocity (Lemma 3.2), and the last one by lemma 4.6. We observe that χ Ind G H H,χ Ind G N N G = d ρ m ρ χ ρ,χ ρ G N ker(ρ) = N ker(ρ) d ρ m ρ and thus is proportional to the probability N i ker(σ i ). We complete the proof with an argument similar to that used in Serre [22] in the proof of Mackey s criterion. By Frobenius reciprocity χ Ind G H H,χ Ind G N N G = χ H,χ ResH Ind G N N H. Decomposing the restricted induction (see [22, 7.4]) we have χ Ind G H H,χ Ind G N N G = g H\G/N χ H,χ Ind H Hg H where H g is a subgroup of H, and g is runs over all representatives of the double cosets H \g/n of G. Using Frobenius reciprocity again we see that χ Ind G H H,χ Ind G N N G = g H\G/N = H \ G/N. χ H,χ H Hg However, N is normal in G. Hence for any g G, H \ g/n = HNg. Furthermore, as H is a group and N is normal in G, HN is also a group. Hence, H \ G/N = / HN. Thus, which is at most half when N H. Pr [N i kerσ] = σ D H H = H N i ker(ρ) HN m ρ d ρ = H HN 5 A Negative Result: Determining Triviality in S n In this section we show that a well-known reduction of graph isomorphism to finding a hidden subgroup over S n will not work using Algorithm.2. Graph Automorphism is the problem of determining if a graph G has a nontrivial automorphism, and is easier than Graph Isomorphism [6]. A natural special case occurs when the graph G consists of two disjoint connected rigid graphs G,G 2 (i.e., Aut(G ) = Aut(G 2 ) = {e}). In this case there are two possibilities for the automorphism group of G: 2

13 Claim 5.. If G G 2, then Aut(G) = {e}. If G G 2 then Aut(G) = {e,σ} where σ S n is a permutation with n/2 disjoint 2 cycles. Proof. For the first part notice that any automorphism maps a connected component onto a connect component. In our case we have two connected components G and G 2. However, G and G 2 are not isomorphic and have no non-trivial automorphisms. For the second part, let σ reflect an automorphism between G and G 2. Now, suppose there was another non-trivial automorphism τ. Then στ is also an automorphism, and στ maps the connected component of G onto G, and G 2 onto G 2. As G and G 2 have no non-trivial automorphisms it follows that στ =, τ = σ = σ. Thus, if one knows how to solve the HSP for S n, or if one knows how to distinguish between cosets of a trivial subgroup and the cosets of a non-trivial subgroup, one can give an efficient quantum algorithm for Graph Automorphism. In particular, one might try the following algorithm for reconstructing H = Aut(G). Algorithm 5.. Input a graph G s.t. either Aut(G) = {e} or Aut(G) = {e, σ}.. Compute Σ π Sn π,π(g) and measure the second register π(g). The resulting superposition is Σ h H ch f(ch) for some coset ch of H. Furthermore, c is uniformly distributed over G. 2. Compute the Fourier transform of the coset state, which is d ρ H ( ) ρ(ch) ρ,i, j. h H i, j 3. Measure the first register and observe a representation ρ. We show that even for this particular case of Graph Isomorphism (and Graph Automorphism) the algorithm fails. Theorem 6. Let G and G 2 be two rigid, connected graphs with n vertices. LetD N (ρ) be the probability of sampling ρ in Algorithm 5. when G G 2, andd I (ρ) the probability when G G 2. Then D N D I 2 Ω(n). Proof. We present the proof from [0] which simplifies the proof of [3]. When G G 2, H = {e}, so D N (ρ) = d 2 ρ /n! by Theorem 4. When G G 2, and G and G 2 are both connected and rigid, H = {e,τ}. By Theorem 4, H has only two elements, e and τ, hence χ χ ρ D I (ρ) = H d ρ χ χ ρ H. H = 2 (χ ρ(e)+χ ρ (τ)) = 2 (d ρ + χ ρ (τ)). 3

14 That is,d I (ρ) = d ρ n! (d ρ + χ ρ (τ)) and so, D I (ρ) D N (ρ) = ρ n! ρ n! ρ d 2 ρ ρ d ρ χ ρ (τ) χ ρ (τ) 2 n! ρ χ ρ (τ) 2 by the Cauchy-Schwartz inequality and Equation (2). By Equation (), χ ρ(τ) 2 = / {τ} where {τ} is the conjugacy class of τ. However, two permutations share the same conjugacy class if and only if they have the same cycle decomposition. In our case τ has cycle decomposition into n/2 pairs. Thus {τ} = ( n ) n/2 ( n 2 )! 2 n/2 where ( ) n n/2 is the number of possibilities for choosing the first element in each of the n/2 pairs, ( n 2 )! is the number of possibilities for arranging the remaining n/2 elements in the pairs, and each ordering is counted exactly 2 n/2 times. Altogether, as desired. D I (ρ) D N (ρ) ρ n! = n! {τ} 2 (n/2) (n/2)! n! 2 Ω(n), 6 Finding Hidden Subgroups in Hamiltonian Groups A group G is Hamiltonian if all subgroups are normal. In light of Theorem 4, a hidden subgroup of a Hamiltonian group G is determined with high probability by O(log ) samples of the distribution induced by Algorithm.2. In this section we show that, for Hamiltonian groups, generators for the hidden subgroup can be computed efficiently from these samples. As the Fourier transform over such groups can be efficiently computed, this gives an efficient quantum algorithm for the HSP over Hamiltonian groups. All Abelian groups are Hamiltonian; the only non-abelian Hamiltonian groups are of form G = Z k 2 B Q, where Q = {±,±i,± j,±k} is the quaternian group and B is an Abelian group with exponent b coprime with 2. For a detailed description of such groups, see Rotman s excellent book [20]. We begin by briefly discussing the case when G is Abelian. If G is simply the cyclic group Z n, the representations are the functions ρ s : z exp(2πisz/n) and the reconstruction algorithm, when it succeeds, yields a collection {ρ s s S} with the property that H = s S ker ρ s. Observe that ρ s (h)ρ t (h) = ρ s+t mod n (h) and that ρ s (h) = implies ρ st mod n (h) = for all t Z. Hence s kerρ s = kerρ d where d is the greatest common divisor of n and the elements in S. Then H is the cyclic subgroup of Z n generated by n/d. In general, an Abelian group G is isomorphic to a direct sum Z n Z nk, and we assume that this decomposition is known. The irreducible representations are the functions ρ s,...,s k (z,...,z k ) = k j= exp(2πis jz j /n j ), and, as above, we begin with a collection {ρ s s = (s,...,s k ) S} so that H = 4

15 S kerρ s. Then (h,...,h k ) H s S, exp ( j 2πis j h j n j ) = s S, s j q j h j 0 mod N, (4) j where N is the least common multiple of the n j, and q j = N/n j. For convenience, we treat the family of equalities appearing in line (4) as a system of equations over the ring Z N ; then a solution h of this system corresponds to the element (h mod n,...,h k mod n k ) of H. Collect these equations together into a matrix R. Though Z N may not be a field, it is easy to check that a matrix over Z N may be diagonalized in polynomial time with the following two operations: for some pair i j, swap row (column) i with row (column) j, for some pair i j, add a multiple of row (column) i to row (column) j. This results in a system D F h = 0, where D is diagonal and F is invertible. Any vector h for which D h = 0 may be then transformed into a solution h of the original equation and, moreover, if h is selected at random in the null space of D, then the resulting h will give rise to a random element of H. Selection of O(log ) random elements in this way yields a generating set for H with high probability. Finally, consider a Hamiltonian group of form G = Z k 2 B Q. An irreducible representation ρ of G is a tensor product ζ β κ where ζ Ẑk 2, β ˆB, and κ ˆQ. (As Z k 2 and B are Abelian, in this case the tensor product may be replaced with the regular product in C.) We briefly review the representation theory of the quaternian group. Q has 5 irreducible representations: four one-dimensional and one two-dimensional. The one-dimensional representations arise as the irreducible representations of the Abelian quotient Q/{±} = Z 2 Z 2. The two-dimensional representation τ realizes Q as a subgroup of SU 2, where [ 0 τ() = 0 [ i 0 τ( j) = 0 i ] [ 0, τ(i) = 0 ], τ(k) = [ 0 i i 0 ], ], and τ( q) = τ(q) for each q Q. As above, we assume that we have a set of samples S so that H = ζ β κ S ker(ζ β κ). It is sufficient to show that for a given element q Q, one can generate a collection of random elements of H Z k 2 B {q}, for if these collections are large enough, then their union yields a set of generators for H with high probabilty. Fixing an element q Q, consider a specific sample ζ β κ. There are two cases to consider: If κ is one-dimensional, the condition ζ(z) β(b) κ(q) = may be interpreted as an equation over Z N, where N = b2 k+, as in the Abelian case above. (Note that κ(q) = ± contributes a constant to the equation; as 2 N, this constant can be suitably represented as exp(2πit/n) for t = N/2.) 5

16 If κ is two-dimensional, the condition ζ(z) β(b) κ(q) = [ 0 0 cannot be satisfied unless q = ±. When q = ±, this may be interpreted as a pair of equations over Z N, where N = 2 k+ b, each equation corresponding to a diagonal entry of the matrix. (Note that κ(q) contributes a constant ± to each equation; as 2 N, these constants can be suitably represented as exp(2iπt/n) for t = N/2.) Now the solution proceeds as in the Abelian case. For each q Q, the above procedure is used to compute clog random elements of H Z n 2 B {q} (unless this intersection is empty). If c is chosen appropriately, the union of these sets generates H with high probability. 7 Acknowledgement The authors thank Umesh Vazirani for many helpful discussions and the simplification of several of the proofs. ] References [] Noga Alon and Joel H. Spencer. The Probabilistic Method. John Wiley & Sons, Inc., 992. [2] Robert Beals. Quantum computation of Fourier transforms over symmetric groups. In Proceedings of the Twenty-Ninth Annual ACM Symposium on Theory of Computing, pages 48 53, El Paso, Texas, 4 6 May 997. [3] Dan Boneh and Richard J. Lipton. Quantum cryptanalysis of hidden linear functions (extended abstract). In Don Coppersmith, editor, Advances in Cryptology CRYPTO 95, volume 963 of Lecture Notes in Computer Science, pages Springer-Verlag, 27 3 August 995. [4] Persi Diaconis and Daniel Rockmore. Efficient computation of the Fourier transform on finite groups. J. Amer. Math. Soc., 3(2): , 990. [5] Mark Ettinger and Peter Høyer. On quantum algorithms for noncommutative hidden subgroups. In Symposium on Theoretical Aspects in Computer Science, University of Trier, 4 6 March 999. [6] Mark Ettinger and Peter Høyer. Quantum state detection via elimination. Technical report, quantph/ , 999. [7] Mark Ettinger and Peter Høyer. A quantum observable for the graph isomorphism problem. Technical report, quant-ph/990029, 999. [8] Mark Ettinger and Peter Høyer and Emanuel Knill. Hidden subgroup states are almost orthogonal. Technical report, quant-ph/990034, 999. [9] M. Goldmann and A. Russell. The computational complexity of solving systems of equations over finite groups. In Fourteenth Annual IEEE Conference on Computational Complexity, Atlanta, Georgia, 4 6 May

17 [0] Michaelangelo Grigni, Leonard Schulman, Monica Vazirani and Umesh Vazirani. Quantum mechanical algorithms for the nonabelian hidden subgroup problem. In Proceedings of the Thirty-Third Annual ACM Symposium on Theory of Computing, Crete, Greece, 6 8 July 200. [] Lisa Hales and Sean Hallgren. Quantum fourier sampling simplified. In Proceedings of the Thirty-First Annual ACM Symposium on Theory of Computing, pages , Atlanta, Georgia, 4 May 999. [2] Lisa Hales and Sean Hallgren. An Improved Quantum Fourier Transform Algorithm and Applications. In Proceedings of the 4st Annual Symposium on Foundations of Computer Science, Redondo Beach, California, 2 4 November [3] Sean Hallgren, Alexander Russell, and Amnon Ta-Shma. Normal Subgroup Reconstruction and Quantum Computation Using Group Representations. In Proceedings of the Thirty-Second Annual ACM Symposium on Theory of Computing, pages , Portland, Oregon, 2 23 May [4] Joe Harris and William Fulton. Representation Theory. Number 29 in Graduate Texts in Mathematics. Springer-Verlag, New York, NY, 99. [5] Alexey Yu. Kitaev. Quantum measurements and the abelian stabilizer problem. Technical report, quant-ph/95026, 995. [6] Johannes Köbler, Uwe Schöning, and Jacobo Torán. The graph isomorphism problem: its structural complexity. Birkhäuser Boston Inc., Boston, MA, 993. [7] Michele Mosca and Artur Ekert The Hidden Subgroup Problem and Eigenvalue Estimation on a Quantum Computer. In C.P. Williams, editor, Proceedings if the st NASA International Conference on Quantum Computing and Quantum Communications, volume 509 of Lecture Notes in Computer Science, pages 74 88, 999. [8] Rajeev Motwani and Prabhakar Raghavan. Randomized Algorithms. Cambridge University Press, Cambridge, England, 995. [9] Martin Rötteler and Thomas Beth. Polynomial-time solution to the hidden subgroup problem for a class of non-abelian groups. Technical report, quant-ph/982070, 998. [20] Joseph Rotman. An Introduction to the Theory of groups. Number 48 in Graduate Texts in Mathematics. Springer-Verlag, New York, NY, 995. [2] B. E. Sagan The Symmetric Group: Representations, Combinatorial Algorithms, and Symmetric Functions. [22] J. P. Serre. Linear representations of finite groups, Number 42 in Graduate Texts in Mathematics. Springer-Verlag, New York, NY, 977. [23] Peter W. Shor. Polynomial-time algorithms for prime factorization and discrete logarithms on a quantum computer. SIAM Journal on Computing, 26(5): , October 997. [24] Daniel R. Simon. On the power of quantum computation. SIAM Journal on Computing, 26(5): , October 997. [25] Audrey Terras. Fourier Analysis on Finite Groups and Applications Number 43 in London Mathematical Society Student Texts. Cambridge University Press, 999. [26] Umesh Vazirani. Personal Communication,