Symmetries in the Entropy Space

Transcription

1 Symmetries in the Entropy Space Jayant Apte, Qi Chen, John MacLaren Walsh Abstract This paper investigates when Shannon-type inequalities completely characterize the part of the closure of the entropy region n that is symmetric under the action of a specified rom variable permutation group This question is answered exhaustively for every group permuting n 4 n 5 rom variables, while multiple examples for arbitrary n 6 are provided for both tightness non-tightness of the Shannontype inequalities For instance, a new result shows that non- Shannon-type inequalities are necessary under the action of the cyclic group C n, the dihedral group D n, as well as S C n S D n, for every n 6 I INTRODUCTION Let N be a set with cardinality n H n R 2N For any h 2H n, we call h entropic if there exits a set of n jointly distributed discrete rom variables X N, {X i : i 2 N} such that ha) HX A ) for any A N, where X A, {X i : i 2 A} HX ; )0by convention The vector h is called the entropy function of X N so H n is called the entropy space of order n Let n be the set of all entropy functions n be the closure of n, the set of almost entropic functions For all A, B N, entropy is a non-negative ha) 0), non-decreasing ha) apple hb) if A B), submodular ha)+hb) ha \ B)+hA [ B)) function, thus a polymatroid [2] The set of all polymatroids, ie the set of all non-negative, non-decreasing, submodular set functions on N, is denoted by n The inequalities bounding n are also called Shannon-type information inequalities since they can be derived by the nonnegativity of Shannon measures [, Chapter 3-5] Due to the existence of infinitely many linear non-shannon-type inequalities [3], [6] when n 4, n is a non-polyhedral convex cone with n n for n 4, whose boundaries have yet to be exhaustively characterized It is thus a fundamental open problem in information theory to characterize n n for n 4, as they determine all fundamental information inequalities It has been shown that characterizing the entropy region is equivalent to determining the capacity regions of all networks information storage systems under coding, determining all fundamental inequalities regarding sizes of subgroups in group theory, as well as several other fundamental problems [, Chapter 3-6, 2] All three authors were with the Dept of ECE, Drexel University, this work was supported by the National Science Foundation under the award CCF-422 Jayant Apte John MacLaren Walsh are now with the Dept of ECE, Drexel University jsa46@drexeledu, jwalsh@coedrexeledu) Qi Chen has now returned to the Institute of Network Coding, The Chinese University of Hong Kong qichen@iecuhkeduhk), where he has received additional support via a grant from the University rant Committee of the Hong Kong Special Administrative Region Project No AoE/E-02/0) Shenzhen Key Laboratory of Network Coding Key Technology Application, Shenzhen, China ZDSY ) In this paper, we wish to contribute to this problem by studying the part of the entropy region n consisting of those almost entropic vectors which obey a specified collection of symmetries As a collection of symmetries must be closed under composition have inverses that are also symmetries, they naturally admit a group structure Formally, we focus on symmetries arising from permuting rom variables, consider any apple S n, a collection of permutations forming a subgroup of the symmetric group on N, defining the associated group action on H n by h)a) h A)) for any 2 h 2H n Let fix be the set of all fixed points the fixed set) of this action, ie, fix {h 2H n : ha) hb) if 9 2 st A) B} ) Inspired by groups of rom variable symmetries that arise in rate regions for network coding problems, [4] defined the -symmetric entropy region, n \ fix its outer bound, the -symmetric polymatroidal region n \ fix, respectively This definition built upon [5], which considered the special case of n,,n t )-partition groups p S N S N2 S Nt, wherein p {N,N 2,,N t } is a t-partition of N with n i N i,i2{,,t}, proved the following important theorem Thm : For N 4, p p if only if p or {{i},n \{i}} for some i 2 N The theorem states that the closure of p -symmetric entropy region can be fully characterized by Shannon-type inequalities if only if p is the -partition or a 2-partition with one of its blocks being singleton Building on this theorem, in this paper we study when for arbitrary rom variable permutation groups Naturally, the key quantity in determining the regions is the orbits in the power set under the group action, denoted as 2 N //, several results are given regarding the structure of these orbits in II, as well the structure of these orbit structures themselves under the action of different groups in III It is shown that Shannon sufficiency is implied under refinement of the associated partition of the power set, hence to supergroups of a group, while insufficiency of

2 Shanon-type inequalities is implied to subgroups This simple implication, in turn, enables one to leverage the main result of [5] to show that unless the group acts transitively on N or fixes an element of this set while acting transitively on the remaining n elements, Shannon-type inequalities will be insufficient Bearing this in mind, we first undertake an exhaustive solution to the Shannon-type sufficiency question for four five variables This is easily carried out with computational tools from AP ITAP, the results are detailed in IV-A summarized at the top of Fig 4 Shifting attention to n 6 the remaining open cases, which are exclusively associated with transitive subgroups of S n or S n, it becomes clear that the priorities for investigation are containment minimal or maximal transitive groups, as the implication of super/subgroup sufficiency/insufficiency of Shannon-type inequalities can be leveraged to solve more tightness questions One collection of containment minimal transitive groups are the cyclic groups C n, Shannon type inequalities were shown in IV-A to be sufficient for n 4, 5 for these groups, thus their supergroups, the dihedral groups D 4 D 5 However, we prove in IV-B that Shannon type inequalities are insufficient to characterize C n, as well as S C n S D n for n 6 Together with the results from [5], their implications to subgroups as described in III, collectively this forms a wealth of examples of both tightness non-tightness of the Shannon-type inequalities on group symmetric entropy regions summarized in Fig 4 II ORBIT STRUCTURE For a given apple S n, for any 2 A N, A) { i) :i 2 A} 2) defines an action of on 2 N For A N, let O A) { A) : 2 } be the orbit of the action that contains A If there is no ambiguity, we simply call O A) an orbit of Let O {O A) :A N} 2 N // be the set of all orbits of we call it the orbit structure of Then one can easily check that ) can be rewritten as fix {h 2H n : ha) hb) if A, B 2O, O2O } 3) It can be seen from 3) that O uniquely determines fix, ie, fix R O So it also determines in some sense, whether they are equal Therefore, in this section, we study the structure of O There exists a partial order in O defined below Definition : For O, O 2 2 O, O appleo 2 if for any A 2O, there exists B 2O 2 such that A B The definition immediately implies the following Prop : For A B N, O A) appleo B) Note that O may not be a lattice, see Fig 2b), the orbit structure of cyclic group C 5 Let {N,N 2 } be a partition of N, 2 be permutation groups on N N 2, respectively We define O O 2 {O 2 N :{A \ N : A 2O}2O, {A \ N 2 : A 2O}2O 2 } Note that O O 2 is just the Cartesian product of O O 2 For A N, it can be checked that O 2 A) O A \ N ) O 2 A \ N 2 ) For O O ) O ) 2 O 2 O 2) O ) 2 in O O 2, O appleo 2 if only if both O ) appleo 2) O ) 2 appleo 2) 2 Prop 2: O 2 O O 2 Proof Let B 2O 2 A) Then there exists 2 2 such that B A) Let 2, where Then B \N A\N 2 ) which implies B \N 2 O A \ N ) Similarly B \ N 2 2O 2 A \ N 2 ) Therefore, O 2 A) O A \ N ) O 2 A \ N 2 ) 2 O O 2 The other inclusion direction can be verified similarly Cor Q : Let p {N,,N t } be a partition of N Let t i i be a permutation group on N i be permutation groups on N i, respectively Then O Q t i O i According to Corollary, if a permutation group on N can be written as the direct product of i on N i, where N i is a block of a partition of N the orbit structure O is the direct product of O i We call such groups decomposable if t 2 call other groups undecomposable An orbit structure of a decomposable group is called a decomposable orbit structure An orbit structure which is not equal to the orbit structure of any decomposable group is called an undecomposable orbit structure Note that the orbit structure of an undecomposable group can be decomposable See Fig 3, the orbit structure of undecomposable group S5 3, h2)45), 345)i is decomposable since it is equal to the orbit structure of decomposable group S 2 S 3 To study the orbit structure of an arbitrary permutation group, we only need to focus on those undecomposable obit structures treat them as the building blocks For symmetric group S n, the orbit structure is a chain with length n +, ie, O Sn {O i {A N : A i} : i 0,,n} O i appleo j if only if i apple j When n, 2, 3, all undecomposable orbit structure is orbit structure of symmetric group S n, while O S3 is also equal to O A3 For n 4, 5, the hasse diagrams of all undecomposable orbit structures up to conjugacy) are listed in Figs 2, respectively To save space, in the label of the figures, we denote a set by juxtaposing its elements For example, {, 2} is denoted by 2 in Fig c), for group V, O2) O V {, 2}) {{, 2}, {3, 4}} In Fig 3, we give an examples of a decomposable orbit structure S 2 S 3 Remark When apple S n is the trivial group, each A N forms an orbit of O become isomorphic to the boolean lattice fix H n Then n When S n, the symmetric group itself, as we discussed above, O becomes a chain When is a partition group, O is the direct product of chains, see Fig 3 for the case of partition group S 2 S 3

3 O23) O23) O2) O3) O2) O) O) a) O S4 O A4 b) O D4 O C4 O23) O34) O23) O4) O3) O4) O3) O2) O34) O2) O) O3) O) c) O V d) O S 2 4 Fig Undecomposable orbit structures on N {, 2, 3, 4} O234) O234) O23) O24) O23) O2) O3) O2) O) O) a) O S5 O A5 O A,5) b) O D5 O C5 Fig 2 Undecomposable orbit structures on N {, 2, 3, 4, 5} O234) O345) O23) O34) O2) O3) O34) O) O3) Fig 3 Orbit structures of S 2 S 3 S5 3 III RELATIONS BETWEEN ORBIT STRUCTURES In the previous section, we discussed the orbit structure of a permutation group In this section, we discuss the structure among different orbit structures, a partial order defined below Definition 2 Structure of the orbit structures): For, 2 apple S n, O apple O 2 if O is a refinement of O 2 For example, by Figs b) c), we see that O V apple O C4 Prop 3: For apple 2 apple S n, O apple O 2 Proof Since O O 2 are both partitions of 2 N, to prove O apple O 2, it suffices to prove that each orbit of 2 is the union of the orbits of Let l 2 /, 2,, l be the right cosets of on 2, where i 2 2 is the identity of 2 also ) Then for any A N, O 2 A) { A) : 2 2 } l[ O i A)) i which proves the proposition l[ { i A)) : 2 } i Lem : For any apple S n, n \ fix Lemma can be proved by a method similar to that of Theorem 4 in [5] The proof is omitted Thm 2: For, 2 apple S n with O apple O 2, ) if 2 2 ; 2) if 2 2 Proof For O apple O 2, by 3), we have fix 2 fix Now if, by Lemma, we have 2 n \ fix 2 n \ fix \ fix 2 ) n \ fix \ fix 2 \ fix 2 \ fix 2 n \ fix \ fix 2 n \ fix 2 2 which proves the first part of the theorem If 2 2 there exists h 2 2 such that h 62 2 n \ fix 2 Since h 2 2 n \ fix 2, h 2 fix 2 which implies that h 62 n so h 62 n \ fix One the other h, since 2, h 2 Therefore which proves the second part of the theorem Proposition 3 Theorem 2 immediately imply the following Corollary Cor 2: For < 2 apple S n, ) if 2 2 ; 2) if 2 2 The purpose of stating Thm 2 in terms of orbit structures rather than the subgroups that generate them as presented in corollary 2, more generally the structure of the orbit structures rather than simply the lattice of subgroups, is that multiple groups can generate the same orbit structure in the

4 power set, as we shall see multiple examples in the next section as depicted in Fig 4 Indeed, as can be seen by Fig4, which we shall derive in detail in the next section, while apple 2 ) O apple O 2, the converse is not true in general Remark According to the the contrapositive of Thm, if p N//, {{ i) : 2 } : i 2 N} has three or more blocks, or has two blocks each with more than one element, then It leaves open only the following remaining cases to determine whether : ) is transitive, ie N// 2) fixes one element of N acts transitively on the remaining n elements, so that N// {N \ i, {i}} for some i 2 N Furthermore, we only need to focus on one of these two cases, depending on whether or not we are trying to show sufficiency or insufficiency, as shown by the following theorem Thm 3: S S ), or equivalently, ) S S Proof Observe that the orbits O S can be found by taking each orbit in O, repeating it twice, once without the element fixed by S, once with it Letting N 0 N [{n+}, we see that the projection proj N S meaning keeping only the elements in the entropy function associated with subsets of the ground set N), since the containment holds from the extra variable, inequality, equalities in the polyhedral cone on the left h side, the direction holds from the fact that any polymatroid h 2 can trivially be extended to a polymatroid h 0 2 S by defining h 0 A) ha \ N) for all A N 0 A parallel argument shows that proj N S, since an almost entropic vector can always be extended in this trivial way by making the new rom variable deterministic Thus mapping through the projection operation, S S implies The other statement is simply the negation of this statement Finally, to best leverage subgroup supergroup implications set out by Corollary 2, the study should focus on maximal or minimal subgroup, respect equivalences the ordering between orbit structures from Thm 2 IV RESULTS: TIHTNESS OF POLYMATROIDAL REION FOR SOME SYMMETRICAL ENTROPY FUNCTIONS Bearing the results of the previous section in mind, in this section we exhaustively determine all distinct orbit structures for n 4, 5 rom variables, determine for which ones the Shannon outer bound is tight Then, we outline several new cases beyond those covered by Thm for which the Shannon outer bound is not tight A Symmetrical entropy functions of 4 or 5 rom variables All subgroups of S 4 S 5 are easily generated within AP [7] When doing so, for each conjugacy class of subgroups, we need to consider only one representative, since the orbit structures of different groups in the same conjugacy class can be obtained from each other by permuting the elements of the ground set Furthermore, we can consider two representatives of different conjugacy classes with isomorphic orbit structures equivalent for our purpose of determining whether Up to conjugacy, there are classes of permutation groups for n 4 9 classes of permutation groups for n 5 On the other h, up to orbit structure isomorphism, there are only classes of permutation groups respectively for n 4 5, representatives from the conjugacy classes of groups associated with these equivalence classes are listed in the nodes in Fig 4 under n 4 n 5, respectively As described in the previous section, the structure of orbit structures can be partially ordered by partition refinement, this partial ordering is also depicted via the edges in Fig 4 Thm 4: Fig 4 indicates correctly for each node whether Proof The orbits in the power set can be found with AP [7], while the extreme rays of the associated can be calculated by entering the equality constrained regions into lrs [9] The data located at [0] contains each of these regions their extreme rays Paying attention to the structure among the orbits, we can organize the results as follows n 4:The only remaining cases not already hled by Thms 2is that of the Normal Klein 4-group V : h2)34), 3)24)i the cyclic/dihedral groups C 4,D 4 We prove V V as this implies, by Theorem 2, C4 C 4 ITAP [] was used to construct linear polymatroids for all of the rays of V, showing that each of them are almostentropic thus V V as detailed in [0], [] n 5 : It suffices to prove C5 C 5 S C 4 S C 4, as Thms 2together with this information hle the other cases via implication C5 C 5 can be proven by using, eg ITAP to construct a linear polymatroid for each of the rays of C 5 as detailed in [0] Next, among the extreme rays of S C 4, one finds the polymatroid 0 if A ;, < 2 if A, ha) 3 if A 2 A {2, 4} or {3, 5}, : 4 if A {2, 4}, {3, 5} or A 3 By restricting the polymatroid on {, 2, 3, 4}, we have 0 if A ;, < 2 if A, eha) 3 if A 2 A {2, 4}, : 4 if A {2, 4} or A 3, which violates Zhang-Yeung inequality, is therefore nonentropic Thus S C 4 S C 4 B Cyclic dihedral Symmetric Entropy Functions, n 6 From the remark at the end of III, we know that for apple S n, to determine whether, we should consider transitive groups that are either minimal or maximal In this 4)

5 Fig a) Symmetric & Alternating: S4 : h234), 2)i A4 : h23), 2)34)i n4: Fig b) Dihedral & Cyclic: D4 : h234), 3)i 2, 2)-Partition: C4 : h234)i Fig c) S2 S2 : h2), 34)i Normal Klein 4-group: V : h2)34), 3)24)i Double Transp: S42 : h2)34)i Fig d) n 6: Sn, Sn {,, n}// not a, n S, S 6,, 3)-partition: S S S3 : h345), 34)i S S A3 : h345)i,, 2)-Partition: S S S2 : h34)i S Sn S, 4)-Partition: S S4 : h2345), 23)i S A4 : h2345), 23)45)i Cyclic & Dihedral: C5 : h2345)i D5 : h2345), 25)34)i Fig 2b) Fix, Rest Cyclic or Dihedral: S C4 : h2345)i S D4 : h2345), 24)i, 2, 2)-partition: S S2 S2 : h23), 45)i,,, 2)-partition: S S S S2 : h45)i S Sn ) or n) partition ) S, Fig 2a) Symmetric, Alternating, & en Affine: S5 : h2345), 2)i, A5 : h2345), 23)i A, 5) : h2345), 2345)i Fig 3 2, 3)-Partition: S2 S3 : h2), 345), 34)i S2 A3 : h2), 345)i S53 : h2)45), 345)i, 3)-Partition: S S3 : h234), 23)i S A3 : h234), 243)i Trivial: h)i n5: Fix, Rest Klein 4-group: S V : h23)45), 24)35)i S S42 : h23)45)i trivial: h)i Fig 4 Summary of results in this paper about the tightness of Shannon outer bounds to group symmetrical entropy regions Equivalent subgroups of Sn yielding the same orbit structures in the power set are grouped together, the partial order based on refinement of the associated partition in the power set is displayed For each subgroup, a sample representation is provided in terms of its generators When the orbit structure has been depicted elsewhere in the paper, the appropriate figure is also indicated vein, we consider the cyclic group dihedral group, which we have just shown to have Shannon-type inequalities tight for n 4, 5 in the result below Thm 5: For n S, S 6,, S, S Proof For Sn, let O2, {O A) 2 O : A 2 O, A 2} Note that O2, {{ {, + i}) : 2 }, i,, bn/2c} We pick { {, 4}) : 2 } 2 O2, Let h 2 Hn be defined by 0 if A ;, <2 if A, ha) 3 if A 2 A 62 { {, 4}) : 2 }, : 4 if A 2 { {, 4}) : 2 } or A 3 This h 2 n h 2 fix which implies h 2 By restricting h on {, 2, 3, 4}, we have the ray 4) which violates Zhang-Yeung inequality, hence is non-entropic, impling that h is also non-entropic For dihedral groups, n 6, note that O2,D O2,C thus These in turn imply S S, S S for n 7 by Thm 3 All that remains is S C5 S C5, S D5 S D5 Among the extreme rays of for this case, one finds 0 if A ;, <2 if A, ha) 3 if A 2 A 2 / O {2, 4}), : 4 otherwise By restricting the polymatroid on {, 4, 5, 6}, we obtain a ray isomorphic to the ray 4) which violates Zhang-Yeung inequality, therefore non-entropic, implying h is also nonentropic hence for S C5 S D 5 V C ONCLUSION This paper studied whether or not Shannon-type inequalities fully characterize the part of the entropy region that is symmetric under the action of various permutation groups on the rom variables Several results were outlined regarding implications between different groups of the sufficiency/insufficiency of Shannon-type inequalities Aside from these implications, the main results of the paper are summarized in Fig 4 R EFERENCES [] R W Yeung, Information Theory Network Coding, Springer, 200 [2] S Fujishige, Polymatroidal dependence structure of a set of rom variables, Info Contr, 39: pp 55-72,97 [3] Z Zhang R W Yeung, On characterization of entropy function via information inequalities, IEEE Trans Info Theory, vol 44, pp , Nov 99 [4] J Apte J Walsh, Symmetry in network coding, in 205 IEEE International Symposium on Information Theory ISIT), June 205 [5] Q Chen R W Yeung, Partition-symmetrical entropy functions, accepted by IEEE Trans Inform Theory [6] F Mat usˇ, Infinitely many information inequalities, IEEE Int Symp Info Theory, Nice, France, June 2007 [7] The AP roup, AP roups, Algorithms, Programming, [] J Apte J M Walsh, Information Theoretic Achievability Prover-A AP4 Package, [9] D Avis, A revised implementation of the reverse search vertex enumeration algorithm, Polytopes Combinatorics Computation, pp77 9, Springer, 2000 [0] J Apte, Computer Proofs for Symmetrical entropy functions, [] J Apte J M, Walsh, Constrained linear representability of polymatroids algorithms for computing achievability proofs in network coding, submitted to IEEE Trans Info Theory,