Symmetries in Entropy Space

Transcription

1 1 Symmetries in Entropy Space Jayant Apte, Qi Chen, John MacLaren Walsh Department of Electrical and Computer Engineering Drexel University Philadelphia, PA Thanks to NSF CCF

2 Co-Authors & Software Advertisement Jayant Apte (Ph.D) Defended 10-Aug-2016 Dissertation presents symmetry exploiting algorithms for proving network coding capacity regions. Try our software: ITAP & ITCP Qi Chen, Ph.D. Ph.D. CUHK w/ R. W. Yeung Drexel Fall, 2015 INC, CUHK, 2016 On the Job Market! NEW IT&NC SOFTWARE TOOL RELEASES!!!! (ASPITR homepage & ithub) Information Theoretic Achievability Prover (ITAP): For a given network, determine entire rate region associated w/ a class of linear codes! Also, can determine achievability of a specified rate point, bounds for linear secret sharing, and test where a specified polymatroid is linear over a specified field. Information Theoretic Converse Prover (ITCP): For a given network coding problem, computes polyhedral cones outer bounding its capacity region using a custom new symmetry exploiting polyhedral projection algorithm! Both Shannon & non-shannon outer bounds are supported. Also works with secret sharing.

3 Outline 1. Entropy Region N & Shannon Outer Bound N 2. Portion Symmetric Under roup, and 3. Power-Set Orbit Structures & Hierarchical Results 4.? for N 4, 5 (exhaustive) and N 6.

4 Entropy Region N & Shannon Outer Bound 1. X (X 1,...,X N ) N discrete RVs 2. every subset X A (X i,i 2 A) A {1,...,N} [N] has joint entropy h(x A ). 3. h (h(x A ) A [N]) 2 R 2N 1 entropic vector Example: for N 3, h (h 1,h 2,h 3,h 12,h 13,h 23,h 123 ). 4. a h o 2 R 2N 1 is entropic if 9 joint PMF p X s.t. h(p X )h o. 5. Region of entropic vectors N 6. Closure N is a convex cone h 1 h 12 N is an unknown non-polyhedral convex cone for N 4. N h 2 Shannon bound N : h 2 R 2N 1 s.t. h A + h B h A[B + h A\B (1) h A[B h A (2) i.e. submodular & non.-dec. for N 2{2, 3}, n N, but for N 4, N ( N. determining capacity regions of all networks under network coding, complete characterization of N

5 Entropy Region N WhyCare? All fundamental laws of information theory Network Coding Capacity Region (Streaming, Distributed Storage, & Caching) Implications among conditional independences (raphical Models & Machine Learning) Fundamental Limits for Secret Sharing Schemes More (inequalities for subgroup sizes, matrix rank inequalities, Kolmogorov Complexities, etc.)

6 roup Action & Symmetries h 12 (12) (0,1,1) (1,0,1) h 2 h 1 Any permutation 2 S n, : {1,...,n}!{1,...,n} a bijection, is a symmetry of n. Under, thervs(x 1,...,X n ) 7! (X (1),...,X (n) ), so h A 7! h (A),with (A) :{ (i) i 2A}. Example: (12), ([h 1,h 2,h 12,h 3,h 13,h 23,h 123 ] T )[h 2,h 1,h 12,h 3,h 23,h 13,h 123 ] T. Clearly, if h 2 n or n, (h). (definition is insensitive to ordering of RVs)

7 Key Question:? h 12 Fix S2 S 2 S2 h 2 h 1 What about those h 2 n fixed under a group of problem symmetries S n? Define Fix ( n ) and Fix ( n). Fix (H) :{h 2H (h) h 8 2 } (3) For what types of problem symmetries can we expect Shannon-type inequalities to be su cient? i.e., for which does?

8 Orbits in the Power Set N {1,...,n} Fix,andhence and on depend on through 2 N // theorbits in the power set. (h) h 8 2 () h A h (A) 8 2, 8A N. O (A) :{ (A) 2 }. h A h B 8B 2 O (A) O O (A) A 2 2 N,thepower set orbits. (Partitions 2 N ) Note: Multiple groups can yield the same power set orbits. i.e. can have O O 0 for 6 0.

9 Power Set Orbits, N 4 {N} {N} {N} {N} O(134) O(14) O(14) O(34) O(3) (a) O S4 O A4 (b) O D4 O C4 (c) O V h(12)(34),(13)(24)i (d) O S 2 4 h(12)(34)i Figure 1: Indecomposable power set orbits on N {1, 2, 3, 4} Power Set Orbits under di erent groups are also ordered by refinement of associated partition of the power set.

10 Power Set Orbits, N 5 {N} {N} O(1234) O(1234) O(124) (a) O S5 O A5 O A(1,5) (b) O D5 O C5 Figure 2: Indecomposable orbit structures on N {1, 2, 3, 4, 5} Observe that while a poset, need not be a lattice.

11 ? Some Implications If apple 0,thenO apple O 0, i.e. partition O refines O 0 (extra group elements can force more equivalences) Thm. 2: IfO apple O 0,and, then 0 0. Equiv., if O apple O 0,and 0 ( 0,then (.

12 Chen & Yeung Partition Symmetrical Entropy Functions:? for S n1 S n2 S nk Thm. 1 (Qi Chen & R. W. Yeung): Let p {N 1, N 2,...,N t } be a t-partition of N, and p S N1 S N2 S Nt.For N 4, p p if and only if p {N} or {{i},n \{i}} for some i 2 N. What about general??remainingcases? For any, letp N // {{ (i) 2 } i 2N},then apple p Thm. 1: ( unless N // or N // {i, N\{i}}, i 2N. N // : is transitive. N // {i, N \}: S 1 0, fixes some i, then 0 on remainder is transitive on N\{i}. Priority (Pessimistic) Maximal Transitive Subgroups Priority (Optimistic) Minimal Transitive Subgroups

13 ? Another Implication Thm. 3: Let S 1 act on N[{n +1} (fix n +1, acts on N ). Then S 1 S 1 ). Proof: orbits of O S1 eachorbitino repeated twice: once w/o n +1, and once w/ n +1. proj N S 1 : First2 N 1 coordinates are same subsets obeying same inequalities, must also have the remaining coordinates exist obeying more inequalities. : Takeh 2, extendtoh 0 on N[{n +1} via h 0 A h A\N,then h 0 2 S 1. proj N S1 : TakeX 1,...,X n from X 1,...,X n+1 realizing h 0 2 S1.Realizesa h 2. : Extend as previous, X n+1 0,deterministic.

14 ? Complete Power Set Orbits, N 4 Set Orbits, N 4 Answer for N 4 O(14) power set orbits on N O(14) {1, 2, 3, 4} 14) O(34) s are also ordered by refinement of associated O(3) OV h(12)(34),(13)(24)i (d) OS 2 h(12)(34)i 4 O(134) O(14) Symmetric & Alternating: O(34) S4 : (1234), (12) A : (123), O(3) (12)(34) 4 O(134) 2) O(14) Dihedral & Cyclic: O(14) 6 (a) OS4 OA4 (b) OD4 OC4 (c) O (d) O O(34) 2 D4 : (1234), (13) V h(12)(34),(13)(24)i S4 h(12)(34)i ts, N 4 (2, 2)-Partition: O(3) C4 : (1234) S2 2, 3,S4} 2 : (12), (34) Figure 1: Indecomposable power set orbits on N {1, Normal Klein 4-group: Power Set Orbits under di erent groups are also ordered by refinement of associated (c) OV h(12)(34),(13)(24)i (d) OS 2 h(12)(34)iv : (12)(34), (13)(24) 4 partition of the power set. O(134) (a) OS4 OA4 6 Trivial: () (b) O (1, 3)-Partition:Figure 1: Inde S1 S3 : (234), (23) S1 Power A3 : Set (234), (243) Orbits under di partition of the power set. Double Transp.: S42 : (12)(34) 6 (1, 1, 2)-Partition: S1 S1 S2 : (34) 6

15 ? Complete Answer for N 5 Symmetric, Alternating, & en. Affine: S5 : (12345), (12), A5 : (12345), (123) A(1, 5) : (12345), (2345) 6 (2, 3)-Partition: S2 S3 : (12), (345), (34) S2 A3 : (12), (345) S53 : (12)(45), (345) 6 (1, 1, 3)-partition: S1 S1 S3 : (345), (34) S1 S1 A3 : (345) 6 6 Fix 1, Rest Cyclic or Dihedral: S1 C4 : (2345) S1 D4 : (2345), (24) 6 S1 S42 O(1234) O(124) 6 O O S 5 4-group: A5 (a) Fix 1, Rest Klein S1 V : (23)(45), (24)(35) 6 (1, 2, 2)-partition: S1 S2 S2 : (23), (45) (1, 1, 1, 2)-partition: S1 S1 S1 S2 : (45) Power Set Orbits, N 5 O(1234) (1, 4)-Partition: Cyclic & Dihedral: S1 S4 : (2345), (23) C5 : (12345) S1 A4 : (2345), (23)(45) D5 : (12345), (25)(34) : (23)(45) OA(1,5) Figure 2: Indecomposable orbit structures on N {1, 2, 3, 4, 5} Observe that while a poset, need not be a lattice. 6 trivial: () (b) OD5 OC5

16 ? Some Answers for N 6 Thm. 5: For n 6, Cn ( C n, Dn ( D n, S1 C n 1 ( S 1 C n 1, & S1 D n 1 ( S 1 D n 1. Proof shows that the ray with coordinates matching Vámos projection lies in C n is cut o by Zhang-Yeung non-shannon inequality. but

17 Summary Fig. 1(a) Symmetric & Alternating: S4 : (1234), (12) A4 : (123), (12)(34) n4: Fig. 1(b) Dihedral & Cyclic: 6 D4 : (1234), (13) (2, 2)-Partition: C4 : (1234) Fig. 1(c) S2 S2 : (12), (34) Normal Klein 4-group: V : (12)(34), (13)(24) (1, 3)-Partition: S1 S3 : (234), (23) S1 A3 : (234), (243) Double Transp.: S42 : (12)(34) Fig. 1(d) 6 n 6: Sn, Sn {1,..., n}// not a (1, n 6 S1 Cn, Cn 1 6 Dn 6 S1 Cn S1 Sn S1 Sn S1 Dn 6 (1, 4)-Partition: S1 S4 : (2345), (23) S1 A4 : (2345), (23)(45) 6 6 Fix 1, Rest Klein 4-group: S1 V : (23)(45), (24)(35) S1 S42 : (23)(45) trivial: () 6 Cyclic & Dihedral: C5 : (12345) D5 : (12345), (25)(34) Fig. 2(b) 6 Fix 1, Rest Cyclic or Dihedral: S1 C4 : (2345) S1 D4 : (2345), (24) 1 6 (1, 2, 2)-partition: S1 S2 S2 : (23), (45) (1, 1, 1, 2)-partition: S1 S1 S1 S2 : (45) Dn S1 Dn 6 Symmetric, Alternating, & en. Affine: S5 : (12345), (12), A5 : (12345), (123) A(1, 5) : (12345), (2345) 6 (1, 1, 3)-partition: S1 S1 S3 : (345), (34) S1 S1 A3 : (345) (1, 1, 2)-Partition: S1 S1 S2 : (34) 1) or (n) partition ), 1 Trivial: () Cn Fig. 2(a) 6 Fig. 3 (2, 3)-Partition: S2 S3 : (12), (345), (34) S2 A3 : (12), (345) S53 : (12)(45), (345) 6 n5:

18 The Way Forward 1. For n 6, answer? for other maximal transitive groups. 2. Permutations of the ground set form just one set of interesting symmetries for N, others include Combinatorial Symmetry roup: permutations of the extreme rays which leave the face lattice intact. (Huge Subgroup of S M,w/M # of extreme rays) A ne Symmetry roup: Those Combinatorial symmetries whose ray permutation can be generated by multiplying by a 2 N 1 2 N 1 invertible matrix. (Also a large group) Restricted A ne Symmetry roup: Those a ne symmetries associated with vectors representing the rays of fixed length. (can be computed w/ sympol) 3. Which of the latter are also symmetries of N? 4. What sort of dimensionality reduction can be achieved by exploiting the a ne symmetries when calculating rate regions for network coding, storage repair tradeo s, caching regions, etc?