Symmetries in Entropy Space

Similar documents
Symmetries in the Entropy Space

Partition Symmetrical Entropy Functions*

Non-isomorphic Distribution Supports for Calculating Entropic Vectors

Symmetry in Network Coding

Exploiting Symmetry in Computing Polyhedral Bounds on Network Coding Rate Regions

Entropic Vectors: Polyhedral Computation & Information Geometry

Multiterminal Networks: Rate Regions, Codes, Computations, & Forbidden Minors

A computational approach for determining rate regions and codes using entropic vector bounds

Network Combination Operations Preserving the Sufficiency of Linear Network Codes

Bounding the Entropic Region via Information Geometry

Algorithms for Computing Network Coding Rate Regions via Single Element Extensions of Matroids

Characterizing the Region of Entropic Vectors via Information Geometry

. Relationships Among Bounds for the Region of Entropic Vectors in Four Variables

Extremal Entropy: Information Geometry, Numerical Entropy Mapping, and Machine Learning Application of Associated Conditional Independences.

Symmetry in Network Coding

Matroid Bounds on the Region of Entropic Vectors

A new computational approach for determining rate regions and optimal codes for coded networks

Computing and Communications 2. Information Theory -Entropy

Algebraic matroids are almost entropic

Entropy Vectors and Network Information Theory

Extremal Entropy: Information Geometry, Numerical Entropy Mapping, and Machine Learning Application of Associated Conditional Independences

LP Duality: outline. Duality theory for Linear Programming. alternatives. optimization I Idea: polyhedra

On Multi-source Multi-Sink Hyperedge Networks: Enumeration, Rate Region. Computation, and Hierarchy. A Thesis. Submitted to the Faculty

Received: 1 September 2018; Accepted: 10 October 2018; Published: 12 October 2018

Linearly Representable Entropy Vectors and their Relation to Network Coding Solutions

Lecture 1: The Multiple Access Channel. Copyright G. Caire 12

Belief Propagation, Information Projections, and Dykstra s Algorithm

0 Sets and Induction. Sets

On Secret Sharing Schemes, Matroids and Polymatroids

Network Coding on Directed Acyclic Graphs

On Successive Lumping of Large Scale Systems

5 Mutual Information and Channel Capacity

Capacity Region of the Permutation Channel

Inverse and Implicit Mapping Theorems (Sections III.3-III.4)

Fundamental rate delay tradeoffs in multipath routed and network coded networks

1 Ensemble of three-level systems

Combinatorics of the Cell Decomposition of Affine Springer Fibers

Submodularity in Machine Learning

ACO Comprehensive Exam October 18 and 19, Analysis of Algorithms

Secret Sharing. Qi Chen. December 14, 2015

VC-DENSITY FOR TREES

Endotrivial modules. Nadia Mazza. June Lancaster University

Class Equation & Conjugacy in Groups

Orbitopes. Marc Pfetsch. joint work with Volker Kaibel. Zuse Institute Berlin

hal , version 1-12 Jun 2014

Computational Challenges in Perfect form theory

A multiplicative deformation of the Möbius function for the poset of partitions of a multiset

This document is downloaded from DR-NTU, Nanyang Technological University Library, Singapore.

U = γkt N. is indeed an exact one form. Thus there exists a function U(S, V, N) so that. du = T ds pdv. U S = T, U V = p

Symmetry, Outer Bounds, and Code Constructions: A Computer-Aided Investigation on the Fundamental Limits of Caching

Fundamental Domains for Integer Programs with Symmetries

18. Counting Patterns

Rate region for a class of delay mitigating codes and P2P networks

Orbit coherence in permutation groups

Rational Catalan Combinatorics: Intro

The Inflation Technique for Causal Inference with Latent Variables

Counting chains in noncrossing partition lattices

Copositive matrices and periodic dynamical systems

Some notes on Coxeter groups

Hodge theory for combinatorial geometries

MATH 25 CLASS 21 NOTES, NOV Contents. 2. Subgroups 2 3. Isomorphisms 4

Chapter 2: Entropy and Mutual Information. University of Illinois at Chicago ECE 534, Natasha Devroye

Space-Time Coding for Multi-Antenna Systems

Definitions. Notations. Injective, Surjective and Bijective. Divides. Cartesian Product. Relations. Equivalence Relations

Dimension reduction for semidefinite programming

Amobile satellite communication system, like Motorola s

Quick Sort Notes , Spring 2010

On Common Information and the Encoding of Sources that are Not Successively Refinable

COS597D: Information Theory in Computer Science September 21, Lecture 2

Asymptotics of generating the symmetric and alternating groups

EE595A Submodular functions, their optimization and applications Spring 2011

Theorems and Definitions in Group Theory

120A LECTURE OUTLINES

Lecture 4: Orbits. Rajat Mittal. IIT Kanpur

Lecture 6: Gaussian Channels. Copyright G. Caire (Sample Lectures) 157

SF2729 GROUPS AND RINGS LECTURE NOTES

On Dependence Balance Bounds for Two Way Channels

Conditional Information Inequalities for Entropic and Almost Entropic Points

Before you begin read these instructions carefully.

Improving the Linear Programming Technique in the Search for Lower Bounds in Secret Sharing

Mathematics 222a Quiz 2 CODE 111 November 21, 2002

GIT-Equivalence and Semi-Stable Subcategories of Quiver Representations

The extreme rays of the 5 5 copositive cone

Least squares regularized or constrained by L0: relationship between their global minimizers. Mila Nikolova

Entanglement Inequalities

Entropies & Information Theory

RING ELEMENTS AS SUMS OF UNITS

Introduction to Information Theory

A Classi cation of Dodecaphonic Groups

Hidden Symmetry Subgroup Problems

Partial, Total, and Lattice Orders in Group Theory

Maximum Correlation Analysis of Nonlinear S-boxes in Stream Ciphers

A Hilton-Milner-type theorem and an intersection conjecture for signed sets

Pattern avoidance in compositions and multiset permutations

Cutting planes from extended LP formulations

Math 4320, Spring 2011

Topological properties

Reverse Edge Cut-Set Bounds for Secure Network Coding

arxiv: v2 [math.co] 11 Oct 2016

Extending Brickell-Davenport Theorem to Non-Perfect Secret Sharing Schemes

Transcription:

1 Symmetries in Entropy Space Jayant Apte, Qi Chen, John MacLaren Walsh Department of Electrical and Computer Engineering Drexel University Philadelphia, PA jwalsh@ece.drexel.edu Thanks to NSF CCF-1421828

Co-Authors & Software Advertisement Jayant Apte (Ph.D) Defended 10-Aug-2016 Dissertation presents symmetry exploiting algorithms for proving network coding capacity regions. Try our software: ITAP & ITCP Qi Chen, Ph.D. Ph.D. CUHK w/ R. W. Yeung Postdoc @ Drexel Fall, 2015 Postdoc @ INC, CUHK, 2016 On the Job Market! NEW IT&NC SOFTWARE TOOL RELEASES!!!! (ASPITR homepage & ithub) Information Theoretic Achievability Prover (ITAP): For a given network, determine entire rate region associated w/ a class of linear codes! Also, can determine achievability of a specified rate point, bounds for linear secret sharing, and test where a specified polymatroid is linear over a specified field. Information Theoretic Converse Prover (ITCP): For a given network coding problem, computes polyhedral cones outer bounding its capacity region using a custom new symmetry exploiting polyhedral projection algorithm! Both Shannon & non-shannon outer bounds are supported. Also works with secret sharing. http://www.ece.drexel.edu/walsh/aspitrg/software.html

Outline 1. Entropy Region N & Shannon Outer Bound N 2. Portion Symmetric Under roup, and 3. Power-Set Orbit Structures & Hierarchical Results 4.? for N 4, 5 (exhaustive) and N 6.

Entropy Region N & Shannon Outer Bound 1. X (X 1,...,X N ) N discrete RVs 2. every subset X A (X i,i 2 A) A {1,...,N} [N] has joint entropy h(x A ). 3. h (h(x A ) A [N]) 2 R 2N 1 entropic vector Example: for N 3, h (h 1,h 2,h 3,h 12,h 13,h 23,h 123 ). 4. a h o 2 R 2N 1 is entropic if 9 joint PMF p X s.t. h(p X )h o. 5. Region of entropic vectors N 6. Closure N is a convex cone h 1 h 12 N is an unknown non-polyhedral convex cone for N 4. N h 2 Shannon bound N : h 2 R 2N 1 s.t. h A + h B h A[B + h A\B (1) h A[B h A (2) i.e. submodular & non.-dec. for N 2{2, 3}, n N, but for N 4, N ( N. determining capacity regions of all networks under network coding, complete characterization of N

Entropy Region N WhyCare? All fundamental laws of information theory Network Coding Capacity Region (Streaming, Distributed Storage, & Caching) Implications among conditional independences (raphical Models & Machine Learning) Fundamental Limits for Secret Sharing Schemes More (inequalities for subgroup sizes, matrix rank inequalities, Kolmogorov Complexities, etc.)

roup Action & Symmetries h 12 (12) (0,1,1) (1,0,1) h 2 h 1 Any permutation 2 S n, : {1,...,n}!{1,...,n} a bijection, is a symmetry of n. Under, thervs(x 1,...,X n ) 7! (X (1),...,X (n) ), so h A 7! h (A),with (A) :{ (i) i 2A}. Example: (12), ([h 1,h 2,h 12,h 3,h 13,h 23,h 123 ] T )[h 2,h 1,h 12,h 3,h 23,h 13,h 123 ] T. Clearly, if h 2 n or n, (h). (definition is insensitive to ordering of RVs)

Key Question:? h 12 Fix S2 S 2 S2 h 2 h 1 What about those h 2 n fixed under a group of problem symmetries S n? Define Fix ( n ) and Fix ( n). Fix (H) :{h 2H (h) h 8 2 } (3) For what types of problem symmetries can we expect Shannon-type inequalities to be su cient? i.e., for which does?

Orbits in the Power Set N {1,...,n} Fix,andhence and on depend on through 2 N // theorbits in the power set. (h) h 8 2 () h A h (A) 8 2, 8A N. O (A) :{ (A) 2 }. h A h B 8B 2 O (A) O O (A) A 2 2 N,thepower set orbits. (Partitions 2 N ) Note: Multiple groups can yield the same power set orbits. i.e. can have O O 0 for 6 0.

Power Set Orbits, N 4 {N} {N} {N} {N} O(134) O(14) O(14) O(34) O(3) (a) O S4 O A4 (b) O D4 O C4 (c) O V h(12)(34),(13)(24)i (d) O S 2 4 h(12)(34)i Figure 1: Indecomposable power set orbits on N {1, 2, 3, 4} Power Set Orbits under di erent groups are also ordered by refinement of associated partition of the power set.

Power Set Orbits, N 5 {N} {N} O(1234) O(1234) O(124) (a) O S5 O A5 O A(1,5) (b) O D5 O C5 Figure 2: Indecomposable orbit structures on N {1, 2, 3, 4, 5} Observe that while a poset, need not be a lattice.

? Some Implications If apple 0,thenO apple O 0, i.e. partition O refines O 0 (extra group elements can force more equivalences) Thm. 2: IfO apple O 0,and, then 0 0. Equiv., if O apple O 0,and 0 ( 0,then (.

Chen & Yeung Partition Symmetrical Entropy Functions:? for S n1 S n2 S nk Thm. 1 (Qi Chen & R. W. Yeung): Let p {N 1, N 2,...,N t } be a t-partition of N, and p S N1 S N2 S Nt.For N 4, p p if and only if p {N} or {{i},n \{i}} for some i 2 N. What about general??remainingcases? For any, letp N // {{ (i) 2 } i 2N},then apple p Thm. 1: ( unless N // or N // {i, N\{i}}, i 2N. N // : is transitive. N // {i, N \}: S 1 0, fixes some i, then 0 on remainder is transitive on N\{i}. Priority (Pessimistic) Maximal Transitive Subgroups Priority (Optimistic) Minimal Transitive Subgroups

? Another Implication Thm. 3: Let S 1 act on N[{n +1} (fix n +1, acts on N ). Then S 1 S 1 ). Proof: orbits of O S1 eachorbitino repeated twice: once w/o n +1, and once w/ n +1. proj N S 1 : First2 N 1 coordinates are same subsets obeying same inequalities, must also have the remaining coordinates exist obeying more inequalities. : Takeh 2, extendtoh 0 on N[{n +1} via h 0 A h A\N,then h 0 2 S 1. proj N S1 : TakeX 1,...,X n from X 1,...,X n+1 realizing h 0 2 S1.Realizesa h 2. : Extend as previous, X n+1 0,deterministic.

? Complete Power Set Orbits, N 4 Set Orbits, N 4 Answer for N 4 O(14) power set orbits on N O(14) {1, 2, 3, 4} 14) O(34) s are also ordered by refinement of associated O(3) OV h(12)(34),(13)(24)i (d) OS 2 h(12)(34)i 4 O(134) O(14) Symmetric & Alternating: O(34) S4 : (1234), (12) A : (123), O(3) (12)(34) 4 O(134) 2) O(14) Dihedral & Cyclic: O(14) 6 (a) OS4 OA4 (b) OD4 OC4 (c) O (d) O O(34) 2 D4 : (1234), (13) V h(12)(34),(13)(24)i S4 h(12)(34)i ts, N 4 (2, 2)-Partition: O(3) C4 : (1234) S2 2, 3,S4} 2 : (12), (34) Figure 1: Indecomposable power set orbits on N {1, Normal Klein 4-group: Power Set Orbits under di erent groups are also ordered by refinement of associated (c) OV h(12)(34),(13)(24)i (d) OS 2 h(12)(34)iv : (12)(34), (13)(24) 4 partition of the power set. O(134) (a) OS4 OA4 6 Trivial: () (b) O (1, 3)-Partition:Figure 1: Inde S1 S3 : (234), (23) S1 Power A3 : Set (234), (243) Orbits under di partition of the power set. Double Transp.: S42 : (12)(34) 6 (1, 1, 2)-Partition: S1 S1 S2 : (34) 6

? Complete Answer for N 5 Symmetric, Alternating, & en. Affine: S5 : (12345), (12), A5 : (12345), (123) A(1, 5) : (12345), (2345) 6 (2, 3)-Partition: S2 S3 : (12), (345), (34) S2 A3 : (12), (345) S53 : (12)(45), (345) 6 (1, 1, 3)-partition: S1 S1 S3 : (345), (34) S1 S1 A3 : (345) 6 6 Fix 1, Rest Cyclic or Dihedral: S1 C4 : (2345) S1 D4 : (2345), (24) 6 S1 S42 O(1234) O(124) 6 O O S 5 4-group: A5 (a) Fix 1, Rest Klein S1 V : (23)(45), (24)(35) 6 (1, 2, 2)-partition: S1 S2 S2 : (23), (45) (1, 1, 1, 2)-partition: S1 S1 S1 S2 : (45) Power Set Orbits, N 5 O(1234) (1, 4)-Partition: Cyclic & Dihedral: S1 S4 : (2345), (23) C5 : (12345) S1 A4 : (2345), (23)(45) D5 : (12345), (25)(34) : (23)(45) OA(1,5) Figure 2: Indecomposable orbit structures on N {1, 2, 3, 4, 5} Observe that while a poset, need not be a lattice. 6 trivial: () (b) OD5 OC5

? Some Answers for N 6 Thm. 5: For n 6, Cn ( C n, Dn ( D n, S1 C n 1 ( S 1 C n 1, & S1 D n 1 ( S 1 D n 1. Proof shows that the ray with coordinates matching Vámos projection lies in C n is cut o by Zhang-Yeung non-shannon inequality. but

Summary Fig. 1(a) Symmetric & Alternating: S4 : (1234), (12) A4 : (123), (12)(34) n4: Fig. 1(b) Dihedral & Cyclic: 6 D4 : (1234), (13) (2, 2)-Partition: C4 : (1234) Fig. 1(c) S2 S2 : (12), (34) Normal Klein 4-group: V : (12)(34), (13)(24) (1, 3)-Partition: S1 S3 : (234), (23) S1 A3 : (234), (243) Double Transp.: S42 : (12)(34) Fig. 1(d) 6 n 6: Sn, Sn {1,..., n}// not a (1, n 6 S1 Cn, Cn 1 6 Dn 6 S1 Cn S1 Sn 6 1 6 S1 Sn S1 Dn 6 (1, 4)-Partition: S1 S4 : (2345), (23) S1 A4 : (2345), (23)(45) 6 6 Fix 1, Rest Klein 4-group: S1 V : (23)(45), (24)(35) S1 S42 : (23)(45) trivial: () 6 Cyclic & Dihedral: C5 : (12345) D5 : (12345), (25)(34) Fig. 2(b) 6 Fix 1, Rest Cyclic or Dihedral: S1 C4 : (2345) S1 D4 : (2345), (24) 1 6 (1, 2, 2)-partition: S1 S2 S2 : (23), (45) (1, 1, 1, 2)-partition: S1 S1 S1 S2 : (45) Dn S1 Dn 6 Symmetric, Alternating, & en. Affine: S5 : (12345), (12), A5 : (12345), (123) A(1, 5) : (12345), (2345) 6 (1, 1, 3)-partition: S1 S1 S3 : (345), (34) S1 S1 A3 : (345) (1, 1, 2)-Partition: S1 S1 S2 : (34) 1) or (n) partition ), 1 Trivial: () Cn Fig. 2(a) 6 Fig. 3 (2, 3)-Partition: S2 S3 : (12), (345), (34) S2 A3 : (12), (345) S53 : (12)(45), (345) 6 n5:

The Way Forward 1. For n 6, answer? for other maximal transitive groups. 2. Permutations of the ground set form just one set of interesting symmetries for N, others include Combinatorial Symmetry roup: permutations of the extreme rays which leave the face lattice intact. (Huge Subgroup of S M,w/M # of extreme rays) A ne Symmetry roup: Those Combinatorial symmetries whose ray permutation can be generated by multiplying by a 2 N 1 2 N 1 invertible matrix. (Also a large group) Restricted A ne Symmetry roup: Those a ne symmetries associated with vectors representing the rays of fixed length. (can be computed w/ sympol) 3. Which of the latter are also symmetries of N? 4. What sort of dimensionality reduction can be achieved by exploiting the a ne symmetries when calculating rate regions for network coding, storage repair tradeo s, caching regions, etc?

2024 © sciencedocbox.com Privacy Policy | Terms of Service | Feedback