The Capacity of a Network

Transcription

1 The Capacity of a Network April Rasala Lehman MIT Collaborators: Nick Harvey and Robert Kleinberg MIT

2 What is the Capacity of a Network? Source a Source b c d e Sink f Sink

3 What is the Capacity of a Network? Source Source a 1/2 c 1/2 b 1/2 1/2 e 1/2 d 1/2 f Sink Sink

4 What is the Capacity of a Network? Source bit x Source bit y a x x c y x y b y e Sink wants y d f Sink wants x

5 Information Fluid We understand fluid flows fairly well, but... Information can be copied and encoded. There is as yet no unified theory of network information flow. But there can be no doubt that a complete theory of communication networks would have wide implications for the theory of communication and computation. - Cover & Thomas, Elements of Information Theory. Little known about the capacity of information networks!

6 Theme: Blend Information Theory and Combinatorics Information theory: deep understanding of complex communication problems over structurally simple networks. Little understanding of arbitrary network structures Combinatorial techniques: deep understanding of simple (fluid) flows through structurally complex networks. But information can flow in different ways. Use ideas from both fields to understand capacity of information networks.

7 Combinatorial Background Ford-Fulkerson 62: Max-flow min-cut theorem. Hu 63: Two-commodity max-flow min-cut theorem. Okamura-Seymour 81: Gap between min-cut and max multicommodity flow. Leighton-Rao 99: Approximate max-flow min-cut theorem. Gap between sparsest cut and max-flow is O(log n). Expander example in which this gap is achieved.

8 Information Theory Background Shannon 48, Fujishige 78: Entropy is submodular, nonnonnegative, non-decreasing set function. Ahlswede et al Multicast capacity depends on a mincut condition. Jaggi et al Algorithm achieving capacity for multicast. Song, Yeung and Cai 03 - Outer (upper) bound on capacity for DAGS. Li and Li 04 - Conjecture: in undirected networks, max information flow = max multicommodity flow. Jain et al. graphs Verified conjecture for interesting class of

9 This Talk: Outer Bounds on Capacity Model for information flow in general graphs. Consider natural cut-based outer bound on capacity. Show bound is not tight. Develop useful information theoretic techniques. Improve bound using information theoretic techniques. Derive information inequalities from graph structure. Combine these using facts about entropy. Yields best known outer bound for general graphs. Special Case: use outer bound to verify Li and Li conjecture on same class of graphs as Jain et al. 05.

10 The Network Coding Problem S b T r S r T b Given: Graph G. c(u, v) for each edge (u, v). k-commodities: Set of sources. Set of sinks. Demand d i.

11 The Network Coding Problem S b T r S r T b Given: Graph G. c(u, v) = 1. k-commodities: Single source. Single sink. d i = 1.

12 Network Coding Solution A solution is a sequence of rules: S b M b M b M r M r S r Each rule specifies information transmitted on some edge. T r M b M r M b M r M r M b T b Source edge S i transmits M i. A non-source edge (u, v) transmits a function of information previously sent to u. Sink edge T i transmits M i. Allow multiple transmissions across each internal edge.

13 Rate of a Solution Messages, M i s, selected independently at random. Sequence of symbols transmitted on each edge is a R.V. Rate r solution, H(S i ) r for all commodities and Edges don t exceed capacity. - Directed graph: H(e) c(e). - Undirected graph: H( e ) + H( e ) c(e).

14 This Talk: Outer Bounds on Capacity Model for information flow in general graphs. Consider natural cut-based outer bound on capacity. Show bound is not tight. Develop useful information theoretic techniques. Improve bound using information theoretic techniques. Derive information inequalities from graph structure. Combine these using facts about entropy. Yields best known outer bound for general graphs. Special Case: use outer bound to verify Li and Li conjecture on same class of graphs as Jain et al. 05.

15 Sparsity Sparsity of a cut is: capacity of edges in cut # commodities with no remaining source-sink path Sparsity of a graph is minimum sparsity over all cuts. There exist directed graphs in which the maximum rate > sparsity. Sparsity = 1/2 Rate = 1

16 Meagerness A set of commodities P is separated by a cut if there is no remaining path from a source of any commodity in P to a sink of any commodity in P. The meagerness of a graph is the minimum over all sets of commodities P and cuts that separate P of capacity of edges in cut The maximum rate meagerness in directed graphs. P Meagerness = 1 Rate = 1

17 Sometimes Max Rate < Meagerness The meagerness is 1. This flow solution has rate 2/3. Best possible?

18 Sometimes Max Rate < Meagerness The meagerness is 1. This flow solution has rate 2/3. Best possible?

19 Sometimes Max Rate < Meagerness The meagerness is 1. This flow solution has rate 2/3. Best possible?

20 Better Bounds Through Entropy Show max rate 2/3 for previous example. Obtain strictly better bounds on rate through entropy arguments. Implies meagerness is a loose upper bound on rate. Motivates stronger outer bound based on structural conditions and characteristics of information.

21 Sg Informational Dominance Sb Sr V informationally dominates U if the information transmitted on edges in V is sufficient to determine the information transmitted on edges in U. F G If V informationally dominates U, H(V ) = H(U, V ). Tr Tb Tg Ex: {S g, F } informationally dominates T r. H(S g, F ) = H(S g, T r, F ). Ex: {S r, G} informationally dominates T g. H(S r, G) = H(S r, T g, G).

22 Sg Informational Dominance Sb Sr V informationally dominates U if the information transmitted on edges in V is sufficient to determine the information transmitted on edges in U. F G If V informationally dominates U, H(V ) = H(U, V ). Tr Tb Tg Ex: {S g, F } informationally dominates T r. H(S g, F ) = H(S g, T r, F ). Ex: {S r, G} informationally dominates T g. H(S r, G) = H(S r, T g, G).

23 Proof: Max Rate = 2/3

24 H(S g, F) = H(S g, T r, F) =

25 + = + H(S g, F) + H(S r, G) = H(S g, T r, F) + H(T g, S r, G)

26 + = +

27 + = + sources = sinks

28 + = + > + submodularity: H(A) + H(B) H(A B) + H(A B)

29 + = + > + informational dominance

30 + = + > + sources = sinks

31 + = + > + = 2(H(S g ) + H(S r )) + H(S b )

32 Max Rate = 2/3 + 2(H(S g ) + H(S r )) + H(S b )

33 Max Rate = 2/ (H(S g ) + H(S r )) + H(S b )

34 Max Rate = 2/ (H(S g ) + H(S r )) + H(S b )

35 Max Rate = 2/ (H(S g ) + H(S r )) + H(S b ) H(S g ) H(S r )

36 Max Rate = 2/3 + H(S g ) + H(S r ) + H(S b )

37 Max Rate = 2/3 2 + H(S g ) + H(S r ) + H(S b )

38 Max Rate = 2/ r

39 Generalize Proof Ideas Combine facts about entropy (e.g. submodularity) with constraints based on the structure of the graph. Generalize these ideas get a set of linear constraints on a network coding solution. Results in a large linear program whose optimal value is best known upper bound on capacity.

40 Constraints on Network Coding Solution Information Theoretic: Entropy is a submodular, nonnegative and nondecreasing set function. Independence of sources: H(S A, S B ) = H(S A ) + H(S B ). Rate of sources: H(S A ) r. sources = sinks: H(S A, U) = H(T A, U) for all U. Structural: Edge capacity: H(e) c(e). Informational dominance: Information transmitted on edges in A sufficient to compute information transmitted on edges in B: H(A) = H(A B) c(a) H(A B).

41 Constraints on Network Coding Solution Information Theoretic: Entropy is a submodular, nonnegative and nondecreasing set function. Independence of sources: H(S A, S B ) = H(S A ) + H(S B ). Rate of sources: H(S A ) r. sources = sinks: H(S A, U) = H(T A, U) for all U. Structural: Edge capacity: H(e) c(e). Informational dominance: Information transmitted on edges in A sufficient to compute information transmitted on edges in B: H(A) = H(A B) c(a) H(A B).

42 Informational Dominance Is A dominates B? decidable? Yes. We give a polynomial-time algorithm. Based on structural characterization of informational dominance. Examples of structural conditions implying informational dominance. In-edges to a set of nodes V informationally dominates all edges adjacent to V. A set of edges A informationally dominates S(i) and T (i) if S(i) and T (i) are in different weakly connected components of G \ A. These rules sufficient to prove interesting theorems.

43 This Talk: Outer Bounds on Capacity Model for information flow in general graphs. Consider natural cut-based outer bound on capacity. Show bound is not tight. Develop useful information theoretic techniques. Improve bound using information theoretic techniques. Derive information inequalities from graph structure. Combine these using facts about entropy. Yields best known outer bound for general graphs. Special Case: use outer bound to verify Li and Li conjecture on same class of graphs as Jain et al. 05.

44 Li and Li Conjecture Conjecture 1 In an undirected graphs: max network coding rate = max multicommodity flow rate Interesting cases when max flow < sparsity. Sparsity is an upper bound on rate for undirected graphs. Conjecture trivially true if multicommodity flow = sparsity. Use capacity constraints to verify for an infinite class of interesting graphs. Demonstrate proof for a small example.

45 Okamura-Seymour Example Undirected graph. 4-commodities. Demand 1 for all commodities. Capacity 1 for all edges. Sparsity 1. Max multicommodity flow = 3/4. Prove max network coding rate = 3/4.

46 = informational dominance

47 + = + informational dominance

48 + + = + + informational dominance

49 + + = + + sources = sinks

50 + + = submodularity

51 + + = submodularity

52 + + = informational dominance

53 + + = informational dominance

54 + + = H(S) + 2(H(S) + H(S) + H(S)) independence of sources

55 + + H(S) + 2(H(S) + H(S) + H(S))

56 + + + H(S) + 2(H(S) + H(S) + H(S)) Submodularity

57 H(S) + 2(H(S) + H(S) + H(S)) Submodularity

58 H(S) + 2(H(S) + H(S) + H(S)) Submodularity

59 H(S) H(S) + 2(H(S) + H(S) + H(S))

60 H(S) + + H(S) H(S) + 2(H(S) + H(S) + H(S))

61 H(S) + + H(S) + + H(S) + H(S) + 2(H(S) + H(S) + H(S))

62 + + H(S) + H(S) + H(S) + H(S)

63 + + H(S) + H(S) + H(S) + H(S) + H(S) + H(S) + H(S) + H(S)

64 + + H(S) + H(S) + H(S) + H(S) + + H(S) + H(S) + H(S) + H(S) 6 c(e) 2( H(S) + H(S) + H(S) + H(S) )

65 + + H(S) + H(S) + H(S) + H(S) + + H(S) + H(S) + H(S) + H(S) 6 8 r

66 General Theorem Theorem 2 Maximum network coding rate equals maximum multicommodity flow rate for graph which: are undirected and bipartite, have capacity 1 for all edges, have demand 1 for all commodities, and distance 2 between source and sink for all commodities. Corollary 3 First gap between sparsity and capacity of an information network.

67 Summary First outer bound on capacity for general graphs. Combined constraints based on graph structure with characteristics of entropy. Identified importance of informational dominance. Polynomial-time algorithm for informational dominance. Verified Li and Li conjecture for an infinite class of graphs. Demonstrated a gap between sparsity and capacity of information networks.

68 Open Questions Do we need non-shannon type information inequalities to get a tight upper bound on capacity? Is the Li and Li conjecture true for all undirected graphs? Is there a Ω(log n) between sparsity and capacity for 3- regular expanders? How do we use the full capacity of a network?