Entropy Power Inequalities: Results and Speculation

Transcription

1 Entropy Power Inequalities: Results and Speculation Venkat Anantharam EECS Department University of California, Berkeley December 12, 2013 Workshop on Coding and Information Theory Institute of Mathematical Research The University of Hong Kong Contributions involve joint work with Varun Jog V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

2 Outline 1 Setting the stage 2 The Brunn Minkowski inequality 3 Proofs of the EPI 4 Mrs. Gerber s Lemma 5 Young s inequality 6 Versions of the EPI for discrete random variables 7 EPI for Groups or Order 2 n 8 The Fourier transform on a finite abelian group 9 Young s inequality on a finite abelian group 10 Brunn Minkowski inequality on a finite abelian group 11 Speculation V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

3 Outline 1 Setting the stage 2 The Brunn Minkowski inequality 3 Proofs of the EPI 4 Mrs. Gerber s Lemma 5 Young s inequality 6 Versions of the EPI for discrete random variables 7 EPI for Groups or Order 2 n 8 The Fourier transform on a finite abelian group 9 Young s inequality on a finite abelian group 10 Brunn Minkowski inequality on a finite abelian group 11 Speculation V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

4 Differential entropy and entropy power V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

5 Differential entropy and entropy power Let X be an R n -valued random variable with density f. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

6 Differential entropy and entropy power Let X be an R n -valued random variable with density f. The differential entropy of X is h(x) := E[ log f (X)]. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

7 Differential entropy and entropy power Let X be an R n -valued random variable with density f. The differential entropy of X is h(x) := E[ log f (X)]. The entropy power of X is N(X) := 1 2πe e 2 n h(x). V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

8 Shannon s entropy power inequality V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

9 Shannon s entropy power inequality Theorem (Entropy Power Inequality) If X and Y are independent R n -valued random variables, then e 2 n h(x+y) e 2 n h(x) + e 2 n h(y), with equality if and only if X and Y are Gaussian with proportional covariance matrices. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

10 Notable applications of the EPI-1 V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

11 Notable applications of the EPI-2 V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

12 Notable applications of the EPI-3 V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

13 Notable applications of the EPI-4 V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

14 Outline 1 Setting the stage 2 The Brunn Minkowski inequality 3 Proofs of the EPI 4 Mrs. Gerber s Lemma 5 Young s inequality 6 Versions of the EPI for discrete random variables 7 EPI for Groups or Order 2 n 8 The Fourier transform on a finite abelian group 9 Young s inequality on a finite abelian group 10 Brunn Minkowski inequality on a finite abelian group 11 Speculation V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

15 The Minkowski sum of two subsets of R n V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

16 The Minkowski sum of two subsets of R n Minkowski sum of a square and a circle V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

17 The Minkowski sum of two subsets of R n Minkowski sum of a square and a circle In general the Minkowski sum of two sets A, B R n is A + B := {a + b : a A and b B}. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

18 The Brunn Minkowski inequality V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

19 The Brunn Minkowski inequality Let A, B R n be convex bodies (compact convex sets with nonempty interior). V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

20 The Brunn Minkowski inequality Let A, B R n be convex bodies (compact convex sets with nonempty interior). Then Vol(A + B) n Vol(A) n + Vol(B) n. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

21 The Brunn Minkowski inequality Let A, B R n be convex bodies (compact convex sets with nonempty interior). Then Vol(A + B) n Vol(A) n + Vol(B) n. Equality holds iff A and B are homothetic, i.e. equal up to translation and dilation. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

22 The Brunn Minkowski inequality Let A, B R n be convex bodies (compact convex sets with nonempty interior). Then Vol(A + B) n Vol(A) n + Vol(B) n. Equality holds iff A and B are homothetic, i.e. equal up to translation and dilation. An equivalent form is that for convex bodies A, B R n and 0 < λ < 1 we have Vol((1 λ)a + λb) 1 n (1 λ)vol(a) 1 n + λvol(b) 1 n. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

23 The Brunn Minkowski inequality Let A, B R n be convex bodies (compact convex sets with nonempty interior). Then Vol(A + B) n Vol(A) n + Vol(B) n. Equality holds iff A and B are homothetic, i.e. equal up to translation and dilation. An equivalent form is that for convex bodies A, B R n and 0 < λ < 1 we have Vol((1 λ)a + λb) 1 n (1 λ)vol(a) 1 n + λvol(b) 1 n. Another equivalent form is that for convex bodies A, B R n and 0 < λ < 1 we have Vol((1 λ)a + λb) 1 n min(vol(a), Vol(B)). V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

24 Parallels between the EPI and the Brunn Minkowski inequality V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

25 Parallels between the EPI and the Brunn Minkowski inequality The distribution of a random variable corresponds to a (measurable) subset of R n. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

26 Parallels between the EPI and the Brunn Minkowski inequality The distribution of a random variable corresponds to a (measurable) subset of R n. Addition of independent random variables corresponds to the Minkowski sum. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

27 Parallels between the EPI and the Brunn Minkowski inequality The distribution of a random variable corresponds to a (measurable) subset of R n. Addition of independent random variables corresponds to the Minkowski sum. Balls play the role of spherically symmetric Gaussians. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

28 Parallels between the EPI and the Brunn Minkowski inequality The distribution of a random variable corresponds to a (measurable) subset of R n. Addition of independent random variables corresponds to the Minkowski sum. Balls play the role of spherically symmetric Gaussians. For instance, an equivalent form of the EPI is that if X and Y are independent R n -valued random variables, and 0 < λ < 1, then h( λx + 1 λy) λh(x) + (1 λ)h(y). V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

29 Outline 1 Setting the stage 2 The Brunn Minkowski inequality 3 Proofs of the EPI 4 Mrs. Gerber s Lemma 5 Young s inequality 6 Versions of the EPI for discrete random variables 7 EPI for Groups or Order 2 n 8 The Fourier transform on a finite abelian group 9 Young s inequality on a finite abelian group 10 Brunn Minkowski inequality on a finite abelian group 11 Speculation V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

30 Some history V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

31 Some history The EPI was conjectured by Shannon, who showed that X and Y being Gaussian with proportional covariance matrices is a stationary point of the difference between the entropy power of the sum and sum of the entropy powers. This did not exclude the possibility of it being a local maximum or a saddle point. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

32 Some history The EPI was conjectured by Shannon, who showed that X and Y being Gaussian with proportional covariance matrices is a stationary point of the difference between the entropy power of the sum and sum of the entropy powers. This did not exclude the possibility of it being a local maximum or a saddle point. Stam gave the first rigorous proof of the EPI in 1959, using de Bruijn s identity which relates Fisher information and differential entropy. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

33 Some history The EPI was conjectured by Shannon, who showed that X and Y being Gaussian with proportional covariance matrices is a stationary point of the difference between the entropy power of the sum and sum of the entropy powers. This did not exclude the possibility of it being a local maximum or a saddle point. Stam gave the first rigorous proof of the EPI in 1959, using de Bruijn s identity which relates Fisher information and differential entropy. Blachman s 1965 paper gives a simplified and succinct version of Stam s proof. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

34 Some history The EPI was conjectured by Shannon, who showed that X and Y being Gaussian with proportional covariance matrices is a stationary point of the difference between the entropy power of the sum and sum of the entropy powers. This did not exclude the possibility of it being a local maximum or a saddle point. Stam gave the first rigorous proof of the EPI in 1959, using de Bruijn s identity which relates Fisher information and differential entropy. Blachman s 1965 paper gives a simplified and succinct version of Stam s proof. Subsequently, several proofs have appeared, some of which will be mentioned here for our story. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

35 Blachman s (1965) proof strategy for the EPI following Stam (1959) V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

36 Blachman s (1965) proof strategy for the EPI following Stam (1959) Step 1: Prove de Bruijn s identity for scalar random variables V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

37 Blachman s (1965) proof strategy for the EPI following Stam (1959) Step 1: Prove de Bruijn s identity for scalar random variables Let X t = X + tn, where N N (0, 1). Let X p(x), and X + tn p t (x t ). V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

38 Blachman s (1965) proof strategy for the EPI following Stam (1959) Step 1: Prove de Bruijn s identity for scalar random variables Let X t = X + tn, where N N (0, 1). Let X p(x), and X + tn p t (x t ). De Bruijn s Identity d dt h(x t) = 1 2 J(X t) = 1 2 ( p t (x t )) 2 dx t x t p t (x t ) V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

39 Blachman s (1965) proof strategy for the EPI following Stam (1959) Step 1: Prove de Bruijn s identity for scalar random variables Let X t = X + tn, where N N (0, 1). Let X p(x), and X + tn p t (x t ). De Bruijn s Identity d dt h(x t) = 1 2 J(X t) = 1 2 The proof follows from differentiating h(x t ) = ( p t (x t )) 2 dx t x t p t (x t ) p t (x t ) log p t (x t )dx t, with respect to t, and using the heat equation t p t(x t ) = t p t(x 2 t ). V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

40 Blachman s proof strategy (contd.) V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

41 Blachman s proof strategy (contd.) Step 2: Prove Stam s inequality V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

42 Blachman s proof strategy (contd.) Step 2: Prove Stam s inequality If X and Y are independent, and Z = X + Y then Stam s Inequality J(Z) J(X ) J(Y ) V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

43 Blachman s proof strategy (contd.) Step 2: Prove Stam s inequality If X and Y are independent, and Z = X + Y then Stam s Inequality J(Z) J(X ) J(Y ) The proof goes by showing that ( E a p X (x) p X (x) + b p Y (y) ) p Y (y) Z = z = (a + b) p Z (z) p Z (z), for all a, b. Then squaring both sides, using Jensen s inequality and optimizing over the choice of a, b establishes the inequality. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

44 Blachman s proof strategy (contd.) V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

45 Blachman s proof strategy (contd.) Step 3: Set up the one parameter flow defined by the heat equation starting with the two initial distributions, with suitable time reparametrizations f (t), g(t) for each, respectively V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

46 Blachman s proof strategy (contd.) Step 3: Set up the one parameter flow defined by the heat equation starting with the two initial distributions, with suitable time reparametrizations f (t), g(t) for each, respectively Let f (t) and g(t) be increasing functions tending to infinity with t, and consider the function s(t) = e2h(x f ) + e 2h(Yg ) e 2h(Z f +g ). V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

47 Blachman s proof strategy (contd.) Step 3: Set up the one parameter flow defined by the heat equation starting with the two initial distributions, with suitable time reparametrizations f (t), g(t) for each, respectively Let f (t) and g(t) be increasing functions tending to infinity with t, and consider the function s(t) = e2h(x f ) + e 2h(Yg ) e 2h(Z f +g ). Intuitively, lim t s(t) = 1, since both initial distributions become increasingly Gaussian as time progresses along the flow. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

48 Blachman s proof strategy (contd.) Step 3: Set up the one parameter flow defined by the heat equation starting with the two initial distributions, with suitable time reparametrizations f (t), g(t) for each, respectively Let f (t) and g(t) be increasing functions tending to infinity with t, and consider the function s(t) = e2h(x f ) + e 2h(Yg ) e 2h(Z f +g ). Intuitively, lim t s(t) = 1, since both initial distributions become increasingly Gaussian as time progresses along the flow. Choosing f (t) = exp(2h(x f )) and g (t) = exp(2h(y g )), and using Stam s inequality gives s (t) 0. This gives s(0) 1, proving the EPI. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

49 Lieb s EPI proof as interpreted by Verdú and Guo V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

50 Lieb s EPI proof as interpreted by Verdú and Guo Reparametrization of the de Bruijn identity gives the Guo-Shamai-Verdú I-MMSE relation: and thus d dγ I (X ; γx + N) = 1 mmse(x, γ), 2 h(x ) = 1 2 log 2πe γ mmse(x, γ)dγ. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

51 Lieb s EPI proof as interpreted by Verdú and Guo Reparametrization of the de Bruijn identity gives the Guo-Shamai-Verdú I-MMSE relation: and thus d dγ I (X ; γx + N) = 1 mmse(x, γ), 2 h(x ) = 1 2 log 2πe γ mmse(x, γ)dγ. The EPI in its equivalent form, h(x 1 cos α + X 2 sin α) cos 2 αh(x 1 ) + sin 2 αh(x 2 ), can be proved using these ideas. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

52 Verdú and Guo s proof: Proof strategy V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

53 Verdú and Guo s proof: Proof strategy h(x 1 cos α + X 2 sin α) cos 2 αh(x 1 ) + sin 2 αh(x 2 ) = ( 1 mmse(x 1 cos α + X 2 sin α) cos 2 α mmse(x 1, γ) 2 ) + sin 2 α mmse(x 2, γ) dγ V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

54 Verdú and Guo s proof: Proof strategy h(x 1 cos α + X 2 sin α) cos 2 αh(x 1 ) + sin 2 αh(x 2 ) = ( 1 mmse(x 1 cos α + X 2 sin α) cos 2 α mmse(x 1, γ) 2 ) + sin 2 α mmse(x 2, γ) dγ Consider Z 1 = γx 1 + N 1, Z 2 = γx 2 + N 2 and Z = Z 1 cos α + Z 2 sin α. Then mmse(x 1 cos α + X 2 sin α Z) mmse(x Z 1, Z 2 ) immediately gives the term inside the integral is 0. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

55 Szarek and Voiculescu s proof of the EPI V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

56 Szarek and Voiculescu s proof of the EPI Szarek and Voiculescu (1996) give a proof of the EPI working directly with typical sets, using a restricted version of the Brunn-Minkowski inequality. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

57 Szarek and Voiculescu s proof of the EPI Szarek and Voiculescu (1996) give a proof of the EPI working directly with typical sets, using a restricted version of the Brunn-Minkowski inequality. The restricted Minkowski sum of sets A and B, based on Θ A B is defined as A + Θ B := {x + y : (x, y) Θ}. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

58 Szarek and Voiculescu s proof of the EPI Szarek and Voiculescu (1996) give a proof of the EPI working directly with typical sets, using a restricted version of the Brunn-Minkowski inequality. The restricted Minkowski sum of sets A and B, based on Θ A B is defined as A + Θ B := {x + y : (x, y) Θ}. Szarek and Voiculescu s restricted B-M inequality says: For every ɛ, there exists δ such that if Θ is large enough, viz V 2n (Θ) (1 δ) n V n (A)V n (B), then V n (A + Θ B) 2 n (1 ɛ)(vn (A) 2 n + Vn (B) 2 n ). V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

59 Szarek and Voiculescu s proof (contd.) V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

60 Szarek and Voiculescu s proof (contd.) To prove the EPI, Szarek and Voiculescu replace A and B by typical sets for n i.i.d. copies of X and Y respectively, and define Θ as all pairs (x n, y n ) (each marginal typical) such that x n + y n is typical for X + Y. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

61 Szarek and Voiculescu s proof (contd.) To prove the EPI, Szarek and Voiculescu replace A and B by typical sets for n i.i.d. copies of X and Y respectively, and define Θ as all pairs (x n, y n ) (each marginal typical) such that x n + y n is typical for X + Y. For this choice of restriction, they determine suitable sequence of typicality defining constants (ɛ n, δ n ) going to 0, thus proving the EPI. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

62 Outline 1 Setting the stage 2 The Brunn Minkowski inequality 3 Proofs of the EPI 4 Mrs. Gerber s Lemma 5 Young s inequality 6 Versions of the EPI for discrete random variables 7 EPI for Groups or Order 2 n 8 The Fourier transform on a finite abelian group 9 Young s inequality on a finite abelian group 10 Brunn Minkowski inequality on a finite abelian group 11 Speculation V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

63 Mrs. Gerber s Lemma for Gaussian random variables V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

64 Mrs. Gerber s Lemma for Gaussian random variables n log(e 2 n x + e 2 n y ) is the minimum achievable differential entropy 2 of X + Y where X and Y are independent R n -valued random variables with differential entropies x and y respectively. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

65 Mrs. Gerber s Lemma for Gaussian random variables n log(e 2 n x + e 2 n y ) is the minimum achievable differential entropy 2 of X + Y where X and Y are independent R n -valued random variables with differential entropies x and y respectively. Note that (x, y) n 2 log(e 2 n x + e 2 n y ) is convex in (x, y). V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

66 Mrs. Gerber s Lemma for Gaussian random variables n log(e 2 n x + e 2 n y ) is the minimum achievable differential entropy 2 of X + Y where X and Y are independent R n -valued random variables with differential entropies x and y respectively. Note that (x, y) n 2 log(e 2 n x + e 2 n y ) is convex in (x, y). In particular, for fixed x, is convex in y and for fixed y, is convex in x. y n 2 log(e 2 n x + e 2 n y ) x n 2 log(e 2 n x + e 2 n y ) V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

67 Outline 1 Setting the stage 2 The Brunn Minkowski inequality 3 Proofs of the EPI 4 Mrs. Gerber s Lemma 5 Young s inequality 6 Versions of the EPI for discrete random variables 7 EPI for Groups or Order 2 n 8 The Fourier transform on a finite abelian group 9 Young s inequality on a finite abelian group 10 Brunn Minkowski inequality on a finite abelian group 11 Speculation V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

68 Young s inequality V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

69 Young s inequality The Hausdorff-Young inequality gives an estimate on the norm of the Fourier transform: Ff p f p, for 1 p 2. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

70 Young s inequality The Hausdorff-Young inequality gives an estimate on the norm of the Fourier transform: Ff p f p, for 1 p 2. The sharp constant in the inequality was found by Beckner Ff p C p f p, where C p := p 1/p. p 1/p V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

71 Young s inequality The Hausdorff-Young inequality gives an estimate on the norm of the Fourier transform: Ff p f p, for 1 p 2. The sharp constant in the inequality was found by Beckner Ff p C p f p, where C p := p 1/p. p 1/p This leads to Young s inequality. If p, q, r > 0 satisfy 1 p + 1 q = r, then f g r C pc q C r f p g q, if p, q, r 1, f g r C pc q C r f p g q, if 1 p, q, r. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

72 Young s inequality The Hausdorff-Young inequality gives an estimate on the norm of the Fourier transform: Ff p f p, for 1 p 2. The sharp constant in the inequality was found by Beckner Ff p C p f p, where C p := p 1/p. p 1/p This leads to Young s inequality. If p, q, r > 0 satisfy 1 p + 1 q = r, then f g r C pc q C r f p g q, if p, q, r 1, f g r C pc q C r f p g q, if 1 p, q, r. The second half of this is called the reverse Young s inequality. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

73 EPI from the Young s inequality V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

74 EPI from the Young s inequality For p > 1, h p (X) := X f. p 1 p f p is called the Renyi entropy of V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

75 EPI from the Young s inequality For p > 1, h p (X) := p f 1 p p is called the Renyi entropy of X f. N p (X) := 1 /p 2π p p f 2p /n p is the Renyi entropy power of X f. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

76 EPI from the Young s inequality For p > 1, h p (X) := p f 1 p p is called the Renyi entropy of X f. N p (X) := 1 /p 2π p p f 2p /n p is the Renyi entropy power of X f. As p 1 these converge respectively to the differential entropy and the entropy power of X. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

77 EPI from the Young s inequality For p > 1, h p (X) := p f 1 p p is called the Renyi entropy of X f. N p (X) := 1 /p 2π p p f 2p /n p is the Renyi entropy power of X f. As p 1 these converge respectively to the differential entropy and the entropy power of X. r Given r > 1 and 0 < λ < 1 define p(r) := and (1 λ)+λr q(r) := r. λ+(1 λ)r V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

78 EPI from the Young s inequality For p > 1, h p (X) := p f 1 p p is called the Renyi entropy of X f. N p (X) := 1 /p 2π p p f 2p /n p is the Renyi entropy power of X f. As p 1 these converge respectively to the differential entropy and the entropy power of X. r Given r > 1 and 0 < λ < 1 define p(r) := and (1 λ)+λr r q(r) :=. λ+(1 λ)r Young s inequality gives N r (X + Y) ( Np (X) 1 λ ) 1 λ ( ) λ Nq (Y). λ V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

79 EPI from the Young s inequality For p > 1, h p (X) := p f 1 p p is called the Renyi entropy of X f. N p (X) := 1 /p 2π p p f 2p /n p is the Renyi entropy power of X f. As p 1 these converge respectively to the differential entropy and the entropy power of X. r Given r > 1 and 0 < λ < 1 define p(r) := and (1 λ)+λr r q(r) :=. λ+(1 λ)r Young s inequality gives N r (X + Y) ( Np (X) 1 λ ) 1 λ ( ) λ Nq (Y). λ Optimize over λ and let r 1 to get the EPI. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

80 Brunn Minkowski inequality from the reverse Young s inequality V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

81 Brunn Minkowski inequality from the reverse Young s inequality The limit as r 0 in Young s inequality leads to the Prékopa-Leindler inequality: If 0 < λ < 1 and f, g, h are nonnegative integrable functions on R n satisfying h((1 λ)x + λy) f (x) 1 λ g(y) λ for all x, y R n, then ( ) 1 λ ( ) λ h(x)dx R n f (x)dx R n g(x)dx R n. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

82 Brunn Minkowski inequality from the reverse Young s inequality The limit as r 0 in Young s inequality leads to the Prékopa-Leindler inequality: If 0 < λ < 1 and f, g, h are nonnegative integrable functions on R n satisfying h((1 λ)x + λy) f (x) 1 λ g(y) λ for all x, y R n, then ( ) 1 λ ( ) λ h(x)dx R n f (x)dx R n g(x)dx R n. Setting f := 1 A, g := 1 B, and h := 1 (1 λ)a+λb, this gives Vol((1 λ)a+λb) Vol(A) 1 λ Vol(B) λ min(vol(a), Vol(B)). V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

83 Outline 1 Setting the stage 2 The Brunn Minkowski inequality 3 Proofs of the EPI 4 Mrs. Gerber s Lemma 5 Young s inequality 6 Versions of the EPI for discrete random variables 7 EPI for Groups or Order 2 n 8 The Fourier transform on a finite abelian group 9 Young s inequality on a finite abelian group 10 Brunn Minkowski inequality on a finite abelian group 11 Speculation V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

84 Mrs. Gerber s lemma V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

85 Mrs. Gerber s lemma Let h be the binary entropy function, and h 1 be its inverse. Wyner and Ziv (1973) showed that for a binary X and arbitrary U, if Z Bern(p) is independent of (X, U) then h(x Z U) h(h 1 (H(X U) p). V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

86 Mrs. Gerber s lemma Let h be the binary entropy function, and h 1 be its inverse. Wyner and Ziv (1973) showed that for a binary X and arbitrary U, if Z Bern(p) is independent of (X, U) then h(x Z U) h(h 1 (H(X U) p). The result follows immediately from the following lemma, using Jensen s inequality - Lemma The function is convex on 0 x log 2. x h(h 1 (x) p) V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

87 Wyner and Ziv s proof of Mrs. Gerber s lemma V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

88 Wyner and Ziv s proof of Mrs. Gerber s lemma Introduce the parametrization α = 1 h 1 (x), let p = 1 a, and 2 2 differentiate wrt α, to get that f (x) 0 a(1 α 2 ) log 1 + α 1 α (1 aα)2 log 1 + aα 1 aα, which can be verified using the series expansion of log 1+x 1 x. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

89 Shamai and Wyner s EPI V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

90 Shamai and Wyner s EPI Shamai and Wyner show that - EPI for binary sequences For binary independent processes with entropies H(X ) and H(Y ), σ(z) σ(x ) σ(y ), where Z = X Y and σ(x ) = h 1 (H(X )), σ(y ) = h 1 (H(Y ) and σ(z) = h 1 (H(Z)). V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

91 Shamai and Wyner s proof strategy for the binary EPI V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

92 Shamai and Wyner s proof strategy for the binary EPI H(Z 0 Z 1 n ) H(Z 0 Z 1 n, X 1 n, Y 1 n ) = H(Z 0 X 1 n, Y 1 n ), = P(X 1 n = x)p(y 1 n = y)h(z 0 X 1 n = x, Y 1 n = y) = P(X 1 n = x)p(y 1 n = y)h(α(x) β(y)) Where α(x) = P(X 0 = 1 X 1 n = x), β(x) = P(Y 0 = 1 Y 1 n = y). V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

93 Shamai and Wyner s proof strategy for the binary EPI H(Z 0 Z 1 n ) H(Z 0 Z 1 n, X 1 n, Y 1 n ) = H(Z 0 X 1 n, Y 1 n ), = P(X 1 n = x)p(y 1 n = y)h(z 0 X 1 n = x, Y 1 n = y) = P(X 1 n = x)p(y 1 n = y)h(α(x) β(y)) Where α(x) = P(X 0 = 1 X 1 n = x), β(x) = P(Y 0 = 1 Y 1 n = y). Using convexity in MGL twice, summing over x and then y, we get H(Z 0 Z 1 n ) h(h 1 (H(X 0 X 1 n ) h 1 (H(Y 0 Y 1 n )). Taking n proves the result. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

94 Harremoës s EPI for the Binomial family V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

95 Harremoës s EPI for the Binomial family Let X n Binomial(n, 1 ), then for all m, n 1, 2 e 2H(Xn+Xm) e 2H(Xn) + e 2H(Xm). V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

96 Harremoës s EPI for the Binomial family Let X n Binomial(n, 1 ), then for all m, n 1, 2 e 2H(Xn+Xm) e 2H(Xn) + e 2H(Xm). The proof follows by showing that Y n = e2h(xn) is an increasing n sequence, and thus is super additive, i.e Y m+n Y m + Y n. Figure : Values of Y n V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

97 Our approach V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

98 Our approach Suppose the random variables take values on a finite abelian group G. We define the function f G : [0, log G ] [0, log G ] [0, log G ] by f G (x, y) = min H(X + Y ) H(X )=x,h(y )=y V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

99 Our approach Suppose the random variables take values on a finite abelian group G. We define the function f G : [0, log G ] [0, log G ] [0, log G ] by f G (x, y) = We now ask the questions- min H(X + Y ) H(X )=x,h(y )=y Does f G have a succinct description? V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

100 Our approach Suppose the random variables take values on a finite abelian group G. We define the function f G : [0, log G ] [0, log G ] [0, log G ] by f G (x, y) = We now ask the questions- min H(X + Y ) H(X )=x,h(y )=y Does f G have a succinct description? Does f G have any nice properties? V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

101 Our approach Suppose the random variables take values on a finite abelian group G. We define the function f G : [0, log G ] [0, log G ] [0, log G ] by f G (x, y) = We now ask the questions- min H(X + Y ) H(X )=x,h(y )=y Does f G have a succinct description? Does f G have any nice properties? We try to exploit the group structure to answer these questions. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

102 Mimicking Blachman s proof? V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

103 Mimicking Blachman s proof? Find a 1-parameter group to evolve the probability distributions towards the uniform distribution on the group? V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

104 Mimicking Blachman s proof? Find a 1-parameter group to evolve the probability distributions towards the uniform distribution on the group? Perhaps the analogs of the Gaussian distributions are the distributions of the form 1 G 1 p(0) = 1 ɛ, p(g) = ɛ for g 0, 0 ɛ? G 1 G V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

105 Mimicking Blachman s proof? Find a 1-parameter group to evolve the probability distributions towards the uniform distribution on the group? Perhaps the analogs of the Gaussian distributions are the distributions of the form 1 G 1 p(0) = 1 ɛ, p(g) = ɛ for g 0, 0 ɛ? G 1 G But these are not extremal for the EPI. For instance, for G = Z 2 Z 2, if ɛ is chosen so that h(ɛ) + ɛ log 3 = log 2 (note that ɛ < 3 ), then we have 4 h(1 2ɛ ɛ2 ) + 2ɛ(1 2 ɛ) log 3 > log 2. 3 V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

106 Mimicking Blachman s proof? Find a 1-parameter group to evolve the probability distributions towards the uniform distribution on the group? Perhaps the analogs of the Gaussian distributions are the distributions of the form 1 G 1 p(0) = 1 ɛ, p(g) = ɛ for g 0, 0 ɛ? G 1 G But these are not extremal for the EPI. For instance, for G = Z 2 Z 2, if ɛ is chosen so that h(ɛ) + ɛ log 3 = log 2 (note that ɛ < 3 ), then we have 4 h(1 2ɛ ɛ2 ) + 2ɛ(1 2 ɛ) log 3 > log 2. 3 However, for the uniform distribution on a subgroup, its convolution with itself has entropy log 2. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

107 Mimicking Lieb s proof? V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

108 Mimicking Lieb s proof? Given group valued random variables X 1 and X 2, what is the analog of X 1 cos α + X 2 sin α? V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

109 Mimicking Lieb s proof? Given group valued random variables X 1 and X 2, what is the analog of X 1 cos α + X 2 sin α? There is a version of Stam s inequality for finite abelian groups (Gibilisco and Isola, 2008). Given a set of generators Γ := {γ 1, γ 2,..., γ m } for the group G, define the Fisher information of a random variable X by J(X ) := ( ) px (g) p X (γ 1 2 g) p X (g). p X (g) γ Γ g G 1 Then one has 1 J(X 1 +X 2 ) random variables X 1 and X J(X 1 ) J(X 2 ) for independent G-valued V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

110 Mimicking Lieb s proof? Given group valued random variables X 1 and X 2, what is the analog of X 1 cos α + X 2 sin α? There is a version of Stam s inequality for finite abelian groups (Gibilisco and Isola, 2008). Given a set of generators Γ := {γ 1, γ 2,..., γ m } for the group G, define the Fisher information of a random variable X by J(X ) := ( ) px (g) p X (γ 1 2 g) p X (g). p X (g) γ Γ g G 1 Then one has J(X 1 +X 2 ) J(X 1 ) J(X 2 for independent G-valued ) random variables X 1 and X 2. However this notion of Fisher information is merely mimicking formal properties of the Fisher information from the continuous case. It seems to have little to do with estimation, which, for finite groups ought to refer to likelihood ratio type quantities. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

111 Mimicking Szarek and Voiculescu s proof? V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

112 Mimicking Szarek and Voiculescu s proof? This is a promising direction. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

113 Mimicking Szarek and Voiculescu s proof? This is a promising direction. The question remains, what is the correct analog of the restricted Minkowski sum? V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

114 Mimicking Szarek and Voiculescu s proof? This is a promising direction. The question remains, what is the correct analog of the restricted Minkowski sum? The key lemma driving the proof of Szarek and Voiculescu appears to be Euclidean in character. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

115 Outline 1 Setting the stage 2 The Brunn Minkowski inequality 3 Proofs of the EPI 4 Mrs. Gerber s Lemma 5 Young s inequality 6 Versions of the EPI for discrete random variables 7 EPI for Groups or Order 2 n 8 The Fourier transform on a finite abelian group 9 Young s inequality on a finite abelian group 10 Brunn Minkowski inequality on a finite abelian group 11 Speculation V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

116 Our results V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

117 Our results For a group G of order 2 n, we explicitly describe f G in terms of f Z2. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

118 Our results For a group G of order 2 n, we explicitly describe f G in terms of f Z2. We also describe the minimum entropy achieving distributions on G, these distributions are analogs of Gaussians in this sense. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

119 Our results For a group G of order 2 n, we explicitly describe f G in terms of f Z2. We also describe the minimum entropy achieving distributions on G, these distributions are analogs of Gaussians in this sense. The f G (x, y) function obtained has the property that it is convex in x for a fixed y. This is yet another version of Mrs. Gerber s Lemma. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

120 Statement of our results V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

121 Statement of our results Theorem f G depends only on the size of G, and is denoted by f 2 n, where f 2 (x k log 2, y k log 2) + k log 2, f 2 n(x, y) = if k log 2 x, y (k + 1) log 2, max(x, y) otherwise. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

122 Visualizing our results V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

123 Visualizing our results (contd.) V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

124 A conjecture V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

125 A conjecture Generalized Mrs. Gerber s Lemma If G is a finite group, then f G (x, y) is convex in x for a fixed y, and by symmetry convex in y for a fixed x. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

126 A conjecture Generalized Mrs. Gerber s Lemma If G is a finite group, then f G (x, y) is convex in x for a fixed y, and by symmetry convex in y for a fixed x. If one is less optimistic, one might make this conjecture only for finite abelian groups. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

127 A conjecture Generalized Mrs. Gerber s Lemma If G is a finite group, then f G (x, y) is convex in x for a fixed y, and by symmetry convex in y for a fixed x. If one is less optimistic, one might make this conjecture only for finite abelian groups. Simulations for Z 3 and Z 5 seem to support this conjecture. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

128 A conjecture Generalized Mrs. Gerber s Lemma If G is a finite group, then f G (x, y) is convex in x for a fixed y, and by symmetry convex in y for a fixed x. If one is less optimistic, one might make this conjecture only for finite abelian groups. Simulations for Z 3 and Z 5 seem to support this conjecture. We saw already that f 2 n satisfies the conjecture. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

129 EPI for abelian groups of order 2 n : proof strategy V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

130 EPI for abelian groups of order 2 n : proof strategy First observe that for Z 2 the function f Z2 (x, y) is explicitly given by f Z2 (x, y) = h(h 1 (x) h 1 (y)). V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

131 EPI for abelian groups of order 2 n : proof strategy First observe that for Z 2 the function f Z2 (x, y) is explicitly given by f Z2 (x, y) = h(h 1 (x) h 1 (y)). The proof proceeds by first proving the EPI for Z 4, using convexity properties of f Z2 and concavity of entropy. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

132 EPI for abelian groups of order 2 n : proof strategy First observe that for Z 2 the function f Z2 (x, y) is explicitly given by f Z2 (x, y) = h(h 1 (x) h 1 (y)). The proof proceeds by first proving the EPI for Z 4, using convexity properties of f Z2 and concavity of entropy. We then use induction, where we assume that the result holds for all groups of size 2 n and prove it groups of size 2 n+1. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

133 EPI for abelian groups of order 2 n : proof strategy First observe that for Z 2 the function f Z2 (x, y) is explicitly given by f Z2 (x, y) = h(h 1 (x) h 1 (y)). The proof proceeds by first proving the EPI for Z 4, using convexity properties of f Z2 and concavity of entropy. We then use induction, where we assume that the result holds for all groups of size 2 n and prove it groups of size 2 n+1. The induction part of the proof has almost exactly the same structure as the proof for Z 4. Thus proving the result for Z 4 is the key step in our proof. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

134 Proof for Z 4 V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

135 Proof for Z 4 For Z 4, splitting the support of distributions on Z 4 into two parts, {0, 2} (even part) and{1, 3} (odd part) and using precisely two ingredients: concavity of entropy, and convexity in MGL we arrive at the lower bound Lower bound f 4 (x, y) min u,v f 2(u, v) + f 2 (x u, y v) V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

136 Proof for Z 4 For Z 4, splitting the support of distributions on Z 4 into two parts, {0, 2} (even part) and{1, 3} (odd part) and using precisely two ingredients: concavity of entropy, and convexity in MGL we arrive at the lower bound Lower bound Lemma f Z2 f 4 (x, y) min u,v f 2(u, v) + f 2 (x u, y v) To evaluate the minimum, we prove some additional properties of f Z2, specifically regarding its behavior along lines through the origin. The key Lemma we use is (and by symmetry, f Z 2 ) strictly decreases along lines through the x y origin V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

137 Proving the lemma V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

138 Proving the lemma We use the parametrization h 1 (x) = p and h 1 (y) = q. Our strategy involves brute force differentiation, and a series of steps where we conclude that proving the lemma is equivalent to proving F (p) = p 2 (log(p)) 2 (1 p) 2 (log(1 p)) 2 + (1 2p) (log(p) log(1 p) + p log(p) + (1 p) log(1 p)) 0 V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

139 Proving the lemma We use the parametrization h 1 (x) = p and h 1 (y) = q. Our strategy involves brute force differentiation, and a series of steps where we conclude that proving the lemma is equivalent to proving F (p) = p 2 (log(p)) 2 (1 p) 2 (log(1 p)) 2 + (1 2p) (log(p) log(1 p) + p log(p) + (1 p) log(1 p)) 0 We show that proving F (p) 0 is the same as proving d 5 dp 5 F (p) 0, we differentiate F five times, use a polynomial approximation of log, and finally use Sturm s theorem to prove that the resulting polynomial is non-positive. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

140 Outline 1 Setting the stage 2 The Brunn Minkowski inequality 3 Proofs of the EPI 4 Mrs. Gerber s Lemma 5 Young s inequality 6 Versions of the EPI for discrete random variables 7 EPI for Groups or Order 2 n 8 The Fourier transform on a finite abelian group 9 Young s inequality on a finite abelian group 10 Brunn Minkowski inequality on a finite abelian group 11 Speculation V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

141 Norm of the Fourier transform on a finite abelian group V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

142 Norm of the Fourier transform on a finite abelian group Every finite abelian group G has a dual group Ĝ comprised of the characters of G, i.e. the homomorphisms from G to the unit circle in the complex plane. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

143 Norm of the Fourier transform on a finite abelian group Every finite abelian group G has a dual group Ĝ comprised of the characters of G, i.e. the homomorphisms from G to the unit circle in the complex plane. Ĝ has the same cardinality and structure as G and G can be thought of as the dual of Ĝ. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

144 Norm of the Fourier transform on a finite abelian group Every finite abelian group G has a dual group Ĝ comprised of the characters of G, i.e. the homomorphisms from G to the unit circle in the complex plane. Ĝ has the same cardinality and structure as G and G can be thought of as the dual of Ĝ. The Fourier transform on the group is the map from functions on 1 G to functions on Ĝ given by F(f )(γ) := G g γ(g)f (g). 1/2 V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

145 Norm of the Fourier transform on a finite abelian group Every finite abelian group G has a dual group Ĝ comprised of the characters of G, i.e. the homomorphisms from G to the unit circle in the complex plane. Ĝ has the same cardinality and structure as G and G can be thought of as the dual of Ĝ. The Fourier transform on the group is the map from functions on 1 G to functions on Ĝ given by F(f )(γ) := G g γ(g)f (g). 1/2 With counting measure on G and Ĝ, we have, for p, q > 0, F : L p q (G) L (Ĝ). What is the norm of this map? V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

146 Norm of the Fourier transform on a finite abelian group Every finite abelian group G has a dual group Ĝ comprised of the characters of G, i.e. the homomorphisms from G to the unit circle in the complex plane. Ĝ has the same cardinality and structure as G and G can be thought of as the dual of Ĝ. The Fourier transform on the group is the map from functions on 1 G to functions on Ĝ given by F(f )(γ) := G g γ(g)f (g). 1/2 With counting measure on G and Ĝ, we have, for p, q > 0, F : L p q (G) L (Ĝ). What is the norm of this map? Resolved by Gilbert and Rzeszotnik (2010). V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

147 Visualizing the norm of the FT on a finite abelian group Norm of the Fourier Transform on a finite abelian group V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

148 Outline 1 Setting the stage 2 The Brunn Minkowski inequality 3 Proofs of the EPI 4 Mrs. Gerber s Lemma 5 Young s inequality 6 Versions of the EPI for discrete random variables 7 EPI for Groups or Order 2 n 8 The Fourier transform on a finite abelian group 9 Young s inequality on a finite abelian group 10 Brunn Minkowski inequality on a finite abelian group 11 Speculation V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

149 Young s inequality on a finite abelian group V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

150 Young s inequality on a finite abelian group There is a forward Young s inequality, which reads: For every p, q, r 1 such that 1 p + 1 q = r we have f g r f p g q. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

151 Outline 1 Setting the stage 2 The Brunn Minkowski inequality 3 Proofs of the EPI 4 Mrs. Gerber s Lemma 5 Young s inequality 6 Versions of the EPI for discrete random variables 7 EPI for Groups or Order 2 n 8 The Fourier transform on a finite abelian group 9 Young s inequality on a finite abelian group 10 Brunn Minkowski inequality on a finite abelian group 11 Speculation V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

152 Brunn Minkowski inequality on a finite abelian group V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

153 Brunn Minkowski inequality on a finite abelian group For a finite abelian group G, consider the function µ G (r, s) := min{ A + B : A, B G, A = r, B = s}. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

154 Brunn Minkowski inequality on a finite abelian group For a finite abelian group G, consider the function µ G (r, s) := min{ A + B : A, B G, A = r, B = s}. Then we have µ G (r, s) = min d divides G ( r d + s d 1) d. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

155 Brunn Minkowski inequality on a finite abelian group For a finite abelian group G, consider the function µ G (r, s) := min{ A + B : A, B G, A = r, B = s}. Then we have µ G (r, s) = min d divides G ( r d + s d 1) d. In particular for G a finite abelian group of size n = 2 k, we have µ G (r, s) = r s where r s is the Hopf-Stiefel function defined as the smallest positive integer m for which the polynomial (x + y) m falls in the ideal generated by x r and y s in the polynomial ring F 2 [x, y]. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

156 Brunn Minkowski inequality on a finite abelian group For a finite abelian group G, consider the function µ G (r, s) := min{ A + B : A, B G, A = r, B = s}. Then we have µ G (r, s) = min d divides G ( r d + s d 1) d. In particular for G a finite abelian group of size n = 2 k, we have µ G (r, s) = r s where r s is the Hopf-Stiefel function defined as the smallest positive integer m for which the polynomial (x + y) m falls in the ideal generated by x r and y s in the polynomial ring F 2 [x, y]. Involved history of partial results. See Eliahu and Kervaire (2007) for the history. V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

157 Outline 1 Setting the stage 2 The Brunn Minkowski inequality 3 Proofs of the EPI 4 Mrs. Gerber s Lemma 5 Young s inequality 6 Versions of the EPI for discrete random variables 7 EPI for Groups or Order 2 n 8 The Fourier transform on a finite abelian group 9 Young s inequality on a finite abelian group 10 Brunn Minkowski inequality on a finite abelian group 11 Speculation V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

158 Questions V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

159 Questions What form does the reverse Young s inequality take on finite abelian groups? V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

160 Questions What form does the reverse Young s inequality take on finite abelian groups? Less ambitiously, what form does the reverse Young s inequality take on finite groups of size 2 k? V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

161 Questions What form does the reverse Young s inequality take on finite abelian groups? Less ambitiously, what form does the reverse Young s inequality take on finite groups of size 2 k? Is the EPI we proved for finite abelian groups of size 2 k derivable as a limit from the Young s inequality for such groups? V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

162 Questions What form does the reverse Young s inequality take on finite abelian groups? Less ambitiously, what form does the reverse Young s inequality take on finite groups of size 2 k? Is the EPI we proved for finite abelian groups of size 2 k derivable as a limit from the Young s inequality for such groups? Is the Brunn Minkowski inequality for finite abelian groups of size 2 k (expressed via the Hopf-Stiefel function) derivable as a limit from the appropriate reverse Young s inequality for such groups? V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

163 Questions What form does the reverse Young s inequality take on finite abelian groups? Less ambitiously, what form does the reverse Young s inequality take on finite groups of size 2 k? Is the EPI we proved for finite abelian groups of size 2 k derivable as a limit from the Young s inequality for such groups? Is the Brunn Minkowski inequality for finite abelian groups of size 2 k (expressed via the Hopf-Stiefel function) derivable as a limit from the appropriate reverse Young s inequality for such groups? Is the general Brunn Minkowski inequality for finite abelian groups derivable as a limit from the appropriate reverse Young s inequality for such groups? V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

164 Questions What form does the reverse Young s inequality take on finite abelian groups? Less ambitiously, what form does the reverse Young s inequality take on finite groups of size 2 k? Is the EPI we proved for finite abelian groups of size 2 k derivable as a limit from the Young s inequality for such groups? Is the Brunn Minkowski inequality for finite abelian groups of size 2 k (expressed via the Hopf-Stiefel function) derivable as a limit from the appropriate reverse Young s inequality for such groups? Is the general Brunn Minkowski inequality for finite abelian groups derivable as a limit from the appropriate reverse Young s inequality for such groups? Will a limit of the Young s inequality for a general finite Abelian group give rise to an EPI for each such group? V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57

165 The End V. Jog and V. Anantharam (UC Berkeley) EPI December 12, / 57