Improved log-sobolev inequalities, hypercontractivity and uncertainty principle on the hypercube

Transcription

1 Improved log-sobolev inequalities, hypercontractivity and uncertainty principle on the hypercube Yury Polyanskiy a, Alex Samorodnitsky b a Department of Electrical Engineering and Computer Science, MIT, Cambridge, MA 039 USA. b School of Engineering and Computer Science, The Hebrew University of Jerusalem, Jerusalem 9904, Israel. Abstract We develop a new class of log-sobolev inequalities (LSIs) that provide a nonlinear comparison between the entropy and the Dirichlet form. For the hypercube, these LSIs imply a new version of the hypercontractivity for functions of small support. As a consequence, we derive a sharp form of the uncertainty principle for the hypercube: a function whose energy is concentrated on a set of small size, and whose Fourier energy is concentrated on a small Hamming ball must be zero. The tradeoff we derive is asymptotically optimal. We observe that for the Euclidean space, an analogous (asymptotically optimal) tradeoff follows from the sharp form of Young s inequality due to Beckner. As an application, we show how uncertainty principle implies a new estimate of the metric properties of linear maps F k F n. Keywords: Hamming space, log-sobolev inequality, hypercontractivity, Fourier transform on the hypercube, uncertainty principle, coding theory Contents Introduction. Definitions, background Motivation and Organization Non-linear log-sobolev inequalities 4 3 New LSIs and hypercontractivity for the hypercube 9 4 Uncertainty principle on the hypercube 4 4. Background Sharp uncertainty principle on F n addresses: yp@mit.edu (Yury Polyanskiy), salex@cs.huji.ac.il (Alex Samorodnitsky) June 3, 06

2 4.3 Discussion A similar result for Euclidean space An application to coding theory 5. Method - graph covers Method (analytic) Introduction.. Definitions, background We introduce some standard notions for continous-time semigroups on finite state spaces, e.g. [, Section.7.]. Consider a finite alphabet X and a matrix (L x,y ) x,y X such that ) L x,y 0 for x y; and ) y X L x,y = 0 for all x. Then T t = e tl is a stochastic semigroup, for which we assume that π is a stationary measure. We define f p = E p [ f p ] and (f, g) = E [fg] with expectation over π. The Dirichlet form of semigroup T t is E(f, g) = x,y L x,y f(y)g(x)π(x) = E π [( Lf)g]. We also define Tt n form is given by a product semigroup on X n and notice that its Dirichlet E n (f, g) = n h(xˆk) π(x j ), () xˆk X n j k k= where xˆk = (x,..., x k, x k+,..., x n ) and h(xˆk) = E(f(xˆk, ), g(xˆk, )) is the action of Dirichlet form E on k-th coordinate of f and g with other coordinates held frozen. We say that a semigroup admits a p-logarithmic Sobolev inequality (p-lsi for short) if for some constant α p Ent π (f p ) α p E(f, f p ), () where Ent π (g) = E π [g(x) ln g E [g] ] = E [g]d(π(g) π), where π (g) (x) = g(x)π(x) E [g] and D( ) is the Kullback-Leibler divergence. We note that E(f, f p ) 0 for p > and E(f, f p ) < 0 for p < and this implies corresponding signs for constants α p. As p we have E(f, f p ) 0 and so we need to renormalize by (p ) in this limit. Consequently, we define -LSI as Ent π (f) E(f, ln f), α

3 which is required to hold for all f > 0 on X. We will mostly deal in this paper with a special case of a hypercube. Namely, we set X = {0, }, L x,y = {x = y} + /, π = Bern(/) and T t f(x) = f(x) + e t + f( x) e t. (3) For this case the best LSI constants are α p = (p ) p, see [, Theorem..8], and the Dirichlet form takes particularly simple form: E n (f, g) = ( f, g), f(x) = (f(y) f(x)) (4) y:y x E n (f, f) = (f(x) f(y)) n, (5) 4 x y where x y denotes any x, y {0, } n differing in precisely one coordinate. We also quote one inequality known as Mrs. Gerber s lemma [3], or MGL: n Ent π n(tt n ( f) ln m t, E [f] n ) Ent π n(f), (6) E [f] where m(t, x) = h(h (ln x) e t ), where h(x) = x ln x ( x) ln( x) is the binary entropy function, h : [0, ln ] [0, /] is its functional inverse and a b = ( a)b + ( b)a is the binary convolution. A less cryptic restatement of MGL is the following: Ent π n(tt n f) Ent π n(tt n f iid ), (7) where f iid (x) = n k= f (x k ) with f ( ) selected so that a) in (7) the equality holds for t = 0; b) E [f iid ] = E [f]. In other words, MGL states that among all functions f on the hypercube, Ent decreases slowest for product functions... Motivation and Organization We motivate our investigation by the following two questions: Log-Sobolev inequality implies an estimate of the form Ent(T n t f) e Ct Ent(f). However, for the hypercube a stronger estimate is given by the MGL (6). Can MGL be derived from some strengthening of LSI? Note that by a method of comparison of Dirichlet forms, results derived from log-sobolev inequalities can then be extended to semigroups other than Tt n. As an illustration, note that [4, Example 3.3] estimates speed of convergence of the Metropolis chain on {0,... n} by comparing to Tt n. Our methods allow to show better estimates, similar to (6). 3

4 Hypercontractivity inequality for the hypercube (known as Bonami or Bonami-Beckner theorem) says T n t f p f p0, p(t) = + (p 0 )e t, p 0. (8) This is well known to be tight in the sense that for any q > p(t) we can find f s.t. T t f q > f p0. However, such f will be very close to identity (for this particular semigroup). Is it possible to improve the range of (p, q) in (8) provided f is far from identity (say in the sense of supp f n )? For example, it is clear that Tt n = ( + e t ) n. If f has small support, we have f p0 e nρ0 f, where ρ 0 = ( p 0 ) n ln n supp f and thus T t f f p0, t ln (9) e ρ0 which is a significant improvement of (8) for large times t. Both of these questions will be answered positively. The structure of the paper is the following. In Section we define the main concept of this paper the non-linear LSIs and prove some of its consequences, such as refined hypercontractivity and general MGL. In Section 3 we switch from general theory to a particular case of the hypercube. We establish explicit forms of new LSIs and new hypercontractive estimates for functions of small support. In Section 4 we derive apply the latter to establish a sharp version of the uncertainty principle on the hypercube (i.e. tradeoff between the sizes of supports of the function and its Fourier image). Finally, Section 5 applies the uncertainty principle to showing an intriguing property of linear maps F k F n.. Non-linear log-sobolev inequalities Let us introduce a family of non-linear log-sobolev inequalities. Definition. For p and a concave, non-negative function Φ p : [0, ) [0, ) with Φ p (0) = 0, let us define a (p, Φ p )-LSI as Ent(f p ) E [f p ] ( E(f, f p ) ) Φ p E [f p, (0) ] where for p = we understand E(f, f p ) = E(f, ln f). For p < and a concave, non-negative function Φ p : (, 0] [0, ) we define (p, Φ p )-LSI as (0). When convenient, we will restate (p, Φ p )-LSI in the form ± E(f, f p ) E [f p ] ( Ent(f p ) ) b p E [f p, () ] where b p : [0, ) [0, ] is convex and increasing with b p (0) = 0 (i.e. b p = ±Φ p with b p = if Φ p is constant for x x 0 ). The ± is taken to be + for p and for p <. 4

5 Remark. For convenience we define Φ p and b p on [0, ) even though the arguments in (0) and () may be universally bounded by constants <. Note also that concave non-negative function on [0, ) must be increasing on [0, a) and then constant on [a, ) (either interval could be empty). It is clear from concavity of Φ p that the linear-lsis () are obtained by taking α p = d dx x=0 Φ p. We briefly review history of such inequalities: For a Lebesgue measure on R n and E(f, g) = ( f, g) the p = inequality takes the form: f (x) ln f dx n ( ) R ln f dx, f =. nπe n R n It appeared in information theory [5, (.3)] and [6] as a solution to the problem of minimizing Fisher information subject to differential entropy constraint (the minimizer is Gaussian density). More generally, the p = inequalities were introduced into operator theory by Davies and Simon [7] under the name of entropy-energy inequalities; see [8] for a survey. A p = inequality for the hypercube was proved in [9] for the purpose of showing that the Faber-Krahn problem on the hypercube is asymptotically solved by a Hamming ball. Same reference mentioned [9, paragraph after ()] a tightening of hypercontractivity (8) for p 0 = and functions of large entropy, although no proof was published at the time. Here we prove general results about non-linear LSIs. Theorem. (Tensorization) Suppose that (p, Φ p )-LSI holds for semigroup (X, π, T t, E). Then for all n the (p, nφ p ( n ))-LSI holds for (X n, π n, Tt n, E n ). In other words, for all f : X n R + we have Ent π n(f p ( ) E n (f, f p ) ) n E π n[f p Φ p ] n E π n[f p, () ] where π n = n k= π a product measure on X n and E n is the Dirichlet form associated to the product semigroup (). Theorem. (General MGL) Suppose a semigroup T t admits a (, Φ )-LSI. Let b = Φ be a convex, strictly increasing inverse of Φ and assume that the differential equation d dt ρ(t) = b ( ρ(t)) has a C -solution ρ(t) on [0, t 0 ) with ρ(0) > 0. Then for any f : X n R + with ρ(0) we have n Ent(f) E [f] Ent(T n t f) n ρ(t)e [f] 0 t < t 0. 5

6 Theorem 3. (Hypercontractivity) Fix a non-constant function f : X n R + and p 0 (, ). Then there is a finite t 0 and a unique function p(t) on [0, t 0 ) satisfying Tt n f p(t) = f p0. This function is C -smooth, strictly increasing and surjective onto [p 0, ) with p(0) = p 0. Furthermore, if a semigroup T t admits a (p, Φ p )-LSI for each p p 0, then ( ) d p(t)(p(t) ) p(t)ρ0 p(t) b p(t), ρ 0 = dt ρ 0 p(t) n ln f p 0. (3) f Proof of Theorem. Let us consider the case n = only. For a function f(x, x ) denote by Ent πi (f p ) the entropy evaluated only along x i, i =,. Then, from standard chain-rule and convexity of Ent we have Ent π π (f p ) = E X [Ent π (f p )] + Ent π (E X [f p ]) (4) E X [Ent π (f p )] + E X [Ent π (f p )] (5) Now, we apply Φ p -LSI to each term (not forgetting appropriate normalization). For example, for the first term we get [ ( E X [Ent π (f p Eπ (f, f p ) ] ) )] E X Φ p E X [f p E X [f p ] (6) ] Φ p ( E X [E π (f, f p )] E X,X [f p ] where in the second step we used Jensen s and the fact that (x, y) Φ( x y )y ) E X,X [f p ], (7) is jointly concave for any concave function Φ. Now plugging (7) (and its analog for the second term) into (5) and after applying Jensen s again we get Ent π π (f p ( ) E X [E π (f, f p )] + E X [E π (f, f p ) )] E X,X [f p Φ p ] E X,X [f p, ] which is precisely (). The n > is treated similarly. Proof of Theorem. Since the statement is scale-invariant, we assume E [f] =. Define ρ(t) = n n Ent(Tt f). Consider the identity d n Ent(Tt f) = E(Tt n f, ln Tt n f). dt From tensorizing the (, Φ )-LSI we get and hence n E(Tt f, ln Tt n f) b (ρ(t)), n ρ (t) b (ρ(t)). 6

7 Let us introduce α(t) = ln ρ(t) ln ρ(t), then we have for α(t) the following ( ) α (t) e α(t) Ψ( ρ(t)e α(t) ) Ψ( ρ(t)), (8) where Ψ(x) = b (x)/x is a non-decreasing function of x 0. We know α(0) 0. Suppose that for some t 0 > 0 we have α(t 0 ) > 0. Let t = sup{0 t < t 0 : α(t) = 0}. From continuity of α we have t < t 0, α(t ) = 0 and α(t) > 0 for all t (t, t 0 ]. From mean value theorem, we have for some t (t, t 0 ) that α (t ) > 0. But then from monotonicity of Ψ, we have Ψ( ρ(t )e α(t) ) Ψ( ρ(t )) 0, contradicting (8). Hence α(t 0 ) 0 for all t 0 > 0. Proof of Theorem 3. Since all the statements are scale-invariant, we assume E [f] =. To avoid clutter, we will write T t instead of Tt n. We define on R + the following function φ(t, ξ) = ln T t f ξ. It is clear that φ is monotonically decreasing in ξ. Steepness of φ in ξ encodes information about non-uniformity of T t f. As time progresses, ξ φ(t, ξ) converges to an all-zero function. MGL, LSI and hypercontractivity are estimates on the speed of this relaxation. We summarize the information we have about φ(t, ξ) assuming f is nonconstant: A consequence of Hölder s inequality, cf. [0, Theorems 96-97], implies ξ ln g ξ is strictly convex, unless g = c S (a scaled indicator), in which case the function of is linear in ξ. Thus φ(t, ξ) is convex in ξ. We have φ(0, ξ ) φ(0, ξ ) + (ξ ξ ) ln π n [supp f] ξ < ξ with equality iff f = c S (scaled indicator). Note that T t f = 0 has only f = 0 as solution (indeed, det e tl = e tr L 0 since X is finite). So φ(t, ξ) is finite and infinitely differentiable in (t, ξ). The function t φ(t, ξ) is strictly decreasing from φ(0, ξ) to 0 for any ξ < and strictly increasing from φ(0, ξ) to 0 for ξ >. Indeed, T t f p = f p implies f is constant. Furthermore, T t f E [f] = since T tf E [f] ξ as t. Consequently, for each ξ 0 the fiber {t : φ(t, ξ 0 ) = c} (9) 7

8 consists of at most one point. Define t 0 as the unique solution of φ(t 0, 0) = nρ 0. Solution exists from continuity of φ and the fact that φ(0, 0) = ln f > ρ 0 > φ(+, 0) = 0. We have the standard identities: ξ ) φ ξ = Ent(f (0) E [f ξ ] φ 0, ξ = t = E(Ttf,(Ttf) ξ ) (), ξ E [(T tf) ξ ] Since f is non-constant, so is T t f for all t 0 (for otherwise f E [f] is in the kernel of T t ). Therefore, φ ξ < 0 for all (ξ, t). Thus, for any t [0, t 0] there is at most one solution ξ of φ(t, ξ(t)) = φ(0, p 0 ) = nρ 0 () It is clear that ξ(t) is non-increasing. Since fibers (9) are singletons, we also conclude that ξ(t) is strictly decreasing. From implicit function theorem and φ ξ 0, we infer that solution ξ(t) of () is C -smooth. As we mentioned ξ φ(t, ξ) is convex and strictly decreasing. Furthermore, it is strictly convex for t > 0. From this convexity and (0) we infer the following important consequences: Ent(f r ) E [f r ] r Ent(f r ) E [f r ] ln f r ln f r increasing in r (0, ); strictly unless f = c S (3) r > (4) We now set p(t) = ξ(t), where ξ(t) was found from solving (). From observation () we already inferred that t p(t) is well-defined, strictly increasing and C -smooth. The fact that p(t) is surjective follows from ξ(t) 0 as t t 0. It remains to show (3). This follows from differentiating (): where we defined ξ (t) = E(t) E(t), E(t) = E(T t f, (T t f) p(t) ) n E [(T t f) p(t) ] (5) E(t) = Ent((T t f) p(t) ). (6) n E [(T t f) p(t) ] 8

9 From (p, Φ p )-LSI we get then Finally, from (4) we get ξ (t) b p(t)(e(t)) E(t). (7) E(t) φ(t, ξ(t)) n ξ(t) = ρ 0 ξ(t) (8) From convexity of b p ( ), the function bp(e) E further upper-bound ξ (t) as is increasing in E and so we can Noticing that ξ (t) = p (t) p (t) ξ (t) b p(t)( p(t)ρ0 p(t) ). p(t)ρ 0 p(t) we get (3). 3. New LSIs and hypercontractivity for the hypercube An elegant idea of Stroock and Varopoulos allows to compare Dirichlet forms E(f, f p ) with E(f p, f p ) immediately leading to the conclusion that for any reversible semigroup (i.e. Tt = T t in L (π)) we have b p (x) 4 p p b (x) x 0 p (, ) \ {} (9) b p (x) p p b (x) x 0 p <. (30) (see, e.g., [8] for p = and [] for p = ). Thus, we can get non-trivial nonlinear p-lsis by only establishing p =, cases (of which p = was already done in [9]). However, we can also find the sharpest non-linear LSIs for all p explicitly. Theorem 4 (-LSI for the hypercube). For all f : {0, } n (0, ) with E [f] = we have ( ) n E(f, ln f) b n Ent(f), (3) where Dirichlet form is given by (4), all expectations and Ent are with respect to uniform probability measure on {0, } n and b : [0, ln ) [0, ) is a convex increasing function given by b (ln h(y)) = ( y) ln y y, y (0, /]. (3) 9

10 Proof. By Theorem, we only need to work out the case n =. Then, the space of all f can be parameterized by f(0) = f() = y with y (0, /]. With such a choice we have Ent(f) = ln h(y) and E(f, ln f) = ( y y) ln y. Thus, we only need to verify that b is increasing and convex. Monotonicity follows from the fact that both factors in the RHS of (3) are decreasing in y. For convexity, we have b (ln h(y)) = b (ln h(y)) = ln y y y y( y) ln y y d y dy y( y) ln y y (33) (34) Since ln y y < 0, it suffices to prove d dy y y( y) ln y y 0. derivative and rearranging, we need to show ln y y y z = ln y y this is the same as Computing the y+y. Substituting ln z z z + (35) for z. Both sides of (35) vanish at z =. Comparing the derivatives, we need to verify /z 4z, which is the same as ( z + ) (z +) 4z and hence is true. Corollary 5. Classical MGL (6) holds. Proof. We only need to invoke Theorem ( with ) b taken from (3). Indeed, notice that setting ρ(t) = ln m to equal the RHS of (6) solves ρ(t) = b ( ρ(t)). t, Ent(f) ne [f] Next, we proceed to LSI s with p. Theorem 6 (p-lsi for the hypercube). Fix p (, ) \ {, 0}. f : {0, } n R + with E [f p ] = (and f > 0 if p < ) we have ( ) n sgn(p )E(f, f p ) b p n Ent(f p ), For all where Dirichlet form is given by (4), all expectations and Ent are with respect to uniform probability measure on {0, } n and b p : [0, ln ] [0, ) is a convex increasing function given by b p (ln h(y)) = with 0 < y. sgn(p ) ) ( y p ( y) p y p ( y) p, (36) 0

11 Proof. By Theorem, we only need to work out the case n =. Then, the space of all f can be parameterized by f(0) = (y) p, f() = ( y) p with y [0, /]. Thus we only need verify monotonicity and convexity. First, consider the case p >. Let q = p p. Taking the first derivative, we get b p(ln h(y)) = ln y y ( p ( ( y y ( ( y + q ) /q ( y y y ) /q ) ) /p ( y y ) /p ) ) (37) From here, monotonicity of b p follows from the fact that the RHS is negative ( y y > y y ). We proceed to showing convexity. Let z = y y. Then 0 < z and, taking another derivative, we have p(ln h(y)) = ( y) b ln z [ ( d dz ln z p ( ( ) ) /q z /q z ( ( ) ) /p )] z /p (38) + q z Since ln z < 0 for z <, it would suffice ( to argue that the derivative w.r.t. z ( on RHS is nonnegative. Let r(z) = ) ( /q p z z /q) ( + ) /p q z z /p). We need to show z ln z ( r ) r. Making another substitution of variables, let w = ln z, that is < w 0. Let t(w) = r(z) = r (e w ). Substituting and simplifying, we need to show wt (w) t(w). ( We have t(w) = p e w/q e w/q) ( + q e w/p e w/p). Hence t (w) = ( e w/q + e w/q + e w/p + e w/p) and pq t (w) = ( q pq e w/q + q ew/q p e w/p + p ) ew/p In particular, t is a decreasing convex function on (, 0] which vanishes at 0, and wt (w) t(w) is satisfied. Next, consider the case 0 < p <. We repeat the computation above, multiplying throughout by = sgn(p ). Since in this case q < 0, the sign change cancels out, and the convexity argument, with minor changes as needed, goes through. For monotonicity, observe that again the signs of both terms in the RHS of (37) are negative (the front sign is canceled by sgn(p )). Finally, for the case p < 0, observe that we can set g = f p and apply the p already proven inequality to the pair (g, p ) since p p (0, ).

12 Our chief goal is to derive hypercontractivity inequality tighter than Bonami s (8) for functions with small support. We will replace constraint on the support supp f nr with an analytical proxy: f p0 e nρ0 f, ρ 0 = ( p 0 )( R) ln, as discussed in (9). We get the following result: Theorem 7. Fix < p 0 < and 0 ρ 0 ( p 0 ) ln. Then differential equation ( ) u (t) = C ρ 0 ( + e u(t) ), C(ln h(y)) = 4 y( y) ln h(y) has unique solution on [0, ) with u(0) = ln(p 0 ). f : {0, } n R + with f p0 e nρ0 f we have (39) Furthermore, for any T n t f p(t) f p0, p(t) = + e u(t). (40) Remark. Ref. [9] showed that C(x) : [0, ln ] [, / ln ] is a smooth, convex and strictly increasing bijection. Consequently, the p(t) in (40) is smooth and satisfies p(t) > + (p 0 )e t t > 0 thereby strictly improving Bonami s hypercontractivity inequality (8). Furthermore, it satisfies p(t) = p 0 + p (0)t + p (0)t + o(t ), t 0, (4) p (0) = (p 0 )C(x 0 ) (4) ( p (0) = (p 0 ) C(x 0 ) C (x 0 )C(x 0 ) x ) 0 (43) p 0 x 0 = ρ 0p 0 p 0. (44) Remark 3. Our estimate is locally optimal at t = 0 in the following sense: for every q(t) such that q(0) = p 0 and q (0) < p (0) there exists a function f with f p0 e nρ0 f and T t f q(t) > f p0 for a sequence of t 0. This follows from the fact that had a counter-example q(t) existed, it would imply that the second half of the proof of Theorem 9 could be improved to contradict the first half. Proof. First, notice that C(x) = 4b(x) x, where b was defined in Theorem 6. Let p (t) be the function defined by T n t f p(t) = f p0. Theorem 3 showed this function to be smooth and growing faster than (3). From (3) and using (9) to lower-bound b p ( ) via b ( ) we get that ( ) p p (t)ρ 0 (t) (p (t) )C, p (t)

13 or introducing u (t) = ln(p (t) ) that ( ) u (t) C ρ 0 ( + e u(t) ). The case of ρ 0 = p 0 corresponds to f supported on a single point and can be dealt with separately. So we assume ρ 0 < p 0, in which case the map u C ( ρ 0 ( + e u ) ) is smooth on some interval (ln(p 0 ) ɛ, ). Consequently, (39) possesses unique solution with u(0) = ln(p 0 ) and Chaplygin-type theorem, e.g. [, Theorem 4.], implies u (t) u(t). For p 0 =, we also prove an alternative estimate p(t) via a method tailored to the hypercube. Theorem 8. In the setting of Theorem 7 assume p 0 =. Then (40) holds with p(t) given as p(t) = + e t ρ(s) = ρ 0p 0 p 0 ln 0 C( ρ(s) 0)ds (45) ( ) + e s. (46) Remark 4. Using convexity of C we get C(x 0) C(x 0 ) + (x x 0 )C (x 0 ). Similarly, ln +e t t. Therefore, altogether we get an explicit estimate: p(t) + e C(x0)t C (x 0 ) t, (47) where x 0 = ρ(0) is from (44). The t term here is, however, worse than that of (4). Proof. We return to (7). Recalling that ξ(t) = p(t) and lower-bounding b p by b via (9) we get d ln(p(t) ) C(E(t)), dt (with E(t) from (6)), which implies via Gronwall inequality p(t) + e s 0 C(E(s))ds. Thus, it only suffices to prove that E(s) ρ(s) 0, or equivalently (since E 0 by definition) the estimate ( ) E(t) ( R) ln ln + e t (48) 3

14 Next, we obtain a lower bound on Tt n f. To that end introduce a function t on {0, } n with the property Tt n f = Λ n t f. Note that Λ n t (x) = Λ n ( e t ) x ( + e t ) n x, where x denotes the Hamming weight of x. Clearly Λ n t 0. Furthermore, without loss of generality we may assume f 0. Then, we have Tt n f = Tt n f, Tt n f = Λ n t f, Λ n t f = Λ n t Λ n t, f f = Λ n t, f f ( + e t n Λ n t (0) (f f)(0) = ) n f To prove (48), observe that by (3) and (4) and by the preceding calculation, E(t) n n Tt f ln Tt n = f n n Tt f ln f ( n ln f f ln ) ( ( R) ln ln + e t + e t ) = 4. Uncertainty principle on the hypercube 4.. Background Uncertainty principle asserts that a function and its Fourier transform cannot be simultaneously narrowly concentrated. There are several approaches to quantifying this statement, and here we adopt the Hilbert space point of view, cf. [3, Chapter 3]. Namely, for a pair of subspaces V, V of a Hilbert space with inner product (, ) and f = (f, f) we define cos (V, V ) = (f, f ) sup. f V,f V f f For the uncertainty principle, we will select sets S and Σ and define subspaces V S = {f : supp f S} (49) ˆV Σ = {f : supp ˆf Σ}, (50) where ˆf denotes the corresponding Fourier transform (we will define it precisely). Uncertainty principle corresponds to bounding cos (V S, ˆV Σ ) away from, thus establishing to what extent functions can simultaneously concentrate on (S, Σ). There is a number of equivalent ways to think of cos (V, V ). Letting P i be an orthogonal projection on V i and Pi projection on Vi, it can be shown [3, 4

15 Chapter 3]: cos (V, V ) θ f V : P f θ f (5) Furthermore, there is also a simple criterion: f V : P f θ f (5) λ max (P P P ) θ (53) P P θ (54) f : f ( P θ f + P f ) (55) cos (V, V ) < V V = {0} and (V + V ) closed, where for finite-dimensional V i s the closedness condition is vacuous (but not in general). Finally, as shown in Fuchs [4] and Landau-Pollak [5], knowledge of cos (V, V ) is sufficient for completely characterizing the two-dimensional region {( P f, P f )} Before proceeding to our own results, we briefly review the history of results for R n. First, Slepian and Landau [6] computed cos (V S, ˆV Σ ) for S, Σ being two balls (in fact they computed λ max (P P P ) and named eigenfunctions of the latter prolate spheroidal functions). Next, Benedicks in 974 (published as [7]) showed that vol(s), vol(σ) < = V S ˆV Σ = {0}. Later, Amrein and Berthier [8] strengthened this to vol(s), vol(σ) < = cos (V S, V Σ ) <. Finally, for n = Nazarov [9] showed vol(s), vol(σ) < = cos (V S, V Σ ) < ce c vol(s) vol(σ). Lately, there were a number of extensions and improvements of Nazarov s theorem for n >, e.g. [0]. 4.. Sharp uncertainty principle on F n Define the characters, indexed by v F n, χ v (x) = χ j (x) = ( ) <v,x>, j:v j= where < v, x >= n j= v jx j is a non-degenerate bilinear form on F n. The Fourier transform of f : F n C is ˆf(ω) = χ ω (x)f(x) = n (f, χ ω ), ω F n. x F n 5

16 We denote by x the Hamming weight of x F n and by B r = {x : x r} Hamming ball. Theorem 9. For any ρ, ρ [0, /] satisfying ( ρ ) + ( ρ ) >. (56) there exist an ɛ > 0 and n 0 such that for any n n 0, any S F n with S e nh(ρ) and Σ = B ρn we have where V S, ˆV Σ are defined in (49)-(50). Conversely, for ρ, ρ [0, /] satisfying cos (V S, ˆV Σ ) e nɛ, (57) ( ρ ) + ( ρ ) <, (58) there exist ɛ > 0 and n 0 such that for all n n 0 we have cos (V S, ˆV Σ ) e nɛ, S = B ρn, Σ = B ρn. (59) Proof. For the case (58), fix α R, α ± and consider the following Fourier pair: f(x) = α x (60) ( ) ω α ˆf(ω) = c, + α c = ( + α) n. (6) Then, it is easy to see that the L -energy of f is concentrated around x +α n. Thus, whenever radius ρ > +α, we have for some ɛ > 0 f(x) e ɛn f(x). x: x >ρ n x F n Similarly, whenever ρ > +β, where β = α +α, we have ˆf(ω) e ɛn ˆf(ω). ω: ω >ρ n ω F n Whenever, (58) holds, it is not hard to see that there exists a choice of α (0, ) satisfying both ρ > +α and ρ > +β. Thus, taking corresponding f and using (55) we get (59). Next, we assume (56). We define the Fourier projection operators Π a as Π a f(ω) = ˆf(ω){ ω = a}, a = 0,,..., n. (6) When ρ > (or ρ > ), the result (59) also holds by reducing to ρ =. This is possible since cos (V S, ˆV Σ ) is monotone in S, Σ. 6

17 and set Π r = r a=0 Π a. By (5) we need to show that for any function f with support supp f e nh(ρ) we have Π ρnf e nɛ f, for some ɛ > 0 independent of n and f. Note that T t ω t f(ω) = e ˆf(ω). Thus, comparing eigenvalues we have e ta T t Π a (in the sense of positive-semidefiniteness). Consequently, Π a f = (Π a f, f) e at (T t f, f) e at f q T t f p, (63) where p and q are Hölder conjugates. Since supp f e nh(ρ) we have from Theorem 7 with p 0 = and ρ 0 = ln h(ρ) : T t f p(t) f, p(t) = + p (0)t + o(t), where the value of p (0) is given in (4) Taking p = p(t) > in (63) we need upper-bound f q, which we again do by invoking the bound on support f q f n( R)( q ), q. (64) Overall, we have shown for all a and t that Π a f e at e n(ln h(ρ))( p(t) ) f. Analyzing this inequality for t close to 0 we conclude that whenever ρ < p (0) 4 (ln h(ρ )) (65) we necessarily have for some ɛ > 0 (depending on the gap in the inequality above and on the local bound for p (t) at 0) that for all a ρ n Π a f e nɛ f. Using expression for p (0) in (4), we see that (65) is equivalent to which is in turn equivalent to (56). ρ < ρ ( ρ ), (66) For completeness, we also provide a criterion for when two subspaces have a common element (for the special case of two balls as S, Σ. It demonstrates that there is a discontinuity between the regime cos e O(n) and cos =. For extracting explicit constants, one may invoke (47) instead. 7

18 Proposition 0. Let S = B r and Σ = B r in F n. Then V S ˆV Σ {0} cos (V S, ˆV Σ ) = r + r n. Proof. If r + r n, then take f(x) = {x r+ = = x n = 0}. Its Fourier transform is supported on {ω : ω = = ω r = 0}. Thus f V S ˆV Σ. On the other hand, suppose there is f V S ˆV Σ. By averaging over permutations of coordinates (both subspaces are invariant to such), we conclude that f(x) = f ( x ). As such, it can be expanded in terms of Krawtchouk polynomials: f ( x ) = n a k K k ( x ), k=0 where each K k ( ) is a degree k univariate polynomial. Constraint supp ˆf B r is equivalent to requiring a k = 0 for k > r. Thus, we conclude that f on integers inside [0, n] coincides with a degree r polynomial, and hence has r zeros. Thus, r n r as claimed Discussion In this section we collect some remarks regarding Theorem 9. First, it is instructive to provide an equivalent statement as follows: Theorem (Restatement ( of the uncertainty principle). For any δ < / and 0 < E < R LP (δ) = h ) δ( δ) there is ɛ > 0 with the following property. Let f(x,..., x n ) be polynomial of total degree at most δn. Then, for any S {±} n of size S e ne we have f(x) e nɛ f(x). x S x {±} n This shows that any low-degree polynomial (restricted to the hypercube) smears its energy so evenly that one needs to sum e nr LP (δ) top values in order to obtain a sizable fraction of its overall energy. It is interesting to compare this with [] showing that any f that is a) a degree δn polynomial and b) f 0 satisfies max f(x) x {±} e n(ln h( δ ))+o(n) f(x). (67) n x {±} n We conjecture that (67) holds for all f : {±} n R of degree δn. This could be called an L -version of the uncertainty principle. If true, it would imply that the sum of any e nrham(δ), R Ham (δ) = ln h( δ ) values of f(x) is negligible compared to the sum over all of {±} n. Finally, we discuss to what extent one can relax condition that Σ = B ρn in Theorem 9. First, notice that clearly the same conclusion holds for Σ which is an image of a ball B ρn under a linear isomorphism F n F n. This provides a wealth of examples of Σ that are less contiguous than B ρn. 8

19 At the same time, we cannot extend Theorem 9 to Σ being an arbitrary subset of the same cardinality as B ρn. Indeed, a simple computation shows that when S and Σ are linear subspaces of F n we have cos (V S, ˆV Σ S Σ ) = S, (68) where S = {x :< x, v >= 0, v S} is the dual of S. Thus, if we take S to be a linear subspace of dimension αn, 0 < α <, and Σ = S (of dimension ( α)n) and solve for ρ and ρ in h(ρ ) = α ln, h(ρ ) = ( α) ln we conclude that these ρ and ρ always satisfy ( ρ ) + ( ρ ) >, while from (68) we have cos (V S, ˆV Σ ) =. Consequently, we leave open the question of determining the more general uncertainty principle, i.e. characterizing the best pairs (E, E ) for which one can prove implication S e ne, Σ e ne = cos (V S, ˆV Σ ) ɛ. A partial result easily follows from the Hausdorff-Young inequality: Proposition. For any E, E (0, ln ) satisfying E +E < ln there exist ɛ > 0 and n 0 such that for all n n 0, all S, Σ F n with S = e ne, Σ = e ne we have cos (V S, ˆV Σ ) ɛ. Conversely, for any positive integers k, k n such that k +k n there exist S = k and Σ = k such that cos (V S, ˆV Σ ) =. Proof. Second part follows from (68). For the first part, let θ = cos (V S, ˆV Σ ) and E + E = ln δ for δ > 0. We will show that θ δ n ln ln max(e, E ). (69) Without loss of generality, suppose E E. Recall that a simple consequence of the Hausdorff-Young inequality is the Hirschmann (or entropic) uncertainty principle [, Exercise 4..0]: For any f : F n R we have Ent(f ) E [f ] + Ent( ˆf ) E [ ˆf ] 9 ln F n = n ln. (70)

20 Thus, taking f supported on S we estimate (from Jensen s inequality) Ent(f ) E [f ] n ln ln S = n(ln E ). (7) Suppose that E [ ˆf Σ] E [ ˆf ] F n with = θ, and introduce random variable U taking values in P[U = u] = ˆf (u) ω ˆf (ω). Then, we have Ent( ˆf ) E [ ˆf = n ln H(U), with H( ) denoting the Shannon entropy. ] Introducing also T = {U Σ} we get by the chain rule n ln Ent( ˆf ) E [ ˆf ] Altogether, from (70), (7) and (74) we get (69) A similar result for Euclidean space = H(U) = H(U, T ) = H(T ) + H(U T ) (7) ln + θ ln Σ + ( θ ) ln Σ c (73) ln + n(θ E + ( θ ) ln ). (74) It is interesting to observe that a result in (R n, Leb) analogous to Theorem 9 follows from the sharp form of Young s inequality due to Beckner [3]. This provokes us to hypothesize that the refined hypercontractivity result on the hypercube (Theorem 7) could play the role of the sharp Young inequality (and/or Babenko-Beckner inequality) in R n. Notation: we define B r = {x R n : x r}, x = (x, x), (x, y) = n k= x ky k, S the Lebesgue measure of S, and for f L L ˆf(ω) = e πi(ω,x) f(x)dx, R n ω R n, with the standard extension by continuity to all of f L. Theorem 3. For any ρ, ρ > 0 satisfying ρ ρ < 4π (75) there exist an ɛ > 0 and n 0 such that for any n n 0, any S R n with S = B ρ n and Σ = B ρ n we have where V S, ˆV Σ are defined in (49)-(50). cos (V S, ˆV Σ ) e nɛ, (76) 0

21 Conversely, for ρ, ρ 0 satisfying ρ ρ > 4π there exist ɛ > 0 and n 0 such that for all n n 0 we have (77) cos (V S, ˆV Σ ) e nɛ, S = B ρ n, Σ = B ρ n. (78) Proof. Since the statement is asymptotic, we will use the standard fact and thus ln B = n ln πe n ln(πn) + O( n ) ln B ρ n = n ln(πeρ ) + O(ln n). (79) To prove the second part, consider the Fourier pair (for any σ > 0): f(x) = e x (πσ ) n σ (80) ˆf(ω) = e π σ ω. (8) Choose σ > 0 so that ρ > σ and ρ > 8σπ (which is possible due to (77)). From concentration of Gaussian measure, it is easy to check that for some ɛ > 0 we have f Bρ n ( e nɛ ) f (8) ˆf Bρ n ( e nɛ ) ˆf (83) and therefore (78) follows from (55). For the first part, recall a sharp form of the Young inequality on R n (from [3]) ( ) n Cp C q f g r f p g q, C s = e C r ( ln s s + s s ln( s )) (84) valid for p, q, r and p + q = + r. Consider the heat semigroup e t f = f φ t, φ t (x) = (4πt) n e x 4t. Let γ > 0 be a constant to be specified later, and for a real t < γ we set γt. Then apply (84) with r = p(t), p = and q = q(t) given by p(t) q(t) = = γt to get, after some calculations, a hypercontractive inequality e t f p(t) e ne(t) f, (85) where E(t) = γt ln γ πe + o(t), t 0.

22 Now, we proceed as in the proof of Theorem 9 with (85) replacing the use of the more precise hypercontractivity for the cube. Namely, we define the ball-multiplier operator Π r f(ω) = ˆf(ω) Br (ω). Now consider a function f supported on S and note the chain Π r f = (Π r f, f) e 4π r t (e t f, f) (86) e 4π r t f q(t) e t f p(t) (87) e 4π r t+ne(t) f f q(t) (88) e 4π r t+ne(t)+γt ln S f, (89) where in (86) we used the fact that êt f(ω) = e 4π ω ˆf(ω), in (87) we used Hölder s inequality with q(t) denoting the conjugate of p(t), (88) is by (85), and (89) is by invoking the bound on the support of S via Hölder s inequality f q f S q. Taking r = ρ n and using (79) to estimate S, we conclude that whenever there is a γ > 0 such that Π r f e nɛ f, (90) 4π ρ + γ ln ρ γ e < 0. Since min γ>0 γ ln(aγ) = ea, we get that (90) holds whenever 4π ρ 4ρ < 0, which is equivalent to (85). The structure of the proof for R n suggests that perhaps it is worthwhile to look for a general inequality on the hypercube that could replace the use of hypercontractivity in the proof of Theorem 9, i.e. play a role similar to that of the sharp Young inequality on R n (of course, the Young inequality itself cannot be sharpened on the hypercube, or on any finite group). 5. An application to coding theory Theorem 4. For any 0 < R < R < there exists δ n 0 such that for any linear map f : F k F n with k n = R there exists an x Fk s.t. where δ LP (h(ρ)) = ρ( ρ). n f(x) δ LP (R ) + δ n (9) ( ) R k x δ LP δ n, (9) R

23 Remark 5. This estimate significantly outperforms previously best known bounds of this kind [4, Theorem ], but only applies to linear maps. We give two different proofs, in two subsections below. Note that the two proofs take slightly different points of view. The first proof deals with linear maps f : F k F n, while the second proof looks rather at images of these maps, linear codes in F n. In particular, in the second proof we assume that the image of f is of dimension k (i.e. f is of full rank). 5.. Method - graph covers Proof. To every linear map f : F k F n we associate the following increasing sequence of numbers: d r (f) = min{ f(x) : x r}, where d is just the minimum distance of f. Following, Friedman and Tillich [5] we also associate to f a Cayley graph Γ with vertices F k and generators taken as columns c i of k n matrix describing f. (We will use freely facts from [5], perhaps in a somewhat different formulation, from now on.) Then { } (Ah, h) n d r = max h : ĥ = 0 on ball B(0, r ), (93) where A is the adjacency matrix of Γ. Note that A is also a convolution operator on F k : n Ah = h ( δ ci ). As in [5], select a covering map F n F k and take B F n to be the Hamming ball of radius nρ, with 0 < ρ < found as h(ρ ) = R. There exists a function g B : F n R, supported on B with the property: i= (A C g B, g B ) λ B g B, where A C is the adjacency matrix of the n-dimensional hypercube, and λ B = n ρ ( ρ ) + o(n). Hence, there exists a function h B : F k R supported on the image of B under the covering map with the property: (Ah B, h B ) λ B h B, Then to get a lower bound on (93) we set h = h B Π <r h B, where Π <r = a<r Π a and Π a is from (6). Note that A and Π commute and so (AΠh, h) n(πh, h), then we have : (Ah, h) = (Ah B, h B ) (AΠ <r h B, h B ) λ B h B n(π <r h B, h B ) 3

24 Thus, whenever n d r n ρ ( ρ ) + o(n) (94) Π <r h B h B o(). Using the uncertainty principle for the k-dimensional cube (Theorem 9) we estimate Π <r h B h B, as long as where ρ is found from h(ρ ) = h(ρ) that (94)-(95) are equivalent to (9)-(9). r k < ρ ( ρ ), (95) R = R R. After simple algebra, we see 5.. Method (analytic) We start an uncertainty-type claim for subspaces of F n. Let C be a k-dimensional linear subspace C of F n. Given a basis v = {v,..., v k } of C, denote the length of representation of a vector x C in terms of V by x v. Lemma 5. Let f be a function supported on a subset A F n. Let 0 r k n be integer parameters such that k log A, and, moreover, writing A = h(ρ) k, ( k r) = h(ρ ) k, we have ( ρ ) + ( ρ ) >. Then, for any k-dimensional subspace C of F n and for any basis v of C holds f (ω) f (ω) ω C ω C, ω v r Here the sign means that the LHS is exponentially smaller than the RHS. Proof. Let F = C f C. Note that F is constant on cosets of C and that F (ω) = { f(ω) if ω C 0 otherwise. Let M be a k n matrix with rows v,..., v k. We define a function g on F k as follows. For x F k, the pre-image {y F n, My = x} is a coset of C, and we set g(x) to be the (fixed) value of F on this coset. Next, we calculate the Fourier transform of g. Let α F k. Let ω = α t M F n. We claim that ĝ(α) = F (ω). To see this, note that for any y such that My = x holds (x, α) = (My, α) = (y, M t α) = (y, ω). Using this we compute ĝ(α) = x F k g(x)( ) (x,α) = x F k y:my=x F (y)( ) (y,ω) = F (ω) 4

25 Next, we apply the uncertainty principle for g on F k. Observe that the cardinality of the support of g is given by the number of cosets of C intersecting A, which is at most A. The constraints on A, k, and r imply α r ĝ (α) α ĝ (α), which is equivalent to the claim of the lemma. We now prove Theorem 4, first restating it for linear codes rather than for linear maps. Theorem 6. Let } 0 < R <. Let C F n be a linear code of rate k = Rn. Let v = {v,..., v k be a basis of C. Then for any 0 R < R there is a vector x C with ( ) n x δ LP (R R ) + δ n and k x v δ LP δ n R Proof. Let r = h (R ) n. Let B be the Hamming ball of radius r around zero in F n. As in [5], let g B be a function supported on B, with the property: (A C g B, g B ) λ B g B, where A C is the adjacency matrix of the n-dimensional hypercube, and λ B = n r n ( r n ) + o(n). ( Let d = n λ B+. Note that d = ) h (R ) ( h (R )) n + o(n) = δ LP (R ) n + o(n). Note that B = R n C = k. We introduce two additional parameters with a view towards using Lemma 5. Let ρ be such that B = h(ρ) k, and let ρ satisfy ( ρ ) + ( ρ ) =. Computing explicitly, ( ) ( ) R ρ = h R and ρ = δ LP R R We proceed with the following computation, as in [6]. Let F = C g B C. Compute (A C F, F ) in two ways. On one hand, since A C commutes with convolutions, we have (A C F, F ) λ B (F, F ). On the other hand, (A C F, F ) = ((n x ) F (x), F (x)) = x (n x ) F (x). Note that n d = λ B. Hence, taking into account both expressions gives, after a simple calculation: n ĝ B (x) = n F (x) = ĝ B (x) x C x C, x d x d F (x) x ( ) We now apply Lemma 5 for g B. Let r k < ρ R = δ LP R. By the lemma, ignoring negligible factors, n ĝ B (x) > 0 x C, x d, x v>r ĝ B (x) x C 5

26 This means that there exists a vector x C such that x d and x v > r, proving the claim of the theorem. [] D. Bakry, Functional inequalities for Markov semigroups, in: Probability measures on groups, Tata Institute of Fundamental Research, Mubai, 006, pp [] L. Saloff-Coste, Lectures on finite Markov chains, in: Lectures on probability theory and statistics, Springer, 997, pp [3] A. D. Wyner, J. Ziv, A theorem on the entropy of certain binary sequences and applications I, IEEE Trans. Inf. Theory 9 (6) (973) [4] P. Diaconis, L. Saloff-Coste, Logarithmic Sobolev inequalities for finite Markov chains, Ann. Appl. Probab. 6 (3) (996) [5] A. Stam, Some inequalities satisfied by the quantities of information of Fisher and Shannon, Inf. Contr. () (959) 0. [6] M. Costa, T. Cover, On the similarity of the entropy power inequality and the Brunn-Minkowski inequality (corresp.), IEEE Trans. Inf. Theory 30 (6) (984) [7] E. B. Davies, B. Simon, Ultracontractivity and the heat kernel for Schrödinger operators and Dirichlet Laplacians, J. Func. Anal. 59 () (984) [8] D. Bakry, L hypercontractivité et son utilisation en théorie des semigroupes, in: Lectures on probability theory, Springer, 994, pp. 4. [9] A. Samorodnitsky, A modified logarithmic sobolev inequality for the hamming cube and some applications, arxiv preprint arxiv: [0] G. H. Hardy, J. E. Littlewood, G. Polya, Inequalities, Cambridge University Press, 988. [] E. Mossel, K. Oleszkiewicz, A. Sen, On reverse hypercontractivity, Geometric and Functional Analysis 3 (3) (03) [] P. Hartman, Ordinary differential equation, John Wiley & Sons, New York, USA, 964. [3] V. Havin, B. Jöricke, The uncertainty principle in harmonic analysis, Springer, 994. [4] W. H. J. Fuchs, On the magnitude of fourier transforms, in: Proc. Int. Math. Cong., Amsterdam, 954, pp [5] H. J. Landau, H. O. Pollak, Prolate spheroidal wave functions, Fourier analysis and uncertainty II, Bell Syst. Tech. J. 40 () (96)

27 [6] D. Slepian, H. O. Pollak, Prolate spheroidal wave functions, Fourier analysis and uncertainty I, Bell Syst. Tech. J. 40 () (96) [7] M. Benedicks, On Fourier transforms of functions supported on sets of finite Lebesgue measure, J. Math. Anal. Appl. 06 () (985) [8] W. Amrein, A. Berthier, On support properties of l p -functions and their Fourier transforms, J. Func. Anal. 4 (3) (977) [9] F. L. Nazarov, Local estimates for exponential polynomials and their applications to inequalities of the uncertainty principle type, Algebra i Analiz (in Russian) 5 (4) (993) [0] P. Jaming, Nazarov s uncertainty principles in higher dimension, J. Approximation Th. 49 () (007) [] A. Samorodnitsky, Extremal properties of solutions for Delsarte s linear program, preprint. URL [] T. Tao, V. H. Vu, Additive combinatorics, Vol. 05, Cambridge University Press, 006. [3] W. Beckner, Inequalities in Fourier analysis, Ann. Math. 0 () (975) [4] Y. Polyanskiy, On metric properties of maps between hamming spaces and related graph homomorphisms, Arxiv preprint arxiv: [5] J. Friedman, J.-P. Tillich, Generalized alon boppana theorems and errorcorrecting codes, SIAM Journal on Discrete Mathematics 9 (3) (005) [6] M. Navon, A. Samorodnitsky, Linear programming bounds for codes via a covering argument, Discrete & Computational Geometry 4 () (009)