Hamming Cube and Other Stuff

Transcription

1 Hamming Cube and Other Stuff Sabrina Sixta Tuesday, May 27, 2014 Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12

2 Table of Contents Outline 1 Definition of a Hamming cube 2 Properties of the Hamming cube 3 Functions over the Hamming cube 4 l -norm 5 A counting measure on the Hamming cube 6 Expectation 7 Conditional expectation 8 Martingales 9 Azuma s inequality Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12

3 Definition of a Hamming cube The Hamming Cube Definition The Hamming Cube of dimension n denoted by {0, 1} n is a collection of all binary strings of length n. That is for σ {0, 1} n, σ = σ 1 σ 2...σ n where σ i {0, 1} Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12

4 Definition of a Hamming cube The Hamming Cube Definition The Hamming Cube of dimension n denoted by {0, 1} n is a collection of all binary strings of length n. That is for σ {0, 1} n, σ = σ 1 σ 2...σ n where σ i {0, 1} The Hamming distance: For σ, τ {0, 1} n, d(σ, τ) = #{i : σ i τ i } Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12

5 Definition of a Hamming cube The Hamming Cube Definition The Hamming Cube of dimension n denoted by {0, 1} n is a collection of all binary strings of length n. That is for σ {0, 1} n, σ = σ 1 σ 2...σ n where σ i {0, 1} The Hamming distance: For σ, τ {0, 1} n, d(σ, τ) = #{i : σ i τ i } The normalized Hamming distance: For σ, τ {0, 1} n, d(σ, τ) = #{i : σ i τ i } n Since the (normalized) Hamming distance is a metric (it satisfies the separation, symmetry and triangle inequality properties), the pair ({0, 1} n, d) and ({0, 1} n, d) are both metric spaces. Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12

6 Properties of the Hamming cube Properties of the Hamming Cube Diameter: diam({0, 1} n ) = sup σ,τ d(σ, τ) = n Open Ball: B r (σ) = {τ {0, 1} n d(σ, τ) < r} Metrically homogenous: The metric space X is said to be metrically homogeneous if for all x, y X there exists an isometry i : X X such that i(x) = y. The hamming cube, {0, 1} n, is metrically homogeneous. Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12

7 Functions over the Hamming cube Functions over the Hamming cube The weight function: w(σ) = #{i : σ i = 1} The normalized weight function: w(σ) = #{i:σ i =1} n. The function π i : {0, 1} n {0, 1} that sends a string, σ, to its ith bit: π i (σ) = σ i The function that truncates a string: Let k < n, π n k = {0, 1}n {0, 1} k The distance from the set A {0, 1} n function: d A (σ) = inf a A d(x, a) Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12

8 Lipschitz continuity Functions over the Hamming cube A real-valued function f on {0, 1} n is called Lipschitz with Lipschitz constant L if for σ, τin{0, 1} n, f (σ) f (τ) Ld(σ, τ). Since {0, 1} n is a finite set, all functions over {0, 1} n are Lipschitz. The more interesting question is the value of L, the Lipschitz constant. Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12

9 l -norm l -norm Definition: Let f : X R be a bounded function on a non-empty set. The l -norm of f is defined as: f = sup f (x) x X The l -norm has the following properties for f, g : X R where X is a finite set: f f 0 λf = λ f forλ R f + g f + g fg f g e f e f. Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12

10 A counting measure on the Hamming cube A counting measure on the Hamming cube Take # to be a counting measure on {0, 1} n such that for A {0, 1} n, #A =the number of elements in A. The following are some properties: # = 0, #{0, 1} n = 2 n. #A 0 for any A {0, 1} n Let A 1, A 2,... A k,... be a sequence of disjoint subsets of the Hamming cube. Then n n #( A i ) = #(A i ). i=1 If A B then #(A) < #(B). We can also define the normalized counting measure, µ # (A) = #A 2 n. For the normalized counting measure, µ # {0, 1} n = 1. All other properties from the counting measure hold. Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12 i=1

11 Expectation Expectation Let f : {0, 1} n R be an arbitrary function. The integral of f with respect to the normalized counting measure is defined as σ {0,1} E[f ] = f (σ)dµ # (σ) = n f (σ) {0,1} n 2 n. Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12

12 Expectation Expectation Let f : {0, 1} n R be an arbitrary function. The integral of f with respect to the normalized counting measure is defined as σ {0,1} E[f ] = f (σ)dµ # (σ) = n f (σ) {0,1} n 2 n. Example: The expected value of the normalized weight function is E[w(σ)] = 1 2. Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12

13 Expectation Expectation Let f : {0, 1} n R be an arbitrary function. The integral of f with respect to the normalized counting measure is defined as σ {0,1} E[f ] = f (σ)dµ # (σ) = n f (σ) {0,1} n 2 n. Example: The expected value of the normalized weight function is E[w(σ)] = 1 2. Markov s Inequality: Let f : X R +, then µ # {x X f (x) 1} E[f ]. Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12

14 Conditional expectation Conditional expectation A standard partition Ω k ({0, 1} n ) of the Hamming cube consists of all sets A τ (τ {0, 1} k ) where, A τ {σ {0, 1} n πk n (σ) = τ}. Let Ω, Υ be two partitions of X. One says Ω Υ (Ω refines Υ) if for every A Ω there is a B Υ such that A B. Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12

15 Conditional expectation Conditional expectation A standard partition Ω k ({0, 1} n ) of the Hamming cube consists of all sets A τ (τ {0, 1} k ) where, A τ {σ {0, 1} n πk n (σ) = τ}. Let Ω, Υ be two partitions of X. One says Ω Υ (Ω refines Υ) if for every A Ω there is a B Υ such that A B. Conditional expectation: Let f : X R and let Ω be a partition over {0, 1} n. The conditional expectation of f given Ω, denoted E[f Ω] is a function from X to R: x A E[f Ω](a) = f (x) #(A) where a A. Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12

16 Conditional expectation Conditional expectation A standard partition Ω k ({0, 1} n ) of the Hamming cube consists of all sets A τ (τ {0, 1} k ) where, A τ {σ {0, 1} n πk n (σ) = τ}. Let Ω, Υ be two partitions of X. One says Ω Υ (Ω refines Υ) if for every A Ω there is a B Υ such that A B. Conditional expectation: Let f : X R and let Ω be a partition over {0, 1} n. The conditional expectation of f given Ω, denoted E[f Ω] is a function from X to R: x A E[f Ω](a) = f (x) #(A) where a A. Let Ω Υ be two partitions of X and f : X R, then E[f Υ] = E[E[f Ω] Υ]. Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12

17 Martingales Martingales Let (Ω k ({0, 1} n )) n k=0 be a refining sequence of partitions of {0, 1}n. That is Ω n ({0, 1} n ) Ω n 1 ({0, 1} n )... Ω 0 ({0, 1} n ). A martingale formed by a refining sequence of partitions is a collection of functions such that E[f i Ω i 1 ] = f i 1. A martingale difference of the martingales (f 0, f 1,..., f n ) on a set X is defined as d i = f i f i 1. So, E[d i Ω i 1 ] = 0. Application to Hamming cube. Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12

18 Azuma s inequality Azuma s inequality Let f : X R be a finite set on X and let (f i ) n i=0 be a corresponding martingale to a sequence of refining partitions ofx, (Ω i ) n i=0. Then for c > 0, ( c 2 ) µ # {x X f (x) E[f ] c} 2exp 2 n i=1 d i 2. Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, / 12