Sparse Fourier Transform (lecture 1)

Transcription

1 1 / 73 Sparse Fourier Transform (lecture 1) Michael Kapralov 1 1 IBM Watson EPFL St. Petersburg CS Club November 2015

2 2 / 73 Given x C n, compute the Discrete Fourier Transform (DFT) of x: x i = 1 x n j ω ij, j [n] where ω = e 2πi/n is the n-th root of unity.

3 2 / 73 Given x C n, compute the Discrete Fourier Transform (DFT) of x: x i = 1 x n j ω ij, j [n] where ω = e 2πi/n is the n-th root of unity. Assume that n is a power of 2.

4 2 / 73 Given x C n, compute the Discrete Fourier Transform (DFT) of x: x i = 1 x n j ω ij, j [n] where ω = e 2πi/n is the n-th root of unity. Assume that n is a power of 2. compression schemes (JPEG, MPEG) signal processing data analysis imaging (MRI, NMR)

5 DFT has numerous applications: 3 / 73

6 4 / 73 Fast Fourier Transform (FFT) Computes Discrete Fourier Transform (DFT) of a length n signal in O(nlogn) time

7 4 / 73 Fast Fourier Transform (FFT) Computes Discrete Fourier Transform (DFT) of a length n signal in O(nlogn) time Cooley and Tukey, 1964

8 5 / 73 Fast Fourier Transform (FFT) Computes Discrete Fourier Transform (DFT) of a length n signal in O(nlogn) time Cooley and Tukey, 1964 Gauss, 1805

9 6 / 73 Fast Fourier Transform (FFT) Computes Discrete Fourier Transform (DFT) of a length n signal in O(nlogn) time Cooley and Tukey, 1964 Gauss, 1805 Code=FFTW (Fastest Fourier Transform in the West)

10 7 / 73 Sparse FFT Say that x is k-sparse if x has k nonzero entries time frequency

11 8 / 73 Sparse FFT Say that x is k-sparse if x has k nonzero entries Say that x is approximately k-sparse if x is close to k-sparse in some norm (l 2 for this lecture) time frequency

12 9 / 73 Sparse approximations Given x, compute x, then keep top k coefficients only for k N Used in image and video compression schemes (e.g. JPEG, MPEG)

13 10 / 73 Sparse approximations JPEG = Given x, compute x, then keep top k coefficients only for k N Used in image and video compression schemes (e.g. JPEG, MPEG)

14 11 / 73 Computing approximation fast Basic approach: FFT computes x from x in O(nlogn) time compute top k coefficients in O(n) time.

15 11 / 73 Computing approximation fast Basic approach: FFT computes x from x in O(nlogn) time compute top k coefficients in O(n) time. Sparse FFT: directly computes k largest coefficients of x (approximately formal def later) Running time O(k log 2 n) or faster Sublinear time!

16 12 / 73 Sample complexity Sample complexity=number of samples accessed in time domain.

17 13 / 73 Sample complexity In medical imaging (MRI, NMR), one measures Fourier coefficients x of imaged object x (which is often sparse)

18 13 / 73 Sample complexity In medical imaging (MRI, NMR), one measures Fourier coefficients x of imaged object x (which is often sparse)

19 14 / 73 Sample complexity Measure x C n, compute the Inverse Discrete Fourier Transform (IDFT) of x: x i = x j ω ij. j [n]

20 14 / 73 Sample complexity Measure x C n, compute the Inverse Discrete Fourier Transform (IDFT) of x: x i = x j ω ij. j [n] Given x C n, compute the Discrete Fourier Transform (DFT) of x: x i = 1 x n j ω ij. j [n]

21 15 / 73 Sample complexity Measure x C n, compute the Inverse Discrete Fourier Transform (IDFT) of x: x i = x j ω ij. j [n] Given x C n, compute the Discrete Fourier Transform (DFT) of x: x i = 1 x n j ω ij. j [n]

22 15 / 73 Sample complexity Measure x C n, compute the Inverse Discrete Fourier Transform (IDFT) of x: x i = x j ω ij. j [n] Given x C n, compute the Discrete Fourier Transform (DFT) of x: x i = 1 x n j ω ij. j [n] Sample complexity=number of samples accessed in time domain. Governs the measurement complexity of imaging process.

23 16 / 73 Given access to signal x in time domain, find best k-sparse approximation to x approximately Minimize 1. runtime 2. number of samples

24 17 / 73 Algorithms Randomization Approximation Hashing Sketching... Signal processing Fourier transform Hadamard transform Filters Compressive sensing...

25 18 / 73 Lecture 1: summary of techniques from Gilbert-Guha-Indyk-Muthukrishnan-Strauss 02, Akavia-Goldwasser-Safra 03, Gilbert-Muthukrishnan-Strauss 05, Iwen 10, Akavia 10, Hassanieh-Indyk-Katabi-Price 12a, Hassanieh-Indyk-Katabi-Price 12b Lecture 2: Algorithm with O(k logn) runtime (noiseless case) Hassanieh-Indyk-Katabi-Price 12b Lecture 3: Algorithm with O(k log 2 n) runtime (noisy case) Hassanieh-Indyk-Katabi-Price 12b Lecture 4: Algorithm with O(k logn) sample complexity Indyk-Kapralov-Price 14, Indyk-Kapralov 14

26 19 / 73 Outline 1. Computing Fourier transform of 1-sparse signals fast 2. Sparsity k > 1: main ideas and challenges

27 20 / 73 Outline 1. Computing Fourier transform of 1-sparse signals fast 2. Sparsity k > 1: main ideas and challenges

28 21 / 73 Sparse Fourier Transform (k = 1) Warmup: x is exactly 1-sparse: x f = 0 when f f for some f f time frequency Note: signal is a pure frequency Given: access to x Need: find f and x f

29 22 / 73 Two-point sampling Input signal x is a pure frequency, so x j = a ω f j

30 22 / 73 Two-point sampling Input signal x is a pure frequency, so x j = a ω f j Sample x 0,x 1

31 23 / 73 Two-point sampling Input signal x is a pure frequency, so x j = a ω f j Sample x 0,x 1 We have x 0 = a x 1 = a ω f x 1 = a ω f x 0 = a

32 24 / 73 Two-point sampling Input signal x is a pure frequency, so x j = a ω f j Sample x 0,x 1 We have x 0 = a x 1 = a ω f ω f x 0 = a x 1 = a ω f So x 1 /x 0 = ω f unit circle

33 25 / 73 Two-point sampling Input signal x is a pure frequency, so x j = a ω f j Sample x 0,x 1 We have x 0 = a x 1 = a ω f ω f x 0 = a x 1 = a ω f So x 1 /x 0 = ω f Can read frequency from the angle! unit circle

34 Two-point sampling Input signal x is a pure frequency, so x j = a ω f j Sample x 0,x 1 We have x 0 = a x 1 = a ω f ω f x 0 = a x 1 = a ω f So x 1 /x 0 = ω f Can read frequency from the angle! unit circle Pro: constant time algorithm Con: depends heavily on the signal being pure 26 / 73

35 27 / 73 Two-point sampling Input signal x is a pure frequency+noise, so x j = a ω f j +noise Sample x 0,x 1 x 1 = a ω f We have So x 0 = a+noise x 1 = a ω f +noise x 1 /x 0 = ω f +noise Can read frequency from the angle! ω f x 0 = a unit circle

36 28 / 73 Two-point sampling Input signal x is a pure frequency+noise, so x j = a ω f j +noise Sample x 0,x 1 x 1 = a ω f We have So x 0 = a+noise x 1 = a ω f +noise x 1 /x 0 = ω f +noise Can read frequency from the angle! ω f x 0 = a unit circle

37 Two-point sampling Input signal x is a pure frequency+noise, so x j = a ω f j +noise Sample x 0,x 1 x 1 = a ω f We have x 0 = a+noise x 1 = a ω f +noise So x 1 /x 0 = ω f +noise ω f x 0 = a Can read frequency from the angle! unit circle Pro: constant time algorithm Con: depends heavily on the signal being pure 29 / 73

38 30 / 73 Sparse Fourier Transform (k = 1) Warmup part 2: x is 1-sparse plus noise time frequency Note: signal is a pure frequency plus noise Given: access to x Need: find f and x f

39 31 / 73 Sparse Fourier Transform (k = 1) Warmup part 2: x is 1-sparse plus noise head time frequency Note: signal is a pure frequency plus noise Given: access to x Need: find f and x f

40 32 / 73 Sparse Fourier Transform (k = 1) Warmup part 2: x is 1-sparse plus noise head tail noise time frequency Note: signal is a pure frequency plus noise Given: access to x Need: find f and x f

41 33 / 73 Sparse Fourier Transform (k = 1) Warmup part 2: x is 1-sparse plus noise head tail noise time frequency Note: signal is a pure frequency plus noise Ideally, find pure frequency x that approximates x best: min 1 sparse x x x 2

42 34 / 73 Sparse Fourier Transform (k = 1) Warmup part 2: x is 1-sparse plus noise head time frequency Note: signal is a pure frequency plus noise Ideally, find pure frequency x that approximates x best: min 1 sparse x x x 2

43 35 / 73 Sparse Fourier Transform (k = 1) Warmup part 2: x is 1-sparse plus noise head time frequency Note: signal is a pure frequency plus noise Ideally, find pure frequency x that approximates x best: min 1 sparse x x x 2 = tail noise 2

44 36 / 73 l 2 /l 2 sparse recovery Ideally, find pure frequency x that approximates x best Need to allow approximation: find ŷ such that x ŷ 2 C tail noise 2 where C > 1 is the approximation factor.

45 37 / 73 l 2 /l 2 sparse recovery Ideally, find pure frequency x that approximates x best Need to allow approximation: find ŷ such that x ŷ 2 3 tail noise 2

46 Approximation guarantee Find ŷ such that x ŷ 2 3 tail noise /

47 Approximation guarantee Find ŷ such that x ŷ 2 3 tail noise /

48 Approximation guarantee Find ŷ such that x ŷ 2 3 tail noise 2 Note: only meaningful if or, equivalently, x 2 > 3 tail noise 2 f f x f 2 ε a /

49 Approximation guarantee Find ŷ such that x ŷ 2 3 tail noise 2 Note: only meaningful if or, equivalently, x 2 > 3 tail noise 2 f f x f 2 ε a 2 (assume this for the lecture) /

50 Approximation guarantee Find ŷ such that x ŷ 2 3 tail noise 2 Note: only meaningful if or, equivalently, x 2 > 3 tail noise 2 f f x f 2 ε a 2 (assume this for the lecture) /

51 Approximation guarantee Find ŷ such that x ŷ 2 3 tail noise 2 Note: only meaningful if or, equivalently, x 2 > 3 tail noise 2 f f x f 2 ε a 2 (assume this for the lecture) head tail noise / 73

52 44 / 73 A robust algorithm for finding the heavy hitter head tail noise frequency Assume that x f 2 ε a 2 f f Describe algorithm for the noiseless case first (ε = 0) Suppose that x j = a ω f j.

53 A robust algorithm for finding the heavy hitter head tail noise frequency Assume that x f 2 ε a 2 f f Describe algorithm for the noiseless case first (ε = 0) Suppose that x j = a ω f j. Will find f bit by bit (binary search). 44 / 73

54 45 / 73 Bit 0 Suppose that f = 2f +b, we want b Compute x 0 = a x n/2 = a ω f (n/2)

55 45 / 73 Bit 0 Suppose that f = 2f +b, we want b Compute x 0 = a x n/2 = a ω f (n/2) Claim We have x n/2 = x 0 ( 1) b (Even frequencies are n/2-periodic, odd are n/2-antiperiodic) Proof. x n/2 = a ω f (n/2) = a ( 1) 2f +b = x 0 ( 1) b

56 46 / 73 Bit 0 Suppose that f = 2f +b, we want b Compute x 0+r = a ω f r x n/2+r = a ω f (n/2+r) Claim For all r [n] we have x n/2+r = x 0+r ( 1) b (Even frequencies are n/2-periodic, odd are n/2-antiperiodic) Proof. x n/2+r = a ω f (n/2+r) = a ω f r ( 1) 2f +b = x0+r ( 1) b

57 47 / 73 Bit 0 Suppose that f = 2f +b, we want b Compute x r = a ω f r x n/2+r = a ω f (n/2+r) Claim For all r [n] we have x n/2+r = x r ( 1) b (Even frequencies are n/2-periodic, odd are n/2-antiperiodic) Proof. x n/2+r = a ω f (n/2+r) = a ω f r ( 1) 2f +b = xr ( 1) b

58 48 / 73 Bit 0 Suppose that f = 2f +b, we want b Compute x r = a ω f r x n/2+r = a ω f (n/2+r) Claim For all r [n] we have x n/2+r = x r ( 1) b (Even frequencies are n/2-periodic, odd are n/2-antiperiodic) Proof. x n/2+r = a ω f (n/2+r) = a ω f r ( 1) 2f +b = xr ( 1) b Will need arbitrary r s for the noisy setting

59 49 / 73 Bit 0 test Set b 0 0 if x n/2+r +x r > x n/2+r x r b 0 1 o.w.

60 49 / 73 Bit 0 test Set b 0 0 if x n/2+r +x r > x n/2+r x r b 0 1 o.w. Correctness: If b = 0, then x n/2+r +x r = 2 x r = 2 a and x n/2+r x r = 0

61 49 / 73 Bit 0 test Set b 0 0 if x n/2+r +x r > x n/2+r x r b 0 1 o.w. Correctness: If b = 0, then x n/2+r +x r = 2 x r = 2 a and x n/2+r x r = 0 If b = 1, then x n/2+r +x r = 0 and x n/2+r x r = 2 x r = 2 a

62 50 / 73 Bit 1 Can pretend that b 0 = 0. Why? Claim (Time shift theorem) If y j = x j ω j, then ŷ f = x f. Proof. ŷ f = 1 n j ω j [n]y fj = 1 x n j ω j ω fj j [n] = 1 j (f ) x n j ω j [n] = x f

63 50 / 73 Bit 1 Can pretend that b 0 = 0. Why? Claim (Time shift theorem) If y j = x j ω j, then ŷ f = x f. Proof. ŷ f = 1 n j ω j [n]y fj = 1 x n j ω j ω fj j [n] = 1 j (f ) x n j ω j [n] = x f If b 0 = 1, then replace x with y j := x j ω j b 0.

64 51 / 73 Bit 1 Assume b 0 = 0. Then we have f = 2f, so x j = a ω f j = a ω 2f j = a ω f j N/2.

65 51 / 73 Bit 1 Assume b 0 = 0. Then we have f = 2f, so x j = a ω f j = a ω 2f j = a ω f j N/2. Let ẑ j := x 2j, i.e. spectrum of z contains even components of spectrum of x

66 Bit 1 Assume b 0 = 0. Then we have f = 2f, so x j = a ω f j = a ω 2f j = a ω f j N/2. Let ẑ j := x 2j, i.e. spectrum of z contains even components of spectrum of x Then (x 0,...,x N/2 1 ) = (z 0,...,z N/2 1 ) are time samples of z j ; and ẑ f = a is the heavy hitter in z. 51 / 73

67 Bit 1 Assume b 0 = 0. Then we have f = 2f, so x j = a ω f j = a ω 2f j = a ω f j N/2. Let ẑ j := x 2j, i.e. spectrum of z contains even components of spectrum of x Then (x 0,...,x N/2 1 ) = (z 0,...,z N/2 1 ) are time samples of z j ; and ẑ f = a is the heavy hitter in z. So by previous derivation z N/4+r = z r ( 1) b 1 And hence x n/4+r ω (n/4+r)b 0 = x r ω r b0 ( 1) b 1 51 / 73

68 Bit 1 Assume b 0 = 0. Then we have f = 2f, so x j = a ω f j = a ω 2f j = a ω f j N/2. Let ẑ j := x 2j, i.e. spectrum of z contains even components of spectrum of x Then (x 0,...,x N/2 1 ) = (z 0,...,z N/2 1 ) are time samples of z j ; and ẑ f = a is the heavy hitter in z. So by previous derivation z N/4+r = z r ( 1) b 1 And hence x n/4+r ω (n/4)b 0 = x r ( 1) b 1 52 / 73

69 53 / 73 Decoding bit by bit Set b 0 0 if x n/2+r +x r > x n/2+r x r b 0 1 o.w.

70 53 / 73 Decoding bit by bit Set b 0 0 if x n/2+r +x r > x n/2+r x r b 0 1 o.w. Set b 1 0 if ω (n/4)b 0x n/4+r +x r > ω (n/4)b 0x n/4+r x r b 1 1 o.w.

71 53 / 73 Decoding bit by bit Set b 0 0 if x n/2+r +x r > x n/2+r x r b 0 1 o.w. Set b 1 0 if ω (n/4)b 0x n/4+r +x r > ω (n/4)b 0x n/4+r x r b 1 1 o.w.... ω (n/8)(2b 1+b 0 ) x n/8+r +x r > ω (n/8)(2b 1+b 0 ) x n/8+r x r...

72 53 / 73 Decoding bit by bit Set b 0 0 if x n/2+r +x r > x n/2+r x r b 0 1 o.w. Set b 1 0 if ω (n/4)b 0x n/4+r +x r > ω (n/4)b 0x n/4+r x r b 1 1 o.w.... ω (n/8)(2b 1+b 0 ) x n/8+r +x r > ω (n/8)(2b 1+b 0 ) x n/8+r x r... Overall: O(logn) samples to identify f. Runtime O(logn)

73 54 / 73 Noisy setting (dealing with ε) We now have x j = a ω f j + f f x f ω fj = a ω f j + µj (µ j is the noise in time domain) Argue that µ j is usually small?

74 54 / 73 Noisy setting (dealing with ε) We now have x j = a ω f j + f f x f ω fj = a ω f j + µj (µ j is the noise in time domain) Argue that µ j is usually small? Parseval s equality: noise energy in time domain is proportional to noise energy in frequency domain: N 1 j=0 µ j 2 = n f f x f 2.

75 Noisy setting (dealing with ε) We now have x j = a ω f j + f f x f ω fj = a ω f j + µj (µ j is the noise in time domain) Argue that µ j is usually small? Parseval s equality: noise energy in time domain is proportional to noise energy in frequency domain: N 1 j=0 µ j 2 = n So on average µ j 2 is small: E j [ µ j 2 ] f f x f 2. f f x f 2 ε a 2 54 / 73

76 55 / 73 Need to ensure that: 1. f is decoded correctly 2. a is estimated well enough to satisfy l 2 /l 2 guarantees: x ŷ 2 C x x 2

77 56 / 73 Decoding in the noisy setting Bit 0: set b 0 0 if x n/2+r +x r > x n/2+r x r and b 0 1 o.w. Claim If µ n/2+r < a /2 and µ r < a /2, then outcome of the bit test is the same.

78 56 / 73 Decoding in the noisy setting Bit 0: set b 0 0 if x n/2+r +x r > x n/2+r x r and b 0 1 o.w. Claim If µ n/2+r < a /2 and µ r < a /2, then outcome of the bit test is the same. Suppose b 0 = 1.

79 56 / 73 Decoding in the noisy setting Bit 0: set b 0 0 if x n/2+r +x r > x n/2+r x r and b 0 1 o.w. Claim If µ n/2+r < a /2 and µ r < a /2, then outcome of the bit test is the same. Suppose b 0 = 1. Then x n/2+r +x r µ n/2+r + µ r < a

80 56 / 73 Decoding in the noisy setting Bit 0: set b 0 0 if x n/2+r +x r > x n/2+r x r and b 0 1 o.w. Claim If µ n/2+r < a /2 and µ r < a /2, then outcome of the bit test is the same. Suppose b 0 = 1. x r = a ω f r + µ r Then x n/2+r +x r µ n/2+r + µ r < a and x n/2+r x r 2 a µ n/2+r µ r > a x n/2+r = a ω f (n/2+r) + µ n/2+r

81 57 / 73 Decoding in the noisy setting On average µ j 2 is small: E j [ µ j 2 ] x f 2 ε a 2 f f By Markov s inequality Pr j [ µ j 2 > a 2 /4] Pr j [ µ j 2 > (1/(4ε)) E j [ µ j 2 ]] 4ε

82 57 / 73 Decoding in the noisy setting On average µ j 2 is small: E j [ µ j 2 ] x f 2 ε a 2 f f By Markov s inequality Pr j [ µ j 2 > a 2 /4] Pr j [ µ j 2 > (1/(4ε)) E j [ µ j 2 ]] 4ε By a union bound Pr r [ µ r a /2 and µ n/2+r a /2] 1 8ε

83 57 / 73 Decoding in the noisy setting On average µ j 2 is small: E j [ µ j 2 ] x f 2 ε a 2 f f By Markov s inequality Pr j [ µ j 2 > a 2 /4] Pr j [ µ j 2 > (1/(4ε)) E j [ µ j 2 ]] 4ε By a union bound Pr r [ µ r a /2 and µ n/2+r a /2] 1 8ε Thus, a bit test is correct with probability at least 1 8ε.

84 58 / 73 Decoding in the noisy setting Bit 0: set b 0 to zero if x n/2+r +x r > x n/2+r x r and to 1 otherwise For ε < 1/64 each test is correct with probability 3/4. Final test: perform T 1 independent tests, use majority vote. How large should T be? Success probability?

85 59 / 73 Decoding in the noisy setting For j = 1,...,T let We have E[Z j ] 3/4. { 1 if j-th test is correct Z j = 0 o.w.

86 59 / 73 Decoding in the noisy setting For j = 1,...,T let We have E[Z j ] 3/4. Chernoff bounds { 1 if j-th test is correct Z j = 0 o.w. T Pr[ Z j < T /2] < e Ω(T ). j=1 Set T = O(loglogn) Majority is correct with probability at least 1 1/(16log 2 n) So all bits correct with probability 15/16

87 60 / 73 Estimating the value of heavy hitter Recall that x r = a ω f r + µr (noise) Our estimate: pick random r [n] and output est x r ω f r

88 60 / 73 Estimating the value of heavy hitter Recall that x r = a ω f r + µr (noise) Our estimate: pick random r [n] and output est x r ω f r Expected squared error? E r [ est a 2 ]

89 60 / 73 Estimating the value of heavy hitter Recall that x r = a ω f r + µr (noise) Our estimate: pick random r [n] and output est x r ω f r Expected squared error? E r [ est a 2 ] = E r [ x r ω f r a 2 ]

90 60 / 73 Estimating the value of heavy hitter Recall that x r = a ω f r + µr (noise) Our estimate: pick random r [n] and output est x r ω f r Expected squared error? E r [ est a 2 ] = E r [ x r ω f r a 2 ] = E r [ x r a ω f r 2 ]

91 60 / 73 Estimating the value of heavy hitter Recall that x r = a ω f r + µr (noise) Our estimate: pick random r [n] and output est x r ω f r Expected squared error? E r [ est a 2 ] = E r [ x r ω f r a 2 ] = E r [ x r a ω f r 2 ] = E r [ µ r 2 ] Now by Markov s inequality Pr r [ est a 2 > 4ε a 2 ] < 1/4.

92 Putting it together: algorithm for 1-sparse signals Let { est if f = f ŷ f = 0 o.w. By triangle inequality ŷ x 2 ŷ f a 2 + ŷ f x f 2 2 ε a + ε a = 3 x x 2. Thus, with probability 2/3 our algorithm satisfies l 2 /l 2 guarantee with C = / 73

93 62 / 73 Runtime=O(logn loglogn) Sample complexity=o(logn loglogn)

94 62 / 73 Runtime=O(logn loglogn) Sample complexity=o(logn loglogn) Ex. 1: reduce sample complexity to O(logn), keep O(poly(logn)) runtime Ex. 2: reduce sample complexity to O(log 1/ε n)

95 62 / 73 Runtime=O(logn loglogn) Sample complexity=o(logn loglogn) Ex. 1: reduce sample complexity to O(logn), keep O(poly(logn)) runtime Ex. 2: reduce sample complexity to O(log 1/ε n) What about k > 1

96 63 / 73 Outline 1. Sparsity: definitions, motivation 2. Computing Fourier transform of 1-sparse signals fast 3. Sparsity k > 1: main ideas and challenges

97 Sparsity k > 1 Let x best k-sparse approximation of x Our goal: find ŷ such that x ŷ 2 C x x 2 where C > 1 is the approximation factor. (This is the l 2 /l 2 guarantee) 1 head frequency 64 / 73

98 65 / 73 Sparsity k > 1 Main idea: implement hashing to reduce to 1-sparse case: hash frequencies into k bins run 1-sparse algo on isolated elements Assumption: can randomly permute frequencies (will remove in next lecture) Implement hashing? Need to design a bucketing scheme for the frequency domain

99 Partition frequency domain into B k buckets time frequency 66 / 73

100 Partition frequency domain into B k buckets time frequency 67 / 73

101 67 / 73 Partition frequency domain into B k buckets time frequency For each j = 0,...,B 1 let { û j xf f =, if f j-th bucket 0 o.w. Restricted to a bucket, signal is likely approximately 1-sparse!

102 68 / 73 Partition frequency domain into B k buckets time frequency For each j = 0,...,B 1 let { û j xf f =, if f j-th bucket 0 o.w. Restricted to a bucket, signal is likely approximately 1-sparse!

103 69 / 73 Partition frequency domain into B k buckets time frequency For each j = 0,...,B 1 let { û j xf f =, if f j-th bucket 0 o.w. Restricted to a bucket, signal is likely approximately 1-sparse!

104 69 / 73 Partition frequency domain into B k buckets time frequency For each j = 0,...,B 1 let { û j xf f =, if f j-th bucket 0 o.w. Restricted to a bucket, signal is likely approximately 1-sparse!

105 69 / 73 Partition frequency domain into B k buckets time frequency For each j = 0,...,B 1 let { û j xf f =, if f j-th bucket 0 o.w. Restricted to a bucket, signal is likely approximately 1-sparse!

106 69 / 73 Partition frequency domain into B k buckets time frequency For each j = 0,...,B 1 let { û j xf f =, if f j-th bucket 0 o.w. Restricted to a bucket, signal is likely approximately 1-sparse!

107 69 / 73 Partition frequency domain into B k buckets time frequency For each j = 0,...,B 1 let { û j xf f =, if f j-th bucket 0 o.w. Restricted to a bucket, signal is likely approximately 1-sparse!

108 69 / 73 Partition frequency domain into B k buckets time frequency For each j = 0,...,B 1 let { û j xf f =, if f j-th bucket 0 o.w. Restricted to a bucket, signal is likely approximately 1-sparse!

109 70 / 73 1 Zero-th bucket signal u 0 : { û 0 xf, if f [ n f = 2B : 2B n ] 0 o.w frequency

110 70 / 73 1 Zero-th bucket signal u 0 : { û 0 xf, if f [ n f = 2B : 2B n ] 0 o.w frequency We want time domain access to u 0 : for any a = 0,...,n 1, compute u 0 a = f û 0 f ω f a

111 70 / 73 1 Zero-th bucket signal u 0 : { û 0 xf, if f [ n f = 2B : 2B n ] 0 o.w frequency We want time domain access to u 0 : for any a = 0,...,n 1, compute u 0 a = f û 0 f ω f a = n 2B f n 2B x f ω f a

112 70 / 73 1 Zero-th bucket signal u 0 : { û 0 xf, if f [ n f = 2B : 2B n ] 0 o.w frequency We want time domain access to u 0 : for any a = 0,...,n 1, compute u 0 a = f û 0 f ω f a = n 2B f n 2B x f ω f a = n 2B f n 2B where y j = x j+a (y is a time shift of x by the time shift theorem). ŷ f,

113 71 / 73 We want time domain access to u 0 : for any a = 0,...,n 1, compute ua 0 = n 2B f n 2B ŷ f, where y j = x j+a (y is a time shift of x).

114 71 / 73 We want time domain access to u 0 : for any a = 0,...,n 1, compute ua 0 = n 2B f n 2B ŷ f, where y j = x j+a (y is a time shift of x). Let { [ 1, if f n Ĝ f = 2B : 2B n ] 0 o.w. Then u 0 a = n 2B f n 2B ŷ f

115 71 / 73 We want time domain access to u 0 : for any a = 0,...,n 1, compute ua 0 = n 2B f n 2B ŷ f, where y j = x j+a (y is a time shift of x). Let { [ 1, if f n Ĝ f = 2B : 2B n ] 0 o.w. Then u 0 a = n 2B f n 2B ŷ f = ŷ f Ĝf f [n]

116 71 / 73 We want time domain access to u 0 : for any a = 0,...,n 1, compute ua 0 = n 2B f n 2B ŷ f, where y j = x j+a (y is a time shift of x). Let { [ 1, if f n Ĝ f = 2B : 2B n ] 0 o.w. Then u 0 a = n 2B f n 2B ŷ f = ŷ f Ĝf = (ŷ Ĝ)(0) f [n]

117 71 / 73 We want time domain access to u 0 : for any a = 0,...,n 1, compute ua 0 = n 2B f n 2B ŷ f, where y j = x j+a (y is a time shift of x). Let { [ 1, if f n Ĝ f = 2B : 2B n ] 0 o.w. Then u 0 a = n 2B f n 2B ŷ f = f [n] ŷ f Ĝf = (ŷ Ĝ)(0) = ( x +a Ĝ)(0)

118 72 / 73 Need to evaluate for j = 0,...,B 1. ( ( x Ĝ) j n ) B We have access to x, not x...

119 72 / 73 Need to evaluate for j = 0,...,B 1. ( ( x Ĝ) j n ) B By the convolution identity We have access to x, not x... x Ĝ = (x G)

120 72 / 73 Need to evaluate for j = 0,...,B 1. ( ( x Ĝ) j n ) B By the convolution identity We have access to x, not x... x Ĝ = (x G) Suffices to compute x G j n,j = 0,...,B 1 B

121 Suffices to compute x G j n,j = B/2,...,B/2 1 B Sample complexity? Runtime? time frequency 72 / 73

122 Suffices to compute x G j n,j = B/2,...,B/2 1 B Sample complexity? Runtime? time frequency 72 / 73

123 Suffices to compute x G j n,j = B/2,...,B/2 1 B Sample complexity? Runtime? time frequency 72 / 73

124 Suffices to compute x G j n,j = B/2,...,B/2 1 B Sample complexity? Runtime? time frequency Computing x G takes Ω(N) time and samples! 73 / 73

125 Suffices to compute x G j n,j = B/2,...,B/2 1 B Sample complexity? Runtime? time frequency Computing x G takes Ω(N) time and samples! Design a filter supp(g) k? Truncate sinc? Tolerate imprecise hashing? Collisions in buckets? 73 / 73