1 minimization without amplitude information. Petros Boufounos

Transcription

1 1 minimization without amplitude information Petros Boufounos

2 The Big 1 Picture Classical 1 reconstruction problems: min 1 s.t. f() = 0 min 1 s.t. f() 0 min 1 + λf()

3 The Big 1 Picture Sparsity Model f() = Φ y 2 Classical 1 reconstruction problems: min 1 s.t. f() = 0 min 1 s.t. f() 0 min 1 + λf() Data Fidelity: f() =ɛ Φ y 2

4 Data without amplitude information min 1 + λf() Q: What if data provide no amplitude information on?

5 Data without amplitude information min 1 + λf() Q: What if data provide no amplitude information on? f(α) =αf() Minimizing solution is degenerate: =0 Impose an amplitude constraint.

6 Case I: Sampling the Zero Crossings

7 Signal Reconstruction from Zero Crossings (t) t1 t2... tn Q: Given only the zero crossings {t1,t2,...,tn} of a signal can we reconstruct it? -2(t)

8 Signal Reconstruction from Zero Crossings (t) t1 t2... tn Q: Given only the zero crossings {t1,t2,...,tn} of a signal can we reconstruct it? -2(t) (t) {t1,t2,...,tn} Clock Easy implementation: only need a comparator and a clock

9 Logan s Theorem (t) t1 t2... tn -2(t) Q: Given only the zero crossings {t1,t2,...,tn} of a signal can we reconstruct it? Logan s Theorem: YES. Signals bandlimited to [B,2B) are uniquely determined by their zero crossings. BUT: an arbitrary set of zero crossings might not correspond to a signal bandlimited to [B,2B). Reconstruction is not robust. There is ambiguity.

10 Logan s Theorem (t) t1 t2... tn -2(t) Q: Given only the zero crossings {t1,t2,...,tn} of a signal can we reconstruct it? Logan s Theorem: YES. Signals bandlimited to [B,2B) are uniquely determined by their zero crossings. BUT: an arbitrary set of zero crossings might not correspond to a signal bandlimited to [B,2B). Reconstruction is not robust. There is ambiguity. Introduce sparsity to resolve the ambiguity!

11 Signal Representation Fourier series of (t): (t) = n B [a n cos(2πnt)+b n sin(2πnt)] Vector of coefficients: = a n1. a nn/2 b n1. b nn/2

12 Sampling Operator Given {t1,t2,...,tn}, Φ {tk } = cos (2πn 1 t 1 )... cos ( ) 2πn N/2 t 1 cos (2πn 1 t 2 )... cos ( ) 2πn N/2 t 2... cos (2πn 1 t N )... cos ( ) 2πn N/2 t N sin (2πn 1 t 1 )... sin ( ) 2πn N/2 t 1 sin (2πn 1 t 2 )... sin ( ) 2πn N/2 t sin (2πn 1 t N )... sin ( ) 2πn N/2 t N Samples the signal at those times: Φ {tk } = (t 1 )... (t N )

13 Reconstruction Problem Φ {tk } = (t 1 )... (t N ) If T={t1,t2,...,tN} are the zero crossings, then the desired signal is in the nullspace of Φ: Φ T =0. Logan s theorem ΦT has a one-dimensional nullspace.

14 Signal Acquisition and Reconstruction (t) {t1,t2,...,tn} Clock

15 Signal Acquisition and Reconstruction (t) {t1,t2,...,tn} Clock T={t1,t2,...,tN} Find 1-D Build ΦT nullspace (e.g. SVD)

16 Signal Acquisition and Reconstruction (t) {t1,t2,...,tn} Clock T={t1,t2,...,tN} Find 1-D Build ΦT nullspace (e.g. SVD) SVD(Φ T ) In practice: noise and quantization. No nullspace! Many small singular values. Ambiguity!

17 Sparse Reconstruction 1 minimization: = arg min 1 subject to Φ =0

18 Sparse Reconstruction 1 minimization: = arg min 1 subject to Φ =0 Relaation: = arg min 1 + λ 2 Φ 2 2

19 Sparse Reconstruction 1 minimization: = arg min 1 subject to Φ =0 Relaation: = arg min 1 + λ 2 Φ 2 2 Unit energy constraint: = arg min 1 + λ 2 Φ 2 2 subject to 2 =1

20 Fied Point Equilibrium = arg min 1 + λ 2 Φ 2 2 subject to 2 =1 Unconstrained minimization: Cost() = g()+ λ 2 f(φ) Cost () = g ()+ λ 2 Φ f (Φ) (g ()) i = where: 1 i < 0 [ 1, 1] i =0 +1 i > 0 No change if gradients are projected on unit sphere

21 Minimization Algorithm (based on FPC [Hale, Yin, Zhang, 07]) Big Picture: Gradient descent until equilibrium. Initialization parameters:, τ 1. Compute quadratic gradient: h = Φ T Φ 2. Project onto sphere: h p = h, h 3. Quadratic gradient descent: τh p 4. Shrink ( 1 gradient descent): i sign( i ) ma { i τ λ, 0 } 5. Normalize: 6. Iterate until equilibrium.

22 Optimization on the Sphere = arg min 1 + λ 2 Φ 2 2 subject to 2 =1 Optimization is not conve. Convergence to global optimum not guaranteed.

23 Optimization on the Sphere = arg min 1 + λ 2 Φ 2 2 subject to 2 =1 Optimization is not conve. Convergence to global optimum not guaranteed. Eploit randomness: Eecute L times with random initializations. Pick best solution. If P=P(success for 1 eecution), then P(overall success)=1-(1-p) L

24 Results Probability of Success 1 Probability of success L=1 L=5 L=10 L= % Signal Sparsity (2K/N) L=number of random initializations N=256 coefficients

25 Results Probability of Success 1 Probability of success SVD L=1 0.2 L=5 L= L= % Signal Sparsity (2K/N) L=number of random initializations N=256 coefficients

26 Further Relaation (w/ Cinmay Hegde) Optimization on sphere: = arg min 1 + λ 2 Φ 2 2 subject to 2 =1 Relaation of sphere constraint: = arg min 1 + λ 1 2 Φ λ We can now use standard 1 algorithms!

27 1 minimization formulation = arg min 1 + λ 1 2 Φ λ Let: [ c ] ( λ2 ) 2 Φ = Φ,c= λ 1 At equilibrium: = arg min 1 + λ 1 2 Φ [ c 0 ] 2 2

28 Reweighted FPC algorithm Initialization parameters:, λ 1, λ 2 ] ) 2 1. Build Φ = [ c Φ,c= ( λ2 λ 1 2. Estimate using FPC: = arg min 1 + λ 1 2 Φ [ c 0 ] Iterate until equilibrium.

29 Results Probability of Success 1 Probability of success SVD L=1 0.2 L=5 L= L= % Signal Sparsity (2K/N) L=number of random initializations N=256 coefficients

30 Results Probability of Success 1 Probability of success SVD L=1 L=5 0.2 L=10 Reweighted FPC L= % Signal Sparsity (2K/N) L=number of random initializations N=256 coefficients

31 Case II: 1-bit Compressive Sensing

32 1-Bit Compressive Sensing Q: Can we quantize measurements to 1-bit: y = sign(φ) y i = sign( φ i, ) and recover the signal (within a positive scaling factor)?

33 1-Bit Compressive Sensing Q: Can we quantize measurements to 1-bit: y = sign(φ) y i = sign( φ i, ) and recover the signal (within a positive scaling factor)? 1-bit measurements are inepensive. Focus on bits rather than measurements. Eact recovery is not possible.

34 Reconstruction from 1-bit Measurements Sign information from 1-bit measurements: y i = sign(φ) i y i (Φ) i 0 Reconstruction should enforce model. Reconstruction should be consistent with measurements. = arg min 1 subject to y i (Φ) i 0

35 Reconstruction from 1-bit Measurements Sign information from 1-bit measurements: y i = sign(φ) i y i (Φ) i 0 Reconstruction should enforce model. Reconstruction should be consistent with measurements. Reconstruction should enforce a non-trivial solution. = arg min 1 subject to y i (Φ) i 0 and 2 =1

36 Information in 1-bit Measurements

37 Information in 1-bit Measurements

38 Information in 1-bit Measurements

39 Information in 1-bit Measurements

40 Information in 1-bit Measurements

41 Information in 1-bit Measurements

42 Information in 1-bit Measurements

43 Information in 1-bit Measurements

44 Information in 1-bit Measurements

45 Constraint Relaation = arg min 1 subject to y i (Φ) i 0 and 2 =1

46 Constraint Relaation = arg min 1 subject to y i (Φ) i 0 and 2 =1 We rela the inequality constraints: = arg min 1 + λ 2 subject to 2 =1 i f (y i (Φ)) where f() is a one sided quadratic: f() = { >0 λ

47 Fied point equilibrium = arg min 1 + λ 2 subject to 2 =1 i f (y i (Φ)) Unconstrained minimization: Y diag(y) Cost() = g()+ λ 2 f(yφ) (g ()) i = Cost () = g ()+ λ 2 (YΦ)T f(yφ) 1 i < 0 [ 1, 1] i =0 +1 i > 0 and ( f ) () 2 i = { i i 0 0 i > 0 No change if gradients are projected on unit sphere.

48 Minimization Algorithm Big Picture: Gradient descent until equilibrium. Initialization parameters:, τ 1. Compute quadratic gradient: h = (YΦ) T f (YΦ) 2. Project onto sphere: h p = h, h 3. Quadratic gradient descent: τh p 4. Shrink ( 1 gradient descent): i sign( i ) ma { i τ λ, 0 } 5. Normalize: 6. Iterate until equilibrium.

49 Results Reconstruction Error (N=512) %&'()*+,!!"!!#! )-,(./" (1& "!647(1& (!!"!!#! )6,(./$# (1& "!647(1& (!$! (! "!!! #!!! $!!!!$! (! "!!! #!!! $!!!! )5,(./0> (! )*,(./"#? ( %&'()*+,!"!!#! (1& "!647(1&!$! (! "!!! #!!! $!!! %()%8-39:8;8<73(=:(6473,!"!!#! (1& "!647(1&!$! (! "!!! #!!! $!!! %()%8-39:8;8<73(=:(6473,

50 1-bit Sampling of Images If the signal is an image, we have more information! (i.e., a better signal model) Images are sparse in wavelets and positive: = Wα i 0 and α is sparse Incorporate better model in the reconstruction: ˆα = arg min α α 1 subject to y i (ΦW) i 0 and (Wα) i 0 and α 2 =1

51 Results Original Image 4096 piels 256 levels

52 Results Original Image 4096 piels 256 levels Classical Compressive Sensing, 1 bit per piel 4096 measurements 1 bit per measurement 2048 measurements 2 bits per measurement 1024 measurements 4 bits per measurement 512 measurements 8 bits per measurement

53 Results Reconstruction on unit sphere 1 bit per piel 4096 measurements 1 bit per measurement Original Image 4096 piels 256 levels Classical Compressive Sensing, 1 bit per piel 4096 measurements 1 bit per measurement 2048 measurements 2 bits per measurement 1024 measurements 4 bits per measurement 512 measurements 8 bits per measurement

54 Results Reconstruction on unit sphere Original Image 4096 piels 256 levels Classical Compressive Sensing 4096 measurements 1 bit per measurement 4096 bits (1 bit per piel) 512 measurements 8 bits per measurement

55 Results Reconstruction on unit sphere Original Image 4096 piels 256 levels Classical Compressive Sensing 4096 measurements 1 bit per measurement 4096 bits (1 bit per piel) Reconstruction on unit sphere 512 measurements 8 bits per measurement 512 measurements 1 bit per measurement 512 bits (0.125 bits per piel)

56 Concluding Remarks Practical systems may eliminate amplitude information Reconstruction on the unit sphere is necessary The sphere is a well-behaved manifold Unit sphere constraint reduces the search space Several still open questions