Blind Deconvolution Using Convex Programming Jiaming Cao
Problem Statement The basic problem Consider that the received signal is the circular convolution of two vectors w and x, both of length L. How can we recover the vectors w and x from the single received signal? 0 y = w x (or y ll = ll w ll * x[ll ll * 1 + 1] ) w =? x =?
Problem Statement Structural assumptions Assume that w and x live in subspaces with dimensions K and N respectively, i.e. where B is a L K matrix, and C is a L N matrix. Knowing matrices B and C, reconstructing w and x is euivalent to reconstructing m and h.
Problem Statement An intuition Ahmed et al (2014)
Proposed Algorithm Matrix Observation Expand the convolution euation using the structural assumption, y = m 1 w C < + + m N w C > = circ C < circ C > m 1 w m N w where circ C C denotes the L L circulant matrix constructed be nth column of matrix C.
Proposed Algorithm Matrix Observation Take the Fourier transform, let the DFT matrix by F. Then use CF = FC, BF = FB, and, yg = Fy = Δ < BF Δ > BF m 1 h m N h where Δ C = diag( LCF C ) Related to outer product of h and m, hm = m 1 h m N h
Proposed Algorithm Matrix Observation yg = Fy = Δ < BF Δ > BF m 1 h m N h Let X Q = hm, using the observation that the operation to get yg is linear, we can note the expression as, yg = A(X Q ) Further, X Q is a rank 1 matrix by definition. Now we have a way to formulate the recovery of X Q.
Proposed Algorithm Formulation arg min rank X s. t. yg = A(X) Xe = Convex relaxation arg min X s. t. yg = A(X) Let σ_u_ < v_ < be the best rank 1 approximation to Xc, then set h c = σ_u_ < and md = σ_v_ <
Performance Guarantee Definitions and Assumptions WOLG, assume columns in B to be orthonormal, such that, Define, B B = Bg Bg = h b g jb g j = I 0 jk< Let, μ nop μ y = L max <sjs0 μ nwc = L K max <sjs0 bg j h, bg j = L K min <sjs0 bg j [1, L/K] [0,1] [1, Kμnop ](h unity norm) C l, n ~N(0, L < )
Performance Guarantee Theorem 1 Under the above assumptions, fix α 1. Then there exists a constant C = O(α) depending only on α such that if, max (μ nop K, μ L y N) C (log L)ˆ then X Q = hm is the uniue solution to the neuclear norm minimization problem with probability 1 O(L < ), and we can recover both w and x within a scalar multiple from y = w x. When the coherences are low (i.e. μ nop and μ y are on the order of a constant), the ineuality is tight to within a logarithmic factor, as we always have max (K, N) L
Performance Guarantee Theorem 1 max μ nop K, μ y N L C (log L)ˆ As we would like to have the lower bound low, small μ nop and μ y (i.e. B spread out in freuency domain, or incoherent ) are preferred. Eg. when B = I K, μ 0 nop = μ nwc = 1
Performance Guarantee Theorem 2 (stability in presence of noise) Let the noisy observation be, yg = A X Q + z where z is an unknown noise vector with z δ. The optimization problem is now, arg min s. t. X yg A(X) δ
Performance Guarantee Theorem 2 (stability in presence of noise) Let λ nwc and λ nop be the smallest and largest non- zero eigenvalues of AA, then with probability 1 L <, the solution to the modified optimization problem will obey, Xc X Q for a fixed constant C. C λ nop λ nwc min (K, N)δ When A is sufficiently underdetermined, NK L(log L), then with high probability, λ nop λ nwc ~ μ nop μ nwc
Performance Guarantee Theorem 2 (stability in presence of noise) Set δ = Xc X Q, the there exists a constant C such that, h αh c Cmin ( m 1 α md Cmin ( δ, h h ) δ, m m ) for some scalar multiple α.
Numerical Simulations Phase Transition Ahmed et al (2014)
Numerical Simulations In Presence of Noise Ahmed et al (2014)
Toy Example Image Deblurring x R 0 represents an image of 256x256 pixels, and w R 0 represents a blur kernel with the same dimension. Therefore, L = 256 256 = 65536. Let C be a set wavelet basis, and m be the active coefficients in wavelet domain. Let B be formed by a subset of columns in I matrix, and h is an unknown short vector.
Toy Example Image Deblurring
Toy Example Image Deblurring Knowing the support of the original image in wavelet domain Not knowing the support of the original image in wavelet domain
Comments and Related Works Novelty: casting blind deconvolution to a low- rank matrix recovery problem Drawback: it is known that SDP is feasible but very expensive, esp. at large scales (Li et al.) Speed up using non- convex methods (in presence of noise): min n,y yÿ A(mh )
Sketch of Proof (Theorem 2) Let Xc = X Q + h, P A be the projection operator onto the row space of A. By triangular ineuality and definition, A(h) yÿ A X Q + A Xc yÿ 2δ Recovery error can be decomposed as, h = P A (h) + P A (h) It can be shown that (details not included, Proposition 1) since P A (h) lies in the null space of A, X Q + P A h X Q C P P A h By triangular ineuality, after rearranging, P P A h C P A h C min (K, N) P A (h) It can be shown that (details not included) since P A (h) lies in the null space of A, P P A h 2λnop P P A h Therefore, P A (h) = P P A h + P P A h (2λnop + 1) P P A h
Sketch of Proof (Theorem 2) Plug into the expression for recovery error, we get, h P A h + 2λ nop + 1 P P A h Knowing that Frobenius norm is no greater than nuclear norm, by applying the previous bound on nuclear norm, we have, A(h) P A h + C(2λ nop + 1)min (K, N) P A (h) Absorbing all constants into C, h Cλ nop min (K, N) P A h Cλ nop min (K, N) A A(h) where A is the pseudoinverse of A, whose norm is λ nwc. Also use the previously established ineuality on A(h), the conclusion follows.
Discussion What is the benefit of viewing blind deconvolutionas a low rank recovery problem?
References 1. Ahmed, Ali, Benjamin Recht, and Justin Romberg. "Blind deconvolution using convex programming." IEEE Transactions on Information Theory 60.3 (2014): 1711-1732. 2. Li, Xiaodong, et al. "Rapid, robust, and reliable blind deconvolution via nonconvex optimization." arxiv preprint arxiv:1606.04933 (2016).