Supplementary Material of A Novel Sparsity Measure for Tensor Recovery

Transcription

1 Supplementary Material of A Novel Sparsity Measure for Tensor Recovery Qian Zhao 1,2 Deyu Meng 1,2 Xu Kong 3 Qi Xie 1,2 Wenfei Cao 1,2 Yao Wang 1,2 Zongben Xu 1,2 1 School of Mathematics and Statistics, Xi an Jiaotong University 2 Ministry of Education Key Lab of Intelligent Networks and Network Security, Xi an Jiaotong University timmy.zhaoqian@gmail.com dymeng@mail.xjtu.edu.cn xq.liwu@stu.xjtu.edu.cn {caowenf2015, yao.s.wang}@gmail.com zbxu@mail.xjtu.edu.cn 3 School of Mathematical Sciences, Liaocheng University xu.kong@hotmail.com Abstract In this supplementary material, we present details of the algorithms for solving the proposed tensor recovery models with folded-concave relaxations, and present more experimental results. 1. Solving tensor tecovery models with foldedconcave relaxations In the maintext, we have described how to solve the proposed tensor recovery models with the trace norm relaxation. Here, we present the strategy for solving the models with folded-concave relaxations Locally linear approximation for the foldedconcave penalties First, we introduce the idea of the locally linear approximation (LLA) for the folded-concave penalties, which is adopted from [10] and [6]. It is easy to verify that the following result holds: Proposition 1 ([6]) Let f(x) be a concave function on (0, ). Then with equality if x = x 0. f(x) f(x 0 ) + f (x 0 )(x x 0 ), (1) Apply this proposition to the folded-concave penalties, we can get the LLAs for them: ψ mcp (t) ψ mcp (t 0 ) + ψ mcp(t 0 )(t t 0 ) ψ mcp (t t 0 ), (2) and ψ scad (t) ψ scad (t 0 ) + ψ scad(t 0 )(t t 0 ) ψ scad (t t 0 ), (3) where ψ mcp(t 0 ) = with (x) + = max{x, 0}, and ( λ t ) 0, a + ψ scad(t 0 ) = λi{t 0 λ} + (aλ t 0) + I{t 0 > λ}, a 1 with I{A} being the indicator function which equals 1 if A is true and 0 otherwise. Using these LLAs, we can solve the original models with a two-stage approach, as discussed in the next subsection Two-stage algorithms for solving the models with folded-concave relaxations We mainly discuss the algorithm for RP-TC mcp in the maintext, and the idea can be easily generalized to other models. Recall the RP-TC mcp model: where X 1 C 2 S mcp (X ), s.t. X Ω = T Ω, (4) S mcp (X ) = d = d P mcp ( X(n) ) ( ) ψ mcp(σ (n)i ), i with σ (n)i being the ith singular value of X (n). Now assug that we have an initial estimation for X, denoted as X 0, we can apply (2) to ψ mcp (σ (n)i ) to obtain ψ mcp (σ (n)i σ 0 (n)i ) = ψ mcp(σ 0 (n)i )+ψ mcp(σ 0 (n)i )(σ (n)i σ 0 (n)i ), where σ 0 (n)i is the ith singular value of the unfolding matrix X 0 (n) along the nth model of X 0. Then we can replace ψ mcp (5)

2 with ψ mcp, and approximate S mcp (X ) as follows: where Ŝ mcp (X X 0 ) = d ( ) P mcp X (n) X 0 (n), (6) ( ) P mcp X (n) X 0 (n) = ψ mcp (σ (n)i σ 0 i (n)i ). From the above discussion, we can give a two-stage algorithm for solving (4). In the first stage, we solve RP-TC trace to get X 0, and then in the second stage, we solve the following problem: X 1 C 2 Ŝ mcp (X X 0 ), s.t. X Ω = T Ω. (7) Then the result can be used as X 0 to solve (7) again in an iterative way. However, Zou and Li [10] showed that onestep re-estimation already can provide good result in sparse linear regression. Therefore, we also use this one-step strategy which is empirically effective in our experiments. Now we discuss how to solve (7). We use the similar ADMM as for solving RP-TC trace, which has been discussed in the maintext. First, we introduce d auxiliary tensors M n (1 n d) and reformulate (7) as follows: 1 d ( ) P mcp X (n) X 0 X,M 1,,M d C 2 (n) s.t., X Ω = M Ω, X = M n, 1 n d. The augmented Lagrangian function for (8) is L ρ(x, M 1,, M d, Y 1,, Y d ) = 1 d ( P mcp M(n) X 0 ) d (n) + X M n, Y n C 2 + ρ 2 d X M n 2 F, where Y n (1 n d) are the Lagrange multipliers and ρ is a positive scalar. Then the problem can be solved within the ADMM framework. With M l (l n) fixed, M n can be updated by solving the following problem: (8) L ρ (X, M 1,, M d, Y 1,, Y d ), (9) M n which, with some algebra, is equivalent to ) α n Pmcp (M n(n) X 0 (n) + X M n, Y n + ρ M n 2 X M n 2 F, (10) where α n = 1 d ( ) P mcp M l(l) X 0 C 2 l=1,l n (l), 1 n d. Note that ( ) P mcp M n(n) X 0 (n) = ψ mcp (σ (n)k σ 0 k (n)k ) = i ψ mcp(σ 0 (n)k ) + ψ mcp(σ 0 (n)k )(σ (n)k σ 0 (n)k ), where σ (n)k is the k th singular value of M n(n), and then Eq. (10), by discarding constants irrelevant to σ (n)k s, can be further reduced to α n M n k ψ mcp(σ(n)k 0 )σ (n)k+ X M n, Y n + ρ 2 X M n 2 F, (11) which is intrinsically a weighted nuclear (trace) norm imization problem [2, 7]. Since σ(n)k 0 (k = 1, 2,..., r n) are of non-increasing order, and ψ mcp( ) is non-decreasing on (0, ), then ψ mcp(σ (n)k 0 ) (k = 1, 2,..., r n) are of nondecreasing order. Therefore, according to [7], (11) can be analytically solved by ( ( ( 1 M n = fold n D αn unfold ρ,vn n X + ρ Y ) )) n, (12) where v n = ( ψ mcp(σ 0 (n)1 ), ψ mcp(σ 0 (n)2 ),..., ψ mcp(σ 0 (n)r n )) T, and D τ,w (X) = UΣ τ,w V T is the generalized shrinkage operator with Σ τ,w = diag(max(σ i τw i, 0)) (σ i s are the singular values of X, and w i s are elements of w). Similarly, with M n s fixed, X can be updated by solving X L ρ(x, M 1,, M d, Y 1,, Y d ), (13) which also has the closed-form solution: X = 1 d ρd (ρm n Y n ). (14) Then the Lagrangian multipliers are updated by Y n := Y n ρ(m n X ), 1 n d, (15) and ρ is increased to µρ with some constant µ > 1. It is straightforward to generalize this two-stage algorithm to solving the proposed RP-TC scad, RP-TRPCA mcp and RP-TRPCA scad, and we omit them here for conciseness. 2. Additional experimental results 2.1. Detailed results on TC experiments In the maintext, we show the averaged results of the multispectral image completion. Here, we list the results on separated images in Table 1-8 for a more comprehensive comparison. It can be seen that the proposed methods can achieve the best or second best results in all situations in terms of the 5 PQIs utilized.

3 2.2. Additional results on TRPCA experiments In the maintext, we show the averaged results of the simulated multispectral image restoration. Here, we list the results on separated images in Table 9-18, and also show the visual results on 4 sample images in Fig We can see that the proposed methods can produce comparatively better restoration results than other competing methods, both quantitatively and visually. References [1] D. Goldfarb and Z. T. Qin. Robust low-rank tensor recovery: Models and algorithms. SIAM Journal on Matrix Analysis and Applications, 35(1): , [2] S. Gu, L. Zhang, W. Zuo, and X. Feng. Weighted nuclear norm imization with application to image denoising. In CVPR, [3] Z. Lin, M. Chen, and Y. Ma. The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices. arxiv preprint arxiv: , [4] J. Liu, P. Musialski, P. Wonka, and J. P. Ye. Tensor completion for estimating missing values in visual data. IEEE TPAMI, 34(1): , [5] Y. Peng, D. Y. Meng, Z. B. Xu, C. Q. Gao, Y. Yang, and B. Zhang. Decomposable nonlocal tensor dictionary learning for multispectral image denoising. In CVPR, [6] S. Wang, D. Liu, and Z. Zhang. Nonconvex relaxation approaches to robust matrix recovery. In IJCAI, [7] Q. Xie, D. Meng, S. Gu, L. Zhang, W. Zuo, X. Feng, and Z. Xu. On the optimal solution of weighted nuclear norm imization. arxiv preprint arxiv: , [8] Y. Xu, R. Hao, W. Yin, and Z. Su. Parallel matrix factorization for low-rank tensor completion. arxiv preprint arxiv: , [9] Z. M. Zhang, G. Ely, S. Aeron, N. Hao, and M. E. Kilmer. Novel methods for multilinear data completion and denoising based on tensor-svd. In CVPR, [10] H. Zou and R. Li. One-step sparse estimates in nonconcave penalized likelihood models. Annals of statistics, 36(4):1509, 2008.

4 Table 1. The performance comparison of 7 competing TC methods with different sampling rates on balloons image. MC-ALM [3] HaLRTC [4] t-svd [9] TMac [8] RP-TC trace RP-TC mcp RP-TC scad Table 2. The performance comparison of 7 competing TC methods with different sampling rates on beads image. MC-ALM [3] HaLRTC [4] t-svd [9] TMac [8] RP-TC trace RP-TC mcp RP-TC scad Table 3. The performance comparison of 7 competing TC methods with different sampling rates on cloth image. MC-ALM [3] HaLRTC [4] t-svd [9] TMac [8] RP-TC trace RP-TC mcp RP-TC scad Table 4. The performance comparison of 7 competing TC methods with different sampling rates on jellybeans image. MC-ALM [3] HaLRTC [4] t-svd [9] TMac [8] RP-TC trace RP-TC mcp RP-TC scad

5 Table 5. The performance comparison of 7 competing TC methods with different sampling rates on peppers image. MC-ALM [3] HaLRTC [4] t-svd [9] TMac [8] RP-TC trace RP-TC mcp RP-TC scad Table 6. The performance comparison of 7 competing TC methods with different sampling rates on watercolors image. MC-ALM [3] HaLRTC [4] t-svd [9] TMac [8] RP-TC trace RP-TC mcp RP-TC scad Table 7. The performance comparison of 7 competing TC methods with different sampling rates on Scene 4 of Natural Scenes MC-ALM [3] HaLRTC [4] t-svd [9] TMac [8] RP-TC trace RP-TC mcp RP-TC scad Table 8. The performance comparison of 7 competing TC methods with different sampling rates on Scene 8 of Natural Scenes MC-ALM [3] HaLRTC [4] t-svd [9] TMac [8] RP-TC trace RP-TC mcp RP-TC scad

6 Table 9. The performance comparison of 7 competing TRPCA methods on balloons image. Noisy image TDL [5] RPCA [3] HoRPCA [1] t-svd [9] RP-TRPCA trace RP-TRPCA mcp RP-TRPCA scad Table 14. The performance comparison of 7 competing TRPCA methods on feathers image. Noisy image TDL [5] RPCA [3] HoRPCA [1] t-svd [9] RP-TRPCA trace RP-TRPCA mcp RP-TRPCA scad Table 10. The performance comparison of 7 competing TRPCA methods on beads image. Noisy image TDL [5] RPCA [3] HoRPCA [1] t-svd [9] RP-TRPCA trace RP-TRPCA mcp RP-TRPCA scad Table 15. The performance comparison of 7 competing TRPCA methods on flowers image. Noisy image TDL [5] RPCA [3] HoRPCA [1] t-svd [9] RP-TRPCA trace RP-TRPCA mcp RP-TRPCA scad Table 11. The performance comparison of 7 competing TRPCA methods on chart and stuffed toy image. Noisy image TDL [5] RPCA [3] HoRPCA [1] t-svd [9] RP-TRPCA trace RP-TRPCA mcp RP-TRPCA scad Table 16. The performance comparison of 7 competing TRPCA methods on photo and face image. Noisy image TDL [5] RPCA [3] HoRPCA [1] t-svd [9] RP-TRPCA trace RP-TRPCA mcp RP-TRPCA scad Table 12. The performance comparison of 7 competing TRPCA methods on cloth image. Noisy image TDL [5] RPCA [3] HoRPCA [1] t-svd [9] RP-TRPCA trace RP-TRPCA mcp RP-TRPCA scad Table 17. The performance comparison of 7 competing TRPCA methods on pompoms image. Noisy image TDL [5] RPCA [3] HoRPCA [1] t-svd [9] RP-TRPCA trace RP-TRPCA mcp RP-TRPCA scad Table 13. The performance comparison of 7 competing TRPCA methods on egyptian statue image. Noisy image TDL [5] RPCA [3] HoRPCA [1] t-svd [9] RP-TRPCA trace RP-TRPCA mcp RP-TRPCA scad Table 18. The performance comparison of 7 competing methods on watercolors image. Noisy image TDL [5] RPCA [3] HoRPCA [1] t-svd [9] RP-TRPCA trace RP-TRPCA mcp RP-TRPCA scad

7 (a) Original image (b) Noisy image (f) t-svd (c) TDL (g) RP-TRPCAtrace (d) RPCA (h) RP-TRPCAmcp (e) HoRPCA (i) RP-TRPCAs cad Figure 1. (a) The original image from chart and stuffed toy; (b) The corresponding noisy image; (c)-(i) The recovered images by TDL[5], RPCA [3], HoRPCA [1], t-svd [9], RP-TRPCAtrace, RP-TRPCAmcp and RP-TRPCAscad, respectively. (a) Original image (b) Noisy image (f) t-svd (g) RP-TRPCAtrace (c) TDL (d) RPCA (h) RP-TRPCAmcp (e) HoRPCA (i) RP-TRPCAs cad Figure 2. (a) The original image from feathers; (b) The corresponding noisy image; (c)-(i) The recovered images by TDL[5], RPCA [3], HoRPCA [1], t-svd [9], RP-TRPCAtrace, RP-TRPCAmcp and RP-TRPCAscad, respectively.

8 (a) Original image (b) Noisy image (f) t-svd (c) TDL (g) RP-TRPCAtrace (d) RPCA (h) RP-TRPCAmcp (e) HoRPCA (i) RP-TRPCAs cad Figure 3. (a) The original image from flowers; (b) The corresponding noisy image; (c)-(i) The recovered images by TDL[5], RPCA [3], HoRPCA [1], t-svd [9], RP-TRPCAtrace, RP-TRPCAmcp and RP-TRPCAscad, respectively. (a) Original image (b) Noisy image (f) t-svd (g) RP-TRPCAtrace (c) TDL (d) RPCA (h) RP-TRPCAmcp (e) HoRPCA (i) RP-TRPCAs cad Figure 4. (a) The original image from photo and face; (b) The corresponding noisy image; (c)-(i) The recovered images by TDL[5], RPCA [3], HoRPCA [1], t-svd [9], RP-TRPCAtrace, RP-TRPCAmcp and RP-TRPCAscad, respectively.