Surrogate Functional Based Subspace Correction Methods for Image Processing

Transcription

1 SpezalForschungsBerech F 3 Karl Franzens Unversta t Graz Technsche Unversta t Graz Medznsche Unversta t Graz Surrogate Functonal Based Subspace Correcton Methods for Image Processng M. Hntermueller A. Langer SFB-Report No December 0 A 800 GRAZ, HEINRICHSTRASSE 36, AUSTRIA Supported by the Austran Scence Fund (FWF)

2 SFB sponsors: Austran Scence Fund (FWF) Unversty of Graz Graz Unversty of Technology Medcal Unversty of Graz Government of Styra Cty of Graz

3 Surrogate Functonal Based Subspace Correcton Methods for Image Processng Mchael Hntermüller and Andreas Langer Introducton Recently n [3, 4, 5] subspace correcton methods for non-smooth and non-addtve problems have been ntroduced n the context of mage processng, where the nonsmooth and non-addtve total varaton (TV) plays a fundamental role as a regularzaton technque, snce t preserves edges and dscontnutes n mages. We recall, that for u L (Ω), V(u,Ω) := sup { Ω udvφdx : φ [C c(ω)], φ } s the varaton of u. In the event that V(u,Ω)< we denote Du (Ω)= V(u,Ω) and call t the total varaton of u n Ω []. In ths paper, as n [5], we consder functonals, whch consst of a non-smooth and non-addtve regularzaton term and a weghted combnaton of anl -term and a quadratcl -term; see () below. Ths type of functonal has been shown to be partcularly effcent to elmnate smultaneously Gaussan and salt-and-pepper nose. In [5] an estmate of the dstance of the lmt pont obtaned from the proposed subspace correcton method to the global mnmzer s establshed. In that paper the exact subspace mnmzaton problems are mnmzed, whch are n general not easly solved. Therefore, n the present paper we analyse a subspace correcton approach n whch the subproblems are approxmated by so-called surrogate functonals, as n [3, 4]. In ths stuaton, as n [5], we are able to acheve an estmate for the dstance of the computed soluton to the real global mnmzer. Wth the help of ths estmate we show n our numercal experments that the proposed algorthm generates a sequence whch converges to the expected mnmzer. Notatons For the sake of brevty we consder a two dmensonal settng only. We defne Ω = {x <...<x N } {y <...<y N } R, and H =R N N, where N N. For u H we wrte u = u(x) = u(x,y j ), where, j {,...,N} and x Ω. Let h = x + x = y j+ y j be the equdstant step-sze. We defne the scalar product of u,v H by u,v H = h x Ω u(x)v(x) and the scalar product of p,q H by p,q H = h x Ω p(x),q(x) R wth z,w R = j= z jw j for every z=(z,z ) R and w= Department of Mathematcs, Humboldt-Unversty of Berln, Unter den Lnden 6, 0099 Berln, Germany hnt@math.hu-berln.de Insttute for Mathematcs and Scentfc Computng, Unversty of Graz, Henrchstraße 36, A-800 Graz, Austra andreas.langer@un-graz.at

4 Mchael Hntermüller and Andreas Langer (w,w ) R. We also use u l p (Ω)= ( h x Ω u(x) p) /p, p<, u l (Ω)= sup x Ω u(x) and, when any norm can be taken. The dscrete gradent u s denoted by ( u)(x) = (( u) (x),( u) (x)) wth ( u) (x) = h (u(x +,y j ) u(x,y j )) f < N and ( u) (x) = 0 f = N, and ( u) (x) = h (u(x,y j+ ) u(x,y j )) f j < N and ( u) (x) = 0 f j = N, for all x Ω. For ω H we defne ϕ :R R by ϕ( ω )(Ω) := h x Ω ϕ( ω(x) ), where z = z + z. In partcular we defne the total varaton of u by settng ϕ(t) = t and ω = u,.e., u (Ω) := h x Ω u(x). For an operator T we denote by T ts adjont. Further we ntroduce the dscrete dvergence dv : H H defned by dv = ( s the adjont of the gradent ), n analogy to the contnuous settng. The symbol ndcates the constant vector wth entry values and D s the characterstc functon of D Ω. For a convex functonal J : H R, we defne the subdfferental of J at v H as the set valued mappng J(v) := /0 f J(v)= and J(v) :={v H : v,u v H + J(v) J(u) u H} otherwse. It s clear from ths defnton that 0 J(v) f and only f v s a mnmzer of J. Whenever the underlyng space s mportant, then we wrte V J or H J. 3 Subspace Correcton Approaches As n [5] we are nterested n mnmzng by means of subspace correcton the followng functonal J(u)=α S Su g S l (Ω) + α T Tu g T l (Ω) + ϕ( u )(Ω), () where S,T : H H are bounded lnear operators, g S,g T H are gven data, and α S,α T 0 wth α S +α T > 0. We assume that J s bounded from below and coercve,.e., {u H : J(u) C} s bounded n H for all constants C > 0, n order to guarantee that () has mnmzers. Moreover we assume that (A ϕ ) ϕ :R R s a convex functon, nondecreasng nr + wth () ϕ(0)=0. () cz b ϕ(z) cz+b, for all z R + for some constant c>0 and b 0. Note that for the partcular example ϕ(t)= t, the thrd term n () becomes the well-known total varaton of u n Ω and we call () the L -L -TV model. Now we seek to mnmze () by decomposng H nto two subspaces V and V such that H = V +V. Note that a generalzaton to multple splttngs can be performed straghtforwardly. However, here we wll restrct ourselves to a decomposton nto two domans only for smplcty. By V c we denote the orthogonal complement of V n H and we defne by π V c the correspondng orthogonal projecton onto V c for =,.

5 Surrogate Functonal Based Subspace Correcton Methods for Image Processng 3 Wth ths splttng we want to mnmze J by sutable nstances of the followng alternatng algorthm: Choose an ntal u (0) =: u (0) + u (0) V +V, for example, u (0) = 0, and terate u (n+) = arg mn u V J(u + u (n) ), u (n+) = arg mn J(u (n+) u V + u ), u (n+) := u (n+) + u (n+). Dfferently from the case n [5], where the authors solved the exact subspace mnmzaton problem n (), whch s n general not always easly manageable, we suggest now to approxmate the subdoman problems by so-called surrogate functonals (cf. [, 3, 4]): Assume a,u V, u V, and defne J s ( (u + u,a+u ) := J(u + u )+α T δ u + u (a+u ) l (Ω) (3) T(u + u (a+u )) ) l (Ω) ) = J(u + u )+α T (δ u a l (Ω) T(u a) l (Ω) for =, and {,}\{}, where δ > T. Then an approxmate soluton to mn u V J(u + u ) s realzed by usng the followng algorthm: For u (0) V, u (l+) = arg mn J s (u + u,u (l) u V + u ), l 0, where u V for =, and {,}\{}. The alternatng doman decomposton algorthm reads then as follows: Choose an ntal u (0) =: ũ (0) + ũ (0) V +V, for example, u (0) = 0, and terate u (n+,0) = ũ (n), u (n+,l+) = arg mn u V J s (u + ũ (n),u(n+,l) u (n+,0) = ũ (n), + ũ (n) ), l=0,...,l, u (n+,m+) = arg mn J s (u (n+,l) u V + u,u (n+,m) + u (n+,l) ), m=0,...,m, u (n+) := u (n+,l) + u (n+,m), ũ (n+) = χ u (n+), ũ (n+) = χ u (n+), (4) where χ, χ H have the propertes () χ + χ = and () χ V for =,. Let κ := max{ χ, χ }<. The parallel verson of the algorthm n (4) reads as follows: Choose an ntal u (0) =: ũ (0) + ũ (0) V +V, for example, u (0) = 0, and terate ()

6 4 Mchael Hntermüller and Andreas Langer u (n+,0) = ũ (n), u (n+,l+) = arg mn J s (u + ũ (n) u V,u(n+,l) + ũ (n) ), l=0,...,l, u (n+,0) = ũ (n), u (n+,m+) = arg mn J s (ũ (n) u V + u,u (n+,m) + ũ (n) ), m=0,...,m, (5) u (n+) := u(n+,l) ũ (n+) = χ u (n+), ũ (n+) = χ u (n+). +u (n+,m) +u (n), Note that we prescrbe a fnte number L and M of nner teratons for each subspace, respectvely. Hence we do not get a mnmzer of the orgnal subspace mnmzaton problems n (), but approxmatons of such mnmzers. Moreover, observe that u (n+) = ũ (n+) + ũ (n+), wth u (n+) ũ (n+), for =,, n general. 3. Convergence Propertes In ths secton we state convergence propertes of the subspace correcton methods n (4) and (5). In partcular, the followng three propostons are drect consequences of statements n [3, 4, 5]. Proposton. The algorthms n (4) and (5) produce a sequence (u (n) ) n n H wth the followng propertes:. J(u (n) )>J(u (n+) ) for all n N (unless u (n) = u (n+) );. lm n u (n+,l+) u (n+,l) l (Ω) = 0 and lm n u (n+,m+) 0 for all l {0,...,L } and m {0,...,M }; 3. lm n u (n+) u (n) l (Ω) = 0; 4. the sequence (u (n) ) n has subsequences that converge n H. u (n+,m) l (Ω) = The proof of ths proposton s analogous to the one n [3, Proposton 5.4] and [4, Theorem 5.]. Proposton. The sequences (ũ (n) ) n for =, generated by the algorthm n (4) or (5) are bounded n H and hence have accumulaton ponts ũ ( ), respectvely. Proof. By the boundedness of the sequence (u (n) ) n we obtan ũ (n) = χ u (n) κ u (n) C< and hence (ũ (n) ) n s bounded for =,. Remark. From the prevous proposton t drectly follows by the coercvty assumpton on J that the sequences (u (n,l) ) n and (u (n,m) ) n are bounded for all l {0,...,L} and m {0,...,M}.

7 Surrogate Functonal Based Subspace Correcton Methods for Image Processng 5 Proposton 3. Let u ( ), u ( ), and ũ ( ) be accumulaton ponts of the sequences (u (n,l) ) n, (u (n,m) ) n, and (ũ (n) ) n generated by the algorthms n (4) and (5), then u ( ) = ũ ( ), for =,. One shows ths statement analogous to the frst part of the proof of [3, Theorem 5.7]. Moreover, as n [5] we are able to establsh an estmate of the dstance of the lmt pont obtaned from the subspace correcton method to the true global mnmzer. Theorem. Let u be a mnmzer of J and u ( ) an accumulaton pont of the sequence (u (n) ) n generated by the algorthm n (4) or (5). Then we have that ether. u ( ) s a mnmzer of J or. there exsts a constant ε > 0 such that for α T δ < ε we have u ( ) u l (Ω) ˆη l (Ω) ε α T δ, where ˆη l (Ω) = mn{ η( ) l (Ω), η( ) l (Ω)} and η( ) s an accumulaton pont of the sequence (η (n) ) n for =,. In partcular, under the addtonal assumptons that α T > 0 and T T s postve defnte wth smallest Egenvalue σ > 0, then we have for δ < σ. Proof. We have that u (n+,l) u u ( ) l (Ω) ˆη l (Ω) α T (σ δ), = arg mn J s (u + ũ (n) u V,u(n+,L ) + ũ (n) ) { = argmn J s (u+ũ (n) u H,u(n+,L ) + ũ (n) ) : π V }. cu=0 Then, by [6, Theorem..4, p. 305] there exsts an η (n+) Range(π V c ) V c such that 0 H J s ( +ũ (n),u(n+,l ) + ũ (n) )(u(n+,l) )+η (n+). (6) Snce (u (n+,l) ) n, (u (n+,l ) ) n, and (ũ (n) ) n are bounded and based on the fact that J s (ξ, ξ) s compact for any ξ, ξ H we obtan that (η (n) ) n s bounded, cf. [5, Corollary 4.7]. By notng that (u (n+,l) ) n and (u (n+,l ) ) n have the same lmt for n, see Proposton, we subtract a sutable subsequence(n k ) k wth lmts η ( ), u ( ), and ũ ( ) such that (6) s stll vald, cf. [7, Theorem 4.4, p 33],.e., 0 H J ( +ũ ( ),u ( ) + ũ ( ) )(u ( ) )+η ( ). By the defnton of the subdfferental and Proposton 3 we obtan

8 6 Mchael Hntermüller and Andreas Langer J(u ( ) )=J s (u ( ),u ( ) ) J s (v,u ( ) )+ η ( ),u ( ) v H J s (v,u ( ) )+ η ( ) l (Ω) u( ) v l (Ω) for all v H. Smlarly one can show that J(u ( ) ) J s (v,u ( ) )+ η ( ) l (Ω) u( ) for all v H, and hence we have v l (Ω) J(u ( ) ) J s (v,u ( ) )+ ˆη l (Ω) u( ) v l (Ω) for all v H, where ˆη l (Ω) = mn{ η( ) l (Ω), η( ) l (Ω) }. Let u argmn u H J(u). Then there exsts a ρ 0 such that J(u ( ) )=J(u )+ρ.. If ρ = 0, then mmedately follows that u ( ) s a mnmzer of J.. If ρ > 0, then snce u ( ) u l (Ω) β <+, there exsts an ε > 0 such that εβ ρ and hence we get J(u ( ) ) J(u )+ε u ( ) u l (Ω). Settng v=u above and usng the last nequalty we obtan ) α T (δ u u ( ) l (Ω) T(u u ( ) ) l (Ω) + ˆη l (Ω) u( ) u l (Ω) (7) ε u ( ) u l (Ω). From the latter nequalty we get ˆη (ε α T δ) u ( ) u l (Ω) and consequently for α T δ < ε ˆη l (Ω) ε α T δ u( ) u l (Ω). If addtonally α T > 0 and T T s symmetrc postve defnte wth smallest Egenvalue σ > 0, then we get that ε = α T σ, cf. [5]. Then (7) reads as follows α T (σ δ) u u ( ) l (Ω) + α T T(u u ( ) ) l (Ω) ˆη l (Ω) u( ) u l (Ω) and by usng once more the symmetrc postve defnteness assumpton on T T we obtan from the latter nequalty that α T (σ δ) u u ( ) l (Ω) ˆη l (Ω) u( ) u l (Ω). If σ > δ then we get u u ( ) l (Ω) ˆη l (Ω) α T (σ δ). 4 Numercal Experments We present numercal experments obtaned by the parallel algorthm n (5) for the applcaton of mage deblurrng,.e., S = T are blurrng operators and ϕ( u )(Ω) = u (Ω) (the total varaton of u n Ω). The mnmzaton problems of the subdo-

9 Surrogate Functonal Based Subspace Correcton Methods for Image Processng 7 (a) (b) Fg. Image of sze pxels whch s corrupted by Gaussan blur wth kernel sze 5 5 pxels and standard devaton, 4% salt-and-pepper nose, and Gaussan whte nose wth zero mean and varance 0.0. In (a) decomposton of the spatal doman nto strpes and n (b) nto wndows. mans are mplemented n the same way as descrbed n [5] by notng that the functonal to be consdered n each subdoman s now the strctly convex functonal n (3). We consder an mage of sze pxels whch s corrupted by Gaussan blur wth kernel sze 5 5 pxels and standard devaton. Addtonally 4% saltand-pepper nose (.e., 4% of the pxels are ether flpped to black or whte) and Gaussan whte nose wth zero mean and varance 0.0 s added. In order to show the effcency of the parallel algorthm n (5) for decomposng the spatal doman nto subomans, we compare ts performance wth the L -L -TV algorthm presented n [5], whch solves the problem on all of Ω wthout any splttng. We consder splttngs of the doman n strpes, cf. Fgure (a), and n wndows as depcted n Fgure (b) for dfferent numbers of subdomans (D=4,6,64). The algorthms are stopped as soon as the energy J reaches a sgnfcance level J,.e., when J(u (n) ) J for the frst tme. For reason of comparson we expermentally choose J = ,.e., we once restored the mage of nterest untl we observed a vsually satsfyng restoraton and the assocated energy-value as J. In the subspace correcton algorthm as well as n the L -L -TV algorthm we restore the mage by settng α S = 0.5, α T = 0.4, and δ =.. The computatons are done n Matlab on a computer wth 56 cores and the multthreadng-opton s actvated. Table presents the computatonal tme and number of teratons the algorthms need to fulfll the stoppng crteron for dfferent number of subdomans. We clearly see that the doman decomposton algorthm for D=4,6,64 subdomans s much faster than the L -L -TV algorthm (D=). Snce a blurrng operator s n general non-local, n each teraton u (n) has been communcated to each subdoman. Therefore the communcaton tme becomes substantal for splttngs nto 6 or more domans such that the algorthm needs more tme to reach the stoppng crteron. In Fgure we depct the progress of the mnmal Lagrange multpler η (n) := mn { η (n) l (Ω)}, whch ndcates that the parallel algorthm ndeed converges to a mnmzer of the functonal J.

10 8 Mchael Hntermüller and Andreas Langer Table Restoraton of the mage n Fgure : Computatonal performance (CPU tme n seconds and the number of teratons) for the global L -L -TV algorthm and for the parallel doman decomposton algorthms wth α = 0.5, α = 0.4 for dfferent numbers of subdomans (D=4,6,64). # Domans wndow-splttng strpe-splttng D= (L -L -TV alg.): 944 s / 3 t D=4: 374 s / 7 t 340 s / 7 t D=6: 94 s / 7 t 98 s / 7 t D=64: 7833 s / 7 t 8797 s / 8 t Fg. The progress of the mnmal Lagrange multpler η (n). 4.5 x 0 3 Norm of the Lagrange multpler n frst doman Acknowledgements Ths work was supported by the Austran Scence Fund FWF through the START Project Y 305-N8 Interfaces and Free Boundares and the SFB Project F3 04-N8 Mathematcal Optmzaton and Its Applcatons n Bomedcal Scences as well as by the German Research Fund (DFG) through the Research Center MATHEON Project C8 and the SPP 53 Optmzaton wth Partal Dfferental Equatons. M.H. also acknowledges support through a J. Tnsely Oden Fellowshp at the Insttute for Computatonal Engneerng and Scences (ICES) at UT Austn, Texas, USA. References. Ambroso, L., Fusco, N., D., P.: Functons of Bounded Varaton and Free Dscontnuty Problems. Oxford Mathematcal Monographs. Oxford: Clarendon Press. xv, Oxford (000). Fornaser, M., Km, Y., Langer, A., Schönleb, C.B.: Wavelet decomposton method for l /tvmage deblurrng. SIAM J. Imagng Scences 5, (0) 3. Fornaser, M., Langer, A., Schönleb, C.B.: A convergent overlappng doman decomposton method for total varaton mnmzaton. Numer. Math. 6, (00) 4. Fornaser, M., Schönleb, C.B.: Subspace correcton methods for total varaton and l - mnmzaton. SIAM J. Numer. Anal. 47, (009) 5. Hntermüller, M., Langer, A.: Subspace correcton methods for non-smooth and non-addtve convex varatonal problems n mage processng. SFB-Report 0-0, Insttute for Mathematcs and Scentfc Computng, Karl-Franzens-Unversty Graz (0) 6. Hrart-Urruty, J.B., Lemaréchal, C.: Convex Analyss and Mnmzaton Algorthms I, Vol. 305 of Grundlehren der mathematschen Wssenschaften. Sprnger-Verlag, Berln (996) 7. Rockafellar, R.T.: Convex Analyss. Prnceton Unversty Press, Prncton, NJ (970)