A New Projected Gradient Algorithm for Total Variation Based Image Denoising

Transcription

1 Noname manuscript No. (will be inserted by te editor) A New Projected Gradient Algoritm or Total Variation Based Image Denoising Ming-Jun Lai Leopold Matamba Messi te date o receipt and acceptance sould be inserted later Abstract Tis paper is concerned wit te numerical approximation o te minimizer o te continuous Rudin-Oser- Fatemi (ROF) model. A new projected gradient algoritm or computing a numerical solution o te ROF model in te discrete setting is proposed. We sow tat te new algoritm converges. Te solution o our discrete algoritm is ten used to construct a continuous piecewise linear spline wic approximates te solution o te continuous ROF model in te L norm. We sow tat i te noised image is in te space Lip(, L ()), 0 < 1, our piecewise linear approximations converge to te solution o te ROF model. Finally, we demonstrate wit numerical experiments tat te new algoritm is as good as te traditional projected gradient algoritm and Cambolle s ixed point algoritm. 1 Introduction Since te seminal work o Rudin, Oser, and Fatemi[13] total variation based model or image restoration ave received a great deal o attention. Tey are now used in image denoising, image deblurring, and image inpainting. In eac case, te problem is ormulated as a minimization o a unctional o te orm argmin Du (), subject to te constraints F (u), (1.1) were Du () is te total-variation o te unction u on, and F (u) is a suitable set o constraints satisied by u, and R is te domain o te image u. For image de-noising, te problem was ormulated as ollows: recover te true image u rom a contaminated version = u + n, were n is a wite noise wit mean 0 and standard deviation σ. Te corresponding minimization problem is argmin Du () subject to te constraints (1.) u(x)dx = (x)dx and u(x) (x) dx σ. (1.3) It was sown (c. [6,13]) tat problem (1.) is equivalent to te ollowing unconstrained minimization argmin Du () + 1 (u ) dx (1.4) u BV () λ were λ > 0 is a Lagrange multiplier, and BV () is te Banac space o unctions o bounded variation. Te existence and uniqueness o te minimizer o te above problem was establised in [1] and [6]. Ming-Jun Lai Department o Matematics University o Georgia Atens, GA 3060, USA Tel.: Fax: mjlai@mat.uga.edu Leopold Matamba Messi Department o Matematics University o Georgia Atens, GA 3060, USA Tel.: Fax: lmatamba@mat.uga.edu

2 Ming-Jun Lai, Leopold Matamba Messi To ind numerical approximations o te solution o (1.4), one as to understand ow to discretize te total variation term Du (). All te inite dierence metods proposed in te literature are based on te ollowing act (see or example [,10] or details) ( ) ( ) u u Du () = u dx = + dx, u W 1,1 (). (1.5) x 1 x As suc, te total variation is ten approximated using a combination o a quadrature ormula or te integral, and a irst order inite dierence approximation o te derivative. Several algoritms to ind approximations o te solution o problem (1.4) ave been proposed and studied in te literature. See or example [3,6,13]. Indeed, a time dependent nonlinear PDE associated wit te Euler-Lagrange equation or (1.4) was used to approximate te minimizer o (1.4) and ten solved by inite dierence metods (c. [13]) or inite elements metods (c. [7]). In te discrete setting, a primal-dual algoritm (a popular approac in optimization) based on a straigtorward discretization o te TV term and L term was introduced by Cambolle in [?] or image denoising. Recently, Duval et al [8] proposed a projected-gradient algoritm or te discrete L TV regularization to compute an approximation o te minimizer. A numerical algoritm based on completely symmetric discretization o te TV term was studied in [15]. In particular, te researcers in [15] were able to sow tat teir numerical solution approximates te minimizer o problem (1.4) in L norm. Furtermore, te convergence o te discrete solution o te inite dierence metod or time dependent nonlinear PDE was recently establised in [11]. Tese motivate us to study te convergence o te discrete solutions o Cambolle algoritm and projected gradient algoritm to te solution o problem (1.4). Ater studying te convergence issue o tese two algoritms, we realize tat one as to use a new discretization o te total variation term wic leads to a new version o te projected-gradient algoritm. Wit tis new discretization, we are able to use te solution in te discrete setting to construct a piecewise linear interpolatory spline wic can be sown to approximate te solution o te original problem (1.4). Our contributions in tis paper are te ollowing: (1) we irst prove te existence and uniqueness o te solution based on te new discrete ROF unctional (see Teorem 1); () We ormulate a projected gradient algoritm or te new minimization problem (c. Algoritm 31) and prove its convergence (c. Teorem 3); (3) We demonstrate numerically tat our new version o te projected-gradient algoritm perorms sligtly better tan te original projected-gradient algoritm in [5,8], and is at least as good as Cambolle s ixed point iterative algoritm[4], as measured by te Peak Signal to Noise Ratio (PSNR) (see Section 5); (4) We obtain an error bound in L () between te piecewise linear interpolatory spline o te discrete solution and te solution o te continuous ROF model (see Teorem ). Preliminaries and Notations In tis section, we give preliminary results, and introduce te notations tat we sall use in te paper..1 Basic notations Let be an open subset o R, we denote te indicator unction o te set by 1, x, 1 (x) = 0, x /. For a given η R, we sall denote by τ η te image o te set under te translation wit te vector η, i.e τ η := x + η : x }. For a unction u : R, we denote by τ ηu te unction wose domain is τ η and is deined by τ ηu(x) = u(x + η), x τ η. Next, we recall te deinitions o some unctional spaces tat are used in tis work. Let 1 p < be ixed. L p () is te standard Banac space o p integrable unctions } L p () := u : R : u(x) p dx <. L p loc () is te set o unctions u tat are locally p integrable, i.e } L p loc u () = : R : u(x) p dx <, K compact. K

3 A New Projected Gradient Algoritm or Total Variation Based Image Denoising 3 Te norm o an element u L p () is given by ( 1/p u p = u(x) dx) p. It is well known tat te translation operator τ η is a bounded linear operator rom L p () into L p ( τ η ). Let > 0 be given. Te p modulus o continuity o order, o a unction u L p (), is deined by ω p(u, ) = τ ηu u p, (.1) η were η stands or te Euclidean norm o η. For u L p loc () and A a subset o wose closure Ā is a compact, te p modulus o continuity o u wit respect to A, denoted ω p(u, ) A, is given by ω p(u, ) A = ω p(u1 A, ). (.) Let 0 < 1. We denote Lip(, L ()) te subspace o L () deined by } Lip(, L ()) = u L () : ω (u, ) < >0.. A discretization o te ROF unctional. In te sequel sall denote te unit square [0, 1] o R. We subdivide into (N 1) square sub-domains o side lengt to get a uniorm quadrangulation. A triangulation is ten obtained rom by splitting eac square into two triangles using te Nortwest-Souteast diagonal as sown in Figure 1. ω i,j+1 ω i+1,j+1 T u i,j T d i,j ω i,j ω i+1,j Fig. 1: Type I triangulation o te domain wit vertexes ω i,j. T u i,j is te triangle wit vertexes ω i+1,j, ω i+1,j+1, ω i,j+1 and T d i,j is te triangle wit vertexes ω i,j, ω i+1,j, ω i,j+1. Let ω 1,1 be te lower let corner o. We denote te set o vertices o te triangulation by and deine a partition } o subordinate to V by V = ω 1,1 + Z } := ω i,j : 1 i, j N}, ( := ω i,j + [ /, /] ). Let E 1 λ (u) := Du () + (u ) dx. (.3) λ

4 4 Ming-Jun Lai, Leopold Matamba Messi Trougout te paper E λ (u) will be reerred to as te continuous ROF unctional. We are interested in devising a numerical sceme or computing an approximation o te minimizer o E λ (u). However, since Du is a measure, discretizing E λ (u) solely based on u is a delicate matter. Many researcers [5,1] ave used te closed orm o Du () given in (1.5) to designed discretization sceme o te ROF unctional E λ (u) tat combine a ormal inite dierence approximation o u wit a quadrature approximation o te integral. Our discretization o te ROF unctional E λ (u) as a similar lavor. We irst discretize te total variation as ollows J (u) := 1 i,j<n u i+1,j u i,j + u i,j+1 u i,j + 1 i,j<n u i+1,j+1 u i,j+1 + u i+1,j+1 u i+1,j, (.4) were u i,j is a suitable discretization o te unction u at te vertexes o te triangulation. For example i u is a continuous piecewise linear spline wit respect to, on can easily sow tat J (u) = Du (); tis is te reason wy we use (.4) to discretize Du (). Te resulting discrete ROF-unctional E λ, (u) is given by E λ, (u) := J (u) + 1 λ 1 i,j N ui,j i,j, u R N N (.5) were i,j := (ω i,j ) is a suitable discretization on o te datum. From tis point onwards, E λ, (u) will be reerred to as te discrete ROF unctional and te associated miniization problem sall be called te discrete ROF model. argmin E λ, (u) (.6) u R N N.3 Embedding and Projection Operators We introduce various operators tat will elp us construct a piecewise linear approximation o te solution to te continuous L ROF model. In te previous section, we gave a ormal discrete approximation o te ROF unctional. We now clariy ow tis discretization is obtained. Te sampling operator, Q : L () R N N, is deined by (Q ) i,j := 1 (x)dx. (.7) Q will also denote te projection o L () onto te linear space o piecewise constant unction wit respect to te partition i,j : 1 i, j N} o, in wic case Q is deined by Q (y) := (Q ) i,j = 1 (x)dx, or all y. (.8) Next, we denote by l (V ) te vector space R N N endowed wit te inner product u, v := N u i,j v i,j, (.9) and te corresponding norm u := u, u 1/. Te ollowing properties o Q ollows rom Jensen s Inequality. i,j=1 Lemma 1 Te sampling operator Q deined above as te ollowing properties Q, and Q, or all L (). (.10)

5 A New Projected Gradient Algoritm or Total Variation Based Image Denoising 5 Next, we explain two metods o constructing a unction o bounded variation rom an element u l (V ). First, we create a piecewise constant unction deined on via te partition : 1 i, j N} as ollows C u(y) = u i,j, i y. (.11) We denote by C : l (V ) BV () te piecewise constant embedding o l (V ) into BV () deined by (.11). Second, we deine a piecewise linear interpolatory operator. P : l (V ) BV (). For u l (V ), P u is te continuous piecewise linear polynomial deined by P u(y) = 1 i,j N wit φ i,j te continuous piecewise linear unction suc tat u i,j φ i,j (y), (.1) φ i,j (ω i,j ) = 1, and φ i,j (ω) = 0, ω V \ ω i,j. (.13) Wit a sligt abuse o notation, P will also denote te piecewise linear interpolation o unctions o class C k ( ), k 0 over te set V ; in wic case it will be deined by P (y) = 1 i,j N (ω i,j )φ i,j (y), y, C k ( ). Lemma Suppose tat is endowed wit te triangulation. Ten or all u l (V ), tere olds C u = u l (V ) and P u 3 u l (V ), (.14) P u C u ω(u, 1) l (V ) (.15) were ω(u, m) l (V ) denotes te discrete modulus o smootness o order m N and is deined by ω(u, m) l (V ) = =( 1, ) Z, 1 i,i+ 1 N max( 1, ) m 1 j,j+ N ui+1,j+ u i,j. (.16) Proo Let u l (V ) be given. Te inequalities (.14) ollow rom te deinition o te Euclidean norm on l (V ) and te act tat te area o te port o φ i,j is at most 3. Next we prove te inequality (.15). By deinition, we ave P u C u = P u(y) C u(y) dy 1 i,j N ( ) ul,k u i,j φl,k (y) dy 1 i,j N 1 l,k N ul,k u i,j φ l,k(y)dy 1 i,j N 1 l,k N i,j ul,k u i,j ω(u, 1) l (V ), 1 i,j N 1 l N, l i =1 1 k N, k j =1 wic completes te proo. Remark 1 Let u R N N be ixed, and let ω(u, m) be te modulus o continuity o u R N N wit respect to te Euclidian norm u = u, u. It is easy to see tat ω(u, m) l (V ) ω(u, m). (.17).4 Periodic Extension operators In tis section, we construct a periodic extension o u R N N to Z. We also present te construction o a periodic extension to R o a unction deined on.

6 6 Ming-Jun Lai, Leopold Matamba Messi Te discrete periodic extension Let u R N N be ixed. We construct a discrete unction Ext(u): Z R deined in two steps as ollows: 1) First, we extend u into an element u N R N N as ollows u N i,j = u i,j i 1 i, j N, u N i+1,j i i > N, 1 j N, u i,n j+1 i 1 i N, j > N, else. u N i+1,n j+1 (.18) ) Ext(u) is te periodic extension o u N to all o Z. In te sequel, we will denote by u N te restriction o te unction Ext(u) to te set Ψ N := (i, j) Z Z: 1 i, j N}. For = ( 1, ) Z and u := (u i,j ) i,j Z a discrete unction, we deine te translation τ u o u as ollows: (τ u) i,j = u i+1,j+, i, j Z. Te restriction o τ u to a set o te orm Ψ M := (i, j): 1 i, j M} sall be denoted u M. Te continuous periodic extension Te construction o te periodic extension tat ollows was presented in [16, 15]. Let u : R be given. First, we extend u to u : [0, ] R by te ormula u (x 1, x ) = u(x 1, x )1 (x 1, x ) + u( x 1, x )1 τ1,0(x 1, x ) + u(x 1, x )1 τ0,1(x 1, x ) + u( x 1, x )1 τ1,1(x 1, x ), (.19) were 1 A denotes te indicator unction o te set A, and τ m,n = (x 1 + m, x + n): (x 1, x ) }. Te extension o u is te periodic unction Ext(u) : R R wit period [0, ] tat coincides wit u on [0, ]. Since as compact closure, it is easy to sow tat te continous periodic extension operator, Ext, maps L p () into L p loc (R ). Proposition 1 Suppose tat L (). Ten or any 0 < 1, we ave were C > 0 is a constant independent o and 1 = ω (Ext(), ) 1 Cω (, ), (.0) (m,n) Z (m,n) 1 τ (m,n). Proo Let L () be given, and η R be ixed wit η. By deinition τ η(ext()1 1) Ext()1 1 = Ext()(x + η) Ext()(x) dx 1 τ η 1 1 i,j i,j 1 1 m 1 m i =1 1 n 1 n j =1 Ext()(x + η) Ext()(x) dx τ (m,n) τ (i,j) η Ext()(x + η) Ext()(x) dx τ (i,j) ( τ η) (x + η) (x) dx = 18 τ η. τ η As a consequence, we ave ω (Ext(), ) 1 3 ω (, ). Lemma 3 For any L () and 0 < 1, tere olds were K > 0 is a constant independent o. C Q K ω (, ), (.1)

7 A New Projected Gradient Algoritm or Total Variation Based Image Denoising 7 Proo By deinition, we ave C Q = = 1 i,j N 1 i,j N 1 i,j N 4 (x) 1 (y)dy dx 1/ ( ) 1/ 1 (x) (y) dy dx ( 4 z : z Ext()(x) Ext()(x + z) dz) dx } ( 4 z : z Ext()(x) Ext()(x + z) dz) dx } z : z } ( C Q 8πω (Ext(), ) 1. 1/ Ext()(x) Ext()(x + z) dx) dz Inequality (.1) ollows rom te latter inequality tanks to Proposition 1, and te act tat ω (Ext(), ) 1 ω (Ext(), ) 1. 1/ 1/ 3 A total variation based model or digital image denoising In te sequel we treat gray-scale images o size N N as rectangular matrices o dimension N N and denote by X := R N N te maniold o gray-scale images o size N N. 3.1 Te model and its properties To compute te total variation o elements o X, we introduce two discrete gradient operators + = ( x +, y + ) and = ( x, y ), wic are linear operators rom X into Y := X X, deined by ( x 0, i i = N or j = N +u) i,j = (3.1) u i+1,j u i,j oterwise; ( y + u) 0, i i = N or j = N i,j = (3.) u i,j+1 u i,j oterwise; and ( x 0, i i = 1 or j = 1 u) i,j = u i,j u i 1,j oterwise; ( y u) 0, i i = 1 or j = 1 i,j = u i,j u i,j 1 oterwise. (3.3) (3.4) We associate to te discrete gradient operators + and te discrete divergence operators, div + := + : Y X and div := : Y X, deined respectively by div + (p) i,j = + 0 i i = N or j = N p 1 i,j oterwise 0 i i = N or j = N p i,j oterwise 0 i i = 1 or j = N p 1 i 1,j oterwise 0 i i = N or j = 1 p i,j 1 oterwise

8 8 Ming-Jun Lai, Leopold Matamba Messi and div (p) i,j = + 0 i i = N or j = 1 p 1 i+1,j oterwise 0 i i = 1 or j = N p i,j+1 oterwise 0 i i = 1 or j = 1 p 1 i,j oterwise 0 i i = 1 or j = 1 p i,j oterwise Te total variation o a gray-scale image u X is ten deined by J(u) := 1 1 i N 1 j N ( ( + u) i,j + ( u) i,j ), (3.5) were or an element p := (p i,j ) := (p 1 i,j, p i,j) Y, pi,j is te Euclidean norm o p ij := (p 1 i,j, p i,j) in R. Consequently, te digital ROF-unctional is given by E 1 d,λ (u) := J(u) + λ u, (3.6) were X is te Euclidean norm on X = R N N, and X. Since te unctional E d,λ (u) is strictly convex and coercive, it ollows (c. [9, Proposition 1., page 35]) tat te minimization problem argmin E d,λ (u) as a unique solution. Henceort, we will reer to te problem u X argmin E d,λ (u) (3.7) u X as te digital ROF-model. Te next result, similar to te one proved in [15,16], sows tat problem (3.7) is stable under small perturbations in te datum. Lemma 4 (Stability) Let u be te minimizer o E d,λ (u) and ug te minimizer o Eg d,λ (u). Ten u u g g. (3.8). We now prove a tecnical lemma tat asserts tat te discrete ROF model is compatible wit translation o te datum Lemma 5 Let R N N be ixed and u be te minimizer o te unctional E d,λ (u). Ten, un is a minimizer o E N d,λ (u). Moreover, or any Z suc tat = 1, te minimizer o E N d,λ (u) is un,, te restriction o τu to Ψ N. E N u R N N Proo We irst sow tat u N = argmin d,λ (u). It is easy to see tat or all u RN N, we ave E N d,λ (un ) = 4E d,λ (u) Furtermore, or every u R N N, tere exists x u R N N suc tat E N N (u) E x u be te constant vector tat equals u i0,j 0 E N d,λ (xn u ). Consequently, N min E u R N N d,λ (u) = d,λ d,λ (xn u ). In eect, letting wit (i 0, j 0 ) argmin u i,j i,j, it is easy to veriy tat E N d,λ (u) 1 i,j N N min E u R N N d,λ (un ) = 4 min E u R N N d,λ (u). Hence, u N is te minimizer o E N d,λ (u) wit respect to RN N. Next let Z wit = 1. For clarity o te argument, we ix = (1, 0). An argument identical to te one above sows tat N min E u RN N d,λ (u) = min N E u RN N d,λ (un ). Let k N be ixed. Ten, or any periodic discrete unction g wit period [1, N] [1, N], we ave k E gn d,λ (un g ) = E gnk d,λ (unk g ) = min E g Nk u R Nk Nk d,λ (u), (3.9)

9 A New Projected Gradient Algoritm or Total Variation Based Image Denoising 9 were u g is te minimizer o E gn d,λ (u), and unk g is te restriction o te periodic (wit period [1, N] [1, N]) extension o u g to [1, Nk] [1, Nk]. Moreover, or any u R N N, tere olds E gnk d,λ (unk ) = E gnk d,λ (unk ) kd(u), (3.10) were D(u) = ( + (u) 1,j + (u) 1,j ). (3.11) 1 j<n E N E N d,λ u R N N Let y R N N be suc tat y N = argmin d,λ (yn we ave (u). We want to prove tat yn ). Let us consider te minimization problem associated to Nk. Since u N k E N d,λ (yn k E N d,λ (yn ) = E Nk d,λ (ynk ) Dividing te last inequality above by k, we obtain E N d,λ (yn = E Nk d,λ (ynk ) kd(y) by equation (3.10) E Nk d,λ (unk ) kd(y) by equation (3.9) E Nk d,λ (unk, ) + k(d(u ) D(y)) by equation (3.10) ) k E N d,λ (un,) + k(d(u ) D(y)) by equation (3.9). ) E N d,λ (un Passing to te limit as k in (3.1) yields E N = u N N,, i.e E d,λ (un, ) is te minimizer o E N λ (u),,) + (D(u ) D(y)), k N, k 1. (3.1) k d,λ (yn ) E N d,λ (un, E N u R N N ). Tus, un, = argmin ). d,λ (un Our argument above works mutatis mutandis or any := ( 1, ) wit one o te components being zero. For (1, 1), (1, 1), ( 1, 1), ( 1, 1)}, te proo ollows rom te previous case by observing tat τ = τ (1,0) τ (0,). Remark We observe tat or any u R N N E λ, (u) = E / d,λ/ (u/). (3.13) Tereore, Lemma 4 and Lemma 5 remain valid or te unctional E λ, (u). Te equation (3.13) gives te relation between te discrete ROF unctional (.5) and te digital ROF unctional (3.6). 3. Primal-Dual ormulation We establis te primal-dual ormulation o te minimization problem associated to te unctional E d,λ (u). We also prove te existence and uniqueness o te minimizer. First, we establis an alternate ormula or te discrete total variation J(u) deined in (3.5). By Riesz representation Teorem, we can rewrite te discrete total variation J(u) above in te ollowing orm: J(u) = 1 = 1 = 1 ( + u) i,j, p i,j + ( u) i,j, q i,j 1 i N p i,j R, p i,j 1 q i,j R, q i,j 1 1 j N + u, p Y + 1 u, q p Y, p 1 Y q Y, q 1 div + (p) X + 1 u, div (q) p Y, p 1 u, X q Y, q 1 J(u) = p Y, p 1 q Y, q 1 u, 1 div +(p) + 1 div (q) X (3.14)

10 10 Ming-Jun Lai, Leopold Matamba Messi were or p = (p 1, p ) and q = (q 1, q ) in Y, p, q Y = 1 i,j N p 1 i,jqi,j 1 + p i,jqi,j, and p = max (p 1 i,j ij ) + (p ij ). Tereore, te minimization problem (3.7) is equivalent to te ollowing saddle-point problem argmin u X p Y, p 1 q Y, q 1 u, 1 div +(p) + 1 div (q) X + 1 λ u. (3.15) Te saddle-point problem (3.15) above is reerred to as te primal-dual ormulation o problem (3.7), wit primal variable u and dual variable p. Let and L(u : p, q) be te unctional deined on X Y Y by B Y = p Y : p 1}, L (u; p, q) := u, 1 div +(p) + 1 div (q) + 1 λ u. (3.16) We ave te ollowing result or te existence o a solution to problem (3.15). Lemma 6 Te unctional L : X Y Y R deined by (3.16) as a saddle point (ū; p, q) in te set X B Y B Y. Furtermore te component ū o any saddle point o L is te solution o te minimization problem (3.7). Proo We observe tat L(u; p, q) is a strictly convex quadratic unction in u and linear in (p, q). Moreover, bot te mappings u L(u; p, q) and (p, q) L(u; p, q) are Gâteaux-dierentiable, wit derivatives ul(u; p, q) = 1 (div +(p) + div (q)) 1 (u ), (3.17) [ λ 1 p,ql(u; p, q) = +u, 1 ] u. (3.18) Let ū = + λ (div +( p) + div ( q)), (3.19) wit ( p, q) argmin p,q B Y λ (div + (p) + div (q)) +. (3.0) We sow tat (ū; p, q) is a saddle-point or L(u; p, q) wit respect to te set X B Y B Y. To tis end, it suices to ceck tat ul(ū; p, q), u ū 0, u X p,ql(ū; p, q), (p p, q q) 0, p, q B Y. wic ollows by deinition o ū and ( p, q). In eect, by deinition o ū, we ave ul(ū; p, q) = 0. Tus, Next, it is easy to see tat or all p, q Y we ave ul(ū; p, q), u ū = 0, u X. p,ql(ū; p, q), (p p, q q) = 1 λ λ(div + p + div q) +, λ(div + (p p) + div (q q)). Now, we recall tat λ(div + ( p) + div ( q)) is te ortogonal projection o onto te closed convex set λk. Consequently, by te caracterization o te ortogonal projection, we ave λ(div + p + div q) +, λ(div + (p p) + div (q q)) 0, p, q B Y. Hence, by (3.18) p,ql(ū; p, q), (p p, q q) 0 or all p, q B Y.

11 A New Projected Gradient Algoritm or Total Variation Based Image Denoising 11 Remark 3 I (ū; p, q) is a saddle-point o L(u; p, q) wit respect to te set X B Y B Y, ten L(ū; p, q) = in J(u) + 1 u X λ u = J(ū) + 1 ū λ = max p,q B Y λ λ 8 div +(p) + div (q) + λ. Hence, or any saddle-point (ū; p, q) o L(u; p, q), we ave ( p, q) argmin p,q B Y λ (div + (p) + div (q)) +. We ave proved te ollowing caracterization o a minimizer o te discrete ROF unctional E d,λ. Teorem 1 A vector u X is te solution o problem (3.7) i and only i (u ) is te ortogonal projection o onto te closed convex set λk := λ [div + (p) + div (q)] : p, q B Y }. (3.1) Furtermore, te solution u is given by wit ( p, q) argmin p,q B Y λ (div + (p) + div (q)) +. u = + λ (div +( p) + div ( q)), For a closed convex subset A o a Euclidean space, we denote by π A te ortogonal projection onto A. In particular or A = B Y, it can be sown tat ( ) p 1 i,j π BY (p) i,j = max(1, p i,j ), p i,j, 1 i, j N. (3.) max(1, p i,j ) Te ormula above ollows rom te observation tat B Y is te cartesian product o unit disks o R, and R can be isometrically embedded in Y, wit bot spaces endowed wit teir natural Euclidian norms. Tereore, or 1 i, j N, π BY (p) i,j is te ortogonal projection o p i,j = (p 1 i,j, p i,j) onto te unit disk o R, given by (3.). Te ollowing result is a straigtorward consequence o te caracterization o minimizers o a Gâteaux-dierentiable unction and orms te basis o our projected-gradient algoritm. Proposition I ( p, q) argmin λ (div + (p) + div (q)) +, ten or any τ > 0, tere olds p,q B Y p = π BY ( p + τ + [div + ( p) + div ( q) + /λ]), q = π BY ( q + τ [div + ( p) + div ( q) + /λ]). (3.3) Proo Let ( p, q) argmin p,q B Y λ(div + (p) + div (q) + ) be ixed. Ten, p argmin p B Y div + (p) + div ( q) + /λ, q argmin q B Y div + ( p) + div (q) + /λ). We notice tat te mappings p div + (p) + div ( q) + /λ and q div + ( p) + div (q) + /λ are Gâteauxdierentiable wit dierentials + (div + (p) + div ( q) + /λ) and (div + ( p) + div (q) + /λ), respectively. Tereore (see [9, Proposition.1, page 37]), we ave It ten ollows tat or any τ > 0 + (div + ( p) + div ( q) + /λ), p p 0, p B Y, (div + ( p) + div ( q) + /λ), q q 0, q B Y. p [ p + τ + (div + ( p) + div ( q) + /λ)], p p 0, p B Y, q [ q + τ (div + ( p) + div ( q) + /λ)], q q 0, q B Y, wic is equivalent to (3.3).

12 1 Ming-Jun Lai, Leopold Matamba Messi 3.3 A projected gradient algoritm By Teorem 1, computing te minimizer o te discrete ROF unctional E d,λ (u) is equivalent to projecting onto te set λk deined by (3.1). In view o Proposition, we propose a projected gradient algoritm or computing te projection o onto λk. We prove te convergence o te proposed algoritm by adapting te argument used in [8]. Algoritm 31 (Projected-Gradient) Let τ > 0 be ixed. For n = 0, let p 0 = q 0 = 0. Step 1: Compute u n u n = + λ [div +(p n) + div (q n)]. (3.4) Step : Compute p n+1 and q n+1 p n+1 = π BY (p n + τ + [div + (p n) + div (q n) + /λ]), q n+1 = π BY (q n + τ [div + (p n) + div (q n) + /λ]), (3.5) wit π BY given by te equation (3.). Step 3: Increment n by 1, ten go back to step 1. We ave te ollowing convergence result or Algoritm 31. Proposition 3 I 0 < τ < 1/8, ten Algoritm 31 converges. More precisely, or any ( p, q) satisying (3.0), we ave lim n (div +(p n) + div (q n)) = div + ( p) + div ( q). Proo Let ( p, q) satisying (3.3) be ixed and τ > 0 be given. Since te projection mapping π BY iner rom (3.3) tat p n+1 p + q n+1 q Id τa ( p n p + q n q ), is non-expansive, we were A : Y Y Y Y is te linear operator ( ) + div A := + + div. div + div Consequently, as long as τ > 0 is suc tat κ := Id τa = 1, we get ( ) ( ) pn+1 p pn p q n+1 q, n 0. (3.6) q n q Next, we sow tat κ = 1 or 0 < τ < 1/8. We note to begin tat A is sel-adjoint and nonnegative deinite. Indeed or p, q Y, we ave A ( ) ( ) p p, q q Y Y = + (div + (p) + div (q)), p Y + (div + (p) + div (q)), q Y = div + (p) + div (q), div + (p) X + div + (p) + div (q), div (q) X = div + (p) + div (q), div + (p) + div (q) X 0. Tus, Y Y = ker(a) F,were F is te closure i Im(A). Moreover, all o te eigenvalues o A are nonnegative and κ = max(1, 1 τ A ), were A denotes te spectral norm o A (te largest eigenvalue o A). So, or any τ suc tat 0 < τ / A, we ave Id τa = 1. Now, we compute an upper-bound or A. We recall tat by deinition A = (p,q) (0,0) A ( ) p, A q ( ) p q Y Y p + q.

13 A New Projected Gradient Algoritm or Total Variation Based Image Denoising 13 By Caucy-Scwarz inequality and te deinition o te norm o a linear operator, we ave ( ) ( ) p p A, A q q A = Tus, it is te case tat (p,q) (0,0) p + q Y Y ( + + ) div + (p) + div (q) (p,q) (0,0) p + q ( + + ) (p,q) (0,0) ( + + ) max( div +, div ) By deinition o div +, or all u X and all p Y div + p + div + div p q + div q p + q A ( + + ) max( div +, div ). (3.7) + u, + u Y = u, div + ( + u) X div + (p), div + (p) X = p, + (div + (p)) Y. Consequently, by Caucy-Scwarz inequality and te deinition o te norm o a linear operator, we obtain + div + + and div + + div + wic sows tat + = div +. A similar argument sows tat = div. Now, or u X, + u = (u i+1,j u i,j ) + (u i,j+1 u i,j ) 1 i,j N 1 1 i,j N 1 u i+1,j + u i,j + u i,j+1 8 u. Hence, + 8. A similar argument sows tat div = 8. Tus, by (3.7) we ave tat A 16. (3.8) Tereore, or any 0 < τ < 1/8, tere olds te inequality κ = Id τ A = 1. We now sow tat te sequence (div + (p n) + div (q n)) n 0 converges by sowing tat all o its convergent subsequences ave te same limit. Let (div + (p nk ) + div (q nk )) k 0 be a convergent subsequence. Tere exists a urter subsequence, wic we denote again by n k, suc tat p nk p and q nk q. Since te projection π BY is non-expansive, it ollows rom (3.5) and (3.6) tat p nk +1 p, q nk +1 q, and ) ) (p p = ( p p q q, q q were is te Euclidean norm on Y Y induced by te Euclidean norm o Y. As a consequence o te above equality and te equation p = π BY ( p + τ + (div + ( p) + div ( q))), we ave But te latter inequality is nonsense. Tus, q = π BY ( q + τ (div + ( p) + div ( q))), ) ( p p ) = (p p ) q q ( p p q q (Id τa) q q ) ) ( p p ( p + 1 τ A p q q q q ker(a) F ) < ( p p ) ( p p i / ker(a) and 0 < τ < 1/8. q q q q ) ( p p ker(a), or equivalently, q q div + ( p p) + div ( q q) ker( + ) ker( )

14 14 Ming-Jun Lai, Leopold Matamba Messi Indeed, by deinition o te divergence operators div + ( p p) + div ( q q) = p p, + (div + ( p p) + div ( q q)) Y q q, (div + ( p p) + div ( q q)) Y = p p, 0 Y q q, 0 Y = 0. Hence, div + ( p p) + div ( q q) = 0, or equivalently, div + ( p) + div ( q) = div + ( p) + div ( q). We ave sown tat any convergent subsequence o (div + (p n) + div (q n)) n 0 converges to div + ( p) + div ( q), tus te sequence (div + (p n) + div (q n)) n 0 converges to div + ( p) + div ( q). Remark 4 One may obtain an alternating version o Algoritm 31 by modiying te second step as ollows p n+1 = π BY (p n + τ + [div + (p n) + div (q n) + /λ]), q n+1 = π BY (q n + τ [div + (p n+1 ) + div (q n) + /λ]). (3.9) Wile te proo o convergence o te alternating version o Algoritm 31 is still eluding us, te numerical experiments suggest tat one sould be able to prove a convergence result or 0 < τ < 1/4. 4 Numerical Approximation o te continuous ROF model In tis section, we study ow to approximate te minimizer o te continuous ROF model u = argmin E λ (u), (4.1) u BV () were E λ (u) is deined in (.3) Our approac consists in extending te minimizer z, o te discrete ROF unctional (.5) associated wit Q into a continuous piecewise linear polynomial P z, wit respect to te triangulation. We sall obtain an error bound in L () between P z, and u. To begin, we recall te ollowing standard act about te ROF unctional (c. [16,15]), and provide a proo or convenience. Lemma 7 Let u BV () be te minimizer o te unctional E λ (u) deined by (.3). Ten, or any v BV (), tere olds ( ) v u λ E λ (v) E λ (u ). (4.) Moreover, E λ (u ) = 1 λ ( u ). (4.3) Proo Since u is te minimizer o E λ (u) and E λ is convex, it ollows tat 0 E λ (u ). Since, E λ (u ) = Du () + (u )/λ, we iner tat ( u )/λ Du (). Tus, or any v BV (), we ave As a consequence, we ave Dv () Du () u, v u λ. (4.4) E λ (v) E λ (u ) = Dv () Du () + 1 λ u λ E λ (v) E λ (u ) 1 λ v u. ( ) v u, v u + u, v u λ + 1 λ v u To sow te equality (4.3), we coose v in (4.4) to be equal to 0 and u, respectively. Using te act tat te total variation unctional is positively 1-omogeneous, we iner tat Du () = u, u λ. Tereore E λ (u ) = u λ, u + u λ, u = 1 λ ( u ). Te next result is an upper bound o te dierence between te energy o te discrete minimizer and te energy o its piecewise linear interpolation.

15 A New Projected Gradient Algoritm or Total Variation Based Image Denoising 15 Lemma 8 Let L () be given, and z, te minimizer o te unctional E λ, (u). Ten E λ (P z, ) E λ, (z,) C λ were C is a positive constant depending on and λ. [ ω(z,, 1) + ω (, ) ], (4.5) Proo Since J (z, ) = DP z, (), we ave ( ) λ E λ (P z, ) E λ, (z, = P z, C z, C Q P z, C z, ( ) P z, C z, + C z, + C z, C z, C Q Now, let us ind an upper bound or te dierence C z, C z, C Q. ( C z, C z, C Q = C z, (x) (x) C z, (x) C Q (x) ) dx = (C Q (x) (x)) ( C z, (x) C Q (x) (x) ) dx C Q C z, C Q. Te inequality (4.5) ollows rom Lemma togeter wit (.17), and Lemma 3 by observing tat C z, C Q 4 and C z,. Let 1 := x R : dist(x, ) < 1}. We denote by ū an extension (see [, Proposition 3.1 page 131]) o u to a unction in BV (R ) suc tat ū = 0 in R \ 1, and Dū ( ) = 0. Let ( ρ(x) = c exp 1 x 1 ) 1 x 1} (x), x R be te standard convolution kernel, and ρ ɛ} ɛ>0 te ensuing amily o molliiers. Te constant c > 0 in te deinition o ρ above is cosen suc tat ρ(x)dx = 1. For ɛ > 0 ixed, we denote by ū ɛ := ρɛ ū te ɛ-molliication o ū. Ten, it is known tat (see [,10]) Dū ɛ () Dū ( ) = Dū () = Du (). (4.6) Lemma 9 Suppose L (). Let u be te minimizer o te ROF-unctional E λ (u), and z, te minimizer o te discrete ROF-unctional E λ, (u). Ten, or any 0 < < 1 and or 1 E λ, (z,) E λ (u ) + K 1 (1 )/ + K ū ū (4.7) were ū is te /4 molliication o ū, K 1 and K are positive constants tat depend only on λ,, and. Proo Let ɛ > 0 be ixed. By deinition o z,, we ave E λ, (z,) E λ, (Q ū ɛ ) = J (Q ū ɛ ) + 1 λ C Q ū ɛ C Q. (4.8) Moreover by deinition o te operators C and Q, it is easy to establis tat C Q ū ɛ C Q ū ɛ u + ū ɛ ū ( u + ū ɛ ū ) Next, we obtain an upper bound on J (Q ū ɛ ). By deinition J (Q ū ɛ ) = 1 i,j<n + 1 i,j<n u + 6 ū ɛ ū using (4.3). (4.9) (Q ū ɛ ) i+1,j (Q ū ɛ ) i,j (Q ū ɛ ) i+1,j+1 (Q ū ɛ ) i,j+1 (Q + ū ɛ ) i,j+1 (Q ū ɛ ) i,j (Q + ū ɛ ) i+1,j+1 (Q ū ɛ ) i+1,j. (4.10)

16 16 Ming-Jun Lai, Leopold Matamba Messi Now, let 1 i, j < N be ixed. Ten, by te Taylor ormula, we ave (Q ū ɛ ) i+1,j (Q ū ɛ ) i,j = 1 ɛ ū (x 1 +, x ) ū ɛ (x 1, x ) dx = 1 ( ū ɛ (x) + ) 1 ū ɛ (x 1 + t, x )dt dx. x 1 0 Tanks to Jensen s inequality, we iner rom te latter equation tat (Q ū ɛ ) i+1,j (Q ū ɛ ) i,j 1 ū ɛ (x) x 1 dx + Bij(), 1 (4.11) were B 1 i,j() is given by B 1 i,j() = ū ɛ (x) ū ɛ x 1 x 1 (x 1 + t, x ) dtdx x 1 0 ū ɛ (x 1 + t, x ) dtdx. (4.1) Excanging te order o integration and using Caucy-Scwarz inequality, we obtain an upper bound on Bi,j() 1 as ollows: Bi,j() 1 1 ū ɛ (x) x 1 dx + + ū ɛ (x) dx. (4.13) 4 Similarly, we can sow tat (Q ū ɛ ) i,j+1 (Q ū ɛ ) i,j 1 wit B ij() deined by B i,j() = ū ɛ (x) ū ɛ x x and satisying te inequality Bi,j() 1 (x 1, x + t) dtdx ū ɛ x dx i+1,j x 1 x 1 ū ɛ (x) x dx + Bij(), (4.14) ū ɛ (x 1, x + t) dtdx, (4.15) x ū ɛ dx. (4.16) Combining (4.11) and (4.14) in equation (4.10), and using Jensen s inequality, we obtain or 1 J (Q ū ɛ ) Dū ɛ () + 1/ Dū ɛ () + ū ɛ ū ɛ x + 1 x dx ) Du () + ( Du 1/ ū ɛ () + + ū ɛ dx by (4.6). (4.17) Now, using te isometric embedding o L 1 () into te space o Radon measure, we ave or i = 1, ū ɛ x i dx = ū ɛ ( ) ρ ɛ x φ dx = ū i φ C( ) φ 1 x φ dx i φ C( ) φ 1 ρ ɛ u φ C( ) x i φ 1 Substituting te latter inequality into (4.17) gives φ K ɛ, x 1 J (Q ū ɛ ) (1 + ) Du () + K Finally, letting ɛ = /4 wit < 1 and combining (4.19) and (4.9), we get x x wit K = u ρ x i E λ, (z,) E λ (u ) + K 1 (1 )/ + K ū ū,. (4.18) ɛ. (4.19) were ū is te /4 molliication o ū, and te K is are positive constants depending only on, λ, and. We are now ready to establis an error bound or te L norm o P z, u.

17 A New Projected Gradient Algoritm or Total Variation Based Image Denoising 17 Teorem Let u be te minimizer o te ROF unctional E λ (u) in BV (), and z, te minimizer o te discrete ROF unctional E λ, (u). Ten, or any 0 < < 1 and 0 < 1 P z, u K 3 ω (, ) + K K 5 ū ū, (4.0) were K 3, K 4, and K 5 are positive constants independent o. Proo Using (4.), we ave P z, u λ ( = λ ) E λ (P z, ) E λ (u ) Now, tanks to te Lemmas 8 and 9, i.e., (4.5) and (4.7), we ave tat ( E λ (P z, ) E λ, (z,) + E λ, (z,) E λ (u ) P z, u C(ω(z,, 1) + ω (, )) + λ(k 1 (1 )/ + K ū ū ). (4.1) Next, we notice tat by Lemmas 4 and 5, we ave ω(z,, 1) = =( 1, ) 1 i+ 1 N =1 1 j+ N =( 1, ) =1 =( 1, ) =1 =( 1, ) 1 + =1 z N,, z N, (z, ) i+1,j+ (z, ) i,j (Q ) N (Q ) N, by (3.8) (Q ) N (Q ) N ω(z,, 1) 8 (Q ) i+1,j+ (Q ) i,j. =( 1, ) 1 i+ 1 N 1 + =1 1 j+ N From te latter inequality, te deinition o Q, and Jensen s inequality, we obtain ω(z,, 1) 8 =( 1, ) 1 i+ 1 N 1 + =1 1 j+ N 8 8 =( 1, ) 1 + =1 =( 1, ) 1 + =1 1 i+ 1 N 1 j+ N (Q ) i+1,j+ (Q ) i,j 1 (x + ) (x) dx (x + ) (x) dx 8 ω (, ). ) Tus, we ave ω(z,, 1) 4 ω (, ). (4.) Finally, we combine te inequalities (4.1) and (4.), to obtain inequality (4.0) wit K 3 = 5C, K 4 = λk 1, and K 5 = λk. Corollary 1 Let 0 < β 1 be given. I Lip(β, L ()), ten P z, u 0 as 0. Proo Suppose tat Lip(β, L ()). Tere exists M β > 0 suc tat or any 0 < 1, β ω (, ) M β. As a consequence, (4.0) becomes wit = β/ P z, u K 3 M β β + K 4 ( β)/4 + K 5 ū ū. Since ū ū 0 as 0, we iner tat P z, u 0 as 0, and te proo is complete.

18 18 Ming-Jun Lai, Leopold Matamba Messi Fig. : Convergence o ALG1, ALG and ALG3 or te image in Figure 3a wit σ = 5 and λ = 1/13. In algoritms ALG1 and ALG we set τ = 0.05, wile τ = in ALG3. In tis case we used te value o λ tat gave te best PSNRs or our coices o τ. 10 ROF: λ=1/16 ALG1 ALG ALG3 Mean Square Error Iterations 5 Numerical experiments In tis section, we report te result o te numerical experiments wit Algoritm 31. We compare te perormance o te algoritm proposed above to Cambolle s ixed-point algoritm proposed in [4], and te projected-gradient algoritm proposed in [5]. Te test images used are presented in Figure 3. We sall use te ollowing abbreviations to identiy te tree algoritms under consideration ere. ALG1: Te ixed point iterative algoritm described in [4]. ALG: Te projected-gradient algoritm described in [5,8]. ALG3: Te projected-gradient algoritm presented in Algoritm 31. It sould be noted tat in our tests, we did not attempt to coose te parameters τ and λ or optimal perormance o te algoritms. Table 1 troug Table 4 below sow te capability o Algoritm 31 to remove noise or various noise level. Te inputs or all tree algoritms are obtained by adding a zero mean Gaussian noise wit standard deviation σ to te images in Figure 3. Our experiments sow tat te new projected gradient algoritm, Algoritm 31, is sligtly more eicient tan ALG. Finally in Figure, we sow te asymptotic beavior o te tree algoritms: ALG1, ALG, and ALG3. All o te tree algoritms display te same asymptotic beavior as te number o iterations goes to ininity. O course, tis agreement o asymptotic beaviors is expected since all tree algoritms in te discrete setting were proved to converge to te solution o te ROF model. Reerences 1. R. Acar and C.R. Vogel: Analysis o bounded variation penalty metods or ill-posed problems, Inverse Problems 10 (1994), L. Ambrosio, N. Fusco, and D. Pallara, Functions o bounded variation and ree discontinuity problems, Oxord University Press, USA (000). 3. T. F. Can, G. H. Golub, and P. Mulet, A nonlinear primal-dual metod or total variation-based image restoration, SIAM J. Sci. Comput. 0 (1999), no. 6, A. Cambolle, An algoritm or total variation minimization and applications. Special issue on matematics and image analysis, J. Mat. Imaging Vision 0 (004), no. 1-, pp A. Cambolle, Total variation minimization and a class o binary MRF models, in Energy minimization metods in computer vision and pattern recognition, Lecture Notes in Computer Science, Vol. 3757/005 (005), pp A. Cambolle and P.-L. Lions, Image recovery via total variation minimization and related problems, Numer. Mat. 76 (1997), no., D. C. Dobson and C. R. Vogel, Convergence o an iterative metod or total variation denoising, SIAM J. Numer. Anal. 34 (1997), V. Duval, J.-F. Aujol, and L. A. Vese, Matematical modeling o textures: Application to color image decomposition wit a projected gradient algoritm, J Mat Imaging Vis. 37 (010), I. Ekeland and R. Temam, Convex analysis and variational problems, Society or Industrial Matematics, Piladelpia, PA, USA (1987). 10. E. Giusti, Minimal suraces and unctions o bounded variation, Monograps in Matematics, 80, Birkäuser, Basel (1984).

19 A New Projected Gradient Algoritm or Total Variation Based Image Denoising 19 Fig. 3: Te images used in te numerical experiments. Te images in te top row are o size 56 56, wile tose in te bottom row ave resolution (a) Lena (b) Peppers (c) Bank. (d) Boats 11. Q. Hong, M. J. Lai, and J. Wang, Convergence Analysis o a Finite Dierence Sceme or te Gradient Flow associated wit te ROF Model, submitted (010). 1. M. -J. Lai, B. Lucier, and J. Wang, Te convergence o a central-dierence discretization o Rudin-Oser-Fatemi model or image denoising, in Scale Space and Variational Metods in Computer Vision, Lecture Notes in Computer Science, Vol. 5567/009 (009), L.I. Rudin, S. Oser, and E. Fatemi, Nonlinear total variation based noise removal algoritms, Pysica D: Nonlinear Penomena 60 (199), no. 1-4, C. R. Vogel and M. E. Oman, Iterative metods or total variation denoising, SIAM J. Sci. Comput. 17 (1996), no. 1, J. Wang and Lucier, B., Error bounds or inite-dierence metods or Rudin-Oser-Fatemi image smooting, to appear in SIAM J. Numer. Analysis (010). 16. J. Wang, Error bound or numerical metods or te ROF image smooting model, PD Dissertation, Purdue University, IN, USA (008).

20 0 Ming-Jun Lai, Leopold Matamba Messi Table 1: Comparison o te algoritms using te image in Figure 3a. Eac algoritm is terminated wen te dierence between te Mean Square Error (MSE) at two consecutive steps is less tan σ λ τ ALG1 ALG τ ALG3 15 1/ / / / / / / / / / Table : Comparison o te Algoritms using te image in Figure 3b. Eac algoritm is terminated wen te dierence between te Mean Square Error (MSE) at two consecutive steps is less tan σ λ τ ALG1 ALG τ ALG3 15 1/ / / / / / / / / / Table 3: Comparison o te Algoritms using te image in Figure 3c. Eac algoritm is terminated wen te dierence between te Mean Square Error (MSE) at two consecutive steps is less tan σ λ τ ALG1 ALG τ ALG3 15 1/ / / / / / / / / /

21 A New Projected Gradient Algoritm or Total Variation Based Image Denoising 1 Table 4: Comparison o te Algoritms using te image in Figure 3d. Eac algoritm is terminated wen te dierence between te Mean Square Error (MSE) at two consecutive steps is less tan σ λ τ ALG1 ALG τ ALG3 15 1/ / / / / / / / / /