Variational Image Restoration

Size: px
Start display at page:

Download "Variational Image Restoration"

Transcription

1 Variational Image Restoration Yuling Jiao School of and Statistics and Mathematics ZNUFE Dec 30, 2014

2 Outline 1 1 Classical Variational Restoration Models and Algorithms 1.1 Variational Image Restoration Modeling 1.2 Algorithms for min u J (u) = F (Ku, f ) + λr(u) 2 2 Proposed Model and Algorithm 3 3 Numerical Results 3.1 Denoising 3.2 Deblurring 3.3 Inpainting (ZNFUE) Dec 30, / 118

3 1 Classical Variational Restoration Models and Algorithms

4 Variational Image Restoration Modeling f = Ku n, f Y, u X, K L(X, Y ). +, n is gaussian =, n is multiplicative n(u), n is image dependent I, Denoising convolution, Deblurring K = projection, Inpainting upsample, Zomming Dec 30, / 118

5 oringal image Dec 30, / 118

6 SNR 3.84dB,PSNR 7.89dB June 5, / 118

7 SNR 16.92dB,PSNR 29.45dB,It: 28 June 5, / 118

8 oringal image June 5, / 118

9 SNR 1.72dB,PSNR 11.99dB June 5, / 118

10 SNR 24.47dB,PSNR 35.64dB,It: 51 June 5, / 118

11 original image June 5, / 118

12 inpainted June 5, / 118

13 Variational Image Restoration Modeling Recover u from observed f by MAP: } u = arg max {P u f u u = arg min u { = arg max u { Pf u (f u)p u (u)} P f (f ) } log P f u (f u) + log P u (u) eg: restoration f = u + n with Gaussian distributed n and Gaussian prior of u, i.e., ( P f u (f u) exp f ) u 2 L 2 2σ 2 n and ( P u (u) exp ) u 2 L 2 2σ 2. u June 5, / 118

14 Variational Image Restoration Modeling Then u is recovered by min u f u 2 L 2 (Ω) + λ u H 1 (Ω) (Tikhonov regularization). So, one can recover u by the regularized variational form Then, min J (u) = F (Ku, f ) + λr(u). u proper data fitting term + proper regularization term+ fast algorithm + proper regularization parameter = good restoration June 5, / 118

15

16 Variational Image Restoration Modeling Data Fitting Term F (Ku, f ) 1 2 Ku f 2 L 2 (Ω), Gaussian Ku f G, texture Ku f L 1 (Ω), outlier F = Ku f log Ku, poisson Ω Ω Ku + f Ku, multiplicative (Ku f ) 2, sample Ω u 2 (Ku f ) 2 Ku, ultrasound Ω (WHU) June 5, / 118

17 Variational Image Restoration Modeling Regularization Term (prior of u) u L2 (Ω), Tikhonov regularizer, 1960 s u H 1 (Ω), Tikhonov regularizer, 1960 s u, Γ M S, Mumford-Shar regularizer, 1980 s u TV (Ω), TV regularizer, 1992, Osher Wu L1 (Ω), Wavelet regularizer, 1990 s R(u) = u E, Euler s elastica regularizer, 1980 s, M-S u TV 2 (Ω), TV 2 regularizer, 2000 s, Scherzer u HOR : M, Reduced TV 2 regularizer, 2004, T.Chan u ICTV (Ω), Inf-conv regularizer, 1997, Lions, Chambolle u TGV (Ω),... Tgv regularizer, 2010, Kunisch (WHU) June 5, / 118

18 Variational Image Restoration Modeling u L 2 (Ω) convex differentiable but no geometrical information and poor noise killing u H 1 (Ω) convex differentiable but do not keep jumps u(x, y), (x, y) Ω, Ω = (a, b) (c, d), for a.e y (c, d), u(, y) H 1 (a, b) H ölder(1/2). Mumford-Shar λ ] [(1 v i+0.5,j )(u i+1,j u i,j ) 2 +(1 v i,j +0.5 )(u i,j +1 u i,j ) 2 +µ i,j i,j R M S (u, Γ) = λ u 2 + µ Γ h Ω\Γ ensure spatial regularity and keep edges but non-convex (v i+0.5,j + v i,j +0.5 ) (WHU) June 5, / 118

19 Variational Image Restoration Modeling u TV { } Du := sup u gdx : p C0 (Ω, R2 ), p 1. Ω Ω Denote the Rodon measure Ω Du by u TV. Important property: Du = u + u + d h 1 and Du = dsdl Γ Γ Ω {u=l} edge preserving convex but non differentiable and cause stair case Euler s elastica (α + β κ 2 )ds Γ + u E = (α + β κ 2 )dsdl = (α + β( u {u=l} Ω u )2 ) Du preserve both jumps and smoothness but nonconvex (WHU) June 5, / 118

20 Variational Image Restoration Modeling u TV 2 u TV 2 { } := sup u 2 vdx : v C0 (Ω, S 2 2 ), v F 1 Ω convex, take care of flatten domain but blur the the boundary. u M u TV 2 { } := sup u vdx : v C0 (Ω), v 1 Ω convex, weaker smoothness but also do not keep jump since u L 1 (Ω) and u M (Ω) u W 1,1 (Ω) (WHU) June 5, / 118

21 Variational Image Restoration Modeling u ICTV (Ω) { } u ICTV (Ω) := inf α u v v TV (Ω) + β v TV 2 (Ω) { = inf α v sup p 1 p C 0 (Ω,R2 ) Ω (u v) pdx + β sup q 1 q C 0 (Ω,S 2 2 ) = sup inf. and inf is attained when α p = p,q v β 2 q Ω } v 2 qdx convex and utilizing structure of the image preserve both smoothness and constant jump (WHU) June 5, / 118

22 Variational Image Restoration Modeling u GTV (Ω) u GTV (Ω) := sup u 2 qdx (let p = q in u ICTV ) q α Ω q β q C 0 (Ω,S 2 2 ) Wu L 1 (Ω) W can be FT, let, dictionary... natural images can be sparsely approximate under let bases large let coefficients appear at the singularities different let has different vanishing moment let transform can be seen as an multi-differential operator which can detect the underling regularity of the image adaptively (WHU) June 5, / 118

23 Variational Image Restoration Modeling HOR MODEL TO REDUCE STAIR CASE A. Chambolle and P. Lions min u 1+u 2 λ 1 u 1 TV + λ 2 H (u 2 ) F L 1 + K (u 1 + u 2 ) f 2 L 2 T. Chan, S. Esedoglu, and F. Park min λ 1 u 1 u TV + λ 2 u 2 p 1+u 2 L 1 + K (u 1 + u 2 ) f 2 L, p = 1, 2 2 T. Chan, A. Marquina, and P. Mulet min u λ 1 u TV + λ 2 Φ( u ) (Lu) 2 L 1 + Ku f 2 L 2 (WHU) June 5, / 118

24 Variational Image Restoration Modeling K. Kunisch,K. Bredies, and T. Pock min u,v λ 1 ε(v) + λ 2 u v L 1 + Ku f 2 L 2 Osher, S, Tai, X.C.,Rahman, T min τ TV + 1 τ f 2, s.t. τ = 0 τ 2λ L 2 min u u TV u, τ τ, s.t. Ku f L 2 σ(n) L 2 (WHU) June 5, / 118

25 Variational Image Restoration Modeling Tight Frames Let X L 2 (R) to be countable, X is a tight if f 2 2 = < f, h > 2, f L 2 (R). h X It is equivalent to f = h X < f, h > h, f L 2 (R). Representation of f in terms of h is not unique. The sequence {< f, h >} is called the canonical frame coefficients. (WHU) June 5, / 118

26 Variational Image Restoration Modeling Wavelet Frames A wavelet system is the collection of the dilations and the shifts of a finite set Ψ L 2 (R) X (Ψ) = {2 k/2 ψ(2 k x j ) : ψ Ψ, k, j Z} Theorem X (Ψ) is a tight frame iff its "dual Gramian" is the identity, i.e., ^ψ(2 k ω) 2 = 1, ψ Ψ k Z a.e and ^ψ(2 k ω) ^ψ(2 k (ω + (2j + 1)2π)) = 0, a.e j Z. ψ Ψ k=0 (WHU) June 5, / 118

27 Variational Image Restoration Modeling Wavelet Frame from Multiresolution Analysis Start with a function refinable ψ L 2 (R) with mask h 0, i.e. φ(x ) = 2 k Z h 0 (k)φ(2x k). Let Ψ = {ψ 1, ψ 2,, ψ r }, where ψ i (x ) = 2 k Z h i (k)φ(2x k), with wavelet masks h i. (WHU) June 5, / 118

28 Variational Image Restoration Modeling Matrix Representation Let rows of W be a frame, i.e. W T W = I. Decomposition: c = Wf. Reconstruction: f = W T c. W can be generated by tight frame filters obtained via UEP. Framelet Based Balanced Model { 1 min K W T c f p c p + α (I WW T )c } 2 + β c L p (Ω) 2 l 2 l 1, p = {1, 2} (WHU) June 5, / 118

29 Algorithms for min u J (u) = F (Ku, f ) + λr(u) First Optimization then Discretization first derive Euler-Lagrange equation { F (Ku, f ) + λ R(u) = 0 B.D. then time marching: u = F (Ku, f ) λ R(u) t u(0, ) = f B.C. it does work but high nonlinearity + nonsmoothness = bad CFL condition (slow) (WHU) June 5, / 118

30 Algorithms for min u J (u) = F (Ku, f ) + λr(u) First Discretization and Optimization (based on operator splitting,alternative direction,nesterov s acceleration ) State-of-the-Art-Methods Augmented-Lagrangian-Type Methods Forward-Backward Splitting Nesterov s acceleration A Unified Primal-Dual Algorithm (WHU) June 5, / 118

31 Algorithms for min u J (u) = F (Ku, f ) + λr(u) Augmented-Lagrangian-Type Methods minimize subject to H (u) + G(v) Au + Bv = b Algorithm 1 ADMM(GADM) Require: v 0 R Nv, λ 0 R N b, τ > 0 1: for k = 0, 1, do 2: u k+1 = arg min u H (u)+ < λ k, Au > + τ 2 b Au Bv k u u k 2 Q 3: v k+1 = arg min u G(u)+ < λ k, Bv > + τ 2 b Au k+1 Bv v v k 2 P 4: λ k+1 = λ k + τ(b Au k+1 Bv k+1 ) 5: end for (WHU) June 5, / 118

32 Algorithms for min u J (u) = F (Ku, f ) + λr(u) minimize subject to H (u) + G(v) Au + Bv = b Algorithm 2 AMA Require: λ 0 R N b, τ > 0 1: for k = 0, 1, do 2: u k+1 = arg min u H (u)+ < λ k, Au > 3: v k+1 = arg min u G(u)+ < λ k, Bv > + τ 2 b Au k+1 Bv 2 4: λ k+1 = λ k + τ(b Au k+1 Bv k+1 ) 5: end for (WHU) June 5, / 118

33 Algorithms for min u J (u) = F (Ku, f ) + λr(u) Forward-Backward Splitting minimize J (x ) Algorithm 3 Gradient Descent 1: for k = 0, 1, do 2: x k+1 = x k + τ J (x k ) 3: end for F = H + G Algorithm 4 FBS 1: for k = 0, 1, do 2: x k+1 = J τ G (x k τ H (x k )) 3: end for (WHU) June 5, / 118

34 Algorithms for min u J (u) = F (Ku, f ) + λr(u) Nesterov s acceleration minimize J (x ) Algorithm 5 Nesterov s Optimal Gradient Descent Require: α 0 = 1, x 0 = y 1 R N, τ < 1/L( F ) 1: for k = 1, 2, 3, do 2: x k = y k τ J (y k ) 3: α k+1 = (1 + 4α 2 k + 1)/2 4: y k+1 = x k + (α k 1)(x k x k 1 )/α k+1 5: end for (WHU) June 5, / 118

35 Algorithms for min u J (u) = F (Ku, f ) + λr(u) J = H + G Algorithm 6 FISTA Require: y 1 = x 0 R N, α 1 = 1, τ < 1/L( G) 1: for k = 1, 2, 3, do 2: x k = J τ G (y k τ H (y k )) 3: α k+1 = (1 + 4α 2 k + 1)/2 4: y k+1 = x k + α k 1 α k+1 (x k x k 1 ) 5: end for (WHU) June 5, / 118

36 Algorithms for min u J (u) = F (Ku, f ) + λr(u) minimize subject to H (u) + G(v) Au + Bv = b Algorithm 7 Fast AMA Require: α 0 = 1, λ 1 = ^λ 0 R N b, τ < ρ(a T A) 1: for k = 0, 1, 2, do 2: u k = arg min H (u)+ < ^λ k, Au > 3: v k = arg min G(v)+ < ^λ k, Bv > + τ 2 b Au k Bv 2 4: λ k = ^λ k + τ(b Au k Bv k ) 5: α k+1 = ( α 2 k )/2 6: ^λ k+1 = λ k + α k 1 α k+1 (λ k λ k 1 ) 7: end for (WHU) June 5, / 118

37 Algorithms for min u J (u) = F (Ku, f ) + λr(u) minimize H (u) + G(v) subject to Au + Bv = b Algorithm 8 Fast ADMM Require: v 1 = ^v 0 R Nv, λ 1 = ^λ 0 R N b, τ > 0 1: for k = 1, 2, 3, do 2: u k = arg min H (u)+ < ^λ k, Au > + τ 2 b Au B ^v k 2 3: v k = arg min G(v)+ < ^λ k, Bv > + τ 2 b Au k Bv 2 4: λ k = ^λ k + τ(b Au k Bv k ) 5: if E k > 0 then 6: α k+1 = ( α 2 k )/2 7: ^v k+1 = v k + α k 1 α k+1 (v k v k 1 ) 8: ^λ k+1 = λ k + α k 1 α k+1 (λ k λ k 1 ) 9: else 10: α k+1 = 1, ^v k+1 = v k, ^λ k+1 = λ k 11: end if 12: end for (WHU) June 5, / 118

38 Algorithms for min u J (u) = F (Ku, f ) + λr(u) A Unified Primal-Dual Algorithm min F (Kx ) + G(x ), max x X ( ) G ( K y) + F (y) y Y min max < Kx, y > +G(x ) F (y) x X y Y Initialization: Choose τ, σ > 0, θ [0, 1], (x 0, y 0 ) X Y and set x 0 = x 0. Iterations (n 0): Update x n, y n, x n as follows: y n+1 = (I + σ F ) 1 (y n + σk x n ) x n+1 = (I + τ G) 1 (x n τk y n+1 ) x n+1 = x n+1 + θ(x n+1 x n ) (WHU) June 5, / 118

39 Algorithms for min u J (u) = F (Ku, f ) + λr(u) min x Ω Du + λ 2 u g 2 2 min max < u, div p > X + λ u X p Y 2 u g 2 2 δp(p) F (p) = δp(p) G(u) = λ 2 u g 2 2 p = (I + σ F ) 1 ( p) p i,j = p i,j max(1, p i,j ) (WHU) June 5, / 118

40 Algorithms for min u J (u) = F (Ku, f ) + λr(u) min u Ω Du + λ u g 1 min max < u, div p > X +λ u g u X p Y 1 δp(p) ũ i,j τλ u = (I + τ G) 1 (ũ) ũ i,j + τλ g i,j if ũ i,j g i,j > τλ if ũ i,j g i,j < τλ if ũ i,j g i,j τλ (WHU) June 5, / 118

41 Algorithms for min u J (u) = F (Ku, f ) + λr(u) min u Ω Du + λ 2 Au g 2 2 min max < u, div p > X + λ u X p Y 2 Au g 2 2 δp(p) u = (I + τ G) 1 (ũ) u = arg min u u ũ 2τ F 1( τλf(g)f(k A ) + F(ũ) τλf(k A ) λ 2 k A u g 2 2 ) (WHU) June 5, / 118

42 Algorithms for min u J (u) = F (Ku, f ) + λr(u) min u X Φu 1 + λ 2 Au g 2 2 min u X Φu 1 + λ 2 u = (I + τ G) 1 (ũ) u i,j = ũ i,j ũ i,j +τλg i,j 1+τλ i,j D\I if (i, j ) I else (u i,j g i,j ) 2 (WHU) June 5, / 118

43 2 Proposed Model and Algorithm

44 Proposed Model and Algorithm Proposed Model { } 1 min u p Ku f p L p (Ω) + α u TV + β ωwu l, p = {1, 2}, = {a, i} 1 there exist at least one minimizer. convex, utilizes edge preserving property of total variation and the adaptivity of the framlet transform to the underlying regularity of image. Therefore, preserve both jumps and smoothness. (WHU) June 5, / 118

45 Proposed Model and Algorithm Algorithm:(ADMM for Fixed α and β) 1 min u 2 Ku f 2 L 2 (Ω) + α u TV + β ωwu l (P) 1 min max u,v,z p,q L(u, v, z ; p, q) (PD) L(u, v, z ; p, q) = 1 2 Ku f 2 + u v, p + r 2 u v α v 1 + Wu z, q + s 2 Wu z β ωz 1 (WHU) June 5, / 118

46 Algorithm for Proposed Model algorithm ( ) u 0 = f, v 0 = 0, z 0 = 0; p 0 = 0, q 0 = 0, given α 0, β 0 γ > 0 for l = 1, 2, 3,, for k = 1, 2, 3,, (u k, v k, z k ) = arg min u,v,z L α l 1,β l 1(u, v, z ; pk 1, q k 1 ) p k = p k 1 r(p k 1 u k ) q k = q k 1 s(q k 1 Wz k ) stop or k k + 1 end for α l = 1 γ+1 ( 1 2 β l = 1 γ+1 ( 1 2 stop or l l + 1 end for Ku k f βl 1 ωwu k l 1)/ u k 1 Ku k f αl 1 u k 1 )/ ωwu k l 1 (WHU) June 5, / 118

47 Algorithm Detail Alternative Direction Minimization for (u k, v k, z k ) = arg min u,v,z L α l 1,β l 1(u, v, z ; pk 1, q k 1 ) fix v = v k 1,z = z k 1, solving u k u k = arg min L α l 1,β l 1(u, v k 1, z k 1 ; p k 1, q k 1 ) u the optimal conditon is a linear systerm. fix u = u k, solving v k, z k v k = arg min L α l 1,β l 1(u k, v, z k 1 ; p k 1, q k 1 ) v z k = arg min z L α l 1,β l 1(u k, v k, z ; p k 1, q k 1 ) two close form solution defined by threshold functions. (WHU) June 5, / 118

48 Complexity and Convergence Computational Complexity: O(m n) or O(m n) log(m n) Convergence Analysis: Convergence for u k with fix α l, β l : X = R m n, Y = X X, Z = W (X ), A = [, W ] : X X Y Z F (u) = 1 2 Ku f 2 2 : X R, G(v) = α v 1 1 +β ωv 2 1 : Y Z : R then, min u,v F (u) + G(v), s.t. Au = v by ADMM is inner loop of algorithm ( ) Convergence for α l, β l : {α l }, {β l } are monotone in the same way depend on α 0, β 0. (WHU) June 5, / 118

49 3 Numerical Results

50 Denoising oringal image (WHU) June 5, / 118

51 Denoising observed image,psnr 23.70dB (WHU) June 5, / 118

52 Denoising tv restoration,psnr 34.29dB, CPU 0.84s (WHU) June 5, / 118

53 Denoising balanced restoration,psnr 31.53dB, CPU 2.17s (WHU) June 5, / 118

54 Denoising chan 4th ordr restoration,psnr 35.25dB, CPU 8.33s (WHU) June 5, / 118

55 Denoising tgv restoration,psnr 35.72dB, CPU 27.69s (WHU) June 5, / 118

56 Denoising tv frame restoration,psnr 33.95dB, CPU 5.44s (WHU) June 5, / 118

57 Denoising oringal image (WHU) June 5, / 118

58 Denoising observed image,psnr 23.47dB (WHU) June 5, / 118

59 Denoising tv restoration,psnr 28.35dB, CPU 0.39s (WHU) June 5, / 118

60 Denoising balanced restoration,psnr 27.10dB, CPU 4.47s (WHU) June 5, / 118

61 Denoising chan 4th ordr restoration,psnr 28.01dB, CPU 8.19s (WHU) June 5, / 118

62 Denoising tgv restoration,psnr 28.07dB, CPU 26.92s (WHU) June 5, / 118

63 Denoising tv frame restoration,psnr 28.23dB, CPU 5.97s (WHU) June 5, / 118

64 Denoising oringal image (WHU) June 5, / 118

65 Denoising observed image,psnr 22.82dB (WHU) June 5, / 118

66 Denoising tv restoration,psnr 26.19dB, CPU 0.83s (WHU) June 5, / 118

67 Denoising balanced restoration,psnr 25.72dB, CPU 2.78s (WHU) June 5, / 118

68 Denoising chan 4th ordr restoration,psnr 25.98dB, CPU 8.94s (WHU) June 5, / 118

69 Denoising tgv restoration,psnr 25.58dB, CPU 23.84s (WHU) June 5, / 118

70 Denoising tv frame restoration,psnr 26.21dB, CPU 6.45s (WHU) June 5, / 118

71 Denoising oringal image (WHU) June 5, / 118

72 Denoising observed image,psnr 24.10dB (WHU) June 5, / 118

73 Denoising tv restoration,psnr 32.14dB, CPU 0.52s (WHU) June 5, / 118

74 Denoising balanced restoration,psnr 30.38dB, CPU 5.44s (WHU) June 5, / 118

75 Denoising chan 4th ordr restoration,psnr 31.46dB, CPU 12.97s (WHU) June 5, / 118

76 Denoising tgv restoration,psnr 32.19dB, CPU 16.27s (WHU) June 5, / 118

77 Denoising tv frame restoration,psnr 31.86dB, CPU 9.28s (WHU) June 5, / 118

78 Deblurring oringal image (WHU) June 5, / 118

79 Deblurring observed image,psnr 16.60dB (WHU) June 5, / 118

80 Deblurring tv restoration,psnr 33.12dB, CPU 2.78s (WHU) June 5, / 118

81 Deblurring balanced restoration,psnr 30.57dB, CPU 24.53s (WHU) June 5, / 118

82 Deblurring tv frame restoration,psnr 30.93dB, CPU 13.80s (WHU) June 5, / 118

83 Deblurring oringal image (WHU) June 5, / 118

84 Deblurring observed image,psnr 18.13dB (WHU) June 5, / 118

85 Deblurring tv restoration,psnr 29.44dB, CPU 2.88s (WHU) June 5, / 118

86 Deblurring balaced restoration,psnr 28.84dB, CPU 18.69s (WHU) June 5, / 118

87 Deblurring tv frame restoration,psnr 29.08dB, CPU 7.86s (WHU) June 5, / 118

88 Deblurring oringal image (WHU) June 5, / 118

89 Deblurring observed image,psnr 18.55dB (WHU) June 5, / 118

90 Deblurring tv restoration,psnr 24.72dB, CPU 2.84s (WHU) June 5, / 118

91 Deblurring balanced restoration,psnr 24.33dB, CPU 27.03s (WHU) June 5, / 118

92 Deblurring tv frame restoration,psnr 23.92dB, CPU 11.19s (WHU) June 5, / 118

93 Deblurring oringal image (WHU) June 5, / 118

94 Deblurring observed image,psnr 18.32dB (WHU) June 5, / 118

95 Deblurring tv restoration,psnr 27.39dB, CPU 1.70s (WHU) June 5, / 118

96 Deblurring blanced restoration,psnr 27.53dB, CPU 24.80s (WHU) June 5, / 118

97 Deblurring tv frame restoration,psnr 27.55dB, CPU 11.50s (WHU) June 5, / 118

98 Inpainting oringal image (WHU) June 5, / 118

99 Inpainting observed image,psnr 21.68dB (WHU) June 5, / 118

100 Inpainting tv restoration,psnr 43.01dB, CPU 95.63s (WHU) June 5, / 118

101 Inpainting balanced restoration,psnr 31.60dB, CPU 34.14s (WHU) June 5, / 118

102 Inpainting tvframe restoration,psnr 42.20dB, CPU s (WHU) June 5, / 118

103 Inpainting oringal image (WHU) June 5, / 118

104 Inpainting observed image,psnr 14.27dB (WHU) June 5, / 118

105 Inpainting tv restoration,psnr 27.82dB, CPU 13.03s (WHU) June 5, / 118

106 Inpainting balanced restoration,psnr 27.35dB, CPU 32.02s (WHU) June 5, / 118

107 Inpainting tgframe restoration,psnr 27.65dB, CPU s (WHU) June 5, / 118

108 Inpainting oringal image (WHU) June 5, / 118

109 Inpainting observed image,psnr 18.86dB (WHU) June 5, / 118

110 Inpainting tv restoration,psnr 34.27dB, CPU 11.02s (WHU) June 5, / 118

111 Inpainting balanced restoration,psnr 31.20dB, CPU 9.03s (WHU) June 5, / 118

112 Inpainting tvframe restoration,psnr 32.32dB, CPU 47.75s (WHU) June 5, / 118

113 Inpainting oringal image (WHU) June 5, / 118

114 Inpainting observed image,psnr 23.05dB (WHU) June 5, / 118

115 Inpainting tv restoration,psnr 27.24dB, CPU 55.55s (WHU) June 5, / 118

116 Inpainting balanced restoration,psnr 26.82dB, CPU 29.52s (WHU) June 5, / 118

117 Inpainting tvframe restoration,psnr 26.85dB, CPU s (WHU) June 5, / 118

118 Thank you!

IMAGE RESTORATION: TOTAL VARIATION, WAVELET FRAMES, AND BEYOND

IMAGE RESTORATION: TOTAL VARIATION, WAVELET FRAMES, AND BEYOND IMAGE RESTORATION: TOTAL VARIATION, WAVELET FRAMES, AND BEYOND JIAN-FENG CAI, BIN DONG, STANLEY OSHER, AND ZUOWEI SHEN Abstract. The variational techniques (e.g., the total variation based method []) are

More information

Convex Hodge Decomposition and Regularization of Image Flows

Convex Hodge Decomposition and Regularization of Image Flows Convex Hodge Decomposition and Regularization of Image Flows Jing Yuan, Christoph Schnörr, Gabriele Steidl April 14, 2008 Abstract The total variation (TV) measure is a key concept in the field of variational

More information

A General Framework for a Class of Primal-Dual Algorithms for TV Minimization

A General Framework for a Class of Primal-Dual Algorithms for TV Minimization A General Framework for a Class of Primal-Dual Algorithms for TV Minimization Ernie Esser UCLA 1 Outline A Model Convex Minimization Problem Main Idea Behind the Primal Dual Hybrid Gradient (PDHG) Method

More information

A GENERAL FRAMEWORK FOR A CLASS OF FIRST ORDER PRIMAL-DUAL ALGORITHMS FOR CONVEX OPTIMIZATION IN IMAGING SCIENCE

A GENERAL FRAMEWORK FOR A CLASS OF FIRST ORDER PRIMAL-DUAL ALGORITHMS FOR CONVEX OPTIMIZATION IN IMAGING SCIENCE A GENERAL FRAMEWORK FOR A CLASS OF FIRST ORDER PRIMAL-DUAL ALGORITHMS FOR CONVEX OPTIMIZATION IN IMAGING SCIENCE ERNIE ESSER XIAOQUN ZHANG TONY CHAN Abstract. We generalize the primal-dual hybrid gradient

More information

Accelerated Dual Gradient-Based Methods for Total Variation Image Denoising/Deblurring Problems (and other Inverse Problems)

Accelerated Dual Gradient-Based Methods for Total Variation Image Denoising/Deblurring Problems (and other Inverse Problems) Accelerated Dual Gradient-Based Methods for Total Variation Image Denoising/Deblurring Problems (and other Inverse Problems) Donghwan Kim and Jeffrey A. Fessler EECS Department, University of Michigan

More information

Coordinate Update Algorithm Short Course Operator Splitting

Coordinate Update Algorithm Short Course Operator Splitting Coordinate Update Algorithm Short Course Operator Splitting Instructor: Wotao Yin (UCLA Math) Summer 2016 1 / 25 Operator splitting pipeline 1. Formulate a problem as 0 A(x) + B(x) with monotone operators

More information

Gauge optimization and duality

Gauge optimization and duality 1 / 54 Gauge optimization and duality Junfeng Yang Department of Mathematics Nanjing University Joint with Shiqian Ma, CUHK September, 2015 2 / 54 Outline Introduction Duality Lagrange duality Fenchel

More information

LINEARIZED BREGMAN ITERATIONS FOR FRAME-BASED IMAGE DEBLURRING

LINEARIZED BREGMAN ITERATIONS FOR FRAME-BASED IMAGE DEBLURRING LINEARIZED BREGMAN ITERATIONS FOR FRAME-BASED IMAGE DEBLURRING JIAN-FENG CAI, STANLEY OSHER, AND ZUOWEI SHEN Abstract. Real images usually have sparse approximations under some tight frame systems derived

More information

Inverse problems Total Variation Regularization Mark van Kraaij Casa seminar 23 May 2007 Technische Universiteit Eindh ove n University of Technology

Inverse problems Total Variation Regularization Mark van Kraaij Casa seminar 23 May 2007 Technische Universiteit Eindh ove n University of Technology Inverse problems Total Variation Regularization Mark van Kraaij Casa seminar 23 May 27 Introduction Fredholm first kind integral equation of convolution type in one space dimension: g(x) = 1 k(x x )f(x

More information

A GENERAL FRAMEWORK FOR A CLASS OF FIRST ORDER PRIMAL-DUAL ALGORITHMS FOR TV MINIMIZATION

A GENERAL FRAMEWORK FOR A CLASS OF FIRST ORDER PRIMAL-DUAL ALGORITHMS FOR TV MINIMIZATION A GENERAL FRAMEWORK FOR A CLASS OF FIRST ORDER PRIMAL-DUAL ALGORITHMS FOR TV MINIMIZATION ERNIE ESSER XIAOQUN ZHANG TONY CHAN Abstract. We generalize the primal-dual hybrid gradient (PDHG) algorithm proposed

More information

A posteriori error control for the binary Mumford Shah model

A posteriori error control for the binary Mumford Shah model A posteriori error control for the binary Mumford Shah model Benjamin Berkels 1, Alexander Effland 2, Martin Rumpf 2 1 AICES Graduate School, RWTH Aachen University, Germany 2 Institute for Numerical Simulation,

More information

FAST ALTERNATING DIRECTION OPTIMIZATION METHODS

FAST ALTERNATING DIRECTION OPTIMIZATION METHODS FAST ALTERNATING DIRECTION OPTIMIZATION METHODS TOM GOLDSTEIN, BRENDAN O DONOGHUE, SIMON SETZER, AND RICHARD BARANIUK Abstract. Alternating direction methods are a common tool for general mathematical

More information

Continuous Primal-Dual Methods in Image Processing

Continuous Primal-Dual Methods in Image Processing Continuous Primal-Dual Methods in Image Processing Michael Goldman CMAP, Polytechnique August 2012 Introduction Maximal monotone operators Application to the initial problem Numerical illustration Introduction

More information

A Dual Formulation of the TV-Stokes Algorithm for Image Denoising

A Dual Formulation of the TV-Stokes Algorithm for Image Denoising A Dual Formulation of the TV-Stokes Algorithm for Image Denoising Christoffer A. Elo, Alexander Malyshev, and Talal Rahman Department of Mathematics, University of Bergen, Johannes Bruns gate 12, 5007

More information

A fast primal-dual method for generalized Total Variation denoising

A fast primal-dual method for generalized Total Variation denoising Appl. Math. Inf. Sci. 6, No. 3, 401-409 (2012) 401 Applied Mathematics & Information Sciences An International Journal A fast primal-dual method for generalized Total Variation denoising Francesc Aràndiga,

More information

A Tutorial on Primal-Dual Algorithm

A Tutorial on Primal-Dual Algorithm A Tutorial on Primal-Dual Algorithm Shenlong Wang University of Toronto March 31, 2016 1 / 34 Energy minimization MAP Inference for MRFs Typical energies consist of a regularization term and a data term.

More information

A Total Fractional-Order Variation Model for Image Restoration with Non-homogeneous Boundary Conditions and its Numerical Solution

A Total Fractional-Order Variation Model for Image Restoration with Non-homogeneous Boundary Conditions and its Numerical Solution A Total Fractional-Order Variation Model for Image Restoration with Non-homogeneous Boundary Conditions and its Numerical Solution Jianping Zhang and Ke Chen arxiv:509.04237v [cs.cv] 6 Sep 205 Abstract

More information

Recent developments on sparse representation

Recent developments on sparse representation Recent developments on sparse representation Zeng Tieyong Department of Mathematics, Hong Kong Baptist University Email: zeng@hkbu.edu.hk Hong Kong Baptist University Dec. 8, 2008 First Previous Next Last

More information

PDEs in Image Processing, Tutorials

PDEs in Image Processing, Tutorials PDEs in Image Processing, Tutorials Markus Grasmair Vienna, Winter Term 2010 2011 Direct Methods Let X be a topological space and R: X R {+ } some functional. following definitions: The mapping R is lower

More information

arxiv: v1 [math.na] 3 Jan 2019

arxiv: v1 [math.na] 3 Jan 2019 arxiv manuscript No. (will be inserted by the editor) A Finite Element Nonoverlapping Domain Decomposition Method with Lagrange Multipliers for the Dual Total Variation Minimizations Chang-Ock Lee Jongho

More information

Convex Hodge Decomposition of Image Flows

Convex Hodge Decomposition of Image Flows Convex Hodge Decomposition of Image Flows Jing Yuan 1, Gabriele Steidl 2, Christoph Schnörr 1 1 Image and Pattern Analysis Group, Heidelberg Collaboratory for Image Processing, University of Heidelberg,

More information

Solving DC Programs that Promote Group 1-Sparsity

Solving DC Programs that Promote Group 1-Sparsity Solving DC Programs that Promote Group 1-Sparsity Ernie Esser Contains joint work with Xiaoqun Zhang, Yifei Lou and Jack Xin SIAM Conference on Imaging Science Hong Kong Baptist University May 14 2014

More information

Math 273a: Optimization Overview of First-Order Optimization Algorithms

Math 273a: Optimization Overview of First-Order Optimization Algorithms Math 273a: Optimization Overview of First-Order Optimization Algorithms Wotao Yin Department of Mathematics, UCLA online discussions on piazza.com 1 / 9 Typical flow of numerical optimization Optimization

More information

Dual methods for the minimization of the total variation

Dual methods for the minimization of the total variation 1 / 30 Dual methods for the minimization of the total variation Rémy Abergel supervisor Lionel Moisan MAP5 - CNRS UMR 8145 Different Learning Seminar, LTCI Thursday 21st April 2016 2 / 30 Plan 1 Introduction

More information

Adaptive discretization and first-order methods for nonsmooth inverse problems for PDEs

Adaptive discretization and first-order methods for nonsmooth inverse problems for PDEs Adaptive discretization and first-order methods for nonsmooth inverse problems for PDEs Christian Clason Faculty of Mathematics, Universität Duisburg-Essen joint work with Barbara Kaltenbacher, Tuomo Valkonen,

More information

Shiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 9. Alternating Direction Method of Multipliers

Shiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 9. Alternating Direction Method of Multipliers Shiqian Ma, MAT-258A: Numerical Optimization 1 Chapter 9 Alternating Direction Method of Multipliers Shiqian Ma, MAT-258A: Numerical Optimization 2 Separable convex optimization a special case is min f(x)

More information

Inexact Alternating Direction Method of Multipliers for Separable Convex Optimization

Inexact Alternating Direction Method of Multipliers for Separable Convex Optimization Inexact Alternating Direction Method of Multipliers for Separable Convex Optimization Hongchao Zhang hozhang@math.lsu.edu Department of Mathematics Center for Computation and Technology Louisiana State

More information

Adaptive Primal Dual Optimization for Image Processing and Learning

Adaptive Primal Dual Optimization for Image Processing and Learning Adaptive Primal Dual Optimization for Image Processing and Learning Tom Goldstein Rice University tag7@rice.edu Ernie Esser University of British Columbia eesser@eos.ubc.ca Richard Baraniuk Rice University

More information

Large-Scale L1-Related Minimization in Compressive Sensing and Beyond

Large-Scale L1-Related Minimization in Compressive Sensing and Beyond Large-Scale L1-Related Minimization in Compressive Sensing and Beyond Yin Zhang Department of Computational and Applied Mathematics Rice University, Houston, Texas, U.S.A. Arizona State University March

More information

Deep Learning: Approximation of Functions by Composition

Deep Learning: Approximation of Functions by Composition Deep Learning: Approximation of Functions by Composition Zuowei Shen Department of Mathematics National University of Singapore Outline 1 A brief introduction of approximation theory 2 Deep learning: approximation

More information

Variational Methods in Image Denoising

Variational Methods in Image Denoising Variational Methods in Image Denoising Jamylle Carter Postdoctoral Fellow Mathematical Sciences Research Institute (MSRI) MSRI Workshop for Women in Mathematics: Introduction to Image Analysis 22 January

More information

Lecture 4 Colorization and Segmentation

Lecture 4 Colorization and Segmentation Lecture 4 Colorization and Segmentation Summer School Mathematics in Imaging Science University of Bologna, Itay June 1st 2018 Friday 11:15-13:15 Sung Ha Kang School of Mathematics Georgia Institute of

More information

Variational Methods in Signal and Image Processing

Variational Methods in Signal and Image Processing Variational Methods in Signal and Image Processing XU WANG Texas A&M University Dept. of Electrical & Computer Eng. College Station, Texas United States xu.wang@tamu.edu ERCHIN SERPEDIN Texas A&M University

More information

Investigating the Influence of Box-Constraints on the Solution of a Total Variation Model via an Efficient Primal-Dual Method

Investigating the Influence of Box-Constraints on the Solution of a Total Variation Model via an Efficient Primal-Dual Method Article Investigating the Influence of Box-Constraints on the Solution of a Total Variation Model via an Efficient Primal-Dual Method Andreas Langer Department of Mathematics, University of Stuttgart,

More information

A memory gradient algorithm for l 2 -l 0 regularization with applications to image restoration

A memory gradient algorithm for l 2 -l 0 regularization with applications to image restoration A memory gradient algorithm for l 2 -l 0 regularization with applications to image restoration E. Chouzenoux, A. Jezierska, J.-C. Pesquet and H. Talbot Université Paris-Est Lab. d Informatique Gaspard

More information

A Multilevel Proximal Algorithm for Large Scale Composite Convex Optimization

A Multilevel Proximal Algorithm for Large Scale Composite Convex Optimization A Multilevel Proximal Algorithm for Large Scale Composite Convex Optimization Panos Parpas Department of Computing Imperial College London www.doc.ic.ac.uk/ pp500 p.parpas@imperial.ac.uk jointly with D.V.

More information

Convex representation for curvature dependent functionals

Convex representation for curvature dependent functionals Convex representation for curvature dependent functionals Antonin Chambolle CMAP, Ecole Polytechnique, CNRS, Palaiseau, France joint work with T. Pock (T.U. Graz) Laboratoire Jacques-Louis Lions, 25 nov.

More information

Fast and Accurate HARDI and its Application to Neurological Diagnosis

Fast and Accurate HARDI and its Application to Neurological Diagnosis Fast and Accurate HARDI and its Application to Neurological Diagnosis Dr. Oleg Michailovich Department of Electrical and Computer Engineering University of Waterloo June 21, 2011 Outline 1 Diffusion imaging

More information

Sparse Regularization via Convex Analysis

Sparse Regularization via Convex Analysis Sparse Regularization via Convex Analysis Ivan Selesnick Electrical and Computer Engineering Tandon School of Engineering New York University Brooklyn, New York, USA 29 / 66 Convex or non-convex: Which

More information

Operator Splitting for Parallel and Distributed Optimization

Operator Splitting for Parallel and Distributed Optimization Operator Splitting for Parallel and Distributed Optimization Wotao Yin (UCLA Math) Shanghai Tech, SSDS 15 June 23, 2015 URL: alturl.com/2z7tv 1 / 60 What is splitting? Sun-Tzu: (400 BC) Caesar: divide-n-conquer

More information

Novel integro-differential equations in image processing and its applications

Novel integro-differential equations in image processing and its applications Novel integro-differential equations in image processing and its applications Prashant Athavale a and Eitan Tadmor b a Institute of Pure and Applied Mathematics, University of California, Los Angeles,

More information

First-order methods for structured nonsmooth optimization

First-order methods for structured nonsmooth optimization First-order methods for structured nonsmooth optimization Sangwoon Yun Department of Mathematics Education Sungkyunkwan University Oct 19, 2016 Center for Mathematical Analysis & Computation, Yonsei University

More information

Sparse Approximation: from Image Restoration to High Dimensional Classification

Sparse Approximation: from Image Restoration to High Dimensional Classification Sparse Approximation: from Image Restoration to High Dimensional Classification Bin Dong Beijing International Center for Mathematical Research Beijing Institute of Big Data Research Peking University

More information

arxiv: v1 [math.na] 17 Nov 2018

arxiv: v1 [math.na] 17 Nov 2018 A New Operator Splitting Method for Euler s Elastica Model Liang-Jian Deng a, Roland Glowinski b, Xue-Cheng Tai c a School of Mathematical Sciences, University of Electronic Science and Technology of China,

More information

ACCELERATED FIRST-ORDER PRIMAL-DUAL PROXIMAL METHODS FOR LINEARLY CONSTRAINED COMPOSITE CONVEX PROGRAMMING

ACCELERATED FIRST-ORDER PRIMAL-DUAL PROXIMAL METHODS FOR LINEARLY CONSTRAINED COMPOSITE CONVEX PROGRAMMING ACCELERATED FIRST-ORDER PRIMAL-DUAL PROXIMAL METHODS FOR LINEARLY CONSTRAINED COMPOSITE CONVEX PROGRAMMING YANGYANG XU Abstract. Motivated by big data applications, first-order methods have been extremely

More information

A First Order Primal-Dual Algorithm for Nonconvex T V q Regularization

A First Order Primal-Dual Algorithm for Nonconvex T V q Regularization A First Order Primal-Dual Algorithm for Nonconvex T V q Regularization Thomas Möllenhoff, Evgeny Strekalovskiy, and Daniel Cremers TU Munich, Germany Abstract. We propose an efficient first order primal-dual

More information

Finite Elements. Colin Cotter. February 22, Colin Cotter FEM

Finite Elements. Colin Cotter. February 22, Colin Cotter FEM Finite Elements February 22, 2019 In the previous sections, we introduced the concept of finite element spaces, which contain certain functions defined on a domain. Finite element spaces are examples of

More information

Multilevel Preconditioning and Adaptive Sparse Solution of Inverse Problems

Multilevel Preconditioning and Adaptive Sparse Solution of Inverse Problems Multilevel and Adaptive Sparse of Inverse Problems Fachbereich Mathematik und Informatik Philipps Universität Marburg Workshop Sparsity and Computation, Bonn, 7. 11.6.2010 (joint work with M. Fornasier

More information

Recent Developments of Alternating Direction Method of Multipliers with Multi-Block Variables

Recent Developments of Alternating Direction Method of Multipliers with Multi-Block Variables Recent Developments of Alternating Direction Method of Multipliers with Multi-Block Variables Department of Systems Engineering and Engineering Management The Chinese University of Hong Kong 2014 Workshop

More information

Wavelet Bi-frames with Uniform Symmetry for Curve Multiresolution Processing

Wavelet Bi-frames with Uniform Symmetry for Curve Multiresolution Processing Wavelet Bi-frames with Uniform Symmetry for Curve Multiresolution Processing Qingtang Jiang Abstract This paper is about the construction of univariate wavelet bi-frames with each framelet being symmetric.

More information

Selected Methods for Modern Optimization in Data Analysis Department of Statistics and Operations Research UNC-Chapel Hill Fall 2018

Selected Methods for Modern Optimization in Data Analysis Department of Statistics and Operations Research UNC-Chapel Hill Fall 2018 Selected Methods for Modern Optimization in Data Analysis Department of Statistics and Operations Research UNC-Chapel Hill Fall 08 Instructor: Quoc Tran-Dinh Scriber: Quoc Tran-Dinh Lecture 4: Selected

More information

ENERGY METHODS IN IMAGE PROCESSING WITH EDGE ENHANCEMENT

ENERGY METHODS IN IMAGE PROCESSING WITH EDGE ENHANCEMENT ENERGY METHODS IN IMAGE PROCESSING WITH EDGE ENHANCEMENT PRASHANT ATHAVALE Abstract. Digital images are can be realized as L 2 (R 2 objects. Noise is introduced in a digital image due to various reasons.

More information

Functionals for signal and image reconstruction: properties of their minimizers and applications

Functionals for signal and image reconstruction: properties of their minimizers and applications Functionals for signal and image reconstruction: properties of their minimizers and applications Research report Mila NIKOLOVA Centre de Mathématiques et de Leurs Applications (CMLA) - CNRS-UMR 8536 -

More information

Gradient Descent and Implementation Solving the Euler-Lagrange Equations in Practice

Gradient Descent and Implementation Solving the Euler-Lagrange Equations in Practice 1 Lecture Notes, HCI, 4.1.211 Chapter 2 Gradient Descent and Implementation Solving the Euler-Lagrange Equations in Practice Bastian Goldlücke Computer Vision Group Technical University of Munich 2 Bastian

More information

Total Variation Theory and Its Applications

Total Variation Theory and Its Applications Total Variation Theory and Its Applications 2nd UCC Annual Research Conference, Kingston, Jamaica Peter Ndajah University of the Commonwealth Caribbean, Kingston, Jamaica September 27, 2018 Peter Ndajah

More information

Algorithms for Nonsmooth Optimization

Algorithms for Nonsmooth Optimization Algorithms for Nonsmooth Optimization Frank E. Curtis, Lehigh University presented at Center for Optimization and Statistical Learning, Northwestern University 2 March 2018 Algorithms for Nonsmooth Optimization

More information

On a multiscale representation of images as hierarchy of edges. Eitan Tadmor. University of Maryland

On a multiscale representation of images as hierarchy of edges. Eitan Tadmor. University of Maryland On a multiscale representation of images as hierarchy of edges Eitan Tadmor Center for Scientific Computation and Mathematical Modeling (CSCAMM) Department of Mathematics and Institute for Physical Science

More information

Recovering piecewise smooth multichannel images by minimization of convex functionals with total generalized variation penalty

Recovering piecewise smooth multichannel images by minimization of convex functionals with total generalized variation penalty SpezialForschungsBereich F 32 Karl Franzens Universität Graz Technische Universität Graz Medizinische Universität Graz Recovering piecewise smooth multichannel images by minimization of convex functionals

More information

Bayesian Paradigm. Maximum A Posteriori Estimation

Bayesian Paradigm. Maximum A Posteriori Estimation Bayesian Paradigm Maximum A Posteriori Estimation Simple acquisition model noise + degradation Constraint minimization or Equivalent formulation Constraint minimization Lagrangian (unconstraint minimization)

More information

A first-order primal-dual algorithm for convex problems with applications to imaging

A first-order primal-dual algorithm for convex problems with applications to imaging A first-order primal-dual algorithm for convex problems with applications to imaging Antonin Chambolle, Thomas Pock To cite this version: Antonin Chambolle, Thomas Pock. A first-order primal-dual algorithm

More information

Calculus of Variations Summer Term 2014

Calculus of Variations Summer Term 2014 Calculus of Variations Summer Term 2014 Lecture 20 18. Juli 2014 c Daria Apushkinskaya 2014 () Calculus of variations lecture 20 18. Juli 2014 1 / 20 Purpose of Lesson Purpose of Lesson: To discuss several

More information

Convex Optimization on Large-Scale Domains Given by Linear Minimization Oracles

Convex Optimization on Large-Scale Domains Given by Linear Minimization Oracles Convex Optimization on Large-Scale Domains Given by Linear Minimization Oracles Arkadi Nemirovski H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology Joint research

More information

arxiv: v1 [math.oc] 3 Jul 2014

arxiv: v1 [math.oc] 3 Jul 2014 SIAM J. IMAGING SCIENCES Vol. xx, pp. x c xxxx Society for Industrial and Applied Mathematics x x Solving QVIs for Image Restoration with Adaptive Constraint Sets F. Lenzen, J. Lellmann, F. Becker, and

More information

You should be able to...

You should be able to... Lecture Outline Gradient Projection Algorithm Constant Step Length, Varying Step Length, Diminishing Step Length Complexity Issues Gradient Projection With Exploration Projection Solving QPs: active set

More information

Double Smoothing technique for Convex Optimization Problems with Linear Constraints

Double Smoothing technique for Convex Optimization Problems with Linear Constraints 1 Double Smoothing technique for Convex Optimization Problems with Linear Constraints O. Devolder (F.R.S.-FNRS Research Fellow), F. Glineur and Y. Nesterov Center for Operations Research and Econometrics

More information

Randomized Coordinate Descent with Arbitrary Sampling: Algorithms and Complexity

Randomized Coordinate Descent with Arbitrary Sampling: Algorithms and Complexity Randomized Coordinate Descent with Arbitrary Sampling: Algorithms and Complexity Zheng Qu University of Hong Kong CAM, 23-26 Aug 2016 Hong Kong based on joint work with Peter Richtarik and Dominique Cisba(University

More information

Lecture 9 Approximations of Laplace s Equation, Finite Element Method. Mathématiques appliquées (MATH0504-1) B. Dewals, C.

Lecture 9 Approximations of Laplace s Equation, Finite Element Method. Mathématiques appliquées (MATH0504-1) B. Dewals, C. Lecture 9 Approximations of Laplace s Equation, Finite Element Method Mathématiques appliquées (MATH54-1) B. Dewals, C. Geuzaine V1.2 23/11/218 1 Learning objectives of this lecture Apply the finite difference

More information

I P IANO : I NERTIAL P ROXIMAL A LGORITHM FOR N ON -C ONVEX O PTIMIZATION

I P IANO : I NERTIAL P ROXIMAL A LGORITHM FOR N ON -C ONVEX O PTIMIZATION I P IANO : I NERTIAL P ROXIMAL A LGORITHM FOR N ON -C ONVEX O PTIMIZATION Peter Ochs University of Freiburg Germany 17.01.2017 joint work with: Thomas Brox and Thomas Pock c 2017 Peter Ochs ipiano c 1

More information

A Linearly Convergent First-order Algorithm for Total Variation Minimization in Image Processing

A Linearly Convergent First-order Algorithm for Total Variation Minimization in Image Processing A Linearly Convergent First-order Algorithm for Total Variation Minimization in Image Processing Cong D. Dang Kaiyu Dai Guanghui Lan October 9, 0 Abstract We introduce a new formulation for total variation

More information

Sparsity Regularization

Sparsity Regularization Sparsity Regularization Bangti Jin Course Inverse Problems & Imaging 1 / 41 Outline 1 Motivation: sparsity? 2 Mathematical preliminaries 3 l 1 solvers 2 / 41 problem setup finite-dimensional formulation

More information

Multiresolution Analysis

Multiresolution Analysis Multiresolution Analysis DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Frames Short-time Fourier transform

More information

Source Reconstruction for 3D Bioluminescence Tomography with Sparse regularization

Source Reconstruction for 3D Bioluminescence Tomography with Sparse regularization 1/33 Source Reconstruction for 3D Bioluminescence Tomography with Sparse regularization Xiaoqun Zhang xqzhang@sjtu.edu.cn Department of Mathematics/Institute of Natural Sciences, Shanghai Jiao Tong University

More information

Dual Proximal Gradient Method

Dual Proximal Gradient Method Dual Proximal Gradient Method http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes Outline 2/19 1 proximal gradient method

More information

Agenda. Fast proximal gradient methods. 1 Accelerated first-order methods. 2 Auxiliary sequences. 3 Convergence analysis. 4 Numerical examples

Agenda. Fast proximal gradient methods. 1 Accelerated first-order methods. 2 Auxiliary sequences. 3 Convergence analysis. 4 Numerical examples Agenda Fast proximal gradient methods 1 Accelerated first-order methods 2 Auxiliary sequences 3 Convergence analysis 4 Numerical examples 5 Optimality of Nesterov s scheme Last time Proximal gradient method

More information

Low-Rank Factorization Models for Matrix Completion and Matrix Separation

Low-Rank Factorization Models for Matrix Completion and Matrix Separation for Matrix Completion and Matrix Separation Joint work with Wotao Yin, Yin Zhang and Shen Yuan IPAM, UCLA Oct. 5, 2010 Low rank minimization problems Matrix completion: find a low-rank matrix W R m n so

More information

Convex Optimization and l 1 -minimization

Convex Optimization and l 1 -minimization Convex Optimization and l 1 -minimization Sangwoon Yun Computational Sciences Korea Institute for Advanced Study December 11, 2009 2009 NIMS Thematic Winter School Outline I. Convex Optimization II. l

More information

Adaptive Corrected Procedure for TVL1 Image Deblurring under Impulsive Noise

Adaptive Corrected Procedure for TVL1 Image Deblurring under Impulsive Noise Adaptive Corrected Procedure for TVL1 Image Deblurring under Impulsive Noise Minru Bai(x T) College of Mathematics and Econometrics Hunan University Joint work with Xiongjun Zhang, Qianqian Shao June 30,

More information

Enhanced Compressive Sensing and More

Enhanced Compressive Sensing and More Enhanced Compressive Sensing and More Yin Zhang Department of Computational and Applied Mathematics Rice University, Houston, Texas, U.S.A. Nonlinear Approximation Techniques Using L1 Texas A & M University

More information

Comparison of formulations and solution methods for image restoration problems

Comparison of formulations and solution methods for image restoration problems INSTITUTE OF PHYSICS PUBLISHING Inverse Problems 7 (2) 977 995 INVERSE PROBLEMS PII: S266-56()2336-7 Comparison of formulations and solution methods for image restoration problems Tommi Kärkkäinen, Kirsi

More information

TWO-PHASE APPROACH FOR DEBLURRING IMAGES CORRUPTED BY IMPULSE PLUS GAUSSIAN NOISE. Jian-Feng Cai. Raymond H. Chan. Mila Nikolova

TWO-PHASE APPROACH FOR DEBLURRING IMAGES CORRUPTED BY IMPULSE PLUS GAUSSIAN NOISE. Jian-Feng Cai. Raymond H. Chan. Mila Nikolova Manuscript submitted to Website: http://aimsciences.org AIMS Journals Volume 00, Number 0, Xxxx XXXX pp. 000 000 TWO-PHASE APPROACH FOR DEBLURRING IMAGES CORRUPTED BY IMPULSE PLUS GAUSSIAN NOISE Jian-Feng

More information

Numerical Solution of Nonsmooth Problems and Application to Damage Evolution and Optimal Insulation

Numerical Solution of Nonsmooth Problems and Application to Damage Evolution and Optimal Insulation Numerical Solution of Nonsmooth Problems and Application to Damage Evolution and Optimal Insulation Sören Bartels University of Freiburg, Germany Joint work with G. Buttazzo (U Pisa), L. Diening (U Bielefeld),

More information

EE 367 / CS 448I Computational Imaging and Display Notes: Image Deconvolution (lecture 6)

EE 367 / CS 448I Computational Imaging and Display Notes: Image Deconvolution (lecture 6) EE 367 / CS 448I Computational Imaging and Display Notes: Image Deconvolution (lecture 6) Gordon Wetzstein gordon.wetzstein@stanford.edu This document serves as a supplement to the material discussed in

More information

An Algorithmic Framework of Generalized Primal-Dual Hybrid Gradient Methods for Saddle Point Problems

An Algorithmic Framework of Generalized Primal-Dual Hybrid Gradient Methods for Saddle Point Problems An Algorithmic Framework of Generalized Primal-Dual Hybrid Gradient Methods for Saddle Point Problems Bingsheng He Feng Ma 2 Xiaoming Yuan 3 January 30, 206 Abstract. The primal-dual hybrid gradient method

More information

Optimization for Learning and Big Data

Optimization for Learning and Big Data Optimization for Learning and Big Data Donald Goldfarb Department of IEOR Columbia University Department of Mathematics Distinguished Lecture Series May 17-19, 2016. Lecture 1. First-Order Methods for

More information

Accelerated primal-dual methods for linearly constrained convex problems

Accelerated primal-dual methods for linearly constrained convex problems Accelerated primal-dual methods for linearly constrained convex problems Yangyang Xu SIAM Conference on Optimization May 24, 2017 1 / 23 Accelerated proximal gradient For convex composite problem: minimize

More information

Parameter Identification in Partial Differential Equations

Parameter Identification in Partial Differential Equations Parameter Identification in Partial Differential Equations Differentiation of data Not strictly a parameter identification problem, but good motivation. Appears often as a subproblem. Given noisy observation

More information

arxiv: v1 [math.na] 30 Jan 2018

arxiv: v1 [math.na] 30 Jan 2018 Modern Regularization Methods for Inverse Problems Martin Benning and Martin Burger December 18, 2017 arxiv:1801.09922v1 [math.na] 30 Jan 2018 Abstract Regularization methods are a key tool in the solution

More information

Inverse problem and optimization

Inverse problem and optimization Inverse problem and optimization Laurent Condat, Nelly Pustelnik CNRS, Gipsa-lab CNRS, Laboratoire de Physique de l ENS de Lyon Decembre, 15th 2016 Inverse problem and optimization 2/36 Plan 1. Examples

More information

Wavelet Frames on the Sphere for Sparse Representations in High Angular Resolution Diusion Imaging. Chen Weiqiang

Wavelet Frames on the Sphere for Sparse Representations in High Angular Resolution Diusion Imaging. Chen Weiqiang Wavelet Frames on the Sphere for Sparse Representations in High Angular Resolution Diusion Imaging Chen Weiqiang Overview 1. Introduction to High Angular Resolution Diusion Imaging (HARDI). 2. Wavelets

More information

Introduction to Nonlinear Image Processing

Introduction to Nonlinear Image Processing Introduction to Nonlinear Image Processing 1 IPAM Summer School on Computer Vision July 22, 2013 Iasonas Kokkinos Center for Visual Computing Ecole Centrale Paris / INRIA Saclay Mean and median 2 Observations

More information

ALGORITHMS FOR MINIMIZING DIFFERENCES OF CONVEX FUNCTIONS AND APPLICATIONS

ALGORITHMS FOR MINIMIZING DIFFERENCES OF CONVEX FUNCTIONS AND APPLICATIONS ALGORITHMS FOR MINIMIZING DIFFERENCES OF CONVEX FUNCTIONS AND APPLICATIONS Mau Nam Nguyen (joint work with D. Giles and R. B. Rector) Fariborz Maseeh Department of Mathematics and Statistics Portland State

More information

Nonlinear Flows for Displacement Correction and Applications in Tomography

Nonlinear Flows for Displacement Correction and Applications in Tomography Nonlinear Flows for Displacement Correction and Applications in Tomography Guozhi Dong 1 and Otmar Scherzer 1,2 1 Computational Science Center, University of Vienna, Oskar-Morgenstern-Platz 1, 1090 Wien,

More information

A fast nonstationary preconditioning strategy for ill-posed problems, with application to image deblurring

A fast nonstationary preconditioning strategy for ill-posed problems, with application to image deblurring A fast nonstationary preconditioning strategy for ill-posed problems, with application to image deblurring Marco Donatelli Dept. of Science and High Tecnology U. Insubria (Italy) Joint work with M. Hanke

More information

Image restoration: numerical optimisation

Image restoration: numerical optimisation Image restoration: numerical optimisation Short and partial presentation Jean-François Giovannelli Groupe Signal Image Laboratoire de l Intégration du Matériau au Système Univ. Bordeaux CNRS BINP / 6 Context

More information

A Localized Linearized ROF Model for Surface Denoising

A Localized Linearized ROF Model for Surface Denoising 1 2 3 4 A Localized Linearized ROF Model for Surface Denoising Shingyu Leung August 7, 2008 5 Abstract 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 1 Introduction CT/MRI scan becomes a very

More information

Introduction to Alternating Direction Method of Multipliers

Introduction to Alternating Direction Method of Multipliers Introduction to Alternating Direction Method of Multipliers Yale Chang Machine Learning Group Meeting September 29, 2016 Yale Chang (Machine Learning Group Meeting) Introduction to Alternating Direction

More information

Solution-driven Adaptive Total Variation Regularization

Solution-driven Adaptive Total Variation Regularization 1/15 Solution-driven Adaptive Total Variation Regularization Frank Lenzen 1, Jan Lellmann 2, Florian Becker 1, Stefania Petra 1, Johannes Berger 1, Christoph Schnörr 1 1 Heidelberg Collaboratory for Image

More information

Primal-dual coordinate descent A Coordinate Descent Primal-Dual Algorithm with Large Step Size and Possibly Non-Separable Functions

Primal-dual coordinate descent A Coordinate Descent Primal-Dual Algorithm with Large Step Size and Possibly Non-Separable Functions Primal-dual coordinate descent A Coordinate Descent Primal-Dual Algorithm with Large Step Size and Possibly Non-Separable Functions Olivier Fercoq and Pascal Bianchi Problem Minimize the convex function

More information

Optimal mass transport as a distance measure between images

Optimal mass transport as a distance measure between images Optimal mass transport as a distance measure between images Axel Ringh 1 1 Department of Mathematics, KTH Royal Institute of Technology, Stockholm, Sweden. 21 st of June 2018 INRIA, Sophia-Antipolis, France

More information

Fast Multilevel Algorithm for a Minimization Problem in Impulse Noise Removal

Fast Multilevel Algorithm for a Minimization Problem in Impulse Noise Removal Fast Multilevel Algorithm for a Minimization Problem in Impulse Noise Removal Raymond H. Chan and Ke Chen September 25, 7 Abstract An effective 2-phase method for removing impulse noise was recently proposed.

More information