Optimisation in imaging

Transcription

1 Optimisation in imaging Hugues Talbot ICIP 2014 Tutorial Optimisation on Hierarchies H. Talbot : Optimisation 1/59

2 Outline of the lecture 1 Concepts in optimization Cost function Constraints Duality 2 Inverse problems in imaging Useful formulation in imaging 3 Formulations in imaging 4 DCTV model 5 Generalization of TV models Algorithms 6 Applications in image processing 7 Non-convex regularization 8 Conclusion H. Talbot : Optimisation 2/59

3 Concepts in optimization Introduction Mathematical optimization is a domain of applied mathematics relevant to many areas including statistics, mechanics, signal and image processing. Generalizes many well known techniques such as least squares, linear programming, convex programming, integer programming, combinatorial optimization and others. In this talk we will overview both the continuous and discrete formulations. We follow the notations of [1]. H. Talbot : Optimisation 3/59

4 Concepts in optimization General form Cost function and constraints An optimization problem generally has the following form minimize f 0 (x) subject to f i (x) b i, i = 1,..., m (1) x = (x 1,..., x n ) is a vector of R n called the optimization variable of the problem; f 0 : R n R is the cost function functional; the f i : R n R are the constraints and the b i are the bounds (or limits). A vector x is is optimal, or is a solution to the problem, if it has the smallest objective value among all vectors that satisfy the constraints. H. Talbot : Optimisation 4/59

5 Concepts in optimization Types of optimization problems The type of the variables, the cost function and the constraints determine the type of problems we are dealing with. Optimization problems, in their most general form, are usually unsolvable in practice. NP-complete problems (traveling salesperson, subset-sum, etc) can classically be put in this form and so can many NP-hard problems. Some mathematical regularity is necessary to be able to find a solution: for example, linearity or convexity in all the functions. Requiring integer solutions usually, but not always, makes things much harder: Diophantine vs linear equations for instance. H. Talbot : Optimisation 5/59

6 Concepts in optimization Example: least-squares Least squares with no constraints minimize f 0 (x) = Ax b 2 2 = k a x i b i (2) i=1 The system is quadratic, so convex and differentiable. The solution to (2) is unique and reduces to the linear equation and so the analytical solution is (A A)x = A b (3) x = (A A) 1 A b (4) H. Talbot : Optimisation 6/59

7 Concepts in optimization Regularization: Tikhonov Even with something as simple as least-squares, if A A is ill-conditioned, the solution will be very sensitive to noise, e.g. in the example of deconvolution or tomography. One solution is to use regularization. Ill-posed least-squares problems The simplest regularization strategy is due to Tikhonov [3]. minimize f 0 (x) = Ax b Γx 2 2, (5) where Γ is a well-chosen operator, e.g. λi or x or a wavelet operator. The solution is given analytically by x = (A A + Γ Γ) 1 A b (6) H. Talbot : Optimisation 7/59

8 Concepts in optimization Linear programming Linear programming with constraints minimize c x subject to a i x b i; i = 1,..., n (7) No analytical solution. Well established family of algorithms: the Simplexe (Dantzig 1948) ; interior-point (Karmarkar 1984) Not always easy to recognize. Important for compressive sensing. H. Talbot : Optimisation 8/59

9 Concepts in optimization Duality in the LP case Primal / Dual linear programs Primal minimize c x subject to a i x b i; i = 1,..., n (8) Dual maximize b y subject to a i x c i ; i = 1,..., m (9) A primal/dual pair of LP problems can be obtained by transposing the constraint matrix and swapping cost function and constraint bounds. The primal and dual optima, if they exist, are the same, and can be easily deducted from each other. H. Talbot : Optimisation 9/59

10 Concepts in optimization Duality in convex optimization The same concept of duality applies in convex optimization Duality allows one to swap constraints for terms in the objective function Two concepts of duality : Lagrange and Fenchel. Both are equivalent. H. Talbot : Optimisation 10/59

11 Inverse problems in imaging Motivation: inverse problems in imaging Images we observe are nearly always blurred, noisy, projected versions of some reality. We wish to dispel the fog of acquisition by removing all the artefacts as much as possible to observe the real data. This is an inverse problem. H. Talbot : Optimisation 11/59

12 Inverse problems in imaging Maximum Likelihood We want to estimate some statistical parameter θ on the basis of some observation x. If f is the sampling distribution, f (x θ) is the probability of x when the population parameter is θ. The function θ f (x θ) is the likelihood. The Maximum Likelihood estimate is ˆθ ML (x) = argmax θ f (x θ) E.g, if we have a linear operator H (in matrix form) and Gaussian deviates, then argmax x f (x) = Hx y 2 2 = x H Hx + 2y Hx y y is a quadratic form with a unique maximum, provided by f (x) = 2H Hx + 2H y = 0 θ = (H H) 1 H y H. Talbot : Optimisation 12/59

13 Inverse problems in imaging Strengths and drawbacks of MLE When possible, MLE is fast and effective. Many imaging operators have a MLE interpretation: Gaussian smoothing ; Wiener filtering ; Filtered back projection for tomography ; Principal component analysis... However these require a very descriptive model (with few degrees of freedom) and a lot of data, typically unsuitable for images because we do not have a suitable model for natural images. When we do not have all these hypotheses, sometimes the Bayesian Maximum A Posteriori approach can be used instead. H. Talbot : Optimisation 13/59

14 Inverse problems in imaging Maximum A Posteriori If we assume that we know a prior distribution g over θ, i.e. some a-priori information. Following Bayesian statistics, we can treat θ as a random variable and compute the posterior distribution of θ: (i.e. the Bayes theorem). f (x θ)g(θ) θ f (θ x) = ϑ Θ f (x ϑ)g(ϑ)dϑ Then the Maximum a Posteriori is the estimate ˆθ MAP (x) = argmax θ MAP is a regularization of ML. f (θ x) = argmax θ f (x θ)g(θ) H. Talbot : Optimisation 14/59

15 Inverse problems in imaging Markov Random Fields So far this is statistics theory. What is the link between MAP and imaging? We need an imaging model. A Markov Random Field is a model made of a set of sites (a.k.a. pixels) S = {s 1,..., s n }, a set of random variables y = {y 1,..., y n } associated with each pixel, and a set of neighbours N 1,...,n at each pixel location. N p describes the neighborhood at pixel p. Obeys the Markov condition, i.e. Pr(y p y S\p ) = Pr(y p N p ) I.e.: the probability of a pixel p depends only on its immediate neighbours. H. Talbot : Optimisation 15/59

16 Inverse problems in imaging Formulating the MAP of an MRF Now let us express a MAP formulation for an MRF Given a set of observables x = {x 1,..., x n }, We derive a MAP ŷ = argmax Pr(y 1...n x) (10) y 1...n = argmax n Pr(x n y n ) Pr(y 1...n ) (11) y 1...n n=1 = argmax n log[pr(x n y n )] + log[pr(y 1...n )] (12) y 1...n n=1 n = argmin U p (y p ) + P u,p (y u, y p ) y 1...n p=1 u N p (13) (Geman & Geman, PAMI 1984). H. Talbot : Optimisation 16/59

17 Inverse problems in imaging Solving the MAP-MRF formulation This last sum is an energy contains unary terms U p (y p ) and pairwise terms P u,p (y u, y p ). We now have an optimization problem. Depending on the expression of the probability functions, can solve it by i: statistical means, e.g. EM, ii: physical analogies, e.g. simulated annealing or iii: via linear/convex optimization techniques. With some restrictions, graph cuts are able to optimize these energies. H. Talbot : Optimisation 17/59

18 Inverse problems in imaging MRF and Graph Cuts For instance, consider the binary segmentation problem. With unary weights the above can be written: argmin Ê(G) = v i V w i (V i ) + λ w ij δ Vi V j (14) e ij E V i is 1 if v i V s and 0 if v i V t, i.e. it is 1 if pixel i belongs to the partition containing s and 0 otherwise. δ Vi V j is 1 if the corresponding e ij is on the cut, and 0 otherwise. The first sum contains the pairwise terms, and sums the cost of the cut in the image plane. The second sum contains the unary terms, and adds the cost of a pixel to belong to either the partition containing s or the partition containing t. H. Talbot : Optimisation 18/59

19 Inverse problems in imaging Illustration S S T (a) T (b) Figure : Segmentation with unary weights. In this case weighted edges link the source and the sink to all the pixels in the image (a). The min-cut is a surface separating s from t (b). Some strong edge weights can ensure the surface crosses the pixel plane, enforcing topology constraints. H. Talbot : Optimisation 19/59

20 Inverse problems in imaging Segmentation example (a) (b) Figure : Binary segmentation with unary weights and no markers (Boykov-Jolly segmentation model, ICCV 2001). H. Talbot : Optimisation 20/59

21 Inverse problems in imaging Image restoration and graph cuts GC are able to optimize some MRF energies exactly (globally) in the binary case More generally, submodular (e.g. discrete-convex) energies can be at least locally optimized using graph cuts Using various constructions, e.g. Ishikawa PAMI 2003, it is possible to map restoration (denoisng) problems to GC. Many GC optimization approaches have been invented to solve the corresponding energies: α-expansions, α β moves, convex moves, etc (Veksler 1999). They were essentially known before in other communities (Murota 2003). More recent approaches are able to optimize the same kind of energies using different techniques: Belief propagation, Primal-dual Tree-Reweighted, etc (Kolmogorov PAMI 2006). H. Talbot : Optimisation 21/59

22 Inverse problems in imaging Graph-based energies These formulation are very useful but suffer from the purely discrete graph framework Formulations and solutions are not isotropic (grid bias) Graph based formulation can be resource-intensive (memory and speed) They are hard to parallelize Hard to incorporate extra constraints and projection/linear operators. H. Talbot : Optimisation 22/59

23 Formulations in imaging Continuous image restoration model We suppose there exists some unknown image x R N. However we do observe some data y R Q via some linear operator H, which is corrupted by some noise: y = Hx + u, H R Q N x y H. Talbot : Optimisation 23/59

24 Formulations in imaging Recovery We seek to recover a good approximation ˆx of x from H and y. H can be: Model for camera, including defocus and motion blur MRI, PET, X-Ray tomography... u often modeled by Additive White Gaussian Noise, but can be Poisson, Poisson Gauss, Rician, etc. Simplest case: least squares: ˆx = argmin x Hx y 2 2 analytical, simple, effective, but not robust to outliers. H. Talbot : Optimisation 24/59

25 Formulations in imaging Recovery We seek to recover a good approximation ˆx of x from H and y. H can be: Model for camera, including defocus and motion blur MRI, PET, X-Ray tomography... u often modeled by Additive White Gaussian Noise, but can be Poisson, Poisson Gauss, Rician, etc. Tikhonov regularization: ˆx = argmin x x λ Hx y 2 2 reflect the prior assumption that we want to avoid large x. Also analytical and more robust but not sparse. H. Talbot : Optimisation 24/59

26 Formulations in imaging Recovery We seek to recover a good approximation ˆx of x from H and y. H can be: Model for camera, including defocus and motion blur MRI, PET, X-Ray tomography... u often modeled by Additive White Gaussian Noise, but can be Poisson, Poisson Gauss, Rician, etc. Enforced sparsity: ˆx = argmin x x 0 + λ Hx y 2 If we know x to be sparse (many zero elements) in some space (e.g. Wavelets). Highly non-convex. H. Talbot : Optimisation 24/59

27 Formulations in imaging Recovery We seek to recover a good approximation ˆx of x from H and y. H can be: Model for camera, including defocus and motion blur MRI, PET, X-Ray tomography... u often modeled by Additive White Gaussian Noise, but can be Poisson, Poisson Gauss, Rician, etc. Compressive sensing: ˆx = argmin x x 1 + λ Hx y 2 If we know x to be sparse (many zero elements) in some space (e.g. Wavelets). Smallest convex approximation of the l 0 pseudo-norm. H. Talbot : Optimisation 24/59

28 Formulations in imaging Formal context Penalized optimization problem Find where ( ) min F (x) = Φ(Hx y) + λr(x), x R N Φ Fidelity to data term, related to noise R Regularization term, related to some a priori assumptions λ Regularization weight Here, x is sparse in a dictionary V of analysis vectors in R N F 0 (x) = Φ(Hx y)+λ l 0 (V x) H. Talbot : Optimisation 25/59

29 Formulations in imaging Formal context Penalized optimization problem Find where ( ) min F (x) = Φ(Hx y) + λr(x), x R N Φ Fidelity to data term, related to noise R Regularization term, related to some a priori assumptions λ Regularization weight Here, x is sparse in a dictionary V of analysis vectors in R N F δ (x) = Φ(Hx y)+λ C c=1 ψ δ (V c x) H. Talbot where : Optimisation ψ δ is a differentiable, non-convex approximation of the 25/59 l 0 norm.

30 Formulations in imaging Benefits and drawbacks of the continuous approach pros flexible theory (not just denoising; deblurring, tomography, MRI reconstruction, etc) large library of algorithms, many more than in the discrete case isotropic convergence proofs and characterization of solutions. cons non-explicit discretization non-flexible structure deriving projections operators sometimes inefficient or impossible conditions for convergence. H. Talbot : Optimisation 26/59

31 Formulations in imaging Discrete and continuous approaches Both the previous discrete and continuous formulation have a MAP interpretation. Total Variation (TV) minimization: good regularization tool Weighted TV : penalization of the gradient leading to improved results Our contribution General combinatorial formulation of the dual TV problem : easily suitable to various graphs Generic constraint in the dual problem : more flexible penalization of the gradient sharper results H. Talbot : Optimisation 27/59

32 Formulations in imaging Outline 1 Generalization of TV models 2 Parallel Proximal Algorithm as an efficient solver 3 Results H. Talbot : Optimisation 28/59

33 DCTV model Total variation regularization Given an original image f Deduce a restored image u Weighted anisotropic TV model [Gilboa and Osher 2007] ( 1/2dx min w x,y (u y u x ) dy) (u x f x ) 2 dx u 2λ }{{}}{{} regularization R(u) data fidelity Φ(u) where λ ]0, + [ regularization parameter H. Talbot : Optimisation 29/59

34 DCTV model Equivalent dual formulation Weighted anisotropic TV model [Gilboa and Osher 2007] ( 1/2dx min w x,y (u y u x ) dy) 2 + Φ(u) u is equivalent [Chan, Golub, Mulet 1999] to the min-max problem min max wx,y 1/2 (u y u x )p x,y dxdy + Φ(u) u p 1 with p a projection vector field. Main idea p was introduced in practice to compute a faster solution constraining p can promote better results H. Talbot : Optimisation 30/59

35 DCTV model Segmentation Same model as denoising, with a labeled fidelity term Same regularisation. This includes very widespread models such as watershed, region growing, minimal curves and surfaces, geodesic active contours, and more. H. Talbot : Optimisation 31/59

36 DCTV model Deblurring, tomography Deblurring / tomography simply composes a linear term within the fidelity. Same model for regularization as before Possible to do very advanced applications: local tomography, angular integration tomography, dual image deblurring, etc. Also applicable with wavelets, etc. Any linear operator can serve. H. Talbot : Optimisation 32/59

37 Generalization of TV models Discrete formulation on graphs - notations Graph of N vertices, M edges Incidence matrix A R M N A = p 1 p 2 p 3 p 4 e e e e e A gradient operator A divergence operator allows general formulation of problems on arbitrary graphs For more details: L. Grady and J.R. Polimeni, Discrete Calculus: Applied Analysis on Graphs for Computational Science, Springer, H. Talbot : Optimisation 33/59

38 Generalization of TV models Discrete formulations of TV and its dual Let u R N be the restored image. [Bougleux et al. 2007] min u n ( ) 1/2 w i,j (u j u i ) 2 + Φ(u) j N i i=1 where N i = {j {1,..., n} e i,j E}. We introduce the following combinatorial formulation for the primal dual problem min u max p ((Au) w) + Φ(u) p 1, p R M H. Talbot : Optimisation 34/59

39 Generalization of TV models Dual constrained TV based formulation Constraining the projection vector Introducing the projection vector F R M = p w Constraining F to belong to a convex set C min sup u R N F C F (Au) }{{} regularization + 1 2λ u f 2 2 }{{} data fidelity C = m 1 i=1 C i where C 1,..., C m 1 closed convex sets of R M. Given g R N, θ i R M, α 1, C i = {F R M θ i F α g i }. H. Talbot : Optimisation 35/59

40 Generalization of TV models Dual constrained TV based formulation min sup u R N F C F (Au) }{{} regularization + 1 2λ u f 2 2 }{{} data fidelity C = m 1 i=1 C i, C i = {F R M θ i F α g i }, α 1. Example adapted to image denoising g i R N weight on vertex i, inversely function of the gradient of f at node i. Flat area : weak gradient strong g i strong F i,j weak local variations of u. Contours : strong gradient weak g i weak F i,j large local variations of u allowed. g j4 Fj4,i Fj1,i g j1 g i g j3 Fj3,i Fj2,i g j2 C i = {F R M Fj,i 2 g i } j N i H. Talbot : Optimisation 36/59

41 Generalization of TV models Illustration of constraining flow Illustration of constraining flow. H. Talbot : Optimisation 37/59

42 Generalization of TV models Sharper results Noisy image DCTV Weighted TV H. Talbot : Optimisation 38/59

43 Generalization of TV models Extension of our DCTV based formulation min sup u R N F C f R Q, observed image u R N, restored image F (Au) }{{} regularization + 1 2λ u f 2 2 }{{} data fidelity F R M, dual solution : projection vector H. Talbot : Optimisation 39/59

44 Generalization of TV models Extension of our DCTV based formulation min sup u R N F C f R Q, observed image u R N, restored image F (Au) }{{} regularization + 1 2λ Hu f 2 2 }{{} data fidelity F R M, dual solution : projection vector H R Q N, degradation matrix H. Talbot : Optimisation 39/59

45 Generalization of TV models Extension of our DCTV based formulation min sup u R N F C F (Au) }{{} regularization f R Q, observed image u R N, restored image + 1 2λ Hu f η 2 Ku 2 }{{} data fidelity F R M, dual solution : projection vector H R Q N, degradation matrix K R N N : projection onto Ker H, η 0 H. Talbot : Optimisation 39/59

46 Generalization of TV models Extension of our DCTV based formulation min sup u R N F C F (Au) }{{} regularization f R Q, observed image u R N, restored image (Hu f ) Λ 1 (Hu f ) + η 2 Ku 2 }{{} data fidelity F R M, dual solution : projection vector H R Q N, degradation matrix K R N N, projection onto Ker H, η 0 Λ R Q Q, matrix of weights, positive definite H. Talbot : Optimisation 39/59

47 Generalization of TV models Primal formulation min u R N σ C (Au) + 1 }{{} 2 (Hu f ) Λ 1 (Hu f ) + η 2 Ku 2 }{{} regularization data fidelity C = m 1 i=1 C i where C 1,..., C m 1 closed convex sets of R M. σ C support function of the convex set C σ C : R M ], + ]: a sup F a. F C H. Talbot : Optimisation 40/59

48 Generalization of TV models Dual problem The problem admits a unique solution û. Fenchel-Rockafellar dual problem: min F R M m 1 i=1 ι Ci (F ) +f m (F ) }{{} f i (F ) where ι C is the indicator function of the convex C (equal to 0 inside C and + outside), f m : F 1 2 F AΓA F F AΓH Λ 1 f, and Γ = (H Λ 1 H + ηk) 1. If F is a solution to the dual problem, ( û = Γ H Λ 1 f A F ). H. Talbot : Optimisation 41/59

49 Generalization of TV models Families of algorithms in continuous optimization Contour-based algorithms Snakes Level sets Region-based algorithms Primal only algorithms Primal-dual algorithms H. Talbot : Optimisation 42/59

50 Generalization of TV models Parallel ProXimal Algorithm (PPXA) optimizing DCTV [2] γ > 0, ν ]0, 2[. Repeat until convergence For (in parallel) { r = 1,..., s + 1 P Cr (y r ) if r s π r = (γaγa + I ) 1 (γaγh Λ 1 f + y s+1) otherwise z = 2 s+1 (π π s+1 ) F For (in parallel) r = 1,..., s + 1 yr = y r + ν(z p r ) F = F + ν 2 (z F ) H. Talbot : Optimisation 43/59

51 Generalization of TV models Parallel ProXimal Algorithm (PPXA) optimizing DCTV [2] γ > 0, ν ]0, 2[. Repeat until convergence For (in parallel) { r = 1,..., s + 1 P Cr (y r ) if r s π r = (γaγa + I ) 1 (γaγh Λ 1 f + y s+1) otherwise z = 2 s+1 (π π s+1 ) F For (in parallel) r = 1,..., s + 1 yr = y r + ν(z p r ) F = F + ν 2 (z F ) Simple projections onto hyperspheres H. Talbot : Optimisation 43/59

52 Generalization of TV models Parallel ProXimal Algorithm (PPXA) optimizing DCTV [2] γ > 0, ν ]0, 2[. Repeat until convergence For (in parallel) { r = 1,..., s + 1 P Cr (y r ) if r s π r = (γaγa + I ) 1 (γaγh Λ 1 f + y s+1) otherwise z = 2 s+1 (π π s+1 ) F For (in parallel) r = 1,..., s + 1 yr = y r + ν(z p r ) F = F + ν 2 (z F ) Linear system resolution H. Talbot : Optimisation 43/59

53 Generalization of TV models Quantitative perfomances Speed : competitive with the most efficient algorithm for optimizing weighted TV Denoising a image with an Alternated Direction of Multiplier Method: 0.4 seconds with the Parallel Proximal Algorithm: 0.7 seconds Quantitative denoising experiments on standard images show improvements of SNR (from 0.2 to 0.5 db) for images corrupted with Gaussian noise of variance σ 2 from 5 to 25. H. Talbot : Optimisation 44/59

54 Applications in image processing Results in image denoising Original image Noisy SNR=10.1dB Weighted TV SNR=13.4dB DCTV SNR=13.8dB H. Talbot : Optimisation 45/59

55 Applications in image processing Results in image denoising Weighted TV SNR=13.4dB DCTV SNR=13.8dB H. Talbot : Optimisation 46/59

56 Applications in image processing Comparison with more standard TV Figure : Left hand side: Standard deviation of each test image compared with the standard deviation of the denoising results, averaged results with (σ 2 = 5, 10, 15, 20, 25, 50). Right hand side: mean SNR over the experiments, H. Talbot : Optimisation 47/59

57 Applications in image processing Image denoising and deconvolution Original Noisy, blurred DCTV image image SNR=12.3dB result SNR=17.2dB H. Talbot : Optimisation 48/59

58 Applications in image processing Image fusion Original Noisy blurry DCTV image SNR=7.2dB SNR=11.6dB SNR=16.3dB H. Talbot : Optimisation 49/59

59 Applications in image processing Mesh denoising Original Noisy DCTV regularization mesh mesh on spatial coordinates H. Talbot : Optimisation 50/59

60 Applications in image processing (d) Noisy sampled (e) Taubin filtered (f) DCTV result H. Talbot : SNR Optimisation = 22.1 db result [?] SNR = 19.4 db (λ = 0.5) SNR 51/59 = 23.3 db Irregular graph (a) Original (b) Bottlenosed (c) Sampled image dolphin structure image

61 Applications in image processing Non-local regularization Original image Noisy PSNR=28.1dB (a) Nonlocal graph (figure P. Coupé, [?] Figure : Example of Non-Local Graph. Nonlocal DCTV PSNR=35 db H. Talbot : Optimisation 52/59

62 Non-convex regularization Non-convex optimization The current frontier. Many interesting applications thought to be very hard to solve: blind deblurring Many current methods extend to the Non-Convex case Generally only a local minimum is reached, but this might be OK. The miimum might be of high quality : stochastic optimization. For instance: see results achieved by deep-learning methods. H. Talbot : Optimisation 53/59

63 Non-convex regularization l 2 -l 0 regularization functions We consider the following class of potential functions: 1 ( δ (0, + )) ψ δ is differentiable. 2 ( δ (0, + )) lim t ψ δ (t) = 1. 3 ( δ (0, + )) ψ δ (t) = O(t 2 ) for small t. Examples: ψ δ (t) = t2 2δ 2 +t 2 ψ δ (t) = 1 exp( t2 2δ 2 ) H. Talbot : Optimisation 54/59

64 Non-convex regularization Majorize-Minimize principle [Hunter04] Objective: Find ˆx Arg min x F δ (x) For all x, let Q(., x ) a tangent majorant of F δ at x i.e., Q(x, x ) F δ (x), Q(x, x ) = F δ (x ) x, MM algorithm: j {0,..., J}, x j+1 Arg min x Q(x, x j ) H. Talbot : Optimisation 55/59

65 Non-convex regularization Image reconstruction Original image x Noisy sinogram y SNR=25 db { H Radon projection matrix y = H x + u with u Gaussian noise ( ˆx Arg min 1 x 2 Hx y 2 + λ c ψ δ(vc x) ) Non convex penalty / convex penalty H. Talbot : Optimisation 56/59

66 Non-convex regularization Results: Non convex penalty Reconstructed image SNR = 20.4 db MM-MG algorithm: Convergence in 134 s H. Talbot : Optimisation 57/59

67 Non-convex regularization Results: Convex penalty Reconstructed image SNR = 18.4 db MM-MG algorithm: Convergence in 60 s H. Talbot : Optimisation 58/59

68 Conclusion Conclusion Optimization is a very powerful, general methodology We ve drawn a panorama of interesting methodologies in image processing Extension of TV models via dual formulations Many applications in inverse problems including segmentation Proposed algorithm efficiently solves convex and non-convex problems Application to arbitrary graphs Generally optimization problems are unsolvable without some regularity assumptions. There exist a trade-off between the generality of a framework and the efficiency of associated algorithms. On to new things: hierarchies of partitions. H. Talbot : Optimisation 59/59

69 Conclusion S.P. Boyd and L. Vandenberghe. Convex optimization. Cambridge Univ Press, P. L. Combettes and J.-C. Pesquet. Proximal splitting methods in signal processing. In H. H. Bauschke, R. Burachik, P. L. Combettes, V. Elser, D. R. Luke, and H. Wolkowicz, editors, Fixed-Point Algorithms for Inverse Problems in Science and Engineering. Springer-Verlag, New York, Andrey Nikolayevich Tikhonov. On the stability of inverse problems. In Dokl. Akad. Nauk SSSR, volume 39, pages , H. Talbot : Optimisation 59/59