Regolarizzazione adattiva in spazi di Lebesgue a esponente variabile. Claudio Estatico Dipartimento di Matematica, Università di Genova, Italy

Transcription

1 GNCS 2018 Convegno Biennale del Gruppo Nazionale per il Calcolo Scientifico Montecatini Terme, febbraio 2018 Regolarizzazione adattiva in spazi di Lebesgue a esponente variabile Claudio Estatico Dipartimento di Matematica, Università di Genova, Italy Progetto di ricerca GNCS 2017: Metodi numerici non lineari per problemi inversi e applicazioni

2 Progetto di ricerca GNCS 2017: Metodi numerici non lineari per problemi inversi e applicazioni CNR Bari: Teresa Laudadio Univ. Bologna: Germana Landi, Damiana Lazzaro, Elena Loli Piccolomini, Fabiana Zama Univ. Cagliari: Luisa Fermo, Giuseppe Rodriguez; Anna Concas, Patricia Diaz De Alba Univ. Genova: Claudio Estatico, Alberto Sorrentino Univ. Insubria: Marco Donatelli; Davide Bianchi, Alessandro Buccini Univ. L Aquila: Pietro Dell Acqua Univ. Modena e Reggio Emilia: Marco Prato; Carla Bertocchi Univ. Padova: Elena Morotti Univ. Sapienza, Roma: Silvia Noschese RWTH Aachen University, Germany: Caterina Fenu

3 Outline I - Inverse Problems, variational approach and regularization by iterative methods in (Hilbert and) Banach spaces. II - A sketch on variable exponent Lebesgue spaces. III - Application of variable exponent Lebesgue spaces in iterative regularization methods. IV - Numerical results in image deblurring.

4 Inverse Problem By the knowledge of some observed data g (i.e., the effect), find an approximation of some model parameters f (i.e., the cause). Given the (noisy) data g G, find (an approximation of) the unknown f F such that Af = g where A : F G is a known linear operator, and F,G are two functional (here Hilbert or Banach) spaces. True image Blurred (noisy) image Restored image Inverse problems are usually ill-posed, they need regularization techniques.

5 Solution of inverse problems by minimization Variational approaches are very useful to solve the functional equation Af = g. These methods minimize the Tikhonov-type variational functional Φ α Φ α (f) = Af g p G + αr(f), where 1 < p < +, R : F [0, + ) is a (convex and proper) functional, and α > 0 is the regularization parameter. The data-fitting term Af g p G is called residual (usually in mathematics) or cost function (usually in engineering). The penalty term R(f) is often f q F, or f q F or Lf q F, for q 1 (such as the Hölder conjugate of p) and a differential operator L which measures the non-regularity of f.

6 Numerical linear algebra lives in Hilbert spaces A Hilbert space is a complete vector space endowed with a scalar product that allows (lengths and) angles to be defined and computed. In any Hilbert spaces, what happens in our Euclidean world (that is, in our part of universe...) always holds: Pythagorean theorem: if u v, then u + v 2 = u 2 + v 2 Parallelogram identity: u + v 2 + u v 2 = 2( u 2 + v 2 ) Gradient of (the square of) the norm: 1 2 u 2 = u The scalar product allows also for a natural definition of: orthogonal projection (best approximation) of u on v P (u) = u, v v/ v 2 SVD decomposition (or eigenvalues/eigenvectors dec.) of linear operators (for separable Hilbert spaces, as any finite dimensional Hilbert space...)

7 Regularization: the recent framework in Banach spaces Several regularization methods for ill-posed functional equations by variational approaches have been first formulated as minimization problems in Hilbert spaces (i.e., the classical approach). Later they have been extended to Banach spaces setting (i.e., the more recent approach). Examples: L 1 for sparse recovery or L p, 1 < p < 2 for edge restoration. Hilbert spaces Banach spaces L 2 (R n ), H s = W s,2 L p (R n ), W s,p, (p 1); BV Benefits Easier computation Better restoration (Spectral theory, of the discontinuities; eigencomponents) Sparse solutions Drawbacks Over-smoothness Theoretically tricky (bad localization of edges) (Convex analysis required)

8 Minimization of the residual by gradient-type it. methods For the simple residual functional Φ(f) = Af g p G, the basic minimization approach is the gradient-type iteration, which reads as f k+1 = f k τ k ψ(f k, g) where ψ(f k, g) ( Af g p G), i.e., ψ(f k, g) is an approximation of the (sub-)gradient of the residual functional Φ at point f k, and τ k > 0 is the step length. For the least square functional Φ(f) = 2 1 Af g 2 2 in L2 Hilbert space ) ) Φ(f) = Φ(f) = ((( 2 1 u 2 ) u=af g J (Af g) = ((Af g) A) = A (Af g) we have the simplest iterative method, i.e., the Landweber method, f k+1 = f k τa (Af k g) where τ (0, 2/ A 2 2 ) is a fixed step length.

9 From Hilbert to Banach spaces f k+1 = f k τa (Af k g) Formally, A is the dual operator of A, that is, the operator A : G F such that g (Af) = (A g )(f), f F and g G, where F and G are the dual spaces of F and G. If F and G are Hilbert spaces, then F is isometrically isomorph to F and G is isometrically isomorph to G (by virtue of Riesz Theorem), and the operator A : G F can be identified with A : G F. However, in general Banach spaces are not isometrically isomorph to their duals. This way, the Landweber iteration above is well defined in Hilbert spaces (...only!) The key point: To generalize from Hilbert to Banach spaces we have to consider the so-called duality maps.

10 Minimization in Banach spaces by duality maps A duality map is a function which allows us to associate (in a special way) an element of a Banach space B with an element (or a subset of elements) of its dual B as follows: Theorem (Asplund [1968]) Let B be a Banach space and p > 1. A duality map J B : B 2 B is the sub-differential of the convex functional f defined as f(b) = ( ) p 1 b p B : 1 J B = f = p p B. By chaining rule, the (sub-)differential of the residual Φ(f) = 1 p Af g p G, by means of the duality map J G : G 2 G, is the following ( ) 1 Φ(f) = p Af g p G = A J G (Af g).

11 Landweber iterative method in Hilbert spaces A : F G A : G F Φ(f) = 1 2 Af g 2 G f k+1 = f k τa (Af k g) Landweber iterative method in Banach spaces A : F G A : G F Φ(f) = p 1 Af g p G ( ) f k+1 = J F J F f k τ k A J G (Af k g) Some remarks. In the Banach space L p, with 1 < p < +, we have J L p(f) = f p 1 sgn(f). J L p is a non-linear, single-valued, diagonal operator, which cost O(n) operations, and does not increase the global numerical complexity O(n log n) of shift-invariant image restoration problems solved by FFT.

12 Landweber iterative method in Hilbert spaces A : F G A : G F Φ(f) = 1 2 Af g 2 G f k+1 = f k τa (Af k g) Landweber iterative method in Banach spaces A : F G A : G F Φ(f) = p 1 Af g p G ( ) f k+1 = J F J F f k τ k A J G (Af k g) Some remarks. Numerical linear algebra and related image restoration tools developed usually in Hilbert spaces are still useful in Banach spaces (preconditioning, trigonometric matrix algebras for boundary conditions, super-resolution... ). All is buried inside the discretized setting.

13 Duality maps are the basic tool for generalizing classical iterative methods for linear systems to Banach spaces: Landweber method, CG, Mann iterations, Gauss-Newton Gradient type iterations (for nonlinear problems). Basic hypotheses for the convergence (and regularization behavior): uniformly smooth and (uniformly) convex Banach spaces. [Schöpfer, Louis, Hein, Scherzer, Schuster, Kazimierski, Kaltenbacher, Q. Jin, Tautenhahn, Neubauer, Hofmann, Daubechies, De Mol, Fornasier, Tomba,....] To reduce over-smoothness, these methods have been implemented in the context of L p Lebesgue spaces with 1 < p 2. p > 1 Low regularization Good recovery of edges and discontinuities Improve the sparsity p 2 High regularization Higher stability Over-smoothness

14 A numerical evidence in L p Lebesgue space, 1 < p 2 A 2D Image restoration with Landweber method (200 iterations) True image x PSF Blurred and noisy image Hilbert Restoration (p = 2) Banach Restoration (p = 1.5) Banach Restoration (p = 1.2)

15 A new framework: variable exponent Lebesgue spaces L p( ) In image restoration, often different regions of the image require different amount of regularization. Setting different levels of regularization is useful because background, low intensity, and high intensity values require different filtering levels (see Nagy, Pauca, Plemmons, Torgersen, J Opt Soc Am A, 1997). The idea: the ill-posed functional equation Af = g is solved in L p( ) Banach spaces, namely, the variable exponent Lebesgue spaces, a special case of the so-called Musielak-Orlicz functional spaces (first proposed in two seminal papers in 1931 and 1959, but intensively studied just in the last 10 years). In a variable exponent Lebesgue space, to measure a function f, instead of a constant exponent p all over the domain, we have a pointwise variable (i.e., a distribution) exponent 1 p( ) +. This way, different regularization levels on different regions of the image to restore can be automatically and adaptively assigned.

16 A sketch on variable exponent Lebesgue spaces L p( ) L p (Ω) L p(x) (Ω) 1 p p(x) : Ω [1, ] p is constant p(x) is a measurable function ( 1/p ( ) 1/??? f p = Ω dx) f(x) p f p( ) = Ω f(x) p(x) dx f = ess sup f(x)... f L p (Ω) Ω f(x) p dx < f L p( ) (Ω)??? In the following, Ω = {x Ω : p(x) = } has zero measure.

17 Ω = [ 5, 5] p(x) = { 2 se 5 x 0 3 se 0 < x 5

18 Ω = [ 5, 5] p(x) = { 2 se 5 x 0 3 se 0 < x 5 f(x) = 1 x 1 1/3 / Lp( ) ([ 5, 5])

19 Ω = [ 5, 5] p(x) = { 2 se 5 x 0 3 se 0 < x 5 f(x) = 1 x+1 1/3 Lp( ) ([ 5, 5]) f(x) = 1 x 1 1/3 / Lp( ) ([ 5, 5])

20 The norm of variable exponent Lebesgue spaces In the conventional case L p, the norm is f L p = ( Ω f(x) p dx) 1/p. In L p( ) Lebesgue spaces, the definition and computation of the norm is not straightforward, since we have not a constant value for computing the ( mandatory ) radical. ( 1/???. f L p( ) = f(x) dx) p(x) The solution: compute first the modular (for 1 p( ) < + ) ρ(f) = f(x) p(x) dx, Ω Ω and then obtain the (so called Luxemburg [1955]) norm by solving a 1D minimization problem f L p( ) = inf { ( f λ > 0 : ρ λ) } 1.

21 The elements of a variable exponent Lebesgue space The Lebesgue space is a Banach space. ρ(f) = f L p( ) = inf L p( ) (Ω) = Ω f(x) p(x) dx, { ( f λ > 0 : ρ λ) } 1. { } f : Ω R f L p( ) < In the case of a constant function exponent p(x) = p, this norm is exactly the classical one f p, indeed ( f ( f(x) ) pdx 1 ρ = = λ) Ω λ λ p f(x) p dx = 1 Ω λ p f p p and { 1 } inf λ > 0 : λ p f p p 1 = f p

22 Modulus VS Norm In (classical) L p, norm and modulus are the same apart from a p-root: f p < f(x) p dx < In L p( ), norm and modulus are really different: f p( ) < ρ(f) < Ω Indeed, the following holds ( f f p( ) < there exist a λ > 0 s.t. ρ λ) < (and notice that λ can be chosen large enough... ).

23 A simple example of a strange behavior ρ(f) = Ω = [1, ) f(x) 1 p(x) = x 1 Indeed ρ(f/λ) = 1 x dx = BUT f p( ) ( 1 λ ) x dx = 1 λ log λ <, for any λ > 1

24 The vector case: the Lebesgue spaces of sequences l p( ) = (l p n n ) The unit circle of x = (x 1 ; x 2 ) in R 2 with variable exponents p = (p 1 ; p 2 ). x p x p( ) Inclusion if (p 1, p 2 ) (q 1, q 2 ) (as classical) No inclusion in general

25 Properties of variable exponent Lebesgue spaces L p( ) Let p = ess inf Ω p(x), and p + = ess sup Ω p(x). If p + =, then L p( ) (Ω) is a bad (although very interesting) Banach space, with poor geometric properties (i.e., not useful for our regularization schemes). If 1 < p p + <, then L p( ) (Ω) is a good Banach space, since many properties of classical Lebesgue spaces L p still hold. This is the natural framework for our iterative methods in Banach spaces, because: L p( ) is uniformly smooth, uniformly convex, and reflexive, ( its dual space is well defined, L p( )) L q( ), where 1 p(x) + q(x) 1 = 1, we can define its duality map.

26 The duality map of the variable exponent Lebesgue space By extending the duality maps, we can define into L p( ) all the iterative methods developed in L p (Landweber, Steepest descent, CG, Mann iter.). For any constant 1 < r < +, we recall that the duality map, that is, the (sub-)differential of the functional r 1 f r L p, in the classical Banach space L p, with constant 1 < p < +, is defined as follows ( J L p(f) ) (x) = f(x) p 1 sgn(f(x)) f p r p By generalizing a result of P. Matei [2012], we have that the corresponding duality map in variable exponent Lebesgue space is defined as follows ( ) J L p( )(f) (x) = Ω 1 p(x) f(x) p(x) f p(x) p( ) dx p(x) f(x) p(x) 1 sgn(f(x)) f p(x) r p( ) where any product and any ratio have to be considered as pointwise..,

27 The adaptive algorithm in variable exponent Lebesgue spaces It is a numerical evidence that, in L p image deblurring, dealing with small 1 p << 2 improves sparsity and allows a better restoration of the edges of the images and of the zero-background, dealing with p 2 (even p > 2), allows a better restoration of the regions of pixels with the highest intensities. The idea: to use a scaled into [1, 2] version of the (re-)blurred data g as distribution of the exponent p( ) for the variable exponent Lebesgue spaces L p( ) where computing the solution. Example (linear interpolation): p( ) = 1 + [A g( ) min(a g)]/[max(a g) min(a g)] The Landweber (i.e., fixed point) iterative scheme in this L p( ) Banach space can be modified as adaptive iteration algorithm, by recomputing, after each fixed number of iterations, the exponent distribution p k ( ) by means of the k-th restored image f k (instead of the first re-blurred data Ag), that is p k ( )= 1 + [f k ( ) min(f k )]/[max(f k ) min(f k )]

28 The conjugate gradient method in L p( ) for image restoration Let p( ) = 1 + [A g( ) min(a g)]/[max(a g) min(a g)] ρ 0 = A J Lp( ) r (Af 0 g) For k = 1, 2, 3,... α k = arg min α 1 r A(f k + αρ k ) g r L p( ) f k+1 = f k + α kρ k f k+1 = J (Lp( ) ) s (f k+1 ) β k+1 = γ Af k+1 g r L p( ) Af k g r L p( ) with γ < 1/2 ρ k+1 = A J Lp( ) r (Af k+1 g) + β k+1 ρ k and recompute p( ) = 1 + [f k ( ) min(f k )]/[max(f k ) min(f k )] each m iterations by using the last iteration f k. [Proof of convergence for L p (constant) Lebesgue spaces in 2017].

29 Numerical results

30 True image Point Spread Function Blurred image (noise = 4.7%) Special thanks to Brigida Bonino (Univ. Genova).

31 True image Blurred (noise = 4.7%) p = 2 (0.2692) p = 1.3 (0.2681) p = (0.2473) and irreg. (0.2307)

32 True image Point Spread Function Blurred image (noise = 4.7%)

33 A remote sensing application: spatial resolution enhancement of microwave radiometer data It is an under-determined problem (with a special structured matrix), since we want to reconstruct the High frequency components reduced by the Low Pass filter of the radiometer. Joint work with: Matteo Alparone, Flavia Lenti, Maurizio Migliaccio, Nando Nunziata (Dip. di Ingegneria, Univ. Napoli Parthenope)

34 RECT E DOUBLE RECT p=2 p= Variable p pmin = 1.2; pmax = 2; lambda = 0.3; maxiter = 2000; ethr = 0.12; fkind = 1; Variable p pmin = 1.2; pmax = 2; lambda = 0.3; maxiter = 2000; ethr = 0.12; fkind = 1;

35 300 Signal Res En Constant p = 1.2 xconst TB Res En variable p xvar TB

36 Conclusions Basic tools for regularization methods in Banach spaces have been reviewed, and the variable exponent Lebesgue spaces has been introduced. Iterative methods in variable exponent Lebesgue spaces can give adaptive regularization. The framework is new (previously only used in the penalty term as weighted total variation (f) p( ) dx in [T. Chan et al 1997], without any link to Banach space setting). TO DO: Theoretical convergence is under analysis. TO DO: Application of L p( ) duality maps for the acceleration of R-L and ISRA has given very good numerical result. It should be better analyzed.

37 Thank you for your attention

38 References [1] F. Schöpfer, A.K. Louis, T. Schuster, Nonlinear iterative methods for linear ill-posed problems in Banach spaces, Inverse Problems, vol. 22, pp , [2] M. O. Scherzer, M. Grasmair, H. Grossauer, M. Haltmeier, F. Lenzen, Variational Methods in Imaging, Springer, Berlin, [3] L. Diening, P. Harjulehto, P. Hästö, M. Ruzicka, Lebesgue and Sobolev Spaces with Variable Exponents, Lecture Notes in Mathematics, vol. 2017, Springer, [4] Schuster, T., Kaltenbacher, B., Hofmann, B., and Kazimierski, K. S Regularization Methods in Banach Spaces. Radon Series on Computational and Applied Mathematics, vol. 10, De Gruyter. [5] P. Brianzi, F. Di Benedetto, C. Estatico, Preconditioned iterative regularization in Banach spaces, Comp. Optim. Appl., vol. 54, pp , [6] P. Dell Acqua, C. Estatico, Acceleration of multiplicative iterative algorithms for image deblurring by duality maps in Banach spaces, Appl. Num. Math., vol. 99, pp , [7] C. Estatico, S. Gratton, F. Lenti, D. Titley-Peloquin, A conjugate gradient like method for p-norm minimization in functional spaces, Numer. Math., vol. 137, pp , 2017.