A Double Regularization Approach for Inverse Problems with Noisy Data and Inexact Operator

Size: px

Start display at page:

Download "A Double Regularization Approach for Inverse Problems with Noisy Data and Inexact Operator"

Dina Welch
5 years ago
Views:

1 A Double Regularization Approach for Inverse Problems with Noisy Data and Inexact Operator Ismael Rodrigo Bleyer Prof. Dr. Ronny Ramlau Johannes Kepler Universität - Linz Florianópolis - September, Doctoral Program Computational Mathematics Numerical Analysis and Symbolic Computation supported by Bleyer, Ramlau JKU Linz 1/ 27

2 Overview Introduction Proposed method: DBL-RTLS Computational aspects Numerical illustration Outline and future work Bleyer, Ramlau JKU Linz 2/ 27

3 Introduction Overview Introduction Proposed method: DBL-RTLS Computational aspects Numerical illustration Outline and future work Bleyer, Ramlau JKU Linz 2/ 27

4 Introduction Inverse problems Inverse problems are concerned with determining causes for a desired or an observed effect [Engl, Hanke, and Neubauer, 2000] Consider a linear operator equation Ax = y. Inverse problems most oft do not fulfill Hadamard s postulate [1902] of well posedness (existence, uniqueness and stability). Computational issues: observed effect has measurement errors or perturbations caused by noise. Bleyer, Ramlau JKU Linz 3/ 27

Introduction 1st Case: noisy data Solve Ax = y 0 out of the measurement y δ with y0 y δ δ. Need apply some regularization technique minimize Ax yδ 2 +α Lx 2.

5 Introduction 1st Case: noisy data Solve Ax = y 0 out of the measurement y δ with y0 y δ δ. Need apply some regularization technique minimize Ax yδ 2 +α Lx 2. x Tikhonov regularization fidelity term (based on LS); regularization parameter α; stabilization term (quadratic). [Tikhonov, 1963, Phillips, 1962] Bleyer, Ramlau JKU Linz 4/ 27

6 Introduction 1st Case: noisy data Solve Ax = y 0 out of the measurement y δ with y0 y δ δ. Need apply some regularization technique minimize Ax yδ 2 +αr(x). x Tikhonov-type regularization fidelity term (based on LS); regularization parameter α; R is a proper, convex and weakly lower semicontinuous functional. [Burger and Osher, 2004, Resmerita, 2005] Bleyer, Ramlau JKU Linz 4/ 27

7 Introduction Subgradient The Fenchel subdifferential of a functional R : U [0,+ ] at ū U is the set F R(ū) = {ξ U R(v) R(ū) ξ, v ū v U}. First in 1960 by Moreau & Rockafellar and extended by Clark Optimality condition: If ū minimizes R then 0 F R(ū) Bleyer, Ramlau JKU Linz 5/ 27

8 Introduction Example Consider the function R(u) = u Figure: Function (left) and its subdifferential (right). Bleyer, Ramlau JKU Linz 6/ 27

9 Introduction 2nd Case: inexact operator and noisy data Solve A 0 x = y 0 under the assumptions (i) noisy data y0 y δ δ. (ii) inexact operator A0 A ǫ ǫ. What have been done so far? Linear case - based on TLS [Golub and Van Loan, 1980]: R-TLS: Regularized TLS [Golub et al., 1999]; D-RTLS: Dual R-TLS [Lu et al., 2007]. Nonlinear case: no publication (?) LS: y δ and A 0 minimize y y yδ 2 subject to y R(A 0 ) TLS: y δ and A ǫ minimize [A,y] [Aǫ,y δ ] F subject to y R(A) Bleyer, Ramlau JKU Linz 7/ 27

10 Introduction 2nd Case: inexact operator and noisy data Solve A 0 x = y 0 under the assumptions (i) noisy data y0 y δ δ. (ii) inexact operator A0 A ǫ ǫ. What have been done so far? Linear case - based on TLS [Golub and Van Loan, 1980]: R-TLS: Regularized TLS [Golub et al., 1999]; D-RTLS: Dual R-TLS [Lu et al., 2007]. Nonlinear case: no publication (?) LS: y δ and A 0 minimize y y yδ 2 subject to y R(A 0 ) TLS: y δ and A ǫ minimize [A,y] [Aǫ,y δ ] F subject to y R(A) Bleyer, Ramlau JKU Linz 7/ 27

11 Introduction Illustration Solve 1D problem: am = b, find the slope m. 3 TLS vs LS 2.5 Given: 1. b δ, a ǫ (red) Solution: 1. LS solution (blue) 2. TLS solution (green) LS solution TLS solution noisy data true data slope Example: arctan(1) = 45 o [Van Huffel and Vandewalle, 1991] Bleyer, Ramlau JKU Linz 8/ 27

12 Introduction R-TLS The R-TLS method [Golub, Hansen, and O leary, 1999] minimize A Aǫ 2 + y yδ 2 { subject to Ax = y Lx 2 M. If the inequality constraint is active, then ( A T ǫ A ǫ +αl T L+βI )ˆx = A T ǫ y δ and Lˆx = M with α = µ(1+ ˆx 2 ), β = Aǫˆx y δ 2 multiplier. 1+ ˆx 2 and µ > 0 is the Lagrange Difficulty: requires a reliable bound M for the norm Lx 2. Bleyer, Ramlau JKU Linz 9/ 27

13 Proposed method: DBL-RTLS Overview Introduction Proposed method: DBL-RTLS Computational aspects Numerical illustration Outline and future work Bleyer, Ramlau JKU Linz 9/ 27

14 Proposed method: DBL-RTLS Consider the operator equation B(k,f) = g 0 where B is a bilinear operator (nonlinear) B : U V H (k,f) B(k,f) and B is characterized by a function k 0. K = B( k, ) compact linear operator for a fixed k U F = B(, f) linear operator for a fixed f V B(k0, ) V H C k0 U ; B(k,f) H C k U f V ; Example: B(k,f)(s) := Ω k(s,t)f(t)dt. Bleyer, Ramlau JKU Linz 10/ 27

15 Proposed method: DBL-RTLS Consider the operator equation B(k,f) = g 0 where B is a bilinear operator (nonlinear) B : U V H (k,f) B(k,f) and B is characterized by a function k 0. K = B( k, ) compact linear operator for a fixed k U F = B(, f) linear operator for a fixed f V B(k0, ) V H C k0 U ; B(k,f) H C k U f V ; Example: B(k,f)(s) := Ω k(s,t)f(t)dt. Bleyer, Ramlau JKU Linz 10/ 27

16 Proposed method: DBL-RTLS We want to solve B(k 0,f) = g 0 out of the measurements k ǫ and g δ with (i) noisy data g 0 g H δ δ. (ii) inexact operator k0 k U ǫ ǫ. We introduce the DBL-RTLS minimize k,f J (k,f) := T(k,f,k ǫ,g δ )+R(k,f) where T measures of accuracy (closeness/discrepancy) R promotes stability. Bleyer, Ramlau JKU Linz 11/ 27

17 Proposed method: DBL-RTLS DBL-RTLS where minimize k,f J (k,f) := T(k,f,k ǫ,g δ )+R(k,f) (1) T(k,f,k ǫ,g δ ) = 1 B(k,f) gδ 2 2 H + γ k kǫ 2 2 U R(k,f) = α Lf 2 2 V +βr(k) T is based on TLS method, measures the discrepancy on both data and operator; L : V V is a linear bounded operator; α, β are the regularization parameters and γ is a scaling parameter; double regularization [You and Kaveh, 1996], R : U [0,+ ] is proper convex function and w-lsc. Bleyer, Ramlau JKU Linz 12/ 27

18 Proposed method: DBL-RTLS Main theoretical results Assumption: (A1) B is strongly continuous, ie, if (k n,f n ) ( k, f) then B(k n,f n ) B( k, f) Proposition Let J be the functional defined on (1) and L be a bounded and positive operator. Then J is positive, weak lower semi-continuous and coercive functional. Theorem (existence) Let the assumptions of Proposition 1 hold. Then there exists a global minimum of minimize J (k,f). Bleyer, Ramlau JKU Linz 13/ 27

19 Proposed method: DBL-RTLS Theorem (stability) δ j δ and ǫ j ǫ g δj g δ and k ǫj k ǫ α,β > 0 (k j,f j ) is a minimizer of J with g δj and k ǫj Then there exists a convergent subsequence of (k j,f j ) j (k jm,f jm ) ( k, f) where ( k, f) is a minimizer of J with g δ,k ǫ,α and β. Bleyer, Ramlau JKU Linz 14/ 27

20 Proposed method: DBL-RTLS Theorem (stability) δ j δ and ǫ j ǫ g δj g δ and k ǫj k ǫ α,β > 0 (k j,f j ) is a minimizer of J with g δj and k ǫj Then there exists a convergent subsequence of (k j,f j ) j (k jm,f jm ) ( k, f) where ( k, f) is a minimizer of J with g δ,k ǫ,α and β. Bleyer, Ramlau JKU Linz 14/ 27

21 Proposed method: DBL-RTLS Consider the convex functional Φ(k,f) := 1 Lf 2 +ηr(k) 2 where the parameter η represents the different scaling of f and k. For convergence results we need to define Definition We call (k,f ) a Φ-minimizing solution if (k,f ) = argmin{φ(k,f) B(k,f) = g 0 }. (k,f) Bleyer, Ramlau JKU Linz 15/ 27

22 Proposed method: DBL-RTLS Theorem (convergence) δ j 0 and ǫ j 0 gδj g 0 δj and kǫj k 0 ǫj α j = α(ǫ j,δ j ) and β j = β(ǫ j,δ j ), s.t. α j 0, β j 0, lim j δ 2 j +γǫ2 j α j = 0 and lim j β j α j = η (k j,f j ) is a minimizer of J with g δj, k ǫj, α j and β j Then there exists a convergent subsequence of (k j,f j ) j (k jm,f jm ) (k,f ) where (k,f ) is a Φ-minimizing solution. Bleyer, Ramlau JKU Linz 16/ 27

23 Proposed method: DBL-RTLS Theorem (convergence) δ j 0 and ǫ j 0 gδj g 0 δj and kǫj k 0 ǫj α j = α(ǫ j,δ j ) and β j = β(ǫ j,δ j ), s.t. α j 0, β j 0, lim j δ 2 j +γǫ2 j α j = 0 and lim j β j α j = η (k j,f j ) is a minimizer of J with g δj, k ǫj, α j and β j Then there exists a convergent subsequence of (k j,f j ) j (k jm,f jm ) (k,f ) where (k,f ) is a Φ-minimizing solution. Bleyer, Ramlau JKU Linz 16/ 27

24 Computational aspects Overview Introduction Proposed method: DBL-RTLS Computational aspects Numerical illustration Outline and future work Bleyer, Ramlau JKU Linz 16/ 27

25 Computational aspects Optimality condition If the pair ( k, f) is a minimizer of J (k,f), then (0,0) J ( k, f). Theorem Let J : U V R be a nonconvex functional, J(u,v) = ϕ(u)+q(u,v)+ψ(v) where Q is a nonlinear differentiable term and ϕ, ψ are lsc convex functions. Then J(u,v) = { ϕ(u)+d u Q(u,v)} { ψ(v)+d v Q(u,v)} = { u J(u,v)} { v J(u,v)} Bleyer, Ramlau JKU Linz 17/ 27

26 Computational aspects Remark: is difficult to solve wrt both (k,f) J is bilinear and biconvex (linear and convex to each one) applied alternating minimization method. Alternating minimization algorithm Require: g δ,k ǫ,l,γ,α,β 1: n = 0 2: repeat 3: f n+1 argmin f J(k,f k n ) 4: k n+1 argmin k J(k,f f n+1 ) 5: until convergence Bleyer, Ramlau JKU Linz 18/ 27

27 Computational aspects Remark: is difficult to solve wrt both (k,f) J is bilinear and biconvex (linear and convex to each one) applied alternating minimization method. Alternating minimization algorithm Require: g δ,k ǫ,l,γ,α,β 1: n = 0 2: repeat 3: f n+1 argmin f J(k,f k n ) 4: k n+1 argmin k J(k,f f n+1 ) 5: until convergence Bleyer, Ramlau JKU Linz 18/ 27

28 Computational aspects Proposition The sequence generated by the function J(k n,f n ) is non-increasing, J(k n+1,f n+1 ) J(k n,f n+1 ) J(k n,f n ). Assumptions: (A1) B is strongly continuous, ie., if (k n,f n ) ( k, f) then B(k n,f n ) B( k, f) (A2) B is weakly sequentially closed, ie., if (k n,f n ) ( k, f) and B(k n,f n ) g then B( k, f) = g (A3) the adjoint of B is strongly continuous, ie., if (k n,f n ) ( k, f) then B (k n,f n ) z B ( k, f) z, z D(B ) Bleyer, Ramlau JKU Linz 19/ 27

29 Computational aspects Proposition The sequence generated by the function J(k n,f n ) is non-increasing, J(k n+1,f n+1 ) J(k n,f n+1 ) J(k n,f n ). Assumptions: (A1) B is strongly continuous, ie., if (k n,f n ) ( k, f) then B(k n,f n ) B( k, f) (A2) B is weakly sequentially closed, ie., if (k n,f n ) ( k, f) and B(k n,f n ) g then B( k, f) = g (A3) the adjoint of B is strongly continuous, ie., if (k n,f n ) ( k, f) then B (k n,f n ) z B ( k, f) z, z D(B ) Bleyer, Ramlau JKU Linz 19/ 27

30 Computational aspects Theorem Given regularization parameters 0 < α α and β, compute AM algorithm. The sequence {(k n+1,f n+1 )} n+1 has a weakly convergent subsequence, namely (k n j+1,f n j+1 ) ( k, f) and the limit has the property J( k, f) J( k,f) and J( k, f) J(k, f) for all f V and for all k U. Proposition Let {(k n,f n )} n be a weakly convergent sequence generated by AM algorithm, where k n k and f n f. Then there exists a subsequence {k n j } nj such that k n j k and there exists {ξ n j k } n j with ξ n j k kj(k n j,f n j ) such that ξ n j k 0. Bleyer, Ramlau JKU Linz 20/ 27

31 Computational aspects Theorem Given regularization parameters 0 < α α and β, compute AM algorithm. The sequence {(k n+1,f n+1 )} n+1 has a weakly convergent subsequence, namely (k n j+1,f n j+1 ) ( k, f) and the limit has the property J( k, f) J( k,f) and J( k, f) J(k, f) for all f V and for all k U. Proposition Let {(k n,f n )} n be a weakly convergent sequence generated by AM algorithm, where k n k and f n f. Then there exists a subsequence {k n j } nj such that k n j k and there exists {ξ n j k } n j with ξ n j k kj(k n j,f n j ) such that ξ n j k 0. Bleyer, Ramlau JKU Linz 20/ 27

32 Computational aspects Proposition Let {n} be a subsequence of N such that the sequence {(k n,f n )} n generated by AM algorithm satisfies k n k and f n f. Then f n j f and there exists {ξ n j f } n j with ξ n j f f J(k n j,f n j ) such that ξ n j f 0. Remark: Graph of subdifferential mapping is sw-closed, ie., if v n v and ξ n ξ with ξ n ϕ(v n ), then ξ ϕ( v). Theorem Let {(k n,f n )} n be the sequence generated by the AM algorithm, then there exists a subsequence converging towards to a critical point of J, ie., (0,0) J ( k, f). Bleyer, Ramlau JKU Linz 21/ 27

33 Computational aspects Proposition Let {n} be a subsequence of N such that the sequence {(k n,f n )} n generated by AM algorithm satisfies k n k and f n f. Then f n j f and there exists {ξ n j f } n j with ξ n j f f J(k n j,f n j ) such that ξ n j f 0. Remark: Graph of subdifferential mapping is sw-closed, ie., if v n v and ξ n ξ with ξ n ϕ(v n ), then ξ ϕ( v). Theorem Let {(k n,f n )} n be the sequence generated by the AM algorithm, then there exists a subsequence converging towards to a critical point of J, ie., (0,0) J ( k, f). Bleyer, Ramlau JKU Linz 21/ 27

34 Numerical illustration Overview Introduction Proposed method: DBL-RTLS Computational aspects Numerical illustration Outline and future work Bleyer, Ramlau JKU Linz 21/ 27

35 Numerical illustration First numerical result Convolution in 1D Ω k(s t)f(t)dt = g(s) characteristic kernel and hat function; space: Ω = [0,1], discretization: N = 2048 points; R(k) = k w,p with p = 1 Haar wavelet for {φ} λ and J = 10; initial guess: k 0 = k ǫ, τ = 1.0; 1st. relative error: 10% and 10%. 2nd. relative error: 0.1% and 0.1%. Bleyer, Ramlau JKU Linz 22/ 27

36 Numerical illustration 1.2 noisy kernel 1.2 solution 0.2 noisy data approach kernel 1.2 solution function 0.2 computed data kernel: ker 1.2 function: func 0.2 data: convolution Bleyer, Ramlau JKU Linz 23/ 27

37 Numerical illustration 1.2 noisy kernel 1 solution 0.2 noisy data approach kernel 1.2 solution function 0.2 computed data kernel: ker 1.2 function: func 0.2 data: convolution Bleyer, Ramlau JKU Linz 24/ 27

38 Outline and future work Overview Introduction Proposed method: DBL-RTLS Computational aspects Numerical illustration Outline and future work Bleyer, Ramlau JKU Linz 24/ 27

39 Outline and future work Outline and future work So far: introduced a method for nonlinear equation (bilinear operator) with noisy data and inexact operator; proved existence, stability and convergence; study of source conditions and convergence rates (k and f); suggested an iterative implementation; proved convergence of AM algorithm to a critical point; For further work: study variational inequalities; how to choose the best regularization parameter? a priori and a posteriori choice; implementations and numerical experiments (2D); Bleyer, Ramlau JKU Linz 25/ 27

40 Outline and future work Outline and future work So far: introduced a method for nonlinear equation (bilinear operator) with noisy data and inexact operator; proved existence, stability and convergence; study of source conditions and convergence rates (k and f); suggested an iterative implementation; proved convergence of AM algorithm to a critical point; For further work: study variational inequalities; how to choose the best regularization parameter? a priori and a posteriori choice; implementations and numerical experiments (2D); Bleyer, Ramlau JKU Linz 25/ 27

41 Outline and future work M. Burger and S. Osher. Convergence rates of convex variational regularization. Inverse Problems, 20(5): , ISSN doi: / /20/5/005. URL H. W. Engl, M. Hanke, and A. Neubauer. Regularization of Inverse Problems. Kluwer Academic Publishers, Dordrecht, G. H. Golub and C. F. Van Loan. An analysis of the total least squares problem. SIAM J. Numer. Anal., 17(6): , ISSN G. H. Golub, P. C. Hansen, and D. P. O leary. Tikhonov regularization and total least squares. SIAM J. Matrix Anal. Appl, 21: , S. Lu, S. V. Pereverzev, and U. Tautenhahn. Regularized total least squares: computational aspects and error bounds. Technical Report 30, Ricam, Linz, Austria, URL D. L. Phillips. A technique for the numerical solution of certain integral equations of the first kind. J. Assoc. Comput. Mach., 9:84 97, ISSN E. Resmerita. Regularization of ill-posed problems in Banach spaces: convergence rates. Inverse Problems, 21(4): , ISSN doi: / /21/4/007. URL A. N. Tikhonov. On the solution of incorrectly put problems and the regularisation method. In Outlines Joint Sympos. Partial Differential Equations (Novosibirsk, 1963), pages Acad. Sci. USSR Siberian Branch, Moscow, S. Van Huffel and J. Vandewalle. The total least squares problem, volume 9 of Frontiers in Applied Mathematics. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, ISBN Computational aspects and analysis, With a foreword by Gene H. Golub. Y.-L. You and M. Kaveh. A regularization approach to joint blur identification and image restoration. Image Processing, IEEE Transactions on, 5(3): , mar ISSN Bleyer, Ramlau JKU Linz 26/ 27

42 Outline and future work Thank you for your kind attention! Bleyer, Ramlau JKU Linz 27/ 27

A Double Regularization Approach for Inverse Problems with Noisy Data and Inexact Operator

A Double Regularization Approach for Inverse Problems with Noisy Data and Inexact Operator Ismael Rodrigo Bleyer Prof. Dr. Ronny Ramlau Johannes Kepler Universität - Linz Cambridge - July 28, 211. Doctoral