Iterative Methods for Ill-Posed Problems

Transcription

1 Iterative Methods for Ill-Posed Problems Based on joint work with: Serena Morigi Fiorella Sgallari Andriy Shyshkov Salt Lake City, May, 2007

2 Outline: Inverse and ill-posed problems Tikhonov regularization Regularization by truncated iteration Multigrid methods Application to image restoration

3 Inverse problems Inverse problems arise when one seeks to determine the cause of an observed effect. Helioseismology: Determine the structure of the sun by measurements from earth or space. Medical imaging, e.g., electrocardiographic imaging, computerized tomography. Image restoration: Determine the unavailable exact image from an available contaminated version. Inverse problems often are ill-posed.

4 Ill-posed problems A problem is said to be ill-posed if it has at least one of the properties: the problem does not have a solution, the problem does not have a unique solution, the solution does not depend continuously on the data.

5 Linear discrete ill-posed problems Ax = b arise from the discretization of linear ill-posed problems (Fredholm integral equations of the first kind) or, naturally, in discrete form (image restoration). The matrix A is of ill-determined rank, possibly singular. System may be inconsistent. The right-hand side b represents available data that generally is contaminated by an error.

6 Available contaminated, possibly inconsistents, linear system Ax = b (1) Unavailable associated consistent linear system with error-free right-hand side Ax = ˆb (2) Let ˆx denote the desired solution of (2), e.g., the minimal-norm solution. Task: Determine an approximate solution of (1) that is a good approximation of ˆx.

7 Define the error e = ˆb b noise and let δ = e How much damage can a little noise really do? How much noise requires the use of special techniques?

8 Example: Fredholm integral equation of the 1st kind π 0 exp( st)x(t)dt = 2 sinh(s) s, 0 s π 2. Determine solution x(t) = sin(t). Discretize integral by Galerkin method using piecewise constant functions. Code baart from Regularization Tools.

9 This gives a linear system of equations Ax = ˆb, A R , ˆb R 200. A is numerically singular. Let the noise vector e in b have normally distributed entries with mean zero and δ = e = 10 3 b b = ˆb + e i.e., 0.1% relative noise

10 0.26 right hand side no added noise added noise/signal=1e

11 0.14 exact solution

12 1.5 x 1013 Solution Ax=b: noise level

13 Tikhonov regularization

14 Solve the Tikhonov minimization problem where A R m n ; min x { Ax b 2 + µ Lx 2 }, L R p n, p n, is the regularization operator. Common choices: L = I or a finite difference operator; µ > 0 is the regularization parameter to be determined.

15 Normal equations associated with Tikhonov minimization problem: (A T A + µl T L)x = A T b. Have unique solution iff N (A) N (L) = {0}.

16 Important to determine a suitable value of µ: L = I: Solution x µ := (A T A + µi) 1 A T b. lim µ 0 x µ = A b, lim µ x µ = 0.

17 Regularization by truncated iteration

18 CGNR: CG applied to A Ax = A b Define the Krylov subspace K k (A A, A b) = span{a b, (A A)A b,..., (A A) k 2 A b, (A A) k 1 A b}. Then x k K k (A A, A b) and Ax k b = min Ax b x K k (A A,A b) Therefore discrepancy d j = b Ax j satisfies b d 1... d k.

19 Stopping Criterion Discrepancy principle Let α > 1 be fixed, e = ˆb b = δ. The iterate x k satisfies the discrepancy principle if Ax k b αδ Stopping rule Terminate the iterations as soon as iterate x k satisfies Ax k b αδ Ax k 1 b > αδ Denote the termination index by k δ.

20 An iterative method is a regularization method if lim sup δ 0 e δ x kδ ˆx = 0 CGNR is a regularization method; see Nemirovskii, Hanke.

21 A multilevel method

22 Consider Ω k(s, t)x(s)ds = b(t), t Ω Let S 1 S 2... S k L 2 (Ω) nested subspaces R i : L 2 (Ω) S i restriction operator b i = R i b, b δ i = R i b δ, A i = R i AR i P i : S i 1 S i prolongation operator

23 Cascadic multilevel method: - Solve integral equation in S 1, map solution to S 2, - Solve integral equation in S 2 for correction, map to S 3, - Solve integral equation in S 3 for correction, map to S 4... Assume that b b δ = δ and b i b δ i cδ, c > 1, i = 1, 2,...

24 Apply CGNR or MR-II in S 1. Yields iterates x 1,j, j = 1, 2,.... Terminate iterations as soon as A 1 x 1,j b δ 1 cδ. Proceed similarly on higher levels. Theorem: The multilevel method outlined is a regularization method.

25 Example: Baart integral equation discretized by trapeziodal rule on mesh with 1025 point. δ b One-Grid CGNR m(δ) x δ 8,m(δ) ˆx ˆx

26 δ b Multilevel CGNR m i (δ) x δ 8,m 8 (δ) ˆx ˆx , 1, 1, 1, 1, 1, 1, , 2, 1, 3, 1, 1, 1, , 3, 2, 1, 1, 1, 1, , 3, 3, 3, 2, 1, 1,

27 Noise level 10 3 : CGNR (red curve), ML-CGNR (green curve), exact solution (blue curve)

28 Example: Phillips integral equation discretized by trapeziodal rule on mesh with 1025 point. δ b One-Grid CGNR m(δ) x δ 8,m(δ) ˆx ˆx

29 δ b Multilevel CGNR m i (δ) x δ 8,m 8 (δ) ˆx ˆx , 1, 1, 1, 1, 1, 1, , 5, 3, 2, 1, 1, 1, , 6, 6, 4, 3, 1, 1, , 13, 9, 9, 5, 4, 3,

30 Application to image restoration

31 Nonlinear Prolongation Operators Return to Tikhonov regularization: { } min (h u f δ ) 2 + µr(u)dx u Ω Associated Euler-Lagrange equations: u t = h (h u f δ ) + µ D(u), Example: u 0 = f δ R(u) = u 2 = D(u) = 2 u R(u) = u = D(u) = div( u u ) TV operator

32 We consider a weighted total variation operator: ( ) u D 1 (u) = u q ɛ u q, u q ɛ = ( u 2 + ɛ 2) q/2 ɛ with 1 q 2, and a Perona-Malik operator: D 2 (u) = (g( u 2 ) u), g(s 2 ) = 1/(1 + s 2 /ρ 2 ) with g the diffusivity function.

33 The prolongation operators are determined by integration of for j = 1 or j = 2. u t = D j(u) Example: Restoration pixel image contaminated by 10% noise and Gaussian blur.

34 Original blur- and noise-free image

35 Blurred and noisy image

36 Restoration using multigrid with piecewise linear prolongation

37 Restoration using multigrid with D 1 prolongation (TV-norm)

38 Restoration using multigrid with D 2 prolongation (Perona-Malik)

39 Example: Contaminate by Gaussian blur and 0.5% noise. Restored with 1 or 3 level multigrid method. Then apply edge detector from gimp.

40 1-level CGNR

41 3-level multigrid with piecewise linear prolongation

42 1-level CGNR with D 2 applied after convergence

43 3-level multigrid with D 2 applied after convergence

44 Example: Original image

45 Image contaminated by Gaussian blur and 0.5% noise

46 Restored image using prolongation D2

47 More iterations...

48 Restored image using nonlinear PDE model based on D2