Generalized Local Regularization for Ill-Posed Problems

Size: px

Start display at page:

Download "Generalized Local Regularization for Ill-Posed Problems"

Leslie Greene
5 years ago
Views:

1 Generalized Local Regularization for Ill-Posed Problems Patricia K. Lamm Department of Mathematics Michigan State University AIP29 July 22, 29 Based on joint work with Cara Brooks, Zhewei Dai, and Xiaoyue Luo

2 Basic problem Problem: Solve Au = f for ū dom(a) X, where the ideal data f R(A) X is only approximately given. We assume the operator A may be linear or nonlinear; X is a Hilbert or reflexive Banach space; Existence/uniqueness of solution ū for f R(A), which does not depend continuously on f ; The data is only approximately given by f δ X, with f f δ X < δ. (Precise conditions on A to be given later.)

3 Examples of (related) regularization methods Equation for th order Tikhonov regularization: Let α > and find uα δ dom(a) X minimizing Au f δ 2 X + α u 2 X. If A bounded linear, the regularized solution uα δ satisfies A Au + αu = A f δ. (1) Features (for generic linear case): Pros: Cons : Solution of (1) easily Stability of (A A) 1 vs A 1 handled for f δ X A nonlinear = modify or discard (1) All-purpose regularizer Ignores special structure of A Easy to implement Can be oversmoothing

4 Examples of (related) regularization methods Equation for Lavrent ev (or simplified) regularization: In contrast to the Tikhonov regularization equation for linear A, A Au + αu = A f δ, the method of Lavrent ev finds uα δ dom(a) satisfying Au + αu = f δ. Features: Pros: Underlying operator is still A, so preserves structure of equation; Method the same for both linear and nonlinear problems Easy to implement Cons: Solvability guaranteed only for certain A Numerical results not always satisfactory

5 Diagonal quantities Comparisons: Both Tikhonov (linear case) and Lavrent ev methods stabilize using the diagonal quantity αu. But local regularization also involves a diagonal quantity : Tikhonov and Lavrent ev regularization: * αu added to the underlying operator, which makes a small shift of singular values of the operator away from zero. Local regularization: * We first subtract a diagonal quantity from the operator... use this to construct a diagonal αu to be added back in. = αu more closely linked to A = can preserve structure * The subtraction effectively splits the underlying operator (As we ll see later, the splitting should be related to the sensitivity of the data to local changes in the solution.)

6 Local regularization The equation for local regularization: For each α (, ᾱ] and X α X, let u δ α dom(a α ) X α satisfy A α u + a α u = f δ α for given (nonlinear) operator A α : dom(a α ) X α X, parameter (function/operator) a α ( ), f δ α X α defined via f δ α = T α f δ, and for for a family {T α } of bounded linear operators, T α : X X α, T α f Xα f X, uniformly in f X. [E.g., T α = I, or T α = a (normalized) mollification operator, or a number of other choices.]

7 Compare Tikhonov, Lavrent ev, and local regularization Comments about the local regularization equation, A α u + a α u = T α f δ Tikhonov and Lavrent ev are special cases, with parameter-independent A α and X α = X for each: Method A α a α T α Tikhonov: A A α (scalar) A Lavrent ev: A α (scalar) I In local regularization, T α plays the role of consolidating a local piece of the function f δ : Tikhonov: T α = A (not local at all) Lavrent ev: T α = I (may be too local)

8 Operators/parameters for local regularization Selection of A α, a α, and T α for local regularization method: Motivation: Look at what s needed to show stability and convergence to ū of uα δ, the solution of the (generic) equation A α u + a α u = T α f δ. (2) To simplify the motivation: * Take operators A, A α bounded linear, α >. * Take X α = X, α >. Will make comparisons with: * stability/convergence results for Tikhonov/Lavrent ev (since these methods also take the form of equation (2) ).

9 Stability of the three methods (1) Stability for methods based on A α u + a α u = T α f δ : Want: solution u δ α unique + depends continuously on data f δ. = need (A α + a α I ) invertible with bounded inverse. This is easy for Tikhonov regularization: A α = A A automatically nonnegative self-adjoint. Not automatic for Lavrent ev or local regularization: Need added conditions on A α. E.g., assume A α nonnegative self-adjoint. Note difference: For Lavrent ev = assumption is on A α = A; For local reg. = assumption is on A α (we construct).

10 Convergence for the three methods So assume we already have stability, or that ( ) 1 (A α + a α I ) 1 = O c(α) where c(α) as α ; here denotes the operator norm. (2) Convergence for methods based on A α u + a α u = T α f δ : Must show: For each f R(A) there is a choice of α = α(δ, f δ ) for which α(δ, f δ ) as δ, and for which u δ α(δ,f δ ) ū as δ, for all f δ X satisfying f f δ X δ.

11 Convergence for the three methods, cont d. Verification of convergence of uα δ to ū, where A α uα δ + a α uα δ = T α f δ. (3) with A α, a α, T α given: First note ū X uniquely satisfies Aū = f, = T α Aū = T α f, = A α ū + a α ū = T α f + (A α ū + a α ū T α Aū). (4) So subtract: equation (3) (4) to obtain (A α + a α I )(uα δ ū) = T α d δ (A α ū + a α ū T α Aū), where d δ (f δ f ) X satisfies d δ X δ.

12 Convergence for the three methods, cont d. We then have (u δ α ū) = (A α + a α I ) 1 T α d δ + (A α + a α I ) 1 [(T α Aū A α ū) a α ū], where ) (Aα + a α I ) 1 T α d δ X δ = O(. c(α) Goal: select α = α(δ, f δ ) so that α(δ, f δ ) as δ, and so that both terms above in (uα δ ū) go to zero as δ. That is, The term due to data error goes to zero, i.e., δ c ( α(δ, f δ as δ. ) ) The term due to method error goes to zero: (A α + a α I ) 1 (D α ū a α ū) as δ. Here we have defined D α = T α A A α.

13 Convergence for the three methods, cont d. Note: There is generally little difficulty in making an a priori choice of α = α(δ, f δ ) so that α(δ, f δ ) and δ c ( α(δ, f δ as δ. ) ) The problem then is to show that for a selection of α-values with α, the operator (A α + a α I ) is continuously invertible, and (A α + a α I ) 1 (D α ū a α ū), as α. It is well-known that these last statements are true (for linear A) in the case of: Tikhonov regularization, for A bounded; Lavrent ev regularization, for a narrower class of operators A (e.g. A nonnegative self-adjoint).

14 Local regularization: choice of A α and a α For local regularization, we ll use the desired convergence condition, (A α + a α I ) 1 (D α ū a α ū) as α, (5) to motivate the selection of A α and a α. (Recall D α T α A A α.) That is, ū satisfies Aū = f = T α Aū = T α f, where the operator T α A may be split as follows: T α A = A α + (T α A A α ) = T α A = A α + D α. Using (5), this splitting of T α A needs to be such that (roughly) D α ū a α ū for α, for some choice of a α, i.e., D α diagonal when applied to ū.

15 Local regularization: choice of A α and a α So in local regularization: We first select T α (selection criteria to be discussed later). We then select A α and a α (and define D α T α A A α ) so that for α > we have: 1 (A α + a α I ) continuously invertible, and 2 the operator (D α a α I ) satisfies (A α + a α I ) 1 (D α a α I ) ū as α. X Roughly, we take the modified equation T α Au = T α f, and split its leading operator via T α A = A α + D α, where A α = the bigger, nearly continuously invertible part of T α A, D α = the smaller, diagonal-like part of T α A.

16 Statement of convergence The above statement of convergence, namely, (A α + a α I ) 1 (D α a α I ) ū as α. (6) X is obtained for Tikhonov & Lavrent ev, without splitting T α A: Tikhonov: T α A = A α = A A, D α =, a α = α. Lavrent ev: T α A = A α = A, D α =, a α = α. In these cases the convergence as α in (6) takes the form (respectively) of (A A + αi ) 1 ( αū), (A + αi ) 1 ( αū). X X In contrast: With local regularization we can obtain a stronger statement of convergence (than (6)) for many problems.

17 Local regularization: choice of A α and a α Stronger statement of convergence: A stronger result than (A α + a α I ) 1 (D α ū a α ū) as α, X is obtained if the method satisfies D α ū a α ū X = o (c(α)) as α (7) (recall that (A α + a α I ) 1 = O( 1 c(α) ) ). The stronger result (7) cannot hold for Tikhonov or Lavrent ev (in general). For these methods the usual case is (A α + αi ) 1 ( = O 1 ) α as α, i.e., c(α) = α, but D α ū a α ū X = αū X = O(α) = D α ū a α ū X o ( c(α) ) as α.

18 Local regularization: the selection of T α How to select T α in the method of local regularization? We will select T α in order to: Facilitate a regularization method which is localized in an appropriate sense. Work toward the goal of finding a method which is easier/faster than classical regularization methods. Effectively utilize information about the sensitivity of the data f to (local) changes in u (for f, u satisfying the Au = f ). We will demonstrate the selection of T α, as well as of A α and a α, through some examples.

19 Examples of local regularization Three motivating examples... from where local regularization theory is the most complete, i.e., for Volterra operators. 1 Linear problem on X = L p (, 1), 1 < p : t k(t, s)u(s) ds = f (t), a.e. t [, 1]. 2 Nonlinear Hammerstein problem on X = C[, 1]: t k(t, s)g(s, u(s)) ds = f (t), a.e. t [, 1], under (standard) conditions on g. 3 Nonlinear autoconvolution problem, X = L 2 (, 1), C[, 1]: t u(t s)u(s) ds = f (t), a.e. t [, 1].

20 Linear Volterra example First consider the linear Volterra problem Au(t) = t k(t, s)u(s) ds = f (t), a.e. t [, 1], in the special case of X = L 2 (, 1), and for k L 2 ([, 1] [, 1]). To determine an appropriate choice for T α, we first look at sensitivities of the data f = Au to local parts of u: A is bounded linear on X = the Frechet differential of A at u X with increment u X is given by i.e., DA(u; u)(t) = DA(u; u) = A u, t k(t, s) u(s) ds, a.e. t [, 1].

21 Sensitivities of data for linear Volterra problem Graphs of: (1) u = unit impulse of width.1, (1st & 3rd rows of graphs above), and (2) the response DA(u; u) (2nd & 4th rows), immediately below each u. (Here the kernel for A is k(t, s) = 1.)

22 Sensitivities of data for linear Volterra problem Combined graphs of DA(u; u) for each unit impulse u of width.1 (As before, the kernel for A is k(t, s) = 1.)

23 Sensitivities of data for linear Volterra problem Combined graphs of DA(u; u) for each unit impulse u of width.1 (Here the kernel for A is k(t, s) = t s.)

24 Sensitivities of data for linear Volterra problem Combined graphs of DA(u; u) for each unit impulse u of width.1 (Here the kernel for A is k (t, s) = (t s) 2.)

25 Sensitivities of data for linear Volterra problem Combined graphs of DA(u; u) for each unit impulse u of width.1 (Here the kernel for A is k(t, s) = (t s) 3.)

26 Selection of T α for linear Volterra problem = for the linear Volterra problem, the solution u at t most influences data f on the interval [t, t + α], for some α > small. The length α of the interval is the regularization parameter. (Alternatively, use two parameters one for sensitivity region and another for regularization. E.g., see [PKL & Dai, 25] ) One possible choice of T α : T α f (t) = 1 α α f (t + ρ) dρ, (averaging function on [t, t + α]). More generally, T α f (t) = α a.e. t [, 1 α], f (t + ρ) dη α(ρ) α dη, a.e. t [, 1 α], α(ρ) for suitable measure η α on [, α].

27 Selection of A α, a α for linear Volterra problem. For example, we will apply the simple averaging T α to the original linear Volterra equation. Assume α (, ᾱ], for ᾱ > small: Start with the original equation Au = f, or t k(t, s)u(s) ds = f (t), a.e. t [, 1]. Next compute T α Au = T α f, for a.e. t [, 1 α], 1 α α t+ρ k(t + ρ, s)u(s) ds dρ = 1 α α f (t + ρ) dρ. Now split: T α A = A α + D α, where D α is nearly diagonal : T α Au(t) = 1 α α t + 1 α α t+ρ k(t + ρ, s)u(s) ds dρ t k(t + ρ, s)u(s) ds dρ.

28 Selection of A α, a α for linear Volterra problem. So the splitting of T α A is thus given by where, for a.e. t [, 1 α], A α u(t) = T α A = A α + D α, t D α u(t) = 1 α and k α (t, s) 1 α k α (t, s)u(s) ds, α t+ρ α k(t + ρ, s) dρ. = A α is of the same form as A. t k(t + ρ, s)u(s) ds dρ, Finally: select a α = a α (t) so that D α ū a α ū for α small.

29 Selection of a α for linear Volterra problem. Need a α = a α (t) so that, for a.e. t [, 1 α], D α u(t) = 1 α when u = ū. α t+ρ t k(t + ρ, s)u(s) ds dρ a α (t)u(t), If we approximate u on the local interval [t, t + α] by the constant u(t), then we have (formally) for D α u(t) ( 1 α α t+ρ t ) k(t + ρ, s)ds dρ u(t) = a α (t)u(t), a α (t) = 1 α α t+ρ t k(t + ρ, s)ds dρ, a.e. t [, 1 α].

30 Local regularization for the linear Volterra problem To summarize: The particular construction we have chosen leads to a local regularization equation for the linear Volterra problem given, for noisy data f δ, by A α u + a α u = f δ α on X α = L 2 (, 1 α), where for a.e. t [, 1 α], f δ α (t) = T α f δ (t) = 1 α A α u(t) = = α t t a α (t) = 1 α f δ (t + ρ) dρ, k α (t, s)u(s) ds ( 1 α ) k(t + ρ, s) dρ u(s) ds, α α t+ρ t k(t + ρ, s)ds dρ.

31 Local regularization for the linear Volterra problem Result: For a large class of kernels k and for α sufficiently small, 1 The quantity a α, satisfies a α (t) = 1 α α t+ρ t k(t + ρ, s)ds dρ, < c 1 c(α) a α (t) c 2 c(α), a.e. t [, 1 α], for some c(α) > with the property that c(α) as α ; and, 2 (A α + a α I ) is invertible with (A α + a α I ) 1 ( ) 1 = O c(α) as α.

32 Stronger convergence result Question: So why does the local regularization method satisfy the stronger convergence result? Recall that in this case we need to verify (D α ū a α ū) Xα = o (c(α)) as α. = 1 α In contrast to Tikhonov and Lavrent ev, where X α = X and (D α ū αū) Xα = αū Xα o(α), the method of local regularization gives α t+ρ D α ū(t) a α (t)ū(t) = ( 1 k(t+ρ, s)ū(s) ds dρ α t α t+ρ t ) k(t + ρ, s) ds dρ ū(t).

33 Stronger convergence result, cont d So for local regularization we have D α ū(t) a α (t)ū(t) = 1 α α t+ρ t k(t + ρ, s) (ū(s) ū(t)) ds dρ, a.e. t [, 1 α), and thus as α, ( ) D α ū a α ū Xα = O a α ( ) L S α ū ū Xα where = O ( c(α) ) S α ū ū Xα S α ū(t) = 1 α α ū(t + s) ds. Note that S α ū(t) is a special case of the Hardy-Littlewood maximal function for ū at t.

34 Stronger convergence result, cont d So we have D α ū a α ū Xα = O ( c(α) ) S α ū ū Xα, and it follows from the Lebesgue differentiation theorem that for any ū X. Thus S α ū ū Xα as α, D α ū a α ū Xα = o ( c(α) ) as α, and the stronger convergence result is obtained. For smoother ū, it s obvious that from this same estimate we can obtain a rate of convergence.

35 General results for linear Volterra problems More general results for the linear Volterra problem ( [PKL] for earlier work]; [Brooks thesis, 27] and [Brooks & PKL, 29] for L p results and discrepancy principle) f R(A) X = L p [, 1], f δ L p [, 1], 1 < p. For ν-smoothing problems, with a large class of measures (depending on ν) used to define suitable operators T α. Convergence as δ of u δ α to ū in X α = L p [, 1 α] for a priori choices of α = α(δ, f δ ). Optimal rates of convergence for the ν-smoothing problem. A discrepancy principle of the form where ( d(α) α m d(α) = τδ, f δ ) X T α f δ A α uα δ T α f δ Xα. Xα

36 Nonlinear Volterra problems Nonlinear Hammerstein problem: Start with the usual local regularization equation A α u + a α u = T α f δ, except now A α and the term a α u are both nonlinear in u. How to handle the nonlinearity? Use the local regularization interval to facilitate linearization of the nonlinear problem [Luo thesis 7; Brooks, PKL, Luo 9] i.e., for t >, let τ = min{t, α}, and linearize a α = a α (t, u(t)) about the prior u-value u(t τ). Numerical implementation of the linearized problem: u = (T α f δ A α u)/a α ( ), t > = solve a linear problem for u(t) sequentially, since A α u and (the linearized) a α only involve values of u prior to t.

37 Nonlinear Volterra problems, cont d Nonlinear autoconvolution problem: For the autoconvolution problem (as with Hammerstein), the usual local regularization equation A α u + a α u = T α f δ (8) involves nonlinearities in both A α and the term a α u. First: Solve (8) for u(t), t [, α] (α > small): * Nonlinear equation (8) not too difficult to solve on [, α]; * Need only a reasonable approximation to u on [, α] for convergence Then: Solve (8) for u(t), t (α, 1] using linear methods: * The (approximate) solution u on (α, 1] can be found sequentially from (8) using linear techniques. Thus, linearization of (8) not required in practice for the autoconvolution problem (but used as tool in convergence theory). [Dai thesis, 26; Dai & PKL, 28; Brooks, Dai, & PKL, 29]

38 Nonlinear Volterra problems, cont d Numerical results for nonlinear Volterra problems: In general, Qualitatively equivalent to Tikhonov solution, except not oversmoothed; considerably better than Lavrent ev solution. Many times faster than a nonlinear Tikhonov (iterative) approach in practice. (See above references.) Other non-volterra work: 2-D image deblurring Older work: [Cui thesis], [Cui, Lamm, Scofield, 7]. Newer results (to appear) are based on generalized local regularization approach = less computationally intensive. General n-dimensional problems governed by a nonlinear monotone operator (to appear; from generalized theory).

39 Sample numerical results

40 Sample numerical results, cont d

A generalized approach to local regularization of linear Volterra problems in L p spaces

A generalized approach to local regularization of linear Volterra problems in L p spaces Cara D. Brooks and Patricia K. Lamm Department of Mathematics, Michigan State University, East Lansing, MI 48824