Computational methods for large distributed parameter estimation problems in 3D

Transcription

1 Computational methods for large distributed parameter estimation problems in 3D U. M. Ascher E. Haber March 8, 03 Abstract This paper considers problems of distributed parameter estimation from data measurements on solutions of diffusive partial differential equations (PDEs). A nonlinear functional is minimized to approximately recover the sought parameter function (i.e., the model). This functional consists of a data fitting term, involving the solution of a finite volume or finite element discretization of the forward differential equation, and a Tikhonov-type regularization term, involving the discretization of a mix of model derivatives. We develop methods for the resulting constrained optimization problem. The method directly addresses the discretized PDE system which defines a critical point of the Lagrangian. This system is strongly coupled when the regularization parameter is small. We then apply such methods for electromagnetic data inversion in 3D, both in frequency and in time domains. This involves developing appropriate discretizations for the forward problems as well as handling very large systems of algebraic equations. Finally, we explore the use of the so-called Huber s norm for the recovery of piecewise smooth model functions. Since our differential operators are compact, there are many different models that give rise to fields that fit practical data sets well. 1 Introduction This paper mostly describes and surveys our efforts over the past several years to design efficient numerical techniques for electromagnetic data inversion in 3D. The full Department of Computer Science, University of British Columbia, Vancouver, BC, V6T 1Z4, Canada. (ascher@cs.ubc.ca). Supported in part under NSERC Research Grant Department of Mathematics and Computer Science, Emory University, Atlanta, Ga, 303, USA. (haber@mathcs.emory.edu). 1

2 details are developed and presented in a series of papers including [15, 13, 1, 14,, 16], and we fall back on these sources as we try to paint the general picture. In Section we set up the scene for the problems considered, both the forward problems and those of data inversion for the recovery of a distributed parameter model. A nonlinear functional is minimized to approximately recover the sought parameter function (i.e., the model). This functional consists of a data fitting term, involving the solution of a finite volume or finite element discretization of the forward differential equation, and a Tikhonov-type regularization term, involving the discretization of a mix of model derivatives. We develop methods for the resulting constrained optimization problem in Section 3. The methods directly address the discretized PDE system which defines a critical point of the Lagrangian. This system is strongly coupled when the regularization parameter is small. In Section 4 we then apply such methods for electromagnetic data inversion in 3D, both in frequency and in time domains. This involves developing appropriate discretizations for the forward problems as well as handling very large systems of algebraic equations. The last section of this paper, Section 5, is new. We consider the task of recovering a piecewise smooth model, i.e. one that admits discontinuities. We use Huber s norm for this and discuss the choice of its parameter. However, with the diffusive forward operators considered in this paper (and elsewhere) we must also cast some doubt on efforts which attempt to define the model too uniquely for realistic data. Indeed, we feel that often there are many scenarios that reasonably match the measured data in practice. Forward problems and distributed parameter estimation Several different applications give rise to a partial differential equation (PDE) of the form (σ u) = q. (1) Here, the field u(x) satisfies the PDE plus boundary conditions for given sources q = q(x) and a conductivity function σ = σ(x). Applications include DC resistivity [5], magnetotelluric inversion [1], diffraction tomography [8], impedance tomography [6], oil reservoir simulation [9] and aquifer calibration [11]. Other applications of interest give rise to the more involved Maxwell s equations, e.g. written for the time harmonic case as (µ 1 E) ıω σ E = ıωs. ()

3 3 σ=0 (air) Ω Γ (air-earth interface) σ>0 (ground) Ω 1 Figure 1: A typical domain cross-section for a geophysical application See, e.g., [18, 19, 15] and references therein. A typical domain Ω in 3D on which such a PDE is defined is depicted in Figure 1. In a typical application we may have several PDEs such as (1) or () corresponding, e.g., to different frequencies and different sources and sinks. Whatever of these we have, we group them as well as the given boundary conditions into a unified notation A(m)u = q where the model m(x) typically is a function of σ, e.g., σ(x) = e m(x), x Ω. For the numerical solution of such a boundary value PDE we discretize it on a tensor product grid, not necessarily uniform, using a finite volume technique [13, 1]. We assume the material properties to be constant in each cell and call the resulting grid functions u, q and m. If needed, they are ordered into vectors. Thus, on a grid depicted in Figure we obtain our discretized problem A(m)u = q. (3) We assume further that the large and sparse matrix A is nonsingular. This assumption, as well as others regarding the type of grid and the use of the same grid for u and for m, are not strictly necessary, but they simplify both the exposition and the programming. The forward problem is to find u satisfying (3) given m (and q, which is always assumed given). The inverse problem is to recover the model m, given measurements b on the field u (and/or its derivatives, but let s do just u here) such that (3) holds. However, it is well-known that while the forward problem is well-posed the inverse problem is not. Indeed, in practice for the available, noisy data typically there is no

4 4 h Ω h Ω 1 Figure : The discretized domain: a cross-section of a 3D grid of cells. unique solution, i.e., there are many models m which yield a field u which is close to b to within the noise level, and moreover, such models m may vary wildly. Thus, we must add a priori information and isolate noise effects. Then the problem becomes to choose, from all the possible model solutions, the one closest to our a priori information. This Tikhonov-type regularization [6] leads to the optimization problem min m,u φ = 1 Qu b + βr(m) (4) subject to A(m)u q = 0, where Q is a matrix consisting of unit rows which projects to data locations, and R(m) is a regularization term. The parameter β > 0 is the regularization parameter whose choice has been the subject of many a paper. Also, throughout this paper the notation refers to the least squares norm. For the regularization term, consider a same-grid discretization of R(m) = ρ( m ) + ˆα(m m ref ) (5) Ω where ˆα is a (typically very small, positive) parameter and m ref is a given reference function (typically, the half-space solution). We next present several options for the choice of the function ρ, but reserve most of the discussion to Section 5. The most popular least squares choice is ρ(τ) = 1 τ, R (m) m.

5 5 In geophysics often depth is characteristically different from the horizontal direction. The weighted least squares choice is for some diagonal matrix a. R (m) a m The total variation (TV) choice (e.g. [8, 10]) is ρ(τ) = τ, R (m) ( m m ) A mix of least squares and total variation is the so-called Huber s norm, { τ, τ γ, ρ(τ) = τ /(γ) + γ/, τ < γ ( R (m) min{ 1 ) γ, 1 m } m Whichever choice is made for ρ, there is a very large, nonlinear optimization problem to solve in (4). This problem is significantly harder to solve when total variation or Huber s norm are used, and it also gets harder the smaller β becomes. It is not unusual in our experience to solve problems of the form (4) for half a million unknowns, so care must be taken to do this efficiently: One cannot simply assume the existence of a general software package to do the job. Fortunately, the matrices appearing in the necessary conditions for (4) are all very sparse and correspond to discretizations of elliptic PDEs, so we proceed below to exploit the special structure thus afforded. 3 Solving the optimization problem The unconstrained approach The most obvious first step in order to solve (4), taken by many, is to eliminate the field u using the forward problem. Thus we obtain a much smaller, although still large (and dense) unconstrained minimization problem min m φ(m) = 1 QA(m) 1 q b + βr(m). (6) The unconstrained approach is to devise numerical methods for solving this unconstrained optimization problem.

6 6 Newton and Gauss-Newton methods are well-known. Iterations involve positive definite linear systems of form (J T J + βr ) δm = p where the vector p and the matrices J and R are evaluated based on a current iterate m and the next iterate is obtained as m m + α δm, for a suitable step size 0 < α 1. The matrix J is the sensitivity matrix, and it will be reintroduced in more detail below. The conventional wisdom is that it is good to have an unconstrained minimization problem. Indeed, the matrix H red = J T J + βr is positive definite. However, this matrix is full and designing good preconditioners for it is a difficult task. In the large, sparse context the superiority of the unconstrained approach may thus be challenged. Let us proceed to widen the scope. The constrained approach We continue to consider solving the constrained (4). Introducing the Lagrangian L(m, u, λ) = 1 Qu b + βr(m) + λ T (A(m)u q) where λ is the vector (or more generally, a grid function like u) of Lagrange multipliers, we need to find an extremum (saddle) point of this Lagrangian. This leads to the large system of nonlinear equations L λ = Au q = 0, (7a) L u = Q T (Qu b) + A T λ = 0, (7b) L m = βr (m) + G T λ = 0; where G = A(m)u m. (7c) Note that all matrices appearing in (7), including the newly introduced G, are sparse. For the numerical solution of (7) consider using a variant of Newton s method (e.g. Lagrange-Newton, SQP, Gauss-Newton; see for instance []). A Gauss-Newton instance could read A 0 G Q T Q A T 0 0 G T βr δu δλ = δm L λ L u L m. (8) In (8) we have ordered the equations and the unknowns in a deliberate, unsymmetric fashion. Rather than highlighting the indefinite nature of the KKT matrix in its symmetric form we have placed on the main (block) diagonal the blocks corresponding to the highest derivative terms. For instance, if β were not small then the

7 7 resulting matrix would have been block-diagonal dominant. (However, β is small and life is not that simple.) As for the unconstrained case, the system (8) describes the equations to be solve at each iteration in order to find the correction vector for a current iterate (u, λ, m). We can distinguish the following tasks at each iteration: 1. Calculate the Gradients and Hessian.. Solve the large, sparse Hessian system. 3. Update the iteration. Of these tasks the first is straightforward in the present context. The third has been the subject of many studies (involving methods such as damped Newton, trust region, directly updating λ, etc.) that need not be repeated here: see, e.g. [] or other texts. We thus concentrate on the remaining, second task. There are two approaches here: the reduced Hessian approach and the simultaneous, all-at-once approach. Reduced Hessian approach Eliminating δu from the first block row of (8) (this involves solving the forward problem) and then δλ from the second block row of (8) (this involves solving the adjoint of the forward problem) we obtain for δm a system of the form where H red δm (J T J + βr ) δm = p (9) J = QA 1 G (10) is the sensitivity matrix. This sensitivity matrix is large and full for applications of the type considered here, so it is never evaluated or stored. To solve (9) we use a Preconditioned Conjugate Gradient method (e.g., []), i.e. the conjugate gradient method for M 1 (J T J + βr )δm = M 1 p, where M is a preconditioner, e.g. M = R. This conjugate gradient method requires only the evauation of matrix-vector products involving H red. Unfortunately, evaluating H red v for a given vector v is expensive!! It involves evaluating Jv and J T v. While multiplying a vector by G or Q or their adjoints is fast, there is A 1 in (10) as well. The forward and adjoint problems must be solved for this purpose to a relatively high accuracy. The system (9) is similar to that arising in the unconstrained approach. We now proceed to a different one, developed in [14,, 3, 4, 5].

8 8 Solving for the update direction simultaneously (all-at-once) The following rationale is well-known: When the iterate for a nonlinear problem is far from the optimal solution, it is wasteful to solve the linearized problem to a high accuracy. Thus, one wants to balance accuracies of inner and outer iterations. This leads to inexact-newton type methods; see, e.g., []. But now we apply the same rationale within the linear iteration: When the iterate for the update direction is far from the solution to the linear system, it is wasteful to eliminate some variables accurately in terms of others! Thus, one wants to balance accuracies inside the linear solver. This is where the unconstrained approach and the reduced Hessian approach described above fall over. We are thus led to consider methods where δu, δλ and δm are eliminated simultaneously. This approach is more natural also when considering the origin of the systems (7) and (8). These are, after all, discretizations of systems of PDEs, and the approach of eliminating some PDEs in terms of others instead of solving the given system as one is less usual. We therefore consider solving the linear problem (8) at once. Unfortunately, the matrix is no longer symmetric or positive definite. Moreover, β is typically small, so the problem corresponds to a strongly coupled PDE system. In [14] we proposed a preconditioned QMR method for the symmetrized system (8). The preconditioner consists of applying iterations towards the solution of the reduced problem (9). The catch is that this no longer has to be performed to a high accuracy. Biros & Ghattas had proposed an incredibly similar approach for a different application in [3, 4]. In the next section we demonstrate the performance of this method. In [] we proposed a multigrid method for the strongly coupled system (8). The resulting method is very nice and fast, but we have applied it only for the problem (1). The staggered discretization applied for the Maxwell equations in Section 4 complicates the multigrid technique significantly, so we chose the preconditioned QMR approach for the latter application. 4 3D electromagnetic data inversion in frequency and time domain The inversion results reported in this section were obtained using the weighted least squares norm on m for the regularization functional R, where m(x) = ln σ(x). This transformation automatically takes care of the positivity constraint on σ, and it also reduces the contrast in the conductivity, which is reasonable for the mining exploration application and commensurate with the use of a least squares norm in R.

9 9 Maxwell s equations in frequency domain The PDE system is written as E + αµh = s H in Ω, H ˆσE = s E in Ω, n H = 0 on Ω, where ˆσ = σ +αɛ and α = ıω. The permeability µ(x) is assumed known also during data inversion and has potential (moderate) jumps only where σ(x) has larger ones. Typical parameter regimes in these applications satisfy 0 ɛ 1 and exclude high frequencies ω. The operator introduces a nontrivial nullspace. In [15] we have therefore reformulated these equations using the Helmholtz decomposition and a Coulomb gauge law, E = A + φ A = 0. Thus, A is the electric field induced by magnetic fluxes, and φ is due to charge accumulation. Next, we differentiate as in the pressure-poisson equation in CFD (e.g. [4]) and stabilize, yielding the system (µ 1 A) (µ 1 A) + α σ(a + φ) = αs σ(a + φ) = s. For σ = σ > 0 this is an elliptic, diagonally dominant system. The corresponding boundary conditions are ( A) n = 0, Ω A n = φ = 0, Ω n Ω φdv = 0. Ω See [13] for more. A finite volume discretization is then applied on a staggered grid [15, 13]. This is summarized in Figure 3. The resulting system is written as ( Lµ + αm σ αm σ h h M σ h M σ h ) ( ) A φ = ( αs ) h s

10 10 Figure 3: Elements of a staggered, finite volume discretization. Here A, φ and s are located at face centers (like E); φ and A are at cell centers (like m); ˆσ is obtained by harmonic averaging at face centers; and µ is obtained by arithmetic averaging at edge centers (like H). In a multiple source/frequency experiment, we write the above as A k (m)u k = q k. Then the forward problem is A 1 (m) A(m)u = Example 1 A (m) u 1 u q. =. = q. (11).. A s (m) u s q s In [16] an experiment of an inversion of frequency domain data is described. It involves a transmitter and receiver geometry from an actual CSAMT field survey (from Penasquito, Mexico), although the conductivity model is synthesized. The transmitter is a 1km grounded wire a few kilometers away from survey area this is dealt with using a special procedure. The data is measured on 5 field components at frequencies 16, 64 and 51 Hz at 8 stations spaced 50m apart on each of 11 lines with linespacing of 100m. This gives a total of 308 data locations and 46 data values. The true model has two conductive bodies and one resistive body. This is used to generate true data, which are then contaminated by Gaussian q 1

11 11 β = 100 Final misfit = 0.06 Nonlinear iter KKT iter Au q / q Rel-grad 1 4 3e e-1 4 e 4 3e- 3 3 e 6 5e-4 β = 1e0 Final misfit = 0.03 Nonlinear iter KKT iter Au q / q Rel-rad 1 8 1e 6 3e-3 6 8e 7 9e-4 Table 1: Inverting electromagnetic data in the frequency domain. noise, % in amplitude and degrees in phase. The 3350m 3000m 00m volume is discretized into = 96, 000 cells. A simple continuation process in β is applied, where we start with a large β and decrease it until the data is deemed to be fitted sufficiently well. Results are accummulated in Table 1. The misfit reported here and later on is Qu b / b, i.e., the relative l norm of the predicted minus the observed data. The overall number of KKT iterations is rather reasonable here. Of the additional results displayed and discussed in [16] we only show Figure 4, to which we return in Section 5. Maxwell s equations in time domain We continue to summarize [16]. The PDE system now reads E + µ H = 0 in Ω, t H σe ɛ E t = s r(t) in Ω, n H = 0 on Ω. For the choice of time discretization we note that (i) the relevant parameter regime over non-short time scales yields heavy dissipation, i.e., very stiff problems in time; (ii) the field is measured only beyond the initial, transient layer, so the details of this layer can be skipped; (iii) we cannot expect high accuracy, as sources are typically only continuous in time; and (iv) Lagrange multipliers (solution of the adjoint problem for a rough right hand side) are of low continuity. All these together point at the unique choice of the backward Euler method for discretizing in time, α n = (t n t n 1 ) 1 ˆσ n = σ + α n ɛ

12 1 y (m) y (m) y (m) Re(Ex True) x (m) x x (m) x x (m) x 10 4 x x x Im(Ex True) Im(Ex Obs) Im(Ex Pred) x x x Re(Hy True) Re(Hy Obs) Re(Hy Pred) x x x Im(Hy True) Im(Hy Obs) Im(Hy Pred) x x x Figure 4: The accurate E x, H y data for 51 Hz are shown in the top row. The error contaminated data are shown in the middle row and the bottom row displays the data predicted from the inverted model. E n + α n µh n = α n H n 1 s H in Ω H n ˆσ n E n = s n r α n ɛe n 1 s E in Ω n H n = 0 on Ω. Now we realize that the latter system has the same form as in the frequency domain. Thus, we apply the same treatment. We obtain L µ + α n M σ α n M σ h 0 A n α n s h M σ h M σ h 0 φ n = h s. αn 1 Mµ 1 h 0 I H n H n 1 For the nth time step, we write the above as B n u n 1 + A n (m)u n = q n. Then forward problem for s time steps is A 1 (m) u 1 q 1 B A (m) u q. A(m)u = =. = q. (1) B s A s (m) u s q s Example For an inversion of time domain data we considered in [16] a square loop 50m 50m just above earth s surface. Data is acquired at depths in each of 4 boreholes

13 13 β = 1e 1 Final misfit = 0.1 Nonlinear iter KKT iter Au q / q Rel-grad 1 3e 3 1e- 3 e 4 4e-3 3 7e 6 1e-3 4 9e 7 3e-4 β = 1e Final misfit = 0.04 Nonlinear iter KKT iter Au q / q Rel-grad 1 7 4e 6 e-3 5 6e 7 7e-4 β = 1e 3 Final misfit = 0.0 Nonlinear iter KKT iter Au q / q Rel-grad 1 8 e 6 3e-3 7 8e 7 9e-4 Table : Inverting electromagnetic data in the time domain. surrounding a conductive body at 18 logarithmically spaced times between sec. The true model is a conductive sphere of radius 15m buried in a uniform halfspace. The time discretization involves 3 steps equally spaced on a log-grid sec. The inversion grid in space was 3. The initial guess is a uniform halfspace equal to the true background conductivity. Results are accummulated in Table similarly to those for the previous example. The performance of our algorithm is again very satisfactory, as the total number of KKT iterations is only about three dozens. 5 Discontinuous solutions and Huber s norm Recall that our argument leading to the minimization problem (4) has been that we introduce a priori information about smoothness of the model this way. But in many cases, including the examples of the previous section, our a priori knowledge is that the model probably contains jump discontinuities! So, in the regularization term [ ] R(m) = ρ( m ) + ˆα(m m ref ) Ω (where the subscript h implies that the integral has been discretized) we want to limit the effect of penalty through a jump discontinuity in m, as this should not be penalized for non-smoothness. h

14 14 Huber s norm Let us concentrate for a moment on the choice of norm in R(m) that will allow discontinuities. For this we may assume that u = m, which leads one to the vast literature on image demoising, e.g. [3, 8]. Here we stick to the multiresolution view of exploring functions on a given grid as corresponding to a discretization of some limit process. Note that: For m, m is integrable but m is not. This suggests inadequacy of the least squares norm in the presence of discontinuities. For m 0, m yields problems when differentiating it to obtain necessary conditions, but m does not. These observations suggest to combine the two, which yields the so-called Huber s norm [17, 10, 3] { τ, τ γ, ρ(τ) = (13) τ /(γ) + γ/, τ < γ ( R (m) min{ 1 ) γ, 1 m } m. (Strictly speaking, Huber s norm is not a norm, because a multiplication of the argument by a constant may cause a switch, but this does not bother us here.) Another way to handle the problem with the TV semi-norm is to modify the discretization m h = (D +,x m) + (D +,y m) into m h = (D +,x m) + (D +,y m) + ɛ, where the parameter 0 < ɛ 1 ensures that m 1 h does not become too large. See, e.g., [8, 7]. If ɛ is too small then the resulting image tends to be blocky; if it s too large then discontinuities are again smeared, as in the least squares case. We have preferred to consider (13), even though it s rougher and harder to analyze, because the parameter γ appears more naturally. Indeed, γ in (13) is interpreted as answering the question, what is a jump? i.e., it is the maximal size of a local change in m (no ɛ needed here) which is still interpreted as a smooth change in m on the scale of the current grid. This depends on the application. A lower value of γ favours the interpretation of a significant change in m as a jump, whereas a higher value favours the smooth interpretation. We have found the following automatic choice to be particularly useful in practice. We set γ = h Ω h [ m Ω ]. (14) h

15 15 Thus, γ depends on the solution, and indeed we adjust its value through the iteration in an obvious fashion. Others choose this parameter using an expression from robust statistics involving medians of m : see [3] and references therein. Also, in the image processing literature one often chooses to penalize even less through discontinuities than when using Huber s norm (e.g. using a Gaussian function or a Tukey biweight). However, this leads to non-convex functionals and local minima, which seems excessive in our more complex context where the forward problem is not simply the identity. Consider solving the equations for the unconstrained approach (6) where R uses Huber s norm or the modified TV. If we set QA(m) 1 q = m then an iterative method called lagged diffusivity suggests itself, whereby in the current iterate we set ( R δm min{ 1 ) γ, 1 m } δm, (15) with a similar expression for TV, and solve the resulting weighted least squares problem. This iteratively reweighted least squares (IRLS) algorithm has been analyzed in the image denoising context to show global convergence; see [8] and references therein. If γ is picked adaptively then we are short of a proof, although in practice no difficulty ever seems to arise. Difficulties do arise, of course, when the forward model is (1) or (), when even the least squares regularization functional can be testy. However, the lagged diffusivity approach can still be generalized. Indeed, for the nonlinear system (7) we use (15) for defining R in the linearization (8). The all-at-once methodology follows unchanged. Recovering discontinuous surfaces from diffusive operators? We have obtained good results using the techniques outlined in this section for examples in denoising and in SPECT tomography [1]. However, in the context of the present paper one wonders, is there really enough accurate data in applications to allow an honest identification of discontinuities using our diffusive forward operators?! Consider, for instance, Figure 4. Note how well the recovered field approximates the unpolluted data! Indeed, the solution of a PDE such as (1) or a low frequency instance of Maxwell s equations cannot tolerate high level noise the forward problem is a low-pass filter. The recovered model fits the true model much less closely, and the differences are easily visible to the eye [16]. It would appear that many, rather different models for m could fit the data within the noise tolerance. Does it then make sense at all to seek a model with pinpointed discontinuities? A full investigation of these questions is well beyond the scope of this paper. Here, to focus the discussion further, let us consider an example.

16 Figure 5: Contour plot of the true model for Example 3. Example 3 This example is in D, and Ω = [ 1, 1]. To allow maximum possibility for recovery of discontinuities we consider the differential operator (1) in the form (m 1 u) = q, i.e. without the exponential transformation from σ to m, with natural boundary conditions, and assume (unrealistically) that data on u are available everywhere on a given grid. The right hand side is chosen with source and sink, q = exp( 10((x + 0.6) + (y + 0.6) )) exp( 10((x 0.6) + (y 0.6) )), and the true model, depicted in Figure 5, contains discontinuities. We use this true model to generate a field on a cell-centered grid and contaminate this with 1% noise to yield the observed data, b, depicted in Figure 6. A standard finite volume discretization is applied to the forward problem (with harmonic averaging for m 1 ). For the optimization problem we use a conjugate gradient solver with a multigrid preconditioner (see, e.g., [7]). Our multigrid preconditioner employs operator-induced, node-based prolongation. The advantages of using this preconditioner (over one based on simpler multigrid) will be discussed elsewhere. During the iteration we do not allow m to decrease below a pre-fixed, positive value. Convergence does not always come easy here, even though we do not use tight tolerances; but this is not our focus in this example.

17 Figure 6: Data for Example 3. Figure 7 displays the recovered model using least squares with β = The resulting misfit is , which is not awfully far from the target of 1%. When reducing β to β = , Figure 8 is obtained and the misfit reduces to Reducing β to 10 6 produces a model where noise-related artifacts are apparent, so β should not be reduced further. In comparison, our result using Huber s norm with β = 10 5, which yields a final γ = 4.6 and a misfit , is displayed in Figure 9. The reconstructed fields in all of these inversions are far smoother than the observed data and closer to the noiseless data (i.e. the exact discrete field) which is displayed in Figure 10. Recall our similar observation for Example 1. The good news when comparing these figures is that the more careful Huber norm does yield better results, both in terms of misfit and in terms of closeness to the true model. On the other hand, the difference between a misfit of 1% and 1.5% is much too fine to define a cutting edge between a good and a bad model in realistic situations, where we do not know a true model either. One never has such a precise knowledge of the noise level (nor of its statistical distribution) in practice, and data are scarcely given everywhere - see Examples 1 and. Of course, it can be said that, using the a priori information that the model contains discontinuities, we generate one such model which fits the data well enough and that is that. However, there are other models with discontinuities which fit the data well! For the present example we took the recovered model of Figure 8 and

18 Figure 7: Recovered model using least squares with β = Figure 8: Recovered model using least squares with β =

19 Figure 9: Recovered model using Huber s norm with β = Figure 10: The noiseless data, viz., the field corresponding to the true model using the applied discretization.

20 Figure 11: The model of Figure 8 replaced by a piecewise constant approximation with 5 constant values. applied a simple thresholding procedure, whereby the range [m min, m max ] was divided into 5 equal subintervals, and then all values of m within each such subinterval were replaced by the midpoint value. The result is displayed in Figure 11. Clearly, the resulting model is rather far from the true model of Figure 5, and yet the misfit is a respectable. 10. Our point with the above thresholding experiment is that in more realistic examples we may not know whether our recovered discontinuities are, even approximately, in the right place. Simply trusting that the misfit is small enough is insufficient for this purpose. (Considering the maximum norm of the predicted minus the observed fields proves insufficient as well.) It can then be argued that displaying a smooth blob, such as when using least squares regularization, is less committing than displaying a discontinuous solution (especially with only a few constant values), and as such is more commensurate with the actual information at hand. Nonetheless, the results using Huber s norm for this example are encouraging, especially if we disregard questions of cost in obtaining them. References [1] U. Ascher and E. Haber. Grid refinement and scaling for distributed parameter estimation problems. Inverse Problems, 17: , 01.

21 1 [] U. Ascher and E. Haber. A multigrid method for distributed parameter estimation problems. ETNA, 01. [3] G. Biros and O. Ghattas. Parallel Newton-Krylov methods for PDE-constrained optimization. In Proceedings of CS99, Portland Oregon, [4] G. Biros and O. Ghattas. Parallel preconditioners for KKT systems arising in optimal control of viscous incompressible flows. In Proceedings of Parallel CFD. North Holland, May 3-6, Williamsburg, VA. [5] G. Biros and O. Ghattas. Parallel Lagrange-Newton-Krylov-Schur methods for PDE-constrained optimization Parts I,II. Preprints, 00. [6] L. Borcea, J. G. Berryman, and G. C. Papanicolaou. High-contrast impedance tomography. Inverse Problems, 1, [7] T. F. Chan, G. H. Golub, and P. Mulet. A nonlinear primal-dual method for total variation-based image restoration. SIAM Journal on Scientific Computing, : , [8] A. J. Devaney. The limited-view problem in diffraction tomography. Inverse Problems, 5:510 53, [9] R. Ewing (Ed.). The mathematics of reservoir simulation. SIAM, Philadelphia, [10] C. Farquharson and D. Oldenburg. Non-linear inversion using general measures of data misfit and model structure. Geophysics J., 134:13 7, [11] S. Gomez, A. Perez, and R. Alvarez. Multiscale optimization for aquifer parameter identification with noisy data. In Computational Methods in Water Resources XII, Vol., [1] E. Haber. Numerical Strategies for the Solution of Inverse Problems. PhD thesis, University of British Columbia, [13] E. Haber and U. Ascher. Fast finite volume simulation of 3D electromagnetic problems with highly discontinuous coefficients. SIAM J. Scient. Comput., : , 01. [14] E. Haber and U. Ascher. Preconditioned all-at-one methods for large, sparse parameter estimation problems. Inverse Problems, 17: , 01. [15] E. Haber, U. Ascher, D. Aruliah, and D. Oldenburg. Fast simulation of 3D electromagnetic using potentials. J. Comput. Phys., 163: , 00.

22 [16] E. Haber, U. Ascher, and D. Oldenburg. Inversion of 3D electromagntic data in frequency and time domain using an inexact all-at-once approach. 0. Manuscript. [17] P. J. Huber. Robust estimation of a location parameter. Ann. Math. Stats., 35:73 101, [18] G. Newman and D. Alumbaugh. Three-dimensional massively parallel electromagnetic inversion i. theory. Geophysical journal international, 18: , n. [19] G. Newman and D. Alumbaugh. Three-dimensional massively parallel electromagnetic inversion ii, analysis of a crosswell electromagnetic experiment. Geophysical journal international, 18: , n. [] J. Nocedal and S. Wright. Numerical Optimization. New York: Springer, [1] R. L. Parker. Geophysical Inverse Theory. Princeton University Press, Princeton NJ, [] Y. Saad. Iterartive Methods for Sparse Linear Systems. PWS Publishing Company, [3] G. Sapiro. Geometric Partial Differential Equations and Image Analysis. Cambridge, 01. [4] D. Sidilkover and U. Ascher. A multigrid solver for the steady state navier-stokes equations using the pressure-poisson formulation. Comp. Appl. Math. (SBMAC), 14:1 35, [5] N.C. Smith and K. Vozoff. Two dimensional DC resistivity inversion for dipole dipole data. IEEE Trans. on geoscience and remote sensing, GE :1 8, Special issue on electromagnetic methods in applied geophysics. [6] A.N. Tikhonov and V.Ya. Arsenin. Methods for Solving Ill-posed Problems. John Wiley and Sons, Inc., [7] U. Trottenberg, C. Oosterlee, and A. Schuller. Multigrid. Academic Press, 01. [8] C. Vogel. Computational methods for inverse problem. SIAM, Philadelphia, 0.