A parallel method for large scale time domain electromagnetic inverse problems

Size: px
Start display at page:

Download "A parallel method for large scale time domain electromagnetic inverse problems"

Transcription

1 A parallel method for large scale time domain electromagnetic inverse problems Eldad Haber July 15, 2005 Abstract In this work we consider the solution of 3D time domain electromagnetic inverse problems Solving such problems is an open challenge as they require very high computational resources We therefore explore a method to parallelize the inverse problem by using time decomposition We show that our approach can reduce the computational time although it does not scale optimally 1 Introduction In this work we consider an inverse problem where the forward problem is the Quasi Static approximation to Maxwell s equations in time which we write in their H form σ 1 H + µh t = 0; H Ω (1) H(x, 0) = H 0 n H Ω = 0 Here H(x, t) is the magnetic field, σ is the conductivity and µ is the magnetic susceptibility Such an approximation is valid for eddy current computation in conductive media for long integration times and it is commonly used in geophysical prospecting as well as in medical imaging and computation of eddy currents in electrical devices [18, 4, 23] In the forward problem we assume that the conductivity σ is known and that we require to calculate the magnetic field H In the inverse problem we assume that H is known at some discrete locations and times and the goal is to evaluate the conductivity σ(x) Assuming that an appropriate finite volume or finite element mimicking discretization scheme has been applied in space [10, 12, 14, 18] we can rewrite the resulting system as a large scale ODE in time u t = A(m)u (2) u(0) = u 0 Dept of Mathematics and Computer Science, Emory University, Atlanta GA haber@mathcsemoryedu 1

2 where A(m) is a symmetric matrix which results from the discretization of the operator σ 1, u is a discretization of the magnetic field and m = log(σ) is a discretization of a the log conductivity Here, commonly to many other inverse conductivity formulations, we use the log conductivity because the conductivity may change over a few orders of magnitudes [22] To solve the discrete forward problem given a discrete log conductivity m we need to integrate the ODE to obtain u In the inverse problem we are given some observations on the solution u and we require to recover m There are many practical challenges to solving the inverse problem First, a fast, accurate and reliable algorithm for 3D forward modeling in time is required Second, the sensitivities for such problems are too numerous to be formed or stored in a reasonable amount of time and space Furthermore, for the type of problems we consider here, storing the solution of the forward problem stretches our computational ability Finally, finding the minimum of the objective function obtained by matching the data and incorporating a priori information on the distributed parameter can be difficult due to the nonlinearity and sensitivity of the problem Further difficulties arise when trying to generate algorithms which are easily parallelized Since Maxwell s equations are an initial value problem (IVP), they do not easily lend themselves to parallelization However, it is important to note that even though the forward problem is an IVP the solution of the inverse problem results in a system of equations which is equivalent to a boundary value problem (BVP) in space-time Therefore, in the spirit of domain decomposition methods which are routinely applied to BVP s it is possible to develop efficient numerical techniques which utilize the structure of the problem In this paper we present two approaches for the solution of the problem The first is a classical solution approach based on the Tichonov regularization [5] combined with a Gauss-Newton reduced space optimization technique Although the approach is routinely used in other inverse problems it has not been applied to the inversion of time domain electromagnetic problems The disadvantage of the classical reduced space approach is that although we use sparse, large-scale techniques it is serial in nature The second approach is based on a time decomposition method where we decouple the forward problem The method has many similarities to instantaneous control methods and in particular we are motivated by the work of Heinkenschloss on the time dependent optimal control problem in fluid dynamics (see [16] and reference within) There are major differences between our problem and instantaneous control First, unlike the control problem the log conductivity, m does not change in time Second, while the algorithms above are serial in nature, we aim to develop a highly parallelization algorithm and finally, our problem is nonlinear Our method can be understood in the context of iterative methods for nonlinear equations We show that it is equivalent to a nonlinear splitting of the Euler Lagrange equations We use a discretize-optimize approach where in the first stage we discretize the forward problem and then solve the finite dimension (but very large) discrete optimization problem An optimize-discretize approach can be used as well, however, if we use mimicking discretization then the resulting discrete equations are identical Although our fundamental structure here may be considered discrete it is useful to view it as an instance of a family of finer and 2

3 finer discretizations; see, eg, [1, 8, 2] For brevity we use the same notation for discretized and continuous variables The rest of the paper is as follows In Section 2 we discuss the discretization of the problem in time as well as the regularization we use in order to obtain a solution In Section 3 we suggest a straight-forward numerical method for the solution of the optimization problem based on the reduced Hessian SQP method In Section 4 we suggest a new timedecomposition method that allow us to easily parallelize the solution of the problem In Section 5 we analyze the proposed method Finally in Section 6 we conduct numerical examples and demonstrate the effectiveness of our techniques as well as summarize the paper 2 Discretization and problem formulation In this section we lie the mathematical background needed for the development of our algorithms We start by discretizing the forward problem in time Since Maxwell s equations in conductive media are very stiff, we use the Backward Euler method (although other methods for stiff equations can be used) and obtain the system B 1 (m) I B 2 (m) I u 1 = 0 B s (m) 0 where B j = I k j A(m) and k i = (t i t i 1 ) 1 is the time step In the inverse problem we have measurements of a subset of the vector u given by the projection matrix m at the discrete times [t 1,, t s ] u s d i = d(t i ) = Qu i + ɛ i i = 1,, s The problem of recovering m given d is known to be ill-posed and therefore regularization is needed In this work we consider the Tichonov regularizations which can be obtained by solving a sequence of optimization problems of the form 1 2 s Qu i d i 2 + αr(m m ref ) = min (4) i=1 subject to the (discrete) forward problem (3) In the classical Tichonov regularization one solves the problem a few times, changing the regularization parameter α until the data misfit is small enough In this work we assume that the regularization operator R( ) is some function of the gradients of m We have implemented two common regularization functionals First, for a smooth log conductivity function, we set R(m m ref ) = 1 2 h(m m ref ) 2 3 u 0 (3)

4 where h is a standard short differences discretization of the gradient Second, for a piecewise smooth log conductivity function, we consider regularization of the form R(m m ref ) = ρ( h (m m ref ) ) where ρ is the Huber function (see [17, 2]) { 1 ρ(z; θ) = 2θ z2 + θ z θ 2 z z > θ It is easy to verify that the optimality conditions for (4) with the equality constraint (3) leads to the following nonlinear system of equations B 1 (m) u 1 u 0 I B 2 (m) = (5a) I Bs(m) u s 0 B 1 (m) I λ 1 Q (d 1 Qu 1 ) B s 1 (m) I = (5b) B s (m) λ s Q (d s Qu s ) g r = ( G(m, u 1 ) G(m, u s ) ) + αr m(m m ref ) = 0 where λ are Lagrange multipliers and the sparse matrices G(m, u j ) are defined as G j := G(m, u j ) = (B j(m)u j ) m and its computation is discussed in [15] Equation (5a) is the forward problem, which (given m) can be solved forward in time Equation (5b) is the adjoint problem which (given m and u) can be solved backward in time Finally, equation (5c) couples the variables and can be thought as an equation for m If for a given m one solves the forward problem (5a) and the adjoint problem (5b) and substitutes u and λ into (5c) then (5c) is the reduced gradient (see [13] for discussion) The whole system (5) can be thought of as a discretization of a nonlinear elliptic boundary value problem for u, λ and m in space-time This nonlinear system has to be solved for a few regularization parameters α s We now discuss two numerical approaches for the solution of the problem λ 1 λ s (5c) 3 A Reduced space method There are many options for the solution of the nonlinear system of equations (5) In [15, 11] we have advocated linearization of the system followed by an all-at-once approach for the 4

5 solution of the linearized system For the applications we consider here, current hardware does not support such algorithms The problem is the shear size of the system For example, if the domain is discretized using voxels in space and 128 time steps then we requires roughly 20Gb of RAM just to store the vectors u and λ If we further require to use an iterative method then we require roughly 100Gb of RAM (depends on the iterative solver and our choice of preconditioner) We therefore use a variant of a reduced space approach [20, 13] In the context of the classical application of the reduced Hessian method (see [20]), the linearized constraint (forward problem) has the form Ã(m + s)u b (Ã(m) ) ( ) u G s b = 0 where Ã(m) is the forward modeling matrix in (3), G = [G 1,, G s ], b = [u 0, 0,, 0] and s is a perturbation to m Note that unlike classical reduced Hessian methods we have linearized the constraint with respect to u and not a perturbation in u In our case, where the constraint is linear with respect to u this does not make a difference For the reduced Hessian we need to construct (but not to directly compute) the active and null space matrices of the linearized constraint These matrices are ( ) ( ) Ã(m) Z = 1 G I and Y = I O It is easy to verify that Z spans the null space and Y spans the active space of the linearized forward problem We now follow the reduced Hessian method as presented in [20] p-552 The method approximately solves the Jacobian of the Euler-Lagrange Equations (5) At iteration j we first solve the forward problem for u (j) We then solve the adjoint problem for λ (j) and calculate and approximation to the the reduced gradient by substituting λ (j) in (5c) A steepest descent type method is obtained by taking a step in the negative direction of the reduced gradient A better algorithm can be obtained by using a Newton-like algorithm which requires an approximation to the reduced Hessian Here we use a Quasi-Newton algorithm suggested in [9] where the reduced Hessian, H is approximated by H J J + α R mm where R mm is an approximation to the Hessian of R and J approximates the sensitivities J QÃ 1 G where Q = diag[q,, Q] Methods to approximate the sensitivity matrix are beyond the scope of this paper and can be based on either physics approximations [6, 7] or on algebraic approximations [9, 3] 5

6 In this work we have used an approximation suggested in [9] although other approximations can be used We then solve the system Hs = g r (6) for the step s (using Conjugate Gradient) and update m (j+1) = m (j) + γs using a soft line search In the classical reduced Hessian method, the vector u is updated by using the linearized forward model In our case, due to the linearity of the forward problem with respect to u we simply solve the forward problem to update u (which follows in the next iteration) The process is summarized in algorithm 1 Algorithm 1 Time domain Inversion: m TDI(α, m ref ) Set m m ref, while true do Solve (5a) for u Solve (5b) for λ Calculate the reduced gradient (5c) Solve the linear system (6) update m = m + γs where γ 1 Check for convergence end while Note that one does not require to store the vectors λ when solving the adjoint problem (5b) by backward substitution Since we require only the reduced gradient, there is no need to keep λ j Instead, it is possible to initialize the reduced gradient with αr m and after the computation of the j th product G(m, u j ) λ j, simply add it to the current reduced gradient approximation A second point is that due to the linearity of the forward problem we obtain feasibility after each iteration Therefore, the parameter γ in the line search is chosen solely based on the size of the objective function There is a major disadvantages to the Reduced Hessian process The algorithm is not easily parallelizable Although this is true in general it particularly effects our problem where a four dimensional constrained optimization problem is solved We now discuss how to improve this algorithm by using a time decomposition method 4 Time Decomposition The process of solving the Euler-Lagrange equations (5) using the reduced Hessian algorithm (1) is not easily parallelizable Solving both the forward and the adjoint problems are sequential in nature and though there is recent development in parallelizing time domain codes, such a process is not trivial We therefore suggest a different algorithm which allows for easy parallelization 6

7 Assume first we want to parallelize our algorithm over two processors To do that, we divide the time domain into two macro-times The early time [t 0, t p ] and the late time [t p+1, t s ] We now rewrite the early time system first B 1 (m) u 1 u 0 = (7a) I B p (m) u p 0 B 1 (m) I λ Q (d 1 Qu 1 ) 1 = (7b) B p (m) λ p Q (d p Qu p ) + λ p+1 we then write the late time system B p+1 (m) I B p+1 (m) I B s (m) B s (m) u p+1 u s λ p+1 λ s = = u p 0 Q (d p+1 Qu p ) Q (d s Qu s ) (7c) (7d) Finally, we write the reduced gradient as a combination g r = ( G(m, u 1 ) G(m, u p ) ) λ 1 λ p ( G(m, up+1 ) G(m, u s ) ) + λ p+1 λ s + αr m (m m ref ) = 0 (7e) The early-late systems are almost decoupled If we have λ p+1 then we could simply solve the early time system (7a)-(7b) If we have u p then we could also solve for the late times system (7c)-(7d) We could then combine these solution in order to obtain an approximation to the reduced gradient This observation suggests a strategy to obtain a highly parallelizable algorithm The basic idea is as follows Guess an approximation to u p and λ p+1 Using this guess, solve the systems (7a)-(7b) and (7c)-(7d) in parallel and update the reduced gradient Finally, use a Newton-like step to update the log conductivity m Using the updated models we can now go back to the decoupled forward and adjoint problem (7a)-(7b) and (7c)-(7d), update the matrices B j (m) and repeat this calculation Note that after a single sweep of this algorithm we can also update u p and λ p+1 7

8 This of-course generalized to more than two macro-time steps In general we can assume l + 1 macro time steps, at times [t p1,, t pl ] The forward-adjoint problems decouple into l systems of the form B pk (m) B pk (m) I B pk+1 1(m) I B pk+1 1(m) u pk u pk+1 1 λ pk λ pk+1 1 = = u pk 1 0 (8) Q (d pk Qu pk ) Q (d pk+1 1 Qu pk+1 1) + λ pk+1 which can be solved in parallel The reduced gradient is then computed by summing the contribution of each of the systems and a Newton like step as per equation (6) follows Note that only the solution of the decoupled forward and adjoint problems can be done in parallel, however, since the majority of the computing is done in this stage such parallelization can significantly reduce the overall computational time The algorithm involves with solving linear systems of equations at each step over a shorter time span and therefore the storage requires is only for the field u over this short time rather than the whole time Thus we are able to tackle very large problems even with modest computational power It is clear that at the end of the above process feasibility is lost, that is, the solution u is feasible only in the first macro time step The solution at later macro steps are not feasible due to the wrong initial guess If we get too far from being feasible then, similar to secondary correction in SQP methods [20] and to [13] we preform a step towards feasibility by solving the forward problem However, since the problem is stiff, approximating the solution at late time by the solution of the previous model is usually not a bad approximation The algorithm is summarized in algorithm 2 5 More on time decomposition To explain some of the properties of the time decomposition method we note that the Euler Lagrange system (5) can be written of the following set of non linear equations B 1 (m) 0 u 1 I B s (m) 0 Q u s Q B 1 (m) I 0 λ 1 = Q Q B s (m) 0 λ s G 1 G s αl m u 0 0 Q d 1 Q d s αlm ref (9) 8

9 Algorithm 2 Parallel Time domain Inversion: m PTDI(α, m ref, [p 1, p l ]) Initialize: m m ref, [λ (0) p 1 +1,, λ (0) p l +1] = 0 Solve the forward problem (3) with m and store {u (0) for j = 1, 2, do for k = 1l (in parallel) do p k 1; k = 1 l} Given [λ (j 1) p k+1, u (j 1) p k 1, m] solve (8) for [u pk,, u pk+1 1] and [λ pk,, λ pk+1 1] end for Calculate the reduced gradients g r using (7e) if g r < tol then break end if Use g r calculate the step s by solving (6) update m m + γs where γ 1 update [λ (j 1) p k+1, u (j 1) p k 1 ] end for For simplicity we write the matrix in (9) in block form Ã(m) u Q Q Ã(m) λ = G αl m b Q d αlm ref (10) where as introduced in Section 3, Ã(m) is the forward problem matrix in (3), b = [u 0,, 0], G = [G 1,, G s ] and Q = diag(q,, Q) In the time decomposition, we modify the forward and adjoint problems Assume that we have only two macro-time steps Zooming on Ã, we see that decomposing in time is equivalent to the following simple matrix decomposition à = Ã1 + Ã2 where B 1 (m) I à 1 = B 2 (m) 0 B p+1 (m) I B s 1 (m) I B s (m) 9

10 and 0 à 2 = I 0 0 Substituting the decomposition into the large system (10) and as usual moving terms with Ã2 to the right hand side we see that an iteration in the time decomposition method presented above can be written as the following nonlinear operator splitting à 1 Q Q à 1 G Q Q à 1 G u λ = αl m λ + = αl m + b Q d αlm ref à 2 à 2 u λ (11) m Following the time decomposition algorithm we note that it can be thought as a nonlinear iteration of the form à 1 u + b à 2 u Q d (12) αlm ref This iteration can be thought of as a block nonlinear Jacobi iteration where we freeze u p and λ p and solve for the rest of the variables Local convergence properties of the nonlinear Jacobi iteration was analyzed in [21] and although it is possible to use the theory it is practically very hard to assess if the conditions that the theory depend on actually hold Therefore, as previously discussed in Algorithm 2 we do not simply accept the next iterate but rather compute a perturbation s = m + m and using a line search accept only steps that decrease the value of the objective function It is interesting to see that the nonlinear system (12) is in fact the Euler-Lagrange equations for the problem minimize 1 2 à 2 λ m s Qu + i d i 2 + αr(m + m ref ) (13a) i=1 subject to à 1 (m)u + = b Ã2(m)u (13b) with a particular initialization of the Lagrange multipliers This implies that a simple implementation of the problem can be done using the same optimization code for the unconstrained problem, replacing the constraint which is coupled in time to an approximate constraint which is uncoupled in time 10

11 Figure 1: First test model and experiment The source is a loop on the surface of the earth and the receivers are in boreholes 6 Numerical Examples In this section we experiment with our method demonstrate its effectiveness and explore its weaknesses To do that we experiment with two model problems Both problems are commonly used to test 3D time domain electromagnetic inversion codes for geophysical applications 61 Examples I In our first example we consider the model of two anomalies in a homogenous media with conductivity of 10 2 S/m One of the anomalies has the conductivity of 10 1 S/m while the other have the conductivity of 10 3 S/m The conductive block is m and it is berried 30m below the ground The resistive block is m and it is berried 20m below the ground The source is a square current loop on the top of the earth with sides that are 130m each 32 time measurements are taken after an instant step function shut down of the current in the loop and they are measured on a log grid starting at 10 6 Sec to 10 3 Sec The measurements are taken in boreholes which are located at the corners of the loop The depth of each borehole is 100m and we place 18 receivers in each borehole Each receiver measure 3 components of the magnetic field The total number of data is 18(recievers) 32(times) 4(boreholes) 3(fields) = 6612 We discretize the problem on a grid size The size of the unknown discrete conductivities vector is A sketch of the model is plotted in Figure 1 The data at time 10 6 is plotted in Figure 2 and it is polluted with 1% noise 11

12 Figure 2: 3 components of the magnetic fields at each borehole at t = 10 6 Sec Red-Observed and Blue-Recovered 12

13 β NP Out Iter Av In Iter Rel Time Misfit Table 1: Experiments for 3D EM problem I Solving the forward problem on this mesh on a single 22Ghz processor takes roughly 160 minutes Since an inverse problem can be involved with as many as 50 forward/adjoint problems, a typical inverse problem is solved in a few days It is therefore important to try to reduce the solution time by parallelization In our experiment we solve the problem using 1, 2, 4, 8 and 16 processors For each experiment we record the number of outer iterations, the number of inner iterations (the iterations of the decoupled problems), the relative time compared with a single processor and the misfit We experiment with two regularization parameters, one which yields a misfit which is slightly larger than needed and another which yields a misfit which is close to the χ 2 misfit (see [22]) The results of this experiment are presented in Table 1 The Table show that as expected the methods does not scale perfectly when the number of processors grows Nevertheless, the computational time is reduced to roughly 50% for 8 processors The results of the inversion are plotted in Figure 3 We see that both anomalies have been clearly identified although their dynamical range is reduced This is typical to electromagnetic inverse problems [19] 62 Examples II In our second example we consider the model of a single anomaly of 1S/m in a homogenous media with conductivity of S/m Unlike the previous example, the conductivity contrast is high which makes the inverse problem less linear The size of the conductive block is m and it is berried 50m below the ground The source is a square current loop on the top of the earth with sides that are 1000m each The center of the loop is lactated 1000m from the center of the block Such a configuration is typical in exploration scenarios [23] The model is plotted in Figure 4 62 time measurements are taken after an instant step function shut down of the current in the loop and they are measured on a log grid starting at 10 6 Sec to 10 2 Sec The measurements 3 component magnetic fields and they are taken on the surface above the 13

14 Figure 3: Recovered model for the first test problem Figure 4: The second test model 14

15 Figure 5: The second test model block on a grid of The total number of data is 41 2 (recievers) 62(times) 3(fields) = The observed data is polluted with 1% noise The observed and recovered data is plotted in Figure 2 To solve the problem we discretize on a grid of size The size of the unknown discrete conductivities vector is once again use 1, 2, 4, 8 and 16 processors The results of this experiments are presented in Table 3 The table demonstrates that although the conductivity contrast is much higher, the effectiveness of the method is unchanged Once again we get most of the benefit by using 4-8 processors Using more processors yields only a modest improvement in computation time Finally, we plot the recovered conductivity in Figure 5 7 Conclusion In this paper we have explored a method to parallelize 3D time domain electromagnetic inverse problems We have investigated the proposed scheme as a variant of an operator splitting and show that it can be thought of as a nonlinear block Jacobi method for the solution of the Euler-Lagrange equations As expected by other block Jacobi methods, the approach does not have an optimal scaling properties and the performance of the method deteriorate as the number of sub-domain grows Nevertheless, the approach can reduce the computational time in two even given modest computational tools 15

16 Observed Predicted Table 2: The vertical magnetic fields for times 1, 20, 40 and 62 16

17 β ND Out Iter Av In Iter Rel Time Misfit References Table 3: Experiments for 3D EM problem II [1] U Ascher and E Haber Grid refinement and scaling for distributed parameter estimation problems Inverse Problems, 17: , 2001 [2] U Ascher, E Haber, and H Haung On effective methods for implicit piecewise smooth surface recovery Submitted to SISC, 2004 [3] G Biros and O Ghattas Parallel Lagrange-Newton-Krylov-Schur methods for PDEconstrained optimization Parts I,II Preprints, 2000 [4] R Casanova, A Silva, and A R Borges A quantitative algorithm for parameter estimation in magnetic induction tomography Meas Sci Technol, 15: , 2004 [5] HW Engl, M Hanke, and A Neubauer Regularization of Inverse Problems Kluwer, 1996 [6] C G Farquharson and D W Oldenburg Approximate sensitivities for the electromagnetic inverse problem Geophysical Journal International, 126: , 1996 [7] C G Farquharson and D W Oldenburg Approximate Sensitivities for the Multi- Dimensional Electromagnetic Inverse Problem, In Three-Dimensional Electromagnetics, ML Oristaglio and BR Spies (eds) Society of Exploration Geophysicists, 1999 [8] E Haber A multilevel, level-set method for optimizing eigenvalues in shape design problems JCP, 115:1 15, 2004 [9] E Haber Quasi-newton methods methods for large scale electromagnetic inverse problems Inverse Peoblems, 21,

18 [10] E Haber and U Ascher Fast finite volume simulation of 3D electromagnetic problems with highly discontinuous coefficients SIAM J Scient Comput, 22: , 2001 [11] E Haber and U Ascher Preconditioned all-at-one methods for large, sparse parameter estimation problems Inverse Problems, 17: , 2001 [12] E Haber, U Ascher, D Aruliah, and D Oldenburg Fast simulation of 3D electromagnetic using potentials J Comput Phys, 163: , 2000 [13] E Haber, U Ascher, and D Oldenburg On optimization techniques for solving nonlinear inverse problems Inverse problems, 16: , 2000 [14] E Haber, U Ascher, and D Oldenburg Inversion of frequency and time domain electromagnetic data Geophysics, 69: , 2004 n5 [15] E Haber, D Oldenburg, and U Ascher Inversion of 3D electromagnetic data - a constrained optimization approach In SEG, Calgary, Aug 2000 [16] M Heinkenschloss Time domain decomposition iterative methods for the solution of distributed linear quadratic optimal control problems Technical report, CAAM, Rice University, 2002 [17] P J Huber Robust estimation of a location parameter Ann Math Stats, 35:73 101, 1964 [18] J Jin The Finite Element Method in Electromagnetics John Wiley and Sons, 1993 [19] G Newman Three dimensional magnetotelluric modeling and inversion Proceedings of the Second Symposium on 3D Electromagnetics, 1999 Salt Lake City Utah [20] J Nocedal and S Wright Numerical Optimization New York: Springer, 1999 [21] J M Ortaga and W C Rheinboldt Iterative Solution of Nonlinear Equations in Several Variables (2nd edition) SIAM, 2003 [22] R L Parker Geophysical Inverse Theory Princeton University Press, Princeton NJ, 1994 [23] SH Ward and GW Hohmann Electromagnetic theory for geophysical applications Electromagnetic Methods in Applied Geophysics, 1: , 1988 Soc Expl Geophys 18

Parallelizing large scale time domain electromagnetic inverse problem

Parallelizing large scale time domain electromagnetic inverse problem Parallelizing large scale time domain electromagnetic inverse problems Eldad Haber with: D. Oldenburg & R. Shekhtman + Emory University, Atlanta, GA + The University of British Columbia, Vancouver, BC,

More information

A multilevel, level-set method for optimizing eigenvalues in shape design problems

A multilevel, level-set method for optimizing eigenvalues in shape design problems A multilevel, level-set method for optimizing eigenvalues in shape design problems E. Haber July 22, 2003 Abstract In this paper we consider optimal design problems that involve shape optimization. The

More information

Computational methods for large distributed parameter estimation problems with possible discontinuities

Computational methods for large distributed parameter estimation problems with possible discontinuities Computational methods for large distributed parameter estimation problems with possible discontinuities Uri Ascher Department of Computer Science, University of British Columbia, Vancouver, BC, V6T 1Z4,

More information

Achieving depth resolution with gradient array survey data through transient electromagnetic inversion

Achieving depth resolution with gradient array survey data through transient electromagnetic inversion Achieving depth resolution with gradient array survey data through transient electromagnetic inversion Downloaded /1/17 to 128.189.118.. Redistribution subject to SEG license or copyright; see Terms of

More information

A multigrid method for large scale inverse problems

A multigrid method for large scale inverse problems A multigrid method for large scale inverse problems Eldad Haber Dept. of Computer Science, Dept. of Earth and Ocean Science University of British Columbia haber@cs.ubc.ca July 4, 2003 E.Haber: Multigrid

More information

A wavelet multiscale method for inversion of Maxwell equations

A wavelet multiscale method for inversion of Maxwell equations Appl. Math. Mech. -Engl. Ed. 30(8), 35 44 (2009) DOI:.07/s483-009-08-1 c Shanghai University and Springer-Verlag 2009 Applied Mathematics and Mechanics (English Edition) A wavelet multiscale method for

More information

Direct Current Resistivity Inversion using Various Objective Functions

Direct Current Resistivity Inversion using Various Objective Functions Direct Current Resistivity Inversion using Various Objective Functions Rowan Cockett Department of Earth and Ocean Science University of British Columbia rcockett@eos.ubc.ca Abstract In geophysical applications

More information

Inversion of 3D electromagnetic data in frequency and time domain using an inexact all-at-once approach

Inversion of 3D electromagnetic data in frequency and time domain using an inexact all-at-once approach GEOPHYSICS, VOL. 69, NO. 5 (SEPTEMBER-OCTOBER 2004); P. 1216 1228, 9 FIGS., 2 TABLES. 10.1190/1.1801938 Inversion of 3D electromagnetic data in frequency and time domain using an inexact all-at-once approach

More information

Simulation based optimization

Simulation based optimization SimBOpt p.1/52 Simulation based optimization Feb 2005 Eldad Haber haber@mathcs.emory.edu Emory University SimBOpt p.2/52 Outline Introduction A few words about discretization The unconstrained framework

More information

Simultaneous estimation of wavefields & medium parameters

Simultaneous estimation of wavefields & medium parameters Simultaneous estimation of wavefields & medium parameters reduced-space versus full-space waveform inversion Bas Peters, Felix J. Herrmann Workshop W- 2, The limit of FWI in subsurface parameter recovery.

More information

Computational methods for large distributed parameter estimation problems in 3D

Computational methods for large distributed parameter estimation problems in 3D Computational methods for large distributed parameter estimation problems in 3D U. M. Ascher E. Haber March 8, 03 Abstract This paper considers problems of distributed parameter estimation from data measurements

More information

GEOPHYSICAL INVERSE THEORY AND REGULARIZATION PROBLEMS

GEOPHYSICAL INVERSE THEORY AND REGULARIZATION PROBLEMS Methods in Geochemistry and Geophysics, 36 GEOPHYSICAL INVERSE THEORY AND REGULARIZATION PROBLEMS Michael S. ZHDANOV University of Utah Salt Lake City UTAH, U.S.A. 2OO2 ELSEVIER Amsterdam - Boston - London

More information

AM 205: lecture 19. Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods

AM 205: lecture 19. Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods AM 205: lecture 19 Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods Optimality Conditions: Equality Constrained Case As another example of equality

More information

ETNA Kent State University

ETNA Kent State University Electronic Transactions on Numerical Analysis. Volume 5, pp. -7, 23. Copyright 22,. ISSN 68-963. ETNA A MULTIGRID METHOD FOR DISTRIBUTED PARAMETER ESTIMATION PROBLEMS U. M. ASCHER AND E. HABER Abstract.

More information

Inversion of 3D Electromagnetic Data in frequency and time domain using an inexact all-at-once approach

Inversion of 3D Electromagnetic Data in frequency and time domain using an inexact all-at-once approach Inversion of 3D Electromagnetic Data in frequency and time domain using an inexact all-at-once approach Eldad Haber UBC-Geophysical Inversion Facility Department of Earth and Ocean Sciences University

More information

AM 205: lecture 19. Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods

AM 205: lecture 19. Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods AM 205: lecture 19 Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods Quasi-Newton Methods General form of quasi-newton methods: x k+1 = x k α

More information

Numerical Methods for PDE-Constrained Optimization

Numerical Methods for PDE-Constrained Optimization Numerical Methods for PDE-Constrained Optimization Richard H. Byrd 1 Frank E. Curtis 2 Jorge Nocedal 2 1 University of Colorado at Boulder 2 Northwestern University Courant Institute of Mathematical Sciences,

More information

Trust-Region SQP Methods with Inexact Linear System Solves for Large-Scale Optimization

Trust-Region SQP Methods with Inexact Linear System Solves for Large-Scale Optimization Trust-Region SQP Methods with Inexact Linear System Solves for Large-Scale Optimization Denis Ridzal Department of Computational and Applied Mathematics Rice University, Houston, Texas dridzal@caam.rice.edu

More information

1.2 Derivation. d p f = d p f(x(p)) = x fd p x (= f x x p ). (1) Second, g x x p + g p = 0. d p f = f x g 1. The expression f x gx

1.2 Derivation. d p f = d p f(x(p)) = x fd p x (= f x x p ). (1) Second, g x x p + g p = 0. d p f = f x g 1. The expression f x gx PDE-constrained optimization and the adjoint method Andrew M. Bradley November 16, 21 PDE-constrained optimization and the adjoint method for solving these and related problems appear in a wide range of

More information

Parameter Identification in Partial Differential Equations

Parameter Identification in Partial Differential Equations Parameter Identification in Partial Differential Equations Differentiation of data Not strictly a parameter identification problem, but good motivation. Appears often as a subproblem. Given noisy observation

More information

PARALLEL LAGRANGE-NEWTON-KRYLOV-SCHUR METHODS FOR PDE-CONSTRAINED OPTIMIZATION. PART I: THE KRYLOV-SCHUR SOLVER

PARALLEL LAGRANGE-NEWTON-KRYLOV-SCHUR METHODS FOR PDE-CONSTRAINED OPTIMIZATION. PART I: THE KRYLOV-SCHUR SOLVER PARALLEL LAGRANGE-NEWTON-KRYLOV-SCHUR METHODS FOR PDE-CONSTRAINED OPTIMIZATION. PART I: THE KRYLOV-SCHUR SOLVER GEORGE BIROS AND OMAR GHATTAS Abstract. Large scale optimization of systems governed by partial

More information

Multi-source inversion of TEM data: with field applications to Mt. Milligan

Multi-source inversion of TEM data: with field applications to Mt. Milligan Multi-source inversion of TEM data: with field applications to Mt. Milligan D. Oldenburg, E. Haber, D. Yang Geophysical Inversion Facility, University of British Columbia, Vancouver, BC, Canada Summary

More information

Direct Methods. Moritz Diehl. Optimization in Engineering Center (OPTEC) and Electrical Engineering Department (ESAT) K.U.

Direct Methods. Moritz Diehl. Optimization in Engineering Center (OPTEC) and Electrical Engineering Department (ESAT) K.U. Direct Methods Moritz Diehl Optimization in Engineering Center (OPTEC) and Electrical Engineering Department (ESAT) K.U. Leuven Belgium Overview Direct Single Shooting Direct Collocation Direct Multiple

More information

IBM Research Report. E. Haber Department of Mathematics University of British Colombia Vancouver, Canada

IBM Research Report. E. Haber Department of Mathematics University of British Colombia Vancouver, Canada RC24978 (W1004-055) April 16, 2010 Mathematics IBM Research Report Numerical Methods for the Design of Large-Scale Nonlinear Discrete Ill-Posed Inverse Problems E. Haber Department of Mathematics University

More information

A Robust Preconditioner for the Hessian System in Elliptic Optimal Control Problems

A Robust Preconditioner for the Hessian System in Elliptic Optimal Control Problems A Robust Preconditioner for the Hessian System in Elliptic Optimal Control Problems Etereldes Gonçalves 1, Tarek P. Mathew 1, Markus Sarkis 1,2, and Christian E. Schaerer 1 1 Instituto de Matemática Pura

More information

Chapter 6. Finite Element Method. Literature: (tiny selection from an enormous number of publications)

Chapter 6. Finite Element Method. Literature: (tiny selection from an enormous number of publications) Chapter 6 Finite Element Method Literature: (tiny selection from an enormous number of publications) K.J. Bathe, Finite Element procedures, 2nd edition, Pearson 2014 (1043 pages, comprehensive). Available

More information

An Introduction to Algebraic Multigrid (AMG) Algorithms Derrick Cerwinsky and Craig C. Douglas 1/84

An Introduction to Algebraic Multigrid (AMG) Algorithms Derrick Cerwinsky and Craig C. Douglas 1/84 An Introduction to Algebraic Multigrid (AMG) Algorithms Derrick Cerwinsky and Craig C. Douglas 1/84 Introduction Almost all numerical methods for solving PDEs will at some point be reduced to solving A

More information

PARALLEL LAGRANGE NEWTON KRYLOV SCHUR METHODS FOR PDE-CONSTRAINED OPTIMIZATION. PART I: THE KRYLOV SCHUR SOLVER

PARALLEL LAGRANGE NEWTON KRYLOV SCHUR METHODS FOR PDE-CONSTRAINED OPTIMIZATION. PART I: THE KRYLOV SCHUR SOLVER SIAM J. SCI. COMPUT. Vol. 27, No. 2, pp. 687 713 c 2005 Society for Industrial and Applied Mathematics PARALLEL LAGRANGE NEWTON KRYLOV SCHUR METHODS FOR PDE-CONSTRAINED OPTIMIZATION. PART I: THE KRYLOV

More information

2.29 Numerical Fluid Mechanics Spring 2015 Lecture 9

2.29 Numerical Fluid Mechanics Spring 2015 Lecture 9 Spring 2015 Lecture 9 REVIEW Lecture 8: Direct Methods for solving (linear) algebraic equations Gauss Elimination LU decomposition/factorization Error Analysis for Linear Systems and Condition Numbers

More information

Review for Exam 2 Ben Wang and Mark Styczynski

Review for Exam 2 Ben Wang and Mark Styczynski Review for Exam Ben Wang and Mark Styczynski This is a rough approximation of what we went over in the review session. This is actually more detailed in portions than what we went over. Also, please note

More information

1 Computing with constraints

1 Computing with constraints Notes for 2017-04-26 1 Computing with constraints Recall that our basic problem is minimize φ(x) s.t. x Ω where the feasible set Ω is defined by equality and inequality conditions Ω = {x R n : c i (x)

More information

MS&E 318 (CME 338) Large-Scale Numerical Optimization

MS&E 318 (CME 338) Large-Scale Numerical Optimization Stanford University, Management Science & Engineering (and ICME) MS&E 318 (CME 338) Large-Scale Numerical Optimization 1 Origins Instructor: Michael Saunders Spring 2015 Notes 9: Augmented Lagrangian Methods

More information

A new inversion method for dissipating electromagnetic problem

A new inversion method for dissipating electromagnetic problem A new inversion method for dissipating electromagnetic problem Elena Cherkaev Abstract The paper suggests a new technique for solution of the inverse problem for Maxwell s equations in a dissipating medium.

More information

On effective methods for implicit piecewise smooth surface recovery

On effective methods for implicit piecewise smooth surface recovery On effective methods for implicit piecewise smooth surface recovery U. M. Ascher E. Haber H. Huang October 15, 2004 Abstract This paper considers the problem of reconstructing a piecewise smooth model

More information

Downloaded 01/09/13 to Redistribution subject to SEG license or copyright; see Terms of Use at

Downloaded 01/09/13 to Redistribution subject to SEG license or copyright; see Terms of Use at 3D modeling of IP effects on electromagnetic data in the time domain David Marchant, Eldad Haber, Laurens Beran and Douglas W. Oldenburg, University of British Columbia Downloaded /9/3 to 37.82.7.. Redistribution

More information

Linear Solvers. Andrew Hazel

Linear Solvers. Andrew Hazel Linear Solvers Andrew Hazel Introduction Thus far we have talked about the formulation and discretisation of physical problems...... and stopped when we got to a discrete linear system of equations. Introduction

More information

On the modified conjugate gradient method in cloth simulation

On the modified conjugate gradient method in cloth simulation The Visual Computer manuscript No. (will be inserted by the editor) On the modified conjugate gradient method in cloth simulation Uri M. Ascher 1, Eddy Boxerman 2 Department of Computer Science, University

More information

Lecture 9 Approximations of Laplace s Equation, Finite Element Method. Mathématiques appliquées (MATH0504-1) B. Dewals, C.

Lecture 9 Approximations of Laplace s Equation, Finite Element Method. Mathématiques appliquées (MATH0504-1) B. Dewals, C. Lecture 9 Approximations of Laplace s Equation, Finite Element Method Mathématiques appliquées (MATH54-1) B. Dewals, C. Geuzaine V1.2 23/11/218 1 Learning objectives of this lecture Apply the finite difference

More information

Full-Waveform Inversion with Gauss- Newton-Krylov Method

Full-Waveform Inversion with Gauss- Newton-Krylov Method Full-Waveform Inversion with Gauss- Newton-Krylov Method Yogi A. Erlangga and Felix J. Herrmann {yerlangga,fherrmann}@eos.ubc.ca Seismic Laboratory for Imaging and Modeling The University of British Columbia

More information

Algorithms for Constrained Optimization

Algorithms for Constrained Optimization 1 / 42 Algorithms for Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University April 19, 2015 2 / 42 Outline 1. Convergence 2. Sequential quadratic

More information

Preface to the Second Edition. Preface to the First Edition

Preface to the Second Edition. Preface to the First Edition n page v Preface to the Second Edition Preface to the First Edition xiii xvii 1 Background in Linear Algebra 1 1.1 Matrices................................. 1 1.2 Square Matrices and Eigenvalues....................

More information

On Lagrange multipliers of trust region subproblems

On Lagrange multipliers of trust region subproblems On Lagrange multipliers of trust region subproblems Ladislav Lukšan, Ctirad Matonoha, Jan Vlček Institute of Computer Science AS CR, Prague Applied Linear Algebra April 28-30, 2008 Novi Sad, Serbia Outline

More information

Fast Iterative Solution of Saddle Point Problems

Fast Iterative Solution of Saddle Point Problems Michele Benzi Department of Mathematics and Computer Science Emory University Atlanta, GA Acknowledgments NSF (Computational Mathematics) Maxim Olshanskii (Mech-Math, Moscow State U.) Zhen Wang (PhD student,

More information

Preconditioning Techniques Analysis for CG Method

Preconditioning Techniques Analysis for CG Method Preconditioning Techniques Analysis for CG Method Huaguang Song Department of Computer Science University of California, Davis hso@ucdavis.edu Abstract Matrix computation issue for solve linear system

More information

An introduction to PDE-constrained optimization

An introduction to PDE-constrained optimization An introduction to PDE-constrained optimization Wolfgang Bangerth Department of Mathematics Texas A&M University 1 Overview Why partial differential equations? Why optimization? Examples of PDE optimization

More information

A Multilevel Method for Image Registration

A Multilevel Method for Image Registration A Multilevel Method for Image Registration Eldad Haber Jan Modersitzki May 1, 004 Abstract In this paper we introduce a new framework for image registration. Our formulation is based on consistent discretization

More information

DUAL REGULARIZED TOTAL LEAST SQUARES SOLUTION FROM TWO-PARAMETER TRUST-REGION ALGORITHM. Geunseop Lee

DUAL REGULARIZED TOTAL LEAST SQUARES SOLUTION FROM TWO-PARAMETER TRUST-REGION ALGORITHM. Geunseop Lee J. Korean Math. Soc. 0 (0), No. 0, pp. 1 0 https://doi.org/10.4134/jkms.j160152 pissn: 0304-9914 / eissn: 2234-3008 DUAL REGULARIZED TOTAL LEAST SQUARES SOLUTION FROM TWO-PARAMETER TRUST-REGION ALGORITHM

More information

Anisotropic 2.5D Inversion of Towed Streamer EM Data from Three North Sea Fields Using Parallel Adaptive Finite Elements

Anisotropic 2.5D Inversion of Towed Streamer EM Data from Three North Sea Fields Using Parallel Adaptive Finite Elements Anisotropic 2.5D Inversion of Towed Streamer EM Data from Three North Sea Fields Using Parallel Adaptive Finite Elements K. Key (Scripps Institution of Oceanography), Z. Du* (PGS), J. Mattsson (PGS), A.

More information

Part 4: Active-set methods for linearly constrained optimization. Nick Gould (RAL)

Part 4: Active-set methods for linearly constrained optimization. Nick Gould (RAL) Part 4: Active-set methods for linearly constrained optimization Nick Gould RAL fx subject to Ax b Part C course on continuoue optimization LINEARLY CONSTRAINED MINIMIZATION fx subject to Ax { } b where

More information

On Lagrange multipliers of trust-region subproblems

On Lagrange multipliers of trust-region subproblems On Lagrange multipliers of trust-region subproblems Ladislav Lukšan, Ctirad Matonoha, Jan Vlček Institute of Computer Science AS CR, Prague Programy a algoritmy numerické matematiky 14 1.- 6. června 2008

More information

Anna Avdeeva Dmitry Avdeev and Marion Jegen. 31 March Introduction to 3D MT inversion code x3di

Anna Avdeeva Dmitry Avdeev and Marion Jegen. 31 March Introduction to 3D MT inversion code x3di Anna Avdeeva (aavdeeva@ifm-geomar.de), Dmitry Avdeev and Marion Jegen 31 March 211 Outline 1 Essential Parts of 3D MT Inversion Code 2 Salt Dome Overhang Study with 3 Outline 1 Essential Parts of 3D MT

More information

LINEAR AND NONLINEAR PROGRAMMING

LINEAR AND NONLINEAR PROGRAMMING LINEAR AND NONLINEAR PROGRAMMING Stephen G. Nash and Ariela Sofer George Mason University The McGraw-Hill Companies, Inc. New York St. Louis San Francisco Auckland Bogota Caracas Lisbon London Madrid Mexico

More information

Newton s Method and Efficient, Robust Variants

Newton s Method and Efficient, Robust Variants Newton s Method and Efficient, Robust Variants Philipp Birken University of Kassel (SFB/TRR 30) Soon: University of Lund October 7th 2013 Efficient solution of large systems of non-linear PDEs in science

More information

Inverse Problems and Optimal Design in Electricity and Magnetism

Inverse Problems and Optimal Design in Electricity and Magnetism Inverse Problems and Optimal Design in Electricity and Magnetism P. Neittaanmäki Department of Mathematics, University of Jyväskylä M. Rudnicki Institute of Electrical Engineering, Warsaw and A. Savini

More information

SUMMARY INTRODUCTION. Traditionally, the inverse problem is approached by considering. DOI

SUMMARY INTRODUCTION. Traditionally, the inverse problem is approached by considering. DOI Parametrized inversion framework for proppant volume in a hydraulically fractured reservoir Lindsey J. Heagy, A. Rowan Cockett, Douglas W. Oldenburg, UBC Geophysical Inversion Facility, University of British

More information

CHAPTER 10: Numerical Methods for DAEs

CHAPTER 10: Numerical Methods for DAEs CHAPTER 10: Numerical Methods for DAEs Numerical approaches for the solution of DAEs divide roughly into two classes: 1. direct discretization 2. reformulation (index reduction) plus discretization Direct

More information

Numerical Optimization Professor Horst Cerjak, Horst Bischof, Thomas Pock Mat Vis-Gra SS09

Numerical Optimization Professor Horst Cerjak, Horst Bischof, Thomas Pock Mat Vis-Gra SS09 Numerical Optimization 1 Working Horse in Computer Vision Variational Methods Shape Analysis Machine Learning Markov Random Fields Geometry Common denominator: optimization problems 2 Overview of Methods

More information

Migration with Implicit Solvers for the Time-harmonic Helmholtz

Migration with Implicit Solvers for the Time-harmonic Helmholtz Migration with Implicit Solvers for the Time-harmonic Helmholtz Yogi A. Erlangga, Felix J. Herrmann Seismic Laboratory for Imaging and Modeling, The University of British Columbia {yerlangga,fherrmann}@eos.ubc.ca

More information

An Inexact Newton Method for Nonlinear Constrained Optimization

An Inexact Newton Method for Nonlinear Constrained Optimization An Inexact Newton Method for Nonlinear Constrained Optimization Frank E. Curtis Numerical Analysis Seminar, January 23, 2009 Outline Motivation and background Algorithm development and theoretical results

More information

Tu Efficient 3D MT Inversion Using Finite-difference Time-domain Modelling

Tu Efficient 3D MT Inversion Using Finite-difference Time-domain Modelling Tu 11 16 Efficient 3D MT Inversion Using Finite-difference Time-domain Modelling S. de la Kethulle de Ryhove* (Electromagnetic Geoservices ASA), J.P. Morten (Electromagnetic Geoservices ASA) & K. Kumar

More information

THE solution of the absolute value equation (AVE) of

THE solution of the absolute value equation (AVE) of The nonlinear HSS-like iterative method for absolute value equations Mu-Zheng Zhu Member, IAENG, and Ya-E Qi arxiv:1403.7013v4 [math.na] 2 Jan 2018 Abstract Salkuyeh proposed the Picard-HSS iteration method

More information

A PRECONDITIONER FOR THE HELMHOLTZ EQUATION WITH PERFECTLY MATCHED LAYER

A PRECONDITIONER FOR THE HELMHOLTZ EQUATION WITH PERFECTLY MATCHED LAYER European Conference on Computational Fluid Dynamics ECCOMAS CFD 2006 P. Wesseling, E. Oñate and J. Périaux (Eds) c TU Delft, The Netherlands, 2006 A PRECONDITIONER FOR THE HELMHOLTZ EQUATION WITH PERFECTLY

More information

Key words. preconditioned conjugate gradient method, saddle point problems, optimal control of PDEs, control and state constraints, multigrid method

Key words. preconditioned conjugate gradient method, saddle point problems, optimal control of PDEs, control and state constraints, multigrid method PRECONDITIONED CONJUGATE GRADIENT METHOD FOR OPTIMAL CONTROL PROBLEMS WITH CONTROL AND STATE CONSTRAINTS ROLAND HERZOG AND EKKEHARD SACHS Abstract. Optimality systems and their linearizations arising in

More information

Higher-Order Methods

Higher-Order Methods Higher-Order Methods Stephen J. Wright 1 2 Computer Sciences Department, University of Wisconsin-Madison. PCMI, July 2016 Stephen Wright (UW-Madison) Higher-Order Methods PCMI, July 2016 1 / 25 Smooth

More information

Numerical Solution I

Numerical Solution I Numerical Solution I Stationary Flow R. Kornhuber (FU Berlin) Summerschool Modelling of mass and energy transport in porous media with practical applications October 8-12, 2018 Schedule Classical Solutions

More information

5.1 2D example 59 Figure 5.1: Parabolic velocity field in a straight two-dimensional pipe. Figure 5.2: Concentration on the input boundary of the pipe. The vertical axis corresponds to r 2 -coordinate,

More information

Quasi-Newton Methods

Quasi-Newton Methods Newton s Method Pros and Cons Quasi-Newton Methods MA 348 Kurt Bryan Newton s method has some very nice properties: It s extremely fast, at least once it gets near the minimum, and with the simple modifications

More information

The TAO Linearly-Constrained Augmented Lagrangian Method for PDE-Constrained Optimization 1

The TAO Linearly-Constrained Augmented Lagrangian Method for PDE-Constrained Optimization 1 ARGONNE NATIONAL LABORATORY 9700 South Cass Avenue Argonne, Illinois 60439 The TAO Linearly-Constrained Augmented Lagrangian Method for PDE-Constrained Optimization 1 Evan Gawlik, Todd Munson, Jason Sarich,

More information

CS 542G: Robustifying Newton, Constraints, Nonlinear Least Squares

CS 542G: Robustifying Newton, Constraints, Nonlinear Least Squares CS 542G: Robustifying Newton, Constraints, Nonlinear Least Squares Robert Bridson October 29, 2008 1 Hessian Problems in Newton Last time we fixed one of plain Newton s problems by introducing line search

More information

Lecture 14: October 17

Lecture 14: October 17 1-725/36-725: Convex Optimization Fall 218 Lecture 14: October 17 Lecturer: Lecturer: Ryan Tibshirani Scribes: Pengsheng Guo, Xian Zhou Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:

More information

A Simple Explanation of the Sobolev Gradient Method

A Simple Explanation of the Sobolev Gradient Method A Simple Explanation o the Sobolev Gradient Method R. J. Renka July 3, 2006 Abstract We have observed that the term Sobolev gradient is used more oten than it is understood. Also, the term is oten used

More information

A DECOMPOSITION PROCEDURE BASED ON APPROXIMATE NEWTON DIRECTIONS

A DECOMPOSITION PROCEDURE BASED ON APPROXIMATE NEWTON DIRECTIONS Working Paper 01 09 Departamento de Estadística y Econometría Statistics and Econometrics Series 06 Universidad Carlos III de Madrid January 2001 Calle Madrid, 126 28903 Getafe (Spain) Fax (34) 91 624

More information

Iterative Methods for Solving A x = b

Iterative Methods for Solving A x = b Iterative Methods for Solving A x = b A good (free) online source for iterative methods for solving A x = b is given in the description of a set of iterative solvers called templates found at netlib: http

More information

Some notes about PDEs. -Bill Green Nov. 2015

Some notes about PDEs. -Bill Green Nov. 2015 Some notes about PDEs -Bill Green Nov. 2015 Partial differential equations (PDEs) are all BVPs, with the same issues about specifying boundary conditions etc. Because they are multi-dimensional, they can

More information

High Performance Nonlinear Solvers

High Performance Nonlinear Solvers What is a nonlinear system? High Performance Nonlinear Solvers Michael McCourt Division Argonne National Laboratory IIT Meshfree Seminar September 19, 2011 Every nonlinear system of equations can be described

More information

7.2 Steepest Descent and Preconditioning

7.2 Steepest Descent and Preconditioning 7.2 Steepest Descent and Preconditioning Descent methods are a broad class of iterative methods for finding solutions of the linear system Ax = b for symmetric positive definite matrix A R n n. Consider

More information

An Active Set Strategy for Solving Optimization Problems with up to 200,000,000 Nonlinear Constraints

An Active Set Strategy for Solving Optimization Problems with up to 200,000,000 Nonlinear Constraints An Active Set Strategy for Solving Optimization Problems with up to 200,000,000 Nonlinear Constraints Klaus Schittkowski Department of Computer Science, University of Bayreuth 95440 Bayreuth, Germany e-mail:

More information

1. Introduction. In this work we consider the solution of finite-dimensional constrained optimization problems of the form

1. Introduction. In this work we consider the solution of finite-dimensional constrained optimization problems of the form MULTILEVEL ALGORITHMS FOR LARGE-SCALE INTERIOR POINT METHODS MICHELE BENZI, ELDAD HABER, AND LAUREN TARALLI Abstract. We develop and compare multilevel algorithms for solving constrained nonlinear variational

More information

Numerical optimization

Numerical optimization Numerical optimization Lecture 4 Alexander & Michael Bronstein tosca.cs.technion.ac.il/book Numerical geometry of non-rigid shapes Stanford University, Winter 2009 2 Longest Slowest Shortest Minimal Maximal

More information

Numerical Analysis of Electromagnetic Fields

Numerical Analysis of Electromagnetic Fields Pei-bai Zhou Numerical Analysis of Electromagnetic Fields With 157 Figures Springer-Verlag Berlin Heidelberg New York London Paris Tokyo Hong Kong Barcelona Budapest Contents Part 1 Universal Concepts

More information

Incomplete Cholesky preconditioners that exploit the low-rank property

Incomplete Cholesky preconditioners that exploit the low-rank property anapov@ulb.ac.be ; http://homepages.ulb.ac.be/ anapov/ 1 / 35 Incomplete Cholesky preconditioners that exploit the low-rank property (theory and practice) Artem Napov Service de Métrologie Nucléaire, Université

More information

Multigrid and Iterative Strategies for Optimal Control Problems

Multigrid and Iterative Strategies for Optimal Control Problems Multigrid and Iterative Strategies for Optimal Control Problems John Pearson 1, Stefan Takacs 1 1 Mathematical Institute, 24 29 St. Giles, Oxford, OX1 3LB e-mail: john.pearson@worc.ox.ac.uk, takacs@maths.ox.ac.uk

More information

Final Examination. CS 205A: Mathematical Methods for Robotics, Vision, and Graphics (Fall 2013), Stanford University

Final Examination. CS 205A: Mathematical Methods for Robotics, Vision, and Graphics (Fall 2013), Stanford University Final Examination CS 205A: Mathematical Methods for Robotics, Vision, and Graphics (Fall 2013), Stanford University The exam runs for 3 hours. The exam contains eight problems. You must complete the first

More information

New Fast Kalman filter method

New Fast Kalman filter method New Fast Kalman filter method Hojat Ghorbanidehno, Hee Sun Lee 1. Introduction Data assimilation methods combine dynamical models of a system with typically noisy observations to obtain estimates of the

More information

An Inexact Sequential Quadratic Optimization Method for Nonlinear Optimization

An Inexact Sequential Quadratic Optimization Method for Nonlinear Optimization An Inexact Sequential Quadratic Optimization Method for Nonlinear Optimization Frank E. Curtis, Lehigh University involving joint work with Travis Johnson, Northwestern University Daniel P. Robinson, Johns

More information

Preconditioned all-at-once methods for large, sparse parameter estimation problems

Preconditioned all-at-once methods for large, sparse parameter estimation problems INSTITUTE OF PHYSICS PUBLISHING Inverse Problems 17 (2001) 1847 1864 INVERSE PROBLEMS PII: S0266-5611(01)20474-2 Preconditioned all-at-once methods for large, sparse parameter estimation problems E Haber

More information

FETI domain decomposition method to solution of contact problems with large displacements

FETI domain decomposition method to solution of contact problems with large displacements FETI domain decomposition method to solution of contact problems with large displacements Vít Vondrák 1, Zdeněk Dostál 1, Jiří Dobiáš 2, and Svatopluk Pták 2 1 Dept. of Appl. Math., Technical University

More information

Scalable Non-Linear Compact Schemes

Scalable Non-Linear Compact Schemes Scalable Non-Linear Compact Schemes Debojyoti Ghosh Emil M. Constantinescu Jed Brown Mathematics Computer Science Argonne National Laboratory International Conference on Spectral and High Order Methods

More information

Numerical optimization. Numerical optimization. Longest Shortest where Maximal Minimal. Fastest. Largest. Optimization problems

Numerical optimization. Numerical optimization. Longest Shortest where Maximal Minimal. Fastest. Largest. Optimization problems 1 Numerical optimization Alexander & Michael Bronstein, 2006-2009 Michael Bronstein, 2010 tosca.cs.technion.ac.il/book Numerical optimization 048921 Advanced topics in vision Processing and Analysis of

More information

Optimization and Root Finding. Kurt Hornik

Optimization and Root Finding. Kurt Hornik Optimization and Root Finding Kurt Hornik Basics Root finding and unconstrained smooth optimization are closely related: Solving ƒ () = 0 can be accomplished via minimizing ƒ () 2 Slide 2 Basics Root finding

More information

Numerical Optimization Algorithms

Numerical Optimization Algorithms Numerical Optimization Algorithms 1. Overview. Calculus of Variations 3. Linearized Supersonic Flow 4. Steepest Descent 5. Smoothed Steepest Descent Overview 1 Two Main Categories of Optimization Algorithms

More information

INTERGRID OPERATORS FOR THE CELL CENTERED FINITE DIFFERENCE MULTIGRID ALGORITHM ON RECTANGULAR GRIDS. 1. Introduction

INTERGRID OPERATORS FOR THE CELL CENTERED FINITE DIFFERENCE MULTIGRID ALGORITHM ON RECTANGULAR GRIDS. 1. Introduction Trends in Mathematics Information Center for Mathematical Sciences Volume 9 Number 2 December 2006 Pages 0 INTERGRID OPERATORS FOR THE CELL CENTERED FINITE DIFFERENCE MULTIGRID ALGORITHM ON RECTANGULAR

More information

A Sobolev trust-region method for numerical solution of the Ginz

A Sobolev trust-region method for numerical solution of the Ginz A Sobolev trust-region method for numerical solution of the Ginzburg-Landau equations Robert J. Renka Parimah Kazemi Department of Computer Science & Engineering University of North Texas June 6, 2012

More information

AMS526: Numerical Analysis I (Numerical Linear Algebra)

AMS526: Numerical Analysis I (Numerical Linear Algebra) AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 24: Preconditioning and Multigrid Solver Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 5 Preconditioning Motivation:

More information

Lecture Notes: Geometric Considerations in Unconstrained Optimization

Lecture Notes: Geometric Considerations in Unconstrained Optimization Lecture Notes: Geometric Considerations in Unconstrained Optimization James T. Allison February 15, 2006 The primary objectives of this lecture on unconstrained optimization are to: Establish connections

More information

Constrained optimization: direct methods (cont.)

Constrained optimization: direct methods (cont.) Constrained optimization: direct methods (cont.) Jussi Hakanen Post-doctoral researcher jussi.hakanen@jyu.fi Direct methods Also known as methods of feasible directions Idea in a point x h, generate a

More information

Reduced-Hessian Methods for Constrained Optimization

Reduced-Hessian Methods for Constrained Optimization Reduced-Hessian Methods for Constrained Optimization Philip E. Gill University of California, San Diego Joint work with: Michael Ferry & Elizabeth Wong 11th US & Mexico Workshop on Optimization and its

More information

Worst Case Complexity of Direct Search

Worst Case Complexity of Direct Search Worst Case Complexity of Direct Search L. N. Vicente May 3, 200 Abstract In this paper we prove that direct search of directional type shares the worst case complexity bound of steepest descent when sufficient

More information

5 Handling Constraints

5 Handling Constraints 5 Handling Constraints Engineering design optimization problems are very rarely unconstrained. Moreover, the constraints that appear in these problems are typically nonlinear. This motivates our interest

More information

A Hybrid Method for the Wave Equation. beilina

A Hybrid Method for the Wave Equation.   beilina A Hybrid Method for the Wave Equation http://www.math.unibas.ch/ beilina 1 The mathematical model The model problem is the wave equation 2 u t 2 = (a 2 u) + f, x Ω R 3, t > 0, (1) u(x, 0) = 0, x Ω, (2)

More information