Simulation based optimization
|
|
- Janis Sims
- 5 years ago
- Views:
Transcription
1 SimBOpt p.1/52 Simulation based optimization Feb 2005 Eldad Haber Emory University
2 SimBOpt p.2/52 Outline Introduction A few words about discretization The unconstrained framework Calculation of the gradient Getting a decent descent direction Globalization Summary
3 SimBOpt p.3/52 Simulation and Optimization The problem: min J (y, u) subject to c(y, u) = 0 Work within the discretize-optimize framework
4 SimBOpt p.4/52 Discretize-Optimize Optimize-Discretize: Can yield inconsistent gradients of the objective functionals. The approximate gradient obtained in this way is not a true gradient of anything not of the continuous functional nor of the discrete functional. Discretize-Optimize Requires to differentiate computational facilitators such as turbulence models, shock capturing devices or outflow boundary treatment. M. Gunzburger Want to use the wealth of optimization algorithms
5 SimBOpt p.5/52 Simulation and Optimization Need to discretize the PDE (constraint) Parameters change - modeling need to be flexible Need to optimize - derivatives
6 SimBOpt p.6/52 Discretizing c(y, u) = 0 - difficulties Stability with respect to parameters Explicit vs Implicit c(y, u) = y t uy xx Explicit: c h (y h, u h ) = y n+1 h y n h u h δt δx 2Lyn h = 0
7 SimBOpt p.7/52 Discretizing c(y, u) = 0 - difficulties Stability requires u h δt δx 2 But if we do not know u it may be hard to guarantee stability. Code has to make sure discretization is compatible Implicit methods are unconditionally stable
8 SimBOpt p.8/52 Discretizing c(y, u) = 0 - difficulties Differentiability of the discretization c(y, u) = ǫy xx + uy x = 0 Common discretization, upwind ǫ h 2(y j+1 2y j + y j 1 ) + u j h (max(u, 0)(y j y j 1 ) + min(u, 0)(y j+1 y j )) = 0
9 SimBOpt p.9/52 Discretizing c(y, u) = 0 - difficulties The continuous problem is continuously differentiable w.r.t u The discrete problem is not differentiable w.r.t u h Depends on the application - can be hard to deal with
10 SimBOpt p.10/52 The optimization problem - example Example - the mother of all elliptic problems (u y) = q Finite volume discretization A(u h )y h = D diag(n(u h ))D y h = q h Comment - N(u) harmonic averaging
11 SimBOpt p.11/52 The optimization problem Constrained approach, solve min J (y,u) subject to c(y,u) = 0 (e.g. A(u)y q = 0) Unconstrained approach, eliminate y to obtain min J (y(u),u) J (A(u) 1 q,u)
12 SimBOpt p.12/52 Constrained vs Unconstrained Constrained approach, Saddle point problem Algorithmically hard No need to solve the constraints Unconstrained approach Simple from an optimization standpoint Need to solve the constraint equation PDE Becomes even messier for nonlinear PDE s
13 Constrained vs Unconstrained SimBOpt p.13/52
14 SimBOpt p.14/52 Derivatives - Unconstrained approach In the inverse minimize min J (y(u), u) Linearize y(u + s) y(u) + δy {}}{ y u s Need to compute sensitivities C = y u
15 SimBOpt p.15/52 Computing Derivatives Rewrite the constraint 0 = c(y + δy, u + s) = c y δy + c u s = ( C {}}{ y u + c u )s c y Therefore C = y u = c 1 y c u
16 SimBOpt p.16/52 Computing Derivatives - example Then c(y,u) = A(u)y q = D diag(u)d y q c y c u = A(u) = (D diag(u)d y) u = (D diag(dy)u) u = D diag(dy) Then C = A(u) 1 D diag(dy)
17 SimBOpt p.17/52 The sensitivities C = y u = c 1 y c u A(u) 1 D diag(dy) c y is a discretized (linearized) PDE c 1 y is (usually) dense Do not compute C directly Whenever needed Cv use: w = Cv = c 1 y c u v Solve c y w = c u v
18 SimBOpt p.18/52 Computing the gradient The optimization problem min u J (y(u),u) Gradients: use chain rule J u (y(u),u) = y u J y + J u = C J y + J u
19 SimBOpt p.19/52 Computing the gradient Gradients g(u) = J u (y(u),u) = C J y + J u To compute the gradients need to calculate w = C J y = c u c y To compute J y Solve the adjoint problem c y z = J y Set w = c u z
20 Optimization algorithms The optimization problem min u J (y(u),u) Optimization algorithms - Framework Guess u 0 while not converge Evaluate J (u k ), g(u k ) and an approximation to the HessianB(u k ) Compute δu = B(u k ) 1 g(u k ) Take a step u k+1 = u k + αδu 0 < α 1 SimBOpt p.20/52
21 SimBOpt p.21/52 Getting a decent descent direction In a nut-shell, difference between optimization algorithms, how to choose B steepest descent B = I Newton B = J uu Quasi-Newton use [g k j,g k j+1,...,g k 1 ], [s k j,s k j+1,...,s k 1 ] to construct an approximation to the Hessian
22 SimBOpt p.22/52 More about the Newton direction Need to compute the Hessian. H = g(u) u = (C J (y,u) y ) u + J (y,u) uu Evaluating the second term is usually easy.
23 SimBOpt p.23/52 More about the Newton direction Need to compute the Hessian. H = g(u) u = (C J (y,u) y ) u + J (y,u) uu To evaluate (C J (y,u) y ) u use chain rule (J (y,u) y ) u = J (y,u) yy C
24 SimBOpt p.24/52 More about the Newton direction Need to compute the Hessian. H = g(u) u = (C J (y,u) y ) u + J (y,u) uu Gauss-Newton family - Ignore the dependency of C(u). H C J (y,u) yy C + J uu If J yy, J uu are SPD, then H is SPD
25 SimBOpt p.25/52 Computing the GN direction Need to solve (C J (y,u) yy C + J uu )δu = g(u) Problem is large, natural choice - Conjugate Gradient For each CG iteration Multiply C v and Cw Require one forward and one adjoint solve
26 SimBOpt p.26/52 Computing the GN direction Need to solve (C J (y,u) yy C + J uu )δu = g(u) Cost per iteration (#ITER CG + 1) (COST FORWARD + COST ADJOINT ) Typically do not solve the system to high tolerance (inexact Gauss-Newton)
27 SimBOpt p.27/52 Computing the GN direction Need to solve (C J (y,u) yy C + J uu )δu = g(u) Open question Preconditioning? Use Quasi-Newton approximate Hessians as preconditioners [Nocedal, Haber, Bradsly & Vogel Newman & Boggs...] Some problem dependent preconditioners [Mackie, Vogel, Farquharson...] Waiting for the big break
28 SimBOpt p.28/52 More about Quasi-Newton Use previous gradients and descent directions [g k j,g k j+1,...,g k 1 ], [s k j,s k j+1,...,s k 1 ] to construct an approximation to the Hessian Basic idea, Taylor s expansion B k (s k s k 1 ) = g k g k 1 Given s k s k 1 and g k g k 1 update B k
29 SimBOpt p.29/52 More about Quasi-Newton Cheap - Need not solve extra PDE s Very effective for some problems. Most popular - LBFGS, DFP Recent active research on application and improvement [Bradsly & Vogel, Navon, Haber...]
30 SimBOpt p.30/52 Globalization Make sure that J (u k+1 ) < J (u k ) line search: approximately min α J (u k + αs) trust region: approximately min J (u k + w) s.t w [s,g(u)], w homotopy: Solve a sequence of problems g(u,α k ) = 0
31 SimBOpt p.31/52 Globalization Every back tracking iteration requires the solution of a PDE. Important to get the most we can from a step
32 SimBOpt p.32/52 Grid Sequencing The problems we solve have an underline continuous structure. Use this structure for continuation Main idea: Solution of the problem on a coarse grid can approximate the problem on a fine grid. Use coarse grids to evaluate parameters within the optimization. Burger, Ascher & Haber, Haber & Modersitzki, Haber, Moŕe (see talk)
33 SimBOpt p.33/52 Algorithm Solve the optimization problem on a coarse grid H Refine the grid to a fine grid h Interpolate the solution from H to h and use it as initial guess H h In many cases grid continuation is sufficient to have global convergence. No proof that this is always the case
34 SimBOpt p.34/52 Application: Impedance Tomography Joint project with R. Knight and A. Pidlovski, Stanford Environmental Geophysics Group
35 Application: Impedance Tomography SimBOpt p.35/52
36 SimBOpt p.36/52 Application: Impedance Tomography Reference potential electrode V cone-mounted potential electrode permanent current electrodes
37 Application: Impedance Tomography SimBOpt p.37/52
38 SimBOpt p.38/52 The mathematical problem The constraint (PDE) (with some BC) c(y,u) = exp(u) y q j = 0 j = 1...k The Objective function min 1 2 Q(y(u) yobs ) 2 }{{} misfit + α }{{} regpar regularization {}}{ R(u)
39 Discretization SimBOpt p.39/52
40 SimBOpt p.40/52 Discretization use cells # of states = k # of controls In practical experiments k
41 SimBOpt p.41/52 The discrete mathematical problem The constraint (PDE) c h (y h,u h ) = A(u h )y h q h = D T S(u h )Dy h q h = 0 The Objective function min 1 2 Q(A(u h) 1 q h y obs ) 2 }{{} misfit + α }{{} regpar regularization {}}{ R(u h )
42 The Data - 63 sources SimBOpt p.42/52
43 The Inversion SimBOpt p.43/52
44 SimBOpt p.44/52 Computational Cost α misfit Total iterations forward sol s IGN QN PIGN IGN QN PIGN IGN - inexact Gauss-Newton QN - Quasi Newton PIGN - QN preconditioner to IGN
45 SimBOpt p.45/52 Application - Image Registration Joint work with J. Modesitzki, Lubic, Germany Given Template image Reference image T(x) = T(x 1,x 2,x 3 ) R(x) = R(x 1,x 2,x 3 ) find a transformation u(x) = [u(x),v(x),w(x)] such that T(x + u(x)) R(x)
46 SimBOpt p.46/52 Example I R T T 0 R = =100.00% Start animation - HeadSpin
47 SimBOpt p.47/52 Example - ML R T T 0 R = =100.00%
48 SimBOpt p.48/52 Example - ML R T T 6 T 0 R =1.666=100.00% T 1 R = = 17.68%
49 SimBOpt p.49/52 Example - ML R T T 3 T 0 R =1.1637=100.00% T 1 R = = 14.62%
50 SimBOpt p.50/52 Example - ML R T T 3 T 0 R = =100.00% T 1 R = = 16.72%
51 SimBOpt p.51/52 Example - ML R T T 4 T 0 R = =100.00% T 1 R = = 23.61%
52 SimBOpt p.52/52 Summary Introduction discretization of PDE s The unconstrained framework Calculation of the gradient Getting a decent descent direction Globalization Summary
Parallelizing large scale time domain electromagnetic inverse problem
Parallelizing large scale time domain electromagnetic inverse problems Eldad Haber with: D. Oldenburg & R. Shekhtman + Emory University, Atlanta, GA + The University of British Columbia, Vancouver, BC,
More informationA multigrid method for large scale inverse problems
A multigrid method for large scale inverse problems Eldad Haber Dept. of Computer Science, Dept. of Earth and Ocean Science University of British Columbia haber@cs.ubc.ca July 4, 2003 E.Haber: Multigrid
More informationNonlinear Optimization: What s important?
Nonlinear Optimization: What s important? Julian Hall 10th May 2012 Convexity: convex problems A local minimizer is a global minimizer A solution of f (x) = 0 (stationary point) is a minimizer A global
More informationA multilevel, level-set method for optimizing eigenvalues in shape design problems
A multilevel, level-set method for optimizing eigenvalues in shape design problems E. Haber July 22, 2003 Abstract In this paper we consider optimal design problems that involve shape optimization. The
More informationPDE Solvers for Fluid Flow
PDE Solvers for Fluid Flow issues and algorithms for the Streaming Supercomputer Eran Guendelman February 5, 2002 Topics Equations for incompressible fluid flow 3 model PDEs: Hyperbolic, Elliptic, Parabolic
More informationAchieving depth resolution with gradient array survey data through transient electromagnetic inversion
Achieving depth resolution with gradient array survey data through transient electromagnetic inversion Downloaded /1/17 to 128.189.118.. Redistribution subject to SEG license or copyright; see Terms of
More informationSimultaneous estimation of wavefields & medium parameters
Simultaneous estimation of wavefields & medium parameters reduced-space versus full-space waveform inversion Bas Peters, Felix J. Herrmann Workshop W- 2, The limit of FWI in subsurface parameter recovery.
More informationNonlinearOptimization
1/35 NonlinearOptimization Pavel Kordík Department of Computer Systems Faculty of Information Technology Czech Technical University in Prague Jiří Kašpar, Pavel Tvrdík, 2011 Unconstrained nonlinear optimization,
More informationA parallel method for large scale time domain electromagnetic inverse problems
A parallel method for large scale time domain electromagnetic inverse problems Eldad Haber July 15, 2005 Abstract In this work we consider the solution of 3D time domain electromagnetic inverse problems
More information2.29 Numerical Fluid Mechanics Spring 2015 Lecture 9
Spring 2015 Lecture 9 REVIEW Lecture 8: Direct Methods for solving (linear) algebraic equations Gauss Elimination LU decomposition/factorization Error Analysis for Linear Systems and Condition Numbers
More informationComputational methods for large distributed parameter estimation problems with possible discontinuities
Computational methods for large distributed parameter estimation problems with possible discontinuities Uri Ascher Department of Computer Science, University of British Columbia, Vancouver, BC, V6T 1Z4,
More informationNumerical Optimization Algorithms
Numerical Optimization Algorithms 1. Overview. Calculus of Variations 3. Linearized Supersonic Flow 4. Steepest Descent 5. Smoothed Steepest Descent Overview 1 Two Main Categories of Optimization Algorithms
More informationAM 205: lecture 19. Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods Optimality Conditions: Equality Constrained Case As another example of equality
More informationOPER 627: Nonlinear Optimization Lecture 14: Mid-term Review
OPER 627: Nonlinear Optimization Lecture 14: Mid-term Review Department of Statistical Sciences and Operations Research Virginia Commonwealth University Oct 16, 2013 (Lecture 14) Nonlinear Optimization
More informationPartial Differential Equations
Partial Differential Equations Introduction Deng Li Discretization Methods Chunfang Chen, Danny Thorne, Adam Zornes CS521 Feb.,7, 2006 What do You Stand For? A PDE is a Partial Differential Equation This
More informationDirect Current Resistivity Inversion using Various Objective Functions
Direct Current Resistivity Inversion using Various Objective Functions Rowan Cockett Department of Earth and Ocean Science University of British Columbia rcockett@eos.ubc.ca Abstract In geophysical applications
More informationNumerical optimization
Numerical optimization Lecture 4 Alexander & Michael Bronstein tosca.cs.technion.ac.il/book Numerical geometry of non-rigid shapes Stanford University, Winter 2009 2 Longest Slowest Shortest Minimal Maximal
More informationReview for Exam 2 Ben Wang and Mark Styczynski
Review for Exam Ben Wang and Mark Styczynski This is a rough approximation of what we went over in the review session. This is actually more detailed in portions than what we went over. Also, please note
More informationA Sobolev trust-region method for numerical solution of the Ginz
A Sobolev trust-region method for numerical solution of the Ginzburg-Landau equations Robert J. Renka Parimah Kazemi Department of Computer Science & Engineering University of North Texas June 6, 2012
More informationBasic Aspects of Discretization
Basic Aspects of Discretization Solution Methods Singularity Methods Panel method and VLM Simple, very powerful, can be used on PC Nonlinear flow effects were excluded Direct numerical Methods (Field Methods)
More informationOptimization Methods
Optimization Methods Decision making Examples: determining which ingredients and in what quantities to add to a mixture being made so that it will meet specifications on its composition allocating available
More informationTrajectory-based optimization
Trajectory-based optimization Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Winter 2012 Emo Todorov (UW) AMATH/CSE 579, Winter 2012 Lecture 6 1 / 13 Using
More informationLectures Notes Algorithms and Preconditioning in PDE-Constrained Optimization. Prof. Dr. R. Herzog
Lectures Notes Algorithms and Preconditioning in PDE-Constrained Optimization Prof. Dr. R. Herzog held in July 2010 at the Summer School on Analysis and Numerics of PDE Constrained Optimization, Lambrecht
More informationNumerical Optimization Professor Horst Cerjak, Horst Bischof, Thomas Pock Mat Vis-Gra SS09
Numerical Optimization 1 Working Horse in Computer Vision Variational Methods Shape Analysis Machine Learning Markov Random Fields Geometry Common denominator: optimization problems 2 Overview of Methods
More informationOptimal control problems with PDE constraints
Optimal control problems with PDE constraints Maya Neytcheva CIM, October 2017 General framework Unconstrained optimization problems min f (q) q x R n (real vector) and f : R n R is a smooth function.
More informationComputational methods for large distributed parameter estimation problems in 3D
Computational methods for large distributed parameter estimation problems in 3D U. M. Ascher E. Haber March 8, 03 Abstract This paper considers problems of distributed parameter estimation from data measurements
More informationAM 205: lecture 19. Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods Quasi-Newton Methods General form of quasi-newton methods: x k+1 = x k α
More informationConstrained optimization. Unconstrained optimization. One-dimensional. Multi-dimensional. Newton with equality constraints. Active-set method.
Optimization Unconstrained optimization One-dimensional Multi-dimensional Newton s method Basic Newton Gauss- Newton Quasi- Newton Descent methods Gradient descent Conjugate gradient Constrained optimization
More informationNumerical optimization. Numerical optimization. Longest Shortest where Maximal Minimal. Fastest. Largest. Optimization problems
1 Numerical optimization Alexander & Michael Bronstein, 2006-2009 Michael Bronstein, 2010 tosca.cs.technion.ac.il/book Numerical optimization 048921 Advanced topics in vision Processing and Analysis of
More informationHigh Performance Nonlinear Solvers
What is a nonlinear system? High Performance Nonlinear Solvers Michael McCourt Division Argonne National Laboratory IIT Meshfree Seminar September 19, 2011 Every nonlinear system of equations can be described
More informationParameter Identification in Partial Differential Equations
Parameter Identification in Partial Differential Equations Differentiation of data Not strictly a parameter identification problem, but good motivation. Appears often as a subproblem. Given noisy observation
More informationComparison between least-squares reverse time migration and full-waveform inversion
Comparison between least-squares reverse time migration and full-waveform inversion Lei Yang, Daniel O. Trad and Wenyong Pan Summary The inverse problem in exploration geophysics usually consists of two
More information1. Introduction. In this work we consider the solution of finite-dimensional constrained optimization problems of the form
MULTILEVEL ALGORITHMS FOR LARGE-SCALE INTERIOR POINT METHODS MICHELE BENZI, ELDAD HABER, AND LAUREN TARALLI Abstract. We develop and compare multilevel algorithms for solving constrained nonlinear variational
More informationKasetsart University Workshop. Multigrid methods: An introduction
Kasetsart University Workshop Multigrid methods: An introduction Dr. Anand Pardhanani Mathematics Department Earlham College Richmond, Indiana USA pardhan@earlham.edu A copy of these slides is available
More informationQuasi-Newton Methods. Zico Kolter (notes by Ryan Tibshirani, Javier Peña, Zico Kolter) Convex Optimization
Quasi-Newton Methods Zico Kolter (notes by Ryan Tibshirani, Javier Peña, Zico Kolter) Convex Optimization 10-725 Last time: primal-dual interior-point methods Given the problem min x f(x) subject to h(x)
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 24: Preconditioning and Multigrid Solver Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 5 Preconditioning Motivation:
More information1. Introduction. In this work, we consider the solution of finite-dimensional constrained optimization problems of the form
SIAM J. SCI. COMPUT. Vol. 31, No. 6, pp. 4152 4175 c 2009 Society for Industrial and Applied Mathematics MULTILEVEL ALGORITHMS FOR LARGE-SCALE INTERIOR POINT METHODS MICHELE BENZI, ELDAD HABER, AND LAUREN
More informationChapter 3 Numerical Methods
Chapter 3 Numerical Methods Part 2 3.2 Systems of Equations 3.3 Nonlinear and Constrained Optimization 1 Outline 3.2 Systems of Equations 3.3 Nonlinear and Constrained Optimization Summary 2 Outline 3.2
More informationTwo-Scale Wave Equation Modeling for Seismic Inversion
Two-Scale Wave Equation Modeling for Seismic Inversion Susan E. Minkoff Department of Mathematics and Statistics University of Maryland Baltimore County Baltimore, MD 21250, USA RICAM Workshop 3: Wave
More informationOptimization. Escuela de Ingeniería Informática de Oviedo. (Dpto. de Matemáticas-UniOvi) Numerical Computation Optimization 1 / 30
Optimization Escuela de Ingeniería Informática de Oviedo (Dpto. de Matemáticas-UniOvi) Numerical Computation Optimization 1 / 30 Unconstrained optimization Outline 1 Unconstrained optimization 2 Constrained
More informationHigher-Order Methods
Higher-Order Methods Stephen J. Wright 1 2 Computer Sciences Department, University of Wisconsin-Madison. PCMI, July 2016 Stephen Wright (UW-Madison) Higher-Order Methods PCMI, July 2016 1 / 25 Smooth
More informationInexact Newton Methods and Nonlinear Constrained Optimization
Inexact Newton Methods and Nonlinear Constrained Optimization Frank E. Curtis EPSRC Symposium Capstone Conference Warwick Mathematics Institute July 2, 2009 Outline PDE-Constrained Optimization Newton
More informationAn introduction to PDE-constrained optimization
An introduction to PDE-constrained optimization Wolfgang Bangerth Department of Mathematics Texas A&M University 1 Overview Why partial differential equations? Why optimization? Examples of PDE optimization
More informationComplexity analysis of second-order algorithms based on line search for smooth nonconvex optimization
Complexity analysis of second-order algorithms based on line search for smooth nonconvex optimization Clément Royer - University of Wisconsin-Madison Joint work with Stephen J. Wright MOPTA, Bethlehem,
More informationLINEAR AND NONLINEAR PROGRAMMING
LINEAR AND NONLINEAR PROGRAMMING Stephen G. Nash and Ariela Sofer George Mason University The McGraw-Hill Companies, Inc. New York St. Louis San Francisco Auckland Bogota Caracas Lisbon London Madrid Mexico
More informationFinite Elements for Magnetohydrodynamics and its Optimal Control
Finite Elements for Magnetohydrodynamics and its Karl Kunisch Marco Discacciati (RICAM) FEM Symposium Chemnitz September 25 27, 2006 Overview 1 2 3 What is Magnetohydrodynamics? Magnetohydrodynamics (MHD)
More informationLecture 7 Unconstrained nonlinear programming
Lecture 7 Unconstrained nonlinear programming Weinan E 1,2 and Tiejun Li 2 1 Department of Mathematics, Princeton University, weinan@princeton.edu 2 School of Mathematical Sciences, Peking University,
More informationAnna Avdeeva Dmitry Avdeev and Marion Jegen. 31 March Introduction to 3D MT inversion code x3di
Anna Avdeeva (aavdeeva@ifm-geomar.de), Dmitry Avdeev and Marion Jegen 31 March 211 Outline 1 Essential Parts of 3D MT Inversion Code 2 Salt Dome Overhang Study with 3 Outline 1 Essential Parts of 3D MT
More informationLine Search Methods for Unconstrained Optimisation
Line Search Methods for Unconstrained Optimisation Lecture 8, Numerical Linear Algebra and Optimisation Oxford University Computing Laboratory, MT 2007 Dr Raphael Hauser (hauser@comlab.ox.ac.uk) The Generic
More informationA Line search Multigrid Method for Large-Scale Nonlinear Optimization
A Line search Multigrid Method for Large-Scale Nonlinear Optimization Zaiwen Wen Donald Goldfarb Department of Industrial Engineering and Operations Research Columbia University 2008 Siam Conference on
More informationMatrix Derivatives and Descent Optimization Methods
Matrix Derivatives and Descent Optimization Methods 1 Qiang Ning Department of Electrical and Computer Engineering Beckman Institute for Advanced Science and Techonology University of Illinois at Urbana-Champaign
More informationNonlinear Multigrid and Domain Decomposition Methods
Nonlinear Multigrid and Domain Decomposition Methods Rolf Krause Institute of Computational Science Universit`a della Svizzera italiana 1 Stepping Stone Symposium, Geneva, 2017 Non-linear problems Non-linear
More informationIndex. higher order methods, 52 nonlinear, 36 with variable coefficients, 34 Burgers equation, 234 BVP, see boundary value problems
Index A-conjugate directions, 83 A-stability, 171 A( )-stability, 171 absolute error, 243 absolute stability, 149 for systems of equations, 154 absorbing boundary conditions, 228 Adams Bashforth methods,
More informationA Multiphysics Framework Using hp-finite Elements for Electromagnetics Applications
Univ. Carlos III of Madrid A Multiphysics Framework Using hp-finite Elements for Electromagnetics Applications D. Pardo, L. E. García-Castillo, M. J. Nam, C. Torres-Verdín Team: D. Pardo, M. J. Nam, V.
More informationEAD 115. Numerical Solution of Engineering and Scientific Problems. David M. Rocke Department of Applied Science
EAD 115 Numerical Solution of Engineering and Scientific Problems David M. Rocke Department of Applied Science Taylor s Theorem Can often approximate a function by a polynomial The error in the approximation
More informationNumerical Optimization of Partial Differential Equations
Numerical Optimization of Partial Differential Equations Part I: basic optimization concepts in R n Bartosz Protas Department of Mathematics & Statistics McMaster University, Hamilton, Ontario, Canada
More informationBindel, Fall 2011 Intro to Scientific Computing (CS 3220) Week 6: Monday, Mar 7. e k+1 = 1 f (ξ k ) 2 f (x k ) e2 k.
Problem du jour Week 6: Monday, Mar 7 Show that for any initial guess x 0 > 0, Newton iteration on f(x) = x 2 a produces a decreasing sequence x 1 x 2... x n a. What is the rate of convergence if a = 0?
More informationNumerical solutions of nonlinear systems of equations
Numerical solutions of nonlinear systems of equations Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan E-mail: min@math.ntnu.edu.tw August 28, 2011 Outline 1 Fixed points
More informationContents. Preface. 1 Introduction Optimization view on mathematical models NLP models, black-box versus explicit expression 3
Contents Preface ix 1 Introduction 1 1.1 Optimization view on mathematical models 1 1.2 NLP models, black-box versus explicit expression 3 2 Mathematical modeling, cases 7 2.1 Introduction 7 2.2 Enclosing
More informationAn Inexact Newton Method for Optimization
New York University Brown Applied Mathematics Seminar, February 10, 2009 Brief biography New York State College of William and Mary (B.S.) Northwestern University (M.S. & Ph.D.) Courant Institute (Postdoc)
More informationOptimization II: Unconstrained Multivariable
Optimization II: Unconstrained Multivariable CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Justin Solomon CS 205A: Mathematical Methods Optimization II: Unconstrained Multivariable 1
More informationREPORTS IN INFORMATICS
REPORTS IN INFORMATICS ISSN 0333-3590 A class of Methods Combining L-BFGS and Truncated Newton Lennart Frimannslund Trond Steihaug REPORT NO 319 April 2006 Department of Informatics UNIVERSITY OF BERGEN
More informationChapter 5. Methods for Solving Elliptic Equations
Chapter 5. Methods for Solving Elliptic Equations References: Tannehill et al Section 4.3. Fulton et al (1986 MWR). Recommended reading: Chapter 7, Numerical Methods for Engineering Application. J. H.
More informationEAD 115. Numerical Solution of Engineering and Scientific Problems. David M. Rocke Department of Applied Science
EAD 115 Numerical Solution of Engineering and Scientific Problems David M. Rocke Department of Applied Science Multidimensional Unconstrained Optimization Suppose we have a function f() of more than one
More informationStochastic Analogues to Deterministic Optimizers
Stochastic Analogues to Deterministic Optimizers ISMP 2018 Bordeaux, France Vivak Patel Presented by: Mihai Anitescu July 6, 2018 1 Apology I apologize for not being here to give this talk myself. I injured
More informationA projected Hessian for full waveform inversion
CWP-679 A projected Hessian for full waveform inversion Yong Ma & Dave Hale Center for Wave Phenomena, Colorado School of Mines, Golden, CO 80401, USA (c) Figure 1. Update directions for one iteration
More informationSeismic imaging and optimal transport
Seismic imaging and optimal transport Bjorn Engquist In collaboration with Brittany Froese, Sergey Fomel and Yunan Yang Brenier60, Calculus of Variations and Optimal Transportation, Paris, January 10-13,
More informationSpring 2014: Computational and Variational Methods for Inverse Problems CSE 397/GEO 391/ME 397/ORI 397 Assignment 4 (due 14 April 2014)
Spring 2014: Computational and Variational Methods for Inverse Problems CSE 397/GEO 391/ME 397/ORI 397 Assignment 4 (due 14 April 2014) The first problem in this assignment is a paper-and-pencil exercise
More informationReduced-Hessian Methods for Constrained Optimization
Reduced-Hessian Methods for Constrained Optimization Philip E. Gill University of California, San Diego Joint work with: Michael Ferry & Elizabeth Wong 11th US & Mexico Workshop on Optimization and its
More informationFast Iterative Solution of Saddle Point Problems
Michele Benzi Department of Mathematics and Computer Science Emory University Atlanta, GA Acknowledgments NSF (Computational Mathematics) Maxim Olshanskii (Mech-Math, Moscow State U.) Zhen Wang (PhD student,
More informationAffine covariant Semi-smooth Newton in function space
Affine covariant Semi-smooth Newton in function space Anton Schiela March 14, 2018 These are lecture notes of my talks given for the Winter School Modern Methods in Nonsmooth Optimization that was held
More informationNewton-Multigrid Least-Squares FEM for S-V-P Formulation of the Navier-Stokes Equations
Newton-Multigrid Least-Squares FEM for S-V-P Formulation of the Navier-Stokes Equations A. Ouazzi, M. Nickaeen, S. Turek, and M. Waseem Institut für Angewandte Mathematik, LSIII, TU Dortmund, Vogelpothsweg
More informationLinear and Non-Linear Preconditioning
and Non- martin.gander@unige.ch University of Geneva July 2015 Joint work with Victorita Dolean, Walid Kheriji, Felix Kwok and Roland Masson (1845): First Idea of After preconditioning, it takes only three
More informationAlgorithms for Constrained Optimization
1 / 42 Algorithms for Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University April 19, 2015 2 / 42 Outline 1. Convergence 2. Sequential quadratic
More informationA Multilevel Method for Image Registration
A Multilevel Method for Image Registration Eldad Haber Jan Modersitzki May 1, 004 Abstract In this paper we introduce a new framework for image registration. Our formulation is based on consistent discretization
More informationNonlinear Optimization for Optimal Control
Nonlinear Optimization for Optimal Control Pieter Abbeel UC Berkeley EECS Many slides and figures adapted from Stephen Boyd [optional] Boyd and Vandenberghe, Convex Optimization, Chapters 9 11 [optional]
More informationETNA Kent State University
Electronic Transactions on Numerical Analysis. Volume 5, pp. -7, 23. Copyright 22,. ISSN 68-963. ETNA A MULTIGRID METHOD FOR DISTRIBUTED PARAMETER ESTIMATION PROBLEMS U. M. ASCHER AND E. HABER Abstract.
More information1 Newton s Method. Suppose we want to solve: x R. At x = x, f (x) can be approximated by:
Newton s Method Suppose we want to solve: (P:) min f (x) At x = x, f (x) can be approximated by: n x R. f (x) h(x) := f ( x)+ f ( x) T (x x)+ (x x) t H ( x)(x x), 2 which is the quadratic Taylor expansion
More informationIn the derivation of Optimal Interpolation, we found the optimal weight matrix W that minimizes the total analysis error variance.
hree-dimensional variational assimilation (3D-Var) In the derivation of Optimal Interpolation, we found the optimal weight matrix W that minimizes the total analysis error variance. Lorenc (1986) showed
More informationNumerical Optimization Techniques
Numerical Optimization Techniques Léon Bottou NEC Labs America COS 424 3/2/2010 Today s Agenda Goals Representation Capacity Control Operational Considerations Computational Considerations Classification,
More informationOptimization and Root Finding. Kurt Hornik
Optimization and Root Finding Kurt Hornik Basics Root finding and unconstrained smooth optimization are closely related: Solving ƒ () = 0 can be accomplished via minimizing ƒ () 2 Slide 2 Basics Root finding
More informationAn Inexact Newton Method for Nonlinear Constrained Optimization
An Inexact Newton Method for Nonlinear Constrained Optimization Frank E. Curtis Numerical Analysis Seminar, January 23, 2009 Outline Motivation and background Algorithm development and theoretical results
More informationMathematical optimization
Optimization Mathematical optimization Determine the best solutions to certain mathematically defined problems that are under constrained determine optimality criteria determine the convergence of the
More informationMATHEMATICS FOR COMPUTER VISION WEEK 8 OPTIMISATION PART 2. Dr Fabio Cuzzolin MSc in Computer Vision Oxford Brookes University Year
MATHEMATICS FOR COMPUTER VISION WEEK 8 OPTIMISATION PART 2 1 Dr Fabio Cuzzolin MSc in Computer Vision Oxford Brookes University Year 2013-14 OUTLINE OF WEEK 8 topics: quadratic optimisation, least squares,
More informationKey words. preconditioned conjugate gradient method, saddle point problems, optimal control of PDEs, control and state constraints, multigrid method
PRECONDITIONED CONJUGATE GRADIENT METHOD FOR OPTIMAL CONTROL PROBLEMS WITH CONTROL AND STATE CONSTRAINTS ROLAND HERZOG AND EKKEHARD SACHS Abstract. Optimality systems and their linearizations arising in
More informationA Robust Preconditioner for the Hessian System in Elliptic Optimal Control Problems
A Robust Preconditioner for the Hessian System in Elliptic Optimal Control Problems Etereldes Gonçalves 1, Tarek P. Mathew 1, Markus Sarkis 1,2, and Christian E. Schaerer 1 1 Instituto de Matemática Pura
More informationInversion of 3D electromagnetic data in frequency and time domain using an inexact all-at-once approach
GEOPHYSICS, VOL. 69, NO. 5 (SEPTEMBER-OCTOBER 2004); P. 1216 1228, 9 FIGS., 2 TABLES. 10.1190/1.1801938 Inversion of 3D electromagnetic data in frequency and time domain using an inexact all-at-once approach
More informationThe Inversion Problem: solving parameters inversion and assimilation problems
The Inversion Problem: solving parameters inversion and assimilation problems UE Numerical Methods Workshop Romain Brossier romain.brossier@univ-grenoble-alpes.fr ISTerre, Univ. Grenoble Alpes Master 08/09/2016
More informationConstrained Optimization
1 / 22 Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University March 30, 2015 2 / 22 1. Equality constraints only 1.1 Reduced gradient 1.2 Lagrange
More informationUnconstrained optimization
Chapter 4 Unconstrained optimization An unconstrained optimization problem takes the form min x Rnf(x) (4.1) for a target functional (also called objective function) f : R n R. In this chapter and throughout
More informationVasil Khalidov & Miles Hansard. C.M. Bishop s PRML: Chapter 5; Neural Networks
C.M. Bishop s PRML: Chapter 5; Neural Networks Introduction The aim is, as before, to find useful decompositions of the target variable; t(x) = y(x, w) + ɛ(x) (3.7) t(x n ) and x n are the observations,
More informationInversion of 3D Electromagnetic Data in frequency and time domain using an inexact all-at-once approach
Inversion of 3D Electromagnetic Data in frequency and time domain using an inexact all-at-once approach Eldad Haber UBC-Geophysical Inversion Facility Department of Earth and Ocean Sciences University
More informationNumerical Optimization
Numerical Optimization Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Spring 2010 Emo Todorov (UW) AMATH/CSE 579, Spring 2010 Lecture 9 1 / 8 Gradient descent
More informationChapter 7 Iterative Techniques in Matrix Algebra
Chapter 7 Iterative Techniques in Matrix Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128B Numerical Analysis Vector Norms Definition
More informationBang bang control of elliptic and parabolic PDEs
1/26 Bang bang control of elliptic and parabolic PDEs Michael Hinze (joint work with Nikolaus von Daniels & Klaus Deckelnick) Jackson, July 24, 2018 2/26 Model problem (P) min J(y, u) = 1 u U ad,y Y ad
More informationLecture # 20 The Preconditioned Conjugate Gradient Method
Lecture # 20 The Preconditioned Conjugate Gradient Method We wish to solve Ax = b (1) A R n n is symmetric and positive definite (SPD). We then of n are being VERY LARGE, say, n = 10 6 or n = 10 7. Usually,
More informationPose Tracking II! Gordon Wetzstein! Stanford University! EE 267 Virtual Reality! Lecture 12! stanford.edu/class/ee267/!
Pose Tracking II! Gordon Wetzstein! Stanford University! EE 267 Virtual Reality! Lecture 12! stanford.edu/class/ee267/!! WARNING! this class will be dense! will learn how to use nonlinear optimization
More informationProgramming, numerics and optimization
Programming, numerics and optimization Lecture C-3: Unconstrained optimization II Łukasz Jankowski ljank@ippt.pan.pl Institute of Fundamental Technological Research Room 4.32, Phone +22.8261281 ext. 428
More informationLecture Notes: Geometric Considerations in Unconstrained Optimization
Lecture Notes: Geometric Considerations in Unconstrained Optimization James T. Allison February 15, 2006 The primary objectives of this lecture on unconstrained optimization are to: Establish connections
More informationINTRODUCTION TO FINITE ELEMENT METHODS ON ELLIPTIC EQUATIONS LONG CHEN
INTROUCTION TO FINITE ELEMENT METHOS ON ELLIPTIC EQUATIONS LONG CHEN CONTENTS 1. Poisson Equation 1 2. Outline of Topics 3 2.1. Finite ifference Method 3 2.2. Finite Element Method 3 2.3. Finite Volume
More information