STCE. Adjoint Code Design Patterns. Uwe Naumann. RWTH Aachen University, Germany. QuanTech Conference, London, April 2016
|
|
- Alicia Hubbard
- 6 years ago
- Views:
Transcription
1 Adjoint Code Design Patterns Uwe Naumann RWTH Aachen University, Germany QuanTech Conference, London, April 2016
2 Outline Why Adjoints? What Are Adjoints? Software Tool Support: dco/c++ Adjoint Code Design Patterns Naumann, April 2016, Adjoint Code Design Patterns 2
3 Why Adjoints? Setting the Stage In U.N. and J. du Toit: Adjoint Algorithmic Differentiation Tool Support for Typical Numerical Patterns in Computational Finance. Under Review. Also appeared as NAG Technical Report TR 3/14, The Numerical Algorithms Group. we price a simple European Call option written on an underlying S = S(t) : IR + IR +, described by the SDE ds(t) = S(t) r dt + S(t) σ(t, S(t)) dw (t) with time t 0, maturity T > 0, strike K > 0, constant interest rate r > 0, volatility σ = σ(t, S) : IR + IR + IR +, and Brownian motion W = W (t) : IR + IR. We use this simple case study for illustration of common patterns found in financial simulations in the context of Adjoint Algorithmic Differention (AAD). Naumann, April 2016, Adjoint Code Design Patterns 3
4 Why Adjoints? Setting the Stage The value of a European call option driven by S is given by the expectation V = Ee r T (S(T ) K) + for given interest r, strike K, and maturity T. It can be evaluated using Monte Carlo simulation or restated as a PDE and solved by a finite difference method. We compare the computation of the gradient of the option price V IR + wrt. the problem parameters (K, S(0), r, T, and the variable number of parameters of the local volatility surface) by finite differences with an adjoint code based on dco/c++... Naumann, April 2016, Adjoint Code Design Patterns 4
5 Why Adjoints? Setting the Stage Monte Carlo method using 10 4 paths performing 360 time steps each; gradient of size n = 222; running on standard PC with 3GB of RAM available primal central FD naive AAD robust AAD run time (sec.) observing gradient with machine accuracy by AAD prohibitive memory requirement > 40GB for naive AAD (quasi-)constant factor (here approx. 10) wrt. primal run time for AAD within given memory bounds aiming for robust and efficient gradient (and Hessian) code download adjoint option pricers from including first- and second-order adjoint ensembles and evolutions Naumann, April 2016, Adjoint Code Design Patterns 5
6 What Are Adjoints? Let y = F (x), F : IR n IR m, be continuously differentiable. Then (approximate) tangents (directional derivatives) IR m l Ẏ = F (x) Ẋ IR m n IR n l yield (approximate) Jacobian at O(n) Cost(F );... adjoints IR n l X = F (x) T IR n m Ȳ IR m l yield Jacobian at O(m) Cost(F ) (cheap gradients). Challenge: Reversal of data flow yields potentially prohibitive memory requirement (O(Cost(F )))! Naumann, April 2016, Adjoint Code Design Patterns 6
7 Tangents by Overloading 5: y 0( ) 6: y 1(/) v4 x0 1/x1 4: exp v4 3: sin v4/x 2 1 cos(v2) 2: x1 x0 0: x 0 1: x 1 t := e sin(x0 x1) y 0 := x 0 t y 1 := t x 1 v 2 = x 0 x 1 v 2 = ẋ 0 x 1 + x 0 ẋ 1 v 3 = sin(v 2 ) v 3 = cos(v 2 ) v 2 v 4 = exp(v 3 ) v 4 = v 4 v 3 y 0 = x 0 v 4 ẏ 0 = ẋ 0 v 4 + x 0 v 4 y 1 = v 4 /x 1 ẏ 1 = v 4 /x 1 v 4 ẋ 1 /x 2 1 5: ẏ 0 6: ẏ 1 v4 x0 1/x1 4: v 4 v4 3: v 3 v4/x 2 1 cos(v2) 2: v 2 x1 x0 0: ẋ 0 1: ẋ 1 Naumann, April 2016, Adjoint Code Design Patterns 7
8 Adjoints by Overloading 5: y 0( ) 6: y 1(/) v4 x0 1/x1 4: exp v4 3: sin v4/x 2 1 cos(v2) 2: x1 x0 0: x 0 1: x 1 t := e sin(x0 x1) y 0 := x 0 t y 1 := t x 1 // primal... v 2 = v 3 = v 4 := 0 x 1 = v 4 /x 2 1 ȳ 1 v 4 + = 1/x 1 ȳ 1 v 4 + = x 0 ȳ 0 x 0 + = v 4 ȳ 0 v 3 + = v 4 v 4 v 2 + = cos(v 2 ) v 3 x 1 + = x 0 v 2 x 0 + = x 1 v 2 5: ȳ 0 6: ȳ 1 v4 x0 1/x1 4: v 4 v4 3: v 3 v4/x 2 1 cos(v2) 2: v 2 x1 x0 0: x 0 1: x 1 Naumann, April 2016, Adjoint Code Design Patterns 8
9 Software Tool Support: dco/c++ dco/c++ (derivative code by overloading in C++) features tangents and adjoints of arbitrary order through recursive template instantiation for numerical simulation code implemented in C++ front-ends for Fortran and C# (beta) optimized assignment-level gradient code through expression templates cache-optimized internal representation in various incarnations vector modes / detection and exploitation of sparsity external adjoint / Jacobian interfaces user-defined intrinsics intrinsic NAG Library functions (e.g. Linear Algebra, Interpolation, Root Finding, Nearest Correlation Matrix) support for parallelism: thread-safe data structures, adjoint MPI library, GPU coupling Naumann, April 2016, Adjoint Code Design Patterns 9
10 Software Tool Support: dco/c++ Moreover, we have implemented an industrial-strength software engineering infrastructure 1 including professional software development and quality assurance (NAG ) multi-platform (Windows, Linux, Mac OS X) / compiler (vcc, gcc, clang, icc) support overnight building / testing version control user documentation / examples numerous successful applications, including QuantLib, OpenFOAM, PETSc, ICON, Telemac, Jurassic2, ACE+, and several confidential projects with tier-1 investment banks 1 building on academic prototype Naumann, April 2016, Adjoint Code Design Patterns 10
11 Adjoint Code Design An algorithmic adjoint of F : IR n IR m : y z q = F (x) computes X = F T Ȳ = IR n l IR m l Z {}} k 1 { F 1T... F kt (z k 1 ) ( F k+1t... ( F qt Ȳ )...) }{{} Z k for x z 0 and based on adjoint code modules (ACModules) Z i 1 = F it Zi for i = q,..., 1. The art of differentiating computer programs / algorithmic differentiation (AD) is about provision of a correct, robust, efficient, scalable, flexible ACModule hierarchy. z 4 z 5 z 3 (F 3 F 2 ) z 1 z 0 0 z 0 1 Naumann, April 2016, Adjoint Code Design Patterns 11
12 Adjoint Code Design with dco/c++ Mind the Gap Naumann, April 2016, Adjoint Code Design Patterns 12
13 Adjoint Code Design with dco/c++ Case Study We consider a Euler-Maruyama scheme performing µ explicit Euler integration steps along ν independent Monte Carlo sample paths. A basic implementation of this method evaluates a function of the following structure: y = F (x) = G Pµ 1(P 0 µ 2(. 0.. P0 0 (S 0 (x))...))., ν 1 ν 1 (Pµ 2 (... P0 (S ν 1 (x))...)) P ν 1 µ 1 where the arguments x (strike price, time to maturity, interest rate(s), price of underlying and market observed volatilities) are scattered over the individual paths by the S i, i = 0,..., ν 1, each path performs integration steps P i j, j = 0,..., µ 1, and the results are gathered by G to yield y (option price). Naumann, April 2016, Adjoint Code Design Patterns 13
14 Adjoint Code Design Patterns with dco/c++ Case Study: ACModule Class Diagram Naumann, April 2016, Adjoint Code Design Patterns 14
15 Adjoint Code Design Patterns with dco/c++ Case Study: ACModule Object Diagram See for doxygen. Naumann, April 2016, Adjoint Code Design Patterns 15
16 Adjoint Code Design Patterns with dco/c++ Further Scenarios Symbolic Adjoints of Implicit Functions Adjoint Linear Systems Adjoint Nonlinear Systems Adjoint Nonlinear (Locally) Convex Objective Functions Local Finite Differences Black-Box Primal Active Control Flow Dependence Nonsmooth Time Stepping Preaccumulation of Local Jacobians Detection and Exploitation of Sparsity Elimination Techniques on Linearized DAGs Reverse Accumulation for Adjoint Fix-Point Iteration Dynamic Call Tree Reversal Verified Enclosures through Interval Arithmetic / Convex Envelopes... Naumann, April 2016, Adjoint Code Design Patterns 16
17 Adjoint Code Design Patterns with dco/c++ Selected Scenarios (Toy) x 2 p = 0 x d dp differentiate ( x 2 x dx ) ( dp 1 = 2 x x dx ) x = 0 dp = p (x, G) = S(x 0, p) record perturb ẋ = S(x 0, p + h) p = S( p 0, x, x) x p interpret p = S(G, x) p = x 2 p = x 2 x p p x ẋ x h U.N. at al.: Algorithmic Differentiation of Numerical Methods: Tangent and Adjoint Solvers for Parameterized Systems of Nonlinear Equations, ACM TOMS, Naumann, April 2016, Adjoint Code Design Patterns 17
18 Adjoint Code Design Patterns with dco/c++ Local Finite Differences (Notation: v v (1) ) Naumann, April 2016, Adjoint Code Design Patterns 18
19 Adjoint Code Design Patterns with dco/c++ A(p) x(p) = b(p) A T b = x; Ā = b x T Naumann, April 2016, Adjoint Code Design Patterns 19
20 Adjoint Code Design Patterns with dco/c++ F (x(p), p) = 0 xf T z = x; p = pf T z Naumann, April 2016, Adjoint Code Design Patterns 20
21 Adjoint Code Design Patterns with dco/c++ argmin x f(x(p), p) xxf T z = x; p = xpf T z Naumann, April 2016, Adjoint Code Design Patterns 21
22 Conclusion The quality of an adjoint AD solution / tool is defined by robustness wrt. language features of target code efficiency of adjoint propagation flexibility wrt. design scenarios sustainability wrt. dynamics in user requirements, personnel, hard- and software You want (sooner or later) top-quality AAD tools and documentation, e.g, dco/c++ adjoint numerical libraries, e.g, NAG AD Library training, e.g, 1-3 day in-house courses support / consultancy, e.g, responsive and competent help desk Naumann, April 2016, Adjoint Code Design Patterns 22
23 Outlook dco/c++ v4.x tape compression / decomposition / debugging run time code generation file / MPI tapes heapless taping on GPUs fully automatic checkpointing / adjoints exploratory spawning for globalization and nonsmoothness multi-core taping approximate / gut computing Naumann, April 2016, Adjoint Code Design Patterns 23
24 Follow-up Naumann, April 2016, Adjoint Code Design Patterns 24
The Art of Differentiating Computer Programs
The Art of Differentiating Computer Programs Uwe Naumann LuFG Informatik 12 Software and Tools for Computational Engineering RWTH Aachen University, Germany naumann@stce.rwth-aachen.de www.stce.rwth-aachen.de
More informationSTCE. Mind the Gap in the Chain Rule. Uwe Naumann. Software Tool Support for Advanced Algorithmic Differentiation of Numerical Simulation Code
Mind the Gap in the Chain Rule Software Tool Support for Advanced Algorithmic Differentiation of Numerical Simulation Code Uwe Naumann Software and Tools for Computational Engineering, RWTH Aachen University
More informationNested Differentiation and Symmetric Hessian Graphs with ADTAGEO
Nested Differentiation and Symmetric Hessian Graphs with ADTAGEO ADjoints and TAngents by Graph Elimination Ordering Jan Riehme Andreas Griewank Institute for Applied Mathematics Humboldt Universität zu
More informationAlgorithmic Differentiation of Numerical Methods: Tangent and Adjoint Solvers for Parameterized Systems of Nonlinear Equations
Algorithmic Differentiation of Numerical Methods: Tangent and Adjoint Solvers for Parameterized Systems of Nonlinear Equations UWE NAUMANN, JOHANNES LOTZ, KLAUS LEPPKES, and MARKUS TOWARA, RWTH Aachen
More informationSTCE. Mixed Integer Programming for Call Tree Reversal. J. Lotz, U. Naumann, S. Mitra
Mixed Integer Programming or J. Lotz, U. Naumann, S. Mitra LuFG Inormatik 12: Sotware and Tools or Computational Engineering () RWTH Aacen University and Cemical Engineering, Carnegie Mellon University
More informationNumerical Nonlinear Optimization with WORHP
Numerical Nonlinear Optimization with WORHP Christof Büskens Optimierung & Optimale Steuerung London, 8.9.2011 Optimization & Optimal Control Nonlinear Optimization WORHP Concept Ideas Features Results
More informationMulti-Factor Finite Differences
February 17, 2017 Aims and outline Finite differences for more than one direction The θ-method, explicit, implicit, Crank-Nicolson Iterative solution of discretised equations Alternating directions implicit
More informationSensitivity analysis by adjoint Automatic Differentiation and Application
Sensitivity analysis by adjoint Automatic Differentiation and Application Anca Belme Massimiliano Martinelli Laurent Hascoët Valérie Pascual Alain Dervieux INRIA Sophia-Antipolis Project TROPICS Alain.Dervieux@inria.fr
More information1.2 Derivation. d p f = d p f(x(p)) = x fd p x (= f x x p ). (1) Second, g x x p + g p = 0. d p f = f x g 1. The expression f x gx
PDE-constrained optimization and the adjoint method Andrew M. Bradley November 16, 21 PDE-constrained optimization and the adjoint method for solving these and related problems appear in a wide range of
More informationAutomatic Differentiation Algorithms and Data Structures. Chen Fang PhD Candidate Joined: January 2013 FuRSST Inaugural Annual Meeting May 8 th, 2014
Automatic Differentiation Algorithms and Data Structures Chen Fang PhD Candidate Joined: January 2013 FuRSST Inaugural Annual Meeting May 8 th, 2014 1 About 2 3 of the code for physics in simulators computes
More informationFast evaluation of mixed derivatives and calculation of optimal weights for integration. Hernan Leovey
Fast evaluation of mixed derivatives and calculation of optimal weights for integration Humboldt Universität zu Berlin 02.14.2012 MCQMC2012 Tenth International Conference on Monte Carlo and Quasi Monte
More informationMalliavin Calculus in Finance
Malliavin Calculus in Finance Peter K. Friz 1 Greeks and the logarithmic derivative trick Model an underlying assent by a Markov process with values in R m with dynamics described by the SDE dx t = b(x
More informationAn Active Set Strategy for Solving Optimization Problems with up to 200,000,000 Nonlinear Constraints
An Active Set Strategy for Solving Optimization Problems with up to 200,000,000 Nonlinear Constraints Klaus Schittkowski Department of Computer Science, University of Bayreuth 95440 Bayreuth, Germany e-mail:
More informationModule III: Partial differential equations and optimization
Module III: Partial differential equations and optimization Martin Berggren Department of Information Technology Uppsala University Optimization for differential equations Content Martin Berggren (UU)
More informationNumerical Modelling in Fortran: day 10. Paul Tackley, 2016
Numerical Modelling in Fortran: day 10 Paul Tackley, 2016 Today s Goals 1. Useful libraries and other software 2. Implicit time stepping 3. Projects: Agree on topic (by final lecture) (No lecture next
More informationEdwin van der Weide and Magnus Svärd. I. Background information for the SBP-SAT scheme
Edwin van der Weide and Magnus Svärd I. Background information for the SBP-SAT scheme As is well-known, stability of a numerical scheme is a key property for a robust and accurate numerical solution. Proving
More informationVerified High-Order Optimal Control in Space Flight Dynamics
Verified High-Order Optimal Control in Space Flight Dynamics R. Armellin, P. Di Lizia, F. Bernelli-Zazzera K. Makino and M. Berz Fourth International Workshop on Taylor Methods Boca Raton, December 16
More informationWRF performance tuning for the Intel Woodcrest Processor
WRF performance tuning for the Intel Woodcrest Processor A. Semenov, T. Kashevarova, P. Mankevich, D. Shkurko, K. Arturov, N. Panov Intel Corp., pr. ak. Lavrentieva 6/1, Novosibirsk, Russia, 630090 {alexander.l.semenov,tamara.p.kashevarova,pavel.v.mankevich,
More informationScikit-learn. scikit. Machine learning for the small and the many Gaël Varoquaux. machine learning in Python
Scikit-learn Machine learning for the small and the many Gaël Varoquaux scikit machine learning in Python In this meeting, I represent low performance computing Scikit-learn Machine learning for the small
More informationLow-Memory Tour Reversal in Directed Graphs 1. Uwe Naumann, Viktor Mosenkis, Elmar Peise LuFG Informatik 12, RWTH Aachen University, Germany
Low-Memory Tour Reversal in Directed Graphs 1 Uwe Naumann, Viktor Mosenkis, Elmar Peise LuFG Informatik 12, RWTH Aachen University, Germany 1 EuroAD 9, INRIA, Nov. 2009 Contents Motivation: Derivative
More informationAdjoint approach to optimization
Adjoint approach to optimization Praveen. C praveen@math.tifrbng.res.in Tata Institute of Fundamental Research Center for Applicable Mathematics Bangalore 560065 http://math.tifrbng.res.in Health, Safety
More informationIntroduction to numerical simulations for Stochastic ODEs
Introduction to numerical simulations for Stochastic ODEs Xingye Kan Illinois Institute of Technology Department of Applied Mathematics Chicago, IL 60616 August 9, 2010 Outline 1 Preliminaries 2 Numerical
More informationA Symbolic Numeric Environment for Analyzing Measurement Data in Multi-Model Settings (Extended Abstract)
A Symbolic Numeric Environment for Analyzing Measurement Data in Multi-Model Settings (Extended Abstract) Christoph Richard 1 and Andreas Weber 2? 1 Institut für Theoretische Physik, Universität Tübingen,
More informationExploiting Fill-in and Fill-out in Gaussian-like Elimination Procedures on the Extended Jacobian Matrix
2nd European Workshop on AD 1 Exploiting Fill-in and Fill-out in Gaussian-like Elimination Procedures on the Extended Jacobian Matrix Andrew Lyons (Vanderbilt U.) / Uwe Naumann (RWTH Aachen) 2nd European
More informationIMPLEMENTATION OF AN ADJOINT THERMAL SOLVER FOR INVERSE PROBLEMS
Paper ID: ETC217-336 Proceedings of 12th European Conference on Turbomachinery Fluid dynamics & Thermodynamics ETC12, April 3-7, 217; Stockholm, Sweden IMPLEMENTATION OF AN ADJOINT THERMAL SOLVER FOR INVERSE
More informationDiffSharp An AD Library for.net Languages
DiffSharp An AD Library for.net Languages Atılım Güneş Baydin 1 Barak A. Pearlmutter 2 Jeffrey Mark Siskind 3 1 University of Oxford gunes@robots.ox.ac.uk 2 Maynooth University barak@pearlmutter.net 3
More informationNumerical Methods I Solving Nonlinear Equations
Numerical Methods I Solving Nonlinear Equations Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 October 16th, 2014 A. Donev (Courant Institute)
More informationImplicitly and Explicitly Constrained Optimization Problems for Training of Recurrent Neural Networks
Implicitly and Explicitly Constrained Optimization Problems for Training of Recurrent Neural Networks Carl-Johan Thore Linköping University - Division of Mechanics 581 83 Linköping - Sweden Abstract. Training
More informationMath and Numerical Methods Review
Math and Numerical Methods Review Michael Caracotsios, Ph.D. Clinical Associate Professor Chemical Engineering Department University of Illinois at Chicago Introduction In the study of chemical engineering
More informationParallel Polynomial Evaluation
Parallel Polynomial Evaluation Jan Verschelde joint work with Genady Yoffe University of Illinois at Chicago Department of Mathematics, Statistics, and Computer Science http://www.math.uic.edu/ jan jan@math.uic.edu
More informationECS289: Scalable Machine Learning
ECS289: Scalable Machine Learning Cho-Jui Hsieh UC Davis Sept 27, 2015 Outline Linear regression Ridge regression and Lasso Time complexity (closed form solution) Iterative Solvers Regression Input: training
More informationSolving equilibrium problems using extended mathematical programming and SELKIE
Solving equilibrium problems using extended mathematical programming and SELKIE Youngdae Kim and Michael Ferris Wisconsin Institute for Discovery University of Wisconsin-Madison April 30, 2018 Michael
More informationVisual Tracking via Geometric Particle Filtering on the Affine Group with Optimal Importance Functions
Monday, June 22 Visual Tracking via Geometric Particle Filtering on the Affine Group with Optimal Importance Functions Junghyun Kwon 1, Kyoung Mu Lee 1, and Frank C. Park 2 1 Department of EECS, 2 School
More informationDense Arithmetic over Finite Fields with CUMODP
Dense Arithmetic over Finite Fields with CUMODP Sardar Anisul Haque 1 Xin Li 2 Farnam Mansouri 1 Marc Moreno Maza 1 Wei Pan 3 Ning Xie 1 1 University of Western Ontario, Canada 2 Universidad Carlos III,
More informationSensitivity Analysis AA222 - Multidisciplinary Design Optimization Joaquim R. R. A. Martins Durand
Sensitivity Analysis AA222 - Multidisciplinary Design Optimization Joaquim R. R. A. Martins Durand 165 email: joaquim.martins@stanford.edu 1 Introduction Sensitivity analysis consists in computing derivatives
More informationCurrent Developments and Applications related to the Discrete Adjoint Solver in SU2
Current Developments and Applications related to the Discrete Adjoint Solver in SU2 Tim Albring, Max Sagebaum, Ole Burghardt, Lisa Kusch, Nicolas R. Gauger Chair for Scientific Computing TU Kaiserslautern
More informationDesign and implementation of a new meteorology geographic information system
Design and implementation of a new meteorology geographic information system WeiJiang Zheng, Bing. Luo, Zhengguang. Hu, Zhongliang. Lv National Meteorological Center, China Meteorological Administration,
More informationOverview of the PyLith Finite Element Code
Overview of the PyLith Finite Element Code www.geodynamics.org Charles Williams (GNS Science) Brad Aagaard (USGS) Matt Knepley (University of Chicago) CIG Software PyLith is a Tool for Crustal Deformation
More informationStatistical Clustering of Vesicle Patterns Practical Aspects of the Analysis of Large Datasets with R
Statistical Clustering of Vesicle Patterns Mirko Birbaumer Rmetrics Workshop 3th July 2008 1 / 23 Statistical Clustering of Vesicle Patterns Practical Aspects of the Analysis of Large Datasets with R Mirko
More informationPROJECTION METHODS FOR DYNAMIC MODELS
PROJECTION METHODS FOR DYNAMIC MODELS Kenneth L. Judd Hoover Institution and NBER June 28, 2006 Functional Problems Many problems involve solving for some unknown function Dynamic programming Consumption
More informationNeural Networks in Structured Prediction. November 17, 2015
Neural Networks in Structured Prediction November 17, 2015 HWs and Paper Last homework is going to be posted soon Neural net NER tagging model This is a new structured model Paper - Thursday after Thanksgiving
More informationShort-time expansions for close-to-the-money options under a Lévy jump model with stochastic volatility
Short-time expansions for close-to-the-money options under a Lévy jump model with stochastic volatility José Enrique Figueroa-López 1 1 Department of Statistics Purdue University Statistics, Jump Processes,
More informationCompute the behavior of reality even if it is impossible to observe the processes (for example a black hole in astrophysics).
1 Introduction Read sections 1.1, 1.2.1 1.2.4, 1.2.6, 1.3.8, 1.3.9, 1.4. Review questions 1.1 1.6, 1.12 1.21, 1.37. The subject of Scientific Computing is to simulate the reality. Simulation is the representation
More informationVariational Bayesian Inference Techniques
Advanced Signal Processing 2, SE Variational Bayesian Inference Techniques Johann Steiner 1 Outline Introduction Sparse Signal Reconstruction Sparsity Priors Benefits of Sparse Bayesian Inference Variational
More informationSMO vs PDCO for SVM: Sequential Minimal Optimization vs Primal-Dual interior method for Convex Objectives for Support Vector Machines
vs for SVM: Sequential Minimal Optimization vs Primal-Dual interior method for Convex Objectives for Support Vector Machines Ding Ma Michael Saunders Working paper, January 5 Introduction In machine learning,
More informationLINEAR AND NONLINEAR PROGRAMMING
LINEAR AND NONLINEAR PROGRAMMING Stephen G. Nash and Ariela Sofer George Mason University The McGraw-Hill Companies, Inc. New York St. Louis San Francisco Auckland Bogota Caracas Lisbon London Madrid Mexico
More informationLecture 8: Fast Linear Solvers (Part 7)
Lecture 8: Fast Linear Solvers (Part 7) 1 Modified Gram-Schmidt Process with Reorthogonalization Test Reorthogonalization If Av k 2 + δ v k+1 2 = Av k 2 to working precision. δ = 10 3 2 Householder Arnoldi
More informationProbabilistic Graphical Models
Probabilistic Graphical Models Brown University CSCI 295-P, Spring 213 Prof. Erik Sudderth Lecture 11: Inference & Learning Overview, Gaussian Graphical Models Some figures courtesy Michael Jordan s draft
More informationGPU Applications for Modern Large Scale Asset Management
GPU Applications for Modern Large Scale Asset Management GTC 2014 San José, California Dr. Daniel Egloff QuantAlea & IncubeAdvisory March 27, 2014 Outline Portfolio Construction Outline Portfolio Construction
More informationAM 205: lecture 19. Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods Quasi-Newton Methods General form of quasi-newton methods: x k+1 = x k α
More informationStochastic Volatility and Correction to the Heat Equation
Stochastic Volatility and Correction to the Heat Equation Jean-Pierre Fouque, George Papanicolaou and Ronnie Sircar Abstract. From a probabilist s point of view the Twentieth Century has been a century
More informationDELFT UNIVERSITY OF TECHNOLOGY
DELFT UNIVERSITY OF TECHNOLOGY REPORT -09 Computational and Sensitivity Aspects of Eigenvalue-Based Methods for the Large-Scale Trust-Region Subproblem Marielba Rojas, Bjørn H. Fotland, and Trond Steihaug
More informationOpen-source finite element solver for domain decomposition problems
1/29 Open-source finite element solver for domain decomposition problems C. Geuzaine 1, X. Antoine 2,3, D. Colignon 1, M. El Bouajaji 3,2 and B. Thierry 4 1 - University of Liège, Belgium 2 - University
More informationVariance Reduction Techniques for Monte Carlo Simulations with Stochastic Volatility Models
Variance Reduction Techniques for Monte Carlo Simulations with Stochastic Volatility Models Jean-Pierre Fouque North Carolina State University SAMSI November 3, 5 1 References: Variance Reduction for Monte
More informationA new approach for investment performance measurement. 3rd WCMF, Santa Barbara November 2009
A new approach for investment performance measurement 3rd WCMF, Santa Barbara November 2009 Thaleia Zariphopoulou University of Oxford, Oxford-Man Institute and The University of Texas at Austin 1 Performance
More informationEfficient robust optimization for robust control with constraints Paul Goulart, Eric Kerrigan and Danny Ralph
Efficient robust optimization for robust control with constraints p. 1 Efficient robust optimization for robust control with constraints Paul Goulart, Eric Kerrigan and Danny Ralph Efficient robust optimization
More informationComputing the Geodesic Interpolating Spline
Computing the Geodesic Interpolating Spline Anna Mills 1, Tony Shardlow 1, and Stephen Marsland 2 1 The University of Manchester, UK 2 Massey University, NZ Abstract. We examine non-rigid image registration
More informationGLoBES. Patrick Huber. Physics Department VT. P. Huber p. 1
GLoBES Patrick Huber Physics Department VT P. Huber p. 1 P. Huber p. 2 General Long Baseline Experiment Simulator GLoBES is a software package designed for Simulation Analysis Comparison of neutrino oscillation
More informationGeog 469 GIS Workshop. Managing Enterprise GIS Geodatabases
Geog 469 GIS Workshop Managing Enterprise GIS Geodatabases Outline 1. Why is a geodatabase important for GIS? 2. What is the architecture of a geodatabase? 3. How can we compare and contrast three types
More informationAM 205: lecture 19. Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods Optimality Conditions: Equality Constrained Case As another example of equality
More informationREPORTS IN INFORMATICS
REPORTS IN INFORMATICS ISSN 0333-3590 A class of Methods Combining L-BFGS and Truncated Newton Lennart Frimannslund Trond Steihaug REPORT NO 319 April 2006 Department of Informatics UNIVERSITY OF BERGEN
More informationHydra. A library for data analysis in massively parallel platforms. A. Augusto Alves Jr and Michael D. Sokoloff
Hydra A library for data analysis in massively parallel platforms A. Augusto Alves Jr and Michael D. Sokoloff University of Cincinnati aalvesju@cern.ch Presented at NVIDIA s GPU Technology Conference,
More informationCP2K. New Frontiers. ab initio Molecular Dynamics
CP2K New Frontiers in ab initio Molecular Dynamics Jürg Hutter, Joost VandeVondele, Valery Weber Physical-Chemistry Institute, University of Zurich Ab Initio Molecular Dynamics Molecular Dynamics Sampling
More informationA nonlinear equation is any equation of the form. f(x) = 0. A nonlinear equation can have any number of solutions (finite, countable, uncountable)
Nonlinear equations Definition A nonlinear equation is any equation of the form where f is a nonlinear function. Nonlinear equations x 2 + x + 1 = 0 (f : R R) f(x) = 0 (x cos y, 2y sin x) = (0, 0) (f :
More informationNumerical Methods for Large-Scale Nonlinear Equations
Slide 1 Numerical Methods for Large-Scale Nonlinear Equations Homer Walker MA 512 April 28, 2005 Inexact Newton and Newton Krylov Methods a. Newton-iterative and inexact Newton methods. Slide 2 i. Formulation
More informationA computational architecture for coupling heterogeneous numerical models and computing coupled derivatives
This is a preprint of the following article, which is available from http://mdolab.engin.umich.edu John T. Hwang and J. R. R. A. Martins. A computational architecture for coupling heterogeneous numerical
More informationWhat s New in Active-Set Methods for Nonlinear Optimization?
What s New in Active-Set Methods for Nonlinear Optimization? Philip E. Gill Advances in Numerical Computation, Manchester University, July 5, 2011 A Workshop in Honor of Sven Hammarling UCSD Center for
More informationCalculation of Sound Fields in Flowing Media Using CAPA and Diffpack
Calculation of Sound Fields in Flowing Media Using CAPA and Diffpack H. Landes 1, M. Kaltenbacher 2, W. Rathmann 3, F. Vogel 3 1 WisSoft, 2 Univ. Erlangen 3 inutech GmbH Outline Introduction Sound in Flowing
More informationEfficient Serial and Parallel Coordinate Descent Methods for Huge-Scale Convex Optimization
Efficient Serial and Parallel Coordinate Descent Methods for Huge-Scale Convex Optimization Martin Takáč The University of Edinburgh Based on: P. Richtárik and M. Takáč. Iteration complexity of randomized
More informationParallel-in-Time Optimization with the General-Purpose XBraid Package
Parallel-in-Time Optimization with the General-Purpose XBraid Package Stefanie Günther, Jacob Schroder, Robert Falgout, Nicolas Gauger 7th Workshop on Parallel-in-Time methods Wednesday, May 3 rd, 2018
More informationUsing an Auction Algorithm in AMG based on Maximum Weighted Matching in Matrix Graphs
Using an Auction Algorithm in AMG based on Maximum Weighted Matching in Matrix Graphs Pasqua D Ambra Institute for Applied Computing (IAC) National Research Council of Italy (CNR) pasqua.dambra@cnr.it
More informationParallelization of Multilevel Preconditioners Constructed from Inverse-Based ILUs on Shared-Memory Multiprocessors
Parallelization of Multilevel Preconditioners Constructed from Inverse-Based ILUs on Shared-Memory Multiprocessors J.I. Aliaga 1 M. Bollhöfer 2 A.F. Martín 1 E.S. Quintana-Ortí 1 1 Deparment of Computer
More informationLecture 1 Numerical methods: principles, algorithms and applications: an introduction
Lecture 1 Numerical methods: principles, algorithms and applications: an introduction Weinan E 1,2 and Tiejun Li 2 1 Department of Mathematics, Princeton University, weinan@princeton.edu 2 School of Mathematical
More informationOptimization Tools in an Uncertain Environment
Optimization Tools in an Uncertain Environment Michael C. Ferris University of Wisconsin, Madison Uncertainty Workshop, Chicago: July 21, 2008 Michael Ferris (University of Wisconsin) Stochastic optimization
More informationWork in Progress: Reachability Analysis for Time-triggered Hybrid Systems, The Platoon Benchmark
Work in Progress: Reachability Analysis for Time-triggered Hybrid Systems, The Platoon Benchmark François Bidet LIX, École polytechnique, CNRS Université Paris-Saclay 91128 Palaiseau, France francois.bidet@polytechnique.edu
More informationNumerical Modelling in Fortran: day 9. Paul Tackley, 2017
Numerical Modelling in Fortran: day 9 Paul Tackley, 2017 Today s Goals 1. Implicit time stepping. 2. Useful libraries and other software 3. Fortran and numerical methods: Review and discussion. New features
More informationD03PEF NAG Fortran Library Routine Document
D03 Partial Differential Equations D03PEF NAG Fortran Library Routine Document Note. Before using this routine, please read the Users Note for your implementation to check the interpretation of bold italicised
More informationTreecodes for Cosmology Thomas Quinn University of Washington N-Body Shop
Treecodes for Cosmology Thomas Quinn University of Washington N-Body Shop Outline Motivation Multipole Expansions Tree Algorithms Periodic Boundaries Time integration Gravitational Softening SPH Parallel
More informationThe Chemical Kinetics Time Step a detailed lecture. Andrew Conley ACOM Division
The Chemical Kinetics Time Step a detailed lecture Andrew Conley ACOM Division Simulation Time Step Deep convection Shallow convection Stratiform tend (sedimentation, detrain, cloud fraction, microphysics)
More informationParallelization of the Dirac operator. Pushan Majumdar. Indian Association for the Cultivation of Sciences, Jadavpur, Kolkata
Parallelization of the Dirac operator Pushan Majumdar Indian Association for the Cultivation of Sciences, Jadavpur, Kolkata Outline Introduction Algorithms Parallelization Comparison of performances Conclusions
More informationReservoir Computing and Echo State Networks
An Introduction to: Reservoir Computing and Echo State Networks Claudio Gallicchio gallicch@di.unipi.it Outline Focus: Supervised learning in domain of sequences Recurrent Neural networks for supervised
More informationChapter 6. Differentially Flat Systems
Contents CAS, Mines-ParisTech 2008 Contents Contents 1, Linear Case Introductory Example: Linear Motor with Appended Mass General Solution (Linear Case) Contents Contents 1, Linear Case Introductory Example:
More informationNumerical methods in molecular dynamics and multiscale problems
Numerical methods in molecular dynamics and multiscale problems Two examples T. Lelièvre CERMICS - Ecole des Ponts ParisTech & MicMac project-team - INRIA Horizon Maths December 2012 Introduction The aim
More informationDynamic Risk Measures and Nonlinear Expectations with Markov Chain noise
Dynamic Risk Measures and Nonlinear Expectations with Markov Chain noise Robert J. Elliott 1 Samuel N. Cohen 2 1 Department of Commerce, University of South Australia 2 Mathematical Insitute, University
More informationA Fast Augmented Lagrangian Algorithm for Learning Low-Rank Matrices
A Fast Augmented Lagrangian Algorithm for Learning Low-Rank Matrices Ryota Tomioka 1, Taiji Suzuki 1, Masashi Sugiyama 2, Hisashi Kashima 1 1 The University of Tokyo 2 Tokyo Institute of Technology 2010-06-22
More informationHybrid Analog-Digital Solution of Nonlinear Partial Differential Equations
Hybrid Analog-Digital Solution of Nonlinear Partial Differential Equations Yipeng Huang, Ning Guo, Kyle Mandli, Mingoo Seok, Yannis Tsividis, Simha Sethumadhavan Columbia University Hybrid Analog-Digital
More informationRecent Progress of Parallel SAMCEF with MUMPS MUMPS User Group Meeting 2013
Recent Progress of Parallel SAMCEF with User Group Meeting 213 Jean-Pierre Delsemme Product Development Manager Summary SAMCEF, a brief history Co-simulation, a good candidate for parallel processing MAAXIMUS,
More informationA particle-in-cell method with adaptive phase-space remapping for kinetic plasmas
A particle-in-cell method with adaptive phase-space remapping for kinetic plasmas Bei Wang 1 Greg Miller 2 Phil Colella 3 1 Princeton Institute of Computational Science and Engineering Princeton University
More informationUsing AD to derive adjoints for large, legacy CFD solvers
Using AD to derive adjoints for large, legacy CFD solvers J.-D. Müller Queen Mary, University of London collaborators: D. Jones, F. Christakpoulos, S. Xu, QMUL S. Bayyuk, ESI, Huntsville AL c Jens-Dominik
More information2326 Problem Sheet 1. Solutions 1. First Order Differential Equations. Question 1: Solve the following, given an explicit solution if possible:
2326 Problem Sheet. Solutions First Order Differential Equations Question : Solve the following, given an explicit solution if possible:. 2 x = 4x3, () = 3, ( ) The given equation is a first order linear
More informationAirfoil shape optimization using adjoint method and automatic differentiation
Airfoil shape optimization using adjoint method and automatic differentiation Praveen. C praveen@math.tifrbng.res.in Tata Institute of Fundamental Research Center for Applicable Mathematics Bangalore 5665
More informationFAS and Solver Performance
FAS and Solver Performance Matthew Knepley Mathematics and Computer Science Division Argonne National Laboratory Fall AMS Central Section Meeting Chicago, IL Oct 05 06, 2007 M. Knepley (ANL) FAS AMS 07
More informationNumerical Integration of SDEs: A Short Tutorial
Numerical Integration of SDEs: A Short Tutorial Thomas Schaffter January 19, 010 1 Introduction 1.1 Itô and Stratonovich SDEs 1-dimensional stochastic differentiable equation (SDE) is given by [6, 7] dx
More informationAdvances in Extreme Learning Machines
Advances in Extreme Learning Machines Mark van Heeswijk April 17, 2015 Outline Context Extreme Learning Machines Part I: Ensemble Models of ELM Part II: Variable Selection and ELM Part III: Trade-offs
More informationParsimonious Adaptive Rejection Sampling
Parsimonious Adaptive Rejection Sampling Luca Martino Image Processing Laboratory, Universitat de València (Spain). Abstract Monte Carlo (MC) methods have become very popular in signal processing during
More informationZero-Order Methods for the Optimization of Noisy Functions. Jorge Nocedal
Zero-Order Methods for the Optimization of Noisy Functions Jorge Nocedal Northwestern University Simons Institute, October 2017 1 Collaborators Albert Berahas Northwestern University Richard Byrd University
More informationElectronics 101 Solving a differential equation Incorporating space. Numerical Methods. Accuracy, stability, speed. Robert A.
Numerical Methods Accuracy, stability, speed Robert A. McDougal Yale School of Medicine 21 June 2016 Hodgkin and Huxley: squid giant axon experiments Top: Alan Lloyd Hodgkin; Bottom: Andrew Fielding Huxley.
More informationFEniCS Course Lecture 13: Introduction to dolfin-adjoint. Contributors Simon Funke Patrick Farrell Marie E. Rognes
FEniCS Course Lecture 13: Introduction to dolfin-adjoint Contributors Simon Funke Patrick Farrell Marie E. Rognes 1 / 25 dolfin adjoint automatic adjoint models for FEniCS P. E. Farrell, S. W. Funke, D.
More informationAdjoint code development and optimization using automatic differentiation (AD)
Adjoint code development and optimization using automatic differentiation (AD) Praveen. C Computational and Theoretical Fluid Dynamics Division National Aerospace Laboratories Bangalore - 560 037 CTFD
More informationDual Reciprocity Method for studying thermal flows related to Magma Oceans
Dual Reciprocity Method for studying thermal flows related to Magma Oceans Tyler Drombosky Ph.D. Student, Applied Mathematics Scientific Computation Department of Mathematics University of Maryland, College
More information