Steepest descent minimization applied to a simple seismic inversion
|
|
- Osborn Farmer
- 5 years ago
- Views:
Transcription
1 Steepest descent minimization applied to a simple seismic inversion Sylvain Barbot Friday, December 10, Problem Definition The problem one tries to address is the estimation of the location of a seismic wave source. One considers the seismic wave to propagate straight from the epicenter to the seismograph without reflection. Besides, one knows the wave propagation velocity v. And the relation 1 holds: (x xi )2 + (y y i )2 t i = v Equation 1 is called the direct model: (1) d = g(m) One has a set of time of wave front arrival measured from different seismographs scattered on the surface (the problem has been reduced to a two-dimensional space). One writes t i the time of arrival measured from the seismograph i d = C d = Where d is data column matrix and C d the corresponding covariance matrix. One finally writes the a priori information on model parameters m prior and its covariance matrix C m. These two last objects are necessary for the gradient formulation but as no real information is available on the seismic source, one defines arbitrarily the seismic source in the middle of the seismograph but with a high uncertainty (the source might be around a 10km circle): m prior = [ ] C m = We first draw the posterior probability density [ ] (2) (3) σ m (m) = exp ( S(m)) (4) 1
2 Figure 1: Probability Density σ m. Left: x and y vary from 0 to 20 km; Right: a zoom, x and y vary from 14 to 16 km. where 2 S(m) = (g(m) d obs ) T C 1 d This σ m as represented in figure 1. (g(m) d obs )+(m m prior ) T C 1 m (m m prior ) (5) 2
3 2 Implementation and Results 2.1 Steepest Descent Algorithm OOne then implements the steepest descent algorithm to minimize S(m). The steepest ascent vector γ is defined by relation 6: γ k = C m G T k C 1 d (g(m k) d obs ) + (m k m prior ).... G k = δf i δf i δx δy.. (6) And we use the following algorithm which converges towards the best estimation in the least square sense fitting the data parameters with respect to our direct model expression: m k+1 = m k εγ k (7) Finally, the a posteriori covariance matrix holds: C m = (G T k C 1 d G k + C 1 m ) 1 (8) One has implemented the steepest descent algorithm on IDL to find numerically the solution of this simple seismic inversion and to display the probability density shape. One has chosen ε arbitrarily with the trial and error method. Values close to 1 fail because too large to allow the convergence: we have chosen the value because we have verified in a first computation that the steepest ascent vector has components of the order of 10 4, and we want our jumps to be of the order of one km. Should the value have not ensures convergence, we would have taken half the value. After iterations, the computer answers: Least Squares Solution: Covariance Matrix:
4 2.2 Quasi-Newton Algorithm One has then implemented the quasi-newton algorithm. The difference holding in the calculation of the hessian matrix ξ k at each iteration: The algorithm becomes: ξ k = (G T k C 1 D G k + C 1 D ) 1 (9) γ k = G T k C 1 d (g(m k) d obs ) + C 1 M (m k m prior ) (10) m k+1 = m k ξ k γ k After 5 iterations, the programs displays: Least Squares Solution: Covariance Matrix:
5 3 Appendix FUNCTION seismic_proba_density d_obs = TRANSPOSE([3.12, 3.26, 2.98, 3.12, 2.84, 2.98]) data_cov = DIAG_MATRIX(REPLICATE(.1^2, 6)) m_prior = TRANSPOSE([4, 15.5]) prior_cov = DIAG_MATRIX(REPLICATE(10.^2, 2)) S = FLTARR(100, 100) FOR l = 0, DO BEGIN y = l / 5. FOR c = 0, DO BEGIN x = c / 5. S[c, l] = 1./2. * (TRANSPOSE(seismic_direct([x, y]) - d_obs) ## $ INVERT(data_cov) ## (seismic_direct([x, y]) - d_obs)) S[c, l]+= 1./2. * (TRANSPOSE([[x], [y]] - m_prior) ## $ INVERT(prior_cov) ## ([[x], [y]] - m_prior)) ENDFOR ENDFOR distribution = EXP(- S) END Table 1: IDL code calculating the probability density ρ M = exp ( S(m)) 5
6 PRO seismic_inversion iteration = & eps = 1e-5 d_obs = TRANSPOSE([3.12, 3.26, 2.98, 3.12, 2.84, 2.98]) data_cov = DIAG_MATRIX(REPLICATE(.1^2, 6)) m_prior = TRANSPOSE([10, 2]) prior_cov = [[1d, 0d], [0, 100d^2]] Mk = m_prior FOR k = 0, iteration - 1 DO BEGIN B = seismic_direct(mk) - d_obs LUDC, data_cov, index matrice = TRANSPOSE(LUSOL(data_cov, index, TRANSPOSE(B))) Gk = seismic_gradient(mk) vector = prior_cov ## TRANSPOSE(Gk) ## matrice + (Mk - m_prior) mk = mk - eps * vector ENDFOR post_cov = INVERT(TRANSPOSE(Gk) ## $ INVERT(data_cov) ## Gk + INVERT(data_cov)) PRINT, Least Squares Solution:, mk PRINT, Covariance Matrix:, post_cov Table 2: Procedure running the steepest descent algorithm core. 6
7 FUNCTION seismic_direct, m x = [3., 3., 4., 4., 5., 5.] y = [15., 16., 15., 16., 15., 16.] v = 5 RETURN, TRANSPOSE(SQRT((m[0] - x)^2 + (m[1] - y)^2) / v) END FUNCTION seismic_gradient, m x = [3., 3., 4., 4., 5., 5.] y = [15., 16., 15., 16., 15., 16.] v = 5 result = [[(m[0] - x) / (v * SQRT((m[0] - x)^2 + (m[1] - y)^2))],$ [(m[1] - y) / (v * SQRT((m[0] - x)^2 + (m[1] - y)^2))]] RETURN, TRANSPOSE(result) END Table 3: Ancillary functions calculating the direct model and the gradient matrix. 7
8 PRO seismic_inversion_quasi_newton iteration = 5 d_obs = TRANSPOSE([3.12, 3.26, 2.98, 3.12, 2.84, 2.98]) data_cov = DIAG_MATRIX(REPLICATE(.1^2, 6)) m_prior = TRANSPOSE([10, 2]) prior_cov = [[1d, 0d], [0, 100d^2]] Mk = m_prior FOR k = 0, iteration - 1 DO BEGIN B = seismic_direct(mk) - d_obs LUDC, data_cov, index matrice = TRANSPOSE(LUSOL(data_cov, index, TRANSPOSE(B))) Gk = seismic_gradient(mk) hessian = INVERT(TRANSPOSE(Gk) ## idata_cov ## Gk + iprior_cov) vector = TRANSPOSE(Gk) ## matrice + iprior_cov ## (Mk - m_prior) mk = mk - hessian ## vector ENDFOR PRINT, Least Squares Solution:, mk PRINT, Covariance Matrix:, hessian Table 4: Procedure running the Quasi-Newton algorithm 8
Problem Set 5: Iterative methods with generalized least squares
Problem Set 5: Iterative methods with generalized least squares GEOS 627: Inverse Problems and Parameter Estimation, Carl Tape Assigned: February 20, 2017 Due: March 6, 2017 Last compiled: July 7, 2017
More informationnonrobust estimation The n measurement vectors taken together give the vector X R N. The unknown parameter vector is P R M.
Introduction to nonlinear LS estimation R. I. Hartley and A. Zisserman: Multiple View Geometry in Computer Vision. Cambridge University Press, 2ed., 2004. After Chapter 5 and Appendix 6. We will use x
More informationApplication of new Monte Carlo method for inversion of prestack seismic data. Yang Xue Advisor: Dr. Mrinal K. Sen
Application of new Monte Carlo method for inversion of prestack seismic data Yang Xue Advisor: Dr. Mrinal K. Sen Overview Motivation Introduction Bayes theorem Stochastic inference methods Methodology
More informationECE 680 Modern Automatic Control. Gradient and Newton s Methods A Review
ECE 680Modern Automatic Control p. 1/1 ECE 680 Modern Automatic Control Gradient and Newton s Methods A Review Stan Żak October 25, 2011 ECE 680Modern Automatic Control p. 2/1 Review of the Gradient Properties
More informationThe Inversion Problem: solving parameters inversion and assimilation problems
The Inversion Problem: solving parameters inversion and assimilation problems UE Numerical Methods Workshop Romain Brossier romain.brossier@univ-grenoble-alpes.fr ISTerre, Univ. Grenoble Alpes Master 08/09/2016
More informationCourse on Inverse Problems
California Institute of Technology Division of Geological and Planetary Sciences March 26 - May 25, 2007 Course on Inverse Problems Albert Tarantola Institut de Physique du Globe de Paris Lesson XVI CONCLUSION
More informationCourse on Inverse Problems Albert Tarantola
California Institute of Technology Division of Geological and Planetary Sciences March 26 - May 25, 27 Course on Inverse Problems Albert Tarantola Institut de Physique du Globe de Paris CONCLUSION OF THE
More informationLAB. Balboa Heights, Panama. Boulder, Colorado. Mexico City, Mexico. Data Table. Difference Between P-wave and S-wave. S-wave Arrival Time
Name: Date: Lab Period: Locating the Epicenter of an Earthquake Introduction: The epicenter is the point on Earth's surface directly above an earthquake. Seismic stations detect earthquakes by the tracings
More informationNumerical Optimization Algorithms
Numerical Optimization Algorithms 1. Overview. Calculus of Variations 3. Linearized Supersonic Flow 4. Steepest Descent 5. Smoothed Steepest Descent Overview 1 Two Main Categories of Optimization Algorithms
More informationMultivariate Newton Minimanization
Multivariate Newton Minimanization Optymalizacja syntezy biosurfaktantu Rhamnolipid Rhamnolipids are naturally occuring glycolipid produced commercially by the Pseudomonas aeruginosa species of bacteria.
More informationOptimization method examples in Matlab Carl Tape, Qinya Liu, Albert Tarantola July 2, The forward problem 3
Optimization method examples in Matlab Carl Tape, Qinya Liu, Albert Tarantola July 2, 27 This file : /home/carltape/latex/notes/tromp/optimization_guide.tex Matlab code : /opt/seismo-util/source/optim_matlab/optimization.m
More informationNote that, when perturbing c(x) instead of m(x), an additional Taylor approximation is necessary: 1 c 2 (x) 1. c 2 0(x) 2εc 1(x).
Chapter 3 Scattering series In this chapter we describe the nonlinearity of the map c u in terms of a perturbation (Taylor) series. To first order, the linearization of this map is called the Born approximation.
More informationQuasi-Newton Methods
Newton s Method Pros and Cons Quasi-Newton Methods MA 348 Kurt Bryan Newton s method has some very nice properties: It s extremely fast, at least once it gets near the minimum, and with the simple modifications
More informationOptimization II: Unconstrained Multivariable
Optimization II: Unconstrained Multivariable CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Justin Solomon CS 205A: Mathematical Methods Optimization II: Unconstrained Multivariable 1
More informationMA/OR/ST 706: Nonlinear Programming Midterm Exam Instructor: Dr. Kartik Sivaramakrishnan INSTRUCTIONS
MA/OR/ST 706: Nonlinear Programming Midterm Exam Instructor: Dr. Kartik Sivaramakrishnan INSTRUCTIONS 1. Please write your name and student number clearly on the front page of the exam. 2. The exam is
More informationOptimization II: Unconstrained Multivariable
Optimization II: Unconstrained Multivariable CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Doug James (and Justin Solomon) CS 205A: Mathematical Methods Optimization II: Unconstrained
More informationZero-Order Methods for the Optimization of Noisy Functions. Jorge Nocedal
Zero-Order Methods for the Optimization of Noisy Functions Jorge Nocedal Northwestern University Simons Institute, October 2017 1 Collaborators Albert Berahas Northwestern University Richard Byrd University
More information4. DATA ASSIMILATION FUNDAMENTALS
4. DATA ASSIMILATION FUNDAMENTALS... [the atmosphere] "is a chaotic system in which errors introduced into the system can grow with time... As a consequence, data assimilation is a struggle between chaotic
More informationMethods that avoid calculating the Hessian. Nonlinear Optimization; Steepest Descent, Quasi-Newton. Steepest Descent
Nonlinear Optimization Steepest Descent and Niclas Börlin Department of Computing Science Umeå University niclas.borlin@cs.umu.se A disadvantage with the Newton method is that the Hessian has to be derived
More informationLogistic Regression. Seungjin Choi
Logistic Regression Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationAugmented Reality VU numerical optimization algorithms. Prof. Vincent Lepetit
Augmented Reality VU numerical optimization algorithms Prof. Vincent Lepetit P3P: can exploit only 3 correspondences; DLT: difficult to exploit the knowledge of the internal parameters correctly, and the
More informationNonlinear Optimization: What s important?
Nonlinear Optimization: What s important? Julian Hall 10th May 2012 Convexity: convex problems A local minimizer is a global minimizer A solution of f (x) = 0 (stationary point) is a minimizer A global
More informationOptimization. Totally not complete this is...don't use it yet...
Optimization Totally not complete this is...don't use it yet... Bisection? Doing a root method is akin to doing a optimization method, but bi-section would not be an effective method - can detect sign
More informationFALL 2018 MATH 4211/6211 Optimization Homework 4
FALL 2018 MATH 4211/6211 Optimization Homework 4 This homework assignment is open to textbook, reference books, slides, and online resources, excluding any direct solution to the problem (such as solution
More informationApproximate Message Passing
Approximate Message Passing Mohammad Emtiyaz Khan CS, UBC February 8, 2012 Abstract In this note, I summarize Sections 5.1 and 5.2 of Arian Maleki s PhD thesis. 1 Notation We denote scalars by small letters
More informationGEOPHYSICAL INVERSE THEORY AND REGULARIZATION PROBLEMS
Methods in Geochemistry and Geophysics, 36 GEOPHYSICAL INVERSE THEORY AND REGULARIZATION PROBLEMS Michael S. ZHDANOV University of Utah Salt Lake City UTAH, U.S.A. 2OO2 ELSEVIER Amsterdam - Boston - London
More informationLab Activity Locating Epicenters
Lab Activity Locating Epicenters Key Features: Analyzing Seismograms P-waves & S-waves Using ESRT page 11 Drawing Epicenter Circles By Kaitlin Coder https://www.teacherspayteachers.com/store/nys-earth-science-and-living-environment-regents
More informationVasil Khalidov & Miles Hansard. C.M. Bishop s PRML: Chapter 5; Neural Networks
C.M. Bishop s PRML: Chapter 5; Neural Networks Introduction The aim is, as before, to find useful decompositions of the target variable; t(x) = y(x, w) + ɛ(x) (3.7) t(x n ) and x n are the observations,
More informationECE580 Exam 1 October 4, Please do not write on the back of the exam pages. Extra paper is available from the instructor.
ECE580 Exam 1 October 4, 2012 1 Name: Solution Score: /100 You must show ALL of your work for full credit. This exam is closed-book. Calculators may NOT be used. Please leave fractions as fractions, etc.
More information5 Quasi-Newton Methods
Unconstrained Convex Optimization 26 5 Quasi-Newton Methods If the Hessian is unavailable... Notation: H = Hessian matrix. B is the approximation of H. C is the approximation of H 1. Problem: Solve min
More informationRegression with Numerical Optimization. Logistic
CSG220 Machine Learning Fall 2008 Regression with Numerical Optimization. Logistic regression Regression with Numerical Optimization. Logistic regression based on a document by Andrew Ng October 3, 204
More informationNonlinear equations and optimization
Notes for 2017-03-29 Nonlinear equations and optimization For the next month or so, we will be discussing methods for solving nonlinear systems of equations and multivariate optimization problems. We will
More information1 Newton s Method. Suppose we want to solve: x R. At x = x, f (x) can be approximated by:
Newton s Method Suppose we want to solve: (P:) min f (x) At x = x, f (x) can be approximated by: n x R. f (x) h(x) := f ( x)+ f ( x) T (x x)+ (x x) t H ( x)(x x), 2 which is the quadratic Taylor expansion
More informationOptimization Methods for Machine Learning
Optimization Methods for Machine Learning Sathiya Keerthi Microsoft Talks given at UC Santa Cruz February 21-23, 2017 The slides for the talks will be made available at: http://www.keerthis.com/ Introduction
More informationCoE 3SK3 Computer Aided Engineering Tutorial: Unconstrained Optimization
CoE 3SK3 Computer Aided Engineering Tutorial: Unconstrained Optimization Jie Cao caoj23@grads.ece.mcmaster.ca Department of Electrical and Computer Engineering McMaster University Feb. 2, 2010 Outline
More information, b = 0. (2) 1 2 The eigenvectors of A corresponding to the eigenvalues λ 1 = 1, λ 2 = 3 are
Quadratic forms We consider the quadratic function f : R 2 R defined by f(x) = 2 xt Ax b T x with x = (x, x 2 ) T, () where A R 2 2 is symmetric and b R 2. We will see that, depending on the eigenvalues
More informationTruncated Conjugate Gradient Method for History Matching in Reservoir Simulation
Trabalho apresentado no CNAC, Gramado - RS, 2016. Proceeding Series of the Brazilian Society of Computational and Applied athematics Truncated Conjugate Gradient ethod for History atching in Reservoir
More informationUncertainty quantification for inverse problems with a weak wave-equation constraint
Uncertainty quantification for inverse problems with a weak wave-equation constraint Zhilong Fang*, Curt Da Silva*, Rachel Kuske** and Felix J. Herrmann* *Seismic Laboratory for Imaging and Modeling (SLIM),
More informationComparative study of Optimization methods for Unconstrained Multivariable Nonlinear Programming Problems
International Journal of Scientific and Research Publications, Volume 3, Issue 10, October 013 1 ISSN 50-3153 Comparative study of Optimization methods for Unconstrained Multivariable Nonlinear Programming
More informationMATH 4211/6211 Optimization Quasi-Newton Method
MATH 4211/6211 Optimization Quasi-Newton Method Xiaojing Ye Department of Mathematics & Statistics Georgia State University Xiaojing Ye, Math & Stat, Georgia State University 0 Quasi-Newton Method Motivation:
More informationσ(a) = a N (x; 0, 1 2 ) dx. σ(a) = Φ(a) =
Until now we have always worked with likelihoods and prior distributions that were conjugate to each other, allowing the computation of the posterior distribution to be done in closed form. Unfortunately,
More informationUncertainty quantification for Wavefield Reconstruction Inversion
Uncertainty quantification for Wavefield Reconstruction Inversion Zhilong Fang *, Chia Ying Lee, Curt Da Silva *, Felix J. Herrmann *, and Rachel Kuske * Seismic Laboratory for Imaging and Modeling (SLIM),
More informationStrong Lens Modeling (II): Statistical Methods
Strong Lens Modeling (II): Statistical Methods Chuck Keeton Rutgers, the State University of New Jersey Probability theory multiple random variables, a and b joint distribution p(a, b) conditional distribution
More informationData Fitting and Uncertainty
TiloStrutz Data Fitting and Uncertainty A practical introduction to weighted least squares and beyond With 124 figures, 23 tables and 71 test questions and examples VIEWEG+ TEUBNER IX Contents I Framework
More informationOptimization: Nonlinear Optimization without Constraints. Nonlinear Optimization without Constraints 1 / 23
Optimization: Nonlinear Optimization without Constraints Nonlinear Optimization without Constraints 1 / 23 Nonlinear optimization without constraints Unconstrained minimization min x f(x) where f(x) is
More informationNonlinear Least Squares Curve Fitting
Nonlinear Least Squares Curve Fitting Jack Merrin February 12, 2017 1 Summary There is no general solution for the minimization of chi-squared to find the best estimates of the parameters when the model
More informationChapter 3 Numerical Methods
Chapter 3 Numerical Methods Part 2 3.2 Systems of Equations 3.3 Nonlinear and Constrained Optimization 1 Outline 3.2 Systems of Equations 3.3 Nonlinear and Constrained Optimization Summary 2 Outline 3.2
More informationEpicenter - a point, directly above the true center of disturbance, from which the shock waves of an earthquake apparently radiate.
Name: Date: Period: Earthquakes Epicenter - a point, directly above the true center of disturbance, from which the shock waves of an earthquake apparently radiate. Define each of the following terms in
More informationEarthquake information
Name: Date: Block Earthquake information The graph below shows travel time in minutes and distance traveled for primary and secondary waves. Primary and secondary waves start at the same time but do not
More informationTrajectory-based optimization
Trajectory-based optimization Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Winter 2012 Emo Todorov (UW) AMATH/CSE 579, Winter 2012 Lecture 6 1 / 13 Using
More informationK/U /39 T/I /50 C /102 A
Name: Partner: K/U /39 T/I /50 C /102 A Purpose: What is the relationship between the magnitude of the force causing the acceleration and the frequency of revolution of an object in uniform circular motion?
More informationA Robot that Learns an Evaluation Function for Acquiring of Appropriate Motions
A Robot that Learns an Evaluation Function for Acquiring of Appropriate Motions Katsunari Shibata and Yoich Okabe Research Center for Advanced Science and Technology, Univ. of Tokyo -6-1 Komaba, Meguro-ku,
More informationIntroduction. New Nonsmooth Trust Region Method for Unconstraint Locally Lipschitz Optimization Problems
New Nonsmooth Trust Region Method for Unconstraint Locally Lipschitz Optimization Problems Z. Akbari 1, R. Yousefpour 2, M. R. Peyghami 3 1 Department of Mathematics, K.N. Toosi University of Technology,
More informationMachine Learning Brett Bernstein. Recitation 1: Gradients and Directional Derivatives
Machine Learning Brett Bernstein Recitation 1: Gradients and Directional Derivatives Intro Question 1 We are given the data set (x 1, y 1 ),, (x n, y n ) where x i R d and y i R We want to fit a linear
More informationQuantitative Information Flow. Lecture 7
Quantitative Information Flow Lecture 7 1 The basic model: Systems = Information-Theoretic channels Secret Information Observables s1 o1... System... sm on Input Output 2 Probabilistic systems are noisy
More informationMachine Learning for Signal Processing Expectation Maximization Mixture Models. Bhiksha Raj 27 Oct /
Machine Learning for Signal rocessing Expectation Maximization Mixture Models Bhiksha Raj 27 Oct 2016 11755/18797 1 Learning Distributions for Data roblem: Given a collection of examples from some data,
More informationChapter 9. Derivatives. Josef Leydold Mathematical Methods WS 2018/19 9 Derivatives 1 / 51. f x. (x 0, f (x 0 ))
Chapter 9 Derivatives Josef Leydold Mathematical Methods WS 208/9 9 Derivatives / 5 Difference Quotient Let f : R R be some function. The the ratio f = f ( 0 + ) f ( 0 ) = f ( 0) 0 is called difference
More informationEngineering Part IIB: Module 4F10 Statistical Pattern Processing Lecture 5: Single Layer Perceptrons & Estimating Linear Classifiers
Engineering Part IIB: Module 4F0 Statistical Pattern Processing Lecture 5: Single Layer Perceptrons & Estimating Linear Classifiers Phil Woodland: pcw@eng.cam.ac.uk Michaelmas 202 Engineering Part IIB:
More informationConvex Optimization CMU-10725
Convex Optimization CMU-10725 Quasi Newton Methods Barnabás Póczos & Ryan Tibshirani Quasi Newton Methods 2 Outline Modified Newton Method Rank one correction of the inverse Rank two correction of the
More informationMulti-layer Neural Networks
Multi-layer Neural Networks Steve Renals Informatics 2B Learning and Data Lecture 13 8 March 2011 Informatics 2B: Learning and Data Lecture 13 Multi-layer Neural Networks 1 Overview Multi-layer neural
More informationApproximate- vs. full-hessian in FWI: 1D analytical and numerical experiments
Approximate- vs. full-hessian in FWI: 1D analytical and numerical experiments Raul Cova and Kris Innanen ABSTRACT Feasibility of using Full Waveform Inversion (FWI) to build velocity models has been increasing
More informationBayesian Computation in Color-Magnitude Diagrams
Bayesian Computation in Color-Magnitude Diagrams SA, AA, PT, EE, MCMC and ASIS in CMDs Paul Baines Department of Statistics Harvard University October 19, 2009 Overview Motivation and Introduction Modelling
More informationOptimization Methods
Optimization Methods Decision making Examples: determining which ingredients and in what quantities to add to a mixture being made so that it will meet specifications on its composition allocating available
More informationVariational data assimilation
Background and methods NCEO, Dept. of Meteorology, Univ. of Reading 710 March 2018, Univ. of Reading Bayes' Theorem Bayes' Theorem p(x y) = posterior distribution = p(x) p(y x) p(y) prior distribution
More informationTraining Schro dinger s Cat
Training Schro dinger s Cat Quadratically Converging algorithms for Optimal Control of Quantum Systems David L. Goodwin & Ilya Kuprov d.goodwin@soton.ac.uk comp-chem@southampton, Wednesday 15th July 2015
More information4 Newton Method. Unconstrained Convex Optimization 21. H(x)p = f(x). Newton direction. Why? Recall second-order staylor series expansion:
Unconstrained Convex Optimization 21 4 Newton Method H(x)p = f(x). Newton direction. Why? Recall second-order staylor series expansion: f(x + p) f(x)+p T f(x)+ 1 2 pt H(x)p ˆf(p) In general, ˆf(p) won
More informationLecture 10. Neural networks and optimization. Machine Learning and Data Mining November Nando de Freitas UBC. Nonlinear Supervised Learning
Lecture 0 Neural networks and optimization Machine Learning and Data Mining November 2009 UBC Gradient Searching for a good solution can be interpreted as looking for a minimum of some error (loss) function
More informationHOMEWORK #4: LOGISTIC REGRESSION
HOMEWORK #4: LOGISTIC REGRESSION Probabilistic Learning: Theory and Algorithms CS 274A, Winter 2019 Due: 11am Monday, February 25th, 2019 Submit scan of plots/written responses to Gradebook; submit your
More informationAdaptive Filter Theory
0 Adaptive Filter heory Sung Ho Cho Hanyang University Seoul, Korea (Office) +8--0-0390 (Mobile) +8-10-541-5178 dragon@hanyang.ac.kr able of Contents 1 Wiener Filters Gradient Search by Steepest Descent
More informationHow Do Scientists Find the Epicenter of an Earthquake?
3.4 Explore How Do Scientists Find the Epicenter of an Earthquake? Seismograph data says that the earthquake is 100 km (62 mi) away, but at which point on the circle is the earthquake located? EE 116 3.4
More informationNumerical Methods for PDE-Constrained Optimization
Numerical Methods for PDE-Constrained Optimization Richard H. Byrd 1 Frank E. Curtis 2 Jorge Nocedal 2 1 University of Colorado at Boulder 2 Northwestern University Courant Institute of Mathematical Sciences,
More informationMidterm. Introduction to Machine Learning. CS 189 Spring Please do not open the exam before you are instructed to do so.
CS 89 Spring 07 Introduction to Machine Learning Midterm Please do not open the exam before you are instructed to do so. The exam is closed book, closed notes except your one-page cheat sheet. Electronic
More informationUniversität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen. Bayesian Learning. Tobias Scheffer, Niels Landwehr
Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen Bayesian Learning Tobias Scheffer, Niels Landwehr Remember: Normal Distribution Distribution over x. Density function with parameters
More informationNonlinearOptimization
1/35 NonlinearOptimization Pavel Kordík Department of Computer Systems Faculty of Information Technology Czech Technical University in Prague Jiří Kašpar, Pavel Tvrdík, 2011 Unconstrained nonlinear optimization,
More informationProgramming, numerics and optimization
Programming, numerics and optimization Lecture C-3: Unconstrained optimization II Łukasz Jankowski ljank@ippt.pan.pl Institute of Fundamental Technological Research Room 4.32, Phone +22.8261281 ext. 428
More informationRegression. Oscar García
Regression Oscar García Regression methods are fundamental in Forest Mensuration For a more concise and general presentation, we shall first review some matrix concepts 1 Matrices An order n m matrix is
More informationManaging Uncertainty
Managing Uncertainty Bayesian Linear Regression and Kalman Filter December 4, 2017 Objectives The goal of this lab is multiple: 1. First it is a reminder of some central elementary notions of Bayesian
More informationNumerical Methods I Solving Nonlinear Equations
Numerical Methods I Solving Nonlinear Equations Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 October 16th, 2014 A. Donev (Courant Institute)
More informationGradient Descent. Dr. Xiaowei Huang
Gradient Descent Dr. Xiaowei Huang https://cgi.csc.liv.ac.uk/~xiaowei/ Up to now, Three machine learning algorithms: decision tree learning k-nn linear regression only optimization objectives are discussed,
More informationOptimization. Yuh-Jye Lee. March 21, Data Science and Machine Intelligence Lab National Chiao Tung University 1 / 29
Optimization Yuh-Jye Lee Data Science and Machine Intelligence Lab National Chiao Tung University March 21, 2017 1 / 29 You Have Learned (Unconstrained) Optimization in Your High School Let f (x) = ax
More informationMathematical optimization
Optimization Mathematical optimization Determine the best solutions to certain mathematically defined problems that are under constrained determine optimality criteria determine the convergence of the
More informationIterative linearized migration and inversion
Stanford Exploration Project, Report 123, October 31, 2005, pages 101 128 Iterative linearized migration and inversion Huazhong Wang 1 ABSTRACT The objective of seismic imaging is to obtain an image of
More informationParallel Algorithms for Four-Dimensional Variational Data Assimilation
Parallel Algorithms for Four-Dimensional Variational Data Assimilation Mie Fisher ECMWF October 24, 2011 Mie Fisher (ECMWF) Parallel 4D-Var October 24, 2011 1 / 37 Brief Introduction to 4D-Var Four-Dimensional
More information2.098/6.255/ Optimization Methods Practice True/False Questions
2.098/6.255/15.093 Optimization Methods Practice True/False Questions December 11, 2009 Part I For each one of the statements below, state whether it is true or false. Include a 1-3 line supporting sentence
More informationIntroduction to unconstrained optimization - direct search methods
Introduction to unconstrained optimization - direct search methods Jussi Hakanen Post-doctoral researcher jussi.hakanen@jyu.fi Structure of optimization methods Typically Constraint handling converts the
More information8 Numerical methods for unconstrained problems
8 Numerical methods for unconstrained problems Optimization is one of the important fields in numerical computation, beside solving differential equations and linear systems. We can see that these fields
More informationLecture 4: Types of errors. Bayesian regression models. Logistic regression
Lecture 4: Types of errors. Bayesian regression models. Logistic regression A Bayesian interpretation of regularization Bayesian vs maximum likelihood fitting more generally COMP-652 and ECSE-68, Lecture
More informationNon-linear least-squares inversion with data-driven
Geophys. J. Int. (2000) 142, 000 000 Non-linear least-squares inversion with data-driven Bayesian regularization Tor Erik Rabben and Bjørn Ursin Department of Petroleum Engineering and Applied Geophysics,
More informationNumerical Optimization Prof. Shirish K. Shevade Department of Computer Science and Automation Indian Institute of Science, Bangalore
Numerical Optimization Prof. Shirish K. Shevade Department of Computer Science and Automation Indian Institute of Science, Bangalore Lecture - 13 Steepest Descent Method Hello, welcome back to this series
More informationSQUAREM. An R package for Accelerating Slowly Convergent Fixed-Point Iterations Including the EM and MM algorithms.
An R package for Accelerating Slowly Convergent Fixed-Point Iterations Including the EM and MM algorithms Ravi 1 1 Division of Geriatric Medicine & Gerontology Johns Hopkins University Baltimore, MD, USA
More informationConstrained optimization. Unconstrained optimization. One-dimensional. Multi-dimensional. Newton with equality constraints. Active-set method.
Optimization Unconstrained optimization One-dimensional Multi-dimensional Newton s method Basic Newton Gauss- Newton Quasi- Newton Descent methods Gradient descent Conjugate gradient Constrained optimization
More informationInverse Kinematics. Mike Bailey.
Inverse Kinematics This work is licensed under a Creative Commons Attribution-NonCommercial- NoDerivatives 4.0 International License Mike Bailey mjb@cs.oregonstate.edu inversekinematics.pptx Inverse Kinematics
More informationNumerical Optimization
Numerical Optimization Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Spring 2010 Emo Todorov (UW) AMATH/CSE 579, Spring 2010 Lecture 9 1 / 8 Gradient descent
More informationMaximum Likelihood Estimation
Maximum Likelihood Estimation Guy Lebanon February 19, 2011 Maximum likelihood estimation is the most popular general purpose method for obtaining estimating a distribution from a finite sample. It was
More informationNumerical Optimization: Basic Concepts and Algorithms
May 27th 2015 Numerical Optimization: Basic Concepts and Algorithms R. Duvigneau R. Duvigneau - Numerical Optimization: Basic Concepts and Algorithms 1 Outline Some basic concepts in optimization Some
More informationOptimization Part 1 P. Agius L2.1, Spring 2008
Optimization Part 1 Contents Terms and definitions, formulating the problem Unconstrained optimization conditions FONC, SONC, SOSC Searching for the solution Which direction? What step size? Constrained
More information. α β γ δ ε ζ η θ ι κ λ μ Aμ ν(x) ξ ο π ρ ς σ τ υ φ χ ψ ω. Α Β Γ Δ Ε Ζ Η Θ Ι Κ Λ Μ Ν Ξ Ο Π Ρ Σ Τ Υ Φ Χ Ψ Ω. Friday April 1 ± ǁ
. α β γ δ ε ζ η θ ι κ λ μ Aμ ν(x) ξ ο π ρ ς σ τ υ φ χ ψ ω. Α Β Γ Δ Ε Ζ Η Θ Ι Κ Λ Μ Ν Ξ Ο Π Ρ Σ Τ Υ Φ Χ Ψ Ω Friday April 1 ± ǁ 1 Chapter 5. Photons: Covariant Theory 5.1. The classical fields 5.2. Covariant
More informationMachine Learning CS 4900/5900. Lecture 03. Razvan C. Bunescu School of Electrical Engineering and Computer Science
Machine Learning CS 4900/5900 Razvan C. Bunescu School of Electrical Engineering and Computer Science bunescu@ohio.edu Machine Learning is Optimization Parametric ML involves minimizing an objective function
More informationECE580 Exam 2 November 01, Name: Score: / (20 points) You are given a two data sets
ECE580 Exam 2 November 01, 2011 1 Name: Score: /100 You must show ALL of your work for full credit. This exam is closed-book. Calculators may NOT be used. Please leave fractions as fractions, etc. I do
More informationFrom the propagation of uncertainties to the propagation of distributions
From the propagation of uncertainties to the propagation of distributions Carlo Carobbi Dipartimento di Elettronica e Telecomunicazioni Università degli Studi di Firenze Istituto di Fisica Applicata "Nello
More information