Some Comments on Accelerating Convergence of Iterative Sequences Using Direct Inversion of the Iterative Subspace (DIIS)


 Hollie Jefferson
 4 years ago
 Views:
Transcription
1 Some Comments on Acceleratng Convergence of Iteratve Sequences Usng Drect Inverson of the Iteratve Subspace (DIIS) C. Davd Sherrll School of Chemstry and Bochemstry Georga Insttute of Technology May 1998 Introducton Most standard textbook approaches to solvng systems of lnear equatons or dagonalzng matrces are descrbed as drect methods, and they typcally requre a fxed number of mathematcal operatons whch depends on the dmensons of the problem. These methods generally requre access to matrx elements n random order, whch poses serous dffcultes for the very large matrces typcally encountered n computatonal quantum chemstry: random access of large dsk fles becomes prohbtvely expensve, and often the matrces are too large even to store on dsk! In such cases, one may avod the need for random access to ndvdual matrx elements by turnng to teratve technques, whch requre only the repeated evaluaton of matrxvector products. Unfortunately, teratve methods are not guaranteed to converge, and they can have dffcultes when the matrx s not dagonally domnant or when there are nearly degenerate solutons. The wellknown Davdson method [1] for the teratve soluton of the lowest few egenvalues and egenvectors of large, symmetrc matrces combnes some of the features of drect and teratve technques. Although only matrxvector operatons are requred, and there s no need to explctly store the Hamltonan matrx, Davdson s method also uses drect methods to dagonalze a small Hamltonan matrx formed n the subspace of all tral CI vectors that have been consdered up to the present teraton. The current estmates of the egenvalues of the full Hamltonan matrx are obtaned as the egenvalues the small Hamltonan matrx, and the current CI vectors are obtaned as the lnear combnatons of the tral vectors whose coeffcents are gven by the egenvectors of the small Hamltonan matrx. Pople and coworkers later used related deas to teratvely solve the large systems of lnear equatons occurng n the coupledperturbed HartreeFock method [2]. In 1980, Pulay publshed a somewhat smlar method [3] known as the drect nverson of the teratve subspace (DIIS). Lke the Davdson method, DIIS apples drect methods to a small lnear algebra problem (now a system of lnear equatons nstead of an egenvalue problem) n a subspace 1
2 formed by takng a set of tral vectors from the fulldmensonal space. Pulay found that DIIS could be useful for acceleratng the convergence of selfconsstentfeld (SCF) procedures and, to a lesser extent, geometry optmzatons. The Mathematcs of DIIS Suppose that we have a set of tral vectors {p } whch have been generated durng the teratve soluton of a problem. Now let us form a set of resdual vectors defned as p = p +1 p. (1) The DIIS method assumes that a good approxmaton to the fnal soluton p f can be obtaned as a lnear combnaton of the prevous guess vectors p = c p, (2) where m s the number of prevous vectors (n practce, only the most recent few vectors are used). The coeffcents c are obtaned by requrng that the assocated resdual vector p = ( c ) p (3) approxmates the zero vector n a leastsquares sense. Furthermore, the coeffcents are requred to add to one, c = 1. (4) The motvaton for the latter requrement can be seen as follows. Each of our tral solutons p can be wrtten as the exact soluton plus an error term, p f + e. Then, the DIIS approxmate soluton s gven by p = ( c p f + e ) (5) = m p f c + c e. Hence, we wsh to mnmze the actual error, whch s the second term n the equaton above (of course, n practce, we don t know e, only p ); dong so would make the second term vansh, leavng only the frst term. For p = p f, we must have m c = 1. 2
3 Thus, we wsh to mnmze the norm of the resduum vector p p = c c j p p j, (6) j subject to the constrant (4). These requrements can be satsfed by mnmzng the followng functon wth Lagrangan multpler λ where B s the matrx of overlaps L = c Bc λ ( 1 ) c, (7) B j = p p j. (8) We can mnmze L wth respect to a coeffcent c k to obtan (assumng real quanttes) L = 0 = c j B kj + c B k λ (9) c k j = 2 c B k λ. We can absorb the factor of 2 nto λ to obtan the followng matrx equaton, whch s eq. (6) of Pulay [3]: B 11 B 12 B 1m 1 c 1 0 B 21 B 22 B 2m 1 c 2 0 = (10) B m1 B m2 B mm 1 c m λ 1 Programmng DIIS The DIIS procedure seems so smple that further comment on specfc computatonal mplementatons mght appear superfluous. However, I have found that the precse computatonal detals are absolutely crucal for effectve nterpolaton. Hence, I wll descrbe here my mplementaton of DIIS for the optmzaton of orbtals n a twostep CASSCF program. There are probably many varatons on ths mplementaton whch would also work, but often seemngly nconsequental chnages can make dramatc dfferences n effcency. In the twostep CASSCF procedure, one begns wth a set of guess orbtals, solves the full CI problem n the actve space, determnes the gradent for orbtal rotatons, takes a step n orbtal 3
4 rotaton (theta) space down the gradent drecton (.e., obtans new guess orbtals), and repeats the process teratvely untl convergence. To allow DIIS nterpolaton, one can express the current set of guess orbtals as the result of the multplcaton of a set of Gvens rotaton matrces by a matrx of reference orbtals (C µp = q C o µqu qp, see [4]). The rotaton angles whch defne the untary transformaton U (a product of Gvens rotaton matrces) comprse a vector of parameters, p. In ths case, one can defne the error vectors as the dfferences between subsequent sets of orbtal rotaton angles, or one could also reasonably choose the orbtal gradent vector. In my detcas program, the regular theta step s determned usng a NewtonRaphson approach wth an approxmate, dagonal orbtal Hessan. Ths s equvalent to scalng the orbtal gradent to a new coordnate system. Snce the step n theta space s just the scaled gradent, the scaled gradent s the same as the dfference between successve theta vectors (apart from a sgn) before the DIIS procedure starts. However, I have found t much better to assocate the gradent vector wth the next teraton s theta vector, not wth the theta vector from whch t was computed. In other words, t s best to change eq. (1) to the followng: p +1 = p +1 p. (11) Another general consderaton s that one does not want to add an nterpolated vector to the lst of vectors {p} unless t contans some new character to add to the subspace. Otherwse, lnear dependences can result. An outlne of my DIIS procedure for the detcas program s gven below: 1. Usng current orbtals p, obtan scaled orbtal gradent g. 2. Take NewtonRaphson step p +1 = p g. 3. Add p +1 to lst of vectors. Add p +1 = g to lst of error vectors. 4. Perform DIIS nterpolaton to obtan new guess vector. Overwrte p +1 wth DIIS nterpolant. (Ths vector wll never be added to lst of vectors). 5. Increment, begn new cycle. 4
5 References [1] E. R. Davdson, J. Comp. Phys. 17, 87 (1975). [2] J. A. Pople, R. Krshnan, H. B. Schlegel, and J. S. Bnkley, Int. J. Quantum Chem. Symp. 13, 225 (1979). [3] P. Pulay, Chem. Phys. Lett. 73, 393 (1980). [4] M. HeadGordon and J. A. Pople, J. Phys. Chem. 92, 3063 (1988). 5
PHYS 705: Classical Mechanics. Calculus of Variations II
1 PHYS 705: Classcal Mechancs Calculus of Varatons II 2 Calculus of Varatons: Generalzaton (no constrant yet) Suppose now that F depends on several dependent varables : We need to fnd such that has a statonary
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationVector Norms. Chapter 7 Iterative Techniques in Matrix Algebra. CauchyBunyakovskySchwarz Inequality for Sums. Distances. Convergence.
Vector Norms Chapter 7 Iteratve Technques n Matrx Algebra PerOlof Persson persson@berkeley.edu Department of Mathematcs Unversty of Calforna, Berkeley Math 128B Numercal Analyss Defnton A vector norm
More informationSingular Value Decomposition: Theory and Applications
Sngular Value Decomposton: Theory and Applcatons Danel Khashab Sprng 2015 Last Update: March 2, 2015 1 Introducton A = UDV where columns of U and V are orthonormal and matrx D s dagonal wth postve real
More informationMMA and GCMMA two methods for nonlinear optimization
MMA and GCMMA two methods for nonlnear optmzaton Krster Svanberg Optmzaton and Systems Theory, KTH, Stockholm, Sweden. krlle@math.kth.se Ths note descrbes the algorthms used n the author s 2007 mplementatons
More information8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS
SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 493 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces you have studed thus far n the text are real vector spaces because the scalars
More informationGeneralized Linear Methods
Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set
More informationCSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography
CSc 6974 and ECSE 6966 Math. Tech. for Vson, Graphcs and Robotcs Lecture 21, Aprl 17, 2006 Estmatng A Plane Homography Overvew We contnue wth a dscusson of the major ssues, usng estmaton of plane projectve
More informationU.C. Berkeley CS294: Beyond WorstCase Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond WorstCase Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationInexact Newton Methods for Inverse Eigenvalue Problems
Inexact Newton Methods for Inverse Egenvalue Problems Zhengjan Ba Abstract In ths paper, we survey some of the latest development n usng nexact Newtonlke methods for solvng nverse egenvalue problems.
More informationLectures  Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures  Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationSome modelling aspects for the Matlab implementation of MMA
Some modellng aspects for the Matlab mplementaton of MMA Krster Svanberg krlle@math.kth.se Optmzaton and Systems Theory Department of Mathematcs KTH, SE 10044 Stockholm September 2004 1. Consdered optmzaton
More informationSTAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 16
STAT 39: MATHEMATICAL COMPUTATIONS I FALL 218 LECTURE 16 1 why teratve methods f we have a lnear system Ax = b where A s very, very large but s ether sparse or structured (eg, banded, Toepltz, banded plus
More informationwhere the sums are over the partcle labels. In general H = p2 2m + V s(r ) V j = V nt (jr, r j j) (5) where V s s the snglepartcle potental and V nt
Physcs 543 Quantum Mechancs II Fall 998 HartreeFock and the Selfconsstent Feld Varatonal Methods In the dscusson of statonary perturbaton theory, I mentoned brey the dea of varatonal approxmaton schemes.
More information1 GSW Iterative Techniques for y = Ax
1 for y = A I m gong to cheat here. here are a lot of teratve technques that can be used to solve the general case of a set of smultaneous equatons (wrtten n the matr form as y = A), but ths chapter sn
More information1 Matrix representations of canonical matrices
1 Matrx representatons of canoncal matrces 2d rotaton around the orgn: ( ) cos θ sn θ R 0 = sn θ cos θ 3d rotaton around the xaxs: R x = 1 0 0 0 cos θ sn θ 0 sn θ cos θ 3d rotaton around the yaxs:
More informationChapter Newton s Method
Chapter 9. Newton s Method After readng ths chapter, you should be able to:. Understand how Newton s method s dfferent from the Golden Secton Search method. Understand how Newton s method works 3. Solve
More informationρ some λ THE INVERSE POWER METHOD (or INVERSE ITERATION) , for , or (more usually) to
THE INVERSE POWER METHOD (or INVERSE ITERATION)  applcaton of the Power method to A some fxed constant ρ (whch s called a shft), x λ ρ If the egenpars of A are { ( λ, x ) } ( ), or (more usually) to,
More informationReport on Image warping
Report on Image warpng Xuan Ne, Dec. 20, 2004 Ths document summarzed the algorthms of our mage warpng soluton for further study, and there s a detaled descrpton about the mplementaton of these algorthms.
More information4DVAR, according to the name, is a fourdimensional variational method.
4DVaratonal Data Assmlaton (4DVar) 4DVAR, accordng to the name, s a fourdmensonal varatonal method. 4DVar s actually a drect generalzaton of 3DVar to handle observatons that are dstrbuted n tme. The
More informationSalmon: Lectures on partial differential equations. Consider the general linear, secondorder PDE in the form. ,x 2
Salmon: Lectures on partal dfferental equatons 5. Classfcaton of secondorder equatons There are general methods for classfyng hgherorder partal dfferental equatons. One s very general (applyng even to
More informationQuantum Mechanics I  Session 4
Quantum Mechancs I  Sesson 4 Aprl 3, 05 Contents Operators Change of Bass 4 3 Egenvectors and Egenvalues 5 3. Denton....................................... 5 3. Rotaton n D....................................
More informationErrors for Linear Systems
Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons Â and ˆb avalable. Then the best thng we can do s to solve Âˆx ˆb exactly whch
More informationRelaxation Methods for Iterative Solution to Linear Systems of Equations
Relaxaton Methods for Iteratve Soluton to Lnear Systems of Equatons Gerald Recktenwald Portland State Unversty Mechancal Engneerng Department gerry@pdx.edu Overvew Techncal topcs Basc Concepts Statonary
More informationLecture 21: Numerical methods for pricing American type derivatives
Lecture 21: Numercal methods for prcng Amercan type dervatves Xaoguang Wang STAT 598W Aprl 10th, 2014 (STAT 598W) Lecture 21 1 / 26 Outlne 1 Fnte Dfference Method Explct Method Penalty Method (STAT 598W)
More informationSIO 224. m(r) =(ρ(r),k s (r),µ(r))
SIO 224 1. A bref look at resoluton analyss Here s some background for the Masters and Gubbns resoluton paper. Global Earth models are usually found teratvely by assumng a startng model and fndng small
More informationInner Product. Euclidean Space. Orthonormal Basis. Orthogonal
Inner Product Defnton 1 () A Eucldean space s a fntedmensonal vector space over the reals R, wth an nner product,. Defnton 2 (Inner Product) An nner product, on a real vector space X s a symmetrc, blnear,
More informationCME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 13
CME 30: NUMERICAL LINEAR ALGEBRA FALL 005/06 LECTURE 13 GENE H GOLUB 1 Iteratve Methods Very large problems (naturally sparse, from applcatons): teratve methods Structured matrces (even sometmes dense,
More informationPoisson brackets and canonical transformations
rof O B Wrght Mechancs Notes osson brackets and canoncal transformatons osson Brackets Consder an arbtrary functon f f ( qp t) df f f f q p q p t But q p p where ( qp ) pq q df f f f p q q p t In order
More informationA New Refinement of Jacobi Method for Solution of Linear System Equations AX=b
Int J Contemp Math Scences, Vol 3, 28, no 17, 819827 A New Refnement of Jacob Method for Soluton of Lnear System Equatons AX=b F Naem Dafchah Department of Mathematcs, Faculty of Scences Unversty of Gulan,
More informationn α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0
MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector
More informatione  c o m p a n i o n
OPERATIONS RESEARCH http://dxdoorg/0287/opre007ec e  c o m p a n o n ONLY AVAILABLE IN ELECTRONIC FORM 202 INFORMS Electronc Companon Generalzed Quantty Competton for Multple Products and Loss of Effcency
More informationYong Joon Ryang. 1. Introduction Consider the multicommodity transportation problem with convex quadratic cost function. 1 2 (x x0 ) T Q(x x 0 )
KangweonKyungk Math. Jour. 4 1996), No. 1, pp. 7 16 AN ITERATIVE ROWACTION METHOD FOR MULTICOMMODITY TRANSPORTATION PROBLEMS Yong Joon Ryang Abstract. The optmzaton problems wth quadratc constrants often
More informationCHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE
CHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE Analytcal soluton s usually not possble when exctaton vares arbtrarly wth tme or f the system s nonlnear. Such problems can be solved by numercal tmesteppng
More informationOutline and Reading. Dynamic Programming. Dynamic Programming revealed. Computing Fibonacci. The General Dynamic Programming Technique
Outlne and Readng Dynamc Programmng The General Technque ( 5.3.2) 1 Knapsac Problem ( 5.3.3) Matrx ChanProduct ( 5.3.1) Dynamc Programmng verson 1.4 1 Dynamc Programmng verson 1.4 2 Dynamc Programmng
More informationOPTIMISATION. Introduction Single Variable Unconstrained Optimisation Multivariable Unconstrained Optimisation Linear Programming
OPTIMIATION Introducton ngle Varable Unconstraned Optmsaton Multvarable Unconstraned Optmsaton Lnear Programmng Chapter Optmsaton /. Introducton In an engneerng analss, sometmes etremtes, ether mnmum or
More informationPreconditioning techniques in Chebyshev collocation method for elliptic equations
Precondtonng technques n Chebyshev collocaton method for ellptc equatons ZhWe Fang Je Shen HaWe Sun (n memory of late Professor Benyu Guo Abstract When one approxmates ellptc equatons by the spectral
More informationNorms, Condition Numbers, Eigenvalues and Eigenvectors
Norms, Condton Numbers, Egenvalues and Egenvectors 1 Norms A norm s a measure of the sze of a matrx or a vector For vectors the common norms are: N a 2 = ( x 2 1/2 the Eucldean Norm (1a b 1 = =1 N x (1b
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More informationThe Exact Formulation of the Inverse of the Tridiagonal Matrix for Solving the 1D Poisson Equation with the Finite Difference Method
Journal of Electromagnetc Analyss and Applcatons, 04, 6, 008 Publshed Onlne September 04 n ScRes. http://www.scrp.org/journal/jemaa http://dx.do.org/0.46/jemaa.04.6000 The Exact Formulaton of the Inverse
More informationSolution of Linear System of Equations and Matrix Inversion Gauss Seidel Iteration Method
Soluton of Lnear System of Equatons and Matr Inverson Gauss Sedel Iteraton Method It s another wellknown teratve method for solvng a system of lnear equatons of the form a + a22 + + ann = b a2 + a222
More information= = = (a) Use the MATLAB command rref to solve the system. (b) Let A be the coefficient matrix and B be the righthand side of the system.
Chapter Matlab Exercses Chapter Matlab Exercses. Consder the lnear system of Example n Secton.. x x x y z y y z (a) Use the MATLAB command rref to solve the system. (b) Let A be the coeffcent matrx and
More informationHongyi Miao, College of Science, Nanjing Forestry University, Nanjing ,China. (Received 20 June 2013, accepted 11 March 2014) I)ϕ (k)
ISSN 17493889 (prnt), 17493897 (onlne) Internatonal Journal of Nonlnear Scence Vol.17(2014) No.2,pp.188192 Modfed Block JacobDavdson Method for Solvng Large Sparse Egenproblems Hongy Mao, College of
More informationFinite Element Modelling of truss/cable structures
Pet Schreurs Endhoven Unversty of echnology Department of Mechancal Engneerng Materals echnology November 3, 214 Fnte Element Modellng of truss/cable structures 1 Fnte Element Analyss of prestressed structures
More informationC/CS/Phy191 Problem Set 3 Solutions Out: Oct 1, 2008., where ( 00. ), so the overall state of the system is ) ( ( ( ( 00 ± 11 ), Φ ± = 1
C/CS/Phy9 Problem Set 3 Solutons Out: Oct, 8 Suppose you have two qubts n some arbtrary entangled state ψ You apply the teleportaton protocol to each of the qubts separately What s the resultng state obtaned
More informationA Hybrid Variational Iteration Method for Blasius Equation
Avalable at http://pvamu.edu/aam Appl. Appl. Math. ISSN: 19329466 Vol. 10, Issue 1 (June 2015), pp. 223229 Applcatons and Appled Mathematcs: An Internatonal Journal (AAM) A Hybrd Varatonal Iteraton Method
More informationWorkshop: Approximating energies and wave functions Quantum aspects of physical chemistry
Workshop: Approxmatng energes and wave functons Quantum aspects of physcal chemstry http://quantum.bu.edu/pltl/6/6.pdf Last updated Thursday, November 7, 25 7:9:55: Copyrght 25 Dan Dll (dan@bu.edu) Department
More information2.3 Nilpotent endomorphisms
s a block dagonal matrx, wth A Mat dm U (C) In fact, we can assume that B = B 1 B k, wth B an ordered bass of U, and that A = [f U ] B, where f U : U U s the restrcton of f to U 40 23 Nlpotent endomorphsms
More informationFeb 14: Spatial analysis of data fields
Feb 4: Spatal analyss of data felds Mappng rregularly sampled data onto a regular grd Many analyss technques for geophyscal data requre the data be located at regular ntervals n space and/or tme. hs s
More information1 Convex Optimization
Convex Optmzaton We wll consder convex optmzaton problems. Namely, mnmzaton problems where the objectve s convex (we assume no constrants for now). Such problems often arse n machne learnng. For example,
More informationAPPENDIX A Some Linear Algebra
APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,
More informationProf. Dr. I. Nasser Phys 630, T Aug15 One_dimensional_Ising_Model
EXACT OEDIMESIOAL ISIG MODEL The onedmensonal Isng model conssts of a chan of spns, each spn nteractng only wth ts two nearest neghbors. The smple Isng problem n one dmenson can be solved drectly n several
More informationIntroduction to Density Functional Theory. Jeremie Zaffran 2 nd yearmsc. (Nanochemistry)
Introducton to Densty Functonal Theory Jereme Zaffran nd yearmsc. (anochemstry) A Hartree appromatons Born Oppenhemer appromaton H H H e The goal of computatonal chemstry H e??? Let s remnd H e T e
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More informationCurve Fitting with the Least Square Method
WIKI Document Number 5 Interpolaton wth Least Squares Curve Fttng wth the Least Square Method Mattheu Bultelle Department of BoEngneerng Imperal College, London Context We wsh to model the postve feedback
More informationStructure and Drive Paul A. Jensen Copyright July 20, 2003
Structure and Drve Paul A. Jensen Copyrght July 20, 2003 A system s made up of several operatons wth flow passng between them. The structure of the system descrbes the flow paths from nputs to outputs.
More informationIntroduction to Vapor/Liquid Equilibrium, part 2. Raoult s Law:
CE304, Sprng 2004 Lecture 4 Introducton to Vapor/Lqud Equlbrum, part 2 Raoult s Law: The smplest model that allows us do VLE calculatons s obtaned when we assume that the vapor phase s an deal gas, and
More informationFeature Selection: Part 1
CSE 546: Machne Learnng Lecture 5 Feature Selecton: Part 1 Instructor: Sham Kakade 1 Regresson n the hgh dmensonal settng How do we learn when the number of features d s greater than the sample sze n?
More informationA SEPARABLE APPROXIMATION DYNAMIC PROGRAMMING ALGORITHM FOR ECONOMIC DISPATCH WITH TRANSMISSION LOSSES. Pierre HANSEN, Nenad MLADENOVI]
Yugoslav Journal of Operatons Research (00) umber 5766 A SEPARABLE APPROXIMATIO DYAMIC PROGRAMMIG ALGORITHM FOR ECOOMIC DISPATCH WITH TRASMISSIO LOSSES Perre HASE enad MLADEOVI] GERAD and Ecole des Hautes
More informationLecture 12: Discrete Laplacian
Lecture 12: Dscrete Laplacan Scrbe: Tanye Lu Our goal s to come up wth a dscrete verson of Laplacan operator for trangulated surfaces, so that we can use t n practce to solve related problems We are mostly
More informationInternational Journal of Pure and Applied Sciences and Technology
Int. J. Pure Appl. Sc. Technol., 4() (03), pp. 530 Internatonal Journal of Pure and Appled Scences and Technology ISSN 9607 Avalable onlne at www.jopaasat.n Research Paper Schrödnger State Space Matrx
More informationConvexity preserving interpolation by splines of arbitrary degree
Computer Scence Journal of Moldova, vol.18, no.1(52), 2010 Convexty preservng nterpolaton by splnes of arbtrary degree Igor Verlan Abstract In the present paper an algorthm of C 2 nterpolaton of dscrete
More informationDifference Equations
Dfference Equatons c Jan Vrbk 1 Bascs Suppose a sequence of numbers, say a 0,a 1,a,a 3,... s defned by a certan general relatonshp between, say, three consecutve values of the sequence, e.g. a + +3a +1
More informationPRECONDITIONING TECHNIQUES IN CHEBYSHEV COLLOCATION METHOD FOR ELLIPTIC EQUATIONS
ITERATIOAL JOURAL OF UMERICAL AALYSIS AD MODELIG Volume 15 umber 12 Pages 277 287 c 2018 Insttute for Scentfc Computng and Informaton PRECODITIOIG TECHIQUES I CHEBYSHEV COLLOCATIO METHOD FOR ELLIPTIC
More informationBOUNDEDNESS OF THE RIESZ TRANSFORM WITH MATRIX A 2 WEIGHTS
BOUNDEDNESS OF THE IESZ TANSFOM WITH MATIX A WEIGHTS Introducton Let L = L ( n, be the functon space wth norm (ˆ f L = f(x C dx d < For a d d matrx valued functon W : wth W (x postve semdefnte for all
More informationSTAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 17. a ij x (k) b i. a ij x (k+1) (D + L)x (k+1) = b Ux (k)
STAT 309: MATHEMATICAL COMPUTATIONS I FALL 08 LECTURE 7. sor method remnder: n coordnatewse form, Jacob method s = [ b a x (k) a and Gauss Sedel method s = [ b a = = remnder: n matrx form, Jacob method
More informationMultigrid Methods and Applications in CFD
Multgrd Metods and Applcatons n CFD Mcael Wurst 0 May 009 Contents Introducton Typcal desgn of CFD solvers 3 Basc metods and ter propertes for solvng lnear systems of equatons 4 Geometrc Multgrd 3 5 Algebrac
More informationAn Interactive Optimisation Tool for Allocation Problems
An Interactve Optmsaton ool for Allocaton Problems Fredr Bonäs, Joam Westerlund and apo Westerlund Process Desgn Laboratory, Faculty of echnology, Åbo Aadem Unversty, uru 20500, Fnland hs paper presents
More informationCHAPTER 14 GENERAL PERTURBATION THEORY
CHAPTER 4 GENERAL PERTURBATION THEORY 4 Introducton A partcle n orbt around a pont mass or a sphercally symmetrc mass dstrbuton s movng n a gravtatonal potental of the form GM / r In ths potental t moves
More informationAdvanced Quantum Mechanics
Advanced Quantum Mechancs Rajdeep Sensarma! sensarma@theory.tfr.res.n ecture #9 QM of Relatvstc Partcles Recap of ast Class Scalar Felds and orentz nvarant actons Complex Scalar Feld and Charge conjugaton
More informationA Local Variational Problem of Second Order for a Class of Optimal Control Problems with Nonsmooth Objective Function
A Local Varatonal Problem of Second Order for a Class of Optmal Control Problems wth Nonsmooth Objectve Functon Alexander P. Afanasev Insttute for Informaton Transmsson Problems, Russan Academy of Scences,
More informationEEL 6266 Power System Operation and Control. Chapter 3 Economic Dispatch Using Dynamic Programming
EEL 6266 Power System Operaton and Control Chapter 3 Economc Dspatch Usng Dynamc Programmng Pecewse Lnear Cost Functons Common practce many utltes prefer to represent ther generator cost functons as sngle
More information5.04, Principles of Inorganic Chemistry II MIT Department of Chemistry Lecture 32: Vibrational Spectroscopy and the IR
5.0, Prncples of Inorganc Chemstry II MIT Department of Chemstry Lecture 3: Vbratonal Spectroscopy and the IR Vbratonal spectroscopy s confned to the 005000 cm  spectral regon. The absorpton of a photon
More informationMATH Sensitivity of Eigenvalue Problems
MATH 537 Senstvty of Egenvalue Problems Prelmnares Let A be an n n matrx, and let λ be an egenvalue of A, correspondngly there are vectors x and y such that Ax = λx and y H A = λy H Then x s called A
More informationCollege of Computer & Information Science Fall 2009 Northeastern University 20 October 2009
College of Computer & Informaton Scence Fall 2009 Northeastern Unversty 20 October 2009 CS7880: Algorthmc Power Tools Scrbe: Jan Wen and Laura Poplawsk Lecture Outlne: Prmaldual schema Network Desgn:
More informationThe KMO Method for Solving Nonhomogenous, m th Order Differential Equations
The KMO Method for Solvng Nonhomogenous, m th Order Dfferental Equatons Davd Krohn Danel MarñoJohnson John Paul Ouyang March 14, 2013 Abstract Ths paper shows a smple tabular procedure for fndng the
More informationOn a direct solver for linear least squares problems
ISSN 20666594 Ann. Acad. Rom. Sc. Ser. Math. Appl. Vol. 8, No. 2/2016 On a drect solver for lnear least squares problems Constantn Popa Abstract The Null Space (NS) algorthm s a drect solver for lnear
More informationLecture 14: Forces and Stresses
The Nuts and Bolts of FrstPrncples Smulaton Lecture 14: Forces and Stresses Durham, 6th13th December 2001 CASTEP Developers Group wth support from the ESF ψ k Network Overvew of Lecture Why bother? Theoretcal
More informationNumerical Heat and Mass Transfer
Master degree n Mechancal Engneerng Numercal Heat and Mass Transfer 06FnteDfference Method (Onedmensonal, steady state heat conducton) Fausto Arpno f.arpno@uncas.t Introducton Why we use models and
More informationGlobal Sensitivity. Tuesday 20 th February, 2018
Global Senstvty Tuesday 2 th February, 28 ) Local Senstvty Most senstvty analyses [] are based on local estmates of senstvty, typcally by expandng the response n a Taylor seres about some specfc values
More informationCS4495/6495 Introduction to Computer Vision. 3CL3 Calibrating cameras
CS4495/6495 Introducton to Computer Vson 3CL3 Calbratng cameras Fnally (last tme): Camera parameters Projecton equaton the cumulatve effect of all parameters: M (3x4) f s x ' 1 0 0 0 c R 0 I T 3 3 3 x1
More informationSolutions to exam in SF1811 Optimization, Jan 14, 2015
Solutons to exam n SF8 Optmzaton, Jan 4, 25 3 3 OO 4 \ / \ / The network: \/ where all lnks go from left to rght. /\ / \ / \ 6 OO 5 2 4.(a) Let x = ( x 3, x 4, x 23, x 24 ) T, where the varable
More informationThe Prncpal Component Transform The Prncpal Component Transform s also called KarhunenLoeve Transform (KLT, Hotellng Transform, oregenvector Transfor
Prncpal Component Transform Multvarate Random Sgnals A real tme sgnal x(t can be consdered as a random process and ts samples x m (m =0; ;N, 1 a random vector: The mean vector of X s X =[x0; ;x N,1] T
More informationMean Field / Variational Approximations
Mean Feld / Varatonal Appromatons resented by Jose Nuñez 0/24/05 Outlne Introducton Mean Feld Appromaton Structured Mean Feld Weghted Mean Feld Varatonal Methods Introducton roblem: We have dstrbuton but
More informationImplicit Integration Henyey Method
Implct Integraton Henyey Method In realstc stellar evoluton codes nstead of a drect ntegraton usng for example the RungeKutta method one employs an teratve mplct technque. Ths s because the structure
More informationEEE 241: Linear Systems
EEE : Lnear Systems Summary #: Backpropagaton BACKPROPAGATION The perceptron rule as well as the Wdrow Hoff learnng were desgned to tran sngle layer networks. They suffer from the same dsadvantage: they
More informationPhysics 5153 Classical Mechanics. D Alembert s Principle and The Lagrangian1
P. Guterrez Physcs 5153 Classcal Mechancs D Alembert s Prncple and The Lagrangan 1 Introducton The prncple of vrtual work provdes a method of solvng problems of statc equlbrum wthout havng to consder the
More informationPhysics 5153 Classical Mechanics. Principle of Virtual Work1
P. Guterrez 1 Introducton Physcs 5153 Classcal Mechancs Prncple of Vrtual Work The frst varatonal prncple we encounter n mechancs s the prncple of vrtual work. It establshes the equlbrum condton of a mechancal
More informationFall 2012 Analysis of Experimental Measurements B. Eisenstein/rev. S. Errede
Fall 0 Analyss of Expermental easurements B. Esensten/rev. S. Errede We now reformulate the lnear Least Squares ethod n more general terms, sutable for (eventually extendng to the nonlnear case, and also
More informationPoint cloud to point cloud rigid transformations. Minimizing Rigid Registration Errors
Pont cloud to pont cloud rgd transformatons Russell Taylor 600.445 1 600.445 Fall 000015 Mnmzng Rgd Regstraton Errors Typcally, gven a set of ponts {a } n one coordnate system and another set of ponts
More informationA particle in a state of uniform motion remain in that state of motion unless acted upon by external force.
The fundamental prncples of classcal mechancs were lad down by Galleo and Newton n the 16th and 17th centures. In 1686, Newton wrote the Prncpa where he gave us three laws of moton, one law of gravty,
More informationOn the Multicriteria Integer Network Flow Problem
BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 5, No 2 Sofa 2005 On the Multcrtera Integer Network Flow Problem Vassl Vasslev, Marana Nkolova, Maryana Vassleva Insttute of
More informationLagrangian Field Theory
Lagrangan Feld Theory Adam Lott PHY 391 Aprl 6, 017 1 Introducton Ths paper s a summary of Chapter of Mandl and Shaw s Quantum Feld Theory [1]. The frst thng to do s to fx the notaton. For the most part,
More informationGrover s Algorithm + Quantum Zeno Effect + Vaidman
Grover s Algorthm + Quantum Zeno Effect + Vadman CS 2942 Bomb 10/12/04 Fall 2004 Lecture 11 Grover s algorthm Recall that Grover s algorthm for searchng over a space of sze wors as follows: consder the
More informationIntegrals and Invariants of EulerLagrange Equations
Lecture 16 Integrals and Invarants of EulerLagrange Equatons ME 256 at the Indan Insttute of Scence, Bengaluru Varatonal Methods and Structural Optmzaton G. K. Ananthasuresh Professor, Mechancal Engneerng,
More informationA linear imaging system with white additive Gaussian noise on the observed data is modeled as follows:
Supplementary Note Mathematcal bacground A lnear magng system wth whte addtve Gaussan nose on the observed data s modeled as follows: X = R ϕ V + G, () where X R are the expermental, twodmensonal proecton
More informationTopic 5: NonLinear Regression
Topc 5: NonLnear Regresson The models we ve worked wth so far have been lnear n the parameters. They ve been of the form: y = Xβ + ε Many models based on economc theory are actually nonlnear n the parameters.
More informationProblem Set 9 Solutions
Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem
More informationMEM 255 Introduction to Control Systems Review: Basics of Linear Algebra
MEM 255 Introducton to Control Systems Revew: Bascs of Lnear Algebra Harry G. Kwatny Department of Mechancal Engneerng & Mechancs Drexel Unversty Outlne Vectors Matrces MATLAB Advanced Topcs Vectors A
More informationLeast squares cubic splines without Bsplines S.K. Lucas
Least squares cubc splnes wthout Bsplnes S.K. Lucas School of Mathematcs and Statstcs, Unversty of South Australa, Mawson Lakes SA 595 emal: stephen.lucas@unsa.edu.au Submtted to the Gazette of the Australan
More information