CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 13
|
|
- Merilyn Hodge
- 5 years ago
- Views:
Transcription
1 CME 30: NUMERICAL LINEAR ALGEBRA FALL 005/06 LECTURE 13 GENE H GOLUB 1 Iteratve Methods Very large problems (naturally sparse, from applcatons): teratve methods Structured matrces (even sometmes dense, but use teratve technques) Gven Ax = b, wrte Mx = Nx + b and construct the teraton Subtractng these equatons, we obtan Mx (k+1) = Nx (k) + b M(x x (k+1) ) = N(x x (k) ) Therefore f we denote the error n x (k) by e (k) = x x (k), then e (k+1) = M 1 Ne (k) Be (k) Thus e (k) = B k e (0), whch suggests the followng theorem: Theorem e (k) 0 as k for all e (0) f and only f ρ(b) < 1 Convergence can stll occur f ρ(b) = 1, but n that case we must be careful n how we choose x (0) Note that from e (k) = B k e (0), t follows that e (k) e (0) B k The Jacob Method We now develop a smple teratve method If we rewrte Ax = b as n a x = b, = 1,, n, =1 then x = b a x, or x = 1 [ b a x ]
2 In other words, a 11 M =, a nn 0 a 1 a 1n N = a 1 a n1 a n,n 1 0 Our teraton s therefore x (k+1) = 1 [ b a x (k)], known as the Jacob method, wth a 0 1 a 11 M 1 a 1 N = a a n1 a a nn n,n 1 a nn 0 a 1n a 11 B J So, f M 1 N = max 1 n a < 1, e f B J s strctly dagonally domnant, then the teraton converges For example, suppose A = Then B J = 1, so the Jacob method converges rapdly On the other hand, f 1 1 A = 1, 1 whch arses from dscretzng the Laplacan, then B J = 1 A more subtle analyss can be used to show convergence, but t s slow Note that for these two examples, x (0) x (1) when all elements of x (1) have been computed Ths s a waste of storage; we need only n + elements of storage of A above Ths shows that the orderng of equatons s very mportant If we reorder the equatons n such a way that oddnumbered equatons and even-numbered equatons are grouped separately, then we obtan, for the
3 latter example, A = Then, we can solve for all odd ndces, then all even ndces, ndependently of each other Not only does ths approach save storage space but t also lends tself to parallelsm 3 The Gauss-Sedel Method In the Jacob method, we compute x (k+1) usng the elements of x (k), even though x (k+1) 1,, x (k+1) 1 are already known The Gauss-Sedel method s desgned to take advantage of the latest nformaton avalable about x: x (k+1) = 1 [ 1 n ] b a x (k+1) a x (k) =1 =+1 To derve ths method, we wrte A = L + D + U where 0 a 11 0 a 1 a 1n a L = 1, D =, U = an 1,n a n1 a n,n 1 0 a nn 0 Thus the Gauss-Sedel teraton can be wrtten as or whch yelds Dx (k+1) = b Lx (k+1) Ux (k), (D + L)x (k+1) = b Ux (k) x (k+1) = (D + L) 1 Ux (k) + (D + L) 1 b Thus the teraton matrx for the Gauss-Sedel method s B GS = (D + L) 1 U, as opposed to the teraton matrx for the Jacob method, B J = D 1 (L + U) In some cases, ρ(b GS ) = (ρ(b J )), so the Gauss-Sedel method converges twce as fast On the other hand, note that Gauss-Sedel s very sequental; e t does not lend tself to parallelsm 4 Posson s Equaton Consder the standard problem of solvng Posson s equaton on a doman R n two dmensons, u = f, (x, y) R, u = u xx + u yy, u = g, (x, y) R We take R to be the unt rectangle [0, 1] [0, 1] and dscretze the problem usng a unform grd wth spacng h = 1/(N + 1) n the x and y drectons, and grdponts x = h, = 0,, N + 1, 3
4 and y = h, = 0,, N + 1 Then, for, = 1,, N, we replace the dfferental equaton by a dfference approxmaton u 1, + u u +1, h + u,+1 + u u,+1 h = f, where u = u(x, y ) and f = f(x, y ) From the boundary condtons, we have u 0 = g(x 0, y ), = 1,,, N, and smlar condtons for the other grdponts along the boundary Let u = [u 1,, u N ] Then where u 1 + T u u +1 = f h f 1 + g(x 0, ) = 1 T = 1, [ f ] = h f =,, N 1 h f N + g(x N, ) = N 14 Thus we can solve the problem on the entre doman by solvng Au = f where T I I T I A = I I T We say that A s a block trdagonal matrx A s also a band matrx, but the band s sparse and Gaussan elmnaton may fll-n the whole band However, the equatons can be re-ordered to avod fll-n Department of Computer Scence, Gates Buldng B, Room 80, Stanford, CA E-mal address: golub@stanfordedu 4
5 CME 30: NUMERICAL LINEAR ALGEBRA FALL 005/06 LECTURE 15 GENE H GOLUB 1 Convergence of Iteratve Methods Recall the basc teratve methods based on the splttng A = D + L + U, the Jacob method and the Gauss-Sedel method Dx (k+1) = (L + U)x (k) + b (D + L)x (k+1) = Ux (k) + b These are examples of one-step statonary method, whch s an teraton of the form Mx (k+1) = Nx (k) + b, where A = M N Let B = M 1 N, and defne e (k) = x x (k) Then e (k+1) = Be (k) = B k+1 e (0) Recall that f ρ(b k ) < 1 then e (k) 0 for all choces of x (0) Also, recall that for all consstent norms, ρ(b) B Therefore, a suffcent condton for convergence of the Jacob method s B < 1 where, b = 0 = Note that f B s dagonally domnant Now, defne Then we have the followng result: B = max r = a, a < 1 r = max r Theorem If r < 1, then ρ(b GS ) < 1 In other words, the Gauss-Sedel teraton converges f A s dagonally domnant Proof The proof proceeds usng nducton on the elements of e (k) We have whch can be wrtten as =1 a e (k+1) (D + L)e (k+1) = Ue (k), = =+1 a e (k), = 1,, N
6 Thus For = 1, we have e (k+1) = Assume that for p = 1,, 1, Then, Therefore from whch t follows that snce r < 1 =+1 e (k+1) 1 a e (k) = 1 =1 a e(k) a e (k+1), = 1,, N e (k) r 1 e (k+1) p e (k) r p r e (k) e (k+1) 1 =1 a e(k+1) 1 r e (k) + =1 e (k) = r e (k) r e (k) a a =+1 + e a e(k) =+1 e (k+1) r e (k) r k+1 e (0), lm k e(k) = 0 We see that the Jacob method and the Gauss-Sedel method both converge f A s dagonally domnant, but convergence can be slow n some cases For example, f 1 1 A = 1 1 s of sze N N then 0 1/ D 1 1/ (L + U) = 1/ 1/ 0 and therefore π ρ(b J ) = cos N + 1 = cos πh 1 π h + whch s approxmately 1 for small h = 1 N+1 We would lke to develop a method where ρ(b) 1 ch Now, suppose B = B Then e (k) e (0) B k = ρ(b) k a
7 We want e (k) / e (0) ɛ, so f we let ρ k = ɛ, then k = log ɛ log ρ s the number of teratons necessary for convergence The quantty log ρ s called the rate of convergence The SOR Method The method of successve overrelaxaton (SOR) s the teraton = ω 1 [b a x (k+1) x (k+1) =1 =+1 ] a x (k) + (1 ω)x (k) The quantty ω s called the relaxaton parameter If ω = 1, then the SOR method reduces to the Gauss-Sedel method In matrx form, the teraton can be wrtten as whch can be rearranged to obtan or Defne Then x (k+1) = Dx (k+1) = ω(b Lx (k+1) Ux (k) ) + (1 ω)dx (k) (D + ωl)x (k+1) = ωb + [(1 ω)d ωu]x (k) ( ) 1 1 [( ) ] ( ) ω D + L ω 1 D U x (k) + ω D + L b L ω = ( ) 1 1 [( ) ] 1 ω D + L ω 1 D U ( ) 1 1 [( 1 det L ω = det ω D + L det = 1 det ( 1 det ω D + L) ω n = n =1 = (1 ω) n [( 1 ω 1 (1 ω) n n =1 ω n ) ω 1 ) ] D U ] D U Therefore, n =1 λ = (1 ω) n where λ 1,, λ n are the egenvalues of L ω, wth λ 1 λ n Therefore λ 1 n (1 ω) n Snce we must have λ 1 < 1 for convergence, t follows that a necessary condton for convergence of SOR s 0 < ω < 3 Block Methods Recall that n solvng Posson s equaton on a rectangle, we needed to solve systems of the form Ths can be accomplshed usng an teraton v + T v v +1 = g T v (k+1) = g + v (k) 1 + v(k) +1, 3
8 whch s an example of a block Jacob teraton, snce t nvolves solvng the system Au = g by applyng the Jacob method to A, except each block of sze N N s treated as a sngle element Smlarly, we can use the block Gauss-Sedel teraton Consder the teraton T v (k+1) = g + v (k+1) 1 + v (k) 4 Rchardson Method x (k+1) = (I αa)x (k) + αb = x (k) + α(b Ax (k) ) = x (k) + αr (k) Ths s known as the Rchardson method If we defne the error e (k) = x x (k), then e (k+1) = B α e (k) where B α = I αa; we want to choose the parameter α a pror so as to mnmze B α Suppose A s symmetrc postve defnte, wth egenvalues µ 1 µ µ n > 0 Snce B = I αa, λ = 1 αµ We want to choose α so that B α s mnmzed; e mn α The optmal parameter ˆα s found by solvng max λ (α) = mn max 1 αµ 1 n α 1 n 1 ˆαµ n = (1 ˆαµ 1 ) whch yelds ˆα = µ 1 + µ n Note that When 1 αµ n = 1 that the teraton dverges, from whch t follows that the method converges for 0 < α < /µ n However, ths teraton s senstve to perturbaton, and therefore bad numercally For example, f µ 1 = 10 and µ n = 10 4, then the optmal α s /( , but ths s close to a value of α for whch the teraton dverges, α = /10 Also, note that and smlarly, λ 1 (ˆα) = 1 λ n (ˆα) = µ 1 µ n µ 1 + µ n = µ 1 + µ n µ 1 = µ n µ 1 µ 1 + µ n, µ 1 µ n 1 µ 1 µ n + 1 = κ(a) 1 κ(a) + 1 Therefore the convergence rate depends on κ(a) For example, consder the Helmholtz equaton on a rectangle R, u (k+1) + σ(x, y)u (k) = f, u = g, Usng a fnte dfference approxmaton for gves T I I A = I I T and thus the teraton has the form Au (k+1) + h Σu (k) = f (x, y) R (x, y) R 4
9 where Σ = σ 11 σ nn, σ = σ(x, y ) We wsh to determne the rate of convergence We defne the error operator by Therefore But and Therefore e (k+1) = (h A 1 Σ)e (k) e (k+1) h A 1 Σ e (k) Σ = max σ, λ mn = 4 4 cos πh = 4(1 cos πh) ( ) πh = 8 sn e (k+1) max, σ ) e max, σ π e (k) ( sn xh/ h/ and thus the sze of the problem mesh has dsappeared, and the method converges f max, σ 0 The rate of convergence s essentally ndependent of h, whch s very desrable Department of Computer Scence, Gates Buldng B, Room 80, Stanford, CA E-mal address: golub@stanfordedu 5
10 Problem 1 A = gallery( posson, n) returns the block trdagonal matrx A of order n resultng from dscretzng Posson s equaton wth the 5-pont operator on an n- by-n mesh Choose b so that Ax = b has soluton vector x of all 1 s For n =5, 10, approxmate x by usng the Jacob, Gauss-Sedel, and SOR wth optmal ω Problem Do the same procedure for Hlbert matrx H of sze n =50, 100
STAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 17. a ij x (k) b i. a ij x (k+1) (D + L)x (k+1) = b Ux (k)
STAT 309: MATHEMATICAL COMPUTATIONS I FALL 08 LECTURE 7. sor method remnder: n coordnatewse form, Jacob method s = [ b a x (k) a and Gauss Sedel method s = [ b a = = remnder: n matrx form, Jacob method
More informationSTAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 16
STAT 39: MATHEMATICAL COMPUTATIONS I FALL 218 LECTURE 16 1 why teratve methods f we have a lnear system Ax = b where A s very, very large but s ether sparse or structured (eg, banded, Toepltz, banded plus
More informationRelaxation Methods for Iterative Solution to Linear Systems of Equations
Relaxaton Methods for Iteratve Soluton to Lnear Systems of Equatons Gerald Recktenwald Portland State Unversty Mechancal Engneerng Department gerry@pdx.edu Overvew Techncal topcs Basc Concepts Statonary
More informationVector Norms. Chapter 7 Iterative Techniques in Matrix Algebra. Cauchy-Bunyakovsky-Schwarz Inequality for Sums. Distances. Convergence.
Vector Norms Chapter 7 Iteratve Technques n Matrx Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematcs Unversty of Calforna, Berkeley Math 128B Numercal Analyss Defnton A vector norm
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationA New Refinement of Jacobi Method for Solution of Linear System Equations AX=b
Int J Contemp Math Scences, Vol 3, 28, no 17, 819-827 A New Refnement of Jacob Method for Soluton of Lnear System Equatons AX=b F Naem Dafchah Department of Mathematcs, Faculty of Scences Unversty of Gulan,
More informationρ some λ THE INVERSE POWER METHOD (or INVERSE ITERATION) , for , or (more usually) to
THE INVERSE POWER METHOD (or INVERSE ITERATION) -- applcaton of the Power method to A some fxed constant ρ (whch s called a shft), x λ ρ If the egenpars of A are { ( λ, x ) } ( ), or (more usually) to,
More informationLecture 21: Numerical methods for pricing American type derivatives
Lecture 21: Numercal methods for prcng Amercan type dervatves Xaoguang Wang STAT 598W Aprl 10th, 2014 (STAT 598W) Lecture 21 1 / 26 Outlne 1 Fnte Dfference Method Explct Method Penalty Method (STAT 598W)
More informationLecture 12: Discrete Laplacian
Lecture 12: Dscrete Laplacan Scrbe: Tanye Lu Our goal s to come up wth a dscrete verson of Laplacan operator for trangulated surfaces, so that we can use t n practce to solve related problems We are mostly
More information[7] R.S. Varga, Matrix Iterative Analysis, Prentice-Hall, Englewood Clis, New Jersey, (1962).
[7] R.S. Varga, Matrx Iteratve Analyss, Prentce-Hall, Englewood ls, New Jersey, (962). [8] J. Zhang, Multgrd soluton of the convecton-duson equaton wth large Reynolds number, n Prelmnary Proceedngs of
More informationThe Exact Formulation of the Inverse of the Tridiagonal Matrix for Solving the 1D Poisson Equation with the Finite Difference Method
Journal of Electromagnetc Analyss and Applcatons, 04, 6, 0-08 Publshed Onlne September 04 n ScRes. http://www.scrp.org/journal/jemaa http://dx.do.org/0.46/jemaa.04.6000 The Exact Formulaton of the Inverse
More informationErrors for Linear Systems
Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch
More informationDUE: WEDS FEB 21ST 2018
HOMEWORK # 1: FINITE DIFFERENCES IN ONE DIMENSION DUE: WEDS FEB 21ST 2018 1. Theory Beam bendng s a classcal engneerng analyss. The tradtonal soluton technque makes smplfyng assumptons such as a constant
More informationAppendix B. The Finite Difference Scheme
140 APPENDIXES Appendx B. The Fnte Dfference Scheme In ths appendx we present numercal technques whch are used to approxmate solutons of system 3.1 3.3. A comprehensve treatment of theoretcal and mplementaton
More information2 Finite difference basics
Numersche Methoden 1, WS 11/12 B.J.P. Kaus 2 Fnte dfference bascs Consder the one- The bascs of the fnte dfference method are best understood wth an example. dmensonal transent heat conducton equaton T
More informationMultigrid Methods and Applications in CFD
Multgrd Metods and Applcatons n CFD Mcael Wurst 0 May 009 Contents Introducton Typcal desgn of CFD solvers 3 Basc metods and ter propertes for solvng lnear systems of equatons 4 Geometrc Multgrd 3 5 Algebrac
More informationOne-sided finite-difference approximations suitable for use with Richardson extrapolation
Journal of Computatonal Physcs 219 (2006) 13 20 Short note One-sded fnte-dfference approxmatons sutable for use wth Rchardson extrapolaton Kumar Rahul, S.N. Bhattacharyya * Department of Mechancal Engneerng,
More informationwhere the sums are over the partcle labels. In general H = p2 2m + V s(r ) V j = V nt (jr, r j j) (5) where V s s the sngle-partcle potental and V nt
Physcs 543 Quantum Mechancs II Fall 998 Hartree-Fock and the Self-consstent Feld Varatonal Methods In the dscusson of statonary perturbaton theory, I mentoned brey the dea of varatonal approxmaton schemes.
More information1 GSW Iterative Techniques for y = Ax
1 for y = A I m gong to cheat here. here are a lot of teratve technques that can be used to solve the general case of a set of smultaneous equatons (wrtten n the matr form as y = A), but ths chapter sn
More informationEEE 241: Linear Systems
EEE : Lnear Systems Summary #: Backpropagaton BACKPROPAGATION The perceptron rule as well as the Wdrow Hoff learnng were desgned to tran sngle layer networks. They suffer from the same dsadvantage: they
More informationInner Product. Euclidean Space. Orthonormal Basis. Orthogonal
Inner Product Defnton 1 () A Eucldean space s a fnte-dmensonal vector space over the reals R, wth an nner product,. Defnton 2 (Inner Product) An nner product, on a real vector space X s a symmetrc, blnear,
More informationSome Comments on Accelerating Convergence of Iterative Sequences Using Direct Inversion of the Iterative Subspace (DIIS)
Some Comments on Acceleratng Convergence of Iteratve Sequences Usng Drect Inverson of the Iteratve Subspace (DIIS) C. Davd Sherrll School of Chemstry and Bochemstry Georga Insttute of Technology May 1998
More informationInexact Newton Methods for Inverse Eigenvalue Problems
Inexact Newton Methods for Inverse Egenvalue Problems Zheng-jan Ba Abstract In ths paper, we survey some of the latest development n usng nexact Newton-lke methods for solvng nverse egenvalue problems.
More informationSingular Value Decomposition: Theory and Applications
Sngular Value Decomposton: Theory and Applcatons Danel Khashab Sprng 2015 Last Update: March 2, 2015 1 Introducton A = UDV where columns of U and V are orthonormal and matrx D s dagonal wth postve real
More informationNumerical Heat and Mass Transfer
Master degree n Mechancal Engneerng Numercal Heat and Mass Transfer 06-Fnte-Dfference Method (One-dmensonal, steady state heat conducton) Fausto Arpno f.arpno@uncas.t Introducton Why we use models and
More information1 Matrix representations of canonical matrices
1 Matrx representatons of canoncal matrces 2-d rotaton around the orgn: ( ) cos θ sn θ R 0 = sn θ cos θ 3-d rotaton around the x-axs: R x = 1 0 0 0 cos θ sn θ 0 sn θ cos θ 3-d rotaton around the y-axs:
More informationP A = (P P + P )A = P (I P T (P P ))A = P (A P T (P P )A) Hence if we let E = P T (P P A), We have that
Backward Error Analyss for House holder Reectors We want to show that multplcaton by householder reectors s backward stable. In partcular we wsh to show fl(p A) = P (A) = P (A + E where P = I 2vv T s the
More informationCSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography
CSc 6974 and ECSE 6966 Math. Tech. for Vson, Graphcs and Robotcs Lecture 21, Aprl 17, 2006 Estmatng A Plane Homography Overvew We contnue wth a dscusson of the major ssues, usng estmaton of plane projectve
More informationAPPENDIX A Some Linear Algebra
APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationMMA and GCMMA two methods for nonlinear optimization
MMA and GCMMA two methods for nonlnear optmzaton Krster Svanberg Optmzaton and Systems Theory, KTH, Stockholm, Sweden. krlle@math.kth.se Ths note descrbes the algorthms used n the author s 2007 mplementatons
More informationDeriving the X-Z Identity from Auxiliary Space Method
Dervng the X-Z Identty from Auxlary Space Method Long Chen Department of Mathematcs, Unversty of Calforna at Irvne, Irvne, CA 92697 chenlong@math.uc.edu 1 Iteratve Methods In ths paper we dscuss teratve
More informationfor Linear Systems With Strictly Diagonally Dominant Matrix
MATHEMATICS OF COMPUTATION, VOLUME 35, NUMBER 152 OCTOBER 1980, PAGES 1269-1273 On an Accelerated Overrelaxaton Iteratve Method for Lnear Systems Wth Strctly Dagonally Domnant Matrx By M. Madalena Martns*
More informationLectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationComputational Astrophysics
Computatonal Astrophyscs Solvng for Gravty Alexander Knebe, Unversdad Autonoma de Madrd Computatonal Astrophyscs Solvng for Gravty the equatons full set of equatons collsonless matter (e.g. dark matter
More informationFirst day August 1, Problems and Solutions
FOURTH INTERNATIONAL COMPETITION FOR UNIVERSITY STUDENTS IN MATHEMATICS July 30 August 4, 997, Plovdv, BULGARIA Frst day August, 997 Problems and Solutons Problem. Let {ε n } n= be a sequence of postve
More informationYong Joon Ryang. 1. Introduction Consider the multicommodity transportation problem with convex quadratic cost function. 1 2 (x x0 ) T Q(x x 0 )
Kangweon-Kyungk Math. Jour. 4 1996), No. 1, pp. 7 16 AN ITERATIVE ROW-ACTION METHOD FOR MULTICOMMODITY TRANSPORTATION PROBLEMS Yong Joon Ryang Abstract. The optmzaton problems wth quadratc constrants often
More informationSolution of Linear System of Equations and Matrix Inversion Gauss Seidel Iteration Method
Soluton of Lnear System of Equatons and Matr Inverson Gauss Sedel Iteraton Method It s another well-known teratve method for solvng a system of lnear equatons of the form a + a22 + + ann = b a2 + a222
More informationNorms, Condition Numbers, Eigenvalues and Eigenvectors
Norms, Condton Numbers, Egenvalues and Egenvectors 1 Norms A norm s a measure of the sze of a matrx or a vector For vectors the common norms are: N a 2 = ( x 2 1/2 the Eucldean Norm (1a b 1 = =1 N x (1b
More informationPART 8. Partial Differential Equations PDEs
he Islamc Unverst of Gaza Facult of Engneerng Cvl Engneerng Department Numercal Analss ECIV 3306 PAR 8 Partal Dfferental Equatons PDEs Chapter 9; Fnte Dfference: Ellptc Equatons Assocate Prof. Mazen Abualtaef
More informationMatrix Approximation via Sampling, Subspace Embedding. 1 Solving Linear Systems Using SVD
Matrx Approxmaton va Samplng, Subspace Embeddng Lecturer: Anup Rao Scrbe: Rashth Sharma, Peng Zhang 0/01/016 1 Solvng Lnear Systems Usng SVD Two applcatons of SVD have been covered so far. Today we loo
More informationStatistical Mechanics and Combinatorics : Lecture III
Statstcal Mechancs and Combnatorcs : Lecture III Dmer Model Dmer defntons Defnton A dmer coverng (perfect matchng) of a fnte graph s a set of edges whch covers every vertex exactly once, e every vertex
More informationModule 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:
More information2.29 Numerical Fluid Mechanics Fall 2011 Lecture 12
REVIEW Lecture 11: 2.29 Numercal Flud Mechancs Fall 2011 Lecture 12 End of (Lnear) Algebrac Systems Gradent Methods Krylov Subspace Methods Precondtonng of Ax=b FINITE DIFFERENCES Classfcaton of Partal
More informationSolutions to exam in SF1811 Optimization, Jan 14, 2015
Solutons to exam n SF8 Optmzaton, Jan 4, 25 3 3 O------O -4 \ / \ / The network: \/ where all lnks go from left to rght. /\ / \ / \ 6 O------O -5 2 4.(a) Let x = ( x 3, x 4, x 23, x 24 ) T, where the varable
More informationPerron Vectors of an Irreducible Nonnegative Interval Matrix
Perron Vectors of an Irreducble Nonnegatve Interval Matrx Jr Rohn August 4 2005 Abstract As s well known an rreducble nonnegatve matrx possesses a unquely determned Perron vector. As the man result of
More informationNUMERICAL DIFFERENTIATION
NUMERICAL DIFFERENTIATION 1 Introducton Dfferentaton s a method to compute the rate at whch a dependent output y changes wth respect to the change n the ndependent nput x. Ths rate of change s called the
More informationStatistical pattern recognition
Statstcal pattern recognton Bayes theorem Problem: decdng f a patent has a partcular condton based on a partcular test However, the test s mperfect Someone wth the condton may go undetected (false negatve
More informationC4B Machine Learning Answers II. = σ(z) (1 σ(z)) 1 1 e z. e z = σ(1 σ) (1 + e z )
C4B Machne Learnng Answers II.(a) Show that for the logstc sgmod functon dσ(z) dz = σ(z) ( σ(z)) A. Zsserman, Hlary Term 20 Start from the defnton of σ(z) Note that Then σ(z) = σ = dσ(z) dz = + e z e z
More informationLecture 10: May 6, 2013
TTIC/CMSC 31150 Mathematcal Toolkt Sprng 013 Madhur Tulsan Lecture 10: May 6, 013 Scrbe: Wenje Luo In today s lecture, we manly talked about random walk on graphs and ntroduce the concept of graph expander,
More informationApplication of B-Spline to Numerical Solution of a System of Singularly Perturbed Problems
Mathematca Aeterna, Vol. 1, 011, no. 06, 405 415 Applcaton of B-Splne to Numercal Soluton of a System of Sngularly Perturbed Problems Yogesh Gupta Department of Mathematcs Unted College of Engneerng &
More informationLecture 6/7 (February 10/12, 2014) DIRAC EQUATION. The non-relativistic Schrödinger equation was obtained by noting that the Hamiltonian 2
P470 Lecture 6/7 (February 10/1, 014) DIRAC EQUATION The non-relatvstc Schrödnger equaton was obtaned by notng that the Hamltonan H = P (1) m can be transformed nto an operator form wth the substtutons
More informationMEM 255 Introduction to Control Systems Review: Basics of Linear Algebra
MEM 255 Introducton to Control Systems Revew: Bascs of Lnear Algebra Harry G. Kwatny Department of Mechancal Engneerng & Mechancs Drexel Unversty Outlne Vectors Matrces MATLAB Advanced Topcs Vectors A
More informationU.C. Berkeley CS294: Spectral Methods and Expanders Handout 8 Luca Trevisan February 17, 2016
U.C. Berkeley CS94: Spectral Methods and Expanders Handout 8 Luca Trevsan February 7, 06 Lecture 8: Spectral Algorthms Wrap-up In whch we talk about even more generalzatons of Cheeger s nequaltes, and
More informationMATH 5630: Discrete Time-Space Model Hung Phan, UMass Lowell March 1, 2018
MATH 5630: Dscrete Tme-Space Model Hung Phan, UMass Lowell March, 08 Newton s Law of Coolng Consder the coolng of a well strred coffee so that the temperature does not depend on space Newton s law of collng
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More informationOverlapping additive and multiplicative Schwarz iterations for H -matrices
Lnear Algebra and ts Applcatons 393 (2004) 91 105 www.elsever.com/locate/laa Overlappng addtve and multplcatve Schwarz teratons for H -matrces Rafael Bru a,1, Francsco Pedroche a, Danel B. Szyld b,,2 a
More informationHidden Markov Models & The Multivariate Gaussian (10/26/04)
CS281A/Stat241A: Statstcal Learnng Theory Hdden Markov Models & The Multvarate Gaussan (10/26/04) Lecturer: Mchael I. Jordan Scrbes: Jonathan W. Hu 1 Hdden Markov Models As a bref revew, hdden Markov models
More informationReport on Image warping
Report on Image warpng Xuan Ne, Dec. 20, 2004 Ths document summarzed the algorthms of our mage warpng soluton for further study, and there s a detaled descrpton about the mplementaton of these algorthms.
More informationHongyi Miao, College of Science, Nanjing Forestry University, Nanjing ,China. (Received 20 June 2013, accepted 11 March 2014) I)ϕ (k)
ISSN 1749-3889 (prnt), 1749-3897 (onlne) Internatonal Journal of Nonlnear Scence Vol.17(2014) No.2,pp.188-192 Modfed Block Jacob-Davdson Method for Solvng Large Sparse Egenproblems Hongy Mao, College of
More informationLecture 16 Statistical Analysis in Biomaterials Research (Part II)
3.051J/0.340J 1 Lecture 16 Statstcal Analyss n Bomaterals Research (Part II) C. F Dstrbuton Allows comparson of varablty of behavor between populatons usng test of hypothess: σ x = σ x amed for Brtsh statstcan
More informationChapter 12. Ordinary Differential Equation Boundary Value (BV) Problems
Chapter. Ordnar Dfferental Equaton Boundar Value (BV) Problems In ths chapter we wll learn how to solve ODE boundar value problem. BV ODE s usuall gven wth x beng the ndependent space varable. p( x) q(
More informationLecture 5.8 Flux Vector Splitting
Lecture 5.8 Flux Vector Splttng 1 Flux Vector Splttng The vector E n (5.7.) can be rewrtten as E = AU (5.8.1) (wth A as gven n (5.7.4) or (5.7.6) ) whenever, the equaton of state s of the separable form
More informationSolution 1 for USTC class Physics of Quantum Information
Soluton 1 for 018 019 USTC class Physcs of Quantum Informaton Shua Zhao, Xn-Yu Xu and Ka Chen Natonal Laboratory for Physcal Scences at Mcroscale and Department of Modern Physcs, Unversty of Scence and
More informationProfessor Terje Haukaas University of British Columbia, Vancouver The Q4 Element
Professor Terje Haukaas Unversty of Brtsh Columba, ancouver www.nrsk.ubc.ca The Q Element Ths document consders fnte elements that carry load only n ther plane. These elements are sometmes referred to
More informationTHE CHINESE REMAINDER THEOREM. We should thank the Chinese for their wonderful remainder theorem. Glenn Stevens
THE CHINESE REMAINDER THEOREM KEITH CONRAD We should thank the Chnese for ther wonderful remander theorem. Glenn Stevens 1. Introducton The Chnese remander theorem says we can unquely solve any par of
More informationTopic 5: Non-Linear Regression
Topc 5: Non-Lnear Regresson The models we ve worked wth so far have been lnear n the parameters. They ve been of the form: y = Xβ + ε Many models based on economc theory are actually non-lnear n the parameters.
More informationBOUNDEDNESS OF THE RIESZ TRANSFORM WITH MATRIX A 2 WEIGHTS
BOUNDEDNESS OF THE IESZ TANSFOM WITH MATIX A WEIGHTS Introducton Let L = L ( n, be the functon space wth norm (ˆ f L = f(x C dx d < For a d d matrx valued functon W : wth W (x postve sem-defnte for all
More informationA FORMULA FOR COMPUTING INTEGER POWERS FOR ONE TYPE OF TRIDIAGONAL MATRIX
Hacettepe Journal of Mathematcs and Statstcs Volume 393 0 35 33 FORMUL FOR COMPUTING INTEGER POWERS FOR ONE TYPE OF TRIDIGONL MTRIX H Kıyak I Gürses F Yılmaz and D Bozkurt Receved :08 :009 : ccepted 5
More informationDynamic Systems on Graphs
Prepared by F.L. Lews Updated: Saturday, February 06, 200 Dynamc Systems on Graphs Control Graphs and Consensus A network s a set of nodes that collaborates to acheve what each cannot acheve alone. A network,
More informationFeb 14: Spatial analysis of data fields
Feb 4: Spatal analyss of data felds Mappng rregularly sampled data onto a regular grd Many analyss technques for geophyscal data requre the data be located at regular ntervals n space and/or tme. hs s
More informationFall 2012 Analysis of Experimental Measurements B. Eisenstein/rev. S. Errede
Fall 0 Analyss of Expermental easurements B. Esensten/rev. S. Errede We now reformulate the lnear Least Squares ethod n more general terms, sutable for (eventually extendng to the non-lnear case, and also
More informationDynamic Programming. Preview. Dynamic Programming. Dynamic Programming. Dynamic Programming (Example: Fibonacci Sequence)
/24/27 Prevew Fbonacc Sequence Longest Common Subsequence Dynamc programmng s a method for solvng complex problems by breakng them down nto smpler sub-problems. It s applcable to problems exhbtng the propertes
More informationNew Method for Solving Poisson Equation. on Irregular Domains
Appled Mathematcal Scences Vol. 6 01 no. 8 369 380 New Method for Solvng Posson Equaton on Irregular Domans J. Izadan and N. Karamooz Department of Mathematcs Facult of Scences Mashhad BranchIslamc Azad
More informationALGORITHM FOR THE CALCULATION OF THE TWO VARIABLES CUBIC SPLINE FUNCTION
ANALELE ŞTIINŢIFICE ALE UNIVERSITĂŢII AL.I. CUZA DIN IAŞI (S.N.) MATEMATICĂ, Tomul LIX, 013, f.1 DOI: 10.478/v10157-01-00-y ALGORITHM FOR THE CALCULATION OF THE TWO VARIABLES CUBIC SPLINE FUNCTION BY ION
More informationn α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0
MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector
More informationLECTURE 9 CANONICAL CORRELATION ANALYSIS
LECURE 9 CANONICAL CORRELAION ANALYSIS Introducton he concept of canoncal correlaton arses when we want to quantfy the assocatons between two sets of varables. For example, suppose that the frst set of
More informationLecture 20: Lift and Project, SDP Duality. Today we will study the Lift and Project method. Then we will prove the SDP duality theorem.
prnceton u. sp 02 cos 598B: algorthms and complexty Lecture 20: Lft and Project, SDP Dualty Lecturer: Sanjeev Arora Scrbe:Yury Makarychev Today we wll study the Lft and Project method. Then we wll prove
More informationSalmon: Lectures on partial differential equations. Consider the general linear, second-order PDE in the form. ,x 2
Salmon: Lectures on partal dfferental equatons 5. Classfcaton of second-order equatons There are general methods for classfyng hgher-order partal dfferental equatons. One s very general (applyng even to
More informationMATH Homework #2
MATH609-601 Homework #2 September 27, 2012 1. Problems Ths contans a set of possble solutons to all problems of HW-2. Be vglant snce typos are possble (and nevtable). (1) Problem 1 (20 pts) For a matrx
More informationCHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE
CHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE Analytcal soluton s usually not possble when exctaton vares arbtrarly wth tme or f the system s nonlnear. Such problems can be solved by numercal tmesteppng
More informationNON-CENTRAL 7-POINT FORMULA IN THE METHOD OF LINES FOR PARABOLIC AND BURGERS' EQUATIONS
IJRRAS 8 (3 September 011 www.arpapress.com/volumes/vol8issue3/ijrras_8_3_08.pdf NON-CENTRAL 7-POINT FORMULA IN THE METHOD OF LINES FOR PARABOLIC AND BURGERS' EQUATIONS H.O. Bakodah Dept. of Mathematc
More informationSIO 224. m(r) =(ρ(r),k s (r),µ(r))
SIO 224 1. A bref look at resoluton analyss Here s some background for the Masters and Gubbns resoluton paper. Global Earth models are usually found teratvely by assumng a startng model and fndng small
More informationLossy Compression. Compromise accuracy of reconstruction for increased compression.
Lossy Compresson Compromse accuracy of reconstructon for ncreased compresson. The reconstructon s usually vsbly ndstngushable from the orgnal mage. Typcally, one can get up to 0:1 compresson wth almost
More informationTransfer Functions. Convenient representation of a linear, dynamic model. A transfer function (TF) relates one input and one output: ( ) system
Transfer Functons Convenent representaton of a lnear, dynamc model. A transfer functon (TF) relates one nput and one output: x t X s y t system Y s The followng termnology s used: x y nput output forcng
More informationChapter Newton s Method
Chapter 9. Newton s Method After readng ths chapter, you should be able to:. Understand how Newton s method s dfferent from the Golden Secton Search method. Understand how Newton s method works 3. Solve
More informatione - c o m p a n i o n
OPERATIONS RESEARCH http://dxdoorg/0287/opre007ec e - c o m p a n o n ONLY AVAILABLE IN ELECTRONIC FORM 202 INFORMS Electronc Companon Generalzed Quantty Competton for Multple Products and Loss of Effcency
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Experment-I MODULE VII LECTURE - 3 ANALYSIS OF COVARIANCE Dr Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur Any scentfc experment s performed
More informationCommon loop optimizations. Example to improve locality. Why Dependence Analysis. Data Dependence in Loops. Goal is to find best schedule:
15-745 Lecture 6 Data Dependence n Loops Copyrght Seth Goldsten, 2008 Based on sldes from Allen&Kennedy Lecture 6 15-745 2005-8 1 Common loop optmzatons Hostng of loop-nvarant computatons pre-compute before
More informationProf. Dr. I. Nasser Phys 630, T Aug-15 One_dimensional_Ising_Model
EXACT OE-DIMESIOAL ISIG MODEL The one-dmensonal Isng model conssts of a chan of spns, each spn nteractng only wth ts two nearest neghbors. The smple Isng problem n one dmenson can be solved drectly n several
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More informationLecture 3. Ax x i a i. i i
18.409 The Behavor of Algorthms n Practce 2/14/2 Lecturer: Dan Spelman Lecture 3 Scrbe: Arvnd Sankar 1 Largest sngular value In order to bound the condton number, we need an upper bound on the largest
More informationPh 219a/CS 219a. Exercises Due: Wednesday 23 October 2013
1 Ph 219a/CS 219a Exercses Due: Wednesday 23 October 2013 1.1 How far apart are two quantum states? Consder two quantum states descrbed by densty operators ρ and ρ n an N-dmensonal Hlbert space, and consder
More informationA PROCEDURE FOR SIMULATING THE NONLINEAR CONDUCTION HEAT TRANSFER IN A BODY WITH TEMPERATURE DEPENDENT THERMAL CONDUCTIVITY.
Proceedngs of the th Brazlan Congress of Thermal Scences and Engneerng -- ENCIT 006 Braz. Soc. of Mechancal Scences and Engneerng -- ABCM, Curtba, Brazl,- Dec. 5-8, 006 A PROCEDURE FOR SIMULATING THE NONLINEAR
More informationBezier curves. Michael S. Floater. August 25, These notes provide an introduction to Bezier curves. i=0
Bezer curves Mchael S. Floater August 25, 211 These notes provde an ntroducton to Bezer curves. 1 Bernsten polynomals Recall that a real polynomal of a real varable x R, wth degree n, s a functon of the
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationDiscretization. Consistency. Exact Solution Convergence x, t --> 0. Figure 5.1: Relation between consistency, stability, and convergence.
Chapter 5 Theory The numercal smulaton of PDE s requres careful consderatons of propertes of the approxmate soluton. A necessary condton for the scheme used to model a physcal problem s the consstency
More informationProblem Set 9 Solutions
Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem
More informationLecture 13 APPROXIMATION OF SECOMD ORDER DERIVATIVES
COMPUTATIONAL FLUID DYNAMICS: FDM: Appromaton of Second Order Dervatves Lecture APPROXIMATION OF SECOMD ORDER DERIVATIVES. APPROXIMATION OF SECOND ORDER DERIVATIVES Second order dervatves appear n dffusve
More informationA new construction of 3-separable matrices via an improved decoding of Macula s construction
Dscrete Optmzaton 5 008 700 704 Contents lsts avalable at ScenceDrect Dscrete Optmzaton journal homepage: wwwelsevercom/locate/dsopt A new constructon of 3-separable matrces va an mproved decodng of Macula
More information