Numerical Properties of the LLL Algorithm
|
|
- Eunice Shelton
- 5 years ago
- Views:
Transcription
1 Numercal Propertes of the LLL Algorthm Frankln T. Luk a and Sanzheng Qao b a Department of Mathematcs, Hong Kong Baptst Unversty, Kowloon Tong, Hong Kong b Dept. of Computng and Software, McMaster Unv., Hamlton, Ontaro L8S 4L7, Canada ABSTRACT The LLL algorthm s wdely used to solve the nteger least squares problems that arse n many engneerng applcatons. As most practtoners dd not understand how the LLL algorthm works, they avoded the ssue by referrng to the method as an nteger Gram Schmdt approach (wthout explanng what they mean by ths term). Luk and Tracy 1 were frst to descrbe the behavor of the LLL algorthm, and they presented a new numercal mplementaton that should be more robust than the orgnal LLL scheme. In ths paper, we compare the numercal propertes of the two dfferent LLL mplementatons. Keywords: LLL algorthm, unmodular transformaton, QR decomposton, reduced bass, Gauss transformaton, plane reflecton, numercal overflow and underflow. 1. INTRODUCTION The famous algorthm due to Lenstra, Lenstra and Lovasz 2 has many mportant applcatons; for example, wreless communcaton, cryptography, and GPS (see Hassb and Vkalo 3 and references theren). In some of these applcatons, researchers use the LLL algorthm as a precondtoner n solvng an nteger least squares problem. Although the LLL algorthm s often referred to as an nteger Gram-Schmdt procedure, no one has explaned the workngs of such a process. Luk and Tracy 1 acheved a breakthrough by showng how an LLL reducton can be mplemented usng orthogonal nstead of Gauss transformatons. The purpose of ths paper s to compare the two dfferent numercal mplementatons of the LLL method. Ths paper s organzed as follows. In Sectons 2 and 3, we descrbe the orgnal 2 and new 1 mplementatons of the LLL algorthm. In Secton 4, we present the result 1 that the two dfferent mplementatons gve the same answers n exact arthmetc. Lastly, n Secton 5, we conclude the paper by presentng examples to compare the numercal propertes of the two mplementatons. 2. LLL ALGORITHM Gven a nonsngular matrx B R n n, an dea n Lenstra et al. 2 s to construct a unmodular matrx M Z n n so that the columns of BM become almost orthogonal; a usual consequence s that the condton number of BM wll become much smaller than that of B. Defnton 1. A nonsngular matrx M s unmodular f det(m) = ±1. Lemma 1. A nonsngular nteger matrx M s unmodular f and only f M 1 s an nteger matrx. A key concept n the LLL paper 2 s that of a reduced bass. Consder the QR decomposton of B: where Q R n n s orthogonal, D dag(d ) R n n s dagonal wth Q T B = DU, (1) d > 0, for = 1, 2,..., n, and U (u,j ) R n n s upper trangular wth ones on ts dagonal: Send correspondence to S. Qao: qao@mcmaster.ca u, = 1, for = 1, 2,..., n.
2 Defnton 2. The columns of B form a reduced bass f and where 0.25 < ω < 1 s a parameter that controls the rate of convergence. u,j 0.5, for 1 < j n, (2) d 2 (ω u2 1,j )d2 1, for 2 n, (3) Condton (2) states that the absolute value of any strctly upper trangular element of U s at most 0.5. Condton (3) states that the dagonal elements of U must be ordered n a certan manner. Lemma 2. Snce the value of the quantty nsde the parentheses n (3) s always less than one, an upper trangular matrx B R n n wth a constant dagonal satsfes condton (3). Example 1. The columns of ths trangular matrx B R n n form a reduced bass: B = (4) The matrx s very ll-condtoned, for Luk and Tracy 1 show that ts condton number ncreases lke (1.5) n 2 /2. Let us descrbe the actons of the LLL algorthm by showng how condtons (2) and (3) are enforced. Condton (2) s easy to mpose on U (u,j ), an upper trangular matrx wth a unt dagonal. We begn by defnng elementary unmodular transformaton. Let < j, and let e Z n and e j Z n denote the unt coordnate vectors n the -th and j-th drectons, respectvely. Defne M j Z n n by where γ s an nteger. M j I γe e T j, (5) Lemma 3. The matrx M j defned n (5) s an nteger unmodular transformaton. We use M j to ensure that the (, j)-th element of U s suffcently small. Suppose that (2) s not satsfed for some and j; that s, u,j > 0.5. Calculate γ as the nteger closest to u,j : γ = u,j. (6) Construct the unmodular matrx M j wth ts (, j)-th element equal to γ. Apply M j to B and to U: B BM j and U UM j. (7) The (, j)-th element of the new U satsfes (2). We summarze the actons to enforce (2) n the next procedure. PROCEDURE DECREASE(, j) Gven B and U, calculate M j and γ usng (5) and (6), respectvely. Apply M j to B and U: B BM j and U UM j. Notaton 1. The matrx Π Z n n denotes a permutaton n the ( 1, ) plane, where 2 n. Notaton 2. The matrx X R n n denotes a transformaton n the ( 1, ) plane, where 2 n. It has the form: I 2 X µ 1 ξµ 1 ξ. (8) I n
3 For condton (3), we use the two numercal transformatons defned n the two notatons. Note that and that X 1 s gven by X 1 = det(x ) = 1, (9) 2 ξ 1 ξµ 1 µ I n. (10) The matrx X 1 s made up of a product of two Gauss transformatons; here s a quck llustraton: [ [ [ ξ 1 ξµ 1 ξ 0 1 =. 1 µ µ Ths matrx X 1 s a workhorse n the LLL algorthm, and the followng relaton s key: [ [ [ [ ξ 1 ξµ 1 µ ξ = 1 µ (11) In words, equaton (11) says that the matrx X 1 restores the trangularty of a permuted trangular matrx. Note that both upper trangular matrces n (11) have ones on ther dagonals. Suppose that the (3) s not satsfed for some : d 2 < (ω u2 1, )d2 1. We nterchange columns and 1 of B and those of U: We then use the transformaton X 1 B BΠ and U UΠ. (12) of (10) to restore U to trangular form: U X 1 U. (13) Lenstra et al. 2 gve the formulas that are used to update the squares of the dagonal elements d 1 and d of D. Specfcally, ˆd 2 1 = d2 + µ2 d 2 1 and ˆd2 = (d 2 d2 1 )/ ˆd 2 1, (14) where ˆd 1 and ˆd are the new dagonal elements. The paper 2 also gves the values of ξ and µ n (8). As s obvous from (11), µ s gven by µ = u 1,. (15) In addton, ξ s gven by The actons to enforce (3) are wrtten out n the next procedure. ξ = µ d 2 1 /(d2 + µ2 d 2 1 ). (16) PROCEDURE SWAP() Gven D 2, B, and U, update D 2, swap columns 1 and and those of B and of U, and use the transformaton X 1 to transform U back to trangular form: D 2 D 2 new, B BΠ, and U X 1 UΠ. (17) The matrx Dnew 2 1 s obtaned by (14) and X s computed by the equatons (10), (15), and (16). Luk and Tracy 1 use the two procedures, Decrease and Swap, to construct an algorthmc descrpton of the LLL algorthm. The orgnal LLL paper 2 contans a proof of convergence, but not an algorthmc descrpton, of the method. It s far to say that the algorthmc descrpton nspred Luk and Tracy 1 to derve ther new mplementaton. Although the LLL algorthm has shown to be an effectve tool 3 to reduce the condton number of most ll-condtoned matrces that occur n practce, t does not modfy the ll-condtoned matrx B of (4) because ts columns already form a reduced bass.
4 ALGORITHM LLL Gven B, transform ts columns so that they wll form a reduced bass. compute the QR decomposton of B to get D 2 and U; set k 2; whle k n f u k 1,k > 0.5 then DECREASE(k 1, k); f d 2 k < (ω u2 k 1,k )d2 k 1 then SWAP(k); k max(k 1, 2); else for = k 2 down to 1 f u,k > 0.5 then DECREASE(, k); k k A NEW IMPLEMENTATION Luk and Tracy 1 extend the dea of a reduced bass formed by column vectors to that of a reduced trangular matrx. Let B R n n be nonsngular. Consder ts QR decomposton: Q T B = R, (18) where Q R n n s orthogonal and R (r,j ) R n n s upper trangular wth a postve dagonal: r, > 0, for = 1, 2,..., n. Ths extenson 1 leads to a new algorthm to transform a gven matrx B to a reduced trangular matrx R. Defnton 3. The columns of B form a reduced bass f and r, 2 r,j, for 1 < j n, (19) r 2, [ω (r 1, /r 1, 1 ) 2 r 2 1, 1, for 2 n, (20) where 0.25 < ω < 1 s a parameter that controls the rate of convergence. Defnton 4. An upper trangular matrx R s reduced f ts elements satsfy the condtons (19) and (20). Proposton 1. Gven B R n n, the new algorthm generates an orthogonal matrx Q R n n and a unmodular matrx M Z n n to transform B nto a trangular matrx R: Q T BM = R, (21) so that R s reduced. The columns of BM form a reduced bass as defned n the LLL paper. The new approach 1 enforces condtons (19) and (20). Whle condton (19) states that any dagonal element of R s at least twce as large as any other element of R along the same row, condton (20) states that the dagonal elements of R must be ordered n a certan way. We use M j of (5) to ensure that the (, j)-th element of R s suffcently small relatve to r,. Suppose that (19) s not satsfed for some and j; that s, Calculate γ as the nteger closest to r,j /r, : r, < 2 r,j. γ = r,j /r,. (22) Construct the unmodular matrx M j wth ts (, j)-th element equal to γ. Apply M j to R: R RM j, (23)
5 and accumulate the transformatons n M: M MM j. It s easy to check that the (, j)-th element of the new R n (23) satsfes (19). For condton (20) we need to use a plane reflecton 4 (a basc numercal tool that s closely related to the more famlar plane rotaton). PROCEDURE NEWDECREASE(, j) Gven R and M, calculate M j and γ usng (5) and (22), respectvely, and apply M j to both R and M: R RM j and M MM j. Notaton 3. The symmetrc matrx J R n n denotes a plane reflecton n the ( 1, ) plane, where 2 n. It has the form: I 2 J c s s c, (24) where c 2 + s 2 = 1. Note that I n det(j ) = 1, (25) just as det(x ) = 1 n (9). Luk and Tracy 1 use plane reflectons nstead of plane rotatons because the X s are closely related to plane reflectons, as wll be seen n the next secton. Suppose that (20) s not satsfed for some : r 2, < [ω (r 1,/r 1, 1 ) 2 r 2 1, 1. We nterchange columns and 1 of R: and use a plane reflecton J to restore R to trangular form: We accumulate the transformatons n M and Q: R RΠ, (26) R J R. (27) M MΠ and Q QJ. Now, we have all the tools to present our new algorthm as a matrx decomposton technque. PROCEDURE NEWSWAP() Gven R, M, and Q, swap columns 1 and of R and those of M, use a plane reflecton J to transform the permuted R back to trangular form, and update Q: R J RΠ, M MΠ and Q QJ. (28) ALGORITHM NEW compute B = QR; set M I and k 2; whle k n f r k 1,k 1 < 2 r k 1,k then NEWDECREASE(k 1, k); f r 2 k,k < [ω (r k 1,k/r k 1,k 1 ) 2 r 2 k 1,k 1 then NEWSWAP(k); k max(k 1, 2); else for = k 2 down to 1 f r, < 2 r,k then NEWDECREASE(, k); k k + 1.
6 4. EQUIVALENCE RESULT There are many smlartes between Algorthms New and LLL. Both algorthms am to reduce the gven matrx B to a trangular form. A major dfference les n the transformatons used. Algorthm New apples plane reflectons J of (24) drectly to R, whle Algorthm LLL apples specal transformatons X 1 of (10) to U and D 2 separately. A sgnfcant result 1 s that the two transformatons are related va J = D 1 X 1 D 2, (29) where D 1 and D 2 are n n dagonal matrces. Thus, we may vew X 1 as a scaled plane reflecton. Luk and Tracy 1 show that n exact arthmetc, the two algorthms produce dentcal numercal results. Representng the effect of transformatons (26) and (27) by we wrte out the key 2-by-2 transformatons as follows: [ [ ˆα ˆγ c s = 0 ˆβ s c Defne a new transformaton Y by If we choose Y [ 1/ˆα 0 0 1/ˆβ then we get 1 Y = and [ 1 ξ 0 1 R new = J RΠ, [ α γ 0 β [ c s s c [ [ α 0 0 β. (30). (31) ξ = ˆγ/ˆα and µ = γ/α, (32) = Y [ ξ 1 ξµ 1 µ [ 1 µ 0 1 [ (33) Note that (33) s exactly equaton (11) for the LLL method. Also, we can easly prove that the µ and ξ as defned n (32) have the same values as the µ and ξ as defned n (15) and (16). Thus, the transformaton Y of (31) s exactly the 2 2 part of the workhorse X 1 of the LLL algorthm. Let E 1 D α 0 0 β, (34) where E 1 R ( 2) ( 2) and E 2 R (n ) (n ) are postve dagonal matrces. Defne E 1 E1 1 D 1 ˆα 0 0 ˆβ and D 2 1/α 0 0 1/β E 2 E 2 E 1 2. (35) Then J = D 1 X 1 D 2. (36) We see that D 2 reduces R to a unt-dagonal trangular matrx (namely U), and that D 1 gves the new dagonal of D 2 R after beng transformed by X 1. Therefore, we conclude that Algorthms LLL and New produce the same numercal results n exact arthmetc. It also follows that the convergence result for Algorthm LLL s applcable to Algorthm New. The former algorthm s numercally more effcent n that t avods the computaton of square roots, whch s one reason why t updates D 2 nstead of D. Thus, we may vew the transformatons n the LLL method as square-root-free plane reflectons. The potental cost for ths effcecy s a possble loss n numercal accuracy, as wll be shown n the next secton.
7 5. NUMERICAL PROPERTIES As ponted out n the last secton, a sgnfcant dfference between Algorthms New and LLL s that New works drectly on R whle LLL works on U and D 2 ndvdually. Put t smply, New computes r, whle LLL calculates d 2. Consequently, Algorthm LLL s susceptble to underflow (respectvely overflow) exceptons when the dagonal elements d s are small (respectvely large). For our dscusson, we assume standard IEEE floatng-pont arthmetc. In sngle precson, we would have mnmum exponent value e mn = 126 and maxmum exponent value e max = 127. Due to the presence of denormals, a number x underflows f x < = 2 149, whereas the number x overflows f x Even f the quanttes d 2 s are not small or large enough to cause exceptons, a straghtforward mplementaton of the LLL algorthm could stll result n errors. Let ω = 0.75, and consder the followng 2-by-2 upper trangular matrx [ α µ α R = α The condton (20) s not satsfed when µ < 0.5. Recall the updatng formula (14): ˆd 2 = (d2 d2 1 )/ ˆd 2 1. The numerator (0.5 α 4 ) may readly underflow or overflow; for example, n sngle precson, an underflow would occur f 2 1 α 4 < or α < , and an overflow would occur f 2 1 α or α , Although t may be possble to avod an excepton n (14) by dong the dvson before the multplcaton, we cannot apply the same technque to prevent a possble underflow n the calculaton of the numerator n (16): ξ = (µ d 2 1 )/(d2 + µ2 d 2 1 ), where small values of µ and α could cause the product (µ α 2 ) to underflow. As experments, we programmed Algorthms LLL and New n Matlab, whch supports IEEE double precson. In double precson, we would have mnmum exponent value e mn = 1022 and maxmum exponent value e max = After both programs were run hundreds of tmes wth dentcal random data nput, we observed nether underflows nor overflows and the output results were numercally ndstngushable. ACKNOWLEDGMENTS Ths work s partally supported by Natural Scences and Engneerng Research Councl of Canada. REFERENCES 1. F. T. Luk and D. M. Tracy, An mproved LLL algorthm, Lnear Algebra and Its Applcatons, pp. x x, to appear n A. Lenstra, H. Lenstra, and L. Lovasz, Factorng polynomals wth ratonal coeffcents, Mathematcsche Annalen 261, pp , B. Hassb and H. Vkalo, On the sphere-decodng algorthm : Expected complexty, IEEE Transactons on Sgnal Processng 53, pp , G. Golub and C. V. Loan, Matrx Computatons, 3rd Ed., The Johns Hopkns Unversty Press, Baltmore, MD, 1996.
Lecture 3. Ax x i a i. i i
18.409 The Behavor of Algorthms n Practce 2/14/2 Lecturer: Dan Spelman Lecture 3 Scrbe: Arvnd Sankar 1 Largest sngular value In order to bound the condton number, we need an upper bound on the largest
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationHongyi Miao, College of Science, Nanjing Forestry University, Nanjing ,China. (Received 20 June 2013, accepted 11 March 2014) I)ϕ (k)
ISSN 1749-3889 (prnt), 1749-3897 (onlne) Internatonal Journal of Nonlnear Scence Vol.17(2014) No.2,pp.188-192 Modfed Block Jacob-Davdson Method for Solvng Large Sparse Egenproblems Hongy Mao, College of
More informationLectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationSingular Value Decomposition: Theory and Applications
Sngular Value Decomposton: Theory and Applcatons Danel Khashab Sprng 2015 Last Update: March 2, 2015 1 Introducton A = UDV where columns of U and V are orthonormal and matrx D s dagonal wth postve real
More informationThe Order Relation and Trace Inequalities for. Hermitian Operators
Internatonal Mathematcal Forum, Vol 3, 08, no, 507-57 HIKARI Ltd, wwwm-hkarcom https://doorg/0988/mf088055 The Order Relaton and Trace Inequaltes for Hermtan Operators Y Huang School of Informaton Scence
More informationErrors for Linear Systems
Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch
More informationYong Joon Ryang. 1. Introduction Consider the multicommodity transportation problem with convex quadratic cost function. 1 2 (x x0 ) T Q(x x 0 )
Kangweon-Kyungk Math. Jour. 4 1996), No. 1, pp. 7 16 AN ITERATIVE ROW-ACTION METHOD FOR MULTICOMMODITY TRANSPORTATION PROBLEMS Yong Joon Ryang Abstract. The optmzaton problems wth quadratc constrants often
More informationInner Product. Euclidean Space. Orthonormal Basis. Orthogonal
Inner Product Defnton 1 () A Eucldean space s a fnte-dmensonal vector space over the reals R, wth an nner product,. Defnton 2 (Inner Product) An nner product, on a real vector space X s a symmetrc, blnear,
More informationCSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography
CSc 6974 and ECSE 6966 Math. Tech. for Vson, Graphcs and Robotcs Lecture 21, Aprl 17, 2006 Estmatng A Plane Homography Overvew We contnue wth a dscusson of the major ssues, usng estmaton of plane projectve
More informationA New Refinement of Jacobi Method for Solution of Linear System Equations AX=b
Int J Contemp Math Scences, Vol 3, 28, no 17, 819-827 A New Refnement of Jacob Method for Soluton of Lnear System Equatons AX=b F Naem Dafchah Department of Mathematcs, Faculty of Scences Unversty of Gulan,
More informationLecture 12: Discrete Laplacian
Lecture 12: Dscrete Laplacan Scrbe: Tanye Lu Our goal s to come up wth a dscrete verson of Laplacan operator for trangulated surfaces, so that we can use t n practce to solve related problems We are mostly
More informationCanonical transformations
Canoncal transformatons November 23, 2014 Recall that we have defned a symplectc transformaton to be any lnear transformaton M A B leavng the symplectc form nvarant, Ω AB M A CM B DΩ CD Coordnate transformatons,
More informationMEM 255 Introduction to Control Systems Review: Basics of Linear Algebra
MEM 255 Introducton to Control Systems Revew: Bascs of Lnear Algebra Harry G. Kwatny Department of Mechancal Engneerng & Mechancs Drexel Unversty Outlne Vectors Matrces MATLAB Advanced Topcs Vectors A
More informationDifference Equations
Dfference Equatons c Jan Vrbk 1 Bascs Suppose a sequence of numbers, say a 0,a 1,a,a 3,... s defned by a certan general relatonshp between, say, three consecutve values of the sequence, e.g. a + +3a +1
More informationOn a direct solver for linear least squares problems
ISSN 2066-6594 Ann. Acad. Rom. Sc. Ser. Math. Appl. Vol. 8, No. 2/2016 On a drect solver for lnear least squares problems Constantn Popa Abstract The Null Space (NS) algorthm s a drect solver for lnear
More informationSection 8.3 Polar Form of Complex Numbers
80 Chapter 8 Secton 8 Polar Form of Complex Numbers From prevous classes, you may have encountered magnary numbers the square roots of negatve numbers and, more generally, complex numbers whch are the
More informationn α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0
MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector
More informationAppendix B. The Finite Difference Scheme
140 APPENDIXES Appendx B. The Fnte Dfference Scheme In ths appendx we present numercal technques whch are used to approxmate solutons of system 3.1 3.3. A comprehensve treatment of theoretcal and mplementaton
More informationDeriving the X-Z Identity from Auxiliary Space Method
Dervng the X-Z Identty from Auxlary Space Method Long Chen Department of Mathematcs, Unversty of Calforna at Irvne, Irvne, CA 92697 chenlong@math.uc.edu 1 Iteratve Methods In ths paper we dscuss teratve
More informationThe lower and upper bounds on Perron root of nonnegative irreducible matrices
Journal of Computatonal Appled Mathematcs 217 (2008) 259 267 wwwelsevercom/locate/cam The lower upper bounds on Perron root of nonnegatve rreducble matrces Guang-Xn Huang a,, Feng Yn b,keguo a a College
More informationLecture 2: Gram-Schmidt Vectors and the LLL Algorithm
NYU, Fall 2016 Lattces Mn Course Lecture 2: Gram-Schmdt Vectors and the LLL Algorthm Lecturer: Noah Stephens-Davdowtz 2.1 The Shortest Vector Problem In our last lecture, we consdered short solutons to
More informationLecture 10: May 6, 2013
TTIC/CMSC 31150 Mathematcal Toolkt Sprng 013 Madhur Tulsan Lecture 10: May 6, 013 Scrbe: Wenje Luo In today s lecture, we manly talked about random walk on graphs and ntroduce the concept of graph expander,
More informationFoundations of Arithmetic
Foundatons of Arthmetc Notaton We shall denote the sum and product of numbers n the usual notaton as a 2 + a 2 + a 3 + + a = a, a 1 a 2 a 3 a = a The notaton a b means a dvdes b,.e. ac = b where c s an
More informationMMA and GCMMA two methods for nonlinear optimization
MMA and GCMMA two methods for nonlnear optmzaton Krster Svanberg Optmzaton and Systems Theory, KTH, Stockholm, Sweden. krlle@math.kth.se Ths note descrbes the algorthms used n the author s 2007 mplementatons
More information1 Matrix representations of canonical matrices
1 Matrx representatons of canoncal matrces 2-d rotaton around the orgn: ( ) cos θ sn θ R 0 = sn θ cos θ 3-d rotaton around the x-axs: R x = 1 0 0 0 cos θ sn θ 0 sn θ cos θ 3-d rotaton around the y-axs:
More informationMathematical Preparations
1 Introducton Mathematcal Preparatons The theory of relatvty was developed to explan experments whch studed the propagaton of electromagnetc radaton n movng coordnate systems. Wthn expermental error the
More informationProblem Set 9 Solutions
Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More informationPerron Vectors of an Irreducible Nonnegative Interval Matrix
Perron Vectors of an Irreducble Nonnegatve Interval Matrx Jr Rohn August 4 2005 Abstract As s well known an rreducble nonnegatve matrx possesses a unquely determned Perron vector. As the man result of
More informationA new Approach for Solving Linear Ordinary Differential Equations
, ISSN 974-57X (Onlne), ISSN 974-5718 (Prnt), Vol. ; Issue No. 1; Year 14, Copyrght 13-14 by CESER PUBLICATIONS A new Approach for Solvng Lnear Ordnary Dfferental Equatons Fawz Abdelwahd Department of
More informationCME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 13
CME 30: NUMERICAL LINEAR ALGEBRA FALL 005/06 LECTURE 13 GENE H GOLUB 1 Iteratve Methods Very large problems (naturally sparse, from applcatons): teratve methods Structured matrces (even sometmes dense,
More informationELASTIC WAVE PROPAGATION IN A CONTINUOUS MEDIUM
ELASTIC WAVE PROPAGATION IN A CONTINUOUS MEDIUM An elastc wave s a deformaton of the body that travels throughout the body n all drectons. We can examne the deformaton over a perod of tme by fxng our look
More informationThe Jacobsthal and Jacobsthal-Lucas Numbers via Square Roots of Matrices
Internatonal Mathematcal Forum, Vol 11, 2016, no 11, 513-520 HIKARI Ltd, wwwm-hkarcom http://dxdoorg/1012988/mf20166442 The Jacobsthal and Jacobsthal-Lucas Numbers va Square Roots of Matrces Saadet Arslan
More informationSTAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 16
STAT 39: MATHEMATICAL COMPUTATIONS I FALL 218 LECTURE 16 1 why teratve methods f we have a lnear system Ax = b where A s very, very large but s ether sparse or structured (eg, banded, Toepltz, banded plus
More informationQuantum Mechanics I - Session 4
Quantum Mechancs I - Sesson 4 Aprl 3, 05 Contents Operators Change of Bass 4 3 Egenvectors and Egenvalues 5 3. Denton....................................... 5 3. Rotaton n D....................................
More informationAn efficient algorithm for multivariate Maclaurin Newton transformation
Annales UMCS Informatca AI VIII, 2 2008) 5 14 DOI: 10.2478/v10065-008-0020-6 An effcent algorthm for multvarate Maclaurn Newton transformaton Joanna Kapusta Insttute of Mathematcs and Computer Scence,
More informationAffine transformations and convexity
Affne transformatons and convexty The purpose of ths document s to prove some basc propertes of affne transformatons nvolvng convex sets. Here are a few onlne references for background nformaton: http://math.ucr.edu/
More informationU.C. Berkeley CS294: Spectral Methods and Expanders Handout 8 Luca Trevisan February 17, 2016
U.C. Berkeley CS94: Spectral Methods and Expanders Handout 8 Luca Trevsan February 7, 06 Lecture 8: Spectral Algorthms Wrap-up In whch we talk about even more generalzatons of Cheeger s nequaltes, and
More informationAPPENDIX A Some Linear Algebra
APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,
More information3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X
Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number
More informationThe Exact Formulation of the Inverse of the Tridiagonal Matrix for Solving the 1D Poisson Equation with the Finite Difference Method
Journal of Electromagnetc Analyss and Applcatons, 04, 6, 0-08 Publshed Onlne September 04 n ScRes. http://www.scrp.org/journal/jemaa http://dx.do.org/0.46/jemaa.04.6000 The Exact Formulaton of the Inverse
More informationHomework Notes Week 7
Homework Notes Week 7 Math 4 Sprng 4 #4 (a Complete the proof n example 5 that s an nner product (the Frobenus nner product on M n n (F In the example propertes (a and (d have already been verfed so we
More information2.3 Nilpotent endomorphisms
s a block dagonal matrx, wth A Mat dm U (C) In fact, we can assume that B = B 1 B k, wth B an ordered bass of U, and that A = [f U ] B, where f U : U U s the restrcton of f to U 40 23 Nlpotent endomorphsms
More informationSalmon: Lectures on partial differential equations. Consider the general linear, second-order PDE in the form. ,x 2
Salmon: Lectures on partal dfferental equatons 5. Classfcaton of second-order equatons There are general methods for classfyng hgher-order partal dfferental equatons. One s very general (applyng even to
More informationLecture 5 Decoding Binary BCH Codes
Lecture 5 Decodng Bnary BCH Codes In ths class, we wll ntroduce dfferent methods for decodng BCH codes 51 Decodng the [15, 7, 5] 2 -BCH Code Consder the [15, 7, 5] 2 -code C we ntroduced n the last lecture
More information12 MATH 101A: ALGEBRA I, PART C: MULTILINEAR ALGEBRA. 4. Tensor product
12 MATH 101A: ALGEBRA I, PART C: MULTILINEAR ALGEBRA Here s an outlne of what I dd: (1) categorcal defnton (2) constructon (3) lst of basc propertes (4) dstrbutve property (5) rght exactness (6) localzaton
More informationOn the Interval Zoro Symmetric Single-step Procedure for Simultaneous Finding of Polynomial Zeros
Appled Mathematcal Scences, Vol. 5, 2011, no. 75, 3693-3706 On the Interval Zoro Symmetrc Sngle-step Procedure for Smultaneous Fndng of Polynomal Zeros S. F. M. Rusl, M. Mons, M. A. Hassan and W. J. Leong
More informationTHE CHINESE REMAINDER THEOREM. We should thank the Chinese for their wonderful remainder theorem. Glenn Stevens
THE CHINESE REMAINDER THEOREM KEITH CONRAD We should thank the Chnese for ther wonderful remander theorem. Glenn Stevens 1. Introducton The Chnese remander theorem says we can unquely solve any par of
More informationSection 3.6 Complex Zeros
04 Chapter Secton 6 Comple Zeros When fndng the zeros of polynomals, at some pont you're faced wth the problem Whle there are clearly no real numbers that are solutons to ths equaton, leavng thngs there
More informationInexact Newton Methods for Inverse Eigenvalue Problems
Inexact Newton Methods for Inverse Egenvalue Problems Zheng-jan Ba Abstract In ths paper, we survey some of the latest development n usng nexact Newton-lke methods for solvng nverse egenvalue problems.
More informationSpeeding up Computation of Scalar Multiplication in Elliptic Curve Cryptosystem
H.K. Pathak et. al. / (IJCSE) Internatonal Journal on Computer Scence and Engneerng Speedng up Computaton of Scalar Multplcaton n Ellptc Curve Cryptosystem H. K. Pathak Manju Sangh S.o.S n Computer scence
More informationEstimating the Fundamental Matrix by Transforming Image Points in Projective Space 1
Estmatng the Fundamental Matrx by Transformng Image Ponts n Projectve Space 1 Zhengyou Zhang and Charles Loop Mcrosoft Research, One Mcrosoft Way, Redmond, WA 98052, USA E-mal: fzhang,cloopg@mcrosoft.com
More informationHidden Markov Models & The Multivariate Gaussian (10/26/04)
CS281A/Stat241A: Statstcal Learnng Theory Hdden Markov Models & The Multvarate Gaussan (10/26/04) Lecturer: Mchael I. Jordan Scrbes: Jonathan W. Hu 1 Hdden Markov Models As a bref revew, hdden Markov models
More informationReport on Image warping
Report on Image warpng Xuan Ne, Dec. 20, 2004 Ths document summarzed the algorthms of our mage warpng soluton for further study, and there s a detaled descrpton about the mplementaton of these algorthms.
More informationDeveloping an Improved Shift-and-Invert Arnoldi Method
Avalable at http://pvamu.edu/aam Appl. Appl. Math. ISSN: 93-9466 Vol. 5, Issue (June 00) pp. 67-80 (Prevously, Vol. 5, No. ) Applcatons and Appled Mathematcs: An Internatonal Journal (AAM) Developng an
More informationTime-Varying Systems and Computations Lecture 6
Tme-Varyng Systems and Computatons Lecture 6 Klaus Depold 14. Januar 2014 The Kalman Flter The Kalman estmaton flter attempts to estmate the actual state of an unknown dscrete dynamcal system, gven nosy
More informationNotes on Frequency Estimation in Data Streams
Notes on Frequency Estmaton n Data Streams In (one of) the data streamng model(s), the data s a sequence of arrvals a 1, a 2,..., a m of the form a j = (, v) where s the dentty of the tem and belongs to
More informationVector Norms. Chapter 7 Iterative Techniques in Matrix Algebra. Cauchy-Bunyakovsky-Schwarz Inequality for Sums. Distances. Convergence.
Vector Norms Chapter 7 Iteratve Technques n Matrx Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematcs Unversty of Calforna, Berkeley Math 128B Numercal Analyss Defnton A vector norm
More informationDISCRIMINANTS AND RAMIFIED PRIMES. 1. Introduction A prime number p is said to be ramified in a number field K if the prime ideal factorization
DISCRIMINANTS AND RAMIFIED PRIMES KEITH CONRAD 1. Introducton A prme number p s sad to be ramfed n a number feld K f the prme deal factorzaton (1.1) (p) = po K = p e 1 1 peg g has some e greater than 1.
More informationModule 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:
More informationProblem Do any of the following determine homomorphisms from GL n (C) to GL n (C)?
Homework 8 solutons. Problem 16.1. Whch of the followng defne homomomorphsms from C\{0} to C\{0}? Answer. a) f 1 : z z Yes, f 1 s a homomorphsm. We have that z s the complex conjugate of z. If z 1,z 2
More informationExample: (13320, 22140) =? Solution #1: The divisors of are 1, 2, 3, 4, 5, 6, 9, 10, 12, 15, 18, 20, 27, 30, 36, 41,
The greatest common dvsor of two ntegers a and b (not both zero) s the largest nteger whch s a common factor of both a and b. We denote ths number by gcd(a, b), or smply (a, b) when there s no confuson
More informationSupport Vector Machines. Vibhav Gogate The University of Texas at dallas
Support Vector Machnes Vbhav Gogate he Unversty of exas at dallas What We have Learned So Far? 1. Decson rees. Naïve Bayes 3. Lnear Regresson 4. Logstc Regresson 5. Perceptron 6. Neural networks 7. K-Nearest
More informationarxiv: v1 [math.ho] 18 May 2008
Recurrence Formulas for Fbonacc Sums Adlson J. V. Brandão, João L. Martns 2 arxv:0805.2707v [math.ho] 8 May 2008 Abstract. In ths artcle we present a new recurrence formula for a fnte sum nvolvng the Fbonacc
More informationFor now, let us focus on a specific model of neurons. These are simplified from reality but can achieve remarkable results.
Neural Networks : Dervaton compled by Alvn Wan from Professor Jtendra Malk s lecture Ths type of computaton s called deep learnng and s the most popular method for many problems, such as computer vson
More information= = = (a) Use the MATLAB command rref to solve the system. (b) Let A be the coefficient matrix and B be the right-hand side of the system.
Chapter Matlab Exercses Chapter Matlab Exercses. Consder the lnear system of Example n Secton.. x x x y z y y z (a) Use the MATLAB command rref to solve the system. (b) Let A be the coeffcent matrx and
More informationCommunication Complexity 16:198: February Lecture 4. x ij y ij
Communcaton Complexty 16:198:671 09 February 2010 Lecture 4 Lecturer: Troy Lee Scrbe: Rajat Mttal 1 Homework problem : Trbes We wll solve the thrd queston n the homework. The goal s to show that the nondetermnstc
More informationMA 323 Geometric Modelling Course Notes: Day 13 Bezier Curves & Bernstein Polynomials
MA 323 Geometrc Modellng Course Notes: Day 13 Bezer Curves & Bernsten Polynomals Davd L. Fnn Over the past few days, we have looked at de Casteljau s algorthm for generatng a polynomal curve, and we have
More informationComparison of Wiener Filter solution by SVD with decompositions QR and QLP
Proceedngs of the 6th WSEAS Int Conf on Artfcal Intellgence, Knowledge Engneerng and Data Bases, Corfu Island, Greece, February 6-9, 007 7 Comparson of Wener Flter soluton by SVD wth decompostons QR and
More informationLECTURE 9 CANONICAL CORRELATION ANALYSIS
LECURE 9 CANONICAL CORRELAION ANALYSIS Introducton he concept of canoncal correlaton arses when we want to quantfy the assocatons between two sets of varables. For example, suppose that the frst set of
More information1 GSW Iterative Techniques for y = Ax
1 for y = A I m gong to cheat here. here are a lot of teratve technques that can be used to solve the general case of a set of smultaneous equatons (wrtten n the matr form as y = A), but ths chapter sn
More informationThe internal structure of natural numbers and one method for the definition of large prime numbers
The nternal structure of natural numbers and one method for the defnton of large prme numbers Emmanul Manousos APM Insttute for the Advancement of Physcs and Mathematcs 3 Poulou str. 53 Athens Greece Abstract
More informationFormulas for the Determinant
page 224 224 CHAPTER 3 Determnants e t te t e 2t 38 A = e t 2te t e 2t e t te t 2e 2t 39 If 123 A = 345, 456 compute the matrx product A adj(a) What can you conclude about det(a)? For Problems 40 43, use
More informationGrover s Algorithm + Quantum Zeno Effect + Vaidman
Grover s Algorthm + Quantum Zeno Effect + Vadman CS 294-2 Bomb 10/12/04 Fall 2004 Lecture 11 Grover s algorthm Recall that Grover s algorthm for searchng over a space of sze wors as follows: consder the
More informationSolution 1 for USTC class Physics of Quantum Information
Soluton 1 for 018 019 USTC class Physcs of Quantum Informaton Shua Zhao, Xn-Yu Xu and Ka Chen Natonal Laboratory for Physcal Scences at Mcroscale and Department of Modern Physcs, Unversty of Scence and
More informationBOUNDEDNESS OF THE RIESZ TRANSFORM WITH MATRIX A 2 WEIGHTS
BOUNDEDNESS OF THE IESZ TANSFOM WITH MATIX A WEIGHTS Introducton Let L = L ( n, be the functon space wth norm (ˆ f L = f(x C dx d < For a d d matrx valued functon W : wth W (x postve sem-defnte for all
More informationAn Algorithm to Solve the Inverse Kinematics Problem of a Robotic Manipulator Based on Rotation Vectors
An Algorthm to Solve the Inverse Knematcs Problem of a Robotc Manpulator Based on Rotaton Vectors Mohamad Z. Al-az*, Mazn Z. Othman**, and Baker B. Al-Bahr* *AL-Nahran Unversty, Computer Eng. Dep., Baghdad,
More informationAdvanced Circuits Topics - Part 1 by Dr. Colton (Fall 2017)
Advanced rcuts Topcs - Part by Dr. olton (Fall 07) Part : Some thngs you should already know from Physcs 0 and 45 These are all thngs that you should have learned n Physcs 0 and/or 45. Ths secton s organzed
More informationModule 9. Lecture 6. Duality in Assignment Problems
Module 9 1 Lecture 6 Dualty n Assgnment Problems In ths lecture we attempt to answer few other mportant questons posed n earler lecture for (AP) and see how some of them can be explaned through the concept
More informationCHAPTER 14 GENERAL PERTURBATION THEORY
CHAPTER 4 GENERAL PERTURBATION THEORY 4 Introducton A partcle n orbt around a pont mass or a sphercally symmetrc mass dstrbuton s movng n a gravtatonal potental of the form GM / r In ths potental t moves
More informationOn the correction of the h-index for career length
1 On the correcton of the h-ndex for career length by L. Egghe Unverstet Hasselt (UHasselt), Campus Depenbeek, Agoralaan, B-3590 Depenbeek, Belgum 1 and Unverstet Antwerpen (UA), IBW, Stadscampus, Venusstraat
More informationAssortment Optimization under MNL
Assortment Optmzaton under MNL Haotan Song Aprl 30, 2017 1 Introducton The assortment optmzaton problem ams to fnd the revenue-maxmzng assortment of products to offer when the prces of products are fxed.
More informationMATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS
MATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS These are nformal notes whch cover some of the materal whch s not n the course book. The man purpose s to gve a number of nontrval examples
More informationThe Minimum Universal Cost Flow in an Infeasible Flow Network
Journal of Scences, Islamc Republc of Iran 17(2): 175-180 (2006) Unversty of Tehran, ISSN 1016-1104 http://jscencesutacr The Mnmum Unversal Cost Flow n an Infeasble Flow Network H Saleh Fathabad * M Bagheran
More informationECE559VV Project Report
ECE559VV Project Report (Supplementary Notes Loc Xuan Bu I. MAX SUM-RATE SCHEDULING: THE UPLINK CASE We have seen (n the presentaton that, for downlnk (broadcast channels, the strategy maxmzng the sum-rate
More informationTransfer Functions. Convenient representation of a linear, dynamic model. A transfer function (TF) relates one input and one output: ( ) system
Transfer Functons Convenent representaton of a lnear, dynamc model. A transfer functon (TF) relates one nput and one output: x t X s y t system Y s The followng termnology s used: x y nput output forcng
More informationFundamental loop-current method using virtual voltage sources technique for special cases
Fundamental loop-current method usng vrtual voltage sources technque for specal cases George E. Chatzaraks, 1 Marna D. Tortorel 1 and Anastasos D. Tzolas 1 Electrcal and Electroncs Engneerng Departments,
More informationC/CS/Phy191 Problem Set 3 Solutions Out: Oct 1, 2008., where ( 00. ), so the overall state of the system is ) ( ( ( ( 00 ± 11 ), Φ ± = 1
C/CS/Phy9 Problem Set 3 Solutons Out: Oct, 8 Suppose you have two qubts n some arbtrary entangled state ψ You apply the teleportaton protocol to each of the qubts separately What s the resultng state obtaned
More information763622S ADVANCED QUANTUM MECHANICS Solution Set 1 Spring c n a n. c n 2 = 1.
7636S ADVANCED QUANTUM MECHANICS Soluton Set 1 Sprng 013 1 Warm-up Show that the egenvalues of a Hermtan operator  are real and that the egenkets correspondng to dfferent egenvalues are orthogonal (b)
More informationLecture 21: Numerical methods for pricing American type derivatives
Lecture 21: Numercal methods for prcng Amercan type dervatves Xaoguang Wang STAT 598W Aprl 10th, 2014 (STAT 598W) Lecture 21 1 / 26 Outlne 1 Fnte Dfference Method Explct Method Penalty Method (STAT 598W)
More informationCHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE
CHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE Analytcal soluton s usually not possble when exctaton vares arbtrarly wth tme or f the system s nonlnear. Such problems can be solved by numercal tmesteppng
More informationComputation of Units in Number Fields
UTRECHT UNIVERSITY MASTER THESIS Computaton of Unts n Number Felds Author: Bas JACOBS Supervsor: Prof. dr. Frts BEUKERS Second revewer: Prof. dr. Gunther CORNELISSEN June 14, 2016 Abstract We dscuss three
More informationfind (x): given element x, return the canonical element of the set containing x;
COS 43 Sprng, 009 Dsjont Set Unon Problem: Mantan a collecton of dsjont sets. Two operatons: fnd the set contanng a gven element; unte two sets nto one (destructvely). Approach: Canoncal element method:
More informationCollege of Computer & Information Science Fall 2009 Northeastern University 20 October 2009
College of Computer & Informaton Scence Fall 2009 Northeastern Unversty 20 October 2009 CS7880: Algorthmc Power Tools Scrbe: Jan Wen and Laura Poplawsk Lecture Outlne: Prmal-dual schema Network Desgn:
More informationSome Comments on Accelerating Convergence of Iterative Sequences Using Direct Inversion of the Iterative Subspace (DIIS)
Some Comments on Acceleratng Convergence of Iteratve Sequences Usng Drect Inverson of the Iteratve Subspace (DIIS) C. Davd Sherrll School of Chemstry and Bochemstry Georga Insttute of Technology May 1998
More informationEcon107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4)
I. Classcal Assumptons Econ7 Appled Econometrcs Topc 3: Classcal Model (Studenmund, Chapter 4) We have defned OLS and studed some algebrac propertes of OLS. In ths topc we wll study statstcal propertes
More information5 The Rational Canonical Form
5 The Ratonal Canoncal Form Here p s a monc rreducble factor of the mnmum polynomal m T and s not necessarly of degree one Let F p denote the feld constructed earler n the course, consstng of all matrces
More informationDIFFERENTIAL FORMS BRIAN OSSERMAN
DIFFERENTIAL FORMS BRIAN OSSERMAN Dfferentals are an mportant topc n algebrac geometry, allowng the use of some classcal geometrc arguments n the context of varetes over any feld. We wll use them to defne
More informationa b a In case b 0, a being divisible by b is the same as to say that
Secton 6.2 Dvsblty among the ntegers An nteger a ε s dvsble by b ε f there s an nteger c ε such that a = bc. Note that s dvsble by any nteger b, snce = b. On the other hand, a s dvsble by only f a = :
More information