Supplement: Proofs and Technical Details for The Solution Path of the Generalized Lasso

Size: px
Start display at page:

Download "Supplement: Proofs and Technical Details for The Solution Path of the Generalized Lasso"

Transcription

1 Supplement: Proofs and Techncal Detals for The Soluton Path of the Generalzed Lasso Ryan J. Tbshran Jonathan Taylor In ths document we gve supplementary detals to the paper The Soluton Path of the Generalzed Lasso. We use the prefx GL when referrng to equatons, sectons, etc. n the orgnal paper, as n equaton GL-1 or Secton GL-1 ths stands for Generalzed Lasso. 1 Proof of the boundary lemma We prove the boundary lemma when D D 1d, but frst we gve a helpful lemma. Lemma 1. Let T λ denote the functon that truncates outsde of the nterval λ, λ: λ f x < λ T λ x x f x λ λ f x > λ. Then for any λ 0, λ and x, y, T λ0 x T λ y max{ x y, λ 0 λ }. Proof. Suppose wthout a loss of generalty that λ 0 > λ. We enumerate the possble cases: x > λ, y > λ : T λ0 x T λ y λ 0 λ; x λ, y > λ : x > λ, y λ : T λ0 x T λ y x y ; T λ0 x T λ y x y ; x λ, y λ : T λ0 x T λ y x y. The remanng cases follow by symmetry. Proof of the boundary lemma. Our approach for the proof s a lttle unusual: we consder the use of coordnate descent to fnd the soluton û λ, startng at the pont û λ0 as an ntal guess. Because the coordnate updates are especally smple, we can track how the terates change, and hence we can guarantee that û λ and û λ0 are close together. Namely, we show that û λ0 û λ λ 0 λ, whch mples the desred result. Frst we descrbe the coordnate descent updates for fndng the soluton û λ of the dual GL-13, when D D 1d. We note that any lmt pont of the coordnate descent algorthm s ndeed a soluton by Theorem 4.1 of 1. We take u 0 û λ0 as an ntal guess, and cycle through the coordnates n 1

2 the order 1,... n 1. To derve the th update, we fx u j for all j and mnmze over u. Because of the smple structure of D, we only need to consder two terms: mnmze u 1 y u u y u +1 u subject to u λ. Ths s just a quadratc constraned to le n an nterval, and so the th coordnate update s y+1 y + u +1 + u 1 u T λ, where we let u 0 u n 0 for notatonal convenence. Therefore n the frst teraton of the coordnate descent algorthm, we get u 1 y+1 y + u 0 +1 T + u1 1 λ. Usng the fact that û λ0 s tself the soluton correspondng to λ 0, û λ0, u 1 T y+1 y + û λ0,+1 + û λ0, 1 y+1 y + u 0 +1 λ 0 T + u1 1 λ. But ths s max{ û λ0, 1 u 1 1 /, λ 0 λ} by the helpful lemma. Therefore, by nducton, t follows that û λ0 u 1 λ 0 λ. Contnung the same lne of argument shows that û λ0 u k λ 0 λ for all teratons k. Lettng k, we get û λ0 û λ λ 0 λ, as desred. It s mportant to note that f DD T s dagonally domnant, n other words DD T j DD T j for each 1,... m, then the proof of the boundary lemma s smlar to that gven for the 1d fused lasso case. The coordnate updates are now Dy j u T DDT j u j λ DD T, but the rest of the proof remans the same, so the boundary lemma stll holds. Dervaton detals for Algorthm GL- Ths secton s dvded nto two parts: 1 detals for the algorthm s steps at each teraton, and the nserton-deleton lemma. The frst part reles on the nserton-deleton lemma when verfyng the KKT condtons hence establshng the algorthm s correctness, and we present and prove ths lemma n the second part for the sake of clarty. The nserton-deleton lemma also proves that constructed soluton path s contnuous over λ.

3 .1 The algorthm at the kth teraton We propose a soluton û λ fλ, wth γ gλ and α hλ, n order to satsfy the KKT condtons. Frst we defne fλ B λs fλ B D B D B T + y λdb T s. Next we defne gλ and hλ to satsfy the statonarty equaton GL-4. We examne ths n two blocks: the nteror coordnates, B, and the boundary coordnates, B. For the frst block, the equaton s: DD T fλ B Dy B + αγ B λd B D B T s + D B D B T D B D B T + y λdb T s Dy B + αγ B αγ B. 1 Now for the second block: DD T fλ B Dy B + αγ B D B I D B T D B D B T + y λdb T s + αγ B. We want to choose γ gλ and α hλ to make both 1 and equal to zero. Consder defnng hλ D B I D B T D B D B T + y λdb T s 1. If hλ 0 then we let gλ to be any subgradent of fλ. Otherwse we let gλ B 0 and gλ B 1 hλ D B I D B T D B D B T + y λdb T s. Now we must check that the constrants are met wth û λ fλ, γ gλ, and α hλ as defned above. Frst we consder the case λ λ k : GL-5a: Ths holds because fλ k B λ k, and fλ k B λ k by Lemma 3 we delay presentng ths lemma untl Secton.. GL-5b: Ths s true by constructon. GL-5c: Ths s true because fλ k λ k when B, and otherwse hλ k 0. GL-5d and GL-5e: Here we need to show that gλ k s ndeed a subgradent of fλ k. Ths s true by defnton when hλ k 0, so suppose hλ k 0. Note frst that gλ k 1 1 by constructon. Further, sgn gλ k B sgn fλk B by Lemma 4 presented n Secton., and hence gλ k T fλ k fλ k. Ths verfes the subgradent constrant. As we decrease λ, note that only two of the above condtons can break: fλ B λ, or sgn gλ B sgn fλb. The frst one wll break when one of the nteror coordnate paths crosses the boundary. Ths occurs at the next httng tme. Wrtng fλ B a λb and solvng a λb ±λ for / B, we fnd that the httng tmes are D B T + y t ht a b ± 1 D B T + D B T s 3, ± 1

4 where only one of +1 or 1 above wll yeld a value n 0, λ k. For correspondng to the coordnate that left the boundary n the last teraton, the value of ±1 here s fxed at the sgn of the boundary opposte to the one t left. Thus the next httng tme s h max t ht, and the httng coordnate and ts sgn are ht argmax t ht and s ht fh sgn ht. The second condton, sgn gλ B sgn fλb, can be expressed as s D B I D B T D B D B T + y λdb T s 0 for all B. Lettng we can rewrte ths as c s D B I D B T D B D B T + y d s D B I D B T D B D B T + D B T s c λd 0 3 for all B. Because we know that c λ k d 0, the nequalty 3 can only fal at some λ λ k f c and d are both negatve. Accordngly, the leavng tmes are { t leave c /d f c < 0 and d < 0 0 otherwse. Therefore the next leavng tme s and the leavng coordnate and ts sgn are leave argmax t leave For the fnal step of the teraton, we take l max t leave, and s leave sgn λ max{h, l }. fl leave Ths ensures the algorthm s correctness through the kth teraton, because we have satsfed the KKT condtons for all λ λ. In preparaton for the next teraton: f h > l we add the httng coordnate ht to B and append ts sgn sht to s; otherwse we delete the leavng coordnate leave from B and ts sgn s leave from s.. The nserton-deleton lemma The nserton-deleton lemma s mportant because t leads to Lemmas 3 and 4, whch we used n the prevous secton to argue the correctness of our constructed path û λ. Moreover, t drectly gves the contnuty of û λ wth respect to λ.,. 4

5 Whle t may appear complcated, ts concept s pretty smple: the nserton-deleton lemma states that the pont fλ s the same wth f as defned n teraton k or teraton k + 1 n other words, t s the same f we defne f usng the boundary set and sgns from teraton k or teraton k + 1. Note that teraton k could have ended n one of two ways: a coordnate was added to B nserton, or a coordnate was removed from B deleton. Therefore the lemma has two statements, correspondng to these two cases. Lemma The nserton-deleton lemma. At the kth teraton of the algorthm, let B and s denote the boundary coordnates and ther sgns, and let B and s denote the same quanttes at the begnnng of the next teraton. The two possbltes are: 1. Inserton If a coordnate ht the boundary at λ, that s, B and s are gven by addng elements to B and s, then: fλ B fλ ht ht D B T + y λ D B T s λ s ht. 4. Deleton If a coordnate left the boundary at λ, that s, B and s are gven by deletng elements from B and s, then: D fλ B T + DB y λ D B T s B leave λ s leave D B T + DB y λ D B T s. 5 leave Proof. The proof of each part reles on a block matrx decomposton. conceptually dffcult but detaled. We treat the two cases separately. Case 1: Inserton. Let x fλ ht fλ B x1 ht, The arguments are not the left-hand sde of 4. By defnton ht hts the boundary at λ, so that exactly x fλ ht λ s ht. Now we consder x 1. Assume wthout a loss of generalty that ht s the last of the nteror coordnates. Then we can wrte x1 fλ x B D B D B T + y λ D B T s. The pont x 1, x T s the mnmum l norm soluton to the lnear equaton: D B D B T x1 D x B y λ D B T s. Decomposng ths nto blocks, we get D B D B T D ht D B D B T D ht T D ht D ht T x1 x D ht y λd B T s. 5

6 Solvng for x 1 gves x 1 D B D B T + y λ D B T s D ht D B D B T + y λ D B T s + b, T x + b where b null D B T. The value of b can be determned by consderng the squared norm of x 1, x 1 D B D B T + y λ D B T s + b, whch s mnmal when b 0. Ths completes the proof. Case : Deleton. Ths case s smlar but a lttle more complcated. Let D B T + DB y λ D B T s x1 leave x D B T + DB y λ D B T s. leave If we assume wthout a loss of generalty that leave s the largest of all the boundary coordnates, then x 1, x T s the mnmum l norm soluton of the equaton: D BD B T D B D leave D leave D B T D leave T D leave T x1 x D leave y λd B T s. Solvng ths system for x 1 n terms of x yelds x 1 D B D B T + y λ D B T s T D leave x + b, where b null D B T, and as we argued before, we must have b 0 n order for x 1 to have mnmal l norm. Therefore t suffces to show that x λ s leave. To ths end, we contnue the block elmnaton and solve for x. After a bt of algebra, ths s x D leave P T 1D D leave leave P y λ D B T T leave s + λ D leave s, 6 where P P null. But by defnton of leave, D leave P y λ D B T s 0. Furthermore D leave P T D leave s just a scalar, and t s nonzero otherwse ths mples that P T D leave 0 as P s a projecton matrx, and so λ 0 by defnton of the leavng tme, whch makes the result trval. Therefore 6 becomes x λ s leave, whch completes the proof. Now we gve Lemmas 3 and 4, whch we used n Secton.1 to verfy the KKT condtons. Lemma 3. At the kth teraton of the algorthm, fλ k B λ k. Proof. The proof s a straghtforward applcaton of nducton and Lemma. At k 0 ths s trvally true snce λ 0. Now assume the statement holds for teraton k. Dependng on whether a coordnate ht or left the boundary n teraton k, the statement for k + 1 s verfed by takng the l norm of the rght-hand sde of 4 or 5, respectvely. 6

7 Lemma 4. At the kth teraton of the algorthm, sgn gλ k B sgn fλk B. Proof. Agan we use nducton. At k 0 ths s trvally true because B. Suppose that the statement holds for all teratons k. Gven that we have already proved Lemma 3, the nductve hypothess s really that the constructed path û λ s the soluton path for all λ λ. Let B, s, f, and g refer to the versons defned at the begnnng of teraton k + 1. By Lemma we know that û λ fλ s ndeed the soluton at λ. Hence ˆβ λ y D T fλ s ndeed the prmal soluton at λ. Notng that gλ B D B ˆβλ, and recallng the relatonshp GL-15, we have sgn gλ B sgn fλ B. 3 Proof of the prmal-dual correspondence for a general D Here we prove that the prmal soluton changes slope at λ f and only f the null space of D B changes from teratons k to k +1. Agan we use the notaton B, s and B, s to denote the boundary set and sgns at teraton k, respectvely k + 1. Ths was clamed n Secton GL-6. for the case X I, and later n Secton GL-7.1 for a general X wth rankx p. We dvde our proof nto two parts, accordngly. 3.1 The case X I Consder the vector of coordnate-wse slopes of the soluton path ˆβ λ, as a functon of λ. Usng GL-33, the lmts of ths as λ λ from above and below are a + P null D B T s and a P null D B T s, 7 respectvely. Suppose that a coordnate ht the boundary at λ. Then we have D B T s D B T s + T ht D ht s, and f nulld B nulld B then a P null D B T T ht s + P null D ht s a +, T ht where the dentty P null D ht s 0 follows from T ht D ht s rowd B nulld B. A smlar argument holds n the case that a coordnate left the boundary at λ. Therefore the slope of ˆβ λ changes at λ only f nulld B nulld B. The converse statement, that the slope of ˆβ λ changes at λ f nulld B nulld B, s only true for Lebesgue almost every y R n. Hence, for any reasonable model of the data y, t holds wth probablty one. To show ths, we frst note that the lmts of ˆβ λ as λ λ from above and below can be expressed as ˆβ + P null y λ a + and ˆβ P null y λ a, respectvely, where a, a + are defned n 7. By the contnuty of ˆβ λ, we know that ˆβ + ˆβ. Suppose that nulld B nulld B. Then these two lnear spaces dffer n dmenson by one dependng on whether or not a coordnate ht or left the boundary at λ. Hence P null y P null y for almost every y R n. Therefore, for any such y, we must have a + a n order to satsfy ˆβ + ˆβ. 7

8 3. The case of a general X, rankx p The proof s qute smlar. Now the lmtng slopes are, from equaton GL-39, a + X + P null D B T s and a X + P null D B T s. Recallng that D B D B X +, we have nulld B nulld B null D B null D B, whch mples that a a +, usng the same arguments as we gave for the case X I. The converse s agan true for almost every y R n. Ths s because nulld B nulld B null D B null D B, as X + has rank p, whch mples that a + a for almost every y usng smlar arguments to those gven above. 4 Proof of Lemma GL-3 Note that for a set B {1,... m}, the matrx D B P null may have some rows that are entrely zero. We let ZB denote the set of such rows. Now defne N λ { x : D P nullx λd P nulld B T s } R n, B,s B\ZB where the frst unon s taken over all subsets B {1,... m} and all sgn vectors s { 1, 1} B. Note that N λ s a fnte unon of affne subspaces of dmenson n 1, and hence has measure zero. Ths establshes part a of the lemma. Now, for y / N λ, let û λ y be a dual soluton wth boundary set B and sgns s. We show that: 1. there s a neghborhood U of y such that for any y U, there exsts a dual soluton û λ y wth the same boundary set B and sgns s;. f u λ y s another dual soluton at y, wth a dfferent boundary set B and sgns s, then λd B T s + rowd B λd B T s + rowd B. If we show these two statements then ths would mply part b of the lemma. 4.1 Proof of statement 1 Frst note that we can rewrte the optmalty condtons GL-4 and GL-5a GL-5e for our dual problem as û λ λ 8 Dy D T û λ K, 9 where K R m s the cone generated by { sgnû λ, e : {j : u j λ} } and e denotes the th standard bass vector. Focusng frst on the pont y, let a D y D T û λ y K, and note that K s generated by {s e : B}. Note also that a B 0, and usng the fact that û λ,b y λ s, ths means D B y λdb T s D B D B T û λ, B y 0. 8

9 Hence we can wrte the dual soluton û λ y as û λ,b y λ s û λ, B y D B D B T + y λdb T s + b, where b null D B T. By defnton of the boundary set, û λ, B y < λ. Now we can also wrte a B D B P null y λdb T s. Some rows of the matrx D B P null may be entrely zero; recall that these are denoted by ZB. Snce y / N λ, we know that a 0 for all B \ ZB. For a new pont y, consder defnng û λ y as û λ,b y λ s û λ, B y D B D B T + y λd B T s + b. By contnuty of the affne mappng note that B, s, b are fxed x D B D B T + x λdb T s + b, there exsts a neghborhood U 1 of y such that û λ, B y < λ for all y n U 1. Ths establshes the frst optmalty condton 8 and shows that û λ y has boundary set B and sgns s, for all y U 1. To establsh the second optmalty condton 9, we must check that a DP null y λd B T s K. Well a B 0 and also a ZB 0. By contnuty of the affne mappng agan B, s are fxed x D B\ZB P null x λdb T s, there exsts another neghborhood U of y such that a 0 and further sgna sgna for all B \ ZB and y U. As a K, ths means that a K for all y U. Lettng U U 1 U, we have verfed that û λ y has boundary set B and sgns s, and s ndeed a dual soluton, for all y U. 4. Proof of statement Gven another dual soluton u λ y at y wth a dfferent boundary set B and sgns s, we know from statement 1 that there s a neghborhood U of y such that u λ,b y λ s u λ, B y D B D B T + y λd B T s + b s a dual soluton for all y U, where b null D B T. By unqueness of the dual ft, we have λd B T s + P row y λd B T s λd B T s + P row y λd B T s for all y U U, and therefore References λd B T s + rowd B λd B T s + rowd B. 1 Tseng, P. 001, Convergence of a block coordnate descent method for nondfferentable mnmzaton, Journal of Optmzaton Theory and Applcatons 1093,

Lectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix

Lectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could

More information

Lecture 20: November 7

Lecture 20: November 7 0-725/36-725: Convex Optmzaton Fall 205 Lecturer: Ryan Tbshran Lecture 20: November 7 Scrbes: Varsha Chnnaobreddy, Joon Sk Km, Lngyao Zhang Note: LaTeX template courtesy of UC Berkeley EECS dept. Dsclamer:

More information

Affine transformations and convexity

Affine transformations and convexity Affne transformatons and convexty The purpose of ths document s to prove some basc propertes of affne transformatons nvolvng convex sets. Here are a few onlne references for background nformaton: http://math.ucr.edu/

More information

APPENDIX A Some Linear Algebra

APPENDIX A Some Linear Algebra APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,

More information

More metrics on cartesian products

More metrics on cartesian products More metrcs on cartesan products If (X, d ) are metrc spaces for 1 n, then n Secton II4 of the lecture notes we defned three metrcs on X whose underlyng topologes are the product topology The purpose of

More information

MMA and GCMMA two methods for nonlinear optimization

MMA and GCMMA two methods for nonlinear optimization MMA and GCMMA two methods for nonlnear optmzaton Krster Svanberg Optmzaton and Systems Theory, KTH, Stockholm, Sweden. krlle@math.kth.se Ths note descrbes the algorthms used n the author s 2007 mplementatons

More information

College of Computer & Information Science Fall 2009 Northeastern University 20 October 2009

College of Computer & Information Science Fall 2009 Northeastern University 20 October 2009 College of Computer & Informaton Scence Fall 2009 Northeastern Unversty 20 October 2009 CS7880: Algorthmc Power Tools Scrbe: Jan Wen and Laura Poplawsk Lecture Outlne: Prmal-dual schema Network Desgn:

More information

Feature Selection: Part 1

Feature Selection: Part 1 CSE 546: Machne Learnng Lecture 5 Feature Selecton: Part 1 Instructor: Sham Kakade 1 Regresson n the hgh dmensonal settng How do we learn when the number of features d s greater than the sample sze n?

More information

Lecture 10 Support Vector Machines II

Lecture 10 Support Vector Machines II Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed

More information

princeton univ. F 17 cos 521: Advanced Algorithm Design Lecture 7: LP Duality Lecturer: Matt Weinberg

princeton univ. F 17 cos 521: Advanced Algorithm Design Lecture 7: LP Duality Lecturer: Matt Weinberg prnceton unv. F 17 cos 521: Advanced Algorthm Desgn Lecture 7: LP Dualty Lecturer: Matt Wenberg Scrbe: LP Dualty s an extremely useful tool for analyzng structural propertes of lnear programs. Whle there

More information

Inner Product. Euclidean Space. Orthonormal Basis. Orthogonal

Inner Product. Euclidean Space. Orthonormal Basis. Orthogonal Inner Product Defnton 1 () A Eucldean space s a fnte-dmensonal vector space over the reals R, wth an nner product,. Defnton 2 (Inner Product) An nner product, on a real vector space X s a symmetrc, blnear,

More information

Difference Equations

Difference Equations Dfference Equatons c Jan Vrbk 1 Bascs Suppose a sequence of numbers, say a 0,a 1,a,a 3,... s defned by a certan general relatonshp between, say, three consecutve values of the sequence, e.g. a + +3a +1

More information

Foundations of Arithmetic

Foundations of Arithmetic Foundatons of Arthmetc Notaton We shall denote the sum and product of numbers n the usual notaton as a 2 + a 2 + a 3 + + a = a, a 1 a 2 a 3 a = a The notaton a b means a dvdes b,.e. ac = b where c s an

More information

n α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0

n α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0 MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector

More information

8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS

8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 493 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces you have studed thus far n the text are real vector spaces because the scalars

More information

Lecture 12: Discrete Laplacian

Lecture 12: Discrete Laplacian Lecture 12: Dscrete Laplacan Scrbe: Tanye Lu Our goal s to come up wth a dscrete verson of Laplacan operator for trangulated surfaces, so that we can use t n practce to solve related problems We are mostly

More information

2.3 Nilpotent endomorphisms

2.3 Nilpotent endomorphisms s a block dagonal matrx, wth A Mat dm U (C) In fact, we can assume that B = B 1 B k, wth B an ordered bass of U, and that A = [f U ] B, where f U : U U s the restrcton of f to U 40 23 Nlpotent endomorphsms

More information

Solutions to exam in SF1811 Optimization, Jan 14, 2015

Solutions to exam in SF1811 Optimization, Jan 14, 2015 Solutons to exam n SF8 Optmzaton, Jan 4, 25 3 3 O------O -4 \ / \ / The network: \/ where all lnks go from left to rght. /\ / \ / \ 6 O------O -5 2 4.(a) Let x = ( x 3, x 4, x 23, x 24 ) T, where the varable

More information

The Geometry of Logit and Probit

The Geometry of Logit and Probit The Geometry of Logt and Probt Ths short note s meant as a supplement to Chapters and 3 of Spatal Models of Parlamentary Votng and the notaton and reference to fgures n the text below s to those two chapters.

More information

Assortment Optimization under MNL

Assortment Optimization under MNL Assortment Optmzaton under MNL Haotan Song Aprl 30, 2017 1 Introducton The assortment optmzaton problem ams to fnd the revenue-maxmzng assortment of products to offer when the prces of products are fxed.

More information

Random Walks on Digraphs

Random Walks on Digraphs Random Walks on Dgraphs J. J. P. Veerman October 23, 27 Introducton Let V = {, n} be a vertex set and S a non-negatve row-stochastc matrx (.e. rows sum to ). V and S defne a dgraph G = G(V, S) and a drected

More information

Chapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems

Chapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons

More information

1 GSW Iterative Techniques for y = Ax

1 GSW Iterative Techniques for y = Ax 1 for y = A I m gong to cheat here. here are a lot of teratve technques that can be used to solve the general case of a set of smultaneous equatons (wrtten n the matr form as y = A), but ths chapter sn

More information

Module 9. Lecture 6. Duality in Assignment Problems

Module 9. Lecture 6. Duality in Assignment Problems Module 9 1 Lecture 6 Dualty n Assgnment Problems In ths lecture we attempt to answer few other mportant questons posed n earler lecture for (AP) and see how some of them can be explaned through the concept

More information

Yong Joon Ryang. 1. Introduction Consider the multicommodity transportation problem with convex quadratic cost function. 1 2 (x x0 ) T Q(x x 0 )

Yong Joon Ryang. 1. Introduction Consider the multicommodity transportation problem with convex quadratic cost function. 1 2 (x x0 ) T Q(x x 0 ) Kangweon-Kyungk Math. Jour. 4 1996), No. 1, pp. 7 16 AN ITERATIVE ROW-ACTION METHOD FOR MULTICOMMODITY TRANSPORTATION PROBLEMS Yong Joon Ryang Abstract. The optmzaton problems wth quadratc constrants often

More information

3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X

3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number

More information

Singular Value Decomposition: Theory and Applications

Singular Value Decomposition: Theory and Applications Sngular Value Decomposton: Theory and Applcatons Danel Khashab Sprng 2015 Last Update: March 2, 2015 1 Introducton A = UDV where columns of U and V are orthonormal and matrx D s dagonal wth postve real

More information

SELECTED SOLUTIONS, SECTION (Weak duality) Prove that the primal and dual values p and d defined by equations (4.3.2) and (4.3.3) satisfy p d.

SELECTED SOLUTIONS, SECTION (Weak duality) Prove that the primal and dual values p and d defined by equations (4.3.2) and (4.3.3) satisfy p d. SELECTED SOLUTIONS, SECTION 4.3 1. Weak dualty Prove that the prmal and dual values p and d defned by equatons 4.3. and 4.3.3 satsfy p d. We consder an optmzaton problem of the form The Lagrangan for ths

More information

Stanford University CS359G: Graph Partitioning and Expanders Handout 4 Luca Trevisan January 13, 2011

Stanford University CS359G: Graph Partitioning and Expanders Handout 4 Luca Trevisan January 13, 2011 Stanford Unversty CS359G: Graph Parttonng and Expanders Handout 4 Luca Trevsan January 3, 0 Lecture 4 In whch we prove the dffcult drecton of Cheeger s nequalty. As n the past lectures, consder an undrected

More information

= = = (a) Use the MATLAB command rref to solve the system. (b) Let A be the coefficient matrix and B be the right-hand side of the system.

= = = (a) Use the MATLAB command rref to solve the system. (b) Let A be the coefficient matrix and B be the right-hand side of the system. Chapter Matlab Exercses Chapter Matlab Exercses. Consder the lnear system of Example n Secton.. x x x y z y y z (a) Use the MATLAB command rref to solve the system. (b) Let A be the coeffcent matrx and

More information

COS 521: Advanced Algorithms Game Theory and Linear Programming

COS 521: Advanced Algorithms Game Theory and Linear Programming COS 521: Advanced Algorthms Game Theory and Lnear Programmng Moses Charkar February 27, 2013 In these notes, we ntroduce some basc concepts n game theory and lnear programmng (LP). We show a connecton

More information

Linear, affine, and convex sets and hulls In the sequel, unless otherwise specified, X will denote a real vector space.

Linear, affine, and convex sets and hulls In the sequel, unless otherwise specified, X will denote a real vector space. Lnear, affne, and convex sets and hulls In the sequel, unless otherwse specfed, X wll denote a real vector space. Lnes and segments. Gven two ponts x, y X, we defne xy = {x + t(y x) : t R} = {(1 t)x +

More information

CSCE 790S Background Results

CSCE 790S Background Results CSCE 790S Background Results Stephen A. Fenner September 8, 011 Abstract These results are background to the course CSCE 790S/CSCE 790B, Quantum Computaton and Informaton (Sprng 007 and Fall 011). Each

More information

Lecture Notes on Linear Regression

Lecture Notes on Linear Regression Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume

More information

6.854J / J Advanced Algorithms Fall 2008

6.854J / J Advanced Algorithms Fall 2008 MIT OpenCourseWare http://ocw.mt.edu 6.854J / 18.415J Advanced Algorthms Fall 2008 For nformaton about ctng these materals or our Terms of Use, vst: http://ocw.mt.edu/terms. 18.415/6.854 Advanced Algorthms

More information

MATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS

MATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS MATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS These are nformal notes whch cover some of the materal whch s not n the course book. The man purpose s to gve a number of nontrval examples

More information

FINITELY-GENERATED MODULES OVER A PRINCIPAL IDEAL DOMAIN

FINITELY-GENERATED MODULES OVER A PRINCIPAL IDEAL DOMAIN FINITELY-GENERTED MODULES OVER PRINCIPL IDEL DOMIN EMMNUEL KOWLSKI Throughout ths note, s a prncpal deal doman. We recall the classfcaton theorem: Theorem 1. Let M be a fntely-generated -module. (1) There

More information

Module 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

Module 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:

More information

Section 3.6 Complex Zeros

Section 3.6 Complex Zeros 04 Chapter Secton 6 Comple Zeros When fndng the zeros of polynomals, at some pont you're faced wth the problem Whle there are clearly no real numbers that are solutons to ths equaton, leavng thngs there

More information

Review of Taylor Series. Read Section 1.2

Review of Taylor Series. Read Section 1.2 Revew of Taylor Seres Read Secton 1.2 1 Power Seres A power seres about c s an nfnte seres of the form k = 0 k a ( x c) = a + a ( x c) + a ( x c) + a ( x c) k 2 3 0 1 2 3 + In many cases, c = 0, and the

More information

The Order Relation and Trace Inequalities for. Hermitian Operators

The Order Relation and Trace Inequalities for. Hermitian Operators Internatonal Mathematcal Forum, Vol 3, 08, no, 507-57 HIKARI Ltd, wwwm-hkarcom https://doorg/0988/mf088055 The Order Relaton and Trace Inequaltes for Hermtan Operators Y Huang School of Informaton Scence

More information

MA 323 Geometric Modelling Course Notes: Day 13 Bezier Curves & Bernstein Polynomials

MA 323 Geometric Modelling Course Notes: Day 13 Bezier Curves & Bernstein Polynomials MA 323 Geometrc Modellng Course Notes: Day 13 Bezer Curves & Bernsten Polynomals Davd L. Fnn Over the past few days, we have looked at de Casteljau s algorthm for generatng a polynomal curve, and we have

More information

BOUNDEDNESS OF THE RIESZ TRANSFORM WITH MATRIX A 2 WEIGHTS

BOUNDEDNESS OF THE RIESZ TRANSFORM WITH MATRIX A 2 WEIGHTS BOUNDEDNESS OF THE IESZ TANSFOM WITH MATIX A WEIGHTS Introducton Let L = L ( n, be the functon space wth norm (ˆ f L = f(x C dx d < For a d d matrx valued functon W : wth W (x postve sem-defnte for all

More information

Solutions HW #2. minimize. Ax = b. Give the dual problem, and make the implicit equality constraints explicit. Solution.

Solutions HW #2. minimize. Ax = b. Give the dual problem, and make the implicit equality constraints explicit. Solution. Solutons HW #2 Dual of general LP. Fnd the dual functon of the LP mnmze subject to c T x Gx h Ax = b. Gve the dual problem, and make the mplct equalty constrants explct. Soluton. 1. The Lagrangan s L(x,

More information

1 Matrix representations of canonical matrices

1 Matrix representations of canonical matrices 1 Matrx representatons of canoncal matrces 2-d rotaton around the orgn: ( ) cos θ sn θ R 0 = sn θ cos θ 3-d rotaton around the x-axs: R x = 1 0 0 0 cos θ sn θ 0 sn θ cos θ 3-d rotaton around the y-axs:

More information

Exercise Solutions to Real Analysis

Exercise Solutions to Real Analysis xercse Solutons to Real Analyss Note: References refer to H. L. Royden, Real Analyss xersze 1. Gven any set A any ɛ > 0, there s an open set O such that A O m O m A + ɛ. Soluton 1. If m A =, then there

More information

U.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017

U.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017 U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that

More information

12 MATH 101A: ALGEBRA I, PART C: MULTILINEAR ALGEBRA. 4. Tensor product

12 MATH 101A: ALGEBRA I, PART C: MULTILINEAR ALGEBRA. 4. Tensor product 12 MATH 101A: ALGEBRA I, PART C: MULTILINEAR ALGEBRA Here s an outlne of what I dd: (1) categorcal defnton (2) constructon (3) lst of basc propertes (4) dstrbutve property (5) rght exactness (6) localzaton

More information

CSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography

CSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography CSc 6974 and ECSE 6966 Math. Tech. for Vson, Graphcs and Robotcs Lecture 21, Aprl 17, 2006 Estmatng A Plane Homography Overvew We contnue wth a dscusson of the major ssues, usng estmaton of plane projectve

More information

Problem Do any of the following determine homomorphisms from GL n (C) to GL n (C)?

Problem Do any of the following determine homomorphisms from GL n (C) to GL n (C)? Homework 8 solutons. Problem 16.1. Whch of the followng defne homomomorphsms from C\{0} to C\{0}? Answer. a) f 1 : z z Yes, f 1 s a homomorphsm. We have that z s the complex conjugate of z. If z 1,z 2

More information

Appendix for Causal Interaction in Factorial Experiments: Application to Conjoint Analysis

Appendix for Causal Interaction in Factorial Experiments: Application to Conjoint Analysis A Appendx for Causal Interacton n Factoral Experments: Applcaton to Conjont Analyss Mathematcal Appendx: Proofs of Theorems A. Lemmas Below, we descrbe all the lemmas, whch are used to prove the man theorems

More information

Section 8.3 Polar Form of Complex Numbers

Section 8.3 Polar Form of Complex Numbers 80 Chapter 8 Secton 8 Polar Form of Complex Numbers From prevous classes, you may have encountered magnary numbers the square roots of negatve numbers and, more generally, complex numbers whch are the

More information

A 2D Bounded Linear Program (H,c) 2D Linear Programming

A 2D Bounded Linear Program (H,c) 2D Linear Programming A 2D Bounded Lnear Program (H,c) h 3 v h 8 h 5 c h 4 h h 6 h 7 h 2 2D Lnear Programmng C s a polygonal regon, the ntersecton of n halfplanes. (H, c) s nfeasble, as C s empty. Feasble regon C s unbounded

More information

Problem Solving in Math (Math 43900) Fall 2013

Problem Solving in Math (Math 43900) Fall 2013 Problem Solvng n Math (Math 43900) Fall 2013 Week four (September 17) solutons Instructor: Davd Galvn 1. Let a and b be two nteger for whch a b s dvsble by 3. Prove that a 3 b 3 s dvsble by 9. Soluton:

More information

THE CHINESE REMAINDER THEOREM. We should thank the Chinese for their wonderful remainder theorem. Glenn Stevens

THE CHINESE REMAINDER THEOREM. We should thank the Chinese for their wonderful remainder theorem. Glenn Stevens THE CHINESE REMAINDER THEOREM KEITH CONRAD We should thank the Chnese for ther wonderful remander theorem. Glenn Stevens 1. Introducton The Chnese remander theorem says we can unquely solve any par of

More information

Problem Set 9 Solutions

Problem Set 9 Solutions Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem

More information

Formulas for the Determinant

Formulas for the Determinant page 224 224 CHAPTER 3 Determnants e t te t e 2t 38 A = e t 2te t e 2t e t te t 2e 2t 39 If 123 A = 345, 456 compute the matrx product A adj(a) What can you conclude about det(a)? For Problems 40 43, use

More information

Salmon: Lectures on partial differential equations. Consider the general linear, second-order PDE in the form. ,x 2

Salmon: Lectures on partial differential equations. Consider the general linear, second-order PDE in the form. ,x 2 Salmon: Lectures on partal dfferental equatons 5. Classfcaton of second-order equatons There are general methods for classfyng hgher-order partal dfferental equatons. One s very general (applyng even to

More information

20. Mon, Oct. 13 What we have done so far corresponds roughly to Chapters 2 & 3 of Lee. Now we turn to Chapter 4. The first idea is connectedness.

20. Mon, Oct. 13 What we have done so far corresponds roughly to Chapters 2 & 3 of Lee. Now we turn to Chapter 4. The first idea is connectedness. 20. Mon, Oct. 13 What we have done so far corresponds roughly to Chapters 2 & 3 of Lee. Now we turn to Chapter 4. The frst dea s connectedness. Essentally, we want to say that a space cannot be decomposed

More information

For now, let us focus on a specific model of neurons. These are simplified from reality but can achieve remarkable results.

For now, let us focus on a specific model of neurons. These are simplified from reality but can achieve remarkable results. Neural Networks : Dervaton compled by Alvn Wan from Professor Jtendra Malk s lecture Ths type of computaton s called deep learnng and s the most popular method for many problems, such as computer vson

More information

Example: (13320, 22140) =? Solution #1: The divisors of are 1, 2, 3, 4, 5, 6, 9, 10, 12, 15, 18, 20, 27, 30, 36, 41,

Example: (13320, 22140) =? Solution #1: The divisors of are 1, 2, 3, 4, 5, 6, 9, 10, 12, 15, 18, 20, 27, 30, 36, 41, The greatest common dvsor of two ntegers a and b (not both zero) s the largest nteger whch s a common factor of both a and b. We denote ths number by gcd(a, b), or smply (a, b) when there s no confuson

More information

Learning Theory: Lecture Notes

Learning Theory: Lecture Notes Learnng Theory: Lecture Notes Lecturer: Kamalka Chaudhur Scrbe: Qush Wang October 27, 2012 1 The Agnostc PAC Model Recall that one of the constrants of the PAC model s that the data dstrbuton has to be

More information

Chapter Newton s Method

Chapter Newton s Method Chapter 9. Newton s Method After readng ths chapter, you should be able to:. Understand how Newton s method s dfferent from the Golden Secton Search method. Understand how Newton s method works 3. Solve

More information

a b a In case b 0, a being divisible by b is the same as to say that

a b a In case b 0, a being divisible by b is the same as to say that Secton 6.2 Dvsblty among the ntegers An nteger a ε s dvsble by b ε f there s an nteger c ε such that a = bc. Note that s dvsble by any nteger b, snce = b. On the other hand, a s dvsble by only f a = :

More information

Maximizing the number of nonnegative subsets

Maximizing the number of nonnegative subsets Maxmzng the number of nonnegatve subsets Noga Alon Hao Huang December 1, 213 Abstract Gven a set of n real numbers, f the sum of elements of every subset of sze larger than k s negatve, what s the maxmum

More information

General viscosity iterative method for a sequence of quasi-nonexpansive mappings

General viscosity iterative method for a sequence of quasi-nonexpansive mappings Avalable onlne at www.tjnsa.com J. Nonlnear Sc. Appl. 9 (2016), 5672 5682 Research Artcle General vscosty teratve method for a sequence of quas-nonexpansve mappngs Cuje Zhang, Ynan Wang College of Scence,

More information

MAT 578 Functional Analysis

MAT 578 Functional Analysis MAT 578 Functonal Analyss John Qugg Fall 2008 Locally convex spaces revsed September 6, 2008 Ths secton establshes the fundamental propertes of locally convex spaces. Acknowledgment: although I wrote these

More information

Report on Image warping

Report on Image warping Report on Image warpng Xuan Ne, Dec. 20, 2004 Ths document summarzed the algorthms of our mage warpng soluton for further study, and there s a detaled descrpton about the mplementaton of these algorthms.

More information

On a direct solver for linear least squares problems

On a direct solver for linear least squares problems ISSN 2066-6594 Ann. Acad. Rom. Sc. Ser. Math. Appl. Vol. 8, No. 2/2016 On a drect solver for lnear least squares problems Constantn Popa Abstract The Null Space (NS) algorthm s a drect solver for lnear

More information

Lecture Space-Bounded Derandomization

Lecture Space-Bounded Derandomization Notes on Complexty Theory Last updated: October, 2008 Jonathan Katz Lecture Space-Bounded Derandomzaton 1 Space-Bounded Derandomzaton We now dscuss derandomzaton of space-bounded algorthms. Here non-trval

More information

arxiv: v2 [math.st] 4 Nov 2012

arxiv: v2 [math.st] 4 Nov 2012 The Lasso Problem and Unqueness Ryan J. Tbshran arxv:1206.0313v2 [math.st] 4 Nov 2012 Carnege Mellon Unversty Abstract The lasso s a popular tool for sparse lnear regresson, especally for problems n whch

More information

Lecture 20: Lift and Project, SDP Duality. Today we will study the Lift and Project method. Then we will prove the SDP duality theorem.

Lecture 20: Lift and Project, SDP Duality. Today we will study the Lift and Project method. Then we will prove the SDP duality theorem. prnceton u. sp 02 cos 598B: algorthms and complexty Lecture 20: Lft and Project, SDP Dualty Lecturer: Sanjeev Arora Scrbe:Yury Makarychev Today we wll study the Lft and Project method. Then we wll prove

More information

find (x): given element x, return the canonical element of the set containing x;

find (x): given element x, return the canonical element of the set containing x; COS 43 Sprng, 009 Dsjont Set Unon Problem: Mantan a collecton of dsjont sets. Two operatons: fnd the set contanng a gven element; unte two sets nto one (destructvely). Approach: Canoncal element method:

More information

STAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 16

STAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 16 STAT 39: MATHEMATICAL COMPUTATIONS I FALL 218 LECTURE 16 1 why teratve methods f we have a lnear system Ax = b where A s very, very large but s ether sparse or structured (eg, banded, Toepltz, banded plus

More information

Convexity preserving interpolation by splines of arbitrary degree

Convexity preserving interpolation by splines of arbitrary degree Computer Scence Journal of Moldova, vol.18, no.1(52), 2010 Convexty preservng nterpolaton by splnes of arbtrary degree Igor Verlan Abstract In the present paper an algorthm of C 2 nterpolaton of dscrete

More information

Math 217 Fall 2013 Homework 2 Solutions

Math 217 Fall 2013 Homework 2 Solutions Math 17 Fall 013 Homework Solutons Due Thursday Sept. 6, 013 5pm Ths homework conssts of 6 problems of 5 ponts each. The total s 30. You need to fully justfy your answer prove that your functon ndeed has

More information

Grover s Algorithm + Quantum Zeno Effect + Vaidman

Grover s Algorithm + Quantum Zeno Effect + Vaidman Grover s Algorthm + Quantum Zeno Effect + Vadman CS 294-2 Bomb 10/12/04 Fall 2004 Lecture 11 Grover s algorthm Recall that Grover s algorthm for searchng over a space of sze wors as follows: consder the

More information

Kernel Methods and SVMs Extension

Kernel Methods and SVMs Extension Kernel Methods and SVMs Extenson The purpose of ths document s to revew materal covered n Machne Learnng 1 Supervsed Learnng regardng support vector machnes (SVMs). Ths document also provdes a general

More information

Linear Feature Engineering 11

Linear Feature Engineering 11 Lnear Feature Engneerng 11 2 Least-Squares 2.1 Smple least-squares Consder the followng dataset. We have a bunch of nputs x and correspondng outputs y. The partcular values n ths dataset are x y 0.23 0.19

More information

Additional Codes using Finite Difference Method. 1 HJB Equation for Consumption-Saving Problem Without Uncertainty

Additional Codes using Finite Difference Method. 1 HJB Equation for Consumption-Saving Problem Without Uncertainty Addtonal Codes usng Fnte Dfference Method Benamn Moll 1 HJB Equaton for Consumpton-Savng Problem Wthout Uncertanty Before consderng the case wth stochastc ncome n http://www.prnceton.edu/~moll/ HACTproect/HACT_Numercal_Appendx.pdf,

More information

REAL ANALYSIS I HOMEWORK 1

REAL ANALYSIS I HOMEWORK 1 REAL ANALYSIS I HOMEWORK CİHAN BAHRAN The questons are from Tao s text. Exercse 0.0.. If (x α ) α A s a collecton of numbers x α [0, + ] such that x α

More information

Errors for Linear Systems

Errors for Linear Systems Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch

More information

Some modelling aspects for the Matlab implementation of MMA

Some modelling aspects for the Matlab implementation of MMA Some modellng aspects for the Matlab mplementaton of MMA Krster Svanberg krlle@math.kth.se Optmzaton and Systems Theory Department of Mathematcs KTH, SE 10044 Stockholm September 2004 1. Consdered optmzaton

More information

SL n (F ) Equals its Own Derived Group

SL n (F ) Equals its Own Derived Group Internatonal Journal of Algebra, Vol. 2, 2008, no. 12, 585-594 SL n (F ) Equals ts Own Derved Group Jorge Macel BMCC-The Cty Unversty of New York, CUNY 199 Chambers street, New York, NY 10007, USA macel@cms.nyu.edu

More information

Affine and Riemannian Connections

Affine and Riemannian Connections Affne and Remannan Connectons Semnar Remannan Geometry Summer Term 2015 Prof Dr Anna Wenhard and Dr Gye-Seon Lee Jakob Ullmann Notaton: X(M) space of smooth vector felds on M D(M) space of smooth functons

More information

Linear Approximation with Regularization and Moving Least Squares

Linear Approximation with Regularization and Moving Least Squares Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...

More information

Suggested solutions for the exam in SF2863 Systems Engineering. June 12,

Suggested solutions for the exam in SF2863 Systems Engineering. June 12, Suggested solutons for the exam n SF2863 Systems Engneerng. June 12, 2012 14.00 19.00 Examner: Per Enqvst, phone: 790 62 98 1. We can thnk of the farm as a Jackson network. The strawberry feld s modelled

More information

Determinants Containing Powers of Generalized Fibonacci Numbers

Determinants Containing Powers of Generalized Fibonacci Numbers 1 2 3 47 6 23 11 Journal of Integer Sequences, Vol 19 (2016), Artcle 1671 Determnants Contanng Powers of Generalzed Fbonacc Numbers Aram Tangboonduangjt and Thotsaporn Thanatpanonda Mahdol Unversty Internatonal

More information

SCALED STEEPEST DESCENT METHOD

SCALED STEEPEST DESCENT METHOD SCALED SEEPES DESCEN MEHOD We want to solve the same problem 0. mn x R n f x + x usng another approach, whch we called Scaled Steepest Descent method SSD n short. We propose to solve 0. by tang the safeguarded

More information

Maximal Margin Classifier

Maximal Margin Classifier CS81B/Stat41B: Advanced Topcs n Learnng & Decson Makng Mamal Margn Classfer Lecturer: Mchael Jordan Scrbes: Jana van Greunen Corrected verson - /1/004 1 References/Recommended Readng 1.1 Webstes www.kernel-machnes.org

More information

The Minimum Universal Cost Flow in an Infeasible Flow Network

The Minimum Universal Cost Flow in an Infeasible Flow Network Journal of Scences, Islamc Republc of Iran 17(2): 175-180 (2006) Unversty of Tehran, ISSN 1016-1104 http://jscencesutacr The Mnmum Unversal Cost Flow n an Infeasble Flow Network H Saleh Fathabad * M Bagheran

More information

On the Global Linear Convergence of the ADMM with Multi-Block Variables

On the Global Linear Convergence of the ADMM with Multi-Block Variables On the Global Lnear Convergence of the ADMM wth Mult-Block Varables Tany Ln Shqan Ma Shuzhong Zhang May 31, 01 Abstract The alternatng drecton method of multplers ADMM has been wdely used for solvng structured

More information

Lecture 3. Ax x i a i. i i

Lecture 3. Ax x i a i. i i 18.409 The Behavor of Algorthms n Practce 2/14/2 Lecturer: Dan Spelman Lecture 3 Scrbe: Arvnd Sankar 1 Largest sngular value In order to bound the condton number, we need an upper bound on the largest

More information

10. Canonical Transformations Michael Fowler

10. Canonical Transformations Michael Fowler 10. Canoncal Transformatons Mchael Fowler Pont Transformatons It s clear that Lagrange s equatons are correct for any reasonable choce of parameters labelng the system confguraton. Let s call our frst

More information

DIFFERENTIAL FORMS BRIAN OSSERMAN

DIFFERENTIAL FORMS BRIAN OSSERMAN DIFFERENTIAL FORMS BRIAN OSSERMAN Dfferentals are an mportant topc n algebrac geometry, allowng the use of some classcal geometrc arguments n the context of varetes over any feld. We wll use them to defne

More information

CS 229, Public Course Problem Set #3 Solutions: Learning Theory and Unsupervised Learning

CS 229, Public Course Problem Set #3 Solutions: Learning Theory and Unsupervised Learning CS9 Problem Set #3 Solutons CS 9, Publc Course Problem Set #3 Solutons: Learnng Theory and Unsupervsed Learnng. Unform convergence and Model Selecton In ths problem, we wll prove a bound on the error of

More information

Complete subgraphs in multipartite graphs

Complete subgraphs in multipartite graphs Complete subgraphs n multpartte graphs FLORIAN PFENDER Unverstät Rostock, Insttut für Mathematk D-18057 Rostock, Germany Floran.Pfender@un-rostock.de Abstract Turán s Theorem states that every graph G

More information

ON A DETERMINATION OF THE INITIAL FUNCTIONS FROM THE OBSERVED VALUES OF THE BOUNDARY FUNCTIONS FOR THE SECOND-ORDER HYPERBOLIC EQUATION

ON A DETERMINATION OF THE INITIAL FUNCTIONS FROM THE OBSERVED VALUES OF THE BOUNDARY FUNCTIONS FOR THE SECOND-ORDER HYPERBOLIC EQUATION Advanced Mathematcal Models & Applcatons Vol.3, No.3, 2018, pp.215-222 ON A DETERMINATION OF THE INITIAL FUNCTIONS FROM THE OBSERVED VALUES OF THE BOUNDARY FUNCTIONS FOR THE SECOND-ORDER HYPERBOLIC EUATION

More information

Support Vector Machines CS434

Support Vector Machines CS434 Support Vector Machnes CS434 Lnear Separators Many lnear separators exst that perfectly classfy all tranng examples Whch of the lnear separators s the best? + + + + + + + + + Intuton of Margn Consder ponts

More information

First day August 1, Problems and Solutions

First day August 1, Problems and Solutions FOURTH INTERNATIONAL COMPETITION FOR UNIVERSITY STUDENTS IN MATHEMATICS July 30 August 4, 997, Plovdv, BULGARIA Frst day August, 997 Problems and Solutons Problem. Let {ε n } n= be a sequence of postve

More information