A Tour of the Lanczos Algorithm and its Convergence Guarantees through the Decades
|
|
- Caroline Small
- 6 years ago
- Views:
Transcription
1 A Tour of the Lanczos Algorithm and its Convergence Guarantees through the Decades Qiaochu Yuan Department of Mathematics UC Berkeley Joint work with Prof. Ming Gu, Bo Li April 17, 2018 Qiaochu Yuan Tour of Lanczos April 17, / 43
2 Timeline 1950) Lanczos iteration algorithm 1965) Lanczos algorithm for SVD 1974, 1977) algorithm 2009, 2010, 2015) Randomized , 1971, 1980) Convergence 1975, 1980) bounds for Convergence Lanczos bounds for 2015) Convergence bounds for Randomized Qiaochu Yuan Tour of Lanczos April 17, / 43
3 Timeline 1950) Lanczos iteration algorithm 1965) Lanczos algorithm for SVD 1974, 1977) algorithm 2009, 2010, 2015) Randomized Our work: 2015) 1966, 1971, 1980) Convergence 1975, 1980) 1) Convergence bounds Convergence for bounds for bounds for Convergence RBL, arbitrary b Randomized Lanczos bounds 2) for Superlinear convergence for typical data matrices Qiaochu Yuan Tour of Lanczos April 17, / 43
4 Timeline 1950) Lanczos iteration algorithm 1965) Lanczos algorithm for SVD 1974, 1977) algorithm 2009, 2010, 2015) Randomized , 1971, 1980) Convergence 1975, 1980) bounds for Convergence Lanczos bounds for 2015) Convergence bounds for Randomized Qiaochu Yuan Tour of Lanczos April 17, / 43
5 Lanczos Iteration Algorithm Developed by Lanczos in 1950 [Lan50]. Widely used iterative algorithm for computing the extremal eigenvalues and corresponding eigenvectors of a large, sparse, symmetric matrix A. Goal Given a symmetric matrix A R n n, with eigenvalues λ 1 > λ 2 > > λ n and associated eigenvectors u 1,, u n, want to find approximations for λ i, i = 1,, k, the k largest eigenvalues of A u i, i = 1,, k, the associated eigenvectors where k n. Qiaochu Yuan Tour of Lanczos April 17, / 43
6 Lanczos - Details General idea: 1 Select an initial vector v. 2 Construct Krylov subspace K A, v, k) = span{v, Av, A 2 v,, A k 1 v}. 3 Restrict and project A to the Krylov subspace, T = proj K A K 4 Use eigen values and vectors of T as approximations to those of A. In matrices: K k = [ v Av A k 1 v ] R n k Q k = [ ] q 1 q 2 q k qr Kk ) T k = Q T k AQ k R k k Qiaochu Yuan Tour of Lanczos April 17, / 43
7 Lanczos - Details A [ ] [ ] q 1 q j = q1 q j q j+1 At each step j = 1,, k of Lanczos iteration: Use the three-term recurrence: Calculate the αs, βs as: AQ j = Q j T j + β j q j+1 e T j+1 Aq j = β j 1 q j 1 + α j q j + β j q j+1 α 1 β 1 β βj 1 β j 1 α j β j α j = q T j Aq j 1) r j = A α j I) q j β j 1 q j 2) β j = r j 2, q j+1 = r j /β j 3) Qiaochu Yuan Tour of Lanczos April 17, / 43
8 Timeline 1950) Lanczos iteration algorithm 1965) Lanczos algorithm for SVD 1974, 1977) algorithm 2009, 2010, 2015) Randomized , 1971, 1980) Convergence 1975, 1980) bounds for Convergence Lanczos bounds for 2015) Convergence bounds for Randomized Qiaochu Yuan Tour of Lanczos April 17, / 43
9 Convergence of Lanczos How well does λ k) i, the eigenvalues of T k, approximate λ i, the eigenvalues of A, for i = 1,, k? First answered by Kaniel in 1966 [Kan66] and Paige in 1971 [Pai71]. Theorem Kaniel-Paige Inequality) If v is chosen to be not orthogonal to the eigenspace associated with λ 1, then 0 λ 1 λ k) 1 λ 1 λ n ) tan2 θ u 1, v) Tk 1 2 γ 4) 1) where T i x) is the Chebyshev polynomial of degree i, θ, ) is the angle between two vectors, and γ 1 = λ 1 λ 2 λ 2 λ n Qiaochu Yuan Tour of Lanczos April 17, / 43
10 Convergence of Lanczos Later generalized by Saad in 1980 [Saa80]. Theorem Saad Inequality) For i = 1,, k, if v is chosen such that u T i v 0, then ) 0 λ i λ k) L k) 2 i tan θ u i, v) i λ i λ n ) 5) T k i γ i ) where T i x) is the Chebyshev polynomial of degree i, θ, ) is the angle between two vectors, and γ i = λ i λ i+1 λ i+1 λ n L k) i = i 1 j=1 λ k) j λ n if i 1 λ k) j λ i 1 if i = 1 Qiaochu Yuan Tour of Lanczos April 17, / 43
11 Aside - Chebyshev Polynomials Recall T j x) = 1 x + ) j x x ) ) j x 2 1 6) When j is large, T j x) 1 2 and when g is small, T j 1 + g) 1 2 x + ) j x g + ) j 2g determines the convergence of Lanczos with ) λi λ i+1 g = Θ λ i+1 λ n Chebyshev, Lanczos Monomial, power method Qiaochu Yuan Tour of Lanczos April 17, / 43
12 Timeline 1950) Lanczos iteration algorithm 1965) Lanczos algorithm for SVD 1974, 1977) algorithm 2009, 2010, 2015) Randomized , 1971, 1980) Convergence 1975, 1980) bounds for Convergence Lanczos bounds for 2015) Convergence bounds for Randomized Qiaochu Yuan Tour of Lanczos April 17, / 43
13 Algorithm Introduced by Golub and Underwood in 1977 [GU77] and Cullum and Donath in 1974 [CD74]. The block generalization of the Lanczos method uses, instead of a single initial vector v, a block of b vectors V = [ v 1 v b ], and builds the Krylov subspace in q iterations as K q A, V, q) = span { V, AV,, A q 1 V }. k b, bq n. Compared to classical Lanczos, block Lanczos is more memory and cache efficient, using BLAS3 operations. has the ability to converge to eigenvalues with cluster size > 1. has faster convergence with respect to number of iterations. Qiaochu Yuan Tour of Lanczos April 17, / 43
14 Algorithm - Details A [ ] [ ] Q 1 Q j = Q1 Q j Q j+1 At each step j = 1,, k of Lanczos iteration: AQ = QT j + Q j+1 [ 0 0 Bj ] Use the three-term recurrence: A 1 B1 T. B B T j 1 B j 1 A j B j Calculate the As, Bs as: AQ j = Q j 1 B T j 1 + Q j A j + Q j+1 B j A j = Q T j AQ j Q j+1 B j qr ) AQ j Q j A j Q j Bj 1 T 7) 8) Qiaochu Yuan Tour of Lanczos April 17, / 43
15 Timeline 1950) Lanczos iteration algorithm 1965) Lanczos algorithm for SVD 1974, 1977) algorithm 2009, 2010, 2015) Randomized , 1971, 1980) Convergence 1975, 1980) bounds for Convergence Lanczos bounds for 2015) Convergence bounds for Randomized Qiaochu Yuan Tour of Lanczos April 17, / 43
16 Convergence of First analyzed by Underwood in 1975 [Und75] with the introduction of the algorithm. Theorem Underwood Inequality) Let λ i, u i, i = 1,, n be the eigenvalues and eigenvectors of A respectively. Let U = [ ] u 1 u b. If U T V is of full rank b, then for i = 1,, b 0 λ i λ q) i λ 1 λ n ) tan2 Θ U, V) Tq 1 2 ρ 9) i) where T i x) is the Chebyshev polynomial of degree i, cos Θ U, V) = σ min U T V ), and ρ i = λ i λ b+1 λ b+1 λ n Qiaochu Yuan Tour of Lanczos April 17, / 43
17 Convergence of Theorem Saad Inequality [Saa80]) Let λ j, u j, j = 1,, n be the eigenvalues and eigenvectors of A respectively. Let U i = [ u i u i+b 1 ]. For i = 1,, b, if U T i V is of full rank b, then ) 0 λ i λ q) L q) 2 i tan Θ U i, V) i λ i λ n ) 10) T q i ˆγ i ) where T i x) is the Chebyshev polynomial of degree i, cos Θ U, V) = σ min U T V ), and ˆγ i = λ i λ i+b λ i+b λ n L q) i = i 1 j=1 λ q) j λ n if i 1 λ q) j λ i 1 if i = 1 Qiaochu Yuan Tour of Lanczos April 17, / 43
18 Aside - Block Size Recall, when j is large and g is small T j 1 + g) g + j 2g) 11) 2 ) λi λ i+1 classical: g = Θ λ i+1 λ n ) λi λ i+b block: g b = Θ λ i+b λ n g g b Suppose eigenvalue distributed as λ j > 1 + ɛ)λ j+1 for all j: Chebyshev, Lanczos g b ɛ) b 1 + ɛ) b λ i b λ i ɛ b g Monomial, power method Qiaochu Yuan Tour of Lanczos April 17, / 43
19 Timeline 1950) Lanczos iteration algorithm 1965) Lanczos algorithm for SVD 1974, 1977) algorithm 2009, 2010, 2015) Randomized , 1971, 1980) Convergence 1975, 1980) bounds for Convergence Lanczos bounds for 2015) Convergence bounds for Randomized Qiaochu Yuan Tour of Lanczos April 17, / 43
20 Randomized In recent years, there has been increased interest in algorithms to compute low-rank approximations of matrices, with high computational efficiency requirement, running on large matrices low approximation accuracy requirement, 2-3 digits of accuracy Applications mostly in big-data computations compression of data matrices matrix processing techniques, e.g. PCA optimization of the nuclear norm objective function Qiaochu Yuan Tour of Lanczos April 17, / 43
21 Randomized Grew out of work done in randomized algorithms, in particular Randomized Subspace Iteration, by Rokhlin, Szlam, and Tygert in 2009 [RST09] Halko, Martinsson, and Tropp in 2011 [HMST11] Gu [Gu15], and Musco [MM15] in 2015 Idea: Instead of taking any initial set of vectors V, an unfortunate choice of which could result in poor convergence, choose V = AΩ, a random projection of the columns of A, to better capture the range space. Qiaochu Yuan Tour of Lanczos April 17, / 43
22 RSI & RBL Pseudocode Algorithm 1 Randomized Subspace Iteration RSI) pseudocode Input: A R m n, target rank k, block size b, number of iter. q Output: B k R m n, a rank-k approximation 1: Draw Ω R n b. ω ij N 0, 1). 2: Form K = AA T ) q 1 AΩ. 3: Orthogonalize qrk) Q. 4: B k = QQ T A) k = QQ T A) k. M) k indicates the k-truncated SVD of matrix M. Algorithm 2 Randomized Block Lanczos RBL) pseudocode Input: A R m n, target rank k, block size b, number of iter. q Output: B k R m n, a rank-k approximation 1: Draw Ω R n b. ω ij N 0, 1). 2: Form K =[AΩ, AA T )AΩ,, AA T ) q 1 AΩ]. 3: Orthogonalize qrk) Q. 4: B k = QQ T A) k = QQ T A) k. requires bq k. numerically unstable. Qiaochu Yuan Tour of Lanczos April 17, / 43
23 Timeline 1950) Lanczos iteration algorithm 1965) Lanczos algorithm for SVD 1974, 1977) algorithm 2009, 2010, 2015) Randomized , 1971, 1980) Convergence 1975, 1980) bounds for Convergence Lanczos bounds for 2015) Convergence bounds for Randomized Qiaochu Yuan Tour of Lanczos April 17, / 43
24 Lanczos for SVD A modification of the Lanczos algorithm can be used to find the extremal singular value pairs of a non-symmetric matrix A R m n. Utilizing the connections between the singular value decomposition of A and the eigen decompositions of AA T and A T A, 1 Select a block of initial vectors V = [ v 1 v b ]. 2 Construct Krylov subspaces K V A T A, V, q ) and K U AA T, AV, q ). 3 Restrict and project A to form B = proj KU A KV 4 Use singular values and vectors of B as approximations to those of A. Qiaochu Yuan Tour of Lanczos April 17, / 43
25 Golub-Kahan Bidiagonalization [GK65] A [ ] [ ] V 1 V j = U1 U j A 1 B T A T [ ] [ ] U 1 U j = V1 V j V j+1 A 1... B T j 1 A j B B j 1 At each step j = 1,, k of the bidiagonalization, calculate As and Bs as: ) U j A j qr AV j U j 1 Bj 1 T 12) ) V j+1 B j qr A T U j V j A j 13) A j B j Qiaochu Yuan Tour of Lanczos April 17, / 43
26 Timeline 1950) Lanczos iteration algorithm 1965) Lanczos algorithm for SVD 1974, 1977) algorithm 2009, 2010, 2015) Randomized , 1971, 1980) Convergence 1975, 1980) bounds for Convergence Lanczos bounds for 2015) Convergence bounds for Randomized Qiaochu Yuan Tour of Lanczos April 17, / 43
27 Convergence of RBL Analyzed by Musco and Musco in 2015 [MM15]. Theorem ) ) For block size b k, in q = Θ log n ɛ iterations of RSI and q = Θ log n ɛ iterations of RBL, with constant probability 99/100, the following inequalities bounds are satisfied: σi 2 σi 2 B k ) ɛσk+1 2,, for i = 1,, k 14) ) In the event that σ b+1 cσ k with c < 1, taking q = Θ ) and q = Θ logn/ɛ) min1,σk /σ b+1 1) logn/ɛ) min1,σ k /σ b+1 1) suffices for RSI and RBL respectively. Qiaochu Yuan Tour of Lanczos April 17, / 43
28 Timeline 1950) Lanczos iteration algorithm 1965) Lanczos algorithm for SVD 1974, 1977) algorithm 2009, 2010, 2015) Randomized ) 1966, 1971, 1980) Our work: Convergence Convergence 1975, 1980) 1) Convergence bounds bounds for for bounds for Convergence RBL, arbitrary brandomized Lanczos bounds 2) for Superlinear convergence for typical data matrices Qiaochu Yuan Tour of Lanczos April 17, / 43
29 Aside: Convergence of RSI Many of our analysis techniques are similar to those used by Gu to analyze RSI [Gu15]. The bounds for RBL resulting from the current work will be similar in form to the bounds for RSI. Theorem RSI convergence) Let B k be the approximation returned by the RSI algorithm on A = UΣV T, with target rank k, block size b k, and q iterations. If ˆΩ 1 has full row rank in V T Ω = [ ] ˆΩ T 1 ˆΩ T T 2, then σ j σ j B k ) σ j 1 + ˆΩ ˆΩ σb+1 σ j ) 4q+2 15) Qiaochu Yuan Tour of Lanczos April 17, / 43
30 Current Work - Core Ideas The core pieces of our analysis are: 1 the growth behavior of Chebyshev polynomials, 2 the choice of a clever orthonormal basis for Krylov subspace, 3 the creation of a spectrum gap, by separating the spectrum of A into those singular values that are close to σ k, and those that are sufficiently smaller in magnitude. Qiaochu Yuan Tour of Lanczos April 17, / 43
31 Setup & Notation For block size b, random Gaussian matrix Ω R n b, we are interested in the column span of [ K q = AΩ AA T ) AΩ AA T ) ] q 1 AΩ Let the SVD of A R m n be UΣV T. For any 0 p q, define ˆK p : = UT 2p+1 Σ) [ ˆΩ Σ 2 ˆΩ Σ 2q p 1) ˆΩ ] = UT 2p+1 Σ)V q p Note V q p R n bq p). We require bq p) k, the target rank. Lemma With ˆK p and K q as previously defined, { } span {K q } span ˆK p 16) Qiaochu Yuan Tour of Lanczos April 17, / 43
32 Convergence of RBL Theorem Main Result) Let B k be the approximation returned by the RBL algorithm. With notation as previously defined, if the random starting Ω is initialized such that the block Vandermonde formed by the sub-blocks of V q p is invertible, then for all 1 j k, and all choices a of s, r, σ j σ j B k ) 1 + C 2 T 2 2p+1 σ j+s σj σ j+s+r+1 σ j+s+r+1 ) 17) where p = q k+r b, and C is a constant independent of q. a s is chosen to be non-zero to handle multiple singular values, and can be set to zero otherwise. Qiaochu Yuan Tour of Lanczos April 17, / 43
33 Summary of Convergence Bounds Rewriting bounds in comparable forms: citation bound req. on b Saad [Saa80] Musco [MM15] spec. indep. Musco [MM15] spec. dep. Current Work λ q) j σ q) j σ q) j 1+L q) j σ q) j λ j 2 tan 2 ΘU,V)T 2 q j σ j 1+C 2 1 log2 n) q 2 σ2 k+1 σ 2 j σ j 1+C 2 ne q min1,σ k /σ b+1 1) σ2 k+1 σ 2 j 1+2 λ j λ j+b λ j+b ) b k σ j ) 1+C3 2T σ j σ j+r+1 2q+1 2k+r)/b σ j+r+1 b k b k b 1 bq k + r Qiaochu Yuan Tour of Lanczos April 17, / 43
34 Typical Case - Superlinear Convergence Recall: {a q } convergence superlinearly to a if a q+1 a lim = 0 18) q a q a In practice, Lanczos algorithms classical, block, randomized) often exhibit superlinear convergence behavior. It has been shown that classical Lanczos iteration is theoretically superlinearly convergent under certain assumptions about the singular spectrum [saa94, Li10]. We show this for block Lanczos algorithms, i.e., that under certain assumptions about the singular spectrum, block Lanczos produces rank k approximations B k such that σ j B k ) σ j superlinearly. Qiaochu Yuan Tour of Lanczos April 17, / 43
35 Typical Case - Superlinear Convergence A typical data matrix might have singular value spectrum decaying to 0, i.e., σ j 0. In this case our bound suggests that convergence is governed by a q := Cr)T 1 p 1 + g) ) 2 Cr) 1 2 with σj = 2 σ j+r+1 g = 2 σ j σ j+r+1 p = 2 q k + r b ) + 1 = 2q g + 2g) p ) 2 0 ) 1 σ j+r k + r b We argue that a q+1 /a q 0 as follows: for all ɛ > 0, choose 1 r so that 1 + g ɛ 1 2. Then, a q+1 ɛ a q 1 Recall 1) our main result holds for all r; 2) k + r = q p)b, and so choosing r amounts to choosing q. Qiaochu Yuan Tour of Lanczos April 17, / 43 )
36 Effect of r There is some optimal value of r, typically non-zero, which achieves the best convergence factor. The balance is between larger smaller) values of r, which implies lower higher) Chebyshev degree but bigger smaller) gap. ) Figure: Value of reciprocal convergence factor T 1 2q+1 2k+r)/b) σ j σ j+r+1 σ j+r+1 as r varies, for Daily Activities and Sports Dataset, k = j = 100, b = 10, q = r Qiaochu Yuan Tour of Lanczos April 17, / 43
37 Numerical Example Experimentally, choices of smaller block sizes 1 b < k appear favorable with superlinear convergence for all block sizes. Figure: Daily Activities and Sports Dataset - A R Daily Activities Dataset, target k = 200, j = 200 j - j B k ))/ j rel. err. = RSI b=200+2 RSI b=200+5 RSI b= RBL b=200 RBL b=100 RBL b=50 RBL b=20 RBL b=10 RBL b= Number of MATVECs = b*1+2*q) Qiaochu Yuan Tour of Lanczos April 17, / 43
38 Numerical Example Figure: Eigenfaces Dataset - A R Eigenfaces Dataset, target k = 100, j = 100 j - j B k ))/ j rel. err. = RSI b=100 RSI b=100+2 RSI b=100+5 RBL b=100 RBL b=50 RBL b=20 RBL b=10 RBL b=2 RBL b= Number of MATVECs = b*1+2*q) Qiaochu Yuan Tour of Lanczos April 17, / 43
39 Conclusions Both the theoretical analysis and numerical evidence suggest that, holding the number of matrix vector operations constant, RBL with smaller block size b is better. For matrices with decaying spectrum, RBL achieves superlinear convergence. However the preference for smaller b must be balanced with the advantages of a larger b for computational efficiency and numerical stability reasons in a practical implementation, and should be further investigated. Qiaochu Yuan Tour of Lanczos April 17, / 43
40 References I Jane Cullum and William E Donath, A block lanczos algorithm for computing the q algebraically largest eigenvalues and a corresponding eigenspace of large, sparse, real symmetric matrices, Decision and Control including the 13th Symposium on Adaptive Processes, 1974 IEEE Conference on, vol. 13, IEEE, 1974, pp Gene Golub and William Kahan, Calculating the singular values and pseudo-inverse of a matrix, Journal of the Society for Industrial and Applied Mathematics, Series B: Numerical Analysis ), no. 2, Gene Howard Golub and Richard Underwood, The block lanczos method for computing eigenvalues, Mathematical software, Elsevier, 1977, pp Qiaochu Yuan Tour of Lanczos April 17, / 43
41 References II Ming Gu, Subspace iteration randomization and singular value problems, SIAM Journal on Scientific Computing ), no. 3, A1139 A1173. Nathan Halko, Per-Gunnar Martinsson, Yoel Shkolnisky, and Mark Tygert, An algorithm for the principal component analysis of large data sets, SIAM Journal on Scientific computing ), no. 5, Shmuel Kaniel, Estimates for some computational techniques in linear algebra, Mathematics of Computation ), no. 95, Cornelius Lanczos, An iteration method for the solution of the eigenvalue problem of linear differential and integral operators, United States Governm. Press Office Los Angeles, CA, Qiaochu Yuan Tour of Lanczos April 17, / 43
42 References III Ren-Cang Li, Sharpness in rates of convergence for the symmetric lanczos method, Mathematics of Computation ), no. 269, Cameron Musco and Christopher Musco, Randomized block krylov methods for stronger and faster approximate singular value decomposition, Advances in Neural Information Processing Systems, 2015, pp Christopher Conway Paige, The computation of eigenvalues and eigenvectors of very large sparse matrices., Ph.D. thesis, University of London, Vladimir Rokhlin, Arthur Szlam, and Mark Tygert, A randomized algorithm for principal component analysis, SIAM Journal on Matrix Analysis and Applications ), no. 3, Qiaochu Yuan Tour of Lanczos April 17, / 43
43 References IV Yousef Saad, On the rates of convergence of the lanczos and the block-lanczos methods, SIAM Journal on Numerical Analysis ), no. 5, Theoretical error bounds and general analysis of a few lanczos-type algorithms, Richard Underwood, An iterative block lanczos method for the solution of large sparse symmetric eigenproblems, Tech. report, Qiaochu Yuan Tour of Lanczos April 17, / 43
Normalized power iterations for the computation of SVD
Normalized power iterations for the computation of SVD Per-Gunnar Martinsson Department of Applied Mathematics University of Colorado Boulder, Co. Per-gunnar.Martinsson@Colorado.edu Arthur Szlam Courant
More informationrandomized block krylov methods for stronger and faster approximate svd
randomized block krylov methods for stronger and faster approximate svd Cameron Musco and Christopher Musco December 2, 25 Massachusetts Institute of Technology, EECS singular value decomposition n d left
More informationLarge-scale eigenvalue problems
ELE 538B: Mathematics of High-Dimensional Data Large-scale eigenvalue problems Yuxin Chen Princeton University, Fall 208 Outline Power method Lanczos algorithm Eigenvalue problems 4-2 Eigendecomposition
More informationThe Lanczos and conjugate gradient algorithms
The Lanczos and conjugate gradient algorithms Gérard MEURANT October, 2008 1 The Lanczos algorithm 2 The Lanczos algorithm in finite precision 3 The nonsymmetric Lanczos algorithm 4 The Golub Kahan bidiagonalization
More informationApplications of Randomized Methods for Decomposing and Simulating from Large Covariance Matrices
Applications of Randomized Methods for Decomposing and Simulating from Large Covariance Matrices Vahid Dehdari and Clayton V. Deutsch Geostatistical modeling involves many variables and many locations.
More informationLARGE SPARSE EIGENVALUE PROBLEMS. General Tools for Solving Large Eigen-Problems
LARGE SPARSE EIGENVALUE PROBLEMS Projection methods The subspace iteration Krylov subspace methods: Arnoldi and Lanczos Golub-Kahan-Lanczos bidiagonalization General Tools for Solving Large Eigen-Problems
More informationLARGE SPARSE EIGENVALUE PROBLEMS
LARGE SPARSE EIGENVALUE PROBLEMS Projection methods The subspace iteration Krylov subspace methods: Arnoldi and Lanczos Golub-Kahan-Lanczos bidiagonalization 14-1 General Tools for Solving Large Eigen-Problems
More informationBlock Bidiagonal Decomposition and Least Squares Problems
Block Bidiagonal Decomposition and Least Squares Problems Åke Björck Department of Mathematics Linköping University Perspectives in Numerical Analysis, Helsinki, May 27 29, 2008 Outline Bidiagonal Decomposition
More informationSingular Value Decomposition
Singular Value Decomposition CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Doug James (and Justin Solomon) CS 205A: Mathematical Methods Singular Value Decomposition 1 / 35 Understanding
More informationSingular Value Decomposition
Singular Value Decomposition Motivatation The diagonalization theorem play a part in many interesting applications. Unfortunately not all matrices can be factored as A = PDP However a factorization A =
More informationDimensionality Reduction: PCA. Nicholas Ruozzi University of Texas at Dallas
Dimensionality Reduction: PCA Nicholas Ruozzi University of Texas at Dallas Eigenvalues λ is an eigenvalue of a matrix A R n n if the linear system Ax = λx has at least one non-zero solution If Ax = λx
More informationSTA141C: Big Data & High Performance Statistical Computing
STA141C: Big Data & High Performance Statistical Computing Lecture 5: Numerical Linear Algebra Cho-Jui Hsieh UC Davis April 20, 2017 Linear Algebra Background Vectors A vector has a direction and a magnitude
More informationPrincipal Component Analysis
Machine Learning Michaelmas 2017 James Worrell Principal Component Analysis 1 Introduction 1.1 Goals of PCA Principal components analysis (PCA) is a dimensionality reduction technique that can be used
More informationComputation of eigenvalues and singular values Recall that your solutions to these questions will not be collected or evaluated.
Math 504, Homework 5 Computation of eigenvalues and singular values Recall that your solutions to these questions will not be collected or evaluated 1 Find the eigenvalues and the associated eigenspaces
More informationMain matrix factorizations
Main matrix factorizations A P L U P permutation matrix, L lower triangular, U upper triangular Key use: Solve square linear system Ax b. A Q R Q unitary, R upper triangular Key use: Solve square or overdetrmined
More informationTotal least squares. Gérard MEURANT. October, 2008
Total least squares Gérard MEURANT October, 2008 1 Introduction to total least squares 2 Approximation of the TLS secular equation 3 Numerical experiments Introduction to total least squares In least squares
More informationSTA141C: Big Data & High Performance Statistical Computing
STA141C: Big Data & High Performance Statistical Computing Numerical Linear Algebra Background Cho-Jui Hsieh UC Davis May 15, 2018 Linear Algebra Background Vectors A vector has a direction and a magnitude
More informationParallel Numerical Algorithms
Parallel Numerical Algorithms Chapter 6 Matrix Models Section 6.2 Low Rank Approximation Edgar Solomonik Department of Computer Science University of Illinois at Urbana-Champaign CS 554 / CSE 512 Edgar
More informationAPPLIED NUMERICAL LINEAR ALGEBRA
APPLIED NUMERICAL LINEAR ALGEBRA James W. Demmel University of California Berkeley, California Society for Industrial and Applied Mathematics Philadelphia Contents Preface 1 Introduction 1 1.1 Basic Notation
More informationLow-Rank Approximations, Random Sampling and Subspace Iteration
Modified from Talk in 2012 For RNLA study group, March 2015 Ming Gu, UC Berkeley mgu@math.berkeley.edu Low-Rank Approximations, Random Sampling and Subspace Iteration Content! Approaches for low-rank matrix
More informationUNIT 6: The singular value decomposition.
UNIT 6: The singular value decomposition. María Barbero Liñán Universidad Carlos III de Madrid Bachelor in Statistics and Business Mathematical methods II 2011-2012 A square matrix is symmetric if A T
More informationA fast randomized algorithm for approximating an SVD of a matrix
A fast randomized algorithm for approximating an SVD of a matrix Joint work with Franco Woolfe, Edo Liberty, and Vladimir Rokhlin Mark Tygert Program in Applied Mathematics Yale University Place July 17,
More informationSolution of eigenvalue problems. Subspace iteration, The symmetric Lanczos algorithm. Harmonic Ritz values, Jacobi-Davidson s method
Solution of eigenvalue problems Introduction motivation Projection methods for eigenvalue problems Subspace iteration, The symmetric Lanczos algorithm Nonsymmetric Lanczos procedure; Implicit restarts
More informationRandomized methods for computing the Singular Value Decomposition (SVD) of very large matrices
Randomized methods for computing the Singular Value Decomposition (SVD) of very large matrices Gunnar Martinsson The University of Colorado at Boulder Students: Adrianna Gillman Nathan Halko Sijia Hao
More information4.8 Arnoldi Iteration, Krylov Subspaces and GMRES
48 Arnoldi Iteration, Krylov Subspaces and GMRES We start with the problem of using a similarity transformation to convert an n n matrix A to upper Hessenberg form H, ie, A = QHQ, (30) with an appropriate
More informationSUBSPACE ITERATION RANDOMIZATION AND SINGULAR VALUE PROBLEMS
SUBSPACE ITERATION RANDOMIZATION AND SINGULAR VALUE PROBLEMS M. GU Abstract. A classical problem in matrix computations is the efficient and reliable approximation of a given matrix by a matrix of lower
More informationPreliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012
Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.
More informationSVD, PCA & Preprocessing
Chapter 1 SVD, PCA & Preprocessing Part 2: Pre-processing and selecting the rank Pre-processing Skillicorn chapter 3.1 2 Why pre-process? Consider matrix of weather data Monthly temperatures in degrees
More informationComputational Methods. Eigenvalues and Singular Values
Computational Methods Eigenvalues and Singular Values Manfred Huber 2010 1 Eigenvalues and Singular Values Eigenvalues and singular values describe important aspects of transformations and of data relations
More informationSingular Value Decomposition
Chapter 6 Singular Value Decomposition In Chapter 5, we derived a number of algorithms for computing the eigenvalues and eigenvectors of matrices A R n n. Having developed this machinery, we complete our
More informationA randomized algorithm for approximating the SVD of a matrix
A randomized algorithm for approximating the SVD of a matrix Joint work with Per-Gunnar Martinsson (U. of Colorado) and Vladimir Rokhlin (Yale) Mark Tygert Program in Applied Mathematics Yale University
More informationA fast randomized algorithm for overdetermined linear least-squares regression
A fast randomized algorithm for overdetermined linear least-squares regression Vladimir Rokhlin and Mark Tygert Technical Report YALEU/DCS/TR-1403 April 28, 2008 Abstract We introduce a randomized algorithm
More informationFall TMA4145 Linear Methods. Exercise set Given the matrix 1 2
Norwegian University of Science and Technology Department of Mathematical Sciences TMA445 Linear Methods Fall 07 Exercise set Please justify your answers! The most important part is how you arrive at an
More informationLecture 9: Krylov Subspace Methods. 2 Derivation of the Conjugate Gradient Algorithm
CS 622 Data-Sparse Matrix Computations September 19, 217 Lecture 9: Krylov Subspace Methods Lecturer: Anil Damle Scribes: David Eriksson, Marc Aurele Gilles, Ariah Klages-Mundt, Sophia Novitzky 1 Introduction
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 18 Outline
More informationNORMS ON SPACE OF MATRICES
NORMS ON SPACE OF MATRICES. Operator Norms on Space of linear maps Let A be an n n real matrix and x 0 be a vector in R n. We would like to use the Picard iteration method to solve for the following system
More informationMatrix Approximations
Matrix pproximations Sanjiv Kumar, Google Research, NY EECS-6898, Columbia University - Fall, 00 Sanjiv Kumar 9/4/00 EECS6898 Large Scale Machine Learning Latent Semantic Indexing (LSI) Given n documents
More informationMathematical foundations - linear algebra
Mathematical foundations - linear algebra Andrea Passerini passerini@disi.unitn.it Machine Learning Vector space Definition (over reals) A set X is called a vector space over IR if addition and scalar
More informationRandomization: Making Very Large-Scale Linear Algebraic Computations Possible
Randomization: Making Very Large-Scale Linear Algebraic Computations Possible Gunnar Martinsson The University of Colorado at Boulder The material presented draws on work by Nathan Halko, Edo Liberty,
More informationNotes on singular value decomposition for Math 54. Recall that if A is a symmetric n n matrix, then A has real eigenvalues A = P DP 1 A = P DP T.
Notes on singular value decomposition for Math 54 Recall that if A is a symmetric n n matrix, then A has real eigenvalues λ 1,, λ n (possibly repeated), and R n has an orthonormal basis v 1,, v n, where
More informationApplied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in
More informationSolution of eigenvalue problems. Subspace iteration, The symmetric Lanczos algorithm. Harmonic Ritz values, Jacobi-Davidson s method
Solution of eigenvalue problems Introduction motivation Projection methods for eigenvalue problems Subspace iteration, The symmetric Lanczos algorithm Nonsymmetric Lanczos procedure; Implicit restarts
More informationarxiv: v1 [cs.lg] 26 Jul 2017
Updating Singular Value Decomposition for Rank One Matrix Perturbation Ratnik Gandhi, Amoli Rajgor School of Engineering & Applied Science, Ahmedabad University, Ahmedabad-380009, India arxiv:70708369v
More informationRandomized Methods for Computing Low-Rank Approximations of Matrices
University of Colorado, Boulder CU Scholar Applied Mathematics Graduate Theses & Dissertations Applied Mathematics Spring 1-1-2012 Randomized Methods for Computing Low-Rank Approximations of Matrices Nathan
More informationThe University of Texas at Austin Department of Electrical and Computer Engineering. EE381V: Large Scale Learning Spring 2013.
The University of Texas at Austin Department of Electrical and Computer Engineering EE381V: Large Scale Learning Spring 2013 Assignment Two Caramanis/Sanghavi Due: Tuesday, Feb. 19, 2013. Computational
More informationData Mining Lecture 4: Covariance, EVD, PCA & SVD
Data Mining Lecture 4: Covariance, EVD, PCA & SVD Jo Houghton ECS Southampton February 25, 2019 1 / 28 Variance and Covariance - Expectation A random variable takes on different values due to chance The
More informationProbabilistic upper bounds for the matrix two-norm
Noname manuscript No. (will be inserted by the editor) Probabilistic upper bounds for the matrix two-norm Michiel E. Hochstenbach Received: date / Accepted: date Abstract We develop probabilistic upper
More informationAM 205: lecture 8. Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition
AM 205: lecture 8 Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition QR Factorization A matrix A R m n, m n, can be factorized
More informationApplied Mathematics 205. Unit II: Numerical Linear Algebra. Lecturer: Dr. David Knezevic
Applied Mathematics 205 Unit II: Numerical Linear Algebra Lecturer: Dr. David Knezevic Unit II: Numerical Linear Algebra Chapter II.3: QR Factorization, SVD 2 / 66 QR Factorization 3 / 66 QR Factorization
More informationLinear Algebra & Geometry why is linear algebra useful in computer vision?
Linear Algebra & Geometry why is linear algebra useful in computer vision? References: -Any book on linear algebra! -[HZ] chapters 2, 4 Some of the slides in this lecture are courtesy to Prof. Octavia
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 19: More on Arnoldi Iteration; Lanczos Iteration Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 17 Outline 1
More informationA Randomized Algorithm for the Approximation of Matrices
A Randomized Algorithm for the Approximation of Matrices Per-Gunnar Martinsson, Vladimir Rokhlin, and Mark Tygert Technical Report YALEU/DCS/TR-36 June 29, 2006 Abstract Given an m n matrix A and a positive
More informationIterative methods for symmetric eigenvalue problems
s Iterative s for symmetric eigenvalue problems, PhD McMaster University School of Computational Engineering and Science February 11, 2008 s 1 The power and its variants Inverse power Rayleigh quotient
More informationLet A an n n real nonsymmetric matrix. The eigenvalue problem: λ 1 = 1 with eigenvector u 1 = ( ) λ 2 = 2 with eigenvector u 2 = ( 1
Eigenvalue Problems. Introduction Let A an n n real nonsymmetric matrix. The eigenvalue problem: EIGENVALE PROBLEMS AND THE SVD. [5.1 TO 5.3 & 7.4] Au = λu Example: ( ) 2 0 A = 2 1 λ 1 = 1 with eigenvector
More informationNumerical Methods I Singular Value Decomposition
Numerical Methods I Singular Value Decomposition Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 October 9th, 2014 A. Donev (Courant Institute)
More informationGolub-Kahan iterative bidiagonalization and determining the noise level in the data
Golub-Kahan iterative bidiagonalization and determining the noise level in the data Iveta Hnětynková,, Martin Plešinger,, Zdeněk Strakoš, * Charles University, Prague ** Academy of Sciences of the Czech
More informationNumerical Methods in Matrix Computations
Ake Bjorck Numerical Methods in Matrix Computations Springer Contents 1 Direct Methods for Linear Systems 1 1.1 Elements of Matrix Theory 1 1.1.1 Matrix Algebra 2 1.1.2 Vector Spaces 6 1.1.3 Submatrices
More informationParallel Singular Value Decomposition. Jiaxing Tan
Parallel Singular Value Decomposition Jiaxing Tan Outline What is SVD? How to calculate SVD? How to parallelize SVD? Future Work What is SVD? Matrix Decomposition Eigen Decomposition A (non-zero) vector
More informationLast Time. Social Network Graphs Betweenness. Graph Laplacian. Girvan-Newman Algorithm. Spectral Bisection
Eigenvalue Problems Last Time Social Network Graphs Betweenness Girvan-Newman Algorithm Graph Laplacian Spectral Bisection λ 2, w 2 Today Small deviation into eigenvalue problems Formulation Standard eigenvalue
More informationLinear Algebra Primer
Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................
More informationMATH 22A: LINEAR ALGEBRA Chapter 4
MATH 22A: LINEAR ALGEBRA Chapter 4 Jesús De Loera, UC Davis November 30, 2012 Orthogonality and Least Squares Approximation QUESTION: Suppose Ax = b has no solution!! Then what to do? Can we find an Approximate
More informationContribution of Wo¹niakowski, Strako²,... The conjugate gradient method in nite precision computa
Contribution of Wo¹niakowski, Strako²,... The conjugate gradient method in nite precision computations ªaw University of Technology Institute of Mathematics and Computer Science Warsaw, October 7, 2006
More informationCS168: The Modern Algorithmic Toolbox Lecture #8: How PCA Works
CS68: The Modern Algorithmic Toolbox Lecture #8: How PCA Works Tim Roughgarden & Gregory Valiant April 20, 206 Introduction Last lecture introduced the idea of principal components analysis (PCA). The
More information(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax =
. (5 points) (a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? dim N(A), since rank(a) 3. (b) If we also know that Ax = has no solution, what do we know about the rank of A? C(A)
More informationECS231 Handout Subspace projection methods for Solving Large-Scale Eigenvalue Problems. Part I: Review of basic theory of eigenvalue problems
ECS231 Handout Subspace projection methods for Solving Large-Scale Eigenvalue Problems Part I: Review of basic theory of eigenvalue problems 1. Let A C n n. (a) A scalar λ is an eigenvalue of an n n A
More informationExpectation Maximization
Expectation Maximization Machine Learning CSE546 Carlos Guestrin University of Washington November 13, 2014 1 E.M.: The General Case E.M. widely used beyond mixtures of Gaussians The recipe is the same
More informationSolving Constrained Rayleigh Quotient Optimization Problem by Projected QEP Method
Solving Constrained Rayleigh Quotient Optimization Problem by Projected QEP Method Yunshen Zhou Advisor: Prof. Zhaojun Bai University of California, Davis yszhou@math.ucdavis.edu June 15, 2017 Yunshen
More informationTBP MATH33A Review Sheet. November 24, 2018
TBP MATH33A Review Sheet November 24, 2018 General Transformation Matrices: Function Scaling by k Orthogonal projection onto line L Implementation If we want to scale I 2 by k, we use the following: [
More informationNumerical Methods for Solving Large Scale Eigenvalue Problems
Peter Arbenz Computer Science Department, ETH Zürich E-mail: arbenz@inf.ethz.ch arge scale eigenvalue problems, Lecture 2, February 28, 2018 1/46 Numerical Methods for Solving Large Scale Eigenvalue Problems
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences)
AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences) Lecture 19: Computing the SVD; Sparse Linear Systems Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical
More informationResearch Article Some Generalizations and Modifications of Iterative Methods for Solving Large Sparse Symmetric Indefinite Linear Systems
Abstract and Applied Analysis Article ID 237808 pages http://dxdoiorg/055/204/237808 Research Article Some Generalizations and Modifications of Iterative Methods for Solving Large Sparse Symmetric Indefinite
More informationMultivariate Statistical Analysis
Multivariate Statistical Analysis Fall 2011 C. L. Williams, Ph.D. Lecture 4 for Applied Multivariate Analysis Outline 1 Eigen values and eigen vectors Characteristic equation Some properties of eigendecompositions
More informationLecture 5: Randomized methods for low-rank approximation
CBMS Conference on Fast Direct Solvers Dartmouth College June 23 June 27, 2014 Lecture 5: Randomized methods for low-rank approximation Gunnar Martinsson The University of Colorado at Boulder Research
More informationOnline Tensor Factorization for. Feature Selection in EEG
Online Tensor Factorization for Feature Selection in EEG Alric Althoff Honors Thesis, Department of Cognitive Science, University of California - San Diego Supervised by Dr. Virginia de Sa Abstract Tensor
More information5 Selected Topics in Numerical Linear Algebra
5 Selected Topics in Numerical Linear Algebra In this chapter we will be concerned mostly with orthogonal factorizations of rectangular m n matrices A The section numbers in the notes do not align with
More informationPrincipal Component Analysis
Principal Component Analysis CS5240 Theoretical Foundations in Multimedia Leow Wee Kheng Department of Computer Science School of Computing National University of Singapore Leow Wee Kheng (NUS) Principal
More informationarxiv: v5 [math.na] 16 Nov 2017
RANDOM PERTURBATION OF LOW RANK MATRICES: IMPROVING CLASSICAL BOUNDS arxiv:3.657v5 [math.na] 6 Nov 07 SEAN O ROURKE, VAN VU, AND KE WANG Abstract. Matrix perturbation inequalities, such as Weyl s theorem
More informationLinear Subspace Models
Linear Subspace Models Goal: Explore linear models of a data set. Motivation: A central question in vision concerns how we represent a collection of data vectors. The data vectors may be rasterized images,
More information1. Diagonalize the matrix A if possible, that is, find an invertible matrix P and a diagonal
. Diagonalize the matrix A if possible, that is, find an invertible matrix P and a diagonal 3 9 matrix D such that A = P DP, for A =. 3 4 3 (a) P = 4, D =. 3 (b) P = 4, D =. (c) P = 4 8 4, D =. 3 (d) P
More informationLinear Algebra Review. Vectors
Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors
More informationSingular Value Decompsition
Singular Value Decompsition Massoud Malek One of the most useful results from linear algebra, is a matrix decomposition known as the singular value decomposition It has many useful applications in almost
More informationLecture 2: Numerical linear algebra
Lecture 2: Numerical linear algebra QR factorization Eigenvalue decomposition Singular value decomposition Conditioning of a problem Floating point arithmetic and stability of an algorithm Linear algebra
More informationLinear Algebra- Final Exam Review
Linear Algebra- Final Exam Review. Let A be invertible. Show that, if v, v, v 3 are linearly independent vectors, so are Av, Av, Av 3. NOTE: It should be clear from your answer that you know the definition.
More informationMTH 2032 SemesterII
MTH 202 SemesterII 2010-11 Linear Algebra Worked Examples Dr. Tony Yee Department of Mathematics and Information Technology The Hong Kong Institute of Education December 28, 2011 ii Contents Table of Contents
More informationEnabling very large-scale matrix computations via randomization
Enabling very large-scale matrix computations via randomization Gunnar Martinsson The University of Colorado at Boulder Students: Adrianna Gillman Nathan Halko Patrick Young Collaborators: Edo Liberty
More informationCourse Notes: Week 1
Course Notes: Week 1 Math 270C: Applied Numerical Linear Algebra 1 Lecture 1: Introduction (3/28/11) We will focus on iterative methods for solving linear systems of equations (and some discussion of eigenvalues
More informationBindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 Algorithms Notes for 2016-10-31 There are several flavors of symmetric eigenvalue solvers for which there is no equivalent (stable) nonsymmetric solver. We discuss four algorithmic ideas: the workhorse
More informationMaths for Signals and Systems Linear Algebra in Engineering
Maths for Signals and Systems Linear Algebra in Engineering Lectures 13 15, Tuesday 8 th and Friday 11 th November 016 DR TANIA STATHAKI READER (ASSOCIATE PROFFESOR) IN SIGNAL PROCESSING IMPERIAL COLLEGE
More information2. Review of Linear Algebra
2. Review of Linear Algebra ECE 83, Spring 217 In this course we will represent signals as vectors and operators (e.g., filters, transforms, etc) as matrices. This lecture reviews basic concepts from linear
More informationThe Singular Value Decomposition (SVD) and Principal Component Analysis (PCA)
Chapter 5 The Singular Value Decomposition (SVD) and Principal Component Analysis (PCA) 5.1 Basics of SVD 5.1.1 Review of Key Concepts We review some key definitions and results about matrices that will
More informationdimensionality reduction for k-means and low rank approximation
dimensionality reduction for k-means and low rank approximation Michael Cohen, Sam Elder, Cameron Musco, Christopher Musco, Mădălina Persu Massachusetts Institute of Technology 0 overview Simple techniques
More informationMethods for sparse analysis of high-dimensional data, II
Methods for sparse analysis of high-dimensional data, II Rachel Ward May 26, 2011 High dimensional data with low-dimensional structure 300 by 300 pixel images = 90, 000 dimensions 2 / 55 High dimensional
More informationMaths for Signals and Systems Linear Algebra in Engineering
Maths for Signals and Systems Linear Algebra in Engineering Lecture 18, Friday 18 th November 2016 DR TANIA STATHAKI READER (ASSOCIATE PROFFESOR) IN SIGNAL PROCESSING IMPERIAL COLLEGE LONDON Mathematics
More informationCS 229r: Algorithms for Big Data Fall Lecture 17 10/28
CS 229r: Algorithms for Big Data Fall 2015 Prof. Jelani Nelson Lecture 17 10/28 Scribe: Morris Yau 1 Overview In the last lecture we defined subspace embeddings a subspace embedding is a linear transformation
More informationKrylov Subspace Methods to Calculate PageRank
Krylov Subspace Methods to Calculate PageRank B. Vadala-Roth REU Final Presentation August 1st, 2013 How does Google Rank Web Pages? The Web The Graph (A) Ranks of Web pages v = v 1... Dominant Eigenvector
More informationITERATIVE REGULARIZATION WITH MINIMUM-RESIDUAL METHODS
BIT Numerical Mathematics 6-3835/3/431-1 $16. 23, Vol. 43, No. 1, pp. 1 18 c Kluwer Academic Publishers ITERATIVE REGULARIZATION WITH MINIMUM-RESIDUAL METHODS T. K. JENSEN and P. C. HANSEN Informatics
More informationChapter 7: Symmetric Matrices and Quadratic Forms
Chapter 7: Symmetric Matrices and Quadratic Forms (Last Updated: December, 06) These notes are derived primarily from Linear Algebra and its applications by David Lay (4ed). A few theorems have been moved
More informationMath 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
More informationEigenvalue Problems Computation and Applications
Eigenvalue ProblemsComputation and Applications p. 1/36 Eigenvalue Problems Computation and Applications Che-Rung Lee cherung@gmail.com National Tsing Hua University Eigenvalue ProblemsComputation and
More informationMatrices, Moments and Quadrature, cont d
Jim Lambers CME 335 Spring Quarter 2010-11 Lecture 4 Notes Matrices, Moments and Quadrature, cont d Estimation of the Regularization Parameter Consider the least squares problem of finding x such that
More information