COMPLEXITY OF SINGULAR VALUE DECOMPOSITION (SVD)
|
|
|
- Ethelbert Lewis
- 5 years ago
- Views:
Transcription
1 COMPLEXITY OF SINGULAR VALUE DECOMPOSITION (SVD) INPUT : Matrix M R n n in fullmatrix format OPERATION : SVD of M n = 256 Storage 1 2 MB Time (Seconds) LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
2 COMPLEXITY OF SINGULAR VALUE DECOMPOSITION (SVD) INPUT : Matrix M R n n in fullmatrix format OPERATION : SVD of M n = 256 n = 512 Storage Time (Seconds) 1 2 MB MB LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
3 COMPLEXITY OF SINGULAR VALUE DECOMPOSITION (SVD) INPUT : Matrix M R n n in fullmatrix format OPERATION : SVD of M Storage Time (Seconds) 1 n = MB 0.0 n = MB 0.3 n = MB LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
4 COMPLEXITY OF SINGULAR VALUE DECOMPOSITION (SVD) INPUT : Matrix M R n n in fullmatrix format OPERATION : SVD of M Storage Time (Seconds) 1 n = MB 0.0 n = MB 0.3 n = MB 3.3 n = MB LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
5 COMPLEXITY OF SINGULAR VALUE DECOMPOSITION (SVD) INPUT : Matrix M R n n in fullmatrix format OPERATION : SVD of M Storage Time (Seconds) 1 n = MB 0.0 n = MB 0.3 n = MB 3.3 n = MB 40.0 n = MB n = MB 48 m n = GB 7 h n = GB days n = GB 20 days n = GB 5 months n = GB 4 years LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
6 COMPLEXITY OF CSVD IN rkmatrix FORMAT INPUT : Matrix M R n n in rkmatrix format, n = , rank k OPERATION : SVD of M n = Storage Time (Seconds) k = 4 16 MB LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
7 COMPLEXITY OF CSVD IN rkmatrix FORMAT INPUT : Matrix M R n n in rkmatrix format, n = , rank k OPERATION : SVD of M n = Storage Time (Seconds) k = 4 16 MB 0.08 k = 8 32 MB LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
8 COMPLEXITY OF CSVD IN rkmatrix FORMAT INPUT : Matrix M R n n in rkmatrix format, n = , rank k OPERATION : SVD of M n = Storage Time (Seconds) k = 4 16 MB 0.08 k = 8 32 MB 0.21 k = MB LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
9 COMPLEXITY OF CSVD IN rkmatrix FORMAT INPUT : Matrix M R n n in rkmatrix format, n = , rank k OPERATION : SVD of M n = Storage Time (Seconds) k = 4 16 MB 0.08 k = 8 32 MB 0.21 k = MB 0.60 k = MB LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
10 COMPLEXITY OF CSVD IN rkmatrix FORMAT INPUT : Matrix M R n n in rkmatrix format, n = , rank k OPERATION : SVD of M n = Storage Time (Seconds) k = 4 16 MB 0.08 k = 8 32 MB 0.21 k = MB 0.60 k = MB 2.10 k = MB LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
11 COMPLEXITY OF CSVD IN rkmatrix FORMAT INPUT : Matrix M R n n in rkmatrix format, n = , rank k OPERATION : SVD of M n = Storage Time (Seconds) k = 4 16 MB 0.08 k = 8 32 MB 0.21 k = MB 0.60 k = MB 2.10 k = MB 7.64 LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
12 ALGORITHM FOR CSVD IN rkmatrix FORMAT void csvd rkmatrix(prkmatrix r, double * u, double * s, double * v, int *k ){ } LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
13 ALGORITHM FOR CSVD IN rkmatrix FORMAT void csvd rkmatrix(prkmatrix r, double * u, double * s, double * v, int *k ){ double * Qa, *Ra, *Qb, *Rb, *Z, Us, Vs; int kt = r ->kt, rows = r ->rows, cols = r ->cols;... allocate... }... deallocate... LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
14 ALGORITHM FOR CSVD IN rkmatrix FORMAT void csvd rkmatrix(prkmatrix r, double * u, double * s, double * v, int *k ){ double * Qa, *Ra, *Qb, *Rb, *Z, Us, Vs; int kt = r ->kt, rows = r ->rows, cols = r ->cols;... allocate... qr factorisation(r->a, rows, kt, Qa, Ra); qr factorisation(r->b, cols, kt, Qb, Rb); }... deallocate... LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
15 ALGORITHM FOR CSVD IN rkmatrix FORMAT void csvd rkmatrix(prkmatrix r, double * u, double * s, double * v, int *k ){ double * Qa, *Ra, *Qb, *Rb, *Z, Us, Vs; int kt = r ->kt, rows = r ->rows, cols = r ->cols;... allocate... qr factorisation(r->a, rows, kt, Qa, Ra); qr factorisation(r->b, cols, kt, Qb, Rb); multrans2 lapack(kt, kt, kt, Ra, Rb, Z); csvd factorisation(z, Us, s, Vs, kt, k); }... deallocate... LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
16 ALGORITHM FOR CSVD IN rkmatrix FORMAT void csvd rkmatrix(prkmatrix r, double * u, double * s, double * v, int *k ){ double * Qa, *Ra, *Qb, *Rb, *Z, Us, Vs; int kt = r ->kt, rows = r ->rows, cols = r ->cols;... allocate... qr factorisation(r->a, rows, kt, Qa, Ra); qr factorisation(r->b, cols, kt, Qb, Rb); } multrans2 lapack(kt, kt, kt, Ra, Rb, Z); csvd factorisation(z, Us, s, Vs, kt, k); mul lapack(rows, kt, *k, Qa, Us, u); mul lapack(cols, kt, *k, Qb, Vs, v );... deallocate... LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
17 (A2) CROSSAPPROXIMATION, FULL PIVOTING Input: Matrix M R n m, rank k Output:R k = P k ν=1 aν (b ν ) T M for ν = 1,...,k do end for LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
18 (A2) CROSSAPPROXIMATION, FULL PIVOTING Input: Matrix M R n m, rank k Output:R k = P k ν=1 aν (b ν ) T M for ν = 1,...,k do Determine Index of largest absolute entry (i ν, j ν ) := argmax (i,j) M i,j, δ := M iν,j ν. end for LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
19 (A2) CROSSAPPROXIMATION, FULL PIVOTING Input: Matrix M R n m, rank k Output:R k = P k ν=1 aν (b ν ) T M for ν = 1,...,k do Determine Index of largest absolute entry (i ν, j ν ) := argmax (i,j) M i,j, δ := M iν,j ν. if δ = 0 then Stop with rank ν 1, exact representation R ν 1 = M. else end if end for LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
20 (A2) CROSSAPPROXIMATION, FULL PIVOTING Input: Matrix M R n m, rank k Output:R k = P k ν=1 aν (b ν ) T M for ν = 1,...,k do Determine Index of largest absolute entry (i ν, j ν ) := argmax (i,j) M i,j, δ := M iν,j ν. if δ = 0 then Stop with rank ν 1, exact representation R ν 1 = M. else Compute entries of a ν, b ν : (a ν ) i := M i,j ν, i I, (b ν ) j := M iν,j /δ, j J. end if end for LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
21 (A2) CROSSAPPROXIMATION, FULL PIVOTING Input: Matrix M R n m, rank k Output:R k = P k ν=1 aν (b ν ) T M for ν = 1,...,k do Determine Index of largest absolute entry (i ν, j ν ) := argmax (i,j) M i,j, δ := M iν,j ν. if δ = 0 then Stop with rank ν 1, exact representation R ν 1 = M. else Compute entries of a ν, b ν : (a ν ) i := M i,j ν, i I, (b ν ) j := M iν,j /δ, j J. Subtract new rank 1 approximation end if end for M i,j := M i,j (a ν ) i (b ν ) j, i I, j J. LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
22 (A3) CROSSAPPROXIMATION, PARTIAL PIVOTING Input: Matrix M R n m, rank k Wähle ein i 1. for ν = 1,...,k do Output:R k = P k ν=1 aν (b ν ) T M end for LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
23 (A3) CROSSAPPROXIMATION, PARTIAL PIVOTING Input: Matrix M R n m, rank k Wähle ein i 1. for ν = 1,...,k do Determine Index of largest absolute entry (i ν, j ν ) := argmax j M iν,j, δ := M i ν,j ν. Output:R k = P k ν=1 aν (b ν ) T M end for LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
24 (A3) CROSSAPPROXIMATION, PARTIAL PIVOTING Input: Matrix M R n m, rank k Wähle ein i 1. for ν = 1,...,k do Determine Index of largest absolute entry (i ν, j ν ) := argmax j M iν,j, δ := M i ν,j ν. if δ = 0 then Stop with rank ν 1, hopefully R ν 1 = M? else Output:R k = P k ν=1 aν (b ν ) T M end if end for LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
25 (A3) CROSSAPPROXIMATION, PARTIAL PIVOTING Input: Matrix M R n m, rank k Wähle ein i 1. for ν = 1,...,k do Determine Index of largest absolute entry (i ν, j ν ) := argmax j M iν,j, δ := M i ν,j ν. if δ = 0 then Stop with rank ν 1, hopefully R ν 1 = M? else Compute entries of a ν, b ν : ν 1 X (a ν ) i := M i,j (a µ ) i (b µ), i I, ν j ν (b ν ) j := 1 `Mi δ ν,j µ=1 ν 1 X µ=1 Output:R k = P k ν=1 aν (b ν ) T M (a µ) i ν (b µ ) j, j J. end if end for LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
26 (A3) CROSSAPPROXIMATION, PARTIAL PIVOTING Input: Matrix M R n m, rank k Wähle ein i 1. for ν = 1,...,k do Determine Index of largest absolute entry (i ν, j ν ) := argmax j M iν,j, δ := M i ν,j ν. if δ = 0 then Stop with rank ν 1, hopefully R ν 1 = M? else Compute entries of a ν, b ν : ν 1 X (a ν ) i := M i,j (a µ ) i (b µ), i I, ν j ν (b ν ) j := 1 `Mi δ ν,j µ=1 ν 1 X µ=1 New Pivot index i ν+1 = argmax i iν a ν i end if end for Output:R k = P k ν=1 aν (b ν ) T M (a µ) i ν (b µ ) j, j J. (or some other heuristic) LARS GRASEDYCK (RWTH AACHEN) HIERARCHICAL MATRICES SUMMERSCHOOL / 1
Matrix-Product-States/ Tensor-Trains
/ Tensor-Trains November 22, 2016 / Tensor-Trains 1 Matrices What Can We Do With Matrices? Tensors What Can We Do With Tensors? Diagrammatic Notation 2 Singular-Value-Decomposition 3 Curse of Dimensionality
Linear Algebra (Review) Volker Tresp 2017
Linear Algebra (Review) Volker Tresp 2017 1 Vectors k is a scalar (a number) c is a column vector. Thus in two dimensions, c = ( c1 c 2 ) (Advanced: More precisely, a vector is defined in a vector space.
MATH 2331 Linear Algebra. Section 2.1 Matrix Operations. Definition: A : m n, B : n p. Example: Compute AB, if possible.
MATH 2331 Linear Algebra Section 2.1 Matrix Operations Definition: A : m n, B : n p ( 1 2 p ) ( 1 2 p ) AB = A b b b = Ab Ab Ab Example: Compute AB, if possible. 1 Row-column rule: i-j-th entry of AB:
Linear Algebra Section 2.6 : LU Decomposition Section 2.7 : Permutations and transposes Wednesday, February 13th Math 301 Week #4
Linear Algebra Section. : LU Decomposition Section. : Permutations and transposes Wednesday, February 1th Math 01 Week # 1 The LU Decomposition We learned last time that we can factor a invertible matrix
An Introduction to Hierachical (H ) Rank and TT Rank of Tensors with Examples
An Introduction to Hierachical (H ) Rank and TT Rank of Tensors with Examples Lars Grasedyck and Wolfgang Hackbusch Bericht Nr. 329 August 2011 Key words: MSC: hierarchical Tucker tensor rank tensor approximation
Matrix decompositions
Matrix decompositions Zdeněk Dvořák May 19, 2015 Lemma 1 (Schur decomposition). If A is a symmetric real matrix, then there exists an orthogonal matrix Q and a diagonal matrix D such that A = QDQ T. The
Linear Algebra (Review) Volker Tresp 2018
Linear Algebra (Review) Volker Tresp 2018 1 Vectors k, M, N are scalars A one-dimensional array c is a column vector. Thus in two dimensions, ( ) c1 c = c 2 c i is the i-th component of c c T = (c 1, c
Matrix operations Linear Algebra with Computer Science Application
Linear Algebra with Computer Science Application February 14, 2018 1 Matrix operations 11 Matrix operations If A is an m n matrix that is, a matrix with m rows and n columns then the scalar entry in the
Chapter 2 Notes, Linear Algebra 5e Lay
Contents.1 Operations with Matrices..................................1.1 Addition and Subtraction.............................1. Multiplication by a scalar............................ 3.1.3 Multiplication
The Singular Value Decomposition
The Singular Value Decomposition Philippe B. Laval KSU Fall 2015 Philippe B. Laval (KSU) SVD Fall 2015 1 / 13 Review of Key Concepts We review some key definitions and results about matrices that will
be a Householder matrix. Then prove the followings H = I 2 uut Hu = (I 2 uu u T u )u = u 2 uut u
MATH 434/534 Theoretical Assignment 7 Solution Chapter 7 (71) Let H = I 2uuT Hu = u (ii) Hv = v if = 0 be a Householder matrix Then prove the followings H = I 2 uut Hu = (I 2 uu )u = u 2 uut u = u 2u =
Review of Linear Algebra
Review of Linear Algebra Dr Gerhard Roth COMP 40A Winter 05 Version Linear algebra Is an important area of mathematics It is the basis of computer vision Is very widely taught, and there are many resources
The QR Factorization
The QR Factorization How to Make Matrices Nicer Radu Trîmbiţaş Babeş-Bolyai University March 11, 2009 Radu Trîmbiţaş ( Babeş-Bolyai University) The QR Factorization March 11, 2009 1 / 25 Projectors A projector
Pseudoinverse & Moore-Penrose Conditions
ECE 275AB Lecture 7 Fall 2008 V1.0 c K. Kreutz-Delgado, UC San Diego p. 1/1 Lecture 7 ECE 275A Pseudoinverse & Moore-Penrose Conditions ECE 275AB Lecture 7 Fall 2008 V1.0 c K. Kreutz-Delgado, UC San Diego
POLI270 - Linear Algebra
POLI7 - Linear Algebra Septemer 8th Basics a x + a x +... + a n x n b () is the linear form where a, b are parameters and x n are variables. For a given equation such as x +x you only need a variable and
ENGG5781 Matrix Analysis and Computations Lecture 8: QR Decomposition
ENGG5781 Matrix Analysis and Computations Lecture 8: QR Decomposition Wing-Kin (Ken) Ma 2017 2018 Term 2 Department of Electronic Engineering The Chinese University of Hong Kong Lecture 8: QR Decomposition
MODULE 7. where A is an m n real (or complex) matrix. 2) Let K(t, s) be a function of two variables which is continuous on the square [0, 1] [0, 1].
![MODULE 7. where A is an m n real (or complex) matrix. 2) Let K(t, s) be a function of two variables which is continuous on the square [0, 1] [0, 1].](/thumbs/95/122529712.jpg "MODULE 7. where A is an m n real (or complex) matrix. 2) Let K(t, s) be a function of two variables which is continuous on the square [0, 1] [0, 1].")
Topics: Linear operators MODULE 7 We are going to discuss functions = mappings = transformations = operators from one vector space V 1 into another vector space V 2. However, we shall restrict our sights
Matrix Algebra & Elementary Matrices
Matrix lgebra & Elementary Matrices To add two matrices, they must have identical dimensions. To multiply them the number of columns of the first must equal the number of rows of the second. The laws below
Homework Set #8 Solutions
Exercises.2 (p. 19) Homework Set #8 Solutions Assignment: Do #6, 8, 12, 14, 2, 24, 26, 29, 0, 2, 4, 5, 6, 9, 40, 42 6. Reducing the matrix to echelon form: 1 5 2 1 R2 R2 R1 1 5 0 18 12 2 1 R R 2R1 1 5
Review Let A, B, and C be matrices of the same size, and let r and s be scalars. Then
1 Sec 21 Matrix Operations Review Let A, B, and C be matrices of the same size, and let r and s be scalars Then (i) A + B = B + A (iv) r(a + B) = ra + rb (ii) (A + B) + C = A + (B + C) (v) (r + s)a = ra
(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax =
. (5 points) (a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? dim N(A), since rank(a) 3. (b) If we also know that Ax = has no solution, what do we know about the rank of A? C(A)
Bindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 A cautionary tale Notes for 2016-10-05 You have been dropped on a desert island with a laptop with a magic battery of infinite life, a MATLAB license, and a complete lack of knowledge of basic geometry.
Model Order Reduction Techniques
Model Order Reduction Techniques SVD & POD M. Grepl a & K. Veroy-Grepl b a Institut für Geometrie und Praktische Mathematik b Aachen Institute for Advanced Study in Computational Engineering Science (AICES)
Conceptual Questions for Review
Conceptual Questions for Review Chapter 1 1.1 Which vectors are linear combinations of v = (3, 1) and w = (4, 3)? 1.2 Compare the dot product of v = (3, 1) and w = (4, 3) to the product of their lengths.
Linear Algebraic Equations
Linear Algebraic Equations 1 Fundamentals Consider the set of linear algebraic equations n a ij x i b i represented by Ax b j with [A b ] [A b] and (1a) r(a) rank of A (1b) Then Axb has a solution iff
AMS 209, Fall 2015 Final Project Type A Numerical Linear Algebra: Gaussian Elimination with Pivoting for Solving Linear Systems
AMS 209, Fall 205 Final Project Type A Numerical Linear Algebra: Gaussian Elimination with Pivoting for Solving Linear Systems. Overview We are interested in solving a well-defined linear system given
Assignment #10: Diagonalization of Symmetric Matrices, Quadratic Forms, Optimization, Singular Value Decomposition. Name:
Assignment #10: Diagonalization of Symmetric Matrices, Quadratic Forms, Optimization, Singular Value Decomposition Due date: Friday, May 4, 2018 (1:35pm) Name: Section Number Assignment #10: Diagonalization
A fast randomized algorithm for the approximation of matrices preliminary report
DRAFT A fast randomized algorithm for the approximation of matrices preliminary report Yale Department of Computer Science Technical Report #1380 Franco Woolfe, Edo Liberty, Vladimir Rokhlin, and Mark
C&O367: Nonlinear Optimization (Winter 2013) Assignment 4 H. Wolkowicz
C&O367: Nonlinear Optimization (Winter 013) Assignment 4 H. Wolkowicz Posted Mon, Feb. 8 Due: Thursday, Feb. 8 10:00AM (before class), 1 Matrices 1.1 Positive Definite Matrices 1. Let A S n, i.e., let
Chapter 3. Directions: For questions 1-11 mark each statement True or False. Justify each answer.
Chapter 3 Directions: For questions 1-11 mark each statement True or False. Justify each answer. 1. (True False) Asking whether the linear system corresponding to an augmented matrix [ a 1 a 2 a 3 b ]
The Singular Value Decomposition
The Singular Value Decomposition An Important topic in NLA Radu Tiberiu Trîmbiţaş Babeş-Bolyai University February 23, 2009 Radu Tiberiu Trîmbiţaş ( Babeş-Bolyai University)The Singular Value Decomposition
Main matrix factorizations
Main matrix factorizations A P L U P permutation matrix, L lower triangular, U upper triangular Key use: Solve square linear system Ax b. A Q R Q unitary, R upper triangular Key use: Solve square or overdetrmined
Standard Form An LP is in standard form when: All variables are non-negativenegative All constraints are equalities Putting an LP formulation into sta
Chapter 4 Linear Programming: The Simplex Method An Overview of the Simplex Method Standard Form Tableau Form Setting Up the Initial Simplex Tableau Improving the Solution Calculating the Next Tableau
Exploiting Fill-in and Fill-out in Gaussian-like Elimination Procedures on the Extended Jacobian Matrix
2nd European Workshop on AD 1 Exploiting Fill-in and Fill-out in Gaussian-like Elimination Procedures on the Extended Jacobian Matrix Andrew Lyons (Vanderbilt U.) / Uwe Naumann (RWTH Aachen) 2nd European
APPROXIMATING GAUSSIAN PROCESSES
1 / 23 APPROXIMATING GAUSSIAN PROCESSES WITH H 2 -MATRICES Steffen Börm 1 Jochen Garcke 2 1 Christian-Albrechts-Universität zu Kiel 2 Universität Bonn and Fraunhofer SCAI 2 / 23 OUTLINE 1 GAUSSIAN PROCESSES
Numerical Linear Algebra
Numerical Linear Algebra Direct Methods Philippe B. Laval KSU Fall 2017 Philippe B. Laval (KSU) Linear Systems: Direct Solution Methods Fall 2017 1 / 14 Introduction The solution of linear systems is one
Linear Algebra Methods for Data Mining
Linear Algebra Methods for Data Mining Saara Hyvönen, Saara.Hyvonen@cs.helsinki.fi Spring 2007 The Singular Value Decomposition (SVD) continued Linear Algebra Methods for Data Mining, Spring 2007, University
Orthonormal Transformations and Least Squares
Orthonormal Transformations and Least Squares Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo October 30, 2009 Applications of Qx with Q T Q = I 1. solving
Chapter 2: Matrix Algebra
Chapter 2: Matrix Algebra (Last Updated: October 12, 2016) These notes are derived primarily from Linear Algebra and its applications by David Lay (4ed). Write A = 1. Matrix operations [a 1 a n. Then entry
Properties of Matrices and Operations on Matrices
Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,
Finite Math - J-term Section Systems of Linear Equations in Two Variables Example 1. Solve the system
Finite Math - J-term 07 Lecture Notes - //07 Homework Section 4. - 9, 0, 5, 6, 9, 0,, 4, 6, 0, 50, 5, 54, 55, 56, 6, 65 Section 4. - Systems of Linear Equations in Two Variables Example. Solve the system
Topics. Vectors (column matrices): Vector addition and scalar multiplication The matrix of a linear function y Ax The elements of a matrix A : A ij
Topics Vectors (column matrices): Vector addition and scalar multiplication The matrix of a linear function y Ax The elements of a matrix A : A ij or a ij lives in row i and column j Definition of a matrix
Math 1553 Introduction to Linear Algebra
Math 1553 Introduction to Linear Algebra Lecture Notes Chapter 2 Matrix Algebra School of Mathematics The Georgia Institute of Technology Math 1553 Lecture Notes for Chapter 2 Introduction, Slide 1 Section
Householder reflectors are matrices of the form. P = I 2ww T, where w is a unit vector (a vector of 2-norm unity)
Householder QR Householder reflectors are matrices of the form P = I 2ww T, where w is a unit vector (a vector of 2-norm unity) w Px x Geometrically, P x represents a mirror image of x with respect to
Kevin James. MTHSC 3110 Section 2.1 Matrix Operations
MTHSC 3110 Section 2.1 Matrix Operations Notation Let A be an m n matrix, that is, m rows and n columns. We ll refer to the entries of A by their row and column indices. The entry in the i th row and j
Chapter 4 No. 4.0 Answer True or False to the following. Give reasons for your answers.
MATH 434/534 Theoretical Assignment 3 Solution Chapter 4 No 40 Answer True or False to the following Give reasons for your answers If a backward stable algorithm is applied to a computational problem,
A fast randomized algorithm for approximating an SVD of a matrix
A fast randomized algorithm for approximating an SVD of a matrix Joint work with Franco Woolfe, Edo Liberty, and Vladimir Rokhlin Mark Tygert Program in Applied Mathematics Yale University Place July 17,
Solving linear equations with Gaussian Elimination (I)
Term Projects Solving linear equations with Gaussian Elimination The QR Algorithm for Symmetric Eigenvalue Problem The QR Algorithm for The SVD Quasi-Newton Methods Solving linear equations with Gaussian
Fall Inverse of a matrix. Institute: UC San Diego. Authors: Alexander Knop
Fall 2017 Inverse of a matrix Authors: Alexander Knop Institute: UC San Diego Row-Column Rule If the product AB is defined, then the entry in row i and column j of AB is the sum of the products of corresponding
AM205: Assignment 2. i=1
AM05: Assignment Question 1 [10 points] (a) [4 points] For p 1, the p-norm for a vector x R n is defined as: ( n ) 1/p x p x i p ( ) i=1 This definition is in fact meaningful for p < 1 as well, although
1. The Polar Decomposition
A PERSONAL INTERVIEW WITH THE SINGULAR VALUE DECOMPOSITION MATAN GAVISH Part. Theory. The Polar Decomposition In what follows, F denotes either R or C. The vector space F n is an inner product space with
Chapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations
Chapter 1: Systems of linear equations and matrices Section 1.1: Introduction to systems of linear equations Definition: A linear equation in n variables can be expressed in the form a 1 x 1 + a 2 x 2
There are six more problems on the next two pages
Math 435 bg & bu: Topics in linear algebra Summer 25 Final exam Wed., 8/3/5. Justify all your work to receive full credit. Name:. Let A 3 2 5 Find a permutation matrix P, a lower triangular matrix L with
Least Squares. Tom Lyche. October 26, Centre of Mathematics for Applications, Department of Informatics, University of Oslo
Least Squares Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo October 26, 2010 Linear system Linear system Ax = b, A C m,n, b C m, x C n. under-determined
Orthogonal Transformations
Orthogonal Transformations Tom Lyche University of Oslo Norway Orthogonal Transformations p. 1/3 Applications of Qx with Q T Q = I 1. solving least squares problems (today) 2. solving linear equations
Numerical Linear Algebra
Chapter 3 Numerical Linear Algebra We review some techniques used to solve Ax = b where A is an n n matrix, and x and b are n 1 vectors (column vectors). We then review eigenvalues and eigenvectors and
Lecture 02 Linear Algebra Basics
Introduction to Computational Data Analysis CX4240, 2019 Spring Lecture 02 Linear Algebra Basics Chao Zhang College of Computing Georgia Tech These slides are based on slides from Le Song and Andres Mendez-Vazquez.
Chapter 2. Matrix Arithmetic. Chapter 2
Matrix Arithmetic Matrix Addition and Subtraction Addition and subtraction act element-wise on matrices. In order for the addition/subtraction (A B) to be possible, the two matrices A and B must have the
A new truncation strategy for the higher-order singular value decomposition
A new truncation strategy for the higher-order singular value decomposition Nick Vannieuwenhoven K.U.Leuven, Belgium Workshop on Matrix Equations and Tensor Techniques RWTH Aachen, Germany November 21,
Singular Value Decompsition
Singular Value Decompsition Massoud Malek One of the most useful results from linear algebra, is a matrix decomposition known as the singular value decomposition It has many useful applications in almost
1 Number Systems and Errors 1
Contents 1 Number Systems and Errors 1 1.1 Introduction................................ 1 1.2 Number Representation and Base of Numbers............. 1 1.2.1 Normalized Floating-point Representation...........
Linear Algebra Review. Vectors
Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors
7. Symmetric Matrices and Quadratic Forms
Linear Algebra 7. Symmetric Matrices and Quadratic Forms CSIE NCU 1 7. Symmetric Matrices and Quadratic Forms 7.1 Diagonalization of symmetric matrices 2 7.2 Quadratic forms.. 9 7.4 The singular value
1 Non-negative Matrix Factorization (NMF)
2018-06-21 1 Non-negative Matrix Factorization NMF) In the last lecture, we considered low rank approximations to data matrices. We started with the optimal rank k approximation to A R m n via the SVD,
Image Registration Lecture 2: Vectors and Matrices
Image Registration Lecture 2: Vectors and Matrices Prof. Charlene Tsai Lecture Overview Vectors Matrices Basics Orthogonal matrices Singular Value Decomposition (SVD) 2 1 Preliminary Comments Some of this
Orthonormal Transformations
Orthonormal Transformations Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo October 25, 2010 Applications of transformation Q : R m R m, with Q T Q = I 1.
Chapter 4. Determinants
4.2 The Determinant of a Square Matrix 1 Chapter 4. Determinants 4.2 The Determinant of a Square Matrix Note. In this section we define the determinant of an n n matrix. We will do so recursively by defining
y b where U. matrix inverse A 1 ( L. 1 U 1. L 1 U 13 U 23 U 33 U 13 2 U 12 1
LU decomposition -- manual demonstration Instructor: Nam Sun Wang lu-manualmcd LU decomposition, where L is a lower-triangular matrix with as the diagonal elements and U is an upper-triangular matrix Just
A New Scheme for the Tensor Representation
J Fourier Anal Appl (2009) 15: 706 722 DOI 10.1007/s00041-009-9094-9 A New Scheme for the Tensor Representation W. Hackbusch S. Kühn Received: 18 December 2008 / Revised: 29 June 2009 / Published online:
An Adaptive Hierarchical Matrix on Point Iterative Poisson Solver
Malaysian Journal of Mathematical Sciences 10(3): 369 382 (2016) MALAYSIAN JOURNAL OF MATHEMATICAL SCIENCES Journal homepage: http://einspem.upm.edu.my/journal An Adaptive Hierarchical Matrix on Point
Rank Revealing QR factorization. F. Guyomarc h, D. Mezher and B. Philippe
Rank Revealing QR factorization F. Guyomarc h, D. Mezher and B. Philippe 1 Outline Introduction Classical Algorithms Full matrices Sparse matrices Rank-Revealing QR Conclusion CSDA 2005, Cyprus 2 Situation
1. Determine by inspection which of the following sets of vectors is linearly independent. 3 3.
1. Determine by inspection which of the following sets of vectors is linearly independent. (a) (d) 1, 3 4, 1 { [ [,, 1 1] 3]} (b) 1, 4 5, (c) 3 6 (e) 1, 3, 4 4 3 1 4 Solution. The answer is (a): v 1 is
Linear Systems of n equations for n unknowns
Linear Systems of n equations for n unknowns In many application problems we want to find n unknowns, and we have n linear equations Example: Find x,x,x such that the following three equations hold: x
Numerical Methods. Elena loli Piccolomini. Civil Engeneering. piccolom. Metodi Numerici M p. 1/??
Metodi Numerici M p. 1/?? Numerical Methods Elena loli Piccolomini Civil Engeneering http://www.dm.unibo.it/ piccolom elena.loli@unibo.it Metodi Numerici M p. 2/?? Least Squares Data Fitting Measurement
Ack: 1. LD Garcia, MTH 199, Sam Houston State University 2. Linear Algebra and Its Applications - Gilbert Strang
Gaussian Elimination CS6015 : Linear Algebra Ack: 1. LD Garcia, MTH 199, Sam Houston State University 2. Linear Algebra and Its Applications - Gilbert Strang The Gaussian Elimination Method The Gaussian
MODEL REDUCTION BY A CROSS-GRAMIAN APPROACH FOR DATA-SPARSE SYSTEMS
MODEL REDUCTION BY A CROSS-GRAMIAN APPROACH FOR DATA-SPARSE SYSTEMS Ulrike Baur joint work with Peter Benner Mathematics in Industry and Technology Faculty of Mathematics Chemnitz University of Technology
Numerical Analysis Fall. Gauss Elimination
Numerical Analysis 2015 Fall Gauss Elimination Solving systems m g g m m g x x x k k k k k k k k k 3 2 1 3 2 1 3 3 3 2 3 2 2 2 1 0 0 Graphical Method For small sets of simultaneous equations, graphing
This can be accomplished by left matrix multiplication as follows: I
1 Numerical Linear Algebra 11 The LU Factorization Recall from linear algebra that Gaussian elimination is a method for solving linear systems of the form Ax = b, where A R m n and bran(a) In this method
Math 369 Exam #2 Practice Problem Solutions
Math 369 Exam #2 Practice Problem Solutions 2 5. Is { 2, 3, 8 } a basis for R 3? Answer: No, it is not. To show that it is not a basis, it suffices to show that this is not a linearly independent set.
7.5 Operations with Matrices. Copyright Cengage Learning. All rights reserved.
7.5 Operations with Matrices Copyright Cengage Learning. All rights reserved. What You Should Learn Decide whether two matrices are equal. Add and subtract matrices and multiply matrices by scalars. Multiply
AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences)
AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences) Lecture 19: Computing the SVD; Sparse Linear Systems Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical
Row Reduction and Echelon Forms
Row Reduction and Echelon Forms 1 / 29 Key Concepts row echelon form, reduced row echelon form pivot position, pivot, pivot column basic variable, free variable general solution, parametric solution existence
Mon Feb Matrix algebra and matrix inverses. Announcements: Warm-up Exercise:
Math 2270-004 Week 5 notes We will not necessarily finish the material from a given day's notes on that day We may also add or subtract some material as the week progresses, but these notes represent an
6-2 Matrix Multiplication, Inverses and Determinants
Find AB and BA, if possible. 1. A = A = ; A is a 1 2 matrix and B is a 2 2 matrix. Because the number of columns of A is equal to the number of rows of B, AB exists. To find the first entry of AB, find
Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008
Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008 Exam 2 will be held on Tuesday, April 8, 7-8pm in 117 MacMillan What will be covered The exam will cover material from the lectures
ANALYTICAL MATHEMATICS FOR APPLICATIONS 2018 LECTURE NOTES 3
ANALYTICAL MATHEMATICS FOR APPLICATIONS 2018 LECTURE NOTES 3 ISSUED 24 FEBRUARY 2018 1 Gaussian elimination Let A be an (m n)-matrix Consider the following row operations on A (1) Swap the positions any
3.4 Elementary Matrices and Matrix Inverse
Math 220: Summer 2015 3.4 Elementary Matrices and Matrix Inverse A n n elementary matrix is a matrix which is obtained from the n n identity matrix I n n by a single elementary row operation. Elementary
Applied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in
The Singular Value Decomposition (SVD) and Principal Component Analysis (PCA)
Chapter 5 The Singular Value Decomposition (SVD) and Principal Component Analysis (PCA) 5.1 Basics of SVD 5.1.1 Review of Key Concepts We review some key definitions and results about matrices that will
Dimensionality Reduction: PCA. Nicholas Ruozzi University of Texas at Dallas
Dimensionality Reduction: PCA Nicholas Ruozzi University of Texas at Dallas Eigenvalues λ is an eigenvalue of a matrix A R n n if the linear system Ax = λx has at least one non-zero solution If Ax = λx
Recovering any low-rank matrix, provably
Recovering any low-rank matrix, provably Rachel Ward University of Texas at Austin October, 2014 Joint work with Yudong Chen (U.C. Berkeley), Srinadh Bhojanapalli and Sujay Sanghavi (U.T. Austin) Matrix
22A-2 SUMMER 2014 LECTURE 5
A- SUMMER 0 LECTURE 5 NATHANIEL GALLUP Agenda Elimination to the identity matrix Inverse matrices LU factorization Elimination to the identity matrix Previously, we have used elimination to get a system
Matrix Basic Concepts
Matrix Basic Concepts Topics: What is a matrix? Matrix terminology Elements or entries Diagonal entries Address/location of entries Rows and columns Size of a matrix A column matrix; vectors Special types
2.1 Matrices. 3 5 Solve for the variables in the following matrix equation.
2.1 Matrices Reminder: A matrix with m rows and n columns has size m x n. (This is also sometimes referred to as the order of the matrix.) The entry in the ith row and jth column of a matrix A is denoted
Math 671: Tensor Train decomposition methods
Math 671: Eduardo Corona 1 1 University of Michigan at Ann Arbor December 8, 2016 Table of Contents 1 Preliminaries and goal 2 Unfolding matrices for tensorized arrays The Tensor Train decomposition 3
Since the determinant of a diagonal matrix is the product of its diagonal elements it is trivial to see that det(a) = α 2. = max. A 1 x.
APPM 4720/5720 Problem Set 2 Solutions This assignment is due at the start of class on Wednesday, February 9th. Minimal credit will be given for incomplete solutions or solutions that do not provide details
a Short Introduction
Collaborative Filtering in Recommender Systems: a Short Introduction Norm Matloff Dept. of Computer Science University of California, Davis matloff@cs.ucdavis.edu December 3, 2016 Abstract There is a strong
Sparse factorization using low rank submatrices. Cleve Ashcraft LSTC 2010 MUMPS User Group Meeting April 15-16, 2010 Toulouse, FRANCE
Sparse factorization using low rank submatrices Cleve Ashcraft LSTC cleve@lstc.com 21 MUMPS User Group Meeting April 15-16, 21 Toulouse, FRANCE ftp.lstc.com:outgoing/cleve/mumps1 Ashcraft.pdf 1 LSTC Livermore
0.0.1 Section 1.2: Row Reduction and Echelon Forms Echelon form (or row echelon form): 1. All nonzero rows are above any rows of all zeros.
0.0.1 Section 1.2: Row Reduction and Echelon Forms Echelon form (or row echelon form): 1. All nonzero rows are above any rows of all zeros. 2. Each leading entry (i.e. left most nonzero entry) of a row
Maths for Signals and Systems Linear Algebra in Engineering
Maths for Signals and Systems Linear Algebra in Engineering Lectures 13 15, Tuesday 8 th and Friday 11 th November 016 DR TANIA STATHAKI READER (ASSOCIATE PROFFESOR) IN SIGNAL PROCESSING IMPERIAL COLLEGE