Numerical tensor methods and their applications
|
|
- Fay Stanley
- 5 years ago
- Views:
Transcription
1 Numerical tensor methods and their applications 8 May 2013
2 All lectures 4 lectures, 2 May, 08:00-10:00: Introduction: ideas, matrix results, history. 7 May, 08:00-10:00: Novel tensor formats (TT, HT, QTT). 8 May, 08:00-10:00: Advanced tensor methods (eigenproblems, linear systems). 14 May, 08:00-10:00: Advanced topics, recent results and open problems.
3 Brief recap of Lecture 2 Tensor Train format Arithmetics Rounding QTT-format (idea of tensorization) Cross approximation
4 Plan of lecture 2 QTT-format: explicit representations of functions QTT-format: explicit representation of operators Classification theory QTT-Fourier transform QTT-convolution Linear systems Eigenvalue problems
5 QTT-format (publications-first) [1] S. V. Dolgov, B. N. Khoromskij, and D. V. Savostyanov. Superfast Fourier transform using QTT approximation. J. Fourier Anal. Appl., 18(5): , [2] V. Kazeev, B. N. Khoromskij, and E. E. Tyrtyshnikov. Multilevel Toeplitz matrices generated by tensor-structured vectors and convolution with logarithmic complexity. Technical Report 36, MPI MIS, Leipzig, [3] V. A. Kazeev and B. N. Khoromskij. Low-rank explicit QTT representation of the Laplace operator and its inverse. SIAM J. Matrix Anal. Appl., 33(3): , [4] B. N. Khoromskij. O(d log n) Quantics approximation of N d tensors in high-dimensional numerical modeling. Constr. Appr., 34(2): , [5] I. V. Oseledets. Approximation of 2 d 2 d matrices using tensor decomposition. SIAM J. Matrix Anal. Appl., 31(4): , 2010.
6 QTT-format We have a vector v of values of a function f on a uniform grid with 2 d points: v(i) = f (x i ), x i = a + ih, h = (b a)/(n 1).
7 QTT-format We have a vector v of values of a function f on a uniform grid with 2 d points: v(i) = f (x i ), x i = a + ih, h = (b a)/(n 1). Reshaping into a tensor: i (i 0, i 1,..., i d 1 ). i = i 1 + 2i 2 + 4i d 1 i d V (i 1,..., i d ) = v(i).
8 QTT-format Finally, we have: V (i 1,..., i d ) = f (t t d ), t k = a d + 2k i k h
9 Exponential function f (x) = exp λx, Then f (t t d ) = exp(λt 1 )... exp(λt d ), it has rank 1!
10 Linear function f (x) = x f (t t d ) = t t d
11 Linear function(full steps) t 1 + t 2 + t 3 + t 4 =
12 Linear function(full steps) t 1 + t 2 + t 3 + t 4 = ( t 1 1 ) ( 1 t 2 + t 3 + t 4 )
13 Linear function(full steps) t 1 + t 2 + t 3 + t 4 = ( t 1 1 ) ( ) 1 t 2 + t 3 + t 4 = ( t 1 1 ) ( ) ( ) t 2 1 t 3 + t 4 =
14 Linear function(full steps) t 1 + t 2 + t 3 + t 4 = ( t 1 1 ) ( ) 1 t 2 + t 3 + t 4 = ( t 1 1 ) ( ) ( ) = t 2 1 t 3 + t 4 = ( t 1 1 ) ( ) ( ( ) t 2 1 t 3 1) t 4 =
15 Sine function Similar representation can be obtained for a sine function: f (x) = sin λx f (t t d ) = sin(t t d ) = = ( ) ( ) ( ) ( ) sin t 2 cos t 2 sin td 1 cos t sin t 1 cos t 1... d 1 cos xd cos t 2 sin t 2 cos t d 1 sin t d 1 sin x d The rank is still 2!
16 General result Theorem Let f be such that (O. Const. Approx., 2013) f (x + y) = r u α (x)v α (y) α=1 then the QTT-ranks are bounded by r Interesting example: rational functions
17 TT-format for matrices What about matrices?
18 TT-format for matrices What about matrices? Solution - a vector x associated with a d-tensor X (i 1,..., i d ) Linear operators, acting on such tensors, can be indexed as A(i 1,..., i d ; j 1,..., j d ). Terminology: d-level matrix
19 Two-level matrix: illustration Even in 2d it is interesting: A, B are n n, Kronecker product a 11 B a 12 B a 13 B a 14 B C = A B = a 21 B a 22 B a 23 B a 24 B a 31 B a 32 B a 33 B a 34 B a 41 B a 42 B a 43 B a 44 B
20 Two-level matrix: illustration In the index form: C = A B C(i 1, i 2 ; j 1 j 2 ) = A(i 1, j 1 )B(i 2, j 2 )
21 Two-level matrix: illustration In the index form: C = A B C(i 1, i 2 ; j 1 j 2 ) = A(i 1, j 1 )B(i 2, j 2 ) Exactly rank-1 decomposition under permutation
22 Back to d-dimensions Let A be a d-level matrix: A(i 1,..., i d ; j 1,..., j d ) (say, d-dimensional Laplace operator)
23 Back to d-dimensions Let A be a d-level matrix: A(i 1,..., i d ; j 1,..., j d ) (say, d-dimensional Laplace operator) No low TT-ranks if considered as a 2d-array!
24 Back to d-dimensions Right way: permute indices B(i 1 j 1 ; i 2 j 2 ;... i d j d ) = A(i 1,..., i d ; j 1,..., j d ) A(i 1,..., i d ; j 1,..., j d ) = A 1 (i 1, j 1 )A 2 (i 2, j 2 )... A d (i d, j d )
25 Matrix-by-vector product A(i 1,..., i d ; j 1,... j d ) = A 1 (i 1, j 1 )... A d (i d, j d ) X (j 1,..., j d ) = X 1 (j 1 )... X d (j d ) Y (I ) = J A(I, J)X (J) Exercise: Find a formula
26 QTT-matrix Matrices in the QTT-format: a ij = 1 i j + 0.5, i, j = 1,..., 2d. Let us see what are the ranks. (Demo)
27 Laplace operator Consider an operator d 2 dx with Dirichlet boundary 2 conditions, discretized using the simplest finite-difference scheme: (Illustration for n = 4) = (Demo)
28 Laplace operator V. Kazeev: QTT-ranks of the Laplace operator are bounded by 3
29 Laplace operator: general results We will need a special matrix-by-matrix product: [ ] [ ] A 11 A 12 B 11 B 12 = A 21 A 22 B 21 B 22 [ ] A 11 B 11 + A 12 B 21 A 11 B 12 + A 12 B 22 = A 21 B 11 + A 22 B 21 A 21 B 12 + A 22 B 22 Doing Kronecker product for the blocks!
30 Basic QTT-blocks ( ) ( ) ( ) I =, J =, P =, ( ) ( ) I 2 =, I 1 =, ( ) 1 1 E =, F = 1 1 ( ) 1 1, 1 1 ( ) ( ) K =, L =,
31 QTT representation of 1D-Laplace ] (d) DD [I = J I J J J J J (d 2) 2I J J J J
32 QTT representation in the Toolbox In the TT-Toolbox, it is defined via the function tt_qlaplace_dd
33 Wavelets and tensor trains QTT can be applied to matrices: A(i, j) A(i 1,..., i d, j 1,..., j d ) A(i 1, j 1, i 2, j 2,..., i d, j d )
34 Wavelets and tensor trains Smooth function and/or special function: Hilbert image : a ij = 1.0/(i j + 0.5) 540 bytes (QTT) vs 8 KB (JPEG)
35 Wavelets and tensor trains QTT compression of simple images Good: Triangle BAD: Circle 50 bytes (QTT) vs 8 KB (JPEG) 70 KB (QTT) vs 8 KB (JPEG)
36 Wavelets and tensor trains Wavelet tensor train: One step of TT-SVD is equivalent to: U A = v 11 v v 21 v v 31 v v 41 v 42...
37 Problems with a circle Wavelet: First (dominant) rows compress further, others are sparse QTT: Leave only rows of large norms (large singular values)
38 Problems with a circle That is why it is bad: r n, mem = O(n 2 )
39 Idea with the circle Leaving sparse singular vectors a novel digital data compression technique (to be combined with others)! Wavelet decomposition with adaptive number of moments!
40 Image compression New: 50 bytes (WTT) vs 8 KB (JPEG)
41 Image compression latex Lena 7 KB(WTT), PSNR KB (WTT), PSNR KB (WTT), PSNR 38.79
42 Fourier transform Consider now an important operation: the Fourier transform (FFT) Example of a matrix with large QTT-ranks! (We can test) y = Fx, F = w kl, w = exp 2πi n Question: Given x in the QTT-format, can we compute y?
43 FFT matrix and tensor networks Let us do quantization again: k = k 1 + 2k d 1 k d l = l 1 + 2l d 1 l d w kl = pq G pq(k p, l q ) G pq (k p, l q ) = w 2p 1 k p 2 q 1 l q Rank-1 two-dimensional tensor network!
44 FFT matrix and tensor networks(2) k 1, l 1 k 1, l 2 k 1, l 3 k 1, l 4 k 2, l 1 k 2, l 2 k 2, l 3 k 2, l 4 k 3, l 1 k 3, l 2 k 3, l 3 k 3, l 4 k 4, l 1 k 4, l 2 k 4, l 3 k 4, l 4
45 FFT of a vector ( ) l 1,...,l d pq G pq(k p, l q ) x 1 (l 1 )... x d (l d ) It can be rewritten as a Hadamard product of d rank-2 tensors! Hint: l k takes only two values Complexity: O(d 2 r 3 log n)
46 Convolution The convolution operation is defined as c i = j a i jb j. How to do it in the QTT-format?
47 Convolution Idea: Write convolution as c i = jk E ijka k b j E ijk = δ k i+j. Write down the tensor E: E ijk = E(i 1,..., i d, j 1,..., j d, k 1,..., k d+1 ). Permute dimensions: (i 1, j 1, k 1, i 2, j 2, k 2,...). Find an explicit representation
48 Convolution: results Summary of the results (Hackbusch and Kazeev, Khoromskij, Tyrtyshnikov) Toeplitz matrix generated by QTT-vectors of rank r has rank 2r Convolution of two vectors of rank r has rank 2r Multidimensional Toeplitz matrices have the same rank bound, but two QTT-matvecs required
49 Solving eigenvalue problems and linear systems Now, let us go to more advanced problems: Ax = λx Ax = f, with x = X (j 1,..., j d ), f = F (i 1,..., i d ), A(i 1,..., i d ; j 1,..., j d )
50 Tensor-structured solvers Path 1: Do iterative methods with truncation (Krylov, preconditioning, multigrid, etc.) Path 2: Use the information about the structure of the solution
51 Using the information about the solution How to use the information about the solution?
52 Using the information about the solution How to use the information about the solution? Formulate as an optimization problem!
53 Formulation as an optimization problem Ax = λx, A = A, We can minimize the Rayleigh quotient: (Ax, x) (x, x) min, Minimize not over the whole space, but over the set of structured tensors! Nonconvex optimization problem Have to guess the ranks
54 Idea of alternating iterations The idea of alternating iterations is simple: Fix all except X k (i k ). The local problem reduces to the linear eigenvalue problem Guess the rank!
55 Idea of DMRG Here comes the wonderful idea of DMRG (Density Matrix Renormalization Group, S. White, 1993) Generalization of the Wilson renormalization group (=ALS) Optimize not over one core, but over a pair of cores, X k and X k+1!
56 Idea of DMRG X (i 1, i 2, i 3, i 4 ) = X 1 (i 1 )X 2 (i 2 )X 3 (i 3 )X 4 (i 4 ) = = W 12 (i 1, i 2 )X 3 (i 3 )X 4 (i 4 ). Solve for W 12 Split back by the SVD: W 12 (i 1, i 2 ) = X 1 (i 1 )X 2 (i 2 ). The rank is determined adaptively!
57 DMRG and QTT The DMRG method creates modes of size n 2. Very good for spin systems (n = 2 or n = 4) Very good for the QTT (n = 2).
58 DMRG and QTT: papers Holtz, S., Rohwedder, T., Schneider, R., The alternating linear scheme for tensor optimization in the tensor train format (idea) S.V. Dolgov,, Solution of linear systems and matrix inversion in the TT-format (working code)
59 Difficulties in the DMRG S.V. Dolgov, Solution of linear systems and matrix inversion in the TT-format SVD-based truncation: L 2 -norm approximation of x, but the equation can be differential How to avoid local minima Fast solution of local systems Appplication to matrix inversion
60 Large mode sizes What if the mode size is large? Basically, the question is now how to increase the k-th rank. Answer: Using (projected) residuals of the Krylov methods!
61 Large mode sizes The k-th ALS-step: The x is in the special linear subspace: x = (U I V )φ, Analogously to the rounding procedure U and V are structured-orthogonal matrix and φ is r k 1 n k r k. The local problem is then Û AÛφ = Ûf. Can enrich the basis with the (low-rank) approximation of the two-block residual
62 AMEN: further details S.V. Dolgov and D.V. Savostyanov, 2013: Alternating minimal energy methods for linear systems in higher dimensions. Part I/II For more details: convergence estimates, algorithmic details and so on. Still A LOT to be done on algorithms...
63 Solving linear systems and eigenvalue problems (Demo) Solving the high-dimensional Poisson equation in the QTT-format: u = f.
64 Other important problems Solving block eigenvalue problems (minimizing block Rayleigh quotient) Solving nonstationary problems dy dt = Ay, how to rewrite as a minimization problem
65 Next lecture The plan for the next (and the last!) Applications, new results, open problems
Numerical tensor methods and their applications
Numerical tensor methods and their applications 14 May 2013 All lectures 4 lectures, 2 May, 08:00-10:00: Introduction: ideas, matrix results, history. 7 May, 08:00-10:00: Novel tensor formats (TT, HT,
More informationIntroduction to the Tensor Train Decomposition and Its Applications in Machine Learning
Introduction to the Tensor Train Decomposition and Its Applications in Machine Learning Anton Rodomanov Higher School of Economics, Russia Bayesian methods research group (http://bayesgroup.ru) 14 March
More informationMath 671: Tensor Train decomposition methods II
Math 671: Tensor Train decomposition methods II Eduardo Corona 1 1 University of Michigan at Ann Arbor December 13, 2016 Table of Contents 1 What we ve talked about so far: 2 The Tensor Train decomposition
More informationfür Mathematik in den Naturwissenschaften Leipzig
ŠܹÈÐ Ò ¹ÁÒ Ø ØÙØ für Mathematik in den Naturwissenschaften Leipzig Quantics-TT Approximation of Elliptic Solution Operators in Higher Dimensions (revised version: January 2010) by Boris N. Khoromskij,
More informationNUMERICAL METHODS WITH TENSOR REPRESENTATIONS OF DATA
NUMERICAL METHODS WITH TENSOR REPRESENTATIONS OF DATA Institute of Numerical Mathematics of Russian Academy of Sciences eugene.tyrtyshnikov@gmail.com 2 June 2012 COLLABORATION MOSCOW: I.Oseledets, D.Savostyanov
More informationTensor networks and deep learning
Tensor networks and deep learning I. Oseledets, A. Cichocki Skoltech, Moscow 26 July 2017 What is a tensor Tensor is d-dimensional array: A(i 1,, i d ) Why tensors Many objects in machine learning can
More informationNEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING
NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING Institute of Numerical Mathematics of Russian Academy of Sciences eugene.tyrtyshnikov@gmail.com 11 October 2012 COLLABORATION MOSCOW:
More informationDirect Numerical Solution of Algebraic Lyapunov Equations For Large-Scale Systems Using Quantized Tensor Trains
Direct Numerical Solution of Algebraic Lyapunov Equations For Large-Scale Systems Using Quantized Tensor Trains Michael Nip Center for Control, Dynamical Systems, and Computation University of California,
More informationMath 671: Tensor Train decomposition methods
Math 671: Eduardo Corona 1 1 University of Michigan at Ann Arbor December 8, 2016 Table of Contents 1 Preliminaries and goal 2 Unfolding matrices for tensorized arrays The Tensor Train decomposition 3
More informationMatrix-Product-States/ Tensor-Trains
/ Tensor-Trains November 22, 2016 / Tensor-Trains 1 Matrices What Can We Do With Matrices? Tensors What Can We Do With Tensors? Diagrammatic Notation 2 Singular-Value-Decomposition 3 Curse of Dimensionality
More informationMax Planck Institute Magdeburg Preprints
Thomas Mach Computing Inner Eigenvalues of Matrices in Tensor Train Matrix Format MAX PLANCK INSTITUT FÜR DYNAMIK KOMPLEXER TECHNISCHER SYSTEME MAGDEBURG Max Planck Institute Magdeburg Preprints MPIMD/11-09
More informationTENSORS AND COMPUTATIONS
Institute of Numerical Mathematics of Russian Academy of Sciences eugene.tyrtyshnikov@gmail.com 11 September 2013 REPRESENTATION PROBLEM FOR MULTI-INDEX ARRAYS Going to consider an array a(i 1,..., i d
More informationAPPLIED NUMERICAL LINEAR ALGEBRA
APPLIED NUMERICAL LINEAR ALGEBRA James W. Demmel University of California Berkeley, California Society for Industrial and Applied Mathematics Philadelphia Contents Preface 1 Introduction 1 1.1 Basic Notation
More informationLow Rank Approximation Lecture 7. Daniel Kressner Chair for Numerical Algorithms and HPC Institute of Mathematics, EPFL
Low Rank Approximation Lecture 7 Daniel Kressner Chair for Numerical Algorithms and HPC Institute of Mathematics, EPFL daniel.kressner@epfl.ch 1 Alternating least-squares / linear scheme General setting:
More informationLecture 1: Introduction to low-rank tensor representation/approximation. Center for Uncertainty Quantification. Alexander Litvinenko
tifica Lecture 1: Introduction to low-rank tensor representation/approximation Alexander Litvinenko http://sri-uq.kaust.edu.sa/ KAUST Figure : KAUST campus, 5 years old, approx. 7000 people (include 1400
More informationTENSOR APPROXIMATION TOOLS FREE OF THE CURSE OF DIMENSIONALITY
TENSOR APPROXIMATION TOOLS FREE OF THE CURSE OF DIMENSIONALITY Eugene Tyrtyshnikov Institute of Numerical Mathematics Russian Academy of Sciences (joint work with Ivan Oseledets) WHAT ARE TENSORS? Tensors
More informationTensor networks, TT (Matrix Product States) and Hierarchical Tucker decomposition
Tensor networks, TT (Matrix Product States) and Hierarchical Tucker decomposition R. Schneider (TUB Matheon) John von Neumann Lecture TU Munich, 2012 Setting - Tensors V ν := R n, H d = H := d ν=1 V ν
More informationTensor-Product Representation of Operators and Functions (7 introductory lectures) Boris N. Khoromskij
1 Everything should be made as simple as possible, but not simpler. A. Einstein (1879-1955) Tensor-Product Representation of Operators and Functions (7 introductory lectures) Boris N. Khoromskij University
More informationAn Introduction to Hierachical (H ) Rank and TT Rank of Tensors with Examples
An Introduction to Hierachical (H ) Rank and TT Rank of Tensors with Examples Lars Grasedyck and Wolfgang Hackbusch Bericht Nr. 329 August 2011 Key words: MSC: hierarchical Tucker tensor rank tensor approximation
More informationLecture 4. Tensor-Related Singular Value Decompositions. Charles F. Van Loan
From Matrix to Tensor: The Transition to Numerical Multilinear Algebra Lecture 4. Tensor-Related Singular Value Decompositions Charles F. Van Loan Cornell University The Gene Golub SIAM Summer School 2010
More informationMotivation: Sparse matrices and numerical PDE's
Lecture 20: Numerical Linear Algebra #4 Iterative methods and Eigenproblems Outline 1) Motivation: beyond LU for Ax=b A little PDE's and sparse matrices A) Temperature Equation B) Poisson Equation 2) Splitting
More informationLecture Notes for Inf-Mat 3350/4350, Tom Lyche
Lecture Notes for Inf-Mat 3350/4350, 2007 Tom Lyche August 5, 2007 2 Contents Preface vii I A Review of Linear Algebra 1 1 Introduction 3 1.1 Notation............................... 3 2 Vectors 5 2.1 Vector
More information4. Multi-linear algebra (MLA) with Kronecker-product data.
ect. 3. Tensor-product interpolation. Introduction to MLA. B. Khoromskij, Leipzig 2007(L3) 1 Contents of Lecture 3 1. Best polynomial approximation. 2. Error bound for tensor-product interpolants. - Polynomial
More informationFinal Examination. CS 205A: Mathematical Methods for Robotics, Vision, and Graphics (Fall 2013), Stanford University
Final Examination CS 205A: Mathematical Methods for Robotics, Vision, and Graphics (Fall 2013), Stanford University The exam runs for 3 hours. The exam contains eight problems. You must complete the first
More informationComputational methods for large-scale linear matrix equations and application to FDEs. V. Simoncini
Computational methods for large-scale linear matrix equations and application to FDEs V. Simoncini Dipartimento di Matematica, Università di Bologna valeria.simoncini@unibo.it Joint work with: Tobias Breiten,
More informationLecture 9 Approximations of Laplace s Equation, Finite Element Method. Mathématiques appliquées (MATH0504-1) B. Dewals, C.
Lecture 9 Approximations of Laplace s Equation, Finite Element Method Mathématiques appliquées (MATH54-1) B. Dewals, C. Geuzaine V1.2 23/11/218 1 Learning objectives of this lecture Apply the finite difference
More informationTensor Product Approximation
Tensor Product Approximation R. Schneider (TUB Matheon) Mariapfarr, 2014 Acknowledgment DFG Priority program SPP 1324 Extraction of essential information from complex data Co-workers: T. Rohwedder (HUB),
More informationNumerical Methods in Matrix Computations
Ake Bjorck Numerical Methods in Matrix Computations Springer Contents 1 Direct Methods for Linear Systems 1 1.1 Elements of Matrix Theory 1 1.1.1 Matrix Algebra 2 1.1.2 Vector Spaces 6 1.1.3 Submatrices
More informationInstitute for Computational Mathematics Hong Kong Baptist University
Institute for Computational Mathematics Hong Kong Baptist University ICM Research Report 08-0 How to find a good submatrix S. A. Goreinov, I. V. Oseledets, D. V. Savostyanov, E. E. Tyrtyshnikov, N. L.
More informationMain matrix factorizations
Main matrix factorizations A P L U P permutation matrix, L lower triangular, U upper triangular Key use: Solve square linear system Ax b. A Q R Q unitary, R upper triangular Key use: Solve square or overdetrmined
More informationM.A. Botchev. September 5, 2014
Rome-Moscow school of Matrix Methods and Applied Linear Algebra 2014 A short introduction to Krylov subspaces for linear systems, matrix functions and inexact Newton methods. Plan and exercises. M.A. Botchev
More information1. Structured representation of high-order tensors revisited. 2. Multi-linear algebra (MLA) with Kronecker-product data.
Lect. 4. Toward MLA in tensor-product formats B. Khoromskij, Leipzig 2007(L4) 1 Contents of Lecture 4 1. Structured representation of high-order tensors revisited. - Tucker model. - Canonical (PARAFAC)
More informationPreliminary Examination, Numerical Analysis, August 2016
Preliminary Examination, Numerical Analysis, August 2016 Instructions: This exam is closed books and notes. The time allowed is three hours and you need to work on any three out of questions 1-4 and any
More informationBook of Abstracts - Extract th Annual Meeting. March 23-27, 2015 Lecce, Italy. jahrestagung.gamm-ev.de
GESELLSCHAFT für ANGEWANDTE MATHEMATIK und MECHANIK e.v. INTERNATIONAL ASSOCIATION of APPLIED MATHEMATICS and MECHANICS 86 th Annual Meeting of the International Association of Applied Mathematics and
More informationLecture 18 Classical Iterative Methods
Lecture 18 Classical Iterative Methods MIT 18.335J / 6.337J Introduction to Numerical Methods Per-Olof Persson November 14, 2006 1 Iterative Methods for Linear Systems Direct methods for solving Ax = b,
More informationStatistical Geometry Processing Winter Semester 2011/2012
Statistical Geometry Processing Winter Semester 2011/2012 Linear Algebra, Function Spaces & Inverse Problems Vector and Function Spaces 3 Vectors vectors are arrows in space classically: 2 or 3 dim. Euclidian
More informationTensor approach to optimal control problems with fractional d-dimensional elliptic operator in constraints
Tensor approach to optimal control problems with fractional d-dimensional elliptic operator in constraints Gennadij Heidel Venera Khoromskaia Boris N. Khoromskij Volker Schulz arxiv:809.097v2 [math.na]
More informationName: INSERT YOUR NAME HERE. Due to dropbox by 6pm PDT, Wednesday, December 14, 2011
AMath 584 Name: INSERT YOUR NAME HERE Take-home Final UWNetID: INSERT YOUR NETID Due to dropbox by 6pm PDT, Wednesday, December 14, 2011 The main part of the assignment (Problems 1 3) is worth 80 points.
More informationHands-on Matrix Algebra Using R
Preface vii 1. R Preliminaries 1 1.1 Matrix Defined, Deeper Understanding Using Software.. 1 1.2 Introduction, Why R?.................... 2 1.3 Obtaining R.......................... 4 1.4 Reference Manuals
More informationLecture 1: Center for Uncertainty Quantification. Alexander Litvinenko. Computation of Karhunen-Loeve Expansion:
tifica Lecture 1: Computation of Karhunen-Loeve Expansion: Alexander Litvinenko http://sri-uq.kaust.edu.sa/ Stochastic PDEs We consider div(κ(x, ω) u) = f (x, ω) in G, u = 0 on G, with stochastic coefficients
More informationQ1 Q2 Q3 Q4 Tot Letr Xtra
Mathematics 54.1 Final Exam, 12 May 2011 180 minutes, 90 points NAME: ID: GSI: INSTRUCTIONS: You must justify your answers, except when told otherwise. All the work for a question should be on the respective
More informationMcGill University Department of Mathematics and Statistics. Ph.D. preliminary examination, PART A. PURE AND APPLIED MATHEMATICS Paper BETA
McGill University Department of Mathematics and Statistics Ph.D. preliminary examination, PART A PURE AND APPLIED MATHEMATICS Paper BETA 17 August, 2018 1:00 p.m. - 5:00 p.m. INSTRUCTIONS: (i) This paper
More informationThe multiple-vector tensor-vector product
I TD MTVP C KU Leuven August 29, 2013 In collaboration with: N Vanbaelen, K Meerbergen, and R Vandebril Overview I TD MTVP C 1 Introduction Inspiring example Notation 2 Tensor decompositions The CP decomposition
More informationKrylov subspace methods for linear systems with tensor product structure
Krylov subspace methods for linear systems with tensor product structure Christine Tobler Seminar for Applied Mathematics, ETH Zürich 19. August 2009 Outline 1 Introduction 2 Basic Algorithm 3 Convergence
More information( nonlinear constraints)
Wavelet Design & Applications Basic requirements: Admissibility (single constraint) Orthogonality ( nonlinear constraints) Sparse Representation Smooth functions well approx. by Fourier High-frequency
More informationScientific Computing with Case Studies SIAM Press, Lecture Notes for Unit VII Sparse Matrix
Scientific Computing with Case Studies SIAM Press, 2009 http://www.cs.umd.edu/users/oleary/sccswebpage Lecture Notes for Unit VII Sparse Matrix Computations Part 1: Direct Methods Dianne P. O Leary c 2008
More informationLecture 1: Introduction to low-rank tensor representation/approximation. Center for Uncertainty Quantification. Alexander Litvinenko
tifica Lecture 1: Introduction to low-rank tensor representation/approximation Alexander Litvinenko http://sri-uq.kaust.edu.sa/ KAUST Figure : KAUST campus, 5 years old, approx. 7000 people (include 1400
More informationContributors and Resources
for Introduction to Numerical Methods for d-bar Problems Jennifer Mueller, presented by Andreas Stahel Colorado State University Bern University of Applied Sciences, Switzerland Lexington, Kentucky, May
More informationMax-Planck-Institut fur Mathematik in den Naturwissenschaften Leipzig H 2 -matrix approximation of integral operators by interpolation by Wolfgang Hackbusch and Steen Borm Preprint no.: 04 200 H 2 -Matrix
More informationfür Mathematik in den Naturwissenschaften Leipzig
ŠܹÈÐ Ò ¹ÁÒ Ø ØÙØ für Mathematik in den Naturwissenschaften Leipzig Quantics-TT collocation approximation of parameter-dependent and stochastic elliptic PDEs by Boris N. Khoromskij, and Ivan V. Oseledets
More informationTensor properties of multilevel Toeplitz and related. matrices.
Tensor properties of multilevel Toeplitz and related matrices Vadim Olshevsky, University of Connecticut Ivan Oseledets, Eugene Tyrtyshnikov 1 Institute of Numerical Mathematics, Russian Academy of Sciences,
More informationAlgebra C Numerical Linear Algebra Sample Exam Problems
Algebra C Numerical Linear Algebra Sample Exam Problems Notation. Denote by V a finite-dimensional Hilbert space with inner product (, ) and corresponding norm. The abbreviation SPD is used for symmetric
More informationLecture4 INF-MAT : 5. Fast Direct Solution of Large Linear Systems
Lecture4 INF-MAT 4350 2010: 5. Fast Direct Solution of Large Linear Systems Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo September 16, 2010 Test Matrix
More informationGeometric Modeling Summer Semester 2012 Linear Algebra & Function Spaces
Geometric Modeling Summer Semester 2012 Linear Algebra & Function Spaces (Recap) Announcement Room change: On Thursday, April 26th, room 024 is occupied. The lecture will be moved to room 021, E1 4 (the
More informationA FAST SOLVER FOR ELLIPTIC EQUATIONS WITH HARMONIC COEFFICIENT APPROXIMATIONS
Proceedings of ALGORITMY 2005 pp. 222 229 A FAST SOLVER FOR ELLIPTIC EQUATIONS WITH HARMONIC COEFFICIENT APPROXIMATIONS ELENA BRAVERMAN, MOSHE ISRAELI, AND ALEXANDER SHERMAN Abstract. Based on a fast subtractional
More informationDimensionality Reduction: PCA. Nicholas Ruozzi University of Texas at Dallas
Dimensionality Reduction: PCA Nicholas Ruozzi University of Texas at Dallas Eigenvalues λ is an eigenvalue of a matrix A R n n if the linear system Ax = λx has at least one non-zero solution If Ax = λx
More informationSTA141C: Big Data & High Performance Statistical Computing
STA141C: Big Data & High Performance Statistical Computing Numerical Linear Algebra Background Cho-Jui Hsieh UC Davis May 15, 2018 Linear Algebra Background Vectors A vector has a direction and a magnitude
More informationComputational Methods. Eigenvalues and Singular Values
Computational Methods Eigenvalues and Singular Values Manfred Huber 2010 1 Eigenvalues and Singular Values Eigenvalues and singular values describe important aspects of transformations and of data relations
More informationTopics. The CG Algorithm Algorithmic Options CG s Two Main Convergence Theorems
Topics The CG Algorithm Algorithmic Options CG s Two Main Convergence Theorems What about non-spd systems? Methods requiring small history Methods requiring large history Summary of solvers 1 / 52 Conjugate
More informationThis work has been submitted to ChesterRep the University of Chester s online research repository.
This work has been submitted to ChesterRep the University of Chester s online research repository http://chesterrep.openrepository.com Author(s): Daniel Tock Title: Tensor decomposition and its applications
More informationPreconditioned inverse iteration and shift-invert Arnoldi method
Preconditioned inverse iteration and shift-invert Arnoldi method Melina Freitag Department of Mathematical Sciences University of Bath CSC Seminar Max-Planck-Institute for Dynamics of Complex Technical
More informationProjected Schur Complement Method for Solving Non-Symmetric Saddle-Point Systems (Arising from Fictitious Domain Approaches)
Projected Schur Complement Method for Solving Non-Symmetric Saddle-Point Systems (Arising from Fictitious Domain Approaches) Jaroslav Haslinger, Charles University, Prague Tomáš Kozubek, VŠB TU Ostrava
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences)
AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences) Lecture 1: Course Overview; Matrix Multiplication Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical
More informationEigenvalue Problems CHAPTER 1 : PRELIMINARIES
Eigenvalue Problems CHAPTER 1 : PRELIMINARIES Heinrich Voss voss@tu-harburg.de Hamburg University of Technology Institute of Mathematics TUHH Heinrich Voss Preliminaries Eigenvalue problems 2012 1 / 14
More informationA fast and well-conditioned spectral method: The US method
A fast and well-conditioned spectral method: The US method Alex Townsend University of Oxford Leslie Fox Prize, 24th of June 23 Partially based on: S. Olver & T., A fast and well-conditioned spectral method,
More information6.4 Krylov Subspaces and Conjugate Gradients
6.4 Krylov Subspaces and Conjugate Gradients Our original equation is Ax = b. The preconditioned equation is P Ax = P b. When we write P, we never intend that an inverse will be explicitly computed. P
More informationSTA141C: Big Data & High Performance Statistical Computing
STA141C: Big Data & High Performance Statistical Computing Lecture 5: Numerical Linear Algebra Cho-Jui Hsieh UC Davis April 20, 2017 Linear Algebra Background Vectors A vector has a direction and a magnitude
More informationMachine Learning with Quantum-Inspired Tensor Networks
Machine Learning with Quantum-Inspired Tensor Networks E.M. Stoudenmire and David J. Schwab Advances in Neural Information Processing 29 arxiv:1605.05775 RIKEN AICS - Mar 2017 Collaboration with David
More informationAnnounce Statistical Motivation Properties Spectral Theorem Other ODE Theory Spectral Embedding. Eigenproblems I
Eigenproblems I CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Doug James (and Justin Solomon) CS 205A: Mathematical Methods Eigenproblems I 1 / 33 Announcements Homework 1: Due tonight
More informationThe automatic solution of PDEs using a global spectral method
The automatic solution of PDEs using a global spectral method 2014 CBMS-NSF Conference, 29th June 2014 Alex Townsend PhD student University of Oxford (with Sheehan Olver) Supervised by Nick Trefethen,
More informationDynamical low rank approximation in hierarchical tensor format
Dynamical low rank approximation in hierarchical tensor format R. Schneider (TUB Matheon) John von Neumann Lecture TU Munich, 2012 Motivation Equations describing complex systems with multi-variate solution
More informationSPARSE SIGNAL RESTORATION. 1. Introduction
SPARSE SIGNAL RESTORATION IVAN W. SELESNICK 1. Introduction These notes describe an approach for the restoration of degraded signals using sparsity. This approach, which has become quite popular, is useful
More informationAnnounce Statistical Motivation ODE Theory Spectral Embedding Properties Spectral Theorem Other. Eigenproblems I
Eigenproblems I CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Doug James (and Justin Solomon) CS 205A: Mathematical Methods Eigenproblems I 1 / 33 Announcements Homework 1: Due tonight
More informationFall TMA4145 Linear Methods. Exercise set Given the matrix 1 2
Norwegian University of Science and Technology Department of Mathematical Sciences TMA445 Linear Methods Fall 07 Exercise set Please justify your answers! The most important part is how you arrive at an
More informationContents. Acknowledgments
Table of Preface Acknowledgments Notation page xii xx xxi 1 Signals and systems 1 1.1 Continuous and discrete signals 1 1.2 Unit step and nascent delta functions 4 1.3 Relationship between complex exponentials
More informationDynamical low-rank approximation
Dynamical low-rank approximation Christian Lubich Univ. Tübingen Genève, Swiss Numerical Analysis Day, 17 April 2015 Coauthors Othmar Koch 2007, 2010 Achim Nonnenmacher 2008 Dajana Conte 2010 Thorsten
More informationA New Scheme for the Tensor Representation
J Fourier Anal Appl (2009) 15: 706 722 DOI 10.1007/s00041-009-9094-9 A New Scheme for the Tensor Representation W. Hackbusch S. Kühn Received: 18 December 2008 / Revised: 29 June 2009 / Published online:
More informationarxiv: v2 [math.na] 13 Dec 2014
Very Large-Scale Singular Value Decomposition Using Tensor Train Networks arxiv:1410.6895v2 [math.na] 13 Dec 2014 Namgil Lee a and Andrzej Cichocki a a Laboratory for Advanced Brain Signal Processing,
More informationSTAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 9
STAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 9 1. qr and complete orthogonal factorization poor man s svd can solve many problems on the svd list using either of these factorizations but they
More informationThe tensor structure of a class of adaptive algebraic wavelet transforms
The tensor structure of a class of adaptive algebraic wavelet transforms V. Kazeev and I. Oseledets Research Report No. 2013-28 August 2013 Seminar für Angewandte Mathematik Eidgenössische Technische Hochschule
More informationFrom Stationary Methods to Krylov Subspaces
Week 6: Wednesday, Mar 7 From Stationary Methods to Krylov Subspaces Last time, we discussed stationary methods for the iterative solution of linear systems of equations, which can generally be written
More informationLinear Algebra. Session 12
Linear Algebra. Session 12 Dr. Marco A Roque Sol 08/01/2017 Example 12.1 Find the constant function that is the least squares fit to the following data x 0 1 2 3 f(x) 1 0 1 2 Solution c = 1 c = 0 f (x)
More informationSolving Symmetric Indefinite Systems with Symmetric Positive Definite Preconditioners
Solving Symmetric Indefinite Systems with Symmetric Positive Definite Preconditioners Eugene Vecharynski 1 Andrew Knyazev 2 1 Department of Computer Science and Engineering University of Minnesota 2 Department
More informationLeast Squares Optimization
Least Squares Optimization The following is a brief review of least squares optimization and constrained optimization techniques. I assume the reader is familiar with basic linear algebra, including the
More informationAMS 147 Computational Methods and Applications Lecture 17 Copyright by Hongyun Wang, UCSC
Lecture 17 Copyright by Hongyun Wang, UCSC Recap: Solving linear system A x = b Suppose we are given the decomposition, A = L U. We solve (LU) x = b in 2 steps: *) Solve L y = b using the forward substitution
More informationUNDERSTANDING ENGINEERING MATHEMATICS
UNDERSTANDING ENGINEERING MATHEMATICS JOHN BIRD WORKED SOLUTIONS TO EXERCISES 1 INTRODUCTION In Understanding Engineering Mathematic there are over 750 further problems arranged regularly throughout the
More informationQualifying Examination
Summer 24 Day. Monday, September 5, 24 You have three hours to complete this exam. Work all problems. Start each problem on a All problems are 2 points. Please email any electronic files in support of
More informationNumerical Linear and Multilinear Algebra in Quantum Tensor Networks
Numerical Linear and Multilinear Algebra in Quantum Tensor Networks Konrad Waldherr October 20, 2013 Joint work with Thomas Huckle QCCC 2013, Prien, October 20, 2013 1 Outline Numerical (Multi-) Linear
More informationGeometric Modeling Summer Semester 2010 Mathematical Tools (1)
Geometric Modeling Summer Semester 2010 Mathematical Tools (1) Recap: Linear Algebra Today... Topics: Mathematical Background Linear algebra Analysis & differential geometry Numerical techniques Geometric
More informationA First Course in Wavelets with Fourier Analysis
* A First Course in Wavelets with Fourier Analysis Albert Boggess Francis J. Narcowich Texas A& M University, Texas PRENTICE HALL, Upper Saddle River, NJ 07458 Contents Preface Acknowledgments xi xix 0
More informationHomework 1 Elena Davidson (B) (C) (D) (E) (F) (G) (H) (I)
CS 106 Spring 2004 Homework 1 Elena Davidson 8 April 2004 Problem 1.1 Let B be a 4 4 matrix to which we apply the following operations: 1. double column 1, 2. halve row 3, 3. add row 3 to row 1, 4. interchange
More informationHOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION)
HOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION) PROFESSOR STEVEN MILLER: BROWN UNIVERSITY: SPRING 2007 1. CHAPTER 1: MATRICES AND GAUSSIAN ELIMINATION Page 9, # 3: Describe
More informationAn Accurate Fourier-Spectral Solver for Variable Coefficient Elliptic Equations
An Accurate Fourier-Spectral Solver for Variable Coefficient Elliptic Equations Moshe Israeli Computer Science Department, Technion-Israel Institute of Technology, Technion city, Haifa 32000, ISRAEL Alexander
More informationApplied Machine Learning for Biomedical Engineering. Enrico Grisan
Applied Machine Learning for Biomedical Engineering Enrico Grisan enrico.grisan@dei.unipd.it Data representation To find a representation that approximates elements of a signal class with a linear combination
More informationPreconditioning for Nonsymmetry and Time-dependence
Preconditioning for Nonsymmetry and Time-dependence Andy Wathen Oxford University, UK joint work with Jen Pestana and Elle McDonald Jeju, Korea, 2015 p.1/24 Iterative methods For self-adjoint problems/symmetric
More informationAM 205: lecture 8. Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition
AM 205: lecture 8 Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition QR Factorization A matrix A R m n, m n, can be factorized
More informationLinear Algebra Review
January 29, 2013 Table of contents Metrics Metric Given a space X, then d : X X R + 0 and z in X if: d(x, y) = 0 is equivalent to x = y d(x, y) = d(y, x) d(x, y) d(x, z) + d(z, y) is a metric is for all
More informationCOMP 558 lecture 18 Nov. 15, 2010
Least squares We have seen several least squares problems thus far, and we will see more in the upcoming lectures. For this reason it is good to have a more general picture of these problems and how to
More informationCommon Core Algebra 2 Review Session 1
Common Core Algebra 2 Review Session 1 NAME Date 1. Which of the following is algebraically equivalent to the sum of 4x 2 8x + 7 and 3x 2 2x 5? (1) 7x 2 10x + 2 (2) 7x 2 6x 12 (3) 7x 4 10x 2 + 2 (4) 12x
More informationComputational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf-Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2018/19 Part 4: Iterative Methods PD
More information