Tensor Product Approximation

Size: px

Start display at page:

Download "Tensor Product Approximation"

Angel Kerry Harper
5 years ago
Views:

1 Tensor Product Approximation R. Schneider (TUB Matheon) Mariapfarr, 2014

2 Acknowledgment DFG Priority program SPP 1324 Extraction of essential information from complex data Co-workers: T. Rohwedder (HUB), A. Uschmajev (EPFL Laussanne) W. Hackbusch, B. Khoromskij, M. Espig (MPI Leipzig), I. Oseledets (Moscow) C. Lubich (Tübingen), O. Legeza (Wigner I - Budapest), Vandereycken (Princeton), M. Bachmayr, L. Grasedyck (RWTH Aachen),... J. Eisert (FU Berlin - Physics), F. Verstraete (U Wien), Z. Stojanac, H. Rauhhut Students: M. Pfeffer, S. Holtz...

3 I. High-dimensional problems

4 PDE s in R d, (d >> 3) Equations describing complex systems with multi-variate solution spaces, e.g. stationary/instationary Schrödinger type equations i t Ψ(t, x) = ( V ) Ψ(t, x), }{{} H HΨ(x) = EΨ(x) describing quantum-mechanical many particle systems stochastic SDEs and the Fokker-Planck equation, p(t, x) d ( = fi (t, x)p(t, x) ) + 1 d 2 ( Bi,j (t, x)p(t, x) ) t x i 2 x i x j i=1 i,j=1 describing mechanical systems in stochastic environment, x = (x 1,..., x d ), where usually, d >> 3! parametric PDEs (stochastic PDEs) e.g. x a(x, y 1,..., y d ) x u(x, y 1,..., y d ) = f (x) x Ω, y R d, + b.c. on Ω.

5 Quantum physics - Fermions For a (discs.) Hamilton operator H and given h q p, g r,s H = d p,q=1 h q pa T p a q + d p,q,r,s=1 g p,q r,s a T r a T s a p a q. the stationary (discrete) Schrödinger equation is HU = EU, U ( 0 1 where A := 0 0 and discrete annihilation operators and creation operators ) d C 2 C (2d ). j=1 (, A T 0 0 = 1 0 ) ( 1 0 S := 0 1 a p a p := S... S A (p) I... I p,q R, ), a p a T p := S... S AT (p) I... I

6 Curse of dimensions For simplicity of presentation: discrete tensor product spaces V := d i=1 V i, e.g.: V = d i=1 Rn i = R (Πd i=1 n i ) we consider tensor as multi-index arrays (I i = 1,..., n i ) U = ( U x1,x 2,...,x d )x i =1,...,n i, i=1,...,d V, or equivalently functions of discrete variables (K = R or C) U : d i=1 I i K, x = (x 1,..., x d ) U = U(x 1,..., x d ) V, d = 1: n-tuples (U x ) n x=1, or x U(x), or d = 2: matrices ( U x,y ) or (x, y) U(x, y). dim V = O(n d ) Curse of dimensionality! e.g. n = 100, d = basis functions, coefficient vectors of Bytes = 800 Exabytes n = 2, d = 500: then >> the estimated number of atoms in the universe!

7 Curse of dimensions State of art: FEM : d = 3 time-dep. FEM and optimal control ω [0, T ], d = 4 2d-BEM: Integral Op. d = 4 radiative transport d = 5 The HD problems are considered as not tractable. Common arguments Since the numerical solutions of this problem is impossible, we do... density functional theory- DFT (nobel price 98) or Monte Carlo (rules the stock market) need quantum computers technological revolution do not consider these problems at al (Beatles: Let it Be!) Circumventing the curse of dimensionality is one of the most challenging problems in computational sciences

8 I. Classical and novel tensor formats B {1,2,3,4,5} B {1,2,3} B {4,5} B {1,2} U 3 U 4 U 5 U 1 U 2 U {1,2} U {1,2,3} (Format representation closed under linear algebra manipulations)

9 Setting - Tensors of order d For simplicity: Tensors - multi-indexed arrays (multi-variate functions): u : x = (x 1,..., x d ) u(x) = u(x 1,..., x d ), x i = 1,..., n i, with indices (or variables) x i, i = 1,..., d, x I = d i=1 I i = d i=1 {1,..., n i} other notations (x 1,..., x d ) u[x] = u[x 1,..., x d ] or u = ( u x1,...,x d ) V Examples: 1. d = 1: n-tuple, vectors, signals : x u(x), u = ( u x )x I V = Rn 2. d = 2: matrices, digital images (x, y) u(x, y), u = ( u x,y )x I 1,y I 2 V = R n 1n 2 3. d = 4: two-electron integrals V p,q r,s...

10 Setting - Tensors of order d Tensor product of vectors: u v(x, y) := u(x)v(y) = ( uv T ) x,y Tensor product of vector spaces V 1 V 2 := span{u v : u V 1, v V 2 } Goal: Generic perspective on methods for high-dimensional problems, i.e. problems posed on tensor spaces, Notation: V := d i=1 V i, e.g.: V = d i=1 Rn = R (nd ) Main problem: x = (x 1,..., x d ) U = U(x 1,..., x d ) V = H dim V = O(n d ) Curse of dimensionality! e.g. n = 100, d = basis functions, coefficient vectors of Bytes = 800 Exabytes

11 Setting - Tensors of order d Goal: Generic perspective on methods for high-dimensional problems, i.e. problems posed on tensor spaces, V := d i=1 V i, e.g.: V = d i=1 Rn = R (nd ) Main problem: dim V = O(n d ) Curse of dimensionality! e.g. n = 100, d = basis functions, coefficient vectors of Bytes = 800 Exabytes Approach: Some higher order tensors can be constructed (data-) sparsely from lower order quantities. As for matrices, incomplete SVD: r ( A(x 1, x 2 ) σ k uk (x 1 ) v k (x 2 ) ) k=1

12 Setting - Tensors of order d Goal: Generic perspective on methods for high-dimensional problems, i.e. problems posed on tensor spaces, V := d i=1 V i, e.g.: V = d i=1 Rn = R (nd ) Main problem: dim V = O(n d ) Curse of dimensionality! e.g. n = 100, d = basis functions, coefficient vectors of Bytes = 800 Exabytes Approach: Some higher order tensors can be constructed (data-) sparsely from lower order quantities. As for matrices, incomplete SVD: r ( A(x 1, x 2 ) σ k uk (x 1 ) v k (x 2 ) ) k=1

13 Setting - Tensors of order d Goal: Generic perspective on methods for high-dimensional problems, i.e. problems posed on tensor spaces, V := d i=1 V i, e.g.: V = d i=1 Rn = R (nd ) Main problem: dim V = O(n d ) Curse of dimensionality! e.g. n = 100, d = basis functions, coefficient vectors of Bytes = 800 Exabytes Approach: Some higher order tensors can be constructed (data-) sparsely from lower order quantities. Canonical decomposition for order-d-tensors: r ( U(x 1,..., x d ) d i=1 u i,k (x i ) ). k=1

14 d = 2: Sparsity versus rank sparsity Example: Given (x, y) A(x, y) be an n m matrix A. If A is rank one r = 1 A(x, y) = u(x)v(y), A = u v = u T v requires n + m (instead nm) coefficients u(x), v(y), A sparse matrix is the linear combination of δ i,k = e i e k. A = r α µ δ iµ,k µ = µ=1 r α µ e iµ e kµ µ=1 r sparse matrix are also of rank r. But example u T = (1,..., 1), v := 2u then u v = is not sparse!

15 d = 2: Singular value decomposition (SVD) (other names: Schmidt decomposition, proper orthogonal dec. (POD), principle component analysis (PCA), Karhunen-Loeve transformation) E. Candes: The SVD is the swiss army knife for computational scientists The case d = 2 is an exceptional, since V W L(V, W), the space of linear operators, and spectral theory could be applied Theorem (Eckhardt-Young (36), E. Schmidt (07)) n ( U(x 1, x 2 ) = σ k uk (x 1 ) v k (x 2 ) ) n = (ũk (x 1 )s(k, j)ṽ j (x 2 ) ) k=1 k,j=1 where u k, u l = δ k,l, v k, v l = δ k,l, σ 1 σ 2 0. The best rank r approximation of U in V,. 2 is given by setting σ r+1 = = σ n = 0. Observation: U T U is symmetric and positively semi-definite U T Uv i = σ 2 i v i Since U T U = σ 2 i v i v i, if U = 1, U T U (formally) is a density matrix.

16 d = 2: Singular value decomposition (SVD) tensor d = 2: U : x, y U(x, y) various forms U = = r r σ k u k v k = s(k, l)u k v k k=1 r u k ṽ k = k=1 k=1 k,l=1 r ũ k v k where u k, u l = δ k,l, v k, v l = δ k,l and S = ( s(k, l) ) r k,l=1 Rr r, det S 0. Subspaces: U 1 V 1,U 2 V 2, U 1 := span{u k : 1 k r} = span{ũ k : 1 k r} = Im U U 2 := span{v k : 1 k r} = span{ṽ k : 1 k r} = Im U T

17 Singular value decomposition (SVD) and canonical format There are other less optimal low rank approximations, e.g. LU decomposition QR decomposition etc.. Example: Two electron-integrals Cholesky decomposition vs. density fitting V p,q r,s = k L k p,ql k r,s = i,j T i p,qm i,j T j r,s T i p,q = ψ p (x 1 )ψ q (x 1 )θ i (x 1 )dx 1, M i,j = θ i (x 1 )θ j (x 2 ) r 1 r 2 dx 1 Canonical decomposition: example MP2 calculation F (a, b, i, j) 1 := 1 ɛ a + ɛ b ɛ i ɛ j = 0 k e s(ɛa+ɛ b ɛ i ɛ j ) ds α k e s k (ɛ a+ɛ b ɛ i ɛ j ) = k α k e s k ɛ a e s k ɛ b e s k ( ɛ i ) e s k ( ɛ j )

18 Subspace approximation

19 Subspace approximation Tucker format (Q: MCTDH(F)) - robust Segre -variety (closed) But complexity O(r d + ndr) Is there a robust tensor format, but polynomial in d? Univariate bases x i ( U i (k i, x i ) ) r i ( Graßmann man.) k i =1 U(x 1,.., x d ) = r 1 k 1 =1... r d k d =1 B(k 1,.., k d ) d U i (k i, x i ) i=1 {1,2,3,4,5}

20 Subspace approximation Tucker format (Q: MCTDH(F)) - robust Segre -variety (closed) But complexity O(r d + ndr) Is there a robust tensor format, but polynomial in d? Hierarchical Tucker format (HT; Hackbusch/Kühn, Grasedyck, Meyer et al., Thoss & Wang, Tree-tensor networks) Tensor Train (TT-)format Matrix product states (MPS) r 1 r d 1 d U(x) =... B i (k i 1, x i, k i ) = B 1 (x 1 ) B d (x d ) {1,2,3,4,5} k 1 =1 k d 1 =1 i=1 {1} {2,3,4,5} {2} {3,4,5} U 1 U 2 U 3 U 4 U 5 r 1 r 2 r 3 r 4 n 1 n 2 n 3 n 4 n 5 {3} {4,5} {4} {5}

21 Hierarchical tensor (HT) format Canonical decomposition Subspace approach (Hackbusch/Kühn, 2009) (Example: d = 5, U i R n k i, B t R k t k t1 k t2 )

22 Hierarchical tensor (HT) format Canonical decomposition not closed, no embedded manifold! Subspace approach (Hackbusch/Kühn, 2009) (Example: d = 5, U i R n k i, B t R k t k t1 k t2 )

23 Hierarchical tensor (HT) format Canonical decomposition not closed, no embedded manifold! Subspace approach (Hackbusch/Kühn, 2009) (Example: d = 5, U i R n k i, B t R k t k t1 k t2 )

24 Hierarchical tensor (HT) format Canonical decomposition not closed, no embedded manifold! Subspace approach (Hackbusch/Kühn, 2009) B {1,2,3,4,5} B {1,2,3} B {4,5} B {1,2} U 3 U 4 U 5 U 1 U 2 (Example: d = 5, U i R n k i, B t R k t k t1 k t2 )

25 Hierarchical tensor (HT) format Canonical decomposition not closed, no embedded manifold! Subspace approach (Hackbusch/Kühn, 2009) B {1,2,3,4,5} B {1,2,3} B {4,5} B {1,2} U 3 U 4 U 5 U 1 U 2 U {1,2} (Example: d = 5, U i R n k i, B t R k t k t1 k t2 )

26 Hierarchical tensor (HT) format Canonical decomposition not closed, no embedded manifold! Subspace approach (Hackbusch/Kühn, 2009) B {1,2,3,4,5} B {1,2,3} B {4,5} B {1,2} U 3 U 4 U 5 U 1 U 2 U {1,2} U {1,2,3} (Example: d = 5, U i R n k i, B t R k t k t1 k t2 )

27 Hierarchical tensor (HT) format Canonical decomposition not closed, no embedded manifold! Subspace approach (Hackbusch/Kühn, 2009) B {1,2,3,4,5} B {1,2,3} B {4,5} B {1,2} U 3 U 4 U 5 U 1 U 2 U {1,2} U {1,2,3} (Example: d = 5, U i R n k i, B t R k t k t1 k t2 )

28 Hierarchical Tucker as subspace approximation 1) D = {1,..., d}, tensor space V = j D V j 2) T D dimension partition tree, vertices α T D are subsets α T D, root: α = D 3) V α = j α V j for α T D 4) U α V α subspaces of dimension r α with the characteristic nesting U α U α1 U α2 (α 1, α 2 sons of α) 5) v U D (w.l.o.g. U D = span (v)).

29 Subspace approximation: formulation with bases U α = span {b (α) i : 1 i r α } b (α) l = r α1 r α2 c (α,l) ij b (α 1) i b (α 2) j (α 1, α 2 sons of α T D ). i=1 j=1 Coefficients c (α,l) ij form the matrices C (α,l). Final representation of v is v = r D c (D) i b (D) i, (usually with r D = 1).

30 Recursive definition by bases representations U α = span{b (α) i : 1 i r α } b (α) l = r α1 r α2 c (α,l) i,j b (α 1) i b (α 2) j (α 1, α 2 sons of α T D ). i=1 j=1 The tensor is recursively defined by the transfer or component tensors (l, i, j) c (α,l) i,j in R k t k 1 k 2. Data complexity O(dr 3 + dnr)! (r := max{r α }) Linear algebra operations like U + V or scalar product U, V (and application of linear operators) require at most O(dr 4 + dnr 2 ) operations.

31 TT - Tensors - Matrix product representation Noteable special case of HT: TT format (Oseledets & Tyrtyshnikov, 2009) matrix product states (MPS) in quantum physics Affleck, Kennedy, Lieb &Tagasaki (87)., Römmer & Ostlund (94), Vidal (03), HT tree tensor network states in quantum physics (Cirac, Verstraete, Eisert... ) TT tensor U can be written as matrix product form = r 1 k 1 =1.. r d 1 k d 1 =1 U(x) = U 1 (x 1 ) U i (x i ) U d (x d ) U 1 (x 1, k 1 )U 2 (k 1, x 1, k 2 )...U d 1 (k d 2 x d 1, k d 1 )U d (k d 1, x d, k d ) with matrices or component functions U i (x i ) = ( u k i k i 1 (x i ) ) R r i 1 r i, r 0 = r d := 1. Redundancy: U(x) = U 1 (x 1 )GG 1 U 2 (x 2 ) U i (x i ) U d (x d ).

32 Example Any canonical representation with r terms r U 1 (x 1, k) U d (x d, k) k=1 is also TT with ranks r i r, i = 1,..., d 1. But conversely canonical r term representation is bounded by r 1 r d 1 = O(r d 1 ) Hierarchical ranks could be much smaller than canonical rank. Example x i [ 1, 1], i = 1,..., d, i.e r = d, but U(x 1,..., x d ) = d x d = x 1 I + I x 2 I, i=1 U(x 1,..., x d ) = (1, x 1 ) ( 1 x2 0 1 ) ( 1 xd ) ( xd 1 ) here r 1 =... = r d 1 = 2.

33 TT - Tensors - Matrix product representation Noteable special case of HT: TT format (Oseledets & Tyrtyshnikov, 2009) matrix product states (MPS) in quantum physics Affleck, Kennedy, Lieb &Tagasaki (87)., Römmer & Ostlund (94), Vidal (03), HT tree tensor network states in quantum physics (Cirac, Verstraete, Eisert... ) TT tensor U can be written as matrix product form = r 1 k 1 =1... U(x 1,..., x d ) = U 1 (x 1 ) U i (x i ) U d (x d ) r d 1 k d 1 =1 U 1 (x 1, k 1 )U 2 (k 1, x 2, k 2 )... U d 1 (k d 2 x d 1, k d 1 )U d (k d 1, x d ) with matrices or component functions U i (x i ) = ( U ki 1,k i (x i ) ) R r i 1 r i, r 0 = r d := 1. Redundancy: U(x) = U 1 (x 1 )GG 1 U 2 (x 2 ) U i (x i ) U d (x d ).

34 Matrix product states introduced in quantum lattice systems: Affleck, Kennedy, Lieb &Tagasaki (87) (FCS), S. White (DMRG, 91), Römmer & Ostlund (94), Vidal (03), Schöllwock, Cirac, Verstraete,... TT format (Oseledets & Tyrtyshnikov, 2009), HT (Hackbusch & Kühn, 2009) The tensor x U(x) can be written in matrix product form = r 1 k 1 =1... U(x 1,..., x d ) = U 1 (x 1 ) U i (x i ) U d (x d ) r d 1 k d 1 =1 U 1 (x 1, k 1 )U 2 (k 1, x 2, k 2 )... U d 1 (k d 2 x d 1, k d 1 )U d (k d 1, x d ) with matrices or component functions U i (x i ) = ( u ki 1 ;k i (x i ) ) K r i 1 r i, r 0 = r d := 1. Complexity: O ( ndr 2), r = max{r i : i = 1,..., d 1}, Redundancy: U(x) = U 1 (x 1 )GG 1 U 2 (x 2 ) U i (x i ) U d (x d ).

35 Matrix product states introduced in quantum lattice systems: Affleck, Kennedy, Lieb &Tagasaki (87) (FCS), S. White (DMRG, 91), Römmer & Ostlund (94), Vidal (03), Schöllwock, Cirac, Verstraete,... TT format (Oseledets & Tyrtyshnikov, 2009), HT (Hackbusch & Kühn, 2009) The tensor x U(x) can be written in matrix product form = r 1 k 1 =1... U(x 1,..., x d ) = U 1 (x 1 ) U i (x i ) U d (x d ) r d 1 k d 1 =1 U 1 (x 1, k 1 )U 2 (k 1, x 2, k 2 )... U d 1 (k d 2 x d 1, k d 1 )U d (k d 1, x d ) with matrices or component functions U i (x i ) = ( u ki 1 ;k i (x i ) ) K r i 1 r i, r 0 = r d := 1. Complexity: O ( ndr 2), r = max{r i : i = 1,..., d 1}, Redundancy: U(x) = U 1 (x 1 )GG 1 U 2 (x 2 ) U i (x i ) U d (x d ).

36 Matrix product states introduced in quantum lattice systems: Affleck, Kennedy, Lieb &Tagasaki (87) (FCS), S. White (DMRG, 91), Römmer & Ostlund (94), Vidal (03), Schöllwock, Cirac, Verstraete,... TT format (Oseledets & Tyrtyshnikov, 2009), HT (Hackbusch & Kühn, 2009) The tensor x U(x) can be written in matrix product form = r 1 k 1 =1... U(x 1,..., x d ) = U 1 (x 1 ) U i (x i ) U d (x d ) r d 1 k d 1 =1 U 1 (x 1, k 1 )U 2 (k 1, x 2, k 2 )... U d 1 (k d 2 x d 1, k d 1 )U d (k d 1, x d ) with matrices or component functions U i (x i ) = ( u ki 1 ;k i (x i ) ) K r i 1 r i, r 0 = r d := 1. Complexity: O ( ndr 2), r = max{r i : i = 1,..., d 1}, Redundancy: U(x) = U 1 (x 1 )GG 1 U 2 (x 2 ) U i (x i ) U d (x d ).

37 Successive SVD for e.g. TT tensors - Vidal (2003), Oseledets (2009), Grasedyck (2009) 1. Matricisation - unfolding F (x 1,..., x d ) F x 2,...,x d x 1. low rank approximation up to an error ɛ 1, e.g. by SVD. F x 2,...,x d x 1 = r 1 k 1 =0 u k 1 x 1 v x 2,...,x d k 1, U 1 (x 1, k 1 ) := u k 1 x 1, k 1 = 1,..., r Decompose V (k 1, x 2,..., x d ) via matricisation up to an accuracy ɛ 2, V x 3,...,x d k 1,x 2 = r 2 k 2 u k2 k 1,x 2 v x 3,...,x d k 2 U 2 (k 1, x 2, k 2 ) := u k 2 k 1,x repeat with V (k 2, x 3,... x d ) until one ends with [ vkd 1,x d ] Ud (k d 1, x d ).

38 Example The function U(x 1,..., x d ) = x x d H in canonical representation U(x 1,..., x d ) = x x x d In Tucker format, let U i,1 = 1, U i,2 = x i be the basis of U i (non-orthogonal), r i = 2. U intt or MPS representation U(x) = x x d = ( x 1 1 ) ( ) x 2 1 (non-orthogonal), r 1 =... = r d 1 = 2 ( ) ( ) x d 1 1 x d

39 Tensor networks Diagram - (Graph) of a tensor network: vector matrix tensor x x x 1 2 x 1 u [x] u [x 1, x 2 ] = x 1 x 2 k u 1 [x 1, k] x 1 k 2 = u [x 1,..., x 8 ] u 2 u 4 [x 4, k 3, k 4 ] = k i TT format HT format Diagramm: x 1 x 8 Each node corresponds to a factor u ν, ν = 1,..., d, each line to a variable x ν or a contraction variable k µ. Since k ν is an edge connecting 2 tensors u µ1 and u µ2, one has to sum over connecting lines (edges).

40 Successive SVD for e.g. TT tensors The above algorithm can be extended to HT tensors! Quasi best approximation Error analysis F(x) U(x 1 )... U d (x d ) 2 ( d 1 i=1 ɛ 2 ) 1/2 i d 1 inf U T r F(x) U(x) 2 Exact recovery: if U T r, then is will be recovered exactly! (up ot rounding errors)

41 Comparison: Ranks and Complexity Grouping indices I = {x i1,..., x il } I d = {x 1,..., x d } into row or column index of A I matricisation or unfolding of (x 1,..., x d ) U(x 1,..., x d ) A I d \I I, Ranks are multi-integer r = (r 1,..., r d ) Canonical format: not well defined Tucker : r i = rank A I d \{x i } x i TT format r i = rank A x i+1,...,x d x 1,...,x i, entanglement E i := log r i, HT format r I = rank A I d \I I, Complexity: r := max{r i }, (r Tucker r HT /TT r CP ) canonical format (CP): O(ndr C ) Tucker: O(ndr + r d Tucker ) TT format T r : O(ndr 2 TT ) HT format M r, O(ndr HT + dr 3 HT ) Although the HT (TT) is suboptimal in complexity, by now, there does not exist an alternative mathematical approach for tackling highly entangled problems.

42 Comparison: Ranks and Complexity Grouping indices I = {x i1,..., x il } I d = {x 1,..., x d } into row or column index of A I matricisation or unfolding of (x 1,..., x d ) U(x 1,..., x d ) A I d \I I, Ranks are multi-integer r = (r 1,..., r d ) Canonical format: not well defined Tucker : r i = rank A I d \{x i } x i TT format r i = rank A x i+1,...,x d x 1,...,x i, entanglement E i := log r i, HT format r I = rank A I d \I I, Complexity: r := max{r i }, (r Tucker r HT /TT r CP ) canonical format (CP): O(ndr C ) Tucker: O(ndr + r d Tucker ) TT format T r : O(ndr 2 TT ) HT format M r, O(ndr HT + dr 3 HT ) Although the HT (TT) is suboptimal in complexity, by now, there does not exist an alternative mathematical approach for tackling highly entangled problems.

43 Comparison: Ranks and Complexity Grouping indices I = {x i1,..., x il } I d = {x 1,..., x d } into row or column index of A I matricisation or unfolding of (x 1,..., x d ) U(x 1,..., x d ) A I d \I I, Ranks are multi-integer r = (r 1,..., r d ) Canonical format: not well defined Tucker : r i = rank A I d \{x i } x i TT format r i = rank A x i+1,...,x d x 1,...,x i, entanglement E i := log r i, HT format r I = rank A I d \I I, Complexity: r := max{r i }, (r Tucker r HT /TT r CP ) canonical format (CP): O(ndr C ) Tucker: O(ndr + r d Tucker ) TT format T r : O(ndr 2 TT ) HT format M r, O(ndr HT + dr 3 HT ) Although the HT (TT) is suboptimal in complexity, by now, there does not exist an alternative mathematical approach for tackling highly entangled problems.

44 Closedness A tree T D is characterized by the property, if one remove one edge yields two separate trees. Observation: Let A i with ranks ranka i r. If lim i A i A 2 = 0 then ranka r: closedness of Tucker and HT tensor in T r (Falco & Hackbsuch). T r = s r Ts H is closed! due to Hackbusch & Falco Landsberg & Ye: If a tensor network has not a tree structure, the set of all tensor of this form need not to be closed!

45 Summary For Tucker and HT redundancy can be removed (see next talk) Table: Comparison canonical Tucker HT complexity O(ndr) O(r d + ndr) O(ndr + dr 3 ) TT- O(ndr 2 ) ++ + rank no defined defined r c r T r HT, r T r HT closedness no yes yes essential redundancy yes no no recovery?? yes yes quasi best approx. no yes yes best approx. no exist exist but NP hard but NP hard

46 TT approximations of Friedman data sets f 2 (x 1, x 2, x 3, x 4 ) = (x (x 2x 3 1 x 2 x 4 ) 2, f 3 (x 1, x 2, x 3, x 4 ) = tan 1 ( x 2 x 3 (x 2 x 4 ) 1 x 1 ) on 4 D grid, n points per dim. n 4 tensor, n {3,..., 50}. full to tt (Oseledets, successive SVDs) and MALS (with A = I) (Holtz & Rohwedder & S.)

47 Solution of U = b using MALS/DMRG Dimension d = 4,..., 128 varying Gridsize n = 10 Right-hand-side b of rank 1 Solution U has rank 13

48 Example: Eigenvalue problem in QTT in courtesy of B. Khoromskij, I. Oseledets, QTT: Toward bridging high-dimensional quantum molecular dynamics and DMRG methods, HΨ = ( 1 + V )Ψ = EΨ 2 with potential energy surface given by Henon-Heiles potential V (q 1,..., q f ) = 1 f qk λ f 1 ( qk 2 q k+1 1 ) 3 q3 k. k=1 k=1 Dimensions f = 4,..., 256; 1-D grid size n = 128 = 2 7 = 2 d ; QTT-tensors 7f i=1 R2 = R }{{} 7f =1792.

Tensor networks, TT (Matrix Product States) and Hierarchical Tucker decomposition

Tensor networks, TT (Matrix Product States) and Hierarchical Tucker decomposition R. Schneider (TUB Matheon) John von Neumann Lecture TU Munich, 2012 Setting - Tensors V ν := R n, H d = H := d ν=1 V ν