Computations in Quantum Tensor Networks

Transcription

1 Comutations in Quantum Tensor Networks T Huckle a,, K Waldherr a, T Schulte-Herbrüggen b a Technische Universität München, Boltzmannstr 3, Garching, Germany b Technische Universität München, Lichtenbergstr 4, Garching, Germany Abstract The comutation of the ground state ie the eigenvector related to the smallest eigenvalue is an imortant task in the simulation of quantum many-body systems As the dimension of the underlying vector sace grows exonentially in the number of articles, one has to consider aroriate subsets romising both convenient aroximation roerties and efficient comutations The variational ansatz for this numerical aroach leads to the minimization of the Rayleigh quotient The Alternating Least Squares technique is then alied to break down the eigenvector comutation to roblems of aroriate size, which can be solved by classical methods Efficient comutations require fast comutation of the matrix-vector roduct and of the inner roduct of two decomosed vectors To this end, both aroriate reresentations of vectors and efficient contraction schemes are needed Here aroaches from many-body quantum hysics for one-dimensional and two-dimensional systems Matrix Product States and Projected Entangled Pair States are treated mathematically in terms of tensors We give the definition of these concets, bring some results concerning uniqueness and numerical stability and show how comutations can be executed efficiently within these concets Based on this overview we resent some modifications and generalizations of these concets and show that they still allow efficient comutations such as alicable contraction schemes In this context we consider the minimization of the Rayleigh quotient in terms of the arafac CP formalism, where we also allow different tensor artitions This aroach makes use of efficient contraction schemes for the calculation of inner roducts in a way that can easily be extended to the ms format but also to higher dimensional roblems Keywords: Quantum many-body systems, Density Matrix Renormalization Grou dmrg, Matrix Product States ms, Tensor Trains, Projected Entangled-Pair States, Canonical Decomosition candecom or arafac Corresonding author addresses: huckle@intumde T Huckle, waldherr@intumde K Waldherr, tosh@tumde T Schulte-Herbrüggen 1

2 2 1 Introduction Comutations with tensors are getting increasingly imortant in high dimensional roblems In articular in quantum many-body hysics, a tyical roblem amounts to finding an accurate aroximation to the smallest eigenvalue ie, the ground-state energy of a hermitian matrix reresenting the Hamiltonian that is larger than one can store even on a owerful comuter To this end, in quantum hysics techniques like Matrix Product States ms or Projected Entangled Pair States es have been develoed for reresenting vectors, viz eigenstates of quantum systems efficiently In mathematics, besides the Tucker decomosition and the canonical decomosition, concets like Tensor Trains tt were introduced The examles of ms or es in hysics and tt in mathematics exress a common interest in owerful numerical methods secifically designed for coing with high-dimensional tensor networks Unifying variational aroaches to ground-state calculations [1] in a common framework of tensor aroximations will be highly useful, in articular in view of otimizing numerical algorithms [2, 3, 4, 5] Here it is the goal to cast some of the recent develoments in mathematical hysics into such a common frame exressed in the terminology of multilinear algebra Moreover we resent numerical results on the Ising-tye Hamiltonian underscoring the wealth and the otential of such an aroach In this aer, we address one-dimensional and two-dimensional methods in a unified frame related to ms and es We will introduce some generalization of ms and the canonical decomosition Furthermore, we give a short descrition of tensor-decomosition methods for 2D roblems Scoe and Organization The aer is organized as follows: Sections 2, 3 and 4 contain an overview of already-known concets and describe them in the multilinear algebra language: in Section 2 we introduce the hysical background of the roblem setting and define the matrices involved, in Section 3 we resent reresentation schemes for states in hysically motivated 1D and 2D arrays and we show how comutations can be erformed efficiently and Section 4 finally fixes some basics and notations for tensors and tensor-decomosition schemes In Section 5 we resent new ideas of how to generalize these basic concets, how to execute calculations efficiently and how to aly them to the groundstate aroximation roblem First numerical results will show the benefit of these newly develoed concets 2 Physical Model Systems Consider vectors x in a comlex Hilbert sace H reresenting states of ure quantum systems The differential equation ẋ = ihx Schrödinger s equation of motion then governs quantum dynamics neglecting relaxation with the Hamiltonian H being the generator of unitary time evolution The Hamiltonian

3 3 catures the energies of the constituent subsystems eg sins as well as the interaction energies between couled subsystems For instance, a linear chain of five sins couled by nearest neighbor interactions can be deicted as in Figure 1a In the case of oen boundary conditions a 1D system of 5 sins with nearest-neighbor interaction and eriodic boundary conditions H = P z P z I I I + I P z P z I I + I I P z P z I + I I I P z P z + P z I I I P z b Reresentation of the Hamiltonian related to the hysical system illustrated by Figure 1a Figure 1: Examle of a linear hysical system a and the related Hamiltonian b OBC there is no couling interaction between article 1 and 5, while for eriodic boundary conditions PBC there is a non vanishing couling interaction between 1 and 5 21 Hamiltonian Reresentations For sin 1 2 articles such as electrons or rotons, the sin angular momentum oerator describing their internal degree of freedom ie sin-u and sin-down is usually exressed in terms of the Pauli matrices i 1 0 P x =, P 1 0 y = and P i 0 z = Being traceless and Hermitian, {P x, P y, P z } forms a basis of the Lie algebra su2, while by aending the identity matrix I one obtains a basis of the Lie algebra u2 Now, sin Hamiltonians are built by summing M terms, each of them reresenting a hysical interaction These terms are themselves tensor roducts of Pauli matrices or identities M M H = α k Q k 1 Q k 2 Q k H k, 2 }{{} k=1 =:H k = k=1

4 4 where Q k j can be P x, P y, P z or I In each summand H k most of the Q k j are I: local terms have just one nontrivial tensor factor, while air interactions have two of them Higher m-body interactions with m > 2 usually do not occur as hysical rimitives, but could be reresented likewise by m Pauli matrices in the tensor roduct reresenting the m-order interaction term 1 For instance, in the Ising ZZ model [7] for the 1D chain with sins and oen boundary conditions, the sin Hamiltonian takes the form 1 H = I k 1 P z k P z k+1 I k 1 k=1 + λ I k 1 P x k I k, k=1 3 where the index k denotes the osition in the sin chain and the real number λ describes the ratio of the strengths of the magnetic field and the air interactions For simlicity, we will henceforth dro the tensor owers of the identity and tacitly assume aroriate embedding Then a Hamiltonian for an oen-boundary 1D Heisenberg XY model [8, 9] reads 1 H = J x I P x k P x k+1 I + J y I P y k P y k+1 I k=1 + λ I P x k I k=1 4 with real constants J x, J y and λ Models with all coefficients being different are called anisotroic Being a sum 2 of Kronecker roducts of structured 2 2 matrices many Hamiltonians have secial roerties: they can be multilevel-circulant [10, 11] or skew-circulant, diagonal or ersymmetric [12], which can be exloited to derive roerties of the resective eigenvalues and eigenvectors 22 Comutation of Ground States: Physical Background A key to understand some of the motivating guidelines lies in the somewhat striking fact that quantum dynamical systems tyically evolve in a way that looks non-generic from a mathematical oint of view Yet the very structure of quantum dynamics aves the way to tailored arameterizations based on tensor comression that are efficient in the sense of scaling only olynomially in hysical system size Some of these motivating guidelines established in quantum hysics may be sketched as follows: Comosing a quantum system 1 For further details, a reader wishing to aroach quantum hysics from linear and multilinear algebra may refer to [6]

5 5 from its comonents takes a joint Hilbert sace that is the tensor roduct of the individual Hilbert saces Likewise a linear oerator on the joint Hilbert sace can be comosed by taking sums or weighted linear combinations of tensor factors like in Eqn 2 Clearly, in general a linear combination of tensor roducts does not take the form of a tensor roduct itself Thus a quantum state sace grows exonentially with the number of constituents in contrast to a classical configuration sace just growing linearly However, correlating quantum interactions tyically become smaller and smaller with increasing distance between subsystems articles : for instance, in Eqn 3 only nearest-neighbor interactions had to be taken into account On a general scale, this can be made recise in terms of area laws, where the correlations are quantified by a measure termed entanglement entroy of ground states [13, 14, 15], see also the recent review in [16] Remarkably, this entroy of the reduced state of a subregion is not extensive: it tyically grows with the boundary region area between the subregion and its comlement rather than with the volume of the subregion In one-dimensional systems, a rigorous area law has recently been roven for all systems with a ga between the smallest and the second smallest eigenvalue [17] Extending the results to twodimensional lattice systems, however, currently requires stronger assumtions on the eigenvalue distribution [18] Guided by these area laws, long-distance correlations may be neglected in the sense that eigenvectors ground states of hysical systems are well aroximated within truncated subsets, as has been quantitatively established, eg, for ms [19] Moreover ms-aroximations to ground states can rovably be calculated efficiently [20] Along similar lines, consecutive artitionings have been exloited in unitary and tensor networks addressing ground states and dynamics of large-scale quantum systems Related techniques for truncating the Hilbert sace to ertinent arameterized subsets not only include Matrix Product States ms [21, 22] of Density Matrix Renormalization Grous dmrg [23, 24], but also rojected entangled air states es [25, 26], weighted grah states wgs [27], Multi-scale Entanglement Renormalization Aroaches mera [28], string-bond states sbs [29] as well as combined methods [1, 30] To conclude, the evolution of hysical systems does not exloit the generic state sace with long-distance many-body interactions, but roceeds via welldefined subsaces of short-distance and mainly airwise interactions that can be arameterized by data-sarse formats which allow for tensor-contraction schemes 23 Comutation of Ground States: Numerical Asects The ground state energy of a hysical system modeled by the Hamiltonian H corresonds to the smallest eigenvalue of H, which is the minimum of the Rayleigh quotient [31] x H Hx min x H x H x 5 As long as, the number of articles, is not too large, any standard numerical method for comuting eigenvalues and eigenvectors can be used

6 6 But with increasing, the size of the Hamiltonians grows like 2 Thus for > 50 neither matrices nor even vectors of this size can be stored So, similar to the descrition of the Hamiltonian 2 we need a sarse aroximate reresentation of eigenvectors Assume that we have already chosen an aroriate subset U H, the goal is to find aroximations for the eigenvector in this set Hence we consider the minimization of the Rayleigh quotient 5 only on the subset U: x H Hx min x U x H x 6 An aroriate subset of vectors should allow for easy comutation of Hx and of inner roducts y H x Therefore we consider vector reresentations with a less number of coefficients where subsets of the indices can be groued in artitions corresonding to the binary tensor structure of the Hamiltonian 2 Please note that, in general, the chosen subsets do not form linear subsaces 24 Alternating Least Squares An imortant tool for minimizing the Rayleigh quotient for an aroriate subset U is the Alternating Least Squares aroach als, see [32, 33] Here, all subsets of the artitioning u to one are assumed to be fixed, and then the minimization is reduced to the remaining subset As introductory examle let us look at a set of vectors defined by x = x 1 x 2 x = x 1;i1 x 2;i2 x ;i i1,,i = x i i=0,,2 1 7 with vectors x i of length 2, and i = i 1,, i 2 the binary reresentation of i with i j {0, 1} Hence we envisage the vector x as a -tensor So in our examle 7 we assume all subsets fixed u to x r, and then the minimization is simlified to min x r x H Hx x H x M H x 1 x α k Q k 1 Q k x 1 x = min k=1 x r x 1 x H x 1 x M α k x H 1 Q k 1 x1 xrh Q k r x r x H Q k x k=1 = min x r x 1H x 1 x rh x r x H x = min x r H M x r α k β k Q k r k=1 x rh γi x r x r = min x r x H r R rx r, x rh x r a standard eigenvalue roblem in the effective Hamiltonian R r = M α k β k k=1 γ Q k r More generally, if we work with more comlex reresentations such as Matrix Product States, the minimization of the Rayleigh quotient 6 will lead to min x r x H Hx x H x = min x r x r H R r x r x rh N r x r 8

7 7 with an effective Hamiltonian related to the generalized eigenvalue roblem in R r and N r So far, Eq 8 describes the general situation, the articular secification of the matrices R r and N r will be given when we consider different reresentation schemes for the eigenvector Then, x r is set to the eigenvector with smallest eigenvalue We can reeat this rocedure ste by ste for all x j to get aroximations for the minimal eigenvalue of H The main costs are caused by matrix-vector roducts Hx and inner roducts y H x lus the solution of the relatively small generalized eigenvalue roblem 8 It is therefore imortant to have efficient schemes for the evaluation of inner roducts of two vectors out of the chosen subset U We emhasize that this aroach and adated modifications allows to overcome the curse of dimensionality as it is only olynomial in the maximum length of the small vectors x j, in the number of such vectors which can be uer bounded by and in the number of local terms M The ansatz 8 may cause roblems if the denominator matrix N r is singular In that case one would aly an orthogonal rojection on the nonsingular subsace of N r 3 Reresentations of States for Comuting Ground States Obviously, in general the above simlistic aroach based on a single tensor roduct cannot give good aroximations to the eigenvector Therefore, we have to find a clever combination of such terms The choice naturally deends on the dimension and the neighborhood relation of the hysical setting So first we consider the 1D linear setting, and in a following section we look at the 2D roblem 31 1D Systems: Aroximation by Matrix Product States The Matrix Product State ms formalism goes back to several sources: early ones are by Affleck, Kennedy, Lieb, and Tasaki [34, 9] including their revival by Fannes, Nachtergaele, and Werner [21, 22], while a more recent treatment is due to Vidal [35] The alication to the eigenvalue roblem was discussed by Delgado et al [36] A seemingly indeendent line of thought resorts to the fact that Density Matrix Renormalization Grou DMRG methods as develoed by Wilson and White [37, 38] have a natural interretation as otimization in the class of ms states, see, eg, Refs [23, 24, 39] As has been mentioned already, ground states of gaed 1D Hamiltonians are faithfully reresented by ms [19], where the ms-aroximation can be comuted efficiently [39, 20], the rationale being an area law [17] 311 Formalism and Comutations For ms small D j D j+1 -matrices are used for describing vectors in a comact form The advantage is due to the fact that D := max{d j } the bond dimension has to grow only olynomially in the number of articles in order to aroximate ground states with a given recision [19]

8 8 In ms, the vector comonents are given by x i = x i1,,i = tr A i1 1 A i2 2 A i = D 1 m 1=1 D m =1 a i1 1;m 1,m 2 a i2 2;m 2,m 3 a i ;m,m 1 9 The matrix roducts lead to indices and summation over m 2, m, and the trace introduces m 1 The uer hysical or given indices i j identify which of the two ossible matrices are used at each osition, and thereby they determine the vector comonents So, eg, the last comonent is described by x 2 1 = x 1,,1 = tra 1 1 A1 2 A 1 = a 1 1;m 1,m 2 a 1 2;m 2,m 3 a 1 ;m,m 1, m 1,,m where we always choose the matrix index i j = 1 The additional summation indices are called ancilla indices The above trace form is related to the eriodic case In the oen boundary case, there is no connection between first and last article, and therefore the index m 1 can be neglected In this case we have D 1 = D +1 = 1 and thus the matrices at the ends are of size 1 D 2 and D 1 resectively x i1,,i = A i1 1 A i2 2 A i 1 1 A i = D 2 m 2=1 m 1,,m i 1 D m =1 a i1 1;1,m 2 a i2 2;m 2,m 3 a i 1 1;m 1,m a i ;m,1 10 By introducing the unit vectors e i = e i1,,i = e i1 e i with unit vectors e ij of length 2, another useful reresentation of the ms vector is given by x = x i1,,i e i1,,i = tra i1 1 A i e i1,,i i 1,,i i 1,,i = a i1 1;m 1,m 2 a i ;m,m 1 e i1,,i i 1,,i m 1,,m = 11 a i1 1;m 1,m 2 e i1 a ;m i,m 1 e i = m 1,,m a 1;m1,m 2 a ;m,m 1 with length 2 vectors a r;mr,m r+1 where the two comonents are airwise entries in the matrices A 0 r and A 1 r at osition m r, m r+1 : a 0 r;m a r;mr,m r+1 := r,m r+1 a 1 r;m r,m r+1 i

9 9 Uniqueness of MPS and Normal Forms In this section, we want to summarize some known results concerning the uniqueness of MPS For further details, see, eg, [40] Obviously the reresentation of an ms vector is not unique So, for a vector with comonents x i = x i1,,i = tr we can relace the matrices by A ij j M 1 j A ij j A i1 1 A i2 2 A i 1 1 M j+1, A i1 1 A i1 1 M 2, A i A i 12 M 1 A i 13 with nonsingular matrices M j C Dj Dj, j = 2,, The absence of uniqueness also causes roblems in the solution of the effective generalized eigenvalue roblem, because the matrix N r in Eqn 8 might be ositive semidefinite, but singular To avoid this roblem, we switch from a given ms reresentation to a reresentation based on unitary matrices To this end, we combine the matrix air A ir r comute the SVD: A 0 r A 1 r = U r for i r = 0, 1 to a rectangular matrix and Λr U r 0 V 0 r = U r 1 Λ r V r 14 where the U r ir are the left art of U r Now we can relace at osition r in the ms vector the matrix air A ir r by the air U r ir and multily the remaining SVD factor Λ r V r from the left to the right neighbor air A ir+1 r+1 without changing the vector: tr A i1 1 A i2 2 A r ir A ir+1 r+1 A i 1 1 A i tr A i1 1 A i2 2 U r ir Λ r V r A ir+1 r+1 A i 1 1 A i So we can start from the left, normalizing first A i1 1, always moving the remaining SVD art to the right neighbor, until we reach A i During this rocedure the ms matrix airs A ij j, j = 1,, 1 are relaced by arts of unitary matrices U ij j, which fulfil the gauge condition U ijh j U ij j = I 15 i j In the case of oen boundary conditions, the right unnormalized matrices A i are column vectors and thus i A ih A i is only a scalar γ, which corresonds to the squared norm of the ms vector: x H x = A i1 1 A i A i1 1 A i i 1,,i

10 10 = i 1,,i = i A ih 15 = i A ih H A i1 1 A i A i1 1 A i i 2 A i = γ A i2h 2 i 1 A i1h 1 A i1 1 A i2 2 A i Thus, if x has norm one, the gauge condition 15 is also fulfilled for j = The same rocedure can be alied in order to move the remaining SVD art to the left neighbor To this end, we comute A 0 r A 1 r = V r Λ r 0 U r = V r Λ r U r 0 U r 1 16 Similarly we can move from right to left and relace the matrix airs A ij j =,, 2 by the unitaries U ij j take the form i j j, until we reach A i1 1 Now the gauge conditions U ij j U ijh j = I 17 Analogously, for oen boundary conditions the remaining left matrices A i1 1 are row vectors and so i A i1 1 A i1h 1 is simly a scalar, which is 1 for a norm 1 vector So far, for the normalization rocess only one matrix air A ij j was involved Similar to the two-site DMRG aroach [24], it is also ossible to consider the matrices related to two neighboring sites at once [41] To this end, we consider the two matrix airs A ij j C Dj Dj+1 and A ij+1 j+1 C Dj+1 Dj+2 The four matrix roducts A ij j notation and an SVD is carried out: A 0 j A 1 j A 0 j+1 A 1 j+1 = A ij+1 j+1 C Dj Dj+2 are now re-arranged in matrix A 0 j A 0 j+1 A 0 j A 1 j+1 A 1 j A 0 j+1 A 1 j A 1 j+1 U 0 j = U 1 j Σ j V 0 j+1 V 1 j+1 If we swee from left to right, we relace the matrices A ij j by arts of unitary matrices U ij j, shift the remaining art to the right neighbor, ie A ij+1 j+1 Σ j V ij+1 j+1 and roceed with the adjacent sites j + 1 and j + 2 Accordingly, if we swee from right to left, we relace the matrices A ij+1 j+1 by the unitaries V ij+1 j+1, shift the remaining art to site j, ie A ij j U ij j Σ j

11 11 and roceed with the index air j 1, j There exist even stronger normalization conditions allowing reresentations which are unique u to ermutations and degeneracies in the singular values, see, eg [42, 40] The roof of the existence of such normal forms is based on the SVD of secial matricizations of the vector to be reresented, see [42, 43] In the SVD-TT algorithm [44], the same technique is in use In [43] we resent normal forms for ms which allow for exressing certain symmetry relations The resented normalization techniques for ms vectors have various advantages They introduce normal forms for ms vectors which lead to better convergence roerties For the minimization of the Rayleigh quotient 8, the gauge conditions circumvent the roblem of bad conditioned N r matrices in the denominator and therefore arove numerical stability So far, the resented normalization technique only changes the reresentation of the vector but does not change the overall vector However, the SVD can also be used as a truncation technique This could be interesting if we want to kee the matrix dimensions limited by some D = D max As an examle we mention the PEPS format [45], where such an SVD-based reduction aears, comare Subsection 32 Sum of ms Vectors Unfortunately, the ms formalism does not define a linear subsace According to [41] the sum of two ms vectors x and y, which are both in PBC form, can be formulated as x + y = i 1,,i tr = i 1,,i tr = i 1,,i tr = i 1,,i tr A i 1 1 A i e i1,,i + [ [ A i 1 1 A i A i 1 1 B i 1 1 C i 1 1 C i i 1,,i tr B i 1 1 B i e i1,,i A i B i 1 1 B i e i1,,i ] B i e i1,,i ] e i1,,i In the oen boundary case, we can define the interior matrices C 2,, C 1 in the same way For reasons of consistency, the C matrices at the boundary sites 1 and have to be secified to be vectors, ie C i1 1 = A i1 1, B i1 1, C i = A i B i Hence, the sum of ms vectors is again an ms vector, but of larger size In order to kee the sizes of the ms matrices limited by a constant D max, one could aly an SVD-based truncation of the resulting ms formalism and consider only the D max dominant terms 18

12 12 Proerties of ms Every unit vector can be reresented by an ms with bond dimension D = 1: let j = j 1,, j be the binary form of j, then e j can be formulated as an ms with 1 1 ms matrices A ir r = δ ir,j r : e j = 1 i 1,,i =0 δi1,j 1 δ i,j ei1,,i In view of Eqn 18 this observation may be extended as follows: every sarse vector with at most D non zero entries can be written as an ms with bond dimension D Comutation of Matrix-Vector Products To solve the minimization of the Rayleigh quotient 8 in an als way as introduced in Subsection 24, we have to comute y k = H k x = α k Q k 1 Q k a 1;m1,m 2 a ;m,m 1 = m 1,,m = m 1,,m and derive a sum of ms vectors m 1,,m Q k 1 a 1;m 1,m 2 b 1,k;m1,m 2 b,k;m,m 1 k=1 Q k a ;m,m 1 M M y = Hx = H k x = y k 19 Therefore, we can comute all small vectors Q k j a j;mj,m j+1 in O2D 2 M oerations For reasons of efficiency there are also concets to exress the Hamiltonian in terms of a Matrix Product Oerator mo as defined in 22 This aroach enables short reresentations for mo ms roducts Comutation of Inner Products Furthermore, we have to comute inner roducts of two ms vectors bi 1 i 1;k 1,k 2 b ;k,k 1 i 1,,i m 1,,m k 1,,k k=1 a i1 1;m 1,m 2 a i ;m,m 1 20 To deal with these sums, it is essential to secify in which order the summations have to be conducted In the next figures we dislay such an efficient ordering for the eriodic case To this end, we introduce the following notation Each box in the following figures describes one factor, eg, a ir r;m r,m r+1, in these collections of sums Little lines legs describe the connection of two such boxes via a

13 13 common index Hence, the number of legs of a box is exactly the number of indices So in this case, most of the boxes have three legs Figure 2a shows the sum and the first terms, res boxes, with their connections This also reresents a Tensor Network Now, in a first ste we reorder the sums, and execute a contraction relative to index i 1 This is given by the artial sum bi1 1;k 1,k 2 a i1 1;m 1,m 2 = c k1,k 2;m 1,m 2 21 i 1 This eliminates two boxes, but leads to a new four leg tensor c k1,k 2;m 1,m 2 as shown in 2b and 3a Now we contract index k 2, as shown in Figure 3b, i1,k1,k2 i2,k2,k3 i3,k3,k4 i1,k1,k2 i2,k2,k3 i3,k3,k4 i1 i1,m1,m2 i2,m2,m3 i3,m3,m4 i1,m1,m2 i2,m2,m3 i3,m3,m4 a Comutation of the inner roduct 20 of two ms vectors b Contraction of the two ms vectors concerning index i 1 Figure 2: Contraction of the inner roduct of two ms vectors The dashed red line illustrates the index being contracted leading to 3c In the following ste we go from 3c to 3d by contracting i 2 and m 2, deriving at Figure 3d Now we are in the same situation as in Figure 3a, and we can roceed exactly in the same way, until all sums are executed, res all indices have been contracted The main costs deend on the size of the contracted index, eg 2 for contracting i r or D for contracting m r or k r, and on the size of the other indices that aear in the contracted boxes Hence, eg the contraction in Figure 3b costs D 2D 3 = 2D 4 oerations, and Figure 3c costs 2D D 4 = 2D 5 oerations The total costs for the inner roduct is therefore less than 4D 5, because contractions have to be done until all boxes are removed, that is 2times Therefore, the costs for comuting x H Hx based on 19 are less than 4D 5 M In the oen boundary case, we start at the left or right side and therefore only contractions of boxes of length 3 can occur and thus Figure 3c only costs 2D D 2 = 2D 3 instead of 2D 5 in the eriodic case Thus, the overall costs for the inner roduct is less than 4D 3 and for x H Hx less than 4D 3 M Minimization of the Rayleigh Quotient in Terms of ms Now, we use the ms formalism as vector ansatz to minimize the Rayleigh quotient 6 Therefore we start the als rocedure, udating A i by an imroved estimate by solving the generalized effective eigenvalue roblem In addition, we relace A i by a unitary air via SVD, moving the remaining

14 14 k2 k1,m1,k2,m2 i2,k2,k3 i3,k3,k4 k1,m1,k2,m2 i2,k2,k3 i3,k3,k4 i2,m2,m3 i3,m3,m4 i2,m2,m3 i3,m3,m4 a After the i 1 -contraction, we get a four leg tensor b Contraction concerning index k 2 k1,m1,i2,m2,k3 i3,k3,k4 k1,m1,k3,m3 i3,k3,k4 i2 m2 i2,m2,m3 i3,m3,m4 i3,m3,m4 c Contraction concerning the indices i 2 and m 2 d After the contraction concerning k 2, m 2 and i 2, we are in the same situation as in Figure 3a Figure 3: Contraction of the inner roduct of two ms vectors SVD term to the left neighbor, and so on A nice side effect here is that in the case of oen boundary conditions the matrix N r is the identity because all the matrix airs to index different from r are arts of unitary matrices and thus fulfil one of the gauge conditions 15 or 17 Together with the fact that the ms matrices at both ends are simly vectors we obtain x H x = = = trāi1 1 Āi i 1,,i Āi1 1 Āi i 1,,i tr i 1,,i tr = tr i 1,,i = tr j = tr i r i j A ir r tra i1 1 A i Āi1 1 A i1 1 Āi A i1 1 A i Āi1 1 A i1 1 Āi Āij j A ij j A r irh A i A i In the case of eriodic boundary conditions the SVD-based normalization can only be erformed for all u to one matrix air A ir r and so the denominator matrix N r is non-trivial However, the normalization is utilized to make the roblem numerically stable see [45]

15 15 The matrix H r for the eigenvalue roblem is given by x H Hx = = i 1,,i i 1,,i k = i 1,,i i 1,,i k = i 1,,i i 1,,i k = k = k trāi1 1 Āi tra i 1 1 A i tr Āi1 1 Āi A i 1 1 A i tr Āi1 1 A i 1 1 Āi A i [ tr i 1,,i i 1,,i tr j e i 1 H Q k 1 e i 1 e i H Q k e i Q k 1;i Qk 1,i1 Q k 1;i Qk 1,i1 ;i,i ;i,i ] Q k 1;i Āi1 1,i1 1 A i 1 1 Q k ;i Āi,i A i Q k j;i Āij i j,ij j A i j j j,ij The effective Hamiltonian can be comuted by contracting all indices excet the indices reresenting the unknowns in matrix air A ir r, i r, i r, m r, m r+1, k r, k r+1 leading to a 2D 2 2D 2 matrix H r So the costs for solving the eigenvalue roblem are in this case OD 4 In the eriodic case also a SVD for N r has to be comuted to solve the generalized eigenvalue roblem numerically stable, which leads to costs of order D Matrix Product Density Oerators The Matrix Product Oerator aroach extends the idea behind ms from vectors to oerators Matrix Product Density Oerators mdo [46] have the form tr A i1,i 1 1 A i,i e i e T i 22 i,i with unit vectors e i Matrix Product Oerators mo [4] read A i1 1 A i P i1 P i 23 i 1,,i tr with 2 2 matrices P, eg the Pauli matrices 1 Similarly, the tt format has also been extended to the matrix case [47] These mo concets may be used for a reresentation of the Hamiltonian, such that the alication of the Hamiltonian on an ms vector leads to a sum of ms vectors with less addends

16 D Systems: Aroximation by Projected Entangled Pair States It is natural to extend the couling toology of interest from linear chains to 2D arrays reresenting an entire lattice with oen or eriodic boundary conditions To this end, the matrices in ms are relaced by higher-order tensors thus giving rise to Projected Entangled Pair States es [25] Again, the underlying guideline is an area law [48, 18] The extension to higher dimensions, however, comes at considerably higher comutational cost: calculating exectation values becomes NP-hard actually the comlexity class is #P -comlete [49] This is one reason, why comuting ground states in two-dimensional arrays remains a major challenge to numerics [5] For 2D sin systems, the interaction between articles is also of 2D form, eg, as described by Figure 4 This leads to 2D generalization of ms using a k 42 k 43 k 44 i 41 i 42 i 43 i 44 k 41 k 42 k 43 k 44 k 32 k 33 k 34 i 31 i 32 i 33 i 34 k 31 k 32 k 33 k 34 k 22 k 23 k 24 i 21 i 22 i 23 i 24 k 21 k 22 k 23 k 24 k 12 k 13 k 14 i 11 i 12 i 13 i 14 Figure 4: A 2D system in the oen boundary case with hysical indices i j and ancilla indices k and k tensor network with small boxes related to tensor a ir,s with 5 legs k r,k r+1; k s, k s+1 Thus, an inner roduct of two such vectors would look like bi r,s k r,k r+1 ; k s, k s+1 a ir,s k r,k r+1; k s, k s+1 In a es vector, matrices are relaced by higher order tensors with one hysical index related to the given vector x and the other ancilla indices related to nearest neighbors according to the lattice structure of the tensor network [50] A es vector can be formally written as x = I R C R {A I R }e IR, where I stands for all x-indices in Region R, C R reresents the contraction of indices following the nearest neighbor structure To comute the inner roduct of two es vectors, we have to find an efficient ordering of the summations The related tensor network for the oen boundary case is dislayed in Figure 5a

17 17 In a first ste all airwise contractions relative to the vector indices i r,s are comuted Furthermore, in the roduced boxes the indices are newly numbered by combining k r,s with m r,s to larger index k r,s This generates Figure 5b i41,m42, m41, m51 i42,m42,m43, m42, m52 i43,m43,m44, m43, m53 k 42, k 41, k 51 k 42,k 43, k 42, k 52 k 43,k 44, k 43, k 53 i41,k42, k41, k51 i42,k42,k43, k42, k52 i43,k43,k44, k43, k53 i31,m32, m31, m41 i32,m32,m33, m32, m42 i33,m33,m34, m34, m43 k 32, k 31, k 41 k 32,k 33, k 32, k 42 k 33,k 34, k 33, k 43 i31,k32, k31, k41 i32,k32,k33, k32, k42 i33,k33,k34, k33, k43 i21,m22, m21, m31 i22,m22,m23, m22, m32 i23,m23,m24, m23, m33 k 22, k 21, k 31 k 22,k 23, k 22, k 32 k 23,k 24, k 23, k 33 i21,k22, k21, k31 i22,k22,k23, k22, k32 i23,k23,k24, k23, k33 i11,m12, m21 i12,m12,m13, m22 i13,m13,m14, m23 k 12, k 21 k 12,k 13, k 22 k 13,k 14, k 23 i11,k12, k21 i12,k12,k13, k22 i13,k13,k14, k23 a Contracting hysical indices b Grouing index airs Figure 5: Contraction of two es vectors: After the contraction of the hysical indices related to the first column a the index airs m r,s and k r,s are groued to larger indices k r,s Now the first and second column are contracted starting eg from the bottom, resulting in the network dislayed in Figure 6a Unfortunately, the boxes in the newly generated left column have more indices than in the starting column So we cannot roceed in this way directly In order to kee the number of indices constant, an aroximation ste is introduced The main idea is to reduce the number of indices by considering the first left column as ms with indices related to connections with the right neighbors as original hysical indices longer than 2 and aroximating the boxes by little tensors with only one leg instead of two to the vertical neighbors Such a reduction can be derived by the following rocedure We want to reduce the rank of size D 2 in index air {k 2,1, k 2,2 } to a new index k 2,1 of size D We can rewrite the whole summation in three arts, where c contains the contracted summation over all indices that are not involved in the actual reduction rocess This leads to the sum a{k 2,3,k 2,1 },{k 3,1,k 3,2 } b {k 3,1,k 3,2 },{k 3,3,k 4,1,k 4,2 } c {k 3,3,k 4,1,k 42 },{k 2,3,k 2,1 } The entries a in the above sum build the ms matrices A k 2,3 k 2,1,k 2,2,k 3,1,k 3,2 Collecting these matrices in a large rectangular matrix, we can aly the SVD on this matrix Now we can truncate the diagonal art of the SVD to reduce the matrices A from size D 2 D 2 to size D 2 D If we reeat this rocess along the first column all indices are reduced to length D Note that this aroximation

18 18 ste is not affected by any accuracy demand but it is erformed to kee the number of indices constant, which allows to roceed in an iterative way Reducing the rank of the indices in the first column leads to the network illustrated by Figure 6b with the same structure as Figure 5b, so we can contract column by column until we are left with only one column that can be contracted following Figure 7 { k 51, k 52} }, k 43, { k 41, k 42 k 43, k 44, k 43, k 53 k 52,k 43, k 42 k 43,k 44, k 43, k 53 { k 31, k 32} }, k 33, { k 41, k 42 k 33, k 34, k 33, k 43 k 32,k 33, k 42 k 33,k 34, k 33, k 43 { k 31, k 32} }, k 23, { k 21, k 22 k 23, k 24, k 23, k 33 k 32,k 23, k 22 k 23,k 24, k 23, k 33 } k 13, { k 21, k 22 k 13, k 14, k 23 k 13, k 22 k 13,k 14, k 23 a Contracting index airs b Result of the rank reduction Figure 6: Contraction scheme for es vectors: after the contracting illustrated in Fig 5b, The newly generated first column in a has sets of indices leading to a more comlex contraction rocess as illustrated by the double lines After the rank reduction b, we are in the same situation as in Fig 5b and can thus roceed in the same manner k 52,k 43, k 42 k 52,k 43, k 42 k 52,k 43, k 42 k 52,k 43, k 42 k 52,k 43, k 42 k 32,k 33, k 42 k 32,k 33, k 42 k 32,k 33, k 42 k 32,k 33, k 42 k 33, k 42 k 32,k 23, k 22 k 32,k 23, k 22 k 32,k 23 k 32 k 13, k 22 k 22 Figure 7: After alication of the column-by-column contraction scheme, we end u with the last column, which can be contracted in the illustrated way The overall costs deend on D 10, res D 20 in the eriodic case The socalled virtual dimension D is usually chosen a riori by the hysicists eg D 5, such that the comutations can still be afforded [45]

19 19 j 3,1 a 3,1;j3,1,j 2,1,j 2,2 j 2,1 j 2,2 a 2,1;j2,1,j 1,1,j 1,2 a 2,2;j2,2,j 1,3,j 1,4 j 1,1 j 1,2 j 1,3 j 1,4 a 1,1;j1,1,i 1,i 2 a 1,2;j1,2,i 3,i 4 a 1,3;j1,3,i 5,i 6 a 1,4;j1,4,i 7,i 8 i 1 i 2 i 3 i 4 i 5 i 6 i 7 i 8 Figure 8: Scheme of the Tree Tensor States tts 33 Tree Tensor States and Multi-scale Entanglement Renormalization Ansatz In ms the new sin articles are added one by one leading to a new matrix in the tensor ansatz A modified rocedure leads to Tree Tensor States tts In this ansatz the articles are groued airwise leading to a smaller subsace of the original Hilbert sace This rocedure is reeated with the blockairs until only one block is left This is dislayed in Figure 8 and formula x i1,,i 8 = j 1,1,,j 2,2 a 1,1;j1,1,i 1,i 2 a 1,4;j1,4,i 7,i 8 a 2,1;j2,1,j 1,1,j 1,2 a 2,2;j2,2,j 1,3,j 1,4 a 3,1;j3,1,j 2,1,j 2,2 where each block is isometric: ā k,i,j a k,i,j = δ k,k i,j Hence, in this scheme the network consists in a binary tree built by small isometric tensors with three indices It can be shown that all ms can be reresented by tts see [51] A further generalization of tts and ms is given by the Multi-Scale Entanglement Renormalization Ansatz mera, see [28] Besides the to tensor t with two indices, the network is built by two tyes of smaller tensors: three leg isometric tensors isometries and four leg unitary tensors disentanglers This formalism is dislayed by Figure 9 In connection with eigenvalue aroximation for mera, als cannot be used in order to otimize over the isometries that reresent the degrees of freedom; instead other otimization methods have to be alied 4 Reresentations of Tensors As any state in a hysical system see Sec 2 with articles may be seen as a -th order tensor, we want to resent some basics about tensors within this section In the next section, we will give some modifications and generalizations

20 20 t = tj,k j k l w = wα,β l α β γ u = u β,γ µ,ν µ ν i1 i2 i3 i4 i5 i6 i7 i8 i9 i10 i11 i12 i13 i14 i15 i16 Figure 9: The Multi-scale Entanglement Renormalization Ansatz mera with eriodic boundary conditions, consisting of isometries w and unitaries u 41 Some Basics about Tensors In its most general form, a th-order tensor A = a i1,,i R n1 n is a multiway array with indices A first-order tensor is thus a vector, a second-order tensor corresonds to a matrix If x and y are vectors ie firstorder tensors it is well known that the outer roduct x y := xy T is a rank-one matrix As generalization, if a 1,, a are vectors, the tensor A := a 1 a, which is defined as a i1,,i = a 1;i1 a 2;i2 a ;i is a rank-one tensor Hence, the alication of outer roducts constructs tensors of higher order The Kronecker roduct of matrices just corresonds to this definition of the outer roduct, but it reshaes the single tensor entries into a matrix of larger size To begin, we write a tensor A as a sum of rank-one tensors: A = R j=1 a j 1 a j 2 a j 24 If R is minimal in the reresentation 24 of A, we define the tensor rank of A as equal to R For illustrating tensors, matrices are often in use One ossibility to bring back a general tensor to a matrix is given by the mode-n-unfolding see [52]: When alying this technique, the tensor A = a i1,,i n,,i is reresented by the matrix A n := a i1,,i n,,i i n,{i 1,,i n 1,i n+1,,i }

21 21 The n-mode tensor matrix roduct is given as follows: Let A = a i1,,i n,,i be a -th order tensor and U = u j,in a matrix, then the mode-n-roduct n is defined as A n U = a i1,,i n u j,in i n i 1,,i n 1,j,i n+1,,i Beside the total rank of a tensor there also exist a local rank concet: the mode-n-rank of a tensor is the rank of the collection of all vectors belonging to index n If a given tensor A has the n-mode ranks r n, we define the multilinear rank of A as r 1,, r 42 Decomosition of Tensors For aroximating tensors x i1,,i there are two basic methods see [52]: the Tucker decomosition and the canonical decomosition The Tucker decomosition [53] x i1,,i = D m 1,,m y m1,,m a 1;i1,m 1 a 2;i2,m 2 a ;i,m 25 reresents the given tensor by a tensor y m1,,m with less dimension in each direction; D is called the Tucker rank, y m1,,m is called core tensor This concet is illustrated in Figure 10a In the case of a binary tensor x R 2 2, this is not meaningful, because then the Tucker rank is already 2 The canonical decomosition candecom, which is also known as Parallel Factorizationarafac, has the form x i1,,i = D s=1 a s 1;i 1 a s 2;i 2 a s ;i 26 Hence, the arafac decomoses the given tensor into a sum of rank one tensors see [54, 55], which is illustrated by Figure 10b If we think of x as a vector this is equivalent to x = D s=1 a s 1 a s with tensor roducts of smaller vectors One often finds the normalized form x = D s=1 λ s a s 1 a s 27 with vectors a s i of norm 1 If D is minimal in the reresentation 26 of x, it is called the canonical rank

22 22 Tensor Train Schemes Alication of the concets 25 or 26 would be a generalization of SVD that allows a substantial reduction in number of arameters for deriving a good aroximation for a given tensor Unfortunately, these decomositions have disadvantages like still exonential growth, lack of robust algorithms for rank reduction Oseledets and Tyrtyshnikov [56] roosed the following Tensor Train scheme tt as an attemt to overcome these difficulties In a first ste a dyadic decomosition is introduced by cutting the index set and the tensor in two arts introducing an additional summation with a newly introduced ancilla index m: x i1,,i k ;i k+1,,i = a 1;i1,,i k,m a 2;ik+1,,i,m 28 m This is the first ste of the Tree Tucker [55] decomosition Now we may aly this rocess recursively to a 1 and a 2 If we always use the index artitioning i j ; i j+1,, i, we arrive at the tt format x i1,,i = m 2,,m a 1;i1,m 2 a 2;i2,m 2,m 3 a ;i,m 29 Again we can distinguish between given, hysical indices i j and ancilla indices Figure 10c illustrates the tt decomosition concet y m1,,m m 1 m 2 m 3 m a 1;i1,m 1 a 2;i2,m 2 a 3;i3,m 3 a ;i,m i 1 i 2 i 3 a The Tucker decomosition scheme s i a 1;i1,s a 2;i2,s a 3;i3,s a ;i,s i 1 i 2 b The canonical decomosition scheme i 3 i a 1;i1,m 2 m 2 m 3 m 4 m a ;i,m a 2;i2,m 2,m 3 a 3;i3,m 3,m 4 i 1 i 2 c The Tensor Train decomosition scheme i 3 i Figure 10: Tensor decomosition schemes

23 23 The tt scheme 29 is exactly the ms form 9 for oen boundary conditions: x i1,,i = G i1 1 G i2 2 G i 30 with matrices G ij j of size D j D j+1, D 1 = D +1 = 1, where the matrix sizes differ and are called the comression rank In [57], Khoromskij generalizes the tt concet to Tensor Chains via the definition x i1,,i = a 1;i1,m 1,m 2 a 2;i2,m 2,m 3 a ;i,m,m 1, 31 m 1,,m which corresonds to ms with eriodic boundary conditions, ie x i1,,i = tr G i1 1 G i2 2 G i 32 Starting with Formula 28 this ste can also be alied recursively to the two newly introduced tensors a 1 and a 2 with any cutting oint Cutting a tensor in two arts should introduce as many ancilla indices as described by the tensor network connecting the two arts So in 1D there is only one neighboring connection that is broken u and introduces 1 additional index In the 2D network described in Figure 4 we could aly this method by cutting in a first ste the down left knot i 11, thus introducing two ancilla indices labeled with i 1,1, i 1,2, res i 1,1, i 2,1, reresenting the broken connections to the neighbors of i 1,1 Proceeding in this way knot by knot leads to the es form But generalizing the aroach we can also consider general cuts of region R of the tensor network in two regions R 1 and R 2, relacing the given tensor by a sum over a tensor roduct of two smaller tensors with indices related to R 1, res R 2, introducing as many ancilla indices as broken connections in the cut 5 Modifications of arafac and ms Reresentations In this section we want to introduce some own ideas concerning the generalization and modification of both the arafac decomosition scheme see 42 and the ms tt format These newly resented formats will be used as vector ansatz for the minimization of the Rayleigh quotient 6 As we have ointed out in Section 23, we are looking for reresentation schemes that allow both roer aroximation roerties and efficient comutations such as fast contractions for inner roduct calculations and chea matrix-vector evaluations In view of these requirements we resent both theoretical results concerning the comutational comlexity and numerical results showing the benefit of our concets At this oint we want to emhasize that our roblem does not consist in the decomosition of a known tensor but to find an aroriate ansatz for vectors and to formulate algorithms for the ground state comutation working

24 24 on such vector reresentations Hence, in this context we focus on algorithmic considerations and investigate, which modifications and generalizations of the resented concets are still affordable to solve hysical roblems, where we a riori work with aroximations with ranks which are given by the hysicists It turns out that the usage of low-rank aroaches suffices to give roer results, comare, eg, [58] 51 arafac Formats Any state x C 2 of a hysical system can be tensorized artificially in several ways In the easiest way we may rewrite x as a th ordered binary tensor of the form x = x i1,,i il =0,1 33 But we may also define blockings of larger size, which will follow the interactions of the hysical system eg nearest-neighbor interaction These blocking concets introduce formats with a larger number of degree of freedoms romising better aroximation roerties but still allow efficient comutations of matrix-vector and inner roducts For reroducing the hysical structure such as internal and external interactions we will also allow formats with different overlaing blockings in the addends, comare Subsection 53 Such blockings of indices can be seen as tensorizations of lower order q : x = x i1,,i s1,,i sq 1 +1,,i sq = x j1,,j q 34 where the kth mode combines t k := s k s k 1 binary indices and has therefore index j k = i sk 1 +1,, i sk of size 2 t k for reasons of consistency we define s 0 = 0 and s q = One way to find a convenient reresentation is to consider aroriate decomositions of the tensorization 34 In this context we don t consider the Tucker format 25 as it is only meaningful to decomose very large mode sizes meaning large sets of blocked indices Hence, from now on we consider the arafac concet and choose the ansatz vector to be a sum of rank-1 tensors see Figure 11 In the simlest case 33, the arafac decomosition takes the form D D x = x s 1 x s α s = 1 α s β s 1 β s, 35 s=1 s=1 a sum of tensor roducts of length 2-vectors In view of 11, the decomosition 35 can be seen as a secial ms form Indeed, every arafac reresentation 35 with D addends corresonds to an ms term with D D diagonal matrices This fact becomes clear from the construction A 0 r = α 1 r α D r, A1 r = β 1 r β D r

25 25 More generally, the arafac scheme for the tensorization 34 leads to the ansatz D x = x l 1 xl q 36 l=1 with vectors x l j of moderate size 2 tj Figure 11 illustrates such a decomosition x = x 1 1 x 1 2 x 1 3 x 1 q + x 2 1 x 2 2 x 2 3 x 2 q + x D 1 x D 2 x D 3 x D q Figure 11: arafac ansatz for a chosen tensorization of the vector x to be reresented Comutational Costs Inner roduct calculations of two reresentations of the form 36 reduce to the inner roduct of the block vectors: y H x = D y l H l 1 x 1 y l H l 2 x l,l =1 2 y l q H x l Therefore, the total costs for each of the D 2 inner roducts are 22 t1 + 2 t tq + q q2l + 1 2l + 1 q 37 where t := max{t 1, t 2,, t q } is the maximum number of groued indices and thus l = 2 t denotes the largest block size in x If all q index sets have the same size t ie t = /q, the costs can be bounded by q2 2 /q + 1 To comute the matrix-vector roduct efficiently, we grou the Hamiltonian H 2 in the same way, ie H = = M k=1 M k=1 k α k Q 1 Q k k s 1 Q s q 1+1 Qk α k H k 1 H k q

26 26 For the matrix-vector roduct we thus obtain M D Hx = α k H k 1 H q k x l 1 xl q = = k=1 M D k=1 l=1 M D k=1 l=1 α k H k 1 x l 1 y k,l 1 y k,l q l=1 H k q x l q 38 However, tyically we do not need the matrix-vector roducts 38 exlicitly, we only require them for inner roducts of the form y H Hx comare the nominator of the Rayleigh quotient y H Hx = M k=1 α k D l,l =1 y l 1 H H k 1 x l 1 y l q H H k q x l q 39 The roducts H k i x l i can be comuted imlicitly without constructing the matrices H k i exlicitly Each of these small matrix-vector roducts can be comuted linearly in the size of H k i ie 2 ti Thus, the total costs for each addend in the inner roduct 39 are 32 t1 + 2 t tq + q and can again be bounded by q3 2 /q + 1 in the case of equal block sizes Hence, the costs for 39 are 2MD 2 q3 2 /q + 1 Using arafac for the Ground State Problem Let us now aly the arafac aroach 36 as ansatz for the Rayleigh quotient minimization Then, Eq 6 reads D H D H min x l 1 xl q l=1 D x l 1 xl q l=1 H D x l 1 xl q l=1 x l 1 xl q l=1 40 This minimization task can be realized by an als-based rocedure In a first way we could think about the following roceeding: We start with a arafac reresentation of rank 1 D = 1, otimize it via als and then we successively add one summand and otimize it in an als-based way In the first stage we would have to minimize min x x H Hx x H x = min x 1 x 2 x q H H x 1 x 2 x q x 1,,x q x 1 x 2 x q H x 1 x 2 x q This otimization roblem can be solved via als: considering all of the x j u to x i as fixed, we obtain x 1 x i x q H H x 1 x i x q min x i x 1 x i x q H 41 x 1 x i x q

27 27 Following 39 and 37, we may contract all indices u to i and obtain min x i = min x i M α k x H 1 H k 1 x 1 x H i H k x i x H q H q k x q k=1 M k=1 x 1H x 1 x ih x i x H q x q α k β k x H i H k i x i γx ih x i = min x i i M H α x k β k i γ H k i x i k=1, x ih x i a standard eigenvalue roblem for a matrix of size 2 ti 2 ti that can be solved via classical iterative methods which only require the comutation of matrix vector roducts These roducts can be executed imlicitly without constructing the matrices exlicitly Now we suose that we have already otimized D 1 addends in the reresentation 36 and we want to find the next otimal addend This means that the ansatz vector is now of the form x = x 1 x q }{{} q = x j j=1 D 1 + l=1 y l 1 y l q }{{} =:y l with already otimized y l j -terms and vectors x i that have to be otimized via als Contracting over all terms u to x i comare Eq 39 we obtain x H Hx = x i H H i x i + u i H x i + x i H u i + β, where u i and β comrise the contractions with the y j terms For the denominator we analogously obtain x H Hx = x i H γix i + v i H x i + x i H v i + ρ Altogether we have to solve the generalized eigenvalue roblem H xi 1 H i u i xi H u i β 1 min 42 x i xi H 1 γi v i v i H ρ xi 1 It turns out that the denominator matrix can be factorized via a Cholesky factorization Therefore we have to solve a standard eigenvalue roblem of moderate size However, the roosed rocedure may cause roblems which result from the fact that, for general tensors, the best rank-d aroximation does not have to comrise the best rank-d 1 aroximation, see, eg, [52] Our numerical results Figure 12 will arove this fact