TENSORS AND COMPUTATIONS

Transcription

1 Institute of Numerical Mathematics of Russian Academy of Sciences 11 September 2013

2 REPRESENTATION PROBLEM FOR MULTI-INDEX ARRAYS Going to consider an array a(i 1,..., i d ) of size n }. {{.. n }. d times We have no hope to store all n d elements. For any practical computation we need special structure and condensed representations of d-arrays.

3 REDUCTION OF DIMENSIONALITY i 1 i 2 i 3 i 4 i 5 i 6 i 1 i 2 i 3 i 4 i 5 i 6 i 1 i 2 i 3 i 4 i 5 i 6 i 3 i 4 i 5 i 6

4 HOW RANK-ONE DECOMPOSITION BECOMES TENSOR TRAIN Consider a rank-one separation of variables a(i 1, i 2, i 3 ) = g 1 (i 1 ) g 2 (i 2 ) g 3 (i 3 ). Now, consider g 1 (i 1 ), g 2 (i 2 ), g 3 (i 3 ) as matrices of agreed sizes so that the product is a scalar. Then a(i 1, i 2, i 3 ) = r 1 r 2 α 1 =1 α 2 =1 g 1 (i 1, α 1 ) g 2 (α 1, i 2, α 2 ) g 3 (α 2, i 3 ).

5 TENSOR TRAIN (TT) DECOMPOSITION a(i 1,..., i d ) = d g k (α k 1, i k, α k ) k=1 Assume summation over repeated indices. 1 i k n k for 1 k d 1 α k r k for 0 k d and r 0 = r d = 1 r k are called TT ranks

6 2D TENSOR TRAIN EXAMPLE a(i 1, i 2 ) = g 1 (i 1, α 1 ) g 2 (α 1, i 2 ) This is the skeleton (dyadic) decomposition of a matrix! A = G 1 G 2 A is n 1 n 2, G 1 is n 1 r 1, G 2 is r 1 n 2 r 1 rank A

7 3D TENSOR TRAIN EXAMPLE a(i 1, i 2, i 3 ) = g 1 (i 1, α 1 ) g 2 (α 1, i 2, α 2 ) g 3 (α 2, i 3 )

8 TENSOR TRAIN IS EASY TO GET For a 3-tensor we need two skeleton (dyadic) decompositions for associated unfolding matrices: a(i 1, i 2 i 3 ) = g 1 (i 1, α 1 ) a 1 (α 1, i 2 i 3 ) a 1 (α 1 i 2, i 3 ) = g 2 (α 1 i 2, α 2 ) g 3 (α 2, i 3 ) For a d-tensor we need d 1 skeleton (dyadic) decompositions.

9 IF WE APPROXIMATE USING SVD THEN LOCAL ERROR IN EACH SKELETON DECOMPOSITION DOES NOT BLOW UP THEOREM. If the Frobenius-norm error for kth skeleton decompostion is ε k, then the overall error E is upper bounded by E d 1 ε 2 k. k=1 I. Oseledets, E. Tyrtyshnikov, TT-cross approximation for multidimensional arrays, Linear Algebra Appl., 432 (2010), pp

10 TWO TYPES OF OPTIMIZATION PROBLEMS Given a functional f (x), find its approximate minimizer in the tensor train format. DMRG algorithm (White 1993) AMEn algorithm (Dolgov-Savostyanov 2013) Given a functional f (x), chase its global minimum using tensor trains. Application to the docking problem as an alternative to genetic algorithms.

11 GLOBAL SEARCH A general heuristic scheme includes: Choose a reasonably small set M of optima suspects. Inflate M to a reasonably larger set M e.g. by mutation and crossover operations in the genetic or simulating annealing algorithms Assign some probabilities to the points of M and deflate it to M of the same cardinality as M. Set M := M and repeat.

12 GLOBAL SEARCH IN A LOW-RANK MATRIX Find a low-rank skeleton representation or approximation e.g. by the cross interpolation algorithm Find the maximal element using the skeletons e.g. by reducing to the eigenvalue problem for a structured diagonal matrix

13 COLUMN-AND-ROW INTERPOLATION OF MATRICES [ ] A11 A A = 12 A 21 A 22 A 11 is r r A can be interpolated on the first r columns and rows by ] [ ] A11 A 12 [ A11 A 21 A 1 11

14 COLUMN-AND-ROW INTERPOLATION OF MATRICES [ ] A11 A 12 A 21 A 22 = [ A11 A 21 ] A 1 11 [ A11 A 12 ] [ ] A 22 A 21 A 1 11 A 12

15 MAXIMAL VOLUME PRINCIPLE THEOREM (Goreinov, Tyrtyshnikov) Let [ ] A11 A A = 12, A 21 A 22 where A 11 is a r r block with maximal determinant in modulus (volume) among all r r blocks in A. Then the rank-r matrix ] A r = [ A11 A 21 A 1 11 [ A11 A 12 ] approximates A with the Chebyshev-norm error at most in (r + 1) 2 times larger than the error of best approximation of rank r.

16 MAXIMIZATION VIA CROSS INTERPOLATION DEFINITION We call r r submatrix A of rectangular m n matrix A maximum volume submatrix, if it has maximum determinant in modulus among all possible r r submatrices of A.

17 MAXIMIZATION VIA CROSS INTERPOLATION DEFINITION We call r r submatrix A of rectangular n r matrix A of full rank dominant, if all the entries of are not greater than 1 in modulus. AA 1 DEFINITION We call r r submatrix A of rectangular m n matrix A dominant, if it is dominant in the columns and rows it occupies.

18 MAXIMIZATION VIA CROSS INTERPOLATION THEOREM If A is a dominant r r submatrix of a m n matrix A of rank r, then A A /r 2.

19 MAXIMIZATION VIA CROSS INTERPOLATION THEOREM If A is maximum-volume r r (nonsingular) submatrix of m n matrix A, then A A /(2r 2 + r). S. Goreinov, I. Oseledets, D. Savostyanov, E. Tyrtyshnikov, N. Zamarashkin, How to find a good submatrix, Matrix Methods: Theory, Algorithms and Applications. Devoted to the Memory of Gene Golub (eds. V.Olshevsky and E.Tyrtyshnikov), World Scientific Publishers, Singapore, 2010, pp

20 MINIMIZATION VIA MAXIMIZATION Φ n (x) := exp{ n(f (x) f n )} Assume that Φ n (x n+1 ) 1 C Φ(x min). Then exp{ n(f n+1 f n )} 1 C exp{ n(f min f n )} f n+1 f min log C n

21 MATRIX CROSS ALGORITHM Given initial column indices j 1,..., j r. Find good row indices i 1,..., i r in these columns. Find good column indices in the rows i 1,..., i r. Proceed choosing good columns and rows until the skeleton cross approximations stabilize. E.E.Tyrtyshnikov, Incomplete cross approximation in the mosaic-skeleton method, Computing 64, no. 4 (2000),

22 TENSOR-TRAIN CROSS ALGORITHM Let a 1 = a(i 1, i 2, i 3, i 4 ). Seek crosses in the unfolding matrices. On input: r initial columns in each. Select good rows. A 1 = [a(i 1 ; i 2, i 3, i 4 )], J 1 = {i (β 1) 2 i (β 1) 3 i (β 1) 4 } A 2 = [a(i 1, i 2 ; i 3, i 4 )], J 2 = {i (β 2) 3 i (β 2) 4 } A 3 = [a(i 1, i 2, i 3 ; i 4 )], J 3 = {i (β 3) 4 } rows matrix skeleton decomposition I 1 = {i (α 1) 1 } a 1 (i 1 ; i 2, i 3, i 4 ) a 1 = α 1 g 1 (i 1 ; α 1 ) a 2 (α 1 ; i 2, i 3, i 4 ) I 2 = {i (α 2) 1 i (α 2) 2 } a 2 (α 1, i 2 ; i 3, i 4 ) a 2 = α 2 g 2 (α 1, i 2 ; α 2 ) a 3 (α 2, i 3 ; i 4 ) I 3 = {i (α 3) 1 i (α 3) 2 i (α 3) 3 } a 3 (α 2, i 3 ; i 4 ) a 3 = α 3 g 3 (α 2, i 3 ; α 3 ) g 4 (α 3 ; i 4 ) Finally a = α 1,α 2,α 3,α 4 g 1 (i 1, α 1 ) g 2 (α 1, i 2, α 2 ) g 3 (α 2, i 3, α 3 ) g 4 (α 3, i 4 )

23 TENSOR TRAIN FROM CROSSES IN UNFOLDING MATRICES d A(i 1... i d ) = A(J k 1, i k, J >k ) [A(J k, J >k )] 1 k=1 I. Oseledets, E. Tyrtyshnikov, TT-cross approximation for multidimensional arrays, Linear Algebra Appl., 432 (2010), pp

24 QUASIOPTIMALITY THEOREM FOR TENSOR TRAINS THEOREM (Savostyanov 2013) Assume that a d-tensor A is approximated by à on the maximal volume crosses in the unfolding matrices, and let the error is upper bounded by ε A C in each matrix. Then for sufficiently small ε we have A à C 2drε A C.

25 DIRECT DOCKING IN THE DRUG DESIGN ACCOMMODATION OF LIGAND INTO PROTEIN LIGAND

26 DIRECT DOCKING IN THE DRUG DESIGN ACCOMMODATION OF LIGAND INTO PROTEIN LIGAND

27 MATHEMATICAL COMPONENTS OF THE DOCKING PROBLEM Define which degrees of freedom describe the ligand and the target protein and parametrize all possible interactions between them. Define the scoring function to be optimized. Find an efficient optimization algorithm over all selected degrees of freedom.

28 DOCKING AS A GLOBAL OPTIMIZATION PROBLEM DIFFICULTIES: Degrees of freedom amount to and higher. Many local minima. Singularities with large values of energy. High complexity of evaluation of the energy function.

29 OPTIMIZATION USING TT INPUT: f (x 1,..., x d ) and n n grid. OUTPUT: approximation to the global minimum. IN THE LOOP: Step 1: Transformation of the functional s.t. arg max g(x) = arg min f (x). E.g. g(x) = arcctg(f (x) f ). Step 2: TT-CROSS interpolation with the adaptive choice of pseudo-max nodes. Step 3: Local optimizations of pseudo-max nodes. Step 4: Renewal of f.

30 TTDock vs SOL: chk1_8

31 TTDock vs SOL: urokinase_7

32 TTDock vs SOL: erk2_000124

33 DOCKING PROGRAM SOL (DIMONTA)

34 COMPARISON OF SOL AND TT-DOCK

35 TENSOR TRAIN DOCKING (TTdock) Tensor Train Decomposition opens new prospects in Global Minimum Search TTdock more than 10 times faster than SOL Direct docking: direct calculaion of all interactions between ligand and protein atoms Tensor Train Mining Minima: Global + Local Minima D.Zheltkov, E.T. in collaboration with V.Sulimov and DIMONTA

36 WHY SHOULD WE USE TENSOR TRAINS