Tensor Sparsity and Near-Minimal Rank Approximation for High-Dimensional PDEs

Size: px

Start display at page:

Download "Tensor Sparsity and Near-Minimal Rank Approximation for High-Dimensional PDEs"

Angelina Pope
5 years ago
Views:

Collaborators: Markus Bachmayr, Ron DeVore, Lars Grasedyck,

1 Tensor Sparsity and Near-Minimal Rank Approximation for High-Dimensional PDEs Wolfgang Dahmen, RWTH Aachen Collaborators: Markus Bachmayr, Ron DeVore, Lars Grasedyck, Endre Süli Paris, Oct.11, 2013 W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

2 Contents Contents 1 Motivation, Background 2 Regularity Theorems Problem Setting Sparsity Models Main Results 3 Optimal Rank Approximation What is Known? Basic Strategy Main Result First Experiments W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

3 Motivation High-Dimensional Problems Data mining Parameter dependent PDEs: d = spatial dim. + parameter dim. Stochastic PDEs: d = Electronic Schrödinger equation: d = 3N Fokker-Planck equations for polymeric fluids: d = 3K, K = length of polymer chains Curse of Dimensionality: intractability results (Novak/Woźniakowski) Remedies (?): Accuracy ɛ comp. cost ɛ d/s Excessive regularity Hidden sparsity with respect to a problem dependent dictionary... separation of variables... W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

4 Main Paradigms Motivation Parameter dependent PDEs: (e.g. Reduced Basis Method) [Maday, Patera,...BCDDGW] a(u, v; p) = f, v, v X, p P R m, u(x, p) = n c i (p)u(x, p i ) High dimensional phase space: e.g. Fokker-Planck eqs operator splittings high-dimensional diffusion equation on product domain [Barrett/Süli] W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

5 Products help... Motivation f C s ([0, 1] d ) f (x) = f (x 1,..., x d ) ν n c ν ψ ν (x) r f k,1 (x 1 ) f k,d (x d ) k=1 r ( d ) l=1 c l,j ψ j (x l ) k=1 d.o.f. : n d =: N rdn =: N j n } {{ } f k,l (x l ) accur.: O(n s ) = O(N s/d ) rdn s = rd 1+s N s work/acc. : N ε d/s N r 1/s d 1+1/s ε 1/s W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

6 Regularity Theorems Problem Setting Setting: e.g. D = d j=1 2 x j = d D = I 1 I i 1 D i I i+1 I d, D j : H j (Ω j ) (H j (Ω j )) H j -elliptic, d j=1 Ω j =: Ω R dp d { H := L2 (Ω 1 ) L 2 (Ω j 1 ) H j (Ω j ) L 2 (Ω j+1 ) L 2 (Ω d ) } j=1 v 2 H := Dv, v, a(u, v) := Du, v v, w H D : H H, H L 2 (Ω) H, v 2 H s = v 2 s := D s v, v, H = H 1 Solution structure (?) a(u, v) = f, v, v H W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

7 Main Objectives Regularity Theorems Problem Setting Regularity : a(u, v) = f, v, v H... suppose that f is tensor sparse... n(ε)?... = u = D 1 f is also tensor sparse? u u k,1 u k,d k=1 Computatibility: Compute tensor sparse appr s to D 1 f with near-minimal cost realize (near-)minimal ranks find (near-)optimally sparse representations of tensor factors Issues: stability of tensor formats [Lathauwer, Hackbusch, Grasedyck, Oseledets,...] continuous versus discrete... a scaling trap W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

8 Tensor Sparsity Regularity Theorems Sparsity Models Model 1: Σ n := { g = r k=1 d g k,j : g k,j = j=1 µ Γ k,j c k,j,µ e j,µ, r d k=1 j=1 } #(Γ k,j ) n σ n (f ) H t := inf { f g H t : g Σ n } Given a growth sequence γ(n), n How to read this... A γ (1) ((Σ n), H t ) := {f H t : f γ,t := sup γ(n)σ n (f ) H t < }, n N v A γ (1) ((Σ n), H t ) σ n (v) H t γ(n) 1 v γ,t ε v ε Σ γ 1 ( v γ,t /ε), such that v v ε t ε...it takes γ 1 ( v γ,t /ε) d.o.f./rank to achieve accuracy ε in H t W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

9 Tensor Sparsity Regularity Theorems Sparsity Models Model 2: { Σ s,b n := g = n d k=1 j=1 } g k,j : g k H s, g k s, g s b, Σ n := Σ s,b n b>0 t < s : σ s,b n (f ) H t := inf g Σ s,b n f g t f γ,b,t,s := sup n N γ(n)σn s,b (f ) H t A γ (2) ((Σ n), H t ) := {f H t : b <, s.t. f γ,b,t,s < } W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

10 Sparsity Results Regularity Theorems Main Results Theorem 1: where i {1, 2} : f A γ (i) ((Σ n), H 1+ζ ) u = D 1 f Aˆγ (i)((σ n ), H 1 ) Complexity: ˆγ(n) := (γ G 1 )(n), G(n) := κ(ζ)n ( log(γ(n)c(f )) ) 2 i = 1: u ˆγ,1 2 f γ, 1+ζ, accur. ε #(c(u ε )) ˆγ 1( 2 f γ, 1+ζ /ε ) i = 2: #(c(u ε )) ( C 1 (f, ζ)γ 1 (ε 1 )(log(ε)) 2 d ) 1+1/ζ ε 1/ζ ( #ops(u ε ) = O (d ζ log(df(ε))f (ε) ζ ), F(ε) := κγ 1 C ε Examples: i = 1: γ(n) = n α ˆγ(n) (n/c log n) α i = 2: γ(n) = e αn C(ζ,f )(αn)1/3 ˆγ(n) e )( log(ε) ε ) 2 W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

11 Regularity Theorems Main Results A Tool: Exponential Sums...[Braess/Hackbusch] τ = τ 1 τ d sup x 1 sr (x) Ce π r, sr (x) = x [1, ] D 1 τ s r (D)τ := PROPOSITION 1: r ω r,k e α r,k D τ = k=1 For 1 t s 1, τ H t, one has r ω r,k e α r,k x k=1 r ( d ) ω r,k e α r,k D j τ j }{{} j=1 g j k=1 D 1 sr (2 s+t)π (D) H t H s Ce 2 r Eigensystem for the D j : {e j,k } k N, D j e j,k = λ j,k e j,k e ν : = e 1,ν1 e d,νd, De ν = λ ν e ν, λ ν = λ 1,ν1 + + λ d,νd v 2 s = ν N d λs ν v, e ν 2 W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

12 Complexity Regularity Theorems Main Results cost(d 1 f, ε) := computational cost of solving Du = f with accuracy ε f = τ: evaluate exponentials s r (D)τ = r ( d ) ω r,k e α r,k D j τ j, r = r(ε) log ε 2 k=1 j=1 e td j τ j = 1 2πi Γ e tγ (γi D j ) 1 τ j dγ, iy λ j Γ...truncation, sinc-quadrature d log ε solves at cost (ε/d) 1/ζ for log ε 2 terms x cost(d 1 τ, ε) < d 1+1/ζ ε 1/ζ log ε 3 (instead of: ε d/ζ ) W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

13 Inventory Optimal Rank Approximation What is Known? Some facts: [de Silva, Lathauwer, Hackbusch, Grasedyck, Oseledets, Schneider...] canonical format k N u k,1 u k,d i.g. unstable optimal subspace methods: unique best approximation exists and is realized by orthogonal projections - T- /(H-T)-formats HOSVD near minimimal rank approximation efficient numerical tools [Espig, Kolda,...] Operator equations: immediate reduction to a fixed discrete system accuracy considerations detached from continuous solution approximation error and residuals are measured in the same (Euclidean) norm - scaling trap accuracy and rank growth cannot be controlled simultaneously PGD...convergence, ranks?... [Falcó, Chinesta, Nouy,...] W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

14 Optimal Rank Approximation Basic Strategy Reduction to Problem in l 2 Universal background basis: {ψ ν = ψ ν1 ψ νd : ν J d } O.N.B. for L 2 (Ω) {( d ) Ψ = 2 2 ν 1 i 2 ψ ν =: s ν ψ ν }ν J Riesz-basis for H L 2(Ω) d Du = f Au = f, A = ( s ν a(ψ ν, ψ µ )s µ )ν,µ J, f = ( f, s ν ψ ν ) ν J Theorem: κ(a) := A A 1 < 1 u H u = (u ν ) ν J d l 2 (J d ) W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

15 Optimal Rank Approximation Basic Strategy Scheme: Perturbed Ideal Iteration Algorithm: u k+1 = C ε3 (k)( Pε2 (k)(u k + ω(f Au k )) ) u u k+1 ρ u u k, ρ < 1 Mode frames: U (j) k Tucker format: u = k 1 =1 k d =1 l 2 (J ), k N, j = 1,..., d, U (i) u, U (1) k 1 k, U(i) l = δ kl, k, l N U (d) (1) k d U k 1 U (d) k d =: c k U k k N d Hierarchical Tucker (H-T)-format: hierarchical factorization of core tensor (c k ) k N d W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

16 Optimal Rank Approximation Some New Ingredients... Basic Strategy Wavelet techniques for the 1D-tensor factors (coarsening, best N-term approximation) Thresholding Lemma: restoring a near-minimal rank approximation to the unknown solution from given approximations Contractions: π (i) (u) = ( π ν (i) i (u) ) ( ( ν i J := ) u ν 2 1 ) 2 ˇν i ν i J π (i) ν (u) = ( k U (i) ν,k 2 σ (i) 2 ) 1 2 k, π ν (i) (P U(u),r u) π ν (i) (u), ν J Exponetial sum approximation to (non-separable) scaling matrices S = (s ν δ ν,µ ) ν,µ J d in A = STS W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

17 Optimal Rank Approximation Optimal Convergence Benchmarks/Assumptions (cf. Model 2): Main Result u is tensor sparse u A γu ((Σ HT n ), l 2 (J d )) =: A γu HT A is tensor sparse - can be well approximated by low rank matrices π (i) (u) A s, i d, (v A s sup n n s (inf supp z n v z ) =: v A s < ) The low-rank approximations to A are s -compressible with s > s Theorem 2: For ε > 0 the Algorithm produces a u ε with u u ε ε s.t.: rank u ε γu 1 (C u A γu /ε), HT supp i (u) := k N supp U(i) k u ε A γu C u A γu, HT HT d ( d ) 1/s #(supp i (u ε )) < π (i) (u) A s/ε. Stability in Aγu HT, As : d d π (i) (u ε ) A s < π (i) (u) A s #(ops) < log ε C(A,f,log d)( d ) 1 max{ π (i) (u) A s, π (i) s (f) A s}/ε W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

18 Optimal Rank Approximation Numerical Experiments Experiments t (Tv)(t) := vds, 0 (I ω d d ) T u = f, f = d 2π χ[0,1/π] cos(2π 2 ) d = 32 d = 64 d = 128 W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

19 Optimal Rank Approximation Numerical Experiments Experiments d = 32 d = 64 d = 128 W. Dahmen (RWTH Aachen) Tensor Sparsity Oct.11, / 29

An Introduction to Hierachical (H ) Rank and TT Rank of Tensors with Examples

An Introduction to Hierachical (H ) Rank and TT Rank of Tensors with Examples Lars Grasedyck and Wolfgang Hackbusch Bericht Nr. 329 August 2011 Key words: MSC: hierarchical Tucker tensor rank tensor approximation