A regularized least-squares method for sparse low-rank approximation of multivariate functions

Transcription

1 Workshop Numerical methods for high-dimensional problems April 18, 2014 A regularized least-squares method for sparse low-rank approximation of multivariate functions Mathilde Chevreuil joint work with Prashant Rai, Loic Giraldi, Anthony Nouy GeM Institut de Recherche en Génie Civil et Mécanique LUNAM Université UMR CNRS 6183 / Université de Nantes / Centrale Nantes

2 Motivations Chorus ANR project Aero-thermal regulation in an aircraft cabin 39 random parameters Data basis of 2000 evaluations of the model Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 2

3 Motivations In telecommunication: electromagnetic field and the Specific Absorption Rate (SAR) induced in the body Over 4 random parameters FDTD method: 2 days/run. Few evaluations of the model available Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 3

4 Motivations In telecommunication: electromagnetic field and the Specific Absorption Rate (SAR) induced in the body Over 4 random parameters FDTD method: 2 days/run. Few evaluations of the model available Aim Construct a surrogate model of the true model from a small collection of evaluations of the true model that allows fast evaluations of output quantities of interest, observables or objective function. Propagation: estimation of quantiles, sensitivity analysis... Optimization or identification Probabilistic inverse problem Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 3

5 Uncertainty quantification using functional approaches Stochastic/parametric models Uncertainties represented by simple random variables ξ = (ξ 1,, ξ d ) : Θ Ξ defined on a probability space (Θ, B, P). ξ Model u(ξ) Ideal approach Compute an accurate and explicit representation of u(ξ): u(ξ) α IP u αφ α(ξ), ξ Ξ where the φ α(ξ) constitute a suitable basis of multiparametric functions Polynomial chaos [Ghanem and Spanos 1991, Xiu and Karniadakis 2002, Soize and Ghanem 2004] Piecewise polynomials, wavelets [Deb 2001, Le Maître 2004, Wan 2005] Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 4

6 Motivations Aproximation spaces S P = span{φ α(ξ) = φ (1) α 1 (ξ 1)... φ (d) α d (ξ d ); α I P } with a pre-defined index set I P, e.g. { } { } α N d ; α r α N d ; α 1 r { } α N d ; α q r, 0 < q < 1 Issue Approximation of a high dimensional function u(ξ), ξ Ξ R d #(I P ) 10, 10 10, , ,... Use of deterministic solvers in a black box manner Numerous evaluations of possibly fine deterministic models Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 5

7 Motivations Objective Compute an approximation of u S P u(ξ) u αφ α(ξ) α IP using few samples {u(y q )} Q q=1 where {y q } Q q=1 is a collection of sample points and the u(y q ) are solutions of the deterministic problem Exploit structures of u(ξ) u(ξ) can be sparse on particular basis functions u(ξ) can have suitable low rank representations Can we exploit sparsity within low rank structure of u? Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 6

8 Outline 1 Motivations and framework 2 Sparse low rank approximation 3 Tensor formats and algorithms Canonical decomposition Tensor Train format 4 Conclusion Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 7

9 Outline 1 Motivations and framework 2 Sparse low rank approximation 3 Tensor formats and algorithms Canonical decomposition Tensor Train format 4 Conclusion Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 7

10 Low rank approximation Approximation of function u using tensor approximation methods Exploit the tensor structure of function space ( S P = S 1 P 1... S d P d ; S k P k = span φ (k) i ) Pk i=1 Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 8

11 Low rank approximation Approximation of function u using tensor approximation methods Exploit the tensor structure of function space Low rank tensor subsets M with dim(m) = O(d) ( S P = S 1 P 1... S d P d ; S k P k = span M = {v = F M (p 1, p 2,..., p n)} [Nouy 2010, Khoromskij and Schwab 2010, Ballani 2010, Beylkin et al 2011, Matthies and Zander 2012, Doostan et al 2012,...] φ (k) i ) Pk i=1 Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 8

12 Low rank approximation Approximation of function u using tensor approximation methods Exploit the tensor structure of function space Low rank tensor subsets M with dim(m) = O(d) ( S P = S 1 P 1... S d P d ; S k P k = span M = {v = F M (p 1, p 2,..., p n)} [Nouy 2010, Khoromskij and Schwab 2010, Ballani 2010, Beylkin et al 2011, Matthies and Zander 2012, Doostan et al 2012,...] φ (k) i ) Pk i=1 Sparse low rank tensor subsets M m-sparse, ideally M m-sparse = {v = F M (p 1, p 2,..., p n); p i 0 m i ; 1 i n} with dim(m m-sparse ) dim(m). Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 8

13 Low rank approximation Least-squares in low rank subsets Approximation of v(ξ) M defined by min u v M v 2 Q with u v 2 Q = [Beylkin et al 2011, Doostan et al 2012] Q u(y k ) v(y k ) 2 k=1 Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 9

14 Low rank approximation Least-squares in low rank subsets Approximation of v(ξ) M defined by min u v M v 2 Q with u v 2 Q = [Beylkin et al 2011, Doostan et al 2012] Approximation of v(ξ) M m sparse defined by Q u(y k ) v(y k ) 2 k=1 min v M u v 2 Q s.t. p i 0 m i Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 9

15 Low rank approximation Least-squares in low rank subsets Approximation of v(ξ) M defined by min u v M v 2 Q with u v 2 Q = [Beylkin et al 2011, Doostan et al 2012] Approximation of v(ξ) M m sparse defined by Q u(y k ) v(y k ) 2 k=1 min v M u v 2 Q s.t. p i 0 m i min v M u v 2 Q + n λ i p i 1 i=1 (Lasso) Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 9

16 Low rank approximation Least-squares in low rank subsets Approximation of v(ξ) M defined by min u v M v 2 Q with u v 2 Q = [Beylkin et al 2011, Doostan et al 2012] Approximation of v(ξ) M m sparse defined by Q u(y k ) v(y k ) 2 k=1 min v M u v 2 Q s.t. p i 0 m i min v M u v 2 Q + n λ i p i 1 i=1 (Lasso) Alternating least-squares with sparse regularization For 1 i n and for fixed p j with j i min p i u F M (p 1,..., p i,..., p n) 2 Q+λ i p i 1 Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 9

17 Outline 1 Motivations and framework 2 Sparse low rank approximation 3 Tensor formats and algorithms Canonical decomposition Tensor Train format 4 Conclusion Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 10

18 Approximation in canonical tensor subset Rank-one canonical tensor subset R 1 = {w = w (1)... w (d) ; w (k) S k Pk s.t. w (k) (ξ k ) = φ (k) (ξ k ) T w (k)} Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 11

19 Approximation in canonical tensor subset Rank-one canonical tensor subset R 1 = where φ = { w = φ, w (1)... w (d) ; w (k) R P k (φ (1)... φ (d) ) (ξ) and with dim(r 1 ) = d k=1 P k } Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 11

20 Approximation in canonical tensor subset Rank-one canonical tensor subset where φ = R γ 1 = { w = φ, w (1)... w (d) ; w (k) R P k, w (k) 1 γ k } (φ (1)... φ (d) ) (ξ) and with dim(r γ 1 ) = d k=1 P k Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 11

21 Approximation in canonical tensor subset Rank-one canonical tensor subset where φ = R γ 1 = { w = φ, w (1)... w (d) ; w (k) R P k, w (k) 1 γ k } (φ (1)... φ (d) ) (ξ) and with dim(r γ 1 ) = d k=1 P k Rank-m tensor subsets R γ1,...,γ m m ={v = m i=1 w i ; w i R γi 1 } Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 11

22 Approximation in canonical tensor subset Rank-one canonical tensor subset where φ = R γ 1 = { w = φ, w (1)... w (d) ; w (k) R P k, w (k) 1 γ k } (φ (1)... φ (d) ) (ξ) and with dim(r γ 1 ) = d k=1 P k Rank-m tensor subsets R γ1,...,γ m m ={v = = { m i=1 v = φ, w i ; w i R γi 1 } m i=1 w (1) i... w (d) i ; w (k) 1 γk i i } Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 11

23 Algorithm for adaptive sparse tensor approximation Algorithms Progressive construction based on corrections in R γ 1 greedy construction of a basis {w i } m i=1 selected in a tensor subset Rγi 1 Compute u m = m i=1 α i w i using regularized least-squares [MC, R. Lebrun, A. Nouy, P. Ray, A least-squares method for sparse low rank approximation of multivariate functions, arxiv: , 2013] Direct approximation in R γ1,...,γ m m Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 12

24 Algorithm for adaptive sparse tensor approximation Algorithm for progressive construction Let u 0 = 0. For m 1, Compute a sparse rank-one correction w m R γ 1 by solving min u u m 1 w 2 w R γ Q 1 Computed using alternating minimization on the parameters of R γ 1. Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 13

25 Algorithm for adaptive sparse tensor approximation Algorithm for progressive construction Let u 0 = 0. For m 1, Compute a sparse rank-one correction w m R γ 1 by solving min u u m 1 φ, w (1)... w (d) 2 w (1) R P 1,...,w (d) R P Q + d Computed using Alternating regularized Least Squares d λ k w (k) 1 k=1 Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 13

26 Algorithm for adaptive sparse tensor approximation Algorithm for progressive construction Let u 0 = 0. For m 1, Compute a sparse rank-one correction w m R γ 1 by solving min u u m 1 φ, w (1)... w (d) 2 w (1) R P 1,...,w (d) R P Q + d Computed using Alternating regularized Least Squares: for 1 j d d λ k w (k) 1 k=1 min z Φ (j) w (j) 2 w (j) R n 2 + λ j w (j) 1 j where (Φ (j) ) qi = φ (j) i (y q j ) w (k) (y q k ) k j Lasso problem computed with LARS algorithm Optimal solution selected using the fast LOO CV error estimate [Blatman, Sudret 2011] Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 13

27 Algorithm for adaptive sparse tensor approximation Algorithm for progressive construction Let u 0 = 0. For m 1, Compute a sparse rank-one correction w m R γ 1 by solving min u u m 1 φ, w (1)... w (d) 2 w (1) R P 1,...,w (d) R P Q + d Computed using Alternating regularized Least Squares: for 1 j d d λ k w (k) 1 k=1 min z Φ (j) w (j) 2 w (j) R n 2 + λ j w (j) 1 j where (Φ (j) ) qi = φ (j) i (y q j ) w (k) (y q k ) k j Lasso problem computed with LARS algorithm Optimal solution selected using the fast LOO CV error estimate [Blatman, Sudret 2011] Set U m = span{w i } m i=1 (reduced approximation space) Compute u m = m i=1 α iw i R γ1,...,γ m m using sparse regularization min u m α i w i 2 α=(α 1,...,α m) R m Q + λ α 1 Best rank selected using cross validation method Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 13 i=1

28 Illustration: checker-board function Rank-2 function: u(ξ 1, ξ 2) = 2 i=1 w (1) i (ξ 1)w (2) i (ξ 2) w 1 (1) (ξ1 ) w 2 (1) (ξ1 ) 1 ξ 1 ξ 1 5/6 (a) w (1) 1 (ξ 1) (b) w (1) 2 (ξ 1) 4/6 3/6 w 1 (2) (ξ2 ) w 2 (2) (ξ2 ) 2/6 ξ 2 (c) w (2) 1 (ξ 2) with dimension: d = 2 ξ i U(0, 1). Ξ = (0, 1) 2. (d) w (2) 2 (ξ 2) ξ 2 1/ /6 2/6 3/6 4/6 5/6 1 Approximation of u in S 1 P 1 S 2 P 2 Piecewise polynomials of degree p defined on a uniform partition of Ξ k of s intervals: S k P k = P p,s Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 14

29 Illustration: checker-board function Performance of the method for sparse low rank approximation Q = 200 samples Optimal rank m op selected using 3-fold cross validation Relative error ε estimated with Monte Carlo integration Comparison of different regularizations within Alternated Least Squares OLS l 2 l 1 Approximation space ε m op ε m op ε m op R m(p 2,3 P 2,3 ), P = R m(p 2,6 P 2,6 ), P = R m(p 2,12 P 2,12 ), P = R m(p 10,6 P 10,6 ), P = With few samples: l 1-regularization detects sparsity and gives accurate results Rank 2 is retrieved Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 15

30 Illustration: Friedman function Friedman function f (ξ) = 10sin(πξ 1ξ 2) + 20(ξ 3 0.5) ξ 4 + 5ξ 5 Dimensions: d = 5 ξ i, i = 1,..., 5 are uniform random variables over [0, 1]. Approximation in S P = 5 k=1 Sk P k Polynomials of degree p: S k P k = P p Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 16

31 Illustration: Friedman function Friedman function f (ξ) = 10sin(πξ 1ξ 2) + 20(ξ 3 0.5) ξ 4 + 5ξ 5 Dimensions: d = 5 ξ i, i = 1,..., 5 are uniform random variables over [0, 1]. Approximation in S P = 5 k=1 Sk P k Polynomials of degree p: S k P k = P p What is the sufficient number of samples Q just needed given an a priori underlying approximation space? Q = f (p, d, m) [G. Migliorati, F. Nobile, E. von Schwerin, and R. Tempone, 2011]: sufficient condition for a stable approximation of a multivariate function using OLS: Q (#(I P )) 2 Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 16

32 Illustration: Friedman function Number of samples needed (no regularization): rank-1 Number of samples: Q = cd(p + 1) c R + Number of samples: Q = cd(p + 1) 2 c R c= c=3 c=5 c=10 c= c=0.5 c=1 c=1.5 c= c=3 Error 0.14 Error Polynomial degree Polynomial degree Q = d(p + 1) 2 Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 17

33 Illustration: Friedman function Number of samples needed (no regularization): rank-4 Number of samples: Q = cmd(p + 1) c R + Number of samples: Q = cmd(p + 1) 2 c R c=1 c=3 c=5 c=10 c= c=0.5 c=1 c=1.5 c=2 c=3 Error Error Polynomial degree Polynomial degree Q = md(p + 1) 2 Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 17

34 Illustration: vibration analysis Discrete problem u C N, ( ω 2 M + iωc + K)u = f where K = E K and C = iωηe K with { ξ 1 on horizontal plate, E = ξ 2 on vertical plate, { ξ 3 on horizontal plate, η = ξ 4 on vertical plate, where the ξ k U( 1, 1), k = 1,, 4. Ξ = ( 1, 1) 8. Approximation of a Variable of Interest I (u) in S P = 5 k=1 Sk P k Polynomials of degree p: S k P k = P p I (u)(ξ) = log u c, Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 18

35 Illustration: vibration analysis Evolution of error w.r.t. p Q = 80 Sparsity ratio w.r.t. p Q = 80 Q = 200 Q = 200 dashed lines: OLS, solid lines: with l 1 regularization Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 19

36 Illustration: vibration analysis Evolution of error w.r.t. p dashed lines: OLS, solid lines: with l 1 regularization Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 20

37 Outline 1 Motivations and framework 2 Sparse low rank approximation 3 Tensor formats and algorithms Canonical decomposition Tensor Train format 4 Conclusion Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 21

38 Tensor Train format ξ 1... ξ d v = r 1 i 1 =1 v (1) 1,i 1 v (2,...,d) i 1 ξ 1 ξ 2... ξ d ξ 2... ξ d 2 ξ d 1 ξ d ξ d 1 ξ d [Oseldets 2009,...] Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 22

39 Tensor Train format ξ 1 ξ 1... ξ d ξ 2... ξ d v = v = r 1 i 1 =1 r 1 i 1 =1 v (1) 1,i 1 v (2,...,d) i 1 v (1) 1,i 1 r 2 i 2 =1 v (2) i 1 i 2... r d 1 i d 1 =1 v (d 1) i d 2 i d 1 v (d) i d 1,1 ξ 2... ξ d 2 ξ d 1 [Oseldets 2009,...] ξ d 1 ξ d ξ d Tensor Train subsets TT (1,r1,...,r d 1,1) = TT r The set of tensors TT r (S) is defined by { TT r = v = } v (k) i k 1 i k ; v (k) i k 1 i k S k P k. i I k where I = {i = (i 0, i 1,..., i d 1, i d ); i k {1,..., r k }} with r 0 = r d = 1 Parameterization } TT r = {v = F r (v 1,..., v d ); v k (R P k ) r k 1 r k Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 22

40 Algorithms Alternating least-squares in TT r For a given rank vector r min u v 2 Q + v TT r d λ k vec(v k ) 1 k=1 Question of selection of rank vector r Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 23

41 Algorithm for adaptive sparse tensor approximation: DMRG Re-parameterization Consider the tensor w (k) (S k P k ) r k 1 (S k+1 P k+1 ) r k+1 : w (k) = rk ik =1 v (k, ) i k v (k+1, ) i k v = F k r (v, w (k) ) = r 1 i 1 =1... r k 1 r k+1 i k 1 =1 i k+1 =1... r d 1 i d 1 =1 Compute sparse low-rank w (k) with adaptive rank Modified alternating least-squares algorithm For k {1,, d 1} Compute w (k) (S k P k ) r k 1 (S k+1 P k+1 ) r k+1 by solving min w (k) R (r k 1 P k ) (r k+1 P k+1 ) u F k (v, w (k) ) v (1) 1i 1... w (k) i k 1 i k+1... v (d) i d Q Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 24

42 Algorithm for adaptive sparse tensor approximation: DMRG Re-parameterization Consider the tensor w (k) (S k P k ) r k 1 (S k+1 P k+1 ) r k+1 : w (k) = rk ik =1 v (k, ) i k v (k+1, ) i k v = F k r (v, w (k) ) = r 1 i 1 =1... r k 1 r k+1 i k 1 =1 i k+1 =1... r d 1 i d 1 =1 Compute sparse low-rank w (k) with adaptive rank Modified alternating least-squares algorithm For k {1,, d 1} Compute sparse w (k) (S k P k ) r k 1 (S k+1 P k+1 ) r k+1 by solving min w (k) R (r k 1 P k ) (r k+1 P k+1 ) u F k (v, w (k) ) v (1) 1i 1... w (k) i k 1 i k+1... v (d) i d Q + λ k vec(w (k) ) 1 Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 24

43 Algorithm for adaptive sparse tensor approximation: DMRG Re-parameterization Consider the tensor w (k) (S k P k ) r k 1 (S k+1 P k+1 ) r k+1 : w (k) = rk ik =1 v (k, ) i k v (k+1, ) i k v = F k r (v, w (k) ) = r 1 i 1 =1... r k 1 r k+1 i k 1 =1 i k+1 =1... r d 1 i d 1 =1 Compute sparse low-rank w (k) with adaptive rank Modified alternating least-squares algorithm For k {1,, d 1} Compute sparse w (k) (S k P k ) r k 1 (S k+1 P k+1 ) r k+1 by solving min w (k) R (r k 1 P k ) (r k+1 P k+1 ) u F k (v, w (k) ) v (1) 1i 1... w (k) i k 1 i k+1... v (d) i d Q + λ k vec(w (k) ) 1 Compute best low-rank approximation in (S k P k ) r k 1 (S k+1 P k+1 ) r k+1 using SVD adaptive rank rk v = r 1 i 1 =1... rk... i k =1 r d 1 i d 1 =1 v (1) 1i 1... v (k, ) i k 1 i k v (k+1, ) i k i k+1... v (d) i d 1 1 Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 24

44 Illustration: sine of a sum Sine function: with ξ i U( 1, 1). Ξ = ( 1, 1) 6. u(ξ) = sin(ξ 1 + ξ ξ 6) Evolution of error with respect to sample size Q Approx. in S P = 6 k=1 Sk P k ;S k P k = P 2 Approx. in S P = 6 k=1 Sk P k ;S k P k = P Error 10 1 Tensor Train (MALS) Error Tensor Train (MALS) Least Square+l 1 Greedy R Least Square+l 1 Greedy R 1 R m 10 3 R m Sample Size Sample Size Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 25

45 Illustration: borehole function The Borehole function models water flow through a borehole: Dimension: d = 8 2πT u(h u H l ) f (ξ) = ( ) 2LT ln(r/r w ) 1 + u + Tu ln(r/r w )rw 2 Kw T l r w radius of borehole (m) N(µ = 0.10, σ = ) r radius of influence (m) LN(µ = 7.71, σ = ) T u transmissivity of upper aquifer (m 2 /yr) U[63070, ] H u potentiometric head of upper aquifer (m) U[990, 1110] T l transmissivity of lower aquifer (m 2 /yr) U[63.1, 116] H l potentiometric head of lower aquifer (m) U[700, 820] L length of borehole (m) U[1120, 1680] K w hydraulic conductivity of borehole (m/yr) U[9855, 12045] Approximation in S P = 8 k=1 Sk P k Polynomials of degree p: S k P k = P p p = 2: P = 6561 p = 3: P = Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 26

46 Illustration: borehole function Behavior of the algorithm Evolution of error w.r.t. Q TT ranks (Q=200, p=3) r 1 p=2 r p=3 15 r 3 r 4 r 5 Error Rank 10 r 6 r Sample Size (Q) DMRG iteration Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 27

47 Illustration: Canister Stochastic PDE u (κ u) + c(d u) = σu t u = ξ 1 on Γ 1 Ω t on Ω1 Ω2 u = 0 on Γ 2 Ω t u,n = 0 on ( Ω\(Γ 1 Γ 2)) Ω t with ξ 1 u(t = 0) U[0.8, 1.2] on Ω ξ 2 σ U[8, 12] on Ω 2 ξ 3 σ U[0.8, 1] on Ω 1 ξ 4 c U[1, 5] ξ 5 κ U[0.02, 0.03] Approximation of a Variable of Interest I (u) in S P = 5 k=1 Sk P k I (u) = u(x, t)dxdt T Ω 3 Polynomials of degree p: S k P k = P p Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 28

48 Illustration: Canister Evolution of error with respect to sample size Q Approx. in S P = 5 k=1 Sk P k ;S k P k = P 2 Approx. in S P = 5 k=1 Sk P k ;S k P k = P 3 Tensor Train (MALS) Least Square+l 1 Tensor Train (MALS) Least Square+l 1 Error 10 1 Greedy R 1 Error 10 1 Greedy R Sample Size Sample Size Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 29

49 Illustration: Canister Order of separation of variables ξ 1... ξ 5 ξ 1 ξ 2... ξ 5 ξ 2 ξ 3ξ 4ξ 5 ξ 3 ξ 4ξ ξ 4 ξ Error Sample Size Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 30

50 Illustration: Canister Order of separation of variables ξ 1... ξ 5 ξ 1 ξ 2... ξ 5 ξ 2 ξ 3ξ 4ξ 5 ξ 3 ξ 4ξ ξ 4 ξ Error Sample Size Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 30

51 Illustration: Canister Order of separation of variables ξ 1... ξ 5 ξ 1... ξ 5 ξ 1 ξ 2... ξ 5 ξ 1 ξ 2... ξ 5 ξ 2 ξ 3ξ 4ξ 5 ξ 5 ξ 2ξ 3ξ 4 ξ 3 ξ 4ξ 5 ξ 2 ξ 3ξ ξ 4 ξ ξ 3 ξ Error Error Sample Size Sample Size Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 30

52 Outline 1 Motivations and framework 2 Sparse low rank approximation 3 Tensor formats and algorithms Canonical decomposition Tensor Train format 4 Conclusion Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 31

53 Conclusion Least-squares method for sparse low rank approximation of high dimensional functions A non intrusive method Detects and exploits low-rank and sparsity Adaptive rank Outlook More analyses on the suffisant number of samples to find an approximation in a tensor subset Include adaptivity with respect to polynomial degree for underlying approximation spaces Strategies for optimal separation of variables (choice of tree) This research was supported by EADS Innovation Works and by the French National Research Agency (ANR) (CHORUS project) Motivations and framework Sparse LR approx. Tensor formats & alg. Conclusion 32