On generalized sparse grids

Transcription

1 On generalize sparse gris Michael Griebel Outline. High-imensional problems an curse o imensionality. Dimension ecomposition o unctions 3. Sparse gris 4. Energy-norm base sparse gris 5. Aaptive sparse gris 6. Applications

2 High(er imensional problems Classical physics: most problems in 3 space+time, compl. geometry Higher imensional problems? PDEs rom mathematical moelling, stochastics iusion equation, Fokker-Planck equation, iusion approximation o iscrete processes, networks (Mitzla,Dai viscoelasticity in polymer luis (Rousse, reaction mechanisms in biology an chemistry (Sjoeberg, Loetstet, Heglan,, option pricing, homogenization with multiple scales (Cioranescu,, Hoang, Matache, Schwab quantum mechanics, Schröinger equation (Yserentant, Fla ata analysis, statistical learning (Garcke, Heglan stochastic PDEs (Toor,Schwab,Matthies Domain simple, prouct structure [0,], [-a,a], hypersphere S, R with ecay or x i

3 : ( Curse o imension, V (r Bellmann 6: curse o imension, r isotropic smoothness N H C( N Fin situations where curse can be broken? Trivial: restrict to s r / O( N r / sr H r O( N O( N but practically not very relevant c / c O( N

4 Curse o imension Consier class o unctions o where in L FL => expect But Barron 93 showe class o unctions with Fourier transorm O( N Meanwhile other unction classes known Raial basis schemes, Gaussian bumps, (Y. Meyer Niyogi, Girosi 98: ball in Besov space Stochastic sampling techniques, MC N Spaces with boune mixe erivatives with FL / In any case: some smoothness changes with N O( N / B r, (

5 Concentration o measure What means smoothness or anyway? Concentration o measure:(milman 88, Talagran 95, Gromov 99 Lipschitz with constant L on -sphere, P normalize Lebesgue measure, X uniormly istribute Then: P( ( X E ( X t c exp( ct / L => every Lipschitz unction on suiciently highimensional omain is well approximate by constant unction! (Heglan, Pozzi 05

6 Kolmogorov 56: Lemma o Kolmogorov ex. + cont. strictly increasing unctions ex. constants or some (non-smooth, i i i but: non-constructive result, epenent on : (0, (0, G.,Braun 009: recent constructive proo in Constructive Approximation IBC, weighte RK Hilbert spaces, Wozniakowski, Sloan =>There is hope or high-imensional problems j i ( x,..., x g( ( x g C(0, j i j

7 Approach Basic principles: im series expansion with ecay -im prouct construction Trunctation o resulting multivariate expansion Eect: reuction o cost complexity nearly same accuracy as ull prouct necessary: certain smoothness requirements

8 Introuctory examples Napier s multiplication (John Napier ( Archimees approach or pi an Cavalieri s/fubini s theorem Sparse gris or integration, approximation o unctions an PDEs, etc.

9 N H s Summary Classical approach: curse o imension an intractability c( N,..,3/ 4 O( N r / r / sr H Stronger regularity/norms curse only wrt log-terms r ( / N s c( N (log( N H H or no curse at all r N s c( N s H H but still not tractable, constant grows exponentially,..,0/ r mix sr mix Lower eective imension an lower-im. maniols no curse ue to eective imension e e r / N s c( N s H H an constant grows exponentially only wrt eective imension, e,..,0 r

10 Function ecomposition splitting o one-imensional space V C W projection P : V C onto subspace o constants splitting o -im. space by tensor prouct V ( ( i ( i i ( C i i j jk W i i j ( C i C C ( ( C W C ( ( W W ( i ( i ( i W C W W W ( j ( ( ( ( j C W ( k ( C (

11 Function ecomposition splitting o associate -im unction ( x,..., x u ( xu u{,..., } 0 i( xi i, j ( xi, x j i i i j i i j jk subspaces, terms ( xi, x j, xk,..., ( x,..., x i, j, k ecomposition into correlations, clusters Choice o one-imensional projector P? integral mean => ANOVA ecomposition, inuces ecomposition o variance o evaluation at one ixe point => Anchor ANOVA (Eron, Stein, Wahba, Owen, Hickernell

12 Function ecompositions Example: ( V C C W C C W W W 3 C C W 3 C C C 3 C W W 3 C W C 3 W W C 3 W C C 3 W C W 3 W W W 3 Example: (3 V, ( ( (, (, 0 x x x x x x,, (, (, (, ( ( ( (,, ( 3,,3 3,3 3,3, x x x x x x x x x x x x x x x

13 Function ecompositions approximation by truncation ater q-orer terms q q q q 0 C C ( x W C3 C C3,..., x u ( xu u{,..., }, u q C C W W3 C W3 C W3 W W W W W C3 C C3 W W3 ( x, x, x3 0 ( x 3 x3 ( x ( x, x ( x, x ( x, x, x,, (,3 3,3 x3,,3( x3

14 Function ecompositions Fast ecay o series or even inite orer q <<? surely not in general, but: consier as input-output moel x,..., x ( x,..., x ( Correlate eects o the input variables? Many boy expansion o potential energy surace o molecular systems: mostly only two-, three- or ourboy potentials (i.e. q=4 or physical reasons Cluster expansions in statistical mechanics Statistics: secon orer, covariances but i.g. not more Data-mining: MARS, only up to q=5,..,7 or real ata

15 Truncation Truncation ater q terms introuces a moelling error The remaining subspaces nees to be initely represente => iscretization error Ater truncation ater q terms no more balancing o moelling error an subsequent iscretization error possible. Unnatural istinction between moelling error an subsequent iscretization error better: relate it somehow

16 Further ecomposition o W: Sparse gris ecompose subspace W urther tensor prouct an subsequent truncation => sparse gri representation Fourier series or polynomials (global => Korobov-spaces, hyperbolic cross approximation piecewise polynomials (local hierarchical basis, interpolets, wavelets, multilevel basis => sparse gri inite element spaces W l W l

17 re-invente several times: 957 Korobov, Babenko 963 Smolyak 97 Goron 980 Delvos, Posor 990 Zenger, G. 998 Stromberg, evore History o Sparse Gris hyperbolic cross points blening metho Boolean interpolation sparse gris hyperbolic wavelets application areas inclue: quarature (Novak, Ritter interpolation ata compression solution o PDEs integral equations eigenvalue problems

18 Example: Hierarchical basis l W l W l 3 parabola ( x ( x ( x in [-,] W 3 conventional coeicients no ecay rom level to level hierarchical coeicients ecay by ¼ rom level to level

19 Tensor prouct hierarchical basis Generalization to higher imension by tensor prouct l l l 3 l l l 3 Table o subspaces W l ecay in x- an y-irection by /4 l ecay in iagonal irection by /6 Iea: Omit points with small associate hierarchial coeicient values

20 Regular sparse gris

21 cost complexity (=,interior points N N / N / 4 N / N / N / N / log( N * N N * N / 4* N / 4...

22 ( x, x, ( x, x l l l l l, l W l l ( l l l l 3,, mix l l l 3 Accuracy: Table o subspaces Contribution truncate at level n ( x, x l,,, ( x, ( i, l i l i l i, x i, i l ( n ( 4 8, mix 3 l ( n ( 4 8, mix 3 l 3 ( n ( 4 8, mix 3 ( n ( n ( n3 ( , mix 3 urther summation results in SG n n 0 c n, mix

23 Properties o sparse gris Cost: Accuracy: Smoothness: : Sparse gris log N O( N instea o O( N O ( N O( N O( N log N x... x Space an seminorm: H mix, c Full gris O( N i H x i, c L -norm energy-norm breaks curse o imension o conventional ull gris at least to some extent Note: higher regularity in mixe erivative, r~

24 L For orthogonal wavelets an general stable multiscale systems we can even obtain Hint: estimate irectly or square error. Complexities with bounary terms: Cost: same orer but aitional actor o Accuracy same orer norm-base sparse gris O( N ( / log N Smoothness assumptions relate to variation o Hary an Krause Start multiscale series with constant then linear etc.

25 Representation cost per subspace norm-base sparse gris l beneit or accuracy 3 choice o best subspaces? => restricte global optimization problem, => local beneit /cost ratio V (, opt n L l n W l ( x ( x l l x x,.., x l l,.., l im( W l ( l l n l l( x W l l ( l n O ( l isoline regular sparse gri space

26 Energy-norm base sparse gris energy norm beneit or accuracy / ( : x x x j j E l l / ( j l E j j l j c b 4 ( / ( l l l l ( ( / j l j O l l isolines: energy-norm base sparse gri space n l W l V E n, ( 4 4 (4 log / 5 4 ( log / 5 n n j l j l Now beneit/cost ratio

27 Properties: complexities now inepenent o im( V What about the constants? c, E /3 n im( Vn ( e Thus: Energy-norm base sparse gris O(, E n n c c E n O( E 5 n E ( / ( ( 3 4 c E n, E n E c im( Vn with constant c c n O c (, E im( Vn E n n n

28 Further generalizations H s -norm base optimal sparse gri spaces (G., Knapek s(, s s s ( l, l isolines s 0 s l s More general subspace patterns Anisotropic general subset o subspaces sparse gris V ( W l l l l set o inices l

29 Further generalizations t l H, mix t, l tle tle Hmix ( I : H ( I H ( I k k H ( I : H ( I H k mix ( I spaces an regularity assumption (G., Knapek Mixture o the stanar Sobolev space H s ( I an the space o ominating mixe erivative H t mix ( I Norm equivalency (or stable ecompositions, wavelets H H ( I 0, s mix ( I t, 0 mix t l l l, H t l l mix l 0

30 Further generalizations Otimization allows again to etermine the best sparse T gri spaces T s s In { l N : l t l ( n t n} with approximations like in vv T n V n v li H T n s c ( lts n For a large range o smoothness parameters s,t,l any log-term is avoie in the cost an accuracy estimates But the constants may epen strongly on W l u H t, l mix l T l n BTW: The solution o Schröinger s equation lives in H ((, 3 mix H (( 3/ 4, 3 mix

31 Dimension-aapte sparse gris So ar: unction class known, an a-priori choice o best subspaces by optimization Size o beneit/cost ratio inicate i subspace is active => patterns or Now: or single given unction aaptively buil up a set o active inices Neee: local error inicator or subspace reinement strategy to buil new inex set global stopping criterion W l

32 Dimension-aaptive methos A proper aaptive algorithm then uses lower resolution in less important imensions an correlations an thus automatically etects important imensions important correlations between the imensions large reuction o cost i important imensions are ew (small eective imension, inite orer weight spaces, curse o imensionality broken But: no nee to know unction class a-priori (Heglan 0, Gerstner,G. 03, Garcke 04

33 Evolution o the algorithm: Example (Inex Sets inex sets: corresponing gris: Special ata structures or the bookkeeping o the ierent inex sets require. O(

34 Error Estimation ierential integral or inex l l l can be use as local error estimate problem: too early stopping (no saturation solution: consier also involve work n Wl an use as estimate l max w,( w with weight w[0,]. l n n l (Gerstner, G. 03

35 A simple example

36 High nominal but low eective imension Moel problem We expect a behavior o the metho as or a smooth q-imensional unction an cost with cost or one smooth q-imensional problem q u u u j x j g } {,..., ( (x 5,,here with (, ( ( x x B x g N q q O N q

37 High nominal but low eective imension Cost increase actors or a ixe error 0.000, right cloumn

38 Decay o importance o the imensions Weighte moel problem x x g( x ( ( (x j x j u{,..., } ju u q 0 3/ ( j with /, x0 0.8, w j w g(

39 Decay o importance o the imensions Weighte moel problem x x g( x ( ( (x j x j u{,..., } ju u q 0 3/ ( j with /, x0 0.8, w j w g(

40 Problem u h in on n (0,...,0 [0,] 0 [0,] PDE solver right han sie o inite orer q=4 (x u{,..., } u q ju g( x x g( x ( ( with /, x0 x j 0 0.8

41 Locally aaptive sparse gris or PDEs principle: reine near points with large hierarchical coeicient nonlinear N-term approximation or Besov spaces: same rates as isotropic nonlinear reinement schemes (wavelets, aaptive inite elements (Nitsche, Schwab line/ace singularities aligne with coorinate axes are cheap to resolve

42 D Navier-Stokes equation D mixing layer Chorin projection scheme, incompressible low Re=U/n = 6000 pertubations or initial conition evolution o vorticity an aaptive gris

43 3D Navier-Stokes equations 3D Mixing layer initial conitions analogous to D Re=4000 iscretization as beore number o DOF between... million three ierent isosuraces o vorticity

44 Implementation or higher imensional PDEs naive implementation o sparse gris or PDEs: ~ work count O( N ~ ~ storage N O( o O( N new ata structures an multigri algorithms, use o uniirectional principle, hash techniques separable, non-constant coeicient ~ unctions now: work count O( N ~ storage O( N elliptic PDEs possible with up to 0 imensions with homogeneous bc an prouct-type right han sie (Feuersänger.

45 Implementation or higher imensional PDEs implementation uses semi-orthogonal prewavelets instea o piecewise linear hat unctions orthogonality between levels simpliies mass matrix contributions an results in improve complexity w.r.t imension Full orthogonal wavelets ~ woul reuce complexity to just O( N but are more iicult to work with

46 Example: n orer PDE ( x k b k ( x ( x 0, x k ( x c( x ( x r( x, x [0,] Here: regular L L norm base sparse gri convergene rates with up to We see the inluence o the log( N ( -terms an energy norm approach an aaptivity is necessary

47 Caveat the regularity term,mix might cause problems an can postpone the onset o convergence Example : ( x x sin( k x D (,.. j j j Thus at most 5-8 imensions treatable in practice j j ( x x ( (k sin( k (, mix j k j j j x j

48 Example: Gaussian ( x x / ( exp T x x / Energy-norm base sparse gri in [ 5, 5 ] Rate or relative error in energy norm is asymptotically / n Necessary: n / ( n / 0

49 Tensor prouct sparse gris So ar: one-imensional omain, multiscale basis, -ol tensor prouct, proper truncation Now: E.g. two general omains,, each with its imension, an its smoothness s, s its isotropic multilevel basis (one level inex tensor prouct between the two omains an multiscale bases Mixe regularity H s mix s s ( : H ( H ( s

50 Examples: Tensor prouct sparse gris space time, 3,, parabolic problems space angle 3,, raiosity space parameters 3, 0 0 but smooth in parameter variables space stochastics 3, but analytic in stochastic variables Main result: curse o imension only w.r.t. the larger imension an/or the lower smoothness Time comes or ree, angle space comes or ree, parametrization/stochastics comes or ree, just space imension matters

51 Optimize general sparse gri space Multiscale analyses on i, i, approximation orer r i ( i ( i ( i V V V... L Complementary spaces: (i i ( i W li Vl Wl V i i Anisotropic sparse gri space: Parameter now allows to optimize with respect to imensions an smoothness with associate ( 0 i n V ( ( i l ( l / n l l W s, s W ( l, i l l

52 The sparse gri space Vn ~ egrees o reeom. s, s For a given H ( mix we have or the accuracy in V n n n L ( Properties (G.+Harbrecht 0 possesses with No log-terms in many situations nmin{ s n V J ns /, s } ( ( Analogous results by simple shit or other error q norms like, q ( than just or L ( H mix nmax{ n / /, n } n H H s, s mix s, s mix i 0 s s r i r, 0 / / i i s s / s / s

53 Approximation error an necessary regularity u n in V Space-time sparse gris u u 0 n c H ( L (0, T n How realistic are these regularity assumptions? - x t u is also neee or error estimates o conventional iscretization methos - classical regularity theory shows (Layzenskaja, Wloka u H ( H Space-time sparse gris possess the same approximation rate as conventional ull space-time gris but only the cost complexity o space problem ( => time coorinate comes or ree n ((0, T u H H ((0, T

54 Examples o space time sparse gris space imension, space-time sparse gri, Euler case space imension, space-time sparse gri, Cranck-Nicolson case, n=4,5: in each time slice there is a conventional ull gri

55 Instationary istribute control problems space imension, aaptive space time gris Problem: y y t t p p 0 3 p 0 in (0,], y in [0,, with homogeneous initial/en an bounary conitions Aaptivity with 5 reinement steps starting at level 3 t=0

56 Instationary istribute control problems control t=0 t=0.8 t=0. state t=.0

57 Instationary istribute control problems space imension=3, aaptive space time gris Problem: t y y 0 3 p 0 in (0,], 3 (, \ ( t p p y in [0,, with homogeneous initial/en an bounary conitions Aaptivity with 4 reinement steps starting at level 3 t=0, state variable, our isosuraces t=, control variable, our isosuraces,0 3

58 Stochastic an parametric PDEs Solutions ( x, x o stochastic/parametric PDEs A( x ( x, x r( x, x live on prouct o spatial omain ( x, x with an stochastic/parametric omain large or even ininity. Oten: Very high smoothness in -part with Here: especially weighte analyticity or the ierent coorinates ue to ecay in covariance Thereore, even ininite-imensional treatable inepenently o,,3 x become

59 Stochastic an parametric PDEs Sparse gris methos can be use or with its high imensionality! The stochastic part is smooth or even analytic sparse gris with spectral, polynomial bases to cope The stochastic coorinates are not equally important weights/ecay o the ierent coorinate irections relate to the eigenvalues o covariance o parameters, algebraic or exponential! Then, anisotropic sparse gris (with spectral, polynomial bases an imension-aaptive sparse gris are successully use The ecay kills the curse, the sparse gri approach then reuces the imension-epenency o the constant Moreover: The two-variate sparse gri prouct approach works ine between the spatial omain an the parametric/ stochastic omain.

60 Sparse gris an analytic unctions What may happen i the unction is too smooth? -D: L -orthogonal polynomial basis { k ( x i } k 0 -D: Prouct o polynomials k x k ( x k ( Here: the isotropic case: ( k... k ( x c Representation regular sparse gri cost: accuracy : k k 0 n( n / ull gri Anisotropic case: no curse, but still -epenent constants k k k ( x ( k x O ( n! same orer in n O( n n n O( n aitional log term O( k nn k

61 Weighte analytic approximation space or Let be given an orere real sequence a (a a a a... an a ixe base b A Stochastic an parametric PDEs 3 ki / ai i, a( L ( : b k k 0 Characterization o ( x k 0 k a -weighte analytic unctions ( x k k cb i i i k i / a i with C

62 Discrete anisotropic, regular gri subspace Inex set, brick-type with successively smaller size I ( n : { k 0 : ki / ai n or all i }, a Corresponing subspace V : span { k ( x, k I,, a, n n a ( } k 3 / a 3 k 3 k / a k k / a k

63 Discrete regular gri subspace Degrees o reeom With the summability conition we get, inepenently o, i a i A im( V exp(, a, n n A Accuracy: about linear in, mainly inepenent o V, a, n im( L ( V a ~ b b b ( n,, ( exp( n n ai n a i n ( n in n any or i case ( n : min /a i r i r / a { s na ( r r s 0 }

64 Stochastic an parametric PDEs With the sequence V V, a,0, a, V, a,... L ( an associate sequence o complimentary spaces V W V W, a, j, a. j a, j, a, j we get, together with the usual sequence o complementary spaces on, a two-variate sparse gri construction on, which is inepenent o the imension (even i. The sparse gri prouct approach works ine between the spatial an the parametric/ stochastic omains.

65 N H s Summary Classical approach:,.., 3 or 4 curse o imension an intractability c( N Stronger regularity/norms curse only wrt log-terms r ( / N s c( N (log( N H H or no curse at all r N s c( N H H but still not tractable, constant grows exponentially sr mix O( N r / r / sr H sr mix Lower eective imension an lower-im. maniols no curse ue to eective imension an constant grows exponentially only w.r.t. eective imension e r / N s c( N H H e sr,.., 0 to e,..., 00,.., 0

66 Literature M. Griebel, Sparse tensor prouct spaces or stochastic an parametric approximations, INS Report, in preparation, to appear 03. M. Griebel, H. Harbrecht, On the construction o sparse tensor prouct spaces, Math. Comp., 0. M. Griebel an H. Harbrecht. A note on the construction o L-ol sparse tensor prouct spaces. Constructive Approximation, 0. M. Griebel, H. Harbrecht, Approximation o two-variate unctions: singular value ecomposition versus sparse gris, IMA Journal o Numerical Analysis, 0. M. Griebel, S. Knapek, Optimize general sparse gri approximation spaces or operator equations, Math. Comp. 78, 3-57, 009. M. Griebel an D. Oeltz. A sparse gri space-time iscretization scheme or parabolic problems. Computing, 8(:-34, 007. D. Oeltz. Ein Raum-Zeit Dünngitterverahren zur Diskretisierung parabolischer Dierentialgleichungen. Dissertation, INS, Universität Bonn, 006. M. Griebel. Sparse gris an relate approximation schemes or higher imensional problems. In L. Paro, A. Pinkus, E. Suli, an M. To, eitors, Founations o Computational Mathematics (FoCM05, Santaner, pages Cambrige University Press, 006. H.-J. Bungartz an M. Griebel. Sparse gris. Acta Numerica, 3:-3, 004.