Sparse Grid Tutorial

Transcription

1 Sparse Grid Tutorial Jochen Garcke Centre for Mathematics and its Applications Mathematical Sciences Institute Australian National University Canberra ACT 0200, Australia August 12, Introduction The sparse grid method is a special discretization technique, which allows to cope with the curse of dimensionality of grid based approaches to some extent. It is based on a hierarchical basis [Fab09, Yse86, Yse92], a representation of a discrete function space which is equivalent to the conventional nodal basis, and a sparse tensor product construction. The method was originally developed for the solution of partial differential equations [Zen91, Gri91, Bun92, Bal94, Ach03] and is now also successfully used for integral equations [FHP96, GOS99], interpolation and approximation [Bas85, Tem89, SS99, GK00, Kna00, KW05b]. Furthermore there is work on stochastic differential equations [ST03a, ST03b], differential forms in the context of the Maxwell-equation [GH03] and with a wavelet-based sparse grid discretization parabolic problems are treated in [vps04]. Besides Galerkin finite element approaches there are also finite differences on sparse grids [Gri98, Sch99, Kos02, GK03] and finite volume approaches [Hem95]. Besides working directly in the hierarchical basis a sparse grid representation of a function can also be computed using the combination technique [GSZ92], where a certain sequence of partial functions is linearly combined. Applications include eigenvalue problems [GG00], numerical integration [GG98, BD03, GG03], parabolic equations for options pricing [Rei04], machine learning [GGT01, GG02, Gar04, GG05] and data analysis [LNSH05] and it is used for solving the stochastic master equation applied to gene regulatory networks [HBS + 06]. The underlying idea of sparse grids can be traced back to the Russian mathematician Smolyak [Smo63], who used it for numerical integration. The concept is also closely related to hyperbolic crosses [Bab60, Tem89, Tem93a, Tem93b], boolean methods [Del82, DS89], discrete blending methods [BDJ92] and splitting extrapolation methods [LLS95]. 1

2 For the representation of a function f defined over a d-dimensional domain the sparse grid approach employs O(h 1 n log(h 1 n ) d 1 ) grid points in the discretization process, where h n := 2 n denotes the mesh size. It can be shown that the order of approximation to describe a function f, under certain smoothness conditions, is O(h 2 n log(h 1 n ) d 1 ). This is in contrast to conventional grid methods, which need O(h d n ) for an accuracy of O(h2 n ). Since the curse of dimensionality of full grid method arises for sparse grids at this much smaller extent they can be used for higher dimensional problems. For ease of presentation we will consider the domain Ω = [0, 1] d here and in the following. This situation can be achieved for bounded rectangular domains by a proper rescaling. 2 Sparse grids We introduce some notation while describing the conventional case of a piecewise linear finite element basis. Let l = (l 1,..., l d ) d denote a multi-index. We define the anisotropic grid Ω l on Ω with mesh size h l := (h l1,..., h ld ) := (2 l1,..., 2 l d ), it has different, but equidistant mesh sizes in each coordinate direction t. This way the grid Ω l consists of the points x l,j := (x l1,j 1,..., x ld,j d ), (1) with x lt,j t := j t h lt = j t 2 lt and j t = 0,..., 2 lt. For a grid Ω l we define an associated space V l of piecewise d-linear functions V l := span{φ l,j j t = 0,..., 2 lt, t = 1,..., d}, (2) which is spanned by the usual basis of d-dimensional piecewise d-linear hat functions d φ l,j (x) := φ lt,j t (x t ). (3) t=1 The one-dimensional functions φ l,j (x) with support [x l,j h l, x l,j + h l ] [0, 1] = [(j 1)h l, (j + 1)h l ] [0, 1] are defined by: { 1 x/hl j, x [(j 1)h φ l,j (x) = l, (j + 1)h l ] [0, 1]; 0, otherwise. (4) See Figure 1(a) for a one-dimensional example and Figure 2 for a two-dimensional basis function. 2.1 Hierarchical subspace-splitting Till now and in the following the multi-index l d denotes the level, i.e. the discretization resolution, of a grid Ω l, a space V l or a function f l, whereas the multi-index j d gives the position of a grid point x l,j or the corresponding basis function φ l,j ( ). 2

3 (a) Nodal basis für V 3 (b) Hierarchical basis für V 3 Figure 1: Nodal and hierarchical basis of level 3 Figure 2: Basis function φ 1,1 on grid Ω 2,1. 3

4 We now define a hierarchical difference space W l via W l := V l \ d V l et, (5) t=1 where e t is the t-th unit vector. In other words, W l consists of all φ k,j V l which are not included in any of the spaces V k smaller 1 than V l. To complete the definition, we formally set V l := 0, if l t = 1 for at least one t {1,..., d}. As can easily be seen from (2) and (5), the definition of the index set { } B l := j d j t = 1,..., 2 lt 1, j t odd, t = 1,..., d, if l t > 0, (6) j t = 0, 1, t = 1,..., d, if l t = 0 leads to W l = span{φ l,j j B l }. (7) These hierarchical difference spaces now allow us the definition of a multilevel subspace decomposition. We can write V n := V n as a direct sum of subspaces V n := n n W l = W l. (8) l 1=0 l d =0 l n Here and in the following refers to the element-wise relation. l := max 1 t d l t and l 1 := d t=1 l t are the discrete L - and the discrete L 1 -norm of l, respectively. The family of functions {φ l,j j B l } n l=0 (9) is just the hierarchical basis [Fab09, Yse86, Yse92] of V n, which generalizes the one-dimensional hierarchical basis [Fab09], see Figure 1(b), to the d-dimensional case with a tensor product ansatz. Observe that the supports of the basis functions φ l,j (x), which span W l, are disjunct. See Figure 3 for a representation of the supports of the basis functions of the difference spaces W l1,l 2 forming V 3,3. Now each function f V n can be represented as f(x) = α l,j φ l,j (x), (10) j B l l n where α l,j are the coefficients of the representation in the hierarchical tensor product basis. The number of basis functions, which describe a f V n in nodal or hierarchical basis is (2 n + 1) d. For example a resolution of 17 points in each dimensions, i.e. n = 4, for a ten-dimensional problem therefore needs coefficients, we encounter the curse of dimensionality. 1 We call a discrete space V k smaller than a space V l if tk t l t and t : k t < l t. In the same way a grid Ω k is smaller than a grid Ω l. 4

5 W 0,0 W 1,0 W 2,0 W 3,0 W 0,1 W 1,1 W 2,1 W 3,1 W 0,2 W 1,2 W 2,2 W 3,2 W 0,3 W 1,3 W 2,3 W 3,3 Figure 3: Supports of the basisfunctions of the hierarchical subspaces W l of the space V 3 Note, that for the spaces V l the following decomposition holds V l := l 1 k 1=0 l d k d =0 W k = k l W k. 2.2 Properties of the hierarchical subspaces Now consider the d-linear interpolation of a function f V by a f n V n, i.e. a representation as in (10). First we look at the linear interpolation in one dimension, for the hierarchical coefficients α l,j, l 1, holds α l,j = f(x l,j ) f(x l,j h) + f(x l,j + h) 2 5 = f(x l,j ) f(x l,j 1) + f(x l,j+1 ). 2

6 φ φ φ 3,1 2,1 φ 1,1 3,3 φ 3,5 φ 2,3 φ 3,7 Figure 4: Interpolation with the hierarchical basis This and Figure 4 illustrate why the α l,j are also called hierarchical surplus, they specify what has to be added to the hierarchical representation from level l 1 to obtain the one of level l. We can rewrite this in the following operator form [ α l,j = ] f 2 2 and with that we generalize to the d-dimensional hierarchization operator as follows ( d [ α l,j = ] ) f (11) 2 2 t=1 Note that the coefficients for the basis functions associated to the boundary are just α 0,j = f(x 0,j ), j = 0, 1. Now let us define the so-called Sobolev-space with dominating mixed derivative Hmix 2 in which we then will show approximation properties of the hierarchical basis. We define the norm as f 2 H = k 1 2 mix s x k f, and the space Hmix 2 in the usual way: 0 k s l,j l t,j t H s mix := {f : Ω : f 2 H s mix < } 2 Furthermore we define the semi-norm f H 2 mix := f H 2 mix f k H := k 1 2 mix x k f, 2 6

7 Note that the continuous function spaces H s mix, like the discrete spaces V l, have a tensor product structure [Wah90, Hoc99, GK00, HKZ00] and can be represented as a tensor product of one dimensional spaces: H s mix = H s H s. We now look at the properties of the hierarchical representation of a function f, especially at the size of the hierarchical surplusses. We recite the following proofs from [Bun92, Bun98, BG99, BG04], see these references for more details on the following and results in other norms like or E. For ease of presentation we assume f H0,mix 2 ( Ω), i.e. zero boundary values, and l > 0 to avoid the special treatment of level 0, i.e. the boundary functions in the hierarchical representation. Straightforward calculation shows Lemma 1. For any piecewise d-linear basis function φ l,j holds φ l,j 2 C(d) 2 l 1/2. Lemma 2. For any hierarchical coefficient α l,j of f H 2 0,mix ( Ω) it holds α l,j = d t=1 h t 2 Proof. In one dimension partial integration provides Ω φ l,j 2 f(x)dx x 2 = = xl,j +h x l,j h Ω φ l,j D 2 f(x)dx. (12) φ l,j 2 f(x)dx x 2 [ φ l,j f(x)dx x ] xl,j +h x l,j h xl,j 1 = x l,j h h f(x)dx + x xl,j +h φ l,j x l,j h x f(x)dx x xl,j +h 1 x l,j h f(x)dx x = 1 h (f(x l,j h) 2f(x l,j ) + f(x l,j + h)) = 2 h α l,j The d-dimensional result is achieved via the tensor product formulation (11). Lemma 3. Let f H0,mix 2 ( Ω) be as in hierarchical representation above, it holds α l,j C(d) 2 (3/2) l 1. H 2 mix f supp(φl,j ) 7

8 Proof. d α l,j = h t 2 t=1 Ω φ l,j D 2 f(x)dx d h t 2 φ l,j 2 D 2 f supp(φl,j ) t=1 C(d) 2 (3/2) l 1 2 f supp(φl,j ) H 2 mix Lemma 4. f H 2 0,mix ( Ω) is as above in hierarchical representation. For its components f l W l holds f l 2 C(d) 2 2 l 1 f H 2 mix. (13) Proof. Since the supports of all φ l,j are mutually disjoint we can write f l 2 2 = α l,j φ l,j (x) j B l With Lemma 3 and 1 it now follows f l 2 2 C(d) 2 3 l 1 j B l 2 2 = j B l α l,j 2 φ l,j 2 2 f supp(φl,j ) 2 Hmix 2 C(d) 2 l 1 C(d) 2 4 l 1 f 2 H 2 mix which completes the proof. 2.3 Sparse grids Motivated by the relation (13) of the importance of the hierachical components f l Zenger [Zen91] and Griebel [Gri91] introduce the so-called sparse grids, where hierarchical basis functions with a small support, and therefore a small part in the function representation, are not included in the discrete space of level n anymore. Formally we define the sparse grid function space Vn s V n as Vn s := W l. (14) l 1 n We replace in the definition (8) of V n in terms of hierarchical subspaces the condition l n with l 1 n. In Figure 3 the used subspaces are given in black, the difference spaces W l which are not used anymore, in comparison to (8), are given in grey. Every f Vn s can now be represented, analogue to (10), as fn s (x) = α l,j φ l,j (x). (15) j B l l 1 n 8

9 Figure 5: Two-dimensional sparse grid (left) and three-dimensional sparse grid (right) of level n = 5 each. The resulting grids corresponding to the approximation space Vn s are called sparse grids. Note that sparse grids were introduced in [Zen91, Gri91] using the definition V0,n s := W l. (16) l 1 n+d 1 This definition is useful for the numerical treatment of partial differential equations with Dirichlet boundary conditions where no degrees of freedom exist on the boundary. Using V0,n s the level of refinement n in the sparse grid corresponds to the full grid case again. Examples in two and three dimensions are given in Figure 5. Sparse grids are studied in detail in [Bun92, Bun98, BG99, Kna00, BG04] besides others. The following results hold for both definitions Vn s and V 0,n s. The proofs are somewhat easier without basis functions on the boundary, so we use Vn,0 s in the following, like it is done in the referenced source publications. First we look at the size of the sparse grid space. Lemma 5. The dimension of the sparse grid space V0,n s, i.e. the number of inner grid points, is given by V0,n s = O(h 1 n log(h 1 n ) d 1 ) (17) Proof. We follow [BG04] and use in first part the definition (16), the size of a hierarchical subspace W l = 2 l 1 1 and that there are ( i 1 d 1) possibilities to 9

10 represent i as a sum of d natural numbers. V0,n s = W l = 2 l 1 1 = l 1 n+d 1 l 1 n+d 1 = = n+d 1 i=d n+d 1 i=d n 1 2 i i=0 2 i d l 1=i 1 ( ) i 1 2 i d d 1 ( ) i + d 1 d 1 We now represent the summand as the (d 1)-derivation of a function evaluated at x = 2 = = n 1 ( ) i + d 1 2 i d 1 i=0 n 1 1 (d 1)! 1 (d 1)! i=0 ( x i+d 1 ) (d 1) x=2 (x d 1 1 xn 1 x ) (d 1) 1 d 1 ( d 1 = (d 1)! i i=0 d 1 ( n + d 1 = ( 1) d + 2 n i i=0 x=2 ) (x d 1 x n+d 1) (i) ) ( 2) d 1 i. The summand for i = d 1 is the largest one and it holds 2 n (n + d 1)! (d + 1)!n! ( ) n = 2 n d 1 (d 1)! + O(nd 2 ) ( ) (d 1 i) 1 1 x x=2 which gives a total order of O(2 n n d 1 ) or in other notation, with h n = 2 n, of O(h 1 n log(h 1 n ) d 1 ). This is far less than the size of the corresponding full grid space V n = O(h d n ) = O(2 d n ) and allows the treatment of higher dimensional problems. We now look at approximation properties of sparse grids. For the proof we again follow [BG04] and first look at the error for the interpolation of a function f H0,mix 2 by f 0,n V0,n s which can be written in regard to the partial functions from the hierarchical subspaces as f f0,n s = f l f l = f l. l l 1 n+d 1 l 1>n+d 1 10

11 For any norm now holds f f s 0,n l 1>n+d 1 f l. (18) We need the following lemma to estimate the interpolation error Lemma 6. For s it holds ( ) i + n + d 1 2 s l 1 = 2 s n 2 s d 2 s i d 1 l 1>n+d 1 i=0 ( ) n 2 s n 2 s d d 1 2 (d 1)! + O(nd 2 ), Proof. The steps of the proof are similar to the ones in the previous lemma, first we get Since l 1>n+d 1 2 s l 1 = i=n+d 2 s i l 1=i i=0 1 ( ) i 1 = 2 s i d 1 i=n+d ( ) i + n + d 1 = 2 s n 2 s d 2 s i. d 1 ( ) ( ) (d 1) i + n + d 1 x i = x n x i+n+d 1 d 1 (d 1)! i=0 i=0 ( ) (d 1) = x n (d 1)! x n+d x = x n (d 1)! d 1 ( d 1 k k=0 d 1 ( n + d 1 = k k=0 it follows with x = 2 s the relation ) ) (x n+d 1) (k) ( ) d 1 k x 1 x ( ) i + n + d 1 d 1 ( ) n + d 1 2 s i 2 d 1 k i=0 which finishes the proof. k=0 ( 1 ) (d 1 k) 1 x 1 1 x, ( ) n d 1 = 2 (d 1)! + O(nd 2 ) 11

12 Theorem 1. For the interpolation error of a function f H0,mix 2 in the sparse grid space V0,n s holds Proof. f f s n 2 l 1>n+d 1 which gives the wanted relation. f f s n 2 = O(h 2 n log(h 1 n ) d 1 ). (19) f l 2 C(d) 2 2 l 1 f H 2 mix ( ) n C(d) 2 2 n d 1 f H 2 mix (d 1)! + O(nd 2 ), Note that same results holds in the maximum-norm as well: for f H 2 0,mix. f f s n = O(h 2 n log(h 1 n ) d 1 ) 2.4 Sparse grid combination technique The so-called combination technique [GSZ92], which is based on multi-variate extrapolation [BGR94], is another method to achieve a function representation on a sparse grid. The function is discretized on a certain sequence of grids using a nodal discretization.a linear combination of these partial functions then gives the sparse grid representation. This approach can have numerical advantages over working directly in the hierarchical basis, where e.g. the stiffness matrix is not sparse and the on-the-fly computation of the matrix-vector-product, which complexity scales better, is challenging in the implementation [Ach03, Bal94, Bun98]. In particular, we discretize the function f on a certain sequence of anisotropic grids Ω l = Ω l1,...,l d with uniform mesh sizes h t = 2 lt in the t-th coordinate direction. These grids possess in general different mesh sizes for the different coordinate directions. To be precise, we consider all grids Ω l with l 1 := l l d = n q, q = 0,.., d 1, l t 0. Note that in the original [GSZ92] and other papers as well, a slightly different definition was used, again due to Dirichlet boundary conditions, l 1 := l l d = n + (d 1) q, q = 0,.., d 1, l t > 0. The grids employed by the combination technique of level 4 in two dimensions are shown in Figure 6. A finite element approach with piecewise d-linear functions φ l,j (x) on each grid Ω l now gives the representation in the nodal basis f l (x) = 2 l 1 j 1= l d j d =0 α l,j φ l,j (x). 12

13 Ω 4,0 Ω 3,1 Ω 2,2 Ω 1,3 Ω 0,4 Ω 3,0 Ω 2,1 Ω 1,2 Ω 0,3 = fn c = f l1,l 2 l 1+l 2=n l 1+l 2=n 1 f l1,l 2 Ω s 4 Figure 6: Combination technique with level n = 4 in two dimensions Finally, we linearly combine the discrete partial functions f l (x) from the different grids Ω l according to the combination formula d 1 ( ) d 1 fn(x) c := ( 1) q f l (x). (20) q q=0 l 1=n q The resulting function fn c lives in the sparse grid space V n s, the combined interpolant is identically with the hierarchical sparse grid interpolant fn s [GSZ92]. This can be seen by rewriting each f l in their hierarchical representation (10) and some straightforward calculation using the telescope sum property, i.e. the hierarchical functions get added and subtracted. We write it exemplary in the two dimensional case, using ˆf l1+l 2 W l1,l 2 instead of all the basis functions of W l1,l 2 for ease of presentation fn c = f l1,l 2 l 1+l 2=n = l 1 n k 1 l 1 k 2 n l 1 = = = k 1 l 1 ˆfk1,1 + k 1 l 1 ˆfk1,1 + k 1+t n ˆf k1,t l 1+l 2=n 1 ˆfk1,k 2 l 1 n 1 k 1 l 1 f l1,l 2 ˆfk1,k 2 l 1 n 1 k 1 l 1 k 2 n 1 l 1 k 2 n l 1 ˆfk1,n l 1 l 1 n 1 k 1 l 1 with t := n + 1 l 1. This is exactly (15). ˆfk1,k 2 k 2 n 1 l 1 ˆfk1,k 2 13

14 Note that the solution obtained with the combination technique fn c for the numerical treatment of partial differential equations is in general not the sparse grid solution fn s. However, the approximation property is of the same order as long a series expansion of the error f f l = d i=1 j 1,...,j m 1,...,d c j1,...,j m (h j1,..., h jm ) h p j 1... h p j m, (21) with bounded c j1,...,j m (h j1,..., h jm ) κ exists, see [GSZ92]. Its existence was shown for model-problems in [BGRZ94]. Let us again consider the two dimensional case and consider the error of the combined solution f fn c following [GSZ92]. We have f fn c = f f l1,l 2 + = l 1+l 2=n l 1+l 2=n l 1+l 2=n 1 (f f l1,l 2 ) l 1+l 2=n 1 f l1,l 2 (f f l1,l 2 ). Plugging in the error expansion (21) leads to f fn c ( = c1 (h l1 ) h 2 l 1 + c 2 (h l2 ) h 2 l 2 + c 1,2 (h l1, h l2 ) h 2 l 1 h 2 ) l 2 = l 1+l 2=n l 1+l 2=n 1 ( ( c1 (h l1 ) h 2 l 1 + c 2 (h l2 ) h 2 l 2 + c 1,2 (h l1, h l2 ) h 2 l 1 h 2 l 2 ) c 1 (h n ) + c 2 (h n ) + l 1+l 2=n c 1,2 (h l1, h l2 ) 4 l 1+l 2=n 1 c 1,2 (h l1, h l2 ) And we get the estimation, using c i κ, f fn c 2κ h2 n + c 1,2 (h l1, h l2 ) 4 c 1,2 (h l1, h l2 ) h2 n l 1+l 2=n l 1+l 2=n 1 2κ h 2 n + c 1,2 (h l1, h l2 ) h 2 n + 4 c 1,2 (h l1, h l2 ) h 2 n l 1+l 2=n 2κ h 2 n + κ nh2 n + 4κ(n 1)h2 n = κ h 2 n(5n 2) = κ h 2 n(5 log(h 1 n ) 2) = O(h 2 n log(h 1 n )) l 1+l 2=n 1 ) h 2 n Observe that terms are canceled for h li with l i n and the accumulated h 2 l 1 h 2 l 2 result in log(h 1 n )-term. The approximation order O(h 2 n log(h 1 n )) is just as in Theorem 1. See [GSZ92, PZ99, Rei04] for results in higher dimension. The combination technique was used in fluid dynamics [GHSZ93, GH95, GT95, Hub96, Kra02], for the Lamé-equation [Heu97], for eigenvalue-problems [Gar98, GG00], singular perturbation problems [NH00], machine learning and 14

15 data analysis [GGT01, GG02, Gar04, GG05, LNSH05] and parabolic equations for option pricing [Rei04]. There are close connections to so-called boolean [Del82, DS89] and discrete blending methods [BDJ92], as well as the splitting extrapolation-method [LLS95]. Furthermore, there are relations between the sparse grid combination technique and additive models like e.g. ANOVA [Wah90] or MARS [Fri91]. A Sparse grids in python We give the listing of some python code for a sparse grid without functions on the boundary, i.e. according to formula (16). If you are interested in the source files, write to jochen.garcke@anu.edu.au. A MATLAB implementation of sparse grids [KW05a, Kli06] can be found here import math Listing 1: sparse grid representation class g r i d P o i n t : def i n i t ( s e l f, index=none, domain=none ) : s e l f. hv = [ ] # h i e r a r c h i c a l v a l u e s e l f. fv = [ ] # f u n c t i o n v a l u e i f index i s None : s e l f. pos = [ ] # p o s i t i o n o f g r i d p o i n t else : s e l f. pos = s e l f. p o i n t P o s i t i o n ( index, domain ) def p o i n t P o s i t i o n ( s e l f, index, domain=none ) : coord = l i s t ( ) i f domain i s None : for i in range ( l e n ( index ) ) : coord. append ( index [ i ] [ 1 ] / 2. index [ i ] [ 0 ] ) else : for i in range ( l e n ( index ) ) : coord. append ( ( domain [ i ] [ 1 ] domain [ i ] [ 0 ] ) \ index [ i ] [ 1 ] / 2. index [ i ] [ 0 ] + domain [ i ] [ 0 ] ) return coord def p r i n t P o i n t ( s e l f ) : i f s e l f. pos i s [ ] : pass else : out = for i in range ( l e n ( s e l f. pos ) ) : out += s t r ( s e l f. pos [ i ] ) + \ t 15

16 print out class sparsegrid : def i n i t ( s e l f, dim=1, l e v e l =1): s e l f. dim = dim s e l f. l e v e l = l e v e l s e l f. gp = {} s e l f. i n d i c e s = [ ] s e l f. domain = ( ( 0. 0, 1. 0 ), ) dim s e l f. a c t i o n = ( ) def printgrid ( s e l f ) : print s e l f. hspace def evalaction ( s e l f ) : b a s i s = ( s e l f. evalperdim [ 0 ] [ s e l f. hspace [ 0 ] 1 ] [ 0 ], ) value = s e l f. evalperdim [ 0 ] [ s e l f. hspace [ 0 ] 1 ] [ 1 ] # compute index o f non zero b a s i s f u n c t i o n in hspace and i t s v a l u e on for i in range ( 1, s e l f. dim ) : value = s e l f. evalperdim [ i ] [ s e l f. hspace [ i ] 1 ] [ 1 ] b a s i s += ( s e l f. evalperdim [ i ] [ s e l f. hspace [ i ] 1 ] [ 0 ], ) s e l f. value += s e l f. gp [ b a s i s ]. hv value def evalfunct ( s e l f, x ) : s e l f. value = 0. 0 s e l f. evalperdim = [ ] for i in range ( s e l f. dim ) : s e l f. evalperdim. append ( [ ] ) for j in range ( 1, s e l f. l e v e l +1): # which b a s i s i s unzero on x f o r dim i and l e v e l j pos = ( x [ i ] s e l f. domain [ i ] [ 0 ] ) / ( s e l f. domain [ i ] [ 1 ] \ s e l f. domain [ i ] [ 0 ] ) b a s i s = i n t ( math. c e i l ( pos 2 ( j 1)) 2 1) i f b a s i s == 1: b a s i s = 1 s e l f. evalperdim [ i ]. append ( [ ( j, b a s i s ) ] ) else : s e l f. evalperdim [ i ]. append ( [ ( j, b a s i s ) ] ) # v a l u e o f t h i s b a s i s f u n c t i o n on x [ i ] s e l f. evalperdim [ i ] [ j 1]. append ( evalbasis1d ( x [ i ], \ s e l f. evalperdim [ i ] [ j 1 ] [ 0 ], s e l f. domain [ i ] ) ) s e l f. a c t i o n = s e l f. evalaction s e l f. loophierspaces ( ) return s e l f. value # go through the h i e r a r c h i c a l subspaces o f s e l f. l e v e l 16

17 def loophierspaces ( s e l f ) : for i in range ( 1, s e l f. l e v e l +1): s e l f. hspace = [ i ] s e l f. loophierspacesrec ( s e l f. dim 1, s e l f. l e v e l (i 1)) def loophierspacesrec ( s e l f, dim, l e v e l ) : i f dim > 1 : for i in range ( 1, l e v e l +1): s e l f. hspace. append ( i ) s e l f. loophierspacesrec ( dim 1, l e v e l (i 1)) s e l f. hspace. pop ( ) else : for i in range ( 1, l e v e l +1): s e l f. hspace. append ( i ) s e l f. a c t i o n ( ) s e l f. hspace. pop ( ) # f i l l s e l f. gp with the p o i n t s f o r the i n d i c e s generated beforehand def g e n e r a t e P o i n t s ( s e l f ) : s e l f. i n d i c e s = s e l f. generatepointsrec ( s e l f. dim, s e l f. l e v e l ) for i in range ( l e n ( s e l f. i n d i c e s ) ) : s e l f. gp [ s e l f. i n d i c e s [ i ] ] = g r i d P o i n t ( s e l f. i n d i c e s [ i ], s e l f. domain ) def generatepointsrec ( s e l f, dim, l e v e l, a k t l e v e l=none ) : b a s i s a k t = l i s t ( ) i f a k t l e v e l == None : a k t l e v e l = 1 for i in range ( 1, 2 ( a k t l e v e l )+1,2): b a s i s a k t. append ( ( a k t l e v e l, i ) ) i f dim == 1 and a k t l e v e l == l e v e l : return b a s i s a k t e l i f dim == 1 : b a s i s a k t += s e l f. generatepointsrec ( dim, l e v e l, a k t l e v e l +1) return b a s i s a k t e l i f a k t l e v e l == l e v e l : return c r o s s ( b a s i s a k t, \ s e l f. generatepointsrec ( dim 1, l e v e l a k t l e v e l +1)) else : return c r o s s ( b a s i s a k t, s e l f. generatepointsrec ( dim 1,\ l e v e l a k t l e v e l +1)) \ + s e l f. generatepointsrec ( dim, l e v e l, a k t l e v e l +1) # conversion from nodal to h i e r a r c h i c a l b a s i s in one dimension def nodal2hier1d ( s e l f, node, i, j, dim ) : l e f t = ( i 1, j /2) r i g h t = ( i 1, j /2+1) 17

18 while l e f t [1]%2 == 0 and l e f t [ 0 ] > 0 : l e f t = ( l e f t [0] 1, l e f t [ 1 ] / 2 ) while r i g h t [1]%2 == 0 and r i g h t [ 0 ] > 0 : r i g h t = ( r i g h t [0] 1, r i g h t [ 1 ] / 2 ) i f i s i n s t a n c e ( node [ 0 ], t u p l e ) : precurdim = node [ 0 : dim ] postcurdim = node [ dim : l e n ( node )+1] index = precurdim + ( ( i, j ), ) + postcurdim l e f t = precurdim + ( l e f t, ) + postcurdim r i g h t = precurdim + ( r i g h t, ) + postcurdim else : i f dim == 0 : index = ( ( i, j ), ) + ( node, ) l e f t = ( l e f t, ) + ( node, ) r i g h t = ( r i g h t, ) + ( node, ) else : #dim w i l l be 1 o t h e r w i s e in t h i s case index = ( node, ) + ( ( i, j ), ) l e f t = ( node, ) + ( l e f t, ) r i g h t = ( node, ) + ( r i g h t, ) i f l e f t [ dim ] [ 0 ] == 0 : i f r i g h t [ dim ] [ 0 ]!= 0 : s e l f. gp [ index ]. hv = 0.5 s e l f. gp [ r i g h t ]. hv e l i f r i g h t [ dim ] [ 0 ] == 0 : s e l f. gp [ index ]. hv = 0.5 s e l f. gp [ l e f t ]. hv else : s e l f. gp [ index ]. hv = 0. 5 ( s e l f. gp [ l e f t ]. hv + s e l f. gp [ r i g h t ]. hv ) # conversion from nodal to h i e r a r c h i c a l b a s i s def nodal2hier ( s e l f ) : for i in range ( l e n ( s e l f. i n d i c e s ) ) : s e l f. gp [ s e l f. i n d i c e s [ i ] ]. hv = s e l f. gp [ s e l f. i n d i c e s [ i ] ]. fv for d in range ( 0, s e l f. dim ) : for i in range ( s e l f. l e v e l,0, 1): i n d i c e s = s e l f. generatepointsrec ( s e l f. dim 1, s e l f. l e v e l i +1) for j in range (1,2 i +1,2): for k in range ( l e n ( i n d i c e s ) ) : s e l f. nodal2hier1d ( i n d i c e s [ k ], i, j, d ) # compute cross product o f args def c r o s s ( args ) : ans = [ [ ] ] for arg in args : ans = [ x+[y ] for x in ans for y in arg ] ans2 = [ ] for i in range ( l e n ( ans ) ) : i f i s i n s t a n c e ( ans [ i ] [ 1 ] [ 0 ], t u p l e ) : 18

19 dummy = ( ans [ i ] [ 1 ] [ 0 ], ) for j in range ( 1, l e n ( ans [ i ] [ 1 ] ) ) : dummy = (dummy [ 0 : l e n (dummy) ] ) + ( ans [ i ] [ 1 ] [ j ], ) ans2. append ( ( ans [ i ] [ 0 ], ) + dummy) else : ans2. append ( ( ans [ i ] [ 0 ], ans [ i ] [ 1 ] ) ) return ans2 # e v a l u a t i o n o f the b a s i s f u n c t i o n s in one dimension def evalbasis1d ( x, basis, i n t e r v a l=none ) : i f i n t e r v a l i s None : return 1. abs ( x 2 b a s i s [0] b a s i s [ 1 ] ) else : pos = ( x i n t e r v a l [ 0 ] ) / ( i n t e r v a l [1] i n t e r v a l [ 0 ] ) return 1. abs ( pos 2 b a s i s [0] b a s i s [ 1 ] ) Listing 2: unit test for listing 1 # d e f i n e the u n i t t e s t s import pysg import u n i t t e s t import math class t e s t F u n c t e s t ( u n i t t e s t. TestCase ) : def testsgnobound ( s e l f ) : sg = pysg. sparsegrid ( 3, 3 ) sg. g e n e r a t e P o i n t s ( ) s e l f. a s s e r t E q u a l ( l e n ( sg. i n d i c e s ), 3 1 ) for i in range ( l e n ( sg. i n d i c e s ) ) : sum = 1. 0 pos = sg. gp [ sg. i n d i c e s [ i ] ]. pos for j in range ( l e n ( pos ) ) : sum = 4. pos [ j ] (1.0 pos [ j ] ) sg. gp [ sg. i n d i c e s [ i ] ]. fv = sum sg. nodal2hier ( ) for i in range ( l e n ( sg. i n d i c e s ) ) : s e l f. a s s e r t E q u a l ( sg. gp [ sg. i n d i c e s [ i ] ]. fv, \ sg. evalfunct ( sg. gp [ sg. i n d i c e s [ i ] ]. pos ) ) # t e s t i n g i f name == m a i n : u n i t t e s t. main ( ) References [Ach03] Achatz, S. Higher Order Sparse Grid Methods for Elliptic Partial Differential Equations with Variable Coefficients. Computing, 19

20 71(1):1 15, [Bab60] [Bal94] [Bas85] [BD03] [BDJ92] [BG99] [BG04] [BGR94] Babenko, KI. Approximation of periodic functions of many variables by trigonometric polynomials. Dokl. Akad. Nauk SSSR, 132: , Russian, Engl. Transl.: Soviet Math. Dokl. 1: , Balder, R. Adaptive Verfahren für elliptische und parabolische Differentialgleichungen auf dünnen Gittern. Dissertation, Technische Universität München, Baszenski, G. N th order polynomial spline blending. In Schempp, W and Zeller, K, editors, Multivariate Approximation Theory III, ISNM 75, pages Birkhäuser, Basel, Bungartz, HJ and Dirnstorfer, S. Multivariate Quadrature on Adaptive Sparse Grids. Computing, 71:89 114, August Baszenski, G, Delvos, FJ, and Jester, S. Blending approximations with sine functions. In Braess, D, editor, Numerical Methods in Approximation Theory IX, ISNM 105, pages Birkhäuser, Basel, Bungartz, HJ and Griebel, M. A Note on the Complexity of Solving Poisson s Equation for Spaces of Bounded Mixed Derivatives. J. Complexity, 15: , Also as Report No 524, SFB 256, Univ. Bonn, Bungartz, HJ and Griebel, M. Sparse Grids. Acta Numerica, 13: , Bungartz, HJ, Griebel, M, and Rüde, U. Extrapolation, combination, and sparse grid techniques for elliptic boundary value problems. Comput. Methods Appl. Mech. Eng., 116: , Also in C. Bernardi and Y. Maday, Editoren, International conference on spectral and high order methods, ICOSAHOM 92. Elsevier, [BGRZ94] Bungartz, HJ, Griebel, M, Röschke, D, and Zenger, C. Pointwise convergence of the combination technique for the Laplace equation. East-West J. Numer. Math., 2:21 45, [Bun92] Bungartz, HJ. Dünne Gitter und deren Anwendung bei der adaptiven Lösung der dreidimensionalen Poisson-Gleichung. Dissertation, Institut für Informatik, Technische Universität München, [Bun98] Bungartz, HJ. Finite Elements of Higher Order on Sparse Grids. Habilitation, Institut für Informatik, Technische Universität München and Shaker Verlag, Aachen,

21 [Del82] [DS89] Delvos, FJ. d-variate Boolean Interpolation. J. Approx. Theory, 34:99 114, Delvos, FJ and Schempp, W. Boolean Methods in Interpolation and Approximation. Pitman Research Notes in Mathematics Series 230. Longman Scientific & Technical, Harlow, [Fab09] Faber, G. Über stetige Funktionen. Mathematische Annalen, 66:81 94, [FHP96] Frank, K, Heinrich, S, and Pereverzev, S. Information Complexity of Multivariate Fredholm Integral Equations in Sobolev Classes. J. of Complexity, 12:17 34, [Fri91] Friedman, JH. Multivariate adaptive regression splines. Ann. Statist., 19(1):1 141, [Gar98] Garcke, J. Berechnung von Eigenwerten der stationären Schrödingergleichung mit der Kombinationstechnik. Diplomarbeit, Institut für Angewandte Mathematik, Universität Bonn, [Gar04] Garcke, J. Maschinelles Lernen durch Funktionsrekonstruktion mit verallgemeinerten dünnen Gittern. Doktorarbeit, Institut für Numerische Simulation, Universität Bonn, [GG98] Gerstner, T and Griebel, M. Numerical Integration using Sparse Grids. Numer. Algorithms, 18: , [GG00] [GG02] Garcke, J and Griebel, M. On the computation of the eigenproblems of hydrogen and helium in strong magnetic and electric fields with the sparse grid combination technique. Journal of Computational Physics, 165(2): , Garcke, J and Griebel, M. Classification with sparse grids using simplicial basis functions. Intelligent Data Analysis, 6(6): , (shortened version appeared in KDD 2001, Proc. of the Seventh ACM SIGKDD, F. Provost and R. Srikant (eds.), pages 87-96, ACM, 2001). [GG03] Gerstner, T and Griebel, M. Dimension Adaptive Tensor Product Quadrature. Computing, 71(1):65 87, [GG05] [GGT01] Garcke, J and Griebel, M. Semi-supervised learning with sparse grids. In Amini, MR, Chapelle, O, and Ghani, R, editors, Proceedings of ICML, Workshop on Learning with Partially Classified Training Data, pages Garcke, J, Griebel, M, and Thess, M. Data mining with sparse grids. Computing, 67(3): ,

22 [GH95] Griebel, M and Huber, W. Turbulence simulation on sparse grids using the combination method. In Satofuka, N, Periaux, J, and Ecer, A, editors, Parallel Computational Fluid Dynamics, New Algorithms and Applications, pages North-Holland, Elsevier, [GH03] Gradinaru, V and Hiptmair, R. Multigrid for discrete differential forms on sparse grids. Computing, 71(1):17 42, [GHSZ93] Griebel, M, Huber, W, Störtkuhl, T, and Zenger, C. On the parallel solution of 3D PDEs on a network of workstations and on vector computers. In Bode, A and Cin, MD, editors, Parallel Computer Architectures: Theory, Hardware, Software, Applications, volume 732 of Lecture Notes in Computer Science, pages Springer Verlag, [GK00] [GK03] [GOS99] [Gri91] [Gri98] Griebel, M and Knapek, S. Optimized tensor-product approximation spaces. Constructive Approximation, 16(4): , Griebel, M and Koster, F. Multiscale Methods for the Simulation of Turbulent Flows. In Hirschel, E, editor, Numerical Flow Simulation III, volume 82 of Notes on Numerical Fluid Mechanics and Multidisciplinary Design, pages Springer-Verlag, Griebel, M, Oswald, P, and Schiekofer, T. Sparse Grids for Boundary Integral Equations. Numer. Mathematik, 83(2): , Griebel, M. A parallelizable and vectorizable multi-level algorithm on sparse grids. In Hackbusch, W, editor, Parallel Algorithms for partial differential equations, Notes on Numerical Fluid Mechanics, volume 31, pages Vieweg Verlag, Braunschweig, Also as SFB Bericht, 342/20/90 A, Institut für Informatik, TU München, Griebel, M. Adaptive Sparse Grid Multilevel Methods for elliptic PDEs based on finite differences. Computing, 61(2): , [GSZ92] Griebel, M, Schneider, M, and Zenger, C. A combination technique for the solution of sparse grid problems. In de Groen, P and Beauwens, R, editors, Iterative Methods in Linear Algebra, pages IMACS, Elsevier, North Holland, [GT95] Griebel, M and Thurner, V. The efficient solution of fluid dynamics problems by the combination technique. Int. J. Num. Meth. for Heat and Fluid Flow, 5(3): , Also as SFB Bericht 342/1/93 A, Institut für Informatik. TU München,

23 [HBS + 06] Hegland, M, Burden, C, Santoso, L, MacNamara, S, and Booth, H. A solver for the stochastic master equation applied to gene regulatory networks. Journal of Computational and Applied Mathematics, In Press, Corrected Proof, [Hem95] Hemker, P. Sparse-Grid finite-volume multigrid for 3D-problems. Advances in Computational Mathematics, 4:83 110, [Heu97] Heußer, N. Berechnung einer Dünngitterlösung der dreidimensionalen Lamé-Gleichung mit der Kombinationsmethode. Diplomarbeit, Institut für Angewandte Mathematik, Rheinische Friedrich- Wilhelms-Universität Bonn, [HKZ00] Hochmuth, R, Knapek, S, and Zumbusch, G. Tensor products of Sobolev spaces and applications. submitted, Also as Technical Report 685, SFB 256, Univ. Bonn. [Hoc99] [Hub96] [Kli06] [Kna00] [Kos02] [Kra02] [KW05a] [KW05b] Hochmuth, R. Wavelet Bases in numerical Analysis and Restrictal Nonlinear Approximation. Habilitation, Freie Universität Berlin, Huber, W. Turbelenzsimulation mit der Kombinationsmethode auf Workstation-Netzen und Parallelrechnern. Dissertation, Institut für Informatik, Technische Universität München, Klimke, A. Sparse Grid Interpolation Toolbox User s Guide. Technical Report IANS report 2006/001, University of Stuttgart, Knapek, S. Approximation und Kompression mit Tensorprodukt- Multiskalenräumen. Doktorarbeit, Universität Bonn, April Koster, F. Multiskalen-basierte Finite Differenzen Verfahren auf adaptiven dnnen Gittern. Doktorarbeit, Universität Bonn, January Kranz, CJ. Untersuchungen zur Kombinationstechnik bei der numerischen Strömungssimulation auf versetzten Gittern. Dissertation, Institut für Informatik, Technische Universität München, Klimke, A and Wohlmuth, B. Algorithm 847: Spinterp: piecewise multilinear hierarchical sparse grid interpolation in MATLAB. ACM Trans. Math. Softw., 31(4): , Klimke, A and Wohlmuth, B. Computing expensive multivariate functions of fuzzy numbers using sparse grids. Fuzzy Sets and Systems, 154(3): , [LLS95] Liem, CB, Lü, T, and Shih, TM. The Splitting Extrapolation Method. World Scientific, Singapore,

24 [LNSH05] Laffan, SW, Nielsen, OM, Silcock, H, and Hegland, M. Sparse Grids: a new predictive modelling method for the analysis of geographic data. International Journal of Geographical Information Science, 19(3): , [NH00] [PZ99] Noordmans, J and Hemker, P. Application of an Adaptive Sparse Grid Technique to a Model Singular Perturbation Problem. Computing, 65: , Pflaum, C and Zhou, A. Error analysis of the combination technique. Numer. Math., 84(2): , [Rei04] Reisinger, C. Numerische Methoden für hochdimensionale parabolische Gleichungen am Beispiel von Optionspreisaufgaben. Ph.D. thesis, Ruprecht-Karls-Universität Heidelberg, In Vorbereitung. [Sch99] [Smo63] [SS99] [ST03a] Schiekofer, T. Die Methode der Finiten Differenzen auf dünnen Gittern zur Lösung elliptischer und parabolischer partieller Differentialgleichungen. Doktorarbeit, Universität Bonn, Smolyak, SA. Quadrature and interpolation formulas for Tensor Products of Certain Classes of Functions. Dokl. Akad. Nauk SSSR, 148: , Russian, Engl. Transl.: Soviet Math. Dokl. 4: , Sickel, W and Sprengel, F. Interpolation on Sparse Grids and Nikol skij Besov spaces of dominating mixed smoothness. J. Comput. Anal. Appl., 1: , Schwab, C and Todor, R. Sparse finite elements for stochastic elliptic problems higher order moments. Computing, 71(1):43 63, [ST03b] Schwab, C and Todor, RA. Sparse finite elements for elliptic problems with stochastic loading. Numer. Math., 95(4): , [Tem89] [Tem93a] Temlyakov, VN. Approximation of functions with bounded mixed derivative. Proc. Steklov Inst. Math., 1, Temlyakov, VN. Approximation of Periodic Functions. Nova Science, New York, [Tem93b] Temlyakov, VN. On approximate recovery of functions with bounded mixed derivative. J. Complexity, 9:41 59, [vps04] von Petersdorff, T and Schwab, C. Numerical Solution of Parabolic Equations in High Dimensions. Mathematical Modeling and Numerical Analysis, To appear. 24

25 [Wah90] Wahba, G. Spline models for observational data, volume 59 of Series in Applied Mathematics. SIAM, Philadelphia, [Yse86] Yserentant, H. On the Multi-Level Splitting of Finite Element Spaces. Numerische Mathematik, 49: , [Yse92] [Zen91] Yserentant, H. Hierarchical bases. In R. E. O Malley, J et al., editors, Proc. ICIAM 91. SIAM, Philadelphia, Zenger, C. Sparse Grids. In Hackbusch, W, editor, Parallel Algorithms for Partial Differential Equations, Proceedings of the Sixth GAMM-Seminar, Kiel, 1990, volume 31 of Notes on Num. Fluid Mech., pages Vieweg-Verlag,