arxiv: v2 [math.na] 29 May 2017

Transcription

1 Unversal Scalable Robust Solvers from Computatonal Informaton Games and fast egenspace adapted Multresoluton Analyss Houman Owhad and Clnt Scovel arxv: v2 [math.na] 29 May 2017 May 31, 2017 Abstract We show how the dscovery/desgn of robust scalable numercal solvers for arbtrary bounded lnear operators can, to some degree, be addressed/automated as a Game/Decson Theory problem by reformulatng the process of computng wth partal nformaton and lmted resources as that of playng underlyng herarches of adversaral nformaton games. When the soluton space s a Banach space B endowed wth a quadratc norm, the optmal measure (mxed strategy) for such games (e.g. the adversaral recovery of u B, gven partal measurements ([φ, u]) {1,...,m} wth φ B, usng relatve error n -norm as a loss) s a centered Gaussan feld ξ solely determned by the norm, whose condtonng (on measurements) produces optmal bets. When measurements are herarchcal, the process of condtonng ths Gaussan feld produces a herarchy of elementary gambles/bets (gamblets). These gamblets generalze the noton of Wavelets and Wanner functons n the sense that they are adapted to the norm and nduce a multresoluton decomposton of B that s adapted to the egensubspaces of the operator defnng the norm. When the operator s localzed, we show that the resultng gamblets are localzed both n space and frequency and ntroduce the Fast Gamblet Transform (FGT) wth rgorous accuracy and (near-lnear) complexty estmates. As the FFT can be used to solve and dagonalze arbtrary PDEs wth constant coeffcents, the FGT can be used to decompose a wde range of contnuous lnear operators (ncludng arbtrary contnuous lnear bectons from H0(Ω) s to H s (Ω) or to L 2 (Ω)) nto a sequence of ndependent lnear systems wth unformly bounded condton numbers and leads to O(N polylog N) solvers and egenspace adapted Multresoluton Analyss (resultng n near lnear complexty approxmaton of all egensubspaces). Calforna Insttute of Technology, Computng & Mathematcal Scences, MC 9-94 Pasadena, CA 91125, owhad@caltech.edu Calforna Insttute of Technology, Computng & Mathematcal Scences, MC 9-94 Pasadena, CA 91125, clntscovel@gmal.com 1

2 Contents 1 Introducton Motvatons and hstorcal perspectves On unversal solvers On the game theoretc approach to numercal analyss On unversal optmal recovery measures On operator adapted wavelets Outlne of the paper On the degree of unversalty of the method The Gamblet Transform on R N and on Sobolev spaces The exact gamblet transform on R N The gamblet transform for arbtrary symmetrc lnear operators on Sobolev spaces Exponental decay of gamblets Fast Gamblet Transform Non dvergence form and non symmetrc operators The gamblet transform on a Banach space Settng Herarchy of nested measurements The Gamblet Transform Bounded condton numbers Herarchcal computaton of gamblets Dscrete Gamblet transform Dscrete gamblet transform/computaton Operator nverson wth the gamblet transform The nverse problem and ts varatonal formulaton Identfcaton of Q and varatonal formulaton of Lu = g General case General case wth D = L Self-adont case wth the energy norm When B 2 = B Identfcaton of measurement functons Usng the gamblet transform to solve nverse problems/lnear systems Computatonal Informaton Games From Informaton Based Complexty to Game Theory The game theoretc soluton to numercal approxmaton Measures, cylnder measures, weak dstrbutons, and Gaussan felds Mxed extenson of the game and optmal mnmax solutons Repeated games across herarches of ncreasng levels of complexty

3 5.6 Probablstc nterpretaton of numercal errors Gaussan flterng Emergence of probablstc computaton, quantum mechancs and the smulaton hypothess Unversal worst case measure for optmal recovery Exponental decay and localzaton of gamblets Subspace decomposton Condtons on dual and mage spaces Condtons on localzed measurement functons Numercal homogenzaton Fast Gamblet Transform and Solve The algorthm Complexty Equvalent condtons for localzaton Correspondence between Numercal Analyss, Approxmaton Theory, and Statstcal Inference 69 9 Addtonal propertes of the gamblet transform Dual varatonal formulaton and dual gamblets Dual measurement functons of the herarchy of orthogonalzed gamblets Mnmum angle between gamblets Drac deltas as measurement functons, averagng vs sub-samplng, hgher order PDEs and relaxed condtons Condtons for Algorthm 4 on dscretzed Banach spaces Identfcaton of measurement functons Condtons on dscretzed Banach spaces Alternatve ntalzaton of Algorthm Proofs of the results of Sectons 3 and Proof of Proposton Proof of Theorem Proof of Theorem Theorem Proof of Theorems 3.16 and Proof of Theorem Nested computaton Unformly bounded condton numbers Proof of Theorem Proof of Theorem Inverse problems Proof of Lemma

4 Proof of Theorem Proof of Proposton Proofs of the results of Secton Proof of Theorem Proof of Theorem Proof of Theorem Proof of Theorem Proof of Proposton Proof of Theorem Proof of Proposton Proof of Theorem Gaussan cylnder measures as weak lmts of Gaussan measures Proofs of the results of Secton Proofs of the results of Subsecton Proof of Lemma Proof of Lemma Proof of Theorem Null space of measurement functons and quotent operator Proofs of the results of Subsectons 6.2 and Proof of Theorem Proof of Theorem Proof of Proposton Proof of Lemma Proof of Theorem Proof of Theorem Proof of Theorem Numercal homogenzaton Proof of Theorem Control of the exponental decay of nterpolaton matrces Numercal homogenzaton wth localzed bass functons Proof of Corollary Proofs of the results of Secton Proof of Theorems 2.5 and Proof of Theorem Proof of Theorem Proof of Theorem Proof of Theorem Proof of Proposton Proof of Theorem Proof of Theorem Proof of Theorem

5 14 Proofs of the results of Secton Error propagaton across scales Proof of Theorem Equvalent condtons for localzaton Proof of Theorem Proof of Theorem Proofs of the results of Secton Proof of Theorem Proof of Theorem Proof of Proposton Proof of Proposton Proof of Proposton Proof of Theorem Proof of Lemma Introducton 1.1 Motvatons and hstorcal perspectves On unversal solvers Is t possble to dentfy/desgn a scalable solver that could be appled to nearly all lnear operators? One ncentve to ask ths queston s the vast and ncreasng lterature on the numercal approxmaton of lnear operators where the number of lnear solvers seems to tral the number of possble lnear systems. Paraphrasng Sard s asserton, one reason not to ask ths queston s the hstorcal presupposton that [195, pg. 223] of course no one method of approxmaton of a lnear operator can be unversal. Indeed, ths asserton s reasonable and s now rgorously supported by No Free Lunch theorems n Learnng Theory (see [67, Thm. 7.2] and [249]) and n Optmzaton [250]. However, such profound results do not preclude the exstence of weak assumptons under whch unversal algorthms may exst. For example, the recent success of Support Vector Machnes [211] s an astonshng example whch has transformed Learnng Theory. In ths paper we nvestgate the possblty of achevng some degree of unversalty n answerng ths queston n the settng of lnear operators on Banach spaces wth quadratc norms (whch nclude matrx equatons). We show that, to some degree, a postve answer can be obtaned when the operator s a contnuous becton under the followng condtons on the mage space: exstence of a compact embeddng and a mult-resoluton decomposton (thereby generalzng the results of [169]). The only condton on the actual operator s that t s a contnuous becton. 5

6 1.1.2 On the game theoretc approach to numercal analyss Another purpose of ths paper s to show that the dscovery of these solvers can, to some degree, be automated through a game/decson theoretc approach to numercal approxmaton and algorthm desgn based on the observatons that (1) to compute fast one must compute wth partal nformaton over herarches of ncreasng levels of complexty (2) computng effcently wth partal nformaton requres solvng mnmax problems aganst the mssng nformaton (3) these mnmax problems are repeated and mxed (game theoretc) optmal strateges emerge as natural solutons (and lead to natural Bayesan nterpretatons of the resultng methods and approxmaton errors). The connecton between Informaton Theory and Numercal Analyss emerges naturally from the Informaton Based Complexty [251, 181, 226, 158, 252] noton that computaton on a contnuous space (nfnte dmensonal space) can only be done wth partal nformaton. Here ths noton wll be expanded to the prncple that fast computaton requres computng wth partal nformaton over a herarchy of levels of complexty. Consder, for nstance, the problem of nvertng a matrx. A method that would requre computng wth all the entres of that matrx at once would lead to a slow method. To obtan a fast method one must compute wth a few number of features of that matrx and these features typcally do not represent all the entres of the matrx. Therefore one must brdge the nformaton gap between these few features and the whole matrx. To be made effcent, ths prncple must be repeated over a herarchy (e.g. nformaton gaps must be brdged between 4, 16, 64,... degrees of freedom). Ths prncple s evdently present n classcal fast solvers such as multgrd methods [85, 31, 98, 99, 191, 215], multlevel fnte element splttng [255], multlevel precondtonng [232], stablzed herarchcal bass methods [233, 234, 235], multresoluton methods [34, 26, 7, 71, 81], the Fast Multpole Method [95], Herarchcal matrces [100, 19], Cholesky and multgrd solvers for Graph Laplacans [133, 132] and fast solvers for Symmetrc Dagonally Domnant Matrces [55, 208, 209, 125, 120]. Whle, for classcal solvers, these nformaton gaps have been brdged by essentally guessng the form of nterpolaton operators, here we wll, as n [169] (see also [177, 170, 196]), reformulate the process of brdgng these gaps as that of playng repeated adversaral games aganst the mssng nformaton and dentfy optmal mxed strateges for playng such games over herarches of ncreasng levels of complexty On unversal optmal recovery measures The game theoretc approach s relevant to numercal analyss/approxmaton for two man reasons: (1) Inaccurate approxmatons, n repeated ntermedary calculatons, lead to loss n CPU tme and the total CPU tme requred to nvert a gven lnear operator s the sum of these losses. Therefore fndng optmal strateges for the repeated games descrbng ntermedate numercal approxmaton steps translates nto the mnmzaton of the overall requred CPU tme. (2) As exposed n the reemergng feld of probablstc numercs/computng [45, 197, 167, 104, 102, 35, 57, 169, 177, 48, 184, 186, 47, 196] (we refer to Secton 8 for an overvew) by usng a probablstc descrpton of numercal 6

7 errors t s possble to seamlessly combne model and numercal errors n an encompassng Bayesan framework. However, whle confdence ntervals obtaned from arbtrary prors may be hard to ustfy to a numercal analyst, worst case measures (dentfed as optmal mxed strateges) are robust n adversaral envronments. In ths paper we wll show (Secton 5) that gven a Banach space B endowed wth a quadratc norm, one can dentfy a Gaussan cylnder measure, solely determned by the norm, whose condtonng produces optmal mxed and pure strateges for playng the adversaral games nherent to numercal approxmaton. Ths measure s unversal n the sense that t does not depend on the partal nformaton enterng n the numercal approxmaton problem. Furthermore ts condtonng produces not only a saddle pont n the game theoretc formulaton of numercal approxmaton, but also optmal recovery solutons [93, 153] that are optmal n the determnstc mnmax formulaton. In that sense these unversal optmal recovery measures form a natural brdge between probablstc numercs and classcal numercal analyss/approxmaton. On the relaton wth Decson Theory. One of the landmark dscoveres n Wald s theory of Statstcal Decson Functons [238, 239], was the result that, under mld condtons, the optmal statstcal decson functon was obtaned by extendng the correspondng two person game to ts mxed extenson, obtanng a worst case measure as one component of a saddle pont of the mxed extenson, and then for the prmary player to play as f the second player (nature) used ths worst case measure as ther strategy. In Secton 5 we show that the mnmax optmal soluton to an optmal recovery problem can be recovered n a smlar way. However, n ths case, the scalng propertes of the loss functon of Secton 5.2 lead to a mxed extenson whch ncorporates those propertes. Although no true measure can be a component n a saddle pont (.e. be a worst case measure) for ths extenson, we obtan (Secton 5.9) approxmate saddle ponts to any degree of approxmaton usng Gaussan measures whose covarance operators are easly constructed (computable). These Gaussan measures converge to a Gaussan cylnder measure whose covarance Q : B B s the same as that determnng the nner product, := [Q 1, ] of B, establshng that such a Gaussan cylnder measure s a unversal worst case (weak) probablty measure, snce, as dscussed above, t s ndependent of the choce of the measurement functons (ths unversal Gaussan cylnder measure s an sometry from B to Gaussan space characterzed by the fact that the mage of φ B s a real valued centered Gaussan random varable wth varance φ 2 where s the dual norm of ) On operator adapted wavelets Wavelets [146, 62, 56] have transformed sgnal and mage processng. Could they have a smlar mpact on numercal analyss? Ths queston has stmulated the development of adapted/adaptve wavelets amed at solvng PDEs (or boundary ntegral equatons) [25, 14, 13, 4, 114, 59, 60, 24, 80, 231, 43, 155, 106, 54, 44, 50, 18, 51, 52, 216, 61, 199, 6, 82, 254] or performng MRA on the solutons of PDEs [87, 84, 202]. Whle frst generaton adaptve wavelets (such as b-orthogonal wavelets [53], see [212] for an overvew) can be 7

8 constructed wth arbtrarly hgh preassgned regularty (for adaptaton to the regularty of the elements of the soluton space of the operator) and can replace mesh refnement [49] n numercal approxmaton (as an adaptaton to the local regularty of a partcular soluton) ther shft (and possble scale) nvarance prevents ther adaptaton to rregular domans or non-homogeneous coeffcents. Second generaton wavelets [219, 234, 235, 216] (see [216, Sec. 1.2] for an overvew) offer stronger adaptablty at the cost of a possble loss n shft and scale nvarance. The man dea of second generaton wavelets s to start wth a [219] lazy multresoluton decomposton of the soluton space (such as herarchcal bass methods [255, 17]) that may not possess desrable propertes (such as scale orthogonalty wth respect to the scalar product defned by the operator and vanshng polynomal moments) and then modfy the herarchy of bass functons to acheve desrable propertes, usng constructon technques such as the lftng scheme of Sweldens, the stable constructon technque of Carncer, Dahmen and Peña [43], the orthogonalzaton procedure of Lounsbery et al. [142], the wavelet-modfed herarchcal bass of Vasslevsk and Wang [234, 235], and the stable completon, Gram-Schmdt orthogonalzaton, and approxmate Gram-Schmdt orthogonalzaton of Sudarshan [216]. As emphaszed n [216, p. 83] deal adapted wavelets should be characterzed by 3 propertes: (a) scale-orthogonalty (wth respect to the scalar product assocated wth the operator norm to ensure block-dagonal stffness matrces) (b) local support (or rapd decay) of the bass functons (to ensures that the ndvdual blocks are sparse) and (c) Resz stablty n the energy norm (to ensure that the blocks are well-condtoned). However, as dscussed n [216, p. 83], although adapted wavelets achevng 2 of these propertes have been constructed, t s not known f there s a practcal technque for ensurng all the three propertes smultaneously n general. In ths paper we wll ntroduce operator adapted wavelets (gamblets) exhbtng all 3 propertes for local contnuous lnear bectons on Banach spaces. Gamblets are dentfed by condtonng the unversal measure dscussed Subsecton wth respect to a (non operator adapted) multresoluton decomposton of the dual (or mage) space and have (as a consequence) optmal adversaral approxmaton propertes (n both frameworks of optmal recovery and game-theory). They are not only adapted to the regularty of the elements of the soluton space but also to the egen-subspaces of the operator tself (Theorem 3.19). Through ths adaptaton, gamblets nduce a near optmal sparse compresson of the operator (3.21) and provde near-lnear complexty (Secton 7) solutons to the problem of fndng localzed (Secton 6) Wanner functons [149, 122, 241, 79, 179, 169, 177, 108, 110] (lnear combnatons of egenfunctons concentrated around a gven egenvalue, that are localzed n space). 1.2 Outlne of the paper We wll present the man results and algorthms (wth numercal Illustratons) n Sectons 2 to 9 and proofs (along wth further results) n Sectons 10 to 15. Secton 2 presents the Gamblet Transform for the lnear system Ax = b n R N and for arbtrary contnuous lnear bectons mappng H s 0 (Ω) to H s (Ω) or to L 2 (Ω). The man purpose 8

9 of Secton 2 s to, at the cost of some redundancy, facltate the accessblty of the paper. Secton 3 ntroduces the Gamblet Transform (and ts dscrete verson) on a Banach space B gfted wth a quadratc norm. The Gamblet Transform, whch could be seen as a generalzed Wavelet Transform [63, 150] that s adapted to (B, ), turns a multresoluton decomposton of B nto a multresoluton decomposton of B wth bass functons, called gamblets, that span orthogonal subspaces of B, akn to an egenspaces, enablng the mult-resoluton decomposton of any element u B nto components that are localzed n space and frequency. As the Fourer Transform can be used to solve lnear PDEs wth constant coeffcents, Secton 4 shows that the Gamblet Transform can be used to transform an arbtrary contnuous lnear operator L mappng B to another Banach space B 2 nto a sequence of ndependent lnear systems wth unformly bounded condton numbers. Secton 5 ntroduces the Computatonal Informaton Games framework and shows how gamblets can be dscovered and nterpreted as elementary gambles/bets enablng computaton wth partal nformaton of herarches of ncreasng levels of complexty. Sequences of approxmatons form a martngale under the mxed strategy emergng from the underlyng games and underlyng approxmaton errors are decomposed as sums of ndependent Gaussan felds actng at dfferent levels of resoluton. In partcular Secton 5 provdes a probablstc descrpton of numercal errors (n terms of posteror dstrbutons) that can be used, as n Probablstc Numercs or Scentfc Computng, to seamlessly combne numercal errors wth model errors n an encompassng Bayesan framework. Secton 6 proves that gamblets are localzed (exponentally decayng) based on propertes of the dual space B or the mage space B 2. Therefore gamblets could also be seen, as a generalzaton of Wanner bass functons [149, 122, 241]. Ths exponental decay also provdes a rgorous ustfcaton of the screenng effect seen n Krgng [210] where condtonng a spatal random feld on a (homogeneously dstrbuted) cloud of ponts leads to exponental decay n correlatons. Based on the exponental decay, Secton 7 ntroduces the Fast Gamblet Transform (FGT) whose complexty s O(N ln 3d N) to compute the herarchy of gamblets, and O(N ln d+1 N) to decompose u B over the gamblet bass or nvert a lnear system Lu = g wth g B 2. Snce gamblets nduce a multresoluton decomposton of B that s adapted to the egensubspaces of the operator Q : B B defnng the norm, the FGT can not only be used as a fast solver, but also as fast proecton on approxmatons of these egensubspaces, as a fast operator compresson algorthm, as a near-lnear complexty PCA algorthm [116], or as a near-lnear complexty actve subspace decomposton method [58, 141]. Fnally, Secton 8 provdes a short revew of the reemergng and fascnatng nterplay between the felds of Numercal Analyss, Approxmaton Theory, and Statstcal Inference that provdes both the hstorcal background of the paper and ndcates future developments that are currently avalable. 1.3 On the degree of unversalty of the method Gven a Banach space (B, ) and a nested herarchy of measurement functons (a herarchy of elements φ (k) B such that each level k measurement functon φ (k) s a 9

10 lnear combnaton of level k+1 measurement functons φ (k+1) ), the method (the gamblet transform), produces under stablty condtons, a herarchy of localzed elements of B that are scale-orthogonal wth respect to the scalar product nduced by wth well condtoned stffness matrces. These stablty condtons (Condtons 3.18 and (6.11)) are condtons nvolvng the nterplay between the norm (or equvalently ts dual form) and the measurement functons φ (k). The unversalty of the method s derved from the fact that, gven the measurement functons φ (k), these stablty condtons are an nvarant (modulo proportonal change of constants) of the equvalent class of the norm,.e. f these stablty condtons are satsfed by another norm 1 on B such that C 1 1 C 2 1, then they must also be satsfed by (wth constants scaled by C 1 and C 2 ). As a consequence, f s the operator norm of a contnuous lnear becton L between (B, ) and another Banach space (B 2, 2 ) (wth quadratc norm 2 ) then, as shown n Theorem 4.14, the stablty condtons can be expressed as condtons on the norm 2 placed on the mage space and the values of the contnuty constants of L and L 1. Consequently, modulo ths dependence on the contnuty constants, these condtons are ndependent of the operator L tself. Sad another way, the stablty condtons do not depend on the structure of the operator L but only on ts contnuty constants. In the Sobolev space settng of Secton 2, ths transfer of stablty condtons allows us to show that the method s effcent when s the operator norm of an arbtrary contnuous (symmetrc, postve) lnear becton between H0 s(ω) and H s (Ω) by selectng measurement functons satsfyng the requred stablty condtons for the H s (Ω) norm (Condtons 2.13 and 2.22). Smlarly, n the lnear algebra settng of Secton 2, nvolvng the nverson of the matrx system Ax = b (where A s symmetrc and postve), the stablty condtons (Condtons 2.3 and 7.5) are nvarant (modulo a proportonal change of constants) wth respect to the quadratc form defned by A. Therefore those stablty condtons are satsfed f A s obtaned by dscretzng (usng a stable numercal method) a contnuous lnear becton between H0 s(ω) and H s (Ω). 2 The Gamblet Transform on R N and on Sobolev spaces 2.1 The exact gamblet transform on R N Here we wrte for the Eucldean norm on R N. Consder an N N symmetrc postve defnte matrx A, and let A denote the A-norm defned by u A := u T Au and let, A defned by u, v A := u T Av denote the correspondng nner product. We say that two vectors u, v of R N are A-orthogonal f they are orthogonal wth respect to the A- nner product,.e. f u T Av = 0. For V a lnear subspace of R N, note that v V s the A-orthogonal proecton of u on V f v = argmn w u w A. Hereafter we refer to such A-orthogonal proectons smply as A-proectons. Relabel {1,..., N} usng an ndex tree I (q) of depth q N defned below. Defnton 2.1. We say that I (q) s an ndex tree of depth q f t s the fnte set of q-tuples of the form = ( 1,..., q ). For 1 k q and = ( 1,..., q ) I (q), wrte 10

11 (k) := ( 1,..., k ) and I (k) := { (k) : I (q) }. Wrte I (k) for the I (k) I (k) dentty matrx. Constructon 2.2. For k {1,..., q 1} let π (k,k+1) be a I (k) I (k+1) matrx such that π (k,k+1) (π (k,k+1) ) T = I (k). Algorthm 1 below (the gamblet transform) computes a change of bass on R N through an A-orthogonal decomposton R N = V (1) A W (2) A A W (q), (2.1) where A s the A-orthogonal drect sum, the terms of whch wll be defned shortly. To analyze the performance of Algorthm 1, we wll develop some condtons on A regardng ts relatonshp wth the matrces π (k,k+1) of Constructon 2.2. To that end, for k {1,..., q} wrte π (k,q) := π (k,k+1) π (k+1,k+2) π (q 1,q) (2.2) and let π (q,k) be the transpose of π (k,q). Wrte π (q,q) = I (q). Condton 2.3. There exsts constants C d 1 and H (0, 1) such that the followng condtons are satsfed C d λmn (A) Hk nf x Im(π (q,k) ) x T A 1 x x for k {1,..., q}. 2. sup x Ker(π (k,q) ) x T A 1 x x C d λmn (A) Hk for k {1,..., q 1}. Remark 2.4. Item 2 of Condton 2.3 s equvalent to the Poncaré nequalty z nf z π (q,k) y C d H k T Az y R I(k) λ mn (A) for z RN and k {1,..., q}. (2.3) Item 1 of Condton 2.3 s mpled by the nverse Poncaré nequalty x Cd 1 H k T Ax λ mn (A) x for x Im(π(q,k) ) and k {1,..., q}. (2.4) We wll show n Subsecton 2.2 and the followng sectons that f A s obtaned as the dscretzaton of a contnuous lnear becton L between Sobolev spaces, then matrces π (k,k+1) satsfyng Condton 2.3 can naturally be dentfed from a mult-resoluton decomposton of the mage space, ndependently from the operator tself (.e. ths dentfcaton s easy for lnear systems obtaned by dscretzng contnuous lnear bectons between Sobolev spaces). Graph Laplacans are other prototypcal examples of practcal mportance [21] and Remark 2.4 mples that Condton 2.3 represent natural analytcal (Poncaré and nverse Poncaré) nequaltes nvolvng the nterplay between the matrces π (k,k+1) and the structure of the Graph Laplacan. Although (2.1) s not an exact egenspace decomposton, the followng Theorem shows, under Condton 2.3, that t shares many of ts mportant characterstcs. 11

12 Theorem 2.5. Under Condton 2.3 t holds true that, for some constant C dependng only on C d, CH 2 λ mn (A) vt Av Av 2 C for v V(1) (2.5) and for k {2,..., q}, CH 2k λ mn (A) vt Av Av 2 CH2k 2 for v W (k) (2.6) Gven b R N, Algorthm 1 also computes the soluton u R N of the lnear system Au = b (2.7) and performs the A-orthogonal decomposton of u over the rght hand sde of (2.1),.e. u = v (1) + v (2) + + v (q). (2.8) Snce the decomposton n (2.8) s A-orthogonal, v (1) s the A-proecton of u on V (1) and, for k {2,..., q}, v (k) s the A-proecton of u on W (k). Wrte u (1) := v (1) and, for k {2,..., q}, u (k) := v (1) + v (2) + + v (k) for the correspondng sequence of successve approxmatons of u wth A-orthogonal ncrements. Wrtng V (k) = V (1) A W (2) A A W (k), t follows that V (k) V (k+1) and u (k) s the A-proecton of u on V (k). The followng Theorem demonstrates how Condton 2.3 mples performance guarantees when usng Algorthm 1 to solve the lnear system (2.7). Theorem 2.6. We have u (q) = u and, under Condton 2.3, there exsts a constant C dependng only on C d such that, for k {1,..., q 1}, we have u u (k) A C λmn (A) Hk b. (2.9) Let us now descrbe Algorthm 1. For 1 < r < k and a k-tuple of the form = ( 1,..., k ) we wrte (r) := ( 1,..., r ). Constructon 2.7. For k {2,..., q} let J (k) be a fnte set of k-tuples of the form = ( 1,..., k ) such that { (k 1) J (k) } = I (k 1) and for I (k 1), Card{ J (k) (k 1) = } = Card{s I (k) s (k 1) = } 1. Wrte J (k) for the J (k) J (k) dentty matrx. Constructon 2.8. For k = 2,..., q let W (k) be a J (k) I (k) matrx such that Im(W (k),t ) = Ker(π (k 1,k) ) and W (k) (W (k) ) T = J (k). Algorthm 1 takes A, b, the matrces π (k,k+1) and W (k) as nputs and produces the followng outputs: (1) u the soluton of (2.7) and ts decomposton (2.8) (2) famles of nested vectors of R N, (ψ (k) ) ndexed by k {1,..., q} and I (k) spannng the 12

13 Algorthm 1 Exact Gamblet Transform/Solve on R N. 1: A (q), = A 2: ψ (q) 3: b (q) = e = b 4: for k = q to 2 do 5: B (k) = W (k) A (k) W (k),t 6: w (k) = B (k), 1 W (k) b (k) 7: For J (k), χ (k) = (k) I (k) W, ψ(k) χ (k) 8: v (k) = J (k) w(k) 9: N (k) = A (k) W (k),t B (k), 1 10: R (k 1,k) = π (k 1,k) (I (k) N (k) W (k) ) 11: b (k 1) = R (k 1,k) b (k) 12: A (k 1) = R (k 1,k) A (k) R (k,k 1) 13: For I (k 1), ψ (k 1) = I 14: end for 15: w (1) = A (1), 1 b (1) 16: v (1) = I (1) w(1) ψ (1) 17: u = v (1) + v (2) + + v (q) (k) R(k 1,k), ψ (k) nested lnear subspaces V (k) := span{ψ (k) I (k) } of R N (3) famles of vectors of R N, (χ (k) ) ndexed by k {2,..., q} and J (k) spannng A-orthogonal lnear subspaces W (k) := span{χ (k) J (k) } of R N such that the decomposton (2.1) holds and V (k) = V (k 1) A W (k) (4) postve defnte I (k) I (k) matrces A (k) ndexed by k {1,..., q} (5) postve defnte J (k) J (k) matrces B (k). Snce the subspaces enterng n the decomposton (2.1) are A-orthogonal and adapted to the egenspaces of A, (2.7) can be solved ndependently on each one of them by solvng q well condtoned lnear systems. More precsely Algorthm 1, transforms the N N lnear system (2.7) nto q ndependent lnear systems A (1) w (1) = b (1) and B (k) w (k) = W (k) b (k), k {2,..., q}, (2.10) and the followng theorem guarantees that these systems are well condtoned. Theorem 2.9. Under Condton 2.3, there exsts a constant C dependng only on C d such that C 1 I (1) A(1) λ mn (A) CH 2 I (1), Cond(A (1) ) CH 2. and for k {2,..., q} C 1 H 2(k 1) J (k) B(k) λ mn (A) CH 2k J (k) and Cond(B (k) ) CH 2. Let R (k,k+1) be the I (k) I (k+1) nterpolaton matrces computed n Algorthm 1, wrte R (k,q) = R (k,k+1) R (q 1,q) and let R (q,k) be ts I (q) I (k) matrx transpose. The matrces A (k) correspond to a I (k) I (k) matrx compresson of A n the sense that, 13

14 under Condton 2.3, (A 1 R (q,k) A (k), 1 R (k,q) )b A C λmn (A) Hk b (2.11) for b R N, where the constant C depends only on C d. The compresson (2.11) can be nterpreted as numercal homogenzaton and [23] and [169, Sec. 2.5] establsh ts optmalty (up to a multplcatve constant when compared to an exact egenspace decomposton). The vectors ψ (k) and χ (k) can be nterpreted as algebrac wavelets (n Secton 5 we wll ustfy the name gamblets) adapted to the matrx A. The vectors ψ (k) V (k), satsfy the nestng relaton ψ (k) = I (k) I (k) W, ψ(k) (k+1) R(k,k+1), ψ (k+1), I (k), spannng. The vectors χ (k) =, J (k), spannng W (k), are obtaned by A-orthogonalzaton wth respect to the decomposton V (k) = V (k 1) A W (k). Furthermore for k {1,..., q} and w R (k), the mnmzer of v A over v R N subect to w = π (k,q) v s v = I (k) w ψ (k). 2.2 The gamblet transform for arbtrary symmetrc lnear operators on Sobolev spaces Defnton Gven Ω a bounded open subset of R d wth unformly lpschtz boundary and s N, let H s (Ω) be the Sobolev space [90, Sec ] gfted wth the norm u 2 H s (Ω) := s D t u 2 L 2 (Ω) for u Hs (Ω) (2.12) t=0 where D t u s the total dervatve of u of order t, D 0 u = u, D t u = (D t u D t u) 1 2 and D t u D t v = d t u t v 1,..., t=1 1 t 1 t. Wrte H0 s (Ω) for the closure of the set of smooth functons wth compact support n Ω wth respect to the norm H s (Ω). Wrte k the k-th terate of the Laplacan. For s = 2k, wrte u H s 0 (Ω) = k u L 2 (Ω). For s = 2k + 1 wrte u H s 0 (Ω) = k u L 2 (Ω). Recall [90, Thm. 2.2] that H s 0 (Ω) defnes a norm that s equvalent to H s (Ω) on H0 s(ω). Let (H s (Ω), H s (Ω)) be the dual of (H0 s(ω), H0 s(ω) ) usng the usual dual parng obtaned from the Gelfand trple H0 s(ω) L 2 (Ω) H s (Ω) of Sobolev spaces; for g H s (Ω), g H s (Ω) = sup v H s 0 (Ω) Let Ω gv v H s 0 (Ω). L : H s 0(Ω) H s (Ω) (2.13) be an symmetrc contnuous lnear becton between H0 s(ω) and H s (Ω), and wrte C L := sup u H s 0 (Ω) Lu H s (Ω)/ u H s 0 (Ω) and C L 1 := sup u H s 0 (Ω) u H s 0 (Ω)/ Lu H s (Ω) for ts contnuty constants. Wrte [, ] for the dualty parng between H s (Ω) and H0 s(ω) defned by the L2 (Ω) ntegral [g, v] := Ω gv, g H s (Ω), v H0 s (Ω). Let be the norm on H0 s(ω) defned by u := [Lu, u]

15 and wrte, for the correspondng scalar product. Let I (k) and π (k,k+1) be defned as n Defnton 2.1 and Constructon 2.2. Let (φ (q) ) I (q) be orthonormal elements of L 2 (Ω) and, for k {1,..., q 1} and I (k) defne φ (k) va nducton by φ (k) = π (k,k+1) I (k+1) φ (k+1) (2.14) For k {1,..., q} and I (k), let ψ (k) be the mnmzer of the followng varatonal problem. { Mnmze ψ Subect to ψ H0 s (Ω) and [φ(k), ψ] = δ, for I (k) (2.15) We wll call the elements ψ (k) gamblets. For k {1,..., q} let Θ (k) be the I (k) I (k) symmetrc matrx defned by Θ (k), := [φ(k), L 1 φ (k) ]. Theorem For k {1,..., q}, Θ (k) s postve defnte, and wrtng Θ (k), 1 ts nverse, we have for I (k), ψ (k) = Θ (k), 1, I (k) L 1 φ (k) (2.16) For k {2,..., q}, let J (k) and W (k) be as n Constructons 2.7 and 2.8. k {2,..., q} and J (k) wrte χ (k) := W (k), ψ(k) I (k) For (2.17) For k {1,..., q} wrte V (k) := span{ψ (k) I (k) } and for k {2,..., q} wrte W (k) := span{χ (k) J (k) }. Let W (q+1) be the, -orthogonal complement of V (q) n H0 s(ω). Wrte for the, -orthogonal drect sum. Theorem For k {2,..., q}, V (k 1) V (k) and V (k) = V (k 1) W (k). In partcular, H0(Ω) s = V (1) W (2) W (q) W (q+1) (2.18) Furthermore, k {1,..., q}, the, orthogonal proecton of u H s 0 (Ω) onto V(k) s u (k) = [φ (k) I (k), u]ψ (k) (2.19) and, for k {2,..., q}, the, orthogonal proecton of u H s 0 (Ω) onto W(k) s u (k) u (k 1). 15

16 For k {1,..., q}, wrte Φ (k) := span{φ (k) I (k) }. (2.20) The analogue here to Condton 2.3 for the lnear algebra case s as follows. Condton There exsts constants C s 1 and H (0, 1) such that the followng condtons are satsfed for k {1,..., q}. 1. φ L 2 (Ω) C s H k φ H s (Ω) for φ Φ (k). 2. φ H s (Ω) C s H k φ L 2 (Ω) for φ L 2 (Ω) Φ (k). For k {1,..., q} let A (k) be the stffness matrx of the operator L n V (k),.e. A (k), = ψ (k), ψ (k) for, I (k). For k {2,..., q} let B (k) be the stffness matrx of the operator L n W (k),.e. B (k), = χ (k), χ (k) for, J (k). Theorem Assume Condton 2.13 to be satsfed. Then there exsts a constant C dependng only on C s, C L and C L 1 such that for u H0 s(ω) u u (k) H s 0 (Ω) CH k Lu L 2 (Ω). (2.21) Furthermore, C 1 I (1) A(1) λ mn (A) CH 2 I (1), Cond(A (1) ) CH 2, and for k {2,..., q}, C 1 H 2(k 1) J (k) B(k) λ mn (A) CH 2k J (k) and Cond(B (k) ) CH 2. Condton Assume that there exsts an ndex tree Ī(q), as n Defnton 2.1, and a fnte set ℵ = {1,..., ℵ } such that I (k) = Ī(k) ℵ for all k {1,... q},.e. ndependent of k, each label (t, α) Ī(k) ℵ s n one to one correspondence wth a label I (k) and we wrte ℵ := t for = (t, α). From the Constructon 2.7 of J k+1, we wrte (k),ℵ := t for J (k+1) and (k) = (t, α) I (k). Assume that π (k,k) and W (k) are cellular,.e. (1) π (k 1,k), = 0 for (k 1),ℵ ℵ and (2) W (k), = 0 for (k 1),ℵ (k 1),ℵ for k {2,... q}. Remark When ℵ = 1, Condton 2.15 smplfes to (1) π (k 1,k), = 0 for (, ) I (k 1) I (k) wth (k 1) and (2) W (k), = 0 for (, ) J (k) I (k) wth (k 1) (k 1). We refer to [169, Constructon 4.13] for a possble cellular choce for W (k) so that W (k) W (k),t = J (k) when π (k,k+1) s cellular wth constant non-zero entres n each cell. Example Let h, δ (0, 1). For an open non-vod convex subset τ of Ω R d wrte P s 1 (τ) the space of d-varate polynomals on τ of degree at most s 1. Let n = ( ) s+d 1 d be the dmenson of P s 1 (τ). Let ℵ = {1,..., n}. Let Ī(q) and I (q) be ndex trees defned as n Condton Let (τ (k) t ) t Ī (k) be convex unformly Lpschtz convex sets formng (k) a nested partton of Ω,.e. Ω = Ī (k)τ s a dsont unon except for the boundares, for k {1,..., q} and τ (k) = Ī (k+1) : (k) = τ (k+1) for k {1,..., q 1}. Assume that 16

17 each τ (k), contans a ball of radus h k, and s contaned n a ball of radus δh k. For Ī(k), let (φ (k),α ) α ℵ be an L 2 (τ (k) )-orthonormal bass of P s 1 (τ (k) ). Note that the set of ndces (, α) Ī(q) ℵ form the ndex tree I (q) and I (k) = Ī(k) ℵ. Proposton Condton 2.13 s satsfed wth the measurement functons presented n Example 2.17 wth H = h s and a constant C s dependng only on d, δ and s. Remark The gamblet transform turns the resoluton of Lu = g nto that of a sequence of ndependent lnear systems wth unformly bounded condton numbers. When L s a local operator, and the measurement functons are localzed, Subsecton 2.3 shows that the resultng gamblets are also localzed, the fast gamblet transform descrbed n Subsecton 2.4 enables the computaton of gamblets n Lu = g n O(N log 3d (N)) complexty and the resoluton of Lu = g (up to grd-sze accuracy n H0 s (Ω)-norm) n O(N log d+1 (N)) complexty. 2.3 Exponental decay of gamblets Consder the operator L ntroduced n (2.13). Constructon 2.20 (Localty of measurement functons). Let h > 0 and δ (0, 1). For ℵ, ℶ N, Let ℵ, ℶ be the fnte sets {1,..., ℵ } and {1,..., ℶ }. Let (τ ) ℶ be a partton of Ω such that each τ s convex, unformly Lpschtz, contans a ball of radus δh and s contaned n a ball of radus h. Let (φ,α ) (,α) ℶ ℵ H s (Ω) be such that the support of φ,α s contaned n τ. For ℶ, let Ω Ω such that Ω contans τ, Ω c Ω s at dstance at least h from τ, and Ω s Lpschtz. Defnton Let I be a fnte ndex set. For an I I matrx X, we defned the graph dstance d X on I as follows. For (, ) I I we defne d X,, the graph dstance of X between and, as the mnmal length of paths connectng and wthn the matrx graph of X,.e. d X, s the smallest number m such that there exsts ndces 0, 1,..., m I wth 0 =, m = and X t 1, t 0 for t {1,..., m} wth the conventon that d X, = + f no such path exsts. Moreover, dx, = 0 f and only f =. Let C be the ℶ ℶ (connectvty) matrx defned by C, = 1 f there exsts a (χ, χ ) H0 s(ω ) H0 s(ω ) such that χ, χ 0, and C, = 0 otherwse. Let d := d C be the graph dstance on ℶ nduced by the connectvty matrx C. For n N, let Ω,n := :d(,) n Ω, and note that Ω,0 = Ω, ℶ. For (, α) ℶ ℵ, let ψ,α be the mnmzer of ψ over ψ H0 s (Ω) subect to [φ,β, ψ] = δ, δ α,β for (, β) ℶ ℵ. In addton, for each subset Ω, ℶ and each α ℵ, let ψ,α n be the mnmzer of ψ over ψ Hs 0 (Ω,n) subect to [φ,β, ψ] = δ, δ α,β for (, β) ℶ ℵ. Wrte Φ := span{φ,α (, α) ℶ ℵ}. Condton There exsts C mn, C max > 0 such that for ϕ H s (Ω). C mn nf ϕ φ Φ φ 2 H s (Ω) nf ϕ φ Φ φ 2 H s (Ω ) C max nf ϕ φ Φ φ 2 H s (Ω). (2.22) ℶ 17

18 Theorem Gven Condton 2.22, there exsts a constant C dependng only on C L, C L 1 and C max /C mn such that for (, α) ℶ ℵ, ψ,α H s 0 (Ω\Ω,n ) ψ,α 0 e n/c and ψ,α ψ,α n H s 0 (Ω) ψ,α e 0 n/c, n N. (2.23) Condton Gven the Constructon 2.20 wth values δ and h, and Condton 2.22, defne V := {f H s 0 (Ω) [φ,α, f] = 0 for (, α) ℶ ℵ}. There exsts a constant C l ℵ such that and D t f L 2 (Ω) C l h s t f H s 0 (Ω) for t {0, 1,..., s}, f V, (2.24) [φ,α, f] 2 ( C l f 2 L 2 (Ω) + h2s f 2 ) H0 s, f H (Ω) s 0 (Ω), (2.25) ℶ,α ℵ x 2 C l h 2s α ℵ x α φ,α 2 H s (τ ), ℶ, x Rℵ. (2.26) Theorem Assume that the operator L s local n the sense that ψ, ψ = 0 f ψ and ψ have dsont supports. Let (φ,α ) (,α) ℶ ℵ satsfy Condton 2.24 wth values δ and h from Constructon Then there exsts a constant C d,δ,s dependng only on d, δ and s such that Condton 2.22 holds true wth Cmn 1 C d,δ,s and C max C d,δ,s. In partcular, there exsts a constant C dependng only on d, δ, s, C L and C L 1 such that for (, α) ℶ ℵ, ψ,α H s 0 (Ω\Ω,n ) Ce n/c and ψ,α ψ n,α H s 0 (Ω) Ce n/c (2.27) For a local operator, Theorem 2.25 demonstrates that Condton 2.24 s suffcent to guarantee the localzaton of the gamblets. The followng result demonstrates that the examples of measurement functons that follow t satsfy the localty Condton Theorem The measurement functons of Examples 2.27, 2.28 and 2.29, satsfy Condton Example Let ℵ = 1 and ℶ = m. Let δ, h (0, 1). Let τ 1,..., τ m be a partton of Ω such that each τ s Lpschtz, convex, contaned n a ball of radus h and contans a ball of radus δh. Let φ = 1 τ / τ where 1 τ s the ndcator functon of τ. Example Let δ, h (0, 1). Let τ 1,..., τ ℶ be a partton of Ω such that each τ s Lpschtz, convex, contaned n a ball of radus h and contans a ball of radus δh. Let ℵ = ( ) s+d 1 d. For {1,..., ℶ }, let (φ,α ) α ℵ be an L 2 (τ ) orthonormal bass of P s 1 (τ ), the space of d-varate polynomals on τ of degree at most s 1. Example Assume that s > d/2. Let δ, h (0, 1). Let τ 1,..., τ m be a partton of Ω such that each τ s convex, unformly Lpschtz, contans a ball of center x radus δh and s contaned n a ball of center x and radus h. Let ℵ = 1 and φ (x) := h d/2 δ(x x ). 18

19 2.4 Fast Gamblet Transform When A s sparse (e.g. banded) then Algorthm 1 can be accelerated by usng sparse matrces π (k 1,k) and W (k). For nstance, the matrces π (k 1,k) and W (k) can be chosen to be cellular, as defned n Condton Under these sparsty condtons, we ntroduce n Secton 7 the Fast Gamblet Transform (Algorthm 6) obtaned by truncatng the stffness matrces A (k) and localzng the computaton of the vectors ψ (k) n Algorthm 1. More precsely, f A has O(N) non-zero entres and under the graph dstance (Defnton 2.21) nduced by A on I (q), a ball of radus ρ has O(ρ d ) elements, and f Condtons 3.23, 2.15, and 7.5 are satsfed, then Theorem 7.6 proves that the complexty of the Fast Gamblet Transform s at most O(N ln 3d N) to compute the gamblets ψ (k), χ (k) and ther stffness matrces A (k) and B (k), and O(N ln d+1 N) to nvert the lnear system (2.7) and perform the decomposton (2.8) up to A -norm accuracy O(H q ). The mechansm behnd ths acceleraton s based on the localzaton (exponental decay) of each gamblet ψ (k) away from. Item 2 of Condton 7.5, whch provdes a suffcent condton for ths exponental decay, s the matrx verson of Condton Returnng to Sobolev spaces, recall that Theorem 2.26 asserts that Condton 2.24 s satsfed for a large common class of measurement functons. Moreover, when L s local Theorem 2.25 asserts that Condton 2.24 mples that Condton 2.22 s satsfed, whch s a condton on the range space and does not depend on the operator tself. Consequently, as wth Condton 2.3 for matrx systems, f A s obtaned as the dscretzaton of a local contnuous becton L, then the equvalence between Item 2 of Condton 7.5 and Condton 2.22 mentoned above mples Item 2 of Condton 7.5 s a condton on the mage space of that operator (ndependent from the operator tself) and the matrces π (k,k+1), that s, Item 2 of Condton 7.5 s naturally satsfed for contnuous local bectons on Sobolev spaces. 2.5 Non dvergence form and non symmetrc operators Let L : H0 s(ω) L2 (Ω) be a contnuous lnear becton. A prototypcal example of L s the non-dvergence form operator Lu = a α (x)d α u for u H0(Ω) s (2.28) 0 α s where a s a tensor wth entres n L (Ω) such that L 1 s well defned and contnuous. The nverse problem Lu = g s equvalent to L L = L g where L s the adont of L. Let Q 1 be the lnear operator mappng H0 s(ω) to H s (Ω) defned as Q 1 := L L. Observe that Q 1 and Q are contnuous and, by applyng the results of Subsectons 2.2, 2.3 and 2.4 to the operator Q 1, the nverse problem Lu = g can be solved n O(N polylog N)-complexty by the solvng the lnear system Q 1 u = L Lu = L g usng gamblets defned by the norm nduced by Q,.e. by usng gamblets mnmzng the norm where u = [Q 1 u, u] 1 2 = Lu L 2 (Ω) for u H0 s (Ω) and measurement functons satsfyng Condton 2.22 (as n Example 2.28). 19

20 3 The gamblet transform on a Banach space 3.1 Settng For any topologcal vector space V, we wrte the dual parng between V and t topologcal dual V by [, ]. Let (B, ) be a reflexve separable Banach space such that the norm s quadratc,.e. u 2 = [Q 1 u, u] for u B, and Q s a symmetrc postve bectve lnear operator mappng B to B ([v, Qw ] = [w, Qv ] and [v, Qv ] 0 for v, w B ). Wrte, the correspondng nner product on B defned by u, v := [Q 1 u, v] for u, v B. (3.1) Although B s also a Hlbert space under the quadratc norm, we wll keep usng the Banach termnology to emphasze the fact that our dual parngs wll not be based on the, scalar product but on nonstandard (va the Resz representaton) realzaton of the dual space. Let (B 0, 0 ) be a Banach subspace of B such that the natural embeddng : B 0 B s compact and dense. We summarze ths embeddng n the followng dagram Q 1 B B ( B 0 ) (3.2) Q Wrte, the scalar product on B defned by φ, φ = [φ, Qφ] and let be the correspondng norm. Observe that s the natural norm nduced by dualty on B,.e. φ = sup v B, v 0 [φ, v]/ v for φ B, and Q and Q 1 are sometres n the sense that Q 1 u = u for u B. Example 3.1. As a smple prototypcal runnng example we wll consder B = H0 1(Ω) and B 0 = L 2 (Ω) wth the followng specfcatons. Ω s a bounded open subset of R d (of arbtrary dmenson d N ) wth pecewse Lpschtz boundary. 0 = L 2 (Ω) and u 2 = u 2 a = Ω ( u)t a u s the energy norm defned by a unformly ellptc d d symmetrc matrx a wth entres n L (Ω) (.e. satsfyng λ mn (a) l 2 l T a(x)l λ max (a) l 2 for l R d wth some strctly postve fnte constants λ max (a) and λ mn (a)). Note that for ths example the nduced dual space B can be dentfed wth H 1 (Ω) wth the norm ϕ = sup v H 1 0 (Ω) Ω ϕv/ v a. Under ths dentfcaton the nduced operator Q 1 s the dfferental operator dv(a ). Example 3.2. As another runnng example we wll consder (B 0, 0 ) = (L 2 (Ω), L 2 (Ω)) and (B, ) = (H0 s(ω), ), where u 2 = [Q 1 u, u] and Q s a symmetrc contnuous lnear becton mappng H s (Ω) to H0 s (Ω) such that, for some constant C e 1, Ce 1 u H s 0 (Ω) u C e u H s 0 (Ω), for u H0(Ω) s. (3.3) Note that usng [90, Thm. 2.2] the left and/or rght hand sdes of (3.3) could be replaced by u H s (Ω) modulo changes n norm-equvalence constants. Note that for ths example the nduced dual space B can be dentfed wth H s (Ω) wth the nduced norm φ =, and (3.3) mples that sup v H s 0 (Ω) [φ,v] v C 1 e φ H s (Ω) φ C e φ H s (Ω), for φ H s (Ω) (3.4) 20

21 Fgure 1: See Illustraton 3.5. Ω = (0, 1) 2. Ω k corresponds to a unform partton of Ω nto 2 k 2 k squares. The bottom row shows the support of φ (1), φ (2) and φ (3) s. Note that (1) = s (1) = and s (2) =. The top row shows the entres of π (1,2), and π (2,3),. 3.2 Herarchy of nested measurements Let q N { }. Let I (q) be an ndex tree of level q defned as n 2.1. For a fnte set J wrte J := Card(J ) the number of elements of J. Constructon 3.3. For k {1,..., q 1} let π (k,k+1) be a I (k) I (k+1) matrx of rank I (k). Note that 3.3 does not requre the orthonormalty condton π (k,k+1) (π (k,k+1) ) T = I (k) of 2.2. Constructon 3.4. Let (φ (q) ) I (q) be lnearly ndependent elements of B. For k {1,..., q 1} and I (k) let φ (k) B be defned by nducton va φ (k) =, φ (k+1). (3.5) I (k+1) π (k,k+1) Observe that rank(π (k,k+1) ) = I (k) mples the lnear ndependence of the elements (φ (k) ) I (k). For k {1,..., q}, wrte Φ (k) := span{φ (k) I (k) }. (3.6) Observe that (3.5) mples that the spaces Φ (k) are nested,.e., Φ (k) Φ (k+1). 21

22 Illustraton 3.5. As a runnng llustraton consder Example 3.1 wth Ω = (0, 1) 2 llustrated n Fgure 1. For k N, let Ω k be a regular grd partton of Ω nto 2 k 2 k squares τ (k) and let φ (k) = 1 τ (k) τ (k) where 1 τ (k) s the ndcator functon of τ (k) and τ (k) s the volume of τ (k). The nestng of the ndcator functons mples that of the measurement functons,.e. (3.5). In ths partcular example, the nestng matrces π (k,k+1) are also cellular (n the sense that π (k,k+1), = 0 for (k) ) and orthonormal (n the sense that π (k,k+1) (π (k,k+1) ) T = I (k) where I (k) s the I (k) I (k) dentty matrx). Note that the measurement functons φ (k) form a multresoluton decomposton of L 2 (Ω). Example 3.6. Consder, for Example 3.2, the measurement functons ntroduced n Example Observe that the measurement functons (φ (k) ) I (k) are nested as n (3.5) and satsfy Ω φ(k) φ (k) = δ, for, whch mples π (k,k+1) (π (k,k+1) ) T = I (k). Observe also that the matrces π (k,k+1) are cellular n the sense of Condton 2.15,.e. π (k,k+1), = 0 for (k),ℵ ℵ. Note also that the matrces W (k) can naturally be chosen so that W (k) W (k),t = J (k) and W (k), = 0 for (k),ℵ (k),ℵ. Illustraton 3.5 presents a partcular nstance of the proposed measurement functons for s = The Gamblet Transform Due to ther game theoretc orgn and nterpretaton (presented n Secton 5), we wll refer to the followng defned herarchy of elements ψ (k) as gamblets. Defnton 3.7. (Gamblets) For k {1,..., q} and I (k), let ψ (k) be the mnmzer of { Mnmze ψ Subect to ψ B and [φ (k), ψ] = δ, for I (k). (3.7) For k {1,..., q} let Θ (k) be the symmetrc postve defnte I (k) I (k) matrx defned by Θ (k), := [φ(k), Qφ (k) ], (3.8) and wrte Θ (k), 1 := (Θ (k) ) 1. The gamblets have the followng explct form. Theorem 3.8. For k {1,..., q} and I (k), we have ψ (k) = Θ (k), 1, I (k) Qφ (k). (3.9) Illustraton 3.9. In the context of Example 3.1 and Illustraton 3.5, Fgure 2 provdes a numercal llustraton of the gamblets ψ (k) (1 k 7) usng the Dscrete Gamblet Transform descrbed n Subsecton

23 Fgure 2: Gamblets ψ (k) for 1 k 7. See Illustraton 3.9. The followng proposton provdes the lnk between gamblets and orthogonal proecton n the mmedately followng Theorem We say that a lnear operator Q : B B s postve symmetrc f [ϕ 1, Qϕ 2 ] = [ϕ 2, Qϕ 1 ] and [ϕ 1, Qϕ 1 ] 0 for ϕ 1, ϕ 2 B. When such a Q s a contnuous becton t determnes a Hlbert space nner product by ϕ 1, ϕ 2 Q := [ϕ 1, Qϕ 2 ]. Proposton Let B be a separable Banach space, and B be a realzaton ts dual wth [, ] the correspondng dual parng. For a postve symmetrc becton Q : B B, consder the Hlbert space B equpped wth the nner product u 1, u 2 := [Q 1 u 1, u 2 ] and the Hlbert space B equpped wth the nner product ϕ 1, ϕ 2 := [ϕ 1, Qϕ 2 ]. Consder a collecton φ 1,..., φ m of m lnearly ndependent elements of B, and let Φ B denote ts span. Defne the Gram matrx Θ by and the elements ψ B, = 1,... m by Θ := [φ, Qφ ],, = 1,... m, (3.10) ψ := m =1 Θ 1 Qφ, = 1,..., m. (3.11) The collecton {φ, ψ,, = 1,... m} s a borthogonal system, n that [φ, ψ ] = δ,, = 1,... m. Moreover, the operator P : B B defned by P := m =1 ψ φ s the orthogonal proecton onto QΦ, and the operator P : B B defned by P := m =1 Q 1 ψ Qφ s the orthogonal proecton onto Φ. In addton, P s the adont of P n the sense that [ϕ, P ψ] = [P ϕ, ψ], ϕ B, ψ B, and we have P = Q 1 P Q. Consequently, we can characterze the gamblets as components of an orthogonal 23

24 proecton. For k {1,..., q}, wrte V (k) := span{ψ (k) I (k) }. (3.12) Theorem We have V (k) = QΦ (k) and V (k) V (k+1). Furthermore, the mappng u (k) : B B defned by u (k) (u) :=, u]ψ (k) (3.13) I (k) [φ (k) s the orthogonal proecton of B onto V (k) and therefore has the varatonal formulaton u u (k) (u) = nf u v, u B. (3.14) v V (k) Fgure 3: See Illustratons 3.5 and Entres of W (2) t,, W (2) l, and W (2) r, wth t (1) = l (1) = r (1) =. For 1 < r < k and a k-tuple of the form = ( 1,..., k ) we wrte (r) := ( 1,..., r ). For k = 2,..., q let J (k) be defned as n 2.7. Constructon For k = 2,..., q let W (k) be a J (k) I (k) matrx such that Im(W (k),t ) = Ker(π (k 1,k) ). Note that Constructon 3.12 does not requre the orthonormalty condton W (k) W (k),t = J (k) of Constructon 2.8. Illustraton We refer to Fgure 3 for an llustraton of the entres of W (k) n the context dscussed n Illustraton 3.5 and Fgure 1. Note that In ths partcular example, the matrces W (k) are also cellular (n the sense that W (k), = 0 for (k) ) and orthonormal (n the sense that W (k) (W (k) ) T = J (k) where J (k) s the J (k) J (k) dentty matrx). Defnton For k {2,..., q} and J (k), wrte := χ (k) W (k), ψ(k) I (k) 24 (3.15)

25 and W (k) := span{χ (k) J (k) }. (3.16) Fgure 4: Gamblets ψ (1) and orthogonalzed gamblets χ (k) for 2 k 7. See Illustraton Illustraton In the context of Example 3.1 and Illustratons 3.5 and 3.13, Fgure 2 provdes a numercal llustraton of the Gamblets ψ (1) and orthogonalzed gamblets χ (k) (2 k 7) usng the Dscrete Gamblet Transform descrbed n Subsecton 3.6. Wrte W (q+1) for the, -orthogonal complement of V (q) n B. Theorem For k {2,..., q}, W (k) s the orthogonal complement of V (k 1) n V (k) wth respect to the scalar product,,.e. wrtng the, -orthogonal drect sum, we have B = V (1) W (2) W (q) W (q+1). (3.17) Furthermore, u = u (1) (u) + (u (2) (u) u (1) )(u) + + (u (q) (u) u (q 1) (u)) + (u u (q) (u)) s the, -orthogonal decomposton of u B n correspondng to (3.17). Illustraton Fgure 5 provdes, n the context of Example 3.1 and Illustratons 3.5 and 3.13, an llustraton of the Gamblet Transform descrbed n Subsecton 3.3. Observe that the Gamblet Transform turns the herarchy of nested measurements functons φ (k) nto the herarchy of nested gamblets ψ (k). As t s commonly done wth nested wavelets [63, 150], the gamblets are then orthogonalzed nto the elements χ (k). These new gamblets enable the multresoluton decomposton of any element u B that could not only be 25

26 Fgure 5: The Gamblet Transform. employed for desgnng fast solvers, but also for operator compresson, PCA or actve subspace analyss. Observe that the transformaton of the measurement functons φ (k) ) B nto elements ψ (k) B s also a generc method for automatcally producng large and new classes of wavelets that are adapted to the space (B, ). More precsely we wll show n Subsecton 3.4 that the Gamblet transform turns a multresoluton decomposton of the compact embeddng (B 0, 0 ) (B, ) nto a multresoluton decomposton of the operator Q 1 : (B, ) (B, ), whch n the context of Example 3.1 and Illustraton 3.5, corresponds to turnng the multresoluton decomposton of the compact embeddng (L 2 (Ω), L 2 (Ω)) (H 1 (Ω), ) nduced by Haar bass functons, nto the multresoluton decomposton of the operator dv(a ) : (H0 1(Ω), a) (H 1 (Ω), ). 3.4 Bounded condton numbers Let A (k) be the I (k) I (k) stffness matrx defned by A (k), = ψ (k), ψ (k) (3.18) Let B (k) be the J (k) J (k) (stffness) matrx B (k), := χ (k), χ (k). Observe that B (k) = W (k) A (k) W (k),t (3.19) 26

27 For k {2,..., q} let Φ (k),χ := {φ Φ (k) φ = x φ (k) wth x Ker(π (k 1,k) )} (3.20) I (k) Let I (k) be the dentty I (k) I (k) matrx and let J (k) be the dentty J (k) J (k) matrx. Condton There exsts some constants C Φ 1 and H (0, 1) such that 1. sup φ B0 φ φ 0 C Φ C Φ H k φ nf φ Φ (k) φ 0 for k {1,..., q}. 3. sup ϕ B0 nf φ Φ (k) ϕ φ ϕ 0 C Φ H k for k {1,..., q}. 4. sup φ Φ (k),χ φ φ 0 C Φ H k 1 for k {2,..., q} C Φ x 2 I (k) x φ (k) 2 0 C Φ x 2 for k {1,..., q} and x R I(k). 6. C 1 Φ J (k) W (k) W (k),t C Φ J (k) for k {2,..., q}. We wrte Cond(X) := λmax(x) λ mn (X) matrx X. for the condton number of a symmetrc postve Theorem Under Condton 3.18 t holds true that there exsts a constant C dependng only on C Φ such that u u (k) (u) CH k Q 1 u 0, (3.21) for u QB 0 and k {1,..., q}. Furthermore, C 1 I (1) A (1) CH 2 I (1), Cond(A (1) ) CH 2, and C 1 H 2(k 1) J (k) B (k) CH 2k J (k) (3.22) for k {2,..., q}. In partcular, Cond(B (k) ) CH 2. (3.23) Illustraton Fgure 6 provdes, n the context of Example 3.1 and Illustratons 3.5 and 3.13, an llustraton of the condton numbers of the matrces A (k) and B (k). We defne contrast as the rato λ max (a)/λ mn (a). Fgure 7 provdes an llustraton of the ranges of the egenvalues of A (7), A (1), B (2),..., B (7). Whle the subspaces V (1), W (2), W (3),... are not exact egenspaces (e.g. n the context of Example 3.1 they are not orthogonal n L 2 (Ω) and the angle between two successve subspace s of the order of a power of H) they retan several mportant characterstcs of egensubspaces: (1) Theorem 3.16 shows 27

28 Fgure 6: Condton numbers of A (k) and B (k), n log 10 scale, for 1 k 7. Illustraton See that they are orthogonal n the, scalar product (e.g., for Example 3.1, n the scalar product assocated wth the energy norm a ) (2) Theorem 3.19 (and Fgure 7) shows that the ranges of egenvalues of the operator Q wthn each subspace defne ntervals of unformly bounded lengths n log scale (3) [177] shows that the proectons of the solutons of the hyperbolc and parabolc versons of 7 on subspaces W (k) (obtaned usng, possbly complex valued gamblets, that are non only adapted to the coeffcents of the PDE but also to the mplct numercal scheme used for ts resoluton) produces space-tme multresoluton decompostons of those solutons (the evoluton of ther proected soluton on W (k) s slow for k small and fast for k large). In that sense, gamblets nduce a multresoluton decomposton of B that s, to some degree, adapted to the egen-subspaces of the operator Q. 28

29 Fgure 7: Ranges of the egenvalues of A (7), A (1), B (2),..., B (7), n log 10 scale. Illustraton See 3.5 Herarchcal computaton of gamblets Wrte B (k), 1 the nverse of B (k). Let N (k) be the I (k) J (k) matrx defned by N (k) := A (k) W (k),t B (k), 1, (3.24) and wrte N (k),t ts transpose. Wrte π (k+1,k) the transpose of π (k,k+1). rank(π (k,k+1) ) = I (k) mples that π (k,k+1) π (k+1,k) s nvertble. Let π (k,k+1) be the I (k) I (k+1) matrx defned as the pseudo-nverse of π (k+1,k),.e. π (k,k+1) = (π (k,k+1) π (k+1,k) ) 1 π (k,k+1). (3.25) Snce the spaces V (k) are nested there exsts a I (k) I (k+1) matrx R (k,k+1) such that for 1 k q 1 and I (k) ψ (k) =, ψ (k+1) (3.26) I (k+1) R (k,k+1) It s natural, usng an analogy wth multgrd theory, to refer to R (k,k+1) as the restrcton matrx and to ts transpose R (k+1,k) := (R (k,k+1) ) T as the nterpolaton/prolongaton matrx. The followng theorem s the bass of Algorthm 2, whch descrbes how gamblets are computed n a nested manner (from level q to level 1) by solvng well condtoned lnear systems. Theorem It holds true that for k {1,..., q 1} and I (k), ψ (k) = π (k,k+1),l ψ (k+1) l ( π (k,k+1) N (k+1) ), χ (k+1). (3.27) l I (k+1) J (k+1) In partcular, R (k,k+1) = π (k,k+1) (I (k+1) N (k+1) W (k+1) ). (3.28) 29

30 Algorthm 2 Herarchcal computaton of gamblets. 1: A (q), = ψ (q), ψ (q) // Level q, I (q) I (q) stffness matrx 2: for k = q to 2 do 3: B (k) = W (k) A (k) W (k),t // Eq. (3.19) 4: For J (k), χ (k) = (k) I (k) W, ψ(k) // Eq. (3.15) 5: π (k 1,k) = (π (k 1,k) π (k,k 1) ) 1 π (k 1,k) // Eq : N (k) = A (k) W (k),t B (k), 1 // Eq. (3.24) 7: R (k 1,k) = π (k 1,k) (I (k) N (k) W (k) ) // Eq. (3.28) 8: A (k 1) = R (k 1,k) A (k) R (k,k 1) // Eq. (10.9) 9: For I (k 1), ψ (k 1) = I (k) R(k 1,k), ψ (k) // Eq. (3.26) 10: end for 3.6 Dscrete Gamblet transform Let u B. For k {1,..., q} let b (k) R I(k) that (3.26) mples b (k 1) = R (k 1,k) b (k) ). Let v (1) := w (1) ψ (1) I (1) be defned by b (k) := ψ (k), u (observe (3.29) where w (1) R I(1) s the soluton of A (1) w (1) = b (1). (3.30) Let where w (k) R J (k) v (k) := s the soluton of w (k) J (k) χ (k) (3.31) B (k) w (k) = W (k) b (k). (3.32) The followng theorem s the bass of Algorthm 3, whch descrbes how the gamblet transform of u B,.e. the decomposton of u B over V (1) W (2) W (q), can be obtaned by solvng q ndependent and well condtoned (under Condton 3.18, by Theorem 3.19) lnear systems. Theorem Let u B and, for k {1,..., q}. It holds true that u (q) (u) = q k=1 v(k) where the v (k) are defned n (3.29) and (3.31) va the solutons of the (ndependent) lnear systems (3.30) and and (3.32). Furthermore v (1) = u (1) (u) and for k {2,..., q}, v (k) = u (k) (u) u (k 1) (u). 30

31 Algorthm 3 Gamblet transform of u B. 1: For I (q), b (q) = ψ (q), u 2: for k = q to 2 do 3: w (k) = B (k), 1 W (k) b (k) // Eq. (3.32) // Thm. 3.22, v (k) := u (k) u (k 1) W (k) χ (k) 4: v (k) = J (k) w(k) 5: b (k 1) = R (k 1,k) b (k) // Eq. (3.26) 6: end for 7: w (1) = A (1), 1 b (1) // Eq. (3.30) 8: v (1) = I (1) w(1) ψ (1) // Thm : u (q) (u) = v (1) + v (2) + + v (q) // Thm Dscrete gamblet transform/computaton When B s nfnte dmensonal the practcal applcaton of the gamblet transform may requre ts dscretzaton. Algorthm 2 and 3 only requre the specfcaton of these level q gamblets and ther stffness matrces for ther applcatons. Therefore, when explct/analytcal formulas are avalable for level q gamblets ψ (q) ths dscretzaton can naturally be done on the correspondng (meshless) gamblet bass. However, when such formulas are not avalable the gamblet transform must be appled to a pror dscretzaton B d of the space B. Ths dscretzaton can be done usng lnearly ndependent bass elements (Ψ ) N spannng B d := span{ψ N }, (3.33) a fnte-dmensonal subspace of B. N s a fnte-set and we wrte N := N. Let I (q) be an ndex tree of depth q (as n Def. 2.1) relabelng N. For k {2,..., q} let π (k 1,k) and W (k) be as n Constructon 3.3 and Algorthm 4 descrbes the nested calculaton of dscrete gamblets from the bass functons (Ψ ) N and the correspondng gamblet transform of u B d. Algorthm 4 Dscrete Gamblet computaton and transform of u B d. 1: For I (q), ψ (q) = Ψ // Level q gamblets 2: Use Algorthm 2 to compute the gamblets ψ (k), χ (k) and matrces A (k), B (k) and R (k 1,k) 3: Use Algorthm 3 to compute the gamblet transform of u // u = u (q) (u) = v (1) + v (2) + + v (q) Let A be the I (q) I (q) stffness matrx defned by Defne π (k,q) as n (2.2). A, = Ψ, Ψ. (3.34) 31

32 Condton There exsts constants C d 1 and H (0, 1) such that the followng condtons are satsfed. 1. Cd 1 J (k) W (k) W (k),t C d J (k) for k {2,..., q}. 2. C 1 d I (k) π (k,q) π (q,k) C d I (k) for k {1,..., q 1}. 3. sup x R I (q) nf (x π (q,k) y) T A 1 (x π (q,k) y) y R I(k) x 1} C d λmn (A) Hk nf y R I (k) 5. sup y Ker(π (k 1,k) ) y T π (k,q) A 1 π (q,k) y y y T π (k,q) A 1 π (q,k) y y C d λmn (A) Hk for k {1,..., q for k {1,..., q}. C d λmn (A) Hk 1 for k {2,..., q}. Theorem Let A (1) and (B (k) ) k {2,...,q} be the matrces computed n Algorthm 4. Under Condtons 3.23 there exsts a constant C dependng only on C d such that C 1 I (1) A(1) λ mn (A) CH 2 I (1) and Cond(A (1) ) CH 2. Furthermore, for k {2,..., q} t holds true that C 1 H 2(k 1) J (k) B(k) λ mn (A) CH 2k J (k) and Cond(B (k) ) CH 2. Remark If π (k,q) π (q,k) s a multple of dentty for k {1,..., q 1} then tems 3 and 5 of Condton 3.23 are equvalent to x T A 1 C x d λmn (A) Hk x for x Ker(π (q,k) ) and k {1,..., q 1}. Fgure 8: The fne grd and a n log 10 scale. See Illustraton Illustraton Our runnng numercal example s obtaned from the numercal dscretzaton of Example 3.1. More precsely we consder the unform grd of Ω = (0, 1) 2 wth 2 q 2 q nteror ponts (q = 7) llustrated n Fgure 8. a s pecewse constant on 32

33 Fgure 9: I (1), I (2) and I (3). See Illustraton each square of that grd, and gven by a(x) = ( 7 k= cos ( 2 k π( 2 q q +1 )))( sn ( 2 k π( 2 q q +1 ))), as llustrated n log 10 scale n Fgure To construct the herarchy of ndces I (k) we partton the unt square nto nested sub-squares τ (q) of sde 2 k as llustrated n Fgure 9 and we label each node I (q) of the fne mesh (of resoluton 2 q ) by the square τ (q) of sde 2 q contanng that node, and we determne the herarchy of labels through set ncluson:.e. for I (q), (k) I (k) s the label of the square of sde 2 k contanng. The fnte-element dscretzaton of B s obtaned usng contnuous nodal blnear bass elements Ψ spanned by {1, x 1, x 2, x 1 x 2 } n each square of the fne mesh. As descrbed n Algorthm 4, these fne mesh blnear fnte elements form our level q gamblets (.e. ψ (q) = Ψ). Although the contnuous Gamblet Transform reles on the specfcaton of the measurement functons φ (k), the dscrete Gamblet Transform only requres the specfcaton of the matrces π (k,k+1) and W (k). For our numercal example these matrces are the same as those presented n a contnuous settng n Illustratons 3.5 and We refer to Fgures 2 and 4 for the correspondng llustratons of the gamblets ψ (k) and χ (k). We refer to Fgures 6 and 7 for the correspondng llustratons of the condton numbers of A (k), B (k) and the ntervals contanng ther egenvalues. 4 Operator nverson wth the gamblet transform Here we dscuss the gamblet transform n the context of nverse problems and ts relatonshp wth the choce of error norms n the context of nonstandard dual parngs. 4.1 The nverse problem and ts varatonal formulaton Let (B 2, 2 ) be a separable Banach space and let L : B B 2 be a contnuous lnear becton. Snce L s contnuous, and therefore bounded, t follows from the open mappng 33

34 theorem that L 1 s also bounded. Let us defne ts contnuty constants to be L 1 g C L 1 := sup g B 2 g 2 For a gven g B 2, consder solvng the nverse problem Lv 2 and C L := sup v B v. (4.1) Lu = g (4.2) for u B. Snce Q 1 u = Q 1 L 1 g and u, v = [Q 1 u, v], (4.2) s equvalent to the weak formulaton (n the error norm ) u, v = [Q 1 L 1 g, v] for v B. (4.3) Example 4.1. We wll consder the followng prototypcal PDE as a runnng llustratve example, { dv ( a(x) u(x) ) = g(x) x Ω; (4.4) u = 0 on Ω, where Ω s as n Example 3.1, and a s a unformly ellptc d d matrx (whch may or may not be symmetrc) wth entres n L (Ω). We defne λ mn (a) as the largest constant and λ max (a) as the smallest constant such that for all x Ω and l R d, λ mn (a) l 2 l T a(x)l and l T a T (x)a(x)l ( λ max (a) ) 2 l 2. (4.5) 4.2 Identfcaton of Q and varatonal formulaton of Lu = g The practcal applcaton of the gamblet transform to the resoluton of (4.3) requres the dentfcaton of the operator Q dentfyng the error norm General case The dentfcaton of the operator Q can be done by selectng G, a self-adont postve contnuous lnear becton from B 2 onto B2 and D, a contnuous lnear becton from B2 onto B such that DGL s self-adont and postve. Q s then defned by Q = L 1 G 1 D 1,.e. Q 1 = DGL (4.6) makng the followng dagram B Q 1 L B 2 B B 2 D G (4.7) commutatve. Throughout the rest of ths paper all such dagrams wll be commutatve. G can, a pror, be chosen ndependently from L and the defnton u 2 = [Q 1 u, u] leads to u 2 = [DGLu, u] (4.8) 34

35 The varatonal formulaton of the equaton Lu = g s then [Q 1 u, Ψ] = [DGg, Ψ] for Ψ B (4.9) whch can be wrtten (usng the scalar product u, Ψ = [Q 1 u, Ψ]) u, Ψ = [DGg, Ψ] for Ψ B (4.10) Example 4.2. Consder the prototypcal Example 4.1 and assume a to be symmetrc. Let B = H0 1(Ω) and defne as the energy norm u 2 = u 2 a := Ω ( u)t a u (as n Example 3.1). Let B 2 = H 1 (Ω) and let g 2 = g H 1 (Ω) = sup v H 1 0 (Ω) Ω gv v H 1 0 (Ω) where v 2 H0 1(Ω) := Ω v 2. Observe that B2 = H1 0 (Ω) and the (dual) norm on B 2 s v 2 = v H0 1 (Ω). The gamblet transform for the PDE (4.4) can be defned by (1) consderng the operator L = dv(a ) mappng B onto B 2 (2) takng D = and G = D 1 = 1 n Dagram 4.7, where D = s the Laplace-Drchlet operator mappng H0 1(Ω) onto H 1 (Ω) (note that v 2 2 = [G 1 v, v]). Under these choces Q 1 s (as n Example 3.1) the operator dv(a ) llustrated n the followng dagram H 1 0 (Ω) dv(a ) H 1 (Ω) (4.11) Q 1 1 H 1 (Ω) H 1 0 (Ω). The varatonal formulaton of the PDE (4.4) can then be wrtten ( Ψ) T a u = Ψg for Ψ H0 1 (Ω). (4.12) Ω Ω Observe that by placng the norm g H 1 (Ω) on B 2 we have C L λ max (a) and C L 1 1/ λ mn (a). Example 4.3. Let L be a contnuous, symmetrc, postve lnear becton from H0 s(ω) to H s (Ω) and defne Q := L 1 to be ts nverse. Wrte u := [Lu, u] 1 2 and let be the dual norm of. Defne (B, ) and (B, ) as n Example 3.2. Consder the nverse problem (4.2). Let B 2 = H s (Ω) and let g 2 = g H s (Ω). Observe that B2 = Hs 0 (Ω) and the (dual) norm on B 2 s v 2 = v H0 s(ω). The gamblet transform for the nverse problem (4.2) can then be defned by consderng the operator L mappng B onto B 2 (2) takng D = ( ) s and G = D 1 n Dagram 4.7, where ( ) s s the operator defned by the s-th terate of the Laplacan mappng H0 s(ω) to H s (Ω). Under these choces Q 1 s the operator L as llustrated n the followng dagram H s 0 (Ω) L H s (Ω) (4.13) Q 1 H s (Ω) H s 0 (Ω). ( ) s G 35

36 Lemma 4.4. The norm nequaltes (3.3) n Example 3.2 are equvalent to the contnuty of the operator L (mappng H s 0 (Ω) to H s (Ω)) and of ts nverse n Example 4.3. Moreover, let us ntroduce two-parameter versons C 1 e u H s 0 (Ω) u C e,2 u H s 0 (Ω), for u H s 0(Ω) (4.14) of (3.3) n Example 3.2 and ts consequence C 1 e φ H s (Ω) φ C e,2 φ H s (Ω), for φ H s (Ω). (4.15) Then, f C e and C e,2 are the smallest constants such that 4.14 holds, then C 2 e,2 = C L, C 2 e C L 1, and C e C L C 2 L 1. Example 4.5. A partcular nstance of Example 4.3 s the self-adont dfferental operator L mappng H0 s(ω) to H s (Ω) and defned by Lu = ( 1) α D α (a α,β (x)d β u), for u H0(Ω) s, (4.16) 0 α, β s where a s symmetrc wth entres n L (Ω) and α, β are d dmensonal mult-ndces α = (α 1,..., α d ) (wth α N and α = d =1 α ). Lemma 4.4 mples that (3.3) s equvalent to the contnuty of L and L 1. The contnuty of L (rght nequalty n (3.3)) follows from the unform bound on the entres of a. The left hand sde of (3.3) s a classcal coercvty condton (ensurng the well-posedness of (4.2)) and we refer to [1] for ts characterzaton General case wth D = L Wrte L the adont of L defned as the operator mappng B 2 onto B such that for (v, w ) B B 2, [w, Lv] = [L w, v]. Consder the general case dscussed n Subsecton and take D = L. In that case, we have Q 1 = L GL,.e. Q = L 1 G 1 L 1,, (4.17) and Dagram 4.7 reduces to B L B 2 (4.18) Q 1 G B B 2. L Furthermore, u 2 = [GLu, Lu] and the varatonal formulaton of the equaton Lu = g s then [Q 1 u, Ψ] = [Q 1 L 1 g, Ψ], for Ψ B, (4.19) whch can be wrtten (usng the scalar product u, Ψ = [Q 1 u, Ψ] = [GLu, LΨ]) u, Ψ = [L Gg, Ψ], for Ψ B. (4.20) 36

37 Example 4.6. Consder the prototypcal Example 4.1. The gamblet transform for the PDE (4.4) can be defned by (1) consderng the operator L = dv(a ) mappng B = H 1 0 (Ω) onto B 2 = H 1 (Ω) (2) takng G = 1 n Dagram 4.18,.e. the nverse of, the Laplace-Drchlet operator mappng H 1 0 (Ω) onto H 1 (Ω). Under these choces Q 1 s the operator dv(a T ) 1 dv(a ) llustrated n the followng dagram H 1 0 (Ω) dv(a ) H 1 (Ω) (4.21) Q 1 1 H 1 (Ω) H 1 0 (Ω). dv(a T ) Furthermore, s the flux norm u = u flux ntroduced n [23] (and generalzed n [220, 240], see also [20] for ts applcaton to low rank approxmaton wth hgh contrast coeffcents), defned by u 2 flux = 1 Lu 2 L 2 (Ω) (recall that [23] u flux = (a u) (L pot where (a u) 2 (Ω)) d pot s the potental part of the vector feld a u). The varatonal formulaton of the PDE (4.4) can then be wrtten (a Ψ) T pot(a u) pot = ( Ψ) T a T 1 g for Ψ H0 1 (Ω). (4.22) Ω Wrtng u 2 H 1 0 (Ω) := Ω u 2, recall [23] that for u H 1 0 (Ω), Ω λ mn (a) u H 1 0 (Ω) u flux λ max (a) u H 1 0 (Ω). (4.23) Observe also that f u s the soluton of (4.4) then u flux = 1 g H 1 0 (Ω). Snce the flux norm of u s ndependent from a, C L and C L 1 are also ndependent from a. In partcular placng the norm g 2 := g H 1 (Ω) = 1 g L 2 (Ω) on B 2 = H 1 (Ω) we have C L = 1 and C L 1 = 1). Therefore, under these choces, the effcency of the correspondng gamblet transform s robust to hgh contrast n a (e.g. the condtons numbers of the stffness matrx B (k) are unformly bounded ndependently from a). Example 4.7. Let L be a contnuous lnear becton from H s 0 (Ω) to H s (Ω). We do not assume L to be symmetrc. The nverse problem (4.2) s equvalent to L ( ) s L = L ( ) s g (4.24) Let Q = (L ( ) s L) 1 be the symmetrc operator mappng H s (Ω) to H0 s (Ω). Wrte u := [Q 1 u, u] and let be the dual norm of. Defne (B, ) and (B, ) as n Example 3.2. Consder the nverse problem (4.2). Let B 2 = H s (Ω) and let g 2 = g H s (Ω). Observe that B2 = Hs 0 (Ω). The gamblet transform for the nverse problem (4.2) can then be defned by consderng the operator L mappng B onto B 2 (2) takng D = L and G = ( ) s n Dagram 4.7. Under these choces Q 1 s the (contnuous, symmetrc, postve) operator L ( ) s L as llustrated n the followng 37

38 dagram H s 0 (Ω) L H s (Ω) (4.25) Q 1 ( ) s H s (Ω) H s 0 (Ω). L Note that, under the norm, the soluton u of (4.2) satsfes u = g H s (Ω) whch leads to the robustness of the correspondng gamblets wth respect to the coeffcents of L Self-adont case wth the energy norm From the general case dscussed n Subsecton 4.2.2, assume that B 2 = B, L s selfadont and postve defnte (.e. for v, w B, [Lv, w] = [Lw, v], [Lv, v] 0 and [Lvv, v] = 0 s equvalent to v = 0). Take G = L 1,.e. Q 1 = L. In that case s the energy norm u 2 = [Lu, u] on B and Dagram 4.18 reduces to L B B. (4.26) Q The varatonal formulaton of the equaton Lu = g can then be wrtten (usng the scalar product u, Ψ = [Lu, Ψ]) u, Ψ = [g, Ψ], for Ψ B. (4.27) Observe that, snce g 2 = g = Q 1 u = u, we have C L = 1 and C L 1 = 1. Example 4.8. Consder the prototypcal Example 4.1 when a = a T. The gamblet transform for the PDE (4.4) can be defned by (1) consderng the (self-adont) operator L = dv(a ) mappng B = H 1 0 (Ω) onto B 2 = B 1 = H 1 (Ω) (2) takng G = L 1 n Dagram Under these choces Q 1 = L and Dagram 4.18 reduces to H0 1(Ω) dv(a ) H 1 (Ω). (4.28) Q s the energy norm u = u a, defned by u 2 a := Ω ( u)t a u, for u H0 1 (Ω). Ω g 2 = g s the dual norm g = sup gv v H 1 0 (Ω) v a = u a. Therefore, C L = 1 and C L 1 = 1. The varatonal formulaton of the PDE (4.4) can then be wrtten as n (4.12) When B 2 = B 2 From the general case dscussed n Subsecton 4.2.2, assume that B 2 = B 2 and choose G := d to be the dentty operator on B 2. Then Q 1 = L L (.e. Q = L 1 L, 1 ), u 2 = [Lu, Lu] for u B, and Dagram 4.18 reduces to B L B 2 (4.29) Q 1 d B B 2. L 38

39 The varatonal formulaton of the equaton Lu = g can then be wrtten (usng the scalar product u, Ψ = [Lu, LΨ]) u, Ψ = [g, LΨ] for Ψ B. (4.30) Note that, by usng the norm g 2 = [g, g] on B 2, we have C L = 1 and C L 1 = 1. Example 4.9. Consder the prototypcal Example 4.1. The gamblet transform for the PDE (4.4) can be defned by (1) consderng the operator L = dv(a ) mappng B = {u H0 1(Ω) dv(a u) L2 (Ω)} onto B 2 = L 2 (Ω) (2) takng G = d as n Dagram Under these choces Q 1 = L L and u = Lu L 2 (Ω). The varatonal formulaton of the PDE (4.4) can then be wrtten dv(a Ψ) dv(a u) = ( Ψ) T a g for Ψ B. (4.31) Ω Note that, by takng 2 = L 2 (Ω), we have C L = C L 1 = 1. Ω Example Let L be a contnuous lnear becton from H0 s(ω) to L2 (Ω). nverse problem (4.2) s equvalent to The L L = L g (4.32) Let Q = (L L) 1 be the symmetrc operator mappng H s (Ω) to H0 s (Ω). Wrte u := Lu L 2 (Ω) = [Q 1 u, u] and let be the dual norm of. Defne (B, ) and (B, ) as n Example 3.2. Consder the nverse problem (4.2). Let B 2 = L 2 (Ω) and let g 2 = g L 2 (Ω). Observe that B2 = B 2. The gamblet transform for the nverse problem (4.2) can then be defned by consderng the operator L mappng B onto B 2 (2) takng D = L and G = I d n Dagram 4.7. Under these choces Q 1 s the operator L L (wth the assocated norm u = Lu L 2 (Ω)) as llustrated n the followng dagram H s 0 (Ω) L L 2 (Ω) (4.33) Q 1 I d H s (Ω) L 2 (Ω). L Example A partcular nstance of Example 4.10 s the dfferental operator L : H s 0 (Ω) L2 (Ω) defned by Lu = 0 α s a α (x)d α u, for u H s 0(Ω), (4.34) where a s a tensor wth entres n L (Ω) such that L 1 s well defned and contnuous. 39

40 4.3 Identfcaton of measurement functons The applcaton of the gamblet transform to the resoluton of (4.2) requres the pror dentfcaton of the measurement functons φ (k) B that satsfy Condton We wll now show that these measurement functons can smply be obtaned as the mage, by DG of a multresoluton decomposton on B 2. Let (B c, c ) be a Banach subspace of B 2 such that the natural embeddng : B c B 2 s compact and dense. Constructon Let (f (q) ) I (q) be lnearly ndependent elements of B c. For k {1,..., q 1} and I (k) let f (k) B c be defned by nducton va f (k) =, f (k+1) (4.35) I (k+1) π (k,k+1) where the π (k,k+1) and I (k) are as n Constructon 2.1 and 3.3. For k {1,..., q} let For k {2,..., q} let F (k) := span{f (k) I (k) }. (4.36) F (k),χ := {f F (k) f = x f (k) wth x Ker(π (k 1,k) )} (4.37) I (k) The analogue of Condton 3.18 to the multresoluton decomposton on B 2 s as follows. Condton There exsts some constants C F 1 and H (0, 1) such that 1. sup f Bc f 2 f c C F C F H k f nf 2 f F (k) f 0 for k {1,..., q}. 3. sup g Bc nf f F (k) g f 2 g c C F H k for k {1,..., q}. 4. sup f F (k),χ f 2 f c C F H k 1 for k {2,..., q} C F x 2 I (k) x f (k) 2 c C F x 2 for k {1,..., q} and x R I(k) C F J (k) W (k) W (k),t C F J (k) for k {2,..., q}. Let B 0 = DGB c and φ 0 = G 1 D 1 φ c. Observe that B 0 s a Banach subspace of B and the natural embeddng : B 0 B s compact and dense. For k {1,..., q} and I (k) let φ (k) = DGf (k) (4.38) 40

41 Theorem If Condton 4.13 s satsfed then Condton 3.18 s satsfed wth a constant C Φ dependng only on C L, C L 1 and C F. Furthermore, the (fnte-element) soluton of (4.3) n V (k) s u (k) (u), whch satsfes u u (k) (u) CH k g c. Example Consder Examples 3.1 and 4.2 and take (B 0, 0 ) = (L 2 (Ω), L 2 (Ω)). Snce DG s the dentty operator, Condton 4.13 translates to (1) the compact embeddng nequalty φ H 1 (Ω) C φ L 2 (Ω) for φ L 2 (Ω) (2) the nverse Sobolev nequalty φ L 2 (Ω) CH k φ H 1 (Ω) for φ Φ (k) (3) the approxmaton property nf φ Φ (k) ϕ φ H 1 (Ω) CH k ϕ L 2 (Ω) for ϕ L 2 (Ω) (4) the Poncaré nequalty φ H 1 (Ω) CH k 1 φ L 2 (Ω) for φ Φ (k),χ (5) and the Resz bass/frame nequalty 1 C x 2 I (k) x φ (k) 2 L 2 (Ω) C x 2 for x R I(k). These condtons are therefore natural regularty condtons on the elements φ (k) and t s easy to check that they are satsfed n the context of Illustratons 3.5 and 3.13 wth H = 1/2. Example Consder Examples 3.2 and 4.3 (or a partcular nstance, Example 4.5) and take (B 0, 0 ) = (L 2 (Ω), L 2 (Ω)). Snce DG s the dentty operator, Condton 4.13 translates to (1) the compact embeddng nequalty φ H s (Ω) C φ L 2 (Ω) for φ L 2 (Ω) (2) the nverse Sobolev nequalty φ L 2 (Ω) CH k φ H s (Ω) for φ Φ (k) (3) the approxmaton property nf φ Φ (k) ϕ φ H s (Ω) CH k ϕ L 2 (Ω) for ϕ L 2 (Ω) (4) the Poncaré nequalty φ H s (Ω) CH k 1 φ L 2 (Ω) for φ Φ (k),χ (5) and the Resz bass/frame nequalty 1 C x 2 I (k) x φ (k) 2 L 2 (Ω) C x 2 for x R I(k). These condtons are, as n Example 4.15, natural regularty condtons on the elements φ (k). Proposton The condtons equvalent to Condtons 4.13, derved n Example 4.16, are satsfed wth the measurement functons presented n Examples 2.17 and 3.6 wth H = h s and a constant C dependng only on d, δ and s. Algorthm 5 Dscrete Gamblet solve of (4.39). 1: For I (q), ψ (q) = Ψ // Level q gamblets 2: Use Algorthm 2 to compute the gamblets ψ (k), χ (k) and matrces A (k), B (k) and R (k 1,k) 3: For I (q), b (q) = [DGg, ψ (q) ] 4: for k = q to 2 do 5: w (k) = B (k), 1 W (k) b (k) // Eq. (3.32) 6: v (k) = J (k) w(k) // Thm. 3.22, v (k) := u (k) u (k 1) W (k) 7: b (k 1) = R (k 1,k) b (k) // Eq. (3.26) 8: end for 9: w (1) = A (1), 1 g (1) // Eq. (3.30) χ (k) 10: u (1) = I (1) w(1) ψ (1) // Thm : u d = u (1) + v (2) + + v (q) // Thm

42 4.4 Usng the gamblet transform to solve nverse problems/lnear systems Consder elements (Ψ ) N B, as n Subsecton 3.7, to be used n the dscretzaton of the operator L to ther span B d := span{ψ, N } B. For g B 2 let u d B d be the Galerkn approxmaton of the soluton of Lu = g n B d,.e. the soluton of the dscrete lnear system (obtaned from the varatonal formulaton (4.10)) u d, Ψ = [DGg, Ψ] for Ψ B d. (4.39) When B s fnte-dmensonal one can select B d = B and obtan u d = u. When B s nfnte-dmensonal u d s only an approxmaton of u whose error corresponds to the dstance between u and B d (u d = arg mn v B d u v ). Algorthm 5, whch s a drect varant of the dscrete gamblet transform ntroduced n Algorthm 3, turns the resoluton of (4.39) nto that of q lnearly ndependent lnear systems (wth unformly bounded condton numbers under Condton 3.23). Remark By decomposng the nverson of (4.39) nto the nverson of q lnear systems of unformly bounded condton numbers, Algorthm 5 provdes an alternatve regularzaton of ll-condtoned lnear systems. Recall that classcal regularzaton methods nclude sngular value truncaton and the tradtonal (least square) Tkhonov regularzaton [159]. Recall also that well condtoned lnear systems can be solved effcently usng teratve methods. such as the Conugate Gradent (CG) method [105]. Fgure 10: Decomposton of u wth smooth g. See Illustraton

43 Illustraton Consder the numercal example descrbed n Illustraton 3.26 of Example 3.1. Fgures 10 and 11 show the decomposton the fnte-element soluton u of (4.4) nto subband solutons u (1) and u (k+1) u (k) along wth the relatve energy content of each subband. We consder two rght hand sdes g, one smooth gven by g(x) = ( N cos(3z,1 +z,2 )+sn(3z,2 )+sn(7z,1 5z,2 ) ) Ψ (x) (wrtng z the postons of the nteror nodes of the fne mesh) and one sngular defned as the (approxmate) dscrete mass of Drac g(x) = 4 q ψ 0 where 0 s the label of an nteror node of the fne mesh n the center of the square. Observe that when g s regular then the energy content n hgher subbands quckly decreases towards 0 (and therefore those subband solutons may not need to be computed dependng on the desred accuracy whch enables computaton n sublnear complexty) whereas when g sngular, the energy content n hgher subbands reman sgnfcant and all subband solutons may need to be computed. Fgure 11: Decomposton of u wth a sngular g. See Illustraton Computatonal Informaton Games Although the full development of our approach to Computatonal Informaton Games wll nvolve Gelfand trples of Banach and Hlbert spaces, Gaussan cylnder measures and ther correspondng Wener measures n the context of the nterplay between Numercal Analyss, Approxmaton Theory and Statstcal Decson Theory, as descrbed at length n Secton 8, to reduce the complexty of ths paper, here we consder Hlbert spaces pared wth non-standard realzatons of ther dual spaces and consder Gaussan cylnder measures n the form of Gaussan felds, all to be defned. To begn, we now descrbe 43

44 the connecton between the Informaton-Based Complexty approach to the problem of Optmal Recovery and ts connectons wth Game Theory. 5.1 From Informaton Based Complexty to Game Theory It s well understood n Informaton Based Complexty [252] (IBC) that computaton n nfnte dmensonal spaces can only be done wth partal nformaton. In the settng of Subsecton 3.1 ths means that, f B s nfnte dmensonal space, then one cannot drectly compute wth u but only wth a fnte number of features of u. These features can represented as the vector u m := ( [φ 1, u],..., [φ m, u] ) R m where φ 1,..., φ m are m lnearly ndependent elements of B. Smlarly one can, for g B 2, defne g m R m (as a functon of g) representng fnte nformaton about g. To solve the nverse the problem (4.2), snce one cannot drectly compute wth u and g but only wth u m and g m, one must dentfy a reduced operator mappng u m nto g m. If we know the mappng L and the mappng g g m then, as llustrated n (5.1), ths dentfcaton requres the determnaton of a mappng from u m to u, brdgng the nformaton gap between R m and B. u u m L Reduced operator Dscretzed nverse problem g g m (5.1) We apply the Optmal Recovery approach, see e.g. Mcchell and Rvln [153], to brdgng the nformaton gap as follows: Correspondng to the collecton Φ := φ 1,..., φ m of ( m lnearly ndependent elements of B, let Φ : B R m be defned by Φ(u) := [φ1, u],..., [φ m, u] ) be the nformaton operator. Moreover, t should cause no confuson to also denote by Φ B the span Φ := span Φ. A soluton operator s a possbly nonlnear map Ψ : R m B whch uses only the values of the nformaton operator Φ. For any soluton operator Ψ and any state u B, the relatve error correspondng to the dentty operator on B and the nformaton operator Φ : B R m can be wrtten E(u, Ψ) := u Ψ(Φ(u)) u see e.g. [152], from whch the error assocated wth the soluton operator Ψ s and the optmal soluton error s E = nf Ψ u Ψ(Φ(u)) E(Ψ) := sup, u B u, E(Ψ) = nf sup u Ψ(Φ(u)). (5.2) Ψ u B u Mcchell [152, Thm. 2] provdes the soluton to ths problem. In the settng of nonstandard dual parngs t appears as follows: 44

45 Theorem 5.1 (Mcchell). Let B be a separable Banach space, and B be a realzaton ts dual wth [, ] the correspondng dual parng. For a postve symmetrc becton Q : B B, consder the Hlbert space B equpped wth the nner product u 1, u 2 := [Q 1 u 1, u 2 ]. Correspondng to the collecton Φ := φ 1,..., φ m of m lnearly ndependent elements of B, let Φ : B R m be defned by Φ(u) := ( [φ 1, u],..., [φ m, u] ) be the nformaton operator. Defne the Gram matrx Θ by and the elements ψ B, = 1,... m by Θ := [φ, Qφ ],, = 1,... m, ψ := m =1 Θ 1 Qφ, = 1,..., m. (5.3) Then the mappng Ψ : R m B defned by Ψ(x) := m =1 x ψ, x R m s the unque optmal mnmax soluton to (5.2). The mnmax problem (5.2) corresponds naturally to an adversaral zero sum game nvolvng two players. Let us denote by L(Φ, B) (5.4) the set of ( σ(φ), σ(b) ) -measurable functons, where σ(φ) denotes the Borel σ-algebra generated by Φ and σ(b) the Borel σ- algebra of B. In ths notaton, usng the fact, see e.g. [29, Thm ], that v L(Φ, B) s equvalent to v = Ψ Φ for some Borel measurable functon Ψ, the game can be formulated as n the followng dagram (Player I) u I u II (Player II) (5.5) max u I u II (u I ) u I and the obectve of Player II (from a determnstc worst case numercal perspectve) would be to mnmze u I u II (u I ) nf sup u I. (5.6) u II L(Φ,B) u I B Usng the notaton Φ := span Φ B we obtan the followng corollary. Corollary 5.2 (Mcchell). Consder the stuaton of Theorem 5.1. Then the optmal mnmax soluton to (5.6) s the orthogonal proecton P QΦ onto QΦ B and P QΦ = Ψ Φ, where Ψ s the optmal mnmax soluton to (5.2). mn 45

46 5.2 The game theoretc soluton to numercal approxmaton Bvarate loss functons such as E(u I, u II ) := ui u II (u I ) wth u I B, u II L(Φ, B), do u I not, n general have a saddle pont. In fact t s easy to see that sup nf u I B u II L(Φ,B) u I u II (u I ) u I = 0. We know, from Von Neumann s remarkable mnmax theorem [236], that, at least fnte games, although mnmax problems do not, n general, have a saddle pont of pure strateges, saddle ponts of mxed strateges do always exst. These mxed strateges are randomzed strateges obtaned by lftng mnmax problems to dstrbutons over pure strateges [237, 157]. Although the nformaton game descrbed n (5.5) s zero sum, t s not fnte. Nevertheless, we also know, from Wald s Decson Theory [238], that under suffcent regularty condtons such games can be made compact and can, as a result, be approxmated by a fnte game (we also refer to Le Cam s substantal generalzatons [136, 137] and also to [213]). Instead of lookng for the determnstc worst case (numercal analyss) soluton we wll therefore lft (5.6) to a mnmax problem over measures and look for a game theoretc, mxed strategy, saddle pont/soluton. Our motvaton n dong so s twofold: (1) The game theoretc soluton s n general easer to dentfy than the numercal analyss soluton (2) When solvng a large lnear system, mnmax problems such as (5.5) occur n a repeated manner (over a range of levels of complexty) and mxed strateges are the optmal solutons of such repeated games. Although, n the context of (5.5), mxed strateges for Player II correspond to selectng u II at random by placng a probablty dstrbuton over L(Φ, B), we wll show that the optmal mxed strategy for Player II s a pure (non random) strategy, obtaned by (1) dentfyng the optmal mxed strategy of Player I as selectng u I at random by placng a (weak) probablty dstrbuton π I on B (and proectng that dstrbuton onto the orthogonal complement of QΦ) (2) takng the condtonal expectaton of π I on the observaton of Φ (.e. the vector ([φ 1, u I ],..., [φ m, u I ])). Furthermore the optmal weak dstrbuton for π I corresponds to that of a Gaussan feld on B wth covarance operator Q. To that end, we now ntroduce basc measure theoretc termnology, ncludng the notons of weak dstrbutons n ther equvalent form of cylnder measures, and descrbe ther relaton wth Gaussan felds. 5.3 Measures, cylnder measures, weak dstrbutons, and Gaussan felds Let us begn by establshng some notatonal conventons and basc facts. For a topologcal space B we let σ(b) denote the correspondng Borel σ-algebra. For measurable spaces (X 1, Σ 1 ) and (X 2, Σ 2 ) the notaton f : (X 1, Σ 1 ) (X 2, Σ 2 ) wll ndcate that the functon f : X 1 X 2 s measurable, that s f 1 (A) Σ 1 for A Σ 2. Let Φ := {φ, = 1,.., m} denote the collecton of functons φ : B R defned by the elements φ B, = 1,..., m and let σ(φ) denote the nduced σ-algebra of subsets of B generated by the collecton Φ. Correspondng to such a collecton, let us denote 46

47 by Φ : B R m the mappng defned by Φ(x) := m =1 x φ, x R m. Snce each component ( φ s ) contnuous t s Borel measurable. Therefore, ( ) a functon f : B B s σ(φ), σ(b) -measurable, that s f : (B, σ(φ)) B, σ(b), f and only f f = ψ Φ where ψ : R m B s Borel measurable, that s ( σ(r m ), σ(b) ) measurable. Consequently, the added assumpton that the soluton map Ψ be measurable s equvalent to the functon u II := Ψ Φ beng ( σ(φ), σ(b) ) ( ) -measurable. Recall the set L(Φ, B) of σ(φ), σ(b) -measurable functons ntroduced n (5.4). Accordng to Gross [97, Pg. 33], the noton of a weak dstrbuton, ntroduced by Segal [200], s equvalent to that of a cylnder measure. To descrbe the latter noton, for a Banach space X, the cylnder sets are sets of the form F 1 (B) where F : X R n for some n s contnuous, and B s a Borel subset of R n. The cylnder set algebra s the σ- algebra generated by all choces of F, n, and B. Accordng to Bogachev [28, Thm. A.3.7] when X s separable, ths σ-algebra s the Borel σ-algebra. Accordng to Bogachev [28], we say that ν s a cylnder measure f ν s fntely addtve set functon on the cylnder set σ-algebra such that for every contnuous lnear map F : X R n, the pushforward F ν, defned by F ν(b) := ν(f 1 (B)) for Borel sets B R n, s a true measure. When these are centered Gaussan measures, we say that ν s a Gaussan cylnder measure. To defne a Gaussan feld, we say that a lnear subspace W L 2 (Ω, Σ, µ), where (Ω, Σ, µ) s probablty space, s a Gaussan space f each element w W s a centered Gaussan random varable. If W s a closed subspace, we say that t s a Gaussan Hlbert space. Let us now summarze our defnton of a weak Gaussan dstrbuton defned n terms of a Gaussan feld n the settng of a Banach space and realzaton of ts dual. Defnton 5.3 (Gaussan Feld). Let B be a separable Banach space, and B be a realzaton ts dual wth [, ] the correspondng dual parng. For a postve symmetrc becton Q : B B, consder the Hlbert space B equpped wth the nner product ϕ 1, ϕ 2 Q := [ϕ 1, Qϕ 2 ]. Then we say that ξ s a Gaussan feld wth covarance operator Q, whch we wrte ξ N (0, Q), f ξ : (B,, Q ) L 2 (Ω, Σ, µ) s an sometry to a Lebesgue probablty space such that mage ξ(b ) s a Gaussan space. Let us now menton a partcularly useful abuse of notaton that we wll use throughout the paper. Consder a Gaussan feld ξ : (B,, Q ) L 2 (Ω, Σ, µ) and an element ϕ B. Then the random varable ξ(ϕ) L 2 (Ω, Σ, µ) s a real-valued functon ξ(ϕ) : Ω R on Ω. Snce, for ω fxed, the functon ξ(ϕ)(ω) of ϕ B s lnear, for each ω Ω, there exsts an element ˆξ(ω) n the algebrac dual B, so that ξ(ϕ)(ω) = [ϕ, ˆξ(ω)], ω Ω, where the bracket [, ] s the bracket correspondng to the algebrac dual. If we abuse notaton by removng the hat from ˆξ and usng the same bracket notaton for algebrac dual and topologcal dual, then we obtan the notaton ξ(ϕ) = [ϕ, ξ] where the functon on the rghthand sde s defned by [ϕ, ξ](ω) = [ϕ, ξ(ω)], ω Ω. Consequently, usng ths notaton, the sometrc nature of the Gaussan feld ξ can be 47

48 wrtten as E [ [ϕ 1, ξ][ϕ 2, ξ] ] = [ϕ 1, Qϕ 2 ], ϕ 1, ϕ 2 B. Moreover, snce B s reflexve, usng the close relatonshp between the algebrac dual of B and ts topologcal dual B, ξ has the nterpretaton as a B-valued (weak) random varable. Consequently, we say that a Gaussan feld ξ : (B,, Q ) L 2 (Ω, Σ, µ) s a Gaussan feld on B. Remark 5.4. Observe that f s the norm on B defned by u = [Q 1 u, u] 1 2 and f ξ N (0, Q) then for ϕ, φ B, [φ, ξ] N (0, ϕ 2 ) and E [ [ϕ, ξ][φ, ξ] ] = ϕ, φ where s the dual norm of and, ts assocated scalar product. The condtonal expectaton of a Gaussan feld s determned as the feld of condtonal expectatons. That s, for a Gaussan feld ξ and a sub σ-algebra Σ Σ, E[ξ Σ ] : B L 2 (Ω, Σ, µ) s defned by E[ξ Σ ](ϕ) := E[ξ(ϕ) Σ ], (5.7) where here and throughout the rest of the paper wll refran from constantly mentonng almost everywhere. Recall that we use the symbol Φ B for the span of Φ. Snce ξ(ϕ) L 2 (Ω, Σ, µ) s a centered Gaussan random varable for each element ϕ Φ and Φ s fnte dmensonal, t follows that ξ(φ) L 2 (Ω, Σ, µ) s a Gaussan Hlbert space. Consequently, f we let P ξ(φ) : L 2 (Ω, Σ, µ) L 2 (Ω, Σ, µ) denote the orthogonal proecton onto ξ(φ) and let σ(φ) denote the σ-algebra generated by Φ, usng the standard relaton between condtonng on a σ-algebra and condtonng on the set of random varables generatng t, accordng to Janson [113, Thm. 9.1], for all ϕ B we have E[ξ(ϕ) σ(φ)] = P ξ(φ) ( ξ(ϕ) ) Consequently, the condtonal expectaton s also a Gaussan feld and has the partcularly smple and useful form E[ξ σ(φ)] = P ξ(φ) ξ, (5.8) where the Gaussan feld P ξ(φ) ξ : B L 2 ( ) (Ω, Σ, µ) s defned by(p ξ(φ) ξ)(ϕ) := P ξ(φ) ξ(ϕ), ϕ B. In ths paper, we wll use the canoncal nstantaton of the ambent space L 2 (Ω, Σ, µ) for the Hlbert space B throughout the rest of the paper. Consder the countable product R N equpped wth the product γ N of standard Gaussan measure on R and the resultng Lebesgue space L 2 (R N, σ(r N ), γ N ). It s well known that σ(r N ) = σ(r) N. We smplfy notaton by wrtng ths space as L 2 (R N, γ N ). Let f, N denote the set of functons on RN whch satsfy f (x) = x, x R N. Select an orthonormal bass {e B, N} for B, and consder the mappng ξ : B L 2 (R N, γ N ) determned by defnng t on the bass elements as ξ(e ) = é and extendng t by lnearty to the rest of B. Clearly, ξ(e ) = é s a centered Gaussan random varable on R N wth varance 1. Moreover, Parseval s formula can be used to prove, that for ϕ B, ξ(ϕ) s a centered Gaussan random varable of varance ϕ 2. It follows that 48

49 ξ : B L 2 (R N, γ N ) s an sometry and therefore a Gaussan feld by Defnton 5.3. Ths feld can be shown to be ndependent of the chosen orthonormal bass. Moreover, as asserted by Strasser [213, Ex ], t follows that such a Gaussan feld corresponds to a Gaussan cylnder measure. Therefore, henceforth we restrct the ambent space to be L 2 (R N, γ N ) chosen n ths way. Remark 5.5. We have mentoned the equvalences between cylnder measures and weak dstrbutons, and the equvalence between Gaussan cylnder measures, Gaussan weak dstrbutons and Gaussan felds. Moreover, unless t s mportant to make the dstncton we may refer to such weak obects smply as measures. 5.4 Mxed extenson of the game and optmal mnmax solutons In Secton 5.9 we descrbe how weak dstrbutons arse naturally as worst case (weak) measures for the optmal recovery problem, and demonstrate that Gaussan felds are unversal worst case measures n the sense that they are worst case measures ndependent of the measurement functons Φ. The followng theorem shows that ξ N (0, Q) s such a unversal worst case measure, producng the optmal mnmax strategy. Theorem 5.6. The optmal strategy of (5.5) for Player II s the pure strategy u II = E ξ N (0,Q) [ ξ σ(φ) ] correspondng to the mxed strategy ξ N (0, Q) of player I whch s a worst case (weak) measure n the sense descrbed n Secton 5.9. In partcular, the functon u II L(Φ, B) defned by u II (u) = E ξ N (0,Q) [ ξ [φ, ξ] = [φ, u] for = 1,..., m ], u B (5.9) s the optmal mnmax soluton of (5.6). Moreover, the gamblets (5.3) determned to be optmal by Theorem 5.1, have the representaton ψ = E ξ N (0,Q) [ ξ [φ, ξ] = δ,, = 1,... m ]. (5.10) 5.5 Repeated games across herarches of ncreasng levels of complexty It s not only computaton wth contnuous operators that requres wth partal nformaton, to compute fast one must also compute wth partal nformaton. For example, the nverson of a matrx would be a slow process f one tres to compute wth all the entres of that matrx at once, the only way to compute fast s to compute wth a few features of that matrx (that could be mapped to 64 degrees of freedom) and these features typcally do not represent all the matrx entres. Smlarly, to obtan near optmal complexty solvers, one must compute wth partal nformaton over herarches of ncreasng levels of complexty and brdge herarches of nformaton gaps. In the proposed framework, we use the herarchy of measurement functons φ (k) ntroduced n Subsecton 3.2 to generate a fltraton on B representng a herarchy of partal nformaton about u B. As n Subsecton 5.1, the process of brdgng nformaton gaps across ths herarchy can then be formulaton as an adversaral game, n whch Player 49

50 I selects u I B and Player II s shown the level k measurements ( [φ (k), u I ] ) and I (k) must approxmate u I and level k + 1 measurements (.e., ( [φ (k), u I ] ) ). Ths game I (k+1) s repeated across k (the herarchy of partal nformaton/measurements about u I ) and the choce of Player I does not change as k progresses from 1 to q. We now extend Theorem 5.6 to the herarchy. Theorem 5.7. The optmal strategy of (5.5) at level Φ (k) for Player II s the pure strategy u (k) := E ξ N (0,Q) [ ξ σ(φ (k) ) ] correspondng to the mxed strategy ξ N (0, Q) of player I. That s, the functon u (k) L(Φ (k), B) defned by u (k) [ (k) (u) := E ξ N (0,Q) ξ [φ, ξ] = [φ (k), u] for = 1,..., m ], u B (5.11) s the optmal mnmax soluton of (5.6) at level Φ (k). Moreover, the gamblets at level k, defned n 3.7 wth explct representaton n Theorem 3.8, have the representaton ψ (k) = E [ ξ [φ (k), ξ] = δ,, I (k)], I (k) (5.12) and the nterpolaton matrx mplctly defned n (3.26) has the representaton R (k,k+1), = E [ [φ (k+1), ξ] [φ (k) l, ξ] = δ,l, l I (k)], I (k), I (k+1). (5.13) Fnally, the measure ξ N (0, Q) s a worst case measure at all levels of the herarchy. For a Gaussan feld ξ, let ξ (k) := E [ ξ σ(φ (k) ) ] (5.14) denote the condtonal expectaton wth respect to the σ-algebra σ(φ (k) ) generated by the observaton functons at the k-th level. We now conclude the theoretcal porton of ths secton wth mportant Martngale propertes of our constructons. We say that a sequence ξ (k), k = 1,... of Gaussan felds wth common doman s a martngale f, for each ϕ n ts doman, ts sequence of mages [ϕ, ξ (k) ] s a martngale. Theorem 5.8. It holds true that (1) σ(φ (1) ),..., σ(φ (q) ) forms a fltraton,.e. σ(φ (k) ) σ(φ (k+1) ) (2) ξ (k) s a martngale wth respect to the fltraton ( σ(φ (k) ) ) k 1,.e. ξ(k) = E [ ξ (k+1) σ(φ (k) ) ] (3) ξ (1) and the ncrements (ξ (k+1) ξ (k) ) k 1 are ndependent Gaussan felds. Furthermore, ξ (k) = [φ (k), ξ] (5.15) I (k) ψ (k) Theorem 5.7 shows that, f (5.5) s used to measure loss, then u (k) s not only optmal n a Galerkn sense (.e. t s the best approxmaton of u n V (k) as shown n Theorem 3.11), t s also the optmal (pure) bet for Player II for playng the repeated game descrbed n ths secton. Furthermore, Theorem 5.7 and the followng Theorem 5.8 show that the elements ψ (k) obtaned n Subsecton 3.3 form a bass of elementary 50

51 gambles/bets for playng the game, provdng the motvaton for referrng to them as gamblets. Note that (5.13) shows that R (k,k+1), can be dentfed as the best bet of Player II on the value of [φ (k+1), u] gven the nformaton that [φ (k) l, u] = δ,l for l I (k). Moreover, Theorem 5.8 enables the applcaton of classcal results concernng martngales to the numercal analyss of ξ (k) (and therefore u (k) ). In partcular (1) Martngale (concentraton) nequaltes can be used to control the fluctuatons of ξ (k) (2) Optmal stoppng tmes can be used to derve optmal strateges for stoppng numercal smulatons based on loss functons mxng computaton costs wth the cost of mperfect decsons (3) Takng q = n the constructon of the bass elements ψ (k) and usng the martngale convergence theorem mply that, for all φ B, [φ, ξ (k) ] [φ, ξ ( ) ] as k (a.s. and n L 1 ). Furthermore, the ndependence of the ncrements ξ (k+1) ξ (k) s related to the orthogonal multresoluton decompostons (3.17). Let us now descrbe these results n the context of Example 3.1, Defnton 3.14, and Illustratons 3.5 and Fgure 12: ψ (k) for I (k). χ (k) W, for I (k). R (k,k+1), s the best bet of Player II on the value of u B gven that [φ (k), u] = δ, s the best bet of Player II on the value of u B gven that [φ (k), u] = that [φ (k), u] = δ, for I (k). See Illustraton 5.9. s the best bet of Player II on the value of [φ (k+1), u] gven Illustraton 5.9. Consder Fgure 12 and the context of Example 3.1 and Illustratons 3.5 and 3.13 where φ (k) = 1 (k) τ / τ (k) and (τ s (k), s I (k) ) s a nested rectangular partton of Ω = (0, 1) 2. ψ (k) s Player II s best bet on the value of the soluton u (H0 1(Ω), a) gven τ (k) u = τ (k) δ, for I (k) (b) χ (k) s Player II s best bet on u gven τ (k) u = τ (k) (δ, δ +,) for I (k) (where + s an adacent square of ) (c) R (k,k+1), s Player II s best bet on τ (k+1) 51 u gven τ (k) s u = τ (k) s δ,s for s I (k).