ITALY. Abstract. Keywords: Monte Carlo, Latin Hypercube Sampling, Quasi Monte Carlo, Sobol sequences, High Dimensional Integration. 1.

Transcription

1 Comparso of Lat Hypercube ad Quas Mote Carlo Samplg Techques Serge Kuchereko *, Dael Albrecht, Adrea Saltell CPSE, Imperal College Lodo, Lodo, SW7 AZ, UK The Europea Commsso, Jot Research Cetre, TP 36, 07 ISPRA(VA), ITALY Abstract Three samplg methods are compared for effcecy o a umber of test problems of varous complexty for whch aalytc quadratures are avalable. The methods compared are Mote Carlo wth pseudoradom ad Lat Hypercube Samplg ad the Quas Mote Carlo method wth samplg based o Sobol sequeces. Geerally, results show superor performace of the Quas Mote Carlo approach based o Sobol sequeces le wth theoretcal predctos. There are also some types of fuctos for whch Lat Hypercube Samplg ca be more effcet tha the Mote Carlo method. For the same fuctos types t ca be more effcet tha the Quas Mote Carlo method at small umber of sampled pots. Keywords: Mote Carlo, Lat Hypercube Samplg, Quas Mote Carlo, Sobol sequeces, Hgh Dmesoal Itegrato.. Itroducto I may practcal applcatos (rsk aalyss, face ad ecoomcs, evrometal studes to ame a few) umercal models ca be expesve to ru ad t s sometmes mpossble to have as may model rus as eeded to acheve the desred accuracy. I such cases a effcecy of the Mote Carlo samplg techque ca make the dfferece betwee a relable ferece or a msleadg result. The smple Mote Carlo techque s kow to have low effcecy. Varace reducto methods ca be employed to crease ts effcecy. Oe of the most popular varace reducto techque s stratfed samplg usg Lat Hypercube Samplg (LHS) [8]. May studes have bee made over the years to develop LHS wth better space fllg propertes. Some authors have proposed to mprove LHS space fllg ot oly oe dmesoal projecto, but also hgher dmesos [4]. Dfferet optmalty crtero such as etropy, tegrated mea square error, mmax ad maxm dstaces, etc ca be used for optmzg LHS [3]. The maxm crtero cosstg maxmzg the mmal dstace betwee the pots to avod samplg desgs wth too close pots was used Morrs ad Mtchell [0]. Iooss et al. [] compared the performace of dfferet types of space fllg desgs, the class of the optmal LHS, terms of the effcecy of the subsequet metamodel fttg. They cocluded that optmzed LHS wth mmal wrap-aroud dscrepacy are partcularly well-suted for the Gaussa process metamodel fttg. * Correspodg author at: Imperal College Lodo, Lodo, SW7 AZ, UK. E-mal address: s.kuchereko@mperal.ac.uk (S. Kuchereko).

2 A detaled study o LHS ad varous modfed desgs of LHS was gve by Owe [], [3], Koehler ad Owe [5], Tag [36], Fag ad co-authors [5-6] ad some other authors. Kalagaam ad Dwekar [4] compared the performace of the Lat hypercube ad Hammersley samplg techques o a umber of test problems. The umber of samples requred for covergece to mea ad varace was used as a measure of performace. It was show that the samplg techque based o the low dscrepacy Hammersley pots requres far fewer samples to coverge to the varace of the tested dstrbutos: for the selected test cases Hammersley requred up to 40 tmes fewer sampled pots tha LHS. Wag et al. [37] developed a ew samplg techque combg LHS ad Hammersley sequece samplg ad compared t wth MC, LHS ad the Hammersley sequece samplg methods for calculatg ucertaty rsk aalyss. They foud that the ew (LHS+Hammersley) samplg techque ad the Hammersley sequece samplg techque behaved cosstetly better tha MC ad pure LHS. Whle ts applcato areas lke expermetal desg ad metamodel fttg s well studed, the effcecy of LHS other areas such as hgh dmesoal tegrato has ot bee aalyzed systematcally. I hgh dmesoal tegrato, effcecy depeds upo the degree of uformty of the sampled pots the -dmesoal space. I ths work we compare effceces of three samplg methods: the MC method wth pseudo-radom ad LHS samplg ad the QMC method wth samplg based o Sobol sequeces. We dscuss three dfferet LHS desgs. Despte of the recet advaces optmzato of hgh dmesoal propertes of LHS, due to the P complexty of optmzato, t s lmted to relatvely small desgs. The most effcet ad hece expesve for optmzato LHS desgs ca ot be used for hgh dmesoal tegrato. QMC samplg s performed by usg the hgh-dmesoal Sobol sequece geerator wth advaced uformty propertes: Property A for all dmesos ad Property A for adjacet dmesos (see Secto 5 for detals). Three dfferet classes of test fuctos are cosdered for hgh dmesoal tegrato. Classfcato of fuctos s based o ther effectve dmeso (Kuchereko et al. [7]). We report the absolute values of tegrato errors as well as the covergece rate. Ths paper s orgazed as follows: the ext Secto we descrbe Mote Carlo tegrato method. Secto 3 cotas a bref descrpto of LHS. The QMC method s preseted Secto 4. Secto 5 troduces Sobol Low Dscrepacy Sequeces (LDS). A frst ddactc set of results comparg dfferet techques for populatg a ut square s gve Secto 6. Dscrepacy as a quattatve measure for the devato of sampled pots from the deal (desred) uform dstrbuto s descrbed Secto 7, where L dscrepacy for the three cosdered samplg methods s also reported. Global sestvty aalyss ad effectve dmesos are troduced Secto 8. Applcatos of the three samplg techques are dscussed Sectos 9 ad 0. Secto cocludes our aalyss.. Mote Carlo tegrato Cosder the tegral of the fucto f (x) over the -dmesoal ut hypercube H

3 Fucto f ( x ) s assumed to be tegrable I[ f] = f( xdx ). (.) H H. The MC quadrature formula s based o the probablstc terpretato of a tegral. For a H radom varable x that s uformly dstrbuted I[ f] = E[ f( x)], where E[ f( x )] s the mathematcal expectato. A approxmato to ths expectato s I [ f] = f( x ), (.) = where (,..., x = x x ), =,..., s a sequece of radom pots H of legth. I other words, s a umber of sampled pots. Approxmato (.) s kow as the smple (or crude) Mote Carlo estmate of the tegral. The approxmato I[ f ] coverges to I[ f ] wth probablty. It follows from the Cetral Lmt Theorem that the expectato of a tegrato error ε, where ε = I[ f] I [ f] s where σ ( f ) s the varace gve by σ ( f ) E( ε ) =,. σ ( f ) = f ( xdx ) ( f ( xdx ) ) H H The expresso for the root mea square error of the MC method s σ ( f ) E The covergece rate of MC does ot deped o the umber of varables although t s rather low. Varous varace reducto techques ca be appled to reduce the value of the umerator (.3), whch does ot chage the MC tegrato covergece rate of / ε = ( ( )) ε =. (.3) / / O(/ ). The effcecy of MC methods s determed by the propertes of radom umbers. It s kow that radom umber samplg s proe to clusterg: for ay samplg there are always empty areas as well as regos whch radom pots are wasted due to clusterg; as ew pots are added radomly, they do ot ecessarly fll the gaps betwee already sampled pots. The qualty of a quadrature by a fte umber of sampled pots depeds o the uformty of the pots dstrbuto, ot ther radomess. Samplg strateges amed at placg pots more uformly clude the LHS ad LDS (also kow as quas radom) desgs addressed the preset aalyss. 3. Lat Hypercube Samplg LHS s wdely appled computatoal egeerg. It was developed by McKay et al. [8]. It s oe form of stratfed samplg that ca reduce the varace the Mote Carlo estmate of the tegrad. It was further aalyzed by Ima ad Shortecarrer [0] ad the by Ste [35]. It has bee show that 3

4 LHS ca mprove the effcecy compared to the Mote Carlo approach, though we show here (Secto 9) that ths oly holds for certa classes of fuctos. Cosder the rage [0,] dvded to tervals of the equal legth /. Oe pot s selected at radom from each terval formg a sequece of pots H { },,.., x =. Smlarly but depedetly we costruct aother sequece{ x }, =,..,. The two sequeces { },,.., x = ad { },,.., x = ca be pared to populate a bdmesoal space. These pars ca tur radomly be combed wth the values of sequece of s formed. 3 { },,..., x = to form trplets, ad so o utl a -dmesoal The algorthm formally ca be preseted as follows. Let { π k}, k =,..., be depedet radom permutatos of {,..., } each uformly dstrbuted over all! possble permutatos. Set k k π k() + U x =, =,...,, k =,...,, (3.) k where U are radomly sampled pots o [0,] terval. k It s easy to see that oly oe pot of { x }, =,...,, k =,..., falls betwee (-)/ ad /, =,.., for each dmeso k =,..,. However, ths stratfcato scheme s bult by supermposg well stratfed oe-dmesoal samples, ad caot be expected to provde prcple good uformty propertes a -dmesoal ut hypercube H. I ths work we used the code for the LHS from Burkardt []. Further t s called stadard LHS. The effcecy of the stadard LHS ca be mproved by takg the stadard LHS as a startg desg ad the optmzg t accordg to some optmzato crtero: f.e maxmzg the mmum dstace betwee ay two pots (maxm crtero), mmzg the dstace betwee a pot of the put doma ad the pots of the desg or mmzg the dscrepacy. Optmzato ca be doe by usg dfferet methods: choce of the best ( terms of the chose crtera) LHS amogst a large umber of dfferet LHS, colum wse par wse exchage algorthms or usg optmzato techques such as geetc algorthms, smulated aealg, etc (Fag et al. [6]). It has bee show by Iooss et al. [] that mmzg the dscrepacy leads to a better space-fllg desg compared to the oe where the mmum dstace s maxmzed. I partcular they compared the two-dmesoal projectos of the maxm LHS ad low wrap-aroud dscrepacy LHS wth = 00 pots ad dfferet tal dmesos (Iooss et al. []). They foud that the tal LHS desg optmzed wth the wrap-aroud dscrepacy has gve the best results. These results were further developed Dambl et al. [4]. We use ther code to produce optmzed LHS. It s obtaed by usg optmzato based o smulated aealg. We also used maxm LHS obtaed by selectg a LHS desg from a set of four dfferet desgs accordg to the maxm crtero. A umber of chose desgs was lmted by the CPU tme ad a requremet to use a very large umber of sampled pots (up to = 0 averaged over 50 depedet rus, see Secto 9). 4

5 4. Quas Mote Carlo LDS are specfcally desged to place sample pots as uformly as possble. LDS are also kow as quas radom umbers. The QMC algorthm for the evaluato of the tegral (.) has a form smlar to (.) I = f( q). (4.) = Here { q } s a set of LDS pots uformly dstrbuted a ut hypercube H, q q q (,..., ) =. The Koksma- Hlawka equalty (ederreter []) gves a upper boud for the QMC tegrato error: Here, V (f) s the varato of f ( x ) the sese of Hardy ad Krause, ε V( f) D. (4.) D s the sample dscrepacy (ts defto s gve the ext Secto). For a oe-dmesoal fucto wth a cotuous frst dervatve t s smply V ( f ) = df ( x)/ dx dx. H I hgher dmesos, the Hardy-Krause varato may be defed terms of the tegral of partal dervatves. Further t s assumed that f ( x ) s a fucto of bouded varato. The smaller the dscrepacy D, the better the covergece of the QMC tegrato. It was later show expermetally by Morokoff ad Caflsch [9] that the QMC tegrato error s determed by the varace of the tegrad ad ot by ts varato. It s geerally accepted that the rate of the dscrepacy meag by ths the rate at whch dscrepacy coverges as a fucto of, determes the expected rate of the accuracy. I fact oe ca use the followg estmate of the QMC covergece rate (see Secto 7 for detals): O(l ) ε QMC =. (4.3) The QMC rate of covergece (4.3) s much faster tha that for the MC method (.3), although t depeds o the dmesoalty. Cosequetly, the smaller the value of, the better ths estmate. I practce at > the rate of covergece appears to be approxmately equal to O ( α ), 0< α. Hece, the QMC method most cases outperforms MC terms of covergece. 5. Sobol sequeces There are a few well-kow ad commoly used LDS. May practcal studes have prove that the Sobol LDS s may aspects superor to other LDS. "Prepoderace of the expermetal evdece amassed to date pots to Sobol' sequeces as the most effectve quas-mote Carlo method for 5

6 applcato facal egeerg." Glasserma [7] (see also Jaeckel et al. []). For ths reaso Sobol sequeces were used ths work. Sobol LDS am to meet three ma requremets (Sobol' [30]):. Best uformty of dstrbuto as.. Good dstrbuto for farly small tal sets. 3. A very fast computatoal algorthm. Pots geerated by the Sobol LDS produce a uform-lookg fllg of the space, eve for rather small umbers of pots. Best uformty of dstrbuto s defed terms of low dscrepacy ad addtoal uformty propertes A ad A (Sobol' [3]): Defto. A low-dscrepacy sequece s sad to satsfy Property A f for ay bary segmet (ot a arbtrary subset) of the -dmesoal sequece of legth there s exactly oe pot each hyper-octat that results from subdvdg the ut hypercube alog each of ts legth extesos to half. Defto. A low-dscrepacy sequece s sad to satsfy Property A f for ay bary segmet (ot a arbtrary subset) of the -dmesoal sequece of legth 4 there s exactly oe pot each 4 hyper-octat that results from subdvdg the ut hypercube alog each of ts legth extesos to four equal parts. 6. Comparso of sample dstrbutos geerated by dfferet techques o a ut square I ths secto we compare dstrbutos of = 4 ( Fg. ) ad = 6 ( Fg. ) pots o a ut square gve by fve dfferet samplg techques: MC, stadard LHS, maxm LHS, optmzed LHS ad Sobol LDS. Ths provdes a qualtatve pcture of the uformty propertes of these samplg techques. I the frst case the ut square s dvded to ad / respectvely. I the secod case the ut square s dvded to ad / ad / respectvely. squares of measure / ad squares of measure Fgs.,,j show -dmesoal dstrbutos of Sobol pots satsfyg Property A. Oe ca see that each of the 4 small squares cotas exactly oe Sobol pot (Fgs., ). Ths s ot the case for stadard LHS, see Fg. (c). Projectos to both -dmesoal subspaces also cota pot each of the 4 tervals for stadard LHS (Fg.,d), for maxm LHS (Fg.,f), for optmsed LHS (Fg.,h) ad for LDS-Sobol (Fg.,j). Radom samplg (Fg.,b) does ot possess ether of these propertes. Ths secod effect s also true for LHS. Radom samplg does ot possess ether of these propertes (Fgs., a,b). LDS also possess a more geeral property: all projecto of the -dmesoal LDS o s- dmesoal subspaces form s-dmesoal LDS (ederreter []). These addtoal stratfcato propertes of Sobol LDS result creased uformty of Sobol samplg. The stadard LHS samplg does ot possess addtoal stratfcato propertes hgher dmesos. The LHS desg ca be 6

7 mproved by optmzg t further. However, ths optmzato beg very CPU tme cosumg s oly possble for a small umber of pots ad lmted umber of dmesos. Although maxm LHS (Fgs., e,f) vsually shows a mprovemet a space dstrbuto over stadard LHS, sampled pots do t satsfy Property A. However, pots produced by optmzed LHS do satsfy Property A (Fgs., g,h). 7

8 (a) (b) (c) (d) (e) (f) (g) (h) () (j) Fg.. Dstrbutos of 4 pots two dmesos. The ut square s dvded to 4 (o the left) ad 6 (o the rght) squares. (a,b) MC, (c,d) stadard LHS, (e,f) maxm LHS, (g,h) optmzed LHS, (,j) Sobol LDS. 8

9 (a) (b) (c) (d) (e) (f) (g) (h) () (j) Fg.. Dstrbutos of 6 pots two dmesos. The ut square s dvded to 6 (o the left) ad 56 squares (o the rght). (a,b) MC, (c,d) stadard LHS, (e,f) maxm LHS, (g,h) optmzed LHS, (,j) Sobol LDS. 9

10 Fg. shows dstrbutos of 6 pots two dmesos. From Fg.,,j t s clear that Sobol pots satsfy Property A dmesos: each of the =6 subsquares cotas exactly Sobol pot (Fg., ). It s ot true for all types of LHS (Fg., c, e, g) ad MC (Fg., a) samplgs: clusterg ad empty subsquares are clearly vsble from these plots. Optmzed LHS gves the best dstrbuto amog all other LHS desgs. Projectos of the -dmesoal samplg to both -dmesoal subspaces also cota pot each of the 6 tervals (Fgs., j). Ths s also true for LHS (Fg., d, f, h) but ot for MC (Fg., b). We ca coclude that Sobol LDS samplg gves a much better way of arragg pots dmesos tha MC ad stadard LHS. Although LHS samplg ca be mproved through optmzato, ths procedure s lmted oly for small sample szes ad low dmesos. Aother mportat lmtato of optmzed LHS s that mproved propertes of LHS are ot possble to keep whe cremetally creasg the sze of samples (Iooss et al. []). 7. Dscrepacy Dscrepacy s a quattatve measure for the devato of sampled pots from the uform dstrbuto. Cosder a umber of pots from a sequece { x }, =,.., a -dmesoal Q rectagle Q whose sdes are parallel to the coordate axes ad whch s a subset of H : Q H. Q has a volume mqt ( ( )) = [0, t[... [0, t [, where t = ( t,..., t ) are the rght had sde coordates of the rectagle wth the org the cetre of coordates. The local dscrepacy s defed as (see f.e. ederreter []) where s the total umber of pots sampled two dmesos. The star-dscrepacy s defed as Q ht () = mqt ( ()), (7.) H. Fg. 3 llustrates the oto of the local dscrepacy D * = sup h( t). (7.) By defto a low dscrepacy sequece (LDS) s oe satsfyg the upper boud codto: Q H * (l ) D c( ). (7.3) Costat c ( ) depeds upo the sequece as well as upo the dmeso, but does ot deped o. For radom umbers the expected dscrepacy s D = O((l l ) / ). * / From the Koksma-Hlawka equalty (4.) t follows that asymptotcally at QMC has a covergece rate O(/ ) whch s much hgher tha the rate O(/ ) that ca be acheved by MC ad 0

11 LHS. However, ths rate ca be attaed oly at the umber of sampled pots hgher tha the threshold * ~ exp( ) whch s practcally ot possble at large. The value of dscrepacy depeds equally o all dmesos. I practcal applcatos, whe oe attempts to calculate the quadrature of a gve fucto o ths space, some varables (dmesos) are more mportat tha others gve the fucto. Ths explas why eve for hgh dmesoal problems wth hudreds of varables ad a umber of sampled pots < * stll QMC has a better covergece rate tha MC. The theory behd ths behavor s based o global sestvty aalyss ad s preseted the ext secto. Calculato of the star-dscrepacy practcal purposes. The L dscrepacy s defed as L D has a closed form (Warock [38]): whch was used ths work. * D s very dffcult ad the L dscrepacy s used stead for L D = h () t dt H /. (7.4) D x x x d L k k k d = ( max( ) ( ( ) ) 3 j +, j= k= = k= (7.5) Fg. 3. Illustrato of local dscrepacy ht () two dmesos wth Sobol LDS samplg of = 56 pots. Q s a umber of pots the shaded rectagular.

12 There are also other deftos of dscrepaces. Hckerell [8] poted out that the dscrepacy (7.) s achored to the org because the terval [0; t) appears ts defto ad also there s referece to t = (; ; ) sce t appears the formula for the varato (4.). He defed the socalled cetered dscrepacy ad varato that refer to the ceter of the hypercube at t = (0.5,, 0.5). Ths dscrepacy ad varato are varat uder reflectos of a set of pots about ay plae t j = 0.5. The cetered dscrepacy has some advatages over the stadard L dscrepacy (7.4) whe the objectve s to detfy dffereces varous samplg desgs. Hckerell also troduced the wrap-aroud dscrepacy whch measures uformty ot oly over the -dmesoal ut hypercube all the projecto of a set of pots to s-dmesoal subspaces of s H H but also over, s <. These results were further developed by Fag et al. [6] who appled searchg algorthms by mmzg dscrepacy as the objectve fucto for optmzg LHS desgs. We compared uformty propertes of MC, LHS ad Sobol sequeces for dfferet dmesos usg L D defed by (7.5). Results preseted Fg. 4 were obtaed wth a stadard radom umber geerator as provded by MATLAB, the LHS geerator developed Burkardt [] ad the SobolSeq89 geerator (see BRODA webpage [], Sobol' et al. [34]). The maxmum umber of sampled pots used for calculatos was 3768.

13 0. MC LHS QMC 0.0 D_L e (a) 0.0 MC LHS QMC D_L (b) e-07 MC LHS QMC D_L e-08 e L Fg. 4. D dscrepacy dmesos = 5 (a), = 0 (b), = 40 (c) versus for MC (blue broke le), LHS (thck sold red le) ad QMC (th sold gree le) samplgs. (c) It ca be see that for low dmesos ( < 0) LDS s superor to MC ad LHS as t shows the lowest values for the dscrepacy. I hgher dmesos ( > 40) the L dscrepacy of Sobol LDS becomes 3

14 comparable to that of MC ad LHS. However as atcpated, dscrepacy values are ot suffcet to judge the performace of the cosdered samplg techques whe appled to practcal problems such as hgh dmesoal tegrato. The performace s determed by the effectve dmeso values for a problem at had. The oto of the effectve dmeso s troduced the ext secto. Due to the large umber of requred sampled pots ad hgh dmesos t was ot possble to cosder optmzed LHS. We also cosdered maxm LHS usg procedure descrbed above. Comparso showed that the results obtaed wth stadard ad maxm LHS are smlar, therefore we preset oly the result of stadard LHS. These fdgs also cocer the results for fucto tegratos preseted Secto 9. It s qute possble that optmzed LHS would show a lower L D dscrepacy tha that of MC. However, optmzed LHS ca oly be used at very low umber of sample pots ad relatvely low dmesos, whle all these tests requred a very hgh umber of. Fag et al showed that LHS has a lower expectato of square cetered dscrepacy tha that of smple radom desgs (Fag et al. [5]). They oted that ths result s cosstet wth the result that LHS s better tha radom desgs the sese of small-varace as show by Saltell [8]. 8. Global Sestvty Aalyss ad Effectve dmesos Global Sestvty Aalyss (SA) provdes formato o the geeral structure of a fucto by quatfyg the varato the output varables respose to varato the put varables. Cosder a tegrable fucto f ( x ) defed the ut hypercube the followg form: f ( x) f f ( x,..., x ) 0... s s= <... < s s H. It ca be expaded = +. (8.) Expaso (8.) s a sum of compoets. It ca also be preseted as Each of the compoets s s f ( x) = f + f ( x ) + f ( x, x ) f ( x, x,..., x ). 0 j j... < j f... ( x,..., x ) s a fucto of a uque subset of varables from x. The compoets f ( x ) are called frst order terms, f ( x, x ) - secod order terms ad so o. j j It ca be prove (Sobol' [3]) that the expaso (8.) s uque f f... ( x,..., x ) dx = 0, k s, s s (8.) k 0 whch case t s called a decomposto to summads of dfferet dmesos. Ths decomposto s also kow as the AOVA (Aalyss Of VAraces) decomposto. The AOVA decomposto s orthogoal,.e. for ay two subsets u v a er product 4

15 It follows from (8.) ad (8.) that 0 f ( x) f ( x) dx= 0. u v 0 f ( xdx )... dx = f, 0 f ( x) dxk = f0 + f( x), (8.3) 0 k 0 f ( x) dx = f + f ( x ) + f ( x ) + f ( x, x ) k, j k 0 j j, j j ad so o. For square tegrable fuctos, the varaces of the terms the AOVA decomposto add up to the total varace of the fucto where σ... f... ( x,..., x ) dx,..., x 0 σ =. s s s s σ... s s= < < s Sobol defed the global sestvty dces as the ratos =, (8.4) S σ σ... =... s /. s All S are o egatve ad add up to oe... s... s s= <... < s S... s =. S ca be vewed as a atural sestvty measure of a set of varables fracto of the total varace gve by f... ( x,..., x ). s s x,..., s x. It correspods to a Sobol also troduced sestvty dces for subsets of varables (Sobol' [3]). Cosder two complemetary subsets of varables y ad z: x = ( yz, ). Let y= ( x,..., x ), <... <, K = (,..., ). The varace correspodg to y s defed as m m m σ y cludes all partal varaces σ m y σ... s= ( < < ) K =. s σ, σ,, σ such that ther subsets of dces... s s (,..., ) K s. The total varace tot ( σ y ) s defed as 5

16 ( ) tot σ y cossts of all... s tot ( σ ) = σ σ y z σ such that at least oe dex p K whle the remag dces ca belog to the complemetary to K set K [30]. The correspodg global sestvty dces are defed as S S y tot y = σ = σ / σ, y / σ. tot S tot y = S, z S tot y S accouts for all teractos betwee y ad z. y The mportat dces practce are S ad tot S (Saltell et al. [6], Saltell ad Taratola [7]). Samplg based computatoal strateges to compute these measures are dscussed Saltell [8], Saltell et al. [9]. The frst order dex, wrtte as s detcal to Pearso s correlato rato [5]. I most cases kowledge of S V = S ad X ( E ( Y X )) X S tot V (Y) provdes suffcet formato to determe the sestvty of the aalyzed fucto to dvdual put varables. Ther values also ca be used to determe fucto effectve dmesos. The oto of the effectve dmeso was troduced Caflsch et al. [3]. Let y be a cardalty of a subset y. d S such that Defto. The effectve dmeso of f ( x ) the superposto sese s the smallest teger S y (8.5) 0 < y < d s The threshold 0.99 s arbtrary. Codto (8.5) meas that the fucto f ( x ) s almost a sum of d S dmesoal fuctos. A small value of d mples that there are o hgh-order teractos. S Defto. The effectve dmeso the trucato sese d s defed as T Sy (8.6) y {,,..., d } T A subscrpt y {,,..., d } meas that a subset y cotas puts wth dces from a set T {,,..., d } T. A low value of d mples that there are few mportat varables. T The effectve dmeso the trucato sese d T loosely speakg s equal to the umber of mportat factors the model. Idetfcato of mportat ad ot mportat varables allows to fx ot mportat varables at ther omal values. The resultat model would have lower complexty wth dmesoalty reduced from to d. A codto T dt << ofte occurs for practcal problems. 6

17 The value d does ot deped o the order whch the put varables are sampled, whle S d T does. The followg equalty s always satsfed: d S d T. For some problems chagg the order whch put varables are sampled ca dramatcally decrease d ( see f.e. Caflsch et al. [3], Sobol' ad T Kuchereko [33], Kuchereko ad Shah [6]). A straghtforward evaluato of the effectve dmesos from ther deftos s ot practcal geeral. Global sestvty aalyss allows to estmate the effectve dmesos at reasoable computatoal costs as dscussed Kuchereko et al. [7]. 9. Results. Itegrato It ths secto we cosder the tegral evaluato for dfferet classes of fuctos, we aalyze the tegrato error ad the rate of covergece. Classfcato of fuctos developed Kuchereko et al. [7] s based o ther effectve dmesos: Type A: Fuctos wth ot equally mportat varables. For type A models the effectve dmeso the trucato sese d <<. T Type B: Fuctos wth equally or almost equally mportat varables ad wth domat low order teracto terms AOVA decomposto. For such models the effectve dmeso the superposto sese d <<, whle S d. T Type C: Fuctos wth equally or almost equally mportat varables ad wth domat hgh order teracto terms AOVA decomposto. For type C models ds dt. Ths classfcato s summarzed Table. Iformato about relatve effceces of the three samplg techques s preseted the last two colums. 7

18 Fucto type A B C Table Classfcato of fuctos based o the effectve dmesos Descrpto A few domat varables o umportat subsets; oly low-order teracto terms are preset o umportat subsets; hghorder teracto terms are preset Relatoshp betwee S ad S tot S y tot / y >> S z tot / z S S j,, j S / S tot, S S j,, j S / S tot <<, d T d S QMC s more effcet tha MC << << Yes o << Yes Yes o o LHS s more effcet tha MC For types A ad B fuctos QMC tegrato ca atta the rate of covergece close to the theoretcal lmt O(/ ) regardless of the omal dmeso, although the presece of hgh order teracto terms the AOVA decomposto ca somewhat decrease the covergece rate. The AOVA decomposto a geeral case ca be preseted as f ( x) = f0 + f( x) + r( x), where rx ( ) are the AOVA terms correspodg to hgher order teractos. Ste [35] showed that varace computed wth the LHS desg s E( ε LHS ) = [ r( x)] dx O( ) +, H whle for MC E( ε MC ) = [ f ( )] [ ( )] ( ) x dx r x dx O + + H H. I the AOVA decomposto of type B fuctos, the effectve dmeso d S s small, hece rx ( ) s also small comparg to the ma effects. I the extreme case t s equal to, ad a fucto f(x) ca be preseted as a sum of oe-dmesoal fuctos f ( x) f f ( x) = +. Ths meas that oly oe-dmesoal projectos of the sampled pots play a role the fucto approxmato. For type B fuctos LHS ca acheve a much hgher covergece rate tha that of the stadard MC. To test the classfcato preseted above MC, LHS ad QMC tegrato methods were compared cosderg test fuctos preseted Tables -4. All fuctos are defed H. The theoretcal values of all tegrals apart from the fucto A are equal to. For the fucto A a value of a tegral s equal to I[ f ] = ( ( ) )

19 Table Covergece results for Type A fuctos Idex Fucto f ( x ) Dm Slopeα MC A ( ) A Slopeα QMC Slopeα LHS x j = j = 4x + a = + a a = a = 0 a 3 = = a 00 = 6.5 Table 3 Covergece results for Type B fuctos Idex Fucto f ( x ) Dm Slopeα MC B B B B x = 0. 5 x = Slopeα QMC Slopeα LHS x = + x = Table 4 Covergece results for Type C fuctos Idex Fucto f ( x ) Dm Slopeα C = / MC Slopeα QMC Slopeα LHS x C ( ) / = x Fgs. 5-8 show the root mea square error (RMSE) versus the umber of sampled pots. The root mea square error s defed as K / k ([ I f] I[ f]), k= ε = K k where K s a umber of depedet rus, I [ f ] s a value of the MC/QMC estmate of I[ f ] wth the use of sampled pots at k-th depedet ru. For the MC ad LHS method all rus were statstcally depedet. For QMC tegrato for each ru a dfferet part of the Sobol' LDS was used. For all tests K=50. The RMSE s approxmated by the formula c α,0< α <. Smaller c meas smaller RMSE. The value of α defes the covergece rate. The tred les ad correspodg values for α parethess are preseted Fgs. 5-8 ad the last three colums of Tables -4. 9

20 (a) (b) Fg. 5. RMSE versus the umber of sampled pots for type A models. (a) fucto A ( = 360 ); (b) fucto A ( = 00 ). 0

21 (a) (b) Fg. 6. RMSE versus the umber of sampled pots for type B models, fucto B. (a) fucto B ( = 30 ); (b) fucto B ( = 00 ).

22 (a) (b) Fg. 7. RMSE versus the umber of sampled pots for type B models, fucto B. (a) fucto B ( = 30 ); (b) fucto B ( = 00 )

23 (a) (b) Fg. 8. RMSE versus the umber of sampled pots for type C models. (a) fucto C ( = 0 ); (b) fucto C ( = 0 ) 3

24 Fg. 9. Values of a tegral for Fucto A versus for MC, QMC ad LHS samplg methods ( = 360 ). These results show that agreemet wth theory, QMC tegrato s superor to that of MC ad LHS both terms of the rate of covergece (larger α ) ad the absolute value of the tegrato error (costat c) all test cases. For fucto A at = =360 (a very hgh dmesoal problem) the expoet for algebrac decay the case of QMC tegrato α = 0.94 s very close to the theoretcally predcted asymptotcal value α = (Fgs. 5). The costatt c s lower for the QMC method tha that for both MC ad LHS methods. The LHS method shows the same covergece rate α 0.5 as the MC method ad t has oly a margally smaller costat c tha that of MC. For fucto A at = 00 the effcecy of QMC s hgher tha that of both MC ad LHS, although α s oly close to 0.7. As for the prevous case effceces of MC ad LHS methods are QMC QMC smlar wth LHS havg a margally smaller costat c. Itegrals for type B fuctos are cosdered at two dfferet dmesos ( = 30 ad = 00) to show a terestg effect of terplay betwee the effceces of QMC ad LHS methods (Fgs. 6,7). At = α QMC 0.96, whle 0.6 α LHS < 0.69 ad α MC 0.5. The costat c s lower for the QMC method tha that for the MC method at all ad the LHS method at α QMC 0 >. At = 00 the effcecy of the QMC method slghtly drops , whle the effcecy of the LHS method slghtly creases 0. α.7 LHS The effcecy of the MC method remas uchaged: α MC The costat c s lower for the LHS method at tha that for the QMC method, 5 whle at hgher > due to the hgher gradet the QMC method becomes more effcet tha the c LHS method. More research s eeded to expla the crease of wth. Here of sampled pots such that costats c LHS fuctos for whch LHS s sgfcatly more effcet tha MC. c s a crtcal umber c ( ) = c ( c QMC ). We ote, that type B s the oly type of 4

25 For the most dffcult type C fuctos we preseted results oly for a relatvely low dmesoal case of = 0 (Fgs. 8). Here the QMC method remas the most effcet method, whle LHS shows practcally o better performace tha MC. However, hgher dmesos the covergece rate of QMC tegrato drops ad becomes close to that of the other two samplg methods. Fg. 9 presets covergece plots for the fucto A wthout averagg over depedet rus. For the QMC method the covergece s mootoc ad very fast. A covergece curve reaches the theoretcal value 4, whle for both MC ad LHS covergece curves are oscllatg ad approach 9 the theoretcal value oly at. The QMC method s approxmately 30 tmes faster tha the LHS method. The advatage of usg MC or QMC schemes over LHS s that ew pots ca be added cremetally: formulas (.) ad (4.) ca be wrtte as I = I + f( x). Here f ( x ) s the ew updated value of a tegrad (for the QMC method ts value s equal to f ( q )). It s relatvely easy to have a termato crtero that ca be voked cremetally. The drawback of LHS s that t s ot possble to cremetally add a ew pot whle keepg the old LHS desg: formula (3.) radom permutatos of {,..., } are dfferet from radom permutatos of {,..., }. Ths meas that addg a ew th pot to a already sampled set of pots ad a set { f( x )}, =,..., of fucto values requres to resample all pots ad to recalculate the whole set { f( x )}, =,..., of fucto values at ew { x}, =,...,. 0. Results. Evaluato of quatles for ormally dstrbuted varates May practcal smulato problems requre the geerato of ormally dstrbuted varables. I ths secto we compare MC, LHS ad QMC samplg methods for evaluato of quatles. We cosder evaluato of low 5% ad hgh 95% percetles for the cumulatve dstrbuto fucto of a radom varable f ( x) = x, where x are depedet stadard ormal varates, s the dmeso, = 5. = The cosdered fucto s dstrbuted accordg to the ch-squared dstrbuto wth 5 degrees of freedom: f x χ ( ) (5). The values of 5% ad hgh 95% percetles are.46 ad.07, respectvely. The algorthm for geeratg -dmesoal radom ormal varables usg uformly dstrbuted varables cossts of the followg steps: a) geerate -dmesoal radom vector u uformly dstrbuted betwee 0 ad usg radom umbers, LHS or quas radom sequeces; b) trasform every elemet of u to a stadard ormal vector x wth zero mea ad ut varace usg the verse ormal cumulatve dstrbuto fucto: x F ( u ) =. 5

26 Fg. 0 presets covergece of umercal estmates to the true values of quatles. Log(MeaError Low) "MC" "QMC" "LHS" Expo. ("QMC") Expo. ("MC") Expo. ("LHS") Log() (a) Log(MeaError Hgh) "MC" "QMC" "LHS" Expo. ("MC") Expo. ("QMC") Expo. ("LHS") Log() (b) Fg. 0. Comparso betwee MC, LHS ad QMC methods for evaluato of quatles: RMSE error versus. (a) Low quatle = 0.05; (b) Hgh quatle = 0.95, dmeso = 5. MC damods, LHS tragles, QMC squares. Results show a superor covergece of the QMC method. LHS provdes margally hgher covergece tha MC. 6

27 . Coclusos We compared effceces of three samplg methods: MC, LHS ad QMC based o Sobol sequeces. It was show that Sobol sequeces possess uformty propertes whch other samplg techques do ot have (Propertes A ad A ). Optmzed LHS satsfes Property A low dmesos but hgh dmesos due to the cost hgh of optmzato t s very dffcult to verfy ths Property. I ay case t ca oly be verfed emprcally for optmzed LHS, whle Sobol sequeces possess these addtoal uformty propertes by costructo. Comparso of L dscrepaces revealed that the QMC method has the lowest dscrepacy up to dmeso 0. We used a umber of test fuctos of varous complextes for hgh dmesoal tegrato. Comparso showed that for types A ad C fuctos LHS shows oly a slght mprovemet over MC. The covergece rate of the QMC method for types A ad B fuctos s close to O(/), whle the MC method has the covergece rate close to O(/ ) ad the LHS method outperformg MC oly for type B fuctos. For type B fuctos superorty of the QMC method over the LHS method ca set oly at some large umber of sampled pots, whle at small umber of sampled pots the LHS method ca have a smaller varace tha the QMC method. For type C fuctos covergece of the QMC method sgfcatly drops, however t stll remas the most effcet method amog the three samplg techques. We also compared MC, LHS ad QMC samplg methods for evaluato of quatles ad showed that QMC remas the most effcet method amog the three techques. Although there has bee a sgfcat progress mprovg space fllg propertes of LHS, due to the hgh cost of optmzato volved, eve the most effcet techques ca ot be appled for solvg hgh dmesoal problems such as tegrato usg optmzed LHS. O the other sde optmzed LHS have prove to be very effectve applcato to metamodellg where ofte oly a small umber of sampled pots are requred ad a samplg desg of such pots ca be effcetly optmzed. A mportat practcal dsadvatage of LHS over MC or QMC schemes s that ew pots ca ot be added sequetally, whch makes t very dffcult to use tegrato where termato crtera are ofte voked cremetally ad other methods such as adaptve samplg. Ackowledgemets The authors would lke to thak Bertrad Iooss for hs valuable commets ad provdg codes for producg optmzed LHS. 7

28 Refereces [] BRODA Ltd [] J. Burkardt, Departmet of Scetfc Computg at Florda State Uversty; GU LGPL lcese [3] R.E. Caflsch, W. Morokoff, A.B. Owe, Valuato of mortgage backed securtes usg Browa brdges to reduce effectve dmeso, The J. of Comput. Fac. (997) [4] G. Dambl, M. Couplet, B. Iooss, umercal studes of space fllg desgs: optmzato algorthms ad subprojecto propertes, J. of Smul., submtted 03. [5] K.T. Fag, M. Chag-Xg, P. Wker, Cetered L-dscrepacy of radom samplg ad Lat hypercube desg, ad costructo of uform desgs, Math. Comput. 7(37) (00) [6] K.T. Fag, R. L, A. Sudjato, Desg ad Modelg for Computer Expermets, Chapma ad Hall/CRC, ew York, 006. [7] P. Glasserma, Mote Carlo methods facal egeerg, Sprger, 003. [8] F. Hckerell, A geeralzed dscrepacy ad quadrature error boud, Math. Comp. 67 (998) [9] T. Homma, A. Saltell, Importace measures global sestvty aalyss of olear models, Relab. Eg. Syst. Saf. 5 (996) 7. [0] R. Ima, M. Shortecarrer, A FORTRA 77 program ad users gude for the geerato of Lat hypercube ad radom samples for use wth computer models, Sada atoal Laboratores, Albuquerque (M), Rep. SAD83, 984. [] B. Iooss, L. Boussouf, V. Feullard, A. Marrel, umercal studes of the metamodel fttg ad valdato processes, It. J. of Adv. Syst. ad Meas. 3 (00) -. [] P. Jaeckel, Mote Carlo methods face, Lodo, Joh Wley & Sos, 00. [3] R. J, W. Che, A. Sudjato, A effcet algorthm for costructg optmal desg of computer expermets, J. of Stat. Pla. ad Iferece 34 (005) [4] J. Kalagaam, U. Dwekar, A effcet samplg techque for off-le qualty cotrol, Techometrcs 39(3) (997) [5] J.R. Koehler, A.B. Owe, Computer expermets, Hadb. of Stat. S. Ghosh ad C.R. Rao ed., Elsever Scece B.V. 3, 996, [6] S. Kuchereko,. Shah, The Importace of beg Global, Applcato of global sestvty aalyss Mote Carlo opto prcg, Wlmott J. July (007) 8-9. [7] S. Kuchereko, B. Fel,. Shah, W. Mautz, The detfcato of model effectve dmesos usg global sestvty aalyss, Relab. Eg. ad Syst. Saf. 96 (0) [8] M. McKay, R. Beckma, W. Coover, A comparso of three methods for selectg values of put varables the aalyss of output from a computer code, Techometrcs () (979) [9] W. Morokoff, R.E. Caflsch, Quas Mote-Carlo tegrato, J. of Comput. Phys. (995)

29 [0] M.D. Morrs, T.J. Mtchell, Exploratory desgs for computatoal expermets, J. of Stat. Pla. ad Iferece 43 (995) [] H. ederreter, Radom umber geerato ad quas-mote Carlo methods, Soc. for Id. ad Appl. Math, 99. [] A. Owe, A cetral lmt theorem for Lat hypercube samplg, J. of the R. Stat. Soc. Ser. B 54(3) (99) [3] A. Owe, Cotrollg correlatos Lat hypercube samples, J. of Am. Stat. Assoc. 89 (994) [4] J.S. Park, Optmal Lat-hypercube desgs for computer expermets, J. of Stat. Pla. ad Iferece 39 (993) 95. [5] K. Pearso, O the geeral theory of skew correlato ad o-lear regresso, Vol. IV of Math. Cotrb. to the Theory of Evol., Drapers' Compay Research Memors, Reprted : Early, Lodo:Dulau & Co., 905. [6] A. Saltell, S. Taratola, F. Campologo, Sestvty aalyss as a gredet of modelg, Stat. Sc. 5(4) (000) [7] A. Saltell, S. Taratola, O the relatve mportace of put factors mathematcal models: Safety assessmet for uclear waste dsposal, J. of Am. Stat. Assoc. 97 (00) [8] A. Saltell, Makg best use of model valuatos to compute sestvty dces, Comput. Phys. Commu. 45 (00) [9] A. Saltell, P. Ao, I. Azz, F. Campologo, M. Ratto, S. Taratola, Varace based sestvty aalyss of model output, Desg ad estmator for the total sestvty dex, Comput. Phys. Commu. 8() (00) [30] I. Sobol', O the dstrbuto of pots a cube ad the approxmate evaluato of tegrals, Comput. Math. ad Math. Phys. 7 (967) 86-. [3] I. Sobol', Uformly dstrbuted sequeces wth addtoal uformty propertes, USSR Comput. Math. ad Math. Phys. 6(5) (976) [3] I. Sobol', Sestvty estmates for olear mathematcal models, Mat. Modelrovae () (990) -8 ( Russa), traslated I.M. Sobol', Sestvty estmates for olear mathematcal models, Math. Model. ad Comput. Exp. (993) [33] I. Sobol, S. Kuchereko, O global sestvty aalyss of quas-mote Carlo algorthms, Mt.- Carlo Method ad Smul. () (005) -9. [34] I. Sobol', D. Asotsky, A. Kre, S. Kuchereko, Costructo ad comparso of hgh-dmesoal Sobol geerators, Wlmott J. ov (0) [35] M. Ste, Large sample propertes of smulatos usg Lat hypercube samplg, Techometrcs 9 (987) [36] B. Tag, Orthogoal array-based Lat hypercube, J. of Am. Stat. Assoc. 88 (993) [37] R. Wag, U. Dwekar, C.E. Grégore Padró, Effcet samplg techques for ucertates rsk aalyss, Evro. Prog. 3() (004)

30 [38] T.T Warock, Computatoal vestgatos of low-dscrepacy pot sets, Applcatos of umber Theory to umercal Aalyss, Academc Press, ew York, 97, pp