A benchmark study on intelligent sampling techniques in Monte Carlo simulation

Transcription

1 624 A benchmark study on ntellgent samplng technques n Monte Carlo smulaton Abstract In recent years, new, ntellgent and ecent samplng technques or Monte Carlo smulaton have been developed. However, when such new technques are ntroduced, they are compared to one or two exstng technques, and ther perormance s evaluated over two or three problems. A lterature survey shows that benchmark studes, comparng the perormance o several technques over several problems, are rarely ound. Ths artcle presents a benchmark study, comparng Smple or Crude Monte Carlo wth our modern samplng technques: Importance Samplng Monte Carlo, Asymptotc Samplng, Enhanced Samplng and Subset Smulaton; whch are studed over sx problems. Moreover, these technques are combned wth three schemes or generatng the underlyng samples: Smple Samplng, Latn Hypercube Samplng and Antthetc Varates Samplng. Hence, a total o teen samplng strategy combnatons are explored heren. Due to space constrans, results are presented or only three o the sx problems studed; conclusons, however, cover all problems studed. Results show that Importance Samplng usng desgn ponts s extremely ecent or evaluatng small alure probabltes; however, ndng the desgn pont can be an ssue or some problems. Subset Smulaton presented very good perormance or all problems studed heren. Although smlar, Enhanced Samplng perormed better than Asymptotc Samplng or the problems consdered: ths s explaned by the act that n Enhanced Samplng the same set o samples s used or all support ponts; hence a larger number o support ponts can be employed wthout ncreasng the computatonal cost. Fnally, the perormance o all the above technques was mproved when combned wth Latn Hypercube Samplng, n comparson to Smple or Antthetc Varates samplng. K.R.M. dos Santos a A.T. Beck b a,b Unversty o São Paulo São Carlos School o Engneerng Department o Structural Engneerng Avenda Trabalhador São Carlense, 400, , São Carlos, SP, Brazl. Correspondng autor: a ketson.santos@gmal.com b atbeck@sc.usp.br Receved Accepted Avalable onlne Keywords Structural relablty; Monte Carlo smulaton; ntellgent samplng technques; benchmark study.

2 625 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton INTRODUCTION Snce the early begnnngs n the sxtes and seventes, structural relablty analyss has reached a mature stage encompassng sold theoretcal developments and ncreasng practcal applcatons. Structural relablty methods have permeated the engneerng proesson, ndng applcatons n code calbraton, structural optmzaton, le extenson o exstng structures, le-cycle management o nrastructure rsks and costs, and so on. Durng the past 30 years, sgncant advances were obtaned n terms o transormaton methods (FORM, SORM), as well as n terms o smulaton technques. Transormaton methods were ound to be ecent n the soluton o problems o moderate dmensons and moderate non-lnearty. Smulaton technques (Metropols and Ulam, 949; Metropols et al., 953; Robert and Casella, 20) have always allowed the soluton o hghly nonlnear hgh-dmensonal problems, although computatonal cost used to be a serous lmtaton. Ths s especally true when alure probabltes are small and lmt state unctons are gven numercally (e.g., nte element models) (Beck and Rosa, 2006). Wth the recent and exponental advance o computatonal processng power, Monte Carlo smulaton usng ntellgent samplng technques s becomng ncreasngly more vable. Several ntellgent samplng technques or Monte Carlo smulaton have been proposed n recent years (Au and Beck, 200; Au, 2005; Au et al., 2007; Bucher, 2009; Schan et al., 20a; Schan et al., 20b; Schan et al., 204; Naess et al., 2009; Naess et al., 202). However, when such technques are ntroduced, they are generally compared wth one or two exstng technques, and ther perormance s evaluated over two or three problems. It s dcult to nd n the publshed lterature benchmark studes where several samplng technques are compared or a larger number o problems (Au et al., 2007; Engelund and Rackwtz, 993; Schuëller and Prandlwarter, 2007). Ths artcle presents a benchmark study, comparng Smple or Crude Monte Carlo wth our modern samplng technques: Importance Samplng Monte Carlo, Asymptotc Samplng, Enhanced Samplng and Subset Smulaton over sx problems. Moreover, these technques are combned wth three schemes or generatng the underlyng samples: Smple Samplng, Latn Hypercube Samplng and Antthetc Varates Samplng. Hence, a total o teen samplng strategy combnatons are explored heren. Due to space constrans, results are presented or only three o the sx problems studed. The conclusons, however, cover the sx problems studed. The remander o the artcle s organzed as ollows. The structural relablty problem s ormulated n Secton. The basc technques or generatng the underlyng samples are presented n Secton 2. The ntellgent samplng technques or alure probablty evaluaton are presented n Secton 3. Problems are studed n Secton 4, and Concludng Remarks are presented n Secton 5. 2 FORMULATION 2. Relablty problem Let { } =, 2,..., m be a random varable vector descrbng uncertantes n loads, materal strengths, geometry, and models aectng the behavor o a gven structure. A lmt state equaton g( ) s wrtten such as to dvde the alure and survval domans (Madsen et al., 986; Melchers, 999): Latn Amercan Journal o Solds and Structures 2 (205)

3 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton 626 Ω = { x g( x) 0 } { x g( x) } s the alure doman Ω = > 0 s the survval doman s () The alure probablty s gven by: Ω ( ) P = P Ω = x dx (2) where ( x ) s the probablty densty uncton o random vector. The bggest challenge n solvng the smple mult-dmensonal ntegral n equaton (2) s that the ntegraton doman s generally not known n closed orm, but t s gven as the soluton o a numercal (e.g., nte element) model. Monte Carlo smulaton solves the problem stated n equaton (2) by generatng samples o random varable vector, accordng to dstrbuton uncton ( x ), and evaluatng weather each sample belongs to the alure or survval domans. 2.2 Crude Monte Carlo Smulaton One straghtorward way o perormng the ntegraton n equaton (2) s by ntroducng an ndcator uncton I[ x ], such that I x = x Ω and I [ x ] = 0 x Ωs. Hence, the ntegraton can be perormed over the whole sample space: P Ω ( ) = I d E I x x x x (3) In equaton (3), one recognzes that the rght-hand term s the expected value ( E []. ) o the ndcator uncton. Ths expected value can be estmated rom a sample o sze n by: n E P = P = I x = x = (4) n I j n j= n where n s the number o samples whch belong to the alure doman and n s the total number o samples. The varance o P s gven by: n n j= ( ) 2 j Var P = I P x (5) In Crude Monte Carlo smulaton, sample vector x j can be generated usng the Smple Samplng, Antthetc Varates Samplng or Latn Hypercube Samplng, as detaled n the sequence. 3 BASIC SAMPLING TECHNIQUES In ths paper three basc samplng technques are nvestgated: Smple Samplng, Latn Hypercube Samplng and Antthetc Varates Samplng. These basc samplng schemes are not specc or the Latn Amercan Journal o Solds and Structures 2 (205)

4 627 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton soluton o structural relablty problems: they can be employed n the numercal soluton o ntegrals (lke equaton (2)), n the spatal dstrbuton o ponts n a gven doman (or surrogate modelng, or nstance), and so on. These technques are urther combned wth specc technques or solvng structural relablty problem va Monte Carlo smulaton, as descrbed n Secton Smple Samplng The use o Monte Carlo smulaton n solvng general problems nvolvng random varables (and/or stochastc processes) requres the generaton o samples rom random varable vector. The most straghtorward way o generatng samples o a vector o random varables s by an nverson o ther cumulatve dstrbuton uncton F ( x ): I. Generate a random vector o components v ( m) = { vj}, j=,..., m, unormly dstrbuted between 0 and ; II. Use the nverse o the cumulatve dstrbuton uncton, such that { xj} = { F ( vj) }, j=,..., m. When components o vector are correlated, the correlaton structure can be mposed by premultplcaton by the Cholesky-decomposton o the correlaton matrx. Detals are gven n Madsen et al. (986) and Melchers (999). 3.2 Antthetc Varate Samplng In Smple Samplng, a set o random numbers u ( n ) = { uj, u2j,..., unj} s employed to obtan n samples o random varable j. In Antthetc Varate samplng, the dea s to dvde the total t number o samples by two, and to obtan two vectors u = { u, u2,..., u n /2} and t u = { u, u2,..., u n /2}. Now consder that any random quantty P (ncludng the alure a b probablty, P ) can be obtaned by combnng two unbased estmators P and P, such that: t P c a b P + P = (6) 2 The varance o ths estmator s: Var P c a b a b Var P Var P Cov P P = ( 2, ) (7) = u and P b = ( u ), a negatve correlaton s mposed, Cov P a, b P becomes negatve, hence the varance o P c a b s reduced n comparson to the varances o P or P. The Antthetc Varates technque, when appled by tsel, may not lead to sgncant mprovement; when appled n combnaton wth other technques, more sgncant mprovements can be acheved. Thus, by makng P a ( ) 3.3 Latn Hypercube Samplng The Latn Hypercube Samplng (LHS) was ntroduced by McKay et al. (979). The dea o Latn Hypercube Samplng s to dvde the random varable doman n strpes, where each strpe s sam- Latn Amercan Journal o Solds and Structures 2 (205)

5 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton 628 pled only once (McKay et al., 979; Olsson et al., 2003), such as n Fgure. Ths procedure guarantees a sparse but homogeneous cover o the samplng space. Fgure : Latn Hypercube Samplng. To obtan the Latn Hypercube, let m be the number o random varables and n the number o samples. A matrx P ( n m) s created, where each column s a random permutaton o,...,n. A matrx R( n m ) s created, whose elements are unorm random numbers between 0 and. Then, matrx S s obtaned as (Olsson et al., 2003): The samples are obtaned rom S such that: S= ( P R) (8) n j j ( ) x = F s (9) j where F s the nverse cumulatve dstrbuton uncton o random varable j j. In order to reduce memory consumpton, equaton (8) can be solved s scalar ashon. The ollowng algorthm s adopted:. Start the loop or random varablej : 2. Generate the rst column o matrx P, as a random permutatonn o,...,n ; 3. Start the loop or the number o smulatons ; 4. Generate a sngle unorm random number between 0 and ; thus, t s not necessary to create matrx R ; 5. Compute a number s, and use equaton (9) to compute the element x j. Thus, matrx S also does not need to be computed; 6. Repeat step 4 untl = n ; 7. Repeat step 2 untl j= m. Latn Amercan Journal o Solds and Structures 2 (205)

6 629 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton For llustraton purposes, Fgure 2 shows hstograms obtaned by Smple Samplng (Fgure 2a), Antthetc Varates Samplng (Fgure 2b) and Latn Hypercube Samplng (Fgure 2c), or a random varable wth normal dstrbuton, wth mean equal to 0.0 and standard devaton equal to 0.5. Three thousand samples were used to compute these hstograms. One observes the smoother dstrbutons obtaned by means o Latn Hypercube Samplng. a) Smple Samplng b) Antthetc Samplng c) Latn Hypercube Samplng Fgure 2: Hstograms obtaned or 3000 samples o a sngle random varable ~N(0, 0.5). 4 INTELLIGENT SAMPLING TECHNIQUES 4. Importance Samplng Monte Carlo Importance Samplng Monte Carlo centered on desgn ponts s a powerul technque to reduce the varance n problems nvolvng small and very small alure probabltes. The drawback s that t needs pror locaton the desgn ponts. Desgn ponts can be located usng well-known technques o the Frst Order Relablty Method, or FORM (Madsen et al., 986; Melchers, 999). However, ndng the desgn pont can be a challenge or hghly non-lnear problems. Recall the undamental Monte Carlo smulaton equaton (equaton 3). I numerator and denomnator o ths expresson are multpled by a convenently chosen samplng uncton h ( x ), the result s unaltered: ( x) ( x) P I = h ( ) d E I Ω x x x h x h ( x) ( x) (0) The expected value, n the rght-hand sde o equaton (0), can be estmated by samplng usng: P = I x = ( x) ( x) n I n x = h () By properly choosng the samplng uncton h ( x ), one can ncrease the number o successes, or the number o sampled ponts allng n the alure doman. Ths s normally accomplshed by centerng the samplng uncton h ( x ) n the desgn pont. In other words, the samplng uncton s x, but wth the mean replaced by the desgn usually the jont probablty densty uncton ( ) Latn Amercan Journal o Solds and Structures 2 (205)

7 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton 630 * pont coordnates ( ) x. However, observe that n comparson to equaton (4), each sampled pont allng n the alure doman ( I x = ) s assocated to a samplng weght ( x) h( x). For structures or components wth multple alure modes assocated as a seres system, the sam- desgn pont, such that: plng uncton can be constructed by a weghted sum o unctons h ( x ), centered at the th h nls ( ) = ph ( ) x x (2) = where nls s the number o lmt states and p s the weght related to the lmt state. For a seres system, where alure n any mode characterzes system alure, p s obtaned as: p = nls = ( β) Φ Φ ( β) (3) For parallel system, smlar expressons are gven n (Melchers, 999). The applcaton o Importance Samplng Monte Carlo n combnaton wth Smple Samplng or Anttethc Varates Samplng s straghtorward. When applyng Importance Samplng n combnaton wth Latn Hypercube Samplng, there s a possblty that most samples be generated on one sde o the lmt state. Ths can be avoded by adoptng a transormaton proposed by Olsson et al. (2003). Ths transormaton rotates the Latn Hypercube, on the standard normal space, accordng the orentaton o the desgn pont. Ths procedure s presented n Fgure 3. Further detals are gven n Olsson et al. (2003). Fgure 3: Orgnal and rotated Latn Hypercube, adapted rom Olsson et al. (2003). 4.2 Asymptotc Samplng The Asymptotc Samplng technque (Bucher, 2009; Schan et al., 20a; Schan et al., 20b; Schan et al., 204) was developed based on the asymptotc behavor o alure probabltes as the Latn Amercan Journal o Solds and Structures 2 (205)

8 63 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton standard devaton o the random varables tends to zero. One advantage over Importance Samplng s that t does not requre prevous knowledge o the desgn pont. Asymptotc Samplng s based on choosng actors <, related to the standard devatons o the random varables as: σ σ = (4) where σ s the standard devaton o random varable and σ s the moded standard devaton or the same random varable. For small values o <, larger standard devatons, hence also larger alure probabltes are obtaned. For each pre-selected value o a Monte Carlo Smulaton s perormed, n order to obtan the relablty ndex β ( ). Followng Bucher (2009), the asymptotc behavor o β wth respect to can be descrbed by a curve: β = C A + B (5) Where A, B and C are constants to be determned by nonlnear regresson (e.g. least squares method), usng (, β ) as support ponts. Ater ndng the regresson coecents, the relablty ndex or the orgnal problem s estmated by makng = n equaton (3), such that β= A+ B. Fnally, the probablty o alure s obtaned as P =Φ( β), where Φ (). s the standard normal cumulatve dstrbuton uncton. The Monte Carlo smulatons or derent values o β (support ponts) can be obtaned by Smple Samplng, Latn Hypercube Samplng or Antthetc Varates Samplng. The parameters nluencng the perormance o Asymptotc Samplng are the number o support ponts and the range o values consdered. In ths paper, these parameters are not studed: they are xed at values provdng satsactory responses: 5 support ponts are employed wth varyng rom 0.4 to 0.7 or rom 0.5 to 0.8, dependng on the problem. 4.3 Enhanced Samplng The Enhanced Samplng technque was proposed by Naess et al. (2009, 202) and s based on explorng the regularty o the tals o the PDF s. It ams at estmatng small or very small alure probabltes or systems. The orgnal lmt state uncton M = g( x, x2,..., xm) s used to construct M λ, wth 0 λ, such that: a set o parametrc unctons ( ) M ( ) M ( ) M λ = λ µ (6) where µ M s the mean value o M. Thus, t s assumed that the behavor o alure probabltes wth respect to λ can be represented as: P q a b ( λ) exp ( λ ) Latn Amercan Journal o Solds and Structures 2 (205) c wth λ (7)

9 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton 632,, the parameters n equaton 5 can be ound by nonlnear regresson. The probablty o alure or the orgnal problem s estmated or λ=. One large advantage over Asymptotc Samplng s that, rom one sngle Monte Carlo smulaton run, the whole range o parametrc unctons M( λ ) can be evaluated. Hence, a large number o support ponts can be used, wth no penalty n terms o computatonal cost. For systems, each lmt state or component Mj = gj( x, x2,..., xm), wth j=,..., nls, must be evaluated. Thus, a parametrc set o equatons s obtaned, such as: Usng a set o support ponts λp ( λ) M ( ) M ( ) λ = λ µ (8) j j M j Thereore, or a seres system the probablty o alure s obtaned as: P = P M λ For a parallel system, the probablty o alure s obtaned as: nls { j( ) 0} (9) j= P = P M λ nls { j( ) 0} (20) j= and or a seres system wth parallel subsystems, the probablty o alure s gven by: where C j s a subset o,...,nls, or j=,..., l. l P = P j= Cj { M( λ) 0} (2), and the values o λ or whch alure probabltes are evaluated. Wthn ths paper, these values are kept xed: 00 support ponts are used, wth λ varyng rom 0.4 to 0.9. The parameters o Enhanced Samplng are the number o support ponts λp ( λ) 4.4 Subset Smulaton The Subset Smulaton technque was proposed by Au and Beck (200) amng to estmate small and very small probabltes o alure n structural relablty. The basc dea o ths technque s to decompose the alure event, wth very small probablty, nto a sequence o condtonals events wth larger probabltes o occurrence. For the later, small sample Monte Carlo smulaton should be sucent. Snce Smple Samplng s not a good opton to generate condtonal samples, the Markov Chan Monte Carlo and the moded Metropols-Hastngs algorthms are used. The estmaton o condtonal probabltes n Subset Smulaton depends on the choce o the ntermedates events. Consder the alure event E. The probablty o alure assocated to t s gven by: Latn Amercan Journal o Solds and Structures 2 (205)

10 633 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton ( ) ( ) P E = P Ω = x d x= I x x d x (22) Consderng a decreasng sequence o events E, such as E E2... Em = E, thus: Ω Ω E k k = E, k=,..., m (23) = Hence, the probablty o alure s evaluated as: m m P = P E P E P E P E E = = (24) = = 2 In equaton (22), P E can be evaluated by means o Crude Monte Carlo, usng Smple Samplng, Latn Hypercube Samplng or Antthetc Varates Samplng. On the other hand, the condtonal probabltes P E E are estmated by means o Markov Chans usng the Moded Metropols-Hastngs algorthm (Au and Beck, 200; Au, 2005; Au et al., 2007). In Subset Smulaton the ntermedate events E are chosen n an adaptatve way. In structural relablty the probablty o alure s estmated by: As the alure event s dened by: The ntermedate alure events are dened by: { ( ) 0} Latn Amercan Journal o Solds and Structures 2 (205) P = P E = P g (25) { ( ) 0} E= g (26) { ( ) b} E = g (27) wth =,..., m. Hence, the sequence o ntermedate events E s dened by the set o ntermedate lmt states. For convenence, the condtonal probabltes are establshed prevously, such that P[ E E ] = P0. Also, the number o samples n SS (e.g. n SS = 500) at each subset s prevously establshed. Hence, sets o samples 0,k, wth k=,..., nss are obtaned. The lmt state unctons are evaluated or 0,k, resultng n vector Y0, k= g( 0, k). Components o vector Y 0,k are arranged n ncreasng order, resultng n vector Y 0,k +. The ntermedate lmt o alure b s establshed as the sample Y 0,k + + or whch k= Pn 0 SS, such that P[ { Y b} ] = P 0, Pn 0. Thus, there are 0 SS Pn 0 SS samples on the alure doman dened by ntermedate lmt b. From each o these samples, by means o Markov Chan smulaton, ( P0) nss condtonal samples ( 0 ),k are generated, wth dstrbuton P(. E ). The lmt state uncton s evaluated or these samples resultng n vector Y ( 0 ), k= g( ( 0 ), k) and n ordered vector Y + ( 0 ),k, both related to ntermedate lmt o alure b 2, + where k= Pn 0 SS. Thereore, P[ { Y b} ] ( ) 2 = P0. Thus, the next ntermedate event E 2= 0, Pn 0 SS

11 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton 634 { Y b 2 } s dened. One can notce that P[ E2 E] = PY [ b2 Y b] = P0. The Pn 0 SS condtonal samples wll be the seeds o the condtonal samples or the ollowng level. Repeatng the process, one generates condtonal samples untl the nal lmt b m s reached, such that b m = 0. Ths process s llustrated n Fgure 4. Fgure 4: Subset sample generaton usng Markov Chans. The random walk s dened by ts probablty dstrbuton (e.g. unorm) and by the standard RW devaton σ, whch s consdered as the product o a value α by the standard devaton o the RW problem, such that σ = α σ. Parameters o ths algorthm are α, the number o samples or each subset ( n SS ) and the condtonal probablty P 0. Derent values are used or these parameters or derent problems, as detaled n the next Secton. In all cases, the random walk s modeled by a unorm probablty dstrbuton uncton. 5 COMPARATIVE PERFORMANCE OF SIMULATION STRATEGIES In ths study, three basc samplng technques are combned wth Crude Monte Carlo and wth our modern samplng technques: Importance Samplng Monte Carlo, Asymptotc Samplng, Enhanced Samplng and Subset Smulaton. Hence, teen samplng schemes are nvestgated wth respect to ther perormance n solvng sx structural relablty problems. Due to space lmtatons, only results or the three most relevant problems are presented heren. Conclusons relect the sx problems studed. The ollowng analyss procedure conssts n two steps. The rst step s a study o the convergence o the probablty o alure and ts coecent o varaton or ncreasng numbers o samples. Snce the requred number o samples or Subset Smulaton s much lower than or the other technques, the convergence study s not perormed or Subset Smulaton. The second step s a comparson o the results or all technques, ncludng Subset Smulaton, consderng a small number o samples. For Examples and 2, the lmt state unctons are analytc; hence processng tme s not a relevant ssue. Example 3 nvolves a Fnte Element model wth physcal and geometrcal nonlneartes; hence processng tme s more relevant. Latn Amercan Journal o Solds and Structures 2 (205)

12 635 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton 5. Example : non-lnear lmt state uncton The rst example has a nonlnear lmt state uncton, where the random varables are modeled by non-gaussan probablty dstrbuton unctons. The lmt state uncton s (Melchers and Ahamed, 2004): g(, 2, 3, 4, 5, 6) = (28) 8 where, wth =,...,6, are the random varables. The parameters o the probablty dstrbutons are gven n Table. Random Varable Dstrbuton Mean St. Dev. Webull (mnma) Log-normal Gumbel Unorm Exponencal Normal Table : Random varables o Example (Melchers and Ahamed, 2004). The reerence probablty o alure s obtaned usng Crude Monte Carlo smulaton wth Smple Samplng and samples. The reerence probablty o alure s (β = 4.03). Ths example ams to evaluate the perormance o the ntellgent samplng technques n a problem wth nonlnear lmt state uncton nvolvng non-gaussan dstrbuton unctons. Fgures 5, 6 and 7 show convergence plots or the mean and coecent o varaton (c.o.v) o the P, or Smple Samplng, LHS and Asymptotc Samplng, respectvely. Convergence results were evaluated or number o samples varyng rom 0 3 to 0 6. Results are shown or Crude, Importance, Asymptotc and Enhanced Samplng. For Asymptotc Samplng, the parameter vares rom 0.4 to 0.7, wth 5 support ponts n ths range. Subset smulaton s not ncluded, because computatons are truly expensve or large numbers o samples. On the other hand, only a ew samples are requred to acheve smlar results wth Subset smulaton, as observed n Table 2. Two strkng results can be observed n Fgures 5 to 7. For all basc samplng technques, the c.o.v. or Importance Samplng converges very ast to near zero. Ths s a very postve result. On the other hand, t s observed that the alure probablty also converges very ast, but wth a bas w.r.t. the reerence result. Ths bas, although small and acceptable, s o some concern, and s ntroduced by the samplng uncton. It s also observed, n Fgures 5 to 7, that or Asymptotc Samplng the c.o.v. convergence s qute unstable, wth results oscllatng sgncantly, and convergence or P also shows some bas. Both results are especally true or Smple and Anthytetc Varates Samplng, and less so or LHS. In act, s observed that LHS mproves the results or all samplng technques llustrated n Fgures 5 to 7. Latn Amercan Journal o Solds and Structures 2 (205)

13 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton 636 a) Mean b) Coecent o varaton. Fgure 5: Convergence o P (a) and ts c.o.v. (b) usng Smple Samplng n Example. a) Mean b) Coecent o varaton. Fgure 6: Convergence o P (a) and ts c.o.v. (b) usng Latn Hypercube Samplng n Example. a) Mean b) Coecent o varaton. Fgure 7: Convergence o P (a) and ts c.o.v. (b) usng Antthetc Samplng n Example. A quanttatve comparson o the perormance o all samplng technques, n soluton o Problem, s presented n Table 2. These results are computed or a much smaller number o samples: 2,300. In Subset Smulaton the ollowng parameters are adopted: n SS = 500, α =.5, and P 0 = 0.. Latn Amercan Journal o Solds and Structures 2 (205)

14 637 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton One observes n Table 2 that Crude Monte Carlo and Asymptotc Samplng do not lead to any results or such a small number o samples. Importance Samplng leads to the smallest c.o.v.s, but wth devatons o around 8 to 30% rom the reerence result. Enhanced Samplng works, but devatons and c.o.v.s are rather large. For such a small number o samples, Subset Smulaton provdes the best results, wth acceptable c.o.v.s and devatons varyng rom around 5 to 60%. These devatons could be urther reduced by a margnal ncrease n the number o samples. Estmaton Technque P c.o.v. Devaton (%) Smple Samplng Crude Monte Carlo Importance Samplng Asymptotc Samplng Enhanced Samplng Latn Hypercube Samplng Subset Smulaton Crude Monte Carlo Importance Samplng Asymptotc Samplng Enhanced Samplng Subset Smulaton Crude Monte Carlo Antthetc Varates Samplng Importance Samplng Asymptotc Samplng Enhanced Samplng Subset Smulaton Table 2: Comparson o results or 2,300 samples n Example. 5.2 Example 2: Hper-estatc structural system (truss) In order to nvestgate the perormance o the ntellgent samplng technques n a structural system problem, an hper-estatc truss s studed. Servce alure s characterzed by alure o any component (bar) o the truss, whch can be due to bucklng (compressed bars) or to yeldng (tensle bars). System alure, characterzed by alure o a second bar (any), gven alure o the hper-estatc bar (any), s not consdered n ths study (Verzenhass, 2008). The truss and ts dmensons are shown n Fgure 8. The geometrcal propertes o truss bars are shown n Table 3. Table 4 shows the random varables consdered. Sx lmt state unctons are employed to solve the problem: our unctons related to elastc bucklng o bars, 2, 3 and 6: (,,, ) ( av bh) g E LVH 2 L Latn Amercan Journal o Solds and Structures 2 (205) πei = + +, wth =,2,3,6 (29)

15 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton 638 and two unctons related to the yeldng o bars 4 and 5: (,,, ) ( ) g A V H = A av+ bh, wth = 4,5 (30) y y where E s the Young s modulus, I s the moment o nerta o the U-shaped steel secton, L s bar length, A s the transversal secton area, y s the steel yeldng stress, V s the vertcal load, H s the horzontal load, a s the racton o the vertcal load actng at each bar, b s the racton o the horzontal load actng at each bar. The random varables are descrbed n Table 4. A correlaton o 0. between V and H s consdered. Fgure 8: Hper-statc truss studed n Example 2. Bar U Shape A (cm 2 ) I (cm 4 ) U 50x25x U 50x25x U 50x25x U 50x25x U 50x25x U 75x40x Table 3: Geometrcal propertes o the truss bars o Example 2 (Verzenhass, 2008). Random Varable Dstrbuton mean c.o.v E (MN/cm²) Lognormal y (kn/cm²) Lognormal V (kn) Lognormal H (kn) Lognormal Table 4: Random varables o Example 2. Latn Amercan Journal o Solds and Structures 2 (205)

16 639 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton The reerence probablty o alure s obtaned rom a Crude Monte Carlo smulaton wth samples usng Smple Samplng: P = (β = 3.686). Fgures 9, 0 and show the convergence plots or the mean and coecent o varaton (c.o.v) o the P, or Smple Samplng, LHS and Asymptotc Samplng, respectvely. Convergence results were evaluated or number o samples varyng rom 0 3 to 0 5. Results are shown or Crude, Asymptotc and Enhanced Samplng. For Asymptotc Samplng, the parameter vares rom 0.5 to 0.8, wth 5 support ponts. Importance Samplng s not ncluded because the samplng uncton s composed ollowng equaton (2). Hence, the convergence plot could not be obtaned. Results or Importance Samplng and or Subset smulaton are shown n Table 5, or a xed (and smaller) number o samples. One can notce n Fgures 9, 0 and that Enhanced Samplng perorms very well, comparng to Crude Monte Carlo and to Asymptotc Samplng, w.r.t. convergence o the P and o ts coecent o varaton. The use o LHS (Fgure 0) s advantageous or the studed technques, snce aster and smoother convergence s observed. a) Mean b) Coecent o varaton. Fgure 9: Convergence o P (a) and ts c.o.v. (b) usng Smple Samplng n Example 2. a) Mean b) Coecent o varaton. Fgure 0: Convergence o P (a) and ts c.o.v. (b) usng Latn Hypercube Samplng n Example 2. Latn Amercan Journal o Solds and Structures 2 (205)

17 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton 640 a) Mean b) Coecent o varaton. Fgure : Convergence o P (a) and ts c.o.v. (b) usng Antthetc Samplng n Example 2. The comparson o all ntellgent samplng technques s presented n Table 5, computed or a smaller number o samples: 3,700. For Subset Smulaton, the ollowng parameters are adopted: n SS =,000; α =.5; and P 0 = 0.. Smple Samplng Estmaton Technque P c.o.v. Devaton (%) Crude Monte Carlo Importance Samplng Asymptotc Samplng Enhanced Samplng Latn Hypercube Samplng Subset Smulaton Crude Monte Carlo Importance Samplng Asymptotc Samplng Enhanced Samplng Subset Smulaton Crude Monte Carlo Antthetc Varates Samplng Importance Samplng Asymptotc Samplng Enhanced Samplng Subset Smulaton Table 5. Comparson o results or 3,700 samples n Example 2. In Table 5, one observes that Crude Monte Carlo does not lead to any results or such a small number o samples. Importance Samplng presents a very good perormance n comparson wth the Latn Amercan Journal o Solds and Structures 2 (205)

18 64 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton other technques, wth very small c.o.v.s and small and acceptable devatons rom the reerence P. Subset Smulaton also presents an acceptable perormance, wth small c.o.v.s and larger, but stll acceptable devatons rom the reerence result. The use o Latn Hypercube Samplng s benecal or all technques, or ths problem. 5.3 Example 3: non-lnear steel rame tower An optmzed plane steel rame transmsson lne tower (Fgure 2) s analyzed by nte elements. The problem s based on Gomes and Beck (203), where structural optmzaton consderng expected consequences o alure was addressed. The mechancal problem s modeled by beam elements wth three nodes, wth three degrees o reedom per node. The rame s composed by L- shaped steel beams. The Fnte Element Method wth postonal ormulaton (Coda and Greco, 2004; Greco et al., 2006) s adopted to solve the geometrcal non-lnear problem. Moreover, the materal s assumed elastc perectly plastc. Fgure 2: FE model o the power lne tower addressed n Example 3. The lmt state uncton s dened based on the load-dsplacement curve (Fgure 3) or top nodes and 2. Because several conguratons had to be tested n the optmzaton analyss perormed n Gomes and Beck (203), a robust lmt state uncton was mplemented. The same lmt state uncton s employed heren: g( xd, ) = 75 tan ( δ L), where δ s the ncrement n mean dsplacement n centmeters, or nodes and 2; L s the ncrement n the nondmensonal load actor; and 75 o s the crtcal angle consdered. Latn Amercan Journal o Solds and Structures 2 (205)

19 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton 642 Fgure 3: Load actor mean dsplacement dagram or top nodes and Loadng The gravty load rom algned cables s consdered. The desgn wnd load on cables and structure s calculated based on Brazlan code ABNT NBR 623: Wnd loads n buldngs. The desgn wnd s taken asv 0 = 45m/s. Thus, the characterstc wnd load s calculated as vk = SSSv 2 3 0, where S, S 2 and S 3 are the topographcal ( S = ), the rugosty ( S 2 =.06) and statstcal ( S 3 = ) actors, yeldng v k = 47.7 m/s. Ths characterstc wnd s taken at a heght o 0 meters. The varaton o wnd velocty wth heght ollows a parabolc shape, such that v( z) = 0. z vk, where z 2 s heght. To model uncertanty n the wnd load, an non-dmensonal random varable V s ntroduced, such that V( z) = v( z) V. Thus, the wnd pressure s evaluated rom wnd veloctes as q( zv, ) = 0.63 ( v( z) V) 2, where pressure s gven n N/m 2, or normal atmospherc condtons ( atm) and temperature (5 C). The wnd load acts at each element on the tower. The drag coecent s C a = 2., whch s the maxmum value or prsmatc beams wth L-shaped sectons. Cables o 2.52 cm dameter were adopted, wth an nluence area o 300 m and drag coecent o C a =.2. For these values, one obtans the random orce Fa( V ) actng n the horzontal drecton on nodes 2 and 2, such that Fa( V) 9, V. Random varables consdered n ths problem are the Young s modulus (E ) and the nondmensonal wnd varable (V ). The random varables are descrbed n Table 6. Random Varable Dstrbuton Mean c.o.v V Gumbel E (GPa) Lognormal Table 6: Random varables o Example 3 (Gomes and Beck, 203) Results In Gomes and Beck (203), probablty o alure s evaluated by FORM, and values, FORM P = (β = ) are obtaned or the tower conguraton n Fgure 2. In ths paper, sever- Latn Amercan Journal o Solds and Structures 2 (205)

20 643 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton al Monte Carlo Smulaton technques are adopted or evaluatng ths alure probablty. FORM s very ecent, and ecency s undamental or solvng the structural optmzaton problem. However, FORM provdes only approxmate results or problems wth non-lnear lmt state unctons. The reerence probablty o alure s evaluated by Crude and by Importance Samplng Monte Carlo smulaton, usng Smple Samplng, Latn Hypercube Samplng and Antthetc Varates Samplng. 0 5 samples are employed or each soluton. Results are gven n Table 7, where one observes that Crude Monte Carlo wth LHS and Importance Samplng Monte Carlo present probabltes o alure very close to the result obtaned by FORM. Ths shows that, n spte o the nonlnearty o the lmt state uncton, FORM provdes accurate results. For the remander o the analyss, the average value o P = s used as a reerence. Crude Monte Carlo Importance Samplng Monte Carlo Estmaton Technque P c.o.v. Processng Tme Devaton rom average (%) Smple Samplng h e 2 mn Latn Hypercube Samplng h e 7 mn 0.36 Antthetc Varates Samplng h e mn Smple Samplng h e 36 mn 0.27 Latn Hypercube Samplng h e 4 mn 0.64 Antthetc Varates Samplng h e 43 mn 0.00 Table 7: Comparson o Crude Monte Carlo and Importance Samplng, usng 0 5 samples. Fgures 4, 5 and 6 show the convergence plots or the mean and coecent o varaton (c.o.v) o the P, or Smple Samplng, LHS and Asymptotc Samplng, respectvely. Convergence results were evaluated or number o samples varyng rom 0 3 to 0 5. Results are shown or Crude, Importance, Asymptotc and Enhanced Samplng. In Asymptotc Samplng, parameter vares rom 0.5 to 0.8, wth 5 support ponts. a) Mean b) Coecent o varaton. Fgure 4: Convergence o P (a) and ts c.o.v. (b) usng Smple Samplng n Example 3. Latn Amercan Journal o Solds and Structures 2 (205)

21 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton 644 a) Mean b) Coecent o varaton. Fgure 5: Convergence o P (a) and ts c.o.v. (b) usng Latn Hypercube Samplng n Example 3. a) Mean b) Coecent o varaton. Fgure 6: Convergence o P (a) and ts c.o.v. (b) usng Antthetc Samplng n Example 3. In Fgs. 4 to 6, one observes that convergence o Importance Samplng, or ths problem, s very ast and accurate, or all basc samplng technques. Also, t can be clearly seen that LHS mproves the results or all samplng technques, reducng oscllatons durng convergence. The convergence behavor o Asymptotc Samplng s very unstable, both or the c.o.v. and the P ; hence ths technque does not perorm very well or ths problem. Enhanced Samplng works much better, provdng results smlar to Crude samplng or ths number o samples. A quanttatve comparson o the perormance o all samplng technques, n soluton o Problem 3, s presented n Table 8. These results are computed or 4,800 samples. In Subset Smulaton the ollowng parameters are adopted: n SS = 4000, α =.5, P 0 = 0.. One observes n Table 8 that Importance Samplng and Subset smulaton out-perorm the other methods n terms o small c.o.v. and small devaton rom the reerence soluton. Asymptotc and Enhanced samplng show a smlar and average perormance. Latn Hpercube Samplng mproves the results or most methods, especally or Subset Smulaton Processng Tme The processng tme to compute the results n Table 8 (4,800 samples) are gven n Table 9. One observes that processng tmes are smlar or all technques, except or Importance Samplng: ths Latn Amercan Journal o Solds and Structures 2 (205)

22 645 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton technque takes a lttle longer to compute the weghts or each sample, ollowng equaton (). Processng tmes are very smlar or the basc samplng technques. By comparng results o Table 9 wth results o Table 7, one observes the massve gan n processng tme that s obtaned usng the ntellgent samplng technques addressed heren. Computng tmes are much smaller, but the qualty o the solutons (small devaton and c.o.v.) are smlar. Estmaton Technque P c.o.v. Devaton (%) Smple Samplng Crude Monte Carlo Importance Samplng Asymptotc Samplng Enhanced Samplng Latn Hypercube Samplng Subset Smulaton Crude Monte Carlo Importance Samplng Asymptotc Samplng Enhanced Samplng Subset Smulaton Crude Monte Carlo Antthetc Varates Samplng Importance Samplng Asymptotc Samplng Enhanced Samplng Subset Smulaton Table 8: Comparson o results or 4,800 samples n Example 3. Smple LHS Anthtetc Crude Monte Carlo h and 37 mn h and 3 mn h and 38 mn Importance Monte Carlo 2 h 2 h and 2 mn 2 h and 2 mn Asymptotc Samplng h and 6 mn h and 5 mn h and 4 mn Enhanced Samplng h and 27 mn h and 30 mn h and 30 mn Subset Smulaton h and 52 mn h and 50 mn h and 5 mn Table 9: Processng tme or 4,800 samples n Example Processng Tme In order to urther evaluate the perormance o Subset Smulaton usng smaller numbers o samples, the ollowng parameters are adopted: n SS = 2,000; α = 2; and P 0 = 0.. Results are presented n Table 0. Latn Amercan Journal o Solds and Structures 2 (205)

23 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton 646 Technque Number o Samples P c.o.v. Devaton (%) Processng tme (mn) Smple Samplng 9, LHS 7, Asymptotc Samplng 7, Table 0: Comparson o results usng Subset Smulaton or Example 3. Results presented n Table 0 show that Subset Smulaton s an eectve tool or estmatng small probabltes o alure n problems nvolvng numercal evaluaton o lmt state unctons. The orgnal soluton va Crude Monte Carlo took approxmately hours to compute a probablty o alure wth coecent o varaton o 0.27 and relatve devatons o: 28.68% (Smple Samplng),.% (Latn Hypercube Samplng) and 37.87% (Antthetc Varates Samplng). In comparson, Subset Smulaton took approxmately 45 mnutes to estmate the same probablty o alure wth coecent o varaton o 0.25 and devatons around 3.7%. 5 CONCLUDING REMARKS Ths paper presented a benchmark study on ntellgent and ecent samplng technques n Monte Carlo Smulaton. Crude Monte Carlo smulaton was compared to Importance Samplng, Asymptotc Samplng, Enhanced Samplng and to Subset Smulaton. These ve samplng schemes were combned wth Smple Samplng, Latn Hypercube Samplng and Antthetc Varates Samplng, resultng n teen samplng strategy combnatons. The perormance o these strateges was nvestgated or sx problems, but results or only three were presented heren. The conclusons below relect the sx problems studed. It was observed that use o Latn Hypercube Samplng had a sgncant and postve nluence or all samplng technques wth whch t was combned. LHS has led to smoother convergence curves and more accurate results or most cases studed. Antthetc Varates also produced better results than Smple Samplng. Importance Samplng usng desgn ponts was ound to be one o the most ecent technques or solvng problems wth small and very small alure probabltes. Convergence s extremely ast, and wth a couple o samples one obtans accurate alure probabltes wth small samplng error (c.o.v.). However, or one o the problems studed, t was observed that the samplng uncton ntroduced some bas n the results, makng alure probabltes converge to an nexact yet acceptable value. Also, t s known that use o Importance Samplng can be a problem or hghly non-lnear problems or whch the desgn pont(s) cannot be ound. Results obtaned wth Asymptotc Samplng were naccurate or a number o problems, perhaps because a small number o support ponts were used. The drawback wth ths scheme s that a new set o samples has to be computed or each addtonal support pont; hence there s a strng compromse between the number o support ponts and the accuracy wth whch alure probabltes can be estmated or each support pont. Although smlar to Asymptotc Samplng, the Enhanced Samplng technque presented consstent results or all problems studed. Enhanced samplng has a large advantage over Asymptotc Latn Amercan Journal o Solds and Structures 2 (205)

24 647 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton Samplng, because the same set o samples s used or all support ponts. Hence, there s no computatonal penalty or usng many support ponts, and the accuracy o the regresson s mproved. Subset Smulaton perormed extremely well or all problems studed, resultng n accurate estmates o alure probabltes, wth very small samplng errors. Ths explans why Subset Smulaton has become so popular among structural relablty researchers, and s beng appled extensvely n the soluton o both tme nvarant and tme varant relablty problems. Fnally, t can be sad that Subset Smulaton, Enhanced Samplng and Importance Samplng, aded by Latn Hypercube samplng, are ecent ways o solvng relablty problems, wth a much smaller number o samples than requred n Crude Monte Carlo Smulaton. Acknowledgments The authors wsh to thank the Natonal Councl or Research and Development (CNPq) or the support to ths research. Reerences ABNT NBR623 (998). Wnd loads n buldngs. ABNT - Brazlan Assocaton o Techncal Codes, Ro de Janero (n Portuguese). Au, S.K., (2005). Relablty-based desgn senstvty by ecent smulaton. Comput Struct 83: Au, S.K., Beck, J.L., (200). Estmaton o small alure probabltes n hgh dmensons by subset smulaton. Prob. Eng. Mech. 6(4): Au, S.K., Chng, J., Beck, J.L., (2007). Applcaton o subset smulaton methods to relablty benchmark problems. Struct Saety 29: Beck, A.T., Rosa, E., (2006). Structural Relablty Analyss Usng Determnstc Fnte Element Programs. Latn Amercan Journal o Solds and Structures 3: Bucher, C., (2009). Asymptotc samplng or hgh-dmensonal relablty analyss. Prob. Eng. Mech. 24: Coda, H.B., Greco, M.A., (2004). smple FEM ormulaton or large delecton 2D rame analyss based on poston descrpton. Comput Method Appl Mech Eng 93: Engelund, S., Rackwtz, R., (993). A benchmark study on mportance samplng technques n structural relablty. Structural Saety 2: Gomes, W.J.S., Beck, A.T., (203). Global structural optmzaton consderng expected consequences o alure and usng ANN surrogates. Comput Struct 26: Greco, M., Gesualdo, F.A.R., Venturn, W.S., Coda, H.B., (2006). Nonlnear postonal ormulaton or space truss analyss. Fnte Elem Anal Des 42: Madsen, H.O., Krenk, S., Lnd, N.C., (986). Methods o structural saety. Prentce Hall, Englewood Cls. McKay, M.D., Beckman, R.J., Conover, W.J., (979). A comparson o three methods or selectng values o nput varables n the analyss o output rom a computer code. Technometrcs 2(2): Melchers, R.E., (999). Structural Relablty Analyss and Predcton. John Wley & Sons, Chchester. Melchers, R.E., Ahamed, M.A., (2004). Fast approxmate method or parameter senstvty estmaton n Monte Carlo structural relablty. Comput Struct 82: Metropols, N. et al., (953). Equaton o state calculaton by ast computng machnes. J Chem Phys 2(6): Metropols, N., Ulam, S., (949). The Monte Carlo method. J Am Statst Assoc 44(247): Latn Amercan Journal o Solds and Structures 2 (205)

25 K.R.M dos Santos and A.T. Beck / A benchmark study on ntellgent samplng technques n Monte Carlo smulaton 648 Naess, A., Lera, B.J., Batsevych, O., (2009). System relablty analyss by enhanced Monte Carlo smulaton. Struct Saety 3: Naess, A., Lera, B.J., Batsevych, O., (202). Relablty analyss o large structural systems. Prob. Eng. Mechancs 28: Olsson, A., Sandeberg, G., Dahlblom, O., (2003). On latn hypercube samplng or structural relablty analyss. Struct Saety 25: Robert, C., Casella, G., (20). A short hstory o Markov Chan Monte Carlo: Subjectve recollectons rom ncomplete data. Statst sc 26(): Schuëller, G.I., Prandlwarter, H.J., (2007). Benchmark study on relablty estmaton n hgher dmensons o structural systems An overvew. Struct Saety 29: Schan, M.T., Nelsen, S.R.K., Bucher, C., (20a). Applcatons o asymptotc samplng on hgh dmensonal structural dynamc problems. Struct saety 33: Schan, M.T., Nelsen, S.R.K., Bucher, C., (20b). Ecent estmaton o rst passage probablty o hghdmensonal nonlnear systems. Prob. Eng. Mech. 26: Schan, M.T., Nelsen, S.R.K., Bucher, C., (204). Closure to Applcatons o asymptotc samplng on hgh dmensonal structural dynamc problems. Struct Saety 46: -2. Verzenhass, C.C., (2008). Otmzação de rsco estrutural baseada em conabldade. Masters Dssertaton, São Carlos School o Engneerng, Unversty o São Paulo. Latn Amercan Journal o Solds and Structures 2 (205)