Evaluation of Accuracy and Efficiency of some Simulation and Sampling Methods in Structural Reliability Analysis

Transcription

1 Waye State Uiversity Civil ad Evirometal Egieerig Faculty Research Publicatios Civil ad Evirometal Egieerig Evaluatio of Accuracy ad Efficiecy of some Simulatio ad Samplig Methods i Structural Reliability Aalysis Christopher D. Eamo Mississippi State Uiversity, Starkville, MS, christopher.eamo@waye.edu Michel D. Thompso Mississippi State Uiversity, Starkville, MS Zheyu Liu Mississippi State Uiversity, Starkville, MS Recommeded Citatio Eamo, C. D., Thompso, M., ad Liu, Z. (2005). "Evaluatio of accuracy ad efficiecy of some simulatio ad samplig methods i structural reliability aalysis." Structural Safety, 27(4), , doi: /j.strusafe Available at: This Article is brought to you for free ad ope access by the Civil ad Evirometal Egieerig at DigitalCommos@WayeState. It has bee accepted for iclusio i Civil ad Evirometal Egieerig Faculty Research Publicatios by a authorized admiistrator of DigitalCommos@WayeState.

2 Evaluatio of Accuracy ad Efficiecy of some Simulatio ad Samplig Methods i Structural Reliability Aalysis Christopher D. Eamo 1, Michel Thompso 2, ad Zheyu Liu 2 1 Assistat Professor of Civil Egieerig 2 Graduate Studet, Departmet of Civil Egieerig Mississippi State Uiversity, MS, Abstract Numerous simulatio ad samplig methods ca be used to estimate reliability idex or failure probability. Some poit samplig methods require oly a fractio of the computatioal effort of direct simulatio methods. For may of these methods, however, it is ot clear what trade-offs i terms of accuracy, precisio, ad computatioal effort ca be expected, or for which types of fuctios they are most suited. This study uses ie procedures to estimate failure probability ad reliability idex of approximately 200 limit state fuctios with characteristics commo i structural reliability problems. The effects of fuctio liearity, type of radom variable distributio, variace, umber of radom variables, ad target reliability idex are ivestigated. It was foud that some methods have the potetial to save tremedous computatioal effort for certai types of limit state fuctios. Recommedatios are made regardig the suitability of particular methods to evaluate particular types of problems. Itroductio For may practical problems i structural reliability, a aalytical determiatio of failure probability is uattaiable. This is especially so whe complex structures are modeled with umerical techiques such as the fiite elemet method, where geeratio of a explicit 1

3 expressio of load or resistace, ad thus limit state fuctio, is typically ot possible. Although a direct Mote Carlo simulatio is a valid approach for such cases, it is ofte practically ifeasible to coduct due to the large umber of simulatios required for sufficiet accuracy. To address this problem, umerous simulatio ad samplig methods were developed by various researchers that ca aid the determiatio of failure probability or reliability idex. Curretly, there exists a large umber of competig schemes to estimate statistical parameters from a sigificatly reduced umber of samples. A review of may of these methods ca be foud i [1]. Ufortuately, whe preseted i the literature, the accuracy ad efficiecy of these schemes are typically evaluated for oly a small umber of simple example cases ad without sigificat compariso to other methods [1-12]. For may schemes, the relatioship betwee computatioal effort ad accuracy is ot apparet. Nor is it clear which schemes are most suitable for a particular type of problem, i terms of umber of radom variables, degree of oliearity, radom variable distributios ad variaces, ad so o. The lack of a readily-available evaluatio ad compariso amog competig methods presets a obstacle to proper method selectio, particularly for those without extesive experiece usig these methods. This paper attempts to fill this gap by examiig a umber of simulatio ad samplig methods that may be used to estimate failure probability (P f ) or reliability idex (β) i structural reliability aalysis. I this study, a subset of methods with specific typical formulatios is cosidered. Of particular iterest i this study are poit samplig methods, which promise to provide large reductios i computatioal effort. These methods require a fixed umber of simulatios based o the umber of radom variables i the problem. Three poit samplig methods are cosidered: Roseblueth s 2+1 poit estimatio method [6] 2

4 (ROS) ad Zhou s +1 (N+1) ad 2 (2N) poit itegratio methods [7]. These methods are characterized by havig a fixed umber of pre-determied samples ad are used to calculate β. Five simulatio methods are also cosidered: traditioal Mote Carlo Simulatio (MCS), Lati Hypercube (LH) stratificatio [8], Importace Samplig [9] (IS), Plae-based Adaptive Importace Samplig [13] (AISP), ad Curvature-based Adaptive Importace Samplig (AISC) [10]. These methods have a ope-eded umber of samples, ad are used to calculate P f directly. The Rackwitz-Feissler procedure [5] (RF), a well-kow first order aalytical method suitable for o-ormal distributios, is cosidered as well. By evaluatig 198 differet fuctios, these seve methods are compared i terms of accuracy, precisio, ad computatioal effort for parameters that iclude liearity of the limit state fuctio, umber of radom variables (RVs), type of RV distributio, variace, ad target reliability idex. As the results preseted here are empirical, they caot be used to make uiversal coclusios for a method s effectiveess for all possible fuctios. However, the results may serve to aid the process of iitially selectig a simulatio or samplig method appropriate for a particular type of problem, give specific accuracy, precisio, or computatioal effort costraits. Methods Cosidered Thorough discussios of MCS are preseted elsewhere [4, 15-17]. With MCS, as the accuracy of the estimate of P f depeds o the umber of simulatios, a desired degree of precisio must be chose. I this study, the umber of MCS samples eeded is estimated by [18]: COV ( 1 f = (1) N P P ) f 3

5 Where P f is the failure probability of the limit state fuctio to be estimated ad N the umber of samples. To keep the umber of simulatios reasoable, N was chose such that the expected COV of the result is 5%. There are several stratificatio techiques available [8, 16, 19]. I geeral, the Lati Hypercube (LH) techique ivolves partitioig the probability desity fuctio of each RV ito vertical strata. For each simulatio, a radom strata is sampled for each RV, with the coditio that every strata is sampled oly oce i the etire procedure. I this study, strata are formed from equal probability weights. To reduce the possibility of spurious correlatios occurrig, a correlatio reductio techique as described by Ima ad Coover [19] is employed. The umber of strata (samples) was chose such that the accuracy results were close to those obtaied from MCS. This umber varies depedig o fuctio failure probability. Specific quatitative results are discussed later. The Rackwitz-Feissler Procedure [5] is a commoly used first-order iterative method that calculates β rather tha P f. It accouts for o-ormal distributios by covertig these to equivalet ormal distributios at the desig poit o the failure boudary. It provides excellet results for liear problems, but has two potetial sources of iaccuracies for oliear limit states. First, as it computes β, the shortest distace from the failure regio to the origi of reduced coordiates, the same reliability idex may be achieved for failure regios of differet (hyper)volumes ad thus failure probabilities. The coversio of reliability idex to failure probability (P f = Φ(-β)) is therefore ot always accurate. A secod source of error may occur if the failure (hyper)surface is ot cosistetly covex or cocave, but has multiple local 4

6 miimums. Sice the iteratio is begu at a sigle trial desig poit (typically at the mea values), the search algorithm will stop whe it fids the first local miimum. This may or may ot be the global miimum. Note that RF first requires oe evaluatio of the limit state fuctio (g) to isure that the desig poit is o the failure boudary (this will require more tha 1 evaluatio if at least 1 RV is ot liear with respect to g), the a small umber of iteratios as the search algorithm attempts to fid the miimum distace from the failure surface to the origi of reduced coordiates. Each iteratio requires the computatio of oe derivative per RV i the problem (the partial derivative of g with respect to the reduced RV), as well as aother evaluatio of g to place the updated desig poit o the failure boudary oce agai. If the limit state is ot explicitly give, such as i the case where a fiite elemet procedure is used, RF must be evaluated umerically. Derivatives ca be evaluated with a fiite differece approach, which would require at miimum oe evaluatio of g (if a forward or backwards differece scheme is used), or call to the fiite elemet code, per RV. I this case, the total umber of samples required for RF is a miimum of 1 + (+1)i, where is the umber of RVs ad i is the umber of iteratios required for β covergece. This does ot iclude the special case where if all RVs are oliear with respect to g. Here, the uit values i the above expressio must be replaced with a ukow umber of iteratios, based o the specific problem ad oliear solver used, to allow the desig poit to coverge o the failure boudary. The umber of iteratios i eeded for covergece varies, but is ofte small. For this study, three iteratios were sufficiet for most cases. Thus, 4+3 samples would be required to evaluate the limit states cosidered. 5

7 The fudametal approach of importace samplig (IS) is to shift the probability desities of the RVs close to the desig poit, such that the modified limit state will produce a high rate of failures. These failures ca the be captured with a much-reduced umber of samples, ad a failure probability ca be calculated. The fial failure probability is the adjusted to accout for the shifted distributios. Two key eeds of IS are 1) to idetify the desig poit (the poit of maximum likelihood), or a poit close to it, ad 2) to choose a appropriate importace samplig fuctio, which is used to adjust the fial calculated failure probability. There are may ways proposed i the literature to satisfy 1) ad 2) [3, 9, 20-26]. For this study, the desig poit is foud with the RF method, while the importace samplig fuctio is take as the joit probability desity fuctio of the origial limit state, with RV meas shifted to the desig poit. The failure probability is evaluated with 50 MCS rus. This procedure was foud to give close results to the crude MCS method for most fuctios studied. Quatitative results are discussed later. Allowig for three RF iteratios, this procedure required [(4+3) + 50] total samples. Sice RF is used to idetify the desig poit for use i IS, this procedure is subject to the same errors as RF. Adaptive Importace Samplig (AIS) schemes adjust the samplig regio as the aalysis progresses, refiig the samplig space to maximize samplig i the failure regio. For highly o-liear or o-ormal limit states, depedig of the shape of the failure surface, samplig uiformly aroud the MPP, as with IS, may ot produce a high failure rate ad thus may give a poor estimate of failure probability. For AIS, after the MPP is idetified, a small umber of iitial samples are take. If few samples will fall withi the failure regio, the samplig boudary is adjusted, ad a ew set of samples is take. The process repeats util a satisfactory 6

8 umber of failures is achieved. As with IS, the computed failure probability is the adjusted to accout for the shifted samplig regio. May adaptive methods exist [2, 9, 10, 13, 14, 27-30]. I this study, two AIS methods are cosidered, as described by Wu [10, 13,14]. I oe method, the limitig surface is parabolic, ad is refied by adjustig curvatures at the MPP (AISC), while the other method uses a plaar surface (AISP), ad is refied by shiftig the plae closer or further away from the MPP. The AIS process cosidered here is as follows. After the MPP is foud (such as from RF), the limit state is liearized at that poit the coverted to a parabolic surface as described by Tvedt [31]. Failure probability is the estimated usig a secod-order method [10]. Te iitial MCS samples are the take by evaluatig the actual limit state fuctio at the estimated MPP withi the cofies of the estimated parabolic failure surface. If samples fall outside of the true failure regio, for AISC, the curvatures of the limitig surface are reduced to further eclose the failure regio (or distace of the plaar surface i the case of AISP), such that the chage i estimated failure probability is approximately 10%. Here additioal samples are take oly withi the chaged regio. The failure probability estimate is the updated. This process of adjustig limitig surface curvatures ad resamplig i the icremeted regio is repeated util the failure probability estimates coverge. Allowig three icremets ad te samples per icremet for AISC ad 5 icremets ad 15 samples for AISP (AISP typically requires more samples for equivalet results), the scheme requires (-1)/2 samples for AISC ad 79+3 samples for AISP. As with IS, sice the AIS methods deped o a iitial locatio of the MPP, the method used to fid the MPP may ifluece the quality of the fial results. 7

9 May poit samplig methods exist [6-8, 32]. The focus of this study is o those methods that specify a small umber of simulatios, which realize the least computatioal costs. The N+1 ad 2N poit itegratio methods are based o Gauss quadrature. The derivatio of the methods is give elsewhere [7]. Poit values are give as a fuctio of the umber of radom variables i the limit state, the positio of a RV i the cosidered set, ad the simulatio ru umber. N+1 poit values are give i Table 1. I the table, Z ij refers to the simulatio ru umber i ad radom variable j, while refers to the umber of radom variables i the problem. Here i is from 1 to +1 while j is from 1 to. For 2N, each sample is coducted by settig all RVs equal to 0 (i.e. the mea value i stadard ormal space) except oe, which is set to 1/2. To coduct the simulatio, the RV that takes the value of 1/2 is alterated util all RVs are cosidered, to geerate samples. additioal samples are geerated by repeatig the above procedure but by usig - 1/2 as the value for the o-zero RV. Applicatio of the poit itegratio methods geerates stadard ormal space values for each of the system radom variables. These values are the trasformed to basic variable space values for iput to the limit state fuctio usig stadard trasformatios, just as with MCS or LH. From the total sample of limit state evaluatios, the mea value ad stadard deviatio of the fuctio are calculated. β is the estimated by dividig mea value by stadard deviatio. Zhou [7] provides o evaluatio as to the expected effectiveess or limitatios of the methods, or could this be foud i the available literature. However, based o the small umber of samples, it is expected that accuracy will be degraded for oliear fuctios. The derivatio ad use of Roseblueth s 2+1 poit estimate is substatially differet from N+1 ad 2N [6]. Although it is referred to as 2+1, if β is to be estimated, oly 2 samples are 8

10 eeded (a sigle additioal sample is required if the fuctio s mea value is eeded). It is used by first evaluatig the limit state fuctio with oe of the RVs shifted upward i value by oe stadard deviatio (y + i ), while the remaiig RVs i the fuctio are kept at their mea values (y). This is repeated util the fuctio is evaluated times, where each time a differet RV has its value shifted. This process is the repeated times agai, but ow with the value of each RV shifted dowward by oe stadard deviatio (y - i ). The COV of the fuctio ca the be calculated as [6, 32]: Where 2 ( 1+ ) 1 COV Y = V yi (2) i= 1 y y V yi = (3) y + y + i + i i i Reliability idex ca the be estimated by takig the iverse of the fuctio s COV. The method reportedly works best for fuctios that are liear with low skew ad low variatio. As either oliearities or distributios are take ito accout, reliability idex computed from this method may ot provide a accurate idicatio of failure probability for o-ormal ad oliear fuctios. Limit State Fuctios Cosidered Each of the methods discussed above will be used to evaluate the failure probability ad reliability idex of a geeral limit state fuctio uder the ifluece of a variety of parameters. The parametric cases cosidered are give i Table 2. I the table, mixed meas that the RVs are give differet distributios or coefficiets of variatio (COV). For the o-mixed cases, RVs are give idetical distributios or COVs. Each of the 22 cases i Table 2 is repeated three 9

11 times, varyig the umber of elemets i g that are RVs, to create a total of 66 subcases. I tur, each of these subcases is repeated three times, with a differet target reliability idex, for a total of 198 differet limit state fuctios. A summary of the parameters cosidered is as follows: Liearity: ½ of the fuctios are liear, ½ are oliear. Distributios: Approximately ¼ have all ormal RVs, ¼ are all logormal, ¼ are all extreme I, ad ¼ have mixed distributio types. Variace: Approximately 1 / 3 have all RVs at 5% COV, 1 / 3 have all RVs at 35% COV, ad 1 / 3 have differet RV COVs (mixed amog 5, 15, ad 35%). Oe exceptio is the extreme I distributios, i which o mixed COV cases were cosidered. Radom Variables: 1 / 3 have 2 RVs, 1 / 3 have 5 RVs, ad 1 / 3 have 15 RVs. Target Reliability Idex: 1 / 3 have a low β, 1 / 3 have moderate β, ad 1 / 3 have high β. The low group has a average β of 1.3 with rage of 0.3 to 2. The moderate group has a average β of 3.5 with rage of 2 to 5. The high group has a average β of 10 with rage of 5 to 15. Specific parameter values were chose such that the mea reliability idex is cosistet amog differet parametric compariso groups. This is importat because a fuctio s failure probability iflueces the accuracy at which it ca be estimated. For example, the ormal ad o-ormal distributio groups have the same average β, as do fuctios with 2, 5, ad 15 RVs. Two exceptios to this rule are the COV groups (the low COV fuctios have, o average, a higher β tha the higher COV fuctios), ad liearity (the liear fuctios have a higher average β tha the oliear fuctios). Whe makig data comparisos for β, COV, ad 10

12 liearity parameters, the bias that target reliability idex brigs to the results is accouted for ad is discussed below i the results sectio. To assess the effectiveess of the methods, a geeral fuctio g was developed i which parameters such as umber of radom variables, target reliability idex, ad liearity could be adjusted i a cosistet way. The limit state fuctios have the geeral form: g = k k di c i= 1 j= 1 L w 4 j E j I j j (4) Although g represets specific umerical problems for this study, fuctio g was chose such that its form is mathematically similar to may typical limit states ecoutered i structural reliability; those cotaiig sums of products ad quotiets of RVs. Depedig o the choice of RVs, three forms of g (with respect to liearity) were cosidered: liear, oliear with positive expoets, ad oliear with positive ad egative expoets (i.e. iverse) problems. The RVs, distributios, ad COVs for each RV subcase (2, 5, 15) are give i Table 3. Some reliability problems may ivolve hudreds or thousads of RVs, but may practical problems i structural reliability, particularly those used for code calibratio, ad eve some system reliability problems, ivolve a doze or less RVs. Solvig reliability problems with a large umber of RVs geerally requires tremedous computatioal effort ad are beyod the scope of this work. However, some importat treds ca be see based o the umber of RVs cosidered here. For the mixed cases i Table 3, distributio type ad COV values chose for the RVs were iflueced based o the problem that g represets, as discussed i the ext paragraph. 11

13 As specific formulatios of g must be chose for evaluatio, costats as well as RV values were defied such that g represets oe (for 2 RV problems), two (for 5 RV problems), or five (for 15 RV problems) simple, uiformly-loaded beams i parallel with multiple load effects (such as dead load ad live load) applied, ad failure defied as exceedig a o-determiistic midspa deflectio limit. For all problems, c = 5/384, L = 6.1 (m), E = 2x10 8 (kpa), ad I = 6.452x10-4 (m 4 ), ad w = (N/m), w DL = (N/m) ad w LL = (N/m). These values may be either costats or the mea values of RVs, depedig o how the elemet is used i the specific problem. Values for d ad k are problem-specific ad chose such that three differet target reliability idices (withi the rages of low, medium, ad high, quatified above) could be examied for each subcase. These values are give i tables 4 ad 5. Clearly, as COV or distributio chages, the rage of possible β values also chages, ad β is particularly limited for high COV or extreme I distributios. I each case, the itet was to examie the effectiveess of the seve cosidered procedures over a wide rage of failure probabilities. For the 2 RV problems, =k=1. For the 2 RV liear problems, d 1 ad w 1 are the RVs. For the 2 RV oliear problems, d 1 ad L 1 are the RVs. For the 5 RV liear problems, =2 ad k=3, while d i ad w i are take as RVs. For 5 RV oliear problems, =k=1 ad d 1, w 1, E 1, I 1, ad L 1 are RVs. For the 15 RV liear problems, =5, k=10, ad RVs are d i ad w i. For the 15 RV oliear problems, =k=3 ad RVs are d i, w i, E i, I i, ad L i. Substitutig these values ito equatio (4) results i the followig, where radom variables are i bold ad subscripted: The 2 RV Liear Case is: g 4 cl = kd 1 w 1 (5) EI The 5 RV Liear Case is: 12

14 g = k 4 cl EI ( d + d ) ( w + w + ) 1 2 DL1 LL1 wll2 (6) The 15 RV Liear Case is: g = k ( d + d + d + d + d ) cl (7) ( wdl1+ wll1+ wdl2+ wdl2+ wdl3+ wll3+ wdl4+ wll4+ wdl5+ wll5) EI The 2 RV Noliear Case is: g 4 wl1 = kd1 c (8) EI The 5 RV Noliear Case is: g w L = kd1 c (9) E1I1 The 15 RV Noliear Case is: w L w L w L g = k (10) ( d + + ) d2 d3 c E1I1 E2I2 E3I3 Special Problems Egelud ad Rackwitz [9] idetified six limit states that posed potetial problems for various importace samplig schemes. These limit states have bee icluded i this study for evaluatio as well. Oe limit state was a liear fuctio that cosidered various umbers of radom variables ad failure probabilities, the characteristics of which are fully covered i the liear limit states above. The remaiig five limit states are listed below. 13

15 1. Expoetial oliearity. This highly oliear limit state is give by: g 20 i= 1 [ Φ( u )] C = ± l i ± (11) Where u i are ormally distributed radom variables. Although Egelud ad Rackwitz cosidered various values for C, i this study the fuctio of highest curvature is cosidered, C=41.05, which gave the poorest reported results [9]. RVs are give meas of 1.0 ad stadard deviatios of Reliability idex is approximately Multiple reliability idices. This limit state is hyperbolic with two miimum ad oe maximum beta values. It is give by: g = x x (12) 1 2 Where the meas of x 1 ad x 2 are ad , while their stadard deviatios are: , Both are ormally distributed. Reliability idex is approximately Series System. The series system is give by: ( g, g g ) g = mi (13) Where: g = x 2 1 2, g = x + 2x g 2 = x + 2x 2 + 2x x + x 4 + x x + x 5 + x 5 5 5x 5x 6 5x

16 x 1 -x 5 are logormally distributed with meas of 60.0 ad stadard deviatios of 6.0, while x 5 ad x 6 have extreme type I distributios with meas of 20 ad 25 ad stadard deviatios of 6.0 ad 7.5, respectively. Reliability idex is approximately Parallel System. This limit state is give by: ( g, g, g g ) g = max (14) Where: g g g g = u = u 1 2 3, 1 = u = u u 2 u u 3 4 u 5 4 All g i are stadard ormal radom variables. Reliability idex is approximately Noisy Limit State. This limit state has a fluxuatig boudary ad is: x2+ 2x3+ x4 5x5 5x si(100 x i ) i= 1 g = x (15) All x i are logormal. x 1 -x 4 have meas of 120 ad stadard deviatios of 12; x 5 has mea of 50 ad stadard deviatio of 15; ad x 6 has mea of 40 ad stadard deviatio of 12. Reliability idex is approximately

17 Results Tables 6-8 ad figures 1-10 preset results i terms of β, while tables 9-11 ad figures give results i terms of P f. I figures 1-10, results are preseted for each method based o a subset of cases, to examie the effect of the various parameters cosidered (liearity, ormality, COV, umber of RVs, target β). Results are measured i two ways: accuracy ad precisio. Accuracy refers to closeess to the exact value, ad is measured by mea value. This is calculated by first ormalizig the β computed by a particular method to the exact value (calculated β / exact β) for each limit state i the subset, the the mea value of this fractio is take for the subset. Precisio refers to the degree of cosistecy of the results. This is measured by coefficiet of variatio of the mea (COV). Note that a exact solutio would have a (ormalized) mea of 1.0 ad a COV of 0. For figures 11-20, the same procedure is used to report results i terms of P f. Overall i the figures refers to the results of all of the limit states (198) evaluated by a particular method. The tables preset data specifically for each of the 66 subcases ad results are calculated for the subset of 3 limit states composig each subcase (i.e. the mea ad COV are computed from the results cosistig of fuctios with the same umber of radom variables, distributio types, ad COV, but differet βs). For the simulatio methods, some fuctios had P f beyod the practical rage of applicability. These cases are desigated with a /a i the tables. Because such a small set of data (3) is cosidered for mea ad COV results preseted i the tables, results should ot be cosidered as reliable as those preseted i the figures. However, results i the tables are useful to idetify treds as well as specific problem areas, as discussed i detail below. 16

18 Geeral Observatios For all of the methods cosidered, it is clear by observig the patters i the tables that some types of fuctios are iheretly easier or more difficult to estimate well. I terms of β, most of the poit samplig methods gave high precisio results for cases 1-4, 9,10, ad 14 (table 6). These cases are primarily liear ad cotai distributios other tha extreme I. Cases estimated with high accuracy by most samplig methods were similar: 1-3, 9, ad 14. These are primarily liear ad cotai ormal distributios. Low precisio as well as low accuracy results for most samplig methods were predomiately foud for oliear cases with o-ormal distributios (5, 8, 12, 13, 15, 16, i table 7). The simulatio methods give good results for most cases. There is some degradatio i accuracy ad precisio for two cases, however (13 ad 21). These are oliear fuctios with high variatio. Overall, MCS, IS, ad AISC gave best results for accuracy, while AISP, AISC, ad RF gave best results for precisio. The aalytical method (RF) also provides very good overall results, with early all cases estimated with high accuracy ad precisio (table 8). Quality does degrade as umber of RVs icreases. The 15 RV fuctios for cases 13, 16, 19, ad 21 (oliear with high variatio) are particularly troublesome. As the umber of RVs icreases, the failure surface becomes more complex, with potetially may local miimum. As such, the search algorithm may get stuck i a local but ot global miimum, thereby itroducig error i β estimatio. This is evideced by the fact that RF always overestimates β if it errs, but ever uderestimates it. 17

19 If P f is cosidered, the differeces betwee the poit samplig methods ad simulatio methods are magified, the former providig poor performace. For RF, as with β results, the most iaccurate ad imprecise cases are 13, 16, 18, 19, ad 21. These are primarily oliear with high variatio. The cases that have primarily low target β are 2, 10, 13, 16, ad 21. The most accurate ad precise cases as estimated by most of the poit samplig methods cosidered ted to fall ito this group. For example, cases 2, 10, 13, 18, ad 21 are the most accurate of the samplig methods, while case 2 is the most precise (while the remaiig cases show little differece; see table 9). Although this is ot as clearly defied for the simulatio methods (table 10), there is still sigificat overlap; cases 21 ad 7 are the most precise, while cases 2, 21, 11, ad 18 are the most accurate. Treds become clearer whe iter-depedet factors are cotrolled for, as discussed below. Although β precisio results are quite reasoable i may cases, overall precisio of P f results was low. Although superior to all other methods except RF, the precisio of MCS was also lower tha expected, havig a overall COV of P f results of 0.18 (but 0.06 for β) rather tha the 0.05 predicted usig eq. (1). It should be oted that this approximate formula does ot adjust for fuctio characteristics (RV distributio, liearity, or RV COV), which, i additio to P f, also affect precisio. With regard to LH, for lower failure probabilities (P f ), the method could reduce the umber of MCS simulatios by a factor of 50 for similar accuracy, but for higher P f, proportioally more samples (relative to MCS) were eeded. This varied from o reductio (for P f 0.10), while for 0.10 P f , a reductio factor of about 10 was achieved. It should be oted that, although the LH approach used here provided equivalet accuracy to MCS, a higher degree of variatio was observed tha with MCS (overall COV for β 18

20 = 0.08; overall COV for P f = 0.34). Clearly, icreasig the umber of samples for both of these methods would reduce variatio. Overall, MCS, LH, AISC, ad RF gave best results for accuracy, while MCS, AISP, AISC, ad RF gave best results for precisio. Note that although may of the procedures studied produced large errors i P f i some cases, to keep the results i practical perspective, most civil egieerig structures desiged by codes that lack a reliability-based calibratio (pre-load ad Resistace Factor Desig versios) typically have variatios i P f of orders of magitude. Eve with calibrated codes, the expected variatio of P f is typically greater tha a factor of 2 [33]. Liearity Because the oliear fuctios, take as a whole, have a lower average β tha the liear fuctios (mea β of liear subset = 5.6; mea β of oliear subset = 4), results may be biased, as accuracy ad precisio are ot idepedet of target reliability idex (low β fuctios are i geeral estimated with higher accuracy). To elimiate this bias, a smaller set of fuctios was also cosidered (about half the of the origial), such that the limit states which have a sigificat differece i β betwee their liear ad oliear forms were ot icluded i the set (the average β of the cotrolled liear ad oliear sets are both equal to 3.1). The liear ad oliear β- cotrolled results are idicated as li B ctrl ad oli, B ctrl i figures 1-2 ad It is these results that should be compared to uderstad the true effect that liearity has o accuracy ad precisio. As show i figures 1-2 ad 11-12, for the β -cotrolled results, all methods display better accuracy for liear fuctios ad worse accuracy for oliear fuctios. However, i terms of β (figure 1), differeces due to liearity are mior for the most part. 19

21 Exceptios are ROS ad RF, which display a sigificat bias i accuracy toward liearity. Whe failure probability is cosidered, these differeces are magified, particularly with N+1, which demostrates a sigificat advatage i accuracy with the liear fuctios. For precisio, the same tred is realized, with a few exceptios. It appears that MCS (figures 2 ad 12) results are worse for the β -cotrolled liear cases tha the overall results. This is ot a geeral tred, however, but due to a high-error outlyig sample which throws off the results. If this poit (withi case 3, 5 RV) is elimiated from the set, the tred returs to that which is expected (liear cases are more precise). Normality I figures 2-4 ad 13-14, fuctios are grouped ito those that have all RVs with ormal distributios ( ormal ) ad those that have ay or all RVs with a o-ormal distributio ( oormal ). The mea β for each of these groups is idetical. For the poit samplig methods, RV distributio clearly affect accuracy as well as precisio, where all-ormal fuctios have a distict advatage. This is ot so with the simulatio or RF methods, however, where there is o differece i accuracy. I terms of precisio, the simulatio ad RF methods appear to have a slight advatage for all-ormal fuctios. Two exceptios are ROS (figure 3), ad MCS (figures 4 ad 14), which show a reverse tred. This agai, is due to a sigle outlyig datum (withi case 13, 15 RV for ROS ad case 13, 2 RV for MCS), ad if this datum is removed, ROS ad MCS also shows better results for all-ormal fuctios. 20

22 Number of RVs I figures 5-6 ad 15-16, fuctios are grouped ito those which have 2 ad 15 RVs. The mea β for each of these groups is idetical. For all of the poit samplig methods, accuracy decreases as umber of RVs icreases. I each, there is a sigificat differece i the 2 RV ad 15 RV cases. Iterestigly, for precisio, however, the reverse tred is true, such that the high RV cases are most precise. MCS appears to decrease i accuracy for a high umber of RVs if failure probability is measured but to icrease i accuracy for a high umber of RVs if β is measured. This discrepacy occurs because of the oliear relatioship betwee β ad failure probability (figure 21, discussed further below uder the effect of target β). For the 2 RV cases, data i two outlyig cases (10, 12) cotai large errors. These have low average β (1.4). For the 15 RV cases, fuctios with the worst results (withi cases 5, 6, 10, 12, 14, 15, 20), have a much higher average β (5.0). The errors i this latter group, oce coverted to P f, are greatly icreased relative to those of the low β set. Thus, a differet tred with regard to β ad P f is observed. If these worst cases are ot icluded i the data sets, there is o differece is results betwee the 2 ad 15 RV cases. I terms of precisio, it appears that MCS has a icreased precisio for a low umber of RVs. This is also due to the results of two outlyig data poits (withi cases 13, 21). Elimiatig these two poits results i the low ad high RV case havig the same COV result (0.02). Thus for MCS, the umber of RVs does ot appear to affect accuracy or precisio. Whe LH is cosidered, both accuracy ad precisio degrade as the umber of RVs icreases. This is a overall tred that ca be see i tables 7 ad 10 ad figures 5-6 ad ad is ot due to oe or two very poor results. This result is iterestig, as the expectatio is that LH would show the 21

23 same treds as MCS. It appears that the particular process of PDF stratificatio used i this study [19] may cause results to degrade for a higher umber of RVs. For RF ad IS, as expected, both accuracy ad precisio decrease for the high RV case. This is also true for AISP ad AISC, though to a lesser extet. COV Regardless of the limit state load ad resistace values, each limit state has a upper boud of reliability idex depedig o the fuctio variace. Although fuctios with low variace ca have high βs, fuctios with high variace caot. The resultig effect is that, the low variace fuctios have both high ad low β results, but the high variace fuctios oly have (relatively) low β results. As target reliability idex affects accuracy ad precisio, this effect must be cotrolled for. I figures 7-8 ad 17-18, the category 5, low B represets the set of limit state fuctios i which all RVs have a COV of 5%, ad for which the high β cases were removed, such that the mea β of this set is equal to the mea β of the set of limit states with high variatio (COV for all RVs = 35%). To determie the ubiased effect of COV o accuracy ad precisio, these two cases ( 5, low B ad 35 ) should be compared. I geeral, the tred is that as COV icreases, accuracy ad precisio decrease. The figures dealig with failure probability (17-18) clearly show that the low COV cases, cotrolled for target β ( 5, low B ) are better tha the high COV cases, as well as the ucotrolled low COV cases (5% ucotrolled, which icludes high failure probability cases). Oe exceptio is LH, which has a outlyig datum (case 18, 15 RVs). If this poit is elimiated, the tred matches those of the other methods. Cosiderig reliability idex (figures 7-8), otice 22

24 that for most results, the results for all low COV cases ( 5 ) are better tha the low COV cases for which β is cotrolled ( 5, low B ). Low β cases i geeral show a worseig of results due to the oliear trasformatio betwee failure probability ad reliability idex. This occurs because the low COV cases have a higher average reliability idex tha the β-cotrolled low- COV cases. Errors for the higher β cases, whe coverted to failure probability, are reduced much more tha those of the lower β set. However, the low COV, cotrolled β cases ( 5, low B ) are of course still more accurately ad precisely predicted tha the high COV cases (35%). Here there are two exceptios i terms of accuracy, LH ad IS, which agai are caused by outliers (case 18, 15 RV for LH, ad case 7, 15 RV for IS). Target Reliability Idex I the figures (9-10, 19-20), fuctios are grouped ito the low target β set (β of 2 ad below, with mea of 1.3), a high β set (all β greater tha 2, with mea of 6.75). Sice all high COV fuctios must be withi the low β group, results may be biased, as COV affects accuracy ad precisio as well. Therefore, a additioal data set is cosidered which cotrols for COV (low, low V). This is the set of fuctios that have low COV (i.e. COVs other tha 35%) ad low β. These should be compared to the high β set (which also all have low COV) to ascertai the affect that target reliability idex has o the results. As ca be see from the failure probability figures (19-20), target reliability idex may have a sigificat effect o accuracy ad precisio. I geeral, as failure probability decreases (ad reliability idex icreases), results worse. For the simulatio methods (MCS, LH), this is because failure probability is calculated as the umber of failures sampled (i.e. where g < 0) divided by the total umber of samples. Clearly, at low failure probabilities there are fewer failures to sample, so a error here ca have sigificat effect 23

25 o the overall computed result. For low target βs, sice may more failures are expected as a proportio of the total samples, errors i assessig the umber of failures is ot as sigificat. The poit samplig methods also display a decrease i precisio ad accuracy for low failure probability fuctios, but for a differet reaso. As the poit samplig methods are used to compute β, the result must be trasformed to failure probability. As reliability idex icreases, small errors i β cause icreasigly larger errors i failure probability (fig 21). For accuracy ad precisio, the low β, COV-cotrolled case (low, low V), as expected, is better tha the high β case. The differece is extreme for N+1 ad 2N. A exceptio is LH, which is agai due to a outlyig datum. Because of the reliability idex-failure probability trasformatio, whe β is cosidered (figs. 9 ad 10), the expectatio is that the high β cases would show a icrease i accuracy ad the low β cases a decrease i accuracy. I geeral, the expected tred occurs. This is particularly so with N+1 ad 2N. For precisio (figure 10), four of the seve methods (N+1, 2N, LH, RF) gives poor results for the low- β, COV-cotrolled cases. For each method, the same sigle fuctio withi case 20, for 15 RVs ad a low target reliability idex (0.80), is very poorly predicted. Elimiatig this outlyig datum returs the treds to match that of the three other methods. Special Problems Tables 12 ad 13 give results for the five special problems cosidered. Here the calculated reliability idex is divided by the exact value for compariso. For the oliear limit state (problem 1), all simulatio methods gave good results, most withi a few percet of the exact value. All three poit estimatio methods, however, failed to provide a meaigful result, as 24

26 each predicted a coefficiet of variatio very close or equal to zero, leadig to a ureasoably high reliability idex (200+). This is because each of the 20 RVs i the problem have idetical mea, COV, as well as low sesitivity of the limit state for both positive ad egative perturbatios. I the case of ROS, the sum of the positive ad egative perturbatios cacel out (see equatios 2 ad 3), resultig i zero COV. For N+1 ad N+2, the low sesitivity of the limit state to ay sigle RV results i a gross uder-predictio of COV, as the specified perturbatio value leaves the limit state practically uchaged. The multiple beta value case (problem 2) gave trouble for each of the simulatio methods that use a aalytical method to locate the MPP. Just as with RF, the maximum rather tha the miimum MPP was foud, ad thus results were sampled about a local rather tha a global miimum, over-predictig reliability idex by about 35% i this case. Here use of prior kowledge of the proper MPP locatio would have preveted this. Note that MCS ad LH, which sample without referece to the MPP, provided accurate results. The poit estimatio methods gave results of varyig quality, with ROS close to the exact value. The series system (problem 3) was accurately evaluated by all simulatio methods, as well as RF, although the poit methods uiformly over-predicted reliability idex by about 30%. The parallel system (problem 4) was likewise accurately idetified by the simulatio methods. Here RF sigificatly uder-predicted reliability idex, while the poit estimatio methods gave varyig results. 25

27 The oisy limit state (problem 5) was accurately predicted by all methods except N+1. It appears that small fluxuatios i the limit state boudary are ot early so difficult for the umerical algorithms to deal with tha large-scale shape oliearities. Efficiecy If accuracy ad precisio were the oly cosideratios for structural reliability calculatios, it is clear that poit estimatio methods would have little usefuless, as i most cases they are outperformed by the simulatio methods (as well as RF) by a wide margi. However, the performace of the samplig methods varies greatly with the type of limit state cosidered (i terms of liearity, umber of RVs, distributio type, variace, ad target reliability idex), ad i some istaces, may provide suitable accuracy ad precisio to solve practical problems. Tables 14 ad 15 preset such data. First, the results for all fuctios cosidered i this study are give for a particular method ( OVERALL ). The ext colums list a subset of fuctios for which a particular method was foud to give best results i accuracy ad precisio. These results are highlighted i the table for each method. Note some methods (LH, IS, AISP, AISC, RF) estimate several differet types of limit states equally well, ad so have more tha oe result highlighted i the table. The data reveals that the poit samplig methods ca be very accurate ad precise for certai types of limit states. I terms of reliability idex (table 14), the N+1, 2N, ad ROS methods are equivalet to MCS (overall) accuracy for limit states that are: ormal with low COV (N+1); have low COV (2N); ad have a low umber of RVs (ROS). Moreover, these fuctios may eve exceed MCS (overall) i terms of precisio for limit states that are ormal ad liear, based o the umber of MCS samples coducted i this study. Such results make it 26

28 clear that, at least as importat as the umber of samples chose is how the samples are chose (i.e. radomly, as i MCS, versus selected, as with the poit estimatio methods). Whe cosiderig failure probability, the poit methods do ot show as great of a improvemet, relative to MCS (overall), as with β. N+1 ad 2N, for example, for low β, low COV fuctios, still do ot match MCS (overall) accuracy or precisio. However, cosiderig the rage of variatio expected with failure probability, results from these methods are very reasoable. For ormal, liear fuctios, ROS provides excellet results, ad may exceed MCS (overall). Most precise results for N+1 ad 2N appear for low COV, low RV fuctios, which appears differet from the best results for β (ormal, liear). Here the oliear trasformatio from failure probability to reliability idex effects results. Although the very best results are highlighted i the tables, otice that the actual differeces betwee some types of fuctios are small. The total umber of samples used for a particular type of limit state give a particular method is show i Table 16. All methods cosidered except MCS ad LH have a lower limit of the umber of required simulatios based o the umber of radom variables i the problem. As show i the table, the umber of simulatios is held costat for all limit states of a give umber of RVs for statistical comparisos to be cosistet ad meaigful. The basis for the umber of samples take is described previously i Methods Cosidered, but i geeral these were chose such that most limit states evaluated by a particular method gave results close to that of MCS. Note that for MCS ad the AIS methods, the umber of simulatios idicated is approximate. Of course, icreasig the umber of simulatios for all methods for 27

29 which this is possible (i.e. all but the poit samplig methods), should produce more accurate ad precise results. The differece i computatioal time eeded to execute a sigle loop i ay of these methods for most complex problems is egligible. That is, for practical problems where miimizig computatioal effort would be a cocer (such as those which call a FEA code to evaluate g) the computatioal effort eeded to evaluate the limit state far exceeds that for the reliability algorithm to utilize that iformatio (o more tha a fractio of a secod). Thus, computatioal effort is directly proportioal to the umber of simulatios used as give i Table 16, idepedet of reliability method. By comparig the umber of simulatios used ad the results of the various methods for certai types of limit states, it is clear that, i some cases, the poit methods ca reduce the required umber of samples tremedously, ofte by several orders of magitude, ad yet may retai equivalet accuracy ad precisio as the simulatio methods. Coclusios ad Recommedatios For certai types of fuctios, it was foud that it is possible to tremedously reduce computatioal effort usig poit samplig methods ad yet still maitai competitive accuracy ad precisio. It was also foud that all problems could be accurately evaluated by at least several methods. To select a appropriate method for use, several importat factors must be cosidered: 1) what computatioal resources are available? 2) what accuracy ad precisio are required? 28

30 3) what are the characteristics of the limit state fuctio (umber of RVs, distributio type, variace, liearity, ad target reliability idex)? 4) is reliability idex or failure probability of primary iterest? Based o the data preseted i tables 14 ad 15, some results ad recommedatios ca be summarized. The versios of MCS, LH, IS, ad AIS ivestigated i this study ca be expected to have similar ad good overall results for a wide rage of limit states, ad are recommeded for early all problem types. For all types of limit states, MCS predicts average reliability idex withi 1%, with mea COV of 6%, while LH over predicts reliability idex by 4%, with COV of 8%. For LH, results appear to degrade for a higher umber of RVs. For reliability idex, maximum COV is 15% for 2 RV cases with a typical COV of 3% or less, with a typical beta value withi of 1-2% of the true value (Tables 7, 14). The 15 RV case, however, has a maximum COV of 37% with a value of 15%. Results are similar for beta value. For high RV problems, it may be prudet to avoid the formulatio of LH used i this study. IS predicts reliability idex withi 1% with COV of 6%, while AISP over predicts reliability idex by 4%, with 3% COV. AISP almost always gives a slightly higher (usually withi a few percet) estimate of beta value tha AISC, which predicts average reliability idex withi 1% ad COV of 3%. RF ca excel at early all types of limit states (typical beta ad COV are early exact) except those that have a high umber of RVs or that are highly oliear; here results may be very poor (15 RV forms of cases 13,16,19, ad 21 have a estimated beta over twice the actual value ad COVs of 26-45%; see table 8). Thus it ca be recommeded for all but these types of problems. 29

31 Cosiderig all types of limit states, the poit methods give mixed results. Overall, N+1 uder predicts reliability idex by 20%, with COV of 15%. 2N uder predicts reliability idex 12%, with COV of 15%, while ROS over predicts reliability idex by 12%, with COV of 12%. Limitig results to specific types of problems, if reliability idex is of primary iterest, all three poit methods appear to give excellet results for fuctios that have ormally distributed RVs, ad are liear ad/or have low variace. Specifically, for ormal, low COV problems, all methods give early-exact results, geerally withi a few percet for beta, ad all simulatio methods are withi 2%. ROS is the worse performer here, o average over predictig beta by 13%. For COV, all simulatio methods provide a average COV withi 2% ad a beta value o higher tha 2%. All three poit methods result i a COV o higher tha 6%. If failure probability is eeded ad will ot be coverted to reliability idex, i additio to the characteristics above, fuctios that have low target β (aroud 2 or less, or expected failure probabilities of about 0.02 or more) may be estimated well with the poit methods. For ormal, liear problems, all methods likewise give excellet results withi 1% o average for beta. Here the worst method is N+1, uder predictig average beta by 10%. Most also give good estimates for COV, as all methods are withi 4%, where all poit methods are withi 1%. Therefore, for these specific types of problems, poit methods, icludig +1, ca be useful ad accurate. For the special types of problems cosidered, results are mixed. Poit methods will fail or give very poor results for cases like special problem 1 (may RVs with the same mea ad COV ad the limit state has a low sesitivity to each), ad thus caot be used for this problem type. For problems with multiple beta values, RF ad importace samplig methods may focus o a local 30