Reliability Assessment with Correlated Variables using Support Vector Machines

Transcription

1 Reliability Assessment with Correlated Variables using Support Vector Machines Peng Jiang, Anirban Basudhar, and Samy Missoum Aerospace and Mechanical Engineering Department, University o Arizona, Tucson, Arizona, 8572 This paper presents an approach to estimate probabilities o ailure in cases where the random variables are correlated. An explicit limit state unction is constructed in the uncorrelated standard normal space using the Nata transormation and a support vector machine (SVM). The method o explicitly constructing the limit state unction is reerred to as explicit design space decomposition (EDSD), which also includes an adaptive sampling strategy to build an accurate SVM approximation. Several analytical examples with various distributions and also multiple ailure modes are presented. I. Introduction In reliability, the vast majority o studies use uncorrelated random variables to represent uncertainties. However, in many applications (such as biomechanical problems), random variables associated with a system are oten correlated. Not accounting or such correlation can lead to erroneous probability o ailure estimates. Although there are techniques to calculate such a probability o ailure or speciic cases (e.g., normal distributions with inexpensive unction evaluations), there is a need or approaches that can handle highly nonlinear limit state unctions and arbitrary probabilistic distributions while maintaining a reasonable number o unction evaluations. The general approach or addressing problems with correlated random variables is to convert them into uncorrelated standard Gaussian variables. Two o the common methods are Rosenblatt and Nata 2, 3 transormations. The diiculty with implementing Rosenblatt transormation lies in its requirement o the knowledge o the joint probability density unction, which is not always available. In addition, the transormation is not invariant with respect to the order by which the variables are transormed. The Nata transormation, used in this work, only requires the marginal probability density unctions, and is thereore easier to implement. The Nata transormation is explained in detail in Section III. In the literature, most articles dealing with correlated random variables in reliability assessment use moment-based methods such as First and Second Order Reliability Methods (FORM and SORM) 4 It is, however, known that accuracy o such methods is hampered i the limit state is highly nonlinear. Simulationbased methods, such as Monte Carlo Sampling 5 in the uncorrelated space have also been used, but application o such methods is not practical in general due to the large number o samples. 6 In addition, when dealing with correlated variables, the MCS samples generated in the standard Gaussian space need to be transormed back into the original space to evaluate the responses. In this work in progress, a general approach to calculate probabilities with correlated variables while limiting the computational cost is proposed. This is achieved by replacing the threshold-level contour o the actual limit state unction with by a Support Vector Machine (SVM) classiier in the uncorrelated standard normal space. 7, 8, 6 This method o constructing an explicit approximation o the ailure boundary is reerred to as Explicit Design Space Decomposition (EDSD). 9, 6 In order to obtain an accurate SVM and limit the number o unction evaluations, an adaptive sampling technique is used., To determine the class (ailure or sae), each sample used or the construction o the SVM is mapped back to the original correlated Graduate Student, AME Department, and AIAA Student Member. Assistant Proessor, AME Department, AIAA Senior Member o 7 American Institute o Aeronautics and Astronautics

2 space using an inverse Nata Transormation. Because each sample requires a unction evaluation (e.g., a inite element analysis), their number must be limited. Once an SVM is obtained, MCS is perormed in the uncorrelated standard Gaussian space based on the approximation. 6 It should be noted that the MCS samples do not require inverse transormation to the original space because the explicit SVM limit state unction in the uncorrelated space is available. in addition, the proposed approach introduces the probability o misclassiication by SVM using Probabilistic Support Vector machines (PSVMs) in the probability o ailure estimate. This leads to a relatively conservative ailure probability estimate. 2 The use o SVM is appealing due to several reasons. It is based on the classiication o the space into a sae and ailure domains. Construction o the boundary depends only on the class o samples instead o the actual response values. Thereore, the method is applicable to binary (pass or ail) problems 3 or problems with discontinuities. In addition, SVM presents a natural way to handle multiple ailure modes 9,, 4 especially i the system considered is serial since a sample would be considered ailed i any one mode is active. The paper is organized as ollows. Section II describes the basic methodology. Section III presents the Nata transormation along with the Jacobian o the transormation. Section IV to VI provide the basics o EDSD, SVM, and adaptive sampling. Section VII introduces a relatively conservative ailure probability estimate based on Probabilistic Support Vector machines (PSVMs). 2 Finally, Section VIII provides several examples o analytical problems with various probabilistic distributions. II. Basic methodology The basic methodology or calculating probability o ailure with correlated variables, using Support Vector Machines (SVMs) (Section V), 7, 8 is presented in this section. The generation o MCS samples directly in the correlated space (X-space) is possible only or speciic distribution types, such as Gaussian or Truncated Gaussian. In order to generalize the calculation o probabilities o ailure or arbitrary distributions, this works proposes to construct an SVM-based limit state unction in the uncorrelated Standard Gaussian space (Uspace). The probability o ailure is then calculated in using MCS in the U-space. The basic procedure or constructing a limit state approximation in the U-space is demonstrated in Figure. The main steps are: Generation o an initial uniorm design o experiments (DOE) 5, 6, 7 in uncorrelated standard Gaussian space (U-space). Inverse Nata transormation 8 (Section III) o the DOE samples to the original correlated space (Xspace) or response evaluation and class (ail or sae) assignment. Construction o an initial approximation o the limit state in the U-space using explicit design space decomposition (EDSD) (Section IV). Calculation o initial probability o ailure estimate based on the SVM approximation, using MCS in uncorrelated space. Probability o ailure is calculated as: P = N = N MC I g (u i ) () N MC N MC where u i represents Monte Carlo samples in U-space, N MC is total number o Monte Carlo samples and N is number o Monte Carlo samples lying in ailure region based on the SVM approximation. I g (u) is an indicator unction that is or samples in the sae region and or those in the ailure region: I g (u) = { i= s(u) s(u) > (2) 2 o 7 American Institute o Aeronautics and Astronautics

3 Reinement o the limit state approximation using adaptive sampling. Each additional sample is transormed back to the correlated space to obtain its response. The probability o ailure estimate is updated ater each sample evaluation. ailed samples u 2 u 2 u 2 U-space to X-space (inverse Nata) sae samples u u Responses in X-space associated with U-space samples limit state approximation u g(x) x 2 Response evaluation and classiication x 2 x x Figure. Summary o the method or limit state approximation in U-space. III. Nata transormation A. Basic principle The Nata transormation is perormed in two steps. The irst step transorms the variables o any arbitrary distributions to correlated standard normal variables. The second step transorms the latter into uncorrelated standard normal variables.. Step Consider n correlated random variables X i whose marginal cumulative distribution unctions, F Xi and correlation matrix ρ are known. The irst transormation generates correlated standard normal variables U i, with correlation matrix ρ, obtained as ollows: 8, 5 Φ(u ) = F X (x ) Φ(u 2) = F X2 (x 2 ). Φ(u n) = F Xn (x n ) (3) where Φ is the cumulative standard normal distribution. 3 o 7 American Institute o Aeronautics and Astronautics

4 One o the critical aspects o the Nata transormation is to relate the correlation coeicients ρ ij and ρ ij. For this purpose, we use the act that the joint density unction o two correlated variables X i and X j can be expressed as a unction o the joint normal probability density unction o the correlated variables Ui and using the ollowing equation: U j XiX j (x i, x j ) = φ 2 (u i, u j, ρ ij) X i (x i ) Xj (x j ) φ(u i )φ(u j ) (4) where φ 2 (u i, u j, ρ ij ) is the bivariate standard normal probability density unction with correlation coeicient o ρ ij. The expression o φ 2 is: φ 2 (u i, u j, ρ ij) = exp [ 2π (u i )2 2ρ ij u i u j + ] (u j )2 (ρ ij )2 φ(u i )φ(u j ) (5) The relation between ρ and ρ is derived as ollows: ρ ij = cov(x i, X j ) σ Xi σ Xj = E(X ix j ) E(X i )E(X j ) σ Xi σ Xj = σ Xi σ Xj ( + + x i x j Xi,X j (x i, x j )dx i dx j + + ) x i Xi (x i )dx i x j Xj (x j )dx j (6) Using Xi (x i )dx i = φ(u i )du i, the double integral can be simpliied as: ρ ij = = u i u j Xi,X j (x i, x j )dx i dx j u i u jφ 2 (u i, u j; ρ ij)du i du j (7) where u i = (x i µ i )/σ i and u j = (x j µ j )/σ j. We need to solve the double integral to ind the corresponding ρ ij or a given ρ ij. In engineering applications, this process was oten replaced by polynomial approximation 9 in order to remove the need to solve the double integral. For instance, consider a two variable problem with uniorm-exponential variables and correlation coeicient equal to.2. In this case, the relation is approximated as: ρ 2 = Rρ 2 R = ρ 2 2 (8) Although, these relations can be very accurate, it is nowadays possible to solve the double integral numerically in an eicient manner. This latter approach has been used in this paper. 2. Step 2 Once all the elements o the ρ correlation matrix are ound, the second step o the Nata Transormation consists o decorrelating the variables to obtain uncorrelated standard normal variables U i. 8, 2 The ollowing linear transormation is used: U = AU + B (9) U and U here are vectors containing the correlated and uncorrelated standard normal variables respectively. A and B are matrices. By taking the mean o both sides, we obtain: E(U ) = E(AU + B) = AE(U) + B = () Because both variables are standard normal, we obtain B =. A is related to the correlation matrix ρ with the ollowing equation: 4 o 7 American Institute o Aeronautics and Astronautics

5 AA T = ρ () This relation can be rather simply proven by the act that ρ ij = E(u i u j ). Thereore, A can be obtained by Cholesky decomposition o ρ, which is a positive deinite matrix. For a two dimensional problem: [ ] A = (2) (ρ 2 ) 2 For a three variable problem: A = ρ 2 (ρ 2 ) 2 B. Jacobian ρ 3 ρ 2 ρ 23 ρ 3 ρ 2 (ρ 2 2 ) 2 (ρ 3 )2 (ρ 23 ρ 3 ρ 2 )2 (ρ 2 )2 (3) The Jacobian matrix 8 deining the mapping between the original correlated space to the uncorrelated space is deined by its general term as: u j / x i. Using the chain rule and the relation between dx i and du i : The Jacobian is: dx i = φ(u i ) Xi (x i ) du i (4) [ ] φ(u J = diag i ) A (5) Xi (x i ) As an example, the determinant o the Jacobian or a two dimensional problem with exponentialexponential variables, is depicted in Figure 2. The Figure shows how a grid will transorm through the Nata Transormation. For this case, the determinant o the Jacobian is always positive and thus the mapping between the X-space and the U-space is unique. The small values close to the origin o the X-space lead to large distances in the U-space. x2( ) Jacobian determiant rom X to U x ( ) x Figure 2. Contour o Jacobian determinant rom X to U space. 5 o 7 American Institute o Aeronautics and Astronautics

6 x Grid in original space u2 Deormed grid in uncorrelated space ater transormation x u Figure 3. Grid in X-space Figure 4. Grid in U-space ater transormation IV. Explicit Design Space Decomposition (EDSD) Explicit Design Space Decomposition is a classiication-based method that can be used or constructing approximations o limit state unctions. The basic idea in EDSD 9, 6 is to explicitly decompose the space into regions o distinct behavior, e.g. sae and ailed. Instead o approximation o responses, EDSD is based only on their classiication. As a result, the method has several advantages, such as handling o discontinuous and binary responses, 9, 6, 3, 4 and multiple ailure modes. 2, 22 The main steps o EDSD are as ollows. Generation o an initial design o experiments (DOE). In this article samples are selected in the uncorrelated space using Centroidal Voronoi Tessellations 6, 7 (CVT). Response evaluation at the DOE samples. Classiication o the samples as sae (+) or ailed ( ) based on a threshold responses or using clustering. Construction o an explicit boundary separating the + and classes. In this article, SVMs are used to approximate ailure boundaries. V. Support Vector Machines (SVMs) SVMs 7, 8 belong to a class o machine learning techniques that can be used or both regression and classiication. In the context o EDSD, SVMs are used or classiication. SVMs have gained popularity in the computer science community due to their good generalization capability in classiication. In the context o reliability assessment also, lexibility o SVMs in approximating highly nonlinear limit state unctions has been shown in several papers. 23, 6,,, 22 In the case o multiple ailure modes, a single SVM can be used to represent all the modes. The basic theory o SVMs or binary classiication is presented briely in this section. An SVM is used to construct an explicit equation that separates samples belonging to two classes labeled as + and. Given a set o N training samples u i in a n-dimensional U-space, and the corresponding class labels, an SVM boundary is given as: s(u) = b + N λ i y i K(u i, u) = (6) i= where b is a scalar reerred to as the bias, λ i are Lagrange Multipliers obtained rom the quadratic programming optimization problem used to construct the SVM, and K is a kernel unction. The classiication o any 6 o 7 American Institute o Aeronautics and Astronautics

7 arbitrary point u is given by the sign o s(u). The training samples or which the Lagrange Multipliers are non-zero are reerred to as the support vectors. The number o support vectors N SV is usually much smaller than N, and thereore, only a small raction o the samples aects the SVM equation. The kernel unction K in Equation 6 can have several orms, such as polynomial or Gaussian radial basis kernel. The Gaussian kernel (Equation 7) is used in this article. ( K(u i, u) = exp u ) i u 2σ 2 (7) where σ is the width parameter o the Gaussian kernel. It should be noted that although the SVM equation is presented or U-space in terms o u, it is valid or any space. In this article, SVMs are constructed in U-space. VI. Adaptive Sampling EDSD using a static DOE may not be suicient or predicting accurate ailure probabilities, unless a large number o samples N is used. Thereore, to obtain high accuracy with limited samples, an initially sparse DOE in U-space is adaptively updated. The initial DOE is used to construct the irst SVM approximation o the limit state unction. This gives an initial idea o regions that are important or reining the SVM approximation. Additional samples are then added iteratively to update the SVM boundary. The SVM update algorithm consists o two types o samples: Primary sample Secondary samples A primary sample is selected such that it lies on the SVM boundary in a sparsely populated region. Such a point lying near the boundary has high probability o misclassiication. It is selected by maximizing the distance to the closest training sample: max u u u nearest s.t. s(u) = (8) The objective unction in Equation 8 is not dierentiable everywhere in the space. The problem is reormulated to make it dierentiable: max u,z z s.t. u u i z i =, 2,..., N s(u) = (9) A secondary sample is selected in a region in the vicinity o SVM boundary that has the highest local unbalance o data rom the two classes. Unbalance o data is quantiied as d d +, where d and d + are the distances to the closest and + samples. A high unbalance o data may be an indicator o a phenomenon known as locking o SVM, which may result in a very low rate o convergence o SVM to the actual limit state unction. Although change in SVM due to primary samples may be negligible in such cases, secondary samples are useul in removing locking. This is shown conceptually in Figure 5. Selection o a secondary sample is a two step process: Selection o a point u c on s(u) = with highest measure o unbalance. max u (d (u) d + (u)) 2 Selection o secondary sample within a hypersphere centered at u c. s(u) = (2) min u sign(d (u c ) d + (u c ))s(u) u u c 4 d (u) d + (u) (2) 7 o 7 American Institute o Aeronautics and Astronautics

8 , 22 More details about SVM locking can be ound in previous articles. A dedicated adaptive sampling method to calculate accurate ailure probabilities using SVMs can also be ound. 22 SVM locking secondary sample selection SVM update with locking removal d d u (arbitrary point) - class + class region with unbalanced data (no + sample) circle centered at u c R u c(point on SVM with largest unbalance) secondary sample in shaded region x c updated SVM evaluated secondary sample Figure 5. Locking o SVM with locally unbalanced data in the vicinity o SVM (let), selection o a secondary sample (mid), and update o SVM due to the evaluated secondary sample (right). VII. Monte-Carlo Simulations based on Probabilistic Support Vector Machines (PSVMs) Adaptive sampling (Section VI) increases the accuracy o SVM approximation and reduces the probability o misclassiication. However, the error in the SVM prediction might result in an inaccurate probability o ailure. Underestimation o probability o ailure is especially hazardous as it may lead to an unsae design. Thereore, it is useul to have a relatively conservative estimate o probability o ailure to compensate or errors in SVM classiication. A. Conservative Failure Probability Estimate A method based on PSVMs is used to provide a relatively conservative probability o ailure compared to corresponding deterministic SVMs. 2 Unlike deterministic SVMs that provide binary classiication, PSVMs provide a conditional probability o belonging to a speciic class or any sample. Thus, or any Monte Carlo Sample lying in sae region or + class based on a deterministic SVM, the corresponding PSVM provides a probability o belonging to the or ailure class (P ( u)). The probability o ailure given by Equation is replaced by: ( NMCS ) P P SV M = γ ( u), N MCS i= u i Ω γ ( u) = u i Ω sd Ω misc (22) P ( u) u i Ω misc The dierence between the traditional binary indicator unction I g (u) (Equation 2) and γ( u) lies in their values in the region Ω misc. Ω misc is the region in sae domain, based on deterministic SVM, or which a probability o misclassiication is considered in Equation 22. The contribution o MCS samples lying in Ω misc to the probability o ailure is zero in Equation. However, when using the PSVM-based method, these samples have a non-zero contribution in Equation 22. Thereore, Equation 22 provides a relatively conservative probability o ailure. The region Ω misc or considering the probability o misclassiication by the SVM is: Ω misc = Ω sd Ω ( s(u) < s(u)(d + (u) d (u) )), (23) where Ω sd is the sae domain based on the deterministic SVM, and d + (u) and d (u) are the distances o u to the closest + and training samples in U-space. Ω misc consists o two kinds o regions in the + class. 8 o 7 American Institute o Aeronautics and Astronautics

9 One is the SVM margin in the sae class and the other is the region with d + (u) d (u) (Figure 6). The calculation o γ ( u) in the region Ω misc, equal to P ( u), is explained in the ollowing section. sae + class sae + class s(u) d ( u) d ( u) s(u) d (u) d (u) ailed - class ailed - class Figure 6. Deinition o the region Ω misc or considering the probability o misclassiication. Ω misc is the union o the two shaded regions in the let and the right igures. B. Probabilistic Support Vector Machines (PSVMs) Probabilistic support vector machines are used to map SVM values to conditional probabilities o belonging to a speciic class. One o the commonly used PSVM models proposed by Platt involves the itting o a sigmoid unction using maximum likelihood. 24 This model, however, ignores the spatial distribution o samples. It only accounts or SVM values while calculating P (+ u) and P ( u). To overcome this issue, a modiied sigmoid model is used in this article that accounts or the spatial distribution o samples. 25 I u is ixed in space, PSVM provides the probability o belonging to + or sae class: P (+ u) = d (u) As(u)+B( + e A < d + (u)+τ d + (u) d (u)+τ ) 3, B <, (24) min(s max, s min ) where d (u) and d + (u) are the distances to the closest - and + samples. τ is a small quantity (set equal to in this work) added in order to avoid numerical issues at the evaluated training samples. s max is the maximum SVM value among all training samples and s min is the minimum SVM value among all training samples. The PSVM model in Equation 24 satisies the ollowing conditions: P (+ u) i s(u) or d + (u) P (+ u) i s(u) or d (u) P (+ u).5 i s(u) and d (u) d + (u) Parameters A and B o the PSVM model are calculated using maximum likelihood. For urther details, the reader may reer to. 25 The conditional probability o belonging to or ailure class is: P ( u) = P (+ u) (25) VIII. Results Several examples are presented in this section to demonstrate the calculation o probabilities o ailure using correlated variables. To show the generality o the proposed approach, various distribution types, namely Gaussian, Exponential and Weibull, are used. The examples presented consist o two and three random variables. An example showing the use o PSVM or ailure probability calculation is also presented. Probabilities o ailure are calculated using 6 MCS samples or all the examples. In all o the ollowing examples, g(x) represents ailure domain. The ollowing notations are used to present the results: P SV M is the probability o ailure predicted by SVM 9 o 7 American Institute o Aeronautics and Astronautics

10 P P SV M is the probability o ailure predicted by PSVM P Re P UC is the probability o ailure by direct transormation is the probability o ailure ignoring correlation 95% CI o P Re is 95% conidence interval o P Re ɛ SV M is the relative error between P SV M and P Re ɛ SV M = P SV M P Re P Re % (26) ɛ UC is the relative error between P UC and P Re ɛ UC = P UC P Re P Re % (27) A. Two dimensional examples Two examples with dierent distributions are presented in this section. The second example consists o our ailure modes, all o which are represented using a single SVM. In both examples, the initial SVM in U-space is constructed with 3 samples. The SVM approximations are reined using adaptive sampling. The update is terminated i the relative change in probability o ailure is less than.% or ive consecutive iterations.. Exponential-Exponential Distributions The actual limit state unction or this example, deined in X-space, is: g(x, x 2 ) = 3 (x 2) 3 3(x 2 ) = (28) Both variables have exponential distributions with mean o and the correlation coeicient is -.4. Using the polynomial approximation 9 to solve or ρ 2 ( R 2 = ρ ρ 2 2 and ρ 2 = R 2 ) we obtain ρ 2 =.56 while the numerical solution through double integration yields The relative error is.7% thus showing that either approach is satisactory. The transormation matrix A is then determined by Cholesky decomposition o the correlation matrix ρ : A = [ The limit state unctions in X-space and U-space are depicted in Figures 8 and 9. The updated SVM boundary in U-space is shown in Figure. MCS samples used or ailure probability calculation are plotted in Figure. The inal SVM is constructed with 5 samples. Results o SVM update and ailure probability calculation are presented in Table 2. For inormation, Figure 2 depicts the MCS samples when correlation is included and Figure 3 when correlation is ignored. The convergence plot or this problem is provided in Figure 4. ] (29) 2. Exponential-Weibull Distributions with multiple ailure modes This example consists o our ailure modes. A sample is considered ailed i any o the modes is active (i.e., it is a series system). The ailure domain Ω is union o the ollowing modes: g (x, x 2 ) = 3 (x 2) 3 3(x 2 ) g 2 (x, x 2 ) = 2.5 x + x 2 g 3 (x, x 2 ) = x 2 5x + x g 3 (x, x 2 ) = 2x 2 x (3) o 7 American Institute o Aeronautics and Astronautics

11 Table. Results o 2D and 3D problems. Results Exp-Exp Exp-Wbl Norm-Exp-Wbl P SV M ( 2 ) P Re ( 2 ) ɛ SV M (%) % CI o P Re ( 2 ) [5.629, 5.722] [5.2, 5.8] [.2673,.2883] P UC ( 2 ) ɛ UC (%) * Assuming x and x 2 are uncorrelated,, samples are used in Monte Carlo simulation. Calculated by transorming back to original space PDF o x PDF o x PDF.4 PDF x x 2 Figure 7. Distribution o correlated input variables x Limit state in x x 2 (correlated) plane Limit state u2 Limit state in u u 2 (uncorrelated) plane 3 Limit state x u Figure 8. Limit state unction in X-space Figure 9. Limit state unction in U-space o 7 American Institute o Aeronautics and Astronautics

12 5 Evolving SVM in uncorrelated space Limit state SVM Sae Failure u u Figure. Updated SVM ater adaptive sampling Figure. Monte Carlo samples in the uncorrelated standard normal space. Figure 2. Transorming uncorrelated samples back to original space. Figure 3. Direct sampling in original space ignoring correlation. 8 Convergence plot o P 3% error % error P Relative error (%) Number o adaptive samples Figure 4. Convergence plot o probability o ailure P. x is an exponential variable with mean o and x 2 ollows Weibull distribution with α = β = 2.The 2 o 7 American Institute o Aeronautics and Astronautics

13 distribution plots o the two variables are shown in Figure 5. The composite limit state unctions in X-space and U-space are shown in Figures 6 and 7. The updated SVM boundary in U-space is shown in Figure 8. MCS samples used or ailure probability calculation are plotted in Figure 9. The updated SVM is constructed with 25 samples. Failure probability results are presented in Table 2. PDF o x PDF o x PDF.4 PDF x x 2 Figure 5. Distribution o correlated Exp-Wbl variables Limit state unction in x x 2 (correlated) plane Limit state Limit state 2 Limit state 3 Limit state 4 Sae Limit state unction in u u 2 (uncorrelated) plane Limit state Limit state 2 Limit state 3 Limit state 4 x2 3 u x u Figure 6. Multiple limit states in X-space Figure 7. Multiple limit states in U-space B. Three dimensional example. Normal-Exponential-Weibull Distribution Extending the application to 3D case, consider the limit state unction: g(x, x 2, x 3 ) = x 3 2x 2 3 x 3 3.5x x 2 x 3 + 9x 2 + 8x x x 54x 2 54x = (3) x is a normal variable with mean and standard deviation o. x 2 is an exponential variable with mean o.x 3 ollows a Weibull distribution with α =.5 β = 2.5. The distribution plots o the two variables are shown in Figure 2. The limit state unctions in X-space and U-space are shown in Figures 22 and 23. The updated SVM boundary in U-space is shown in Figure 24. MCS samples used or ailure probability calculation are plotted in Figure 25. The initial SVM is built with 5 CVT 6, 7 samples. At convergence, 3 o 7 American Institute o Aeronautics and Astronautics

14 SVM ater adaptive sampling u2 4 SVM Sae 3 Failure u Figure 8. Updated SVM ater adaptive sampling Figure 9. Monte Carlo samples classiication based on updated SVM 3 25 Convergence plot o P P 3% error % error Relative error(%) Number o adaptive samples Figure 2. Convergence plot o probability o ailure P. the inal SVM is constructed with 79 samples in total. calculation are presented in Table. Results o SVM update and ailure probability C. Two dimensional example with Probabilistic SVM (PSVM) We consider the same two-dimensional problem with exponential-exponential distributions as presented in Section. The method based on PSVMs (see Section A) is used to provide a relatively conservative probability o ailure with less number o unction evaluations compared to the corresponding deterministic SVM. The convergence o probabilities o ailure using the deterministic SVM and the PSVM are plotted in Figure 27. Ater adding 53 adaptive samples, it is observed that the probability o ailure based on PSVM will always be larger than the reerence probability calculated by direct transormation. On the contrary, the deterministic SVM can underestimate the probability o ailure even with many more sample evaluations. 4 o 7 American Institute o Aeronautics and Astronautics

15 PDF o x µ =, σ = PDF o x2 µ = PDF o x3 a:.5 b: PDF.4 PDF.4 PDF x x x3 Figure 2. Distributions o correlated input variables or three dimensional problem Figure 22. Limit state unction in X-space Figure 23. Limit state unction in U-space Figure 24. Updated SVM ater adaptive sampling or the three dimensional problem Figure 25. Monte Carlo samples classiication based on updated SVM However these observations are only valid or this example and cannot be generalized. More generally, the PSVM approach will require less unction evaluations because it will always be, by construction, more conservative than the estimate based on the deterministic SVM. For example, i a less tight convergence criterion o % (instead o.%) relative change o probability is used, the number o samples needed is 34. The deterministic SVM will give a lower probability o ailure with an error o 6.366%, while the probability calculated based on PSVM will give an error o.64% (see Table 2). 5 o 7 American Institute o Aeronautics and Astronautics

16 Relative error(%) Convergence plot o P P 3% error % error Number o adaptive samples Figure 26. Convergence plot o probability o ailure P or the three dimensional problem. Convergence plot o P Relative change o P Relative error(%) 5 5 Deterministic P PSVM based P 3% error % error Relative change (%) P change % error.% error No. o adaptive samples No. o adaptive samples Figure 27. Convergence plot o deterministic P and P based on PSVM. Figure 28. Relative change o P or stopping criteria Table 2. Results o 2D Exp-Exp using PSVM. Results.% criteria % criteria No. o adaptive samples P SV M ( 2 ) P P SV M ( 2 ) P Re ( 2 ) ɛ SV M (%) ɛ P SV M (%) Calculated by transorming back to original space IX. Conclusion This work in progress introduces an approach to evaluate probabilities o ailure when the random variables are correlated. The technique is based on the explicit construction o the limit state unction in the 6 o 7 American Institute o Aeronautics and Astronautics

17 uncorrelated standard normal space ater Nata transormation. The construction o an accurate limit state unction is based on SVM explicit design space decomposition with adaptive sampling. Results with various random distributions and multiple ailure modes are presented. The next steps o this research will involved higher dimensional problems and reliability-based design optimization. X. Acknowledgments The support o the National Science Foundation through grant CMMI is grateully acknowledged. Reerences Rosenblatt, M., Remarks on a multivariate transormation, The Annals o Mathematical Statistics, Vol. 23, No. 3, 952, pp Nata, A., Determination des distribution dont les marges sont donnees, Comptes rendus de lacademie des sciences, Vol. 225, 962, pp Liu, P. and Der Kiureghian, A., Multivariate distribution models with prescribed marginals and covariances, Probabilistic Engineering Mechanics, Vol., No. 2, 986, pp Haldar, A. and Mahadevan, S., Probability, reliability, and statistical methods in engineering design, Wiley, 2. 5 Melchers, R., Structural reliability analysis and prediction, Vol. 437, John Wiley & Sons, Basudhar, A., Missoum, S., and Harrison Sanchez, A., Limit state unction identiication using Support Vector Machines or discontinuous responses and disjoint ailure domains, Probabilistic Engineering Mechanics, Vol. 23, No., 28, pp.. 7 Vapnik, V.N., Statistical Learning Theory, John Wiley & Sons, Gunn, S.R., Support vector machines or classiication and regression, Tech. Rep. ISIS--98, Department o Electronics and Computer Science, University o Southampton, Missoum, S., Ramu, P., and Hatka, R.T., A Convex Hull Approach or the Reliability-based Design o Nonlinear Transient Dynamic Problems, Computer Methods in Applied Mechanics and Engineering, Vol. 96, No. 29, 27, pp Basudhar, A. and Missoum, S., Adaptive explicit decision unctions or probabilistic design and optimization using support vector machines, Computers & Structures, Vol. 86, No. 9 2, 28, pp Basudhar, A. and Missoum, S., An improved adaptive sampling scheme or the construction o explicit boundaries, Structural and Multidisciplinary Optimization, Vol. 42, No. 4, 2, pp Basudhar, A. and Missoum, S., Reliability Assessment using Probabilistic Support Vector Machines (PSVMs), 5 st AIAA/ASME/ASCE/AHS/ASC conerence on Structures, Dynamics and Materials. Paper AIAA , Orlando, Florida, April 2. 3 Layman, R., Missoum, S., and Vande Geest, J., Simulation and probabilistic ailure prediction o grats or aortic aneurysm, Engineering Computations Journal, Vol. 27, No., 2. 4 Basudhar, A. and Missoum, S., A sampling-based approach or probabilistic design with random ields, Computer Methods in Applied Mechanics and Engineering, Vol. 98, No , 29, pp Montgomery, D.C., Design and Analysis o Experiments, Wiley and Sons, Romero, Vincente J., Burkardt, John V., Gunzburger, Max D., and Peterson, Janet S., Comparison o Pure and Latinized Centroidal Voronoi Tesselation Against Various Other Statistical Sampling Methods, Journal o Reliability Engineering and System Saety, Vol. 9, No , Du, Q., Faber, V., and Gunzburger, M., Centroidal Voronoi tessellations: applications and algorithms, SIAM review, Vol. 4, No. 4, 999, pp Lemaire, M., Structural Reliability, ISTE and John Wiley, st ed., Kiureghian, A. D. and Liu, P.-L., Structural Reliability Under Incomplete Probability Inormation, Journal o Engineering Mechanics, Vol. 2, Noh, Y., Choi, K., and Du, L., Reliability-based design optimization o problems with correlated input variables using a Gaussian Copula, Structural and Multidisciplinary Optimization, Vol. 38, No., 29, pp Arenbeck, H., Missoum, S., Basudhar, A., and Nikravesh, P.E., Reliability-Based Optimal Design and Tolerancing or Multibody Systems Using Explicit Design Space Decomposition, Journal o Mechanical Design, Vol. 32, No. 2, 2, pp Basudhar, A. and Missoum, S., Local Update o Support Vector Machine Decision Boundaries, 5 th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conerence, Palm Springs, Caliornia, May Hurtado, Jorge E., An examination o methods or approximating implicit limit state unctions rom the viewpoint o statistical learning theory, Structural Saety, Vol. 26, July 24, pp Platt, J. C., Probabilistic Outputs or Support Vector Machines and Comparisons to Regularized Likelihood Methods, Advances in Large Margin Classiiers, MIT Press, 999, pp Basudhar, A. and Missoum, S., Constrained Eicient Global Optimization with Support Vector Machines (SVMs), 3th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conerence, Dallas, TX, September 2. 7 o 7 American Institute o Aeronautics and Astronautics