Stochastic optimization - how to improve computational efficiency?

Transcription

1 Stochastic optimization - how to improve computational efficiency? Christian Bucher Center of Mechanics and Structural Dynamics Vienna University of Technology & DYNARDO GmbH, Vienna Presentation at Czech Technical University in Prague, 27 June 2014

2 Overview Introduction Stochastic design optimization Reliability analysis Response surface method Example - ten bar truss Deterministic optimization Robust optimization Concluding remarks 2/47 CTU 27 June 2014 c Christian Bucher 2014

3 Uncertainties in optimization Design variables (e.g. manufacturing tolerances) Objective function (e.g. tolerances, external factors) Constraints (e.g. tolerances, external factors) Design Variable 2 Contour lines of objective function Infeasible Domain Design Variable 1 3/47 CTU 27 June 2014 c Christian Bucher 2014

4 Traditional design approach Introduce safety factors into the constraints Leads to results satisfying safety requirement, but not necessarily optimal designs Design Variable 2 Infeasible Domain Safety margin Design Variable 1 4/47 CTU 27 June 2014 c Christian Bucher 2014

5 Motivating Example 1 Deterministic problem C H A B Maximize area of a rectangle A = B H Constraint: limited circumference C = 2(B + H) 4L 0 Solution: B = H = L Stochastic problem B and H are random variables with mean values B and H and standard deviations σ B and σ H, respectively (e.g. normally distributed) 5/47 CTU 27 June 2014 c Christian Bucher 2014

6 Motivating Example 2 Maximize expected value of A Ā = E[B H] or mean plus (or minus) multiple of standard deviation Ā ± k σ A Satisfy constraint condition with certain probability P C 1 Prob[C 0] = Prob[2(B + H) 4L 0] P c L may be random, too Objective function Constraint condition Ā kσ A Max.! Prob[C 0] = Prob[2(B + H) 4L 0] 1 P c = P F 6/47 CTU 27 June 2014 c Christian Bucher 2014

7 Numerical example 1 Assume B and H correlated with ρ, L uncorrelated from B and H Assume all variables to be normally distributed, all have the same coefficient of variation COV Probability of constraint violation: C = 2( B + H) 4 L; L = 1 σc 2 = 4(σB 2 + 2ρσ B σ H + σh) σL 2 [ ] Prob[C > 0] = Φ 0.1 σc C Expected value of area E[BH] = B H + ρσ B σ H Maximize expected area under probability constraint 7/47 CTU 27 June 2014 c Christian Bucher 2014

8 1 Numerical example 2 Solution B, H COV = 0.05 COV = 0.10 COV = 0.25 COV = Coefficient of correlation ρ 8/47 CTU 27 June 2014 c Christian Bucher 2014

9 Numerical example 3 Different probability measures pf = P [C > 0] β = Φ 1 (pf) B = H More linear dependence using β 5 9/47 CTU 27 June 2014 c Christian Bucher 2014

10 Robustness Optimal design should not be sensitive to small variations of uncertain parameters For the case of stochastic uncertainty, robustness can be measured in statistical terms Choice of robustness measures Variance-based: Global behavior, relatively frequent events Probability-based: Specific behavior (often safety-related), rare events 10/47 CTU 27 June 2014 c Christian Bucher 2014

11 Robustness measures 1 Use probabilistic quantities in objective and/or constraints Add multiple of standard deviation of objective to the mean values to form new objective function R = f + kσ f Case k = 0 corresponds to game theory (large portfolio management, insurance, bank loans, fair gambling,...) Use probability of constraint violation as new constraint P [f k (x )] > 0] ε In safety related constraints, the probability level ε may be very small Better to use probability-related measure such as reliability index β = Φ 1 (P [f k (x )) 11/47 CTU 27 June 2014 c Christian Bucher 2014

12 Optimization strategies Conventional strategy Optimize check robustness/reliability if not sufficient, optimize again with modified constraints Easily implemented, may lead to non-optimal results Full RDO strategy incorporate robustness/reliability measure into optimization problem Solve directly for design satisfying robustness/reliability constraint Leads directly to desired solution, may be very expensive 12/47 CTU 27 June 2014 c Christian Bucher 2014

13 The need for speed... Complex system (many parameters, computationally expensive, slow,...) Needed: Fast and reasonably accurate response prediction (e.g. for real-time applications such as control systems) Possible choices: Reduce model complexity based on essential physical features (reduced order model) Replace model based on mathematical simplicity (metamodel) Stochastic analysis needs to be very efficient 13/47 CTU 27 June 2014 c Christian Bucher 2014

14 Reduced order model Need to understand and represent physics May be applicable for many different load cases Very suitable for time dependent phenomena (structural dynamics, convection-diffusion processes) Can be difficult in the presence of strong nonlinearity Typical examples Modal analysis Proper orthogonal decomposition (POD) 14/47 CTU 27 June 2014 c Christian Bucher 2014

15 Metamodel Mathematically formulated black box Suitable for arbitrarily nonlinear input-output relations Requires extensive training data Very difficult to extrapolate Time-dependent problems may be tricky Typical example: Linear response surface model 15/47 CTU 27 June 2014 c Christian Bucher 2014

16 Model Correction factor method Hybrid between reduced order model and meta-model Replaces detailed mechanical model by simple mechanical model Adjust parameters of simple model by calibration to the detailed model Calibration based on probabilistic criteria (e.g. FORM) 16/47 CTU 27 June 2014 c Christian Bucher 2014

17 Common properties Based on previous experience Knowledge of physical processes Acquired experience through training Limited range of applicability Nonlinearities Number of input variables NOTE: Approaches complement each other Combination may be better than the sum of the individual parts! 17/47 CTU 27 June 2014 c Christian Bucher 2014

18 Efficient reliability estimation procedures FORM/SORM (approximation of the limit state) Monte Carlo methods Crude MC - typically used as reference solution when developing a method Importance sampling - requires identification of probabilistically relevant regions in the space of random variables Asymptotic sampling - obtain reliability from results generated with artificially increased standard deviations of the basic variables 18/47 CTU 27 June 2014 c Christian Bucher 2014

19 Essential Steps Sensitivity Analysis to determine the most relevant uncertain parameters Model order reduction for reducing the number of degrees of freedom and complexity Efficient probabilistic analysis to reduce number of samples Validation procedures to detect omissions and errors (includes expert judgement) 19/47 CTU 27 June 2014 c Christian Bucher 2014

20 Reliability analysis Mechanical system F M e M pl Failure condition F = {(F, L, M pl ) : F L M pl } = {(F, L, M pl ) : 1 F L M pl 0 Failure probability P (F) = L P (F) = P [{X : g(x) 0}... I g (x)f X1...X n dx 1... dx n I g (x 1... x n ) = 1 if g(x 1... x n ) 0 and I g (.) = 0 else 20/47 CTU 27 June 2014 c Christian Bucher 2014

21 First-order reliability method (FORM) Rosenblatt-Transformation, e.g. for Nataf model Y i = Φ 1 [F Xi (X i )]; i = 1n Inverse transformation U = L 1 Y; X i = F 1 X i Computation of design point C YY = LL T [ ( n )] Φ L ik U k k=1 u : u T u Min.; subject to: g[x(u)] = 0 Linearize at the design point (in standard Gaussian space) 21/47 CTU 27 June 2014 c Christian Bucher 2014

22 FORM procedure Find point u with minimal distance β from origin in standard Gaussian space u 2 s 2 β u s 1 g(u) = 0 ḡ(u) = 0 u 1 ḡ : n i=1 u i s i + 1 = 0; n P (F) = Φ( β) 1 s 2 i=1 i = 1 β 2 22/47 CTU 27 June 2014 c Christian Bucher 2014

23 Optimization loop Outer optimization loop controls the structural design Probability of constraint violation computed by FORM Inner optimization driven by random variables Both loops need gradients Start optimization loop Repeat for gradients Create one design x j Compute objective f 0 (x j ) Compute probability of constraint violation P k = P [f k (x j ) > 0] Check convergence FORM optimization loop FE analysis 23/47 CTU 27 June 2014 c Christian Bucher 2014

24 Response surface method Reduce computational effort by replacing expensive FE analyses Establish meta-models in terms of simple mathematical functions Fit model parameters to FE solution using regression analysis η x 2 x 1 24/47 CTU 27 June 2014 c Christian Bucher 2014

25 Regression Adjust a model to experiments Set of parameters Y = f (X, p) p = [p 1, p 2,..., p n ] T Experimental values for input X and output Y (X (k), Y (k) ), k = 1... m Search for best model by minimizing the residual S(p) = m k=1 [ Y (k) f (X (k), p)] 2 ; p = argmin S(p) 25/47 CTU 27 June 2014 c Christian Bucher 2014

26 Linear regression Linear dependence on parameters (not on variables!) n f (X, p) = p i g i (X) Necessary condition for a minimum i=1 S p j = 0; j = 1... n Solution { } m n g j (X k )[Y k p i g i (X k )] = 0; j = 1... n k=1 i=1 Qp = q 26/47 CTU 27 June 2014 c Christian Bucher 2014

27 Quality of regression Coeffient of determination (CoD): correlation between experimental data and model predictions ( ) E[Y Z] 2 R 2 = ; Z = σ Y σ Z n p i g i (X) Adjusted (reduced) CoD for small sample sizes (penalize overfitting) Radj 2 = R 2 n 1 ( ) 1 R 2 m n If an additional test data set T is available: Coefficient of Prognosis (CoP) ( ) E[T ZT ] 2 CoP = ; Z T = σ Y σ Z i=1 n p i g i (X T ); 0 CoP 1 i=1 27/47 CTU 27 June 2014 c Christian Bucher 2014

28 Example Adapt polynomial function to 6 data points Variable Y Training set (X k, Y k ) Variable X Variable Y Test set (X k, Y k ) Variable X 28/47 CTU 27 June 2014 c Christian Bucher 2014

29 Regression result Adjust model to training data, cross-check with test data Variable Y Quadratic Training set Cubic Variable X Variable Y Quadratic Test set Cubic Variable X 29/47 CTU 27 June 2014 c Christian Bucher 2014

30 Quality depending on model order Repeat regression and test for different polynomial orders Quadratic model has best prediction capability n R 2 CoP Radj Variable Y Quadratic Cubic Variable X 30/47 CTU 27 June 2014 c Christian Bucher 2014

31 What to approximate? For reliability analysis, the limit state function g(x) is needed Immediate approximation of g may introduce unwanted nonlinear dependencies on input variables F V Example: shear stresses in a console with square cross section B τ xy = 3F H 2B 2 ; τ xz = 3F V 2B 2 B Failure criterion (v. Mises) g(f H, F V ) = β F 3(τxy 2 + τxz) 2 0 F H 31/47 CTU 27 June 2014 c Christian Bucher 2014

32 Comparison Stresses τ xy and τ xz are linear functions of F H and F v Limit state function g(f H, F V ) is highly nonlinear and contains a singularity Try to use easy formulation for constraint functions τ xy τ xz g F H F V F H F V F H F V 32/47 CTU 27 June 2014 c Christian Bucher 2014

33 Example - ten bar truss Configuration: L = 360, F 1 = F 2 = F 1 F L L L Objective: Minimize structural mass Constraints: All member stresses < All nodal displacements < 2 33/47 CTU 27 June 2014 c Christian Bucher 2014

34 Deterministic optimization Gradient-based solver (Conmin) DEMO 34/47 CTU 27 June 2014 c Christian Bucher 2014

35 Optimal design Objective function: m = 5065 Cross sectional areas Member Area Member Area Active constraints: no member stress, displacements in nodes 3 and 6 Effort: 200 FE analyses 35/47 CTU 27 June 2014 c Christian Bucher 2014

36 Robustness evaluation Take into account randomness of loads (F 1 and F 2 independent, log-normally distributed, coefficient of variation = 0.3) Take into account randomness of cross sections (all independent, normally distributed, coefficient of variation = 0.15) Compute variability of constraints Estimate probability of constraint violation(s) using a distribution hypothesis (Gaussian) High probability of violating active constraints, but also 3 inactive ones Effort: 100 FE analyses Member P σ Member P σ Node P u Node P v /47 CTU 27 June 2014 c Christian Bucher 2014

37 Upscaled deterministic solution Increase cross sectional areas of design uniformly to match reliability constraint Use repeated FORM analysis COV m opt Effort: 300 FE analyses for each β 37/47 CTU 27 June 2014 c Christian Bucher 2014

38 Reliability-based optimization Model randomness of loads (F 1 and F 2 independent, log-normally distributed) Model randomness of cross sections (all independent, normally distributed) Accept constraint violation(s) with a probability corresponding to a reliability index β = 3 Use FORM to obtain β for all designs during the optimization process Best designs depend on the coefficient of variation of F 1 and F 2 Effort: FE analyses (factor 300 vs. deterministic case) 38/47 CTU 27 June 2014 c Christian Bucher 2014

39 Robust optima COV m opt Objective Upscaled Direct Coefficient of variation of load Direct: Computational effort: FE runs for 1 COV Upscaled: Computational effort: 1500 FE runs for 1 COV 39/47 CTU 27 June 2014 c Christian Bucher 2014

40 Response surface Use 500 support points in 12-dimensional space Assess quality of regression Cross validation by splitting samples 2/1 Compute coefficient of prognosis (CoP) from applying regression model to unused samples Quantity σ 1 σ 2 σ 3 σ 4 σ 5 CoP Quantity σ 6 σ 7 σ 8 σ 9 σ 10 CoP Quantity u 2 u 3 u 5 u 6 CoP Quantity v 2 v 3 v 5 v 6 CoP /47 CTU 27 June 2014 c Christian Bucher 2014

41 Approximation of displacements 2 Vertical displacement of node 6 Compare displacement computed from FE analysis to response surface results (500 random samples) Fitted displacement True displacement 6 41/47 CTU 27 June 2014 c Christian Bucher 2014

42 Approximation of stresses Stress in member 5 Compare stress computed from FE analysis to response surface results (full quadratic, 500 random samples) Fitted stress True stress 5 42/47 CTU 27 June 2014 c Christian Bucher 2014

43 Alternative approximation of stresses Stress in member 5 Compare stress computed from FE analysis to response surface results (thin plate spline, 500 random samples) Fitted stress True stress 5 43/47 CTU 27 June 2014 c Christian Bucher 2014

44 Reliability-bsed optimization using RSM Gradient-based optimizer (Conmin), FORM based on response surfaces DEMO 44/47 CTU 27 June 2014 c Christian Bucher 2014

45 Robust optima from RSM COV m opt Objective Upscaled RSM Direct Coefficient of variation of load Computational effort reduced by a factor of 100 for one COV (300 for 3 COVs) 45/47 CTU 27 June 2014 c Christian Bucher 2014

46 Compare robust designs All elements are strengthened with larger COV of load Relative member cross section ratios remain similar (not too different from upscaled deterministic solution) COV = 0.1 COV = /47 CTU 27 June 2014 c Christian Bucher 2014

47 Benefits of robust optimization Avoids highly specialized designs, reduces imperfection sensitivity Naturally includes statistical uncertainties into the design optimization process Allows the inclusion of quality control measures (manufacturing, maintenance) into the design process BUT: computationally very expensive unless based on approximations such as response surface models 47/47 CTU 27 June 2014 c Christian Bucher 2014