4M020 Design tools Algorithms for numerical optimization L.F.P. Etman Department of Mechanical Engineering Eindhoven University of Technology Wednesday September 3, 2008 1 / 32
Outline 1 Problem formulation: classes and properties 2 Optimization algorithms 2 / 32
Problem formulation: classes and properties The mathematical problem formulation is to the heart of the success of design optimization 3 / 32
Two-bar truss example d S F S h min x f(x) = C 0 x2 2 S 2 + x1 2 s.t. g 1 (x) = C 1 x 1 1 0 g 2 (x) = C 2 S 2 + x1 2 x2 2x 1 1 0 ( S 2 + x1 2 g 3 (x) = C 3 x2 4 x 1 χ : x 1, x 2 > 0, ) 3/2 1 0 How can we solve this optimization problem? 4 / 32
Mathematical problem formulation Minimize f(x) x = (column) vector of design variables x subject to h j (x) = 0 j = 1,..., m h g k (x) 0 k = 1,..., m g x X R n [Papalambros & Wilde 2000: Principles of optimal design] 5 / 32
Formulation examples Design variables: sizing (dimensions) shape (geometry of boundary) topological (material distribution) 6 / 32
Formulation examples Design variables x: sizing (dimensions) shape (geometry of boundary) topological (material distribution) Objective function f(x): profit (cost, efficiency, weight,...) Constraint functions h j (x) and g k (x): geometrical (width, length, height,...) structural (stresses, displacements,...) dynamical (accelerations, eigenfrequency,...) physical (temperatures, pressures,...) 7 / 32
Formulation classes Design variables: continuous or discrete single-variable or multi-variable Objective function: minimization or maximization single-objective or multi-objective Constraint functions: unconstrained or constrained equality or inequality 8 / 32
Formulation classes: linear programming LP temp 2006 Linear programming (LP) problem: Linear objective function Linear constraint functions Continuous design variables g 3 f Example: x 2 g 2 min x R 2 f(x) = 2x 1 x 2 s.t. g 1 (x) = x 1 + 2x 2 8 0 g 2 (x) = 2x 1 2x 2 3 0 g 3 (x) = 2x 1 + 1 0 g 4 (x) = 2x 2 + 1 0 (Some) design variables discrete: (Mixed-)integer linear programming problem: (M)ILP F x 1 g 4 g 1 9 / 32
Formulation classes: quadratic programming QP temp 2006 Quadratic programming (QP) problem: Quadratic objective function Linear constraint functions Continuous design variables f g 3 Example: x 2 min x R 2 f(x) = 3x 2 1 2x 1 + 5x 2 2 + 30x 2 s.t. g 1 (x) = 2x 1 3x 2 + 8 0 g 2 (x) = 3x 1 + 2x 2 15 0 g 3 (x) = x 2 5 0 F g 2 g 1 (Some) design variables discrete: (Mixed-)integer quadratic programming problem: (M)IQP x 1 10 / 32
Formulation classes: nonlinear programming NLP temp 200 Nonlinear programming (NLP) problem: Non-linear objective function Non-linear constraint functions Continuous design variables Example: min f(x) = 3x 1 + 3x 2 x R 2 s.t. g 1 (x) = 18 x 1 + 6 3 x 2 3 0 g 2 (x) = 5.73 x 1 0 g 3 (x) = 7.17 x 2 0 (Some) design variables discrete: (Mixed-)integer nonlinear programming problem: (M)INLP x 2 g 2 g 3 g 1 F f x 1 11 / 32
Formulation properties Design variables: Domain: {0, 1}, N, N +, Z, R, R + Objective and constraint functions: Linearity: linear or nonlinear Continuity: none/once/twice-differentiable Optimization problem: Modality: uni-modal or multi-modal 12 / 32
Two-bar truss example d S F S h min x f(x) = C 0 x2 2 S 2 + x1 2 s.t. g 1 (x) = C 1 x 1 1 0 g 2 (x) = C 2 S 2 + x1 2 x2 2x 1 1 0 ( S 2 + x1 2 g 3 (x) = C 3 x2 4 x 1 χ : x 1, x 2 > 0, ) 3/2 1 0 Exercise 1: introduction to the Matlab algorithm fmincon 13 / 32
Optimization algorithms Select an optimization algorithm in accordance with the optimization problem class and properties 14 / 32
Searching for a zero ( ) 0 15 / 32 Solving a set of m nonlinear equations with m unknowns: Φ(q) = 0 Example: 1 + e q 2 q 2 2 e q 1 = 0 1 q 1 e q 2 + 2q 2 e q 1 = 0
Searching for a minimum ( ) Finding the minimum of a function with n variables: Minimize x f(x) Example: Min x f(x) = x 1 + x 2 + x 1 e x 2 + x 2 2 e x 1 16 / 32
Zero-search is special case of minimum-search Finding the solution to a set of nonlinear equations: Φ(q) = 0 can be reformulated as a minimization problem: Minimize q e T e with the error (deviations to zero) defined as: e = Φ(q) 17 / 32
Analytical versus numerical solution Some optimization problems can be analytically solved: Min x f(x) = 2x 1 + x 2 1 + 2x 2 + x 2 2 Many optimization problems can only be numerically solved: Min x f(x) = x 1 + x 2 + x 1 e x 2 + x 2 2 e x 1 An optimization algorithm is an iterative procedure to solve an unconstrained or constrained minimization (maximization) problem. [Papalambros & Wilde 2000: Principles of optimal design] 18 / 32
Classification of optimization algorithms Four classifiers of an optimization algorithm: unconstrained / constrained search local / global search determistic / stochastic search 0 th / 1 th / 2 nd -order search 19 / 32
Unconstrained / constrained search Unconstrained search: aims to minimize a nonlinear, possibly multi-modal, objective function in n-dimensional space Constrained search: aims to minimize a (non)linear objective function in n-dimensional space while accounting for (non)linear equality and/or inequality constraint functions 20 / 32
Local / global search Local search: seeks improvement based on the Taylor series expansion; in a sufficiently small region any nonlinear function can be represented by a quadratic approximation: f(x) = f(x )+ n i=1 f(x ) x i (x i x i )+ 1 2 n n i=1 j=1 2 f(x ) x i x j (x i x i )2 Global search: seeks improvement by design space exploration 21 / 32
Deterministic / stochastic search Deterministic search: gives exactly the same search path when running the algorithm twice for unchanged algorithmic settings Stochastic search: generates random search paths for every run of the algorithm 22 / 32
0 th / 1 th / 2 nd -order search A 0 th -order algorithm uses function evaluations only A 1 th -order algorithm uses function and gradient evaluations A 2 nd -order algorithm uses function, gradient, and Hessian evaluations 23 / 32
Newton type of algorithms Characteristics line2 temp 2006/6/26 Matlab Optimization 14:30 trust2 temp page toolbox 1 #1 2006 unconstrained/constrained fminunc local fmincon deterministic 2 nd -order x 1 x 2 x 0 s 0 x 3 s 1 x s 2 2 s 3 x 2 0 1 x 1 x 2 2 s 0 s 1 s2 x 0 x 3 3 Line search x 1 Trust-region x 1 24 / 32
Bio-inspired algorithms Characteristics unconstrained/constrained global stochastic 0 th -order Algorithms Particle swarms Genetic algorithms (GAs) Simulated Annealing Matlab GA toolbox 2Dparticleswarm temp ga 2 1 25 / 32
Direct search algorithms Characteristics unconstrained/constrained local deterministic 0 th -order Algorithms Nelder-Mead simplex search Generalized Pattern Search Matlab Optimization 2Dsimplex temp toolbox fminsearch Matlab Direct Search toolbox patternsearch 2 1 26 / 32
Two-bar truss example d S F S h min x f(x) = C 0 x2 2 S 2 + x1 2 s.t. g 1 (x) = C 1 x 1 1 0 g 2 (x) = C 2 S 2 + x1 2 x2 2x 1 1 0 ( S 2 + x1 2 g 3 (x) = C 3 x2 4 x 1 χ : x 1, x 2 > 0, ) 3/2 1 0 Exercise 2: visualization of the optimization search path 27 / 32
Gradients Calculating derivatives: Analytically (by hand) Symbolically (e.g. Mathematica, Maple) Finite differencing By the CAE analysis code (e.g. MARC) Automatic differentiation (e.g. ADIFOR, ADIC) Matlab: fmincon and fminunc calculate gradients by finite differencing if the user does not provide them Matlab Symbolic Toolbox Matlab Automatic Differentiation Toolbox 28 / 32
Conditions for optimality unc infwell2d temp 2007/4/4 15:12 ineq opt temp page 1 #12006/6/20 10:00 page 1 #1 Karush-Kuhn-Tucker (KKT) conditions x 2 f 10 9 8 x 2 F f x 2 g 1 F f 3 = 0 g 3 P 2 g g 1 g 1 2 g 2 g 2 P 1 x 1 x 1 x 1 Unconstrained: gradients objective function zero Constrained: linear combination of gradients objective and gradients active constraints zero (i.e. gradients Lagrange function L zero) 29 / 32
Termination criteria Condition on optimality L(x k+1 ) < ε g j (x k+1 ) < ε and h j (x k+1 ) < ε (Matlab: TolFun) (Matlab: TolCon) or a condition on change in x x k+1 x k < ε (Matlab: TolX) or a condition on the number of iterations k k max (Matlab: MaxIter) or a combination of these, with ε > 0 30 / 32
Scaling Scale design variables, objective function and constraint functions to avoid numerical difficulties and premature termination 31 / 32
Two-bar truss example d F h min x f(x) = m s.t. g 1 (x) = C 1 x 1 1 0 g 2 (x) = σ(x) σ y 1 0 g 3 (x) = P(x) P c (x) 1 0 χ : x 1, x 2 > 0, S S Exercise 3: FEM-model in the optimization loop 32 / 32