A Simple Heuristic Based on Alternating Direction Method of Multipliers for Solving Mixed-Integer Nonlinear Optimization

Size: px

Start display at page:

Download "A Simple Heuristic Based on Alternating Direction Method of Multipliers for Solving Mixed-Integer Nonlinear Optimization"

Blanche Washington
5 years ago
Views:

1 A Simple Heuristic Based on Alternating Direction Method of Multipliers for Solving Mixed-Integer Nonlinear Optimization Yoshihiro Kanno Satoshi Kitayama The University of Tokyo Kanazawa University May 21 24, 2018 (ACSMO 2018)

2 a challenging optim. prob. mixed-integer nonlinear programming (MINLP): Min. f (x) s. t. g i (x) 0 (i= 1,..., m), x 1,..., x r : discrete, x r+1,..., x n : continuous. hard to solve

3 a challenging optim. prob. mixed-integer nonlinear programming (MINLP): Min. f (x) s. t. g i (x) 0 (i= 1,..., m), x 1,..., x r : discrete, x r+1,..., x n : continuous. hard to solve our heuristic simple computationally cheap often finds good solutions metaheuristics generally, computationally expensive

4 ex. of MINLP (1/2) optimal design of pressure vessel [Sandgren 90] 4 design variables: s, t : discrete (chosen from catalog) L, R : continuous objective: min. manufacturing cost (nonconvex) constraints: ASME code (nonconvex) t R L s R

5 ex. of MINLP (2/2) optimal design of truss w/ discrete cross-sections design variables: member cross-sectional areas : discrete (from catalog) state variables (e.g., displacement) : continuous objective: min. structural weight (topology is fixed) constraints: stress & displacement equilibrium eq. (nonconvex)

6 basic idea of our heuristic multiple restart of ADMM from random initial points

7 basic idea of our heuristic multiple restart of ADMM from random initial points ADMM=alternating direction method of multipliers each iteration consists of... solving an NLP (nonlinear programming) problem rounding a vector to the nearest integer vector updating dual variable (by vector addition) often converges w/n 20 iterations simple & computationally cheap

8 ADMM=alternating direction method of multipliers an algorithm for convex optimization: Min. f (x)+g(z) s. t. Ax+Bz=c. f, g : convex x, z : variables

9 ADMM=alternating direction method of multipliers an algorithm for convex optimization: Min. f (x)+g(z) s. t. Ax+Bz=c. f, g : convex x, z : variables classically, [Glowinski & Marrocco 75], [Gabay & Mercier 76] distributed optim. for convex prob. a precursor: method of multipliers recently, heuristic for nonconvex prob. data science [Chartrand 07], [Kanamori & Takeda 14] mixed-integer convex program [Takapoui, Moehle, Boyd, & Bemporad 18]

10 iteration of ADMM target ( f, g : convex): Min. f (x)+g(z) s. t. Ax+Bz=c. augmented Lagrangian: L ρ (x,z;y)= f (x)+g(z) + y (Ax+Bz c)+ ρ 2 Ax+Bz c 2 y : Lagrange multiplier ρ > 0 : penalty parameter

11 iteration of ADMM target ( f, g : convex): Min. f (x)+g(z) s. t. Ax+Bz=c. augmented Lagrangian: L ρ (x,z;y)= f (x)+g(z) + y (Ax+Bz c)+ ρ 2 Ax+Bz c 2 y : Lagrange multiplier ρ > 0 : penalty parameter ADMM (primal variables are updated alternately): x k+1 minimizer of L ρ (x,z k ;y k ), (primal update-1) z k+1 minimizer of L ρ (x k+1,z;y k ), (primal update-2) y k+1 y k +ρ(ax k+1 + Bz k+1 c). (dual update)

12 iteration of ADMM target ( f, g : convex): Min. f (x)+g(z) s. t. Ax+Bz=c. augmented Lagrangian: L ρ (x,z;y)= f (x)+g(z) + y (Ax+Bz c)+ ρ 2 Ax+Bz c 2 y : Lagrange multiplier ρ > 0 : penalty parameter method of multipliers (= precursor of ADMM): (x k+1,z k+1 ) minimizer of L ρ (x,z;y k ), (primal update) y k+1 y k +ρ(ax k+1 + Bz k+1 c). (dual update)

13 our method (1/3) discrete variables only (for simple presentation): Min. f (x) s. t. g i (x) 0 (i= 1,..., m), ( write x G) x 1,..., x n : from a list. ( write x D)

14 our method (1/3) discrete variables only (for simple presentation): Min. f (x) s. t. g i (x) 0 (i= 1,..., m), ( write x G) x 1,..., x n : from a list. ( write x D) reformulation: Min. f (x) s. t. x=z ( -1) x G, z D. ( -2) augmented Lagrangian: L ρ (x,z;y)= f (x)+ y (x z)+ ρ 2 x z 2 w/ ( -2). scaling v := y/ρ (as usually done in ADMM): L ρ (x,z;v)= f (x)+ ρ 2 x z+v 2 ρ 2 v 2 w/ ( -2).

15 our method (2/3) We now have augmented Lagrangian: L ρ (x,z;v)= f (x)+ ρ 2 x z+v 2 ρ 2 v 2 with g(x) 0, z from a list. ADMM iteration: x k+1 minimizer of L ρ (x,z k ;v k ), (primal update-1) z k+1 minimizer of L ρ (x k+1,z;v k ), (primal update-2) v k+1 v k + x k+1 z k+1. (dual update)

16 our method (2/3) We now have augmented Lagrangian: L ρ (x,z;v)= f (x)+ ρ 2 x z+v 2 ρ 2 v 2 with g(x) 0, z from a list. primal update: x k+1 minimizer of L ρ (x,z k ;v k ), (primal update-1) z k+1 minimizer of L ρ (x k+1,z;v k ), (primal update-2) update of x : Min. f (x)+ ρ 2 x zk + v k 2 s. t. g(x) 0. apply nonlin. prog. approach only local optimality (but, sufficient for heuristic)

17 our method (2/3) We now have augmented Lagrangian: L ρ (x,z;v)= f (x)+ ρ 2 x z+v 2 ρ 2 v 2 with g(x) 0, z from a list. update of z : ρ Min. 2 (xk+1 + v k ) z 2 s. t. z from a list. rounding x k+1 + v k to the nearest point in the list : point in the list x k+1 j +v k j z k+1 j

18 our method (3/3) heuristic for nonconvex problems: not guaranteed to converge solution may depend on initial point & penalty parameter not guaranteed to be globally optimal but, often effective

19 numerical experiments Matlab implementation NLP solver: fmincon (Matlab built-in func.) multiple restart of ADMMM from 100 random initial points penalty parameter ρ = 100 ρ 10ρ if ADMM converges to an infeasible solution. 3 benchmark problems

20 ex.) 3-bar truss truss w/ discrete member cross-sections [Rao 96] Min. 2x 1 + x 2 + 2x 3 s. t. 3x x 3 1.5x 1 x 2 + 1, 2x 2 x x 1 x x x 3 1.5x 1 x 2 + 1, 2x 2 x x 1 x 3 0.5x 1 2x x 1 x 2 + 1, 2x 2 x x 1 x 3 x 1, x 2, x 3 {0.1, 0.2, 0.3, 0.5, 0.8, 1.0, 1.2}. (disc.) 3 disc. variables 4 nonlin. ineq. cstr.

21 ex.) 3-bar truss truss w/ discrete member cross-sections [Rao 96] x 1, x 2, x 3 {0.1, 0.2, 0.3, 0.5, 0.8, 1.0, 1.2} min. structural weight under stress cstr. x 1 x 2 x 3 proposed method finds global opt. x = (1.2, 0.5, 0.1) #trial #conv. #best #iter. time final ρ s 10 2

22 ex.) 10-bar truss truss w/ discrete member cross-sections [Cai & Thierauf, 93] min. structural weight Case D1: x i {1.62, 1.80, 1.99, 2.13,..., 33.50} under stress & displacement cstr. (42 candidates) Case D2: x i {0.100, 0.347, 0.440, 0.539,..., } (30 candidates)

23 ex.) 10-bar truss truss w/ discrete member cross-sections [Cai & Thierauf, 93] proposed method finds: obj = (case D1) obj = (case D2) [best known] [2nd-best known] case #trial #conv. #best #iter. time final ρ D s 10 3 D s 10 2 existing results in literature: case D1: obj= [Groenwold, Stander, & Snyman, 99], [Juang & Chang, 06] case D2: obj= [Thierauf & Cai, 98]

24 ex.) pressure vessel variables: 2 discrete & 2 continuous [Sandgren 90] Min x 1 x 3 x x 2 x x 1 2 x x 1 2 x 3 (nonconvex) s. t. πx 3 2 x πx (nonconvex) x x 3, x x 3 10 x 3 200, 10 x x 1, x 2 { , ,..., } (disc.) t R L s R

25 ex.) pressure vessel proposed method finds obj = #trial #conv. #best #iter. time final ρ s 10 3, 10 4 existing results of metaheuristics: obj [Kitayama, Arakawa, & Yamazaki 06] [He, Prempain, & Wu, 04] [Hsu, Sun, & Leu, 95] [Kannan & Kramer, 94] [Qian, Yu, & Zhou, 93] [Sandgren, 90] nd-best known solution was obtained.

26 conclusions heuristic for MINLP (mixed-integer nonlinear programming) based on ADMM (alternating direction method of multipliers) multiple restart from random initial points proposed ADMM no guarantee, but... simple, computationally cheap, and... each iteration consists of an NLP and a projection effective i.e., often finds a feasible solution w/ good objective value

Distributed Optimization via Alternating Direction Method of Multipliers

Distributed Optimization via Alternating Direction Method of Multipliers Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato Stanford University ITMANET, Stanford, January 2011 Outline precursors dual decomposition