Nonlinear Programming Models: A Survey

Transcription

1 Nonlinear Programming Models: A Survey Robert Vanderbei and David Shanno NJ-INFORMS Nov 16, 2000 Rutgers University Operations Research and Financial Engineering, Princeton University rvdb

2 1 Outline Convex Problems Portfolio Optimization Structural Optimization Digital Audio Filters Antenna-Array Design Nonconvex Problems Putting on an Uneven Green Range Maximization for a Hang Glider Goddard Rocket Problem

3 1 Outline Convex Problems Portfolio Optimization Structural Optimization Digital Audio Filters Antenna-Array Design Nonconvex Problems Putting on an Uneven Green Range Maximization for a Hang Glider Goddard Rocket Problem

4 1 Outline Convex Problems Portfolio Optimization Structural Optimization Digital Audio Filters Antenna-Array Design Nonconvex Problems Putting on an Uneven Green Range Maximization for a Hang Glider Goddard Rocket Problem

5 1 Outline Convex Problems Portfolio Optimization Structural Optimization Digital Audio Filters Antenna-Array Design Nonconvex Problems Putting on an Uneven Green Range Maximization for a Hang Glider Goddard Rocket Problem

6 1 Outline Convex Problems Portfolio Optimization Structural Optimization Digital Audio Filters Antenna-Array Design Nonconvex Problems Putting on an Uneven Green Range Maximization for a Hang Glider Goddard Rocket Problem

7 1 Outline Convex Problems Portfolio Optimization Structural Optimization Digital Audio Filters Antenna-Array Design Nonconvex Problems Putting on an Uneven Green Range Maximization for a Hang Glider Goddard Rocket Problem

8 1 Outline Convex Problems Portfolio Optimization Structural Optimization Digital Audio Filters Antenna-Array Design Nonconvex Problems Putting on an Uneven Green Range Maximization for a Hang Glider Goddard Rocket Problem

9 1 Outline Convex Problems Portfolio Optimization Structural Optimization Digital Audio Filters Antenna-Array Design Nonconvex Problems Putting on an Uneven Green Range Maximization for a Hang Glider Goddard Rocket Problem

10 1 Outline Convex Problems Portfolio Optimization Structural Optimization Digital Audio Filters Antenna-Array Design Nonconvex Problems Putting on an Uneven Green Range Maximization for a Hang Glider Goddard Rocket Problem

11 1 Outline Convex Problems Portfolio Optimization Structural Optimization Digital Audio Filters Antenna-Array Design Nonconvex Problems Putting on an Uneven Green Range Maximization for a Hang Glider Goddard Rocket Problem

12 2 Convex vs. Nonconvex NLPs Nonlinear Programming (NLP) minimize f(x) subject to h i (x) = 0, i E, h i (x) 0, i I. NLP is convex if h i s in equality constraints are affine; h i s in inequality constraints are concave; f is convex; NLP is smooth if All are twice continuously differentiable.

13 2 Convex vs. Nonconvex NLPs Nonlinear Programming (NLP) minimize f(x) subject to h i (x) = 0, i E, h i (x) 0, i I. NLP is convex if h i s in equality constraints are affine; h i s in inequality constraints are concave; f is convex; NLP is smooth if All are twice continuously differentiable.

14 2 Convex vs. Nonconvex NLPs Nonlinear Programming (NLP) minimize f(x) subject to h i (x) = 0, i E, h i (x) 0, i I. NLP is convex if h i s in equality constraints are affine; h i s in inequality constraints are concave; f is convex; NLP is smooth if All are twice continuously differentiable.

15 3 Portfolio Optimization Start with historical data: Year US US S&P Wilshire NASDAQ Lehman EAFE Gold 3-Month Gov Composite Bros. T-Bills Long Corp. Bonds Bonds Notation: R j (t) = return on investment j in time period t.

16 4 Portfolio Optimization Continued maximize Eu j x j R j 1 T T u j x j R j (t) t=1 subject to j x j = 1 x j 0 for all investments j where u(x) = 1 1 γ x1 γ γ 1 log(x) γ = 1 time (secs) LOQO 0.11 LANCELOT 0.12 SNOPT 0.12

17 4 Portfolio Optimization Continued maximize Eu j x j R j 1 T T u j x j R j (t) t=1 subject to j x j = 1 x j 0 for all investments j where u(x) = 1 1 γ x1 γ γ 1 log(x) γ = 1 time (secs) LOQO 0.11 LANCELOT 0.12 SNOPT 0.12

18 4 Portfolio Optimization Continued maximize Eu j x j R j 1 T T u j x j R j (t) t=1 subject to j x j = 1 x j 0 for all investments j where u(x) = 1 1 γ x1 γ γ 1 log(x) γ = 1 time (secs) LOQO 0.11 LANCELOT 0.12 SNOPT 0.12

19 5 Portfolio Optimization Continued γ Gold US Lehman NASDAQ S&P EAFE Mean Std. 3-Month Bros. Composite 500 Dev. T-Bills Corp. Bonds

20 6 Portfolio Optimization Efficient Frontier Plot of risk (i.e., std. dev.) vs. reward (i.e., mean return): EAFE µ=0 NASDAQ Composite Long Bonds Wilshire 5000 S&P 500 µ=0.1 µ= Corp. Bonds µ=4 µ=2 4 T Bills 2 Gold µ= µ=

21 7 Structural Design Given: A region of space in which to build something. Thing is essentially planar but with varying thickness. A place (or places) where the thing will be anchored. A place (or places) where loads will be applied. A certain total amount of material out of which to build the thing. Objective: Design the thing to be as strong as possible. Approach: Partition 2-D region into finite elements. Assign a thickness to each element.

22 7 Structural Design Given: A region of space in which to build something. Thing is essentially planar but with varying thickness. A place (or places) where the thing will be anchored. A place (or places) where loads will be applied. A certain total amount of material out of which to build the thing. Objective: Design the thing to be as strong as possible. Approach: Partition 2-D region into finite elements. Assign a thickness to each element.

23 7 Structural Design Given: A region of space in which to build something. Thing is essentially planar but with varying thickness. A place (or places) where the thing will be anchored. A place (or places) where loads will be applied. A certain total amount of material out of which to build the thing. Objective: Design the thing to be as strong as possible. Approach: Partition 2-D region into finite elements. Assign a thickness to each element.

24 7 Structural Design Given: A region of space in which to build something. Thing is essentially planar but with varying thickness. A place (or places) where the thing will be anchored. A place (or places) where loads will be applied. A certain total amount of material out of which to build the thing. Objective: Design the thing to be as strong as possible. Approach: Partition 2-D region into finite elements. Assign a thickness to each element.

25 8 Structural Design Continued minimize p T w where subject to V A e w T K e w 1, e E p = applied load w = node displacements; optimization vars V = total volume A e = thickness of element e K e = element stiffness matrix ( 0) E = set of elements

26 9 Specific Example: Michel Bracket element grid 40x72 20x36 5x9 constraints variables time (secs) LOQO MINOS (IL) (BS) LANCELOT SNOPT - (IS) (BS)

27 10 Solution

28 11 Finite Impulse Response (FIR) Filter Design Audio is stored digitally in a computer as a stream of short integers: u k, k Z. When the music is played, these integers are used to drive the displacement of the speaker from its resting position. For CD quality sound, short integers get played per second per channel

29 12 FIR Filter Design Continued A finite impulse response (FIR) filter takes as input a digital signal and convolves this signal with a finite set of fixed numbers h n,..., h n to produce a filtered output signal: y k = n i= n h iu k i. Sparing the details, the output power at frequency ν is given by H(ν) where H(ν) = n k= n h ke 2πikν, Similarly, the mean squared deviation from a flat frequency response over a frequency range, say L [0, 1], is given by 1 L L H(ν) 1 2 dν

30 13 Filter Design: Woofer, Midrange, Tweeter minimize ρ subject to 1 0 ( Hw (ν) + H m (ν) + H t (ν) 1 ) 2 dν ɛ ( 1 W ( 1 M W M H 2 w (ν)dν ) 1/2 ρ W = [.2,.8] H 2 m (ν)dν ) 1/2 ρ M = [.4,.6] [.9,.1] ( 1 T T H 2 t (ν)dν ) 1/2 ρ T = [.7,.3] where H i (ν) = h i (0) + 2 n 1 k=1 h i(k) cos(2πkν), i = W, M, T h i (k) = filter coefficients, i.e., decision variables

31 14 Specific Example filter length: n = 14 integral discretization: N = 1000 constraints 4 variables 43 time (secs) LOQO 79 MINOS 164 LANCELOT 3401 SNOPT 35 Ref: J.O. Coleman, U.S. Naval Research Laboratory, CISS98 paper available: engr.umbc.edu/ jeffc/pubs/abstracts/ciss98.html Click here for demo

32 14 Specific Example filter length: n = 14 integral discretization: N = 1000 constraints 4 variables 43 time (secs) LOQO 79 MINOS 164 LANCELOT 3401 SNOPT 35 Ref: J.O. Coleman, U.S. Naval Research Laboratory, CISS98 paper available: engr.umbc.edu/ jeffc/pubs/abstracts/ciss98.html Click here for demo

33 15 Antenna Array Design Given: a 2-D array of radar antennae. An incoming signal produces a signal at each antenna. A linear combination of the signals is made to produce one output signal. Coefficients of the linear combination can be chosen to accentuate and/or attenuate the output signal s strength as a function of the input signal s source direction. Similar to FIR filter design. The set of antennae is analogous to the set of time delays in FIR filter design. The direction of the input signal is analogous to frequency in FIR filter design.

34 16 2-D Antenna-Array Design Problem minimize ρ subject to A(p) 2 ρ, p S A(p 0 ) = 1, where A(p) = w l e 2πip l, p S l {array elements} w l = complex-valued design weight for array element l S = subset of unit hemisphere: sidelobe directions p 0 = look direction

35 17 Specific Example: Hexagonal Lattice of 61 Elements ρ = 20 db = 0.01 S = 889 points outside 20 from look direction p 0 = 40 from zenith constraints 839 variables 123 time (secs) LOQO 722 MINOS > LANCELOT SNOPT

36 18 Solution

37 19 Putting on an Uneven Green Given: z(x, y) elevation of the green. Starting position of the ball (x 0, y 0 ). Position of hole (x f, y f ). Coefficient of friction µ. Find: initial velocity vector so that ball will roll to the hole and arrive with minimal speed. Variables: u(t) = (x(t), y(t), z(t)) position as a function of time t. v(t) = (v x (t), v y (t), v z (t)) velocity. a(t) = (a x (t), a y (t), a z (t)) acceleration. T time at which ball arrives at hole.

38 19 Putting on an Uneven Green Given: z(x, y) elevation of the green. Starting position of the ball (x 0, y 0 ). Position of hole (x f, y f ). Coefficient of friction µ. Find: initial velocity vector so that ball will roll to the hole and arrive with minimal speed. Variables: u(t) = (x(t), y(t), z(t)) position as a function of time t. v(t) = (v x (t), v y (t), v z (t)) velocity. a(t) = (a x (t), a y (t), a z (t)) acceleration. T time at which ball arrives at hole.

39 19 Putting on an Uneven Green Given: z(x, y) elevation of the green. Starting position of the ball (x 0, y 0 ). Position of hole (x f, y f ). Coefficient of friction µ. Find: initial velocity vector so that ball will roll to the hole and arrive with minimal speed. Variables: u(t) = (x(t), y(t), z(t)) position as a function of time t. v(t) = (v x (t), v y (t), v z (t)) velocity. a(t) = (a x (t), a y (t), a z (t)) acceleration. T time at which ball arrives at hole.

40 19 Putting on an Uneven Green Given: z(x, y) elevation of the green. Starting position of the ball (x 0, y 0 ). Position of hole (x f, y f ). Coefficient of friction µ. Find: initial velocity vector so that ball will roll to the hole and arrive with minimal speed. Variables: u(t) = (x(t), y(t), z(t)) position as a function of time t. v(t) = (v x (t), v y (t), v z (t)) velocity. a(t) = (a x (t), a y (t), a z (t)) acceleration. T time at which ball arrives at hole.

41 20 Putting Two Approaches Problem can be formulated with two decision variables: v x (0) and v y (0) and two constraints: x(t ) = x f and y(t ) = y f. In this case, x(t ), y(t ), and the objective function are complicated functions of the two variables that can only be computed by integrating the appropriate differential equation. A discretization of the complete trajectory (including position, velocity, and acceleration) can be taken as variables and the physical laws encoded in the differential equation can be written as constraints. To implement the first approach, one would need an ode integrator that provides, in addition to the quantities being sought, first and possibly second derivatives of those quantities with respect to the decision variables. The modern trend is to follow the second approach.

42 20 Putting Two Approaches Problem can be formulated with two decision variables: v x (0) and v y (0) and two constraints: x(t ) = x f and y(t ) = y f. In this case, x(t ), y(t ), and the objective function are complicated functions of the two variables that can only be computed by integrating the appropriate differential equation. A discretization of the complete trajectory (including position, velocity, and acceleration) can be taken as variables and the physical laws encoded in the differential equation can be written as constraints. To implement the first approach, one would need an ode integrator that provides, in addition to the quantities being sought, first and possibly second derivatives of those quantities with respect to the decision variables. The modern trend is to follow the second approach.

43 20 Putting Two Approaches Problem can be formulated with two decision variables: v x (0) and v y (0) and two constraints: x(t ) = x f and y(t ) = y f. In this case, x(t ), y(t ), and the objective function are complicated functions of the two variables that can only be computed by integrating the appropriate differential equation. A discretization of the complete trajectory (including position, velocity, and acceleration) can be taken as variables and the physical laws encoded in the differential equation can be written as constraints. To implement the first approach, one would need an ode integrator that provides, in addition to the quantities being sought, first and possibly second derivatives of those quantities with respect to the decision variables. The modern trend is to follow the second approach.

44 20 Putting Two Approaches Problem can be formulated with two decision variables: v x (0) and v y (0) and two constraints: x(t ) = x f and y(t ) = y f. In this case, x(t ), y(t ), and the objective function are complicated functions of the two variables that can only be computed by integrating the appropriate differential equation. A discretization of the complete trajectory (including position, velocity, and acceleration) can be taken as variables and the physical laws encoded in the differential equation can be written as constraints. To implement the first approach, one would need an ode integrator that provides, in addition to the quantities being sought, first and possibly second derivatives of those quantities with respect to the decision variables. The modern trend is to follow the second approach.

45 20 Putting Two Approaches Problem can be formulated with two decision variables: v x (0) and v y (0) and two constraints: x(t ) = x f and y(t ) = y f. In this case, x(t ), y(t ), and the objective function are complicated functions of the two variables that can only be computed by integrating the appropriate differential equation. A discretization of the complete trajectory (including position, velocity, and acceleration) can be taken as variables and the physical laws encoded in the differential equation can be written as constraints. To implement the first approach, one would need an ode integrator that provides, in addition to the quantities being sought, first and possibly second derivatives of those quantities with respect to the decision variables. The modern trend is to follow the second approach.

46 21 Putting Continued Objective: minimize v x (T ) 2 + v y (T ) 2. Constraints: v = u a = v ma = N + F mge z u(0) = u 0 u(t ) = u f, where m is the mass of the golf ball. g is the acceleration due to gravity. e z is a unit vector in the positive z direction. and...

47 21 Putting Continued Objective: minimize v x (T ) 2 + v y (T ) 2. Constraints: v = u a = v ma = N + F mge z u(0) = u 0 u(t ) = u f, where m is the mass of the golf ball. g is the acceleration due to gravity. e z is a unit vector in the positive z direction. and...

48 21 Putting Continued Objective: minimize v x (T ) 2 + v y (T ) 2. Constraints: v = u a = v ma = N + F mge z u(0) = u 0 u(t ) = u f, where m is the mass of the golf ball. g is the acceleration due to gravity. e z is a unit vector in the positive z direction. and...

49 21 Putting Continued Objective: minimize v x (T ) 2 + v y (T ) 2. Constraints: v = u a = v ma = N + F mge z u(0) = u 0 u(t ) = u f, where m is the mass of the golf ball. g is the acceleration due to gravity. e z is a unit vector in the positive z direction. and...

50 22 Putting Continued N = (N x, N y, N z ) is the normal force: N z = m g a x(t) z x a y(t) z y + a z(t) ( z x )2 + ( z y )2 + 1 N x = z x N z N y = z y N z. F is the force due to friction: F = µ N v v.

51 22 Putting Continued N = (N x, N y, N z ) is the normal force: N z = m g a x(t) z x a y(t) z y + a z(t) ( z x )2 + ( z y )2 + 1 N x = z x N z N y = z y N z. F is the force due to friction: F = µ N v v.

52 22 Putting Continued N = (N x, N y, N z ) is the normal force: N z = m g a x(t) z x a y(t) z y + a z(t) ( z x )2 + ( z y )2 + 1 N x = z x N z N y = z y N z. F is the force due to friction: F = µ N v v.

53 23 Putting Specific Example Discretize continuous time into n = 200 discrete time points. Use finite differences to approximate the derivatives. constraints 597 variables 399 time (secs) LOQO 14.1 LANCELOT > SNOPT 4.1

54 24 Hang Gliding Range Maximization maximize x(t ) subject to v x = ẋ, v x = 1 m ( L v y u a (x) v r v y = ẏ, v y = 1 m (L v x vr 0 c L c L max D v x vr ), D v y u a (x) v r ) g where v r = v 2 x + (v y u a (x)) 2, L = 1 2 c LρSv 2 r, D = 1 2 c D(c L )ρsvr 2 c D (c L ) = c 0 + kc 2 L c L = lift coefficient = control variable u a (x) = downrange updraft profile (see next slide) and m, S, ρ, g, c L max, c 0, and k are constants. Everything else varies with time t.

55 24 Hang Gliding Range Maximization maximize x(t ) subject to v x = ẋ, v x = 1 m ( L v y u a (x) v r v y = ẏ, v y = 1 m (L v x vr 0 c L c L max D v x vr ), D v y u a (x) v r ) g where v r = v 2 x + (v y u a (x)) 2, L = 1 2 c LρSv 2 r, D = 1 2 c D(c L )ρsvr 2 c D (c L ) = c 0 + kc 2 L c L = lift coefficient = control variable u a (x) = downrange updraft profile (see next slide) and m, S, ρ, g, c L max, c 0, and k are constants. Everything else varies with time t.

56 24 Hang Gliding Range Maximization maximize x(t ) subject to v x = ẋ, v x = 1 m ( L v y u a (x) v r v y = ẏ, v y = 1 m (L v x vr 0 c L c L max D v x vr ), D v y u a (x) v r ) g where v r = v 2 x + (v y u a (x)) 2, L = 1 2 c LρSv 2 r, D = 1 2 c D(c L )ρsvr 2 c D (c L ) = c 0 + kc 2 L c L = lift coefficient = control variable u a (x) = downrange updraft profile (see next slide) and m, S, ρ, g, c L max, c 0, and k are constants. Everything else varies with time t.

57 25 Hang Gliding Updraft Profile *exp(-(x/ )**2)*(1-(x/ )**2)

58 26 Hang Gliding Optimal Trajectory 1000 "y_vs_x" "x_vs_t" "vx_vs_t" Figure 1: y vs x Figure 3: x vs t Figure 5: v x vs t 1.4 "cl_vs_t" "y_vs_t" "vy_vs_t" Figure 2: cl vs t Figure 4: y vs t Figure 6: v y vs t

59 27 Hang Gliding Solver Comparison constraints 600 variables 449 time (secs) LOQO 10.7 LANCELOT > 3600 SNOPT 244.1

60 28 Goddard Rocket Problem Objective: maximize h(t ); Constraints: v = ḣ a = v θ = cṁ ma = (θ σv 2 e h/h 0 ) gm 0 θ θ max m m min h(0) = 0 v(0) = 0 m(0) = 3 where θ = Thrust, m = mass θ max, g, σ, c, and h 0 are given constants h, v, a, T h, and m are functions of time 0 t T.

61 28 Goddard Rocket Problem Objective: maximize h(t ); Constraints: v = ḣ a = v θ = cṁ ma = (θ σv 2 e h/h 0 ) gm 0 θ θ max m m min h(0) = 0 v(0) = 0 m(0) = 3 where θ = Thrust, m = mass θ max, g, σ, c, and h 0 are given constants h, v, a, T h, and m are functions of time 0 t T.

62 29 Goddard Rocket Problem Solution constraints 399 variables 599 time (secs) LOQO 5.2 LANCELOT SNOPT (IL) (IL)