Convex Optimization in Electrical Engineering Stephen Boyd and Lieven Vandenberghe EE Dept., Stanford ISL EE370 Seminar, 2/9/96 1 Main idea many electrical engineering design problems can be cast as convex optimization problems such problems can appear very dicult, but can be solved very eciently by recently developed methods (unfortunately) convexity is often not recognized, hence not exploited 2
Example 1 linear elastic structure; forces f 1 ;:::;f 100 induce deections d 1 ;:::;d 300 0 f i F max i, several hundred other constraints: max load per oor, max wind load per side, etc. f 1 f 2 f 3 f 4 Problem 1a: nd worst-case deection, i.e., max i jd i j Problem 1b: nd worst-case deection, with each force \on" or \o", i.e., f i =0orF max i 3 Example 1 problem 1a is very easy readily solved in a few minutes on small workstation general problem has polynomial complexity problem 1b is very dicult could take weeks to solve... general problem NP-complete 4
Example 2 polytope described by linear inequalities, a T i x b i, i =1;:::;L a 1 0 Problem 2a: nd point closest to origin, i.e., min kxk Problem 2b: nd point farthest from origin, i.e., max kxk 5 Example 2 problem 2a is very easy readily solved on small workstation polynomial complexity problem 2b is very dicult dicult even with supercomputer NP-complete 6
moral: very dicult and very easy problems can look quite similar 7 Outline Convex optimization Examples Interior-point methods 8
Convex optimization minimize f 0 (x) subject to f 1 (x) 0;:::f L (x) 0 f i :R n!r are convex, i.e., for all x, y, 0 1, f i (x +(1,)y)f i (x) +(1,)f i (y) can have linear equality constraints dierentiability not needed examples: linear programs (LPs), problems 1a, 2a other formulations possible (feasibility, multicriterion) 9 (roughly speaking,) Convex optimization problems are fundamentally tractable computation time is small, grows gracefully with problem size and required accuracy large problems solved quickly in practice what \solve" means: { nd global optimum within a given tolerance, or, { nd proof (certicate) of infeasibility not widely enough appreciated 10
Outline Convex optimization Examples Interior-point methods 11 Well-known example: FIR lter design transfer function: H(z) = n X i=0 h i z,i design variables: x =[h 0 h 1 ::: h n ] T sample convex constraints: H(e j0 )=1(unity DC gain) H(e j! 0 )=0(notch at! 0) jh(e j! )j0:01 for! s! (min. 40dB atten. in stop band) jh(e j! )j1:12 for 0!! b (max. 1dB upper ripple in pass band) h i = h n,i (linear phase constraint) s(t) = t X i=0 h i 1:1H(e j0 ) (max. 10% step response overshoot) 12
h(t) FIR lter design example (M. Grant) 4 sample rate 2=n sec,1 linear phase max 1dB ripple up to 0:4Hz min 40dB atten above 0:8Hz minimize max i jh i j some solution times: n = 255: 5 sec n = 2047: 4 min H(e 2jf ) 10 3 2 1-1 -2-3 -1-0.8-0.6-0.4-0.2 0.2 0.4 0.6 0.8 1 10-1 10-2 10-3 t 10 10 1 10 2 f 13 14
Beamforming omnidirectional antenna elements at positions p 1 ;:::;p n 2 R 2 plane wave incident from angle : exp j(k() T p,!t); k() =,[cos sin ] T p 1 k() demodulate to get y i = exp(jk() T p i ) form weighted sum y() = n X i=1 w i y i design variables: x =[Re w T Im w T ] T (antenna array weights or shading coecients) G() = jy()j antenna gain pattern 15 Sample convex constraints: y( t )=1(target direction normalization) G( 0 )=0(null in direction 0 ) w is real (amplitude only shading) jw i j1(attenuation only shading) Sample convex objectives: max fg() jj, t j5 g (sidelobe level with 10 beamwidth) X 2 jw i j 2 (noise power in y) i 16
Open-loop trajectory planning discrete-time linear system, input u(t) 2 R p, output y(t) 2 R q sample convex constraints: ju i (t)ju (limit on input amplitude) ju i (t+1),u i (t)js(limit on input slew rate) l i (t) y i (t) u i (t) (envelope bounds for output) sample convex objective: max t;i jy i (t), y des i (t)j (peak tracking error) 17 Robust open-loop trajectory planning input must work well with multiple plants one input u applied to L plants; outputs are y (1) ;:::;y (L) constraints are to hold for all y (i) sample objectives: weighted sum of objectives for each i (average performance) max over objectives for each i (worst-case performance) 18
Example two plants 0 u(t) 1 ju(t)j1:25=sec minimize worst-case peak tracking error 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 Plant step responses 128 time samples (variables) 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 t u opt y 1 ; y 2 ; y des 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 t 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 t 19 VLSI transistor sizing subcircuit during transition contains Rs, Cs: `on' transistor resistances; drain, source to ground capacitances R 2 R 3 v in R 1 C 2 C 3 R 5 R 4 v 3 v high v in v i R 6 C 6 C 1 C 4 C 5 D i x i : size (width) of transistors, so total area is A(x) =f T x+g `on' conductance of transistor i: g i = x i capacitance at node j : c j = a T j x + b j sample problem: minimize area s.t. max delay, limits on widths 20
Approximation via dominant time constant circuit dynamics: C(x) dv dt =,G(x)v(t) conductance matrix G(x), capacitance matrix C(x) ane in x solutions have form X v(t) = i e it i eigenvalues 0 > 1 2 n given by det( i C(x) +G(x)) = 0 slowest (\dominant") time constant given by T dom =,1= 1 (related to delay) T dom T max () (,1=T max )C(x) +G(x)0 Problem: (convex, in fact, a semidenite program) minimize A(x) subject to (,1=T max )C(x) +G(x)0; x min i x i x max i 21 Transmitter power allocation m transmitters, mn receivers all at same frequency transmitter i wants to transmit to n receivers labeled (i; j), j =1;:::;n transmitter k receiver (i; j) transmitter i A ijk is path gain from transmitter k to receiver (i; j) N ij is (self) noise power of receiver (i; j) variables: transmitter powers p k, k =1;:::;m 22
signal power at receiver (i; j): S ij = A iji p i noise plus interference power at receiver (i; j): I ij = X k6=i A ijk p k + N ij signal to interference/noise ratio (SINR) at receiver (i; j): S ij =I ij Problem: choose p i to maximize smallest SINR: maximize min i;j X k6=i A iji p i A ijk p k + N ij subject to 0 p i p max... a (quasi-) convex problem (same methods work) 23 Other examples control system analysis and design array signal processing, e.g., broadband beamforming lter/controller realization experiment design & system identication structural optimization, e.g., truss design design centering & yield maximization statistical signal processing computational geometry 24
Outline Convex optimization Examples Interior-point methods 25 Interior-point convex programming methods history: Dikin; Fiacco & McCormick's SUMT (1960s) Karmarkar's LP algorithm (1984); many more since then Nesterov & Nemirovsky's general formulation (1989) general: # iterations small, grows slowly with problem size (typical number: 5,,50) each iteration is basically least-squares problem (hence can exploit problem structure via conjugate-gradients) 26
choose (potential fct.) ' s.t. ' smooth Basic idea (oversimplied) '(x)! +1 as x! feasible set boundary '(x)!,1as x! optimal minimize ' by (modied) Newton method each Newton step is (mostly) a least-squares problem if ' is properly chosen: algorithm is polynomial, extremely ecient in practice 27 Conclusions diverse EE design problems can be cast as convex optimization problems, hence eciently solved applications: numerical engines for CAD tools, embedded systems as available computing power increases, this observation becomes more relevant 28
How many practical problems are convex? many, but certainly not all, or even most myth # 1: very few practical problems are convex myth # 2: it's very hard to determine convexity in practice these are self-propagating 29... the great watershed in optimization isn't between linearity and nonlinearity, but convexity and nonconvexity. R. Rockafellar, SIAM Review 1993 30
(Pointers to) references where: anonymous ftp to isl.stanford.edu, in pub/boyd URL http://www-isl.stanford.edu/ boyd what: survey articles with many references code lecture notes for EE392: Introduction to Convex Optimization with Engineering Applications Course: EE364, Winter 1996-7 31