Technical Report Doc ID: TR March-2013 (Last revision: 23-February-2016) On formulating quadratic functions in optimization models.

Transcription

1 Technical Repor Doc ID: TR March-203 (Las revision: 23-Februar-206) On formulaing quadraic funcions in opimizaion models. Auhor: Erling D. Andersen Convex quadraic consrains quie frequenl appear in opimizaion problems and hence i is imporan o know how o formulae hem efficienl. In his noe i is argued ha reformulaing hem o separable form is advanageous because i makes he convexi check rivial and usuall leads o a faser soluion ime requiring less sorage. The suggesed echnique is paricularl applicable in porfolio opimizaion where a facor model for he covariance marix is emploed. Moreover, we discuss how o formulae quadraic consrains using quadraic cones and he benefis of doing ha. Inroducion Consider he quadraic opimizaion problem where Q is smmeric 2 xt Qx + a T x + b 0 Q = Q T. Normall here are some srucure in he Q marix e.g. Q ma be a low rank marix or have a facor srucure. The problem () is said o be separable if Q is a diagonal marix. Subsequenl i is demonsraed how he problem () alwas can be reformulaed o have a separable srucure. For simplici i is assumed ha he problem () onl has one consrain. However, i should be obvious how o exend he suggesed echniques o a problem wih an arbirar number of consrains. Moreover, i is imporan o noe ha he echniques oulined are applicable o problems wih a quadraic objecive as well. Indeed he problem + 2 xt Qx (2) () is equivalen o which has he form (). + 2 xt Qx (3) 2 The convexi assumpion The problem () is an eas problem o solve if i is a convex problem. I is well known ha he problem () is convex if and onl if he quadraic funcion is convex. Furhermore, he following saemens are equivalen. f(x) = x T Qx (4) Page of 6

2 Funcion Sorage Operaional cos cos f 2 n2 2 n2 g np np Table : Sorage and evaluaion coss. i) f is convex. ii) Q posiive semidefinie. iii) There exis a marix H such ha Q = HH T. Observe using Q = HH T we have x T Qx = x T HH T x = H T x 2 0. Noe ha H is no unique in general, for insance H ma be he Cholesk facor or Q 2. Moreover, in pracice usuall he marix H is known and no Q and his precisel he reason wh i can be concluded Q posiive semidefinie. Since opimizaion sofware picall is no informed abou H bu is onl given Q hen he sofware checks convexi assumpion b compuing a Cholesk facorizaion of Q. Unforunael his is no a robus convexi check as he following example demonsraes: assume ha [ ] α Q =. α α This marix is b consrucion posiive semidefinie. Nex assume compuaions are done in finie precision using 3 digis of accurac and α = 5 so he problem is o check wheher [ 2.24 ] is posiive semidefinie. However, i is no posiive semidefinie! Hence, if rounding errors are presen as he are on on a compuer hen he rounded Q marix ma no posiive semidefinie. In pracice compuers emplos abou 6 figures of accurac so wrong conclusions abou he convexi does no appear ofen bu canno be ruled ou. The conclusion is ha a convexi check is no fool proof in pracice. Neverheless for he special case where he marix Q is a diagonal marix hen check is simple and fool proof, since he check consis of checking wheher all he diagonal elemens are nonnegaive. 3 Separable form is he winner (usuall) In he previous secion i was demonsraed ha if Q is a diagonal marix hen i is eas o check he convexi assumpion. The purpose of his secion is o demonsrae ha he problem () alwas can be made separable given he convexi assumpion. In addiion we demonsrae ha he reformulaion likel leads o a faser soluion ime. Firs define g(x) = H T x 2 (5) and clearl f(x) = g(x) = x T Qx holds. Therefore, we can use (5) insead of (4) if deemed worhwhile. Assuming ha H R n p and Q is a dense marix, hen Table 3 lis how much sorage ha is required o sore f and g on a compuer respecivel. Moreover, he able lis how man operaions ha is required o evaluae he wo funcions respecivel. B operaions we mean he number of basic arihmeic operaions like + ha is needed o evaluae he funcion. Page 2 of 6

3 Table 3 shows ha if p is much smaller han n, hen using he formulaion (5) saves a huge amoun of sorage and work. This observaion can be used o reformulae () o 2 T + a T x + b 0, H T x = 0. (6) The formulaion (6) is larger han () because p addiional linear equaliies and variables have been added. However, if p is much smaller han n, hen he formulaion (6) requires much less sorage which in mos cases leads o a much faser soluion imes. Before coninuing hen le us ake a look a an imporan special case. Assume a vecor v is known such ha Hv = a For insance if H is nonsingular, hen v exiss and can be compued as H a. I man pracical applicaions a v will be riviall known. Now if we le H T x = v hen Therefore, problem (6) and 2 = H T x + v 2 = x T HH T x + v T v + 2x T Hv = x T Qx + 2a T x + v T v 0.5 T 0.5v T v + b 0, H T x = v. (7) is equivalen. The reformulaion (7) is sparser han (6) in he consrain marix because he problem no longer conains a and ma herefore be preferable. Now le us consider a sligh generalizaion i.e. le us assume ha Q = D + HH T (8) where D is posiive semidefinie marix. This implies Q is posiive semidefinie. Furhermore, D is assumed o be simple e.g. a diagonal marix. Using he srucure in (8) hen () can be cas as 2 (xt Dx + T ) + a T x + b 0, H T x = 0. (9) In porfolio opimizaion arising in finance n ma be 000 and p is less han, sa 50. In ha case (9) will require abou 0 imes less sorage compared o () assuming Q is dense. This will ranslae ino dramaic faser soluion imes. 4 Conic reformulaion I is alwas possible and ofen worhwhile o reformulae convex quadraic and quadraicall consrained opimizaion problems on conic form. In his secion we will discuss ha possibili. We will use he definiions K q := {x R n x x 2:n } and K r := {x R n 2x x 2 x 3:n 2, x, x 2 0}. Hence, K q is he quadraic cone and K r is he roaed quadraic cone. Le us firs consider he conic reformulaion of (6) which is + a T x + b = 0, H T x = h, s =, s r. (0) Page 3 of 6

4 or more compacl Nex consider (7) which is infeasible if c T x + a T x + b = 0, H T x + h r. () 0.5v T v b < 0. Now assume ha is no he case hen (7) can be saed on he form or compacl H T x = v, = [ ] 0.5v T v b, (2) q [ c T x ] 0.5vT v b H T x + v q. Nex le us consider he problem (9) which can be reformulaed as a conic quadraic opimizaion problem as follows. Firs we scale he x variable b D 2 i.e. we replace x b D 2 x o obain (3) c T D 2 x 2 (xt x + T ) + a T x + b 0, H T D 2 x = 0. (4) which equivalen o he conic quadraic problem c T D 2 x + a T D 2 x + b = 0, H T D 2 x = 0, x r. (5) When he opimal soluion o (5) is compued hen original soluion can obained from x = D 2 x. 5 Linear leas squares problems wih inequaliies An example of generalized linear leas square problem is H T x + h Ax b. (6) Here we will discuss how o reformulae ha as a conic opimizaion problem. The problem (6) can be seen as a quadraic opimizaion problem because minimizing H T x + h or H T x + h 2 = x T HH T x + 2h T Hx + h T h is equivalen. Therefore, he problem (6) ma be saed as he quadraic opimizaion problem x T HH T x + 2h T Hx + h T h Ax b. (7) Page 4 of 6

5 The conic quadraic reformulaion of (6) is rivial because i is [ Ax ] b, H T x + h q. (8) whereas he conic reformulaion of (7) is + 2h T Hx + h T h Ax b, 0.5 H T x r. (9) Now he quesion is should (8) or (9) be preferred? Consider he following problem x n j= x j α, x 0. (20) For n = 0 and α = 0 4, hen MOSEK version 7 requires 22 ieraions o solve (9) whereas onl 6 ieraions are required o solve he formulaion (8) where he accurac of he repored soluion is abou he same in boh cases. If α is reduced o he wo mehods formulaion are equall good in erms of he number ieraions. This confirms our experience ha he formulaion (8) usuall leads o he bes resuls. The reason for his is migh be ha he norm is a nicer funcion han he squared norm in he sense ha he norm of somehing is closer o one han he squared norm. We herefore offer he advice ha a leas square objecive as in (6) is no convered o a quadraic problem which hen is convered o a conic problem. Raher i should direcl be saed on is naural conic quadraic form (8). 6 On he numerical benefis of a conic reformulaion In his secion we will demonsrae ha a conic reformulaion of quadraic consrain ofen leads o a beer scaling. Consider he quadraic consrains x T x 0 2 and The conic reformulaions are and [ x x T x 0 2. ] = 0 6, q [ ] = 0 6, x q. respecivel. Observe, he numbers appearing in he conic reformulaion is much closer o one and hence he problems are much beer scaled. Nex consider he quadraic consrain which has he conic quadraic reformulaion 0 2 x x x = 0 4, = 0 x, 2 = 0 3 x 2, [ ] 3 = 0 4 x 3, Page 5 of 6

6 Observe ha he conic reformulaion is beer scaled. Indeed he relaive difference beween he bigges and smalles number is reduced b 3 orders of magniude b doing he conic reformulaion. Finall, we ma use he subsiuion = 0 2 ȳ o obain = 0 2, ȳ = 0 x, ȳ 2 = 0 x 2, [ ȳ] 3 = 0 2 x 3, ȳ which improves he scaling furher. Clearl, he conic reformulaion is bigger because here are more variables and hence ma ake longer ime o solve. However, he addiional consrains and variables are ver sparse and normall his means onl slighl higher compuaional coss per ieraion. On oher he reformulaion ma lead o fewer so in man cases he soluion ime will be shorer afer reformulaion. 7 Conclusion In his noe we have showed ha when a quadraic funcion occur in an opimizaion problem hen here migh be differen was of represening hem. Moreover, given a suiable srucure in he quadraic erm hen a reformulaion o separable form ma lead o much more effecive represenaion. In addiion he reformulaion leads o a much simpler and fool proof convexi check. Finall, we have discussed how o formulae quadraic consrains using quadraic cones. In paricular we argued when a leas leas square objecive or leas squares pe consrains occur hen he should no be convered o quadraic form and hen convered o conic form. Raher he leas square erms should be represened naurall in conic framework wihou squaring he erm. Page 6 of 6

7 he fas pah o opimum MOSEK ApS provides opimizaion sofware which help our cliens o make beer decisions. Our cusomer base consiss of financial insiuions and companies, engineering and sofware vendors, among ohers. The compan was esablished in 997 b Erling D. Andersen and Knud D. Andersen and i specializes in creaing advanced sofware for soluion of mahemaical opimizaion problems. In paricular, he compan focuses on soluion of large-scale linear, quadraic, and conic opimizaion problems. Mosek ApS Fruebjergvej Copenhagen Denmark info@mosek.com