2012c12 $ Ê Æ Æ 116ò 14Ï Dec., 2012 Oerations Research Transactions Vol.16 No.4 Some results of convex rogramming comlexity LOU Ye 1,2 GAO Yuetian 1 Abstract Recently a number of aers were written that resent low-comlexity interior-oint methods for different classes of convex rograms. To guarantee the olynomiality of the rocedure, in this aer we show that the logarithmic barrier function associated with these rograms is self-concordant. In other words, we will resent two different lemmas with different logarithmic barrier functions and aly them to several classes of structured convex otimization roblems, using the self-concordancy. Keywords convex rogramming, self-concordant barrier functions, entroy rogramming, otimization comlexity Chinese Library Classification O221.2 2010 Mathematics Subject Classification 90C25 'u à5y KE,5ïÄ(J ð 1,2 U 1 Á 8c ulœþïäˆaøóà5y$e,ýæn¼ê { Ù. ^ gúnø éøóaaà5y KEƒAéêæN¼ê ÏLü Úny²ùà 5y KƒAéêæN¼êÑ vgú ŠâNesterov ÚNemirovskyóŠy² KS:Ž{äkõ ªE,5. ' c à5y gúæn¼ê 5y `ze,5 ã aò O221.2 êæ aò(2010) 90C25 0 Introduction Convex otimization deals with the following roblem (CL) inf x R n f 0 (x), s.t. x C, where C R n is a closed convex set and f 0 : C R is a convex function defined on C. ÂvFϵ2012c1120F 1. Deartment of Mathematics, Shanghai University, Shanghai 200444, China; þ ŒÆêÆX, þ, 200444 2. þ ÆEâ Æ þ, 201800; Shanghai Vocational College of Science and Technology, Shanghai 201800, China ÏÕŠö Corresonding author
4Ï Some results of convex rogramming comlexity 113 A fundamental ingredient in the elaboration of these methods is barrier method. Namely, consider the following arameterized family of unconstrained minimization roblem: (CL µ ) inf x R n f 0 (x) µ + F (x), where arameter µ belongs to R ++ and is called the barrier arameter. The constraint x C of the original roblem (CL) has been relaced by a enalty term F (x) in the objective function, which tends to + as x tends to the boundary of C and whose urose is to avoid that the iterates leave the feasible set. Assuming existence of a minimizer x(µ) for each of these roblems(strong convexity of F ensures uniqueness of such a minimizer x(µ)), the set {x(µ) µ > 0} C is called the central ath for the roblem (CL). It is intuitively clear that as µ tends to zero, the first term roortional to the original objective f0(x) µ becomes reonderant in the sum, which imlies that the central ath converges to a solution that is otimal for the original roblem. The rincile behind interioroint methods will thus be to follow this central ath until an iterate that is sufficiently close to the otimum is found. The efficiency of a barrier method for solving convex rograms strongly deends on the roerties of the barrier function used. A key roerty that is sufficient to rove olynomial convergence for barrier methods is the roerty of self-concordance introduced in [1]. This condition not only allows a roof of olynomial convergence, but numerical exeriments in [2-3] and others further indicate that numerical algorithms based on self-concordant barrier functions are of ractical interest and effectively exloit the structure of the underlying roblems. In this article we resent two different lemmas, if the barrier function satisfies the condition of corresonding lemma, the function is also self-concordant. We will show that for several classes of convex roblems for which interior-oint methods were resented in the literature the logarithmic barrier function is self-concordant. 1 Some general comosition rules Firstly we give a recise definition of self-concordance as given by [1]. Definition 1.1 A function F : C R is called (κ, ν)-self-concordant for the convex set C R n if and only if F is a barrier function and the following two conditions hold for all x intc and h R n : 3 F (x)[h, h, h] 2κ( 2 F (x)[h, h]) 3 2, (1.1) F (x) T ( 2 F (x)) 1 F (x) ν (1.2) (Note that the square root is well defined since its argument 2 F (x)[h, h] is ositive of the requirement that F is convex). Furthermore, we call a barrier function F (x) to be ν-logarithmically homogeneous if F (tx) = F (x) ν log t
114 LOU Ye, GAO Yuetian 16ò for all x F 0 and t > 0. Just as the definition of ordinary convexity, self-concordancy is a line-roerty, i.e., the definition of a self-concordant function can be restricted to any line lying in the domain. To see this, let Then, d(t) := F (x + th). d (1) (0) = F (x)[h], d (2) (0) = 2 F (x)[h, h], d (3) (0) = 3 F (x)[h, h, h]. Therefore, F (x) is a self-concordant function satisfying 3 F (x)[h, h, h] 2κ(h T 2 F (x)h) 3/2. if and only if it is a self-concordant function restricted to any line in its domain, i.e. d (3) (0) 2κ(d (2) (0)) 3/2, for any given x in its domain and any given feasible direction h. This observation allows us to rove the self-concordant roerty of a function by roving this roerty for the function restricted to an arbitrary line in its domain. We are now in osition to sketch a short-ste algorithm. Given a roblem of tye (CL), a barrier function F for C, an uer bound on the roximity measure τ > 0, a decrease arameter 0 < θ < 1 and an initial iterate x 0 such that δ(x 0, µ 0 ) < τ, we set k 0 and erform the following main loo: (1) µ k+1 µ k (1 θ); (2) x k+1 x k + n µk+1 (x k ); (3) k k + 1. The key is to choose arameters τ and θ such that δ(x k, µ k ) < τ imlies δ(x k+1, µ k+1 ) < τ in order to reserve roximity to the central ath. This crucial question is answered by the remarkable theory of self-concordant functions. The following theorem gives the final comlexity results in [11]. Theorem 1.1 Given a convex otimization roblem (CL), a (κ, ν)-self-concordant barrier F for C and an initial iterate x 0 such that δ(x 0, µ 0 ) < 1 13.42κ, one can find a solution with accuracy ε in (1.03 + 7.15κ ν) log 1.29µ 0κ ν. ε This theorem imlies that with the hel of self-concordancy of the barrier function F, short-ste interior-oint methods is low-comlexity. The next lemma gives some helful comosition rules for self-concordant functions. The roof follows immediately from the definition of self-concordance. Lemma 1.1 (Nesterov and Nemirovsky [1] )
4Ï Some results of convex rogramming comlexity 115 (1) (addition and scaling) Let ϕ i be κ i -self-concordant on F 0 i, i = 1, 2, and ρ 1, ρ 2 R + ; then ρ 1 ϕ 1 + ρ 2 ϕ 2 is κ-self-concordant on F 0 1 F 0 2, where κ = max{κ 1 / ρ 1, κ 2 / ρ 2 }. (2) (affine invariance) Let ϕ be κ-self-concordant on F 0 and let B(x) = Bx + b : R k R n be an affine maing such that B(R k ) F 0. Then ϕ(b( )) is κ-self-concordant on {x : B(x) F 0 }. 2 Alication I Now consider a standard convex rogramming roblem (CP) min c T x s.t. Ax = b, f i (x) 0, i = 1,, m, where f i (x) is smooth and convex, i = 1,, m. For simlicity, let m = 1 and f(x) = f 1 (x). Also let the decision variable now be and the roblem data as 0 c := 0 c x := q x R 1 R 1 R n, R 1 R 1 R n, b := 1 0 b R 1 R 1 R m (2.1) and Ā := 1 0 0 T 0 1 0 T 0 0 A R (m+2) (n+2). (2.2) Let K = cl{ x > 0, q f(x/) 0} R n+2, which is a closed cone. Ye [12] has roved that it is also convex. An equivalent formulation for (CP) is given by (CCP) min s.t. c T x Ā x = b, x K. Naturally, a 2-logarithmically homogeneous and convex barrier function for K is F ( x) = log log(q f(x/)).
116 LOU Ye, GAO Yuetian 16ò u It holds that K = cl s = v v > 0, u vf (s/v) 0 s and F ( s) = log v log(u vf (s/v)) is a 2-logarithmically homogeneous barrier function for K. The result above was roved by Zhang [13]. The function in this article is of the tye F ( x) = log log(q f(x/)). Now let us consider the following function in R 1 F () = log log g(), where g() = q f(x/) 0. Simly calculation shows that F (1) () = g(1) () g() F (2) () = g(2) () g() F (3) () = g(3) () g() 1, (2.3) + [ g (1) ] 2 () + 1 g() + 3g(2) ()g (1) () (g()) 2 2 2, (2.4) [ g (1) ] 3 () 2 g() 3. (2.5) The next lemma gives a sufficient condition for an objective function F to guarantee that combined with the logarithmic barrier function for the ositive orthant R n + of R is self-concordant. This lemma will hel to simlify self-concordance roofs in the sequel. Lemma 2.1 If there exists a k > 0 such that g() satisfies > 0, then F () is self-concordant. g (3) () k g(2) (), (2.6) Proof In that case g() = q f(x/) 0( > 0), because f is convex, we can derive that g (2) () = 1 3 x T 2 f(x/)x 0. The three terms on the right-hand side of (1.4) are nonnegative, i.e., the right-hand side can be abbreviated by F (2) () = a 2 + b 2 + c 2, with a 2 = g(2) () g(), b2 = [ ] g (1) 2 () g(), c 2 = 1. Obviously, 2 a F (2) () 1/2, b F (2) () 1/2, c F (2) () 1/2.
4Ï Some results of convex rogramming comlexity 117 So we can bound the right-hand side of (1.5) by F (3) () g (3) () g() + 3g (2) ()g (1) () [ (g()) 2 + g (1) ] 3 2 () g() + 2 3 k g (2) () g() + 3 g (2) () g (2) () g() g() + 2 g (2) 2 () g() + 2 1 = k c a 2 + 3a 2 b + 2 b 3 + 2 c 3 k F (2) () 3/2 + 3 F (2) () 3/2 + 2 F (2) () 3/2 + 2 F (2) () 3/2 = (7 + k) F (2) () 3/2. In articular, if f is a convex quadratic function, of course it satisfies g (3) () k g(2) (), F ( x) is self-concordant roved by Zhang in [13]. Now we consider the general formulation of (CP) where m 1. Similarly we have its reresentation (CCP) with m K = K i, where K i = cl{ x > 0, q f i (x/) 0} R n+2, i = 1,, m. The natural 2m-logarithmically homogeneous barrier function for K is F ( x) = m log m log(q f i (x/)). 3 The dual cone of K is m m K = cl(k1 Km) = cl s i = u i v i s i v i > 0, u i v i fi ( si v i ) 0, i = 1,, m and the dual barrier function for K is also 2m-logarithmically homogeneous, is given as follows m F ( s 1,, s m ) = [log v i + log(u i v i fi (s i /v i ))]. Extended entroy otimization The extended entroy rogramming roblem is defined as (EEP) min c T x + s.t. Ax = b, x 0. g i (x i ) Where A is an m n matrix and c and b are n-and m-dimensional vectors, resectively. Moreover, it is assumed that the scalar functions g i C 3 satisfy g (3) i (x i ) k i g (2) i (x i )/x i, i = 1,, n. In the case of entroy rogramming we have g i (x i ) = x i ln x i, for all i, and k i = 1.
118 LOU Ye, GAO Yuetian 16ò An equivalent formulation for (EEP) is given by min Z s.t. c T x + Ax = b, x 0. g i (x i ) Z 0, Consequently, we can get another equivalent formulation (CEEP) for (EEP) (CEEP) min s.t. Z Ā x = b, { ( x K = cl x c T > 0, q x n + ( xi ) ) } g i + Z 0, x 0. Where f(x) = c T x + n g i (x i ) Z and Ā, x, b are defined as (1.1), (1.2). Lemma 2.2 Suose that g (3) i (x i ) k i g (2) i (x i )/x i, i = 1,, n, let x = q x R n+2, then the logarithmic barrier function F ( x) = log(q f(x/)) log for the extended entroy rogramming roblem (CEEP) is self-concordant in the domain > 0 and q f(x/) > 0 where f(x) = c T x + n g i (x i ) Z. Proof Consider an arbitrary line in the domain of F ( x). If the comonent remains constant along this line, then by Lemma 1.2 the function F ( x) is self-concordant on the line. Let us consider the case where changes along the line. We may assume that the line is arameterized by, i.e. q = a 0 + b 0, x i = a i + b i (i = 1,, n), Z = a n+1 + b n+1 characterize the line, where serves as the arameter. We have that ( c T x g() = q = a 0 + b 0 + n ( xi ) ) g i + Z ( ai ) c i (a i + b i ) g i + b i + a n+1 + b n+1.
4Ï Some results of convex rogramming comlexity 119 Simly calculation shows that g (1) () = b 0 + b n+1 c i b i + [ ( ai ) g i + b i a ( i ai ) ] g(1) i + b i, g (2) a 2 ( i ai ) () = 3 g(2) i + b i, [ 3a g (3) 2 ( () = i ai ) ( 4 g(2) i + b i + a3 i ai ) ] 5 g(3) i + b i. Let u i = ai + b i > 0, we have ai = u i b i. Obviously the extended entroy rogramming roblem satisfies g (3) i (x i ) k i g (2) i (x i )/x i, i = 1,, n. Consequently, [ g (3) 3a 2 i () 4 g(2) i ( a i + b i) + a 3 i 5 g(3) i ( a ] i + b i) [ 3a 2 i = 4 g(2) i (u i ) + a 3 ] i 5 g(3) i (u i ) [ ] 3a 2 i 4 g(2) i (u i ) + k a 2 i i 4 (u i b i ) g(2) i (u i ) 1 ( 3 + k i(1 b ) i ) a 2 i u i 3 g(2) i (u i ), let k = max k i, M = max 1 bi u i, i = 1,...n, then the above exression can be abbreviated by g (3) () 3 + km a 2 i 3 g(2) i (u) = 3 + km g (2) (). Follow the Lemma 1.2, naturally F (3) (x) [7 + (3 + km)] F (2) (x) 3/2 = (10 + km) F (2) (x) 3/2. This shows that F ( x) is self-concordant in all directions within its domain. The desired result is roved. Similarly, we can use the result of Lemma 1.2 in dual geometric rogramming roblem, Monteiro and Adler s condition and other smoothes condition in [14]. Theorem 2.1 Consider convex otimization roblem (CP) where f i (x) are all convex and g i () = q f i (x/) satisfies the condition g (2) g i () k (2) i () i, i = 1,, n, then the barrier function for the formulation (CCP) F ( x) = n log log(q f i (x/)) is self-concordant with a comlexity value in the order of n. u i
120 LOU Ye, GAO Yuetian 16ò 3 Alication II This lemma we are going to resent deals with the first-concordancy condition. Let us first introduce two auxiliary function r 1 and r 2 : { } { } γ γ + 1 + 1/γ r 1 : R R : γ max 1, and r 2 : R R : γ max 1,. 3 2/γ 3 + 4/γ + 2/γ 2 Both of these functions are equal to 1 for γ 1 and strictly increasing for γ 1, with the asymtotic aroximations r 1 (γ) γ 3 and r 2 (γ) γ+1 3 when γ tends to +. Lemma 3.1 Let us suose F is a convex function with effective domain C R n + and that there exists a constant γ such that 3 F (x)[h, h, h] 3γ 2 F (x)[h, h] n h 2 i for all x intc, h R n. (3.1) We have that x 2 i F 1 : C R : x F (x) log x i satisfies the first condition of self-concordancy (1.2) with arameter κ 1 = r 1 (γ) on its domain C and F 2 : C R R : (x, u) log(u F (x)) log x i satisfies the first condition of self-concordancy (1.2) with arameter κ 2 = r 2 (γ) on its domain ei F = {(x, u) F (x) u} This lemma is roved in [11]. While the revious lemma is use to tackle the first condition of self-concordant (1.1), it does not say anything about of the second condition (1.2). The following corollary about the second barrier F 2 might rove useful in the resect. Corollary 3.1 Let F satisfy the assumtion of Lemma 3.1. Then the second barrier is (r 2 (γ), n + 1)-self-concordant. F 2 : C R R : (x, u) log(u F (x)) log x i Glineur in [11] has used the results of Lemma 3.1 and Corollary 3.2 to several classes of structured convex otimization roblems, such as extended entroy otimization, dual geometric otimization and l -norm otimization, which are shown to admit a self-concordant logarithmic barrier. In this article we will extend the alication to other convex rogramming. The dual l -rogramming roblem
4Ï Some results of convex rogramming comlexity 121 Let q i be such that 1/ i + 1/q i = 1, 1 i m, and let the rows of a matrix A be a i, i = 1,, m, and the rows of a matrix B be b k, k = 1,, r. Then, the dual of the l -rogramming roblem (PL ) is (DL ) inf c T y + d T z + A T y + B T z = η, z 0 r z k k=1 i I k (1/q i ) y i /z k qi, (If y i 0 and z k = 0, then z k y i /z k qi is defined as ). The above roblem is equivalent to inf c T y + d T z + s qi i z qi+1 m t i /q i, k t i, i I k, k = 1,, r, y s, y s, A T y + B T z = η, z 0, s 0. Similarly, the constraints s qi i z qi+1 k t i are relaced by the equivalent constraints t ρi i z ρi+1 k s i, where 0 < ρ i := 1/q i 1, and the redundant constraints s 0 are relaced by t 0. The new reformulated dual l -rogramming roblem becomes (DL ) inf c T y + d T z + m t i /q i, s i t ρi i z ρi+1 k, i I k, k = 1,, r, y s, y s, A T y + B T z = η, z 0, s 0. Note that the original roblem (DL ) has r inequalities, and the reformulated roblem (DL ))4m + r. We now have the following results. After doing some straightforward calculations to F (t, z) := t ρ z ρ+1, 0 < ρ < 1, there h T = (h 1, h 2 ), we obtain for the second-order term h T F (t, z)h =ρ(1 ρ)t ρ 3 z ρ 2 (tz 3 h 2 1 + t 3 zh 2 2 2t 2 z 2 h 1 h 2 ) =ρ(1 ρ)t ρ 3 z ρ 2 (zh 1 th 2 ) 2 tz,
122 LOU Ye, GAO Yuetian 16ò and for the third-order term 3 F (t, z)[h, h, h] =ρ(1 ρ)t ρ 3 z ρ 2 (ρ 2)z 3 h 3 1 (ρ + 1)t 3 h 3 2 3(ρ 1)tz 2 h 2 1h 2 + 3ρt 2 zh 1 h 3 2 =ρ(1 ρ)t ρ 3 z ρ 2 (zh 1 th 2 ) 2 (ρ 2)zh 1 (ρ + 1)th 2 ρ(1 ρ)(ρ + 1)t ρ 3 z ρ 2 (zh 1 th 2 ) 2 (z h 1 + t h 2 ). Finally we obtain 3 F (t, z)[h, h, h] 2(ρ + 1)h T h1 F (t, z)h 2 t 2 + h 2 2 z 2, where ρ i 1. This exlains log(t ρi i z ρi+1 k s 1 ) log z k log t i is (1, 1 + 3 1 2(ρi + 1))-self-concordant. Monteiro and Adler s condition Monteiro and Adler [15] considered minimization roblems with linear equality constraints and a searable convex objective function on the ositive orthant of R n. The objective function f(x) = g i (x i ) must satisfy the following condition. i There exist ositive numbers T and such that for all reals x > 0 and y > 0 and all i = 1,, n, we have y g (3) i (y) T max {( ) x ( } y, y x) g (2) i (x) substituting y = x in the above condition, it is easy to see that g i satisfies γ = T 3, i.e., that the logarithmic barrier function for such a roblem is (1, 1 + T 3 )-self-concordant. Scaled Lischitz condition Interior-oint methods are given and analyzed for roblems with linear equality constraints and convex objective function f(x) on the ositive orthant of R n +. The objective function has to satisfy the following scaled Lischitz condition: There exists M > 0, such that for any ω, 0 < ω < 1, X( f(x + x) f(x) 2 f(x) x) M x T 2 f(x) x (3.2) whenever x > 0 and X 1 x ω. (Here, is the Euclidean norm). This condition is also covered by the self-concordance condition if f is three times continuously differentiable in the interior of the feasible domain. More recisely we can obtain that the corresonding logarithmic barrier function is (1, 1 + 2 3 M)-self-concordant. Since f C 3, we may exand f as follows: f(x + x) = f(x) + 2 f(x) x + 1 2 3 f(x)[ x, x, ] + o( x 2 ),
4Ï Some results of convex rogramming comlexity 123 where f(x)[ x, x, ] is a vector whose ith comonent is equal to j,k 3 f(x) x i x j x k x j x k. Relacing x by λ x in (3.2), inserting the above exansion, dividing by λ 2, and taking the limit λ 0, we obtain X 3 f(x)[ x, x, ] 2M x T 2 f(x) x. Considering X 3 f(x)[ x, x, ] as a column vector, we may continue X 3 f(x)[ x, x, ] (X 1 x) T X 1 x X 3 f(x)[ x, x, ] = 3 f(x)[ x, x, x] X 1 x and obtain that 3 f(x)[ x, x, x] 2M X 1 x x T 2 f(x) x. Let h = x, and notice that = X 1 x, we obtain that the logarithmic barrier function for such a roblem is (1, 1 + 2 3 M)-self-concordant. h 2 i x 2 i 4 Concluding remarks Before we conclude this work, we would like to briefly oint out a class of roblems considered in [16] which does not have a self-concordant logarithmic barrier function. But for most alications however, we believe that the self-concordance condition is more ractical. In this article we resent two different conditions, and we have the conclusion that if the barrier function satisfies the condition, it is also self-concordant. This work rovides us a ath to study convex rogramming roblems, and needed us to study more in the future work. References [1] Nesterov Y E, Nemirovsky A S. Interior oint olynomial algorithms in convex rogramming [J]. SIAM Studies in Alied Mathematics, 1994, 13. [2] Alizadeh E. Otimization over the ositive definite cone: interior-oint methods and combinatorial alications [M]// Advances in Otimization and Parallel Comuting, New York: Elsevier Science Inc, 1992. [3] Lustig I J, Mal sten R E, Shanno D E. On imlementing Mehrotra s redictor-corrector interior oint method for linear rogramming [J]. SIAM Journal on Otimization, 1992, 2: 435-449. [4] Han C G, Pardalos E M, Ye Y. On interior-oint algorithms for some entroy otimization roblems [R]. Technical Reort CS 91-02, Comuter Science Deartment, Pennsylvania State University, Pennsylvania: University Park, PA, 1991.
124 LOU Ye, GAO Yuetian 16ò [5] Kortanek K O, No H. A second order affine scaling algorithm for the geometric rogramming dual with logarithmic barrier [J]. Otimization, 1990, 23: 501-507. [6] Peterson E L, Ecker J G. Geometric rogramming: duality in quadratic rogramming and i aroximation I [C]// Proceedings of the International Symosium of Mathematical Programming. Princeton: Princeton University Press, 1970, 445-479. [7] Peterson E L, Ecker J G. Geometric rogramming: duality in quadratic rogramming and i aroximation II [J]. SIAM Journal on Alied Mathematics, 1969, 17: 317-340. [8] Peterson E L, Eeker J G. Geometric rogramming: duality in quadratic rogramming and i aroximation III [J]. Journal of Mathematical Analysis and Alications, 1970, 29: 365-383. [9] Kortanek K O, Zhu J. A olynomial barrier algorithm for linearly constrained convex rogramming roblems [J]. Mathematics of Oerations Researeh, 1993, 18: 116-128. [10] Zhu J. A ath following algorithm for a class of convex rogramming roblems [J]. Zeitschrift für Oerations Research, 1992, 36(4): 359-377. [11] Glineur F. Toics in convex otimization: interior-oint methods, conic deality and aroximations [D]. Belgium: U Mons, 2001. [12] Ye Y. Interior algorithms for linear quadratic and linearly constrained convex rogramming [D]. San Francisco: Stanford University, 1987. [13] Zhang S. A new self-dual embedding method for convex rogramming [J]. Journal of Global Otimization, 2004, 29: 479-496. [14] Gao Yuetian, Wu Donghua. A new self-concordant function method for some classes of structured convex rogramming roblems [J]. Alied Mathematical Sciences. [15] Monteiro R D C, Adler I. An extension of Karmarkar tye algorithms to a class of convex searable rogramming roblems with global rate of convergence [J]. Mathematics of Oerations Research, 1989, 15(3): 408-422. [16] Mehrotra S, Sun J. An interior oint algoritbm for solving smooth convex rograms based on Newton s method [J]. Contemorary Mathematics, 1990, 114: 265-284.