CORE DISCUSSION PAPER 2006/30 Constructing self-concordant barriers for convex cones Yu. Nesterov March 21, 2006 Abstract In this paper we develop a technique for constructing self-concordant barriers for convex cones. We start from a simple proof for a variant of standard result [1] on transformation of a ν-self-concordant barrier for a set into a self-concordant barrier for its conic hull with parameter 3.08 ν + 3.57) 2. Further, we develop a convenient composition theorem for constructing barriers directly for convex cones. In particular, we can construct now good barriers for several interesting cones obtained as a conic hull of epigraph of a univariate function. This technique works for power functions, entropy, logarithm and exponent function, etc. It provides a background for development of polynomial-time methods for separable optimization problems. Thus, our abilities in constructing good barriers for convex sets and cones become now identical. Keywords: primal-dual conic optimization problem, self-concordant barriers, interiorpoint methods, barrier calculus. Center for Operations Research and Econometrics CORE), Catholic University of Louvain UCL), 34 voie du Roman Pays, 1348 Louvain-la-Neuve, Belgium; e-mail: nesterov@core.ucl.ac.be. The research results presented in this paper have been supported by a grant Action de recherche concertè ARC 04/09-315 from the Direction de la recherche scientifique - Communautè française de Belgique. The scientific responsibility rests with its author.
1 Introduction Motivation. In the last years, the theory of Interior-Point Methods was developing mainly for symmetric cones. At some moment, one could think that for general convex cones it may be too costly to implement the principles of symmetric duality up to the level of particular applications see [4] for discussion). And indeed, some of existing results are quite pessimistic. The general barrier calculus was developed in [8] only for convex sets. In Section 5.1.1 [8] there was analyzed a transformation of convex set into a cone and corresponding transformation of self-concordant barrier. It was shown that this transformation leads to multiplication of the parameter of the barrier by an absolute factor κ. However, the proposed value was κ = 800. Later, in [1] this factor was significantly reduced, especially for barriers with a large value of the parameter. However, it seems that this improvement is not well known. At least, up to now we can see publications dealing with the old version see, for example, [13]). The absence of acceptable barriers significantly decreased the research activity in the development of numerical schemes. We can mention only few theoretical developments related to this topic [5], [10], [11], [12]). However, in all cases the proposed schemes are based on an extensive usage of both primal and dual barriers. Since we have difficulties even with the primal one, such level of demand may look excessive. Recently, the situation started to change. In [7] there was developed a non-symmetric approach to primal-dual conic problems. It was suggested to run the main correction) process in the primal space. When the point becomes close to the primal central path, we stop and apply a special lifting procedure, which generates a strictly feasible primaldual pair linked by an exact scaling relation. Hence, it is possible to ine a generalized affine-scaling direction exactly in the same way as for self-scaled barriers. This direction can be used for prediction step along the primal-dual central path. In order to compute an appropriate step length, we need to use the dual barrier, etc. For self-scaled cones, this approach leads to well known search directions. At the same time, for general cones it looks quite promising since we can hope that for particular problem instances the local structure of the central path may be not too bad. In this paper we are going to support the progress in optimization technique by an improvement of our abilities of constructing barriers for convex cones. In [7] there was also proposed a simple 4n-self-concordant barrier for the epigraph of n-dimensional p-norm, 1 p. In this paper we show that similar technique can be applied in a much more general framework. As a result, we can get good barriers for conic hulls of epigraphs of different functions of one variable arising in Separable Optimization. Our approach can be seen as a modification of the technique developed in Section 5.1.2B, [8], towards the needs of conic optimization. We also present a simple proof for a variant of standard result [1] on automatic generation of self-concordant barrier for conic hull from a self-concordant barrier for convex set. We argue that this operation must be applied only to the barrier of full feasible set, when the value of the parameter is already big. Contents. In Section 2 we estimate the parameter of a barrier for conic hull of convex set obtained from a ν-self-concordant barrier of the set. The value of new parameter does not exceed 3.08 ν + 3.57) 2. We show that under assumptions made, the asymptotic growth of this estimate in ν cannot be improved. In Section 3 we adapt the technique 1
of Section 5.1.2 [8] for developing self-concordant barrier for convex cones. In Section 4 we give examples of barriers for the epigraph of p-norm, entropy function, hypograph of geometric mean, etc. In Section 5 we describe a general rule helping to construct a selfconcordant barrier for conic hull of epigraph of convex univariate function. The values of corresponding parameters usually vary between three and four. In Section 6 we show how to rewrite the Geometrical Programming problem in primal and dual conic forms, both endowed with good self-concordant barriers. Notation and generalities. Let E be a finite dimensional real vector space with dual space E. We denote the corresponding scalar product by s, x, where x E and s E. If E = R n, then E = R n and we use the standard scalar product s, x = n s i) x i), x = x, x 1/2, s, x R n. The actual meaning of the notation, can be always clarified by the space containing the arguments. For a linear operator A : E E 1 we ine its adjoint operator A : E 1 E in a standard way: Ax, y = A y, x, x E, y E 1. If E 1 = E, we can talk of self-adjoint operators: A = A. Recall that function F x), x int Q, is called self-concordant, if it is a barrier and D 3 F x)[h, h, h] 2D 2 F x)[h, h] 2 F x)h, h, x int Q, h E. 1.1) If h is a recession direction of Q, then at any x int Q we have F x)h, h 1/2 F x), h. 1.2) We say that F is a ν-self-concordant barrier if, in addition to 1.1), F x), h 2 ν F x)h, h, x int Q, h E. 1.3) The value ν 1 is called the parameter of the barrier. If function fx) satisfies inequality D 3 F x)[h, h, h] 2M F x)h, h, then function ˆF x) = M 2 F x) is self-concordant. Let K E be a convex cone. We call it proper if it is a closed pointed cone with nonempty interior. For a proper cone, its dual cone K = {s E : s, x 0 x K} is also proper. For interior-point methods IPM), the cone K must be represented by a self-concordant barrier F x), x int K, with parameter ν 1 see Chapter 4 in [6] for main results). The important examples of convex cones are the positive orthant: R+ n = {x R n : x 0}, F x) = n ln x i), ν = n, 2
the Lorentz cone L n = {τ, x) R R n : τ x, x 1/2 }, F τ, x) = lnτ 2 x, x ), ν = 2. and the cone of positive semiinite matrices S n + = {X S n : X 0}, F X) = ln det X, ν = n, In all these examples, the cones are symmetric and the barriers are self-scaled [9]. The natural barriers for cones are logarithmically homogeneous barriers: F τx) F x) ν ln τ, x int K, τ > 0. 1.4) Note that for convex F, identity 1.4) implies 1.3). It is important that the dual barrier F s) = max x { s, x F x) : x int K}, s int K, is a ν-self-concordant logarithmically homogeneous barrier for K. 2 Barrier for conic hull of convex set Let Q E be a closed convex set and F x) be a ν-self-concordant barrier for Q. Consider the conic hull K = Cl { τ, x) : x τ int Q, τ > 0} R + E. Theorem 1 For any κ 3 2 ν function Φτ, x) = γν, κ) [ F x τ ) κ ln τ ], γν, κ) = ] ) κ 3/2 3/2 2 + 1 κ ν) 3/2 κ [1 + κ κ ν, 2.1) is a ˆν-self-concordant barrier for K with ˆν = κ γν, κ). For value κ = 4ν we have ˆν = 1 27 16 ν + 7 3/2 ) 2 < 3.08 ν + 3.57) 2 2.2) Proof: Consider an arbitrary point τ, x) int K and direction l = τ, x ) R E. Denote δ τ = τ τ, ω = ωτ, x) = x τ, and ω ω = Dωτ, x)[l] = x τ τ x τ 2 = x τ δ τ ω, = D 2 ωτ, x)[l, l] = τ x τ 2 + δ 2 τ ω δ τ ω = 2δ τ ω. 2.3) Denote Φτ, x) = F ) x τ κ ln τ, and Dk = D k Φτ, x)[ l,..., l ], k = 1,..., 3. Let us }{{} k times compute these values. D 1 = F ω), ω κδ τ. 3
D 2 = F ω)ω, ω + F ω), ω + κδ 2 τ 2.3) = F ω)ω, ω 2δ τ F ω), ω + κδ 2 τ. D 3 = D 3 F x)[ω, ω, ω ] + 2 F ω)ω, ω +2δ 2 τ F ω), ω 2δ τ F ω)ω, ω 2δ τ F ω), ω 2κδ 3 τ 2.4) 2.5) 2.3) = D 3 F x)[ω, ω, ω ] 6δ τ F ω)ω, ω + 6δ 2 τ F ω), ω 2κδ 3 τ. Denote σ = F ω)ω, ω. Then, D 2 1.3) σ 2 δ τ ν σ 1/2 + κδ 2 τ = Since ν 2 3 κ, we have D 2 κ 3 δ2 τ. Therefore σ 1/2 ν δ τ ) 2 + κ ν)δ 2 τ. 2.6) 1.3) D 3 2σ 3/2 6δ τ σ + 6δ 2 τ F ω), ω 2κδ 3 τ 2.4) = 2σ 3/2 6δ τ σ + 3δ τ [ σ + κδ 2 τ D 2 ] 2κδ 3 τ = 2σ 3/2 [ 3δ τ D2 + σ κ ] 3 δ2 τ 2σ 3/2 + 3 δ τ [ D 2 + σ κ ] 3 δ2 τ = 2σ 3/2 + 3 δ τ [D 2 + σ] κ δ τ 3 2σ 3/2 + 2 κ [D 2 + σ] 3/2. On the other hand, in view of 2.6), we have D 2 κ δ τ ν κ σ1/2) 2 + 1 ν κ) σ. Hence, γν, κ) D 3 2D 3/2 2 γ 1/2 ν, κ), = ] ) κ 3/2 3/2 2 + 1 κ ν) 3/2 κ [1 + κ κ ν. Thus, we have proved that Φ is a self-concordant function. Since its degree of logarithmic homogeneity is equal to ˆν, we conclude that Φ is a ˆν-self-concordant barrier. Finally, for κ = 4ν, we have ˆν = κ 2 κ ν) 3/2 + [ 1 + κ κ ν ] 3/2 ) 2 = 16 ) ) ν 3/2 2 3 + 7 3 3 = 1 27 16 ν + 7 3/2 ) 2 < 3.08 ν + 3.57) 2. Note that our estimate cannot be significantly improved under assumption made. Indeed, we can expect that On the other hand, it could happen that D 3 2σ 3/2 3δ τ D 2 3δ τ σ + κδ 3 τ. δ τ F ω), ω δ τ F ω), [F ω)] 1 F ω) 1/2 F ω)ω, ω 1/2 δ τ ν σ 1/2. 4
Then D 2 κ δ τ ν κ σ1/2) 2 + 1 ν κ) σ. ) Therefore, choosing δ τ = ν κ σ1/2, we obtain that D 3 2D 2 κ 3/2. κ ν Thus, we cannot expect better asymptotic dependence of ˆν in the parameter ν than that of 2.2). To conclude this section, let us obtain a representation of the barrier for the dual cone K = {λ, s) R E : λτ + s, x 0 x τq, τ > 0} = {λ, s) R E : λ + s, y 0 y Q} = {λ, s) R E : λ ξ Q s)}, where ξ Q s) = max s, y. Assume that the we can compute dual function y Q F s) = max{ s, x F x)}. x Theorem 2 For ˆν-self-concordant barrier Φτ, x) = γf x τ ) ˆν ln τ, with γ 1 and ˆν = 4γν, the dual barrier can be represented as follows: Φ λ, s) = max τ [ λτ + γf τ γ s ) + ˆν ln τ Univariate objective function in this maximization problem is concave and self-concordant. Proof: Let us choose λ > ξ Q s). Then Φ λ, s) = max [ λτ s, x γf x τ,x τ ) + ˆν ln τ] Let us estimate the derivatives of function ψ: In view of Theorem 2.4.2 [8], we have Therefore, ]. = max[ λτ τ s, y γf y) + ˆν ln τ] τ,y [ = max ψτ) ] = λτ + γf τ τ γ s) + ˆν ln τ. ψ τ) = λ + s, F τ γ s) + ˆν τ. s, F τ νγ2 γ s)s. 2.7) τ 2 ψ τ) = 1 γ s, F τ 2.7) ˆν γ s)s τ 2 γν ˆν τ 2 = 3γν τ 2. ψ 1 τ) = D γ 3 F 2 τ 1.1) ˆν γ s)[s, s, s] + 2 2 s, F τ 3 γ 2 τ γ s)s 3/2 + 2 ˆν τ 3 2.7) 2 γ 2 νγ 2 τ 2 ) 3/2 + 2 ˆν τ 3 = 2 γν3/2 +4γν τ 3. 5
Thus, ψ τ) 2 ψ τ)) 3/2 ν3/2 +4ν 3γν) 3/2 < 5 3 3 < 1. We have seen that the generation of a barrier for conic hull results in a significant increase of the barrier parameter, especially if the parameter is relatively small. Therefore it is recommended to apply this operation only once, when the barrier for the whole feasible set of optimization problem is already constructed, and the parameter is already big. 3 Composite barriers In this section we adapt the general framework of Section 5.1.2 B in [8] for convenient development of self-concordant barriers for convex cones. Consider a function ξx) : E 1 E 2 ined on a closed convex set Q 1 E 1. Assume that ξ is three times continuously differentiable and concave with respect to a convex closed cone K E 2 : D 2 ξx)[h, h] K x int Q, h E 1. 3.1) It is convenient to write this inclusion as D 2 ξx)[h, h] K 0. Let F x) be a ν-self-concordant barrier for Q 1 and β 1. We say that ξ is β-compatible with F, if for all x int Q 1 and h E 1 we have D 3 ξx)[h, h, h] K 3β D 2 ξx)[h, h] F x)h, h 1/2. 3.2) Note that the set of β-compatible functions is a convex cone: if functions ξ 1 and ξ 2 are β-compatible with F, then any sum αξ 1 +βξ 2, with α, β > 0, is β-compatible with F also. In this section we construct a self-concordant barrier for a composition of the set and a convex set Q 2 E 2 E 3, that is S 1 = {x, y) Q E 2 : ξx) y}, Q = {x, z) : ξx) K y, y, z) Q 2 }. For that we use a µ-self-concordant barrier Φy, z) for Q 2. We assume that all directions from the cone K 0 = K {0} E 2 E 3 are the recession directions of the set Q 2. Consider the barrier Ψx, z) = Φξx), z) + β 3 F x). Let us fix a point x, z) int Q and choose an arbitrary direction d = x, z ) E 1 E 3. Denote ξ = Dξx)[h], ξ = D 2 ξx)[h, h], ξ = D 3 ξx)[h, h, h], l = ξ, z ). Denote ψx, z) = Φξx), z). Consider the following directional derivatives: 1 = Dψx, z)[d] = Φ yξx), z), ξ + Φ zξx), z), z = Φ ξx), z), l. 6
Note that l lx). Therefore l = Dlx)[d] = ξ, 0) K 0. Thus, we can continue: 2 = D 2 ψx, z)[d, d] = Φ ξx), z)l, l + Φ ξx), z), l = Φ ξx), z)l, l + Φ yξx), z), ξ = σ 1 + σ 2. 3.3) Since l is a recession direction for Q, we have σ 2 0. Finally, 3 = D 3 ψx, z)[d, d, d] = D 3 Φξx), z)[l, l, l] + 3 Φ ξx), z)l, l + Φ yξx), z), ξ. 3.4) Again, since l is a recession direction for Q, Φ ξx), z)l, l Φ ξx), z)l, l 1/2 Φ ξx), z)l, l 1/2 1.2) Φ ξx), z)l, l 1/2 Φ ξx), z), l = σ 1/2 1 σ 2. Further, denote σ 3 = F x)h, h. Since ξ is β-compatible with F, we have Φ yξx), z), ξ 3β Φ yξx), z), ξ σ 1/2 3 = 3β σ 2 σ 1/2 3. Thus, substituting these inequalities to 3.4) and using 1.1), we obtain 3 2σ 3/2 1 + 3σ 1/2 1 σ 2 + 3β σ 2 σ 1/2 3. Consider now D k, k = 1... 3, the directional derivatives of function Ψ. Note that Therefore, D 2 = 2 + β 3 σ 3 = σ 1 + σ 2 + β 3 σ 3 σ 1 + σ 2 + β 2 σ 3. 3.5) D 3 = 3 + β 3 D 3 F x)[h, h, h] 1.1) 3 + 2β 3 σ 3/2 3 2σ 3/2 1 + 3σ 1/2 1 σ 2 + 3β σ 2 σ 1/2 3 + 2β 3 σ 3/2 3 = σ 1/2 1 + βσ 1/2 3 )2σ 1 2βσ 1/2 1 σ 1/2 3 + 2β 2 σ 3 + 3σ 2 ) 3.5) σ 1/2 1 + βσ 1/2 3 )3D 2 σ 1/2 1 + βσ 1/2 3 ) 2 ) 2D 3/2 2. Thus, we come to the following statement. Theorem 3 Let function ξx) : E 1 E 2 satisfies the following conditions. It is concave with respect to a convex cone K E 2. It is β-compatible with self-concordant barrier F x) for a set Q dom ξ. 7
Assume in addition that the cone K {0} E 2 E 3 contains only recession directions of some closed convex set Q 2 E 2 E 3 endowed with a µ-self-concordant barrier Φy, z), y, z) int Q 2. Then function Ψx, z) = Φξx), z) + β 3 F x). is a self-concordant barrier for the set Q = {x, z) : ξx) K y, y, z) Q 2 } with parameter ˆν = µ + β 3 ν. Proof: We need only to justify the value ˆν. Indeed, D 1 = Φ ξx), z), l + β 3 F x), h ν σ 1/2 1 + β 3 µ σ 1/2 3 max { ν σ 1/2 1 + β 3 µσ 1/2 3 : σ 1 + β 3 3.5) σ 3 D 2 } = ˆν D 1/2 2. σ 1,σ 3 0 Thus, in view of 1.3), ˆν can be taken as a parameter of the barrier Ψ. 4 Examples of self-concordant barriers for cones Despite to its complicated formulation, Theorem 3 is very convenient for constructing good self-concordant barrier for convex cones. Let us confirm this claim by examples. Note that the barriers below resemble very much the barriers proposed in Section 5.3.1 and Section 5.3.2 of [8]. However, the new barriers describe now the convex cones. 1. Self-dual power cone and epigraph of p-norm. Let us fix some α 0, 1). Our goal is to find a barrier function for the following cone Let us choose K α = {x 1), x 2), z) R 2 + R : x 1) ) α x 2) ) 1 α z }. Q 1 = R 2 +, F x) = ln x 1) ln x 2), ν = 2, ξx) = x 1) ) α x 2) ) 1 α, E 2 = R, K = R +, Q 2 = {y, z) : y z }, Φy, z) = lny 2 z 2 ), µ = 2. Thus, all conditions of Theorem 3 are clearly satisfied except β-compatibility. check it. Let us choose a direction h R 2 and x int R+. 2 Denote Let us δ 1 = h1) x 1), Let us compute the directional derivatives: Dξx)[h] = [ αh 1) x 1) δ 2 = h2) x 2), σ = δ 2 1 + δ2 2. ] + 1 α)h2) ξx) = [αδ x 2) 1 + 1 α)δ 2 ] ξx), 8
D 2 ξx)[h, h] = [αδ 2 1 + 1 α)δ2 2 ] ξx) + [αδ 1 + 1 α)δ 2 ] Dξx)[h] = α1 α)δ 1 δ 2 ) 2 ξx), D 3 ξx)[h, h, h] = 2α1 α)δ 1 δ 2 ) δ 2 1 δ2 2 ) ξx) α1 α)δ 1 δ 2 ) 2 Dξx)[h] = ξx) α1 α)δ 1 δ 2 ) 2 [2δ 1 + 2δ 2 αδ 1 1 α)δ 2 ] = D 2 ξx)[h, h] [2 α)δ 1 + 1 + α)δ 2 ]. Since 2 α)δ 1 + 1 + α)δ 2 [2 α) 2 + 1 + α) 2 ] 1/2 σ 1/2 < 3σ 1/2, we conclude that ξ is 1-compatible with F. Therefore, in view of Theorem 3, function Φx, z) = ln x 1) ) 2α x 2) ) 21 α) z 2) ln x 1) ln x 2) is a 4-self-concordant barrier for cone K α. This result was proved in [7] by a particular version of Theorem 3. It is interesting that the barrier ) φx, z) = ln x 1) ) α x 2) ) 1 α) z ln x 1) ln x 2) was mentioned in [3] without justification) as a 3-self-concordant barrier for corresponding hypograph. Thus, φx, z) + φx, z) becomes a 6-self-concordant barrier for K α. Note that the barrier Φx, z) can be used for constructing 4n-self-concordant barrier for the epigraph of a p-norm in R n see [7]): } K p = {τ, z) R R n : τ z p), 1 p, where z p) = [ n ] 1/p z i) p. Without loss of generality, let us assume α = 1 p 0, 1). Then, it is easy to prove that the point τ, z) belongs to K p if and only if there exist x R n + satisfying conditions x i) ) α τ 1 α z i), i = 1,..., n, n x i) = τ. 4.1) Thus, a self-concordant barrier for the cone K p can be implemented by restricting the 4n)-self-concordant barrier Φ p τ, x, z) = n onto the hyperplane n x i) = τ. [ ln x i) ) 2α τ 21 α) z i) ) 2) ] + ln x i) + ln τ 4.2) 2. Conic hull of the epigraph of entropy function. We need to describe the conic hull of the following set: { } x 1), z) : z x 1) ln x 1), x 1) > 0. 9
Introducing a projective variable x 2) > 0, we obtain the cone { } Q = x 1), x 2), z) : z x 1) [ ln x 1) ln x 2) ], x 1), x 2) > 0. 4.3) Let us represent it in the format of Theorem 3. Q 1 = R 2 +, F x) = ln x 1) ln x 2), ν = 2, ξx) = x 1) [ ln x 1) ln x 2) ], E 2 = R, K = R +, Q 2 = {y, z) : y + z 0}, Φy, z) = lny + z), µ = 1. Let us show that ξ is 1-compatible with F. We use the notation of the previous example. Dξx)[h] = δ 1 ξx) x 1) [δ 1 δ 2 ]. D 2 ξx)[h, h] = δ 2 1 ξx) + δ 1 Dξx)[h] h 1) [δ 1 δ 2 ] + x 1) [δ 2 1 δ2 2 ] = x 1) [ 2δ 1 δ 1 δ 2 ) + δ 2 1 δ2 2 ] = x1) δ 1 δ 2 ) 2. D 3 ξx)[h, h, h] = h 1) δ 1 δ 2 ) 2 + 2x 1) δ 1 δ 2 ) δ 2 1 δ2 2 ) = x 1) δ 1 δ 2 ) 2 [ δ 1 + 2δ 1 + δ 2 )] = D 2 ξx)[h, h] [δ 1 + 2δ 2 ]. Since δ 1 + 2δ 2 5 σ 1/2 < 3σ 1/2, we conclude that ξ is 1-compatible with F. Therefore, in view of Theorem 3, function ) Φx, z) = ln z x 1) ln x1) ln x 1) ln x 2) 4.4) x 2) is a 3-self-concordant barrier for cone Q. It is interesting that the same barrier can describe also the epigraph of logarithmic and exponent functions. Indeed, Q {x : x 1) = 1} = {x 2), z) : z ln x 2) } = {x 2), z) : x 2) e z } Let us show how to use the 3-self-concordant barrier ψx, y, τ) = ln τ ln y τ x) ln y ln τ, x, y, τ) int E = { } y τe x/τ, τ > 0 R 3, 4.5) in more complicated situations. Consider a conic hull of the epigraph of following function: n ) f n x) = ln e xi), x R n, 4.6) Q = { x, t, τ) R n R R : t τf x ) } m τ, τ > 0. Clearly x, t, τ) Q if and only if f n 1 τ x t e) ) 1, 10
where e R n is the vector of all ones. Therefore, we can model Q as a projection of the following cone: ˆQ = {x, y, t, τ) R n R n R R : y i) τe xi) t)/τ, i = 1,..., n, n y i) = τ}. This cone admits 3n-self-concordant barrier, obtained as a restriction of the function Φx, y, t, τ) = n [ ) ] ln t + τ ln y i) x i) τ ln τ + ln y i) + ln τ, 4.7) onto the hyperplane n y i) = τ. 3. Geometric mean. Let x R n + and a n = { y R n : loss of generality, we can consider x and a with positive component. Denote ξx) = x a = n x i) ) ai). n } y i) = 1. Without Let us write down the directional derivatives of this function along some h R n. Denote δ i) x h) = hi) x i), i = 1,..., n, δ x h) = δ 1) x F x) = n ln x i). h),..., δ x n) T h)), Clearly, h x = F x)h, h 1/2 = δ x h). Note that Dln ξx))[h] = 1 ξx) Dξx)[h] = a, δ xh). Thus, Dξx)[h] = ξx) a, δ x h). Denoting by [x] k R n a component-wise power of vector x R n, we obtain: D 2 ξx)[h, h] = ξx) a, δ x h) 2 ξx) a, [δ x h)] 2 = ξx) a, [δ x h) a, δ x h) e] 2. Further, denoting by x y R n a component-wise product of two vectors x, y R n, and δ = δ x h), we obtain: D 3 ξx)[h, h, h] = ξx) a, δ a, [δ a, δ e] 2 ξx) a, δ a, δ e) [δ] 2 + a, [δ] 2 e) = ξx) a, a, δ [δ] 2 2 a, δ 2 δ + a, δ 3 e ξx) a, [δ] 3 + a, δ [δ] 2 + a, [δ] 2 δ a, δ a, [δ] 2 e = ξx) a, [δ] 3 2 a, δ [δ] 2 + 2 a, δ 2 δ a, δ 3 e 11
Thus, D 3 ξx)[h, h, h] = ξx) a, [δ a, δ e] 3 + ξx) a, δ a, [δ] 2 a, δ 2 e = ξx) a, [δ a, δ e] 3 + ξx) a, δ a, [δ a, δ e] 2 [ ] ξx) a, [δ a, δ e] 2 max 1 i n {δi) a, δ } + a, δ D 2 ξx)[h, h] F x)δ, δ 1/2. Thus, we prove that ξ is 1-compatible with F. This means that the function Ψx, t) = lnξx) t) + F x), x > 0 R n, 4.8) is an n + 1)-self-concordant barrier for the hypograph of function ξ compare with [2]). Moreover, since the set of β-compatible functions is a convex cone, we conclude that any sum ξx) = m α k x a k, 4.9) with α k > 0, and a k n, k = 1,..., m, is 1-compatible with F. Hence, for such ξx) the formula 4.8) is also applicable. Moreover, the parameter of this barrier is still n + 1. Note that the functions in the form 4.9) sometimes arise in optimization problems related to polynomials. Indeed, assume we need to solve the problem max y { py) = m k=1 k=1 } α k y b k : y 0, y d) 1, [ where all b k belong to d n. Then the transformation of variables y i) = x i)] 1/d, i = 1,..., n, leads to a convex problem with a concave objective ξx) given by 4.9). 5 Conic hull of two-dimensional epigraph We have seen that the most difficult part in applying Theorem 3 is the proof of β- compatibility of function ξx). Hence, any sufficient conditions for this property are very useful. In this section, we give a condition for β-compatibility of function ξx), x R+, 2 which is obtained as a homogenization of a concave univariate function compare with Proposition 5.3.1 in [8]). This condition covers many problem arising in Separable Optimization. Lemma 1 Let ζτ) be a C 3 smooth concave function of τ > 0. Assume that for some constants γ 1 < γ 2 and arbitrary τ > 0 we have Then function ξx) = x 2) ζ where γ 1 ζ τ) ζ τ) τ γ 2 ζ τ). 5.1) x 1) x 2) ) is β-compatible with barrier F x) = ln x 1) ln x 2), β = max {1, pγ 1 ), pγ 2 )}, 5.2) with pγ) = 1 3 9 6γ + 2γ 2. For γ 1, γ 2 [0, 3] we can take β = 1. 12
Proof: Let us fix arbitrary x int R 2 + and direction h R 2. Denote ω = ωx) = x1) x 2), δ 1 = h1) x 1), and δ 2 = h2) x 2). Then ω ω = Dωx)[h] = h1) x1) h 2) = δ x 2) x 2) ) 2 1 δ 2 ) ω, = D 2 ωx)[h, h] = δ 2 1 δ2 2 ) ω + δ 1 δ 2 ) ω = 2δ 2 ω. 5.3) Since ξx) = x 2) ζωx)), we have Dξx)[h] = h 2) ζω) + x 2) ζ ω)ω, Finally, ξ 2 = D 2 ξx)[h, h] = 2h 2) ζ ω)ω + x 2) ζ ω)ω ) 2 + x 2) ζ ω)ω 5.3) = x 2) ζ ω)ω ) 2. 5.4) D 3 ξx)[h, h, h] = h 2) ζ ω)ω ) 2 + x 2) ζ ω)ω ) 3 + 2x 2) ζ ω)ω ω 5.3) = 3h 2) ζ ω)ω ) 2 + x 2) ζ ω)ω ) 3 5.4) = 3δ 2 ξ 2 + x 2) ζ ω)ω ) 3 Thus, in view of assumption 5.1), we get 5.3) = 3δ 2 ξ 2 + x 2) ζ ω)ω ω ) 2 δ 1 δ 2 ). D 3 ξx)[h, h, h] ξ 2 max{γ 2 δ 1 + 3 γ 2 )δ 2, γ 1 δ 1 + 3 γ 1 )δ 2 }. Since F x)h, h = σ1 2 + σ2 2, we justify 5.2). Corollary 1 Let function ζτ) satisfies conditions of Lemma 1. Then the convex cone K = { ) ) } x 1), x 2), z R 3 : x 2) ζ x 1) z, x 1), x 2) > 0 x 2) admits a 1 + 2β 3 )-logarithmically homogeneous self-concordant barrier with β given by 5.2). ) ) F x 1), x 2), z) = ln x 2) ζ x 1) z β 3 ln x 1) β 3 ln x 2) x 2) 13
6 Conic formulation for Geometric Programming As an application example, let us present now a Geometric Programming problem in a conic form. The initial formulation of this problem looks as follows: inf x R n p 0x), s.t. p j x) 1, j = 1,..., m, 6.1) x i) > 0, i = 1,..., n, where all functional components are posinomials: p j x) = m j k=1 c k) j n x i)) A k,i) j, j = 0,..., m, with c j int R m j +, j = 0,..., m. Denoting we obtain the following problem: b k) j = ln c k) j, k = 1,..., m j, j = 0,..., m, y i) = ln x i), i = 1,..., n, min f m u j,y 0 u 0 ), s.t. u j = A j y + b j R m j, j = 0,..., m, 6.2) f mj u j ) 0, j = 1,..., m, where functions f mj ) are ined by 4.6). For any function f mj, the conic hull of its epigraph can be described by the barrier 4.7). Imposing the equality constraints for the additional projective variables, we obtain the primal conic reformulation of problem 6.1). Thus, the feasible cone of this problem can be described by 3N-self-concordant barrier, where N = m m j j=0 monomials in 6.1). The dual variant of problem 6.1) can be obtained by representation is the number of { f m u) = max s, u m } ηs k) ) : e, s = 1, 6.3) s R m k=1 where ητ) = τ ln τ. Introducing the dual multipliers λ R m + for inequality constraints 14
in 6.2), we can transform this problem as follows: [ ] min max f m0 A 0 y + b 0 ) + m λ j) f mj A j y + b j ) y R m λ 0 = min y R m = min y R m max λ 0) =1, λ 0 max λ 0) =1, λ 0 j=1 [ m { λ j) m j max s j, A j y + b j j=0 s j,e =1 k=1 [ m max j=0 s j,e =λ j) η s k) j { s j, A j y + b j λ j) m j η k=1 ) }] s k) j λ j) )}] Exchanging now minimum and maximum, and eliminating variables y, we obtain the following formulation:. max { λ,s,z m [ s j, b j z j, e ] : j=0 m A j s j = 0, j=0 λ 0) = 1, λ j) 0, j = 1,..., m, s j, e = λ j), j = 0,..., m, 6.4) s k) j, λ j), z k) ) Q, k = 1,..., m j, j = 0,..., m}, j where Q is given by 4.3). Recall that Q admits a 3-self-concordant barrier F s, λ, z) = ln z s ln s λ) ln λ ln s. Thus, the parameter of the barrier for the feasible cone of problem 6.4) is equal to 3N. It is interesting, that in both primal and dual variants of our problem we use the same cone Q. In order to apply to problem 6.4) nonsymmetric primal-dual IPM see [7]), it is necessary to compute also the value and the gradient of the dual barrier F x, τ, u) = max λ,s,z { sx λτ zu F s, λ, z)}. 6.5) Unfortunately, there is no closed form solution for this problem. However, it can be approximated by an efficient numerical procedure. Denote = z s ln s λ. Then the first-order optimality conditions for 6.5) can be written as follows: 1 ln s + 1 s = x, 1 s λ + 1 λ = τ, 1 = u. Thus, = 1 u, λ = 1 τ 1 + su), and optimal value of s can be found from the equation 1 s + u ln 1 s = x + u, 15
where u > 0. Denoting t = 1 s and a = x + u, we come to equation t + u ln t = a with concave and increasing left-hand side. Thus, it can be easily solved by a quadratically convergent procedure. References [1] R. W. Freund, F. Jarre, and S. Schaible. On self-concordant barrier functions for conic hulls and fractional programming. Mathematical Programming, 74, 237 246 1996) [2] A.S. Lewis and H.S. Sendov. Self-concordant barriers for hyperbolic means. Mathematical Programming, 91, 1-10 2001). [3] Nemirovski, A. Polynomial time methods in Convex Programming. In: J. Renegar, M. Shub and S. Smale, Eds., Lectures in Applied Mathematics, v. 32 1996): AMS, Providence, 543-589. [4] A. Nemirovski, L. Tuncel. Cone-free path-following and potential reduction polynomial time interior-point algorithms. Mathematical Programming, On line First issue, DOI: 10.1007/s10107-004-0545-4 2004). [5] Yu. Nesterov. Long-Step Strategies in Interior-Point Primal-Dual Methods. Mathematical Programming, 761), 47 94 1996) [6] Yu. Nesterov. Introductory Lectures on Convex Optimization. Kluwer, Boston, 2004. [7] Yu. Nesterov. Towards nonsymmetric conic optimization. CORE Discussion Paper #2006/28, 2006) [8] Yu. Nesterov and A. Nemirovsky, Interior Point Polynomial Algorithms in Convex Programming, SIAM, Philadelphia, 1994. [9] Yu. Nesterov, M. J. Todd. Self-scaled Barriers and Interior-Point Methods for Convex Programming. Mathematics of Operation Research, 221), 1 42 1997) [10] Yu.Nesterov, M.J.Todd and Y.Ye. Infeasible-start Primal-Dual Methods and Infeasibility Detectors. Mathematical Programming, 84, 227-267 1999) [11] J. Renegar. A Mathematical View of Interior-Point Methods in Convex Optimization. MPS/SIAM Series on Optimization 3. SIAM Publications, Philadelphia, 2001. [12] L. Tuncel. Generalization of primal-dual interior-point methods to convex optimization problems in conic form. Foundations of Computational Mathematics, 1, 229-254 2001). [13] G. Xue, Y. Ye. An efficient algorithm for minimizing a sum of p-norms. SIAM Journal on Optimization, 101998) 2, 551 579. 16