TECHNICAL NOTE. A Finite Algorithm for Finding the Projection of a Point onto the Canonical Simplex of R"

Transcription

1 JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS: VoL 50, No. 1, JULY 1986 TECHNICAL NOTE A Finite Algorithm for Finding the Projection of a Point onto the Canonical Simplex of R" C. M I C H E L O T I Communicated by A. V. Fiacco Abstract. An algorithm of successive location of the solution is developed for the problem of finding the projection of a point onto the canonical simplex in the Euclidean space R ~. This algorithm converges in a finite number of steps. Each iteration consists in finding the projection of a point onto an affine subspace and requires only explicit and very simple computations. Key Words. Nonlinear programming, quadratic programming, projection onto a simplex, optimality conditions. 1. Introduction It is very important to know how to compute the projection of a point onto a polyhedron. This problem arises in many contexts, in particular in constrained linear optimization methods, such as gradient projection methods (Refs. 1-3). A great deal of papers have been devoted to this problem, and the authors applied themselves to describing stable and finite algorithms (Refs. 4-8). In this t)aper, we present a finite projection algorithm for the canonical simplex of R". This procedure is recursive. At each iteration, using Lagrange multipliers, we locate the solution and we are going to solve the same problem in a strict lower-dimensional space. The maximum number of iterations is equal to the initial dimension of the space. Moreover, each iteration requires only explicit and very simple computations. t MaStre Assistant, Laboratoire d'analyse Num6rique, Universit6 de Dijon, Dijon, France /86/ / P~enum Publishing Corporation

2 196 JOTA: VOL. 50, NO. 1, JULY Preliminaries We denote by X =R" [resp., Y=W"] the n-dimensional [resp., m- dimensional] Euclidean space, by (., ), and II" II. [resp., (., ),, and I1" II m] the usual inner product and the Euclidean norm of X [resp., Y]. Let x = (xl, x2,.., x,) ~ R". Then, we write x ~ 0, if x~ I> 0 for all i = 1, 2,..., n. A r denotes the transpose of the m n matrix A. Let K C X be a nonempty, closed, convex set; and let c be a point such that c~ K. We denote by pk(e) the projection of c onto K. pk(c) is the optimal solution to the following problem (P): (P) inf ½1Ix - cll~, x ~ K. Let us suppose that K is a polyhedron defined by K ={xcxlax=b,x~o}, where A is an m n matrix with full row rank m and b is a vector of Y. For solving (P), we can introduce the Lagrangian L, defined by ~½tlx-clt2+(X, A x - b ) m - ( m x )., x e R ~,,~ er '~, ~z~o, L(x; (& ~)) = t -oo, otherwise. Then, we have the following well-known result (Ref. 9). Theorem 2.1. x* is an optimal solution to (P) if and only if there exist X* e Y,/z* e X such that (x*; ()t*, tz*)) is a saddle point of L. Let V C X be the affine subspace defined by V = {x e X[Ax = b}. Then, we have the following result. Corollary 2.1. Let c e V, c ~ K. Then, x* ek is an optimal solution to (P) if and only if there exists/x* ~ X such that X* -- C + A r ( A A T ) - I A I ~ * =/x*, (x*, Ix*), = 0, ~*~0. Proof. (x*; ()t*,/x *)) is a saddle point of L if and only if the following statements are satisfied: x*ck~ x*-c+ara*=tx *, (x*, t~ *), = O, ~*~0. (1) (2) (3) (4) (5) (6) (7)

3 .IOTA: VOL. 50, NO. 1, JULY If we premultiply both sides of Eq. (5) by A, we get AA TA * = Al,~ *. Since A is of rank m, the operator AA r has an inverse. Then, we can substitute for A* into Eq. (5), and the result follows. D In fact, we can always assume that e c V in view of the following theorem. Theorem 2.2. Let V C X be an affine subspace; let K C V be a nonempty, closed, convex set; and let c ~ V. Then, we have pk (c) = p,, (pv(c)). Proof. From the characterization of the projection of e on K, we have (c--pk(c),x--pk(c)),<.o, forall xe K. This inequality can be rewritten as (e-pv(c),x-pk(c)}+(pv(c)--pk(e),x-pk(c))~o, forall x~ K. Now, we have x -pk(c) e V and c -pv(c) ~ V I, where W- is the orthogonal subspace to V. This implies that (pv(c)-dk(c),x-pr(c))<~o, forall x~ K. Then, the proof is complete. [] 3. Projection of a Point onto the Canonical Simplex In this section, we shall give a method for solving (P) in the following case: where (P) inf½llx - cli~, x ~ K, K = {x ~ Xli~=l x~= l, x >~O }. We begin with some additional notations. We denote 1, = {1, 2,..., n}, V= x c X ~ i x ~ = l.

4 198 JOTA: VOL. 50, NO. 1, JULY 1986 For an arbitrary subset I of In, Xx = {x ~ XIxi = 0, for all i ~ I} v~ = x~ ~ v, K1 = Xr ~ K, nr = dim(x1), where X1 is a linear subspace of X. Let c ~ Xx, c~ Kx. Consider the following problem (P~): (PI) inf½llx- cll2~, x~ gi. From Corollary 2.1, x* ~ Ks is an optimal solution to (P~) if and only if there exists/x*~ XI such that (x*,/x*), = 0, (9) /x* >i O. (10) By means of these optimality conditions, we obtain the main result. Theorem 3.1. Let ICI,,, c~ VI, and put I={i~IIc~<O}. Then, if [# ~, we have pk,(c) ~ Kxj. Proof. Putx* =pk,(c),andlet i~ L By assumption, i~ L The left-hand side of equality (8) is strictly positive. Then, we have x* = 0, since the corresponding multiplier/x* is different from zero. This means that x*c Kij. [] Now, Theorem 2.2 and Theorem 3.1 give a very simple procedure, in three stages, for solving (P): Find the projection onto an affine subspace VI; locate PK (c) in a convex set Kj with I C J; find the projection onto the linear subspace X~; replace I by J, and repeat the process. We obtain in more detail the following algorithm. 4. Algorithm Description Step 1. Initialization. Put I = Q and x = c. Step2. Iteration. Let ICI,,, and let xcxj, such that x~vr. Compute ~ = Pv~(X). If ~ >10, then stop: ~ = PK(c). Otherwise, replace I by Iu{i[~i<O}, and replace x by Px,( ).

5 JOTA: VOL 50, NO. t, JULY Convergence. The algorithm converges with at most n iterations, since the dimension nl of the subspace Xt decreases by at least one unit. Implementation. From a practical point of view, the algorithm requires the computation of ~ = Pvx(X), which is explicitly made by the relation: :~i=xi--(~xi--1)/n,, if i~i, ~i=0, if i~i. The algorithm requires also the computation of x = Px, (x), which consists only in putting xi = ~, i f ~ > 0, x~ = 0, otherwise. 5. Concluding Remarks In general, the algorithm is very efficient, given that at each iteration the dimension nl of the subspace X1 often decreases by several units. It is also to be noted that the method can be generalized to the case where the convex set K is given by where ~i is a strictly positive real, i = 1, 2..., n. References 1. ROSEN, J. B., The Gradient Projection Method of Nonlinear Programming, Part t, Linear Constraints, SIAM Journal on Applied Mathematics, Vol. 8, pp , ROSEN, J. B., The Gradient Projection Method of Nonlinear Programming, Part 2, Nonlinear Constraints, SIAM Journal on Applied Mathematics, Vol. 9, pp , 196t. 3. FIACCO, A. V., and MCCORMICK, G. P., Nonlinear Programming: Sequential Unconstrained Minimization Techniques, John Wiley and Sons, New York, New York, BAZARAA, M. S., GOODE, J. J., and RARDIN, R. L., An Algorithm for Finding the Shortest Element of a Polyhedral Set with Application to Lagrangian Duality, Journal of Mathematical Analysis and Applications, Vol. 65, pp , 1978.

6 200 JOTA: VOL. 50, NO. 1, JULY GOLUB, G. H., and SAUNDERS, M. A., Linear Least Squares and Quadratic Programming, Integer and Nonlinear Programming, Edited by J. Abadie, North- Holland, Amsterdam, Holland, MIFFLIN, R., A Stable Method for Solving Certain Constrained Least Squares Problems, Mathematical Programming, Vol. 16, pp , 19r9. 7. WOLFE, P.~ Algorithm for a Least-Distance Programming Problem, Mathematical Programming Studies, VoL 1, pp , WOLFE, P., Finding the Nearest Point in a Polytope, Mathematical Programming, Vol. 11, pp , ROCKAFELLAR~ R. T., Convex Analysis, Princeton University Press, Princeton, New Jersey, 1970.