An FPTAS for a Vector Subset Search Problem

Transcription

1 ISSN , Journal of Applied and Industrial Mathematics, 2014, Vol. 8, No. 3, pp c Pleiades Publishing, Ltd., Original Russian Text c A.V. Kel manov, S.M. Romanchenko, 2014, published in Diskretnyi Analiz i Issledovanie Operatsii, 2014, Vol. 21, No. 3, pp An FPTAS for a Vector Subset Search Problem A. V. Kel manov 1, 2* and S.M.Romanchenko 1** 1 Sobolev Institute of Mathematics, pr. Akad. Koptyuga 4, Novosibirsk, Russia 2 Novosibirsk State University, ul. Pirogova 2, Novosibirsk, Russia Received November 11, 2013; in final form, January 29, 2014 Abstract Under study is a strongly NP-hard problem of finding a subset of a given size of a finite set of vectors in Euclidean space which minimizes the sum of squared distances from the elements of this subset to its center. The center of the subset is defined as the average vector calculated with all subset elements. It is proved that, unless P=NP, in the general case of the problem there is no fully polynomial time approximation scheme (FPTAS). Such a scheme is provided in the case when the dimension of the space is fixed. DOI: /S Keywords: finding a vector subset, Euclidean space, minimum of the sum of squared distances, NP-hardness, fully polynomial time approximation scheme INTRODUCTION The subject of this article is an intractable optimization problem that is induced, in particular, by an actual problem of data classification. The goal of our study is a justification of an approximation polynomial algorithm for solving the problem. The problem is close in the statement to the classical [6, 9, 11] NP-hard [5] Problem MSSC (Minimum Sum-of-Squares Clustering) of data analysis. Recall that in Problem MSSC we need to find a partition of a given set of vectors in Euclidean space into a family of nonempty subsets, called clusters, so that to minimize the sum over all clusters of the sums of squared distances of all cluster elements to its center. The center of each cluster is defined by the average vector calculated with all cluster elements. Problem MSSC models (for instance, see [6, 8, 9]) the substantial problem of splitting a table with the results of multiple measurements of a set of numerical characteristics of several objects into nonempty subsets that contain the results of measurements of each of the objects and estimate the characteristics of these objects under condition that the correspondence between the object and the result of the measurement is not available, while each measurement is done with a certain error. In the problem under consideration we need to find only one subset of a given size in a finite set of vectors in Euclidean space so that the sum of the squared distances from the elements of the subset to its center is minimized. This problem models (for example, see [1, 2]) a substantial problem simpler than the above-stated. We assume that the table contains measurement results of the characteristics of only one important object (e.g., the subject of the study). Also, this table includes the measurements of some other (irrelevant or random) objects. Moreover, only the characteristics of the single object has an informational value, although it is unknown which measurement results correspond exactly to this object, and the measurement error is present. The formal statement of the problem and some related results are given in Section 1. Here note only that, despite the simplicity of the statement, this problem is strongly NP-hard [1]. Therefore, it is known [7] for this problem (unless P=NP) that there exists neither polynomial nor pseudopolynomial exact algorithm. Finally, unless P=NP, in the general case of this problem there does not exist a fully polynomial time approximation scheme (FPTAS) (see Section 2). Hence, it seems important to find some subclasses and particular cases of this problem for which an FPTAS exists. For one of these cases (when the space dimension is fixed), an FPTAS is provided below. * kelm@math.nsc.ru ** rsm@math.nsc.ru 329

2 330 KEL MANOV, ROMANCHENKO 1. STATEMENT OF THE PROBLEM AND AVAILABLE RESULTS The substantial problem of the introduction can formally be stated as follows [1, 2]: Problem VS-2 (Vector Subset 2). Given are some set Y = {y 1,...,y N } of vectors from R q and a positive integer M>1. Find C Yof size M such that F (C) = y C y y(c) 2 min, (1) where y(c) =1/ C y C y is the center of C. The number 2 in the name of the problem (VS-2) stands for its sequence number in the list [1] of the polynomially equivalent NP-hard problems that are induced by the same substantial problem. As it is known in the problems of statistics, if the elements of some nonempty set Z R q are sampling random numbers then z = 1 Z z, 1 Z 1 F (Z) are nonbiased sampling estimations of the expectation and variance correspondingly. In the deterministic statements of the problems, these quantities define the average value on the set and the variation of the elements with respect to the average. The less the variation of the elements in the set, the more compact they are. Simultaneous search for the most compact subset of elements and calculation of the average of the elements of this subset are the essence of Problem VS-2 in the deterministic case. If the elements of Y are treated as random numbers then Problem VS-2 can be interpreted as the simultaneous extraction from the littered sampling Y a subset C of the elements that correspond to the sampling data (from a certain distribution) with the minimal variance and estimation of the expectation of this distribution by the subset of selected data. Let us recall the available algorithmic results. In [2] for the general case of Problem VS-2, the 2- approximation polynomial algorithm was proposed with time complexity O(qN 2 ). The polynomial time approximation scheme (PTAS) was proposed in [4] that allows us to find an approximate solution with relative error >0in O(qN 2/+1 (9/) 3/ ) time. As stated in the introduction, in the general case of Problem VS-2 there does not exist (unless P=NP) any exact polynomial or pseudopolynomial algorithm. Moreover, in the case when the size M of the subset being found is not a part of the input (i.e., it is fixed), the problem is solvable in polynomial time. Indeed, in order to find an exact solution, it suffices to calculate in O(qM) time the value of the function (1) for each feasible subset of size M of Y and choose the minimum among the O(N M ) values that are found. Further, an introduction of the additional constraints on the problem input data allows us to construct an exact pseudopolynomial algorithm [3] in the case when the space dimension q is fixed and the components of the vectors in the input set are integer. The time complexity of this algorithm is O(qN(2MB +1) q ),whereb is the maximal absolute value of the vector coordinates in the input set. The algorithm proposed in this paper for the fixed space dimension q allows us to find a (1 + )- approximate solution for given (0, 1) in O(N 2 (M/) q ) time. Taking it into account that M N, this algorithm obviously provides an FPTAS. 2. GEOMETRIC FOUNDATIONS OF THE ALGORITHM First, let us show that there is no a fully polynomial time approximation scheme in the general case of Problem VS-2 under the assumption that P NP. Theorem 1. If P NP then there does not exist any FPTAS for Problem VS-2.

3 AN FPTAS FOR A VECTOR SUBSET SEARCH PROBLEM 331 Proof. In [1], it is shown that Problem VS-3 with integer inputs is strongly NP-hard. VS-3 is a problem of finding in the finite set Y of vectors from R q some subset C that minimizes the function H(C) = x y 2 x C y C with the same constraint on the size of C as in Problem VS-2. Furthermore, in [1], the strongly NP-hard problem [10] of finding a clique of a givensize ina regular graph is reduced to Problem VS-3 with binary vectors. Therefore, Problem VS-3 is strongly NP-hard in the case of integer input data. It is easy that, for the integer input data, the value of H(C) is always integer and bounded by the polynomial of the maximal value of components of the input vectors. By [7], this implies that Problem VS-3 does not admit any FPTAS unless P=NP. Recall [1] that the objective functions of Problems VS-2 and VS-3 satisfy the equality H(C) =2MF(C). Suppose now that there exists some FPTAS for Problem VS-2 with an algorithmic solution C A.Then, by the definition of FPTAS, the following holds: F (C A ) (1 + )F (C ), where C is an optimal solution of the problem. This implies H(C A )=2MF(C A ) (1 + )2MF(C )=(1+)H(C ), which contradicts the inexistence of an FPTAS for Problem VS-3. The proof of Theorem 1 is complete. To justify the algorithm let us prove several auxiliary statements. Lemma 1. Given an arbitrary vector x R q and a finite set Z R q,wehave z x 2 = z z 2 + Z x z 2, (2) where z = 1 z Z is the center of Z. Proof. Summing up the obvious identities z x 2 = z z + z x 2 = z z 2 +2 z z,z x + z x 2, where, is the inner product of vectors, we obtain z x 2 = z z 2 +2 z z,z x + Z z x 2. (3) If suffices to note that the sum of the inner products on the right-hand side of (3) equals zero since (z z) =0. The proof of Lemma 1 is complete.

4 332 KEL MANOV, ROMANCHENKO Given a nonempty finite set Z of vectors from R q,put D(Z) = max x y, x,y Z R(Z) =max z z, where z is the center of Z, while D(Z) is the diameter of Z, andr(z) is the maximal distance of the elements of this set to its center. Some relation between D(Z) and R(Z) is given by Lemma 2. Let Z R q be a nonempty finite set of vectors. Then 1 D(Z) R(Z) D(Z). 2 Proof. The triangle inequality for arbitrary x, y Zand z implies the upper bound x y x z + y z 2max z z =2R(Z). Since this holds for arbitrary x, y Z; therefore, D(Z) = max x y 2R(Z), x,y Z which gives the lower bound for R(Z). Suppose that x Zunder conditions of Lemma 1. Then (2) implies ( z x 2 x z 2 ) 0. z z 2 = z x 2 Z x z 2 = Therefore, at least one of the summands in brackets is nonnegative. Hence, for every x Z, there exists z x Zsuch that x z x z x D(Z). Since this inequality holds for an arbitrary x Z, we have the upper bound The proof of Lemma 2 is complete. max x z = R(Z) D(Z). x Z Given a finite set Z of vectors R q,anaturalm Z, and an arbitrary vector x R q,define the set Z M (x, Z) ={z i z i x z j x ; z i,z j Z,j>M} (4) that consists of M elements of Z closest (by distance) to x and define the maximal distance r M (x, Z) = from x to the vectors from Z M (x, Z). From (4) and (5) it is immediate that we have max z x (5) M (x,z) Lemma 3. Let Z be a finite set of vectors from R q, and let X be a subset of size M of Z. Then (i) for every vector x R q, z x 2 z x 2 ; M (x,z) z X (ii) for every z X, r M (z,z) D(X ). (6) Let us justify the approach for obtaining an approximate solution of Problem VS-2. Definition (4) implies that Z M (x, Y) Y is a feasible solution of VS-2 for every x R q. By Lemmas 1 3, we can state an important property of this subset:

5 AN FPTAS FOR A VECTOR SUBSET SEARCH PROBLEM 333 Lemma 4. Let C be an optimal solution of Problem VS-2, let y(c )= 1 M y C y be the center of C, and let Z M (x, Y) be a feasible solution of Problem VS-2 generated by some x R q.then F (Z M (x, Y)) F (C )+M x y(c ) 2. (7) Proof. Denote the center of the feasible solution of Problem VS 2 by y(z M (x, Y)) = 1 y. M y Z M (x,y) By (1) and (4), we have the following chain of equalities and inequalities: F (Z M (x, Y)) = y y(z M (x, Y)) 2 y x 2 y Z M (x,y) y Z M (x,y) y C y x 2 = y C y y(c ) 2 + M x y(c ) 2 = F (C )+M x y(c ) 2. In this chain the first inequality and last but one equality follow from Lemma 1; while the second inequality, from Lemma 3. The proof is over. Lemma 4 shows that the quality of the feasible solution Z M (x, Y) obtained by some vector x R q can be estimated by the distance from this vector to the center y(c ) of the optimal solution. The closer x to y(c ), the better is solution Z M (x, Y). The essence of the above approach consists in constructing the multidimensional grid whose nodes contain a vector close to the center of the optimal cluster. The sufficient condition forthe meshsize of the grid is provided by Lemma 5. Assume that the conditions of Lemma 4 hold, x R q,andy C.Thenifx satisfies x y(c ) 2 4M r2 M(y,Y) then, for fixed >0, thesetz M (x, Y) is an (1 + )-approximate solution of Problem VS-2. Proof. Lemma 2 for C implies 1 4 D2 (C ) R 2 (C )=max u C u y(c ) 2 u C u y(c ) 2 = F (C ). (8) Furthermore, from (6) for x, C,andY we have Combining (8) and (9), note that x y(c ) 2 Applying (10) to the right-hand side of (7), see that 4M r2 M(y,Y) 4M D2 (C ). (9) M x y(c ) 2 F (C ). (10) F (Z M (x, Y)) (1 + )F (C ). Hence, Z M (x, Y) is a (1 + )-approximate solution of Problem VS-2, which completes the proof of Lemma 5.

6 334 KEL MANOV, ROMANCHENKO Some property of the vectors from an optimal solution is stated by Lemma 6. Assume that the conditions of Lemma 4 hold. Then, for every y C, y y(c ) 2 MrM(y,Y). 2 (11) Proof. From the optimality of C it follows that y y(c ) 2 x y(c ) 2 = F (C ) F (Z M (y,y)). (12) x C On the other hand, by the definitions (4) and (5) of Z M (y,y) and r M (y,y), the following chain of inequalities holds: F (Z M (y,y)) x y 2 MrM 2 (y,y). (13) x Z M (y,y) Combining (12) and (13), we obtain the claim of the lemma. Actually, Lemma 5 defines the area (a ball with center y(c )) such that every element of this area allows us to obtain an approximate solution with given performance guarantee. Moreover, Lemma 6 defines the area of search which contains an unknown center of an optimal solution. 3. AN ALGORITHM FOR SOLVING THE PROBLEM Given an arbitrary y Yand positive h and H, define the vector set B(y,h,H) ={b b = y + h(j 1,...,j q ),j i Z, h j i H, i =1,...,q}. (14) The elements of this set correspond to the nodes of the uniform grid with the center y and the meshsize h. The parameter H defines the geometric sizes of the grid. The number of points of the grid at each coordinate evidently does not exceed H H h h +1 and is independent of y. Remark 1. For each x R q such that y x H,wherey is the center of B(y,h,H), the distance from x to the closest node of B(y,h,H) does not exceed qh/2 since, for each of the q coordinates, this distance obviously does not exceed h/2. Given arbitrary >0 and y Y,put h = qm r M (y,y), (15) H = Mr M (y,y), (16) where r M (y,y) is defined by (5). Let us construct some algorithm for solving the problem that bases on the search for the vector close to the center of an optimal solution. Algorithm A. Input. Some set Y and numbers M and. For each y Y, perform Steps 1 4. Step 1. Calculate r M (y,y), h, andh by (5), (15), and (16) correspondingly. Step 2. If r M (y,y) =0then construct Z M (y,y) by (4) and declare it as the result of the algorithm. Otherwise go to Step 3. Step 3. Construct B(y,h,H) according to (14).

7 AN FPTAS FOR A VECTOR SUBSET SEARCH PROBLEM 335 Step 4. For each b B(y,h,H), constructz M (b, Y) by (4) and calculate F (Z M (b, Y)). Step 5. The result of the algorithm is Z M (b, Y) for which the value of the goal function F (Z M (b, Y)) is minimal. Output. Theorem 2. Given >0, Algorithm A finds a (1 + )-approximate solution of Problem VS-2 in time O(qN 2 (2 qm/ +1) q ). Proof. By the description of the algorithm, there can be two cases at Step 2. In the case of r M (y,y) =0, Lemma 5 implies y y(c ) =0, which, together with Lemma 4, prove the optimality of Z M (y,y). Consider the case r M (y,y) > 0. It is obvious that, while executing the algorithm, each of the vectors y C will be chosen. By Lemma 6, (11) holds for all y C. Then (11) and (16) imply y y(c ) H; i.e., the center of the optimal cluster lies in the area of the grid B(y,h,H). Consider the vector b =arg min b b B(y,h,H) y(c ) closest to the center of C. By Remark 1, from (15) we have b y(c ) 2 qh2 4 = 4M r2 M (y,y). Thus, b satisfies the conditions of Lemma 5; hence, Z M (b, Y) is a (1 + )-approximate solution of Problem VS-2. Let us estimate the time complexity of the algorithm. At Steps 1 and 2, to calculate r M (y,y) and construct Z M (y,y) it suffices to find M vectors from Y closest to y. It can be done with O(N) operations on vectors of dimensionq; i.e., witho(qn) elementary operations [12]. The time of construction of B(y,h,H) at Step 3 is determined by its size. From (15) and (16) it follows that B(y,h,H) (2H/h +1) q =(2 qm/ +1) q. Thus, Step 3 requires O(q(2 qm/ +1) q ) elementary operations. At Step 4, for each vector from B, there are calculated M nearest elements from Y. These calculations require at most O(qN) elementary operations for each vector from B and O(qN(2 qm/ +1) q ) operations for allvectors from B [12]. Therefore, the total complexity of Steps 1 4 for each y Y equals O ( qn(2 qm/ +1) q). Steps 1 4 are executed for each y Y; i.e., N times in total. This implies the final estimation for the algorithm time complexity. The proof of Theorem 2 is complete. Let us show that, for a fixed dimension q, the proposed algorithm provides an FPTAS. Indeed, if (0, 1) then for one of the multipliers in the algorithm time complexity estimation we have ( 2 qm q ( +1) = 2 ( M q + 1 )) q 2 (2 ( ) M q q) q q +1 (2 ( ) M q ( ) M q q) q 2 q =2 2q q q/2 = O((M/) q ) since M/ +1 2M/. This implies that, under the above-stated conditions, the time complexity of Algorithm A equals O(N 2 (M/) q ).SinceM N, the time complexity of Algorithm A is bounded with the polynomial of the problem input size and 1/. This means that the algorithm provides an FPTAS [7].

8 336 KEL MANOV, ROMANCHENKO CONCLUSION Some FPTASis provided in thespecial case ofthe strongly NP-hard problem of finding in a finite set of vectors in Euclidean space a subset of a given size which minimizes the sum of the squared distances from the elements of the subset to its geometric center. The case of fixed space dimension is actual for applications. It is shown that, in the general case of this problem no FPTAS can be constructed unless P=NP. An important direction for further research is the justification of the randomized algorithm in the general case of this problem. ACKNOWLEDGMENTS The authors were supported by the Russian Foundation for Basic Research (projects nos and ) and the Target Program of the Siberian Division of the Russian Academy of Sciences (Integration project no. 7B). REFERENCES 1. A. V. Kel manov and A. V. Pyatkin, NP-Completeness of Some Problems of Choosing a Vector Subset, Diskretn. Anal. Issled. Oper. 17 (5), (2010) [J. Appl. Indust. Math. 5 (3), (2011)]. 2. A. V. Kel manov and S. M. Romanchenko, An Approximation Algorithm for Solving a Problem of Search for a Vector Subset, Diskretn. Anal. Issled. Oper. 18 (1), (2011) [J. Appl. Indust. Math. 6 (1), (2012)]. 3. A. V. Kel manov and S. M. Romanchenko, Pseudopolynomial Algorithms for Certain Computationally Hard Vector Subset and Cluster Analysis Problems, Avtomat. i Telemekh. No 2, (2012) [Automat. Remote Control 73 2), (2012)]. 4. V. V. Shenmaier, An Approximation Scheme for a Problem of Search for a Vector Subset, Diskretn. Anal. Issled. Oper. 19 (2), (2019) [J. Appl. Indust. Math. 6 (3), (2012)]. 5. D. Aloise, A. Deshpande, P. Hansen, and P. Popat, NP-Hardness of Euclidean Sum-of-Squares Clustering, Les Cahiers du GERAD, G (2008) [Machine Learning, 75 (2), (2009)]. 6. K. Anil and K. Jain, Data Clustering: 50 Years Beyond k-means, Pattern Recognit. Lett. 31, (2010). 7. M. R. Garey and D. S. Johnson, Computers and Intractability: a Guide to the Theory of NP- Completeness (Freeman, San Francisco, 1979). 8. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction (Springer, New York, 2001). 9. J. B. MacQueen, Some Methods for Classification and Analysis of Multivariate Observations, in Proceedings of the 5th Berkeley Symposium on Mathematics, Statistics, and Probability (Berkeley, June 21 July 18, 1965; December 27, 1965 January 7, 1966), Vol. 1 (Univ. of California Press, Berkeley, 1967), pp C. H. Papadimitriou, Computational Complexity (Addison-Wesley, New York, 1994). 11. M. Rao, Cluster Analysis and Mathematical Programming, J. Amer. Stat. Assoc. 66, (1971). 12. H. Wirth, Algorithms + Data Structures = Programs (Prentice Hall, New Jersey, 1976).