Concavity in Data Analysis

Transcription

1 Journal of Classification 20:77-92 (2003) 77 Concavity in Data Analysis Michael P. Windham University of South Alabama Abstract: Concave functions play a fundamental role in the structure of and minimization of badness-of-fit functions used in data analysis when etreme values of either parameters or data need to be penalized. This paper summarizes my findings about this role. I also describe three eamples where concave functions are useful: building measures of badness-of-fit, building robust M-estimators, and building clustering objective functions. Finally, using the common thread of concavity, all three will be combined to build a comprehensive, fleible procedure for robust cluster analysis. Keywords: Optimization, Objective functions, Least squares, Partitioning, Badness-of-fit, k-means, Miture analysis, Fuzzy clustering, M-Estimators, Robust cluster analysis. Author s address: Michael P. Windham, Department of Mathematics and Statistics, University of South Alabama, Mobile, AL , USA

2 78 Michael P. Windham 1. Introduction Twenty years ago I gave a lecture on cluster analysis wherein I stated that certain clustering algorithms involved minimizing an objective function built with a concave function. A member of the audience insisted that one does not minimize concave functions. Admitting that this person was correct, I spent the remainder of the year trying to eplain to him and to myself why concavity was there and why it was the reason the algorithm worked. Since then I have continued to work in cluster analysis and also in robust statistical estimation and optimization. I have found many times that concave functions have played a fundamental role in the structure of and minimization of badness-of-fit functions used in these areas of data analysis. This paper describes that role. As eamples, I will describe three instances of the use of concave functions: building measures of badness-of-fit, building robust M-estima tors, and building clustering objective functions. In each of these cases etreme values of either parameters or data need to be penalized. I will discuss how and why concavity is involved in the penalizing process. Finally, I will put all three together to build a comprehensive, fleible fit criterion for robust cluster analysis. Concavity also facilitates the minimization of the fit criteria using iterative majorization (Heiser, 1995) as will be described in Section 7. This paper organizes and unifies the common thread of concavity that runs through the several eamples. The role of concavity has not always been recognized in data analysis, and a fundamental role it is. The latter is the main point of this paper. 2. Concave Functions Concave functions are involved in data analysis in reducing the influence of etreme values of some measure of fit and optimizing an objective function built from the measure. Before eamining specific uses of concave functions, I will review the basic facts about them that will be useful here. There are many characterizations of concave functions, but the following is the best for our purposes. A function, f : R p R is concave on a set S, if for each r in S there is a 1 p matri, F(r), satisfying for all r and r 0 in S f(r) F(r 0 )(r r 0 ) + f(r 0 ). (1) When f is tiable, the matri F(r) is its derivative and concavity simply says that the graph of f always lies below its tangent planes. For

3 Concavity in Data Analysis 79 None Delayed 1 Smooth, unbounded Bounded Figure 1: Types of concave functions. this paper, I will use lower case letters for a concave function and the same upper case letter for the 1 p matri valued function required for the linear term in (1). Another, perhaps more common definition of concavity is that for any r 1 and r 2 in a conve set S and 0 1 f( r 1 + (1 )r 2 ) (r 1 ) + (1 )f(r 2 ) That is, the value of f on a conve combination of vectors eceeds the conve combination of the values. Suppose that r represents a measure of fit between data and parameters describing data for which large values suggest poor fit. The of concave functions on reducing the influence of etreme values is shown in Figure 1. In each case, f(r) increases as r does, but f(r) r and the increases as r increases, so that the influence of large values of r is reduced. In building the measures of fit themselves, Section 3, concavity is used to penalize etreme values of the parameters in the measure, so that desirable minima eist. In robust estimation, Section 4, concavity reduces the influence of outliers, and in cluster analysis, Section 5, concavity localizes on a cluster and reduces the influence in describing the cluster of data from other clusters.

4 80 Michael P. Windham 3. Parameter Estimation Using Measures of Fit The basic goal of eploratory data analysis is to summarize data with manageable quantities that describe some informative characteristics of the data. For eample, for a set X of n data points in R d, the mean = /n gives a central location for the data, and the covariance or scatter matri, Ŝ = ( )( )T /n describes the shape of the data set in terms of ellipsoids determined by a natural choice of a metric, ( ) T Ŝ 1 ( ). It is even more satisfying when a natural choice is a best choice in some sense. For eample, the squared Euclidean norm, m 2, is a measure of the fit between a data point with a possible center of concentration m, in that the further m is from, the more incompatible they are with each other. It is well-known that the mean is the value of m that minimizes m 2, that is is a best choice for a center of a data set in that it is on the average least incompatible with the data in the set. It would be nice to have a similar measure of fit that leads to Ŝ being a best choice. A natural candidate would be to minimize d(m, S) = ( m)t S 1 ( m), particularly since for any positive semidefinite S, we have d(, S) d(m, S) for all m. It then to minimize d(, S) = ( ) T S 1 ( ) = n tr(ŝs 1 ), where tr( ) denotes the trace of a matri. Unfortunately, the function d has no minimum as a function of S. The problem is that the larger S is (in terms of say eigenvalues), the smaller d is. People have added penalty terms to the measure to discourage large values of scale. Finding such terms leads to the first use of concavity, namely using concave functions to penalize large values of scale in measuring fit. The first step is to describe how to etend a function g : R R to a function assigning symmetric matrices to symmetric matrices. The function g is etended to a symmetric matri S by letting g(s) = B diag(g( 1),..., g( d))b T, where B is an orthogonal matri whose columns are a basis of eigenvectors of S and 1,..., d are the corresponding eigenvalues. Using this notion, we have the following. If g is concave and increasing, and G satisfies g(s) G(s 0 )(s s 0 ) + g(s 0 ) for all s and s 0, then D(m, S) = ( m) T G(S)( m) + tr(g(s) G(S)S) (2) is minimized by m = and S = Ŝ, the sample mean and covariance matri. Moreover, if g(0) 0 then ( m) T G(S)( m) + tr(g(s) G(S)S) 0 for all, m, and positive semidefinite S.

5 !!! Concavity in Data Analysis 81 This result was presented in Windham (2000) without proof, so I will give one here. Clearly, for a fied, positive semidefinite S, D(m, S) is minimized for m =. It then to show that D(, S) = n tr(g(s)(ŝ S)+g(S)) is minimized as a function of S by S = Ŝ. Letting ˆ be a diagonal matri of eigenvalues of Ŝ, it follows by moving orthogonal matrices around in the trace function that, it to show that tr(g(s)( ˆ S)+g(S)) g(ˆ i i) for all positive semidefinite S. Letting S = " " T and again moving the orthogonal matri B, we have from the concavity of g tr(g(s)(! ˆ S) + g(s)) = i G( i) j b 2 ijˆ j i + g( i) i g j b 2 ij ˆ j. Since rows and columns of B are orthonormal vectors, i b 2 ij = j b 2 ij = 1, so that j b2 ijˆ j is a conve combination of the eigenvalues of Ŝ. Therefore, from the concavity of the function g, we have i g j b2 ijˆ j i j b2 g(ˆ ij j) = j g(ˆ j) and the result follows. Moreover, if g(0) 0, then as above, ( m) T G(S)( m) + tr(g(s) G(S)S) = D(m, S) for a data set consisting of alone, so that D(m, S) is minimized by m = and S = 0. That is, the minimum value of ( m) T G(S)( m) + tr(g(s) G(S)S) is D(, 0) = g(0) 0, completing the proof. It should be noted that since g is increasing, we have G 0, so that G(S) is at least positive semidefinite. Therefore, ( m) T G(S)( m) + tr(g(s) G(S)S) acts like a metric on the data space. The goal was to build a fit function that is minimized by the mean and scatter matri of the data. Any function of the form in (2) is minimized by as long as G(S) is positive semidefinite, which requires only that G be non-negative. If so, then one needs to minimize tr(g(s)(ŝ S) + g(s)) as a function of S. For simplicity consider the one-dimensional case with g any twice tiable function. For d(s) = g# (s)(ŝ s) + g(s), we have d# (s) = g# # (s)(ŝ s), so that ŝ is a critical point. It is the concavity of g that ensures that ŝ is, in fact, a minimizer. For eample, with g(s) = log(s), then G(s) = 1/s, and we have the measure ( m) T S 1 ( m) + log det S, ignoring constant terms. This

6 82 Michael P. Windham eample is popular, but does not satisfy the condition that g(0) = 0, which ensures that the measure of fit is non-negative, and facilitates its use in the applications that follow. The function g(s) = log(1 + s), for eample, does satisfy g(0) = 0 and yields ( m) T (I + S) 1 ( m) + tr(log(i + S) (I + S) 1 S). The second column of Table 1 lists several functions that can be used for g. For functions in the table with two descriptions, ecept for the Cauchy, the first epression applies for 0 r 1 and the second for r > 1. We have now a procedure for building badness-of-fit measures that are minimized by the mean and scatter matri of a data set and that have properties that facilitate their use in data analysis. 4. Robust M-estimation The notion of choosing descriptors for a data set by minimizing total badness-of-fit between points in the set and some summary descriptor of the data is called M-estimation in the contet of robust statistics. Huber (1981) and Hampel, Ronchetti, Rousseeuw, and Stahel (1986) discuss M-estimators in detail and give a variety of procedures for estimating location and shape. In general, if t is a descriptor of a data set and r(, t) is a measure of fit between a data point and a descriptor t, then the M-estimator for t is the value of t that minimizes r(, t). The discussion of the previous section provides measures of fit for M-estimation of location and shape. Some estimators are not robust, in that their values are sensitive to a few data points that may not represent the information in the bulk of the data. The mean, for eample, is sensitive to outliers. The median, on the other hand, is robust to the presence of outliers. The median is also an M-estimator, in that, it is the value of m that minimizes m. The measure of fit used for the median is the composition with the measure used for the mean and a concave, increasing function. In particular, m = m 2. In other words, we have f(r) = r with r = m 2. Therefore, the median should be less sensitive than the mean to outliers because the concavity of the square root reduces their influence. This observation is easily generalized to provide a structure for robustizing M-estimators. For a function r(, t) measuring fit between and t and a concave, increasing function h, let $ (, t) = h(r(, t)). Since h is increasing, $ is also a measure of fit with the influence of etreme incompatibility reduced by the concavity of h. Therefore, minimizing $ (, t) = h(r(, t)) should lead to more robust estimates of t

7 Concavity in Data Analysis 83 Table 1: Robustizers. h(r) % None r 0 Median r Huber r 2r Biweight 1 3 (1 (1 r)3 ) Cauchy for &(' ) ((1 + r) 1 ) 1),&+*= 1 log(1 + r), & = Fair 2(r 1 2 log(1 + r 1 2 )) Logistic 2 log(cosh(r 1 2 )) Talwar r , Welsch 1 e r Andrews 2 2 (1 cos(- r 1 2 )) For functions with two descriptions, ecept for the Cauchy, the first epression applies for 0. r. 1 and the second for r > 1. Tuning constants / are discussed in Section 4. 4, 2 than would be obtained by minimizing r(, t). There are many robust M-estimators of location in the literature. What is not so widely known is that often these can be obtained by composing m 2 with some concave function. A list of the estimators is given in Table 1 along with the concave function h. These concave functions can be used with any measures of fit as was illustrated by Verboon and Heiser (1992) who applied the Huber and biweight functions to squared residuals for orthogonal Procrustes analysis. One caution that must be observed is that the measure of fit is changed by simply rescaling the data. Doing so artificially changes the point in the data space where the robustizing has a particular level of influence. Tuning constants are used to compensate for data scale, that is, use $ (, t) = h(0 r(, t)) 0 for a positive tuning constant 0. Dividing by 0 is not necessary, but doing so often results in 0 = 0 corresponding to no robustizing. This situation occurs when h is tiable at zero and h# (0) = H(0) = 1. Under these conditions $ (, t) approaches r(, t) as 0

8 84 Michael P. Windham approaches zero. In fact, whenever h is tiable at zero, adjusting h by constants will make h(0) = 0 and H(0) = 1 without the fit interpretation. The functions in Table 1 satisfy these conditions ecept for the median. Tuning constants can also compensate somewhat for the t levels concavity in t choices for h. Possible tuning constants for each method are also given in Table 1. These are based on values appearing in Holland and Welsch (1977) for achieving 95% 5 in estimating location under a standard normal model. For clarity, I will not include tuning constants in what follows, but one can apply them as above, if desired. 5. Cluster Analysis Cluster analysis attempts to identify substructure in a data set or population by grouping objects together into clusters based on their being more similar to each other than to other objects in the study. Objective function methods determine descriptions of clusters that minimize a measure of fit between the descriptions and data describing the objects. The problem is that one needs to identify and describe a subset of the data that forms a cluster in the presence of other data that should not belong to the cluster. We need to find a description of a cluster by identifying concentrations of objects that best fit a common description while reducing the of objects tly removed from the description that they should be identified with other clusters. Concavity does precisely this. We will see that the contribution to the total fit of a data point is greatest for the cluster description with which it is most compatible, while the influence of fit for other clusters is reduced or even ignored. For an individual cluster, we will use r(, t) as above to measure fit between a data point and a cluster descriptor t, such as location and shape. Three famous objective function clustering procedures are partitioning methods, miture analysis, and fuzzy clustering. Partitioning: For C 1,..., C k a partition of the data into clusters and descriptors t 1,..., t k, one for each cluster, i 6 C i r(, t i ) is the total fit between the data and the given cluster structure, so that one finds the C 1,..., C k and t 1,..., t k that minimize the badness-of-fit.

9 Concavity in Data Analysis 85 Miture analysis: This approach assumes that the data are a sample from a population whose probability density is a finite miture of densities of the form e r(,t i) and the structure is determined by finding t and p = (p 1,..., p k ) to maimize log i p i e r(,t i) where p i is the probability of belonging to the i-th subpopulation. In this case, t and p are maimum likelihood estimates. Redner and Walker (1984) survey the topic. Fuzzy clustering: This method is based on the notion that some concepts that appear to define sets really do not. For eample, who is in the set of tall people? There is no clear 879 in height that would universally be accepted as the dividing line between tall and not tall. A fuzzy set is a function that assigns to each object a number between zero and one that measures the degree to which the object has the property that the set represents. A fuzzy partition of R d is a vector of k such functions defined on R d, u = (u 1,..., u k ), with u i the membership function for the i-th cluster and restricted so that i u i() = 1 for each. A fuzzy clustering is obtained by finding u and t to minimize i u m i () r(, t i) where m > 1 is a tuning constant used to adjust the fuzziness of the clusters. The larger m is the fuzzier the clusters will be and the closer m is to one, the closer the results are to partitioning. Bezdek and Pal (1991) survey the literature on the subject. These three objective functions can be converted into compositions of the measures fit for each of the clusters given by r(, t) = (r(, t 1 ),..., r(, t k )) with concave functions, increasing in each variable. These functions are obtained for partitioning and fuzzy clustering, by replacing the clusters with the sets C 1,..., C k and membership functions u 1,..., u k that minimize the respective objective functions for given descriptors, namely C i = { : r(, t i ) = min j r(, t j )} and u i () = r(, t i ) 1/(1 m) / j r(, t j) 1/(1 m). The conversion for mitures just puts a minus sign in front to change maimizing to minimizing. The function r:i (, t, p) = r(, t i) log p i is used to measure fit for mitures. In fact, the same can be done for partitioning and fuzzy clustering, so that all

10 86 Michael P. Windham Table 2: Concave functions in cluster analysis. Method a A i Partitioning min i (r i ) 1 if r i = min j (r j ) 0 otherwise Miture log( i e r i ) e r i j e r j Fuzzy ( i r1/(1 m) i ) 1 m r 1/(1 m) i j r1/(1 m) j m a P 0 r 0 1 (a) r a a (b) F M Figure 2: Concavity in cluster analysis. three contain a parameter p = (p 1,..., p k ) satisfying i p i = 1 and describing how the data are apportioned among the clusters. The concave functions, a, are shown in Table 2. The resulting fit functions are Partitioning: min i r:i (, t, p) Miture: log i e r; i (,t,p) Fuzzy clustering: i r: 1 m i (, t, p)1/1 m A simplified illustration will show how concavity facilitates cluster analysis. In Figure 2(a) the line r 0 represents a measure of incompatibility with 0 in that the further is from 0 the greater the measure. Similarly, r 1 represents a measure of incompatibility with 1. Figure 2(b)

11 Concavity in Data Analysis 87 shows the associated functions from Table 2 applied to r 0 and r 1, namely, for partitioning a P = min(r 0, r 1 ), for mitures a M = log(e r 0 + e r 1 ) translated by a constant for the purpose of the illustration so that its graph goes through goes through 0 and 1, and for fuzzy clustering a F = (r0 1 + r1 1 ) 1, using m = 2. In terms of incompatibility, a P ignores incompatibility with 1 for closer to 0 than to 1 and ignores incompatibility with 0 for closer to 1 than to 0. The concavity ensures that near 0 the sense of incompatibility is preserved, that is incompatibility with 0 increases as you move away from 0 and similarly for near 1. The functions for miture and fuzzy clustering are simply smoother versions of the same phenomenon. One disadvantage to this approach is that the sense of cluster is lost. The parameters t i describe the clusters in terms of characteristics like location and shape, but what are the clusters as sets of objects? Fortunately, the notion of cluster membership is still present in the A i s. In particular we have the following interpretations. Partitioning: C i = { : r:i (, t, p) = min j r:j (, t, p)} = { : A i(r: (, t, p)) = 1} is the subset of the data most compatible with the i-th cluster description, that is C i is the i-th cluster. Mitures: A i (r: (, t, p)) = p i e r(,ti) / j p je r(,tj) is the estimate for the conditional probability of belonging to the i-th subpopulation knowing the data. Fuzzy clustering: The fuzzy membership function for the ith fuzzy cluster is u i () = A i (r: (, t, p)) 1 m = r:i (, t, p)1/(1 m) / j r:j (, t, p)1/(1 m). The clustering procedures I have discussed so far use continuously varying parameters to describe the clusters. Many procedures do not. For eample, the medoid method described in Kaufman and Rousseeuw (1990) attempts to identify objects among those under study to serve as descriptors for the location of the clusters, the medoids. The cluster associated to a given medoid consists of the objects the least dissimilar to it. The medoids are chosen to be the objects for which the sum of the dissimilarities to them of the objects associated to them is the least. We can formulate a badness-of-fit function whose minimum identifies

12 88 Michael P. Windham the medoids and their clusters as follows. Let {j 1,..., j k } be the indices of k distinct objects that are candidates for medoids, and d jl denote the dissimilarity between the j-th and l-th objects, then the badness-of-fit function is min min d ji l {j 1,...,j k } i l The concave function min( ) plays its usual role. In particular, min i ignores objects not associated to the i-th medoid and min {j1,...,j k } ignores objects not chosen to be medoids. Windham (1985) describes a fuzzy version of the same idea. The reader is encouraged to create a miture version as well. 6. Robust Cluster Analysis Robust cluster analysis is now quite straight forward to achieve. Simply replace r(, t i ) by h(r(, t i )). This approach was introduced for miture analysis in Windham (2000), but the concavity in both cluster analysis and M-estimation makes it possible to use the same idea in partitioning and fuzzy clustering. In particular, for any function a from Table 2, the badness-of-fit function becomes a(r(, t 1 ) log p 1,..., r(, t k ) log p k ) = a(r: (, t, p)) a(h(r(, t 1 )) log p 1,..., h(r(, t k )) log p k ) = a(< : (, t, p)) Figure 3 shows the impact of robustizing in an eample presented in Windham (2000). Figure 3(a) shows the results of finding three clusters using a miture clustering function with a Cauchy measure of fit, based on g(s) = log(1+s). The confusion caused by the presence of the outliers is apparent. Figure 3(b) shows the results obtained with the miture clustering, Cauchy fit, and the Welsch robustizer with tuning constant The robustized result clearly identifies the structure of the bulk of the data without being unduly influenced by the outliers. 7. Minimization Procedures The role that concavity plays in constructing minimizing algorithms has been recognized for some time. A comprehensive discussion is given in Heiser (1995) where it is viewed as a special case of iterative majorization. It is also described in Lange, Hunter and Yang (2000), which

13 Concavity in Data Analysis 89 (a) (b) Figure 3: Cluster analysis with outliers. includes the algorithm for finding the median as an eample. Both of these articles provide a wealth of references to many other applications of the idea in constructing algorithms. The iterative majorization with concave functions is based on the following consequence of concavity. From (1), one can see that if you are at r 0 then you can decrease f(r) by decreasing F(r 0 )r. In particular, if F(r 0 )r F(r 0 )r 0, (3) then f(r) F(r 0 )(r r 0 ) + f(r 0 ) f(r 0 ). Iteratively applying (3) will produce a decreasing sequence of values of f. If f is bounded below, the sequence of function values, f(r), will converge. Finally, if f is also increasing in each variable, then F i (r 0 ) 0, so F(r 0 )r can be decreased by simply decreasing each variable, r i, and decreasing f(r) has then been reduced to decreasing a weighted version of r. Decreasing r may not be = 3> In fact, in applications r is often a function of some parameter important to the contet and we are seeking the value of the parameter that minimizes f(r). The function F(r 0 )r is often, at least locally, a conve function of that parameter and can be easily minimized. The articles mentioned above also contain detailed discussions of the convergence properties of the algorithms. I mentioned that when the objective function is bounded below a decreasing sequence of iterates of the function itself will converge. More important is the question of whether the corresponding sequence of parameters converges. In fact,

14 90 Michael P. Windham there is no guarantee that they do converge and even when they do they may not converge to minimizer. These articles along with Bezdek, Hathaway, Howard, Wilson and Windham (1987) and Windham (1987) discuss this problem. The following describes the iterative procedures for robust estimation, cluster analysis, and finally robust cluster analysis. Each of them is the implementation of (3). The function F becomes a function H, such as the derivative of an h from Table 1 in robust estimation, and A = (A 1,..., A k ) from Table 2 in cluster analysis. Robust Estimation: Robust estimators can be computed from the iterative process based on (3) simply by using the appropriate choice of H, the derivative of a concave, increasing function, in (4) below. The iterations proceed from the current value of t, t c, to net, t +. We will use arg min l(t) to denote the value of t that minimizes l(t). Beginning t with the initial t c to be the value of t that minimizes r(, t), the iteration is t + = arg min H(r(, t c ))r(, t) (4) t The algorithm reduces the problem of minimizing the robustized criterion to iteratively minimizing a weighted version of the unrobustized function, a problem that is usually easy to solve. For eample, if t = (m, S) and r(, t) = ( m) T G(S)( m) + tr(g(s) G(S)S) as in Section 3, then the parameter updates are as follows. Letting w c () = H(r(, t c ))/ H(r(, tc )), we have m + = S + = w c () w c ()( m + )( m + ) T. The value w c () simply weights the contribution of in accordance with the of the concave function h used for robustness. Clustering: The algorithm for minimizing the objective functions iterates from t c, p c to t +, p + by t + i = arg min t p + i = arg ma p i = A i (r: (, t c, p c ))r(, t) A i (r: (, t +, p c )) log p i A i(r: (, t +, p c )) j A j(r: (, t +, p c )).

15 Concavity in Data Analysis 91 In the case where the parameters are location and scale, that is t i = (m i, S i ), the updates for those parameters are m + i = S + i = w c i () w c i ()( m + )( m + ) T where wi c() = A i(r: (, t c, p c ))/ A i(r: (, t c, p c )) weights the contribution of to the net parameter estimates for the i-th cluster according to its membership in the i-th cluster as described by the current parameter estimates. Robust Clustering: t + i = arg min t p + i = A i (< : (, t c, p c ))H(r(, t c i ))r(, t) A i(< : (, t +, p c )) j A j(< : (, t +, p c )) When the parameters are location and scale, the updates for those parameters are where the weight m + i = S + i = w c i () w c i ()( m+ )( m + ) T w c i () = A i (< : (, t c, p c ))H(r(, t c i ))/ A i (< : (, t c, p c ))H(r(, t c i )) has two terms, one for clustering and the other for robustness. 8. Conclusion The influence of etreme values on parameter estimation in data analysis can be controlled with concave functions without seriously complicating the problem to be solved. This paper illustrates this fact with three eamples where t things need to be controlled. In the first, etreme values of scale were controlled so that well-behaved measures of fit could be built that would have the covariance or scatter matri as

16 92 Michael P. Windham a minimizer. In the M-estimation eample, the influence of outliers on parameter estimates was controlled with concave functions. Finally, in finding a description of a cluster in a cluster analysis, the influence of data from other clusters is reduced by combining fits to cluster descriptions with concave functions. Moreover, the common thread of applying concave functions to control etreme values allows one to combine all three to build a procedure for robust cluster analysis, that estimates location and scale within a cluster. References BEZDEK, J.C., HATHAWAY, R.J., HOWARD, R.E., WILSON, C.A. and WIND- HAM, M.P. (1987), Local convergence analysis of a grouped variable version of coordinate descent, Journal of Optimization Theory and Applications, 54, BEZDEK, J.C. and PAL, S.K. (1991), Fuzzy Models for Pattern Recognition, New York: IEEE Press. HAMPEL, F.R., RONCHETTI, E.M., ROUSSEEUW, P.J. and STAHEL, W.A. (1986), Robust Statistics: The Approach Based on Influence Functions, New York: Wiley. HEISER, W.J. (1995), Convergent computation by iterative majorization: theory and applications in multidimensional data analysis, Recent Advances in Descriptive Multivariate Analysis, Ed. W. Krzanowski, Oford: Clarendon Press, HOLLAND, P.W. and WELSCH, R.E. (1977), Robust regression using iteratively reweighted least-squares, Communications in Statistics Theory and Methods, A6(9), HUBER, P.J. (1981), Robust Statistics, New York: Wiley. KAUFMAN, L. and Rousseeuw, P. (1990), Finding Groups in Data: An Introduction to Cluster Analysis, New York: Wiley. LANGE, K., HUNTER, D. and YANG, I. (2000), Optimization transfer using surrogate objective functions, Journal of Computational and Graphical Statistics, 9, REDNER, R.A. and WALKER, H.F. (1984), Miture densities, maimum likelihood and the EM algorithm, SIAM Review, 26, VERBOON, P. and HEISER, W.J. (1992), Resistant orthogonal Procrustes analysis, Journal of Classification, 9, WINDHAM, M.P. (1985), Numerical classification of proimity data with assignment measures, Journal of Classification, 2, WINDHAM, M.P. (1987), Parameter modification for clustering criteria, Journal of Classification, 4, WINDHAM, M.P. (2000), Robust clustering, Data Analysis: Scientific Modeling and Practical Application, Eds. W. Gaul, O. Opitz and M. Schader, Berlin: Springer,