PROXSCAL. Notation. W n n matrix with weights for source k. E n s matrix with raw independent variables F n p matrix with fixed coordinates

Transcription

1 PROXSCAL PROXSCAL perfors ultidiensional scaling of proxiity data to find a leastsquares representation of the obects in a low-diensional space. Individual differences odels can be specified for ultiple sources. A aorization algorith guarantees onotone convergence for optionally transfored, etric and nonetric data under a variety of odels and constraints. Detailled atheatical derivations concerning the algorith can be found in Coandeur and Heiser (1993). Notation he following notation is used throughout this chapter, unless stated otherwise. For the diensions of the vectors and atrices are used: n Nuber of obects Nuber of sources p Nuber of diensions s Nuber of independent variables h axiu(s, p) l Length of transforation vector r Degree of spline t Nuber of interior nots for spline he input and input-related variables are: n n atrix with raw proxiities for source W n n atrix with weights for source E n s atrix with raw independent variables F n p atrix with fixed coordinates Output and output-related variables are: 1

2 D ˆ n n atrix with transfored proxiities for source Z n p atrix with coon space coordinates A p p atrix with space weights for source X n p atrix with individual space coordinates for source Q n h atrix with transfored independent variables B h p atrix with regression weights for independent variables S l ( r+ t) atrix of coefficients for the spline basis Special atrices and functions are: J D( X ) n n V n n B( X ) n n I 11 / 1 1, centering atrix of appropriate size d, atrix with distances, with eleents { i } where d i = ( xi x )( xi x ) atrix with eleents { v i } atrix with eleents { b i } wi f ( δi ) ( X ) wi for i n, where vi = wil for i = l i, where if di > 0 and i di ( X ) bi = 0 if di ( X ) = 0 and i n bil if i = l i Introduction he following loss function is iniized by PROXSCAL, n 1 σ ˆ w d i di i X, (1.1) = 1 i< which is the weighted ean squared error between the transfored proxiities and the distances of n obect within sources. he transforation function for the

3 3 Preliinaries proxiities provides nonnegative, onotonically nondecreasing values for the transfored proxiities d ˆi. he distances d i ( X ) are siply the Euclidean distances between the obect points, with the coordinates in the rows of X. he ain algorith consists of the following aor steps: 1. find initial configurations X, and evaluate the loss function;. find an update for the configurations X ; 3. find an update for the transfored proxiities d ˆi ; 4. evaluate the loss function; if soe predefined stop criterion is satisfied, stop; otherwise, go to step. At the start of the procedure, several preliinary coputations are perfored to handle issing weights or proxiities, and initialize the raw proxiities. Missings On input, issing values ay occur for both weights and proxiities. If a weight is issing, it is set equal to zero. If a proxiity is issing, the corresponding weight is set equal to zero. Proxiities Only the upper or lower triangular part (without the diagonal) of the proxiity atrix is needed. In case both triangles are given, the weighted ean of both triangles is used. Next, the raw proxiities are transfored such that siilarities becoe dissiilarities by ultiplying with 1, taing into account the conditionality, and setting the sallest dissiilarity equal to zero. ransforations For ordinal transforations, the nonissing proxiities are replaced by their ascending ran nubers, also taing into account the conditionality. For spline transforations, the spline basis S is coputed.

4 4 Noralization he proxiities are noralized such that the weighted squared proxiities equal the su of the weights, again, taing into account the conditionality. Step 1: Initial Configuration PROXSCAL allows for several initial configurations. Before deterining the initial configuration, issings are handled, and the raw proxiities are initialized. Finally, after one of the starts described below, the coon space Z is centered on the origin and optially dilated in accordance with the noralized proxiities. Siplex Start he siplex start consists of a ran-p approxiation of the atrix VBJ. Set H, an n p colunwise orthogonal atrix, satisfying HH= I p equal to I p, where I p denotes the atrix with the first p coluns of the identity atrix. he nonzero rows are selected in such a way that the first Z = B( J) H contains the p coluns of BJ with the largest diagonal eleents. he following steps are coputed in turn, until convergence is reached: 1. For a fixed Z, H = PQ, where PQ is taen fro the singular value decoposition BJZ = PLQ ;. For a fixed H, Z = 1/ V B J H, where V is the pseudo-inverse of V. For a restricted coon space Z, the second step is adusted in order to fullfill the restictions. his procedure was introduced in Heiser (1985). orgerson Start he proxiities are aggregated over sources, squared, double centered and ultiplied with 0.5, after which an eigenvalue decoposition is used to deterine the coordinate values, thus * 0.5JD J = Q 4, where eleents of * D are defined as * ˆ di = widi wi = 1 = 1 1

5 5 1/ followed by Z = Q, where only the first p positive ordered eigenvalues ( λ 1 λ K λ n ) and eigenvectors are used. his technique, classical scaling, is due to orgerson (195, 1958) and Gower (1966) and also nown under the naes orgerson scaling or orgerson-gower scaling. (Multiple) Rando Start he coordinate values are randoly generated fro a unifor distribution using the default rando nuber generator fro the SPSS syste. User-Provided Start he coordinate values provided by the user are used. Step : Configuration Update Update for the Coon Space X he coon space Z is related to the individual spaces = 1,..., through the odel X = ZA, where A are atrices containing space weights. Assue that weight atrix A is of full ran. Only considering Z defines (1.1) as σ z = c + z Hz z t, (3.1) where ( Z) z vec, 1 H ( AA V), = 1 1 vec, = 1 t B X X A (3.) for which a solution is found as z = H t (3.3) Several special cases exist for which (3.3) can be siplified. First, the weights atrices W ay all be equal, or even all equal to one. In these cases H will siplify, as will the pseudo-inverse of H. Another siplification is concerned with the different odels, reflected in restrictions for the space weights. Equation

6 6 (3.3) provides a solution for Z, where A is of full ran. his odel is the generalized Euclidean odel, also nown as IDIOSCAL (Carroll and Chang, 197). he weighted Euclidean odel, or INDSCAL, restricts A to be diagonal, which does siplify H, but not the pseudo-inverse. he identity odel requires A I for all, and does siplify H and its pseudo-inverse, for the ronecer = product vanishes. o avoid coputing the pseudo-inverse of a large atrix, PROXSCAL uses three technical siplifications when appropriate. First, the pseudo-inverse can be replaced by a proper inverse by adding the nullspace, taing the proper inverse and then subtracting the nullspace again as 1 H = H+ N N N= 11 / 1 1. where Furtherore, a diensionwise approach (Heiser and Stoop, 1986) is used which results in a solution for diension a of Z as z = V z, a a a where 1 V V e A A e, a = a a = 1 where e a is the a-th colun of an identity atrix, and 1 za = ( ) a a, B X X A V P A A e = 1 with P a an n p atrix equal to Z, but with the a-th colun containing zeros. Still, the proper inverse of a n n atrix is required. he final siplification is concerned with a aorization function in which the largest eigenvalue of V allows for an easy update (Heiser, 1987; Groenen, Heiser, and Meulan, 1999). Instead of the largest eigenvalue itself, an upper bound is used for this scalar (Wolowicz and Styan, 1980). Update for the Space Weights An update for the space weights ( = 1,..., ) odel is given by A for the generalized Euclidean -1 ( ) A = Z V Z Z B X X. (3.4)

7 7 Suppose, PLQ is the singular value decoposition of A, for which the diagonal atrix with singular values L is in nonincreasing order. hen, for the reduced ran odel, the best r( r < p) ran approxiation of A is given by R, where R contains the first r coluns of PL, and contains the first r coluns of Q. For the weighted Euclidean odel, (3.4) reduces to a diagonal atrix -1 ( ) A = diag Z V Z diag Z B X X. he space weights for the identity odel need no update, since A = I for all. Siplifications can be obtained if all weights W are equal to one and for the reduced ran odel, which can be done in r diensions, as explained in Heiser and Stoop (1986). Restrictions Fixed coordinates If soe of the coordinates of Z are fixed by the user, then only the free coordinates of Z need to be updated. he diensionwise approach is taen one step further, which results in an update for obect i on diension a as p z = e i a i B X X A e = 1 a e A A e V = 1 z e V z % eve eve i a i ia i a a i a ia where the ath colun of Z is divided into za = z% ia +ziaei, with ei the ith colun of 1 the identity atrix, and Va = eaaeav. = 1 his update procedure will only locally iniize (3.1) and repeatedly cycling through all free coordinates until convergence is reached, will provide global optiization. After all free coordinates have been updated, Z is centered on the origin. On output, the configuration is adapted as to coincide with the initial fixed coordinates. Independent variables Independent variables Q are used to express the coordinates of the coon space Z as a weighted su of these independent variables as

8 8 Z QB q b. h = = = 1 An update for Z is found by perforing the following calculations for = 1,..., h: 1.. U h = qb 1 = = 1 = 1 1 C V U A A, where C B( X ) X A = 3. update b as 1 1 = = 1 b q V q A A q 4. optionally, copute optially transfored variables by regressing 1 1 q = b + I V q %, where is greater than or equal to the largest eigenvalue of variable V 1 = = 1 baabv and V, on the original q. Missing eleents in the original variable are replaced with the corresponding values fro Finally, set h = 1 q%. Z QB q b. = = Independent variables restrictions were introduced for the MDS odel in Bentler and Wees (1978), Bloxo (1978), de Leeuw and Heiser (1980) and Meulan and Heiser (1984). If there are ore diensions (p) than independent variables (s), p-s duy variables are created and treated copletely free in the analysis. he transforations for the independent variables fro Step 4 are identical to the transforations of the proxiities, except that the nonnegativety constraint does not apply. After transforation, the variables q are centered on the origin, noralized on n, and the reverse noralization is applied to the regression weights b.

9 9 Step 3: ransforation Update Conditionality wo types of conditionalities exist in PROXSCAL. Conditionality refers to the possible coparison of proxiities in the transforation step. For unconditional transforations, all proxiities are allowed to be copared with each other, irrespective of the source. Matrix-conditional transforations only allow for coparison of proxiities within one atrix, in PROXSCAL refered to as one source. Here, the transforation is coputed for each source seperately (thus ties). ransforation Functions All transforation functions in PROXSCAL result in nonnegative values for the transfored proxiities. After the transforation, the transfored proxiities are noralized and the coon space is optially dilated accordingly. he following transforations are available. Ratio D ˆ =. No transforation is necessary, since the scale of ˆD is adusted in the noralization step. Interval D ˆ = α + β. Both α and β are coputed using linear regression, in such a way that both paraeters are nonnegative. Ordinal D ˆ = WMON, :. Weighted onotone regression (WMON) is coputed using the up-and-down-blocs iniu violators algorith (Krusal, 1964; Barlow et al., 197). For the secondary approach to ties, ties are ept tied, the proxiities within tieblocs are first contracted and expanded afterwards.

10 10 Spline ( D ˆ ) vec = Sb. PROXSCAL uses onotone spline transforations (Rasay, 1988). In this case, the spline transforation gives a sooth nondecreasing piecewise polynoial transforation. It is coputed as a weighted regression of D on the spline basiss. Regression weights b are restricted to be nonnegative and coputed using nonnegative alternating least squares (Groenen, van Os and Meulan, 000). Noralization After transforation, the transfored proxiities are noralized such that the su-of-squares of the weighted transfored proxiities are equal to n(n-1)/ in the unconditional case and equal to n(n-1)/ in the atrix-conditional case. Step 4: erination After evaluation of the loss function, the old function value and new function values are used to decide whether iterations should continue. If the new function value is saller than or equal to the iniu Stress value MINSRESS, provided by the user, iterations are terinated. Also, if the difference in consecutive Stress values is saller than or equal to the convergence criterion DIFFSRESS, provided by the user, iterations are terinated. Finally, iterations are terinated if the current nuber of iterations, exceeds the axiu nuber of iterations MAXIER, also provided by the user. In all other cases, iterations continue. Reaining Issues Acceleration For the identity odel without further restictions, the coon space can be new update old updated with acceleration as Z = Z Z, also refered to as the relaxed update. Lowering diensionality For a restart in p-1 diensions, the p-1 ost iportant diensions need to be identified. For the identity odel, the first p-1 principal axes are used. For the

11 11 weighted Euclidean odel, the p-1 ost iportant space weights are used, and for the generalized Euclidean and reduced ran odels, the p-1 largest singular values of the space weights deterine the reaining diensions. Stress easures he following statistics are used for the coputation of the Stress easures: η η η η ρ ρ κ ( Dˆ ) n = ˆ widi = 1 i< n 4 4 ( Dˆ ) = ˆ widi = 1 i< n i i = 1 i< n 4 4 i i = 1 i< n ( X) = w d ( X ) ( X) = w d ( X ) ( X) = w dˆ d ( X ) i i i = 1 i< n i i i = 1 i< ( X) = w dˆ d ( X ) n ( X) = w d ( X ) d( X) = 1 i< i i where d ( X ) is the average distance. he loss function iniized by PROXSCAL, noralized raw Stress, is given by: σ = ( Dˆ ) + ( X) ( X) η ( Dˆ ) η η α ρ α, with ρ α = η ( X) X. Note that at a local iniu of X, α is equal to one. he other Fit and Stress easures provided by PROXSCAL are given by: η ( Dˆ ) + η ( αx) ρ( αx) η ( Dˆ ) Stress-I:, with α = η αx ρ X.

12 1 Stress-II: ( Dˆ ) + ( X) ( X) η η α ρ α κ ( αx) S-Stress: η 4 ( ˆ ) + η 4 ( α ) ρ ( α ), with ( D) η ˆ α = ρ ρ D X X, with α = η Dispersion Accounted For (DAF): ucer s coefficient of congruence: 1 σ. 1-σ. ( X) 4 X. X. Decoposition of noralized raw Stress Each part of noralized raw Stress, as described before, is assigned to obects and sources. Either su over obects or su over sources are equal to total noralized raw Stress. ransforations on output On output, whenever fixed coordinates or independent variables do not apply, the odels are not unique. In these cases transforations of the coon space and the space weights are in order. For the identity odel, the coon space Z is rotated to principal axes. For the weighted Euclidean odel, = n ( diag ) 1/ diag ( ZZ ) = n Z Z Z Z so that I, and reverse tranforations are applied to the space weights A. Further, the su over sources of the squared space weights are put in descending order as to specify the iportance of the diensions. For the generalized Euclidean odel, the Cholesy decoposition ZZ= LL specifies the coon space on output as 1 Z= nz L, so that ZZ= ni.

13 13 References Barlow, R. E., Bortholoew, D. J., Brener, J. M. and Brun, H. D. (197). Statistical inference under order restrictions. New Yor: Wiley. Bentler, P. M. and Wees, D. G. (1978). Restricted ultidiensional scaling odels. Journal of Matheatical Psychology, 17, Bloxo, B. (1978). Constrained ultidiensional scaling in n spaces. Psychoetria, 43, Carroll, J. D. and Chang, J. J. (197, March). IDIOSCAL (Individual Differences In Orientation SCALing): A generalization of INDSCAL allowing idiosyncratic reference systes as well as analytic approxiation to INDSCAL. Paper presented at the spring eeting of the Psychoetric Society. Princeton, NJ. Coandeur, J. J. F. and Heiser, W. J. (1993). Matheatical derivations in the proxiity scaling (PROXSCAL) of syetric data atrices (ech. Rep. No. RR ). Leiden, he Netherlands: Departent of Data heory, Leiden University. De Leeuw, J. (1977). Applications of convex analysis to ultidiensional scaling. In J. R. Barra, F. Brodeau, G. Roier, and B. van Cutse (Eds.), Recent developents in statistics (pp ). Asterda, he Netherlands: North- Holland. De Leeuw, J. and Heiser, W. J. (1980). Multidiensional scaling with restrictions on the configuration. In P. R. Krishnaiah (Ed.), Multivariate analysis (Vol. V, pp ). Asterda, he Netherlands: North-Holland. Gower, J. C. (1966). Soe distance properties of latent root and vector ethods used in ultivariate analysis. Bioetria, 53, Groenen, P. J. F., Heiser, W. J. and Meulan, J. J. (1999). Global optiization in least squares ultidiensional scaling by distance soothing. Journal of Classification, 16, Groenen, P. J. F., van Os, B. and Meulan, J. J. (000). Optial scaling by alternating length-constained nonnegative least squares, with application to distance-based analysis. Psychoetria, 65,

14 14 Heiser, W. J. (1985). A general MDS initialization procedure using the SMACOF algorith-odel with constraints (ech. Rep. No. RR-85-3). Leiden, he Netherlands: Departent of Data heory, Leiden University. Heiser, W. J. (1987). Joint ordination of species and sites: he unfolding technique. In P. Legendre and L. Legendre (Eds.), Developents in nuerical ecology (pp ). Berlin, Heidelberg: Springer-Verlag. Heiser, W. J. and De Leeuw, J. (1986). SMACOF-I (ech. Rep. No. UG-86-0). Leiden, he Netherlands: Departent of Data heory, Leiden University. Heiser, W. J. and Stoop, I. (1986). Explicit SMACOF algoriths for individual differences scaling (ech. Rep. No. RR-86-14). Leiden, he Netherlands: Departent of Data heory, Leiden University. Krusal, J. B. (1964). Nonetric ultidiensional scaling: A nuerical ethod. Psychoetria, 9, 8-4. Meulan, J. J. and Heiser, W. J. (1984). Constrained Multidiensional Scaling: ore Directions than Diensions. In. Havrane et al (Eds.), COMPSA 1984 (pp ). Wien: Physica Verlag. Rasay, J. O. (1988). Monotone regression splines in action. Statistical Science, 3(4), Stoop, I., Heiser, W. J. and De Leeuw, J. (1981). How to use SMACOF-IA. Leiden, he Netherlands: Departent of Data heory, Leiden University. Stoop, I. and De Leeuw, J. (198). How to use SMACOF-IB. Leiden, he Netherlands: Departent of Data heory, Leiden University. orgerson, W. S. (195). Multidiensional scaling: I. heory and ethod. Psychoetria, 17, orgerson, W. S. (1958). heory and ethods of scaling. New Yor: Wiley. Wolowicz, H. and Styan, G. P. H. (1980). Bounds for eigenvalues using traces. Linear algebra and its applications, 9,