Composite Likelihood Estimation for Latent Variable Models with Ordinal and Continuous, or Ranking Variables

Transcription

1 Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Social Sciences 86 Composite Likelihood Estimation for Latent Variable Models with Ordinal and Continuous, or Ranking Variables MYRSINI KATSIKATSOU ACTA UNIVERSITATIS UPSALIENSIS UPPSALA 2013 ISSN ISBN urn:nbn:se:uu:diva

2 Dissertation presented at Uppsala University to be publicly examined in Hörsal 2, Ekonomikum, Kyrkogårdsgatan 10, Uppsala, Friday, February 15, 2013 at 10:15 for the degree of Doctor of Philosophy. The examination will be conducted in English. Abstract Katsikatsou, M Composite Likelihood Estimation for Latent Variable Models with Ordinal and Continuous, or Ranking Variables. Acta Universitatis Upsaliensis. Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Social Sciences pp. Uppsala. ISBN The estimation of latent variable models with ordinal and continuous, or ranking variables is the research focus of this thesis. The existing estimation methods are discussed and a composite likelihood approach is developed. The main advantages of the new method are its low computational complexity which remains unchanged regardless of the model size, and that it yields an asymptotically unbiased, consistent, and normally distributed estimator. The thesis consists of four papers. The first one investigates the two main formulations of the unrestricted Thurstonian model for ranking data along with the corresponding identification constraints. It is found that the extra identifications constraints required in one of them lead to unreliable estimates unless the constraints coincide with the true values of the fixed parameters. In the second paper, a pairwise likelihood PL) estimation is developed for factor analysis models with ordinal variables. The performance of PL is studied in terms of bias and mean squared error MSE) and compared with that of the conventional estimation methods via a simulation study and through some real data examples. It is found that the PL estimates and standard errors have very small bias and MSE both decreasing with the sample size, and that the method is competitive to the conventional ones. The results of the first two papers lead to the next one where PL estimation is adjusted to the unrestricted Thurstonian ranking model. As before, the performance of the proposed approach is studied through a simulation study with respect to relative bias and relative MSE and in comparison with the conventional estimation methods. The conclusions are similar to those of the second paper. The last paper extends the PL estimation to the whole structural equation modeling framework where data may include both ordinal and continuous variables as well as covariates. The approach is demonstrated through an example run in R software. The code used has been incorporated in the R package lavaan version ). Keywords: latent variable models, factor analysis, structural equation models, Thurstonian model, item response theory, composite likelihood estimation, pairwise likelihood estimation, maximum likelihood, weighted least squares, ordinal variables, ranking variables, lavaan Myrsini Katsikatsou, Uppsala University, Department of Statistics, SE Uppsala, Sweden. Myrsini Katsikatsou 2013 ISSN ISBN urn:nbn:se:uu:diva

3 List of papers This thesis is based on the following papers, which are referred to in the text by their Roman numerals. I Katsikatsou, M., and Yang-Wallentin, F. 2012) On the identification of the unrestricted Thurstonian model for ranking data. II Katsikatsou, M., Moustaki, I., Yang-Wallentin, F., and Jöreskog, K. G. 2012) Pairwise likelihood estimation for factor analysis models with ordinal data. Computational Statistics and Data Analysis, 56, p III IV Katsikatsou, M. 2012) Composite likelihood estimation for Thurstonian models with ranking data. Katsikatsou, M. 2012) Pairwise likelihood estimation for structural equation modeling with ordinal and continuous variables. Reprints were made with permission from the publishers.

4

5 Contents 1 Research Goal Framework Ordinal variables Ranking variables Relationship between ordinal and ranking variables Latent variable modeling Structural Equation Modeling Item Response Theory approach The Thurstonian model Estimation methods overview Estimation in Structural Equation Modeling Estimation in Item Response Theory approach Estimation in Thurstonian modeling Composite likelihood estimation methods Summary of Papers Paper I Paper II Paper III Paper IV Contribution of the thesis Acknowledgments References

6

7 1. Research Goal The main research goal of this thesis is to study the current estimation methods employed within latent variable modeling with data sets of ordinal and continuous variables, or data sets of ranking variables, and develop an alternative, hopefully better, estimation method. The criteria of comparison among the methods are their practical feasibility and the statistical properties of the provided estimators such as unbiasedness and consistency. The motivation for this research interest lies on the fact that maximum likelihood ML) estimation is computationally infeasible for large latent models with the aforementioned type of data sets. On the other hand, the conventional step-wise limited information estimation approaches Muthén, 1984) require the estimation of a weight matrix to compute correct standard errors, the dimension of which grows rapidly with the number of observed variables. Besides, relatively large sample sizes are needed to get a reliable estimate of the matrix. For this, a composite likelihood estimation method is developed for the type of models and data in question. The merits of the proposed approach are that it is computationally feasible regardless of the model size, it does not require the estimation of a weight matrix to provide correct standard errors, and it yields an estimator which is asymptotically unbiased, consistent and normally distributed Lindsay, 1988; Varin, 2008; Varin et al., 2011). To achieve the main goal the research project was split into four major parts resulting into the four papers composing this thesis. Firstly, before proceeding to any studies on the estimation of latent variable models with ranking variables, it was necessary to investigate the special nature of ranking data and how they can be analyzed within the framework of latent variable modeling. The latter can be done by applying the Thurstonian model Thurstone, 1927). To get the model identified two approaches are suggested in the literature. The one is more well established but inference on a basic question ranking data aim to answer becomes tricky. The other, which has been recently suggested Maydeu-Olivares & Böckenholt, 2005), makes the inference on all basic questions straightforward. This second identification strategy is investigated in Paper I by mainly checking how it affects the parameter estimates. Ranking variables set up a more challenging framework than ordinal variables do, mainly due to their comparative and discrete nature. For this, the study of estimation methods starts off with a somehow simpler theoretical structure. Paper II focuses on factor analysis models with ordinal variables. The existing estimation methods are discussed and a composite, namely a pairwise likelihood PL) estimation is developed. The performance of the latter is 7

8 studied in terms of bias and mean squared error MSE) and compared with the performance of the conventional estimation methods via a simulation study and through some real data examples. The positive results of the second paper combined with the conclusions of the first paper lead to Paper III. That firstly presents an overview of the existing estimation methods for Thurstonian models with ranking data followed by the development of a composite likelihood estimation method. In that paper as well, a simulation study investigates the performance of the suggested approach with respect to bias and MSE and in comparison with the current estimation approaches. Paper IV extends the method presented in the second paper to the whole structural equation model where covariates may be included and the data may consist of both ordinal and continuous variables. The proposed method is demonstrated with an example of empirical data run in R software R Development Team, 2008) and using the R package lavaan Rosseel, 2012; Rosseel et al., 2012) which our self-written R code has been incorporated into. The R commands used in the example are provided. Hopefully, this way, the thesis accomplishes the main research purpose and adds to the knowledge of composite maximum likelihood methods; in particular, how they can be applied within latent variable models with ranking variables or with ordinal and continuous variables, and how they perform with respect to bias and MSE and in comparison with the mainstream estimation methods. 8

9 2. Framework 2.1 Ordinal variables In social sciences, the variables that are mainly encountered are categorical, ordinal and/or nominal. These are used to measure, among others, attitudes, beliefs, and abstract characteristics. The ordinal scale mostly employed is the Likert scale typically consisted of four, five, or seven points. Most of the time the points are labeled with numbers such as 1, 2, 3, etc., and with verbal text such as "Strongly disagree", "Disagree", "Agree", etc. Examples of ordinal variables are given in Table 2.1. Possible observations are, for example, 1,2,4), 3,4,5), 1,2,2), 2,5,5), etc. Within each vector, the numbers denote the response category chosen for each variable. In a statistical analysis with such variables, there are two important things that one should take into account. Firstly, the numbers of the response categories do not have metric properties; they just label ordered categories simply indicating different levels of a characteristic without giving information on the degree that the levels differ. Hence, addition or subtraction) and multiplication or division) are meaningless for ordinal variables. Secondly, whenever ordinal variables are used, an assumption is always implied, that the respondents understand the questions/ statements and use the response scale in the same way. Violation of this assumption leads to interpersonally incomparable responses which, in turn, are highly probable to lead to incorrect statistical results and inference e.g. Brady, 1989). 2.2 Ranking variables Ranking variables can also be used to measure attitudes, beliefs, and abstract characteristics. They are discrete as ordinal variables are but free of scale. In a ranking experiment, a set of m objects is presented to the respondents who are asked to assign a rank, from 1 to m, to all objects complete ranking) or to a subset of them partial ranking) according to their personal preference or a presspecified criterion. Usually 1 is defined as the rank to be assigned to the most preferred object and m to the least preferred one. An example of a ranking variable is given in Table 2.2. Possible responses are, for example, 1,2,3), 2,1,3), and 1,2,2) if ties are allowed. The main questions ranking variables seek to answer are a) which preference patterns are dominant, and b) what the inter-relationships among the ranked objects are. Adjacent 9

10 Table 2.1. Example of ordinal variables Policy A is appropriate in order to deal with problem X Strongly Agree Agree Neither Agree Disagree Strongly Disagree nor Disagree Policy B is appropriate in order to deal with problem X Strongly Agree Agree Neither Agree Disagree Strongly Disagree nor Disagree Policy C is appropriate in order to deal with problem X Strongly Agree Agree Neither Agree Disagree Strongly Disagree nor Disagree ranks systematically assigned to certain objects indicate that these objects are perceived similarly and can possibly substitute each other. A detailed introduction to ranking variables and how they can be modeled and analyzed can be found in Marden 1995). 2.3 Relationship between ordinal and ranking variables The relationship between a set of ordinal variables and a ranking variable can be illustrated through the examples discussed in sections 2.1 and 2.2. The first thing to note is that in the ordinal variable set-up the response pattern of an individual is a vector of dimension equal to the number of ordinal variables, while in the ranking set-up for each ranking variable a vector of size equal to the number of ranked objects is observed. The observations 1, 2, 4) and 3,4,5) for the three ordinal variables correspond to the observation 1,2,3) for the ranking variable and the observations 1,2,2) and 2,5,5) correspond to the ranking 1,2,2). Hence, ordinal data can be transformed into ranking data but not the other way round. A ranking response does not convey any information about the level an object is preferred. Therefore, the inference based on ranking data can only be made in relative terms and is valid only for the set of objects included in the ranking experiment. On the other hand, ranking designs impose less restrictions on the response mechanism than ordinal ones. They do not restrict participants to use a certain scale to provide their answer. This way, they avoid the assumption that all participants understand and use the scale in the same way. Because of these advantages and disadvantages 10

11 of ordinal and ranking variables, some researchers prefer designs where both types of variables are combined appropriately. Table 2.2. Example of a ranking variable Policy A, Policy B, and Policy C are three alternatives that can be applied in order to deal with problem X. Rank the three policies assigning rank 1 to the policy you consider as the most suitable, 2 to the next most suitable, and 3 to the least suitable. Policy A B C Rank _ 2.4 Latent variable modeling Latent variable modeling is a multivariate type of modeling that can be applied for any type of observed variables. The observed variables are also referred to as indicators or items in the literature. The core idea of the analysis is that the latent variables account for the dependencies among the observed variables in the sense that if the former are held fixed, the latter are independent. The latent variables are also referred to as factors or constructs. Theoretically, latent variable model analysis can be distinguished into exploratory and confirmatory, but practice usually lies between the two. In exploratory analysis the goal is to summarize the information observed in a large data set of p variables into a smaller set of q latent variables, where q is much smaller than p. A typical example of such situation is questionnaires, which usually include at least 15 questions. In confirmatory analysis, the objective is to verify a theory where the variables of interest are abstract constructs such as those often faced in social and behavioral sciences. Examples of such variables are political beliefs, intelligence, emotional conditions, attitudes, etc. Hence, a latent variable model is specified in advance, the latent variables are measured through indicating variables, and the fit of the model to the empirical data is tested. Within latent variable modeling, there are two approaches, the Structural Equation Modeling SEM) approach e.g. Bollen, 1989; Jöreskog, 1990, 1994, 2002; Lee et al., 1990, 1992; Muthén, 1984) and the Item Response Theory IRT) approach e.g. Bartholomew et al., 2011; Muraki, 1990; Muraki & Carlson, 1995; Samejima, 1969). However, Bartholomew et al. 2011) explain how SEM, where both observed and latent variables are assumed normally distributed, can be seen as a special case of the more general IRT framework. A more detailed description of the two approaches is provided below Structural Equation Modeling SEM mainly aims to test the hypothesis that the covariance matrix of the observed variables is equal to the covariance matrix implied by a hypothesized 11

12 latent variable model. SEM was developed first for continuous variables and then extended to ordinal ones by adopting the underlying response variable URV) approach Jöreskog, 1990, 1994; Muthén, 1984; Olsson, 1979). Let x be an observed p dimensional vector of continuous variables assumed to follow a multivariate normal distribution. The SEM in its general form is written as follows: x = ν + Λξ + Kw + δ 2.1) ξ = α + Bξ + Γz + ζ, 2.2) where ξ is a q dimensional vector of latent variables, w and z are vectors of covariates, δ and ζ are vectors of error, and ν and α are vectors of intercepts. The standard assumptions of the model are that: a) x w,z N p µ,σ), b) ξ follows a multivariate normal distribution, c) δ N p 0,Θ) with Θ being diagonal, d) ζ N q 0,Ψ), e) Covξ,δ) = Covξ,ζ ) = Covδ,ζ ) = 0, and f) I B is not singular, where I is the identity matrix. Equation 2.1) is referred to as the measurement model and links the observed variable vector x with the latent variable vector ξ. The effect of ξ on x is given by the matrix of loadings Λ and the measurement model also reads as ξ is measured by x. Equation 2.2) is referred to as the structural model and shows the relationships among the latent variables. The regression coefficients are included in matrix B. In both equations covariates may be included. The only restriction is that w and z should contain different covariates for identification reasons. Based on the model, it follows that: and µ = E x w,z) = ν + ΛI B) 1 α + Γz) + Kw, [ Σ = Covx w,z) = ΛI B) 1 Ψ I B) 1] Λ + Θ, Covx w,z,ξ ) = Θ. The last two equations along with the fact that Θ is typically assumed to be diagonal implies that, given the covariates, SEM imposes a certain structure on the covariance matrix of x, and accounts for all correlations among the observed variables once the values of latent variables are also given. For a detailed and instructive introduction to SEM with continuous variables see Bollen 1989). To include ordinal observed variables into the model the URV) approach is adopted which assumes that the ordinal variables are generated by underlying continuous variables. The connection between an ordinal variable x i and its underlying continuous counterpart x i is x i = c i, j τ i, j 1 < x i < τ i, j, 2.3) where c i, j is the j-th response category of variable x i, j = 1,...,C i, τ i, j is the j-th threshold of variable x i, and = τ i,0 < τ i,1 <... < τ i,ci 1 < τ i,ci = +. 12

13 Since only ordinal information is available in the data, the distribution of the underlying variable x i is determined only up to a monotonic transformation. In practice it is often assumed that x i N0,1) and the thresholds are free to be estimated. An alternative parametrization where it is assumed x i N µ i,σ ii ) is also possible e.g. Jöreskog, 2002). Based on the URV approach, a p dimensional vector of observed ordinal variables x is matched to its underlying continuous counterpart x. It is the latter that is involved in SEM, particularly in Equation 2.1). The probability of a response pattern for variable x is written in terms of the distribution of x as follows: πx ) = π x1 = c 1, j,...,x ) τ1, j τp, j p = c p, j =... f x)dx, 2.4) τ 1, j 1 τ p, j 1 where f x) is the assumed p dimensional normal ) distribution of vector x. If x the observed variable vector is x = 1, where x x 1 is a vector of p 1 observed ordinal variables and x 2 is a vector ) of p 2 observed continuous variables, 2 x1 p 1 + p 2 = p, then the vector x = is defined, where x x 1 is the vector of 2 the p 1 underlying continuous variables. Again, it is the latter which is involved in the measurement model. For an introduction of SEM with ordinal variables see for example Jöreskog 2002). A special case of SEM is when only the measurement model is considered. Then, the model is usually called factor analysis model. The standard assumptions of SEM as well as the way that ordinal variables can be incorporated into the model remain the same Item Response Theory approach The core assumption of IRT is that of conditional independence or otherwise local independence. It is assumed that the latent variables account for all the dependencies among the observed variables. This way, the conditional distribution of x given ξ, gx ξ ), can be written as the product of the univariate conditional distributions of x i given ξ, gx i ξ ), i.e. gx ξ ) = p i=1 gx i ξ ). This assumption cannot be tested but rather has the role of an axiom. IRT assumes the model: f x) = gx ξ )hξ )dξ R ξ 13

14 which, based on the assumption of local independence, is simplified to f x) = R ξ p i=1 gx i ξ )hξ )dξ, 2.5) where f x) is the joint distribution of variable vector x, hξ ) is the joint distribution of latent variables, and R ξ is the latent variable area. The model is very general and it accommodates any type and combination of variables. Each observed variable can be of different type, since only its univariate conditional distribution gx i ξ ) needs to be determined. In the case of an ordinal observed variable xi the conditional distribution gxi ξ ) is a multinomial one, i.e. gx i ξ ) = C i j=1 π x i = c i, j ξ ) Ix i =c i, j), where Ix i = c i, j ) is the indicator variable whether x i falls into the response category c i, j. A measurement model is applied to the cumulative probabilities γ x i c i, j ξ ), where π x i = c i, j ξ ) = γ x i c i, j ξ ) γ x i c i, j 1 ξ ). A typical model for γ xi c i, j ξ ) in IRT is γ x i c i, j ξ ) = F α i, j q k=1 β ik ξ k ), 2.6) where the α i, j s are thresholds = α i,0 < α i,1 <... < α i,ci 1 < α i,ci = + ), the β ik s are loadings, and F is a link function e.g. logit, probit, etc.). A very rigorous introduction to IRT and its links to SEM can be found in Bartholomew et al. 2011) The Thurstonian model In an experiment of complete ranking of m objects, {O 1,...,O m }, where no ties are allowed the sampling distribution is a multinomial one with m! categories. The log-likelihood function of a random sample of n ranking vectors is of the form: lnlθ;r 1,...,r n )) = m! c=1 n c lnπ c θ), 2.7) where r i is the i-th observed ranking which is an m dimensional vector of permuted integers from 1 to m, θ is a parameter vector, n c is the observed frequency of ranking pattern c, m! c=1 n c = n, and π c θ) is the corresponding probability under the model, π c θ) > 0, and m! c=1 π cθ) = 1. 14

15 One of the most influential models for ranking data in the literature is that suggested by Thurstone 1927). Maydeu-Olivares & Böckenholt 2005) show how a Thurstonian model can be incorporated in the more general framework of SEM. The basic idea of analysis is similar to that of the URV approach. Each ranked object is assumed to have an underlying continuous utility, i.e. the observed ranks assigned to the objects are assumed to be the result of the underlying object utilities. Furthermore, it is assumed that the differences in object utility assessments follow a multinormal distribution. Let u j be the underlying utility of object O j, j = 1,...,m, and u = u 1,u 2,...,u m ) be the m-dimensional random vector of underlying utilities. As said, it is assumed that u N m µ,σ). Let also: O c h) denote the object which has been assigned the rank h given the complete ranking pattern c, h = 1,...,m, c = 1,...,m!, ũ c i = u O c u O i) c be the utility difference between objects with adjacent i+1) ranks within the ranking pattern c, i = 1,...,m 1, ũ c be the m 1)-dimensional vector containing all the above utility differences, C c be an m 1) m contrast matrix transforming vector u into ũ c, i.e. its exact form depends on the ranking pattern c in question, and D c = [diagc c ΣC c)] 1/2, where diag is the function which takes as an argument a square matrix and returns a diagonal matrix with the same main diagonal elements. The probability of the ranking pattern c, π c, is modeled as follows: π c µ,σ) = Φ m 1 Dc C c µ; D c C c ΣC cd c ), 2.8) where Φ m 1 D c C c µ;d c C c ΣC cd c ) is the m 1)-dimensional cumulative normal distribution with correlation matrix D c C c ΣC cd c evaluated at the point D c C c µ. In the unrestricted Thurstonian model where no structure is assumed for µ and Σ, the above model faces serious identification issues. For this reason, the m 1)-dimensional random vector ũ is considered instead, where ũ contains all the object utility differences with respect to the utility of a reference object e.g. Chan & Bentler, 1998; Yao & Böckenholt, 1999). Choosing object O 1 as a reference object, the random vector ũ is of the form ũ = ũ 1,ũ 2,...,ũ m 1 ) = u 1 u 2,u 1 u 3,...,u 1 u m ). It holds that: ũ = Bu, 2.9) 15

16 where B is a m 1) m contrast matrix of the form: B = Hence, ũ N m 1 µ, Σ) with µ = Bµ and Σ = BΣB. The ranking probability π c is modified to: π c µ, Σ ) = Φ m 1 D c C c µ; D c C c ΣC c D c ), 2.10) where D c = [ diag C c ΣC c)] 1/2 and C c is now an m 1) m 1) contrast matrix transforming vector ũ into ũ c. 16

17 3. Estimation methods overview The standard method of maximum likelihood ML) estimation becomes computationally demanding or even impractical for relatively large latent models where some of the observed variables are ordinal or ranking and the latent variables are assumed normally distributed. This has motivated the development of limited information estimation methods, especially within SEM with ordinal variables and Thurstonian ranking models. A brief discussion about estimation under the three types of models mentioned in Section 2 is given below. 3.1 Estimation in Structural Equation Modeling Full maximum likelihood estimation is not practical under SEM with ordinal variables even when the number of ordinal variables is small Lee et al., 1992; Liu, 2007; Poon &) Lee, 1987). To demonstrate this, consider the variable x vector x = 1 and the SEM defined in Section Then, the loglikelihood function for one observation x 2 is: lnlθ;x ) = lnπx 1 x 2 ;θ) + ln f x 2 ;θ), 3.1) where the parameter vector θ includes the thresholds and the SEM parameters, f x 2 ;θ) is the multinormal distribution of the observed continuous variables x 2, and πx 1 x 2;θ) is the probability of the response pattern for x 1 given x 2. The latter is written as in Equation 2.4) with the difference that the distribution to be integrated is the conditional multinormal distribution of the underlying continuous variables x 1 given the observed continuous variables x 2, f x 1 x 2 ;θ). Hence, ML requires the evaluation of p 1 dimensional normal probabilities the computation of which is possible only for a very small number p 1. In practice though, this number is very rarely small. The impractical nature of ML has initiated the development of limited information estimation methods that usually require the evaluation of up to bivariate normal probabilities. Among them the three-stage estimation method proposed by Muthén 1984) is widely employed as well as implemented in commercial software such as LISREL Jöreskog & Sörbom, 1996) and Mplus Muthén & Muthén, 2010). In the first stage of the method, the thresholds of the underlying variables and the product-moment correlations, if continuous 17

18 observed variables are present, are estimated. The thresholds are estimated by maximizing one by one all the univariate likelihood functions of ordinal variables. In the second stage, given the threshold estimates, the polychoric correlations of all pairs of ordinal variables and the polyserial correlations of all pairs of one ordinal and one observed continuous variable are estimated by maximizing one by one the corresponding bivariate likelihood functions. These computations involve the evaluation of one- and two-dimensional normal probabilities only. Finally, in the third stage, given the estimates of all types of correlations, the SEM parameters are estimated by applying a type of least squares, i.e. weighted least squares WLS), diagonally weighted least squares DWLS), or unweighted least squares ULS) for comparisons see e.g. Forero et al., 2009; Yang-Wallentin et al., 2010). The weight matrix is an estimate of the asymptotic covariance matrix of the estimated correlations. The full weight matrix is needed in all three versions of least squares to compute correct standard errors. Then, DWLS and ULS are called robust DWLS RD- WLS) and robust ULS RULS). A point to consider with respect to this threestage estimation is that the size of the weight matrix grows very rapidly with the number of observed variables, e.g. for 15 observed variables, the matrix is This way, the computational simplicity of the approach achieved in the first two stages is somehow canceled out by the size of the weight matrix. Besides, for a reliable estimate of the matrix a fairly large sample size is desired. 3.2 Estimation in Item Response Theory approach Maximum likelihood estimation is the standard approach in IRT modeling. The E-M or Newton-Raphson algorithm are usually employed to carry out the maximization of the objective function. In both cases evaluations of multiple integrals the dimension of which is equal to the number of latent variables are required. These computations are performed numerically using methods such as Gauss-Hermite quadrature, adaptive quadrature, and Monte Carlo for a discussion of different algorithms see e.g. Feddag & Bacci, 2009; Schilling & Bock, 2005). However, for all these algorithms, for a fixed level of computational accuracy, the computation time and burden grows fast with the dimension of the integration. Besides, the convergence of the EM algorithm slows down as the number of latent variables increases. As a result, ML becomes infeasible beyond a certain number of latent variables. 3.3 Estimation in Thurstonian modeling The maximization of the log-likelihood in 2.7) requires the evaluation of m 1) dimensional normal probabilities as defined in 2.10). To evalu- 18

19 ate such probabilities, the algorithm suggested by Schervish 1984) can be employed. However, the computational time increases very rapidly with the number of objects m so that ML is practically infeasible for even a moderate m. As a consequence, limited information estimation methods employing mainly generalized least squares GLS) have been proposed Brady, 1989; Chan & Bentler, 1998; Maydeu-Olivares & Böckenholt, 2005). In these approaches, the estimates are obtained by minimizing a fit function of the type: F θ) = p π θ) ) W p π θ) ), where θ = µ, [ vech Σ )] ), vech is the function transforming a symmetric matrix into a vector by stacking the elements of the lower triangular of the matrix, including its main diagonal, columnwise, µ and Σ are as defined in Section 2.4.3, p is the vector of observed low-order ranking probabilities, π θ) is the vector of the corresponding probabilities under the model, W is a sample estimate of the covariance matrix of the random vector p, and W is the generalized inverse of W. The generalized inverse is computed as W is singular due to the inter-relationships among low-order rankings. Although GLS aims to be an estimation method of low computational complexity, the size of W grows extremely fast with the number of objects m rendering the inversion of the matrix and subsequently, the method computationally demanding. Maydeu-Olivares & Böckenholt 2005) show how this GLS approach can be carried out by the conventional three-stage estimation approach applied in SEM and described in Section Composite likelihood estimation methods Composite likelihood estimation methods, already applied to a range of models, are gaining more research attention recently because they can substantially simplify the computations involved in estimation and at the same time yield asymptotically unbiased, consistent, and normally distributed estimators Lindsay, 1988). Varin 2008) and Varin et al. 2011) give an extensive overview of these methods and their application areas. In situations where the likelihood function either cannot be specified or is impractical to work with due to high computational complexity, one could consider instead a composite likelihood function. The latter is defined as follows Lindsay, 1988; Varin, 2008; Varin et al., 2011): let x be a p dimensional random vector with probability density f x; ω) for some unknown vector parameter ω Ω. Let {A 1,...,A K } be a set of measurable marginal or conditional events with associated likelihoods L k ω;x) f x A k ;ω). A composite likelihood CL) function is the weighted product of the likelihoods cor- 19

20 responding to each single event, CLω;x) = K k=1 L k ω;x) w k, where w k are non-negative weights to be chosen. The maximum composite likelihood estimator ˆω MCL is obtained by maximizing the function CLω;x) over the parameter ω. Under regularity conditions on the component likelihoods, the central limit theorem for the composite likelihood score statistic can be applied leading to the result n ˆωMCL ω) d N 0,G 1 ω) ), where Gω) is the Godambe also called sandwich) information matrix of a single observation. In particular, Gω) = Hω)J 1 ω)hω), where Hω) = E { 2 lnclω;x) }, and Jω) = Var { lnclω;x)}. In general, the identity Hω) = Jω) does not hold because the assumed independence among the likelihood terms forming the composite function is usually not valid when the full likelihood is considered. Varin et al. 2011) discuss some further qualities of the composite likelihood approach. It can be seen as a robust alternative in terms of modeling. It is easier and more straightforward to model lower order dimensional distributions while modeling uncertainty increases with dimensionality. By applying composite likelihood, possible misspecification of the higher order dimensional distributions can be avoided. In addition, a model assumed for lower order distributions can be compatible with more than one possible modeling options available for higher dimensional distributions. 20

21 4. Summary of Papers 4.1 Paper I The framework of Paper I is that described in Section The research interest lies on the identification of the model written with respect to µ and Σ see Equation 2.8)) and the research question is what impact the identification strategy proposed by Maydeu-Olivares & Böckenholt 2005) has on the parameter estimates. More specifically, due to the discrete and comparative nature of ranking data, the utility random vector u is unique only up to a linear transformation. The origin and the unit of the utility scale should be defined. This is usually done by fixing the utility mean and variance of one object equal to 0 and 1, respectively. Let µ 1 = 0 and σ 11 = 1. As a consequence, the elements of µ and Σ should be interpreted in relative terms and the inference should be based on the estimates of the standardized parameters, i.e. the correlations among object utilities ρ i j, i j = 1,...,m, the ratios of object utilities variances, e.g. σ j j /σ 11, and the standardized mean utility differences, e.g. µ 1 µ j )/ σ 11 2σ 1 j + σ j j, j = 2,...,m. However, the model written with respect to µ and Σ is identified only if at least m extra constraints, additional to those defining the scale origin and unit, are set. Maydeu-Olivares & Böckenholt 2005) suggest that one could fix the covariances of the utility of the m-th object with all other object utilities equal to 0, i.e. σ mi = 0, i = 1,...,m 1, and the variance of the last object σ mm equal to 1. These m extra constraints imply that the correlations ρ mi, i = 1,...,m 1, are equal to 0 and the variance ratio σ mm /σ 11 is equal to 1, something which may not hold in the population. Consequently, the model to be estimated may be misspecified and then, the estimates of the free correlations and variance ratios are expected to present bias and relatively high MSE. Note that the goodness-of-fit statistics are unable to detect possible model misspecifications coming from the extra identification constraints. To investigate the size of the misspecification impact on the estimates a simulation study is conducted. 36 different experimental conditions derived by nine different misspecification situations, two model sizes m = 4, 7), and two sample sizes n = 500,1000), are examined. The bias and MSE of the estimates obtained under no misspecification within each combination of model and sample size are used as benchmarks. The results indicate that the identification approach suggested in the literature leads to reliable estimates of all the standardized parameters as long as the extra identification constraints coincide 21

22 with the true values of the constrained parameters. When this does not hold, the estimates of almost all correlations and variance ratios are seriously biased and present relatively high MSE. The level of bias and MSE increases with the misspecification level and not in a uniform way for all parameters. An increase in the sample size seems to have very marginal effect in decreasing the bias and MSE. As a result, the approach should be used with great caution. Before adopting any extra constraints one should resort to already existing theory or previous empirical studies concerning the specific set of objects to confirm that the extra constraints are reasonable and can be justified for the population in question. 4.2 Paper II Papers II focuses on latent variable model analysis, both confirmatory and exploratory, of ordinal variables. Factor analysis models both under the URV approach and the IRT approach, as described in sections and 2.4.2, respectively, are considered. The research objective is to examine the conventional estimation methods employed within these two types of modeling, and propose a new one that is of low computational complexity regardless of the model size and performs equally well as the current methods. Within factor analysis, where the URV approach is adopted, the model is x = Λξ + δ, 4.1) where x is the vector of underlying continuous variables corresponding to the vector of the observed ordinal variables x, ξ is the vector of latent variables, and δ is the vector of errors. It is assumed that each underlying continuous variable follows standard normal distribution, ξ N q 0,Φ), where Φ has ones on its main diagonal, δ N p 0,Θ) with Θ = I diagλφλ ), and Covξ,δ) = 0. The parameter vector to be estimated is θ = λ,ϕ,τ ), where λ and ϕ are the vectors of free non-redundant parameters in matrices Λ and Φ, respectively, and τ is the vector of all free thresholds. As explained in Section 3.1, ML estimation of θ is practically infeasible and the three-stage limited information estimation presented by Muthén 1984) is applied. Under the IRT approach, the model in Equation 2.6) in Section is considered. As said in Section 3.2, ML is the standard estimation method but it becomes impractical when the number of latent variables is large. We propose a pairwise maximum likelihood PML) method under the factor analysis model and the URV approach to estimate the parameter θ. The PML estimator ˆθ PML is the value of θ maximizing the pairwise log-likelihood 22

23 function. The form of the latter for one observation is: plθ;x ) = lnlθ;xi,xi )) = 4.2) i<i = i<i C i j=1 C i j =1 I x i = c i, j,x i = c i, j ) lnπci, j c i, j θ), 4.3) where, I xi = c i, j,xi ) = c i, j is the indicator variable taking the value 1 if the variables xi and xi fall into the categories c i, j and c i, j, respectively, and 0 otherwise; based on the Equation 2.4), π ci, j c i, j θ) = π x i = c i, j,x i = c i, j ;θ) = = Φ 2 τi, j,τ i, j ;ρ x i x i ) Φ2 τi, j,τ i, j 1;ρ xi x i ) Φ 2 τi, j 1,τ i, j ;ρ x i x i ) + Φ2 τi, j 1,τ i, j 1;ρ xi x i ), ρ xi x i θ) = λ i )Φλ i ), and λ i is a 1 q row vector containing the elements of the i-th row of matrix Λ. Since PML belongs to the general family of composite likelihood methods, the general result reported in Section 3.4 can be applied. Hence, the estimator ˆθ PML is asymptotically unbiased, consistent, and normally distributed. The advantage of PML over ML is mainly computational since the former involves the evaluation of integrals of bivariate normal distributions only, regardless of the number of observed ordinal variables or factors. The main advantages of PML over the three-stage limited information estimators are that all model parameters are estimated in one single step and there is no need of estimating a weight matrix to obtain correct standard errors. The performance of PML estimator in finite samples with respect to bias and MSE is studied via a simulation study under eight experimental conditions four different sample sizes and two model sizes). It is also compared with the performance of the three-stage approaches RDWLS and RULS, and that of ML as implemented under the IRT approach. Moreover, PML is demonstrated and compared with RDWLS, RULS, and ML within some real data examples both in an exploratory and confirmatory analysis set-up. The general conclusions are that: a) PML estimates and their standard errors have bias and MSE very close to zero, both decreasing with the sample size, b) all the methods considered in the study provide very similar results, and c) there is a tendency for the PML and RDWLS estimates and standard errors to be slightly closer to those of ML than those of the RULS approach. 4.3 Paper III The promising results of the second paper lead to Paper III. The framework is the unrestricted Thurstonian model with ranking data as described in Section 23

24 Based on the results of the first paper we consider the estimation of the model written with respect to µ and Σ Equation 2.10)). This model is identified by only defining the unit of the scale of utility differences. An overview of the existing estimation methods is given in the paper, a summary of which is provided in Section 3.3. The composite likelihood estimation proposed is based on the notion of trinary rankings. Trinary rankings are the relative rankings of triplets of objects implied by a given complete ranking of m objects. Chan & Bentler 1998) explain that it is enough to consider only the m 1)m 2)/2 triplets which include the reference object. The trinary rankings of the rest of the object triplets contain redundant information. This way, we suggest a trinary composite likelihood TCL) estimation where, for one observation, the log-likelihood function to be maximized is: l ) ) tc θ;r = lnl i j θ;r O1,O i+1,o j+1 ), 4.4) m 2 i=1 m 1 j=i+1 where θ = µ, [ vech Σ )] ), r O1,O i+1,o j+1 ) is the observed trinary ranking of the triplet O 1,O i+1,o j+1 ) implied ) by the complete ranking of the m objects r, and lnl i j θ;r O1,O i+1,o j+1 ) is the log-likelihood function for this triplet of objects. The specific form of the latter is: ) lnl i j θ;r O1,O i+1,o j+1 ) = 6 t=1 ) I r O1,O i+1,o j+1 ) = t lnπ O 1,O i+1,o j+1 ) t θ), ) 4.5) where I r O1,O i+1,o j+1 ) = t is the indicator variable which takes the value 1 if the observed trinary ranking for the triplet O 1,O i+1,o j+1 ) is equal to pattern t and 0 otherwise, and π O 1,O i+1,o j+1 ) t θ) is the corresponding probability under the model. Based on Equation 2.10), this probability is written as follows: π O 1,O i+1,o j+1) i j) t θ) = Φ 2 D t C t µ i j) ; D t C t Σ i j) C td t ), ) c ) if i = 1 where µ i j) µi =, Σ µ i j) σ j1 σ j j = ), j σii otherwise σ ji σ j j [ i j) 1/2, D t = diagc t Σ i j) C t)] Ct is a 2 2 contrast matrix, and c is a known positive constant. Note that σ 11 is fixed to a constant c, c > 0, to define the scale unit. The estimator ˆ θ TCL shares the asymptotic properties of the composite likelihood estimators detailed in Section 3.4. The method is computationally gen- 24 i j) i j)

25 eral as it involves the evaluation of only bivariate normal probabilities regardless of the number of objects m. Compared to the three-stage SEM estimation methods, the TCL approach estimates all parameters simultaneously and it does not require the estimate a weight matrix to get correct standard errors. The performance of TCL estimation in finite samples under different model sizes and sample sizes is investigated with respect to relative Bias and relative MSE through a simulation study. It is also compared with the performance of RDWLS and RULS as implemented within SEM with ordinal variables. It is found that all three methods yield similar estimates and standard errors for all experimental conditions with TCL and RULS performing slightly better than RDWLS with respect to relative bias. Interestingly enough, the great deal of redundant information that is used within the three-stage RULS and RDWLS approaches does not affect the accuracy and efficiency of the methods as those are compared with the accuracy and efficiency of the TCL method. 4.4 Paper IV Paper IV extends the pairwise estimation method proposed in the second paper to the whole SEM where covariates may be included and the indicators of the latent variables can be both ordinal and continuous. The variable vector x and the SEM model introduced in Section the model consisting of equations 2.1) and 2.2)) are considered. Hence, the pairwise log-likelihood function for one observation of Equation 4.2) is modified to: pl θ;x ) = i<i lnlθ;x 1i,x 1i ))+ lnlθ;x 2k,x 2k ))+ k<k i lnlθ;x1i,x 2k )), k where lnl θ;x 1i,x 1i ) ) is the bivariate log-likelihood function of a pair of ordinal variables, lnlθ;x 2k,x 2k )) is the bivariate log-likelihood of a pair of observed continuous variables, and lnlθ;x 1i,x 2k)) is the bivariate loglikelihood function of a pair of one ordinal and one continuous observed variables. The form of lnl θ;x 1i,x 1i ) ) is as defined in Equation 4.3), the function lnlθ;x 2k,x 2k )) is a bivariate normal log-likelihood, and lnlθ;x 1i,x 2k )) = ln f x 1i x 2k ;θ) + ln f x 2k ;θ) = = C i I x1i = c i, j x 2k )lnπx1i = c i, j x 2k ;θ) + ln f x 2k ;θ). j=1 The function ln f x 2k ;θ) is the log-likelihood of a univariate normal distribution, I x 1i = c i, j x 2k ) is the indicator variable taking the value 1 if the ordinal variable x 1i falls into the category c i, j given the value of x 2k and 0 otherwise, and πx 1i = c i, j x 2k ;θ) is the probability that I x 1i = c i, j x 2k ) takes the value 1. The latter is written in terms of univariate normal probabilities. 25

26 It is worthy to note that the proposed method does not apply only to SEM but to any kind of model that assumes a parametric ) structure for µ and Σ, x1 where µ = E x w,z), Σ = Covx w,z), x =, and x x 1 and x 2 are the 2 vectors of underlying and observed continuous variables, respectively. Observe also that PL estimation assumes only bivariate normality among the elements of the vector x given w and z and not joint normality, i.e. x w,z N µ,σ), as SEM does. That it is an important difference since bivariate normality is an assumption that can be tested see e.g. Jöreskog, 2002). The suggested method is demonstrated using an example with empirical data. To run the model an R code has been written and later on incorporated into the R package lavaan version ). The R code used for the example is provided. Interestingly enough, ML as implemented in Mplus version 5.21) and LISREL version 9.10) is not feasible for the demonstrated example. On the other hand, RDWLS gives very similar estimates and slightly smaller standard errors than those of PL. 26

27 5. Contribution of the thesis In this thesis a composite, namely a pairwise likelihood estimation method is developed for SEM with ordinal and continuous variables where covariates may also be included, and is appropriately adjusted for Thurstonian models with ranking variables. The performance of the method with respect to bias and MSE in finite samples is studied within factor analysis models with ordinal variables, and within Thurstonian models with ranking variables. Moreover, in both types of models, the performance of PL is compared with that of RDWLS and RULS. ML as implemented under the IRT approach is also considered in the performance comparisons in the case of factor analysis. The main conclusion of the simulation studies is that PL shows a close to zero bias and MSE decreasing with the sample size and it is competitive to the other estimation methods. The proposed approach is also demonstrated with examples of empirical ordinal data in the case of exploratory and confirmatory factor analysis, and in the case of SEM. To run all these models a code in R has been written and later on a part of it has been incorporated in the R package lavaan version ). This way, PL for SEM with ordinal variables is accessible to researchers at the time this thesis is written. A secondary result of the thesis is that one should be careful with the formulation of the unrestricted Thurstonian ranking model. A certain parametrization requires extra identification constraints than the typical ones which are highly probable to affect the quality of the estimates in terms of bias and MSE negatively. The thesis adds knowledge on composite likelihood estimation methods applied to latent variable models with ordinal and continuous variables, and latent variables models with ranking variables. Optimistically, it could be used as the starting point for further research on the performance of PL under more experimental conditions, especially under large models with small sample size; on the development of chi-square test statistics and model selection criteria under PL for SEM and Thurstonian models; and on the treatment of missing values when PL is applied. 27

28 Acknowledgments I am enormously thankful to my supervisors, Prof. Fan Yang-Wallentin, Prof. Irini Moustaki, and Prof. Karl Jöreskog, who taught me and guided me scientifically as well as supported me psychologically. It is self-evident that without their tremendous help I would not have been able to accomplish the demanding task of PhD. Fan, thank you so much for being always available for advice, solving various annoying administrative issues for me, and taking the effort to ensure more than enough financial support for our project. Irini, thank you so much for "bringing me up" in the world of statistics from my day one up to now with understanding and patience. The knowledge and encouragement you offered me during my master thesis were decisive in applying for a PhD. Thank you for actively supervising my PhD without being paid! Karl, it was an honor and privilege to have such an outstanding researcher of the field like you as my supervisor. Your expertise and experience were more than valuable. I am very thankful to Prof. Yves Rosseel of Department of Data Analysis of Ghent University in Belgium who enthusiastically accepted to cooperate with me in order to incorporate my amateurish R code in his amazing R package lavaan. Apart from the fact that my programming skills have been improved thanks to this cooperation, I find that my PhD gets more value as part of my research becomes easily accessible to other researchers through his extremely user-friendly package. I want to express my appreciation to all professors, teachers, PhD students, and staff of the department. I could not wish a better working environment. The fruitful statistical discussions, the substantial help with various tasks during the PhD, the encouragement, the financial support, and of course the refreshing coffee breaks are also important to carry on with and carry out a PhD successfully. Sincere thanks go to all the teaching staff of the Department of Statistics of Athens University of Economics and Business in Greece who worked during They offered a high quality and inspiring education. I got a solid background in statistics, indispensable prerequisite for continuing further my studies. My dearest friends, many many thanks for being supportive and encouraging but mostly for the great fun we have together! Dino, thank you so much for being my co-traveler in the world of statistics from my very first days, for all the statistical jokes, the crazy fun, and the great deal of traveling! Finally, I am grateful to my family for the endless love and the unconditional support, psychological and financial. Thanks mum and dad for what I 28

29 am today. You also played a very small role in this thesis! Thanks mum for teaching me how to compose a piece of text properly! Thanks dad for helping me with my math homework! These were the very first steps! Giorgo, thank you for the good pieces of advice on how to deal with the challenging PhD life abroad. Σας ευ χαρισ τώ πoλ ύ! The current PhD was partly funded by the Swedish Research Council projects: "Structural Equation Models with Ordinal Ipsative Variables" and "Structural Equation Modeling with Ordinal Variables"). 29

30 References Bartholomew, D., Knott, M., & Moustaki, I. 2011). Latent Variable Models and Factor Analysis: A Unified Approach. John Wiley series in Probability and Statistics, 3rd ed. Bollen, K. A. 1989). Structural Equations with Latent Variables. Wiley Series in Probability and Mathematical Statistics. New York: Wiley. Brady, H. 1989). Factor and ideal point analysis for interpersonally incomparable data. Psychometrika, 54, Chan, W., & Bentler, P. 1998). Covariance structure analysis of ordinal ipsative data. Psychometrika, 63, Feddag, M.-L., & Bacci, S. 2009). Pairwise likelihood for the longitudinal mixed Rasch model. Computational Statistics and Data Analysis, 53, Forero, C. G., Maydeu-Olivares, A., & Gallardo-Pujol, D. 2009). Factor analysis with ordinal indicators: A Monte Carlo study comparing DWLS and ULS estimation. Structural Equation Modeling: A Multidisciplinary Journal, 16, Jöreskog, K. G. 1990). New developments in LISREL: Analysis of ordinal variables using polychoric correlations and weighted least squares. Quality and Quantity, 24, Jöreskog, K. G. 1994). On the estimation of polychoric correlations and their asymptotic covariance matrix. Psychometrika, 59, Jöreskog, K. G. 2002). Structural equation modeling with ordinal variables using LISREL. Jöreskog, K. G., & Sörbom, D. 1996). LISREL 8 User s Reference Guide. Chicago, IL: Scientific Software International. Lee, S., Poon, W., & Bentler, P. 1990). Full maximum likelihood analysis of structural equation models with polytomous variables. Statistics and Probability Letters, 9, Lee, S., Poon, W., & Bentler, P. 1992). Structural equation models with continuous and polytomous variables. Psychometrika, 57, Lindsay, B. 1988). Composite likelihood methods. Contemporary Mathematics, 80, Liu, J. 2007). Multivariate ordinal data analysis with pairwise likelihood and its extension to SEM. Ph.D. thesis, University of California, Los Angeles, Marden, J. 1995). Analyzing and Modeling Rank Data. Chapman & Hall. Maydeu-Olivares, A., & Böckenholt, U. 2005). Structural equation modeling of paired-comparison and ranking data. Psychological Methods, 10, Muraki, E. 1990). Fitting a polytomous item response model to likert-type data. Applied Psychological Measurement, 14, Muraki, E., & Carlson, E. 1995). Full-information factor analysis for polytomous item responses. Applied Psychological Measurement, 19,

31 Muthén, B. 1984). A general structural equation model with dichotomous, ordered, categorical, and continuous latent variables indicators. Psychometrika, 49, Muthén, L. K., & Muthén, B. O. 2010). Mplus 6 [Computer Software]. Muthén and Muthén, Los Angeles. Olsson, U. 1979). Maximum likelihood estimation of the polychoric correlation coefficient. Psychometrika, 44, Poon, W. Y., & Lee, S. Y. 1987). Maximum likelihood estimation of multivariate polyserial and polychoric correlation coefficients. Psychometrika, 52, Rosseel, Y. 2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48 2), 1 36, Rosseel, Y., Oberski, D., Byrnes, J., Vanbrabant, L., Savalei, V., & Merkle, E. 2012). Package lavaan. Samejima, F. 1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, No. 17. Schervish, M. 1984). Algorithm AS 195: Multivariate normal probabilities with errors bound. Applied Statistics, 3, Schilling, S., & Bock, R. 2005). High-dimensional maximum marginal likelihood item factor analysis by adaptive quadrature. Psychometrika, 70, Team, R. D. C. 2008). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, Thurstone, L. L. 1927). A law of comparative judgment. Psychological Review, 34, Varin, C. 2008). On composite marginal likelihoods. Advances in Statistical Analysis, 92, Varin, C., Reid, N., & Firth, D. 2011). An overview of composite likelihood methods. Statistica Sinica, 21, Yang-Wallentin, F., Jöreskog, K. G., & Luo, H. 2010). Confirmatory factor analysis of ordinal variables with misspecified models. Structural Equation Modeling: A Multidisciplinary Journal, 17, Yao, G., & Böckenholt, U. 1999). Bayesian estimation of Thurstonian ranking models based on the Gibbs sampler. British Journal of Mathematical and Statistical Psychology, 52,

32 Acta Universitatis Upsaliensis Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Social Sciences 86 Editor: The Dean of the Faculty of Social Sciences A doctoral dissertation from the Faculty of Social Sciences, Uppsala University, is usually a summary of a number of papers. A few copies of the complete dissertation are kept at major Swedish research libraries, while the summary alone is distributed internationally through the series Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Social Sciences. Distribution: publications.uu.se urn:nbn:se:uu:diva ACTA UNIVERSITATIS UPSALIENSIS UPPSALA 2013

33 Paper I

34

35 On the identication of the unrestricted Thurstonian model for ranking data Myrsini Katsikatsou Fan Yang-Wallentin Department of Statistics, Uppsala University, Sweden Abstract The identication issues of the unrestricted Thurstonian model for ranking data is the focus of the current paper. Within the Thurstonian framework, each object of those to be ranked is associated with a latent continuous variable, often interpreted as the utility of the object. The unrestricted Thurstonian model, due to the discrete and comparative nature of ranking data, faces more serious identication problems than the indeterminacy of the latent scale origin and unit. Most researchers resort to the study of the unrestricted model referring to the dierences of the object utilities but then the inference on object utilities becomes tricky. Maydeu-Olivares & Böckenholt 2005) suggest a strategy to overcome the identication problem of the unrestricted model referring to object utilities but this requires many extra identication constraints, additional to the ones needed for dening the scale origin and unit. In the current paper, we study the general applicability of the suggested identication approach. Our simulation study indicates that the estimates obtained can be seriously biased with relatively large mean squared errors MSE) when the extra constraints deviate from the true values of the parameters. Besides, the bias and MSE do not seem to decrease with increase in the sample size, and the eect of the constraints is not uniform on all estimated parameters. Keywords: data. unrestricted Thurstonian model; identication; ranking 1 Introduction Ranking experimental designs nd many applications in social and behavioral sciences when the research interest lies in investigating preferences of individuals regarding a set of objects or options e.g. Hult et al., 1997; Florig 1

36 et al., 2001; Morgan et al., 2001; Oakes & Slotterback, 2002). For example, respondents are asked to rank or order) a set of similar products or services, alternative policies on a certain issue, alternative leisure activities or sports, a group of candidates for a certain position, etc. A ranking design can be even used with animals in order to study their preferences in food, for instance e.g. Nombekela et al., 1993). Ranking designs aim to answer the following questions. Which objects are mostly or least preferred among a given set of objects? Is there any dominant preference structure? Are there any relationships among the objects? Objects which are perceived similarly, i.e. have a positive relationship, are expected to have very close or even adjacent ranks systematically. On the contrary, when two objects are systematically apart in rankings, that is an indication of negative relationship. Ranking data dier from ordinal data despite their common discrete nature. In ranking designs, an integer is assigned to each object to denote its ranking within a set of objects. For ordinal data, the integer denotes the position of an object on a prespecied ordinal scale. In other words, individuals are asked to use a given ordinal scale to provide their answers, while answers in ranking data are free of scale. Thus, in ranking designs dierent understandings or usages of a scale by respondents, something that it may occur in ordinal designs, are completely avoided Brady, 1989). The price for this advantage is that the provided rankings are only valid for the specic set of objects studied. For modeling ranking data a variety of models has been suggested; Marden 1995), and Flinger & Verducci 1993) give an extensive overview of these models. In the current study, we concentrate on Thurstonian model class Thurstone, 1927; Böckenholt, 2006) where the basic idea of modeling and analysis is very similar to factor analysis. Each object is assumed to have a latent continuous utility driving the observed rankings. The higher the utility of an object compared to others, the higher rank it gets. The latent vector of utilities is assumed to follow a multinormal distribution and the inference is based on the estimated mean vector and covariance matrix. In the unrestricted Thurstonian model e.g. Chan & Bentler, 1998; Maydeu- Olivares, 1999, 2002; Maydeu-Olivares & Böckenholt, 2005; Yao & Böckenholt, 1999) these two parameters are allowed to be free of any structure. Due to the discrete and comparative nature of ranking data, the unrestricted model has more identication problems than the indeterminacy of the scale origin and unit of latent variables Chan & Bentler, 1998; Yao & Böckenholt, 1999). Hence, researchers resort to the study of the unrestricted model which refers to the dierences of object utilities where the denition of the scale unit is enough to identify the model. The disadvantage though, is that the inference, especially about object relationships, becomes tricky. To overcome the identication problems of the unrestricted model writ- 2

37 ten in terms of object utilities, Maydeu-Olivares & Böckenholt 2005) suggest that one could x additional parameters to those needed for dening the scale origin and unit. A natural question that can be raised is whether these extra constraints aect the general suitability of the approach. We expect that the estimates of the free parameters will be biased and inconsistent unless the extra constraints coincide with the true values of the constrained parameters. We also presume that the more the deviation between the true values and the values the parameters are constrained to, the larger the bias and inconsistency of the estimates will be. To investigate these questions we have conducted a simulation study where the suggested identication approach is employed under dierent types and levels of misspecication. The eect of sample size and model size have also been considered. The structure of the current paper is as follows: Section 2 briey presents the basics about ranking data and Thurstonian ranking modeling followed by an overview of the approach suggested by Maydeu-Olivares & Böckenholt 2005) in Section 3. This approach is discussed in Section 4. In Section 5 we report the results of the simulation study and Section 6 provides an implication of the results for the factor analytic Thurstonian model. In the last section the main conclusions are summarized. 2 Ranking data and Thurstonian modeling Ranking experimental designs Ranking designs can be distinguished into two main categories, complete and partial e.g. Marden, 1995). In a complete ranking design, respondents are presented with a set of m objects {O 1, O 2,..., O m } also referred to as options, choices, alternatives, items, or stimuli in the literature) listed in a prespecied random order and they are asked to assign a rank to each object according to their preference or a certain criterion. The set of ranks is {1,..., m}, where 1 is often dened as the rank to be given to the most preferred object and m to the least preferred one. In partial ranking, respondents are asked to rank only a certain subset of the initial set of objects or provide, in preference order, their k favorite objects k < m) or their k 1 favorite and k 2 least favorite objects k 1 + k 2 < m). In general, any design where not all objects are ranked falls into the category of partial ranking. Another issue regarding the design is whether ties are allowed e.g. Marden, 1995). When they are not allowed, respondents are not permitted to show equal preference to two or more objects. That means that each rank should be used only once. Hence, in the case of complete ranking with no ties the response is an m-dimensional vector of permuted integers from 1 to m and the sample space consists of m! ranking vectors. This is the design we 3

38 focus on in the current study. Note that ranking and ordering objects is the two sides of the same coin. There is a one-to-one correspondence between ranking and ordering vectors. Log-likelihood function Let r = r O1, r O2,..., r Om ) be an observed m-dimensional vector of rankings, where r Oj is the rank assigned to object O j, j = 1,..., m. In complete ranking with no ties there are m! possible ranking patterns and the loglikelihood function for a random sample of size n is: m! ln L θ; r 1,..., r n )) = n c ln π c θ), where θ is a q-dimensional parameter vector q should be much less than m!), n c is the observed frequency of ranking pattern c, with m! c=1 n c = n, and π c θ) is the probability under the model of ranking pattern c, with π c θ) > 0 and m! c=1 π cθ) = 1. The Thurstonian model class Thurstone 1927) introduced a class of models for ranking data that has been highly inuential in the literature e.g. Böckenholt, 1992, 1993, 2006; Chan & Bentler, 1998; Maydeu-Olivares, 1999, 2002; Maydeu-Olivares & Böckenholt, 2005; Yao & Böckenholt, 1999). There are two main assumptions in Thurstonian models: a) the observed ranks assigned to the objects are assumed to be the result of latent continuous variables, often interpreted as object utilities, and b) individual dierences in object utility assessments are assumed to follow a multinormal distribution. Based on the rst assumption, a higher ranking of an object compared to another reects that a higher utility is perceived in the rst object than in the second. Let c=1 u j be the latent utility of object O j, j = 1,..., m, and u = u 1, u 2,..., u m ) be the m-dimensional latent random vector containing all objects utilities. The second assumption of Thurstonian models implies that: u N m µ, Σ), where the parameters µ and Σ may be unrestricted or assumed to follow a specic parametric structure. To write the probability of a ranking vector π c in terms of the parameters µ and Σ let: 4

39 Oh) c denote the object having been assigned rank h when the complete ranking vector falls into category c, h = 1,..., m, c = 1,..., m!, ũ c i = u O c i) u O c i 1) be the utility dierence between objects with adjacent ranks within ranking pattern c, i = 1,..., m 1, ũ c be the m 1)-dimensional vector containing all the aforementioned utility dierences, C c be an m 1) m contrast matrix transforming vector u into ũ c ; its exact form depends on the ranking pattern c in question, and D c = [diagc c ΣC c)] 1/2. The ranking probability π c can be written as follows: ) π c µ, Σ) = P rr c ) = P r u O > u c1) O >... > u c2) O cm) ) = P r u O c 1) u O c 2) > 0 & u O c 2) u O c 3) > 0 & & u O c m 1) u O c m) > 0 = P r ũ c 1 > 0 & ũ c 2 > 0 & & ũ c m 1 > 0 ) = P rũ c > 0) = P rc c u > 0) = Φ m 1 Dc C c µ; D c C c ΣC cd c ), 1) where Φ m 1 D c C c µ; D c C c ΣC cd c ) is the m 1)-variate cumulative normal distribution with correlation matrix D c C c ΣC cd c evaluated at D c C c µ. Identication of the unrestricted Thurstonian model Due to the discrete and comparative nature of ranking data, the utility random vector u is unique only up to a linear transformation. To have a unique solution for µ and Σ, the origin and the unit of the utility scale should be dened. The origin is usually dened by setting the utility mean of an object equal to 0, e.g. µ m = 0, and the unit by xing a utility variance equal to 1, e.g. σ mm = 1. As a consequence, the elements of µ and Σ should be interpreted in relative terms and the inference should be based on the estimates of standardized parameters, such as correlations ρ ij, i j = 1,..., m, variance ratios, e.g. σ jj /σ 11, j = 2,..., m, and standardized mean utility dierences, e.g. µ 1 µ j )/ σ 11 2σ 1j + σ jj, j = 2,..., m. However, in the case of the unrestricted model where µ and Σ are free of structure, the identication problem is more serious than the indeterminacy of the origin and unit of the utility scale. Note that µ and Σ are the parameters of an m-dimensional multinormal distribution but the ranking probability π c is linked to these parameters through an m 1)- dimensional cumulative normal distribution see Equation 1)). 5

40 To reformulate the model in 1) so that the probability π c can be written in terms of the parameters of an m 1)-dimensional normal distribution, many researchers e.g. Chan & Bentler, 1998; Yao & Böckenholt, 1999) suggest that one should consider the random vector ũ which contains all utility dierences with respect to the utility of a certain object. The reference object is arbitrarily chosen. For example, for m = 4 with O 1 as a reference object, ũ gets the form ũ = u 1 u 2, u 1 u 3, u 1 u 4 ) and the connection with the initial random vector u is where B is a contrast matrix of the form B = ũ = Bu, 2) In the general case, with O 1 being the reference object, B is of dimension m 1) m and of form similar as above, and ũ N m 1 µ, Σ ), where µ = Bµ and Σ = BΣB. Hence, the model in 1) is modied as follows: π c µ, Σ ) = Φ m 1 D c C c µ; Dc C c ΣC c Dc ), 3) where D c = [ diagc c ΣC c )] 1/2, and Cc is now an m 1) m 1) contrast matrix transforming vector ũ into ũ c. Since the model in 3) refers to utility dierences, only the unit of the utility scale is needed to be dened. That can be done by xing either a dierence utility mean, e.g. µ 1 = a, a 0, or a dierence utility variance, e.g. σ 1 = b, b > 0. As Chan & Bentler 1998) point out, xing a variance may be preferable because one avoids to decide the sign of constant a. Having dened the scale unit, the unrestricted model in 3) is completely identied. Interpretation and limitations of Thurstonian modeling The ranking of the estimated utility means µ j, j = 1,..., m, gives the ranking pattern with the highest probability under the model which, in turn, reveals the dominant preference in the population with respect to the studied objects. As mentioned before, the parameters of interest are the 6

41 standardized ones. The correlations of object utilities reect the relationships among the objects. High correlations indicate that the corresponding objects are perceived very similarly and are expected to have adjacent or very close ranks regardless of their exact position in the complete ranking. High negative correlations imply that the corresponding objects are seen as contradictory alternatives and are expected to be quite apart in the full ranking vector. The utility variance ratios and the ranking of the object utilities variances σ jj, j = 1,..., m, indicate the relative degree of consensus within the population about the utilities of the objects. Standardized mean dierences show how closely the object utilities are perceived on average. In ranking designs inference is valid only for the specic set of objects studied. Besides, Thurstonian models face certain limitations. The assumption that the object utilities follow multinormal distribution requires that the ranking data come from a homogeneous population and forces the modeled multinomial distribution to be unimodal. Thurstone 1927) had already warned that normality is not a safe assumption if respondents are split into groups in some systematic way. If systematic relationships between dierent types of respondents and dierent objects are suspected then other types of models like mixture models e.g. Marden, 1995) and latent class models e.g. Croon & Luijkx, 1993) are probably more appropriate. Furthermore, the modeled multinomial distribution under the normality assumption is forced to be monotone to some extent in the sense that the probabilities of ranking patterns are decreasing as rankings depart from the mode ranking in distance. This monotonicity becomes stricter in the case where all covariances of object utilities are assumed to be zero. 3 Identifying the unrestricted model written with respect to µ and Σ Alternative representation of ranking vectors Maydeu-Olivares & Böckenholt 2005) employ an alternative representation of ranking vectors see also e.g. Maydeu-Olivares, 1999, 2002) which facilitates the presentation of a Thurstonian ranking model as a Structural Equation Model SEM) with ordinal observed variables. In particular, any m-dimensional complete ranking vector can be written as an m-dimensional vector of binary variables, where m is the number of all possible pairs out of m objects, m = mm 1)/2. Each binary variable corresponds to a pairwise object comparison and, for a given complete ranking pattern, takes the value 1 if the rst object of the pair is preferred, and 0 otherwise. Let y c = y {Oi,O j } be the binary variable corresponding to the comparison between objects O i and O j, i = 1,..., m 1, j = i + 1,..., m, j running faster 7

42 than i, and c = 1,..., m. Then, { 1 y c = y {Oi,O j } = 0 if O i is preferred to O j otherwise. Hence, a ranking vector r = r O1, r O2,..., r Om ) can be transformed into a vector of binary variables y = y 1, y 2,..., y m ) without loss of information. In the general case of m binary variables, there are 2 m possible response patterns, but in complete ranking designs only the m! response patterns of binary variables corresponding to the m! dierent complete rankings are used, 2 m > m! for m > 2. To illustrate this alternative representation, let m = 4 and let the 6 binary variables be dened as follows: y 1 = y {O1,O 2 }, y 2 = y {O1,O 3 }, y 3 = y {O1,O 4 }, y 4 = y {O2,O 3 }, y 5 = y {O2,O 4 }, and y 6 = y {O3,O 4 }. Then, the ranking vector 2, 1, 4, 3), for instance, can be written as 0, 1, 1, 1, 1, 0). On contrary, the pattern 1, 1, 1, 1, 0, 1) does not correspond to a complete ranking vector since the answers to variables y 4, y 5, and y 6 do not allow a complete ranking of objects O 2, O 3, and O 4. Formulating a Thurstonian model as SEM It is assumed that there are m underlying continuous variables partially observed through their corresponding m binary variables and the connection between the two is: y c = { 1 0 ũ c > 0 ũ c < 0, 4) where ũ c is the utility dierence of objects O i and O j both members of pair c, ũ c = u O i u Oj, c = 1,..., m, i = 1,..., m 1, and j = i+1,..., m. The m-dimensional vector ũ of underlying pairwise utility dierences measures the m-dimensional latent vector of object utilities u in a deterministic way: ũ = Au, 5) where A is a known m m contrast matrix. Each row of the matrix corresponds to a pair and each column to an object. The elements of row c which corresponds to the pair {O i, O j } take the values: 1 [A] c k = 1 0 if k = i if k = j otherwise 8, k = 1,..., m.

43 For example, for m = 4, ũ = u 1 u 2, u 1 u 3, u 1 u 4, u 2 u 3, u 2 u 4, u 3 u 4 ) and A is of value A = Hence, Equation 5) is a very special measurement model where the loading matrix is known and equal to A, and there are no measurements errors. Thus, ũ N m µ, Σ ) with µ = Aµ and Σ = AΣA. Note that Σ is a singular covariance matrix, since the rank of matrix A is m 1 Maydeu-Olivares, 1999). The structural model in the case of the unrestricted Thurstonian model written with respect to µ and Σ is: u = µ + ζ, where ζ N m 0, Σ). 6) Thus, the SEM consists of equations 5) and 6) and the ranking probability π c is written as follows: π c µ, Σ) = ˆ R m ˆ φ m ũ ; Aµ, AΣA )dũ, R 1 7) where φ m ũ ; Aµ, AΣA ) is the m-dimensional normal distribution of ũ with { mean vector Aµ and covariance matrix AΣA, and 0, ) if ũ c > 0 R c =, 0) if ũ c < 0, c = 1,..., m. Model identication As discussed in Section 2, the model in terms of the parameters µ and Σ faces more serious identication problem than the standard indeterminacy of the origin and unit of the latent utility scale. Apart from xing a utility mean and variance to 0 and 1, respectively, one should set m extra constraints to get the model identied. Maydeu-Olivares & Böckenholt 2005) suggest the following constraints: all the covariances of the m-th object with all the other objects are xed to 0, σ mi = 0, i = 1,..., m 1, and the variance of the last object σ mm is xed to 1. Model estimation The big advantage of formulating a Thurstonian ranking model as SEM with binary variables is that the conventional 3-stage estimation approach of SEM 9

44 can be applied and the standard SEM packages can be used. To facilitate the application of the three-stage estimation, the standardized form of ũ is used, namely [ z = diag Σ )] 1/2 ũ µ ). This way, both the thresholds and the tetrachoric correlations of z are modeled, τ z = [diagaσa )] 1/2 Aµ, and P z = [diagaσa )] 1/2 AΣA [diagaσa )] 1/2. Hence, in the rst stage of the estimation, the thresholds τ z are estimated by maximizing the likelihood functions of pairs of objects. In the second stage, given the threshold estimates, the tetrachoric correlations are estimated by maximizing the likelihood functions of triplets of objects and pairs of object pairs. Finally, in the third stage, given the estimates of stages one and two, the model parameters µ and Σ are estimated using generalized least squares. Unweighted least squares ULS) or diagonally weighted least squares DWLS) are mostly used. The degrees of freedom should be appropriately adjusted as there are redundancies among the thresholds τ z and the tetrachoric correlations coming from the redundancies among the binary and trinary 1 rankings Brady, 1989; Maydeu-Olivares, 1999). In particular, df = m m + 1)/2) p s, where m m + 1)/2 is the total number of m thresholds and m m 1)/2 tetrachoric correlations, p is the number of parameters to be estimated, and s is the number of redundancies among the thresholds and tetrachoric correlations which is equal to mm 1)m 2)/6 Maydeu-Olivares, 1999; Maydeu-Olivares & Böckenholt, 2005). 4 Discussion of the identication approach suggested in the literature To obtain a better insight into the identication approach suggested by Maydeu-Olivares & Böckenholt 2005), it is important to realize the differences between the two types of parametrization of a ranking probability π c. When π c is written as a function { of the} parameters µ and Σ Equation 3)), the set of parameters µ, vech Σ) contains m 1)m + 2)/2 non-redundant parameters vech is the function transforming a symmetric 1 For a given complete ranking of m objects, binary rankings are the relative rankings within pairs of objects implied by the complete ranking vector, and trinary rankings are the relative rankings within triplets of objects implied by the complete ranking vector see e.g. Brady, 1989). 10

45 matrix into a vector by stacking the elements of its lower triangular, including the main diagonal elements, columnwise). Since the model in 3) refers to object utility dierences only the unit of the utility scale needs to be dened which is done by constraining one parameter. This constraint is enough for the identication of the model. When π c is written as a function of the parameters µ and Σ equations 1) and 7)) the set of parameters {µ, vechσ)} contains mm + { 3)/2 non-redundant } parameters, i.e. m + 1 more parameters than the set µ, vech Σ). The models in 1) and in 7) refer to object utilities, so, two constraints are required to dene the origin and the unit of the utility scale. Recall that the relationship among the two parametrizations is µ = Bµ and Σ = BΣB, where B is as dened in Section 2. Given a certain vector µ and the constraint for determining the scale origin, the vector µ can be retrieved from the equation µ = Bµ. However, given a certain matrix Σ and the constraint for the scale unit, the matrix Σ cannot be uniquely determined by the equation Σ = BΣB. The latter can be seen as a linear system with mm 1)/2 equations and mm + 1)/2 unknowns. There are always m more unknowns than equations and therefore innite solutions for Σ unless at least m elements of Σ are xed. Hence, it becomes clear that { there is} no a one-to-one mapping between the two sets of parameters, µ, vech Σ) and {µ, vechσ)}. In Thurstonian ranking models, the interest lies in estimating the standardized parameters as accurately and eciently as possible. In the unrestricted model, no structure is assumed about the standardized parameters, we let the data speak. Fixing the mean and the variance of an object utility to dene the origin and the unit of the utility scale, respectively, does not imply any structure on the standardized parameters. These two constraints result only in rescaling the population distribution leaving the standardized parameters unchanged. However, any further identication constraints on the elements of the covariance matrix Σ imply a certain structure or value to some correlations and/or variance ratios, something which may not hold in the population. Consequently, misspecications of the model may occur and then, the estimates of the free correlations and variance ratios are expected to be biased. Note that any extra constraints in Σ have no impact on the estimates of standardized mean dierences µ 1 µ j )/ σ 11 2σ 1j + σ jj, j = 2,..., m. These parameters can be also estimated by the unrestricted model written with respect to µ and Σ. It holds that µ 1 µ j )/ σ 11 2σ 1j + σ jj = µ i / σ ii, i = 1,..., m 1. On the contrary, it is not possible to derive the correlations ρ ij and variance ratios σ jj /σ 11 of object utilities from the variances of utility dierences σ ii. The identication approach suggested by Maydeu-Olivares & Böckenholt 2005) makes, in fact, assumptions about the value of a certain correlation 11

46 and variance ratio. Let assume that the scale unit is dened by xing σ 11 = 1. The approach suggests to x additionally the covariances of the utility of the last object with the other object utilities and the variance of the utility of the last object, i.e. to x σ mi = k i, i = 1,..., m 1, and σ mm = k m, where k i 's and k m are constants. These m extra constraints along with σ 11 = 1 lead to the constraints: ρ m1 = k 1 km and σ mm σ 11 = k m. As Maydeu-Olivares & Böckenholt 2005) comment, the above set of m extra identication constraints is not unique and other identication constraints can be specied that yield equivalent model ts. However, whichever set of m elements of Σ we choose to x, additionally to σ 11 = 1, a certain structure or value on some correlations and/or variance ratios is implied. If one chooses to x m covariances, instead of m 1 covariances and one variance, then a certain ratio of correlations is assumed to be equal to a constant. To illustrate this, let σ mi = k i, k i 's are constants, i = 1,..., m 1, and σ st = k st for a specic s and t, s = 2,..., m 1, t = 1,..., s 1, and k st is constant. Then, it is implied that ρ ms ρ mt ρ st ρ 2 m1 = k sk t k st k1 2. If one chooses instead to x the rest m 1 variances additionally to σ 11 = 1) and one covariance, this assumes that all variance ratios and one correlation are equal to certain constants. To illustrate this, let σ ii = k i, i = 2,..., m, k i 's are constants, and σ st = k st for a specic s and t, s t = 1,..., m, and k st is constant. The direct implications of these constraints are: σ ii σ 11 = k i, i = 2,..., m, and ρ st = k st ks k t. Therefore, any m extra constraints in Σ imply in fact that the data come from a population where certain correlations and/or variance ratios are equal to specic values or follow a specic structure. The question that follows naturally is how these extra constraints aect the estimates of the remaining free correlations and variance ratios. If the data indeed come from the implied population, then there is no misspecication and the estimates are expected to be as accurate and ecient as the estimation method permits. But when there are deviations, how large are the bias and MSE of the estimates? Is the size of bias and MSE uniform on all estimates? The simulation study presented in Section 5 tries to give an answer to this question. 12

47 u N 4 µ, Σ) with µ 1 µ = µ 2 µ 3, Σ = 0 Model I m = 4) 1 σ 21 σ 22 σ 31 σ 32 σ Model II m = 7) u N 7 µ, Σ) with µ 1 1 µ 2 σ 21 σ 22 µ 3 σ 31 σ 32 σ 33 µ = µ 4, Σ = σ 41 σ 42 σ 43 σ 44 µ 5 σ 51 σ 52 σ 53 σ 54 σ 55 µ 6 σ 61 σ 62 σ 63 σ 64 σ 65 σ Table 1: The models to be estimated It is necessary to emphasize that the goodness-of-t statistics are unable to detect possible misspecications induced by these extra identication constraints. As mentioned earlier, all possible sets of m extra constraints in Σ yield equivalent model ts. Hence, we cannot distinguish between constraints where no misspecication occurs from those inducing serious misspecications. The values of goodness-of-t statistics are exactly the same for both the unrestricted model with respect to µ and Σ and the unrestricted model with respect to µ and Σ. Both models use the same information from data and estimate the same number of parameters. Hence, if the observed data come indeed from a latent multinormal distribution, then both models will present the same good t. However, the unrestricted model with respect to µ and Σ may involve serious misspecications. The two models involve two dierent sets of parameters with the size of the sets being dierent. The issue of models with equal t here is not the same as that in factor analysis models where rotation of the solution is possible. Observe, that in factor analysis, one scale is dened for each latent variable, while, in the ranking framework, only one scale is dened for all m latent object utilities so that inference about object preferences can be made. 5 Simulation study Set-up of the simulation study The simulation study focuses on the estimation of the unrestricted model written with respect to µ and Σ where the identication constraints suggested by Maydeu-Olivares & Böckenholt 2005) are adopted, i.e. the con- 13

48 and Σ = Model I m = 4) Model II m = 7) u N 4 µ, Σ) with u N 7 µ, Σ) with µ = µ = and [σ 41 ] [σ 42 ] [σ 43 ] [σ 44 ] Σ = [σ 71 ] [σ 72 ] [σ 73 ] [σ 74 ] [σ 75 ] [σ 76 ] [σ 77 ] where where σ 41 σ 42 σ 43 σ 44 σ 71 σ 72 σ 73 σ 74 σ 75 σ 76 σ 77 situation situation situation situation situation situation situation situation situation Table 2: The population models from which data are generated straints µ m = 0 and σ 11 = 1 to dene the scale origin and unit, respectively, as well as σ m1 =... = σ mm 1) = 0 and σ mm = 1. The objective is to study the impact of the last m extra constraints on the estimates of all the standardized parameters i.e. correlations, variance ratios, and standardized mean dierences) in terms of bias and mean squared error MSE) under different types and levels of misspecication 9 dierent situations), model sizes m = 4, 7), and sample sizes n = 500, 1000). In total, we examine 36 different experimental conditions where 2000 replicates have been carried out for each condition. The two models considered are referred to as Model I and Model II. Model I is a small model with m = 4 objects to be ranked, while Model II is larger with m = 7. Both models are also studied under correct specication. The results under the correctly specied models are used as a benchmark to make comparisons with the results obtained under dierent types and levels of misspecication. By correct specication we mean that the population model used to generate the data is the same as the model to be estimated, 14

49 while these two dier in the case of misspecication. In our study, in order to generate dierent types and levels of misspecication, we keep the model to be estimated the same and change the population model. For Model I the model to be estimated is a four-dimensional normal distribution and for Model II it is a seven-dimensional normal distribution with the structure of parameters µ and Σ due to the identication constraints being as specied in Table 1. For both models the implications of the identication constraints on the standardized parameters are that ρ mi = 0, i = 1,..., m 1, and σ mm /σ 11 = 1. The population models used to generate the data are given in Table 2. Situation 1 corresponds to the case of correctly specied model, while situations 2-9 to cases where the model is misspecied. Situations 2-4 refer to three dierent levels of misspecication of covariance σ m1. The true values of σ m1 are 0.1, 0.3, and 0.6, respectively, but it is xed to 0 in the model to be estimated. That means that the true values of ρ m1 are 0.1, 0.3, and 0.6, respectively, but in the estimation the correlation is xed to 0. Situations 5-7 refer to three dierent levels of misspecication of the variance σ mm, which, in turn, implies misspecication of the variance ratio σ mm /σ 11. The true values of σ mm and of σ mm /σ 11 are 1.1, 1.5, and 2, respectively, but in the estimated model both the variance and the variance ratio are constrained to be equal to 1. Finally, situations 8-9 represent the cases where all covariances σ mi, i = 1,..., m 1, and the variance σ mm are misspecied. The level of misspecication is low in situation 8 and larger in situation 9 for all these parameters. These misspecications result in misspecifying all correlations ρ mi, i = 1,..., m 1, as well as the variance ratio σ mm /σ 11, the true values of which are given in Table 3. These nine situations of misspecication are investigated under two different sample sizes, 500 and 1000, within each model. Data generation and analysis The following steps have been taken in order to generate data for each experimental condition: 1. An m-dimensional random vector u is generated from N m µ, Σ) with µ and Σ as dened in Table The random vector u is transformed to the m-dimensional vector ũ using Equation 5). 3. The m-dimensional vector ũ is transformed to the m-dimensional vector of binary variables y using expression 4). 15

50 Model I Model II ρ 41 ρ 42 ρ 43 σ 44 /σ 11 ρ 71 ρ 72 ρ 73 ρ 74 ρ 75 ρ 76 σ 77 /σ 11 situation situation Table 3: The true values of the misspecied standardized parameters in situations 8 and 9 4. Steps 1-3 are repeated n times to generate the required sample size within each experimental condition. Then, the analysis proceeds as follows: 1. For each of the 2000 generated data sets, the model in question given in Table 1 is estimated. For this we used Mplus version 5.21), and the same input syntax le as Maydeu-Olivares & Böckenholt 2005) used in their study provided on the webpage supplemental/met_10_3_285/met_10_3_285_supp.html. 2. All 2000 replications are screened for non-admissible solutions, which in our case are non-positive estimated variances and correlations whose absolute value is larger than 1. The replications with improper solutions are considered as invalid observations and are eliminated from the rest of the analysis. 3. The estimates of the free standardized parameters i.e. standardized mean dierences, variance ratios, and correlations) are calculated based on the estimates of µ and Σ, and the bias and MSE are computed following the formulas below. Performance criteria Taking into account only the valid replications within each experimental condition, the bias and MSE of the standardized parameter estimates are calculated as follows: Bias = 1 R R ˆθi θ), and i=1 MSE = 1 R R i=1 ) 2 ˆθi θ, where R is the number of valid replicates, ˆθ i is the estimate of a standardized parameter at the i-th valid replication, and θ is its corresponding true value. 16

51 Results There are three experimental conditions where the problem of non-admissible solutions occurs and all three concern the larger model, Model II, and the misspecication situations 8-9, i.e. the situations where all extra identication constraints deviate from the true values of the parameters. The specic cases and the corresponding percentage of valid replications are given in Table 4. The percentage is remarkably low when all parameters σ m1,..., σ mm are seriously misspecied situation 9). Moreover, once the level of misspecication is relatively high, the sample size has a small eect in ameliorating the problem of non-admissibility. Under Model II and situation 9, doubling the sample size from 500 to 1000 caused an increase of the percentage of valid replications from 36.5% to only 42.05%. The bias and MSE for all standardized parameters and all experimental conditions are reported in tables 7-14 given in the Appendix. In these tables, for each parameter it is reported: the true value, the mean of the replicated estimates, the bias, and the MSE. Figures 1-9 depict the bias and MSE of the correlation and variance ratio estimates for all experimental conditions. To get a better picture of the impact of dierent misspecications, the average bias and MSE over the parameters of the same type within the same experimental condition has been calculated. Table 5 reports the results. Based on all these tables and gures the major conclusions are as follows: With no surprise, the bias and MSE for all standardized parameters, both models, within situation 1 correctly specied model) are very close to zero and decreasing with increases in the sample size. As expected, the bias and MSE of the estimates of the standardized mean dierences are not aected by any kind and level of misspecication. They are very close to zero for all experimental conditions. As expected, the bias and MSE of the estimates of the variance ratios and correlations are aected by the dierent types and levels of misspecication. In particular, the absolute value of bias and the MSE increase as the degree of misspecication gets higher. The increase of both the bias in absolute value) and the MSE is not uniform for all parameters. An interesting pattern can be seen in Figure 5, which depicts the eect of misspecication of σ 71 on the estimates of correlations within Model II. The bias and MSE of the estimates of ρ i1, i = 2,..., 6 are on average smaller than the bias and MSE of the estimates of the rest correlations. When only one parameter is misspecied situations 2-7), the bias and MSE of the variance ratio estimates seem to be much more af- 17

52 Experimental condition Valid replications Model II - situation 8 - n = % Model II - situation 9 - n = % Model II - situation 9 - n = % Table 4: Experimental conditions with valid replications less than 100% fected when the misspecied parameter is a covariance situations 2-4) rather than a variance situations 5-7), while these two types of misspecication seem to cause similar impact on the bias and MSE of correlation estimates. Moreover, when the variance σ mm is misspecied situations 5-7) within Model I, the average bias and average MSE of variance ratio estimates do not seem to be aected by the degree of misspecication. On the contrary, they are very close to the level of average bias and average MSE found under the correctly specied model situation 1). For Model II, this observation applies only for the average MSE of variance ratio estimates. When only the covariance σ m1 is misspecied situations 2-4), both variance ratios and correlations are systematically overestimated. When only the variance σ mm is misspecied situations 5-7), the correlations are overestimated systematically, while the variance ratios are overestimated when their true values are smaller than 1 and underestimated when their true values are larger than 1. When all parameters σ mi, i = 1,..., m, and σ mm are misspecied situations 8-9), then almost all variance ratios and correlations are underestimated. Interestingly, once the model is misspecied, an increase in the sample size seems to play marginal role in decreasing the MSE and the bias. This can be seen in gures 1-9 where the shapes of the lines and the level of values of bias and MSE are very similar on the graphs referring to the same experimental condition except for the sample size. To get a better idea about the eect of the sample size we calculated the relative absolute dierence of average bias RAD Av.Bias ) and the relative dierence of average MSE RD Av.MSE ) for the two sample sizes within the same misspecication situation and model. In particular, RAD Av.Bias = Average Bias n=1000 Average Bias n=500 Average Bias n=500 RD Av.MSE = Average MSE n=1000 Average MSE n=500 Average MSE n= , and

53 A minus sign is interpreted as a decrease in the absolute average bias or average MSE after increasing the sample size. Table 6 presents the results. As it can be seen, the RAD Av.Bias for the correlations are very close to zero for almost all situations of misspecication in both models. This implies that doubling the sample size played almost no role in decreasing the absolute average bias of the correlation estimates. Using the relative dierences measured under the correctly specied model situation 1) as a benchmark, we observe that, after doubling the sample size, the decrease in absolute average bias and average MSE is smaller for most of the misspecication situations compared to the decrease observed in situation 1. Finally, there are few cases where the relative dierence is positive implying that despite the double sample size the absolute average bias and the average MSE did not get any smaller at all. 6 Implications for the exploratory factor analytic Thurstonian model The main conclusions of Section 5 are valid for the exploratory factor analytic Thurstonian model written with respect to µ and Σ since similar identication constraints are needed. Within that framework the covariance matrix Σ is assumed to follow the structure imposed by exploratory factor analysis, namely Σ = ΛΛ + Ψ, where Ψ is a diagonal matrix. Due to the discrete and comparative nature of ranking data, the exploratory factor analytic Thurstonian model has more identication problems than a conventional exploratory factor model with ordinal data e.g. Brady, 1989; Maydeu-Olivares & Böckenholt, 2005). Apart from the rotational indeterminacy of the loading matrix Λ, one needs to dene the origin of the utility scale as well. This may result in a matrix Λ of the following form Maydeu- Olivares & Böckenholt, 2005): λ Λ = 0,. λ m 2)k λ m 1)1... λ m 1)k where k is the number of common factors. Such a structure of Λ along with a diagonal Ψ results in a covariance matrix Σ with one row where all the covariances are equal to 0. In other words, we get a covariance matrix Σ with 19

54 Model I m = 4) Variance ratios Correlations n = 500 Average Bias Average MSE Average Bias Average MSE situation situation situation situation situation situation situation situation situation n = 1000 situation situation situation situation situation situation situation situation situation Model II m = 7) Variance ratios Correlations n = 500 Average Bias Average MSE Average Bias Average MSE situation situation situation situation situation situation situation situation situation n = 1000 situation situation situation situation situation situation situation situation situation Table 5: Average bias and MSE over parameters of same type 20

55 the same constraints as before. This problem is avoided if one considers the factor analytic Thurstonian model written with respect to µ and Σ instead. 7 Summary and Conclusions The Thurstonian framework oers a class of models for ranking data with an easy interpretation. The mean vector µ reveals the dominant preference structure, the standardized mean dierences µ i µ j )/ σ ii 2σ ij + σ jj show how similarly object utilities are perceived on average, the variance ratios σ ii /σ jj describe the relative variability of the perceived utility for each object, and the correlations ρ ij give information about the relationships among the objects. Any Thurstonian model can be written with respect to either object utilities or the dierences of object utilities. The parameter sets of the two models are connected with a certain deterministic relationship but there is no a one-to-one correspondence between them. The aim is to estimate the standardized parameters, i.e. the standardized mean differences, the correlations, and the variance ratios of object utilities. This can be done only within the model written with respect to object utilities. Under the model written in terms of the dierences of object utilities we can only estimate the standardized mean dierences. The latter makes the inference about object relationships dicult. On the other hand, the unrestricted Thurstonian model written with respect to object utilities faces serious identication problems and requires extra identication constraints, additional to those dening the origin and unit of the utility scale. Maydeu- Olivares & Böckenholt 2005) suggest a set of such extra constraints. The general applicability of this identication strategy is studied in the current paper. We nd that the suggested approach leads to reliable estimates of the standardized parameters as long as the extra identication constraints coincide with or are very close to) the true values of the constrained parameters. When this condition does not hold, the estimates of almost all correlations and variance ratios present a considerable amount of bias and MSE. The level of bias and MSE increases with the misspecication level and not in a uniform way for all parameter estimates. An increase in the sample size seems to have very marginal eect in decreasing the bias and MSE. Besides, the goodness-of-t statistics are unable to detect possible misspecications induced by the identication constraints. As a consequence, the approach should be used with great caution. Before adopting these extra identication constraints one should resort to theoretical or previous empirical results to conrm that the extra constraints are reasonable and justied for the population in question. Further research on statistics that could be used to investigate the data at hand and decide whether the suggested extra identication constraints are reasonable for the observed data could 21

56 Variance ratios Correlations Model I m = 4) RAD Av.Bias RD Av.MSE RAD Av.Bias RD Av.MSE situation situation situation situation situation situation situation situation situation Model II m = 7) situation situation situation situation situation situation situation situation situation Table 6: Relative absolute dierence of average bias RAD Av.Bias ) and average MSE RD Av.MSE ) enhance the general suitability of the identication approach suggested in the literature. 22

57 Figure 1: The eect of σ 14 misspecication on bias and MSE of variance ratios and correlations, m = 4 Figure 2: The eect of σ 44 misspecication on bias and MSE of variance ratios and correlations, m = 4 23

58 Figure 3: The eect of σ 14, σ 24, σ 34, σ 44 misspecications on bias and MSE of variance ratios and correlations, m = 4 Figure 4: The eect of σ 17 misspecication on bias and MSE of variance ratios, m = 7 24

59 Figure 5: The eect of σ 17 misspecication on bias and MSE of correlations, m = 7 Figure 6: The eect of σ 77 misspecication on bias and MSE of variance ratios, m = 7 25

60 Figure 7: The eect of σ 77 misspecication on bias and MSE of correlations, m = 7 Figure 8: The eect of σ 17, σ 27,..., σ 77 misspecications on bias and MSE of variance ratios, m = 7 26

61 Figure 9: The eect of σ 17, σ 27,..., σ 77 misspecications on bias and MSE of correlations, m = 7 27

62 Appendix Model I m = 4) Situation 1: n = 1000 Correctly specied model True Mean Bias MSE σ 22 /σ σ 33 /σ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ Situation 2 σ 14 = 0.1) Situation 3 σ 14 = 0.3) True Mean Bias MSE True Mean Bias MSE σ 22 /σ σ 33 /σ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ Situation 4 σ 14 = 0.6) Situation 5 σ 44 = 1.1) True Mean Bias MSE True Mean Bias MSE σ 22 /σ σ 33 /σ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ Situation 6 σ 44 = 1.5) Situation 7 σ 44 = 2) True Mean Bias MSE True Mean Bias MSE σ 22 /σ σ 33 /σ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ Situation 8 small missp/tion in all σ 4i ) Situation 9 larger missp/tion in all σ 4i ) True Mean Bias MSE True Mean Bias MSE σ 22 /σ σ 33 /σ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ Table 7: Bias and MSE of the estimates of standardized parameters, n = 1000, Model I 28

63 Model I m = 4) Situation 1: n = 500 Correctly specied model True Mean Bias MSE σ 22 /σ σ 33 /σ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ Situation 2 σ 14 = 0.1) Situation 3 σ 14 = 0.3) True Mean Bias MSE True Mean Bias MSE σ 22 /σ σ 33 /σ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ Situation 4 σ 14 = 0.6) Situation 5 σ 44 = 1.1) True Mean Bias MSE True Mean Bias MSE σ 22 /σ σ 33 /σ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ Situation 6 σ 44 = 1.5) Situation 7 σ 44 = 2) True Mean Bias MSE True Mean Bias MSE σ 22 /σ σ 33 /σ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ Situation 8 small missp/tion in all σ 4i ) Situation 9 larger missp/tion in all σ 4i ) True Mean Bias MSE True Mean Bias MSE σ 22 /σ σ 33 /σ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ Table 8: Bias and MSE of the estimates of standardized parameters, n = 500, Model I 29

64 Model II m = 7) Situation 1: n = 1000 Correctly specied model True Mean Bias MSE σ 22 /σ σ 33 /σ σ 44 /σ σ 55 /σ σ 66 /σ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ σ11 µ 1 µ 2 2σ 12 +σ µ 1 µ 3 σ11 2σ 13 +σ µ 1 µ 4 σ11 2σ 14 +σ µ 1 µ 5 σ11 2σ 15 +σ µ 1 µ 6 σ11 2σ 16 +σ µ 1 µ 7 σ11 2σ 17 +σ Table 9: Bias and MSE of the estimates of standardized parameters for the baseline model, n = 1000, Model II 30

65 Model II m = 7) Situation 2 σ 17 = 0.1) Situation 3 σ 17 = 0.3) n = 1000 True Mean Bias MSE True Mean Bias MSE σ 22 /σ σ 33 /σ σ 44 /σ σ 55 /σ σ 66 /σ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ µ 1 µ 5 σ11 2σ 15+σ µ 1 µ 6 σ11 2σ 16+σ µ 1 µ 7 σ11 2σ 17+σ Situation 4 σ 17 = 0.6) Situation 5 σ 77 = 1.1) True Mean Bias MSE True Mean Bias MSE σ 22 /σ σ 33 /σ σ 44 /σ σ 55 /σ σ 66 /σ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ µ 1 µ 5 σ11 2σ 15+σ µ 1 µ 6 σ11 2σ 16+σ µ 1 µ 7 σ11 2σ 17+σ Table 10: Bias and MSE of the estimates of standardized parameters, n = 1000, Model II 31

66 Model II m = 7) Situation 6 σ 77 = 1.5) Situation 7 σ 77 = 2) n = 1000 True Mean Bias MSE True Mean Bias MSE σ 22 /σ σ 33 /σ σ 44 /σ σ 55 /σ σ 66 /σ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ µ 1 µ 5 σ11 2σ 15+σ µ 1 µ 6 σ11 2σ 16+σ µ 1 µ 7 σ11 2σ 17+σ Situation 8 small missp/tion in all σ 7i ) Situation 9 larger missp/tion in all σ 7i ) 841 valid replications out of 2000) True Mean Bias MSE True Mean Bias MSE σ 22 /σ σ 33 /σ σ 44 /σ σ 55 /σ σ 66 /σ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ µ 1 µ 5 σ11 2σ 15+σ µ 1 µ 6 σ11 2σ 16+σ µ 1 µ 7 σ11 2σ 17+σ Table 11: Bias and MSE of the estimates of standardized parameters, n = 1000, Model II 32

67 Model II m = 7) Situation 1: n = 500 Correctly specied model True Mean Bias MSE σ 22 /σ σ 33 /σ σ 44 /σ σ 55 /σ σ 66 /σ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ σ11 µ 1 µ 2 2σ 12 +σ µ 1 µ 3 σ11 2σ 13 +σ µ 1 µ 4 σ11 2σ 14 +σ µ 1 µ 5 σ11 2σ 15 +σ µ 1 µ 6 σ11 2σ 16 +σ µ 1 µ 7 σ11 2σ 17 +σ Table 12: Bias and MSE of the estimates of standardized parameters for the baseline model, n = 500, Model II 33

68 Model II m = 7) Situation 2 σ 17 = 0.1) Situation 3 σ 17 = 0.3) n = 500 True Mean Bias MSE True Mean Bias MSE σ 22 /σ σ 33 /σ σ 44 /σ σ 55 /σ σ 66 /σ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ µ 1 µ 5 σ11 2σ 15+σ µ 1 µ 6 σ11 2σ 16+σ µ 1 µ 7 σ11 2σ 17+σ Situation 4 σ 17 = 0.6) Situation 5 σ 77 = 1.1) True Mean Bias MSE True Mean Bias MSE σ 22 /σ σ 33 /σ σ 44 /σ σ 55 /σ σ 66 /σ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ µ 1 µ 5 σ11 2σ 15+σ µ 1 µ 6 σ11 2σ 16+σ µ 1 µ 7 σ11 2σ 17+σ Table 13: Bias and MSE of the estimates of standardized parameters, n = 500, Model II 34

69 Model II m = 7) Situation 6 σ 77 = 1.5) Situation 7 σ 77 = 2) n = 500 True Mean Bias MSE True Mean Bias MSE σ 22 /σ σ 33 /σ σ 44 /σ σ 55 /σ σ 66 /σ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ µ 1 µ 5 σ11 2σ 15+σ µ 1 µ 6 σ11 2σ 16+σ µ 1 µ 7 σ11 2σ 17+σ Situation 8 small missp/tion in all σ 7i ) Situation 9 larger missp/tion in all σ 7i ) 1994 valid replications out of 2000) 730 valid replications out of 2000) True Mean Bias MSE True Mean Bias MSE σ 22 /σ σ 33 /σ σ 44 /σ σ 55 /σ σ 66 /σ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ σ11 2σ µ 1 µ 2 12+σ µ 1 µ 3 σ11 2σ 13+σ µ 1 µ 4 σ11 2σ 14+σ µ 1 µ 5 σ11 2σ 15+σ µ 1 µ 6 σ11 2σ 16+σ µ 1 µ 7 σ11 2σ 17+σ Table 14: Bias and MSE of the estimates of standardized parameters, n = 500, Model II 35

70 References Böckenholt, U. 1992). Thurstonian representation for partial ranking data. British Journal of Mathematical and Statistical Psychology, 45, Böckenholt, U. 1993). Applications of Thurstonian models to ranking data. Probability Models and Statistical Analyses for Ranking Data, edited by Fligner, M. & Verducci, J., New York, NY: Springer-Verlag, pp ). Böckenholt, U. 2006). Thurstonian-based analyses: past, present, and future utilities. Psychometrika, 71, Brady, H. 1989). Factor and ideal point analysis for interpersonally incomparable data. Psychometrika, 54, Chan, W., & Bentler, P. 1998). Covariance structure analysis of ordinal ipsative data. Psychometrika, 63, Croon, M. A., & Luijkx, R. 1993). Probability models and statistical analyses for ranking data, chap. Latent structure models for ranking data, pp. 5374). NY:Springer-Verlag. Flinger, M. A., & Verducci, J. S. 1993). Multistage ranking models. Journal of the American Statistical Association, 83, Florig, H. K., Morgan, M. G., Morgan, K. M., Jenni, K. E., Fischho, B., Fischbeck, P. S., & DeKay, M. L. 2001). A deliberative method for ranking risks i): Overview and test bed development. Risk Analysis, 21, Hult, G. T. M., Neese, W., & Bashaw, R. E. 1997). Faculty perceptions of marketing journals. Journal of Marketing Education, 19, Marden, J. 1995). Analyzing and Modeling Rank Data. Chapman & Hall. Maydeu-Olivares, A. 1999). Thurstonian modeling of ranking data via mean and covariance structure analysis. Psychometrika, 64, Maydeu-Olivares, A. 2002). Limited information estimation and testing of Thurstonian models for preference data. Mathematical Social Sciences, 43, Maydeu-Olivares, A., & Böckenholt, U. 2005). Structural equation modeling of paired-comparison and ranking data. Psychological Methods, 10,

71 Morgan, K. M., DeKay, M., Fischbeck, P. S., Morgan, M. G., Fischho, B., & Florig, H. K. 2001). A deliberative method for ranking risks ii): Evaluation of validity and agreement among risk managers. Risk Analysis, 21, Muthén, L. K., & Muthén, B. O. 2010). Mplus 6 [Computer Software]. Muthén and Muthén, Los Angeles. Nombekela, S. W., Murphy, M. R., Gonyou, H. W., & Marden, J. I. 1993). Dietary preferences in early lactation cows as aected by primary tastes and some common feed avors. Journal of Dairy Science, 77, Oakes, M. E., & Slotterback, C. S. 2002). The good, the bad, and the ugly: Characteristics used by young, middle-aged, and older men and women, dieters and non-dieters to judge healthfulness of foods. Appetite, 38, Thurstone, L. L. 1927). A law of comparative judgment. Psychological Review, 34, Yao, G., & Böckenholt, U. 1999). Bayesian estimation of Thurstonian ranking models based on the Gibbs sampler. British Journal of Mathematical and Statistical Psychology, 52,

72

73 Paper II

74

75 Computational Statistics and Data Analysis ) Contents lists available at SciVerse ScienceDirect Computational Statistics and Data Analysis journal homepage: Pairwise likelihood estimation for factor analysis models with ordinal data Myrsini Katsikatsou a,, Irini Moustaki b, Fan Yang-Wallentin a, Karl G. Jöreskog a,c a Department of Statistics, University of Uppsala, Box 513, SE-75120, Sweden b Department of Statistics, London School of Economics, WC2A 2AE, United Kingdom c Department of Economics, Norwegian School of Management, 0442 Oslo, Norway a r t i c l e i n f o a b s t r a c t Article history: Received 26 September 2011 Received in revised form 16 April 2012 Accepted 16 April 2012 Available online 24 April 2012 Keywords: Composite maximum likelihood Factor analysis Ordinal data Pairwise likelihood Three-stage estimation Item response theory approach A pairwise maximum likelihood PML) estimation method is developed for factor analysis models with ordinal data and fitted both in an exploratory and confirmatory set-up. The performance of the method is studied via simulations and comparisons with full information maximum likelihood FIML) and three-stage limited information estimation methods, namely the robust unweighted least squares 3S-RULS) and robust diagonally weighted least squares 3S-RDWLS). The advantage of PML over FIML is mainly computational. Unlike PML estimation, the computational complexity of FIML estimation increases either with the number of factors or with the number of observed variables depending on the model formulation. Contrary to 3S-RULS and 3S-RDWLS estimation, PML estimates of all model parameters are obtained simultaneously and the PML method does not require the estimation of a weight matrix for the computation of correct standard errors. The simulation study on the performance of PML estimates and estimated asymptotic standard errors investigates the effect of different model and sample sizes. The bias and mean squared error of PML estimates and their standard errors are found to be small in all experimental conditions and decreasing with increasing sample size. Moreover, the PML estimates and their standard errors are found to be very close to those of FIML Elsevier B.V. All rights reserved. 1. Introduction Factor analysis is frequently employed in social sciences where the main interest lies in measuring and relating unobserved constructs, such as emotions, attitudes, beliefs and behavior. The main idea behind the analysis is that the latent variables referred to also as factors) account for the dependencies among the observed variables referred to also as items or indicators) in the sense that if the factors are held fixed, the observed variables would be independent. Theoretically, factor analysis can be distinguishable between exploratory and confirmatory analysis, but in practice the analysis always lies between the two. In exploratory factor analysis the goal is the following: for a given set of observed variables x 1,..., x p one wants to find a set of latent factors ξ 1,..., ξ k, fewer in number than the observed variables k < p), that contain essentially the same information. In confirmatory factor analysis, the objective is to verify a social theory. Hence, a factor model is specified in advance and its fit to the empirical data is tested. The data usually encountered in social sciences is of categorical nature ordinal or nominal). In the literature, there are two main approaches for the analysis of ordinal variables with factor models. The Underlying Response Variable URV) Corresponding author. Tel.: addresses: myrsini.katsikatsou@statistik.uu.se M. Katsikatsou), i.moustaki@lse.ac.uk I. Moustaki), fan.yang@statistik.uu.se F. Yang-Wallentin), karl.joreskog@statistik.uu.se K.G. Jöreskog) /$ see front matter 2012 Elsevier B.V. All rights reserved. doi: /j.csda