Introduction to latent class model

Size: px

Start display at page:

Download "Introduction to latent class model"

Bethanie Wilson
5 years ago
Views:

1 Epi 950, Fall Introduction to latent class model Latent variables Manifest variables Categorical Continuous Categorical Latent class analysis Latent trait analysis Continuous Latent profile analysis Factor analysis The latent class model (LCM) is a statistical method for classifying individuals into population subgroups by using their responses to the manifest items LCM explains the relationship among categorical manifest items by positing the existence of a latent classifier

2 Epi 950, Fall

3 Epi 950, Fall Example Consider the following 2 items from a questionnaire: 1. Is the federal government too big? 2. Should Congress pass a capital gains tax cut this year? Item 1 Yes No Item 2 Yes No X 2 = 36.74, df = 1, p = 0.00; odds ratio = 4.7 These statistics were used to make claims that those two items are related

4 Epi 950, Fall But, if you adjust for another classification, such as fiscal conservatism, Conservative Item 2 Item 1 Not Item 1 Yes No conservative Yes No Yes Yes 5 20 No 20 5 Item 2 No Conservative: X 2 = 0.01, p = 0.93; odds ratio = 0.95 Not conservative: X 2 = 0.00, p = 1.00; odds ratio = 1.00 There is no relationship between Item 1 and Item 2 The item association is explained by conservatism means items 1 and 2 are conditionally independent given conservatism

5 Epi 950, Fall What if we don t or can t observe fiscal conservatism which should be conditioned on? We could fit LCM in order to allocate an object to one of these latent classes on the basis of the responses to the manifest items Questions to answer: How many underlying classes are there? What is the prevalence in each of the latent classes What is the probability that a particular individual will be in a particular class?

6 Epi 950, Fall Application Many medical conditions cannot be diagnosed directly because the root of the condition is deep-seated. For example, a definite reference test for AIDS is not observable. However, some diagnostic tests for HIV are available, and it s assumed that they are imperfect indicators of the true AIDS status. It would be useful if we could use observations of a patient s test results to estimate the sensitivity and the specificity. An LCM may help us to do this.

7 Epi 950, Fall LCM is one of the most used of all latent variable modeling. Major uses include: to reduce the dimension of data by explaining the associations between the observed variables in terms of membership of a small number of latent classes to understand the inter-relationships between observed variables to estimate the probabilities of belonging to each class on the basis of the individual s response pattern to allocate an individual to one of latent classes

8 Epi 950, Fall Review of Bernoulli trial The Bernoulli trial process is the mathematical abstraction of coin tossing that satisfies the following assumption: Each trial has two possible outcomes: 1 if success Y = 0 if failure The trials are independent The probability of success is P(Y = 1) = p and the probability of failure is P(Y = 0) = 1 p It has probability function, P(Y = y) = p y (1 p) 1 y, y = 0, 1

9 Epi 950, Fall When each trial has K (> 2) possible outcomes (e.g., rolling a dice): The random variable Y can take any value from 1 to K The probability for each event is P(Y = k) = p k for k = 1, 2,...,K with constraint K k=1 p k = 1 It has probability function, P(Y = k) = p k where I(Y = k) = I(Y =1) I(Y =2) I(Y =K) = p1 p2 pk K I(Y =k) = pk, k = 1, 2,..., K, k=1 1 if Y = k 0 if Y k

10 Epi 950, Fall When we have M (> 2) independent trials where mth trial has r m possible outcomes (e.g., rolling a dice M times): The random variable Y m can take any value from 1 to r m for m = 1, 2,...,M The probability for each event is P(Y m = k) = p mk with constraint r m k=1 p mk = 1 for m = 1, 2,...,M It has probability function, P(Y 1 = k 1, Y 2 = k 2,..., Y M = k M ) = = = M m=1 M m=1 k=1 M m=1 p I(Y m=1) m1 p I(Y m=2) m2 p I(Y m=r m ) mr m r m p mkm p I(Y m=k) mk, k m = 1, 2,..., r m ; m = 1, 2,..., M

11 Epi 950, Fall LCM: the mathematical model Y = (Y 1,...,Y M ) : M discrete items measuring latent classes Y m = 1, 2,...,r m L : the variable of latent class membership L = 1, 2,...,C L 3 Y 1 Y 2. Y M

12 Epi 950, Fall If the class membership observed, the joint probability that an individual belongs to class l and provides responses y = (y 1,...,y M ) would be where P(Y = y, L = l) = P(L = l) P(Y = y L = l) MY = P(L = l) P(Y m = y m L = l) = γ l M Y r m Y m=1 k=1 m=1 ρ I(Y m=k) mk l, γ l = P(L = l): the probability of belonging to latent class l ρ mk l = P(Y m = k L = l): the probability of response k to the mth item given a class membership in l

13 Epi 950, Fall Within a latent class l, each manifest item Y m has a multinomial distribution, where the probability of response k to the mth item is ρ mk l for k = 1,...,r m Y 1,...,Y M are independent (conditional independence) The marginal probability of a particular response pattern y = (y 1,...,y M ) without regard for the unseen class membership is P(Y = y) = = CX P(Y = y, L = l) l=1 CX l=1 γ l M Y r m Y m=1 k=1 ρ I(y m=k) mk l.

14 Epi 950, Fall Information about the underlying class structure is conveyed through: the smallest number of classes (C) that adequately describes the associations among the manifest items the response probabilities (ρ-parameters) of manifest items in a specific class the latent class prevalence (γ-parameter)

15 Epi 950, Fall An example Attitudes toward abortion are measured by following six questions: 1. If the woman s own health in seriously endangered by the pregnancy? (woman s health) 2. If there is a strong chance of serious defect in the baby? (birth defect) 3. If she became pregnant as the result of rape? (rape) 4. If the family has a very low income and cannot afford any more children? (too poor) 5. If she is married and does not want any more children? (no more children) 6. If she is not married and does not want to marry the man? (single mom)

16 Epi 950, Fall Latent classes Items I II III woman s health birth defect rape too poor no more children single mom Class prevalence The structure of the attitudes toward abortion is best understood in terms of three categories: 1. those who approve of for both social and ethical/medical reasons 2. those who approve of for ethical/medical reasons but oppose legal abortions for social reasons 3. those who oppose for both social and ethical/medical reasons

17 Epi 950, Fall Fitting a latent class model Maximum likelihood method The most common approach to estimate parameters in an LCM is the ML method using the EM algorithm Upon convergence, standard errors for the estimated parameters are obtained by inverting the Hessian matrix (i.e., the negative second derivative matrix of loglikelihood function) Bayesian method As an alternative to ML, one can apply the Bayesian method via MCMC to estimate parameters in an LCM MCMC may produce greater flexibility in model fit assessment and various hypothesis tests without appealing to large-sample approximation

18 Epi 950, Fall Estimation: ML method EM algorithm Direct maximization of the loglikelihood is complicated ML estimates can be easily calculated if class membership were known Iterating two steps (E-step and M-step) produces a sequence of parameter estimates that converges reliably to a local or global maximum of loglikelihood

19 Epi 950, Fall Expectation step (E-step) We compute the posterior probability of the class membership for each individual y i = (y i1,...,y im ), i = 1,...,n, δ l yi = P(L = l Y = y i ) = P(Y = y i, L = l) P(Y = y i ) = γ l Q M m=1 Q rm P C j=1 γ j Maximization step (M-step) Q M m=1 We update the parameter estimates by γ l = P n i=1 δ l y i n, ρ mk l = k=1 ρ I(y im=k) mk l Q rm k=1 ρ I(y im=k) mk j P n i=1 δ l y i I(y im = k) P n i=1 δ l y i

20 Epi 950, Fall Estimation: Bayesian method MCMC algorithm We re interested in describing an observed-data posterior, posterior Likelihood prior We can simulate (correlated) draws of parameters from posterior distribution via MCMC. The posterior distribution is difficult to portray If the latent class membership for each response pattern were known, the augmented-data posterior would be easy to simulate Iterating two steps (I-step and P-step) produces the stream of parameter values which can be summarized as Bayesian estimates

21 Epi 950, Fall Imputation step (I-step) We compute the posterior probability of the class membership and draw the class to which each individual belongs. Posterior step (P-step) New random values for γ = (γ 1,...,γ C ) are drawn from γ Dirichlet(n 1 +.5,..., n C +.5), where n l is the number of subjects classified into class l. New random values for ρ m l = (ρ m1 l,...,ρ mrm l) are drawn from ρ m l Dirichlet(n m1 l +.5,..., n mrm l +.5), where n mk l is the number of subjects whose response k to item m in class l.

22 Epi 950, Fall Computational issues Problems with ML Well-known computational problems associated with ML: non-identified solutions, maxima on the boundary multiple modes standard errors poor summary of uncertainty LR test cannot be used to determine number of latent classes Problems with Bayesian Bayesian inference by MCMC overcomes many of these problems, but introduces a few more: requires a prior (subjectivity) label switching problem

23 Epi 950, Fall Model selection The choice of number of latent classes should be driven in as a parsimonious fashion as possible by a balanced judgment that takes into the substantive knowledge and objective measures available for assessing model fit The loglikelihood-ratio statistics (LRT) are standard statistics to assess the model fit for LCM: where G 2 = 2 npatt r=1 O r log O r E r χ 2 df, npatt = number of possible response patterns df = npatt 1 number of free parameters

24 Epi 950, Fall Difference in LRT for testing the relative fit of a C-class model against a (C + 1)-class alternative does not have limiting chi-square distributions These measures can be used to assess the relative fit of two competing models with different number of classes (available in Mplus or Latent GOLD): information criteria (e.g., AIC, BIC, CAIC) adjusted LRT posterior probability check distribution (PPCD)

25 Epi 950, Fall Allocation to classes We wish to allocate the individuals to the identified classes using their responses LCM provides a set of allocation probabilities, called the posterior probability, P(L = l y 1, y 2,...,y M ), measuring the probability that individuals with a specific behavioral profile belongs to a specific class If most of the probabilities are close to zero or one, there is little doubt as to the class to which individual should be allocated

26 Epi 950, Fall Posterior probability By Bayes theorem, the posterior probability is δ l y = P(L = l Y = y) = = P(Y = y, L = l) P(Y = y) γ l Q M m=1 Q rm P C j=1 γ j Q M m=1 k=1 ρ I(y m=k) mk l Q rm k=1 ρ I(y m=k) mk j The posterior probability at the final solution provides an objective basis for assigning individuals into the class In contrast, γ l = P(L = l) can be thought of as the probability that a randomly chosen individual belongs to a class l

27 Epi 950, Fall Example Abortion data Response Allocated patterns (y) P(L = 1 y) P(L = 2 y) P(L = 3 y) class

28 Epi 950, Fall Example: Monitoring the Future How have attitudes toward and use of marijuana among high-school seniors changed from 1977 to 2001? Items on marijuana use: 1-3. On how many occasions (if any) have you used marijuana (grass, pot) or hashish (hash, hash oil) in your lifetime? during the last 12 months? during the last 30 days? 4. How likely is it that you will use marijuana in the next 12 months? Items on attitudes toward marijuana use: 5-7. Do YOU disapprove of people (who are 18 or older) doing each of the following? trying marijuana (pot, grass) once or twice smoking marijuana occasionally smoking marijuana regularly

29 Epi 950, Fall Model selection Number of Relative classes 2 loglikelihood G 2 df decrement The number of possible response patterns (cells) is = 3200 Chi-square approximation is not appropriate because estimated expected counts for many cells are close to 0 Differences in G 2 should not be compared with chi-square because LRT for testing the fit of an C-class model against an (C + 1)-class model do not have limiting chi-square distribution

30 Epi 950, Fall Figure 1: Estimated item-response probabilities for Lifetime and TryMJ under the (a) four-class and (b) five-class model (a) Four-class model Lifetime use (1=any use) Try marijuana (1=approve) Year 0.0 Year (b) Five-class model Year 0.0 Year Class 1 Class 2 Class 3 Class 4 Class 1 Class 2 Class 3 Class 4 Class 5

31 Epi 950, Fall Latent classes Items I II III IV Use of marijuana Lifetime months days Next 12 months Attitudes toward marijuana TryMJ OccUse RegUse The relationships among 7 items are well explained by four-class model: 1. Non-users who strongly disapprove of marijuana use 2. Non-users who approve of others doing so on experimental basis 3. Experimental users who disapprove of occasional use 4. Regular users who generally approve of use

32 Epi 950, Fall Figure 2: Historical trends in class prevalence prob Class 1 (disapproving non-users) Class 2 (approving non-users) Class 3 (disapproving experimenters) Class 4 (approving regular users) Year prob Users (Class 3+Class 4) Approvers (Class 2+Class 4) Year

Epi 950, Fall 2006 33 Using R Install and load

33 Epi 950, Fall Using R Install and load a package called polca (released at 06/01/2006)

34 Epi 950, Fall Example: sexual attitudes The data set is extracted from the 1990 British Social Attitudes Survey. It concerns contemporary sexual attitudes. The questions addressed to 1077 individuals were as follows. 1. Should divorce be easier? 2. Do you support the law against sexual discrimination? 3. View on pre-marital sex: (wrong/not wrong) 4. View on extra-marital sex: (wrong/not wrong) 5. View on sexual relationship between individuals of the same sex: (wrong/not wrong) 6. Should gays teach in school? 7. Should gays teach in higher education? 8. Should gays hold public positions? 9. Should a female homosexual couple be allowed to adopt children? 10. Should a male homosexual couple be allowed to adopt children?

35 Epi 950, Fall > descript(attitude, n.print=45) Descriptive statistics for attitude data-set Sample: 10 items and 1077 sample units Proportions for each level of response: [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [1,] [2,] Frequencies of total scores: Freq Pairwise Associations: Item i Item j p.value

36 Epi 950, Fall > ## Data should be integer values in polca > w <- attitude == 0 > attitude[w] <- 2 > > ## Formula for LCA > f <- cbind(v1, V2, V3, V4, V5, V6, V7, V8, V9, V10) 1 > > ## two-class model > lca2 <- polca(f, attitude, nclass=2) > two.cls <- c(lca2$llik, lca2$gsq, lca2$aic, lca2$bic) > > ## three-class model > lca3 <- polca(f, attitude, nclass=3) > three.cls <- c(lca3$llik, lca3$gsq, lca3$aic, lca3$bic) > > ## four-class model > lca4 <- polca(f, attitude, nclass=4) > four.cls <- c(lca4$llik, lca4$gsq, lca4$aic, lca4$bic)

37 Epi 950, Fall > ## five-class model > lca5 <- polca(f, attitude, nclass=5) > five.cls <- c(lca5$llik, lca5$gsq, lca5$aic, lca5$bic) > > ## six-class model > lca6 <- polca(f, attitude, nclass=6) > six.cls <- c(lca6$llik, lca6$gsq, lca6$aic, lca6$bic) > > ## seven-class model > lca7 <- polca(f, attitude, nclass=7) > seven.cls <- c(lca7$llik, lca7$gsq, lca7$aic, lca7$bic)

38 Epi 950, Fall > ## Combine model selection criteria > model.sel <- rbind(two.cls, three.cls, four.cls, five.cls, six.cls, seven.cls) > dimnames(model.sel) <- list(c("two", "three", "four", "five", "six", "seven"), c("llik", "G2", "AIC", "BIC")) > model.sel llik G2 AIC BIC two.cls three.cls four.cls five.cls six.cls seven.cls > > ## Class prevalence > round(lca4$p, 3) [1] > round(lca5$p, 3) [1]

39 Epi 950, Fall > ## Item response probabilities > lca4$probs > lca5$probs 1 Probs. of positive response Class 1 Class 2 Class 3 Class Items 1 Probs. of positive response Class 1 Class 2 Class 3 Class 4 Class Items

40 Epi 950, Fall > ## Posterior probabilities and allocated class for the first 15 individuals > cbind( round(lca4$posterior[1:15,], 3), lca4$predclass[1:15] ) [,1] [,2] [,3] [,4] [,5] [1,] [2,] [3,] [4,] [5,] [6,] [7,] [8,] [9,] [10,] [11,] [12,] [13,] [14,] [15,]

41 Epi 950, Fall > ## Numbers of individuals assigned to Class 1 > w <- lca5$predclass == 1 > sum(w) [1] 136 > ## Posterior probabilities for first 15 individuals in Class 1 > round(lca4$posterior[w,], 3)[1:15,] [,1] [,2] [,3] [,4] [1,] [2,] [3,] [4,] [5,] [6,] [7,] [8,] [9,] [10,] [11,] [12,] [13,] [14,] [15,] > round(apply(lca4$posterior[w,], 2, mean), 3) [1]

42 Epi 950, Fall > ## If you do it for Classes 2, 3, and 4, you will get > round(m.post.4, 3) cls.1 cls.2 cls.3 cls.4 cls cls cls cls > ## With a 5-class model, > round(m.post.5, 3) cls.1 cls.2 cls.3 cls.4 cls.5 cls cls cls cls cls

43 Epi 950, Fall > ## Observed frequencies > Obs <- lca4$predcell[,"observed"] > ## Expected frequencies > Exp.4 <- lca4$predcell[,"expected"] > Exp.5 <- lca5$predcell[,"expected"] > ## Standardized residuals > resid.4 <- (Obs-Exp.4) / sqrt(exp.4) > resid.5 <- (Obs-Exp.5) / sqrt(exp.5) > ## Combine results > fit <- cbind(obs, Exp.4, resid.4, Exp.5, resid.5) > tmp <- sort(obs, index.return=t, decreasing=t) > fit <- fit[tmp$ix,]

44 Epi 950, Fall > fit[1:20,] Obs Exp.4 resid.4 Exp.5 resid.5 [1,] [2,] [3,] [4,] [5,] [6,] [7,] [8,] [9,] [10,] [11,] [12,] [13,] [14,] [15,] [16,] [17,] [18,] [19,] [20,]

45 Epi 950, Fall > ## Corresponding response patterns > lca4$predcell[tmp$ix[1:20],1:10] V1 V2 V3 V4 V5 V6 V7 V8 V9 V

What is Latent Class Analysis. Tarani Chandola

What is Latent Class Analysis. Tarani Chandola What is Latent Class Analysis Tarani Chandola methods@manchester Many names similar methods (Finite) Mixture Modeling Latent Class Analysis Latent Profile Analysis Latent class analysis (LCA) LCA is a