Package MixGLasso. August 31, 2013

Size: px

Start display at page:

Download "Package MixGLasso. August 31, 2013"

Kelley Crawford
6 years ago
Views:

1 Package MixGLasso August 31, 2013 Type Package Title The MixGLasso-package is a powerful tool for clustering high-dimensional data. Version 1.0 Date Author Nicolas Staedler Maintainer Nicolas Staedler <staedler.n@gmail.com> The MixGLasso-package is a powerful tool for clustering high-dimensional data. It is based on a Gaussian mixture model. Each component is characterized by a mean vector an a covariance matrix. The inverse covariance matrices are sparse and represent a Gaussian graphical models (GGM). Estimation is performed using a novel penalized EM algorithm. Several strategies to select an optimal number of mixture components are implemented. Imports glasso, mclust, multicore, mvtnorm License GPL-2 Collate hmmgl_mixgl backup-mixglasso-only.r R topics documented: MixGLasso-package bwprun_mixglasso EXPStep.mix func.uinit glasso.invcor glasso.invcov glasso.parcor loglik_mix mixglasso mixglasso_init mixglasso_path MStepGlasso simmix

2 2 bwprun_mixglasso sparse_conc sumoffdiag symmkldist tr w.kldist Index 18 MixGLasso-package MixGLasso-package The MixGLasso-package is a powerful tool for clustering high-dimensional data. It is based on a Gaussian mixture model. Each component is characterized by a mean vector an a covariance matrix. The inverse covariance matrices are sparse and represent a Gaussian graphical models (GGM). Estimation is performed using a novel penalized EM algorithm. Several strategies to select an optimal number of mixture components are implemented. Details MixGLasso provides an optimal number of mixture components; optimal cluster assignments; Cluster-specific networks (sparse inverse covariance matrices); Cluster-specific means. References St\"adler, N. and Mukherjee, S. (2012). Penalized estimation in high-dimensional hidden Markov models with comp-specific graphical models. To appear in the Annals of Applied Statistics. http: //arxiv.org/abs/ bwprun_mixglasso bwprun_mixglasso Mixglasso with backward pruning bwprun_mixglasso(x, n.comp.min = 1, n.comp.max, lambda = sqrt(2 * nrow(x) * log(ncol(x)))/2, pen = "glasso.parcor", selection.crit = "mmdl", term = 10^{ -3 }, min.compsize = 5, init = "kmeans.hc", my.cl = NULL, modelname.hc = "VVV", nstart.kmeans = 1, iter.max.kmeans = 10, reinit.out = FALSE, reinit.in = FALSE, mer = TRUE, del = TRUE,...)

3 bwprun_mixglasso 3 x Input data matrix n.comp.min Minimum number of components. Take n.comp.min=1! n.comp.max lambda pen Maximum number of components Regularization parameter. Default=sqrt(2*n*log(p))/2 Determines form of penalty: glasso.parcor (default), glasso.invcov, glasso.invcor selection.crit Selection criterion. Default= mmdl term min.compsize init my.cl modelname.hc Termination criterion of EM algorithm. Default=10^-3 Stop EM if any(compsize)<min.compsize; Default=5 Initialization. Method used for initialization init= cl.init, r.means, random, kmeans, kmeans.hc, h Default= kmeans.hc Initial cluster assignments; need to be provided if init= cl.init (otherwise this param is ignored). Default=NULL Model class used in hc. Default="VVV" nstart.kmeans Number of random starts in kmeans; default=1 iter.max.kmeans Maximal number of iteration in kmeans; default=10 reinit.out Re-initialization if compsize<min.compsize (at the start of algorithm)? reinit.in mer del Details Re-initialization if compsize<min.compsize (at the bwprun-loop level of algorithm)? Merge closest comps for initialization Delete smallest comp for initialization... Other arguments. See mixglasso_init This function runs mixglasso with various number of mixture components: It starts with a too large number of components and iterates towards solutions with smaller number of components by initializing using previous solutions. list consisting of selcrit res.init comp.name Selcrit for all models with number of components between n.comp.min and n.comp.max Initialization for all components List of names of components. Indicates which states where merged/deleted during backward pruning re.init.in Logical vector indicating whether re-initialization was performed or not fit.mixgl.selcrit Results for model with optimal number of components. List see mixglasso_init

4 4 EXPStep.mix Examples ########################################### ##This an example of how to use MixGLasso## ########################################### rm(list=ls()) ##generate data n < K <- 3 p <- 10 mix.prob <- rep(1/k,k) SigInv <- sparse_conc(p,k,s=p,s.common=0,scale.parcor=true) Sig <- array(na,dim=c(p,p,k)) for (k in 1:K){ Sig[,,k] <- solve(siginv[[k]]) } Mu <- matrix(0,p,k) nonzero.mean <- split(sample(1:p),rep(1:k,length=p)) for(k in 1:K){ Mu[nonzero.mean[[k]],k] <- -2/sqrt(ceiling(p/K)) } dat <- simmix(n,k,mix.prob,mu,sig) ##run mixglasso fit1 <- mixglasso_path(dat$x,n.comp=1:6,save.allfits=false,show.trace=true) fit1$bic# fit2 <- mixglasso_path(dat$x,n.comp=6,save.allfits=false,show.trace=true) fit2$bic# fit3 <- mixglasso_path(dat$x,n.comp=1:6,lambda=0,save.allfits=false,show.trace=true) fit4 <- mixglasso_path(dat$x,n.comp=1:6,lambda=inf,save.allfits=false,show.trace=true) fit5 <- bwprun_mixglasso(dat$x,n.comp=1,n.comp.max=6,selection.crit= bic ) ##compare bic plot(fit1$bic,type= l,lty=1,ylim=range(c(fit1$bic,fit3$bic,fit4$bic)), ylab= bic,xlab= no components ) lines(fit3$bic,lty=2) lines(fit4$bic,lty=3) EXPStep.mix Performs EStep Performs EStep

5 func.uinit 5 EXPStep.mix(logphi, mix.prob) logphi mix.prob list consiting of u LL responsibilities loglikelihood func.uinit Initialization of MixGLasso Initialization of responsibilities func.uinit(x, n.comp, init = "kmeans", my.cl = NULL, nstart.kmeans = 1, iter.max.kmeans = 10, modelname.hc = "EII") x n.comp init my.cl Observed data Number of mixture components Method used for initialization init= cl.init, r.means, random, kmeans, kmeans.hc, hc Initial cluster assignments; need to be provided if init= cl.init (otherwise this param is ignored) Number of random starts in kmeans; default=1 nstart.kmeans iter.max.kmeans Maximal number of iteration in kmeans; default=10 modelname.hc Model class used in hc; default= EII a list consisting of u responsibilities

6 6 glasso.invcov glasso.invcor Graphical Lasso based on inverse covariance penalty Graphical Lasso based on inverse covariance penalty glasso.invcor(s, rho, penalize.diagonal, term = 10^{ -3 }) s rho penalize.diagonal term w; wi; iter glasso.invcov Graphical Lasso based on inverse correlation penalty Graphical Lasso based on inverse correlation penalty glasso.invcov(s, rho, penalize.diagonal, term = 10^{ -3 }) s rho penalize.diagonal term

7 glasso.parcor 7 w; wi; iter glasso.parcor Graphical Lasso based on partial correlation penalty Graphical Lasso based on partial correlation penalty glasso.parcor(s, rho, penalize.diagonal, maxiter = 1000, term = 10^{ -3 }) s rho penalize.diagonal maxiter term w; wi; iter loglik_mix Log-likelihood for mixture model Log-likelihood for mixture model loglik_mix(x, mix.prob, Mu, Sig)

8 8 mixglasso x mix.prob Mu Sig log-likelihood mixglasso mixglasso mixglasso x mixglasso(x, n.comp, lambda = sqrt(2 * nrow(x) * log(ncol(x)))/2, pen = "glasso.parcor", init = "kmeans.hc", my.cl = NULL, modelname.hc = "VVV", nstart.kmeans = 1, iter.max.kmeans = 10, term = 10^{ -3 }, min.compsize = 5,...) n.comp lambda pen init my.cl modelname.hc Input data matrix Number of mixture components Regularization parameter. Default=sqrt(2*n*log(p))/2 Determines form of penalty: glasso.parcor (default), glasso.invcov, glasso.invcor Initialization. Method used for initialization init= cl.init, r.means, random, kmeans, kmeans.hc, h Default= kmeans Initial cluster assignments; need to be provided if init= cl.init (otherwise this param is ignored). Default=NULL Model class used in hc. Default="VVV" nstart.kmeans Number of random starts in kmeans; default=1 iter.max.kmeans Maximal number of iteration in kmeans; default=10 term min.compsize Termination criterion of EM algorithm. Default=10^-3 Stop EM if any(compsize)<min.compsize; Default=5... Other arguments. See mixglasso_init

9 mixglasso_init 9 Details This function runs mixglasso see return mixglasso_init. list consisting of mix.prob Mu Sig SigInv iter loglik bic mmdl u comp compsize pi.comps warn -loglik+log(n)*df/2 -loglik+penmmdl/2 responsibilities component assignments size of components warnings during optimization mixglasso_init mixglasso_init mixglasso_init (initialization and lambda set by user) mixglasso_init(x, n.comp, lambda, u.init, mix.prob.init, gamma = 0.5, pen = "glasso.parcor", penalize.diagonal = FALSE, term = 10^{ -3 }, miniter = 5, maxiter = 1000, min.compsize = 5, show.trace = FALSE)

10 10 mixglasso_init x n.comp lambda u.init mix.prob.init gamma Input data matrix Number of mixture components Regularization parameter Initial responsibilities Initial component probablities Determines form of penalty pen Determines form of penalty: glasso.parcor (default), glasso.invcov, glasso.invcor penalize.diagonal Should the diagonal of the inverse covariance matrix be penalized? Default=FALSE (recommended) term miniter maxiter min.compsize show.trace Termination criterion of EM algorithm. Default=10^-3 Minimal number of EM iteration before stop EM if any(compsize)<min.compsize applies. Default=5 Maximal number of EM iteration. Default=1000 Stop EM if any(compsize)<min.compsize; Default=5 Should information during execution be printed? Default=FALSE Details This function runs mixglasso; requires initialization (u.init,mix.prob.init) list consisting of mix.prob Mu Sig SigInv iter loglik bic mmdl u comp compsize pi.comps warn Component probabilities Component specific mean vectors Component specific covariance matrices Component specific inverse covariance matrices Number of EM iterations Log-likelihood -loglik+log(n)*df/2 -loglik+penmmdl/2 Component responsibilities Component assignments Size of components Component probabilities Warnings during EM algorithm

11 mixglasso_path 11 mixglasso_path mixglasso_path mixglasso_path mixglasso_path(x, n.comp, lambda = sqrt(2 * nrow(x) * log(ncol(x)))/2, pen = "glasso.parcor", init = "kmeans.hc", my.cl = NULL, modelname.hc = "VVV", nstart.kmeans = 1, iter.max.kmeans = 10, term = 10^{ -3 }, min.compsize = 5, save.allfits = TRUE, filename = NULL, mc.set.seed = FALSE, mc.preschedule = FALSE,...) x n.comp lambda pen init my.cl modelname.hc Input data matrix Number of mixture components Regularization parameter. Default=sqrt(2*n*log(p))/2 Determines form of penalty: glasso.parcor (default), glasso.invcov, glasso.invcor Initialization. Method used for initialization init= cl.init, r.means, random, kmeans, kmeans.hc, h Default= kmeans Initial cluster assignments; need to be provided if init= cl.init (otherwise this param is ignored). Default=NULL Model class used in hc. Default="VVV" nstart.kmeans Number of random starts in kmeans; default=1 iter.max.kmeans Maximal number of iteration in kmeans; default=10 term min.compsize Termination criterion of EM algorithm. Default=10^-3 Stop EM if any(compsize)<min.compsize; Default=5 save.allfits Save output of mixglasso for all k s? filename mc.set.seed Output of mixglasso with filename_fit.mixgl_k.rda See mclapply. Default=FALSE mc.preschedule See mclapply. Default=FALSE... Other arguments. See mixglasso_init Details Runs mixglasso (in parallel) with various number of mixture components

12 12 mixglasso_path list consisting of bic comp iter warn Bic for all fits Components assignments for all fits Number of iteration of EM for all fits Warning infos for all fits Examples ########################################### ##This an example of how to use MixGLasso## ########################################### rm(list=ls()) ##generate data n < K <- 3 p <- 10 mix.prob <- rep(1/k,k) SigInv <- sparse_conc(p,k,s=p,s.common=0,scale.parcor=true) Sig <- array(na,dim=c(p,p,k)) for (k in 1:K){ Sig[,,k] <- solve(siginv[[k]]) } Mu <- matrix(0,p,k) nonzero.mean <- split(sample(1:p),rep(1:k,length=p)) for(k in 1:K){ Mu[nonzero.mean[[k]],k] <- -2/sqrt(ceiling(p/K)) } dat <- simmix(n,k,mix.prob,mu,sig) ##run mixglasso fit1 <- mixglasso_path(dat$x,n.comp=1:6,save.allfits=false,show.trace=true) fit1$bic# fit2 <- mixglasso_path(dat$x,n.comp=6,save.allfits=false,show.trace=true) fit2$bic# fit3 <- mixglasso_path(dat$x,n.comp=1:6,lambda=0,save.allfits=false,show.trace=true) fit4 <- mixglasso_path(dat$x,n.comp=1:6,lambda=inf,save.allfits=false,show.trace=true) fit5 <- bwprun_mixglasso(dat$x,n.comp=1,n.comp.max=6,selection.crit= bic ) ##compare bic plot(fit1$bic,type= l,lty=1,ylim=range(c(fit1$bic,fit3$bic,fit4$bic)),

13 MStepGlasso 13 ylab= bic,xlab= no components ) lines(fit3$bic,lty=2) lines(fit4$bic,lty=3) MStepGlasso MStep of MixGLasso MStep of MixGLasso MStepGlasso(x, chromosome = NULL, u, v = NULL, lambda, gamma, pen, penalize.diagonal, equal.prob.trans = NULL, term, model = "hmm") x chromosome u v lambda gamma pen penalize.diagonal equal.prob.trans term model list consisting of mix.prob, Mu, Sig, SigInv

14 14 sparse_conc simmix Simulate from mixture model with MVN components Simulate from mixture model with MVN components simmix(n, n.comp, mix.prob, Mu, Sig) n n.comp mix.prob Mu Sig sample size number of mixture components ("comps") mixing probablities matrix of component-specific mean vectors array of component-specific covariance matrices a list consisting of S X component assignments observed data matrix sparse_conc Generates sparse inverse covariance matrices Generates sparse inverse covariance matrices sparse_conc(p, K, s, s.common, magn.nz = 0.5, scale.parcor = TRUE) p Dimensionality of inverse covariance matrix K Number of inverse covariance matrices s Number of non-zero entries per inverse covariance matrix s.common Number of non-zero entries shared across different inverse covariance matrices magn.nz Magnitude of non-zero elements scale.parcor Should SigInv be scaled to have diagonal equal one, siginv=parcor?

15 sumoffdiag 15 SigInv: list of inverse covariance matrices sumoffdiag Sum of non-diag elements of a matrix Sum of non-diag elements of a matrix sumoffdiag(m) m Sum of non-diag elements symmkldist Compute symmetric kull-back leibler distance Compute symmetric kull-back leibler distance symmkldist(mu1, mu2, sig1, sig2) mu1 mu2 sig1 sig2

16 16 w.kldist symmetric kull-back leibler distance tr Compute trace of matrix Compute trace of matrix tr(m) m trace of matrix w.kldist Distance between comps based on symm. kl-distance Distance between comps based on symm. kl-distance w.kldist(mu, Sig) Mu Sig

17 w.kldist 17 list consisting of comp.kldist min.comp.kldist

18 Index bwprun_mixglasso, 2 EXPStep.mix, 4 func.uinit, 5 glasso.invcor, 6 glasso.invcov, 6 glasso.parcor, 7 loglik_mix, 7 mixglasso, 8 MixGLasso-package, 2 mixglasso_init, 9 mixglasso_path, 11 MStepGlasso, 13 simmix, 14 sparse_conc, 14 sumoffdiag, 15 symmkldist, 15 tr, 16 w.kldist, 16 18

Package scio. February 20, 2015

Package scio. February 20, 2015 Title Sparse Column-wise Inverse Operator Version 0.6.1 Author Xi (Rossi) LUO and Weidong Liu Package scio February 20, 2015 Maintainer Xi (Rossi) LUO Suggests clime, lorec, QUIC