Package breakaway. R topics documented: March 30, 2016

Transcription

1 Title Species Richness Estimation and Modeling Version 3.0 Date Author and John Bunge Maintainer Package breakaway March 30, 2016 Species richness estimation is an important problem in biodiversity analysis. This package provides methods for total species richness estimation (observed plus unobserved) and a method for modelling total diversity with covariates. breakaway() estimates total (observed plus unobserved) species richness. Microbial diversity datasets are characterized by a large number of rare species and a small number of highly abundant species. The class of models implemented by breakaway() is flexible enough to model both these features. breakaway_nof1() implements a similar procedure however does not require a singleton count. betta() provides a method for modelling total diversity with covariates in a way that accounts for its estimated nature and thus accounts for unobserved taxa, and betta_random() permits random effects modelling. License GPL-2 NeedsCompilation no Repository CRAN Date/Publication :14:27 R topics documented: breakaway-package apples betta betta_pic betta_random breakaway breakaway_nof chao chao1_bc chao_bunge hawaii

2 2 breakaway-package wlrm_transformed wlrm_untransformed Index 19 breakaway-package Species richness estimation and modelling in the high-diversity setting Details Species richness estimation is an important problem in biodiversity analysis. This package provides methods for total species richness estimation (observed plus unobserved) and a method for modelling total diversity with covariates. breakaway estimates total (observed plus unobserved) species richness. Microbial diversity datasets are characterized by a large number of rare species and a small number of highly abundant species. The class of models implemented by breakaway is flexible enough to model both these features. breakaway_nof1 implements a similar procedure however does not require a singleton count. betta provides a method for modelling total diversity with covariates in a way that accounts for its estimated nature and thus accounts for unobserved taxa, and betta_random permits random effects modelling. Package: breakaway Type: Package Version: 3.0 Date: License: GPL-2 The function breakaway estimates the total (observed plus unobserved) number of classes (usually, distinct species) based on a sample of the frequency counts. Standard errors and model fits are also given. The algorithm is based on theory of characterization of distributions by ratios of their probabilities. Parameter estimation is done via nonlinear regression. The class of models available is usually broad enough to account for the high-diversity case, which is often observed in microbial diversity datasets. Since many classical estimation procedures either fail to provide an estimate or provide poor fits in the microbial setting, breakaway addresses this data structure. Additionally, since sequencing errors may result in an inflated singleton count, breakaway_nof1 performs a similar procedure but does not require a singleton count. It can be used as an exploratory tool for investigating the plausibility of the given singleton count. betta runs a regression-type analysis of estimated total diversity, thus permitting accounting for unobserved taxa. It does not enforce use of breakaway for diversity estimation. A mixed-model approach accounts for the differing levels of confidence in the diversity estimates, and covariates constitute the fixed effects. Support of this work from Cornell University s Department of Statistical Sciences is gratefully acknowledged. & John Bunge

3 apples 3 Maintainer: <adw96@cornell.edu> Willis, A. and Bunge, J. (2015). Estimating diversity via frequency ratios. Biometrics. Willis, A. (2015). Species richness estimation with high diversity but spurious singletons. Under review. Willis, A., Bunge, J., and Whitman, T. (2015). Inference for changes in biodiversity. arxiv preprint. Rocchetti, I., Bunge, J. and Bohning, D. (2011). Population size estimation based upon ratios of recapture probabilities. Annals of Applied Statistics, 5. Chao, A. and Bunge, J. (2002). Estimating the number of species in a stochastic abundance model. Biometrics, 58. Chao, A. (1984). Nonparametric estimation of the number of classes in a population. Scandinavian Journal of Statistics, 4. breakaway(apples) breakaway(apples,plot=false,print=false,answers=true) breakaway_nof1(apples[-1,]) betta(c(1552,1500,884),c(305,675,205)) apples apples dataset Source Frequency count table for a microbial community sampled in the soil of an apple orchard in Switzerland. apples Walsh et. al. (2014). Restricted streptomycin use in apple orchards did not adversely alter the soil bacteria communities. Frontiers in Microbiology, 4. breakaway breakaway(apples)

4 4 betta betta modelling total diversity This function tests for heterogeneity of total diversity (observed plus unobserved) across multiple sites. It can account or test for fixed effects that may explain diversity. It returns the significance of the covariates in explaining diversity and a hypothesis test for heterogeneity. betta(chats, ses, X = NA) chats ses X A vector of estimates of total diversity at different sampling locations. breakaway estimates are suggested in the high-diversity case but not enforced. The standard errors in chats, the diversity estimates. A numeric matrix of covariates. If not supplied, an intercept-only model will be fit. Value table cov ssq_u homogeneity global blups blupses A coefficient table for the model parameters. The columns give the parameter estimates, standard errors, and p-values, respectively. This model is only as effective as your diversity estimation procedure; for this reason please confirm that your estimates are appropriate and that your model is not misspecified. betta_pic may be useful for this purpose. Estimated covariance matrix of the parameter estimates. The estimate of the heterogeneity variance. The test statistic and p-value for the test of homogeneity. The test statistic and p-value for the test of model explanatory power. The conditional expected values of the diversity estimates (conditional on the random effects). The authors propose that if the practitioner believes that information from one diversity estimator may inform the others, then using the condfits as estimators of total diversity rather than Chats may reduce variance of diversity estimates by sharing strength across the samples. The estimated standard deviation (standard errors) in the blups. Note Ecologists who are interested in the way species richness varies with covariate information often run a regression-type analysis on the observed diversity using their covariate information as predictors. However, in many settings (especially microbial), rare and unobserved taxa play a hugely important role in explaining the subtleties of the ecosystem, however, a regression analysis on the observed

5 betta_pic 5 diversity level fails to account for these unobserved taxa. By predicting the total level of diversity (for example, via breakaway) and estimating the standard error in the estimate, one can take account of these unobserved, but important, taxa. In order to account for the estimated nature of the response, a mixed model approach is taken, whereby the varying levels of confidence in the estimates contributes to a diagonal but heteroscedastic covariance matrix. Given covariates constitute the fixed effects in the mixed model, and significance of the random effect term sigsq_u reflects heterogeneity in the sample, that is, variability that cannot be explained by only the covariates. The authors believe this to be the first attempt at modelling total diversity in a way that accounts for its estimated nature. Willis, A., Bunge, J., and Whitman, T. (2015). Inference for changes in biodiversity. arxiv preprint. Willis, A. and Bunge, J. (2015). Estimating diversity via frequency ratios. Biometrics. breakaway; breakaway_nof1; betta_pic; apples betta(c(2000, 3000, 4000, 3000), c(100, 200, 150, 180), cbind(1, c(100, 150, 100, 50))) ## handles missing data betta(c(2000, 3000, 4000, 3000), c(100, 200, 150, NA)) ## A test for heterogeneity of apples diversity estimates vs butterfly estimates betta(c(1552,1500,884),c(305,675,205), cbind(1,c(0,0,1))) betta_pic function for plotting total diversity A simple plotting interface for comparing total diversity across samples or a covariate gradient. betta_pic(y, se, x=1:length(y), ylimu=na, myy=na, mymain=na, mycol=rep("black", length(y)), labs=na, mypch=rep(16, length(y)), myxlim=c(0.8*min(x, na.rm=t), 1.2*max(x, na.rm=t)))

6 6 betta_random y se x ylimu myy mymain mycol labs mypch myxlim A vector of estimates of total diversity. Other parameter estimates are accessible; this method may be used for plotting any parameter estimates.. The standard errors in y, the diversity (or other parameter s) estimates. A vector of covariates to form the x-coordinates of the intervals. If no argument is given, defaults to the order. The upper endpoint of the y-axis. A label for the y-axis. Default is none. A main title for the plot. Default is none. Colors for the plotting points. Default is black. x-axis labels. Default is none (non x-axis plotted). Plotting characters for the estimates. Defaults to circles. A vector of x-axis limits. Default is 0 to 10% greater than the maximum x value. Willis, A., Bunge, J., and Whitman, T. (2015). Inference for changes in biodiversity. arxiv preprint. betta betta_pic(c(1552,1500,884),c(305,675,205),mymain="example title") betta_random modelling total diversity with random effects This function extends betta() to permit random effects modelling. betta_random(chats, ses, X = NA, groups)

7 betta_random 7 chats ses X groups A vector of estimates of total diversity at different sampling locations. The standard errors in chats, the diversity estimates. A numeric matrix of covariates corresponding to fixed effects. If not supplied, an intercept-only model will be fit. A categorical variable representing the random-effects groups that each of the estimates belong to. Value table cov ssq_u ssq_g homogeneity global blups A coefficient table for the model parameters. The columns give the parameter estimates, standard errors, and p-values, respectively. This model is only as effective as your diversity estimation procedure; for this reason please confirm that your estimates are appropriate and that your model is not misspecified. betta_pic may be useful for this purpose. Estimated covariance matrix of the parameter estimates. The estimate of the heterogeneity variance. Estimates of within-group variance. The estimate will be zero for groups with only one observation. The test statistic and p-value for the test of homogeneity. The test statistic and p-value for the test of model explanatory power. The conditional expected values of the diversity estimates (conditional on the random effects). Estimates of variability for the random effects case are unavailable at this time; please contact the maintainer if needed. Willis, A., Bunge, J., and Whitman, T. (2015). Inference for changes in biodiversity. arxiv preprint. betta; betta_pic betta_random(c(2000, 3000, 4000, 3000), c(100, 200, 150, 180), X = cbind("int"=1, "Cont_var"=c(100, 150, 100, 50)), groups = c("a", "a", "a", "b")) ## handles missing data betta_random(c(2000, 3000, 4000, 3000), c(100, 200, 150, NA), groups= c("a", NA, "b", "b"))

8 8 breakaway breakaway function for species richness estimation This function implements the species richness estimation procedure outlined in Willis & Bunge (2015). The diversity estimate, standard error, estimated model coefficients, model details and plot of the fitted model are returned. breakaway(data, print = TRUE, plot = TRUE, answers = FALSE, force = FALSE) Value data print plot The sample frequency count table for the population of interest. The first row must correspond to the singletons. Acceptable formats include a matrix, data frame, or file path (csv or txt). The standard frequency count table format is used: two columns, the first of which contains the frequency of interest (eg. 1 for singletons, species observed once, 2 for doubletons, species observed twice, etc.) and the second of which contains the number of species observed this many times. Frequencies (first column) should be ordered least to greatest. At least 6 contiguous frequencies are necessary. Do not concatenate large frequencies. See dataset apples for sample formatting. Logical: whether the results should be printed to screen. If FALSE, answers should be set to TRUE so that results will be returned. Logical: whether the data and model fit should be plotted. answers Logical: whether the function should return an argument. If FALSE, print should be set to TRUE. force Logical: force breakaway to run in the presence of frequency count concatenation. breakaway checks that the user has not concatenated multiple upper frequencies. force=true will force breakaway to fit models in the presence of this. breakaway s diversity estimates cannot be considered reliable in this case. code A category representing algorithm behaviour. code=1 indicates no nonlinear models converged and the transformed WLRM diversity estimate of Rocchetti et. al. (2011) is returned. code=2 indicates that the iteratively reweighted model converged and was returned. code=3 indicates that iterative reweighting did not converge but a model based on a simplified variance structure was returned (in this case, the variance of the frequency ratios is assumed to be proportional to the denominator frequency index). Please peruse your fitted model before using your diversity estimate.

9 breakaway 9 name para est seest full ci The name of the selected model. The first integer represents the numerator polynomial degree and the second integer represents the denominator polynomial degree of the model for the frequency ratios. See Willis & Bunge (2015) for details. Estimated model parameters and standard errors. The estimate of total (observed plus unobserved) diversity. The standard error in the diversity estimate. The chosen nonlinear model for frequency ratios. An asymmetric 95% confidence interval for diversity. Note breakaway presents an estimator of species richness that is well-suited to the high-diversity/microbial setting. However, many microbial datasets display more diversity than the Kemp-type models can permit. In this case, the log-transformed WLRM diversity estimator of Rocchetti et. al. (2011) is returned. The authors experience suggests that some datasets that require the log-transformed WLRM contain false diversity, that is, diversity attributable to sequencing errors (via an inflated singleton count). The authors encourage judicious use of diversity estimators when the dataset may contain these errors, and recommend the use of breakaway_nof1 as an exploratory tool in this case. Willis, A. and Bunge, J. (2015). Estimating diversity via frequency ratios. Biometrics. Rocchetti, I., Bunge, J. and Bohning, D. (2011). Population size estimation based upon ratios of recapture probabilities. Annals of Applied Statistics, 5. breakaway_nof1; apples breakaway(apples) breakaway(apples,plot=false,print=false,answers=true)

10 10 breakaway_nof1 breakaway_nof1 species richness estimation without singletons This function permits estimation of total diversity based on a sample frequency count table. Unlike breakaway, it does not require an input for the number of species observed once, making it an excellent exploratory tool for microbial ecologists who believe that their sample may contain spurious singletons. The underlying estimation procedure is similar to that of breakaway and is outlined in Willis & Bunge (2014). The diversity estimate, standard error, estimated model coefficients and plot of the fitted model are returned. breakaway_nof1(data, print = TRUE, plot = TRUE, answers = FALSE, force = FALSE) data print plot answers force The sample frequency count table for the population of interest. The first row must correspond to the doubletons. Acceptable formats include a matrix, data frame, or file path (csv or txt). The standard frequency count table format is used: two columns, the first of which contains the frequency of interest (eg. 1 for singletons, species observed once; 2 for doubletons, species observed twice, etc.) and the second of which contains the number of species observed this many times. Frequencies (first column) should be ordered least to greatest. At least 6 contiguous frequencies are necessary. Do not concatenate large frequencies. See dataset apples for sample formatting. Logical: whether the results should be printed to screen. If FALSE, answers should be set to TRUE so that results will be returned. Logical: whether the data and model fit should be plotted. Logical: whether the function should return an argument. If FALSE, print should be set to TRUE. Logical: force the procedure to run in the presence of frequency count concatenation. A basic check procedure confirms that the user has not appeared to concatenate multiple upper frequencies. force=true will force the procedure to fit models in the presence of this. breakaway_nof1 s diversity estimates cannot be considered reliable in this case. Value code A category representing algorithm behaviour. code=1 indicates no nonlinear models converged and the transformed WLRM diversity estimate of Rocchetti et. al. (2011) is returned. code=2 indicates that the iteratively reweighted model converged and was returned. code=3 indicates that iterative reweighting did not converge but a model based on a simplified variance structure was

11 breakaway_nof1 11 name para est seest full returned (in this case, the variance of the frequency ratios is assumed to be proportional to the denominator frequency index). Please peruse your fitted model before using your diversity estimate. The name of the selected model. The first integer represents the numerator polynomial degree and the second integer represents the denominator polynomial degree. See Willis & Bunge (2014) for details. Estimated model parameters and standard errors. The estimate of total (observed plus unobserved) diversity. The standard error in the diversity estimate. The chosen nonlinear model for frequency ratios. Note It is common for microbial ecologists to believe that their dataset contains false diversity. This often arises because sequencing errors result in classifying already observed organisms as new organisms. breakaway_nof1 was developed as an exploratory tool in this case. Practitioners can run breakaway on their dataset including the singletons, and breakaway_nof1 on their dataset excluding the singletons, and assess if the estimated levels of diversity are very different. Great disparity may provide evidence of an inflated singleton count, or at the very least, that breakaway is especially sensitive to the number of rare species observed. Note that breakaway_nof1 may be less stable than breakaway due to predicting based on a reduced dataset, and have greater standard errors. Willis, A. (2015). Species richness estimation with high diversity but spurious singletons. Under review. Willis, A. and Bunge, J. (2015). Estimating diversity via frequency ratios. Biometrics. breakaway; apples breakaway_nof1(apples[-1,]) breakaway_nof1(apples[-1,],plot=false,print=false,answers=true)

12 12 chao1 chao1 species richness lower bound This function implements the Chao1 diversity index. chao1(data, print=true, answers=false) data print answers The sample frequency count table for the population of interest. See breakaway for details. Logical: whether the results should be printed to screen. Logical: whether the function should return an argument. Value name est seest ci The name of the model. The estimate of total (observed plus unobserved) diversity. The standard error in the diversity estimate. An asymmetric 95% confidence interval for diversity. Note This estimator is based on the assumption of equal species detection probabilities, that is, every group in the population is equally likely to be observed. This assumption never applies, thus this should never be used for species richness estimation. However, it provides a reliable lower bound for species richness, and has historical significance. breakaway, breakaway_nof1 and chao_bunge provide viable alternatives for species richness estimation and are included in this package. Chao, A. (1984). Nonparametric estimation of the number of classes in a population. Scandinavian Journal of Statistics, 4. breakaway; apples; wlrm_untransformed; chao1_bc; chao_bunge

13 chao1_bc 13 chao1(apples) chao1_bc species richness estimator under equal detection probabilities This function implements the bias-corrected Chao1 species richness estimator. chao1_bc(data, print=true, answers=false) data print answers The sample frequency count table for the population of interest. See breakaway for details. Logical: whether the results should be printed to screen. Logical: whether the function should return an argument. Value name est seest ci The name of the model. The estimate of total (observed plus unobserved) diversity. The standard error in the diversity estimate. An asymmetric 95% confidence interval for diversity. Note This estimator is based on the assumption of equal species detection probabilities, that is, every group in the population is equally likely to be observed. This assumption never applies, thus this estimator should never be used for species richness estimation. breakaway, breakaway_nof1 and chao_bunge provide viable alternatives and are included in this package. Chao, A. (1984). Nonparametric estimation of the number of classes in a population. Scandinavian Journal of Statistics, 4. breakaway; apples; wlrm_untransformed; chao1; chao_bunge

14 14 chao_bunge chao1_bc(apples) chao_bunge function for species richness estimation This function implements the species richness estimation procedure outlined in Chao and Bunge (2002). chao_bunge(data, cutoff=10, print=true, answers=false) data The sample frequency count table for the population of interest. See breakaway for details. cutoff The maximum frequency count to use for prediction. Defaults to 10. print answers Logical: whether the results should be printed to screen. Logical: whether the function should return an argument. Value name est seest ci The name of the model. The estimate of total (observed plus unobserved) diversity. The standard error in the diversity estimate. An asymmetric 95% confidence interval for diversity. Note This estimator is based on the negative binomial model and for that reason generally produces poor fits to microbial data. The result is artificially low standard errors. Caution is advised. Chao, A. and Bunge, J. (2002). Estimating the number of species in a stochastic abundance model. Biometrics, 58.

15 hawaii 15 breakaway; apples; wlrm_untransformed; chao1 chao_bunge(apples) hawaii hawaii dataset Frequency count table for a microbial community sampled in the soil near the Kohala Volcano, Hawaii. hawaii Source Chadwick, O.A. et al. (2007). Precontact vegetation and soil nutrient status in the shadow of Kohala Volcano, Hawaii, Geomorphology, 89 1, breakaway breakaway(hawaii) wlrm_transformed function for species richness estimation This function implements the transformed version of the species richness estimation procedure outlined in Rocchetti, Bunge and Bohning (2011). wlrm_transformed(data, print=true, plot=false, answers=false, cutoff=na)

16 16 wlrm_transformed data print plot answers cutoff The sample frequency count table for the population of interest. See breakaway for details. Logical: whether the results should be printed to screen. Logical: whether the data and model fit should be plotted. Logical: whether the function should return an argument. The maximum frequency count to use for prediction. Defaults to maximal. Value name para est seest full ci The name of the model. Estimated model parameters and standard errors. The estimate of total (observed plus unobserved) diversity. The standard error in the diversity estimate. The chosen linear model for frequency ratios. An asymmetric 95% confidence interval for diversity. Note While robust to many different structures, model is almost always misspecified. The result is usually implausible diversity estimates with artificially low standard errors. Extreme caution is advised. Rocchetti, I., Bunge, J. and Bohning, D. (2011). Population size estimation based upon ratios of recapture probabilities. Annals of Applied Statistics, 5. breakaway; apples; wlrm_untransformed wlrm_transformed(apples) wlrm_transformed(apples,plot=false,print=false,answers=true)

17 wlrm_untransformed 17 wlrm_untransformed function for species richness estimation This function implements the untransformed version of the species richness estimation procedure outlined in Rocchetti, Bunge and Bohning (2011). wlrm_untransformed(data, print=true, plot=false, answers=false, cutoff=na) data print plot answers cutoff The sample frequency count table for the population of interest. See breakaway for details. Logical: whether the results should be printed to screen. Logical: whether the data and model fit should be plotted. Logical: whether the function should return an argument. The maximum frequency count to use for prediction. Defaults to maximal. Value name para est seest full ci The name of the model. Estimated model parameters and standard errors. The estimate of total (observed plus unobserved) diversity. The standard error in the diversity estimate. The chosen linear model for frequency ratios. An asymmetric 95% confidence interval for diversity. Note This estimator is based on the negative binomial model and for that reason generally produces poor fits to microbial data. The result is usually artificially low standard errors. Caution is advised. Rocchetti, I., Bunge, J. and Bohning, D. (2011). Population size estimation based upon ratios of recapture probabilities. Annals of Applied Statistics, 5.

18 18 wlrm_untransformed breakaway; apples; wlrm_transformed wlrm_untransformed(apples)

19 Index Topic datasets apples, 3 hawaii, 15 Topic diversity betta, 4 betta_pic, 5 betta_random, 6 breakaway, 8 breakaway_nof1, 10 chao1, 12 chao1_bc, 13 chao_bunge, 14 wlrm_transformed, 15 wlrm_untransformed, 17 Topic error breakaway_nof1, 10 Topic microbial breakaway, 8 breakaway_nof1, 10 Topic models breakaway, 8 breakaway_nof1, 10 chao1, 12 chao1_bc, 13 chao_bunge, 14 wlrm_transformed, 15 wlrm_untransformed, 17 Topic nonlinear breakaway, 8 breakaway_nof1, 10 chao1, 12, 13, 15 chao1_bc, 12, 13 chao_bunge, 12, 13, 14 hawaii, 15 wlrm_transformed, 15, 18 wlrm_untransformed, 12, 13, 15, 16, 17 apples, 3, 5, 9 13, 15, 16, 18 betta, 2, 4, 6, 7 betta_pic, 5, 5, 7 betta_random, 6 breakaway, 2, 3, 5, 8, 10 13, 15, 16, 18 breakaway-package, 2 breakaway_nof1, 2, 5, 9, 10 19