Calculating Confidence Intervals on Proportions of Variability Using the. VARCOMP and IML Procedures

Transcription

1 Calculating Confidence Intervals on Proportions of Variability Using the VARCOMP and IML Procedures Annette M. Green, Westat, Inc., Research Triangle Park, NC David M. Umbach, National Institute of Environmental Health Sciences, Research Triangle Park, NC ABSTRACT A common problem in study design is determining the consistency between measurements taken on different occasions. In other words, do you need to get several readings on a subject and then take the average of them, or is the variability low enough between readings that one reading is a valid indicator of the true response? Using confidence intervals on proportions of variability can be an effective method of making this determination. If the proportion of variability due to "occasion" is relatively smail when compared to the proportion due to "subject", then we do not need to take the average of multiple measurements for that particular response. This paper describes a technique to calculate such confidence intervals for a two-factor nested design with unequal subsamples using the V ARCOMP procedure from SAS/ST A -r in conjunction with SASIIML. INTRODUCTION A common problem in designing field studies is to assess consistency or repeatability between measurements taken on different occasions. If the variability between occasions is small enough, measuring a response at a single time point should provide a precise picture of a subject's "typical" response. However, if the variability between occasions is relatively large, the researcher would want to take measurements at several distinct times and use their average as a more precise description of that subject's response. Confidence intervals on the proportions of variability due to subjects and due to occasions are important tools for assessing consistency. Graybill and Wang (1979) pointed out that no exact confidence intervals are available for proportions of variability in two-factor nested designs. They therefore developed approximate confidence intervals for designs in which each cell at a given level of the hierarchy had an eqnal number of observations. Burdick, Birch, and Graybill (1986) extended these earlier results to situations where the second level of the hierarchy had uneqnal replication but the lowest level of subsampling retained eqnal replication. Because our data had uneqnal replication at the lowest level of snbsampling, we required an alternative approach to calculating confidence intervals. We employed maximum likelihood (ML) techniques to develop large-sample confidence intervals for proportions of variability and used the V ARCOMP and SASIIML procedures for the calculations. BACKGROUND This work arose in the context of a short-term dietaty study of ninety-four post-menopausal women (Baird et al., 1995). Blood was drawn from each subject on two separate occasions, one week apart. On each occasion, four separate draws were made at twenty minute intervals and equal aliquots of serum from each draw were pooled into a single sample. The serum was pooled in order to reduce the measurement variability caused by the pulsatile release of some of the hormones of interest in the study. One such pulsatile hormone is luteinizing hormone (LH), which is the response variable in our particular example. Radio-immunoassays were performed on either two or four subsamples from the pooled serum 1058

2 for each subject on each week. The data were recorded in the following variables: RESPONSE the response to be modeled (in our example, loge ill concentration), SUBJECT the identifier of individual subjects, TIME the identifier of the occasion (in this case, week of blood draws (first or second». METHODS A random effects model for the two-factor nested design used in our study can be represented as: Y"k="+S +T- +E k ljrll]l] (I) i = 1,...,1; j = I,...,J i ; k = 1,...,Kij; where Jl is an unknown constant (fixed effect), and the S;'s, T ij's, and Eijk'S are jointly independent normal random variables with zero d means an vanances cr s. cr t. and () e. respectively. We call these three variances the variance components. The variance of a single observation (or total variance), denoted by (52, is the sum of the individual variance components, that is, (52 = (52 s + (52t + (52 e' In our example, the normal random variables represent random effects due to subjects (S), time of blood draw (T), and the "error" in multiple measurements on blood from a single pooled serum sample (E). We are interested in how the variance of an observation is apportioned among the three variance components. In other words, we are interested in two different parameters which give the proportions of variability attributable to subjects and to occasions. The new parameters are related to the variance components as follows: In each case the numerator consists of the variance component for the random effect of interest (i.e. subject or occasion). The denominator is composed of the total variance (i.e. the sum of the variance components from each random effect inciuding the unexplained error). Our goal is to provide point and interval estimates of these proportions of variability. (Note that Ps is the intraciass correlation between two observations on the sarne subject at different time periods and that Ps + Pt is the intraciass correlation between two observations on the sarne subject at the sarne time period.) Point estimates of these proportions of variability can be made by substituting estimates of the variance components into equation (2). In particular, the resulting point estimates are maximum likelihood estimates when the ML estimates of the variance components are used. Our approach to interval estimation is a straightforward application of maximum likelihood theory. First, we transform the asymptotic covariance matrix for the three variance components, V, into a covariance matrix for the first two variance components and the total variance, E. Next, we use a Taylor series approximation (multivariate delta method) (Bishop et al., 1975) to derive the asymptotic covariance matrix of the proportions of variability, W, from E. The delta method requires a matrix of partial derivatives (Jacobian matrix, J _ M) of each proportion of variability with the total variance. The square roots ofthe diagonal elements ofw are the standard errors of the proportions of variability. Confidence intervals can then be calculated in a simple fashion as,estimate ± z(j. (standard error of estimate) where Za. is the two-sided size-a. standard normal critical value. The following generic code can be used to produce the estimates of the variance components and their covariance matrix for a linear model with two nested factors. proc varcomp data=one method=ml; class subject time; model response = subject time(subject); ru~; Table 1 shows the output from our example. Estimates of the variance components are given in the last iteration from the maximum likelihood procedure. Using those variance 1059

3 components and the asymptotic covariance matrix of the estimates, one can calculate point and intelval estimates of the proportions of variability using SASfIML as follows. PROCIML; C = {Cu, Cll' C31}; 'The vector C contains the point estimates of the variance components: Cll = &s2, Cll = &l, and C31 = &/ which are produced by the V ARCOMP procedure. V= {Vll V12 Vll V22 V31 V32 'The matrix V is the asymptotic covariance matrix of estimates produced from the V ARCOMP procedure (see Table I). In our example this is a 3X3 matrix. L= {I o I I I 0, 0, I}; 'L is the contrast matrix which transforms the vector of three variance components into a vector consisting of the first two variance components and the total variance. I 0 o I 0 I I I necessary for subject point estimate necessary for time point estimate necessary for total variance time, and total variance, i.e. for CNEW. (Note: L' is the matrix transpose of L). RHOHAT_S= CNEW[1,l]fCNEW[3,l]; RHOHAT_T = CNEW[2,l]fCNEW[3,l]; 'The scalar RHOHAT_S contains the point estimate of the proportions of variability for subjects, ps; RHOHAT_T contains the point estimate of the proportion of variability for. " occasions, Pt- J_M =J(2,3,0); J_M[l,l] = licnew[3,l] J_M[1,3] = - CNEW[1,l]fCNEW[3,l]**2; J_M[2,2] = l/cnew[3,l]; J_M[2,3] = - CNEW[2,1]/CNEW[3,l]**2;, o 122 where J_Mll andj_m22 are equal to 11&2 where J-M13 equals -&If(&2)2 where J_M23 equals _&?1(&2)2 J_M is the Iacobian matrix of transformation. This is a matrix of partial derivatives of the proportions of variability with respect to the first two variance components and the total variance. CNEW= L*C; 'The vector CNEW contains the point estimates of the variance components &s2, a?, and "2 <5. E = L*V*L'; *E is the 3X3 asymptotic covariance matrix for the variance component estimates for subject, *W is the covariance matrix for the proportions of variability. The diagonal elements are variances and the off diagonal elements are covariances. Although variances will always be positive, covariances may not be. The DlAG function allows us to focus on the variance values (and to avoid taking the square root of negative covariance estimates). W _SQRT = sqrt(diag(w); 1060

4 *The first element of W _ SQRT (Le. W _SQRT 11) is the estimated standard error for Ps whereas W _ SQRT 22 is the estimated standard error for Pt. With these standard errors, confidence intervals for proportions of variability can now be calculated. The asymptotic 95% confidence intervals for the proportion of variability were obtained by the following formulae: LOWER_S ~ RHODAT_S - (1.96 * W_SQRT[l,l)); UPPER_S ~ RHODAT_S + (1.96*W_SQRT[l,l)); LOWER_T ~ RHODA T _ T -(1.96*W _SQRT[2,2)); UPPER_T RHODAT_T+(1.96*W _SQRT[2.2)); PRINT LOWER_S UPPER_S LOWER_T UPPER] QUIT; RESULTS The results of this particular analysis show that the variance component due to occasion (estimated as ) is much smaller than the variance component due to subjects (estimated as ). The proportions of variability reflect this same relationship: the proportion of variability attributable to occasions (Pt) is much smaller than that attributable to subjects (PS> (estimated as versus ). The corresponding confidence intervals using our approach are (0.0355, ) for occasions and (0.8987, ) for subjects. The confidence limits obtained from our method using all the available data are similar to those obtained from Burdick et al. 's procedure after randomly removing some dati so that every occasion bas the same number of measurements (as required by their method). The 95% confidence intervals produced by their method were (0.0344,0.0857) for occasions and (0.8898,0.9488). Thus, for this example, both confidence intervals are comparable, with the maximum likelihood intervals being slightly shorter. CONCLUSION With the aid of the V ARCOMP and SASIIML software from SAS Institute one can easily calculate an estimate and confidence interval for a proportion of variability. Other statistical information such as graphics and reports can also be readily produced utilizing the information provided by this procedure. While the method of Burdick, et al. (1986) could not be used with our data due to unequal subsampling, their intervals seem io have good coverage properties even in relatively small samples (as seen in their simulation study). Our likelihood-based method is guaranteed to have nominal 95% coverage only in large samples and may not provide proper coverage in small samples (we have yet to do a simulation study). However, our method is simpler to apply and, for the data in our example, both methods provide similar confidence intervals. Our particular example dealt with a simple linear model with two nested factors. One can readily apply this method to models with more layers of hierarchy by expanding the various matrices used. REFERENCES D.D. Baird, D.M. Umbach, L. Landsdell, C.L. Hughes, KD.R. Setchell, C.R. Weinberg, A.F. Haney, AJ. Wilcox, la. McLachlan, Dietary intervention study to assess estrogenicity of dietary soy among postlt1enopausal women. Journal of Clinical Endocrinology and Metabolism. (In press). Y.M.M. Bishop, S.E. Fienberg, and P.W. Holland, Discrete Multivariate 1061

5 Analysis: Theory and Practice. The MIT Press. Cambridge, MA. pp RK. Burdick, N.J. Birch, and FA Graybill, Confidence intervals on measures of variability in an unbalanced two-fold nested design with equal subsampling. J. Statist. Comput. Simul., 1986, 25: FA Graybill and C-M. Wang, Confidence intervals for proportions of variability in two-factor nested variance component models. J. Amer. Statist. Assoc., 1979,74(366): ACKNOWLEDGMENTS We would like to thank Dr. Donna Day Baird, Epidemiology Branch at the National Institute of Environmental Health Sciences for presenting us with the problem that motivated this work and providing us with the data for our example. SAS, SASIIML, and SAS/ST AT are registered trademarks or trademarks of SAS Institute Inc. in the USA and other courntries. indicates USA registration. Other brand and product names are registered trademarks or trademarks of their respective companies. SAS/IML 1M User's Guide, Release 6.03 Edition, SAS/STA -r User's Guide, Volume 2, GLM- V ARCOMP, Version 6, Fourth Edition,

6 Table 1. Example of output from the V ARCOMP Procedure Invoked using the ML option. Variance component estimates and the entries of the covariance matrix are indicated in boldface type. Dependent Variable: RESPONSE Model RESPONSE = SUBJECT TIME(SUBJECT) Method = Maximum Likelihood (mi) Maximum Likelihood Variance Components Estimation Procedure Iteration Objective Var(SUBJECT) Var(TIME(SUBJECT) Var(Error) I Convergence criteria met Asymptotic Covariance Matrix of Estimates Var(SUBJECT) Var(TlME(SUBJECT) Var(SUBJECT) Var(TlME(SUBJECT) Var(Error) Var(Error)