Multi-factor analysis of variance

Transcription

1 Faculty of Health Sciences Outline Multi-factor analysis of variance Basic statistics for experimental researchers 2016 Two-way ANOVA and interaction Matched samples ANOVA Random vs systematic variation Julie Forman and Claus Thorn Ekstrøm Department of Biostatistics, University of Copenhagen Mixed models Repeatability and reproducibility 2 / 67 Two-way analysis of variance Example: Tumor growth What is the effect of two treatments in combination? B A 1 2 c 1 2. r Quantification / test of interaction. Effect of treatment A? Effect of treatment B? Do the two treatments interact? Possible with multiple observations for each combination. not possible with one observation for each combination. Randomized experiment with 28 animats. Two treatments: days (0/1): time of treatment / at day one. radiation (Control/10 Gy): nicely balanced Outcome: tumor volume. Note: one measurement on each animal we have independent data! 3 / 67 4 / 67

2 Data from combined treatment groups Interaction plot Radiation Day Mean (SD) Control (36.1) Control (46.1) Gy (73.3) Gy (47.3) > load("tumgrow.rda") > tumgrow$days <- factor(tumgrow$days) > plot(tumvol ~ interaction(radiation, days), + data=tumgrow, + col="lightblue", cex.lab=1.4, cex.axis=1.4) TumVol Control.0 10 Gy.0 Control.1 10 Gy.1 interaction(radiation, days) > with(tumgrow, interaction.plot(days, radiation, TumVol) ) mean of TumVol days radiation Control 10 Gy Sample means for combined treatments: Do we see the same effect of days with and without radiation? 5 / 67 6 / 67 Interaction What is interaction? Interaction between two treatments (or factors) means that The effect of the two treatments depend on one another. When interaction is present: Quantify differences between treatment combinations as in a one-way ANOVA. Estimated effects of the one treatment must be presented for each value of the other treatment in turn (and vice versa). Interaction is also called effect modification Because the effect of one treatment is modified by the other. 7 / 67 8 / 67

3 Model with or without interaction Parameter estimates Model: for k th anima with combination of i th and j th treatment. Y ijk = µ + α i + β j + γ ij + ε ijk µ: mean of reference group (no radiation, day 0). α: effect of radiation (at day 0) β: effect of time (without radiation) γ: possible interaction (i.e. increase/decrease in anticipated effect when the two treatments are combined). ε ijk s are the error terms assumed independent N(0, σ 2 ) In case γ = 0 we say that there is no interaction or that the treatment effects are additive. We can test this as a hypothesis. Model: With interaction Without interaction Effect Estimate (95% CI) Estimate (95% CI) Intercept (117.9; 204.8) (133.3; 205.9) Radiation (-67.2; 47.8) (-64.3; 16.1) Days (-15.7; 107.3) (-10.6;69.0) Interaction (-110.3; 52.4) assumed = 0! Hence the estimated effect of radiation is lower (95% CI to 52.4) for an animal examined at day 1 compared to an animal examined at day 0. The interaction is not significant, but the confidence is wide so we cannot completely rule out a possible effect modification. 9 / / 67 Interpretation of parameter estimates. Expected tumor volume for each treatment combination radiation day control Gy = = = = Here we have added the estimated treatment effects to the mean of the reference group. Testing interaction in R lm(tumvol ~ radiation*days, data=tumgrow) or similarly lm(tumvol ~ radiation + days + radiation:days, data=tumgrow) Note that: Use lm for two-way ANOVA Both factors must be factors either in the call or in the data frame. If reference groups are not chosen R uses the first in alphabetic order. Use relevel to change the reference level for a factor. Both the main effects and the interaction must be included in the model formula (either manually or through *). 11 / / 67

4 Output: test for interaction > result <- lm(tumvol ~ radiation*days, data=tumgrow) > drop1(result, test="chisq") Single term deletions Model: TumVol ~ radiation * days Df Sum of Sq RSS AIC Pr(>Chi) <none> radiation:days Type III-type tests: Drop one factor while keeping the others. Hierarchical: Do not test main effects if interaction is present. Output: parameter estimates > result <- lm(tumvol ~ radiation*days, data=tumgrow) > summary(result) Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) e-08 *** radiation10 Gy days radiation10 Gy:days Signif. codes: 0 *** ** 0.01 * Residual standard error: on 24 degrees of freedom Multiple R-squared: ,Adjusted R-squared: F-statistic: 1.43 on 3 and 24 DF, p-value: / / 67 Lots of useful information! Post hoc testing: Interaction Post hoc testing: Interaction If interaction is present, compare all treatment combinations with: > library(lsmeans) > all <- lsmeans(result, ~ radiation*days) > pairs(all) contrast estimate SE df t.ratio p.value Control,0-10 Gy, Control,0 - Control, Control,0-10 Gy, Gy,0 - Control, Gy,0-10 Gy, Control,1-10 Gy, Or assess the effect of each treatment given the other with: > all <- lsmeans(result, ~ radiation*days days) > pairs(all) days = 0: contrast estimate SE df t.ratio p.value Control - 10 Gy days = 1: contrast estimate SE df t.ratio p.value Control - 10 Gy P value adjustment: tukey method for comparing a family of 4 estimates 15 / / 67

5 Post hoc testing: No interaction If there is no interaction, the treatment effects are additive. Hence, asses each treatment in turn with: > result2 <- lm(tumvol ~ radiation + days, data=tumgrow) > lsmeans(result2, ~ radiation) radiation lsmean SE df lower.cl upper.cl Control Gy Results are averaged over the levels of: days Confidence level used: 0.95 > lsmeans(result2, ~ days) days lsmean SE df lower.cl upper.cl Model checking The error terms ε rt s are assumed to be independent (this we know to be true). normally distributed with zero mean and equal variances Use the residuals for model checking: Probability or QQ-plot of residuals. Plot of residuals vs expected values and/or factors. Any outliers in the data? Results are averaged over the levels of: radiation Confidence level used: / 67 or use summary() since there are only two levels of each factor. Expected values and residuals 18 / 67 Diagnostic plots Expected value: for radiation=gy 10, day=1: ŷ ij = ˆµ + ˆα i + ˆβ j + ˆγ ij = = > library(mess) > residualplot(result) > qqnorm(residuals(result)) > qqline(residuals(result)) Normal Q Q Plot Residual: for the last animal: ε ijk r ijk = observed expected = y ijk ŷ ij ε st = = Stud.res Fitted values Sample Quantiles Theoretical Quantiles 19 / / 67

6 Outline Overview: comparison of treatment groups Two-way ANOVA and interaction Matched samples ANOVA Random vs systematic variation Mixed models Repeatability and reproducibility number independent paired of groups samples samples 2 unpaired paired t-test t-test 2 one-way two-way analysis of variance analysis of variance or mixed model Analysis of variance: Last week: t-tests and one-way ANOVA. Today: Two-way ANOVA and mixed models. 21 / / 67 Example + exercise: Gene expressions Spaghetti-o-gram Four treatments applied to five cell lines (from lecture 2). Treatment ctrl A B C > library(lattice) > load("geneexp.rda") > xyplot(ge ~ treatment, + groups=cellline, + data=geneexp, type="l") > xyplot(log(ge) ~ treatment, + groups=cellline, + data=geneexp, type="l") log(ge) ge Do we see: differences among treatments? differnces among cell lines? (Is this interesting?) Interaction? (Not possible to test and not that interesting) 23 / Ctrl A B C treatment Ctrl A B C treatment The cell lines should be roughly parallel and equally variable 24 / 67 Log-transformed seems better than raw data.

7 Two-way ANOVA model Measurement for subject s with treatment t: Y st = µ + α s + β t + ε st Test of treatment effect > result <- lm(log(ge) ~ treatment + cellline, data=geneexp) > drop1(result, test="chisq") Single term deletions µ is the intercept (mean of reference) α s describe expected differences between cell lines. β s describe expected differences between treatments. The error terms ε st s are assumed to be independent normally distributed with equal variances The model assumptions should be checked / 67 Model: log(ge) ~ treatment + cellline Df Sum of Sq RSS AIC Pr(>Chi) <none> treatment e-05 *** cellline ** --- Signif. codes: 0 *** ** 0.01 * We find significant differences among treatments (interesting) and among cell lines (not that interesting... ). 26 / 67 Parameter estimates > summary(result) Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) *** treatmenta *** treatmentb treatmentc cellline cellline cellline cellline ** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 12 degrees of freedom Multiple R-squared: ,Adjusted R-squared: F-statistic: on 7 and 12 DF, p-value: Note: The control treatment has been chosen as reference (the way the factor was created). Treatment A, B, and C parameter estimates are expected differences wrt the control... on 27 / 67 log-scale!. Estimates of treatment effect As compared to the control group: Treatment log-scale back-transformed A 1.16 (0.58;1.74) +218% (+79%;+467%) B 0.17 (-0.41;0.75) +19% (-33%;+111%) C 0.13 (-0.45;0.70) +13% (-36%;+102%) i.e. treatment A approximately triples the gene expression level 100 {exp( ) 1} 100 ( ) 218. Multiple comparisons: could be performed by e.g. using lsmeans or the multiple comparison approaches from last week! 28 / 67

8 Outline Two-way ANOVA and interaction Matched samples ANOVA Random vs systematic variation Mixed models Repeatability and reproducibility 29 / 67 Gene expressions again From before \begin{verbatim} Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) ***.. cellline cellline cellline cellline ** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 12 degrees of freedom Multiple R-squared: ,Adjusted R-squared: F-statistic: on 7 and 12 DF, p-value: Overall significant differences in gene expression levels were found among the cell lines (P=0.0012), and estimates show that some cell lines differ. 30 / 67 Should this be reported as an interesting finding? Fixed and random effects Fixed effects such as treatment, dose, and time. Typically a limited number of carefully selected groups. Group names are specific and cannot be shuffled. Each group must have a decent size in order to reach interesting conclusions (statistical power). Random effect such as rat, cell line, experiment or operator. Possibly a large number of different groups. Group names are non-informative (number of rat or cell line) and could be shuffled without consequence. Allows inference to be extended beyond the subjects in the experiment and to the population they were sampled from. The number of groups matters not the size of the groups. Example: experiment with rabbits R = 6 rabbits vaccinated. On S = 6 spots on the back of each. Response: swelling in cm 2 Model: Which one? A one-way ANOVA with a random effect; the rabbit-factor > load("rabbit.rda") > plot(swelling ~ rabbit, + data=rabbit, + cex.axis=1.4, + cex.lab=1.4) swelling rabbit 31 / / 67

9 One-way ANOVA with random variation Comparison of k groups/clusters, satisfying: The groups are of no individual interest and it is of no relevance to test whether they have identical means. The groups may be thought of as a random sample from a population, that we want to describe. Example: Swelling was measured 6 times consecutively on a sample of 6 rabbits. 33 / 67 What response can we expect in the population? Test for identical rabbits means: P = (one-way ANOVA) is not very helpful in this regard, and neither are the estimates of differences between specific rabbits. Mean swelling with 95% CL (and normal range) is better. Random effects ANOVA model Model for response of s th spot on r th rabbit: Y rs = µ + a r + ε rs µ is the grand mean (i.e. of the rabbit population). a r is the between-rabbit deviation (i.e. how does rabbit r deviate from the grand mean). ε rs is the within-rabbit deviation (i.e. how does spot s deviate from its rabbit s mean). It is assumed that all error terms (a r s and ε rs s) are independent and normally distributed: a r N(0, ω 2 B), ε rs N(0, σ 2 W ) The deviations between rabbits are considered random and their variance ωb 2, is called the between-rabbit variance component. 34 / 67 Implications of random effects anova Each single observations is sampled from the same population assumed to follow the normal distribution: Y rs N(µ, ω 2 B + σ 2 W ) Population mean µ (the grand mean). Population variance ω 2 B + σ2 W (the total variation). But: Measurements made on the same rabbit are correlated with the so-called intra-class correlation More about correlation next lecture Parameter estimates Grand mean (µ): 7.37 (6.68;8.05). Variance components: Variation Variance component Estimate (95% CI) %of variation Between ω 2 B 0.33 (0.03; 1.04) 36% Within ω 2 W 0.58 (0.06; 2.48) 64% Total ω 2 B + σ2 W % Corr(y r1, y r2 ) = ρ = ω 2 B ω 2 B + σ2 W Warning: Confidence intervals for the variance components may be invalid due to the tiny sample size (only six rabbits). I.e. measurements made on the same rabbit tend to look more alike than measurements made on different rabbits 35 / / 67

10 Interpretation of variance components Mixed models in R Typical difference between spots on the same rabbit: y rs1 y rs2 = µ + α r + ε rs1 (µ + α r + ε rs2 ) = ε rs1 + ( ε rs2 ) N(0, 2 ωw 2 ) Normal region: ± = ± 2.16 cm 2 Typical difference between spots on different rabbits: y r1 s 1 y r2 s 2 = α r1 + ( α r2 ) + ε rs1 + ( ε rs2 ) N(0, 2 (σb 2 + ωw 2 )) Normal region: ± 2 2 ( ) = ± 2.70 cm 2 > library(lme4) > rabbit$rabbit <- factor(rabbit$rabbit) > result <- lmer(swelling ~ 1 + (1 rabbit), data=rabbit) Syntax is similar to lm with a model formula specifying the relationship between outcome and covariates. Categorical variables must be set to be factors. Random effects are specified by (1 group) in the model formula. Note: If Y 1 N(µ 1, σ 2 1) and Y 2 N(µ 2, σ 2 2) are independent normal variables, then their difference is normal Y 1 Y 2 N(µ 1 µ 2, σ σ 2 2). 37 / / 67 R: Mixed model output > summary(result) Linear mixed model fit by REML ['lmermod'] Formula: swelling ~ 1 + (1 rabbit) Data: rabbit REML criterion at convergence: 91.5 Scaled residuals: Min 1Q Median 3Q Max Random effects: Groups Name Variance Std.Dev. rabbit (Intercept) Residual Number of obs: 36, groups: rabbit, 6 Fixed effects: Estimate Std. Error t value (Intercept) Always check that numerical optimisation has converged. Finally: 39 / 67 Parameter estimates, and something like tests. Negative variance components Warning: It may happen that some programs reports a zero-estimate for the variation between, ω 2 B. By coincidence. Thus the model is OK. As a result of competition within clusters. 40 / 67 Example: yield of plants grown in the same pot. Thus, the model is wrong as the clustering leads to dissimilarities (negative correlation) rather than similarities (positive correlation) in outcome.

11 Comparison of modeling strategies Comments on the strategies: Quantifying overall swelling Four strategies for estimating the grand mean of the rabbit population method estimate (s.e.) 1: forget rabbit (0.155) 2: fixed rabbit (0.127) 3: rabbit averages (0.267) 4: random rabbit (0.267) 1. We (wrongfully) assume independence all 36 measurements 2. We estimate the mean swelling by classical one-way anova. 3. We reduce the data to six averages from the individual rabbits and then compute mean and SE. 4. We estimate the mean swelling in the random effects anova model. 1. Ignoring the clustering is wrong! leads to systematic underestimation of the standard error. 2. In the fixed effect one-way anova the grand mean has a different interpretation!... as the mean swelling of these six particular rabbits. leads to systematic underestimation of the standard error. 3. Looking at the sample of averages may be OK. At least in balanced designs (otherwise the individual averages have unequal variances and the standard error may be affected) But we loose information on within subject variation. 41 / / 67 Unbalanced data We delete the 3 smallest measurements from rabbit 2 (largest level) so that the data becomes unbalanced and the results change: method estimate (s.e.) 1: forget rabbit (0.163) 2: fixed rabbit (0.136) 3: rabbit averages (0.333) 4: random rabbit (0.298) Full sample (0.267) 1 we have omitted some of the largest observations 2 rabbit 2 has a lower weight in the average (only 3 observations) 3 average for rabbit 2 has increased 4 rabbit 2 has a lower weight in the average due to a larger standard error Design considerations Plan an experiment with: R rabbits (independent or true replicates). S spots for each rabbit (repeated measurements or pseudo replicates). R S measurements. Then variance of mean estimate var(ȳ) = ω2 B R + σ2 W RS, decreases with R and S. standard error rabbits The different curves correspond to S varying from 1 to / / 67

12 Effective sample size How many rabbits would we need to obtain the same precision in estimating the grand mean if we had only one measurement on each of R 1 rabbits? Solve the equation for var(ȳ): R 1 = R S 1 + ρ(s 1) where ρ is the within rabbit correlation. Outline Two-way ANOVA and interaction Matched samples ANOVA Random vs systematic variation Mixed models Estimate: ρ = ω2 B ω 2 B +σ2 W = = R 1 = 12.8 Repeatability and reproducibility I.e. one measurement on each of thirteen rabbits gives the same precision as six measurements on each of six rabbits! 45 / / 67 Linear mixed models Multi-level models Generalisations of ANOVA and GLM models involving both fixed effects (covariates) and several sources of random variation, the so-called variance components. Environmental variation. Between clinics, regions or countries. Biological variation. Between patients, animals, or cell lines. Within-individual variation. Between injection sites, tumors, slices. Variation due to uncontrollable circumstances. E.g. day to day, assay, observer. Measurement error. E.g. duplicates, triplicates. Mixed models are also called variance component models. Often we have a multi-level model with hierarchical ordering of the levels. We have variation (i.e. a variance component) on each level. And possibly fixed effects (covariates) on each level. individual context/cluster context/cluster level 1 level 2 level 3 spots rabbits slices tumors mice duplicates experiments operators Arrows indicate simplification or grouping. 47 / / 67

13 Merits of mixed models Drawbacks of mixed models We get a better understanding of the various sources of variation. Certain effects may be estimated more precisely (higher power), since some sources of variation are eliminated, e.g. by making comparisons within the same subject. This is analogous to the paired comparison situation. When planning subsequent investigations, the knowledge of the relative sizes of the variance components will (in principle) be of help in deciding the number of repetitions needed at each level. Independent (sometimes called true) replicates Repeated measurements (called pseudo replicates) Their statistical analysis is more difficult. When making inference (estimation and testing), it is important to take all sources of variation into account. Results may be biased if one or more sources of variation are disregarded! Only few statistical software can do the correct analyses. 49 / / 67 Testing fixed effects Testing fixed effects Imagine that rabbits are grouped in two (e.g. treatments): Rabbit 1 3 is group 1, 4 6 is group 2 level variation covariates 1 within rabbit spot 2 between rabbits group Part of the variation between rabbits could be explained by systematic differences between groups. Part of the variation within rabbits could be explained by systematic differences between spots. > rabbit$group <- factor(rabbit$rabbit %in% c("4", "5", "6"), + labels=c("grp1", "Grp2")) > result <- lmer(swelling ~ spot + group + (1 rabbit), + data=rabbit) > result Linear mixed model fit by REML ['lmermod'] Formula: swelling ~ spot + group + (1 rabbit) Data: rabbit REML criterion at convergence: Random effects: Groups Name Std.Dev. rabbit (Intercept) Residual Number of obs: 36, groups: rabbit, 6 Fixed Effects: (Intercept) spotb spotc spotd spote spotf groupgrp Output: rabbit < larger than before Residual < smaller than before 51 / / 67

14 Testing fixed effects with lmer (May need to restart R session) > library(lmertest) > result <- lmer(swelling ~ spot + group + (1 rabbit), + data=rabbit) > summary(result) Linear mixed model fit by REML t-tests use Satterthwaite approximations to degrees of freedom [lmermod] Formula: swelling ~ spot + group + (1 rabbit) Data: rabbit REML criterion at convergence: 84.3 Scaled residuals: Min 1Q Median 3Q Max Random effects: Groups Name Variance Std.Dev. rabbit (Intercept) Residual Number of obs: 36, groups: rabbit, 6 Fixed effects: Estimate Std. Error df t value Pr(> t ) (Intercept) e-08 *** spotb spotc spotd spote spotf / 67 groupgrp u n i v e rsignif. s i t y ocodes: f c o p e0 n *** h a g e0.001 n ** 0.01 * Correlation of Fixed Effects: (Intr) spotb spotc spotd spote spotf Summary spotb spotc spotd spote spotf groupgrp Measurements belonging together in the same cluster tend to look alike (they are correlated). If we fail to take this into account, we will experience: Possible bias in estimates (in unbalanced data). Too small standard errors (type 1 error) for estimates of level 2 effects (between-cluster effects). Too low efficiency (type 2 error) for evaluation of level 1 covariates (within-cluster effects) Disregarding repeated measurements When the random rabbit variation is ignored: Too small standard errors for estimates of difference between groups and too large standard errors for estimates of differences between spots! > result <- lm(swelling ~ spot + group, + data=rabbit) > summary(result) Call: lm(formula = swelling ~ spot + group, data = rabbit) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) <2e-16 *** spotb spotc spotd spote spotf groupgrp Signif. codes: 0 *** ** 0.01 * Residual standard error: on 29 degrees of freedom 54 Multiple / 67 R-squared: , Adjusted R-squared: F-statistic: on 6 and 29 DF, p-value: Outline Two-way ANOVA and interaction Matched samples ANOVA Random vs systematic variation Mixed models Repeatability and reproducibility 55 / / 67

15 Comparing measurement devices Illustration of all data Example: Peak expiratory flow rate, l/min: 17 subjects, 2 measurement devices, two replicates with each method. subject Wright mini Wright id Y 1p1 Y 1p2 Y 2p1 Y 2p Average SD Reference: Bland and Altman, Lancet (1986). 57 / / 67 Aim of investigation Simple approaches Quantify the precision of each measuring device Repeatability (variability=measurement error) Quantify the agreement between the two devices. Bias of one method compared to the other. Variance of one method compared to the other. Can the devices be used interchangably? For reliability of each method separately we could: make Bland Altman plots of differences vs averages. compute limits of agreement, i.e. the 95% normal range of the differences. For reproducibility (method comparison) we might: compare the averages in a Bland-Altman plot? Not good - unless you also do averages in clinic! For both at the same time: Mixed model for variance between and within methods. 59 / / 67

16 Variance component models Stratified analyses For each method (i = 1, 2) we have a variance component model 61 / 67 Y ijk = µ i + a ij + ε ijk µ i population mean as anticipated by method i. a ij deviation of subject j from population mean, assumed normally distributed N(0, σ 2 i ). ε ijk deviation for replicate k (measurement error), assumed normally distributed N(0, ω 2 i ). > load("wright.rda") > lmer(flow ~ 1 + (1 id), data=wright, subset=(method=="mini")) Linear mixed model fit by REML ['mermodlmertest'] Formula: flow ~ 1 + (1 id) Data: wright Subset: (method == "mini") REML criterion at convergence: Random effects: Groups Name Std.Dev. id (Intercept) Residual Number of obs: 34, groups: id, 17 Fixed Effects: (Intercept) > lmer(flow ~ 1 + (1 id), data=wright, subset=(method=="wright")) Linear mixed model fit by REML ['mermodlmertest'] Formula: flow ~ 1 + (1 id) Data: wright Subset: (method == "wright") REML criterion at convergence: Random effects: Groups Name Std.Dev. id (Intercept) Residual Number of obs: 34, groups: id, 17 Fixed Effects: (Intercept) 62 / Joint model for both methods Advanced analysis For methods (i = 1, 2): Y ijk = µ i + a ij + ε ijk ε ijk assumed normally distributed N(0, ω 2 i ) and independent across methods. a ij assumed normally distributed N(0, σ 2 i ) and correlated with ρ = Cor(a i1, a i2 ). Anticipated means for the same subject ought to look a lot like each other, so the a ij s are likely to be correlated across methods. > library(methcomp) > mydata <- Meth(wright, meth=3, item=4, repl=6, y=5) The following variables from the dataframe "wright" are used as the Meth variables: meth: method item: id repl: repl y: flow #Replicates Method 2 #Items #Obs: 68 Values: min med max mini wright / / 67

17 Advanced analysis Repeatability > BA.est(mydata, linked=false) Conversion between methods: alpha beta sd.pred LoA-lo LoA-up To: From: mini mini wright wright mini wright Variance components (sd): IxR MxI res mini wright Typical differences (approximate 95% normal range) between two measurement with the same method: Wright: ˆω 2 1 = ±2 2ω 2 1 ±43.3 Mini: ˆω 2 2 = ±2 2ω 2 2 ±56.3 Seemingly Wright is more precise, but is the difference significant? F = = 1.69 F (17, 17) P = 0.14 Don t form too firm a conclusion with too small data. 65 / / 67 Reproducibility No evidence of systematic differences between the two methods. Estimated bias +6.0 for mini vs wright. Typical differences between the two methods: var(y 1jk Y 2jk ) = var(a 1j a 2j + ε 1jk ε 2jk ) = σ σ 2 2 2σ 12 + ω ω 2 2 Limits-of-agreement: 6.03 ± = ( 69.3, 81.3). 67 / 67