Overview of statistical methods used in analyses with your group between 2000 and 2013

Transcription

1 Department of Epidemiology and Public Health Unit of Biostatistics Overview of statistical methods used in analyses with your group between 2000 and 2013 January 21st, 2014 PD Dr C Schindler Swiss Tropical and Public Health Institute University of Basel christianschindler@unibasch

2 Contents of the talk The major part of the talk will be devoted to the bootstrap method I will start with examples from papers with your group where bootstrap was used and also address some interpretational aspects Then I will explain the idea of bootstrap and provide some theoretical background Finally, if time permits, I will briefly talk on two-dimensional decision functions 23 Januar 2014 Präsentationstitel 2

3 observed decrase in mortality rate = 6%* observed decrease in mean cost per patient = 2060 $** cost-effectiveness plane * from 10 to 4 % ** from 6840 to 4780 $ N = 93 (BNP-guidance) N = 97 (control group) C Müller et al, Use of B-Type Natriuretic Peptide for the Managemet of Women with Dyspnea, Am J Cardiol 2004; 94:

4 observed decrase in 2 yr rate of MACE = 11% (from 31% to 20%) observed decrease in mean cost per patient = 150 $ (from to 15950$) N = 269 IVUS = intravascular ultrasound guided PTCA ANGIO = angiographic guided PTCA C Mueller et al, Cost-effectiveness of intracoronary ultrasound for percutaneous coronary Interventions, Am J Cardiol 2003; 91:143-47

5 Closer look at the bootstrap results for cost-effectiveness The percentages of bootstrap results in the four quadrants are often interpreted as probabilities for the true cost-effectiveness parameter to fall into the respective quadrant This interpretation is only correct in the framework of Bayesian statistics and does not make any sense in the framework of traditional frequentist statistics

6 Frequentist statistics assumes that there is only one reality which may at best be described with high accuracy by a concrete model defined by fixed parameters Therefore only a statement like «the parameter pair (Θ 1, Θ 2 ) lies in the 1 st (2 nd etc) quadrant with a probability of 0% or 100%» would make sense However, in Bayesian statistics, a different concept of probability is used Here, probabilities are interpreted as degrees of belief and may therefore vary from person to person (subjective probabilities)

7 If we have absolutely no idea about the true cost-effectiveness parameter a priori, any point within a large rectangular domain might be equally likely according to our prior belief Θ 2 = cost Θ 1 = effect Then, under some assumptions, the percentages of bootstrap results in the 4 quadrants may be interpreted as posterior (Bayesian) probabilities (or updated degrees of belief) for (Θ 1, Θ 2 ) to lie in the respective quadrant

8 Nonparametric bootstrap was used to estimate 95% CIs for the areas under the receiver operating characteristic (ROC) curves (each simulation using 5000 bootstrap samples drawn from the original dataset) Cut-off points were calculated using Youden s J index To assess the potential clinical relevance of BNP measurements, we used likelihood ratio tests to determine whether logistic regression models that included measurements of BNP and ASA class provided a significantly better fit than logistic regression models including ASA class alone18 The exact version of the SAS-procedure Logistic was used to estimate odds ratios (ORs) and their confidence limits The predictor variables considered in this model included preoperative BNP values and ASA classes; first considered separately and then in combination T Breidthardt et al, B-Type natriuretic peptide in patients undergoing orthopaedic surgery: a prospective cohort study, Eur J Anaesthesiol 2010; 27:690-5

9 Basics of the bootstrap and some theoretical aspects 23 Januar 2014 Präsentationstitel 9

10 A motivating example from my course with the 2nd year medical students: Among 118 female medical students, the mean value and standard deviation of BMI was 2057 and 197 kg/m 2, respectively Aim: to compute the 95%-confidence interval for the true mean value of BMI in the population of female medical students* * assuming that this population is well-defined and our sample is a true random sample from this population

11 Standard error of the sample mean SE = σ / n Sample size Standard deviation of individual values of the variable X (X = BMI in our example) in the underlying population In practice, we generally must replace σ by the standard deviation s of X in the sample

12 95%-confidence interval for the population mean µ of X ( 0975; 1 /, ; 1 / ) where and s denote the mean and the standard deviation of X in the given sample and t 0975;n-1 is the 975th percentile of the t-distribution with n-1 degrees of freedom For large n s we may replace the above t-quantile by 196, the 975th percentile of the standard normal distribution Validity assumption: This formula is absolutely correct only if X is normally distributed in the underlying population But it provides a good approximation if the sampling distribution of is close to normal, which is the case for large n s

13 Coming back to our example of BMI with n = 118, =2057 kg/m2 and s = 197 kg/m2 The standard error of equals: 197 / 118 = 197 / 1086 = 0181 And so the approximate 95% - confidence interval equals ( , ) = (2022, 2092) But can we assume that the sample size is large enough for the sampling distribution of to be sufficiently close to normal

14 Question Is our sample size n large enough to assume that the statistical error of comes from an approximate normal distribution? Dilemma How can we judge this if we don t really know the distribution of X in the underlying population?

15 Trick We clone the data of the sample to obtain a virtual population which, if the sample is sufficiently representative of the distribution of X in the underlying population, will resemble the true distribution of X Observed sample virtual population

16 From this virtual population we now draw many random samples of the same size n as the original sample In these virtual samples, some of the elements of the original sample will occur more than once while others will be missing virtual samples This approach is referred to as bootstrapping

17 Shape of the distribution of the bootstrap sample means (QQ-plots) observed Individdual values N = 1 observed Means of samples of 5 observations N = expected if distribution were normal expected if distribution were normal observed Means of samples of 10 observations N = 10 bobserved Means of samples of 20 observations N = expected if distribution were normal expected if distribution were normal

18 Conclusion: Already with a sample size of 20, the distribution of the bootstraped means is almost perfectly normal Therefore we infer that the same would be true for repeated samples of size 20 from the true underlying population, and thus even more so for samples of size 118, as in our case The assumption that the statistical error of our observed mean comes from an approximate normal distribution is certainly justified

19 The bootstrap method This idea of cloning the data of the sample to construct a virtual population and repeatedly sampling from this virtual population is the basic idea of the bootstrap If the cloned sample is large enough it is representative of the underlying population so that the virtual population obtained by cloning will resemble the original population In practice, it is not necessary to clone the original sample Generating bootstrap samples by a sequence of random draws in which each selected element is immediately replaced produces the same results («sampling with replacement»)

20 Bradley Efron, * 1938, invented the bootstrap in 1979 He is professor of Statistics and Biostatistics at the University of Stanford Wikipedia: The term (ie, bootstrap) is sometimes attributed to the story The Surprising Adventures of Baron Munchausen, where the main character pulls himself (and his horse) out of a swamp by his hair (specifically, his pigtail), but Baron Münchhausen ( ) does not, in fact, pull himself out by his bootstraps

21 Approximate confidence intervals Often the standard deviation of sample estimatesθ can be approximately calculated or estimated using bootstrap or jackknife The standard deviation of sample estimates is called «standard error» and often denoted by SE(Θ) or just SE An approximate 95%-confidence interval forθ is then given by ( Θ 196 SE(Θ), Θ SE(Θ) ) For large sample sizes, these approximate confidence intervals are generally quite accurate

22 The jackknife technique To estimate a standard error using the jackknife technique, one proceeds as follows Think of a sample with values x 1,x 2,, x n from which the parameter estimateθ is computed 1 Recompute the parameter estimate for each of the samples obtained from the original one by omitting one of the values The parameter estimate computed from the sample x 2,, x n (after omitting x 1 ) is denoted byθ 1, the one computed from the sample without x 2 byθ -2,, and the one computed from x 1,x 2,, x n-1 byθ -n 2 The jackknife standard error ofθ is then obtained via the formula SE () = 1 1! 2 + 2! 2 + +! 2 $

23 Bootstrap standard error estimate To estimate a standard error using the bootstrap technique, one proceeds as follows Think of a certain sample with values x 1,x 2,, x n from which the original parameter estimateθ is computed 1 Draw a large number B of bootstrap samples from x 1,x 2,, x n, all of the same size n Recompute the parameter estimate for each of these samples This gives a sequence of bootstrap estimates Θ %,Θ &,,Θ ( 2 Then the bootstrap estimate of the standard error of Θ equals the classical standard deviation of these estimates: SE ) () = 1 ) 1 1! 2 + 2! )! 2 $ = 1 ) * ) +

24 Bootstrap percentile confidence intervals Think of a certain sample with values y 1,y 2,, y n from which the original parameter estimateθ is computed 1 Draw a large number B of bootstrap samples from y 1,y 2,, y n, all of the same size n Recompute the parameter estimate for each of these samples This gives a sequence of estimates Θ %,Θ &,,Θ ( 2 Determine the 25th and the 975th percentiles of these boostrap estimates Let s denote them by P 0025 (Θ, : i=1,,b) and P 0975 (Θ, : i=1,,b) respectively 3 Then the non-parametric 95%-confidence interval for is (P 0025 (Θ, : i=1,,b), P 0975 (Θ, : i=1,,b) ) For instance, if B = 10000, then P 0025 is the 250 th smallest boostrap estimate and P 0975 is the 250 th largest bootstrap estimate

25 Bootstrap distribution of AUC-values of BNP as a predictor of in-hospital cardiovascular events after orthopaedic surgery Density bootstrap replicates 25% AUC 25% observed AUC = 086 SD of bootstrap distribution = 009 Bootstrap confidence intervals a) based on assumption of normality of bootstrap distribution: 086 +/ = (068, 104) b) Percentile method: P25 = 073, P975 = Januar 2014 Präsentationstitel 25

26 Even if there is only one reality which can be described by a specific model, we need to estimate the parameters of this model This implies that we are a faced with an infinity of possible models a priori For simplicity, we will assume in the following that we are only interested in a specific parameter Θ of the true model

27 Assumptions about these possible models A1 The sampling distribution of the parameter estimate Θ is (approximately) normal with a man value of Θ and a standard error SE which is (almost) independent of Θ* In this case, the 95%-confidence interval may be estimated by (Θ 196 SE(Θ), Θ SE(Θ) where SE(Θ) can be estimated in different ways (by explicit formula, jackknife or bootstrap method) * This assumption is generally tenable if the above confidence interval is narrow

28 A2 There exists a order preserving transformation g* such that the sampling distribution of g(θ) is (approximately) normal with a mean value of g(θ) and a standard error SE which is (almost) independent of g(θ)* In this case, the 95%-confidence interval may be estimated by ( P 0025 (Θ, : i=1,,b), P 0975 (Θ, : i=1,,b) ) * Example: g(θ) = ln(θ) and g(θ) = ln(θ), as for Θ = odds ratio or Θ = relative risk

29 A3 There exists a order preserving transformation g such that the sampling distribution of g(θ) can be apprimately described as follows g(θ) = g(θ) + (1 + a g(θ)) (Z z 0 ) where Z has a standard normal distribution (ie, with mean 0 standard deviation 1) In this case, the 95%-confidence interval may be estimated by the so-called BCa-method *bias corrected and accelerated bootstrap (Efron, 1987)

30 If z 0 is different from 0, then the median of the distribution of g(θ) equals g(θ) - (1 + a g(θ)) z 0 which is different from g(θ) In this case, the estimate g(θ) is biased and the confidence intervals given under A1 and A2 would also be biased But the BCa-method can correct this bias The parameter z 0 can be estimated by determining the proportion p of bootstrap estimatesθ,, withθ, <Θ And then z 0 equals the p-quantile of the standard normal distribution The parameter a is more difficult to estimate

31 The three assumptions and associated methods are in a hierarchical order: A1 holds A2 holds A3 holds 95%-CI valid under A1 95%-CI valid under A2 95%-CI valid under A3

32 Coming back to the use of bootstrap in my analyses for your group Confidence ellipses for the true two-dimensional cost-effectiveness parameter pair (Θ 1, Θ 2 ) = (true in mean outcome, true in mean cost) were derived under assumption A1 When doing so, one implicitly assumes that the sampling distribution of the two dimensional parameter estimate (Θ %,Θ & ) = (observed in mean outcome, observed in mean cost) is (about) normally distributed around the true parameter pair (Θ 1, Θ 2 ) with fixed SE(Θ % ) and SE(Θ & ) and fixed correlaton betweenθ % andθ & independent of (Θ 1, Θ 2 )

33 In all other applications, I have used assumption A2 and determined confidence intervals according to the percentile method

34 Diagnostic decision functions

35 T Reichlin et al, One-Hour Rule-out and rule-in of acute myocardial infarction using high-sensitivity cardiac troponin T, Arch Intern Med 2012; 172:1211-8

36 0() = 0() = 1 ( 25) 1+1 ( 25) 1 5 ( 25) ( 25) The function f(x) approximates the diagnostic decision function d(x) = 1 for x > 25 (disease diagnosed) d(x) = 0 for x 25 (disease not diagnosed)

37 Assume that the disease in question is diagnosed if a) either the first diagnostic marker X 1 > 25 or b) the second diagnostic marker X 2 > 10 Then the diagnostic function d(x 1,x 2 ) can be approximated by 2 () = 0 25 ( 1 ) ( 2 ) 0 25 ( 1 ) 0 10 ( 2 ) where 0 4 () = 1 5 ( 4) ( 4)

38 Function table X 1 X 2 f 25 (X 1 ) f 10 (X 2 ) d (X 1,X 2 ) < 25 < = 0 < 25 > = 1 > 25 < = 1 > 25 > = 1 The disease is diagnosed if d (X 1,X 2 ) = 1

39 By means of a non-linear regression model for the parameters a and b 5 6 = 0 7 (8 16 )+0 9 (8 26 ) 0 7 (8 16 ) 0 9 (8 26 ) the parameters a and b can be determined in such a way that the number of observations with Y i d (X 1i,X 2i ),ie, the prediction error rate, is minimized* This provides the rule-in function By giving observations with Y i = 1 much more weight than to observations with Y i = 0, then the number of false predcitions for cases, ie, Y i = 1 despite d (X 1i,X 2i ) = 0, can be minimized and a rule-out decision function can be obtained * at least locally, since there may be different local minima and there is no guarantee that the best local minimum is found

40 Thank you for your attention 23 Januar 2014 Präsentationstitel 40