Distribution-free ROC Analysis Using Binary Regression Techniques

Transcription

1 Distribution-free Analysis Using Binary Techniques Todd A. Alonzo and Margaret S. Pepe As interpreted by: Andrew J. Spieker University of Washington Dept. of Biostatistics Introductory Talk

2 No, not that!

3 This

4 Ability to discern...

5 Ability to discern...

6 (Receiver Operating Characteristic) Medical diagnostic tests: Discriminate between condition and no condition Dichotomous test, often generated from continuous variable True Positive Rate (TPR - Sensitivity ): #{Condition, Positive} #{Condition, Positive} + #{Condition, Negative} False Negative Rate (FPR Specificity ): #{No Condition, Positive} #{No Condition, Positive} + #{No Condition, Negative}

7 (Receiver Operating Characteristic) The Curve : Set of ordered pairs: {(TPR, 1 FPR)}... {(Sensitivity, Specificity)}... taken at all possible cut-points

8 How good is my test?

9 How good is my test?

10 How good is my test?

11 How good is my test?

12 How good is my test?

13 - How is it affected? Of interest to us: How is accuracy of diagnostic test affected by covariates? Example: Childhood BMI to predict adult obesity... Influenced by age at which BMI was measured! (becomes a better predictor with in increasing age) May be a better measure of obesity risk in females than in males This is our motivation for regression.

14 - How is it affected? In particular, let... Y D denote test results from diseased population Y D denote test results from non-diseased population... (presume Y more indicative of disease) X denote covariates common to both diseased and non-diseased population (e.g., age, gender, race) X D denote covariates specific to diseased state (e.g., severity of obesity, stage of prostate cancer)

15 - Methods Existing approaches: models for test outcome; infers covariate effects on corresponding curves... for area under the curve (AUC)... Parametric, distribution-free () approach...

16 - Methods Existing approaches: models for test outcome; infers covariate effects on corresponding curves (Tosteson and Begg, 1985) This is an inherently an all-around parametric approach. Tosteson and Begg note that often, the parametric assumption is that Y Di N (µ D, σ 2 D ) and Y Di N (µ D, σ 2 D))

17 - Methods Existing approaches: for area under the curve (Thompson and Zucchini, 1989) This is also a parametric approach, using the common measure of accuracy, area under the curve (AUC)... While area can be a good measure of diagnostic accuracy, there are certain places one would rather accrue it than others.

18 How good is my test?

19 How good is my test?

20 How good is my test?

21 - Methods Existing approaches: Parametric, distribution-free () approach (Pepe, 1997, 2000) This is the one we will focus on...

22 -... Assumes parametric model for curve but it distribution-free with respect to test itself... Flexible model: Can accommodate continuous covariates Can accommodate the fact that certain models are only relevant to a restricted portion of the curve

23 - Let... F D,X,XD (c) = P(Y D c X, X D ) F D,X (c) = P(Y D c X ) Key Observation: (t) = F D,X,XD ( F 1 D,X (t) ), where t (0, 1) denotes the FPR.

24 - Ordinarily, our regression model is given by: X,XD (t) = Φ ( γ 1 + γ 2 Φ 1 (t) + βx + β D X D ) (binormal model; curves for different values of X differ on the probit scale, although other link functions can be chosen)... that s the parametric part!

25 - Can fit this regression model based on binary indicators: U ij = 1(Y Di F 1 D,X i (t)) for each i, t Key Observation: E[U it ] = Xi,X Di (t)... that s the distribution-free part!

26 - Algorithm to estimate parameters of interest: Specify a set of FPRs, T For each t T, estimate F D,X 1 i Compute U it = 1(Y Di ˆF D,X 1 (t)) for each i and each t i Fit the model E[U it ] = Φ(γ 1 + γ 2 Φ 1 (t) + βx + β D X D ) using standard GLM procedures

27 - Symmetrized Fitting Procedure: (t) = Φ ( α (1) + β (1) Φ 1 (t) ) t, where t is the FPR On the other hand, (s) = Φ ( α (2) + β (2) Φ 1 (s) ), where s is the TPR How to translate between two wolds: α (2) = α (1) /β (1) ; β (2) = 1/β (2) Estimate (α, β) using each of these models (optimally weighted) In practice, use bootstrap to estimate optimal weights

28 - Symmetrized Fitting Procedure: ˆα = (σ2 1 + σ 12)ˆα (1) + (σ σ 12)ˆα (2) σ σ σ 12 ˆβ = (τ τ 12) ˆβ (1) + (τ τ 12) ˆβ (2) τ τ τ 12

29 Next time! Statistical efficiency is of interest to us. Via simulation, we will Compare with symmetrized approach Compare with maximum likelihood Investigate the choice of the false positive rate set, T. Practically speaking... Would like an application to obesity example