ST495: Survival Analysis: Hypothesis testing and confidence intervals Eric B. Laber Department of Statistics, North Carolina State University April 3, 2014
I remember that one fateful day when Coach took me aside. I knew what was coming. You don t have to tell me, I said. I m off the team, aren t I? Well, said Coach, you never were really ON the team. You made that uniform you re wearing out of rags and towels, and your helmet is a toy space helmet. You show up at practice and then either steal the ball and make us chase you to get it back, or you try to tackle people at inappropriate times. It was all true what he was saying. And yet, I thought something is brewing inside the head of this Coach. He sees something in me, some kind of raw talent that he can mold. But that s when I felt the handcuffs go on. Philip Glass
Then and now Last time we discussed graphical methods QQ-plots PP-plots Linearized plots Today we ll discuss Who is Philip Glass Inference with parametric models Prediction intervals
Warm-up Explain to your stat buddy 1. What is a likelihood ratio test? 2. How can you invert a hypothesis test to form a confidence interval? 3. What is a prediction interval? How does this difference from a confidence interval? 4. Who was Philip Glass? True or false: (T/F) A likelihood ratio test is uniformly most powerful for simple hypotheses? (T/F) The ratio of two (scaled) independent chi-squared random variables has an F-distribution (T/F) The most expensive Donkey cheese in the world sells for several thousand dollars a pound
Inference and prediction intervals Quantities estimated from data should (must!) be accompanied by measures of uncertainty Hypothesis tests and confidence intervals regard population-level parameters, assess uncertainty and strength of evidence Prediction intervals reflect both estimation uncertainty and variability in the generative distribution
Shaking off the rust Suppose T 1,..., T n are i.i.d. with density f (t; θ) Inference and hypothesis testing typically concern θ Prediction regards uncertainty in a new draw Tn+1 from f (t; θ) Suppose X 1,..., X n are i.i.d. N(µ, 1), with your stat buddy: 1. Give a 95% confidence interval for µ 2. Construct a test of H 0 : µ = µ 0 vs. H a : µ µ 0 with α Type I error 3. Give a 95% prediction interval for X n+1
Confidence intervals Let {(T i, δ i )} n i=1 denote censored observation times, survival distn f (t; θ) ML approach for CIS: Construct MLE θ n Construct observed fisher information I ( θ) Use asymptotically normality of MLE I 1/2 ( θ n )( θ n θ) N(0, 1), to obtain θ n ± z 1 α/2 I 1/2 ( θ n )
The example you love to hate Let {(T i, δ i )} n i=1 denote observation times subject to non-informative right-censoring Assume the true survival distribution is exponential with density f (t; θ) = θ 1 exp t/θ construct a 95% CI for θ Answer: on board Problem! The log-lh is not approximated well by a quadratic in small-samples when the amount of censoring is large Recall precision varies inversely with number of uncensored observations (Why?) Normal approximation may be poor
The example you love to hate cont d Local approximation of log-lh by a quadratic, T exp(1) Log LH with n=100, Σδ i = 10 Log LH with n=100, Σδ i = 25 Log LH with n=100, Σδ i = 50 l(θ) 200 150 100 50 l(θ) 30 28 26 24 22 l(θ) 53 52 51 50 49 48 47 0.0 0.5 1.0 1.5 2.0 θ 0.4 0.6 0.8 1.0 1.2 1.4 1.6 θ 0.6 0.8 1.0 1.2 1.4 θ
Inverting a test: review Consider testing the null H 0 : θ = θ 0 vs. H a : θ θ 0 Let T n (θ 0 ) be a test statistic and c 1 α an α-level critical value so that the test rejects when T n (θ 0 ) > c 1 α E.g., H0 : µ = µ 0 vs. H a : µ µ 0 and T n (µ 0 ) = n( X n µ 0 ) 2 / σ 2, and c 1 α = χ 2 n 1,1 α If θ is the true value of θ then P (T n (θ ) c 1 α ) 1 α, thus {θ : T n (θ) c 1 α } is a valid (1 α) 100% CI for θ (Why?)
Inverting a test: likelihood Likelihood ratio statistic is Λ(θ) = 2l( θ) 2l(θ) If θ is the true parameter value then Λ(θ) is asymptotically χ 2 p where p = dim θ (1 α) 100% LH ratio CI for θ { θ : Λ(θ) χ 2 p,1 α } Often preferred to the standard MLE interval in small samples
LH ratio CI for an exponential RV {(T i, δ i )} n i=1 observation times subj to non-informative right-censoring, assume f (t; θ) = θ 1 exp{ t/θ} With your stat buddy: compute a LR CI for θ
LH ratio CI for an exponential RV cont d See lrtci.r
Sprott s adjustment Sprott showed that the transformation φ = θ 3 has a symmetrizing effect on the log-lh Idea: Construct usual ML based interval for φ using I 1/2 1 ( φ)( φ φ) N(0, 1), where I 1 ( φ) is the observed Fisher info for φ, then solve for a CI for θ This method performs similarly to the LRT and does not extend generally so we will not consider it further
Exact methods the exponential distribution Under certain censoring schemes exact confidence intervals are possible Type II censoring Testing with replacement These are somewhat specialized examples but you should know they exist as they are useful when applicable (see pp. 153-154)
Likelihood ratio test Goal is to test null hypothesis H 0 Restricted MLE is θ = arg max θ satisfies H0 L(θ) Ex. X 1,..., X n N(µ, σ 2 ) compute the restricted MLE under H 0 : µ = 1 Ex. X 1,..., X n N(µ, σ 2 ) compute the restricted MLE under H 0 : µ = σ LRT statistic Λ = 2l( θ) 2l( θ), when H 0 is true Λ χ 2 d where d is difference in the degrees of freedom between restricted and unrestricted models
LRT example Suppose we observe right-censored failure times from m different groups that we believe have exponential lifetime distns. Observed data {(Tij, δ ij )}, j = 1,..., n i, i = 1,..., m Want to test H0 : θ 1 = = θ m With your stat buddy: derive unrestricted and restricted MLEs Test statistic Λ = 2l( θ) 2l( θ) reject H0 if Λ > χ 2 m 1,1 α (Why?)
LRT example cont d See lrtfourmachines.r
Other distributions and censoring mechanisms As long as we have a likelihood we can apply these methods The book gives additional details for other distributions Gamma, Inverse-gaussian In more general settings numerical methods must be used to maximize the LH Since we know how to compute the LH for other censoring mechanisms (e.g., left, interval) we can apply the proposed methods directly Don t forget about bootstrap CIs! 1 1 Bootstrap tests are also possible but we won t discuss them in this class. See Bickel and Ren for info.
Prediction intervals Confidence intervals quantify uncertainty in estimated coefficients, the do not include uncertainty about future observations Give prognosis to newly diagnosed patient Predict failure time of new manufacturing process Implementing redundancies...
Prediction intervals cont d Let X 1,..., X n, X n+1 be iid, a (1 α) level prediction interval for X n+1 based on X 1,..., X n are functions L(X 1,..., X n ) and U(X 1,..., X N ) so that P {L(X 1,..., X n ) X n+1 U(X 1,..., X n )} 1 α E.g., in the warm-up example X 1,..., X n, X n+1 iid N(µ, 1), L(X 1,..., X n ) = X n z 1 α/2 1 + 1/n, and U(X 1,..., X n ) = X n + z 1 α/2 1 + 1/n
Pivots G(X 1,..., X n, X n+1 ) that does not depend on θ is called pivot X n X n+1 is a pivot the N(µ, 1) example Solve for a, b such that P (a G(X 1,..., X n, X n+1 ) b) 1 α, then {x : G(X 1,..., X n, x) b} is (1 α) 100% PI In survival problems obtaining a pivot can be difficult, instead: Plug-in intervals Projection intervals Calibrate the interval using simulation
Plug-in intervals Compute θ and take percentiles of distn f (t; θ) Asymptotically correct when θ is consistent Ignores uncertainty in estimated θ Ex. {(T i, δ i )} n i=1 censored lifetimes assumed exp(θ) Solve S(t; θ) = 1 p for t to obtain t p (θ) = log(1 p)θ 2 Approximate ( prediction interval) log(1 α/2) θ, log(α/2) θ See approxatepi.r 2 Here we re using f (t; θ) = θ 1 exp{ t/θ}
Projection intervals Plugin PIs ignore uncertainty in θ and thus may be anti-conservative Projection intervals incorporate uncertainty in θ but can be conservative Let C 1 α (θ 0 ) be a (1 α) 100% plugin PI using assumed parameter θ 0 Let ζ 1 η be a (1 η) 100% confidence region for θ Projection interval is a valid 1 α η PI (Why?) θ 0 ζ 1 η C 1 α (θ 0 ),
Calibration Investigate the performance of PI procedure view α as tuning parameter Choose α so that the coverage is near nominal levels in simulated examples