Constrained estimation for binary and survival data

Size: px

Start display at page:

Download "Constrained estimation for binary and survival data"

Erick Wilcox
5 years ago
Views:

1 Constrained estimation for binary and survival data Jeremy M. G. Taylor Yong Seok Park John D. Kalbfleisch Biostatistics, University of Michigan May, 2010 () Constrained estimation May, / 43

2 Outline Motivation Two Binomial probabilities, p 1 p 2 Survival functions, S 1 (t) S 2 (t) () Constrained estimation May, / 43

3 Motivation Challenges What estimator to use? General Approaches Restricted MLE Isotonic regression Pooled adjacent violators algorithm Bayesian: Impose restriction through prior distribution Inference: Confidence intervals () Constrained estimation May, / 43

4 Motivation Motivation New cancer treatment. Drug 3 levels, d 1 < d 2 < d 3 Possible toxic side effects p j = P (T oxicity d j ) Know p 1 p 2 p 3 Utilize this information in the analysis Data Y 1 Binomial(n 1, p 1 ) Y 2 Binomial(n 2, p 2 ) Y 3 Binomial(n 3, p 3 ) Want ˆp 1 ˆp 2 ˆp 3 Why Gain efficiency, e.g. n 1 = 15, n 2 = 3, n 3 = 14 Consistent with truth () Constrained estimation May, / 43

5 Binomial: Two groups Restricted MLE for two binomial probabilities Y j Binomial(n j, p j ) p 1 p 2 restricted MLE is given by ˆp 1n = min {d 1 /n 1, (d 1 + d 2 )/(n 1 + n 2 )} ˆp 2n = max {d 2 /n 2, (d 1 + d 2 )/(n 1 + n 2 )}. Construction of confidence intervals is difficult if p 1 is close to p 2 Inference is difficult near or on boundary of parameter space () Constrained estimation May, / 43

6 Binomial: Two groups Simulation results: Biases and Efficiency Table: Restricted MLE and the unrestricted MLE ( n 1 = 50, n 2 = 100). Restricted MLE p 1 p 2 bias p 1 = 0.5, p 2 = p 1 = 0.5, p 2 = p 1 = 0.5, p 2 = p 1 = 0.5, p 2 = Efficiency: Var(Restricted)/Var(Unrestricted) p 1 = 0.5, p 2 = p 1 = 0.5, p 2 = p 1 = 0.5, p 2 = p 1 = 0.5, p 2 = () Constrained estimation May, / 43

7 Binomial: Two groups Theorem 0: CLT. Suppose that p 1 < p 2. Then nj (ˆp jn p j ) d N(0, p j (1 p j )). Theorem 1. Suppose that p 1 = p 2, lim n n 2 /n 1 = c, and 0 < c <. Then { } 1 c n1 (ˆp 1n p 1 ) d min W 1, 1 + c W c W 2, and { c n2 (ˆp 2n p 2 ) d max W 2, 1 + c W 1 + c } 1 + c W 2, as n, where W 1 and W 2 are independent and identically distributed as N(0, p 1 (1 p 1 )). Asymptotic results not useful or accurate for small n () Constrained estimation May, / 43

8 Binomial: Two groups Theorem 2. Suppose that p 2 = p 1 + / n 1, lim n n 2 /n 1 = c, and 0 < c <. We have, when p 1 = p 2, ( 1 c n1 (ˆp 1n p 1 ) d min W 1, 1 + c W c W 2 + c ) 1 + c, and n2 (ˆp 2n p 2 ) d max ( W 2, c 1 + c W 1 + c 1 + c W 2 ) c 1 + c, as n, where W 1 and W 2 are independent with distribution N(0, p 1 (1 p 1 )). Confidence intervals don t have good coverage rates () Constrained estimation May, / 43

9 Binomial: Two groups Bootstrap Confidence Intervals Group 1, n 1 observations, (0,1,1,0,1,...,0) Group 2, n 2 observations, (1,1,0,0,0,...,1) Resample within groups Bootstrap percentile confidence intervals Coverage rates good at moderate sample sizes Coverage rates OK at small sample sizes () Constrained estimation May, / 43

10 Binomial: Two groups Table: Simulation: Coverage rates of 95% confidence intervals n 1 = 50, n 2 = 100 p 1 = 0.5 p 2 = 0.5 p 2 = 0.52 p 2 = 0.7 p 2 = 0.9 Restricted MLE p Theorem 2 p percentile bootstrap CI p based on restricted MLE p p 1 = 0.8 p 2 = 0.8 p 2 = 0.82 p 2 = 0.85 p 2 = 0.9 Restricted MLE p Theorem 2 p percentile bootstrap CI p based on restricted MLE p () Constrained estimation May, / 43

11 Survival functions Estimation of Survival Functions () Constrained estimation May, / 43

12 Survival functions Stochastic Ordering Survival Function S(t) = P r(t > t) Definition of Stochastic Ordering: T 1 st T 2 if P r(t 1 > t) P r(t 2 > t) for t R One-sample Case: Estimation of S 1 (t) Bounded Below: S 1 (t) S 2 (t), where S 2 (t) is known; Bounded Above: S 1 (t) S 2 (t), where S 2 (t) is known. Two-sample Case: S 1 (t) S 2 (t), S 1 (t) and S 2 (t) are unknown. () Constrained estimation May, / 43

13 Survival functions Motivation - Survival Analysis in Cancer Study Pr(T>t) t (years) Stage 1 Stage 2 Figure: Kaplan-Meier plots of larynx cancer patients(kardaun, 1983) () Constrained estimation May, / 43

14 Survival functions Motivation - Constrained Estimator Pr(T>t) t (years) Stage 1 Stage 2 Figure: Constrained NPMLE () Constrained estimation May, / 43

15 Survival functions Motivation - Cont. Wide Range of Applications. biomedical research; engineering sciences; economics; software reliability. Estimators from separate samples may not satisfy constraint random variation; small sample size; Constrained Estimator Potential to gain efficiency Realistic estimate () Constrained estimation May, / 43

16 Survival functions Literature C-NPMLE: Two-sample case without censoring. Brunk et al. (1966). C-NPMLE: One- & two-sample with right censoring. Dykstra (1982) - Correct in Bounded Below Case Some possible outcomes were not properly handled. May not be C-NPMLE in bounded above and two-sample case. Alternative: One-sample case. Puri and Singh (1992); Rojo and Ma (1996). Alternative: Two-sample case. Lo (1987) - swapping estimates if violated; Rojo (2004) - averaging estimates if violated; Park et al (2010) - pointwise constrained MLE. () Constrained estimation May, / 43

17 Survival functions One sample, No constraints NPMLE: Kaplan-Meier estimator. Product limit estimator. Distribution is discrete. Jumps at the event times. h i = log[s(t i )/S(t i 1 )] Discrete hazard = 1 exp(h i ) S(t i ) = exp[ i j=1 h j] d i = number of events at time t i n i = number at risk at time t i The NPMLE of S( ) is given by log(1 d i ) d i > 0 ĥ i = n i 0 d i = 0 () Constrained estimation May, / 43

18 Method: One-sample Bounded Above One-sample Bounded Above () Constrained estimation May, / 43

19 Method: One-sample Bounded Above Problem Data (Y 1i, 1i ), i = 1,, n; 1i = 1 if event occurred or 1i = 0 if right censored Goal Estimate S 1 (t) under S 1 (t) S 2 (t). Likelihood L = n i=1 [S 1(Y 1i ) S 1 (Y 1i )] 1i S 1 (Y 1i ) 1 1i Discrete Case: L = m j=1 [S 1(a j 1 ) S 1 (a j )] d 1i S 1 (a j ) c 1i () Constrained estimation May, / 43

20 Method: One-sample Bounded Above Definitions C-NPMLE Constrained Nonparametric MLE: nonparametric estimator that maximizes the likelihood amongst those that satisfy the constraint. C-NPMLE may not be unique. MC-NPMLE Maximum C-NPMLE, which is C-NPMLE that maximizes the estimate of the survivor function in the class of all C-NPMLE. () Constrained estimation May, / 43

21 Method: One-sample Bounded Above Theorem Theorem: Bounded Above For S 1 ( ) and S 2 ( ) discrete the MC-NPMLE of S 1 ( ) is given by log(1 d 1i n 1i ˆk ) d i 1i > 0 ĥ 1i = [ i i 1 ] min 0, h 2j ĥ 1j d 1i = 0 j=1 and ˆk i = min a i max b i min(k (a, b), n 1b ), where (Dykstra 1982: ˆk i = min a i max b i K (a, b)) K (a, b) = max{0, K(a, b)} and K (a, b) is the solution of b j=a log(1 d 1j n 1j +k ) b j=a h 2j = 0. j=1 () Constrained estimation May, / 43

22 Method: One-sample Bounded Above Algorithm Algorithm: Bounded Above 1 Set i0 = 0, l = 1, m = max(i : n 1i > 0). 2 Let il = min b>il 1 {b : H(i l 1 + 1, b, 0) > 0}. If no such i l exists, set i l = m and k l = 0 and go to step 6, otherwise go to step 3. 3 If d1il = 0 and H(i l 1 + 1, i l, n 1il ) 0, then set k l = n 1il and go to step 5, otherwise set k l = K(i l 1 + 1, i l ) and go to step 4. 4 Let I = minb>il {b : n 1b > k l and H(i l + 1, b, k l ) > 0}. If no such I exists, then go to step 5. Otherwise, set i l = I and go to step 3. 5 Let ĥ 1j = log[1 d 1j /(n 1j k l )], i l j i l 1 ĥ 1il = { log[1 d1il /(n 1il k l )], if k l < n 1il il j=il 1 +1 ĥ2j i l 1 j=i l 1 +1 ĥ1j, if k l = n 1il 6 If il = m, stop. Otherwise, set l = l + 1 and go to step 2. () Constrained estimation May, / 43

23 Method: One-sample Bounded Above Algorithm Proof that Algorithm gives MC-NPMLE Constrained optimization problem Maximize likelihood subject to some constraints Max log(l(h 1,..., h k ) s.t. S 1 (t) S 2 (t), h j 0 Kuhn-Tucker conditions Lagrange multipliers () Constrained estimation May, / 43

24 Method: One-sample Bounded Above Example Example: Bounded Above Pr(T>t) t Upper Bound Kaplan Meier( 10.65) MC NPMLE( 12.4) Dykstra( 13.22) () Constrained estimation May, / 43

25 Extension: continuous constraint function One-sample Bounded Above with Continuous Constraint () Constrained estimation May, / 43

26 Extension: continuous constraint function Example Example Pr(T>t) t l Upper Bound Kaplan Meier( 10.65) () Constrained estimation May, / 43

27 Extension: continuous constraint function Naïve Method Naïve Method limit approaching Use the limit of a discrete function to approach a continuous one; For example Choose R evenly spaced times between 0 and max(y 1i ) as potential death times and obtain the limiting estimate of Ŝ1(t) with discrete method as R goes to infinity; Drawback Computationally intensive. () Constrained estimation May, / 43

28 Extension: continuous constraint function Naïve Method 12 potential event times Pr(T>t) t l Upper Bound Kaplan Meier( 10.65) MC NPMLE( 12.4) () Constrained estimation May, / 43

29 Extension: continuous constraint function Naïve Method 36 potential event times Pr(T>t) t l Upper Bound Kaplan Meier( 10.65) MC NPMLE( 12.82) () Constrained estimation May, / 43

30 Extension: continuous constraint function Naïve Method 360 potential event times Pr(T>t) t l Upper Bound Kaplan Meier( 10.65) MC NPMLE( 13.02) () Constrained estimation May, / 43

31 Extension: continuous constraint function Simple Algorithm Simple algorithm Let C i, i = 1,, n c be all distinct observed censoring times and let X i be the time just before observed death time X i. 1 Let X i, i = 1, 2,, n tot be the distinct ordered set of times from the union of X i, X i and C i ; 2 Estimate Ŝ 1 (t), which is the MC-NPMLE with potential death times at X i, i = 1,, n tot; 3 Let S1 (t) = min(ŝ1(t), S 2 (t)), which is the MC-NPMLE of S 1 (t) subject to S 1 (t) S 2 (t) for t > 0. () Constrained estimation May, / 43

32 Extension: continuous constraint function Simple Algorithm Simple Algorithm in Example Pr(T>t) t Upper Bound l Kaplan Meier( 10.65) S^ 1( 13.03) MC NPMLE( 13.03) () Constrained estimation May, / 43

33 Two-sample Case Two-sample Case () Constrained estimation May, / 43

34 Two-sample Case Problem Problem - Two sample case Data (Y gi, gi ), g = 1, 2, i = 1,, n g ; gi = 1 if event occurred or gi = 0 if right censored Goal Estimate S 1 (t), S 2 (t) under S 1 (t) S 2 (t). Likelihood L = 2 g=1 ng i=1 [S g(y gi ) S g (Y gi )] gi S g (Y gi ) 1 gi Discrete Case: L = 2 g=1 { m j=1 [S g(a j 1 ) S g (a j )] d gi S g (a j ) c gi } () Constrained estimation May, / 43

35 Two-sample Case Theorem Theorem for two-sample case The C-NPMLE of S 1 ( ) and the MC-NPMLE of S 2 ( ) are given by S 1 (t) = exp( a i t h 1i) and S 2 (t) = exp( a i t h 2i), where ĥ 1i = log(1 d 1i n 1i + ˆk ) i log(1 d 2i n 2i ˆk ) d i 2i > 0 ĥ 2i = [ i i 1 ] min 0, h 1j ĥ 2j d 2i = 0 j=1 j=1 and ˆk i = min a i max b i min(k 2 + (a, b), n 2b), (Dykstra 1982: ˆk i = min a i max b i K 2 + (a, b)) where K 2 + (a, b) = max(k 2(a, b), 0) and K 2 (a, b) is the solution of b j=a (log(1 d 1j n 1j +k ) = b j=a (log(1 d 2j n 2j k ). () Constrained estimation May, / 43

36 Two-sample Case Example Example - Two sample Pr(T>t) t KM 1 KM 2 () Constrained estimation May, / 43

37 Two-sample Case Example Example - C-NPMLE, Dykstra Pr(T>t) KM 1 KM 2 ( ) C NPMLE 1 MC NPMLE 2 ( ) Dykstra 1 Dykstra 2 ( ) t () Constrained estimation May, / 43

38 Simulation Simulation in Two-sample Case () Constrained estimation May, / 43

39 Simulation Simulation - Two-sample case Finite sample property MSE = (Ŝ(t) S(t))2 ; Pointwise criteria Event distributions and sample sizes S 1 (t) = exp( t), n 1 = 100; S 2 (t) = exp( 1.2t), n 2 = 40. Scenarios 1 Same censoring: S c 1 (t) = S2 c (t) = exp( 1.5t); 2 Excessive censoring 1: S c 1 (t) = exp( 3t), S2 c (t) = 1; 3 Excessive censoring 2: S c 1 (t) = 1, S2 c (t) = exp( 3t). () Constrained estimation May, / 43

40 Simulation Simulation - estimators in comparison 1 C-NPMLE from this paper: 2 Dykstra (1982): 3 Lo (1987): Ŝ L 1 (t) = max(s 1 (t), S 2 (t)) Ŝ L 2 (t) = min(s 1 (t), S 2 (t)); 4 Rojo (2004): Ŝ 1 R(t) = max(s 1 (t), n 1S1 (t)+n 2S2 (t) n 1 +n 2 ) Ŝ2 R(t) = min( n 1S1 (t)+n 2S2 (t), S2 (t)); n 1 +n 2 5 Park et al (2010): PC-NPMLE (pointwise C-NPMLE) Ŝ P 1 (t) = S 1 (t; t), ŜP 2 (t) = S 2 (t; t) where S 1 (t; x) and S 2 (t; x) are the MLE subject to S 1 (x) S 2 (x) at fixed time x. () Constrained estimation May, / 43

41 Simulation Pointwise C-NPMLE Fix a time x of interest Find NPMLE Ŝ1(t) and Ŝ2(t) such that Ŝ1(x) Ŝ2(x) This gives Ŝ1(t) and Ŝ2(t) at t = x Repeat for all x () Constrained estimation May, / 43

42 Simulation Simulation - same censoring distributions RMSE C NPMLE Rojo Dykstra Lo PC MLE RMSE C NPMLE Rojo Dykstra Lo PC MLE S 1 (t) S 2 (t) Figure: Same censoring distributions () Constrained estimation May, / 43

43 Simulation Simulation - different censoring dist n RMSE C NPMLE Rojo Dykstra Lo PC MLE RMSE C NPMLE Rojo Dykstra Lo PC MLE S 1 (t) S 2 (t) Figure: Excessive censoring group 1 () Constrained estimation May, / 43

44 Simulation Simulation - different censoring dist n RMSE C NPMLE Rojo Dykstra Lo PC MLE RMSE C NPMLE Rojo Dykstra Lo PC MLE S 1 (t) S 2 (t) Figure: Excessive censoring group 2 () Constrained estimation May, / 43

45 Conclusion and Related Problems Conclusion 1 Developed methods to obtain the C-NPMLE in the one- and two-sample cases; Including a correction of Dykstra s(1982) estimator and computationally efficient algorithms; 2 Developed a simple method to obtain the MC-NPMLE in the one-sample situation with a bounded above constraint when the constraint survivor function is continuous; 3 C-NPMLE is better than Dykstra s estimator; C-NPMLE and Rojo s estimator outperform each other at different situations; Pointwise C-NPMLE performs better in all cases considered. () Constrained estimation May, / 43

46 Conclusion and Related Problems Related Problems 1 3 groups. S1 (t) S 2 (t) S 3 (t) 2 4 groups. S1 (t) S 2 (t) S 4 (t) and S 1 (t) S 3 (t) S 4 (t) 3 Inference: Confidence Intervals () Constrained estimation May, / 43

Statistics 262: Intermediate Biostatistics Non-parametric Survival Analysis

Statistics 262: Intermediate Biostatistics Non-parametric Survival Analysis Jonathan Taylor & Kristin Cobb Statistics 262: Intermediate Biostatistics p.1/?? Overview of today s class Kaplan-Meier Curve