Maximum likelihood: counterexamples, examples, and open problems. Jon A. Wellner. University of Washington. Maximum likelihood: p.
|
|
- Aileen Todd
- 5 years ago
- Views:
Transcription
1 Maximum likelihood: counterexamples, examples, and open problems Jon A. Wellner University of Washington Maximum likelihood: p. 1/7
2 Talk at University of Idaho, Department of Mathematics, September 15, http: // Maximum likelihood: p. 2/7
3 Outline Introduction: maximum likelihood estimation Maximum likelihood: p. 3/7
4 Outline Introduction: maximum likelihood estimation Counterexamples Maximum likelihood: p. 3/7
5 Outline Introduction: maximum likelihood estimation Counterexamples Beyond consistency: rates and distributions Maximum likelihood: p. 3/7
6 Outline Introduction: maximum likelihood estimation Counterexamples Beyond consistency: rates and distributions Positive examples Maximum likelihood: p. 3/7
7 Outline Introduction: maximum likelihood estimation Counterexamples Beyond consistency: rates and distributions Positive examples Problems and challenges Maximum likelihood: p. 3/7
8 1. Introduction: maximum likelihood estimation Setting 1: dominated families Maximum likelihood: p. 4/7
9 1. Introduction: maximum likelihood estimation Setting 1: dominated families Suppose that X 1,...,X n are i.i.d. with density p θ0 respect to some dominating measure µ where p θ0 P= {p θ : θ Θ} for Θ R d. with Maximum likelihood: p. 4/7
10 1. Introduction: maximum likelihood estimation Setting 1: dominated families Suppose that X 1,...,X n are i.i.d. with density p θ0 respect to some dominating measure µ where p θ0 P= {p θ : θ Θ} for Θ R d. The likelihood is with L n (θ) = n p θ (X i ). i=1 Maximum likelihood: p. 4/7
11 1. Introduction: maximum likelihood estimation Setting 1: dominated families Suppose that X 1,...,X n are i.i.d. with density p θ0 respect to some dominating measure µ where p θ0 P= {p θ : θ Θ} for Θ R d. The likelihood is with L n (θ) = n p θ (X i ). i=1 Definition: A Maximum Likelihood Estimator (or MLE) of θ0 is any value ˆθ Θ satisfying L n (ˆθ) =supl n (θ). θ Θ Maximum likelihood: p. 4/7
12 Equivalently, the MLE ˆθ maximizes the log-likelihood log L n (θ) = n log p θ (X i ). i=1 Maximum likelihood: p. 5/7
13 Example 1. Exponential (θ). X1,...,X n are i.i.d. p θ0 where p θ (x) =θ exp( θx)1 [0, ) (x). Maximum likelihood: p. 6/7
14 Example 1. Exponential (θ). X1,...,X n are i.i.d. p θ0 where Then the likelihood is p θ (x) =θ exp( θx)1 [0, ) (x). L n (θ) =θ n exp( θ n X i ), 1 Maximum likelihood: p. 6/7
15 Example 1. Exponential (θ). X1,...,X n are i.i.d. p θ0 where Then the likelihood is p θ (x) =θ exp( θx)1 [0, ) (x). L n (θ) =θ n exp( θ n X i ), 1 so the log-likelihood is log L n (θ) =n log(θ) θ n 1 X i Maximum likelihood: p. 6/7
16 Example 1. Exponential (θ). X1,...,X n are i.i.d. p θ0 where Then the likelihood is p θ (x) =θ exp( θx)1 [0, ) (x). L n (θ) =θ n exp( θ n X i ), 1 so the log-likelihood is log L n (θ) =n log(θ) θ n 1 X i and ˆθ n =1/X n. Maximum likelihood: p. 6/7
17 0.4 1/n power of likelihood, n = Maximum likelihood: p. 7/7
18 1/n times log-likelihood, n = Maximum likelihood: p. 8/7
19 MLE pˆθ(x) and true density p θ0 (x) Maximum likelihood: p. 9/7
20 Example 2. Monotone decreasing densities on (0, ). X 1,...,X n are i.i.d. p 0 Pwhere P = all nonincreasing densities on (0, ). Maximum likelihood: p. 10/7
21 Example 2. Monotone decreasing densities on (0, ). X 1,...,X n are i.i.d. p 0 Pwhere P = all nonincreasing densities on (0, ). Then the likelihood is L n (p) = n i=1 p(x i); Maximum likelihood: p. 10/7
22 Example 2. Monotone decreasing densities on (0, ). X 1,...,X n are i.i.d. p 0 Pwhere P = all nonincreasing densities on (0, ). Then the likelihood is L n (p) = n i=1 p(x i); L n (p) is maximized by the Grenander estimator: ˆp n (x) = left derivative at x of the Least Concave Majorant C n of F n where F n (x) =n 1 n i=1 1{X i x} Maximum likelihood: p. 10/7
23 Maximum likelihood: p. 11/7
24 4 Figure 5: Grenander Estimator, n = Maximum likelihood: p. 12/7
25 Maximum likelihood: p. 13/7
26 2 1.5 Figure 5: Grenander Estimator, n = Maximum likelihood: p. 14/7
27 Setting 2: non-dominated families Maximum likelihood: p. 15/7
28 Setting 2: non-dominated families Suppose that X 1,...,X n are i.i.d. P 0 Pwhere P is some collection of probability measures on a measurable space (X, A). Maximum likelihood: p. 15/7
29 Setting 2: non-dominated families Suppose that X 1,...,X n are i.i.d. P 0 Pwhere P is some collection of probability measures on a measurable space (X, A). If P {x} denotes the measure under P of the one-point set {x}, the likelihood of X 1,...,X n is defined to be L n (P )= n P {X i }. i=1 Maximum likelihood: p. 15/7
30 Setting 2: non-dominated families Suppose that X 1,...,X n are i.i.d. P 0 Pwhere P is some collection of probability measures on a measurable space (X, A). If P {x} denotes the measure under P of the one-point set {x}, the likelihood of X 1,...,X n is defined to be L n (P )= n P {X i }. i=1 Then a Maximum Likelihood Estimator (or MLE) of P 0 can be defined as a measure ˆP n Pthat maximizes L n (P ); thus L n ( ˆP )=supl n (P ) P P Maximum likelihood: p. 15/7
31 Example 3. (Empirical measure) Maximum likelihood: p. 16/7
32 Example 3. (Empirical measure) If P =all probability measures on (X, A), then n where δ x (A) =1 A (x). ˆP n = 1 n i=1 δ Xi P n Maximum likelihood: p. 16/7
33 Example 3. (Empirical measure) If P =all probability measures on (X, A), then where δ x (A) =1 A (x). Thus ˆP n = 1 n n i=1 δ Xi P n ˆP n (A) = 1 n = 1 n n δ Xi (A) i=1 n i=1 1 A (X i )= #{1 i n : X i A} n Maximum likelihood: p. 16/7
34 Consistency of the MLE: Wald (1949) Maximum likelihood: p. 17/7
35 Consistency of the MLE: Wald (1949) Kiefer and Wolfowitz (1956) Maximum likelihood: p. 17/7
36 Consistency of the MLE: Wald (1949) Kiefer and Wolfowitz (1956) Huber (1967) Maximum likelihood: p. 17/7
37 Consistency of the MLE: Wald (1949) Kiefer and Wolfowitz (1956) Huber (1967) Perlman (1972) Maximum likelihood: p. 17/7
38 Consistency of the MLE: Wald (1949) Kiefer and Wolfowitz (1956) Huber (1967) Perlman (1972) Wang (1985) Maximum likelihood: p. 17/7
39 Consistency of the MLE: Wald (1949) Kiefer and Wolfowitz (1956) Huber (1967) Perlman (1972) Wang (1985) van de Geer (1993) Maximum likelihood: p. 17/7
40 Counterexamples: Neyman and Scott (1948) Maximum likelihood: p. 18/7
41 Counterexamples: Neyman and Scott (1948) Bahadur (1958) Maximum likelihood: p. 18/7
42 Counterexamples: Neyman and Scott (1948) Bahadur (1958) Ferguson (1982) Maximum likelihood: p. 18/7
43 Counterexamples: Neyman and Scott (1948) Bahadur (1958) Ferguson (1982) LeCam (1975), (1990) Maximum likelihood: p. 18/7
44 Counterexamples: Neyman and Scott (1948) Bahadur (1958) Ferguson (1982) LeCam (1975), (1990) Barlow, Bartholomew, Bremner, and Brunk (4B s) (1972) Maximum likelihood: p. 18/7
45 Counterexamples: Neyman and Scott (1948) Bahadur (1958) Ferguson (1982) LeCam (1975), (1990) Barlow, Bartholomew, Bremner, and Brunk (4B s) (1972) Boyles, Marshall, and Proschan (1985) Maximum likelihood: p. 18/7
46 Counterexamples: Neyman and Scott (1948) Bahadur (1958) Ferguson (1982) LeCam (1975), (1990) Barlow, Bartholomew, Bremner, and Brunk (4B s) (1972) Boyles, Marshall, and Proschan (1985) bivariate right censoring Tsai, van der Laan, Pruitt Maximum likelihood: p. 18/7
47 Counterexamples: Neyman and Scott (1948) Bahadur (1958) Ferguson (1982) LeCam (1975), (1990) Barlow, Bartholomew, Bremner, and Brunk (4B s) (1972) Boyles, Marshall, and Proschan (1985) bivariate right censoring Tsai, van der Laan, Pruitt left truncation and interval censoring Chappell and Pan (1999) Maximum likelihood: p. 18/7
48 Counterexamples: Neyman and Scott (1948) Bahadur (1958) Ferguson (1982) LeCam (1975), (1990) Barlow, Bartholomew, Bremner, and Brunk (4B s) (1972) Boyles, Marshall, and Proschan (1985) bivariate right censoring Tsai, van der Laan, Pruitt left truncation and interval censoring Chappell and Pan (1999) Maathuis and Wellner (2005) Maximum likelihood: p. 18/7
49 2. Counterexamples: MLE s are not always consistent Counterexample 1. (Ferguson, 1982). Suppose that X 1,...,X n are i.i.d. with density f θ0 f θ (x) =(1 θ) 1 δ(θ) f 0 ( x θ δ(θ) ) + θf 1 (x) where for θ [0, 1] where f 1 (x) = [ 1,1](x) Uniform[ 1, 1], f 0 (x) =(1 x )1 [ 1,1] (x) Triangular[ 1, 1] and δ(θ) satisfies: δ(0) = 1 0 <δ(θ) 1 θ δ(θ) 0 as θ 1. Maximum likelihood: p. 19/7
50 Density f θ (x) for c =2, θ = Maximum likelihood: p. 20/7
51 F θ (x) for c =2, θ = Maximum likelihood: p. 21/7
52 Ferguson (1982) shows that ˆθ n a.s. 1 no matter what θ 0 is true if δ(θ) 0 fast enough. Maximum likelihood: p. 22/7
53 Ferguson (1982) shows that ˆθ n a.s. 1 no matter what θ 0 is true if δ(θ) 0 fast enough. In fact, the assertion is true if δ(θ) =(1 θ)exp( (1 θ) c +1) with c > 2. (Ferguson shows that c =4works.) Maximum likelihood: p. 22/7
54 Ferguson (1982) shows that ˆθ n a.s. 1 no matter what θ 0 is true if δ(θ) 0 fast enough. In fact, the assertion is true if δ(θ) =(1 θ)exp( (1 θ) c +1) with c > 2. (Ferguson shows that c =4works.) If c =2, Ferguson s argument shows that sup n 1 log L n (θ) 0 θ 1 n 1 n D d log(m n/2) + 1 n log 1 M n δ(m n ) Maximum likelihood: p. 22/7
55 where P (D y) =exp ( ) 1, y log(2). 2(y log 2) That is, with E an Exponential(1) random variable D d =log2+ 1 2E. Maximum likelihood: p. 23/7
56 Maximum likelihood: p. 24/7
57 Counterexample 2. (4 B s, 1972). A distribution F on [0,b) is star-shaped if F (x)/x is non-decreasing on [0,b). Thus if F has a density f which is increasing on [0,b) then F is star-shaped. Maximum likelihood: p. 25/7
58 Counterexample 2. (4 B s, 1972). A distribution F on [0,b) is star-shaped if F (x)/x is non-decreasing on [0,b). Thus if F has a density f which is increasing on [0,b) then F is star-shaped. Let F star be the class of all star-shaped distributions on [0,b) for some b. Maximum likelihood: p. 25/7
59 Counterexample 2. (4 B s, 1972). A distribution F on [0,b) is star-shaped if F (x)/x is non-decreasing on [0,b). Thus if F has a density f which is increasing on [0,b) then F is star-shaped. Let F star be the class of all star-shaped distributions on [0,b) for some b. Suppose that X 1,...,X n are i.i.d. F F star. Maximum likelihood: p. 25/7
60 Counterexample 2. (4 B s, 1972). A distribution F on [0,b) is star-shaped if F (x)/x is non-decreasing on [0,b). Thus if F has a density f which is increasing on [0,b) then F is star-shaped. Let F star be the class of all star-shaped distributions on [0,b) for some b. Suppose that X 1,...,X n are i.i.d. F F star. Barlow, Bartholomew, Bremner, and Brunk (1972) show that the MLE of a star-shaped distribution function F is ˆF n (x) = 0, x < X (1) ix nx (n), X (i) x<x (i+1),i=1,...,n 1, 1, x X (n). Maximum likelihood: p. 25/7
61 Moreover, BBBB (1972) show that if F (x) =x for 0 x 1, then ˆF n (x) a.s. x 2 x for 0 x 1. Maximum likelihood: p. 26/7
62 MLE n =5and true d.f Maximum likelihood: p. 27/7
63 MLE n = 100 and true d.f Maximum likelihood: p. 28/7
64 MLE n = 100 and limit Maximum likelihood: p. 29/7
65 Note 1. Since X(i) d = Sj /S n+1 where S i = i j=1 E j with E j i.i.d. Exponential(1) rv s, the total mass at order statistics equals 1 nx (n) n n d X (i) = i=1 p 1 i=1 1 S i = n 1 ns n S n n 0 n j=1 (1 t)dt =1/2. ( 1 j 1 ) E j n Maximum likelihood: p. 30/7
66 Note 1. Since X(i) d = Sj /S n+1 where S i = i j=1 E j with E j i.i.d. Exponential(1) rv s, the total mass at order statistics equals 1 nx (n) n n d X (i) = i=1 p 1 i=1 1 S i = n 1 ns n S n n 0 n j=1 (1 t)dt =1/2. ( 1 j 1 ) E j n Note 2. BBBB (1972) present consistent estimators of F star-shaped via isotonization due to Barlow and Scheurer (1971) and van Zwet. Maximum likelihood: p. 30/7
67 Counterexample 3. (Boyles, Marshall, Proschan (1985). A distribution F on [0, ) is Increasing Failure Rate Average if 1 x { log(1 F (x))} 1 x Λ(x) is non-decreasing; that is, if Λ is star-shaped. Maximum likelihood: p. 31/7
68 Counterexample 3. (Boyles, Marshall, Proschan (1985). A distribution F on [0, ) is Increasing Failure Rate Average if 1 x { log(1 F (x))} 1 x Λ(x) is non-decreasing; that is, if Λ is star-shaped. Let F IFRA be the class of all IFRA- distributions on [0, ). Maximum likelihood: p. 31/7
69 Suppose that X 1,...,X n are i.i.d. F F IFRA. Boyles, Marshall, and Proschan (1985) showed that the MLE ˆF n of a IFRA-distribution function F is given by ˆλ j, X (j) x<x (j+1), log(1 ˆF n (x)) = j =0,...,n 1, x > X (n) where ˆλ j = j i=1 ( n X 1 (i) log k=i X ) (k) n k=i+1 X (k). Maximum likelihood: p. 32/7
70 Moreover, BMP (1985) show that if F is exponential(1), then 1 ˆF n (x) a.s. (1 + x) x exp( x), so 1 x ˆΛ n (x) a.s. log(1 + x) 1. Maximum likelihood: p. 33/7
71 MLE n = 100 and true d.f. 1 exp( x) Maximum likelihood: p. 34/7
72 MLE n = 100 and limit d.f.(1 + x) x Maximum likelihood: p. 35/7
73 More counterexamples: bivariate right censoring: Tsai, van der Laan, Pruitt Maximum likelihood: p. 36/7
74 More counterexamples: bivariate right censoring: Tsai, van der Laan, Pruitt left truncation and interval censoring: Chappell and Pan (1999) Maximum likelihood: p. 36/7
75 More counterexamples: bivariate right censoring: Tsai, van der Laan, Pruitt left truncation and interval censoring: Chappell and Pan (1999) bivariate interval censoring with a continuous mark: Hudgens, Maathuis, and Gilbert (2005) Maathuis and Wellner (2005) Maximum likelihood: p. 36/7
76 3. Beyond consistency: rates and distributions Le Cam (1973); Birgé (1983): optimal rate of convergence r n = r opt n determined by If nr 2 n =logn [] (1/r n, P) (1) log N [] (ɛ, P) (1) leads to the optimal rate of convergence r opt n = n γ/(2γ+1). K ɛ 1/γ (2) Maximum likelihood: p. 37/7
77 On the other hand, bounds (from Birgé and Massart (1993)), yield achieved rates of convergence for maximum likelihood estimators (and other minimum contrast estimators) r n = rn ach determined by nr 2 n = r 1 n cr 2 n If (2) holds, this leads to the rate { n γ/(2γ+1) log N [] (ɛ, P)dɛ if γ>1/2 n γ/2 if γ<1/2. Thus there is the possibility that maximum likelihood is not (rate-)optimal when γ<1/2. Maximum likelihood: p. 38/7
78 Typically 1 γ = d α where d is the dimension of the underlying sample space and α is a measure of the smoothness of the functions in P. Maximum likelihood: p. 39/7
79 Typically 1 γ = d α where d is the dimension of the underlying sample space and α is a measure of the smoothness of the functions in P. Hence leads to γ<1/2. α< d 2 Maximum likelihood: p. 39/7
80 Typically 1 γ = d α where d is the dimension of the underlying sample space and α is a measure of the smoothness of the functions in P. Hence leads to γ<1/2. α< d 2 But there are many examples/problem with γ>1/2! Maximum likelihood: p. 39/7
81 Optimal rate and MLE rate as a function of γ Maximum likelihood: p. 40/7
82 Difference of rates γ/(2γ +1) γ/ Maximum likelihood: p. 41/7
83 4. Positive Examples (some still in progress!) Interval censoring (Groeneboom) case 1, current status data case 2 (Groeneboom) Maximum likelihood: p. 42/7
84 4. Positive Examples (some still in progress!) Interval censoring (Groeneboom) case 1, current status data case 2 (Groeneboom) panel count data (Wellner and Zhang, 2000) Maximum likelihood: p. 42/7
85 4. Positive Examples (some still in progress!) Interval censoring (Groeneboom) case 1, current status data case 2 (Groeneboom) panel count data (Wellner and Zhang, 2000) k monotone densities (Balabdaoui and Wellner, 2004) Maximum likelihood: p. 42/7
86 4. Positive Examples (some still in progress!) Interval censoring (Groeneboom) case 1, current status data case 2 (Groeneboom) panel count data (Wellner and Zhang, 2000) k monotone densities (Balabdaoui and Wellner, 2004) competing risks current status data (Jewell and van der Laan; Maathuis) Maximum likelihood: p. 42/7
87 Example 1. (interval censoring, case 1) Maximum likelihood: p. 43/7
88 Example 1. (interval censoring, case 1) X F, Y G independent Observe (1{X Y },Y) (,Y). Goal: estimate F. MLE ˆF n exists Maximum likelihood: p. 43/7
89 Example 1. (interval censoring, case 1) X F, Y G independent Observe (1{X Y },Y) (,Y). Goal: estimate F. MLE ˆF n exists Global rate: d =1, α =1, γ = α/d =1. γ/(2γ +1)=1/3, sor n = n 1/3 : n 1/3 h(p ˆFn,p 0 )=O p (1) and this yields n 1/3 ˆF n F 0 dg = O p (1). Maximum likelihood: p. 43/7
90 Interval censoring case 1, continued: Maximum likelihood: p. 44/7
91 Interval censoring case 1, continued: Local rate: (Groeneboom, 1987) n 1/3 ( ˆF n (t 0 ) F (t 0 )) d { F (t0 )(1 F (t 0 ))f 0 (t 0 ) 2g(t 0 ) where Z = argmin{w (t)+t 2 } } 1/3 2Z Maximum likelihood: p. 44/7
92 Example 2. (interval censoring, case 2) Maximum likelihood: p. 45/7
93 Example 2. (interval censoring, case 2) X F, (U, V ) H, U V independent of X Observe i.i.d. copies of (,U,V) where = ( 1, 2, 3 ) = (1{X U}, 1{U <X V }, 1{V < X}) Maximum likelihood: p. 45/7
94 Example 2. (interval censoring, case 2) X F, (U, V ) H, U V independent of X Observe i.i.d. copies of (,U,V) where = ( 1, 2, 3 ) = (1{X U}, 1{U <X V }, 1{V < X}) Goal: estimate F. MLE ˆF n exists. Maximum likelihood: p. 45/7
95 (interval censoring, case 2, continued) Maximum likelihood: p. 46/7
96 (interval censoring, case 2, continued) Global rate (separated case): If P (V U ɛ) =1, d =1, α =1, γ = α/d =1 γ/(2γ +1)=1/3, sor n = n 1/3 n 1/3 h(p ˆF n,p 0 )=O p (1) and this yields n 1/3 ˆF n F 0 dµ = O p (1) where µ(a) =P (U A)+P (V A), A B 1 Maximum likelihood: p. 46/7
97 (interval censoring, case 2, continued) Maximum likelihood: p. 47/7
98 (interval censoring, case 2, continued) Global rate (nonseparated case): (van de Geer, 1993). n 1/3 (log n) 1/6 h(p ˆFn,p 0 )=O p (1). Although this looks worse in terms of the rate, it is actually better because the Hellinger metric is much stronger in this case. Maximum likelihood: p. 47/7
99 (interval censoring, case 2, continued) Global rate (nonseparated case): (van de Geer, 1993). n 1/3 (log n) 1/6 h(p ˆFn,p 0 )=O p (1). Although this looks worse in terms of the rate, it is actually better because the Hellinger metric is much stronger in this case. Maximum likelihood: p. 47/7
100 (interval censoring, case 2, continued) Local rate (separated case): (Groeneboom, 1996) { } n 1/3 ( ˆF f0 (t 0 ) 1/3 n (t 0 ) F 0 (t 0 )) d 2Z 2a(t 0 ) where Z = argmin{w (t)+t 2 } and a(t 0 )= h 1(t 0 ) F 0 (t 0 ) + k 1(t 0 )+ k 2 (t 0 )+ h 2(t 0 ) 1 F 0 (t 0 ) k 1 (u) = k 2 (v) = M u v 0 h(u, v) F 0 (v) F 0 (u) dv h(u, v) F 0 (v) F 0 (u) du Maximum likelihood: p. 48/7
101 (interval censoring, case 2, continued) Local rate (non-separated case): (conjectured, G&W, 1992) (n log n) 1/3 ( ˆF n (t 0 ) F 0 (t 0 )) d { 3 4 where Z = argmin{w (t)+t 2 } f 0 (t 0 ) 2 } 1/3 2Z h(t 0,t 0 ) Maximum likelihood: p. 49/7
102 (interval censoring, case 2, continued) Local rate (non-separated case): (conjectured, G&W, 1992) (n log n) 1/3 ( ˆF n (t 0 ) F 0 (t 0 )) d { 3 4 where Z = argmin{w (t)+t 2 } Monte-Carlo evidence in support: Groeneboom and Ketelaars (2005) f 0 (t 0 ) 2 } 1/3 2Z h(t 0,t 0 ) Maximum likelihood: p. 49/7
103 MSE histogram / MSE of MLE f 0 (t) = n=1000 n=2500 n=5000 n= Maximum likelihood: p. 50/7
104 MSE histogram / MSE of MLE f 0 (t) =4(1 t) n=1000 n=2500 n=5000 n= Maximum likelihood: p. 51/7
105 MSE histogram / MSE of MLE f 0 (t) =1 9 8 n=1000 n=2500 n=5000 n= Maximum likelihood: p. 52/7
106 MSE histogram / MSE of MLE f 0 (t) =4(1 t) n=1000 n=2500 n=5000 n= Maximum likelihood: p. 53/7
107 Example 3. (k-monotone densities) Maximum likelihood: p. 54/7
108 Example 3. (k-monotone densities) A density p on (0, ) is k monontone (p D k )ifitis non-negative and nonincreasing when k =1; and if ( 1) j p (j) (x) 0 for j =0,...,k 2 and ( 1) k 2 p (k 2) is convex for k 2. Maximum likelihood: p. 54/7
109 Example 3. (k-monotone densities) A density p on (0, ) is k monontone (p D k )ifitis non-negative and nonincreasing when k =1; and if ( 1) j p (j) (x) 0 for j =0,...,k 2 and ( 1) k 2 p (k 2) is convex for k 2. Mixture representation: p D k iff p(x) = 0 k (y x)k 1 yk + df (y) for some distribution function F on (0, ). k =1: monotone decreasing densities on R + k =2: convex decreasing densities on R + k 3:... k = : completely monotone densities = scale mixtures of exponential Maximum likelihood: p. 54/7
110 (k-monotone densities, continued) The MLE ˆp n of p 0 D k exists and is characterized by 0 { k (y x) k + 1, for all y 0 y k dp n (x) ˆp n (x) =1, if ( 1) k ˆp (k 1) n (y ) > ˆp (k 1) n (y+) Maximum likelihood: p. 55/7
111 (k-monotone densities, continued) The MLE ˆp n of p 0 D k exists and is characterized by 0 { k (y x) k + 1, for all y 0 y k dp n (x) ˆp n (x) =1, if ( 1) k ˆp (k 1) n (y ) > ˆp (k 1) n (y+) k =1Grenander estimator: r n = n 1/3 Maximum likelihood: p. 55/7
112 (k-monotone densities, continued) The MLE ˆp n of p 0 D k exists and is characterized by 0 { k (y x) k + 1, for all y 0 y k dp n (x) ˆp n (x) =1, if ( 1) k ˆp (k 1) n (y ) > ˆp (k 1) n (y+) k =1Grenander estimator: r n = n 1/3 Global rates and finite n minimax bounds: Birgé (1986), (1987), (1989) Maximum likelihood: p. 55/7
113 (k-monotone densities, continued) The MLE ˆp n of p 0 D k exists and is characterized by 0 { k (y x) k + 1, for all y 0 y k dp n (x) ˆp n (x) =1, if ( 1) k ˆp (k 1) n (y ) > ˆp (k 1) n (y+) k =1Grenander estimator: r n = n 1/3 Global rates and finite n minimax bounds: Birgé (1986), (1987), (1989) Local rates: Prakasa Rao (1969), Groeneboom (1985), (1989) Kim and Pollard (1990) { n 1/3 p0 (t 0 ) p 0 (ˆp n (t 0 ) p 0 (t 0 )) (t } 1/3 0) d 2Z 2 Maximum likelihood: p. 55/7
114 (k-monotone densities, continued) k =2; convex decreasing density d =1, α =2, γ =2, γ/(2γ +1)=2/5, sor n = n 2/5 (forward problem) Global rates: nothing yet Local rates and distributions: Groeneboom, Jongbloed, Wellner (2001) Maximum likelihood: p. 56/7
115 (k-monotone densities, continued) k =2; convex decreasing density d =1, α =2, γ =2, γ/(2γ +1)=2/5, sor n = n 2/5 (forward problem) Global rates: nothing yet Local rates and distributions: Groeneboom, Jongbloed, Wellner (2001) k 3; k - monotone density d =1, α = k, γ = k, γ/(2γ +1)=k/(2k +1),so r n = n k/(2k+1) (forward problem)? Global rates: nothing yet Local rates: should be r n = n k/(2k+1) progress: Balabdaoui and Wellner (2004) local rate is true if a certain conjecture about Hermite interpolation holds Maximum likelihood: p. 56/7
116 Direct and Inverse estimators k =3, n = (1a) - LSE, k=3, n=100 (direct problem) (1b) - MLE, k=3, n=100 (direct problem) (2a) - LSE, k=3, n=100 (inverse problem) (2b) - MLE, k=3, n=100 (inverse problem) Maximum likelihood: p. 57/7
117 Direct and Inverse estimators k =3, n = 1000 (1a) - LSE, k=3, n=1000 (direct problem) (1b) - MLE, k=3, n=1000 (direct problem) (2a) - LSE, k=3, n=1000 (inverse problem) (2b) - MLE, k=3, n=1000 (inverse problem) Maximum likelihood: p. 58/7
118 Direct and Inverse estimators k =6, n = (1a) - LSE, k=6, n=100 (direct problem) (1b) - LSE, k=6, n=100 (direct problem) (2a) - LSE, k=6, n=100 (inverse problem) (2b) - MLE, k=6, n=100 (inverse problem) Maximum likelihood: p. 59/7
119 Direct and Inverse estimators k =6, n = 1000 (1a) - LSE, k=6, n=1000 (direct problem) (1b) - MLE, k=6, n=1000 (direct problem) (2a) - LSE, k=6, n=1000 (inverse problem) (2b) - MLE, k=6, n=1000 (inverse problem) Maximum likelihood: p. 60/7
120 Example 4. (Competing risks with current status data) Maximum likelihood: p. 61/7
121 Example 4. (Competing risks with current status data) Variables of interest (X, Y ); X = failure time; Y = failure cause X R +, Y {1,...,K} T = an observation time, independent of (X, Y ) Maximum likelihood: p. 61/7
122 Example 4. (Competing risks with current status data) Variables of interest (X, Y ); X = failure time; Y = failure cause X R +, Y {1,...,K} T = an observation time, independent of (X, Y ) Observe: (,T), =( 1,..., K, K+1 ) where j =1{X T,Y = j}, j =1,...,K K+1 =1{X >T}. Maximum likelihood: p. 61/7
123 Example 4. (Competing risks with current status data) Variables of interest (X, Y ); X = failure time; Y = failure cause X R +, Y {1,...,K} T = an observation time, independent of (X, Y ) Observe: (,T), =( 1,..., K, K+1 ) where j =1{X T,Y = j}, j =1,...,K K+1 =1{X >T}. Goal: estimate F j (t) =P (X t, Y = j) for j =1,...,K Maximum likelihood: p. 61/7
124 Example 4. (Competing risks with current status data) Variables of interest (X, Y ); X = failure time; Y = failure cause X R +, Y {1,...,K} T = an observation time, independent of (X, Y ) Observe: (,T), =( 1,..., K, K+1 ) where j =1{X T,Y = j}, j =1,...,K K+1 =1{X >T}. Goal: estimate F j (t) =P (X t, Y = j) for j =1,...,K MLE ˆF n =(ˆF n,1,..., ˆF n,k ) exists! Characterization of ˆF n involves an interacting system of slopes of convex minorants Maximum likelihood: p. 61/7
125 (competing risks with current status data, continued) Global rates. Easy with present methods. n 1/3 K k=1 ˆF n,k (t) F 0,k (t) dg(t) =O p (1) Maximum likelihood: p. 62/7
126 (competing risks with current status data, continued) Global rates. Easy with present methods. n 1/3 K k=1 ˆF n,k (t) F 0,k (t) dg(t) =O p (1) Local rates? Conjecture r n = n 1/3. New methods needed. Tricky. Maathuis (2006)? Maximum likelihood: p. 62/7
127 (competing risks with current status data, continued) Global rates. Easy with present methods. n 1/3 K k=1 ˆF n,k (t) F 0,k (t) dg(t) =O p (1) Local rates? Conjecture r n = n 1/3. New methods needed. Tricky. Maathuis (2006)? Limit distribution theory: will involve slopes of an interacting system of greatest convex minorants defined in terms of a vector of dependent two-sided Brownian motions Maximum likelihood: p. 62/7
128 n 2/3 MSE of MLE and naive estimators of F1 n=25 n=250 MLE NE SNE TNE MLE NE SNE TNE t n 2 3 MSE t n 2 3 MSE n=2500 n=25000 MLE NE SNE TNE MLE NE SNE TNE t n 2 3 MSE t Maximum likelihood: p. 63/7 n 2 3 MSE
129 n 2/3 MSE of MLE and naive estimators of F 2 n=25 n=250 n 2 3 MSE MLE NE SNE TNE n 2 3 MSE MLE NE SNE TNE t t n=2500 n=25000 n 2 3 MSE MLE NE SNE TNE n 2 3 MSE MLE NE SNE TNE t t Maximum likelihood: p. 64/7
130 Example 5. (Monotone densities in R d ) Maximum likelihood: p. 65/7
131 Example 5. (Monotone densities in R d ) Entropy bounds? Natural conjectures: α =1, d, γ =1/d, soγ/(2γ +1)=1/(d +2) Maximum likelihood: p. 65/7
132 Example 5. (Monotone densities in R d ) Entropy bounds? Natural conjectures: α =1, d, γ =1/d, soγ/(2γ +1)=1/(d +2) Biau and Devroye (2003) show via Assouad s lemma and direct calculations: r opt n = n 1/(d+2) Maximum likelihood: p. 65/7
133 Example 5. (Monotone densities in R d ) Entropy bounds? Natural conjectures: α =1, d, γ =1/d, soγ/(2γ +1)=1/(d +2) Biau and Devroye (2003) show via Assouad s lemma and direct calculations: r opt n = n 1/(d+2) Rate achieved by the MLE: Natural conjecture: r ach n = n 1/2d, d > 2 Maximum likelihood: p. 65/7
134 Example 5. (Monotone densities in R d ) Entropy bounds? Natural conjectures: α =1, d, γ =1/d, soγ/(2γ +1)=1/(d +2) Biau and Devroye (2003) show via Assouad s lemma and direct calculations: r opt n = n 1/(d+2) Rate achieved by the MLE: Natural conjecture: r ach n = n 1/2d, d > 2 Biau and Devroye (2003) construct generalizations of Birgé s (1987) histogram estimators that achieve the optimal rate for all d 2. Maximum likelihood: p. 65/7
135 5. Problems and Challenges More tools for local rates and distribution theory? Maximum likelihood: p. 66/7
136 5. Problems and Challenges More tools for local rates and distribution theory? Under what additional hypotheses are MLE s globally rate optimal in the case γ>1/2? Maximum likelihood: p. 66/7
137 5. Problems and Challenges More tools for local rates and distribution theory? Under what additional hypotheses are MLE s globally rate optimal in the case γ>1/2? More counterexamples to clarify when MLE s do not work? Maximum likelihood: p. 66/7
138 5. Problems and Challenges More tools for local rates and distribution theory? Under what additional hypotheses are MLE s globally rate optimal in the case γ>1/2? More counterexamples to clarify when MLE s do not work? What is the limit distribution for interval censoring, case 2? (Does the G&W (1992) conjecture hold?) Maximum likelihood: p. 66/7
139 5. Problems and Challenges More tools for local rates and distribution theory? Under what additional hypotheses are MLE s globally rate optimal in the case γ>1/2? More counterexamples to clarify when MLE s do not work? What is the limit distribution for interval censoring, case 2? (Does the G&W (1992) conjecture hold?) When the MLE is not rate optimal, is it still preferable from some other perspectives? For example, does the MLE provide efficient estimators of smooth functionals (while alternative rate -optimal estimators fail to have this property)? Compare with Bickel and Ritov (2003). Maximum likelihood: p. 66/7
140 Problems and challenges, continued More rate and optimality theory for Maximum Likelihood Estimators of mixing distributions in mixture models with smooth kernels: e.g. completely monotone densities (scale mixtures of exponential), normal location mixtures (deconvolution problems) Maximum likelihood: p. 67/7
141 Problems and challenges, continued More rate and optimality theory for Maximum Likelihood Estimators of mixing distributions in mixture models with smooth kernels: e.g. completely monotone densities (scale mixtures of exponential), normal location mixtures (deconvolution problems) Stable and efficient algorithms for computing MLE s in models where they exist (e.g. mixture models, missing data). Maximum likelihood: p. 67/7
142 Problems and challenges, continued More rate and optimality theory for Maximum Likelihood Estimators of mixing distributions in mixture models with smooth kernels: e.g. completely monotone densities (scale mixtures of exponential), normal location mixtures (deconvolution problems) Stable and efficient algorithms for computing MLE s in models where they exist (e.g. mixture models, missing data). Results for monotone densities in R d? Maximum likelihood: p. 67/7
143 Selected References Bahadur, R. R. (1958). Examples of inconsistency of maximum likelihood estimates. Sankhya, Ser. A 20, Barlow, R. E., Bartholomew, D. J., Bremner, J. M., and Brunk, H. D. (1972). Statistical Inference under Order Restrictions. Wiley, New York. Barlow, R. E. and Scheurer, E. M (1971). Estimation from accelerated life tests. Technometrics 13, Biau, G. and Devroye, L. (2003). On the risk of estimates for block decreasing densities. J. Mult. Anal. 86, Bickel, P. J. and Ritov, Y. (2003). Nonparametric estimators which can be plugged-in. Ann. Statist. 31, Birgé, L. (1987). On the risk of histograms for estimating decreasing densities. Ann. Statist. 15, Maximum likelihood: p. 68/7
144 Birgé, L. (1989). The Grenander estimator: a nonasymptotic approach. Ann. Statist. 17, Birgé, L. and Massart, P. (1993). Rates of convergence for minimum contrast estimators. Probab. Theory Relat. Fields 97, Boyles, R. A., Marshall, A. W., Proschan, F. (1985). Inconsistency of the maximum likelihood estimator of a distribution having increasing failure rate average. Ann. Statist. 13, Ferguson, T. S. (1982). An inconsistent maximum likelihood estimate. J. Amer. Statist. Assoc. 77, Ghosh, M. (1995). Inconsistent maximum likelihood estimators for the Rasch model. Statist. Probab. Lett. 23, Maximum likelihood: p. 69/7
145 Hudgens, M., Maathuis, M., and Gilbert, P. (2005). Nonparametric estimation of the joint distribution of a survival time subject to interval censoring and a continuous mark variable. Submitted to Biometrics. Le Cam, L. (1990). Maximum likelihood: an introduction. Internat. Statist. Rev. 58, Maathuis, M. and Wellner, J. A. (2005). Inconsistency of the MLE for the joint distribution of interval censored survival times and continuous marks. Submitted to Biometrika. Pan, W. and Chappell, R. (1999). A note on inconsistency of NPMLE of the distribution function from left truncated and case I interval censored data. Lifetime Data Anal. 5, Maximum likelihood: p. 70/7
Maximum likelihood: counterexamples, examples, and open problems
Maximum likelihood: counterexamples, examples, and open problems Jon A. Wellner University of Washington visiting Vrije Universiteit, Amsterdam Talk at BeNeLuxFra Mathematics Meeting 21 May, 2005 Email:
More informationNonparametric estimation under shape constraints, part 2
Nonparametric estimation under shape constraints, part 2 Piet Groeneboom, Delft University August 7, 2013 What to expect? Theory and open problems for interval censoring, case 2. Same for the bivariate
More informationwith Current Status Data
Estimation and Testing with Current Status Data Jon A. Wellner University of Washington Estimation and Testing p. 1/4 joint work with Moulinath Banerjee, University of Michigan Talk at Université Paul
More informationNonparametric estimation under Shape Restrictions
Nonparametric estimation under Shape Restrictions Jon A. Wellner University of Washington, Seattle Statistical Seminar, Frejus, France August 30 - September 3, 2010 Outline: Five Lectures on Shape Restrictions
More informationChernoff s distribution is log-concave. But why? (And why does it matter?)
Chernoff s distribution is log-concave But why? (And why does it matter?) Jon A. Wellner University of Washington, Seattle University of Michigan April 1, 2011 Woodroofe Seminar Based on joint work with:
More informationThe International Journal of Biostatistics
The International Journal of Biostatistics Volume 1, Issue 1 2005 Article 3 Score Statistics for Current Status Data: Comparisons with Likelihood Ratio and Wald Statistics Moulinath Banerjee Jon A. Wellner
More informationSTATISTICAL INFERENCE UNDER ORDER RESTRICTIONS LIMIT THEORY FOR THE GRENANDER ESTIMATOR UNDER ALTERNATIVE HYPOTHESES
STATISTICAL INFERENCE UNDER ORDER RESTRICTIONS LIMIT THEORY FOR THE GRENANDER ESTIMATOR UNDER ALTERNATIVE HYPOTHESES By Jon A. Wellner University of Washington 1. Limit theory for the Grenander estimator.
More informationNonparametric estimation under shape constraints, part 1
Nonparametric estimation under shape constraints, part 1 Piet Groeneboom, Delft University August 6, 2013 What to expect? History of the subject Distribution of order restricted monotone estimates. Connection
More informationarxiv:math/ v2 [math.st] 17 Jun 2008
The Annals of Statistics 2008, Vol. 36, No. 3, 1031 1063 DOI: 10.1214/009053607000000974 c Institute of Mathematical Statistics, 2008 arxiv:math/0609020v2 [math.st] 17 Jun 2008 CURRENT STATUS DATA WITH
More informationA Law of the Iterated Logarithm. for Grenander s Estimator
A Law of the Iterated Logarithm for Grenander s Estimator Jon A. Wellner University of Washington, Seattle Probability Seminar October 24, 2016 Based on joint work with: Lutz Dümbgen and Malcolm Wolff
More informationScore Statistics for Current Status Data: Comparisons with Likelihood Ratio and Wald Statistics
Score Statistics for Current Status Data: Comparisons with Likelihood Ratio and Wald Statistics Moulinath Banerjee 1 and Jon A. Wellner 2 1 Department of Statistics, Department of Statistics, 439, West
More informationISOTONIC MAXIMUM LIKELIHOOD ESTIMATION FOR THE CHANGE POINT OF A HAZARD RATE
Sankhyā : The Indian Journal of Statistics 1997, Volume 59, Series A, Pt. 3, pp. 392-407 ISOTONIC MAXIMUM LIKELIHOOD ESTIMATION FOR THE CHANGE POINT OF A HAZARD RATE By S.N. JOSHI Indian Statistical Institute,
More informationNonparametric estimation for current status data with competing risks
Nonparametric estimation for current status data with competing risks Marloes Henriëtte Maathuis A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy
More informationNONPARAMETRIC CONFIDENCE INTERVALS FOR MONOTONE FUNCTIONS. By Piet Groeneboom and Geurt Jongbloed Delft University of Technology
NONPARAMETRIC CONFIDENCE INTERVALS FOR MONOTONE FUNCTIONS By Piet Groeneboom and Geurt Jongbloed Delft University of Technology We study nonparametric isotonic confidence intervals for monotone functions.
More informationShape constrained estimation: a brief introduction
Shape constrained estimation: a brief introduction Jon A. Wellner University of Washington, Seattle Cowles Foundation, Yale November 7, 2016 Econometrics lunch seminar, Yale Based on joint work with: Fadoua
More informationarxiv:math/ v2 [math.st] 17 Jun 2008
The Annals of Statistics 2008, Vol. 36, No. 3, 1064 1089 DOI: 10.1214/009053607000000983 c Institute of Mathematical Statistics, 2008 arxiv:math/0609021v2 [math.st] 17 Jun 2008 CURRENT STATUS DATA WITH
More informationLes deux complices. Piet Groeneboom, Delft University. Juin 28, 2018
Les deux complices Piet Groeneboom, Delft University Juin 28, 2018 Monsieur Teste (Paul Valéry) Je me suis rarement perdu de vue Je me suis détesté je me suis adoré Puis, nous avons vieilli ensemble (La
More informationLeast squares under convex constraint
Stanford University Questions Let Z be an n-dimensional standard Gaussian random vector. Let µ be a point in R n and let Y = Z + µ. We are interested in estimating µ from the data vector Y, under the assumption
More informationAdditive Isotonic Regression
Additive Isotonic Regression Enno Mammen and Kyusang Yu 11. July 2006 INTRODUCTION: We have i.i.d. random vectors (Y 1, X 1 ),..., (Y n, X n ) with X i = (X1 i,..., X d i ) and we consider the additive
More informationInconsistency of the MLE for the joint distribution. of interval censored survival times. and continuous marks
Inconsistency of the MLE for the joint distribution of interval censored survival times and continuous marks By M.H. Maathuis and J.A. Wellner Department of Statistics, University of Washington, Seattle,
More informationarxiv:math/ v1 [math.st] 21 Jul 2005
The Annals of Statistics 2005, Vol. 33, No. 3, 1330 1356 DOI: 10.1214/009053605000000138 c Institute of Mathematical Statistics, 2005 arxiv:math/0507427v1 [math.st] 21 Jul 2005 ESTIMATION OF A FUNCTION
More informationEmpirical likelihood ratio with arbitrarily censored/truncated data by EM algorithm
Empirical likelihood ratio with arbitrarily censored/truncated data by EM algorithm Mai Zhou 1 University of Kentucky, Lexington, KY 40506 USA Summary. Empirical likelihood ratio method (Thomas and Grunkmier
More informationEstimation of a k-monotone density, part 2: algorithms for computation and numerical results
Estimation of a k-monotone density, part 2: algorithms for computation and numerical results Fadoua Balabdaoui 1 and Jon A. Wellner 2 University of Washington September 2, 2004 Abstract The iterative (2k
More informationMaximum Likelihood Estimation under Shape Constraints
Maximum Likelihood Estimation under Shape Constraints Hanna K. Jankowski June 2 3, 29 Contents 1 Introduction 2 2 The MLE of a decreasing density 3 2.1 Finding the Estimator..............................
More informationOn Efficiency of the Plug-in Principle for Estimating Smooth Integrated Functionals of a Nonincreasing Density
On Efficiency of the Plug-in Principle for Estimating Smooth Integrated Functionals of a Nonincreasing Density arxiv:188.7915v2 [math.st] 12 Feb 219 Rajarshi Mukherjee and Bodhisattva Sen Department of
More informationAsymptotic normality of the L k -error of the Grenander estimator
Asymptotic normality of the L k -error of the Grenander estimator Vladimir N. Kulikov Hendrik P. Lopuhaä 1.7.24 Abstract: We investigate the limit behavior of the L k -distance between a decreasing density
More informationINCONSISTENCY OF BOOTSTRAP: THE GRENANDER ESTIMATOR
Submitted to the Annals of Statistics INCONSISTENCY OF BOOTSTRAP: THE GRENANDER ESTIMATOR By Bodhisattva Sen, Moulinath Banerjee and Michael Woodroofe Columbia University, University of Michigan and University
More informationFULL LIKELIHOOD INFERENCES IN THE COX MODEL
October 20, 2007 FULL LIKELIHOOD INFERENCES IN THE COX MODEL BY JIAN-JIAN REN 1 AND MAI ZHOU 2 University of Central Florida and University of Kentucky Abstract We use the empirical likelihood approach
More informationand Comparison with NPMLE
NONPARAMETRIC BAYES ESTIMATOR OF SURVIVAL FUNCTIONS FOR DOUBLY/INTERVAL CENSORED DATA and Comparison with NPMLE Mai Zhou Department of Statistics, University of Kentucky, Lexington, KY 40506 USA http://ms.uky.edu/
More informationNonparametric estimation of log-concave densities
Nonparametric estimation of log-concave densities Jon A. Wellner University of Washington, Seattle Seminaire, Institut de Mathématiques de Toulouse 5 March 2012 Seminaire, Toulouse Based on joint work
More informationEfficient Semiparametric Estimators via Modified Profile Likelihood in Frailty & Accelerated-Failure Models
NIH Talk, September 03 Efficient Semiparametric Estimators via Modified Profile Likelihood in Frailty & Accelerated-Failure Models Eric Slud, Math Dept, Univ of Maryland Ongoing joint project with Ilia
More informationLecture 8: Information Theory and Statistics
Lecture 8: Information Theory and Statistics Part II: Hypothesis Testing and I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 23, 2015 1 / 50 I-Hsiang
More informationGeneralized continuous isotonic regression
Generalized continuous isotonic regression Piet Groeneboom, Geurt Jongbloed To cite this version: Piet Groeneboom, Geurt Jongbloed. Generalized continuous isotonic regression. Statistics and Probability
More informationf(x θ)dx with respect to θ. Assuming certain smoothness conditions concern differentiating under the integral the integral sign, we first obtain
0.1. INTRODUCTION 1 0.1 Introduction R. A. Fisher, a pioneer in the development of mathematical statistics, introduced a measure of the amount of information contained in an observaton from f(x θ). Fisher
More informationAn Information Criteria for Order-restricted Inference
An Information Criteria for Order-restricted Inference Nan Lin a, Tianqing Liu 2,b, and Baoxue Zhang,2,b a Department of Mathematics, Washington University in Saint Louis, Saint Louis, MO 633, U.S.A. b
More informationEditorial Manager(tm) for Lifetime Data Analysis Manuscript Draft. Manuscript Number: Title: On An Exponential Bound for the Kaplan - Meier Estimator
Editorial Managertm) for Lifetime Data Analysis Manuscript Draft Manuscript Number: Title: On An Exponential Bound for the Kaplan - Meier Estimator Article Type: SI-honor of N. Breslow invitation only)
More informationM- and Z- theorems; GMM and Empirical Likelihood Wellner; 5/13/98, 1/26/07, 5/08/09, 6/14/2010
M- and Z- theorems; GMM and Empirical Likelihood Wellner; 5/13/98, 1/26/07, 5/08/09, 6/14/2010 Z-theorems: Notation and Context Suppose that Θ R k, and that Ψ n : Θ R k, random maps Ψ : Θ R k, deterministic
More informationChapter 2 Inference on Mean Residual Life-Overview
Chapter 2 Inference on Mean Residual Life-Overview Statistical inference based on the remaining lifetimes would be intuitively more appealing than the popular hazard function defined as the risk of immediate
More informationEstimation of a discrete monotone distribution
Electronic Journal of Statistics Vol. 3 (009) 1567 1605 ISSN: 1935-754 DOI: 10.114/09-EJS56 Estimation of a discrete monotone distribution Hanna K. Jankowski Department of Mathematics and Statistics York
More informationReliability of Coherent Systems with Dependent Component Lifetimes
Reliability of Coherent Systems with Dependent Component Lifetimes M. Burkschat Abstract In reliability theory, coherent systems represent a classical framework for describing the structure of technical
More informationPreservation Theorems for Glivenko-Cantelli and Uniform Glivenko-Cantelli Classes
Preservation Theorems for Glivenko-Cantelli and Uniform Glivenko-Cantelli Classes This is page 5 Printer: Opaque this Aad van der Vaart and Jon A. Wellner ABSTRACT We show that the P Glivenko property
More informationNonparametric estimation under Shape Restrictions
Nonparametric estimation under Shape Restrictions Jon A. Wellner University of Washington, Seattle Statistical Seminar, Frejus, France August 30 - September 3, 2010 Outline: Five Lectures on Shape Restrictions
More informationLikelihood Ratio Tests for Monotone Functions
Likelihood Ratio Tests for Monotone Functions Moulinath Banerjee 1 and Jon A. Wellner 2 University of Washington July 20, 2001 Abstract We study the problem of testing for equality at a fixed point in
More informationA note on the asymptotic distribution of Berk-Jones type statistics under the null hypothesis
A note on the asymptotic distribution of Berk-Jones type statistics under the null hypothesis Jon A. Wellner and Vladimir Koltchinskii Abstract. Proofs are given of the limiting null distributions of the
More informationNonparametric estimation of log-concave densities
Nonparametric estimation of log-concave densities Jon A. Wellner University of Washington, Seattle Northwestern University November 5, 2010 Conference on Shape Restrictions in Non- and Semi-Parametric
More informationCURE MODEL WITH CURRENT STATUS DATA
Statistica Sinica 19 (2009), 233-249 CURE MODEL WITH CURRENT STATUS DATA Shuangge Ma Yale University Abstract: Current status data arise when only random censoring time and event status at censoring are
More informationLearning Multivariate Log-concave Distributions
Proceedings of Machine Learning Research vol 65:1 17, 2017 Learning Multivariate Log-concave Distributions Ilias Diakonikolas University of Southern California Daniel M. Kane University of California,
More informationNonparametric Bayes Estimator of Survival Function for Right-Censoring and Left-Truncation Data
Nonparametric Bayes Estimator of Survival Function for Right-Censoring and Left-Truncation Data Mai Zhou and Julia Luan Department of Statistics University of Kentucky Lexington, KY 40506-0027, U.S.A.
More informationA central limit theorem for the Hellinger loss of Grenander-type estimators
Delft University of Technology A central limit theorem for the Hellinger loss of Grenander-type estimators Lopuhaa, Rik; Musta, Eni DOI./stan.53 Publication date 8 Document Version Other version Published
More informationEfficiency of Profile/Partial Likelihood in the Cox Model
Efficiency of Profile/Partial Likelihood in the Cox Model Yuichi Hirose School of Mathematics, Statistics and Operations Research, Victoria University of Wellington, New Zealand Summary. This paper shows
More informationBayesian estimation of the discrepancy with misspecified parametric models
Bayesian estimation of the discrepancy with misspecified parametric models Pierpaolo De Blasi University of Torino & Collegio Carlo Alberto Bayesian Nonparametrics workshop ICERM, 17-21 September 2012
More informationEstimation of a k monotone density, part 3: limiting Gaussian versions of the problem; invelopes and envelopes
Estimation of a monotone density, part 3: limiting Gaussian versions of the problem; invelopes and envelopes Fadoua Balabdaoui 1 and Jon A. Wellner University of Washington September, 004 Abstract Let
More informationLikelihood Based Inference for Monotone Response Models
Likelihood Based Inference for Monotone Response Models Moulinath Banerjee University of Michigan September 5, 25 Abstract The behavior of maximum likelihood estimates (MLE s) the likelihood ratio statistic
More informationCURRENT STATUS LINEAR REGRESSION. By Piet Groeneboom and Kim Hendrickx Delft University of Technology and Hasselt University
CURRENT STATUS LINEAR REGRESSION By Piet Groeneboom and Kim Hendrickx Delft University of Technology and Hasselt University We construct n-consistent and asymptotically normal estimates for the finite
More informationNonparametric estimation: s concave and log-concave densities: alternatives to maximum likelihood
Nonparametric estimation: s concave and log-concave densities: alternatives to maximum likelihood Jon A. Wellner University of Washington, Seattle Statistics Seminar, York October 15, 2015 Statistics Seminar,
More informationUniversity of California, Berkeley
University of California, Berkeley U.C. Berkeley Division of Biostatistics Working Paper Series Year 2001 Paper 100 Maximum Likelihood Estimation of Ordered Multinomial Parameters Nicholas P. Jewell John
More informationICSA Applied Statistics Symposium 1. Balanced adjusted empirical likelihood
ICSA Applied Statistics Symposium 1 Balanced adjusted empirical likelihood Art B. Owen Stanford University Sarah Emerson Oregon State University ICSA Applied Statistics Symposium 2 Empirical likelihood
More informationRATES OF CONVERGENCE OF ESTIMATES, KOLMOGOROV S ENTROPY AND THE DIMENSIONALITY REDUCTION PRINCIPLE IN REGRESSION 1
The Annals of Statistics 1997, Vol. 25, No. 6, 2493 2511 RATES OF CONVERGENCE OF ESTIMATES, KOLMOGOROV S ENTROPY AND THE DIMENSIONALITY REDUCTION PRINCIPLE IN REGRESSION 1 By Theodoros Nicoleris and Yannis
More informationEstimating monotone, unimodal and U shaped failure rates using asymptotic pivots
Estimating monotone, unimodal and U shaped failure rates using asymptotic pivots Moulinath Banerjee University of Michigan March 17, 2006 Abstract We propose a new method for pointwise estimation of monotone,
More informationStat583. Piet Groeneboom. April 27, 1999
Stat583 Piet Groeneboom April 27, 1999 1 The EM algorithm and self-consistency We recall the basic model and notation of Chapter 7, section 1, of Stat582, but use for reasons that soon will become clear)
More informationStatistics - Lecture One. Outline. Charlotte Wickham 1. Basic ideas about estimation
Statistics - Lecture One Charlotte Wickham wickham@stat.berkeley.edu http://www.stat.berkeley.edu/~wickham/ Outline 1. Basic ideas about estimation 2. Method of Moments 3. Maximum Likelihood 4. Confidence
More informationNonparametric estimation of. s concave and log-concave densities: alternatives to maximum likelihood
Nonparametric estimation of s concave and log-concave densities: alternatives to maximum likelihood Jon A. Wellner University of Washington, Seattle Cowles Foundation Seminar, Yale University November
More informationA MODIFIED LIKELIHOOD RATIO TEST FOR HOMOGENEITY IN FINITE MIXTURE MODELS
A MODIFIED LIKELIHOOD RATIO TEST FOR HOMOGENEITY IN FINITE MIXTURE MODELS Hanfeng Chen, Jiahua Chen and John D. Kalbfleisch Bowling Green State University and University of Waterloo Abstract Testing for
More informationEstimation for two-phase designs: semiparametric models and Z theorems
Estimation for two-phase designs:semiparametric models and Z theorems p. 1/27 Estimation for two-phase designs: semiparametric models and Z theorems Jon A. Wellner University of Washington Estimation for
More informationA talk on Oracle inequalities and regularization. by Sara van de Geer
A talk on Oracle inequalities and regularization by Sara van de Geer Workshop Regularization in Statistics Banff International Regularization Station September 6-11, 2003 Aim: to compare l 1 and other
More informationarxiv: v3 [math.st] 29 Nov 2018
A unified study of nonparametric inference for monotone functions arxiv:186.1928v3 math.st] 29 Nov 218 Ted Westling Center for Causal Inference University of Pennsylvania tgwest@pennmedicine.upenn.edu
More information12 - Nonparametric Density Estimation
ST 697 Fall 2017 1/49 12 - Nonparametric Density Estimation ST 697 Fall 2017 University of Alabama Density Review ST 697 Fall 2017 2/49 Continuous Random Variables ST 697 Fall 2017 3/49 1.0 0.8 F(x) 0.6
More informationDepartment of Statistics, School of Mathematical Sciences, Ferdowsi University of Mashhad, Iran.
JIRSS (2012) Vol. 11, No. 2, pp 191-202 A Goodness of Fit Test For Exponentiality Based on Lin-Wong Information M. Abbasnejad, N. R. Arghami, M. Tavakoli Department of Statistics, School of Mathematical
More informationEstimating Bivariate Survival Function by Volterra Estimator Using Dynamic Programming Techniques
Journal of Data Science 7(2009), 365-380 Estimating Bivariate Survival Function by Volterra Estimator Using Dynamic Programming Techniques Jiantian Wang and Pablo Zafra Kean University Abstract: For estimating
More informationPreservation Theorems for Glivenko-Cantelli and Uniform Glivenko-Cantelli Classes
Preservation Theorems for Glivenko-Cantelli and Uniform Glivenko-Cantelli Classes Aad van der Vaart and Jon A. Wellner Free University and University of Washington ABSTRACT We show that the P Glivenko
More informationEstimation of the Bivariate and Marginal Distributions with Censored Data
Estimation of the Bivariate and Marginal Distributions with Censored Data Michael Akritas and Ingrid Van Keilegom Penn State University and Eindhoven University of Technology May 22, 2 Abstract Two new
More informationTo appear in The American Statistician vol. 61 (2007) pp
How Can the Score Test Be Inconsistent? David A Freedman ABSTRACT: The score test can be inconsistent because at the MLE under the null hypothesis the observed information matrix generates negative variance
More informationAnalysis of Gamma and Weibull Lifetime Data under a General Censoring Scheme and in the presence of Covariates
Communications in Statistics - Theory and Methods ISSN: 0361-0926 (Print) 1532-415X (Online) Journal homepage: http://www.tandfonline.com/loi/lsta20 Analysis of Gamma and Weibull Lifetime Data under a
More informationLikelihood Based Inference for Monotone Response Models
Likelihood Based Inference for Monotone Response Models Moulinath Banerjee University of Michigan September 11, 2006 Abstract The behavior of maximum likelihood estimates (MLEs) and the likelihood ratio
More informationAsymptotic Properties of Nonparametric Estimation Based on Partly Interval-Censored Data
Asymptotic Properties of Nonparametric Estimation Based on Partly Interval-Censored Data Jian Huang Department of Statistics and Actuarial Science University of Iowa, Iowa City, IA 52242 Email: jian@stat.uiowa.edu
More informationGoodness-of-Fit Tests for Time Series Models: A Score-Marked Empirical Process Approach
Goodness-of-Fit Tests for Time Series Models: A Score-Marked Empirical Process Approach By Shiqing Ling Department of Mathematics Hong Kong University of Science and Technology Let {y t : t = 0, ±1, ±2,
More informationEstimation of a Two-component Mixture Model
Estimation of a Two-component Mixture Model Bodhisattva Sen 1,2 University of Cambridge, Cambridge, UK Columbia University, New York, USA Indian Statistical Institute, Kolkata, India 6 August, 2012 1 Joint
More informationAsymptotically optimal estimation of smooth functionals for interval censoring, case 2
Asymptotically optimal estimation of smooth functionals for interval censoring, case 2 Ronald Geskus 1 and Piet Groeneboom 2 Municipal Health Service Dept Public Health and Environment Nieuwe Achtergracht
More informationis a Borel subset of S Θ for each c R (Bertsekas and Shreve, 1978, Proposition 7.36) This always holds in practical applications.
Stat 811 Lecture Notes The Wald Consistency Theorem Charles J. Geyer April 9, 01 1 Analyticity Assumptions Let { f θ : θ Θ } be a family of subprobability densities 1 with respect to a measure µ on a measurable
More informationOn the Convolution Order with Reliability Applications
Applied Mathematical Sciences, Vol. 3, 2009, no. 16, 767-778 On the Convolution Order with Reliability Applications A. Alzaid and M. Kayid King Saud University, College of Science Dept. of Statistics and
More informationMinimax lower bounds I
Minimax lower bounds I Kyoung Hee Kim Sungshin University 1 Preliminaries 2 General strategy 3 Le Cam, 1973 4 Assouad, 1983 5 Appendix Setting Family of probability measures {P θ : θ Θ} on a sigma field
More informationBayesian Regularization
Bayesian Regularization Aad van der Vaart Vrije Universiteit Amsterdam International Congress of Mathematicians Hyderabad, August 2010 Contents Introduction Abstract result Gaussian process priors Co-authors
More informationLikelihood Construction, Inference for Parametric Survival Distributions
Week 1 Likelihood Construction, Inference for Parametric Survival Distributions In this section we obtain the likelihood function for noninformatively rightcensored survival data and indicate how to make
More informationStatistics: Learning models from data
DS-GA 1002 Lecture notes 5 October 19, 2015 Statistics: Learning models from data Learning models from data that are assumed to be generated probabilistically from a certain unknown distribution is a crucial
More informationMore Powerful Tests for Homogeneity of Multivariate Normal Mean Vectors under an Order Restriction
Sankhyā : The Indian Journal of Statistics 2007, Volume 69, Part 4, pp. 700-716 c 2007, Indian Statistical Institute More Powerful Tests for Homogeneity of Multivariate Normal Mean Vectors under an Order
More informationSpatially Smoothed Kernel Density Estimation via Generalized Empirical Likelihood
Spatially Smoothed Kernel Density Estimation via Generalized Empirical Likelihood Kuangyu Wen & Ximing Wu Texas A&M University Info-Metrics Institute Conference: Recent Innovations in Info-Metrics October
More informationBootstrap, Jackknife and other resampling methods
Bootstrap, Jackknife and other resampling methods Part III: Parametric Bootstrap Rozenn Dahyot Room 128, Department of Statistics Trinity College Dublin, Ireland dahyot@mee.tcd.ie 2005 R. Dahyot (TCD)
More informationarxiv:math/ v1 [math.st] 11 Feb 2006
The Annals of Statistics 25, Vol. 33, No. 5, 2228 2255 DOI: 1.1214/95365462 c Institute of Mathematical Statistics, 25 arxiv:math/62244v1 [math.st] 11 Feb 26 ASYMPTOTIC NORMALITY OF THE L K -ERROR OF THE
More informationInformation Measure Estimation and Applications: Boosting the Effective Sample Size from n to n ln n
Information Measure Estimation and Applications: Boosting the Effective Sample Size from n to n ln n Jiantao Jiao (Stanford EE) Joint work with: Kartik Venkat Yanjun Han Tsachy Weissman Stanford EE Tsinghua
More informationOn the generalized maximum likelihood estimator of survival function under Koziol Green model
On the generalized maximum likelihood estimator of survival function under Koziol Green model By: Haimeng Zhang, M. Bhaskara Rao, Rupa C. Mitra Zhang, H., Rao, M.B., and Mitra, R.C. (2006). On the generalized
More informationFULL LIKELIHOOD INFERENCES IN THE COX MODEL: AN EMPIRICAL LIKELIHOOD APPROACH
FULL LIKELIHOOD INFERENCES IN THE COX MODEL: AN EMPIRICAL LIKELIHOOD APPROACH Jian-Jian Ren 1 and Mai Zhou 2 University of Central Florida and University of Kentucky Abstract: For the regression parameter
More information1.1 Basis of Statistical Decision Theory
ECE598: Information-theoretic methods in high-dimensional statistics Spring 2016 Lecture 1: Introduction Lecturer: Yihong Wu Scribe: AmirEmad Ghassami, Jan 21, 2016 [Ed. Jan 31] Outline: Introduction of
More informationLecture 3. Truncation, length-bias and prevalence sampling
Lecture 3. Truncation, length-bias and prevalence sampling 3.1 Prevalent sampling Statistical techniques for truncated data have been integrated into survival analysis in last two decades. Truncation in
More informationTime-varying failure rate for system reliability analysis in large-scale railway risk assessment simulation
Time-varying failure rate for system reliability analysis in large-scale railway risk assessment simulation H. Zhang, E. Cutright & T. Giras Center of Rail Safety-Critical Excellence, University of Virginia,
More informationUnobserved Heterogeneity in Longitudinal Data An Empirical Bayes Perspective
Unobserved Heterogeneity in Longitudinal Data An Empirical Bayes Perspective Roger Koenker University of Illinois, Urbana-Champaign University of Tokyo: 25 November 2013 Joint work with Jiaying Gu (UIUC)
More informationReview and continuation from last week Properties of MLEs
Review and continuation from last week Properties of MLEs As we have mentioned, MLEs have a nice intuitive property, and as we have seen, they have a certain equivariance property. We will see later that
More informationUniversity of California, Berkeley
University of California, Berkeley U.C. Berkeley Division of Biostatistics Working Paper Series Year 24 Paper 153 A Note on Empirical Likelihood Inference of Residual Life Regression Ying Qing Chen Yichuan
More informationFrailty Models and Copulas: Similarities and Differences
Frailty Models and Copulas: Similarities and Differences KLARA GOETHALS, PAUL JANSSEN & LUC DUCHATEAU Department of Physiology and Biometrics, Ghent University, Belgium; Center for Statistics, Hasselt
More informationMath 152. Rumbos Fall Solutions to Assignment #12
Math 52. umbos Fall 2009 Solutions to Assignment #2. Suppose that you observe n iid Bernoulli(p) random variables, denoted by X, X 2,..., X n. Find the LT rejection region for the test of H o : p p o versus
More informationMaximum likelihood estimation of a log-concave density based on censored data
Maximum likelihood estimation of a log-concave density based on censored data Dominic Schuhmacher Institute of Mathematical Statistics and Actuarial Science University of Bern Joint work with Lutz Dümbgen
More information