Method of Conditional Moments Based on Incomplete Data

Size: px
Start display at page:

Download "Method of Conditional Moments Based on Incomplete Data"

Transcription

1 , ISSN X (Online, ISSN (Print, Vol. 20; Issue No. 3; Year 2013, Copyright 2013 by CESER Publications Method of Conditional Moments Based on Incomplete Data Yan Lu 1 and Naisheng Wang 2 1 Department of Mathematics and Statistics University of New Mexico Albuquerque, NM, USA, luyan@math.unm.edu 2 China Securities Index Co., Ltd Shanghai, China, wangnaisheng@yahoo.com.cn ABSTRACT This paper extends the traditional methods of moments to incomplete sample cases. This method is termed as the method of conditional moments since it is obtained via conditioning on observed data. The convergence and asymptotic normality of the conditional moment estimator are investigated under certain conditions. An iterative algorithm is proposed to solve the conditional moment equations. Examples show that the proposed method performs well for incomplete data. Keywords: Asymptotic normality, Censored sample, Conditional moment estimator, Convergence, Grouped sample Mathematics Subject Classification: Introduction Method of moments is one of the oldest and most extensively used methods for parameter estimation. Let X =(X 1,X 2,,X n be independent and identically distributed random variables, each with distribution function F (x, θ for some fixed θ Θ R d. If the moments exist, the moment estimators for θ =(θ 1,,θ d are the solutions of the moment equations: μ (θ =m (X, =1,,d, (1.1 where μ (θ =E(X1 is the -th non-central population moment that depends on the unnown parameter vector θ, whereas m (X is the -th non-central sample moment which does not depend on any unnown parameter. Thus, the method of moments attempt to equate the population moments to the corresponding non-central sample moments to solve for θ (if exists. Under appropriate conditions, certain properties such as consistency and asymptotic normality can be derived for the moment estimators. Method of moment is popular also because of its simplicity in computing. However, an underlying assumption for the method of moments is the

2 completeness of the data set, which is not satisfied in many practical settings. The method of moments fails in those situations. The problem to apply the method of moments to the incomplete data cases lies in the construction of the sample moments m (X. This issue has been investigated by Wang (1992 and Mao and Wang (1997 in some special cases such as type-ii censored Weibull distribution. In this paper we propose the method of conditional moments for parameter estimation when the sample is incomplete. The proposed method equates the population moments to the corresponding sample moments under given conditions. One can apply this method in both complete and incomplete sample cases. This paper is organized as follows. Section 2 describes the proposed method of conditional moments. Section 3 discusses properties of the conditional moment estimators such as consistency and asymptotic normality. Section 4 proposes an efficient iterative algorithm of the method of conditional moments. Illustrative examples are presented in Section 5. Concluding remars are given in Section 6. 2 Method of Conditional Moments 2.1 Definition of the proposed method Define (X 1,,X n, θ, μ (θ and m (X as in Section 1. If the sample is incomplete, it is a problem to obtain m (X and therefore the regular method of moments fails. Let Y be the observed sample and a (θ, Y the conditional expectation of m (X given the observed Y, i.e. a (θ, Y =E{m (X Y}. (2.1 We call a (θ, Y the -th non-central conditional sample moments. It is clear that a (θ, Y depends on both Y and θ. By the property of conditional expectation, we have E{a (θ, Y} = μ (θ. Given the observed incomplete sample Y, the conditional expectation a (θ, Y is the optimal estimation of m (X under the mean square error minimization criteria. Therefore, a (θ, Y is expected to be close to m (X. Thus, we may replace m (X in (1.1 with a (θ, Y to estimate the parameter θ. Let a(θ, Y = (a 1 (θ, Y,,a d (θ, Y and μ(θ = (μ 1 (θ,,μ d (θ, (1.1 becomes: μ(θ =a(θ, Y. (2.2 We call (2.2 the conditional moment equation since a(θ, Y is the non-central conditional sample moment of the complete sample X. ˆθ is called the conditional moment estimator of parameter θ, if it falls in the parameter space Θ and satisfies (2.2. In this paper, we assume that θ is identifiable. The conditional sample moments reduce to the sample moments if data is complete. 3 Properties of the Method of Conditional Moments In this section, we will investigate the properties of the proposed method. In Section 3.1, we discuss convergence and asymptotic normality of the conditional moment estimator. In Section 41

3 3.2, we show that some frequently used incomplete samples such as randomly censored sample, doubly censored sample and grouped sample all fall within the range of the conditions for the convergence and asymptotic normality of the conditional moment estimator. The notation a (n (θ, Y, ˆθ n and a (n (θ, Y is used in this section. The supscript and subscript n stands for the size of the unobserved complete sample X. 3.1 Convergence and Asymptotic Normality Theorem 3.1. Suppose the following conditions are satisfied. (1 parameter space Θ is a compact subset of R d ; (2 μ(θ is continuous with respect to parameter θ; (3 a (n (θ, Y is differentiable with respect to θ, and θ a(n (θ, Y < K< holds for each n and θ, here is Euclidean norm and K is certain positive number. (4 there exits θ 0 Θ, such that a (n (θ, Y a.s μ(θ (as n if and only if θ = θ 0 ; Under the above conditions, we have ˆθ n a.s θ 0, as n Proof. From the compactness of the parameter space Θ, there exists a θ Θ, such that where {ˆθ n } is a subsequence of {ˆθ (n }. Thus By the definition of ˆθ n,wehave Expand a (n (ˆθ n, Y at the point θ, ˆθ n a.s θ, as n μ(ˆθ n μ( θ, as n a (n (ˆθ n, Y μ( θ, as n a (n (ˆθ n, Y =a (n ( θ, Y+(ˆθ n θ θ a(n (θ n, Y where θ n lies between ˆθ n and θ. Since θ a(n (θ n, Y < K, taing limitation on both sides of above equation, we get a (n ( θ, Y μ( θ, as n Under condition (4, θ must equal to θ 0. Since this is true for any limit point of the subsequence ˆθ n, i.e, all the subsequence converges to the same point, the result follows immediately. Theorem 3.1 established the strong convergence of the conditional moment estimator ˆθ n. The asymptotic normality of ˆθ n is presented in the following theorem. 42

4 Theorem 3.2. Provided the following conditions satisfied, (1 conditional sample moment a (n (θ, Y is approximately distributed as n{a (n (θ, Y μ(θ} L N(0, V θ where V θ is a positive matrix. (2 a (n 2 (θ, Y is twice differentiable with respect to θ, and θ θ a(n (θ, Y is bounded for each n and each θ in N θ0, a neighborhood of θ 0. (3 lim n a (n (θ, Y exists and the matrix M θ = lim n θ (a(n (θ, Y μ(θ is of full ran. Suppose ˆθ n is a consistent solution of the conditional moment equation (2.2. We have n(ˆθn θ 0 Σ 1/2 L N(0, Id where Σ 1/2 = M(θ 0 1 V(θ 0 1/2 and I d is a d d unit matrix. Proof. By the consistency of ˆθ n, we can focus on the case ˆθ n N θ0. Expand a (n (ˆθ n μ(ˆθ n at the point θ 0, a (n (ˆθ n μ(ˆθ n =a (n (θ 0, Y μ(θ 0 +(ˆθ n θ 0 θ {a(n (θ, Y μ(θ} θ=θ n, where θ n lies between θ 0 and ˆθ n. Note that a (n (ˆθ n μ(ˆθ n =0, n(ˆθn θ 0 θ {a(n (θ, Y μ(θ} θ=θ n = n{a (n (θ 0, Y μ(θ 0 }. 2 Since θ θ a(n (θ, Y is bounded for each n and each θ in N θ0, using similar arguments as in Theorem 3.1, it can be proved that θ {a(n P (θ, Y μ(θ} θ=θ n M(θ0 as n, where P means to converge in probability. Thus the result follows immediately from the asymptotic normality of a (n (θ, Y. The conditions in Theorem 3.1 and Theorem 3.2 seem a little too strict for the convergence and asymptotic normality of the estimator sequence ˆθ n. But some of the most commonly used incomplete samples such as randomly censored sample, doubly censored sample and grouped sample fall within the range of these conditions. This will be presented in Section Conditional Sample Moments of Censored and Grouped Samples In this section, we will investigate the conditional sample moments of three inds of frequently used incomplete samples including randomly censored sample, doubly Type-II censored sample and grouped sample. Although these three inds of incomplete samples can be investigated in a unified formula as in Dempster, Laird and Rubin (1977, we will not do that for the sae of explicit forms of the conditional sample moments. The convergence and asymptotic normality are displayed in detail. By Theorem 3.1 and Theorem 3.2, estimators by the method of conditional moments have the strong consistency and asymptotic normality. These results can be easily extended to more complicated settings such as multiply censored data. 43

5 3.2.1 Randomly Censoring In many prospective studies, survival data are often subject to random censoring. This typically comes up in the analysis of lifetime data. Under random censoring, rather than X = (X 1,,X n, the complete sample of interest, one observes Z i = min(x i,c i and δ i =1 (Xi C i i =1,,n where 1 (A is the indicator function of A and {C i } is another i.i.d random sequence independent of the {X i } sequence. Obviously, random censoring reduce to the usual Type-I censoring if all the C i s equal ( to a single specified real number. Z 1 Z 2 Z n Let Y =. Then Y is the observed incomplete sample. By the definition δ 1 δ 2 δ n of the conditional sample moments, the th non-central conditional sample moment of X is given by a (n (θ, Y = 1 {δ i Z 1 i + (1 δ i x df (x, θ} (3.1 n 1 F (Z i, θ Z i From the law of large numbers and central limit theorem, the following proposition follows immediately. Proposition 1. Assume 1 dg(x <, here G(x is the common distribution 1 F (x, θ function of C i s. Then (1 if x df (x, θ <, then a (n a.s (θ, Y μ (θ(as n. (2 if x2 df (x, θ <, then n(a (n (θ, Y μ (θ L N(0, 2, where 2 = x2ḡ(xdf (x, θ+ F (x, θ{ z x df (x, θ} 2 dg(x μ 2 (θ Doubly Type-II Censoring Let X =(X 1,,X n be a random sample of size n from distribution F (x, θ, which has a density function f(x, θ. Censor the r 1 smaller and the s largest observations, then the remaining observations Y =(X r:n,,x n s:n constitute a doubly Type-II censored sample, where X i:n is the i th smallest observation in X. Doubly Type-II censoring frequently occurs in survival analysis and other research fields. Tin, Tan and Balarishnan (1986 gave a detailed discussion of the application of doubly Type-II censoring in robust analysis. From (2.1, the th non-central conditional sample moment of X is given as follows (θ, Y = 1 n s n [ r 1 Xi:n + E{Xi:n Y} + a (n i=r = 1 n s n { Xi:n + r 1 F (X r:n,θ i=r Xr:n i=n s+1 x df (x, θ E{X i:n Y}] s + x df (x, θ} (3.2 1 F (X n s:n,θ n s:n Assume equation F (x, θ =p has unique solution ξ p (θ for each p (0, 1. Let i = p i (1 p i /f(ξ pi, θ (i =1, 2 44

6 ρ = p 1 (1 p 2 /p 2 (1 p 1 ( 1 2 ρ 1 2 Σ= ρ By the convergence and asymptotic normality of sample quantiles, the convergence and asymptotic normality of a (θ, Y can also be proved. Proposition 2. Assume (1 x2 df (x, θ < ; (2 there are 0 <p 1 <p 2 < 1, such that r n n = p 1 + o( 1 and n r n = p 1 + o( 1 ; n n n (3 f(x, θ is continuous at the points x = ξ pi (θ and f(ξ pi, θ is positive. Then a (n a.s (θ, Y μ (θ as n (n n(a (θ, Y μ (θ L N(0, 2 where 2 =[ ξ p2 ξ p1 u 2 1 ξp2 f(u, θdu { uf(u, θdu} 2 ] 1 p 2 p 1 The proof is trivial. ξ p Grouping In grouped (or interval censored data, each X i is nown only to lie between two nown constants a j and a j+1, here <a 0 a 1 a 2 a l a l+1 <. Grouped data is common in various fields of applied statistics. Denote the interval (a j,a j+1 by ( I j. Let r j be the number of the observations fell in interval I j. Then the observed data is Y = I 0 I 1 I l r 0 r 1 r l.by definition, the th non-central conditional sample moment of X is a (n (θ, Y = 1 n l r i { F (a j+l,θ F (a j, θ j=0 We rewrite this expression as follows a (n (θ, Y = 1 n l δ ij [ { F (a j+1, θ F (a j, θ j=0 aj+1 a j x df (x, θ} (3.3 aj+1 a j x df (x, θ}] where δ ij = 1 (Xi Ij. Thus, the convergence and asymptotic normality of a (θ, Y can be obtained from the law of large numbers and central limit theorem. Proposition 3. Assume F (a j+1, θ F (a j, θ > 0 for each j. (1 if x d(x, θ <, then a (n a.s. (θ, Y μ (θ, Y as n ; (2 if x2 d(x, θ <, then n(a (n (θ, Y μ (θ L N(0, 2 as n, where 2 = { a j+1 a j x df (x} 2 F (a j+1 F (a j μ2. j=0 45

7 4 Iterative Algorithm The method proposed in Section 2 can be applied to almost all inds of complicated incomplete sample cases as long as the required moments of the underlying population exist. Unfortunately, it is only for relatively simple situations that (2.2 has an explicit solution. The intractability of (2.2 results from F (x, θ Y, the conditional joint distribution function of X, which often has complicated form. Therefore, numerical iterative methods are required for solving the equation to obtain ˆθ. For this, general purpose root finding algorithm such as Newton-Raphson and quasi-newton algorithm are available. The Newton-Raphson algorithm is fast convergent but an analytical expression for the derivatives of a(θ, Y and μ(θ with respect to θ may not be obtained easily in some complicated situations. The deficiency of the quasi-newton algorithm in obtaining ˆθ has been noted in some application (Srivastava and Keen, In this section, we propose a new iterative procedure for solving (2.2 to obtain ˆθ. Suppose θ (p denotes the current value of θ after p step cycles of the algorithm. The next value of θ can be obtained by solving the following equation a(θ (p, Y =μ(θ (4.1 The above equation has the same form as (1.1, the moment equation, as if the whole sample X were observed. Hence θ (p+1 can be easily obtained. Obviously, we can decompose each cycle of this iterative procedure into two steps as follows. Step 1 : Calculate a(θ (p, Y, the conditional expectation of the sample moments of the unobserved complete sample X. Step 2 : Defining θ (p+1 as the solution of (4.1. One may find that these two steps are very similar to those of EM algorithm except that the left side of (4.1 is replaced by the log-lielihood in the latter. The following theorem established the convergence of the proposed iterative procedure. Illustrative examples are given in Section (5. Theorem 4.1. The sequence {θ (p } converges to ˆθ, if the following conditions satisfied. (1 (4.1 has unique solution. (2 a(θ, Y and μ(θ are continuous with respect to θ. Proof. Let {θ p } be any subsequence of {θ (p }. Since Θ is compact, there exists subsequence {θ p l } of {θ p } such that {θ p l } θ, where θ Θ. Since a(θ, Y is continuous about θ, a(θ (p l, Y a( θ, Y,n By (4.1 and continuity of μ(θ, wehave μ(θ (p +1 l μ( θ, (n So a( θ, Y =μ( θ i.e., θ is equivalent to the solution ˆθ from (2.2. Since sequence {θ (p } is arbitrary, the result follows. 46

8 5 Illustrative Examples In this section, we present three examples to illustrate the use of the method discussed in this paper. Example 1. Doubly censored Pareto distribution. Pareto distribution is widely used in economic research. It has been found that the claims on an insurance can be fitted well by Pareto distribution. Recently many researchers use Pareto distribution to fit the tail of the return rate of investments. The distribution function and density function of Pareto distribution are given by F (x, θ =1 (c/x θ and f(x, θ =θc θ /x θ+1 x>c>0,θ >0 where c is a nown positive constant and θ is an unnown parameter to be estimated. By the definition from Resni (1997 of a heavy tailed distribution, Pareto distribution is typically heavy tailed, and θ is the so-called tail indicator. Smaller θ implies a heavier tail. Other forms of the Pareto distribution can be found in Johnson, Kotz and Balarishnan (1994. The mean of Pareto distribution is μ 1 (θ =cθ/(θ 1 (θ >1 Suppose that the observed sample has the form x r+1:n < <x n s:n, that is doubly censored. According to (3.2, the conditional sample mean of X is given by a 1 (θ, Y = 1 n { rcθ θ 1 c 1 ( θ+1 x r+1:n c 1 ( } + n s θ r+1 x r+1:n x i:n + sx n s:nθ θ 1. Then the conditional moment estimator ˆθ of θ can be obtained by solving the conditional moment equation a 1 (θ, Y =μ 1 (θ. The maximum lielihood estimator θ is given by maximizing the following lielihood function L(θ {F (x r+1:n,θ} r n s i=r+1 f(x i:n,θ {1 F (x n s:n,θ} s We compare the conditional moment estimator and maximum lielihood estimator by a simulated study. These simulations are for fixed c =1and several groups of (n, r, s. For each (n, r, s the results are obtained by 5000 simulations. The simulated results in Table (1 are obtained through the proposed iterative algorithm in the previous section. Table (1 shows negligible difference between conditional moment estimator and maximum lielihood estimator. This implies that the performance of the conditional moment estimator is almost as efficient as the MLE s. Example 2. Grouped normal data. The second example concerns with the grouped normal data which has been discussed by several authors including Swan (1969 and Wolnetz (1979. Here we use the conditional method to estimate the unnown mean and standard deviation. Suppose the real line is divided into +1intervals (a 0,a 1, [a 1,a 2,, [a,a +1, here a 0 = 47

9 Table 1: Simulated comparison between the proposed method and maximum lielihood method for doubly censored Pareto distribution. ˆθ is the estimator from proposed method. θ is the estimator from maximum lielihood method. Rbias is relative bias of the simulated estimator. Rmse is the relative mean square error of simulated estimator. θ n r s Rbias(ˆθ Rbias( θ Rmse(ˆθ Rmse( θ

10 ,a +1 =. Let r i be the number of the observations fell in the interval (a i,a i+1. By (3.3 the first conditional sample moments of X are a 1 (μ,, Y =μ + n a 2 (μ,, Y = μ n i=0 i=0 3μ + 2 n φ( a i μ r i Φ( a i μ i=0 a i φ( a i μ r i Φ( a i+1 μ φ( a i+1 μ Φ( a i+1 μ φ( a i μ r i Φ( a i+1 μ φ( a i+1 μ a i+1 φ( a i+1 μ Φ( a i μ Φ( a i μ One can see that the above equations are equivalent to the lielihood ones. Therefore the conditional moment estimators of μ and equal the maximum lielihood estimators. Similarly, the proposed iterative algorithm is equivalent to the algorithm in this case. Example 3: Bivariate normal data with non-response in one variable. The purpose of this example is to estimate the mean vector μ and the covariance matrix Σ of a bivariate normal (X 1,X 2. The sample consists of m completed pairs of observations (x 11,x 12,, (x m1,x m2 and n m observed values x m+1,1,,x n,1 of X 1. Let μ =(μ 1,μ 2, ψ = vec(σ =( 11, 12, 22 and θ =(μ, ψ. The lielihood estimation for parameter θ has no closed form. This example was first discussed by Little and Rubin (2002. This is a typical missing data problem and the proposed estimate method in this paper can( be applied here. x 11 x 12 x 1n Denote by X the interested complete data and Y the observed incomplete data. From the property of multivariate nor- x 21 x 22 x ( 2n x 11 x 1m x 1m+1 x 1n x 21 x 2m.. mal distribution (Rao, 2001, the conditional distribution of X 1 given X 2 is N(μ (X 1 11 μ 1, Hence the first two conditional sample moments of X given Y is given by 11 E{ 1 n E{ 1 n E{ 1 n X i1 Y} = 1 n x i1, E( 1 n X1 Y 2 = 1 n X i2 Y} = 1 m n { x i2 + (n mμ E{ 1 n i=m+1 x 2 i1 (x i1 μ 1 } Xi2 Y} 2 = 1 m n [ x 2 i2 + (n m( i=m+1 X i1 X i2 Y} = 1 m n [ x i1 x i {μ (x i1 μ 1 } 2 ] 11 i=m+1 x i1 {μ (x i1 μ 1 }]

11 Thus we can construct conditional moment equations by equating the above conditional sample moments to the corresponding population moments and the conditional moment estimators for θ are given as follows. here x (h j = 1 h h x ij, S (h j ˆμ 1 = x (n 1, ˆ 11 = S (n 1, ˆ 12 = ρ (m S (n 1 /S (m 1 = 1 h ˆμ 2 = x (m 2 ρ(m S (m 1 ˆ 22 = S (m 2 (S (n 1 S (m h (x ij 1 h ( x (m 1 x (n 1 1 ( ρ(m S (m 2 1 h x ij 2, h is either n or m, ρ (m = 1 m x i1 x i2 m x (m 1 x (m 2. And thus the estimators of mean vector μ and covariance matrix Σ are derived. We can get the lielihood function of Y by n m f(y,μ, i=m+1 m ( ( 2 2 exp{ 1 m x i1 μ 1 Σ 1 x i1 μ 1 2 x i2 μ 2 x i2 μ 2 (x i1 μ 1 2 } This is equivalent to the following expression: n m f(y,μ, 2 1 trσ 1 ( m ( 2 2 exp{ 1 2 [m S (m 1 ρ (m ρ (m S (m 2 + n( x (n 1 μ 1 2 m( x (m 1 μ 1 2 ]} x (m ( 1 μ 1 x (m Σ 1 2 μ 2 + n m 1 2 (ns (n 1 ms (m 1 x (m 1 μ 1 x (m 2 μ 2 where tr is the trace of the matrix. By the factor decomposition theory, ( x (n 1, x(m 1, x (m 2,S (n 1,S (m 2,ρ (m are sufficient statistics. Compared to the maximum lielihood estimators, the conditional moment estimators of parameters have simple explicit forms and are functions of sufficient statistics. 6 Concluding Remars In this paper, we proposed the method of conditional moments as an extension of the traditional method of moments. This method is appealing since it can be used to obtain estimator for various complicated settings with incomplete data set. The conditional moment estimator might be not unique if different conditional sample moments are used. There is no optimal rule for the choice of the conditional sample moments system. We prefer the simplicity of the conditional moment equation (2.2 as a choice. The resulting estimator should be functions of sufficient statistics if possible. We can generalize the proposed method to incomplete time series and investigate the large sample properties similarly. 50

12 References Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977. Maximum lielihood from incomplete data via the em algorithm, Journal of the Royal Statistical Society 39: Johnson, N. L., Kotz, S. and Balarishnan, N. (1994. Continuous univariate distribution, John Wiley & Sons, New Yor. Little, R. J. and Rubin, D. B. (2002. Statistical Analysis with Missing Data, Wiley, New Yor. Mao, S. and Wang, L. (1997. Accelerated life testing, Science press of china. Rao, C. R Linear statistical inference and its application, John Wiley, New Yor. Resni, S. I. (1997. Heavy tail modeling and teletraffic data, Annals of Statistics 25: Srivastava, M. S. and Keen, K. J Estimation of the interclass correlation coefficient, Biometria 75: Swan, A. S. (1969. Algorithm as 16: Maximum lielihood estiamtion from grouped and cencored normal data, Applied statistics 18: Tin, M. L., Tan, W. Y. and Balarishnan, N. (1986. Robust inference, Marcel Deer, Inc. Wang, B. (1992. Statistical inference for weibull distribution, Chinese journal of applied probability and statistics 8: Wolnetz, M. S. (1979. Algorithm as 139: Maximum lielihood estiamtion from confined and cencored normal data, Applied statistics 28:

Analysis of Gamma and Weibull Lifetime Data under a General Censoring Scheme and in the presence of Covariates

Analysis of Gamma and Weibull Lifetime Data under a General Censoring Scheme and in the presence of Covariates Communications in Statistics - Theory and Methods ISSN: 0361-0926 (Print) 1532-415X (Online) Journal homepage: http://www.tandfonline.com/loi/lsta20 Analysis of Gamma and Weibull Lifetime Data under a

More information

Latent Variable Models and EM algorithm

Latent Variable Models and EM algorithm Latent Variable Models and EM algorithm SC4/SM4 Data Mining and Machine Learning, Hilary Term 2017 Dino Sejdinovic 3.1 Clustering and Mixture Modelling K-means and hierarchical clustering are non-probabilistic

More information

Optimization. The value x is called a maximizer of f and is written argmax X f. g(λx + (1 λ)y) < λg(x) + (1 λ)g(y) 0 < λ < 1; x, y X.

Optimization. The value x is called a maximizer of f and is written argmax X f. g(λx + (1 λ)y) < λg(x) + (1 λ)g(y) 0 < λ < 1; x, y X. Optimization Background: Problem: given a function f(x) defined on X, find x such that f(x ) f(x) for all x X. The value x is called a maximizer of f and is written argmax X f. In general, argmax X f may

More information

EM Algorithm II. September 11, 2018

EM Algorithm II. September 11, 2018 EM Algorithm II September 11, 2018 Review EM 1/27 (Y obs, Y mis ) f (y obs, y mis θ), we observe Y obs but not Y mis Complete-data log likelihood: l C (θ Y obs, Y mis ) = log { f (Y obs, Y mis θ) Observed-data

More information

Parameter Estimation of Power Lomax Distribution Based on Type-II Progressively Hybrid Censoring Scheme

Parameter Estimation of Power Lomax Distribution Based on Type-II Progressively Hybrid Censoring Scheme Applied Mathematical Sciences, Vol. 12, 2018, no. 18, 879-891 HIKARI Ltd, www.m-hikari.com https://doi.org/10.12988/ams.2018.8691 Parameter Estimation of Power Lomax Distribution Based on Type-II Progressively

More information

Estimating the parameters of hidden binomial trials by the EM algorithm

Estimating the parameters of hidden binomial trials by the EM algorithm Hacettepe Journal of Mathematics and Statistics Volume 43 (5) (2014), 885 890 Estimating the parameters of hidden binomial trials by the EM algorithm Degang Zhu Received 02 : 09 : 2013 : Accepted 02 :

More information

Ph.D. Qualifying Exam Friday Saturday, January 6 7, 2017

Ph.D. Qualifying Exam Friday Saturday, January 6 7, 2017 Ph.D. Qualifying Exam Friday Saturday, January 6 7, 2017 Put your solution to each problem on a separate sheet of paper. Problem 1. (5106) Let X 1, X 2,, X n be a sequence of i.i.d. observations from a

More information

COMPETING RISKS WEIBULL MODEL: PARAMETER ESTIMATES AND THEIR ACCURACY

COMPETING RISKS WEIBULL MODEL: PARAMETER ESTIMATES AND THEIR ACCURACY Annales Univ Sci Budapest, Sect Comp 45 2016) 45 55 COMPETING RISKS WEIBULL MODEL: PARAMETER ESTIMATES AND THEIR ACCURACY Ágnes M Kovács Budapest, Hungary) Howard M Taylor Newark, DE, USA) Communicated

More information

Type II Bivariate Generalized Power Series Poisson Distribution and its Applications in Risk Analysis

Type II Bivariate Generalized Power Series Poisson Distribution and its Applications in Risk Analysis Type II Bivariate Generalized Power Series Poisson Distribution and its Applications in Ris Analysis KK Jose a, and Shalitha Jacob a,b a Department of Statistics, St Thomas College, Pala, Arunapuram, Kerala-686574,

More information

For iid Y i the stronger conclusion holds; for our heuristics ignore differences between these notions.

For iid Y i the stronger conclusion holds; for our heuristics ignore differences between these notions. Large Sample Theory Study approximate behaviour of ˆθ by studying the function U. Notice U is sum of independent random variables. Theorem: If Y 1, Y 2,... are iid with mean µ then Yi n µ Called law of

More information

A New Class of Positively Quadrant Dependent Bivariate Distributions with Pareto

A New Class of Positively Quadrant Dependent Bivariate Distributions with Pareto International Mathematical Forum, 2, 27, no. 26, 1259-1273 A New Class of Positively Quadrant Dependent Bivariate Distributions with Pareto A. S. Al-Ruzaiza and Awad El-Gohary 1 Department of Statistics

More information

ON THE TRUNCATED COMPOSITE WEIBULL-PARETO MODEL

ON THE TRUNCATED COMPOSITE WEIBULL-PARETO MODEL ON THE TRUNCATED COMPOSITE WEIBULL-PARETO MODEL SANDRA TEODORESCU and EUGENIA PANAITESCU The composite Weibull-Pareto model 2] was introduced as an alternative to the composite Lognormal-Pareto ] used

More information

Maximum Likelihood Estimation. only training data is available to design a classifier

Maximum Likelihood Estimation. only training data is available to design a classifier Introduction to Pattern Recognition [ Part 5 ] Mahdi Vasighi Introduction Bayesian Decision Theory shows that we could design an optimal classifier if we knew: P( i ) : priors p(x i ) : class-conditional

More information

Lecture 3. Inference about multivariate normal distribution

Lecture 3. Inference about multivariate normal distribution Lecture 3. Inference about multivariate normal distribution 3.1 Point and Interval Estimation Let X 1,..., X n be i.i.d. N p (µ, Σ). We are interested in evaluation of the maximum likelihood estimates

More information

Statistical Estimation

Statistical Estimation Statistical Estimation Use data and a model. The plug-in estimators are based on the simple principle of applying the defining functional to the ECDF. Other methods of estimation: minimize residuals from

More information

On nonlinear weighted least squares fitting of the three-parameter inverse Weibull distribution

On nonlinear weighted least squares fitting of the three-parameter inverse Weibull distribution MATHEMATICAL COMMUNICATIONS 13 Math. Commun., Vol. 15, No. 1, pp. 13-24 (2010) On nonlinear weighted least squares fitting of the three-parameter inverse Weibull distribution Dragan Juić 1 and Darija Marović

More information

Lecture 4: Probabilistic Learning. Estimation Theory. Classification with Probability Distributions

Lecture 4: Probabilistic Learning. Estimation Theory. Classification with Probability Distributions DD2431 Autumn, 2014 1 2 3 Classification with Probability Distributions Estimation Theory Classification in the last lecture we assumed we new: P(y) Prior P(x y) Lielihood x2 x features y {ω 1,..., ω K

More information

Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.

Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A. 1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n

More information

Gaussian Mixture Distance for Information Retrieval

Gaussian Mixture Distance for Information Retrieval Gaussian Mixture Distance for Information Retrieval X.Q. Li and I. King fxqli, ingg@cse.cuh.edu.h Department of omputer Science & Engineering The hinese University of Hong Kong Shatin, New Territories,

More information

Submitted to the Brazilian Journal of Probability and Statistics

Submitted to the Brazilian Journal of Probability and Statistics Submitted to the Brazilian Journal of Probability and Statistics Multivariate normal approximation of the maximum likelihood estimator via the delta method Andreas Anastasiou a and Robert E. Gaunt b a

More information

Parametric Techniques Lecture 3

Parametric Techniques Lecture 3 Parametric Techniques Lecture 3 Jason Corso SUNY at Buffalo 22 January 2009 J. Corso (SUNY at Buffalo) Parametric Techniques Lecture 3 22 January 2009 1 / 39 Introduction In Lecture 2, we learned how to

More information

Biostat 2065 Analysis of Incomplete Data

Biostat 2065 Analysis of Incomplete Data Biostat 2065 Analysis of Incomplete Data Gong Tang Dept of Biostatistics University of Pittsburgh October 20, 2005 1. Large-sample inference based on ML Let θ is the MLE, then the large-sample theory implies

More information

The Use of Copulas to Model Conditional Expectation for Multivariate Data

The Use of Copulas to Model Conditional Expectation for Multivariate Data The Use of Copulas to Model Conditional Expectation for Multivariate Data Kääri, Meelis; Selart, Anne; Kääri, Ene University of Tartu, Institute of Mathematical Statistics J. Liivi 2 50409 Tartu, Estonia

More information

Tail negative dependence and its applications for aggregate loss modeling

Tail negative dependence and its applications for aggregate loss modeling Tail negative dependence and its applications for aggregate loss modeling Lei Hua Division of Statistics Oct 20, 2014, ISU L. Hua (NIU) 1/35 1 Motivation 2 Tail order Elliptical copula Extreme value copula

More information

Chapter 3 : Likelihood function and inference

Chapter 3 : Likelihood function and inference Chapter 3 : Likelihood function and inference 4 Likelihood function and inference The likelihood Information and curvature Sufficiency and ancilarity Maximum likelihood estimation Non-regular models EM

More information

Goodness-of-fit tests for randomly censored Weibull distributions with estimated parameters

Goodness-of-fit tests for randomly censored Weibull distributions with estimated parameters Communications for Statistical Applications and Methods 2017, Vol. 24, No. 5, 519 531 https://doi.org/10.5351/csam.2017.24.5.519 Print ISSN 2287-7843 / Online ISSN 2383-4757 Goodness-of-fit tests for randomly

More information

Optimum Test Plan for 3-Step, Step-Stress Accelerated Life Tests

Optimum Test Plan for 3-Step, Step-Stress Accelerated Life Tests International Journal of Performability Engineering, Vol., No., January 24, pp.3-4. RAMS Consultants Printed in India Optimum Test Plan for 3-Step, Step-Stress Accelerated Life Tests N. CHANDRA *, MASHROOR

More information

Parametric Techniques

Parametric Techniques Parametric Techniques Jason J. Corso SUNY at Buffalo J. Corso (SUNY at Buffalo) Parametric Techniques 1 / 39 Introduction When covering Bayesian Decision Theory, we assumed the full probabilistic structure

More information

STAT 730 Chapter 4: Estimation

STAT 730 Chapter 4: Estimation STAT 730 Chapter 4: Estimation Timothy Hanson Department of Statistics, University of South Carolina Stat 730: Multivariate Analysis 1 / 23 The likelihood We have iid data, at least initially. Each datum

More information

ABC methods for phase-type distributions with applications in insurance risk problems

ABC methods for phase-type distributions with applications in insurance risk problems ABC methods for phase-type with applications problems Concepcion Ausin, Department of Statistics, Universidad Carlos III de Madrid Joint work with: Pedro Galeano, Universidad Carlos III de Madrid Simon

More information

Computational Statistics and Data Analysis. Estimation for the three-parameter lognormal distribution based on progressively censored data

Computational Statistics and Data Analysis. Estimation for the three-parameter lognormal distribution based on progressively censored data Computational Statistics and Data Analysis 53 (9) 358 359 Contents lists available at ScienceDirect Computational Statistics and Data Analysis journal homepage: www.elsevier.com/locate/csda stimation for

More information

Estimation of the Parameters of Bivariate Normal Distribution with Equal Coefficient of Variation Using Concomitants of Order Statistics

Estimation of the Parameters of Bivariate Normal Distribution with Equal Coefficient of Variation Using Concomitants of Order Statistics International Journal of Statistics and Probability; Vol., No. 3; 013 ISSN 197-703 E-ISSN 197-7040 Published by Canadian Center of Science and Education Estimation of the Parameters of Bivariate Normal

More information

Label Switching and Its Simple Solutions for Frequentist Mixture Models

Label Switching and Its Simple Solutions for Frequentist Mixture Models Label Switching and Its Simple Solutions for Frequentist Mixture Models Weixin Yao Department of Statistics, Kansas State University, Manhattan, Kansas 66506, U.S.A. wxyao@ksu.edu Abstract The label switching

More information

Chapter 3. Point Estimation. 3.1 Introduction

Chapter 3. Point Estimation. 3.1 Introduction Chapter 3 Point Estimation Let (Ω, A, P θ ), P θ P = {P θ θ Θ}be probability space, X 1, X 2,..., X n : (Ω, A) (IR k, B k ) random variables (X, B X ) sample space γ : Θ IR k measurable function, i.e.

More information

Testing a Normal Covariance Matrix for Small Samples with Monotone Missing Data

Testing a Normal Covariance Matrix for Small Samples with Monotone Missing Data Applied Mathematical Sciences, Vol 3, 009, no 54, 695-70 Testing a Normal Covariance Matrix for Small Samples with Monotone Missing Data Evelina Veleva Rousse University A Kanchev Department of Numerical

More information

P n. This is called the law of large numbers but it comes in two forms: Strong and Weak.

P n. This is called the law of large numbers but it comes in two forms: Strong and Weak. Large Sample Theory Large Sample Theory is a name given to the search for approximations to the behaviour of statistical procedures which are derived by computing limits as the sample size, n, tends to

More information

Estimation Under Multivariate Inverse Weibull Distribution

Estimation Under Multivariate Inverse Weibull Distribution Global Journal of Pure and Applied Mathematics. ISSN 097-768 Volume, Number 8 (07), pp. 4-4 Research India Publications http://www.ripublication.com Estimation Under Multivariate Inverse Weibull Distribution

More information

An Empirical Characteristic Function Approach to Selecting a Transformation to Normality

An Empirical Characteristic Function Approach to Selecting a Transformation to Normality Communications for Statistical Applications and Methods 014, Vol. 1, No. 3, 13 4 DOI: http://dx.doi.org/10.5351/csam.014.1.3.13 ISSN 87-7843 An Empirical Characteristic Function Approach to Selecting a

More information

Constant Stress Partially Accelerated Life Test Design for Inverted Weibull Distribution with Type-I Censoring

Constant Stress Partially Accelerated Life Test Design for Inverted Weibull Distribution with Type-I Censoring Algorithms Research 013, (): 43-49 DOI: 10.593/j.algorithms.01300.0 Constant Stress Partially Accelerated Life Test Design for Mustafa Kamal *, Shazia Zarrin, Arif-Ul-Islam Department of Statistics & Operations

More information

Accelerating the EM Algorithm for Mixture Density Estimation

Accelerating the EM Algorithm for Mixture Density Estimation Accelerating the EM Algorithm ICERM Workshop September 4, 2015 Slide 1/18 Accelerating the EM Algorithm for Mixture Density Estimation Homer Walker Mathematical Sciences Department Worcester Polytechnic

More information

Streamlining Missing Data Analysis by Aggregating Multiple Imputations at the Data Level

Streamlining Missing Data Analysis by Aggregating Multiple Imputations at the Data Level Streamlining Missing Data Analysis by Aggregating Multiple Imputations at the Data Level A Monte Carlo Simulation to Test the Tenability of the SuperMatrix Approach Kyle M Lang Quantitative Psychology

More information

Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach

Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Observed likelihood 3 Mean Score

More information

Multivariate Differentiation 1

Multivariate Differentiation 1 John Nachbar Washington University February 23, 2017 1 Preliminaries. Multivariate Differentiation 1 I assume that you are already familiar with standard concepts and results from univariate calculus;

More information

On Mixture Regression Shrinkage and Selection via the MR-LASSO

On Mixture Regression Shrinkage and Selection via the MR-LASSO On Mixture Regression Shrinage and Selection via the MR-LASSO Ronghua Luo, Hansheng Wang, and Chih-Ling Tsai Guanghua School of Management, Peing University & Graduate School of Management, University

More information

Inequalities Relating Addition and Replacement Type Finite Sample Breakdown Points

Inequalities Relating Addition and Replacement Type Finite Sample Breakdown Points Inequalities Relating Addition and Replacement Type Finite Sample Breadown Points Robert Serfling Department of Mathematical Sciences University of Texas at Dallas Richardson, Texas 75083-0688, USA Email:

More information

The Type I Generalized Half Logistic Distribution Downloaded from jirss.irstat.ir at 11: on Monday August 20th 2018

The Type I Generalized Half Logistic Distribution Downloaded from jirss.irstat.ir at 11: on Monday August 20th 2018 JIRSS (2014) Vol. 13, No. 1, pp 69-82 The Type I Generalized Half Logistic Distribution A. K. Olapade Department of Mathematics, Obafemi Awolowo University, Ile-Ife, Nigeria. Abstract. In this paper, we

More information

Estimating Gaussian Mixture Densities with EM A Tutorial

Estimating Gaussian Mixture Densities with EM A Tutorial Estimating Gaussian Mixture Densities with EM A Tutorial Carlo Tomasi Due University Expectation Maximization (EM) [4, 3, 6] is a numerical algorithm for the maximization of functions of several variables

More information

OPTIMAL B-ROBUST ESTIMATORS FOR THE PARAMETERS OF THE GENERALIZED HALF-NORMAL DISTRIBUTION

OPTIMAL B-ROBUST ESTIMATORS FOR THE PARAMETERS OF THE GENERALIZED HALF-NORMAL DISTRIBUTION REVSTAT Statistical Journal Volume 15, Number 3, July 2017, 455 471 OPTIMAL B-ROBUST ESTIMATORS FOR THE PARAMETERS OF THE GENERALIZED HALF-NORMAL DISTRIBUTION Authors: Fatma Zehra Doğru Department of Econometrics,

More information

Statistics 3858 : Maximum Likelihood Estimators

Statistics 3858 : Maximum Likelihood Estimators Statistics 3858 : Maximum Likelihood Estimators 1 Method of Maximum Likelihood In this method we construct the so called likelihood function, that is L(θ) = L(θ; X 1, X 2,..., X n ) = f n (X 1, X 2,...,

More information

Parameter Estimation

Parameter Estimation Parameter Estimation Chapters 13-15 Stat 477 - Loss Models Chapters 13-15 (Stat 477) Parameter Estimation Brian Hartman - BYU 1 / 23 Methods for parameter estimation Methods for parameter estimation Methods

More information

Lifetime Dependence Modelling using a Generalized Multivariate Pareto Distribution

Lifetime Dependence Modelling using a Generalized Multivariate Pareto Distribution Lifetime Dependence Modelling using a Generalized Multivariate Pareto Distribution Daniel Alai Zinoviy Landsman Centre of Excellence in Population Ageing Research (CEPAR) School of Mathematics, Statistics

More information

An Introduction to Expectation-Maximization

An Introduction to Expectation-Maximization An Introduction to Expectation-Maximization Dahua Lin Abstract This notes reviews the basics about the Expectation-Maximization EM) algorithm, a popular approach to perform model estimation of the generative

More information

STAT 512 sp 2018 Summary Sheet

STAT 512 sp 2018 Summary Sheet STAT 5 sp 08 Summary Sheet Karl B. Gregory Spring 08. Transformations of a random variable Let X be a rv with support X and let g be a function mapping X to Y with inverse mapping g (A = {x X : g(x A}

More information

Estimation for Mean and Standard Deviation of Normal Distribution under Type II Censoring

Estimation for Mean and Standard Deviation of Normal Distribution under Type II Censoring Communications for Statistical Applications and Methods 2014, Vol. 21, No. 6, 529 538 DOI: http://dx.doi.org/10.5351/csam.2014.21.6.529 Print ISSN 2287-7843 / Online ISSN 2383-4757 Estimation for Mean

More information

Empirical likelihood ratio with arbitrarily censored/truncated data by EM algorithm

Empirical likelihood ratio with arbitrarily censored/truncated data by EM algorithm Empirical likelihood ratio with arbitrarily censored/truncated data by EM algorithm Mai Zhou 1 University of Kentucky, Lexington, KY 40506 USA Summary. Empirical likelihood ratio method (Thomas and Grunkmier

More information

Mean. Pranab K. Mitra and Bimal K. Sinha. Department of Mathematics and Statistics, University Of Maryland, Baltimore County

Mean. Pranab K. Mitra and Bimal K. Sinha. Department of Mathematics and Statistics, University Of Maryland, Baltimore County A Generalized p-value Approach to Inference on Common Mean Pranab K. Mitra and Bimal K. Sinha Department of Mathematics and Statistics, University Of Maryland, Baltimore County 1000 Hilltop Circle, Baltimore,

More information

FULL LIKELIHOOD INFERENCES IN THE COX MODEL

FULL LIKELIHOOD INFERENCES IN THE COX MODEL October 20, 2007 FULL LIKELIHOOD INFERENCES IN THE COX MODEL BY JIAN-JIAN REN 1 AND MAI ZHOU 2 University of Central Florida and University of Kentucky Abstract We use the empirical likelihood approach

More information

Maximum Likelihood Estimation

Maximum Likelihood Estimation Chapter 8 Maximum Likelihood Estimation 8. Consistency If X is a random variable (or vector) with density or mass function f θ (x) that depends on a parameter θ, then the function f θ (X) viewed as a function

More information

Bayes Prediction Bounds for Right Ordered Pareto Type - II Data

Bayes Prediction Bounds for Right Ordered Pareto Type - II Data J. Stat. Appl. Pro. 3, No. 3, 335-343 014 335 Journal of Statistics Applications & Probability An International Journal http://dx.doi.org/10.1785/jsap/030304 Bayes Prediction Bounds for Right Ordered Pareto

More information

Part IA Probability. Theorems. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015

Part IA Probability. Theorems. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015 Part IA Probability Theorems Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.

More information

4. CONTINUOUS RANDOM VARIABLES

4. CONTINUOUS RANDOM VARIABLES IA Probability Lent Term 4 CONTINUOUS RANDOM VARIABLES 4 Introduction Up to now we have restricted consideration to sample spaces Ω which are finite, or countable; we will now relax that assumption We

More information

Introduction An approximated EM algorithm Simulation studies Discussion

Introduction An approximated EM algorithm Simulation studies Discussion 1 / 33 An Approximated Expectation-Maximization Algorithm for Analysis of Data with Missing Values Gong Tang Department of Biostatistics, GSPH University of Pittsburgh NISS Workshop on Nonignorable Nonresponse

More information

Exact Inference for the Two-Parameter Exponential Distribution Under Type-II Hybrid Censoring

Exact Inference for the Two-Parameter Exponential Distribution Under Type-II Hybrid Censoring Exact Inference for the Two-Parameter Exponential Distribution Under Type-II Hybrid Censoring A. Ganguly, S. Mitra, D. Samanta, D. Kundu,2 Abstract Epstein [9] introduced the Type-I hybrid censoring scheme

More information

Statistics 135 Fall 2007 Midterm Exam

Statistics 135 Fall 2007 Midterm Exam Name: Student ID Number: Statistics 135 Fall 007 Midterm Exam Ignore the finite population correction in all relevant problems. The exam is closed book, but some possibly useful facts about probability

More information

5.1 Consistency of least squares estimates. We begin with a few consistency results that stand on their own and do not depend on normality.

5.1 Consistency of least squares estimates. We begin with a few consistency results that stand on their own and do not depend on normality. 88 Chapter 5 Distribution Theory In this chapter, we summarize the distributions related to the normal distribution that occur in linear models. Before turning to this general problem that assumes normal

More information

Small area estimation with missing data using a multivariate linear random effects model

Small area estimation with missing data using a multivariate linear random effects model Department of Mathematics Small area estimation with missing data using a multivariate linear random effects model Innocent Ngaruye, Dietrich von Rosen and Martin Singull LiTH-MAT-R--2017/07--SE Department

More information

Primal-dual Covariate Balance and Minimal Double Robustness via Entropy Balancing

Primal-dual Covariate Balance and Minimal Double Robustness via Entropy Balancing Primal-dual Covariate Balance and Minimal Double Robustness via (Joint work with Daniel Percival) Department of Statistics, Stanford University JSM, August 9, 2015 Outline 1 2 3 1/18 Setting Rubin s causal

More information

Brief Review on Estimation Theory

Brief Review on Estimation Theory Brief Review on Estimation Theory K. Abed-Meraim ENST PARIS, Signal and Image Processing Dept. abed@tsi.enst.fr This presentation is essentially based on the course BASTA by E. Moulines Brief review on

More information

TAMS39 Lecture 2 Multivariate normal distribution

TAMS39 Lecture 2 Multivariate normal distribution TAMS39 Lecture 2 Multivariate normal distribution Martin Singull Department of Mathematics Mathematical Statistics Linköping University, Sweden Content Lecture Random vectors Multivariate normal distribution

More information

A note on multiple imputation for general purpose estimation

A note on multiple imputation for general purpose estimation A note on multiple imputation for general purpose estimation Shu Yang Jae Kwang Kim SSC meeting June 16, 2015 Shu Yang, Jae Kwang Kim Multiple Imputation June 16, 2015 1 / 32 Introduction Basic Setup Assume

More information

Lecture 25: Review. Statistics 104. April 23, Colin Rundel

Lecture 25: Review. Statistics 104. April 23, Colin Rundel Lecture 25: Review Statistics 104 Colin Rundel April 23, 2012 Joint CDF F (x, y) = P [X x, Y y] = P [(X, Y ) lies south-west of the point (x, y)] Y (x,y) X Statistics 104 (Colin Rundel) Lecture 25 April

More information

Testing Hypothesis. Maura Mezzetti. Department of Economics and Finance Università Tor Vergata

Testing Hypothesis. Maura Mezzetti. Department of Economics and Finance Università Tor Vergata Maura Department of Economics and Finance Università Tor Vergata Hypothesis Testing Outline It is a mistake to confound strangeness with mystery Sherlock Holmes A Study in Scarlet Outline 1 The Power Function

More information

Overview of Extreme Value Theory. Dr. Sawsan Hilal space

Overview of Extreme Value Theory. Dr. Sawsan Hilal space Overview of Extreme Value Theory Dr. Sawsan Hilal space Maths Department - University of Bahrain space November 2010 Outline Part-1: Univariate Extremes Motivation Threshold Exceedances Part-2: Bivariate

More information

Survival Analysis for Case-Cohort Studies

Survival Analysis for Case-Cohort Studies Survival Analysis for ase-ohort Studies Petr Klášterecký Dept. of Probability and Mathematical Statistics, Faculty of Mathematics and Physics, harles University, Prague, zech Republic e-mail: petr.klasterecky@matfyz.cz

More information

Clustering by Mixture Models. General background on clustering Example method: k-means Mixture model based clustering Model estimation

Clustering by Mixture Models. General background on clustering Example method: k-means Mixture model based clustering Model estimation Clustering by Mixture Models General bacground on clustering Example method: -means Mixture model based clustering Model estimation 1 Clustering A basic tool in data mining/pattern recognition: Divide

More information

1 EM algorithm: updating the mixing proportions {π k } ik are the posterior probabilities at the qth iteration of EM.

1 EM algorithm: updating the mixing proportions {π k } ik are the posterior probabilities at the qth iteration of EM. Université du Sud Toulon - Var Master Informatique Probabilistic Learning and Data Analysis TD: Model-based clustering by Faicel CHAMROUKHI Solution The aim of this practical wor is to show how the Classification

More information

Negative Multinomial Model and Cancer. Incidence

Negative Multinomial Model and Cancer. Incidence Generalized Linear Model under the Extended Negative Multinomial Model and Cancer Incidence S. Lahiri & Sunil K. Dhar Department of Mathematical Sciences, CAMS New Jersey Institute of Technology, Newar,

More information

Experience Rating in General Insurance by Credibility Estimation

Experience Rating in General Insurance by Credibility Estimation Experience Rating in General Insurance by Credibility Estimation Xian Zhou Department of Applied Finance and Actuarial Studies Macquarie University, Sydney, Australia Abstract This work presents a new

More information

ESTIMATING BIVARIATE TAIL

ESTIMATING BIVARIATE TAIL Elena DI BERNARDINO b joint work with Clémentine PRIEUR a and Véronique MAUME-DESCHAMPS b a LJK, Université Joseph Fourier, Grenoble 1 b Laboratoire SAF, ISFA, Université Lyon 1 Framework Goal: estimating

More information

The properties of L p -GMM estimators

The properties of L p -GMM estimators The properties of L p -GMM estimators Robert de Jong and Chirok Han Michigan State University February 2000 Abstract This paper considers Generalized Method of Moment-type estimators for which a criterion

More information

Extreme Value Analysis and Spatial Extremes

Extreme Value Analysis and Spatial Extremes Extreme Value Analysis and Department of Statistics Purdue University 11/07/2013 Outline Motivation 1 Motivation 2 Extreme Value Theorem and 3 Bayesian Hierarchical Models Copula Models Max-stable Models

More information

Inference for the Positive Stable Laws Based on a Special Quadratic Distance

Inference for the Positive Stable Laws Based on a Special Quadratic Distance Inference for the Positive Stable Laws Based on a Special Quadratic Distance Andrew Luong* and Louis G. Doray** 1 *École d actuariat, Université Laval, Cité Universitaire, Ste-Foy, Québec, Canada G1K 7P4

More information

Spring 2012 Math 541B Exam 1

Spring 2012 Math 541B Exam 1 Spring 2012 Math 541B Exam 1 1. A sample of size n is drawn without replacement from an urn containing N balls, m of which are red and N m are black; the balls are otherwise indistinguishable. Let X denote

More information

On Bicomplex Nets and their Confinements

On Bicomplex Nets and their Confinements American Journal of Mathematics and Statistics 0; (): 8-6 DOI: 0 593/jajms0000 On Bicomplex Nets and their Confinements Rajiv K Srivastava *, S Singh Department of Mathematics, Institute of Basic Science,

More information

1 Degree distributions and data

1 Degree distributions and data 1 Degree distributions and data A great deal of effort is often spent trying to identify what functional form best describes the degree distribution of a network, particularly the upper tail of that distribution.

More information

Estimation of the functional Weibull-tail coefficient

Estimation of the functional Weibull-tail coefficient 1/ 29 Estimation of the functional Weibull-tail coefficient Stéphane Girard Inria Grenoble Rhône-Alpes & LJK, France http://mistis.inrialpes.fr/people/girard/ June 2016 joint work with Laurent Gardes,

More information

Multivariate Assays With Values Below the Lower Limit of Quantitation: Parametric Estimation By Imputation and Maximum Likelihood

Multivariate Assays With Values Below the Lower Limit of Quantitation: Parametric Estimation By Imputation and Maximum Likelihood Multivariate Assays With Values Below the Lower Limit of Quantitation: Parametric Estimation By Imputation and Maximum Likelihood Robert E. Johnson and Heather J. Hoffman 2* Department of Biostatistics,

More information

Fractional Hot Deck Imputation for Robust Inference Under Item Nonresponse in Survey Sampling

Fractional Hot Deck Imputation for Robust Inference Under Item Nonresponse in Survey Sampling Fractional Hot Deck Imputation for Robust Inference Under Item Nonresponse in Survey Sampling Jae-Kwang Kim 1 Iowa State University June 26, 2013 1 Joint work with Shu Yang Introduction 1 Introduction

More information

Asymptotic Statistics-III. Changliang Zou

Asymptotic Statistics-III. Changliang Zou Asymptotic Statistics-III Changliang Zou The multivariate central limit theorem Theorem (Multivariate CLT for iid case) Let X i be iid random p-vectors with mean µ and and covariance matrix Σ. Then n (

More information

Notes, March 4, 2013, R. Dudley Maximum likelihood estimation: actual or supposed

Notes, March 4, 2013, R. Dudley Maximum likelihood estimation: actual or supposed 18.466 Notes, March 4, 2013, R. Dudley Maximum likelihood estimation: actual or supposed 1. MLEs in exponential families Let f(x,θ) for x X and θ Θ be a likelihood function, that is, for present purposes,

More information

Expectation Maximization (EM) Algorithm. Each has it s own probability of seeing H on any one flip. Let. p 1 = P ( H on Coin 1 )

Expectation Maximization (EM) Algorithm. Each has it s own probability of seeing H on any one flip. Let. p 1 = P ( H on Coin 1 ) Expectation Maximization (EM Algorithm Motivating Example: Have two coins: Coin 1 and Coin 2 Each has it s own probability of seeing H on any one flip. Let p 1 = P ( H on Coin 1 p 2 = P ( H on Coin 2 Select

More information

Last lecture 1/35. General optimization problems Newton Raphson Fisher scoring Quasi Newton

Last lecture 1/35. General optimization problems Newton Raphson Fisher scoring Quasi Newton EM Algorithm Last lecture 1/35 General optimization problems Newton Raphson Fisher scoring Quasi Newton Nonlinear regression models Gauss-Newton Generalized linear models Iteratively reweighted least squares

More information

Nonparametric Function Estimation with Infinite-Order Kernels

Nonparametric Function Estimation with Infinite-Order Kernels Nonparametric Function Estimation with Infinite-Order Kernels Arthur Berg Department of Statistics, University of Florida March 15, 2008 Kernel Density Estimation (IID Case) Let X 1,..., X n iid density

More information

A Conditional Approach to Modeling Multivariate Extremes

A Conditional Approach to Modeling Multivariate Extremes A Approach to ing Multivariate Extremes By Heffernan & Tawn Department of Statistics Purdue University s April 30, 2014 Outline s s Multivariate Extremes s A central aim of multivariate extremes is trying

More information

Tail Dependence of Multivariate Pareto Distributions

Tail Dependence of Multivariate Pareto Distributions !#"%$ & ' ") * +!-,#. /10 243537698:6 ;=@?A BCDBFEHGIBJEHKLB MONQP RS?UTV=XW>YZ=eda gihjlknmcoqprj stmfovuxw yy z {} ~ ƒ }ˆŠ ~Œ~Ž f ˆ ` š œžÿ~ ~Ÿ œ } ƒ œ ˆŠ~ œ

More information

Maximum Likelihood Estimation

Maximum Likelihood Estimation Chapter 7 Maximum Likelihood Estimation 7. Consistency If X is a random variable (or vector) with density or mass function f θ (x) that depends on a parameter θ, then the function f θ (X) viewed as a function

More information

Discrete Dependent Variable Models

Discrete Dependent Variable Models Discrete Dependent Variable Models James J. Heckman University of Chicago This draft, April 10, 2006 Here s the general approach of this lecture: Economic model Decision rule (e.g. utility maximization)

More information

CS Lecture 18. Expectation Maximization

CS Lecture 18. Expectation Maximization CS 6347 Lecture 18 Expectation Maximization Unobserved Variables Latent or hidden variables in the model are never observed We may or may not be interested in their values, but their existence is crucial

More information

Multiscale Systems Engineering Research Group

Multiscale Systems Engineering Research Group Hidden Markov Model Prof. Yan Wang Woodruff School of Mechanical Engineering Georgia Institute of echnology Atlanta, GA 30332, U.S.A. yan.wang@me.gatech.edu Learning Objectives o familiarize the hidden

More information

Lasso Maximum Likelihood Estimation of Parametric Models with Singular Information Matrices

Lasso Maximum Likelihood Estimation of Parametric Models with Singular Information Matrices Article Lasso Maximum Likelihood Estimation of Parametric Models with Singular Information Matrices Fei Jin 1,2 and Lung-fei Lee 3, * 1 School of Economics, Shanghai University of Finance and Economics,

More information