GEE ESTIMATORS IN MIXTURE MODEL WITH VARYING CONCENTRATIONS

Size: px

Start display at page:

Download "GEE ESTIMATORS IN MIXTURE MODEL WITH VARYING CONCENTRATIONS"

Asher Owen
6 years ago
Views:

1 ACTA UIVERSITATIS LODZIESIS FOLIA OECOOMICA 3( Olesii Doronin *, Rostislav Maiboroda ** GEE ESTIMATORS I MIXTURE MODEL WITH VARYIG COCETRATIOS Abstract. We discuss a seiparaetric ixture odel where soe coponents are paraeterized with coon Euclidean paraeter and others are fully unnown. We introduce GEE (generalized estiating equations approach and adaptive GEE-based approach for paraeter estiation. Derived estiators are consistent and asyptotically noral, and they are optiized in ters of their dispersion atrices. Proposed techniques are tested on siulated saples. eywords: ixture odel, seiparaetric estiation, GEE. 1. ITRODUCTIO The cuulative distribution function (CDF of one observation in a ixture odel is expressed by a linear cobination of soe CDFs F 1,..., F M with 1 M M M probabilities p,..., p, p 1 (i.e. F x p F ( x. ote that 1 ( 1 F is called the CDF of the -th ixture coponent, and p the coponent concentration. In ixture odel with varying concentrations p depends on the observation index: p, j 1,. Thus, p j M F j ( x p j F ( x, j 1, j 1. We consider the case when soe paraetric odel is nown for the first coponents: F ( x F, 1,. Paraeter t is assued to be d Euclidean: t. The true value of t we designate as and assue that it is unnown. The CDFs of the last M ixture coponents are assued to * Ph.D. student, Departent of Probability Theory, Statistics and Actuarial Matheatics, Mechanics and Matheatics Faculty, Taras Shevcheno ational University of yiv. ** Ph.D., Departent of Probability Theory, Statistics and Actuarial Matheatics, Mechanics and Matheatics Faculty, Taras Shevcheno ational University of yiv. [15]

2 16 Olesii Doronin, Rostislav Maiboroda be fully unnown. We also assue that concentrations are nown. Our goal is to estiate. To do this, we derive consistent and asyptotically noral estiators, and optiize the in ters of their dispersion atrices. p j 2. OPARAMETRIC ESTIMATE FOR DISTRIBUTIO FUCTIO CDF of the -th coponent ay be estiated through the weighted epirical distribution function: ˆ 1 F ( x : a j I{ x}. j a j Weights are taen as the solution of the iniization proble of axial variance of unbiased estiates of F (x for all possible CDFs F (i.e. 1 M a p e where p : 1 M M ( p j, : p T p, j 1,, 1, M e : ( { i } i 1,...,M. See Maiboroda et al. (2008 for details. ote that weights a j can be negative. Thus, we can iprove Fˆ ( x by introducing iproved epirical distribution function (see Maiboroda et al. (2005: Fˆ j 1 ( x : in(1, ax F( y. y x 3. GEE ESTIMATE Consider soe set of easurable functions 1 ( d g ;,..., g ( ;. Theoretical oent g F ( dx ay be estiated by the weighted epirical oent as 1 gˆ ( : a j g ( j;. j1 Define the joint weighted epirical oent of g ˆ( : 1 gˆ (. ˆ g ( as

3 GEE estiators In Mixture Model Definition. GEE estiator ˆ for is the easurable function fro saple 1,..., such that gˆ ( ˆ 0. ext we assue that P[ t : gˆ( ˆ 0] 1 as. Exaple. Moent estiators can be represented as GEE estiators. Let h 1,...,h be the set of estiating functions. Denote theoretical oent of h (x as ( : h ( x F ( dx;, H,. g : h ( x H (, 1,. ˆ 1 ( ˆ 1 h : H where 1 Define estiating functions as GEE estiator 1 H is the inversed function to Analogous iproved oent estiate with ˆ can be represented as ( 1 H t : H ( t. can be introduced. Consistency for oent estiators is shown in theore 3.1 fro Doronin (2014a. hˆ : h ( x Fˆ ( dx 4. ASYMPTOTICS OF GEE ESTIMATOR Assue that CDFs F 1,..., F M are absolutely continuous with respect to siga-finite easure on the space of observations. Denote densities of each df df ( x coponent's distributions as f ( x :, 1,, f ( x :, d( x d( x 1, M. Introduce the atrix of estiating functions G g1( x; d G( x :. g x ( ; Expectation of G(x fro the -th coponent designate as : G( x F ( dx, 1,..., M Introduce the following notations.., l 1 l r s r, s: ( r, s : l a j j a j p j p li j, r, s 1, M., 1, 1, l1,

4 18 Olesii Doronin, Rostislav Maiboroda l : ( a j j a j p j, 1, M. 1, l :, l1, li 1 M f ( x, l1, R( x :. 1 T M T dd Z : G( x R( x G( x ( dx G G. 1 V : g t r, s r r, s dd t F ( dx 1. Theore 4.1. (Theore 3.4 fro Doronin (2014a Let ˆ be GEE estiator in introduced definitions, and U be soe open neighborhood of the true paraeter value. Assue the following.: (i ˆ converges in probability to as. ' T (ii Derivatives g g / t exist and are integrable (i.e. ' Et[ g ( ; ] for t U, where Et denotes expectation under condition that the true paraeter value is t, and are the foral rando values with distrubutions F. (iii Functions g ( E [ g ( ; ] are continuous on U. (iv E [suptu g ( ; ]. (v Liit atrix exist and is nonsingular. (vi Matrices r,s and exist. (vii Matrix V is nonsingular. (viii GEE is unbiased, i.e. E [ ( ; ] 0 1 t g t for t U. Then ( ˆ converges in distribution to Gaussian distribution with 1 T zero ean and covariance atrix V ZV. s 5. LOWER BOUD OF DISPERSIO MATRIX FOR GEE ESTIMATOR Assue that the atrix Z and nonsingular atrix V exist. Without loss of generality we can assue that two conditions for GEE estiator ˆ are fulfilled: (i1 g F ( dx 0, 1, (unbiasedness; (i2 atrix V is the unit atrix.

5 GEE estiators In Mixture Model Consider the iniization proble of dispersion atrix Z in Loewner ordering (i.e. A B if A B is non-negatively defined over all g d satisfying conditions (i1, (i2. Thus, we have to iniize c T Zc for all c. The solution of this proble is the set of estiating functions g (x;, which give us the lower bound of dispersion atrix (2014a. 6. ADAPTIVE ESTIMATE Z (see theore 4.1 fro Doronin Unfortunately, it is ipossible to use in practice the optial estiating functions g, which give the lower bound of dispersion atrix. The first reason is that they depend on unnown densities f (x, 1,. The second one is the difficulty to solve the GEE in the general case. Therefore, we consider the adaptive approach. dl Each function g can be approxiated as B u where B L is soe atrix of coefficients to be found, and u is the vector of soe predefined basis functions (e.g. B-splines. Under conditions (i1, (i2 equation ( 0 we can approxiate as g ˆ 1 gˆ ( B uˆ ( ( t. The solution of this approxiated equation is t B ˆ (. u Thus, one can start with soe consistent estiate ~ and define adaptive estiate as ˆ ~ : 1 B ~ ˆ (. u 1 Consistency and asyptotic norality of introduced adaptive estiate is shown in lea 3.3 fro Doronin (2014b. 7. UMERICAL RESULTS We chose a three-coponent ixture odel to siulate. All coponents are taen Gaussian, with paraeter values (, as ( 3.2, ( 3.2, (0. 2, for each coponent, respectively. The first two coponents are assued to be

6 20 Olesii Doronin, Rostislav Maiboroda T paraeterized with ( 1, 2, (different eans, coon standard deviation. Distribution of the third coponent is assued to be fully unnown. Concentrations were also generated as the pseudo-rando values, derived by 2 3 forula /( 1 p j s j s j s j s j where s j is taen fro unifor distribution on [0,1]. Series of saples with sizes 50,, 250, 500, 750, 0, 2000, 5000 were siulated, 2000 saples in each series. Vectors of basis functions u for adaptive estiate were chosen as the set of unifor cubic B-splines with nots at points i, where and are the ean and standard deviation of the -th coponent, respectively, i 5,...,5. Matrices B were chosen to iniize dispersion atrix. Results are shown in Figure 1. COCLUSIOS The ixture odel with varying concentrations is considered. Several estiators for this odel are introduced (oent, GEE, adaptive. The proposed estiators are consistent and asyptotically noral under soe conditions. Perforance of oent and adaptive estiators are copared on siulated saples. Dispersion of introduced estiators converges to its theoretical asyptotic value for saples with 0 and ore observations. REFERECES Doronin O. (2014a, Lower bound of dispersion atrix for seiparaetric estiation in ixture odel. "Theory of Probability and Matheatical Statistics", no. 90, p Doronin O. (2014b, Adaptive estiation in seiparaetric odel of ixture with varying concentrations. "Theory of Probability and Matheatical Statistics", no. 91, p Doronin O. (2012, Robust Estiates for Mixtures with Gaussian Coponent. "Bulletin of Taras Shevcheno ational University of yiv. Series: Physics & Matheatics" (in Urainian, vol. 1, p Maiboroda R., Sugaova O. (2008, Estiation and classification by observations fro ixtures. yiv University Publishers, yiv (in Urainian. Maiboroda R., ubaichu O. (2005, Iproved estiators for oents constructed fro observations of a ixture. "Theory of Probability and Matheatical Statistics", no. 70, p Maiboroda R., Sugaova O., Doronin A. (2013, Generalized estiating equations for ixtures with varying concentrations. "The Canadian Journal of Statistics", no. 41, vol. 2, p

7 GEE estiators In Mixture Model MSE ( ˆ MSE ( ˆ RobVar ( ˆ RobVar ( ˆ

8 22 Olesii Doronin, Rostislav Maiboroda MSE (ˆ RobVar (ˆ Here MSE is the ean squared error of the paraeter estiate ultiplied by nuber of observations. RobVar is the robust estiate of MSE through the interquartile range of paraeter estiate. Sybol indicates the oent estiates (lower line for iproved and upper line for uniproved, and adaptive estiates. White sybols indicate theoretical dispersion. Sybol indicates the lower bound. Figure 1. Dispersion of estiates Source: plots are generated by Wolfra Matheatica using our own script.

9 GEE estiators In Mixture Model Olesii Doronin, Rostislav Maiboroda ESTYMATORY GEE W MODELU MIESZAYM ZE ZMIEYMI WSPÓŁCZYIAMI OCETRACJI Streszczenie. W pracy oówiono seiparaetryczny odel ieszany, w tóry pewne współczynnii są paraetryzowane za poocą wspólnego paraetru eulidesowego, natoiast inne są zupełnie nieznane. Wprowadzono etodę estyacji paraetrów opartą na podejściu GEE (uogólnionych równań estyujących oraz adaptacyjny podejściu GEE. Proponowane estyatory zostały przeanalizowane w badaniu syulacyjny. Słowa luczowe: odel ieszany, estyacja seiparaetryczna, GEE.

10 24 Olesii Doronin, Rostislav Maiboroda

Machine Learning Basics: Estimators, Bias and Variance

Machine Learning Basics: Estimators, Bias and Variance Machine Learning Basics: Estiators, Bias and Variance Sargur N. srihari@cedar.buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Topics in Basics