Estimation of Location Parameter from Two Biased Samples

Transcription

1 Applied Mathematics, 3, 4, Published Online September 3 ( Estimation o Location Parameter rom Two Biased Samples Leonid I. Piterbarg Department o Mathematics, University o Southern Caliornia, Los Angeles, USA piter@usc.edu Received June 5, 3; revised July 5, 3; accepted July, 3 Copyright 3 Leonid I. Piterbarg. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. ABSTRACT We consider a problem o estimating an unknown location parameter rom two biased samples. The biases and scale parameters o the samples are not known as well. A class o non-linear estimators is suggested and studied based on the uzzy set ideas. The new estimators are compared to the traditional statistical estimators by analyzing the asymptotical bias and carrying out Monte Carlo simulations. Keywords: Biasness; Fuzzy Estimator; Location Parameter. Introduction The problem is to estimate an unknown scalar parameter rom two dierent independent samples o size n and n x i bi, i,,, n, x b, i,,, n i i where b and are the bias and scale parameter o the -th sample respectively, =,, and i, i are zero mean independent random noises. A novelty in our set up is that the biases b, =, are assumed to be unknown which makes unidentiiable rom the classical statistics viewpoint, e.g. []. Thus, traditional ormulations o best estimation are not applicable in this situation which nevertheless oten arises in applications. An important example is an assimilation problem in physical oceanography and meteorology where inormation on a certain parameter comes rom both observations and a circulation model [,3]. Typically such a model is biased or a variety reasons: uncertainties in a orcing and dissipation, boundary conditions, model parameters, etc. The bias in observations is mostly due to inaccurate measurements and time/space averaging intrinsically present in any measuring procedure. That type o bias sometimes can be excluded using a learning samples [4,5], however it is oten diicult to ustiy the key assumption that learning and control samples are taken rom the same ensemble. One can simply ignore the biases and apply traditional () least square or maximum likelihood methods which would result in a biased estimate o. As an alternative, we suggest to use uzzy set (possibility theory) ideas [6,7] to construct non-linear estimators or diminishing the bias comparing to the aorementioned approach. With biased observations the ocus is naturally shited rom the variance o an estimator, which can be arbitrarily reduced by increasing a sample size, to its asymptotical bias. More exactly, in the traditional representation o the squared standard error (SE) E ˆ Bˆ Varˆ our primary point o interest is the irst term. Thus, we start with analyzing the asymptotical bias o the suggested estimators and then compare it to that o estimators traditionally used in statistics such as weighted mean or weighted median. Then, SE is addressed or small samples via Monte Carlo simulations when the second term is not negligible. Worthy noting that in the simplest situation with unbiased observations b and known s the unbiased estimator with the smallest SE (least square estimate) is given by the weighted mean, e.g. [8] x x ˆLS where x is the sample mean o the -th sample. Moreover, with normal noises it is the maximum likelihood estimator []. () Copyright 3 SciRes.

2 7 L. I. PITERBARG In the general ormulation () with biased observations a choice o an appropriate measure o the estimation skill is a challenge because the bias B ˆ depends on unknown (nuisance) parameters b, b which never can be identiied rom the available observations. We construct such a measure as ollows. For large samples one can eiciently estimate,, and the bias dierence bbb rom the observations () by subtracting the second sample rom the irst one. Introduce b (3) b Assume that we deal with a class o estimators ˆ or which the asymptotical bias n, n exists and thereby is a unction o all the involved parameters B ; b,, (sub ˆ is dropped as a matter o breivity). Unlike b,, one cannot estimate at all under the given observations. In such a situation one o the ways to order estimators according to their biases is to accept that ˆ is better than ˆ (under the identiiable parameters being ixed) i B π B or a certain class o densities π where k k π (4) B π B ; b,, π d k and B is the asymptotical bias o ˆ k. In simple words under arbitrary distribution o the absolute value o the irst bias is not greater than that o the second one. Our irst inding and pretty surprising one is that the best estimator in sense (4) among a wide class including traditional statistical and uzzy estimators is the simple arithmetic mean ˆ (5) where is any consistent estimator o the center o x i (i.e. b as n ) say the sample mean or median. Does it mean that (5) is the best way to deal with biased observations? O course it is not because ater all a real matter o concern is SE which can be essentially less or the weighted mean () than or (5) under small enough biases b. The next important result o this study is that the suggested uzzy estimators are better than weighted estimators o type () in sense (4). Finally, to decide which estimator should be preered in dealing with small samples we carry out Monte Carlo simulations and use SE averaged over the nuisance parameters as a measure o the estimation skill SE ˆ Eˆ, (6) where the angle brackets mean averaging over parameters deined in (3) and the ratio (7) characterizing the dierence in the noise level o two sources. The reason or including the averaging over identiiable parameter is that we are interested in small samples n, n or which it is not possible to eiciently estimate even identiiable parameters. We then investigate dependence o (6) on the bias level b and noise scale or dierent estimators and suggest recommendations or sensible choice among them. In general, uzzy estimators seem to be preerable or high values o b and in most o scenarios determined by dierent noise distribution (normal, Cauchy), non-probabilistic noise (logistic chaos), and by dierent estimates o the center (mean or median).. Estimators and Their Asymptotical Bias First recall that a uzzy set is a pair A, P where A is a set and P: A, is called the membership unction, the value P x characterizes the degree o membership x in A. The set x A P x > is called the support o A, P. For our purposes it is enough to consider real uzzy sets, A R. Regarding to the ormulation () we consider A as a range o the observed random variable X. Let us introduce a class o uzzy estimators similar to estimators based on the triangular membership unction (possibility distribution) discussed in [9]. Let F x be a cumulative distribution unction, symmetric and increasing, i.e. FxFx, F x. Introduce the membership unctions generated by each o two samples as ollows P x x F i x s x F i x s and, s are consistent estimators o the center and spread o the -th sample respectively,,. In other words we assume b, s with probability as n. Copyright 3 SciRes.

3 L. I. PITERBARG 7 We address the ollowing estimator o henceorth called the uzzy estimator ˆ O d x P x P x dx O P x P x x where O is the set o the Pareto optimal solutions and a b is the minimum o a and b. Pareto set is deined by x O i and only i P x P x ' ' < In other words, a Pareto optimal solution (Pareto optimal) is one in which any improvement o one obective unction in the two obective optimization problem P x max, P x max (8) mean p and the weighted estimator with p. To analyze properties o ˆ, let us ix the bias dierence bb b, the observation error level and introduce dimensionless parameters b b,, a b. Then the bias o (9) as n n n goes to B, b p p p p p p () The asymptotical bias or (8) is given in the next statement. Proposition Let n n n, then or any ixed a the bias B E goes to ˆ ˆ B b () can be achieved only at the expense o another, e.g. []. where A key point in the base o (8) is that we use a standard ap qq uzzy logic aggregation procedure, but integration is car- ried out only over the set o x maximizing both memberap p ships o x. Such an approach goes ar beyond linear esti- with mators traditionally used in statistics. However, the in- a troduced class o estimators allows their analytical study px x Fud u, ax at least regarding to the asymptotical bias. a Together with (8) we consider the class o weighted qx x Fudu ax estimators Proo ˆ p wp wp (9) With no loss o generality assume <, then the Pareto optimal set is simply an interval O, where is a sample mean or other consistent estima- due to the condition F x >. Since the symmetry the tor o the center o -th sample. only intersection point o P x and P x in this The weights in (9) are given by interval is given by p p p wp s s s s s x s s and our primary ocus is on the minimum least square estimator corresponding to p, simple arithmetic and ater an appropriate variable change (8) becomes ˆ ss s s s s ss s s s s s s u F u du s su F u du s F u du s F u du Let us change u to u in the second integrals on the top and at the bottom. Then using the symmetry o ˆ Proceeding to the limit obtain ss s d s s s ss s d d s s s B s s u F u u s su F u du s F u u s F u u n n, accounting or the consistency o estimates, s, b b d b b b b Fudu Fudu b b b u F u u b u F u du F u get and symmetry o F x Copyright 3 SciRes.

4 7 L. I. PITERBARG Notice that in the considered case b. Using dimensionless variables b b,, a b ater some algebra arrive at (). Notice that Proposition I < x <, then Proo Since px x x () x (3) inequality (3) is equivalent to ap xq xap xq x The expression on the right hand side can be written as a S x x a xu F u du ax ax v a vf dv a x then the inequality takes orm equivalent to ax ax > S x S x va va vf v vf v d a x a x d and is (4) Let us break down the integral on LHS into integrals rom a to and rom to a x respectively, change v to v in the second integral, and move it to RHS. Next, make the same variable change on the RHS o (4) and move it to LHS. The goal is to have all the integrals over positive intervals. The result is ax va a va vf d d v vf v x ax x a v a > vf dv x Obviously v a x v a x va x v a x and whenever x and hence the last inequality is true since F u is increasing. The proo is over. Further we restrict ourselves by distributions which decay ast enough as u F u C (5) u From (5) it ollows that x qx px p q we get, since lim and (6) Next we consider the class o all estimators or which the asymptotical bias is expressible in orm B b (7) where x satisies x x, x. i x Notice that both the weighted mean (9) and the uzzy estimator (8) are in this class. For the ormer it is easy to check and or the latter the statement ollows rom (, 3, 6). Now we intend to order estimators rom this class according to their asymptotical bias. Roughly speaking or ixed a and one estimator is better than another i the asymptotical bias o the ormer averaged over is less than that o the latter. Rigorously, or two estimator k k with biases B B given by (7) k, the irst one is better than the second one i (3) holds true or any positive unction π x satisying two conditions x π x d x, π.5 x π.5 x (8) Value.5 is singled out since it corresponds to the case o balanced biases bb. Propos ition 3 I or some inequality is true, then the irst estimator is better than the second one B π B π (9) Proo By direct computations with using symmetry o π obtain or z.5 d z x z πxdx πxd x< dz z.5. A surprising result is readily derived rom Proposition 3. Corollary The trivial estimator Hence implies (9) because ˆ () is the best estimator in the class (7) This statement ollows rom the act that or () x x. The problem with () is that or small biases it is es- ˆ sentially less eicient than and ˆ. Let us compare the eiciency o () to that o ˆ in the case o equal small biases b b. Introduce ratio o MSEs Proposition 4 R E ˆ E ˆ Copyright 3 SciRes.

5 L. I. PITERBARG 73 For any large whenever and C > R C C C or C C 4C C 4 Proo Let s 4 and then () becomes or t s t C () () (3) s tc (4) C Inequality stc is equivalent to (3) and then (4) turns in (). For example assume C =, then () holds true i β >.7887 or β <.3. Then or β =. we have R > i ε <.594. Thus it is worth to compare the suggested uzzy estimators with ˆ and ˆ. For that purpose we need another representation or appearing in () where x TxTx x (5) a p x p x a T x x a u F u du ax Since the denominator in (5) is positive or all x the ollowing statement gives conditions or or the uzzy estimator to dominate ˆ. Proposition 5 i and only i x> x, x, (6) >,, T x T x x (7) For example i the possibility distribution is given by P u u (8) which corresponds to Fu u.5, u <, then (7) is ulilled. Indeed, or (8), t x b T x T x b a a bx b x x ln x ln, x x Then or any ixed and x, b x t b xbx xb x one gets t x, which implies (7). Condition (7) is not always ulilled or th e triangle membership unction deined by the cumulative probability distribution unction u i qu Fu.5 q i u q where q is a ixed quantile. Indeed, direct computations yield []. Proposition 6 x x x xx 3 i x 3 x 3 x x xx i x min, 3 3 x 3 x x i x max, x i x where qa. It can be derived rom that statement that (6) holds true i and only i while or 7 3 the opposite relation holds true: x x or all x. In the intermediate case 73 the dierence x x changes the sign once in,. I then two samples are incompatible. These conclusions are illustrated in Figure by plotting the dierence d x x x or the triangular membership unction (let and middle panel) and or membership (8) (right panel). I then or sure 73 and hence d x or x (let panel). I 3 then two options are possible either d x or d x changes the sign once (central pane l). Finally Copyright 3 SciRes.

6 74 L. I. PITERBARG Figure. The dierence λ(x) x vs x. ) Triangle membership, < γ < ; ) Triangle membership, < γ < 3; 3) Membership given by (8),.5 a 5. or membership () d x or all a. x p x (9) However, it can be s hown [9], that x x when x, thereby the uzzy triangle estimator is better or all x. Then than ˆ in sense (9). G, p x a dx Next, deine another quantitative characteristic or comparing asymptotical biases which is easy to evaluate, Thus under (9) the region G always occupies more unlike B π. First deine the space o t he parame ters o than a hal interest. In particular, or x x the integral on the right hand side is equal to.5 and or, 3, x x x x it is ln The latter ollows rom The range or is ounded in [9]. Then introduce the dierence o biases or the mentioned estimators deined by unctions xdxln 4 x and x respectively In particular, the area o a region in where the B, B B bias o the uzzy estimator (8) is less than th at o ˆ is and deine greater than.5. While the area o a region in where the uzzy triangle estimator is better than ˆ is G, B, greater than Obviously the area o equals a nd hence the irst estimator can be viewed as better than the second one whenever the area o G exceeds. This excess is quantiied in the next statement. Proposition 7 x x, x then I G x xdx Indeed, the solution o x < x equivalent to > x x is rom which the statement readily ollows. The ollowing proposition gives a rough estimate o improving a uzzy estimator comparing to any weighted mean (9). Set, B, G B, Corollary Let or some ixed a p 3. Simulations The goal o the perormed simulations is to compare the eiciency o dierent estimators under dierent noise distributions. Now the asymptotical bias is not o primary concern, but rather the relative standard error.5 E SE ˆ ˆ d d.5 is addressed under small samples. We examine two uzzy estimators (8), irst, determined by (8) (Fuzzy ) and, second, determined by Pu.5e u (Fuzzy ) and three weighted estimators ˆ, ˆ and ˆ given by (9). Sample sizes n n and number o Monte Carlo trials M = or each mesh, are kept the same or all experiments. First, normal noise is tested with, i.e. the x Copyright 3 SciRes.

7 L. I. PITERBARG 75 sample center is estimated by the sample mean (Figure ). At zero bias (let panel) ˆ is best as expected, while Fuzzy is only slightly worse. For modest biases (central panel) ˆ is best or small level o noise while or modest and high level o noise again Fuzzy is better. Finally or high biases (right panel) only two estimates are co mpeting, ˆ and Fuzzy. The ormer is better or small noise while the latter dominates or high noise. ˆ and ˆ are much worse in this case. I the center is estimated by the sample median the conclusions remain basically the same, however the numbers are slightly dierent (experiments are not shown). In the case o Cauchy distribution o the noises (Figure 3) only the median med x was used or estimati ng the centers. One can see that even or zero bias o noises Fuzzy has the smallest SE along with ˆ. For modest biases all the estimators except ˆ are approximately o the same accuracy or the whole range o. The primitive ˆ appe ars to be essentially worse than any other estim ate or high values o. Finally or high biases again Fuzzy is best or intermediate values o while ˆ and Fuzzy are slightly better or small and large s respectively. Comparing to the normal noise the accuracy o all the estimators are somewhat lower. In Figure 4 results are presented or logistic noise generated by k rk k with r = 5. or the irst sample and r = 3 or the second one. The results are closer to the normal noise case rather than to the Cauchy case, however errors in general are smaller than in the normal case. In summary or all three experiments, the classical weighted estimator ˆ is worsening ast as the level o bias is increasing while the uzzy estimators demonstrate a steady skill in all the scenarios and or whole range o and b. At this point it is hard to give a preerence to either Fuzzy or Fuzzy. Similar conclusions can be drawn rom Figures 5 and 6 where other experiments are presented in which the dependence o SE on b were studied or ixed. We Figure. Dependence o standard error on the noise level σ or dierent values o bias scale b normal noise. The mean is taken as an estimate o center. ) b = ; ) b =.5; 3) b =. Figure 3. Dependence o standard error on the noise level σ or dierent values o bias scale b Cauchy noise. The median is taken as an estimate o center. ) b = ; ) b =.5; 3) b =. Copyright 3 SciRes.

8 76 L. I. PITERBARG Figure 4. Dependence o standard error on the noise level σ or dierent values o bias scale b logistic noise. The median is taken as an estimate o center. ) b = ; ) b =.5; 3) b =. Figure 5. Dependence o standard error on the bias scale b or dierent values o noise level σ normal noise. The median is taken as an estimate o center. ) σ =.; ) σ = ; 3) σ = 3. Figure 6. Dependence o standard error on the bias scale b or dierent values o noise level σ Cauchy noise. The median is taken as an estimate o center. ) σ =.; ) σ = ; 3) σ = 3. Copyright 3 SciRes.

9 L. I. PITERBARG 77 want to stress two o them, irst, or high Fuzzy is uniormly better than any other estimate and, second, ˆ is not appropriate or such values o regardless the noise distribution. 4. Conclusions and Discussion [] G. Casella and R. Berger, Statistical Inerence, nd Edition, Brooks/Cole, Paciic Grove, 99. [] M. Ghil, Meteorological Data Assimilation or Oceanographers. Part ; Description and Theoretical Framework, Dynamics o Atmospheres and Oceans, Vol. 3, No., 989, pp doi:.6/377-65(89)94-7 [3] P. Malanotte-Rizzoli, Modern Approaches to Data Assimilation in Ocean Modeling, Elsevier, Amsterdam, 996. [4] T. Krishnamurti, C. Kishtawal, Z. Zhang, T. Larow, D. Bachiochi and E. Williord, Multimodel Ensemble Forecasts or Weather and Seasonal Climate, Journal o Cli- mate, Vol. 3, No. 8,, pp doi:.75/5-44()3<496:meffwa>.. CO; [5] F. J. Lenartz, M. Beckers, J. Chiggiato, B. Mourre, C. Troupin, L. Vandenbulcke and M. Rixen, Super Ensemble Techniques Applied to Wave Forecast: Perormance and Limitations, Ocean Science, Vol. 6, No. 3,, pp doi:.594/os [6] D. Dubois and H. Prade, Possibilty Theory, Plenum Press, New York, London, 988. doi:.7/ [7] G. Shaer, A Mathematical Theory o Evidence, Princeton University Press, Hoboken, 976. [8] T. P. Hill and J. Miller, How to Combine Independent Data Sets or the Same Quantity, Chaos, Vol., No. 3,, p. 3. doi:.63/ [9] L. I. Piterbarg, Parameter Estimation rom Small Biased Samples: Statistics vs Fuzzy Logic, Fuzzy Sets and Systems, Vol. 7, No.,, pp. -. doi:.6/.ss... [] K. Deb, Multi-Obective Optimization Using Evolutionary Algorithms, Willey, Princeton,. [] R. J. Huber, Robust Estimation o a Location Parameter, The Annals o Mathematical Statistics, Vol. 35, No., 964, pp doi:.4/aoms/ A maority o studies in estimating a location parameter address, irst, linear unctionals o either the original sample or its ranking, e.g. [], and, second, unbiased observations. Here a class o essentially nonlinear estimators is suggested based on the uzzy set theory ideas to handle biased observations coming rom two dierent sources. Because any analytical investigation o the standard error or highly non-linear unctions o sample is hard, we concentrated on, irst, analytical studying the asymptotical bias o the suggested estimators and, second, Monte Carlo simulations or small samples with the tra- ditional standard error as an eiciency measure. Conclusions rom both studies are similar. Regarding the symptotical bias, the considered uzzy estimators are uniormly better than the classical least square estimator. As or small samples, the new estimators are o higher accuracy than the traditional least square estimator and its modiications or essential bias and high level o noise. Moreover, even or small bias, the uzzy estimators are only slightly worse than the optimal one. Unexpectedly, the primitive arithmetic mean succesully competes with uzzy estimators or essential biases and modest noise level, but it is o little help or high level o noise or negligable biases. Notice that the developed uzzy estimators are highly non-linear unctionals o observations and apparently are more appropriate or estimating parameters in non-linear complex systems. We tested only two estimators rom a wide class introduced here. It would be interesting to ind an optimal one among them. Another important problem is to address a similar problem or more than two inormation sources. 5. Acknowledgements The support o the Oice o Naval Research under grant N and NSF under grant CMG-5453 is greatly appreciated. REFERENCES Copyright 3 SciRes.