Small area solutions for the analysis of pollution data TIES Daniela Cocchi, Enrico Fabrizi, Carlo Trivisano. Università di Bologna

Transcription

1 Small area solutions for the analysis of pollution data Daniela Cocchi, Enrico Fabrizi, Carlo Trivisano Università di Bologna TIES 00 Genova, June nd 00

2 Summary The small area problem in the environmental context Theoretical framework The linear hierarchical model Special case: ANOVA model Approximating criterion: least squares Prior conjectures Statistic chosen for conditioning The normal case and the departure from normality Comments on least squares approximations A simulated experiment Consequences of misspecifications in prior evaluations High applicability to environmental problems Extensions

3 The small area problem in the environmental context Simultaneous estimation of parameters related to different subpopulations (domains) of a more general population Desired property: additivity Example 1 Erosion from agricultural land in a watershed (Opsomer, Botts, Kim, 001) Population of areas (160 acres plot) Auxiliary variables available Mixed effect linear model 3

4 Example Emissions inventories (CORINAIR project in EU) national estimates of emission volumes (by pollutant) many domain estimates spatial economic sectors time evolution Building inventories Census of major pollution sources Indirect estimation of small and very diffuse sources Activity indicators Emission factors 4

5 Hierarchical classification of emission activities SNAP system (Selected Nomenclature for Air Pollution) E A F pat,, at, pa, A at, : volume of activity a in period t subject to sampling random variation; : emission factor for pollutant p in activity a; seldom F pa, estimated on a sample basis. Approach: bottom up. E N E pt, a, pt, a1 Suggestion: ANOVA model is a realistic proposal for A at, since it does not use auxiliary information which is very heterogeneous and suffers from problems of spatio-temporal definition. 5

6 Theoretical framework Typical of the finite population context It can be solved within The design based approach parameters: unknown characteristics of the subpopulations The model based approach where hyperparameters are considered 6

7 We discuss: Model based approach Hierarchical modeling stressing Bayesian solutions Two main motivations A natural way of building and solving models Managing different aggregation level of the information 7

8 The linear hierarchical model It is a structural model for the population Z, : N1 Z : N p : p 1 Z Z : pq : q1 : q where Z diag i Nk is the small area indicator matrix and the incidental parameter is the vector of small area means object of inference. 8

9 Assumptions on the model (only on the first two moments of the distributions): C 1, Z, Z 0,, E Z,, E 0 V Z V V 1 0,, E Z,, E 0 V Z V B A set of assumptions of conditional independence is able to simplify the model. Many small area models are special cases of this general model. 9

10 Role of covariates a) At the individual level The slightly different Battese et. al. (1988) model X Z V I B i i Z I N 0 p p 1 p with a value on the whole population and small area random effects 10

11 b) At the area level N p, : p1 Z I : p 1 p Fay and Herriott (1979) model adds distributional assumptions e ~ N 0, D, k 1,..., p k k k ~ N 0, A, k 1,..., p Frequentist solutions Empirical Best Linear Unbiased Prediction. Estimating and, and introducing the estimates in the Best Linear Unbiased Predictor of. 11

12 Some basic simplifications Special case: ANOVA model V I N Z Z i 1 N A comprehensive statistical model A sampling model can be added to the structural model,..., ': S e e n N s1 sn where e si is the si column of I N. 1

13 The general least squares principle Approximating criterion: least squares Approximation to a normal distribution with the same first two moments of the exact one. Finite populations: this approximation is conditional on Z, S. Normal approximation on TYPE OF INFERENCE: Bayesian, tzs, and not on the whole,, tzs,. Why approximated solutions when MCMC solutions which approximated posterior distributions are easily available? Because they are analytically approximated and need the elicitation of a relatively small number of prior guesses. 13

14 Be X the data and the parameter, for any l t { a' t}. we define If: ZS, LS g and t t X Z, S lzs, t { az, s t } E t arg min E l t Z, S l =arg min E E t, Z, S l t Z, S l ZS, 1 = E Z, S C, t Z, S { V t Z, S } { t E t Z, S } ZS LS V t min E E t, Z, S E t, Z, S Z, S ZS,, LS l Z, S 1 S = V Z, S C, t Z, S { V t Z, S } C t, ' Z, 14, (Cocchi and Mouchart, 1996)

15 Prior conjectures Only on a certain number of moments Conjecturing on moments is more immediate than conjecturing on distributional assumptions Their type and number depend on the statistic on which conditioning 15

16 Statistic chosen for conditioning Its choice can enrich the solution and determines its complexity a) Solution conditional on t T ZSy: vector of small area totals 0 Priors to be elicitated E Z E b 0 V 0 Z1 V 0 M 0 1 v E Z E v E Z E 16

17 b) Solution conditional on t T T0, T1, T i.e. the vector of small area totals and the sum of squares T : between small areas and T : within small areas Since T contains a polynomial of order d, prior information on moments of order d must be used. For computing the solution the following moments are needed j j E k Z1, 0,, j=3,4 j j E k Z,,, j=3,4 17

18 New priors to be elicitated besides the group of priors above: It is not a normal approximation!,, V V ZS V V ZS,, a E Z S a E Z S ,, f E Z S f E Z S c C Z 0,1,, 0 It does not mean approximating normal distributions with the same first two moments: the model may contain moments up to the 4 th. Gain: this solution can consider asymmetry and kurtosis. 18 S

19 Solution a) For t=t 0 where 0 a 0 1 p a ZS, ELS T gb g y I p w = wv gb0 1 g yv vi y: p1 ZS, LS V T v M gaa 0 a 0 v M v g p p = w w 0 ww : y a k k k a v v n v = v v n v : p 1 19

20 1 1 w wk v ak v nkv : p1 a diag a : p p 1 : k diag w v n v p p w k k y ny v n v v w y wt0 wn k k k k n nk v w i n p k v nkv ZSSZ diag n p p n k : wn wn 1 g M M M y n iy n it n ny n p 0 n n i p : p1 y 1 n T0 : p1 0

21 b) For t=t heavier calculations, no elegant formula because of the difficulty of writing analytically V v v VT Z S v v v p p 0 1 v v v , : ( ) ( ) E T I C i b C y C ZS, LS p * p 0 * ** 1 1 nn T v p v nn T v n p 00 where 01 C* CV 0 c1 v n : p p C** C0v cv 1 C0v cv 1 : p after defining: C T C c p p : 1

22 The normal case and the departure from normality In case of symmetry: VT Z, S 0 1 T 1 1 V T Z, S 0 when normality holds: CT, TV p1n p When c C T Z S 0,1,, 0 Then and 0 V Z, S T C** C T ZS, VTZS, C T 0 ZS, VT 0 ZS, As a consequence ZS LS 0 E T E T ZS,, LS

23 Comments on least squares approximations They borrow strength from the other subgroups and from the prior conjectures The idea recalls two important points : posterior linearity (for estimating the general mean starting from the least squares approximations of the small area parameters) empirical Bayes solutions. The solution is robust since it depends only on moments. The solution seems not to have practical drawbacks linked to the number of small areas or to the fact of not having evidence for some subgroups. 3

24 A simulated experiment Aim: Checking the improvement of conditioning on T instead of T0 in the approximated posterior expectations. Data generation process: random generation of and from normal - normal lognormal - lognormal. Lognormal parameters are set in order to have the same first two moments of the normal distribution. Exact Bayesian least squares solutions i.e. right prior conjectures. 4

25 Design of the experiment 1. generation of 1000 populations of N= 1000 elements with mean =1. 0 under 4 different hypotheses on structural variability: 1, , 0.5, 1 1, 1 and different hypotheses on the domains: p=10; p=40;. generation of 100 samples from each population with n k =10, k=1,, p when p=10 n k =5, k=1,, p when p=40; 3. evaluation of the performances of the posterior expectations by means of mean square errors averaged over the domains. 5

26 Lognormal Lognormal p=10 n k =0 p=40 n k =5 =1 = =0.5 = = = = = Normal Normal ( c ) 0,1 1 p=10 n k =0 p=40 n k =5 =1 = =0.5 = = = = =

27 Consequences of misspecifications in prior evaluations a) Not important: errors in conjectures on variances 0 V V V b) As expected: errors in conjectures on E 0 c) The most dangerous: errors in conjectures on expectations of variances E E d) Exchanging the right lognormal and normal conjectures For normal-normal data generation: the most dangerous are the errors on the 3rd moment. For lognormal - lognormal data generation: the most dangerous are the errors on the 4 th moments. 7

28 High possibility of application to environmental problems Suitable solution for estimating emission inventories since a) the compositeness and heterogeneity of local emissions is so high that a random effect model is in many cases the only practicable solution b) all methods for building emissions inventories involve expert evaluations, which can be managed as priors. introduction of covariates relationships with the design Extensions 8