Kernel Smoothing and Tolerance Intervals for Hierarchical Data

Transcription

1 Clemson University TigerPrints All Dissertations Dissertations Kernel Smooting and Tolerance Intervals for Hierarcical Data Cristoper Wilson Clemson University, Follow tis and additional works at: ttp://tigerprints.clemson.edu/all_dissertations Recommended Citation Wilson, Cristoper, "Kernel Smooting and Tolerance Intervals for Hierarcical Data" (2016). All Dissertations ttp://tigerprints.clemson.edu/all_dissertations/1816 Tis Dissertation is brougt to you for free and open access by te Dissertations at TigerPrints. It as been accepted for inclusion in All Dissertations by an autorized administrator of TigerPrints. For more information, please contact

2 Kernel Smooting and Tolerance Intervals for Hierarcical Data A Dissertation Presented to te Graduate Scool of Clemson University In Partial Fulfillment of te Requirements for te Degree Doctor of Pilosopy Matematical Sciences by Cristoper Wilson December 2016 Accepted by: Dr. Patrick Gerard, Committee Cair Dr. William Bridges Dr. Colin Gallager Dr. Julia Sarp

3 Abstract Multistage sampling is a common sampling tecnique in many studies. A callenge presented by multistage sampling scemes is tat an additional random term sould be introduced to te linear model. Observations are identically distributed but not independent, tus many traditional kernel smooting tecniques, wic assume tat te data is independent and identically distributed, may not produce reasonable estimates for te marginal density. Breunig (2001) proposed a metod to account for te intra-class correlation leading to a complex bandwidt involving ig order derivatives for bivariate kernel density estimate. We consider an alternative approac were te data are grouped into multiple random samples, by taking one observation eac class, ten constructing a kernel density estimate for eac sample. A weigted average of tese kernel density estimates yields a simple expression for te optimal bandwidt tat accounts for te intra-class correlation. For unbalanced data, resampling metods are implemented to ensure tat eac class is included in every random sample. Bot simulations and analytical results are provided. One-sided tolerance intervals are confidence intervals for percentiles. Many autors ave provided metods to estimate one-sided tolerance limits for bot random samples and ierarcical data. Many of tese metods ave assumed tat te population is normally distributed. Since multistage sampling is a popular sampling sceme, we would like to employ metods tat avoid suc assumptions on te population. We explore non-parametric metods tat utilize bootstrapping and/or kernel density estimation to produce data driven percentile estimates. One way to account for ierarcical data is to decompose observations in a way tat is consistent wit decomposition of sum of squares for analysis of a one-way random effects model. We provide simulation study wit two percentiles of interest. ii

4 Dedication Tis work dedicated to my parents, Micael Wilson and Susan Reynolds, wo ave always loved me unconditionally and wose good examples ave taugt me to work ard for te tings tat I aspire to acieve. Tank you to my girlfriend Sivani Sa, for all er love and support iii

5 Acknowledgment I offer my most eartfelt praise to my advisor Dr. Patrick Gerard wo as been an amazing advisor. Witout is knowledgeable advice, insigtful criticisms, and patient encouragement, completing tis dissertation would not be possible. I am grateful tat Dr. William Bridges, Dr. Colin Gallager, and Dr. Julia Sarp provided me wit constant entusiasm and an interest in improving my researc. I would like to tank Dr. Pete Kiessler and Dr. Robert Lund for not only being great friends, but for encouraging me in every step of my journey at Clemson. iv

6 Table of Contents Title Page i Abstract ii Abstract ii Dedication iii Acknowledgments iv List of Tables vi List of Figures vii 1 Hierarcical Linear Models Kernel Smooting for i.i.d. Data Kernel Smooting for Hierarcical Data Percentile Estimation Application Conclusion Future Work Appendices A Marginal Distribution wit Non-Normal Errors or Cluster Effects B Approximate Covariance of Density Estimates in (3.3) C Proofs Bibliograpy v

7 List of Tables 1.1 ANOVA table for a one way random effects model for general ierarcical data sets ANOVA table for a one way random effects model for ierarcical data sets wit balanced data. b = n i for all i = 1,..., a Comparison of coverage probability and alf-widt (HW) of different confidence intervals. Coverage probability refers to te percentage of simulations tat µ = 0 is included in confidence intervals (1.6) and (1.7) Solutions to (3.10) for various values of σ 2 τ, for datasets a = 40 and b = 10, were τ i N(0, σ 2 τ ) and ɛ ij N(0, 1 σ 2 τ ). Additonally, we consider 500 resamples Solutions to (3.18) for various values of σ 2 τ, for datasets a = 40 and b = 10, and τ i N(0, σ 2 τ ) and ɛ ij N(0, 1 σ 2 τ ) List of all possible combinations for distributions of τ i and ɛ ij, were x denotes te combinations tat were considered Simulation results for balanced data wit 40 clusters and 1000 observations for eac cluster, and τ i follow a normal distribution and ɛ ij are normally distributed Simulation results for balanced data wit 40 clusters and 10 observations for eac cluster, and τ i follow a normal distribution and ɛ ij ave a log normal distribution Simulation results for balanced data wit 40 clusters and 10 observations for eac cluster, and τ i follow a normal distribution and ɛ ij ave a double exponential distribution Simulation results for balanced data wit 40 clusters and 10 observations for eac cluster, and τ i follow a log normal distribution and ɛ ij are normally distributed Simulation results for balanced data wit 40 clusters and 10 observations for eac cluster, and τ i follow a double exponential distribution and ɛ ij are normally distributed Simulation results for unbalanced data wit 62 clusters and τ i follow a normal distribution and ɛ ij are normally distributed Simulation results for unbalanced data wit 62 clusters and τ i follow a normal distribution and ɛ ij are log normally distributed Simulation results for unbalanced data wit 62 clusters and τ i follow a normal distribution and ɛ ij ave a double exponential distribution Simulation results for unbalanced data wit 62 clusters and τ i follow a log normal distribution and ɛ ij follow a normal distribution Simulation results for unbalanced data wit 62 clusters and τ i follow a double exponential distribution and ɛ ij follow a normal distribution Simulation results for unbalanced data wit 27 clusters and bot τ i and ɛ ij follow normal distributions Simulation results for unbalanced data wit 27 clusters and τ i follow a normal distribution and ɛ ij are log normally distributed Simulation results for unbalanced data wit 27 clusters and τ i follow a normal distribution and ɛ ij are double exponential distributed vi

8 3.17 Simulation results for unbalanced data wit 27 clusters and τ i follow a log normal distribution and ɛ ij follow a normal distribution Simulation results for unbalanced data wit 27 clusters and τ i follow a double expoential distribution and ɛ ij follow a normal distribution Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij also following a normal distribution. Eac dataset ad 40 clusters and 10 observations from eac cluster Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij following a log normal distribution. Eac dataset ad 40 clusters and 10 observations from eac cluster Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij following a double exponential distribution. Eac dataset ad 40 clusters and 10 observations from eac cluster Results from 1000 simulations were te data is generated wit τ i following log normal distribution and ɛ ij following a normal distribution. Eac dataset ad 40 clusters and 10 observations from eac cluster Results from 1000 simulations were te data is generated wit τ i following double exponential distribution and ɛ ij following a normal distribution. Eac dataset ad 40 clusters and 10 observations from eac cluster Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij also following a normal distribution. Eac dataset ad 62 clusters and 672 observations Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij also following a log normal distribution. Eac dataset ad 62 clusters and 672 observations Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij also following a double exponential distribution. Eac dataset ad 62 clusters and 672 observations Results from 1000 simulations were te data is generated wit τ i following log normal distribution and ɛ ij also following a normal distribution. Eac dataset ad 62 clusters and 672 observations Results from 1000 simulations were te data is generated wit τ i following double exponential distribution and ɛ ij also following a log normal distribution. Eac dataset ad 62 clusters and 672 observations Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij also following a normal distribution. Eac dataset ad 27 clusters and 285 observations Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij following a log normal distribution. Eac dataset ad 27 clusters and 285 observations Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij following a double exponential distribution. Eac dataset ad 27 clusters and 285 observations Results from 1000 simulations were te data is generated wit τ i following log normal distribution and ɛ ij following a normal distribution. Eac dataset ad 27 clusters and 285 observations Results from 1000 simulations were te data is generated wit τ i following double exponential distribution and ɛ ij following a normal distribution. Eac dataset ad 27 clusters and 285 observations vii

9 4.16 Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij also following a normal distribution. Eac dataset ad 40 clusters and 10 observations from eac cluster Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij following a log normal distribution. Eac dataset ad 40 clusters and 10 observations from eac cluster Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij following a double exponential distribution. Eac dataset ad 40 clusters and 10 observations from eac cluster Results from 1000 simulations were te data is generated wit τ i following log normal distribution and ɛ ij following a normal distribution. Eac dataset ad 40 clusters and 10 observations from eac cluster Results from 1000 simulations were te data is generated wit τ i following double exponential distribution and ɛ ij following a normal distribution. Eac dataset ad 40 clusters and 10 observations from eac cluster Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij also following a normal distribution. Eac dataset ad 62 clusters and 672 observations Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij following a log normal distribution. Eac dataset ad 62 clusters and 672 observations Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij following a double exponential distribution. Eac dataset ad 62 clusters and 672 observations Results from 1000 simulations were te data is generated wit τ i following log normal distribution and ɛ ij following a normal distribution. Eac dataset ad 62 clusters and 672 observations Results from 1000 simulations were te data is generated wit τ i following double exponential distribution and ɛ ij following a normal distribution. Eac dataset ad 62 clusters and 672 observations Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij also following a normal distribution. Eac dataset ad 27 clusters and 285 observations Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij following a log normal distribution. Eac dataset ad 27 clusters and 285 observations Results from 1000 simulations were te data is generated wit τ i following normal distribution and ɛ ij following a double exponential distribution. Eac dataset ad 27 clusters and 285 observations Results from 1000 simulations were te data is generated wit τ i following log normal distribution and ɛ ij following a normal distribution. Eac dataset ad 27 clusters and 285 observations Results from 1000 simulations were te data is generated wit τ i following double exponential distribution and ɛ ij following a normal distribution. Eac dataset ad 27 clusters and 285 observations Results from Levene s test for several measure of lumber strengt, as well as summary statistics. Here ICC is te estimated intra-class correlation for eac quantity fo interest Estimate for 10 t and 25 t percentile of measure of lumber strengt viii

10 A.1 Te 10 t and 25 t percentile of te sum of a τ i N(0, σ 2 τ ) and ɛ ij 1 LN(0, 1 σ 2 τ ) wit varying amounts of correlation A.2 Te 10 t and 25 t percentile of te sum of a τ i 1 LN(0, σ 2 τ ) and ɛ ij N(0, 1 σ 2 τ ) wit varying amounts of correlation A.3 Te 10 t and 25 t percentile of te sum of a τ i N(0, σ 2 τ ) and ɛ ij DE(0, 1 σ 2 τ ) wit varying amounts of correlation A.4 Te 10 t and 25 t percentile of te marginal distribution of observations from model (1.1) wen τ i DE(0, σ 2 τ ) and ɛ ij N(0, 1 σ 2 τ ) ix

11 List of Figures 2.1 Comparison between fourt order kernel and a second order kernel (standard normal pdf) Plots of (3.10), as a function of, for variaous values of σ 2 τ, for datasets wit a = 40 and b = 10, and τ i N(0, σ 2 τ ) and ɛ ij N(0, 1 σ 2 τ ). Additonally, we consider 500 resamples Visually summary of MOE, MOR, and MOE/MOR Kernel density estimate of marginal distribution of MOE of board. Te blue kernel density estimate uses resampling to compute te bandwidt, wile te red uses te Seater and Jones plug in metod Kernel density estimate of marginal distribution of MOR of board. Te blue kernel density estimate uses resampling to compute te bandwidt, wile te red uses te Seater and Jones plug in metod Kernel density estimate of marginal distribution of te ratio MOR/MOE of board. Te blue kernel density estimate uses resampling to compute te bandwidt, wile te red uses te Seater and Jones plug in, A.1 Above are plots of te marginal pdf of an observation from model (1.1) if τ i N(0, σ 2 τ ) and ɛ ij LN(0, 1 σ 2 τ ) A.2 Above are plots of te marginal pdf of an observation from model (1.1) if τ i 1 LN(0, σ 2 τ ) and ɛ ij N(0, 1 σ 2 τ ) A.3 Above are plots of te marginal pdf of an observation from model (1.1) if τ i N(0, σ 2 τ ) and ɛ ij DE(0, 1 σ 2 τ ) A.4 Above are plots of te marginal pdf of an observation from model (1.1) if τ i DE(0, σ 2 τ ) and ɛ ij N(0, 1 σ 2 τ ) x

12 Capter 1 Hierarcical Linear Models Hierarcical linear models are statistical models tat ave more tan one source of random variation. Tis type of linear model is considered wen data is collected by means of multistage sampling. Te first stage involves breaking te population into groups, called clusters, and randomly selecting several clusters. Te second stage is collecting a random sample from eac of te selected clusters. Hierarcical linear models are useful because inference can be made about all of te clusters, unlike fixed effects models were te only source of random variation is due to te sampling witin te groups. If a fixed effects model is employed, ten inference sould only be made concerning te clusters tat are represented in te experiment. In oter words, te population being considered by random effects models is larger tan tat of fixed effects models. Hierarcical data occurs often in manufacturing of goods. For instance, tere are many lumber mills in te United States. It is too time consuming and expensive to study lumber from eac mill. For practical purposes, we collect data from a muc smaller number of mills. Tere are many elements of lumber manufacturing tat can impact te quality of te final product. Possible examples are mills could get teir raw materials from different locations around te world, and mills may ave varying production protocols. Also, environmental factors like climate, may cause differences in te final product from eac mill. Te quality of te lumber from te same mill could be impacted by any of tese factors in a very similar way. In oter words, wile boards from te same mill are related, tey are not related to boards from oter mills. We aim to take tis data structure into consideration. We will be studying one-way random effect models wic ave two sources of random 1

13 variation. One source of random variation is due to randomness in te selection of te clusters, and te oter is due to randomness from coosing te units from eac cluster. An example of a linear model for ierarcical data is y ij = µ + τ i + ɛ ij i = 1,..., a, j = 1,..., n i, (1.1) were y ij is te value of te response variable of te j t observation from te i t cluster, and µ is te overall mean. We assume tat {τ 1, τ 2,..., τ a } is a collection of independent and identically distributed (i.i.d.) random variables. In many cases, eac τ i is considered to be a normal random variable wit mean 0 and variance στ 2, owever tese random variables are not necessarily required to be normally distributed. Similarly, {ɛ 11, ɛ 12,..., ɛ ana } are independent and identically distributed random variables. Commonly, {ɛ 11, ɛ 12,..., ɛ ana } are assumed to follow a normal distribution wit mean 0 and variance σɛ 2, but again te normality assumption is not required. Regardless of te distribution assumptions, every τ i and ɛ ij are independent of one anoter. Due to te additional source of random variation, te covariance structure of a ierarcical model is στ 2 + σɛ 2 i = i, j = j Cov(y ij, y i j ) = στ 2 i = i, j j 0 i i, 1 i = i, j = j and Corr(y ij, y i j ) = στ 2 /(στ 2 + σɛ 2 ) i = i, j j 0 i i. (1.2) Te correlation between observations witin te same cluster must be taken into account because as στ 2 increases, te information gained from a single observation can decrease significantly. If te data are independent and Corr(y ij, y ij ) = 0 for all j j, ten eac of te N = i n i observations contain te same amount of information. Alternatively, if te data are perfectly correlated, Corr(y ij, y ij ) = 1, ten eac observation from te same cluster is identical. Tus only one observation from eac cluster is required. In tat scenario tere are only a out of N observations tat contain information. A useful tool for analyzing a dataset is an analysis of variance table (ANOVA). Te ANOVA table is elpful for testing H 0 : σ 2 τ = 0 H 1 : σ 2 τ > 0. (1.3) 2

14 Source df SS MS E[MS] ( ) a n 2 i N a N Between a 1 SSB = n i (ȳ i ȳ ) 2 MSB = SSB/(a 1) σɛ 2 + στ 2 i=1 a 1 i=1 a a n i a Witin (n i 1) SSW = (y ij ȳ i ) 2 MSW = SSW/ (n i 1) σɛ 2 i=1 i=1 j=1 i=1 a n i Total N 1 SST = (y ij ȳ ) 2 i=1 j=1 Table 1.1: ANOVA table for a one way random effects model for general ierarcical data sets. Source df SS MS E[MS] a Between a 1 SSB = b (ȳ i ȳ ) 2 MSB = SSB/(a 1) σɛ 2 + bστ 2 Witin a(b 1) SSW = Total ab 1 SST = i=1 a i=1 j=1 a i=1 j=1 b (y ij ȳ i ) 2 MSW = SSW/(a(b 1)) σɛ 2 b (y ij ȳ ) 2 Table 1.2: ANOVA table for a one way random effects model for ierarcical data sets wit balanced data. b = n i for all i = 1,..., a. Te ANOVA table for te ierarcical linear model (1.1) can be found in Table 1.1. Te ANOVA table for model (1.1) reduces to te ANOVA table in Table 1.2, wen we consider balanced data (b = n i for all i = 1,..., a). Te following is focused on balanced data, were we ave te same number of observations from eac clusters. If te underlying population is normal, te test statistic for testing te ypotesis found in (1.3) is F 0 = MSB/MSW, wic under te null ypotesis follows an F -distribution wit a 1 numerator degrees of freedom and a(b 1) denominator degrees of freedom. Te null ypotesis is rejected if te between group variation is muc larger tan te witin group variation. For a reference for wat large values of F 0 are, under te null ypotesis E[F 0 ] = a(b 1)/(a(b 1) 2) and E[F 0 ] approaces 1 as eiter a or b increases. To find appropriate estimators for te variance components, σ 2 τ and σ 2 ɛ, we can solve te following system of equations, MSB = ˆσ 2 ɛ + bˆσ 2 τ MSW = ˆσ 2 ɛ. 3

15 Te above system yields ˆσ 2 τ = (MSB MSW )/b, ence an estimator for te variance of an observation is ˆσ 2 τ + ˆσ 2 ɛ = MSB/b + (1 1/b)MSW. (1.5) Notice tat bot te between and witin errors are involved to estimate te variance of an observation, as opposed to analyzing a random sample were we would use te sample variance to estimate te population variance. To illustrate possible implications of conducting inappropriate analysis tat ignores te covariance structure of random effects models, we conduct a simulation study to see ow often confidence intervals for te overall mean, µ, actually contain µ and we also record te mean margin of error (MOE). In tis simulation study, we will set µ = 0 and σ 2 τ = 1 σ 2 ɛ, and ten generate datasets via (1.1). We will construct 95% confidence intervals for µ assuming tat te data are a random sample, or tat is tere no correlation between any observations. A 95% confidence interval for µ based on a simple random sample is given by: ȳ ± t 0.975,ab 1 s ab, (1.6) were s is te sample standard deviation and t 0.975,ab 1 is te 97.5 percentile of a t-distribution wit ab 1 degrees of freedom. We also construct te following 95% confidence interval wic sould be used for ierarcical models like (1.1): MSB ȳ ± t 0.975,a 1. (1.7) ab Tere are two important differences to notice in above confidence intervals, te first is we only use te mean squares between (M SB) for te margin of error for (1.7), wile sample standard deviation, s, is used in (1.6). As στ 2 increases we expect te cluster means to become more spread out leading to a larger M SB, wile te sample standard deviation sould not cange dramatically and sould be approximately στ 2 + σɛ 2. Te second difference is tat te degrees of freedom used for te critical values, ab 1 degrees of freedom are used for te critical value in (1.6) and we only use a 1 degrees of freedom in (1.7). We know tat as we increase degrees of freedom, critical values of a t-distribution decrease for a fixed confidence level. Te disparity in te critical values, t 0.975,a 1 and t 0.975,ab 1, becomes more obvious wit fewer clusters. For instance, for a = 3 and b = 2, te 4

16 resulting critical values are t 0.975,2 = and t 0.975,5 = From Table 1.3, we see tat te average MOE beaves differently for te confidence intervals. If στ 2 = 0, and bot a and b are small, confidence intervals computed using (1.7) are muc wider, on average, tan confidence intervals constructed using (1.6); surprisingly confidence interval (1.6) as a iger coverage probability. However, if eiter a or b are reasonably large, ten te discrepancy between te MOE s is less noticeable for στ 2 = 0. Wen στ 2 = 0.95, te difference in te MOE is due to te critical value, since s 2 and MSB sould be relatively close to 1. We observe tat confidence intervals constructed via (1.6) become more narrow as σ 2 τ increases resulting in low coverage probability. On te oter and, confidence intervals constructed using (1.7) become wider as στ 2 increases and maintain coverage probabilities tat are always approximately Tis illustrates te value in using tools tat take te sampling sceme into account as opposed to blindly using formulas assuming data are i.i.d. We will focus on developing metods tailored to ierarcical linear models and we will be making comparisons to metods tat assume tat data are a random sample. Wile estimating te overall population mean is a goal of many studies, it is possible tat estimation of anoter population quantities or curves tat may be of interest. For instance, in te study of manufactured goods or quality control it may be important to estimate te p t percentile, Y p = inf{y : F (y) p}. Tere are many approaces to estimating percentile, wic will be discussed in Capter 4. Capter 2 and 3 will focus applying kernel density estimation to estimation te population density, f(y) = df (y) dy, for bot i.i.d. data (Capter 2), as well as ierarcical data (Capter 3). 5

17 Confidence Interval (1.6), df= ab 1 Confidence Interval (1.7), df= a 1 a b στ 2 Coverage Probability Average HW Coverage Probability Average HW Table 1.3: Comparison of coverage probability and alf-widt (HW) of different confidence intervals. Coverage probability refers to te percentage of simulations tat µ = 0 is included in confidence intervals (1.6) and (1.7). 6

18 Capter 2 Kernel Smooting for i.i.d. Data We seek an estimate for te marginal pdf for an observation from a continuous population using kernel density estimation. Tis capter will be a brief summary of many important topics in kernel density estimation. Furter details can be found in Wand and Jones (1994). Let Y 1, Y 2,..., Y n represent a random sample, ten a kernel density estimate of a pdf at a given value of y can be found by ˆf(y) = 1 n n ( ) y Yi K, (2.1) i=1 were K( ) is te kernel function, and is called te bandwidt. To aid in obtaining reasonable kernel estimates, te following assumptions are typically employed. 1. Te density f is suc tat its second derivative f is continuous, square integrable and ultimately monotone. A function is ultimately monotone if te function is monotonic over bot (, M] and [M, ) for some M > Let = n be a sequence of positive values suc tat lim n = 0 and lim n n =. n n 3. Te kernel K is a bounded probability density function aving finite fourt moment and is symmetric about te origin. Assumptions 1 and 3 can easily be weakened or even replaced wit oter conditions for f and K. It as been sown tat, if te above assumptions are met, ten te coice of te kernel function 7

19 Figure 2.1: Comparison between fourt order kernel and a second order kernel (standard normal pdf). does not severely impact te efficiency of te kernel density estimate. However, as we will see later, selection of plays an important role in te quality of a kernel estimate. If te above assumptions are used, ten µ 2 (K) > 0, were µ s (g) = y s g(y)dy, and K is called a second order kernel. Weakening te tird assumption, in particular not requiring K to be a probability density, can lead to iger order kernels. K is a r t order kernel if µ s = 0 for s = 1, 2,..., r 1 and µ r 0. We denote te r t order kernel by K [r]. Jones and Foster (1993) discovered a recursive formula to compute iger order kernels, K [r+2] (y) = 3 2 K [r](y) yk [r] (y). For example, if K [2] (y) = φ(y), were φ( ) is te standard Gaussian density function, ten a fourt order kernel is K [4] (y) = 1 2 (3 y2 )φ(y). Higer order kernels can lead to density and distribution estimates tat are difficult, or even impossible, to interpret due to possibly producing density estimates wit negative values or a potentially decreasing distribution function estimate. Wile te use of a suboptimal second order kernel function may not greatly impact te 8

20 efficiency of kernel density estimates, te coice of te bandwidt is crucial. Large bandwidts can lead to important features of te data being missed, wile small bandwidts can lead to noisy estimates. Te trade off between estimates being too smoot or too noisy is known as te variancebias trade off. We seek a bandwidt tat leads to a density estimate tat is not too noisy, yet reflects important features. Common bandwidt selection tecniques attempt minimize te asymptotic mean integrated squared error (AMISE) wic is minimized by [ ] 1/5 R(K) AMISE = µ 2 (K) 2 R(f, (2.2) )n were R(g) = g(y) 2 dy, and µ 2 (K) = y 2 K(y)dy. Optimizing te bandwidt requires some knowledge of te population, namely R(f ). Tere are many approaces to bandwidt selection. An example of bandwidt selection is te normal scale rule wic assumes tat te population follows a normal distribution. Oter examples of bandwidt selection are cross validation, bootstrapping metods, and direct plug-in metods. Tere are problems associated wit all of tese metods of bandwidt selection, but we will concentrate on plug-in metods. A direct plug in metod requires an estimate for R(f (s) ). Te relationsip R(f (s) ) = f (s) (y) 2 dy = ( 1) s f (2s) (y)f(y)dy will be elpful, and can easily be sown using integration by parts. R(f (s) ) can be tougt of as an expectation, E[f (r) (y)] = ψ r = ( 1) r/2 f (r) (y)f(y)dy. We estimate ψ r wit ˆψ r = n 1 n i=1 ˆf (r) (y i, g) = n 2 n n i=1 i=1 ( ) L (r) yi y j, (2.3) g were L (r) is te r t derivative of te kernel L and g is te bandwidt. To obtain an estimate for ψ 2 = R(f ) in te denominator of (2.2), we set r = 2 in (2.3). A practical concern would be te selection of te bandwidt, g. To select an appropriate bandwidt to estimate ˆf (y), we can find te bandwidt tat minimizes te AMISE of f (y), wic requires and estimate of te fourt derivative of te population. Te following process for finding bandwidt wic attempts to minimize te AMISE as been suggested by Seater and Jones (1993): 1. Estimate ψ 8 using te normal scale rule estimate ˆψ NS 8 = 105/(32πˆσ 9 ). 2. Estimate ψ 6 using te kernel estimator ˆψ 6 (g 1 ), were g 1 = { 2K (6) (0)/(µ 2 (K) 2 ˆψNS 8 n)} 1/9. 9

21 3. Estimate ψ 4 using te kernel estimator ˆψ 4 (g 2 ), were g 2 = { 2K (4) (0)/(µ 2 (K) 2 ˆψ6 (g 1 )n)} 1/7. 4. Te selected bandwidt is ĥ DP I,2 = {R(K)/(µ 2 (K) 2 ˆψ4 (g 2 )n)} 1/5. Plug-in metods are attractive because tey are completely data driven and do not tend to produce bandwidts tat are as volatile as some oter bandwidt selection procedures. In te following capter, we will discuss kernel density estimation for more complicated sampling tecniques, in particular a ierarcical sample. 10

22 Capter 3 Kernel Smooting for Hierarcical Data In tis capter, we will attempt to estimate te marginal density of an observation obtained from a multistage sampling sceme via kernel smooting. Tere as been minimal work done regarding smooting clustered data. Breunig (2001) proposed a metod tat requires te use of fourt order kernels. Higer order kernels were explored to avoid complicated polynomial expressions tat resulted from including more terms in te Taylor expansion to approximate te AM ISE. Higer order kernels are not required to be density functions, tus te resulting density estimate is not guaranteed to be a density. Tis can make kernel density estimates constructed from iger order kernel difficult to interpret. To account for general ierarcical data, Breunig proposed te following kernel density estimator at a given y ˆf(y) = 1 N a n i ( ) y Yij K i=1 j=1 a were N = n i. (3.1) i=1 If a iger order kernel is used, te bandwidt tat minimizes te AMISE is very difficult to estimate. A data driven bandwidt would involve bivariate kernel density estimation wic requires selecting an appropriate bandwidt matrix. Breunig did not address tis issue, rater e simply assumed tat te population follows a normal distribution and found an optimal bandwidt wit a normal scale rule. We seek an alternative approac tat does not require suc severe assumptions 11

23 te population distribution. Tere are several assumptions tat are typically made wen working wit clustered data. Observations from te same cluster are identically distributed but not independent; wile observations from different clusters are i.i.d. If tere were only one observation from eac cluster, ten we would ave a random sample of size a. An estimate of te population pdf could be constructed by appealing to te classic kernel density estimate approaces outlined in Capter 2. However, multistage samples contain multiple observations from eac cluster. Any combination of a observations tat are all from different clusters can be viewed as an i.i.d. sample and can be used to construct a density estimate. We present two different kernel density estimators tat combine kernel density estimates and are based upon i.i.d. samples. Te first estimator, wic can be used for any ierarcical dataset, is expressed as Q ˆf R (y) = w j ˆf j (y), (3.2) j=1 were were Y ij ˆf j (y) = 1 a a ( y Yij K i=1 ), (3.3) is randomly selected from Y i = {Y i1, Y i2,..., Y ini } wit replacement and eac element of Y i is equally likely to be selected. We also assume tat eac w j is non-negative and j w j = 1. Estimators of tis form ave been studied in Hoffman (2001) to implicitly account for te correlation between observations in te same cluster. Te second estimator selects eac observation witout replacement wic is only appropriate for balanced datasets, were n i = b for all i, in order to ensure tat eac cluster is represented in eac i.i.d. sample. We express te second estimator as b ˆf C (y) = w j ˆfj (y), (3.4) j=1 were ˆf j (y) = 1 a a ( y Yij K i=1 ), (3.5) were Y ij is te j t observed value from te i t cluster, for j = 1,..., n i, and w j 0 and j w j = 1. Hence, we can construct b density estimates so tat eac observation is used only once, and all observations will be used. Next, we will focus on approximating te MISE of (3.2). 12

24 In te following lemmas, we derive te bias and te variance of individual kernel density estimates based on resampled observations, as well as te covariance between two resampled density estimates. Te bias and variance will allow us to compute te AMISE, wic in turn will elp in obtaining an appropriate bandwidt for kernel density estimator (3.2). Proofs are provided in Appendix C. Lemma 3.1 If we consider ˆf j (y), in (3.3), as an estimator for f(y), ten te bias of ˆf j (y) is E[ ˆf j (y) f(y)] = 2 µ 2 (K)f (y) 2 + o( 2 ) and ˆf j (y) as variance Var( ˆf j (y)) = R(K)f(y) a + o((a) 1 ). Additionally, if 0 and a, ten ˆf j (y) is a consistent estimator of f(y). Lemma 3.2 Te covariance between two resampled density estimators, in (3.3), is Cov( ˆf j (y), ˆf a j (y)) = (a) 1 and approaces 0 as a 3. i=1 { R(K)f(y) + (n i 1)στ 2 an i an i 3 K ( ) } 2 y µ + O(a 1 ) wic is We can use Lemma 3.1 and Lemma 3.2 to write an expression for te variance of (3.2), Q Var( ˆf R ) = Var w j ˆf j (y) = = j=1 j=1 Q Q wj 2 Var( ˆf j (y)) + Q j=1 Q j=1 w 2 j w 2 j R(K)f(y) a R(K)f(y) a + + j=1 j j Q j=1 j j Q j=1 ( w j w j Cov ˆf j (y), ˆf ) j (y) w j w j a w j (1 w j ) a a i=1 { R(K)f(y) an i a i=1 + (n i 1)σ 2 τ an i 3 K { R(K)f(y) + (n i 1)στ 2 an i an i 3 K ( ) } 2 y µ + o((a) 1 ) ( ) } 2 y µ + o((a) 1 ). 13

25 Hence, te asymptotic MSE is AMSE( ˆf R (y)) = lim a E[( ˆf R (y) f(y)) 2 ] = lim E[ ˆf R (y) f(y)] 2 + lim Var( ˆf R (y)) a a Q = 4 µ 2 (K) 2 f (y) Q j=1 w j (1 w j ) a + a i=1 j=1 w 2 j R(K)f(y) a { R(K)f(y) + (n i 1)στ 2 an i an i 3 K AMISE is obtained by integrating wit respect to y, wic yields ( ) } 2 y µ. AMISE( ˆf R ) = AMSE( ˆf R (y))dy = 4 µ 2 (K) 2 f (y) 2 Q Q j=1 w j (1 w j ) a = 4 µ 2 (K) 2 R(f ) 4 + Q j=1 w j (1 w j ) a a i=1 + a i=1 j=1 w 2 j R(K)f(y) a { R(K)f(y) + (n i 1)στ 2 an i an i 3 K Q j=1 w 2 j R(K) a ( ) } 2 y µ dy { R(K) + (n i 1)στ 2 R(K } ) an i an i 2. (3.6) Note tat K ( ) y µ 2 dy = K (u) 2 du = R(K ), by letting u = y µ and du = 1 dy. To find te optimal weigts for (3.2), we seek a set of weigts tat minimizes te AMISE subject to j w j = 1. We will use a Lagrangian multiplier wic produces te following function, were λ is a constant tat is not restricted in sign, Q g(y) = λ 1 + Q j=1 j=1 w j w j (1 w j ) a + 4 µ 2 (K) 2 R(f ) 4 a i=1 + Q j=1 w 2 j R(K) a { R(K) + (n i 1)στ 2 R(K } ) an i an i 2. To find te optimal weigts, we take te partial derivative of g(y) wit respect to eac w j. Hence, 14

26 for all j, g(y) w j = λ + 2w jr(k) a + (1 2w j ) Tis leads to te following system of equations, λ = 2w 1R(K) a λ = 2w 2R(K) a λ = 2w QR(K) a. + (1 2w 1 ) + (1 2w 2 ) a i=1 { R(K) a 2 n i + (n i 1)στ 2 R(K } ) set a 2 n i 2 = 0. a { R(K) a 2 n i + (n i 1)στ 2 R(K } ) a 2 n i 2 a { R(K) a 2 n i + (n i 1)στ 2 R(K } ) a 2 n i 2 i=1 i=1 + (1 2w Q ) a i=1 { R(K) a 2 n i + (n i 1)στ 2 R(K } ) a 2 n i 2. Tis system can only be satisfied if all te weigts are equal. For tis to be te case, te optimal weigts are w j = Q 1 for all j. If all weigts are equal to Q 1, ten te AMISE becomes AMISE( ˆf R (y)) = 4 µ 2 (K) 2 R(f ) 4 + Q 1 a aq i=1 + R(K) aq { R(K) + (n i 1)στ 2 R(K ) an i an i 2 To compute te optimal bandwidt, differentiate AM ISE wit respect to yielding }. (3.7) AMISE( ˆf R (y)) = 3 µ 2 (K) 2 R(f ) R(K) a 2 Q Q 1 a aq i=1 { R(K) a 2 + 3(n i 1)στ 2 R(K ) n i an i 4 If we consider a balanced data set, were n i = b for all i, ten (3.8) becomes } set = 0. (3.8) AMISE( ˆf R (y)) = 3 µ 2 (K) 2 R(f ) R(K) a 2 Q Q 1 Q { R(K) ab 2 + 3(b 1)σ2 τ R(K ) ab 4 } set = 0. (3.9) Finding te roots to equation (3.9) is equivalent to finding te roots of te following sevent order 15

27 polynomial in : µ 2 (K) 2 R(f ) 7 (b + (Q 1)) ( ) R(K) 2 3(b 1)(Q 1)σ2 τ R(K ) set = 0. (3.10) abq abq Te solution for (3.10) wen σ 2 τ = 0 is = [ ] 1/5 (b + (Q 1))R(K) µ 2 (K) 2 R(f. )abq Additionally, in te case tat σ 2 τ = 0, an infeasible solution to (3.10) is = 0. Note tat is different from te bandwidt tat minimizes AM ISE for i.i.d. data (2.2), but te bandwidt remains O((ab) 1/5 ). For στ 2 > 0, we are guaranteed at least one zero for > 0. Wen = 0, te left and side of (3.10) is negative and eventually te leading term in te polynomial will dominate, as increases, forcing te value for te left and side of (3.10) to be positive. Using te Intermediate Value Teorem, tere must be at least one positive solution to te sevent order polynomial. To determine if te estimate of AMISE is a convex function of, te second derivative of AMISE wit respect to is 2 AMISE( ˆf R (y)) 2 = 3 2 µ 2 2(K)R(f (b + (Q 1))R(K) ) + 2 ab (b 1)σ2 τ R(K ) ab 5. Notice tat every term in te above equation is positive, ence AMISE is a convex function for > 0. A solution to (3.10) is te global minimum for > 0. As te intra-class correlation increases, te positive root of (3.10) increases. For an example of te beavior of te roots of 3.10, we assume bot τ i and ɛ ij follow a normal distribution, values of roots are given in te Table 3.1 below for selected values of σ 2 τ, wit a = 40, b = 10, and Q=500. Additionally in Figure 1, we see plots of (3.10) wit various values of σ 2 τ, under te assumption tat τ i N(0, σ 2 τ ) and ɛ ij N(0, 1σ 2 τ ). στ 2 Root Table 3.1: Solutions to (3.10) for various values of σ 2 τ, for datasets a = 40 and b = 10, were τ i N(0, σ 2 τ ) and ɛ ij N(0, 1 σ 2 τ ). Additonally, we consider 500 resamples. 16

28 σ τ 2 = 0 Sevent order polynomial in (3.11) σ τ 2 σ τ 2 = 0.1 f() 3e 05 1e 05 1e f() σ τ 2 = 0.25 σ τ 2 = 0.5 f() f() σ τ 2 = 0.75 σ τ 2 = 0.95 f() f() Figure 3.1: Plots of (3.10), as a function of, for variaous values of σ 2 τ, for datasets wit a = 40 and b = 10, and τ i N(0, σ 2 τ ) and ɛ ij N(0, 1 σ 2 τ ). Additonally, we consider 500 resamples. 17

29 An additional property of interest may be te asymptotic distribution of ˆf R (y). Te next teorem will sow tat ˆf R (y) can be asymptotically normal. Teorem 3.1 Assume tat 0, (a) 1/2 2 0, and a 3, let te resampled kernel density estimate be defined as were ˆf R (y) = 1 ˆf j (y), Q ˆf j (y) = 1 a j=1 a ( ) y Yij K k i=1 for very large Q. If (a) 1, and 0 as a, ten (a) 1/2 ( ˆf R (y) f(y)) N ( 0, σ 2), (3.11) were as a. σ 2 = R(K)f(y) Q + Q 1 Q a { R(K)f(y) i=1 an i + (n i 1)σ 2 τ an i 3 K ( )} y µ (3.12) Te assumption (a) 1/2 2 0 ensures tat te squared bias approaces 0 faster tan te variance approaces 0. We also need to assume tat a 3 to ensure tat te asymptotic variance is finite. Teorem 3.1 olds wen a β, were β (1/5, 1/3). If te usual assumptions, namely (a) 1, are made ten te squared bias and variance converge to 0 at te same rate, causing te mean of (a) 1/2 ( ˆf R (y) f(y)) to depend on a. Now we focus on bandwidt selection for estimator (3.2). Since ˆf j (y), for j = 1, 2,..., Q, is based on a random sample, we can use Seater and Jones (1991) metod to find an optimal bandwidt for eac density estimate, tese bandwidts will be denoted by j. Eac of te bandwidts are independent of te amount of intra-class correlation and contain information from eac cluster, making eac one a potential candidate for te bandwidt. We could ten average tese Q bandwidts, R = 1 Q j (3.13) Q j=1 were j is te bandwidt using te jt resample, and use te result as te bandwidt in (3.2). Anoter bandwidt selection metod involves solving eac bandwidt computed via Seater and 18

30 Jones (1991) metods for R(f ) in (2.2), R(f ) = R(K) 5 µ 2 (3.14) 2 (K)a, providing Q estimates for R(f ), wic can be averaged, ten substituted into (2.2) producing [ ] 1/5 R(K) R(f ) = µ 2 (K) 2 R R (f, (3.15) )aq were R R (f ) = 1 Q Q R(K) ( j )5 µ 2 (3.16) 2 (K)a. j=1 We average eac estimate of R(f ) because eac contains independent observations and te same amount of information about teir respective cluster. If a balanced ierarcical dataset is being analyzed, it may not be necessary to use observations more tan once. Tis would avoid te use of resampling, wic significantly reduces te computational burden. Anoter advantage of estimator (3.4) is te variance will be sligtly less complicated compared to te variance of ˆf(y). Te AMISE for ˆf C (y) is AMISE( ˆf C (y)) = 4 µ 2 (K) 2 R(f ) 4 + R(K) ab + (b 1)σ2 τ R(K ) ab 2. (3.17) Te bandwidt tat minimizes (3.17) is te solution to te following sevent order polynomial, ( ) R(K) µ 2 (K) 2 R(f ) 7 2 3(b 1)σ2 τ R(K ) set = 0. (3.18) ab ab Like polynomial (3.10), polynomial (3.18) only as one positive root. Listed below in Table 3.2 are te roots of (3.18) for several values of intra cluster correlation. Te solutions to (3.18) tend to be στ 2 Root Table 3.2: Solutions to (3.18) for various values of στ 2, for datasets a = 40 and b = 10, and τ i N(0, στ 2 ) and ɛ ij N(0, 1 στ 2 ). 19

31 smaller tan te solutions to (3.10). We will also explore analogous bandwidt tecniques to tose proposed for ˆf R (y). We will average b bandwidts, wic are found using Seater and Jones plug in metod. We also estimate R(f ) based on average of b estimates of R(f ), [ ] 1/5 R(K) R(f ) = µ 2 (K) 2 R C (f, (3.19) )ab were R C (f ) = 1 b b R(K) ( j )5 µ 2 (3.20) 2 (K)a. j=1 Next, we conduct simulations to compare kernel density estimates (3.4) and (3.2) by comparing M ISE, using many of te bandwidt selection metods outlined above. Simulations A variety of simulations were conducted to study te beavior of several metods of bandwidt selection. Bot τ i and ɛ ij will be randomly generated from normal, double exponential (eavy-tailed and symmetric distribution), and log normal (skewed distribution) distributions. A brief description of bot te log normal and double exponential distributions as well as parameterizations tat were used are included in Appendix A. Tere are nine possible combinations of distributions for τ i and ɛ ij and te cases we considered are displayed Table 3.3. Additionally six value of intra-class correlation were studied wic ranging for 0 (i.i.d. data) to 0.95 (extremely correlated data) for eac combination of distributions tat were considered. Distribution of τ i Normal Log Normal Double Exponential Distribution of ɛ ij Log Normal x Normal x x x Double Exponential x Table 3.3: List of all possible combinations for distributions of τ i and ɛ ij, were x denotes te combinations tat were considered. In addition to studying te effect of different distributions, we will consider bot balanced and unbalanced data sets. For balanced data, eac simulation will be conducted wit sample size of 400 consisting of 40 clusters wit 10 observations. For unbalanced data, two separate simulation studies were conducted. Eac study ad te same sample size and number of clusters for eac 20

32 simulated dataset. Te first configuration as 62 clusters wit sample size of 672, and te second configuration as 27 clusters wit a sample size of 285. We will be comparing seven metods of bandwidt selection. We observe wic metod produces te smallest M ISE and study te beavior of te bandwidt as te intra-class correlation canges. We implement Seater and Jones direct plug-in metod (SJ), averaging multiple bandwidts computed wit only one observation from eac cluster witout resampling (ĥc), appropriate for balanced data, and wit resampling (ĥr). We can obtain an estimate for R(f ), by averaging R(f ) based on one observation from eac cluster. We introduced two scemes to estimate R(f ), (3.20) used eac observation once ( R(f ) C ), wic relies on balanced data, and (3.16) used observations more tan once ( R(f ) R ). Lastly, we use numerical metods to find a solution to (3.10) and(3.18); tis solution corresponds to te bandwidt tat minimizes our estimate for M ISE. Numerical metods are required to find solutions to equation (3.10) and (3.18). Te root- Solve package in R was used to find tese solutions. If we examine te following Taylor expansion centered about µ, ( ) y Yij 1 K = l K(l) l=0 ( y µ ) (Y ij µ) l, were K (m) indicates te m t derivative of te kernel function wit respect to Y ij, we typically consider terms tat are o( 4 ) or o((n) 1 ) ignorable, but, eac term wit a iger order derivative could be included in te Taylor approximation of covariance of two dependent kernel density estimates. Additionally, te joint beavior of te estimates is not caracterized by tis approximation. Tese metods perform best for extreme levels of correlation, σ 2 τ large, but can yield unexpected results for some of our simulations. Wen bot τ i and ɛ ij are normally distributed, results in Tables 3.4, 3.9 and 3.14, te bandwidts selected via ĥc, ĥr, R(f ) C, and R(f ) R tend to remain similar as σ 2 τ increases. Regardless of te value of σ 2 τ, eac observation marginally follows te standard normal distribution. Tere is not a large difference in MISE for ĥc and ĥr, suggesting tat analysis of a balanced dataset may not require resampling, wic will significantly cut down te computational burden. For low values of σ 2 τ, tere is not an advantage to select bandwidts using ĥc, owever wen σ 2 τ exceeds 0.5, ĥc tends to produce a better bandwidt tan Seater and Jones direct plug-in metod. For σ 2 τ = 0.95, finding te roots of polynomials (3.10) and (3.18) tend to produce bandwidts leading to te smallest MISE. We also notice tat estimating R(f ) via resampling provides a result tat 21