Estimating joinpoints in continuous time scale for multiple change-point models

Transcription

1 Computational Statistics & Data Analysis 5 (2007) wwwelseviercom/locate/csda Estimating joinpoints in continuous time scale for multiple change-point models Binbing Yu a,, Michael J Barrett a, Hyune-Ju Kim b, Eric J Feuer c a Information Management Services, Inc 250 Prosperity Dr Suite 200, Silver Spring, MD 20904, USA b Department of Mathematics, 25 Carnegie Building, Syracuse University, Syracuse, NY , USA c Statistical Research and Applications Branch, National Cancer Institute, 66 Executive Boulevard, Suite 504, Bethesda, MD , USA Received 5 February 2006; received in revised form 2 July 2006; accepted 29 July 2006 Available online September 2006 Abstract Joinpoint models have been applied to the cancer incidence and mortality data with continuous change points The current estimation method [Lerman, PM, 980 Fitting segmented regression models by grid search Appl Statist 29, 77 84] assumes that the joinpoints only occur at discrete grid points However, it is more realistic that the joinpoints take any value within the observed data range Hudson [966 Fitting segmented curves whose join points have to be estimated J Amer Statist Soc 6, ] provides an algorithm to find the weighted least square estimates of the joinpoint on the continuous scale Hudson described the estimation procedure in detail for a model with only one joinpoint, but its extension to a multiple joinpoint model is not straightforward In this article, we describe in detail Hudson s method for the multiple joinpoint model and discuss issues in the implementation We compare the computational efficiencies of the LGS method and Hudson s method The comparisons between the proposed estimation method and several alternative approaches, especially the Bayesian joinpoint models, are discussed Hudson s method is implemented by C++and applied to the colorectal cancer incidence data for men under age 65 from SEER nine registries 2006 Elsevier BV All rights reserved Keywords: Constrained least square; Cancer incidence and mortality; Joinpoint regression; SEER Introduction It is of great importance to describe the trend of cancer incidence and mortality data The joinpoint regression model, which is composed of a few continuous linear phases, is often useful to describe changes in trend data Suppose that for the observations {(x,y ),,(x n,y n )}, x x n, the responses y i = E(y x i ) + e i,i=,,n, with E(e i ) = 0 and V(e i ) = σ 2 i for random errors e i The joinpoint regression models assume that, in each segment, the E(y x) follows a linear model E(y x) = β k,0 + β k, x, if τ k <x τ k, k =,,K +, () where τ 0 =, τ K+ = and E(y x) is continuous throughout [x 0,x n ], such that β k,0 + β k, τ k = β k+,0 + β k+, τ k for k =,,K (2) Corresponding author Tel: ; fax: addresses: yub@imswebcom, whybb@yahoocom (B Yu) /$ - see front matter 2006 Elsevier BV All rights reserved doi:006/jcsda

2 B Yu et al / Computational Statistics & Data Analysis 5 (2007) As the response is continuous at the change points, we call model () the joinpoint model and the τ k s joinpoints (JPs) This model is also called segmented-line regression model or piecewise linear model (Kim et al, 2004) An alternative parameterization of the JP model () is E(y x) = β 0 + β x + K δ k (x τ k ) +, (3) k= where δ k = β k+, β k, and (x τ k ) + = x τ k if x τ k and 0 otherwise This parameterization implicitly satisfies the continuity of E(y x) at τ k The current estimation method is the grid search (LGS) method proposed by Lerman (980), which is implemented by Joinpoint software developed by US National Cancer Institute ( Although the LGS method can be refined such that the JPs could occur at the middle point or quarterly point between two data points, the computation time for finer grid increases dramatically Hence, the LGS method is practical only when the JPs occur at the observed data points Hudson (966) described the continuous algorithm in detail for a one-jp model and discussed its extension to a model with more than two JPs, which is not straightforward Our aims in this paper are to describe the details of the extension to a multiple JP model and to compare computational efficiencies of these two fitting methods Several alternative methods have been proposed to estimate the locations of the change points for single series in different contexts For example, Quandt (958) and Quandt and Ramsey (978) proposed the procedure of estimating a single change point without continuity constraint at response in economics settings, Hinkley (969, 97) discussed the estimation and inference for the joinpoints in one-joinpoint models, Smith (975), Carlin et al (992), Slate and Turnbull (2000) and Tiwari et al (2005) use Bayesian approaches to estimate the change points under different scenarios Most of the available methods estimate the single change/join point The proposed method in the paper estimates the multiple joinpoints in continuous scale, hence it provides a better fit The rest of the paper is organized as follows: The model formulation and notation are described in Section 2 and Hudson s method for a one-jp model is reviewed in Section 3 In Section 4, Hudson s method is extended to a multiple JP model and the issues arising in the implementation are discussed Then the multiple JP model is applied to colorectal cancer incidence data for men under age 65 from the SEER nine registries The relative merits of different approaches are discussed in the final section 2 Model formulation and notation Let the kth segment denoted by S k ={x i : τ k <x i τ k }={x ik +,,x ik } for i 0 = 0 and i K+ = n For each segment S k,k=,,k +, we define that Y k = y ik + y ik x ik +, X k =, ɛ k = x ik e ik + where E(ɛ k ) = 0, Cov(ɛ k ) = Σ k and the weight matrix W k = Σ k Let Y X 0 β Y =, X =, β = Y K+ 0 X K+ e ik β K+ ( ) βk0, β k =, β k, ɛ = ɛ ɛ K+ Notice that Y = (y,,y n ) and ɛ = (e,,e n ) Then, the JP model () can be expressed as Y = Xβ + ɛ, with constraints (2), where E(ɛ) = 0, Cov(ɛ) = Σ Let τ = (τ,,τ K ) To fit this model, we find the estimates (ˆτ, ˆβ), of the JPs τ and the regression coefficients β which minimize the weighted sum of squared error (SSE) R(τ, β) = (Y Xβ) T W(Y Xβ),

3 2422 B Yu et al / Computational Statistics & Data Analysis 5 (2007) where W = Σ When the e i s are independent, Σ k, hence Σ, are diagonal matrices Especially if V(e i ) = σ 2 /w i, then Σ = σ 2 Diag(w,,w n ), and R(τ, β) can be simplified as K+ K+ (Y k X k β k ) W k (Y k X k β k ) = σ 2 k= k= x i S k w i [y i (β k,0 + β k, x i )] 2 In general, however, Σ k and Σ are non-diagonal matrices For example, for AR() model, the ijth element of Σ is σ ij = Cov(e i,e j ) = σ 2 i j /( ), 0 < 3 Review of Hudson s method: one JP τ In this section, we first summarize Hudson s algorithm for the -JP model The procedure to estimate ˆτ is described as follows: (a) For the partition [x,x i ], [x i+,x n ], 2 i n 2, fit the least square (LS) regression for each segment Let y y i+ x x i+ Y =, Y 2 =, X =, X 2 = y i y n x i x n The unconstrained weighted LS estimates are β k = ( β k,0, β k, ) = (X k W kx k ) X k W ky k, k =, 2 (4) (b) Let τ (i) be the solution to the equation β 0 + β τ = β 20 + β 2 τifx i τ (i) <x i+, then ˆτ (i) is called in the right place That means the two unconstrained regression lines cross between the two observations x i and x i+ (b) If ˆτ (i) is in the right place, then let R(i) = ρ + ρ 2, where ρ and ρ 2 are the unconstrained SSE for the two segments (b2) Otherwise, ˆτ (i) / [x i,x i+ ), ie, two unconstrained regression lines cross outside the interval [x i,x i+ ) Then we need to adjust the unconstrained LS estimates β = ( β, β 2 ) to the constrained estimates ˆβ = (ˆβ, ˆβ 2 ) and set ˆτ (i) = x i (Hudson, 966, Appendix 2) The linear constraint is Aˆβ = 0, where A = (,x i,, x i ) and ˆβ = (ˆβ,0, ˆβ,, ˆβ 2,0, ˆβ 2, ) Let ( C = (XWX) (X = W X ) ) 0 0 (X 2 W 2X 2 ) Using the method of Lagrange Multipliers, the constrained LS estimate is ˆβ = β C A [AC A ] A β (5) For -JP model, t = AC A and s = A β are scaler numbers Hence, ˆβ = β s t C A, and the adjusted SSE is given by R(i) = ρ + ρ 2 + s 2 /t (c) Repeat (a) (b) for all of i to choose the τ (i) which minimize R(i), that is τ = arg τ (i) min R(i) However, we do not need to make adjustment (b2) for all i since we may rule out some cases (Hudson, 966) As the adjustment with linear constraint in (b2) always increases the SSE, Hudson (966) proved that (a) we need not try ˆτ (i) = x i or ˆτ (i) = x i+ if ˆτ is in the right place; (b) Even if ˆτ (i) is not in the right place, no further adjustment is necessary if the constrained SSE R(i) = ρ + ρ 2 is larger than some previously obtained SSE

4 B Yu et al / Computational Statistics & Data Analysis 5 (2007) Estimation of multiple JP model in continuous scale For a K-JP model, there are K + segments, S,,S K+ and K JPs The kth JP τ k [x ik,x ik +) divides segments S k and S k+ Recall that the unconstrained LS estimates that minimize R(τ, β) in (4) are β = ( β,, β K+ ) = (X WX) X WY (6) When the e i,i=,,n, are independent, then W is block diagonal and (X W X ) 0 X W Y (X WX) =, X WY =, 0 (X K+ W K+X K+ ) X K+ W K+Y K+ and β k = (X k W kx k ) X k W ky k The kth JP ˆτ k is obtained by solving equation β k,0 + β k, τ k = β k+,0 + β k+, τ k Let T k denote the location of ˆτ k If the estimated JP ˆτ k is in the right place, ie, x ik < ˆτ k <x ik +, then T k = ; otherwise, T k = 2 and further adjustment is needed In the ideal situation, all ˆτ k s from unconstrained LS regression are in the right places, ie, ˆτ k (x ik,x ik +) for k =,,K, no adjustment is needed and (ˆτ,,ˆτ K )are the final estimates of JPs Otherwise, some ˆτ k s need to be adjusted to x ik and the LS estimate β needs to be adjusted subject to continuity constraints Let x i x i 0 0 Q K (2K+2) = 0 x ik x ik x ik x ik The continuity constraint at ˆτ k = x ik implies that Q(k, )β = 0, where Q(k, ) is the kth row of Q Let A be the constraint matrix such that Aβ = 0 For example, if τ k and τ l are both adjusted to x ik and x il, then row k and row l of Q will be added into matrix A Then the estimate of β with constraint Aβ = 0(Plackett, 960, p 53) is ˆβ = β (X WX) A [A(X WX) A ] A β, and the corresponding SSE is R(ˆτ, ˆβ) = R( τ, β) + (A β) [A(X WX) A ] A β In the rest of this section, we discuss several issues arising in the implementation of the Hudson s method for the multiple JP regression Given the estimates of the joinpoints (ˆτ,,ˆτ K ), the covariance matrix of the regression coefficients ˆβ and the confidence intervals for the JPs can be calculated as described by Lerman (980) 4 Comparison of the computations between the LGS method and the Hudson s method To find the global minimum of SSE using the LGS method, the number of necessary trials (LS calculations) is ( G K ), where G is the number of grid points When only data points are used as grid, then G = n and when the midpoints are inserted as grid, then G = 2n When Hudson s method is used, the location of the joinpoints need to be taken in consideration For each partition, you may try all possible adjustments for each partition The number of possible trials for a K-JP model is given by (Hudson, 966) K ( )( ) 2 r K n K 2 r r r=0

5 2424 B Yu et al / Computational Statistics & Data Analysis 5 (2007) Table Maximum number of trials for a K-JP model when n = 30 K Hudson s method ,5 24,60 Grid-search using only data points ,920 Grid-search by inserting one midpoint , ,845 Typically, in the analysis of cancer incidence and mortality data, n 30 and the upper limit for the number of JPs is 4 Table shows the maximum number of trials from the Hudson s method, the LGS method using only data points, and the LGS method by inserting one midpoint between two data points The LGS method using only data points takes the least number of trials The number of trials for the Hudson s method is less than the LGS method with midpoint inserted Practically, no adjustment is needed for the LGS method, so each trial of the Hudson s method takes longer time than that of the LGS method The Joinpoint software uses a permutation test to select the optimal JP model (Kim et al, 2000) The permutation test procedure sequentially conducts the tests of the null hypothesis so that there are k 0 JPs against the alternative of k JPs until we reach to a conclusion, where 0 k 0 <k 3 At each level of testing, the models with k 0 and k joinpoints are fitted for each of the N permuted data and N is usually large to generate the permutation distribution of the test statistic When fitting a JP model without model selection, the difference between the LGS method on data points and the Hudson s method is not noticeable, both finish in a few seconds When the permutation test is used for model selection, the LGS method using data points as the grid is the fastest, and the Hudson s method becomes substantially longer, which is about the time of the LGS method with three grid points inserted between consecutive data points This is because of the extra comparisons needed to check the joinpoint location, which is further discussed below However, the time of the LGS method with nine grid points inserted is daunting One major advantage of Hudson s method is that the location of JP is continuous and it provides a better model fit than the LGS method does 42 Implementation of a multiple JP model IfaJPˆτ k is not in the right location, ie, ˆτ k / [x ik,x ik +), it could be adjusted to either x ik or x ik + If two adjacent JPs ˆτ k and ˆτ k+ are not in the right locations, adjusting one JP could automatically change the adjacent JP to the right location Hence if L JPs are not in the right locations, the maximum number of possible adjustments would be ( L )2 + ( L 2 )22 + +( L L )2L = 3 L To speed up the Hudson s method, only left adjustment is necessary, ie, ˆτ k only needs to be adjusted to x ik The approximate number of adjustments are ( L ) + ( L 2 ) + +( L L ) = 2L, which is substantially less than 3 L The exceptional cases are i k+ i k = 4, when ˆτ k needs to be adjusted to both x ik and x ik +, and i k = n 3, when ˆτ k needs to be adjusted to both x n 3 and x n 2 Let P be one of the partitions and R min be the current minimum of unconstrained SSE The initial value of R min = The steps to find the estimates of (τ, β) for a K-JP model are as follows: () For the partition P, find the unconstrained LS estimates β k for each segment S k Calculate the total unconstrained SSE R (0) P = K+ k= ρ k, where ρ k is the unconstrained SSE for the kth segment If R (0) P R min, then stop and try another partition; otherwise, go to 2 (2) Calculate the JPs ˆτ k of the regression lines from S k and S k+,k=,,k (a) If all ˆτ k s are in the right places, then update R min = R (0) P and go back to step (b) If some τ k s are not in the right places and we need to adjust those τ k to x ik Let A k = (0) indicate whether a JP τ k needs adjustment (or not) For example, for a model with three JPs (τ, τ 2, τ 3 ), the possible adjustments (A,A 2,A 3 ) are (, 0, 0),(0,, 0), (0, 0, ), (,, 0), (, 0, ), (0,, ), (,, ) (i) For each adjustment, check whether the JPs after adjustment are all in the right places, ie, τ k [x ik,x ik +) If they are, calculate the adjusted SSE R (m) P, =,,M; otherwise, set R(m) P = (ii) If min(r () P,,R(M) P ) R min then update R min ; otherwise, go to step (3) Try all possible partitions, then the global minimum SSE is R min and the corresponding estimates of (τ, β) are the final estimates

6 43 Restrictions on the JP locations B Yu et al / Computational Statistics & Data Analysis 5 (2007) Although the JPs can occur anywhere within the range of observed data, some restrictions apply For example, two JPs may not be too close to each other and a JP may not occur too early or too late The default options in the current Joinpoint software restrict that, including the data points that are also JPs, the minimum number of data points between two JPs is 4 and the minimum number of data points from a JP to either end of the data is 3 These restrictions are necessary to calculate the standard errors of the regression coefficients Suppose that the estimated JPs ˆτ k [x ik,x ik +), k =,,K In order to include at least three data points between the ˆτ k and either end, it should satisfy that x 3 ˆτ and ˆτ K x n 3 In order to contain at least four data points between ˆτ k and ˆτ k+, then i k+ i k 4ifˆτ k (x ik,x ik +) and i k+ i k 3ifˆτ k = x ik 5 Application In the Annual Report to the Nation on the Status of Cancer, jointly released by the National Cancer Institute (NCI), the American Cancer Society (ACS), the North American Association of Central Cancer Registries (NAACCR), and the Centers for Disease Control and Prevention (CDC), including the National Center for Health Statistics (NCHS), the rate of new cancer cases and deaths for all cancers combined as well as for most of the top 0 cancer sites were reported The joinpoint regression models were used to analyze the changing trends of cancer incidence and mortality rates over successive segments of time, and to estimate the amount of increase or decrease within each time period The report includes a special section on colorectal cancer, which has the third highest incidence of any cancer site Using JP regression with annual grid, the report shows that overall incidence increased until 985 and then began decreasing steadily at an average rate of 6% per year In this application, we consider the colorectal cancer incidence rates from 976 to 999 for the male less than age 65 and compare the JP model estimated using the LGS method and the one estimated using the Hudson s method The data were extracted from the nine cancer registries in the National Cancer Institute s Surveillance, Epidemiology, and End Results (SEER) program, which covers approximately 0% of the US population The response variable for the JP analysis, y, is the natural logarithms of age-adjusted colorectal cancer incidence rate per 00,000 people The range of the possible number of JPs is set at the default for the Joinpoint software with minimum 0 and maximum 3, as most cancer trend data has up to three joinpoints The trend of the incidence rates is represented by annual percent change (APC), where for the kth segment, the APC is (exp(β k ) ) 00% The permutation test procedure (Kim et al, 2000) was used to select the best model among the zero- to three-jp models When the default restrictions are used, ie, the middle segment should have at least four data points, both methods choose a zero-jp model as the best model The APC is 04% with CI ( 064%, 07%), indicating that the colorectal cancer incidence rates has decreased slowly, but significantly from 976 to 999 To allow more rapid changes, we decrease the number of data points in the middle segment from 4 to 2 The estimates and confidence intervals (CIs) of the JPs and the sum of squared error (SSE) from both methods are shown in Table 2 From Table 2, we see that the SSEs for the zero- and one-jp models are identical for both methods The SSEs for the two-jp model from both methods are very close However, the SSE for the three-jp model from the Hudson s method Table 2 SSE and the estimated JPs with 95% CI from both methods No JPs Grid-search method Hudson s method SSE JP (95% CI) SSE JP (95% CI) τ = 985 (982, 988) 4743 τ = 9850 (9826, 9878) τ = 986 (978, 99) 3634 τ = 9860 (977, 9908) τ 2 = 987 (982, 997) τ 2 = 987 (9830, 9979) τ = 983 (978, 987) 845 τ = 9832 (9773, 9840) τ 2 = 985 (983, 996) τ 2 = 9855 (9840, 9864) τ 3 = 988 (985, 997) τ 3 = 987 (986, 9958)

7 2426 B Yu et al / Computational Statistics & Data Analysis 5 (2007) Fig Plot of final models selected by grid-search and Hudson s methods is much smaller than that from the LGS method As we see from Fig, the 3-JP model from the LGS method only allows the JPs at the data points Also, Hudson s method yields narrower CIs for the JPs, especially for τ 2 under the three-jp model The improvement in SSE and CI is because the JPs are not restricted to be at the observed data points in the Hudson s method When the permutation test procedure is used to select the optimal model, the final model using the grid search method is still a zero-jp model; whereas a three-jp model is selected when the Hudson s method is used The plots of the final models from both methods are shown in Fig From the three-jp model selected by Hudson s method, we see a spike at 9855 Starting from 9832, the incidence rate increased dramatically until 9855, then decreased sharply until 987 This spike in incidence rates might be due to the presidential effect (Brown and Potosky, 990) Brown and Potosky examined the public health impact of mass media coverage of President Reagan s colon cancer episode of 985 They also found a sharp but somewhat transitory increase in public interest following the diagnosis of the President s colon cancer, with a corresponding increase in early detection tests Their analysis of the incidence data showed an increase in early stage colorectal cancers in the months following the President s diagnosis and a decrease in advanced disease in , suggestive of a screening effect The new trend represented by the three-jp model may shed light on the use and usefulness of colorectal cancer screening Although we do not know the true underlying model for the colorectal cancer incidence rates, the Hudson s method is more sensitive and has more power to discover the new trend which is missed by the grid-search method Furthermore, the Hudson s method always provides a smaller SSE, hence more accurate estimates of the regression coefficients and APCs The comparisons between the LGS method and the Hudson s method regarding their effects on inference of the regression parameters are addressed in detail in a companion paper (Kim et al, 2006) 6 Discussion In this paper, we discuss the computational details of estimating multiple joinpoints in continuous time scale, and compare the computational efficiencies of the two fitting methods, the Hudson s method and the Lerman s grid search method In summary, the Hudson s method takes longer time than the basic grid search where only the data points serve as the grid points, but it is more efficient than a grid search with more than four points inserted between the consecutive data points To illustrate other advantages of the Hudson s method, we applied both methods to male colorectal cancer incidence rate data and found that the Hudson s method provides estimates with smaller biases Because the Hudson s method does not restrict that the JPs occur at the data points, it provides a better fit than the grid search method This enables us to describe the cancer incidence and mortality trend more accurately The extension of the Hudson s method is able to fit multiple JP regression model and the computation time of fitting a K-JP model

8 B Yu et al / Computational Statistics & Data Analysis 5 (2007) is faster than the fine grid-search with one midpoint The extended Hudson s method is currently implemented in the Joinpoint software developed by the National Cancer Institute ( Several other alternative approaches are proposed for multiple change/join point problems for single time series (Smith, 975; Carlin et al, 992) The Bayesian approach with Markov Chain Monte Carlo (MCMC) is becoming more popular as the computer is more powerful Using the data concerning prostate specific antigen serial markers for prostate cancer, Slate and Turnbull (2000) compare two joinpoint models, ie, the fully Bayesian hierarchical change point model and the latent disease process model Tiwari et al (2005) compare different model selection procedures in Bayesian joinpoint models The Bayesian method with MCMC usually takes more time to fit a multiple joinpoint model, depending on the number of MCMC runs However, the Bayesian approach is able to produce posterior distributions of the parameters, particularly the posterior distribution of number and the locations of the joinpoints As Tiwari et al pointed out (2005), the Bayesian methods is a useful companion of the frequentist methods, since the posterior distribution of the number and location of the joinpoints gives additional insight to compare different joinpoint models Hence, it is of interest to study how these different estimation methods perform under different situations References Brown, ML, Potosky, AL, 990 The presidential effect: the public health response to media coverage about Ronald Reagan s colon cancer episode The Public Opinion Quarterly 54, Carlin, BP, Gelfand, AE, Smith, AFM, 992 Hierarchical Bayesian analysis of change point problems Appl Statist 4, Hinkley, DV, 969 Inference about the intersection in two-phase regression Biometrika 56, Hinkley, DV, 97 Inference in two-phase regression J Amer Statist Soc 66, Hudson, DJ, 966 Fitting segmented curves whose join points have to be estimates J Amer Statist Soc 6, Kim, H-J, Fay, MP, Feuer, EJ, Midthune, DN, 2000 Permutation tests for joinpoint regression with applications to cancer rates Statist Medicine 9, Kim, H-J, Fay, MP, Yu, B, Barrett, MJ, Feuer, EJ, 2004 Comparability of segmented line regression models Biometrics 60, Kim, H-J, Yu, B, Feuer, EJ, 2006 Inference in segmented line regression: a simulation study Comput Statist Data Anal, under revision Lerman, PM, 980 Fitting segmented regression models by grid search Appl Statist 29, Plackett, RL, 960 Principles of Regression Analysis Clarendon Press, Oxford Quandt, RE, 958 The estimation of the parameters of a linear regression system obeying two separate regimes J Amer Statist Assoc 53, Quandt, RE, Ramsey, JN, 978 Estimating mixtures of normal distributions and switching regressions J Amer Statist Assoc 73, (with discussion) Slate, EH, Turnbull, BW, 2000 Statistical models for longitudinal biomarkers of disease onset Statist Medicine 9, Smith, AFM, 975 A Bayesian approach to inference about a change point in a sequence of random variables Biometrika 62, Tiwari, R, Cronin, K, Davis, W, Feuer, EJ,Yu, B, Chib, S, 2005 Bayesian model selection for join point regression with application to age-adjusted cancer rates Appl Statist 54,