Research Paper. Stochastic Differential Equations in NONMEM \ : Implementation, Application, and Comparison with Ordinary Differential Equations

Transcription

1 Pharmaceutical Research, Vol. 22, No. 8, August 25 ( # 25) DOI:.7/s Research Paper Stochastic Differential Equations in NONMEM \ : Implementation, Application, and Comparison ith Ordinary Differential Equations Christoffer W. Tornøe,,2,3,4 Rune V. Overgaard, 2 Henrik Agersø, Henrik A. Nielsen, 2 Henrik Madsen, 2 and and E. Niclas Jonsson 3 Received December, 24; accepted March 6, 25 Purpose. The obective of the present analysis as to explore the use of stochastic differential equations (SDEs) in population pharmacokinetic/pharmacodynamic (PK/PD) modeling. Methods. The intra-individual variability in nonlinear mixed-effects models based on SDEs is decomposed into to types of noise: a measurement and a system noise term. The measurement noise represents uncorrelated error due to, for example, assay error hile the system noise accounts for structural misspecifications, approximations of the dynamical model, and true random physiological fluctuations. Since the system noise accounts for model misspecifications, the SDEs provide a diagnostic tool for model appropriateness. The focus of the article is on the implementation of the Extended Kalman Filter (EKF) in NONMEM \ for parameter estimation in SDE models. Results. Various applications of SDEs in population PK/PD modeling are illustrated through a systematic model development example using clinical PK data of the gonadotropin releasing hormone (GnRH) antagonist degarelix. The dynamic noise estimates ere used to track variations in model parameters and systematically build an absorption model for subcutaneously administered degarelix. Conclusions. The EKF-based algorithm as successfully implemented in NONMEM for parameter estimation in population PK/PD models described by systems of SDEs. The example indicated that it as possible to pinpoint structural model deficiencies, and that valuable information may be obtained by tracking unexplained variations in parameters. KEY WORDS: degarelix; Extended Kalman Filter; NONMEM; parameter tracking; population PK/PD modeling; stochastic differential equations, systematic model development. INTRODUCTION Traditionally in population pharmacokinetic/pharmacodynamic (PK/PD) modeling, the variability is decomposed Experimental Medicine, Ferring Pharmaceuticals A/S, DK-23, Copenhagen S, Denmark. 2 Informatics and Mathematical Modelling, Technical University of Denmark, DK-28, Lyngby, Denmark. 3 Department of Pharmaceutical Biosciences, Division of Pharmacokinetics and Drug Therapy, Uppsala University, Box 59, 75 24, Uppsala, Seden. 4 To hom correspondence should be addressed. ( christoffer. tornoe@ferring.com) ABBREVIATIONS: EKF, Extended Kalman Filter; FOCE, firstorder conditional estimation; GnRH, gonadotropin releasing hormone; LLOQ, loer limit of quantification; ODE, ordinary differential equation; PK/PD, pharmacokinetics/pharmacodynamics; RSE, relative standard error; SC, subcutaneous; SDE, stochastic differential equation; Variables: e, measurement error;, residual error;, inter-individual variability; K, Kalman gain; W, interindividual covariance; P, state covariance;, individual parameter; R, output prediction covariance; ~, measurement error covariance; s, diffusion term; t, time; q, population mean parameter;, Wiener process; x, state variable; y, measurement. into inter-individual and intra-individual variability. The intra-individual (residual) variability accounts not only for the various environmental errors such as those associated ith assay, dosing, and sampling errors, but also for errors associated ith structural model misspecifications and approximations in the differential equations and variations in model parameters over time. Since most of these errors cannot be considered real sources of uncorrelated measurement noise, the sources of the intra-individual error should preferably be separated (,2). Erroneous dosing, sampling history, as ell as structural model misspecifications may introduce time-dependent or serial correlated residual errors. Karlsson et al. (3) introduce three types of residual error models to population PK/PD data analysis to account for more complicated residual error structures. The study concluded that the inter-individual variability is overestimated hile the residual variability is underestimated hen serial correlations are present in the residuals but not accounted for. Neither accuracy nor precision of the structural parameters as improved by accounting for serial correlations. Only the variance components ere more accurately estimated. Furthermore, timedependent residual errors due to inaccurate recording of sampling times are very difficult to handle hen dosing /5/8-247/ # 25 Springer Science + Business Media, Inc.

2 248 Tornøe et al. patterns are more complex than single-dose or steady-state data. Another source of variability is unaccounted variations in model parameters. Part of this variability can sometimes be linked to surrogate variables (e.g., demographic covariates) but the variability is most often not predictable oing to the governing processes not being fully understood or too complex to model deterministically. More sophisticated methods are therefore required to handle models ith structural model misspecifications. This motivates the folloing questions: () Ho should one formulate a population PK/PD model to account for structural model deficiencies? (2) Given sparsely sampled, incomplete, and noise-corrupted PK/PD data, ho should one optimally estimate the parameters in such models? Grey-box PK/PD modeling provides an attractive approach by combining prior physiological knoledge about the modeled system ith information from experimental data. Stochastic state-space or grey-box models consist of stochastic differential equations (SDEs) describing the dynamics of the system in continuous time and a set of discrete time measurement equations. The advantage of such models is that they allo for decomposition of the noise affecting the system into a system noise term representing unknon or incorrectly specified dynamics and a measurement noise term accounting for uncorrelated errors such as assay error (4Y6). The approach considered in this article aims to pursue the above mentioned questions by implementing SDEs in a nonlinear mixed-effects modeling setup, as previously investigated by Overgaard et al. (7). The intra-individual error is thereby decomposed into to types of noise: uncorrelated measurement noise arising from, for example, assay error, and system noise representing structural model deficiencies. This setup makes identification of structural model misspecification feasible by quantifying the model uncertainty and provides the basis for systematic model development (5,8). The aims of this analysis are () to illustrate ho SDEs can be implemented in NONMEM (9) (the most commonly used program for population PK/PD modeling), (2) demonstrate its application to systematic model development using a clinical PK data example, and finally (3) compare the results to common practice using ordinary differential equations (ODEs). Theory Nonlinear mixed-effects models based on SDEs extend the first-stage model of the hierarchical structure by decomposing the intra-individual variability into measurement and system noise (7). In the folloing, the population likelihood function based on SDEs is formulated along ith a proposed algorithm for parameter estimation in SDE models. To ease the notation, bold symbols refer to vector or matrix representation. Readers not interested in the theory behind SDEs and state filtering methods can skip these sections ithout loss of understanding of the remainder of the article. Nonlinear Mixed-Effects Models Based on SDEs The first-stage model ith SDEs ritten in state-space form consists of a set of continuous time system equations in Eq. () and discrete time measurement equations in Eq. (2) (see [4,5,7]), that is, dx it ¼ gx ð it ; d it ; i Þdt þ d it ; it is 2 Nð; t siþ ðþ y i ¼ f x i ; i Þþe i ; e i 2 here x is the state vector, y is the observation vector, d is the input vector, is the individual parameter vector, t is the time variable, s d is the system noise, I is the identity matrix, and e is the measurement error ith mean zero and The subscript i in Eq. (2) refers to the measurement on individual i at time t i. The structural model function g(i) is called the drift term, the matrix s is a scaling diffusion term, hile is a standard Wiener process also referred to as Bronian motion (). The standard Wiener process is a nonstationary stochastic process ith mutually independent (orthogonal) increments ( t Y s ) hich are Gaussian distributed ith mean zero and variance t Y si. If the diffusion term s is zero, the SDE in Eq. () reduces to an ODE. The usual physiological interpretation of the parameters is thereby preserved in the SDE model formulation. The second-stage model describing the inter-individual variability (IIV) is included in the same ay as for ODEs. The individual parameters i are modeled as i ¼ hð; Z i Þ exp ð i Þ; i 2 Nð; 4Þ here h(i) denotes the structural type parameter model hich is a function of the fixed-effects parameters q, covariates Z i, and random-effects parameters h i influencing i. The random-effects h i are assumed independent and multivariate normal distributed ith zero mean and covariance matrix W. The three levels of random-effects it, e i, and h i are assumed mutually independent for all i, t, and. The population likelihood function ith SDEs can be ritten as ; 4Þ / YN ¼ YN i ¼ Z i ¼ Z exp ðl i Þd i p ðy ini i ; d i Þp 2 ð i 4Þd i here p (I) and p 2 (I) are the distributions associated ith the first- and second-stage models, respectively, Y ini ¼ ½y i ;...; y ini Š represents all n i observations of the ith individual, N is the total number of individuals, hile l i is the individual log-likelihood function (7). Using the first-order conditional estimation (FOCE) method hich approximates the likelihood using an expansion around the conditional estimates of the random effects ˆ i, the population likelihood function ith SDEs can be ritten as ; 4Þ YN i¼ $l =2 i exp ðl i Þˆ i ð2þ ð3þ ð4þ ð5þ

3 Stochastic Differential Equations in NONMEM \ here Dl i is the Hessian of l i. Further information about the FOCE method as implemented in NONMEM can be found in NONMEM Users GuideVpart VII (9). Assuming a Gaussian conditional density for the firststage distribution density, the individual a posteriori loglikelihood function l i and its Hessian Dl i are given by l i ¼ 2 2 $l i ¼ Xni ¼ X ni ¼ T i R ið Þ i þ log 2pR ið Þ 2 T i 4 i log 24 ð6þ r T i R ið Þ r i 4 ð7þ respectively, here r is the gradient of the one-step ˆ i prediction error i ith respect to the random effects h. The one-step output prediction ŷ ið is the optimal prediction of the th measurement before that measurement is taken hile R i( ) is the expected covariance for that prediction. As indicated in (7), the Gaussian density is completely described by h i ð Þ ¼ E y i Y ið Þ ; ŷ i h i ð Þ ¼ V y i Y ið Þ ; R i here ŷ ið Þ and R i( ) are the conditional mean and covariance, respectively, of y i conditioned on all previous measurements up to time t for individual i denoted by Y ið ¼ y i ;...; y ið Þ. The subscript notation i( ) in Eqs. (8) and (9) refers to the th prediction based on all Y previous measurements for individual i. The one-step prediction error are calculated by i ¼ y i ŷ ið ; i 2 N ; R ið Þ ð8þ ð9þ ðþ The one-step prediction ŷ ið Þ and the associated covariance R i( ) are calculated using the Extended Kalman Filter (EKF), hich is described in the folloing section. Extended Kalman Filter The Extended Kalman Filter (EKF) (,2) provides an efficient recursive algorithm to calculate the conditional mean and covariance for the assumed Gaussian conditional densities needed to evaluate the likelihood function in Eq. (5) (,3). The EKF equations can be grouped into to parts: prediction and update equations. The prediction equations predict the state and output variables one step ahead (i.e., until the next measurement) hile the update equations update the state predictions ith the nely obtained measurement. The one-step state prediction equations of the EKF, hich are the optimal (minimum variance) prediction of the mean and covariance, can be calculated by solving the state and state covariance prediction equations from measurement time t until t, that is, dˆx it ð Þ dt ¼ g ˆx it ð Þ ; d i ; i Þ; t 2 t ; t d ˆP it ð Þ ¼ A it P it ð Þ þ P it ð Þ A T it þ T dt ; t 2 t ; t ðþ ð2þ ith initial conditions ˆx ið Þ¼ x i ð3þ P ið Þ ¼ P i ¼ Z t2 t e Aits T e A it s T ds ð4þ here t and t 2 are the sampling times of the to first measurements hile x i are the initial states hich can be prespecified or estimated along ith the other model parameters. The integral in Eq. (4) specifying the initial state covariance P i is taken as the integral of the Wiener process and the dynamics of the system over the time difference beteen the first to measurements. This initialization scheme has proven to be successful in other softare implementations (6). The EKF is an exact solution to the state filtering problem for linear systems, hile it is a first-order approximative filter for nonlinear systems. Hence, the A it matrix is calculated for nonlinear systems by linearizing the state equations in Eq. () using a local first-order Taylor expansion of g(i) about the current state at each time instant t, that is, A ð5þ x ¼ ˆxiðt Þ Next, the EKF one-step output prediction equations are calculated by ð Þ ¼ f i ; ˆx ið ŷ i R i Þ Þ ð6þ ð Þ ¼ C i P ið C T i ð7þ here C i is obtained using a local first-order Taylor expansion of the measurement equation in Eq. (2), that is, C ð8þ x ¼ ˆx i ð Þ The one-step output prediction covariance R i( ) is thus the sum of the state covariance associated ith the observed states (C i P i( ) C T i ) and the covariance of the actual measurement (@). In case of no system noise (s = ), the one-step output prediction ŷ ið and covariance R i( ) ill reduce to the ODE predictions ŷy i and residual typically used in the NONMEM likelihood function. Finally, the one-step state and state covariance predictions are updated by conditioning on the th measurement using the EKF state update equations, that is, ˆx ið P ið Þ ¼ ˆx ið Þ þ K i yi ŷ ið Þ ð9þ Þ ¼ P ið Þ K i R ið K T i ð2þ 249

4 25 Tornøe et al. Table I. Implementation of Recursive EKF Algorithm in NONMEM VI Control stream implementation EKF algorithm Equation Description ˆx i 35Y37 ð Þ ¼ x i (3) Specify initial state conditions Data file P ið Þ ¼ R t2 it s t T ea it Tds s (4) Calculate initial state covariance for i ¼ tondo For each of the N subects for ¼ ton i do For each of the ith subect s n i 4Y6 measurements dˆx it ð Þ ¼ g ˆx dt it ð ; d i ; i Þ () Compute one-step state prediction at time t dp it ð Þ 8Y3 ¼ A it P it ð Þ þ p dt it ð Þ A T it þ T (2) Calculate one-step state covariance prediction 2 ŷ ið ¼ f i ; ˆx ið (6) Compute one-step output prediction at time ŷy ið 72 and 2 R ið ¼ C i P ið Þ C T i þ X (7) Calculate one-step output covariance prediction variance 73Y75 K i ¼ P ið C T i R ið (2) Compute the Kalman Gain 78Y8 ˆx ið Þ ¼ ˆx ið Þ þ K i yi ŷ ið Þ (9) Update state prediction ith the th measurement 82Y87 end for end for P i ð Þ ¼ P ið K i R ið K T i (2) Update state covariance here ˆx ið Þ is the updated state estimate, P i( ) is the updated state covariance, and the Kalman gain K i is calculated by K i ¼ P ið C T i R i ð ð2þ The optimal state estimate at time denoted by ˆx ið Þ is equal to the best state prediction ˆx ið before the measurement is taken plus a correction term consisting of an optimal eighting value times the difference beteen the measurement y i and the one-step prediction of its value. For measurements ith a large the Kalman gain becomes small and the measurement is eighted lightly oing to the little confidence in the noisy measurement. In the limit Y V, the Kalman gain becomes zero and the infinitely noisy measurement is completely ignored in the update. When the system noise is dominant implying uncertainty in the output of the system model, the measurement is heavily eighted. In the limit hen s Y V and P Y V, the Kalman gain ill approach and the updated state ill be equal to the measurement (3). The EKF algorithm specified above and summarized in Table I is recursive by repeating the calculations of the onestep state and output prediction equations in Eqs. ()Y(8) as ell as the state update equations in Eqs. (9)Y(2) for each individual measurement. METHODS Implementation of EKF Algorithm in NONMEM The EKF algorithm for parameter estimation in SDE models is implemented in NONMEM by modifying the standard NONMEM data file and control stream as described later for a clinical PK data example. An overvie of Table II. Extract from the Example SDE Data File ID ORIG TIME AMT CMT DV EVID MDV Y.9. I I I I I I I I I I I I I I I I I I I I I 3 The data file includes the folloing records: ID (patient number), ORIG (original sampling time), TIME (ne time variable), AMT (amount), CMT (compartment), EVID (event identifier), and MDV (missing dependent variable) record.

5 Stochastic Differential Equations in NONMEM \ the necessary modifications is shon in Table I. The necessary control stream and data file modifications are made automatically using an S-PLUS script, hich can be obtained by contacting the corresponding author. Data File Modifications The necessary data file modifications are illustrated by an extract from the example SDE data file in Table II. In order for the EKF to ork, the initial states x i and the associated covariance matrix P i must be specified. The initial states x i are often zero hen dealing ith pharmacokinetics of an exogenous compound but might otherise be specified to some initial amounts or estimated along ith the other model parameters. The integral in Eq. (4) specifying the initial state covariance P i cannot be entered directly into NONMEM. This problem is handled by reinding the time variable so that the estimation starts at t start calculated by t start ¼ t ðt 2 t Þ ð22þ here t and t 2 are the sampling times of the first and second measurements, respectively, hich in the example data in Table II are equal to 7.97 and 26.3 h. The initial conditions P i are thereby calculated exactly as the integral in Eq. (4) at the time of the first measurement. If t start is negative, it is necessary to create a ne time variable since negative time data records are not accepted in NON- MEM. For the example data in Table II, the ne time variable (TIME) is therefore calculated by subtracting t start from the original sampling time variable (ORIG), that is, TIME = ORIG t start. This ne starting time is implemented in the data file by an event identification (EVID) record equal to 2 and the missing dependent variable (MDV) data item set to. The doses in Table II are specified as usual ith a dose event record (EVID = ). Finally, each observation is duplicated and presented as three records in the data file ith an EVID =, EVID = 2, and EVID = 3 record. The EVID = makes the onestep prediction as usual, the EVID = 2 record stores the onestep prediction ^x ið Þ and P i( ) from the EVID = record before the states are reset to zero and updated ith the EKF updates in the subsequent EVID = 3 record. Control Stream Modifications 25 The NONMEM SDE control stream for a one-compartment disposition model ith to absorption components is shon in Appendix. The one-step prediction equations of the EKF are specified in the $DES block of the NONMEM control stream. The kkþ ð Þ 2 one-step state covariance prediction equations in lines 8Y3 associated ith the k one-step state prediction equations in lines 4Y6 are calculated using Eq. (2). The one-step state and state covariance prediction equations (both of hich are systems of ODEs) Table III. Summary of Population Pharmacokinetic Parameter Estimates for the Initial Model ith First-Order Absorption (Initial Model), Tracking Absorption Half-Life Model (Tracking t /2,abs ), and the Final Model ith To Absorption Components (Final Model) Model Initial model Tracking t /2,abs a Final model Method OFV Parameter Units SDE -89 Estimate SDE -323 Estimate SDE Estimate RSE (%) ODE Estimate RSE (%) t /2,abs h 484 V V V V V t /2,abs h V 245 V V V V t /2,fast h V V t /2,slo h V V CL/F h V/F l Fr V V V F rel V.5 V t=2; abs % 25.9 V V V V V t=2; fast % V V t=2; slo % V V CL % V 3.9 V V V V V % V 54. V V V V Fr % V V Frel % 29.8 V s prop % abs s s central 2g ffiffi h p V V V V p 2g ffiffi.b h. b. b V V V t =2;abs s fast pffiffi h V.346 V V V V p 2g ffiffi h V V. b V V V s slo p 2g ffiffi h V V V V a Single<dose data from group 3 only. b Fixed parameter.

6 252 Tornøe et al. are progressed until the time of the next measurement. The one-step output prediction equation in line 2, hich is a function of the central compartment and its volume, is defined in the $ERROR block. The standard deviation of the one-step prediction error is calculated in line 2 as the square root of R i( ) in Eq. (7). Updating the state predictions and covariances is slightly tricky in NONMEM, because this involves first a reset of the compartment and subsequently an update to the ne value, hich is all performed at EVID = 3. Lines 34Y68 store the one-step state and state covariance prediction values, and ensure that these values are not overritten during system reset. Calculation of the measurement updated predictions is based on the stored state values, as performed in lines 7Y87. First, the one-step output prediction variance R i() is calculated. Next, the Kalman gain K i is calculated in lines 74Y76, and the state and state covariance updates are calculated in lines 78Y87. Finally the execution of the update is performed in lines 9Y99. Example Data Clinical PK data of GnRH antagonist degarelix is used to illustrate the application of SDEs in population PK/PD modeling. (LC-MS/MS) method according to current guidelines for bioanalytical samples (4) ith a loer limit of quantification (LLOQ) of.5 ng/ml. Data Analysis The data analysis as performed using NONMEM version VI beta ith subroutines ADVAN6 and ADVAN8 and relative tolerance set to six on a Pentium 4-M 2. GHz computer ith 52 MB RAM running Windos XP. Three significant digits ere requested in the parameter estimates. All models ere estimated using the first-order conditional estimation (FOCE) method. The individual parameter vector i as assumed to be distributed around the population mean parameter vector q according to the exponential IIV model in Eq. (3), that is, i ¼ exp ð i Þ; i 2 Nð; 4Þ ð23þ here the random-effects h i influencing i are assumed multivariate normal distributed ith zero mean and covariance matrix W. The inter-individual covariance matrix W as specified as a diagonal matrix ith 2 k in the k th diagonal element. Correlations beteen all pairs of h i ere investigated in the pre-analysis but ere found to be nonsignificant. Study Design The study as designed as a 6-month, multicenter, openlabeled, equally randomized, parallel group trial investigating the efficacy and safety of three dose regimens of degarelix in prostate cancer patients. The study as performed in accordance ith the Declaration of Helsinki and according to Good Clinical Practice (GCP). The appropriate independent ethics committee approved the protocol prior to study initiation. Written informed consent as obtained from all patients prior to participation in the study. The folloing dose regimens ere included in the study: Group received 8 mg at days and 3, and 4 mg at day 28 and every 4 eeks thereafter (8/8/4). Group 2 received 4 mg at days, 3, 28, and every 4 eeks thereafter (4/4/4). Group 3 received 8 mg at day, 2 mg at day 28, and every 4 eeks thereafter (8/Y/2). The doses ere administered as subcutaneous (SC) inections of the folloing volume and concentrations: 8 mg as to SC inections of 2 ml ith 2 mg/ml. 4 mg as one SC inection of 2 ml ith 2 mg/ml. 2 mg as one SC inection of 2 ml ith mg/ml. One hundred and tenty-nine patients ere randomized to receive at least one dose of degarelix, ith 43 patients in treatment group, 46 patients in treatment group 2, and 4 patients in treatment group 3. Blood samples ere dran before dosing at each visit except for visit. At visit 2, an additional blood sample as taken 8 h after dosing. Degarelix plasma concentrations ere measured using a validated liquid chromatography ith tandem mass spectrometric detection RESULTS To illustrate the application of SDEs in population PK/ PD modeling, clinical PK degarelix data are used in the folloing systematic model development scheme motivated by (8). First, the structural model misspecifications of the initial PK model are pinpointed using the diffusion term estimates. Next, the model is expanded to track unexplained time variations in the entity of the model believed to be deficient, hich leads to an extension of the SDE model. Finally, the parameter estimates of the final SDE model are compared ith the corresponding ordinary differential equation (ODE) model. Initial Model A one-compartment disposition model ith first-order absorption and elimination as selected as the initial PK model describing the SC administration of degarelix. From previous studies, it is knon that the SC release of degarelix does not follo simple first-order absorption kinetics (5). But in order to demonstrate the use of SDEs for systematic model development, it as chosen as the initial PK model. The system of SDEs governing the initial PK model can be ritten in matrix notation together ith the observation equation as d A ¼ A 2! k a A þ F rel Dt ðþ k a A CL V A dt þ abs 2 central d t log c ¼ log A 2 V þ e ð24þ

7 Stochastic Differential Equations in NONMEM \ here A and A 2 are the state variables for the amount of drug in the SC absorption and central compartment, respectively, hile c is the observed plasma degarelix concentration. The diffusion term s as specified as a diagonal matrix to pinpoint possible model misspecifications. The presence of significant parameters in the diffusion term is an indication that the corresponding drift term may be incorrectly specified (8). The system noise s d t is additive hile s prop is the coefficient of variation for the measurement error. The model in Eq. (24) as parameterized in terms of CL and V symbolizing the clearance and volume of the central compartment, respectively, hile the first-order absorption rate constant k a as estimated as the absorption half-life t =2;abs ¼ log 2 k a and D(t) is the input from the SC degarelix doses. The bioavailability of SC inected degarelix doses has previously been reported to be dependent on the concentration of the inected suspension (5,6). The relative bioavailability F rel of the mg/ml doses as fixed to hile the bioavailability of the 2 mg/ml doses relative to the mg/ml doses as estimated as the parameter F 2 rel. IIV as estimated for the parameters t /2,abs and F rel using the exponential model in Eq. (23). The parameter estimates of the initial SDE model are shon in Table III. The diffusion parameters can be used to pinpoint possible model misspecifications and provide information on ho to reformulate the model. The diffusion abs parameter associated ith the absorption compartment s as estimated to 82.2 p g ffiffiffi ith an asymptotic relative standard hr error (RSE) estimate of 4.% hile that of the central compartment s central as insignificant and fixed to zero in the final estimation. This may indicate that the disposition of degarelix is sufficiently explained by a one-compartment disposition model hile the first-order absorption model inadequately describes the SC absorption profile of degarelix. Tracking Absorption Half-Life Model Since the diffusion term on the absorption compartment as significant in the initial PK model, the point of interest for the folloing model expansion should be the absorption model. From previous studies, it is knon that SC administration of degarelix results in the formation of a gel-like in situ SC depot from hich the drug is released into the systemic circulation (5). The depot undergoes a maturation stage here the density of the gel increases and the release rate decreases. The absorption half-life is therefore suspected to change over time. Instead of umping to conclusions by imposing a structural model for this behavior, e ant to track the absorption half-life t /2,abs and thereby confirm/reect hether it is constant throughout the study. This is done by expanding the model ith a state equation for the absorption half-life that fluctuate randomly like a Wiener process. For a given individual, the EKF gives the filtered estimate of the evolution of t /2,abs. The drift term in the state equation for the absorption half-life is set to zero, thereby assuming it is driven only by the random variation t =2;abs d t. If t =2;abs turns out to be significant, it is an indication that the absorption half-life is not constant since the variations in t /2,abs are explained by the diffusion term. The model for tracking the variations in t /2,abs can thereby be ritten as log 2 A þ Dt ðþ A exp log t =2;abs db A 2 A ¼ log 2 A CL exp log t =2;abs V A dt 2 B C log t A abs þb central A d t log c ¼ log A 2 V þ e t =2;abs 253 ð25þ here t /2,abs is the state variable for the absorption half-life ith initial value t /2,abs. To constrain t /2,abs to be non- Fig.. Tracking absorption half-life t /2,abs for patients in group 3 (8/-/2) using single-dose data only. Each line represents the tracked absorption half-life for one individual.

8 254 Tornøe et al. negative, multiplicative (i.e., state dependent) system noise is needed. By log-transforming the state equation for the absorption half-life, the multiplicative system noise on t /2,abs as transformed into additive system noise in Eq. (25) hich can be estimated using the EKF based algorithm (7). IIV as estimated for the parameters CL and V using the exponential model in Eq. (23). For simplicity reasons in the implementation of this model, only single-dose data from group 3 ere included in this estimation since a ne depot is formed at each inection hich ould make it necessary to track the different absorption half-lives from each depot. The estimated parameters for the tracking absorption half-life model are shon in Table III. The diffusion parameter on the absorption compartment s abs as no reduced to.99 p 2g ffiffiffi hile t =2;abs hr as estimated to.346 p ffiffiffi ith an asymptotic RSE estimate hr of 8.99%. This confirms the suspicion that t /2,abs is not constant throughout the study since t =2;abs is significantly different from zero. Tracking of the absorption half-life is shon in Fig. here the updated EKF estimates of t /2,abs using Eq. (9) are plotted as a function of time. The initial absorption half-life estimate is 245 h and remains constant until approximately 3 days after drug administration for most of the patients. From 3 to 4 days the absorption half-life increases and seems to reach a ne constant or slightly decreasing level thereafter. A reasonable approximation of the pattern seen in Fig. ould be a model ith to absorption components, that is, an initial fast absorption phase folloed by a prolonged slo release phase. This interpretation corresponds ell ith the clinical observation that SC administration of degarelix results in the formation of a depot out of hich the drug diffuses. Final Model The final step in this illustrative example is to reformulate the initial PK model ith the information obtained by tracking the absorption half-life. The final PK model ith to first-order absorption components (see Fig. 2) as modeled by the folloing system of SDEs and observation equation d A A 2 A 3 C A ¼ k a; A þ Fr F rel Dt ðþ k a;2 A 2 þ ð FrÞF rel Dt ðþ k a; A þ k a;2 A 2 CL V A 3 þ fast slo central log c ¼ log A 3 V þ e dt C A C A d t (26) here A, A 2, and A 3 are the state variables for the amount of drug in the slo SC absorption, fast SC absorption, and Fig. 2. Schematic illustration of the final PK model for SC inected GnRH antagonist degarelix. The model parameters are explained in the text. central compartment, respectively. The fraction of the SC dose being absorbed via the fast and slo absorption route ere estimated as Fr and ( Y Fr), respectively. The first-order absorption rate constants k a, associated ith the fast absorption components as estimated as the absorption half-life t =2; fast ¼ log 2. To ensure that t =2; slo ¼ log 2 k a;2 as constrained to be larger than the t /2,fast even after taking IIV into account, the folloing parametrization as chosen. k a; t =2; slo; i ¼ t =2; slo fast exp t=2; slo fast; i þ t =2; fast; i ð27þ here t /2, sloyfast is the typical individual s difference beteen t /2, fast and t /2, slo. IIV as estimated for the parameters t /2, fast, t /2, slo, and F rel using the exponential model in Eq. (23). The parameter Fr as constrained to be beteen and by logit-transformation, that is, Fr ¼ log Fr and the individual parameter Fr i as calculated by Fr i ¼ exp ð þ Fr i Þ þ exp ð þ Fri Þ ð28þ ð29þ The SDE control stream for the final PK model is shon in Appendix A. The run times for the estimation of model parameters in Eq. (26) using SDEs as about three times that of the corresponding ODE model. Since the one-step output prediction variance R i( ) in the SDE model depends on the individual h s [see Eq. (7)], the FOCE method ith interaction (FOCE-INTER) as used. For the corresponding ODE model (i.e., s = ), the FOCE method as applied ithout the INTER option since the intraindividual variance is homoscedastic on the log-scale. The population PK parameter estimates from the final PK model ith to absorption components using ODEs and SDEs are summarized in Table III. The diffusion parameters fast central s and s for the fast absorption and central com-

9 Stochastic Differential Equations in NONMEM \ 255 Fig. 3. One-step individual SDE predictions, updates, and prediction interval plotted together ith the individual p ODE predictions for an illustrative patient. The SDE prediction interval is calculated by ŷ ffiffiffiffiffiffiffiffiffiffiffi R. partment, respectively, ere fixed to zero in the final run since they ere found not to be significantly different from zero. The only diffusion parameter that as estimated as therefore the diffusion parameter on the slo absorption compartment. The estimated standard deviation of ˆ slo ¼ 244 p 2g ffiffiffi for the additive system noise indicates that there hr still is a model misspecification in the slo absorption compartment. Hoever, it is relatively small compared to the amount of drug in the slo absorption compartment hich ranges from 2.5 to 8. mg throughout the study. Other more complex absorption models such as diffusion out of a spherical SC depot have previously been suggested to describe the PK profile of SC administered degarelix (5). For this study, the final PK model ith to absorption components representing an initial fast release folloed by a prolonged slo release as considered acceptable at describing the observed data. Since the SDE and ODE models are nested, the significance of including the diffusion parameter s slo can be tested using the likelihood ratio test (LRT) ith DOFV = 3236 (3387) = 5 > c 2 ().95 = 3.84 hich is highly significant ith a p-value <.. When comparing the SDE and ODE parameter estimates, the parameters hich seem affected the most by including system noise on the slo absorption compartment are t =2;slo and s prop. The interindividual variability of t /2,slo as reduced from a coefficient of variation (CV) of 3.5% to 24.7% hile the CV of the proportional measurement error as deflated from 7.6% to 4.5% hen using SDEs instead of ODEs. The RSE estimates ere more or less the same for the SDE and ODE models. The one-step individual SDE predictions, updates, and prediction interval are shon together ith the individual ODE predictions for an illustrative patient from the study in Fig. 3. The observed discrepancy beteen the SDE and ODE predictions are due to the SDE predictions being conditioned on all previous observations and therefore updated at each sampling time (visualized by the vertical lines in the SDE predictions). The ODE predictions are considerably loer than the observed plasma concentrations beteen and 3 months and above thereafter, hich falsifies the statistical assumption of uncorrelated residuals. All except one of the onestep SDE predictions are ithin the SDE prediction interval Fig. 4. Observed and SDE predicted plasma degarelix concentration-time profiles plotted on a semilogarithmic scale ith each line representing data from one patient.

10 256 Tornøe et al. Fig. 5. Observed vs. individual (left) and population SDE predicted (right) plasma degarelix concentrations plotted on a double-logarithmic scale. p in Fig. 3 calculated by ŷ ffiffiffiffiffiffiffiffiffiffiffi R using Eqs. (6) and (7). The only SDE prediction outside the prediction interval is around 4 months here the observed plasma concentration is ng/ml hile the SDE prediction is close to 3 ng/ml, making it hard to falsify this model based on correlation structure or distribution of residuals. The subsequent SDE predictions are again all ithin the SDE prediction interval. The ODE predictions sho systematic deviations from the observed data and exhibit a high degree of autocorrelation. The observed degarelix concentrationytime profiles for all 29 patients in the study are displayed in Fig. 4 together ith the one-step individual predictions and the population predictions from the SDE model. The final PK model seems to describe the observed data ell ith good agreement beteen predicted and observed plasma concentrations as shon in Fig. 5. DISCUSSION The variability in population PK/PD modeling is traditionally divided into inter- and intra-individual variability. This article suggests to further decompose the intra-individual variability into measurement and dynamic noise by considering SDEs instead of ODEs in nonlinear mixedeffects models. The system noise represents structural model misspecifications or true random fluctuations ithin the system. By quantifying the model uncertainty in the system noise, it is possible to identify hich parts of the model are misspecified, and the estimated system noise can be used as a tool for systematic model development. This ould not be possible using similar approaches such as an AR() model (3) since it acts only on the measurement equation and not the dynamics of the system. In the present analysis, the recursive EKF algorithm as implemented in NONMEM version VI beta for parameter estimation in SDE models. The EKF corrects for structural model deficiencies by updating the system ith the individual measurements at each sampling time, unlike the ODE approach hich progresses the system ithout including this additional information. The algorithm orks only ith NONMEM version VI beta because it is necessary to have access to the state estimates in $PK hich is not possible ith NONMEM version V (see lines 49Y57 of the SDE control stream example in Appendix). The application of SDEs for systematic model development as illustrated using a clinical PK data example. Starting from a one-compartment disposition model ith first-order absorption, a PK model for SC inected degarelix as systematically developed by pinpointing model deficiencies in the absorption model. By tracking variations in the absorption half-life, it could be shon that the absorption profile could be approximated by a model ith to absorption components describing the initial fast absorption phase folloed by a prolonged slo release phase. The iterative frameork for systematic model development used in this analysis is summarized belo (8). & Formulate an initial SDE model ith a diagonal diffusion term. & Identify possible model misspecifications by examining the significant diffusion term estimates. & Extend the model ith state equations for the pinpointed model deficiencies. & Track unexplained variations using the updated EKF state estimates. & Use the information obtained by tracking variations to reformulate the model. & Accept/reect model based on the diffusion term estimates of the reformulated model. In a recent publication by Bayard et al. (7), an interacting multiple model (IMM) algorithm for tracking changing PK parameters as presented. The IMM and EKF algorithms are practically identical. The IMM algorithm, hoever, cannot be used for quantifying model uncertainty oing to the multiple model formulation. The introduction of system noise in the differential equations gave a more satisfactory description of the observed clinical PK data compared to the ODE model in hich the residuals ere correlated. The SDE and ODE model population parameter estimates ere nearly identical, hoever, since the system noise as relatively small in the final PK model. For other systems in hich the degree of model misspecification is more dominant (e.g., nonlinear pharmacodynamic models of complicated physiological systems), the discrepancies beteen the SDE and ODE

11 Stochastic Differential Equations in NONMEM \ parameter estimates are expected to be greater since the intra-individual variance decomposition into measurement and system noise resembles the true physiological variations in the modeled system more correctly. In conclusion, the EKF-based algorithm as successfully implemented in NONMEM for parameter estimation in population PK/PD models described by systems of SDEs. A clinical PK data example as used to illustrate the application of SDEs for systematic model development by quantifying model uncertainty and tracking unexplained variations. ACKNOWLEDGMENTS The authors ish to acknoledge the help of Professor Stuart Beal (UCSF), ho instructed us on ho to implement the EKF algorithm in NONMEM. This ork as financially supported by Ferring Pharmaceuticals A/S and Center for Information Technology, Denmark. REFERENCES 257. R. Jelliffe, A. Schumitzky, and M. Van Guilder. Population pharmacokinetics/pharmacodynamics modeling: parametric and nonparametric methods. Ther. Drug Monit. 22(7Y8):354Y365 (2). 2. J. K. Lindsey, B. Jones, and P. Jarvis. Some statistical issues in modelling pharmacokinetic data. Stat. Med. 2:2775Y2783 (2). 3. M. O. Karlsson, S. L. Beal, and L. B. Sheiner. Three ne residual error models for population PK/PD analyses. J. Pharmacokinet. Biopharm. 23:65Y672 (995). 4. C. W. Tornøe, J. L. Jacobsen, and H. Madsen. Grey-box pharmacokinetic/pharmacodynamic modelling of a euglycaemic clamp study. J. Math. Biol. 48:59Y64 (24). 5. C. W. Tornøe, J. L. Jacobsen, O. Pedersen, T. Hansen, and H. Madsen. Grey-box modelling of pharmacokinetic/pharmacodynamic systems. J. Pharmacokinet. Pharmacodyn. 3:4Y47 (24). 6. N. R. Kristensen, H. Madsen, and S. B. Jørgensen. Parameter estimation in stochastic grey-box models. Automatica 4:225Y 237 (24). 7. R. V. Overgaard, E. N. Jonsson, C. W. Tornøe, and H. Madsen. Non-linear mixed-effects models ith stochastic differential equations. Implementation of an estimation algorithm. J. Pharmacokinet. Pharmacodyn. In press (25). 8. N. R. Kristensen, H. Madsen, and S. B. Jørgensen. A method for systematic improvement of stochastic grey-box models. Comput. Chem. Eng. 28:43Y449 (24). 9. S. L. Beal and L. B. Sheiner (Eds.). NONMEM \ Users Guides. NONMEM Proect Group, University of California, San Francisco, A. H. Jazinski. Stochastic Processes and Filtering Theory, Academic Press, Ne York, 97.. R. E. Kalman. A ne approach to linear filtering and prediction problems. Trans. ASME, Ser. D J. Basic Eng. 82:35Y45 (96). 2. R. E. Kalman and R. S. Bucy. Ne results in linear filtering and prediction theory. Trans. ASME, Ser. D J. Basic Eng. 83:95Y8 (96). 3. P. S. Maybeck. Stochastic Models, Estimation, and Control, Academic Press, Ne York, V. P. Shah, K. K. Midha, S. Dighe, I. J. McGilveray, J. P. Skelly, A. Yacobi, T. Layloff, C. T. Visanathan, C. E. Cook, and R. D. McDoall. Analytical methods validation: bioavailability, bioequivalence and pharmacokinetic studies. Conference report. Eur. J. Drug Metab. Pharmacokinet. 6(4):249Y255 (99). 5. C. W. Tornøe, H. Agersø, H. A. Nielsen, H. Madsen, and E. N. Jonsson. Population pharmacokinetic modeling of a subcutaneous depot for GnRH antagonist degarelix. Pharm. Res. 2:574Y 584 (24). 6. H. Agersø, W. Koechling, M. Knutsson, R. Hortkær, and M. O. Karlsson. The dosing solution influence on the pharmacokinetics of degarelix, a ne GnRH antagonist, after s.c. administration to beagle dogs. Eur. J. Pharm. Sci. 2:335Y34 (23). 7. D. S. Bayard and R. W. Jelliffe. A Bayesian approach to tracking patients having changing pharmacokinetic parameters. J. Pharmacokinet. Pharmacodyn. 3:75Y7 (24).

12 258 Tornøe et al. APPENDIX NONMEM SDE Control Stream (See Table I) $PROBLEM SDE -COMP MODEL WITH 2 ABS COMPONENTS 65 A7 = A7 $INPUT ID HOUR TIME AMT CMT DV EVID MDV A8 = A8 $DATA SDE.dta IGNORE=@ A9 = A9 $SUBROUTINE ADVAN6 TOL 6 DP ENDIF 5 $MODEL COMP = (ABS) COMP = (ABS2) COMP = (CENT) ; EKF state update equations COMP = (P) COMP = (P2) COMP = (P3) 7 IF(EVID.EQ.3) THEN COMP = (P22) COMP = (P23) COMP = (P33) ; Output cov. R_ =CP_ C^T + Sig Sig^T RVAR = A9/(A3*A3) + SIG**2 $THETA (,5.5)(,)(,3.)(,9)(,.,) ; Kalman Gain K_ = P_ C^T R_^ (,.7,)(,.2)(,)(,25)(,) K = A6/(A3*RVAR) $OMEGA K2 = A8/(A3*RVAR) $SIGMA FIX K3 = A9/(A3*RVAR) ; State update eq. x_=x_+k_ (y_y_) $PK AUP = A + K*(OBS LOG(A3/V)) 5 CL = THETA() A2UP = A2 + K2*(OBS LOG(A3/V)) V = THETA(2) 8 A3UP = A3 + K3*(OBS LOG(A3/V)) HL = THETA(3)*EXP(ETA()) ; State cov. update eq.p_=p_k_ R_ K_^T HL2 = THETA(4)*EXP(ETA(2)) + HL PUP = A4 K*RVAR*K KA = LOG(2)/HL P2UP = A5 K*RVAR*K2 2 KA2 = LOG(2)/HL2 P3UP = A6 K*RVAR*K3 TFR = THETA(5) 85 P4UP = A7 K2*RVAR*K2 RHO = LOG(TFR/(-TFR)) P5UP = A8 K2*RVAR*K3 FRAC = EXP(RHO+ETA(3))/(+EXP(RHO+ETA(3))) P6UP = A9 K3*RVAR*K3 TBIO = ENDIF 25 IF(CONC.EQ.2) TBIO = THETA(6) ; Update states BIO = TBIO*EXP(ETA(4)) 9 IF(A_FLG.EQ.) THEN SIG = THETA(7) A_() = AUP SGW = THETA(8) A_(2) = A2UP SGW2 = THETA(9) A_(3) = A3UP 3 SGW3 = THETA() A_(4) = PUP F = FRAC*BIO 95 A_(5) = P2UP F2 = (-FRAC)*BIO A_(6) = P3UP ; Initialize state and state covariance equations A_(7) = P4UP IF(NEWIND.NE.2) THEN A_(8) = P5UP 35 AHT = A_(9) = P6UP AHT2 = ENDIF AHT3 = PHT = $DES PHT2 = ; State predicton eq. dx_t/dt = g(x_t,d,phi) 4 PHT3 = DADT() = KA*A() PHT4 = 5 DADT(2) = KA2*A(2) PHT5 = DADT(3) = KA*A() + KA2*A(2) CL/V*A(3) PHT6 = ; State cov. dp_t/dt=a_t P_t+P_t A_t^T+SGW SGW^T ENDIF DADT(4) = 2*KA*A(4) + SGW*SGW 45 ; Store observations for EKF update DADT(5) = (KA+KA2)*A(5) IF(EVID.EQ.) OBS = DV DADT(6) = (KA+CL/V)*A(6) + KA*A(4) + KA2*A(5) ; Store one-step predictions for EKF update DADT(7) = 2*KA2*A(7) + SGW2*SGW2 IF (EVID.NE.3) THEN DADT(8) = (KA2+CL/V)*A(8) + KA*A(5) + KA2*A(7) 5 A = A() DADT(9) = 2*KA*A(6)+2*KA2*A(8)2*CL/V*A(9)+SGW3*SGW3 A2 = A(2) A3 = A(3) 5 $ERROR (OBS ONLY) A4 = A(4) IF (ICALL.EQ.4) THEN A5 = A(5) IF (DV.NE.) Y = LOG(DV) A6 = A(6) RETURN 55 A7 = A(7) ENDIF A8 = A(8) 2 IPRED = LOG(A(3)/V) A9 = A(9) W = SQRT(A(9)/(A(3)*A(3)) + SIG**2) ELSE IRES = DV Y IPRED A = A IWRES = IRES/W 6 A2 = A2 Y = IPRED + W*EPS() A3 = A3 25 $SIM () A4 = A4 $EST METH= INTER A5 = A5 $COV A6 = A6