Directly testing the linearity assumption or assay validation Steven Novick / Harry Yang May, 2013 Manuscript accepted March 2013 or publication in Journal o Chemometrics 1
Steven Novick Associate Director @ GlaxoSmithKline Harry Yang Senior Director o statistics @ MedImmune LLC 2
Purpose Illustrate a novel method to test or linearity in an analytical assay. 3
Analytical assay standard curve We wish to measure the concentration o an analyte (e.g., a protein) in clinical sample. Standards = known concentrations o an analyte. To estimate the concentration, we create a standard curve. 4
Assay Signal Standards with a itted standard curve log 10 Concentration 5
Assay Signal Assay Signal Clinical sample Estimated concentration log 10 Concentration 6
Assay Signal To many, the only interest lies in the linear portion o the curve. Clinical sample??? Estimated concentration log 10 Concentration 7
ICH Q2(R1) guideline http://www.ich.org/ileadmin/public_web_site/ich_products /Guidelines/Quality/Q2_R1/Step4/Q2_R1 Guideline.pd Evaluate linearity by visual inspection 8
Assay Signal log 10 Concentration 9
The EP6-A guidelines Clinical and Laboratory Standards Institute http://www.clsi.org/source/orders/ree/ep6-a.pd Compare straight-line to higher-order polynomial curve its Recommendation: Test higher-order coeicients. 10
Assay Signal log 10 Concentration 11
Notation E[ Y ij i ] = a 1 + b 1 i = g 1 ( i ) E[ Y ij i ] = a 2 + b 2 i + c 2 i 2 = g 2 ( i ) E[ Y ij i ] = a 3 + b 3 i + c 3 i 2 + d 3 i 3 = g 3 ( i ) Assume IID normally distributed errors with equal variance. i = 1, 2,, L concentrations j = 1, 2,, n i replicates 12
Orthogonal polynomials The orthogonal polynomial o degree k is o the orm k r=0 θ r r ( x), r = (k+1) constants (to be estimated) r (x) = (k+1) orthogonal polynomials x = concentration See: Robson, 1959 13
Orthogonal polynomials properties L i=1 n i r ( i )=0 L i=1 n i r ( i ) s ( i )=0 L i=1 n i r 2( i )=1 ortho-normal property See: Robson, 1959 14
OLS: orthogonal polynomials =( Y Y E[Y ]=F θ Var[Y ]=σ 2 I N, N=total sample size 11 Y 12 Y 1n1 Y L 1 Y L 2 Y L nl), F=( 0( 1 ) 0 ( L nl ) θ=f T Y and 1 ( 1 ) 1 ( L nl ) k( 1 ) k ( Ln L )), Var[ θ]=σ 2 I k+1. θ=(θ0 θ 1 θ k) 15
16 F T Y L L L L Ln Ln Ln Ln 3 2 3 1 3 2 2 2 1 2 1 2 1 1 1 0 2 0 1 0 θ [cubic] Y [linear] θ L L L L Ln Ln Ln Ln 3 2 3 1 3 2 2 2 1 2 1 2 1 1 1 0 2 0 1 0 Intercept and slope estimates are same or both!
Literature Krouwer and Schlain (1993) Assume linearity, except at last concentration Ha: max - (a 1 + b 1 max ) 0 Assumes linearity? EP6-A Ha: One or both o c 3, d 3 0 Ha: g k ( i ) g 1 ( i ) <δ Kroll et al. (2000) Composite statistic, ADL or all i = 1, 2,, L L i=1 Wrong power proile Tested without use o inerence {g k ( i ) g 1 ( i )} 2 / i Wrong power proile Diicult to choose Ha: ADL > 17
More literature Hsieh and Liu (2008) Ha: I-U tests g k ( i ) g 1 ( i ) <δ or all i = 1, 2,, L Hsieh, Hsiao, and Liu (2009) Composite statistic, Ha: SSDL < SSDL=L 1 i =1 Generalized pivotal quantity (GPQ) method L Choice o concentrations? Diicult to choose Generally << 2! {g k ( i ) g 1 ( i )} 2. 18
Our proposed hypothesis Ha: g k (x) g 1 ( x) <δ or all x [x L, x U ] Similar to I-U testing, but instead o individual concentrations, perormed across a range o interesting concentrations. Bayesian(or GPQ) methods. Linear models Test is unction o linear contrast 19
Our proposed test statistic g k (x) g 1 ( x) <δ or all x [x L, x U ] max k θr 2 r (x) <δ x [x L, x U ] p(δ, x L, x U )=Pr{ max k r=2 x [ x L, x U ] θ r r ( x) <δ data} Accept linearity i p > p 0 (e.g., p > 0.9). 20
Assay Signal log 10 Concentration 21
How to: with Jerey s prior (or GPQ) Given Y (responses) and F (orth poly design matrix). Assume a kth-degree polynomial. Let and be OLS estimates ˆ ˆ Error degrees o reedom = N-(k+1) 22
Generate two random variables Z ~ N k+1 ( 0, I ) U ~ 2 (N-k-1) Generate B o these Bayes ˆ U N Z k 1 To estimate p(, x L, x U ), count the proportion o times: max x x, x k L U 2 Bayes, r r ( x) 23
From NCCLS EP6-A, Appendix C, ex. 2 Calcium Assay example Dilution Replicate 1 Replicate 2 1 4.7 4.6 2 7.8 7.6 3 10.4 10.2 4 13 13.1 5 15.5 15.3 6 16.3 16.1 Cubic model = best it Compare cubic to linear rom Dilution =1 to Dilution = 6: = 0.9 or testing. NCCLS EP6-A: =0.2 mg/dl 24
+/- =0.9 around cubic it Dilution Mean dierence: Cubic Linear 1-0.53 2-0.13 3 0.42 4 0.74 5 0.42 6-0.93 25
Clear ailure at Dilution = 6 Hsieh and Liu I-U test p-value = 0.61 g or all i = 1, 2,, 6 g3 i 1 Linear it not adequate p(, x L, x U ) = 0.39 x g x g3 1 or all 1 x 6 Linear it not adequate i 1 6 6 2 2 Probability SSDL g g > 0.99 i1 i 1 i Linear it is equivalent to cubic it 3 26
Try again without last dilution From NCCLS EP6-A, Appendix C, ex. 2 Dilution Replicate 1 Replicate 2 1 4.7 4.6 2 7.8 7.6 3 10.4 10.2 4 13 13.1 5 15.5 15.3 6 Quadratic model = best it Compare quadratic to linear rom Dilution =1 to Dilution = 5 with = 0.9 or testing. 27
+/- =0.9 around quadratic it Dilution Mean dierence: Quad Linear 1-0.18 2 0.09 3 0.18 4 0.09 5-0.18 6 28
Linear and quadratic its equivalent Hsieh and Liu I-U test p-value < 0.01 p(, x L, x U ) > 0.99 SSDL probability > 0.99 Linear it is equivalent to quadratic it or dilutions between 1 and 5. 29
Simulation 30
Quadratic vs. Linear Simulation g 2 (x) = 10 + (1-40)x + x 2, 1 x 40 = 0 = linear: g 2 (x) = 10 + x = 0.04 = large quadratic component. Y ~ N( g 2 (x), =3 ). 10 g 2 (x) 50 6 concentrations with two replicates each Testing limit: = 6.1 31
Two sets o concentrations This case is H0/H1 border Set 1: Maximum deviation occurs at x = 1 Set 2: Maximum deviation occurs between points 32
Maximum dierence occurs at x = 1 Because max di occurs at design point, these tests are very similar Test is generally Too powerul SSDL can be tuned with knowledge o unknown curve. 33
Our method retains similar power proile Maximum dierence occurs between points Tests are generally Too powerul SSDL can be tuned with knowledge o unknown curve. 34
Inverting the test Consider the true mean response g k (x) and the reduced model g 1 (x)=a+bx. Let z k (x) = { g k (x) a }/b This is polynomial Y-value back-calculated with best-itting straight line. How close is z k (x) to x? 35
A ew hypotheses to consider z k (x) x < or all x L x x U 100%{z k (x) x}/x < log{z k (x)} log(x) < Many others! 36
We can compute conditional probability Pr{ log{z k (x)} log(x) < data } Or Find such that Pr{ log{z k (x)} log(x) < data } = 0.95 37
Pr{ log{z 3 (x)} log(x) < 0.15 data } = 0.95 or all 1 x 6 Back-calculated values are within 0.15 log10 units o true value. Back-calculated values are within 100%(10 0.15-1) = 40% o true value. 38
Pr{ log{z 2 (x)} log(x) < 0.05 data } = 0.95 or all 1 x 5 Back-calculated values are within 0.05 log10 units o true value. Back-calculated values are within 100%(10 0.05-1) = 12% o true value. 39
Extra bits When k=2 (quadratic vs. linear), the proposed test statistic is central T distributed. When k = 2, by altering the testing limits, the I-U, SSDL, and proposed test methods can be made equal. From simulations, test size or the proposed test statistic appears to be, depending on the experimental design. 40
Summary Test method extends idea o NCCLS EP6-A by computing probability that best-it curve is equivalent to a linear it. Testing perormed across a range o concentrations and not at experimental design points. 41
Reerences: Guidelines ICH (2005), Validation o Analytical Procedures: Text and Methodology Q2(R1) International Conerence on Harminisation o Technical Requirements or Registration o Pharmaceuticals or Human Use, http://www.ich.org/ileadmin/public_web_site/ich_products /Guidelines/Quality/Q2_R1/Step4/Q2_R1 Guideline.pd. Clinical Laboratory Standard Institute (2003), Evaluation o the Linearity o Quantitative Measurement Procedures: A Statistical Approach; Approved Guideline, http://www.clsi.org/source/orders/ree/ep6-a.pd. National Committee or Clinical Laboratory Standards. Evaluation o the linearity o quantitative analytical methods; proposed guideline. NCCLS Publ. EP6-P. Villanova, Pk NCCLS, 1986. 42
Reerences: Linearity tests Krouwer, J. and Schalin, B. (1993). A method to quantiy deviations rom assay linearity, Clinical Chemistry, 39(8), 1689-1693. Kroll MH, Præstgaard J, Michaliszyn E, Styer PE (2000). Evaluation o the extent o nonlinearity in reportable range studies, Arch. Pathol. Lab. Med, 124: 1331 1338. Hsieh E, Liu JP (2008). On statistical evaluation o linearity in assay validation, J. Biopharm. Stat., 18: 677 690. Hsieh, Eric, Hsiao, Chin-Fu, and Liu, Jen-pei (2009). Statistical methods or evaulation the linearity in assay validation, Journal o chemometrics, 23, 56-63. 43
Reerences: Orthogonal polynomials Narula, Sabhash (1979). Orthogonal Polynomial Regression, International Statistical Review, 47 : 1, 31-36. Robson, D.S. (1959). A simple method or constructing orthogonal polynomials when the independent variable is unequally spaced, Biometrics, 15 : 2, 187-191. 44
Reerences: Bayes, GPQ, Fiducial inerence Gelman, A., Carlin, J., Stern, H., and Rubin, D. (2004). Bayesian Data Analysis Second Edition, New York: Chapman & Hall. Hannig, Jan (2009). On Generalized Fiducial Inerence, Statistica Sinica, 19, 491-544. Weerahandi, S. (1993). Generalized Conidence Intervals, Journal o the American Statistical Association 88:899-905. Weerahandi, S. (1995). Exact Statistical Methods or Data Analysis, New York: Springer-Verlag. Weerahandi, S. (2004). Generalized Inerence in Repeated Measures: Exact Methods in MANOVA and Mixed Models, New Jersey: John Wiley & Sons. 45
Thank you! Questions? 46