Unified Approach For Performing An Analytical Methods Comparability Study IVT s LAB WEEK Presenter: Peter M. Saama, Ph.D. Bayer HealthCare LLC

Transcription

1 Unified Approach For Performing An Analytical Methods Comparability Study IVT s LAB WEEK 2015 Presenter: Peter M. Saama, Ph.D. Bayer HealthCare LLC

2 Agenda/ Content Page 1 Analytical Methods for Performing a Comparability Study Session Summary: Overview: Regulatory Requirements and Guidance Motivation: Comparability Sources of Variability in Analytical Methods Statistical Approaches for Comparability Case Studies

3 Regulatory Requirements and Guidance (1 of 3) FDA Guidance, Request for Quality Metrics published on 27th July Manufacturer s should: Understand the sources of variation; Detect the presence and degree of variation; Understand the impact of variation on the process and ultimately on product attributes; Control the variation in a manner commensurate with the risk it represents to the process and product.

4 Regulatory Requirements and Guidance (2 of 3) FDA CFR 601.2(a) data derived from nonclinical laboratory and clinical studies demonstrate that the manufactured product meets prescribed standards for safety, purity and potency (c) demonstration that the modification will provide assurances of the safety, purity, potency and effectiveness of the biological product equal to or greater than the assurance provided by the method or process specified in the general standards or additional standards for biological products

5 Regulatory Requirements and Guidance (3 of 3) FDA Guidance Demonstration of Comparability of Human Biological Products, including Therapeutic Biotechnology-derived Products (July 6, 2005) Comparability Protocols-Protein Drug Products and Biological Products Chemistry, Manufacturing, and Controls Information (Sept 2003 Draft) ICHQ5E Comparability of Biotechnological/Biological Products Subject to Changes in Their Manufacturing Process (Nov 2004)

6 Motivation: Comparability (1 of 3) Changes in analytical methods occur: To ensure that they remain valid over time; Take advantage of improved analytical technology; To monitor new related substances as a result of changes in synthetic or formulation processes; Improve analytical efficiency To account for changes in the manufacturing process, e.g. between sending and receiving site (for a product)

7 Motivation: Comparability (2 of 3) USP 1033(2) indicates the preference for equivalence testing over significance testing. Equivalency is recommended when: Trending is important; In the late-development stage (Phase III) or postregistration; There is a potential impact on results caused by a method change; Following a method transfer

8 Motivation: Comparability (3 of 3) Equivalency is less important when: Trending is unimportant; There is no expectation of equivalent results (e.g., a change to the formulation or test method that causes an intentionally different performance and/or analytical result); Method changes do not affect results; In an early development stage (Phase I or II), when one is making deliberate process changes and optimizing methods.

9 Sources of Variability in Analytical Methods (1 of 5) Biomolecular Assay Type Colorimetric Enzymatic Potential Batch to Batch Variability Unique buffer components Chromogenic reagent Commercial kit active components Unique buffer components Substrates Enzymes Cofactors Detection reagents Commercial kit active components

10 Sources of Variability in Analytical Methods (2 of 5) Biomolecular Assay Type Chromatographic Electrophoretic Potential Batch to Batch Variability Unique buffer components Labile mobile phase solvents Chromatography column resin Derivatization or conjugation reagents Unique electrode buffers Gel matrix reagents Sample treatment reagents Staining reagents Commercial kit components

11 Sources of Variability in Analytical Methods (3 of 5) Biomolecular Assay Type Immunological Potential Batch to Batch Variability Primary antibodies Secondary antibodies Conjugated antibodies Blocking reagents Detection reagents Commercial kit active components Plastic cuvettes or micro titer plates

12 Sources of Variability in Analytical Methods (4 of 5) Biomolecular Assay Type Ligand Binding Potential Batch to Batch Variability Unique buffer components Target receptor Target ligand Detection reagents Commercial kit active components Plastic cuvettes or micro titer plates

13 Sources of Variability in Analytical Methods (5 of 5) Biomolecular Assay Type Cell Based Bioassay Potential Batch to Batch Variability Cell seed stock (homogeneity and viability) Cell culture (passage number and density) Media components Growth factors Antimicrobial agents Harvest reagents (e.g. trypsin) Cell reactants (e.g. induction compounds) Plastic flasks or plates

14 Sources of Variability in Analytical Methods (5 of 5) Biomolecular Assay Type Cell Based Bioassay Potential Batch to Batch Variability Cell seed stock (homogeneity and viability) Cell culture (passage number and density) Media components Growth factors Antimicrobial agents Harvest reagents (e.g. trypsin) Cell reactants (e.g. induction compounds) Plastic flasks or plates

15 Statistical Approaches: Student s t-test (1 of 3) Uses: Compare results from a new analytical method with those from another test method Small samples (n < 30) Tests the hypothesis H 0 : μ x = μ y H a : μ x μ y where H 0 and H a are the null and alternative hypotheses; μ x and μ y are the population means. Is it likely that no difference exists between two sets of analytical test results?

16 Statistical Approaches: Student s t-test (2 of 3) The T statistic is calculated as T = x y S p 1 n x + 1 n y Where x and y are sample means; n x and n y are the sample sizes; S p is the pooled estimate of the variance, S p 2 = n x 1 S x 2 + n y 1 S y 2 n x 1 + n y 1 Reject H 0 : μ x = μ y if T > t α, n x + n y 2

17 Statistical Approaches: Student s t-test (3 of 3) Caveat: Only test if means are not equal, (i.e. different from each other) Implication is that we are attempting to prove that the mean values are different with high confidence rather than attempting to prove that they are the same. Assumes that variances of the two populations are equal Student s T-test for the difference between means is NOT appropriate for demonstrating equivalence Favors samples with large standard deviations Favors small sample sizes

18 Statistical Approaches: ANOVA (F-test) Uses of Analysis of Variance (ANOVA): Determine if analytical test results from three analysts are significantly different Like the t-test, ANOVA tests for zero-difference between the population means. ANOVA has the same problems as the t-test. F is calculated as F = Variation due to Methods Variation due toerror = MSR MSE ANOVA is NOT suitable for demonstrating equivalence.

19 Statistical Approaches: Schuirmann s Two One-sided test (TOST); (1 of 13) Uses: Demonstrate equivalence between two analytical methods or materials or processes Is there an unacceptable difference between two sets of results? Test the hypothesis H 01 : μ x μ y θ L or H 02 : μ x μ y θ U versus H a1 : μ x μ y > θ L and H a2 : μ x μ y < θ U Where θ L and θ U are pre-defined upper and lower acceptable difference limits for equivalence, i.e. LEL, UEL

20 Statistical Approaches: Schuirmann s Two One-sided test (TOST); (2 of 13) The One-sided T statistics are calculated as T L = x y θ L S p 1 n x + 1 n y T U = x y θ U S p 1 n x + 1 n y H 01 is rejected if T L > t 1 α, n x + n y 2 ; i.e. 1-α quantile of the t distribution with n x + n y 2 d.f. H 02 is rejected if T U > t 1 α, n x + n y 2

21 Statistical Approaches: Schuirmann s Two One-sided test (TOST); (3 of 13) Graphical representation of TOST Not Equivalent Not Equivalent

22 Statistical Approaches: Schuirmann s Two One-sided test (TOST); (4 of 13) Power of TOST, 1-β Probability of rejecting H 01 or H 02 when the true mean difference is bounded by Sample size, n, for TOST giving rise to the Power of is 1-β: n 2σ2 2 z α + z β/2 2 At 5% significance level, α =0.05, and 80% power, β=0.2, z 0.05 =1.645 and z 0.2/2 =1.282 No closed form for n with multiple measurements per run, e.g. multiple AI for a product

23 Statistical Approaches: Schuirmann s Two One-sided test (TOST); (5 of 13) Power of TOST, 1-β If variance, σ 2 =2, and largest acceptable difference, = 1.6, the sample size required for TOST at 5% level of significance and 80% Power is: n = 14 Sample size to conclude equivalence between analytical methods with at least 80% power and 95% confidence when the mean difference is 1.6 is 14.

24 Statistical Approaches: Schuirmann s Two One-sided test (TOST); (6 of 13) Power of TOST, 1-β Sample Size for Power of TOST=80% at the 5% Significance Level Variance= n Acceptable Difference between Means 5 Sample size required decreases as acceptable difference increases

25 Statistical Approaches: Schuirmann s Two One-sided test (TOST); (7 of 13) TOST does not assume that there is no difference in the analytical test results TOST expects some acceptable differences (θ L and θ U ; LEL and UEL) in the comparison Does not favor samples with large standard deviations TOST is an appropriate method for determining equivalence. Analysis can be performed in SAS JMP or Minitab 17

26 Statistical Approaches: Schuirmann s Two One-sided test (TOST); (8 of 13) Choice of Samples Homogeneous Lots Representative lots that cover a wide spectrum of results For example, stability samples beyond testing windows or expiry periods Sensitivity and Specificity Uniform and stable process / product Samples represent reasonable range of potential results OOS samples if available

27 Statistical Approaches: Schuirmann s Two One-sided test (TOST); (9 of 13) Apriori selection of an acceptable difference Subject Matter Expert Variability of the methods Sample Size Underlying methods Historical data and/or previous analyses Risk-based approach Process change No historical data

28 Statistical Approaches: Schuirmann s Two One-sided test (TOST); (10 of 13) Apriori selection of an acceptable difference Too restrictive Test results may fail the comparability study even if they are acceptable Too wide Equivalency of the two methods may not be adequately demonstrated

29 Statistical Approaches: Schuirmann s Two One-sided test (TOST); (11 of 13) Acceptance Criteria for method transfer Method Type Acceptance Criteria Assay, content uniformity ± 2% Dissolution ± 5% Impurities, degradation products ± 15-20% and/or comparison of profiles Source: Analytical Method Equivalency Don Chambers, Kelly G., Limentani G., Lister A., Lung K.R., and Warner E. Pharmaceutical Technology. Sept. 2005

30 Statistical Approaches: Schuirmann s Two One-sided test (TOST); (12 of 13) Risk-based approach for determining acceptable difference (AD) Given upper (USL) and/or lower specification (LSL) limits for the process parameter or quality attribute Higher risks should allow only small AD s and larger practical differences. Two-sided Specifications, % Tolerance (USL-LSL) High Risk Medium Risk Low Risk 5-10% 11-25% 26-50% One-sided Specification Limits, % Tolerance ( x-lsl or USL- x) High Risk Medium Risk Low Risk 5-10% 11-25% 26-50% Source: T.A. Little, Equivalence Testing for Comparability, Biopharm Int. Feb. 2015

31 Statistical Approaches: Schuirmann s Two One-sided test (TOST); (13 of 13) Risk-based approach for determining acceptable difference (AD) Consider a test method with USL=11.5% and LSL=10.5% for which a high risk of 7.5% is chosen. The tolerance is (USL- LSL)=( )%=1.0%. Consequently, AD=Risk-Level*Tolerance AD= 0.075*( )=0.075 Risk-level should be assessed by a subject matter expert (SME).

32 Statistical Approaches: Confidence Interval, Goal Post (1 of 3) Uses Demonstrate equivalence between two analytical methods or materials or processes Are means, μ x and μ y, for the two methods equivalent? Yes if μ x μ y Construct a 100(1-2α)% confidence interval for μ x μ y. If interval that is bounded by - and + contains μ x μ y then μ x and μ y can be declared to be equivalent at the α level of significance

33 Statistical Approaches: Confidence Interval, Goal Post (2 of 3) Key Assumption: Equal-tailed confidence intervals, i.e. Symmetry around the target for the assay. Conclusion is same as that from TOST Enjoys same properties as TOST Goal Post approach can be used to demonstrate equivalence between two analytical methods, materials, or processes

34 Statistical Approaches: Confidence Interval, Goal Post (3 of 3) Graphical Representation of Goal Post approach Equivalent Not Equivalent μ x μ y Not Equivalent Equivalent μ x μ y Equivalent μ x μ y Analysis can be performed in SAS JMP or Minitab

35 Statistical Approaches: Normal Tolerance Intervals (1 of 5) TI is the interval within which we expect a stated proportion, p, of the population to lie with some confidence, 1-α. TI x = μ x ± k 2 s TI y = μ y ± k 2 s where k 2 is a tolerance factor (Guenter, 1977), k 2 = z 1 p /2 n n χ 2 1 α,n n 3 χ2 1 α,n 1 2 n Risk assessment by SME is used to determine the uncertainty, p, i.e. 99.9% or 99.0% or 95.0%. Analysis can be performed in Minitab

36 Statistical Approaches: Normal Tolerance Intervals (2 of 5) Two-Sided TI with 99% uncertainty, p=.99, and 95% confidence: k 2 = z /2 n n χ ,n n 3 χ2 0.95,n 1 2 n For method x, Lower equivalence limit, LEL x = x k 2 s For method x, Upper equivalence limit, UEL x = x + k 2 s For method y, Lower equivalence limit, LEL y = y k 2 s For method y, Upper equivalence limit, UEL y = y + k 2 s Estimate Tolerance Intervals for the reference test method

37 Statistical Approaches: Normal Tolerance Intervals (3 of 5) Let USL and LSL be the upper and lower release specification limits for the reference method. Suppose method x is the reference Let L = LEL x LSL for LEL x LSL and U = USL UEL x for USL UEL x If the minimum of L and U, min L, U, is less than the acceptable difference then the two methods can be declared to be equivalent at the α level of significance (USP <1010>). Can be used to show that there is no major difference between two methods or materials or processes

38 Statistical Approaches: Normal Tolerance Intervals (4 of 5) For One-sided TI, estimate k 1 as (Natrella, 1963) k 1 = z p + z 2 p 1 z 2 1 α 2 n 1 1 z 2 1 α 2 n 1 z 2 p z 2 1 α n One-sided TI with 99.0% uncertainty and 95.0% confidence, α =.05 k 1 = 2 z z z Lower equivalence limit, LEL = 2 n 1 1 z n 1 2 z n z 0.99 x y k 1 s

39 Statistical Approaches: Normal Tolerance Intervals (5 of 5) For One-sided TI Suppose method x is the reference For method x, Lower equivalence limit, LEL x = For method x, Upper equivalence limit, UEL x = x k 1 s x + k 1 s If the minimum of L and U, min L, U, is less than the acceptable difference then the two methods can be declared to be equivalent at the α level of significance (USP 1010) Can be used to demonstrate equivalence between two methods or materials or processes Incorporates notion of uncertainty

40 Statistical Approaches: GLM Tolerance Intervals (1 of 4) Uses for General Linear Model (GLM) Tolerance Intervals Demonstrate equivalence between > 2 methods or materials or processes. Suppose we wish to compare a methods. Consider a one-way random effects model Balanced: Equal number of samples for each method; Variances are similar Y ij = μ + τ i + ε ij, j = 1,2,, n; i = 1,2,, a Where μ is the overall mean; τ i s are random effects for the test methods, and are ε ij s random residual errors

41 Statistical Approaches: GLM Tolerance Intervals (2 of 4) One way random effects model Y ij = μ + τ i + ε ij, j = 1,2,, n; i = 1,2,, a 2 Assumptions: τ i ~N 0, σ τ and ε ij ~N 0, σ 2 ε. Thus, Y ij ~N μ, σ 2 τ + σ2 ε Tolerance Intervals for N μ, σ τ 2 + σ ε 2 Let σ 1 2 = nσ τ 2 + σ ε 2 and σ 2 2 = σ ε 2 such that σ τ 2 = σ 1 2 σ 2 2 Here σ 1 2 and σ 2 2 are the variation due to test methods and error, respectively n

42 Statistical Approaches: GLM Tolerance Intervals (3 of 4) Tolerance Intervals for N μ, σ 2 τ + σ 2 ε Lian, 2013): is (Krishnamoorthy and Where ω 1 α = a 1 σ a 2 σ 2 2 μ ± z1+p 2 ω 1 α + a 1 2 σ 1 4 a 1 χ 2 α,a a 2 2 σ 2 4 a n 1 χ 2 α,a(n 1) 1 2 with a 1 = a n and a 2 = 1 1 n

43 Statistical Approaches: GLM Tolerance Intervals (4 of 4) Tolerance Intervals for N μ, σ τ 2 + σ ε 2 Unknown parameters are obtained by Maximum Likelihood Estimation (SAS, SAS JMP, Minitab, R) If overall mean is outside the Tolerance Interval, the methods are not equivalent Can be used to demonstrate equivalence When a =2, comparable to TOST Closed forms for unequal number of samples and heterogeneous variances exist

44 CASE STUDY 1: Equivalency using TOST (1 of 3) HPLC Assay data for tablet strength A: Are the test results different? TPW-II: 3 HPLC s; 3 separate days; independent reference standards Manual by 3 operators on 3 separate days: independent reference standards Source: Lung et. al Journal of Automated Methods & Management in Chemistry. (25):6

45 CASE STUDY 1: Equivalency using TOST (2 of 3) HPLC Assay data for tablet strength A: Are the test results different? Test for equality of variances Variances do not differ (P > 0.05)

46 CASE STUDY 1: Equivalency using TOST (2 of 3) TOST: Acceptable difference of 2%. Thus, LEL=-2.0, UEL=2.0 Means for the automated and manual test methods are equivalent (P > 0.05)

47 CASE STUDY 1: Equivalency using Confidence Intervals (1 of 1) HPLC Assay data for tablet strength A: Are the test results different? Hypothesized difference= =2.0. α=0.05 The actual difference of is significantly different from the hypothesized difference of 2.0. The two methods are equivalent (P<.05).

48 CASE STUDY 1: Equivalency using Normal Tolerance Intervals (1 of 2) HPLC Assay data for tablet strength A: LSL=95.0%, USL=105.0% Let reference=automated Method, p=.99, and α =0.05 Here L = =1.7 and U = = 3.4

49 CASE STUDY 1: Equivalency using Normal Tolerance Intervals (2 of 2) HPLC Assay data for tablet strength A: LSL=95.0%, USL=105.0% Recall that L = =1.7 and U = = 3.4 Therefore, min L, U = min 1.7,3.4 = 1.7 We declare that the two methods are equivalent Findings from Normal Tolerance Intervals agree with TOST and Goal Post approaches SME should be involved in selecting a reference method

50 CASE STUDY 1: Equivalency using GLM Tolerance Intervals (1 of 3) HPLC Assay data for tablet strength A: σ τ 2 = , σ ε 2 = , n = 35, z = χ ,2 1 = , χ2 0.05, 2(25 1) =

51 CASE STUDY 1: Equivalency using GLM Tolerance Intervals (2 of 3) HPLC Assay data for tablet strength A: Our estimate for the intra-class coefficient, ρ = ρ = = 0.95 σ τ 2 σ τ 2 + σ ε 2, is The coverage probabilities of the multivariate TI can be estimated using Monte Carlo Simulations

52 CASE STUDY 1: Equivalency using GLM Tolerance Intervals (3 of 3) Coverage probabilities for multivariate TI s: Balanced Data (a,n ) (5,3) (5,2) (10,2) (20,2) (10,6) (15,5) TI s are accurate for large values of ρ

53 CASE STUDY 2: Equivalency using TOST (1 of 5) HPLC Assay for Injectable Solution following a change in the manufacturing process (New Compounding Tank of Same Size) n = 22 batches Data represented as % of target assay (% Label Claim) Results from Process Capability are compared with TOST No historical data but expect equivalence Risk-based approach is used to determine the acceptable difference using a high risk estimate of 7.5%

54 CASE STUDY 2: Equivalency using TOST (2 of 5) HPLC Assay for Injectable Solution: Capability Analysis: Test for Equivalence of Variances Variances do not differ (P > 0.05).

55 CASE STUDY 2: Equivalency using TOST (3 of 5) HPLC Assay for Injectable Solution: Capability Analysis Process remained capable with a High Level of Confidence (C pk =3.79).

56 CASE STUDY 2: Equivalency using TOST (4 of 5) HPLC Assay for Injectable Solution: LEL=-0.075, UEL=0.075 Test results from Old and New Tank are equivalent (P >.05)

57 CASE STUDY 2: Equivalency using TOST (5 of 5) HPLC Assay for Injectable Solution The 95% confidence interval for the difference between the means for the Old and New tanks is within the equivalence interval (-0.075, 0.075). In the absence of historical data, a risk-based estimate of the acceptable difference can be used Results from TOST are consistent with null hypothesis Results from TOST support findings from the Capability Analysis

58 Comparability: Conclusions (1 of 2) An attempt has been made to present a unified framework for conducting a comparability study Equivalence testing procedures should be used instead of significance testing whenever the objective is to demonstrate equivalence. Schuirmann s Two One-sided test (TOST) can be used to demonstrate equivalency SME should be involved in determining an acceptable difference between two test methods When no historical data are available, the acceptable difference can be obtained using a risk-based approach

59 Comparability: Conclusions (2 of 2) SME should participate in selecting a risk-level for calculating the acceptable difference Under the assumption of symmetry around the target, the Goal Post approach is similar to TOST. Normal Tolerance Intervals are useful for showing that there is no major difference between two analytical methods. SME input is required in selecting the reference method for the comparison. GLM Tolerance Intervals allow for the comparison of more than two methods, simultaneously. Further work is needed to incorporate GLM Tolerance Intervals in software packages

60 Thank you!

61 Forward- Looking Stateme nts This presentation may contain forward-looking statements based on current assumptions and forecasts made by Bayer Group or subgroup management. Various known and unknown risks, uncertainties and other factors could lead to material differences between the actual future results, financial situation, development or performance of the company and the estimates given here. These factors include those discussed in Bayer s public reports which are available on the Bayer website at The company assumes no liability whatsoever to update these forward-looking statements or to conform them to future events or developments.

62 Page 62 Name of presentation February 9, 2011