Validation and Standardization of (Bio)Analytical Methods Prof.dr.eng. Gabriel-Lucian RADU 21 March, Bucharest
Why method validation is important? The purpose of analytical measurement is to get consistent, reliable and accurate data. Incorrect measurement results can lead to tremendous costs. Equal importance for those working in a regulated and in an accredited environment. ISO, AOAC, U.S. FDA, EMEA, EPA
What should be the role of validation? As analytical chemistry is about making decisions validation should support the decision making process limit results
Traditional roles of method validation Establish performance characteristics Linearity Limits of detection/determination Precision: repeatability, intermediate reproducibility Effect of concomitants Present data for approval of method Produce control limits for everyday operation
Validation is an old concept but there are many problems Lack of documented procedures and documented validation results Sampling or Sample preparation step contribute to overall error Accessories and materials used for equipment qualification are not qualified Limits for Operational Qualification Lack of software validation and computer system validation Qualification and validation are done at just one particular point in time Adaptation of acceptance criteria for qualification of new system
Validation starts out from the customer s needs Requirements of customer Report results Report to customer Convert to uncertainty Do analyses Apportion to the different steps Accept/reject contract Express as figures of merit Estimate factual uncertainty Specify validation plan Compute figures of merit Validate
Validation activity including the complete analytical procedure Sampling Sample Preparation Analysis Data Evaluation Reporting
Optimization of validation Additional value and Cost versus Completeness of validation
Considerations prior to method validation Suitability of Instrument Status of Qualification and Calibration Suitability of Materials Status of Reference Standards, Reagents, Placebo Lots Suitability of Analyst Status of Training and Qualification Records Suitability of Documentation Written analytical procedure and proper approved protocol with pre-established acceptance criteria
Validation step Define the application, purpose and scope of the method Analytes? Concentration? Sample matrices? Develop a analytical method Develop a validation protocol Qualification of instrument Qualify/train operator
Qualification of material Perform pre-validation experiments Adjust method parameters and/or acceptance criteria if necessary Perform full validation experiments Develop SOP for executing the method in routine analysis Document validation experiments and results in the validation report
System Suitability Validation Calibration Analyst Method Sample
Validation requirements criteria to be validated Calibration curve Accuracy Precision (repeatability, reproducibility) Limit of quantification (LOQ) Limit of detection (LOD) Sensitivity Specificity/selectivity Stability of the analyte in the matrix under study Others (ruggedness, agreement, )
Role of traceability in validation Provide a firm and identical base of units worldwide Supply this base in a manner stable in time Underpin the hierarchy of measurements/procedures/laboratories...
Standard view of the relation between uncertainty and validation precision bias Measurement uncertainty Uncertainty of a standard Identity Purity Preparation other effects
Way to define the measurand / analyte A) Careful and complete description of circumstances: (exactly) what species What (kind of) samples/concomitants Which environmental conditions (p, T) B) Measure as specified and give the analyte a name
Redesigning validation studies what has been missing so far? Establish the quality of the standards Complete coverage of scope or clear redefinition of the analyte Anchored in the requirements of the customer
Traceability of results and reference values is a central issue in modern laboratory operation. It is not an end in itself, but serves the purpose of achieving a reliable result. Traceability of results can only be claimed if results are accompanied by an uncertainty statement based on traceability of all references, chemical and physical, as well as on procedural contributions to uncertainty. A result must be "fit for purpose", thus estimation of measurement uncertainty from uncertainties of references and procedures is added value for laboratories and simple when guidelines are followed.
Bioanalytical method validation Quantitative determinations of chemicals, drugs in biological samples, such as blood or plasma, play a significant role in evaluation and interpretation of reliable data in (bio)medicine. The bioanalytical method validation should include a full pre-study validation, to support that the method is suitable for its intended use study validation data, to support its performance during the analysis of the samples.
Development / Use of a bioanalytical method Reference standard preparation Method procedure development Application of validated method to routine analysis
Reference (Bio)Standards Calibration standards / Quality control samples Authenticated reference Known identity Known purity Reference standard should be identical to analyte, if possible
Bioanalytical method validation (1) Method validation should include: Accuracy Precision Sensitivity Specificity Recovery Stability Etc.
Bioanalytical method validation (2) Accuracy Closeness of determined value to the true value. Precision The closeness of replicate determinations of a sample by an assay. The acceptance criteria is mean value 15% deviation from true value. At LOQ, 20% deviation is acceptable. Sensitivity The limit of quantitation (LOQ) is the lowest concentration which can be measured with acceptable accuracy and precision.
Bioanalytical method validation (3) Specificity (selectivity) Ability of the method to measure only what it is intended to measure in the presence of other components in the sample. Blank samples of the biological matrix should be tested for interfering peaks. Calibration Curve Sufficient number of calibration standards must be employed Typically 6 8 non-zero standards Blank sample Zero sample Lower Limit of Quantification (LLOQ)
Bioanalytical method validation (4) Stability Stability of the analyte in the biological matrix Sample collection Sample storage Sample analysis -Freeze-thaw stability -Short-term stability -Long-term stability -Post-preparation stability -Stock solution stability Recovery The extraction efficiency of an analytical process, reported as a percentage of the known amount of an analyte carried through the sample extraction and processing steps of the method.
Bioanalytical method validation (5) Dilution integrity Confirm ability to dilute samples above Upper Limit of Quantitation (ULOQ) Accuracy and precision Additional considerations for ligand-binding assays Cross-reactivity Matrix effects Concentration-response relationship
Clinical sample analysis (1) Single determination of calibration standards Calibration curve for each analytical run Extrapolation below or above calibration range not acceptable
Clinical sample analysis (2) Quality control (QC) At least three concentrations Low concentration Midrange concentration High concentration Duplicate determinations 4 out of 6 QC samples must be 15% of nominal value; two outside cannot be at the same concentration
Clinical sample analysis (3) Repeat sample analysis Clear SOP and acceptance criteria Reasons for re-analysis Analytical issues Problematic pharmacokinetic fit Explanation for missing samples
Standard operating procedures Complete written set of SOPs Quality control and assurance All aspects of analysis should be covered including Record keeping Security Chain of sample custody Sample preparation Analytical tools Procedures for quality control and verification of results
It need to have confidence in the data and understand its limitations. A component of the variance in bioanalytical measurements determined by the physiology of the subjects observed
Identification the nature of biological variation What is meant by the term biological variation in the context of clinical biochemistry? A component of the variance in biochemical measurements determined by the physiology of the subjects observed.
Components of variance in clinical chemistry measurements Analytical variance Within Subject biological variance Between Subject biological variance
Biological variation All clinical chemistry measurements change with time. Knowledge of temporal changes useful in diagnosis and interpretation. Rate of change may be useful in prognosis. Understanding of the sources of biological variation in non-diseased subjects is fundamental to the development of reference data.
Sources of biological variation Biological Rhythms (time) Homeostasis Age Sex Ethnicity Pathology Stimuli
Practical significance of biological variation What is the significance of this result? Is the performance of the analytical method appropriate (imprecision, accuracy)? When should I measure it again? Has this result changed significantly over time? Changes in variability be used as a tool?
Models of biological variation Assume values represent random fluctuation around a setting point. More general model allows correlation between successive results. (Time series and non-decayed biological variation)
Quantifying biological variation How are you going to quantify biological variation? You have to dissect out the components of variance: s 2 total = s2 Analytical + s2 Individual + s2 Group
Quantifying biological variation s 2 Analytical = s 2 Individual = s 2 Group = Average variance of replicate assays within run analytical variance Average biological within subject variance Average variance around the homeostatic setting point Variance of true means among subjects Variance in homeostatic setting points
Reference materials in clinical chemistry When there is a change in reference materials this affects the final value it is critical that everyone in both the traceability chain and the clinical decision chain are aware of this effect. Different parties are involved in obtaining a value metrological institutes, manufacturers, and users. The value in itself is of importance only insofar as it can be used to make a clinical decision. There is often confusion between problems with the values and problem with the clinical decisions (interpretation of the values). Reference materials cannot be changed overnight!
Creatinine biological variation Upper Reference Limits: Male = 106 µmol/l Female = 80 µmol/l RCV larger for men than for women
Values in clinical chemistry Reference Material (Metrological Institute) Diagnostic kit Control Materials (Manufacturer) Result (User) Reference Range 100 um/l 80 um/l Clinical Interpretation (Correct) Traceability Chain Clinical Decisions
Calibration curve homogeneity of variance Problem of the homogeneity of variance Cochran's test Homogeneous Non homogeneous "cone shaped"
Full / Partial validation Full validation During method development Partial validation Transfer between laboratories Change in instrument / software Change in anticoagulant Minor changes in sample processing procedures
Analytical Performance Parameter Accuracy Precision Specificity LOD LOQ Linearity Range Ruggedness Data elements required for assay Assay Category 1 Yes Yes Yes No No Yes Yes Yes validation Assay Category 2 Quantitative Yes Yes Yes No Yes Yes Yes Yes * May be required, depending on the nature of the specific test. Limit Tests * No Yes Yes No No * Yes Category 1: Quantitation of major components or active ingredients Category 2: Determination of impurities or degradation products Category 3: Determination of performance characteristics Assay Category 3 * Yes * * * * * Yes
Inter-laboratory proficiency tests Many laboratory tests are used for international or national standard and regulations, for example: food labelling, water quality, animal feeds, pesticide residues. The authorised laboratories must show that their results are comparable with others doing the same tests.
Proficiency testing (1) Participation in proficiency testing schemes offer the following benefits: Show labs how well they compare with others; Help them to reduce the overall variability in testing; Give the regulatory authorities and consumers confidence that their quality criteria are meaningful.
Proficiency testing (2) Organisers of such schemes must ensure that: all participating labs receive identical samples; a true value is assigned for the result of a test (for example it may be the mean of all participants results); participants do not know the true result before they do the test.
Proficiency testing (3) From the results submitted by the participating laboratories, the organisers can: rank the laboratory s performance by calculating its z-score ; identify the laboratories that can produce acceptable results for the test; show divergent laboratories how they need to improve their performance of the test.
Calculation of z scores 10 8 6 4 A laboratory s z-score is calculated from: x X [(x - X)/SD] = the lab result = the true or accepted result z-score SD = target value for the standard deviation 2 0 1-2 -4-6 -8-10 Laboratory identity number The graph above shows the z scores obtained by laboratories who participated in a particular proficiency testing scheme The best performing laboratories have z values close to zero. The acceptable range is +2 to -2
Proficiency testing (4) Z z scores x ~ x IQR 0.7413 z z 2,satisfactory x ~ x Z IQR 0.7413 2< <3,questionable z 3,unsatisfactory x value reported by lab median x~ IQR median and normalised inter-quartile range
What is accreditation? Procedure by which an authoritative body gives formal recognition that a body or person is competent to carry out specific tasks. (From ISO/IEC Guide 2:1996) Key words : authoritative body competent specific tasks What, Why and How?
Laboratory accreditation is an evaluation process by which an authoritative body gives formal recognition that a laboratory is competent to carry out specific testing and/or calibration Evaluation Technical competence Management System Competence to carry out specific testing and/or calibration
Essence of accreditation Accreditation is based on the evaluation and confirmation of the validity of: Methods Procedures Results The standard for accreditation ISO/IEC/EN/SR: 17025:2005 General Requirements for Competence of Testing and Calibration Laboratories