Analysis of Reagent Lot-to-Lot Comparability Tests in Five Immunoassay Items

Transcription

1 Available online at Analysis of Reagent Lot-to-Lot Comparability Tests in Five Immunoassay Items 165 Hyun Soo Kim 1, Hee Jung Kang 1, Dong Hee Whang 2, Seong Gyu Lee 3, Min Jeong Park 1, Ji-Young Park 1, and Kyu Man Lee 1 1 Department of Laboratory Medicine, Hallym University College of Medicine, Seoul; 2 Department of Laboratory Medicine, Inje University Seoul Paik Hospital, Seoul; 3 Department of Laboratory Medicine, Bundang Jesaeng General Hospital, Sungnam, Korea Abstract. We investigated the degree of lot-to-lot reagent variation for 5 common immunoassay items. We measured the commercial as well as in-house controls for α-fetoprotein (AFP), ferritin, CA19-9, quantitative hepatitis B surface antigen (HBsAg), and hepatitis B surface antibody (anti-hbs) 10 times each by using both the old and the new lot of reagents whenever a reagent lot was changed, over a period of 10 months. The differences in the mean control values, the percent difference (% difference), and the difference to between-run standard deviation ratio (D:SD ratio) between successive lots were calculated. The % difference in mean control values between 2 reagent lots ranged from 0.1 to 17.5% for AFP, 1.0 to 18.6% for ferritin, 0.6 to 14.3% for CA19-9, 0.6 to 16.2% for HBsAg, and 0.1 to 17.7% for anti-hbs except negative controls of HBsAg and anti-hbs. The maximum D:SD ratios between 2 lots were 4.37 for AFP, 4.39 for ferritin, 2.43 for CA19-9, 1.64 for HBsAg, and 4.16 for anti-hbs. Thus, we have experienced extensive variability in lot-to-lot reagent variation for 5 immunoassay items, indicating that reagent lot-tolot comparability tests should be continuously performed and that laboratories should determine their own acceptance criteria for each item. Key words: Immunoassay, Quality-control, Lot, Reagent Introduction Lot-to-lot reagent reproducibility is an important issue in immunoassays [1, 2]. Occasionally, measured values of the same quality-control material are quite different between different reagent lots; however, it is difficult to distinguish whether the changes are due to variation between the reagent lots, degeneration of the control material, or other instrumental problems. Accordingly, a reagent lotto-lot comparability test, in which the results of a new reagent lot are compared to those of a previously used lot, using control materials or patient specimens, is usually performed. This allows for the detection of differences between lots, prediction of a change in the control value, and if necessary, for the determination of a new target value and an acceptable range of control values. Address correspondence to Hyun Soo Kim, M.D., Associate Professor, Department of Laboratory Medicine, Hallym University College of Medicine; Yeongdeungpo-dong, Yeongdeungpo-gu, Seoul, , Korea; tel: ; fax: , hskim0901@empal.com Immunoassays show wider inter-lot variations than routine chemistry tests [1-3]. This is mainly due to the characteristics of the process by which reagents are prepared, which involves the binding of specific antigens or antibodies to a solid phase. Ideally, time after time, exact concentrations of antigens or antibodies should be bound to the solid phase during this process, but in reality, this concentration can vary between reagent lots. In addition, the inter-lot difference may be even greater depending on the quality-control measures of the manufacturer [3]. Immunoassays make use of an antigen-antibody reaction; the equilibrium between the antigen, the antibody, and the antigen-antibody complex involves the electrostatic force, hydrogen bonds, van der Waals force, or hydrophobic force, etc. These interactions are influenced by other factors, such as ph, temperature, ionic strength, detergent, protein concentration, lipid concentration, and cross-reacting material. Due to this matrix effect, commercial control materials may indicate erroneous results, /12/ by the Association of Clinical Scientists, Inc.

2 166 Therefore, in this study, we sought to investigate the degree of lot-tolot reagent variation among commonly used immunoassay items in order to present concrete laboratory data that can be used in setting acceptance criteria for reagent comparability tests. Additionally, we intended to establish whether matrix effects also influenced commercial controls, as compared to controls prepared inhouse using only patient serum. such as falsely identical or falsely different results between different lots. Thus, it is usually recommended that the results of a test before and after changing the reagent lot are compared by performing a reagent comparability test, by using specimens from patients. Laboratories should have their own criteria for accepting test results for reagent comparability tests as well as for daily quality-control assays. The limit for acceptance criteria may vary according to the precision, or coefficient of variation, of each test item. Many laboratories in Korea calculate the percent difference between the results obtained using different lots, and use a cutoff value of 10% difference (or 20% in low concentrations) as an acceptable limit in the reagent comparability test. However, there has been little data published on acceptance criteria in reagent comparability tests. Materials and Methods Evaluation items, instruments, and reagents. Five items were tested; viz. α-fetoprotein (AFP), ferritin, cancer antigen 19-9 (CA19-9), hepatitis B surface antigen (HBsAg), and hepatitis B surface antibody (anti-hbs). Whenever a reagent lot was changed, the levels of both commercial and inhouse quality-control materials were measured 10 times each, using both reagents of the extant lot and those of the new lot. Commercial quality-control material of the same lot was used throughout the study period. The systems used were the following (Table 1): ADVIA Centaur (Siemens Healthcare Diagnostics, Inc., NY, USA) for AFP and ferritin; Architect i2000 (Abbott Diagnostics Division, Abbott Park, IL, USA) for quantitative HBsAg and anti-hbs;

3 Lot-to-lot Reagent Variation 167 mixed by gentle inversion prior to use. Commercial controls for AFP, ferritin, and CA19-9 were lyophilized ones and were dissolved by exact amount of distilled water and then mixed. Aliquots of reconstituted controls were stored at -70 C in sterilized microtubes. Reconstituted controls were warmed to room temperature and mixed well before use. and the Roche Cobas E 411 analyzer (Roche Diagnostics, Mannheim, Germany) for CA19-9. Chemiluminescence immunoassay was applied with dedicated reagents and calibrators to all the assays. Commercial controls manufactured by the same company were used for CA19-9, HBsAg, and anti-hbs, but commercial controls for AFP and ferritin were obtained from different manufacturers (Bio-Rad, Hercules, CA, USA). Commercial controls for HBsAg and anti-hbs were liquid, Manufacture of in-house prepared qualitycontrol materials. Patients specimens were collected and stored at -20 C prior to production of the quality-control material that was to be used for the comparability tests between lots. When sufficient specimens had been collected for preparation of the in-house quality-control material, the frozen sera of the patients were thawed, sorted into high, medium, and low concentrations, and each put into a 50-mL falcon tube. After centrifugation at 3,000 rpm for 10 min, the serum supernatants belonging to each category were filtered into a beaker through gauze and mixed, and the concentration of the pooled specimen was then measured. To achieve a

4 168 concentration similar to that of the commercial quality-control material, the specimens with high, medium, and low concentrations were mixed in appropriate ratios, and the resulting concentration established. When the measured concentration was similar to that of the commercial control, the mixed specimen was mixed overnight for >18 h by magnetic stirrer in a refrigerated room. The next day, the concentration of the mixed specimen was measured again, and the specimen was filtered through a 0.22-μm filter. Aliquots (700 1,200 μl) of the specimen were stored at -70 C in sterilized microtubes. In-house-prepared controls were warmed to room temperature and mixed well before use. Precision of the in-house controls and commercial controls. To obtain the mean concentration and the standard deviation (SD) of the in-houseprepared quality-control material, measurements were made 4 times a day (twice in the morning and twice in the afternoon) for 10 days by using 20 aliquots, totaling 40 measurements. For commercial controls, measurements were taken over 40 days in order to establish the mean, SD, and coefficient of variance; the same quality-control material and the same reagent lot were used for these measurements. For each of the 5 items, the commercial controls were measured once a day. Because the acceptable limit for variation in quality-control material in our laboratory was set as mean ± 3SD, re-test or calibration was performed when the results exceeded this limit. Reagent comparability tests at the time of reagent lot changes. For the study period, from November 2009 to August 2010, the commercial quality-control material and the in-house-prepared quality-control material for each of the 5 test items were measured 10 times each by using both the extant reagent and the reagent from the new lot; and the difference between the results obtained by the 2 lots of reagents analyzed. When reagent from the original lot was available at the end of the study period, a reagent comparability test using the first and the last reagent lots was performed. Analysis of the results. Each of the reagent lots was used for measurement 10 times. The difference between results measured using different reagent lots, by using the commercial quality-control material and the in-house-prepared quality-control material, was analyzed by student t-test, with a significance level of P < MedCalc software version 11 (MedCalc Software, Mariakerke, Belgium) was used for analysis. The degree of differences between 2 lots are presented as difference, % difference, and difference to SD ratio (D:SD ratio); the latter 2 approaches were used because many Korean laboratories use the criteria of 10% difference as the acceptable limit for reagent comparability tests, and the D:SD ratio can represent the degree of difference between lots compared to daily variation of measurement. The difference in the mean value (Difference = c2 - c1), the percent difference [%Diff = (c2 - c1)/{(c1 + c2)/2} 100 (%)] [4], and D:SD ratio [D:SD ratio, where SD is the between-run SD, shown in Table 2] were calculated at each change in reagent lots. Results Precision of the quality-control materials. For AFP, the coefficients of variance were % for the in-house-prepared quality-control material, and % for the commercial quality-control material (Table 2). For ferritin, the coefficient of variance was % for the in-house-prepared quality-control material and % for the commercial controls. For CA19-9, the coefficients of variance were % for the in-house-prepared quality-control material and % for the commercialquality-control material. For HBsAg and anti-hbs, the coefficient of variance for the negative control was very large ( %), as expected. Reagent comparability tests at the time of the reagent lot changes: AFP. During the study period, 3 reagent lots were used (lots A, B, and C), and a comparability test was performed twice (March 22 and July 28, 2010); these results are summarized in Table 3. The % differences for mean control values were relatively small ( %) between lots A and B, but much larger ( %) between lots B and C. Similarly, the D:SD ratio was small ( ) for lots A and B, but larger ( ) for lots B and C. The pattern of increase or decrease between 2 reagent

5 Lot-to-lot Reagent Variation 169 lots showed a similar trend at each concentration of the commercial and in-house control materials. Ferritin. For ferritin, a total of 6 reagent lots were available during the study period, and a total of 5 comparability tests were performed (on February 2, February 22, March 12, May 20, and July 28, 2010) (Table 4). As in the case of AFP, the results for the commercial and in-house control materials showed a similar trend for increase or decrease. A significant difference (P < 0.05) was found among nearly all the lots. The %differences of mean control values between 2 reagent lots extended across a range from 1.0% to 18.6%, and the D:SD ratio extended from 0.14 to For the commercial and in-house control materials, the % difference and the D:SD ratio between lots B and C and between lots E and F were found to be 10%, and 2-fold or more. CA19-9. For CA19-9, 3 comparability tests were performed with 3 lots. The % differences in the mean control values between 2 reagent lots ranged from 0.6 to 14.3%. Two of the three lots had significant differences, and % difference 10% was found between lots B and C (commercial control high level, 11.0%; in-house control high level, 12.3%), and between lots C and A (commercial control high level, -14.3%; in-house control high level, -10.6%, commercial control low level, -12.0%; inhouse control high level, -11.2%). The maximum D:SD ratio between 2 lots was As in the case of AFP and ferritin, the results of the measurements of the commercial and in-house control materials at different concentrations showed a similar trend for increase or decrease, based on the change in the reagent lot. (Data were not shown.) HBsAg. A total of 5 lots (lots A, B, C, D, and E) were analyzed for HBsAg. Significant differences (P < 0.05) were found between only a few lots. The % differences in the mean control values between 2 reagent lots ranged from 0.6 to 16.2%, and the D:SD ratio from 0.07 to 1.64, except for the negative controls, which showed a high % difference ( %) due to the low value of the denominator. The means of the negative quality-control materials were all within the IU/mL range, which was not significantly different. (Data were not shown.)

6 170

7 Lot-to-lot Reagent Variation 171 Anti-HBs. The differences among the 5 lots of reagents for anti-hbs were analyzed. In some of the lots, both the commercial and the in-house control materials showed a significant difference at each concentration (Table 5). The % differences in the mean control values between 2 reagent lots ranged from -0.1 to 17.7%, and the D:SD ratio ranged from to 4.16, except for the negative controls. A considerable difference was observed between lots D and E. Discussion As precision and lot-to-lot variability of assays may vary according to the item being assessed and the particular reagents used, it is necessary to identify the characteristics of the relevant reagents. Because it is virtually impossible to manufacture a new reagent lot that would be identical to an older one with respect to all conditions, both the laboratory and the manufacturer should properly determine the acceptable range of difference between the reagent lots. A few studies have reported the occurrence of differences between reagent lots [5, 6, 13]. Since the level of variability between reagent lots differs according to the items being measured and the manufacturers of the reagents, it is advisable for each laboratory to determine the acceptable range for each test item, using cumulative data. Most laboratories in Korea evaluate the % difference and determine the acceptable limit to be 10% difference (20% at low concentrations) in a reagent comparability test. In this study, the value of D:SD ratio was evaluated as another criterion for establishing an acceptable limit, as the commonly used acceptable range for daily quality-control values is the mean ± 2SD or ± 3SD, and this ratio can indicate whether the difference between lots has exceeded the daily acceptable range for qualitycontrol. However, it is noteworthy that the SD value can vary according to the reagent lot itself, the number of lots used in a certain period, the number of measurements, and the period over which measurements are obtained. In this study, most of the SDs of the commercial controls were larger than those of the in-house controls, probably because of the longer period of measurement used for the commercial controls (40 days in a commercial control vs. 10 days in an in-house control), despite the fact that the same number of measurements (40) were made for both. The proficiency testing criteria set out in the Clinical Laboratory Improvement Amendments (CLIA) for acceptable analytical performance [7, 8] should be considered when establishing acceptable limit ranges for reagent comparability tests. These were initially proposed as criteria for establishing acceptable ranges for external quality control, but they are also useful in establishing criteria for acceptable ranges for internal quality control. Furthermore, they may also be useful in establishing acceptance criteria for variance arising from reagent lot changes. For example, the acceptable range of quality control for AFP is the target value ± 3 SD. For other criteria, the quality requirements of the Royal College of Pathologists of Australia (RCPA) [7] may also provide useful insights into establishing criteria for acceptable ranges in reagent comparability tests. In the RCPA quality requirements for AFP tests, when AFP concentration is less than 25 kiu/l (1 ng/ml = 0.83 IU/mL), variance should be less than ± 5 kiu/l; and when AFP concentration is more than 25 kiu/l, variance should be less than ± 20%. In CA19-9 tests, when CA19-9 concentration is <20 ku/l, variance should be ± 3 ku/l; and when CA19-9 concentration is more than 20 ku/l, variance should be less than ± 15%. In this study, AFP and CA19-9 results met these criteria. For ferritin, the difference between lots B and C and between lots E and F exceeded the acceptable criteria of 15% at the 40 mg/l or more. Additionally, the European biological goal and biological allowable total error % [7, 9, 10], and Biological Variation Database [7, 11, 12], which were calculated from the biological variations, were also referenced. According to these biological variations, the allowable total errors of AFP and CA19-9 are 21.8% and 39.0%, respectively; our data met these criteria. For ferritin, however, the difference between lots E and F exceeded the allowable total error (16.9%) for high-concentration commercial quality-control materials. A recently published CLSI guideline for the verification of comparability of patient results within one health care system [13] could also be considered

8 172

9 Lot-to-lot Reagent Variation 173 when establishing acceptable ranges for reagent comparability tests. In this document, acceptable criteria for comparability could be established based on clinical outcome studies, clinician s questionnaire, biological variability, published professional recommendations, goals set by accrediting agencies, or general capability. The second purpose of this study was to establish whether there were matrix effects in commercial controls, as compared with in-house-prepared controls made only from patients sera. As their increase or decrease trends at reagent lot changes were similar between the commercial controls and the in-house controls, it can be concluded that there were negligible matrix effects. If the difference between successive lots using commercial controls or small numbers of patient samples exceeds the acceptable criteria of the laboratory, further assessments using additional patient samples should be performed. If these further tests consistently yield a degree of difference that is unacceptable, claims against the manufacturer for the new reagent lot could be considered, or the laboratory director may consider changing the permissible range for the daily quality-control tests. In summary, we found that some lot-to-lot reagent variation in the tests for all 5 items exceeded the arbitrary acceptance criterion of 10% difference for reagent comparability tests. Therefore, reagent comparability tests should be continuously performed in order to detect large differences between lots. Moreover, we hope that the results and several possible acceptable criteria highlighted in this paper can be used as reference data for individual laboratories to determine their own acceptance criteria for reagent comparability tests. Acknowledgements This study was supported by a grant from the Laboratory Accreditation Program of The Korean Society for Laboratory Medicine in The authors are grateful to Dr. Young-Su Ju for statistical advice and to Ki-Young Kwon for manufacturing in-house controls. References 1. Kitchen AD and Newham JA. Lot release testing of serological infectious disease assays used for donor and donation screening. Vox Sang 2010;98: Holzel W. Analytical variation in immunoassays and its importance for medical decision making. Scand J Clin Lab Invest Suppl 1991;205: Kim HS, Choi SK, Ko YS, Chung D, Kim JM, Kim YJ, et al.evaluation of allergen specific IgE assay on ADVIA Centaur immunoassay system. Korean J Lab Med 2006;26: Relative change and difference. wiki/percent_difference (Updated on 9 November 2010) 5. Hoefner DM and Yeo KT. Lot-to-lot inconsistency of anticardiolipin reagents. Clin Chem 2002;48: Dufour DR. Lot-to-lot variation in anti-hepatitis C signal-tocutoff ratio. Clin Chem 2004;50: Westgard JO. Quality requirement (CLIA and others). (Updated on October 2010) 8. Lasky FD. Book, software, and web site reviews: CLIA final rules for quality systems: Quality assessment issues and answers. Clin Chem 2005;51: Fraser CG, Petersen PH, Ricos C, Haeckel R. Proposed quality specifications for the imprecision and inaccuracy of analytical systems for clinical chemistry. Eur J Clin Chem Clin Biochem 1992;30: Petersen PH, Ricos C, Stockl D, Libeer JC, Baadenhuijsen H, Fraser C, et al. Proposed guidelines for the internal qualitycontrol of analytical results in the medical laboratory. Eur J Clin Chem Clin Biochem 1996;34: Ricos C, Alvarez V, Cava F, Garcia-Lario JV, Hernandez A, Jimenez CV, et al. Current databases on biological variation: pros, cons and progress. Scand J Clin Lab Invest. 1999;59: Ricos C, Iglesias N, Garcia-Lario J, Simon M, Cava F, Hernandez A, et al. Within-subject biological variation in disease: collected data and clinical consequences. Ann Clin Biochem 2007;44: Clinical and Laboratory Standards Institute. Verification of comparability of patient results within one health care system: approved guideline. CLSI document C54-A. Wayne, PA: Clinical and Laboratory Standards Institute, 2008.