A Simple QC Rule for Highly Capable Measurement Procedures

Transcription

1 A Simple QC Rule for Highly Capable Measurement Procedures Curtis A. Parvin, Ph.D. Manager of Advanced Statistical Research, Quality Systems Division

2 Background 2013 AACB Satellite Symposium Setting QC target standard deviation Use the analytical SD of the measurement procedure Base it on the specified quality goal for the analyte Suggested that this is a useful approach for highly capable measurement procedures? I decided to investigate the merits of basing QC rule limits on the specified quality goal for an analyte 1

3 2014 AACC Poster 2

4 Introduction A measurement procedure s process capability can be characterized by computing its sigma-metric value Sigma = (TE a Bias ) / SD TE a is the quality requirement Bias is the absolute bias in the measurement procedure SD is the stable (in-control) standard deviation of the measurement procedure Low sigma-metric processes are hard to QC High sigma-metric processes are easy to QC 3

5 Goal Evaluate the performance of a simple QC rule that is based on the allowable total error specification for an analyte (TE a ). Determine how high a measurement procedure s sigma-metric value must be for the simple QC rule to meet or exceed desirable quality control performance characteristics. 4

6 Methods Notation: μ u = the unbiased concentration of a QC material μ i = instrument s mean concentration for the QC material Bias = μ i μ u SD i = instrument s standard deviation for the QC material TE a = allowable total error specification The QC rule is defined to have rejection limits given as μ u ± f*te a (if TE a is given in analyte result units) μ u *(1 ± f*te a ) (if TE a is given as a percent) f is a constant defining a fixed fraction of TEa The QC rule rejects if any QC measurement exceeds the limits 5

7 QC Rule Performance The QC rule s performance is characterized by; probability of false rejection, P fr Probability of error detection, P ed (SE c ) SE c is defined as the magnitude of shift that would cause 5% of measurement errors to exceed TE a Rule performance will depend on instrument bias Sigma-metric value QC rule performance is evaluated over a range of sigma-metric values assuming different amounts of instrument bias 6

8 Determining Suitable Rejection Limits For a given instrument bias The QC rule s value for f that gives P fr = 0.01 is determined as a function of sigma-metric value The value for f that gives P ed (SE c ) = 0.90 is determined as a function of sigma-metric value Values for f were identified that met both criteria; Pfr 0.01 P ed (SE c ) 0.90 The minimum sigma-metric value with at least one f value that met both criteria is determined. 7

9 Rejection Limits that Meet Criteria P ed (SE c ) = 0.90 P fr =

10 Rejection Limits that Meet Criteria P ed (SE c ) = 0.90 P fr =

11 Rejection Limits that Meet Criteria P ed (SE c ) = 0.90 P fr =

12 Rejection Limits that Meet Criteria P ed (SE c ) = 0.90 P fr =

13 Rejection Limits that Meet Criteria Bias /TE a

14 Rejection Limits that Meet Criteria f = 0.6 meets criteria if Bias 0.25*TE a and Sigma > 5.5 Bias /TE a

15 Rejection Limits that Meet Criteria f = 0.6 meets criteria if Bias 0.25*TE a and Sigma 6 Bias /TE a

16 Rejection Limits that Meet Criteria f = 0.65 meets criteria if Bias 0.25*TE a and Sigma 5.1 Bias /TE a

17 Rule Performance, N Q = 2 µ u ± 0.6*TE a Rejection Limits Bias = 0 Bias / TE a = 0.20 Sigma P fr P ed (SE c ) P fr P ed (SE c ) <

18 Rule Performance, N Q = 2 µ u ± 0.6*TE a Rejection Limits Bias = 0 Bias / TE a = 0.20 Sigma P fr P ed (SE c ) P fr P ed (SE c ) < P fr decreases as Sigma increases 14

19 Rule Performance, N Q = 2 µ u ± 0.6*TE a Rejection Limits Bias = 0 Bias / TE a = 0.20 Sigma P fr P ed (SE c ) P fr P ed (SE c ) < P ed (SE c ) increases as Sigma increases 14

20 Rule Performance, N Q = 2 µ u ± 0.6*TE a Rejection Limits Bias = 0 Bias / TE a = 0.20 Sigma P fr P ed (SE c ) P fr P ed (SE c ) < P ed (SE c ) <

21 Rule Performance, N Q = 2 µ u ± 0.6*TE a Rejection Limits Bias = 0 Bias / TE a = 0.20 Sigma P fr P ed (SE c ) P fr P ed (SE c ) < P fr >

22 Rule Comparison: N Q = 2, Bias = 0 QC Rule Rejection Limits µ u ± 0.6*TE a Sigma P fr P ed (SE c ) <

23 Rule Comparison: N Q = 2, Bias = 0 QC Rule Rejection Limits µ u ± 0.6*TE a µ i ± 3*SD i Sigma P fr P ed (SE c ) P fr P ed (SE c ) < >

24 Rule Comparison: N Q = 2, Bias = 0 QC Rule Rejection Limits µ u ± 0.6*TE a µ i ± 3*SD i Sigma P fr P ed (SE c ) P fr P ed (SE c ) < > P fr, P ed (SE c ) identical at Sigma = 5 15

25 Rule Comparison: N Q = 2, Bias = 0 QC Rule Rejection Limits µ u ± 0.6*TE a µ i ± 3*SD i Sigma P fr P ed (SE c ) P fr P ed (SE c ) < > P fr doesn t change as Sigma increases 15

26 Conclusions QC rules of the form µ i ± k*sd i Are centered on the instrument s mean concentrations Limits are based on multiples of the instrument s SD Multiple instruments measuring the same analyte have different rejection limits QC rules of the form µ u ± f*te a Are centered on the unbiased QC concentrations Limits are based on a fraction of the allowable total error Multiple instruments measuring the same analyte have the same rejection limits 16

27 Conclusions QC rules of the form µ i ± k*sd i Keep the false rejection rate fixed. Increasing sigma-metric values increase the probability of error detection. At the point where the probability of detecting a critical outof-control condition is nearly 1.0 there is no additional QC performance gain for higher sigma-metric values. 17

28 Conclusions QC rules of the form µ u ± f*te a Increasing sigma-metric values simultaneously decrease the false rejection rate and increase the probability of error detection. Beyond the point where the probability of detecting a critical out-of-control condition is nearly 1.0, the false rejection rate continues to decrease for increasing sigmametric values. 18

29 Conclusions For measurement procedures with Bias /TE a 0.25 Sigma-metric value 6 a QC rule with limits = µ u ± 0.6*TE a will provide A false rejection rate 1% An error detection rate 90% for out-of-control conditions that would produce 5% of patient results with measurement errors exceeding TEa. 19

30 Caveats These results assumed that the sigma-metric was independent of concentration (same sigma-metric value at every concentration). In cases where the sigma-metric value varies with concentration, then requiring that the minimum sigma-metric value across all QC concentrations is 6 should produce rule performance characteristics similar to or superior to those shown here. 20

31 Caveats Changing the criteria for; maximum acceptable probability of false rejection, minimum acceptable probability of detecting critical out-ofcontrol conditions, percentage of measurement errors exceeding TEa that defines a critical out-of-control condition would modify the results shown here, but the general insights and conclusions should still apply. 21

32 The End Questions? Comments?

33 A 5 Sigma-Metric Process Bias = 0 5 SDs 1

34 A 5 Sigma-Metric Process Bias /TEa = SDs 1

35 A 5 Sigma-Metric Process Bias /TEa = SDs 1

36 A 5 Sigma-Metric Process Bias /TEa = SDs 1

37 All are 5 Sigma-Metric Processes 1

38 All are 5 Sigma-Metric Processes 1