The Binomial Approach for Probability of Detection

Transcription

1 Vol. No. (Mar 5) - The e-journal of Nondestructive Testing - ISSN The Binomial Aroach for of Detection Carlos Correia Gruo Endalloy C.A. - Caracas - Venezuela carloscorreya@gmail.com Abstract The resent works, resents tree different aroach to calculate the of Detection (POD) Using the Binomial Aroach. The ASME BPVC Sec. V Art. 4 roosal is included. POD will be a major issue during near future insection rojects. The sread of fracture mechanics as first method to establish critical defect size ask for more accurate roved NDT rocedures and insection teams. New methodologies for maintenance are under develoment to include the POD as one of the relevant factors to be considered in Probabilistic Failure Assessment and Risk Based Insection. The Binomial Aroach resent the advantage to be an easy method but relevant drawbacks should be considered.. Introduction During insection, not all detectable discontinuities would be detected. The roortion of detected (detectable) discontinuities, could change from time to time, from one insector to other insector, between different teams of insection or equiments or combinations. These fact is gaining imortance over the NDT industry, and these has lead to the study of the of Detection, which refers to the robability to detect a detectable discontinuity (for easy of exlanation we will incorrectly use the term "defect" instead discontinuity in some aragrah of this aer). It is imortant to kee in mind that POD calculations are made based on detectable indications of certain critical size. There is no sense to calculate POD for non detectable discontinuities (). POD alied in Nondestructive Testing has started at NASA in the early on 97s (), while the henomenon seems be conceived or imorted from other brands of technology as telecommunications, radar and medicine (). Most Nondestructive Testing (NDT) Methods and techniques are based on the interretation of signals which comes from interactions of some hysical stimulus with the medium and the discontinuity. The recetion and interretation of those signals could be affected by many different henomena, like system noise or other unexected signals coming from other rocesses. In that way, the insection results could be affected. NDT as any other technology is suscetible to errors. The consolidated techniques has well know drawbacks, and the human factor is a major issue: Fatigue, lack of roer training and exerience. Imroer rocedures or devices shall also be included in the list of factors that can introduce error. The imortance of POD in the NDT industry could be seen trough the increasing number of aers wrote in the last years as shown in Figure (). of 4 rev

2 Figure. (Courtesy of Jochen Kurz, Fraunhofer Institute for Non-Destructive Testing IZFP ) (). Some industry sectors as art of Nuclear, offshore oil and gas and Aeronautics consider POD as crucial, including the fact that POD will have an incidence in Risk (4) and of Failure (5). The resent aer, shows some techniques and examles to have an estimate of POD from binomial oint of view. It is recognized that the binomial aroach is far from otimum method but simlicity of calculations make it an attractive choice. This is the technique indicated in ASME BPVC Section V, Art.4 ().. Foundations If an exeriment is erformed with trials that are indeendent each other with only two ossible outcomes (failure or success for examle) with robability of success, the henomena can be described by a binomial distribution: Bin(n, ) (E.) where is the actual roortion of success of the oulation. The roortion of fails and success follow the rule: Where q is the roortion of fails. + q = (Ec. ) Known, the robability to have X success in n trials is: (x = X) = q ( ) (Ec. ) of 4 rev

3 But the unknown quantity is, in the NDT case, the roortion of ositive detections in our system, including all factors that could affect the test: environment, rocedure, equiment, oerators and the comonent under evaluation. In the classical examle the roblem is described by cube lenty of green and red ales. If one need to know the roortion of red ales, we can take all the ales and count. This couldn t be done if the oulations is too big. In this case, the method is take a samle of the oulation and calculate the samle roortion instead. If n trials where done with k successess, i.e. if a welded late with n = defects is examinated and X = 8 defects where detected a rough estimate of success roortion (the POD) of the system is: = (E. 4) = 8 =.8 8% The first exeriment seems to indicate that the POD could be 8%. If a new exeriment is erformed under the same conditions of revious one, it is ossible that the "new" samle roortion, the new score will be 6%. If the exeriment is reeated many times, is ossible that different samle roortions will result each case. So, a conservative rocedure is needed to fix a minimum POD from one exeriment. The Statistical way to deal with the revious situation is through the use of confidence intervals. In the resent case, the confidence means that if the test is reeated a certain number of times, we have a ercentage of guarantee (the confidence) that the arameter of interest is inside the interval; i.e. if the confidence is 95% and the calculated lower bound POD is 9%, we can be sure that in 95 of times the POD will be 9% or greater, as resented in Figure % Confidence Figure. 95% Confidence uer Interval. of 4 rev

4 As is unknow, it can be inferred with reasonable statistic suort that (5) : L( ) < < U( ) (Ec. 5) In other words, if ossible to create the interval from the test which roduced. For this interval L( ) is the lower bound and U( ) is the uer bound of the interval. After the interval has been built the lower bound L( ) can be calculated an can be used as a good estimate of minimum ossible POD for a confidence level. Most times a POD of 9% is required with a confidence of 95%. The acceted nomenclature for this value is POD /. This numbers seems to be selected from an arbitrary basis, nevertheless, some recent works connects the POD with failure robability and risk based insection hilosohy to roduce secific requirements for POD (4) & (5). In accordance with the revious discussion, for ractical urose the POD would be the lower bound L( ) of the confidence interval:. Confidence interval L( ) < (E. 6) The confidence is equivalent to the area below the distribution function. If 95% confidence is required, this means that the interval should be defined by 95% the area below the curve (δ =.95) as shown in Figure. Once the confidence has been fixed, the next ste is to find the lower limit k = L( ) d=.95 k Figure. Lower bound confidence interval. In Figure, an examle of the Beta Distribution Function is resented. The lower bound limit k, has the following relathionshi with δ (the confidence coefficient) (5) : 4 of 4 rev

5 δ = Beta(, a, b)d (E.7) Exression (E.7) could be used to calculate the confidence coefficient δ from k = L( ) or vice versa (which would be the most common situation). Different software could be used. Annex A resent a MATLAB scrit which let us find the confidence δ (Confidence = δx) from k = L( ). 4. Bayesian Aroach In the Bayesian aroach, revious knowledge of an asect related with the henomenon, could be used to find a robability of occurrence; i.e. If we know the number of ice creams that was sold in a sunny Sunday, we can estimate if a secific day was a sunny Sunday from the number of ice creams sold that day. In the Bayesian oint of view, is a random variable described by a robability distribution function, which is subjective and based on the exerimenter "revious" knowledge. In that way, if the function is known, it is ossible to fix a Credible Set (the Confidence Interval concet alied in the Bayesian framework) and find his lower bound limit using (E.7). Once the exerimenter select a revious distribution for, the exeriment is conducted and revious distribution is corrected taking in consideration the exeriment results to roduce a osterior distribution. In our case, osterior distribution could be the same tye of function than revious one, but with new arameters. These can be written in the following way (5) : π(θ x) = ( ) ( ) ( ) ( ) (E. 8) Where π(θ) is the revious distribution of the random variable θ, π(θ x) is the osterior distribution under the new evidence x, and f(x θ) is the samle distribution. Aendix of reference (5) resents a clear exlanation of the Bayesian Aroach, and the use of a Beta distribution to reresent the random variable. The Beta distribution use the Beta function with arameters a and b which is also built from Gamma Function as shown below: Where Γ is the gamma function. Beta(, a, b) = ( ) ( ) ( ) ( ) (E. 9) NOTE : In the resent aer the notation Beta(, a, b) is referred instead Beta(x, a, b) or Beta(a, b) to make emhasis in the fact that is a Beta distribution of the random variable. A relevant asect is that Beta distribution is the conjugate of the Binomial distribution. In that way, the denominator of (E.9) can be resolved analytically (5). The easy of calculation introduced by this fact is one of the factors that make The Beta distribution a referred Distribution Function for. Another relevant oint is that Beta is defined in the interval [,]. 5 of 4 rev

6 In the Figure 4, the Beta(,,) distribution function is resented. 8 robability Figure 4. Beta(,,) Distribution function. The Beta Distribution function shae is configured by the arameters a and b. If a = and b =, the Beta distribution becomes the uniform distribution as can be observed in Figure 5. robability Figure 5. Uniform distribution, Beta(,,). One of the most common aroximations is to select Beta(,,), the uniform distribution as the revious distribution. Beta(,,) (Ec. ) This is equivalent to say that the exerimenter doesn't have any reference non revious knowledge of POD, all robabilities are equally ossible. A very areciated roerty of Beta Distribution is that the new distribution arameters are easily obtained from test result following the next exressions: a = a + N (E. ) b = b + (N N ) (E.) Where N is the number of detected defects and N is the total number of defects included in the test. 6 of 4 rev

7 The osterior distribution is: Beta(, a, b ) (E. ) If a =, b =, (E. 4) becomes: Beta[, ( + N, + (N N )] (E.4) The new a arameter is the old a arameter lus the number of successes (detected defects). The new b arameter is the old b arameter lus the number of fails (non detected defects). Some examles are resented below: Examle : if we have a welded late with searated and similar defects (over critical size and with similar size each). We would like to calculate the POD of an insection team using a well tested rocedure and otimal equiment. After the insection has been done, the team successfully detect 8 of defects. So, N = and N = 8. We would like to find L( ) the lower bound limit. The osterior distribution in accordance with (E.4) after the test will be: This distribution is shown in Figure 6. Beta(, 9,).5.5 robability Figure 6. Beta(,9,) Distribution. We need to fix the confidence required for the Test. Suose 9% as a confidence required. The lower bound value is shown in Figure 7. Under the Bayesian aroach the Credible Set lower bound is L( ) =.58 for 9% confidence. This means that if the test is reeated times, 9% of the times.58. Under this conditions our POD could be reresented by this value. 7 of 4 rev

8 Although the score =.8 reresent only one exeriment and could not be used to evaluate the erformance of the insection team most art of times. The lower bound value can be obtained through recognized PC statistics software even using free mobile devices alications (6) k= Figure 7. Lower bound limit for a confidence of.9. Examle Suose the same situation that Examle but the confidence required is now 95%. Posterior distribution is still: Beta(, 9,) Figure 8 shows the Credible Set (Confidence Interval) and the lower bound limit k= Figure 8. Lower bound limit for a confidence of.95. Now: L( ) =.5. 8 of 4 rev

9 5 ASME BPVC Art. 4 () POD Lower Bound ASME code seems to use the Clooer Pearson aroach also called the exact interval (8). Paragrah below ASME notation is used. The code resents the following exression referred to the lower bound confidence interval: α = β( ; D, N D + ) (E.5) The exression seems to be easily comressible in the following form: α = Beta(, D, (N D) + )d (E.6) Where is the lower bound, D reresent the successes (defects detected) and N is the total number of trials (the total number of defects) and α is the confidence coefficient. The oint of view of ASME is close as resented above (E.7). The main difference is the osterior arameter a, which is exactly equal to the number of successes a = N. The other arameter remains the same b = (N N ) +. The osterior distribution after the test will be: Beta(, N, (N N ) + ) (E.7) The (E.7) roduce the same Interval as the Clooer Pearson "Exact" interval as can be seen in the reference (8). More examles are resented below: Examle. Suose the same conditions as the Examle including the same confidence of 9%, N = and N = 8. The osterior Beta distribution in accordance with (E.7) will be Beta(, 8,), as shown in Figure robability % Area.5.5 k= Figure 9. (left) Beta(,8,) Distribution Function. (right) 9% confidence interval and lower bound. In this case L( ) =.55 (as can be seen in one of the ASME examles) of 4 rev

10 Examle 4. Suose the same conditions as Examle, but the confidence is increased to 95%. In this case the new lower bound will be L( ) =.49 as can be seen in Figure % Area.5 k= Figure. Beta(,8,) Distribution for 95% confidence interval and lower bound. This results differs from examle shown in ASME BPVC Sec V. Art. 4 (). of 4 rev

11 Examle 5. For the resent case suose the following values: Total number of defects in the test N =. Defects detected N =, the insection team detects all the defects. The lower bound is required for a confidence of 9%,95% and 99%. According to (E.7) the distribution will be: Beta(,,) This Beta distribution roduces the following results: Confidence Lower Bound 9%.79 95%.74 99%.6 As shown in Figure (i), (ii) and (iii) Confidence=95% 6 Confidence=9% 4 k=.74 4 k= (i) Beta (,,) Distribution with a 9% Confidence Interval. (ii) Beta (,,) Distribution with a 95% Confidence Interval. 8 6 Confidence=99% 4 k= (iii) Beta (,,) Distribution with a 99% Confidence Interval. Figure. Beta (,,) Distribution with 9%, 95% and 99% Confidence Intervals. of 4 rev

12 The revious examle shows a very relevant characteristic of the binomial aroach and the reason why some references (including ASME and ENIQ documents) establish that this aroach is far from otimum. As can be seen, even detecting all the defects included in the test, it is not ossible to guarantee a high POD with a high confidence. If higher POD is requested the total number of defects used in the exeriment should be increased. If a POD / is required, the number of defects shall be increased to a minimum of 9, and the insection team need to detect all 9 defects. This is known as the 9 of 9 rule for the binomial aroach. 6 Some relevant asects regard the Binomial aroach resented in ASME BPVC Art. 4 () One of the drawbacks of the binomial aroach is that all defects should have a very similar size and morhology. If the defects are of different sizes they has different robabilities to be detected and the foundations of binomial aroach are not valid. Considerations about defect orientation and location could be also an issue. If a "Size vs. POD" curve is needed, the test should be reeated for different grous of defects sizes. This increase amazingly the number of defects required. ASME gives the alternative to use the same lates with defects many times in a blind fashion, but this aroach has a well suorted criticism (see reference () MIL-HDBK-8A (9), age 6 oint 6.4). The most critical limitation of Binomial Test is the number of defects required to achieve the common POD /. As Examle 5 shows, 9 of 9 defects shall be detected to achieve POD /. If the insection team have only one miss a 9% POD could not be guaranteed and the lower bound will be L( ) =.84 as can be seen in Figure. Cumulative Beta Distribution Function (Confidence) :.84 alfa: Figure. Confidence coefficients for Distribution Function Beta(,8,), 8 detected from 9 defects resented in the test. of 4 rev

13 If during the Test design, the exerimenter assumes that maybe defects would not be detected by the insection team, 6 defects are needed to achieve POD / as can be seen in Table T- 47. de ASME BPVC Sec V. Art. 4 (). One very imortant oint, not frequently mentioned is that if the total number of defects is reduced, the erformance of good insection teams could be worse than if the number of defects is large. 7 Use of F-Distribution One simle and fast aroach is the use of the lower bound interval (LCL) mentioned in the reference (9). In this case the lower bound could be obtained using: LCL = ( ) (, ) (E.9) Where F (f, f ) comes from the F-Distribution with coefficient of confidence α and f y f the distribution arameters, calculated using: f = (N N ) (E.) f = N 8 Conclusions The Binomial aroach is an easy way to calculate the POD but has some relevant drawbacks. If the confidence and POD should be high, the number of samles should be high with direct imact in the test cost. If the a Size vs. POD curve should be required, the test should be reeated for different grous of defects sizes. 9 Acknowledgments Secially to Ed Ginzel (Materials Research Institute, Canada) for sharing the Cloer Pearson Sread Sheet file and his constant and dedicated suort for all questions. Claudia Antonini (Universidad Simón Bolívar, Venezuela) for his suort regard the Beta Binomial aroach, Jochen Kurz (Fraunhofer Institute for Non-Destructive Testing IZFP, Germany) for share Figure and Matthew Bognar (University of Iowa, USA) for the mobile alet and the online Distribution alication. of 4 rev

14 Aendix. Confidence Associated with a lower bound value. % Confidence calculation given the lower Bound close all; clear all; clc; x=:.:; e=8;% Successes f=;% Fails alfa=e; beta=f+; %Beta Distribution Grah y=betadf(x,alfa,beta); figure(); lot(x,y,'color','r','linewidth',);xlabel('');ylabel('robability') low=.49 %POD roosed Lower bound Confidence=-betacdf(low,alfa,beta) %Confidence associated with low % Carlos Correia - Jan 5 9 Bibliograhy. MIL-HDBK-8A. NONDESTRUCTIVE EVALUATION SYSTEM RELIABILITY ASSESMENT. htt:// Ginzel E., Introduction to Statistics, htt:// Jochen H. KURZ., Anne JÜNGERT., Sandra DUGAN., Gerd DOBMANN. of Detection (POD) determination using ultrasound hased array for considering NDT in robabilistic damage assessments. 8th World Conference on Nondestructive Testing, 6- Aril, Durban, South Africa. 4. O. Cronvall, K. Simola et all. A study on the effect of flaw detection robability assumtions on risk reduction achieved by non-destructive insection, Reliability Engineering and System Safety, 5 (), J. Garza, H. Millwater, Sensitivity of robability-of-failure estimates with resect to robability of detection curve arameters, International Journal of Pressure Vessels and Piing, 9 () L. Gandossi, K. Simola, A BAYESIAN FRAMEWORK FOR THE QUANTITATIVE MODELLING OF THE ENIQ METHODOLOGY FOR QUALIFICATION OF NON-DESTRUCTIVE TESTING. Aendix. Institute for Energy, EUR 675, htts://lay.google.com/store/as/details?id=com.mbognar.robdist. Matthew Bognar. 8. THULIN M., The cost of using exact confidence intervals for a binomial roortion. arxiv:.88v. Mar. 9. Generazio E., DIRECTED DESIGN OF EXPERIMENTS FOR VALIDATING PROBABILITY OF DETECTION CAPABILITY OF NDE SYSTEMS (DOEPOD). NASA, Hamton, VA of 4 rev