IEC Reliability testing Compliance test plans for success ratio. Proposal for review of sequential tests

Transcription

1 The Standard Institution of Israel Technion Israel Institute of Technology IEC Reliability testing Compliance test plans for success ratio Proposal for review of sequential tests המלצה לשיפור תקן IEC תכניות לבדיקת תאימות לשיעור הצלחה Dr. Yefim Haim MICHLIN Senior Research Fellow, Adjunct Associate Professor Faculty of Industrial Engineering & Management, Technion. Nov 2015 Ofer SHAHAM M.Sc., Chief Research Engineer Microelectronics Directorate RD&E Division, Rafael.

2 1. Introduction Standard s edition The Standard includes two main test plans: Sequential Updated knowledge and methods. Fixed trial/failure Its development has been exhausted and nothing to renew. Nov 2015 IEC Proposal Updated sequential test plans 2

3 1. Introduction Standard s actuality and motivation IEC Reliability testing Compliance test plans for success ratio. Chapter 8 and Annex A are dedicated to sequential testing. Sequential tests have significant importance and actuality in reliability and quality assurance. It enables decision-making relying on the smallest average sample number versus any other test. The standard was not revised for many years and in the field of sequential tests it is not compatible with the latest knowledge methods and techniques. In recent years, studies in this area were conducted in the Technion, enabling better planning of the tests, according to the requirements of the industry today. The studies have been published in many professional journals and conferences and scrutinized by experts. Nov 2015 IEC Proposal Updated sequential test plans 3

4 2. SPRT Assumptions & hypothesis Let X 1, X 2, be independent and identically distributed binomial variables assuming values of failure with a constant and unknown probability p and those of success with probability 1-p. The hypothesis H 0 is examined vs. the alternative H 1 : where P a (p) is the probability of accepting H 0 ; test s Operational Characteristics (OC) function; α and β are the Type - I and - II error probabilities; p 1 is the Rejected Quality Limit: p 1 =p 0 d; d >1 is the discrimination ratio. H 0 : p p0 Pa p0 1 H: 1 p p0 Pa p1 Sequential test - the outcome of each observation is one of three decisions: To accept the H 0 hypothesis and stop the test; To reject H 0, prefer H 1 as an alternative, and stop the test; Further observation is required (not enough information for either decision). Nov 2015 IEC Proposal Updated sequential test plans 4

5 Number of failures, r 2. SPRT Description of the test (a) r t TA n t TA f TA (b) h r h a n s =n t -r t +r Number of trials, n s Accept Line Reject Line Number of failures, f Continue zone TA s Accept Reject test points ADP RDP centerline Number of successes, s Fig. 1 - Typical test plane ADP Accept Decision Point RDP Reject Decision Point The parameters h a and h r are adjusted so that the real values of α and β (α real and β real ) will be close to the required ones - α tg and β tg. Nov 2015 IEC Proposal Updated sequential test plans 5

6 Operational Characteristics, P a (p) 2. SPRT Test characteristics α (a) OC Operational Characteristics. ASN Average Sample Number. ENT Expected Number of Trials to decision ASN ENT β 0.0 p0 D p p 0 0 D p1 p1 True faliure ratio, p (logarithmic scale) D Fig. 2 - Typical Operational Characteristics and ASN (ENT) of binomial test (b) Nov 2015 IEC Proposal Updated sequential test plans 6

7 Probability of hitting any point within the boundaries: Probability P a (p) obtained for given test boundaries, summing up those of hitting all ADP, p ADP (f,p): The real α, β values of the truncated test (α real,β real ) are calculated as: TA 1 TA 1 r 1 P p 1 p f, p P p p f, p real a 0 ADP 0 real a 1 ADP 1 f0 f0 Nov 2015 IEC Proposal Updated sequential test plans 7 r where ADP(f) is the s-coordinate of the ADP for given f; P ADP (f,p) the probability of hitting the above at given p. 2. SPRT Characteristics calculation for given boundaries direct method 1 P p p P p p P p s, f s, f 1 s1, f The test ASN is calculated as:, 1 P f p p P p ADP s1, f RDP ADP ASN ( p ) s RDP s P s, p f ADP f P f, p s The method permits calculation of the test s α real, β real, ASN, and OC. f

8 2. SPRT Truncation and quality features OC closeness The real OC vs. the target. Measured by R D - type I and II error probabilities (α real, β real ) gap of truncated test and their targets (α tg, β tg ): R 2 2 R D real tg tg real tg tg Dtg Degree of truncation The test does not continue beyond maximal failure and succsess numbers. Truncation causes change of the OC and ASN Nov 2015 IEC Proposal Updated sequential test plans 8

9 2. SPRT Truncation and quality features (cont.) Test efficacy Truncation increases the ASN. ASN ntr for comparing truncated and non-truncated test. ASN(p i ) constructed for real and real on the same p i values. R ASN for measuring the test efficacy for a given OC. ASN ENT R ASN 5 5 i ntr i i1 i1 5 i1 ASN p ASN p ASN ntr p i where: p i =p 0 d (i-2)/2 ; ASN(p i ) by direct method; ASN ntr (p i ) by Wald s analytic formula. At 5% <R ASN < 15%, the ASN increases insignificantly, whereas the maximum SN is drastically reduced. The optimal test is that with the heaviest possible truncation among those with the required R D and R ASN. Nov 2015 IEC Proposal Updated sequential test plans 9

10 2. SPRT Judicious region of TA and R ASN 0.3 (a) TA f TA R ASN Truncation Apex failure coordinate (TA f ) TA on CL Number of failures, f Continue zone TA s Number of successes, s (b) Accept Reject test points ADP RDP centerline Fig. 3 - R ASN reachable potential Optimal test TA location - the tests with the heaviest truncation, while standing R D and R ASN limitations, are where the TA falls on the CL, and is closest to the origin. Nov 2015 IEC Proposal Updated sequential test plans 10

11 Number of Trials 2. SPRT Optimal test example p 0 p Failure Ratio, p ENT of IEC test ENT of proposed test Truncation of IEC test Truncation of proposed test Fig. 4 - Number of trials characteristics of IEC and the proposed test Accept Line of IEC test Reject Line of IEC test Accept Line of proposed test Reject Line of proposed test Fig. 5. The plane of the same tests. Nov 2015 IEC Proposal Updated sequential test plans 11

12 3. Sequential tests in the current standard 1. The standard relates to any value of the test s parameters (p 0,,, D): o Part of the tests are pre-planned and are ready to use and presented in a table. o Other tests should be planned according to formulas and recommendations. 2. The ready to use tests: o Are not significantly truncated (the maximum sample number is too high) o Are limited to higher than required probabilities and risks. o Are limited to equal risks for the supplier and the customer (the real values are not available and sometimes much differ from the defined) o The average sample number is not available, it can be calculated with significant error. 3. The calculated tests: o The parameters are calculated by Wald s formulas for the non-truncated test o Are not accurate Nov 2015 IEC Proposal Updated sequential test plans 12

13 4. Sequential tests in the proposal Scope 1. The proposal includes tables of many pre-planned tests that are ready to use and a method for planning additional tests. 2. The proposal is an improvement versus the current one in the field of SPRT (Chapter 8 and Annex A): o o o o o o The tests are significantly truncated. The risks are very close to the target. The range of the test is wider (by means of success ratio and risk values). The ready to use tests include tests with unequal risks (). The values of ENT is available in the table. Simple method for planning additional test (out of the table) is given. 3. These improvements will reduce the sample number and the mistakes in the decisions, leading to reducing the production cost. Nov 2015 IEC Proposal Updated sequential test plans 13

14 4. Sequential tests in the proposal Tests table (fragment) Range of the test parameters: p 0-0.2, 0.1, 0.05, 0.02, 0.01, 0.005, 0.002, D - 1.5, 1.75, 2, 3, 5 Nominal risks: α β Nov 2015 IEC Proposal Updated sequential test plans 14

15 4. Sequential tests in the proposal Advantages/comparison p 0 =1-q 0 RMSRE Root Mean Square Relative error of, n t number of trials (sample number) at truncation Nov 2015 IEC Proposal Updated sequential test plans 15

16 4. Sequential tests in the proposal Advantage summary No. Subject Current standard Proposal Proposal advantage 1 Truncation Soft/slight Hard/ significant The tests are shorter and lower cost 2 Risk variety in the ready to use tests 4 combinations of equal risks (5% 10% 20% 30%) 17 combinations High variety of the given tests, including small and unequal values of risks 3 Real/true risks Unknown Given Accurate decisionmaking 4 Average sample number/size Needs to be calculated Given Significant help in choosing the appropriate test 5 Planning of additional tests Complex not accurate Simple and accurate Expanding the variety of tests Nov 2015 IEC Proposal Updated sequential test plans 16

17 5. Conclusion The example of the proposed methodology vs. binomial SPRT in IEC demonstrates its advantages, and the appropriate required changes in the standard. These advantages will extend the use of SPRT and this standard. Recently, the Standard Institution of Israel is concluding a process of adopting IEC including a new proposed Annex (a national deviation). This Annex is a mature proposal for the revision of the international standard. Given all of the above, it is recommended that IEC be revised in accordance with the updated knowledge methodology and tools. Nov 2015 IEC Proposal Updated sequential test plans 17

18 6. Applicable papers of the authors & References Michlin, Y. H. and Shaham, O. Planning of Truncated Sequential Binomial Test via Relative Efficiency. Quality and Reliability Engineering Int., vol. 29 pp Michlin, Y. H., Meshkov, L. and Grunin, I., Improvement of "Sequential Testing" sections of MIL-HDBK-781A and IEC 61124, IEEE Transactions on Reliability, Vol. 57, No. 2, June 2008, pp Michlin, Y. H. and Shaham, O., Comment on the Paper "Closed Sequential and Multistage Inference on Binary Responses with or without Replacement" by Ignatova, Deutsch, and Edwards, The American Statistician, Vol. 68, n. 2, pp.128, 2014 Michlin, Y. H.; Shaham, O.; Lumelskii, Ya. P.; "Substantiation of sequential test parameters for mass-produced electronic devices," Electrical & Electronics Engineers in Israel (IEEEI), 2012 IEEE 27th Convention of, vol., no., pp.1-5, Michlin, Y. H., Ingman, D., Levin-David, L.; "Sequential Test for Reliability Under Allowance for Target Uncertainty," Reliability, IEEE Transactions on, vol.61, no.4, pp , Michlin, Y. H., Kaplunov, V., Ingman, D., Sequential testing for two exponential distributions at arbitrary risks. Int. J. of Quality and Reliability Management, 2012, Vol. 29 No 4, pp Michlin, Y. H., Grabarnik, G., Search boundaries of truncated discrete sequential test, Journal of Applied Statistics, Vol. 37, No. 05, 2010, pp Michlin, Y. H., Grabarnik, G., Leshchenko, E., Comparison of the mean time between failures for two systems under short tests, IEEE Transactions on Reliability, Vol. 58, No. 4, Dec. 2009, pp Michlin, Y. H., Grabarnik G., Sequential testing for comparison of the mean time between failures for two systems, IEEE Transact. on Reliability, Vol. 56, No. 2, 2007, pp References Wald A Sequential Analysis, John Wiley & Sons: New York. GOST R , Dependability technics Compliance test plans for mean operating time to failure or between failures Part 1: Exponential case. Aroian, L. A Sequential analysis-direct method, Technometrics, Vol. 10, pp R. Kenett, Sh. Zacks, and D. Amberti, Modern Industrial Statistics: with applications in R, MINITAB and JMP. John Wiley & Sons, Ltd., 2014 MIL-HDBK-781A, Reliability test methods, plans, and environments for engineering, development, qualification, and production, US DOD, 1996, pp Nov 2015 IEC Proposal Updated sequential test plans 18