Bayesan Plannng of Ht-Mss Inspecton Tests Yew-Meng Koh a and Wllam Q Meeker a a Center for Nondestructve Evaluaton, Department of Statstcs, Iowa State Unversty, Ames, Iowa 5000 Abstract Although some useful general gudelnes est for plannng nondestructve nspecton studes (eg, n MIK- HDBK 83A), statstcal tools provde more defntve gudance and allow comparson among dfferent proposed study plans It s possble to obtan epressons for estmaton precson (eg, gvng the relatve wdth of a confdence nterval), provdng an assessment and comparson of alternatve test plans One problem s that estmaton precson depends on the unknown actual underlyng POD functon Engneers generally have some nformaton about the true POD functon, based on some combnaton of knowledge of the physcs behnd the nspecton method or prevous eperence wth the nspecton method If such uncertan nformaton can be descrbed by a probablty dstrbuton, t s natural to use a Bayesan method to do the test plannng In ths paper we present Bayesan methods to fnd optmum test plans Although the optmum plans have practcal defcences, they provde nsght for developng statstcally effcent compromse plans that are also developed n our work Keywords: Bnary Regresson; Nondestructve Inspecton; Probablty of Detecton; Optmum Test Plan PACS: 050-r, 870 INTRODUCTION Background and Motvaton Nondestructve evaluaton s wdely used to determne the n-servce status of components or the qualty of raw materals and components wthn manufacturng processes Probablty of detecton (POD) s an mportant nspectoncapablty metrc Usually, POD s estmated on the bass of a POD study, n whch a collecton of specmens contanng flaws (eg, cracks) of varyng szes s nspected Commonly asked questons are: How many specmens are needed? and What sze cracks should be used? The answers to these questons depend on several factors, ncludng sources and amounts of varablty n the nspecton process and the degree of precson needed for the POD estmate Related Work Some general gudelnes for desgnng ht-mss POD studes are gven on page 4, Secton 45 of [] The study n [] looks at the effect of sample sze n the desgn of ht-mss POD studes They use smulaton studes to demonstrate some conclusons regardng sample sze choce n these desgns Both these presentatons use the classcal approach that assumes knowledge of the model parameter values In [3] the authors proposed a Bayesan approach for optmum desgn wth a logstc regresson model, usng a pror dstrbuton n place of parameter plannng values In ths paper, we also use a Bayesan approach n a more general bnary regresson model that would also allow the use of pror nformaton for nference We focus partcularly on the mportant test-plannng problem of desgnng a htmss nondestructve nspecton to estmate the probablty of detecton and compare such plans wth more practcal optmzed compromse test plans HIT-MISS RESPONSE LOGISTIC REGRESSION MODEL AND LIKELIHOOD ESTIMATION Let Y denote a bnary random varable, wth
We let for a ht Y = 0 for a mss = log(crack length) Then the logstc bnary regresson model s where logs POD( ) = Pr( Y = ; ) = ( ) ep( ) 0 logs 0 0 ( z ) ep( z ) /[ ep( z )] Ths model can also be epressed as [POD( )] = logs 0 ep( ) where the functon ( p ) = log( p / ( p )) The (possbly transformed) flaw sze that wll be detected wth logs probablty p ( 0 < p <) s = ( ( p) ) / For eample, f a log transformaton s used on the flaw [ p] logs 0 sze then a90 ep( [090] ) s the flaw sze that can be detected wth probablty 090 Generally, the parameters 0 and are unknown and need to be estmated from data Suppose we have k dstnct flaw szes and specmen s nspected once would provde j =,, n Then the lkelhood can be epressed as Data Setup and Lkelhood n flaws at each sze n = n nk bnary responses ( j y 0 j k n e 0 = j= 0 0 y j L(, ) = e e The mamum lkelhood estmates are the values of and that mamze () 0 PRIOR DISTRIBUTION SPECIFICATION Test plannng requres nput nformaton about the model parameters The log flawsze where POD s equal to 05 (known as 50 Data from a POD study n whch each Y ), where =,, k and log( a ) ), and The slope parameter where we can nterpret / as the ncrease n log flawsze when POD s ncreased from 05 to 073 Note that = (05) = 0 logs Informaton about would be more readly avalable as compared to 0 and the elements of (, ), unlke those of ( 0, ), would be epected to be appromately ndependent, makng t easer to specfy pror nformaton, by specfyng only margnal dstrbutons It should be noted that because our formulaton allows pror nformaton to be used n both the test plannng and nference stages, we wll allow for the possblty of usng two potentally dfferent pror dstrbuton specfcatons for these purposes (e a plannng pror and an nferental pror) Ths s justfed because those who plan and conduct the POD eperment have dfferent rsk functons than those who are subject to the effects of nference from the results of the study See Secton IIIC of [5] and references provded there for more dscusson of ths pont ()
that Inferental Pror Dstrbutons ' ' We denote the varance-covarance matr of the nferental pror dstrbuton of =, by S We fnd Var( ) Cov(, ) Var( ) 0 S = Var = = Cov(, ) Var( ) 0 Var( ) () Equvalently, the precson matr of the nferental pror dstrbuton s gven by nferental pror s desred, the matr S wll be set to be the zero matr Plannng Pror Dstrbutons on and S We note that f a dffuse As mentoned prevously, the amount of nformaton one has about the parameters s reflected n the spread of the pror dstrbutons whch are assgned to the parameters We note, however, that the plannng prors cannot be too dffuse as that can cause nstabltes n the optmzaton process For parameter we use a normal dstrbuton pror specfed by where N(, ) s the mean and s the varance For we use a lognormal pror specfed by Lognormal(, ), where aand are the mean and varance, respectvely, of the logs of In our eamples, we wll nvestgate the effect of usng dfferent pror dstrbutons BAYESIAN TEST PLANS The objectve of ths secton s to show how to fnd good POD test plans We would lke to determne the number of levels k at whch specmens are placed and how many specmens are placed at each level (e, the n values for =,, k) to optmze some partcular estmaton crteron We note that optmum test plans tend not to be practcal However, optmum test plans are valuable because they provde nsght nto how to plan a practcal compromse test plan wth good statstcal propertes Optmzed compromse plans can be obtaned by usng constrants to make a test plan practcal Also, optmum test plans suggest useful test-plannng heurstcs Optmzaton Crteron A reasonable crteron for a good test plan s one that mnmzes the varablty of the estmator of [ p], the (log) flawsze whch s detected wth probablty p Usng the delta method, the appromate posteror varance of [ p] can be epressed as ' Var cvar ( ) c (3) ˆ[ p] t, D where Var, t D( ) s the varance-covarance matr of the posteror dstrbuton of, gven the desgn D and the data t and ' c =, logs ( p) / Thus, a reasonable crteron would be to choose a desgn whch mnmzes the epected value of (3) over all possble data That s, to choose D that mnmzes ' C( D) = E D Var, D( ) t c t c (4) Computng CD ( ) s epensve, but when when the total number of specmens n s large (say 60 or more), ' c S I c, (5) C( D) ( D) f ( ) d
provdes an ecellent appromaton Here S s the varance-covarance matr of the nferental pror dstrbuton for and I ( ) ( ) E E ( D) = ( ) ( ) E E s the Fsher nformaton matr, and f ( ) s the plannng pror dstrbuton See [4] and [5] for other, applcatons of ths approach The Optmum Test Plan Denote the total number of specmens by n and the total number of flawszes at whch specmens wll be placed by k For each value of k, we need to determne the dfferent flawszes, and the proporton of specmens placed at each of the k flawszes that wll mnmze (5) The search for the optmum desgn uses the followng algorthm: Choose a value of n (total sample sze) Begn wth k = levels The optmum flawszes [ k=] proportons and by and and the correspondng allocaton [ k=] [ k=] [ k=] are determned We denote the vector of optmum flawszes [ k =] [ k =], [ k=] and the vector of optmum allocatons [ k =] [ k =], by [ k=] 3 Ths process s repeated for larger values of k ( k = 3,4, ) 4 For a partcular value k = K ( K ), f t s found that CD ( ) [gven by (5)] evaluated at the vector of optmum flawszes at the vector of optmum flawszes stopped [ k= K] and optmum allocaton proportons 5 The optmum number of levels n the desgn s then taken to be [ = ] optmum allocaton proportons k K * [ k= K] [ k= K] and optmum allocaton proportons k * = s larger than CD ( ) evaluated [ k= K] K wth optmum flawszes, the process s [ k * = K] and We note that the problem of fndng the vectors and for the desgn whch mnmzes (5) s a constraned optmzaton problem wth constrants: < < < k k = = 0 for =,, k A Compromse Test Plan As mentoned earler, optmum plans provde nsght that are useful for developng heurstcs and other means of determnng compromse plans that meet practcal constrants and have good statstcal propertes For our compromse plans, we use the followng addtonal constrants: The number of levels k s fed
The specmens are equally spaced so that on the log flawsze scale We wll fnd the flawszes < < < k that mnmze (5) by fndng the smallest flawsze at whch specmens should be placed the equal spacng between any two consecutve flawszes Eamples Usng Dfferent Plannng Pror Dstrbutons Ths secton presents eamples usng fve dfferent combnatons of pror dstrbutons for the model parameters, shown n Table For each eample plannng pror dstrbuton, we found the true Optmum Plan and the optmzed Compromse Plan for estmatng both and [09] These pror dstrbutons are epressed graphcally n Fgure Pror dstrbuton on 05 Weakly Informatve Moderately Informatve Pror dstrbuton on γ Moderately Informatve Weakly Informatve FIGURE Mean POD (sold lne) and bands showng the lower 5% and upper 5% quantles of the pror POD dstrbuton as a functon of log crack sze Fgures through 6 show the Optmum and Compromse Plans POD for estmatng (on the left) and [09] (on the rght) To epress the plan n a scale-free manner, the locaton of the k flaws s specfed n the fgures as the POD at the gven crack sze, computed as POD( ) = logs of the R = ep z Var ˆ / [ p ] The precson of each plan s gven n terms precson factor, whch provdes a untless measure of precson of the confdence
ntervals for [ p] A confdence nterval for [ p] would be / R, R [ p] [ p] and thus a value of R close to mples a narrow confdence nterval Fgure 7 shows the effect of ncreasng the number of crack szes k and justfes our choce of k 60 n the comparson eamples TABLE Plannng pror dstrbutons for the eamples (mean = ) (mean = 7) Eample Dstrbuton SD Descrpton Dstrbuton SD Descrpton lognormal 08 weakly lognormal weakly lognormal 0 moderately lognormal weakly 3 lognormal 08 weakly lognormal 0 moderately 4 lognormal 0 moderately lognormal 0 moderately 5 lognormal 00 hghly lognormal 00 hghly FIGURE Test plans for pror dstrbutons that are weakly on both and FIGURE 3 Test plans for pror dstrbutons that are moderately on and weakly on
FIGURE 4 Test plans for pror dstrbutons that are weakly on and moderately on FIGURE 5 Test plans for pror dstrbutons that are moderately on both and FIGURE 6 Test plans for pror dstrbutons that are hghly on both and Some Observatons on Comparng Test Plans The optmum and compromse plans for estmatng (left-hand plots n Fgures -6) are all nearly symmetrc, as epected The optmum plans for estmatng (rght-hand plots n Fgures -6) are asymmetrc, [09]
suggestng that more precson can be obtaned for estmatng by usng more large flaws For the (unrealstc) [09] hghly pror dstrbutons (Eample 5), t s optmum to have only two flaw szes As the prors become more dffuse, the optmum number of flaw szes grows somewhat, up to fve flaw szes n our eamples However, as shown n Fgure 7, precson, although decreasng slghtly, remans good as the number of flaws grows wthout bound The 60-pont unform allocaton compromse plans perform ecellently for estmatng, but can be sub- optmal for estmatng, especally when the pror nformaton s dffuse, due to the unform allocaton [09] constrant Nevertheless, the unform-allocaton test plans generally perform well, relatve to the optmum plans FIGURE 7 R-factors for varous compromse plans The plotted ponts ndcate the value of k and the R precson factor for the respectve optmum plan for each eample ACKNOWLEDGEMENTS Ths materal s based upon work supported by the Ar Force Research Laboratory under Contract # FA8650-04-C-58 at Iowa State Unversty s Center for Nondestructve Evaluaton REFERENCES MIL-HDBK-83A Nondestructve evaluaton system relablty assessment Avalable onlne at http://wwwstatstcalengneerngcom/mh83/mil-hdbk-83a(009)pdf (009) Accessed 6 May 0 C Anns and L Gandoss, Influence of sample sze and other factors on ht/mss probablty of detecton curves, European Network for Inspecton and Qualfcaton Report, (47), 0 3 K Chaloner and K Larntz, Optmal bayesan desgn appled to logstc regresson eperments, Journal of Statstcal Plannng and Inference,, (989) 4 Y Sh and W Q Meeker, Bayesan methods for accelerated destructve degradaton test plannng, IEEE Transactons on Relablty, 6, 45-53 (0) 5 Y Zhang and W Q Meeker, Bayesan methods for plannng accelerated lfe tests, Technometrcs 48, 49 60 (006)