Model-Assisted Probability of Detection for Ultrasonic Structural Health Monitoring

4th European-American Workshop on Reliability of NDE - Th.2.A.2 Model-Assisted Probability of Detection for Ultrasonic Structural Health Monitoring Adam C. COBB and Jay FISHER, Southwest Research Institute, San Antonio, TX USA Jennifer E. MICHAELS, Georgia Institute of Technology, Atlanta, GA USA Abstract. Ultrasonic interrogation of metal alloys has been demonstrated to be effective for monitoring fatigue-induced damage in structural health monitoring (SHM) frameworks. However, traditional probability of detection (POD) approaches as used for nondestructive evaluation (NDE) are not directly applicable given the fixed nature of the ultrasonic probe(s). One difference is that there is a lack of variation in SHM sensor response due to human factors. Thus, the primary source of variability is related to the crack geometry itself. Another difference is that experimental derivations of POD curves for SHM are exceptionally burdensome. Since the probes are fixed, repeated inspections with the same probes and defect provide no additional information. Thus, given a particular ultrasonic SHM method, a model-driven methodology for POD may be necessary. This paper discusses the differences between SHM and NDE and the relationship to POD interpretation. Also, an SHM case study is presented for the development of POD curves using a model-assisted formulation. Variations associated with the structure state, such as the fatigue process, environmental effects and electronic noise, are approximated from experimental observations for inclusion in POD curve creation. The specific application analyzed is monitoring of fatigue cracks near fastener holes with an energy-based feature of through-transmission ultrasonic waveforms. www.ndt.net/index.php?id=8333 Introduction Probability of detection (POD) curves are routinely created for applications in nondestructive evaluation (NDE) to provide a measure of the efficacy of a given inspection technique. These curves ideally quantify the variability associated with all aspects of an inspection procedure, including human factors, for the inspection process. Thus, POD for NDE is a measure of the likelihood of detecting a defect each time the inspection procedure is performed. For structural health monitoring (SHM) applications, the interpretation of POD curves is not readily apparent, as the sources of variability not the same. Several authors have used POD curves in the analysis of SHM systems, but there are few discussions of how the differences in NDE and SHM require different interpretations [1,2]. An obvious difference with POD for NDE and SHM is that whereas sensors, instrumentation and operators are variable for NDE, all three parameters are fixed for SHM. Furthermore, for NDE, the sensors are moveable, resulting in even more opportunity for variability from inspection to inspection. There are many sources of variation for SHM, but they are primarily related to in situ effects, such as load and temperature variations. Further, other sources of variation for SHM are those that arise from structure-to-structure, and include differences in both structural geometry and sensor mounting. A meaningful POD curve for SHM should incorporate variability from all sources mentioned above. If variations from in situ effects are small, the POD curve captures the variability of the SHM system to crack geometry variations, structural differences and sensor mounting inconsistencies. Thus, SHM POD curves primarily describe the 1

percentage of possible defects of a given size that will be detected by a specific SHM system. The remainder of this paper is organized as follows: The procedure for conventional POD curve creation is explained, highlighting the difficulties for SHM POD creation. An ultrasonic SHM method and experimental data are presented, and POD curves are generated and discussed. Lastly, the summary and conclusions are given. Probability of Detection Analysis Procedure As an overview, POD curves can be based on the relationship between the crack size, a, and sensor response, â. Generally, the procedure for calculating the POD involves finding a functional relationship, preferably linear, between â and a such that the residuals are normally distributed with a constant variance [3]. Typically, this linear relationship is found in a log-log space; i.e., log 10 ( a ˆ )= c 1 log 10 ( a)+ c 0 + δ, where δ is a zero mean Gaussian random variable with variance σ δ, and c 1 and c 2 are given by the linear fit. In other words, for a given crack size a, the expected response log(â) is a random variable with mean equal to c 1 log 10 ( a)+ c 0 and standard deviation σ δ. Given a detection threshold, â THRES, the probability of detection, POD(â), can be readily computed from the standard normal cumulative distribution function, Φ( x), as 1 POD ( a) =Φ ( x) = e du x= 2π x ( ) + log ( ) 2 u c log a c a ˆ 2 1 10 0 10 THRES, where σ δ The challenge with applying this â vs. a strategy for SHM applications is in the determination of an appropriate value for â THRES. For NDE, this value would typically be determined from measurements where no flaws are present, and would be set above the background noise floor. It is generally infeasible to measure the noise floor from undamaged parts aside from known calibration specimens. For SHM, however, a sequence of measurements will presumably be available from the undamaged structure before damage occurs. Since the sensors are fixed in place, any initial variability in response is caused by in situ effects such as temperature, load changes, and sensor degradation. Furthermore, the degree of variation is related to the individual sensor installation. The variance in signal response as a function of time from the undamaged structure can thus be used to adaptively set the detection threshold for the monitoring system. Another difficulty with defining â THRES for SHM is the inherent dependence of detecting a crack on the previously recorded responses. For NDE, each measurement is independent, and could either detect or miss the same defect. For SHM, however, the measurements are of an evolving damage site. Once a defect is detected, all subsequent measurements should also indicate a defect. Thus, if there were a large degree of variability in the measurement responses, a response that would not normally indicate damage could be classified as damage because of the prior measurements. The net effect is that there is no practical way to define a fixed value for â THRES. Given the challenges of defining â THRES for â vs. a analysis strategies, a hit/miss approach is preferable for defining POD for SHM. Experimentally, there is a large amount of uncertainty in measuring the true crack size during most SHM applications as well as large costs associated with providing a statistically meaningful sample size. To 2

Location of cracks (size view) Figure 1. Illustration of beam path for monitoring the fatigue crack growth near the fastener hole. overcome these challenges, a model-assisted approach is proposed to generate the necessary information. The procedure is to create an empirical measurement response model that approximates the sensor response for a given defect. Next, a defect propagation model, such as theoretical corrosion rates or a crack growth equation, is used to generate a meaningful set of theoretical SHM sensor responses over time. Finally, a detection strategy that incorporates multiple sensor measurements determines when damage is detected. Conventional hit/miss analysis is performed on the resulting data. The remainder of this paper presents a case study for the development of POD curves for an SHM system. Case Study: Fatigue Crack Depth Sizing Using In Situ Ultrasonic Sensors 2.1 Experimental Overview The SHM system analyzed here monitors fatigue cracks originating from fastener holes using ultrasonic waveforms captured during fatiguing, and has been extensively described in the literature; e.g., [4-6]. A brief summary is given here. Ultrasonic signals were analyzed to calculate an energy-based parameter related to the degree of fatigue damage. The specimens considered were aluminium rectangular coupons with two holes of radius r in the center of the sample. The samples were fatigued by repeating a purely tension fatigue spectrum that was interrupted at regular intervals for ultrasonic measurements. The setup for ultrasonically monitoring the sample was to attach a pair of 10 MHz, 70 shear wave transducers in a through-transmission configuration on opposite sides of a hole with the direction of propagation perpendicular to the direction of crack growth. This transducer configuration maximizes the effect of the crack on the received ultrasonic signal. A sketch of the mounting configurations and beam path is shown in Figure 1. The feature used for data analysis is a ratio of signal energies from samples under tensile loading for opening any cracks to their unloaded counterparts normalized by the ratio from the undamaged specimen. It is thus a measure of the ultrasonic response to load caused by cracks opening. This normalized energy ratio, R(n), where n is the n th measurement, can then be examined versus successive measurements, and the indication of cracking is a drop in the response. An example energy ratio curve is shown in Figure 2. Small variations caused by in situ effects are evident during the initial flat portion of the curve. Figure 2. Example of normalized energy ratio curve. 3

Figure 3. Final crack depth versus sensor response for the 37 experiments [5]. Also shown is the linear fit in the log-log space. A series of fatigue tests were performed using this procedure as described in [5]. Figure 3 summaries the final crack size (depth) versus energy ratio response for each experiment, and is the same data shown in Table 1 of [5]. The crack sizes were determined by fracturing the specimens after the final ultrasonic measurement, photographing the crack surfaces, and measuring the crack dimensions from the high-resolution photographs. These experimental results will be used to develop the necessary measurement model for approximating the sensor response for a given defect size. 2.2 Detection Strategy The authors have developed an algorithm for detecting defects based upon a time series of the normalized energy ratio [6]. The basis of the crack detection algorithm is to compare the current energy ratio to a value predicted from a set of prior energy ratio measurements. This procedure is illustrated in Figure 4 for an example energy ratio curve. Let N be the number of measurement sets used for the prediction, referred to as the prediction set. The prediction set is assumed to be representative of the sample without damage. There is a gap of S measurements between the prediction set and the current measurement, n c, to allow small cracks to have S measurements to propagate prior to detection. Now consider a local linear fit of the energy ratio curve using the prediction set, and let σ e be the calculated standard deviation of the error between the points in the prediction set and the linear fit. A defect is detected if the actual energy ratio at the current measurement is at least three standard deviations (3σ e ) below the linear prediction. If this drop occurs, the crack detection output, n detect, is the current measurement, n c, and all remaining energy ratio measurements correspond to a detected defect. Figure 4. Illustration of the crack detection algorithm with N = 6 and S = 2. 4

2.3 Measurement Model The first modelling aspect is determining a relationship between the ultrasonic measurement response and crack depth. Given the complexity of the ultrasonic energy diffracting around the surface of the hole and interacting with multiple cracks, an empirical approach was employed for approximating this relationship as was done in [5]. The final measurement responses and crack sizes shown in Figure 3 are used in the development of the measurement model. The procedure for calculating the empirical relationship between crack depth and the energy ratio response is similar to the â vs. a POD strategies discussed previously. Following the convention for POD, the crack size, a, is the measured crack depth and the ultrasonic response is â. Here â is taken to be one minus the energy ratio ( a ˆ = 1 R) so that increasing crack depths correspond to increasing values of â. A visual examination of the experimental results of Figure 3 indicates that the preferred linear relationship of log 10 ( a ˆ )= c 1 log 10 ()+ a c 0 is appropriate and the constant variance, σ δ, assumption for the distribution is reasonable. Thus, the expression for the measurement model, M(a), is given in the linear domain as R = M(a) = 1 ˆ a = 1 10 c 0 ac 1. Additionally, the constant standard deviation in the log-log domain, σ δ, is transformed to the linear domain as Q δ = 0.5 10 c 0 ac 1 10 σ δ 10 σ δ ( ) Figure 5 compares the experimental results with the measurement model and the standard deviation in the linear plane, displaying reasonable agreement. For the experimental results summarized in Figure 3, the values for c 0 and c 1 are -0.44 and 1.03, respectively. Note that c 1 is close to unity, implying that the standard deviation is proportional to the crack size. It is important to discuss the implications of the measurement model. For traditional NDE, a measurement model is an indication of the expected range of sensor responses for repeat inspections of a given defect. The source of the variability in response is generally attributed to the inspection procedure itself. For SHM, however, there is little variation in sensor response for a given defect since the sensors are permanently attached to the structure. Consequently, the primary sources of variability are related to the crack geometry and the small perturbations in sensor position. In other words, the variance of the measurement model captures the variability between experiments, not successive Figure 5. Measurement model for relating the crack depth to energy ratio response. 5

measurements on a single experiment. The implication is that for a single experiment, the crack generated will have associated sensor responses that consistently either under or overestimate the crack size by a variable amount when using the measurement model. 2.4 Damage Propagation Model The next modelling aspect is determining an appropriate model for the propagation of the crack near a fastener hole. Based on established mechanics theory, an appropriate model is Paris s power law [7]. Paris s equation defines the incremental crack growth rate per fatigue cycle as da dn = C( ΔK) m. As per [7], the material constants C and m for 7075 Al were taken to be 2.71 10 8 MPa m and 3.7, respectively, with the resulting incremental growth rates in units of mm/cycle. The stress intensity range, K, is given by ΔK = FΔS πa, where F = Fa ()= Q k T 1+ 4.5 ar, where S is the applied stress range for that cycle and F is a geometry correction factor related to the crack shape and specimen geometry. For the work presented here, the F parameter assumes a hole of radius r with symmetric elliptical cracks of depth a and stress concentration factor, k T [8]. For this study, the stress concentration factor is approximately 2.52. The Q variable captures the effect of different elliptical crack shapes; prior efforts have shown that Q = 1 provides the best agreement with the experimental results. 2.5 Simulated Hit/Miss Data Generation Following the modelling and detection strategies discussed above, it is possible to generate simulated hit/miss data for POD analysis. First, artificial crack growth curves are produced using the damage propagation model and an appropriate fatigue spectrum. Each crack growth curve is assumed to have 40 regularly spaced measurement sets. To generate a range of expected crack growth curves, the initial crack size is varied between 5 μm and 7 μm. The result is 21 crack growth curves, a i ( n), where examples are given in Figure 6. Corresponding nominal energy ratio curves, R i (n), are generated using the measurement model, M(a). Each of these nominal energy ratio curves is randomly perturbed to generate 20 unique energy ratio curves, resulting in a total of 420 simulated experiments. Two perturbations are made to simulate the effects of arbitrary crack geometries and intra-measurement variations. As mentioned previously, the crack associated with a single experiment will be consistently under or over-estimated depending on the crack geometry. To approximate this effect, the nominal energy ratio curve is first randomly scaled proportional to the crack size, a, by R i [ ]R i ( n). () n 1+ G a i ( n) Here, G ~N( 0, Q δ ), where N is a normal distribution of mean zero and standard deviation Q δ. The standard deviation for this formulation is given as 6

Figure 6. Example simulated crack growth curves (left) and the associated energy ratio curves (right). Note that the energy ratio curves shown were generated from the same crack growth curve. Q δ 0.5 10 c 0 10 σ δ 10 σ δ [ ( )], which is the standard deviation from the measurement model presented earlier without the crack size term. For intra-measurement variations, a random variable w(n), where w~ N 0,σ e 2 ( ), is added to the scaled energy ratio, R i (n) = [ 1+ G a i ( n) ]R i ( n)+ wn ( ). Based on the 37 experiments shown in Figure 3, a typical value for σ e is approximately 0.02. Examples of the simulated energy curves are given in Figure 6, where each of the curves is from the same nominal energy ratio curve, R i (n). Each of the 420 simulated energy ratio curves, R i (n), can now be processed using the detection algorithm. Given that each of the simulated energy ratio curves has 40 discrete measurements, a total of 16,800 individual hit/miss data points are available for the subsequent POD curve creation. Figure 7 summarizes the results. 2.6 Probability of Detection Curves Using the available hit/miss data, conventional statistical analysis techniques were used for POD curve creation. In summary, logistic regression is used to estimate parameters of a log-odds function to model the binominal data. For each of the 16,800 data points, there is a crack size, a i, and an associated binary response, x i, where x i is 1 for a hit and 0 for a miss. For the log-odds model, the probability of detection for each data point, P i, is given as 1 P i = 1 + e α +β a i, with α and β model parameters. A generalized linear model regression procedure is used to calculate the appropriate α and β for the simulated hit/miss data using the MATLAB software environment and the Statistics Toolbox. The resulting POD curve for the data is shown in Figure 7. From the curve, the POD associated with a 0.53 mm deep crack size is approximately 90%. As with the measurement model interpretation for SHM, however, the interpretation of this POD value is different from the NDE interpretation. For NDE, the implication is that for an arbitrary 0.53 mm defect, there would be a 90% chance of detection for each inspection. Since most 7

Figure 7. The simulated hit/miss data (left) and the associated POD curve. of the variability captured by the SHM POD curve is related to the crack geometry, this 90% value indicates that the SHM system will detect approximately 90% of all possible defects with a maximum depth of 0.53 mm. Conclusions This paper presents data from a case study where a POD curve was calculated for an SHM application. The paper illustrates the development of the POD curve using a model-assisted strategy. Rather than employing conventional analysis strategies, a series of models were developed to simulate measurement responses from an SHM system and their associated crack detection results. Logistic regression was used to model the resulting binomial response data. The interpretation of the POD curves was discussed, with differences between NDE and SHM resulting from the different sources of variability. For NDE, the POD curve indicates the chance of detecting an arbitrary crack for each inspection as a function of crack size. In contrast, for SHM, the POD curve indicates the percent of all possible defects of a given size that would be detected; the major source of variability is the difference among similarly-sized defects. SHM POD results were shown here for a single case study, but no minimum detection experiments were performed to verify the results. It is anticipated that verification of POD will be a major issue in terms of quantifying performance of SHM systems prior to implementation. Also, some sources of variability were not considered, such as crack growth parameters, to simplify the discussion. References [1] S. S. Kulkarni and J. D. Achenbach, Structural health monitoring and damage prognosis in fatigue, Structural Health Monitoring, vol. 7, no. 1, pp. 37 49, 2008. [2] S. Hudak, B. Lanning, G. Light, J. Major, J. Moryl, M. Enright, R. McClung, and H. Millwater, The influence of uncertainty in usage and fatigue damage sensing on turbine engine prognosis, in Materials Damage Prognosis, (Warrendale, PA), TMS, 2005. [3] MIL-HDBK-1823, Nondestructive Evaluation System Reliability Assessment. 1999. [4] J. E. Michaels, T. E. Michaels, and B. Mi, An ultrasonic angle beam method for in situ sizing of fastener hole cracks, Journal of Nondestructive Evaluation, vol. 25, no. 1, pp. 3 16, 2006. [5] A. C. Cobb, J. E. Michaels and T. E. Michaels, Ultrasonic structural health monitoring: A probability of detection case study, Review of Progress in Quantitative Nondestructive Evaluation, vol. 28B, D. O. Thompson and D. E. Chimenti (Eds.), American Institute of Physics, pp. 1800-1807, 2009. [6] A. C. Cobb, J. E. Michaels, T. E. Michaels, An automated time frequency approach for ultrasonic monitoring of fastener hole cracks, NDT&E International, vol. 40, pp. 525 536, 2007. [7] N. E. Dowling, Mechanical Behavior of Materials, 10th Edition, Prentice-Hall, Inc., New Jersey, 1993. [8] P. Lucas, Stress intensity factor for small notch-emanated cracks, Engineering Fracture Mechanics, vol. 26. no. 3, pp 471 473, 1987. 8