Test Uncertainty ASME PTC (Revision of ASME PTC ) Copyright ASME International

Transcription

1 Test Uncertainty A N A M E R I C A N N A T I O N A L S T A N D A R D ASME PTC (Revision of ASME PTC )

2

3 Date of Issuance: October 13, 2006 The 2005 edition of ASME PTC 19.1 will be revised when the Society approves the issuance of the next edition. There will be no Addenda issued to ASME PTC ASME issues written replies to inquiries as code cases and interpretations of technical aspects of this document. Code cases and interpretations are published on the ASME website under the Committee Pages at as they are issued. ASME is the registered trademark of The American Society of Mechanical Engineers. This code or standard was developed under procedures accredited as meeting the criteria for American National Standards. The Standards Committee that approved the code or standard was balanced to assure that individuals from competent and concerned interests have had an opportunity to participate. The proposed code or standard was made available for public review and comment that provides an opportunity for additional public input from industry, academia, regulatory agencies, and the public-at-large. ASME does not approve, rate, or endorse any item, construction, proprietary device, or activity. ASME does not take any position with respect to the validity of any patent rights asserted in connection with any items mentioned in this document, and does not undertake to insure anyone utilizing a standard against liability for infringement of any applicable letters patent, nor assumes any such liability. Users of a code or standard are expressly advised that determination of the validity of any such patent rights, and the risk of infringement of such rights, is entirely their own responsibility. Participation by federal agency representative(s) or person(s) affiliated with industry is not to be interpreted as government or industry endorsement of this code or standard. ASME accepts responsibility for only those interpretations of this document issued in accordance with the established ASME procedures and policies, which preclude the issuance of interpretations by individuals. No part of this document may be reproduced in any form, in an electronic retrieval system or otherwise, without the prior written permission of the publisher. The American Society of Mechanical Engineers Three Park Avenue, New York, NY Copyright 2006 by THE AMERICAN SOCIETY OF MECHANICAL ENGINEERS All rights reserved Printed in U.S.A.

4 CONTENTS Notice... Foreword... Committee Roster... Section 1 Introduction General Harmonization With International Standards Applications... 1 Section 2 Object and Scope Object Scope... 2 Section 3 Nomenclature and Glossary Nomenclature Glossary Section 4 Fundamental Concepts Assumptions Measurement Error Measurement Uncertainty Pretest and Posttest Uncertainty Analyses Section 5 Defining the Measurement Process Overview Selection of the Appropriate True Value Identification of Error Sources Categorization of Uncertainties Comparative Versus Absolute Testing Section 6 Uncertainty of a Measurement Random Standard Uncertainty of the Mean Systematic Standard Uncertainty of a Measurement Classification of Uncertainty Sources Combined Standard and Expanded Uncertainty of a Measurement Section 7 Uncertainty of a Result Propagation of Measurement Uncertainties Into a Result Sensitivity Random Standard Uncertainty of a Result Systematic Standard Uncertainty of a Result Combined Standard Uncertainty and Expanded Uncertainty of a Result Examples of Uncertainty Propagation Section 8 Additional Uncertainty Considerations Correlated Systematic Standard Uncertainties Nonsymmetric Systematic Uncertainty Fossilization of Calibrations Spatial Variation iii vii viii ix

5 8-5 Analysis of Redundant Means Regression Uncertainty Section 9 Step-by-Step Calculation Procedure General Considerations Calculation Procedure Section 10 Examples Flow Measurement Using Pitot Tubes Flow Rate Uncertainty Flow Rate Uncertainty Including Nonsymmetrical Systematic Standard Uncertainty Compressor Performance Uncertainty Periodic Comparative Testing Section 11 References Section 12 Bibliography Figures Illustration of Measurement Errors Measurement Error Components Distribution of Measured Values (Normal Distribution) Uncertainty Interval Generic Measurement Calibration Hierarchy Difference Between Within and Between Sources of Data Scatter Pareto Chart of Systematic and Random Uncertainty Component Contributions to Combined Standard Uncertainty Schematic Relation Between Parameters Characterizing Nonsymmetric Uncertainty Relation Between Parameters Characterizing Nonsymmetric Uncertainty Three Posttest Cases Traverse Points (Example 10-1) Schematic of a6in. 4 in. Venturi Typical Pressure and Temperature Locations for Compressor Efficiency Determination The h-s Diagram of the Actual and Isentropic Processes of an Adiabatic Compressor Installed Arrangement Pump Design Curve With Factory and Field Test Data Shown Comparison of Test Results With Independent Control Conditions Comparison of Test Results Using the Initial Field Test as the Control Tables Circulating Water Bath Temperature Measurements (Example 6-4.1) Systematic Uncertainty of Average Circulating Water Bath Temperature Measurements (Example 6-4.1) Table of Data (Example 7-6.1) Summary of Data (Example 7-6.1) Table of Data (Example 7-6.2) Summary of Data (Example 7-6.2) Burst Pressures (Example 8-1-1) iv

6 Systematic Standard Uncertainty Components for Ŷ Determined from Regression Equation Table of Data Summary of Data Average Values (Example 10-1) Standard Deviations (Example 10-1) Summary of Average Velocity Calculation (Example 10-1) Standard Deviation of Average Velocity (Example 10-1) Uncertainty of Result (Example 10-1) Uncalibrated Case (Example 10-2) Absolute Sensitivity Coefficients in Example 10-2 (Calculated Numerically) Absolute Sensitivity Coefficients in Example 10-2 (Calculated Analytically) Absolute Contributions of Uncertainties of Independent Parameters (Example 10-2: Uncalibrated Case) Summary: Uncertainties in Absolute Terms (Example 10-2: Uncalibrated Case) Relative Uncertainty of Measurement (Example 10-2: Uncalibrated Case) Relative Contributions of Uncertainties of Independent Parameters (Example 10-2: Uncalibrated Case) Summary: Uncertainties in Relative Terms for the Uncalibrated Case Relative Uncertainties of Independent Parameters (Example 10-2: Calibrated Case) Relative Contributions of Uncertainties of Independent Parameters (Example 10-2: Calibrated Case) Summary: Uncertainties in Relative Terms for the Calibrated Case Summary: Comparison Between Calibrated and Uncalibrated Cases Absolute Contributions of Uncertainties of Independent Parameters (Example 10-3: Uncalibrated, Nonsymmetrical Systematic Uncertainty Case) Summary: Uncertainties in Absolute Terms (Example 10-3: Uncalibrated, Nonsymmetrical Systematic Uncertainty Case) Elemental Random Standard Uncertainties Associated With Error Sources Identified in Para Independent Parameters Calculated Result Inlet and Exit Pressure Elemental Systematic Standard Uncertainties Inlet and Exit Temperature Elemental Systematic Standard Uncertainties Evaluation of Analysis Error Pump Design Data (T c p 20 C) Summary of Test Results Uncertainty Propagation for Comparison With Independent Control Summary: Uncertainties in Absolute Terms Summary of Results for Each Test v

7 Uncertainty Propagation for Comparative Uncertainty Sensitivity Coefficient Estimates for Comparative Analysis Nonmandatory Appendices A Statistical Considerations B Uncertainty Analysis Models C Propagation of Uncertainty Through Taylor Series D The Central Limit Theorem vi

8 NOTICE All Performance Test Codes must adhere to the requirements of ASME PTC 1, General Instructions. The following information is based on that document and is included here for emphasis and for the convenience of the user of the Supplement. It is expected that the Code user is fully cognizant of Sections 1 and 3 of ASME PTC 1 and has read them prior to applying this Supplement. ASME Performance Test Codes provide test procedures which yield results of the highest level of accuracy consistent with the best engineering knowledge and practice currently available. They were developed by balanced committees representing all concerned interests and specify procedures, instrumentation, equipment-operating requirements, calculation methods, and uncertainty analysis. When tests are run in accordance with a Code, the test results themselves, without adjustment for uncertainty, yield the best available indication of the actual performance of the tested equipment. ASME Performance Test Codes do not specify means to compare those results to contractual guarantees. Therefore, it is recommended that the parties to a commercial test agree before starting the test and preferably before signing the contract on the method to be used for comparing the test results to the contractual guarantees. It is beyond the scope of any Code to determine or interpret how such comparisons shall be made. vii

9 FOREWORD In March 1979 the Performance Test Codes Supervisory Committee activated the PTC 19.1 Committee to revise a 1969 draft of a document entitled PTC 19.1 General Considerations. The PTC 19.1 Committee proceeded to develop a Performance Test Code Instruments and Apparatus Supplement which was published in 1985 as PTC , Measurement Uncertainty, and which was intended along with its subsequent editions to provide a means of eventual standardization of nomenclature, symbols, and methodology of measurement uncertainty in ASME Performance Test Codes. Work on the revision of the original 1985 edition began in The two-fold objective was to improve the usefulness to the reader regarding clarity, conciseness, and technical treatment of the evolving subject matter, as well as harmonization with the ISO Guide to the Expression of Uncertainty in Measurement. That revision was published as PTC , Test Uncertainty, the new title reflecting the appropriate orientation of the document. The effort to update the 1998 revision began immediately upon completion of that document. This 2005 revision is notable for the following significant departures from the 1998 text: (a) Nomenclature adopted for this revision is more consistent with the ISO Guide. Uncertainties remain conceptualized as systematic (estimate of the effects of fixed error not observed in the data), and random (estimate of the limits of the error observed from the scatter of the test data). The new aspect is that both types of uncertainty are defined at the standard-deviation level as standard uncertainties. The determination of an uncertainty at some level of confidence is based on the root-sum-square of the systematic and random standard uncertainties multiplied times the appropriate expansion factor for the desired level of confidence (usually 2 for 95%). This same approach was used in the 1998 revision but the characterization of uncertainties at the standard-uncertainty level ( standard deviation ) was not as explicitly stated. The new nomenclature is expected to render PTC more acceptable at the international level. (b) There is greater discussion of the determination of systematic uncertainties. (c) There is new text on a simplified approach to determine the uncertainty of straightline regression. ASME PTC was approved by the PTC Standards Committee on September 13, 2005, and was approved as an American National Standard by the ANSI Board of Standards Review on November 3, viii

10 PERFORMANCE TEST CODE COMMITTEE 19.1 ON TEST UNCERTAINTY (The following is the roster of the Committee at the time of the approval of this Supplement.) OFFICERS R. H. Dieck, Chair W. G. Steele, Vice Chair G. Osolsobe, Secretary COMMITTEE PERSONNEL J. F. Bernardin, Pratt & Whitney D. A. Coutts, WSMS R. H. Dieck, Ron Dieck Associates, Inc. R. S. Figliola, Clemson University H. K. Iyer, Colorado State University J. Maveety, Intel Corp. J. A. Rabensteine, Environmental Systems Corp. M. Soltani, Bechtel National Corp. W. G. Steele, Mississippi State University ix

11 PERFORMANCE TEST CODES STANDARDS COMMITTEE OFFICERS J. G. Yost, Chair J. R. Friedman, Vice Chair S. D. Weinman, Secretary COMMITTEE PERSONNEL P. G. Albert P. M. McHale R. P. Allen M. P. McHale J. M. Burns J. W. Milton W. C. Campbell S. P. Nuspl M. J. Dooley A. L. Plumley A. J. Egli R. R. Priestley J. R. Friedman J. A. Rabensteine G. J. Gerber J. W. Siegmund P. M. Gerhart J. A. Silvaggio T. C. Heil W. G. Steele R. A. Johnson J. C. Westcott D. R. Keyser W. C. Wood S. J. Korellis J. G. Yost HONORARY MEMBERS W. O. Hays F. H. Light MEMBERS EMERITI R. L. Bannister G. H. Mittendorf R. Jorgensen R. E. Sommerlad x

12 TEST UNCERTAINTY ASME PTC GENERAL This Supplement has significant additions and Sections that have been rewritten to both add to the available technology for uncertainty analysis and to make it easier for the practicing engineer. Throughout, the intent is to provide a Supplement that can be utilized easily by engineers and scientists whose interest is the objective assessment of data quality, using test uncertainty analysis. 1-2 HARMONIZATION WITH INTERNATIONAL STANDARDS It is recognized that this Supplement and promulgated international uncertainty standards and/ or guides must be in harmony. In rewriting this Supplement, great care was taken to assure continued harmony with the International Organization for Standardization (ISO) Guide to the Expression of Uncertainty in Measurement (GUM) [1]. For the practicing engineer, this harmonization means the elimination of such ambiguous terms as bias, precision, bias limit, and precision index. In addition, careful attention was paid to discriminating between errors, the effects of errors, and the estimation of their limits, which is the uncertainty. The term bias is not used in this Supplement. Instead, the combined terms of systematic error and systematic uncertainty are used. The former describes an error source whose effect is systematic or constant for the duration of a test. The latter describes the limits to which a systematic error may be expected to go with some confidence. The term precision also is not used in this Supplement. Instead the combined terms of random error and random uncertainty are used. The former describes an error source that causes scatter in test data. The latter describes the limits to which a random error may be expected to reach with some confidence. Throughout the Supplement, the term standard uncertainty has been introduced to improve Section 1 Introduction harmony with international guidelines and standards. In this Supplement, standard uncertainties are always equivalent to a single standard deviation of the average. The most common confidence level used in this Supplement is 95% although methods for employing alternate confidences are also given. The confidence level of 95% is applied to expanded uncertainty. This term, too, was included in this Supplement for improved harmony with international guidelines and standards. While this Supplement is in harmony with the ISO GUM, this Supplement emphasizes the effects of errors rather than the basis of the information utilized in the estimation of their limits. The ISO GUM utilizes two major classifications for errors and uncertainties. They are Type A and Type B. Type A uncertainties have data with which to calculate a standard deviation. Type B uncertainties do not have data to calculate a standard deviation and must be estimated by other means. This Supplement utilizes two major classifications for errors and uncertainties. They are systematic and random. Random errors (whose effects are estimated with Random Standard Uncertainties ) cause scatter in test data. Systematic errors (whose effects are estimated with Systematic Standard Uncertainties ) do not. Harmonization of this Supplement with the ISO GUM is achieved by encouraging subscripts with each uncertainty estimate to denote the ISO Type, i.e., using subscripts of either A or B. 1-3 APPLICATIONS This Supplement is intended to serve as a reference to the various other ASME Instruments and Apparatus Supplements (PTC 19 Series) and to ASME Performance Test Codes and Standards in general. In addition, it is applicable for all known measurement and test uncertainty analyses. The paramater values and uncertainty levels used throughout the examples are for illustrative purposes only and are not intended to be typical of standard tests. 1

13 ASME PTC TEST UNCERTAINTY 2-1 OBJECT The object of this Supplement is to define, describe, and illustrate the various terms and methods used to provide meaningful estimates of the uncertainty in test parameters and methods, and the effects of those uncertainties on derived test results. Analysis of test measurement and result uncertainty is useful because it (a) facilitates communication regarding measurement and test results; (b) fosters an understanding of potential error sources in a measurement system and the effects of those potential error sources on test results; (c) guides the decision-making process for selecting appropriate and cost-effective measurement systems and methodologies; (d) reduces the risk of making erroneous decisions; and Section 2 Object and Scope (e) documents uncertainty for assessing compliance with agreements. 2-2 SCOPE The scope of this Supplement is to specify procedures for evaluation of uncertainties in test parameters and methods, and for propagation of those uncertainties into the uncertainty of a test result. Depending on the application, uncertainty sources may be classified either by the presumed effect (systematic or random) on the measurement or test result, or by the process in which they may be quantified (Type A or Type B). The various statistical terms involved are defined in the Nomenclature (subsection 3-1) or Glossary (subsection 3-2). The end result of an uncertainty analysis is a numerical estimate of the test uncertainty with an appropriate confidence level. 2

14 TEST UNCERTAINTY ASME PTC NOMENCLATURE b X p systematic standard uncertainty component of a parameter b Xk p systematic standard uncertainty associated with the k th elemental error source b R p systematic standard uncertainty component of a result b XY p covariance of the systematic errors in X and Y b +,b p upper and lower values of nonsymmetrical systematic standard uncertainty Np number of measurements or sample points or observations available (sample size) Rp result s R p random standard uncertainty of a result s X p standard deviation of a data sample; estimate of the standard deviation of the population x s X p random standard uncertainty of the mean of N measurements SEEp standard error of estimate of a leastsquares regression or curve fit tp Student s t value at a specified confidence level with degrees of freedom, i.e., t 95, up combined standard uncertainty Up expanded uncertainty U +,U p upper and lower values of the nonsymmetrical expanded uncertainty Xp individual observation in a data sample of a parameter Xp sample mean; average of a set of N individual observations of a parameter p (unknown) true systematic error; fixed or constant component of Section 3 Nomenclature and Glossary 3 p (unknown) total error; difference between the assigned value of a parameter or a test result and the true value p (unknown) true random error; random component of p absolute sensitivity p relative sensitivity p (unknown) true average of a population p number of degrees of freedom p (unknown) true standard deviation of a population 2 p (unknown) true variance of a population Indices Ip total number of variables ip counter for variables jp counter for individual measurements Kp total number of sources of elemental errors and uncertainties kp counter for sources of elemental errors and uncertainties Lp total number of correlated sources of systematic error lp counter for correlated sources of systematic error Mp total number of multiple results mp counter for multiple results Np total number of measurements 3-2 GLOSSARY calibration hierarchy: the chain of calibrations that links or traces a measuring instrument to a primary standard. calibration: the process of comparing the response of an instrument to a standard instrument over some measurement range. confidence level: the probability that the true value falls within the specified limits.

15 ASME PTC TEST UNCERTAINTY degrees of freedom ( ): the number of independent observations used to calculate a standard deviation. elemental random error source: an identifiable source of random error that is a subcomponent of total random error. elemental random standard uncertainty (s Xk ): an estimate of the standard deviation of the mean of an elemental random error source. elemental systematic error source: an identifiable source of systematic error that is a subcomponent of the total systematic error. elemental systematic standard uncertainty (b Xk ): an estimate of standard deviation of an elemental systematic error source. expanded uncertainty (U X or U R ): an estimate of the plus-or-minus limits of total error, with a defined level of confidence, (usually 95%). influence coefficient: see sensitivity. mean (X): the arithmetic average of N readings. parameter: quantity that could be measured or taken from best available information, such as temperature, pressure, stress, or specific heat, used in determining a result. The value used is called the assigned value. population mean ( ): average of the set of all population values of a parameter. population standard deviation ( ): a value that quantifies the dispersion of a population. population: the set of all possible values of a parameter. random error ( ): the portion of total error that varies randomly in repeated measurements of the true value throughout a test process. random standard uncertainty of the sample mean (s X ): a value that quantifies the dispersion of a sample mean as given by eq. (4-3.3). result (R): a value calculated from a number of parameters. sample size (N): the number of individual values in a sample. sample standard deviation (s x ): a value that quantifies the dispersion of a sample of measurements as given by eq. (4-3.2). sensitivity: the instantaneous rate of the change in a result due to a change in a parameter. standard error of estimate (SEE): the measure of dispersion of the dependent variable about a least squares regression or curve. statistic: any numerical quantity derived from the sample data. X and s X are statistics. Student s t: a value used to estimate the uncertainty for a given confidence level. systematic error ( ): the portion of total error that remains constant in repeated measurements of the true value throughout a test process. systematic standard uncertainty (b X ): a value that quantifies the dispersion of a systematic error associated with the mean. total error ( ): the true, unknown difference between the assigned value of a parameter or test result and the true value. traceability: see calibration hierarchy. true value: the error-free value of a parameter or test result. Type A uncertainty: uncertainties are classified as Type A when data is used to calculate a standard deviation for use in estimating the uncertainty. Type B uncertainty: uncertainties are classified as Type B when data is not used to calculate a standard deviation, requiring the uncertainty to be estimated by other methods. uncertainty interval: an interval expressed about a parameter or test result that is expected to contain the true value with a prescribed level of confidence. 4

16 TEST UNCERTAINTY ASME PTC ASSUMPTIONS The assumptions inherent in test uncertainty analysis include the following: (a) The test objectives are specified. (b) The test process, including the measurement process and the data reduction process, is defined. (c) The test process, with respect to the conditions of the item under test and the measurement system employed for the test, is controlled for the duration of the test. (d) The measurement system is calibrated and all appropriate calibration corrections are applied to the resulting test data. (e) All appropriate engineering corrections are applied to the test data as part of the data reduction and/or results analysis process. For expanded uncertainty, 95% confidence levels have been used throughout this document in accordance with accepted practice. Other confidence levels may be used, if required. (See Nonmandatory Appendix B.) 4-2 MEASUREMENT ERROR Every measurement has error, which results in a difference between the measured value, X, and the true value. The difference between the measured value and the true value is the total error,. Since the true value is unknown, total error cannot be known and therefore only its expected limits can be estimated. Total error consists of two components: random error and systematic error (see Fig ). Accurate measurement requires minimizing both random and systematic errors (see Fig ) Random Error Random error,, is the portion of the total error that varies randomly in repeated measurements throughout the conduct of a test. The total random error in a measurement is usually the sum of the Section 4 Fundamental Concepts 5 contributions of several elemental random error sources. Elemental random errors may arise from uncontrolled test conditions and nonrepeatabilities in the measurement system, measurement methods, environmental conditions, data reduction techniques, etc Systematic Error Systematic error,, is the portion of the total error that remains constant in repeated measurements throughout the conduct of a test. The total systematic error in a measurement is usually the sum of the contributions of several elemental systematic errors. Elemental systematic errors may arise from imperfect calibration corrections, measurement methods, data reduction techniques, etc. 4-3 MEASUREMENT UNCERTAINTY There is an inherent uncertainty in the use of measurements to represent the true value. The total uncertainty in a measurement is the combination of uncertainty due to random error and uncertainty due to systematic error Random Standard Uncertainty Any single measurement of a parameter is influenced by several different elemental random error sources. In successive measurements of the parameter, the values of these elemental random error sources change resulting in the random scatter evident in the successive measurements. If an infinite number of measurements of a parameter were to be taken following the defined test process, the resulting population of measurements could be described statistically in terms of the population mean,, the population standard deviation,, and the frequency distribution of the population. These terms are illustrated in Fig for a population of measurements that is normally distributed. For measurements with zero systematic

17 `,`,,``,``,`,``,```````,,`-`-`,,`,,`,`,,`--- ASME PTC TEST UNCERTAINTY Fig error (refer to para ), the population mean is equal to the true value of the parameter being measured and the population standard deviation is a measure of the scatter of the individual measurements about the population mean. For a normal distribution, the interval ± will include approximately 68% of the population and the interval ±2 will include approximately 95% of the population. Since only a finite number of measurements are acquired during a test, the true population mean and population standard deviation are unknown but can be estimated from sample statistics. The sample mean, X, is given by X p N X j jp1 N Illustration of Measurement Errors (4-3.1) where X j represents the value of each individual measurement in the sample and N is the number of measurements in the sample. The sample standard deviation, s X, is given by 6 s X p N jp1 (X j X) 2 N 1 (4-3.2) Since the sample mean is only an estimate of the population mean, there is an inherent error in the use of the sample mean to estimate the population mean. For a defined frequency distribution, the random standard uncertainty of the sample mean, s X, can be used to define the probable interval about the sample mean that is expected to contain the population mean with a defined level of confidence. The random standard uncertainty of the sample mean is related to the sample standard deviation as follows: s X p s X N (4-3.3) For a normally distributed population and a large sample size (N > 30), the interval X±s X is expected to contain the true population mean with 68% confidence and the interval X±2s X is expected to contain the true population mean with 95% confidence [where the value 2 represents the Student s t value for 95% confidence and degrees of

18 TEST UNCERTAINTY ASME PTC Fig freedom of greater than or equal to 30 where the degrees freedom for the random standard uncertainty is N 1 (see subsection 6-1)]. In general, increasing the number of measurements collected during a test and used in the preceding formulas is beneficial as (a) it improves the sample mean as an estimator of the true population mean; (b) it improves the sample standard deviation as an estimator of the true population standard deviation; and (c) it typically reduces the value of the random standard uncertainty of the sample mean Systematic Standard Uncertainty Every measurement of a parameter is influenced by several different elemental systematic error sources. Each of these elemental systematic error Measurement Error Components 7 sources contributes a constant, but unknown, error, Xk, to the successive measurements of a parameter for the duration of the test (the subscript k is used to denote a specific elemental error source). As Xk is constant for the test, the error imparted to the average value of successive measurements, X [as given by eq ], is equivalent to the error imparted to each individual measurement. While Xk is unknown, it may be postulated to come from a population of possible error values from which a single sample (error value) is drawn and imparted to the average measurement for the test. Knowledge of the frequency distribution and standard deviation of this population permits describing the uncertainty in X due to this single sample elemental systematic error in terms of a confidence interval. The elemental systematic standard uncertainty, b Xk, is defined as a value

19 ASME PTC TEST UNCERTAINTY Fig that quantifies the dispersion of the population of possible Xk values at the standard deviation level. All of the elemental systematic errors associated with a measurement combine to yield the total systematic error in the measurement, X. As with elemental systematic error, total systematic error is constant, unknown, and may be postulated to come from a population of possible error values from which a single sample (error value) is drawn and imparted to the average measurement for the test. Total systematic standard uncertainty, b X,is defined as a value that quantifies the dispersion of the population of possible X values at the standard deviation level. Typically, total systematic standard uncertainty is quantified by (a) identifying all elemental sources of systematic error for the measurement; (b) evaluating elemental systematic standard uncertainties as the standard deviations of the possible systematic error distributions; and (c) combining the elemental systematic standard uncertainties into an estimate of the total systematic standard uncertainty for the average measurement Identifying Elemental Sources of Systematic Error. Attempting to identify all of the elemental sources of systematic error for a measurement is an important step of an uncertainty analysis, as failure to identify any significant source of systematic error will lead to an underestimation Distribution of Measured Values (Normal Distribution) of test uncertainty. Attempting to identify all elemental sources of systematic error requires a thorough understanding of the test objectives and test process. For further discussion refer to subsection Evaluating Elemental Systematic Standard Uncertainties. Once all elemental sources of systematic error are identified, elemental systematic standard uncertainties for each source are evaluated. By definition, an elemental systematic standard uncertainty is a value that quantifies the dispersion of the population of possible Xk values at the standard deviation level. As Xk is both constant and unknown during a test, successive measurements of a parameter do not provide sufficient data for direct computation of a standard deviation as described in para Therefore, the evaluation of an elemental systematic standard uncertainty requires that a standard deviation be evaluated from engineering judgment, published information, or special data Engineering Judgment. When neither published information or special data is available, it is often necessary to rely upon engineering judgment to quantify the dispersion of errors associated with an elemental error source. In these situations, it is customary to use engineering analyses and experience to estimate the limits of the 8

20 TEST UNCERTAINTY ASME PTC elemental systematic error at 95% confidence. In other words, an interval is estimated which is expected to contain 95% of the population of possible Xk values. Not withstanding information to the contrary, the analyst typically assumes that the population of possible Xk values is normally distributed, that the estimation of the limits of the error is based upon large degrees of freedom, and that the limits of error are symmetric (equally spread in both the positive and negative directions). Based upon these assumptions, the elemental systematic standard uncertainty is estimated as follows: b Xk p B Xk 2 (4-3.4) The variable B Xk in the preceding equation represents the 95% confidence level estimate of the symmetric limits of error associated with the k th elemental error source. In certain situations, knowledge of the physics of the measurement system will lead the analyst to believe that the limits of error are nonsymmetric (likely to be larger in either the positive or negative direction). For treatment of nonsymmetric systematic uncertainty see subsection 8-2. The value of 2 in the equation is based on the assumption that the population of possible systematic errors is normally distributed. If the analyst thinks that the error distribution might be other than normal, such as uniform (rectangular), then a different factor would be used to convert the 95% confidence level estimate of the systematic error limits to an elemental systematic standard uncertainty (see Nonmandatory Appendix B). Also, there is some level of uncertainty associated with the estimate of B Xk. This uncertainty in the estimate can be converted into a degrees of freedom for the systematic standard uncertainty as shown in Nonmandatory Appendix B. Usually, the B Xk estimates are made such that this degrees of freedom will be large ( 30). Using the recommendations in Nonmandatory Appendix B, it can be shown that this large degrees of freedom ( 30) corresponds to an uncertainty in the estimate of B Xk of 13% or less Published Information. For some elemental systematic error sources, published information from calibration reports, instrument specifications, and other technical references may provide quantitative information regarding the dispersion of errors for an elemental systematic error source in terms of a confidence interval, an ISO expanded uncertainty statement, or a multiple of a standard deviation. If the published information is presented as a confidence interval (limits of error at a defined level of confidence), then the elemental systematic standard uncertainty is estimated as the confidence interval divided by a statistic that is appropriate for the frequency distribution of the error population. The specific value of this statistic must be selected on the basis of the defined confidence level and degrees of freedom associated with the confidence interval. For a normal distribution, the Student s t statistic is used. For a 95% confidence level and large degrees of freedom, the value of the Student s t statistic is approximated as 2 and eq. (4-3.4) would apply (refer to Nonmandatory Appendix B for values of the Student s t statistic at other confidence levels and degrees of freedom). For situations in which the frequency distribution and degrees of freedom are unspecified, a normal distribution and large degrees of freedom are often assumed. For situations involving other frequency distributions, refer to an appropriate statistics textbook. If the published information is presented as an ISO expanded uncertainty at a defined coverage factor (sometimes referred to as a k factor ), then the elemental systematic standard uncertainty is estimated as the expanded uncertainty divided by the coverage factor. If the published information is presented as a multiple of a standard deviation, then the elemental systematic standard uncertainty is estimated as the multiple of the standard deviation divided by the multiplier Special Data. For some elemental systematic error sources, special data may be obtained that manifests the dispersion of the population of possible, unknown Xk values. Possible sources of this special data include (a) interlaboratory or interfacility tests; and (b) comparisons of independent measurements that depend on different principles or that have been made by independently calibrated instruments; for example, in a gas turbine test, airflow can be measured with an orifice or a bell mouth nozzle, or computed from compressor speed-flow rig data, turbine flow parameters, or jet nozzle calibrations. For these cases, the elemental systematic standard uncertainty may be evaluated as follows: N X b Xk p k 1 (X kj X k ) 2 N Xk j p 1 N Xk 1 (4-3.5) 9

21 ASME PTC TEST UNCERTAINTY where N Xk p the number of special data values used in the computation of b Xk N Xk p the number of independent samples from the population of possible Xk values that are averaged together in the computation of the average measurement for the test (X) X k p the average of the set of special data X kj p the j th data point of the set of special data that manifests the dispersion of the population of possible Xk values associated with the k th elemental error source For most measurements (especially those made using a single instrument calibrated at a single laboratory and installed in a single location), only a single sample from the population of possible Xk values is included in the computation of the average measurement for the test (X) and hence N Xk p 1. The following illustrate some possible cases where N Xk may be greater than one. (a) Several independent measurement methods that depend on different principles are used to measure the same parameter. The results from each of the measurement methods (each determined as an average value over the duration of the test) are used as input to eq. (4-3.5) to evaluate the elemental systematic standard uncertainty associated with the error inherent to the various measurement methods. If the average measurement reported for the test is the average of the results from all of the measurement methods, then the value for N Xk used in eq. (4-3.5) is equal to the number of independent measurement methods employed. (b) An instrument is sent to multiple laboratories to obtain calibration data for the instrument prior to using the instrument in a test. The results from each of the independent laboratories (each determined as an offset to be applied to the instrument when measuring a specific input level) are used as input to eq. (4-3.5) to evaluate the elemental systematic standard uncertainty associated with the error inherent to the various laboratories. If the average measurement from the instrument reported for the test is based upon application of the average offset from all of the laboratories, then the value for N Xk used in eq. (4-3.5) is equal to the number of independent laboratories employed Combining Elemental Systematic Standard Uncertainties. Once evaluated, all of the elemental systematic standard uncertainties influencing a measurement are combined into an estimate 10 of the total systematic standard uncertainty for the measurement, b X. Provided all elemental systematic standard uncertainties are evaluated in terms of their influence on the parameter being measured and in the units of the parameter being measured, these elemental systematic standard uncertainties are combined per subsection 6-2. Otherwise, these elemental systematic standard uncertainties are combined per subsection 7-4. In some cases, elemental systematic standard uncertainties may arise from the same elemental error source and are therefore correlated. See subsection 8-1 for a detailed discussion Combined Standard Uncertainty and Expanded Uncertainty As mentioned previously, the total uncertainty in a measurement is the combination of uncertainty due to random error and uncertainty due to systematic error. The combined standard uncertainty of the measurement mean, which is the total uncertainty at the standard deviation level, is calculated as follows: where u X p (b X ) 2 +(s X ) 2 (4-3.6) b X p the systematic standard uncertainty s X p the random standard uncertainty of the mean The expanded uncertainty of the measurement mean is the total uncertainty at a defined level of confidence. For applications in which a 95% confidence level is appropriate, the expanded uncertainty is calculated as follows: U X p 2u X (4-3.7) where the assumptions required for this simple equation are presented in subsection 6-4. Expanded uncertainty is used to establish a confidence interval about the measurement mean which is expected to contain the true value. Thus, the interval X±U X is expected to contain the true value with 95% confidence (see Fig ).

22 TEST UNCERTAINTY ASME PTC Fig PRETEST AND POSTTEST UNCERTAINTY ANALYSES (a) The objective of a pretest analysis is to establish the expected uncertainty interval for a test result, prior to the conduct of a test. A pretest uncertainty analysis is based on data and information that exist before the test, such as calibration histories, previous tests with similar instrumentation, prior measurement uncertainty analyses, expert opinions, and, if necessary, special tests. A pretest uncertainty analysis allows corrective action to be taken, prior to expending resources to conduct a test, either to decrease the expected uncertainty to a level consistent with the overall objectives of the test or to reduce the cost of the test while still attaining the objectives. Possible corrective actions include (1) selecting alternative testing methods that rely upon different analysis procedures, testing under different conditions, and/or measurement of different parameters; Uncertainty Interval (2) selecting alternative measurement methods by varying test instrumentation, calibration techniques, installation methods, and/or measurement locations; and (3) increasing sample sizes by increasing sampling frequencies, increasing test duration, and/or conducting repeated testing. Additionally, a pretest uncertainty analysis facilitates communication between all parties to the test about the expected quality of the test. This can be essential to establishing agreement on any deviations from applicable test code requirements and can help reduce the risk that disagreements regarding the testing method will surface after conducting the test. (b) The objective of a posttest analysis is to establish the uncertainty interval for a test result, after conducting a test. In addition to the data and information used to conduct the pretest uncertainty analysis, a posttest uncertainty analysis is based upon the additional data and information gathered for the test including all test measurements, pretest and 11

23 ASME PTC TEST UNCERTAINTY posttest instrument calibration data, etc. A posttest uncertainty analysis serves to (1) validate the quality of the test result by demonstrating compliance with test requirements; (2) facilitate communication of the quality of the test result to all parties to the test; and (3) facilitate interpretation of the quality of the test by those using the test result. 12

24 TEST UNCERTAINTY ASME PTC OVERVIEW Section 5 Defining the Measurement Process The first step in a measurement uncertainty analysis is to clearly define the basic measurement process. This simple step, often overlooked, is essential to successfully develop and apply the uncertainty information. Consideration must be given to the selection of the appropriate true value of the measurement and the time interval for classifying errors as systematic or random. This section provides an overview of how the measurement process should be defined. 5-2 SELECTION OF THE APPROPRIATE TRUE VALUE Depending on the user s perspective, several measurement objectives or goals and hence corresponding true values (measurements with ideal zero error) may exist simultaneously in a measurement process. For example, when analyzing a thermocouple measurement in a gas stream, several starting points or true values can be selected. The starting point for the analysis could begin with the true value defined as the metal temperature of the thermocouple junction, the gas stagnation temperature or junction temperature corrected for probe effects, or the mass flow weighted average of the gas temperature at the plane of the instrumentation. Any of the aforementioned true values may be appropriate. The selection of the true value for the uncertainty analysis must be consistent with the goal of the measurement [3]. 5-3 IDENTIFICATION OF ERROR SOURCES Once the true value has been defined, the errors associated with measuring the true value must be identified. Examples of error sources include imperfect calibration corrections, uncontrolled test conditions, measurement methods, environmental conditions, and data reduction techniques. Estimates to reflect the extent of these errors are 13 represented as uncertainties. These uncertainties in the measurement process can be grouped by source (a) calibration uncertainty (b) uncertainty due to test article and/or instrumentation installation (c) data acquisition uncertainty (d) data reduction uncertainty (e) uncertainty due to methods and other effects Calibration Uncertainty Each measurement instrument may introduce random and systematic uncertainties. The main purpose of the calibration process is to eliminate large, known systematic errors and thus reduce the measurement uncertainty to some acceptable level. Having decided on the acceptable level, the calibration process achieves that goal by exchanging the large systematic uncertainty of an uncalibrated or poorly calibrated instrument for the smaller combination of systematic uncertainties of the standard instrument and the random uncertainties of the comparison. Calibrations are also used to provide traceability to known reference standards or physical constants, or both. Requirements of military and commercial contracts have led to the establishment of extensive hierarchies of standards laboratories. In some countries, a national standards laboratory is at the apex of these hierarchies, providing the ultimate reference for every standards laboratory. Each additional level in the calibration hierarchy adds uncertainty in the measurement process (see Fig ) Uncertainty Due to Test Article and/or Instrumentation Installation Measurement uncertainty can also exist from interactions between (a) the test instrumentation and the test media or (b) between the test article and test facility. Examples of these types of uncertainty are

25 ASME PTC TEST UNCERTAINTY Fig (a) Interactions Between the Test Instrumentation and Test Media: (1) Installation of sensors in the test media may cause intrusive disturbance effects. An example could be the measurement of airflow in an air conditioning duct. Depending on the design of the pitot static probe, it may affect the measured total and static pressure and thus the calculated airflow. (2) Environmental effects on sensors/instrumentation may exist when the sensors experience environmental effects that are different from those observed during calibration. These may be such things as conduction, convection, and radiation on a sensor when installed in a gas turbine. (b) Interactions Between the Test Article and Test Facility: (1) Test-facility limitations for certification testing affects product measurement uncertainty. An example may be an air conditioner that was bench tested in a laboratory but used in an automotive mechanics shop. The effect of the oily air can influence the quoted rating of the unit. A second example is the testing of a gas turbine engine in an altitude facility. The facility simulates altitude by lowering the ambient pressure at the test article exhaust and raising the inlet pressure at the engine inlet. In appli- Generic Measurement Calibration Hierarchy 14 cation the inlet pressure is elevated due to the ram drag effects of the aircraft. A correction factor must be applied that corrects between uninstalled to installed aircraft engine performance. (2) Facility limitations for testing may require extrapolations to other conditions. An example is the testing of an automotive engine. The fuel consumption of an automotive engine changes with altitude and speed. An automotive test facility may only be able to test at specified altitudes and speeds, and the effects at other altitude conditions may need to be extrapolated Data Acquisition Uncertainty Uncertainty in data acquisition systems can arise from errors in the signal conditioning, the sensors, the recording devices, etc. The best method to minimize the effects of many of these uncertainty sources is to perform overall system calibrations. By comparing known input values with their measured results, estimates of the data acquisition system uncertainty can be obtained. However, it is not always possible to do this. In these cases, it is necessary to evaluate each of the elemental