SIM Regional Comparison on The Calibration of Internal and External Diameter Standards SIM.L-K4.2009 FINAL REPORT July 2012 Theodore Doiron (NIST), J. A. Pires Alves (INMETRO), Bruno R. Gastaldi (INTI), and Guillermo Navarrete (CENAM) Contents 1. Introduction. 2 2. Organization 3 3. Descriptions of the Artifacts.. 4 4. Measurement Instructions and Data Reporting.. 4 5. Measurement Methods and Instruments 5 6. Stability of Artifacts.. 5 7. Measurement Results and Uncertainty Components... 6 8. Reference Value.... 7 9. Report of Results... 7 10. Conclusions. 9 Appendix A: Summary of Data 10 Appendix B: Uncertainty Budgets.. 12
1. Introduction The metrological equivalence of national measurement standards and of calibration certificates issued by national metrology institutes is established by a set of key comparisons chosen and organized by the Consultative Committees of the CIPM or by the regional metrology organizations in collaboration with the Consultative Committees. This regional comparison was performed with the National Institute of Standards and Technology (NIST), as the pilot laboratory. The results of this regional comparison will contribute and be included in the agreement for establishing the metrological equivalence. The interregional CCL key comparison will be combined, where necessary, with regional comparisons following the same protocol. Laboratories participating in both, the interregional and the regional comparisons establish the link between the comparisons and assure their equivalence. 2
2. Organization 2.1 Participants 2.1.1 The general requirement for the participating laboratories is the ability to measure, by any primary means, provided it is a measurement service to clients, the diameter of external diameter standards within the range 2 mm to 100 mm and the diameter of internal diameter standards within the range 5 mm to 100 mm. The uncertainty requirements for the diameter measurements is set at approximately 200 nm at k = 1. 2.2 Participants details Contact Person Ted Doiron Guillermo Navarrete Ing. Bruno R. Gastaldi National Metrology Institute Address NIST Building 220, Rm B113 Gaithersburg, MD 20899 USA CENAM Km 4,5 Carretera a Los Cués El Marqués, Querétaro 76246 MEXICO Instituto Nacional De Tecnologia Industrial Centro Regional Cordoba Metrología Dimensional Argentina Tel: / Fax: Email: Tel. +1-301-975-3472 Fax +1-301-975-8291 e-mail: doiron@nist.gov (52-442) 211-05 00 to 05 est. 3285 e-mail: gnavarre@cenam.mx Teléfono (54 351) 4684835/4698304/4681662/4603974 Fax: (54 351) 4681021/ 4699459 e-mail: gastaldi@inti.gov.ar J. A. Pires Alves INMETRO Av. N. Sra. das Graças, 50 ; Vila Operária; Xerém, Duque de Caxias, CEP.: 25250-020, R.J., Brazil Phone Int 005521-6799036, Fax Int 005521-6791505, e-mail: jaalves@inmetro.gov.br Coordinator: Ted Doiron NIST Building 220, Rm B113 Gaithersburg, MD 20899 USA Tel. +1-301-975-3472 Fax +1-301-975-8291 e-mail: doiron@nist.gov Table 1. Participating laboratories 3
3. Description of the Artifacts 3.1 The package contains 4 ring gages made of steel and 5 cylinders made of steel. The thermal expansion coefficient of the diameter artifacts has been supplied by the manufacturer and is assumed to be 11.5 0.5 10-6 K -1. The artifacts are identified in the following table. Ring gages: Identification Nominal diameter (mm) Expansion coeff. Manufacturer (10-6 K -1 ) 2K97 11.95 11.5 0.5 (k = 1) Glastonbury Gage NIST-3 25.0 11.5 0.5 (k = 1) Glastonbury Gage AP-002/02 46.0 11.5 0.5 (k = 1) Glastonbury Gage NIST-6 75.0 11.5 0.5 (k = 1) Glastonbury Gage Cylinders: Identification Nominal diameter (mm) Expansion coeff. Manufacturer (10-6 K -1 ) A1 3.0 11.5 0.5 (k = 1) Glastonbury Gage PI-002 7.0 11.5 0.5 (k = 1) Glastonbury Gage A4 23.0 11.5 0.5 (k = 1) Glastonbury Gage L97 49.3 11.5 0.5 (k = 1) SIP Table 2. Description of Artifacts 4. Measurement Instructions and Data Reporting 4.1 Diameter standards. 4.1.1 Before measurement, the artifacts have to be inspected for damage of the measurement surfaces, particularly at the gaging points. Any damage must be recorded using the appropriate forms in appendix B of the protocol. 4.1.2 The measurement item of interest is the diametrical distance between the nominal gauge points, defined as mid-elevation along the gauge cylinder and in the diameter direction specified by the engraved marks on the gauge. 4
4.1.3 The measurement results must be appropriately corrected to the reference temperature of 20 C using the thermal expansion coefficients given in this document. Additional corrections have to be applied according to the equipment and procedures used by each laboratory. 4.1.4 If any artifacts are found to have a magnetic condition, the magnetism must be removed per individual laboratory practices before the diameter measurements are performed. Note this condition in the comments on the form in appendix B. 4.1.5 A laboratory may submit measurements from more than one measurement system as long as the timetable is adhered to and that each measurement system is available to general clients for measurement services. 5. Measurement Methods and Instruments CENAM: SIP 305m one axis universal measuring machine calibrated with a laser interferometer. The gauges were calibrated by comparison to master gauge blocks and ring gauges. INMETRO: A coordinate measuring machine was used for comparison to master gauge blocks and rings. The ring gauges were calibrated at PTB. An external laser interferometer was used in place of the machine scales for the displacement measurements. INTI: A SIP 420M length measuring machine was used for comparison to master gauge blocks. An external laser interferometer was used in place of the machine scale for the measurements. NIST: A coordinate measuring machine which has laser interferometers for scales was used for all measurements. A precision sphere was used to calibrate the probe. 6. Stability of the Artifacts The pilot laboratory measured the artifacts twice: at the beginning of the comparison (February 2008) and at the end of the artifact circulation (August 2010). Table 3 shows the results. The artifacts were measured using both the M48 CMM and the 1D comparator or laser micrometer at each re-measurement interval. No relevant damage was observed on the artifacts during the circulation. The observed changes were very small with respect to the uncertainties. 5
Gage ID Nominal (mm) Opening Closing Change Uncertainty ring 2K97 11.95 0.284 0.310 0.026 0.110 ring NIST-3 25-0.250-0.266-0.016 0.120 ring AP-002/02 46 16.495 16.528 0.033 0.130 ring NIST-6 75 0.442 0.459 0.017 0.140 plug A1 3 0.346 0.357 0.011 0.120 plug PI-002 7 0.941 0.912-0.029 0.120 plug A4 23 0.437 0.485 0.048 0.140 plug L97 49.3 1.586 1.648 0.062 0.140 Table 3. Apparent changes in diameter between first and last measurements in micrometers. 7. Measurement Results and Uncertainty Components Tables 4 and 5 show the results of the comparison participants. The figures are the deviations from the nominal size of the artifacts in micrometers. Participants were not given instructions on uncertainty budgets. The number of ways of measuring diameter is so large that it was up to each lab to assign their uncertainty as they chose. Gage ID Nominal NIST INMETRO INTI CENAM NIST (mm) ring 2K97 11.95 0.284 0.330 0.400 0.276 0.310 ring NIST-3 25-0.250-0.490-0.200-0.038-0.266 ring AP-002/02 46 16.495 16.480 16.520 16.543 16.528 ring NIST-6 75 0.442 0.330 0.520 0.443 0.459 plug A1 3 0.346 0.160 0.280 0.404 0.357 plug PI-002 7 0.941 0.750 0.880 1.045 0.912 plug A4 23 0.437 0.520 0.400 0.429 0.485 plug L97 49.3 1.586 1.660 1.790 1.680 1.648 Table 4. Deviations from the nominal diameter as measured by each laboratory in micrometers. 6
Gage ID Nominal NIST INMETRO INTI CENAM NIST (mm) ring 2K97 11.95 0.058 0.130 0.150 0.110 0.058 ring NIST-3 25 0.058 0.140 0.150 0.120 0.058 ring AP-002/02 46 0.060 0.140 0.150 0.130 0.060 ring NIST-6 75 0.063 0.140 0.150 0.140 0.063 plug A1 3 0.022 0.100 0.130 0.120 0.022 plug PI-002 7 0.023 0.100 0.130 0.120 0.023 plug A4 23 0.036 0.100 0.130 0.140 0.036 plug L97 49.3 0.060 0.110 0.130 0.140 0.060 8. Reference Value Table 5. Reported uncertainty (k = 1) of participants in micrometers. The simple mean was used as the reference value. It was calculated using the average of the NIST values and then again using both NIST results as separate entries. The changes were found to be negligible. Further analysis was made using the weighted average for the reference value, and again the differences were found to be negligible. Because of the small number of participants and their general agreement, more sophisticated analysis did not seem warranted. The reference value was taken as the average of all four laboratory results, using the first NIST measurement only. The standard deviation of the mean was used as the uncertainty of the reference value. 9. Report of Results The agreement between the laboratories is presented in Table 6 and Figure 1. Table 7 and Figure 2 give E n values for the measurements. The E n value is defined as E n xi x where xi is the measurement result for laboratory i, with uncertainty 2 k )], 2 [ u( xi )] [ u( x u ( x i ) x is the reference value with uncertainty u (x), and k is the coverage factor which in this report we take to be 1. 7
Gage ID Nominal Reference NIST INMETRO INTI CENAM NIST (mm) Value 2K97 11.95 0.323-0.038 0.007 0.077-0.047-0.012 NIST-3 25-0.245-0.006-0.245 0.045 0.207-0.021 AP-002/02 46 16.509-0.015-0.029 0.011 0.034 0.019 NIST-6 75 0.434 0.008-0.104 0.086 0.009 0.025 A1 3 0.297 0.048-0.137-0.017 0.107 0.060 PI-002 7 0.904 0.037-0.154-0.024 0.141 0.008 A4 23 0.447-0.009 0.073-0.047-0.018 0.039 L97 49.3 1.679-0.093-0.019 0.111 0.001-0.031 Table 6. Deviations from the Reference Value in micrometers. Figure 1. The reference value was taken as the average of all four laboratory values. Only the first value from NIST was used to preserve equal weighting of the results. 8
Artifact and NIST INMETRO INTI CENAM Nominal (mm) 11.95 ring -0.30 0.03 0.25-0.20 25 ring -0.03-0.73 0.13 0.68 46 ring -0.12-0.11 0.03 0.13 75 ring 0.06-0.36 0.28 0.03 3 plug 0.43-0.61-0.06 0.41 7 plug 0.28-0.66-0.08 0.52 23 plug -0.11 0.36-0.18-0.06 49.3 plug -0.63-0.08 0.41 0.00 Table 7. Values for E n. All of the values are within the range ± 1. Figure 2. All of the values of E n (at 68% confidence level) are within the range ± 1. There are no obvious systematic trends in the data. 10. Conclusions All participant s results were consistent within the stated uncertainties. This is a gratifying result that has not been achieved in any previous comparison of diameter measurements. In fact, all 9
values of E n values were less than 0.75, nearly a factor of 3 better than required. These results support the claims of the CMCs of all participants, as will be discussed in more detail in the Executive Report. 10
APPENDIX A Laboratory summary graphs are given below. The error bars are k = 2 uncertainties. 11
NOTE: The NIST data of 8-2010 has been offset slightly in nominal diameter so that both data sets and their uncertainties are visible. 12
Appendix B Uncertainty Budgets Each lab provided detailed uncertainty budgets for their measurements. For laboratories that submitted budgets for each artifact, one budget each for internal and external diameter is presented for comparison. 13
Appendix B1 CENAM Components of uncertainty for: External Diameter 1. External diameters were measured against gage blocks calibrated by interferometry, U(k=2) : 0.080 µm for lengths of 3 mm, 7 mm, 23 mm and 50 mm. 2. The scale used to transfer the source of traceability, a laser 5519A, our estimated U: (0.04+0.36*L) µm, L in mm. 3. Elastic deformation (Ed): for each 1 N of force the gage will deform 50 nm, the measurements were done at 0.5 N, but it is suppose there is a variation of 0.1 N, so the U for the Ed is taken as 25 nm. 4. Artifacts alignment is considered to be 0.05 x10-6 L which is a reasonable value because of the resolution of the maximum value indicator device from the comparator (Tesa modul TTA-80). 5. Uncertainty due to temperature influences is 0.28x10-6 L. 6. The parallelism of the flat feelers, it is estimated to be 70 nm on the whole surface between them (Ø 8 mm). 7. Uncertainty due to repeatability is: 60 nm Plug gages nm x i u(x i ), µm (k = 1) u(xi) i c i = l/ x i u i (l) / nm u i (l) L dependent nm/mm 1 L i 40 24 1 40 -- 2 m (scale laser accuracy) 40 + 0.36 L 78 1 40 0.36L 3 E (contact defoermation) 25 85 1 25 -- 4 am (scale alignment) 0.05 L 75 1 -- 0.05L 5 L( a t a - m t m ) u, u tm 80 1 -- 0.28L 6 p (paralelism) 70 95 1 70 -- 7 Repeatibility 60 85 1 60 -- u c, nm (111 + 0.46 L) nm, L in mm Combined standard uncertainty: u c (l) = (111 + 0.46 L) nm, L in mm (1sigma) 14
CENAM Components of uncertainty for: Internal Diameter 1. Internal diameters were measured against gage blocks (GB) calibrated by interferometry, for lengths of 10 mm. The GB is used as standard in order to get the constant diameter in case of using an internal diameter feeler (1 axis probe head). In case of using a U feelers igb (internal fixed GB) are also used to reach the initialization of the U feeler. The GB u = 75 nm, the diameter of sphere has an uncertainty, u i = 60 nm 2. The scale used to transfer the source of traceablity, a laser 5519A, our estimated u 1s : (40+0.36 L)nm, L in mm. 3. Elastic deformation (Ed): for each 1 N of force the gage will deform 50 nm, the measurements were done at 0.5 N, but it is has been measured there is a variation of 0.2 N, so the u for the Ed is taken as 25 nm. 4. Artifacts alignment is considered to be 0.05x10-6 L which is a reasonable value because of the resolution of the maximum value indicator device from the comparator (Tesa modul TTA-80). 5. Uncertainty due to temperature influences is 0.28x10-6 L. 6. Coaxiality of the U feelers, it is estimated to be 24 nm. Ring gages nm x i u(x i ), µm (k = 1) u(xi) i c i = l/ x i u i (l) / nm ui(l) L dependent nm/mm 1 L i GB standard 75 24 1 75 -- 1 Diameter of probe 1 60 28 1 60 -- axis head 2 m (scale laser accuracy) 3 E (contact defoermation) 40 + 0.36 L 18 1 40 0.36 L 25 85 1 25 -- 4 am (scale alignment) 0.05L 90 1 -- 0.05 L 5 L( a t a - m t m ) u, u tm 60 1 -- 0.28 L 6 p (coaxiality) 24 85 1 24 -- u c, nm (110 + 0.46 L) nm, L in mm Combined standard uncertainty: u c (l) = (110 + 0.46 L) nm, L in mm (1sigma) 15
APPENDIX B2 INMETRO Ring Gage Measurement Uncertainty Example 2K97-11,95mm x i u(x i ) i c i = l/ x i (nm/ m) u i (l) / nm m Laser resolution 0,0029 Infinite 1000 2,8868 Variation in indication 0,0058 Infinite 1000 5,7735 Laser alignment 0,0090 Infinite 1000 9,0211 Gage alignment 0,0000 Infinite 1000 0,0000 Dead path 0,0000 Infinite 1000 0,0000 Probing 0,1014 Infinite 1000 101,3908 Repeatability 0,0111 4 1000 11,0718 Gage positioning 0,0866 Infinite 1000 86,6025 Air temperature C nm/ºc Calibration Certificate 0,0100 Infinite 1,1141E+01 0,1114 Variation in indication 0,0130 Infinite 1,1141E+01 0,1447 Gradient 0,0115 Infinite 1,1141E+01 0,1286 Reading 0,0115 Infinite 1,1141E+01 0,1286 Uncorrected error 0,0035 Infinite 1,1141E+01 0,0386 Air pressure Pa nm/pa Calibration Certificate 25,3979 Infinite -0,032031538 0,8135 Variation in indication 2,2484 Infinite -0,032031538 0,0720 Partial vapor pressure Pa nm/pa Calibration Certificate 11,8694 Infinite 0,004427659 0,0526 Wavelength m nm/ m 0,000000035 Infinite -1,47380E+02 0,0000052 Gage temperature C nm/ºc Calibration Certificate 0,0110 Infinite -1,37425E+02 1,5117 Variation in indication 0,0000 Infinite -1,37425E+02 0,0000 Gradient 0,0043 Infinite -1,37425E+02 0,5951 Reading 0,0006 Infinite -1,37425E+02 0,0793 Uncorrected error 0,0023 Infinite -1,37425E+02 0,3174 Thermal expansion coefficient Combined standard uncertainty (u c (l)) nm = 134 k (~95%) = 2 ºC -1 nm*ºc 0,00000115 Infinite 746875,5798 0,8624 16
A1 3mm INMETRO External Diameter Measurement Example x i u(x i ) ( m) i c i = l/ x i (nm/ m) u i (l) / nm m Laser resolution 0,0029 Infinite 1000 2,8868 Variation in indication 0,0058 Infinite 1000 5,7735 Laser alignment 0,0090 Infinite 1000 9,0211 Gage alignment 0,0000 Infinite 1000 0,0000 Dead path 0,0083 Infinite 1000 8,2994 Probing 0,0410 Infinite 1000 40,9648 Repeatability 0,0296 4 1000 29,5904 Gage positioning 0,0866 Infinite 1000 86,6025 Air temperature C nm/ºc Calibration Certificate 0,0100 Infinite 2,7909E+00 0,0279 Variation in indication 0,0052 Infinite 2,7909E+00 0,0145 Gradient 0,0115 Infinite 2,7909E+00 0,0322 Reading 0,0115 Infinite 2,7909E+00 0,0322 Uncorrected error 0,0035 Infinite 2,7909E+00 0,0097 Air pressure Pa nm/pa Calibration Certificate 25,3979 Infinite -0,008051518 0,2045 Variation in indication -2,2484 Infinite -0,008051518 0,0181 Partial vapor pressure Pa nm/pa Calibration Certificate 11,6023 Infinite 0,001111548 0,0129 Wavelength m 0,000000035 Infinite -3,69991E+01 0,0000013 Gage temperature C nm/ºc Calibration Certificate 0,0110 Infinite -3,45000E+01 0,3795 Variation in indication 0,0014 Infinite -3,45000E+01 0,0498 Gradient 0,0101 Infinite -3,45000E+01 0,3486 Reading 0,0006 Infinite -3,45000E+01 0,0199 Uncorrected error 0,0023 Infinite -3,45000E+01 0,0797 Thermal expansion coefficient Combined standard uncertainty (u c (l)) nm = 101 k (~95%) = 2 ºC -1 nm*ºc 0,00000115 Infinite 135000,1731 0,1559 17
APPENDIX B3 INTI Uncertainty for Internal Diameter (Ring) x i u(x i ) i c i = l/ x i u i (l) / nm Laser Wavelength 3,5E-6 nm 200 101923 0 Index of refraction 1,6E-8 50 6,4E7 1 Number wavelengths ring measured 0,011 13 633 7 Number wavelengths gauge block measured 0,011 13-633 7 Ring expansion coefficient 6,64E-7 1/ C 50-11999988 8 Ring temperature measurement 0,02 C 50-863 20 Gauge block expansion coefficient 4,9E-7 1/ C 50 1,9E6 1 Gauge block temperature measurement 0,03 C 50 89 2 Searching the point of maximum diameter 11,55 nm 50 1 12 Ring form deviation 28,87 nm 50 1 29 Length gauge block 10,1 nm 200 1 10 Variation gauge block 10,5 mm (U-shaped) 23,1 nm 200 1 23 Variation gauge block 19,5 mm (U-shaped) 23,1 nm 200 1 23 Variation gauge block 20 mm (U-shaped) 40,4 nm 200 1 40 Dead path 11,5 nm 13 1 12 Optics thermal drift 57,7 nm 13 1 58 Abbe error, XrY (offset < 4,5 mm) 34,8 nm 13 1 35 Cosine error 1,6 nm 13 1 2 Error U-shaped arrangement 46,2 nm 13 1 46 Laser resolution 2,9 nm 200 1 3 Probe stability 28,9 nm 50 1 29 Probe calibration 28,9 nm 50 1 29 Back to zero (initial point measurement) 57,7 nm 50 1 58 Reproducibility 63,5 nm 50 1 64 Standard deviation ring measurement 17,5 nm 15 1 18 Standard deviation gauge block measurement 22,5 nm 15 1 23 Others (1) -------- ------- --------- 25 Combined standard uncertainty ± 147 nm Expanded uncertainty of measurement (k = 2) ± 296 nm (1) Others minor components: change temperature ring or gauge block during measurement, stability wavelength, optics nonlinearity, pressure, temperature and humidity ambient measurement, etc. 18
INTI Uncertainty for External Diameter (Cylinder) x i u(x i ) i c i = l/ x i u i (l) / nm Laser Wavelength 3,5E-6 nm 200 61316 0 Index of refraction 1,6E-8 50 3,9E7 1 Number wavelengths plug measured 0,011 13 633 7 Number wavelengths gauge block measured 0,011 13-633 7 Plug expansion coefficient 6,64E-7 1/ C 50-24158382 16 Plug temperature measurement 0,02 C 50-567 13 Gauge block expansion coefficient block 4,9E-7 1/ C 50 6,6E6 3 Gauge block temperature measurement 0,02 C 50 89 2 Searching the point of maximum diameter 11,55 nm 50 1 12 Form deviation of plug 28,87 nm 50 1 29 Length gauge block 10,1 nm 200 1 10 Variation gauge block 10,5 mm 23,1 nm 200 1 23 Dead path 11,5 nm 13 1 12 Optics thermal drift 57,7 nm 13 1 58 Abbe error, ZrY (offset < 4,5 mm) 28,8 nm 13 1 29 Abbe error, XrZ (offset < 4,5 mm) 14,8 nm 13 1 15 Cosine error 1,6 nm 13 1 2 Laser resolution 2,9 nm 200 1 3 Probe stability 28,9 nm 50 1 29 Probe calibration 28,9 nm 50 1 29 Back to zero (initial point measurement) 57,7 nm 50 1 58 Reproducibility 52,0 nm 50 1 52 Standard deviation plug measurement 20,1 nm 19 1 20 Standard deviation gauge block measurement 22,4 nm 19 1 22 Others (1) -------- ------- --------- 17 Combined standard uncertainty ± 125 nm Expanded uncertainty of measurement (k = 2) ± 252 nm (1) Others minor components: change temperature plug or gauge block during measurement, stability wavelength, optics nonlinearity, pressure, temperature and humidity ambient measurement, etc. 19
APPENDIX B4 NIST External Diameter Uncertainty Budget Laser Micrometer The NIST wire micrometer was used for small diameter cylinders. It uses a laser interferometer as its scale and dead weight to set the force. The uncertainty budget is primarily from the micrometer s long term reproducibility consisting of check standard measurements with every cylinder calibrated over the last few years. Uncertainty Budget Absolute Measurement of Diameter Standards Source of Uncertainty Analysis Method 1 Equivalent Value (in m) Length independent terms Gage repeatability geometry and roundness effects Gage Performance/Control 0.012 Charts Elastic deformation correction or extrapolation uncertainty 5% of expected results/extrapolation data 0.006 Micrometer contact geometry Rectangular dist. of contact form errors 0.012 Length dependent terms Control artifact Performance/Reproducibility Control Charts 0.50L Laser wavelength 2 x 10-8 m 0.02L Velocity of light correction 5 x 10-8 m 0.05L Air pressure measurement 10 Pa 0.04L Abbe offset/alignment error 0.5 mm x < 0.1 s. 0.50L artifact temperature measurement 11.5ppm x 0.02 C error 0.23L Thermal expansion ( ) uncertainty 0.1ºC x 1ppm 0.10L Thermometer calibration 0.01ºC @ 11.5ppm 0.12L Combined uncertainty u c 0.018 + 0.76 x 10-6 L Expanded uncertainty k = 2 0.036 + 1.52 x 10-6 L 20
NIST Ring gage and large Cylinder Calibrations on M48 The Moore M48 coordinate measuring machine was used for ring gages and large cylinders. It has laser interferometers for all three axes and has been thoroughly monitored since moving to the 0.01 ºC laboratory. The primary uncertainty component is the long term reproducibility of measurements of one dimensional standards (cylinders, ring, end standards and step gages) over the last 7 years. Source Calculation μm Residual Positioning Error Multiple rotations of 2D ball plates, holeplates 0.04 Temperature difference in beam paths during mapping Mapping Laser Frequency Difference Measurement Reproducibility (parts in 10 6 ).02ºC maximum difference 0.01 2 x 10-8 0.02 7 years, 100 s of data pts on rings, plugs, step gages 0.04 0.04 Edlén Equation Internationally accepted 0.03 Index of Refraction Air Temperature ± 0.006ºC beam path meas. accuracy Index of Refraction - Air Pressure ± 10 Pascal meas. accuracy 0.04 Index of Refraction Humidity ± 4% meas. accuracy 0.03 Temperature Accuracy 0.003ºC x 12ppm 0.04 Coefficient of Thermal Expansion 0.05ºC x 1ppm 0.05 Contact Deformation Bi-directional, material 0.002 Gage Surface Geometry Csy generation error 0.004 0.01 Combined uncertainty u c 0.05 μm + 0.10 x 10-6 L Expanded uncertainty k = 2 0.11 μm + 0.20 x 10-6 L 21
REFERENCE [1] Taylor, B.N. and Kuyatt, C.E., Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results, National Institute of Standards and Technology Technical Note 1297, U.S. Government Printing Office, Washington, D.C. (1994). 22