Report on the Key Comparison. CCPR-K2.a Spectral Responsivity in the Range of 900 nm to 1600 nm. Final Report November 30, 2009

Transcription

1 Report on the Key Comparison CCPR-K2.a-2003 Spectral Responsivity in the Range of 900 nm to 1600 nm November 30, 2009 Prepared by Steven W. Brown, homas C. Larason, and Yoshi Ohno Optical echnology Division National Institute of Standards and echnology (NIS) USA

2 ii

3 able of Contents 1. Introduction Organization of the Key Comparison Characterization of the ransfer Detectors prior to the Key Comparison Description of the NIS Spectral Comparator Facility Spectral responsivity of transfer detectors Spatial uniformity of responsivity Linearity emperature dependence Stability abulated Results from Participating Laboratories BNM-INM CSIR CSIRO HU IFA-CSIC KRISS NIM NMi-VSL NPL NRC OMH PB SMU VNIIOFI NIS (Pilot lab) Examination and Correction of the Data emperature correction to the transfer detector responsivities Repeatability of the measurements at NIS Stability of the transfer detectors Instability and anomalous behavior of transfer detectors Uncertainties associated with Laboratory-NIS transfer Repeatability of NIS measurement Long-term stability of the comparison scale Wavelength scale uncertainty Bandwidth effect Beam size Alignment of detector Summary of transfer uncertainties associated with photodiode characteristics Data Analysis Analysis method Results of the Comparison Intermediate calculation results Determination of the Key Comparison Reference Values iii

4 7.3 Birge Ratio Summarized laboratory results with respect to the KCRV Discussion of results Conclusions References Appendix A. Degree of Equivalence ables List of ables able 2.1. List of participants. able 2.2. List of the rounds and photodiodes used. able 2.3. List of transfer detectors and rounds that each detector participated in. able BNM-INM laboratory results for photodiode IGA001 (NIS04). able BNM-INM laboratory results for photodiode IGA002 (NIS08). able BNM-INM laboratory results for photodiode IGA003 (NIS10). able CSIR laboratory results for photodiode IGA010 (NIS04). able CSIR laboratory results for photodiode IGA011 (NIS08). able CSIR laboratory results for photodiode IGA012 (NIS15). able CSIRO laboratory results for photodiode IGA007 (NIS06). able CSIRO laboratory results for photodiode IGA008 (NIS14). able CSIRO laboratory results for photodiode IGA009 (NIS03). able HU laboratory results for photodiode IGA004 (NIS06). able HU laboratory results for photodiode IGA005 (NIS10). able HU laboratory results for photodiode IGA006 (NIS15). able IFA laboratory results for photodiode IGA007 (NIS04). able IFA laboratory results for photodiode IGA008 (NIS03). able IFA laboratory results for photodiode IGA009 (NIS11). able KRISS laboratory results for photodiode IGA004 (NIS17). able KRISS laboratory results for photodiode IGA005 (NIS10). able KRISS laboratory results for photodiode IGA006 (NIS13). able NIM laboratory results for photodiode IGA007 (NIS04). able NIM laboratory results for photodiode IGA008 (NIS11). able NIM laboratory results for photodiode IGA009 (NIS09). able NMi-VSL laboratory results for photodiode IGA013 (NIS16). able NMi-VSL laboratory results for photodiode IGA014 (NIS02). able NMi-VSL laboratory results for photodiode IGA015 (NIS05). able NPL laboratory results for photodiode IGA001 (NIS07). able NPL laboratory results for photodiode IGA002 (NIS08). able NPL laboratory results for photodiode IGA003 (NIS14). able NRC laboratory results for photodiode IGA001 (NIS16). able NRC laboratory results for photodiode IGA002 (NIS14). able NRC laboratory results for photodiode IGA003 (NIS08). able OMH laboratory results for photodiode IGA004 (NIS07). able OMH laboratory results for photodiode IGA005 (NIS11). able OMH laboratory results for photodiode IGA006 (NIS15). iv

5 able PB laboratory results for photodiode IGA007 (NIS18). able PB laboratory results for photodiode IGA008 (NIS11). able PB laboratory results for photodiode IGA009 (NIS01). able SMU laboratory results for photodiode IGA004 (NIS07). able SMU laboratory results for photodiode IGA005 (NIS02). able SMU laboratory results for photodiode IGA006 (NIS10). able VNIIOFI laboratory results for photodiode IGA001 (NIS07). able VNIIOFI laboratory results for photodiode IGA002 (NIS14). able VNIIOFI laboratory results for photodiode IGA003 (NIS09). able NIS results for detectors measured by BNM-INM. able NIS results for detectors measured by CSIR. able NIS results for detectors measured by CSIRO. able NIS results for detectors measured by HU. able NIS results for detectors measured by IFA-CSIC. able NIS results for detectors measured by KRISS. able NIS results for detectors measured by NIM. able NIS results for detectors measured by NMi-VSL. able NIS results for detectors measured by NPL. able NIS results for detectors measured by NRC. able NIS results for detectors measured by OMH. able NIS results for detectors measured by PB. able NIS results for detectors measured by SMU. able NIS results for detectors measured by VNIIOFI. able 5.1. Detector measurement temperatures ( o C). able 5.2. Repeatability of NIS measurements of the transfer detectors. able 5.3. Uncertainty contribution u long s NIS from the longterm stability of the comparison scale. able 5.4 ransfer uncertainty contribution u w,p associated with the wavelength scale uncertainty of NIS SCF. able 5.5 ransfer uncertainty contribution u bp,p associated with the effect of bandpass of NIS SCF. able 5.6 Relative difference (%) in measured responsivity for different beam sizes compared to the 1 mm diameter beam used at NIS able 5.7 ransfer uncertainty contribution u bs,ij (%) associated with the differences in beam size used by the participants and the spatial nonuniformity of transfer photodiodes able BNM-INM additional transfer uncertainty contributions. able CSIR additional transfer uncertainty contributions. able CSIRO additional transfer uncertainty contributions. able HU additional transfer uncertainty contributions. able IFA additional transfer uncertainty contributions. able KRISS additional transfer uncertainty contributions. able NIM additional transfer uncertainty contributions. able NMi-VSL additional transfer uncertainty contributions. able NPL additional transfer uncertainty contributions. able NRC additional transfer uncertainty contributions. able OMH additional transfer uncertainty contributions. v

6 able able able PB additional transfer uncertainty contributions. SMU additional transfer uncertainty contributions. VNIIOFI additional transfer uncertainty contributions. able 7.1. BNM-INM intermediate calculation results (values in %). able 7.2. CSIR intermediate calculation results (values in %). able 7.3. CSIRO intermediate calculation results (values in %). able 7.4. HU intermediate calculation results (values in %). able 7.5. IFA intermediate calculation results (values in %). able 7.6. KRISS intermediate calculation results (values in %). able 7.7. NIM intermediate calculation results (values in %). able 7.8. NMi-VSL intermediate calculation results (values in %). able 7.9. NPL intermediate calculation results (values in %). able NRC intermediate calculation results (values in %). able OMH intermediate calculation results (values in %). able PB intermediate calculation results (values in %). able SMU intermediate calculation results (values in %). able VNIIOFI intermediate calculation results (values in %). able Combined uncertainty u c i for each laboratory (no cut-off applied) able Corresponding weights w i used in the calculation of the KCRV if no cutoff value is applied. able u cut-off at each wavelength. able Combined uncertainty u c, adj i of each lab with cut-off applied. able Corresponding weights w i used in the calculation of the KCRV, with cutoff applied. able Birge ratios calculated with and without cutoff (CSIR and KRISS excluded). able he KCRV KCRV and its uncertainty u KCRV and U KCRV with k=2. able Laboratory differences d i from the KCRV, or unilateral Degree of Equivalence D i. able Standard uncertainties ud i in laboratory difference d i. able Expanded uncertainties U d i in laboratory difference, or U D i in DoE, with coverage factor k=2. able Laboratories reported uncertainties u i. able Laboratory name and associated figure label. able A nm Unilateral and Bilateral Degrees of Equivalence (unit: %) able A nm Unilateral and Bilateral Degrees of Equivalence (unit: %) able A nm Unilateral and Bilateral Degrees of Equivalence (unit: %) able A nm Unilateral and Bilateral Degrees of Equivalence (unit: %) able A nm Unilateral and Bilateral Degrees of Equivalence (unit: %) able A nm Unilateral and Bilateral Degrees of Equivalence (unit: %) able A nm Unilateral and Bilateral Degrees of Equivalence (unit: %) able A nm Unilateral and Bilateral Degrees of Equivalence (unit: %) able A nm Unilateral and Bilateral Degrees of Equivalence (unit: %) able A nm Unilateral and Bilateral Degrees of Equivalence (unit: %) vi

7 able A nm Unilateral and Bilateral Degrees of Equivalence (unit: %) able A nm Unilateral and Bilateral Degrees of Equivalence (unit: %) able A nm Unilateral and Bilateral Degrees of Equivalence (unit: %) able A nm Unilateral and Bilateral Degrees of Equivalence (unit: %) able A nm Unilateral and Bilateral Degrees of Equivalence (unit: %) List of Figures Figure 2.1. Figure 3.1. Figure 3.2. Figure 3.3. Figure 3.4. InGaAs photodiode mounted in package used for the key comparison. Spectral responsivities of photodiodes used in the intercomparison separated by vendor. Responsivity spatial uniformity (0.2 % contours) of representative InGaAs photodetectors from (a) Ge Power Devices, (b) Fermionics, (c) EG&G Optoelectronics, and (d) elcom Devices Corporation at 900 mm, 1250 nm and 1600 nm, respectively. Spectral responsivity linearity measured on the NIS Beam Conjoiner Facility. Average temperature dependence of the responsivity of the InGaAs transfer detectors over the range from 20 o C to 30 o C. Figure 5.1(a) Round 1 transfer detector stability. Figure 5.1(b) Round 2 transfer detector stability. Figure 5.1(c) Round 3 transfer detector stability. Figure 5.1(d) Round 4 transfer detector stability. Figure 5.2 (a). Stability of photodiodes over the course of the measurement comparison. Example results for Ge Power Devices photodiodes are shown. Figure 5.2 (b). Stability of photodiodes over the course of the measurement comparison. Example results for Fermionics photodiodes are shown. Figure 5.2 (c). Stability of photodiodes over the course of the measurement comparison. Example results for EGG Optoelectronics photodiodes are shown Figure 5.2 (d). Stability of photodiodes over the course of the measurement comparison. Example results for elcom Devices photodiodes are shown. Figure 5.3. Change in the responsivity of NIS03 after Round 3. Figure 5.4. Spectral responsivity uniformity of NIS06 measured after Round 3. Figure 5.5. Figure 5.6. Laboratory results, plotted as difference from NIS, separated by vendor. An example of the wavelength scale calibration results for the NIS SCF in the near IR region. Figure 5.7 (a). Relative sensitivity coefficient for one of NIS working standard detectors (GPD). Figure 5.7 (b). Relative sensitivity coefficient for one of the elcom Devices photodiodes Figure 5.8 (a). Relative errors due to bandwidth (4 nm) of the NIS SCF for a typical GPD photodiode. Figure 5.8 (b). Relative errors due to bandwidth (4 nm) of the NIS SCF for one of DC photodiodes. Figure 5.9. Figure 7.1. Responsivity spatial nonuniformity of NIS#13 at 1600 nm. Weighting factors with no cut-off value applied to the data. vii

8 Figure 7.2. Weighting factors with cut-off value applied to the data. Figure 7.3(a). Laboratory differences from the KCRV at 900 nm. Error bars correspond to uncertainties listed in able 7.24 Figure 7.3(b). Laboratory differences from the KCRV at 950 nm. Figure 7.3(c). Laboratory differences from the KCRV at 1000 nm. Figure 7.3(d). Laboratory differences from the KCRV at 1050 nm. Figure 7.3(e). Laboratory differences from the KCRV at 1100 nm. Figure 7.3(f). Laboratory differences from the KCRV at 1150 nm. Figure 7.3(g). Laboratory differences from the KCRV at 1200 nm. Figure 7.3(h). Laboratory differences from the KCRV at 1250 nm. Figure 7.3(i). Laboratory differences from the KCRV at 1300 nm. Figure 7.3(j). Laboratory differences from the KCRV at 1350 nm. Figure 7.3(k). Laboratory differences from the KCRV at 1400 nm. Figure 7.3(l). Laboratory differences from the KCRV at 1450 nm. Figure 7.3(m). Laboratory differences from the KCRV at 1500 nm. Figure 7.3(n). Laboratory differences from the KCRV at 1550 nm. Figure 7.3(o). Laboratory differences from the KCRV at 1600 nm. Figure 7.4(a). BNM-INM differences from the KCRV. Figure 7.4(b). CSIR differences from the KCRV. Figure 7.4(c). CSIRO differences from the KCRV. Figure 7.4(d). HU differences from the KCRV. Figure 7.4(e). IFA differences from the KCRV. Figure 7.4(f). KRISS differences from the KCRV. Figure 7.4(g). NIM differences from the KCRV. Figure 7.4(h). NIS differences from the KCRV. Figure 7.4(i). NMi-VSL differences from the KCRV. Figure 7.4(j). NPL differences from the KCRV. Figure 7.2(k) NRC differences from the KCRV. Figure 7.4(l). OMH differences from the KCRV. Figure 7.4(m). PB differences from the KCRV. Figure 7.4(n). SMU differences from the KCRV. Figure 7.4(o). VNIIOFI differences from the KCRV. Figure 8.1. Laboratory differences d i from the KCRV listed in able Figure 8.2. Magnification (-1.0 % to 1.0 %) of the Laboratory differences d i from the KCRV listed in able Figure 8.3. Laboratories reported uncertainties u i listed in able Figure 8.4 (a). Comparison of the results between CCPR K2.a and CCPR K2.b (single photodiodes) at 900 nm. Figure 8.4 (b). Comparison of the results between CCPR K2.a and CCPR K2.b (single photodiodes) at 950 nm. Figure 8.4 (c). Comparison of the results between CCPR K2.a and CCPR K2.b (single photodiodes) at 1000 nm. viii

9 1. Introduction he Mutual Recognition Arrangement (MRA) for national measurements standards and for calibration and measurement certificates issued by National Metrology Institutes (NMIs) was signed in 1999 by the directors of the NMIs of 38 member states of the Metre Convention and representatives of two international organizations. he objectives of the MRA were to establish the degree of equivalence of national measurements standards maintained by NMIs; to provide for the mutual recognition of calibration and measurement certificates issued by NMIs; and to thereby provide governments and other parties with a secure technical foundation for wider agreements related to international trade, commerce and regulatory affairs. Under the MRA, the metrological equivalence of national measurement standards is to be determined by a set of key comparisons chosen and organized by the Consultative Committees of the International Committee for Weights and Measures (Comité International des Poids et Mesures, CIPM) working closely with Regional Metrology Organizations (RMOs). At the annual meeting of the Consultative Committee for Photometry and Radiometry (CCPR) in March 1997, several Key Comparisons were identified in the field of optical radiation metrology, one of which, K2, was spectral responsivity. he Key Comparison of Spectral Responsivity was divided into three separate comparisons covering different spectral regions: K2.a (900 nm to 1600 nm), K2.b (300 nm to 1000 nm), and K2.c (200 nm to 400 nm). he NMI of the United States of America, the National Institute of Standards and echnology (NIS), was chosen as the Pilot Laboratory for K2.a. A small working group of NMIs participating in the Key Comparison was responsible for the organization of the Key Comparison and development of the echnical Protocol. he working group was comprised of representatives from the National Physical Laboratory, United Kingdom (NPL); the Physikalish-echnische Bundesanstalt, Germany (PB); the National Research Council, Canada (NRC), and the NIS. he Pilot Laboratory was responsible for purchasing and characterizing the transfer artifacts used in the Key Comparison. 2. Organization of the Key Comparison 15 laboratories including the pilot laboratory participated in this comparison. able 2.1 shows the list of participants. he names of some of the laboratories have changed since the start of this comparison. In this document, the original names of the laboratories at the time of the start of this comparison are used. he Key Comparison K2.a was designed to cover the spectral range from 900 nm to 1600 nm. hree transfer standard artifacts were sent to each participating laboratory. Laboratories were asked to make three independent measurements of each artifact, for a total of nine spectral responsivity measurements over the full spectral range. Participating laboratories were requested to make spectral responsivity measurements every 50 nm. An incoherent source, with incident power in the range from 1 µw to 100 µw and a beam diameter of 1 mm to 3 mm, was recommended. 5 mm diameter indium gallium arsenide (InGaAs) photodiodes with sapphire windows were selected as transfer artifacts. he photodiodes were mounted in 50.8 mm diameter circular holders (Fig.2.1). A thermistor was included in the mount in close proximity to the photodiode to monitor the temperature. wo standard BNC-type connections were provided on the back plate to measure the signal and the thermistor resistance. he Comparison was organized in a star pattern. ransfer detectors were first measured at NIS, then sent to groups of participating laboratories. hey were again measured upon their return to NIS. here were a total of four rounds in the Comparison. he comparison 1

10 measurements for the four rounds took place in the period from January 1999 to February 2001; each round took approximately 6 months to complete. Each photodiode mount was engraved with a serial number starting with 'NIS' for recordkeeping. o ensure the validity of the independent measurements by participating laboratories, a cover plate was placed over the mount serial number with a second serial number starting with 'IGA'. Records were kept of both IGA and NIS serial numbers for artifacts sent to each participating laboratory. Laboratories reported measurements for photodiodes with IGA serial numbers. able 2.2 shows the list of the rounds each laboratory participated in, the 'IGA' serial numbers of the transfer detectors measured by each laboratory, and the corresponding 'NIS' serial numbers. he transfer detectors, identified by their 'NIS' serial number, the manufacturer and the rounds that each photodiode participated in are listed in able 2.3. able 2.1. List of participants Laboratory Contact Person(s) *6 BNM-INM/CNAM (France) *1 Jeanne-Marie Coutin CSIR (South Africa) *2 CSIRO (Australia) *3 HU (Finland) *4 IFA-CSIC (Spain) KRISS (South Korea) NIS (USA) - Pilot NIM (China) NMi-VSL (Netherlands) NPL (Great Britain) NRC (Canada) OMH (Hungary) *5 PB (Germany) SMU (Slovakia) Bertus heron, Natasha Nel-Sakharova, Elsie Coetzee James Gardner, Peter Manson Erkki Ikonen, Farshid Manoocheri Antonio Corrons Young Boong Chung, Seung Nam Park, Dong-Hoon Lee Steven Brown, Yoshi Ohno Lin YanDong Charles Schrama, Eric van der Ham Nigel Fox, Bill Hartree Phil Boivin, Charles Bamber Gyula Dezsi, George Andor Lutz Werner Peter Nemecek Victor Sapritsky, Raisa Stolyarevskaya VNIIOFI (Russia) Boris Khlevnoy *1 now known as Laboratoire Natioinal de Métrologie et d'essais / Institut National de Métrologie (LNE-INM). *2 now known as National Metrology Institute of South Africa (NMISA). *3 now known as National Metrology Institute of Australia (NMIA). *4 now known as Metrology Research Institute, Centre for Metrology and Accreditation (MIKES). *5 now known as Hungarian rade Licensing Office (MKEH). *6 All the names are listed if the contact person changed during the course of the comparison including report preparation. 2

11 able 2.2. List of the rounds and photodiodes used. Laboratory IGA photodiode Serial number NIS photodiode Serial number Round 1 NPL HU IFA-CSIC Round 2 BNM-INM OMH CSIRO Round 3 NRC SMU NIM Round 4 VNIIOFI KRISS PB CSIR NMi-VSL Figure 2.1. InGaAs photodiode mounted in package used for the key comparison. 3

12 able 2.3. List of transfer detectors and rounds that each detector participated in. Manufacturer NIS serial # Round Fermionics 01 4 Fermionics Fermionics Used in all Rounds Ge Power Devices X Ge Power Devices 05 4 Ge Power Devices Ge Power Devices X EG&G Optoelectronics X EG&G Optoelectronics EG&G Optoelectronics X EG&G Optoelectronics X elcom Devices Corp. 12 Never elcom Devices Corp elcom Devices Corp X elcom Devices Corp Ge Power Devices Ge Power Devices 17 4 Ge Power Devices Characterization of the ransfer Detectors prior to the Key Comparison A total of 18 detectors from 4 different commercial vendors were used in the comparison. Prior to the start of the comparison, the detector spectral responsivities were measured and they were characterized for responsivity uniformity, linearity, stability, and temperature dependence. he polarization dependence was not measured. 3.1 Description of the NIS Spectral Comparator Facility Measurements were made on the Visible to Near-Infrared Spectral Comparator Facility (Vis/NIR SCF) [Larason 1998]. he facility was designed to measure the spectral power responsivity of test artifacts over the spectral range from 350 nm to 1800 nm. From 700 nm to 1800 nm, InGaAs photodiodes are typically used as working standard detectors. Prior to the start of the Key Comparison, the near infrared spectral responsivity scale was derived on a series of reference standard InGaAs detectors by comparison against a cryogenic radiometer [Shaw 2000]. A constant-voltage-controlled, 100 W quartz-tungsten halogen lamp was used as the excitation source of radiant flux. he source is imaged onto the entrance slit of a modified Cary-14 prismgrating monochromator. he monochromator uses a 30 o fused silica prism in series with a 600 line per mm echelette grating. he monochromator slits are set for a 4 nm full-width half 4

13 maximum (FWHM) bandwidth. he instrument has a stray light rejection of approximately 10-8 ; the exit beam is f/9. A circular 1.1 mm diameter aperture located just after the monochromator exit slit determines the beam size. he exit aperture is imaged onto the detector with 1:1 magnification using two cm diameter spherical mirrors and two 7.62 cm diameter flat mirrors. wo orthogonal linear positioning stages are used to translate the test and working standard detectors in front of the excitation source. Calibrations against the reference standard detectors are done using direct substitution; variations in the source intensity during a calibration are corrected using a beam splitter and monitor detector located after the monochromator. he detectors, translation stages and imaging optics are located inside a light-tight enclosure to eliminate ambient background light from the environment. 3.2 Spectral responsivity of transfer detectors he spectral responsivity of each photodiode used in the comparison was measured just prior to the start of the comparison. In Fig. 3.1, the spectral responsivities of the detectors are shown, separated by vendor. Fermionics InGaAs Photodiode Responsivity Summary Ge Power Devices InGaAs Photodiode Responsivity Summary Responsivity Responsivity NIS 01 Responsivity NIS 02 Responsivity NIS 03 Responsivity Fermionics Average Responsivity 0.2 NIS 04 Responsivity NIS 05 Responsivity NIS 06 Responsivity NIS 07 Responsivity GPD Average Responsivity Wavelength Wavelength EG&G Optoelectronics InGaAs Photodiode Responsivity Summary elcom Devices Corp. InGaAs Photodiode Responsivity Summary Responsivity Responsivity NIS 08 Responsivity NIS 09 Responsivity NIS 10 Responsivity NIS 11 Responsivity EG&G Optoelectronics Average Responsivity 0.2 NIS 12 Responsivity NIS 13 Responsivity NIS 14 Responsivity NIS 15 Responsivity DC Average Responsivity Wavelength Wavelength Figure 3.1. Spectral responsivities of photodiodes used in the intercomparison separated by vendor. 5

14 3.3 Spatial uniformity of responsivity he spectral responsivity uniformity of each transfer artifact was measured at 900 nm, 1250 nm and 1600 nm. Representative results for one photodiode from each manufacturer are shown in Fig Note the large decrease in the photodiode responsivity uniformity at 900 nm and 1600 nm compared with the uniformity at 1250 nm. No corrections were made to the InGaAs photodiode responsivities to account for differences in incident beam sizes by participating laboratories. 3.4 Linearity he linearity of all transfer and working standard InGaAs detectors was measured on the NIS Beam Conjoiner Facility [Yoon 2003]. In these measurements, the detectors were underfilled with broad-band illumination; the incident power was changed over 4 orders of magnitude (in the power range of interest for the comparison) using the flux addition method. No dependence on incident power was observed; typical results are shown in Fig 3.3. While noting the possibility of a spectral dependence to the linearity below 1000 nm [Fox 1993, Boivin 2000], no linearity correction was applied to the responsivity of the photodiodes. 3.5 emperature dependence A special thermo-electrically controlled copper mount was designed to measure the temperature dependence of the responsivity of the InGaAs transfer detectors. Using this mount, the spectral responsivities of several transfer artifacts were measured from 20 o C to 30 o C. he average temperature dependence of the responsivities of two photodiodes used in the comparison from each vendor is shown in Fig 3.4. he results for other photodiodes were similar. Because of the large observed temperature dependence, temperature corrections were applied to the data set (see Section 5.1) according to the recommendation by the K2.a Working Group. 3.6 Stability Prior to the start of the comparison, the responsivities and uniformities of several photodiodes were measured, separated in time by 6 months, to assess their short-term stability and compatibility with the Key Comparison protocols. No changes within the measurement reproducibility were observed in any of the photodiodes over that time period. he stabilities of the transfer detectors were also monitored during the comparison (see Sections 5.3). 6

15 900 nm 1250 nm 1600 nm (a) y [mm] y [mm] y [mm] x [mm] x [mm] x [mm] (b) y [mm] y [mm] y [mm] x [mm] x [mm] x [mm] (c) y [mm] y [mm] y [mm] x [mm] x [mm] x [mm] (d) y [mm] y [mm] y [mm] x [mm] x [mm] x [mm] Figure 3.2. Responsivity spatial uniformity (0.2 % contours) of representative InGaAs photodetectors from (a) Ge Power Devices, (b) Fermionics, (c) EG&G Optoelectronics, and (d) elcom Devices Corporation at 900 mm, 1250 nm and 1600 nm, respectively. All plots are on the same scale, from 99 % to 101 %, with 100 % at the center of the photodiode. 7

16 InGaAs NIS01 (Fermionics), FEL Signal/ Flux E E E E E E E E-03 Signal (A) 07094a0 (10^6) 06200a1 (10^4) 07126a3 (10^7) (a) Fermionics photodiode, typical shunt resistance, 15 M. InGaAs NIS18, (Ge Power Devices), FEL Signal/ Flux E E E E E E E E-03 Signal (A) 06222a0 (10^4) 06255a4 (10^6) 06273a0 (10^7) (b) Ge Power Devices photodiode, typical shunt resistance, 2 M. Figure 3.3. Spectral responsivity linearity measured on the NIS Beam Conjoiner Facility. 8

17 Figure 3.4. Average temperature dependence of the responsivity of the InGaAs transfer detectors over the range from 20 o C to 30 o C. 4. abulated Results from Participating Laboratories In this Section, reported results from the participating Laboratories including the results of the pilot laboratory are tabulated. For each Laboratory, the base of the radiometric scale, the source of the radiant flux, the instrument bandpass, the beam size and geometry, and the incident power are reported in a header section. Next, measurement dates are given, along with thermistor readings at the time of the measurement (in k as well as calculated temperatures). Finally, results from each Laboratory are tabulated according to the detector measured. In each able, the wavelength, the responsivity and the relative combined standard uncertainty are reported for each of the three measurements, as required by the protocol. hen the average responsivity and the average uncertainty are reported. he values in these tables are the original results reported by the participants and not corrected for temperature differences. 9

18 4.1 BNM-INM Base of the radiometric scale: Cryogenic radiometer Source: Quartz halogen lamp Bandpass: 10 nm Beam Size: 3 mm Beam Geometry: Incident Power: 10 W to 20 W Laboratory Results: able BNM-INM laboratory results for photodiode IGA001 (NIS04). Measurement dates: Not reported. hermistor (k ): emperature ( o C) W L s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 10

19 able BNM-INM laboratory results for photodiode IGA002 (NIS08). Measurement dates: Not reported. hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg able BNM-INM laboratory results for photodiode IGA003 (NIS10). Measurement dates: hermistor (k ): emperature ( o C) u avg s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 11

20 4.2 CSIR Base of the radiometric scale: Room temperature absolute radiometer (electrical substitution radiometer) Source: Quartz-halogen Bandpass: 5 nm Beam Size: 3 mm for absolute points against absolute radiometer and 2 mm for relative measurements against flat response detector. Beam Geometry: Incident beam convergent at detector surface. Half-angle subtended 6.5 degrees. Incident Power: s 1 = 3 µw to 21 µw; s 2 = 3 µw to 21 µw; s 3 = 14 µw to 22 µw Laboratory Results: able CSIR laboratory results for photodiode IGA010 (NIS04). Measurement dates: 22-Sep Sep Sep-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg

21 able CSIR laboratory results for photodiode IGA011 (NIS08). Measurement dates: 22-Sep Sep Sep-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg able CSIR laboratory results for photodiode IGA012 (NIS15). Measurement dates: 25-Sep Sep Sep-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg

22 4.3 CSIRO Base of the radiometric scale: Cryogenic radiometer Source: ungsten halogen lamp operating at 15V Bandpass: 4 nm Beam Size: 1 mm x 3 mm Beam Geometry: Rectangular image of the monochromator exit slit formed on the photodiode surface, where the beam is converging at f/8. Incident Power: 10 µw (approx.) Details on the derivation of the uncertainty budget are provided in: "he NML units of spectral responsivity nm", Frank Wilkinson, CSIRO echnical Memorandum No. 253, (PM-HAEA-F18/9). Laboratory Results: able CSIRO laboratory results for photodiode IGA007 (NIS06). Measurement dates: 12-Aug Aug Aug-99 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg Note: he third measurement results (S3) of this photodiode were found to be duplication of the 2nd measurement results by an error by the participant. Since the results were submitted this way and approved with no correction after review of Draft A, no corrections were made in the final report. he effect of this error is considered negligible. u avg 14

23 able CSIRO laboratory results for photodiode IGA008 (NIS14). Measurement dates: 12-Aug Aug Aug-99 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg able CSIRO laboratory results for photodiode IGA009 (NIS03). Measurement dates: 12-Aug Aug Aug-99 hermistor (k ): emperature ( o C) u avg s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 15

24 4.4 HU Base of the radiometric scale: Cryogenic absolute radiometer Source: Bandpass: 2.0 nm Beam Size: 1.0 mm x 1.7 mm Beam Geometry: A full cone angle of 10 degrees was used for the measurements. Incident Power: 2 W and 10 W Laboratory Results: able HU laboratory results for photodiode IGA004 (NIS06). Measurement dates: 4-May-99 6-May-99 8-May-99 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 16

25 able HU laboratory results for photodiode IGA005 (NIS10). Measurement dates: 4-May-99 5-May-99 7-May-99 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg able HU laboratory results for photodiode IGA006 (NIS15). Measurement dates: 2-May-99 5-May-99 7-May-99 hermistor (k ): emperature ( o C) u avg s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 17

26 4.5 IFA-CSIC Base of the radiometric scale: Electrically calibrated pyroelectric radiometer Source: Halogen lamp and monochromator Bandpass: 10 nm Beam Size: 3.2 mm Beam Geometry: Incident Power: 18 µw to 67 µw Laboratory Results: able IFA laboratory results for photodiode IGA007 (NIS04). Measurement dates: 4-Apr-99 5-Apr-99 6-Apr-99 hermistor (k ): Error in reported results. s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 18

27 able IFA laboratory results for photodiode IGA008 (NIS03). Measurement dates: 4-Apr-99 5-Apr-99 6-Apr-99 hermistor (k ): Error in reported results. s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg able IFA laboratory results for photodiode IGA009 (NIS11). Measurement dates: 4-Apr-99 5-Apr-99 6-Apr-99 hermistor (k ): Error in reported results. u avg s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 19

28 4.6 KRISS Base of the radiometric scale: Spectrally flat pyroelectric detector calibrated with cryogenic radiometer. Source: ungsten halogen lamp and double monochromator Bandpass: 6 nm Beam Size: 2 mm diameter Beam Geometry: Circular Incident Power: 9 µw to 25 µw Laboratory Results: able KRISS laboratory results for photodiode IGA004 (NIS17). Measurement dates: 20-Nov Nov Nov-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 20

29 able KRISS laboratory results for photodiode IGA005 (NIS10). Measurement dates: 20-Nov Nov Nov-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg able KRISS laboratory results for photodiode IGA006 (NIS13). Measurement dates: 20-Nov Nov Nov-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg u avg 21

30 4.7 NIM Base of the radiometric scale: cryogenic radiometer and pyroelectric power meter Source: 250 W quartz halogen lamp Bandpass: 6 nm Beam Size: 3 mm Beam Geometry: f/3.9 Incident Power: 7.8 W to 80 W Laboratory Results: able NIM laboratory results for photodiode IGA007 (NIS04). Measurement dates: 26-Apr Apr Apr-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 22

31 able NIM laboratory results for photodiode IGA008 (NIS11). Measurement dates: 26-Apr Apr Apr-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg able NIM laboratory results for photodiode IGA009 (NIS09). Measurement dates: 26-Apr Apr Apr-00 hermistor (k ): emperature ( o C) u avg s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 23

32 4.8 NMi-VSL Base of the radiometric scale: Source: 100 W quartz tungsten halogen lamp Bandpass: 9 nm Beam Size: 4 mm in diameter Beam Geometry: f/8 Incident Power: 32 µw to 98µW Laboratory Results: able NMi-VSL laboratory results for photodiode IGA013 (NIS16). Measurement dates: 15-Aug Aug Aug-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 24

33 able NMi-VSL laboratory results for photodiode IGA014 (NIS02). Measurement dates: 15-Aug Aug Aug-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg able NMi-VSL laboratory results for photodiode IGA015 (NIS05). Measurement dates: 15-Aug Aug Aug-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 25

34 4.9 NPL Base of the radiometric scale: Cryogenic radiometer Source: ungsten strip lamp Bandpass: 6 nm Beam Size: 3.0 mm in diameter Beam Geometry: f/7 converging Incident Power: 1 µw to 5 µw Laboratory Results: able NPL laboratory results for photodiode IGA001 (NIS07). Measurement dates: 4-Mar-99 9-Mar Mar-99 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 26

35 able NPL laboratory results for photodiode IGA002 (NIS08). Measurement dates: 4-Mar-99 9-Mar Mar-99 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg able NPL laboratory results for photodiode IGA003 (NIS14). Measurement dates: 8-Mar Mar Mar-99 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg u avg 27

36 4.10 NRC Base of the radiometric scale: Cryogenic radiometer Source: 100W tungsten halogen lamp Bandpass: 5 nm Beam Size: 3 mm X 1.5 mm Beam Geometry: he angular divergence of the radiation is about ±2.6 (f/11). Incident Power: 9 µw to 22 µw Laboratory Results: able NRC laboratory results for photodiode IGA001 (NIS16). Measurement dates: 15-Feb Feb Feb-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 28

37 able NRC laboratory results for photodiode IGA002 (NIS14). Measurement dates: 15-Feb Feb Feb-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg able NRC laboratory results for photodiode IGA003 (NIS08). Measurement dates: 15-Feb Feb Feb-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 29

38 4.11 OMH Base of the radiometric scale: PQE detector in the visible, nonselective pyroelectric trap detector in the IR. Source: 1000 W tungsten halogen lamp Bandpass: 4.1 nm to 4.3 nm Beam Size: 2 mm diameter Beam Geometry: f /8 to f /11 Incident Power: 1 µw Laboratory Results: able OMH laboratory results for photodiode IGA004 (NIS07). Measurement dates: 12-Aug-99 6-Sep Sep-99 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 30

39 able OMH laboratory results for photodiode IGA005 (NIS11). Measurement dates: 13-Aug-99 7-Sep Sep-99 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg able OMH laboratory results for photodiode IGA006 (NIS15). Measurement dates: 8-Sep-99 9-Sep Sep-99 hermistor (k ): emperature ( o C) u avg s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 31

40 4.12 PB Base of the radiometric scale: Laser-based cryogenic radiometer Source: quartz tungsten halogen lamp Bandpass: < 1400 nm: 6.4 nm; >= 1400 nm: 12.9 nm Beam Size: 1.3 mm diameter at half maximum, 2.3 mm diameter at half maximum Beam Geometry: Circular Gaussian like profile, beam divergence: horizontal full angle = 10, vertical full angle = 4, circular flat top profile, beam divergence: horizontal full angle = 2, vertical full angle = 2. Incident Power: 3 µw to 13 µw, 5 µw to 22 µw, 1 µw to 4 µw Laboratory Results: able PB laboratory results for photodiode IGA007 (NIS18). Measurement dates: 28-Aug Oct Oct-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 32

41 able PB laboratory results for photodiode IGA008 (NIS11). Measurement dates: 28-Aug Oct Oct-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg able PB laboratory results for photodiode IGA009 (NIS01). Measurement dates: 28-Aug Oct Oct-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg

42 4.13 SMU Base of the radiometric scale: Absolute pyroelectric radiometer with electrical calibration model Rs-3964 and trap silicon photodiode detector QED-200 Source: 100 W halogen lamp with the monochromator DM 300 Bandpass: 6 nm SHBW Beam Size: 3 mm diameter, Beam Geometry: divergent, 7 o full angle Incident Power: 10 W to 40 W Laboratory Results: able SMU laboratory results for photodiode IGA004 (NIS07). Measurement dates: 21-Feb Feb Feb-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 34

43 able SMU laboratory results for photodiode IGA005 (NIS02). Measurement dates: 21-Feb Feb Feb-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg able SMU laboratory results for photodiode IGA006 (NIS10). Measurement dates: 21-Feb Feb Feb-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg

44 4.14 VNIIOFI Base of the radiometric scale: MAR-1 absolute radiometer Source: tungsten ribbon lamp RU type Bandpass: 2 nm Beam Size: 2 mm x 3 mm Beam Geometry: f/3 Incident Power: 1 µw to 10 µ W Laboratory Results: able VNIIOFI laboratory results for photodiode IGA001 (NIS07). Measurement dates: 20-Jul Aug Aug-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg 36

45 able VNIIOFI laboratory results for photodiode IGA002 (NIS14). Measurement dates: 26-Jul Aug Aug-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg able VNIIOFI laboratory results for photodiode IGA003 (NIS09). Measurement dates: 18-Jul Aug Aug-00 hermistor (k ): emperature ( o C) s 1 u(s 1 ) s 2 u(s 2 ) s 3 u(s 3 ) s avg u avg u avg

46 4.15 NIS (Pilot lab) Base of the radiometric scale: cryogenic radiometer Source: 100 W quartz tungsten halogen lamp Bandpass: 4 nm Beam Size: 1.1 mm in diameter Beam Geometry: f/9 Incident Power: 0.1 µw to 0.6 µw Results 38

47 able NIS results for detectors measured by BNM-INM. Average responsivity IGA001 (NIS04) IGA002 (NIS08). IGA003 (NIS10) Difference = (Post-Pre) /Pre Average responsivity Difference = (Post-Pre) /Pre Average responsivity Difference = (Post-Pre) /Pre u stability u stability u stability able NIS results for detectors measured by CSIR. Average responsivity IGA010 (NIS04). IGA011 (NIS08). IGA012 (NIS15). Difference = (Post-Pre) /Pre Average responsivity Difference = (Post-Pre) /Pre Average responsivity Difference = (Post-Pre) /Pre u stability u stability u stability

48 able NIS results for detectors measured by CSIRO. Average responsivity IGA007 (NIS06) IGA008 (NIS14) IGA009 (NIS03) Difference = (Post-Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre u stability u stability u stability able NIS results for detectors measured by HU. Average responsivity IGA004 (NIS06) IGA005 (NIS10) IGA006 (NIS15) Difference = (Post-Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre u stability u stability u stability

49 able NIS results for detectors measured by IFA. Average responsivity IGA007 (NIS04) IGA008 (NIS03) IGA009 (NIS11) Difference = (Post-Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre u stability u stability u stability able NIS results for detectors measured by KRISS. Average responsivity IGA004 (NIS17) IGA005 (NIS10). IGA006 (NIS13) Difference = (Post-Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre u stability u stability u stability

50 able NIS results for detectors measured by NIM. Average responsivity IGA007 (NIS04). IGA008 (NIS11). IGA009 (NIS09) Difference = (Post-Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre u stability u stability u stability able NIS results for detectors measured by NMi-VSL. Average responsivity IGA013 (NIS16). IGA014 (NIS02) IGA015 (NIS05) Difference = (Post-Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre u stability u stability u stability

51 able NIS results for detectors measured by NPL. Average responsivity IGA001 (NIS07) IGA002 (NIS08) IGA003 (NIS14). Difference = (Post-Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre u stability u stability u stability able NIS results for detectors measured by NRC. Average responsivity IGA001 (NIS16). IGA002 (NIS14). IGA003 (NIS08). Difference = (Post-Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre u stability u stability u stability

52 able NIS results for detectors measured by OMH. Average responsivity IGA004 (NIS07) IGA005 (NIS11) IGA006 (NIS15) Difference = (Post-Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre u stability u stability u stability able NIS results for detectors measured by PB. Average responsivity IGA007 (NIS18) IGA008 (NIS11) IGA009 (NIS01) Difference = (Post-Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre u stability u stability u stability

53 able NIS results for detectors measured by SMU. Average responsivity IGA004 (NIS07) IGA005 (NIS02) IGA006 (NIS10) Difference = (Post-Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre u stability u stability u stability able NIS results for detectors measured by VNIIOFI. Average responsivity IGA001 (NIS07) IGA002 (NIS14) IGA003 (NIS09) Difference = (Post-Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre Average responsivity Difference = (Post- Pre) /Pre u stability u stability u stability

54 5. Examination and Correction of the Data 5.1 emperature correction to the transfer detector responsivities After circulation of Draft A-1, due to relatively large differences in laboratory temperatures of participants, the K2.a Working Group decided to apply temperature corrections for all results based on the thermistor readings for Draft A-2. he average temperature-dependence of the responsivities of detectors from each of the four different vendors (Fig. 3.4) was used to correct the measurements by participating laboratories and the measurements at NIS after a round of measurements. he responsivities of the detectors were corrected for recorded temperature differences from the measurements at NIS prior to a measurement round. he thermistor values recorded during each set of measurements were converted to temperatures; able 5.1 lists the temperatures for each transfer detector measured at NIS before and after measurements at the participating laboratory, as well as the temperatures recorded by each laboratory. One laboratory, IFA, had an error in their thermistor measurements; no temperature data were available for those measurements. One measurement at NIS, the measurement of detector NIS01 following measurements by the PB, was considerably lower than all other NIS temperature measurements. his data point was treated as an outlier and no temperature correction was made to the post NIS measurements of the detector. here was an error in the temperature of NIS measurement of detector NIS13 before being measured by KRISS. he post-measurement temperature was used to correct the KRISS measurements. 46

55 able 5.1. Detector measurement temperatures ( o C). Laboratory Detector Laboratory Measurement NIS Pre NIS Post Vendor BNM-INM NIS GPD NIS EGG NIS EGG CSIR NIS GPD NIS EGG NIS DC CSIRO NIS GPD NIS DC NIS Fermionics HU NIS GPD NIS EGG NIS DC IFA NIS GPD NIS Fermionics NIS EGG KRISS NIS GPD NIS EGG NIS13 xxx DC NIM NIS GPD NIS EGG NIS EGG NMi-VSL NIS GPD NIS Fermionics NIS GPD NPL NIS GPD NIS EGG NIS DC NRC NIS GPD NIS DC (Continued) NIS EGG OMH NIS GPD NIS EGG NIS DC PB NIS GPD NIS EGG NIS Fermionics SMU NIS GPD NIS Fermionics NIS EGG VNIIOFI NIS GPD NIS DC NIS EGG 47

56 5.2 Repeatability of the measurements at NIS An average of the standard deviations of the mean of the three measurements of all the transfer detectors measured at NIS was used as the repeatability of the NIS measurements. he results are summarized in able 5.2. he last column shows the corresponding values for a total of six measurements for each photodiode including before and after transportation. able 5.2. Repeatability of NIS measurements of the transfer detectors. Std. dev. of the mean of 6 measurements (pre, post) 1 photodiode Wavelength Std. dev. of the mean of 3 measurements, 1 photodiode u r s NIS Stability of the transfer detectors Differences in the transfer detectors responsivities measured at NIS Before (pre) and After (post) participants measurements in each round were taken into account as an uncertainty component in the Laboratory i to NIS ratio values. he difference was converted to a standard uncertainty by assuming a rectangular probability distribution: u stability (s ij NIS ) s ij NIS pre s NIS ij post s ij NIS pre. (5.1) s ij NIS is the average of NIS-measured responsivity, before and after the transfer, of the photodiode j measured by laboratory i during the comparison (see eq. 6.3). Figure 5.1 (a, b, c, d) shows the uncertainty due to the transfer detector stability for each round of measurements. 48

57 u Stability IGA001 IGA002 IGA003 IGA004 IGA005 IGA006 IGA007 IGA008 IGA Wavelength Figure 5.1(a) Round 1 transfer detector stability. u Stability Wavelength IGA001 IGA002 IGA003 IGA004 IGA005 IGA006 IGA007 IGA008 IGA009 Figure 5.1(b) Round 2 transfer detector stability. 49

58 u Stability IGA001 IGA002 IGA003 IGA004 IGA005 IGA006 IGA007 IGA008 IGA Wavelength Figure 5.1(c) Round 3 transfer detector stability. u Stability Wavelength IGA001 IGA002 IGA003 IGA004 IGA005 IGA006 IGA007 IGA008 IGA009 IGA010 IGA011 IGA012 IGA013 IGA014 IGA015 Figure 5.1(d) Round 4 transfer detector stability. 50

59 5.4 Instability and anomalous behavior of transfer detectors he responsivities of most of the photodiodes remained stable within the measurement reproducibility over the course of the comparison. Fig. 5.2 (a, b, c, d) shows example results of the long-term stability of the transfer detectors. Note that the data shown in the figure were not corrected for temperature. Notable effects of temperature are clearly seen around 950 nm. he optical characteristics of two transfer detectors changed significantly enough prior to the Fourth Round of measurements to warrant their removal from the Comparison. NIS03 had stability problems, as shown in Fig. 5.3, and NIS06 had a hole in the responsivity, shown in Fig Neither photodiode participated in Rounds 3 and 4 of the Comparison. Laboratory results separated by vendor and plotted as a difference from NIS results are shown in Fig he variance in measurements of EGG photodiodes at 900 nm was significantly larger than the variance in measurements of the photodiodes purchased from other vendors at 900 nm and was significantly larger than the variance in measurements of any photodiodes at other wavelengths. Following distribution of Draft A (Jan. 2003) and discussions of the results with members of the Key Comparison Working Group, it was decided that measurements of EGG photodiodes at 900 nm would be removed from the Key Comparison data set. he difference in the measurement of the DC photodiode by Laboratory #2 (CSIR) at 950 nm was considered anomalous and this measurement was not included in the analysis. 51

60 Ge Power Devices InGaAs Photodiode Responsivity Stability 1.0 Difference, ² = (D1-ref)/ref Jan Jan-99 1-Feb Jun-99 6-Jan Jun Feb-01 Note: he reference value is the average responsivity from the first three measurements CCPR-NIR NIS 04 Ctrl Chrt 2-01 Wavelength Figure 5.2 (a). Stability of photodiodes over the course of the measurement comparison. Example results for Ge Power Devices photodiodes are shown. 1.0 Fermionics InGaAs Photodiode Responsivity Stability Difference, ² = (D1-ref)/ref Jan Jan-99 1-Feb Jun Jan Jun Feb-01 Diode removed from intercomparison after Round 3 due to this change in responsivity. Note: he reference value is the average responsivity from the first three measurements CCPR-NIR NIS 03 Ctrl Chrt 2-01 Wavelength Figure 5.2 (b). Stability of photodiodes over the course of the measurement comparison. Example results for Fermionics photodiodes are shown. 52

61 EG&G Optoelectronics InGaAs Photodiode Responsivity Stability 1.0 Difference, ² = (D1-ref)/ref Jan Jan-99 1-Feb Jun-99 7-Jan Jun Feb-01 Note: he reference value is the average responsivity from the first three measurements CCPR-NIR NIS 11 Ctrl Chrt 2-01 Wavelength Figure 5.2 (c). Stability of photodiodes over the course of the measurement comparison. Example results for EGG Optoelectronics photodiodes are shown. 1.0 elcom Devices InGaAs Photodiode Responsivity Stability Difference, ² = (D1-ref)/ref Jan Jan Jan Jun-99 7-Jan Jun Feb-01 Note: he reference value is the average responsivity from the first three measurements CCPR-NIR NIS 14 Ctrl Chrt 2-01 Wavelength Figure 5.2 (d). Stability of photodiodes over the course of the measurement comparison. Example results for elcom Devices photodiodes are shown. 53

62 1.0 Fermionics InGaAs Photodiode Responsivity Stability Difference, ² = (D1-ref)/ref Jan Jan-99 1-Feb Jun Jan Jun Feb-01 Diode removed from intercomparison after Round 3 due to this change in responsivity. Note: he reference value is the average responsivity from the first three measurements CCPR-NIR NIS 03 Ctrl Chrt 2-01 Wavelength Figure 5.3. Change in the responsivity of NIS03 after Round y [mm] x [mm] Figure 5.4. Spectral responsivity uniformity of NIS06 measured after Round 3. 54

63 Difference (Lab-NIS)/NIS Vendor: GPD Wavelength Lab01 Lab02 Lab03 Lab04 Lab05 Lab06 Lab07 Lab09 Lab09 Lab10 Lab11 Lab12 Lab13 Lab14 Lab15 Difference (Lab-NIS)/NIS Vendor: EGG Wavelength Lab01 Lab01 Lab02 Lab04 Lab05 Lab06 Lab07 Lab07 Lab10 Lab11 Lab12 Lab13 Lab14 Lab15 Vendor: DC Vendor: Fermionics Difference (Lab-NIS)/NIS Lab02 Lab03 Lab04 Lab06 Lab10 Lab11 Lab12 Lab13 Lab15 Difference (Lab-NIS)/NIS Lab03 Lab05 Lab09 Lab Wavelength Wavelength Figure 5.5. Laboratory results, plotted as difference from NIS, separated by vendor. 5.5 Uncertainties associated with Laboratory-NIS transfer he uncertainty in transfer measurements by the Pilot lab consisted of the following components Repeatability of NIS measurement he NIS measurement repeatability data presented in able 5.2 were taken from the repeated measurements of each transfer detector on different dates with re-mounting photodiodes, thus include the repeatability of wavelength scale and the alignment repeatability associated with spatial nonuniformity of the transfer detectors used in this comparison Long-term stability of the comparison scale he comparison scale was the NIS scale maintained at the time of this intercomparison. he scale was maintained by a group of working standard detectors (Larason 1998), which were not recalibrated during the comparison. able 5.3 shows the uncertainty contribution from the longterm stability (reproducibility) of the comparison scale estimated from the data in reference (Larason 2008). his longterm stability uncertainty includes longterm shifts in the NIS instrument characteristics as well as drifts of working standard detectors. he correlated components in the scale realization at NIS are cancelled out in the transfer measurements. 55

64 able 5.3. Uncertainty contribution u long s NIS from the longterm stability of the comparison scale. u long s NIS Wavelength Wavelength scale uncertainty he wavelength scale uncertainty of the NIS SCF, evaluated as the residuals after correction, is typically within 0.1 nm as shown in Fig. 5.6 (blue triangle marks with dashed line). Considering small long-term shifts between calibrations, 0.1 nm (rectangular distribution) is assumed for the entire 900 nm to 1600 nm region. he repeatability of the wavelength scale is much smaller as indicated by the standard deviation curve (orange square box marks). Errors in transfer occur if there are significant random variations or differences in relative spectral responsivity curve between the working standard detectors and photodides measured. he NIS working standard detectors are of the Ge Power Device type. he spectral responsivity curves of Ge Power Devices, Fermionics, and EG&G are similar, but there are significant variations in the curves around 900 and 950 nm. he variations of the sensitivity coefficients among these photodiodes used in the comparison were analyzed as one group. he spectral responsivity curves of elcom Device photodiodes were significantly different from the Ge Power Device, and thus this type of photodiodes was analyzed separately. Figs. 5.7 (a) and (b) show typical relative sensitivity coefficient at each wavelength. he variations and differences from NIS working standard detectors in relative sensivity coefficient among all photodiodes in the two groups were analyzed. able 5.4 shows the summary of results for the transfer uncertainty contributions, denoted u w,p, for the two groups of photodiode measured. Subscript p in u w,p indicates the group of photodiodes. 56

65 Figure 5.6. An example of the wavelength scale calibration results for the NIS SCF in the near IR region. Figure 5.7 (a) Relative sensitivity coefficient for one of NIS working standard detectors (GPD). 57

66 Figure 5.7 (b) Relative sensitivity coefficient for one of the elcom Devices photodiodes. able 5.4 ransfer uncertainty contribution u w,p with the wavelength scale uncertainty of NIS SCF. ransfer uncertainty contribution (%) for GPD, EG&G, Fermionics ransfer uncertainty contribution (%) for elcom associated Bandwidth effect he bandwidth of the monochromator also affects in transfer measurement if there are significant differences in relative spectral responsivity curve between the working standard detectors and photodides measured in the NIS SCF. Figure 5.8 (a) and (b) shows the bandpass errors for two different types of photodiode calculated as the measured value minus the corrected value (Ohno 2005). Even though such data are shown, bandpass error correction was not applied for the NIS measurements in this comparison, thus this factor is evaluated as an uncertainty component. he uncertainty contribution in transfer measurement was analyzed from the 58

67 differences in bandpass error between the NIS working standard detectors and various transfer photodiodes used in the comparison. he estimated errors were converted to standard uncertainty using a rectangular distribution. able 5.5 summarizes the results for the uncertainty contribution in the transfer measurement associated with the effect of bandpass, denoted u bp,p, for the two groups of photodiode. Figure 5.8 (a) Relative errors due to bandwidth (4 nm) of the NIS SCF for a typical GPD photodiode. Figure 5.8 (b) Relative errors due to bandwidth (4 nm) of the NIS SCF for one of DC photodiodes. 59

68 able 5.5 ransfer uncertainty contribution u bp,p with the effect of bandpass of NIS SCF. GPD, EG&G, Fermionics ransfer uncertainty contribution (%) DC ransfer uncertainty contribution (%) associated Beam size he difference in beam size used by the pilot lab and by the participants also affects the results of the comparison due to the spatial nonuniformity of photodiode responsivity. he spatial nonuniformity of all the transfer photodiode were measured at three wavelengths at least once at the beginning of the comparison (section 3.3). Based on these data, the differences in the results for each photodiode in different beam sizes were analyzed. In this calculation, the beam profile is assumed to be Gaussian, and the funtion is fit so that most of the beam power is contained within the circle of reported diamter and the center of the beam is placed at the center of the photodiode s sensitive area. he difference in measured responsivity was obtained as the weighted average of the responsivity uniformity data (normalized at the center) weighted by the Gaussian beam distribution. able 5.6 shows the results. Corrections were not made to the comparison results based on these data, thus these were considered as a part of transfer uncertainty. Based on these data and reported beam size used at each participant laboratory, the relative difference for each photodiode for each laboratory was estimated. For beam sizes other than 2 mm and 3 mm, a palabolic interpolation was used (with the errors at 1 mm beam size to be zero). he results were converted to standard uncertainties, called u bs,ij, and summarized in able 5.7. Since it would be too complicated to deal with this uncertainty at each wavelength for each photodiode, the average of 1250 nm and 1600 nm was used for all wavelengths for simplicity. he 900 nm data was not used because many of the data at this wavelength were not used (highlighted gray), and this wavelength tends to have large nonuniformity but data of neigiboring wavelengths may not necessarily be similar. An example of a problematic spatial nonuniformity of a photodiode is shown in Fig Fortunately this photodiode was sent only to one laboratory, who used 2 mm beam size. 60

69 able 5.6 Relative difference (%) in measured responsivity for different beam sizes compared to the 1 mm diameter beam used at NIS. 2 mm beam size 3 mm beam size Detector NIS # 900 nm 1250 nm 1600 nm 900 nm 1250 nm 1600 nm 01 Fermionics Fermionics Fermionics GPD GPD GPD EG&G EG&G EG&G EG&G elcom elcom elcom elcom GPD GPD GPD * 900 nm data of gray highlighted diodes were not used in the comparison analysis. Position [mm] Position [mm] Figure 5.9 Responsivity spatial nonuniformity of NIS#13 at 1600 nm. 61

70 able 5.7 ransfer uncertainty contribution u bs,ij (%) associated with the differences in beam size used by the participants and the spatial nonuniformity of transfer photodiodes. NMI BNM- INM CSIR CSIRO HU IFA KRISS NIM NMi- VSL NPL NRC OMH Beam size 3 mm 2 mm 1 x 3 mm 1 x 1.7 mm 3.2 mm 2 mm 3 mm 4 mm 3 mm 3 x 1.5 mm 2 mm Detector NIS # *1 900 nm 1250 nm 1600 nm u bs,ij 04 GPD EG&G EG&G GPD EG&G elcom GPD elcom Fermionics GPD EG&G elcom GPD Fermionics EG&G GPD EG&G elcom GPD EG&G EG&G GPD Fermionics GPD GPD EG&G elcom GPD elcom EG&G GPD EG&G elcom PB 18 GPD mm HM *1 11 EG&G Fermionics GPD nm HM *2 11 EG&G Fermionics GPD SMU 3 mm 02 Fermionics EG&G VNIIOFI 07 GPD x 3 14 elcom mm 09 EG&G *1 Average of 1250 nm and 1600 nm. * nm and lower, *3 higher than 1400 nm 62

71 5.5.6 Alignment of detector Due to the significant spatial nonuniformity of the photodiode responsivity, alignment offsets can cause significant variations in results. he repeatability of NIS measurements due to repeatability of alignment is included in the NIS repeatability data in able 5.2 because these repeated measurements at NIS included re-alignment of the photodiodes. he uncertainty contributions from alignment were evaluated from the spatial nonuniformity data of transfer photodiodes, which showed similar degree of uncertainty contributions from alignment accuracy and used only as verification Summary of transfer uncertainties associated with photodiode characteristics he transfer uncertainties due to the wavelength scale uncertainty, bandpass, and differences in beam size are dependent on the characteristics of each particular photodiode. he effect of beam size also depends on the instruments used by the participants. hese components are to be included in the uncertainty analysis for the laboratory NIS difference for each photodiode. he tables below summarizes these uncertainty contributions for each transfer photodioded measured by each laboratory. u c,ij is the combined standard uncertainties of these components. 63

72 able BNM-INM additional transfer uncertainty contributions. NIS04 (GPD) NIS08 (EG&G) NIS10 (EG&G) Wavelength u w,p u bp,p u bs,ij u c,ij u w,p u bp,p able CSIR additional transfer uncertainty contributions. NIS04 (GPD) NIS08 (EG&G) NIS15 (DC) Wavelength u w,p u bp,p u bs,ij u c,ij u w,p u bp,p able CSIRO additional transfer uncertainty contributions. NIS06 (GPD) NIS14 (DC) NIS03 (Fermionics) Wavelength u w,p u bp,p u bs,ij u c,ij u w,p u bp,p u bs,ij u bs,ij u bs,ij u c,ij u c,ij u c,ij u w,p u w,p u w,p u bp,p u bp,p u bp,p u bs,ij u bs,ij u bs,ij u c,ij u c,ij u c,ij 64

73 able HU additional transfer uncertainty contributions. NIS06 (GPD) NIS10 (EG&G) NIS15 (DC) Wavelength u w,p u bp,p u bs,ij u c,ij u w,p u bp,p able IFA additional transfer uncertainty contributions. NIS04 (GPD) NIS03 (Fermionics) NIS11 (EG&G) Wavelength u w,p u bp,p u bs,ij u c,ij u w,p u bp,p able KRISS additional transfer uncertainty contributions. NIS17 (GPD) NIS10 (EG&G) NIS13 (DC) Wavelength u w,p u bp,p u bs,ij u c,ij u w,p u bp,p u bs,ij u bs,ij u bs,ij u c,ij u c,ij u c,ij u w,p u w,p u w,p u bp,p u bp,p u bp,p u bs,ij u bs,ij u bs,ij u c,ij u c,ij u c,ij 65

74 able NIM additional transfer uncertainty contributions. NIS04 (GPD) NIS11 (EG&G) NIS09 (EG&G) Wavelength u w,p u bp,p u bs,ij u c,ij u w,p u bp,p able NMi-VSL additional transfer uncertainty contributions. NIS16 (GPD) NIS02 (Fermionics) NIS05 (GPD) Wavelength u w,p u bp,p u bs,ij u c,ij u w,p u bp,p able NPL additional transfer uncertainty contributions. NIS07 (GPD) NIS08 (EG&G) NIS14 (DC) Wavelength u w,p u bp,p u bs,ij u c,ij u w,p u bp,p u bs,ij u bs,ij u bs,ij u c,ij u c,ij u c,ij u w,p u w,p u w,p u bp,p u bp,p u bp,p u bs,ij u bs,ij u bs,ij u c,ij u c,ij u c,ij 66

75 able NRC additional transfer uncertainty contributions. NIS16 (GPD) NIS14 (DC) NIS08 (EG&G) Wavelength u w,p u bp,p u bs,ij u c,ij u w,p u bp,p able OMH additional transfer uncertainty contributions. NIS07 (GPD) NIS11 (EG&G) NIS15 (DC) Wavelength u w,p u bp,p u bs,ij u c,ij u w,p u bp,p able PB additional transfer uncertainty contributions. NIS18 (GPD) NIS11 (EG&G) NIS01 (Fermionics) Wavelength u w,p u bp,p u bs,ij u c,ij u w,p u bp,p u bs,ij u bs,ij u bs,ij u c,ij u c,ij u c,ij u w,p u w,p u w,p u bp,p u bp,p u bp,p u bs,ij u bs,ij u bs,ij u c,ij u c,ij u c,ij 67

76 able SMU additional transfer uncertainty contributions. NIS07 (GPD) NIS02 (Fermionics) NIS10 (EG&G) Wavelength u w,p u bp,p u bs,ij u c,ij u w,p u bp,p able VNIIOFI additional transfer uncertainty contributions. NIS07 (GPD) NIS14 (DC) NIS09 (EG&G) Wavelength u w,p u bp,p u bs,ij u c,ij u w,p u bp,p u bs,ij u bs,ij u c,ij u c,ij u w,p u w,p u bp,p u bp,p u bs,ij u bs,ij u c,ij u c,ij 68

77 6. Data Analysis he data analysis of this comparison basically follows the Guidelines for CCPR Comparison Report Preparation (CCPR 2006), while different intermediate steps (used in Draft A-1 and accepted by the participants) are used. In the analysis of the data set, we first calculate the mean relative difference, called, between each participant i and the Pilot Laboratory (NIS), weighted by the transfer uncertainty of the measurements of each photodiode. Weighted mean is used in this intermediate step so that results from detectors that experienced larger changes during comparison were less weighted. We then calculate the Key Comparison Reference Value (KCRV) of the relative difference, and calculate individual laboratories results, including results from the Pilot Laboratory, with respect to the KCRV. he KCRV is calculated as a weighted mean with cut-off following the CCPR Guidelines. 6.1 Analysis method he calculation steps are detailed below. he following notations are used. laboratory i (i =0 to 14, where i =0 is the pilot lab.) photodiode j (j=1 to 3 for each participant) repeat of measurement n (n=1 to 3) for each photodiode s ij,n : spectral responsivity measured by a participant s NIS ij,n (pre,post): spectral responsivity measured by the pilot lab (before and after transportation). 1. Calculate the relative difference ij for each laboratory i (=1 to 14) for each photodiode j compared to NIS measurement: ij s ij s NIS ij 1. (6.1) where s ij is the average of the three measurements of detector j at laboratory i : 3 s ij 1 s ij,n, (6.2) 3 n 1 and s NIS ij is the average of the measurements at NIS of detector j measured by laboratory i before (pre) and after (post) each round of the comparison: s NIS ij NIS s ij,n pre 1 3 NIS s ij,n post. (6.3) 3 3 n 1 n 1 2. Calculate the relative transfer uncertainty of ij, calculated as a combination of the repeatability of the laboratory s three measurements, the NIS repeatability, the detector stability measured by NIS, and additional transfer uncertainty components u c,ij : s ij u 2 2 NIS r s NIS u stability s ij u c,ij 2, (6.4) s ij is the standard deviation of the mean of the three reported measurements of u t2 ij u 2 r where u r detector j at laboratory i: u 2 r s ij 1 s k ij s ij 2 ; n = 3, (6.5) n 1 n n k 1 69

78 u r s NIS is the relative NIS measurement repeatability uncertainty given in able 5.2 and u stability (s NIS ij ) is calculated according to Eq. 5.1, and u c,ij are the additional transfer uncertainty component combined as listed in ables to For each laboratory i, calculate the weighted mean (call it i ) of its three ij (j=1 to 3) using the relative transfer uncertainty derived in Step 2: 3 i w ij ij (6.6) where w ij j 1 3 u t ( ij ) 2 j 1 u t ( ij ) 2 For NIS, Calculate the transfer uncertainty of i, call it u t i : u t i 3 j u t2 ij his transfer uncertainty includes the repeatability of the detectors in the participant s measurement as indicated by eq. (6.4). he propagated transfer uncertainty excluding this component, denoted u t,corr i, is given by 2 u t,corr 3 2 (6.7) (6.8) i w 2 ij u 2 r s NIS u 2 NIS stability s ij u c,ij. (6.9) j 1 Note that u c,ij is partially correlated among the three photodiodes measured by each participant, but it is treated as uncorrelated here since the degree of correlation is unknown and it should be taken as approximation. For NIS, 2 u t,corr 0 u 2 NIS r s NIS average u stability s ij 2 average u c,ij 2 3 (6.10) is used. Averages are for all diodes measured by all participants. 5. Apply a cut-off value for the reported uncertainties u i of each laboratory. u adjusted,i u i when u i u cut -off (6.11) u adjusted,i u cut -off when u i u cut -off where u cut-off is the cut-off value for the reported uncertainty (discussed in Section 7.2), and u i average u rel s ij,n (6.12) 6. Calculate the combined standard uncertainty u c i combined standard uncertainty u c, adj i (cut-off not applied) and the adjusted (cut-off applied) for each laboratory i as 70

79 2 u c2 i u i 2 u c, adj 2 u t,corr 2 i u adjusted,i 2 i u long 2 u t,corr (6.13) s NIS 2 i u long (6.14) s NIS u long s NIS is the uncertainty contribution from the longterm stability of the comparison scale as listed in able 5.3. Note that u t,corr i (eq. 6.9) rather than u t i (eq. 6.8) is used above because the reported uncertainty u i already includes the uncertainty contribution from the repeatability of measurements at participants, and this component should not be duplicated. 7. Calculate the KCRV, KCRV, as a weighted mean of i for all participating laboratories, using the adjusted uncertainties in eq. (6.14). 14 KCRV w i i (6.15) where w i 14 i 0 i 0 u 2 c, adj i u 2 c, adj i (6.16) 8. Calculate the propagated uncertainty of kcrv, call it u kcrv, for weighted mean with cut-off, u kcrv N u c 2 i u 4 i 0 c, adj i N u 2 c, adj i (6.17) i 0 9. he difference d i from KCRV for each participant i is given by d i i KCRV (6.18) and the uncertainty in the difference d i is given by ud i u u c2 2 i u 2 ( KCRV ) 2 c i u 2 c, adj i N j 0 u c, adj 2 j (6.19) ud i u c2 i u 2 ( KCRV ) for CSIR and KRISS. (6.20) Note that eq. (6.19) takes into account the partial correlation between i and KCRV. 10. he unilateral Degree of Equivalence (DoE) of laboratory i is given by D i d i (6.21) U i k ud i ; k=2 (coverage factor) (6.22) 11. he bilateral DoE between laboratory i and laboratory m is given by D i,m d i d m (6.23) U i,m k u c2 i 2 u c2 m ; (k =2) (6.24) 71

80 7. Results of the Comparison 7.1 Intermediate calculation results o determine the KCRV at each wavelength, the temperature-corrected, measured values from each laboratory are first analyzed with respect to the values of the pilot laboratory (NIS) using Eqs. (6.1) to (6.9). he results are summarized in ables 7.1 to he four columns for each photodiode sample show the results of calculatioon for ij by eq. (6.1), u r s ij by eq. (6.5), NIS NIS u stability s ij by eq. (5.1), and u t ij by eq. (6.4). u stability s ij is labeled as simply u stability and their values are different from those in ables to because the results in ables 7.1 to 7.14 are corrected for temperature variations. he last two columns of the table show the laboratory s mean difference i by eq. (6.6), the transfer uncertainty u t i i by eq. (6.8), and u t,corr calculated by eq. (6.9). 72

81 able 7.1. BNM-INM intermediate calculation results (values in %). NIS04 NIS08 i,1 u r s i,1 u stability u t i,1 i,2 u r s i,2 u stability u t i,2 NIS10 i,3 u r s i,3 u stability u t i,3 i u t i u t,corr i able 7.2. CSIR intermediate calculation results (values in %). NIS04 NIS08 i,1 u r s i,1 u stability u t i,1 i,2 u r s i,2 u stability u t i,2 NIS15 i,3 u r s i,3 u stability u t i,3 i u t i u t,corr i

82 able 7.3. CSIRO intermediate calculation results (values in %). NIS06 i,1 u r s i,1 u stability u t i,1 NIS14 i,2 u r s i,2 u stability u t i,2 NIS03 i,3 u r s i,3 u stability u t i,3 i u t i u t,corr i able 7.4. HU intermediate calculation results (values in %). NIS06 i,1 u r s i,1 u stability u t i,1 NIS10 i,2 u r s i,2 u stability u t i,2 NIS15 i,3 u r s i,3 u stability u t i,3 i u t i u t,corr i

83 able 7.5. IFA intermediate calculation results (values in %). NIS04 NIS03 i,1 u r s i,1 u stability u t i,1 i,2 u r s i,2 u stability u t i,2 NIS13 i,3 u r s i,3 u stability u t i,3 i u t i u t,corr i able 7.6. KRISS intermediate calculation results (values in %). NIS17 NIS10 i,1 u r s i,1 u stability u t i,1 i,2 u r s i,2 u stability u t i,2 NIS13 i,3 u r s i,3 u stability u t i,3 i u t i u t,corr i

84 able 7.7. NIM intermediate calculation results (values in %). NIS04 NIS11 i,1 u r s i,1 u stability u t i,1 i,2 u r s i,2 u stability u t i,2 NIS09 i,3 u r s i,3 u stability u t i,3 i u t i u t,corr i able 7.8. NMi-VSL intermediate calculation results (values in %). NIS16 i,1 u r s i,1 u stability u t i,1 NIS02 i,2 u r s i,2 u stability u t i,2 NIS05 i,3 u r s i,3 u stability u t i,3 i u t i u t,corr i

85 able 7.9. NPL intermediate calculation results (values in %). NIS07 NIS08 i,1 u r s i,1 u stability u t i,1 i,2 u r s i,2 u stability u t i,2 NIS14 i,3 u r s i,3 u stability u t i,3 i u t i u t,corr i able NRC intermediate calculation results (values in %). NIS16 NIS14 i,1 u r s i,1 u stability u t i,1 i,2 u r s i,2 u stability u t i,2 NIS08 i,3 u r s i,3 u stability u t i,3 i u t i u t,corr i

86 able OMH intermediate calculation results (values in %). NIS07 NIS11 i,1 u r s i,1 u stability u t i,1 i,2 u r s i,2 u stability u t i,2 NIS15 i,3 u r s i,3 u stability u t i,3 i u t i u t,corr i able PB intermediate calculation results (values in %). NIS18 NIS11 i,1 u r s i,1 u stability u t i,1 i,2 u r s i,2 u stability u t i,2 NIS01 i,3 u r s i,3 u stability u t i,3 i u t i u t,corr i

87 able SMU intermediate calculation results (values in %). NIS07 NIS02 i,1 u r s i,1 u stability u t i,1 i,2 u r s i,2 u stability u t i,2 NIS10 i,3 u r s i,3 u stability u t i,3 i u t i u t,nis i able VNIIOFI intermediate calculation results (values in %). NIS07 NIS14 i,1 u r s i,1 u stability u t i,1 i,2 u r s i,2 u stability u t i,2 NIS09 i,3 u r s i,3 u stability u t i,3 i u t i u t,corr i

88 7.2 Determination of the Key Comparison Reference Values he KCRV is derived at each wavelength from the results summarized in Section 7.1 according to the procedures described in Section 6 (Eqs to 6.24). It is based on the weighted mean, with individual weights limited by the adoption of a cut-off value for the reported uncertainties of each laboratory. he following measurements were excluded from the calculation of the KCRV: 1. Measurements by CSIR because they were incomplete and did not conform to the protocol. 2. Measurements by KRISS because KRISS was unable to provide details on their uncertainty budget, as requested of all laboratories following the Working Group discussion of Draft A results in June, he combined uncertainty u c i (Eq. 6.13) for each laboratory with no cutoff value applied are listed in able he corresponding weights are listed in able 7.16 and shown in Fig Note that with no cutoff value applied, the KCRV would be dominated by one laboratory, NRC, with weights ranging from 0.16 to 0.48, depending on the wavelength. In Draft A-1 (Jan. 2003), the uncertainty cutoff value of 0.25 % was selected for all wavelengths. his limited the weight assigned to each laboratory to be less than 0.15, also for all wavelengths. In this final report, following the recommendation of the Working Group that resulted from their meeting in June 2004, the cut-off value u cut-off is determined to be the average uncertainty of all uncertainties less than the median uncertainty: u cut -off Average u i for u i Median u i. (7.1) he cut-off uncertainties are given in able he combined uncertainties applying the cut-off, u c, adj i (Eq. 6.14), are listed in able 7.18 and the resultant weights used in the calculation of the KCRVs (Eq. 6.16) in able 7.19 and Fig Note that in this case no laboratory has a weight larger than 0.17, and four laboratories have almost equal weight at each wavelength. Using the laboratory weights in able 7.19, the weighed mean difference KCRV of the laboratory differences i from the pilot is calculated according to Eq he uncertainty of KCRV, u KCRV, is calculated using Eq KCRV is the KCRV, and the difference i KCRV (eq. 6.18) is the relative laboratory difference from the KCRV. 7.3 Birge Ratio he Birge ratio is the ratio of the external consistency to the internal consistency, with the internal and external consistencies defined as: N 2 u i i 1 2 uint 1 (7.2) 2 u ext N i 1 N 1 x i x w 2 u 2 N x i 1 u 2 x i 1 i, (7.3) where x i is measurement i and x w is the weighted mean of the measurement [aylor 1969]. he Birge ratio R B is defined as: 80

89 R B u ext u int. (7.4) he consistency check fails if the Birge ratio is significantly greater than one. he Birge ratios with and without cut-off are given in able he data of CSIR and KRISS are excluded in the calculation as they are excluded from the calculation of KCRV. he Birge ratio is found to be significangly greater than 1 (1.4 or larger) for all wavelengths, which indicates that the uncertainties reported by participants, on the average, were underestimated. 7.4 Summarized laboratory results with respect to the KCRV able 7.21 summarizes KCRV and its uncertainty u KCRV for each wavelength. Each laboratory s relative percent difference d i i KCRV (eq. 6.18) is listed in able his corresponds to the unilateral Degree of Equivalence (DoE) D i of each laboratory at each wavelength. he uncertainties ud i in the relative difference d i, calculated by eqs and 6.20, are given in able Multiplying the values by 2 in able 7.23 gives U D i, the expanded uncertainty (k=2) in the unilateral DoE, which are listed in able he results are shown for each wavelength in Fig. 7.3 (a) (o), plotting each laboratory s relative difference d i. Error bars corresponds to the values of U D i, thus the expanded uncertainties (k=2) of the relative difference d i given by eq. (6.22), which includes the laboratory s reported uncertainty, transfer uncertainty in the comparison, and the uncertainty of KCRV. For reference, able 7.25 lists the laboratories reported uncertainties (average of the measurements reported in Section 4). In Fig. 7.4 (a) (o), the results are summarized for each laboratory; laboratories are given in able In these figures, the expanded uncertainty (k=2) in the KCRV is plotted as well. he tables of the unilateral and bilateral DoE of all combinations of laboratories are presented in Appendix A. 81

90 able Combined uncertainty u c i for each laboratory (no cut-off applied). BNM CSIR CSIRO HU IFA KRISS NIM NIS NMi-VSL NPL NRC OMH PB SMU VNIIOFI able Corresponding weights w i used in the calculation of the KCRV if no cut-off value is applied. BNM CSIR CSIRO HU IFA KRISS NIM NIS NMi-VSL NPL NRC OMH PB SMU VNIIOFI

91 able u cut-off at each wavelength. Median u rel u cut-off able Combined uncertainty u c, adj i of each lab with cut-off applied. BNM CSIR CSIRO HU IFA KRISS NIM NIS NMi-VSL NPL NRC OMH PB SMU VNIIOFI

92 able Corresponding weights w i used in the calculation of the KCRV, with cut-off applied. BNM CSIR CSIRO HU IFA KRISS NIM NIS NMi-VSL NPL NRC OMH PB SMU VNIIOFI

93 Figure 7.1. Weighting factors with no cut-off value applied to the data. Figure 7.2. Weighting factors with cut-off value applied to the data. 85

94 able Birge ratios calculated with and without cutoff (CSIR and KRISS excluded). Wavelength Birge ratio without cut-off Birge ratio with cut-off able he KCRV KCRV and its uncertainty u KCRV Wavelength KCRV u KCRV and U KCRV with k=2. U KCRV

95 able Laboratory differences d i from the KCRV, or unilateral Degree of Equivalence D i. BNM CSIR CSIRO HU IFA KRISS NIM NIS NMi-VSL NPL NRC OMH PB SMU VNIIOFI able Standard uncertainties ud i in laboratory difference d i. BNM CSIR CSIRO HU IFA KRISS NIM NIS NMi-VSL NPL NRC OMH PB SMU VNIIOFI

96 able Expanded uncertainties U d i in laboratory difference, or U in DoE, with coverage factor k=2. BNM CSIR CSIRO HU IFA KRISS NIM NIS NMi-VSL NPL NRC OMH PB SMU VNIIOFI able Laboratories reported uncertainties u i. BNM CSIR CSIRO HU IFA KRISS NIM NIS NMi-VSL NPL NRC OMH PB SMU VNIIOFI D i 88

97 able Laboratory name and associated figure label. Laboratory Label BNM-INM 1 CSIR 2 CSIRO 3 HU 4 IFA 5 KRISS 6 NIM 7 NIS 8 NMi-VSL 9 NPL 10 NRC 11 OMH 12 PB 13 SMU 14 VNIIOFI 15 89

98 Difference (Lab i - KCRV) (a) 900 nm Laboratory i Figure 7.3(a). Laboratory differences from the KCRV at 900 nm. Error bars correspond to uncertainties listed in able Difference (Lab i - KCRV) (b) 950 nm Laboratory i Figure 7.3(b). Laboratory differences from the KCRV at 950 nm. Error bars correspond to uncertainties listed in able

99 Difference (Lab i - KCRV) (c) 1000 nm Laboratory i Figure 7.3(c). Laboratory differences from the KCRV at 1000 nm. Error bars correspond to uncertainties listed in able Difference (Lab i - KCRV) (d) 1050 nm Laboratory i Figure 7.3(d). Laboratory differences from the KCRV at 1050 nm. Error bars correspond to uncertainties listed in able

100 Difference (Lab i - KCRV) (e) 1100 nm Laboratory i Figure 7.3(e). Laboratory differences from the KCRV at 1100 nm. Error bars correspond to uncertainties listed in able Difference (Lab i - KCRV) (f) 1150 nm Laboratory i Figure 7.3(f). Laboratory differences from the KCRV at 1150 nm. Error bars correspond to uncertainties listed in able

101 Difference (Lab i - KCRV) (g) 1200 nm Laboratory i Figure 7.3(g). Laboratory differences from the KCRV at 1200 nm. Error bars correspond to uncertainties listed in able Difference (Lab i - KCRV) (h) 1250 nm Laboratory i Figure 7.3(h). Laboratory differences from the KCRV at 1250 nm. Error bars correspond to uncertainties listed in able

102 Difference (Lab i - KCRV) (I) 1300 nm Laboratory i Figure 7.3(i). Laboratory differences from the KCRV at 1300 nm. Error bars correspond to uncertainties listed in able Difference (Lab i - KCRV) (j) 1350 nm Laboratory i Figure 7.3(j). Laboratory differences from the KCRV at 1350 nm. Error bars correspond to uncertainties listed in able

103 Difference (Lab i - KCRV) (k) 1400 nm Laboratory i Figure 7.3(k). Laboratory differences from the KCRV at 1400 nm. Error bars correspond to uncertainties listed in able Difference (Lab i - KCRV) (l) 1450 nm Laboratory i Figure 7.3(l). Laboratory differences from the KCRV at 1450 nm. Error bars correspond to uncertainties listed in able

104 Difference (Lab i - KCRV) (m) 1500 nm Laboratory i Figure 7.3(m). Laboratory differences from the KCRV at 1500 nm. Error bars correspond to uncertainties listed in able Difference (Lab i - KCRV) (n) 1550 nm Laboratory i Figure 7.3(n). Laboratory differences from the KCRV at 1550 nm. Error bars correspond to uncertainties listed in able

105 Difference (Lab i - KCRV) (o) 1600 nm Laboratory i Figure 7.3(o). Laboratory differences from the KCRV at 1600 nm. Error bars correspond to uncertainties listed in able

106 Difference from KCRV BNM-INM Wavelength Figure 7.4(a). BNM-INM differences from the KCRV. Error bars correspond to uncertainties listed in able 7.24; KCRV error bars (k=2) from able CSIR Difference from KCRV Wavelength Figure 7.4(b). CSIR differences from the KCRV. Error bars correspond to uncertainties listed in able 7.24; KCRV error bars (k=2) from able

107 2.0 CSIRO Difference from KCRV Wavelength Figure 7.4(c). CSIRO differences from the KCRV. Error bars correspond to uncertainties listed in able 7.24; KCRV error bars (k=2) from able Difference from KCRV HU Wavelength Figure 7.4(d). HU differences from the KCRV. Error bars correspond to uncertainties listed in able 7.24; KCRV error bars (k=2) from able

108 Difference from KCRV IFA Wavelength Figure 7.4(e). IFA differences from the KCRV. Error bars correspond to uncertainties from able 7.24; KCRV error bars (k=2) from able KRISS Difference from KCRV Wavelength Figure 7.4(f). KRISS differences from the KCRV. Error bars correspond to uncertainties from able 7.24; KCRV error bars (k=2) from able

109 NIM Difference from KCRV Wavelength Figure 7.4(g). NIM differences from the KCRV. Error bars correspond to uncertainties from able 7.24; KCRV error bars (k=2) from able Difference from KCRV NIS Wavelength Figure 7.4(h). NIS differences from the KCRV. Error bars correspond to uncertainties listed in able 7.24; KCRV error bars (k=2) from able

110 Difference from KCRV Nmi-VSL Wavelength Figure 7.4(i). NMi-VSL differences from the KCRV. Error bars correspond to uncertainties listed in able 7.24; KCRV error bars (k=2) from able NPL Difference from KCRV Wavelength Figure 7.4(j). NPL differences from the KCRV. Error bars correspond to uncertainties listed in able 7.24; KCRV error bars (k=2) from able

111 2.0 NRC Difference from KCRV Wavelength Figure 7.2(k). NRC differences from the KCRV. Error bars correspond to uncertainties listed in able 7.24; KCRV error bars (k=2) from able OMH Difference from KCRV Wavelength Figure 7.4(l). OMH differences from the KCRV. Error bars correspond to uncertainties listed in able 7.24; KCRV error bars (k=2) from able

112 2.0 PB Difference from KCRV Wavelength Figure 7.4(m). PB differences from the KCRV. Error bars correspond to uncertainties listed in able 7.24; KCRV error bars (k=2) from able SMU Difference from KCRV Wavelength Figure 7.4(n). SMU differences from the KCRV. Error bars correspond to uncertainties listed in able 7.24; KCRV error bars (k=2) from able

113 VNIIOFI Difference from KCRV Wavelength Figure 7.4(o). VNIIOFI differences from the KCRV. Error bars correspond to uncertainties listed in able 7.24; KCRV error bars (k=2) from able

114 8. Discussion of results o summarize the results, Laboratory differences d i from the KCRV listed in able 7.22 of all participants are presented in Figs. 8.1 and 8.2, and the laboratories reported uncertainties u i listed in able 7.25 of all participants are presented in Fig Form Fig. 8.1, except for two or three laboratories, the results of most of the laboratories agreed within ±1 % difference in the entire spectral range, which is considered an expected level of agreement for the near IR range. It should be noted, however, that the uncertainties reported by the participants were generally underestimated as indicated by the Berge ratio (able 7.20), and also as observed in the plots of some of the participants where the uncertainty bars (k=2) are not overlapping with that of KCRV (k=2) in a large portion of the spectral range. It is also observed from Fig. 8.2 that many curves are above the 0 % line, and much fewer curves are below 0 %, which indicates that the KCRV seems to be deviated slightly to the lower direction. his may be caused by one or two laboratories that deviated significantly from the KCRV in lower direction and reported relatively low uncertainties. No results were removed as outliers in this comparison. It would be useful to implement a procedure to exclude outliers in future comparisons. It is also observed from Fig. 8.2 that several laboratories agree within ~0.3 % in most of the wavelength range, which is encouraging. he changes of detector responsivity before and after transportation in this comparison were generally much less than the reported uncertainties of the laboratories (except NRC whose uncertainties were very low), and it can be said that transfer standards served generally well in this comparison. he use of the detectors from four different manufactures was also useful as one type of photodiode showed instability at 900 nm and excluded for that wavelength, but two other photodioes of other types served for those participants for that wavelength. As presented in sections 5.3 and 5.5.5, the responsivity spatial uniformity of some of the InGaAs transfer detectors, especially at 900 nm and 1600 nm, were significantly large. Due to this, the differences in beam size caused considerable measurement variations in some cases as reported in ables 5.6 and 5.7. he 900 nm data of EG&G photodiodes were removed from the analysis due to very large inconsistencies in results (Fig. 5.5), which seems to be caused by other instability factors. he final results (Fig. 7.4 (a) to 7.4 (o)) also showed some anomalous feature at 900 nm for several participants. In the future comparisons of this type, the transfer photodiodes may be selected for better spatial uniformity, and/or the wavelength point of 900 nm (overlapping with K2.b) might be dropped. he wavelength points of 900 nm, 950 nm, and 1000 nm were overlapped with CCPR K2.b, and the consistency of results between K2.a and K2.b were checked. Measurements for K2.b were performed in 2000, about the same time as K2.a. Figures 8.4 (a), (b), and (c) show relative differences from KCRV of the laboratories that participated in both K2.a and K2.b. From these figures, no correlations between the two comparisons were found. 106

115 Figure 8.1. Laboratory differences d i from the KCRV listed in able Figure 8.2. Magnification (-1.0 % to 1.0 %) of the Laboratory differences d i from the KCRV listed in able

116 Figure 8.3. Laboratories reported uncertainties u i listed in able Figure 8.4 (a). Comparison of the results between CCPR K2.a and CCPR K2.b (single photodiodes) at 900 nm. 108