Real-World Performance of Temperature Measurements at Automated Weather Stations How well do we do it?

Transcription

1 CIMO TECO 2016 Australian Bureau of Meteorology Real-World Performance of Temperature Measurements at Automated Weather Stations How well do we do it? Dr Jane Warne and Ian Dollary

2 Hello from the top of the World.

3 Tyranny of Distance! Peru or 2 Frances Libya or 5 Germanys Between Argentina & Algeria or 5 Spains Egypt or 1.6 Ukraines Turkey or 2.6 Italys 1 UK Ireland or 1.7 Switzerlands

4 REAL-world temperature measurement.

5 Growth of the Automated Network Number of Automated Weather Stations

6 Synoptic Stations (Total 864, 750 AWS)

7 Problem how do we maintain a network? Very large country 7,741,000km 2 Very few stations i.e. Australia 1 every 10,000km 2 Switzerland 1 every 26km 2 A long distance between stations, over rough and inhospitable roads Swap in and out would not work

8 Inspection and Maintenance Visits We ended up with. One inspection visit every 12months One maintenance visit every 12 months One emergency visit every 12months This meant most stations should get sited twice per year but would at least get one visit The result is a data set of over 30,000 intercomparisons of travelling standards with AWS in the field, per parameter.

9 Temperature Inspection Results Before After < 1.5%, >U C Sensor Reference in degree C < 5.6%, >U C

10 Length of Time in the Field 2 Sensor - Refernce in degree C % > 0.5C Days in the Field

11 Days between Inspection Visit 2.0 Change in (Sensor-Ref) between visits in degrees C Days between Inspection Visits

12 So what do we know so far? There is no drift in the temperature network over the 17 years the AWS network has operated That the sensors show no significant signs of aging and that if they survive installation they will just keep going until they fail. That the time interval between inspections has little impact So we can consider changing network operations and reducing the frequency of inspections from six month/annual to 3yrs without significant impact.

13 So what's the uncertainty? Uncertainty Budget from Lab Stage Comment 95% Expanded Uncertainty Covera ge Factor Unit Primary to Working Working to Dostmann Field by Working Dostmann in field process Verification in field by Dostmann Operational Uncertainty of Laboratory Working Reference (ITS90 Standards) Calibration of Dostmann Inspection IPRT by Working References Calibration of Field IPRT by Working References Operation Uncertainty of Inspection of IPRT by Working References Field Inspection of Field IPRT by Dostmann Inspection Reference mk C C C C

14 Does the inspection data agree? Uncertainty Budget from Inspections Model fit to distributions of corrections for RTDs (Bin 0.1, Centered 0.05) Frequency Normal Model m=0.0179c std=0.1932c Fit = Normal Model* m=0.0621c std=0.1011c Fit = Sensor Model m=0.0620c std=0.0773c Process Model m=0.0620c std=0.1588c Fit = 3.60 Model U C Sensor C Dostman and Process Lab U C Sensor C Dostman and Process Frequency Normal Model Sensor Model Process Model Combined Actual Model Actual Combined Normal Model* 2000 Actual Model* Correction (C)

15 What is driving the variability? Lots of things can be driving the variability. Instrument types Maintenance practices Skill of the staff Variation in practices across the country Can we identify from this same data some of the causes and possible ways of reducing it?

16 Sensor Type and Design Sensor - Reference in degrees C Before After PRT 1 TC PRT 2 PRT 3 PRT Ukn

17 Type of Network Average Sensor Reference ( C) Staff Auto Climate Aviation Marine Ag Res

18 State to State Variability Difference Sensor Reference ( C) NSW NT QLD SA TAS VIC WA

19 Skills and Training Temperature Probe Dry Bulb U95% Confidence Boundary for Inpector/Tech Competance U95% Confidence Interval for demonstrating statistically significant biases in Inspector Difference between Sensor and Reference ( C) I 001 I 011 I 021 I 031 I 041 I 051 I 061 I 071 I 081 I 091 I 101 I 111 I 121 I 131 I 141 I 151 I 161 I 171 I 181 I 191 I 201 I 211 I 221 Inspector/Technician Number

20 Experience in the Field 0.6 Inspection Data > 1 yr Inspection Data Average Probe Temp Inspection Data > +/ 0.1 C Significant Bias U Difference Sensor Reference ( C) Days of experience as Inspector or Technician 0.8

21 Drivers 1. We need to better control our manufacturers/suppliers of probes 2. Some networks perform better than others. e.g. 3. Staff station have both a lower variability and bias, probably due to better site control. 4. Reference networks are not all linked into the same traceability chain leading to higher variability and bias a lesson for third party networks 5. New staff need more hands on training 6. Not all states appear to be getting the same out comes, this will need further investigation

22 Control Charts Sensor-Ref (degree C) After After (mean) (mean) UCL UCL U2 U2 U1 U1 X U1 -U2 LCL After After X (mean) UCL UCL U2 U1 U2 U1 XU1 U1 X -U2 -U2 U1 LCL -U2 T5 - Linear LCL (u) LCL T8 T5 - Run - Linear Zone (u) C T1 - Extremes (u) T2-2 out of 3 in Zone A + T5 T3 - Linear 4 out of (u) 5 in Zone B + T7 T5 T4 - Linear Runs Avoid >8 Zone (u) (u) C T8 T7 T5 - T5 Linear Avoid Run - Linear Zone (u) (u) C T8 T7 - T7 Avoid Run - Avoid in Zone Zone C C T8 - T8 Run - Run in Zone in Zone C C

23 Control Charts Can be used as a predictor of future failure! Because they monitor the process not the specification they trigger at a lower threshold. They also help identify waste where you are over controlling the process

24 Conclusions Performance management is about understanding what you are achieving and why, then using that information to do a better job Analysis of basic data like these inspection data can yield enormous useful information for an organisation if they take the time to do so. The problem is we so often don't. On the up side we can confidently claim an uncertainty on our data of 0.3C on each and every observation We have a number of means to improve the variability of the data at little or no cost We have a number of means of reducing the cost of operating the program without damage to the network, like reducing the frequency of visits We have the information to argue the case for why particular aspects of the network need to change or stay the same.

25 Sea-Level Measurements What really matters? Lachlan Nicholls, Jane Warne and James Allen Location Dominant Thermal Profile Raw Airgap (Aq-Vg) noise (std, mm) Thermal Grad corrected (Aq-Vg) noise (std, mm) Darwin, Aus Quadratic Broome, Aus Quadratic Hillarys, Aus Quadratic Thevenard, Aus Quadratic Portland, Aus Linear Burnie, Aus Quadratic Spring Bay, Aus Quadratic Port Kembla, Aus Quadratic Rosslyn Bay, Aus Quadratic Cape Ferguson, Aus Linear Lautoka, Fiji Linear Suva, Fiji Linear Majuro, Marshall Is Linear

26 Desktop analysis of commercially available "All in One" and "Compact" weather stations - How do they measure up? Dr Jane Warne Vaisala WXT530 Vaisala WXT520 WeatherHawk 922 Signature For each sensor a specific set of information was sought this Range included WeatherHawk 620 the maximum and minimum measurement range Rainwise Prolog Rainwise MkIII Rainwise Hazmat Magellan Vehicle Magellan Fixed Base Lufft WS601 Lufft WS600 Lufft WS503 Gill GMX700 Environdata Wx Master 3000 Davis 6250 Davis 6163 Davis 6162 Davis 6152 Climatronics AIO resolution time constant drift per year temperature sensitivity coefficient (for humidity sensors etc) uncertainty over a specified operational range Meisei Poteka Compact Vaisala AWS310 MEA Premium HoBo U30 NRC Campbell Scientific Met200 Campbell Scientific GRWS100 Specification Dimensions of the unit Performance Traceability Drift Modularity Manufacture Quality Testable by RIC Passive (2) / Active (0) All in One (0) / Compact (2) Sample Size Screen Volume Heat Source Height Siting

27 So What is Australia's Climate?