EAS 535 Laboratory Exercise Weather Station Setup and Verification Lab Objectives: In this lab exercise, you are going to examine and describe the error characteristics of several instruments, all purportedly measuring the same thing over a one-week period. In the first week of the lab, you will set up or take down several weather stations to measure environmental parameters on the roof of the CIVL civil engineering building. In the second week of the lab, you will analyze the data that was collected, which should illustrate several of the kinds of issues that arise in many measurement situations. The MAWS Vaisala weather station is a high performance instrument manufactured by the company that provides weather monitoring equipment to many of the operational weather services in the world. The Vaisala WXT510 instruments are next generation operational quality equipment, approaching the MAWS in quality. PART I: Instrument Setup and Recording Materials required: MAWS weather station, WXT510 weather stations and accessory hardware As you install the weather stations, use your lab notebook to record procedures, sketches, data, and important notes. Install the MAWS weather station on the roof of CIVL building, following the instructions in the instrument operations manuals and the shortcut step-by-step directions provided on the class web site. Install the WXT510 weather sensors on the railing near the MAWS. Connect the PC to run the assigned weather station and familiarize yourself with the software. Setup a database in the assigned location on the disk. For measurement verification, configure the stations to run with 1 minute sampling. Compare the observations among all sensors to verify proper operation before leaving the site (see below). Before you leave the equipment, change the configuration to run with 10 minute sampling for one week. Measurement Verification Take this time now to fill out your lab notebook to have all the descriptive elements for the experiment. As a reminder, make sure you have noted the following: 1. Date and times of experiments and measurements, and names of persons making measurements. 2. Description and sketches of setup, environmental conditions 3. Unambiguous identification of sensors and equipment 4. Readability of tables of measurements including units and error estimates 1
5. Notes on significant problems 6. As always remember the notebook should be bound no loose pages and neat. Note the information recorded for all of the weather stations that were set up. The sketch should include a map that shows the approximate distance between sensors and other features of the environment, as well as a sketch of the instrument configuration. Note the names here of the person in charge of each station, it is important in case you need to ask someone for further information at a later date. Verify that the observations are reasonable. Make a table and note the values of the parameters measured by all weather stations, along with error bars. Comment on their reasonableness. Verify that the observations agree with other sensors present to within the specified errors. Note any problems in your notebook and fix them or see the TA for help. Download some observations from the sensors into the software database and verify that the software is functioning correctly, and the downloading procedure is clear. Before you leave the equipment, verify that your setup will have sufficient power and datalogging capability to record for the required amount of time. Note what type of power supplies are being used for each weather station, the voltage if it is running on batteries, and give an estimate of when the power supply may need to be replaced if running on batteries. Note the data storage capacity (from the manual or previous experiments), the current sampling rate, and an estimate of how much data will need to be stored over a week. For more information on operation, datalogging, and downloading data from all the equipment, see http://web.ics.purdue.edu/~jhaase/eas535/teaching/labs Your notebook will be checked next lab period, so be sure it is complete. 2
Background: Weather Station Instrument Performance Characterization and Calibration PART II: Data analysis We re going to assume that the higher quality more sensitive weather station is measuring the true value of the temperature at each point in time and use it as a reference instrument. Note that in reality one almost never has the truth, so in practice it is usually difficult to unambiguously decide which of a group of measurements is more reliable. Nevertheless, by understanding what certain kinds of errors look like, it is possible to make a reasonable guess. After investigating the nature of the instrumental errors, we will find a calibration relation that allows us to correct the less accurate instruments to the more accurate MAWS instrument and then comment on our confidence in that calibration relation. The error for a given instrument can be estimated by comparing it to a reference: Error 1 =T 1 -T reference Common measures of instrument performance include the following: 1) the MEAN (or average) error 2) the ROOT-MEAN-SQUARED error 3) the standard deviation of the error The first of these, which is a measure of BIAS, can be obtained by taking the average of all of the elements in a given error column. Use the AVERAGE function in EXCEL to calculate the mean bias whenever it is required. You can use the instrument specifications given in the instrument manuals, and also your intuition on the use of such data in forecasting, to evaluate if these errors are significant. The second of the above scalar measures of error, the RMS error is computed as the SQUARE-ROOT of the AVERAGE of the SQUARED ERROR. Mathematically, this is ( ) 2 Σ ERROR N The third measure is the STANDARD DEVIATION, defined as follows: 1/2 (ERROR MeanError) 2 N 1 1/2 3
Keep in mind that the RMS error represents the typical size of errors in the measurement; the smaller the RMS error is, the better the accuracy and precision of the instrument. Excel data analysis Download the datafiles from each of the weather stations and put them together in an excel spreadsheet. 1. Open the data file in EXCEL. The first set of columns contains the values from the MAWS weather station: date, time, temperature, relative humidity, pressure, wind, solar radiation, rainfall. The next set of columns contains values from the WXT510s. The readings have been taken at 10 minute intervals 2. Characterize the Temperature differences among the instruments. You will be responsible for characterizing the errors for two WXT510 stations, compared to the MAWS. One will be the WXT510 that was assigned to you, plus one other of your choice. 3. a. Plot temperature as a function of time for all three instruments on the same graph. Select the time column and the three temperature columns, then use the EXCEL graphics wizard (for example) and use the plot type scatter. How do these measurements compare? b. In a new column, calculate MAWS Temp - WXT-N Temp (where N is your sensor ID) and plot it as a function of time. Describe the time dependent nature of the differences, if you detect any, and calculate their standard deviation, RMS error, and mean. c. In a new column, calculate MAWS Temp WXT-M Temp (where M is the other sensor ID that you are comparing), and plot it on the same plot as a function of time. Describe the time dependent nature of the differences, if you detect any, and calculate their standard deviation, RMS error, and mean. d. Plot WXT510 temperature (X-axis) versus MAWS temperature (Y-axis). Calculate the correlation coefficient by having EXCEL fit a straight line through the points, and output the equation for the trendline on the plot. Does the line go through zero? Calculate the correlation coefficient with the constraint that the line goes through zero. Is the correlation coefficient the same? If the correlation coefficient goes through zero, then there is no bias. If a given sensor is well calibrated and precise, then the cloud of points should fall very close to the 1:1 line. Larger errors will be indicated by points falling further away from that line. e. Do the same for the other WXT510, it can go on the same plot as d. The equations found from the linear regression in d. and e. are the calibration curves that you could use, if necessary, in later labs to correct your data. 4
f. Plot the differences as a function of Temperature. What can you say about the relative error (dt/t)? g. Plot the solar radiation as a function of time from the MAWS. Does this give you any additional information for interpreting your time dependent or relative errors? 4. Characterize the humidity differences among the instruments. Use the same stations as you used in part 2. a. Plot Relative humidity as a function of time for all three instruments on the same graph. How do these measurements compare? b. In a new column, calculate MAWS RH WXT510-N RH, and plot it as a function of time. Describe the time dependent nature of the differences, if you detect any, and calculate their standard deviation, RMS error, and mean. c. In a new column, calculate MAWS RH WXT510-M RH, and plot it as a function of time on the same graph as b. Describe the time dependent nature of the differences, if you detect any, and calculate their standard deviation, RMS error, and mean. d. Plot WXT510 relative humidity (X-axis) versus MAWS relative humidity (Yaxis). Calculate the correlation coefficient by having EXCEL fit a straight line through the points, and output the equation for the trendline on the plot. Does the line go through zero? Calculate the correlation coefficient with the constraint that the line goes through zero. Is the correlation coefficient the same? If the correlation coefficient without the constraint goes very close to zero, then there is no bias. e. Do the same for the other WXT510 relative humidity and put it on the same plot as d. f. Plot the MAWS- WXT510-N and MAWS- WXT510-M differences as a function of Relative humidity? What can you say about the relative error (drh/rh)? g. Can you interpret the relative humidity errors in terms of the temperature or solar radiation? 5. Characterize the pressure differences among the instruments. Use the same stations that you used in section 2. a. Plot pressure as a function of time for all three instruments on the same graph. How do these measurements compare? b. In a new column, calculate MAWS P WXT510-N P, and plot it as a function of time. Describe the time dependent nature of the differences, if you detect any, and calculate their standard deviation, RMS error, and mean. c. In a new column, calculate MAWS P WXT510-M P, and plot it as a function of time with the plot from b. Describe the time dependent nature of the differences, if you detect any, and calculate their standard deviation, RMS error, and mean. 6. Calibration Equations for Temperature 5
a. The calibration equations to correct the two temperatures (ie WXT510-M and N) to give readings consistent with the MAWS were calculated in 2d. and 3d. above. T MAWS = a T TM 02 + b b. Using these values of a and b, create a new column which is calibrated WXT510-M. c. Calculate the expected error of these calibrated measurements by creating a new column which is delta T = T maws T TM02calibrated and then calculating the RMS of the delta T. d. Using these values of a and b, create a new column which is calibrated WXT510-N. e. Calculate the expected error of these calibrated measurements by creating a new column which is delta T = T maws T TM03calibrated and then calculating the RMS of the delta T. f. How does this accuracy compare with the nominal accuracy of the WXT510 weather sensors given in the specifications in the manual? g. If you were to deploy the MAWS and WXT510 together for another experiment would you feel confident using this calibration equation? Why or why not? 7. Plot wind direction from all the MAWS, and all of the WXT510 sensors on one figure as individual points without lines connecting them. Are you confident that all the sensors are properly aligned? If not, what is the approximate error in any sensor alignment? It may be more clear to plot only points where the wind speed is above 2 m/s. 8. When you discuss the errors in the analysis above, you should try to reach some judgment about the NATURE of the error, such as whether it is RANDOM or SYSTEMATIC, and if it is systematic, what it depends on. PURE RANDOM errors are errors that fluctuate randomly and don t depend on any other variables, including the value of the previous error. The following SYSTEMATIC errors may be present: 1) BIAS a sensor measures a parameter with an average constant offset compared to a reference measurement. 2) calibration DRIFT over time e.g. the sensor measures a parameter more accurately at the beginning of the period than at the end of the period. 3) CONTAMINATION by another environmental variable in this case, a parameter error may be correlated with another measured value, for example contamination by heating by direct sunlight. 4) NONLINEARITY the linear relationship assumed in a calibration equation is not correct. This will typically be manifested as an error that is a smooth function of the reference or true value, and would be evident in a plot, for example, of dt verus T. 6
5) TIME LAGGED RESPONSE the error is due to the sensor not responding to the most rapid fluctuations in the actual parameter, so the measured parameter appears as a smoothed version of the reference or true parameter. This will usually be most obvious in the comparison of the raw time series with the true value or reference value. In the Results section of your report, identify characteristic errors and answer the following questions: Which parameters and which instruments, if any, seem to show RANDOM errors? Which parameters and which instruments, if any, seem to show SYSTEMATIC BIAS errors? Which parameters and which instruments, if any, seem to show SYSTEMATIC DRIFT errors? Which parameters and which instruments, if any, seem to show SYSTEMATIC CONTAMINATION errors and what might be the source of the contamination? Which parameters and which instruments, if any, seem to show SYSTEMATIC NONLINEARITY errors? Which parameters and which instruments, if any, seem to show SYSTEMATIC TIME LAGGED RESPONSE errors? Which instruments would you tend to recommend as giving the most precise measurements? When answering these questions, refer to the graphs that you have created in the previous parts of the exercise. In some cases, the answer might be none. Be sure that your lab report contains all sections as described in the Lab Report handout. Attach a copy of your notebook pages to the lab report. 7