Russ Ingersoll OC 3570 Project Report 16 SEP, Instrument Accuracy Analysis

Size: px

Start display at page:

Download "Russ Ingersoll OC 3570 Project Report 16 SEP, Instrument Accuracy Analysis"

Karen McKenzie
5 years ago
Views:

1 Russ Ingersoll OC 3570 Project Report 16 SEP, 2009 Instrument Accuracy Analysis Introduction: As discussed in the Instrument Accuracy Analysis brief given in Spanagel 332 on September 9 th, the purpose of this project is to demonstrate similarities or differences in measurements of the same environmental parameter, in this case Sea Surface Temperature (SST), as conducted with different instruments. Three types of automated, mounted SST instruments were onboard the RV New Horizon for the July 2009 NPS cruise, and one Hand-held instrument was used for hourly observations by the watch teams. Data from all four sources is utilized in this analysis. Considerations are made as to the strength of correlation among the readings. Strong correlations are shown for certain instruments, despite that trivial differences are present. If differences are found to be non-trivial, the issue is further investigated so as to determine if an appropriate translation (or correction) can be found, so the erroneous instrument s readings can be adjusted to fit the truth. When there is a lack of predictability in the errors of an instrument, it may be impractical to apply a correction. Specifically, the issue of an instrument s predictability (or consistency) must be evaluated with respect to possible influences which may be unique to that instrument. If incorrect readings seem at first glance to be random vs. predictable, the end-user should consider the likelihood of errors due to known spatial &/or temporal influences, or with

2 any reasonable hypothesis of locally induced corruptions of the instrument (e.g., operator error on a particular cruise, or presence of an obscuration atypical to what the instrument is designed for, etc.). When all these factors have been considered and the data bears no sensible association with any of the influences, it must be considered randomly erroneous. This project identified one such sub-set of SST data, and it is suggested this data would not be useable except for very general purposes. It will be shown that simple human error (specifically, variation of personal technique) is the reason this data is not reliable. It will also be shown, that one instrument was erroneous with an especially consistent error margin, and it can be adjusted to the truth reliably. This type of erroneous data might also be referred to as biased. Correcting biased data is applied on a regular basis for a variety of instruments, such as the Infrared sensors utilized on the RV New Horizon cruise. All SST values or corrections discussed hereafter are in degrees Celsius. Operational Importance (Project Motivation): Consider any of the numerous follow-on uses of measured environmental data (research, commerce, at-sea salvage, military operations, etc.), and estimate the importance of data accuracy as it might lead to better or worse calculations for a variety of applications in support of the activity. It is usually essential that the initial measured data be as accurate as possible. The salvage operations conducted on the Rockwater II (pictured left) exemplify how easy it is for in-situ data to be too 2

3 erratic for the follow-on application. During these salvage ops, accurate forecasts for sea state were critical for mission planning, especially with regard to evolutions dependant on the ship s mounted cranes. The bridge crew was a variety of individuals of various backgrounds in weather observation, and these individuals were responsible for sea state observations at the start of the mission. Shore-based forecasters utilizing these observations, noticed varying sea state reports associated with non-varying wind intensities, and the process of providing timely & accurate forecasts became impaired by the discontinuity of the reports from the ship. An individual, well trained Meteorological Technician (Met-Tech) was assigned to the vessel to take over the sea state observation duties on a 24/7 basis. This immediately provided consistency of method as well as the reported conditions, and it facilitated improved forecast accuracy and associated mission planning. If the data is to be reliable, a simple parameter like sea state or SST, must be measured and/or observed with due attention to the reasons why inaccuracies often become introduced. This NPS cruise SST data serves as another example of the vulnerabilities of some types of instruments and the potential errors due to different instrument operators/observers. Instruments: On the RV New Horizon, there were four methods of continuous SST data collection: 1) Boom, 2) Intake, 3) Mounted Radiometer, and 4) Hand-held Radiometer. Details of how these instruments function are as follows: The SST Boom is an in-house sensor fabricated by the Naval Post-Graduate School Oceanography department. It utilizes a Linear Technologies thermistor, housed in PVC 3

pipe, and fitted with electronics to interface it with a cable for communication with onboard data collection SST Boom by laptop computer.

A Seabird temperature probe was used to control water temperature variations and a Rosemont platinum resistance thermometer was used to validate accuracy of the Boom

Communication with the MicroTSG is over an internal, RS-232C link, providing real-time data transmission (Seabird, 2009).

002 C per year) (Seabird, 2009). The underway watch teams recorded the Intake SST readings each hour from the display panel ( repeater ) in the ship s lab.

4 pipe, and fitted with electronics to interface it with a cable for communication with onboard data collection SST Boom by laptop computer. The boom was calibrated in the NPS cal-lab bath. A Seabird temperature probe was used to control water temperature variations and a Rosemont platinum resistance thermometer was used to validate accuracy of the Boom thermistor. The Intake SST data was collected by the In-house system on the RV New Horizon. This is a SBE 45 MicroTSG Thermosalinograph, made by Seabird. Communication with the MicroTSG is over an internal, RS-232C link, providing real-time data transmission (Seabird, 2009). The SBE 45 utilizes a pressure-protected thermistor which has a long history of exceptional accuracy and stability (typical drift is less than C per year) (Seabird, 2009). The underway watch teams recorded the Intake SST readings each hour from the display panel ( repeater ) in the ship s lab. To collect SST values remotely, the Ship was fitted with mounted Infra-red sensors made by Wintronics. KT 15 Specifically, we used Model KT15.82II infrared thermometers (Wintronics, 2009). Two of these were installed side-by-side on the weather decks, each mounted at 45 degree angles, one upward looking and the other downward looking. They accommodate a wavelength band of 8 to 14 micrometers 4

5 and achieve a field of view of 9 degrees. Emissivity was set at 1.0 for both thermometers. These instruments were automatically recording degrees Celsius values on a continual basis while at sea. As mentioned there was also a hand-held instrument (Model KT19) with very similar functionability. However, The KT19 works in the 9 to 11.5 micrometer wavelength band. This band is slightly better for colder temperature ranges. Analysis Methods: Data collection was mostly automated for SST for the duration of the cruise. The exception was the hand-held radiometer (KT19), which was used by each watch section to obtain hourly readings of both SST and sky temperature. These readings were done with the attempt to consistently hold the instrument at an approximate 45 degree angle (both upward and downward). Hourly readings were recorded on the weather observation log, along with SST Intake, cloud cover, and some other parameters not utilized in this report. Post-trip data sets for the four instruments was entered into Matlab so as to create a matrix of relevant vertical data vectors organized with respect to rows that represent each hour that the watch teams recorded the necessary parameters. The first set of observations utilized is assigned row number zero, then all the subsequent hours with good data becomes a vertical Matlab vector representing the total number of samples. The beginning of the first watch until the last observation by the last watch provides 91 total consecutive hours, there-by this Reference Vector is simply a column from 0 to 91. This Reference Vector drives the creation of all other columns in the matrix and facilitates the coordination of any given SST value with a certain date and 5

time of day during the cruise, and it allows full SST vectors to be plotted against each other or other analysis methods to be performed with the ease of having matching vector lengths.

6 time of day during the cruise, and it allows full SST vectors to be plotted against each other or other analysis methods to be performed with the ease of having matching vector lengths. Figure (1) shows the data matrix in full where the ten columns are the Reference Vector, Zulu Time, Pacific Daylight Savings Time, Cloud Cover, KT19 SST, KT19 sky, Intake, KT15 SST, KT15 sky, and lastly Boom SST. ure (1) Fig Initially the Intake and Boom vectors were chosen for analysis to determine correlation, and then find any correction that should be applied if one exists. When these two vectors are co-plotted it is effectively a near perfect overlay, and the correlation coefficient is.995. Figure (2) shows the strong similarity achieved by these two instruments. It was expected for these sensors to be a close fit with no adjustment and for each of them to demonstrate exceptional accuracy given the level of careful calibration and field-tested track record for each instrument. No correction is applied. 6

Figure (2) With these vectors established is good representatives of the truth, the Boom was chosen as a vector to evaluate the accuracy of the KT15 and KT19 information.

7 Figure (2) With these vectors established is good representatives of the truth, the Boom was chosen as a vector to evaluate the accuracy of the KT15 and KT19 information. First, let s look at a simple plot of all four vectors and get a general impression of what may be expected. Consider Figure (3) where the color scheme is as follows: Red=Intake, Blue=Boom, Green=KT19 (hand-held IR), and Black=KT15 (mounted IR). Figure (3) 7

8 With the Intake and Boom established (red and blue), consider the hand-held IR in green vs. the mounted IR in black. Initially it seems there is a problem with the KT19. When this vector is over-layed with the Boom the correlation coefficient drops to.95, notably lower than the.995 for Intake with Boom. This could be considered a decent correlation coefficient, but what is un-nerving in particular is that the KT19 curve varies from greater vs. lesser values than the Boom and there is no organized pattern. There are places where the KT19 almost hugs the Boom curve, and then there are places where it almost hugs the KT15 curve. It was expected the two IR sensors would share stronger similarities with each other throughout the entire time-series. The KT15 curve (black) is obviously not accurate in its raw form, but it is visually very consistent with the Boom curve pattern, so there seems to be consistency in the errors. It seems the KT19 may not be correctable while the KT15 is correctable. This initial evaluation of all four curves plotted together was the method for basic identification of potential problems. In this matrix, only the KT19 is visually disturbing, since it appears to have erratic errors. Let s look a little further and evaluate scatter- plots of each of the IR devices vs. the Boom. Figure (4) shows KT19 vs. Boom, while Figure (5) shows KT15 vs. Boom: Figure (4) Figure (5) 8

Immediately, the mounted sensor indicates that a correction may be applicable with respect to almost all of the temperature values in the range, while there is less hope to correct the hand-held

334 degrees is recommended to correct the KT15 data to fit the truth as represented by the Boom. Let s evaluate this visually and see if it seems to be a good adjustment.

9 Immediately, the mounted sensor indicates that a correction may be applicable with respect to almost all of the temperature values in the range, while there is less hope to correct the hand-held sensor. With Matlab curve fitting methods applied to the Figure (5) vectors, it is found that an adjustment of degrees is recommended to correct the KT15 data to fit the truth as represented by the Boom. Let s evaluate this visually and see if it seems to be a good adjustment. Figures (6) is the Boom with the original KT15 vector, while Figure (7) shows Boom with KT15 curve adjusted by degrees. Figure(6) Figure(7) Visually, this is very satisfying at first glance. It is likely that for many follow-on uses the degree correction would suffice. Taking a closer look at Figure (7), consider the locations along the time-series where there are obvious disparities. At the 0 th, 48 th and slightly beyond the 70 th samples it is identifiable that the adjusted KT15 curve is still below the Boom. Seeing distinct discontinuities occurring every now and again, raises the concern of additional margins of error not yet accounted for in the correction. Other factors contributing to errors will 9

10 not be problem, so long as there is predictability. If we consult the original data matrix, Figure (8), and look at the Pacific Daylight-Saving Time (PDT) as well as the cloud cover (SkyCon) information, we see some key points of interest. Figure (8) reveals that the 0th, 48th and 72nd samples were late in the afternoon (all three events occurring near 1600 local time), and that there was also a decrease in cloud cover at about that same time, allowing some direct sunlight to hit the sea surface. These three data points occur at 25/00, 27/00, and 28/00, all times Zulu. Figure (8) Now consider the unique properties of the downward looking IR sensor. Sensors of this nature are receiving emittance from a very thin portion of the water surface (less than 1 mm), and the ocean does not emit as a blackbody with respect to the water vapor wavelength band (Lind et al, 1989). A certain percentage of the radiance that reaches the IR sensor is reflected radiance from the skyward hemisphere (Lind et al, 1989). 10

It is also noteworthy that the analysis of ocean surface Heat Flux shows spikes at similar times as the enhanced disparities of the adjusted curves.

11 It is also noteworthy that the analysis of ocean surface Heat Flux shows spikes at similar times as the enhanced disparities of the adjusted curves. Figure (9) demonstrates the temporal distribution of Heat Flux variability over the series (Lewis, 2009): Figure (9) The immediate sensitivity to micro-adjustments in photons received from the skin of the sea surface is a benefit in ways, but also something of vulnerability with respect the IR sensor accuracy in a changing environment. However the concern about consistency is satisfied and the biases can be accounted for with further corrections. In a Technical Report from the Naval Post Graduate School (NPS), titled Sea Surface Temperature Fields Derived From Aircraft and Ship Observations During FASINEX 1986, the authors discuss the methodology for fully correcting IR derived temperatures to true SST values by use of the following formula (Lind et al, 1989) : Emittance = 0.986, Reflectance = a 1 T sfc = ( T D ow n T U p ) 4 = a = (1) 11

12 Results: Multiple data vectors were successfully created to facilitate analysis of all four SST instruments utilized on the cruise. The analysis achieved confirmation of the validity of both the Intake (SBE 45) and the Boom (NPS in-house design). With the Boom established as truth, the IR derived vectors were able to be evaluated with confidence. The hand-held IR instrument produced data that cannot be deemed reliable while the mounted IR data can be successfully corrected. The specific reason for the erratic errors in the hand-held IR readings is determined as operator error. By virtue of the consistent performance of the KT15 it seems the only influence on the KT19 was the variation in individual observer s technique operating the device each hour. This is plausible, especially given past situations when the practice of rotating personnel became well established as the root cause of data inconsistencies. Successful correction is made to the mounted IR vector such that the correlation coefficient of the adjusted curve to the Boom is.991. Although further corrections are not performed in this project, it is shown that predictable influences create additional biases in the IR readings, therefore a more sophisticated method of correction must ultimately be applied beyond the translation utilized here. Conclusions and Recommendations: SST measurements, like so many other measurements are vulnerable to error when due care is not taken to properly set up, calibrate, or manually operate the instruments and sensors. Even when properly calibrated instruments are properly mounted, there are local effects that must be compensated for in order to have accurate values at the end of 12

13 the day. When any hand-held instrument is used, the potential for error increases. When multiple operators are assigned to take measurements, the potential for error increases much further. Specific training and minimization in the number of observers is recommended whenever achievable. As was practiced on the NPS RV New Horizon summer cruise, keeping a log of all significant events and of all measurements taken, including the time and location, will tend to facilitate going back to evaluate curiosities that may show up in the data at a later date. Ultimately this project shows the value of having multiple instruments available to monitor the same environmental parameter concurrently. This type of multiplicity should be employed whenever possible. References: 1. Lind, R. J., and W. J. Shaw, 1989: Sea Surface Temperature Fields Derived From Aircraft and Ship Observations During FASINEX 1986, Naval Post Graduate School. 2. Lewis, D.L. 2009: Surface Heat Flux Analysis Class Project, Naval Post Graduate School. 3. Wintronics, Inc, Seabird, Inc,

ISAR (Infrared Sea Surface Temperature Autonomous Radiometer)

5.14.1 ISAR (Infrared Sea Surface Temperature Autonomous Radiometer) (1) Personnel R. Michael Reynolds (RMRCO) Principal Investigator Jeremiah Reynolds (RMRCO) Instrument Technician (2) Objective a. To