Leveraging Existing and Future In-Situ Observational Networks John Horel* Department of Atmospheric Sciences University of Utah john.horel@utah.edu * COI Disclosure: equity stake in Synoptic Data Corp. Workshop goals: Technology development to reduce cost and improve data quality Increase collaboration to advance the science
NWS National Mesonet Program In 2008, the National Academy of Sciences (NAS) identified gaps in NWS observations, particularly boundary layer wind, temperature, and humidity; solar radiation; soil moisture and soil temperature The report recommended the establishment of a multi-sector Network of Networks (NoN) to meet the observing needs and fill the gaps Key Finding of the NAS: Increased coordination among existing surface networks would provide a significant step forward and serve to achieve improved quality checking, more complete metadata, increased access to observations, and broader usage of data serving multiple locally driven needs Congress directed the creation of the National Weather Service s National Mesonet Program, which is now an established Program in the NWS budget Now includes 43 networks covering all 50 states, providing surface data, soil observation, numerous types of profiling technologies, a radar component (CASA), a mobile mesonet capability, and aircraft-based observations (TAMDAR) Curtis Marshall NMP Program Manager
NMP System Networks Earth Networks WeatherFlow Weather Telematics Sonoma Technology University of Oklahoma University of Utah (MesoWest) Utah State University University of Delaware Rutgers, the State Univ. of NJ North Carolina State University Coastal Carolina University Western Kentucky University Texas Tech University University of Georgia University of Missouri University of Illinois Kansas State University University of Alabama - Huntsville University of South Alabama Auburn University University of Florida University of Massachusetts Pennsylvania State University Washington State University State University of New York Louisiana State University Colorado State University Iowa State University Michigan State University Mississippi State University New Mexico State University New York State Mesonet Montana State University University of Nebraska Harris Corporation Radiometrics United Parcel Service DTN Ag Weather Panasonic, Inc. (TAMDAR) Multiple Data Providers Fixed Stations Mobile Platforms Data and Metadata (Multiple connections) MADIS NOAA AWIPS, NCEP, etc. Curtis Marshall NMP Program Manager
NMP Plans for the Future Government funding has been used to establish the backbone of a Nationwide Network of Networks capability via the National Mesonet Program Broaden concept of Mesonet to encompass purchase of commercial data from multiple sources, not just surface networks The networks are expanding their observational reach into the vertical dimension and provide boundary layer profiles of wind, temperature, and humidity as envisioned in NAS From the Ground Up report NWS needs the types of data National Mesonet provided in order to meet its programmatic goals for high resolution forecast products and integrated decision support services Curtis Marshall NMP Program Manager
30,000+ active stations from over 200 networks 1700+ stations added in 2017 300,000+ data records inserted every hour 120+ variables in database Reporting intervals from 1-60+ min http://mashup.mesowest.net/map_stream.html
Metadata Objective: accurate information on what is being measured Identifying extent & gaps of in-situ infrastructure remains incomplete Currently requires too much effort by station/network owners to supply critical metadata to others Critical metadata: sensor operating characteristics sensor siting (location, sensor heights, obstructions, etc.) replacement or recalibration dates Recommendation: Encourage manufacturers to implement automated procedures to transmit operating characteristics & critical metadata updates automatically
Managing Real-Time Metadata and Data Mobile Sensor Platforms: Incident RAWS Best available estimate of locations of over 50000 publicly-accessible stations now available 2017-06-22 (Station relocated) ELEVATION: 8794 from 2830 NAME: IRAWS 41(PANGUITCH LAKE) STATE: UT from ID 2017-08-30 (Station relocated) ELEVATION: 1658 from 2830 NAME: IRAWS 41 (ELK MOUNTAIN) STATE: OR from ID
Data Validity Objective: estimate validity of in-situ observations from multiple sources as data received Sensors of varying quality are deployed for diverse reasons using diverse deployment and reporting strategies Assuming single standard for all in-situ sensor data is not practical, nor likely to insure data validity Current practice for operational data assimilation tends to rely heavily on network allow/deny lists Recommendation: data mining/informatics research required to develop tools beyond traditional quality control metrics
SynopticLabs Real-Time Data Validity Checks Suite of standard validity metrics applied in real-time
Probabilistic vs. Deterministic Observations Internet of Things sensors (low-cost home units, wearables, vehicle sensors, etc.) require R&D to use them in data assimilation systems Weather Telematics
Data Licensing Objective: provide framework to sustain and grow observing networks managing data rights is necessary for networks of in-situ and surface-based remote sensors Agencies such as NOAA often enter into license agreements with network owners to access environmental information with limited rights for the agencies to redistribute that data OSSE experiments required for wide range of use cases to assess the relative value and impacts of different observational data streams
Summary Surface based in-situ sensors reporting continuously in realtime are the foundation from which to monitor and understand the boundary layer NWS National Mesonet Program designed to meet the needs of the NWS Metadata and data validity tools to manage the complex mix of in-situ observations are available but more work required Leverage the diverse nature of existing and rapidly expanding surface-based in-situ networks in combination with remote sensors and other data sources
Selected Surface-Based Remote Sensors Networks available from the National Mesonet Program https://asn.synopticdata.com/
Challenges highlighted at March 2016 AMS Forum on Observing the Environment from the Ground Up How to: maintain as well as expand local and regional surface observing networks relying on diverse data rights and funding strategies support data sharing and integrating environmental data from diverse sources identify, prioritize, and eliminate the surface and boundarylayer observing gaps in the national and international observing network most notably within coastal, urban and complex terrain areas AMS NATIONWIDE NETWORK OF NETWORKS COMMITTEE