Thermodynamic Profiling Technology Workshop Characteristics of a Testbed: NOAA/NWS Testbed Implementation David Helms Office of Science and Technology NOAA National Weather Service Boulder, Colorado April 13, 2011
If given a $100M plus-up (cut), what portfolio investments (re-balancing) should I make to have the greatest (least) improvement (decrease) to short-term predictability Don Berchoff Director, Office of Science and Technology NOAA National Weather Service Testbeds are a means to an end in this regard
U.S. Weather Research Program (USWRP) Testbed Definition A testbed is a working relationship in a quasi-operational framework among measurement specialists, forecasters, researchers, the private sector, and government agencies aimed at solving operational and practical regional problems with a strong connection to the end users. Outcomes from a testbed are more effective observing systems, better use of data in forecasts, improved services, products, and economic/public safety benefits. Testbeds accelerate the translation of R&D findings into better operations, services, and decisionmaking. A successful testbed requires physical assets as well as substantial commitments and partnerships 1 1. Dabberdt, W. F., et al., 2005. Multifunctional mesoscale observing networks. Bulletin of the American Meteorological Society 86(7):961 982
Washington Post - April 10, 2011 Characteristics of Testbeds: 1. To Improve Services 2. To Assess Performance What should be assessed? Are we assessing performance adequately?
Value Chain Analysis: Assessing Linkages Between Service and Capabilities Quality Control and Metadata impact every link in the Value Chain they need to an assessed for impact! B. Quality Control (feedback) D. Metadata, Data Assumptions, Scaling (time, space) Model Guess D. Model Physics, Parameterization Resolution A: Product/Service PM B: Key Products/Services C: Intermediate Products D: Systems Producing Products E: Requirements F: Observing Systems F. D. D. C. C. B. Platforms/ Sensors Raw Data (B1) Data Assimilation Analysis Guidance Products Societal Benefits Decision Quality Control Aids assessment to determine requirements D. satisfaction Verification Human in (over) the Loop Each link in the Value Chain needs rigorous D. C. D. C. D. C. and impact to service outcomes Training D. A., E. Performance Measures and Requirements: Calibration, Logistics, Drift Accuracy, Availability % Signal Retained, Noise Reduced, Accuracy, Resolution, Timeliness Predictability, Timeliness Predictability, Timeliness Outcomes
NOAA s Weather Enterprise Value Tree Hundreds of Products and Services Several Program Areas Three Models One Data Assimilation System Hundreds of Observing Systems Platforms and Sensors Raw Data Data Assimilation Decision Analysis Models Assessment Products Making $ $ $ $ $ $ $
NOAA Testbed Capabilities Testbed Name Aviation Weather Testbed Climate Testbed Developmental Testbed Center GOES Proving Groud Hazardous Weather Testbed HF Radar Testbed HydroMet Testbed HydroMet Testbed - NCEP Joint Center for Satellite Data Assimilation Joint Hurricane Testbed Joint Numerical Testbed National Weather Radar Test Bed (NWRT) OSSE Testbed Regional Storm Surge and Inundation Model Testbed Unmanned Aerial System (UAS) Testbed Organization NWS/NCEP/AWC NWS/NCEP/CPC OAR/ESRL/GSD NESDIS/GOES R OAR/NSSL NOAA/IOOS OAR/ESRL/PSD NOAA/NWS/NCEP/HPC NOAA/NWS/NCEP/EMC NOAA/NWS/NCEP/TPC NOAA/NWS/OST NOAA/OAR/NSSL NOAA/OAR/AOML NOAA/IOOS NOAA/OAR/ERSL Sensor (static and mobile) Test Facilities Air Quality, Boundry Layer Processes - Oar Ridge, TN NOAA/OAR/ARL ASOS Test Facility - Johnstown, PA NOAA/NWS/OPS Buoy, CMAN marine systems - Bay St Louis, MS NOAA/NWS/OPS/NDBC Mobile Profiling Sensors - Boulder, CO NOAA/OAR/PSD P-3, G-IV aircraft - Tampa, FL NOAA/OMAO Radar Operations Center - Norman, OK NOAA/NWS/OPS Sterling Research and Development Center - Sterling, VANOAA/NWS/OPS Urban Processes - DCNet NOAA/OAR/ARL
Decision Analysis Models Assessment Products Making Data Assimilation Raw Data Platforms and Sensors Positioning of NOAA Testbeds Supporting Value Tree Assessments Climate Joint Hurricane Joint Numerical GOES Proving Ground Aviation Weather Center HydroMet Testbed (NCEP) Hazardous Weather HydroMet Testbed (OAR) Storm Surge and Inundation Model Developmental Testbed Center OSSE Testbed Joint Center for Satellite Data Assim UAS National Radar HR Radar Assessments in this portion of the Value Tree may yield cost/benefit alternatives
TPIO Technology Planning and Integration Office Leveraging and Extending the Application of Testbeds: Assessing the Observing System Architecture NOAA Near-term Assessment (2011): Optimized Value Tree Corporate Portfolio Analysis Decision Memorandum: March 3, 2011 Recommend cross-noaa observing system efficiency gains using observing system experiments and other analysis techniques and involving the external community as appropriate, identifying implementation opportunities for FY 12 and plans for further work in FY 13 (lead: NOSC, due: Sep 30, 2011). FY 12-16 Program Decision Memorandum Action to NOSC: March 2010 form a NOAA-wide team to evaluate all systems (space, air and ground based) measuring atmospheric moisture and temperature develop a comprehensive plan to capture these measurements in a more integrated fashion and include recommendations for improvement Annual Guidance Memorandum: On Nov 16, 2010, NOAA released the draft Annual Guidance Memorandum (AGM) that directed: It [NOAA] should strengthen and integrate NOAA s upper air observing capability through analysis of atmospheric profiles of air temperature, water vapor, and wind vectors. 9
TPIO Technology Planning and Integration Office Leveraging and Extending the Application of Testbeds: Assessing the Observing System Architecture NOAA Task Deliverables: Optimized Value Tree 1.Recommended inventory of upper air systems providing the greatest NOAA operational and science requirements satisfaction for atmospheric temp, moisture, winds 2.Timeline of decision points for current and planned upper air observing systems (Now +20 Years) 3.(Internal OSC) Produce Evaluation Technique lessonslearned for future observing systems integrated assessments 10
Workshop 2 Objectives Determination of Techniques for Value Tree Assessments Question 1: What techniques are available to estimate impact of current observing systems, data assimilation, and modeling to product accuracy and/or performance measures? Question 2: What techniques are available to estimate impact of changes to current the observing system, data assimilation, and modeling to product accuracy and/or performance measures? Question 3: What techniques are available to estimate impact of future combinations of observing systems, data assimilation, and modeling to product accuracy and/or performance measures? Question 4: What ancillary processes/information, which impact the quality of the observed data and data analysis, might be considered in the value chain trade space? 11
Bottom Line: NOAA needs Business-Case quality information supporting maintenance of and/or improving services Without strong evidence of efficiencies and/or substantial service improvements, funding will not be forthcoming Observing Systems are one of many dependencies in the Value Tree contributing to service delivery performance Assessments must look at the full set of alternatives that most impact services