Towards Dynamically Adaptive Weather Analysis and Forecasting in LEAD Beth Plale 1, Dennis Gannon 1, Dan Reed 2, Sara Graves 3, Kelvin Droegemeier 4, Bob Wilhelmson 5, Mohan Ramamurthy 6 1 Indiana University 2 Univ of North Carolina Chapel Hill 3 University of Alabama Huntsville 4 Oklahoma University 5 NCSA 6 Unidata
LEAD vision Cyberinfrastructure that allows mesoscale meteorology forecasters to: Dynamically and adaptively respond to weather patterns to produce better faster-than-real-time forecasts, and Run larger multi-model simulations than can do today.
Definitions Mesoscale meteorology - regional scale, severe storm forecasting Tornadoes, flash floods, severe storms Multi-model simulations - ensemble runs 20-100-500 versions of a forecast model with physics tweaked slightly differently. Results analyzed akin to distributed concensus scheme (i.e., voting) - looking for regions of uncertainty Faster-than-real-time - forecast that precedes the storm.
Problem: Conventional Numerical Weather Prediction OBSERVATIONS Radar Data Mobile Mesonets Surface Observations Upper-Air Balloons Commercial Aircraft Geostationary and Polar Orbiting Satellite Wind Profilers GPS Satellites
Conventional Numerical Weather Prediction OBSERVATIONS Radar Data Mobile Mesonets Surface Observations Upper-Air Balloons Commercial Aircraft Geostationary and Polar Orbiting Satellite Wind Profilers GPS Satellites Analysis/Assimilation Quality Control Retrieval of Unobserved Quantities Creation of Gridded Fields
Problem: Conventional Numerical Weather Prediction OBSERVATIONS Radar Data Mobile Mesonets Surface Observations Upper-Air Balloons Commercial Aircraft Geostationary and Polar Orbiting Satellite Wind Profilers GPS Satellites Analysis/Assimilation Quality Control Retrieval of Unobserved Quantities Creation of Gridded Fields Prediction PCs to Teraflop Systems
Problem: Conventional Numerical Weather Prediction OBSERVATIONS Radar Data Mobile Mesonets Surface Observations Upper-Air Balloons Commercial Aircraft Geostationary and Polar Orbiting Satellite Wind Profilers GPS Satellites Analysis/Assimilation Quality Control Retrieval of Unobserved Quantities Creation of Gridded Fields Prediction PCs to Teraflop Systems Product Generation, Display, Dissemination
Problem: Conventional Numerical Weather Prediction OBSERVATIONS Radar Data Mobile Mesonets Surface Observations Upper-Air Balloons Commercial Aircraft Geostationary and Polar Orbiting Satellite Wind Profilers GPS Satellites Analysis/Assimilation Quality Control Retrieval of Unobserved Quantities Creation of Gridded Fields Prediction PCs to Teraflop Systems End Users Product Generation, Display, Dissemination Nat l Weather Service Private Companies Students
Problem: Conventional Numerical Weather Prediction OBSERVATIONS Radar Data Mobile Mesonets Surface Observations Upper-Air Balloons Commercial Aircraft Geostationary and Polar Orbiting Satellite Wind Profilers GPS Satellites Analysis/Assimilation Quality Control Retrieval of Unobserved Quantities Creation of Gridded Fields Prediction PCs to Teraflop Systems The process is is entirely serial and pre-scheduled: no no response to to weather! Product Generation, Display, Dissemination End Users NWS Private Companies Students
Goal: Adaptive Forecast NexRad radar ingest Fetch data products Convert to format suitable for assim Assimilate Into 3D grid Plan 20 run ensemble Forecast model execution (20 versions) Analyze final results of each run 0600 0800 1000 1200 1300 Request to NetRad radar control system 1400 2100 collect data 12 hr forecast 6 hr forecast 3 hr forecast afternoon storms
Selected data gathering possible with NEXRAD Moore OK tornado, 3 May 1999 Steerable, 90 o sweep
Computational needs of forecast models Single 27 km resolution CONUS forecast (continental US) 30 node dual processor 2.0 GHz Pentium IV 6 hrs to generate 84 hour forecast Regional runs take less time Ensemble runs consume more resources
Real-time response: a mirrored system
Requirements of Adaptive System Highly heterogeneous and numerous sources of adaptive events Environment level Application level Service level Hardware level Numerous simultaneous forecasts over limited resources Prioritizing events and users Expecting the unexpected
Bus architecture Serves multiple levels of events
LEAD Architecture: adaptivity service interaction Crosscutting Services User Interface LEAD Portal Desktop Applications IDV WRF Configuration GUI Portlets MyLEAD Visualization Workflow Education Browse Control Ontology Query Monitor Control Client Interface Authorization Authentication Monitoring Notification Configuration and Execution Services Resource Access Services Application Resource Broker (Scheduler) Application & Configuration Services Host Environment Application Host GPIR Geo-Reference GUI Grid FTP GRAM Execution Description Application Description WRF, ADaM, IDV, ADAS Scheduler SSH LDM Workflow Monitor Workflow Engine/Factories OPenDAP VO Catalog THREDDS Workflow Services Catalog Services Generic Ingest Service Stream Service Query Service Decoder/ Resolver Service RLS Control Service Ontology Service Transcoder Service/ ESML OGSA- DAI Data Services Distributed Resources Computation Observations Streams Static Archived Specialized Applications Steerable Instruments Data Bases Storage
Workflow GBPEL (Gannon, IU) Based on Business Processing Execution Language (BPEL) Define workflow in terms of sequence of wellplanned tasks, but also Define task in response to anticipated trigger events Application services publish status notifications to which workflow is responsive Ongoing research in expecting the unexpected
Monitoring Based on Autopilot (Reed, UNC) Monitors progress of workflow Demo at LEAD AHM (May 05): Instrument WRF forecast model to capture dynamic performance data from hardware performance counter, stream result through Autopilot sensor information.
Weather detection Mesoscyclone Detection Algorithm (Graves, UAH) Mesocyclones contain velocity signature. UAH-MDA identifies shear segments based on this segment Applies fast classification techniques Operates on the fly
CASA/LEAD Interaction NEXRAD RADARS Weather Detection Algorithms NETRAD/CLEAR Optimal Control Systems NETRAD Data Commands Data QC & Assimilation Algorithms Hazard Detection Algorithms Optimal Control Systems Processed Data Model Output Algorithm Detections LEAD Experimental Numerical Weather Prediction Models Signal Processing, Quality Control Engineering and Scientific Research: Sensor Hardware, Middleware, Storage, Communications, Computing NETRAD Data NETRAD Data NETRAD Data Meta Data Creation, Real Time Data and Meta Data Distribution via Web, Storage/Archival CASA NETRAD Data Model Output Graphical Products Web Portal Private Sector: Value-Added Products and Services, Decision Support Systems Oklahoma Climate Survey