Bridging the gap in operational hydrologic data assimilation - Challenges and a way forward

Transcription

1 Bridging the gap in operational hydrologic data assimilation - Challenges and a way forward D.-J. Seo 1, Yuqiong Liu 2,3, Haksu Lee 2,4, Victor Koren 2, Jiarui Dong 5,6, Michael Ek 5, Pedro Restrepo 2 1 The University of Texas at Arlington, Arlington, TX, USA 2 NWS/Office of Hydrologic Development, Silver Spring, MD, USA 3 Riverside Technology, Inc., Fort Collins, CO, USA 4 University Corporation for Atmospheric Research, Boulder, CO, USA 5 NWS/National Centers for Environmental Prediction, Camp Springs, MD, USA 6 I. M. Systems Group, Inc., Rockville, MD, USA 1

2 Acknowledgments APRFC And many collaborators in the research and operational communities 2

3 In this presentation Context of this presentation DA in operational hydrology in NWS Past, present Lessons learned Needs Way forward Strategy, opportunities 3

4 Community Hydrologic Prediction System (CHPS) Flexible, open modeling architecture linking program elements Modular software to enhance collaboration and accelerate R2O Extension of the Flood Early Warning System (FEWS) architecture: Incorporates NWS models with models from FEWS, U.S. Army Corps of Engineers (ACE), and academia Implementation Status: AWIPS-II compatible prototype hardware and software for all RFCs Conducting parallel operations at 4 RFCs, remaining by early 2011 Retire legacy system in early 2012 FEWS Models NWS Models FEWS ACE Models From Carter (2010) Other Models 4

5 CHPS and Hydrologic Data Assimilation (DA) From 5

6 DA planning with the RFCs (2008) Develop Phase 2 R&D and research-to-operations (RTO) transition plans for the Hydrologic Ensemble Forecast System (HEFS) Identify ensemble and DA capabilities that may address new and existing service needs (particularly in light of recent experiences and emerging issues) Implementable in CHPS Steer enhancement/evolution of FEWS Strengthen collaborations with the external research community 6

7 (Extra) Motivation One can only forecast based on all available information There are/will be more sources of information There are/will be significant additions to the forecast process They will require additional attention and intervention, and expertise and judgment of human forecasters Making dynamically and statistically coherent/consistent sense out of them is and will be a large challenge It is (past) time to rebalance what human forecasters (should) do and what computers (should) do Forecaster role: Master analyzer, interpreter, judge and translator of all available hydrologic, hydrometeorological and hydroclimatological information In CHPS, we have a miss-it-or-lose-it window of opportunity to do this rebalancing toward realizing the integrated water science and services vision 7

8 A brief history of DA in NWS 8

9 Blending (Adjust-Q) Practiced ever since river forecasting began Merges the observed hydrograph with the simulated Does not correct the underlying cause of the discrepancy Is objective and easy to implement Cannot correct for certain common sources of errors such as erroneous precipitation reports or timing errors 9

10 CHAT (Computed Hydrograph Adjustment Technique) Sittner, W. T. and Krouse, K. M. (1979) Improvement of hydrologic simulation by utilizing observed discharge as an indirect input (Computed Hydrograph Adjustment Technique-CHAT). NOAA Tech. Memo. NWS HYDRO-38, US Dept of Commerce, Silver Spring. Applicable to runoff events not involving snow and to headwater basin outflow hydrographs Adjusts the precipitation input and modifies the shape of the unit hydrograph until the simulation is in satisfactory agreement with the discharge observation Utilizes a unique objective function that models, to some degree, the thought processes which a human forecaster uses in judging the seriousness of a disagreement between the rising limb of a simulated hydrograph and the discharge observations Unlike conventional optimizing techniques, does not seek to minimize the objective function, but rather, attempts to reduce it to an acceptable value 10

11 Assimilation of snow course data Carroll, T. R. (1979) A procedure to incorporate snow course data into the National Weather Service River Forecast System. Proceedings, Modeling of Snow Cover Runoff (AGU, AMS, Corps of Engineers and NWS, Hanover, New Hampshire, September 1978), pp Incorporates snow course data into the NWSRFS Weights the simulated and observed water equivalent values Intended primarily to reduce monthly volume errors Is of particular value where the precipitation data do not accurately represent the snow accumulation at higher elevations Basins where precipitation stations are located primarily at elevations lower than the snow courses 11

12 State updating via Kalman filter Kitanidis, P. K. and Bras, R. L. (1978) Real time forecasting of river flows. Ralph M. Parsons Laboratory for Water Resources and Hydrodynamics, Dept of Civil Engineering, MIT, TR235. Uses SAC (re-)written in continuous state-space form with certain modifications Linearized using various techniques and then integrated to obtain the state transition matrix for a discrete time Kalman filter An automatic procedure to estimate the model noise covariance matrix and input noise covariance A generalized likelihood ratio test to detect input errors Limitations Considered only SAC with a crude procedure to time-distribute channel inflow and route to the gauging station Considered the model and input noise covariance to be constant The procedure used to identify input errors did not consider the pure timing error 12

13 SS-SAC (State-Space Sacramento) NWSRFS Operation No. 22 Georgakakos, K. P., and J. A. Sperflage, 1995: Hydrologic Forecast System - HFS: A user s manual. HRC Tech. Note 1, Hydrologic Research Center, San Diego, CA, 17 pp. Performs real time updating of SAC and channel-routing model storage elements from observations of discharge, and provides discharge forecast-error variance Uses user-supplied degree-of-belief estimates for the variance of SAC parameters and of forecast evapotranspiration demand and precipitation or rain-plus-melt input to SAC The channel model component is a conceptual kinematic channel routing model consisting of a series of nonlinear or linear reservoirs In the latter case, the unit hydrograph coordinates may be used to estimate the channel-routing model parameters 13

14 ASSIM (Assimilator Operation) NWSRFS Operation No.? Koren, V. and Schaake, J. (1993). Nile Technical Note #147: Updating Algorithm and Program. A soil moisture updating technique which modifies current soil moisture states Uses the iterative Rosenbrock Optimization Technique (Rosenbrock, 1960) to estimate the current 'optimal' soil moisture states Initial soil moisture states and input precipitation are varied over the run period and an objective function that includes the difference between observed and simulated flows, precipitation adjustments and state adjustments, is minimized 14

15 SEUS (Snow Estimation & Updating System) McManamon, A., R. K. Hartman, and R. Hills, "Implementation of the Snow Estimation and Updating System (SEUS) in the Clearwater River Basin, Idaho", Proceedings: 63 rd Annual Western Snow Conference, pp , Sparks, NV., April Consists of three components calibration, operational, updating The updating component Modifies the existing snow water equivalent states of the conceptual snow model within NWSRFS based on the weighted contributions of the simulated model snow states and the estimates of the snow states developed using the snow observations The weighting for each estimate reflects the relative uncertainty of the estimate due to such factors as model bias and measurement error 15

16 Snow Updating System Snow updating user manual, Version Updated , Riverside Technology, Inc. An extension to NWSRFS and provides tools to update the snow conditions Consists of a graphical user interface (GUI) and batch programs, which are based on a Principal Components Analysis (PCA) program The batch system can process all or a subset of the basins in a system The GUI simplifies running a subset of the system and also provides displays to view the data and results Initially consisted only of the batch snow update software, which was implemented by Riverside Technology, inc. for the Bonneville Power Administration. The GUI was subsequently added, using BPA support and technologies developed by RTi The PCA software was developed by Dave Garen of the USDA 16

17 Recent prototype development 2DVAR State updating for SAC-UHG Seo et al. (2003) Implemented in the Site Specific Hydrologic Prediction (SSHP) system, used at a number of WFOs and RFCs 4DVAR State updating of gridded SAC and kinematic-wave routing in Research Distributed Hydrologic Model (RDHM) Lee et al. (submitted to AWR) 1DVAR Real-time updating of 3-parameter Muskingum routing Under integration in CHPS via OpenDA 17

18 2DVAR - Comparative evaluation 18

19 FA VAR-aided SAC-UHG forecast FC MOD a -aided SAC-UHG forecast a Run-time modification by human forecaster Seo, D.-J., L. Cajina, R. Corby and T. Howieson, 2009: Automatic State Updating for Operational Streamflow Forecasting via Variational Data Assimilation, 367, Journal of Hydrology,

20 From Snow updating user manual (2003) From When NWSRFS, SEUS, and Snow Intersect. Vernon C. Bissell, Western Snow Conference, Bend, Oregon,

21 Reality NWSRFS Operations in use (as of October 2007) Operation #RFCs ADD/SUB 13 ADJUST-H 1 ADJUST-Q 13 ADJUST-T 2 API-CONT 1 API-HFD 1 API-MKC 1 BASEFLOW 4 CHANGE-T 13 CHANLOSS 11 CLEAR-TS 13 CONSUSE 3 DELTA-TS 3 DWOPER 3 FFG 10 FLDWAV 6 GLACIER 1 LAG/K 13 LAY-COEF 1 LIST-FTW 2 LOOKUP 11 LOOKUP3 6 MEAN-Q 13 MERGE-TS 12 MULT/DIV 4 MUSKROUT 3 NOMSNG 9 PLOT-TS 8 PLOT-TUL 13 RES-SNGL 11 RSNWELEV 4 SAC-SMA 12 SARROUTE 1 RES-J 7 SET-TS 9 SNOW SSARRESV 1 SS-SAC 1 STAGE-Q 13 STAGEREV 1 TATUM 2 TIDEREV 2 UNIT-HG 13 WEIGH-TS 10 Snow Updating System used at NWRFC 2DVAR, implemented in the Site-Specific Prediction System (SSHP), is used by various WFOs and RFCs 21

22 2DVAR - Lessons learned Use the same, operational models (soil moisture accounting, snow, routing, etc.) Model physics and parameters must be the same and completely transferable Allow forecaster control/intervention To reflect any prior or additional information that the forecaster may have Restart (warm or cold) may be necessary if the model deviates from the real world Provide, and effectively present, model-dynamical information that explains the DA results Clearly demonstrate the value of DA through objective comparative verification In the context of the end-to-end forecast process Relative to the current operational practice Training 22

23 Accounting for timing errors due to spatially non-uniform distribution of rainfall and/or varying rainfall intensity Levee failures From MBRFC s presentation in XEFS Phase 2 R&D and RTO Planning (2008) 23

24 From MBRFC s presentation in XEFS Phase 2 R&D and RTO Planning (2008) 24

25 For forecaster control For forecaster control of the end-to-end DA process, versatile, informatics-based graphical user interface would be necessary that allows, e.g., displays of with- and without-da results over multiple time periods for pattern identification. It is possible that, at times, the forecaster may have to restart the DA process, going back to some user-specified time and negating any changes thereafter. The DA tools should be flexible enough to allow such forecaster-controlled warm restarts. From NWS/OHD Strategic Science Plan (2010) 25

26 Science and technology End-to-end DA challenges A suite of integrated DA tools for forcings, hydrologic models and hydraulics models Decomposition, control Synergistically linking land DA and DA for operational hydrologic forecasting 26

27 Closing thoughts Development, implementation and Institutionalization of forecastersupervised automatic DA in operational hydrology will require: Collective efforts from both the research and the operational communities Cross-cutting community planning and action to capitalize on recent advances and to seize the window of opportunity We, the cross-cutting community, need: A science roadmap to Lay out clear and scientifically-sound pathways to operationallyviable practical solutions Identify and articulate specific scientific and technological challenges therein Community-wide tools to Make R&D and RTO transition feasible Improve cost-effectiveness Foster cross-cutting collaborations 27

28 What kind of tools? NOAA NASA EPA New observing systems Experimental Data Assimilation Once validated, new data are fed into operational stream DA Testbed In-Situ Operational Data Assimilation DOD Radar Airborne USGS New physiographic data DOE Operational data feeds baseline developmental stream physiographic Satellite USDA OTHERS Adapted from Integrated Water Science Plan (NWS 2004) 28

29 What kind of tools? (cont.) CHPS/FEWS CHPS-OpenDA Model Adapter OpenDA DATABASE: Forecast Observation MODELS: SNOW-17, SACSMA UNITHG, Lag/K, TS Parameters States VAR EnKF, EnKS MLEF, PF Models DA Tools Community Contribution 29

30 Thank you For more information on the presentation, contact:

31 Vision for Ensemble and Data Assimilation in Hydrologic Forecast Operations Hydrologic Ensemble Prediction System Hydrologic Ensemble Forecast System (HEFS) QPE, QTE, Soil Moisture QPF, QTF Ensemble Pre- Processor Modeling & DA Data Assimilator Streamflow Hydrology & Water Resources Models Ensemble Post- Processor Parametric Uncertainty Processor Hydrologic Ensemble Processor Hydrology & Water Resources Ensemble Product Generator Improved accuracy, Reliable uncertainty estimates, Benefit-cost effectiveness maximized From NWS/OHD Strategic Science Plan (2010)

32 From Snow updating user manual (2003)

33 From Snow updating user manual (2003)

34 NWS Operational DA Strategy MODIS-derived snow cover AMSR-derived SWE 1 MODIS-derived surface temperature MODIS-derived cloud cover AMSR-derived SM 1 Atmospheric forcing Snow models Snowmelt Potential evap. (PE) Precipitation Soil moisture accounting models Runoff Hydrologic routing models Flow In-situ snow water equivalent (SWE) SNODAS SWE In-situ soil moisture (SM) Streamflow or stage Satellite altimetry 1 pending assessment Hydraulic routing models Flow reservoir, etc., models River flow or stage From NWS/OHD Strategic Science Plan (2010)

35 From Sorenson (2002)

36 From Sorenson (2002)

37 From Sorenson (2002)

38 The Red River Flood of 1997 East Grand Forks city engineer Gary Sanders reasoned that predicting a crest at 49.0 ft in 1997 was putting the flood in the same class as the 1979 flood, which crested at 48.8 ft with a peak flow of 82,000 cfs The peak flow in 1997 reached 137,000 cfs "Missing by five ft may not sound like much, but when you talk about flow, the National Weather Service missed by almost 100 percent. (Sanders)

39 DA strategy for operational hydrologic forecasting Decompose into smaller ones such that: The suboptimal solutions from the decomposed problems are close to the optimal solution from the full-blown problem the resulting DA process is forecaster-controllable n m nm n n m m n V V V X X X H H H H H H H H H Z Z Z

40 Multi-Model Soil Moisture Percentile for May 2008 Commonly dry regions: Southeast, Southern Plains, North Dakota, California (a) NARR (b) Noah (c) Mosaic D4 D3 D2 D1 Drought D0 < 20% (a): (b) & (c): D4 D3 D2 D1 40

41 Snow Water Equivalent (a) (b) (c) Snow melt providing water over west mountain region. Pacific Northwest: heavy streamflow might due to snow melt

42 STREAMFLOW FORECAST EXAMPLE 12-DAY LEAD-TIME (APRIL 1, 2006) Analysis (NLDAS) Ensemble Mean Ensemble mean similar to analysis Error ~10% of flow Error=Ens. Mean - analysis Ensemble Spread Ensemble spread comparable to error in ensemble mean From Hou et al

43 Strategy for bridging/coupling land data assimilation and hydrologic assimilation

44 NWS s Integrated Water Science Vision Coupled ocean/atmospheric/ land-surface model (global) Coupled atmospheric/land-surface model (regional) Research Labs WEATHER & CLIMATE FOCUS Common Hydrologic Land-Surface Modeling System (Uncoupled) Land Surface Hydrology & Water Resources Modeling Interface with Estuary Models, Water Quality Models Observing Systems WATER FOCUS From Integrated Water Science Plan (NWS 2004)

45 * stage river V moist soil V evap poten V precip V moist soil X stage river Z moist soil Z evap poten Z precip Z * stage river V moist soil V evap poten V precip V moist soil X stage river Z moist soil Z evap poten Z precip Z * * * * * moisture soil V moisture soil V moist soil X moist soil X moist soil X * stage river V stage river V moist soil V moist soil V evap poten V evap poten V precip V precip V moist soil X stage river Z stage river Z moist soil Z moist soil Z evap poten Z evap poten Z precip Z precip Z Strategy for bridging/coupling land data assimilation and hydrologic assimilation (cont.)

46 Predicting Floods to Droughts In Your Neighborhood River channel Developed Shrubs/Grass Agriculture Wetlands Forecast Point Forecast Basin River Services (600 miles per forecast point) Water Resource Services (6 square mile forecast basins) River Conditions From Carter (2006) Soil Conditions

47 Distributed Modeling System Simulated water balance model and channel routing states over the Arkansas River basin before and after a storm; cumulated rainfall are shown at the top From Koren et al Feb 3, 2010

48 Water Issues: Too Much, Too Little, Poor Quality New strategic plans for NOAA and NWS recognize decision-makers in all water management sectors need: Higher resolution information in space and time Quantification of uncertainty to manage risk Water resource challenges are significant and getting bigger: Population growth and economic development are stressing water supplies and increasing vulnerability A changing climate is impacting water availability and quality Socio-economic risks of floods and droughts are escalating Growing needs for water resource forecasts: Soil moisture for agriculture and forest management Low flow for maintaining water supply Water temperature and salinity forecasts for fisheries management and healthy ecosystems From Carter (2010)