Multiphysics Modeling Simulating the Terrestrial Water Cycle Christopher Duffy, Mukesh Kumar, Gopal Bhatt, Shuangcai Li George Holmes, Wenfang Li, Pat Reed Penn State University
Why HPC?
Issues & Opportunities Multiphysics analysis is a unifying modeling strategy that allow scientists and engineers to explore fully-coupled effects of complex physical phenomena New sensors, communication networks, and characterization tools have made it possible to examine spatial and temporal phenomena over an unprecedented range of scales Environmental observatories are poised to take advantage of sensors and simulation tools to advance predictive understanding and environmental forecasting One measure of a new theory or formulation is its ability to predict new phenomena. 3 examples
Multiphysics analysis is a unifying modeling strategy that allow scientists and engineers to explore fully-coupled effects of complex physical phenomena
Architecture for Multiphysics Modeling Sequential Coupling Approach one process or field is partially solved and then passed to the next physics field to form a partial solution. The iteration process continues until a final solution is achieved. The integration interface provides the services for subsystem interaction, data handling, mesh generation, parallel I/O, visualization, etc. Mike Heath UIUC, 2008
Architecture for Multiphysics Modeling Direct Coupling assembles all the physics equations in one matrix and solves all equations together. In this example an open-source GIS and Geodatabase is used for data management, mesh generation, space-time data analysis, and interface to the system solver and visualization. User Interface Data Management Vector Processing Raster Processing Data Model Loader Parameterization PIHM Kernel Physics Numerical Solver Data Analysis Spatial Temporal Spatio-Temporal Uncertainty Domain Decomposition Static: Conformed, constrained Delaunay & nested triangulation Dynamic: Adaptive triangulations Kumar, Bhatt, Duffy, 2008 Geodatabase (Schema, Data, Relationships) Field, Feature Objects, Non-Spatial Data Data Access Tool (Grid-Shp/ Dbf) Read, (Grid-Shp/ Dbf) Write
Penn State Integrated Hydrologic Model: PIHM Qu and Duffy 2007
Semi-Discrete Approach
Open Source Programming Tools QGIS an open source and free "Programmable Geographic Information System" a mapping tool, a GIS modeling system, and a GIS application programming interface (API) all in one a platform independent Open Source Geographic Information System that runs on Linux, Unix, Mac OSX, and Windows libraries for raster and vector geospatial formats Object relational database is handled using PostgreSQL GRASS compatible Qt/C++ - tools for programming the interface and visualization
Feature/ Time Series Soil Geology Land Cover Property Porosity; Sand, Silt, Clay Fractions; Bulk Density Bed Rock Depth; Horizontal and Vertical Hydraulic Conductivity LAI Manning s Roughtness Topology: From Node To Node, Neighboring Elements; A-Priori Data Sources Source CONUS, SSURGO and STATSGO http://www.soilinfo.psu.edu/index.cgi?soil_data&conus http://datagateway.nrcs.usda.gov/nextpage.asp http://www.ncgc.nrcs.usda.gov/products/datasets/statsgo/ http://www.dcnr.state.pa.us/topogeo/, http://www.lias.psu.edu/emsl/guides/x.html http://glcf.umiacs.umd.edu/data/landcover/data.shtml, http://ldas.gsfc.nasa.gov/ldas8th/mapped.veg/ldasmapveg.shtml; Hernandez et. al., 2000 Derived using PIHMgis (Bhatt et. al., 2008) River Manning s Roughness; Coefficient of Discharge Dingman (2002) ModHms Manual (Panday and Huyakorn, 2004) Forcing DEM Streamflow Groundwater Shape and Dimensions; Precipitation, Temperature Derived from regression using depth, width and discharge data from http://nwis.waterdata.usgs.gov/usa/nwis/measurements Gauge data obtained from MARFC. 6 hourly precipitation point data is spatially gridded such that it conforms to the monthly precipitation distribution map obtained from parameter-elevation regressions on independent slopes model (PRISM) (Daly et. al., 1994, 1997) http://seamless.usgs.gov/ http://nwis.waterdata.usgs.gov/nwis/sw http://nwis.waterdata.usgs.gov/nwis/gw
Conceptual Model + GeoDataBase = A Priori Data Conceptual Model Groundwater flow in the Allegheny Plateau Section. Example of the GIS for the surface geology coverage.
PIHM GIS Framework User Interface Data Management Vector Processing Raster Processing Data Model Loader Parameterization PIHM Kernel Definition Numerical Solver Data Analysis Spatial Temporal Spatio-Temporal Uncertainty Domain Decomposition Static: Conformed, constrained Delaunay & nested triangulation Dynamic: Adaptive triangulations Geodatabase (Schema, Data, Relationships) Field, Feature Objects, Non-Spatial Data Data Access Tool (Grid-Shp/ Dbf) Read, (Grid-Shp/ Dbf) Write
PIHM GIS Interface
Irregular Mesh & Stream Network Elevations Soil Classes Land Cover
New sensors, communication networks, and characterization tools have made it possible to examine spatial and temporal phenomena over an unprecedented range of scales
WSN Architecture
WSN: Wireless Sensor Network IRIS 2.4 GHz radio Stargate Netbridge Gateway 7 Web Services ASA MDA320 Sensor Board DAQ eko Soil moisture temp
Some Sensor Properties/Limitations Heterogeneous sensors: observe the atmosphere, vegetation, surface and subsurface coherently Development of data acquisition for heterogeneous sensors Low power requirement limits availability of sensors Tradeoff between cost, response time, sampling rate Power/charging issues in forest canopies
Sensors Node Design
Shale Hills CZO Prototype Node Web Services
Environmental observatories are poised to take advantage of sensors and simulation to advance predictive understanding and environmental forecasting
One measure of a new theory or formulation is its ability to predict new phenomena. 3 examples
Integrated Modeling in a Coastal Watershed Freshwater Discharge: Groundwater or Surface Water? Smithsonian Environmental Research Center
Domain used in decomposition
SSURGO Soil Classification
30m NLCD Data
Stream Gauge Locations
Freshwater Discharge to the Chesapeake
Distributed Water Budgets & The Virtual Watershed Mukesh Kumar, PhD 09
Little Juniata Watershed 875 sq km 0 5 10 mi Mukesh Kumar PhD 08
Distributed Stream Runoff in Upland 5000000 Observed Modeled Modeled_Cal Precep 0 4500000 0.03 4000000 0.06 Streamflow (m^3/d). 3500000 3000000 2500000 2000000 1500000 0.09 0.12 0.15 0.18 0.21 Precep (m/d) 1000000 0.24 500000 0.27 0 0 30 60 90 120 150 180 210 240 270 300 330 360 Days 0.3
0 Spatial Mean ET from Distributed Simulation b) Daily interception: et0 = 0.000584 m/d c) Daily transpiration et1 = 0.000664 m/d d) Daily ground ET: et2 = 0.000645 m/d e) Monthly totals f) Monthly percentages et0 (m/d) 0.014 0.012 0.01 0.008 0.006 0.004 0.002 0 (b) 0 30 60 90 120 150 180 210 240 270 300 330 360 Day 0.009 0.008 (c) 0.007 0.006 (d) 0.007 0.005 0.006 et1 (m/d) 0.005 0.004 et2 (m/d) 0.004 0.003 0.003 0.002 0.002 0.001 0.001 0 0 0 30 60 90 120 150 180 210 240 270 300 330 360 0 30 60 90 120 150 180 210 240 270 300 330 360 Day Day 0.2 0.16 (e) et0 et1 et2 precep 100% 90% 80% (f) et0 et1 et2 Monthly ET (m). 0.12 0.08 0.04 70% 60% 50% 40% 30% 20% 10% Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct 0% Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct
0 Spatial Mean Energy & ET For Each Land Cover Type Ev_NL DE_BL Mixed WL W_GL Crop Urban Precep LAI 0.0003 0.00025 (a ) 0.2 0.16 (b ) 6 5 Daily Avg. IS (m). 0.0002 0.00015 0.0001 Monthly Precep. (m). 0.12 0.08 4 LAI 3 2 0.00005 0.04 1 Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct 0 Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct 0 Ev_NL DE_BL Mixed WL W_GL Crop Urban Monthly Rn (J/m^2/d). 30000000 25000000 20000000 15000000 10000000 5000000 0.002 (c) 0.0018 (d et0 (m/d) 0.0016 0.0014 0.0012 0.001 0.0008 0.0006 0.0004 ) 0.0002 0 Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct 0 Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Ev_NL DE_BL Mixed WL W_GL Crop Urban Ev_NL DE_BL Mixed WL W_GL Crop Urban 0.006 0.005 0.004 (e ) 0.003 0.0025 0.002 (f) et1 (m/d) 0.003 et2 (m/d) 0.0015 0.002 0.001 0.001 0.0005 0 Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct 0 Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct
Predicting Surface-Groundwater Interaction at Meanders Losing-Gaining Reach Overland Flow Base Flow
0.00 Precep (m/d). 0.03 0.06 0.09 0.12 0.15 700 Left Right 500 Base Flow (m^3/d). 300 100 Nov -100 Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct -300-500 -700 Net BF Losing Net BF Gaining
Groundwater-Surface Water Interaction at Stream Junctions Losing Stream Overland Flow Base Flow
Predicting Gaining-Losing Stream Reaches
Predicted Distributed Stream Runoff at Outlet 18000000 Observed Modeled Modeled_Cal Precep 0 16200000 0.03 14400000 0.06 Streamflow (m^3/d). 12600000 10800000 9000000 7200000 5400000 0.09 0.12 0.15 0.18 0.21 Precep (m/d) 3600000 0.24 1800000 0.27 0 0 30 60 90 120 150 180 210 240 270 300 330 360 0.3 Days
Predicted Distribution of Net Annual Recharge
Predicted ET-Groundwater Relation Model transect 0.0006 S4+S5 GW Depth 100 90 0.0005 80 0.0004 70 S4+S5 (m/d). 0.0003 0.0002 60 50 40 30 GW Depth (m). 0.0001 20 10 0.0000 0.0 2.7 5.3 7.9 9.0 9.8 10.8 12.7 15.1 0 Transect Distance (km).
A 2-D Finite Volume Model for Hydrodynamic and Sediment Transport in Rivers and Estuaries Shuangcai Li Department of Civil and Environmental Engineering The Pennsylvania State University PhD Dec 08
Hydrodynamic model: Shallow water equations! h! t! +! x ( uh)!( vh) +! y = 0 2 2 ( uh) "( u h + gh / ) "( uvh) " + 2 " t " x + " y 2 2 ( vh) "( uvh) "( v h + gh / ) " + + 2 " t " x " y Sediment transport model (# h) "(# uh) "( vh) " + + # " t " x " y " z D! E = " t 1! p = E! D Solute transport model (! h) $ (! uh) $ (! vh) $ $ t + $ x + $ y = # " =! ghs ox! ghs fx =! ghs oy! ghs fy ( Kh# "!)
Malpasset Dam Break The Malpasset dam was located in Reyran river valley, about 12 km upstream of Frejus. It collapsed in Dec. 1959 due to the exceptionally heavy rain. There were 433 casualties. The simulation area is about 520 km 2 The water level in the reservoir was 100 m. In the downstream of the dam, the bottom is set to dry since the discharge is unknown and small compared to the flood wave The Manning coefficient is 0.033. The simulation is run to 2500 s on 26000 elements.
Motivation Challenges: Physical System Rapidly changing multi-scale flow (damaging) under extreme events; Interaction between hydrodynamics and transport; Computational Complicated topography and geometry in real flow field; Wetting/drying processes, and very shallow flow; Stability, accuracy, and computational cost.
Topography of the Reyran valley and the coastal zone
Domain decomposition
Results Flood inundation map t=0 s t=1000 s t=1500 s t=2000 s
Malpasset Dam Break Model & Observations Water surface level (m) 100 90 80 70 60 50 40 30 20 10 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Police survey points Field Model Y oon & Kang (2004) V aliani et al. (2002)
Results Flood inundation map t=2500 s t=3000 s
Computation & Observatory Science Multiphysics analysis is a unifying modeling strategy for exploring fully-coupled effects of complex physical phenomena New sensors, communication networks, and characterization tools have made it possible to examine spatial and temporal phenomena over an unprecedented range of scales Environmental observatories are poised to take advantage of sensors and simulation to advance predictive understanding and environmental forecasting Multiphysics formulations will produce new predictions->open Source Code Sharing