Coupling SUNDIALS with OpenFOAM 6 th OpenFOAM Worshop E. David Hucaby June 14, 2011 EDH-OFW2011
Outline Motivation Mathematics Basic Design Eamples Food Web Steady & Unsteady Pitz-Daily 2
Motivation Why couple SUNDIALS and OpenFOAM? Improve solution convergence over current segregated solution algorithms SUNDIALS vs. alternatives Comprehensive (write own solver Lightweight Trilinos, PetSC User Familiarity (via. Cantera vs. NITSOL [1] Luo, Baum, Lohner, 1998, JCP 46, 644-690 [2] Pawlowsi, Shadid, Simonis, Waler, 2006, SIAM Review 48-2, 700-721 [3] Pernice, Tocci, SIAM J. Sci. Comp. 23-2, 398-418 [4] Evans, Knoll, Pernice, 2006, JCP 219, 404-417 [5] Vui, Saghir, Boerstoel, 2000, IJ Num. Meth. Fluids 33, 1027-1040 [6] Bocelie, Tang, Denison, Wang, Cremer, Chen, 2003, Worshop on Solution Methods for Large Scale Non-Linear Problems [7] Pernice, Zhou, Waler, 1997, University of Utah Center for High Performance Computing 3
SUNDIALS Suite of solvers CVODE ode CVODES add sensitivity analysis KINSOL nonlinear algebraic equations IDA differential algebraic equations (DAE IDAS add sensitivity analysis User supplies the physics function, F F dy = dt F( y = F( y, t 0 ( y, y, t = 0 Features Serial & Parallel, Eact & Ineact Newton Solvers Hindmarsh, Brown, Grant, Lee, Serban, Shumaer, Woodward https://computation.llnl.gov/casc/sundials/main.html 4
5 Nonlinear Iteration At each time step a DAE or ODE system form nonlinear algebraic system Newton Solve Until Approimate Newton (Picard, SIMPLE, Ineact Newton F J s F s J = + = = +1 ( ( ( ( appr F s J = ( ( ( F F s J η < + rel abs F F ε < ε + + ( ( 0 1 Knoll and Keys, 2006, JCP 193, 357 397 Collier, Hindmarsh, Serban, Woodward,KINSOL v.2.6
Ineact Newton J ( s + F( 2 < η F( Solve for s using Jacobian Free Krylov Iteration Method only requires matri-vector product Method Krylov Variable None s F( Objective Function Js MV Product F( + ε s F( ε Right z = P s F( Left s P -1 F( Fied Point s G( = F( - ( P J ( P z 1 F( + ε ( P z F( ε 1 1 1 P J s s J G 1 F( + ε s P F( ε G( + ε s G( ε 6 Knoll and Keys, 2006, JCP 193, 357 397
Design - Interface to SUNDIALS GOAL reduce eplicit dependencies in two pacages SUNDIALS Design - Objected Oriented C Classes: KinMEM, IDAMEM, NVector,.. Member functions, KinSet (KinMEM,. Interface (KINSOL, IDA similar to Cantera s interface to CVODE(S Interface (.H is not dependent on SUNDIALS Dependency is in the implementation (.C Static call bac functions Classes: Kinsol, KinsolUser, Ida, IdaUser physics module which inherits KinsolUser or IdaUser 7
Design Physics Module Physics Modules (OpenFOAM Turn solvers into objects Class contains fields AND equations Similar to turbulence and radiation models No dependence on SUNDIALS SUNDIALS/OpenFOAM Physics Module Inherits from KinsolUser or IDAUser Container for OpenFOAM solver object FoodWeb, Incompressible Flow, 8
Outline Motivation Mathematics Basic Design Eamples Food Web Steady & Unsteady Pitz-Daily 9
Food Web Model Reaction-Diffusion Equation m i c t i = ( d c + f (, c c i : M prey & M predators i i i Model Parameters f = i (, c ci bi g( + a i j g( = 1+ α y + β + y ij c ( sin(4π sin(4π j Steady (KINSOL M = 3 c prey ( = 0 = 1 c (, t = 0 = 3 10 pred d pred = 1 d prey = 0.5 α = 1 β = 0 Unsteady (IDA M = 1 c c prey pred (, t = 0 = 10 + i 5 (, t = 0 = 10 prey ( 16(1 y(1 y d pred = 1 d prey = 0.5 α = 50 β = 1000 5 2 a ij = A = 1 A 0 E E m b prey 0 A E E E = 10 prey = 1 4 = 1 G G A 0 G G 0 A G = 5 10 m b pred prey = 4 0 = 1 10 Brown, 1986, SIAM J. Applied Math, v.46, no. 3 Collier and Serban (KINSOL, Hindmarsh, Serban, Collier (IDA
Food Web Physics Class Calculate increment from the residual OF does time integration (KINSOL void PredatorPrey::calcIncrement(u[].. tmp<fvscalarmatri> CEqn( M_[i]*(fvm::ddt( dc_ - fvc::ddt(dc_ - fvm::laplacian(diff_[i],dc_,"laplacian(diff,conc" + fvm::sp( (rateaa_[i]*freqoverconc*conc_[i],dc_ -Resid ; u P 1 u Time integration outside of OF void PredatorPrey::calcIncrement(Rz[], z[], alpha.. tmp<fvscalarmatri> CEqn( fvm::sp( timecoeff*scalar(alpha*m_[i], dc_ - fvm::laplacian(diff_[i],dc_,"laplacian(diff,conc" + fvm::sp( (rateaa_[i]*freqoverconc*conc_[i],dc_ - Resid ; z ( P ss + Pα t 1 R z 11
Prey Number Density for Kinsol Run problem using tol abs = tol rel =10-5 on a 2020 mesh Kinsol C Kinsol C++ Segregated OpenFOAM Kinsol OpenFOAM Reaction Laplacian + Reaction Diagonal 55 55 55 55 2020 Lower Left Upper Right Prey 1.6253 1.16253 1.16581 1.16558 1.16515 Predator 34875 34875 34973.4 34973.4 34953.4 Prey 1.2637 1.2637 1.25179 1.25179 1.25491 Predator 37908.7 37908.7 37551.8 37551.8 37645.5 12
Kinsol-Open Solutions Food for prey poison for predator Prey/m 3 g ( =1+ y Residuals ( d c + f (, c i i i Pred/m 3 13
Prey Number Density for IDA Run problem using tol abs = tol rel =10-5 on a 2020 mesh Time 0.0 10 10 10 Differences due to: IDA-C Finite Difference space+diag Eact Newton IDA-C++ Finite Difference In-Eact Newton 0.001 10.822 10.827 12.161 0.01 197.34 197.35 264.43 0.1 270.72 270.72 333.65 round-off error FV (cell vs. FD (point Newton methods Preconditioner OpenFOAM Finite Volume 0.4 270.72 270.72 333.65 In-Eact Newton 14
Prey Number Density Green - OpenFOAM (cell Red -IDA-C++ (point Blue IDA-C++ (point->cell 15
Final Prey Number Density IDA - C++ IDA- OpenFOAM 16
Incompressible Flow Simple class for (steady & unsteady incompressible flow based on SIMPLE algorithm Residual & Left - Preconditioner for Ida Residual & Right - Preconditioner for Kinsol Fied-Point Residual Preconditioning for Kinsol Necessarily have a standard SIMPLE solver Advantageous as solver compleity increases <?>SimpleFoam solver Provides interfaces between simple class and SUNDIALS pacages 17
Laminar Pitz Daily Tested on Pitz-Daily with ν = 0.01 m/s 2 Kinsol Right Precond Kinsol Fied Point Picard SIMPLE Time(s 129 s 138 s 198 s 146 s Machine zero Machine zero 1000 iters 1000 iters Mass L2 1.1 e-12 7.6 e-11 2e-7 1.32 Mass abs-ma 8.9 e-12 9.1 e-10 4.77e-6 83.3 Mom. L2 2.1 e-12 9.4e-8 0.016 98.9 Mom. abs-ma 9.0 e-12 1.2e-6 0.080 5314 18
Kinsol-Right Laminar Pitz-Daily +1 in Kinsol-Fied Point Picard-SIMPLE -1 in 0 m/s 5 m/s Velocity Profiles are nearly identical 12 m/s 19
Turbulent Pitz-Daily Solver Approach 1. nonlinear iteration on velocity & pressure followed by 2. N SIMPLE sweeps on U, p & M 1 turbulence corrections 3. M 2 turbulence corrections System is much less robust Selection of input parameters loose tolerance at the beginning tighten toward convergence step-size Turbulence model Reynolds stress seemed to wor better Need Globalization Methods 20
Results Fied Point Right Pre. Not quite converged.. 21
Future Wor Testing and eploration of eisting software interfaces Solution Strategies Improve variable and residual scaling Ida with incompressible flow Include turbulence models in nonlinear iteration Code clean up generalization, templates, better names, comments,.. Etend to other physical systems Additional Capabilities Sensitivity Analysis IDAS, CVodeS,.. Alternative Pacages - NITSOL, Trillinos (NOX,.. 22