CFD Modelling with OpenFOAM

CFD Modelling with OpenFOAM Hrvoje Jasak Faculty of Mechanical Engineering and Naval Architecture, Uni Zagreb, Croatia Wikki Ltd. United Kingdom Oxford University, 13 October 2017 CFD Modelling with OpenFOAM p. 1

Outline Objective Present the activities of the CFD Research group in Zagreb and Wikki Ltd. Review progress on selected topics related to turbomachinery CFD Topics CFD Group at University of Zagreb: Research Activity Harmonic Balance Solver for Turbomachinery flows Block matrix and pressure-based coupled implicit solver Native Overset Mesh in OpenFOAM NUMAP-FOAM Summer School Summary CFD Modelling with OpenFOAM p. 2

Overview of Research Activity Research Group Members, CFD Group at University of Zagreb CFD Research Group attached to the Chair of Turbomachinery 2 professors: Prof. Hrvoje Jasak, Prof. Željko Tuković 1 (+ 1) post doctoral researchers: dr. Vuko Vukčević 6 fully funded PhD students Larger group of Master Thesis and Diploma Thesis students integrated within the group: results of Master Thesis projects directly used in further research Regular external (foreign) visitors working with the group: 3-6 months Communication and Activity Leading developer of OpenFOAM:http://foam-extend.fsb.hr/ Integrated work effort, modern communication and data integration tools Private Media-Wiki: http://spirit.local/mediawiki/ YouTube Channel: 8th Floor CFD@FSB Public web site: http://www.fsb.hr/cfd Approximately 25 (significant) publications per year CFD Modelling with OpenFOAM p. 3

Overview of Research Activity Open Source Software in Research and Engineering Open Source tools are ideal for a research environment: industrial partner gains access not only to physical model equations but also to a working and validated implementation by expert users Deployment of results of research is faster and more reliable Proven track record of model development and delivery First-class students with good technical and CFD background (OpenFOAM) Strong multi-disciplinary group: fluids, structure multi-phase flow modelling, turbulence, optimisation. Premier source of numerics knowledge in collaboration with leading world Universities CFD Modelling with OpenFOAM p. 4

Overview of Research Activity Areas of Research Activity, CFD Group at University of Zagreb CFD simulations in Turbomachinery: basic validation and verification and practical industrial simulations Incompressible flow: pumps and turbines, wind energy devices Compressible flow: compressors and fans with pressure- and density-based CFD solution algorithms Harmonic balance modelling in CFD as a general-purpose tool Gradient-based and gradient-free optimisation: continuous and discrete adjoint CFD in naval hydrodynamics, wave and off-shore structures Fundamental research in numerics: discretisation, solution techniques, HPC performance, inter-equation coupling Complex solid mechanics modelling: large deformation, lubricated contact, conjugate heat transfer Fluid-solid interaction and multi-physics modelling Detailed fuel cell modelling using CFD Acoustics modelling using linearised Euler equations and coupling with CFD CFD Modelling with OpenFOAM p. 5

Free Surface Flow Simulations CFD Modelling with OpenFOAM p. 6

Turbomachinery Simulations Full Support for Turbomachinery CFD Complex rotor-stator interfaces, for all physics models: GGI, mixing plane Incompressible and compressible, MRF and transient Periodic transient mode: harmonic balance solvers CFD Modelling with OpenFOAM p. 7

Complex Physics in Complex Geometry Simulating Complex Physical Processes in Complex Geometry CFD Modelling with OpenFOAM p. 8

Harmonic Balance Solver Expanding the Horizons: Spectral Modelling of Time-Periodic Flows Many CFD problems involve temporally periodic flows Flows induced by periodic boundary condition Flows with periodically moving objects Wave-like phenomena To remove irregular start-up unsteadiness, a number of periods is simulated: expensive, complicated periodic uncertainty issues Harmonic Balance Method Replacing a transient problem with a set of coupled steady-state snapshots by virtue of using periodicity of the time-signal in the solution CFD Modelling with OpenFOAM p. 9

Harmonic Balance Solver Harmonic Balance Method: Work-Flow Variables are developed into Fourier series in time with n-harmonics and substituted into transport equation Transport equation with n sine and n cosine parts + mean part is obtained and written as a set of 2n+1 equations in frequency domain Equations are transformed back to time domain in order to be able to use time-domain boundary conditions and time-domain non-linear flow solver Harmonic Balance in FOAM-Extend Harmonic balance decomposition does not relate to a special physics model: implement HB as a choice of temporal discretisation scheme Geometric aspects of harmonic balance can be tackled without change Currently, HB is implemented in a segregated mode: low memory, and explicit inter-mode coupling terms. Analysis shows issues for HB without a dominant mean flow component: working towards coupled implicit HB solver Author: Gregor Cvijetić, Uni Zagreb CFD Modelling with OpenFOAM p. 10

Harmonic Balance Solver Rapid Simulation of Non-Linear Periodic Flow: Harmonic Balance A variable is presented by a Fourier series, using first n harmonics and the mean: replacing a transient simulation with a set of coupled steady-state problems Periodicity is independently developed in each computational point Non-linear interaction is captured without simplification Example: Harmonic Balance scalar equation set (uq tj ) (γ Q tj ) = 2ω 2n+1 ( 2n i=1 P (i j) Q ti ) P i = n k sin(kωi t) k=1 for i = {1,2n} A transient equation is replaced by a set of n coupled quasi-steady coupled equations of the same type Physical justification: if a functional form of temporal variation is known, the function and its time derivative can be reproduced from a small number of data points by fitting a prescribed harmonic function (spectral time accuracy!) CFD Modelling with OpenFOAM p. 11

Harmonic Balance Solver Harmonic Balance for Navier-Stokes Equations Harmonic balance momentum equation (u tj u tj ) (γ u tj ) = 2ω 2n+1 ( 2n i=1 P (i j) u ti ) Harmonic continuity equation u tj = 0 Harmonic scalar transport (u tj Q tj ) (γ Q tj ) = 2ω 2n+1 ( 2n i=1 P (i j) Q ti ) Physical justification: each t j instance represents a single-time-step solution; time derivative terms couple solution fields to each other CFD Modelling with OpenFOAM p. 12

Harmonic Balance Solver Harmonic Balance Solver: ERCOFTAC Centrifugal Pump Validation of harmonic balance in turbulent incompressible periodic flow HB simulations performed using 1 and 2 harmonics: rotor and stator blade count Results compared against full transient simulation: excellent agreement Integral properties: typical error of 2% Local solution features: pressure on surface in time Mode and nature of flow instability Results are significantly better than expected! Substantial reduction in simulation time: Intel Core i5-3570k, 3.4 GHz computer with 16 GB memory Transient run needs approx. 50 blade passages to become quasi-periodic Transient HB, 1 h HB, 2 h Simulation time 5 hrs/rotation 52 mins 78 mins Iterations 600, dt = 5e-5 s 3000 2400 1 rotation = 0.03 s CFD Modelling with OpenFOAM p. 13

Harmonic Balance Solver Harmonic Balance Solver: ERCOFTAC Centrifugal Pump CFD Modelling with OpenFOAM p. 14

Harmonic Balance Solver Harmonic Balance Solver: ERCOFTAC Centrifugal Pump 0 0 0-200 -200-200 Pressure, Pa -400-600 -800-1000 Transient HB, 1h HB, 2h Pressure, Pa -400-600 -800-1000 Transient HB, 1h HB, 2h Pressure, Pa -400-600 -800-1000 Transient HB, 1h HB, 2h -1200-0.2-0.15-0.1-0.05 0 x-axis -1200-0.15-0.1-0.05 0 0.05 x-axis -1200-0.15-0.1-0.05 0 0.05 x-axis Transient HB, 1h err, % HB, 2h err, % MRF err, % Efficiency 89.72 88.80 1.0 89.76 0.0 89.65 0.07 t = T 3 Head 81.48 81.80 0.4 80.45 1.3 84.12 3.14 Torque 0.0297 0.0302 1.7 0.0294 0.9 0.0308 3.57 Efficiency 89.92 88.78 1.3 89.81 0.1 t = 2T 3 Head 81.48 81.85 0.4 80.6 1.1 Torque 0.0296 0.0302 2.0 0.0295 0.4 Efficiency 89.83 88.85 1.1 89.71 0.1 t = T Head 81.49 81.79 0.4 80.39 1.3 Torque 0.0297 0.0302 1.6 0.0294 1.0 CFD Modelling with OpenFOAM p. 15

Block Matrix in OpenFOAM Background OpenFOAM uses equation mimicking to perform field algebra and discretisation: perfect for simple PDE-s or segregated solution algorithms... but sometimes we use equation segregation inappropriately There exists a family of problems that cannot be solved efficiently without inter-equation coupling; some simulations that work can be performed 10-s or 100-s of times faster than with equivalent segregated algorithms Objective Implement flexible and efficient block-coupled solution infrastructure Re-use all operator-based discretisation schemes, parallelisation and boundary condition tools already available in OpenFOAM Optimise top-level code for efficient execution and ease of assembly Examples Incompressible steady pressure-velocity system (with turbulence) Compressible multi-phase free surface simulations: under-water explosions CFD Modelling with OpenFOAM p. 16

Block Matrix in OpenFOAM Block-Coupled Solution Algorithms For cases of strong coupling between the components of a vector, the components can be solved as a block variable: (u x,u y,u z ) will appear as variables in the same linear system In spite of the fact that the system is much larger, the coupling pattern still exists: components of u in cell P may be coupled to other components in the same point or to vector components in the neighbouring cell With this in mind, we can still keep the sparse addressing defined by the mesh: if a variable is a vector, a tensorial diagonal coefficients couples the vector components in the same cell. A tensorial off-diagonal coefficient couples the components of u P to all components of u N, which covers all possibilities For multi-variable block solution like the compressible Navier-Stokes system above, the same trick is used: the cell variable consists of (ρ,ρu,ρe) and the coupling can be coupled by a 5 5 matrix coefficient Important disadvantages of a block coupled system are Large linear system: several variables are handled together Different kinds of physics can be present, e.g. the transport-dominated momentum equation and elliptic pressure equation. At matrix level, it is impossible to separate them, which makes the system more difficult to solve CFD Modelling with OpenFOAM p. 17

Block Matrix in OpenFOAM Matrix Connectivity and Mesh Structure Irrespective of the level of coupling, the FVM dictates that a cell value will depend only on values in surrounding cells N W P E S We still have freedom to organise the matrix by ordering entries for various components of the solution variable x Global sparseness pattern related to mesh connectivity: easier coefficient assembly CFD Modelling with OpenFOAM p. 18

Block Matrix in OpenFOAM Coupling Coefficient Matrix implemented with block coefficients Consider general linear dependence between two vectors m and n m = Ab Component-wise coupling describes the case where m x depends only on n x, m y on n y and m z on n z 1. Scalar component-wise coupling 2. Vector component-wise coupling 3. Full (block) coupling Explicit methods do not feature here because it is not necessary to express them in terms of matrix coefficients For reference, the linear equation for each cell featuring in the matrix reads A P m P + N A N m N = R CFD Modelling with OpenFOAM p. 19

Coupled p-u Solver Turbulent Steady Incompressible Flows: SIMPLE or Coupled System Equation set contains linear p-u and non-linear U-U coupling u t + (uu) (ν u) = p u = 0 Traditionally, this equation set is solved using the segregated SIMPLE algorithm Low memory peak: solution + single scalar matrix in peak storage p-u coupling is handled explicitly: loss of convergence (under-relaxation) Number of iterations is substantial; not only due to non-linearity Convergence dependent on mesh size: SIMPLE slows down on large meshes Block-implicit p-u coupled solution Coupled solution significantly increases matrix size: 4 blocks instead of 1... but the linear p-u coupling is fully implicit! Iteration sequence only needed to handle the non-linearity in the U-equation Net result: significant convergence improvement (steady or transient) at a cost of increase in memory usage: reasonable performance compromise! CFD Modelling with OpenFOAM p. 20

Coupled p-u Solver SIMPLE-Based Segregated p-u Solver // Momentum equation assembly and solution fvvectormatrix UEqn ( fvm::div(phi, U) + turbulence->divdevreff() ); UEqn.relax(); solve(ueqn == -fvc::grad(p)); // Pressure equation assembly and solution U = UEqn().H()/UEqn.A(); phi = fvc::interpolate(u) & mesh.sf(); fvscalarmatrix peqn ( fvm::laplacian(1/ueqn.a(), p) == fvc::div(phi) ); peqn.solve(); phi -= peqn.flux(); p.relax(); CFD Modelling with OpenFOAM p. 21

Coupled p-u Solver Block-Coupled u p System Matrix Structure u 1 u 2 p 1 p 2 ( a P (uu) a P (pu).. a P (up) a P (pp) ) ( a N (uu) a N (pu) ( a P (uu) a P (pu). a N (up) a N (pp) a P (up) a P (pp) ) )... u 1 p 1 u 2 p 2. = b u1 b p1 b u2 b p2. CFD Modelling with OpenFOAM p. 22

Coupled p-u Solver Coupled Implicit p-u Solver: Source Code fvvectormatrix UEqn ( fvm::div(phi, U) + turbulence->divdevreff() ); fvscalarmatrix peqn ( - fvm::laplacian(ruaf, p) == -fvc::div(fvc::grad(p)) ); BlockLduSystem<vector, vector> pinu(fvm::grad(p)); BlockLduSystem<vector, scalar> UInp(fvm::UDiv(U)); BlockLduMatrix<vector4> A(mesh); blockmatrixtools::insertequation(0, UEqn); blockmatrixtools::insertequation(3, peqn); blockmatrixtools::insertblockcoupling(3, 0, UInp, false); blockmatrixtools::insertblockcoupling(0, 3, pinu, true); UpEqn.solve(); UpEqn.retrieveSolution(0, U.internalField()); UpEqn.retrieveSolution(3, p.internalfield()); CFD Modelling with OpenFOAM p. 23

Performance of the Coupled p-u Solver Performance of the Coupled p-u Solver: Speed and Robustness CFD Modelling with OpenFOAM p. 24

Performance of the Coupled p-u Solver Performance of the Coupled p-u Solver: Speed and Robustness CFD Modelling with OpenFOAM p. 25

Performance of the Coupled p-u Solver Performance of the Coupled p-u Solver: External Aerodynamics 1e+00 1e-01 1e-02 1e-03 U x (simplefoam) U y (simplefoam) U z (simplefoam) p (simplefoam) BiCGStab U x BiCGStab U y BiCGStab U z BiCGStab p SAMG U x SAMG U y SAMG U z SAMG p Residual 1e-04 1e-05 1e-06 1e-07 1e-08 0 500 1000 1500 2000 2500 Iteration CFD Modelling with OpenFOAM p. 26

Performance of the Coupled p-u Solver Performance Improvements and Extensions in the Coupled p-u Solver Improvements in performance for the coupled solver: consistency, numerics Extension to compressible flows, MRF and porous media (implicit!) Major performance jump: block-coupled AMG with additive correction (Hutchinson 1988) Block-coupled k ǫ and k ω SST turbulence models Turbulence equations solved in a single block-coupled system Analysis of source terms to establish favourable cross-equation coupling Implemented in Diploma Thesis assignment: Robert Keser, Uni Zagreb Example: steady (MRF) and transient centrifugal pump CFD Modelling with OpenFOAM p. 27

Performance of the Coupled p-u Solver Performance Improvements and Extensions in the Coupled p-u Solver Initial Residuals 1 0.01 0.0001 Comparison between MRFcoupled and MRFsegregated initial residuals Ux_0 (coupled) Uy_0 (coupled) Uz_0 (coupled) p_0 (coupled) Ux_0 (segregated) Uy_0 (segregated) Uz_0 (segregated) p_0 (segregated) Initial Residuals Comparison between MRFcoupled and MRFsegregated turbulence initial residuals 1 0.1 0.01 0.001 0.0001 k_0 (segregated) omega_0 (segregated) k_0 (coupled) omega_0 (coupled) Head [m] 100 80 60 40 20 0-20 -40-60 -80 0 100 200 300 400 500 Iterations Comparison between MRFcoupled and MRFsegregated OtaBm1 pump head Head (segregated) Head (coupled) -100 0 100 200 300 400 500 Iterations Power [W] 1e+05 90000 80000 70000 60000 50000 40000 30000 20000 10000 Comparison between MRFcoupled and MRFsegregated OtaBm1 pump power 0 0 100 200 300 400 500 Iterations 1e-05 0 100 200 300 400 500 Iterations Power (segregated) Power (coupled) Efficiency [%] 140 120 100 80 60 40 20 Comparison between MRFcoupled and MRFsegregated OtaBm1 pump efficiency Efficiency (segregated) Efficiency (coupled) 0 0 100 200 300 400 500 Iterations CFD Modelling with OpenFOAM p. 28

Overset Mesh Open Source Native Overset Mesh Capability infoam-extend We have been involved in several previous Overset projects... but none reached the maturity or provided flexibility required for a general-purpose CFD platform Project Objective: Complete rewrite without limitations on release Native implementation of Overset Mesh technology in FOAM-Extend without reference to external libraries Operator-based support for Finite Volume (FV) discretisation: implicit and explicit operators (fvc andfvm) Setup viaoverset patch type, with minimal impact on existing code components: full re-use of existing physical models Consistent support for strict mass conservation and second order accurate discretisation in space and time Further requirements Overset mesh functionality must operate on unstructured polyhedral mesh hierarchies Basic layout for massive parallelisation in domain decomposition mode Support for dynamic overset mesh simulations CFD Modelling with OpenFOAM p. 29

Overset Mesh Overset Mesh Capability infoam-extend: An Update Revisited native implementation of Overset Mesh: public release Rewrite of basic Overset functionality to improve performance and parallel scaling Overview of New Overset Capability Explicit and Implicit fringe treatments: chosen per-field, depending on application Rewrite of parallelised Overset assembly New research item: Implicit conservative Overset Additional fringe reduction algorithms Rewrite of parallel communications to improve scaling on HPC machines Publicly available Overset Solver for foam-extend-4.0 CFD Modelling with OpenFOAM p. 30

Overset Mesh: Example Overset Mesh Validation: DTC Propeller Case Transient propeller simulation for a model scale propeller for DTC hull (Duisburg Test Case, a container ship) Case details Model scale propeller for DTC hull, Diameter, D = 0.15 m n = 890 RPM (14.8343 RPS) Advance coefficient (J = v/(n*d) = 0.75) 0.4 Thrust & torque coefficients; K T & 10K Q 0.35 0.3 0.25 0.2 0.15 K T, CFD (overset,transient) K T, EXP K Q, CFD (overset, transient) K Q, EXP 0.1 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 Time; t, s CFD Modelling with OpenFOAM p. 31

Overset Mesh: Parallel Scaling Overset Mesh: Parallel Scaling Test CFD Modelling with OpenFOAM p. 32

Overset Mesh: Parallel Scaling Parallel Efficiency of the Overset Mesh Implementation of Overset interpolation performed similar to GGI Interpolation performed in out-of-core multiplication with parallel comms Parallelised usingmapdistribute tool Parallel scaling test case Scaling test performed on 20M cells submarine mesh Approximately 40K donor/acceptor cells (0.4% of total cell count) Performed 20 iterations with explicit and implicit Overset fringe Parallel speed-up on 64 cores: 41 (implicit) and 46 (explicit) Parallel efficiency on 64 cores: 64% (implicit) and 74% (explicit) 64 32 Explicit fringe Implicit fringe Linear scaling 100 80 Parallel speed-up, S 16 8 4 Parallel efficiency, E, % 60 40 2 20 Explicit fringe Implicit fringe 1 1 2 4 8 16 32 64 Number of cores, n 0 1 2 4 8 16 32 64 Number of cores, n CFD Modelling with OpenFOAM p. 33

NUMAP-FOAM Summer School NUMAP-FOAM Summer School 2017 12th Edition of NUMAP-FOAM Summer School: 21/Aug-1/Sep/2017 https://www.fsb.unizg.hr/numap The idea of the Summer School is to expand the physical modelling knowledge, numerics and programming skills of attendees using OpenFOAM in their research through direct supervision and one-to-one work. This is NOT an introductory OpenFOAM course: significant understanding of the project and software is a pre-requisite for application. The School accepts 10-15 attendees bringing their own projects to the School over a period of 10 working days Work is embedded in the research group with 4 6 tutors providing daily one-to-one attention School is open to young researchers (typically PhD students) but also to industrial users, government labs and professors Strong follow-up collaboration and extensive publication lists Approx 150 attendees to NUMAP-FOAM, from the start in 2008 CFD Modelling with OpenFOAM p. 34

Summary Summary Ongoing research activity at Uni Zagreb on naval hydrodynamics, basic numerics and turbomachinery CFD Actively looking for collaboration partners Current Work Topics Naval hydrodynamics: added resistance in regular and irregular waves, full-scale ship simulation, self-propulsion and manoeuvring, green water and freak wave impact, modelling of irregular sea states Numerics: strongly coupled solution algorithms, Discontinuous Galerkin discretisation, Overset Mesh and Immersed Boundary Turbomachinery: quasi-periodic methods (harmonic balance), LES and instability modelling, implicit pressure- and density-based solvers, turbulence and transition Solid mechanics and FSI: coupled non-linear FSI problems Optimisation: Gradient-free and adjoint-based methods; uncertainty propagation and robust design CFD Modelling with OpenFOAM p. 35

About Me Hrvoje Jasak First degree: mechanical engineering, University of Zagreb, Croatia 1992 PhD, Imperial College London 1993-1996 Senior development engineer, CD-adapco (Siemens), 1996-2000 Technical director, Nabla Ltd. 2000-2004 Consultant on CFD software, numerics and modelling, ANSYS Fluent 2000-2008 Current Work Director, Wikki Ltd: UK-based consultancy company 2004- Professor, University of Zagreb, Croatia 2007- Mercator Fellow, TU Darmstadt, 2016- Various software development and commercial support projects based on OpenFOAM with consultants and large industrial partners Coordinating open source OpenFOAM development to allow contributions from the public domain developers OpenFOAM workshops, lectures and seminars, visiting professorships (TU Delft, Chalmers University and others) CFD Modelling with OpenFOAM p. 36