Verification of Calculations: an Overview of the Lisbon Workshop

Transcription

1 Verification of Calculations: an Overview of the Lisbon Workshop L. Eça IST, Instituto Superior Técnico, Lisbon, Portugal M. Hoekstra Maritime Research Institute Netherlands, Wageningen, The Netherlands. and Patrick J. Roache Socorro, New Mexico, U.S.A. This paper presents an overview of a Workshop dedicated to the Verification of Calculations held in Lisbon in October The Workshop aimed at evaluation of Computational Fluid Dynamics Uncertainty Estimators by comparing results of six groups on two twodimensional incompressible turbulent flows involving separated turbulent flows. Various numerical methods were represented, and calculations were done on smooth grids as well as deliberately poor grids in order to burden the Uncertainty Estimators. Even though local grid convergence was often non-monotone, the overall consensus evaluation was generally favorable for the Grid Convergence Index and the Least Squares version of it, as well as a monogrid Error Transport Equation method. Some residual doubts remain on the inter-group comparisons because of conceptual modelling differences and the possibility of undetected coding errors. These could be eliminated by the use of a realistic exact analytical solution produced by the Method of Manufactured Solutions, and the consensus favored this approach for a second Workshop on this theme. I. Introduction A Workshop on CFD Uncertainty Analysis was held on October 21-22, 2004 at the Instituto Superior Técnico in Lisbon, Portugal. Although its title suggests perhaps more, the Workshop focused on Verification of Calculations. The main objective was to assess the present capabilities in making reliable uncertainty estimates for numerical solutions of steady, incompressible, turbulent flows. To this end two test cases featuring two-dimensional separated turbulent flow were selected from the ERCOFTAC Classic Database. 1 The organisers constructed single-block structured grids of various types, and for each type a family of geometrically similar grids. The purpose was twofold: to let all participants use the same grids and to not exclude the participation with flow solvers unable to handle multi-block grids. The supplied boundary conditions included the inlet profiles for the velocity as well as several turbulence quantities for the following eddy-viscosity models: one-equation models of Spalart & Allmaras 2 and Menter 3 and two-equation k ɛ and k ω models, described for example by Wilcox. 4 Although experimental data are available for these two test cases, these were not included in the information supplied to the participants to emphasize that the topic of the Workshop was Verification of Calculations, not Validation. Assistant Professor, IST, Department of Mechanical Engineering. Senior Research Scientist, Maritime Research Institute Netherlands Consultant 1 of 12

2 The uncertainty estimates by definition target 95% confidence, i.e. the error bands produced should include the exact mathematical solutions in 95% of the cases to which they are applied. In the absence of an exact solution for the selected test cases, the performance of the uncertainty estimators was evaluated for internal consistency, i.e. the uncertainty estimates from various grid sequences and numerical methods should roughly overlap. This of course assumes that the same mathematical problem (including turbulence model) is being calculated by the different numerical methods. Another aspect that was requested from the participants was to check the ability to reduce the iterative error to negligible levels. Groups presenting calculations were from Italy, Germany, USA, France, Bulgaria, Japan, and Portugal/Netherlands. The numerical methods included finite-volume and finite-difference discretization techniques applied to formulations based on artificial compressibility, the strong conservation form or the Contravariant weak formulation of the continuity and momentum equations. The calculations presented at the Workshop were performed with both in-house and commercial codes. Most of the results presented used the one-equation eddy-viscosity model of Spalart & Allmaras. 2 There was also one set of results for the Chang 5 et al. version of the k ɛ model and one group that also applied the one-equation model proposed by Menter. 3 Three uncertainty estimation methods were used by the participants: the 3-grid Grid Convergence Index 6 (GCI) with Factor of Safety Fs = 1.25, the Least-Squares version 7 of the GCI, and an Error Transport Equation Method 8 (with implicit Fs = 1, i.e. an error estimator). The selected flow quantities include the friction and pressure resistance coefficients at the walls, the separation and re-attachment points and the local flow solution at three pre-defined locations. For all these flow quantities the participants were requested to present their numerical solution and the respective error bar. All these data were compiled in tables and plots, facilitating the comparison of the uncertainty intervals obtained by each prediction. The number of submitted solutions in different grids and/or different codes for the same flow with the same turbulence model ranges from 2 (k ɛ model for the Hill flow) to 16 (Spalart & Allmaras model for the backward facing step). The present paper is organized in the following way: section II presents the grid sets proposed and the specified inlet profiles. An overview of the results presented at the Workshop is given in section III. The final remarks and recommendations of this Workshop are summarized in section IV. II. Test cases and grid sets The computational domains of the two selected test cases, both being two-dimensional channel flows, are similar: the inlet and outlet boundaries are vertical lines with x = constant, the top boundary is a flat wall with y = constant and the bottom boundary is also a wall, but with a more difficult shape. The geometry of the two flows is illustrated in Figure 1, which is taken from the Ercoftac Database. 1 In both cases, the computational domain is bounded by two solid walls and the inlet and outlet boundaries, which means that there are only two boundaries with exact (i.e. unambiguous, completely defined) boundary conditions. In the hill case, the Reynolds number based on the hill height, h = 28mm, and on the mean centreline velocity at the inlet, U o = 2.147m/s, is R n = The inlet boundary is an x = constant line located at x = 300mm, i.e. x 10.7h and the outlet boundary is an x = constant line at x = 800mm (x 28.6h). The maximum distance between the two walls is 170mm, i.e. approximately 6h. For the flow over a backward facing step, the angle of the top wall is 0 degrees. The velocity of the uniform incoming flow, U ref, is 44.2 m/s and the step height, h, is 1.27 cm. The inlet is an x = constant section located 4 step heights upstream of the step and the outlet is an x = constant section 40 step heights downstream of the step. The Reynolds number based on U ref and h is In both cases, all the grid sets consist of single-block structured grids. One family of grid lines connects the inlet and outlet boundaries, whereas the other family of grid lines runs between the top and bottom walls. This choice does not lead to an optimal grid (the use of bad grids is even intended in some cases herein), 2 of 12

3 Figure 1. Geometry of the flows over a 2-D Hill and a backward facing step. but the main objective of the present grids is to allow their use with the maximum number of flow solvers possible, and that geometrical similarity of the grids can be guaranteed. To achieve this result a basis grid is generated for each test case. Other members of the grid set are then obtained with a 2-D cubic spline interpolation performed on the basis grid. The desired grid line near-wall distribution is obtained with 1-D coordinate transformations tuned at the inlet and bottom boundaries, using the stretching functions proposed by Vinokur. 9 A. Flow over a hill Two sets of 11 grids have been generated for the flow over a hill. The coarsest grid of each set includes grid nodes and the finest , covering an overall grid refinement ratio of 4. Set A is composed of nearly-orthogonal grids with two families of curvilinear grid lines. The maximum deviations from orthogonality are smaller than 2 o. On the other hand, set B has vertical grid lines (x = constant), which are located at the bottom wall coordinates of the equivalent grids of set A. As a consequence, the typical values of mean and maximum deviations from orthogonality in the interior are about 5 o and 41 o, respectively. The grids are orthogonal at the inlet, outlet and top boundaries, but there is a significant deviation from orthogonality at the bottom wall, with a maximum deviation from orthogonality of 40.5 o in the hill region. Figure 2 presents the coarsest grids of the two sets in the hill region. Set A Set B Figure grid of the two grid sets for the flow over a 2-D Hill. The near-wall grid line distances were selected to allow a proper application of the no-slip condition. The 3 of 12

4 typical values of the maximum y + at the first grid node away from the walls, (y + 2 ) max, are given in table 1. Even for the coarsest grid, the largest value is Grid Typical Number of grid nodes in each direction Set (y 2 + ) max A Top A Bottom B Top B Bottom Table 1. Maximum value of y + at the first grid node away from the walls in grid sets of the flow over a hill. B. Flow over a backward facing step The choice of a single block, structured grid for the present computational domain makes the grid generation a challenging task. The two corners of the bottom wall are not easy to handle in such a grid. It is obvious that it is possible to generate much simpler grids using different topologies than the present choice. A simple example is a multiblock Cartesian grid. However, one of the goals of the Workshop was to test the reliability of the uncertainty estimates in difficult situations and the single block structured grid is a really demanding test case for any flow solver. In this case, 3 sets of 7 geometrically similar grids have been generated ranging from to grid nodes. In the 3 sets, the boundary node distributions at the inlet boundary are equivalent and the values are specified using the stretching functions proposed by Vinokur. 9 In grid set A, there are always grid nodes coincident with the two corners of the step. The bottom wall has an equidistant node distribution along the vertical wall of the step. In each grid, one fifth of the grid nodes are located between the inlet and the top corner of the step. Another fifth is used along the vertical line of the step and the remaining three fifths are placed between the bottom corner and the outlet of the computational domain. The typical mean and maximum deviations from orthogonality are around 15 o and 43 o degrees. However, at the two boundaries where the no-slip condition applies the mean deviation from orthogonality is only 0.5 o. Grid set B is obtained from the grids of set A by connecting the grid nodes of the two horizontal boundaries by straight lines. The two-sided stretching function applied at the inlet is also adopted for all the vertical grid lines of each grid to obtain the desired interior grid line spacing. The resulting grids have the largest deviation from orthogonality of all the grid sets, with mean values around 26 o and the maximum values of 64 o. It is also important to remark that the grids are clearly non-orthogonal at the boundaries. Grid Typical N ξ or N η Set (y 2 + ) max A, C Top A, C Bottom B Top B Bottom Table 2. Maximum value of y + at the first grid node away from the walls in the grids for the flow over a backward facing step. 4 of 12

5 In practical applications, it may be impossible to guarantee the existence of a grid node at every corner of a complex configuration. Therefore, the purpose of the third grid set, C, was to investigate the influence of not having grid nodes at the two corners of the step. The grids are generated with a similar process to the one described above for set A, but the two corner nodes are always located at half-distance of two grid nodes. Reasonably, the deviations from orthogonality are similar to the ones obtained for the grids of set A. Set A Set B Set C Figure grid of set A, B and C for the flow over a backward facing step. 5 of 12

6 Figure 3 presents three views of the coarsest grids of each set in the step region. The typical values of (y + 2 ) max are given in table 2. For the coarsest grid, the largest value is Again, we emphasize that the choice of grids with obviously poor quality in some regions was deliberate in order to burden the Uncertainty Estimators being evaluated in the Workshop. C. Inlet Profiles In both test cases there are no exact (physical) boundary conditions available at the inlet boundary, but experimental data have been reported, The latter have therefore been approximated as well as possible. All the details of the specification of the inlet profiles are given in the Workshop Proceedings. 12 Here we will only mention the most important aspects. The Cartesian velocity component in the x direction, U 1, was defined with the help of analytical profiles using a multi-layer approach. The inlet boundary conditions for the turbulence quantities were chosen so as to lead always to the same eddy-viscosity profile, independently of the turbulence model chosen. Approximate inlet profiles were given for k and ɛ and the profile of the eddy-viscosity, ν t, was obtained using Chien s k ɛ model. 13 The remaining turbulence model quantities used by either the one-equation or the k ω models were derived from the specified k, ɛ and ν t profiles. III. Results Several relevant flow variables were selected for the evaluation of the uncertainty estimation procedures including integral and local quantities. In both test cases, the integral quantities selected are the friction resistance coefficients of the top and bottom walls and the pressure resistance of the bottom wall. The uncertainty in the local flow quantities (the two Cartesian velocity components, the pressure coefficient and the eddy-viscosity) was demanded at three locations, viz. one close to the separation point, one in the middle of the separation region and one close to re-attachment. The position of the re-attachment point of both flows and of the separation point for the hill flow were also to be subjected to analysis. Three uncertainty estimators were applied by the participants. The Grid Convergence Index (GCI) proposed by Roache; 6 the procedure proposed by Eça & Hoekstra 7 (LSGCI) based on a least squares version of the GCI; an Error Transport Equation (ETE) method proposed by Hay. 8 The GCI method in its original form requires three grids to evaluate the uncertainty estimate, whereas the least squares root version requires at least four. On the other hand, the ETE method is a single-grid error estimator. Most of the calculations presented at the Workshop were performed with very tight iterative convergence criteria, and grid convergence was evaluated in several error norms. However, it is pointed out by several participants that the convergence rate of their methods tends to decrease with the increase of the grid density. The goal of overlapping uncertainty estimates was usually but not always met by the Workshop contributions. However, there are several details of the calculations that one has to be careful of in these types of comparisons. Unfortunately, the mathematical definition of the problems in fact varied. One source was differences in boundary conditions: not merely the numerical implementation (which involve only ordered errors and therefore are estimable in a grid convergence test) but in the continuum definitions of smaller terms, particularly at inflow and outflow. As an example of the consequences of the details of the boundary conditions, Figure 4 presents the friction and pressure resistance coefficients of the bottom wall of the flow over the hill obtained with the same code using different types of boundary conditions for the vertical velocity component, U 2, at the inlet. In one case U 2 is set equal to zero whereas the other set of calculations is performed with the axial derivative of U 2 equal to zero. The calculations were performed in grid set B with the Spalart & Allmaras turbulence model. The error bars plotted in Figure 4 were obtained with the least squares version of the GCI. The results plotted in Figure 4 show that the change in the U 2 boundary condition is sufficient to cause 6 of 12

7 2.50x10-02 U 2 =0 at the inlet du 2 /dx=0 at the inlet U= 0.84%, h i U= 0.83%, h i p=1.8, h i p=1.8, h i 2.40x x U 2 =0 at the inlet du 2 /dx=0 at the inlet U= 0.08%, h i U= 0.83%, h i p=4.9, h i p=1.8, h i U 2 =0 at the inlet U= 0.11%, h i p=4.6, h i 2.40x (C F ) b (C F P ) b 2.30x x x h i 2.20x h i h/h i 1 Figure 4. Friction resistance, (C F ) b, and pressure resistance, (C P ) b, at the bottom wall for the flow around the Hill. Calculations performed with different inlet boundary conditions for the vertical velocity component, U 2. a misleading conclusion on the performance of the uncertainty estimators when one compares the different numerical solutions and their respective error bars. In fact, the error bars of these two calculations should not intersect at high resolutions, because the mathematical problem solved is not identical. The choices in the implementation of the turbulence models are another source of possible differences in the mathematical model of each flow solver. Last but not the least, coding errors are also a source of concern, i.e. Code Verification. Code Uncertainty Grid Set (C F ) b U (C F ) b U (C F ) b + U 1 A LSGCI 401x401 A A LSGCI 401x401 B B LSGCI 401x401 A B LSGCI 401x401 B C ETE 401x401 A C ETE 401x401 B D GCI 361x361 A E LSGCI 401x401 A E LSGCI 201x201 A E LSGCI 401x401 B E LSGCI 201x201 B F LSGCI 281x281 A F LSGCI 281x281 B Table 3. hill. Predicted values and error bars for the friction resistance at the bottom wall of the flow around the Although it was recognized by the Workshop participants that there are limitations in the comparison of all the results submitted, the clear consensus was that the results were gratifyingly consistent and provided a favorable evaluation for all three approaches to Uncertainty Estimation. As an example of these comparisons, we have selected an integral and a local flow quantity from each test case. Figure 5 presents the two flow quantities of the flow around the hill: the friction resistance at the bottom wall and the eddy-viscosity, ν t, 7 of 12

8 close to the re-attachment point, x = 5.357h, y = 0.107h. Tables 3 and 4 present the calculated flow values and respective error bars of these two flow quantities and the uncertainty estimation technique used. All these calculations were performed with the Spalart & Allmaras one-equation model. In table 4, there are two estimated uncertainties with a * in front. This indicates that the apparent condition observed is divergence. However, one of the estimated uncertainties is even the smallest of all. This is a consequence of very small differences between the solutions in the different grids, which cause a situation where a nearly grid independent solution produces apparent divergence in the least squares fit to the data. This shows that the determination of the apparent (or observed) order of accuracy is sensitive to small perturbations in the data (e.g. round-off or incomplete iteration errors) for a nearly grid-converged solution, as is well known, but of course apparent order looses its meaning in such a situation. Code Uncertainty Grid Set ν t U ν t U ν t + U 1 A LSGCI 401x401 A A LSGCI 401x401 B B LSGCI 401x401 A B LSGCI 401x401 B C ETE 401x401 A C ETE 401x401 B E LSGCI 401x401 A E LSGCI 201x201 A * E LSGCI 401x401 B E LSGCI 201x201 B F LSGCI 281x281 A * F LSGCI 281x281 B Table 4. Predicted values and error bars for the eddy-viscosity at x = 5.357h, y = 0.107h for the flow around the hill (C F ) b ν t /(U ref h) Calculations submitted Calculations submitted Figure 5. Friction resistance at the bottom wall and eddy-viscosity at x = 5.357h, y = 0.107h for the flow around the Hill. Figure 6 presents the friction resistance at the bottom wall and the re-attachment point for the calculations of the flow around the backward facing step. Tables 5 and 6 present the results and respective error bars of these two flow quantities and the uncertainty estimation technique adopted. All these calculations were performed with the Spalart & Allmaras one-equation model. 8 of 12

9 Code Uncertainty Grid Set (C F ) b U (C F ) b U (C F ) b + U 1 A LSGCI 241x241 A A LSGCI 241x241 B A LSGCI 241x241 C B LSGCI 241x241 B B LSGCI 241x241 C C ETE 241x241 A C ETE 241x241 B C ETE 241x241 C D LSGCI 241x241 B D LSGCI 241x241 B E LSGCI 241x241 A E LSGCI 241x241 B E LSGCI 241x241 C F LSGCI 241x241 A F LSGCI 241x241 B F LSGCI 241x241 C Table 5. Predicted values and its error bars for the friction resistance at the bottom wall of the flow around a backward facing step. It is interesting to remark that the ETE method estimates clearly the largest values of uncertainty for the friction resistance at the bottom, whereas for the re-attachment point the estimated uncertainties are similar to the ones of the LSGCI. As in the results of table 4, there are several uncertainties estimated for cases where apparent divergence is obtained. However, in contrast to Table 4, in this case the values of U are among the largest estimated uncertainties. As expected, the results of grid set C displayed degraded performance but the Uncertainty Estimators were still effective. See the complete Proceedings 12 for further details. The results plotted in Figure 5 and 6 give a good illustration of the comparison between different calcula (C F ) b x ret /h Calculations submitted Calculations submitted Figure 6. Friction resistance at the bottom wall and re-attachment point for the flow around a backward facing step. 9 of 12

10 Code Uncertainty Grid Set (C F ) b U (C F ) b U (C F ) b + U 1 A LSGCI 241x241 A A LSGCI 241x241 B A LSGCI 241x241 C B LSGCI 241x241 B B LSGCI 241x241 C C ETE 241x241 A C ETE 241x241 B C ETE 241x241 C D LSGCI 241x241 B D LSGCI 241x241 B E LSGCI 241x241 A * E LSGCI 241x241 B E LSGCI 241x241 C F LSGCI 241x241 A * F LSGCI 241x241 B * F LSGCI 241x241 C * Table 6. step. Predicted values and error bars for the re-attachment point of the flow around a backward facing tions. It should be mentioned that the internal consistency exercise performed for the same code in different grid sets by each group would lead to overlapping error bars and an unqualified success of the Uncertainty Estimators. However, the comparison of all the codes did not produce an equivalent result. As discussed above, this does not necessarily mean that the Uncertainty Estimators failed. The main problems and tendencies that were identified in the analysis of the results were: Uncertainty estimation for complex turbulent flows is not an easy task. Most of the data presented showed the existence of scatter in the convergence with the grid refinement, even in sets of geometrically similar grids. Post-processing operations, like interpolation and integration, are known to be one of the sources of scatter 14 and this was once more demonstrated. There were several cases which exhibited apparent super-convergence. Most of theses cases are related to data that has very small numerical differences between the finest grids solutions (the pressure resistance plotted in Figure 4 is a good example). Two possibilities may exist in these cases: the solution is nearly grid independent or the convergence is not monotone. The latter case is really more difficult to handle. Nevertheless, the use of observed orders of accuracy larger than the theoretical value in the GCI formula is not recommended because this would produce overly optimistic Uncertainty Estimates. Many of the grid convergence tests displayed non-monotone convergence in local values. It was noted and agreed by the Workshop participants that non-monotone convergence does not indicate that the CFD solutions are necessarily inaccurate, even though it may make uncertainty estimation more problematical. In any case, non-monotone convergence is a property of the CFD solutions on the grid sequence, not of the uncertainty estimation method, and so the recommendation is to not shoot the messenger. 10 of 12

11 As expected, functional values like base pressure or integrated friction behaved better than did local values, i.e. the convergence with the grid refinement was smoother than for the local variables. IV. Final Remarks The Workshop on Verification of Calculations held in Lisbon in October 2004 aimed at assessing our ability to make reliable uncertainty estimates for complex turbulent flows. Although 2-D test cases have been selected for this exercise, the complexity of the flows, which involved significant regions of flow separation, proved to be sufficient for the evaluation of the Uncertainty Estimators. One of the important lessons re-learned in this Workshop is that the problems proposed to the participants must be completely specified, i.e. all boundary conditions should be given. This is a fundamental detail to permit a meaningful comparison of predictions and error bars from different flow solvers. However, not all boundary condition types can be handled by all commercial codes (e.g., normal derivatives of vertical velocity at inflow). This can confuse modelling assumptions and grid convergence behavior, making intercode comparisons problematical at high resolutions. Moreover, the Verification of Calculations should be preceded by Code Verification. The Method of the Manufactured Solutions 6 seems to be the best choice to generate a realistic exact solution for a future Workshop. It was also agreed that any future Workshop should specifically invite participation by groups using unstructured grid (or grid-free) CFD solutions. Known applications of the GCI to unstructured grids using a crude effective grid refinement ratio have been ad hoc and highly limited. A more extensive evaluation using a systematic approach and an exact solution would be welcome. It was generally agreed that more extensive problem sets and more difficult problems should be addressed at some time in the future, but that the next Workshop on the subject should still be limited to one or two simple problems. Comparisons are difficult, even with simple problems. The limitation to simple problems will both encourage participation, and will keep out extraneous factors that could confuse a definitive and convincing evaluation of Uncertainty Estimators from a broadly based segment of CFD practitioners. Acknowledgments The Workshop on CFD Uncertainty Analysis was sponsored by IST, MARETEC, Fundação Luso- Americana para o Desenvolvimento and by the Maritime Research Institute Netherlands. We would like to thank all the participants of the Workshop for the fantastic atmosphere and the open technical discussions that existed during the entire Workshop. References 1 ERCOFTAC Classic Collection Database Spalart P.R., Allmaras S.R. - A One-Equations Turbulence Model for Aerodynamic Flows - AIAA 30th Aerospace Sciences Meeting, Reno, January Menter F.R. - Eddy Viscosity Transport Equations and Their Relation to the k ɛ Model - Journal of Fluids Engineering, Vol. 119, December 1997, pp Wilcox D.C. -Turbulence Modeling for CFD - DWC Industries 1998, 2 nd Edition. 5 Chang K.C., Hsieh W.D., Chen C.S. - A Modified Low-Reynolds-Number Turbulence Model Applicable to Recirculating Flow in Pipe Expansion - Journal of Fluids Engineering, Vol. 117, 1995, pp Roache P.J. - Verification and Validation in Computational Science and Engineering - Hermosa Publishers, Eça L., Hoekstra M. - Uncertainty Estimation : A Grand Challenge for Numerical Ship Hydrodynamics - 6 th Numerical Towing Tank Symposium, Rome, September Hay A. - Etude des stratégies d estimation d erreur numérique et d adaptation locale de maillages non-structurés pour les équations de Navier Stokes en moyenne de Reynolds - PhD Thesis, Université de Nantes, (ftp://ftp.ec-nantes.fr/pub/dmn/thesis) 11 of 12

12 9 Vinokur M. - On One-Dimensional Stretching Functions for Finite-Difference Calculations. - Journal of Computational Physics, Vol. 50, 1983, pp Almeida G.P., Durão D.F.G., Heitor M.V. - Wake Flows behind Two-dimensional Model Hills - Experimental Thermal and Fluid Science, 7, p. 81, Driver D.M., Seegmiller H.L. - Features of a Reattaching Turbulent Shear Layer in Divergent Channel Flow - AIAA Journal, Vol. 23, N. 2, February Proceedings of the Workshop on CFD Uncertainty Analysis - Eça L., Hoekstra Eds., Instituto Superior Técnico, Lisbon, October Chien K.Y - Prediction of Channel and Boundary-Layer Flows with a Low-Reynolds-Number Turbulence Model. - AIAA Journal, January 1982, pp Eça L, Hoekstra M. - An Evaluation of Verification Procedures for CFD Applications - 24 th Symposium on Naval Hydrodynamics, Fukuoka, Japan, July of 12