PhD Prelimiary Oral Exam CHARACTERIZATION AND PREDICTION OF CFD SIMULATION UNCERTAINITIES by Serhat Hosder Chair: Dr. Berard Grossma Committee Members: Dr. Raphael T. Haftka Dr. William H. Maso Dr. Reece Neel Dr. Rimo Arieli Departmet of Aerospace ad Ocea Egieerig Virgiia Tech. Blacksburg, VA Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 0
Outlie of the Presetatio Itroductio Classificatio of CFD Simulatio Ucertaities Objective of the Preset Work Previous Studies Trasoic Diffuser Case Results, fidigs ad discussio about differet sources of ucertaity Coclusios Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 1
Itroductio (1) The Computatioal Fluid Dyamics (CFD) as a aero/hydrodyamic aalysis ad desig tool Icreasigly beig used i multidiscipliary desig ad optimizatio (MDO) problems Differet levels of fidelity (from liear potetial solvers to RANS codes) CFD results have a certai level of ucertaity origiatig from differet sources Sources ad magitudes of the ucertaity importat to assess the accuracy of the results Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 2
Itroductio (2) Drag Polar Results for DLR F-4 Wig at M=0.75, Re c =3x10 6 (take from 1 st Predictio Workshop (DPW), Ref. 1) AIAA Drag Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 3
Classificatio of CFD Simulatio Ucertaities Physical Modelig Ucertaity PDEs describig the flow (Euler, Thi-Layer N-S, Full N-S, etc.) Boudary ad iitial coditios (B.C ad I.C) Auxiliary physical models (turbulece models, thermodyamic models, etc.) Ucertaity due to Discretizatio Error Numerical replacemet of PDEs ad cotiuum B.C with algebraic equatios Cosistecy ad Stability of PDEs Spatial (grid) ad temporal resolutio Ucertaity due to Iterative Covergece Error Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 4
Defiitio of Ucertaity ad Error Oberkampf ad Trucao (Ref. 2) defied Ucertaity as a potetial deficiecy i ay phase or activity of modelig process that is due to the lack of kowledge (ucertaity of turbulece models, geometric dimesios, thermo-physical parameters, etc.) Error as a recogizable deficiecy i ay phase or activity of modelig ad simulatio Discretizatio errors ca be estimated with certai methods by providig certai coditios I this work, we ll refer the iaccuracy i the CFD simulatios due differet sources as ucertaity Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 5
Objective of the Preset Work Characterize differet sources of CFD simulatio ucertaities Cosider differet test cases Apply differet grids, solutio schemes/parameters, ad physical models Try to quatify/predict the magitude ad the relative importace of each ucertaity Compare the magitudes of CFD simulatio ucertaities with other sources of ucertaity (geometric ucertaity, ucertaity i flow parameters, etc.) Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 6
Previous Studies Previous CFD related studies maily focused o discretizatio ad iterative covergece error estimatios Grid Covergece Idex (GCI) by Roache (Ref. 3) Discretizatio Error of Mixed-Order Schemes by C. D. Roy (Ref. 4) Trucao ad Hill (Ref. 5) proposed statistical based validatio metrics for Egieerig ad Scietific Models Hemsch (Ref. 6) performed statistical aalysis of CFD solutios from 1 st AIAA DPW. Kim (Ref. 7) made statistical modelig of simulatio errors (from poorly coverged optimizatio rus) ad their reductio via respose surface techiques Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 7
Descriptio of Trasoic Diffuser Test Case (1) Kow as Sajbe Trasoic Diffuser case i CFD Validatio studies Top wall described by a aalytical equatio Although geometry is simple, the flow-field is complex. The Shock stregth ad the locatio determied by exit-pressure-to-ilettotal pressure ratio P e /P 0i P e /P 0i =0.72 (Strog shock case), P e /P 0i =0.82 (Weak shock case), Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 8
Descriptio of Trasoic Diffuser Test Case (2) Mach cotours for the weak shock case Mach cotours for the strog shock case Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 9
Simulatio Code, Solutio Parameters, ad Grids (1) Geeral Aerodyamic Simulatio Program (GASP) 3-D, structured, multi-block, fiite-volume, RANS code Iviscid fluxes calculated by upwid-biased 3 rd (omial) order spatially accurate Roe-flux scheme All viscous terms were modeled (full N-S) Implicit time itegratio to reach steady-state solutio with Gauss-Seidel algorithm Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 10
Simulatio Code, Solutio Parameters, ad Grids (2) Flux-Limiters Va Albada s limiter Mi-Mod limiter Turbulece Models Spalart-Allmaras (Sp-Al) k-w (Wilcox, 1998 versio) Grids Geerated by a algebraic mesh geerator Grid 1 (g1): 41x26x2 Grid 2 (g2): 81x51x2 Grid 3 (g3): 161x101x2 Grid 4 (g4): 321x201x2 Grid 5 (g5): 641x401x2 (Used oly for Sp-Al, Mi-Mod, strog shock case) y + = 0.53 (for g2) ad y + = 0.26 (for g3) at the bottom wall Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 11
Output Variables (1) Nozzle efficiecy, eff H 0i : Total ethalpy at the ilet H e : Ethalpy at the exit eff = H H H es : Exit ethalpy at the state that would be reached by isetropic expasio to the actual pressure at the exit H H H e es y i = ρ( y) u( y) h 0i 0 y e = ρ ( y) 0 y e oi u( y) h = ρ( y) u( y) h 0 ( y) d y es e ( y) d ( y) y d y 0i 0i H H h es Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 12 e es ( y) = c y = y h t p T oi P ( ) e y P0 i γ 1 γ Throat height
Output Variables (2) Orthogoal Distace Error, E A measure of error i wall pressures betwee the experimet ad the curve represetig the CFD results d i = mi x x ilet x exit E = N exp i= 1 N d exp i { } 2 ( x x ) + ( P ( x) P ( x )) 2 P c : Wall pressure obtaied from CFD calculatios P exp : Experimetal Wall Pressure Value N exp : Total umber of experimetal poits (N exp =36) d i : Orthogoal distace from the i th experimetal data poit to P c (x) curve i c exp i Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 13
Ucertaity due to iterative covergece error (1) Normalized L2 Norm Residual of the eergy equatio for the case with Sp-Al turbulece model, Va-Albada ad Mi-Mod limiters at the strog shock case. Same covergece behavior with respect to the limiters observed for the k-w case. Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 14
Ucertaity due to iterative covergece error (2) Poor L2 orm covergece does ot seem to effect the covergece of the eff results at differet grid levels Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 15
Ucertaity due to iterative covergece error (3) Roy ad Blotter (Ref. 8) proposed a method to estimate, iterative covergece error at time level (cycle) ( eff ) exact = eff eff = eff ( t ) = ( eff ) exact Assumig expoetial decrease for Λ 1 Λ ε 1 eff Λ = ε = αe β + 1 eff eff t + ε Need three time levels i the expoetial regio where 1 eff Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 16 ε 100 eff eff = eff 1 Λ + 1 eff eff % error of eff = 1 Λ eff + 1 eff ε the
Ucertaity due to discretizatio error (1) For each case with a differet turbulece model, grid level (resolutio) ad the flux-limiter affect the magitude of the discretizatio error The effect of the limiter observed at grid levels g1 ad g2 At grid levels g3 ad g4, the effect is much smaller Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 17
Ucertaity due to discretizatio error (2) Richardso s extrapolatio method: f k = f exact + α h p + O( h p+1 ) h: a measure of grid spacig p: The order of the method. Assumptios eeded to use Richardso s method: Grid resolutio is i the asymptotic regio The order of the spatial accuracy, p should be kow. Usually observed order of spatial accuracy is differet tha the omial value. The observed order should be determied. Mootoic grid covergece. Mixed-order schemes ca cause o-mootoic covergece. Roy (Ref. 4) proposed a method for for the discretizatio error estimate of mixed-order schemes. Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 18
Ucertaity due to discretizatio error (3) turb. model limiter Pe/P0i p (eff) exact Sp-Al Va Albada 0.72 1.528 0.71830 Sp-Al Mi-Mod 0.72 1.322 0.71590 Sp-Al Va Albada 0.82 1.198 0.80958 Sp-Al Mi-Mod 0.82 1.578 0.81086 k-w Va Albada 0.82 1.980 0.82962 k-w Mi-Mod 0.82 1.656 0.82889 grid level discretizatio error (%) g1 9.820 g2 4.505 g3 1.562 g4 0.542 g1 14.298 g2 6.790 g3 2.716 g4 1.086 g1 6.761 g2 3.507 g3 1.528 g4 0.666 g1 8.005 g2 3.539 g3 1.185 g4 0.397 g1 3.514 g2 1.459 g3 0.370 g4 0.094 g1 4.432 g2 1.452 g3 0.461 g4 0.146 Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 19
Ucertaity due to discretizatio error (4) p values are depedet o the grid levels used However the differece betwee the ( eff ) exact values are small compared to overall ucertaity turb. model limiter Pe/P0i grid levels used p eff_exact grid level discretizatio error (%) g1 14.405734 Sp-Al Mi-Mod 0.72 g2, g3, ad g4 1.303 0.715235 g2 6.940532 g3 2.812698 g4 1.139865 g5 0.729023 g1 13.728739 Sp-Al Mi-Mod 0.72 g3, g4, ad g5 2.026 0.719492 g2 6.307712 g3 2.204303 g4 0.5413695 g5 0.1329585 Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 20
Ucertaity due to discretizatio error (5) The ucertaity due to discretizatio error is bigger for the cases with strog shock compared to the weak shock results at each grid level. The flow structure has sigificat effect o the discretizatio error. For the mootoic cases, largest errors occur for the Sp- Al, Mi-Mod, strog shock case ad the smallest errors are obtaied for the k-w, Va-Albada, weak shock case No-mootoic covergece behavior for the cases with k-w ad the strog shock as the mesh is refied Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 21
Ucertaity due to discretizatio error (6) Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 22
Ucertaity due to discretizatio error (7) Noise due to discretizatio error observed at grid levels 1 ad 2. Noise error small compared to the systematic discretizatio error betwee each grid level. However, this ca be importat for gradiet-based optimizatio. Kim (Ref. 7) successfully modeled the the oise error due to poor covergece of the optimizatio rus by fittig a probability distributio (Weibull) to the error. The oise error ca be reduced via respose surface modelig. Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 23
Ucertaity due to turbulece models (1) Ucertaity due to turbulece modelig (i geeral physical modelig) should be ivestigated after estimatio of the discretizatio ad iterative covergece error. Difficult to totally separate physical modelig errors from discretizatio errors Validatio of the Egieerig ad Scietific Models deals with accuracy of the physical model Need high-quality experimetal data Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 24
Ucertaity due to turbulece models (2) Orthogoal distace error, E is used for compariso of CFD results with the experimet E for each case is scaled with the maximum value obtaied for k-w, Mi-Mod, strog shock case Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 25
Ucertaity due to turbulece models (3) For each case (strog shock or weak shock), best match with the experimet is obtaied with differet turbulece models at differet grid levels Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 26
Ucertaity due to turbulece models (4) Experimetal ucertaity should be cosidered With the experimetal geometry, a perfect match with CFD ad experimet ca be observed upstream of the shock Upstream of the shock, discrepacy betwee CFD simulatios ad experimet is most likely due to the experimetal ucertaity Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 27
Ucertaity due to turbulece models (5) A better way of usig E for this example would be to evaluate it oly dowstream of the shock The discretizatio ad iterative covergece error should be estimated for E i a similar way used for the ozzle efficiecy A estimate of exact value of (E ) ca be used for approximatig the ucertaity due to turbulece models The relative ucertaity due to the selectio of turbulece models ca also be ivestigated by usig ( eff ) exact values obtaied by Richardso s extrapolatio Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 28
Ucertaity due to turbulece models (6) Hills ad Trucao (Ref. 5) proposed a Maximum Likelihood based model validatio metric to test the accuracy of the model predictios Ucertaity i the experimetal measuremets ad the model parameters are cosidered Model parameters: Material properties Geometry Boudary or Iitial Coditios This method requires prior kowledge about the measuremet ad the model parameter ucertaity (modelig with probabilistic distributios) Looks for statistically sigificat evidece that the model validatios are ot cosistet with the experimetal measuremets Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 29
Ucertaity due to turbulece models (7) PDF(d) : PDF of measuremet vector occurrece PDF(p) : PDF of model parameter vector occurrece PDF(d, p) = PDF(d) x PDF(p) Fid the maximum likely values for the mode of the measuremets d ad the model parameters p Fid the maximum value of Joit PDF via optimizatio Evaluate the probability of obtaiig a smaller PDF assumig that the model is correct If this value is bigger tha the level of sigificace that we assumed for rejectig a good model, tha the model predictios are cosistet with the experimet Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 30
Ucertaity due to turbulece models (8) Possible applicatio to test the accuracy of the turbulece models Takes ito accout the experimetal ucertaity Requires prior kowledge of ucertaity i the measuremets ad the model parameters Selectio of model parameters No simple relatioship with the model parameters ad the output quatities. Usig respose surface techiques may be eeded to fid a fuctioal form. Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 31
Additioal Test Cases Need more cases to geeralize the results obtaied i Trasoic Diffuser Case Next possible case : Steady, turbulet, flow aroud a airfoil (RAE2822 or NACA0012) Cosider trasoic ad subsoic cases Cosider a rage of AOA Output quatities to moitor: Cl, Cd, Cp distributios Orthogoal distace error may be used for characterizig Cp distributios Cosider a case with a more complex geometry Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 32
Coclusios (1) Differet sources of ucertaity i CFD simulatios should be ivestigated separately. Discretizatio ad iterative covergece errors ca be estimated by certai methods i certai coditios Limiters affect the iterative covergece ad the discretizatio error. L2 orm covergece affected by the use of differet limiters Poor L2 orm covergece do ot seem to affect the eff results Asymptotic Grid covergece hard to obtai Flow structure has a strog effect o the magitude of the discretizatio error. Iterative covergece error small compared to the discretizatio error Ucertaity due to turbulece model should be ivestigated after the estimatio of discretizatio ad iterative covergece error. Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 33
Coclusios (2) Compariso with the experimet is eeded to determie the accuracy of the turbulece models Experimetal ucertaity should be cosidered possibly by usig a statistical method More cases eed to be aalyzed to geeralize the results Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 34
Refereces 1. Levy, D. W., Zickuhr, T., Vassberg, J., Agrawal S., Wahls. R. A., Pirzadeh, S., Hemsch, M. J., Summary of Data from the First AIAA CFD Drag Predictio Workshop, AIAA Paper 2002-0841, Jauary 2002 2. Oberkampf, W. L. ad Trucao, T. G., Validatio Methodology i Computatioal Fluid Dyamics. AIAA Paper 2000-2549, Jue 2000 3. Roache, P. J. Vericatio ad Validatio i Computatioal Sciece ad Egieerig.Hermosa Publishers, Albuquerque, New Mexico, 1998. 4. Roy, C. J., Grid Covergece Error Aalysis for Mixed-Order Numerical Schemes, AIAA Paper 2001-2606, Jue 2001 5. Hills, R. G. ad Trucao, T. G., Statistical Validatio of Egieerig ad Scietific Models: A Maximum Likelihood Based Metric, Sadia Natioal Loboratories, SAND2001-1783 6. Hemsch, M. J., Statistical Aalysis of CFD Solutios from the Drag Predictio Workshop, AIAA Paper 2002-0842, Jauary 2002 7. Kim, H., Statistical Modelig of Simulatio Errors ad Their Reductio Via Respose Surface Techiques, PhD dissertatio, VPI&SU, Jue 2001 8. Roy, C. J. ad Blotter F. G., Assesmet of Oe-ad Two-Equatio Turbulece Models for Hypersoic Trasitioal Flows, Joural of Spacecraft ad Rockets, Vol.38, No. 5, September-October 2001 Ph.D Prelimiary Oral Exam, Aerospace ad Ocea Egieerig Departmet, February 27 th 2002 35