Numerical model validation using experimental data: Application of the area metric on a Francis runner

Size: px

Start display at page:

Download "Numerical model validation using experimental data: Application of the area metric on a Francis runner"

Brook Underwood
5 years ago
Views:

IOP Coferece Series: Earth ad Evirometal Sciece PAPER OPEN ACCESS Numerical model validatio usig experimetal data: Applicatio of the area metric o a Fracis ruer To cite this article: Q Chateet et al

1 IOP Coferece Series: Earth ad Evirometal Sciece PAPER OPEN ACCESS Numerical model validatio usig experimetal data: Applicatio of the area metric o a Fracis ruer To cite this article: Q Chateet et al 2016 IOP Cof. Ser.: Earth Eviro. Sci Related cotet - Caoical differetial geometry of strig backgrouds Frederic P. Schuller ad Mattias N.R. Wohlfarth - Propagatio of light i area metric backgrouds Raffaele Puzi, Frederic P Schuller ad Mattias N R Wohlfarth - Radiatio-domiated area metric cosmology Frederic P Schuller ad Mattias N R Wohlfarth Vie the article olie for updates ad ehacemets. This cotet as doloaded from IP address o 20/09/2018 at 00:01

2 Numerical model validatio usig experimetal data: Applicatio of the area metric o a Fracis ruer Q Chateet 1, A Taha 1, M Gago 2 ad J Chamberlad-Lauzo 3 1 Mechaical Egieerig Departmet, École de Techologie Supérieure (ÉTS), 1100 Rue Notre-Dame O, Motréal, QC, Caada H3C1K3 2 Istitut de Recherche d Hydro-Québec (IREQ), 1800 boul. Lioel-Boulet, Varees, QC, Caada J3X1S1 3 Adritz Hydro Caada Ic., 6100 aut. Trascaadiee, Poite Claire, QC, Caada H9R1B9 queti.chateet@gadz.org Abstract. Noadays, egieers are able to solve complex equatios thaks to the icrease of computig capacity. Thus, fiite elemets softare is idely used, especially i the field of mechaics to predict part behavior such as strai, stress ad atural frequecy. Hoever, it ca be difficult to determie ho a model might be right or rog, or hether a model is better tha aother oe. Nevertheless, durig the desig phase, it is very importat to estimate ho the hydroelectric turbie blades ill behave accordig to the stress to hich it is subjected. Ideed, the static ad dyamic stress levels ill ifluece the blade s fatigue resistace ad thus its lifetime, hich is a sigificat feature. I the idustry, egieers geerally use either graphic represetatio, hypothesis tests such as the Studet test, or liear regressios i order to compare experimetal to estimated data from the umerical model. Due to the variability i persoal iterpretatio (reproducibility), graphical validatio is ot cosidered objective. For a objective assessmet, it is essetial to use a robust validatio metric to measure the coformity of predictios agaist data. We propose to use the area metric i the case of a turbie blade that meets the key poits of the ASME Stadards ad produces a quatitative measure of agreemet betee simulatios ad empirical data. This validatio metric excludes ay belief ad criterio of acceptig a model hich icreases robustess. The preset ork is aimed at applyig a validatio method, accordig to ASME V&V 10 recommedatios. Firstly, the area metric is applied o the case of a real Fracis ruer hose geometry ad boudaries coditios are complex. Secodly, the area metric ill be compared to classical regressio methods to evaluate the performace of the method. Fially, e ill discuss the use of the area metric as a tool to correct simulatios. 1. Itroductio Give the complexity of the eeds required to perform the validatio process, the ASME V&V 10 committee [1] prefers ot to recommed specific metrics to evaluate the accuracy or the goodess-tofit of the model, but rather suggest to specify hich metric is used, ad hat is the chose acceptace level durig the validatio pla desig. This ork agrees ithi the scope of the Verificatio & Validatio process as sho i figure 1. Nevertheless, e decided to focus o the validatio part of the process because most of the difficulties occur durig this phase (the validatio phase is sho i Cotet from this ork may be used uder the terms of the Creative Commos Attributio 3.0 licece. Ay further distributio of this ork must maitai attributio to the author(s) ad the title of the ork, joural citatio ad DOI. Published uder licece by Ltd 1

the grey rectagle i Fig. 1). Besides, it is ofte difficult to go through the verificatio process as most softare used are ot ope source; thus, oly the developer has access to the code. Figure 1.

3 the grey rectagle i Fig. 1). Besides, it is ofte difficult to go through the verificatio process as most softare used are ot ope source; thus, oly the developer has access to the code. Figure 1. Verificatio & Validatio activities (ASME V&V 10) Specifically, i the hydropoer field, it is importat for maufacturers or plat operators to estimate the life expectacy of their hydraulic poer plats ad pla he to perform maiteace icludig ispectios. I particular, for hydroelectric turbie ruers, they eed to rely o estimatio of the stress level at the blade s hot spots. Oe of the challeges facig today s desig egieers is determiig the suitable mechaical models (good eough for the iteded purpose) compared to experimetal data, ad be able to choose the best oe. To esure this, egieers have to use a adequate metric to quatify the discrepacy betee the predicted results ad actual data. This metric should take ito accout the ucertaities i both: the simulatio outcomes ad the experimetal outcomes [1]. The metric used to perform model validatio should also satisfy these coditios accordig to ASME V&V 10 recommedatios [1]: A validatio metric provides a method by hich the simulatio outcomes ad the experimetal outcomes ca be compared. A metric is a mathematical 2

4 measure of the differece betee the to outcomes such that the measure is zero oly if the to outcomes are idetical. The proposed validatio metric, i this paper, is the area metric developed by Ferso et al. [2] to itegrate these recommedatios ad excludes ay belief ad acceptace criterio cotrary to classical hypothesis tests [3]. This paper is structured as follos. First, a revie of the area metric ad its characteristics is preseted i Sec 2. Next, i Sec. 3 e apply the method o a real case of a hydraulic turbie ruer blade i order to determie hich model is more accurate. Fially, a discussio o the results of the study case ad a compariso ith classical methods cocludes the paper. 2. Area metric The area metric displays several advatages compared to other validatio methods (i.e.: hypothesis tests, Bayes factor or frequetist s metric) [3]. Oe of these is objectiveess, sice the coclusio give by the area metric does ot deped o ho a egieer coductig the aalysis, or o the assumptios he makes. The area metric allos egieers to coduct the validatio process based o a quatitative measure of the discrepacy betee predictios ad data. At the same time, it gives a graphical represetatio (i the physical uits used) of this discrepacy. Moreover, this metric allos oe to evaluate the differeces across the full rage of predictio distributio hile takig ito accout simulatio ucertaities. I order to apply the area metric method, e eed to compute predictios hich ca be represeted as a cumulative distributio fuctio F( x ) (CDF), here x is the variable of the predictio. Observatios are represeted by a o-decreasig step fuctio ith a costat vertical step of 1 / ( represets the size of the data set). The x-axis value of the steps correspods to the data poits. Usig equatio (1), e ca costruct this fuctio for data x i. I ( x, ) 1, 1 i x xi x i= S ( x) =, here I ( xi, x) = (1) 0 xi > x, The measure of the mismatch betee predictio ad observatio is thus the area betee the predictio fuctio F ad the data distributio S ad ca be evaluated usig equatio (2). d( F, S ) = F( x) S ( x) dx (2) It ca be oted that if the fuctios F ad S are idetical, the the area metric ill be equal to zero, hich satisfies oe of the ASME V&V recommedatios. The figure 2 illustrates the mismatch betee the predictio distributio (sho as the gray curve) ad the experimetal data (sho as the dark step fuctio ith the metric beig the grey shaded area). The latter d( F, S ) is computed usig equatio (2) o the etire rage o both the predictio ucertaity distributio ad empirical data. I this example, a ormal distributio has bee chose to describe umerical model. Isofar as the x-axis uit is expressed i MPa ad the y-axis is uit less, the area is expressed i MPa too, hich is the uit of iterest for stress levels. Moreover, as the metric is ot ormalized, the egieer is able to evaluate the impact of differet values of the area metric accordig to the level of stress he is dealig ith. As a example, if the result of the area metric is 5 MPa for a set of measuremets hose mea is about 10 MPa, the egieer ill ot dra the same coclusios as if the mea value as 100 MPa. 3

robust. Ideed, figure 3 illustrates examples ith to sets of data, both of hich have strictly idetical meas ad stadard deviatios (i.e.: µ 1 = µ 2 = 180 MPa ad σ1 = σ 2 = 40 MPa).

5 Figure 2. Example data set ( = 6 ) As the area metric is applied o the etire ucertaity distributio of the predictios ad the data, the metric is particularly sesitive, hile at the same time remaiig robust. Ideed, figure 3 illustrates examples ith to sets of data, both of hich have strictly idetical meas ad stadard deviatios (i.e.: µ 1 = µ 2 = 180 MPa ad σ1 = σ 2 = 40 MPa). The grey curve represets same ormal distributio o both figures. The results of the metric are respectively MPa ad MPa for those to sets of data. The area metric allos oe to get more iformatio from the validatio aalysis, hereas dealig oly ith the first to statistical parameters ould yield o oticeable differece. Figure 3. Compariso of to differet data sets ith µ 1 = µ 2 ad σ1 = σ 2. 4

6 3. Case study The area metric method has bee applied to idetify the best mechaical model agaist experimetal data. For this purpose, to sets of fiite elemets aalysis (FEA) have bee computed accordig to to differet mechaical models o a real ad ruig Fracis hydraulic turbie ruer (medium head ad omial poer >100MW) Probability distributio of simulatio ucertaity I order to perform the validatio process, it is better to focus o probability distributio rather tha o a sigle predicted value. Ideed, a distributio better reflects the variability of the model outcomes isofar as ucertaities ca be itegrated durig computatio (such as epistemic ucertaities). I his ork, J. Arpi-Pot [4] developed a method of accoutig positio ad orietatio ucertaities, itegratio effect, ad ucertaities due to the elded gauge techology. Actually, it is impossible to accurately positio a elded gauge at a target locatio as sho i figure 4, hich results i positio ucertaities. The method cosists i simulatig virtual strai gauges located aroud the fiite elemets target ode, for umerous differet positios ad orietatios. The purpose of this is to reflect the replicability of the measuremet usig radom distributio model. Thus, ith a Mote Carlo simulatio, a distributio of the strai measured by the virtual gauge is geerated ad ca be compared to the empirical values. Figure 4. Defiitio of ucertaity sources: elded gauge behavior, locatio ucertaity (icludig positioig ad aligmet errors) ad itegratio effect [4] Numerical models Firstly, computatioal fluid dyamics (CFD) aalysis at maximum poer ere performed usig to differet CFD settigs givig to pressure-loadig cases for this turbie. Thus, i terms of FEA softare, differet pressure fields ere imposed to the blade depedig o the model used. Figure 5. Positio of the gauges placed o the hydraulic turbie blade durig measuremets. 5

7 Accordig to previous ork doe by J. Arpi Pot [5], e ere able to evaluate positio ad orietatio errors for each gauge locatio o the blade. Notice that the same Fracis ruer is used i this study. The blade is istrumeted evely o both sides, positio ad orietatio ucertaities are the same for sites 1 ad 3 ad for sites 2 ad 4 respectively. The parameters used to perform the simulatio are preseted i table 1. The simulatio distributio obtaied for FEA models #1 ad #2 are obtaied from strai gauge positio ucertaity o FEA strai results. Locatio Table 1. Positioig ucertaity distributio las used for simulatio. Positio alog target axis X (mm) Ucertaities (95% cofidece level) Positio alog target axis Y (mm) Aligmet agle ( ) Sites 1-3 [-8; 8] [-2; 5] [-2; 2] Sites 2-4 [-12; 6] [-9; 3] [-6; 6] Site 5 [-14; 14] [-3; 6] [-7; 7] 3.3. Experimetal process Accordig to FEA results, strai gauges have bee elded at specific locatios (as sho i figure 4) o each of the to blades alog the pricipal strai directio give by a iitial FEA aalysis. Oe of the assumptios is the strai measured by the gauges is alog the pricipal directio, hich caot be verified sice uidirectioal elded gauges ere used. The use of strai rosette gauges ould eable the directio of the strai to be obtaied ad improve the assessmet of the maximum stress at hot spots. Measured strais ere established from at least to idepedet measuremets made at similar operatig coditios. At each strai gauge site, the empirical distributio fuctio the iclude measuremets at both istrumeted blade ad at several idepedet measuremets i time. 4. Results Area metric results comparig simulatio distributio ad measuremets are sho i table 2, ith detailed graph for site 3 (see figure 6). Table 2. Area metric results. Locatio Model #1 (MPa) Model #2 (MPa) Site 1 Cro outflo SS a Site 2 Bad outflo SS a Site 3 Cro outflo PS b Site 4 Bad outflo PS b Site 5 Cro ceter PS b Sum a Suctio Side b Pressure Side As e ca see i figure 6, the area metric method provides both a umerical result of the mismatch betee predictio ad data ad a graphical represetatio of the result. Because of the smaller area metric value, e ca coclude i this case that the model #1 predictios are closer to experimetal data tha model #2. 6

Figure 6. Site 3 Area metric compariso of model #1 vs. model #2. The dashed lie i figure 6 represets the omial computed stress value at the ode uder the strai gauge. 4.1. Evolutio of the area metric as a fuctio of various operatig coditios.

Hoever, measures ere performed for several opeigs (12% to 100%) i order to study trasiet states i particular.

The aim is to perform validatio process hile lookig for operatig coditios hich better match ith simulatio results.

The area metric icreases sigificatly as icket gate opeig decreases hich suggests that FEA performed ith maximum poer CFD pressures does ot represet lo ad part load measured strais. Table 3.

8 Figure 6. Site 3 Area metric compariso of model #1 vs. model #2. The dashed lie i figure 6 represets the omial computed stress value at the ode uder the strai gauge Evolutio of the area metric as a fuctio of various operatig coditios. The CFD simulatio as doe for the maximum omial poer of the turbie, hich correspods to a icket gates opeig of 100%. Hoever, measures ere performed for several opeigs (12% to 100%) i order to study trasiet states i particular. The, it is possible to compare area metric results depedig o the turbie operatig coditio, as sho i table 3. The aim is to perform validatio process hile lookig for operatig coditios hich better match ith simulatio results. As expected, FEA performed ith maximum poer CFD pressures better predicts measuremets at 100% icket gate opeig. The area metric icreases sigificatly as icket gate opeig decreases hich suggests that FEA performed ith maximum poer CFD pressures does ot represet lo ad part load measured strais. Table 3. Evolutio of the area metric as a fuctio of icket-gate opeig. Wicket gate opeig Locatio 12% 40% 75% 100% Site 1 Cro outflo SS a Site 2 Bad outflo SS a Site 3 Cro outflo PS b Site 4 Bad outflo PS b Site 5 Cro ceter PS b Sum Compariso ith classical methods The least squares liear regressio is oe of the most-used methods i model validatio because it is easy to compute ad to uderstad. Yet this method has severe limitatios. The result of least squares regressio is improved by icreasig sample size ad does ot take ito accout ucertaities o both 7

9 axes, uless oe uses eighted least squares [6, 7] for example. The eighted least squares estimates ( β0 ad β 1 ) are the give by equatios (4) here x ad y are the eighted meas. 1 = i 2 2 ( x ) + ( y ) (3) i ixi β0 = y β1 x x = i, i ( xi x)( yi y) here (4) β1 = 2 i yi i ( xi x) y = i For each poit, eights are assiged as described by equatio (4). Ucertaities o each axis xi ad yi are determied by applyig recommedatios [8], modeled by equatio (5). rage( xi ) xi = (5) 2 3 i Figure 7. Compariso of computed stress ad experimetal stress (from strai measures) at 75% (left) ad 100% (right) opeig. Poits represet the mea of sample data ad probability distributio; bars associated ith poits correspod to ucertaities accordig to each axis. I figure 7, the dashed lie represets the simple liear regressio, hich does ot take ito accout ucertaities, ulike the eighted least squares ith a solid lie. The eighted liear regressio curve is give by: y = β x + β Discussio First, ith respect to the model compariso, e observe that the area metric result is loer for model #1, hich meas this model better reflects empirical observatios. Hoever, this is ot the case for oe of the five sites ith a area metric result loer for model #2. It could be explaied by the fact that e have oly four (4) observatios for this specific site hereas e have eight (8) observatios for others, resultig i a lack of iformatio. Note that compilig area metric results for all sites, cosiderig the mea, model #2 is about three times higher i terms of area metric tha model #1 (34.02 MPa for model #1, MPa for model #2). 8

10 A secod observatio ca be made from table 3. Ideed, for sites 2, 4 ad 5, the further aay from a omial operatig poit, the more the area metric icreases, hich is cosistet sice simulatios have bee made for maximum icket gates opeig. Coversely, for sites 1 ad 3, hich are located symmetrically o opposite sides, the area metric value is the loest for a icket opeig of 75% ad 40%, respectively. This result ca be used to correct FEA simulatios, cosiderig values evolve proportioally as a fuctio of icket-gate opeig. Thus, applyig this correctio for site 1 predictios, the area metric result drops to MPa istead of MPa (for a icket gates opeig of 100%). As sho ith liear regressios i figure 7, ofte used i egieerig to compare models, it is rather difficult to evaluate if the model is right or rog accordig to regressios parameters. Ideed, eve if the slope is close to oe (1), e caot coclude o the model accuracy isofar as poits ca be symmetrically distributed aroud the liear regressio curve. Also, the real turbie geometry is ot take ito accout as a ucertaity source. Hoever, FEA is based o desig geometry hile strai measures are from the real ruer hose geometry varies i the tolerace iterval because of maufacturig costraits. This variability added to gauge positio ucertaities, might explai the mismatch betee model predictios ad experimetal data i some cases. 6. Coclusio I this paper, e preseted a applicatio of the area metric method ad e coducted a validatio process accordig to ASME V&V recommedatios usig the former metric. This method relies o a calculatio area betee to probability distributio: oe from simulatio predictio ad the secod from experimetal data. Thus, accordig to area metric values, egieer is able to choose hich model is the more accurate. Whe applied to i situ measuremets carried out o hydraulic turbie ruer blades, area metric results sho that model #1 is about three times more accurate (o average) tha model #2. Hoever, e observe that i particular locatios, model #2 predictios ca be closer to experimetal values. Likeise, usig results for differet operatig coditios, e sho that usig the area metric, a extrapolatio ca be made to correct some predictio simulatios. Fially, oe of the most difficult questios to aser cocers the level of acceptace. Ideed, validatio pla must predetermie the accuracy requiremets to validate a model accordig to the metric used. Ackoledgmets This study as carried out ith the fiacial support of the Caadia research itership program, Mitacs-Accelerate. The authors ackoledge Adritz Hydro Caada, the Istitut de Recherche d Hydro-Québec (IREQ) ad the École de techologie supérieure (ÉTS) for their support, materials ad advice. Nomeclature d( F, S ) Itegral fuctio σ Stadard deviatio of distributio ( S x ) Step fuctio of empirical data xi Ucertaities alog x-axe Sample size yi Ucertaities alog y-axe x Variable of the predictio i Poit eight F( x ) Cumulative distributio fuctio β Liear regressio slope 1 µ Mea of distributio β Liear regressio Y itercept 0 Refereces [1] ASME 2006 Guide for Verificatio ad Validatio i Computatioal Solid Mechaics vol PTC 60/V&V 10 9

11 [2] Ferso S, Oberkampf W L ad Gizburg L 2008 Model validatio ad predictive capability for the thermal challege problem Computer Methods i Applied Mechaics ad Egieerig [3] Liu Y, Che W, Aredt P ad Huag H-z 2011 Toard a Better Uderstadig of Model Validatio Metrics Joural of Mechaical Desig [4] Arpi-Pot J, Gago M, Taha S A, Coutu A ad Thibault D 2012 Strai gauge measuremet ucertaities o hydraulic turbie ruer blade IOP Coferece Series: Earth ad Evirometal Sciece [5] Arpi-Pot J 2012 Méthode de détermiatio des icertitudes de mesures par jauges de déformatio. (Motréal: École de techologie supérieure) pp 1 ressource e lige (xxiv, 129 p.) [6] Chatterjee S ad Price B 1991 Regressio aalysis by example (Ne York, N.Y.: J. Wiley ad Sos) [7] Rouaud M 2014 Calcul d'icertitudes (Paris, Frace: Creative Commos) [8] ENV13005 N 1999 Guide pour l expressio de l icertitude de mesure. (Paris, Frace: AFNOR) 10

A statistical method to determine sample size to estimate characteristic value of soil parameters

A statistical method to determie sample size to estimate characteristic value of soil parameters Y. Hojo, B. Setiawa 2 ad M. Suzuki 3 Abstract Sample size is a importat factor to be cosidered i determiig