Statstcal tools to perform Senstvty Analyss n the Contet of the Evaluaton of Measurement Uncertanty N. Fscher, A. Allard Laboratore natonal de métrologe et d essas (LNE) MATHMET PTB Berln nd June
Outlne Contet of senstvty analyss n metrology Senstvty analyss accordng to GUM and GUM S. Illustraton based on GUM S eample 9.3 : mass calbraton Senstvty analyss tools Applcaton n fre engneerng Concluson MATHMET PTB Berln nd June
Measurement uncertanty evaluaton Step 3 : Propagaton of uncertanty (LPU, MCM) Step : Quantfcaton of uncertanty sources Standard uncertanty or pdf Step : Analyss of the measurement process Input quanttes Measurement model f(x,..,x n ) Measurand Y = f(x ) Step 4 : Epanded uncertanty y ± U Step 3 : senstvty analyss Feedback MATHMET PTB Berln nd June 3
Contet of senstvty analyss Goal of SA : to understand the nfluence of the nput quanttes or to rank the mportance of uncertantes, hence to gude any addtonal measurement, modellng or R&D efforts Ths work : deals wth provdng a quanttatve tool, that fully descrbes the decomposton of the varance of the measurand n order to dentfy the domnant contrbutons n the varance u (y) doesn t deal wth nvestgatng the senstvty of the output varable to the choce of the nput varables pdfs doesn t address the case of correlated nput varables MATHMET PTB Berln nd June 4
5 MATHMET PTB Berln nd June Senstvty analyss accordng to the GUM In the GUM, senstvty coeffcent and contrbuton for each nput quantty to the uncertanty assocated wth the estmate of the measurand are defned : Senstvty coeffcent : Contrbuton to the varance : Rato n the uncertanty budget : Terms of second order n the Taylor seres epanson (when used) provde nteracton effects between two nput quanttes : ) ( u c = f c ) ( ) ( y u u c ) ( ) ( u u f f f +
Senstvty analyss accordng to the GUM S. The GUM Supplement (Anne B) recommends an approach, known as OAT (One factor At a tme) for senstvty analyss Prncple: holdng all nput quanttes but one fed at ther best estmates whle performng MCM. Ths provdes the pdf for the output quantty havng ust that nput quantty as a varable Generalzaton of the partaldervatve approach Senstvty coeffcent : u( y) c = u( ) Contrbuton to the varance : c u ( ) Rato n the uncertanty budget : u u ( y) ( y) MATHMET PTB Berln nd June 6
GUM S. Eample 9.3 : mass calbraton Measurement equaton 5 nput varables Y = ( mrc dmrc) + ( rhoa ) rhow rhor Varable mrc PDF N( ;.5) dmrc N(.34;.) a U[ ; 3 ] rhow U[7 ;9 ] Senstvty ndces % of V(Y) rhor U[7 95 ;8 5 ] Varable mrc dmrc a rhow LPU st order 86 4 LPU nd order 44 7 OAT GUM S 86 4 rhor Interacton a rhow 48 Interacton a rhor MATHMET PTB Berln nd June 7
Regresson based approach to senstvty analyss Least square can be used to construct the regresson model : p Y = b + = The coeffcents are called the Standardzed Regresson Coeffcents SRC and are often used as measures of varable mportance SRC b sˆ = ρ, Y sˆ = We used the normalzed ndces n order to assess the relatve contrbuton of : SRC b n ( )( y y) n n ( ) ( y y) = = = SRC MATHMET PTB Berln nd June 8
9 MATHMET PTB Berln nd June Rank transformaton If the lnear hypothess s not vald whle one assume there are f monotonc relatonshps, the rank transformaton can be performed to lnearze the relatonshps The usual regresson and correlaton procedures are performed on these ranks The ranktransformed data provde others senstvty ndces, SRRC Standardzed Rank regresson coeffcents 3 5 4 4 z 4 4 3 3 r r z ( ) ( )( ) ( ) ( ) = = = = n y y n n y y y r r r r r r r r r r SRRC ;
GUM S. Eample 9.3 : mass calbraton Measurement equaton 5 nput varables Y = ( mrc dmrc) + ( rhoa ) rhow rhor Varable mrc PDF N( ;.5) dmrc N(.34;.) a U[ ; 3 ] rhow U[7 ;9 ] Senstvty ndces % of V(Y) rhor U[7 95 ;8 5 ] Varable mrc dmrc a rhow LPU st order 86 4 LPU nd order 44 7 OAT GUM S 86 4 Rank 86 4 rhor Interacton a rhow 48 Interacton a rhor MATHMET PTB Berln nd June
Measures of mportance Decomposton of the varance of Y: ( Y) = V( E[ Y X ]) EV [ ( Y )] V + X Varance of the condtonal epectaton of Y, sutable measure of the mportance of : V =V E Y Frst order senstvty ndces As all nput are assumed to be ndependent V V... k = V = V ( E[ Y X, X ] ) V ( E[ Y X, X, X ]) p k V V V k V k V V... p = V( Y) V V Vk = Leads to second order senstvty ndces, sutable measures of the mportance of the nteractons, V S = V Y MATHMET PTB Berln nd June S V = V ( [ ]) [ X ] X V k ( E Y ) V ( Y) ( )
The method of Sobol If S =, the model s addtve The VCE needs computaton of many ntegrals, too comple to estmate Idea : to transform the many ntegrals (dmenson k) n the ntegrals of the products of f() and f( ) (dmenson k) Performng N Monte Carlo trals, ths means to compute : MATHMET PTB Berln nd June
Results of Sobol method SENSIBILITY INDICES (% of V(Y)) Varable LPU LPU OAT st order nd order GUM S mrc 86 44 86 dmrc 4 7 4 rhoa rhow rhor Interacton rhoa rhow 48 Interacton rhoa rhor Rank 86 4 Sobol 44 6 49 Frst order ndces S Second order ndces S model runs to obtan a sutable confdence nterval of the senstvty ndces! MATHMET PTB Berln nd June 3
X Morrs Desgn /4 Many nput varables, bg computatonal tme, comple model Dscretzaton of the nput varable space X MATHMET PTB Berln nd June 4
Morrs Desgn /4 X p nput varables mples p+ eperments P OAT (OneatATme) P3 P X MATHMET PTB Berln nd June 5
Morrs Desgn 3/4 X P P3 P Computng elementary effect for each nput varable X MATHMET PTB Berln nd June 6
Morrs Desgn 4/4 X 4 3 The desgn s repeated R tmes In total : R*(p+) eperments Ths leads to R samples for each elementary effect: 5 X Measures of senstvty Epectaton of the effect std of the effect MATHMET PTB Berln nd June 7
Results of Morrs method Only 6 model runs! Three Groups of varables:. Neglgble effect. Lnear effect 3. Nonlnear effect or/and nteracton MATHMET PTB Berln nd June mrc dmrc rhoa rhow rhor µ*.6.35.39.49.3 s 5. 7 3. 7.48.34. 8
Local polynomal estmaton for senstvty analyss Consderng the random vector (X,Y), we wrte the regresson model : where m( ) = E Y X= and σ( VarYX ) = = Local polynomal fttng conssts n appromatng locally the regresson functon m by a pth order polynomal : p m( z) = β ( ) ( ) Y = m X+σ X ε ( ) = ( z ) for z n a neghbourhood of If mx ˆ s the estmator of the local polynomal condtonal epectaton of Y, an estmator of VEYX = s gven by : n ( ( ) ˆ ), ˆ n ( ), n = = Tˆ S = V( Y) Tˆ = m Xˆ m m = m ˆ X n Then the frst order senstvty analyss s : MATHMET PTB Berln nd June 9
Scatterplot and condtonal epectaton mrc dmrc rhoa rhow rhor MATHMET PTB Berln nd June
Results of local polynomal based method Frst order Indces model runs Second order Indces MATHMET PTB Berln nd June
Summary of senstvty analyss results : Mass calbraton Non lnear model : Y = ( mrc dmrc) + ( rhoa ) rhow rhor Varable mrc dmrc rhoa rhow SENSIBILITY INDICES (% of V(Y)) LPU LPU OAT st order nd order GUM S Rank 86 44 86 86 4 7 4 4 Sobol 44 6 Local Polynomal 46 7 rhor Interacton rhoa rhow 48 49 49 Interacton rhoa rhor Three nput varables to consder n the uncertanty budget : mrc, rhoa, rhow MATHMET PTB Berln nd June
Propagaton of prmary fre source n a room MATHMET PTB Berln nd June 3
Propagaton of prmary fre source n a room Three measurands to determne f the evacuaton ways can reman practcable n the presence of a fre source: Mamal upper layer temperature Mamal lower layer temperature Mnmal layer heght The model to nvestgate s a code called CFAST wth a costly computatonal tme More than nput varables: Dmensons of the room (area, heght,..) Openngs (Heght, wdth) Insde and outsde temperatures Propertes of the fre (Floor area and surface load) MATHMET PTB Berln nd June 4
Fng some nput varables : Morrs method Mamal upper layer temperature Mamal lower layer temperature Morrs desgn MATHMET leads PTB to Berln keep 8 nd June nput varables n the model 5
Senstvty analyss results : frst order ndces Mamal lower layer temperature Mamal upper layer temperature Tet Tnt S H Har War Af Qf Rank 7 7 5 Local Polynomals 6 3 5 Tet Tnt S H Har War Af Qf Rank 8 33 9 3 5 Local Polynomals 7 8 8 9 Total : 86 84 The frst order senstvty ndces eplan 85% of the varance of the measurand Few nteractons n the model MATHMET PTB Berln nd June 6
Concluson Decson tree for the choce of the approprate framework to perform senstvty analyss YES Lnear model? Senstvty ndces Partal dervatves Correlaton coeffcent Regresson coeffcent Rank correlaton Rank regresson YES YES NO Monotonc model? No nteracton effects? NO Computatonal cost? NO Sobol s Measures of mportance Morrs desgn Local polynomal estmaton MATHMET PTB Berln nd June 7
Selected references I.M. Sobol, Senstvty Estmates for Nonlnear Mathematcal Models, Mathematcal Modelng and Computatonal Eperments (993) A.Saltell, K.Chan, E.M.Scott, Senstvty Analyss, Wley () J. Jacques, C. Lavergne, N. Devctor, Senstvty analyss n presence of model uncertanty and correlated nputs, n Proceedngs of SAMO4 (4) S. Da Vega, F. Wahl, F. Gamboa, Local Polynomal Estmaton for Senstvty Analyss on Models Wth Correlated Inputs (8) R Package «senstvty» : http://cran.cct.fr/src/contrb/descrptons/senstvty.html Submtted: A. Allard, N. Fscher, F. Ddeu, B. Iooss, Applcaton of Global Senstvty Analyss n Measurement Scence, Journal of the French Socety of Statstcs, Specal ssue on Stochastc methods for Senstvty Analyss MATHMET PTB Berln nd June 8
Thank you for your attenton MATHMET PTB Berln nd June 9