Introduction to Uncertainty Quantification in Computational Science Handout #1

Transcription

1 Introduction to Uncertainty Quantification in Computational Science Handout #1 Gianluca Iaccarino Department of Mechanical Engineering Stanford University June 29 - July 1, 2009 Scuola di Dottorato di Ricerca in Ingegneria Industriale Dottorato di Ricerca in Ingegneria Aerospaziale, Navale e della Qualita Universita di Napoli Federico II Based on the Lecture Notes for ME470 prepared with Dr. A. Doostan

2 Objectives of the Lectures Introduce Uncertainty Quantification (UQ) Definitions and motivations Classification of various techniques Application examples

3 Objectives of the Lectures Introduce Uncertainty Quantification (UQ) Definitions and motivations Classification of various techniques Application examples Characterize mysterious concepts such as: polynomial chaos stochastic collocation surrogate sampling epistemic uncertainty...

4 Objectives of the Lectures Introduce Uncertainty Quantification (UQ) Definitions and motivations Classification of various techniques Application examples Characterize mysterious concepts such as: polynomial chaos stochastic collocation surrogate sampling epistemic uncertainty... Identify the research areas and the connections to other fields and convey the challenges and the opportunity in UQ

5 Topics

6 Topics Introduction and definitions Types of uncertainties Errors vs. uncertainties Sensitivity analysis

7 Topics Introduction and definitions Types of uncertainties Errors vs. uncertainties Sensitivity analysis Theoretical framework Probability theory Other approaches: Intervals, evidence

8 Topics Introduction and definitions Types of uncertainties Errors vs. uncertainties Sensitivity analysis Theoretical framework Probability theory Other approaches: Intervals, evidence Data representation Estimate of mean, variance, moments

9 Topics Introduction and definitions Types of uncertainties Errors vs. uncertainties Sensitivity analysis Theoretical framework Probability theory Other approaches: Intervals, evidence Data representation Estimate of mean, variance, moments Uncertainty propagation Sampling methods: MC, LHS Surrogate response: collocation Polynomial chaos methods

10 Topics Introduction and definitions Types of uncertainties Errors vs. uncertainties Sensitivity analysis Theoretical framework Probability theory Other approaches: Intervals, evidence Data representation Estimate of mean, variance, moments Uncertainty propagation Sampling methods: MC, LHS Surrogate response: collocation Polynomial chaos methods

11 Why Uncertainty Quantification?

12 Why Uncertainty Quantification? UQ for Decision Making In spite of the wide spread use of Modeling and Simulation (M&S) tools it remains difficult to provide objective confidence levels in the quantitative information obtained from numerical predictions The complexity arises from the uncertainties related to the inputs of any computation attempting to represent a real physical system. Use of M&S predictions in high-impact decisions require a rigorous evaluation of the confidence

13 Why Uncertainty Quantification? UQ for Decision Making Example: hurricane forecasting Image from NOAA

14 Why Uncertainty Quantification? UQ for Decision Making Example: Yucca Mountain Nuclear Waste Repository Image from Peterson, Kastemberg, Corradini, 2004

15 Observations Predictions are strongly subject to uncertainty Forecasts are typically presented as probabilities of outcomes: 20% chance of rain Whether prediction models are continuously reset: extensive observation/measurements are used to calibrate the models and to define initial conditions; on the other hand radiation level models are based on sparse data Several competing models of the important physical processes are typically available

16 Observations Predictions are strongly subject to uncertainty Forecasts are typically presented as probabilities of outcomes: 20% chance of rain Whether prediction models are continuously reset: extensive observation/measurements are used to calibrate the models and to define initial conditions; on the other hand radiation level models are based on sparse data Several competing models of the important physical processes are typically available Keywords: Data rich/sparse environments, calibration, probabilities, models

17 Why Uncertainty Quantification? UQ for Validation The accepted process of evaluating M&S tools and solutions is based on the general concept of Verification and Validation (V&V) The last step of the process is invariably based on comparisons between numerical predictions and physical observations Precise quantification of the errors and uncertainties is required to establish predictive capabilities: UQ is a key ingredient of validation!

18 Why Uncertainty Quantification? UQ for Validation Example: measurements of the speed of light ( ) Image from Christie et al., Los Alamos Science, #29, 2005

19 Why Uncertainty Quantification? UQ for Validation Comparisons of measurements and numerical predictions must include the quantification of the uncertainty

20 Why Uncertainty Quantification? UQ for Validation Comparisons of measurements and numerical predictions must include the quantification of the uncertainty

21 Why Uncertainty Quantification? UQ for Validation Comparisons of measurements and numerical predictions must include the quantification of the uncertainty...but also the precise notion of validated model must be discussed Images from Trucano et al. 2002, and Romero 2008

22 Observations Experimentalists are forced to provide a measure of the uncertainty in their data! Computationalists (???) will be soon... Comparing two uncertain quantities is not easy? what are the metrics? Only few quantities are measures/computed what can we say about other quantities? Measurements from different experimental campaign can be inconsistent, contradictory, etc. Experiments might only cover a limited parameter space (e.g. Re, Mach, etc.): the validation domain. What if the application domain is larger?

23 Observations Experimentalists are forced to provide a measure of the uncertainty in their data! Computationalists (???) will be soon... Comparing two uncertain quantities is not easy? what are the metrics? Only few quantities are measures/computed what can we say about other quantities? Measurements from different experimental campaign can be inconsistent, contradictory, etc. Experiments might only cover a limited parameter space (e.g. Re, Mach, etc.): the validation domain. What if the application domain is larger? Keywords: validation metrics, validation domain, inconsistent data

24 Why Uncertainty Quantification? UQ for Design/Optimization A robust design is one where system performance remains relatively unchanged (stable) when exposed to uncertainties in the operating conditions. Robust design/optimization is a powerful tool for managing the tradeoffs between optimal performance and performance stability. A reliability-based design is one where the probability of failure is less than some acceptable value

25 Why Uncertainty Quantification? UQ for Design/Optimization Example: transonic wing shape optimization Choice of design conditions can dramatically affect performance Impact of flight conditions uncertainties can lead to unknown/unexpected consequences Image from T. Zang, 2003

26 Why Uncertainty Quantification? One more... Slide from M. Anderson, LANL

27 Why Uncertainty Quantification? Simplistic View: Error bars on the numerical results, just like experiments... The results provide an intuitive notion of confidence Unsteady turbulent heat convection with uncertain wall heating Constantine & Iaccarino, 2009

28 Why Uncertainty Quantification? Simplistic View: Error bars on the numerical results, just like experiments... The results provide an intuitive notion of confidence Unsteady turbulent heat convection with uncertain wall heating Constantine & Iaccarino, 2009 One important objective of UQ is to make the intuitive notion of confidence mathematically sound!

29 Definitions

30 Verification and Validation Definitions The American Institute for Aeronautics and Astronautics (AIAA) has developed the Guide for the Verification and Validation (V&V) of Computational Fluid Dynamics Simulations (1998) What is V&V? Verification: The process of determining that a model implementation accurately represents the developer s conceptual description of the model. Validation The process of determining the degree to which a model is an accurate representation of the real world for the intended uses of the model

31 Errors vs. Uncertainties Definitions The AIAA Guide for the Verification and Validation (V&V) of CFD Simulations (1998) defines errors as recognisable deficiencies of the models or the algorithms employed uncertainties as a potential deficiency that is due to lack of knowledge.

32 Errors vs. Uncertainties Definitions The AIAA Guide for the Verification and Validation (V&V) of CFD Simulations (1998) defines errors as recognisable deficiencies of the models or the algorithms employed uncertainties as a potential deficiency that is due to lack of knowledge. What is the relation with V&V?

33 Errors vs. Uncertainties Definitions The AIAA Guide for the Verification and Validation (V&V) of CFD Simulations (1998) defines errors as recognisable deficiencies of the models or the algorithms employed uncertainties as a potential deficiency that is due to lack of knowledge. What is the relation with V&V? Verification aims at answering the question are we solving the equations correctly? it is an exercise in mathematics

34 Errors vs. Uncertainties Definitions The AIAA Guide for the Verification and Validation (V&V) of CFD Simulations (1998) defines errors as recognisable deficiencies of the models or the algorithms employed uncertainties as a potential deficiency that is due to lack of knowledge. What is the relation with V&V? Verification aims at answering the question are we solving the equations correctly? it is an exercise in mathematics Validation aims at answering the question are we solving the correct equations? it is an exercise in physics

35 Errors vs. Uncertainties Definitions The AIAA definition is not very precise: it does not clearly distinguish between the mathematics and the physics.

36 Errors vs. Uncertainties Definitions The AIAA definition is not very precise: it does not clearly distinguish between the mathematics and the physics. What are errors? errors are associated to the translation of a mathematical formulation into a numerical algorithm and a computational code. Acknowledged errors are known to be present but their effect on the results is deemed negligible (round-off, limited convergence of iterative algorithms) Unacknowledged errors are not recognizable but might be present, e.g. implementation mistakes (bugs).

37 Errors vs. Uncertainties Definitions The AIAA definition is not very precise: it does not clearly distinguish between the mathematics and the physics. What are errors? errors are associated to the translation of a mathematical formulation into a numerical algorithm and a computational code. Acknowledged errors are known to be present but their effect on the results is deemed negligible (round-off, limited convergence of iterative algorithms) Unacknowledged errors are not recognizable but might be present, e.g. implementation mistakes (bugs). What are uncertainties? uncertainties are associated to the specification of the input physical parameters required for performing the analysis.

38 Errors vs. Uncertainties Definitions The AIAA definition is not very precise: it does not clearly distinguish between the mathematics and the physics. What are errors? errors are associated to the translation of a mathematical formulation into a numerical algorithm and a computational code. Acknowledged errors are known to be present but their effect on the results is deemed negligible (round-off, limited convergence of iterative algorithms) Unacknowledged errors are not recognizable but might be present, e.g. implementation mistakes (bugs). What are uncertainties? uncertainties are associated to the specification of the input physical parameters required for performing the analysis. "...an uncertain input parameter will lead not only to an uncertain solution but to an uncertain solution error as well.." [Trucano, 2004]

39 Uncertainties Aleatory: it is the physical variability present in the system or its environment. It is not strictly due to a lack of knowledge and cannot be reduced (also referred to as variability, stochastic uncertainty or irreducible uncertainty)

40 Uncertainties Aleatory: it is the physical variability present in the system or its environment. It is not strictly due to a lack of knowledge and cannot be reduced (also referred to as variability, stochastic uncertainty or irreducible uncertainty) It is naturally defined in a probabilistic framework Examples are: material properties, operating conditions manufacturing tolerances, etc.

41 Uncertainties Aleatory: it is the physical variability present in the system or its environment. It is not strictly due to a lack of knowledge and cannot be reduced (also referred to as variability, stochastic uncertainty or irreducible uncertainty) It is naturally defined in a probabilistic framework Examples are: material properties, operating conditions manufacturing tolerances, etc. In mathematical modeling it is also studied as noise

42 Aleatory Uncertainty Manufacturing process Courtesy of NASA

43 Aleatory Uncertainty Flight conditions Difference between measured (balloon) and expected (Global Reference Atmospheric Model) density in the earth atmosphere Image from Desai et al. 2003

44 Uncertainties Epistemic: it is a potential deficiency that is due to a lack of knowledge It can arise from assumptions introduced in the derivation of the mathematical model (it is also called reducible uncertainty or incertitude) Examples are: turbulence model assumptions or surrogate chemical models

45 Uncertainties Epistemic: it is a potential deficiency that is due to a lack of knowledge It can arise from assumptions introduced in the derivation of the mathematical model (it is also called reducible uncertainty or incertitude) Examples are: turbulence model assumptions or surrogate chemical models It is NOT naturally defined in a probabilistic framework

46 Uncertainties Epistemic: it is a potential deficiency that is due to a lack of knowledge It can arise from assumptions introduced in the derivation of the mathematical model (it is also called reducible uncertainty or incertitude) Examples are: turbulence model assumptions or surrogate chemical models It is NOT naturally defined in a probabilistic framework Can lead to strong bias of the predictions

47 Epistemic Uncertainty Model uncertainty CBO Predictions of deficit as a percentage of GDP Source: Congressional Budget Office.

48 Epistemic Uncertainty Model uncertainty Predictions of surface pressure over a transonic bump

49 Reducible vs. Irreducible Uncertainty Epistemic uncertainty can be reduced by increasing our knowledge, e.g. performing more experimental investigations and/or developing new physical models. Aleatory uncertainty cannot be reduced as it arises naturally from observations of the system. Additional experiments can only be used to better characterize the variability.

50 Sensitivity Analysis vs. UQ Sensitivity analysis (SA) investigates the connection between inputs and outputs of a (computational) model The objective of SA is to identify how the variability in an output quantity of interest (q) is connected to an input (ξ) in the model; the result is a sensitivity derivative q/ ξ SA allows to build a ranking of the input sources which might dominate the response of the system

51 Sensitivity Analysis vs. UQ Sensitivity analysis (SA) investigates the connection between inputs and outputs of a (computational) model The objective of SA is to identify how the variability in an output quantity of interest (q) is connected to an input (ξ) in the model; the result is a sensitivity derivative q/ ξ SA allows to build a ranking of the input sources which might dominate the response of the system Note that strong (large) sensitivities derivatives do not necessarily translate in critical uncertainties because the input variability might be very small in a specific device of interest. q q ξ ξ

52 Predictive Scientific Computing = Computations Under Uncertainty

53 Uncertainty Quantification Computational Framework Consider a generic computational model (y R d with d large)

54 Uncertainty Quantification Computational Framework Consider a generic computational model (y R d with d large) How do we handle the uncertainties? 1. Data assimilation: characterize uncertainties in the inputs 2. Uncertainty propagation: perform simulations accounting for the identified uncertainties 3. Certification: establish acceptance criteria for predictions

55 Uncertainty Quantification Computational Framework Consider a generic computational model (y R d with d large) How do we handle the uncertainties? 1. Data assimilation: characterize uncertainties in the inputs 2. Uncertainty propagation: perform simulations accounting for the identified uncertainties 3. Certification: establish acceptance criteria for predictions

56 Uncertainty Quantification Computational Framework Consider a generic computational model (y R d with d large) How do we handle the uncertainties? 1. Data assimilation: characterize uncertainties in the inputs 2. Uncertainty propagation: perform simulations accounting for the identified uncertainties 3. Certification: establish acceptance criteria for predictions

57 Uncertainty Quantification Computational Framework Consider a generic computational model (y R d with d large) How do we handle the uncertainties? 1. Data assimilation: characterize uncertainties in the inputs 2. Uncertainty propagation: perform simulations accounting for the identified uncertainties 3. Certification: establish acceptance criteria for predictions

58 Data assimilation The objective is characterize uncertainties in simulation inputs, based on available information

59 Data assimilation The objective is characterize uncertainties in simulation inputs, based on available information Data sources - Experimental observations - Theoretical arguments - Expert opinions - etc.

60 Data assimilation The objective is characterize uncertainties in simulation inputs, based on available information Data sources - Experimental observations - Theoretical arguments - Expert opinions - etc. Mathematical framework for data assimilation Methods are a function of the amount of data available - Data-rich environments: e.g. stock market - Sparse data environments: e.g. planetary exploration

61 Data assimilation The objective is characterize uncertainties in simulation inputs, based on available information Data sources - Experimental observations - Theoretical arguments - Expert opinions - etc. Mathematical framework for data assimilation Methods are a function of the amount of data available - Data-rich environments: e.g. stock market - Sparse data environments: e.g. planetary exploration inference: determination of the statistical input parameters that best represent the available information

62 Data assimilation What is the end result of this phase?

63 Data assimilation What is the end result of this phase? Identification of all (d) the explicit and hidden parameters (knobs) of the mathematical/computational model: y

64 Data assimilation What is the end result of this phase? Identification of all (d) the explicit and hidden parameters (knobs) of the mathematical/computational model: y Characterization of the associated level of knowledge

65 Data assimilation What is the end result of this phase? Identification of all (d) the explicit and hidden parameters (knobs) of the mathematical/computational model: y Characterization of the associated level of knowledge The mathematical framework for propagating uncertainties is dependent on the data representation chosen Probabilistic propagation Interval analysis Fuzzy logic

66 Uncertainty Propagation Perform simulations accounting for the identified uncertainties Number of choices:

67 Uncertainty Propagation Perform simulations accounting for the identified uncertainties Number of choices: Probabilistic vs. Non-probabilistic framework Intrusive vs. Non-Intrusive methodology Based on Full model vs. Surrogate... It is difficult to build a proper taxonomy of methods

68 Probabilistic Uncertainty Propagation Perform simulations accounting for the uncertainty represented as randomness Define an abstract probability space (Ω, A, P) Introduce uncertain input as random quantities y(ω), ω Ω

69 Probabilistic Uncertainty Propagation Perform simulations accounting for the uncertainty represented as randomness Define an abstract probability space (Ω, A, P) Introduce uncertain input as random quantities y(ω), ω Ω The original problem becomes stochastic with solution u(ω) u(y(ω))

70 Probabilistic Uncertainty Propagation Perform simulations accounting for the uncertainty represented as randomness Define an abstract probability space (Ω, A, P) Introduce uncertain input as random quantities y(ω), ω Ω The original problem becomes stochastic with solution u(ω) u(y(ω))

71 Probabilistic Uncertainty Propagation Perform simulations accounting for the uncertainty represented as randomness Define an abstract probability space (Ω, A, P) Introduce uncertain input as random quantities y(ω), ω Ω The original problem becomes stochastic with solution u(ω) u(y(ω)) Remark: y can affect the boundary conditions, the geometry, the forcing terms or the operator in the computational model.

72 Uncertainty Propagation Intrusive vs. Non-Intrusive Methodology Intrusiveness is defined with respect to the original (deterministic) mathematical model, not the physical description of the problem

73 Uncertainty Propagation Intrusive vs. Non-Intrusive Methodology Intrusiveness is defined with respect to the original (deterministic) mathematical model, not the physical description of the problem Nonintrusive methods only require (multiple) solutions of the original model

74 Uncertainty Propagation Intrusive vs. Non-Intrusive Methodology Intrusiveness is defined with respect to the original (deterministic) mathematical model, not the physical description of the problem Nonintrusive methods only require (multiple) solutions of the original model Intrusive methods require the formulation and solution of a stochastic version of the original model

75 Example of intrusive methods Perturbation methods Consider a very simple mathematical model to study 1D non-linear flow motion (Burgers equation): u t + u u x = 0 x [0 : 1] with uncertain initial conditions u(x, t = 0) = u 0 (ω)

76 Example of intrusive methods Perturbation methods Consider a very simple mathematical model to study 1D non-linear flow motion (Burgers equation): u t + u u x = 0 x [0 : 1] with uncertain initial conditions u(x, t = 0) = u 0 (ω) We seek the solution u(x, t, ω)

77 Example of intrusive methods Perturbation methods Consider a very simple mathematical model to study 1D non-linear flow motion (Burgers equation): u t + u u x = 0 x [0 : 1] with uncertain initial conditions u(x, t = 0) = u 0 (ω) We seek the solution u(x, t, ω) Define a base state u(x, t) and write u(x, t, ω) u + δu

78 Example of intrusive methods Perturbation methods Consider a very simple mathematical model to study 1D non-linear flow motion (Burgers equation): u t + u u x = 0 x [0 : 1] with uncertain initial conditions u(x, t = 0) = u 0 (ω) We seek the solution u(x, t, ω) Define a base state u(x, t) and write u(x, t, ω) u + δu Plug in the original equation to obtain: ( given δu t + u δu x + δu u x = 0 u t + u u ) x = 0

79 Example of intrusive methods Perturbation methods Consider a very simple mathematical model to study 1D non-linear flow motion (Burgers equation): u t + u u x = 0 x [0 : 1] with uncertain initial conditions u(x, t = 0) = u 0 (ω) We seek the solution u(x, t, ω) Define a base state u(x, t) and write u(x, t, ω) u + δu Plug in the original equation to obtain: ( given δu t + u δu x + δu u x = 0 The key assumption is δu 2 0 u t + u u ) x = 0

80 Example of intrusive methods Perturbation methods Consider a very simple mathematical model to study 1D non-linear flow motion (Burgers equation): u t + u u x = 0 x [0 : 1] with uncertain initial conditions u(x, t = 0) = u 0 (ω) We seek the solution u(x, t, ω) Define a base state u(x, t) and write u(x, t, ω) u + δu Plug in the original equation to obtain: ( given δu t + u δu x + δu u x = 0 The key assumption is δu 2 0 u t + u u ) x = 0 The method implies the original problem for the base state and a new, linear problem for the perturbation

81 Why intrusive methods? Perturbation methods are clearly strongly limited, but the assumption of small variability can be lifted Intrusive methods can be strongly customized, highly effective, and provide a connection to the physics/modeling of the problem

82 Why intrusive methods? Perturbation methods are clearly strongly limited, but the assumption of small variability can be lifted Intrusive methods can be strongly customized, highly effective, and provide a connection to the physics/modeling of the problem For realistic engineering problems intrusive methods remain challenging to implement, expensive to use and difficult to verify

83 Why intrusive methods? Perturbation methods are clearly strongly limited, but the assumption of small variability can be lifted Intrusive methods can be strongly customized, highly effective, and provide a connection to the physics/modeling of the problem For realistic engineering problems intrusive methods remain challenging to implement, expensive to use and difficult to verify Answering the question "to intrude or not to intrude" remains a key research area and likely a case-by-case choice.

84 Uncertainty Propagation Full model vs. Surrogate Surrogates or Reduced Order Models (ROM) are based on a simplification of the original mathematical problem

85 Uncertainty Propagation Full model vs. Surrogate Surrogates or Reduced Order Models (ROM) are based on a simplification of the original mathematical problem Examples of ROMs: Coarse grids Subset of the physical phenomena Curve fit of the output quantity of interest (response surface)...

86 Uncertainty Propagation Full model vs. Surrogate Surrogates or Reduced Order Models (ROM) are based on a simplification of the original mathematical problem Examples of ROMs: Coarse grids Subset of the physical phenomena Curve fit of the output quantity of interest (response surface)... Remark: the use of ROMs is only justified by the cost of UQ for the full system.

87 Certification The last step in UQ is also the most important: Quantification of the confidence for decision making Objective analysis of the validity of a physical/mathematical model...

88 Certification The last step in UQ is also the most important: Quantification of the confidence for decision making Objective analysis of the validity of a physical/mathematical model... How to report the wealth of information gathered through a UQ process? For example in a probabilistic framework we can use boxplots

89 Certification The last step in UQ is also the most important: Quantification of the confidence for decision making Objective analysis of the validity of a physical/mathematical model... How to report the wealth of information gathered through a UQ process? For example in a probabilistic framework we can use boxplots In a decision-making context we need to provide a simple and clear summary.

90 Quantification of Margins and Uncertainty: QMU The concept of QMU has been introduced to facilitate communication of numerical results: for a quantity of interest (q) define a performance gate (acceptable operation) predict q for the design conditions (best estimate) evaluate the margin perform an uncertainty analysis define the Confidence M/U From Sharp & Wood-Shultz, 2003

91 Quantification of Margins and Uncertainty: QMU The concept of QMU has been introduced to facilitate communication of numerical results: for a quantity of interest (q) define a performance gate (acceptable operation) predict q for the design conditions (best estimate) evaluate the margin perform an uncertainty analysis define the Confidence M/U From Sharp & Wood-Shultz, 2003 Remark: in general both the predictions and the bounds are uncertain.

92 Example: design a vehicle to explore Titan

93 Planet exploration vehicle The exploration of Saturn s moon Titan was one of the objective of the Huygens mission by NASA and ESA During descent in the atmosphere vehicles experience extreme heating loads The design of the thermal protection system (TPS) is the most critical component of every planetary entry mission TPS design is fundamentally computation-based because no ground-test can reproduce all the aspects of flight Safety (and reliability) requires rigorous evaluation of the uncertainties

94 Planetary Exploration Vehicles Predictions of TPS heating loads during re-entry are challenging Physics Components - Chemistry - Radiation - Turbulence - etc. Computational issues - Strong shocks - Thin boundary layers - Flow separation - etc. In addition, CFD simulations require precise input parameters, but the real vehicle is subject to uncertain conditions...

95 Uncertainty sources (aleatory) Predicting heating rates Imprecise characterization of the environment - speed of sound - angle of attack - temperature fluctuations - gas composition - etc. Static characterization of the vehicle components - structure and material imperfections/inhomogeneity - surface rougheness and out-of-spec geometry - etc. Thermal-fluid processes during operation - ablation rates - surface catalysis - etc.

96 Uncertainty sources (epistemic) Predicting heating rates Physical modeling - High temperature gas properties - dissociation, ionization - Radiation - Transition to turbulence - Surface chemistry - etc. Missing physics (unknown unknowns) - Non-continuum effects - Thermal distortion of the structure - Charring of the ablating material - etc.

97 Probabilistic approach Reaction rates in combustion models Chemical kinetics: 100s of species and 1000s of reactions Uncertainty in the reactions rates, gathered from theory experiments engineering judgment Uncertainty in the reaction rates is described using Gaussian independent variables [ P(k r ) = exp 1 ( ) ] log10 k r /E[k r ] 2 2 σ r From Bose & Wright, 2004

98 Uncertainty analysis The analysis is carried out using Monte Carlo sampling ( 300r.v.) and 4000 simulations (samples) are required to achieve statistical convergence. From Bose & Wright, 2004

99 Primary Uncertainty NASA [Bose & Wright, 2004] has identified the heating from shock layer radiation due to the CN radical formed in the N 2 /CH 4 atmosphere as the primary uncertainty

100 Primary Uncertainty NASA [Bose & Wright, 2004] has identified the heating from shock layer radiation due to the CN radical formed in the N 2 /CH 4 atmosphere as the primary uncertainty Can we predict the CN radical?

101 Primary Uncertainty NASA [Bose & Wright, 2004] has identified the heating from shock layer radiation due to the CN radical formed in the N 2 /CH 4 atmosphere as the primary uncertainty Can we predict the CN radical? State-of-the art knowledge during the design of the Huygen s probe was the Boltzmann model This led to overprediction of the heating rates - conservative design Stimulated work on collisional-radiative (CR) models and gathering of new experimental data From Magin et al., 2007

102 Primary Uncertainty We repeated the evaluation of the TPS heating loads using the CR and the Boltzmann model for radiation in an effort to characterize the epistemic uncertainty

103 Primary Uncertainty We repeated the evaluation of the TPS heating loads using the CR and the Boltzmann model for radiation in an effort to characterize the epistemic uncertainty To first order the results are consistent Maximum heat flux CR Model: Ghaffari, Iaccarino, Magin, 2009 Boltzmann model: Bose & Wright, 2004

104 Primary Uncertainty Ranking on the uncertainty sources is based on correlation plots of output (amount of CN) vs. input (uncertainty in the reaction rates)

105 Primary Uncertainty Ranking on the uncertainty sources is based on correlation plots of output (amount of CN) vs. input (uncertainty in the reaction rates) Correlation plot for N 2 + C CN + N 2 CR Model: Ghaffari, Iaccarino, Magin, 2009 Boltzmann model: Bose & Wright, 2004

106 Primary Uncertainty The decision-making might be affected: Ranking on the uncertainty sources is potentially changed by the new physical model 8 major contributors to uncertainty CR Model: Ghaffari, Iaccarino, Magin, 2009 Boltzmann model: Bose & Wright, 2004

107 Planetary Exploration Vehicles Uncertainty analysis Take home messages: Quantification of the effect of input uncertainties can provide useful information and ranking of the sources

108 Planetary Exploration Vehicles Uncertainty analysis Take home messages: Quantification of the effect of input uncertainties can provide useful information and ranking of the sources this enables focused research efforts in the areas of high return

109 Planetary Exploration Vehicles Uncertainty analysis Take home messages: Quantification of the effect of input uncertainties can provide useful information and ranking of the sources this enables focused research efforts in the areas of high return Epistemic uncertainty (physical modeling) can be a dominant source of uncertainty and can even affect the ranking

110 Planetary Exploration Vehicles Uncertainty analysis Take home messages: Quantification of the effect of input uncertainties can provide useful information and ranking of the sources this enables focused research efforts in the areas of high return Epistemic uncertainty (physical modeling) can be a dominant source of uncertainty and can even affect the ranking For this problem, a prediction of the stagnation point heat flux can be carried out very efficiently and this enables the use of Monte Carlo (1000s repeated computations)

111 Example: analysis of engine intakes in a Formula 1 car

112 Fomula 1 Vehicles The aerodynamics of Fomula 1 cars is very sophisticated and highly optimized One of the few aspects that cannot be actively modified is the wake of the tires The air intakes are critical to achieve engine performance They might be critically affected by the flow in the tire wakes

113 Formula 1 Vehicles Predictions of the air flow in the engine intakes is challenging Physics Components - Unsteadiness - Complex geometry - Turbulence - etc. Computational issues - Rotating tires and flow in brake ducts - Tire contact patch - Flow separation - etc. As for the previous example, CFD simulations require precise input parameters, but the real vehicle is subject to uncertain conditions...

114 Uncertainty sources Intake air flow Imprecise characterization of the environment - Incoming disturbances from preceding cars - Tire deformation - Speed - etc. Physical models - turbulence models - multiple rotating frame of references - etc.

115 Uncertainty sources Intake air flow The geometry of the tire is uncertain Aleatory or epistemic?

116 Uncertainty sources Intake air flow The geometry of the tire is uncertain Aleatory or epistemic? We could reduce it by measuring it??

117 Uncertainty sources Intake air flow The geometry of the tire is uncertain Aleatory or epistemic? We could reduce it by measuring it?? or predict the deformation using a model!

118 Uncertainty sources Intake air flow The geometry of the tire is uncertain Aleatory or epistemic? We could reduce it by measuring it?? or predict the deformation using a model! In this case it is likely that we might introduce more uncertainties, e.g. elastic property of the rubber, tire/road slippage, etc.

119 Uncertainty sources Intake air flow The geometry of the tire is uncertain Aleatory or epistemic? We could reduce it by measuring it?? or predict the deformation using a model! In this case it is likely that we might introduce more uncertainties, e.g. elastic property of the rubber, tire/road slippage, etc. How many degrees of freedom do we need to include to describe the generic shape?

120 Uncertainty sources Uncertainty representation The geometry of the tire is uncertain! We choose to represent it using few (3) parameters

121 Uncertainty sources Uncertainty representation The geometry of the tire is uncertain! Experimental "measure" We choose to represent it using few (3) parameters The justification was a single measurement of the contact patch for a stationary tire

122 Uncertainty sources Uncertainty representation The geometry of the tire is uncertain! Experimental "measure" We choose to represent it using few (3) parameters The justification was a single measurement of the contact patch for a stationary tire

123 Engine Intake Analysis Uncertainty propagation We used a response surface approach based on stochastic collocation

124 Engine Intake Analysis Uncertainty propagation We used a response surface approach based on stochastic collocation We can now estimate the standard deviation of the entire flow field with respect to the assumed variability in the input, e.g. the three parameters representing the tire deformation

125 Engine Intake Analysis Uncertainty propagation We can also estimate the probability of the air intake to provide a certain amount of flow rate to the engine

126 Engine Intake Analysis Uncertainty propagation We can also estimate the probability of the air intake to provide a certain amount of flow rate to the engine We can evaluate the probability of failure associate to the uncertainty in the tire deformation

127 Formula 1 Vehicles Uncertainty analysis Take home messages: Quantification of the effect of input uncertainties can lead to different aerodynamic choices

128 Formula 1 Vehicles Uncertainty analysis Take home messages: Quantification of the effect of input uncertainties can lead to different aerodynamic choices The representation of the uncertainties is an important step: an increase in the number of degrees of freedom considered can lead to exponential growth of the computational effort required

129 Formula 1 Vehicles Uncertainty analysis Take home messages: Quantification of the effect of input uncertainties can lead to different aerodynamic choices The representation of the uncertainties is an important step: an increase in the number of degrees of freedom considered can lead to exponential growth of the computational effort required For typical CFD problems (3D, complex geometry, etc) the cost is prohibitive: 1000s repeated computations are NOT feasible

130 Formula 1 Vehicles Uncertainty analysis Take home messages: Quantification of the effect of input uncertainties can lead to different aerodynamic choices The representation of the uncertainties is an important step: an increase in the number of degrees of freedom considered can lead to exponential growth of the computational effort required For typical CFD problems (3D, complex geometry, etc) the cost is prohibitive: 1000s repeated computations are NOT feasible Changes in the geometry are also associated to changes in the grid resolution and quality. Are we introducing uncertain numerical errors?

131 Example: estimate flight conditions during HyShot flight

132 Validation against flight data Flight data are necessarily one-of-a-kind! Only some quantities can be measured, typically do not provide conclusive evidence. Hyshot is an experimental vehicles to study hypersonic flight and to correlate flight-dat and ground tests The launch occurred in 2002 in Queensland

133 HyShot Vehicle Predictions of the reacting flow in hypersonic vehicles is extremely challenging Physics Components - Hypersonic speed, shocks, high-temperature behavior of the air - Fuel injection, mixing and reactions - Turbulence - etc. Computational issues - Strong shocks - Combustion - Flow separation - etc. To validate our predictions we need to gather the flight data and reproduce the same flow conditions

134 HyShot Vehicle Flight tests During flight pressure and temperature measurement in the combustion chamber condition in the fuel tank allow to build the fuee flow rate as function of time

135 HyShot Vehicle Flight tests During flight pressure and temperature measurement in the combustion chamber condition in the fuel tank allow to build the fuee flow rate as function of time Unfortunately, the radar telemetry did not work and the information about the altitude, speed, etc. were lost

136 HyShot Vehicle Flight tests Problem: given the noisy measurements within the system, can we infer the flight conditions?

137 HyShot Vehicle Flight tests Problem: given the noisy measurements within the system, can we infer the flight conditions? Solution: pose a statistical inverse problem, where the solution is the most probable flight conditions

138 HyShot Vehicle Flight tests Problem: given the noisy measurements within the system, can we infer the flight conditions? Solution: pose a statistical inverse problem, where the solution is the most probable flight conditions Remark: need a model that predicts the flow in the system given the flight condition (forward model).

139 HyShot Vehicle Flight tests Problem: given the noisy measurements within the system, can we infer the flight conditions? Solution: pose a statistical inverse problem, where the solution is the most probable flight conditions Remark: need a model that predicts the flow in the system given the flight condition (forward model). Consider a simplified configuration We have the exact inflow but want to recover them using statistical inversion, we also assume that the measurements are noisy...

140 HyShot Vehicle Simplified configuration Estimate of the inflow Mach number and static pressure using 2 noisy pressure measurements

141 HyShot Vehicle Simplified configuration Estimate of the inflow Mach number and static pressure using 4 noisy pressure measurements

142 HyShot Flight Data Validation analysis Take home messages: Incomplete information from experimental analysis are common, but this might not be a show-stopper

143 HyShot Flight Data Validation analysis Take home messages: Incomplete information from experimental analysis are common, but this might not be a show-stopper The inverse analysis is based on a forward model, so the final estimates are conditioned (dependent) on the model chosen (and its potential inaccuracy)

144 HyShot Flight Data Validation analysis Take home messages: Incomplete information from experimental analysis are common, but this might not be a show-stopper The inverse analysis is based on a forward model, so the final estimates are conditioned (dependent) on the model chosen (and its potential inaccuracy) If all the experimental data is used to infer the operating conditions, nothing is left for validation

145 Further Reading Material 1. T. G. Trucano, Prediction and Uncertainty in Computational Modeling of Complex Phenomena: A Whitepaper, SANDIA Report, SAND T.G. Trucano, L. P. Swiler, T. Igusa, W. L. Oberkampf, M. Pilch, Calibration, Validation and Sensitivity Analysis: What s What, Reliability Engineering and System Safety, V. 91, 2006

146 Basic Notions of Probability Theory

147 The different kinds of probability There are four kinds of probability. A. Probability as intuition Deals with judgements based on intuition. Examples are: She will probably marry him. He probably drove too fast. People buying lottery tickets intuitively believe that certain numbers are more likely to win.

148 The different kinds of probability B. Probability as ratio of favorable to total outcomes (classical theory) This is a non-experimental approach where probability of an event is computed a priori by counting the number of ways N A that an event A can occur among N number of all possible outcomes. Then the probability of event A is P(A) N A N. It is considered that all outcomes are equally probable.

149 The different kinds of probability Example: The die rolling Suppose we throw a pair of fair dice. What is the probability of having a seven? Outcome of throwing two dice P(getting seven) = 6 36 = 1 6.

150 The different kinds of probability C. Probability as a measure of frequency of occurrence This is an experimental approach where an experiment is performed n times. The number of times that event A appears is denoted by n A. Then the probability of A is postulated as Notice that: 0 P(A) 1 n A P(A) = lim n n. We can only estimate P(A) from finite number of trials. Example: One could experimentally estimate the probability of getting seven in the die rolling example.

151 The different kinds of probability D. Probability based on an axiomatic theory In this approach probability is defined based on set, field and event concepts. The probability assignments have to satisfy some constraints called axioms. Modern probability theory is based on this approach

152 Sets, Subsets, and Events Some definitions: Set is a collection of objects. Subset of a set is a collection that is contained within the larger set. Sample space/set of elementary events/certain event is the set of all outcomes of an experiment and is denoted by Ω. Example: The sample space of the experiment of flipping a coin once is Ω = {H, T } Example: The sample space of the experiment of flipping a coin twice is Ω = {HH, HT, TH, TT }. Subsets of Ω are called events. Example: A subset of Ω is {HH, HT, TH} which is the event of getting at least one head in two flips. Remark: If Ω = {ω1,, ω N }, then the number of subsets of Ω is 2 N.

153 Set Algebra Union/Sum of two sets A and B is the set of all elements that are in at least A or B and is denoted by A B. Intersection of two sets A and B is the set of all elements that are common to both A and B and is denoted by A B. Complement of a set A is the set of all elements not in A and is denoted by A c. Notice that, A A c = Ω and A A c = φ, where φ is called the impossible event or the empty set.

154 Fields and Sigma Fields Consider a set Ω and a collection of subsets of Ω. Let A, B,... denote subsets in this collection. This collection of subsets forms a field M if 1. φ M, Ω M 2. If A M and B M, then A B M and A B M 3. If A M, then A c M A sigma (σ)-field F is a field that is closed under any countably infinite set of unions and intersections. Thus if A 1,, A n, belong to F so do i=1 A i and i=1 A i.

155 Probability Space/Probability Triple Definition: A probability space is a triple (Ω, F, P) consisting of the sample space Ω, a σ-field F of subsets of Ω, and a probability function P on (Ω, F). The probability function P : F [0, 1] is defined based on the following three axioms. Axioms of probability: For every event A F, a number P(A) called the probability of A is assigned such that: P(A) 0 P(Ω) = 1 P(A B) = P(A) + P(B) if A B = φ

156 Example - Tossing a Coin Consider the experiment of throwing a coin once. The sample space Ω is Ω = {H, T }. The σ-field of events consists of sets: F = {{H}, {T }, Ω, φ}. For a fair coin, probabilities of events in F are: P({H}) = P({T }) = 1/2, P(Ω) = 1 and P(φ) = 0.

157 Conditional Probabilities and Independence Probability of an event A given that an event B has happened (conditional probability) is denoted by P(A B) and is computed by P(A B) = P(A B), P(B) > 0. P(B) Event A is said to be independent from event B if: Bayes rule: P(A B) = P(A) or P(A B) = P(A)P(B). P(A B) = P(B A)P(A) P(B)

158 Random Variables A random variable X(ω) : Ω R is a function that maps an outcome ω Ω of the random experiment to a number on the real line. Under such mapping an event A B Ω (A B F) is mapped to an interval B on the real line.

159 Probability Distribution Function(PDF/CDF) F X (x) P(X x) P({ω : X(ω) x}) is called the probability distribution function (PDF) of random variable X and has the following properties: F X ( ) = 1, F X ( ) = 0 F X (x) is a nondecreasing function of x, i.e., x 1 x 2 F X (x 1 ) F X (x 2 ) F X (x) is continuous from right, i.e., F X (x) = lim ɛ 0 F X (x + ɛ) ɛ > 0

160 Probability Density Function(pdf) If F X (x) is continuous and differentiable, the probability density function (pdf) is computed as f X (x) df X (x), dx and has the following properties: f X (x) 0 f X (ξ)dξ = F X ( ) F X ( ) = 1 F X (x) = x f X (ξ)dξ = P(X x) F X (x 2 ) F X (x 1 ) = x 2 f X (ξ)dξ x 1 f X (ξ)dξ = x2 x 1 f X (ξ)dξ = P(x 1 < X x 2 )

161 Continuous Random Variables If F X (x) is continuous, then X is said to be a continuous random variable. Examples: Gaussian random variable Uniform random variable

162 Gaussian (Normal) and Uniform Random Variables Gaussian: X N(µ, σ 2 ) f X (x) = 1 2πσ e 1 2 [ x µ σ ]2 Standard normal X N(0, 1) Uniform: X U(a, b) f X (x) = 1 b a I [a,b](x)

163 Mean, Variance, and Moments Mean/Expectation: X E(X) µ(x) X Variance: xf X (x)dx Var(X) σ 2 (X) (X X ) 2 = X 2 X 2 ( ) 2 x 2 f X (x)dx xf X (x)dx K-th moment: m k (X) X k = x k f X (x)dx, k = 1, 2, 3,

164 Functions of a Random Variable Let X be a random variable and g(x) a differentiable function of x, then Y = g(x) is a random variable with F Y (y) = P(Y y) = P(g(X) y). The pdf of Y is computed from f Y (y) = n f X (x i )/ g (x i ) x i = x i (y), g (x i ) 0, i=1 where x i are roots of y = g(x) and g (x i ) dy/dx x=xi. Remark: Y = g(x) = yf Y (y)dy = D Y g(x)f X (x)dx, D X where D Y and D X are the domains of Y and X, respectively.

165 Two Random Variables Let X and Y be two random variables. The joint distribution of X and Y is defined by F XY (x, y) = P(X x, Y y). For a continuous and differentiable F XY (x, y), the joint density of X and Y is defined by f XY (x, y) = Notice that marginal distributions are 2 x y [F XY (x, y)]. F X (x) = F XY (x, ), F Y (y) = F XY (, y).

166 Two Random Variables The marginal densities are obtained from Remark: f X (x) = f Y (y) = X and Y are independent if: or equally, f XY (x, y)dy, f XY (x, y)dx. F XY (x, y) = F X (x)f Y (y) f XY (x, y) = f X (x)f Y (y). X and Y are independent identically distributed (i.i.d.) if they are independent and have the same distribution.

167 Two Random Variables - Correlation and Covariance The covariance C XY of X and Y is defined be C XY (X X )(Y Y ) = XY X Y The correlation coefficient ρ XY of X and Y is defined be with the property that ρ XY 1. Remark: ρ XY C XY σ X σ Y, X and Y are uncorrelated if: C XY = 0 or ρ XY = 0, or equally, XY = X Y. X and Y are orthogonal if XY = 0.

168 Conclusions Random variables are the building block to study uncertainties in a probabilistic framework In general, uncertain quantities cannot be described by a scalar quantities we introduce random vectors, and random fields.

169 Random Processes Figure: Microstructures in a metal (From Professor H. Fujii, JWRI)

170 Random Processes A turbulent flow experiment Figure: Three sample paths of a turbulence velocity component U at a fix location. From: S.B. Pope, Turbulent Flows.

171 Random Processes Earthquake time histories Figure: Acceleration time history of the El Centro 1940 earthquake.

172 White noise v.s. non-white (colored) noise Figure: A sample from a white noise process Figure: A sample from a non-white noise process