Uncertainty Quantification in Performance Evaluation of Manufacturing Processes Manufacturing Systems October 27, 2014 Saideep Nannapaneni, Sankaran Mahadevan Vanderbilt University, Nashville, TN Acknowledgement Funding support: NIST (Cooperative Agreement NANB14H036, Monitor: Sudarsan Rachuri) Technical help: Seung-Jun Shin, Senthilkumaran Kumaraguru, Anantha Narayanan, Sudarsan Rachuri
Why Uncertainty Quantification? Uncertainty Sources Data Sources Process variability Operational data Reliability data Material uncertainty Evaluation Uncertainty Model uncertainty Legacy system data Information Fusion Test data Sensor uncertainty Simulation data Mathematical models Expert Opinion Performance evaluation affected by available data and uncertainty Uncertainty sources linear or non-linear combination, occur at different stages Information sources heterogeneous A systematic approach is necessary for information fusion and uncertainty quantification. 2
Aleatory vs. epistemic uncertainty Four types of quantities 1. Deterministic known value (fixed constant) 2. Stochastic random variability Distribution statistics are known Aleatory, irreducible uncertainty 3. Quantity is deterministic, but unknown Epistemic, reducible uncertainty True value 4. Quantity is stochastic, but distribution characteristics are unknown Has both aleatory and epistemic uncertainty Unknown distribution type Unknown distribution parameters 3
Bayesian network a b d e g a, b,.. component nodes (variables) g system-level output c f U - set of all nodes { a, b,, g } Joint PDF of all variables π(u) = π(a) π(b a) π(c a) π(d c) π(e b, d) π(f ) π(g e, f ) PDF of node g π(g) = π(u) da db df Data m b e With new observed data m π(u, m) = π(u) π(m b) a c d f g 4
Methodology for UQ (1/2) Construction of Bayesian Network Physics-based Models Use available models based on process physics (domain knowledge) Eg: Differential equations Construction of Bayesian Network Data-driven approach Build surrogate models using data e.g., Regression, Polynomial models, Gaussian Process Structure Learning algorithms Constraint-based and Score-based algorithms Hybrid approach Combination of physics-based and data-driven approaches Partial structure of the BN using domain knowledge Learn other parts of network using data Bayesian Model Calibration/Updating Uncertainty Propagation Bartram & Mahadevan, SCHM, 2014 5
Methodology for UQ (2/2) Bayesian Model Calibration/Updating Estimate unknown (epistemic) parameters using data in a probabilistic manner Bayes theorem Likelihood Prior Pr Pr Pr Pr, Posterior Evidence Sampling from Posterior: Markov Chain Monte Carlo (MCMC) Uncertainty Propagation Obtain updated posterior distribution of the Quantity of Interest (QoI) Monte Carlo Sampling - samples from posterior distribution Samples are propagated through the network Construction of Bayesian Network Bayesian Model Calibration, Updating Uncertainty Propagation Posterior distribution of QoI (Performance Metric) 6
Scalability Dimension Reduction Size of Production Network increases Increase in Epistemic Parameters Goal: Obtain a reduced set of variables such that there is no significant change in statistics of QoI Increase in Computational effort Obtain prior sensitivity indices by performing sensitivity analysis using prior distributions Assume a threshold value for sensitivity index. If sensitivity index < threshold value, assume that variable to be deterministic at the nominal value. Approach: Global Sensitivity Analysis Assess the variance contribution of each of the variables to the variance of the QoI., 1 7
Sensitivity Analysis under Epistemic Uncertainty Global Sensitivity Analysis Applicable to Y = G(X) where G is deterministic One value of X One value of Y Define auxiliary variables Can include both aleatory & epistemic sources Data uncertainty Model uncertainty P Transfer Function Distribution X G(X) Y Introduce auxiliary variable U represent variability U (0, 1) X U f ( x P) dx X F -1 (U, P) U, P X Sankararaman & Mahadevan, RESS, 2013 8
Handling Big Data HDFS : Hadoop Distributed File System To store large amounts of data in databases MapReduce A parallel processing framework for large-scale data processing. Model Building using all data computationally unaffordable Data Reduction and Feature Selection techniques required Clustering Selection of Data using Design of Experiments (DoE) Apache Mahout : Machine learning library for Hadoop The reduced dataset can be used for Bayesian Network construction 9
Summary Methodology Construction of Bayesian Network Data Reduction, Dimension Reduction Construction of Bayesian Network Bayesian Model Calibration, Updating Bayesian Model Calibration, Updating Uncertainty Propagation Without Dimension/Data Reduction Uncertainty Propagation With Data/ Dimension Reduction 10
Demonstration Problem Metal Part GTAW Arc Welding Metallic Module OR OR Plastic Grains Plastic Part GMAW Product Module Injection Molding Process Metal Powder Machine1 Metal Part Threaded Fastening e.g. Power Train Die Casting Machine2 Machine3 Turning 11
Injection Molding Goal: UQ in Energy Consumption of an Injection Molding Process Three stages Melting of polymer Injection of polymer into the mold Cooling of polymer to form product Source: http://www.maximintegrated.com/en/app-notes/index.mvp/id/4717 12
Melting Process Power used for melting the dye Volume of shot Energy consumed for melting 1 Flow rate and Average flow rate 0.5 100 Δ 100 Legend = specific gravity of polymer = heat capacity of polymer = Injection temperature = polymer temperature = polymer heat of fusion = Volume of mold = shrinkage Δ = buffer = total volume injected 13
Injection Process Injection time 2 1Δ 1000 Energy consumed in injection process Legend = number of cavities 14
Cooling Process Cooling Time ln 4 Energy consumed in cooling process Legend = maximum wall thickness of mold = thermal diffusivity = Injection temperature = mold temperature = ejection temperature COP = coefficient of performance of cooling equipment 15
Total Energy Consumption Energy consumed in making a part 1 0.75 0.25 1Δ Total Electricity Cost Reset Energy 3600 1000 0.25 Total Cycle Time Reset time t 1 1.75 Legend = maximum wall thickness of mold = Energy consumed for resetting = Power required for basic energy consuming units, when machine is in stand by mode = dry cycle time = maximum clamp stroke in the mold = unit cost in USA = Total cycle time,,,, = Efficiencies of injection, reset, cooling, heating, machine power = depth of the part = output per day = number of days 16
Injection Molding Bayesian Network Constants Calibration Parameters (Epistemic) Deterministic Observations Aleatory 17
Injection Molding Materials Several types of materials used in injection molding Metals Glass Elastomers Thermoplastic polymers Thermosetting polymers Polyethylene properties (assumed) Specific gravity = 960 kg/m 3 Heat capacity = 2250 J/kg-K Thermal diffusivity = 2.27 x 10-7 m 2 /s Shrinkage = 0.019 Heat of fusion = 240 kj/kg Nylon Polyethylene Polypropylene Polyvinyl chloride Polystyrene Acrylic Teflon 18
Injection Molding Process Parameters Synthetic Dataset Parameter Value Injection temperature 210 ( o C) Mold temperature 35 ( o C) Ejection temperature 50 ( o C) Injection pressure Flow rate 90 MPa 1.67e-5 / Coefficient of performance 0.7 All efficiency coefficients 0.7 Number of cavities 1 Fraction Δ 0.015 Thickness 0.0125 m Volume of part 0.002048 m 3 Unit Cost 10 cents/kwh Parts per day 10,000 Number of days 28 19
Injection Molding Process - Synthetic dataset Error in temperature measurements ( o C) ~0,3 Error in cycle time measurements (s) ~0,2 Error in injection pressure measurements (MPa) ~ 0,4 Initial temperature of polymer ( o C) 30,2 Density of polymer (kg/m 3 ) 960,10 Heat Capacity (J/kg-K) 2250,20 Flow Rate (m 3 /s) ~0,1.5 6 Parameter Prior Distribution ( o C) Uniform (205,220) ( o C) Uniform (45,60) ( o C) Uniform (30,45) (MPa) Uniform (88,95) (m 3 /s) Uniform (1.6e-5,1.75e-5). 100 data points are generated and used in calibration process 20
Results Prior Sensitivity Analysis Variable Type of Uncertainty Prior Individual effect Prior - Total effect Remarks Epistemic 0.4857 0.5829 Significant Aleatory 0.0161 0.0363 Significant Epistemic 0.2412 0.3617 Significant Aleatory 0.1226 0.1264 Significant Epistemic 0.0028 0.0078 Insignificant Aleatory 0.0015 0.0035 Insignificant Aleatory 0.0134 0.0137 Significant Sensitivity index < 0.01 (Threshold value) Insignificant Variables retained for Calibration : Injection Temperature, Ejection Temperature 21
Results - Prior and Posterior Plots (Epistemic variables) Injection Temperature Energy consumed per part Ejection Temperature 22
Results Prior and Posterior Sensitivity Analysis Variable Type of Uncertainty Prior Individual effect Prior - Total effect Remarks Posterior Individual effect Posterior Total effect Remarks Epistemic 0.4857 0.5829 Significant 0.0083 0.0125 Insignificant Aleatory 0.0161 0.0363 Significant 0.0686 0.0961 Significant Epistemic 0.2412 0.3617 Significant 0.0063 0.0126 Insignificant Aleatory 0.1226 0.1264 Significant 0.7997 0.818 Significant Epistemic 0.0028 0.0078 Insignificant 0.0018 0.0019 Insignificant Aleatory 0.0015 0.0035 Insignificant 0.0107 0.0418 Significant Aleatory 0.0134 0.0137 Significant 0.0414 0.0798 Significant Uncertainty in and greatly reduced after calibration process (Effects Significant to Insignificant) 23
Illustrative Example Welding Process Energy Consumption (1/2) Goal: UQ in Energy Consumption of a Welding Process Volume of the Weld 3 4 2 Theoretical Minimum Energy (filler and metal are the same) Input Energy (Laser) Efficiency of welding process Legend L = Length of weld = density of material = Heat capacity of material = Final Temperature = Initial Temperature = Latent Heat = Voltage = Current = Welding Speed 24
Energy Consumption in Welding Process (2/2) Total Power Total Electricity Cost Legend = Total weld time = Number of days in service = Unit cost for electricity in USA 25
Welding Process Bayesian Network Constants Calibration Parameters (Epistemic) Deterministic Observations Aleatory 26
Synthetic Dataset (1/2) Parameter Initial Temperature ( ) (K) Final Temperature ( ) (K) Heat Capacity ( ) (J/kgK) Density () (kg/m 3 ) Latent Heat () (kj/kg) Weld Zone parameters (mm) Length of weld () (mm) Value 303,0.3 1628,10 500,5 8238,10 270,3 8.5,0.5 2.6,0.5 2,0.1 15,0.5 11,1 500,10 Voltage () (V) 20 Current () (A) 250 Weld Speed () (mm/min) 700 26 10 cents/kwh 27
Synthetic Dataset(2/2) Error in length measurement (mm), ~ 0,0.01 Error in current measurement (A), ~ 0,2 Observation Data 8.5,0.5 0,0.01 2.6,0.5 0,0.01 11,1 0,0.01 250 0,2 Parameter Prior Distributions Prior Distributions Uniform (8.3,8.6) Uniform (0.2,0.7) Uniform (2.5,2.8) Uniform (0.3,0.6) Uniform (10,13) Uniform (0.8,1.3) Uniform (235,260) Uniform (1.7,2.5) - Standard deviation in measurement error of current 28
Results Prior Sensitivity Analysis Parameter Type of uncertainty Prior Individual effect Prior Total effect Remarks Aleatory 0.0745 0.0872 Significant Epistemic 0.0026 0.0027 Insignificant Epistemic 6.44 x 10-7 7.75 x 10-3 Insignificant Aleatory 0.2147 0.2238 Significant Epistemic 0.0073 0.0074 Insignificant Epistemic 7.056 x 10-8 0.00826 Insignificant Aleatory 0.3153 0.3227 Significant Epistemic 0.2019 0.2032 Significant Epistemic 1.225 x 10-6 0.00569 Insignificant Aleatory 0.044 0.0442 Significant Aleatory 0.1651 0.1655 Significant Aleatory 0.000124 0.000125 Insignificant Aleatory 0.00435 0.0044 Insignificant Aleatory 0.00091 0.00093 Insignificant Aleatory 2.393 x 10-6 2.421 x 10-6 Insignificant Aleatory 0.00225 0.00228 Insignificant Aleatory 0.0347 0.0351 Significant Threshold value 0.01 Significant Epistemic variables - 29
Results Prior and Posterior Plots (Epistemic Variables) Weld parameter e Theoretical energy consumption per weld Prior and Posterior no major change, most of the uncertain variables are aleatory (uncertainty can not be reduced) 30
Results Prior and Posterior Sensitivity Analysis Parameter Type of uncertainty Prior Individual effect Prior Total effect Remarks Posterior Individual effect Posterior Total effect Remarks Aleatory 0.0745 0.0872 Significant 0.1193 0.1226 Significant Epistemic 0.0026 0.0027 Insignificant 0.00012 0.00013 Insignificant Epistemic 6.44 x 10-7 7.75 x 10-3 Insignificant 1.062 x 10-7 6.005 x 10-5 Insignificant Aleatory 0.2147 0.2238 Significant 0.2995 0.3001 Significant Epistemic 0.0073 0.0074 Insignificant 0.00025 0.000247 Insignificant Epistemic 7.056 x 10-8 0.00826 Insignificant 7.208 x 10-10 0.000135 Insignificant Aleatory 0.3153 0.3227 Significant 0.2967 0.2989 Significant Epistemic 0.2019 0.2032 Significant 0.00034 0.00034 Insignificant Epistemic 1.225 x 10-6 0.00569 Insignificant 2.991 x 10-9 0.000147 Insignificant Aleatory 0.044 0.0442 Significant 0.0493 0.0494 Significant Aleatory 0.1651 0.1655 Significant 0.2069 0.2075 Insignificant Aleatory 0.000124 0.000125 Insignificant 0.000153 0.000154 Insignificant Aleatory 0.00435 0.0044 Insignificant 0.00549 0.00554 Insignificant Aleatory 0.00091 0.00093 Insignificant 0.00107 0.00108 Insignificant Aleatory 2.393 x 10-6 2.421 x 10-6 Insignificant 2.054 x 10-6 2.564 x 10-6 Insignificant Aleatory 0.00225 0.00228 Insignificant 0.00313 0.00316 Insignificant Aleatory 0.0347 0.0351 Significant 0.04097 0.0413 Significant Uncertainty in greatly reduced after calibration process (Significant to Insignificant) 31
Comprehensive framework for uncertainty integration and management Bayesian network j Include all available models and data Include calibration, verification and validation results at multiple levels e d Heterogeneous data of varying precision g and cost c Models of varying complexity, accuracy, s cost Data on different but related systems Bayesian Network o n r Facilitates Forward problem: UQ in overall system-level prediction Integrate all available sources of information and results of modeling/testing/monitoring Inverse problem: Resource allocation at various stages of system life cycle Model development, data collection, system design, manufacturing, operations, health monitoring, risk management 32
Summary of Analyses Information Retrieval retrieve required information from database Dimension Reduction Select a subset of features/parameters Data Reduction Reduce amount of data through clustering Model Building Build a Bayesian Network Uncertainty Propagation Monte Carlo simulation Sensitivity Analysis (Aleatory vs. Epistemic) Variance decomposition Performance Prediction Monte Carlo simulation Temporal variation -- Dynamic Bayesian Network Diagnosis, Prognosis Classification, Particle filtering Decision-Making, Resource allocation 33
Future Work Derive Bayesian network from process and model libraries Include text and image data, along with numerical data in UQ Apply to production network with multiple processes Analysis over time with streaming data (Dynamic Bayesian Network) Explore various problems in manufacturing Process Monitoring (Dynamic Tracking) For diagnosis Stochastic Process Optimization Adjust parameters to meet the requirements Risk Management Reduce performance variability to be within desired limits Resource Allocation maximize information maximize reduction in uncertainty 34