Using Markov Chain Monte Carlo to Solve your Toughest Problems in Aerospace

Transcription

1 Using Markov Chain Monte Carlo to Solve your Toughest Problems in Aerospace AIAA Annual Technology Symposium 30 April 2010 Mark A. Powell Attwater Consulting

2 Introduction Review of Monte Carlo Methods Introduce Markov Chain Monte Carlo Methods Examples of Aerospace Problems not Solvable without MCMC Many Details and Resources in Backup Slides, Contact Me for this Presentation or Other Papers Slide # 2

3 Review of Monte Carlo What are Monte Carlo Methods? Numerical Methods to Approximate Solutions for Definite Integrals Used when no Closed Form Analytical Solution is Available Very Useful to Calculate Probabilities for Complex Problems Requirements Ability to Random Sample the Integrand Computer Time Slide # 3

4 A Simple Example Suppose we want to Evaluate this Integral: Might Be Able to Look it Up or Use a Tool Might Recognize the Integrand and Use Probability Tables Or, Recognize and Use Monte Carlo to Approximate Obtain N samples from a N(0,1) using a Built-in Sampler Count the number between -2 and 1 and divide by N Here is the R code to Do this where N=5000 > x<-rnorm(5000,0,1) # get the 5000 standard normal samples > cnt<-0 # zero our counter > for (i in 1:5000){ # Count up those inside the interval + if (x[[i]]>=-2) if (x[[i]]<=1) cnt<-cnt+1} > cnt/5000 [1] Our Approximation Error is only not Bad! Slide # 4

5 Monte Carlo Catch(es) Must be able to Sample the Integrand For Most Problems, Built-in Samplers do not Exist in Any Tool Most Often, Make Assumptions to Allow Use of Built-in Samplers Danger Will Robinson! How Many Samples are Needed? Depends on Needed Precision Depends on Needed Number of Significant Digits Depends on Repeatability of Answer with Needed Precision and Significant Digits - Convergence Can Only be Determined Empirically, Problem Unique Slide # 5

6 Suppose Your Problem Does Not Have a Closed Form Analytical Solution and, No Built-in Samplers Exist for Your Integrand and, You do not Want to Assume Anything You can Use Markov Chain Monte Carlo Does Not Require Built-in Samplers Does Not Require Questionable Assumptions MCMC Developed about 20 Years ago Used Extensively in Europe for Biostatistics Hardly Used at All in Aerospace even Today Can Enable Quick and Easy Solutions to the Toughest Problems Slide # 6

7 Some Remarkable Capabilities with MCMC Markov Chains can be Produced that Cover the Domain for ANY Integrand Named and Recognized Integrands Unrecognized Analytical Integrands Complex Arbitrary Integrands Even Improper Integrands Even Joint Integrands with more than 10,000 Correlated (Dependent) Variables (multivariate) Marginal Integrands for Each Joint Dependent Variable are Immediately Available Full Joint and Conditional Integrands of Complex Functions of the Dependent Variables are Immediately Available Slide # 7

8 However, Even MCMC has a Few Catches Markov Chains for MCMC must be Tuned by Hand (and eye) Need Burn-in for Stationarity Need Adjustment for Sampling Stability The Solutions are Quite Simple Ignore Early Samples for Burn-in Visually Determine if Stability is Achieved The Basic Algorithm Used to Generate the Markov Chains is Unbelievably Simple Metropolis-Hastings Algorithm Easy to Tune, Using Sample Acceptance Ratio and Visual Inspection of Samples As you may have Guessed There are No Tools which can Do MCMC Coding is Needed Fortunately, it is easy! Slide # 8

9 MCMC Algorithms Primary algorithm called Metropolis-Hastings (M-H) Very General Very Simple Gibbs Sampling is a special M-H algorithm with fast convergence to steady state Must have explicit analytical and recognizable Conditional Models BUGS (Bayesian inference Using Gibbs Sampling) software is free and proven Very Few Problems in Engineering Can Use Gibbs Sampling Slide # 9

10 MCMC Summary MCMC Methods Do Not Require Recognized Integrands MCMC Methods Allow Solutions to Engineering and Decision Problems Previously Unsolvable as Stated MCMC Methods Allow Elimination of Assumptions MCMC Algorithms, Tuning, and Extensions are: Embarrassingly Easy! Slide # 10

11 Examples of Aerospace Problems Solvable Only with Markov Chain Monte Carlo Slide # 11

12 Astronaut Bone Fracture Risk On-Orbit Astronaut Bone Fractures could have Severe Consequences To the Astronaut To the Mission Very Low Probability Event No Astronaut has Ever Broken a Bone during a Mission in History Example Risk Questions What is the Risk of Bone Fracture for Long Mars Missions? How Much will the Risk Increase if International Space Station Missions extend from 180 to 365 Days? (Paper Published at IEEE Aerospace Conference March 2010, Contact Me if you Would Like a Copy) Slide # 12

13 The Available Information 977 Astronaut Missions of Varying Lengths (as of May 2005) No Events Observed No Bones Broken Did Observe 977 Mission Lengths without a Broken Bone (DATA) Slide # 13

14 Risk Results Parameterized for Mission Duration Plotted Various Assurance Levels As a Function of Mission Duration For Mars Missions of 270 Days We Can be 95% Certain that Risk of fracture during the Mission is < 3%, Based on the Information Available Quantified Result consistent with Intuition! Slide # 14

15 The ISS Mission Extension Question Bar Legend Left Side at 5 th Quantile Right Side at 95 th Quantile Color Density Proportional to Probability Density Black Bar at Median (50 th Quantile) Decision is now Much Easier Slide # 15

16 Example: US Coast Guard C130 Cockpit Cooling Turbine Cooling Turbine Provides Cooling and Pressurization to the C130 Crew Failure in Service Loss of Cooling, but More Important, Loss of Cabin Pressurization Smoke, Loud, Crew Must Secure Mission Compromised Costs Replacement: $30,000 Refurbishment: $500 Slide # 16

17 The C130 PM Problem 60:1 Ratio, Replacement:Maintain Only Had Five Failure Data: 463, 538, 1652, 1673, and 2462 flight hours Only Had One Survivor Datum: 96 flight hours What PM Interval to Select? USCG Decision Makers Paralyzed (Paper Published at IEEE Aerospace Conference March 2010, Contact Me if you Would Like a Copy) Slide # 17

18 Figure of Merit USCG had No Idea what the Failure in Service Mode was Costing Them for the Fleet of C130 s Needed to Evaluate Cost Savings for the Fleet of C130 s using a Preventative Maintenance Interval Parameterized as a function of Candidate PM Intervals The Integrand for this Problem was Rather Nasty Used MCMC Rather Liberally Slide # 18

19 Results Used MCMC to Sample Joint Uncertainty Model for Weibull Parameters Based Solely on the Data pd(η,β data) Non-Intuitive Porkchop Looking Distribution Lots of Outliers Good! Correlations Not Describable Using any Recognized Models Slide # 19

20 Cost Savings Risks Using a PM Interval - Parameterized Full Distributions Per Flight Hour, Per Bird - CS tpm Based Solely on The Data, Parameterized as a function of PM Interval in flight hours Plotted Only 5 th, Most Likely, and 95 th percentile Cost Savings Risks Peaks all between 100 and 450 flight hours At tpm = 250 hours, 95% Certain, based on the data, that USCG can SAVE at least $17 per flight hour per bird Slide # 20

21 Example: ISS O2 Sensor Drift Problem: Space Station Oxygen Sensor Measurement Accuracy is Observed to drift with Time If the Measured O 2 is in Error by more than ±6mmHG within 270 days since Calibration, it could Kill an Astronaut Already Compensating for Pressure Variations in Measurement Accuracy (Successful) Proposed Solution Options: Test for Drift rates and Compensate for Drift; OR, Redesign O 2 Sensor and Ship Up to ISS Questions: What is the Existing Risk of Sensor Accuracy Drift Beyond Acceptable Limits? Nobody Knew! What is the Risk After the Proposed Drift Compensation? Nobody Knew! Slide # 21

22 O2 Sensor Test Data Drift of the CSA-O2s During Long Life Evaluation (Data is pressure corrected) 8 Accuracy (mmhg) Days Since Calibration 270 Days Linear (1039) Linear (1031) Linear (1026) Linear (1014) Linear (1037) Slide # 22

23 Drift Corrected O2 Sensor Data Drift Time-Corrected CSA-O2s During Long Life Evaluation (Data is pressure corrected) Accuracy (mmhg) Days Since Calibration 270 Days Slide # 23

24 Problem Setup Use Gaussian Model for Measurement Errors Mean of Errors Appears to Drift with Time Model This: μ = μ 0 + μ dot * t We are Now Uncertain about μ 0, μ dot, and σ 2 errors ~ N(μ 0 + μ dot * t, σ 2 ) MCMC Code: Relatively Straightforward Slide # 24

25 Before and After Drift Correction Risk Results Linear Scale Logarithmic Scale Slide # 25

26 O2 Sensor Risk Summary Without Drift Compensation, Risk of Exceeding Accuracy Limits at 270 Days is 36-46% (with 90% Certainty) With Drift Compensation, 95% Sure Risk of Exceeding Accuracy Limits at 270 Days is < 1.5% Additional O2 Level Compensation could Reduce Risk Further Slide # 26

27 Summary Markov Chain Monte Carlo Allows Accurate Solutions for the Tough Problems faced in Aerospace MCMC Requires Coding, but the Coding is Short and Easy Contact Me for More Information Slide # 27

28 Backup Slides with More Details Slide # 28

29 Heuristic Guidance for Numbers of Monte Carlo Samples that are Needed Always use N = 10 n where n is an integer Try a Few Runs at low values of n to Bound the Precision Experiment, and When You Achieve 3 Consecutive MC runs that agree to m+1 significant digits, then your MC answer is very likely to be good to m significant digits Slide # 29

30 The Metropolis-Hastings Algorithm To Start, formulate the Model (density) pd(θ), and select a proposal Step Size dθ Start with any legal value: Θ i = Θ 1 Repeat This Loop to get new samples Propose a new sample: Θ i+1 = Θ i + ΔΘ, where ΔΘ ~U(-dΘ,dΘ), a Simple Uniform Sample Calculate the ratio (α) of Proposed Model Density to Previous Model Density: α = pd(θ i+1 data)/ pd(θ i data) Obtain a sample u from a uniform distribution: u~u(0,1) If u < α, then accept the proposed sample as Θ i+1, else set the new sample to the previous one: Θ i+1 = Θ I Maintain a Count of the Accepted Proposed Samples Slide # 30

31 M-H Algorithm Notes Θ can be a vector of parameters Θ = (θ 1, θ 2, θ 3,, θ n ) Can propose an entire new vector then test for acceptance of the new vector (dθ and ΔΘ are vectors), or Can add an internal loop to propose each new parameter and test it for acceptance to get a new sample vector Each sample of the vector Θ is a sample of the joint Model Can use other distributions for proposal besides U(-dΘ,dΘ) Scatterplots of θ j vs θ k, j k, show correlation or marginal joint densities Set of θ i samples provide Marginal Models for the i th parameter without doing this integral: Slide # 31

32 Some M-H Heuristics Use short test chains (~1,000 points) to tune Means are good starting points for quicker Burn-in Two Std. Deviations are good first proposal step sizes Parameter-by-parameter (inner loop) acceptance testing is easier to tune than vector-at-once acceptance testing Beware Changes on proposal step size for one parameter may change acceptance ratios for other parameters Improper Models can show stationarity in short chains, Track-off to non-stationarity in short chains, then back to stationarity Slide # 32

33 Tuning the M-H Algorithm Subjective judgment is usually all that is needed Need to collect data on numbers of proposed samples accepted (vector or on each parameter) Acceptance Ratios should be within 30-60% range, if not, adjust the proposal step size If ratio is too high, samples will show tracking, Increase step size If ratio is too low, samples will get stuck, Decrease step size Must look at Burn-In, Convergence to a Stationary Markov Chain Slide # 33

34 M-H Convergence to Stationarity Scatterplot Marginal samples to identify point at which Markov Chain becomes stationary, use samples beyond this point Called Burn-In Converges before 1K samples Rightmost 9,000 samples Slide # 34

35 MCMC Tuning by Acceptance Ratio and Eye Tracking Acceptance Ratio: 0.9 Increase Step Size Stuck Acceptance Ratio: 0.1 Decrease Step Size About Right Acceptance Ratio: 0.53 Suitable Step Size Slide # 35

36 Gibbs Sampling Algorithm Recall requirement to have recognizable Conditional Models: Obviously, cannot have Improper Conditional Models Changes to M-H algorithm Use parameter by parameter inner loop algorithm Do not need a proposal step size dθ To obtain a sample vector of the joint Model, loop for i=1,n proposing new sample θ i from the its conditional model and accept no need to test No need to worry about acceptance ratio or tuning Burn-in still required Slide # 36

37 Gibbs Sampling Advantages Obviously, a faster algorithm than pure Metropolis-Hastings Can combine Gibbs and M-H algorithms when you know Conditional Models for some parameters, use standard M-H step for those you don t Slide # 37

38 MCMC Extensions As will be Seen in Later Modules, Can be Used to Predict Distributions of Future Observations By Evaluating Complex Functions of MCMC Samples, Obtain Uncertainty Models for that Function Example: Obtained Joint Samples of Weibull Parameters η and β based on Observed Data Using MCMC Samples of Reliability at 250 Hours Obtained by Simply Evaluating Reliability Equation at MCMC Samples Slide # 38

39 Minimum Cost Test Plan Example Automaker Test Requirements: 95% Reliability at 100,000 miles with 90% Confidence; Cheapest Possible Test Test Cost Function: Test Plan Problem: Find Number of Vehicles n that will Survive to What Mileage T that will Produce 90% Confidence that Vehicle has 95% Reliability Basics Failures Modeled as Weibull Model with β = 3 To Meet 95% Reliability, η 269,141 Miles We need n and T that will produce 90% Confidence from our Test Results (none of the n Vehicles fail by Mileage T) that η 269,141 Miles at Minimum Cost Slide # 39

40 The Classical Test Plan Equation from Nelson to Relate n and T for 90% Confidence Level and 95% Reliability Dodson Solution Uses Graphical Optimization for Cost: n = 5 T = 207,840 miles C = $401,368 C vs. n Slide # 40

41 Just a Couple of Minor Problems What is the Probability that if the Vehicles Meet the 95% Reliability Requirement, that the Test will be Successful? I.e., Probability that All Five Vehicles Survive to 207,840 miles Simple Solution: What do we do if One or More of the Vehicles Fail During the Test? Dodson Recommended Replacing it with another Vehicle, Recalculate the Test Duration, and Continue What does this do to our Test Plan Cost? Is this a Good Test Plan? Slide # 41

42 The Second Problem: Nelson Equation Generalizes to Include the One Failure Unexpected Failures The Classical Statistics Change!!!!! New Maximum Test Duration: T 134,634 miles New Maximum Test Cost: C $315,213 New Test Plan Break One if all Five Survive to 127,534 miles C = $285,213 T vs. Failure Mileage C vs. Failure Mileage Slide # 42

43 The PRA/MCMC Test Plan Use an Ignorance Prior - Worst Case Scenario Use MCMC to find n and T at 90% Assurance Level; i.e., P(η 269,241 miles n Survivors to T miles) Use Dodson Graph to Find Optimum Cost Test Plan n = 2; 3 T = 107,192; 93,641 C = $195,429; $195,759 Slide # 43

44 A Little Comparison Classical Test Plan PRA/MCMC Test Plan n 5 3 T 207,840 93,641 C $401,368 $195,759 P(Success R=0.95) 10% 88.13% Using an Ignorance Prior and the Classical Test Plan with PRA and MCMC we Find: Automaker Pays $205,609 for an Unneeded (unrequired) Extra 9.2% Probability (Assurance) that R(100,000 miles Successful Test) = 95% Slide # 44

45 The PRA/MCMC Plan with a Failure Only 11.87% Probability of a Failure Occurring Can Use PRA/MCMC for a Failure Occurring Test Costs Increase (this makes sense) Comparable to Original Classical Test Plan Probability of Test Success given R(100,000 miles) = 0.95 Approaches Unity - Also makes Sense Expected Test Cost Still Reasonable More Test Planning Options may be Considered Zero Failures One Failure Zero or One Failure Number of Prototypes Cost P(Pass Test) Cost P(Pass Test) Expected Cost P(Pass Test) 1 $215, $713, $244, $195, $517, $222, $195, $465, $219, $203, $441, $225, $214, $431, $236, Slide # 45

46 Example Synopsis Test Plan Derived Using Classical Procedures Has some Problems Overconservative - extra 9.2% Assurance Too Expensive - More than Twice PRA/MCMC Plan Cost Too Unlikely to Succeed - Even if the Requirement is Met Overall Screwy - Consider the Effects of one Failure Test Plan Using PRA/MCMC Procedures Much More Reasonable Realistic, even with Ignorance Prior (Worst Case) Likely to Succeed if Requirement Met % More Intuitive, all the Way Around Allows Much More Flexible Test Planning and Execution Slide # 46

47 References Anderson, Theodore Wilbur, An Introduction to Multivariate Statistical Analysis, 2 nd Edition. John Wiley & Sons, Inc., New York, Berger, James O., Statistical Decision Theory and Bayesian Analysis, Second Edition. Springer-Verlag, New York, Bernardo, Jose M., and Smith, Adrian F. M., Bayesian Theory. John Wiley & Sons, Ltd., 2001 Box, George E. P., and Tiao, George C., Bayesian Inference in Statistical Analysis. John Wiley & Sons, Inc., New York, Clemen, Robert T., and Reilly, Terence, Making Hard Decisions. Duxbury, Pacific Grove, CA, Daniels, Jesse, Werner, Paul W., and Bahill, A. Terry, Quantitative Methods for Tradeoff Analyses. Systems Engineering, Volume 4, John Wiley & Sons, Inc., New York, Gamerman, Dani, Markov Chain Monte Carlo. Chapman & Hall, London, Gelman, Andrew, Carlin, John B., Stern, Hal S., and Rubin, Donald B., Bayesian Data Analysis. Chapman and Hall/CRC, Boca Raton, Florida, Gilks, W. R., Richardson, S., and Spiegelhalter, D. J., Markov Chain Monte Carlo in Practice. Chapman & Hall, Boca Raton, Florida, Hammond, John S., Keeney, Ralph L., and Raiffa, Howard, Smart Choices, A Practical Guide to Making Better Decisions. Harvard Business School Press, Boston, Jefferys, William H., and Berger, James O., Ockham s Razor and Bayesian Analysis. American Scientist, Volume 80, Research Triangle Park, NC, Jeffreys, Harold, Theory of Probability. Oxford University Press, Oxford, Raiffa, Howard, and Schlaifer, Robert, Applied Statistical Decision Theory. John Wiley & Sons, Inc., New York, Robert, Christian P., The Bayesian Choice. Springer-Verlag, New York, Robert, Christian P., and Casella, George, Monte Carlo Statistical Methods. Springer-Verlag, New York, Schmitt, Samuel A., Measuring Uncertainty, An Elementary Introduction to Bayesian Statistics. Addison-Wesley Publishing Company, Inc., Phillipines, Sivia, D. S., Data Analysis, A Bayesian Tutorial. Oxford University Press, Oxford, Venables, William N., and Ripley, Brian D., Modern Applied Statistics with S-Plus. Springer-Verlag, New York, Williams, David, Probability with Martingales. Cambridge University Press, Cambridge, Slide # 47