PHM Engineering Perspectives, Challenges and Crossing the Valley of Death. 30 September, 2009 San Diego, CA

Transcription

1 PHM Engineering Perspectives, Challenges and Crossing the Valley of Death 30 September, 2009 San Diego, CA The views, opinions, and/or findings contained in this article/presentation are those of the author/presenter and should not be interpreted as representing the official views or policies, either expressed or implied, of the Defense Advanced Research Projects Agency or the Department of Defense Stephen J. Engel Northrop Grumman Corporation 1 Approved for Public Release, Distribution Unlimited

2 Steve s Prediction Challenge Problem Raw data are images ~200 x 265 (53,000 pixels) 12 images spanning 92 years (1917 to present) Feature discovered in 636,000 pixels Image Feature [B, H] Feature trend: B, H, B, H, B, H, B, H, B, H, B, H Probability that this pattern happens by chance times in 10,000 2 Approved for Public Release, Distribution Unlimited

3 Raw Data With Feature Image Feature [Balding, Hairy] B H B H B H Lenin Stalin Kruschev Brezchnev Andropov Chernenko B H B H B H Data-Driven Methods Are Very Powerful But Understand The Yeltsin Physical Basis Chernomyrdin of Yeltsin of Discovered Features Putin Before Blindly Nov 6, 1996 Declaring Success! Gorbachev Approved for Public Release, Distribution Unlimited Medvedev 2008-

4 Another Key PHM Challenge Cannot Duplicate (CND), Retest OK OK (RTOK), No No Trouble Found (NTF), No No Defect Found (NDF), No No Fault Found (NFF) Symptoms Symptoms appear appear in in flight flight Fault Fault code code identifies identifies the the culprit culprit Culprit Culprit is is exonerated on on the the ground ground Primary causes: Incorrect Incorrect isolation isolation False False Alarms Alarms Real Real problems problems that that are are not not reproducible Exact Exact conditions conditions are are not not known known and/or and/or not not reproducible on on ground ground Software Software faults faults Intermittent failures failures Solution: Prognostics Health Management (onboard & off off board) Comprehensive data data collection collection Advanced Advanced diagnostics onboard onboard (multi-level reasoning reasoning methods) methods) Prognosis Prognosis (health (health state state prediction) prediction) 4 Approved for Public Release, Distribution Unlimited

5 Given an Indication, What s The Probability That a Defect Truly Exists? Applying Bayes Theorem: the probability that a flaw (F) exists given a positive indication (I) depends on: Sensor s Probability of detection P(I F) probability that there will be an indication given a qualified flaw exists Sensor s Probability of false positive (false alarm) P(I ~F) probability that there will be an indication given a qualified flaw does not exist A Prior probability P(F) that the qualified flaw exists before indications are considered Probability of a flaw given an indication P( F I) = P( I Probability of detection F) P( F) P( I) = A priori probability of a flaw (from prediction) P( I Probability of a false positive P( I F) P( F) F) P( F) + P( I ~ F) P(~ F) Suppose we have a component with failure probability of 10-3 and a fault indication from a test having 100% probability of detection and 1% probability of false alarm. What is the probability that we really have a failure? 5 Approved for Public Release, Distribution Unlimited

6 Probability Considering All the Evidence Highly Dependent on A Priori Probability Probability That Defect Truly Exists Given Positive Indication P(F I) ~ 92% If a priori = 10-1 A Priori Probability ~ 9% If a priori = % Probability of False Positive P(I ~F) A Priori Probability Can Be Provided By Prognosis 6 Approved for Public Release, Distribution Unlimited

7 Prognosis Is More Than Just Trending Expected Failure Time Failure Threshold Trend is median or average Damage Assumption are required about usage from this point forward Observations Makes assumptions about future usage Makes assumptions about failure mode t 0 Current time 7 Approved for Public Release, Distribution Unlimited Time BUT it could fail a little sooner or later depending on actual usage, model & measurement errors etc

8 Prediction With Uncertainty Included Damage Expected Failure Current Time time Shaded area = Failure pdf PoF Failure Threshold 95% The probability of failure at this exact point is zero Half of the time it will already be failed The distribution must be known in order to make a risk-based decision. Initial damage state may not be known exactly d 1 d 0 t 0 Future Usage May Not Be Known Exactly 8 Approved for Public Release, Distribution Unlimited Time

9 Probabilistic Assessments From Deterministic Models Input Examples Initial defect size is a random variable Projections from Monte Carlo simulations f X θ 0 ( x0 1) Particle Size Defect Size Current time P(t) x0 f X x ) ( 1 θ 1 1 d 1 f X θ 2 ( x2 1) Particle Density x 1 Physics Model Flight Hours Usage x 2 t 0 Vertical Slice through projection gives a PDF of defect size at time t 1 t 1 Horizontal Slice through projection gives a PDF in time to reach a defect size of d1 9 Approved for Public Release, Distribution Unlimited

10 DARPA/Northrop Grumman Corp Structural Integrity Prognosis System (SIPS) Power Train Applications FLEET PLANETARY TRANSMISSION LAB TESTS NAVAIR HTTF TEST AC Feature discovery Feature validation 2 seeded fault tests H-60 AC test Sept 2003 Sept 2009 Impact Technologies, Georgia Tech, NAVAIR, Penn State ARL, Sikorsky, University of Maryland, 10 Approved for Public Release, Distribution Unlimited

11 DARPA/Northrop Grumman Corp Structural Integrity Prognosis System (SIPS) Fixed-Wing Structures Application Sub-Component 1 (4.00 x ) Fatigue Element 1 (1.868 x ) Fatigue Coupon 2 (1.868 x ) Fatigue Coupon 4 (1.0 x ) FLEET EA-6B COUPONS, ELEMENTS AND SUBCOMPONENTS 400 lab specimens 10,000 measured cracks RETIRED WING PANELS 2 outer wing panel (OWP) teardowns NAVAIR Tests 3 fullscale OWP tests Active P-3 Death Valley 24 month P3 flight demonstration Sept 2003 Sept 2010 Aero Union Corp, ALCOA, Carnegie Mellon, Cornell, DMI Inc., Impact Technologies, JENTEK Inc., Lehigh, Mississippi State, Oceana Systems, Ohio State, PSI Inc, Rensselaer Polytechnic, Ultra Electronics, University of Pennsylvania, University of Virginia, Vextec, Wash State 11 Approved for Public Release, Distribution Unlimited

12 Motivation for Structural Integrity Prognosis System Navy Legacy Method - Fatigue Life Expended (FLE) Retirement at initiated crack of 0.01 Coarse Measure of True Condition Does Not Provide Useful Reliability Measures No Longer Applies to Aging Fleet (FLE>100%) Conservatism Driven by Uncertainty Life Ends when P crack init = f L (l) P crack init FLE=%100 P crack init = Approved for Public Release, Distribution Unlimited Flight Hours SIPS Provides a More Realistic Assessment of Current & Future Health Condition

13 Adaptation With Negative Indications Catastrophic threshold Defect size Serviceable threshold Detection threshold Anticipated Usage Actual Usage Model plus sensor s negative indication Individual AC distribution FH 1 Current State Estimate Current State FH 2 FH 3 FH m Current State Flight Hours 13 Approved for Public Release, Distribution Unlimited

14 Adaptation With Positive Indications Defect size Catastrophic threshold Serviceable threshold Detection threshold Anticipated Usage Actual Usage Model plus sensor s negative indication Probability Sufficient to Warrant Action FH 1 Current State Estimate Current State FH 2 FH 3 FH m FH m+n Current State Flight Hours 14 Approved for Public Release, Distribution Unlimited

15 Engineers Need Requirements! Four Key Prognosis Requirement Parameters Maximum Allowable Probability of of Failure Bounds Bounds Risk Risk Maximum Tolerable Probability of of Proactive Maintenance Bounds Bounds Unnecessary Maintenance Lead Time Specifies Specifies the the amount amount of of advanced advanced warning warning needed needed for for appropriate actions actions Required Confidence Specifies Specifies when when prognosis prognosis is is sufficiently sufficiently mature mature to to be be used used It is also useful to have a clear definition of failure or end of useful life 15 Approved for Public Release, Distribution Unlimited

16 Maximum Allowable PoF Limits Risk Damage p max p max = area shaded blue is the maximum allowable probability of failure: t max is the point in time where the probability of failure = p max t max Failure pdf Probability of failure avoidance = red area Failure threshold Any point to the left satisfies this requirement Time 16 Approved for Public Release, Distribution Unlimited

17 Maximum Tolerable Probability of Proactive Maintenance [1 - Max Probability of Proactive Maintenance] = p min t min is the point in time where the probability of failure = p min Damage p min = area shaded blue t min t max Failure pdf Probability of unnecessary maintenance = red area Failure threshold Any point to the right satisfies this requirement Time 17 Approved for Public Release, Distribution Unlimited

18 Compliance Interval Satisfies Both Damage The requirements are satisfied as long as we design our prognosis algorithms to predict any time in the compliance interval. Is there an ideal point for validation? t min t max Failure pdf Failure threshold Compliance interval Time 18 Approved for Public Release, Distribution Unlimited

19 Just-In-Time Point & Lead-Time Damage Performance should be evaluated here Current time Just-In-Time point is the time where: PoF=[p max + p min ] Lead Time L t t min 2 t max The JIT point is best for validation purposes Failure pdf 95% Failure Threshold Compliance interval Time t 0 t a 19 Approved for Public Release, Distribution Unlimited

20 Summary Discover features in data, but then explain them Prognosis / Diagnosis Duality Diagnosis supports prognosis, AND vice versa Together they mitigate false alarms, CNDs, RTOK, NTF along with a variety of other benefits Predicting exactly when a failure will happen is not as useful as predicting when an action should be taken The more precise the failure prediction, the lower the probability of it coming true Don t strive to predict an exact time of failure, Quantify the distribution to enable risk-based decisions Well-formed prognostic requirements are the key for transitioning Agnostic Health Managers into PHM practitioners 1. Maximum Allowable Probability of Failure 2. Maximum Tolerable Probability of Proactive Maintenance 3. Lead Time 4. Required Confidence 20 Approved for Public Release, Distribution Unlimited

21 21 Approved for Public Release, Distribution Unlimited