Incident and Accident Metrics

Transcription

1 1 Incident and Accident Metrics Ludwig Drees J. Sembiring, L. Höhndorf, C. Wang, S. Schiele, C. Zaglauer, F. Holzapfel Data Science in Aviation Workshop 21st May 2014, ECC-Paris, France Institute of Flight System Dynamics Technische Universität München Garching, Germany

2 The challenge of small numbers 2 Annual fatal accident rate (per million departures) Rest of the world U.S. & Canadian Operators trough Year Source: Boeing, Statistical Summary of Commercial Jet Airplane Accidents Year How do you measure safety for a single airline? Which safety metrics are appropriate? How do you determine your airline s most likely accident? How can you quantify the effectiveness of risk mitigation measures before implementing them?

3 Background 3 Mission Statement Predicting statistically valid accident probabilities for an individual airline based on available evidence from accident free operation. Accounting for airline-specific factors such as operations, training, etc. Predictive Analysis: Making quantitative statements about the future state based on previous experience and knowledge. BUT: How to implement Predictive Analysis for practical application?

4 Solution? 4? P Incident = Frequency of Incident Number of Flights P Incident = = 0 Classical statistical approach Runway overrun example vs. Simple statistical approach is inappropriate and unsuitable for rare events *Serious incidents as defined in ICAO Annex 13

5 Basic Hypothesis Predictive Analysis: Making quantitative statements about the future state based on: previous experience knowledge 5 previous experience = data/evidence driven recorded data known accident types and their causes knowledge physical relation between contributing factors and accident known cause-consequence-chains Basic Hypothesis: 1. Accidents cannot be directly observed in daily operation, however, the contributing factors still occur at high frequency so they can be measured or observed with statistical significance. 2. The relation between the contributing factors and the accident can be described by the laws of physics and cause-consequence-chains based on operational and procedural knowledge.

6 Predictive Analysis Concept 6 Unknown Contributing Factors Contributing Factors Advanced Statistical Methods Physical Model Incident

7 ... Frequency Predictive Analysis on Runway Overrun 7 Contributing Factors (Model Input) Weight Potential Outcomes Wind Speed Incident Model Overrun Model Output Transition Probabilities Outcome 1 (e.g. hull loss) Outcome 2 Outcome 3 Flaps IATA Start of Braking Incident Probability i.e. Overrun Outcome n

8 Predictive Analysis on Runway Overrun Step 1 Incident metric 8 Runway overrun: Stop margin < 0 Step 2 Functional relationships between contributing factors: Physical relationships u K G Aerodynamics Propulsion Brakes Gravitation EB 1 B = m m g sinθ + ρ 2 V A A 2 S cosβ A C D cosβ A sinβ A C Q Operational relationships Runway Condition Procedures + X P G B + μ m g cosθ cosφ ρ 2 V A A 2 S C L A/BRK Selection DRY WET DRY WET OFF LO MED

9 Incident Model Cause-Consequence Chains Chains 9 Reverser Hydraulics A/SKID... System Failures Human Factor No Go-around performed Over speed Check failed Incident 1 Incident 2 Environment Incident 3. Runway Slope Contaminated RWY Tailwind

10 Cause-Consequence Chain Examples 10 Contaminated Rwy p = 2.0 E-4 Brakes Inoperative p = 6.4 E-9 / flight A/SKID Fault p= 5.6 E-4 / flight Dual BSCU Fault p = 6.8 E-6 / flight Recognition Failed p = 5.0 E-1 / flight Reduced Braking Capability Incident Model Overrun Tailwind not in limits p = 2.3 E-5 / flight No Go-Around p = 9.0 E-1 / flight High Energy at Touchdown IATA Speed not in limits p = 2.9 E-2 / flight Cause-Consequence Chain Chain Probability Rank Chain Incident Probability Total Rank Chain 1 2.6*10E *10E-6 1 Human Performance Environment System Failures Numbers for illustration only! Chain 2 2.8*10E *10E-9 3 Chain 3 2.1*10E *10E-4 2 Chain 4 3.4*10E *10E-11 4 Chain

11 Frequency Frequency Making Data Talk 11 Asking the right question can significantly increase the information we obtain. vs. Yes No Gear Down at 2000 ft AGL? Altitude (AGL) of Gear Down AGL at Gear Down? Quality of statistical statements depend on how we look at the data Gaussian distribution Non-Gaussian distribution Underestimation of high values D Contributing factor

12 Reading Between Lines Parameter Estimation 12 Develop algorithms to extract non-measured contributing factors Estimation algorithms are applied to every single flight Parameter Estimation Implementation during Ground Roll Parameter Expected Value Standard Deviation C D,G C D,GS μ roll μ roll+brake

13 Reading Between Lines Parameter Estimation longitudinal acceleration (m/s 2 ) 13 Proof of Match Measured and Predicted Deceleration During Ground Roll measured predicted Predicted (model) f-x-tot-b [m/s Measured (QAR Data) time [sec] time (45 s)

14 Quantifying Main Drivers 14 What are the main drivers behind the incident probabilities? Postholder ENV Wind Influencable No Quantify the sensitivities of the contributing factors Postholder TRA Flaps Influencable Yes Postholder TRA Speed Deviation Influencable Yes Postholder Influencable

15 Frequency Change Management Contributing Factors (Model Input) Weight 15 Distribution based on actual flight operation (FDM) Distribution proposed by Flight Safety Manager Wind Speed Incident Model Overrun Model Output Potential reduction Flaps Start of Braking Touchdown Incident Probability Incident Metric Predictive analysis allows the assessment of the impact of mitigation actions BEFORE implementing them Impact of mitigation actions to OTHER incidents automatically considered (e.g. runway overrun vs. hard landing vs. tail strike)

16 Dependence Dependence Identifying the Unknown 16 Correlation Coefficient Copula Only captures CONSTANT dependency between two parameters Capable of capturing VARIABLE (nonlinear) dependencies between more than two parameters

17 Future Work Comparison between planned and actual performance Takeoff planning Landing distances Fuel consumption Mismatch can be expressed to quantify growth factors 2. Exploitation and correlation of further data sources: ATM data Weather data Training data Maintenance records Growth Factors Gap Deltas Planned Performance Actual Performance Gap Planned Actual + -

18 Summary 18 Predictive Analysis enables airlines: To QUANTIFY airline-specific incident and accident probabilities BEFORE things go wrong. To IDENTIFY and QUANTIFY HIDDEN and UNKNOWN contributing factors. PREDICTIVE ANALYSIS To QUANTIFY the main drivers behind incidents. To QUANTIFY the effectiveness potential mitigation actions BEFORE implementing them.

19 19 Professor Florian Holzapfel Flight Safety Group Ludwig Drees Javensius Sembiring Lukas Höhndorf Chong Wang Stefan Schiele Christopher Zaglauer Thank you!