Integrated Approach of Risk-based Compliance-based Performance-based Audit and Surveillance Programme

Transcription

1 Integrated Approach of Risk-based Compliance-based Performance-based Audit and Surveillance Programme George WAI, Airworthiness Officer Flight Standards and Airworthiness Division Hong Kong Civil Aviation Department FSSCO, Busan, 29 November

2 Outline Purpose of the integrated approach Methodology Result 2

3 3

4 The Purpose Enhance the prioritisation in the Audit and Surveillance Programme: Introduce a Data-driven, Risk-based approach Integrate it with the existing Compliance-based and Performance-based approaches 4

5 Integrated Approach Risk-based: Risk assessment, areas of importance Compliance-based: Administrative procedures, rules Performance-based: Measureable outcomes, not prescriptive 5

6 The Guidance ICAO Doc 9859 Safety Management Manual 3rd Edition 6

7 Data to be considered AOC Annual Report Accident Report Serious Incident Voluntary Report Biannual Report Audit Report Extended Diversion Time Operations Safety Assessment Foreign Aircraft Mandatory Occurrence Report Requirements Survey Reliability 7

8 Suitable Safety Performance Indicators (SPI) Safety Data Based on Data-driven Use of existing safety oversight activities Consolidated and categorised HIGH LOW MOR EDTO/IFSD Audit findings Aircraft survey Programme Note: IFSD (Inflight Shut Down) 8

9 RISK GROUP AOC MRO H A1 A2 M1 M2 Component Avionics Overhaul Scheduled Flight Unscheduled Flight Base / Engine Maintenance Line Maintenance NDT Plating Only 9

10 Consequence Indicators Monthly Monitoring Annual Monitoring HIGH LOW MOR Audit findings EDTO/IFSD Aircraft Survey Programme(ASP) 10

11 Risk Group High Consequence Indicators Low Consequence Indicators MOR EDTO-IFSD Audit Finding Aircraft Survey A1 A2 M1 M2 H Allocation of SPIs to Risk Groups Weighed cognitively Allocated based on complexity Combined to form 5 SPIs (1) MOR & IFSD SPI (3) Audit & ASP SPI (2) MOR SPI (4) Audit SPI (5) ASP SPI 11

12 MOR & IFSD SPI (Risk Group A1) 12

13 Risk Group High Consequence Indicators Low Consequence Indicators MOR EDTO/IFSD Audit Finding Aircraft Survey A1 A2 M1 M2 H MOR SPI (Risk Group A2 & M1) 13

14 Audit & Aircraft Survey SPI (Risk Group A1) Risk Group High Consequence Indicators Low Consequence Indicators MOR EDTO/IFSD Audit Finding Aircraft Survey A1 A2 M1 M2 H Level 1 Finding: Significant non-compliance lowers the safety standard and hazards seriously the flight safety. Level 2 Finding: Any non-compliance could lower the safety standard and possibly hazard the flight safety. 14

15 Audit SPI (Risk Groups A2, M1, M2, H) 15

16 Aircraft Survey (Risk Group A1) 16

17 Target / Alert Level Made reference to ICAO Doc 9859 Established 5 zones with past 3 years data 17

18 Example: High-consequence Indicator Occurrence (MOR & EDTO/IFSD) & No. of Aircraft Total No. of Occurrence Total No. of Aircraft Organisation Operator-MOR No Risk (RF 1) Low Risk (RF 2 &3) Medium Risk (4& 5) High Risk (6& 7) EDTO IFSD

19 Simple Mathematics.. % 19

20 Occurrence Rate (EDTO/IFSD) and SPI Establishment Occurrence Rate (per aircraft) SPI - Occurrence Rate Organisation Operator-MOR Risk % No Risk (RF 1) % Low Risk (RF 2 &3) % Medium Risk (4& 5) % High Risk (6& 7) % EDTO IFSD % Overall SPI Average SD Lower Target Threshold Average Target Average SD Upper Target Threshold Average + 1 SD 1st Alert Level Average + 2 SD 2nd Alert Level

21 SPI for Occurrence Rate (MOR & EDTO/IFSD) (1) MOR & IFSD SPI 21

22 Audit Frequency Adjustment Dominating Factor 22

23 Audit Frequency Adjustment Region Individual SPI Combined SPI (Dominate SPI Dependent) G1-5% -5% or -10% Unchanged G2 Unchanged Unchanged Unchanged R1 +5% +5% or +10% Unchanged R2 +10% +10% or +15% Unchanged R3 +15% +15% +1 Audit OverseasOrganisation Risk Group(M1, M2 and H) Provided expected audit frequency adjustment based on the integrated approach Overseas M1, M2, H organisations: Audit arranged once per year/two-year cycle. Influenced by small variation by number of finding 23

24 Implementation Held internal meetings Discussed comments Refined this methodology Established 2018 audit & surveillance programme Review SPI target and alert levels annually 24

25 Conclusion Use risk-based approach on oversights Divide AOC/MRO into 5 Risk Groups Find suitable high/low consequence indicators Prioritise oversight based on analysis results 25

26 YOU CAN T HAVE ARAINBOW WITHOUT THE RAIN End of Presentation 26