Unmanned Aircraft System Well Clear Dr. Roland Weibel 18 November 2014 Sponsors: Neal Suchy, FAA TCAS Program Manager, Surveillance Services, AJM-233 Jim Williams, FAA Unmanned Aircraft Systems Integration Office Manager, AFS-80 Distribution Statement A. Approved for public release; distribution is unlimited. This work is sponsored by the Federal Aviation Administration under Air Force Contract #FA8721-05-C-0002. Opinions, interpretations, recommendations and conclusions are those of the author and are not necessarily endorsed by the United States Government.
Need for Quantitative Well Clear Definition for UAS Unmanned Aircraft System (UAS) Detect & Avoid (DAA) equipment is required to maintain well clear and avoid collisions Historical definition of well clear is subjective, impeded development of specific requirements: FAR 91.111:...not operate so close to another aircraft as to create a collision hazard FAR 91.113: Vigilance shall be maintained so as to see and avoid other aircraft pilots shall alter course to pass well clear of other air traffic Analytically-derived, quantitative performance threshold for initial definition of well clear is needed Provides specific baseline for evaluation of safety and interoperability of proposed architectures Establishes basis and methodology for refinement in light of operational considerations or airspace changes UAS Well Clear - 2
Organization and Process Detect and Avoid (DAA) Science and Research Panel (SARP) created in 2011 to coordinate DAA research Supports multi-agency UAS Executive Committee Lack of well clear definition identified as highest priority research gap Well clear working group formed to rapidly deliver recommendation to RTCA SC-228 August 2013 August 2014 Approach and progress reported to UAS ExCom Multi-agency collaboration: FAA, DOD, NASA, MITRE, MIT LL, subject matter experts RTCA SC-228 UAS Executive Committee (ExCom) Senior Executives FAA DOD NASA DHS Senior Steering Group (SSG) Agency Representatives Detect and Avoid Science and Research Panel (SARP) SARP Well Clear Working Group UAS Well Clear - 3
Well Clear as a Separation Standard Recognized as a separation standard, informed by operational acceptability Future trajectories Initially defined as the relative state where a level of risk is achieved regardless of ownship or intruder avoidance maneuvering Aircraft 1 Relative state between aircraft Aircraft 2 Objective: define risk threshold and map to other states (range, altitude, time, ) Collision Risk Acceptable Risk Threshold well clear Relative State Established Airspace Separation Standard Methodology Applied to Detect and Avoid UAS Well Clear - 4
Outline Background Analysis & Results Overview Risk Threshold Determination Collision Risk Modeling SARP Recommendation Summary UAS Well Clear - 5
Analysis Overview rse Trade Sp pace Analysis Well Clear Definitions Boundaries Collision Risk Modeling Probabilistic Encounters Risk TCAS Alerts Risk Threshold Initial Sizing presentation focus Coa Fine Anal lysis of Individual Definitions Representative algorithms No maneuvers Operational Characterization Probabilistic Encounters Stressing Trajectories Human in the Loop Simulation Additional Performance Metrics Collision Risk Intruder TCAS Alerts Controller Acceptability Course Deviation Alert & Maneuver Rate UAS Well Clear - 6
Acceptable Risk Objective Assumptions on Safety Management System (SMS) severity Well clear violation Minor Near midair collision (NMAC) - Hazardous Derived requirement: failure of implemented DAA would exhibit a conditional probability of NMAC given a well clear violation of 10-4 Air Traffic Control Operational Error Classification Conformance Categories as a percentage of the separation standard 100% 75% 50% 25% 0% Cat. D (>90) Cat. C (90-75) Cat. B (75-34) Cat. A (34-0) NMAC Minimal 1x10(-1) Minor 1x10(-3) Major 1x10(-5) Hazardous 1x10(-7) Catastrophic 1x10(-9) P (Hazardous Minor): 10(-4) Associated ATO SMS Outcome Severity & Acceptable Rate (/hr) UAS Well Clear - 7
Derived Conditional Risk Threshold More frequent Eve ents and Rat tes Encounter Safety Target Less frequent Separation Failure Events Well clear separation failure No action P(NMAC WCV ) Separation risk reduction Collision avoidance risk reduction no action Notes: WCV = Well Clear Violation NMAC = Near Midair Collision MAC = Midair i Collision i CA = Collision Avoidance 5% 10% 2% WCV: Minor Product 10-4 NMAC: Hazardous MAC Objective: derive risk threshold without maneuver actions Assumptions made on achievable DAA performance CA (2%) similar to TCAS against TCAS risk ratio Separation (10%) as near- equal contribution Conditional risk, P(NMAC WCV) of 5% meets 10-4 target Conditional Risk Threshold Agnostic to Architecture, Environment, Absolute Target UAS Well Clear - 8
Well Clear Boundary Concepts Concept 1: Horizontal: time & distance Vertical: distance Modified Tau: Range 2 DMOD 2 Range * Range Rate and Horizontal Miss Distance Relative Altitude Concept 2: Horizontal: time & distance Vertical: time & distance Modified Tau: Range 2 DMOD 2 Range * Range Rate and Horizontal Miss Distance Relative Altitude or Time to Co-Altitude Concept 3: Horizontal & Vertical: scaled boundaries with speed & altitude (ownship and intruder) UAS Well Clear - 9
High Fidelity Encounter Models Models based on continuous radar data feed from the U.S. Air Force 84 th Radar Evaluation Squadron (RADES) CONUS Radar Coverage Data enables characterization of traffic environments and development of safety analysis models Models recently expanded to encompass littoral/oceanic regions and for self-separation system assessment Development Model Encounter Construction 134 radar sites across the US ~10 GB per day Probabilistic initial conditions Aircraft 1 Radar tracker Radar tracker Radar tracker Process tracks and extract t track data Probabilistic dynamic update ( 1 s -1 ) Aircraft 2 Randomly initialize AC2 trajectory with respect to AC1 Developed using a Bayesian approach UAS Well Clear - 10 * Kochenderfer et al., Airspace Encounter Models for Estimating Collision Risk, AIAA Journal of Guidance, Control, and Dynamics, 2010.
Trade Space Analysis Monte Carlo encounter modeling used to coarsely assess risk across large trade space of boundary threshold combinations Encounters and visualization tool provided to collaborators to size boundaries for follow-on analysis Risk threshold refined to 1.5%, informed by operational suitability Contours of Conditional Risk for Binned Thresholds (Concept 1 Example) Tau 35 s Altitude 700 ft Miss Distance* 4000 ft Range* 4000 ft P(NMAC WCV) = 1.5% P(TCAS RA WCV) = 2.1% *Boundaries held constant for illustration UAS Well Clear - 11 Note: 1 Million Encounters generated from MIT LL Uncorrelated Encounter Model v2.0
Initial Operational Characterization Multiple collaborative activities produced performance metrics to support initial well clear recommendation TCAS alert rate Controller acceptability Maneuver initiation point Mitigated risk ratio Miss distance distributions Track Deviation Well clear violation rate Mitigated risk ratio Well clear violation rate Metric importance and performance rated qualitatively by working group experts Total weighted score determined SARP recommendation Recognized potential need to further refine definitions Higher Weighted Metrics Mid / Lower Weighted Metrics Well Clear Candidate Evaluation Against Metrics Cleared for Open Publication: 14-S-2344 Final SARP operational suitability performance assessment scores 13 UAS Well Clear - 12
Well Clear Boundary Recommendation 35 s Modified Tau: Range 2 DMOD 2 4000 (4) ft Range * Range Rate and Horizontal Miss Distance Relative Altitude 700 ft 4000 ft SARP recommendation delivered to RTCA SC-228 on 20 August 2014 FAA ATO reviewed and refined definition to achieve improved operational suitability To mitigate concerns about controller acceptability with existing VFR/IFR separation standards Reduced altitude threshold to 450 ft, added advisory alert at 700 ft Current basis for SC-228 requirements development UAS Well Clear - 13
Outline Background Analysis & Results Overview Risk Threshold Determination Collision Risk Modeling SARP Recommendation Summary UAS Well Clear - 14
Summary Collision risk modeling central to defining well clear Leveraged accepted processes for defining airspace separation performance thresholds Provides common baseline for comparison of diverse definitions Specific well clear definition enables progress toward DAA requirements Provides analytical approach that can be used to evaluate future airspace capabilities Additional analyses underway Detailed characterization of system requirements in the context of proposed DAA operation and architecture (coordination with ATC, surveillance errors, display configuration, etc.) Assessment of interoperability with current and future collision avoidance systems UAS Well Clear - 15