INTELLIGENT AUTONOMY FOR MULTIPLE, COORDINATED UAVS PRESENTED BY: Dr. Lora G. Weiss PRESENTED TO: The Institute for Computational Sciences The Pennsylvania State University January 31, 2005
COORDINATED UNMANNED VEHICLE CHALLENGES Autonomously and Intelligently: Navigate and Search Large Areas Coordinate Sensor and Payload Capabilities Maintain Stealth Enable Communications Autonomously Reposition Avoid Threats and Maneuver Multi-Sensor Fusion Accomplish Mission Even if a Unit is Removed (robustness)
COORDINATED UNMANNED VEHICLE ISSUES ADDRESSED What Control Architectures Enable Coordinated Ops? How is Collaboration Enabled Within the Architecture? How are Improvements Associated with Collaboration Measured?
UNMANNED TEST VEHICLES TAKE-OFF AIR-FRAME LANDING
INTELLIGENT CONTROL ARCHITECTURE SENSOR INPUTS PAYLOAD INPUTS Sensor 1... Sensor N Messages Contacts In-Situ Environmental Data Orders Advice Perception Sensor Data Fusion Information Integration Inferencing and Interpretation Situational Awareness Incoming Missile Mine field Terrain Characterization Obstacle INTELLIGENT CONTROLLER Response Operational Assessment Dynamic Planning and Replanning Plan Execution Launch Counter Missile Monitor Situation Survey Minefield Environmental Path Planning Avoid Obstacle Configure Team Messages Conventional Control Systems Human Collaborator Other Autonomous Controllers
CONTINUOUS INFERENCE NETWORKS Size 1.0 Range.3.3 Speed Maneuvers 1.0 1.0 1.0 1.0 CONFIDENCE FACTOR OF A THREAT TARGET 1.0 Deployed Decoys Counterfire
WEIGHTED FUZZY ANDs and ORs Two weighted clues, A and B, with weights wa and wb, respectively, are merged with weighted fuzzy AND and fuzzy OR : Weighted Fuzzy AND: ANDw (A, wa, B, wb) = (1 - wa + wa*a)(1 - wb + wb*b) Weighted Fuzzy OR: ORw (A, wa, B, wb) = 1 - (1 - wa*a)(1 wb*b) These obey DeMorgan's laws and map to conventional Boolean logic when the logical values are 0 and 1 and the weights are 1.
TYPICAL CINET NODE STRUCTURE C CF(P 1 ) W 1 W 1 ADAPTATION FUNCTION SITUATION CF(P 2 ) W 2 W n C CF(Q) C CF(P n ) C C is the Node Connective Function (Continuous): I n I, I [ 01, ] W i is a weight specifying relative significance of P i to Q Relative significance may change over time (situation) Node output is the confidence factor for existence of Q C may be mathematical model of - Fuzzy and (necessity) - Fuzzy or (sufficiency) - Fuzzy M-out-of-N - Weighted Average - A Neural Network - Statistical Classifier
FUZZY LOGICAL OPERATOR FOR SIMILAR Cf(Similar) = u + (1 u) sin(π ([ (ω 6 * Cf(SA)) 2 + (ω 7 * Cf(SP)) 2 (2u 2)] / [2( 2 u)] 0.5)), where u = Max {ω 6 * Cf(SA), ω 7 * Cf(SP)} and Cf(SA) = {[1 ω 5 + ω 5 {0.5[b 16 + b 14 + (b 16 b 14 ) sin(π ((DD b 13 ) / (b 15 b 13 ) 0.5)))]}] *[1 ω 4 + ω 4 {[1 ω 3 + ω 3 {0.5 [b 12 + b 10 + (b 12 b 10 ) sin(π((dz b 9 ) / (b 11 b 9 ) 0.5))]}] *[1 ω 2 + ω 2 {0.5 [b 8 + b 6 + (b 8 b 6 ) sin(π((dy b 5 ) / (b 7 b 5 ) 0.5))]}] *[1 ω 1 + ω 1 {0.5 [b 4 + b 2 + (b 4 b 2 ) sin(π((dx b 1 ) / (b 3 b 1 ) 0.5))]}] } (0.1 + 0.9 exp( 0.3(ω 1 + ω 2 + ω 3 1))) ] } (0.1 + 0.9 exp( 0.3(ω 4 + ω 5 1))) where b 1,..., b 16 and ω 1,..., ω 7 are adaptation parameters
ADVISORY SYSTEMS Programs Target Anesthesia/Analgesia Control Neonatal Oxygenation Control Driver Warning Systems Human-In-The-Loop Applications Information Overload Subtle Combinatorial Changes?
MULTIPLE - UAV RESPONSE MISSION MANAGER NAVIGATE AVOID ASSIST SEARCH ATTACK INVESTIGATE TAKEOFF LAND EVADE CONFIRM SCAN ASSISTED ATTACK INDEPENDENT ATTACK RE-EXAMINE TRANSIT LOITER COMMUNICATE SUPERVISE INFORM DELEGATE MONITOR MISSION
MULTIPLE CONTROLLER INTERACTIONS Messages Intelligent Controller #1 Effector Commands Smart Sensor Data PERCEPTION RESPONSE Messages Messages Supervisor Intelligent Controller HUMAN GUI Peer-to-Peer Smart Sensor Data PERCEPTION RESPONSE Messages Effector Commands Smart Sensor Data Intelligent Controller #2 PERCEPTION RESPONSE Messages Effector Commands Messages
= UAV #1 ARL SAMPLE SURVEILLANCE OPERATIOINS Smart Sensor Data - Radar 1 Detected PERCEPTION Smart Sensor Data and Messages Data Fusion Algorithms Physical Properties Continuous Inference Network (Interpretation) BEHAVIORS BEHAVIOR - 3 RESPONSE Context for Sensor Interpretation Offboard Comms Orders MANEUVER BEHAVIOR Wants Control - To Get Higher Signal Strength Physical Properties TARGET CINET - Radar 1 Data - Data w/in Freq. Band - Data Associated w/ Target on List - PRF, Pulse Type, Pulse Width => High Confidence Target of Interest Inferred Properties Perceived Objects(Classes) Behavior Interface Property Class Instances BEHAVIOR - 2 BEHAVIOR - 1 Plan Execution ACTION - 1 ACTION - N ACTIONS MISSION MNGR Effector Commands Messages MISSION MANAGER - Cycles Behaviors - No Obstacles =>Maneuver OK Other ICs SIGNAL CINET - Target CINET Data Plus - Signal Level => Signal Strength Low OBSTACLE CINET => No Obstacles in Vicinity AUTOPILOT COMMAND - UAV Repositioned
ARL SAMPLE SURVEILLANCE OPERATIOINS Smart Sensor Data - Radar 1 Detected From New UAV Position PERCEPTION Smart Sensor Data and Messages Data Fusion Algorithms Physical Properties Continuous Inference Network (Interpretation) BEHAVIORS BEHAVIOR - 3 RESPONSE Context for Sensor Interpretation COMMUNICATE BEHAVIOR - Requests and Receives Control to Send Messages Offboard Comms Orders Behavior Interface Property BEHAVIOR - 2 BEHAVIOR - 1 MISSION MNGR Physical Properties TARGET CINET High Confidence Target of Interest SIGNAL CINET Signal Strength Good for Analysis Inferred Properties Perceived Objects(Classes) Class Instances Plan Execution ACTION - 1 ACTION - N ACTIONS Effector Commands Messages Other ICs MESSAGES - Msg w/ UAV Posit Goes to Sup UAV and Other UAVs = UAV #1 after maneuver = UAV #2 MESSAGES - 2 nd UAV Provides Its Posit to Sup UAV and other UAVs
= Sup UAV ARL SAMPLE SURVEILLANCE OPERATIOINS Sup UAV Sensor Data - UAV #1 and UAV#2 Posit Data PERCEPTION Smart Sensor Data and Messages Data Fusion Algorithms Physical Properties Continuous Inference Network (Interpretation) BEHAVIORS BEHAVIOR - 3 RESPONSE Context for Sensor Interpretation COMMUNICATE BEHAVIOR - Requests and Receives Control to Send Messages Offboard Comms Orders Behavior Interface Property BEHAVIOR - 2 BEHAVIOR - 1 MISSION MNGR Physical Properties Inferred Properties Perceived Objects(Classes) Class Instances Plan Execution ACTION - 1 ACTION - N ACTIONS Effector Commands Messages Other ICs TARGET CINET Cross Fix / Fusion of UAV#1 & UAV#2 Contact Data MESSAGES - Msg from Sup UAV to UAVs To Maintain Target Contact
MULTI-VEHICLE CONTROL DEMONSTRATION - 1 RUN
MULTI-VEHICLE CONTROL DEMONSTRATION - 2
MULTI-VEHICLE CONTROL DEMONSTRATION - 3
MEASURING COLLABORATIVE GAIN Multi-UAV Coordination Should Increase Likelihood of Mission Success Above the Base Capability Base Capability = Success of N Individual UAVs Accomplishing Their Missions (no coordination/collaboration) Collaborative Gain = Gain Above Base Capability and Results from Communications Information Exchange Coordinated Mission Control Collaborative Gain is Not Automatic; It Is Possible to Design Systems Where Collaboration Results in Conflicts, Causing Deterioration in Overall Mission Success
MEASURING COLLABORATIVE GAIN Collaborative Gain Can be Measured By CG = M i= 1 α i β T i i α i = weight reflecting importance of overall mission β i = success of task, 0 β i 1 T i = ith of M mission tasks Goal: Maximize CG
MEASURING COLLABORATIVE GAIN - 1 M = 3 (one for each UAV) α i = 1, i = 1,2,3 β 2 = 1, β 3 = 1 β 1 CG Base Capability β 1 = 0 CG = 2 Partial Success Incorporated β 1 = 0.6 CG = 2.6 UAVs Cooperate β 1 = 1 CG = 3
MEASURING COLLABORATIVE GAIN - 2 M = 5 (one for each UAV) α i = 1, i = 1,,5 β 2 = 1, β 5 = 1 β CG Base Capability β i = 0, i = 1,3,4 CG = 2 Partial Success Incorporated β 3 = 0.1, β 4 = 0.2 CG = 2.3 UAVs Cooperate β i = 1, i = 1,3,4 CG = 5
MEASURING COLLABORATIVE GAIN - 3 M = 3 (one for each UAV) α i = 1, i = 1,,3 β 1 = 1, β 3 = 1 β CG Base Capability β 2 = 0 CG = 2 Partial Success Incorporated β 2 = 0.1 CG = 2.1 UAVs Cooperate β 2 = 1 CG = 3
SUMMARY Multi-UAV Controller Architecture Supports Fully Autonomous Vehicles Modular Open Architecture for Mission Control Improved Situational Awareness Dynamic Planning and Re-planning Heterogeneous Vehicles w/ Disparate Payload Packages Interactions Between UAVs and Humans as Desired Coordination Between UAV with Disparate Capabilities Add or Remove Payloads w/out Affecting Controller Design Control Software Akin to Middleware Cooperative Capability For Complex Missions Hierarchical Control and Collaborative Control Collaboration Measured by Collaborative Gain Index