INTEGRATED HEALTH MANAGEMENT AND CONTROL OF COMPLEX DYNAMICAL SYSTEMS

Transcription

1 The Pennsylvania State University The Graduate School Department of Mechanical Engineering INTEGRATED HEALTH MANAGEMENT AND CONTROL OF COMPLEX DYNAMICAL SYSTEMS A Thesis in Mechanical Engineering by Devendra K Tolani c 2005 Devendra K Tolani Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy August 2005

2 Sanskrit shloka from the Brihad-aranyaka Upanishad (8th-7th century BCE) asato mā sadgamayā tamaso mā jyotirgamayā mŗtyor mā amŗtan gamayā om shāntiĥ shāntiĥ shāntiĥ From falsehood, lead me to the truth; from darkness to light; from death to Immortality Om! Peace! Peace! Peace!

3 The thesis of Devendra K. Tolani was reviewed and approved* by the following: Asok Ray Distinguished Professor of Mechanical Engineering Thesis Co-Adviser Co-Chair of Committee Joseph F Horn Assistant Professor of Aerospace Engineering Thesis Co-Adviser Co-Chair of Committee Constantino M Lagoa Associate Professor of Electrical Engineering Alok Sinha Professor of Mechanical Engineering Qian Wang Assistant Professor of Mechanical Engineering Richard C. Benson Professor and Head Department of Mechanical and Nuclear Engineering * Signatures are on file in the Graduate School.

4 iii Abstract A comprehensive control and health management strategy for human-engineered complex dynamical systems is formulated for achieving high performance and reliability over a wide range of operation. Results from diverse research areas such as Probabilistic Robust Control (PRC), Damage Mitigating/Life Extending Control (DMC), Discrete Event Supervisory (DES) Control, Symbolic Time Series Analysis (STSA) and Health and Usage Monitoring System (HUMS) have been employed to achieve this goal. Continuous-domain control modules at the lower level are synthesized by PRC and DMC theories, whereas the upper-level supervision is based on DES control theory. In the PRC approach, by allowing different levels of risk under different flight conditions, the control system can achieve the desired trade off between stability robustness and nominal performance. In the DMC approach, component damage is incorporated in the control law to reduce the damage rate for enhanced structural durability. The DES controller monitors the system performance and, based on the mission requirements (e.g., performance metrics and level of damage mitigation), switches among various lower-level controllers. The core idea is to design a framework where the DES controller at the upper-level, mimics human intelligence and makes appropriate decisions to satisfy mission requirements, enhance system performance and structural durability. Recently developed tools in STSA have been used for anomaly detection and failure prognosis. The DMC deals with the usage monitoring or operational control part of health management, where as the issue of health monitoring is addressed by the anomaly detection tools. The proposed decision

5 and control architecture has been validated on two test-beds, simulating the operations of rotorcraft dynamics and aircraft propulsion. iv

6 v Table of Contents List of Tables x List of Figures xi Acknowledgments xiv Chapter 1. Introduction Motivation Research Objectives Background and Literature Survey Contributions of the Proposed Research Organization Chapter 2. Discrete Event Systems Introduction Language Measure Optimal Discrete Event Supervisory Control Chapter 3. Symbolic Time Series Analysis for Anomaly Detection Introduction Nonlinear Time Series Analysis Symbolic Time Series Analysis (STSA) and Encoding Pattern Identification

7 vi The ɛ-machine The Suboptimal D-Markov Machine Statistical Mechanical Concept of D-Markov Machine Comparison of D-Markov Machine and ɛ-machine Anomaly Measure and Detection Anomaly Detection Procedure Chapter 4. Probabilistic Robust Control of Future Generation Rotorcraft Introduction Control System Architecture Rotorcraft Dynamics Dynamic Controller Signal Conditioning and Signal Validation Risk Assessment Subsystem Upper Tier Control at the Flight Management Level System Identification The Plant Model Controller Design Procedure Sample Generation Procedure Practical Implementation Issues Simulation Results and Discussion Conclusion Chapter 5. Hierarchical Control for Rotorcraft

8 5.1 Hierarchical Control for a bank of Probabilistic Robust Controllers Hierarchical Control for a bank of Damage Mitigating Controllers.. 97 vii Chapter 6. Hierarchical Control of Aircraft Propulsion Systems: Discrete Event Supervisor Approach Introduction Description of the test bed for the propulsion system simulation Synthesis of Discrete Event Supervisory(DES) Controllers Engine Level DES control Propulsion Level DES control Simulation Experiments: Results and Discussion Evaluation of State Transition Matrix Parameters Selection of Characteristic Values for Quantitative Evaluation of DES Controllers Optimal DES controller synthesis and evaluation Summary and Conclusions Chapter 7. Anomaly Detection for Health Management of Aircraft Gas Turbine Engines Introduction Symbolic Dynamics Radial Basis Function Neural Network Principal Component Analysis Experimental Results

9 Chapter 8. Summary, Conclusions and Future Work Future Work viii Appendix A. Rotorcraft Simulation and Control Test-bed (RSC) Appendix B. Damage Mitigating Control B.1 Introduction B.2 Control Scheme B.2.1 Open Loop Plant Model B.2.2 Controller Design and Implementation B.3 Damage Model B.3.1 Theoretical Stress and Crack Growth Model B.3.2 Implementation in GENHEL B.4 Results and Discussion B.4.1 Outer Loop Controller B.4.2 Time History Results B.4.3 Handling Qualities Results B.5 Conclusions Appendix C C.1 Standard Information-Theoretic Quantities C.2 Finite-type Shift and Sofic Shift Appendix D. Nomenclature Appendix E. Acronyms

10 References ix

11 x List of Tables 1.1 Nomenclature of Signals in Figure Nomenclature of Signals Risk and Performance of Controllers Event List for the Mission scenario DFSA Model State List for the Mission scenario DFSA Model Simulated Performance of DES Controllers Event List for the Engine Level DFSA Model State List for the Engine Level DFSA Model Event List for the Propulsion Level DFSA Model State List for the Propulsion Level DFSA Model Π matrix of the Propulsion Level DFSA Model Iterations for Optimal DES Controller Synthesis Language Measure and Performance for Different Supervisors

12 xi List of Figures 1.1 Overall architecture Detailed architecture Continuous Dynamics to Symbolic Dynamics State Machine with D=2, and A = Rotorcraft control architecture Roll axis frequency response Yaw axis frequency response Model uncertainty in the roll axis Augmented plant model Augmented model Time history results for high risk controller Recovery from instability High risk/low risk controller response Energy content of the roll rate response System recovery from instability Choosing Appropriate DMC DES Representation Pilot following DES commands Pilot not following DES commands

13 xii 5.8 Comparison of damage in the two cases Autopilot using Damage Mitigation Autopilot not using Damage Mitigation Crack Length and Damage Weights Overall architecture Engine Level Plant/DES controller Propulsion Level DES controller Engine Level DES plant model Engine Level DES controller model Power Lever Angle input Simulation output for the unsupervised case Simulation output for the supervised case Effect of propulsion level DES controller on Engine Effect of propulsion level DES controller on Engine Convergence of event costs Wavelet transforms of time series data Times series data (T c ) for various efficiency drops Times series data (T c ) for efficiency drop = 2.5% Comparison of different methods of anomaly detection A.1 RSC test-bed architecture B.1 Model-Following Control System Schematic

14 xiii B.2 Typical Twin-Engine Helicopter Transmission Schematic B.3 Main Bevel Pinion/Main Bevel Gear Schematic B.4 Main Bevel Pinion Tooth Profile B.5 Main Bevel Pinion Tooth Stress Profile B.6 Step Climb Results (80 Knots)- a B.7 Step Climb Results (80 Knots)- b B.8 Medium-Length Terrain-Following Maneuver Results - a B.9 Medium-Length Terrain-Following Maneuver Results - b B.10 Extended-Length Terrain-Following Maneuver Results with Damage Weight Switching - a B.11 Extended-Length Terrain-Following Maneuver Results with Damage Weight Switching - b B.12 Height Response Plot (V = 40 Knots)

15 xiv Acknowledgments I dedicate this thesis to my parents. I am, who I am; I am, what I am; I am, where I am; because of them. This work has been made possible by the active support of number of people. The most direct support came from my supervisors, Dr. Asok Ray and Dr. Joseph Horn. Dr. Ray has been more than an adviser, he has been in the true sense of word, a mentor to me. His dedication to research should be a topic of research in itself. Dr. Horn provided me active support and guidance throughout my graduate studies, especially in the Aerospace field which was totally alien to me at the beginning of my dissertation. I would like to thank Dr. Constantino Lagoa for clarifying my innumerable doubts both in and outside the classroom and I am indebted to him for his active support and encouragement. I would like to thank Dr. Alok Sinha and Dr. Qian Wang for their invaluable suggestions and constructive criticisms. Special thanks to Murat, Jialing, and Derek, who were an integral part of this project. Finally, I would like to thank my house-mates and friends at Penn State, who have been like a home away from home. I will cherish their friendship and fond memories. Last but certainly not the least I would like to thank the sponsors of this research effort: U.S. Army Research Office(Grant No. DAAD ); NASA Glenn Research Center(Grant Nos. NAG and NNC04GA49G); National Science Foundation (Grant No. CMS ).

16 1 Chapter 1 Introduction The ultimate aim of the research initiated in this dissertation is to formulate a comprehensive control and health management strategy for human-engineered complex dynamical systems to achieve high performance and reliability over a wide range of operation. This goal could be achieved by a hierarchically structured decision and control system that synergistically combines the results from diverse research areas: Probabilistic Robust Control (PRC), Damage Mitigating/Life Extending Control (DMC), Discrete Event Supervisory (DES) Control, Symbolic Time Series Analysis (STSA) and Health and Usage Monitoring System (HUMS). While the PRC and DMC form the backbone of the lower-level continuous-domain control, the discrete-domain supervision at the upperlevel is based on DES control. Recently developed tools in STSA have been used for anomaly detection/failure prediction. The DMC deals with the usage monitoring or operational control part of health management, where as the task of health monitoring is taken care by the anomaly detection tools. In essence, an integrated strategy has been proposed for the comprehensive health management and control of complex dynamical systems. For testing and validating the proposed decision and control architecture, two independent simulation test-beds of a rotorcraft dynamics and aircraft propulsion, have been constructed in the laboratories of the Complex Systems Failure (CSF) Collaboratory at Pennsylvania State University.

17 2 Before going into the technical details and the implementation issues, a brief explanation regarding the thesis title, Hierarchical Control and Health Management of Complex Dynamical System, is called for. Hierarchical Control: Hierarchically structured information and control systems occur for at least two related reasons: first, the complexity of many natural and engineered systems limits the ability of humans and machines to describe and comprehend them; and, second, the inherent limitations on the information processing capacity of feedback mechanisms results in the decision and control systems being organized in hierarchical configurations. The notion of Discrete Event Supervisory (DES) control blends with the concept of Hierarchical architecture. Intuitively, lower level control (i.e., the domain of continuous control) implies the traditional frequency domain or time domain based control strategies that are designed to follow certain specifications (e.g., the regulator problem). In essence, a lower level control is more precise and less intelligent. On the other hand the proposed DES decision and control at the upper level has a hierarchical structure to mimic human intelligence [75]. Health Management: The topic of health management covers primarily two areas listed below: Damage Mitigating Control (DMC): Component damage (e.g., crack development in a critical component) is incorporated in the optimization of the control system in order to reduce damage rate (e.g., crack growth rate). The damage level is monitored or predicted in real time or near-real time, and the

18 3 controller is adjusted to reduce the damage rate as damage accumulates on the system. This notion of health management can be extended even further: there can be a fleet of similar subsystems (say two engines, being part of the propulsion system of an aircraft), which may have different health status. The notion of nested hierarchies is also explored, where a subsystem (e.g., a gas turbine in the above example) is nested inside another hierarchical scheme (e.g., propulsion system). The hierarchical control scheme described above assigns the workload. In the above example of gas turbine engine: thrust from individual engines is regulated based on their health status. Anomaly Detection: In short, anomaly means deviation from nominal condition. Early detection of anomalies and their characterization are essential for health management that includes prognosis of impending failures in critical components and taking actions for mitigating their detrimental effects on the system. Complex System : A leading authority on the subject, the Santa Fe Institute s website lists over three hundred definitions of complex systems. Instead of trying to define, it is better to understand complex systems in terms of one or more properties (listed below), that they might possess. Emergence: What distinguishes a complex system from a merely complicated one is that some behaviors and patterns emerge in complex systems as a result of the patterns of relationship between the elements.

19 4 Short-range relationships: Typically, the relationships between elements in a complex system are short-range. That is, information is normally received from the nearest neighbors. Richness of connections implies communications pass across the system but are probably modified on the way. Non-Linear and (possibly) time-varying relationships: There are rarely simple cause and effect relationships between elements. A small stimulus may cause a large effect or no effect at all. (The butterfly effect is a phrase that encapsulates the more technical notion of sensitive dependence on initial conditions. The idea is that small variations in the initial conditions of a dynamical system can produce large variations in the long term behavior of the system.) Nested structure: Another key aspect of complex systems is that the components of the system - usually referred to as agents - are themselves complex systems. For example, an economy is made up of organizations, which are made up of people, who are systems of organs controlled by their nervous systems and endocrine systems, that in turn, are made up of cells, all of which, are complex systems, at each level in the hierarchy. Another example which is covered in much detail in this thesis is the propulsion system of an aircraft. The propulsion system is made up of individual gas turbine engines, which themselves are complex systems. Feedback loop configuration: Both negative (damping) and positive (regenerative) feedback are key ingredients of complex systems. The effects of

20 5 an agent s actions are fed back to the agent and this, in turn, affects the way the agent behaves in the future. This set of constantly adapting nonlinear relationships lies at the heart of what makes a complex system special. Open architecture: Complex systems are open systems, i.e., energy and information are constantly being imported and exported across system boundaries. This causes the complex systems to be subjected to fluctuations. Under normal circumstances, complex systems are at quasi-static equilibrium in the thermodynamic sense [4] and are subjected to small fluctuations. under abnormal situations, these fluctuations may rapidly grow and the equilibrium (or stability) condition is lost. The goal of the decision and control system is to forecast such a situation and take appropriate actions to circumvent potential instabilities. The whole not being sum of the parts: There is a sense in which elements in a complex system cannot know what is happening in the system as a whole. If they could, all the complexity would have to be present in that element. A corollary of this hypothesis is that no single element in the system is capable of individually controlling the system. Imprecise boundaries: It is usually difficult to determine the boundaries of a complex system. The decision is usually based on the observer s needs and prejudices rather than any intrinsic property of the system itself. This thesis is based on detailed simulation experiments performed on two testbeds, mentioned earlier: Rotorcraft Simulation and Control (RSC) test-bed and Gas

21 6 Turbine Simulation (GTS) test-bed. The structure of the hierarchical control architecture is largely similar in both cases. The aim here is to develop, explain and validate the theory by making these two test-beds as platforms. These test-beds exhibit salient features of complex systems listed above. Rotorcraft Simulation and Control (RSC) test-bed: A mathematical description or simulation of rotorcraft s flight dynamics needs to embody the complex aerodynamic, structural and other internal dynamic effects (e.g., engine, actuation) that combine to influence the response of the aircraft to pilot s controls (handling qualities) and external atmospheric disturbances (ride qualities) [58]. This interaction is highly complex and the dynamic behavior of the rotorcraft is often limited by local effects that rapidly grow in their influence to inhibit larger or faster motion, e.g., blade stall. A non-linear dynamic model of the UH-60A Black Hawk helicopter has been adopted for this study [37]. The GENHEL rotorcraft simulation code is widely used by industry and the U.S. government and is accepted as a validated engineering model for handling qualities analysis and flight control design. The code models non-linear aerodynamic effects, and includes fuselage rigid body dynamics, rotor blade flapping and lagging dynamics, rotor inflow dynamics, engine / fuel control dynamics, actuators, and a model of the existing UH-60A automatic flight controls systems (AFCS). The code has been modified to allow for the disengagement of existing AFCS channels and for the integration of the controllers presented in this thesis.

22 7 Gas Turbine Simulation (GTS) test-bed: The gas turbine engine is a complex machine involving operation at extremes of pressure and temperature and interactions between various units, e.g., fan/compressor, combustor and turbine/nozzle. In addition, interactions between aerodynamics, thermodynamics and mechanical integrity for a particular component introduces further complexity. The high power to weight ratio has made it the propulsion system of choice in aircraft applications. It is also used extensively in the oil, gas, power and process industries. The GTS test-bed is based on the non linear dynamic model of a generic gas turbine engine [20] [27]. It has now been extended to simulate a propulsion system involving two gas turbines engines. This introductory chapter is composed of four sections. The first section deals with the motivation of this dissertation. Second section covers the objectives of this dissertation. Third section describes the background and contribution of this research. The last section of this introductory chapter covers the organization of the rest of the thesis. 1.1 Motivation Discrete-event dynamical behavior of physical plants (e.g., rotorcraft and gas turbine engines) is often modeled as regular languages that can be realized by finitestate automata [14] [62]. This thesis focuses on development of intelligent decision and control algorithms based on Discrete Event Supervisory (DES) control of complex dynamical systems like rotorcraft and aircraft propulsion systems. A good feature of

23 8 the proposed DES control approach is that the control policy can be adaptively updated on-line and that the system is tolerant to small anomalies and component faults. Although the theory of DES control has been developed for almost two decades [62], only a very few applications have been reported in literature. A possible explanation is that, until recently, no quantitative analytical tool was available for design and evaluation of DES controllers. The work reported here makes use of a quantitative measure of regular languages [81] [95] [67], and is a novel application of hierarchical DES control synthesis for complex dynamical systems. The real-time implementation of the DES scheme is challenging because it requires integration of several disciplines such as systems theory, computer hardware and software, and domain knowledge of the dynamical system being controlled. The proposed hierarchical scheme is a natural choice for future-generation rotorcraft, that will need to meet more stringent handling qualities requirements in order to perform difficult tasks such as air combat, target tracking, and operating in degraded visual environments. The introduction of Rotorcraft Unmanned Aerial Vehicles (RUAV s) may raise the requirements even higher in terms of the achievable agility and precision maneuvering of rotorcraft. As a result, future-generation rotorcraft control systems should provide higher bandwidth, improved attitude quickness, less cross-coupling and better disturbance rejection than currently operational rotorcraft. Traditionally, the limits on flight control performance for rotorcraft have been more restrictive than those of high performance fixed-wing aircraft. The out-of-plane (flapping) and in-plane (lagging) motion of the rotor blades produce a number of additional dynamic modes that can couple with the rigid body motion of the fuselage and the flight control system. Rotor

24 9 dynamics can become dynamically coupled with the fuselage dynamics for high feedback gains. The air resonance phenomenon occurs when one of the lagging modes becomes very lightly damped or even unstable due to this coupling, and has been observed on helicopters with high bandwidth control systems [21]. Modern rotorcraft are complex dynamical systems which, by virtue of their missions and operating environments, endure a very high level of wear and tear during their service life [10]. Specifically, the main rotor drive system and dynamic components, including the engines, transmission and drive-shaft, experience large amounts of periodic and random vibration, which result in both high-cycle and low-cycle fatigue damage. As there is no analytical method in place to detect and quantify the extent of the fatigue damage, current procedures focus on preventative maintenance (specifically, a regimen of inspection and replacement of components based on time in service rather than their actual condition). This conservative maintenance approach, coupled with the relative complexity of the vehicle leads to higher operational costs of rotorcraft, that are significantly higher than those of equivalent fixed-wing aircraft. In addition, due to the ever-increasing workload imposed on pilots of military rotorcraft, advances in handling qualities are of considerable interest, especially in the areas of bandwidth and response quickness. Recent studies demonstrate that flight control systems with improved performance also cause an increase in the damage rate of certain rotorcraft components [70] [69]. It has been shown that controllers with increased bandwidth in the pitch and yaw axes increase the damage rate of the main rotor pitch horn on a UH-60A Black Hawk. Thus, the dual objectives of reducing operational costs and improving handling qualities may cause conflict. The incorporation of component

25 10 damage in the flight control design and optimization process is warranted, but it must be balanced with the handling qualities requirements of the mission. Another motivation of this thesis is to detect anomalies in dynamical systems (e.g., gas turbine engines) at an early stage. Traditionally, engine condition monitoring has led the way for condition-based maintenance and health management technologies because of the safety and dispatch requirements of aircraft engines [39]. Therefore, the results, conclusions, and recommendations presented in this thesis can be generalized to all types of equipment, systems, and vehicles, i.e., to EHM (Equipment Health Management), IVHM (Integrated Vehicle Health Management) and ISHM (Integrated System Health Management). Anomalous operation of a dynamical system is undesirable from the perspectives of both system operation (e.g., the engine itself) and mission management (e.g overall operation of the aircraft, where engine is just a part). Early detection of anomalies and their characterization are essential for health management, that includes prognosis of impending failures in critical components and mitigation of their detrimental effects on the system operation. For detection of these slow time scale deviations, it might be necessary to rely on the time series data generated from the available sensors and other relevant information based on user (e.g., pilot in the above example) experience. Since sufficiently accurate and computationally tractable modeling of the system is often infeasible solely based on the fundamental principles of physics, small changes in the system behavior may be inferred from both time series analysis of the sensor data and model-based information [2] [44]. By taking advantage of the available model-based information, the time series data can be converted, by phase-space partitioning, into a

26 symbol sequence [5] that, in turn, generate a finite-state machine model of the dynamical system behavior Research Objectives The objective of this research is to use mathematics of failures in complex systems to develop future generation health management and control systems that address the following issues: Reliable operation over wide range Achieving maximum performance with reduced risk Reduction of the probability of catastrophic failures and accommodation of minor faults Reduction of maintenance costs and development costs Damage reduction (with no significant loss of performance) via life extending control Early detection of incipient anomalies Real-time decision-making and increased autonomy, mimicking human intelligence To achieve the above research goals, three key areas will be explored: Probabilistic Robust Control (PRC), Damage Mitigating Control (DMC) and Discrete Event Supervisory (DES) Control as displayed in Figure 1.1. Although these topics may appear to be diverse at the first glance, they are intimately related from the perspective of plant

27 monitoring and control. For example, PRC and DMC form the backbone of the lower 12 level control strategy as seen in Figure 1.2. Their unification and supervision of the entire system under a hierarchical architecture is performed by DES. Figure 1.1 shows the overall supervisory architecture of the RSC testbed. Figure 1.2 fills in the details. The signal nomenclature of Figure 1.2 is presented in Table 1.1. Another objective of this work is to address the problem of early detection of anomalies. This thesis addresses the forward problem (i.e., analysis and comparison of sensor signals using various pattern recognition tools when the injected anomalies are a-priori known). The inverse problem (i.e., Real time early detection of anomalies by analysis of sensor signals) and its integration with the DES control architecture described above, is suggested for future work. 1.3 Background and Literature Survey State of the art in Rotorcraft Control: A feasible approach for achieving higher performance from rotorcraft flight control systems is rotor state feedback [34] [83]. This allows a full state feedback with good stability margins, but it also requires specialized sensors to measure the motion of the rotor blades. Another alternative is to use dynamic compensators based on an accurate high-order model of the coupled fuselage / rotor dynamics [38]. This approach tends to result in high-order controllers, and there will always be inherent uncertainties due to modeling errors, parametric variations, and changing operating conditions. Robust control theory allows synthesis of control systems based on a simple loworder plant model with well-defined uncertainty bounds that account for model

28 13 simplifications, non-linearity, and variations in operating conditions. Furthermore, approximate low-order plant models are more readily identified from flight test data and result in less complicated control designs. A number of simulation studies have investigated robust control methods on rotorcraft using both H -based and µ- synthesis techniques [73], and an H -based controller has been tested in flight [79]. Probabilistic Robust Control (PRC): It is well known that the demands on system stability robustness and desired nominal performance could be contradictory to each other. The deterministic worst-case robust design could cause unnecessary conservativeness and thus degrade system nominal performance. Instead of stability guarantee under worst-case uncertainties, recent results in probabilistic robust control indicate that complexity of the controller can be greatly reduced and/or system performance can be significantly improved by allowing a small risk of instability [47] [48]. Furthermore, by specifying different levels of risk at different flight regimes, the control design could obtain a trade off between robustness and nominal performance. This forms the basis of Probabilistic Robust Control (PRC) synthesis [36]. While performing various mission tasks in military rotorcraft, specifications of handling qualities [92] dictate different levels of flight control system performance. For example, if the rotorcraft is in cruise flight, the bandwidth and attitude quickness requirements are relatively low, and a low-risk / low-performance controller would be adequate. On the other hand, when performing aggressive combat tasks or

29 precision maneuvers, it may be desirable to achieve the maximum available performance. A high-risk controller might be used if there is a mechanism to recover, 14 in the event that the controller initiates instability. The upper-tier supervisory controller can govern the acceptable level of risk as well as the desired level of performance. Such a system would need to monitor the response of the vehicle to detect degradation in performance or stability, and also take into account external inputs such as the current mission task and environmental conditions. Discrete Event Supervisory (DES) Control: Traditional procedures of control synthesis are based on Continuously Varying Dynamical Systems (CVDS) theory. Classical examples of such systems can be found in the areas of of Mechanics and Electrodynamics [4], and are based on continuous time and/or discrete-time state-space models that are real-valued and evolve continuously in time. The CVDS controllers process the sensor data to generate actuator commands that drive the plant through continuously manipulated actuators. In spite of the theoretical elegance of CVDS, many systems cannot be adequately modeled by ordinary or partial differential equations. Complex human-engineered systems (e.g., aircraft, power plants, networked robotic systems) require Discrete Event Supervisory (DES) decision and control for enhanced reliability, performance, and operating range [67]. A Discrete Event System is a dynamical system that evolves according to asynchronous occurrence of certain discrete changes, called events. For example, an event may be the arrival of a customer in a queue, completion of a task of failure

30 15 of a machine in a manufacturing system, transmission of a message in a communication network, termination of a computer program, variation of a set point in a control system, etc. Thus, examples of discrete-event systems include many humanengineered systems such as computer and communication networks, robotics and manufacturing systems, computer programs, and automated traffic systems. Unlike a physical system, a discrete-event system has a discrete set of states that take symbolic values rather than real values; for example a machine is either idle, working or broken. State transitions in such systems occur at asynchronous discrete instants of time in response to events, which may also take symbolic values. Discrete Event Supervisory (DES) control is a new research area pioneered by Ramadge and Wonham [63] and subsequently extended by several researchers. Instead of processing continuous numerical data, DES controllers disable or enable certain controllable events in the physical plant based on the control specification and observed event strings. The algorithms for DES control synthesis have evolved based on the automata theory and formal languages in the discipline of Computer Science. The upper-tier supervision would be an appropriate application of DES Control. Health and Usage Monitoring System (HUMS) and Damage Mitigating Control (DMC): Another motivation that brought DES control in focus is that it is potentially very useful for Health Management and Damage Mitigating Control (DMC), also known as Life Extending Control (LEC) [68]. A related and well researched topic is carefree maneuvering (CFM) systems that use advanced

31 16 flight controls and cueing systems to avoid flight envelope limits [74]. Algorithms built into the CFM system are used to detect approaching limits, and then assist in avoiding the limit either by directly restricting control inputs through the automatic flight control system (AFCS), or by issuing a tactile or other type of cue to the pilot. Since many envelope limits are associated with structural requirements on the aircraft (e.g., torque limits, load factor limits), CFM might significantly extend the life of structural components without any appreciable loss of performance. There is also an added benefit of reducing pilot workload since CFM relieves the pilot of monitoring envelope limits. However, carefree maneuvering technology does not contribute towards life extension or damage mitigation while the aircraft is operating within the prescribed flight envelope. Damage mitigating control (DMC) is a relatively new concept in control design [68], where component damage is incorporated in the optimization of the flight control system in order to minimize damage rate. The damage level is monitored or predicted in real time or near-real time, and the controller is adjusted to reduce the damage rate as damage accumulates on the system. Previous work has demonstrated the concept of DMC for a number of aerospace applications including rocket engines [68], fixed-wing aircraft structures [13], and helicopter rotor systems [69]. Damage mitigating control may result in a tradeoff between flight control performance (in terms of handling qualities) and system damage; for example, a damage mitigating control system may cause the aircraft to revert to a degraded mode of operation if a specified level of damage is detected. Such a controller might also

32 17 reduce the rate of damage without any significant impact on handling characteristics if there is sufficient redundancy in control inputs [10] [9]. Damage mitigating control can also minimize the probability of a catastrophic failure and allow the aircraft to react intelligently to minor failures encountered as damage accumulates. The development of Health and Usage Monitoring Systems (HUMS) also provides new opportunities to reduce the operational costs of rotorcraft. In particular, the concept of condition based maintenance (CBM) uses advanced data fusion algorithms to identify damaged components so that their replacement can be based on their actual condition rather than hours of usage [28]. By integrating HUMS with the flight control system, it is possible for the flight control system to react to the damage level detected by the HUMS, as in DMC, but it is also possible that the performance of the HUMS could be augmented through the flight control system. As the HUMS detects increasing damage levels, the control system is modified to reduce the damage rate, possibly by sacrificing some performance in terms of handling qualities so that allowable aircraft tasks would be restricted. Likewise, the control system might be used to inject small disturbances in order to assist in the diagnosis of damage and anomalies as well as prognosis of potential failures. In DMC, it is necessary to model the damage progress and gather the damage information, and then decision can be made to reduce the damage and extend the life of the component without significantly reducing the performance. The information collecting and decision making process is conveniently characterized in discrete settings, which requires the concept of DES control to model the system.

33 18 Anomaly Detection: The anomaly detection algorithm [64] is built upon twotime-scale analysis of stationary behavior of dynamical systems using the principles of Symbolic Time Series Analysis (STSA) [45] [51], Information Theory [3], Automata Theory [76], and Pattern Recognition [29]. STSA captures the essential dynamical features of the physical process through phase-space partitioning. Information Theory allows modeling of incipient catastrophic failures and (possibly) chaotic behavior that is analogous to thermodynamic phase transitions. Automata Theory generates finite-state machine models of the dynamical system behavior under nominal and anomalous conditions. Pattern discovery methods infer anomalies through quantitative evaluation of the deviations in statistical patterns of the respective state machines from those under the nominal condition. These features are described in detail in subsequent chapters. A review of engine monitoring systems for commercial aviation was conducted and reported by Tumer and Bajwa in [90]. Engine performance monitoring has proven effective in providing early warning against impending failures; however, high number of false alarms, has caused reluctance among commercial users to rely on the results. Space shuttle and helicopters have more advanced engine monitoring capabilities than commercial aircraft. Although the cost of implementing Health and Usage Monitoring System (HUMS) is still high, the benefits have been steadily increasing. Tumer and Bajwa [90] identified two practical problems facing the Engine Health Management (EHM) community:

34 19 too many false alarms insufficient sampling and data storage On-going research areas in the field of Engine Health Management (EHM) are: anomaly detection replacing standard threshold method with feature extraction automated fault diagnosis combination of theory, knowledge, and test information to develop more reliable fault libraries combination of rule-based (e.g., expert system) diagnosis with Artificial Neural Network (ANN) or Fuzzy Logic (FL) knowledge discovery Health monitoring of gas turbine engines can be viewed as a class of slow time scale problem [93]. For gas turbine engines, health condition of an engine changes in hundreds of hours, whereas the engine runs at a much faster time scale, usually in the order of fractions of seconds. Identification of the current state of the engine health is very important for maintenance engineers because and necessary repairs must be carried out before the engine becomes permanently non-operable. Thus, it is essential to monitor slow-time-scale anomalies for gas turbine engines from the time series data of the engine response. A generic gas turbine engine simulation [27] test bed has been used to validate the anomaly detection techniques.

35 Contributions of the Proposed Research The primary contribution of this thesis is to provide an integrated and comprehensive architecture for health management and control of complex dynamical systems. The core areas are high performance and reliability over a wide range of operation. The lower-tier controller is designed using a probabilistic robust control (PRC) and the Damage Mitigating Control (DMC). In the PRC approach, by allowing different levels of risk under different flight conditions, the control system can achieve the desired trade off between stability robustness and nominal performance [36] [35]. In the DMC approach, component damage is incorporated in the control law to minimize damage rate and extend the operational life of the system. The DES based upper-tier supervisory controller monitors the system response and based on the mission requirements ( i.e., performance metrics, level of damage mitigation etc) switches between various lower level controllers. The core idea of this thesis is to design a framework where the upper level DES mimics human intelligence and chooses what is best for the system and the mission requirements. The proposed concept of integrated health management and hierarchical control has not apparently been reported in open literature. This thesis will primarily focus on Probabilistic Robust control and the details of implementation are provided in later chapter. Damage Mitigating Control work has been primarily taken from [9] [10] [11]. It has been provided in Appendix B for the sake of completion. Appendix B deals with the design and implementation of a damage mitigating control system for the heave and rotor speed degrees of freedom of a military helicopter, which has been previously attempted with a more complex controller designed

36 21 using H synthesis [10] [9]. Damage is measured in terms of the length of a crack in the main bevel pinion of the main rotor transmission. Various sets of maneuvers are simulated to examine the time histories of damage and other relevant flight conditions. The impact of damage mitigation on handling qualities is investigated by evaluating the controller according the applicable sections of the ADS-33E handling qualities specification [92]. Following are the specific contributions of this dissertation: Design and development of an extensive bank of lower level lateral directional probabilistic robust controllers for the UH-60 black Hawk Rotorcraft. These controllers have a wide operating range ( knots) and their high performance results from risk adjusted robust control synthesis instead of traditional robust control synthesis procedure. The reliability of the overall control architecture stems from DES based hierarchical scheme under which both DMC and PRC work in tandem. The proposed scheme addresses the diverse issues of rotorcraft health management and high performance, reliable, wide range control in single integrated setting. An effective event generation scheme (C/D convertor) is proposed which converts the continuous-time plant model data into discrete asynchronous events. This area of DES control has by and large been ignored. This work focuses on the implementation of recently developed symbolic dynamics and wavelets based techniques for event generation [87] coupled with traditional frequency based methods.

37 22 A Discrete Event Supervisory (DES) control system with a two-layer hierarchical architecture is designed to coordinate the operations of a twin-engine propulsion system. Each engine is operated under a continuously varying feedback control system that maintains the specified performance under the supervision of a local (lower level) discrete-event controller for condition monitoring and life extension. The two engines are individually controlled to achieve enhanced performance and reliability, as necessary for fulfilling the mission objectives. A global (upper level) DES controller is designed for load balancing and overall health management of the propulsion system. A comparison of different pattern recognition algorithms to identify slow time scale anomalies for health management of aircraft gas turbine engines. A new tool of anomaly detection, based on STSA and Information Theory [64], is compared with traditional pattern recognition tools of Principal Component Analysis (PCA) and Artificial Neural Network (ANN). Time series data of the observed variables on the fast time scale are analyzed at slow time scale epochs for early detection of anomalies. 1.5 Organization This Thesis is organized into eight chapters including the present one and five appendices. Chapter 2 covers upper level Control based on Discrete event systems and the notion of language measure. Chapter 3 reviews anomaly detection methods based on Symbolic Time Series Analysis (STSA). Chapter 4 describes the Probabilistic Robust

38 23 Control (PRC) synthesis of rotorcraft. Chapter 5 deals with the issue of hierarchical control of rotorcraft. Chapter 6 deals with hierarchical control of the propulsion system of an aircraft. In chapter 7 compares the efficacy of symbolic time series analysis with traditional pattern recognition methods for anomaly detection. The thesis is concluded in Chapter 8 along with suggestions for future work. Appendix A elaborates on the Rotorcraft Simulation and Control (RSC) test-bed. Appendix B covers the topic of Damage Mitigating Control (DMC). Appendix C covers the background theoretical material related to Symbolic Time Series Analysis (STSA). Appendix D and Appendix E list the nomenclature and acronyms used in the thesis respectively. Figures and tables are placed at the end of each chapter.

39 24 Table 1.1. Nomenclature of Signals in Figure 1.2 Signal Description S 1 S 2 S 3 S 4 S 5 S 6 S 7 S 8 S 9 S 10 S 11 S 12 S 12 External commands treated as uncontrollable events Other information Observable events Disabling controllable events Input to Command generator from DES Input to Control Switching module from command generator Input to the Plant from Control Switching module Reference Signals Plant / Controller output fed to the Analysis module Output of the Analysis module fed to the Event generator Output of the Plant/Controller fed to the Control Switching module Output of the Analysis module fed to the Control Switching module Output of the Event generator fed to the DES plant model

40 25 Fig Overall architecture

41 26 Fig Detailed architecture

42 27 Chapter 2 Discrete Event Systems 2.1 Introduction Discrete event systems belong to a special class of dynamical systems. The states of a discrete event system may take discrete (or symbolic) values and change only at (possibly asynchronous) discrete instants of time, in contrast to the familiar continuously varying dynamical systems of the physical world, which can be modelled by differential or difference equations. The dynamics of many human-engineered systems evolve asynchronously in time via complex interactions of various discrete-valued events with continuously varying physical processes. The relatively young discipline of discrete event systems has undergone rapid growth over the last three decades with the evolution of human engineered complex systems, such as integrated control and communication systems, distributed sensing and monitoring of large-scale engineering systems, manufacturing and production systems, software fault management, and military Command, Control, Computer, Communication, Intelligence, Surveillance, and Reconnaissance (C 4 ISR) systems. The discipline of discrete event systems was initiated with simulation of humanengineered processes about four decades ago in the middle of nineteen sixties. The art of discrete event simulation emerged with the development of a simulation software package, called GPSS, that was followed by numerous other software simulation tools, such as SIMSCRIPT II.5, SLAM II, and SIMAN [50]. Shortly thereafter, computer

43 28 scientists and control theorists entered the field and brought in theoretical concepts of languages and automata in modelling discrete event systems. In the late nineteen sixties, Arbib [42] showed how algebraic methods could be used to explore the structure of finite automata to model dynamical systems. Around that time, computer scientists focused on formal languages, automata theory, and computational complexity for application of language-theoretic concepts (e.g., regular expressions and context-free grammars) in software development including design of compilers and text processors [33]. In the late nineteen seventies and early nineteen eighties, Ho and co-workers introduced the concept of finite perturbation in discrete event systems for modelling and analysis of humanengineered systems [31]. So far, no concrete theoretical concept and mathematical tools had been available for analysis and synthesis of discrete event control systems. The concept of discrete event supervisory (DES) control was first introduced in the seminal paper of Ramadge and Wonham [62] and this important paradigm has been subsequently extended by other researchers (for example, see citations in [46] [14], and the October 2000 issue of Part B of IEEE Transactions on Systems, Man, and Cybernetics). These efforts have led to the evolution of a new discipline in decision and Control, called Supervisory Control Theory (SCT), that requires partitioning the discrete-event behavior of a physical process, called the plant, into legal and illegal categories. The legal behavior of the plant dynamics is modelled by a deterministic finite-state automaton, abbreviated as DFSA in the sequel. The DFSA model is equivalent to a regular language that is built upon an alphabet of finitely many events; the event alphabet is partitioned into subsets of controllable events (that can be disabled) and uncontrollable events (that

44 29 cannot be disabled). Based on the regular language of an unsupervised plant, SCT synthesizes a DES controller as another regular language, having the common alphabet with the plant language, that guarantees restricted legal behavior of the controlled plant based on the desired specifications. Instead of continuously handling numerical data, DES controllers are designed to process event strings to disable certain controllable events in the physical plant. A number of algorithms for DES control synthesis have evolved based on the automata theory and formal languages relying on the disciplines of Computer Science and Control Science. In general, a supervised plant DFSA is synthesized as a parallel composition of the unsupervised plant DFSA and a supervisor DFSA. The supervised plant DFSA yields a sublanguage of the unsupervised plant language, which enables restricted legal behavior of the supervised plant [62] [46] [14]. These concepts have been extended to several practical applications, including hierarchical Command, Control, Communications, and Intelligence (C 3 I) systems [60]. Apparently, there have been no quantitative methods for evaluating the performance of supervisory controllers and establishing thresholds for their performance. The concept of permissiveness has been used in DES control literature [14] [46] to facilitate qualitative comparison of DES controllers under the language controllability condition. Design of maximally permissive DES controllers has been proposed by many researchers based on different conditions. However, maximal permissiveness does not necessarily imply best performance of the supervised plant from the perspectives of plant operational objectives. For example, in the travelling salesman problem, a maximally permissive supervisor would not usually yield the least expensive way of visiting the scheduled cities and returning to the starting point because no quantitative measure of

45 30 performance is addressed in this type of supervisor design. Hence, permissiveness is not an adequate measure of performance. The controlled behavior of a given plant DFSA under different supervisors could vary if they are designed to meet different control specifications. As such the respective controlled sublanguages of the plant language form a partially ordered set that is not necessarily totally ordered. Since the literature on DES control does not apparently provide a language measure, it may not be possible to quantitatively evaluate the performance of a DES controller. Therefore, it is necessary to formulate a mathematically rigorous concept of language measure(s) to quantify performance of individual supervisors such that the measures of controlled plant behavior, described by a partially ordered set of controlled sublanguages, can be structured to form a totally ordered set. From this perspective, it is necessary to formulate a signed real measure µ that can be assigned to sublanguages of the regular language of the unsupervised plant to achieve the following objective [95] [81] [66]: Given that the relation induces a partial ordering on a set of controlled sublanguages {L(S j /G), j = 1,..., N} of the plant language L(G) under supervisors whose languages are {L(S j ), j = 1,..., N}, the language measure µ induces a total ordering on {µ(l(s j /G))}. The major motivation here is to present a signed real measure of regular languages, which can be used for quantitative analysis and synthesis of DES control systems for physical plants, instead of relying on permissiveness as the (qualitative) performance

46 31 index. Construction of the proposed language measure follows Myhill-Nerode Theorem [32] and Hahn Decomposition Theorem [71], where a state-based partitioning of the (unsupervised) plant language yields equivalence classes of finite-length event strings. Each marked state is characterized by a signed real value that is chosen based on the designer s perception of the state s impact on the system performance. Conceptually similar to the conditional probability, each event is assigned a cost based on the state at which it is generated. This procedure permits a string of events, terminating on a good (bad) marked state, to have a positive (negative) measure; each of the remaining strings of events in the language, terminating on an unmarked state, is assigned zero measure. In this setting, a supervisor can be designed with the goal of eliminating bad strings and retaining good strings by disabling controllable event(s) at selected states. Different supervisors may attempt to achieve this goal in different ways and generate a partially ordered set of controlled languages. The language measure then creates a total ordering on the performance of the controlled languages, which provides a precise quantitative comparison of the controlled plant behavior under different supervisors. The language measure has been used as a performance index for analysis and synthesis of DES control systems. For example, Fu et al. [26] [24] [25] have proposed optimal control and robust optimal control of regular languages where a state-based optimal control policy is obtained by selectively disabling controllable events to maximize the measure of the controlled plant language. The concept of DES control has been validated by laboratory experimentation on mobile robots as well as on simulation test beds for applications to gas turbine engines, software fault management, C 3 I systems, and control of malicious executables in computers.

47 32 This chapter is organized in three sections including the present one. In Section 2.2, theoretical foundations of the recently developed Language Measure Theory for DES control [81] [95] [67] are reviewed. Section 2.3 summarizes the theory of optimal DES control [65]. 2.2 Language Measure This section reviews the previous work on language measure [81] [95] [67]. It provides the background information necessary to develop a performance index for the optimal DES control law. Let the dynamical behavior of a physical plant be modeled as a deterministic finite state automaton (DFSA) G i (Q, Σ, δ, q i, Q m ) with Q = n and Σ = m; the Kleene closure is the set of all finite-length strings of events including the empty string ε ; the (possibly partial) function of δ : Q Q represents state transitions and ˆδ : Q Q is an extension of δ; and Qm Q is the set of marked (or accepted) states. Definition 1 : A DFSA G i, initialized at q i Q, generates the language L(G i ) {s Σ : ˆδ(q i, s) Q} and its marked sublanguage L m (G i ) {s Σ : ˆδ(q i, s) Q m }. Now, it is possible to construct a signed real measure µ : 2 L(G i ) R (, ) that allows quantitative evaluation of every event string s L(G i ) based on state-based decomposition of L(G i ) into null (i.e., L 0 (G i )), positive (i.e., L + m (G i ) ), and negative (i.e., L m (G i ) ) sublanguages of L(G i ). Definition 2 : The language of all strings that, starting at q i Q, terminates at q j Q, is denoted as L(q j, q i ). That is, L(q j, q i ) {s L(G i ) : ˆδ(q i, s) = q j }.

48 Definition 3 : The characteristic function that assigns a signed real weight to state-partitioned sublanguages L(q j, q i ) is defined as: χ : Q [ 1, 1] such that 33 χ j χ(q j ) [ 1, 0) if q j Q m {0} if q j Q m independent of q i (0, 1] if q j Q + m The (n 1) characteristic vector is denoted as: χ [χ 1 χ 2... χ n ] T Definition 4 : The event cost is conditioned on a DFSA state at which the event is generated, and is defined as: π : Σ Q [0, 1] such that q j Q, σ k Σ, s Σ, π[σ k q j ] = 0 if δ(q j, σ k ) is undefined; π[ɛ q j ] = 1; π[σ k q j ] π jk [0, 1); Σ k π jk < 1; π[σ s k q j ] = π[σ k q j ] π[s δ(q j, σ k )]. The (n m) event cost matrix is denoted as: Π = [ πij ]. Definition 5 : The state transition cost of DFSA is defined as a function π : Q Q [0, 1) such that q j, q k Q, π(q k q j ) = σ Σ:δ(q j,σ)=q k π(σ q j ) π jk

49 and π jk = 0 if [σ Σ : δ(q j, σ)] =. The n n state transition cost matrix, denoted as 34 Π, is defined as: Π = π 11 π 12 π 1n π 21 π 22 π 2n π n1 π n2 π nn Now we define the measure of any sublanguage of the plant language L(G i ) in terms of the signed characteristic function χ and the non-negative event cost π. Definition 6 : The signed real measure µ of a singleton string set {s} is defined as: µ({s}) χ(q j ) π(s q i ) s L(q j, q i ) L(G i ). The signed real measure of L(q j, q i ) is defined as: µ(l(q j, q i )) µ({s}) s L(q j,q i ) as The signed real measure of a DFSA G i, initialized at the state q i Q, is defined µ i µ(l(g i )) = j µ(l(q j, q i )). The n 1 real signed measure vector is denoted as: µ [µ 1 µ 2 µ n ] T

50 It has been shown in [81] [95] [67]that the measure of the language L(G i ), where G i = (Q, Σ, δ, q i, Q m ) can be expressed as: µ i = j π ij µ j + χ i. Equivalently, in vector notation: µ = Π µ+ χ. Since Π is a contraction operator, the measure vector µ is uniquely determined as: µ = [I Π] 1 χ. 35 In the formulation of the optimal control policy for a plant automaton G, the performance index vector is defined as the measure vector µ s = [I Π s ] 1 χ of the controlled plant sublanguage L(S/G) controlled under the supervisor S without incorporating the cost of event disabling. Note that the transition cost matrix Π s of the controlled plant S/G is obtained by disabling certain controllable events at selected states and hence Π s Π (elementwise). 2.3 Optimal Discrete Event Supervisory Control This section reviews the theory of optimal DES control [65] based on a specified measure with no event disabling cost. In gas turbine engine application, the cost of disabling controllable events (e.g., redistribution of thrust between two engines, and nozzle area reduction for individual engines) is set to zero as these manipulations do not require any special efforts. The state-based optimal control policy is obtained by selectively disabling controllable events to maximize the measure of the controlled plant language without any constraints. In each iteration, the optimal control algorithm attempts to disable all controllable events leading to bad marked states and enable all controllable events leading

51 36 to good marked states. It has been also shown in [65] that computational complexity of the control synthesis is polynomial in the number of plant states. Let S {S 0, S 1,, S n } be a finite set of all supervisory control policies for the unsupervised plant automaton G where S 0 is the null controller (i.e., no event being disabled) implying that L(S 0 /G) = L(G). Therefore,Π(S 0 ) Π 0 = Π plant, i.e., the state transition cost matrix Π S 0 is equal to the Π matrix of unsupervised plant automaton G. For a supervisor S i, i {1, 2,, n}, the control policy selectively disables certain controllable events, and therefore the following element-wise inequality holds: Π k Π(S k ) Π 0 and L(S k /G) L(G) S k S Definition 7 : For any supervisor S S and any language measure vector ν R n, the affine operator T (S) : R n R n is defined as:t (S) ν = Π ν + χ. The following propositions that are proved in [65] are restated here for the sake of completeness. Proposition 1 : S S, T (S) is a contraction operator and there exists a unique measure vector µ(s) such that µ(s) = T (S) µ(s). The fixed point of the contraction operator T (S k ) is: µ k = [I Π k ] 1 χ where µ k µ(s k ) and Π k Π(S k ). Furthermore, the operator [I Π k ] has a real positive determinant. Proposition 2 : If µ k j = min l {1,2,,n} µk l. If µk j 0 then χ j 0 ; and if µ k j < 0 then χ j < 0. Likewise, if µk j = max l {1,2,,n} µk l. If µk j 0 then χ j 0 ; and if µ k j > 0 then χ j > 0.

52 37 Proposition 3 : Given Π(S 0 ) Π 0 = Π plant and µ k = [I Π k ] 1 χ, let Π k+1 be generated from Π k for k 0 as follows: i, j {1, 2,, n}, the ij th element of Π k+1 is modified as: π k+1 ij π k+1 ij = π k+1 ij if µ k j > 0 if µ k j = 0 and Πk Π 0 k π k+1 ij if µ k j < 0 Then, µ k+1 µ k element-wise and equality holds if and only if Π k+1 = Π k. Moreover, if µ k j 0 and Πk+1 is generated from Π k by disabling controllable events that lead to state q j, then µ k+1 j < 0. Proposition 4 : Iteration of the algorithm in Proposition 3 leads to an optimal state transition cost matrix Π that maximizes the performance vector µ = [I Π ] 1 χ element-wise. Proposition 5 : The control policy induced by the Π matrix is unique in the sense that the controlled language is most permissive among all controller(s) having the best performance. The algorithm for synthesis of the optimal control policy is summarized as follows: Let G be the DFSA plant model without any constraint of operational specifications. Let the state transition cost matrix of the unsupervised plant be: Π plant R n n and the characteristic vector be: χ j [ 1, 1] j {1, 2,, n}. Starting with k = 0 and Π 0 Π plant, the control policy is constructed by the following two-step procedure:

53 38 Step 1 : For every state q j for which µ 0 j < 0, disable controllable events leading to q j. Now, Π 1 = Π 0 0, where 0 0 is composed of event costs corresponding to all controllable events that have been disabled at k = 0. Step 2 : Starting with k = 1, if µ k j 0, re-enable all controllable events leading to q j, which were disabled in Step 1. The cost matrix is updated as: Π k+1 = Π k + k for k 1, where k 0 is composed of event costs corresponding to all currently reenabled controllable events. The iteration is terminated if no controllable event leading to q j remains disabled for which µ k j 0. At this stage, the optimal performance is µ = [I Π ] 1 χ. The theory developed in this chapter has been applied to simulation test bed of a generic gas turbine engine, similar to that reported in [20] (see chapter 6). A DES control system with a two-layer hierarchical architecture is proposed and designed to coordinate the operations of a twin-engine propulsion system. Each engine is operated under a continuously varying feedback control system that maintains the specified performance under the supervision of a local (lower level) discrete-event controller for condition monitoring and life extension. The two engines are individually controlled to achieve enhanced performance and reliability, as necessary for fulfilling the mission objectives. A global (upper level) DES controller is designed for load balancing and overall health management of the propulsion system.

54 39 Chapter 3 Symbolic Time Series Analysis for Anomaly Detection 3.1 Introduction This chapter covers the second aspect of the topic of health management, i.e., Anomaly Detection. A novel anomaly detection concept is proposed for complex dynamical systems using tools of Symbolic Time Series Analysis (STSA), Finite State Automata, and Pattern Recognition, where time series data of the observed variables on the fast time-scale are analyzed at slow time-scale epochs for early detection of (possible) anomalies [64]. Anomaly in a dynamical system is defined as a deviation from its nominal behavior and can be associated with parametric or non-parametric changes that may gradually evolve in the system. Early detection of anomalies in complex dynamical systems is essential not only for prevention of cascading catastrophic failures, but also for enhancement of performance and availability [53]. For anomaly detection, it might be necessary to rely on time series data generated from sensors and other sources of information [2], because accurate and computationally tractable modelling of complex system dynamics is often infeasible solely based on the fundamental principles of physics. This symbolic dynamics based anomaly detection tool has been tested on the Gas Turbine Simulation (GTS) test-bed.

55 40 This chapter formulates a novel concept for detection of slowly evolving anomalies in complex dynamical systems. Often such dynamical systems are either self-excited or can be stimulated with a priori known exogenous inputs to recognize (possible) anomaly patterns from the observed stationary response. Early detection of an anomaly (i.e., small parametric or non-parametric changes) has motivated formulation and validation of the proposed Symbolic Dynamic approach to pattern recognition, which is based on the following assumptions: The system behavior is stationary at the fast time scale of the process dynamics; An observable non-stationary behavior of the dynamical system can be associated with anomaly(ies) evolving at a slow time scale. The theme of anomaly detection, formulated in this chapter, is built upon the concepts of STSA [51] [45], Finite State Automata [32], and Pattern Recognition [29] as a means to qualitatively describe the (fast-time-scale) dynamical behavior in terms of symbol sequences [5] [7]. Appropriate phase-space partitioning of the dynamical system yields an alphabet to obtain symbol sequences from time series data [2] [19] [43]. Then, tools of Computational Mechanics [16] are used to identify statistical patterns in these symbolic sequences through construction of a (probabilistic) finite-state machine from each symbol sequence. Transition probability matrices of the finite state machines, obtained from the symbol sequences, capture the pattern of the system behavior by information compression. For anomaly detection, it suffices that a detectable change in the pattern represents a deviation of the nominal behavior from an anomalous one. The state probability vectors, which are derived from the respective state transition matrices

56 41 under the nominal and an anomalous condition, yield a vector measure of the anomaly, which provides more information than a scalar measure such as the complexity measure [76]. In contrast to the ɛ-machine [16] [76] that has an a priori unknown structure and yields optimal pattern discovery in the sense of mutual information [22] [15], the state machine adopted in this chapter has an a priori known structure that can be freely chosen. Although the proposed approach is suboptimal, it provides a common state machine structure where physical significance of each state is invariant under changes in the statistical patterns of symbol sequences. This feature allows unambiguous detection of possible anomalies from symbol sequences at different (slow-time) epochs. The proposed approach is apparently computationally faster than the ɛ-machine [76], because of significantly fewer number of floating point arithmetic operations. These are the motivating factors for introducing this new anomaly detection concept that is based on a fixed-structure fixed-order Markov chain, called the D-Markov machine in the sequel. The anomaly detection problem is separated into two parts [82]: (i) forward problem of Pattern Discovery to identify variations in the anomalous behavior patterns, compared to those of the nominal behavior; and (ii) inverse problem of Pattern Recognition to infer parametric or non-parametric changes based on the learnt patterns and observed stationary response. The inverse problem could be ill-posed or have no unique solution. That is, it may not always be possible to identify a unique anomaly pattern based on the observed behavior of the dynamical system. Nevertheless, the feasible range of parameter variation estimates can be narrowed down from the intersection of the information generated from inverse images of the responses under several stimuli.

57 42 It is envisioned that complex dynamical systems will acquire the ability of selfdiagnostics through usage of the proposed anomaly detection technique that is analogous to the diagnostic procedure employed in medical practice in the following sense. Similar to the notion of injecting medication or inoculation on a nominally healthy patient, a dynamical system would be excited with known stimuli (chosen in the forward problem) in the idle cycles for self diagnosis and health monitoring. The inferred information on health status can then be used for the purpose of self-healing control or life-extending control [96]. This chapter focuses on the forward problem. The chapter is organized in five sections including the present one. Section 3.2 briefly introduces the notion of nonlinear time series analysis. Section 3.3 provides a brief overview of symbolic dynamics and encoding of time series data. Section 3.4 presents two ensemble approaches for statistical pattern representation. It also presents information extraction based on the ɛ-machine [16] and the D-Markov machine, as well as their comparison from different perspectives. Section 3.5 presents the notion of anomaly measure to quantify the changing patterns of anomalous behavior of the dynamical system form the information-theoretic perspectives, followed by an outline of the anomaly detection procedure and the application to the Gas Turbine Simulation(GTS) test-bed. 3.2 Nonlinear Time Series Analysis This section presents nonlinear time series analysis (NTSA) that is needed to extract relevant physical information on the dynamical system from the observed data. NTSA techniques are usually executed in the following steps [2]:

58 43 1. Signal Separation: The (deterministic) time-dependent signal {y(n) : n N}, where N is the set of positive integers, is separated from noise, using time-frequency and other types of analysis. 2. Phase Space Reconstruction: Based on the Takens Embedding theorem [84], time lagged or delayed variables are used to construct the state vector x(n) in a phase space of dimension d E (which is diffeomorphically equivalent to the attractor of the original dynamical system) as follows: x(n) = [y(n), y(n + T ),, y(n + (d E 1)T )] (3.1) where the time lag T is determined using mutual information; and one of the ways to determine d E is the false nearest neighbors test [2]. 3. Signal Classification: Signal classification and system identification in nonlinear chaotic systems require a set of invariants for each subsystem of interest followed by comparison of observations with those in the library of invariants. The invariants are properties of the attractor and could be independent of any particular trajectory. These invariants can be divided into two classes: fractal dimensions and Lyapunov exponents. Fractal dimensions characterize geometrical complexity of dynamics (e.g., spatial distribution of points along a system orbit); and Lyapunov exponents describe the dynamical complexity (e.g., stretching and folding of an orbit in the phase space) [57].

59 44 4. Modeling and Prediction: This step involves determination of the parameters of the assumed model of the dynamics, which is consistent with the invariant classifiers (e.g., Lyapunov exponents, and fractal dimensions). The first three steps show how chaotic systems may be separated from stochastic ones and, at the same time, provide estimates of the degrees of freedom and the complexity of the underlying dynamical system. Based on this information, Step 4 formulates a state-space model that can be used for prediction of anomalies and incipient failures. The functional form often used in this step, includes orthogonal polynomials and radial basis functions. This chapter has adopted an alternative class of discrete models inspired from Automata Theory, which is built upon the principles of STSA as described in the following section. 3.3 Symbolic Time Series Analysis (STSA) and Encoding This section introduces the concept of STSA and its usage for encoding nonlinear system dynamics from observed time series data. Let a continuously varying physical process be modelled as a finite-dimensional dynamical system in the setting of an initial value problem: dx(t) dt = f(x(t), θ(t s ); x(0) = x 0, (3.2) where t [0, ) denotes the (fast-scale) time; x R n is the state vector in the phase space; and θ R l is the (possibly anomalous) parameter vector varying in (slow-scale) time t s. Sole usage of the model in Eq. (3.2) may not always be feasible due to unknown parametric and non-parametric uncertainties and noise. A convenient way of learning the

60 dynamical behavior is to rely on the additional information provided by (sensor-based) time series data [2] [7]. A tool for behavior description of nonlinear dynamical systems is based on the concept of formal languages for transitions from smooth dynamics to a discrete symbolic description [5]. The phase space of the dynamical system in Eq. (3.2) is partitioned into a finite number of cells, so as to obtain a coordinate grid of the space. A compact (i.e., closed and bounded) region Ω R n, within which the (stationary) motion under the specific exogenous stimulus is circumscribed, is identified. Encoding of Ω is accomplished by introducing a partition B {B 0,, B m 1 } consisting of m mutually exclusive (i.e., B j B k = j k), and exhaustive (i.e., m 1 j=0 B j = Ω) cells. The dynamical system describes an orbit by the time series data as: O {x 0, x 1, x k }, x i Ω, which passes through or touches the cells of the partition B. Let us denote the cell visited by the trajectory at a time instant as a random variable S that takes a symbol value s A. The set A of m distinct symbols that label the partition elements is called the symbol alphabet. Each initial state x 0 Ω generates a sequence of symbols defined by a mapping from the phase space into the symbol space as: x 0 s i0 s i1 s i2 s ik (3.3) 45 The mapping in Eq. (3.3) is called Symbolic Dynamics as it attributes a legal (i.e., physically admissible) symbol sequence to the system dynamics starting from an initial state. (Note: A symbol alphabet A is called a generating partition of the phase space Ω if every legal symbol sequence uniquely determines a specific initial condition

61 46 x 0, i.e., every symbolic orbit uniquely identifies one continuous space orbit.) Figure 3.1 pictorially elucidates the concepts of partitioning a finite region of the phase space and mapping from the partitioned space into the symbol alphabet. This represents a spatial and temporal discretization of the system dynamics defined by the trajectories. Figure 3.1 also shows conversion of the symbol sequence into a finite-state machine as explained in later sections. Symbolic dynamics can be viewed as coarse graining of the phase space, which is subjected to (possible) loss of information resulting from granular imprecision of partitioning boxes, measurement noise and errors, and sensitivity to initial conditions. However, the essential robust features (e.g., periodicity and chaotic behavior of an orbit) are expected to be preserved in the symbol sequences through an appropriate partitioning of the phase space [5]. Although the theory of phase-space partitioning is well developed for one-dimensional mappings, very few results are known for two and higher dimensional systems [7]. 3.4 Pattern Identification Given the intricacy of phase trajectories in complex dynamical systems, the challenge is to identify their patterns in an appropriate category by using one of the following two alternative approaches: The single-item approach, which relies on Kolmogorov Chiatin (KC) complexity, also known as algorithmic complexity [15], for exact pattern regeneration

62 The ensemble approach, which regards the pattern as one of many possible experimental outcomes, for estimated pattern regeneration. 47 While the single-item approach is common in coding theory and computer science, the ensemble approach has been adopted in this chapter due to its physical and statistical relevance. As some of the legal symbol sequences may occur more frequently than others, a probability is attributed to each observed sequence. The collection of all legal symbol sequences S M S 2 S 1 S 0 S 1 S N, N, M = 0, 1, 2, defines a stochastic process that is a symbolic probabilistic description of the continuous system dynamics. Let us symbolically denote a discrete-time, discrete-valued stochastic process as: S, S 2 S 1 S 0 S 1 S 2 (3.4) where each random variable S i takes exactly one value in the (finite) alphabet A of m symbols (see Section 3.3). The symbolic stochastic process S is dependent on the specific partitioning of the phase space and is non-markovian, in general. Even if a partitioning that makes the stochastic process a Markov chain exists, identification of such a partitioning is not always feasible because the individual cells may have fractal boundaries instead of being simple geometrical objects. In essence, there is a trade-off between selecting a simple partitioning leading to a complicated stochastic process, and a complicated partitioning leading to a simple stochastic process. Recent literature has reported a comprehensive numerical procedure for construction phase-space partitions from the time series data [43]. Having defined a partition of the phase space, the time series data is converted to a symbol sequence that, in turn, is used for construction of

63 48 a finite state machine using the tools of Computational Mechanics [16] as illustrated in Figure 3.1. This chapter considers two alternative techniques of finite-state machine construction from a given symbol sequence S: (i) the ɛ-machine formulation [76]; and (ii) a new concept based on D th order Markov chains, called the D-Markov machine, for identifying patterns based on time series analysis of the observed data [64]. Both techniques rely on information-theoretic principles (see C.1) and are based on Computational Mechanics [16] The ɛ-machine Like Statistical Mechanics [23] [7], Computational Mechanics is concerned with dynamical systems consisting of many partially correlated components. Whereas Statistical Mechanics deals with the local space-time behavior and interactions of the system elements, Computational Mechanics relies on the joint probability distribution of the phase-space trajectories of a dynamical system. The ɛ-machine construction [16] [76] makes use of the joint probability distribution to infer the information processing being performed by the dynamical system. This is developed using the statistical mechanics of orbit ensembles, rather than focusing on the computational complexity of individual orbits. Let the symbolic representation of a discrete-time, discrete-valued stochastic process be denoted by: S S 2 S 1 S 0 S 1 S 2 as defined earlier in Section 3.4. At any instant t, this sequence of random variables can be split into a sequence S t of the past and a sequence S t of the future. Assuming conditional stationarity of the symbolic

64 49 process S (i.e., P [ S t S t = s ] being independent of t), the subscript t can be dropped to denote the past and future sequences as S and S, respectively. A symbol string, made of the first L symbols of S, is denoted by S L. Similarly, a symbol string, made of the last L symbols of S, is denoted by S L. Prediction of the future S necessitates determination of its probability conditioned on the past S, which requires existence of a function ɛ mapping histories s to predictions P ( S s ). In essence, a prediction imposes a partition on the set S of all histories. The cells of this partition contain histories for which the same prediction is made and are called the effective states of the process under the given predictor. The set of effective states is denoted by R; a random variable for an effective state is denoted by R and its realization by ρ. The objective of ɛ-machine construction is to find a predictor that is an optimal partition of the set S of histories, which requires invoking two criteria in the theory of Computational Mechanics [18]: 1. Optimal Prediction: For any partition of histories or effective states R, the conditional entropy (see C.1) H[ S L R] H[ S L S ], L N, S S, is equivalent to remembering the whole past. Effective states R are called prescient if the equality is attained L N. Therefore, optimal prediction needs the effective states to be prescient. 2. Principle of Occam Razor: The prescient states with the least complexity are selected, where complexity is defined as the measured Shannon information of the

65 50 effective states: H[R] = ρ R P (R = ρ) log P (R = ρ) (3.5) Equation (3.5) measures the amount of past information needed for future prediction and is known as Statistical Complexity denoted by C µ (R) (see C.1). For each symbolic process S, there is a unique set of prescient states known as causal states that minimize the statistical complexity C µ (R). Let S be a (conditionally) stationary symbolic process and S be the set of histories. Let a mapping ɛ : S Υ( S ) from the set S of histories into a collection Υ( S ) of measurable subsets of S be defined as: Γ Υ( S ), ɛ( s ) { s S such that P ( S Γ S = s ) = P ( S Γ S = s )} (3.6) Then, the members of the range of the function ɛ are called the causal states of the symbolic process S. The i th causal state is denoted by q i and the set of all causal states by Q Υ( S ). The random variable corresponding to a causal state is denoted by Q and its realization by q. Given an initial causal state and the next symbol from the symbolic process, only successor causal states are possible. This is represented by legal transitions among the causal states, and the probabilities of these transitions. Specifically, the probability of

66 51 transition from state q i to state q j on a single symbol s is expressed as: T (s) ij ( S ) 1 = P = s, Q = qj Q = q i q i, q j Q (3.7) T (s) ij s A q j Q = 1 (3.8) The combination of causal states and transitions is called the ɛ-machine (also known as the causal state model [76]) of a given symbolic process. Thus, the ɛ-machine represents the way in which the symbolic process stores and transforms information. It also provides a description of the pattern or regularities in the process, in the sense that the pattern is an algebraic structure determined by the causal states and their transitions. The set of labelled transition probabilities can be used to obtain a stochastic matrix [6] given by: T = s A T s where the square matrix T s is defined as: T s = [T ij s ] s A. Denoting p as the left eigenvector of T, corresponding to the eigenvalue λ = 1, the probability of being in a particular causal state can be obtained by normalizing p l1 = 1. A procedure for construction of the ɛ-machine is outlined below. The original ɛ-machine construction algorithm is the subtree-merging algorithm as introduced in [16] [18]. The default assumption of this technique was employed by Surana et al. [82] for anomaly detection. This approach has several shortcomings, such as lack of a systematic procedure for choosing the algorithm parameters, may return non-deterministic causal states, and also suffers from slow convergence rates. Recently, Shalizi et al. [76] have developed a code known as Causal State Splitting Reconstruction (CSSR) that is based on state splitting instead of state merging as was done in the earlier algorithm of subtree-merging [16]. The CSSR algorithm starts with a

67 52 simple model for the symbolic process and elaborates the model components only when statistically justified. Initially, the algorithm assumes the process to be independent and identically distributed (iid) that can be represented by a single causal state and hence zero statistical complexity and high entropy rate. At this stage, CSSR uses statistical tests to determine when it must add states to the model, which increases the estimated complexity, while lowering the entropy rate h µ (see C.1). A key and distinguishing feature of the CSSR code is that it maintains homogeneity of the causal states and deterministic state-to-state transitions as the model grows. Complexity of the CSSR algorithm is: O(m L max) + O(m 2L max +1 ) + O(N), where m is the size of the alphabet A; N is the data size and L max is the length of the longest history to be considered. Details are given in [76] The Suboptimal D-Markov Machine This section presents a new alternative approach for representing the pattern in a symbolic process, which is motivated from the perspective of anomaly detection. The core assumption here is that the symbolic process can be represented to a desired level of accuracy as a D th order Markov chain, by appropriately choosing D N. A stochastic symbolic stationary process S S 2 S 1 S 0 S 1 S 2 is called D th order Markov process if the probability of the next symbol depends only on the previous (at most) D symbols, i.e. the following condition holds: P (S i S i 1 S i 2 S i D ) = P (S i S i 1 S i D ) (3.9)

68 Alternatively, symbol strings S, S S become indistinguishable whenever the respective substrings S D and S D, made of the most recent D symbols, are identical. 53 This can be interpreted as follows: S, S S such that S D and S D, ( S ɛ( S ) and S ɛ( S ) ) iff S D = S D. Thus, a set { S L : L D} of symbol stings can be partitioned into a maximum of A D equivalence classes where A is the symbol alphabet, under the equivalence relation defined in Eq. (3.6). Each symbol string in { S L : L D} either belongs to one of the A D equivalence classes or has a distinct equivalence class. All such symbol strings belonging to the distinct equivalence class form transient states, and would not be of concern to anomaly detection for a (fast-time-scale) stationary condition under (slowly changing) anomalies. Given D N and a symbol string s with s = D, the effective state q (D, s ) is the equivalence class of symbol strings as defined below: q(d, s ) = { S S : S D = s } (3.10) and the set Q(D) of effective states of the symbolic process is the collection of all such equivalence classes. That is, Q(D) = {q(d, s ) : s S D } (3.11) and hence Q(D) = A D. A random variable for a state in the above set Q of states is denoted by Q and the j th state as q j. The probability of transitions from state q j to

69 54 state q k is defined as: π jk = P (s S 1 q j Q, (s, q j ) q k ) ; π jk = 1; (3.12) k Given an initial state and the next symbol from the original process, only certain successor states are accessible. This is represented as the allowed state transitions resulting from a single symbol. Note that π ij = 0 if s 2 s 3 s D s 1 s D 1 whenever q i s 1 s 2 s D and q j s 1 s 2 s. Thus, for a D-Markov machine, the stochastic D ] matrix Π [π ij becomes a branded matrix with at most A D+1 nonzero entries. The construction of a D-Markov machine is fairly straightforward. Given D N, the states are as defined in Eqs. (3.10) and (3.11). On a given symbol sequence S, a window of length (D + 1) is slided by keeping a count of occurrences of sequences s i1 s id s id+1 and s i1 s id which are respectively denoted by N(s i1 s id s id+1 ) and N(s i1 s id ). Note that if N(s i1 s id ) = 0, then the state q s i1 s id Q has zero probability of occurrence. For N(s i1 s id ) 0), the transitions probabilities are then obtained by these frequency counts as follows: π jk = P (s i s 1 id s) P (s i1 s id ) N(s i s 1 id s) N(s i1 s id ) (3.13) where the corresponding states are denoted by: q j s i1 s i2 s id and q k s i2 s id s. As an example, Figure 3.2 shows the finite state machine and the associated state transition matrix for a D-Markov machine, where the alphabet A = {0, 1}, i.e., alphabet size A = 2; and the states are chosen as words of length D=2 from a symbol sequence S. Consequently, the total number of states is A D = 4, which is the number

70 55 of permutations of the alphabet symbols within a word of length D; and the set of states Q = {00, 01, 10, 11}. The state transition matrix on the right half of Figure 3.2 denotes the probability π ij = p ij of occurrence of the symbol 0 A at the state q ij, where i, j A. The states are joined by edges labelled by a symbol in the alphabet. The state machine moves from one state to another upon occurrence of an event as a new symbol in the symbol sequence is received and the resulting transition matrix has at most A D+1 = 8 non-zero entries. The machine language is complete in the sense that there are different outgoing edges marked by different symbols; however, it is possible that some of these arcs may have zero probability Statistical Mechanical Concept of D-Markov Machine This section outlines an analogy between the structural features of the D-Markov machine and those of spin models in Statistical Mechanics. The main idea is derived from the doctoral dissertation of Feldman [23] who has demonstrated how measures of patterns from Information Theory and Computational Mechanics are captured in the construction of ɛ-machines. In general, the effects of an anomaly are reflected in the respective state transition matrices. Thus, the structure of the finite state machine is fixed for a given alphabet size A and window length D. Furthermore, the number of edges is also finite because of the finite alphabet size. The elements of the state transition matrix (that is a stochastic matrix [6]) are identified from the symbol sequence. For A = 2 and D = 2, the finite-state machine construction is (to some extent) analogous to the one-dimensional Ising model of spin-1/2 systems with nearest neighbor interactions, where the z-component of each spin takes on one of the two possible values

71 56 s = +1 or s = 1 [23] [59]. For A 3, the machine would be analogous to onedimensional Potts model, where each spin is directed in the z-direction with A different discrete values s k : k 1, 2,, A ; for a j/2-spin model, the alphabet size A = j + 1 [7]. For D 2, the spin interactions extend up to the (D 1) th neighbor Comparison of D-Markov Machine and ɛ-machine An ɛ-machine seeks to find the patterns in the time series data in the form of a finite-state machine, whose states are chosen for optimal prediction of the symbolic process; and a finite-state automaton can be used as a pattern for prediction [76]. An alternative notion of the pattern is one which can be used to compress the given observation. The first notion of the pattern subsumes the second, because the capability of optimal prediction necessarily leads to the compression as seen in the construction of states by lumping histories together. However, the converse is not true in general. For the purpose of anomaly detection, the second notion of pattern is sufficient because the goal is to represent and detect the deviation of an anomalous behavior from the nominal behavior. This has been the motivating factor for proposing an alternative technique, based on the fixed structure D-Markov machine. It is possible to detect the evolving anomaly, if any, as a change in the probability distribution over the states. Another distinction between the D-Markov machine and ɛ-machine can be seen in terms of finite-type shifts and sofic shifts [51] (see C.2). Basic distinction between finite-type shifts and sofic shifts can be characterized in terms of the memory: while a finite-type shift has finite-length memory, a sofic shift uses finite amount of memory in representing the patterns. Hence, finite-type shifts are strictly proper subsets of sofic

72 57 shifts. While, any finite-type shift has a representation as a graph, sofic shifts can be represented as a labelled graph. As a result, the finite-type shift can be considered as an extreme version of a D-Markov chain (for an appropriate D) and sofic shifts as an extreme version of a Hidden Markov process [91], respectively. The shifts have been referred to as extreme in the sense that they specify only a set of allowed sequences of symbols (i.e., symbol sequences that are actually possible, but not the probabilities of these sequences). Note that a Hidden Markov model consists of an internal D-order Markov process that is observed only by a function of its internal-state sequence. This is analogous to sofic shifts that are obtained by a labelling function on the edge of a graph, which otherwise denotes a finite-type shift. Thus, in these terms, an ɛ-machine infers the Hidden Markov Model (sofic shift) for the observed process. In contrast, the D-Markov Model proposed in this chapter infers a (finite-type shift) approximation of the (sofic shift) ɛ-machine. 3.5 Anomaly Measure and Detection The machines described in the subsections and recognize patterns in the behavior of a dynamical system that undergoes anomalous behavior. In order to quantify changes in the patterns that are representations of evolving anomalies, we induce an anomaly measure on these machines, denoted by M. The anomaly measure M can be constructed based on the following information-theoretic quantities: entropy rate, excess entropy, and complexity measure of a symbol string S (see C.1). The entropy rate h µ (S) quantifies the intrinsic randomness in the observed dynamical process.

73 58 The excess entropy E(S) quantifies the memory in the observed process. The statistical complexity C µ (S) of the state machine captures the average memory requirements for modelling the complex behavior of a process. Given two symbol strings S and S 0, it is possible to obtain a measure of anomaly by adopting any one of the following three alternatives: M(S, S 0 ) = h µ (S) h µ (S 0 ), or E(S) E(S 0 ), or C µ (S) C µ (S 0 ) Note that each of the anomaly measures, defined above, is a pseudo metric [55]. For example, let us consider two periodic processes with unequal periods, represented by S and S 0. For both processes, h µ = 0, so that M(S, S 0 ) = 0 for the first of the above three options, even if S S 0. The above measures are obtained through scalar-valued functions defined on a state machine and do not exploit the rich algebraic structure represented in the state machine. For example, the connection matrix T associated with the ɛ-machine (see the subsection 3.4.1), can be treated as a vector representation of any possible anomalies in the dynamical system. The induced 2-norm of the difference between the T - matrices for the two state machines can then be used as a measure of anomaly, i.e., M(S, S 0 ) = T T 0 2. Such a measure, used in [82], was found to be effective. However, there is some subtlety in using this measure on ɛ-machines, because ɛ-machines do not guarantee that

74 59 the machines formulated from the symbol sequences S and S 0 have the same number of states; and these states do not necessarily have similar physical significance. In general, T and T 0 may have different dimensions and different physical significance. However, by encoding the causal states, T could be embedded in a larger matrix, and an induced norm of the difference between T matrices for these two machines can be defined. Alternatively, a (vector) measure of anomaly can be derived directly from the stochastic matrix T as the left eigenvector p corresponding to the unit eigenvalue of T, which is the state probability vector under a stationary condition. This chapter has adopted the D-Markov machine approach, described in the subsection to build the state machines. Since D-Markov machines have a fixed state structure, the state probability vector p associated with the state machine have been used for a vector representation of anomalies, leading to the anomaly measure M(S, S 0 )) as a distance function between the respective probability vectors p and p 0 (that are of identical dimensions), or any other appropriate functional Anomaly Detection Procedure Having discussed various tools and techniques, this section outlines the steps of the forward problem and the inverse problem [64]. Following are the steps for the forward problem: F1. Selection of an appropriate set of input stimuli. F2. Signal-noise separation, time interval selection, and phase-space construction.

75 F3. Choice of a phase space partitioning to generate symbol alphabet and symbol sequences. 60 F4. State Machine construction using generated symbol sequence(s) and determining the connection matrix. F5. Selection of an appropriate metric for the anomaly measure M. F6. Formulation and calibration of a (possibly non-parametric) relation between the computed anomaly measure and known physical anomaly under which the time series data were collected at different (slow-time) epochs. Following are the steps for the inverse problem: I1. Excitation with known input stimuli selected in the forward problem. I2. Generation of the stationary behavior as time series data for each input stimulus at different (slow-time) epochs. I3. Embedding the time series data in the phase space determined for the corresponding input stimuli in Step F2 of the forward problem. I4. Generation of the symbol sequence using the same phase-space partition as in Step F3 of the forward problem. I5. State Machine construction using the symbol sequence and determining the anomaly measure. I6. Detection and identification of an anomaly, if any, based on the computed anomaly measure and the relation derived in Step F6 of the forward problem.

76 61 Having obtained the phase plots from the time series data, the next step is to find a partition of the phase space for symbol sequence generation. This is a difficult task especially if the time series data is noise-contaminated. Several methods of phasespace partitioning have been suggested in literature (for example, [2], [19], and [43]). Apparently, there exist no well-established procedure for phase-space partitioning of complex dynamical systems; this is a subject of active research. In this chapter, we have introduced a new concept of symbol sequence generation, which uses wavelet transform to convert the time series data to time-frequency data for generating the symbol sequence. The graphs of wavelet coefficients versus scale at selected time shifts are stacked starting with the smallest value of scale and ending with its largest value and then back from the largest value to the smallest value of the scale at the next instant of time shift. The resulting scale series data in the wavelet space is analogous to the time series data in the phase space. Then, the wavelet space is partitioned into segments of coefficients on the ordinate separated by horizontal lines. The number of segments in a partition is equal to the size of the alphabet and each partition is associated with a symbol in the alphabet. For a given stimulus, partitioning of the wavelet space must remain invariant at all epochs of the slow time scale. Nevertheless, for different stimuli, the partitioning could be chosen differently. (The concept of proposed wavelet-space partitioning would require significant theoretical research before its acceptance for application to a general class of dynamical systems for anomaly detection; and its efficacy needs to be compared with that of existing phase-space partitioning methods such as false nearest neighbor partitioning [43].)

77 62 The procedure, described in the subsection constructs a D-Markov machine and obtains the connection matrix T and the state vector p from the symbol sequence corresponding to each β. For this analysis, the wave space generated from each data set has been partitioned into eight (8) segments, which makes the alphabet size A = 8 to generate symbol sequences from the scale series data. At each value of β, the generated symbol sequence has been used to construct several D-Markov Machines starting with D=1 and higher integers. It is observed that, the dominant probabilities of the state vector (albeit having different dimensions) for different values of D are virtually similar. Therefore, a fixed-structure D-Markov Machine with alphabet size A = 8 and depth D=1, which yields the number of states A D = 8, is chosen to generate state probability (p) vectors for the symbol sequences. In conclusion, this chapter reviews a novel concept of anomaly detection in complex systems based on the tools of STSA, Finite State Automata, and Pattern Recognition [64]. This concept has been validated on the Gas Turbine Simulation (GTS) test-bed in Chapter 7.

78 63 Fig Continuous Dynamics to Symbolic Dynamics Fig State Machine with D=2, and A = 2

79 64 Chapter 4 Probabilistic Robust Control of Future Generation Rotorcraft 4.1 Introduction This chapter presents an innovative approach for reliable control system design of high performance rotorcraft to enhance handling qualities. The proposed control system has a two-tier hierarchical architecture. The lower-tier controller is designed using a probabilistic robust control approach. By allowing different levels of risk under different flight conditions, the control system can achieve the desired trade off between stability robustness and nominal performance [36] [35]. The upper-tier supervisory controller monitors the system response for any anomalous behavior that might lead to potential instability or loss of performance. The supervisor may then switch between low-level robust controllers with different levels of risk and performance. One approach for achieving higher performance from rotorcraft flight control systems is rotor state feedback [34] [83]. This allows a full state feedback with good stability margins, but it also requires specialized sensors to measure the motion of the rotor blades. Another alternative is to use dynamic compensators based on an accurate high-order model of the coupled fuselage / rotor dynamics [38]. This approach tends to result in high-order controllers, and there will always be inherent uncertainties due to modeling errors, variations in aircraft properties, and changing operating conditions.

80 65 Robust control theory allows the design of control systems based on a simple loworder plant model with well-defined uncertainty bounds that account for model simplifications, non-linearity, and variations in operating conditions. Furthermore, approximate low-order plant models are more readily identified from flight test data and result in less complex control designs. A number of simulation studies have investigated robust control methods on rotorcraft using both H and µ-synthesis techniques [73], and an H based controller has been tested in flight [79]. It is well known that the demands on system stability robustness and desired nominal performance could be contradictory to each other. The deterministic worst-case robust design could cause unduly conservativeness and thus degrade system nominal performance. Instead of stability guarantee under worst-case uncertainties, recent results in probabilistic robust control indicate that complexity of the controller can be greatly reduced and/or system performance can be significantly improved by allowing a small risk of instability; e.g., see [47] [48]. Furthermore, by specifying different levels of risk at different flight regimes, the control design could obtain a trade off between robustness and nominal performance. Military rotorcraft handling qualities specifications dictate different levels of flight control system performance when performing various mission tasks [92]. For example, if the rotorcraft is in cruise flight, the bandwidth and attitude quickness requirements are relatively low, and a low-risk / low-performance controller would be adequate. On the other hand, when performing aggressive combat tasks or precision maneuvers it may be desirable to achieve the maximum available performance. A high risk controller might be used if there is a mechanism to recover, in the event that the controller initiates instability. The upper-tier supervisory controller can govern the acceptable level of

81 66 risk as well as the desired level of performance. Such a system would need to monitor the response of the vehicle to detect degradation in performance or stability, and also take into account external inputs such as the current mission task and environmental conditions. The upper-tier supervision would be an appropriate application of discreteevent control. This chapter investigates the application of advanced control theory for high bandwidth flight control on a military rotorcraft using a high fidelity non-linear simulation model. The control architecture is formulated as a hierarchical control system. Discreteevent control is used for upper-tier supervision; probabilistic robust control is used as the lower-tier controller. The supervisory control chooses from a bank of robust control designs with different levels of risk and performance. This chapter focuses on the lower-tier probabilistic robust controller, as applied for lateral-directional control of a helicopter operating in the low speed flight regime. Frequency weighted uncertainty perturbations are derived based on the discrepancy between the nominal low order linear model and the identified frequency response of the non-linear model for a range of off-design flight conditions. Several different controllers are designed using µ-synthesis. The radius of uncertainty and the performance weighting for each controller are varied to produce a set of controllers with varying levels of risk and performance. The risk associated with each controller is assessed using Monte-Carlo simulations, in which the uncertainty perturbations are modeled using random transfer functions [49]. The controllers are then tested using the non-linear simulation model. These results clearly show that switching between a high risk / high performance controller to a low risk / low performance controller can recover the aircraft from the onset of limit cycle instability.

82 67 This chapter is organized in eight sections including the present section: Section 4.2 presents the proposed two-tier control architecture. Section 4.3 describes the plant modeling and system identification. Section 4.4 presents the equations of motion of rotorcraft and the augmented plant model used for synthesis. Section 4.5 presents the details of controller design. Section 4.6 presents a method for generating sets of random transfer functions for use in the Monte Carlo analysis of the controllers probabilistic robustness. Section 4.7 discusses some issues for practical implementation of the controllers on aircraft. Section 4.8 discusses the results of simulation experiments. This chapter is summarized and concluded in section Control System Architecture Figure 4.1 shows the two-tier architecture of rotorcraft control system and its components are described below. The signals are explained in greater detail in table Rotorcraft Dynamics This functional block receives the input of continuous feed-forward control signals S 1, from the upper-tier controller, exogenous disturbance signal vector S 6, and feedback control input vector S 4 from the dynamic controller subsystem. The sensor data S 7, a vector of electronic signals is the output of this subsystem that serve as inputs to the Signal Conditioning subsystem. The sensor data set S 8 serves as an input to the risk assessment subsystem.

83 Dynamic Controller This subsystem provides the basic inner loop control of the rotorcraft in order to stabilize the rotorcraft and satisfy the handling qualities requirements (i.e., performance) over a wide range of operating conditions. In this study, a probabilistic robust controller is designed for a subset of operating conditions. This subsystem receives input signal S 2 from the upper-tier controller, S 10 from risk assessment subsystem and S 5 from Signal conditioning and validation subsystem. This subsystem sends the feedback control vector S 4 to the plant. Given a bound on the risk of instability that is provided by the uppertier supervisor, the risk-adjusted controller maximizes the nominal performance. In other words, the controller is designed based on a tradeoff between nominal performance and risk of instability Signal Conditioning and Signal Validation The input set S 7 to this subsystem consists of the sensor signals (e.g. acceleration, velocity, and angular rate). It also receives information on the linear controller (A, B, C, D matrices) S 13. The subsystem provides the processed auxiliary feedback signal S 5 to the dynamic controller. The signal set S 11 is used by the risk assessment subsystem and the signal set S 12 is the input to the upper tier controller Risk Assessment Subsystem This subsystem receives input from the signal conditioning subsystem S 11, sensor data from rotorcraft S 8 and external inputs S 9. Based on these inputs it assesses the risk factor associated with the different controllers using probabilistic methods and sends

84 this information to the robust feedback control system S 10 and the upper-tier controller S Upper Tier Control at the Flight Management Level A discrete event supervisor (DES) will be used to choose a particular controller from a bank of controllers for the robust feedback control system. This subsystem receives the input from the risk assessment subsystem S 14 and the signal conditioning and validation subsystem S 12. The outputs of this subsystem are the reference command signals S 1 and input signals S 2 to the dynamic controller. This chapter focuses on the flight control level of the proposed scheme. 4.3 System Identification Frequency domain identification methods have been shown to be an effective method for identifying accurate linear models of aircraft and particularly rotorcraft. The Comprehensive Identification from FrEquency Responses (CIFER) analysis tool was developed for this purpose (Tischler, 1991) [86]. An advantage of this method is that linear models can be extracted directly from flight data, and the control designer does not rely on theoretical flight dynamics models. Although flight data was not available for this study, a non-linear simulation model is used as a proxy for a real helicopter. The controller is designed using linear models extracted directly from time history data of the non-linear simulation, and then the controllers are tested using this nonlinear model. The simulation model generates time histories of the vehicle response to a sinusoidal input of varying frequency. The frequency response characteristics are identified, and a linear

85 70 state space model is derived to fit the frequency responses using the CIFER analysis tool. This software uses the chirp-z transform and composite optimal windowing methods to identify the multi-input / multi-output frequency responses, and a non-linear search algorithm is used to fit a state space model to these frequency responses given a known model structure. Figure 4.2 and 4.3 show the frequency response of the roll rate due to lateral control and yaw rate due to directional control respectively, for the linear model that is identified from the non-linear simulation. The coherence function is a useful metric to verify that the flight data are satisfactory for system identification. The coherence function γ xy (or partial coherence for a multiple-input multiple-output system) indicates how well the output y (any of the estimated helicopter states) is linearly correlated with a particular input x over the examined frequency range. It is computed together with the system s frequency responses, from the cross spectrum G xy, and the input and output auto-spectra G xx and G yy respectively (note that the partial coherence is derived from the conditioned spectrum); the mathematical definition is given by 4.1 γ xy 2 = G xy 2 G xx G yy (4.1) A value of 0.6 for the coherence function is usually used as a limit. In our case we have taken the limit to be 0.8. For lower coherence values the identified frequency responses will have a too-high random error. The linear model of the aircraft is subject to inherent uncertainties. The model is relatively low order (as discussed in the following section) and it matches the full

86 71 order dynamics over a very limited frequency range. Furthermore, the controller is designed for the low speed flight regime. It is not normally possible to obtain accurate airspeed measurements below 20 knots on a helicopter, and therefore it is not possible to schedule the controllers with airspeed in this regime. Instead, the controller is designed based on a single nominal linear model for hover, and the controller must be robust to perturbations in the model as the operating point changes within the low speed envelope. To estimate the uncertainty bounds associated with varying operating conditions, the frequency response characteristics for five flight conditions were calculated: hovering, 20 knots forward, 20 knots right sideward, 20 knots rearward, and 20 knots left sideward flight. Uncertainty bounds are estimated based on the maximum difference between the nominal linear model and the five sets of frequency response data. This is illustrated in Figure 4.4, which shows the multiplicative error in each case. The uncertainty weighting, W, is designed to cover all operating conditions. 4.4 The Plant Model A reduced order plant model is used for design of a lateral-directional controller. The controller is designed to achieve a rate response type in the roll and yaw axes. A simple linear model is optimized to represent the roll and yaw rate dynamics in the range of 1 to 10 rad/sec. The state variables are roll rate, yaw rate, and lateral flapping angle. The control inputs are lateral and pedal control in equivalent inches of stick position. The plant outputs are yaw and roll rate measured in deg/sec. The state space model of the plant can be written as equation 4.2. This is a relatively simple third order linear model. A number of dynamic states have been truncated. The complexity of the model

87 72 could be increased by including lateral speed and roll attitude to better match the low frequency dynamics. Flapping rate and lag dynamics could also be included to match the high frequency dynamics. This would result in smaller uncertainty bounds, but a more complex plant model. The approach here is to use a simplified plant dynamic model and allow the uncertainty bounds to account for the discrepancies. ẋ = Ax + Bu y = Cx (4.2) Where: p δ x = r ; u = lat ; A = δ ped β 1s B = ; C = ; y = p r ; p = roll rate (rad/sec) r = yaw rate (rad/sec) β 1s = lateral flapping angle (rad) δ lat = lateral control (inches) δ ped = pedal control (inches)

88 73 The augmented plant model includes additional dynamics due to the ideal response models, frequency weighted performance, and frequency weighted uncertainty bounds. Figure 4.5 shows a schematic of the augmented plant model. Output multiplicative uncertainty is used to account for the uncertainties that arise from model simplification and variation in operating point. The uncertainty associated with the plant is represented by, (s) = G(s) G nom (s) G nom (s) (4.3) The frequency variation of the uncertainty is calculated using equation 4.3, where G(jω) is identified from the non-linear model and G nom (jω) is the frequency response of the simplified linear model. A weighting function must be designed that covers the model uncertainty over the entire low speed flight envelope. The following uncertainty weighting matrix 4.4 that was derived to cover the plant uncertainties. W = (4.4) 11 = (s s )(s s ) (s s )(s s ) 22 = 1.789(s s )(s s ) (s s )(s s ) Where r i is the radius of uncertainty used for the controller. When r i =1, the plant uncertainty is entirely covered by the weighting functions as shown in Figure 4.4.

89 The figure shows that the model is relatively accurate in the frequency range of rad/sec,but the model error approaches 100% for lower and higher frequencies. When r i is selected to be less than unity, less conservative uncertainty bounds are used with the risk. The control design is based on an implicit model following approach. Ideal response characteristics for the yaw and roll axes are specified in the augmented plant in order to meet or exceed the handling qualities criteria for military rotorcraft [92]. The ideal response model provides a guideline in terms of relative system performance in terms of bandwidth, which is defined in the handling qualities specification as the frequency where the aircraft attitude lags the pilot input by 135 o. This corresponds to the frequency where the phase angle of the angular rate response is 45 o. Simple first order transfer functions for the response of roll rate due to roll rate command and yaw rate due to yaw rate command are specified as in equation 4.5. G ideal (s) = 1 0.1s s+1 (4.5) If the ideal response were tracked perfectly, then the bandwidth frequency is equal to the inverse of the time constant in the ideal response model. Thus, the controller is designed to achieve a bandwidth of 10 rad/sec and 5 rad/sec in the roll and yaw axes respectively. In this study, the objective is to significantly improve upon the agility of current rotorcraft, so these values are significantly larger than the bandwidth requirements currently specified for military rotorcraft [92]. In practice, perfect tracking is not achieved, so the actual bandwidth is expected to be lower that is specified in the ideal

90 75 response model. The degree to which the aircraft follows the ideal response model will be largely based on the relative weighting of tracking and actuator performance as well as the effective uncertainty bounds used in the probabilistic robust control design. The augmented plant includes frequency weighed performance for the tracking and actuator performance. The tracking performance weights given by equation 4.6 are designed to emphasize low frequency tracking and minimize steady-state error: W p (s) = P weight s+0.2 s s+0.2 s+0.01 (4.6) The actuator weights are designed to penalize high frequency actuator activity and allow steady-state actuator motion as shown in equation 4.7: W a (s) = 0.2 P weight s s s s+0.2 (4.7) The performance weighting parameter P weight can be adjusted to achieve varying levels of performance. Increasing this performance weight and relaxing the radius of uncertainty, r i is expected to improve the tracking performance but allow greater risk of closed loop instability. Note that the simplified linear model discussed above is used for flight control design in the hover and low speed flight regime. Similar models have been derived for forward flight conditions in order to design a suite of controllers for the entire flight envelope. Although, the control design is based on simple linear models, the controllers are eventually tested on the full order non-linear model [37].

91 Controller Design Procedure If the full radius of uncertainty, r i = 1, were used in equation 4.4, and the controller were designed to achieve a closed-loop structured singular value less than or equal to unity, then the system would have guaranteed closed loop stability for all possible complex uncertainty perturbations. Such a controller could not be found for the plant model used in this study. The uncertainty perturbations appear to be too severe. When r i = 1, even as the performance weighting parameter P weight approaches zero, the structured singular value could not be made less than one, which theoretically implies that there is no controller that can robustly stabilize the plant. However, it is known that the robust control design is conservative, and in practice a controller synthesized with a reduced radius of uncertainty will result in a stable closed-loop system. To design the risk-adjusted controller, a grid of the interval [0, 1] was first determined. For each of the points of the grid r i, the µ-toolbox of MATLAB was used to design a controller that robustly stabilizes the closed loop system subject to uncertainty of radius r i and maximizes nominal performance. This was achieved by iteratively increasing the performance weight, P weight, until the optimization resulted in a µ value equal to 1. The controllers obtained were of 25 th order. Hankel norm model reduction techniques were employed to lower the order of the controllers to 9. These controllers do not robustly stabilize the plant, so Monte Carlo simulation was performed to determine the associated risk of instability for each controller. A set of random transfer functions were generated using the algorithm in [49] to model uncertainty perturbations (this process is discussed in the following section). A total of 10, 000 samples were used to

92 77 estimate the risk factor associated with each controller, where risk factor is defined as the percentage of cases where the theoretical uncertainty perturbation results in instability. Hence, we obtain a set of controllers with varying levels of risk, as presented in Table 4.2. Each controller was also evaluated in terms of performance using the nominal plant model. The roll axis and yaw axis bandwidths were calculated according to handling qualities specifications [92] and are shown in the table in units of radians/sec. The results in Table 4.2 show the expected trend in risk/performance tradeoff. ADS-33 defines the attitude bandwidth of an aircraft as the frequency where the attitude response lags the primary control input by 135 o. A rotorcraft with high bandwidth flight controls will respond more quickly to pilot control inputs and will more readily track pilot commands at higher frequencies. As discussed in section 4, the ideal response models were chosen to achieve a roll and yaw bandwidth of 10 and 5 rad/sec, respectively. These values are substantially higher than those required for Level I handling qualities in ADS-33. However, future rotorcraft missions may require more agile rotorcraft, which would require higher bandwidth flight controls. Thus bandwidth is a suitable measure of performance. The bandwidths for the high-risk controllers are relatively high, and nearly achieve the bandwidth specified in ideal response model, but the risk analysis indicates those controllers also have a higher risk of inducing closed-loop instability. For the low risk controllers, the bandwidth is substantially degraded, but those controllers have a much lower risk of initiating instability. As discussed in the following section, the measure of risk is determined by generating a large sample of perturbations uniformly distributed over the entire space of possible uncertainties as defined by the weighting functions in Equation 4.4. A risk

93 78 factor of X% does not mean that there is an X% chance that the aircraft will become unstable, only that X% of the theoretical uncertainty perturbations result in instability. In practice the probability of the uncertainty perturbations is not uniformly distributed. Perturbations within some portions of the space of uncertainty are more or less likely to occur than others. Although the risk factor in Table 4.2 gives some measure of risk relative to the other controllers, it does not explicitly define the probability of instability for a given controller. For example, controller 9 was found to have a risk factor of 8%, but in extensive simulation testing with the full non-linear model it never resulted in instability. 4.6 Sample Generation Procedure To address the problem of risk assessment in the presence of dynamic uncertainty, recent results on probabilistic robustness were applied. The algorithm in [49] was used to generate a set of random transfer functions to represent the uncertainty perturbations, (s). The algorithm generates random discrete-time transfer functions in the set: H(z) = h 0 + h 1 z h n 1 z n 1 F n = H(z) + z n G(z) 1 for some stable G(z) (4.8) This is a set of random transfer function on the discrete time domain, which can be completed to a transfer function with infinity norm less or equal than one. The random transfer functions are then transformed to the continuous time domain using Tustin transformations. The algorithm is described below:

94 Step 1. Let k = 0. Generate N samples of h 0 uniformly distributed over the interval [-1, 1]. Step 2. Let k = k + 1. For every generated sample (h l 0, hl 1,..., hl ), generate k 1 samples of h k uniformly over the interval M l k, ml k where 79 m l k = Y (Hl ) T Y T 1 Y Y T M l k = Y (Hl ) T Y T + 1 Y Y T H l = H(0, h l 0, hl 1,..., hl k 2 ) (4.9) Y = [h l k 1,..., hl 1, hl 0 ](I (Hl ) T H l ) 1/2 h k... h 1 h 0 h H(h 0, h 1,..., h k ) = 1... h h Step 3.If k = n 1, stop. Else go to Step Practical Implementation Issues The controllers discussed above provide a roll rate command and yaw rate command response for the hover and low speed flight regime. This is just one step required for practical implementation of a lateral-directional controller on an aircraft. The controller needs to be extended to operate in forward flight conditions and it needs to incorporate more advanced autopilot modes such as attitude hold, turn coordination, and heading

95 80 hold. The issue of control switching must also be addressed. The operating range of the controller has been extended into forward flight up to 140 knots. The same design procedure is repeated for forward flight conditions at 40 knots, 80 knots, and 120 knots: 1. Identify the linear dynamics at nominal flight condition 2. Identify the dynamics at off-design conditions 3. Define the uncertainty weights that cover the multiplicative uncertainty 4. Define a suite of controllers with varying risk and performance The uncertainty weights are based on frequency response data obtained at airspeeds ±20 knots off of the nominal operating point. The probabilistic robust controllers discussed above only provide inner loop stabilization and results in a rate command response type. Outer loop controllers can then be used to achieve autopilot functions such as roll attitude hold, heading hold, and turn coordination. Once the inner loop is closed, the lateral-directional dynamics of the aircraft behaves as a pair of decoupled first order systems, as dictated by the ideal response model. Thus, simple classical control theory can be used to design proportional or proportional plus integral compensators for the outer loop. Figure 4.5 shows a schematic of an outer-loop controller that can be used to achieve for roll attitude command / attitude hold response in the roll axis and turn coordination in the yaw axis. If a proportional gain, K, is used in the in the roll attitude compensator, then the effective closed loop transfer function for roll attitude response is given by Equation 4.11, where τ is the effective time constant in the roll rate response as dictated by the ideal response model in Equation 4.5. The gain K can be chosen to achieve the desired

96 2 nd order dynamics in attitude response. For turn coordination, the control law derived in [72] is used to regulate lateral acceleration, a y, given by Equation r cmd = (a ycmd a y + wp + gsinφcosθ) u (4.10) where u, w, p, φ, and θ represent the longitudinal velocity, vertical velocity, roll rate, roll attitude, and pitch attitude of the aircraft, respectively. Another topic that must be addressed is the issue of switching between controllers. As discussed before, the controller architecture results in a bank of controllers with different risk and performance levels. Furthermore, as discussed above, several banks of controllers are designed for different airspeeds. The system will switch between controllers as the aircraft transitions to different airspeeds or when a higher level supervisor determines that it should switch to a higher or lower risk controller. The issue of instability due to switching between the controllers has to be addressed. A switching law is proposed which guarantees stability of the closed-loop system while the controllers are being switched. It is shown in [54] that when all subsystems are Hurwitz stable, then the entire system is exponentially stable for any switching signal if the time between two consecutive switching operations, called the dwell time, is sufficiently large. [30], extend the concept of dwell time to average dwell time, which means the average time interval between two consecutive switching operations is no less than a specified constant. It was shown that if the average dwell time is sufficiently large, then the switched system is exponentially stable. The dwell time concept is a reasonable approach for real-time implementation since it is counter-intuitive and counter-productive to switch controllers too frequently.

97 82 For this system, an average dwell time is selected that is sufficiently large to guarantee the stability when switching between any of the controllers. A dwell time of 2 seconds was found to be sufficient for this application (this value was found experimentally and was not derived from rigorous analysis). When the decision is made to switch controllers, the current controller and the new controller are run simultaneously. The control signal is gradually switched between the two controllers over the two second dwell time. A blend parameter is ramped in over the two second interval and used to generate a weighted average of the two control signals. This approach was found to be sufficient to demonstrate the concepts in this chapter, and a more rigorous approach to switching is left to future work. 4.8 Simulation Results and Discussion This section presents and discusses the pertinent results of simulation experiments conducted on rotor control system model described earlier. The controllers were implemented on the non-linear dynamic model. Consequently, the very high-risk controllers (C 1 and C 2 ) that exhibited instability were eliminated from the family of robust controller designed earlier. The controllers with medium-to-high risk tended to perform well but occasionally exhibited instability as the operating conditions were varied or for significantly large disturbances. The low-risk controllers resulted in significantly degraded performance but these controllers never resulted in instability in the non-linear simulation. The most common form of instability resulted in sustained oscillations at a highfrequency range when the lag progressing mode of the rotor became unstable. While

98 83 this type of instability could be very dangerous and uncomfortable to the pilot, it also would be easily identifiable to a supervisory controller since the oscillations tend to have a unique frequency range. In some cases, it was observed that instability could occur due to a real pole passing the origin, which resulted in a very slow divergent mode. Such instability could be considered less dangerous on piloted aircraft, because pilots can easily compensate for it. However, this might be more problematic if the method is applied to unmanned vehicles since it would take longer for the supervisor to detect it. This type of instability could be eliminated by including attitude and velocity feedback loops or by making modest improvements to the nominal plant model and thus are not considered to be significant. Two sets of time history results are presented in Figure 4.7 shows the response of the aircraft using a relatively high-risk controller C 3. For moderately large inputs, the controller was found to cause limit cycle instability due to destabilization of the lag progressing mode. Figure 4.8 starts with the same high risk controller but as soon as instability is detected by growing oscillations, a lower risk controller C 7 is phased in between 3 and 5 seconds using a control blending parameter. State space representations of controllers C 3 and C 7 are run simultaneously, and the output the controller is a weighted average based on the blending parameter. By switching to the low risk controller instability is prevented. 4.9 Conclusion This chapter presents a two-tier hierarchical architecture of future-generation rotorcraft control systems for enhanced performance and reliability. Probabilistic robust

99 84 control is proposed and the concept is validated on a non-linear model of rotorcraft dynamics. A bank of µcontrollers is designed where the robust stability requirements are relaxed with a specified probability in order to achieve better performance. Extensive Monte Carlo simulations were conducted to demonstrate the expected trend in the risk level and performance as the weights were varied. The controller was demonstrated for the inner loop stabilization of the lateral-directional dynamics of a helicopter in hover / low speed flight. The approach for extending the controller to operate over the entire flight enveloped and to provide outer loop autopilot functions was also discussed.current work has focused on extending the operating regime out to 140 knots forward speed and developing a more rigorous approach to control switching. The issues of early detection of stability are addressed in [87]. It was observed that no controllers could be found that robustly stabilize the plant given the relatively large uncertainty bounds associated with the simple linear model used in the control synthesis. However, it was observed that several of the controllers were stable and found to operate effectively in the high order, non-linear simulation environment when the uncertainty bounds were relaxed. Clearly, a design approach with strict robust stability requirements results in excessive conservatism. Such an approach would only be feasible if a significantly enhanced plant model with less uncertainty were used, which would in turn increase the complexity of both the system identification process and the controller. Therefore, the use of a probabilistic robust design method is of interest for complex and uncertain dynamic systems such as rotorcraft. The benefits of such an approach can be enhanced by allowing the system to increase the risk of instability to achieve better performance, as long as a method is in place to recover

100 85 upon the onset of instability. The available performance of the flight controller can then effectively be maximized for any given flight condition. The idea of allowing a small well defined risk of instability for enhanced performance works only because there is hierarchical structure in place which detects instability and mitigates it by switching to a more conservative controller. This hierarchical control methodology based on tradeoff between performance and risk of instability can be easily exported to other nonlinear complex dynamical systems that operate over a wide range of conditions. φ φ cmd (s) = K τs 2 + s + K (4.11)

101 86 Table 4.1. Nomenclature of Signals Signal Explanation S 1 S 2 S 3 S 4 S 5 S 6 S 7 S 8 S 9 S 10 S 11 S 12 S 13 S 14 Continuous feed-forward control signals Command signal from upper tier Error in command tracking Feedback control input vector Input from Signal validation module to lower level control Exogenous disturbance signal vector Sensor data for signal conditioning Sensor data for risk assessment External inputs Input from risk assessment module to lower level control Signal set used by risk assessment subsystem Input to the upper tier controller after signal conditioning Information on the linear controller (A, B, C, D matrices) Input to the upper tier controller from risk assessment module Table 4.2. Risk and Performance of Controllers Controller r i P weight Risk Factor Roll BW Yaw BW C C C C C C C C C

102 87 Fig Rotorcraft control architecture

103 88 Fig Roll axis frequency response

104 89 Fig Yaw axis frequency response

105 90 Fig Model uncertainty in the roll axis

106 91 Fig Augmented plant model Fig Augmented model

107 92 Fig Time history results for high risk controller

108 93 Fig Recovery from instability

109 94 Chapter 5 Hierarchical Control for Rotorcraft Hierarchically structured information and control systems occur for at least two related reasons: first, the great complexity of many natural and man-made systems limits the ability of humans and machines to describe and comprehend them, and, second, the inherent limitations on the information processing capacity of feedback regulators results in the regulators (and possibly the control systems) being organized in special (in particular hierarchical) configurations. As stated earlier, the notion of Discrete Event System or Discrete Event Dynamical System (DEDS) fits in beautifully with the idea of Hierarchical architecture. Intuitively, lower level control (the domain of continuous control) implies the traditional frequency or time domain based control strategies which are designed to follow certain specifications ( e.g. regulator problem). In essence, lower level control is usually precise but not intelligent. On the other hand the proposed DES based upper level supervision tries to mimic human intelligence by its heuristic design. In this chapter an upper level supervisory control scheme is proposed for the lower level probabilistic robust control (PRC) [36] [35] and Damage Mitigating Control (DMC) [10]. The goal here is to augment the lower level controller with a higher level supervisor (referred to as Supervisor in the sequel) in the discrete-event setting.

110 95 In the case of PRC the objective to achieve high performance and reliable operation over a wide range of operation. The function of this supervisory control scheme is to autonomously determine the desired level of performance based on environmental and operational conditions. The system should have the capability to detect the onset of instability and switch to a stable controller based on available sensor data and additional information on vehicle operation and maintenance. In the case of DMC the objective of the Supervisor is choose the appropriate damage weight D w ( which in turn affects the choice of DMC) based on mission objectives, to maximize damage mitigation with minimum loss of performance. In this chapter two scenarios are presented : In the first case the Supervisor acts in an advisory mode (i.e, it only suggests what the pilot ought to do) and in the second case the supervisor takes a more proactive role mimicking the actions of a pilot. This chapter is composed of three sections including the present one. The second section implements the hierarchical control scheme for PRC 5.1. In the third section implements the hierarchical control scheme for DMC Hierarchical Control for a bank of Probabilistic Robust Controllers A common form of instabilities that occur in uncertain dynamical systems is of unstable focus type that results in diverging oscillations that can be detected using frequency based methods. For the rotorcraft application used in this study, the lag progressing mode can exhibit this type of instability when using a high bandwidth controller. This type of instability can be detected at an early stage by a supervisory controller since the oscillations tend to have an a priori known narrow frequency range. The top plate of

111 96 Figure 5.1 represents the response of the aircraft using a relatively high-risk controller C 3. For moderately large inputs, the controller causes a high frequency, diverging oscillation that eventually reaches a limit cycle oscillation. Such a controller could not be used without higher level supervision. The bottom plate of Figure 5.1 shows the response of the low risk controller C 7. The controller is stabilizing, but it results in a relatively sluggish response. A frequency-domain method is proposed which uses a moving-window approach that relies on time series data from available sensors. In rotorcraft, the roll rate response is one of the critical variables, which captures the onset of instability. Let us consider the following scenario: Initially aggressive controller is used for better performance and handling qualities. On initiation of instability, it is required that Supervisor switches to a more conservative controller. A moving window approach is used to solve this problem, where a block of 1024 points in the time series data of roll rate are considered at any instant. Fast Fourier analysis of this data set is performed using validated codes [61]. As the system approaches instability, the energy content of the oscillatory modes increases. This energy across the frequency range is normalized so that the maximum energy is unity. Threshold techniques are then employed to determine whether the system is approaching instability. If so, the supervisor issues a command to switch to a relatively more conservative controller from the bank of pre-designed controllers. This process is repeated until the energy content of the high frequency terms is less than the threshold value. Two typical cases are shown in Figure 5.2.

112 97 The bottom plate of Figure 5.3 starts with the high risk controller C 3. As the normalized energy crosses the threshold within the specified frequency range, Supervisor issues a command to switch to a lower risk controller. The lower risk controller C 7 is phased in at the onset of the Supervisor s command and becomes fully effective between 2.2 and 4.4 seconds as shown in the top plate of Figure 5.3. The 2.2 second dwell time in which the controller 7 is blended in, is chosen using the formulation in [97]. The bottom plate of Figure 5.3 shows that this approach clearly stems the incipient instability. 5.2 Hierarchical Control for a bank of Damage Mitigating Controllers Figures 5.4 and 5.5 represent the implementation of hierarchical control scheme for the Damage Mitigating Control of Rotorcraft. The events and states of the corresponding DFSA model are given in Tables 5.1 and 5.2 respectively. As explained before DMC incorporates component damage into the synthesis of the controllers. The controllers are categorized by a variable parameter called Damage Weight (D w ). Normalized D w varies between 0 and 10. The value 10 implies maximum emphasis is being given to damage mitigation and thus the response for this category of controllers is relatively sluggish. On the other extreme the controllers with D w = 0, are a class of extremely agile controllers where the main emphasis is performance and these may cause an increase in damage rate. This section presents a scenario where these controllers are tested. Scenario: The rotorcraft is flying over a terrain which is divided into two territories: Enemy territory and friendly territory. The basic difference is that when flying in enemy territory there is chance that the rotorcraft may be shot down (This risk has also been sub-categorized as high risk of enemy fire and low risk of enemy fire). When

113 98 flying in enemy territory the rotorcraft has to fly close to the ground and at a high speed (nap of the earth flight) to avoid being shot. For better performance, rotorcraft needs maximum agility to avoid obstacles/terrain and damage mitigation is not an option here therefore the most aggressive controllers (low D w ) are employed in this case. For rotorcraft, it has been found that power versus airspeed curve is bucket shaped i.e., for hover and high speed regime the rotorcraft typically need more power compared to intermediate speed (shown by the green region in Figure 5.4). Therefore the case when the rotorcraft is flying in friendly territory and there is no risk (associated with enemy fire), to minimize damage, the rotorcraft flies at an intermediate speed and high altitude. Flying at high altitude is preferred (for damage mitigation) because the torque requirements do not fluctuate with terrain (versus the case of a low flying rotorcraft that has to repeatedly perform aggressive climb and decent maneuvers to follow the terrain). There are two distinct health conditions defined for the rotorcraft based on the current state of the representative damage (crack length in the main bevel pinion of the helicopter transmission). These are represented by the dashed mirror states in the DES representation of the scenario in Figure 5.5. The idea is, when the current damage is low the rotorcraft can afford to use low damage weight controllers for more aggressive maneuvering. For the other case when the current damage state of rotorcraft is high, damage mitigation should be given more importance (by choosing high damage weight controllers) to avoid catastrophic failures. Two sets of results are presented in this section. The first set of Figures 5.6, 5.7, and 5.8 depict a piloted simulation scenario where as the second set of Figures 5.9, 5.10, and 5.11 show the rotorcraft completely working under the control of hierarchical

114 99 control scheme without any inputs from the pilot. In Figure 5.6 the pilot follows the suggestion of the upper level DES control. It can be seen that initially pilot flies at a high altitude and when the rotorcraft enters the enemy territory it follows the terrain very closely, (nap of the earth flight) as suggested by the supervisor and finally regains the high altitude when it leaves the enemy territory. For this simulation run the Damage Mitigation controller was turned on i.e., The Damage Weight D w which varies between 0-10 is chosen based on the flight requirements: High damage weight controllers are used in friendly territory (higher damage mitigation and a relatively sluggish response) and low damage weight controllers are used in enemy territory (lower damage mitigation and a relatively fast response). In Figure 5.7 the pilot chooses not to follow the DES recommendations (e.g., pilot chooses to fly close to earth even when the DES recommends flying at a higher altitude). For this simulation run the Damage Mitigation controller was turned off i.e., The Damage Weight D w is fixed at 0. It can be clearly seen in Figure 5.8 that when pilot chooses to follow the DES recommendations the damage to the rotorcraft (in terms of crack growth) is significantly lower. The second set of results (Figures 5.9, 5.10, and 5.11) show the case where DES takes active part in the simulation (unlike the previous case where it worked in an advisory capacity to the pilot). Figure 5.9 depicts the simulation run with the Damage Mitigation turned on and Figure 5.10 depicts the simulation run with the Damage Mitigation turned off. First plate of Figure 5.11 represents the damage increase (in terms of crack growth) for the two simulation runs. The improvement in terms of damage

115 100 mitigation is very apparent from the figure. The bottom plate of Figure 5.11 depicts the change of Damage Weight D w over the simulation time.

116 101 Table 5.1. Event List for the Mission scenario DFSA Model Event Description Status a Start Mission C b Increase speed over 40 Knots C c Decrease speed below 40 knots C d Increase speed over 80 Knots C e Decrease speed below 80 knots C f Increase altitude over 100 feet C g Decrease altitude below 100 feet C h Increase altitude over 500 feet C i Decrease altitude below 500 feet C j Detection of high damage UC k Detection of low risk of enemy fire UC l Detection of high risk of enemy fire UC m Maintain same altitude and speed C Table 5.2. State List for the Mission scenario DFSA Model State Description Status Q 1 On Ground Unmarked Q 2 Low Altitude Low Speed Unmarked Q 3 Low Altitude Mid Speed Unmarked Q 4 Low Altitude High Speed Marked Q 5 Mid Altitude Low Speed Unmarked Q 6 Mid Altitude Mid Speed Unmarked Q 7 Mid Altitude High Speed Unmarked Q 8 High Altitude Low Speed Unmarked Q 9 High Altitude Mid Speed Marked Q 10 High Altitude High Speed Unmarked Q 11 Unfriendly territory, low risk of enemy fire Unmarked Q 12 Unfriendly territory, high risk of enemy fire Unmarked

117 102 Fig High risk/low risk controller response

118 103 Fig Energy content of the roll rate response

119 104 Fig System recovery from instability

120 105 Fig Choosing Appropriate DMC

121 106 Fig DES Representation

122 107 Fig Pilot following DES commands

123 108 Fig Pilot not following DES commands Fig Comparison of damage in the two cases

124 109 Fig Autopilot using Damage Mitigation

125 110 Fig Autopilot not using Damage Mitigation

126 111 Fig Crack Length and Damage Weights

127 112 Chapter 6 Hierarchical Control of Aircraft Propulsion Systems: Discrete Event Supervisor Approach 6.1 Introduction This chapter presents an application of Discrete Event Supervisory (DES) control theory for intelligent decision and control of a twin-engine aircraft propulsion system. A DES control system with a two-layer hierarchical architecture is proposed and designed to coordinate the operations of a twin-engine propulsion system. Each engine is operated under a continuously varying feedback control system that maintains the specified performance under the supervision of a local (lower level) discrete-event controller for condition monitoring and life extension. The two engines are individually controlled to achieve enhanced performance and reliability, as necessary for fulfilling the mission objectives. A global (upper level) DES controller is designed for load balancing and overall health management of the propulsion system. Discrete-event dynamical behavior of physical plants is often modeled as regular languages that can be realized by finite-state automata [14] [62]. This chapter focuses on development of intelligent decision and control algorithms based on the theory of Discrete Event Supervisory (DES) control for a twin-engine aircraft propulsion system. The DES control system is designed to be hierarchically structured in the following sense: continuously varying control of each engine interacts with its own local DES

128 113 controller for health monitoring and intelligent control; and the operational information is abstracted and reported to the propulsion-level DES controller that coordinates the operation of two engines. Furthermore, the propulsion-level supervisory control system allows interactions with exogenous inputs, such as human operators and inputs from other units (e.g., flight control, structural control, energy management, and avionic systems) of the vehicle management system for flexibility of making on-line modifications in the mission objectives. A good feature of the proposed DES control approach is that the control policy can be adaptively updated on-line at both engine and propulsion levels and that the system is tolerant of small anomalies and component faults. Although the theory of DES control has been developed for almost two decades [62], only very few applications have been reported in literature. An apparent reason is that, until recently, no quantitative analytical tool was available for design and evaluation of DES controllers. The work reported here makes use of a quantitative measure of regular languages [81] [95] [67], and is a novel application of hierarchical DES control synthesis for the non-linear complex dynamical system of twin-engine aircraft propulsion. The realtime implementation of the DES scheme is challenging because it requires integration of several disciplines such as systems theory, computer hardware and software, and domain knowledge of gas turbine engine propulsion. This chapter presents synthesis of a Discrete Event Supervisory (DES) system for intelligent decision and control of twin-engine aircraft propulsion. The DES control of the propulsion system is validated on a simulation test bed. Thus, feasibility of the DES concept is demonstrated for enhanced operation and control of twin-engine aircraft propulsion in the following areas:

129 1. real-time decision-making for propulsion control (e.g., load balancing between engines); damage reduction (with no significant loss of performance) via life extending control; and 3. improvement performance, and reliability of the mission. In this chapter the terms controller and supervisor are used interchangeably, also the phrases Upper level and Propulsion Level are synonymous and similarly the phrases Lower level and Engine Level convey the same meaning. This chapter is organized in five sections including the present one. Section 6.2 describes the real-time simulation test bed of the twin-engine propulsion system. Section 6.3 presents the syntheses of the engine level DES control and propulsion level DES control systems. Section 6.4 presents the simulation results and discusses implications of the controller design. The chapter is summarized and concluded in Section Description of the test bed for the propulsion system simulation This section presents the implementation of Discrete Event Supervisory (DES) control on a real-time simulation test bed of a commercial turbofan engine. The objective is to validate the theory of optimal DES control for a real-world nonlinear complex dynamical system. A DES controlled propulsion system has been designed and implemented on a simulation test bed that consists of three networked computers using the client/server concept. One of the three computers hosts the propulsion system coordinator for health

130 115 monitoring of the engines and accordingly making intelligent decisions (e.g., load balancing and turning on/off of individual engines in the extreme cases). The other two computers run separate copies (which may or may not be different depending on the health of the individual engines) of the aircraft gas turbine engine simulation model including its (continuously varying) control system and a local discrete-event supervisor. The test bed is capable of simulating different dynamics for individual engines due to nonuniform operating conditions. Each of the engine simulators integrates the event-driven discrete dynamics modeled by finite-state automaton as well as time-driven continuous dynamics modeled by ordinary differential equations through continuous-to-discrete and discrete-to-continuous interfaces [27]. Figure 6.1 shows the supervisory control architecture of the engine propulsion control system. This software architecture is flexible to adapt other DFSA models and controller designs for other complex dynamic systems. Each major function in the simulation program has a modular structure as implemented on the three computers of the simulation test bed. One of the computers hosts the propulsion level DES controller whose primary task is load balancing between two engines; and the other two computers execute the commercial turbofan engine simulation program along with engine level DES controllers. In order to superimpose the DES control system on the existing turbofan engine simulation program (FORTRAN language) of the engine plant model and the continuously varying engine control system, a C++ code was written to wrap the FORTRAN simulation code. The wrapper program interfaces the major inputs and outputs with the rest of the program in the C++ environment. This approach is especially useful

131 116 to utilize the available FORTRAN models without any significant changes and to treat them as individual modules of the C++ program. The turbofan engine simulation program, which consists of large order nonlinear differential and difference equations and supporting algebraic equations, has been designed for both steady-state and transient operations of a turbine jet engine. With the proper inputs such as power lever angle (PLA), and ambient conditions (e.g., altitude, forward speed, and ambient temperature), the FORTRAN engine model simulates the complex operation of the engine from a steady state to transients and to (possibly) other steady states. This simulation code is a stand-alone program with a continuous-time gain scheduling controller. The engine simulation model provides various sensor data (e.g., combustion-chamber temperature and high-pressure turbine speed) together with other critical information (e.g., simulation step size and simulation cycle number), which are collected by the C++ wrapper program and exchanged with DES controller through the Message API communication routine. Figure 6.2 shows the architecture of the engine level plant and DES controller implementation, which has two replicas, with possible different parameters and initial conditions, in two different computers. Figure 6.3 shows the organization of the propulsion level DES controller together with its own (discrete) event generator, which is implemented on a third computer and makes use of the messaging interface to communicate with the other two computers. The DES controller design has two important components that serve as interfaces between the continuously-varying control system and the discrete-event supervisory controller: one is Event Generator and the other is Action Generator. Event Generator

132 117 receives continuously varying sensor data from the engines. The data along with other information like estimated state and external inputs are used to generate events that, in turn, are inputs to unsupervised Deterministic Finite State Automata (DFSA) model of engine operation. The unsupervised DFSA model is constructed based on the operation scenario and the details are discussed later in Section 5. The state-based DFSA model serves as state estimator and provides information on engine states and (both controllable and uncontrollable) events for the discrete-event supervisor to take appropriate actions. Event behavior in the state-based DFSA model is dependent on the state where the event is generated and not on the history or the path of how the state is reached. The DES controller represents the control policy applied to the DFSA model of engine operation, and it could be a conventional DES controller based on the control specifications [62] provided by an experienced designer; alternatively, an optimal discrete-event supervisor can be designed by the quantitative method, presented in Section 3. In both cases, the DES controller takes the estimated states as inputs and generates control commands (of controllable event disabling or enabling) as outputs. The control commands are transmitted through a Message API communication routine to the Action Generator. The primary task of the action generator is to convert control commands from the supervisor into necessary input functions for the continuously varying plant. 6.3 Synthesis of Discrete Event Supervisory(DES) Controllers One of the major tasks of the supervisory decision making is fusion of the (possibly) redundant, conflicting, and incomplete information to make timely decisions. Such

133 118 information can be derived from different types of sensor data as well as operational history and the knowledge base generated from pilot s personal experience. Computerbased advanced analytical techniques are necessary for fusion of the time series data available from multiple sensors and other relevant non-sensor-based information (e.g. weather data, analytical damage model data) to make specific inferences that could not be achieved through the sole usage of sensors. However, improved performance may not result simply from an increased volume of sensor data and engine information unless the ensemble of information is systematically processed in the context of the engine operational conditions and mission objectives. In essence, fusion of the heterogeneous information is necessary: to guarantee improvement of resolution and reduction of ambiguity in decision and control to make advantageous trade-offs between probability of false alarms and missed detection [27] The unsupervised dynamics of engine operations are modeled as a DFSA, based on postulated engine operation scenarios. (Note that the model may change for different mission objectives.) The DFSA model assumes that a twin-engine aircraft is carrying out a routine surveillance mission. The mission abortion is allowed at certain states according to the operation scenario. Each engine of the aircraft is equipped with a continuous time controller which is supervised by a local DES controller. The primary objective of the local engine level DES controller is to strike the right balance between the conflicting demands of higher performance from upper level supervisor and limiting

134 the damage to the engine. The upper (i.e., propulsion) level DES controller redistributes the load depending on the health of the engine and thrust demand placed by the pilot Engine Level DES control Figure 6.4 presents the DFSA model of the unsupervised engine operations for discrete-event supervisory control. Table 6.2 lists the events, where C denotes controllable events and UC denotes uncontrollable events; and Table 6.3 lists the plant states. The events and states for the DFSA models at the engine level are denoted by lower case letters (e.g., is the start event and is the engine start state). The engine can operate in two regimes, one is high performance regime (state q 3 ), where the damage rate is also high. The other is low performance regime (state q 5 ) where the damage rate is low. In the high performance regime the engine has a tendency of going to state q 4, where engine variables like combustor temperature have been observed to have oscillatory behavior. Temperature oscillations could be extremely harmful for engine health and therefore must be avoided at all costs. Engine level controller chooses the regime of operation (q 3 or q 5 ) depending on two factors: thrust requirement at the propulsion level and health of the engine as explained below. Health of the engine is determined from the damage accumulation that is a function of high-pressure turbine gas inlet temperature and shaft speed; and in addition, at random time intervals spikes (i.e., sudden jumps) are introduced to simulate real world damage impacts in an engine. The DFSA model of the supervised engine operations is shown in Figure 6.5, where the dashed lines indicate the controllable events that are disabled by the optimal control algorithm described in Section 3.

135 Propulsion Level DES control One of the main tasks of the propulsion level DES controller is to redistribute the load between two engines depending on the current health condition of each engine and the thrust demand. Another important motive of the propulsion level DES controller is to enhance the mission performance. Thus, mission scenario elements, like successful mission and abort the mission are used in the modeling stage. Since propulsion level supervisor in the hierarchical control scheme should be able to carry out the key operational features of the engine level supervisors, in the propulsion level operating regimes of individual engines are included in the model as a Cartesian product of the state sets of two supervised engine DFSA models. However, model order reduction via aggregation and deletion of unrealizable states is needed because the above Cartesian product will cause a very large number of resulting states. The events and states of the unsupervised DFSA model at the propulsion level are listed in Tables 6.4 and 6.5, respectively. Events and states for the DFSA model at the propulsion level are denoted by upper case letters (e.g., is the start event and Q 1 is the aircraft on the ground state). 6.4 Simulation Experiments: Results and Discussion Experiments were conducted on the simulation test bed, described in previous sections, to validate the DES control concept. Upon successful implementation of the software modules on the client and server computers, several sets of simulation experiments were performed. The first set of experiments was performed on the engine level

136 121 DES controller to demonstrate how the engine component damage is reduced and consequently the engine life is enhanced. Then, the propulsion level DES controller that is built upon the engine level DES controllers is investigated. Figure 6.6 exhibits the predetermined input profile (Power Lever Angle - PLA) that excites both unsupervised and supervised propulsion systems. This profile tries to mimic the actions of a pilot. It should be noted that Altitude and Mach Number were kept constant (Altitude= feet and Mach Number=0.8) during the simulation runs. Only engine model is considered here therefore this assumption is reasonable, but for an actual airplane these variables (PLA, Altitude and Mach Number) cannot be varied independently. The most commonly measured engine variables are: gas path temperatures and pressures, rotor speed, fuel flow, throttle position, nozzle position, stator position, oil properties, vibration, and life usage [39]. Time responses of several engine variables (e.g., combustor outlet temperature, high pressure turbine speed, net thrust of the engine, and fuel flow through the main burner) over a period of 12 minutes were observed. The four plates in each of Figures 6.7 and 6.8 exhibit the response profiles of the above set of plant variables for completely unsupervised and completely supervised propulsion at both levels, respectively. A comparison of plots in Figures 6.7 and 6.8 indicates that the DES control at the engine level eliminates the high frequency oscillations that are present in the unsupervised plant responses in Figure 6.7. Physically these oscillation correspond to a limit cycle type of a behavior. The supervisor takes actions immediately upon detection of oscillations. Due to supervisor s actions, the potentially sustained oscillations are quenched very fast and the engine operation is brought to steady state.

137 Thus, sustained oscillations are practically non-existent in the supervised plant responses 122 in Figure 6.8. Since high frequency oscillations of temperature and pressure are the primary sources of fatigue crack damage in the turbine blades, disks, and stationary vanes, the supervisory control becomes very effective for mitigation of structural damage in the engine components. In contrast, in the unsupervised case, the engine health would be adversely affected if the propulsion system is operated in this way to achieve the mission objectives. The propulsion level DES controller has three main tasks. The first task is the intelligent decision making and control of the twin-engine aircraft propulsion systems for mission execution; the second task is to improve the overall mission and operation behavior so that engine health can be enhanced via damage reduction; and the third task is load balancing between two engines so that the propulsion system produces the thrust demanded by the flight control system provided the engine life is enhanced. The issue of load balancing becomes even more important when the health conditions of two engines are significantly different (one can be in bad condition and the other in good condition). If the situation comes to this point, then the aim of the DES controller is judicious redistribution of the load between two engines such that the bad engine carries lower load than the good engine, subject to the condition that the total thrust output of the engines satisfies the mission and safety requirements. Figures 6.9 and 6.10 shows the simulated outputs of two engines, where both engines are in good health condition at the start of simulation. As the mission progresses, one engine deteriorates and starts to run at bad health condition, and after some time the other engine also deteriorates. Thus, the load distribution of the engines varies in three regions. In the beginning, the

138 123 load is equally distributed in Region 1. Then, the good engine takes the responsibility of producing higher thrust, as seen in Region 2 in the plates of Figures 6.9and Later on, both engines are again loaded equally as seen in Region 3 when it is not advisable to impose uneven thrust requirements on the two engines Evaluation of State Transition Matrix Parameters To quantitatively evaluate the impact of the propulsion level DES controller on the overall mission behavior, concept of language measure, described earlier, has been used in the analysis. Given the state weights and the state transition probabilities, the language measure provides a quantitative performance measure of the controllers. Analysis and synthesis of optimal DES controllers require the identification of the event cost matrix. Similar to continuously varying dynamical systems (CVDS), one must use the techniques of system identification to evaluate the language measure parameters of the DFSA plant model, i.e., the elements π ij of the event cost matrix Π. As the number of experiments increases, the identified event costs tend to converge within an appropriately specified tolerance. For stationary operation of the engine, since conditional probabilities of the events can be assumed to be time-invariant, the identified event costs and their uncertainty bounds can be determined. As a typical case, Figure 6.11 presents identification of event costs at state Q 5, where both engines are in lowperformance operation. Fifty experimental runs were conducted for both supervised and unsupervised cases to construct the state transition probability matrix Π. The different visited states and the triggered events were monitored and plotted. After identifying the event cost

139 matrix Π, it is a straightforward task to obtain the state transition cost matrix. Table 6.6 lists the Π-matrix of the DFSA at the propulsion level Selection of Characteristic Values for Quantitative Evaluation of DES Controllers Given the state characteristic values and the state transition probabilities, the language measure serves as a theoretical performance measure for quantitative measure of DES controllers. The characteristic values are assigned based on the designer s perception of the importance of terminating on specific marked states. For the propulsion level DES controller, the weights of the states are selected according to each state s importance (contribution) to the mission management as ( χ = ) The marked states that are assigned non-zero weights are as follows: Q 8 : Both engines failed, State Q 9 : Mission abort, State Q 10 : Mission successful. They have relative weights of -1, -0.2 and 0.3 respectively. The bad marked state, Both engines failed, is assigned the characteristic value of because the aircraft will most likely be destroyed if the DFSA terminates on this state. On the other hand, the good marked state, Mission successful, is assigned the characteristic value of based on its relative importance to the loss of the aircraft. Another bad marked state, Decision to abort mission, has a negative characteristic value which signifies the importance of this state relative to a successful mission and a possible loss of the aircraft. Using µ = [I Π] 1 χ.

140 125 obtained in previous section, the language measures (i.e., the theoretical performance of the propulsion control system) are calculated as µ unsupervised = and µ unsupervised = It is seen that the DES controller used in the propulsion level has a positive effect on the mission behavior of the overall system Optimal DES controller synthesis and evaluation The state transition cost matrix Π is determined from the event cost matrix Π, and the transition function δ of the finite state automaton. After identifying the event cost matrix Π 0 of the unsupervised plant G, the state transition cost matrix Π 0 is formed. State transition cost matrix is basically the only unknown input for the optimal control algorithm to design the optimal DES controller, since the other necessary parameters, such as the characteristic vector χ, are the selected by the designer based on the design requirements. Therefore, given the state transition cost matrix Π 0 of the unsupervised plant automaton and the state characteristic vector χ, the optimal DES controller can be synthesized as described in earlier section. Table 6.7 lists the iterations of optimal control synthesis for the propulsion level supervisory control. The performance measure of the unsupervised plant is negative at the states Q 6, Q 7, Q 8, Q 9, Q 11, and Q 12 as indicated in Table 6.7. All controllable events leading to these states are disabled and the resulting performance measure at Iteration 1 shows sign change at states Q 7, Q 11, and Q 12 as in Table 6.7. All controllable events leading to these states are now re-enabled for further increase in performance at Iteration 2. However, there is no sign change in the performance vector between Iteration 1 and Iteration 2, which immediately shows that the algorithm converged to the

141 126 optimal solution after this iteration. The synthesis is complete in Iteration 2 (i.e., there is no need to go for the Iteration 3) because there is no sign change; moreover, the performance vector at Iteration 2 shows also no improvement after the previous iteration. The performance of the optimal controller was compared with that of unsupervised plant and the supervisor S that was designed using the conventional procedure. Theoretical performance of the supervisors can be associated with the language measure of each supervisor, as described previously in Section 2. The language measure of the unsupervised plant and that of the supervised plant at the propulsion level are listed in Table 6.8. Table 6.1. Simulated Performance of DES Controllers State Relative Weight Unsupervised Supervised Optimal Plane destroyed Q Mission Abort Q Mission Successful Q Experimental outcomes can also be used to evaluate the DES controller performance directly by multiplying each state visit with the relative weight of the state. Table 6.1 shows the number of visits to the states that have non-zero weights. The performance of the unsupervised plant (i.e., null supervisor) and the other two supervisors (i.e., conventionally designed and optimal one) are compared based on the observations of mission execution on the simulation test bed. The mission outcomes of the unsupervised

142 and supervised propulsion systems were recorded during the simulated missions. The numbers of visits at the good and bad marked state were multiplied by the respective 127 relative weights of the states to calculate the system performance. The experimental evaluations of the performance for different supervisors are presented in Table 6.8. Both theoretical and (simulation) experimental evaluations of DES controllers provide better mission management under optimal supervision. It is seen that the theoretical performance of the supervisors is in qualitative agreement with the experimental results, presented in Table 6.8. Optimal DES controller, synthesized using the algorithm described in Section 3, has the highest theoretical performance (i.e., highest language measure) among all controllers; the optimal supervisor is also least restrictive. The results of the simulation experiments concur with the theoretical measure of the controllers in the sense that optimal supervisor yields the best mission performance. Therefore, the optimal supervisor not only yields the best mission performance of all supervisors under evaluation and unsupervised plant, but also functions as an intelligent coordinator at the propulsion level for the load balancing between two engines. These claims are clearly demonstrated by the simulation results. Finally, some remarks about the designed optimal DES controller would be useful to underline certain aspects of the synthesis. At the first step of the iterations, the performance measure of the unsupervised plant is negative at states Q 6, Q 7, Q 8, Q 9, Q 11, and Q 12. Therefore, controllable transitions to these states should be disabled. When states Q 6 and Q 7 are considered, we see that one engine is lost at those states. This means the events leading to the states would have one engine failed condition which is uncontrollable, and cannot be disabled. Similarly for state Q 8, only transition is both

143 128 engines failed. This condition is also uncontrollable, so this transition cannot be disabled either. The transitions leading to states Q 11 and Q 12 are deterioration of engines, which is directly related to the damage information coming from the engine level DES control system. According to this information, the propulsion level supervisor decides on the health condition of the individual engines. Evidently, this kind of sensory events are uncontrollable, so is the engine deterioration event. Remaining state that should be investigated is state Q 9, Mission abort. According to the optimal control algorithm, all mission abort requests should be disabled which will result in performance increase in the mission behavior. The resulting controller doesn t allow mission abortion in any state. This kind of control strategy is aggressive compared to the other ones. However, due to the fact that the characteristic weight for aborting the mission is less than that of a successful mission, this result is not totally unexpected. After running the simulation experiments, aggressiveness of the optimal DES controller seems to increase the mission performance at the propulsion level. 6.5 Summary and Conclusions This chapter presents a quantitative approach to synthesis of an optimal discreteevent supervisory (DES) control of an aircraft propulsion system. The DES control law has been validated on a networked simulation test bed. The plant dynamics in the simulation test bed is built upon the model of a generic turbofan gas turbine engine. The software architecture of the simulation test bed is flexible for adaptation to arbitrary DFSA models and a variety of supervisor designs. The supervisory control laws are quantitatively analyzed using a language measure.

144 129 Table 6.2. Event List for the Engine Level DFSA Model Event Description Status a Start C b Warm Up Complete UC c Shut Engine UC d Detection of Oscillations UC e Nozzle Area Reduction C f Engine Fails UC g Reduce Performance /Reduce Damage C h Increase Performance/ Increase Damage C i State Unchanged C Table 6.3. State List for the Engine Level DFSA Model State Description Status q 1 Engine Start Unmarked q 2 Engine Warm Up Unmarked q 3 High Performance/ High Damage Rate Marked (good) q 4 Oscillations Marked (bad) q 5 Low Performance/ Low Damage Rate Marked (good) q 6 Engine Inoperable Marked (bad)

145 130 Table 6.4. Event List for the Propulsion Level DFSA Model Event Description Status A Start Engine C B Warm Up Complete UC C One Engine Deteriorates UC D Redistribute Load (1) C E Both Engines Deteriorate UC F One Good Engine Fails UC G Both Engines Fail UC H Increase Performance C I Reduce Performance C J Request Abort Mission C K Request Accepted UC L Request Rejected UC M Mission Accomplished UC N Turn off Engines UC O Redistribute Load (2) C P Redistribute Load (3) C Q Request Rejected UC R One Bad Engine Fails UC

146 131 Table 6.5. State List for the Propulsion Level DFSA Model State Description Status Q 1 Engines on ground Unmarked Q 2 Engines warming up Unmarked Q 3 Both engines in High Performance operation Unmarked Q 4 One engine in High one engine in Low Performance Unmarked Q 5 Both engines in Low Performance operation Unmarked Q 6 One engine stopped one engine in High Performance Unmarked Q 7 One engine stopped one engine in Low Performance Unmarked Q 8 Both engines failed Marked (bad) Q 9 Decision for abort mission Marked (bad) Q 10 Mission successful Marked (good) Q 11 High damage detected for one engine Unmarked Q 12 High damage detected for both engines Unmarked Table 6.6. Π matrix of the Propulsion Level DFSA Model States Q 1 Q 2 Q 3 Q 4 Q 5 Q 6 Q 7 Q 8 Q 9 Q 10 Q 11 Q 12 Q Q Q Q Q Q Q Q Q Q Q Q

147 132 Table 6.7. Iterations for Optimal DES Controller Synthesis State Unsupervised Plant Iteration 1 Iteration 2 Q Q Q Q Q Q Q Q Q Q Q Q Table 6.8. Language Measure and Performance for Different Supervisors Plant Indices Unsupervised Supervised Optimal Language Measure(µ) Performance Index

148 133 Fig Overall architecture

149 134 Fig Engine Level Plant/DES controller

150 135 Fig Propulsion Level DES controller Fig Engine Level DES plant model

151 136 Fig Engine Level DES controller model Fig Power Lever Angle input

152 137 Fig Simulation output for the unsupervised case

153 138 Fig Simulation output for the supervised case

154 139 Fig Effect of propulsion level DES controller on Engine-1

155 140 Fig Effect of propulsion level DES controller on Engine-2

156 141 Fig Convergence of event costs

157 142 Chapter 7 Anomaly Detection for Health Management of Aircraft Gas Turbine Engines 7.1 Introduction This chapter presents a comparison of different pattern recognition algorithms to identify slow time scale anomalies for health management of aircraft gas turbine engines. A new tool of anomaly detection, based on Symbolic Dynamics and Information Theory (for details see chapter3), is compared with traditional pattern recognition tools of Principal Component Analysis (PCA) and Artificial Neural Network (ANN). Time series data of the observed variables on the fast time scale are analyzed at slow time scale epochs for early detection of anomalies. The time series data are obtained from a generic engine simulation model. Health monitoring of gas turbine engines based on these techniques is discussed. The traditional Engine Health Management (EHM) approach uses fleet statistical data and signal processing techniques to detect and isolate faults. In the last few years, there has been a prognostic emphasis on the traditional Fault Detection and Isolation (FDI) approach [39] [41]. There has been a significant interest in the area of algorithm development. Although advancements have been made in four general areas: system partition system architecture

158 143 EHM functionalities algorithmic approaches The latter two have represented the majority of the published work. Algorithms are used in a monitoring system to calculate capabilities and margins; algorithms are also used in reasoning processes to make decisions. Certain algorithmic approaches may be more effective for a given EHM functionality than others, but typically, algorithms fall into two categories: data-driven algorithms and model-based algorithms, while a combination of the two, i.e., a hybrid algorithm, may offer the benefits of both [94]. The major requirements of EHM are [89]: automated monitoring, analysis, and decision support accurate results with high confidence robust capabilities against noise and faulty information wide coverage of fault conditions predictive capabilities using existing, or as few as possible, sensing instruments flexible, modular, and open architecture user friendliness Anomalous operation of gas turbine engines is undesirable from the perspectives of both engine operation and aircraft mission management. Early detection of anomalies

159 144 and their characterization are essential for health management, that includes prognosis of impending failures in critical components and mitigation of their detrimental effects on the engine operation. For detection of these slow time scale deviations, it might be necessary to rely on the time series data generated from the available sensors as well as other relevant information regarding pilot s experience. Since sufficiently accurate and computationally tractable modeling of thermo-fluid and structural system dynamics is often infeasible solely based on the fundamental principles of physics, small changes in the system behavior may be inferred from both time series analysis of the sensor data and model-based information [2] [44]. By taking advantage of the available model-based information, the time series data can be converted, by phase-space partitioning, into a symbol sequence [5] that, in turn, will generate a finite-state machine model of the dynamical system behavior. Health monitoring of gas turbine engines can be viewed as a class of slow time scale problem [93]. For gas turbine engines, health condition of an engine changes in hundreds of hours, whereas the engine runs at a much faster time scale, usually in the order of seconds. Identification of the current state of the engine health is very important for maintenance engineers because and necessary repairs must be carried out before the engine becomes permanently non-operable. Thus, it is essential to monitor slow-timescale anomalies for gas turbine engines from the time series data of the engine response. A generic gas turbine engine simulation [27] test bed has been used to validate the anomaly detection techniques. The anomaly detection algorithm is built upon two-time-scale analysis of stationary behavior of dynamical systems using the principles of Symbolic Dynamics [45]

160 145 [51], Information Theory [3], Automata Theory [76], and Pattern Recognition [29]. Symbolic Dynamics captures the essential dynamical features of the physical process through phase-space partitioning. Information Theory allows modeling of incipient catastrophic failures and chaotic behavior that are analogous to thermodynamic phase transitions. Automata Theory generates finite-state machine models of the dynamical system behavior under nominal and anomalous conditions. Pattern Discovery methods infer anomalies through quantitative evaluation of the deviations in statistical patterns of the respective state machines from those under the nominal condition. These features are described briefly in the following section. This chapter is organized in six sections including the present one. Section 2 reviews the concepts of the symbolic dynamics and finite state machine based anomaly detection. Section 3 summarizes the radial basis function approach to solve the anomaly detection problem. Section 4 discusses principle component analysis technique. In section 5, the simulation results are discussed. The chapter is summarized and concluded in Section Symbolic Dynamics Due to the simplicity and ability to capture the state dynamics in a lower dimensional space, Symbolic dynamics is a widely used tool in the pattern recognition applications. In general, state vectors of a process may form a very high dimensional space which is difficult to handle both mathematically and physically. However, symbolic dynamics uses the phase trajectory partitioning of this high order space to represent the continuous dynamics with a predefined alphabet of symbols [18].

161 146 These underlying concepts of symbolic dynamics and partitioning of phase space for anomaly detection of gas turbine engine are briefly introduced in a previous publication [44]. Further details on symbolic dynamics are available in [76] and on phase-space partitioning in [43]. Time series data of combustor outlet temperature, which is used for early detection of incipient faults is converted to a symbol sequence by partitioning the finite region in the phase-space of engine operation dynamics, over which the time series data evolves, into finitely many discrete blocks [2] [5]. Each block is labeled as a symbol σ Σ, where the symbol set Σ is called the alphabet consisting of m different symbols. In this way, a data sequence, obtained from a trajectory of the dynamical system, is converted to a symbol sequence (σ i, σ j, σ k,...) that characterizes the system dynamics represented by the data sequence. Critical steps in the symbol generation process are: (i) partitioning of a finite region in the phase space; and (ii) construction of a mapping from the partitioning into the symbol alphabet, which becomes a representation of the system dynamics defined by the trajectories. Finding the dimensionality of the phase space of relevant system dynamics can be difficult especially if the time series data are noise-corrupted [2] [43]. There are potentially a number of ways to do this. Two such methods to perform the partitioning are described below. Kennel and Buhl [43] have formulated a phase-space partitioning method that is built upon the concept of symbolic false nearest neighbors (SFNN), where a statistic and algorithm is introduced to refine empirical partitions for symbolic state reconstruction. This method avoids topological degeneracy that is an essential feature of a generating partition [5]. The major advantage of this method is that the partitioning is entirely by the algorithm based on the

162 time series data. The partitions are defined with respect to a set of radial-basis influence functions, f k (x) = 147 α k x z k 2, each associated with a symbol s k with the center z k and weight α k. For each element x of the time series data set, one f m (x) is generally expected to be greater than other f k (x) with k m. Then, the data point x in the phase space is transformed to a symbol s in the symbol space. The parameters z k and α k are the free optimization variables, with the constraint α k 0 k. There may be one or more influence functions assigned to each of the symbols in the alphabet. The partitions remain invariant at all epochs of the slow time scale. An alternative scheme for obtaining partitions is based on wavelet transform of time series data, which yields a graph of coefficients versus scale at each time shift. After the wavelet transform [40] is applied to the data, we partition the space of wavelet coefficients that is a function of scale and time. These graphs are stacked from end to end starting with the smallest value of scale and ending with the largest value. For example, the wavelet coefficients versus scale at time shift t k are stacked after the ones at time shift t k 1 to obtain the so-called scale series data in the wavelet space, which is analogous to the time series data in the phase space. In this chapter, the wavelet space is partitioned into horizontal slabs; future work will focus on optimized partitioning of the wavelet space. The number of blocks in a partition is equal to the size of the alphabet and each block of the partition is associated with a symbol in the alphabet. For a given stimulus, the partitioning of wavelet space must remain invariant at all epochs of the slow time scale. After the partitioning is finalized, the sequence of symbols is generated from the time series (or scale series) data at different epochs of the slow time at which anomalies may take place. Then, a probabilistic finite state automaton is

163 148 constructed from the symbol sequence at each time epoch. The anomaly measure at a given epoch is obtained as a distance between the state probability vector of the finite state machine at that epoch and the state probability vector of the finite state machine at the nominal condition. Thus, the above vector measure quantifies growth of anomaly from the nominal condition in the slow time scale. Finite state machines, generated from the symbol sequences of a dynamical system identify its behavioral pattern. As the system trajectory evolves, different states are visited with different frequencies. The number of times a state is visited as well as the number of times a particular symbol is received, while sliding the window from a state leading to another state, is counted. In this way, the state probability vector is calculated from the time series data associated with different anomalous conditions. Having obtained the state probability vectors, the next step is to calculate the anomaly measure that signifies the change in stationary behavior of the dynamical system as the fault progresses. The state probability vector under the nominal case serves as a benchmark. The anomaly measure is obtained as the norm of the difference between the state probability vector associated with faulty behavior and the state probability vector for the benchmark condition. Obviously, the deviation measure at the benchmark condition is zero. The step by step process is depicted in Figure Radial Basis Function Neural Network Neural networks learn complex input-output relationships, use sequential training procedures, and adapt themselves to the data. The most commonly used family of neural networks for pattern classification tasks is the feed-forward network. The learning

164 149 process involves updating network architecture and connection weights so that a network can efficiently perform a specific classification/clustering task. Neural networks provide a new suite of nonlinear algorithms for feature extraction and classification. However, most of the well-known neural network models are implicitly equivalent or similar to classical statistical pattern recognition methods. Despite these similarities, neural networks do offer several advantages such as, unified approaches for feature extraction and classification and flexible procedures for finding good, moderately nonlinear solution. A major class of neural network model is the radial basis function (RBF) neural network (NN) [8] in which the activation of a hidden unit is determined by the distance between the input vector and the prototype vector. The RBF NN is essentially a nearest neighbor type of classifier. The technique used in the anomaly detection is a variation of RBF NN to form the statistical model of nominal data. As new data enters into the system, it is compared with the RBF NN model. If it falls within the boundaries defined by the model than it is considered as a nominal data, if does not than the data is considered as anomalous. The approach is generic and has been applied to a variety of problems including advanced military aircraft subsystems [12][85]. The key aspect of Neural Net approach is the appropriate selection of the radial basis function and the order of the statistics of the model. From this perspective, a radial basis function is given by Equation 7.1. f(x) = exp{ xk µ α θ α } (7.1) where the parameter α (0, ); µ and θ α are the center and central moment of the data set, respectively. For α = 2, f( )becomes Gaussian, which is the typical radial

165 150 basis function used in the neural network literature. From a sampled time series data under the nominal condition, the mean µ and the central moment θ α are calculated as in Equations 7.2 and 7.3. µ = 1 N N x k (7.2) k=1 N θ α = x k µ α (7.3) k=1 The distance between any vector x and the center µ is obtained in Equation 7.4. N x µ lα = ( x k µ α ) α 1 (7.4) k=1 Hence, at the nominal condition, the radial basis function f nom = f(x). For different anomalous conditions, the parameters, µ and θ, are kept fixed; and f k is evaluated from the data set under the (possibly anomalous) condition at the k th epoch at the slow time scale. Then, the anomaly measure is defined as the distance function M given by Equation 7.5 M = d(f nom, f k ) (7.5) 7.4 Principal Component Analysis In the statistical approach, each pattern is represented in terms of d features or measurements and is viewed as a point in a d-dimensional space. The goal is to choose those features that allow pattern vectors belonging to different categories to occupy

166 151 compact and disjoint regions in d-dimensional feature space. The effectiveness of the representation space (feature set) is determined by how well patterns from different classes can be separated. Given a set of training patterns from each class, the objective is to establish decision boundaries in the feature space which separate patterns belonging to different classes. In the statistical decision theoretic approach, the decision boundaries are determined by the specified or learned probability distributions of the patterns belong to each class. Feature extraction methods determine an appropriate subspace of dimensionality m in the original feature space of dimensionality d (m d). The best known linear feature extractor is the Principal Component Analysis (PCA) [29] that makes use of Karhunen-Love expansion to compute the m largest eigenvectors of the d d covariance matrix of the N d-dimensional patterns. Since PCA uses the most expressive features (eigenvectors with the largest eigenvalues), it effectively approximates the data by a linear subspace using the mean squared error criterion. To detect growth in anomaly from time series data, the PCA can be performed for dimensionality reduction. For a time series data with length L, a data matrix of size M N is to be created by dividing the length L time series into M = L/N length N subsections. Each row of the data matrix is a length N subsection obtained above. Then then N covariance matrix can be obtained from the data matrix. After determination the orthonormal eigenvectors υ 1... υ n and eigenvalues λ 1... λ n of the covariance matrix, where the eigenvalues are in increasing orders of magnitude, it is possible to choose the m largest eigenvalues and associated eigenvectors,so the the inequality in Equation 7.6 is satisfied.

167 152 m N λ i > η λ i (7.6) i=1 i=1 Where η < 1 is a real positive number close to 1 (e.g., η = 0.95 ). The feature matrix F is defined by Equation 7.7 ( λ F = 1 Nk=1 υ λ 1... k ) λ m Nk=1 υ λ m k (7.7) The feature matrix F nom for nominal condition and feature matrices F 1, F 2... corresponds to different health conditions are formed. The anomaly measure for health condition k defined by Equation 7.8: M = d(f nom, F k ) (7.8) where d is a metric signifying the difference between the nominal and the anomalous condition. For the results shown in the next section, metric was chosen to be the Euclidian norm. 7.5 Experimental Results This section presents the results of the application of various anomaly detection methods on the simulation test bed of a generic gas turbine engine, similar to that reported in [20]. The model contains steady state performance maps for all the components and has control volumes where continuity and energy balances are maintained. Rotor dynamics and the duct momentum dynamics are also included in this model.

168 153 In the health monitoring of the engine, the degradation of engine efficiency from the nominal condition (e.g., a new or an overhauled engine) can be treated as anomaly. For implementation purposes, the high and low pressure turbine efficiencies, fan and compressor tip velocity ratios are reduced to observe the effects on the time series data. Here, the degradation in efficiencies of different engine components is assumed to be uniform. This assumption is reasonable for the comparative study presented herein. The health condition (e.g., efficiencies) of engine components changes at a very slow time scale. Thus, for a short span of time, the efficiency values are assumed to remain approximately constant for all practical purposes. Replication of these conditions may require hundreds of hours of engine simulation. Since this is not a feasible solution, engine component efficiencies are reduced for each run of the simulation and a certain period of operation is observed for each health condition. This is conceptually similar to experimental methods, where an engine is tested under extreme conditions (accelerated degradation) to simulate many hours of engine operation within small time duration. Only steady state data was used for this analysis. For anomaly detection, the time series data from a number different sensor sources was analyzed Based on these simulation experiments it was found that the combustor outlet temperature (T c ) and the main burner fuel flow rate (F f ) variables, best captured the degradation of the engine. From thermodynamic perspective, it was expected that these two variables would be affected by the change in engine efficiencies. The results of the simulation corroborate this fact. Upon further investigation it was found that the combustor outlet temperature (T c ) essentially captured the dynamics of degradation and using the fuel flow rate (F f ) did not add anything significant to the analysis. In

169 other words, from the perspective of anomaly detection F f and T c are not independent. Therefore, the results on anomaly detection are based on the time series analysis of 154 the combustor outlet temperature. The sets of time-series data were collected after the dynamic response attained the stationary behavior. These data sets were used to compare the anomaly detection capability of the symbolic dynamics approach relative to that of two existing pattern recognition techniques: Principal Component Analysis (PCA) and Radial basis Function Neural Network (RBFNN). Since symbol generation from time series data is the crucial step in symbolic-dynamics-based anomaly detection, this paper investigates two alternative approaches: Symbolic False Nearest Neighbor (SFNN) partitioning and Wavelet Space (WS) partitioning. Time series data belonging to different health conditions (up to 2.5% efficiency drop) has been used for the analysis purpose. It can be seen in Figure 7.2 that the output data at steady state are almost impossible to distinguish with a simple threshold check, visual inspection, or a peak pass test in the frequency domain. However in Figure 7.3 it can be seen that for the extreme case (efficiency drop = 2.5%), high fluctuations in temperature are observed and visual inspection might be sufficient for anomaly detection. However, the goal here is to capture the anomalous behavior as early as possible. Time series data was gathered by executing the simulation model under different anomalous conditions, by altering the efficiency values and flow constants to generate simulated faults. The nominal relative parameter for each efficiency was set to 1.0 and, under anomalous conditions; the parameters were reduced down to The effects of the decreasing efficiency on the output data were observed under persistent excitation. For example, in these simulation experiments, output data was observed by perturbing

170 155 the booster vane angle. The perturbation used for this purpose was a square wave with an amplitude 10% of the nominal value. The four plots in Figure 7.4 compare the anomaly measures obtained by using the afore-described anomaly detection approaches, SFNN, WS, PCA, and RBFNN. The nominal condition is chosen at the time epoch when all transients have decayed. For the wavelet space (WS) partitioning method, the alphabet size used in partitioning the wavelet coefficient space is 8. For the SFNN method, the alphabet size to generate the finite state machine is 4. Comparison of the two Symbolic Dynamics based partitioning techniques shows that the SFNN method performs marginally better than the WS method in predicting the onset of anomaly. For the RBFNN method exponent value = 2(Gaussian RBF) is used. Results are normalized with respect to maximum anomaly condition. For the PCA method time series data is divided into 10 subsections to form the data matrix. PCA technique also agrees with the other ones predicting the health condition of the gas turbine engine before saturation. Figure 7.4clearly shows that the results for RBFNN and PCA based methods are comparatively inferior to the symbolic dynamics based methods in terms of early detection of anomalies. This chapter compares the latest anomaly detection techniques for safety and performance enhancement of aircraft gas turbine engines. A test bed based on a generic gas turbine engine simulation model has been established to validate the various anomaly detection techniques. In this chapter, four methods for anomaly detection are compared quantitatively. Although all techniques were able to predict the saturation point correctly, the best results are obtained using phase space partitioning using SFNN and

171 156 wavelet methods. Computational complexity is an important issue for online implementation of these methods. Symbolic dynamics method based on both, wavelet space partitioning and SFNN, are compared with more traditional techniques using RBF NN and PCA. The wavelet space partitioning technique has been investigated in detail.

172 157 Fig Wavelet transforms of time series data

173 158 Fig Times series data (T c ) for various efficiency drops

174 Fig Times series data (T c ) for efficiency drop = 2.5% 159

175 160 Fig Comparison of different methods of anomaly detection

176 161 Chapter 8 Summary, Conclusions and Future Work The crux of this thesis is the development of a comprehensive control and health management strategy for human-engineered complex dynamical systems for achieving high performance and reliability over a wide range of operation. This goal is achieved by employing the results from diverse research areas such as Probabilistic Robust Control (PRC), Damage Mitigating/Life Extending Control (DMC), Discrete Event Supervisory (DES) Control, Symbolic Time Series Analysis (STSA) and Health and Usage Monitoring System (HUMS). PRC and DMC form the basis of the lower-level continuous-domain control. Upper level supervision is based on Discrete Event Supervisory (DES) control theory. Tools of STSA have been used for anomaly/fault detection. As a specific example, this thesis proposes an innovative approach for reliable control system design of high performance rotorcraft to enhance handling qualities. The proposed decision and control system has a two-tier hierarchical architecture that consists of DES control at upper level and PRC [35] and DMC [10] at the lower level. The central idea of PRC is to allow for a small risk instability (under upper level supervision) to design high performance controllers, whereas the DMC reduces the actual damage to a component in the control design scheme without any significant loss of performance. The first suite of lower-tier controllers was designed using probabilistic robust control approach. By allowing different levels of risk under different flight conditions,

177 162 the control system achieves the desired trade off between stability robustness and nominal performance [36] [35]. In the proposed scheme, a small well-defined risk of instability is acceptable because there is an upper level monitoring and control system that serves as a watch dog to detect the onset of instability. Potential instabilities are predicted sufficiently in advance and hence can be quenched by switching to a more conservative controller. Similarly, an unduly conservative controller may be switched to a more aggressive controller for better performance if there is no imminent risk of instability resulting from this action. These decisions are made by the DES controller based on the information received from the lower level. Therefore for most of the operating regime, aggressive controllers can be used to achieve high performance. In this thesis, this idea has been demonstrated for a rotorcraft but it can be easily extended to other nonlinear complex dynamical systems where high performance control action is required and a single control module may not be suitable for the entire operating range. The second suite of lower-tier controller was designed using a damage mitigating control approach. details are provided in Appendix B for the sake of completion. At the most basic level, the DMC system uses a dynamic gain-scheduled controller. Similar to the traditional gain scheduling, this controller includes parameters that vary with flight condition. However, the controller also includes a parameter (damage weight) that adjusts the level of damage mitigation in the controller. The DMC system is integrated with a Discrete Event Supervisor at the upper level of control. As damage accumulates, the effort of damage mitigation can be increased, possibly to a level such that handling qualities have to be compromised. In that case, the rotorcraft may have to be operated in a degraded mode, possibly within a restricted flight envelope.

178 163 In the second application example, the DES control theory is applied to intelligent decision and control of a twin-engine aircraft propulsion system on the Gas Turbine Simulation (GTS) test-bed. A DES control system with a two-layer hierarchical architecture is proposed and designed to coordinate the operations of a twin-engine propulsion system. Each engine is operated under a continuously varying feedback control system that maintains the specified performance under the supervision of a local (lower level) discrete-event controller for condition monitoring and life extension. The two engines were individually controlled to achieve enhanced performance and reliability, as necessary for fulfilling the mission objectives. A global (upper level) DES controller is designed for load balancing and overall health management of the propulsion system. The objectives of the proposed approach are to achieve: 1. intelligent decision and control of distributed propulsion management systems, where each of the engines has its own local DES control 2. structural damage reduction and life extension of aircraft engines without any significant loss of the system performance 3. decision making and mission planning modifications through a high-level DES coordinator 4. incorporation of optimal control laws for better mission management Traditionally, engine condition monitoring has led the way for condition-based maintenance and health management technologies because of the safety and dispatch requirements of aircraft engines [39]. Therefore, the results, conclusions, and recommendations presented in this thesis can be generalized to all types of equipment, systems,

179 164 and vehicles, i.e., to EHM (Equipment Health Management), IVHM (Integrated Vehicle Health Management) and ISHM (Integrated System Health Management). A STSA-based anomaly detection algorithm has been presented and validated on the GTS test-bed. It is assumed that dynamical systems under consideration exhibit nonlinear dynamical behavior on two time scales. Anomalies occur on a slow time scale that is several orders of magnitude larger than the fast time scale of the system dynamics. It is also assumed that the unforced dynamical system (i.e., in the absence of external stimuli) is stationary at the fast time scale and that any non-stationary behavior is observable only on the slow time scale. The time series data of stationary phase trajectories are collected to create the respective symbolic dynamics (i.e., symbol sequences) using wavelet transform [64]. The resulting state probability vector of the transition matrix is considered as the vector representation of a phase trajectory s stationary behavior. The distance between any two such vectors under the same stimulus is the measure of anomaly that the system has been subjected to. This vector representation of anomalies is more powerful than a scalar measure. The major conclusion of this research is that STSA along with the stimulus-response methodology and having a vector representation of anomaly is effective for early detection of small anomalies. As an application of the above procedure, different pattern recognition algorithms are tested on the GTS testbed to identify slow time scale anomalies for health management of aircraft gas turbine engines. The STSA-based algorithm is compared with traditional pattern recognition tools of Principal Component Analysis (PCA) and Artificial Neural Network (ANN). The work reported in this thesis, has the potential to be extended both in scope and size. A few key areas of interest are listed below.

180 Future Work The methodology proposed in this thesis can be further advanced in number of different areas Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance (C 4 ISR): The Rotorcraft Simulation and Control (RSC) test-bed developed as a part of the dissertation work currently simulates the operation of an Rotorcraft Unmanned Ariel Vehicle (RUAV). Integrating this test-bed with networked robotics laboratory will enable the future researchers to explore the complex scenarios involving C 4 ISR. The coordinated/cooperative control of autonomous vehicle formation has emerged as a topic of significant interest. Application examples are: Cooperative decision-making and control of unmanned Aerial Vehicles (UAVs); formation flying for clusters of micro-satellites; and coordination of mobile robots used for search and rescue missions. Of particular interest is the cooperative control of autonomous, Unmanned Air Vehicle (UAV) teams for missions that include: Cooperative search, acquisition, tracking, and rescue Persistent Intelligence, Surveillance, and Reconnaissance Task decomposition among heterogeneous vehicles for coordinated attack Cooperative timing of tasks Rendezvous/Join-up Simultaneous target intercept

181 166 Task sequencing Partial Observability and Asynchronous Communication: The proposed architecture does not take into account the issues of partial observability and asynchronous communication. In the examples listed above, vehicle, target, and threat information need to be exchanged, in real time, among vehicles on network links, which are likely to have limited bandwidth. These data are subject to randomly varying delays as packets are lost and retransmitted. In addition, network connectivity may be limited because of geographical constraints or electronic countermeasures. Unfortunately, emerging cooperative decision and control strategies are often designed on the unrealistic assumption of idealized information flow between the vehicles, which could lead to degraded performance or even failure to complete a cooperative task. For the control system designer, such treatment is undertaken to reduce algorithmic complexity and obtain a real-time solution. Consequently, communication constraints and their effects on the control algorithms are quantified a posteriori. While vehicle communications provide the opportunity to enhance the system performance, one must pay the associated cost. Therefore, decision and control laws must be synthesized with due regard to their associated communication needs or effects. Comprehensive Prognostics and Health Management (PHM) In this thesis, Damage Mitigating Control addresses the usage monitoring or the operational part of HUMS and the proposed STSA-based anomaly detection tools address the

182 167 the Health Monitoring aspect. These two methods have been implemented on separate test-beds. In future, these methods can be synergistically combined with other key Life Cycle Cost (LCC) variables such as system availability, maintainability, reliability and failure mode observability to build a comprehensive Prognostics and Health Management (PHM) strategy.

183 168 Appendix A Rotorcraft Simulation and Control Test-bed (RSC) A Rotorcraft Simulation and Control (RSC) test-bed was developed for real time simulation and testing of future generation control systems. The test-bed comprises of three computers. The first computer acts as the plant, it uses a non-linear simulation model (GENHEL) of the UH-60A Black Hawk helicopter. The GENHEL rotorcraft simulation code is widely used by industry and the U.S. government and is accepted as a validated engineering model for handling qualities analysis and flight control design. The code models non-linear aerodynamic effects, and includes fuselage rigid body dynamics, rotor blade flapping and lagging dynamics, rotor inflow dynamics, engine/fuel control dynamics, actuators, and a model of the existing UH-60A automatic flight controls systems (AFCS). The code has been modified to allow for the disengagement of existing AFCS channels and for the integration of the controllers presented in this paper. The second computer acts as the flight control computer. It includes both the upper level discrete event supervisory control as well as the lower level probabilistic robust controller. The RSC test-bed has also been used to evaluate a damage mitigating control system [10]. The third computer is used for visualization and runs an open-source, multi-platform, flight simulator called FlightGear. These three computers are connected by Ethernet and utilize Windows Sockets to communicate data with each other. The RSC test-bed

184 is built under Microsoft Visual C++ environment and runs on Windows XP. figure A.1 shows the architecture of RSC test-bed. 169

185 170 Fig. A.1. RSC test-bed architecture

186 171 Appendix B Damage Mitigating Control B.1 Introduction This appendix investigates the feasibility and potential benefits of damage mitigating control (DMC) for rotorcraft [10] [9] [11]. The controller is applied to regulate rotor speed and control vertical speed and pitch attitude on a military helicopter, with the objective of minimizing damage to the main rotor transmission, while maintaining Level 1 handling qualities over a range of flight conditions. An explicit model-following control scheme is used, in which the feedback portion is designed using an LQR solution. The controllers are gain scheduled with damage reduction and total airspeed as parameters. The damage is represented by the length of a crack in the main bevel pinion of the helicopter transmission. The results of nonlinear simulations over a range of airspeeds show good tracking performance with no damage mitigation, and an acceptable tradeoff of performance for damage reduction at higher levels of damage mitigation. Evaluation of handling qualities against the ADS-33E specification shows excellent performance (Level 1) with no damage mitigation [92]. As the amount of damage mitigation increases, handling qualities degrade, moving into Level 2 for large amounts of damage mitigation. The controller has been integrated with an upper level supervisory control system that selects the damage mitigation level based on the current damage level and mission scenario of the aircraft [88].

187 172 B.2 Control Scheme To demonstrate the concept of damage mitigating control, controllers for a military helicopter are developed over its entire speed envelope (hover to 150 knots). These controllers are designed to regulate the heave, pitch, and rotor speed degrees of freedom by providing collective and longitudinal cyclic inputs to the mechanical mixer and an RPM governor input to the engine throttle. A multi-input, multi-output (MIMO) design approach is used, so the controller features integrated flight and fuel controls. There are a number of objectives in the controller design. First, the controller should track vertical speed and pitch angle commands while regulating main rotor speed. This command tracking with no damage mitigation is designed to meet Level 1 handling qualities for height response and pitch bandwidth, as specified in the ADS-33E standard [92]. For higher levels of damage mitigation, the handling qualities requirements are relaxed to Level 2. Secondly, the controller should also be designed to operate effectively over a range of flight speeds (hover to 150 knots). Lastly, the controller is designed to minimize torque loads to the main transmission based on the damage weight, which acts to reduce the damage to the transmission. In order to achieve these objectives, a gain-scheduled controller is developed using an explicit model-following control scheme. B.2.1 Open Loop Plant Model For a range of flight conditions, linear state-space models for the heave, forward motion, and rotor speed degrees of freedom were extracted using perturbation methods in GENHEL, a nonlinear rotorcraft simulation model [37]. Frequency domain response

188 173 methods (using CIF ER R [86]) were applied to verify and refine the linear models. The aircraft model, represented in equation B.1 has 8 states (equation B.2), representing the heave and pitching motion of the aircraft, rotor speed, and 4 engine states, including the load demand spindle angle, fuel flow rate, gas generator speed, and power turbine temperature. The model also has 3 inputs (equation B.3): longitudinal cyclic, collective, and RPM governor. The state-space model of the aircraft varies with flight condition, as shown in equation B.1; specifically, the model depends on the total airspeed of the helicopter. G ac (s) A ac (V ) B ac (V ) C ac (V ) D ac (V ) (B.1) ( x ac = w q θ Ω x LDS W f N G T 41 ) T (B.2) ( u ac = δ lon δ col δ gov ) T (B.3) B.2.2 Controller Design and Implementation The architecture of the damage mitigating control system (shown in figure B.1) is an explicit model-following scheme [80]. Commands in vertical speed and pitch attitude are passed through separate command filters (equation B.4 to equation B.7). The command filter parameters are a function of the level of damage mitigation, or damage weight, D. The natural frequency in the pitch response and the time constant in the vertical speed response are designed to exceed Level 1 handling qualities when D = 0.

189 174 As the value of the damage weight is increased from 0 to 9, a more sluggish response is allowed in pitch and vertical speed. This effectively achieves a tradeoff in terms of flight control performance and damage mitigation, since a more sluggish maneuver is expected to result in smaller transients in torque response. Rotor speed is regulated at a constant set point, so no command filter is required. τ VD = 0.4D + 2 (B.4) F VD = ( s τ VD s+1 1 τ VD s+1 ) (B.5) ω nθ = 3 0.1D (B.6) F θ (s) = ω 2 n θ s ω nθ s + ω 2 n θ ( s 2 s 1 ) T (B.7) The feedback portion of the model-following controller is designed using an LQR full-state feedback approach. The feedback compensation is designed to regulate tracking error in vertical speed, pitch attitude, pitch rate, and rotor speed, as well as the integrated tracking errors in vertical speed, pitch, and rotor speed. The controller is designed by first extracting a modified linear model of the aircraft/engine dynamics in equation B.1 to equation B.3. The aircraft dynamics are augmented to include the integrated state variables for vertical speed, pitch, and rotor speed. The aircraft/engine dynamics are then represented by equation B.8 and equation B.9

190 175 ˆx ac = Â ac (V )ˆx ac + ˆB ac (V )u ac (B.8) ( ˆx ac = V D θ θ Ω QE VD θ Ω ) (B.9) The feedback gains are designed using linear quadratic regulator (LQR) theory to minimize the cost function defined in equation B.10 that includes the state weighting matrix Q (equation B.11) and control weighting matrix R (equation B.12). Note that the state weighting matrix includes a performance penalty for the engine torque, which is a function of damage weight. Thus, as damage weight is increased, the feedback controller is designed to minimize fluctuations in engine torque, which is directly related to damage on the engines, transmission, and drive system of the rotorcraft. The resulting LQR gain matrices are a function of both the airspeed of the helicopter (V), to account for the variation of the plant model with airspeed, and damage weight (D) to account for variation in the cost index with damage mitigation. t J = [x T ac Qx ac + ut ac Ru ac ]dτ (B.10) 0 Q = diag[1, 180 π, 180 π, , D, 0.5, 90 π, 50 π ] (B.11) R = diag[1, 360 π, ] (B.12) An implementation issue that arises with this approach is that the engine torque is not zero in equilibrium flight. A high pass filter is applied to the measured torque, as

191 176 shown in equation B.13. The objective of the damage mitigating controller is to reduce fluctuations in engine torque; therefore, the low frequency steady-state portion of the engine torque signal is of no interest. Q E(filter) = s+10 s Q E (B.13) B.3 Damage Model In order to intelligently adjust the level of damage mitigation, the control system must incorporate either a direct measurement or a model-based prediction of the damage. Furthermore, in order to evaluate the effectiveness of the controller, the simulation should include an analytical model of some representative damage. In this study, the damage is characterized by the length of a crack (a) in the main bevel pinion of the helicopter transmission, shown in schematic form in figures B.2 and B.3. B.3.1 Theoretical Stress and Crack Growth Model To calculate the crack growth in the main bevel pinion, it is first necessary to calculate the stress on one representative tooth, shown in figure B.4. The stress is found from the engine torque using equations B.14 and B.15 from reference [78], which are valid only if the tooth is in contact with a corresponding tooth on the main bevel gear (i.e.,0 o θ p o ). Equations B.16 and B.17 are empirically determined formulas to determine the distances R c and h illustrated in Fig. 4 as functions of θ p, the angular position of the pinion. Since the stress is greatest near the apex of the main bevel pinion as shown in the stress profile of figure B.5, the value at this location is used as a limiting

192 case. Q p = Q E 2 Ω ω p 177 (B.14) S = Q p R c F (6h t 2 tanφ ) (B.15) t R c = θ 2 p θ p (B.16) h = θ 2 p θ p (B.17) The current stress value is then compared to previous stress values to determine local maximum and minimum stresses. These local maximum and minimum stresses are then used to determine the crack opening stress using Newman s formulas [56], which are given in equation B.18 through equation B.24. R = S max S min (B.18) σ 0 = 1 2 (σ y + σ ult ) (B.19) A 0 = ( α α 2 )(cos[ πs max G 2σ 0 ]) 1 α (B.20) A 1 = ( α) S max G σ 0 (B.21) A 3 = 2A 0 + A 1 1 (B.22)

193 178 A 2 = 1 A 0 A 1 A 3 (B.23) S o = S max (A 0 + A 1 R + A 2 R 2 + A 3 R 3 ) (B.24) If the current stress is greater than the crack opening stress, the crack length increases at a rate determined by equation B.25 through equation B.28 below. Otherwise, the crack length remains at its current value. S inc = S max max(s o, S min ) (B.25) K eff = S inc G πa (B.26) da dn = A( K eff )n (B.27) da dt = da Ω dn 2π (B.28) B.3.2 Implementation in GENHEL Certain modifications to the gearbox module of the nonlinear GENHEL model are necessary in order to implement the stress and crack growth models. Within the gearbox module, constants such as the material properties of the pinion, as well as the stress ratio R, are hard-coded. The stress ratio remains constant since both the maximum and minimum stresses vary with the engine torque. An additional loop is

194 179 also included in the gearbox module, in which each of the standard GENHEL timesteps of 0.01 seconds is subdivided into 1000 smaller steps of seconds; this loop is necessary due to the high rotational speed of the main bevel pinion, which is on the order of 6000 revolutions per minute. The angular speed of the pinion is calculated outside of this loop since it varies only with rotor speed, which is updated once per GENHEL timestep. Within the loop, the current maximum stress value is calculated from the actual value of engine torque, using equation B.29, and the angular position of the main bevel pinion is found from the angular speed according to equation B.30. If the representative tooth of the pinion is in contact (0 o θ p o ), the current stress (equation B.14 through equation B.17) and crack opening stress (equation B.20 through equation B.24) values are calculated; however, if the tooth completes a cycle (θ p > 360 o ), the current maximum and minimum stress values are reset to zero. If the current stress is greater than the opening stress, the crack growth rate is found using equation B.25 through equation B.28, and the additional crack length (equation B.31) is added to the value of crack length stored in memory. S max Q E (B.29) θ p = θ p + (ω p t) (B.30) a = a + ( da dt t) (B.31)

195 180 B.4 Results and Discussion The GENHEL simulation code was used to perform a number of maneuvers using the damage mitigating control system. The controller is evaluated by examining its tracking performance, as well as the time history of damage on the aircraft. In addition, the impact of the controller on the handling qualities of the helicopter is measured by using the ADS-33E handling qualities performance specification [92]. B.4.1 Outer Loop Controller To produce the simulation results presented below, it is necessary to include an outer loop controller so that aircraft trajectory commands can be translated into the vertical speed and pitch attitude commands required by the damage mitigating control system. The total airspeed (V ) command is translated into the pitch attitude command, and a height above ground level (h) command is translated into the vertical speed command. The conversion between the total airspeed and pitch attitude commands is accomplished through the use of a proportional-integral (P I) controller, given in equation B.32, which was designed using classical control techniques. The conversion from the height above ground level command to the vertical speed command is performed by a proportional-derivative (P D) controller, given in equation B.33. This controller was also designed using classical control techniques; however, due to the variation of the vertical speed command filter with damage weight, the proportional term of the controller also varies with damage weight, as shown in equation B.34. The outer loop controller

196 can be interpreted as a model of a human pilot for a piloted rotorcraft or an outer loop autopilot and guidance system for a UAV. 181 t θ cmd = K (V P (V ) cmd V ) + K (V I(V ) cmd V )dτ (B.32) o V Dcmd = K P (h) (h cmd h) + K D(h) d dt (h cmd h) (B.33) K P (h) = 1 τ VD (B.34) B.4.2 Time History Results The first set of time history results consists of flight at 1000 feet over level terrain with a step command to 2000 feet above ground after 10 seconds. In addition, the total airspeed of the helicopter is commanded to decrease by 10 knots as it climbs. Figures B.6 and B.7 shows this maneuver for an initial total airspeed of 80 knots. Results for damage weights of 0, 3, 6, and 9 (on a scale from 0 to 9) are shown; the dashed line represents the commanded value for each variable. It can be seen from the figure that, as damage weight increases, performance (in the form of tracking the airspeed and height above ground commands) is slightly degraded in favor of a reduction in damage. These two results are linked through the engine torque; the peak value of engine torque is reduced as damage weight increases, which serves to reduce both the tracking performance and the damage rate. Also, it

197 182 should be noted that, although the height above ground level command is a step command, the commanded value that comes out of the simulation is not a step; this is due to the fact that the control system includes limits on both climb and descent velocities. These limits on vertical speed are gain-scheduled with velocity in order to prevent the helicopter from exceeding its continuous engine torque limit. It is also worth noting that the damage weight also effectively reduces the negative peaks in the engine torque response (which occurs when the aircraft levels off at 2000 feet). The controller with D = 0 actually exhibits lower torque in the transient peak at that time, but due to the nonlinear behavior of the crack growth, the net damage accumulation is higher. The second set of time history results shows a terrain-following maneuver for damage weights of 0, 3, 6, and 9. In this maneuver, the height of the ground varies, and is provided to the helicopter simulation. Figure B.8 and B.9 shows that the simple outer loop controllers can track a height above ground level command while following the ground contours, provided that the slope of the ground is not overly steep. Thus the aircraft does not respond to the building at approximately 110 seconds into the maneuver or the radio tower at approximately 390 seconds into the maneuver. A forward-looking terrain-following controller has not been implemented at his time and is left for future work. In addition, the helicopter simultaneously tracks a total airspeed command, and regulates the main rotor speed. As in the step command results, tracking performance can be traded for damage mitigation. The final time history result, shown in figures B.10 and B.11, is another terrainfollowing maneuver. This maneuver differs from the medium-length maneuver in three ways. First, the terrain for this run is not predefined; rather, the maneuver is flown

198 183 over terrain generated by FlightGear [1], an open-source flight simulation program that interface with GENHEL. Second, the lateral axes of the helicopter are not controlled by the standard UH-60A stability augmentation system (SAS); instead, a probabilistic robust controller is used [35] [87]. Third, the damage weight is varied during this maneuver; this is accomplished through the use of a higher-level discrete event supervisor (DES) [88], which adjusts the damage weight and airspeed and height above ground level commands based on a number of factors, including damage level, current flight condition, and whether the aircraft is in friendly or unfriendly territory. For example, in this maneuver, the helicopter is in unfriendly territory from approximately 240 seconds to 340 seconds, and from approximately 420 seconds to 580 seconds. The DES adjusts the damage weight from its friendly territory level of 8 to a lower level of 5 (or less, if the aircraft is climbing sharply) to allow the helicopter greater maneuverability, in case it encounters hostile fire. In addition, the DES increases the airspeed command from 75 knots to 90 knots, and reduces the height above ground level command from 1000 feet to a value that varies between 75 feet and 300 feet. Once again, the damage mitigating controller tracks the airspeed and height above ground commands well and regulates the main rotor speed. In addition, the controller trades performance for damage mitigation, as evidenced in the plots of damage weight and crack length in the above figures. B.4.3 Handling Qualities Results While the helicopter possesses acceptable command tracking performance, even for high damage weights, it is important to understand how the damage mitigating controller affects the aircraft s handling qualities. In order to quantify these effects, the

199 184 controller was evaluated against the U.S. Army s ADS-33E handling qualities specification [92]; in particular, the controller was tested using the height response (Section ) and pitch bandwidth (Section ) portions of the specification. Figure B.12 shows the height response characteristics of the controller at 40 knots for damage weights of 0, 3, 6, and 9. For damage weights from 0 to 6, the aircraft has Level 1, or excellent, handling qualities. For damage weights above 6, the aircraft has Level 2, or adequate, handling qualities. These results match closely with the vertical speed time constant in equation B.4, which varies with damage weight. Handling qualities results for pitch bandwidth also match closely with the natural frequency of the pitch attitude command filter in equation B.6, which also varies with damage weight. Handling qualities requirements vary with mission tasks. For example, the height response requirement shown below is for aggressive combat maneuvers, and it might be acceptable to allow a more sluggish response for different flight regimes in order to reduced damage. It may also be acceptable to allow a degraded mode of operation if the aircraft has accumulated a large amount of damage. B.5 Conclusions In this appendix, a control system has been developed with the goal of mitigating the amount of damage sustained by a rotorcraft while regulating rotor speed and tracking commands in the longitudinal and heave axes. The control system is an explicit model-following controller, whose feedback portion consists of gain-scheduled controllers designed using LQR theory. The controllers are gain scheduled using two parameters: total airspeed and amount of damage mitigation. Flight simulation results have been

200 185 produced by using a nonlinear simulation model; these results show that a tradeoff exists between damage mitigation and flight performance, which can be managed by adjusting the damage weight. Handling qualities have been evaluated using the ADS-33E specification. A tradeoff also exists between damage mitigation and handling qualities; however, handling qualities are still rated as accurate for the high levels of damage mitigation. In future work, more extensive simulations, including pilot-in-the-loop studies, will be performed to evaluate the control system.

201 186 Fig. B.1. Model-Following Control System Schematic

202 187 Fig. B.2. Typical Twin-Engine Helicopter Transmission Schematic

203 188 Fig. B.3. Main Bevel Pinion/Main Bevel Gear Schematic

204 189 Fig. B.4. Main Bevel Pinion Tooth Profile

205 190 Fig. B.5. Main Bevel Pinion Tooth Stress Profile