ABSTRACT. LIAO, MANG. Optimization Algorithms for Cyber-Physical Security of Wide-Area Power Systems. (Under the direction of Aranya Chakrabortty).

Transcription

1 ABSTRACT LIAO, MANG. Optimization Algorithms for Cyber-Physical Security of Wide-Area Power Systems. (Under the direction of Aranya Chakrabortty). This dissertation work focuses on optimization algorithms for cyber-physical security in wide-area large power system network. In Chapter 2 of this report we first retrospect the power system oscillation model. Our goal is using the measurements from Phasor Measurement Units (PMUs), such as the voltage and current magnitudes and phasors, to estimate the oscillation modes: damping and frequency. To avoid the prior known disturbance input information, we consider the existing well-known modal estimation algorithm, Prony analysis method. The real-time centralized problem is formulated as a global consensus problem, and solved using Alternating Direction Method of Multipliers (ADMM). The power system is divided into multiple non-overlapping areas, each equipped with a local estimator. These local estimators use local sensor measurements of the data from PMUs to carry out a local regression algorithm for generating a local estimate of the characteristic polynomial of the system, and, thereafter, communicate this estimate to a central supervisor. The supervisor computes the average or consensus of all estimates, and broadcasts this consensus variable back to each local estimator to be used in the next round of regression. If all estimates are accurate, then ADMM is guaranteed to converge asymptotically to the true optimal solution of the characteristic polynomial, following which the central supervisor can solve for its roots to obtain the desired eigenvalues. Furthermore, we consider the case of noisy measurements. In the case using the standard ADMM algorithm the estimates cannot be obtained correctly. We fomulate the optimal problem to minimize the influence of the noise into the performance. An efficient approach combining the total least squal (TLS) method and ADMM is provided for the estimation with noisy measurements. However, due to the high cost associated with dedicated fiber-optic communication networks, the communication between the local phasor data concentrator (PDC)s and the central PDC is most likely to happen over an open wide-area communication network. These networks are quite vulnerable to hacking. If one or more of the local estimates are manipulated by attackers, then the resulting consensus

2 variable will be inaccurate using the standared ADMM in Chapter 2, which, in turn, will contaminate the accuracy of every local estimate. Even a small amount of bias at a single iteration can destabilize the entire estimation process. To combat this, in Chapter 3 we first develop an algorithm to show how ADMM can be used by the central supervisor to catch the identities of malicious estimators by simply tracking the quality of every incoming estimate. This algorithm, however, can become computationally expensive if the network size is large. Therefore, we propose another algorithm where the central supervisor, instead of computing the average, employs a Round-Robin technique to generate the consensus variable, and show that by tracking the evolution of only this consensus variable it is possible to identify the malicious estimators. Both large and covert attacks are considered. Results are illustrated using simulations of a IEEE 68-bus power system model. If the measurements are noisy it is difficult to detect the identities of the attacked estimator(s) because the injected false bias may remain hidden in the noise. In Chapter 4, we consider to identify the data manipulators in power system estimation loops with noisy measurement. To combat the phenomenon of false bias hidden in the noise, the central supervisor still uses Round-Robin technique with an appropriate order to generate the consensus variable. We show that by tracking the evolution of this consensus variable it is possible to identify which estimators are malicious. We also show that by choosing the correct order based on the TLS estimates it is possible to amplify the attack signatures, thereby reducing false alarms. Numerical results using PMU measurements from IEEE 68-bus power system model illustrate the effectiveness and efficiency of the proposed approaches Furthermore, we consider the identification in a closed-loop wide-area power system. In Chapter 5 we present an algorithm that can identify potential data-manipulators in distributed control loops of large power system models. The control loop is considered to be implemented through a cloud computing layer, which is typical in wide-area applications. Virtual computing machines (VMs) in the cloud receive state measurements from the generators, communicate these measurements to their neighboring VMs, and collaboratively compute the control signals that are transmitted back to the respective generators. If any of these VMs is faulted or compromised by a malicious attacker, resulting in an incorrect value of the control input, then the entire closed-loop power system can be destabilized. We present a periodic

3 Round-Robbin based algorithm by which the identities of these compromised VMs can be revealed in real-time. The algorithm is based on sparse feedback gains with Round-Robin approach and the values of the states to identify the malicious VMs covertly. Results are illustrated using simulations of an IEEE 68-bus power system model.

4 Copyright 2018 by Mang Liao All Rights Reserved

5 Optimization Algorithms for Cyber-Physical Security of Wide-Area Power Systems by Mang Liao A dissertation submitted to the Graduate Faculty of North Carolina State University in partial fulfillment of the requirements for the Degree of Doctor of Philosophy Electrical Engineering Raleigh, North Carolina 2018 APPROVED BY: Alexandra Duel-Hallen Ning Lu Fen Wu Aranya Chakrabortty Chair of Advisory Committee

6 DEDICATION To my parents. ii

7 BIOGRAPHY Mang Liao was born in Shenyang City, Liaoning Province, China. She received her BE degree in Automatic Control Department of China Civil Aviation University, China in 2009 and PhD degree in Electrical Information Engineering Department of Beihang University, China in Inspired to solve the challenge for the power system to improve the people s lives, she continued to pursue her second Ph. D degree in the Department of Electrical and Computer Engineering of North Carolina State University, Raleigh, NC in During her second Ph. D period, she worked as a research assistant on National Science Foundation focusing on the distributed cyber-physical optimizations of wide-area monitoring and control. Mang Liao will be graduating with her Ph. D in Aug 2018 with the hopes of continuing to build the future power system control and stability through both theoretical and practical techniques. Especially, her research interests are power system dynamics and monitoring, especially in the development and implementation of cyber-physical security algorithms using the Synchrophasor technology. iii

8 ACKNOWLEDGEMENTS First and foremost, I would like to thank Dr. Aranya Chakrabortty for imparting his knowledge of power system, control theory, and optimization skills on me as well as allowing me venture off into unfamiliar territory and multiple research projects. I cannot thank him enough for his trust and willingness to let me work on the majority of my Ph. D in different state. He has helped me to improve my research skills through involving into several research projects and publishing our own research contributions. Thank you. To my committee members, Drs. Alexandra Duel-Hallen, Ning Lu, and Fen Wu, your advice, encouragement, and support have been an immense help over the years. I would also like to thank Dr. Di Shi from GEIRI North America Institute, San Jose, CA for mentoring me, sharing your wealth of knowledge of power system with me. Working in GEIRI is my invaluable experience. I would not have succeeded without the love and moral support from my parents. My parents, you will never know how much I appreciate all the times you were there for me when I felt run down and being right beside me to celebrate the joyous moments. I appreciate your constant love and support throughout my long educational endeavors. To the rest of my family, thank you for various words of wisdom over the years, helpfulness, delicious meals, and wonderful times when we are able to get together and chat up. To my friends, your unwavering patience and kindness has meant the world to me. Thank you to Seyedbehzad Nabavi, Abhishek Jain, Jianhua Zhang, Nan Xue, Yao Meng, Sayak Mukherjee, and Haoqi Ni for joyfully co-working on our research projects and papers. Jianhua, I do remember our happy time on all kinds of shows at our school theaters. Thank you to my lab mates and the graduate students I have gotten to know throughout my time at NCSU, especially in the FREEDM center. The FREEDM party times, seminars, and random conversations will be treasured memories. iv

9 TABLE OF CONTENTS LIST OF TABLES LIST OF FIGURES vii viii Chapter 1 Introduction Contributions Future Tasks Chapter 2 Wide-Area Oscillation Estimation for Power System using Optimization Algorithms Power System Oscillation Model Distributed Prony Algorithm with ADMM Prony Algorithm Real-Time Distributed Prony Algorithm using ADMM Noisy Measurements Case Conclusion Chapter 3 Identifying Data-Manipulators in Power System Estimation Loops Problem Formulation for Attack Identification Data Manipulations with General Biases Detection of attacks S-ADMM for Identifying Malicious PDCs Round-Robbin ADMM for Detecting the Malicious Users Random Order of RR-ADMM Data Manipulations with Small Biases S-ADMM for Detecting Malicious Users with Small Biases RR-ADMM for Detecting Malicious Users with Small Biases Comparison between S-ADMM and RR-ADMM Simulation Results Conclusion Chapter 4 Identifying Data-Manipulators with Noisy Measurements Problem Formulation for Attack Identification Proposed Round-Robbin ADMM RR-ADMM with Local TLS Estimates Simulation results Conclusions Chapter 5 Identifying Data Manipulators in Distributed Wide-Area Control Loops of Power Systems Problem Formulation for Attack Identification Power System Model v

10 5.1.2 Data updated by the individual PDCs When the VMs at one or multiple areas are faulted or attacked Identify the faulted or attacked VMs Identify the attacked VMs covertly RR controller in fault-free case RR Sparse controller in fault case Simulation Conclusion Chapter 6 Concluding Remarks and Future Research Conclusion Future Research Directions References Appendix Appendix A Flow Charts A.1 Expressions of f ( ) functions in (3.16): vi

11 LIST OF TABLES Table 1.1 Overview of FDIA researches Table 3.1 Comparison of S-ADMM and RR-ADMM for detection of attacks with large biases 33 Table 3.2 S-ADMM and RR-ADMM for detection with multiple small biases Table 5.1 The cost J using sparse LQR with differnt values of p vii

12 LIST OF FIGURES Figure 1.1 A schematic block diagram of a power network, a SCADA system, and a control center Figure 2.1 Distributed architecture for a 4-area power system network Figure 2.2 The trajectories of the data from each local PDC using S-ADMM Figure 2.3 The trajectories of the data from each local PDC using S-ADMM Figure 2.4 Values of the real parts of the four estimated inter-area modes using S-ADMM and TLS-ADMM with noisy measurements, respectively. True values of σ i are shown by dashed lines Figure 3.1 The trajectories of the data from each local PDC using S-ADMM with biases Figure 3.2 The timing diagram of the communication between the central and local PDCs Figure 3.3 Evolution of the norms of the local estimates, and the average z k before and after detection using S-ADMM. For convenience, we only show the first element of z k.. 33 Figure 3.4 Evolution of the average z k before and after detection using RR-ADMM. For convenience, we only show the first element of z k Figure 3.5 Evolution of the norms of the local estimates, and the average z k av before and after detection of attack with small biases using S-ADMM. For convenience, we only show the first element of z k av Figure 3.6 The response of z k before and after detection of attack with small biases using RR-ADMM Figure 3.7 IEEE 68-bus model Figure 3.8 Architecture for a 5-area power system network with 2 malicious PDCs Figure 3.9 Evolution of āa k av, j when S-ADMM is run under attacks Figure 3.10 Values of the real parts of the four inter-area modes before and after detection using S-ADMM. True values of σ i are shown by dashed lines Figure 3.11 Evolutions of z k av and the first four elements of z k av when S-ADMM is run under attacks Figure 3.12 Evolutions of z k when RR-ADMM and S-ADMM are run under attacks Figure 3.13 Values of the real parts of the four inter-area modes before and after detection with RR-ADMM. True values of σ i are shown by dashed lines Figure 3.14 z k rr with random order of RR-ADMM Figure 3.15 The response of āa k av, j with sparse biases and non-sparse biases using S-ADMM. 52 Figure 3.16 The response of z k rr with sparse biases and non-sparse biases using RR-ADMM. 53 Figure 3.17 The response of z k av with sparse biases and non-sparse biases using S-ADMM.. 53 Figure 3.18 Evolutions of āa k av, j with multiple biases and different values of ρ using S-ADMM. 54 Figure 3.19 Values of the real parts of the four estimated inter-area modes before and after detection using S-ADMM with small biases. True values of σ i are shown by dashed lines Figure 4.1 Behavior of z k pre and post detection of attack using RR-ADMM Figure 4.2 Behavior of z k rr with successive order viii

13 Figure 4.3 Figure 4.4 Evolutions of average values of z when TLS-ADMM and RR-ADMM are run under attacks Values of the real parts of the four inter-area modes before and after detection with RR-ADMM Figure 5.1 Timing diagram for the communication protocol among the firewall, PDCs, and VMs. 74 Figure 5.2 The communication between two PDCs and two VMs with and without attack Figure 5.3 The trajectories of the first element of x with different values of p Figure 5.4 The trajectories of the first element of x with different values of p Figure 5.5 The communication between two PDCs and two VMs with and without attack Figure 5.6 The communication between two areas after removing the attacked VM Figure 5.7 Wide-area control using a cloud-in-the loop architecture with IEEE 68-bus model.. 87 Figure 5.8 The trajectories of the first elements of inputs from the detector and VMs with p = 1 88 Figure 5.9 The trajectories of the first elements of inputs from the detector and VMs with p = Figure 5.10 The trajectories of the first elements of inputs from the detector and VMs with p = 3 89 Figure 5.11 The trajectories of the first elements of inputs from the detector and VMs with p = 4 90 Figure 6.1 Timeline of work ix

14 Chapter 1 Introduction Reliable electricity supply via the modern power grid is fundamentally supported by the underlying cyber systems. Power system security involves two main aspects: physical security and cyber security. Physical security represents the ability of a power system to maintain a normal working state in the presence of severe disturbances. Cyber security refers to the security of the communication networks and computer systems which support the power system operation. Weaknesses in cyber security can threaten the physical security of the power systems due to the strong correlation of the physical and cyber systems [1 6]. The wide-area measurement systems (WAMS) technology using Phasor Measurement Units (PMUs) has been regarded as the key to guarantee stability, reliability, state estimation, control, and protection of next-generation power systems [7]. However, with the exponentially increasing number of PMUs deployed in the North American grid, and the resulting explosion in data volume, the design and deployment of an efficient wide-area communication and computing infrastructure, especially from the point of view of resilience against a wide varieties of nefarious cyber-physical attacks, is evolving as one of the greatest challenges to the power system and IT communities. Power system vulnerabilities resulting in the cost of power outages are estimated about 100bn per year in the United States (US) [8]. In August 2003, a cascading outage of generation and transmission facilities in North America caused a tremendous blackout in Northeastern and Midwestern USA, which made around 6-billion dollar lost. 1

15 In the same year, major blackouts also happened in Europe, such as Denmark, Sweden, and Italy [9]. In 2008, the public transport system in Poland was hacked remotely, while in 2010 the Stuxnet worm attacked Iran s Natanz nuclear fuel-enrichment facility. On 23 December 2015, a synchronized and coordinated cyber-attack compromised three Ukrainian regional electric power distribution companies, resulting in power outages affecting approximately customers for several hours [10]. With several thousands of networked PMUs being scheduled to be installed in the United States by 2020, exchange of Synchrophasor data between balancing authorities for any type of wide-area control will involve several thousands of Terabytes of data flow in real-time per event, thereby opening up a wide spectrum of opportunities for adversaries to induce data manipulation attacks [11 13], denial-of-service attacks [14], GPS spoofing [15], attacks on transmission assets [16], and so on. The challenge is even more aggravated by the gradual transition of WAMS from centralized to distributed in order to facilitate the speed of data processing [19 21]. Unlocking the tremendous potential of the smart grid strongly depends on the security of this system. From the advent of the smart grid concept, security has always been a primary concern. In the 2009 White House Cyberspace Policy Review, the US federal government was asked to ensure that security standards are developed and adopted to avoid creating unexpected opportunists to penetrate these systems or conduct large-scale attacks [17]. The US National Institute of Standards and Technology (NIST) has provided guidelines for developers and policy makers, covering cyber security requirements of the smart grid that should be included from the beginning of the development process [18]. Fig. 1 shows a schematic block diagram of a modern power network control sytstem. In the transmission level of power network models, hundreds of buses are spread out over a large geographic area (a region in a country, for example). To monitor and control the behavior of such large-scale systems, Supervisory Control and Data Acquisition (SCADA) systems are used to transmit measurements, status information, and circuit-breaker signals to and from Remote Terminal Units (RTUs) that are connected to substations [22 24]. State estimation plays a very important role in the control center. It calculates the state of every bus, and filters raw measurements by processing the data provided by the SCADA system. An attack on a SCADA system typically affects the power system state estimation process. The incoming data is often fed to a state estimator which provides Energy Management Systems (EMS) and the human 2

16 RTUs Power Network RTUs WAMS/WAMC PMUs SCADA Master AGC Optimal Power Flow EMS x State Estimator SCADA Master Control center Figure 1.1: A schematic block diagram of a power network, a SCADA system, and a control center. operator in the control center with hopefully accurate information at all times. The state estimation results reflect the real time power grid operation state and are essential for operators to make decisions in order to maintain security and stability of the system. Many power system applications, such as economic dispatch (ED), contingency analysis, and so forth, rely on the results of state estimation [25, 39]. Attack can degrade the performance of the state estimates [26], and even worse destroy the security and stability of the system [27] and have catastrophic consequences [28]. Therefore, detecting the identities of the data manipulators has become a significant and inevitable problem in power systems. The false data injection attack (FDIA), first proposed by Liu et al. [11, 12], is a cyber-attack in which power system state estimation outputs are corrupted by injecting false data into meter measurements in a carefully coordinated fashion. The defining feature of a successful FDIA is that the state estimation residual falls below a hypothesis test threshold despite the presence of corrupted measurements, the attack thereby evading detection. Several recent papers such as [4 6, 29 31, 39] have studied how false-data injection attacks may be deceptively injected into a power grid using its state estimation loops, and through data in relays and intelligent electronic devices (IEDs) [32]. Others have proposed estimation- 3

17 based mitigation strategies to secure the grid against many of these attacks [33 36]. In general, research on FDIAs mainly focuses on the following three aspects: theoretical research, application research, and defensive research [37]. In theoretical research, the challenge is the construction of injected vectors capable of evading detection by the control center under different situations [38], for example, when the attacker has limited access to meters, incomplete information, false topology, or an AC power flow model is used; the attacker then injects bad data into meters. In application research, the purpose is to analyze the impacts of FDIAs on power system operation, mainly on EMS and market management systems (MMS), such as economic dispatch and congestion managements. In defensive research, the aim is proposing defense strategies from the viewpoint of the system operator. In [37] Liang et al. summarized the existing works that can be categorized as shown in Table 1.1. The fundamental approach behind many of these designs is based on the so-called idea of Byzantine consensus [72 74], a fairly popular topic in distributed computing, where the goal is to drive an optimization or optimal control problem to a near-optimal solution despite the presence of a malicious agent. In practice, however, this approach is not acceptable to most WAMS operators as they are far more interested in finding out the identity of a malicious agent if it exists in the system, disconnect it from the estimation or control loop, and continue operation using the remaining non-malicious agents rather than settling for a solution that keeps the attacker unidentified in the loop. This basic question of how to catch malicious agents in distributed wide-area monitoring applications is still an open challenge in the WAMS literature. Motivated by these challenges, the primary focus of this research is to develop distributed optimization algorithms for cyber-physical security for the communication network on the real-time wide-area monitoring of large power systems. The research group led by Dr. Chakrabortty is currently investigating the theory behind distributed estimation and control, and recently proposed development a set of algorithms that can detect the identities of malicious data-manipulators in distributed optimization loops for estimating oscillation modes in large power system models. In this report, we first utilize alternating direction method of multipliers (ADMM)-based Prony algorithm for wide-area oscillation estimation in power system. Secondly considering the attack during the communication between the local and central phasor data centrators (PDC)s, we develop a set of algorithms for detecting the identities of the data 4

18 Table 1.1: Overview of FDIA researches FDIA research Categories References Construct a valid [12, 28, 39 42] FDIA under certain constraints Theoretical researches on constrcting Construct a valid [43 46] a valid FDIA FDIA with incom- plete information of matrix Construct a valid [47 49] FDIA with topology being falsified Construct a valid [50 52] FDIA under AC power flow model Application researches on the impacts of FDIAs Defense strategies against FDIAs Economic attack [48, 49, 53 55] Load redistribution [56, 57] attack Energy deceiving [58] attack Protect a set of basic [12, 39, 59, 60] measurements PMU-based protection [61 64] Other ways of [65 71] defending against FDIAs manipulators. We consider the identification in these cases of both general false-data injection and covert bias. Also the detection of malicious users with noisy measurements is analyzed. We start our research with combining the Prony analysis method and ADMM optimization algorithm for wide-area oscillation estimation in the large power system network. In Chapter 2 we first introduce the power system oscillation model. Our goal is using the measurements obtained from PMU, such as the voltage and current magnitudes and phasors, to estimate the oscillation modes: damping and frequency. To avoid the prior known disturbance input information, we consider the existing well-known modal estimation algorithm, Prony analysis method. It can be recast as a real-time distributed optimization problem by formulating it as a global consensus problem, and solving it using ADMM algorithm [80]. 5

19 Thus, at any iteration, the local estimators receive PMU measurements from within their own respective areas, run a local consensus algorithm, and communicate their estimates to a central estimator. The central estimator averages all estimates, and broadcasts the average back to each local estimator as the consensus variable for their next iteration. It was shown that this average value converges to the global solution as the number of iterations tends to infinity. Besides, we consider the case of noisy measurements. Then the optimization problem is changed to minimize the influence of the noise into the performance. In this case, using the standard ADMM algorithm the oscillation modes cannot be estimated correctly. Thus we develop the traditional ADMM and combine it into total least squal (TLS) method, and then the correct estimates are obtained with noisy measurements. However, due to the high cost associated with dedicated fiber-optic communication networks, the communication between the local PDCs and the central PDC is most likely to happen over an open wide-area communication network. These networks are quite vulnerable to hacking. The key question, therefore, is how to catch malicious agents in distributed wide-area monitoring applications. In Chapter 3, we address this question in the context of identifying malicious data-manipulators in distributed optimization loops for wide-area oscillation monitoring. The specific application of our interest is the estimation of electromechanical oscillation modes or eigenvalues from streaming PMU data following a small-signal disturbance in the grid [75, 76]. If one or more of the local estimates are manipulated by attackers, then the resulting consensus variable will be inaccurate using ADMM mentioned in Chapter 2, which, in turn, will contaminate the accuracy of every local estimate. Even a small amount of bias at a single iteration can destabilize the entire estimation process. To combat this, we first develop an algorithm to show how ADMM can be used by the central supervisor to catch the identities of malicious estimators by simply tracking the quality of every incoming estimate. This algorithm, however, can become computationally expensive if the network size is large. Therefore, we propose another algorithm where the central supervisor, instead of computing the average, employs a Round-Robin technique to generate the consensus variable, and show that by tracking the evolution of only this consensus variable it is possible to identify the malicious estimators. Both large and covert attacks are considered. As mentioned in Chapter 2, when measurements are corrupted by noise, the standard least-squares 6

20 based regression no longer yields accurate results as least square (LS) is inherently a biased estimator. Instead a more robust and noise-tolerant version of LS, namely, total least squares (TLS) [81], need to be used in such scenarios. We, therefore, first develop a distributed version of TLS using ADMM. In Chapter 4 thereafter we show that if some of the local estimators are compromised then attackers at these estimators may send corrupted values of their TLS estimates, and destabilize the estimation loop completely within a few iterations. Since the measurements are noisy it is difficult to detect the identities of the attacked estimator(s) since the injected false bias may remain hidden in the noise. To combat this, in Chapter 4 we propose an algorithm where the central supervisor, instead of computing the average as in the usual ADMM, employs a deterministic ordered Round- Robin technique to generate the consensus variable. We show that by tracking the evolution of this consensus variable it is possible to identify which estimators are malicious. We also show that by choosing the correct order based on the TLS estimates it is possible to amplify the attack signatures, thereby reducing false alarms. In Chapter 5 we address this problem in the context of identifying malicious data-manipulators in wide-area control loops of power system. The fundamental set-up for the controller is based on minimizing a LQR objective function. The power system is divided into multiple non-overlapping areas, each equipped with a PDC. These PDCs collect the measurements of voltage, phase angle, and frequency from PMUs and send the data to virtual computing agents, referred to as virtual machines (VMs), in a cloud computing network. The VMs can communicate to each other. After receiving the data from all the other VMs, each VM calculates the optimal control input by minimizing the LQR objective function and sends them back to the respective generators for actuation. However, if one or more of the VMs are faulted or manipulated by attackers, then the resulting control inputs will be inaccurate, which, in turn, will contaminate the accuracy of every PDC. To combat this, a detector is designed to collect the state information from PDCs and VMs. It requires that the individual VMs send all the states to the detector. Compared to the data from PDCs, the detector can identify the attacked VMs. However, if the attacker is smart, he or she can perceive the detection period due to the different actions of VMs, and then sends the correct data to puzzle the detector and conceal his or her identity. We provide a Round-Robin (RR) technique based on sparse feedback gains to show how the detector catches the malicious VMs by 7

21 simply tracking the magnitude of every control input covertly. In the normal case, the system uses LQR controller for minimizing the cost. When the attacker accesses the network, the firewall [96] or intrusion detection system (IDS) [97] can detect the intruder and broadcast the alarm. The system changes to the sparse controller with RR technique. At this time the attacker just accesses the network, not intrudes the VMs. Each VM keeps the same calculation of the inputs after the alarm. So the attacker believes that the system always uses the RR sparse controller to calculate the inputs and cannot perceive the detection period. We illustrate effectiveness of the attack localization algorithms using simulation results on an IEEE 68-bus power system model. 1.1 Contributions 1. Devolop Standard ADMM and Round Robin ADMM for detecting the identities of the data manipulators with general biases in wide-area power system estimation loops. 2. Devolop Standard ADMM and Round Robin ADMM for detecting the identities of the data manipulators with covert biases in wide-area power system estimation loops. 3. Devolop the algorithm which is combined TLS and Round Robin ADMM for detecting the identities of the data manipulators with noisy measurements in wide-area power system estimation loops. 4. Design the sparse controller based on Round-Robin technique for identifying the attacked or faulted virtual machines in wide-area power system control loops. 1.2 Future Tasks 1. Identify the data manipulators with other attack methods, such as denial-of-service attack and replay attack. 2. Considering the goal of attackers, use game theory to analyze the actions of the defenders and the attackers. 8

22 Chapter 2 Wide-Area Oscillation Estimation for Power System using Optimization Algorithms This chapter studies the oscillation mode estimation problem of a large, geographically distributed power system network using streaming Phasor Measurement Unit (PMU) data following a small-signal disturbance in the grid. Accurate estimation of these oscillation modes is critical to grid operators, as many control and protection decisions depend on whether the damping and residues of the modes are below a chosen threshold or not. The estimation problem can be treated as a centralized Least Square (LS) problem using prony method [75, 76]. The centralized approach, however, becomes computationally untenable as more and more PMUs are installed in the system. Instead a distributed solution is much more efficient. Fortunately, the LS problem can be reformulated as a global consensus problem over a distributed network of N computational areas spanning the entire grid using Alternating Direction Multiplier Method (ADMM) [80]. The fundamental setup for the distributed optimization is based on the ADMM algorithm. The power system is divided into multiple non-overlapping areas, each equipped with a local estimator. These local estimators use local sensor measurements of voltage, phase angle, and frequency from PMUs to carry out a local regression algorithm for generating a local estimate 9

23 of the characteristic polynomial of the system, and, thereafter, communicate this estimate to a central supervisor. The supervisor computes the average or consensus of all estimates, and broadcasts this consensus variable back to each local estimator to be used in the next round of regression. If all estimates are accurate, then ADMM is guaranteed to converge asymptotically to the true optimal solution of the characteristic polynomial, following which the central supervisor can solve for its roots to obtain the desired eigenvalues. Furthermore, we consider the case of noisy measurements. When measurements are corrupted by noise, the standard least-squares based regression no longer yields accurate results as LS is inherently a biased estimator. Instead a more robust and noise-tolerant version of LS, namely, Total Least Squares (TLS) [81], need to be used in such scenarios. For reducing the effect of noise on the estimates, a novel algorithm, called TLS-ADMM is proposed. The actual implementation of this algorithm can be easily adapted to the cyber-physical architecture [82]. Simulation results on a IEEE 68-bus power system model illustrates its effectiveness. 2.1 Power System Oscillation Model Consider a power system network consisting of n synchronous generators and n l loads connected by a given topology. Without loss of generality, we assume buses 1 through n to be the generator buses and buses n + 1 through n + n l to be the load buses. Let P i and Q i denote the total active and reactive powers injected to the i th bus (i = 1,...,m + n l ) from the network that is calculated as: n+n l P i = Vi 2 r ik /z 2 ik +V i V k sin(θ ik α ik )/z ik, k=1 n+n l Q i = k=1 V 2 i x ik /z 2 ik V i V k cos(θ ik α ik )/z ik, (2.1a) (2.1b) where V i θ i is the voltage phasor at the i th bus. r ik and x ik in (2.1) are the resistance and reactance of the transmission line joining buses i and k, respectively. θ ik = θ i θ k, z ik = rik 2 + x2 ik, and α ik = tan 1 (r ik /x ik ). The electromechanical model of the power system can be described as a system of 10

24 differential-algebraic equations (DAE) [83] as follows: δ i = ω s (ω i 1) M i ω i = P mi P ei D i (ω i 1), i = 1,..,m, (2.2a) (2.2b) with associated power balance equations given by P ei + P i P Li = 0,i = 1,...,n, Q ei + Q i Q Li = 0, P k P Lk = 0,k = n + 1,...,n + n l, Q k Q Lk = 0, (2.3) where δ i, ω i, M i, D i, P mi, P ei, and Q ei denote the internal angle, speed, inertia, damping, mechanical power, active and reactive electrical powers produced by the i th generator, respectively. P Lk and Q Lk denote the active and reactive powers of the loads at the k th bus. The DAE in (2.2) can be converted to a system of purely differential equations by relating the algebraic variables V i and θ i in (2.1) to the system state variables (δ,ω) and then substituting them back in (2.2) via Kron reduction. The resulting system is a fully connected network of m second-order oscillators with l n(n 1)/2 tie-lines. Let Ẽ i = E i δ i denote the internal voltage phasor of the i th machine. For i = 1,...,m the electromechanical dynamics of the i th generator in Kron s form can be written as: δ i = ω s (ω i 1), M i ω i = P mi P i D i (ω i 1), (X ik P i = E i E k k Zik 2 sin(δ ik ) R ik Zik 2 cos(δ ik ) ), (2.4a) (2.4b) (2.4c) where R i j and X i j denote the resistance and reactance of the line connecting the i th and j th generator in the Kron s form, respectively. Z 2 i j = R2 i j + X 2 i j and δ ik = δ i δ k. Linearizing (2.4) respect to the equilibrium 11

25 (δ i0,1) results in the small signal state space model: δ = 0 m m ω s I m m 0 + u, ω M 1 L M 1 D ω M 1 e j } {{ } } {{ } A B y = col( δ i, ω i ), for i S, (2.5) [ ] T [ ] T where δ = δ 1 δ m, ω = ω 1 ω m, I m m denote the m m identity matrix. M = diag(m i ) and D = diag(d i ) are the m m diagonal matrices of the generator inertias and damping factors, respectively. e j is the j th unit vector with all elements zero but the j th element that is 1, considering that the input is modeled as a change in the mechanical power in the j th machine. Since we are interested only in the oscillatory modes or eigenvalues of A, this assumption is not necessary. The input can be modeled in any other feasible way, such as faults and excitation inputs. The matrix L in (2.5) is the m m Laplacian matrix of the form: [L ] i, j = E ie j ( Xi j cos(δ i0 δ j0 ) + R i j sin(δ i0 δ j0 ) ) i j, [L ] i,i = Zi 2 j n k=1 [L ] i,k. (2.6) Let ˆλ i denote the i th eigenvalue of the matrix M 1 L. The largest eigenvalue of this matrix is equal to 0, and all other eigenvalues are negative, i.e. ˆλ m ˆλ 2 < ˆλ 1 = 0. The eigenvalues of A are given by λ i = ( σ i ± jω i ), (j = 1), where Ω i = ˆλ i denotes the i th osscillation frequency, and σ i > 0 denotes the i th damping factor. Our purpose is to estimate the oscillation modes, frequency Ω and damping factor σ from the PMU data in a distributed archtecture. Next, we will present Prony method and ADMM algorithm for the estimation. 12

26 2.2 Distributed Prony Algorithm with ADMM For estimating the oscillation modes based on PMU measurements, we introduce the existing well-known modal estimation algorithm, Prony analysis method, and its real-time distributed architecture in this section Prony Algorithm Consider a set of N PMU measurements y(t) = col(y 1 (t) y N (t)) are available at t = 0,1,...,M, in a given power system network described in Section II. Following the linearized state space model shown in Equation (2.5), one can write the continuous-time transfer function between the input u(t) and output y p (t) as below [84]: G p (s) = Y n p(s) U(s) = r p,i, (2.7) i=1 s λ i where λ i is the i th pair of eigenvalues, corresponding to the i th pair of oscillatory modes of the system, r p,i is the residue or amplitude of i th mode, and n is the total system order. If we apply an impulse as input to the system, the output y p (t) can be written as y p (t) = n i=1 r p,i e ( σ i+ jω i )t + r p,ie ( σ i jω i )t, p = 1,,N. (2.8) Note that regardless input is an impulse or a step unit, the linearized system response will always be a sum of exponential terms [75, 76]. This is the form that the Prony method can be applied for modal estimation. When y p (t) is sampled at a constant sampling period t, we have the following discrete form: y p (k) = n i=1 r p,i z k i, (2.9) where z i = e ( σ i± jω i ) t. The modal estimation objective is to find the damping factors σ i, the frequencies Ω i and residues r i = col(r 1,i r p,i r N,i ) for i = 1,...,n. In order to estimate these 2n complex eigenvalues λ k from y(t) one may employ centralized techniques such as Prony s algorithm. Sampling 13

27 y i (t) with a uniform sampling period of T, a generic expression for the z-transform of y i (m) y i (t) t=mt, (m = 0,1,...,M), can be written as y i (z) = b 0i + b 1i z 1 + b 2i z b 2ni z 2n 1 + a 1 z 1 + a 2 z a 2n z 2n, (2.10) where a s and b s are constant coefficients of the characteristic polynomial and the zero polynomial, respectively. The roots of the characteristic polynomial will provide the discrete-time poles of the system. One can, therefore, first estimate the coefficient vector a : {a 1,...,a 2n }, compute the discrete-time poles, and finally convert them to the continuous-time poles to obtain σ k and Ω k, for k = 1,...,2n, as follows [75, 76]: Step 1. Solve for a from y i (2n) y i (2n 1) y i (0) y i (2n + 1) y i (2n) y i (1) =... y i (2n + l) }{{} y i (2n + l 1) }{{ y i (l) } c i H i a 1 a 2, (2.11). a 2n } {{ } a where l is an integer satisfying 2n + l M 1. Concatenating c i and H i in (2.11) for i = 1,..., p, one can find a by solving a LS problem H 1 H p c 1 1 min a 2. a. 2, (2.12) c p where denotes the 2-norm of a vector. Step 2. We next find the roots of the discrete-time characteristic polynomial equation as shown in denominator of (2.10) denoted by z k, k = 1,...,n. Then, the eigenvalues of A in (2.5) are equal to ln(z k )/T, T being the sampling period. 14

28 Step 3. The final step is to find the residues r i in (2.8). This can be done by forming the following so-called Vandermonde equation and solving it for r 1 through r n. r y i (0) y i (1) (z 1 ) 1/T (z 2 ) 1/T (z n ) 1/T r = 1.. (2.13).... r y i (M) (z 1 ) M/T (z 2 ) M/T (z n ) M/T n r n The centralized approach, however, becomes computationally untenable as more and more PMUs are installed in the system. Instead a distributed solution is much more preferable. In the next subsection, the LS problem (2.12) is reformulated as a global consensus problem over a distributed network and ADMM is utilized for finding a Real-Time Distributed Prony Algorithm using ADMM In this subsection, we show how the centralized approach delineated in the above subsection for estimating the oscillation modes from PMU data can be recasted as a distributed optimization problem. The LS problem (2.12) is, in fact, a global consensus problem over a network of N computational areas spanning the entire grid, as shown in our recent paper [19]. Each area may belong to a specific utility company. We briefly recall that distributed architecture before stating our main problem statement. An example of the architecture with four areas is shown in Fig We assume each area to be equipped with one local Phasor Data Concentrator (PDC) (located at its control center). These PDCs receive local PMU measurements from that area, run a local LS using these measurements, and then share the estimated parameters with a supervisory PDC at the Independent System Operator (ISO). For convenience, we will refer to the PDCs inside the areas as local PDC and the PDC at the ISO as central PDC as indicated in Fig Let the indices of the PMUs in the i th area be i 1,i 2,...,i mi. The problem (2.12) then can be 15

29 1 PMU Area 2 Power System Network PMU PMU PMU PMU Area PMU Area 4 Area 1 y6 y8 y 2 PDC 1 H 1, c ) ( 1 y 3 k a 1 PDC 2 PDC 3 H, ) H 3, c ) ( 2 c2 k z k z k a 2 Central PDC k k k k a1 a2 a3 z 4 k z a k 4 ( 3 k a 3 k z k a 4 y y PDC 4 H 4, c ) ( 4 Figure 2.1: Distributed architecture for a 4-area power system network. rewritten as min N a 1,...,a N,z i=1 1 2 ĤH i a i ĉc i 2, sub ject to a i z = 0, (2.14) for i = 1,...,N, where a i is the vector of the primal variables, z is the global consensus variable, ĤH i = [H T i 1, H T i 2,..., H T i mi ] T, and ĉc i = [c i1, c i2,..., c imi ] T. Each block element of ĤH i and ĉc i can be constructed after the disturbance using the data matrices shown in (2.11). The estimators can wait up to a certain number of samples, say 2n + l as indicated in (2.11), and gather the local measurements up to that iteration. The ADMM distributed estimation method uses the Lagrangian multiplier approach to solve (2.14). 16

30 The augmented Lagrangian for (2.14) is defined as L ρ = N i=1 ( 1 2ĤH i a i ĉc i 2 + w T i (a i z) + ρ 2 a i z 2 ), where w i is the vector of the dual variables, or the Lagrange multipliers associated with (2.14), and ρ > 0 denotes the penalty factor. Then the optimal problem in (2.14) can be solved in a distributed way using ADMM [80], which reduces to the following set of recursive updates: w k i = w k 1 i + ρ(a k i z k ), (2.15a) a k+1 i = ((ĤH i ) T ĤH i + ρ I) 1 ((ĤH i ) T ĉc i w k i + ρ z k ), (2.15b) z k+1 = 1 N N i=1 a k+1 i. (2.15c) To distinguish it from other variants of ADMM to be proposed later in the paper, we will refer to (2.15) as the standard ADMM, or S-ADMM in short. In [19] we developed the cyber-physical architecture by which local PDCs and the central PDC can exchange information between each other for executing S-ADMM. We summarize that architecture as follows. Consider the k th iteration. Step 1) any local PDC i runs the dual-primal update for (w k i, ak+1 i ) using (2.15a) and (2.15b), after receiving the consensus variable z k from the central PDC; Step 2) the local PDC i transmits a k+1 i to a central PDC; Step 3) the central PDC calculates the consensus variable z k+1 using (2.15c); Step 4) the central PDC broadcasts z k+1 to the local PDCs in each area for their next update. Since the LS problem is convex, therefore as k, z k in (2.15c) converges to z which is the solution of the centralized problem (2.14). Also, because of consensus, every a k j converges to z, 1 j N; Step 5) finally, the central PDC estimates the eigenvalues of the small-signal model by solving for the roots of the characteristic polynomial given by z. Fig. 2.2 shows the trajectories of the data from each PDC from the IEEE 68-bus power system model using S-ADMM algorithm. The system is divided into 5 areas, each with one local PDC and 3 PMUs. The simulated measurements are obtained using the Power System Toolbox (PST) nonlinear 17

31 Figure 2.2: The trajectories of the data from each local PDC using S-ADMM. dynamics simulation routine s simu and the data file data16m.m [85]. We set ρ = The synchronous generators are assumed to have 6 th -order models for simplicity. Since there are 16 generators, our proposed algorithm should ideally solve a 96 th -order polynomial. However, our previous work on this model as reported in [19] show that choosing 2n = 40 yields a reasonably satisfactory estimate of the inter-area modes. From the Fig. 2.2, the data from each PDC will be convergent to the same value using S-ADMM. Then we can use the equilibrium value of the data from each local PDC to calculate the oscillation modes. Fig. 2.3 shows four selected estimated modes σ per iteration. They converge to their global values within 25 iterations. The dashed lines show the actual values of σ for these four modes obtained from PST Noisy Measurements Case The ordinary LS estimator (2.12) is biased when the measurement vector y(t) is noisy. Let ỹ i ( j) = y i ( j)+ ε i ( j) be the noisy measurement, where ε i ( j) denotes the noise or any other measurement imperfection injected into the j th element of the i th PMU measurement. We assume that the noise components injected into different time-samples of any measured output are independent and identically distributed (i.i.d), and the noise in different PMUs may have different distributions. The matrices H i and c i as in (2.11) are then 18

32 0.5 < 1 Estimates of < i < 2 <3 < Iteration (k) Figure 2.3: The trajectories of the data from each local PDC using S-ADMM. rewritten as H i = c i = y i (2n 1) + ε i (2n 1)... y i (0) + ε i (0) y i (2n) + ε i (2n)... y i (1) + ε i (1)... = H i + ε Hi ; (2.16)... y i (2n + l 1) + ε i (2n + l 1)... y i (l) + ε i (l) ỹ i (2n) y i (2n) + ε i (2n) ỹ i (2n + 1) y i (2n + 1) + ε i (2n + 1) = = c i + ε ci. (2.17).. ỹ i (2n + l) y i (2n + l) + ε i (2n + l) 19

33 Now our goal is change to reduce the effect of noise on the estimates. Then the optimization problem (2.14) can be reformulated as: min a N i=1 1 2 ε i 2, sub ject to ( H i P H ε i )a i = c i P Cl ε i P Cr, a i = z, (2.18) where P H = [I l+1, 0 1 (l+1) ], I l+1 is a (l + 1) (l + 1) identity matrix, P Cl = [0 1 (l+1), I l+1 ], and P Cr = [1, 0 1 (2n 1) ] T. The optimization problem (2.18) is commonly referred to as Total Least Squares or TLS [81]. To solve (2.18) in a distributed way, we follow the general ADMM approach, and define the augmented Lagrangian for (2.18) as L ρ = N ( 1 j=1 2 ε i 2 + w T i (a i z) + ρ 1 2 a i z 2 + u T i + ρ ) 2 H i P H ε i )a i c i + P Cl ε i P Cr, [(H i P H ε i )a i c i + P Cl ε i P Cr ] where a i, ε i and z are the vectors of the primal variables, w i and u i are the vectors of the dual variables or the Lagrange multipliers associated with (2.18), and ρ > 0 and ρ 1 > 0 denote the penalty factors. The 20

34 corresponding update equations, similar to (2.15), can be written as: [ ] 1 i = I + ρ(p Cl P H ) T (P Cl P H ) ] [P TH u ki (a ki ) T P TCl u ki P TCr ρ(p Cl P H ) T (H i a ki c i )(P Cr a ki ) T ε k+1 a k+1 [ ] 1 I + (P Cr a k i )(P Cr a k i ) T, [ i = ρ 1 I + ρ(h i P H εi k+1 ) T (H i P H εi k+1 ) [ u k+1 i = u k i + ρ z k+1 = 1 N ] 1 ] i P Cr ) (2.19a) ρ 1 z k w k i (H i P H ε k+1 ) T u k i + ρ(h i P H εi k+1 ) T (c i P Cl ε k+1, (2.19b) [ (H i P H εi k+1 )a k+1 i c i + P Cl εi k+1 P Cr ], (2.19c) N a k+1 j, (2.19d) j=1 w k+1 i = w k i + ρ 1 (a k+1 i z k+1 ). (2.19e) We refer to (2.19) as TLS-ADMM. The actual implementation of this algorithm can be easily adapted to the cyber-physical architecture that we recently proposed in [82]. We summarize that architecture as follows. Consider the k th iteration. Step 1) any local PDC i runs the primal update for ( ε k+1 i and a k+1 i ) using (2.19a) and (2.19b), after receiving the consensus variable z k from the central PDC; Step 2) the local PDC i transmits a k+1 i to a central PDC; Step 3) the central PDC calculates the consensus variable z k+1 using (2.19d); Step 4) the central PDC broadcasts z k+1 to the local PDCs in each area for their next update; Step 5) the local PDC i updates the dual variables ui k+1 and wi k+1 using (2.19c) and (2.19e). Since the optimal problem in (2.18) is convex, therefore as k, z k in (2.19d) converges to z which is the solution of the optimization problem (2.18) [80]. Also, due to consensus, every a k i converges to z, 1 j N; Step 6) finally, the central PDC estimates the eigenvalues of the small-signal model by solving for the roots of the characteristic polynomial given by z. Fig. 2.4 compares the S-ADMM algorithm in (2.15) with the TLS-ADMM algorithm in (2.19) using noisy PMU measurements generated from the IEEE 68-bus power system model as mentioned before. 21

35 Estimates of <i using TLS-ADMM < 1 < 2 < 3 < 4 Estimates of <i using S-ADMM < 1 < 2 < 3 < Iteration (k) Iteration (k) Figure 2.4: Values of the real parts of the four estimated inter-area modes using S-ADMM and TLS-ADMM with noisy measurements, respectively. True values of σ i are shown by dashed lines. The dash lines shows the true values of the damping coefficients σ i for the four dominant inter-area oscillation modes, while the solid lines show their estimated values. In the left figure, the estimates of σ i, obtained via TLS-ADMM, match their true values. In the right figure, however, the estimates obtained using S-ADMM do not match the true values. Same holds for the imaginary parts of the modes. 2.3 Conclusion In this chapter, we presented the oscillation mode estimation problem of a large, geographically distributed power system network using streaming PMU data following a small-signal disturbance in th grid. Firstly, Prony method is utilized for changing the estimation problem to a traditional centralized LS problem. Then we use ADMM to reformulate the problem as a global consensus problem. If all estimates are accurate, then ADMM is guaranteed to converge asymptotically to the true optimal solution of the characteristic polynomial, following which the central supervisor can solve for its roots to obtain the desired eigenvalues. Furthermore, the case of noisy measurement is considered. For reducing the influence of the noise into the performances and obtaining the correct estimates, we proposed TLS-ADMM algorithm. Simulation 22

36 results on a IEEE 68-bus power system model illustrates the effectiveness of these algorithms. 23

37 Chapter 3 Identifying Data-Manipulators in Power System Estimation Loops Due to the high cost associated with dedicated fiber-optic communication networks, the communication between the local PDCs and the central PDC is most likely to happen over an open wide-area communication network. These networks are quite vulnerable to hack. How to catch malicious agents in distributed wide-area monitoring applications is still an open challenge in the WAMS literature. In this Chapter we address this problem in the context of identifying malicious data-manipulators in distributed optimization loops for wide-area oscillation monitoring. The specific application of our interest is the estimation of electromechanical oscillation modes or eigenvalues from streaming PMU data following a small-signal disturbance in the grid. Accurate estimation of these oscillation modes is critical to grid operators, as many control and protection decisions depend on whether the damping and residues of the modes are below a chosen threshold or not. The fundamental setup for the distributed optimization is based on the Alternating Direction Multiplier Method (ADMM) as mentioned in Chapter 2. If all estimates are accurate, then ADMM is guaranteed to converge asymptotically to the true optimal solution of the characteristic polynomial, following which the central supervisor can solve for its roots to obtain the desired eigenvalues. However, if one or more of the local estimates are manipulated by attackers, then the resulting consensus variable will be inaccurate, which, in turn, will contaminate the accuracy of every 24

38 local estimate. Even a small amount of bias at a single iteration can destabilize the entire estimation process. To combat this, we first develop an algorithm to show how ADMM can be used by the central supervisor to catch the identities of malicious estimators by simply tracking the quality of every incoming estimate. This algorithm, however, can become computationally expensive if the network size is large. Therefore, we propose another algorithm where the central supervisor, instead of computing the average, employs a Round-Robin technique to generate the consensus variable, and show that by tracking the evolution of only this consensus variable it is possible to identify the malicious estimators. Both large and covert attacks are considered. To combat this, we propose an algorithm where the central supervisor, employs the deterministic ordered Round-Robin technique to generate the consensus variable. We show that by tracking the evolution of this consensus variable it is possible to identify which estimators are malicious. We analyze the convergence properties of the proposed algorithms, and illustrate their effectiveness using simulation results on a IEEE 68-bus power system model. 3.1 Problem Formulation for Attack Identification Due to the high cost associated with dedicated fiber-optic communication networks, the communication between the local phasor data concentrator (PDC)s and the central PDC is most likely to happen over an open wide-area communication network. These networks are quite vulnerable to hack. A malicious attacker may hack into one or more local PDCs, and corrupt the value of the messages that are transmitted out of these PDCs to the central PDC. Attacking the local PDCs is usually easier than attacking the central PDC since utilities may have hundreds of local PDCs in the cloud [86] while the independent system operator (ISO) will have only one dedicated central PDC with high security. The attack model is accordingly defined as follows. The attacker is allowed to attack at most N 1 local PDCs for the attack to be detectable. The attacker has access to only the outputs a k+1 i and w k i of the attacked PDCs at every iteration k. It does not, however, have the capability to tamper with the execution of the three equations (2.15a)-(2.15c) running inside the PDCs, meaning that it cannot change these update laws in any way, owing to the encryption of the code. The attacker does not know the power system model, nor does it have access to any of the PMU measurements y i (t). The attack may start at k = 0, i.e., at the instant when 25

39 ADMM starts, or at any k > 0 while ADMM is in progress. Because the ISO does not know that the message is corrupted, it will still calculate the consensus variable by averaging the estimates obtained from all individual local PDCs. Thus z k in (2.15c) will become z k = k + 1 N ( N a k i i=1 ), (3.1) ( ) where k = 1 N k j. Notice that here we consider any number of local PDCs to be attacked, as j=1, j S long as there is at least one unattacked PDC, and that the bias k j may be time-varying and of arbitrary magnitude. Although the matrices H k i and c k i are time-varying matrices, i.e., they are functions of the iteration index k, the convexity of the LS problem will yield the same solution z if these two matrices are replaced by two constant matrices H i and c i, respectively, where the latter is constructed by waiting over a certain number of iterations, and gathering all local measurements up to that iteration. Denoting A j := ((ĤH j ) T ĤH j + ρ I 2n ) 1, and C j := (ĤH j ) T c j, following the expression of the consensus variable with bias as in (3.1), the S-ADMM algorithm in (2.15) can be written in a state-variable form: ak+1 a k = L 11 L 12 I 0 } {{ } L ak a k 1 + P 0 k, (3.2) where a k = a k 1 a k 2. a k N, L 12 = ρa 1 ρa 1 N N ρa N N... ρa N N, P = ρa 1 ρa 2. ρa N, L 12 = ρa 1 ρa 1 N N ρa N N... ρa N N, and 26

40 Figure 3.1: The trajectories of the data from each local PDC using S-ADMM with biases. I + (2 N)ρ N A 1 2ρA 1 N... 2ρA 2 N I + (2 N)ρ 2ρA N A N L 11 = ρA N 2ρA N... N N I + (2 N)ρ N A N The rows of L in (3.2) add up to 1, and so when k j = 0 then the trajectories of ak i 2ρA 1 N for every i = 1,...,N converge asymptotically to consensus, as shown in [80]. However, when the arbitrary disturbance k j is added then these trajectories will diverge unless k j is chosen in a special way so that its entries corresponding to the consensus modes are exactly zeros. That, however, is very unlikely to happen as the attacker will not know the matrix L prior to the attack, and, therefore, cannot use any information about its consensus properties for designing k j. In any case, the attacker would benefit most if ak i start diverging, implying that she has been able to destabilize the estimation loop. Fig. 3.1 shows the trajectories of the data from each PDC from the IEEE 68-bus power system model using S-ADMM algorithm as in Chapter 2. The system is divided into 5 areas, each with one local PDC and 3 PMUs. The simulated measurements are obtained using the Power System Toolbox (PST) nonlinear dynamics simulation routine s simu and the data file data16m.m [85]. We set ρ = From the Fig. 3.1, the data from each PDC are divergent 27

41 because of the false-data injections. Thus, we cannot estimate the correct values of the oscillation modes in this case. Detecting the identity of the corrupted PDC, therefore, is crucial to retain normal operation of the loop. In the following sections, we propose a variety of algorithms to catch the identities of these data manipulators, starting with S-ADMM and then its round-robbin version. We also consider the case when j k is small or covert, and show how S-ADMM can be used for the detection by reducing the penalty parameter ρ. In that situation the round-robbin algorithm can also detect the identities of the manipulators by monitoring the dual variable w k j without requiring any knowledge of the individual estimates ak j, thereby saving computation cost. It should also be noted that the algorithms presented in the following sections are solely based on the computed values of the primal and dual variables of ADMM. They do not need any information about the power system model parameters, nor the PMU measurements y i. This is the main difference between our work and the work in [87]. In [87], the authors derived detectability results based on the properties of the state matrix L in (3.2). In our problem set-up, however, L consists of the Hankel matrices ĤH i and ĉc i, both of which are filled with the measured outputs y i. The inherent assumption is that the central PDC does not have direct access to any y i, and therefore, does not know anything about the matrix L. It only has access to the estimates a i and the dual variables w i, i = 1,..., N, and, therefore, must algorithmically figure out the detection and identification mechanisms based on these two variables only. This is the main contribution of the paper, compared to the model-based results of [87]. The time line for executing ADMM and the attack localization algorithms is shown in Fig Data Manipulations with General Biases Detection of attacks For the ADMM algorithm stated in Chapter 2, local PDCs only need to transmit the primal variable a k j to the central PDC. To make the algorithm attack-resilient, we next show that the central PDC can mandate all the local PDCs to transmit both a k j and wk 1 j. 28

42 1. Central PDC detects that one or more PDCs is compromised 2. Central PDC runs Algorithms 1-4 Pre-fault equilibrium t=0 t=t1 t=t2 t=t3 t=tf 1. Fault occurs in the power system 2. PMUs send data to local PDCs 1. PDCs finish gathering (Hi, ci) 2. ADMM starts 3. Data manipulation starts 1. Central PDC localizes the faulty PDC 2. ADMM continues with the nonfaulty PDCs 1. ADMM terminates, eigenmodes are computed Figure 3.2: The timing diagram of the communication between the central and local PDCs. We denote the value of a k i received at the central PDC at iteration k as āa k i. We define āak = [(āa k 1 )T,(āa k 2 )T,...,(āa k N )T ] T. If i S, āa k i = a k i + k i ; otherwise, āak i = a k i. If there is no attacker in the system, then at the first iteration 1 N N āa 1 i = 1 N N a 1 i = z 1. We assume w 0 j = 0 2n 1, where 0 2n 1 is a 2n 1 i=1 i=1 matrix whose elements are all zeros. According to (2.15a), we have 1 N N i w k+1 i = 1 N N i w k i = 0 2n 1 ; while if any PDC is biased, then z 1 = 1 N N āa 1 i = 1 N N a 1 i + 1 and, hence, 1 N N w 1 i = ρ 1. Thus the central i=1 i=1 i=1 PDC can detect the presence of malicious users at any iteration k > 0 by simply checking the difference between two average values of the dual variables in two successive iterations. Next we will describe two algorithms by which central PDC can identify which local PDCs are malicious S-ADMM for Identifying Malicious PDCs For any pair of PDCs (i, j), we define the quantity d k i, j = āa k i ak j. For i S and j / S, from (3.2) one can write d k+1 i, j = āa k+1 i āa k+1 j = (L i L j )ãa k + (P i P j ) k + k+1 i, (3.3) 29

43 where ãa k = ak a k 1 other hand, for i, j / S,, and L i and P i are the i th (2n 2nN) block rows of L and P, respectively. On the d k+1 i, j = (L i L j )ãa k + (P i P j ) k. (3.4) Comparing (3.3) and (3.4), it can be seen that if the minimum absolute value of a non-zero element of k+1 i is large enough, then the difference of estimates between two non-malicious PDCs can be much smaller compared to the difference between any malicious PDC and any non-malicious PDC. Thus, at any iteration the central PDC will be able to separate the incoming messages into at least two groups based of the values of the biases by simply computing the difference between every pair of messages arriving from the local PDCs. The messages without biases will belong to the same group. We define a threshold γ k a to identify the group members at iteration k as scalar threshold: { āa γa k k = min max āa k min }, N( āa k N min2 āa k min ), (3.5) where āa k max, āa k min, and āak min2 are the maximum, minimum, second minimum values of āak j, 1 j N, respectively. In what follows, we will simply use the symbol to represent Euclidian norm. Note that γ k a is one of many other choices for the threshold. If āa k j āak i γk a, then the central PDC classifies the vectors a k j and ak i to be in the same group; otherwise, a k j and ak i are treated to be in the different groups. Note that, āa k i āak j = ak i + k i ak j, i S and j / S. After a few calculations it can be easily shown that for successful localization, i.e., for making āa k i āak j > γk a, the bias k i must satisfy k i k max N > ak max a k min + a k N j a k i (3.6) k i > N ( a k min2 a k min ) + a k j a k i, (3.7) for all k, where represents maximum norm. Otherwise, if i, j / S āa k i āak j = ak i ak j, to 30

44 guarantee āa k i āak j < γk a, k max must satisfy k max > N ( a k i a k j ) + a k min a k max. (3.8) If the biases satisfy the requirements as in (3.6)-(3.8), then S-ADMM can successfully identify the malicious PDCs by simply tracking the differences āa k i āak j. In reality, however, these lower bounds may not mean much since the fundamental rationale behind the detection and localization are all based on the quality of the estimates, which depend on the numerical magnitude of the measurements that are specific to that particular distance event. Algorithm 1 summarizes the implementation of this simple method. Algorithm 1 Identifying malicious PDCs injected with general biases using S-ADMM Detection: 1) At any iteration k, every local PDC computes a k+1 j in (2.15b) and w k j in (2.15a), j = 1,...,N, and transmits these two messages to the central PDC. 2) If at any iteration k > 0 the central PDC finds 1 N N w k i 1 N N w k 1 i 0 2n 1, it suspects that there i=1 i=1 exists one or more malicious PDCs in the system. Identification: 3) If Step 2 is positive, for all k > k the central PDC computes the difference āa k i āak j, 1 i, j N, and the threshold γa. k It then compares these differences to the threshold, and separates āa k into groups. 4) The central PDC finds the index j of the vectors āa k j, 1 j N, whose 2-norm is minimum. It then picks the group where the vector with this index is located, and classifies this group as unbiased. 5) The central PDC repeats this classification for a sufficiently large iteration s. If the identified nonmalicious PDCs are consistent through these iterations, it finally confirms that these PDCs are unbiased. 6) Onwards from iteration s + k, the central PDC ignores any message coming from the malicious PDCs, and simply carries out S-ADMM with the remaining non-malicious PDCs using (2.15). The final solution of this S-ADMM will lead to the solution of (2.14) as the LS problem is convex with s + k being an initial iteration for the rest of the non-malicious S-ADMM. Since the success of this algorithm is contingent on the quality of the estimates, it is difficult to specify a universial lower bound for s in Step 5. Generally, one would choose s to be large, but it actual value must be decided on a case-by-case basis. Also, note that after identifying the attacked PDCs, Step 6 of the algorithm eliminates these PDCs, and continues normal estimation using the good PDCs. The 31

45 reason why omitting a certain set of PDCs translates into retainment of stability is because the original least-squares problem in (5) is based on consensus. This means that if every node estimates a i correctly, then the ADMM algorithm is guaranteed to converge to the centralized least-squares solution a [80]. More importantly, the solution for this convergence does not depend on how many PDCs are there in the system. As every PDC is trying to reach the same optimal point a, it does not matter whether there are N PDCs, or less than N PDCs. The speed of convergence, of course, may slow down as more and more PDCs are omitted, but the final solution will remain the same. We illustrate the approach with a simple example. Consider a system with 5 local PDCs. The second and third PDCs are respectively injected with k 2 = δ k 2 1 2n 1 and k 3 = δ k 3 1 2n 1, where δ k 2 and δ k 3 are two different arbitrary time-varying numbers and 1 2n 1 is a 2n 1 vector with all elements one. All other PDCs are unbiased. Fig. 3.3 shows the first element of the consensus vector z k before and after detecting the malicious PDCs uing Algorithm 1. In the figure, it can be seen that at iteration k = 2 we have āa 2 1 = , āa2 2 = , āa2 3 = , āa2 4 = , āa2 5 = Thus, āa 2 max = , āa 2 min = , and āa2 min2 = The threshold, therefore, is γ2 a = The estimates āa 2 are separated into three groups: āa 2 1, āa2 4, and āa2 5 are in the first group, āa2 2 and āa2 3 are in the second and the third groups, respectively. The magnitude āa 2 1 in the first group is minimum. Thus, PDCs 1, 4 and 5 are identified as non-malicious, or alternatively, PDCs 2 and 3 are identified as malicious. After iteration 2, the central PDC cuts off communication with PDCs 2 and 3, and only calculates the average of messages received from the unbiased PDCs, leading to z k = 1 3 (ak 1 + ak 4 + ak 5 ), k > 2. The estimates thereby asymptotically converge to the ideal solution z, as expected Round-Robbin ADMM for Detecting the Malicious Users Algorithm 1, though very simple to implement, may suffer from computational challenges as the central PDC has to process 2nN number of elements at every iteration k. In practice, there may be thousands of virtual computers installed in a cloud network serving the purpose of these local PDCs, as shown in our recent paper [88], which makes N a very large number. The computational memory of the central PDC may not be able to track such a large number of variables per iteration in real-time. Instead it may only 32

46 Vector norms PDCs 2 and 3 are detected as malicious Cut off PDCs 2 and 3 k + + Continue to calculate zav = a a a 3 k k k av,1 av,4 av,5 z * (1)= Figure 3.3: Evolution of the norms of the local estimates, and the average z k before and after detection using S-ADMM. For convenience, we only show the first element of z k. Table 3.1: Comparison of S-ADMM and RR-ADMM for detection of attacks with large biases S-ADMM RR-ADMM Variable used for detection a k av, j z k rr N(N 1) Minimum # of computations per iteration 2 1 Minimum # of iterations needed to identify 1 k min + N Bias Magnitude Less stringent More stringent want to track or keep an eye on a much smaller number of variables such as only the consensus vector z k which has 2n number of elements in it. Under that condition, if any local estimate a k i is corrupted by bias k i at every iteration, then it will be impossible for the central PDC to identify the malicious PDCs, or identify which PDCs are unbiased, just by tracking z k. The main question, therefore, is how can the central PDC catch the manipulators by simply tracking z k over every iteration? We next propose a variant of (2.15) using a Round-Robin strategy replacing the averaging step (2.15c) to solve this problem. We refer to this algorithm as Round-Robin ADMM or RR-ADMM in short. The basic strategy for RR-ADMM is as follows. At iteration k = 1, for example, the central PDC 33

47 receives local estimates a 1 j from every local PDC, but computes z1 simply as z 1 = α a 1 1, where α is a constant non-zero number. Then the central PDC sends z 1 back to the local PDCs following Step 4 of S-ADMM. Similarly, in iteration k = 2, the central PDC uses z 2 = α a 2 2, at iteration k = 3 it uses z 3 = α a 3 3, and so on. In general, zk = α a k ((k 1) mod N)+1. N successive iterations constitute one period of RR-ADMM where N is the total number of local PDCs. After N iterations, the central PDC will again start from PDC 1, then PDC 2, and so on. For convenience of expression, we denote the consensus variables of RR-ADMM and S-ADMM at iteration k as z k rr and z k av, the latter being updated in (2.15c). Similarly, the local PDC estimates and its dual variable will be denoted as a k rr, j and wk rr, j, and ak av, j as in (2.15b) and w k av, j as in (2.15a), 1 j N, respectively. Remark: Note that the purpose of using RR-ADMM is only to detect the malicious local PDC, not for obtaining the optimal solution of (2.14). This is because this algorithm is run by the central PDC stealthily, while every local PDC still believes that the central PDC uses S-ADMM to calculate z k, and thereby updates a k+1 rr, j in the same way as in (2.15b). Therefore, this algorithm should be treated more as a S-ADMM with a RR-averaging step, rather than a true RR-ADMM where every step of (2.15) would have to be modified in accordance to the RR strategy. Considering some of the PDCs to be malicious, the ADMM update equations using RR-averaging can be written as: w k rr,i =w k 1 rr,i + ρ(a k rr,i z k rr), (3.9a) a k+1 rr,i =((ĤH k i ) T ĤH k i + ρ I) 1 ((ĤH k i ) T ĉc k i w k i + ρ z k rr), (3.9b) z k+1 rr =αāa k+1 rr,((k 1) mod N)+1 = α(ak+1 rr,((k 1) mod N)+1 + k+1 ((k 1) mod N)+1 ). (3.9c) From (3.9c), it is clear that unlike S-ADMM where z k av only depends on the average value of the biases, the expression for z k rr is influenced by the injection of the [((k 1) mod N) + 1] th PDC, i.e., k ((k 1) mod N)+1, which provides a potential signature to catch the identities of the PDCs with disturbances. In [89], we proposed a way to detect the malicious PDC by finding the maximum ξ i = z i rr zrr i+1 + z i rr z i 1 rr, 1 < i < N + 1, which holds only when there is exactly one malicious PDC. We next provide an 34

48 alternative algorithm that applies for multiple attacks. From (3.9), at iteration k, we can write the consensus vector z k rr as z k rr = α k b + α ( (ĤH k 1 b [ ) ) T (ĤH k 1 1 b ) + ρ I 2n (ĤH k 1 b ] ) T ĉc k 1 w 0 k 1 rr,b ρ (a j rr,b z rr) j + ρ z k 1, b j=1 rr (3.10) where b = ((k 1) mod N) + 1. We assume that the minimum absolute value of the element in the variable k b is large enough. It follows from (3.10) that the minimum value of z rr in one period must be from a non-malicious PDC. Let k min denote the iteration index in the period k = 1,...,N where the magnitude of z k rr is minimum, 1 k N, and define a threshold γ z as γ z = z k min+n rr z k min rr. (3.11) If z k rr > z k min rr +γ z, k min k k min +N 1, then the central PDC infers that the [(k 1) mod N +1] th PDC is attacked. Algorithm 2 summarizes this detection mechanism. Notice that if the biases are constant, the central PDC does not need to search for the minimum value of z k rr, 1 k N, and the threshold γ z can be defined as zrr N+1 z 1 rr. In that case, the central PDC will only wait for N k iterations, not k min + N iterations. Consider the same example as in Fig Fig. 3.4 shows the first element of the consensus vector z k rr before and after detection using RR-ADMM. In the first period, z 1 rr = , z 2 rr = , z 3 rr = , z 4 rr = , and z 5 rr = Thus, z 1 rr is minimum and k min = 1. The threshold is computed as γ z = z 6 rr z 1 rr = Only z 2 rr and z 3 rr are larger than γ z + z 1 rr. So PDCs 2 and 3 are identified as malicious. After detecting these manipulators at iteration 6, the central PDC cuts off the signal from PDCs 2 and 3, and only calculates the average of messages received from the other local PDCs leading to z k av = 1 3 (ak av,1 + ak av,4 + ak av,5 ), k > 6. Note that at the same time the dual variable w k j must be reset to its initial value. At iteration 7, a7 av, j is updated using the value of z6 rr. The requirement on the magnitude of the biases for successful detection (i.e. for minimizing false positives) is much more strict for RR-ADMM than for S-ADMM. This is because the threshold γ z is 35

49 Algorithm 2 Identifying malicious PDC with general biases using RR-ADMM Detection: 1) At any iteration k, every local PDC computes a k+1 j in (2.15b) and w k j in (2.15a), j = 1,...,N, and transmits these two messages to the central PDC. 2) If at any iteration k > 0 the central PDC finds 1 N N w k i 1 N N w k 1 i 0 2n 1, it suspects that there i=1 i=1 exists one or more malicious PDCs in the system. Identification: 3) If Step 2 is positive, for k > k the central PDC switches to RR-ADMM. That is, every local PDC computes a k+1 rr, j in (3.9b) and w k+1 rr, j in (3.9a), j = 1,...,N, and transmits them to the central PDC. 4) When k N + k, the central PDC searches for the minimum value z k min rr iteration index k min. 5) Waiting till k k min + N, the central PDC computes the threshold γ z = z k min+n rr compares z i rr to z k min rr + γ z for k min i k min + N 1. in one period and its z k min rr. It then 6) If z i rr > z k min rr + γ z, then the central PDC identifies the [(i 1) mod N + 1] th PDC to be malicious. 7) The central PDC repeats this classification for a few iteration, say up to iteration s. If the identified non-malicious PDCs are consistent through these iterations, it finally confirms that these PDCs are unbiased. 8) Onwards from iteration s + k, the central PDC ignores any message coming from the malicious PDCs, and simply carries out S-ADMM with the remaining non-malicious PDCs using (2.15). The final solution of this S-ADMM will lead to the solution of (2.14) as the LS problem is convex with s + k being an initial iteration for the rest of the non-malicious S-ADMM. dependent on the speed of divergence of the elements of z rr in one period. For S-ADMM, however, āa k av, j, 1 j N, are compared only at one iteration, and so the biases are less affected by the speed of divergence. Also note that if every element of the bias vector at every iteration is non-zero, and the central PDC knows this information, then the detection can be done by only using any chosen element of āa k av, j, 1 j 2n, and zk rr (for S- and RR-ADMM respectively) instead of using the vector norms. Table 3.1 compares S-ADMM and RR-ADMM for detecting attacks with large magnitudes Random Order of RR-ADMM Note that the order of choosing the PDCs for the averaging step in the RR-ADMM need not be fixed. The algorithm should succeed with any random order as long as every PDC is visited exactly once in a period. For example, if there are 4 local PDCs, then the central PDC need not follow the order as (1, 2, 3, 4) for every period. It can, for instance, choose (1, 2, 3, 4) for the first period, (1, 3, 4, 2) for 36

50 The PDCs 2 and 3 are detected as malicious Cut off PDCs 2 and 3 6 Resetw rr,j k Continue to calculatez av = a k k k av,1+a av,4+aav,5 3 z * (1)= Figure 3.4: Evolution of the average z k before and after detection using RR-ADMM. For convenience, we only show the first element of z k. the second, (2, 4, 3, 1) for the third, and so on. This gives the central PDC the flexibility in choosing z k rr in case any of the local estimates do not arrive on time due to message loss or denial-of-service. In that case, Algorithm 2 needs to be modified slightly to accommodate this random order. After Step 4 in Algorithm 2, the central PDC should find the index of the local PDC corresponding to the iteration index k min. Let this PDC index be m. Let k be the iteration index between N k and 2N + k, considering z k rr = αāa k rr,m. The threshold is then changed to γ z = z k rr z k min rr. After that the central PDC compares z k rr to γ z + z k min rr is identified as malicious., 1 + k < k < N + k. If z k rr > γ z + z k min rr and if z k rr = αāa k rr,i, then the i th PDC We will illustrate this approach using a real power system simulation in Case 3 of Section

51 3.3 Data Manipulations with Small Biases S-ADMM for Detecting Malicious Users with Small Biases The basic approach for this method is the same as in Subsection Recall equations (3.3) and (3.4) as follows. For i S and j / S d k+1 i, j = (L i L j )ãa k + (P i P j ) k + k+1 i, (3.12) if i, j / S, then d k+1 i, j = (L i L j )ãa k + (P i P j ) k = (L i L j )ãa k + ρ(a i A j ) k, (3.13) where the last equation follows from the definition of P in Section 3.1. The problem, however, is that if k+1 i is small, then the value of ρ(a j A i ) k + i k+1 in (3.12) may become comparable to ρ(a i A j ) k in (3.13), thereby leading to incorrect classification. One way to bypass this would be to reduce the value of ρ > 0 such that the difference between the LHS of (3.12) and (3.13) is still large enough for detection despite k+1 i being a small number. Thus, the only difference of this approach from that in Section is that the ISO must ask every local PDC to reduce their penalty factor ρ once it realizes the presence of a false-data injector. Algorithm 3 describes this method. Notice that the penalty factor ρ is a private parameter, and hence unknown to the attacker. However, following the attack model stated in Section 3.1, the communication link connecting the central PDC to any of the local PDCs, is assumed to be uncompromised. Thus, when the central PDC broadcasts the instruction to reduce ρ, then every PDC whether attacked or unattacked, must be able to follow this instruction. This satisfies Step 4 of Algorithm 3. Consider the same example as in Section 3.2. As before, we assume there are five local PDCs and one central PDC. The second and the third local PDCs are malicious, i.e., they send their estimates with two different arbitrary biases k 2 and k 3 at iteration k, respectively. Fig. 3.5 shows the trajectories of the first element of the vector z k av and the first elements of the un-biased vectors āa k av,1, āak av,4, and āak av,5 pre and post 38

52 Algorithm 3 Identifying malicious PDCs with small biases using S-ADMM Detection: 1) 1) At any iteration k, every local PDC computes a k+1 j in (2.15b) and w k j in (2.15a), j = 1,...,N, and transmits these two messages to the central PDC. 2) If at any iteration k > 0 the central PDC finds 1 N N w k i 1 N N w k 1 i 0 2n 1, it suspects that there i=1 i=1 exists one or more malicious PDCs in the system. Identification: 3) If Step 2 is positive, for k > k the central PDC asks every individual PDC j to update aav, k+1 j as in (2.15b) by reducing the value of ρ, j = 1,...,N. 4) Every local PDC updates ρ, and computes aav, k+1 j in (2.15b), zav k+1 in (2.15c), and w k+1 av, j in (2.15a), j = 1,...,N, and transmits them to the central PDC. 5) For all k > k, the central PDC computes the difference āa k av,i āak av, j, 1 i, j N, and the threshold γa. k It then compares these differences to the threshold, and separates āa k av into groups. 6) The central PDC finds the index j of the vectors āa k av, j, 1 j N, whose 2-norm is minimum. It then picks the group where the vector with this index is located, and classifies the group as unbiased. 7) The central PDC repeats this classification for a few iteration, say up to iteration s. If the identified non-malicious PDCs are consistent through these iterations, it finally confirms that these PDCs are unbiased. 8) Onwards from iteration s + k, the central PDC ignores any message coming from the malicious PDCs, resets the value of ρ to its initial value, and simply carries out S-ADMM with the remaining non-malicious PDCs using (2.15). The final solution of this S-ADMM will lead to the solution of (2.14) as the LS problem is convex with s + k being an initial iteration for the rest of the non-malicious S-ADMM. 39

53 Vector norms PDCs 2 and 3 are detected as malicious Cut off PDCs 2 and 3-6 Reset r = 10 Continue to calculate z k av + + = a a a 3 k k k av,1 av,4 av,5 z * (1) = Figure 3.5: Evolution of the norms of the local estimates, and the average z k av before and after detection of attack with small biases using S-ADMM. For convenience, we only show the first element of z k av. detection using S-ADMM. In Fig. 3.5, at the second iteration, with ρ reduced to 10 9, āa 2 av,1 = 0.283, āa 2 av,2 = , āa2 av,3 = , āa2 av,4 = , āa2 av,5 = The threshold is γ2 a = āa 2 av is separated into three groups: āa 2 av,1, āa2 av,4, and āa2 av,5 are in the first group, āa2 av,2 and āa2 av,3 are in the second and the third groups, respectively. The value of āa 2 av,4 in the first group is minimum. Therefore, PDCs 1, 4, and 5 are identified to be non-malicious which matches with the true situation. After the second iteration, the central PDC cuts off communication with PDCs 2 and 3, and only calculates the average of messages received from the other local PDCs leading to z k av = 1 3 (ak av,1 + ak av,4 + ak av,5 ), k > 2. Notice that at iteration 3, ρ should be reset to The estimates asymptotically converge to the true solution z, as expected. 40

54 3.3.2 RR-ADMM for Detecting Malicious Users with Small Biases The basic requirement for successful identification using S-ADMM is that the bias, no matter how small, should be large enough so that the difference of estimates between a malicious PDC and non-malicious PDC is notably larger than that between two non-malicious PDCs. In this section, we employ RR-ADMM using the dual variable w k rr, j instead of the consensus variable zk rr for detection where this requirement is less stringent. From (3.9a) and (3.9c), the difference w k rr,i wk 1 rr,i, considering α = 1, can be written as w k rr,i w k 1 rr,i = ρ(a k rr,i a k rr,(k 1) mod N+1 k (k 1) mod N+1 ). (3.14) If 1 k N and i = k, then (3.14) is reduced to w k rr,k wk 1 rr,k = ρ(ak rr,k ak rr,k k k ) = ρ k k. (3.15) Therefore, if k k = 0 2n 1, then w k rr,k wk 1 rr,k = 0 2n 1, where 0 2n 1 is a 2n 1 vector with all zero elements. This indicates that if the i th PDC is not injected with any bias, then at the i th iteration, w i rr,i wi 1 rr,i = 0 2n 1 ; if this condition is not met, the i th PDC must be malicious. The central PDC can check for the above inequality to identify the malicious PDC. For instance, at iteration 2, the central PDC should calculate the difference w 2 rr,2 w1 rr,2. If the difference is not equal to zero, PDC 2 must be malicious. At iteration 3, the central PDC should check the difference w 3 rr,3 w2 rr,3 ; at iteration N + 1, the central PDC should check the difference w N+1 rr,1 wn rr,1. Thus, after N + 1 iterations, the central PDC will be able to detect the identities of all manipulators no matter what the values of their biases are. Notice that for the exact accuracy of identification, the central PDC should compare each element of w k rr,k to the corresponding elements of w k 1 rr,k. If the central PDC compares the norms wk rr,k to wk 1 rr,k, then it is possible that under the condition k (k 1) mod N+1 0 2n 1, these two norms may turn out to be equal thereby making the identification inaccurate. The upshot is that every local PDC now must also send the dual variable to the central PDC at every iteration, thereby increasing the volume of the transmitted messages. It must be noted, again, that the RR-ADMM should only be used for catching data-manipulators, and not to solve the optimal solution of (2.14). Algorithm 5 summarizes the RR-ADMM strategy. 41

55 Algorithm 4 Identifying malicious PDCs with small biases using RR-ADMM Detection: 1) At any iteration k, every local PDC computes a k+1 j in (2.15b) and w k j in (2.15a), j = 1,...,N, and transmits these two messages to the central PDC. 2) If at any iteration k > 0, (k mod N) = 0, the central PDC finds 1 N N w k i 1 N N w k 1 i 0 2n 1, it i=1 i=1 suspects that there exists one or more malicious PDCs in the system. Identification: 3) If Step 2 is positive, for k > k the central PDC calculates the consensus variable z k rr using RR-ADMM instead of S-ADMM. 4) Every local PDC updates a k rr in (3.9b), z k rr in (3.9c), and w k rr in (3.9a), j = 1,...,N, and transmits them to the central PDC. 5) After N + 1 iterations, the central PDC calculates the differences wrr,i i+k difference w N+1+k w N+k rr,1 rr,1 w i 1+k rr,i, 2 i N, and the. It then checks if these differences are exactly zero or not. PDCs showing non-zero differences are classified as malicious. 6) The central PDC repeats this classification for a few iteration, say up to iteration s. If the identified malicious PDCs are consistent through these iterations, it finally confirms these PDCs are malicious. 7) Onwards from iteration s + k, the central PDC ignores any message coming from the malicious PDC, resets the dual variable w s+k j to its initial value, and simply carries out S-ADMM with the remaining non-malicious PDCs using (2.15). The final solution of this S-ADMM will lead to the solution of (2.14) as the LS problem is convex with s + k being an initial iteration for the rest of the non-malicious S-ADMM. 42

56 Note that in Step 5 of Algorithm 5, the only way for a malicious PDC to fool the central PDC will be to send identical values of its dual variable over two successive iterations. However, fortunately the local PDCs do not know that the central PDC is running RR-ADMM and that too in what order. Therefore, this is a rather unlikely situation. Even if the malicious PDCs send incorrect values of the dual variables, the detection is still possible as (4.1) will still hold. That is, every good PDC must obey Step 4 for the detection to be successful. Consider the same example as in Fig Fig. 3.6 shows the trajectory of the first element of the vectors z k pre and post detection. In the top figure of Fig. 3.6, w 2 rr,2 (1) w1 rr,2 (1) = and w 3 rr,3 (1) w2 rr,3 (1) = The central PDC also calculates w 4 rr,4 w3 rr,4 = 0 2n 1, w 5 rr,5 w4 rr,5 = 0 2n 1, and w 6 rr,1 w5 rr,1 = 0 2n 1, which means PDCs 1, 4, and 5 are non-malicious. After iteration 6, the central PDC cuts off communication with the malicious PDCs, and calculates the consensus vector using S-ADMM which is the same as the one in Fig The bottom figure of Fig. 3.6 shows the consensus variable z k rr before detection and z k av after detection. Notice that at iteration 7, a 7 j is reinitialized to the value based on z 6 rr, and the dual variable w 7 j is reset to its initial value. For calculating the differences of the dual variables over two successive iterations of RR-ADMM, the central PDC has to process only one number at every iteration; while from the S-ADMM algorithm (2.15) it follows that the central PDC has to process at least N(N 1) 2 numbers of computations at every iteration. Thus if N is large then RR-ADMM will require much less computation time than S-ADMM. Table 3.2 compares S-ADMM and RR-ADMM for detection of small biases. Table 3.2: S-ADMM and RR-ADMM for detection with multiple small biases S-ADMM RR-ADMM Variable used for detection a k av, j w k rr, j N(N 1) Minimum # of computations per iteration 2 1 Minimum # of iterations needed to identify 1 N

57 The first element of z k The first element of w k w - w ¹ rr,2 rr,2 w - w ¹ rr,3 rr,3 w - w = rr,4 rr,4 w - w = rr,5 rr,5 w - w = rr,1 rr,1 Iteration (k) k w (1) rr,1 k w (1) rr,2 k w (1) rr,3 k w (1) rr,4 k w (1) rr,5 PDCs 2 and 3 are detected as malicious Cut off PDCs 2 and 3 6 Reset w j k k k k aav,1 + aav,4 + aav,5 Continue to calculate zav = 3 Iteration (k) Figure 3.6: The response of z k before and after detection of attack with small biases using RR-ADMM. 3.4 Comparison between S-ADMM and RR-ADMM It must be noted, again, the RR-ADMM should only be used for catching data-manipulators, not to solve the optimal solution. In fact, in this section we show that the steady-state solutions of S-ADMM and RR-ADMM with k j = 0 are different from each other, which means that if the central PDC only executes RR-ADMM it will not be able to obtain an accurate solution. From Eq. (3.9b), the value of z k rr depends on the coefficient α. Generally, if α 1, z k rr z k av with k j = 0. Thus, we only consider the case of α = 1. We assume w 0 av, j = w0 rr, j and z0 av = z 0 rr. Thus, a 1 av, j = a1 rr, j, 1 j N. However, z1 av, j z 1 rr, j, because in each iteration ĤH i ĤH j and a 1 av, j a 1 av,i, 1 i, j N, i j. When k, zk av = a k av, j. In that situation, without loss of generality, we assume z k rr = a k rr, j. Thus zk av z k rr = a k av, j ak rr, j. Let Ak j = ((ĤHk j) T ĤH k j + ρ I 2n ) 1, and B k j = (ĤHk j) T c k j w0 j. According to Eqs. (2.15b) and (3.9b), when k, the difference 44

58 between a k+1 av, j and a k+1 rr, j can be deduced as where a k+1 av, j ak+1 rr, j = ρ k i=1 f i (A 1 j,...,a k j,ρ)(z i av z i rr), (3.16) f 1 (A 1 j,...,a k j,ρ) =( 1) (k 1) +( 1) k i=1 A i jρ k 1 k 1 + ( 1) l k 1 2ρ k 2 k A i j k 2 i=1 l=2 A l j l=0 k i=k l A i jρ l k ( 1) l=1 ρa k ja l j,k 2, (3.17) f 2 (A 2 j,...,a k j,ρ) =( 1) (k 2) 2 +( 1) k i=2 A i jρ k 2 k 2 + ( 1) l k 1 2ρ k 3 k A i j k 3 i=2 l=3 A l j l=0 k i=k l A i jρ l k ( 1) l=2 ρa k ja l j,k 3, (3.18). f k (A k j) = 2A k j,k 1. (3.19) The expressions for the various functions in (3.16) are derived in the Appendix A.1. Since z k av z k rr, k 1, and the value of f i (A 1 j,...,ak j,ρ), 1 i k, are not equal to zero in general, the difference between a k av, j and ak rr, j is not equal to zero. In fact, the following corollary can be written from (3.16). Corollary I: z k rr = z k av, k only if ĤH i = ĤH j, a 0 av,i = a0 av, j = a0 rr,i = a0 rr, j, and w0 av,i = w0 av, j = w0 rr,i = w 0 rr, j, 1 i, j N, i j. Proof: Based on Eq. (2.15b), because ĤH i = ĤH j and a 0 av,i = a0 av, j, 1 i, j N, i j, a1 av,i = a1 av, j = z1 av. Then this result could be extended to any iteration, i.e., a k av,i = ak av, j = zk av. Due to the same update 45

59 equations of a k av and a k rr, and the same initial values, a k av,i = ak rr,i = zk av = z k rr for every k, which follows directly from (3.16). 3.5 Simulation Results To verify our algorithms we consider the IEEE 68-bus power system model shown in Fig The system is divided into 5 areas, each with one local PDC and 3 PMUs. Fig. 3.8 shows the communication between the 5 local PDCs and the central PDC. The red lines denote that the PDCs sending messages through these links are attacked. In this case, PDC 2 and PDC 3 are malicious. The simulated measurements are obtained using the Power System Toolbox (PST) nonlinear dynamics simulation routine s simu and the data file data16m.m [85]. We set ρ = The synchronous generators are assumed to have 6 th -order models for simplicity. Since there are 16 generators, our proposed algorithm should ideally solve a 96 th -order polynomial. However, our previous work on this model as reported in [19] show that choosing 2n = 40 yields a reasonably satisfactory estimate of the inter-area modes. The true values of the four inter-area modes σ i ± jω i, 1 i 4, obtained by PST are ± j2.2262, ± j3.2505, ± j3.5809, and ± j Next, we use the ADMM algorithm to estimate the oscillation modes. The following cases are considered for the purpose of illustrating the results. Case 1:) S-ADMM for detecting data manipulators with general biases. We next apply Algorithm 1 to catch the malicious PDCs 2 and 3. Fig. 3.9 shows the norms of the vector āa k av,i, 1 i N. At the first iteration, āa1 av,1 = , āa1 av,2 = , āa1 av,3 = 6.839, āa 1 av,4 = , and āa1 av,5 = The threshold γ1 a = āa 1 av is separated into three groups: āa 1 av,1, āa1 av,4, and āa1 av,5 are in the first group, āa1 av,2 and āa1 av,3 are in the second and third groups, respectively. According to Algorithm 1, the minimum value āa 1 av,5 is in the first group. Therefore, PDCs 1, 4, and 5 are identified as non-malicious, which matches the true situation. Fig shows the estimations of σ i, 1 i 4, for the four inter-area modes after PDCs 2 and 3 are disconnected. The dashed lines show the actual values of σ i. The final values of the estimates of σ i in this case match with their actual values again. 46

60 42 67 Area 3 Area 2 Area 1 PMU 41 PMU G G 10 G PMU G PMU G G 13 Area 4 PMU PMU PMU PMU G 12 PMU G PMU G 2 PMU PMU G 3 G PMU G 4 26 Area 5 28 G G G 6 PMU G 7 20 PMU Figure 3.7: IEEE 68-bus model Compromised PDC Compromised PDC PDC 2 PDC 3 H, ) H 3, c ) ( 2 c2 ( 3 PDC 1 H 1, c ) ( 1 k a 1 k z z k k k a2 +Da2 Central PDC k z 5 å 1 = i= 5 a k i z k k k a3 +Da3 z k k a 4 PDC 4 H 4, c ) ( 4 k a 5 k z PDC 5 H, c 5 ) ( 5 Figure 3.8: Architecture for a 5-area power system network with 2 malicious PDCs. 47

61 Figure 3.9: Evolution of āa k av, j when S-ADMM is run under attacks. Considering the same example, Fig shows the norm and the first four elements of the consensus vector z k av using S-ADMM. None of these trajectories has any signature of the biases from PDCs 2 and 3, and hence it is impossible to identify them by just tracking z k av. Case 2:) RR-ADMM for detecting data manipulators with general biases. We next apply Algorithm 2 to detect the malicious PDCs 2 and 3. The top figure of Fig depicts the norm of the consensus vector z k rr using RR-ADMM. From iteration 1 to iteration 5, z 1 rr = is minimum. z 6 rr = and the threshold is γ z = z 2 rr > z 1 rr +γ z and z 3 rr > z 1 rr +γ z, so PDCs 2 and 3 are identified as malicious. The bottom panel of Fig shows the norm of the consensus vector z k av using S-ADMM. The trajectory of z k av does not contain any attack signature, and hence the malicious PDCs cannot be identified from it. Fig shows the estimations of σ i, 1 i 4 for the four inter-area modes after PDCs 2 and 3 are disconnected using RR-ADMM. The dashed lines show the actual values of σ i. The final values of the estimates of σ i in this case match with their true values again. Case 3:) RR-ADMM for detecting the data manipulators with random order. In this case, the central PDC chooses the order of the local PDCs are (1, 2, 4, 5, 3) for the first period, (3, 2, 5, 4, 1) in the second period, and (2, 5, 4, 1, 3) in the third period. α = 0.9. Fig

62 E Cut off the malicious PDCs 2 and 3 1 Figure 3.10: Values of the real parts of the four inter-area modes before and after detection using S-ADMM. True values of σ i are shown by dashed lines Figure 3.11: Evolutions of z k av and the first four elements of z k av when S-ADMM is run under attacks. 49

63 Figure 3.12: Evolutions of z k when RR-ADMM and S-ADMM are run under attacks. shows the trajectory of z k rr. z 1 rr = 0.767, z 2 rr = , z 3 rr = , z 4 rr = , and z 5 rr = z 1 rr is smaller than others in the first period, and α z 1 rr is equal to the data from the third local PDC. So in the second period, the central PDC assigns the estimate of the third PDC as the consensus variable, which is z 10 rr. The threshold is γ z = z 10 rr z 1 rr = z 2 rr > z 1 rr + γ z and z 5 rr > z 1 rr + γ z, and z 2 rr = αāa 2 rr,2 and z5 rr = αāa 5 rr,3. Hence, the PDCs 2 and 3 are identified as malicious, which matches the true situation. Case 4:) S- and RR-ADMM for detecting data manipulators with sparse and non-sparse biases. In this case, we test the sensitivity of S-ADMM and RR-ADMM with respect to the sparsity in the elements of the bias. We compare two scenarios. In both scenarios the biases chosen to be constant over time. We assume PDCs 2 and 3 are attacked. In the first scenario, all entries of 2 and 3 are zero except 2 (5) = 0.1 and 3 (5) = 0.2. In the second example, 2 = 0.1 2n 1 2n 1 and 3 = 0.2 2n 1 2n 1. Thus the norms of the respective bias vectors in the two examples are the same. Fig draws the trajectories of āa k av, j, 1 k 5, using S-ADMM in these two examples. The top figure shows the first scenario, i.e., the case when the bias vector is sparse, while the bottom figure shows the case for non-sparse bias. For both examples, S-ADMM can catch the attacked PDCs 2 and 3 successfully by separating the estimates into 50

64 E Cut off the malicious PDCs 2 and 3 7 Figure 3.13: Values of the real parts of the four inter-area modes before and after detection with RR-ADMM. True values of σ i are shown by dashed lines Figure 3.14: z k rr with random order of RR-ADMM. 51

65 40 with sparse with non-sparse Figure 3.15: The response of āa k av, j with sparse biases and non-sparse biases using S-ADMM. three correct groups, namely PDCs (1, 4, 5) in one group and PDCs (2, 3) in other two groups, respectively. Fig. 3.16, on the other hand, shows the trajectories of z k rr using RR-ADMM for these two examples. The top figures shows the scenario with sparse bias. In the first period, z 1 rr = , z 2 rr = , z 3 rr = 4.534, z 4 rr = , and z 5 rr = z 1 rr is minimum, and z 6 rr = Hence, the threshold is γ z = z 6 rr z 1 rr = z k rr < γ z + z 1 rr, 1 k 5. Therefore, according to Algorithm 2, none of the PDCs are detected to be malicious. When the bias is non-sparse, however, RR-ADMM identifies the malicious PDCs successfully, as shown in the bottom figure. This example shows that S-ADMM is fairly insensitive to the sparsity pattern in the elements of the bias vector since it compares the estimate of every individual local PDC. RR-ADMM, on the other hand, may show false positives when the attack vector is sparse since it relies only on the consensus vector for detection. Finally, Fig shows the trajectories of z k av computed using S-ADMM for both sparse and non-sparse cases. Neither of these trajectories has any signature by which PDCs 2 and 3 can be identified to be malicious. This, again, clearly shows that if the central PDC wants to track only the consensus variable for detection, then S-ADMM is of no use, and it must resort to RR-ADMM. Case 5:) S-ADMM for detecting data manipulators with small biases. 52

66 with sparse with non-sparse Figure 3.16: The response of z k rr with sparse biases and non-sparse biases using RR-ADMM. 40 with non-sparse with sparse Figure 3.17: The response of z k av with sparse biases and non-sparse biases using S-ADMM. 53

67 Figure 3.18: Evolutions of āa k av, j with multiple biases and different values of ρ using S-ADMM. Let the second and the third PDCs be respectively injected with k 2 = n 1 and k 3 = n 1. Fig shows the norm of the vector āa k av,i, 1 i N with different values of ρ. In the top figure of Fig. 3.18, with ρ = 10 6 as before, at the second iteration, āa 2 av,1 = , āa2 av,2 = , āa 2 av,3 = , āa2 av,4 = , āa2 av,5 = The threshold γ2 a = āa 2 av is separated into four groups: āa 2 av,1 and āa2 av,4 are in the first group, āa2 av,2, āa2 av,3, and āa2 av,5 are in the second, third and fourth groups, respectively. The minimum value āa 2 av,4 is in the first group. Therefore, PDCs 2, 3, and 5 are treated as malicious which is a false positive. The bottom figure of Fig depicts the norm of āa k av, j with ρ reduced to At the second iteration, āa 2 av,1 = , āa2 av,2 = , āa2 av,3 = , āa 2 av,4 = , āa2 av,5 = The threshold γ2 a = āa 2 av is separated into three groups: āa 2 av,1, āa2 av,4, and āa2 av,5 are in the first group, āa2 av,2 and āa2 av,3 are in the second and third groups, respectively. āa 2 av,4 in the first group is minimum. Therefore, PDCs 1, 4, and 5 are non-malicious which matches the true situation. Top figure of Fig shows the estimations of σ i, 1 i 4, for the four inter-area modes. The dashed lines show the actual values of σ i. The final values of the estimates of σ i in this case 54

68 1 E E Figure 3.19: Values of the real parts of the four estimated inter-area modes before and after detection using S-ADMM with small biases. True values of σ i are shown by dashed lines. do not match their actual values, due to the undetected bias. The estimates are divergent, but due to the specific choice of, the rate of divergence is slow, and, hence, not very visible. The bottom figure of Fig shows the estimations of σ i, 1 i 4, for the four inter-area modes. After the malicious PDCs 2 and 3 are cut off, the final values of the estimates of σ i match their true values again. 3.6 Conclusion In this paper we developed a set of algorithms for revealing the identities of malicious attackers in power system mode estimation loops. The attack is modeled in terms of data manipulations in the messages passed between distributed estimators. The need for a distributed architecture arises primarily from the computational bottlenecks of this problem. For example, a single centralized estimator may take an enormous amount of time if it has to process hundreds of PMU measurements. Dividing the computational burden among multiple estimators soothes this complexity. The upshot, however, is that increasing the 55

69 number of estimators also means increasing the number of communication links for information exchange, and therefore, increasing the risk of cyber-attacks in these channels. Moreover, in the centralized case, the entire estimation will fail if the estimator is attacked. In the distributed case, however, even if any of the estimators are attacked, the estimation can still be sustained using the proposed detection algorithms. The speed of convergence may suffer due to PDCs being cut off, but the accuracy of the estimation will still be guaranteed following the consensus nature of the problem. The specific application presented here is widearea oscillation estimation, although the same technique can be used for other applications such as distributed dispatch, optimal power flow, and distributed control problems as well. Future work in this area will include generalizing the detection and correction procedures to more specialized attack models such as denial-of-service [91], jamming attacks on GPS signals [92] and [93], and eavesdropping. 56

70 Chapter 4 Identifying Data-Manipulators with Noisy Measurements 4.1 Problem Formulation for Attack Identification The transmission of messages between the local phasor data concentrators (PDC)s and the central PDC will happen through an unsecure wide-area communication network. Thus, it is easy for attackers to hack into the PDCs, and corrupt the value of the estimates. Similar to the Chapter 3, if some of the local estimators are compromised by malicious attackers, they may send corrupted values of their estimates to the central PDC. Because the central PDC does not know that the message is corrupted, it will still calculate the consensus variable by averaging the data from individual local PDC as in (3.1). Notice that the bias k j may be of arbitrary magnitude and sign, and may be time-varying. When k j is injected then the trajectories of the local estimates as well as that of the consensus variable will diverge unless k j is chosen in a special way so that its entries corresponding to the consensus modes are exactly zeros. In any case, the attacker would benefit most if a k i diverges, implying that she has been able to destabilize the estimation loop. Detecting the identity of the corrupted PDC is, therefore, crucial to retain normal operation of the loop. Our goal is to develop an algorithm by which the central PDC can infer which local PDCs may be malicious by simply tracking the consensus variable z k. In Chapter 3, we used standard 57

71 alternating direction multiplier method (S-ADMM) to develop such an algorithm assuming y(t) to be noiseless. In this Chapter, we want to address the same objective when y(t) is noisy. In that situation, localization becomes more difficult as the central PDC may not be able to distinguish the bias from the noise, especially if the bias is small, and particularly for the first several iterations depending on the initial guess for the estimates. Therefore, we derive a new algorithm based on total least square ADMM (TLS-ADMM) but by replacing the averaging step by a deterministic Round-Robbin (RR) operation as follows. 4.2 Proposed Round-Robbin ADMM The purpose of RR-ADMM is only to identify the malicious local PDCs, not for obtaining the optimal solution of (2.18). The algorithm is run by the central PDC stealthily, while every local PDC still believes that the central PDC is using TLS-ADMM to calculate z k, and thereby updates a k+1 j in the same way as in (2.19b). We define ãa k i to be the value of a k i received at the central PDC at iteration k. If the i th PDC is attacked, then ãa k i = a k i + k i ; otherwise, ãak i = a k i. The basic strategy for RR-ADMM is the same as the one in Chapter 3. In general, z k can be written as in. N successive iterations constitute one period of RR-ADMM where N is the total number of local PDCs. After N iterations, the central PDC will again start from PDC 1, then PDC 2, and so on. Notice that the order of choosing the PDCs for the averaging step in the RR-ADMM need not be fixed. The identification should succeed with any random order as long as every PDC is visited exactly once in a period. We denote the consensus variables of RR-ADMM and TLS-ADMM at iteration k as z k rr and z k av, the latter being updated in (2.19d). Similarly, the local PDC estimates and their dual variables will be denoted as ε k rr, j and εk av, j as in (2.19a), ak rr, j and ak av, j as in (2.19b), w k rr, j and wk av, j as in (2.19e), and uk rr, j and uk av, j as in (2.19c), 1 j N, respectively. From (4.2), it is clear that unlike TLS-ADMM where z k av only depends on the average value of the biases, the expression for z k rr is influenced by k ((k 1) mod N)+1 which is the bias of the [((k 1) mod N) + 1]th PDC. This provides a potential signature to catch the identities of the PDCs with biases. In addition, since the updates of the dual parameter w k rr,i are the same as in Chapter 3, if 1 k N and i = k, the difference 58

72 w k rr,i wk 1 rr,i can be written as: w k rr,k wk 1 rr,k = ρ 1(a k rr,k ak rr,k k k ) = ρ 1 k k. (4.1) Therefore, if k k = 0, then wk rr,k wk 1 rr,k = 0. The central PDC can use Algorithm 5 in Chapter 3 to detect the identities of the malicious users with noisy measurements. However, if the attacker is smart enough to know the value of ρ 1, she can counterfeit the value of w k rr,k to fool the central PDC, leading to w k rr,k = wk k + ρ 1 k k. Then w k rr,k wk 1 rr,k = wk k + ρ 1 k k wk 1 rr,k = 0. (4.2) Thus, the central PDC will not be able to identify the malicious users using the above method. To bypass this difficulty we propose the following. In the first period, the central PDC searches for the minimum value of z k rr, 1 k N, say, referred to as z k min rr, and then compares z k min+n rr to z k rr, 1 k N. From (4.2) and (2.19b) z k min+n rr can be expressed as z k min+n rr = f 1 (ε k min+n rr,k min ) = ãa k min+n rr,k min [ = a k min+n rr,k min ρ 1 z k min+n 1 rr w 0 rr,k min ρ 1 k min+n 1 j=1 a j k min +N 1 ] rr,k min + ρ 1 z i rr + f 2 (ε k min+n rr,k min ), (4.3) i=1 where the expressions for f 1 ( ) and f 2 ( ) are given by f 1 (ε k min+n rr,k min ) = [ρ 1 I + ρ(h kmin P H ε k min+n rr,k min ) T (H kmin P H ε k min+n rr,k min ))] 1 (4.4) ) u k min+n + ρ c kmin ρ P Cl ε k min+n rr,k min P Cr (4.5) ( f 2 (ε k min+n rr,k min ) = (H kmin P H ε k min+n rr,k min ) T k min 59

73 Say, the p th local PDC is attacked. From (4.2) and (2.19b), the variable z p rr can be expressed as zrr p = ãa rr,p p = arr,p p + p p [ = f 1 (εrr,p) p ρ 1 zrr p 1 w 0 rr,p ρ 1 p 1 j=1 arr,p j p 1 ] + ρ 1 z i rr + f 2 (εrr,p) p + p p. (4.6) i=1 Thus the difference between the part in the bracket of (4.3) and the one of (4.6) is ρ 1 z k min+n 1 rr w 0 rr,k min + w 0 rr,p ρ 1 k min +N 1 j=1 a j rr,k min +ρ 1 p 1 j=1 a j rr,p +ρ 1 k min +N 1 i=p ρ 1 zrr p 1 z i rr + f 2 (ε k min+n rr,k min ) f 2 (εrr,p). p If k min +N 1 k min +N 1 is much larger than p, then the part ρ 1 z i rr becomes larger. In [90], we conjectured that the k th i=p PDC is identified as malicious if z k rr > z k min+n rr, 1 k N. Thus, p p should be large enough to satisfy zrr p > z k min+n rr for every attacked PDC p, and at the same time the difference between the part in the bracket of (4.3) and the one of (4.6) should not be too large to affect the difference z k rr z k min+n rr much. [ However, if p p is small, i.e., the attack magnitude is covert, then the value of f 1 (ε k min+n rr,k min ) ρ 1 z k min+n 1 rr ] [ w 0 k min +N 1 rr,k min ρ 1 a j k min +N 1 rr,k min +ρ 1 z i rr + f 2 (ε k min+n rr,k min ) may become comparable to f 1 (εrr,p) p ρ 1 zrr p 1 j=1 i=1 ] w 0 p 1 rr,p ρ 1 arr,p j p 1 + ρ 1 z i rr + f 2 (εrr,p) p + p p, thereby leading to incorrect classification. Next we provide a more efficient approach with the assistance of local TLS estimates combined with the j=1 i=1 RR-ADMM approach by which the central PDC may be able to reduce false alarms in these situations RR-ADMM with Local TLS Estimates Next, assume that each local PDC were running a TLS using only their own local measurements, independent of each other. We show that these decoupled TLS estimates can improve the efficiency of RR-ADMM for detecting covert attacks. The decoupled estimates can be generated by running TLS for just once in a batch fashion at every local PDC using their local H i and c i matrices only. We briefly recall the process as follows [81]. Step 1. Define an extended matrix E i = [ H i, c i ]. 60

74 Step 2. Compute the singular value decomposition (SVD) of E i as [ E i = ] U i,2n U i,1 Σ i,2n λ i,2n+1 2n 2n v 2n 1 v 1 2n v 1 1, (4.7) where Σ i,2n = diag(λ i,1,λ i,2,...,λ i,2n ) which satisfies λ i,1 λ i,2... λ i,2n+1. Step 3. The TLS estimate of the i th local PDC is a T LS,i = v 2n 1 v (4.8) Since the effect of the noise is reduced by the SVD, a T LS,i is generally very close to a T LS, j for any pair of PDCs i and j, 1 i, j N,i j. We assume the number of malicious attackers is less than N 2, where N 2 means the largest integer less than or equal to N 2. The steps of the detection algorithm can then be stated as follows. First, every PDC waits till a certain prescribed number of sampling times to gather enough data to construct their H i and c i matrices. Once constructed, we assume that these matrices remain constant for the rest of the detection process. Then at the first iteration, every local PDC calculates its TLS estimate a T LS,i, 1 i N, as in (4.8), and sends it to the central PDC. At iteration k, the central PDC calculates the distances ( di k 1 = 2n ãa k rr,i (p) a T LS,i (p)), 1 i N. Let these distances be arranged in descending order. If 2n p=1 d k i is large, i.e., if it appears to be an outlier compared to the computed values of the other distances, then the central PDC suspects that the i th local PDC may be malicious. Let the indices of the local PDCs whose distances are in the first N 2 of the descending order of d k i be in a set S. Then the central PDC rearranges the order of RR-ADMM by inter-twining the maximum and minimum ordered absolute values of the distances as follows. d k min At iteration k +1, the consensus variable z k+1 rr is minimum. At iteration k +2, the consensus variable z k+2 rr is equal to the one from the local PDC whose distance is equal to the one from the local PDC whose distance 61

75 d k max is maximum. At iteration k +3, the consensus variable z k+3 rr d k min 2 is second minimum. At iteration k +4, the consensus variable z k+4 rr d k max 2 is second maximum, and so on, until one period is complete. is equal to the one from the local PDC whose distance is equal to the one from the local PDC whose distance After that the central PDC will compare the distance between z k rr which is equal to the data from the local PDC whose index is in the set of S and z k 1 rr, to the distance between zrr k 1 and z k+1 rr. If z k rr, 2 k N, is equal to the data from attacked local PDC i and z k 1 rr is from non-attacked local PDC j, 1 i, j N, i j, leading to z k rr = a k rr,i + k i and z k 1 rr written as z k rr = a k rr,i + k i + f 1 (ε k rr,i) [ ρ 1 z k 1 rr w 0 rr,i ρ 1 k 1 j=1 = arr, k 1 j, then zk rr and z k 1 rr can be respectively a j rr,i + ρ 1 k 1 i=1 ] z i rr + f 2 (εrr,i) k + k i, (4.9) z k 1 rr [ = a k 1 rr, j = f 1 (εrr, k 1 j ) ρ 1 z k 2 rr w 0 rr, j ρ 1 k 2 j=1 a j rr, j + ρ 1 k 2 i=1 z i rr + f 2 (ε k 1 rr, j ) ]. (4.10) The difference between the part in the bracket of (4.9) and the one of (4.10) is ρ 1 z k 1 w 0 rr, j ρ 1 k 1 a j rr,i +ρ 1 j=1 rr ρ 1 z k 2 rr w 0 rr,i + k 2 a j rr, j +ρ 1z k 1 + f 2 (εrr,i k ) f 2(εrr, k 1 j ). The part ρ 1z k 1 is much less than the corre- j=1 k min +N 1 sponding part of the difference between (4.3) and (4.6), ρ 1 z i rr. This implies that combining the decoupled TLS estimates with RR-ADMM helps in reducing the effect of the bias in the difference i=p between the bracketed parts of (4.9) and (4.10). In other words, the difference between f 1 (εrr,i [ρ k ) 1 zrr k 1 w 0 rr,i k 1 ρ 1 a j rr,i +ρ k 1 1 j=1 i=1 is still significant enough despite k i z k rr and z k+1 rr ] [ z i rr + f 2 (εrr,i k ) + k i and f 1 (εrr, k 1 j ) ρ 1 zrr k 2 w 0 rr, j ρ 1 k 2 j=1 a j rr, j +ρ 1 k 2 i=1 ] z i rr + f 2 (εrr, k 1 j ) being a small number. The result holds for the difference between too. Thus rearranging the order of RR-ADMM based on the decoupled TLS estimates can amplify the attack signature, and help in identification. The central PDC can identify the i th local PDC as malicious if z k rr z k 1 rr > z rr k 1 z k+1 rr. Algorithm 5 summarizes this entire RR-ADMM strategy combined with decoupled TLS estimation. 62

76 Algorithm 5 Identifying malicious PDCs using RR-ADMM 1) At the first iteration, local PDC i calculates its own TLS estimate a T LS,i, 1 i N, and sends it to the central PDC. 2) For the first few iterations, 1 k k, central PDC computes message z k av as in (2.19d). If z k av, 1 k k, is divergent, then it suspects that there exists one or more malicious users. 3) At iteration k, every local PDC updates a k av in (2.19b), and send it to the central PDC. 4) The central PDC calculates the distances di k, 1 i N, arrays the distances in descending order, builds up the set S, and calculates z k rr using the listed in the four bullets on the top of the left column of this page. 5) After N + 1 iterations, the central PDC calculates the differences z k rr zrr k 1 and z k+1 rr z k 1 rr, where z k rr satisfies z k rr = ãa k rr,i, i S. If z k rr zrr k 1 > z rr k+1 zrr k 1, then the central PDC identifies the i th local PDC is malicious. 6) The central PDC repeats this classification for a few iteration, say up to iteration s. If the identified malicious PDCs are consistent through these iterations, it finally confirms these PDCs are malicious. 7) Onwards from iteration s + k + 1, the central PDC ignores any message coming from the malicious PDC, and simply carries out TLS-ADMM with the remaining non-malicious PDCs using (2.19). The final solution of this TLS-ADMM will lead to the solution of (2.18) as the LS problem is convex with s + k + 1 being an initial iteration for the rest of the non-malicious TLS-ADMM. We close this section by illustrating Algorithm 5 on an example. Consider five local PDCs, where the second and third PDCs are malicious. Fig. 4.1 depicts the trajectory of the average value of the vectors z k pre and post detection using RR-ADMM. At the first iteration, ãa 1 1 = 0.025, ãa 1 2 = , ãa 1 3 = , ãa 1 4 = , and ãa 1 5 = , where ãa 1 i = 1 2n 2n ãa 1 i (p). The TLS estimates from the p=1 local PDCs are as follows: āa T LS,1 = , āa T LS,2 = , āa T LS,3 = , āa T LS,4 = , and āa T LS,5 = , where āa T LS,i = 1 2n 2n a T LS,i (p). Thus, the set S = {2,3}. Using the proposed p=1 rearranging method, the order of RR-ADMM is fixed to be 1,2,5,3,4. Hence, at the second iteration we get z 2 rr = ãa 2 1 ; at the third iteration, z3 rr = ãa 3 2 ; at the fourth iteration, z4 rr = ãa 4 5 ; at the fifth iteration, z 5 rr = ãa 5 3 ; at the sixth iteration, z6 rr = ãa 6 4 ; and at the seventh iteration, z7 rr = ãa 7 1 again. In Fig. 4.1, we see that z 3 rr z 2 rr > z 2 rr z 4 rr and z 5 rr z 4 rr > z 4 rr z 6 rr, which means PDCs 2 and 3 are malicious. After iteration 7, the central PDC cuts off the signal from the malicious PDCs, and calculates the consensus vector using TLS-ADMM. Notice that at iteration 7, a 7 j is reinitialized at the value based on z 6 rr. In addition, the results of Fig. 4.1 show that after disconnecting the malicious node, the consensus variable of TLS-ADMM converges to the ideal solution z, as expected. If the central PDC uses the successive order instead of changing the order based on TLS estimates, Fig. 4.2 shows 63

77 PDCs 2 and 3 are detected as malicious Cut off PDCs 2 and 3 Continue to calculate z k av k = a av,1 + a av,4 + a av,5 3 k k z av ҧ = Figure 4.1: Behavior of z k pre and post detection of attack using RR-ADMM. the trajectory of the average value of the consensus variable z k rr. The detection algorithm is followed by the one provided in [90]. In the first period, z 1 rr is minimum. Thus for 1 k 5, z 3 rr > z 6 rr and z 5 rr > z 6 rr. So the PDCs 3 and 5 are identified as malicious which does not match the true situation. This clearly shows how the proposed sorting method can help in reducing false detections. 4.3 Simulation results We verify our proposed localization algorithm using the IEEE 68-bus power system model. The power system is divided into 5 non-overlapping areas based on their coherency properties, each with one local PDC and 3 PMUs. For details of this partitioning please see [82]. The simulated measurements are obtained using the Power System Toolbox (PST) nonlinear dynamics simulation routine s simu and the data file data16m.m. The top figure of Fig. 4.3 shows the average value of z k av using TLS-ADMM. The trajectory of z k av does not contain any attack signature, and hence the malicious PDCs cannot be identified from it. The 64

78 One period Figure 4.2: Behavior of z k rr with successive order. Average value of zav Average value of zrr Iteration (k) Iteration (k) 7z av 7z rr Figure 4.3: Evolutions of average values of z when TLS-ADMM and RR-ADMM are run under attacks. bottom figure, on the other hand, shows the average value of z k rr using RR-ADMM. At iteration 15, the central PDC calculates d 15 i, which satisfies d 15 3 > d15 2 > d15 5 > d15 1 > d15 4. Thus the order of RR-ADMM 65

79 Cut off the malicious PDCs 2 and 3 Figure 4.4: Values of the real parts of the four inter-area modes before and after detection with RR-ADMM. starting at iteration 16 is (4,3,1,2,5). In other words, we apply z 16 rr = ãa 16 4, z17 rr = ãa 17 3, z18 rr = ãa 18 1, z19 rr = ãa 19 2, z 20 rr = ãa 20, and z21 rr = ãa 21 again. The set S is identified as S = {2,3}. Also, we see the following two 5 inequalities to hold: 4 z 17 rr z 16 rr > z 16 rr z 18 rr, (4.11) z 19 rr z 18 rr > z 18 rr z 20 rr. (4.12) Thus both PDCs 2 and 3 are identified as malicious, which matches the true condition. After identifying PDCs 2 and 3 as malicious, as shown in the bottom figure of Fig. 4.3, the central PDC disconnects these PDCs at the 21 st iteration, and calculates the solution using TLS-ADMM with only non-malicious PDCs. Fig. 4.4 shows the estimation of σ i, 1 i 4, for the four inter-area modes. The dashed lines show the actual values of σ i. The final values of the estimates of σ i match their actual values. 4.4 Conclusions In this paper, we developed an optimization algorithm to detect the identities of malicious users in distributed optimization loops in power systems. We illustrated our results using RR-ADMM with noisy 66

80 measurements and multiple biases. Results show that the central coordinator can catch data manipulators using RR-ADMM only by tracking the consensus variable with a prescribed ordering scheme of roundrobbin based on local TLS estimates. Our future work in this area will include generalizing the detection and correction procedures to more specialized attack models such as denial-of-service, jamming attacks on GPS, all in the context of total least-squares using noisy measurements. 67

81 Chapter 5 Identifying Data Manipulators in Distributed Wide-Area Control Loops of Power Systems In this chapter we address this problem in the context of identifying malicious data-manipulators in widearea control loops of power system. The fundamental set-up for the controller is based on minimizing a LQR objective function. The power system is divided into multiple non-overlapping areas, each equipped with a phasor data concentrator (PDC). These PDCs collect the measurements of voltage, phase angle, and frequency from phasor measurement units (PMUs) and send the data to virtual computing agents, referred to as virtual machines (VMs), in a cloud computing network. The VMs can communicate to each other. After receiving the data from all the other VMs, each VM calculates the optimal control input by minimizing the LQR objective function and sends them back to the respective generators for actuation. However, if one or more of the VMs are faulted or manipulated by attackers, then the resulting control inputs will be inaccurate, which, in turn, will contaminate the accuracy of every PDC. To combat this, a detector is designed to collect the state information from PDCs and VMs. It requires that the individual VMs send all the states to the detector. Compared to the data from PDCs, the detector can identify the attacked VMs. However, if the attacker is smart, he or she can perceive the detection period 68

82 due to different actions of VMs, and then sends the correct data to puzzle the detector and conceal his or her identity. We provide a Round-Robin (RR) technique based on sparse feedback gains to show how the detector catches the malicious VMs by simply tracking the magnitude of every control input covertly. In the normal case, the system uses LQR controller for minimizing the cost. When the attacker accesses the network, the firewall [96] or intrusion detection system (IDS) [97] can detect the intruder and broadcast the alarm. The system changes to the sparse controller with RR technique. At this time the attacker just accesses the network, but not intrudes the VMs. Each VM keeps the same calculation of the inputs after the alarm. So the attacker believes that the system always uses the RR sparse controller to calculate the inputs and cannot perceive the detection period. We illustrate effectiveness of the attack localization algorithms using simulation results on a IEEE 68-bus power system model. 5.1 Problem Formulation for Attack Identification Power System Model Consider a power system with b buses. Without loss of generality, classify the first m buses to be generator buses, and the remaining (b m) buses as load buses. The voltage at the i th bus is denoted as Ṽ i = V i θ i where V i is the magnitude (volts) and θ i is the phase (radians). The internal voltage phasor of a synchronous generator connected to any generator bus is denoted as Ẽ i = E i δ i, i = 1,2,...,m. Each synchronous generator may be modeled as δ i = ω i ω s (5.1) M i ω i = P mi D i (ω i ω s ) P G i (5.2) τ i Ė i = x di x di P G i Q G i E i + x di x di x di V i cos(δ i θ i ) + Ẽ Fi (5.3) ( x di x qi = E iv i x di sin(δ i θ i ) + 2x qi x di = E iv i x di cos(δ i θ i ) x di x qi 2x qi x di ) V 2 i sin(2(δ i θ i )) (5.4) ( ) 1 cos(2(δ i θ i )) Vi 2 (5.5) 69

83 where, the states δ i, ω i, and E i are, respectively, the generator phase angle (radians), rotor velocity (rad/sec), and the quadrature-axis internal emf; ω s is the synchronous frequency or 120π radian/sec; P G i and Q G i are, respectively, the active and reactive power produced by the i th generator, M i is the generator inertia, D i is the generator damping, P mi is the mechanical power input; τ i is the excitation time constant; x di, x di, and x qi are the direct-axis salient reactance, direct-axis transient reactance, and quadrature-axis salient reactance (all in ohms), respectively. The control variable is the field voltage Ẽ Fi, which can be split as Ẽ Fi = Ē Fi + E Fi (5.6) where, the first term is a constant that fixes the equilibrium value, and the second is a designable control input. Following [85] it can be shown that the linearized small-signal dynamic model of the networked power system can be expressed as δ ω ĖE = 0 I 0 M 1 L M 1 D M 1 P T 1 K 0 T 1 J δ ω E I E F, (5.7) where δ = col( δ 1,..., δ m ), ω = col( ω 1,..., ω m ), E = col( E 1,..., E m ), E F = col( E F1,..., E Fm ), M = diag(m 1,...,M m ), T = diag(τ 1,...,τ m ), and the expressions for the various matrices on the RHS can be found in [85]. We consider the mechanical power input to be constant. Next, we consider the network to be divided into n areas, n m, the indices of the generators in the different areas are consider to be as follows. Area 1: {1,2,...,m 1 }, Area 2: {m 1 + 1,...,m 1 + m 2 },..., Area n: { n 1 m g + 1,...,m}. Area i has m i generators with m i controllers and N i = 3m i states, 1 i n. g=1 All states are assumed to be measured by placing PMUs at the generator terminal buses. We consider that every area has a local control center where a PDC, which is simply a data aggregator, collects PMU measurements streaming from the generators located inside that area. It should be noted that the states in (5.7) are stacked as ( δ, ω, E). Rearranging them in tuples of ( δ i, ω i, E i ) for the i th generator and 70

84 then sorting them in terms of the areas, the discrete-time dynamical model of the network is written as: x 1 (t + 1) x 2 (t + 1). x n (t + 1) A 11 A A 1n x 1 (t) A 21 A A 2n x 2 (t) =..... A n1 A n2... A nn x n (t) }{{}}{{} + B 1 A OL B 2... B n } {{ } B u 1 u 2. u n x(t) }{{} u(t) = (A OL + BK)x(t), (5.8) where for area i: x i (t) = [(x states, u i (t) = [(u 1+ i 1 m j j=1 i 1+ i 1 m j j=1 i (t)) T,...,(u (t)) T,...,(x i m j j=1 i i j=1 i m j (t)) T ] T = [x 1 i (t),..., xn i i (t)] T R N i 1 is the vector of (t)) T ] T is the vector of control inputs, and u 1 (t) u 2 (t). u n (t) = K 11 K K 1n K 21 K K 2n K n1 K n2... K nn x 1 (t) x 2 (t). x n (t). (5.9) Here, each block K i j represents the block of feedback gains from the PMUs in area j to the controllers in area i. Thus, the diagonal and off-diagonal blocks represent intra-area and inter-area feedbacks, respectively. K i = [K i1, K i2,..., K in ] is the feedback gains in area i. We assume the open-loop system to be stable, i.e., ρ(a OL ) < 1 where the function ρ( ) denotes spectral radius of a square matrix. Consider that the control inputs are designed by linear state-feedback u(t) = Kx(t) to minimize the LQR objective 71

85 function J = T t=0 (x T (t) Q x(t) + u T (t) R u(t)), (5.10) where Q = Q T 0 and R = R T > 0 are given design matrices. The gains satisfy Algebraic Riccati Equation (ARE); for closed-loop stability they must satisfy ρ(a CL ) < 1, where A CL := A OL + BK. (5.11) Let the value of the closed-loop J in that case be denoted as J LQR, and the optimal feedback gains of the closed-loop system be called LQR feedback gains Data updated by the individual PDCs The model for information updated from these n PDCs is simple: at any time instant t, PDC i, 1 i n, collects its own state x i (t) from generators at area i and sends it to VM i. VM i sends the state to its neighbor VMs. We assume that the VMs are strongly connected. Once the i th VM has received the values of the states from other VMs, it computes u i (t) = K i x(t) in which K i is the optimal LQR feedback gain in area i. Then the VM i sends the feedback data u i (t) to the generators at area i. Finally, generators at area i update the state as x i (t + 1) = A i x i (t) + B i u i (t) and send x i (t + 1) to PDC i When the VMs at one or multiple areas are faulted or attacked Next, assume that the VMs at some areas are faulted or attacked - meaning, they send incorrect values of their states to the other VMs, and send either incorrect or correct feedback data to the generators at the corresponding areas. We denote the measurements x i (t), 1 i n, sent from VM i at instance t as x i (t) = [ x 1 i (t), x2 i (t),..., xn i i (t)] T. We define a set S which includes the indices of faulted VMs. If i S, x i (t) = x i (t) + i (t); otherwise, x i (t) = x i (t), i / S. If the attacked VMs use the same incorrect data to 72

86 calculate the inputs as the ones sending to their neighbors, the control inputs are u l (t) = which means that the closed-loop dynamics will be n K l p x p (t),1 l n, (5.12) p=1 and x l (t + 1) = n q=1 n n A lq x q (t) + B l u l (t) = A lq x q (t) + B l K l p x p (t),1 l n, (5.13) q=1 p=1 x(t + 1) = A OL x(t) + BK x(t) = (A OL + BK)x(t) + BK (t), (5.14) where (t) = [ T 1 (t), T 2 (t),..., T n ] T (t). If i / S, i (t) = 0 Ni 1, 1 i n. If the attacked VMs send correct inputs to their corresponding generators, the updates in (5.14) can be rewritten as x(t + 1) = A OL x(t) + BK( x(t) + β(t)) = (A OL + BK)x(t) + BK( (t) + β(t)). (5.15) where β(t) = [β T 1 (t), β T 2 (t),..., β T n (t)]t. β l (t) = 0, l / S. In general, Eq. (5.14) and Eq. (5.15) are equivalent. Since the value of (t) can be arbitrarily large, the closed-loop states in (5.14) are no longer guaranteed to convergence or be convergent to zero. There exists a nature question that how to detect the identities of the faulted VMs. The possible solution in measurement-based model is provided in the next section. The inherent assumption is that the PDCs only collect the states, and therefore, do not know the model parameters A OL and B. They only has access to the updated states and inputs, and, therefore, must algorithmically figure out the identification mechanism based on these two variables only. This is the main contribution of Chapter 5. If the attacks intrude the VMs, they have to access into the network first. The firewall [96] or the intrusion detection system (IDS) [97] can detect the intrusion. The time-line for executing the attack localization algorithms is shown in Fig

87 The attackers intrude the VMs. The detector identify the attacked VMs t=t0 t=t1 t=t2 t=t3 t=t4 Alarm from firewall or IDS. The detector is added. Each PDC runs the attack localization algorithms. Attacked VMs are removed. Non-malicious VMs volunteer to take up the task. Figure 5.1: Timing diagram for the communication protocol among the firewall, PDCs, and VMs. 5.2 Identify the faulted or attacked VMs To identify the faulted or attacked VMs, we consider the following set up. As soon as the instability due to the data manipulation is detected from the divergence of the generator frequency measurements, the system operator triggers a computing node, which we refer to as a detector, and enables a secure private communication network between this detector and the PDCs. This secure communication network is only reserved for emergency conditions like this (since the cost of this network may be very high owing to its high security), and is not used during normal conditions when the usual cloud-in-the-loop control is working. All the PDCs and VMs will send their data to the detector. The left figure of Fig. 5.2 shows the communication between two PDCs and two VMs without attack, at sampling instant t for 1 i n: (1) PDC i sends x i (t) to VM i. (2) VM i sends its own state x i (t) to its neighbors. (3) After receiving all the states, VM i computes u i (t) = K i x(t), where K i is the LQR controller at area i, and sends u i (t) back to the generators at area i. (4) The generators at area i updates the state x i (t + 1) = A i x(t) + B i u i (t), and send it to PDC i. The right figure of Fig. 5.2 shows the communication among the PDCs, VMs, and the detector with attack, at sampling instant t for 1 i n: (1) PDC i sends x i (t) to VM i and the detector. (2) VM i sends its own state x i (t), 1 i n, to its neighbors; If VM i is attacked or faulted, x i (t) = x i (t) + i (t). (3) After receiving all the states, VM i sends all the states x(t) back to the detector. (4) The generators at area i update the state as x i (t + 1) = A i x(t), and send it to PDC i. 74

88 Without attack Area 1 Area 2 With attack Area 1 Area 2 u m1 (t) 1 G1 G3 u m3 (t) 2 G1 G3 u m2 1 (t) G2 G4 u m4 (t) 2 G2 G4 m1 x (t) 1 x1 m4 x m3 m2 (t) (t) 2 x (t) 2 m1 x (t) 1 m2 x m4 x m3 1 (t) (t) 2 x (t) 2 PDC 1 PDC 2 x (t) 1 x (t) 2 x (t) 2 VM 1 VM 2 x (t) 1 Internet of clouds x1(t) x2 (t) (t) PDC 1 PDC 2 x (t) 1 x (t) 1 Detector x (t) 2 x2 (t) ( t) VM 1 VM 2 x (t) 1 Internet of clouds x (t) 2 x 1 (t) x1(t) x2 (t) (t) Figure 5.2: The communication between two PDCs and two VMs with and without attack. Note that the attacked VMs can send either correct or incorrect data to the detector. Algorithm 6 illustrates how to identify the faulted VMs. There is no limitation on the number of the attacked VMs for Algorithm 6 Identifying the faulted VMs (1) PDC i sends x i (t) to VM i and the detector, 1 i n. (2) VM i sends its own state x i (t) to its neighbors. If VM i is attacked or faulted, x i (t) = x i (t) + i (t). (3) After receiving all the states, VM i sends all the states x(t) back to the detector. (4) The detector compares the data received from PDCs and VMs. If x i (t) x i (t) and if x i (t) is sent from VM j, 1 j n, j i, then the i th VM is identified as malicious. (5) The generators at area i update the state as x i (t + 1) = A i x(t), and send it to PDC i. the identification using Algorithm 6. The assumption in Algorithm 6 is that each malicious VM should send incorrect state to its neighbors. In the fault-free case VMs send the control inputs to the generators; while in the fault case VMs send all the states to the detector for the identification. VMs change the calculation of the control inputs and the receiver. That implies that if the attacker is smart, he or she can perceive the identification duration. Then the attacked VMs can send correct states to the neighbors for puzzling the detector and concealing the attackers identities. So there is a nature question: how to identify the attacked VMs covertly? In the 75

89 next section we will provide an algorithm based on Round Robin technique to solve the problem. 5.3 Identify the attacked VMs covertly If the actions of all the VMs during the detection period are kept the same with the normal case, the attacked VMs should still send incorrect states to their neighbors and incorrect or correct data to their corresponding generators. In this section, we will design the sparse controller based on the RR approach for the covert identification. Let a period be with n p sampling instants, where 1 p n 1. If p is closer to n 2, one period has more sampling instants; on the other hand, if p = 1 or p = n 1, one period has minimum n sampling instants. p denotes the number of zero submatrices in K i, 1 i n. Since there are n p combinations, in one period at different sampling instants the structures of RR feedback gains are different. The order of RR technique is not fixed. At each sampling instant, PDCs decide the positions of zero submatrices in K i at the next sampling instant, and send the information to its corresponding VMs. All the non-malicious VMs utilize the same order. For instance, if p = n 1, there are n sampling instants in one period. At sampling instant q, 1 q n, the feedback gain K q at area i should satisfy ρ(a OL + BK q ) < 1 K q i,1:q 1 = 0 K q i,q+1:end = 0, (5.16) In one period, the value of q should be from 1 to n. With different values of q, the structures of K q i are 76

90 different in one period. Due to the special structure of feedback gains (as in (5.16)), it satisfies K q x 1 (t) x 2 (t). x n (t) = K q 0. 0 x q (t) (5.17) It implies that at sampling instant q, the input u q (t) is only dependent on the state x q (t) from VM q. In general, at one sampling instant the Round Robin sparse feedback gains are dependent on the n p states RR controller in fault-free case We assume in the open-loop the power system is stable, i.e., ρ(a OL ) < 1. And for simplicity, we assume B is an identity matrix. The RR controller at each area is followed by a periodic function. In one period there are n p be written as sampling instants. Thus, the discrete-time dynamical model in one period of the network can where T = n p x(t + T + 1) = T q=1 (A OL + BK q )x(t), (5.18) and K q is the RR sparse controller at the sampling instant q. Theorem 1 If the power system in the open-loop is stable, there exists a RR sparse controller to make the closed-loop system stable. Proof: Since the open-loop system is stable, according to the perturbation theory [99], there exists perturbation matrix K which still guarantees that ρ(a OL + BK q ) < 1, 1 q T. Thus, the eigenvalues 77

91 of matrix T (A OL + BK q ) are in the unit cycle, which implies that the system is stable using RR sparse q=1 controller. In particular, the system uses the sparse LQR feedback gains for minimizing the cost: T K = min J = K (x T (t)qx(t) + u T (t)ru(t)). t=0 s.t. u(t) = Kx(t) K i j = 0, j p,1 i n (5.19) In general, with p increasing, the cost in the closed-loop system J as in (5.19) becomes larger sharply, even using sparse LQR controller with RR technique. Consider an example with five PDCs. Let the initial values of the states at each PDC are all equal to 0.1. Fig. 5.3 shows that the trajectories of the first elements of the states using RR sparse LQR controllers with different values of p, compared with the one using traditional LQR controller. The sparse LQR feedback gains are calculated using the approach in [98]. The states are convergent to zero with different values of p, which implies that the system is stable using RR sparse LQR feedback gains. With p increasing, the states are dynamic more heavily. Table 5.1 lists the cost J as in (5.19) using RR sparse LQR controllers with different values of p. The cost using RR sparse LQR feedback gains is much larger than the one using traditional LQR controller. Therefore, if the system always uses the RR sparse controller in the false-free case, it will take a large number of additional expenses. Fortunately, if the attacker intrudes the VMs, she or he has to access into the network first and the firewall [96] or the intrusion detection system (IDS) [97] can detect the intrusion. After receiving the alarm from firewall or IDS, each VM changes to RR controller for the covert identification. The time-line for executing RR controller and the attack localization algorithms is shown in Fig Note that when the system changes to RR controller, the attacker just accesses to the network. She or he does not know that the actions of VMs are changed. So the attacker does not know the detection period. 78

92 The -rst element of x p = 4 p = 3 p = 2 p = 1 LQR Sampling instant (t) Figure 5.3: The trajectories of the first element of x with different values of p Attackers intrude VMs. Attacked VMs send incorrect data to the neighbors. Each PDC sends the identification report to the detector. The detector identifies the attacked VMs. LQR controller t=t0 t=t1 t=t2 t=t3 t=t4 Alarm from firewall or IDS. Change to RR controller. The detector is added. Each PDC runs attack localization algorithms. Attacked VMs are removed. Non-malicious VMs volunteer to take up the task. Change to LQR controller. Figure 5.4: The trajectories of the first element of x with different values of p 79

93 Table 5.1: The cost J using sparse LQR with differnt values of p. p = 4 p = 3 p = 2 p = 1 p = 0 (LQR) J ( 10 7 ) RR Sparse controller in fault case In this subsection, we consider how to use RR sparse feedback gains to identify the malicious VMs covertly. For simple expression, first we consider p = n 1. In this case, the discrete-time dynamical model of the network is changed to x(t + 1) = (A OL + BK q )x(t) = A OL x(t) + BK q x q (t) , (5.20) where q = [(t 1) mod n] + 1. The control inputs are given by u 1 (t) u 2 (t)... u n (t) = K q 1q K q 2q K q nq x q (t) , (5.21) K q j = [0,..., 0, Kq jq, 0,..., 0] is the feedback gain in area j. VM j, 1 j n, calculates the control input u j (t) = K q j x(t), q = [(t 1) mod n] + 1. In (5.21) at sampling instant t u i (t), 1 i n, depends on x q (t). If x q (t) obtained from VM q is not correct, the values of u i (t), 1 i n, is incorrect. For the identification, one detector is added in the system, and a secure private communication network between this detector and the PDCs. This secure communication network is only reserved for 80

94 emergency condition, and is not used during normal conditions when the usual cloud-in-the-loop control is working. The detector knows the structure of the RR sparse feedback gains at each sampling instant in one period, so it can calculate the correct values of inputs. Fig. 8 shows the communication among the PDCs, detector, and VMs with attack. At sampling instant t for 1 j n: (1) PDC j sends x j (t) to the detector and VM j; (2) The detector calculates the correct control input u d j (t) = Ki jx(t), where K i j is the RR sparse controller, i = [(t 1) mod n] + 1, and sends u d j (t) to PDC j. (3) VM j sends the state to its neighbors. If VM j is faulted or attacked, it will send x j (t) = x j (t) + j (t). (4) After obtaining all the states, VM j calculates u j (t) = K i j x(t), and sends u j (t) to the generators at area j. The feedback gains K i j are the same with the ones used by the detector in Step 2. (5) The generators at area j update the state x i (t + 1) = A i x(t) + B i u i (t), and send both x i (t + 1) and u j (t) to PDC j. (6) PDC j identifies the faulted VMs by comparing the difference between u j (t) and u d j (t). Due to the calculation of the control inputs as in (5.21), if VM q is malicious, then at sampling instant t, (t 1) mod n + 1 = q, the control inputs which are calculated by VM j, j / S, should be incorrect, i.e., u j (t) u d j (t); otherwise, if VM q is non-malicious, then at sampling instant t the control inputs which are obtained by VM j, j / S, should be correct, i.e., u j (t) = u d j (t). Thus, each PDC can compare the control inputs sent from the corresponding VM and the detector to identify whether the q th VM is attacked or not at sampling instant t, (t 1) mod n + 1 = q. After one period, all the attacked VMs can be identified. The order of RR is not fixed. If p = n 1, the inputs at one sampling instant are only dependent on the state from one VM. If the number of attacked VMs is unknown, all the orders of RR approach are equivalent for the identification. Thus, it is not necessary for the PDCs to decide the structure of RR sparse controller at each sampling instant. For simple expression, we assume the order is followed the successive order as in (5.21). Note that the attacked VMs can send any values of control inputs to the corresponding generators at any sampling instant. Only PDCs whose corresponding VMs 81

95 With attack Area 1 Area 2 u m1 (t) 1 G1 G3 u m3 (t) 2 u m2 1 (t) G2 G4 u m4 (t) 2 x u m1 (t) 1 m1 (t) 1 x u m2 1 (t) m2 1 (t) x u m4 (t) 2 m4 (t) 2 x u m3 (t) 2 m3 (t) 2 PDC 1 PDC 2 d u (t) 1 x (t) 1 x (t) 2 detector d u (t) 2 x (t) 1 x (t) 2 x (t) t) 2 ( VM 1 VM 2 x 1 (t) Internet of clouds Figure 5.5: The communication between two PDCs and two VMs with and without attack. 82

96 are non-malicious can identify the attacked VMs correctly. We assume the number of attacked VMs is less than n 2. According to the majority rule, the attacked VMs can be identified. If the attacked VMs also use the same states as the ones used by the non-malicious VMs to calculate the inputs and send them to the corresponding PDCs, there is no limitation on the number of attacked VMs for the successful identification. gains. Algorithm 7 illustrates how to identify the faulted VMs with p = n 1 using RR sparse feedback Algorithm 7 Identifying the faulted VMs using RR sparse feedback gains covertly with p = n 1 1) After detecting there exist faulted or attacked VMs at sampling instant t by firewall or IDS, PDC j sends x j (t ) to the detector and VM j, 1 j n. (2) After receiving all the states, the detector calculates the correct control inputs u d j (t ) = K i jx(t ), where K i j is RR sparse controller, i = [(t 1) mod n] + 1, and sends u d j (t ) to PDC j. (3) VM j sends the state to its neighbors. If VM j is faulted or attacked, it will send x j (t ) = x j (t ) + j (t ). (4) After obtaining all the states, VM j calculates u j (t ) = K i j x(t ), and sends u j (t ) to the generators at area j. K i j is used in the same way as the one used by the detector in Step 2. (5) The generators at area j update the state x i (t + 1) = A i x(t ) + B i u i (t ), and send both x i (t + 1) and u j (t) to PDC j. (6) Waiting for n sampling instants, PDC j compares u j (t +t) with u d j (t +t), 1 t n. If u j (t) u d j (t), the i th VM is identified as malicious, i = [(t +t 1) mod n] + 1. (7) The individual PDCs send an identification report to the detector. (8) The detector can identify all the malicious VMs correctly by majority. Note that it is not necessary to let p = n 1. We provide a lemma to illustrate the relationship among p, the number of the attacked VMs k, and the successful identification. Lemma 1 If and only if p k, k n 2, then the attacked VMs can be identified successfully. Proof: p < k, i.e., n p n k, implies that at each sampling instant t the input u j (t), 1 j n, is dependent on at least k p states from attacked VMs. Thus, in this case the individual PDCs can only detect there exist attacked VMs, but not justify the identities of them, since at each sampling instant u j (t) u d j (t). Therefore, for identifying the attacked VMs successfully, it requires p k. 83

97 In general, the individual PDCs do not need n n p sampling instants, k p n 2, to identify all the attacked VMs. Theorem 2 provides the maximum and minimum numbers of the sampling instants for the successful identification. Theorem 2 The maximum number of the sampling instants for the successful identification is n n p n k n p + 1, k p n 2. If the number of attacked VMs is unknown, the minimum number of the sampling instants for the successful identification is n k n p + k n p (n p). If the number of attacked VMs k is known, then the minimum number of sampling instants for the successful identification is 1 + k. Further, if n k is an integer multiple of n p, then the minimum number of sampling instants for the successful identification is min{ n k n p,1 + k}. Proof: The number of non-malicious VMs is n k. n n p n k n p is the number of all the cases that the inputs from VMs are dependent on at least one incorrect state. So there is at least one case that the inputs from VMs are only dependent on correct states during n n p n k n p + 1 sampling instants. The correct inputs are calculated only by the correct states. If we change one of these correct states to an incorrect one from one attacked VM, the attacked VM can be identified. Since n n p n k n p is the number of all the cases that the inputs from VMs are dependent on at least one incorrect state, all the attacked VMs can be identified in n n p n p +1 sampling instants. If at first several sampling instants the inputs are calculated by the correct states, the number of these sampling instants at most is n k n p. Then the left states from other VMs have to be checked one by one. If the number of attacked VMs is unknown, the minimum number of the sampling instants for the successful identification is n k n p + k n p (n p). Now let us consider the case of knowing the number of attacked VMs. If at the first sampling instant the inputs from non-malicious VMs are calculated by 84

98 the correct states, and if at the following continuous k sampling instants the inputs are all calculated by the same n p 1 correct states and one incorrect states, then all the k malicious VMs can be identified. Further, if n k is an integer multiple of n p, and if at the first n k n p sampling instants the inputs are calculated by the correct states, then the other VMs which send the left k states are malicious. Thus, If the number of attacked VMs k is known and if n k is an integer multiple of n p, then the minimum number of sampling instants for the successful identification is min{ n k n p,1 + k}. According to Theorem 2, the number of sampling instants in one period can be reduced to n n p n k n p + 1. It requires that the detector decides the order of RR method. Also, each non-malicious VM should use the same structure of RR sparse feedback gains at each sampling instant. Without designing the order of RR approach, there are n p sampling instants in one period. Algorithm 8 illustrates how to identify the faulted VMs with n 2 p k. After the malicious VMs are identified, they are removed from the cloud, and one or more of the non-malicious VMs (through a negotiation by the network operator) can volunteer to take up the task of the missing VMs. For example, Fig. 5.6 shows the communication between two areas after removing the attacked VM 2. VM 1 is separated into the original VM 1 and the substitute VM 2, which now receive states from and send control inputs to areas 1 and 2, respectively. After the identification, the left VMs calculate the control inputs using traditional LQR approach. 5.4 Simulation To verify our algorithms we consider the IEEE 68-bus power system model shown in Fig The system is divided into 5 areas, each with one PDC. The computation of the control signals are not done at the PDCs or at the generators, but entirely inside a cloud computing network. The simulated model is obtained using the Power System Toolbox (PST) linear dynamics simulation routine svm mgen and the 85

99 Algorithm 8 Identifying the faulted VMs covertly using sparse RR feedback gains with n 2 p k 1) After detecting the faulted or attacked VMs at sampling instant t, PDC j sends x j (t ) to the detector and VM j, 1 j n. (2) After receiving all the states, the detector calculates the correct control inputs u d j (t ) = K i jx(t ), ( ) ( ) n n k 1 i + 1, where K i j is the sparse controller at sampling instant i, 1 j n, n p n p and sends u d j (t ) to PDC j. (3) VM j sends the state to its neighbors. If VM j is faulted or attacked, it will send x j (t ) = x j (t ) + j (t ). (4) After obtaining all the states, VM j calculates u j (t ) = K i j x(t ), and sends u j (t ) to the generators at area j. K i j is the same as the one used by the detector in Step 2. (5) The generators at area j update the state x i (t + 1) = A i x(t ) + B i u i (t ), and send both x i (t + 1) and u j (t) to PDC j. (6) PDC j compares u j (t +t) with u d j (t +t), 1 t T, T = ( n n p ) ( n k n p ) + 1. If u j (t) = u d j (t), PDC j identifies the corresponding VMs as non-malicious. Otherwise, if u j (t) u d j (t), at least one corresponding VM is malicious. (7) The individual PDCs send an identification report to the detector. (8) The detector can identify the non-malicious VMs correctly by majority. (9) The detector decides the order of RR approach, designs the structure of feedback gains at sampling instant t + 1, and broadcasts to the VMs. The non-malicious VMs will follow the order. (10) Waiting for T sampling instants, the detector can identify all the malicious VMs correctly based on the detection reports from the PDCs by majority. After removal of faulted VMs Area 1 Area 2 u m1 (t) 1 G1 G3 u m3 (t) 2 u m2 (t) 1 G2 G4 u m4 (t) 2 m1 (t) m2 x (t) m4 m3 x1 1 x x 2 (t) 2 (t) PDC 1 PDC 2 x (t) 1 x (t) 2 VM 1 Substitute VM2 Internet of clouds Figure 5.6: The communication between two areas after removing the attacked VM. 86

100 Simulation results IEEE 68-bus power system model is divided into 5 areas, each with one PDC PMU 41 PMU G15 52 PMU G16 G14 PMU PMU 51 G Area PDC 2 PDC 3 Area 3 Area 2 Area 1 PMU G13 PMU G PMU PMU G12 G PMU 54 G2 PMU PMU G G PMU G4 Area G PDC PDC 5 G6 G7 20 G5 PMU PMU Area 3 VM Area 2 VM Area 5 VM PDC 4 Substation layer VM Internet of clouds Figure 5.7: Wide-area control using a cloud-in-the loop architecture with IEEE 68-bus model. 41 data file data16m.m. The synchronous generators are assumed to have 3rd order models for simplicity. Since there are 16 generators, the size of matrix A OL is We assume the VMs at the second and third areas are attacked at t = 9. At t = 0 the firewall or IDS broadcasts the alarm, each VM changes to sparse RR controllers, and the detector is added for the identification. For simplicity, in the simulation we let one period have n n p sampling instants and the attacked VMs use the same states as the ones used by the non-malicious VMs to calculate the inputs and send them to the corresponding generators. It implies that the inputs from individual VMs are equivalent. We will draw the trajectory of the input from one VM. According to Lemma 1, if p 2, the attacked VMs can be identified successfully. Fig. 5.8 shows that the trajectories of the first elements of the inputs from the detector and VMs with p = 1. In the case, there are 5 1 = 5 sampling instants in one period. From sampling instant 11 to 15, the input from VMs u j (t) is not equal to the input from the detector u d j (t). Thus, the attacked VMs cannot be identified which matched with the Lemma 1. Fig. 5.9 shows that the trajectories of the first elements of the inputs from the detector and VMs with p = 2. In the case, there are 5 2 = 10 sampling instants in one period. From sampling instant 11 to 87