arxiv: v1 [cs.cr] 1 Apr 2016

Transcription

1 Data Injecton Attacks on Smart Grds wth Mutpe Adversares: A Game-Theoretc Perspectve Anba Sanjab 1 and Wad Saad 1 1 Wreess@VT, Bradey Department of Eectrca and Computer Engneerng, Vrgna Tech, Backsburg, VA USA, Emas: {anbas,wads}@vt.edu arxv: v1 [cs.cr] 1 Apr 2016 Abstract Data njecton attacks have recenty emerged as a sgnfcant threat on the smart power grd. By aunchng data njecton attacks, an adversary can manpuate the rea-tme ocatona margna prces to obtan economc benefts. Despte the surge of exstng terature on data njecton, most such works assume the presence of a snge attacker and assume no cost for attack or defense. In contrast, n ths paper, a mode for data njecton attacks wth mutpe adversares and a snge smart grd defender s ntroduced. To study the defender-attackers nteracton, two game modes are consdered. In the frst, a Stackeberg game mode s used n whch the defender acts as a eader that can antcpate the actons of the adversares, that act as foowers, before decdng on whch measurements to protect. The exstence and propertes of the Stackeberg equbrum of ths game are studed. To fnd the equbrum, a dstrbuted earnng agorthm that operates under mted system nformaton s proposed and shown to converge to the game souton. In the second proposed game mode, t s consdered that the defender cannot antcpate the actons of the adversares. To ths end, we proposed a hybrd satsfacton equbrum - Nash equbrum game and defned ts equbrum concept. A search agorthm s aso provded to fnd the equbrum of the hybrd game. Numerca resuts usng the IEEE 30-bus system are used to ustrate and anayze the strategc nteractons between the attackers and defender. Our resuts show that by defendng a very sma set of measurements, the grd operator can acheve an equbrum through whch the optma attacks have no effect on the system. Moreover, our resuts show how, at equbrum, mutpe attackers can pay a destructve roe towards each other, by choosng to carry out attacks that cance each other out, eavng the system unaffected. In addton, we compared the obtaned equbrum strateges under the Stackeberg and the hybrd modes and we characterzed the amount of oss that the defender endures due to ts nabty to antcpate the attackers actons. Index Terms Data njecton attacks, smart grd securty, game theory, Stackeberg equbrum, satsfacton equbrum. I. INTRODUCTION The recent ntegraton of advanced meterng nfrastructures, data coecton and communcaton nodes have rendered the smart eectrc grd more vunerabe to cyber-attacks. Partcuary, data njecton attacks have recenty emerged as an exceedngy macous type of cyber-attacks. Usng data njecton, macous adversares can target the state estmator of a power system, by targetng a number of measurement unts, n order to ater the estmate of the rea-tme system state [1], [2]. Data njecton can sgnfcanty mpact the overa we-beng of the power system snce t can target the state estmator, an ntegra component of the grd whch s used by the system operator to montor, protect, contro, and economcay operate the system [1], [2]. Usng data njecton attacks, macous adversares can acheve a varety of goas that range from compromsng the securty of the grd to mpedng the reatme operaton of the system or makng fnanca proft through Ths research was supported by the U.S. Natona Scence Foundaton under Grant CNS energy prces manpuaton. Data njecton attacks are nherenty chaengng due to ther steathness whch makes the task of detectng them arduous [2]. In fact, data njecton attacks can modfy the estmaton process whe remanng unnotced by the operator. Recenty, data njecton attacks have attracted sgnfcant attenton [1] [5]. The work n [1] ntroduces a data njecton scheme that can evade detecton when compromsng a number of measurements. The authors n [2] propose an optma data njecton scheme and derve an optmzed subset of measurements that can be defended to face ths attack. The work n [3] targets coordnated attacks and dscusses efforts for detectng those attacks. An anayss of the economc effects of data njecton on energy markets s dscussed n [4]. In [5], a zero-sum game s formuated between an attacker and a defender n whch the attacker modfes an estmated ne fow to manpuate prces. Whe nterestng, ths exstng body of terature [1] [5] has prmary focused on data njecton attacks wth a snge attacker and assume no cost for attackng or defendng the system. However, n practce, due to ther potenta proftabty and steathness, data njecton attacks can occur concurrenty from mutpe adversares that can target varous state estmaton sensors. Due to the networked nature of the smart grd, the manpuaton of measurements n one part of the system, by an adversary, has an overa effect on the system as a whoe. Hence, an attack executed by one attacker does not ony mpact the grd s performance, but t aso affects the benefts of the other attackers. Such an nterdependence can be, on the one hand, benefca to the grd for cases n whch the dfferent smutaneous attacks mtgate the severty of one another. On the other hand, mutpe attacks can ead to a more severe combned effect on the eectrc grd thus further mpactng ts overa performance. Ceary, there s a necessty for a strategc modeng framework to anayze and understand these nterdependences between attackers. Remarkaby, to our best knowedge, beyond our premnary works on data njecton wth two attackers n [6], no work has anayzed the case of mutpe adversares. The man contrbuton of ths paper s to ntroduce nove game-theoretc approaches to anayze data njecton attacks that nvove a defender and mutpe adversares. In ths regard, two approaches are proposed. In the frst approach, we formuate the probem as a Stackeberg game n whch the defender (.e. grd operator) acts as a eader havng the abty to antcpate the actons of the adversares, whch act as foowers, pror to seectng a subset of measurements to defend. The defender s goa s to reduce the effect of potenta attacks on the system whe optmzng a utty that captures both the benefts and costs of the chosen defense strategy. In response to the eader s strategy, the attackers pay a noncooperatve strategc game n

2 whch each attacker chooses ts optma attack scheme n order to maxmze the trade off between the benefts, obtaned from prces manpuaton, and costs assocated wth the attack. We prove the exstence of a generazed Nash equbrum for the attacker s game and we study the exstence and propertes of the overa game s Stackeberg equbrum. To sove the game, we propose a dstrbuted earnng agorthm whch we prove to converge to a souton of the game usng mted nformaton that can be avaabe to the payers. In the second approach, t s assumed that the defender cannot antcpate the actons of the adversary. To ths end, we use the framework of satsfacton equbrum [7] through a proposed hybrd satsfacton equbrum - Nash equbrum game mode. In ths approach, rather than antcpatng the attackers response and payng a strategy that optmzes ts objectve functon, the defender seeks a defense strategy that meets a certan performance constrant. We ntroduce an equbrum concept of ths game and propose a search agorthm to fnd ths equbrum. The performance of the proposed frameworks s assessed va numerca smuatons usng the IEEE 30-bus test system. Through the numerca anayss, we smuate the strategc nteractons between the attackers and defender over the test system. We show that by defendng a mnma number of measurements, the grd operator can enforce an equbrum n whch the attacker s have no effect on the system. In addton, our resuts shed the ght on the adversara behavor n between the attackers. The resuts show that, at equbrum, the attackers can choose attack strateges that cance each other out resutng n no effect on the grd. In addton, we anayze the equbrum of the hybrd game and compare the obtaned souton to the Stackeberg one. In ths regard, we defne a prce of nformaton ndex whch compares the utty acheved by the defender under the Stackeberg mode and the hybrd mode. Hence, t refects the oss that the defender can be subject to due to ack of nformaton about the potenta reactons of the attackers to the dfferent defense strateges avaabe to the defender. The rest of ths paper s organzed as foows. Secton II presents the system mode and probem formuaton. Secton III ntroduces the formuated Stackeberg game and assocated souton. Secton IV ntroduces our proposed hybrd mode and ts souton concept. Secton V provdes numerca resuts whe concusons are presented n Secton VI. II. SYSTEM MODEL AND PROBLEM FORMULATION A. Energy Markets Compettve energy markets archtectures are often based on day ahead (DA) and rea tme (RT) markets [8]. In the DA market, the system operator ssues houry-based ocatona margna prces (LMPs), µ DA, for the next operatng day based on the DA energy bds submtted by the partcpants [8]. The market cearng process s performed by the grd operator through the souton of a nearze Optma Power Fow (DCOPF) whch returns the optma dspatch for each of the generators partcpatng n the market and the DA LMP at each bus. The most commony used DCOPF formuaton s as foows [8]: mn P G C (P ), (1) s.t. (P D ) = 0, (2) P mn P P max, {1,,G}, (3) (P D )χ, F max, {1,,L}, (4) (P D )χ, F max, {1,,L}, (5) where N, G and L represent, respectvey, the number of buses, generators, and transmsson nes. C corresponds to the offer of generator whep andd are, respectvey, the power njecton and oad at a bus. Thus, P = 0 (D = 0) corresponds to the case n whch no generator (or oad) s connected to bus. The upper and ower mts on generator s output are denoted by P mn and P max. Constrants (4) and (5) pace a mt, F max, on the eve of power that can fow over ne. A reference drecton s assgned to the power fow over each transmsson ne. In ths regard, a power fow opposng ts assgned reference drecton s represented by a negatve quantty. Hence, constrants (4) and (5) correspond to the therma mt of a ne n ts reference and opposte drectons respectvey. X s the generaton shft factor matrx whch defnes the senstvty of the power fow over each ne, F, to changes n power njecton, P, at each bus: F (L 1) = X (L G) P (G 1). (6) Therefore the senstvty of the fow over ne to a change n power njecton at bus s denoted by χ,. In the RT market, actua rea-tme operatng condtons estmated usng the state estmator, n eu of the predctons n DA, are used through an ex-post mode to compute the RT LMPs, µ RT [8]. An ncrementa DCOPF s used to compute the RT LMPs and can be formuated as foows [8]: P mn mn P s.t. G C RT ( P ), (7) ( P ) = 0, (8) P P max, {1,,G}, (9) ( P )χ, 0, C +, (10) ( P )χ, 0, C, (11) where C RT s the RT offer of generator whch s computed usng ts RT power output and ts assocated offer curve [8]. C + (C ) s the set of congested nes whch fow s n (opposte to) ther reference drectons. P max and P mn defne a bandwdth whch s empoyed to aow for souton toerance. In practce, here, we typcay [9] set P mn = 2 MW and P max = +0.1 MW. A proposed aternatve to usng ths feasbty bandwdth s aso avaabe n [9]. Thus, the DA and RT LMPs at each bus,, are computed usng the DA and ex-post DCOPFs. The generated LMPs refect, both, the ncrementa cost of energy at bus and the congeston cost assocated wth the contrbuton of ths bus to the system congeston. A ne s sad to be congested f the fow of power over t reaches ts maxmum mt. The DA and RT LMPs at bus are gven by:

3 µ DA = λ 0 + µ RT L =1 (λ DA, = λ 0 + C (λ RT, λ DA,+ )χ,, (12) λ RT,+ )χ,. (13) C {C + C } s the set of congested nes, n RT, obtaned usng the state estmator.c L wherel = {1,,L} s the set of a nes. λ 0 corresponds to the Lagrange mutper assocated wth the energy baance constrants (2) and (8). λ DA,+ and are the Lagrange mutpers correspondng, respectvey, λ DA, to constrants (4) and (5) for ne L whereas λ RT,+ and λ RT, are the Lagrange mutpers correspondng, respectvey, to constrants (10) and (11) for ne C. When L but / C, λ RT,+ = λ RT, = 0. Moreover, when C +, λ RT, = 0; whe when C,λ RT,+ = 0. Computng the RT LMPs rees on the ex-post DCOPF formuaton whch depends on the output of the state estmator. Hence, data njecton attacks targetng the state estmaton affects the LMPs n (13). Next, we ntroduce the data attacode. B. State Estmaton and Data Injecton Attacks A power system state estmator uses mutpe power measurements coected throughout the grd to estmate the system states [10]. The reaton between the measurement vector, z, and the vector of system states, θ, n a nearzed state estmaton mode (DC SE) s expressed as foows: z = Hθ+e, (14) where H s the measurement Jacoban matrx and e s the vector of random errors assumed to foow a norma dstrbuton, N(0,R). Usng a weghted east square (WLS) estmator the estmated system states are gven by [10]: ˆθ = (H T R 1 H) 1 H T R 1 z = Mz. (15) Usng the estmated states, an estmate of the measurement vector, ẑ, and resduas, r, can be cacuated as foows [10]: ẑ = Hˆθ = Sz, r = z ẑ = (I n S)z = Wz, (16) where I n s the dentty matrx of sze (n n), and n s the tota number of coected measurements. When data njecton attacks are concurrenty carred out by M attackers n the set M = {1,...,M}, the coected measurements are modfed through the addton of ther correspondng attack vectors denoted by {z (1),z (2),...,z (M) } resutng n the foowng atered measurements and resduas: z att = z + z (), r att = r +W z (m). (17) m=1 In the case n whch the measurement errorse foow a norma dstrbuton, the WLS estmator s a maxmum kehood estmator of ocaton of the system states [10]. However, the WLS estmator has a zero robustness aganst outers. To overcome ths drawback, outers detecton and dentfcaton mechansms are used so that the fna state estmate s ony based on good data. The measurement resduas gve an ndcaton of the rea and unknown measurement errors. By repacng the expresson of z from (14) n the expresson of r n (16), the resduas can be expressed n terms of the true errors as foows [10]: r = We (18) Thus, an anayss of the resduas aow for the detecton and dentfcaton of bad data (outers). In ths respect, bad data detecton corresponds to determnng whether the coected measurement set contans bad data or not. On the other hand, bad data dentfcaton corresponds to dentfyng whch measurements may contan bad data. One shoud note here that outers can stem from data njecton as we as other reasons such as meter bases or communcaton nk faures [10]. Bad data detecton s typcay performed usng a test known as the Ch-squares test over the sum of the squares of the resduas [1], [10]. In fact, when the measurement errors vector e s assumed to foow a norma dstrbuton, r 2 2 = n r2 foows a χ 2 dstrbuton wth n N θ degrees of freedom where N θ s the number of states to be estmated [10]. Hence, for a measurement set to be consdered free from bad data, the resduas must satsfy r 2 τ where τ s a detecton threshod [1], [10]. In ths respect, n the presence of M attackers, and snce r att 2 = r+w M m=1 z(m) 2 as shown n (17), each attacker m M shoud reguate Wz (m) to keep the effect of the attacks on the resduas ow to mnmze the chance of beng detected as outers [1]. In the case where the Ch-squares test ndcates the presence of bad data, varous bad data dentfcaton and emnaton tests can be empoyed such as the argest normazed resdua test, or the hypothess testng dentfcaton (HTI), among others, to dentfy and emnate the outers [10]. In our mode, each attacker m M ams at manpuatng RT LMPs, µ RT, to make fnanca beneft va vrtua bddng. Usng vrtua bddng, enttes that do not own any physca generaton nor oad can engage n the energy market settements by submttng so-caed vrtua suppy and demand offers. Snce these energy offers are vrtua, an entty offerng to buy (se) vrtua power at a gven bus n DA s requred to se (buy) that same amount of power at the same bus n RT. Usng such vrtua bds, the grd operator ams at promotng qudty n the energy market whe, on the other hand, vrtua bdders am to reap fnanca proft from possbe msmatch between the DA and RT LMPs [8]. Usng a data njecton attack, a vrtua bdder can, thus, manpuate the RT LMPs to create a ucratve msmatch wth respect to ther DA counterparts. On the other hand, to acheve prcng ntegrty, the system operator ams at protectng the system aganst such attacks. The strategc nteractons between the attackers and the defender (.e. system operator) are modeed and anayzed next. III. ATTACKERS AND DEFENDER STRATEGIC INTERACTION Data njecton attacks nvove nteractons between M attackers, whch are vrtua bdders, and one defender consstng of the grd operator. The defender chooses a set of measurements to secure aganst potenta attacks amng at decreasng the aggregate effect of the mutpe attacks on the system. Securng measurements to bock data njecton attacks s dscussed n [2] and the technques that can be mpemented for securng those measurements are referred to n [2], [5], and [11]. In [2] and [11], protecton of measurements s performed through encrypton of the assocated sensors whe, n [5], protecton of measurements s performed through the mpementaton of a set of hghy

4 secured measurement unts whch are assumed to provde more robustness aganst data attacks. In any case, n practce, attackers can have the abty to detect or watch whch measurements are secured by the defender. In fact, a pacement of new measurement unts can be physcay notceabe by the attackers whe encryptng the measurement sensors outputs can aso be observed by a hacker attemptng to read these outputs. After observng whch measurements are secured, each of the M attackers can choose, accordngy, to carry out a data njecton attack over a subset of measurements. Gven the networked nature of the eectrc grd, the actons and payoffs of the dfferent attackers are nterconnected thus motvatng a game-theoretc approach [12]. Hence, gven that the defender acts frst and the attackers react to the observed defender s acton, the nteracton between the defender and attackers s herarchca. Thus, we formuate a snge eader, mut-foower Stackeberg game [12] between the defender and the M attackers to capture and anayze the strategc nteracton between the two. In ths game, the defender acts as a eader who seects a set of measurements to defend whe the adversares nteract wth one another usng a foowers noncooperatve game to dentfy the optma attack n response to the strategy of the defender. By observng or predctng the ways n whch the attackers react to ts defense strategy, the eader chooses ts optma defense acton. Next, we frst anayze and sove the foowers game and then fnd the Stackeberg souton. A. Attackers Noncooperatve Game Formuaton We formuate a strategc noncooperatve game to anayze the optma decson makng of the M attackers n response to any arbtrary defender strategy. Ths game s formuated n ts norma form as foows: Ξ = M,(Z () ) M,(U ) M, where M s the set of M attackers, Z () s the set of actons (attack vectors z () Z () ) avaabe to attacker M, and U s the utty functon of attacker. Thus, each attacker, m M, seects an attack vector, z (m) Z (m) that maxmzes ts utty U m. Let K m denote the subset of measurements that m can attack. Then, Z (m) can be represented by a coumn vector wth eements equa to 0 except for those n K m whch can take vaues wthn a compact range refectng the range of magntude of the attack. The utty functon of each attacker refects the fnanca beneft obtaned by vrtua bddng. Usng vrtua bddng, each attacker m buys and ses P m MW at, respectvey, buses m and j m n DA whe, conversey n RT, attacker m ses and buys P m MW at, respectvey, buses m and j m. Thus, the goa of attacker m M s to optmze the foowng (Probem 1): max U m (z (m),z (m) )= [ (µ RT z (m) Z (m) m µ DA m )+ s.t. Wz (m) 2 + =1, m (µ DA j m µ RT j m ) ] P m c m (z (m) ), (19) Wz () 2 ǫ m, (20) where c m (z (m) ) s the cost of attack, and z (m) denotes the strategy vector of a payers except m. The number of measurements that can be attacked concurrenty by m as we as the attack eves (the eve of modfcaton of a measurement) are mted by Z (m). Snce r att 2 = r+w M m=1 z(m) 2 r 2 + Wz (m) 2 + M =1, m Wz() 2, m M chooses z (m) as n (20), where ǫ m s a chosen threshod, to mnmze the chance of the attack of beng detected as outers. B. Attackers Game Anayss Due to the networked nature of the eectrc grd, the m attackers actons are nterdependent. In fact, by aterng a set of measurements, an attacker manpuates the whoe estmaton outcome and, thus, affects the actons as we as the payoffs of the other attackers. In the event of concurrent attacks by M attackers, the resutng estmates, ẑ att, are computed as foows: ẑ att = ẑ + Sz (m) ẑ = m=1 Sz (m), (21) m=1 where ẑ represents the change n the generated estmates due to the M attacks. Lkewse, the overa change n the measurement resduas due to the M attacks can be expressed as foows: r = W z (m). (22) m=1 Consequenty, the varous attackers n the system can mpar the abty of attacker m to successfuy manpuate a targeted measurement z as expressed n Remark 1. Remark 1: Dependng on the targeted measurements, the coectve mpact of the M attacks can be ether constructve for the attackers by hepng each one of them to acheve ts goa, or destructve, attenuatng the goba effect of these attacks on the system. In fact, consderng the case of two attackers n whch attacker 1 s (attacker 2 s) am s to ncrease the estmated fow, ẑ (ẑ j ), over a ne ( j ) n order to create a fase congeston. The objectve of attacker 1 (attacker 2) s, hence, to acheve ẑ F max ẑ ( ẑ j F max j ẑ j ). Foowng from (21), the change ntroduced to ẑ and ẑ j by the two attacks s stated as foows: ẑ = s, z (1) +s,j z (2) j, ẑ j = s j,j z (2) j +s j, z (1), (23) where s,j denotes eement (,j) of matrx S. When the measurement errors are ndependent and dentcay dstrbuted (.e. R = σ 2 I n ), S s a symmetrc matrx. Ths property can be proven based on (15) and (16) through showng that S T = S when R = σ 2 I n. Snce S s symmetrc, s,j = s j,. In the event wheres,j < 0, both attackers actons attenuate the effect of one another. In fact, snce s,j < 0, z (2) j (z (1) ) reduces ẑ ( ẑ j ) preventng t from causng any congeston over ne ( j ). In the contrary, f s j > 0, each of the attackers actons woud assst the other n achevng ts objectve. Ths resut can be trvay generazed to the case of M attackers. Moreover, the payoffs of the dfferent attackers (.e. vrtua bdders) are aso sgnfcanty nterdependent. In fact, as shown next, an attacker can coect fnanca beneft or endure oses due to the strateges payed by other attackers. Remark 2: The payoff of each attacker, m, s dependent on the chosen attack strateges of other attackers. Thus, based on ts vrtua bdng nodes, m can acheve a postve or negatve payoff dependng on attacks carred out by other attackers. In ths regard, foowng from (19), attacker m s payoff n the presence of M attackers s governed by:

5 ζ m = (µ RT m µ DA m )+(µ DA j m µ RT j m ). (24) Repacng the expressons of the DA and RT LMPs from (12) and (13) n (24) yeds: L ζ m= [(χ,jm χ,m ) ((λ DA, λ DA,+ )+(λ RT,+ λ RT, ))]. =1 (25) As a resut, foowng the sgn of (χ,jm χ,m ), determned by the choce of vrtua bd nodes m andj m, an attacodfyng the congeston status of a ne between DA and RT can ntroduce a postve or negatve payoff to attacker m. C. Attackers Game Souton The attackers payoff n (19) s a functon of DA and RT LMPs. These LMPs are ndrecty controed by the attack vector, z (m), whch can contro the exstence of congeston over transmsson nes and hence eventuay affect the LMPs n (12) and (13). Thus, (19) can be rewrtten usng (12) and (13) as: U m (z (m),z (m) ) = ζ m P m c m (z (m) ), (26) where ζ m s gven by (25). By droppngp m for beng a constant, the objectve of attacker m s hence to max ζ m c m (z (m) ). (27) z (m) Z (m) We defne the two sets of nes L + m and L m such that L + m = { L χ,jm χ,m > 0} and L m = { L χ,j m χ,m < 0}. Moreover, et L R and L O, such that L = {L R L O }, be the sets of nes over whch the power fows n, respectvey, the reference and opposte to reference drectons. Attacker m seeks to congest or decongest nes n a way that maxmzes (27). In ths regard, for a ne L + m,.e. χ,j m χ,m > 0, attacker m profts from creatng a congeston over n the reference drecton, causng λ RT,+ to be postve. Thus, m ams at creatng a congeston over a ne {L + m LR }. Smary, for L m, m benefts from causng a congeston over ne n the drecton opposte to ts reference drecton, causng λ RT, to be postve. Accordngy, m ams a creatng a congeston over a ne {L m L O }. Combnng these two observatons, attacker m ams at creatng congestons over nes {(L + m L R ) (L m L O )}. In a smar manner, an attacker woud aso seek to remove congeston from a ne n order to set ts λ RT,+ To ths end, m ams at removng congestons from nes {(L + m LO ) (L m LR )}. However, due to the presence of measurement errors, an attacker cannot be competey certan that ts attack w ead to the creaton or remova of congeston over a gven ne. In fact, gven the estmated states n the presence of attack, ˆθ att, att the power fow estmates, ˆF, can be obtaned usng the near matrx denoted by H F reatng the power fows to the system states: ˆF att = H Fˆθatt. Usng the expressons of ˆθ gven by (15), or λ RT, to zero n a way that maxmzes (27). ˆF att = H F M(z + z () ) = ˆF +H F M z (). (28) Repacng z by ts expresson gven by (14) and notng that att MH reduces to the dentty matrx, ˆF can be expressed as: ˆF att = H F θ+h F Me+H F M z (). (29) H F θ represents the true fow denoted by F t. Gven that e att N(0,R), ˆF s aso a random varabe that s aso Gaussan dstrbuted wth the foowng expected vaue and varance: E[ˆF att ] = F t +H F M z (), V[ˆF att ] = H F MR. (30) att Gven that ˆF s a vector of random varabes, attacker m att ams at aterng the expected vaue of ˆF to acheve, wth hghest possbe probabty, the ntended congeston creaton or remova to maxmze (26). In other words, to create (or remove) a congeston over a ne, attacker m desgns ts attack so that E[ˆF att ] F max + δ m (or E[ˆF att ] F max δ m ) and ams at maxmzng ths δ m to ncrease ts chances for achevng ts congeston. Thus, attacker m ams to sove the foowng optmzaton probem where S F H F M (Probem 2): max (δ km γα km ) c m(z (m) ) (31) z (m),δ km,α km s.t. Wz (m) 2 + {L + m L m} =1, m ˆF km +S F z (m) + p M\{m} ˆF km +S F z (m) + p M\{m} ˆF km +S F z (m) + p M\{m} ˆF km +S F z (m) + 0 < δ km βf max p M\{m} Wz () 2 ǫ m, (32) S F z (p) F max +δ km α km {L + m L R }, (33) S F z (p) (F max +δ km )+α km {L m L O }, (34) S F z (p) F max δ km +α km {L m L R }, (35) S F z (p) (F max δ km ) α km {L + m L O }, (36) δ km, 0 < α km β Fax m α km, (37) where z (m) Z (m). The constrants n (37) put some mts on the varabes δ km and α km reatve to the correspondng fow mt F max where β and β correspond to the fracton of Fax m that δ km and α km can take respectvey. Thus, attacker m ams at maxmzng δ km to ncrease the chance of creatng congestons over nes {(L + m LR ) (L m LO )} and removng congestons from nes {(L + m L O ) (L m L R )}. On the other hand, due to resource mtaton, an attacker cannot concurrenty acheve a ts favorabe congestons. Thus, the α km varabes are reaxaton varabes to ensure feasbty of the optmzaton probem. However, ths reaxaton s accompaned wth a penaty factor, γ, present n the objectve functon whch refects a decrease n the objectve functon of the attacker for the case n whch a benefca congeston creaton or remova s not performed. Hence, when such a congeston manpuaton s feasbe, ths penaty factor ensures that the attacker has a hgh ncentve to perform ths congeston manpuaton. Moreover, smary to (19), c m (z (m) ) s the cost assocated wth the attack. Ths cost functon can be represented as a scaed norm of the attack vector, where κ m s the scang factor and m s the ength of vector z (m) : m c m (z (m) ) = κ m (z (m) ) 2. (38)

6 Foowng ths formuaton, one can see that the constrants of the optmzaton probems of each of the attackers are couped. In other words, the strategy space of each attacker depends on the strateges seected by the other attackers. Games n whch the constrants of the dfferent payers are couped are known as generazed Nash equbrum probems (GNEP). A wdey used souton concept of these games s known as the generazed Nash equbrum (GNE) whch s defned as foows [13]: Defnton 1: In a game of N payers n whch the contro varabe,.e. strategy, of each payer {1,...,N}, s denoted by x R n and utty functon s denoted by U : R n1+...+nn R, a GNE s a state of the game n whch each payer ams at max U (x,x, ) s.t. (x,x, ) X, (39) x where x, denotes the optma strateges of a other payers except for payer and X s the shared strategy space n between the N payers. In other words, as a response to optma chosen actons of other payers, a payer ams at choosng the strategy, n the restrctng subset dctated by the choce of the other payers, that maxmzes ts own utty. We next prove the exstence of a GNE for the attackers game. Theorem 1: The attackers game has at east one GNE. Proof: Snce δ km and α km are near functons and c m (z (m) ) s a summaton of strcty concave functons, as shown n (38), each attacker s utty functon gven n (31) s a contnuous and strcty concave functon over the attackers strategy profe. Moreover, Z m s a convex and compact set, and as shown n (37), the sets n whch δ km and α km e are aso compact and convex. Thus, snce a GNEP havng compact and convex acton sets as we as contnuous and quas-concave utty functons has at east one GNE [14] [15, Theorem 4.1], our attackers game has at east one GNE. The souton to GNEP probems can be obtaned usng a number of wdey adopted souton concepts that are avaabe n terature [13], [15], [16] where the appcabty of each technque depends on the characterstcs of the utty functons and acton spaces. Gven the strct concavty of the utty functon of each attacker s probem and the convexty of the acton space, such technques converge to a GNE for our derved formuaton. D. Defender s Sde Anayss Under a gven equbrum of the foowers, the eader (grd operator) seects a defense vector a 0 that determnes whch measurements are to be made secure and abe to bock potenta attacks. The objectve of the defender s to mnmze a cost functon capturng the varaton between the DA and RT LMPs, on a N buses n the system, as foows: mn U 0 (a 0,a 0 )=P 1 L (µ RT µ DA ) a 0 A 0 N 2 +c 0 (a 0 ), (40) s.t a 0 0 B 0, (41) where c 0 (a 0 ) s the cost of defense, P L s the tota system oad and B 0 s the mt on the number of measurements that the operator can defend smutaneousy. In (40), µ RT depends on the strateges taken by the defender, a 0, and attackers, a 0 {z (1),z (2),...,z (M) }. The Stackeberg souton concept s adequate for games wth herarchy n whch the eader enforces ts strategy and the foowers respond, ratonay (.e. optmay), to the eader s strategy [12]. We denote the optma response of the attackers to acton a 0 payed by the defender by R att (a 0 ) {z (1) (a 0 ),z (2) (a 0 ),,z (M) (a 0 )}. Ths optma strategy denotes the equbrum strategy profe of the attackers as a response to the defender s strategy. In ths regard, a 0 A 0 s a Stackeberg equbrum f t mnmzes the eader s (.e. defender s) utty functon U 0. In other words, U 0 (a 0,Ratt (a 0 )) U 0(a 0,R att (a 0 )) a 0 A 0. (42) A Stackeberg equbrum s guaranteed to exst and be unque f the optma response of the foowers s unque n response to every acton of the eader. However, Theorem 1 proves the exstence of at east one GNE for the attackers game. Hence, the foowers can have mutpe optma responses to a eaders strategy. In ths case, the eader can rank the GNEs correspondng to each strategy based on ther mpact on ts utty and retan the one that eads to the worst utty (.e. maxma utty gven that the defender s a utty mnmzer). The eader then seects the pocy that mnmzes ths maxma utty. Ths s known as a herarchca equbrum (HE) [12]. In other words, a 0 A 0 s a herarchca equbrum strategy for the defender f: max U 0(a 0,a 0) = mn a 0 R att (a 0 ) E. Dstrbuted Learnng Agorthm max a 0 A 0 a 0 R att (a 0 ) U 0(a 0,a 0). (43) Here, we provde a methodoogy for fndng a herarchca equbrum of the defender-attackers game as defned by (43). We frst consder the attackers subgame. To fnd an equbrum that can be reached by the attackers, we propose a dstrbuted earnng agorthm, based on the framework of earnng automata that was frst anayzed n [17]. The man drvers behnd ths agorthm are as foows. Frst, ths agorthm s fuy dstrbuted n the sense that each attacker s ony requred to know ts own acton space, and not the shared one, and the observaton of ts own payoff after choosng an acton. In ths regard, knowedge of the acton spaces of other attackers or even ther exstence s not requred. Second, snce the attackers game mght admt mutpe GNEs, the use of ths agorthm, emuatng practca smart grds securty settngs, enabes the characterzaton of the GNE(s) that can be actuay reached n practce. The proposed earnng agorthm s shown n Agorthm 1. In ths agorthm, each attacker m frst ntazes a strategy vector q (m) contanng a probabty dstrbuton over ts attack space 1. For nstance,q (m) (t) corresponds to the probabty that attacker z (m) m chooses attack z (m) at tme nstant t. Then, at tme nstant t, each attacker chooses an attack randomy and ndependenty from ts attack space foowng the probabty dstrbuton avaabe through ts strategy vector. The coecton of the attackers actons at tme nstant t resuts n a payoff for each attacker denoted by r m (t). r m (t) s a postve normazed vaue whch corresponds to a mappng from[um mn max mn,um ] [0,1] whereum and Um max are the mnmum and maxmum achevabe uttes by 1 Ths agorthm requres decrtzaton of the acton space of the attackers; our dscretzaton approach s provded n Secton V.

7 Agorthm 1 Dstrbuted Learnng Automata Input: Number of attackers M Acton space of each attacker Z (m) Output: Strategy vector of each payer q (m) 1: Intaze q (m) (0) 2: whe Not Converged do 3: Randomy seect z (m) (t) based on q (m) (t) 4: Coect payoff r m (t) 5: Update strategy vector q (m) (t+1) = q (m) (t)+br m (t) ( e (m) (t) q (m) (t) ) 6: Check Convergence 7: f Converged then 8: Break 9: end f 10: end whe 11: return Strategy vector q (m) m. Based on the payoff that t receves at tme nstant, each attacker, m M, updates ts strategy vector as foows: q (m) (t+1) = q (m) (t)+br m (t)(e (m) (t) q (m) (t)), (44) where b s an arbtrary sma postve constant and e (m) (t) s a coumn vector of ength equa to the sze of the acton set of attacker m. e (m) (t) s a vector whose eements are equa to 0 except for the eement correspondng to the acton that was seected at tme nstant t. The eement correspondng to the seected acton w have a vaue of 1. Thus, gven that the j th attack was seected by attacker m at tme nstant t; then, e (m) j (t) = 1 and e (m) k (t) = 0 for k j. Ths updatng scheme s known as a near reward-nacton (L R I ) scheme [17]. Hence, wth every teraton, the strategy vector of each attacker s updated and the agorthm repeats unt each of the attackers strategy vectors has a eements equa to 0 except for one eement whch s equa to 1. Such a strategy vector shows whch of the strateges s to be chosen by each attacker. The coecton of these attacks (havng a probabty of 1 each) corresponds to the game s equbrum. Ths agorthm has been dscussed n [17], [18] where t has been proven that, for an arbtrary sma b, ths agorthm asymptotcay converges to a pure strategy Nash equbrum (PSNE) when the game admts a PSNE. Defnton 2: Foowng the notatons of Defnton 1 a PSNE s a state of the game n whch each payer ams at max U (x,x, ) s.t. x X, (45) x where, on the contrary wth GNEP, X s payer s own strategy space whch s ndependent of other payers. Thus, the man dfference between a GNE and a PSNE s that the GNE s an optma acton profe n whch each acton does not voate couped constrants wth other payers. Thus, gven that Agorthm 1 s guaranteed to converge to a PSNE, provng that ths PSNE w never voate the couped constrants s enough to prove the convergence to a GNE. Theorem 2: When apped to Probem 1, Agorthm 1 s guaranteed to asymptotcay converge to a GNE when the step sze b s chosen to be arbtrary sma. Proof: For a strategy z (m) to be a best response strategy (BR) for attacker m, t needs to satsfy the property U m (z (m),z (m) ) U m (z (m),z (m) ) z (m) Z (m). A PSNE s hence a state of the game n whch a payers pay BR strateges wth respect to one another. Thus, a strategy that s not a BR strategy cannot be a PSNE strategy. However, a strategy z (m) that voates the resdua threshod constrants n (20) cannot be a BR strategy. In fact, consder the case n whch attacker m attacks ony one measurement z and. If ths attack voates the resdua threshod ofz, z w be dentfed as outer and dscarded from the measurement set. Thus, from (27), ths resuts n ts attack s denoted by z (m) U m (z (m),z (m) ) = ζ m P m c m (z (m) ) < ζ m P m = U m (0,z (m) ). Thus, z (m) s not a BR snce not aunchng an attack at a returns a hgher U m. Hence, a actons that voate the couped constrants are domnated by the aternatve of not carryng out an attack at a and hence cannot correspond to BR strateges. As a resut, a PSNE s guaranteed not to voate the couped constrants n (20) and hence ths PSNE s a GNE of the game. Snce a PSNEs are GNEs and that agorthm 1 asymptotcay converges to a PSNE for a sma b t, as a resut, converges to a GNE of our game. To be abe to choose a herarchca equbrum strategy, a defender needs to antcpate the worst case GNE of the attacker to each defense strategy. Ths antcpaton can be done through: ) repeatng the earnng agorthm of the attackers, Agorthm 1, startng from dfferent nta condtons to fnd a possbe GNEs from whch worst case GNEs can be extracted, ) usng one of the varous agorthms taored to fnd soutons of GNE probems [13], [15], [16], ) anaytca dervaton based on ts fu knowedge of the cyber-physca system mode, energy market mode and avaabe past data. When beng abe to antcpate a worst case GNEs of the attackers, the defender chooses the strategy that resuts n the best worst case GNE. Ths strategy and ts correspondng GNE corresponds to the HE of the game. IV. GAME MODEL UNDER LIMITED INFORMATION Thus far, we assumed that the defender can antcpate a worst case GNEs of the attackers. However, n some nstances, the defender does not have enough knowedge to antcpate the reacton of the attackers. Thus, t cannot seek a strategy that mnmzes ts utty when the reacton of the foowers to any of ts actons s unknown. As a resut, we empoy the framework of satsfacton equbrum (SE) [7], [19]. Under the satsfacton framework, rather than mnmzng ts objectve functon (40), gven a number of measurements that can be defended concurrenty, b 0, the defender ams at keepng the overa changes n the LMPs at a buses under a desred threshod Γ 0 : r 0 (a 0,a 0 ) = (µ RT µ DA ) 2 Γ 0. (46) Gven our herarchca mode, we present a hybrd SE-Nash mode n whch the defender ams at choosng an acton that satsfes ts performance requrement gven potenta reacton of the attackers whe the attackers observe the acton of the defender and pay a noncooperatve game n whch each attacker ams at maxmzng ts utty. Extendng the SE ogc to the

8 attackers game, an attacker s satsfed by payng one of ts BR strateges facng the actons chosen by other attackers and the defender. We denote an equbrum of ths hybrd mode as a hybrd herarchca equbrum (HHE). Defnton 3: A strategy profe (a 0,z(1),...,z (M) ) s an HHE f r 0 (a 0,a 0) Γ 0 and U m (z (m),z (m) ) U m (z (m),z (m) ) z (m) Z (m) and m {1,...,M}. Wth a proper choce of Γ 0 ths game s guaranteed to have at east one HHE. In fact, f none of the actons avaabe to the defender eads to meetng ts satsfacton eve then ether the satsfacton threshod needs to be ncreased or more resources shoud be empoyed so that a arger number of measurements can be concurrenty secured. When the eader chooses an acton that satsfes (46) t has no ncentve to devate from t. The attackers w respond to ths strategy by payng a GNE. Hence, the attackers woud aso have no ncentve to devate from ths GNE. As a resut, the satsfacton strategy of the defender and ts GNE response by the attackers correspond to an HHE of ths hybrd game. Gven the ack of knowedge about the adversares, the defender has to earn the acton(s) that nsure the satsfacton of ts constrant through tra and observaton. To ths end, to fnd a strategy that satsfes ts performance constrant, the defender can adopt the foowng search agorthm: ) For a maxmum number of teratons, N 0, the defender starts by choosng an acton from ts strategy space A 0 foowng a unform probabty dstrbuton f 0 over ths acton space. The foowers observe ths acton and react by payng a noncooperatve game whose GNE s obtanabe va Agorthm 1. ) The eader observes f the acton t had taken ed to the satsfacton of ts performance constrant. If ths s the case, the strategy s hence deemed satsfactory and the eader has no ncentve to devate from t. The foowers response to the eader s satsfacton acton s a GNE and hence the foowers have no ncentve to devate from ther response as we. Thus, ths resuts n an equbrum. ) If the acton that the defender had chosen dd not ead to the satsfacton of ts constrant, another acton s randomy chosen from ts strategy space and the process repeats. Ths agorthm w eventuay fnd a HHE snce, for a arge number of teratons and gven that at east one acton exsts n ts acton space that satsfes the defender s threshod, ths acton woud eventuay be randomy chosen wth a probabty that s extremey cose to 1. Assume the number of vunerabe measurements to be equa to V and that the defender secures b 0 < V measurements concurrenty. Its acton space has then a cardnaty A 0 = V!/(b 0!(V b 0 )!). Assume that n 0 of the aternatves acheve r 0 Γ 0. Choosng unformy between the aternatves, the probabty of choosng an acton that satsfes the defender s equa to p 0 = n 0 / A 0. Thus, the probabty of not fndng a satsfacton acton n N 0 teratons,.e. tras, s gven by (1 p 0 ) N0 ; and hence, the probabty of fndng a satsfacton acton n N 0 teratons s gven by p 0 = 1 (1 p 0) N0. Moreover, the expected number of teratons needed to fnd a satsfacton equbrum strategy s equa to µ 0 = 1/p 0 whe the varance of that number s equa to v 0 = (1 p 0 )/p 2 0. Hence, TABLE I ATTACKERS VIRTUAL BIDDING CONFIGURATION Attacker VB Bus 1 VB Bus2 Target Lne Attacker 1 Bus 3 Bus 4 Lne 4 Attacker 2 Bus 4 Bus 12 Lne 15 Attacker 3 Bus 6 Bus 7 Lne 9 sgnfcanty ncreasng Γ 0 w typcay ncrease n 0 eadng to an ncrease n p 0 and p 0 and a decrease n µ 0 and v 0. As a resut, one can see the confctng effect between the quaty of the found souton, refected by how ow the satsfacton threshod Γ 0 s, and the speed of fndng a souton. Reducng the requred satsfacton quaty eads to fndng a souton faster whe a hgher satsfacton quaty requrement (ower Γ 0 ) eads to a sower dentfcaton of a souton. V. NUMERICAL RESULTS AND ANALYSIS For performance evauaton, we consder three data njecton attackers and one defender nteractng over the IEEE 30-bus test system whch represents a segment of the Amercan Eectrc Power System [20], [21]. In our numerca settng, each attacker s assumed to have a subset of measurements comprsng three measurements that t can attack. In partcuar, attacker 1 can attack ne fow measurements over nes 3, 4 and 7, attacker 2 can attack ne fow measurements over nes 14, 15 and 16, and attacker 3 can attack ne fow measurements over nes 5, 9 and 11. The attack eve on any of the measurements s assumed to have one of the foowng power eves (n MW): { 3.5,2,0,2,3.5}. The amount of vrtua power that each attacker ses or buys s assumed to be equa to 100 MW and ts attack cost s as shown n (38) where κ m = The DA and RT vrtua bddng (VB) nformaton of the dfferent attackers are shown n Tabe I. In ths tabe, VB Bus 1 corresponds to the bus at whch an attacker ses energy n DA (respectvey buys n RT) and VB Bus 2 corresponds to the bus at whch ths attacker buys energy n DA (respectvey ses n RT). The target ne coumn corresponds to the ne connectng the two VB buses whch the attacker ams to congest. In our smuatons, we assume that the system experences no congeston n DA and that each attacker prmary ams at creatng a fake estmated congeston over ts target ne so as to reap fnanca beneft 2. On the other hand, the defender decdes on a subset of measurements to secure out of a the measurements n the system. In our smuatons, we assume that a measurement devce s paced on every bus and every ne n the system so that every power njecton and every ne fow s measured. In Fg. 1 we show the effect of each attacker s optma attack, when no defense or other attackers are present n the system, on the RT LMPs. As can be seen from Fg. 1, assumng that ony one attacker attacks at a tme, the acton of attacker 3 yeds the most detrmenta effect on the system. Ths can be aso seen from Fg. 2 n whch the mpact of each of the attacks on the system s shown. The goba effect of any attack on the system s captured by the defender s utty functon gven n (40). Fg. 2 2 Snce our man focus s on the attackers and defender s strateges, t s assumed that a market partcpants abde by ther DA schedues and, except for the attacks and defense, no change n system condtons occurs between DA and RT. Thus, n case of no attacks, the DA and RT LMPs match.

9 5 4.5 LMP at each bus under dfferent attacks Attacker 1 Attacker 2 Attacker 3 No Attack TABLE II STACKELBERG GAME SOLUTION FORB 0 = {0,1,2} B 0 Secured Measurements Attacker 1 Attacker 2 Attacker (2,3.5,3.5) (0,0,0) (3.5,3.5,0) 1 ne 9 (0,3.5,2) (0,0,0) (0,0,0) 2 nes 4 and 5 (0,0,0) (0,2,0) (0,3.5,-3.5) LMP($/MWh) Day ahead Stackeberg Souton Crtca Defense Bus Fg. 1. System LMPs under a unque adversary s optma attack wth no defense U,j Defender s Loss ($) Attacker 1 Aggregate effect of each attack on the system Attacker 2 Attacker Attacker Fg. 2. Goba effect of each attack on the system. Payoff of attacker due to a congeston over the target ne of attacker j, U,j for the dfferent attackers U 1,2 U 1,3 U 2,1 U 2,3 U 3,1 U 3,2 Attacker 1 Attacker 2 Attacker 3 Fg. 3. Loss to attacker due to congeston over the target ne of attacker j. shows ndeed that the attack of 3 has the hghest goba effect on the system foowed by that of attacker 1 and of attacker 2 respectvey. In fact, the defender s oss under the attack of attacker 3 s equa to $44.69 as compared to $11.61 under that of attacker 1 and $5.74 under that of attacker 2. Fg. 3 shows the effect that a congeston over the target ne of attacker j has on the payoff of attacker denoted as U,j. Accordngy, Fg. 3 shows how the attack of an attacker affects the payoffs of the others. To ths end, the attackers appear to be n a perfecty confctng stuaton snce fufng the purpose of an attacker resuts n a negatve payoff to a other attackers. Next, we consder the strategc nteractons between the three attackers and the defender based on our Stackeberg mode. In ths regard, we fnd the HE of the game, and the underyng GNE of the attackers, when the defender defends an ncreasng number of measurements as shown n Tabe II. We, namey, treat the cases n whch the number of measurements that can be defended concurrenty, B 0, s 0, 1 and 2. The attackers optma strateges are represented n a vector contanng ther respectve optma attack eves (n MW) such that for attacker LMP($/MWh) Bus Fg. 4. Comparson between HE and crtca ne defense strateges 1 the attacked eves correspond to addtve power fows over (ne3, ne4, ne7), for attacker 2 to addtve power fows over (ne 14, ne 15, ne 16) and for attacker 3 to addtve power fows over (ne5, ne9, ne11). Gven that a congeston occurrng over ne 9 has the argest mpact on the system, one can ntutvey expect the defender to secure the measurement over that ne when ony one measurement can be secured (.e. B 0 = 1). Indeed, from Tabe II, we can see that the HE corresponds to the defender defendng ne 9 and the attackers carryng out ther optma equbrum response. For the case n whch the defender can defend up to two measurements concurrenty,.e. B 0 = 2, one expects the defender to secure the measurements of the two nes, nes 4 and 9, whch congeston has the argest mpact on the system (we refer to ths defense, n ths context, as the crtca defense). However, the HE of the game corresponds to the defender defendng nes 4 and 5 nstead. In fact, by defendng those two nes the attackers optma response yeds no effect on the system hence eavng the RT LMPs unaffected. Ths s a representaton of the anayss provded through Remark 1 n whch mutpe attackers attacks can cance each other out. Fg. 4 provdes a comparson between the LMP manpuaton outcome under the HE strategy as compared to the crtca defense strategy. It can be ceary seen that the HE strategy (.e. Stackeberg souton) competey prevents the manpuaton of the RT LMPs and, hence, s a sgnfcanty better strategy than crtca defense. In fact, the attackers optma response to the crtca defense strategy resuted n a successfu manpuaton of the RT LMPs eadng to a 3% root mean square devaton (RMSD) from the DA LMPs where RMSD = 1 (µ RT µ DA ) N 2. (47) Fg. 5 shows the defender s HE utty for dfferent numbers of concurrenty defended measurements. Ths fgure shows that the attackers have a very arge mpact on the system when no defensve actons are taken. In fact, the aggregate effect of the

10 Aggregate effect on the system ($) Roe of defense n reducng attacks mpact Number of defended measurements Fg. 5. Effect of system defense attacks on the system LMPs, at equbrum, wth no defensve actons s $306. However, when B 0 = 1 (defender s abe to secure one measurement), the HE of the game shows that the goba effect of the mutpe data njectons attacks on the system drops sgnfcanty to $11.6. Moreover, when B 0 = 2 (defender s abe to secure two measurements concurrenty), the HE of the game shows that the defender s equbrum strategy competey protects the system aganst attacks and acheves a zero overa effect of the attacks on the system. We next consder our proposed SE-Nash framework and we defne the prce of nformaton (PI) to be an ndex refectng the oss that the defender endures due to ts ack of nformaton about the possbe reacton of the attackers to ts defense strateges. The PI s defned as foows: PI = U0 HHE U0 HE. (48) The PI hence refects the dfference between the utty acheved under the SE-Nash framework and the one acheved under the Stackeberg mode (correspondng to mnmum possbe utty). We consder frst that the defender can ony defend one measurement at a tme. Gven that there are 9 vunerabe measurement unts n the system, the defender has 9 optons to choose from. Consderng that the defender woud be satsfed by havng RMSD 10%, we run the search agorthm descrbed n Secton IV and the HHE we obtaned s smar to the one we obtaned usng the Stackeberg mode for B 0 = 1. Thus, n ths case, PI = 0. Foowng ths HHE, the defender defends the ne measurement over ne 9 and the attackers GNE corresponds to that shown n Tabe II for B 0 = 1. Ths strategy generates an RM SD = 6.1% < 10% hence meetng the performance requrement. Next, we consder the case n whch the defender secures 2 measurements concurrenty. Thus, the defender has 36 optons to choose from. We consder two dfferent performance requrements. In the frst, the defender seeks to have RMSD 5%. Runnng the search agorthm yeds an HHE dctatng the defense of nes4and9whch corresponds to the crtca defense defned prevousy. Ths HHE resuts n RMSD = 3% < 5% and a PI = $5.74. The second consdered performance requrement seeks to have RMSD 10%. In ths regard, our search agorthm ed to an HHE under whch nes 5 and 9 shoud be defended. Ths HHE resuts n RMSD = 6.1% < 10% and a PI = $ VI. CONCLUSION In ths paper, we have studed the probem of data njecton attacks on the smart grd n the presence of mutpe adversares. The strategc nteractons between the defender and the attackers have been modeed usng a Stackeberg game and a hybrd satsfacton equbrum - Nash equbrum game. In these games, the grd operator acts as the eader and the attackers act as foowers whch pay a noncooperatve strategc game n response to each defender s strategy. The costs of attack and defense have been ntegrated n the utty functons of the payers. We have proven the exstence of a generazed Nash equbrum of the attackers game, studed the exstence and propertes of the equbra of the Stackeberg and the hybrd games and proposed earnng agorthms, and proved ther convergence, to compute the games soutons. Numerca resuts have shown the crtcay mportant roe of the defender n protectng the grd and the potenta confctng nteracton between the mutpe adversares. Our resuts aso hghght potenta oss that the defender can ncur due to a ack of nformaton about the actons of the attackers. REFERENCES [1] Y. Lu, P. Nng, and M. Reter, Fase data njecton attacks aganst state estmaton n eectrc power grds, ACM Transactons on Informaton and System Securty (TISSEC), vo. 14, no. 1, pp. 1 33, May [2] T. Km and H. V. Poor, Strategc protecton aganst data njecton attacks on power grds, IEEE Transactons on Smart Grd, vo. 2, no. 2, pp , June [3] S. Cu, Z. Han, S. Kar, T. Km, H. Poor, and A. Tajer, Coordnated datanjecton attack and detecton n the smart grd: A detaed ook at enrchng detecton soutons, Sgna Processng Magazne, IEEE, vo. 29, no. 5, pp , Sept [4] L. Xe, Y. Mo, and B. Snopo, Integrty data attacks n power market operatons, IEEE Transactons on Smart Grd, vo. 2, no. 4, pp , Dec [5] M. Esmafaak, G. Sh, Z. Han, and L. Song, Bad data njecton attack and defense n eectrcty market usng game theory study, IEEE Transactons on Smart Grd, vo. 4, no. 1, pp , March [6] A. Sanjab and W. Saad, Smart grd data njecton attacks: To defend or not? n 2015 IEEE Internatona Conference on Smart Grd Communcatons (SmartGrdComm), Nov 2015, pp [7] S. Ross and B. Chab-draa, Learnng to pay a satsfacton equbrum., n Workshop on Evoutonary Modes of Coaboraton, [8] A. L. Ott, Experence wth pjm market operaton, system desgn, and mpementaton, IEEE Transactons on Power Systems, vo. 18, no. 2, pp , May [9] F. L, Y. We, and S. Adhkar, Improvng an unjustfed common practce n ex post mp cacuaton, IEEE Transactons on Power Systems, vo. 25, no. 2, pp , May [10] A. Abur and A. G. Exposto, Power system state estmaton: theory and mpementaton. New York: Marce Dekker, [11] F. Mao, Q. Zhu, M. Pajc, and G. Pappas, Codng sensor outputs for njecton attacks detecton, n Decson and Contro (CDC), 2014 IEEE 53rd Annua Conference on, 2014, pp [12] T. Başar and G. J. Osder, Dynamc Noncooperatve Game Theory. Phadepha, PA, USA: SIAM Seres n Casscs n Apped Mathematcs, Jan [13] A. Von Heusnger and C. Kanzow, Reaxaton methods for generazed nash equbrum probems wth nexact ne search, Journa of Optmzaton Theory and Appcatons, vo. 143, no. 1, pp , [14] G. Debreu, A soca equbrum exstence theorem, Proceedngs of the Natona Academy of Scences of the Unted States of Amerca, vo. 38, no. 10, p. 886, [15] F. Facchne and C. Kanzow, Generazed nash equbrum probems, 4OR, vo. 5, no. 3, pp , [16] A. von Heusnger and C. Kanzow, Optmzaton reformuatons of the generazed nash equbrum probem usng nkado-soda-type functons, Computatona Optmzaton and Appcatons, vo. 43, no. 3, pp , [17] K. S. Narendra and M. A. L. Thathachar, Learnng automata: an ntroducton. Engewood Cffs, N.J: Prentce Ha, [18] P. Sastry, V. Phansakar, and M. Thathachar, Decentrazed earnng of nash equbra n mut-person stochastc games wth ncompete nformaton, IEEE Transactons on Systems, Man and Cybernetcs, vo. 24, no. 5, pp , May [19] S. Peraza, H. Tembne, S. Lasauce, and M. Debbah, Quaty-of-servce provsonng n decentrazed networks: A satsfacton equbrum approach, Seected Topcs n Sgna Processng, IEEE Journa of, vo. 6, no. 2, pp , Apr [20] O. Asac and B. Stott, Optma oad fow wth steady-state securty, IEEE Transactons on Power Apparatus and Systems, vo. PAS-93, no. 3, pp , May [21] R. Ferrero, S. Shahdehpour, and V. Ramesh, Transacton anayss n dereguated power systems usng game theory, IEEE Transactons on Power Systems, vo. 12, no. 3, pp , Aug 1997.