DISCRETE TIME ATTACKER-DEFENDER GAME

Transcription

1 JP Journal of Appled Mathematcs Volume 5, Issue, 7, Pages Ishaan Publshng House Ths paper s avalable onlne at DISCRETE TIME ATTACKER-DEFENDER GAME Faculty of Socal and Economc Scences Comenus Unversty n Bratslava Mlynské luhy 4, SK-85 Bratslava Slovaka Abstract We analyze a dscrete tme stochastc attacker-defender game wth smultaneous decsons by the attacker and the defender. Actons are not observable, only states of the game, resultng from actons and random factor, are observable. The set of states s fnte. Attacker maxmzes the probablty of httng the fxed target by hs devce (e.g., rocket, defender mnmzes t. A Markov-Nash equlbrum (a Nash equlbrum n Markov strateges s the soluton concept appled to the game. For each subgame, equlbrum probablty of httng the target s the same n each Markov-Nash equlbrum. When at the tme of each decson at least two non-redundant actons are avalable, reducton of frequency of takng actons by the attacker (defender ncreases (decreases the probablty of httng the target. Thus, t s optmal for the attacker (defender to take an acton only once - n the frst (second perod of the game. Mathematcs Subject Classfcaton: 9A5, 9A5. Keywords and phrases: attacker-defender game, stochastc game, Markov behavoral strateges, Nash equlbrum. Research reported n ths paper s fnancally supported by grant No. APVV-4- of the Agency for Support of Research and Development of the Mnstry of Educaton, Scence, Research, and Sport of the Slovak Republc. Receved February 4, 7

2 36. Introducton In the present paper, we analyze a dscrete tme stochastc dynamc noncooperatve strctly compettve game that we call an attacker-defender game. The attacker tres to ht a fxed target by hs devce (e.g., rocket or arplane; henceforth we use the term rocket. The defender tres to prevent hm from dong so, usng hs rocket. Snce the game s stochastc, the attacker maxmzes and the defender mnmzes the probablty of httng the target. (We could consder httng some fxed neghborhood of the target. Ths would not qualtatvely change our results. Trajectores of rockets need not be lnear. They can be descrbed by arbtrary technologcally feasble functons. Remark. A reader has obvously notced that our game remnds - from the pont of vew of modelled ssue - pursut-evason games. (See, for example, [4], Chapter for ther characterzaton. Nevertheless, from techncal pont of vew, our game s qute dfferent. Frst of all, our game s a dscrete tme one. Moreover, the set of states of the game s fnte. Therefore, mathematcal technques that we use n our analyss are qute dfferent and smpler than those used n the analyss of pursutevason games. (We do not use dfferental equatons to descrbe trajectores of rockets. The ams of players are also dfferent. In our game, the attacker s am s to ht a fxed target on the defender s terrtory, not to avod catchng of hs rocket by the rval s rocket (or to avod gettng wth hs rocket to some specfed neghborhood of the rval s rocket. The defender tres to avod httng the target by the attacker (whch need not necessarly nvolve catchng of the attacker s rocket. We use a Markov-Nash equlbrum (.e., a Nash equlbrum n Markov strateges as the soluton concept for our game. Also, we stress the role of so called non-redundant actons of players. A player s acton s non-redundant f t s needed to tackle some pure strategy of the rval. That s, a pure strategy usng a non-redundant acton guarantees the attacker (defender a hgher (lower probablty of httng the target than the lowest (hghest probablty of httng t that the rval can guarantee hmself when the attacker (defender does not use the non-redundant acton. If each player at each tme of decson has more than one non-redundant acton, behavoral strateges are used n each Markov-Nash equlbrum. For these dfferences, we prefer the name attacker-defender game and avod a survey of the lterature on the current state of the art n research on pursut-evason games (snce t would not be helpful to a reader

3 DISCRETE TIME ATTACKER-DEFENDER GAME 37 of our paper. Also, understandably, our am s not to mprove the current state of the art n research on pursut-evason games, but to provde an alternatve to them, whch s useful for reasons descrbed above. Players actons are not observable (by a rval. Only states of the game (henceforth, states are observable. A state n any perod s characterzed by postons of both rockets n a nonempty fnte subset of a three-dmensonal Eucldean vector space. Ths poston s gven by geographcal length, geographcal wdth, and heght. A state n the frst perod s characterzed by the ntal postons of both rockets (on the launchng ramp. States result from players actons and from the nfluence of random factors. The set of states s fnte. Ths assumpton stems from the followng two facts. Frst, actons are made wth the help of a dgtal computer and the current state, as well as tme, s also recorded by a dgtal computer. Second, there s an upper bound on the number of dgts n the nteger part as well as n the decmal part of a number that a dgtal computer can process. Thus, n any compact subset of a threedmensonal vector space the set of recordable postons of a rocket s fnte. (It s reasonable to assume that both rockets wll stay n some compact subset of a three dmensonal vector space around the target. The set of states has a strct subset contanng absorbng states. (For the termnology on Markov chans see, for example, Chng et al. []. These are all states at whch the attacker s rocket s n the poston of the target (.e., the attacker s msson s accomplshed and all states at whch the two rockets are n the same poston but the attacker s rocket s not n the poston of the target (.e., the defender s msson s accomplshed. At absorbng states players do not take any acton. The set of each player s feasble actons depends on the current non-absorbng state. An acton of each player specfes an ntended path of hs rocket between the current perod and the mmedately followng perod. Of course, the actual path can be affected by a random factor and, therefore, dffer from the ntended one. At each non-absorbng state the set of feasble actons of each player s fnte. Justfcaton of ths assumpton s analogous as n the case of set of states. We restrct attenton to Markov strateges (as defned by Maskn and Trole []. Nevertheless, ths s not unduly restrctve. The attacker - unless he overshoots the target - would not lke to move hs rocket backwards. If he overshoots the target and

4 38 the same poston of hs rocket occurs for the second tme, then the two resultng states dffer n the poston of the defender s rocket. Hence, unless random factors are very strong and cause t, no state wll occur more than once. A player s Markov pure strategy assgns to each non-absorbng state one of hs feasble actons at t. A player s Markov behavoral strategy assgns to each nonabsorbng state a probablty dstrbuton on the set of hs feasble actons at t. (Of course, a Markov pure strategy s a specal case of a Markov behavoral strategy. Although a set of a player s feasble actons depends only on the current poston of hs rocket, an acton, or a probablty dstrbuton on the set of feasble actons, n general depends also on a current poston of the rval s rocket because (by affectng the set of rval s feasble actons t affects probablty of httng the target. For each par of players behavoral strateges transtons between states of the game can be descrbed by a Markov chan. We apply a Markov-Nash equlbrum (henceforth, MNE as the soluton concept for the game. It s the applcaton of Nash [3] equlbrum to Markov strateges. We use an MNE manly for two reasons. Frst, snce our game s a strctly compettve game (prevously called zero-sum game, there cannot be a cooperaton (or colluson between players. Thus, there s no ncentve to make actons dependent on payoff-rrelevant nformaton,.e., n our case on the way n whch a current state was reached. (In an nfnte horzon dscrete tme games, n whch cooperaton s possble, dependence of actons on payoff-rrelevant parts of a hstory s used n a punshment of devatons from a collusve behavor. Second, n our game an MNE s the most general soluton concept that rules out equlbra dependng on rratonal belefs. We show that each MNE s also a Markov perfect equlbrum (as defned by Maskn and Trole []. Moreover, for any subgame, the equlbrum probablty of httng the target n t s the same n each MNE. An acton of a player at any state at whch he takes an acton (.e., n the case of the attacker at any non-absorbng state and n the case of the defender at any nonabsorbng state except the ntal one s non-redundant f t s needed to tackle some pure strategy of the rval. That s, a pure strategy usng t guarantees the attacker (defender a hgher (lower probablty of httng the target than the lowest (hghest

5 DISCRETE TIME ATTACKER-DEFENDER GAME 39 probablty of httng t that the rval can guarantee hmself when the attacker (defender does not use t. In other words, a player cannot replace a non-redundant acton wth another acton wthout makng hmself worse off. (See also Defnton 3 n Secton 3. If at the tme of each acton at least one non-redundant acton s avalable, n each MNE each player uses each of hs non-redundant actons wth a postve probablty. Moreover, there s an MNE n whch each player uses only nonredundant actons wth a postve probablty. Then we dscuss the mpact of less frequent takng of actons by one player (once n two perods, once n four perods, etc. on the probablty of httng the target n equlbrum when at the tme of each acton at least two non-redundant actons are avalable. We show that reducton of frequency of takng actons by the attacker (defender ncreases (decreases the probablty of httng the target. Thus, t s optmal for the attacker (defender to take an acton only once - n the frst (second perod of the game.. The Analyzed Game Throughout the paper, R denotes the set of real numbers, N s the set of postve ntegers, and Z s the set of ntegers. We endow each fnte dmensonal real vector space wth the Eucldean topology. For n n N and A R, con ( A s the convex hull of A. If A s fnte, # ( A stands for the cardnalty of A and ( A n n denotes the set of all probablty dstrbutons on A. For n N, R + = [,. We denote the analyzed attacker-defender game by. There are two players n t: the attacker (player and the defender (player. The tme horzon of the game s N. In the frst perod only the attacker takes an acton. Snce the second perod 6 players take actons smultaneously. The set of states s a fnte set Ω R. Each ω Ω has the form ω = ( ω, ω = ω, ω, ω, ω,., 3 ω3 For each {, }, ω = ω ω s the poston of player s rocket,, 3

6 4 (geographcal length, geographcal wdth, and heght. We denote by Ω ( Ω the projecton of Ω on the coordnates, ω, ω (, ω ω of R. 3, The target that the attacker wants to ht has poston ω =, ω,. The ntal state (n perod s ( ( (, ( ω = ω ω ; ω and 3 ( ω are postons of launchng ramps of the attacker s and the defender s rocket, respectvely. We assume that poston ( ( ω cannot be reached from any other poston of attacker s ω (defender s rocket;.e., nether player can place hs rocket back to the launchng ramp. The defender can postpone launchng of hs rocket - he need not launch t n perod two. The attacker has to launch hs rocket n the frst perod. (Ths s wthout loss of generalty - the perod of launchng the attacker s rocket can be numbered by one. We assume that 6 Ωa = { ω Ω ω = ω or ω = ω } s the subset of absorbng states. Ω ( We denote the set of feasble actons of player {, } at ω Ω by A and assume that A s fnte and # ( A ( for each {, } and each ω Ω. The attacker (defender can take any of the actons n A ( A whenever state ω wth ω = ω = occurs. ω For each {, } let n N\ {, } be such that s the smallest postve real number that can be represented n s dgtal computer, set n = mn{ n, n }, and n N\ { } such that n > and s lower than the hghest postve real number that can be represented n s dgtal computer, and let n n ω

7 DISCRETE TIME ATTACKER-DEFENDER GAME 4 X x R+ 3 3 ( n, n, n3 ({ } UN I [, n ] =. such that {,, 3} x = n k k k 3 3 Snce X s fnte, t s a compact subset of R. We assume that ( ω X, and X. + = \ ω We set X X { ω } and X =. ( ω X, + + X X s the set of postons of s rocket at whch (unless the two rockets crashed he can take a new acton. Further, for each {, } let ~ X ~ x R3 3 (,, [, ] n n n3 Z I n =. such that x = n k {,, 3} k k Clearly, X ~ 3 s a fnte and compact subset of R. For each {, } and each mappng + X x let F ( be a fnte set of functons x nto { n n { n,, K, }} X wth the property that f ( x ( = x for each X x and each f ( = ( f ( x ( n n {,, K, n } F ( x. We assume that # ( F ( x for each + {, } and each x X. When poston of s rocket s x + x X n perod t N, each f ( x F ( x descrbes hs desred trajectory of hs rocket between perods t and t + recorded n tme nstances t, t +, K, t +. An actual poston of hs rocket n each tme nstance s affected also by random factor. For + X each {, } and each x each functon f ( x F ( x s technologcally feasble,.e., player has at hs dsposal techncal means such that, f there was nether random factor affectng the poston of hs rocket nor crash wth the rval s

8 4 rocket, startng from poston x, trajectory of hs rocket startng at x descrbed by f ( x would be ensured. We wll call elements of F ( x player s actons at x. Actually, an acton of player at + x X (occurrng n tme perod t N s a contnuous functon q ( x mappng [, ] nto con( X, where q ( x ( k s the poston of s rocket n tme nstance t + k, k [, ]. Nevertheless, observes the poston of hs rocket only n tme nstances n t +, n {,,, K }. + For each {, }, each x X, and for each f ( x F ( x let D ( x, f ( x be a nonempty fnte set of functons mappng { n n { n,, K, }} nto X ~ + such that for each X, x each f ( x F ( x, and each D ( x, f ( x ( d d ( x, f ( x ( =, ( f ( x ( n + d( x, f ( x ( n X for each n {,, }. + For each {, } and each x X, µ ( x, f ( x s the probablty dstrbuton on D ( x f ( x. We can assume wthout loss of generalty that the support of, µ ( x, f ( x equals D ( x f ( x. We also formally assume that f, for some, + x X, f( x F ( x, d D ( x, f( x, and n {,, K }, f ( x ( n + d ( x f ( x ( n = ω,, then f ( x ( n + d( x, f( x ( n = ω for each { n,, n + K }. (After httng the target the attacker does not take any further acton. Fnally, we formally assume that f the two rockets crash (.e., the defender catches the attacker s rocket n tme nstance t + n for some n

9 DISCRETE TIME ATTACKER-DEFENDER GAME 43 n t N and n N I [, ], then ther poston remans unchanged n each tme nstance t n n n n +, n N I [ +, ]. (Crashng of the two rockets creates an absorbng state. No player takes an acton at t. We can defne state space Ω nductvely as follows. Set Ω ( ( = ω, Ω ( = Ω( = { ω ( }, x X f ω F ω Ω( = & d, ω f ω D ω such that ( f ( + ω, ω d f ω ( ( ( ( ( (, f ω ( ( n n, ( = x Ω ( ( = ω, Ω ( = Ω ( Ω (, Ω( = Ω( U Ω(, ( ω, ( ω A F and A = F for each {, } and each ω Ω(. Then probablty of occurrence of ω Ω( n the current perod condtonal on occurrence of perod s ψ ω (, a = ( ω and the use of acton ( ω a A by the attacker n the frst µ ( (,, ω a d ω a. ( ( ( ( ( ( d ω, a ω D ω, a ω : ( ( n ( (, ( ( ω + n a d ω a ω ( =ω (Wth respect to notaton for Markov strateges used below, we assgn actons n A to whole state ω, not just to ω. A player s acton can depend also on poston of the rval s rocket. It s the sum of probabltes of occurrence of ω (condtonal on occurrence of ( ω and the use of acton = ( ω a A by the attacker n the frst perod over all feasble values of a random factor that lead to

10 44 occurrence of poston ω of the attacker s rocket. (Recall that the defender does not take any acton n the frst perod. For each t N wth t 3 set x X ω ( & ( ( Ω t f ω F ω Ω ( t = & d, f D, f such that n n f ( + d, f ( =, x {, }, Ω( t = ( Ω( t \ Ω a ( t U Ω ( t a, ω Ω( t Ω( t {, } ω Ω ( t & f F & d, f D, f such that n n = f ( + d, f ( = ω {, } & f ( n + d, f ( n f ( n + d, f ( n n n N I [, ] & ( n f n + d, f ( ω U, ω Ω( t Ω( t {, } ω Ω ( t & f F & d, f D, f n & n N I [, ] such that, ether [ f ( n + d, f ( n = f ( n + d, f ( n = ω = ω ] or [ f ( n + d, f ( n = ω = ω & f ( n + d, f ( n = ω ]

11 DISCRETE TIME ATTACKER-DEFENDER GAME 45 Ω( t = Ω( t U Ω( t, and A = F for each {, } and each ω Ω ( t. (In the defnton of Ω a( t we assume that catchng of the attacker s rocket or httng the target s recorded when t s recorded by both players. Therefore, we use n. The followng two formulae, expressng probablty of occurrence of ω Ω( t n the current perod condtonal on occurrence of ω Ω( t and the use of acton a by {, } n the mmedately precedng perod, as well as formulae A (5 and (6 below, may seem complcated and unclear to some readers. Nevertheless, they have clear nterpretaton that we gve below them. In order to avod msunderstandng, we also stress that they do not depend on the use of dgtal computes n makng decsons. Probablty of occurrence of ω Ω( t \ Ω ( t n the current perod condtonal on occurrence of ω Ω( t and the use of acton a by {, } n the mmedately precedng perod s = ψ ω, ( a a, d, a D, a & d, a D, a : ( ( a ω + d, a ( =ω, ( ( (, a ω + d ω a ( =ω, a ( n + d, a ( n a ( n + d, a ( n & a ( n + d, a ( n ω n NI[, n ] a A µ, a ( d, a. µ, a ( d, a It s the sum of probabltes of occurrence of ω n the current perod (condtonal on occurrence of ω and the use of acton a by {, } n the mmedately A precedng perod over all pars of values of a random factor affectng the attacker s rocket and a random factor affectng the defender s rocket that lead to occurrence of ω (and do not lead to crashng of the rockets or httng the target. Probablty of occurrence of ω Ω a ( t n the current perod condtonal on occurrence of ω Ω( t \ Ωa ( t and the use of acton a by {, } n the A

12 46 mmedately precedng perod s ψ ω, ( a, a µ, a ( d, a =. µ, a ( d, a d, a ( (, D ω, a (, & d ω a D ω a : n NI[, n ] such that a ( n n + d, a ( n n a ( n n + d, a ( n n n {, K, n } and ether a ( n n + d, a ( n n = a ( n n + d, a ( n n = ω = ω or a ( n n + d, a ( n n = ω = ω It s the sum of probabltes of occurrence of ω n the current perod (condtonal on occurrence of ω and the use of acton a by {, } n the mmedately A precedng perod over all pars of values of a random factor affectng the attacker s rocket and a random factor affectng the defender s rocket that lead to crashng of the rockets at ω (f ω ω or httng the target (f ω ω, or both (f ω = ω = ω. We wll use the symbol ψ ω, ( a, a for the probablty of occurrence of any state any state a ω Ω n the current perod condtonal on occurrence of ω Ω \ Ω and the use of acton a by {, } n the mmedately precedng perod. A Recall that X and X are fnte sets. Therefore, contnung n the way descrbed above, we reach perod t N \ { } such that Ω ( t = Ω( t + k for each k N. Then we set Ω = Ω( t. Of course, the game makes sense only f the target can be ht wth a postve probablty. Therefore, we assume that there exsts t N \ {, } such that ω Ω( t. Clearly, the current state s payoff relevant. (Recall the defnton of A. No other nformaton s payoff relevant. (A current state determnes the sets of players feasble actons. These actons, together wth the current state, determne probablty of occurrence of each state n the mmedately followng perod. The way n whch a

13 DISCRETE TIME ATTACKER-DEFENDER GAME 47 current state was reached does not affect these probabltes. Thus, all subgames wth the same ntal state are strategcally equvalent. (That s, sets of players feasble actons and ther consequences for the probablty of httng the target are the same n all such subgames. We denote a class of strategcally equvalent subgames of determned by ω Ω \ Ωa by. A Markov pure strategy of player {, } n (as well as n the strategc form of s a functon s : Ω \ Ωa U ω Ω A such that s A for each ω Ω. We denote the set of Markov pure strateges of player {, } by S and set S = S S. A Markov mxed strategy of player {, } n (as well as n the strategc form of s a probablty dstrbuton on S. A Markov behavoral strategy of player {, } n s a functon β : Ω ( such U ω A Ω that β ( A for each ω Ω. We denote the set of behavoral strateges of player {, } by B and set B = B B. The symbol β ( a stands for the probablty that player takes acton a at state ω Ω Ω. We denote by A \ a ( β β the restrcton of behavoral strategy β B of player {, } (restrcton of profle of behavoral strateges β to and by B the set of all behavoral strateges of player n. For {, } and s S, b ( s denotes the behavoral strategy generated by s,.e., b ( s = β, where β ( s = for each ω Ω \ Ωa. It follows from the way of generaton of Ω descrbed above that for each par of players behavoral strateges transtons between states of the game can be descrbed by a Markov chan. Snce strateges are computed by a dgtal computer and the latter can represent only a fnte number of decmal ponts, probablty dstrbutons on sets of feasble actons at all states should be chosen from a fnte set. Ths would make sets of players behavoral strateges also fnte. Nevertheless, followng the usual approach n game theoretc lterature, we allow all probablty dstrbutons on sets of feasble actons. In practcal computatons of strateges on a computer, these probablty

14 48 dstrbutons are approxmated wth the precson gven by the maxmal number of decmal ponts that can be represented by a computer. The way of generaton of Ω descrbed above, together wth the assumpton that Ω ( t for some t N \ {, }, ensures that for each ω Ω there exst ω ω Ω and ( a, a A A such that ψ ω, ( a, a >. Functon π : B Ω \ ( Ω U { ω ( } ( Ω assgns to each par of profle of a behavoral strateges and current non-absorbng (and non-ntal state the probablty dstrbuton on the set of feasble states n the mmedately followng perod. We denote the probablty of occurrence of state ω when the players use strategy profle β and the current state s ω, by π ( β, ω. Functon π s defned by π( β, ω = β ( a β ( a ψ ω, ( a, a (. ( a, a A A Functon ( π : B ( Ω assgns to each profle of behavoral strateges the probablty dstrbuton on the set of feasble states n the second perod. It s defned by ( ( a A ω ( ( ( ( ( π β ω = β ω ( ψ ω ( a, a f ( ω = ω, (3 ( π ( β = f ( ω ω. (4 We assume that π ( β = for each ω Ω wth ω = ω. Thus, the attacker cannot ht the target before the defender can take any acton. The attacker wants to maxmze the probablty of httng the target and the defender wants to mnmze t. In an MNE ths holds for any subgame. The latter probablty depends on strateges used by the players. For subgames, whch do not start at the ntal state, t s gven by functonal values of functon ( { ( u : B Ω \ Ω U ω } [, ] defned by a

15 DISCRETE TIME ATTACKER-DEFENDER GAME 49 u ( β, ω = k k = (, ω(, K, ω( k Ω : n= ω ( n ω n {, K, k }, ω( =ω, ω ( k =ω, ω ( n ω ( n n {, K, k } k π( β, ω( n ( n +. (5 Thus, u ( β, ω s the sum of probabltes that the target wll be ht n perod k of the subgame (and the rockets wll not crash before perod k, expressed by the nner sum n (5, over all perods, 3, K of the subgame. For the whole game (.e., for the subgame startng at the ntal state ths probablty s gven by functonal values of functon ( u : B [, ] defned by ω Ω ( ( [ ( u β = π ( β k k = (, ω(, K, ω( k Ω n= ω ( n ω n {, K, k }, ω( =ω, ω ( k =ω, ω ( n ω ( n n {, K, k } k π( β, ω( n ( n + ]. (6 In (6, the thrd sum s the sum of probabltes that the target wll be ht n perod k of the subgame startng at state ω (and the rockets wll not crash before perod k. The second sum s the sum of the latter probabltes over all perods, 3, K of the subgame startng at state ω. Hence, t s the probablty that the target wll be ht n the subgame startng at state ω. Then ( u ( β s the weghted sum of probabltes of httng the target n the subgame startng at state ω over all states, where weghts are gven by the probabltes of reachng state ω n the second perod of the game. (Recall that the target cannot be ht n the frst perod of the game. Functon w : ( Ω \ Ωa B [, ] assgns to each ω Ω \ Ωa and each β B the probablty of reachng subgame when the players follow β. It s

16 5 defned by w, β = π ( ( β + [ π( β ω Ω \ { ω} k k = (, ω(, K, ω( k Ω ω( n ω n {, K, k }, ω( =ω, ω( k =ω,, n= ω ( n ω ( n n {, K, k } k π( β, ω( n ( n + ], (7 f ( ω ω and by ( ( w ω, β = for each β B. Interpretaton of the sums n (7 s analogous to the case of (6. The frst sum s the probablty that subgame wll be reached n the thrd or later perod of the game. Then w, β s the sum of ths probablty and the probablty that subgame perod of the game. wll be reached n the second The followng assumpton says that, due to random factors, each non-absorbng state s reached wth a postve probablty under any profle of behavoral strateges. We use t only n the proof of propertes (, (, and (v n Proposton n the followng secton. Assumpton. For each ω Ω \ Ωa and each B, β w, β >. We end ths secton by the defnton of an MNE of and the defnton of a Markov perfect equlbrum of that we need n Proposton n the followng secton. Defnton. An MNE of s a profle of Markov behavoral strateges β B such that ( ( ( ( ( ( u β u β, β for each β B, ( ( ( ( u β u β, β for each β B. Thus, a profle of Markov behavoral strateges s an MNE of f the attacker cannot ncrease and the defender cannot decrease the probablty of httng the target by a unlateral devaton. It s mmune also to unlateral devatons to non-markov

17 DISCRETE TIME ATTACKER-DEFENDER GAME 5 strateges. When the attacker (defender uses a Markov strategy n an MNE, then the defender (attacker cannot decrease (ncrease the probablty of httng the target by a unlateral devaton that prescrbes dfferent strategy at two occurrences of the same subgame. If he could, for at least one of these occurrences hs strategy taken at t would decrease (ncrease the probablty of httng the target n ths subgame. Then, however, he could decrease (ncrease the probablty of httng the target n by usng the same acton also at all other occurrences of. Defnton. A Markov perfect equlbrum of s a profle of Markov behavoral strateges subgame. β B such that, for each ω Ω Ω, \ a β s an MNE of Thus, a profle of Markov behavoral strateges s a Markov perfect equlbrum of f ts restrcton to each subgame s an MNE n ths subgame. That s, t has to be mmune to all unlateral devatons to all Markov strateges n each subgame. Ths should hold also for those subgames that are not reached wth a postve probablty n equlbrum. It s mmune also to all unlateral devatons to non-markov strateges n each subgame (see Maskn and Trole []. 3. Results The agent strategc form of s a strategc form non-cooperatve game, n whch to each player {, } and each non-absorbng state ω n game, at whch he takes an acton, corresponds one player, called agent of player at state ω and formally denoted by ω. The set of pure strateges of agent ω s A. (Snce we restrct attenton to Markov strateges, the behavor of player at state ω s the same whenever state ω occurs. Therefore, for each player and each state, at whch he takes an acton, there s only one agent. Each agent of the attacker maxmzes the probablty of httng the target and each agent of the defender maxmzes the latter probablty multpled by. (Agent strategc form s used n defnton of a tremblng hand perfect equlbrum of a fnte extensve form non-cooperatve game. See, Selten [5]. Lemma. Game has an MNE. Proof. Snce strateges n are Markov, Ω\ s fnte, A s fnte for Ωa

18 5 ( each ω Ω \ ( Ω { ω ( a U } and each {, }, and A s fnte, the agent strategc form of s a fnte strategc form non-cooperatve game. Therefore, t has β, a Nash equlbrum n mxed strateges. Ths corresponds to a profle = ( β β B n. In order to show that β s an MNE of, we have to show that t satsfes Defnton. Suppose that there exsts β B such that ( ( ( u, β β > u ( β. Then there s ω Ω \ Ωa and β B such that s reached wth a postve probablty when the players follow β, or ( u β, β, ω > u β, ω ( ( ( u β, β > u ( β f ( ω = ω. (8 (If the attacker s able to ncrease the probablty of httng the target by a unlateral devaton, he s able to do so n at least one subgame reached wth a postve probablty. Moreover, snce Ω \ Ωa s fnte and nfnte cycles of states do not contrbute to a postve probablty of httng the target and after dsregardng them (.e., vewng the last state precedng them as the last state of a termnal hstory, each hstory n any subgame of has a fnte length, there exsts ω Ω \ Ωa such that s reached wth a postve probablty when the players follow β and (8 holds also for some β B that dffers from β only n a probablty dstrbuton on A. Ths mples that n the agent strategc form of the agent of the attacker playng at state ω can ncrease the probablty of httng the target by a unlateral devaton. Ths contradcton wth the fact that β corresponds to a mxed strategy Nash equlbrum of the agent strategc form of proves that there do not exst ω Ω \ Ω a reached wth a postve probablty when the players follow β B satsfyng (8. Thus, that t satsfes requrement ( s analogous. β and β satsfes requrement ( of Defnton. The proof

19 DISCRETE TIME ATTACKER-DEFENDER GAME 53 f Defnton 3. ( For each ω Ω \ Ωa, an acton a A s non-redundant (( ( (, ω mn max u b s, b s s S s S s a : (( ( (, ω < max mn u b s, b s s S s = a s S : f ( ω ω, or mn max S s S : s a s ( u (( b ( s, b ( s ( < max mn u (( b ( s, b( s f s S : s = a s S ( ω = ω. (9 f ( For each ω Ω \ ( Ω { ω ( a U }, an acton a A s non-redundant (( ( (, ω max mn u b s, b s s S s S s a : > mn max u( ( b ( s, b ( s, ω. s S s = a s S : ( If an acton s not non-redundant, then t s called redundant. Thus, an acton s non-redundant f t s needed to tackle some pure strategy of the rval. That s, a pure strategy usng t guarantees the attacker (defender a hgher (lower probablty of httng the target than the lowest (hghest probablty of httng t that the rval can guarantee hmself when the attacker (defender does not use a ( a. We use the symbol  for the set of non-redundant actons of agent ω. Note that, n order to determne whether an acton of agent ω s non-redundant or redundant, we need the nformaton about whole ω, not only ω. The reason s that poston of a rval s rocket affects consequences of s acton at target. ω for httng the Proposton. ( If  for each ω Ω \ Ωa and  for each

20 54 ω Ω β \ ( Ω { ω ( a U }, then n each MNE ( β, of, β ( a > for each ω Ω\ Ω a and each ˆ A, ˆ ω and each a A (. a and β ( a for each ω Ω\ ( Ω U{ ω ( } > ( Each MNE of s also a Markov perfect equlbrum of. ( For each ω Ω \ ( Ω { ω ( a U }, u (( β, β, ω s the same for each β β MNE ( β, of and u ( (( β, s the same for each MNE ( β, of. a β (v If  for each ω Ω \ Ωa and  for each ω Ω \ ( Ω { ω ( a U }, then there exsts an MNE of n whch β ( a > for each ω Ω \ Ωa and each ˆ A, a β ( a = for each ω Ω \ Ωa and each a ( ˆ A ω \ A, β ( a > for each ( { ( ω Ω \ Ω ω a U }} and each a Aˆ, and β ( a = for each ( { ( ω Ω \ Ω ω a U } and each a A Aˆ (. \ ω β Proof. Let the assumpton of Proposton hold and let = ( β, β be an MNE of. ( Consder (arbtrary ω Ω \ ( Ω { ω ( a U }. Assumpton mples that each ω Ω \ Ω a s reached wth a postve probablty along the equlbrum path generated by β. Therefore, β solves the mnmzaton program mn ω Ω a A A subject to β ( ( ( ( ψ ω ( β ω ω a β ω a, a u, β ( a a A (, a A ω β ( a =. (

21 DISCRETE TIME ATTACKER-DEFENDER GAME 55 Part ( of Defnton 3 mples that β ( a for each a Aˆ (. > ω (Suppose that β ( a for some a Aˆ (. Then the attacker can reach = probablty of httng the target n ω no lower than the left hand sde of (. He can reach a probablty hgher than the left hand sde of ( f the defender s strategy does not solve the mnmzaton program on the left hand sde of (. Thus, f u ( β, ω s lower than the left hand sde of (, the attacker can ncrease the probablty of httng the target n by swtchng from β to s solvng the maxmzaton program on the left hand sde of (. If u ( β, ω s equal or hgher than the left hand sde of (, the defender can decrease probablty of httng the target n by swtchng from β to s solvng the mnmzaton program on the rght hand sde of (. In both cases, we have a contradcton wth the assumpton that β s an MNE of. Smlarly, for (arbtrary ω Ω \ Ωa, β solves the maxmzaton program max ω Ω a A A subject to β ( ( ( ( ψ ω ( β ω ω a β ω a, a u, β ( θ ( a, a A (, a A where we replace the objectve functon by f ( = ω. ω β ( θ ( a =, ( ( max ( ( ( ( β ω a ψ ω ω, a u β, ω ω Ω a A ω Part ( of Defnton 3 mples that β ( a for each > a Aˆ (. (Argument showng ths s analogous to the one for the defender we ω use (9 nstead of (.

22 56 Ths proves clam ( of Proposton. ( Assumpton mples that each ω Ω \ Ωa s reached wth a postve probablty along the equlbrum path generated by β. Therefore, f the attacker (defender could ncrease (decrease the probablty of httng the target by a unlateral devaton n subgame for some ω Ω \ ( Ω { ω ( a U }, he could do the same also n. Ths contradcton wth the assumpton that proves that for each ( { ( } ω Ω \ Ω U ω β a s an MNE of fact, Defnton, and Defnton clam ( of Proposton follows. β s an MNE of. From ths that ( Take (arbtrary two MNEs of, β and β. It follows from Defnton ( ( ( u u (( β, β β ( u ( β ( u (( β, β ( u ( β. (3 Ths mples that all weak nequaltes n (3 are satsfed as equaltes. Hence, ( ( ( u β = u ( β. ω Ω It follows from Defnton and part ( of Proposton that for each \ ( Ω U { ω ( }, a, u ( β ω u(( β, β, ω, u ( β ω, u (( β β, ω u ( β, ω. (4 Ths mples that all weak nequaltes n (4 are satsfed as equaltes. Hence, u ( β, ω = u( β, ω. Ths completes the proof of clam ( of Proposton.

23 DISCRETE TIME ATTACKER-DEFENDER GAME 57 (v Consder game ˆ obtaned from by elmnaton of all redundant actons. ˆ has an MNE ˆβ. (The argument showng ths s the same as n the proof of Lemma. Defne strategy profle β n by assgnng each non-redundant acton the probablty t has under βˆ and zero probablty to each redundant acton. Defnton 3 and Assumpton mply that, ( a > for each {, }, each ω Ω β ( { ( \ Ω ω a U }, and each a ˆ A, and ( ( β ω ( a > for each ˆ ω a A ( (. The attacker (defender cannot ncrease (decrease the probablty of httng the target by a unlateral change n probabltes wth whch he uses hs nonredundant actons. (If he could, t would contradct the fact that βˆ s an MNE of ˆ. Suppose that there does not exst an MNE βˆ of ˆ such that β derved from t n the way descrbed above s an MNE of. Snce has an MNE by Lemma, there exst an MNE β of, player {, }, ω Ω Ω, and a A \ Aˆ \ a such that β ( a >. Ths s possble only f the contrbuton of a ( a to an ncrease (decrease of the probablty of httng the target s greater than the contrbuton of each a  ( a ˆ A. Then, however, β ( a = for each a  ( β ( a = for each a ˆ A. Usng Defnton 3, ether the attacker (defender can ncrease (decrease the probablty of httng the target by unlaterally changng hs strategy by usng any a ( a   wth probablty one or the defender (attacker can decrease (ncrease the latter probablty by a unlateral devaton. Therefore, β s not an MNE of, whch s a contradcton. Thus, there exsts an MNE βˆ of ˆ such that way descrbed above s an MNE of. Ths completes the proof. 4. Less Frequent Actons of One Player β derved from t n the Standard pursut-evason games are contnuous tme games but the players decde on the plan of ther actons only once - at the begnnng of the game. Ths rases the queston what happens n our game when one player takes an acton less frequently than once n each perod (e.g., once n two perods, once n four perods, etc.. In the present secton, we answer ths queston. (At the end of ths secton we

24 58 explan why we do not deal wth the opposte queston - what happens when one player takes an acton more frequently. We obtan our results here under the followng assumpton about game. Assumpton. # ( A ˆ for each ω Ω \ Ωa and # ( A ˆ for each ω Ω \ ( Ω U { ω ( }. a Ths assumpton s plausble. It s n the nterest of each player to have, whenever he takes an acton, several optons that are not smlar. When the rval has, whenever he takes an acton, several optons that are not smlar, t s hardly possble to cope wth all of them by one acton. From ths the necessty of havng at least two non-redundant actons follows. Assume that player {, } takes an acton at the begnnng of every second perod (.e., at the begnnng of odd perods f = and at the begnnng of even perods f =, whle player j {, } \ { } contnues to take an acton at the begnnng of each perod. (Conclusons about less frequent takng of an acton can be easly derved from the results of ths case. Then an acton of player ncludes several of hs actons n orgnal game - hs acton at the current state and hs acton at each state n the followng perod. (Some states can occur wth zero probablty n the followng perod. Nevertheless, ths depends also on acton of player j at the current state. Snce n game theory usually a player s set of feasble actons does not depend on acton of any other player taken smultaneously, we defne an acton taken once n two perods n the way descrbed above. Thus, t can combne s nonredundant actons at several states. In ths modfed game, we reserve the term nonredundant acton of player for s actons for two perods that combne a nonredundant acton for one perod at the current state wth a non-redundant acton for one perod at each state n the followng perod. It s better for player to use such acton for two perods than several actons for two perods each of whom prescrbes a non-redundant acton for one perod only at the current state and/or some states n the followng perod. In order to acheve the same mpact on the probablty of httng the target, t s enough to use the former wth a probablty lower than the probablty of any of the latter. The reason s that n the former case actons n the current and n the followng perod are coordnated. There s no danger that wll use at some state n the followng perod an acton that does not utlze the ground prepared n the current perod. Ths leaves the opportunty to ncrease probablty of use of other non-

25 DISCRETE TIME ATTACKER-DEFENDER GAME 59 redundant actons for two perods that, by tacklng other actons of the rval, further affect the probablty of httng the target (ncreases t f = and decreases t f =. Assumpton mples that player has at each state (at whch he takes an acton more than two non-redundant actons for two perods. Thus, n each MNE, at each state he uses more than two actons for two perods wth a postve probablty. Clearly, aganst a fxed strategy of player j, he can do equally well (.e., to acheve the same probablty of httng the target as n the orgnal game, n whch he takes actons for one perod. (He can use each acton for two perods wth a probablty wth whch t s used under hs best response aganst j s strategy n. In computng the latter probablty we use s behavoral strategy and probabltes of occurrences of states n the followng perod. Nevertheless, he can do better when he uses only actons for two perods that combne a non-redundant acton for one perod at the current state wth a non-redundant acton for one perod at each state n the followng perod (as descrbed n the precedng paragraph. Thus, aganst any equlbrum strategy of player j n, player = can ncrease and player = can decrease the probablty of httng the target. Then player j = cannot decrease and player j = cannot ncrease the probablty of httng the target. (If he could, he could do so also n the orgnal game, n whch suffers from lack of coordnaton of actons n the two perods. Therefore, n each MNE n the game wth actons of player = ( = for two perods the equlbrum probablty of httng the target s hgher (lower than n any MNE n. The above results extend to the case when player takes an acton even less frequently. Contnung n ths way, we conclude that for each player t s best to take an acton only once n the game - for the attacker n the frst perod and for the defender n the second perod. (The defender cannot take the acton earler than at the begnnng of the second perod. Thus, each player decdes hs plan of actons only once n the game. An ncrease n frequency of takng an acton by one player (e.g., takng an acton not only at the begnnng, but also n the mddle of each perod, n the case of the defender n the mddle of each perod other than the frst one need not affect the probablty of httng the target. The reason s that, despte Assumpton, n one part of a dvded perod at some states a player need not have a non-redundant acton.

26 6 Then, however, more frequent takng of an acton wll not deepen the problem of lack of coordnaton between actons n the two consecutve parts of a perod. (There wll be no danger that wth a postve probablty the acton taken n the current part of a perod prepares ground for an acton n the followng part that s not taken. 5. Conclusons In the present paper, we analyzed a dscrete tme attacker-defender game. Besdes beng motvated by the use of dgtal computers (that can represent only a fnte number of dgts n both decmal part and nteger part of a number, a dscrete tme game s smpler from the mathematcal pont of vew than a contnuous tme game. It allows any contnuous trajectores of mssles. It s mportant that we allow for randomzed - namely behavoral-strateges. Thus, our game s more n the sprt of game theory than the standard contnuous tme pursut-evason games that use manly methods from the control theory. Moreover, the use of randomzed strateges s a necessary consequence of a plausble assumpton that no player at no state has an acton that can tackle each acton of the rval (recall Assumpton n Secton 4. Nevertheless, there s also relatonshp n the opposte drecton. The latter assumpton can be justfed by the use of randomzed strateges. Even f a player knows the rval s randomzed strategy, he cannot know ts realzaton at states that wll occur n the future. If the rval randomzes between actons that are suffcently dfferent, he cannot handle them wth a pure strategy. Tryng to do so would be lke a behavor of a football goalkeeper who, when facng a penalty, throws hmself (wthout any objectve reason lke, e.g., the record of past behavor of the penalty executor nto one corner of the goal. Therefore, he should also randomze between hs actons. The use of behavoral strateges (that do not reduce to pure strateges generates a postve probablty that a player wll actually use (as a realzaton of a prescrpton of a behavoral strategy at some state at the current state an acton that wll not be coordnated wth hs acton n the followng perod. Each player can mtgate ths problem by takng an acton less frequently. Such adjustment eventually leads each player to take acton only once n the game - the attacker n the frst perod and the defender n the second perod. Nevertheless, each player stll uses a randomzed strategy. (In ths case a behavoral strategy concdes wth a mxed one. Even f the rval restrcts hs attenton to pure strateges n contnuous tme, t s benefcal for a player to use a behavoral strategy n dscrete tme, randomzng

27 DISCRETE TIME ATTACKER-DEFENDER GAME 6 between actons that are suffcently dstant. Frst, t enables hm to compute more easly optmal non-lnear trajectores of hs rocket. Second (and more mportantly, t forces the rval to randomze as well. Tryng to do so n contnuous tme wll mpose on hm an addtonal computatonal burden. Swtchng to decson-makng n dscrete tme wll requre swtchng to another software. In both cases, a delay n hs battlefeld actvtes wll result. References [] W. K. Chng, X. Huang, M. K. Ng and T. K. Su, Markov Chans, Sprnger, Hedelberg, 3. [] E. Maskn and J. Trole, A theory of dynamc olgopoly II: Prce competton, knked demand curves, and Edgeworth cycles, Econometrca 56 (988, [3] J. F. Nash, Non-cooperatve games, Ann. Math. 54 (95, [4] K. M. Ramachandran and C. P. Tsokos, Stochastc Dfferental Games, Atlants Press, Pars,. [5] R. Selten, Reexamnaton of the perfectness concept for equlbrum ponts n extensve games, Int. J. Game Theo. 4 (975, 5-55.