Semideterministic Finite Automata in Operational Research

Transcription

1 Applied Mathematical Sciences, Vol. 0, 206, no. 6, HIKARI Ltd, Semideterministic Finite Automata in Operational Research V. N. Dumachev and N. V. Peshkova Department o Mathematics Voronezh Institute o the Ministry o the Interior o Russia Voronezh, Russia Copyright c 206 V. N. Dumachev and N. V. Peshkova. This article is distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Abstract To describe a state low in situational theory o conlict we propose a model that combines the theory o Markov chains and game theory as a inite-state machine. To calculation o normalized payo we used conjugate Markov chain whose states are the possible players winnings. As an example the rancher problem and the gardener problem as semideterministic automata o Bayes type are considered. Mathematics Subject Classiication: 60J20, 90C40, 9-04, 93C85 Keywords: inite-state automata, game theory, Bayesian strategy Introduction The classical theory o inite-state automata [] supposes its work with the deterministic meaning o the input signal. I.e. or any input symbol, there exists a unique state o the next transition. In other words, deterministic inite-state automata given as quintuple A = (s, x, y,, g), where s is a inite set o states; (x, y) is the input and output alphabet, respectively; is the state-transition unction; g is the output unction. A natural generalization o deterministic inite-state machine is to be used o random

2 748 V. N. Dumachev and N. V. Peshkova variables as its arguments. For example, in [2] Rabin introduced nondeterministic automata, in which the transition unction was deined by a stochastic matrix. In [3] a probabilistic inite-state automaton game type has been suggested. Input o this automaton b 0 b... b m a b 0... a k b m S 0 S S... S S... S S S S... S S... S S n S S... S S... S is a sequence o player s strategies and output g (, a,..., a k ) and (b 0, b,..., b m ) b 0 b... b m a b 0... a k b m S 0 c 0 00 c c 0 0m c c 0 km S c 00 c 0... c 0m c 0... c km S n c n 00 c n 0... c n 0m c n 0... c n km is a payo matrix o games or current state o the inite-state machine. The number o payment matrices is determined by the number o possible states o the automaton (S 0, S,..., S n ): p(s 0 ) = c 0 00 c c 0 0m c 0 0 c 0... c 0 m c 0 k0 c 0 k... c 0 km g,... p(s n) = c n 00 c n 0... c n 0m c n 0 c n... c n m c n k0 c n k... c n km Probabilistic characteristics o the machine arise in cases when one rom two players is the nature. Then the human response strategies can be calculated using, or example, the Bayesian criterion. This machine we will call a Bayesian automaton. In paper [3] with help o this machine the concrete situation o crash o the lood dam during loods on the Amur River in 203 was simulated. In [4] this scheme or situational simulation o the Zeya hydroelectric power station in extreme situations was used. In those articles the system had 3 and 5 states respectively. Since the authors did not ind an analytical solution o tasks in this connection was used StateFlow simulation methods o Simulink Matlab package. In this work we give a calculation technique o analytical solutions or automata o small dimension and suggest several examples or their application..

3 Semideterministic inite automata in operational research Preliminary notes about 2 state automaton Consider steps o analytical solution o Bayesian automata ( ( ) ( ) ) x00 x A = (s 0, s ), 0 y00 y, 0,, g. () x 0 x y 0 y ) I output unction is then payment matrix are: b 0 b a b 0 a b S 0 c 0 00 c 0 0 c 0 0 c 0 S c 00 c 0 c 0 c p(s 0 ) = ( c 0 00 c 0 0 c 0 0 c 0 ), p(s ) = ( c 00 c 0 c 0 c 2) For a given probability o arrival o input signal b k rom Bayesian criterion we ind an optimal strategies or all games a k : ). (a i ) = b 0 c i0 + b c i, a = arg(max(( ), (a ))). Obviously, they do not necessarily coincide. 3) The product a k b m generates an input (i.e. argument unction ) which determines the change in a system state at the next step b 0 b a b 0 a b S 0 S k S k2 S k3 S k4 S S k5 S k6 S k7 S k8 Because our machine has 2 states, the set o all possible transition orm a logical unction o = 8 variables (i.e. total N = 2 8 = 256 automata). However, since the Nash theorem asserts that any inite game has a solution in pure or mixed strategies, the number o dierent Bayesian machine is reduced by hal. Furthermore, we can select the equivalent machines. I.e. machines, which have the same limit states and the average winnings o players. Consider an automaton b 0 b a b 0 a b S 0 S 0 S S 0 S 0 S S S S 0 S

4 750 V. N. Dumachev and N. V. Peshkova which have a statelow diagram b 0 /c00 0 b /c 0 0 b 0 /c 00 b /c 0 S0 S b 0 /c 0 0 a b /c 0 b 0 /c 0 a b /c Here, with help o bold lines we have selected the optimal Bayesian strategies. Further, to simpliy, we introduce the ollowing notation. We write transition unction o the automaton in one line = (S 0, S, S 0, S 0, S, S, S 0, S, ) = (000 0) 2 = 3d h and convert it to hexadecimal. Consider the automaton 55 h b 0 /c00 0 b /c 0 0 b 0 /c 00 b /c 0 b 0 b a b 0 a b S 0 S 0 S S 0 S S S 0 S S 0 S S0 S b 0 /c 0 0 a b /c 0 b 0 /c 0 a b /c It is easy to see that the machines 3d h and 55 h are equivalent. These machines have equal optimal strategies and thereore the same limit state and the average winnings. All equivalent machines split the entire set into 7 groups b0... b3 c0... c b b bb c4... c7... cb b c bc... b cc... c c...

5 Semideterministic inite automata in operational research 75 Here, with help o underline ont a leaders o the equivalence classes or which we can obtain exact analytical solutions were marked. Solutions or the remaining machines can be obtained by simply replacing o component o the payo matrix. 3 The rancher problem Beore we present the table o analytical solutions or considered automata we show a calculating technique or limit states and average wins on the practical example. Every morning, the armer collects cucumbers and decides: to pour his own rancho a or not. Rain can go with probability b = 0.4 and watering his rancho also. Depending on watering day the rancho takes one o two states: S 0 good, or S bad. I.e. depending on the input signal a i b j, the state o rancho varies according to the transition unction b 0 b a b 0 a b S 0 S S 0 S 0 S 0 S S S 0 S 0 S 0 Depending on the state o rancho the armer has payo matrix p(s 0 ) = : a : ( b 0 b ) b 0 b 4 5, p(s 4 4 ) = a ( ) 0 : 2 3 a : 2 2 In other words, this automaton has the ollowing output unction g b 0 b a b 0 a b S S Now we ind optimal strategy o armer with help the Bayes criterion (a i ) = b 0 c i0 + b c i, a = arg(max(( ), (a ))). For this, it is necessary to calculate the average win o armer i he uses strategy or a. For S 0 state: ( ) = = 4.4, (a ) = = 4, a = arg(max(( ), (a ))) =. g

6 752 V. N. Dumachev and N. V. Peshkova For S state: ( ) = = 2.4, (a ) = = 2, a = arg(max(( ), (a ))) =. I.e. or both state the optimal strategy o armer is : do not watering own rancho. We have automaton 88 h -type or which StateFlow diagram has the orm b /c 0 0 b 0 /c00 0 b /c 0 b 0 /c 00 S0 b /c S b 0 /c 0 0 a b /c 0 b 0 /c 0 a Now, we introduce the notation µ 0 = b 0 c b c 0 0, µ = b 0 c 00 + b c 0 and deine the player s payo on irst step as The player s payo on second step is Further, m = µ 0. m 2 = (2 b 0 )µ 0 + b 0 µ. m 3 = (3 2b 0 )µ 0 + 2b 0 µ, m 4 = (4 3b 0 )µ 0 + 3b 0 µ,... m n = ( + (n )b )µ 0 + (n )b 0 µ. It ollows that normalized player s payo or our automaton is m n 88 = lim n n = b µ 0 + b 0 µ. Substituting the initial values we obtain µ 0 = 0 ( ) = 4.4, µ = ( ) = 2.4, 88 = 3.2.

7 Semideterministic inite automata in operational research Analytical solutions or ()-automata Next, we assume that zero strategies are optimal rom the Bayes criterion. This implies that average wins or leaders o equivalence classes are automaton 00 h c h 4 h c3 h c7 h 47 h 43 h payo µ 0 µ µ µ 0 + µ b 0 µ 0 + µ + b 0 b 0 µ 0 + b µ µ 0 + b µ + b 0 5 Preliminary Notes about 3-state automaton In article [5] the automata A = ( ( x00 x (s 0, s, s 2 ), 0 x 0 x ) ( y00 y, 0 y 0 y ) ),, g (2) - type had been studied. In this section we consider steps o analytical solution o Bayesian automata ) I output unction is then payment matrix are: A = (s k, x ik, y ki,, g) ; i = 0, ; k = 0,, 2. (3) b 0 b b 2 a b 0 a b a b 2 S 0 c 0 00 c 0 0 c 0 02 c 0 0 c 0 c 0 2 S c 00 c 0 c 02 c 0 c c 2 S 2 c 2 00 c 2 0 c 2 02 c 2 0 c 2 c 2 2 p(s i ) = c i jk; j = 0, ; i, k = ) For a given probability o arrival o input signal b k rom Bayesian criterion we ind an optimal strategies or all games a k : S n : { (a0 ) = b k c n 0k, (a ) = b k c n k, a (S n ) = arg(max(( ), (a ))). Obviously, they do not necessarily coincide. 3) The product a k b m generates an input (i.e. argument unction ) which determines the change in a system state at the next step

8 754 V. N. Dumachev and N. V. Peshkova b 0 b b 2 a b 0 a b a b 2 S 0 S k S k2 S k3 S k4 S k5 S k6 S S k7 S k8 S k9 S k0 S k S k2 S 2 S k3 S k4 S k5 S k6 S k7 S k8 Because our machine has 3 states, the set o all possible transition orm a logical unction o = 8 variables (i.e. total N = 3 8 = automata). To determine the normalized average payo we construct the conjugate Markov chain. Let the payo matrix allow ollowing vector o optimal strategies a (S n ) = (a (S 0 ), a (S ), a (S 2 )) = (m, n, r); m, n, r = 0,. Then Markov chain has the ollowing states s = (c 0 m0, c 0 m, c 0 m2, c n0, c n, c n2, c 2 r0, c 2 r, c 2 r2). I.e. components o the payo matrix corresponding optimal strategies. Hence the weight o the transition edge l(s i, S k ) is probability that Player will obtain proit c k mn, provided that in the previous step he had proit c i mn. From the condition s P = s we determine the stationary distribution s. Then normalized average win o the machine may be calculated by equation = ( s s). 6 The gardener problem: simple The gardener problem in own classic ormulation [6] suggests the presence o two stochastic transition matrix (qij, 0 qij), i, j =, 2, 3. Each matrix corresponds to one o two gardener strategies about choice o method o care or own garden. By two stochastic matrix corresponds to the payo matrix (p 0 ij, p ij), or which the gardener determines own winnings at the next step. As can be seen rom the statement o the problem the garden itsel and randomly changes own state. In this work we describe this system in the orm o semideterministic inite state machine game type. I.e. we assume that changeover o the garden state do not happen by themselves, but under an inluence o external signalactor (invasion o any parasites or onset o abnormal weather condition). Obviously, the pre- predict the occurrence o such eects are impossible, as there are only the probability o their occurrence.

9 Semideterministic inite automata in operational research 755 Thereore we consider the case where Markov transition matrix q is constant or any state o the garden and depend only on the gardener strategies (, a ): q 0 = b 0 b + b 2 0, q = Then the transition unction o automaton has the orm b 0 b b 2 0 b 0 b + b b 0 b b 2 a b 0 a b a b 2 S 0 S 0 S 0 S 0 S 0 S S 2 S S 0 S 0 S 0 S S 2 S 2 S 2 S 0 S S S 2 S 2 S 2 According to the presented unctions the gardener has two strategies: - spray the garden; a - do not spray the garden. Nature has 3 strategy b 0 - without incident; b - insects attack; b 2 - birds and insects attack. Output unction has the orm. b 0 b b 2 a b 0 a b a b 2 S S S and determines the payo matrix o games with nature p(s 0 ) = ( ) ( 9 8 7, p(s ) = It is evident that vector o optimal strategies are ) ( 5 4 3, p(s 2 ) = 2 0 a (S n ) = (a (S 0 ), a (S ), a (S 2 )) = (a,, ) and StateFlow diagram this automaton has the orm ).

10 756 V. N. Dumachev and N. V. Peshkova b 0 /c00 2 b k /c 0 0k b 0 /c 0 0 b /c b /c0 2 k 0k b 2 /c02 2 S0 S S2 b /c 0 b /c b 0 /c 0 a b /c 0 b 2 /c2 a 2 2 a b k /c 2 k For the urther analysis o this game we deine variables and the average win on the irst step c k = c 0 k, d k = c 0k, e k = c 2 0k, k = 0,, 2. µ 0 = b 0 c 0 + b c + b 2 c 2, µ = b 0 d 0 + b d + b 2 d 2, µ 2 = b 0 e 0 + b e + b 2 e 2. To determine the normalized average payo we construct conjugate Markov chain. This Markov chain has the ollowing state vector s = (c k, d k, e k ). The components o this vector are components o the payo matrix, which corresponds to the optimal strategies. Then the weight o the edge l(s i, S k ) is the probability o obtaining by player o win h k provided that in the previous step he proited h i. The conjugate Markov transition matrix is P = and has the limit state s = b 0 b b b 0 b b b 0 b b b 0 b b b 0 b b b 0 b b b 0 b b b 0 b b b 0 b b b + b 2 (b 0, b, b 2, b 0 ( b 0 ), b ( b 0 ), b 2 ( b 0 ), 0, 0, 0). Then normalized average win or this automaton has the orm = (s s) = µ 0 + ( b 0 )µ + b + b 2. Let b 0 = 0.7, b = 0.2, then µ = (9.6; 8.6; 4.6), =

11 Semideterministic inite automata in operational research The gardener problem: extended The undamental dierence o our system rom [6] is that we have two transition matrix and three payo matrix. Two transition matrices depends on two player strategies. Three payment matrix depend on three system states. In classical ormulation the transition matrix (q 0 ij, q ij) depends on the selected gardener strategy (, a ). In our examples the payment matrix depends only on the state o system but not on strategies. Thereore, the output unction o the automaton has the orm g a S 0 q00 0 c 0 00 q0 0 c 0 0 q02 0 c 0 02 q00 c 0 0 q0 c 0 q02 c 0 2 S 0 c 00 c 0 2 c 02 0 c 0 c 2 c 2 q 0 q 0 q 0 q q q S 2 q20 0 q2 0 q22 0 q20 q2 q22 c 2 00 c 2 0 c 2 02 c 2 0 c 2 c 2 2 and determines the payo matrix o games with nature p(s 0 ) = ( c 0 00 c 0 0 c 0 2 c 0 0 c 0 c 0 2 ), p(s ) = ( c 00 c 0 c 2 c 0 c c 2 Determining the optimal strategy using the Bayesian criterion S n : ), p(s 2 ) = ( c 2 00 c 2 0 c 2 2 c 2 0 c 2 c 2 2 { (a0 ) = q 0 nk cn 0k, (a ) = q nk cn k, a (S n ) = arg(max(( ), (a ))). player-gardener essentially orms the value o the next input signal. Under the inluence o this signal, a machine becomes new state according to the transition unction g a q 0 00 q 0 0 q 0 02 q 00 q 0 q 02 S 0 S 0 S 0 S 0 S 0 S S 2 q 0 0 q 0 q 0 2 q 0 q q 2 S 0 S 0 S 0 S 0 S S 2 S 3 q 0 20 q 0 2 q 0 22 q 20 q 2 q 22 S 0 S 0 S S S 2 S 2 S 2 ). which has StateFlow diagram in the orm

12 758 V. N. Dumachev and N. V. Peshkova 0 q 0k/c 0 0k q 2 0 /c 2 0 q 00 /c 0 q 0 /c 0 q q 0 20 /c 2 2/c 2 0 a q /c 0 a q 22 /c 2 2 a 02 q k 0 /c 0k 2 q 20 0 /c 2 00 q 22 0 /c 2 02 S0 S S2 b k /c 2 k Suppose, as in the previous case, the vector selecting optimal strategies has the orm (a (S 0 ), a (S ), a (S 2 )) = (a,, ). Then the transition matrix or conjugate Markov chain P = gives the limit state vector s = 2 q 00 with normalized win q 00 q 0 q q 0 0 q 0 q q 0 0 q 0 q q 00 q 0 q q 00 q 0 q q 00 q 0 q q 00 q 0 q q 0 0 q 0 q q 0 0 q 0 q ( q 00, q 0, q 02, q 00( q 00), q 0( q 00), q 02( q 00), 0, 0, 0 ) = ( s s) = µ 0 + ( q00)µ, (4) 2 q00 where µ 0 = q0k c 0k, µ = qk 0 c0 k, µ 2 = q2k 0 c0 2k. For example, i q 0 = , q = then µ = (9.2; 7.5; 3), = Conclusions In this paper, a complete classiication o Bayesian automata () was presented. As a result, we received 7 non-equivalent machines. Notice that i graph o

13 Semideterministic inite automata in operational research 759 automaton has branched structure then to get an exact solution is not always possible. Thereore, the analysis o speciic models should be carried out by methods o situational modeling [3,4]. I the model has an exact solution (as (3)) then its analysis is a simple substitution o the payo matrix components into analytical expressions (as (4)). In this paper the computational technique or some simple machines was suggested. In unlike [5] to calculate the normalized wins was used a conjugate Markov chain or which states was the components o the payo matrix o the man-nature game. It was shown that some o the problems o analysis o interaction o human with nature can be described as semideterministic automaton Bayesian type. Reerences [] W. Brauer, Automatentheorie, Teubner, Stuttgart, 984. [2] M.O. Rabin, Probabilistic automata, Inormation and Control, 6 (963), [3] V.N. Dumachev, N.V. Peshkova, A.V. Kalach, A.A. Chudakov, Statelow simulation o crash o the lood dam during Far East loods in the summer o 203, Vestnik Voronezhskogo Instituta GPS MChS Rossii, 4 (203), [4] V.N. Dumachev, N.V. Peshkova, A.V. Kalach, A.A. Chudakov, Statelow simulation o Zeyskaya hydroelectric power station in during loods, Vestnik Voronezhskogo Instituta GPS MChS Rossii, 2 (204), [5] V.N. Dumachev, On semideterministic inite automata games type, Applied Mathematical Sciences, 9 (204), [6] H.M. Taha, Operations Research: An Introduction, Pearson Education, Inc., New Jersey, Received: January 7, 206; Published: March 9, 206