OPTIMAL CONTROL PROBLEM FOR PROCESSES REPRESENTED BY STOCHASTIC SEQUENTIAL MACHINE

Transcription

1 OPTIMA CONTRO PROBEM FOR PROCESSES REPRESENTED BY STOCHASTIC SEQUENTIA MACHINE Yaup H. HACI an Muhammet CANDAN Department of Mathematics, Canaale Onseiz Mart University, Canaale, Turey ABSTRACT In this paper, optimal control problem for processes represente by stochastic sequential machine is analyze. Principle of optimality is proven for the consiere problem. Then by using metho of ynamical programming, solution of optimal control problem is foun. Keywors Optimal control problem, stochastic sequential machine, ynamical programming.. INTRODUCTION Stochastic sequential machine (SSM) is the one of the most eveloping fiel of iscrete system theory [2], [3]. It plays an important role in many areas such as construction of finite ynamical system, imitation moelling problem, coing of iscrete systems an ientification problems. Thus, it points out that it requires a comprehensive research. 2. SSM SSM is generalization of multi-parametric finite sequential machine[3] but it contains probability variable. General form of this system is efine by[6]: K =< X, S, Y, s, p( ), F (.), G > v =,2,... () where X = [ GF( 2) ] r, S = [ GF(2) ] m an Y [ GF(2) ] q s is initial state vector, () = are input, state an output inex (alphabet) respectively; p is etermitistic iscrete probability istribution ( Ω = {, 2,..., p} is finite set, p( ) = { p( i ) : i Ω, p( i ) = }), characteristic Boolean vector functions [7] enote by F ( ) = { F ( ),..., F ( ) } DOI:.52/ijci m i Ω which are also nown as transfer functions are nonlinear functions efine on the set Z [ GF ( 2) ] m [ GF (2)] r an G are an output characteristic functions efine on GF (2) where GF (2) is Galois fiel[8] an the symbol (.) enotes for simplicity. In aition to the efinition, SSM is represente by: c + e ) F v =,2,... (2) =

2 y ( c) = G( (3) where s ( x ( y(c) are m, r an q imensional state, input an output vectors at the point c respectively, (c) c c c Z =, c c c,..., c c c, c Z is point in is a ranom variable [6], { } Z, etermining position i of this process. Z is set of integers an e (,...,,,,...,). i, ( i =,2,... positive integer) is the uration of the stage = In SSM, each ranom variable has a special case. For instance, is an input variable in the ientification problem of the SSM. However, is a set of all possible states in the synthesis of optimal sequential machine Moreover, the state of the system epens on the ranom variable which affects not only parameters of the SSM but also the input variable. Finally, equation (2) is transforme to: c + e ) = F c) v =,2,..., (4) where symbol means that x w is always in input alphabet X. The iscrete optimal processes given by SSM are characterize by functional: = M c ))} (5) where M {.} is a mathematical expecte value of. i 3. OPTIMA CONTRO PROBEM AND PRINCIPE OF OPTIMAITY We can state optimal control problem [2] for processes represente by SSM as below: In orer for given SSM to start from the nown inital state s to go any esire state s * ( c ), to which we expect to access in steps, a control c) X [4] must exist such that the functional in () has a minimal value: c + e ) = Fˆ, c v =,2,... (6) G s ( c ) = s, x c) X G (, c (7) F ˆ ( c + e, Fˆ, c + e ), c + e )) = Fˆ ( c + e Fˆ, c + e ) c + e ) = M c )} min (9) where F ˆ (.) ( =,..., ) ^ enotes the pseuo Boolean expression of the Boolean vector function[7] F (.)( =,..., ) an = is the time uration of this process. It is well-nown that metho of ynamic programming [5] is use for solution of optimal control problem. If we mae use of this metho to solve the consiere problem then, (6)-(9) can be formulate as an optimal problem: c + e ) = Fˆ, c G () (8) 22

3 s (σ ) = ℵ () c) X, c (2) G F ˆ ( e, Fˆ, e ), e )) = Fˆ ( e, Fˆ, e ), e )) c ))} min = M (4) where ℵ is an arbitrary element in S. As it can be seen from ()-(4), if we substitute σ = c or ℵ = s into problem ()-(4) we obtain first problem state above. If the conitions for existence of unique solution are satisfie then for the given initial conition σ ) = ℵ an given c)( c ), we fin a unique s (c). That is, the functional (4) is the function of the G parameters ℵ an c)( c ) : G = J ( ℵ, ) (5) G where ) enotes the range of the control c) on the points c (σ ) : G G ) = { c G } (6) from the unique solution conition of the system (6), we fin that the stochastic process can be investigate in the set G (σ ) an also in the set G { c; c c < σ c c σ } =,..., < Definition. We say that the control ( c ) which minimizes the functional (5) in the G problem ()-(4) is optimal control with respect to the initial pair ( σ, ℵ) on the region (σ ) Suppose that x ( c ) is an optimal control with respect to the initial pair ( c, s ) on the region G an s ( c ) is amissible optimal trajectory. Then x ( c ) is an optimal control with respect to the initial pair ( σ, s ) on the region G (σ ) for every G. Proof. Suppose the contrary. Then there exists c) X, c such that we have J ( ℵ, G ) < J ( ℵ, x ( G ) (8) G G G (7) (3) We choose a new control process ~ c G as follows: ~ x ( c G x ( c) = (9) c G As it can be seen, (9) is an amissible control such that 23

4 J ( s, ~ x ( G )) = J ( s, ~ x ( G G ). (2) Accoring to the conition, s ( σ ) = ℵ σ. Thus we have ~ ( ( ) ( )) = ( ( ), ~ J ( s, x G σ G σ J s σ x ( G ( )) = = J ( ℵ, G )) < J ( ℵ, x ( G )) = = J ( σ ), x ( G )) = J ( s, x ( )) (2) G an by virtue of (2.) an (2.) we can obtain ~ ( J ( s, x G )) < J ( s, x ( )) (22) G Hence, (22) contraicts the hypothesis that the control the proof of the theorem. x ( c G is optimal. This completes et a function (for every fixe σ an ℵ) be corresponing to the optimal value of pseuo Boolean functional in the problem ()-(4): c ))} B( σ, ℵ ) = min M. (23) where minimization on the set of amissible control c. Now, we etermine metho of ynamical programming(it is nown as Bellman equation) [5] for B ( σ, ℵ). Suppose that x ( c G is the amissible control corresponing to ()-(4) with initial conition an s ( c is also the optimal trajectory. et the point G ξ s G ( v =,2,..., ) an any element y( c) X be specifie. If x = y( c), then the state of the system in the point ξ σ is etermine by ξ σ ) = Fˆ y, σ )) (24) We consier the following problem: ξ c) = Fˆ, c G ( ξ σ ) (25) ξ ˆ σ ) = F y) (26) c) X ( c (27) G ξ G c ))} min = M (28) If yˆ ( c G ( ξ σ ) an sˆ ( c G ( ξ σ ) are optimal control an optimal trajectory respectively, then 24

5 M c ))} B( ξ σ, Fˆ σ ))) = (29) can be foun. For ()-(4), let ~ x ( c ) be an amissible control below. ~ y, c = σ x ( c) = (3) yˆ( c G ( ξσ ) Also, ~ s ( c ) can be obtaine by ˆ ~ F (, ℵ,, ( )), = ( σ y σ c σ s c) = (3) sˆ( c), c G ( ξσ ) It is evient that the value of M c ))} { ( ~ φ s ( c ))} = M ˆ( s c ))}= = to control ~ x ( c ) is etermine by M = B( ξ ˆ σ, F σ ))). (32) Since ~ c is not largely optimal control, we can state G M { ( ~ φ s ( c ))} M s ( c ))} = B( σ, ℵ) (33) Thus, we have B( σ, ℵ ) B( ξ σ, Fˆ σ ))) (34) On the other han, if y ( c) = x, then by the principle of optimality [2], ˆ y( c)( c G ( ξ σ )) = x ( c)( c G ( ξ σ )) (35) Therefore, ˆ B( σ, ℵ ) = B( ξ σ, F x, σ ))) (36) By (34) an (35), Bellman equation[5] can be etermine by B( σ, ℵ) = ℵ S min B( ξ Fˆ σ, σ ))), X (37) y 25

6 4. CONCUSIONS It is shown that Bellman equation for optimal processes with stochastic sequential machine is obtaine an the principle of optimality is proven. REFERENCES [] Anerson, J. A., (24) Discrete Mathematics wit Combinatorics, Prentice Hall, New Jersey, p.45 This is my paper, ABC Transactions on ECE, Vol., No. 5, pp2-22. [2] Boltyansii, V. G., (978) Optimal Control of Discrete Systems, John Willey, New Yor, p.363. [3] Gaishun, I. V., (983) Completely Solvable Multiimensional Differential Equations, Naua an Tehnia, Mins, p.23. [4] Hacı, Y., Ozen, K., (29) Terminal Control Problem for Processes Represente by Nonlinear Multi Binary Dynamic System, Control an Cybernetics, Vol. 38, No. 3, pp [5] Bellman, R., (957) Dynamic Programming, Princeton University Press, Princeton, p.2 [6] Yermolyev, Y. M., (976) Stochastic Programming Methos, Naua (in Russian), p.24. [7] Yablonsy, S. V., (989) Introuction to Discrete Mathematics, Mir Publishers, Moscow, p.9. [8] Fraileigh, J. B., (998) A First Course in Abstract Algebra, 6th e. Aison--Wesley Publisher, p Authors Yaup H. HACI receive Ph.D at physics an mathematics in Azerbaijan NationalAcaemy of Sciences at the Institute of Cybernetics. He is currently woring as professor an hea of epartment of Mathematics, in Canaale Onseiz Mart University. His primary research interests are in the areas of the theory of multiparametric binary sequential machines,linear an nonlinear optimization an iscrete mathematics. He is an author(co-author) of over 38 papers. Muhammet CANDAN receive his M.Sc egree from Canaale Onseiz Mart University, Canaale, Turey, in 22. He is currently Ph.D. caniate an woring as a research assistant in Canaale Onseiz Mart University. His research interests cover applie mathematics, finite ynamical systems, graph theory. 26