Analysis of the problem of intervention control in the economy on the basis of solving the problem of tuning

Transcription

1 Anaysis of the probem of intervention contro in the economy on the basis of soving the probem of tuning arxiv: v1 q-fin.gn] 9 Nov st eter Shnurov Nationa Research University Higher Schoo of Economics Moscow, Russia pshnurov@hse.ru Abstract The paper proposes a new stochastic intervention contro mode conducted in various commodity and stoc marets. The essence of the phenomenon of intervention is described in accordance with current economic theory. A review of papers on intervention research has been made. A genera construction of the stochastic intervention mode was deveoped as a Marov process with discrete time, controed at the time it hits the boundary of a given subset of a set of states. Thus, the probem of optima contro of interventions is reduced to a theoretica probem of contro by the specified process or the probem of tuning. A genera soution of the tuning probem for a mode with discrete time is obtained. It is proved that the optima contro in such a probem is deterministic and is determined by the goba maximum point of the function of two discrete variabes, for which an expicit anaytica representation is obtained. It is noted that the soution of the stochastic tuning probem can be used as a basis for soving contro probems of various technica systems in which there is a need to maintain some main parameter in a given set of its vaues. Index Terms controed stochastic processes, absorbing Marov chains, stochastic probem of tuning, mathematica modes of economic interventions, contro in technica systems. I. INTRODUCTION The artice addresses the probem of the deveopment and the anaysis of a stochastic mode of the intervention phenomenon in economic systems. As such a mode, it is proposed to use a Marov stochastic process with discrete time, that is, a Marov chain. In the scientific iterature, papers are nown in which Marov processes with discrete time are used to describe the intervention, but a these modes are buit on the basis of autoregression. A more detaied description of such studies is given in Section III. In this paper, the stochastic intervention mode is constructed on the basis of a specia type of Marov process, namey, Marov chains with absorption. Another fundamenta difference between the proposed mode and the nown ones is the presence of contros, which are reaized after ingression into absorbing states. These contros describe the intervention, that is, externa infuence on an economic system. We aso note that the proposed stochastic Marov mode with discrete time has a universa character and can be used 2 nd Danii Noviov Nationa Research University Higher Schoo of Economics Moscow, Russia even.he@yandex.ru to describe the contro of technica systems, in the operation of which it is necessary to maintain a certain basic parameter in given acceptabe imits. A more detaied anaysis of the possibe appications of the deveoped stochastic mode is given in the fina part of this wor. Short description of the stochastic mode describing the above-mentioned phenomena occurring in technica and economic systems is proposed by.v. Shnurov in 1]. This pubication aso presents the resuts of soving the optima contro probem for this mode, which the author has caed the tuning probem. II. DESCRITION OF THE HENOMENON OF INTERVENTION IN THE ECONOMY To design the mode we need to describe the process behind it. Hence, et s begin with the definition of the intervention and other reated terms from economics. Intervention is commony defined as an economica infuence of one entity on actions and matters of another entity through the investment and the deposit of its own funds. Usuay the intervention operations are carried out via centra ban and the Treasury by means of massive buying or seing currency, securities and the provision of credit in order to normaize the financia system 2]. As the definition indicated, there are two main types of interventions: one of them has its purpose in seing goods, currencies or securities and the other in buying them. These actions are the essence of the so caed intervention. Such activities as intervention purchases and intervention stocs are taing pace in marets for agricutura commodities in Russian Federation and they are aso defined by the aw. We wi refer to these notions foowing the 14th artice of the federa aw 3]: The government intervention purchases, stoc interventions are carried out with a view to stabiizing the prices in the agricutura maret, foods and commodities, as we as to maintaining income-support for agricutura producers. The government intervention purchases are taing pace when prices for agricutura products fa beow the estimated

2 prices. The means of conducting such intervention is through purchasing incuding exchange trades agricutura commodities from agricutura producers or impementing pedge operations in reation to the products in question. Government interventions in the stoc marets are carried out through seing the purchased agricutura products incuding exchange trades when the prices on the agricutura products exceed the maximum estimated prices. Thus, we ve described the essence of an economic phenomenon of interventions. In genera, it consists in a purposefu infuence on the maret of a product, currency or securities in form of buying or seing them. When an intervention purchase is conducted, it is reated to the time when a maret price of a certain product fas beow a minimum imit. Intervention stocs wi be conducted at times when a maret price exceeds the maximum of a predetermined eve. In this respect, we shoud note that the characteristics given above, namey, once price reaches its ower or upper boundary eves, are the main reasons for the intervention. Therefore, the main purpose of the intervention is to reach such a condition of an economic system in commodities or currency maret, where one of its main parameters price of a product or currency is found within the permissibe imits, which is between its minimum and maximum boundary eves. A mathematica mode in intervention research shoud refect, above a, the core eements of the rea process of intervention. III. REVIEW OF RESEARCHES OF INTERVENTION HENOMENON BY MATHEMATICAL METHODS Russian research of intervention. In the Russian scientific iterature there are not a ot of wors dedicated to the intervention phenomenon research using mathematica modes and methods of anaysis of economic systems. Note the research Romaneno I.N. and Evdoimova N.E. 4], 5], 6]. These wors describe the construction of a mathematica mode of intervention purchases and intervention stocs in the Russian grain maret. The basis of this mode is the baance principe. This mode is a set of inear reations that incude equaities and inequaities between the main dynamic characteristics of the system regiona or A-Russian grain maret. Each reation expresses a certain ind of baance or partia equiibrium in the terminoogy of the authors. For exampe, the sae of grain by producers is ineary dependent on the equiibrium price of grain and current stocs of grain producers. The coefficients of these inear reations are cacuated using regression anaysis and refined in the anaysis of data for Anaytica representation of maret equiibrium price as a function of the voume of grain purchases becomes possibe as a resut of the anaysis of this system of reations. This representation is as foows 6]: Ct V te b d + f + 0, 001hK rubc W t + St, h + a g 1 where Ct - equiibrium price of grain; V t - current stocs of maretabe grain from producers; St - trade and purchasing baance positive for purchases; K rub t - rube-doar exchange rate; C W t - price of export contracts; These characteristics depend on the parameter t, which has the sense of time, that is, they can be considered as dynamic. Constants a, b, d, e, f, g, h incuded in the expression for the maret equiibrium price Ct are coefficients whose numerica vaues can be determined on the basis of avaiabe information on the grain maret. This reation is the main theoretica resut of this research. According to the authors 6], the use of this reation wi sove various probems associated with the effective conduct of intervention purchases and intervention stocs. In particuar, this wi aow us to cacuate the voumes of interventions aimed at maintaining indicative prices or optimizing the criterion of the efficiency of the functioning of the grain maret. In addition, this wi mae it possibe to assess the effects of a compex reguatory impact on pricing processes and maret winnings of a grain maret entities, depending on the amount and timing of grain interventions. Let us give some comments on the above concusions given in 4], 5], 6] concerning the mode in question. 1 The presented mode is inear and deterministic. Rea maret processes are random by their nature. 2 Even if we accept the proposed mode, there is no expanation why the formua 1, expressing the price of the product, provided that a certain maret equiibrium is fufied, aows us to optimize the criterion of the effectiveness of the functioning of the maret. Mathematicay, the optimization probem is not posed and is not soved here. 3 The concusion about the possibiity of assessing the effects of compex reguatory impact on pricing processes and the maret winnings of a grain maret entities, depending on the voumes and timing of grain interventions, can ony be taen in terms of the abiity to numericay evauate the effect of individua parameters on others in the inear deterministic mode. Despite these features and significant simpifications, the studies carried out in papers 4], 5], 6] are significant because they form and anayze one of the first mathematica modes describing interventions in the Russian grain maret. A significant research of the Russian grain maret using mathematica methods was carried out in 7]. The goa of the authors of this wor was to find stabe modes of operation of the intervention fund, in which the baance between intervention stocs and intervention purchases is maintained in the ong run. The genera method of D. Forrester s system dynamics is used. A simuation mode of the grain maret in Russia was deveoped, based on a system of dynamic inear reations. In the construction of the mode, a nonequiibrium approach is used. It is assumed that a the demand and suppy functions are nown; b demand and suppy at an arbitrariy chosen point in time are generay not baanced, and

3 imbaances are compensated by the dynamics of grain stocs, incuding the intervention fund. The regression mutidimensiona dependence of the export size on the gross grain harvest in the preceding months, the foreign trade price of grain and the price of grain in the domestic maret was constructed. Estimates of the parameters of this regression mode are found. The obtained resuts are used for quaitative assessments of measures of state reguation of the grain maret in Russia. An agorithm for maing decisions on interventions in the grain maret based on a foating price corridor is proposed. Computer experiments on the basis of this agorithm were carried out, according to their resuts a number of quaitative concusions were made. In 8] the mode of M.orter s competitive forces for the Russian grain maret is constructed. This mode is a too for quaitative anaysis. Based on this mode, the author maes a number of quaitative concusions. There are aso specia studies devoted to the anaysis of interventions in the Russian foreign exchange maret. Among these, we note 9], which anayzed the Ban of Russia s foreign exchange interventions for the period from January 2001 to November In this paper, the authors use methods of quaitative economic anaysis and cassica econometric methods reated to the construction of mathematica modes. In this wor, data on gross foreign exchange interventions of state funds and the Ban of Russia were coected and systematized. It is noted that the main parameter, to which the impact of intervention is directed, is the vaue of the bicurrency baset in rubes, that is, amounts of doar and euro with the corresponding weight coefficients. The ideas used to assess the effectiveness of foreign exchange interventions are characterized. It has been estabished that the Ban of Russia foreign exchange interventions do not have a significant impact on the vaue of the bi-currency baset, and therefore, on the tendency of the rube exchange rate change in the medium and ong term. But at the same time, these interventions significanty affect the voatiity of the vaue of the bi-currency baset. The author uses autoregressive modes GARCH and GARCH-M, describing voatiity and characterizing the impact of interventions on it. The resuts of statistica anaysis of the quaity of the modes considered are presented. We note in addition that the resuts of the conducted studies of autoregressive modes of voatiity are given by the authors in a very concise, overview form, without presentation of the origina mathematica reations. Foreign research of intervention. Scientific studies of the phenomenon of intervention in commodity and currency marets are aso being conducted in other countries. At the same time in the foreign scientific iterature there are much more pubications devoted to the anaysis of the phenomenon of intervention than in the Russian. The bu of research is focused on two areas. The first of them is reated to the research of interventions conducted by state organizations in the marets of grain crops of the various countries. The second is devoted to the study of reguarities and features of interventions conducted by state bans and other state financia structures in foreign exchange marets. Let s give a brief description of some of them, in which mathematica methods are used to some extent. Let s start with the research of interventions in grain marets. Wor 10] is devoted to the research of the grain maret in China. To describe the grain maret, a mutidimensiona autoregressive inear mode is constructed. This mode incudes five main characteristics that describe the maret. The mode is discrete in time, the vaues of the main indicators are determined annuay on the interva from 1986 to The mode taes into account the forms of state impact on the maret quotas for government purchases of grain, set by the government. The import of cereas aso infuences the maret. The authors assume that the vaues of the main indicators at the current time t can depend on the vaues of the other indicators at the moments t and t 1. The paper describes obtaining numerica vaues of estimates of the coefficients of the autoregressive mode. The importance of infuence of some indicators on others is investigated. This autoregressive mode of the grain maret in China is used to simuate the possibe effects of the state on the state of the maret. The author s cacuations show the consistency of the resuts obtained in the modeing and the data on the actua processes that too pace in the Chinese grain maret in the specified period of time. We aso note that the bibiography of scientific research of the grain maret in China is given in the wor 10]. A thorough and extensive study of the various forms of state infuence on the rice maret in India is conducted in 11]. The author considers two main forms of such impact: state purchases of raw unpurified rice grain directy from producers farms at guaranteed minimum government support prices MS and the purchase of processed purified rice from primary grain processors mis using preferentia system of taxation. Thus, direct and indirect forms of support of farms are provided and production is stimuated. The paper uses a mutidimensiona stochastic maret mode that describes the dependence of the reative capita growth of the main participants producers of goods on the voume of investments purchases of buyers. This dependence is expressed in an expicit inear anaytica form and has a dynamic character, that is, it taes into account the dependence on the transaction number time point. We shoud especiay note that in the mode a buyers of grain are divided into two groups: arge and sma, which is taen into account in the nature of the functiona dependence. In this paper, it is assumed that the distribution density of procurement voumes can be approximated by means of Hermite series of order 2. On the basis of the statistica materia, estimates of the unnown parameters of the mode are constructed. After evauating unnown parameters and specifying the mode, the author of the wor uses it to simuate rea processes in the rice maret. Using modeing, estimates of the incomes of grain producers are made and the dependence of these incomes on various factors is investigated: minimum government procurement prices, eves of taxes on product prices of

4 primary processors, and the eve of competition in the maret determined by the number of participants in transactions. The authors of the wor cacuate the observed prices and the corresponding estimates of the producers incomes. Based on the methods of statistica anaysis, the asymptotic distribution of the optima purchase price estimate is determined. Thus, it becomes possibe to determine the estimate for the optima procurement state price and compare the producers incomes in rea conditions using the optima state procurement prices. The paper 12] investigates some of the specia effects associated with interventions in the maret for geneticay unmodified soybeans at the Toyo Stoc Exchange. In particuar, the effect of the receipt of information on the change in the duration of the futures contracts on the highest price of this type of grain is investigated. The autoregressive integrated moving average mode ARIMA from the theory of time series has been used as a mathematica mode to describe the intervention impact on the price. As a resut of the anaysis on specific data, it is estabished that incoming information eads to an increase in the price, but this effect manifests itsef with a deay equa to four months. This means that the maret in question is not effective, because in theory, an effective maret shoud immediatey respond to incoming information. In addition to this resut, this wor is interesting because in the mode under consideration, the roe of direct impact, that is, intervention, is not the voume of rea purchases or saes, but information about changing the rues of the game in the reevant maret. In 13] a mathematica mode of the housing maret is proposed, based on the idea of genera equiibrium. In the roe of externa infuences interventions are tax benefits associated with income from empoyment, income from capita and renta income. The infuence of these interventions on the wefare of buyers is investigated. In particuar, home buyers can perform maret anaysis and determine the most preferabe forms of action in the maret: the acquisition of housing in the property, or rent. In addition, homeowners are given the opportunity to assess the effectiveness of additiona investments in housing and subsequent renta housing. Note aso the paper 14], in which an extensive review of econometric modes and methods used to describe the impact of the state in various areas of the economy is made. Summarizing the review, we wi give a genera description of scientific research on the phenomenon of intervention in the economy, conducted using mathematica modes and methods. Let us note first of a that the probem of investigating interventions in economic systems is important and urgent. It attracts the attention of speciaists in various countries. In the modern scientific iterature there are studies in which crops mathematica modes of interventions in the marets of grain. Such modes are inherenty stochastic, given the random nature of the factors operating in a free competitive maret. They are regressive and autoregressive reations describing the impact of various factors, incuding the extent of interventions, on some of the ey indicators that characterize the maret system under consideration. The unnown parameters incuded in these reations are estimated from the resuts of observations, that is, according to the statistica information at the disposa of the authors. After the creation of such a mode, it becomes possibe to assess the impact of interventions on the most important basic economic indicators of the system. There are other reated studies in 10], 11], 12], 7]. There are aso wors based on the deterministic inear mode of the grain maret 4], 5], 6]. Investigations of the phenomenon of intervention in currency marets using mathematica methods are rarey conducted 15]. In our opinion, this is due to the objectivey existing compexity of this phenomenon, as we as the fact that decisions on such interventions are made at the government eve and depend not ony on economic but aso on poitica factors that are difficut to describe. IV. GENERAL STRUCTURE AND BASIC FEATURES OF THE STOCHASTIC MODEL INTERVENTIONS IN THE ECONOMY In this section, a genera concept of a new stochastic mode wi be proposed, intended to describe the phenomenon of intervention. This concept is based on the genera concept of intervention in the economic system formuated in Section II, as we as on a quaitative anaysis of this phenomenon. In the framewor of this study, the economic system wi be understood as a certain commodity or financia maret in which random factors objectivey act. In this maret, some basic parameter is formed that changes over time. This parameter is a stochastic process that wi be considered a mathematica mode of the functioning of the system under research. The state of this process at an arbitrary point in time wi characterize the state of the system. In the commodity maret system for exampe, the grain maret this parameter is the current unit price of the reevant product. In the grain maret system, this parameter is the product price. This parameter may be used as the present vaue of the bi-currency baset in the financia maret system. Evoution of this process consists of two main stages that go one after another and forming a repeating cyce. At the first stage, the price is formed without externa infuence, according to the interna rues of this maret system. The set of possibe states of the process incudes the specified subset of admissibe states. Externa infuence is not used during the process stay in this admissibe subset. At moments when the process the main parameter goes to the border or beyond the subset of admissibe states, an externa infuence is made. This is the intervention. Conducting such an impact forms the second stage of the process evoution. The purpose of this infuence is the return of the process vaue to one of the state from admissibe subset. This externa infuence is the contro in this probabiity mode. The probem of optima contro is the choice of contro characteristics that give an extremum to some indicator of contro quaity. This indicator shoud have an economic bacground and refect the efficiency of the economic system. We tae the foowing important assumption reated to the nature of the proposed stochastic mode. This assumption is

5 that the process describing the functioning of this system has a Marov property. As is nown, the Marov property consists in the fact that after entering a certain fixed state, the further evoution of the process occurs independenty of the past, and depends ony on the specified state. This property is characteristic of many economic and technica systems and reated stochastic processes. The presence of this property wi aow to use the resuts of a deepy deveoped theory of Marov processes. Iustration of the concept described above - Fig. 1, which shows the possibe trajectory of the main process and possibe contro effect. Fig. 1. A possibe trajectory of the stochastic process stochastic mode of the behavior of the main parameter. { ξ }, which is a Let s give some comments to Figure 1, expaining the aspects of the above stochastic mode. However, we wi first accept certain conventions on the designation of states. Under the natura order of numbering, the boundary states for the set in question are the states {0} and {N}. In this mode, the boundary states are considered invaid, and the interna states {1, 2,..., N 1} - are admissibe. However, in order to buid a subsequent theory, it wi be necessary to immediatey re-designate the states, abandoning the natura numbering. Further in this investigation we wi use the cassica theory of absorbing Marov chains 16]. For convenience in appying this theory, we rewrite the states in the origina set {0, 1, 2,..., N} as foows: the state {0} we eave the previous notation {0}, the state {N} wi be denoted by the symbo {1}, and assign the new notation to the remaining states {1, 2,..., N 1} as {2, 3,..., N}. Thus, in the set of states X {0, 1,..., N} the states {0} and {1} wi be boundary and absorbing states, and the states {2, 3,..., N} - interna, aowabe, non-returnabe. At the initia moment t 0 0 the process starts from an admissibe state 0 {2, 3,..., N}. The evoution { of} the process is described with absorbing Marov chain ξ 0, in which boundary states are absorbing, and interna admissibe states. As is nown from Marovian processes theory, at some moment 0 the process fas into one of the absorbing states; in this exampe ξ After that, an externa infuence contro, as a resut of which the process is transated into some the interna admissibe state 1 {2, 3,..., N } with probabiity ; 1. A simiar externa infuence 2 produced when the process is absorbed in state 0, i.e. if ξ 0 0 0, is described by the discrete probabiity distribution α 0, 2, 3,..., N. After each infuence, irrespective of the past the process begins to evove from the state 1 aong { } the trajectory of the new absorbing Marov chain ξ 1, whose probabiistic characteristics coincide with { } the characteristics of the chain ξ 0. At the moment of faing into one of the boundary absorbing states, an externa contro infuence is again performed, which is described with the discrete probabiity distributions α 0, 2, 3,..., N,, 2, 3,..., N. V. FORMAL CONSTRUCTION OF A STOCHASTIC MODEL IN THE FORM OF A MARKOV ROCESS WITH DISCRETE TIME We now turn to the forma construction of a stochastic Marov mode with discrete time and a finite set of states X {0, 1,..., N}, describing the functioning of a system with periodic externa infuences. We note first that a stochastic objects introduced in the future are assumed to be given on the same initia probabiity space Ω, A,. This space formaizes a random experiment conducted in objective reaity with the system in question. The concept of probabiity space and its properties are described in detai, for exampe, in research 17], 18], 19]. Suppose that a sequence of independent Marov chains { ξ n }, n 0, 1, 2,... is given. We note in particuar that these chains are uncontroabe and describe the evoution of the system under consideration over time periods between successive externa infuences. Foowing the genera concept of the proposed stochastic mode described in Section IV, the evoution of a discretetime Marov process, which wi pay the roe of the basis of this mode, can be described as foows. Suppose that { } some Marov chain ξ n with fixed number n begins to evove in one of the admissibe states {2, 3,..., N}. After a{ finite } time after the beginning of the evoution, the process ξ n with probabiity 1 fas in one of the boundary { } states {0} or {1}. After the process ξ n is absorbed, the mode is subjected to an externa infuence, which consists in the transition from the absorbing state to one of admissibe interna states. Note that such an assumption is fundamentay important for buiding a mode. His anaytica expressions and probabiistic content wi be expained ater. After the transition to the admissibe state {2, 3,..., N} the evoution of the system wi continue irrespective of the

6 { } past and described by the Marov chain ξ n+1 whose probabiistic{ characteristics } coincide with the characteristics of the process ξ n. The further evoution of the process describes anaogousy. Let ν n denote the random { } time from the beginning of the evoution of the process ξ n in one of the interna states of {2, 3,..., N} to the moment of absorption, n 0, 1, 2,.... It foows { from } the properties of the absorbing Marov chains ξ n, n 0, 1, 2,... that the random variabes { ν n, n 0, 1, 2,... } are independent and with a probabiity equa to one, tae ony finite vaues. The distributions of these quantities are the same and depend { } on the initia vaue of the corresponding Marov chains, n 0, 1, 2,.... We ξ n introduce a sequence of random variabes ν n, 0, 1, 2,..., defined by the reations ν 0 ν 0, n ν n ν i + n, i0 n 1, 2,.... Now } we define a random process with discrete time { ξ, which we wi ca the main one in the foowing. We fix some initia state of the process ξ 0 { ξ 0 } 0 0 {2, 3,.}.., N} and assume that the initia state of the process { ξ coincides with the indicated state: } ξ 0 ξ The further evoution of the process { ξ over a period of time before the first entry into one of the boundary states is determined as foows ξ ξ 0, 0, 1, 2,..., ν0. 1 Suppose that the condition ν 0 ν 0 } 0 is satisfied. At the time of 0 the process { ξ is in one of the boundary } states s 0 {0, 1}. The behavior of the process { ξ after getting into this state does not depend on the past, that is, on its behavior up to the moment 0, and is determined by the ratio. ξ0+1 1 ξ 0 0, ξ 1 i 1,..., ξ 0 1 i 0 1, ξ 0 s 0 ξ0+1 1 ξ 0 s 0 α s0 1, 2 1 {2, 3,..., N}, s 0 {0, 1}, where i 1, i 2,..., i 0 1 {2, 3,..., N} } arbitrary states that form the trajectory of the process { ξ on the time period {1, 2,..., 0 1}. rovided that ξ 0+1 1, we assume ξ 1 0 ξ and, further, ξ 0++1 ξ 1, 1, 2,..., ν1 3 : Suppose that for some vaue n, n 0, 1, } 2,... the moment of the n-th hit of the main process { ξ to one of the boundary states is fixed: ν n n, and } ξ n s n, s n {0, 1}. Then the further behavior of the { ξ process does not depend on the past and is determined by the ratio ξn+1 n+1 ξ 0 0, ξ 1 i 1,..., ξ n 1 i n 1, ξ n s n ξn+1 n+1 ξ n s n n+1 {2, 3,..., N}, s n {0, 1}, α sn n+1, 4 where i 1, i 2,..., i n 1 } arbitrary states that form the trajectory of the process { ξ on the time period {1, 2,..., n 1}. rovided that ξ n+1 n+1, we assume ξ n+1 0 ξ n+1 n+1, } and the subsequent trajectory of the process { ξ is determined by the reation process ξ n++1 ξ n+1, 1, 2,..., ν n+1 5 Reations 1-5 competey } describe the behavior of the main stochastic process { ξ. } A random sequence { ξ is a Marov chain. This { } foows from the Marov property of the sequences ξ n, n 0, 1, 2,... and the aw of transitions from boundary to interna states, which are determined by reations 2, 4. The random variabes ν n, n } 0, 1, 2,... are the points in time at which the process { ξ reaches the boundary vaues of the set of states, that is, one of the {0, 1} states. After reaching an arbitrary boundary state, a contro } is made, which consists in transferring the process { ξ to one of the interna states. This transferring is described by a probabiity distribution α 0 α 0, 2, 3,..., N,, 2, 3,..., N. We say that a given pair of probabiity distributions α 0, forms } a strategy for controing the main random process { ξ. At the same } time, the process { ξ is uncontroabe over time periods between hits to the boundary states. The mathematica probem of optima contro in this stochastic mode is to{ find a pair of probabiity } distributions α 0 α 0, 2, 3,..., N, { }, 2, 3,..., N, which deiver an extremum to some stationary cost indicator of efficiency. The definition of such an indicator and the forma formuation of the optima contro probem wi be given ater. We introduce another auxiiary stochastic object, namey, the random sequence {ζ n } n0, associated with the main

7 } process { ξ We wi assume that the process {ζ n }. n0 is determined by the ratio ζ n ξ ν n, n 0, 1, 2,.... Thus, the vaues of the random sequence {ζ n } n0 coincide with the vaues of the main process at the moments when the atter fas into the boundary states. Obviousy, the eements of the random sequence {ζ n } n0 tae vaues in the set Z {0, 1}. Note that the introduced sequence of random variabes {ζ n } n0 forms a Marov chain. Indeed, we fix a random time point ν n n and the state of the process ζ n ξ n s n {0, 1}. Then the further evoution of the process {ζ n } n0 wi } be determined by the evoution of the main process { ξ after the time point n. In turn, the } evoution of the process { ξ immediatey after the n moment wi be determined by the probabiity distribution α sn α sn, 2, 3,..., N and the Marov chain { } transition probabiity matrix. ξ n+1 Let us expain the ast remar in more detai. Under { the } above conditions, the initia state of the chain ξ n+1 is determined by the distribution of α sn, 2, 3,..., N. Further, at a fixed initia state α sn ξ n+1 {0 } n+1 {2, 3,..., N} the probabiity that the chain ξ n+1 enters one of the absorbing states s n+1 {0, 1} is determined. This characteristic is caed the absorption probabiity and is expressed in terms of the eements of the transition matrix { } of the Marov chain ξ n+1, the corresponding formua wi be given in the next section. Thus, under the conditions ν n n and ζ n ξ n s n the vaue of the chain {ζ n } n0 at the next time instant ζ n+1 s n+1 does not depend on the past and is determined by the given probabiity characteristics. In accordance with the terminoogy adopted in the theory of Marov processes, the introduced Marov chain {ζ n } n0 can { } be caed a chain embedded in the main random process ξ. A nested Marov chain {ζ n } n0 wi pay an important roe in anayzing the properties of the constructed stochastic mode and determining the necessary probabiity characteristics. VI. MAIN CHARACTERISTICS OF THE STOCHASTIC MODEL In this section, theoretica resuts for absorbing Marov chains wi be used 16]. { } We assume that for Marov chains ξ n, n 0, 1,... the foowing matrix probabiistic characteristics are given: 00 - matrix of transition probabiities within the set of admissibe states {2, 3,..., N}, has dimension N 1 N 1; 01 - matrix of transition probabiities from admissibe states {2, 3,..., N} to absorbing states {0, 1} in one step of the chain, has dimension N 1 2; 10 - is the matrix of transition probabiities from absorbing states {0, 1} to the admissibe states {2, 3,..., N}. This matrix is zero, has dimension 2 N 1; 11 - matrix of transition probabiities from absorbing states {0, 1}. This matrix is singe, has dimension 2 2. { Then } the transition probabiity matrix of the Marov chain ξ n with an arbitrary number n has the foowing ceuar structure Suppose that the foowing cost characteristics of the mode are given. Denote as} c the income for a singe stay of the main process { ξ in the state {2, 3,..., N} in the period of free evoution without externa infuences. Let c c, {2, 3,..., N} T be the coumn vector of these revenues. Let denote as d s i the vaue of the costs associated with transferring the main process from the boundary state s to the interna state i, s {0, 1}, i {2, 3,..., N}. These costs characterize the price of externa infuence, which determines the transfer of the process. In accordance with their economic content, these vaues are negative. We obtain representations for some additiona probabiistic and cost characteristics of the mode in question, based on the theory of absorbing Marov chains 16]. Let b{ i0, b i1 } be the probabiities of absorption of the Marov chain ξ n, n 0, 1, 2,... in states {0} and {1} respectivey, provided that at the initia time this process is in the i state; ξ n 0 i, i {2, 3,..., N}; note that b i1 1 b i0, i {2, 3,..., N}. Further, r i is the expectation of the income associated with { } the behavior of the Marov chain ξ n, n 0, 1, 2,... for a period of time before absorption, provided that at the initia moment of time this process is in the state i; ξ n 0 i, i {2, 3,..., N}. In this mode, it is assumed that the { income } associated with the behavior of the Marov chains, n 0, 1, 2,..., defined by the parameters c ξ n c, {2, 3,..., N} T, given above, generate incomes } on the corresponding trajectories of the main process { ξ. Then the absorption probabiity matrix B b i0, b i1, i {2, 3,..., N} T is defined by the formua B I , where I - is unit matrix of the corresponding dimension. The vector r r i, i {2, 3,..., N} T can be expressed as foows r I 00 1 c. Thus, to obtain subsequent resuts on soving the optima contro probem in the stochastic mode under consideration, it is necessary to specify the transition probabiity matrix, the income vector characterizing the evoution of the main process over time periods without externa infuences c, and the set of d s i, i {2, 3,..., N}, s {0, 1}, vaues characterizing the costs of externa infuences or main process contros. The remaining necessary probabiistic and cost characteristics are determined on the basis of the above anaytica formuas.

8 VII. ON THE ANALYTICAL RESENTATION OF THE STATIONARY COST INDICATOR OF MANAGEMENT EFFICIENCY We now turn to investigation of the contro probem in a stochastic Marov mode with discrete time and periodic outputs to the state-set boundary described in Section IV of this paper. For this purpose, we introduce the concept of a cost additive functiona associated with a Marov chain. This functiona describes a random income or profit arising from the evoution of the corresponding economic system. Aso functionas are nown in the scientific iterature for exampe 17], chapter 8. Let there be given a Marov chain {θ } with a discrete set of states Z {0, 1, 2,...}. The set of states of a given chain can be finite or countabe. Let define a function D : Z R, which can aso be defined as the set of its vaues ρ i Di, i Z. We wi interpret ρ i as revenue positive or negative obtained by one stay of the process {θ } in state i, i Z. Consider a random sequence γ n n Dθ, n {0, 1,...} Definition 1. A random sequence discrete-time process {γ n } n0 wi be caed the cost additive functiona associated with the Marov chain {θ }. In the course of the further presentation we wi use various concepts and properties from the theory of Marov chains. A detaied presentation of this theory can be found in the foowing fundamenta editions: 16], 18], 19], 20]. We formuate a statement characterizing the behavior of the cost additive functiona {γ n } n0 for a ong evoution of the process {θ }. Statements of this ind are usuay caed ergodic theorems. Let us quote this statement, foowing 17], chapter 8. Theorem. Let the Marov chain {θ } be irreducibe, recurrent, and positive. Suppose aso that the foowing condition hods ρ π <, where π π, Z is the stationary distribution of the Marov chain {θ }. Then for any initia state i 0 Z the foowing reation hods. 1 I im n n Eγ n θ 0 i 0 ] ρ i π i 6 i Z It is natura to ca the vaue on the right-hand side of reation 6 the average stationary specific income associated with the Marov chain {θ }. Returning to the study of the introduced stochastic mode with discrete} time and contros at the time when the main process { ξ reaches the boundary of a given subset of the set of states, we wi consider the Marov chain {ζ n } n0 buit in Section V embedded in the main process as the Marov chain {θ }. The cost characteristics of the mode, defined in Section VI, wi define the additive } cost functiona associated with the main process { ξ and the nested Marov chain {ζ n } n0. According to its economic content, this functionaity wi be a random profit accumuated over a certain period of time. Under some conditions, which wi be formuated beow, one can appy the reduced ergodic theorem. Then the vaue I, determined by the reation 6, wi have the meaning of the average specific profit. Foowing the cassica papers on the theory of contro of Marovian and semi-marov stochastic processes 21], 22], we consider the vaue I as an indicator of contro effectiveness in the mode under consideration. Note that simiar performance indicators are considered in modern studies on the theory of stochastic contro: 23], 24]. In the future it wi be proved that the stationary cost indicator 6 can be represented expicity, through the initia probabiistic and cost characteristics of the mode, defined in Section VI. We note that this indicator depends on the discrete probabiity distributions α 0 α 0 i, i {2, 3,..., N}, j, j {2, 3,..., N}, that specify basic stochastic process contro strategy } { ξ and nested Marov chain {ζ n } n0. Thus, the optima contro probem in this mode can be formuated as the extrema probem I I α 0, extr, α 0, Γ d 7 where Γ d is the set of pairs of vectors of discrete probabiity distributions defined on a finite set of admissibe contros U {2, 3,..., N}. To sove the optima contro probem 7, it is necessary to estabish the structure of the dependence of the functiona I α 0, on the discrete probabiity distributions α 0,. However, before formuating and proving the corresponding resut, we mae severa important remars on the features of the constructed mode and the stated optima contro probem. } Remar 1. The main process contro scheme { ξ considered in this stochastic mode differs from the standard contro scheme adopted in cassica papers 21], 22] and in many subsequent studies. In this mode, decisions about the choice of contro are taen } at some random points in time at which the process { ξ reaches the boundary states. In the standard contro scheme of Marov and semi-marov stochastic processes, decisions about the choice of contro are taen at each moment when a change in the state of the process occurs. } Thus, the probem of optima contro of the process { ξ, which is in form the probem of tuning for a mode with discrete time, is not reduced to the cassica formuation of the probem of stochastic contro with respect to stationary cost indicators of the form 6. Remar 2. We introduce an important assumption reated to the initia probabiity characteristics of the mode. Namey, we wi assume that the absorption probabiities of a Marov

9 { chain ξ n } with an arbitrary number n {0, 1, 2,... } satisfy condition b 0 > 0, b 1 > 0, {2, 3,..., N}; at the same time, as aready noted, b 0 + b 1 1, {2, 3,..., N}.This condition means that a Marov chain with an arbitrary number n {0, 1, 2,... } in a finite time with a probabiity equa to one reaches one of its absorbing states: } {0} or {1}. For the constructed Marov mode { ξ this condition has the foowing probabiistic meaning. As a resut of the } next externa infuence contro, the Marov process { ξ is transferred to one of the interna admissibe states } {2, 3,..., N}. After this, the main process { ξ with a probabiity equa to one reaches one of its boundary states {0} or {1}. Next, the foowing externa infuence contro is performed, as a resut of which the process wi be transferred to one of the interna aowabe states. The further evoution } of the process { ξ occurs independenty of the past and according to the aws described above. Thus, this condition } ensures an infinitey ong evoution of the process { ξ regardess of the contros made at the moments when the set of states reaches the boundary. This property can be caed the stabiity of the constructed stochastic mode. We emphasize that it wi be carried out for any strategy for controing the process { ξ }, which is determined by probabiity distributions α 0, α 0. As wi be shown ater, this condition aso ensures the existence of a singe stationary distribution of some } auxiiary Marov chain embedded in the process { ξ. In the future, we wi require the fufiment of this condition when proving the main resuts. Now we can proceed to the formuation and proof of the statement about the expicit representation of the stationary cost indicator of the effectiveness of contro. Theorem 1. Suppose that the foowing conditions are met in the stochastic mode under consideration: b,0 > 0, b,1 > 0, {2, 3,..., N}. Then the foowing representation taes pace for the stationary cost indicator I I α 0,, defined by the reation 6. I I α 0, where Am 0, m 1 m 02 m 12 m 02 m 12 Am 0, m 1 α 0 m 0 m 1 Bm 0, m 1 α 0 m 0 m 1 8 ] ] d 0 m 0 + r m0 b m1,0 + d 1 m 1 + r m1 b m0,1 9 Bm 0, m 1 b m0,1 + b m1,0 10 roof. We first prove the foowing emma reated to the probabiity characteristics of the auxiiary Marov chain {ζ n } n0. Lemma 1. Suppose that conditions b,0 > 0, b,1 > 0, {2, 3,..., N} are satisfied. Then the transition probabiity of the Marov chain {ζ n } n0 is determined by p ij ζ n+1 j ζ n i i, j Z {0, 1} 2 α i b j, 11 roof of Lemma 1. During the proof, various random events reated to} the trajectories of the main process wi be considered { ξ. In accordance with the initia assumptions about the construction of a stochastic mode Section V, a these events are defined on the probabiity space Ω, A,, that is, eements of the σ- agebra A. We } fix arbitrary consecutive moments of the process { ξ entering the boundary states: ν n n, ν n+1 n+1 and we wi consider random } events associated with the trajectories of the process { ξ on the time interva ] n, n+1]. Now we fix the state i Z {0, 1} and consider the random event ξn i. Events that wi be introduced ater wi be considered under the condition that event ξn i occurred, that is, on the set of eementary outcomes corresponding to this event. Note that if condition ξn i, is satisfied, that is, at } time n process { ξ is in the boundary state i, then by the property of the adopted stochastic mode at time n + 1 it wi be transferred to one of the interna aowabe states {2, 3,..., N}, that is, one of the system ξn+1, {2, 3,..., N} wi be impemented. Thus, there is an embedding N ξn i ξn+1, i Z {0, 1} 2 } Events from system { ξn+1, {2, 3,..., N} are pairwise incompatibe. At the same time, each of them together with the event ξn i, since as a resut of the contro, the transfer from the boundary state i Z can be made to any interna state {2, 3,..., N}. We next consider event ξn+1 j, j Z {0, 1} - a certain boundary state. This event can be reaized ony when, as} a resut of the previous contro, the main process { ξ was transferred to one of the interna states {2, 3,..., N}, that is, one of the system of incompatibe events { ξn+1, {2, 3,..., N}} occurred. Therefore, it can be argued that on the set of outcomes of this experiment, corresponding to the fixed condition ξn i, there is an embedding of events N ξn+1 j 2 ξn+1, j Z {0, 1}.

10 From the comments made it foows that the set of tota α i probabiity is appicabe on the set of eementary outcomes b j, i, j {0, 1} corresponding to the event ξn i. In this case, the roe of the main event, the probabiity of which is necessary It remains to be noted that, within the framewor of the adopted mode, there is a coincidence of events ξn s to determine, wi pay the event ξn+1 j, and the roe of the system of incompatibe } hypotheses events-set ζ n s, s {0, 1}, n 0, 1, 2,.... The assertion of Lemma 1 foows from equaity 17. { ξn+1, {2, 3,..., N}. The probabiities of a roof of Theorem 1. specified events are determined under condition ξn i. Consider the properties of the auxiiary Marov chain Based on the formua of tota probabiity, we have {ζ n } n0. If conditions b 0 > 0, b 1 > 0, {2, 3,..., N}, ξn+1 j ξ are satisfied, then the statement of Lemma 1 reation 11 n i impies that p ij > 0, i, j {0, 1}. Thus, the states of a given chain are interconnected and form one cass, that is, ξn+1 j ξ n i, ξn+1 ξn+1 ξ n i 2 12 At the same time, by the property of the constructed mode see reations 2, 4 and the corresponding expanations for them. ξn+1 ξ n i α i, {2, 3,..., N}, i {0, 1}. 13 We use another property of the constructed stochastic mode. After the next hitting into the boundary state } and the produced contro, the evoution of the process { ξ wi depend ony on the state to which the process was transferred as a resut of the contro. It foows that for any i, j {0, 1}, {2, 3,..., N} there is an equaity ξn+1 j ξ n i, ξn+1 ξn+1 j ξ n+1 14 } The trajectory of the process { ξ on the time interva n + 1, n+1 ]{ coincides } with the trajectory of the absorbing Marov chain ξ n+1 see reation 5 and the corresponding expanations. Thus, the transition probabiity on the right-hand side of equaity 14 { wi coincide } with the absorption probabiity of the chain. ξ n+1 ξn+1 j ξ n+1 ξ n+1 n+1 n 1 j ξn+1 0 b j, 15 {2, 3,..., N}, j {0, 1}. From 14 and 15 it immediatey foows that for any fixed i {0, 1} ξn+1 j ξ n i, ξn+1 b j 16 {2, 3,..., N}, j {0, 1}. Substituting 13 and 16 into reation 12, we obtain ξn+1 j ξ n i this Marov chain is irreducibe. Moreover, the set of states of the chain Z {0, 1} is finite; it foows from this that a states are recurrent and positive. As is we nown 16], 17], a Marov chain with the indicated properties has a unique stationary distribution π π 0, π 1, which satisfies the system of equations π 0 π 0 p 00 + π 1 p 10 π 1 π 0 p 01 + π 1 p 11 π 0 + π The soution of the system of equations 18 is π 0 π 1 p 10 1 p 00 + p 10 1 p 00 1 p 00 + p 10 p 10 p 01 + p 10, p 01 p 01 + p Substituting into equation 19 the formuas for transition probabiities 11 and we obtain π 0 π b 0 20 α 0 b 1 + N b α 0 b 1 21 α 0 b 1 + N b 0 2 We now define the cost additive functiona associated with the auxiiary Marov chain {ζ n } n0. We wi assume that the income ρ i, obtained by a singe stay of this process in state i Z {0, 1}, coincides with the average profit generated in the system under consideration for the time } eapsed from the moment the main stochastic process { ξ hits the boundary state i unti the next state fas into the boundary state. Let v n be the random vaue of the profit generated in the system for the specified period of time. Then ] ρ i E v n ξ n ζ n i, i Z {0, 1} 22.

11 We cacuate the magnitude of the conditiona expectation, which is ocated on the right-hand side of equaity 22. By the property of mathematica expectation ] ρ i E v n ξ n ζ n i 2 ] E v n ξ n ζ n i, ξn+1 ξn+1 ξ n ζ n i, i {0, 1} 23 For the stochastic mode under consideration, the increment in profit over } the time between successive hits of the main process { ξ into boundary states is made up of the costs associated with transferring this process to one of the interna states and random income received during the free evoution unti the next hit to one of the boundary states. From here foows ] E v n ξ n ζ n i, ξn+1 d i + r, {2, 3,..., N}, i {0, 1} 24 From 23, taing into account 24 and 13, we obtain ] rho i E v n ξ n ζ n i 2 d i + r ] α i, i {0, 1} 25 As aready noted, the auxiiary Marov chain {ζ n } n0 is irreducibe, recurrent, and positive. Then the above ergodic theorem for the additive cost functiona associated with the stochastic mode under consideration and the Marov chain {ζ n } n0 is vaid. From the statement of this theorem and reation 6 it foows I ρ 0 π 0 + ρ 1 π 1 26 Substituting into 26 formuas 20, 21, 25, we obtain the foowing representation m2 α m 0 d 0 m + r m ] N 2 I b 0 + N m2 α m 1 d 1 m + r m ] N 2 α 0 b 1 α 0 b 1 + N b Let us transform the expressions in the numerator and denominator of the right side of formua 27. Consider the numerator I 1 α 0, m2 α m 0 d 0 m + r m ] N 2 b 0 + m2 α m 1 d 1 m + r m ] N 2 α 0 b 1 We write the products of sums in the form of a doube sum of pairwise products I 1 α 0, ] d 0 m + r m b 0 α m m2 2 m2 2 d 1 m + r m ] b 1 α 0 m 28 Now we carry out redesignations of the summation indices in the first and second terms of the right-hand side of equaity 28. In the first doube sum we put m m 0, m 1, and in the second m 0, m m 1. After that, combine both amounts into one and get m 0 2 m 1 2 I 1 α 0, ] ] ] d 0 m 0 + r m0 b m1,0 + d 1 m 1 + r m1 b m0,1 α m 0 0 α m Now consider the representation of the denominator on the right side of formua 27 I 0 α 0, 2 α 0 b b 0 We perform the identity transformation of the expression I 0 α 0, taing into account the normaization conditions for probabiity distributions α 0,. I 0 α 0, m2 2 m2 m 2 b 1 α 0 m + α 0 b 1 + m2 2 m2 α 0 m 2 b 0 b 0 α 0 m 30 Next, we wi redesign the summation indices in the first and second terms of the right-hand side of equaity 30 in the same way as was done above when converting the numerator, and then combine these sums into one. Then I 0 α 0, m 02 m 12 b m0,1 + b m1,0] α 0 m 0 m 1 31 From equaity 27, taing into account the transformations of expressions for the numerator and denominator, given by formuas 29, 31, we obtain the representation for the stationary cost indicator I α 0, in the form 8. In this case, the functions Am 0, m 1, Bm 0, m 1 are determined by formuas 9, 10, respectivey. Theorem 1 is proved. From the statement of Theorem 1, it foows that in the stochastic mode under consideration, the stationary cost indicator of contro efficiency I α 0, is represented as a inear fractiona integra functiona defined on a set of pairs of discrete probabiity distributions α 0,, each of which defines a contro strategy. In this regard, to sove the optima contro probem, which is formuated as an extrema probem 7, it is necessary to use the theoretica resuts for the unconditiona extremum probem of a functiona of this form. summary of these resuts wi be given in the next section.