Optimal control of a stochastic network driven by a fractional Brownian motion input

Transcription

1 Optimal control of a stochastic network driven by a fractional Brownian motion inpt arxiv:88.299v [math.pr] 8 Ag 28 Arka P. Ghosh Alexander Roitershtein Ananda Weerasinghe Iowa State University Agst 5, 28 Abstract We consider a stochastic control model driven by a fractional Brownian motion. This model is a formal approximation to a qeeing network with an ON-OFF inpt process. We stdy stochastic control problems associated with the long-rn average cost, the infinite horizon disconted cost, and the finite horizon cost. In addition, we find a soltion to a constrained minimization problem as an application of or soltion to the long-rn average cost problem. We also establish Abelian limit relationships among the vale fnctions of the above control problems. MSC2: primary 6K25, 68M2, 9B22; secondary 9B8. Keywords: stochastic control, controlled qeeing networks, heavy traffic analysis, fractional Brownian motion, self-similarity, long-range dependence. Introdction Self-similarity and long-range dependence of the nderlying data are two important featres observed in the statistical analysis of high-speed commnication networks in heavy traffic, sch as local area networks LAN) see for instance [, 2, 25, 26, 3, 34, 35] and references therein). In theoretical models sch traffic behavior has been sccessflly described by stochastic models associated with fractional Brownian motion fbm) see [4, 5, 9, 3, 32, 33]). It is well known that fbm exhibits both of these featres when the associated Hrst parameter is above. Therefore, nderstanding the behavior and control of these 2 stochastic models are of significant interest. The non-markovian natre of the fractional Brownian motion makes it qite difficlt to stdy stochastic control problems for a state process driven by fbm. The techniqes sch as dynamic programming and analysis of the corresponding Hamilton-Jacobi-Bellmann eqations which are the commonly sed tools in Research spported by National Science Fondation grant DMS Research spported by US Army Research Offce grant W9NF7424.

2 the analysis of the stochastic control problems associated with the ordinary Brownian motion are not available for fbm-models. In this paper, we stdy several basic stochastic control problems for a qeeing model with an inpt described by a fractional Brownian motion process. Similar qeeing models, bt not in the context of control of the state process, were considered for instance in [8, 23, 36]. We are aware of only a few solvable stochastic control problems in fbm setting. Usally, the controlled state process is a soltion of a linear semi-linear in [7]) stochastic differential eqation driven by fbm, and the control typically effects the drift term of the SDE. In particlar, the linear-qadratic reglator control problem is addressed [6, 7] and a stochastic maximm principle is developed and applied to several stochastic control problems in [3]. We refer to [6] and to Chapter 9 of the recent book [4] for frther examples of control problems in this setting. In contrast to the models considered in the above references, the model described here is motivated by qeeing applications and involves processes with stateconstraints. At the end of this section, we discss an example of a qeeing network which leads to or model. Or analysis relies on a copling of the state process with its stationary version see [8, 36]) which enables s to address control problems in a non-markovian setting, and or techniqes are different from those employed in [3, 7, 6, 7]. A real-valed stochastic process W H = W H t) ) is called fractional Brownian motion t with Hrst parameter H,) if W H ) = and W H is a continos zero-mean Gassian process with stationary increments and covariance fnction given by Cov W H s),w H t) ) = 2 [t2h +s 2H t s 2H ], s, t. The fractional Brownian motion is a self-similar process with index H, that is for any a > the process a H WH at) ) has the same distribtion as W t H t) ). If H = then W t 2 H is an ordinary Brownian motion, and if H [,) then the increments of the process are 2 positively correlated and the process exhibits long-range dependence, which means that Cov W H ),W H n+) W H n) ) =. n= Notice that fbm is a recrrent process, and in particlar, for any >, lim W H t)/t = t a.s. and conseqently lim WH t) t ) = a.s. For additional properties and a more t detailed description of the process we refer to [22, 24, 27, 28]. We consider a single server stochastic processing network having deterministic service process with rate µ >. For any time t, the cmlative work inpt to the system over the time interval [,t] is given by λt + W H t), where λ is a fixed constant and W H is a fractional Brownian motion with Hrst parameter H [,). We assme that the service 2 rate µ satisfies µ > λ and that the parameter µ can be controlled. The workload present in the system at time t is given by Xx t) which is defined in 2.) and 2.2) below. Here x is the initial workload and = µ λ > is the control variable. Assming for simplicity that x =, an eqivalent representation for the process Xx is given by see 2.2) below for the general case) X x t) = W H t) t ) inf s [,t] 2 WH t) t ), t..)

3 For a given arrival process W H, this is a common formlation for a simple stochastic network where the server works continosly nless there are no cstomers in the system. The first term above represents the difference between the cmlative nmber of job arrivals and completed services in the time interval [,t], and the last term ensres that the the qeelength is non-negative, and it is a non-decreasing process which increases only when the qee-length process is zero. For more examples of sch formlations for qeeing networks or stochastic processing networks, we refer to [] and [33]. The above qeeing model with W H in.) being a fractional Brownian motion was considered by Zeevi and Glynn in [36]. In particlar, we are motivated by their work. Or goal is to address several stochastic control problems related to the control of the workload process Xx described above. The organization of the paper is as follows. We will conclde this section with a motivating example of a qeeing network which leads to or model. In Section 2, we introdce the model and describe three basic stochastic control problems associated with it, namely the long-rn average cost problem, the infinite horizon disconted cost problem, and the finite horizon control problem. Here we also discss some properties of the reflection map which will be sed in or analysis. In Section 3, we stdy the long-rn average cost problem. Here we obtain an explicit deterministic representation of the cost fnctional for each control >. This enables s to redce the stochastic control problem to a deterministic minimization problem. We obtain an optimal control > and show its niqeness nder additional convexity assmptions for the associated cost fnctions. We show that the vale fnction and the corresponding optimalcontrolareindependent oftheinitialdata. Itiswellknownthattheabovepropertyis tre for the classical long-rn average cost problem associated with non-degenerate diffsion processes. Here we show that it remains valid for or model driven by fbm which is highly non-markovian. The main reslts of this section are given in Theorems 3.4 and 3.5. Their generalizations are given in Theorems 3.6 and 3.7. Or proofs here rely on the se of a copling of the nderlying stochastic process with its stationary version introdced in [8]. In particlar, we show that the copling time has finite moments in Proposition 3.2. Becase of the highly non-markovian character of the fractional Brownian motion it is well known that fbm cannot be represented as a fnction of a finite-dimensional Markov process), copling argments in general do not work for the models associated with fbm we refer to [3] for an exception). In or case, the copling is available de to the niqeness reslts related to the reflection map described in Section 2. We se or reslts in Section 3 to obtain an optimal strategy for a constrained optimization problem in Theorem 4.2 of Section 4. Similar stochastic control problems for systems driven by an ordinary Brownian motion were previosly considered in [, 3]. An interesting application of this model to wireless networks is discssed in []. Or constrained optimization problem in the fbm setting) is a basic example of a general class of problems with an added bonded variation control process in the model. This class of problems has important applications to the control of qeeing networks, bt in fbm setting, it seems to be an nexplored area of research. In Section 5, we address the infinite horizon disconted cost problem associated with a similar cost strctre. An optimal control for this problem is given in Theorem 5.3. In Section 6, we establish Abelian limit relationships among the vale fnctions of the three stochastic control problems introdced in Section 2. The main reslt of this section 3

4 is stated in Theorem 6.3. We show that the long term asymptotic for the finite horizon control problem and the asymptotic for the infinite horizon disconted control problem, as the discont factor approaches zero, share a common limit. This limit trns ot to be the vale of the long-rn average cost control problem. Or proof also shows that the optimal control for the disconted cost problem converges to the optimal control for the long-rn average cost control problem when the discont factor approaches zero. A similar reslt holds also for the optimal control of the finite horizon problem, as the time horizon tends to infinity. For a class of controlled diffsion processes analogos reslts were previosly obtained in [3]. Motivating example. We conclde this section with a description of a qeeing network related to the internet traffic data in which the weak limit of a sitably scaled qee-length process satisfies 2.) and 2.2) which are redced to the above.) when the initial workload x eqals ). For more details on this model we refer to [3] and Sections 7 and 8 of [33]. We begin by defining a seqence of qeeing networks with state dependent arrival and service rates, indexed by an integer n and a non-negative real-valed parameter τ. For each n and τ, the n,τ)-th network has only one server and one bffer of infinite size, and the arrivals and departres from the system are given as follows. There are n inpt sorces e.g. n sers connected to the server), and job reqirements of each ser is given by the so-called ON-OFF process {X n,τ i,i } as defined in [3], namely each ser stays connected to the server for a random ON-period of time with distribtion fnction F, and stays off dring a random OFF-period of time with distribtion fnction F 2. The distribtion F i is assmed to have finite mean m i bt infinite variance, and in particlar, F i x) c i x α i, where < α i < 2 for i =,2. While connected to the server, each ser demands service at nit rate sends data-packets at a nit rate to the server for processing). The server is processing sers reqests at a constant rate, say µ n,τ. Assme that the ON and OFFperiods are all independent for each ser as well as across sers), the ON-OFF processes have stationary increments, and average rate of arrival of jobs packets) from each sorce cstomer) is given by λ = m /m +m 2 ). The qee-length at time t is given by X n,τ t) = X n,τ )+ n i= t X n,τ i s)ds µ n,τ t+l n,τ t), where L n,τ is a non-decreasing process that starts from, increases only when X n,τ is zero, and ensres that X n,τ is always non-negative. Physically, this implies that the server is nonidling, i.e it serves jobs continosly as long as the bffer is non-empty. The second term in the right-hand side of the above eqation represents cmlative nmber of packets sent to the server by all the n cstomers in the interval [,t]. We will assme that X n,τ ) = x n,τ, where x n,τ are fixed non-negative real nmbers for each n and τ. In this setp, τ > represents the time scaling parameter, and it is well known see [29] or Theorem of [33]) that τ H n 2 n i= τ X n,τ i s)ds W H ), 4

5 when n first and then τ. Here W H is a fractional Brownian motion with Hrst parameter H = 3 min{α,α 2 })/2,), and the convergence is the weak convergence 2 in the space D [, ),R ) of right-continos real fnctions with left limits eqipped with the standard J α, topology see [33] for details). From the above convergence reslt it can be derived see Theorem 8.7. in [33]) that if the service rates µ n,τ satisfy the following heavy traffic assmption τ H n 2 n µ nτ nλ)τ, i= as n,τ), then a sitably scaled qee-length satisfies eqations 2.) and 2.2). More precisely, if the above heavy traffic condition is satisfied, and τ H n 2x n,τ x, then the scaled qee-length τ H n 2X n,τ τ ) converges weakly to a limiting process Xx ) that satisfies 2.), 2.2) if we let n first and then τ. Hence, we see that with a sper-imposed ON-OFF inpt sorce and deterministic services times for the qeeing processes, a sitably-scaled qee length in the limit satisfies or model. With a cost strctre similar to either 2.5) or 2.7), or 2.) for the qeeing network problem, one can consider the problem described in this paper as a formal fractional Brownian control problem fbcp) of the corresponding control problem for the qeeing network. We, however, do not attempt to solve the qeeing control problem in this paper. A soltion the limiting control problem provides sefl insights into the qeeing network control problem see for instance [2]). For a broad class of qeeing problems, it has been shown that the vale fnction of the Brownian control problem BCP) is a lower bond for the minimm cost in the qeeing network control problem see [6]). In many sitations, the soltion to the BCP can be tilized to obtain optimal strategies for the qeeing network control problem cf. [2], [5], [9] etc.). Here, we stdy jst the Brownian control problem, which is an important problem in its own right. Or explicit soltion to the fbcp can be considered as an approximate soltion to the qeeing network problem. 2 Basic setp In this section we define the controlled state process Section 2.), describe three standard control problems associated with it Section 2.2), and also discss some basic properties of a reflection mapping which is involved in the definition of the state process Section 2.3). 2. Model Let W H t) ) be a fractional Brownian motion with Hrst parameter H /2 and let σ ) t be a deterministic continos fnction defined on [, ) and taking positive vales. For a given initial vale x and a control variable, the controlled state process Xx is defined by Xx t) = x t+σ)w Ht)+L x t), t, 2.) 5

6 where the process L x is given by L x t) = min{, min x s+σ)wh s) )}, s [,t] t. 2.2) The control variable remains fixed throghot the evoltion of the state process Xx. It follows from 2.) and 2.2) that Xxt) for all t. Notice that the process L x has continos paths, and it increases at times when Xx t) =. The process Xx represents the workload process of a single server controlled qee fed by a fractional Brownian motion, as described in the previos section see also [36]). For a chosen control that remains fixed for all t, the controller is faced with a cost strctre consisting of the following three additive components dring a time interval [t,t+dt] :. a control cost h)dt, 2. a state dependent holding cost C X x) dt, 3. a penalty of pdl x t), if the workload in the system is empty. Here p is a constant, and h and C are non-negative continos fnctions satisfying the following basic assmptions: i) The fnction h is defined on [,+ ), and h is non-decreasing and continos, h), lim h) = ) + ii) The fnctions C is also defined on [, + ), and it is a non-negative, non-decreasing continos fnction which satisfies the following polynomial growth condition: for some positive constants K > and γ >. Cx) K+x γ ) 2.4) We will sometimes assme convexity of h and C in order to obtain sharper reslts, sch as the niqeness of the optimal controls. 2.2 Three control problems Here we formlate three cost minimization problems for or model. In the long-rn average cost minimization problem it is also called ergodic control problem), the controller minimizes the cost fnctional I,x) := limsp T T E = h)+limsp [ h)+c X x t) )] T ) dt+ pdl x t) T T E C Xxt) ) ) dt+pl xt), 2.5) 6

7 sbject to the constraint > for a fixed initial vale x. Notice that since L x t) is an increasing process, the integral with respect to L xt) can be defined an ordinary Riemann- Stieltjes integral. The vale fnction of this problem is given by V x) = inf I,x). 2.6) > In Section 3, we show that I,x) and hence also the vale fnction V x) are actally independent of x. In addition, we show the existence of a finite optimal control > and also prove that is niqe if the fnctions h and C are convex. We apply the reslts on the long rn average cost problem to find an optimal strategy for a constrained optimization problem, in Section 4. In Section 5, we solve the infinite horizon disconted cost minimization problem for the case when σ) in 2.). In this problem it is assmed that the controller wants to minimize the cost fnctional J α,x) := E e αt[ h)+c Xxt) )] ) dt+p e αt dl xt). 2.7) sbject to > for a fixed initial vale x. Here the discont factor α > is a strictly positive constant. The vale fnction in this case is given by V α x) = inf > J α,x). 2.8) We stdy the asymptotic behavior of this model in Section 6. When α approaches zero, we prove that lim α +αj α,x) = I,x) for any control >. Frthermore, we show that lim α +αv αx) = V x) and the optimal controls for the disconted cost problem converges to that of the long-rn average cost problem as α tends to zero. In Section 6, we also consider the finite horizon control problem with the vale fnction Vx,T) defined by where Vx,T) := inf I,x,T), 2.9) > T[ I,x,T) := E h)+c X x t) )] ) dt+pl x T) = h)t +pe T L x T)) +E C ) Xx t)) dt, 2.) Vx,T) and p is a non-negative constant. We prove that lim = V T x). Frthermore, we show that the optimal controls for the finite horizon problem converges to that of the long-rn average cost problem, as T tends to infinity. 2.3 The reflection map The model eqations 2.) and 2.2) have an eqivalent representation which is given below in 2.2) by sing the reflection map. Therefore we briefly discss some basic properties of the reflection map and of the representation 2.2). 7

8 Let C[, ),R) be the space of continos fnctions with domain [, ). The standard reflection mapping Γ : C[, ),R) C[, ),R) is defined by Γ f ) t) = ft)+ sp fs)) +, 2.) s [,t] forf C[, ),R).Hereandhenceforthwesethenotationa + := max{,a}.thismapping is also known as the Skorokhod map or the reglator map in different contexts. For a detailed discssion we refer to [2, 33]. In or model 2.)-2.2), we can write Xx as follows: X xt) = Γ x e+σ)w H ) t), 2.2) where et) := t for t, is the identity map. Notethat according tothe definition, Γ f ) t) fort. Wewill also se thefollowing two standard facts abot Γ see for instance [2, 33]). First, we have sp Γ f ) t) 2 sp ft), 2.3) t [,T] t [,T] for f C[, ),R). Secondly, let f and g be two fnctions in C[, ),R) sch that f) = g) and ht) := ft) gt) is a non-negative non-decreasing fnction in C[, ),R). Then Γ f ) t) Γ g ) t), for all t. 2.4) Weshallalsorelyonthefollowingconvexity propertyofthereflected mapping. Letα,) and f and g be two fnctions in C[, ),R). Then, αf + α)g C[, ),R), and Γ αf + α)g ) t) αγ f ) t)+ α)γ g ) t) 2.5) for all t. The proof is straightforward. Let Fx) = x := max{, x}. Then F is a convex fnction and therefore αfs) + α)gs)) αf s) + α)g s). Conseqently, sp s [,t] αfs) + α)gs)) αsp s [,t] f s) + α)sp s [,t] g s). Since sp s [,t] fs)) + = sp s [,t] f s), the ineqality 2.5) follows from the definition 2.). The reflection map also satisfies the following minimality property: if ψ,η C [, ),R ) are sch that ψ is non-negative, η) =, η is non-decreasing, and ψt) = ϕt) + ηt) for t, then ) +, ψt) Γϕ)t) and ηt) sp ϕs) for all t. 2.6) s [,t] 3 Long-rn average cost minimization problem In this section we address the control problem defined in 2.5)-2.6). First we find a soltion to the control problem for the particlar case when σ) in 2.). This is accomplished in Sections In Section 3.5, we show that the general case can be redced to this simplified version. 8

9 3. Redction of the cost strctre The controlled state space process X x lim corresponding to σ) ) has the form Xx t) = x t+w Ht)+L x t), t. 3.) The following lemma simplifies the expression for the cost fnctional 2.5) by compting E L T x T)). Lemma 3.. Let Xx be given by 3.). Then lim E L T xt) ) =. Proof. Since >, sing 2.2), 2.3), and 2.4), we obtain: Xxt) Xxt) = Γ ) x+w H t) 2 x + sp W H s) ). s [,t] By the self-similarity of fractional Brownian motion process, E sp W H s) ) K T H, s [,T] where K, ) is a constant independent of T see for instance [24, p. 296]). Therefore, E X x T)) 2K x +T H ). 3.2) Conseqently, lim E X T x T)) =.SinceW H T)isamean-zeroGassianprocess, itfollows from 3.) that E L T x T)) = E ) X T x T)) x. Letting T tend to infinity completes the proof of the lemma. Remark. Lemma 3. with literally the same proof as above remains valid if X x satisfies 2.) instead of 3.). With the above lemma in hand, we can represent the cost fnctional2.5) and reformlate the long-rn average cost minimization problem as follows. The controller minimizes I,x) = h)+p ) +limsp T T E C ) Xx t)) dt 3.3) sbject to > andxx givenin3.). Notethattheabove redctionshows thattheoriginal minimization problem 2.5) redces to the case p = with the fnction h) replaced by h)+p. Or next step is to analyze the cost component limsp T E t C X xt) ) ) dt. The following reslts are described in [8] and [8], and are collected in [36] in a convenient form for or application. We smmarize them here sing or notation. i) The random seqence X t) with t converges weakly, as t goes to infinity, to the random variable Z := max s {W Hs) s}. 3.4) 9

10 ii) There is a probability space spporting the processes X, L and hence X x as well as L x for any x ) and a stationary process X = {Xt) : t } sch that and X t) = W Ht) t+max{x ),L t)}, t, 3.5) X t) D = Z, t, 3.6) where D = denotes the eqality in distribtion and Z is defined in 3.4). iii) The tail of the stationary distribtion satisfies where θ ) is given by lim z z2h 2 logpz z) = θ ), 3.7) θ ) = 2H In particlar, all the moments of Z are finite. >. 3.8) 2H 2H 2 H) H) Throghot the rest of the paper, we se this probability space where all these processes are defined. Using 3.) and 2.2), we can write for t, where Xx t) = W Ht) t+max{x,l t)}, 3.9) L t) = inf WH s) s ) = sp s WH s) ). 3.) s [,t] s [,t] The above representation 3.9) and 3.) agrees with 3.5) if the process X x is initialized with X ). 3.2 A copling time The following copling argment is crcial to address the optimal control problems. In particlar, it enables s to deal with the last term of I,x) in 3.3). Proposition 3.2. Let > and the initial point x be fixed. Consider the state process Xx in 3.) and the stationary process X of 3.5) and 3.6). Then the following reslts hold: i) There is a finite stopping time τ sch that X x t) = X t) for all t τ. Frthermore, Eτ β ) < for all β. ii) The cost fnctional I,x) defined in 2.5) is finite and independent of x, that is I,x) = I,) < for x. Conseqently, the vale fnction V x) = inf I,x) > is also finite and independent of x, that is V x) = V ) < for x.

11 Proof. For y > introdce the stopping time λ y = inf{t > : L x t) > x+y}. The stopping time λ y is finite a.s. becase lim t + L x t) lim t WH t) ) = + a.s. t + Define the stopping time τ by τ = inf{t > : L t) > x+x )}. 3.) Here X is the stationary process which satisfies 3.5) and 3.6). That is τ = λ X ) a.s. It follows that for t τ, we have L t) L τ ) = x+x ) and X ). Therefore, it follows from 3.9) and 3.) that X x t) = W Ht) t+l t) = X t) for t τ. Next, we prove that Eτ β ) < + for each β. Clearly, withot loss of generality we can assme that β. We then have: Eτ β ) E λ β m+ ) [ [m X)<m+] Eλ 2β m+ )PX ) m)] /2, 3.2) m= where in the last step we have sed the Cachy-Schwartz ineqality. Since X ) has the same distribtion as Z = sp s {W H s) s}, it follows from 3.7) that for all m large enogh, m= PX ) m) e 2 θ )m 2 H), 3.3) where θ ) is defined in 3.8). Next, we estimate Eλ 2β m) for m and β. For m N, let b m = x+m and T m = 2bm. We have Eλ 2β m ) = 2β T 2β m +2β t 2β Pλ m > t)dt = 2β t 2β PL t) x+m)dt T m t 2β PL t) b m)dt, 3.4) where the second eqality is de to the fact that Pλ m > t) = PL t) x+m) according to the definition of λ m. Notice that P L t) b ) ) m) = P sp {s W H s)} b m P WH t) t b m, 3.5) s [,t] and recall that Z := W Ht) has a standard normal distribtion. Therefore, by 3.5), for t H t > T m we have P L t) b ) ) n P WH t) t b m P W H t) t ) 2 ) = P Z t H. 3.6) 2

12 It follows from 3.4) and 3.6) that Eλ 2β m ) T2β m +2β = T 2β m +2β = T 2β m +E [ 2 Z t 2β P Z t H 2 [ 2Z ) t 2β H P ) dt ] t dt ) 2β H ] = 4β 2βx+m)2β +E [ 2 Z ) 2β ] H <. 3.7) The estimates 3.3) and 3.7) imply that the infinite series in the right-hand side of 3.2) converges. Ths Eτ β ) < for all β, and hence for all β. This completes the proof of the first part of the proposition. We trn now to the proof of part ii). First, we will prove that E C X x t) ) C ) X t)) dt <. 3.8) We will show later that part ii) of the proposition is a rather direct conseqence of this ineqality. Notice that E C X x t) ) C Xt) ) dt )= τ E C X x t) ) C Xt) ) dt ), where τ is given in 3.) and X is given in 3.5) and 3.6). The definition of τ implies L τ ) x+x ). Therefore, it follows from 3.4) and 3.5) that for t [,τ ], max{x xt),x t)} Z +x+x ). Since C is a non-decreasing fnction, this implies max { C X xt) ),C X t) ) } C Z +x+x ) ), and conseqently, sing the Cachy-Schwartz ineqality, τ E C X x t) ) C Xt) ) ) dt E τ C Z +x+x ) )) [ Eτ 2 )E [C Z +x+x ))] 2 )] /2. Since Eτ) 2 < by part i) of the lemma, 3.8) will follow once we show that [C ) E Z +x+x ))] 2 <. 3.9) Recall that X ) and Z have the same distribtion. The tail asymptotic 3.7) implies that any moment of Z is finite. This fact together with 2.4) yield 3.9). 2

13 Wewill nowdedce part ii) ofthepropositionfrom3.8). Toward thisend, first observe that, since X t) is a stationary process, T T E C ) X t)) dt = T T E C ) X )) dt = E CZ ) ), and recall that E CZ ) ) < by 3.7). Then notice that by 3.8), Therefore lim sp T T E C X t)) dt ) T T E C X x dt) t)) limsp T E C X x t) ) C Xt) ) ) dt =. T lim T E C Xx t)) dt ) = E CZ ) ), 3.2) which implies part ii) of the proposition in view of 3.3). Therefore, the proof of the proposition is complete. Remark. The above proposition is in agreement with a reslt in Theorem of [36] which shows that the asymptotic distribtions, as t approaches infinity, for Mt) = max s [,t] Xx s) and M t) = max s [,t] X s) coincide. 3.3 Properties of E CZ ) ). For >, let G) = E CZ ) ). We are interested in the behavior of G) in view of the identity 3.2). Lemma 3.3. Let G) = E CZ ) ) where Z is defined in 3.4). Then the following reslts hold: i) G) is a decreasing and continos fnction of on [, ). ii) If Cx) is a convex fnction then so is G). iii) lim +G) = +. Proof. First we observe that the polynomial bond 2.4) on the growth of C combined with 3.7), which describes the tail behavior of Z, imply that G) = E CZ ) ) is finite for each. It is a decreasing fnction of becase C is non-decreasing while Z Z 2 if > 2. To complete the proof of part i), it remains to show that G) is a continos fnction. Toprovethis, firstnoticethat, accordingtothedefinition3.4), Z isacontinosfnctionof thevariablea.s.,asshownbelow. Indeed, ift isarandomtimeschthatz = W H t ) t and,v), then Z Z v W H t ) vt = Z t v ). 3

14 Hence lim v + Z v = Z. A similar argment shows that lim v Z v = Z. Therefore, since C is a continos fnction, CZ ) is a continos fnction of with probability one. Since CZ ) is monotone in, the dominated convergence theorem implies the continity of G. To prove part ii), fix constants r [,], >, 2 >, and let ū r = r + r) 2. Then Zūr = sp t {W H t) ū r t} rz + r)z 2. If C is a non-decreasing convex fnction, we have E CZūr ) ) re CZ ) ) + r)e CZ 2 ) ). Hence G is convex, and the proof of part ii) is complete. Trning to the part iii), we first notice that Z = sp t W H t) = + with probability one. Let n ) n be any seqence monotonically decreasing to zero. Then, Z n is increasing and hence there exists a limit finite or infinite) lim Z n = L and Z n L for n all n. Ths W H t) n t L for all n and t. By letting n go to infinity we obtain sp t W H t) L, and conseqently L = + with probability one. Therefore lim +Z) = + a.s. Since C is a non-decreasing fnction, the monotone convergence theorem implies that lim +E CZ ) ) = +. This completes the proof of the lemma. 3.4 Existence of an optimal control In the following two theorems we provide a representation of the cost fnctional I, x) as well as the existence and niqeness reslts for the optimal control >. Theorem 3.4. Let I,x) be the cost fnctional of the long-rn average cost problem described in 3.3). Then i) I, x) is independent of x and has the representation I) := I,x) = h)+p+g), 3.2) where G) is given in Lemma 3.3. Frthermore, I) is finite for each > and is continos in >. ii) lim +I) = + and lim I) = +. iii) If hx) and Cx) are convex fnctions, then I) is also convex. Proof. Part i) follows from 3.3), Proposition 3.2, and Lemma 3.3. The first part of claim ii) follows from the fact that I) G) along with part iii) of Lemma 3.3. To verify the second part, notice that I) h) for all >, and lim + h) = +. Conseqently, lim + I) = +. Part iii) follows from the representation 3.2) combined with the part ii) of Lemma 3.3. This completes the proof of the theorem. Theorem 3.5. i) There is an optimal control > sch that for all x we have I ) = min > I,x) where I is given in 3.2). In particlar, is independent of x. 4

15 ii) In the case p >, if h and C are convex fnctions, then is niqe. iii) In the case p =, if h is a strictly convex fnction and C is a convex fnction, then is niqe. Proof. Since I) is a continos fnction, part i) follows from parts i) and ii) of Theorem 3.4. If h andc areconvex fnctions, the representation 3.2) yields that I is a strictly convex fnction when p >. Therefore, is niqe in this case. In the case p =, if h is a strictly convex fnction and C is a convex fnction, the reslts follows from the representation 3.2) in a similar way as in the case p >. 3.5 Generalizations In this section we generalize the reslts in Theorem 3.4 and 3.5 to the more general model introdced in 2.). Notice that for a given control > fixed in 2.), the self-similarity of fbm yields that the process ŴH defined by Ŵ H t) = σ)w H t σ) H is a fractional Brownian motion. Let Yx t) = Xx t σ) H ), t R, ). Then Y x satisfies Yx t t) = x σ) H +ŴHt)+ L x t), 3.22) where L xt) = L x t σ) H ). Using 2.2) and change of the variable s = t L xt) = L x t σ) H ) σ) H { = min, min s [,t], we observe that x s σ) H )} +ŴHs). 3.23) The eqations 3.22) and 3.23) are analogos to 2.) and 2.2). We next consider the change in the cost strctre de to the change of the variable s = t. We notice that σ) H T T C Xx t)) dt = MT) C Yx MT) t)) dt, 5

16 where MT) = σ) HT. Therefore, sing the reslts in Theorem 3.4, we obtain We have the following reslt. T lim T E C ) ) Xx t)) dt = G. 3.24) σ) H Theorem 3.6. Consider the controlled state process X x defined by 2.) and 2.2) with the cost fnctional Ix,) given in 2.5). Define f : [, ) [, ) by Then f) =. 3.25) σ) H I,x) = h)+p+g f) ), 3.26) where the fnction G is given in Lemma 3.3. Frthermore, I,x) is independent of x we will henceforth denote the cost fnction by I)). Proof. The same argment as in the proof of Lemma 3. yields lim E L T x T)) =. Combining this reslt with 3.24) we obtain the representation 3.26). Or next reslt is analogos to Theorems 3.4 and 3.5. Theorem 3.7. Assme that the fnction f in 3.25) is continos and lim +f) =. Then, with I) = I,x) as in 3.26), we have: i) lim +I) = lim I) = +, and I) is a finite continos fnction on [, ). + Frthermore, there is a constant > sch that I ) = min > I). ii) If f is a concave increasing fnction then the statements similar to parts i) and ii) of Theorem 3.5 regarding the niqeness ) hold. The proof of this theorem is a straightforward modification of the proofs of Theorems 3.4 and 3.5 and therefore is omitted. Remark. One can frther generalize or model to cover the following sitation. For given positive continos fnctions b) and σ) let where for >, X x t) = x+σ)w ht) b)t+l x t), L x {, t) = min min x b)s+σ)wh s) )}. s [,t] The optimization problem here is to minimize the cost fnctional I, x) defined in 2.5). Following the time change method described in Section 3.5, one can obtain an analoge of Theorem 3.7 regarding the derivation of the optimal control. In this sitation, the fnction f defined in 3.25) needs to be replaced by f) = b)σ)) H with the assmptions that f is continos and lim +f) =. We omit the details of the proof. 6

17 4 A constrained minimization problem In this section, we address a constrained minimization problem that can be solved by sing or reslts in Section 3. Or model here is of the form Y x t) = x t+σ)w H t)+k t), 4.) where σ is a non-negative continos fnction, K ) is a non-negative non-decreasing rightcontinos with left limits RCLL) process adapted to the natral filtration F t ) t, where F t is the σ-algebra generated by {W H s) : s t} agmented with all the nll sets. Frthermore, K ) = and the process K is chosen by the controller in sch a way that the state process Y is constrained to non-negative reals. In this sitation, the controller is eqipped with two controls: the choice of > and the choice of K process sbject non-negativity of the Y process. Throghot this section we keep the initial state x fixed. We will dedce sing the reslts of the previos section that the vale of this minimization problem as well as the optimal control are not affected by the initial data. Let m > be any fixed positive constant. The constrained minimization problem we wold like to address here is the following: Minimize Sbject to: lim sp lim sp T T E E K T) ) T [ h)+c Y t) )] dt) 4.2) m. 4.3) A controlled optimization problem of this natre for diffsion processes was considered in [], and for a more complete treatment in the case of diffsion processes we refer to [3]. Fix any integer m > and define a class of state processes U m as follows: { U m = Y,K ) : Y E K T) ) } t) for t, 4.) is satisfied, limsp m. T From or reslts in Section 3 it follows that for any m, the pair Xx,L x ) in 2.) and 2.2) belongs to U m, and hence U m is non-empty. Therefore, the constrained minimization problem is to find inf limsp Y,K ) U m T T E [ h)+c Y t) )] dt). In this section we make the following additional assmptions: i) For fnctions h and C we assme: h is strictly convex and satisfies 2.3), C is convex and satisfies 2.4). 4.4) 7

18 ii) Let f) =. Then σ) H f) >, lim +f) =, and f is a convex increasing fnction. 4.5) The following lemma enables s to redce the set U m to the collection of processes X x,l x ) described in the previos section, with m. Lemma 4.. Let > and let Y,K ) be a pair of processes satisfying 4.). Consider X x,l x ) which satisfies 2.), 2.2), and is defined on the same probability space as Y,K ). Then i) L x t) K t) and X x t) Y t) for t. ii) = lim E L T xt) ) limsp E K T) ). T iii) G f) ) = lim E T C X T x t)) dt ) limsp E T C Y t) ) dt ). T Proof. Since, Y, K ) =, and K is non-increasing process, the minimality property of the reflection map stated in 2.6) implies that L xt) K t) and Xxt) Yx t) for t. Next, observe that part ii) of the lemma follows from the reslt in part i) while the identity = lim E L T xt) ) is implied by Lemma 3. see also the remark right after the proof of Lemma 3.). Finally, part iii) of the lemma follows from 3.24), part i), and from the fact that C is non-decreasing. The proof of the lemma is complete. Let V m = {Xx,L x ) : 2.),2.2) are satisfied and in addition m}. From the above lemma it is clear that T inf lim K,Y ) U m T E [ h)+c X x t) )] dt) = inf X x,l x) V m limsp T T E [ h)+c Y t) )] dt). 4.6) Therefore, or minimization problem is redced. Next, we can se the reslts in Section 3.5 and write for any >, T lim T E [ h)+c X x t) )] dt) = h)+g f) ). 4.7) Here G is given by Lemma 3.3, and f is described in 3.25) and 4.5). Conseqently, T inf lim Xx,L x ) Vm T E [ h)+c X x t) )] dt) = inf{h)+g f) ) : < m}. 4.8) 8

19 We next consider the optimal control described in Theorem 3.7 corresponding to the case p =. In virte of the assmptions 4.4) and 4.5), the optimal control is niqe and we will label it by >. We have the following reslt. Theorem 4.2. Let { m if m < m) =, if m, 4.9) where is the niqe optimal control in Theorem 3.7 corresponding to p =. Then the pair X m) x,l m) x ) is an optimal process for the constrained minimization problem 4.2) and 4.3). Frthermore, the optimal control m) is a continos increasing fnction of the parameter m. Proof. Let Λ) = h)+g f) ) where f and G are as in 4.7). Then, by the assmptions 4.4), 4.5) and Theorem 3.7, Λ is a strictly convex fnction which is finite everywhere on, ). Frthermore, lim +Λ) = + and lim Λ) = +, and hence Λ has a niqe + minimm at. Therefore, Λ is strictly increasing on, ). Clearly, with m) defined in 4.9), Λ m) ) = inf m Λ), and m) is the niqe nmber which has this property. By 4.6) and 4.8), we have Λ m) ) = inf limsp Xx,L x ) Vm Consider the pair of processes X m) x Lemma 3., we have lim lim sp T T E [ h)+c Y t) )] dt).,l m) x ) defined in 2.) and 2.2). Then, in virte of E L m) T) ) = m) m, and by Theorem 3.6, T T T E [ h)+c X m) x t) )] ) dt = Λ m) ). Hence X m) x,l m) x ) describes an optimal strategy. This completes the proof of the theorem. Remark. Notice that the above optimal control m) is independent of the initial point x. 5 Infinite horizon disconted cost minimization problem In this section we define an optimal control for the infinite horizon disconted cost fnctional given in 2.7). Throghot this section we assme that σ), the state process X x satisfies 3.), and that the fnctionals h and C are convex in addition to the assmptions stated in 2.3) and 2.4). In contrast with Section 3, or methods here do not readily extend to the case where the fnction σ) is non constant. 9

20 The disconted cost fnctional J α x,) is given by J α x,) = E e αt[ h)+c Xxt) )] dt+ = h) α +E ) e αt pdl xt) e αt[ C ) Xx t)) +αpl x t)] dt. 5.) Here α > is a constant discont fnction. To derive the last eqality above we have sed Fbini s theorem to obtain e αt dl x t) = α e αt L x t)dt. Let J α, x,) = E e αt C ) Xx t)) dt and ) J α,2 x,) = E e αt L xt)dt. 5.2) Next we se the convexity of the reflection mapping described in 2.5) to establish the convexity of the cost fnctional with respect to. Lemma 5.. Let x be fixed and let C be a convex fnction satisfying assmption 2.4). Then, J α, x,) and J α,2 x,) introdced above are finite for each and are convex in -variable. Proof. By 3.2), we have the bond E L xt) ) t+k +t H ), where K > is a constant independent of t. This implies, by sing Fbini s theorem, that J α,2 x,) is finite. Next, by sing 3.8) we obtain J α, x,) E e αt C Xt) ) ) dt E C X x t) ) C Xt) ) ) dt <. Bt, sing the stationary of X, we have E e αt C ) X t)) dt = α E CZ ) ), where Z is given in 3.4). Notice that E CZ ) ) is finite becase C has polynomial growth and in virte of 3.7). Conseqently, J α, x,) is also finite. To establish convexity of J α, x,), first recall that Xx = Γ x+w H e ), where et) t for t, and Γ is the reflection mapping described in Section 2.3. Now let, 2, and r,). Then, for t, x+w H t) r + r) 2 ) et) = r x+w H t) et) ) + r) x+w H t) 2 et) ). Since the reflection map Γ satisfies the convexity property 2.5), we have: Xūr x t) rx x t)+ r)x 2 x t), 5.3) where ū r = r + r) 2. Next, since C is a convex non-decreasing fnction, 5.3) implies that C Xūr x t)) rc X x t)) + r)c X 2 x t)) for t. From this it follows that J α, x,ū r ) rj α, x, )+ r)j α, x, 2 ). 2

21 Hence J α, x,) is convex in -variable. Next, by 2.2) and 2.), we have Γ x+w H e ) t) x+w H e)t) = L x t), t. Then, since Γ is convex in -variable by 2.5), the process L x Conseqently, with ū r = r + r) 2, we obtain is also convex in -variable. Lūr x t) rl x t)+ r)l 2 x t), t. Finally, it is evident that J α,2 x,) is convex in -variable from the definition 5.2). Corollary 5.2. Under the conditions of Lemma 5., the disconted cost fnctional J α x,) is finite for each and is convex in -variable. Proof. Notice that J α x,) = h) α +J α,x,)+αpj α,2 x,). 5.4) By or assmptions, h is a convex fnction and p and α > are constants. Therefore, the claim follows from Lemma 5.. The above lemma and the corollary lead to the following reslt. Theorem 5.3. Consider the Xx process satisfying 3.) and the associated disconted cost fnctional J α x,) described in 5.). Then, for each initial point x, there is an optimal control sch that J α x, ) = inf J αx,) V α x), where V α x) is the vale fnction of the disconted cost problem defined in 2.8). Proof. Fix any x and α >. By Corollary5.2, J α x,) is finite for each and is convex in -variable. By 5.4), we have that J α x,) h) and hence, since lim h) = +, α we obtain lim J α x,) = +. Since J α x,) is convex in -variable, we can conclde that there is a which may depend on x) sch that J α x, ) = inf J α x,). This completes the proof of the theorem. Remark. Notice that in contrast with the long-rn average cost minimization problem, we cannot rle ot the possibility = here. Corollary 5.4. For the special case p =, assme frther that hx) is constant on an interval [,δ] for some δ >. Then, for every initial point x, the optimal control is strictly positive. Proof. It follows from 5.4) that J α x,) = J α, x,). The fnction C is increasing and X x t) < X 2 x t) for all t and > 2. Therefore J α, x, ) J α, x, 2 ) for > 2. Conseqently, J α x,) J α x,)for all >. Hence, we canfind anoptimal control >, and the proof of the corollary is complete. 2

22 6 Finite horizon problem and Abelian limits In this section, we establish Abelian limit relationships among the vale fnctions of three stochastic control problems introdced in Section 2.2. The main reslt is stated in Theorem 6.3 below. Throghot this section, for simplicity, we assme that σ) in the model described in 2.). We begin with the existence of an optimal control for the finite horizon control problem introdced in Section 2.2. Proposition 6.. For any fixed x and T >, we have: i) I,x,T) is finite for each > and is continos in >. ii) lim I,x,T) = +. iii) If h and C are convex fnctions, then I,x,T) is a convex fnction of the variable. Corollary 6.2. For any fixed x and T >, we have: i) There is an optimal control x,t) sch that I x,t) ) = min > I,x,T). ii) In the case p >, if h and C are convex fnctions, then x,t) is niqe. iii) In the case p =, if h is a strictly convex fnction and C is a convex fnction, then x,t) is niqe. The proof of the proposition and of the corollary is a straightforward adaptation of the corresponding proofs given in Section 3, and therefore is omitted. The following theorem is the main reslt of this section. Theorem 6.3. Let X x satisfy 3.) and let V, V α x), Vx,T) be the vale fnctions defined in 2.6), 2.8), and 2.9) respectively. Then the following Abelian limit relationships hold: Vx,T) lim α +αv αx) = lim = V. T We prove this reslt in Propositions 6.5 and 6.6 below. The following technical lemma gathers necessary tools to establish lim α +αv αx) = V. Lemma 6.4. Let > be given and X x satisfy 3.). Consider the cost fnctional J αx,) as defined in 2.7). Then where I) is described in 3.2). lim α +αj αx,) = I), Proof. First consider lim α +αe e αt dl x t)), where L x is as in 2.2). Similarly to 5.), we have ) ) αe e αt dl xt) = α 2 E e αt L xt)dt = α 2 e αt E L xt) ) dt, 6.) 22

23 where we sed Fbini s theorem to obtain the last identity. By 3.), Therefore, α 2 E L xt) ) = t+e X xt) ) x. e αt E L x t)) dt = α 2 e αt tdt+α 2 e αt E Xx t)) dt αx = +α 2 e αt E Xx t)) dt αx. 6.2) Frther, by 3.2), E X x t)) K +t H ), where the constant K > is independent of t and. Ths α 2 e αt E Xx t)) dt K α 2 e αt +t H )dt K α+gammah)α H ), 6.3) where GammaH) := e t t H dt = α +H e αt t H dt is the Gamma fnction evalated at H. Since H,), it follows from 6.3) that lim α +α2 e αt E X x t)) dt =. Hence, sing 6.) and 6.2), we obtain lim α +αe ) e αt dl xt) =. 6.4) We next consider lim α +αe e αt C X xt) ) dt ). It follows from 3.8) that E e αt C ) Xx t)) dt E e αt C X t)) dt) E C X x t) ) C Xt) ) ) dt <, where X is the stationary process described in 3.5) and 3.6). Therefore, lim α +αe e αt C Xx t)) dt ) = lim α +αe = lim α +α e αt C X t)) dt ) e αt E CZ ) ) dt = E CZ ) ) = G), 6.5) where G) is defined in Section 3.3. It follows from 2.7), 6.4), and 6.5) that lim α +αj αx,) = h)+p+g) = I), where I) is given in 3.2). This completes the proof of the lemma. The next proposition contains the proof of the first part of Theorem

24 Proposition 6.5. Let X x satisfy 3.) and V αx) be the corresponding vale fnction defined in 2.8). Then lim α +αv αx) = V <, where V is the vale of the long-rn average cost minimization problem given in 2.6). Proof. Fix the initial point x. For any >, we have αv α x) αj α x,). Hence, by Lemma 6.4, limsp αv α x) lim α + α +αj αx,) = I). Therefore, minimizing the right-hand side over >, we obtain lim sp α + αv α x) inf > I) = V. It remains to show the validity of the reverse ineqality, namely that lim inf α + αv αx) inf > I) = V. 6.6) To this end, consider a decreasing to zero seqence α n >, n N, sch that lim inf α + αv αx) = lim n α n V αn x). 6.7) Fix any ε > and let n >, n N, be a seqence sch that V αn x) + ε > J αn x, n ). Then Letting n we obtain α n V αn x)+α n ε > α n J αn x, n ) h n ). 6.8) lim sp n h n ) limspα n V αn x) V. 6.9) n Since lim x hx) = +, this implies that n is a bonded seqence. That is, there is M > sch that n,m) for all n N. Therefore, withot loss of generality we can assme that n converges as n to some [,M] otherwise, we can consider a convergent sbseqence of n ). Let δ, ). Then, α n J αn x, n ) h n )+αn 2 p e αnt E L n x t)) dt+α n e αnt E [ C Xx δ t))] dt Since E L n x t)) = E Xx nt)) + n t x n t x, we obtain and hence α 2 n e αnt E L n x t)) dt n α n x, α n J αn x, n ) h n )+p n pα n x+α n e αnt E [ C Xx δ t))] dt. 24

25 Therefore, letting n go to infinity and sing 6.5), we obtain lim inf n α nj αn x, n ) h )+p +E [ C Z δ )]. 6.) In particlar, one can conclde that >, becase otherwise, letting δ tend to and sing Lemma 3.3, we wold obtain that liminf n α n J αn x, n ) = + which contradicts the fact that limspα n J αn x, n ) V < according to 6.8) and 6.9). n Therefore >. Letting δ in 6.) tend to and sing again the continity of G) = E [ C )] Z proved in Lemma 3.3, we obtain lim inf n α nj αn x, n ) h )+p +E [ C Z )] = I ) V. 6.) The ineqalities 6.8), 6.9), and 6.) combined together yield lim α nj αn x, n ) = I ) = V, n which completes the proof of the proposition in view of 6.7). Notice that 6.) implies that is an optimal control for the long-rn average cost control problem. The following proposition incldes the second part of Theorem 6.3. Proposition 6.6. Under the conditions of Theorem 6.3, we have Vx,T) lim = V. T Proof. It follows from 2.5) and 2.) that for any x and > we have lim sp I,x,T) T = I), Vx,T) where I) is given in 3.2). Therefore limsp T minimizing the right-hand side over >, we obtain limsp I,x,T) T = I), and, lim sp Vx,T) T inf > I) = V. 6.2) It remains to show that lim inf Vx,T) T V. 6.3) The proof of 6.3) given below is qite similar to that of 6.6). Consider a seqence of positive reals T n ) n N sch that lim n T n = + and lim inf Vx,T) T = lim n Vx,T n ) T n. 25

26 Fix any ε > and for each n N chose n > sch that Vx,T n ) + ε > I n,x,t n ). Then, in view of 2.) and 6.2), we have Vx,T n ) lim sph n ) limsp n n T n V < +. Since lim x hx) = +, this implies that n is a bonded seqence. That is, there is M > sch that n,m) foralln N. Taking afrther sbseqence if necessary, we canassme withot loss of generality that n converges to some [,M], as n. Frthermore, > becase if =, then by 3.2) we obtain V limsp n Vx,T n ) T n limsp n I n,x,t n ) T n E CZ δ ) ) for any δ >. This is impossible in view of part iii) of Lemma 3.3. Therefore >. Let v,v 2 be any nmbers sch that < v < < v 2. Then, by 2.) and an argment similar to the derivation of 6.), we have Vx,T n )+ε > I n,x,t n ) hv )T n +E L v x T n) ) Tn + E [ C X v 2 x t))] dt, for n large enogh. Using Lemma 3. along with 3.2) we dedce that lim inf n Vx,T n ) T n hv )+pv +E CZ v2 ) ). Since v and v 2 are arbitrary nmbers satisfying the above ineqality constraints and h) and G) = E CZ ) ) are continos fnctions, this implies V liminf n Vx,T n ) T n h )+p +E CZ ) ) = I ) V. The proof of 6.3), and hence the proof of the proposition is complete. In fact, the above ineqality also shows that is an optimal control for the long-rn average cost control problem. Propositions 6.5 and 6.6 combined together yield Theorem 6.3. Remark. The proof of Propositions 6.5 and 6.6 imply the following reslts:. Let α n ) n be a seqence of positive nmbers converging to zero and let n be an ε-optimal control for V αn x) in 2.8). Then the seqence n ) n is bonded and any limit point of n ) n is an optimal control for V defined in 2.6). 2. Let T n ) n be a seqence of positive nmbers sch that lim n T n = +. If n be an ε- optimal control for Vx,T n ) in 2.9), then the seqence n ) n is bonded and any limit point of n ) n is an optimal control for V defined in 2.6). 26

27 Acknowledgments: The work of Arka Ghosh was partially spported by the National Science Fondation grant DMS The work of Ananda Weerasinghe was partially spported by the US Army Research Office grant W9NF7424. We are gratefl for their spport. References [] B. Ata, J. M. Harrison, and L. A. Shepp, Drift rate control of a Brownian processing system, Ann. Appl. Probab. 5 25), [2] S. L. Bell and R. J. Williams, Dynamic schedling of a system with two parallel servers in heavy traffic with resorce pooling: asymptotic optimality of a threshold policy, Ann. Appl. Probab 2), [3] F. Biagini, Y. H, B. Oksendal, and A. Slem, A stochastic maximm principle for processes driven by fractional Brownian motion, Stoch. Process. Appl. 22), [4] F. Biagini, Y. H, B. Oksendal, and T. Zhang, Stochastic Calcls for Fractional Brownian Motion and Applications, Springer s series in Probability and its Applications, Springer, 27. [5] A. Bdhiraja and A. P. Ghosh, A large deviations approach to asymptotically optimal control of crisscross network in heavy traffic, Ann. Appl. Probab. 525), [6] A. Bdhiraja and A. P. Ghosh, Diffsion approximations for controlled stochastic networks: an asymptotic bond for the vale fnction, Ann. Appl. Probab. 6 26), [7] T. E. Dncan, Some stochastic systems with a fractional Brownian motion and applications to control, Proceedings of American Control Conference, Jly 27, NY, 4. [8] N. G. Dffield and N O Connell, Large deviations and overflow probabilities for the general single-server qee, with applications, Math. Proc. Cambridge Philos. Soc ), [9] A. P. Ghosh and A. Weerasinghe, Optimal bffer size and dynamic rate control for a qeeing network with impatient cstomers in heavy traffic, sbmitted, apghosh/reneging qee.pdf [] W.-B. Gong, Y. Li, V. Misra, and D Towsley, Self-similarity and long range dependence on the Internet: a second look at the evidence, origins and implications, Compter Networks 48 25), [] J. M. Harrison, Brownian Motion and Stochastic Flow Systems, John Wiley & Sons, New York,