Operations Research. Embedding Capacity of Information Flows in High QoS Renewal Traffic

Transcription

1 Embedding Capacity of Information Flows in High QoS Renewal Traffic Journal: Operations Research Manuscript ID: Draft Manuscript Type: Article Date Submitted by the Author: n/a Complete List of Authors: Marano, Stefano; University of Salerno, Dept. of Information and Electrical Engineering, and Applied Mathematics Matta, Vincenzo; University of Salerno, Dept. of Information and Electrical Engineering, and Applied Mathematics He, Ting; IBM T. J. Watson Research Center, Tong, Lang; Cornell University, ECE Department Keywords: Communications, Applications < Networks/graphs

2 Page of Operations Research 0 0 Submitted to Operations Research manuscript (Please, provide the mansucript number!) Embedding Capacity of Information Flows in High QoS Renewal Traffic Stefano Marano, Vincenzo Matta DIEM, University of Salerno, via Ponte don Melillo I-, Fisciano (SA), Italy, {marano, vmatta}@unisa.it Ting He IBM T. J. Watson Research Center, Yorktown, NY, the@us.ibm.com Lang Tong ECE Department, Cornell University, Ithaca, NY USA, ltong@ece.cornell.edu We study the capacity of embedding secret information flows into normal transmission activities of a communication network. The maximum amount of embedded flows known as the embedding capacity C has been recently characterized analytically for a broad range of traffic patterns modeled as renewal point processes. The existing formulas for approximating C, however, lack accuracy in the regime of small-dispersion traffic, which represents a variety of real-time applications with stringent Quality-of-Service (QoS) requirements. We attempt to fill this gap by converting the characterization of C into the study of a certain random walk with permeable barriers constructed from the optimal flow embedding policy. It has been shown that C is a simple function of the probability for this random walk to exceed the barriers; however, it is generally difficult to compute this probability for arbitrary renewal processes. To this end, we derive a novel closed-form relationship between the time-averaged overshoot beyond the barriers and the overshooting probability. This relationship implies a new formula for approximating C in terms of the average overshoot. Although it is again difficult to compute the overshoot in general, we show that the asymptotic overshoot can be characterized as a function of well-known statistical properties of the original renewal processes in the regime of large or small barriers, therefore readily computable in cases that these properties are known. The new formula offers new insights into the impact of system parameters on the embedding capacity C ; more important, it improves the accuracy of the existing formulas in the regime of high QoS requirements. Key words : information hiding; stepping-stone detection; random walks with permeable barriers. History : Submitted on December, 0. Introduction and motivation Due to rising concerns of security and privacy, modern packet communication networks, both wired and wireless, often operate in the encrypted domain where the packet content is not accessible. While the content of information may be secure via cryptographic techniques, keeping the acts of communication and parties involved in communications private or anonymous requires very

3 Page of 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) different types of techniques. In fact, it is relatively easy, especially in wireless transmissions, to perform timing analysis based on measured timestamps of transmissions at various network nodes. It was such timing analysis that revealed vulnerabilities of the well known Secure Shell (SSH) protocol. Thus, although encryption of communication packets hides the message content and even the packet headers (via anonymous routing protocols) such that in the content domain each packet does not reveal any special characteristic, we can still extract valuable information about the communications, e.g., who is talking to whom, from the act of communication itself, using the technique of timing analysis. This possibility gives rise to a series of questions. For a traffic analyzer, given that the packet content is secret, what can be revealed by timing analysis? Are there practical real-time algorithms for such analysis? Can we trace the route of the information flows across the network? Parties involved in communications who wish to maintain anonymity may raise the same types of questions but in an opposite way: How much information (if any) can be conveyed covertly? What are the guidelines for designing anonymity-oriented traffic patterns? Answering the above questions allows a better understanding of security and privacy issues in modern communication systems and networks; it also represents an important step toward the design of future complex networks (including perhaps social networks and cyber physical systems) where privacy and security may be a critical feature. Motivated by these general considerations, in this paper we address the issue of optimally embedding an information flow into the normal traffic of a network, whose context domain is now described more in detail. In the existing computer networks, hidden information flows not visible in normal network protocols are often indicators of intrusions. For example, in the so-called stepping-stone attacks that account for a large portion of Internet-based attacks [Staniford-Chen and Heberlein ()], the attacker deliberately constructs a multi-hop path to the victim computer by concatenating host-tohost connections (e.g., via SSH protocol tunneling) across multiple compromised hosts, which serve as the stepping stones for the attacking traffic. Because traffic on different connections appears independent from packet headers, the attacker can hide the intrusion path and hence his true identity. In defending against such attacks, it is crucial for network administrators to accurately detect the presence of information flows based on the actual correlation in the traffic.

4 Page of Operations Research 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) A straightforward approach of comparing traffic payload is vulnerable to encryption, e.g., traffic in two SSH sessions can appear uncorrelated even if they carry identical content. A more robust countermeasure is timing analysis [Donoho et al. (00), Blum, Song and Venkataraman (00), Raymond (00)], based on comparing the timing patterns on these connections: if two packet streams carry the same information flow, their transmission timestamps have to be similar up to limited perturbations to achieve reasonable end-to-end performance (e.g., bounded delays). Timing analysis greatly extends our capability in detecting information flows beyond those visible from packet headers or payload. It is, however, vulnerable to deliberate manipulations of transmission timing, e.g., through delaying selected packets or even injecting dummy packets. Theoretically, a smart attacker aware of the presence of the timing analyzer can embed his packets in a larger number of transmissions such that the overall transmission sequences, referred to as cover traffic, follow an arbitrary pattern. In particular, he can make the cover traffic on different hops mutually independent, both mimicking normal transmission patterns, so that the information flow is perfectly hidden. Such practice, however, incurs a loss in efficiency because independently generated transmission timestamps may be non-causal or too far apart, and hence cannot be used to relay information packets. The exact efficiency heavily depends on the model used to generate cover traffic, and measures how suitable this model is for embedding information flows, or equivalently, how vulnerable the network is to attacks employing embedded information flows when normal transmissions follow this model. A dual application of timing analysis is anonymous networking, where a user wishing to hide his identity sends his traffic to a proxy (e.g., Chaum Mix [Chaum ()]), which encrypts the traffic and embeds it into the overall outgoing traffic sent to the destination. The efficiency of embedding in this context measures the performance of the proxy under given models of incoming/outgoing traffic. In light of the above observations, a fundamental question is: given a model of normal traffic, what is the maximum efficiency, measured by the long-term fraction of information packets, in embedding an information flow into cover traffic generated independently from the model? The answer to this question will establish fundamental limits on both timing analysis, in terms of what kind of flows can be detected reliably, and anonymous networking, in terms of how much traffic can

5 Page of 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) be sent anonymously. Furthermore, the answer will provide guidelines for designing transmission schedules in future computer networks so as to control or facilitate embedded information flows. A brief historical sketch of the addressed problem can be so summarized. The detection of information flows has been originally studied in the context of stepping-stone attacks and related countermeasures [Voydock (), Staniford-Chen and Heberlein (), Donoho et al. (00), Blum, Song and Venkataraman (00), Raymond (00)]. The dual problem of hiding information flows finds application in the paradigm of anonymous networking [Raymond (00), Chaum (), Radosavljevic and Hajek (), Venkitasubramaniam, He and Tong (00)]. While earlier work focuses on algorithm design, several recent studies have investigated the fundamental limit on embedding information flows using techniques from the areas of signal processing and information theory [He, Agaskar and Tong (00), He and Tong (00), He, Tong and Swamy (00), Venkitasubramaniam, He and Tong (00)]. In particular, the problem of characterizing the maximum embedding efficiency, termed embedding capacity C, has been recently studied in [Marano, Matta and Tong (00), Marano et al. (0), Marano et al. (0)] for cover traffic modeled as iid (independent, identically distributed) renewal processes with an arbitrary interarrival distribution, leading to a set of approximation formulas for C... The basic scenario To illustrate the concept of embedding capacity, let us consider the following scenario depicted in Fig.. In a communication network, Sally (source) is sending information packets towards the Dave (destination), using the intermediate node Ray as a relay. The information flow originated from Sally and relayed by Ray is embedded in the global traffic of the network; only a fraction of the packets transmitted by Sally will be relayed by Ray, and only a fraction of the packets transmitted by Ray originate from Sally. The content of the packets is unaccessible due to encryption, so that a network analyzer can only measure the timestamps of the packets at the various nodes. Suppose that the network analyzer knows the Quality of Service (QoS) required by Sally, in particular, a maximum per-hop delay induced by the end-to-end delay constraint of Sally s application. Under this constraint, an information flow between Sally and Ray can be modeled as a sequence of pairs of packets, with each pair consisting of a packet at Ray and a packet at Sally, where the former is

6 Page of Operations Research 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) Figure Alice Sally Ray Sally Ray TX epochs of packets travelling from Sally through Ray (information flow) Other TX activity of Sally and Ray (chaff) General information flows co-existing in the wireless network Bob time Packets sent by Sally to Ray must be relayed within a maximum delay Carl time time Dave Notional sketch of the -bounded-delay information flow problem. Above: Pictorial view of a wireless packet communication network, with the output activities of Sally and Ray monitored. Different information flows traverse the network, shown by different colors. Below: The traffic patterns at the output of Sally s and Ray s node as function of time: pairs of packets forming the information flow from Sally through Ray comply with the -delay bound, while the remaining packets are chaff. The fraction of packets belonging to the flow, over infinitely long steams, represents the embedding capacity C a fundamental limit on the rate of undetectable communication. delayed by at least zero and at most with respect to the latter, and no packet can belong to two different pairs. The question of interest is: how many packets can Ray relay for Sally, relative to the total number of packets transmitted by the two nodes, so that no network analyzer can detect the act of relay? The answer is just the capacity C : the efficiency of the best policy for embedding a -delay information flow into a cover traffic distributed according to the normal transmission patterns of Sally and Ray when they are not communicating. To date, the formulas for C Eve available in the literature refer to packet timestamps at the two nodes modeled as iid renewal point processes with an arbitrary interarrival distribution [Marano, Matta and Tong (00), Marano et al. (0), Marano et al. (0)]. Despite being usually accurate for a broad variety of traffic models, these formulas have been shown to lack

7 Page of 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) precision for certain distributions, especially when the randomness of the cover traffic, measured by the dispersion index of the corresponding interarrival distribution, is low. Such low-dispersion traffic models real-world applications with stringent quality-of-service requirements, such as voiceover-ip and video streaming. From the perspective of network administrator, it is highly important to understand the embedding capacity of this category of traffic because it not only constitutes a significant portion of Internet traffic, but also provides an ideal vehicle for embedding information flows. In this regard, a main contribution of this paper is to derive new formulas which allow a better characterization of C in the low-dispersion regime. Moreover, the new formulas reveal new insights about the relationship between the embedding capacity and statistical properties of the cover traffic. To derive the new formulas for C, we exploit the fact, originally discovered in [He, Agaskar and Tong (00)], that C can be related to the asymptotic behavior of a certain type of Markov chains, which we refer to as random walks with permeable barriers. Accordingly, the addressed problem translates into investigating steady-state properties of these random walks. A (one-dimensional) random walk with permeable barriers is a relaxation of a random walk with hard barriers, a classical model for a wandering particle constrained to lie in a given region, say [ τ, τ], τ > 0. Often the impossibility of entering the forbidden region beyond the barriers is an idealization frequently adopted to simplify the model. Permeable barriers model situations where the barriers can be crossed (i.e., they are permeable), but when such crossing occurs, the walking particle is forced to move back towards the allowed region [ τ, τ]. An example of such process taken from biological systems is that of the random motion of molecules in cell membranes enforcing the molecule towards the inside part of the cell, see e.g., [Heitjans and Karger (00)]. Another example, borrowed from statistical physics, is represented by a randomly wandering electrical particle that encounters a potential barrier, constraining the particle within a certain region, see e.g., [Kraprivsky, Redner and Ben-Naim (00)]... Overview of main results and organization We first present the formulation of the problem and some key preliminary results. In particular, we introduce the notion of embedding capacity C the maximum efficiency of embedding information flows in independent point processes. The key to obtain the best embedding

8 Page of Operations Research 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) is based on a simple policy, referred to as Bounded Greedy Match (BGM), originally shown in [Blum, Song and Venkataraman (00)]. This strategy happens to be sample-path optimal, such that the evaluation of C is directly done through the analysis of the BGM. The flow-diagram of the BGM algorithm is easily described by a certain random walk with permeable barriers. The asymptotic fraction of time p spent by the walk outside the barriers is straightforwardly related to C, so that computation of capacity reduces to the study of this special kind of random process. Accordingly, we then focus on the analytical characterization thereof. In particular, in the presence of permeable barriers, one of the quantities of interest is how much in depth the walk enters the forbidden region; such an excess over the barriers is called overshoot. In this paper we find a simple and closed-form relationship between the average value of the overshoot O (whose definition is given in eq. ()), and the probability p that the walk lies outside the barriers, i.e., the fraction of time spent outside [ τ, τ]; this is one of the main results of the paper contained in Theorem of Sect. providing insights about the essential features of these random walks. We also investigate the behavior of the average overshoot O in the limiting cases of large and small barriers, i.e., τ and τ 0, to characterize the stationary distribution of the corresponding random walk. As expected, in the regime of vanishingly small τ, the average overshoot O is ruled by the steps of the chain outside the barriers, see eq. (). Conversely, when τ is very large, O is essentially determined by the steps of the chain inside the barriers, see eq. (). Finally, we apply the findings about the random walk to the characterization of the embedding capacity, which leads to a new exact expression for C, given in Corollary, and asymptotic expressions thereof. This allows a better understanding of the operational meaning of C, and provides a more accurate analytical characterization of the embedding capacity in the regime of small-dispersion traffic complementing the previously known facts about C. The remainder of the paper is organized as follows. In Sect. the problem is introduced, and Sect. deals with the characterization of C. Section presents a theorem about random walks with permeable barriers, with limiting cases dealt with in Sect.. Examples are provided in Sect., with special emphasis on the applications to the computation of C. Final remarks are contained in Sect..

9 Page of 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!). Embedding capacity of information flows Capital letters denote random variables, and the corresponding lowercase their realizations; P and E denote probability and expectation operators, respectively. By a point process on the positive real semi-axis t (0, ) we mean a collection of non-negative random variables S = {S i, i } such that, for i =,,..., S i < S i+ almost surely (a.s.), and lim i S i = a.s. Definition. (Bounded delay information flow) Point processes S = {S i, i } and R = {R i, i }, defined on the real semi-axis t (0, ), form a -bounded-delay information flow in the direction S R if, for every realization {s i, i } of S and {r i, i } of R, there is a one-to-one mapping M : {s i } {r i } between sets {s i } and {r i } such that 0 M(s i ) s i, i. The condition M(s i ) s i 0 reflects a causality constraint, while the one-to-one requirement ensures packet conservation. Let us consider two point processes S = {S i, i } and R = {R i, i } that, in general, do not form a -bounded-delay information flow. We refer to the pair (S, R) as the cover traffic. Given cover traffic (S, R), one can extract subsequences forming a -bounded-delay information flow, which leads to the concept of embedding policy. Definition. (Embedding policy) An embedding policy ϵ selects two subsequences of indices {I k, k } and {J k, k } such that the thinned point processes S ϵ = {S Ik, k } of S and R ϵ = {R Jk, k } of R form a -bounded-delay information flow in the direction S ϵ R ϵ. Thus, the cover traffic (S, R) is decomposed into an information flow (S ϵ, R ϵ ) plus the residual part of (S, R), which is called chaff. The ensemble of all the admissible policies is denoted by E. Definition. (Efficiency) Given cover traffic (S, R), the efficiency of an embedding policy ϵ E is measured by η(ϵ):= lim inf t S ϵ (t) + R ϵ (t) S(t) + R(t), where S(t) denotes the number of points of the process S in the interval (0, t). The embedding capacity C can be now introduced as the best efficiency. Definition. (Embedding capacity) Given cover traffic (S, R), the embedding capacity of a - bounded-delay information flow is C := sup {r [0, ] : ϵ E with η(ϵ) r a.s.}.

10 Page of Operations Research 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) The operational meaning of the above definitions becomes clear upon observing that a perfect (undetectable) masking of information-carrying packets can be obtained by embedding them into the normal transmission patterns corresponding to nodes which are not talking to each other, e.g., corresponding to statistically independent point processes [Blum, Song and Venkataraman (00)]. Conceptually, this can be achieved by supposing that Sally and Ray first generate two independent point processes according to the nominal traffic distribution, and then use the generated timestamps to send their packets: timestamps which meet the causal delay constraint are used to relay information packets, and the remaining timestamps are filled with chaff (i.e., dummy packets). In this way, the nodes mimic the independent processes exactly, and their transmission patterns will appear indistinguishable from those of non-communicating nodes, while an information flow of positive rate is actually sustained. The efficiency here is the fraction of information-carrying packets with respect to the total number of packets in cover traffic. Since the embedding capacity represents the highest achievable efficiency, C establishes a fundamental limit on the amount of information flow which can be embedded; smart nodes can send information flows with a rate up to C without being detected by any timing analyzer, by suitably multiplying the rate of normal transmissions. Therefore, computing the embedding capacity allows comparison of different traffic models in terms of their capability of hiding information flows, and is also relevant to the design of anonymity-compliant protocols for the future generations of networks [He, Agaskar and Tong (00), He, Tong and Swamy (00), He and Tong (00), Marano, Matta and Tong (00), Marano et al. (0), Marano et al. (0)].. Characterization of the embedding capacity C Given a realization of the cover traffic (S, R), there exists a simple algorithm yielding the best achievable embedding efficiency. This algorithm is known as Bounded Greedy Match (BGM), see [Blum, Song and Venkataraman (00)]. Let all the points in the realizations of the two processes S and R be initialized as undetermined. The BGM algorithm repeats the following steps:

11 Page 0 of Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) BGM Algorithm Consider the first (in the direction of increasing time) undetermined point in the process S, say p () ; Find the first undetermined point in the process R in the interval [p (), p () + ], if any, denoted by p () ; If such a point exists, mark both p () and p () as matched; otherwise, mark p () as unmatched. In either cases, mark all undetermined points in the process R before p () as unmatched. Matched and unmatched points are also referred to as flow and chaff packets, respectively. When running over a realization of cover traffic, the BGM algorithm matches pairs of points over the two streams in such a way that these pairs comply with the -delay constraint and that the maximum number of matched pairs is found [Blum, Song and Venkataraman (00)]. Therefore, the BGM algorithm implements the best embedding policy, so that its efficiency equals C. To get a clearer picture of how the BGM algorithm works, it is convenient to introduce an alternative (though equivalent) implementation thereof. To this aim, let us define the interarrivals {T (s) k, k } and {T (r) k, k } of the processes S and R, respectively. We set: T (s) such that S i = i T (s) k= k and R i = i := S, T (s) k :=S k S k for k, T (r) := R, T (r) k :=R k R k for k, T (r) k= k. As shown in [Marano et al. (0)], the BGM algorithms classifies packets according to the following rule, which also implicitly defines a random process {Y n, n 0}:

12 Page of Operations Research 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) BGM Algorithm (nd version) Initialization: I = ; J = ; Y 0 = 0; For each iteration n =,,... (a) Process Update Y n = Y n T (s) I n, if Y n >, Y n + T (r) I n T (s) I n, if 0 Y n, Y n + T (r) I n, if Y n < 0. (b) Packet Classification & Index Update if Y n I >, S In is chaff, n+ = I n +, J n+ = J n, if 0 Y n, S In and R Jn are flow, if Y n < 0, R Jn is chaff, I n+ = I n, J n+ = J n +. I n+ = I n +, J n+ = J n +, As seen from the above box, the BGM algorithm sequentially tests each pair of points in S and R for a match. The indices I n and J n denote the points in S and R, respectively, that are under consideration in the n-th iteration (n =,,... ). The delay between these two points Y n:=r Jn S In is computed recursively, and allows the algorithm to classify the packets. The above formalization is particularly convenient for our purposes, in that the number of samples of {Y n, n 0} lying inside the barriers 0 and equals the fraction of matches computed by the BGM. Thus, the asymptotic analysis of the process {Y n, n 0} is key to the computation of C. To perform such analysis we adhere from now on to the following assumptions. Assumptions. The cover traffic S and R are iid renewal point processes. The common probability density function (pdf) of the interarrivals T (s) n and T (r) n is denoted by f T (t). We only consider random variables admitting a density with respect to the Lebesgue measure, with a finite non-zero first moment, 0 < E[T ] < (occasionally, we omit the subscript n and the superscript (s) or (r), when these are immaterial). Let λ:=/e[t ] be the common rate of the cover traffic S and R. Under the above assumptions, the random process {Y n, n 0} is a Markov chain with the following properties:

13 Page of 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) The step of the chain starting outside the upper barrier (resp. lower barrier 0), is distributed as T (resp. +T ). The step of the chain starting inside the barriers, is distributed as the difference between independent and identically distributed interarrivals. Namely, the pdf of Y n Y n, when Y n [0, ], is given by the convolution between f T (t) and f T ( t), i.e., 0 f T (x)f T (x t) dx. As anticipated, the practical relevance of the Markov chain {Y n, n 0} for computing the embedding capacity stems from the fact that C is straightforwardly related to the fraction of time that the chain spends outside the barriers. Let p be such fraction and assume for now that it is a well defined quantity (the formal definition is in eq. ()). Given p, it is easy to show that the embedding capacity is: C = p p. () The asymptotic characterization of the chain is the main subject of the forthcoming section.. Random walks with permeable barriers It is expedient to symmetrize the Markov chain {Y n, n 0} by shifting it down by τ:= /, which clearly leaves unchanged the asymptotic properties we are interested in. The evolution of the symmetrized Markov chain can be compactly described by the following recursion: Set Y 0 :=0; for n : Y n T n, if Y n > τ, Y n := Y n + Z n, if Y n τ, Y n + T n, if Y n < τ, where {T n, n } is a collection of non-negative iid random variables with common pdf f T (t), {Z n, n } is a collection of iid random variables with common pdf f Z (t), and the two sequences are mutually independent. Note that, from the characterization of the BGM algorithm over renewal processes, the pdf of the inter-barrier steps Z n s is given by the convolution f Z (t) = 0 f T (x)f T (x t) dx, namely, the density of the difference between (independent) interarrivals. However, the Markov chain () is amenable to a more general formulation where the distribution of Z n is not necessarily related to that of T n. Accordingly, we focus on the general case that f Z (t) is still even-symmetric, namely f Z (t) = f Z ( t), but is otherwise arbitrary. With this broader assumption, the Markov chain () () models a wider variety of physical systems described by a particle symmetrically wandering within

14 Page of Operations Research 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) [ τ, τ]; the barriers τ and τ can be crossed but, when this happens, the particle is constrained to move back toward the region [ τ, τ]. The process in () will be accordingly referred to as random walk with permeable barriers. It can be shown that the Markov chain in () is ergodic, see [Marano et al. (0), Th. ]. Then, a little thught reveals that its stationary pdf, denoted by u(t), satisfies the following integral equation: τ u(t) = u(x)f T (t x)dx + + τ τ u(x)f T (x t)dx + u(x)f Z (t x)dx. () τ The asymptotic fraction of time the chain {Y n, n 0} spends outside the barriers [ τ, τ] can be formally defined as: p:= lim n n i= I{ Y i > τ} n a.s., () where I{ } is the indicator function whose value is one if the (event-)argument is true and zero if it is false. From ergodicity of (), it follows that the above limit exists and can be evaluated as p = t >τ u(t)dt. Next, we establish a connection between the probability p and the penetration depth of the chain in the region beyond the barriers, which is characterized in terms of the overshoot, namely, the excess over the barriers when the random walk crosses them. Formally, letting Y i be an out-ofbarrier sample, the corresponding overshoot is Y i τ. We focus on the long-term arithmetic mean of the overshoot, defined as n O:= lim ( Y i= i τ) I{ Y i > τ} n n I{ Y i= i > τ} a.s. () Our goal is to characterize O as a function of p, which is provided in the next result. In what follows, we assume finiteness of the second moments E[T ] and E[Z ]. (A further regularity condition, E[Z ] <, will be needed when developing the large-τ approximations in Sect..) Theorem. (Relationship between O and p) The long-term arithmetic mean of the overshoots is related to the fraction of time the chain spends outside the barriers, via the following formula O = ( p p E[Z ] E[T ] + E[T ) ] τ. () E[T ]

15 Page of 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) Remark. One way to interpret formula () is as follows. Define a random variable W such that Then eq. () can be written as { T with probability p, W = Z with probability p. () O = E[W ] τ, () E[W ] or, in other words, the excess over the barriers for a particle moving according to the Markov chain () is given by a normalized version of the mean square of W, minus the threshold level τ. Proof of Theorem Let us start by considering the segments of the Markov chain {Y n, n 0} in (), made of samples between two successive out-of-barrier values. Specifically, let I k be the index at which {Y n, n 0} exceeds the barriers for the k th time. Then, the k th segment is made of the sequence of random variables Y Ik +,..., Y Ik+, namely, starts at the first sample after the k th overshoot of {Y n, n 0}, and ends at the sample corresponding to the (k + ) th overshoot. The number of samples of the k th segment which are inside the barriers will be denoted by N k. In the case that the k th and the (k + ) th out-of-barrier values are consecutive samples of {Y n, n 0}, we have Y Ik + = Y Ik+, namely, there are no steps inside the barriers, and N k is accordingly set to zero. In terms of the above partitioning, the probability p of staying outside the barriers, see eq. (), can be expressed as p = lim k k k + k j=0 N j a.s., p = + N, () where N = lim k k k j=0 N j a.s. Indeed, as n diverges, the number k of samples beyond the barriers diverges as well, and we go through the same sample paths. By convention, we denote by N 0 the number of samples in the process {Y n, n 0} before the occurrence of the first out-of-barrier sample. We next relate N to O in (), which can be also rewritten as O = lim k k k j= O j a.s., where we have denoted the k th overshoot by O k = Y Ik τ. Now, suppose that the k th out-of-barrier sample of {Y n, n 0} occurs across the lower barrier, i.e., Y Ik = O k τ. Note that I k is determined This is ensured by the fact that the random variable N k, representing the number of internal steps in the k th segment, is finite with probability, see the subsequent discussion about the properties of the stopping times.

16 Page of Operations Research 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) only by Y n for n I k and therefore, subsequent steps are still distributed according to the original step distribution and are independent of Y Ik. Each segment in which the Markov chain has been partitioned defines a random walk that, skipping the explicit dependence upon k, can be simply denoted by {X n, n 0}. We have X 0 := Y Ik + = Y Ik + T = T O k τ, (0) n X n := X 0 + Z i, n, () i= N k := inf{n 0 : X n / [ τ, τ]}, () where we now write T in place of T Ik + for notational simplicity. Note that N k defined by the stopping rule () is a well-behaved random variable with E[N k ] <. Indeed, N k is zero if X 0 exceeds the barriers. Else, N k is the stopping time of a random walk with steps Z n s, starting from X 0 inside the barriers. Denoting by L τ the exit time of a random walk n i= Z i, with starting point 0 but with augmented thresholds τ and τ, we have P[N k n] P[L τ n], irrespectively of O k and of the value of X 0. This implies E[N k ] <, because it is known that E[L τ ] <, see e.g., [Gallager ()]. If the k th overshoot is across the upper threshold, we have X 0 = Y Ik T = O k + τ T, and the stopping time becomes N k = inf {n 0, : O k + τ T + n i= Z i / [ τ, τ]}. However, such N k is statistically equivalent to N k = inf {n 0 : T O k τ + n i= Z i / [ τ, τ]}, thanks to the symmetry of the distribution of the Z i s. Therefore, regardless of which barrier is under consideration, the k th segment between two out-of-barrier samples in the process {Y n, n 0} is statistically described by process () with initial condition (0), and associated stopping time (). Accordingly, the (k + ) th overshoot of the process {Y n, n 0} can be written as Let us define M n :=X n X 0 ne[z ]. We have O k+ = X Nk τ. () E[M n X 0,..., X n ] = E[(X n + Z n ) X 0 ne[z ] X 0,..., X n ] = X n X 0 (n )E[Z ] + X n E[Z n ] = M n.

17 Page of 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) The process {M n, n 0} is thus a martingale with respect to {X n, n 0} (finiteness of the second moments are assumed, when needed). Then, the martingale optional stopping theorem [Ross (), Gallager (), Karlin and Taylor ()] implies 0 = E[M 0 ] = E[M Nk ] = E[X N k ] E[X 0] E[N k ]E[Z ], or, E[X N k ] E[X 0] = E[N k ]E[Z ]. () From eq. () we get E[X N k ] = E[(O k+ + τ) ], that substituted in eq. (), and further using eq. (0) for X 0, gives E[N k ]E[Z ] = E[(O k+ + τ) ] E[(T O k τ) ]. () In order to evaluate the limiting overshoot, let us fix the initial point Y 0 = O τ, with O distributed according to its invariant distribution, a choice which does not affect the long term averages O and N; also the distribution of N k does not change with k, by construction. Thus, {N k, k 0} is stationary and ergodic. We finally have O = E[O k ] = E[O k+ ], E[O k] = E[O k+], N = E[N k ], where all the expectations have been evaluated under the pertinent stationary distributions. Substituting the above equalities in eq. () gives N = E[T ] ( O + τ ) E[T ]. () E[Z ] Using eq. (), the desired claim () follows by straightforward algebra.. Asymptotic approximations of the overshoot Here we provide the characterization of the overshoots in the limit of large values of τ, which turns out to be ruled by the behavior of the chain inside the barriers, under the technical assumption E[ Z ] <. The bottom line of the forthcoming analysis is a simple approximation for O in the regime of large τ: O O := E[H ] E[H], () An alternative way to get the same result is that of focusing on ergodicity of the vector chain {O k, N k } and working in terms of the invariant joint distribution.

18 Page of Operations Research 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) Figure (a) How to select the ladder heights from the original (single-barrier) chain [panel (a)], to build the extracted chain [panel (b)]. Samples above the threshold are marked by filled circles and are left unchanged, while for samples below the threshold, shown as dots in (a), we retain only those complying with the ladder height extraction rule, as described in the main text; these extracted samples are shown as empty circles in (b). The extracted process in (b) (made of filled and empty circles) is a Markov chain formally described by eq. (). where H is the increment of the ladder heights (which are defined below) associated with the random walk n i= Z i. Derivation of formula () Simplified Markov chain with one barrier: To obtain the above result, we start by building a simplified version of the Markov process {Y n, n 0} in eq. (), which retains the relevant properties of the original Markov chain in the regime of large τ, and is amenable to analytical solution. Let us refer to Fig.. We assume, without loss of generality, that the Markov chain evolves from an initial point lying near the upper barrier τ, and let us first make the crude assumption of setting the lower barrier to. The evolution of this simplified (i.e., single-barrier) Markov chain is depicted in Fig. (a), where the samples above τ are shown as filled circles, and samples below τ are shown as dots. Consider now a sample of the chain below the barrier immediately preceded by a sample larger than τ. The characterization of the overshoots is not altered if, starting form this sample, we retain only the increasing samples, until the next sample above τ is encountered. Otherwise stated, we forget about the possible zigzag of the chain due to its negative steps, extracting only the (b) increasing values, known as ascending ladder heights, see [Feller ()]. This procedure leads to

19 Page of 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) the process shown in Fig. (b), where the extracted ladder heights are marked by empty circles. In this way, the single-barrier Markov chain has been replaced by a new chain, whose evolution below τ is dictated by the ladder heights. It is easy to see that the increments of the ladder heights are iid non-negative random variables, and are independent from the T n s in eq. (), such that the new extracted process is again a Markov chain. The increment variable, denoted by H, can be conveniently defined by means of an associated stopping time N H : N H = inf { n : } n Z i > 0, H = i= The overall extracted process exemplified in Fig. (b) can be now formally described as { Yn T Y n := n, if Y n 0, Y n + H n, if Y n < 0, where the barrier τ has been set to 0 for notational ease, without losing generality. Here {H n, n } is a collection of iid random variables distributed as H, with pdf f H (t), and {T n, n } is a collection of non-negative iid random variables with common pdf f T (t); the two sequences are mutually independent. Note that the modified process is always increasing below the threshold (with increments given by the random variables H n s). Conversely, it is always decreasing when above the threshold (with increments T n s). Steady-state distribution of the simplified Markov chain: The study of the extracted Markov chain () is now in order. It is easy to show that the chain () is ergodic and its stationary density u(t) obeys 0 u(t) = u(x)f H (t x)dx N H i= Z i. () u(x)f T (x t)dx. () Equation () is a homogeneous Fredholm equation of the second kind, and can be classified as a convolution-type equation with two distinct kernels. A powerful Fourier-domain We wish to avoid confusion here: the time index n in the above recursion is used with a slight abuse of notation, and should not be confused with the time index n in the original chain (). Indeed, while the characterization of the overshoots can be made looking only at the extracted chain, such an extraction entails a rescaling of the time axis, implying that the information about the time spent by the chain below the threshold τ is lost in this picture. For notational convenience the stationary density of the extracted chain is denoted by the same symbol u(t) used for the stationary density of the original chain.

20 Page of Operations Research 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) approach for this problems can be traced back to Carleman and to Wiener and Hopf, see, e.g., [Polyanin and Manzhirov (00)]. After Fourier transformation, the problem falls in the class of the so-called Riemann-Hilbert boundary value problems, which, in its simplest version, consists of finding two functions, analytic in the upper and lower half planes of the complex plane, respectively, whose difference on the real axis equals a known function [Polyanin and Manzhirov (00), Muskhelishvili (00)]. This kind of approach has been already largely used in the applied sciences, e.g., for solving partial differential equations [Noble ()], in electromagnetics [Jones ()], and has been recently exploited in [Marano, Matta and Tong (00), Marano et al. (0), Marano et al. (0)]. We now show how to apply this approach for a quick solution of eq. (). Let u(t) = u + (t) u (t), where we define u + (t):=u(t)(t), u (t):= u(t)( t), where (t) is the Heaviside function, taking value for t 0 and 0 otherwise. By transforming eq. () in the Fourier domain t f, we have U + (f) U (f) = U (f)ψ H (f) + U + (f)ψ T (f), (0) where denotes complex conjugation, U + (f) and U (f) are the Fourier transforms of u + (t) and u (t), i.e., U + (f) = + u + (t)e iπft dt, U (f) = + u (t)e iπft dt, and Ψ T (f), Ψ H (f) are the Fourier transforms of f T (t), f H (t), respectively. Equation (0) is equivalent to U + (f) Ψ H (f) = U (f) () Ψ T (f). The Fourier integrals appearing in this expression can be certainly extended to the complex plane, in the regions where they are absolutely convergent, by replacing the real variable f with the complex variable z = f + i α. It can easily seen that the left-hand side of () with f replaced by z is an analytic function in the region I(z) > 0, continuous on the real axis z = f, with a single pole located at z = 0. Since Ψ H(0) = iπe[h], this pole is of order one. Similarly, the right-hand

21 Page 0 of Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) side of () is analytic in the region I(z) < 0, continuous on z = f, with a pole of order one at z = 0. These two functions of the complex variable z, defined in the upper and lower half-planes, take the same values on the real axis z = f, as prescribed by (). Then, by analytic continuation [Rudin ()], they define a single function over the whole complex plane z, which is analytic everywhere except for a pole of order one at the origin z = 0. The generalized Liouville theorem [Rudin ()] implies that such a function of the complex variable z takes the simple form c/z where c is an arbitrary complex constant. On the real axis this means or Recalling that u(t) is a pdf, we have U + (f) Ψ H (f) = U (f) Ψ T (f) = c f, U + (f) = c Ψ H(f), U (f) = c Ψ T (f). () f f U(0) = U + (0) U (0) = c i πe[h] c i πe[t ] = c = i π E[H] + E[T ]. Transforming back to the t domain one obtains yielding u + (t) = F H(t) E[H] + E[T ] (t) and u (t) = F T ( t) E[H] + E[T ] ( t) u(t) = F H(t) F T ( t). () E[H] + E[T ] This equation gives the complete asymptotic characterization of the extracted Markov chain (). Steady-state distribution of overshoot: Recall that we are interested in evaluating the long-term average (). To this aim, one can consider the Markov chain made only of overshoots, say it {O k, k }, whence O is easily rewritten as: O = lim k k k j= O j a.s. The generalized Liouville theorem actually states that the function is in the form c/z + c 0, but since we seek for Fourier-transformable functions vanishing at infinity, in our case c 0 = 0.

22 Page of Operations Research 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) Under the assumption that the lower barrier is removed, the invariant pdf of the overshoot Markov chain {O k, k } equals the conditional pdf of the Markov chain () given that the process stays above the threshold, namely f O (t) u(t)(t) u(t)dt = u+ (t) U + (0) = F H(t) (t). () E[H] 0 In the above analysis, we have started from a sample close to the upper barrier, and neglected the lower threshold. The overall analysis can be repeated in a symmetric fashion by starting from a sample lying near the lower barrier and neglecting the upper one, obtaining exactly the same invariant pdf of the overshoot (the invariant density of the chain is in fact simply mirrored). Unfortunately, we know that eq.() is only an approximation, due to the fact that the presence of both barriers should be taken into account, because the event of a very long walk n i= Z i inside the barriers, terminating with a faraway-threshold crossing, is unlikely but not impossible. Thus, the true invariant pdf of the overshoot Markov chain {O k, k } must account for the possibility of this rare switching between upper and lower barriers. This is done by means of the following plausibility argument. Let us start from a sample close to the upper threshold, so that the rare event is the lowerthreshold crossing. The distribution of the overshoot, given that it occurs on the distant threshold, can be approximated by that of the excess of a random walk n i= Z i with respect to a faraway boundary τ. This excess can be equivalently regarded as the so-called residual life [Ross ()] of the descending ladder heights of n i= Z i, whose (negative) increments are distributed as H. The pdf of such residual life (e.g., the first overshoot), in the limit of large barriers, is known from standard renewal theory [Feller ()], and is again F H (t) E[H] (t), namely, the stationary density we found in () when neglecting the distant threshold. The above argument is essentially unchanged when starting from below. This finding suggests taking the pdf () as the invariant pdf of the entire overshoot Markov chain {O k, k }. To be precise, the sought distribution should be computed in the conditional space where the upper threshold is not crossed before the lower threshold. On the other hand, for this rare event to occur, the walk is expected to evolve (and randomly oscillate) for a long time inside the barriers in order to complete the path from τ to τ. The conditioning is accordingly expected to become less and less influent as τ grows, such that we propose to ignore it, and simply consider the excess of n i= Z i with respect to a boundary τ away.

23 Page of 0 0 Article submitted to Operations Research; manuscript no. (Please, provide the mansucript number!) Back to the derivation of eq. (), the average overshoot O can be now evaluated by computing the statistical expectation under the pdf in eq. (), yielding: which is the claimed result. O = E[H] 0 t[ F H (t)] dt = E[H ] E[H], () It remains to provide a convenient expression for the ratio E[H ]/(E[H]) where, we recall, H is the increment of the ladder heights associated to the random walk n i= Z i. It turns out that the statistical characterization of such increment for random walks (both with zero and nonzero drift) is a largely investigated topic, see, e.g., [Spitzer (), Feller (), Lai (), Nagaev (00)]. A formula due to Spitzer [Spitzer ()] (see also [Gut (00), form.. p. ]) adapted to our case where f Z (t) = f Z ( t), yields E[H] = σ z/, where σ z := VAR[Z]. More involved are the explicit formulas for E[H ], see for instance [Lai (), Nagaev (00), Keener ()]. Among the many different methods proposed for evaluating E[H ], we find here convenient to adopt the integral formula provided in [Keener ()], which, tailored to our case, yields E[H ]/(E[H]) in the form σ O = z + π 0 [ ] Ψ Z (f) Ψ Z (f) + df f ( + (πσ z f) ) f, () where Ψ Z (f) is the Fourier transform of f Z (t), and Ψ Z(f) denotes its derivative with respect to the argument f. The integral in () can be easily solved numerically. Before ending this section, we would like to investigate what happens in the case that τ 0, meaning that the process {Y n, n 0} is always outside the barriers, with positive samples enforced toward the negative values and vice versa. Although the behavior of the long-term mean overshoot for zero threshold can be directly obtained from formula (), it is instructive to derive that by means of the invariant density of the corresponding Markov chain. Consider hence a random process evolving according to iteration () when τ is set to zero: { Yn T Y n := n, if Y n 0, () Y n + T n, if Y n < 0, with n, and T n s non-negative iid random variables. This is equivalent to the chain () with H n replaced by T n. Therefore, specializing eq. (), the stationary pdf of the chain is u(t) = F T ( t ) E[T ], yielding: f O (t) = F T (t) E[T ] (t) O zero := E[T ] E[T ]. ()