A Synchronization Scheme for Stored Multimedia Streams

Transcription

1 In B. Butsher, E. Moeller, and H. Push, editors, Interative Distributed Multimedia Systems and Servies (European Workshop IDMS 96, Berlin, Germany), volume 1045 of LNCS, pages Springer Verlag, Heidelberg, Germany, Marh 1996 A Synhronization Sheme for Stored Multimedia Streams Werner Geyer 1, Christoph Bernhardt, Ernst Biersak Institut Euréom 2 Abstrat: Multimedia streams suh as audio and video impose tight temporal onstraints due to their ontinuous nature. Often, different multimedia streams must be played out in a synhronized way. We present a sheme to ensure the ontinuous and synhronous playout of stored multimedia streams. We propose a protool for the synhronized playbak and we ompute the buffer required to ahieve both, the ontinuity within a single substream and the synhronization between related substreams. The sheme is very general beause it only makes a single assumption, namely that the jitter is bounded. 1 Introdution 1.1 Motivation Advanes in ommuniation tehnology lead to new appliations in the domain of multimedia. Emerging high-speed, fiber-opti networks make it feasible to provide multimedia servies suh as Video On-Demand, Tele-Shopping or Distane Learning. These appliations typially integrate different types of media suh as audio, video, text or images. Customers of suh a servie retrieve the digitally stored media from a video server [Ber95] for playbak. 1.2 Multimedia Synhronization Multimedia refers to the integration of different types of data streams inluding both ontinuous media streams (audio and video) and disrete media streams (text, data, images). Between the information units of these streams a ertain temporal relationship exists. Multimedia systems must maintain this relationship when storing, transmitting and presenting the data. Commonly, the proess of maintaining the temporal order of one or several media streams is alled multimedia synhronization [Eff93]. Continuous media are haraterized by a well-defined temporal relationship between subsequent data units. Information is only onveyed when media quanta are presented ontinuously in time. As for video/audio the temporal relationship is ditated by the sampling rate. The problem of maintaining ontinuity within a single stream is referred to as intra-stream synhronization. Moreover, there exist temporal 1 Now with: Praktishe Informatik IV, University of Mannheim, Mannheim, Germany, geyer@pi4.informatik.uni-mannheim.de Route des Crêtes, Sophia-Antipolis Frane, Phone: , FAX: , {bernhard,erbi}@eureom.fr 1

2 relationships between media-units of related streams, for instane, an audio and video stream. The preservation of these temporal onstraints is alled inter-stream synhronization. To solve the problem of stream synhronization we have to regard both issues whih are tightly oupled. One an distinguish between life synhronization for life media streams and syntheti synhronization for stored media streams [Ste93a]. In the former ase, the apturing and playbak must be performed almost at the same time, while in the latter ase, samples are reorded, stored and played bak at a later point of time. For life synhronization, e.g. in teleonferening, the tolerable end-to-end delay is in the order of a few hundred milliseonds only. Syntheti synhronization of reorded media stream is easier to ahieve than life synhronization: higher end-to-end delays are tolerable, and the fat that soures an be influened proves to be very advantageous as will be shown later. It is, for instane, possible to adjust playbak speed or to shedule the start-up times of streams as needed. However, as resoures are limited, it is desirable for both kinds of synhronization to keep the buffers required as small as possible. [Koe94] 1.3 Related Work Esobar et al. [Es94] and Rothermel et al. [Rot95b] propose a sheme that requires globally synhronized loks. Their synhronization mehanism relies on time stamps to determine the different kind of delays eah stream experienes, using time stamps. At the reeiver different delays are equalized to the maximum delay by buffering. Rothermel enhanes this basi mehanism with a buffer level ontrol and a master-slave onept. Rangan et al. [Ran93] present a synhronization tehnique based on feedbak. Synhronization is done at the senders side, assuming that the reeiver stations send bak the number of the urrently displayed media-unit. Asynhrony an be disovered by the use of so-alled relative time stamps (RTS). Synhrony is restored by deleting or dupliating media-units. Trigger pakets are exhanged periodially so to alulate the relative time deviation between sender and reeiver. Agarval et al. [Aga94] adopt the idea of Rangan and enhane the sheme by dropping the assumption of bounded jitter. Our synhronization sheme is inspired by the work of Santoso [San93] intrastream synhronization and the work of Ishibashi et al. [Ish95] on intra-stream synhronization and inter-stream synhronization. Ishibashi proposes a time-stamp-based synhronization and applies a onept based on delay estimations to perform synhronization in ase of unknown delay. One intra-stream synhronization is established, inter-stream synhronization an be maintained with a ertain probability. Corretive ations are taken by skipping/pausing. The sheme assumes no lok drift. 1.4 Context of the Synhronization Problem The synhronization problem addressed in this paper is motivated by our work on salable video servers. We have designed and implemented a video server, alled server array, onsisting of n server nodes. A video is distributed over all server 2

3 nodes using a tehnique alled sub-frame striping: Eah video frame is partitioned into n equal size parts, alled sub-frames, that are stored on the n different servers. If F i = { i, 1,, i, n } denotes the set of sub-frames for frame f, then: i i, j = f i j = 1 n The server array with the synhronization mehanisms presented in this paper has been fully implemented as a prototype [Ber95]. During playbak, eah server node is ontinuously transmitting its (sub-frames) to the lient. The transfer is sheduled so, that all striping bloks that are part of the same frame are ompletely reeived by the lient at the deadline of the orresponding frame. The lient reassembles the frame by ombining the sub-frames from all server nodes. An example for n=3 with eah server sending with a rate of r frames per seond is depited in figure 1. Substream 2 Frame f i sub-frame i,2 Frame f i+1 sub-frame i+1,2 Substream 1 sub-frame i,1 sub-frame i+1,1 Substream 0 sub-frame i,0 1/r sub-frame i+1,0 1/r time Fig. 1. Temporal Relationship for Sub-Frame Striping. 2 Synhronization Protool 2.1 Overview We propose a synhronization sheme for stored media that ahieves both, suitable intra- and inter-stream synhronization. The sheme is reeiver-based and does not assume global loks. To initiate the playbak of a stream in a synhronized manner we introdue a start-up protool. Our protool has been mainly influened by the ideas of Ishibashi [Ish95] with respet to intra- and inter-stream synhronization. Based on Santoso s work [San93] we derive buffer requirements and playout deadlines to assure inter- and intra-stream synhronization. For re-synhronization, we adopt sheme similar to the one desribed by Koehler and Rothermel [Koe94], [Rot95]. 3

4 We derive our synhronization sheme by step-wise refinement: We first develop a solution for the ase of zero jitter and then relax this assumption requiring bounded jitter only. We present two models: Model 1 overs the problem of different but fixed delays on the network onnetions for eah substream. We propose a synhronization protool that ompensates for these delays by omputing well-defined starting times for eah server. The protool allows to initiate the synhronized playbak of a media stream that is omposed of several substreams. Model 2 takes into aount the jitter experiened by media-units travelling from the soure to the destination. Jitter is assumed to be bounded. To smoothen out jitter, elasti buffers are required. Our sheme guarantees a smooth playbak of the stream and has very low buffer requirements. Model 2 overs intra-stream synhronization as well as inter-stream synhronization. For the proposed synhronization sheme, we assume that a lient D is reeiving sub-streams from different servers 3. Client and servers are interonneted via a network (see figure 2). S S Network D S Fig. 2. Distributed Arhiteture for the Synhronization Sheme. Eah of the servers denoted by S delivers an independent substream of mediaunits (sometimes referred to as frames in the following). The prodution rate is driven by the server lok. Arriving media-units are buffered in FIFO queues at the destination D. The playout of the entire stream, omposed of the substreams, is driven by the destination s lok. 3 It is also possible that a single server sends multiple substreams to a lient. Our model is more general and overs this ase too. 4

5 While the use of globally synhronized loks failitates synhronization, our synhronization sheme does not assume the presene of a global time or synhronized loks. 2.2 Soures of Asynhrony Several soures of asynhrony exist in the onfiguration desribed in the previous setion. These are: Different delays: the assumption of independent network onnetions imposes different delays. A synhronization sheme has to ompensate for these differenes in order to display the ontinuous media stream in a timely order. Beside the network delay, media-units experiene a delay for the reasons of paket-size/depaketizing, the proessing through the lower protool layers, and the buffering on the lient site. The variation of delay is defined as jitter. Network jitter: asynhronous transfer destroys synhrony. Jitter arises in intermediate nodes for the reason of buffering. End-system jitter: paketizing and depaketizing of media-units with different size due to enoding introdues jitter as well as passing media-units through the lower protool layers. Clok drift between the loks in the servers and in the lient is present beause we do not assume global loks. Change of the average delay: the synhronization sheme has to be adaptive with respet to a hange of the average delay. Server drop outs due to proess sheduling are a realisti assumption when using non-real-time operating systems. At the same time, the onsideration of drop outs overs the overload probability of statistial admission ontrol strategies to a ertain amount. 2.3 Assumptions Our synhronization mehanism uses time stamps. Eah time a media-unit 4 is sheduled by a server, it is stamped with the urrent loal time. This enables the lient to alulate statistis, suh as for the roundtrip delay, jitter, or inter-arrival times. Moreover, we assume that eah media-unit arries a sequene number for determining media-unit order. In ontrast to other approahes, buffer requirements or fill levels are always stated in terms of media-units or time, instead of the amount of alloated memory. This onsideration is preferred beause synhronization is a problem of time and for ontinuous media, time is represented impliitly by the media-units of a stream. This seems reasonable beause media-unit sizes vary due to enoding algorithms like JPEG or MPEG [Koe94]. However, notie that a mapping of media-units to the alloation of bytes 4 We will also use the abbreviation mu for media-unit. 5

6 must be arried out for implementation purposes. Taking the largest media-unit of a stream as an estimate wastes a lot of memory, espeially when using MPEG ompression. Sophistiated solutions of mapping are subjet of future work. In the following, we will use the term buffer slot to denote the buffer spae for one media-unit. Sine proessing time, e.g. for protool ations does not onern the atual synhronization problem we will neglet it whereas an implementation has to take it into aount. Finally, we assume that ontrol messages are reliably transferred. Model Parameters. n number of server nodes in the server array N number of media-units of a stream i,j, υ media-unit index i, j, υ= 0,..., N-1 k server index k = 0,..., n-1 I j index set of n subsequent media-units starting with media-unit j S k denotes server node k providing substream k D denotes the destination or lient node s i initial sending time of media-unit i in server time [se] s i synhronized sending time of media-unit i in server time [se] a i arrival time of media-unit i in lient time [se] d i roundtrip delay 5 for media-unit i measured at lient site [se] d max maximum roundtrip delay [se] d max, j maximum roundtrip delay for all j element of I j [se] t start starting time of the synhronization protool [se] t ref referene time for the start-up alulation [se] j t ref referene time regarding the set of media-units given by I j [se] t i expeted arrival of the media-unit i at the lient site [se] arrival time differene between media-unit i and j [se] δ ij A set of media-units that needs to be played out at the same time is referred to as synhronization group. We assume that media-units are distributed in a round robin fashion aross the involved server nodes. Hene, we an identify the storage loation of a media-unit by its media-unit number 6, i.e. Server S i mod n stores the media-unit i. (1) 5 The roundtrip delay omprises the delay for a ontrol message that requests a media-unit and the delay for delivering the media-unit 6 This implies that eah substream will send media-units at the same rate. An extension of the sheme to different media-unit rate, eah one being the integer multiple of a base rate is straight forward. 6

7 The leads to the following formulation of the synhronization problem: The lient must playout the media-units of all subsets I j, with j mod n = 0, at the same time. 2.4 Model 1: Start-Up Synhronization Introdution Under the assumption of onstant delay and zero jitter, we solve the synhronization problem by assuring that the first n media-units, whih form a synhronization group, arrive at the same time at the lient. We therefore need t i = t 0 i I 0 (2) The major problem addressed by model 1 is the ompensation for different delays due to the independene of the different substreams. For instane, the geographial distane from server to lient may be different for eah server. Thus, starting transmission of media-units in a synhronized order would lead to different arrival times at the lient with the result of asynhrony. Usually, this is ompensated by delaying mediaunits at the lient site [Es94]. Depending on the loation of the soures large buffers may be required. In order to avoid buffering to ahieve the equalization of different delays, we take advantage of the fat that stored media offers more flexibility: The idea is to initiate playout at the servers suh that media-units arrive at the sink site in a synhronous manner. This is performed by shifting the starting times of the servers on the time axis in orrelation to the network delay of their onnetion to the lient. The proposed startup protool onsists of two phases. In the first phase, alled evaluation phase, roundtrip delays for eah substream are alulated, while In the seond phase, alled synhronization phase, the starting time for eah server is alulated and transmitted bak to the servers. The model is based on the assumption of a onstant end-to-end delay without any jitter. We further exlude hanging network onditions, server drop-outs, and lok drift. In suh a senario, synhronization needs to be done one at the beginning and is maintained afterwards automatially. We need to introdue some more notation to express interdependenies between the parameters of the model. We then give a desription of the start-up protool flow and prove its orretness. We lose the setion with an example for the protool. The starting time t start of the protool equals the beginning of the first phase. Without loss of generality let t start = 0 (3) 7

8 To begin with, we regard the first n media-units of a stream given by I 0 that are distributed aross the n servers. The roundtrip delay d i for the media-unit i is given by the differene between its arrival time a i and the starting time of the synhronization protool d i = a i t start ij, I 0 (4) Equation (5) omputes the maximum of the roundtrip delay for all n substreams d max = max { d i i I 0 } (5) The seond phase of the protool begins at time t ref, whih is determined by the last of the first n media-units that arrives. t ref = max { a i i I 0 } (6) The differene between the arrival times of arbitrary media-units i and j is needed to alulate the starting times of the servers. We define the differene as follows. δ ij = a i a j ij, (7) Start-Up Protool The synhronization protool for starting playbak on the server sites is launhed after all involved parties are ready for playbak. It an be divided into two phases: evaluation phase and synhronization phase. The goal of the first phase is to ompute the roundtrip delays d i i I 0 for eah onnetion, while the seond phase alulates the starting times and propagates them bak to the servers. During start-up, the lient sends two different kinds of ontrol messages to the servers: Eval_Request(i): Client D requests media-unit i from Server S i, i I 0. Syn_Request(i, s i ): Client D transmits the starting time s i to server S i. (a) Evaluation Phase At loal time t start, lient D sends an Eval_Request(i) to Server S i, i I 0. Server S i reeives the Eval_Request(i) at loal time s i, i I 0. Server S i sends media-unit i time-stamped with s i immediately bak to lient D, i I 0. At loal time a i, lient D reeives media-unit i from Server S i, i I 0. At loal time t ref, lient D has reeived the last media-unit. The roundtrip delay d i = a i t start i I 0 and the relative distane between media-unit arrivals δ ij = a i a j ij, I 0 are omputed. 8

9 (b) Synhronization Phase At loal time t ref, lient D omputes t 0 as t 0 = max { t ref + d i i I 0 }, the maximum round trip time as d max = max { d i i I 0 }, the index ν that determines t 0 as υ = { j I 0 t ref + d j = t 0 }, and the delay differenes as δ νi = a ν a i, i I 0 With these results the starting time of Server S i is alulated in server time s i = s i + d max + δ υi, i I 0. Client D sends a Syn_Request(i, s i ) to server S i, i I 0. At loal time s i + d i + ( t ref a i ) server S i reeives the Syn_Request(i, s i ), i I 0. At loal time s i, server S i starts sheduling of the substream by sending mediaunit i, i I 0. At loal time t i, lient D reeives media-unit i, i I 0. At any time, only one synhronization group of n media-units must be buffered at the lient; after the omplete reeption the media-units are played out immediately. To show the orretness of our mehanism we disuss the Calulation of t 0 Calulation of s i (a) Calulation of the Earliest Possible Playout Time t 0 for the First Media-unit We need to hoose t 0 suh that all media-units i I 0 an be delivered and played out in time, i.e they will arrive at their deadline t i given by (2). It is obvious that mediaunit i I 0 delivered by server S i an not be expeted earlier than t ref + d i. Hene, the substream with the largest delay determines t 0. Theorem 1: Let t 0 = max { t ref + d i i I 0 }. Then all media-units an be delivered and played out in time. Proof: Sine the earliest possible arrival time for media-unit i I 0 is t ref + d i we need to show that t i t ref + d i i I 0. Let t 0 = max { t ref + d i i I 0 } t 0 t ref + d i i I 0 (2) t i = t 0 t ref + d i i I 0 t i t ref + d i i I 0 t 0 does not violate the arrival times of other substreams To show that t 0 is minimal, we assume that t 0 < t 0 i 0 I 0 with t 0 < t ref + d i0 (2) t 0 = t i0 < t ref + d i0 ontradition to the earliest possible arrival time. For the alulation of the future starting times s i, i I 0, we define the substream ν determining t 0 as follows: 9

10 ν = { j I 0 t ref + d j = t 0 } (8) (b) Calulation of the Synhronized Sending Time of Media-unit i for Server S i Substream υ an be onsidered ritial sine it determines the starting times of all other initial substreams. It is therefore onsidered as a referene point to whih all other substreams are adjusted. Clearly, the future starting time s i of substream i is omposed of the initial starting time s i plus the maximum roundtrip delay d max. This sum is orreted by the relative arrival time distane δ υi between media-unit i and media-unit υ. This gives the starting time that provides a simultaneous arrival of the media-units of substream i and these of substream υ. The alulation based on (8), (7), (4) is stated in the following theorem. Theorem 2: Let = s i + d max + δ υi, i I 0. Then media-unit i I 0 will arrive at lient time t i. Proof: For eah i I 0 : At lient time t ref, s i is sent bak to server S i whih reeives it at server time s i + d max (see figure 3). If S i sent media-unit i immediately bak to D, it would arrive at lient time t ref + d i. The term s i + d max is orreted by δ υi. Media-unit i will arrive at D at lient time t ref + d i + δ υi = t ref + ( a i t start ) + ( a υ a i ) (3) = t ref + a υ = t ref + d υ (theorem 1) = t 0 (2) = t i One an easily imagine situations where synhronization is needed not only at the beginning of a stream. A typial example is the VCR funtion pause. After having paused it beomes neessary to resynhronize again, starting with the media-unit subsequent to the last one displayed. The desribed sheme an be generalized to any series of subsequent media-units requested by the lient. Example of the Start-Up Protool s i The following example in figure 3 illustrates model 1. We assume n = 3, i.e. three servers, with one substream eah. The alulated starting values are shown in table 1. For eah server and for the lient D a time axis is provided. Arrows indiate ontrol messages or media-units, respetively, that are transferred between lient and servers. With t 0 = max { t ref + d i i I 0 } = max { 23, 18, 24} = 24 we get υ = 2. Substream 2 experienes the longest roundtrip delay d max and determines therefore t ref. Substream 2 is ritial beause it annot be started earlier than s As indiated on the time axis for server 0, substream 0 ould be started earlier but is delayed to arrive at the same time as substream 2. s i 10

11 S 0 S 1 D S 2 t start s 1 Evaluation Phase s 0 d max a 1 s 2 d max a 0 a 2 = t ref Synhronisation Phase δ 02 = 1 s 0 s 1 s 2 d max t 0 = t 1 = t 2 Fig. 3. Example of the Start-Up Synhronization Protool Flow. Server a i d i t ref δ 2i s i S s S s S s Table 1. Example for the Start-Up Calulations. 11

12 2.5 Model 2: Intra- and Inter-Stream Synhronization Introdution Model 1 shows how to ope with different delays for eah substream. However, synhronization is performed under the assumption that jitter does not exist. Model 2 loosens this assumption and takes into aount end-system jitter and network jitter. For our onsiderations, we regard the aumulated value of all soures of jitter desribed in setion. Furthermore, we assume that the jitter is bounded. When subjet to jitter, media-units will not arrive in a synhronized manner although they have been sent in a orret timely order. The temporal relationship within one substream is destroyed and time gaps between arriving media-units vary aording to the ourred jitter. Thus, an isohronous playbak annot be ahieved if arriving media-units of a substream would be played out immediately. Furthermore, jitter may distort the relationship between media-units of a synhronization group. Hene, intra-stream synhronization as well as inter-stream synhronization is disturbed. To smoothen out the effets of jitter, media-units have to be delayed at the sink suh that a ontinuous playbak an be guaranteed. Consequently playout buffers orresponding to the amount of jitter are required. The main point addressed by model 2 is intra- and interstream synhronization and the alulation of the required buffer spae. First, we regard the synhronization of a single substream. Based on a rule of Santoso [San93] we formulate a theorem that states a well defined playout time for a substream suh that intra-stream synhronization an be guaranteed. Using this so-alled playout deadline we derive the required buffer spae. Smooth playout annot be guaranteed if starting before playout deadline. Starting at a later time would require more buffer spae. Afterwards, we will extend our onsiderations to the synhronization of multiple substreams. The main idea in order to ahieve inter-stream synhronization is to maintain intra-stream synhronization for eah substream [Ish95]. Eah one of the substreams is assumed to have a different jitter bound. In this ase, buffer reservation aording to a single substream is not suffiient anymore as inter-stream synhronization will be disturbed for the reason of differenes in the jitter bounds. Additional buffering is required to ompensate for this. Furthermore, the playout deadline is modified with respet to multiple substreams. Finally, we examine the effets of the start-up protool (model 1) on buffer requirements in the ase of jitter. The appliation of model 1 to initiate playbak of the servers in a synhronized manner an introdue an error for the reason of jitter. We give a worst ase estimate for the error and additional buffer requirements are omputed aordingly. We begin with an extension of the model parameters used so far. Model Parameters k substream or server index, k = 0,..., n-1 r requested display rate of eah substream at lient site [mu/se] 12

13 maximum delay for substream k [se] min d k minimum delay for substream k [se] d k average delay for substream k [se] k jitter for substream k [se] max maximum jitter of all substreams [se] + k maximum upper deviation from d k due to jitter for substream k [se] - k maximum lower deviation from d k due to jitter for substream k [se] max+ maximum upper deviation of all substreams [se] b k buffer requirement for substream k on sink site [mu] S b k buffer requirement for substream k on sink site with shifting [mu] M b k buffer requirement for substream k on sink site with max. jitter [mu] B total buffer requirement for a synhronization group [mu] B S total buffer requirement for a synhronization groupwith shifting [mu] B M total buffer requirement for a synhronization group with max. jitter [mu] Throughout this paper we assume bounded jitter and we use the definition of jitter given by Rangan et al. [Ran92] who define jitter as the differene between the maximum delay and the minimum delay 7. d k max k = d k max d k min k (9) max = max { k k { 0 n 1} } (10) In addition to this, we need a jitter bounds defined as the deviation from the average delay d k. Jitter is in general not distributed symmetrially. Thus, k and k must + - not be equal. For further onsiderations, we assume interdependenies as follows. + - k = k + k k d k max d k min + = d k + k = k - d k k k (11) (12) (13) max+ = max { + k k { 0 n 1} } (14) Synhronized Playout for a Single Substream To guarantee the timely presentation of a single stream subjet to jitter, it is neessary to buffer arriving media-units at the sink to ompensate the jitter. The buffer is emptied at a onstant rate for displaying the media-units. 7 Jitter is often defined as the variation of network delay. 13

14 Santoso [San93] has already shown that the temporal relationship within one ontinuous media stream an be preserved by delaying the output of the first media-unit max min for d k d k seonds. Based on this theorem, both the playout deadline and the buffer requirements are be derived. The deadline given by Santoso (ase a) an be lowered in some situations (ase b). Theorem 3: Smooth playout for a substream k an be guaranteed in ase of bounded jitter whenever either one of the following two starting onditions holds true. (a) max min d k d k = k seonds elapsed after the arrival of the first mediaunit, or (b) the ( k r + 1 )-th media-unit has arrived. Proof: A proof for (a) an be found in [San93]. Condition (b) improves (a) in some ases, i.e. playout an start earlier without violating timeliness. Suh a situation is shown in figure 4: the first media-unit experienes the maximum delay, subsequent media-units arrive in a burst (marked gray in figure (4)) suh that after the arrival of the ( k r max min + 1 )-th media-unit the elapsed time is less than d k d k. The average delay of media-units is denoted by dotted lines. Assuming that the ( k r + 1 )-th media-unit just has arrived, we start the playout of the buffered media-units immediately. A number of k r + 1 media-units is at least suffiient for a presentation period of ( k r + 1) r 1 + r 1 seonds. In the worst ase, the ( k r + 1 )-th media-unit experienes its minimum delay and the subsequent media-unit its maximum delay. Then the maximum period without any - arrival is given by is k r k = k + r 1 seonds. ( k r + 1) r 1 gives an upper bound for k + r 1. Consequently, the next media-unit arrives just in time. Following media-units will not arrive later beause the last one has already experiened the largest delay. Theorem 4 enables us to alulate the required minimum buffer spae for the synhronization of a single substream. Theorem 4: To guarantee intra-stream synhronization for a single substream by applying theorem 3, a minimum buffer spae of 2 k r media-units is required. For a proof see [Gey95]. Synhronized Playout for Multiple Substreams The basi idea of the synhronization sheme in model 2 is to ahieve inter-stream synhronization between multiple substreams by intra-stream synhronization. One the latter has been established by satisfying theorem 3 and 4 for eah substream, interstream synhronization is attained [Ish95], [San93]. This holds true if eah substream experienes the same jitter. In the following, we onsider and examine the impat on buffer requirements for different jitter bounds for eah substream. This reflets the 14

15 s 0 S k D d k s 1 + ( k r + 1) s 1 + ( k r + 2) max < d k k - d k min Theorem 3(b) r -1 k + Theorem 3(a) ase that the paths from soures to the destination are independent. We assume that media-units experiene an average delay d on all substream onnetions. The following proofs an also arried out with different delays. We present two methods to ompute the buffer requirements for multiple substreams. The first approah estimates the jitter for all substreams with the maximum jitter value. The seond strategy attempts to refine this oarse-grain estimation by shifting the starting times of eah substream in orrelation to their jitter values in order to save buffer spae. (a) Maximum Jitter Strategy Fig. 4. Worst Case Senario for a Single Substream. Obviously, playout an only start if theorem 3 is satisfied for all substreams. Thus, the playout deadline for a stream given by a synhronization group is defined by the latest substream that satisfies theorem 3. The situation is ompliated by different jitter 15

16 bounds for the orresponding substreams whih lead to different playout deadlines and buffer requirements. We must avoid a situation where substreams with large jitter bounds still wait for their deadlines while the buffer of other substreams with small jitter bounds already overflows. To ope with this problem in a straight forward manner, Ishibashi et al. [Ish95] propose to alloate the buffer aording to the substream with M the largest jitter bound. Hene, the buffer requirement b for eah substream of the group and B M k for the omplete group are given as follows. B M = M b k = 2 max n 1 M b k = k = 0 r n 2 max r (15) (16) S 0 s 0 D S 1 s 0 S 0 D S 1 s 0 s 1 s 1 s 1 s 2 s 2 shift s 2 s 3 s 3 s 3 s 4 s 4 s 0 s 4 s 5 s 1 s 5 t d1 s 6 s 2 s 6 t d2 t d1 t d2 (b) Shifting Strategy Fig. 5. Multiple Substreams without and with Shifting. Depending on the differenes in the jitter values for the substreams, the maximum jitter strategy might lead to a buffer waste. A more sophistiated way to handle this problem is to synhronize the different substreams suh that they reah their playout 16

17 deadline on average at the same time. This is done by shifting the starting points of all substreams aording to the deadline of the substream, with the largest jitter bound. Figure 5 depits suh a senario for two soures 8 where d = 3.5, 0 = 2 and 1 = 6. With theorem 4 we get buffer requirements of four media-units for substream 1 and twelve media-units for substream 2. Substream 1 reahes its playout deadline on average at t d1 and substream 2 at t d2. Without shifting a buffer overflow ours when reeiving the 5th media-unit of substream 1 while substream 2 still has to wait two time units until playout an ommene. By shifting, both substreams arrive at the same time. The amount of the forward shift an be easily derived from theorem 3. The k-th substream has to be shifted forward on time axis with the differene of its jitter to the maximum jitter, i.e. max k seonds. Clearly, substream k has to be started max k seonds later than the substream with the highest jitter. When applying that shift one might onlude that no further buffering is needed exept for the buffer given by theorem 4. In fat, there exists a worst ase that requires additional buffer spae for eah substream. The amount of additional buffering is stated in theorem 5. Theorem 5: Applying a shift of k to the k-th substream, k = 0,..., n-1, and having bounded jitter for eah substream, inter-stream synhronization for multiple dependent substreams an be guaranteed if in addition to the buffer requirement of theorem 4, another max+ + ( k ) r buffer slots are alloated. For a proof see [Gey95]. With the above theorems, the total buffer requirements an be omputed as follows. B S = S b k = n 1 S b k k = Start-up Protool Influene = max 2 k max+ + ( + k ) r n 1 k = 0 2 k max+ + ( + k ) r Until now, we have assumed that substreams are synhronized with respet to their average delay. Model 1 is based on the assumption of zero jitter. When we use the sheme in the ase of bounded jitter, annot guarantee the synhronization of the substreams with respet to their average delay sine it is based on the roundtrip delay values experiened by the first n media-units. If this delay orresponds to the average delay, the start-up protool works orretly. However, the observed delay an be altered due to jitter, hene the alulation introdues an error that must be onsidered. The start-up protool omputation is based on a roundtrip delay for a request paket and one or several pakets arrying a media-unit. However, the transmission of the request paket from sink to soure and the sending of the media-unit bak to the lient are subjet to jitter. Sine the request message is a small paket made up of sev- 8 In ontrast to the definitions for model 1, eah one of the depited substreams delivers equal media-unit numbers. (17) (18) 17

18 eral bytes the following onsiderations we will neglet the jitter experiened by the request paket, supposing that the jitter bounds for the buffer alulations stated in theorem 4 and 5 have been hosen suffiiently large. 2.7 Exat Buffer Requirements Model 2 gives us a framework to ompute buffer requirements for multiple substreams with different jitter bounds to attain inter-stream synhronization by maintaining intra-stream synhronization. Buffer requirements are given by theorem 4 and 5. The error introdued by the start-up protool is orreted by theorem 6. Throughout all theorems, we expressed the time we need to buffer in terms of media-units. The required buffer spae an be optimized by summing up the time to buffer given by theorem 4 and 5 and by transforming the resulting sum into buffer slots. We an summarize the overall buffer requirements b k for a substream k and B for a synhronization group onsisting of n substreams as follows. b k 2 k max+ + = + ( k ) + (19) + max m + k max+ { m k m= 0 n 1} r B= n b k k = 1 = n k = 1 2 k max+ + + k max m + k max+ + + { m k m= 0 n 1} r (20) 3 Conlusion We have presented a sheme for intra- and inter-stream synhronization of stored multimedia streams. The only assumption we make is that jitter is bounded, whih is typially true in todays networks. Having presented a base-version of the synhronization sheme, we make enhanements suh as shifting the start-up times in order to redue the buffer requirements. At the end, we derived exat bounds for the buffer requirements. Aknowledgment The work desribed in this paper was supported by the Siemens Nixdorf AG, Munih. 4 Referenes [Aga94] N. Agarwal and S. Son. Synhronization of Distributed Multimedia Data in an Appliation-speifi Manner. In 2nd ACM International Conferene on Multimedia, 18

19 [Ber95] [Eff93] [Es94] [Gey95] [Ish95] [Koe94] [Ran92] [Ran93] [Rot95b] [Rot95] [San93] [Ste93a] pages , San Franiso, USA, Otober C. Bernhardt and E. Biersak. The Server Array: A Novel Arhiteture for a Salable Video Server. In Proeedings of the Distributed Multimedia Conferene, pages 63 72, Stanford, USA, August W. Effelsberg, T. Meyer, and R. Steinmetz. A Taxonomy on Multimedia-Synhronization. In Proeedings of the Fourth Workshop on Future Trends of Distributed Computing Systems, Lisbon, Portugal, Sep. 1993, pages Eyrolles, J. Esobar, C. Patridge, and D. Deutsh. Flow Synhronization Protool. In ACM Transations on Networking, volume 2, pages IEEE, April W. Geyer. Stream Synhronisation in a Salable Video Server Array. Master s thesis, Institut Eureom, Sophia Antipolis, Frane, September Y. Ishibashi and S. Tasaka. A Synhronization Mehanism for Continuous Media in Multimedia Communiations. In IEEE Infoom 95, volume 3, pages , Boston, Massahusetts, April D. Koehler and H. Mueller. Multimedia Playout Synhronization Using Buffer Level Control. In 2nd International Workshop on Advaned Teleservies and High- Speed Communiation Arhitetures, pages , Heidelberg, Germany, September P. V. Rangan, H. M. Vin, and S. Ramanathan. Designing an On-Demand Multimedia Servie. IEEE Communiations Magazine, 30(7):56 65, July P. Rangan, S. Ramanathan, H. M. Vin, and T. Kaeppner. Tehniques for Multimedia Synhronization in Network File Systems. Computer Communiations, 16(3): , Marh K. Rothermel and T. Helbig. An Adaptive Stream Synhronization Protool. In 5th International Workshop on Network and Operating System Support for Digital Audio and Video, Durham, New Hampshire, USA, April K. Rothermel, T. Helbig, and S. Noureddine. Ativation Set: An Abstration for Aessing Periodi Data Streams. In Multimedia Computing and Networking, volume 2417, San Jose, California, February IS&T/SPIE. H. Santoso, L. Dairaine, S. Fdida, and E. Horlait. Preserving Temporal Signature: A Way to Convey Time Constrained Flows. In IEEE Globeom, pages , Deember R. Steinmetz. Multimedia-Tehnologie. Springer Verlag, Heidelberg, Germany,