IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. XX, NO. XX, MMMMM 20YY 1

Transcription

1 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. XX, NO. XX, MMMMM 2YY 1 Time Varying Dynamic Bayesian Nework for Non-Saionary Evens Modeling and Online Inference Zhaowen Wang*, Ercan E. Kuruoğlu, Senior Member, IEEE, Xiaokang Yang, Senior Member, IEEE, Yi Xu, and Thomas S. Huang, Life Fellow, IEEE Absrac This paper presens a novel Time Varying Dynamic Bayesian Nework (TVDBN) model for he analysis of nonsaionary sequences which are of ineres in many fields. The changing nework srucure and parameer in TVDBN are reaed as random processes whose values a each ime epoch deermine a saionary DBN model; his DBN model is hen used o specify he disribuion of daa sequence a he ime epoch. Under such a hierarchical formulaion, he changing sae of nework can be incorporaed ino he Bayesian framework sraighforwardly. The nework sae is assumed o ransi smoohly in he join space of numerical parameer and graphical opology so ha we can achieve robus online nework learning even wihou abundan observaions. Paricle filering is employed o dynamically updae curren nework sae as well as infer hidden daa values. We implemen our ime varying model for daa sequences of mulinomial and Gaussian disribuions, while he general model framework can be used for any oher disribuion. Simulaions on synheic daa and evaluaions on video sequences boh demonsrae ha he proposed TVDBN is effecive in modeling non-saionary sequences. Comprehensive comparisons have been made agains exising non-saionary models, and our proposed model is shown o be he op performer. Index Terms Bayesian neworks, ime varying, paricle filers, even recogniion. I. INTRODUCTION MODELING he evoluion of emporal sequences is of grea ineres in many areas such as signal processing, auomaion, finance, compuaional biology, ec. Among he numerous ools designed for he analysis of emporal sequences, Dynamic Bayesian Neworks (DBNs) [1] have been he mos successful ones. A DBN is he exension of a Bayesian Nework (BN) o emporal domain, in which condiional dependencies are modeled beween random variables boh wihin and across ime slos. The condiional disribuions are assumed o be homogeneous in DBN; ha is, he srucure and parameer of DBN are fixed hroughou he ime. Under his assumpion, a DBN is effecively consruced by unrolling Copyrigh (c) 21 IEEE. Personal use of his maerial is permied. However, permission o use his maerial for any oher purposes mus be obained from he IEEE by sending a reques o pubs-permissions@ieee.org. Z. Wang and T. S. Huang are wih he Deparmen of Elecrical and Compuer Engineering, Universiy of Illinois a Urbana-Champaign, Urbana, IL, 6181 USA ( wang38@illinois.edu; -huang1@illinois.edu). E. E. Kuruoğlu is wih he Isiuo di Scienza e Tecnologie dell Informazione (ISTI), CNR, Pisa, Ialy ( ercan.kuruoglu@isi.cnr.i). X. Yang and Y. Xu are wih he Insiue of Image Communicaion and Informaion Processing, Shanghai Jiao Tong Universiy, Shanghai, 224 China ( xkyang@sju.edu.cn; xuyi@sju.edu.cn). a BN in ime axis, and he model learning procedure can be grealy simplified. However, his bold assumpion limis he power of DBN in modeling many non-saionary sequences, where he inrinsic relaionships among variables change from ime o ime. Such non-saionary sequences may arise in all aspecs of our life. Some examples include: he seering paern of a vehicle under differen road condiions; he appearance of an objec across muliple cameras; he gene ineracions in differen sages of a life circle; and he sock prices in differen economic periods. A fixed saisical model is obviously inadequae o model hese daa sequences a all ime insances. Incorporaing he emporal variaion of nework srucure and parameer ino DBN is a naural way o handle nonsaionary sequences. However, learning and inferring such a ime varying nework is a non-rivial job. The naive approach o learn a DBN independenly for each ime insance is no feasible since in mos applicaions oo few observaions can be obained from jus one ime insance. One way o ameliorae his daa scarciy problem is o pre-segmen emporal sequences ino saionary epochs, in each of which daa are generaed from he same probabiliy disribuion. Bu he segmenaion iself is hard due o he lack of knowledge for models in each epoch and he large soluion space growing exponenially wih he lengh of sequence. On he oher hand, our observaion daa are ofen corruped wih noise, and here will ineviably be some perurbaions associaed wih he saisics derived from hem. The noise perurbaion can be indisinguishable from he rue variaion of underlying daa disribuion, paricularly when he disribuion is changing gradually. Furhermore, real ime processing is a crucial consrain in some applicaions such as sock price predicion and video surveillance. The demand for adaping DBN model o new daa online poses even greaer challenges. A. Previous Work Faced wih all he difficulies menioned above, researchers have been rying o exend DBN o non-saionary scenarios by imposing various condiions on he form of nework and he way i can change. Earlier works have been mainly focused on non-saionary models wih fixed srucure. Among hem he mos sudied one is he ime varying auoregression (TVAR) model [2], which describes non-saionary linear dynamic sysems wih coninuously changing linear coefficiens and

2 2 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. XX, NO. XX, MMMMM 2YY noise variances. Normalized leas square algorihm can be used o esimae he regression parameers recursively, and he esimaion error is shown bounded when he parameers change smoohly [3]. Due o he well-esablished heory, TVAR model has found wide applicaion in he researches of equiy marke [4], gene expression [5], and elecroencephalogram (EEG) races [6]. Exensions on TVAR have also been made for oher ime varying processes such as non-gaussian auoregression [7] and Poisson couning process [8]. Anoher class of non-saionary model ha has been exensively sudied is he swiching linear dynamic sysem (SLDS), in which a laen Markov chain is employed o describe he piecewise change of linear sysem. Time varying observaion marices are formulaed in [9]; and ime varying dynamic ransiion marices are formulaed in [1], [11]. An SLDS wih arbirary sae duraion disribuion is proposed in [12]. An SLDS wih an unknown number of saionary modes is proposed in [13] by using hierarchical Dirichle process prior. There exis a bunch of learning algorihms for SLDS; besides he commonly used EM mehod, variaional approximaion [9], Grassmann manifold [14], and auxiliary paricle filer [15] have also been applied for efficien and/or online learning. The models of TVAR and SLDS only consider nonsaionary parameer change in dynamic sysems. Recenly, graphical models ha change dynamically in boh parameer and srucure have received more aenion. Wih he assumpion ha daa sequence is piecewise saionary in ime, nonsaionary models are consruced as a cascade of saionary models each of which is learn from a pre-segmened saionary sub-inervals. A number of mehods have been proposed o find hese sub-inervals. In [16], Gaussian graphical model is used o represen he dependency among variables, and hen he exac poserior disribuion of swiching imes beween saionary inervals is evaluaed wih he mehod of [17]. Curve manifold is employed o express ime varying sequences in [18], and segmenaion is carried ou according o he geomeric srucure of manifold. Markov chain Mone Carlo (MCMC) sampling mehods are used in [19], [2], which search MAP saionary sub-inervals hrough ieraive local movemens on nework configuraion. When he dimension of model parameer is no fixed, reversible jump MCMC can be uilized [21]. Change deecion echnique is used o monior he finess beween incoming daa and curren nework [22], and local adapaion is applied o nework when large discrepancy is deeced. In [23], nework changes are represened by a hidden conroller, whose opimal value is searched wih a random hill climbing algorihm. Since he piecewise saionary models are sill no general enough for any applicaions, several more sophisicaed nonsaionary models have been developed wih coninuously changing nework parameers aken ino accoun. In [24], he srucure of a binary nework is reaed as a hidden sae, and is dynamic ransiion follows an exponenial random graph process. Consrains of emporal smoohness and srucural sparsiy are imposed on ime varying linear regression neworks [25] [27], in which linear coefficiens are joinly opimized by minimizing lasso objecive funcions. In [28], he auhors propose o model he dynamics of mixed membership vecor using a logisic normal disribuion whose hyper parameers can evolve over ime according o linear Gaussian model. Unforunaely, all hese algorihms can only be used for off-line nework learning, bu canno adap o new daa sequence on he fly. B. Proposed Mehod In his paper, we propose a novel Time Varying Dynamic Bayesian Nework (TVDBN) model for online inference of he underlying disribuion of non-saionary sequences. We exend he basic DBN model so ha boh he srucure and parameer of nework become random variables ha can change hrough ime. These random variables are reaed as addiional hidden nodes in our graph model, and a smooh ransiion prior is imposed on heir emporal variaion o ease he problem of daa scarciy. This novel represenaion of changing nework allows a unified modeling of boh daa and nework iself under he same dynamic Bayesian framework. In conrary o mos off-line learning mehods repored in lieraures, we employ paricle filer o dynamically infer he hidden saes of nework as well as missing daa, if here are any. This key feaure enables he applicaion of our model in siuaions wih real ime consrain. The framework of our ime varying model is general enough for daa sequence of any disribuion ype; in his work, mulinomial and Gaussian disribuions are sudied in paricular. The effeciveness of he proposed TVDBN model is validaed on boh simulaed non-saionary daa and video sequences, wih preliminary applicaions o racking and even recogniion. I is worh noing he differences beween our model and hose closely relaed o i. A hidden variable is used o represen he change of nework in [23], bu i only serves as an auxiliary variable o faciliae implemenaion. In our model, he srucure and parameer nodes are indispensable componens of he whole graph model, and hey represen he saisical aribues of curren nework. The online adapaion mehods in [15], [22] can only model piecewise consan variaion of nework; while our mehod deals wih boh coninuous change in parameer and discree swich in srucure. Smooh change of nework is ensured in [25] wih a kernel window applied on daa sequences; we achieve similar goal via a smooh ransiion model, which is more favorable from a Bayesian perspecive. In he Gaussian graphical model [16], nework parameer is marginalized ou and nework srucure is he only hing o invesigae. Under our seing, he saes of boh srucure and parameer are inferred o give a full descripion of curren nework. The remainder of his aricle is organized as follows. Firs, we inroduce he overall framework of TVDBN model in Secion II. The ransiion disribuion of nework is deailed in Secion III, in he conex of wo popular daa disribuions: mulinomial and Gaussian. Secion IV shows he procedure of inferring hidden variables in TVDBN using paricle filer algorihm. A se of experimenal resuls are repored in Secion V. Finally, in Secion VI, we conclude he paper and discuss he poenial exension o large scale neworks.

3 WANG e al.: TIME VARYING DYNAMIC BAYESIAN NETWORK FOR NON-STATIONARY EVENTS MODELING AND ONLINE INFERENCE 3 II. TIME VARYING DYNAMIC BAYESIAN NETWORK A. Dynamic Bayesian Nework A Bayesian Nework (BN) [29] is a graph model describing he saisical relaionships among a group of n random variables X = {X i } i=1...n. A BN is deermined by is graph srucure G and disribuion parameer Θ. G is a direced acyclic graph wih n nodes corresponding o random variables X. An edge in G direced from node i o j encodes X j s condiional dependence on X i, and X i is called a paren of X j. A variable X i is independen of is non-descendans given all is parens Pa(X i ) in G. Therefore, he join probabiliy disribuion over X can be decomposed by he chain rule: n p(x) = p(x i Pa(X i )) (1) i=1 The parameer se Θ = {Θ i } i=1...n specifies he parameers of each condiional disribuion p(x i Pa(X i )) in Eq. (1). The meaning of Θ i is inerpreed according o he specific form of he disribuion. When he disribuion is mulinomial, Θ i is simply a condiional probabiliy able; when he disribuion is Gaussian, Θ i may conain he values of mean and variance. A Dynamic Bayesian Nework (DBN) [1], [3] is he exension of a BN o model emporal processes. In DBN, a se of random processes are represened by he X = {X i } i=1...n, and X i [] is he random variable of process X i a discree ime. The nework srucure G now defines he dependency among variables over a period of ime as well as hose wihin he same ime epoch. Usually, he processes X are assumed o be Markovian and causal. So a node in graph G is only allowed o be linked from he oher nodes in he same or previous epoch, i.e., Pa(X i []) {X[ 1], X[]}. Also, all he condiional disribuions are assumed o be saionary. Thus we have: p(x[ + 1] X[ : ]) = p(x[ + 1] X[]), =, 1, 2... (2) The model parameer Θ is defined similarly as in BN. Θ and G are kep consan over ime under he saionariy assumpion. B. Time Varying Nework Represenaion The saionariy assumpion has been favored for is simpliciy in mos applicaions of DBN so far. However, his assumpion is no always valid in real life. For example, a driver will change his driving syle according o differen raffic condiions, and a consan velociy dynamic model is apparenly incapable of describing he movemen of vehicle. Such ime varying characerisic is crucial o a beer undersanding of emporal evens. To his end, we inroduce a Time Varying Dynamic Bayesian Nework (TVDBN) model whose nework srucure and parameer can vary as he underlying disribuion of emporal sequence changes. In our ime varying formulaion, he nework srucure and parameer are modeled as random processes whose values a ime are denoed as G[] and Θ[], respecively. These random variables, regarded as srucure nodes and parameer nodes in our model, are used o consruc a graph ogeher wih daa nodes X[]. Since here is no way o observe he srucure nodes and he parameer nodes direcly, hese nodes are hidden in naure. We can only infer hem from he observaions available on daa nodes. A each ime epoch, we have n parameer nodes: Θ[] = {Θ i []} i=1...n. Each Θ i [] is linked o node X i [] and deermines is condiional disribuion joinly wih oher daa nodes linked o i. Therefore, in TVDBN, he parens of a daa node X i [] are defined as: Pa(X i []) { Pa(X i []), Θ i []} (3) where Pa(X), called X s non-parameer parens, is he se of daa nodes ha are linked o X. The linkage beween daa nodes, or equivalenly heir probabilisic dependency, is deermined by he nework srucure G[]. Here G[] is represened as a se of direced edges: G[] { e ji, e ji i, j = 1...n } (4) where e ji sands for an edge poining from X j [] o X i [], and e ji sands for an edge poining from X j[ 1] o X i []. We only consider he nodes a ime 1 and as X i [] s poenial parens, as required by he Markovian propery 1. To ensure G[] is an acyclic graph, i is also required ha here is no sequence i 1,..., i k such ha e ij,i j+1 G[] for 1 j < k and e ik,i 1 G[]. Given G[], we can rerieve X i [] s nonparameer parens as: Pa(X i []) = {X j [] e ji G[]} {X j [ 1] e ji G[]} (5) I should be noed ha he dependency of X i [] on Θ i [] is fixed and will no be eliminaed over ime. To model he changing processes of Θ[] and G[] in TVDBN, we adop a firs order Markov model p(θ[ + 1], G[ + 1] Θ[], G[]) as heir join ransiion disribuion. Assume ha given srucure G[], each parameer Θ i [] ransis independenly, and we have p(θ[ + 1], G[ + 1] Θ[], G[]) = p(g[ + 1] G[])p(Θ[ + 1] Θ[], G[ + 1]) n = p(g[ + 1] G[]) p(θ i [ + 1] Θ i [], G[ + 1]) (6) i=1 The dependency of Θ[+1] on G[+1] and G[] is due o he fac ha he dimension of disribuion parameer is affeced by nework srucure. An example of TVDBN wih changing parameer and srucure is illusraed graphically in Fig. 1. All he black edges represen he probabilisic dependency among nodes, which play he same role as he edges in a BN or DBN. While he green and blue edges in Fig. 1 have essenially differen meanings. A green edge links a srucure node G[] o he slice of nework enclosed in a corresponding green recangle, which conains all he edges conneced owards X[] from oher daa nodes of ime 1 and. The presence and absence of any of hese edges, or equivalenly all he Pa(X i []) s, are 1 The Markovian resricion is imposed here for convenience in discussion. In more general siuaion, any node a ime earlier han can be paren of X j [].

4 4 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. XX, NO. XX, MMMMM 2YY X 1[1] X 1[2] X 1[3] X 1[4] X 2[1] X 2[2] X 2[3] X 2[4] Θ 1[1] Θ 1[2] Θ 1[3] Θ 1[4] Θ 2[1] Θ 2[2] Θ 2[3] Θ 2[4] G[1] G[2] G[3] G[4] Fig. 1. Graphical represenaion of a TVDBN over ime = 1, 2, 3, 4. A each ime epoch, here are wo daa nodes (above he dash line), wo parameer nodes, and one srucure node (below he dash line). The black edges indicae he probabilisic dependency among nodes. The green edges indicae he nework opologies inside he green recangles are deermined by he srucure nodes. The blue edges indicae he dimensions of parameer nodes are deermined by he srucure nodes. deermined by he saus of node G[]. A blue edge links a srucure node G[] o a parameer node Θ i [] of he same ime epoch, and indicaes ha he dimension of parameer Θ i [] is deermined by he saus of G[]. The dependency encoded in a green or blue edge is deerminisic, as opposed o he probabilisic dependency encoded in a black edge. Therefore, our TVDBN model is differen from radiional DBN model, and a new mehod should be designed for is online inference. Before addressing ha in Secion IV, we will firs discuss he ransiion models for he srucure and parameer in ime varying nework. III. NETWORK TRANSITION DISTRIBUTION The ransiion disribuion in Eq. (6) plays a crucial role in our ime varying model. The firs erm on he righ side of Eq. (6), p(g[ + 1] G[]), characerizes he dynamic ransiion of nework srucure. Dynamic graph processes have been sudied exensively in random graph heory. Some rules have been formulaed o model he ypical evoluion of graphs over ime, such as preferenial aachmen [31], copying [32], and preferenial deleion [33]. However, mos dynamic graph processes are devoed o modeling social and web-like neworks, which usually feaure differen behaviors han Bayesian neworks. Therefore, he specialized rules for random graphs are no applicable o our TVDBN model. Transiion kernels for Bayesian nework are proposed in [34] ha make local modificaion o nework srucure in an ieraive manner. Such ransiion kernel is sufficien for off-line learning of a saionary nework, bu is no flexible enough o model dynamically changing nework. Here we propose o model he sequence of nework srucure {G[]} as a Markov chain, whose sae space G = {G i } consiss of all valid direced acyclic graphs ha connec daa nodes X[] across adjacen ime epochs or wihin he same ime epoch. When he number of daa nodes n is fixed and moderaely large, using Markov chain model is a reasonable approach because he finie se G can be enumeraed. Given he curren nework srucure G i, he probabiliy of ransiing o srucure G j in nex epoch is se o be: p(g[+1] = G j G[] = G i ) exp ( λ 1 G j λ 2 G j G i ) (7) where G j denoes he number of edges in graph G j, and G j G i denoes he number of edges ha have been changed (added or deleed) from G i o G j. λ 1 and λ 2 are parameers conrolling he relaive imporance of he wo erms. The ransiion model of G[] is designed in such a way o favor sparse nework srucure and smooh nework ransiion. Sparseness requires he ime varying model o adap o daa disribuion wih a nework srucure as simple as possible. In his way, he problem of model over-fiing can be avoided. This idea is inspired by Bayesian Informaion Crierion (BIC) score [35], which penalizes complex models according o he number of model parameers. The smoohness resricion is inroduced o deal wih daa scarciy. Typically, he daa observed in one ime epoch alone are far from enough for us o learn he unknown nework srucure. A good prior knowledge of how he nework changes will enable us o esimae is srucure using boh curren and previous daa observaion. Therefore, we impose he resricion ha he nework can only change smoohly over ime; i.e., he oal number of edges being changed from one ime epoch o he nex is expeced o be small. This smoohness assumpion is valid for many daa sequences in real life. For example, in video applicaions we have a much higher signal sampling rae (frame rae) han even occurring rae (human behavior), so he underlying daa disribuion can be hough as changing slowly and smoohly. Fig. 2 illusraes he srucure ransiion model of a TVDBN wih wo daa nodes. To save space, we have only ploed par of all possible nework opologies. The arrows indicae he mos likely ransiions beween differen srucures, which are realized by adding or removing a single edge in he nework. The second erm on he righ side of Eq. (6) is he produc of ransiion probabiliies of all nework parameers. Under he same smoohness assumpion as above, we require ha he parameer ransiion disribuion p(θ i [ + 1] Θ i [], G[ + 1]) should have a large mass in he neighborhood around Θ i []. In his way, we can learn he parameers of nework robusly even in he absence of abundan observaion daa. On he oher hand, he ransiion of Θ i [] also depends on how i parameerizes he disribuion of daa node X i []. We should cusomize he form of p(θ i [ + 1] Θ i [], G[ + 1]) for differen daa disribuions. In he following, we shall discuss his poin in he conex of wo paricular cases where he condiional disribuion of X i [] is mulinomial/gaussian. Daa disribuions of oher ypes can also be incorporaed in he framework of our TVDBN model as long as suiable parameer ransiion models are devised. A. Time Varying Mulinomial DBN When he variables X i [] s are discree, he simples way o represen he condiional dependency among hem is using mulinomial disribuion. The parameer Θ i [] of mulinomial disribuion is a condiional probabiliy able conaining a collecion of probabiliy vecors {Θ ij []}, where Θ ij [] corresponds o he j h configuraion of X i [] s non-parameer

5 WANG e al.: TIME VARYING DYNAMIC BAYESIAN NETWORK FOR NON-STATIONARY EVENTS MODELING AND ONLINE INFERENCE 5 G=4 G=6 G=9 X 1[-1] X 1[] X 2[-1] X 2[] G=21 G=14 G=15 G=17 Fig. 2. Some possible srucures of a TVDBN wih wo daa nodes. Indices are shown for each srucure. The mos likely ransiion pahs beween hem are indicaed by grey arrows. parens Pa j (X i []). Vecor Θ ij [] conains he probabiliies for all possible values of X i [] given he paren: p(x k i [] Pa j (X i []), Θ i []) = θ ijk [] (8) where X k i [] is he kh possible value of X i[], and θ ijk [] is he kh elemen in Θ ij []. The probabiliy vecors are assumed o propagae independenly, so ha he ransiion disribuion of Θ i [] can be decomposed as: p(θ i [ + 1] Θ i [], G[ + 1]) = n i [+1] j=1 p(θ ij [ + 1] Θ i [], G[ + 1]) (9) where n i [ + 1] is he number of possible configuraions of Pa(X i [ + 1]) given he curren nework srucure G[ + 1]. The emporal evoluion of each probabiliy vecor is furher modeled by a hierarchical Dirichle disribuion [36]: p(θ ij [ + 1] Θ i [], G[ + 1]) Dir(Θ ij [ + 1]; α Θ ij ) k θ ijk [ + 1] α θ ijk 1 (1) where Dir(; ) denoes he Dirichle disribuion, Θ ij is he disribuion cener o be derived from Θ i [], and α is a smoohing coefficien. The Dirichle disribuion is well suied for modeling he densiy of probabiliy vecors, since is pdf is suppored on he probabiliy simplex {Θ ij [ + 1] θ ijk [ + 1], k θ ijk[+1] = 1}. Moreover, he hierarchical srucure can resric he variaion of Θ ij [+1] wihin he space around Θ ij, a mechanism keeping he nework parameer change smoohly over ime. The smoohness can be easily adjused via he coefficien α. When he new nework srucure G[ + 1] does no change from is previous value G[] or more precisely, X i [ + 1] shares he same se of paren nodes wih X i [] (wih a shif in ) he Θ ij in Eq. (1) is simply chosen o be he previous probabiliy vecor ha corresponds o he same paren configuraion: Θ ij = Θ ij []. However, if G[ + 1] changes from G[] wih some edges being added and/or deleed, furher elaboraion is required in he design of Θ ij. Consider he node X i [] and is non-parameer parens Pa(X i []). Suppose he nework srucure G[ + 1] changes from G[] such ha Pa(X i [ + 1]) becomes differen from Pa(X i []) wih an addiion of nodes Y and a deleion of nodes Z. More formally, le us denoe Pa(X i []) = Pa(X i [, + 1]) Z[] (11) Pa(X i [ + 1]) = Pa(X i [ + 1, ]) Y[ + 1] (12) where Pa(Xi [, + 1]) and Pa(X i [ + 1, ]) are he nonparameer parens shared by X i [] and X i [ + 1] (wih a shif in ime) a and + 1, respecively. Under he assumpion of smooh model changing, he condiional disribuion of X i [ + 1] can be approximaed as follows: p(x i [ + 1] Pa(X i [ + 1])) p(x i [ + 1] Pa(X i [ + 1, ])) p(x i [] Pa(X i [, + 1])) p(x i [] Pa(X i [, + 1]), Z[])p(Z[] Pa(X i [, + 1])) Z[] Z[] p(x i [] Pa(X i []))p(z[]) (13) We have assumed weak dependency of X i [ + 1] on he newly added parens Y[ + 1]; and he condiional probabiliy p(z[] Pa(X i [, + 1])) is subsiued by p(z[]) for evaluaion convenience. The error of hese approximaions can be oleraed if an appropriae value is chosen for α in Eq. (1). Thus, a good choice of Θ ij can be: Θ ij = p(z k [])Θ il(j,k) [], k l(j,k) s.. Pa (Xi []) = [ Pa j (X i [ + 1, ]), Z k []] (14) j where Pa (Xi [ + 1, ]) is equal o he sub-vecor of Pa j (X i [ + 1]) wih he elemens of Y[ + 1] runcaed. B. Time Varying Gaussian DBN When daa nodes X[] represen coninuous variables, linear Gaussian sysem is ofen used o describe heir join dynamic ransiion: X[] = F[]X[ 1] + v[] (15) where F[] is a n n marix wih elemens {f ij []}, and v[] is a Gaussian noise sampled from N (v[];, Σ[]). Since X[] are join Gaussian, hey can be equivalenly represened by a

6 6 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. XX, NO. XX, MMMMM 2YY Gaussian belief nework [37], and he condiional disribuion of a single node X i [] can be expressed as: p(x i [] Pa(X i []), Θ i []) = N (X i []; m i []+ X j[] Pa(X i[]) b ij [](X j [] m j []), v i []) (16) where m i [] = j f ij[]x j [ 1] is he uncondiional mean of X i []; v i [] is he condiional variance of X i [] given he value of paren nodes Pa(Xi []). Pa(Xi []) are specified by nework srucure G[] for all i s; and b ij [] is a linear coefficien measuring he srengh of dependency of X i [] on X j [] Pa(X i []). The parameer se Θ i [] for node X i [] in his ime varying Gaussian DBN is defined as: Θ i [] = {{f ij []}, v i [], {b ij []}} (17) Obviously, we have v i [] > for all i, and b ij [] = for X j [] / Pa(X i []). As will be seen below in he definiion of parameer ransiion disribuion, i is convenien o adop he form of parameers as in Eq. (17). The ransformaion from {{v i []}, {b ij []}} o Σ[] can be accomplished using he procedure described in [38]. We assume ha all he parameers in Θ i [] propagae independenly: p(θ i [ + 1] Θ i [], G[ + 1]) = p(v i [ + 1] v i []) p(f ij [ + 1] f ij [], G[ + 1]) j p(b ij [ + 1] b ij [], G[ + 1]) (18) Also, Eq. (16) can be inerpreed as a linear regression of X i [] wih coefficiens f ij [], b ij [], and variance v i []. The ime varying auo-regression model [2] offers a convenien way o model he ransiion of hese ime varying parameers. According o [2], he variaion of linear coefficiens can be modeled by random walk processes: p(f ij [ + 1] f ij [], G[ + 1]) = N (f ij [ + 1]; f ij [], σ 2 f ) (19) p(b ij [ + 1] b ij [], G[ + 1]) = N (b ij [ + 1]; b ij [], σ 2 b ), if e ji G[ + 1] (2) where σ f and σ b are sandard deviaions. The coefficiens are se o zero when he corresponding edges are missing in G[+ 1]. To model he change of non-negaive variance v i [], we employ a muliplicaive random walk process as in [2]: v i [] p(v i [ + 1] v i []) = Bea( d; a, b) (21) v i [ + 1] where a and b are he parameers of Bea disribuion, and d is a discoun facor. IV. SEQUENTIAL MONTE CARLO INFERENCE OF TVDBN Afer consrucing he ime varying dynamic Bayesian nework model and is ransiion disribuion, we are ready o do on-line inference of he unknown variables in nework. The unknown variables of our ineres a each ime include unobserved daa 2, nework srucure, and nework parameer. We combine hem as a hidden sae s : s = [X[], G[], Θ[]] (22) The dynamic ransiion disribuion of sae s can be decomposed ino he produc of he ransiion disribuions defined in Secion III: p(s s 1 ) = p(g[] G[ 1])p(Θ[] Θ[ 1], G[]) p(x[] X[ 1], Θ[], G[]) (23) A each ime epoch, we acquire an observaion o, which measures par or all of he daa nodes. The relaionship beween observaion and sae is governed by an observaion disribuion p(o s ). Given he iniial sae s, we are going o recursively esimae he curren sae poserior p(s o 1: ) using he observaions up o now. The dynamic evoluion of s is very complicaed, because s may conain boh discree and coninuous variables, and even is dimensionaliy is no fixed (due o he varying dimension of Θ[]). Therefore, i is generally impossible o ge a closed-form soluion for he poserior disribuion of s. We can only resor o some numerical mehod such as paricle filering o find an approximaed soluion. Paricle filering (or sequenial Mone Carlo) [39], [4] is employed here o esimae he sae poserior because of is capabiliy o handle arbirary sysem and observaion models. In a paricle filer, he poserior disribuion is approximaed by a finie se of N s sae samples (paricles) {s i } and he associaed weighs {w i }: N s p(s o 1: ) wδ(s i s i ) (24) i=1 When N s approaches infiniy, he approximaion can be arbirarily close o he rue disribuion. A each ime epoch, we sar he filering wih he sample se {s i 1, w 1} i i=1...ns of previous ime. New samples are firs drawn from a proposal disribuion: s i q(s s i 1, o ) (25) The proposal disribuion q( ) has is suppor over he whole sae space of s. In his way, differen nework srucures and nework parameers of differen dimensions can be explored wihin he same framework as hidden daa are inferred. This rans-dimensional proposal densiy is similar in spiri o he ransiion kernel of RJMCMC [34]; he difference is ha we have used he idea o learn disribuion in a dynamic sysem. Afer obaining he new samples {s i }, we measure he likelihood of each of hem wih observaion model, and hen updae heir weighs according o w i w 1 i p(o s i )p(s i s i 1) q(s i s i 1, o ) (26) Finally, a resample sep will be aken if he variance of paricle weighs is oo large [4]. The resulan sample se 2 here we denoe unobserved daa using X[] abusively, even if some daa nodes are observable.

7 WANG e al.: TIME VARYING DYNAMIC BAYESIAN NETWORK FOR NON-STATIONARY EVENTS MODELING AND ONLINE INFERENCE 7 TABLE I NETWORK STRUCTURE AND PARAMETERS OF THE SYNTHETIC TIME VARYING DBN. G[] Θ 1 [] Θ 2 [] [1, 5] 6 [51, 1] 4 [11, 15] 15 ( ( ( ) ) ) ( ) {s i, w i } i=1...ns can give poserior esimaions of curren daa and nework. Choosing a good proposal densiy q(s s 1, o ) is criical o he performance of paricle filer. Sae ransiion disribuion p(s s 1 ) is a popular choice for proposal densiy, and i reduces he weigh updaing procedure in Eq. (26) ino simple accumulaion of likelihoods. We will use his disribuion as proposal densiy in mos of our experimens. However, when he dimension of he sae space is high, mos of he samples drawn from he ransiion disribuion will have very small weighs a problem known as degeneracy. In such a siuaion, we should inegrae he knowledge of o ino he proposal densiy in order o increase sampling efficiency. An example of how o do his for mulinomial daa disribuion will be shown in Secion V-A. V. EXPERIMENTAL RESULTS In his secion, several experimens are conduced o validae he effeciveness of he proposed TVDBN model on mulinomial and Gaussian daa disribuions. Our es daa come from boh simulaion and video sequences. In all he experimens we have achieved online model adapaion, which is a key advanage over mos exising algorihms. To ensure fair comparison, we apply all he off-line models in our experimens ( [16], [23], [25]) in a pseudo online manner; i.e., hey are re-esimaed a each epoch from scrach using he observaions up o curren ime. A. Simulaion We firs consider a wo-node TVDBN model as shown in Fig. 2. Boh nodes are discree and observable, and heir condiional disribuions are mulinomial. X 1 [] has 2 saes; X 2 [] has 3 saes. Parameers Θ 1 [] and Θ 2 [] are defined as he condiional probabiliy ables for he wo nodes, which are represened by marices wih 2 and 3 columns respecively. The nework srucure G[] is indexed by a posiive ineger. There are a oal of 21 srucural variaions, some of which are ploed in Fig. 2. Our es daa are generaed by a synheic ime varying DBN over 15 ime epochs. The nework shows hree differen srucures during he whole period, and he corresponding parameers eiher change linearly or keep piecewise consan. A full descripion of his ime varying nework is given in Table I. We draw 3 sample sequences of X[] from he nework and use hem as observaions. We use 5 paricles o esimae he poserior disribuion of nework srucure and parameers. The iniial sae of nework is assumed o be known. Since all he daa nodes are observable, he hidden sae in his experimen is jus s = [G[], Θ[]]. The dimension of mulinomial parameer Θ i [] is usually high, so we ry o improve he sampling efficiency of paricle filer wih a sub-opimal proposal densiy: q(s s 1, o ) = p(g[] G[ 1])q(Θ[] o, Θ[ 1], G[]) n n i[] = p(g[] G[ 1]) q(θ ij [] o, Θ i [ 1], G[]) (27) where i=1 j=1 q(θ ij [] o, Θ i [ 1], G[]) Dir ( Θ ij []; α Θ ij + ss ij (o ) ) (28) The only difference beween his proposal densiy and he sae ransiion disribuion is he inroducion of sufficien saisics ss ij (o ), which couns he insances of X i [] Pa j (X i []) observed from o. Wih his addiional erm, he proposal densiy becomes closer o is opimal form p(s s 1, o ) [39], and a he same ime remains simple o manipulae. The MAP expecaion of paricle filering resul a each ime sep is calculaed as he esimaed nework sae Ĝ[] and ˆΘ[]. 1) Coninuous Non-saionary Change: We firs examine he performance of TVDBN in esimaing parameer Θ 1 [], which changes smoohly over ime. I is shown in Fig. 3 (a) ha he esimaed ˆΘ 1 [] follows closely o ground ruh values. To ge a principled measuremen on how well he esimaed nework parameer maches he rue underlying condiional disribuion of node X 1 [], Kullback-Leibler (KL) divergence beween hem is evaluaed a each ime insance and ploed in Fig. 4 (a). The resulan KL divergence of our mehod is compared wih hose of several oher algorihms including he fully conneced dynamic Bayesian nework (DBN) [1], he kernel re-weighed TVDBN (K-TVDBN) [25], he adapive

8 8 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. XX, NO. XX, MMMMM 2YY θ211[] θ212[] θ213[] ˆθ211[] ˆθ212[] ˆθ213[] θ111[] θ121[] ˆθ111[] ˆθ121[] (a) (b) Fig. 3. Par of he esimaed nework parameers and ground ruh: (a) Θ 11 [] and Θ 12 []; (b) Θ 21 [] PF TVDBN DBN K TVDBN abn SLDS HCHC 1 1 PF TVDBN DBN K TVDBN abn SLDS HCHC (a) (b) Fig. 4. KL Divergences beween ground ruh and node disribuions esimaed wih differen mehods: PF-TVDBN (our mehod); DBN [1]; K-TVDBN [25]; abn [22]; SLDS [11]; HCHC [23]. (a) divergence of p(x 1 [] ); (b) divergence of p(x 2 [] ) Bayesian nework (abn) [22], he swiching linear dynamic sysem (SLDS) [11], and he hidden conroller hill climb algorihm (HCHC) [23]. A separae daa se of 2 sequences is used o rain he las wo models before online inference. I is seen from Fig. 4 (a) ha our mehod achieves he lowes divergence from rue disribuion mos of he ime expec for he ime period [51, 1]. The HCHC and abn mehods perform slighly beer during ha period because heir piecewise saionariy assumpion is consisen wih he unchanging nework sae a ha ime. Alhough he TVDBN model is more suscepible o observaion noise in such siuaion, is adapabiliy is heavily rewarded during non-saionary period when coninuous nework change akes place. 2) Piecewise Saionary Change: The parameer Θ 2 [] changes in a piecewise consan way in he synheic nework, and is esimaion can shed some ligh on TVDBN s performance in piecewise saionary nework. Some elemens of he esimaed ˆΘ 2 [] are ploed in Fig. 3 (b). I is observed ha TVDBN can correcly rack Θ 21 [] and quickly converge o he rue value even when abrup changes occur a ime 51 and 11. The quick response o abrup parameer change is aribued o an adapive selecion of smoohing coefficien α in he parameer ransiion model Eq. (1). We have chosen a large α (1) for parameer ransiion condiioned on unchanging srucure (G[ + 1] = G[]); and a relaively small α (2) for parameer ransiion condiioned on changing srucure (G[ + 1] G[]). In his way, we can obain sable parameer esimaion wihin saionary period and remain sensiive o abrup change poins simulaneously. The KL divergences for he disribuions of node X 2 [] esimaed wih differen mehods are also compared in Fig. 4 (b). The performance of TVDBN is close o piecewise saionary models such as abn and HCHC, and is much beer han oher models under comparison. For piecewise saionary models, large esimaion error will be incurred if a wrong saionary mode is seleced. However, TVDBN is free of his rouble because i can change boh abruply and coninuously.

9 WANG e al.: TIME VARYING DYNAMIC BAYESIAN NETWORK FOR NON-STATIONARY EVENTS MODELING AND ONLINE INFERENCE 9 TABLE II AVERAGE KL DIVERGENCES FOR EACH NODE DISTRIBUTION AND THE OVERALL DISTRIBUTION. DIFFERENT MODELS ARE COMPARED: PF-TVDBN (OUR METHOD); DBN [1]; K-TVDBN [25]; ABN [22]; SLDS [11]; HCHC [23]. Mehod p(x 1 [] ) p(x 2 [] ) p(x[] ) PF-TVDBN DBN K-TVDBN abn SLDS HCHC G[] Ĝ[] iniial srucure index Fig. 5. Esimaed nework srucure index and ground ruh. Fig. 8. The ime needed o converge o rue nework saus from differen iniial nework srucures PF TVDBN DBN K TVDBN abn SLDS HCHC Fig. 6. KL Divergences beween ground ruh and overall daa disribuions p(x[] ) esimaed wih mehods: PF-TVDBN (our mehod); DBN [1]; K- TVDBN [25]; abn [22]; SLDS [11]; HCHC [23]. 3) Overall Performance: The esimaed nework srucure Ĝ[] is ploed in Fig. 5 ogeher wih ground ruh. I can be seen he esimaed value is correc mos of he ime, excep for a few errors around model swiching poins. The effecive exploraion of nework srucure space can guide he esimaion of nework parameers, which is anoher reason for TVDBN s beer performance over is compeiors. Fig. 6 shows he comparison of KL divergences for he disribuion over all nodes. The average KL divergences over ime are summarized in Table II. Our mehod ouperforms all he ohers and has very small KL divergence on average. 4) Random Iniializaion: To furher invesigae he online adapabiliy of TVDBN, we ry o employ i o rack ime varying nework wih unknown iniial saus. Daa sequences generaed by he above synheic nework in ime period [11, 15] are used as observaions. The paricle filer is iniialized wih random nework srucures and parameers. For each of he 21 possible iniial nework srucures, he algorihm is run for 1 rials wih randomly sampled parameers. The esimaed nework saus a = 15 is compared wih ground ruh in Fig. 7. As shown in (a), he rue srucure G[15] = 15 can be recovered wih high probabiliy from mos of he random iniial srucures. In some rials iniialized wih srucure 6, 7, 12, 14, he esimaed nework converges o a wrong mode wih G[15] = 11, which only differs from rue srucure by one edge. For hose rials converging o correc srucure, he daa disribuion represened by he esimaed parameers is furher compared wih ground ruh, and he KL divergences beween hem are ploed in (b). The divergence is low for mos rials, which shows he esimaed parameers also converge o correc values. In Fig. 8, he ime needed o reach convergence is ploed for each iniial nework srucure. Convergence is defined o occur when he poserior probabiliy of rue srucure is higher han.95 and he KL divergence from rue disribuion is less han.1. On average we can reach convergence in less han 2 ime seps from any iniial sae, wih he nework sill evolving in he meanime. 5) Tuning Parameers: In all he experimens above, we have se λ 1 = 1. and λ 2 = 2.5 for he srucure ransiion model in Eq. (7). To ge a beer undersanding of heir

10 1 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. XX, NO. XX, MMMMM 2YY iniial srucure index iniial srucure index (a) (b) Fig. 7. Esimaed nework disribuion a ime 15 wih random srucure iniializaion a ime 11. (a) average poserior probabiliy of rue nework srucure G[15] = 15; (b) KL divergence for p(x[15] ). Median, minimal, maximal rials are shown λ λ λ λ2.1 (a) (b) Fig. 9. The abiliy of TVDBN o converge o rue srucure G[] = 15 from a fully conneced srucure, esed wih differen parameer se {λ 1, λ 2 }. (a) percenage of convergence; (b) average ime aken o reach convergence. roles played in TVDBN model, we ry differen combinaions of λ 1 and λ 2, and repea he previous random iniializaion experimens wih a fully conneced iniial nework srucure. For each pair of λ 1 and λ 2, he percenage and ime of convergence o rue srucure are ploed in Fig. 9. I can be seen ha as he penaly on srucure complexiy (λ 1 ) decreases and he penaly on srucure change (λ 2 ) increases, our model will have a beer chance o converge, bu he ime aken o reach convergence will be longer. Therefore, parameers λ 1 and λ 2 can be uned o conrol he radeoff beween change sensiiviy and esimaion sabiliy. B. Acive Camera Tracking Video objec racking is a popular applicaion of dynamic Bayesian nework where arge sae is described by hidden nodes X[]. In he simples case, le X[] = (X 1 [], X 2 []) T be he 2D image coordinaes of arge posiion a ime. The arge dynamics can be specified by a second order auoregression model of Gaussian disribuion, and he likelihood of sae can be measured from image feaures such as HSV color hisogram [41]. For real ime racking, i is ofen a desirable feaure ha camera can acively follow he moving arge. However, he inconsisen self-moion of camera will be superimposed on arge s moion in image plane, which makes i no longer suiable o model he arge dynamics wih a saionary auoregression process. Therefore, we propose a ime varying Gaussian dynamic disribuion for objec racking wih acive camera: p(x[] X[ 2 : 1], v[]) = N (X[]; 2X[ 1] X[ 2], v[]) (29) where v[] = [v 1 [], ;, v 2 []] is he ime varying covariance marix. The emporal variaion of v 1 [] and v 2 [] inerpres he changing accuracy of auoregression predicion caused by irregular camera moion. In his ime varying Gaussian DBN, v 1 [] and v 2 [] consiue he nework parameer Θ[]. The ransiion of v i [] follows he muliplicaive random walk

11 WANG e al.: TIME VARYING DYNAMIC BAYESIAN NETWORK FOR NON-STATIONARY EVENTS MODELING AND ONLINE INFERENCE 11 (a) (b) Fig. 1. Tracking resuls on acive camera video sequence. (a) frame 146, arge is labeled by a red recangle; (b) frame 16, each paricle is visualized by a blue recangle ˆv 1 [] ˆv 2 [] 2 15 d x [] d y [] (a) (b) Fig. 11. (a) inferred ime varying dynamic variances; (b) iner-frame ranslaion in x/y direcion found by moion esimaion algorihm. disribuion in Eq. (21), wih a = 5, b = 5, d = 1/2. Nework srucure G[] says consan in his experimen. The ime varying Gaussian TVDBN model is esed on a video sequence recorded by a il-pan camera. A yellow oy car in he video is racked using paricle filer and is iniial posiion is labeled manually. Alhough boh of he camera and he car are moving rapidly, our racking resuls mach he arge posiions very well, as can be seen from he snapshos in Fig. 1. The disribuion of paricles is visualized for frame 16 in (b), which shows he variance of paricles becomes large in verical direcion when he camera urns upwards, and he uncerain moion in ha direcion is well handled. The srong correlaion beween dynamic variance and camera moion can be seen more clearly in Fig. 11, which plos he esimaed value of ˆv i [] and he iner-frame image ranslaions found by global moion esimaion. The accuracy of racking is quaniaively measured using racking rae [42], which is defined as he percenage of frames whose racking resul overlaps wih ground ruh by a leas 3%. Fig. 12 plos he racking raes achieved by TVDBN model as well as saionary DBN [1], kernel reweighed TVDBN (K-TVDBN) [25], and segmenaion-based DBN K TVDBN Seg GMM PF TVDBN paricle number Fig. 12. Tracking raes achieved by PF-TVDBN (our mehod), DBN [1], K-TVDBN [25], and seg-ggm [16]. Tess wih various numbers of paricles have been run in he experimen. Gaussian graphic model (seg-ggm) [16]. The comparisons are conduced under differen seings of paricle number, and for each seing 1 rials are run. As seen from he figure, our

12 12 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. XX, NO. XX, MMMMM 2YY TABLE III GRAPH STRUCTURE AND NODE TRANSITION DISTRIBUTION FOR DIFFERENT INTERACTION CLASSES. THE INDICES OF GRAPH STRUCTURE ARE DEFINED AS IN FIG. 2. Ineracion G[] p(x i [] Pa(X i []), G[]) wandering G W = 4 X i [] N (X i []; X i [ 1],.1), i = 1, 2 joining G J = 9 X 1 [] N (X 1 []; X 1 [ 1] + b J [](X 2 [ 1] X 1 [ 1]),.2) X 2 [] N (X 2 []; X 2 [ 1] + b J [](X 1 [ 1] X 2 [ 1]),.2) moving as a group G M = 17 X 1 [] N (X 1 []; X 1 [ 1],.4) X 2 [] N (X 2 []; X 2 [ 1] + b M [](X 1 [] X 1 [ 1]),.3) p(g[] = GJ Z) p(g[] = GW Z) 3.4 p(g[] = GM Z).2 25 Fig. 13. Trajecories of wo arges (marked by colored cross and circle) racked using TVDBN model. The color bar illusraes he ime of rajecory poins. mehod achieves much higher racking raes han he oher mehods under all paricle seings. C. Muliple Targes Ineracion Recogniion In his experimen, TVDBN model is used o recognize he class of ineracion among muliple arges based on he ransiion disribuion of heir rajecories. The similar problem has been invesigaed in [43] using DBN model. Consider he case of wo arges, whose coordinaes in image a ime are represened by (X 1 [], Y 1 []) and (X 2 [], Y 2 []), respecively. Suppose here are hree classes of ineracion beween he arges: wandering, joining, and moving as a group. The condiional dependency among arge saes will be differen for each ineracion class, and can be represened by a disinc nework srucure. In a video, he ineracion beween arges may change from ime o ime, and heir rajecories are modeled by TVDBN as non-saionary sequences. We can recognize he ineracion class by inferring he curren nework srucure of TVDBN model. The nework srucure G[] and he ransiion disribuion of node X i [] are lised in Table III for all ineracions. The ransiion disribuion of Y i [] is defined similarly as ha of X i []. When wo arges are wandering, here is no ineracion beween hem, and he ransiion disribuion is modeled by firs order auoregression. When wo arges are joining, hey approach each oher hrough he line connecing heir previous posiions. The speed of joining is conrolled by ime varying linear coefficien b J []. In he case where wo arges move ogeher as a group, we assume ha X 1 [] is he sae of leader arge which moves by is own will, and X 2 [] is he sae of follower arge which mimics he movemen of Fig. 14. The inferred poserior disribuion of nework srucure G[] over hree possible values. GT PF TVDBN DBN K TVDBN seg GGM NLMS HCHC SLDS Ineracing Fig. 15. GT (ground ruh) of ineracion ype versus ime, and he recogniion resuls found by PF-TVDBN (our mehod), DBN [1], K-TVDBN [25], seg- GGM [16], NLMS [3], HCHC [23], and SLDS [11]. leader arge. The exen o which he follower is affeced is conrolled by ime varying linear coefficien b M []. Therefore, we have wo ime varying parameers in his TVDBN model: Θ[] = {b J [], b M []}. The ransiion of hese parameers follows he random walk model in Eq. (2), wih σ bj =.5 and σ bm =.5. The experimen of ineracion recogniion is carried ou on a video from he CAVIAR daabase [44]. We use 4 paricles o rack he posiions and ineracion class of wo people simulaneously. The arge posiions are racked successfully hroughou he video, as shown by he rajecories in Fig. 13. To recognize he ineracion beween arges, he poserior

13 WANG e al.: TIME VARYING DYNAMIC BAYESIAN NETWORK FOR NON-STATIONARY EVENTS MODELING AND ONLINE INFERENCE Ineracing Ineracing Ineracing Ineracing Ineracing Ineracing Ineracing Ineracing (a) (b) (c) (d) Ineracing Ineracing Ineracing Ineracing Ineracing Ineracing (e) (f) (g) Fig. 16. Confusion marices for ineracion recogniion using: (a) PF-TVDBN (our mehod); (b) DBN [1]; (c) K-TVDBN [25]; (d) seg-ggm [16]; (e) NLMS [3]; (f) HCHC [23]; (g) SLDS [11]. TABLE IV PRECISION OF INTERACTION RECOGNITION ACHIEVED BY DIFFERENT METHODS. Mehod PF-TVDBN DBN [1] K-TVDBN [25] seg-ggm [16] NLMS [3] HCHC [23] SLDS [11] Precision disribuion of srucure G[] is evaluaed as shown in Fig. 14. The ineracion corresponding o he MAP mode of G[] is recognized as he curren ineracion ype. Our recogniion resuls (PF-TVDBN) are ploed in Fig. 15, ogeher wih he ground ruh. The recogniion resuls generaed by several oher non-saionary models are also given for comparison, which include he saionary dynamic Bayesian nework (DBN) [1], he kernel re-weighed TVDBN (K-TVDBN) [25], he segmenaion-based Gaussian graphic model (seg-ggm) [16], he normalized leas square algorihm (NLMS) [3], he hidden conroller hill climb algorihm (HCHC) [23], and he swiching linear dynamic sysem (SLDS) [11]. The confusion marices for each mehod are shown in Fig. 16, and he recogniion precisions are compared in Table IV. Our mehod is shown o give he bes resul among all. Furhermore, he wo models wih performance closes o ours - DBN and NLMS - canno describe ineracion class explicily; heir recogniion is done hrough a careful hreshold on he ime varying parameers. Our TVDBN model is free of such adhoc hreshold and is inference resuls are self-explanaory. VI. CONCLUSION AND DISCUSSION We have proposed a new ime varying dynamic Bayesian nework model ha is capable of describing he evoluion of non-saionary emporal sequences. The nework srucure and parameer are assumed o change smoohly over ime, and heir ransiion disribuions are designed accordingly. Paricle filering is employed for online inference of he hidden nodes and changing nework, so ha daa space and nework configuraion can be explored simulaneously. Our conribuion is hreefold. Firs, he proposed TVDBN model works online. I can adapively rack he curren sae of nework wih he laes daa observaion. Second, our algorihm provides a general framework ha can be applied o non-saionary sequences of any disribuion. The cases of mulinomial and Gaussian disribuions have been sudied in deail. Third, we have validaed he effeciveness of he proposed model wih exensive experimens, including boh simulaion and ess on video sequences. The resuls demonsrae he superioriy of our mehod over oher non-saionary models, and show is prospecive applicaions in acive camera racking and muliple arges ineracion recogniion. Discussion on Scalabiliy Our work is a preliminary exploraion of modeling nonsaionary dynamic neworks. One of a few imporan problems ha remain o be invesigaed is how o model large-scale ime varying neworks, such as geneic nework and social nework. As he number of nodes grows in a nework, he number of possible nework opologies and he dimension of nework parameer will all grow exponenially. This will aggravae he daa scarciy problem and makes online nework adapaion more challenging. One way o resric he exploding soluion space is making use of prior knowledge on nework srucure. For example, in social nework, he maximal degree of a single node is usually bounded and does no increase wih he size of nework. Also, a huge social nework may be decomposed ino many small-sized cliques which are loosely conneced wih each oher. If we can cluser all he nodes ino such cliques (which is also an open quesion), hen modeling he nework will be reduced o modeling each of hese cliques individually. On he oher hand, someimes we are more ineresed in

14 14 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. XX, NO. XX, MMMMM 2YY he global behavior of he nodes in a large-scale nework han he characerisic of an individual node. In such cases, all he nodes can be regarded as homogeneous and a common saisic model can be employed o describe heir disribuions. For example, if he relaionship beween any wo linked nodes is governed by a common poenial funcion, he nework will reduce o a pairwise Markov random field, and he number of is parameers will no longer increase wih nework size. Compuaional complexiy is anoher hing o consider if TVDBN model is used for large-scale neworks. As he size of nework increases, a prohibiively large number of paricles will be required for robus inference, which hampers he real ime applicaion of our model. In order o reduce he number of required paricles, we should design more efficien proposal disribuion by leveraging domain knowledge and hisorical daa. Techniques such as Rao-Blackwellizaion may be used o resric sampling space. The burgeoning field of cloud compuing may also provide a soluion o he compuaion burden associaed wih large neworks. ACKNOWLEDGMENT This work was suppored in par by Hi-Tech Research and Developmen Program of China 863 (26AA1Z124), NSFC (69273, 68281, 61255), 973 Program (21CB73141) and he 111 Projec (B722). Research was sponsored by he Army Research Laboraory and was accomplished under Cooperaive Agreemen Number W911NF The views and conclusions conained in his documen are hose of he auhors and should no be inerpreed as represening he official policies, eiher expressed or implied, of he Army Research Laboraory or he U.S. Governmen. The U.S. Governmen is auhorized o reproduce and disribue reprins for Governmen purposes nowihsanding any copyrigh noaion here on. E. E. Kuruoğlu graefully acknowledges he suppor of he Insiue of Image Communicaion and Informaion Processing, SJTU under 111 projec and parial suppor from CNR-shor erm mobiliy program. REFERENCES [1] K. Murphy, Dynamic Bayesian neworks: Represenaion, inference and learning, Ph.D. disseraion, UC Berkeley, Compuer Science Division, July 22. [2] R. Prado, G. Huera, and M. Wes, Bayesian ime-varying auoregressions: heory, mehods and applicaions, Resenhas, vol. 4, pp , 21. [3] E. Moulines, P. Prioure, and F. Roueff, On recursive esimaion for ime varying auoregressive processes, Annals of Saisics, vol. 33, no. 6, pp , 25. [4] L. D. Johnson and G. Sakoulis, Maximizing equiy marke secor predicabiliy in a Bayesian ime-varying parameer model, Compuaional Saisics and Daa Analysis, vol. 52, no. 6, pp , 28. [5] A. Rao, D. J. Saes, and J. D. Engel, Inferring ime-varying nework opologies from gene expression daa, EURASIP Journal on Bioinformaics and Sysems Biology, vol. 27, p. 12, 27. [6] M. Wes, R. Prado, and A. D. Krysal, Evaluaion and comparison of EEG races: Laen srucure in nonsaionary ime series, Journal of he American Saisical Associaion, vol. 94, no. 446, pp , [7] D. Gencaga, E. E. Kuruoglu, and A. Eruzun, Modeling non-gaussian ime-varying vecor auoregressive processes by paricle filering, Mulidimensional Sysems and Signal Processing, vol. 21, no. 1, pp , 21. [8] A. Ihler, J. Huchins, and P. Smyh, Adapive even deecion wih ime-varying Poisson processes, Proceedings of he 12h ACM SIGKDD inernaional conference on knowledge discovery and daa mining, pp , Augus 26. [9] Z. Ghahramani and G. E. Hinon, Variaional learning for swiching sae-space models, Neural Compuaion, vol. 12, pp , 2. [1] A. Blake, B. Norh, and M. Isard, Learning muli-class dynamics, Neural Informaion Processing Sysems (NIPS), pp , [11] V. Pavlovic, J. M. Rehg, and J. Maccormick, Learning swiching linear models of human moion, Neural Informaion Processing Sysems (NIPS), pp , 2. [12] S. M. Oh, J. M. Rehg, T. Balch, and F. Dellaer, Learning and inferring moion paerns using parameric segmenal swiching linear dynamic sysems, Inernaional Journal of Compuer Vision, vol. 77, pp , 28. [13] E. Fox, E. Sudderh, M. Jordan, and A. Willsky, Nonparameric Bayesian learning of swiching linear dynamical sysems, in Advances in Neural Informaion Processing Sysems 21, 29, pp [14] P. Turaga and R. Chellappa, Locally ime-invarian models of human aciviies using rajecories on he grassmannian, IEEE Conference on Compuer Vision and Paern Recogniion, pp , 29. [15] C. Andrieu, M. Davy, and A. Douce, Efficien paricle filering for jump markov sysems. applicaion o ime-varying auoregressions, IEEE Trans. on Signal Processing, vol. 51, no. 7, pp , 23. [16] X. Xuan and K. Murphy, Modeling changing dependency srucure in mulivariae ime series, in Proceedings of he 24h Inernaional Conference on Machine Learning, 27, pp [17] P. Fearnhead, Exac and efficien bayesian inference for muliple changepoin problems, Saisics and Compuing, vol. 16, pp , 26. [18] K. Wang, J. Zhang, F. Shen, and L. Shi, Adapive learning of dynamic Bayesian neworks wih changing srucures by deecing geomeric srucures of ime series, Knowledge and Informaion Sysems, vol. 17, no. 1, pp , 28. [19] J. W. Robinson and A. J. Haremink, Non-saionary dynamic Bayesian neworks, Neural Informaion Processing Sysems (NIPS), pp , 28. [2] M. Grzegorczyk and D. Husmeier, Non-saionary coninuous dynamic bayesian neworks, in Advances in Neural Informaion Processing Sysems 22 (NIPS), 29, pp [21] E. Punskaya, C. Andrieu, A. Douce, and W. Fizgerald, Bayesian curve fiing using MCMC wih applicaions o signal segmenaion, IEEE Trans. on Signal Processing, vol. 5, no. 3, pp , 22. [22] S. H. Nielsen and T. D. Nielsen, Adaping Bayes nework srucures o non-saionary domains, Inernaional Journal of Approximae Reasoning, vol. 49, no. 2, pp , 28. [23] A. Tucker and X. Liu, A Bayesian nework approach o explaining ime series wih changing srucure, Inelligen Daa Analysis, vol. 8, no. 5, pp , 24. [24] F. Guo, S. Hanneke, W. Fu, and E. P. Xing, Recovering emporally rewiring neworks: a model-based approach, in Proceedings of he 24h Inernaional Conference on Machine Learning, 27, pp [25] L. Song, M. Kolar, and E. P. Xing, Time-varying dynamic bayesian neworks, Proceeding of he 23rd Neural Informaion Processing Sysems (NIPS), 29. [26] M. Kolar, L. Song, and E. P. Xing, Sparsisen learning of varyingcoefficien models wih srucural changes, in Advances in Neural Informaion Processing Sysems 22, 29, pp [27] M. Kolar, L. Song, A. Ahmed, and E. P. Xing, Esimaing ime-varying neworks, Annals of Applied Saisics, vol. 4, pp , 21. [28] W. Fu, L. Song, and E. P. Xing, Dynamic mixed membership block model for evolving neworks, in Proceedings of he 26h Inernaional Conference on Machine Learning, 29. [29] D. Heckerman, A Tuorial on Learning wih Bayesian Neworks. In Learning in Graphical Models, M. Jordan, ed.. MIT Press, Cambridge, MA, [3] N. Friedman, K. Murphy, and S. Russell, Learning he srucure of dynamic probabilisic neworks, in Proceedings of he 14h Annual Conference on Uncerainy in Arificial Inelligence, 1998, pp [31] A.-L. Barabási and R. Alber, Emergence of scaling in random neworks, Science, vol. 286, pp , [32] R. Kumar, P. Raghavan, S. Rajagopalan, D. Sivakumar, A. Tomkins, and E. Upfal, Sochasic models for he web graph, in Proc 41s Annual Symposium on Foundaions of Compuer Science, IEEE Compuer Sociey, 2, pp [33] N. Deo and A. Cami, Preferenial deleion in dynamic models of weblike neworks, Informaion Process Leer, vol. 12, pp , 27.

15 WANG e al.: TIME VARYING DYNAMIC BAYESIAN NETWORK FOR NON-STATIONARY EVENTS MODELING AND ONLINE INFERENCE 15 [34] P. J. Green, Reversible jump Markov Chain Mone Carlo compuaion and Bayesian model deerminaion, Biomerika, vol. 82, pp , [35] G. Schwarz, Esimaing he dimension of a model, The Annals of Saisics, vol. 6, pp , [36] S. Veeramachaneni, D. Sona, and P. Avesani, Hierarchical Dirichle model for documen classificaion, in Inernaional Conference on Machine Learning, Bonn Germany, vol. 119, 25, pp [37] D. Geiger and D. Heckerman, Learning Gaussian neworks, in Proceedings of he 1h Annual Conference on Uncerainy in Arificial Inelligence, 1994, pp [38] R. D. Shacher and C. R. Kenley, Gaussian influence diagrams, Managemen Science, vol. 35, no. 5, pp , [39] A. Douce, S. Godsill, and C. Andrieu, On sequenial Mone Carlo sampling mehods for Bayesian filering, Saisics and Compuing, vol. 1, pp , 2. [4] M. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, A uorial on paricle filers for on-line nonlinear/non-gaussian Bayesian racking, IEEE Trans. on Signal Processing, vol. 5, no. 2, pp , 22. [41] P. Pérez, C. Hue, J. Vermaak, and M. Gangne, Color-based probabilisic racking, Proceedings of he 7h European Conference on Compuer Vision-Par I, pp , 22. [42] R. Collins, X. Zhou, and S. K. Teh, An open source racking esbed and evaluaion web sie, in IEEE Inernaional Workshop on Performance Evaluaion of Tracking and Surveillance (PETS 25), January 25. [43] A. Dore and C. Regazzoni, Ineracion analysis wih a bayesian rajecory model, IEEE Inelligen Sysems, vol. 25, pp. 32 4, 21. [44] CAVIAR, Conex Aware Vision using Image-based Acive Recogniion projec, 25, EC Funded projec/ist , found a hp://homepages.inf.ed.ac.uk/rbf/caviar/. Zhaowen Wang received he B.E. and M.S. degrees in elecrical engineering from he Shanghai Jiao Tong Universiy, Shanghai, China, in 26 and 29. respecively. He is currenly pursuing Ph.D. degree in he Deparmen of Elecrical and Compuer Engineering, Universiy of Illinois a Urbana-Champaign (UIUC), Urbana. Since spring 21, he has been wih Deparmen of Elecrical and Compuer Engineering, UIUC. His research ineress include compuer vision, saisical learning, and video evens analysis. Xiaokang Yang (M SM 4) received he B.S. degree from Xiamen Universiy, Xiamen, China, in 1994, he M.S. degree from he Chinese Academy of Sciences, Shanghai, China, in 1997, and he Ph.D. degree from Shanghai Jiao Tong Universiy, Shanghai, in 2. He is currenly a Professor and he depuy Direcor of he Insiue of Image Communicaion and Informaion Processing, Deparmen of Elecronic Engineering, Shanghai Jiao Tong Universiy. From Augus 27 o July 28, he visied he Insiue for Compuer Science, Universiy of Freiburg, Germany, as an Alexander von Humbold Research Fellow. From Sepember 2 o March 22, he worked as a Research Fellow a he Cenre for Signal Processing, Nanyang Technological Universiy, Singapore. From April 22 o Ocober 24, he was a Research Scienis a he Insiue for Infocomm Research (I2R), Singapore. He has published over 13 refereed papers, and has filed 14 paens. His curren research ineress include visual processing and communicaion, media analysis and rerieval, and paern recogniion. Dr. Yang received he Microsof Young Professorship Award 26, he Bes Young Invesigaor Paper Award a IS&T/SPIE Inernaional Conference on Video Communicaion and Image Processing (VCIP23), and awards from he A-STAR and Tan Kah Kee foundaions. He is currenly a member of Design and Implemenaion of Signal Processing Sysems (DISPS) Technical Commiee of he IEEE Signal Processing Sociey and a member of Visual Signal Processing and Communicaions (VSPC) Technical Commiee of he IEEE Circuis and Sysems Sociey. Yi Xu received her B.S. and M.S. degrees from Nanjing Universiy of Science and Technology, Nanjing, China, in 1996 and 1999, respecively, and Ph.D. degree in Informaion and Communicaion Engineering from Shanghai Jiao Tong Universiy, in 25. Dr. Xu is currenly an assisan professor in he Insiue of Image Communicaion and Informaion Processing, Deparmen of Elecronic Engineering, Shanghai Jiao Tong Universiy, where she works in he area of video analysis and undersanding. Her research ineress and aciviies cover quaernion wavele ransform, image sequence analysis and video conen undersanding, especially recogniion of moving objecs and abnormal evens. Ercan E. Kuruoğlu (M 98-SM 6) was born in 1969 in Ankara, Turkey. He received he B.Sc. and M.Sc. degrees in elecrical and elecronics engineering from Bilken Universiy, Turkey, in 1991 and 1993, and he M.Phil. and Ph.D. degrees in informaion engineering from he Universiy of Cambridge, Cambridge, U.K., in 1995 and 1998, respecively. Upon compleion of his sudies, he joined he Xerox Research Cener Europe, Cambridge, as a permanen member of he Collaboraive Mulimedia Sysems Group. He was an ERCIM Fellow in 2 a INRIA-Sophia Anipolis, France. In January 22, he joined ISTI-CNR, Pisa, Ialy. He was a visiing professor a Georgia Tech-Shanghai in Auumn 27. He is currenly a Senior Researcher and Associae Professor a ISTI- CNR. His research ineress are in he areas of saisical signal and image processing and informaion and coding heory wih applicaions in asrophysics, bioinformaics, elecommunicaions, and inelligen user inerfaces. Dr. Kuruoğlu was an Associae Edior for he IEEE TRANSACTIONS ON SIGNAL PROCESSING beween He is currenly an Associae Edior for he IEEE TRANSACTIONS ON IMAGE PROCESSING and is on he ediorial board of Digial Signal Processing: A Review Journal. He aced as he echnical chair for EUSIPCO 26. He is a member of he IEEE Technical Commiee on Signal Processing Theory and Mehods. Thomas S. Huang (F 1) received he B.S. degree in elecrical engineering from Naional Taiwan Universiy, Taipei, Taiwan, R.O.C., and he M.S. and Sc.D. degrees in elecrical engineering from he Massachuses Insiue of Technology (MIT), Cambridge. He was on he Faculy of he Deparmen of Elecrical Engineering a MIT from 1963 o 1973; and on he Faculy of he School of Elecrical Engineering and Direcor of is Laboraory for Informaion and Signal Processing a Purdue Universiy from 1973 o 198. In 198, he joined he Universiy of Illinois a Urbana-Champaign, where he is now William L. Everi Disinguished Professor of Elecrical and Compuer Engineering, and Research Professor a he Coordinaed Science Laboraory, and a he Beckman Insiue for Advanced Science he is Technology and Co-Chair of he Insiue s major research heme Human Compuer Inelligen Ineracion. His professional ineress lie in he broad area of informaion echnology, especially he ransmission and processing of mulidimensional signals. He has published 21 books, and over 6 papers in nework heory, digial filering, image processing, and compuer vision. Dr. Huang is a Member of he Naional Academy of Engineering; a Member of he Academia Sinica, Republic of China; a Foreign Member of he Chinese Academies of Engineering and Sciences; and a Fellow of he Inernaional Associaion of Paern Recogniion, IEEE, and he Opical Sociey of America. Among his many honors and awards: Honda Lifeime Achievemen Award, IEEE Jack Kilby Signal Processing Medal, and he KS Fu Prize of he In. Asso. for Paern Recogniion.