Decision-Theoretic Approach to Maximizing Observation of Multiple Targets in Multi-Camera Surveillance

Transcription

1 Decision-Theoretic Approach to Maxiizing Observation of Multiple Targets in Multi-Caera Surveillance Prabhu Natarajan, Trong Nghia Hoang, Kian Hsiang Low, and Mohan Kankanhalli Departent of Coputer Science, National University of Singapore Coputing 1, 13 Coputing Drive, Singapore , Republic of Singapore {prabhu, nghiaht, lowkh, ABSTRACT This paper presents a novel decision-theoretic approach to control and coordinate ultiple active caeras for observing a nuber of oving targets in a surveillance syste. This approach offers the advantages of being able to (a) account for the stochasticity of targets otion via probabilistic odeling, and (b) address the tradeoff between axiizing the expected nuber of observed targets and the resolution of the observed targets through stochastic optiization. One of the key issues faced by existing approaches in ulti-caera surveillance is that of scalability with increasing nuber of targets. We show how its scalability can be iproved by exploiting the proble structure: as proven analytically, our decision-theoretic approach incurs tie that is linear in the nuber of targets to be observed during surveillance. As deonstrated epirically through siulations, our proposed approach can achieve high-quality surveillance of up to targets in real tie and its surveillance perforance degrades gracefully with increasing nuber of targets. We also deonstrate our proposed approach with real AXIS 214 PTZ caeras in axiizing the nuber of Lego robots observed at high resolution over a surveyed rectangular area. The results are proising and clearly show the feasibility of our decision-theoretic approach in controlling and coordinating the active caeras in real surveillance syste. Categories and Subject Descriptors I.4.8 [Scene Analysis]: Tracking; I.2.9 [Robotics]: Coercial robots and applications, Sensors General Ters Algoriths, Perforance, Experientation, Security Keywords Active caera networks, Sart caera networks, Multi-caera coordination and control, Surveillance and security 1. INTRODUCTION The use of active caeras in surveillance is becoing increasingly popular due to the recent advances in sart caera technologies [4]. These active caeras are endowed with pan, tilt, and zoo Appears in: Proceedings of the 11th International Conference on Autonoous Agents and Multiagent tes Innovative Applications Track (AAMAS 2012), Conitzer, Winikoff, Padgha, and van der Hoek (eds.), 4-8 June 2012, Valencia, Spain. Copyright c 2012, International Foundation for Autonoous Agents and Multiagent tes ( All rights reserved. capabilities, which can be exploited to provide high-quality surveillance. In order to achieve effective, real-tie surveillance, an efficient collaborative echanis is needed to control and coordinate these caeras actions, which is the focus of our work in this paper. Monitoring a set of targets oving in an environent is a challenging and difficult task because (a) the otion of these targets is often stochastic in nature, (b) it needs to address the non-trivial trade-off between axiizing the expected nuber of observed targets and the resolution of the observed targets, and (c) a caera coordination fraework should be scalable with an increasing nuber of targets. To elaborate, (a) the uncertainty in the targets otion akes it hard for the active caeras to know where to observe in order to keep these targets within their fields of view (fov) and they ay consequently lose track of the observed targets, (b) increasing the resolution of observing soe targets through panning, tilting, or zooing ay result in the loss of other targets being tracked, and (c) when the nuber of targets increases, a caera coordination fraework, if poorly designed, tends to incur exponentially increasing coputational tie, which degrades the perforance of the entire syste. These issues arise in any realworld surveillance applications such as target surveillance, observing a group of players in sports, industrial onitoring of protected sites, etc. Hence, we believe that, by addressing these practical issues, a ore effective surveillance syste can be realized and subsequently deployed in the real world. Note that our proposed surveillance task differs fro the typical sensor coverage proble, the latter of which instead focuses on axiizing the spatial coverage of the caeras that are independent of targets otion. In our work, we try to axiize the coverage of the observed targets in the environent. This paper presents a novel principled decision-theoretic approach to control and coordinate the active caeras for the surveillance of ultiple oving targets (Section 2). This approach is based on the Markov Decision Process (MDP) fraework, which allows the surveillance task to be fraed forally as a stochastic optiization proble (Sections 3 and 4). In particular, our MDP-based approach resolves the above-entioned issues: (a) the otion of the targets can be odeled probabilistically (Section 4.2), and (b) to address the trade-off, the active caeras actions are coordinated to axiize the expected nuber of observed targets while guaranteeing a pre-defined resolution of these observed targets (Section 4.4), and (c) the scalability can be iproved by exploiting the proble structure: as proven analytically (Section 4.5), our MDP-based approach incurs tie that is linear in the nuber of targets to be observed during surveillance. One key proble faced by existing ulti-caera ulti-target surveillance approaches is that of scalability with increasing nuber of targets (Section 2). As deonstrated epirically through siulations (Section 5), our MDP-based approach

2 Table 1: Coparison of related work based on (a) caera:target ratio, (b) priary criterion, and (c) uncertainty in targets otion. Surveillance/tracking strategy n n n = Maxiizing no. of Miniizing uncertainty Uncertainty in observed targets of targets locations targets otion Banerjee et al. [3] Costello et al. [5] toever et al. [9] Qureshi et al. [13] Soto et al. [15] Soerlade et al. [14] Huang et al. [8] Alfy et al. [6] Proposed MDP-based approach can achieve high-quality surveillance of up to targets in real tie and its surveillance perforance degrades gracefully with an increasing nuber of targets. The real-world experients (Section 5.3) show the practicality of our decision-theoretic approach to control and coordinate caeras in surveillance systes. 2. RELATED WORK Our proposed work is copared and contrasted with existing approaches for active caera surveillance based on the following classification: (a) ratio of nuber n of caeras to nuber of targets, (b) priary criterion - the ain objective/goal of the surveillance syste, and (c) uncertainty in targets otion - whether the targets otion uncertainty is considered in caera coordination and optial decision aking. This coparison is shown in Table 1. The caera:target ratio is further classified based on n, n, and n =. The caera:target ratio plays an iportant role in the choice of priary criterion that is used in the existing works, as explained below. The priary criterion is classified into: (i) axiizing the nuber of observed targets with certain guaranteed resolution and (ii) iniizing the uncertainty of individual targets locations. The targets otion is stochastic in nature and hence needs to be predicted and subsequently exploited for coordinating the caeras in a typical surveillance syste. The existing works are also classified based on whether they have accounted for the uncertainty in targets otion in their optiization fraework. Table 1 shows that when the caera:target ratio is either n = [3] or n [5, 6, 8, 9, 13, 14, 15], the priary criterion is to iniize the uncertainty of individual targets locations. By observing individual targets with ore caeras, the uncertainty of targets locations is decreased. In contrast, when the caera:target ratio is n, the priary criterion is to axiize the nuber of observed targets in the environent. In either criterion, the targets otion is inherently non-deterinistic. But, none of the previous works have accounted for the otion uncertainty in their optiization fraework. The works of [2, 12] ai to axiize the coverage of static targets in oni-directional active sensors. Since the targets are static, there is no notion of stochasticity of targets otion. All the above-entioned works use heuristic approaches to select the best actions for the active caeras. Such approaches are therefore tailored specifically to their own objectives and cannot be odified to achieve other objectives. In contrast, our approach is a general fraework in which different surveillance goals can be odeled as foral objective functions. To suarize, our proposed work is different fro the existing works in the following ways: (a) we use a foral, principled Markov Decision Process (MDP) fraework to select the optial actions for active caeras to axiize the expected nuber of observed targets; (b) we account for the uncertainty in targets otion by integrating a probabilistic otion odel into our optiization fraework; and (c) any previous works ([9, 13, 14, 16], etc.) face a serious scalability issue in ters of the nuber of targets to be observed. We shall show in later sections how the state space of the targets can be anaged efficiently by exploiting the structure and properties that are inherent in the surveillance proble. 3. SYSTEM ARCHITECTURE The proposed surveillance fraework consists of a supervised surveillance environent and an MDP controller. The environent consists of targets, static caeras, and active caeras. The targets are the oving objects (e.g., people, vehicles, robots, etc.) in the surveillance environent whose otions are stochastic in nature. The static caeras are wide-view caeras that can only provide low-quality inforation of the surveillance environent. These caeras are assued to be calibrated and can obtain the 3D location, direction, and velocity inforation of the targets. The active caeras are PTZ (pan/tilt/zoo) caeras that can get highresolution iages of the targets in the environent. The MDP controller odels the interaction between the active caeras and the environent, and provides a platfor to choose optial actions for these caeras in order to achieve high-quality surveillance tasks. Fig. 1 shows the top view of a representative surveillance environent where the full fov s of the active caeras are shown in dotted lines and the current active fov s are shaded. For siplicity, the static caeras are not shown. The active caeras are placed such that they can observe the coplete environent by pan/tilt/zoo operations but cannot observe all locations of the environent siultaneously. This akes the proble ore practical and challenging, thus ephasizing the need to control these active caeras. The static caeras deterine the location, direction, and velocity of targets and pass these inforation to the MDP controller. Based on these inforation, the MDP controller coputes the optial actions of active caeras such that the expected utility of the surveillance syste is axiized. The utility of the surveillance syste corresponds to the high-level application goal that can be defined forally using a real-valued objective function, as described in Section 4.4. Forally, the MDP controller is defined as a tuple (S, A, R, T f ) consisting of a set S of discrete states of active caeras and targets, a set A of joint actions of active caeras, a reward function R : S R representing the high-level surveillance goal, and a transition function T f : S A S [0, 1] denoting the probability P (S S, A) of switching fro the current state S S to the next state S S using the joint action A A. In the MDP fraework, the policy function π : S A aps fro each state to

3 Door Door caera 7 MDP Controller Actions for each active caera caera 8 caera 6 Wall caera 5 caera 1 caera 2 caera 3 Figure 1: te architecture. caera 4 a joint action of the caeras. Solving the MDP involves choosing the policy that axiizes the expected reward for any given state. The optial policy, denoted by π, axiizing the expected utility of the syste in the next tie step is given by π (S) = arg ax A A R(S ) P (S S, A). S S The ain challenge in the MDP is anaging the state space S and action space A. This is because the state space grows exponentially in the nuber of active caeras and targets. Hence, the policy coputation tie for our surveillance proble is exponential. In practice, the structure of the proble and environent can usually be exploited to reduce the nuber of states and the tie required to copute the optial policy. We will show in Section 4.5 how the state space can be anaged for our surveillance proble, thus allowing the MDP to be solved ore efficiently. The following assuptions are ade in our surveillance task: The targets are oblivious to the caeras, in particular, non-evasive (i.e., they do not try to escape fro the caeras fields of view) and their otion cannot be controlled nor influenced; The static caeras are calibrated accurately such that the 3D positioning errors of the targets are inial. This can be achieved by placing the caeras at high altitude; The total nuber of targets in the environent can be obtained fro static caeras and/or otion sensors at the entry and exit. 4. PROBLEM FORMULATION Given a set of caeras and targets in a surveillance syste, the MDP controller deterines the optial actions for these caeras such that the expected utility of the surveillance syste is axiized. In this section, we describe how an MDP fraework can be applied to a generic active caera surveillance in order to axiize the expected utility of the surveillance syste. We enuerate each coponent of the MDP fraework and show how these coponents can be forulated for a typical surveillance syste. In this work, the objective/reward function of the MDP odeling the high-level surveillance goal easures the total nuber of targets observed by active caeras with a guaranteed resolution. Maxiizing the nuber of observed targets with a guaranteed resolution is a andatory task in surveillance because we need to obtain the high-resolution iages of targets for bioetric and forensic tasks o 0o 90 o c i = + o + o +90 o target location (0,0) (0,1) (0,2) (0,3) (0,4) (0,5) (1,0) (1,1) (1,2) (1,3) (1,4) (1,5) Wall (2,0) (2,1) (2,2) (2,3) (2,4) (2,5) (3,0) (3,1) (3,2) (3,3) (3,4) (3,5) (a) (b) Figure 2: (a) Caera states and (b) target locations. like target detection, recognition, etc. In this work, we present a decision-theoretic approach for axiizing the expected nuber of targets observed by the active caeras. 4.1 es and Actions A state of the MDP coprises the states of active caeras and targets in the surveillance environent. The passive static caeras are first calibrated based on coon ground plane coordinates and then used to obtain the targets approxiate 3D location, velocity, and the direction inforation. Let n be the nuber of active caeras and be the nuber of targets in the environent such that n. In this anner, the surveillance proble becoes ore challenging and interesting since there are ore targets to be onitored by fewer active caeras. Let the set of possible states of each active caera in the environent be denoted by C such that each state c i C corresponds to a discretized pan/tilt/zoo position of caera i. For exaple, in Fig. 2a, the set of possible states of caera i based on discretized pan angles is given by C = {+90, +, 0,, 90 } and the current state c i is +. Let the state space of a target be represented by a set of tuples of location, direction and velocity, and denoted by T = T l T d T v where T l denotes a set of all possible locations of the target in the environent, T d denotes a set of all possible discretized directions between all pairs of locations in T l, and T v denotes a set of discretized velocities of the target. The surveillance environent is discretized into grid cells such that the centers of the grid cells represent the possible locations of a target, as shown in Fig. 2b. The approxiate 3D location of the target observed by static caeras will be apped to the center of the nearest grid cell. The direction and velocity of the target are deterined based on its current and previous locations. The static caeras detect the targets in their fov s and report their locations, directions, and velocities to the MDP controller. By calibrating the active caeras, the possible target locations in the environent that lie within the fov of each active caera in its various states can be pre-coputed. For each state c i C of active caera i, the subset of locations lying within its corresponding fov is denoted by fov(c i) T l. For exaple, Fig. 3 illustrates the fov (i.e., shaded polygon) of active caera 1 in its current state c 1; the subset of locations that are observed by caera 1 is given by fov(c 1) = {(0, 1), (0, 2),..., (2, 3), (2, 4)}. To observe targets with a guaranteed resolution, the zoo paraeter of an active caera can be adjusted to focus its fov so that iageries of the targets detected within its fov satisfy a pre-defined resolution. This requires liiting the depth of its fov, as depicted by the horizontal line in Fig. 3. As a result, if a target is located within fov(c i) of any caera i, then it is observed with a guaranteed resolution. For exaple, the iniu resolution of the huan face should be pixels, which is the base resolution for face detection [18]. The resolution of the targets should be higher than

4 fov(c 1 ) caera 1 (0,0) (0,1) (0,2) (0,3) (0,4) (0,5) (1,0) (1,1) (1,2) (1,3) (1,4) (1,5) (2,0) (2,1) (2,2) (2,3) (2,4) (2,5) (3,0) (3,1) (3,2) (3,3) (3,4) (3,5) (4,0) (4,1) (4,2) (4,3) (4,4) (4,5) Figure 3: fov(c 1) of caera pixels for other tasks like face recognition and expression analysis, vehicle nuber plate detection and identification, etc. Let the vector C = (c 1, c 2,..., c n) be the joint state of n active caeras in the environent and the vector T = (t 1, t 2,..., t ) be the joint state of targets in the environent where t k T is the state of target k. A state S S = T C n of the MDP is therefore of the for S = (T, C). The actions of an active caera are pan/tilt/zoo coands to ove the caera to a specified state. Let a i be an action of caera i corresponding to a pan/tilt/zoo coand. We assue that the delay in oving the caera to a specified state is negligible as the state-of-the-art caeras are capable of panning at a speed of 3 /sec [1]. The joint action of all caeras at any given tie is a vector A = (a 1, a 2,..., a n) A. Since we assue that the targets otion cannot be controlled, no action can be specified by the MDP controller to influence their otion in the surveillance environent. 4.2 Transition Function T f Recall that the transition function T f of the MDP denotes the probability P (S S, A) of oving fro the current state S to the next state S using the joint action A. In this subsection, we will show how this transition probability can be factored into transition probabilities of individual active caeras and targets using the conditional independence property, which is inherent in the state transition dynaics of the surveillance environent. As a result, the coputation tie of our optial policy is significantly reduced (i.e., fro exponential to linear in the nuber of targets), hence alleviating the scalability issue (see Theore 1). Firstly, the transition probability P (S S, A) can be factored into the transition probabilities of the active caeras and targets (i.e., respectively, P (C C, A) and P (T T )) due to conditional independence (see first equality of (1)). Specifically, the transition probability P (C C, A) of the active caeras is conditionally independent of the targets states. Since the targets are assued to be oblivious to the caeras, the transition probability P (T T ) (i.e., otion odel) of the targets is conditionally independent of the active caeras states and actions. Next, the transition probability P (C C, A) of the active caeras can also be factored into transition probabilities of individual active caeras due to conditional independence. The transition probability of an individual caera i is P (c i c i, a i) where c i, c i C are, respectively, its current and next states, and a i is its action. Since the transition probability of each active caera is conditionally independent of the other caeras given its current state and action, P (C C, A) can be factored into P (c i c i, a i) s for i = 1,..., n (see second equality of (1)). Modern active caeras are equipped with advanced functionalities that enable the to ove to the desired pan/tilt/zoo positions accurately [1]. Hence, it is practical to assue the transition of caera i to be deterinistic and consequently represented by a deterinistic function c i = execute(c i, a i) since P (c i c i, a i) evaluates to either 0 or 1. Siilarly, the transition probability P (T T ) of the targets can be factored into transition probabilities (i.e., otion odels) of individual targets by assuing conditional independence. The transition probability of target k is P (t k t k ) where t k, t k T are, respectively, its current and next states. Since the transition probability of each target is conditionally independent of the other targets given its current state, P (T T ) can be factored into P (t k t k ) s for k = 1,..., (see second equality of (1)). As discussed above, the transition probability P (S S, A) of the MDP can be factored into transition probabilities of individual active caeras and targets after repeatedly applying the conditional independence property: P (S S, A) = P (C C, A) P (T T ) n = P (c i c i, a i) P (t k t k ) i=1 P (t = k t k ) if P (c i c i, a i) = 1 for i = 1,..., n, 0 otherwise. 4.3 Transition Probability P (t k t k) of a Target To calculate the transition probability of a target, we first predict a target s oveent in a surveillance environent using a general velocity-direction otion odel. Specifically, this odel coprises two Gaussian distributions for the velocity v and direction d of the target: v N (µ v, σ v) and d N (µ d, σ d ) where the ean paraeters µ v and µ d are obtained fro the static caeras at every tie step based on the previous location of the target, and the variance paraeters σ v and σ d are learned fro a dataset of targets trajectories in the given supervised surveillance environent. Then, in every tie step t, we draw paired saples of velocity v and direction d of the target fro the Gaussian distributions, copute its corresponding predicted location (x t, y t) in the environent using x t = y t = x t 1 + v cos(d) dt y t 1 + v sin(d) dt and deterine the proportion of saples in each grid cell to produce the transition probability P (t k t k ) of the target. Fig. 4 shows the transition probability distribution of a target that is located at (x t 1, y t 1) = (5, 5) with µ v = 2 cells per tie step and µ d =. The probability distribution of the neighboring locations that the target will ove to in tie step t is shown as black dots. Since the possible locations, directions, and velocities of the target are finite, we can pre-copute the transition probabilities of the target and store the off-line. This helps to reduce the on-line policy coputation tie, as discussed in Theore Objective/Reward Function R The advantage of using MDPs in surveillance systes is that any high-level surveillance goal can be defined forally using a real-valued objective/reward function. In this work, the goal of the surveillance syste is to axiize the nuber of observed targets with a guaranteed resolution. Supposing the states of all targets are known, such a goal can be achieved by defining a reward function (1) (2)

5 Y Transition probability distribution for target Target s location at (5,5) Figure 4: Transition probability distribution of a target. that easures the total nuber of targets lying within the fov of any of the active caeras: R(S) = R((T, C)) = R(t k, C) (3) X R(t k, C) = { 1 if target k s location lies in fov(c), 0 otherwise; where fov(c) = n i=1 fov(ci) denotes a set of target locations in the environent, each of which lies within the fov of at least one active caera when the caeras are in state C. So, if the location of target k lies within fov(c), then it is guaranteed to be observed at a predefined iage resolution, as discussed in Section 4.1, and R(t k, C) = 1 results. 4.5 Policy Coputation The states of the targets in the next tie step are uncertain due to stochasticity of their otion. Therefore, the optial policy π has to instead axiize the expected total nuber of targets that lie within the fov of any of the active caeras in the next tie step: (4) π (S) = π ((T, C)) = arg ax V (T, C, A) (5) A A V (T, C, A) = T T R((T, C )) P (T T ) (6) where T and C are, respectively, the joint states of the targets and active caeras in the next tie step. The next joint state C of the caeras can be deterined deterinistically fro their current joint state C and action A using the function c i = execute(c i, a i) for i = 1,..., n (Section 4.2). Coputing the policy π (5) for a given state S incurs O( A T ) tie, which is exponential in the nuber of targets. Its tie coplexity can be significantly reduced by exploiting the inherent structure of our surveillance proble, in particular, the conditional independence property in the transition odel of the MDP (Section 4.3). As a result, the value function V (6) can be reduced to V (T, C, A) = Ṽ (t k, C ) (7) Ṽ (t k, C ) = t k T R(t k, C ) P (t k t k ). (8) For a detailed derivation of (7), see Appendix A. Coputing the policy π for a given state S consequently incurs linear tie in the nuber of targets, as shown in the result below: THEOREM 1. If (1) holds, then coputing policy π (5) for a given state S incurs O( A T ) tie. To iprove the real-tie coputation of policy π, the values of Ṽ (t k, C ) (8) for all t k T and C C n can be pre-coputed and stored off-line. To do this, the values of P (t k t k ) for all t k, t k T have to be pre-coputed first, which incurs O( T 2 ) tie. The values of R(t k, C ) for all t k T and C C n also have to be pre-coputed, which incurs O( T C n ) tie. Consequently, the values of Ṽ (t k, C ) (8) for all t k T and C C n can be precoputed in O( T 2 C n ) tie. Hence, the total off-line coputation tie is O( T 2 C n ). The on-line coputation tie to derive policy π can then be reduced to O( A ), which includes the tie taken to look up the values of Ṽ (t k, C ) for targets (7) and over A possible joint actions (5). The result below suarizes the coputation tie incurred by the on-line and off-line processing steps: THEOREM 2. If (1) holds, then coputing policy π (5) for a given state S incurs off-line coputation tie of O( T 2 C n ) and on-line coputation tie of O( A ). 5. EXPERIMENTS AND DISCUSSIONS In this section, we present epirical evaluation of our MDPbased approach for axiizing the nuber of targets observed by active caeras. Our proposed approach is siulated in Player/Stage siulator [7] to perfor extensive experientations and ipleented using real AXIS 214 PTZ caeras to deonstrate its feasibility in real surveillance syste. Before describing the, it is iportant to point out that there is no standard benchark surveillance environents and datasets for active caera networks to copare our proposed approach with the other systes in the literature (e.g., [9, 13, 14]). While the priary criterion of these systes is to iniize the uncertainty of targets locations, our objective function is to axiize the nuber of targets observed in high-resolution iages (see Table 1). These existing systes use heuristic approaches that can optiize only their respective objective function and cannot be used for other objective functions. These systes also suffer fro scalability issue when the nuber of targets is increased. Furtherore, the optiization fraeworks of these existing systes deterine the caeras actions (or schedule the caeras) for the current tie step based on the current locations of the targets. In contrast, our approach deterines the caeras actions for the current tie step based on the expected locations of the targets in the next tie step (see (7) and (8)). This akes our approach perfor better than the existing ethods in axiizing the nuber of observed targets. Our MDP-based approach is epirically copared with the following existing heuristic ethods: toever s () Approach: The work of [9] provided an optiization ethod to capture high-resolution iages of a single target. It schedules the tasks for active caeras based on the location of the targets in the current tie step and assues that the targets will not ove out of the fov within the short duration; teatic () Approach: The active caeras pan autoatically in a round robin fashion such that every caera pans to each of its states for a finite duration; ic () Approach: The active caeras are fixed at specific states such that they can cover axiu area to get high-resolution iageries of the targets. Our approach and the above heuristic ethods are evaluated using the following perforance etric: P ercentobs = 100 τm tot τ i=1 M i obs

6 caera 2 caera 1 caera 3 caera 4 caera 4 caera 3 caera 1 caera 2 Figure 5: Setups of corridor and hall environents. where τ (i.e., set to 100 in siulations) is the total nuber of tie steps taken in our experients, M i obs is the total nuber of targets observed by the active caeras at a given tie step i, and M tot is the total nuber of targets present in the environent. That is, the P ercentobs etric averages the percentage of targets being observed by the active caeras over the entire duration of τ tie steps. We will first discuss the environental setup for the siulated experients and analyze the experiental results. Then, we will show the results of the real caera experients. Interested readers can view our deo video Siulated Experients: Setup In Player/Stage siulator, we have designed an active caera odel with functionalities to siulate real active caeras by configuring the nuber of states across pan angles, as discussed in Section 4.1. The targets otion are generated in Player/Stage siulator based on velocity-direction otion odel (see (2)), which resebles real huan otion in surveillance environent. The locations of the targets are deterined by a static caera, which is the siulator itself. We have conducted our experients for two environental setups (Fig. 5): corridor and hall. The sizes of the corridor and hall environents are, respectively, 5 grid cells and grid cells such that T l = 200. The size of a grid cell in the siulator is approxiately apped to 1 2 in real world. We have used up to n = 4 active caeras with C = 3, 5, and tested up to = targets. We have also conducted experients for the caera resolutions fov(c i) 25, 16 by reducing the size of the caera s fov polygon in the siulator. The set fov(c i) of target locations that are observed by each active caera is deterined by calibrating the active caeras in each of its state. 5.2 Siulated Experients: Results Figs. 6 and 7 show the perforance of our MDP-based approach for corridor and hall setups with n = 4, T l = 200, target s velocity v = 3 cells per tie step, and with varying, fov(c i), C, and sizes of clusters of targets that follow the Poisson distribution (λ = 3). The rest of this subsection describes the observations fro our experients. Our MDP-based approach perfors better for any of the target s velocity v = 1, 2, 3 cells per tie step. This is because the caeras are controlled based on the predicted locations of a target by atching its corresponding transition probabilities with respect to its observed state. It can be observed fro the experients that the perforance of MDP is uch better that the other approaches when (a) the velocity of the targets is higher, (b) the targets ove in clusters, and (c) when the resolution of the caeras is increased (i.e., fov(c i) is decreased). This is because when the velocity of the targets is high (i.e., v = 2.5, 3 cells per tie step), all the targets will alost certainly ove out of the fov s of the caeras in approach as the caeras are controlled based on 1 lowkh/caera.htl the current location of the targets, hence producing worse perforance (see Figs. 6a and 7a). When the targets ove in clusters, then the approach suffers even ore perforance degradation because it has high tendency to lose clusters of targets. On the other hand, since the MDP has the correct transition odel, it gives superior perforance even when the targets ove in high velocity. By increasing the resolution of the active caeras (i.e., by reducing fov(c i) 25 to 16), it can be observed that the MDP perfors uch better when copared to the approach (Figs. 6c, 6d, 7c, and 7d). This is because when the targets are oving at a velocity of v = 3 cells per tie step and are observed at higher resolution (i.e., fov(c i) is saller), the chance of losing the targets is high when the caeras are controlled based on current observed locations of the targets. Since MDP has transition odel that predicts the next locations of the targets, it outperfors the other approaches when the targets are clustered and the resolution of the caeras is high. The and approaches perfor worse in alost all cases except when fov(c i) 16 (Figs. 6c, 6d, 7c, and 7d) where the approach perfors better than approach. This is due to the fact that the caeras are controlled independently of the targets inforation in both and approaches. This shows that the targets inforation (e.g., location, direction, etc) play a vital role in achieving high-quality surveillance. But, when fov(c i) 16, the approach perfors slightly better than because the chance of targets oving out of the fov is higher in approach if the velocity of the targets is v = 3 cells per tie step and the fov is reduced to fov(c i) 16. In all cases, the MDP outperfors the and approaches. When the nuber of states of each caera is increased fro C = 3 (Figs. 6c and 7c) to C = 5 (Figs. 6d and 7d), the perforance iproves because ore targets can be observed due to the additional caera states. The MDP-based approach perfors better than the other approaches even when the transition odel is inaccurate. This is tested by keeping the velocity of the targets oving at v = 3 cells per tie step and atching the transition probabilities coputed with velocities v = 2, 2.5, 3 cells per tie step (Figs. 6c, 6d, 7c, and 7d). The perforance of MDP coputed with inaccurate transition probabilities is still uch better than the other approaches. This is because the reward function is optiized with respect to the expected locations of the targets. When the nuber of caeras is increased fro n = 2, 3 to 4, the increase in perforance of MDP is uch better than the other approaches for < 10 targets and coparable to (if not better than) other approaches for > 10. This is because the prediction capability of our approach outperfors the other approaches with every addition of a new caera. The graph with increasing nuber of caeras is not shown here due to space liitation. Fro these observations, we find that our MDP-based approach perfors better than the other tested approaches in all the cases due to its prediction capability. Specifically, it outperfors approach when the velocity of the targets and the resolution of the caeras are high. 5.3 Real Experients We have conducted real experients with n = 3 AXIS 214 PTZ caeras to onitor up to = 6 Lego robots (targets) in an environent with the size of T l = 11 9 grid cells. The size of each grid cell is Each caera has C = 3 states. The states of the caeras are deterined such that all the cells of the environent can be observed at high resolution by at least one caera. Given any joint state C of the caeras, only a subset of cells in the environent can be observed by these caeras, i.e., fov(c) T l.

7 90 (a) (b) (c) MDP(v=2) MDP(v=2.5) (d) MDP(v=2) MDP(v=2.5) Figure 6: Graphs of P ercentobs vs. nuber of targets for corridor setup: (a) non-clustered targets with fov(c i) 25 cells, C = 3 and clustered targets with (b) fov(c i) 25 cells, C = 3, (c) fov(c i) 16, C = 3, (d) fov(c i) 16, C = (a) (b) (c) MDP(v=2) MDP(v=2.5) (d) MDP(v=2) MDP(v=2.5) Figure 7: Graphs of P ercentobs vs. nuber of targets for hall setup: (a) non-clustered targets with fov(c i) 25 cells, C = 3 and clustered targets with (b) fov(c i) 25 cells, C = 3, (c) fov(c i) 16, C = 3, (d) fov(c i) 16, C = 5. This akes the proble challenging for the active caeras to axiize the nuber of observed robots. We have a static caera that can track these robots based on OpenCV Cashift tracker. The static caera is calibrated using [17] to obtain the approxiate locations of the robots at every tie step. The direction and velocity of the robots are deterined based on their previous and current locations. The fov(c) is deterined by calibrating the active caeras in each of its state and deterining the grid cells of the environent in which the robots can be observed at a high resolution. We guarantee the resolution of the robots that are observed by the active caeras to be approxiately ore than pixels. We pre-coputed the transition probabilities of an individual target for all possible locations, directions, and velocities v = 1, 2 cells per tie step. The robots are oved based on the velocity-direction otion odel and are prograed to turn back or stop when they hit the wall or cross other robots. Each robot is initialized with a Cashift tracker in the static caera and is tracked to get its approxiate 3D location, direction, and velocity. We have tested our ipleentation up to = 6 robots but we keep one of the robots static. It can be observed that caeras 2 and 3 coordinate to observe the brown static robot (Fig. 8). Caera 2 pans to another state (see botto two rows of Fig. 8) only when caera 3 takes over the observation of the static target (see top two rows of Fig. 8). This static target can be replaced by a portion of the surveillance environent like the entrance/exit or reception where we need to pay ore attention. Table 2 shows the P ercentobs perforance for the real experients over τ = tie steps. Our proposed approach has soe liitations: (a) only when the observations fro static caeras are near-deterinistic (i.e., with the help of overhead static caeras), our proposed approach is expected to perfor well; (b) MDP observes its targets less well when their otion is ore uncertain. In our future work, we will include an observation odel to handle location uncertainty due to static caeras, which results in a Partially Observable Markov Decision Process fraework. We will also look into deploying active caeras with a tea of obile robots [10, 11] for tracking and surveillance of obile targets. Table 2: Perforance for real experients P ercentobs CONCLUSION This paper describes a novel decision-theoretic approach to control and coordinate ultiple active caeras for observing a nuber of oving targets in a surveillance syste. Specifically, it utilizes the Markov Decision Process fraework, which accounts for the stochasticity of targets otion via a probabilistic otion odel and addresses the trade-off by axiizing the expected nuber of observed targets with a guaranteed resolution via stochastic optiization. The conditional independence property, which is inherent in our surveillance proble, is exploited in the transition odel of the MDP to reduce the exponential policy coputation tie to linear tie. As shown in siulations, our approach can scale up to targets in real tie. We have also ipleented our proposed decision-theoretic approach using real AXIS 214 PTZ caeras to deonstrate its feasibility in real surveillance syste. 7. REFERENCES [1] AXIS 232D+ Network Doe Caera datasheet ( [2] J. Ai and A. A. Abouzeid. Coverage by directional sensors in randoly deployed wireless sensor networks. J. Cob. Opti., 11(1):21 41, [3] S. Banerjee, A. Chowdhury, and S. Ghosh. Video surveillance with PTZ caeras: The proble of axiizing effective onitoring tie. In K. Kant, S. V. Pearaju, and K. M. Sivalinga, editors, ICDCN 2010, volue 5935 of LNCS, pages Springer-Verlag, [4] A. N. Belbachir, editor. Sart Caeras. Springer, [5] C. Costello and I.-J. Wang. Surveillance caera coordination through distributed scheduling. In Proc. CDC, 2005.

8 at tie = 58sec at tie = sec at tie = 20sec caera 1 caera 2 caera 3 static caera Figure 8: Results of real experients: coluns 1 to 3 show the high-resolution iages of Lego robots captured by caeras 1, 2, and 3 while colun 4 shows the targets trajectories tracked by the static caera. [6] H. El-Alfy, D. Jacobs, and L. Davis. Assigning caeras to subjects in video surveillance systes. In Proc. ICRA, [7] B. P. Gerkey, R. T. Vaughan, and A. Howard. The Player/Stage project: Tools for ulti-robot and distributed sensor systes. In Proc. ICAR, pages , [8] C.-M. Huang and L.-C. Fu. Multitarget visual tracking based effective surveillance with cooperation of ultiple active caeras. IEEE Trans. t., Man, Cybern. B, 41(1): , [9] N. toever, T. Yu, S.-N. Li, K. Patwardhan, and P. Tu. Collaborative Real-Tie Control of Active Caeras in Large Scale Surveillance tes. In Proc. M2SFA2, [10] K. H. Low, W. K. Leow, and M. H. Ang, Jr. Task allocation via self-organizing swar coalitions in distributed obile sensor network. In Proc. AAAI, pages 28 33, [11] K. H. Low, W. K. Leow, and M. H. Ang, Jr. Autonoic obile sensor network with self-coordinated task allocation and execution. IEEE Trans. t., Man, Cybern. C, 36(3): , [12] V. P. Munishwar and N. B. Abu-Ghazaleh. Scalable target coverage in sart caera networks. In Proc. ICDSC, [13] F. Qureshi and D. Terzopoulos. Planning ahead for PTZ caera assignent and handoff. In Proc. ICDSC, [14] E. Soerlade and I. Reid. Probabilistic surveillance with ultiple active caeras. In Proc. ICRA, [15] C. Soto, B. Song, and A. K. Roy-Chowdhury. Distributed ulti-target tracking in a self-configuring caera network. In Proc. CVPR, pages , [16] M. T. J. Spaan and P. U. Lia. A decision-theoretic approach to dynaic sensor selection in caera networks. In Proc. ICAPS, pages , [17] R. Y. Tsai. An efficient and accurate caera calibration technique for 3D achine vision. In Proc. CVPR, [18] P. Viola and M. J. Jones. Robust real-tie face detection. IJCV, 57(2): , APPENDIX A. PROOFS Derivation of Equation 7 The value function V (6) is given by V (T, C, A) = R((T, C )) P (T T ) T T = R(t k, C ) P (t i t i) t 1 T,...,t T = R(t k, C ) P (t k t k ) t k T = R(t k, C ) P (t k t k ) t k T = Ṽ (t k, C ) i=1 T k T 1 i k P (t i t i) where T k = (t 1,..., t k 1, t k+1,..., t ). The second equality is obtained using (1) and (3). The fourth equality follows fro P (t i t i) = P (T k T k ) = 1. T 1 i k T k T 1 T k