GEOMETRIC AND STOCHASTIC ERROR MINIMISATION IN MOTION TRACKING. Karteek Alahari, Sujit Kuthirummal, C. V. Jawahar, P. J. Narayanan

Size: px

Start display at page:

Download "GEOMETRIC AND STOCHASTIC ERROR MINIMISATION IN MOTION TRACKING. Karteek Alahari, Sujit Kuthirummal, C. V. Jawahar, P. J. Narayanan"

Angel Ray
5 years ago
Views:

1 GEOMETRIC AND STOCHASTIC ERROR MINIMISATION IN MOTION TRACKING Karteek Alahari, Sujit Kuthirummal, C. V. Jawahar, P. J. Narayanan Centre for Visual Information Tehnology International Institute of Information Tehnology Gahibowli, Hyeraba INDIA. ABSTRACT Traking has been onsiere to be a single view problem onventionally, where one is intereste in the projetions of a partiular objet in a view over time. Even if many views of the same event are available, traking often proees inepenently in eah view. The geometri information ue to the projetion of the same objet onto multiple image planes is not utilize. In this paper, we ouple the stohasti error use by the Kalman filter with a geometri error term erive from multiview geometri onstraints to ahieve improve traking in iniviual views. We present experimental results to evaluate the performane of the algorithm. 1. INTRODUCTION Visual traking has been an ative area of researh in the Computer Vision ommunity over the past eaes. Due to its pratial appliations, traking has gaine importane in both Computer Vision an Computer Graphis [1. Popular methos of traking inlue the Kalman filter [2, the Conensation algorithm [3, et. The Conensation algorithm uses a fatore sampling approah to propagate the probability istribution of the parameters that are to be trake over time. Kalman ilter has been use not only for traking, but also for struture from motion omputations[4 ue to its onvenient form for on-line real time proessing. It is a set of mathematial equations that implement a preitororretor type estimator, an minimizes the error ovariane uner ertain assumptions. Kalman ilter is shown to be optimal in the linear Gaussian environment. It is popular for traking robots [5, raars [1, an for many other pratial appliations. The onensation algorithm, though appliable to non-linear ensities, requires the prior ensity to be preetermine. The problem of traking has inherent hallenges assoiate with it, most important one being olusion of the feature point. This has been taken are of in various ways using the Kalman filter [6 in single views. Due to various reasons errors get introue into the proess of measuring the observations. Hene, it is esirable for the traker to be robust enough to hanle this noise. Kalman filter assumes the noise in the measurement moel to be Gaussian. In the presene of olusion or noise, aitional information from other viewing angles will be of immense help. Traking, whih has been traitionally assume to be a single view problem, has not taken avantage of the reent evelopments in multiview analysis. The algebrai relations among the projetions of a point onto multiple ameras have been stuie extensively in projetive, affine an Euliean frameworks [7. It has been reognise that aitional views an provie information whih helps to solve the problem of extrating meaning an struture from images of the sene. Using a plurality of projetions, the lost information about the thir imension an be reovere. Appliations of the multiview onstraints is not limite to the reonstrution of the thir imension from a set of projetions. They have been applie to view generation, objet reognition, vieo stabilization, et. [7. In the past few years, multiple view traking has gaine onsierable importane [8,, 10. In spite of its importane, the geometri information available in multiple views has selom been use [11. Traking has been performe on ifferent views inepenently without utilizing their interrelationships. We present a solution to the traking problem by moifying the stanar Kalman filter formulation in this paper. Base on the multiview relations among ifferent views of the same sene, a new error term alle the geometri error is use in the traking proess. The stohasti error minimise in the onventional filter has been ouple with this geometri error to get better traking. Traking proees by estimating the state (position) of the objet at eah time instant. The estimate obtaine at eah instant is orrete aoring to the error at that instant. This ynami orretion of the error makes traking real-time an helps ahieving faster results. Due to the inorporation of the multiview information at eah stage, there is more than one measurement with whih we orret our estimate. We present the formulation for two views in this paper; it an

2 + = m + G be extene to any number of views easily. In setion 2 we present the problem formulation for twoview traking. The stanar Kalman filter is briefly introue along with the notations use, motion moels, et. The unerlying assumptions are also isusse. Setion 3 eals with Kalman iltering in two views. Traking in image spae is isusse in Setion 4 along with a isussion on olusion an generalizations. Numerial results to valiate the laims are presente in Setion 5. Setion 6 presents the onlusions. 2. TRACKING WITH GEOMETRIC CONSTRAINTS The Kalman filter estimates the state in a two-stage manner by using feebak ontrol. Base on the feebak obtaine through the noisy measurements it upates the estimates of the state. It is a preitor-orretor algorithm where the preition is one by the time upate equations an the orretion is one by the measurement upate equations [12. The time upate equations are responsible for obtaining the state an error ovariane for the next time step. The measurement upate equations are responsible for upating the estimates using the noisy measurements. The Kalman filter algorithm is summarize below. 1. Initialize estimates of state an error ovariane. 2. Preit the next state an the error ovariane base on the best estimate till the present time instant. 3. Take the measurement at that time instant. 4. Corret the state an error ovariane estimates base on the measurement mae. 5. Repeat steps 2-4 for the next time instant. Now we isuss the multiview traking problem using the Kalman filter base approah. Let be the 3D position of a point at time instant. Let the point move as per a linear motion moel as! (1) represents the proess noise that is assume to be inepenent, with normal probability istribution "$#&%('*) is a matrix whih efines the motion #-,/.102'. of the point. Mathematially, any amera an be represente as a 7843 matrix, known as the amera projetion matrix [7. The relation between a worl point being image an the amera matrix is given by ; < >= / (2) where ; BA enotes the image point in view C at time represente in the homogeneous notation [7 an enotes the 74D3 amera matrix for view C. We assume ; the projetion moel to be affine [7 wherein the Gaussian noise in the worl remains Gaussian in the image spae. E represents the noisy measurement of the projetion at time, ; is the orrete estimate of the projetion at time, while &; is the preite estimate of the projetion at time. We onstrut a measurement vetor by onatenating the measurements in eah view, G H JA K@L8A L. G E is the noisy measurement, the best estimate is enote by G, while the preite estimate is enote by. is the a priori estimate of the loation of the worl point at time given knowlege of the proess till time NM, an O is the a posteriori estimate of the loation of the worl point at time given measurement G E. The measurement moel is given by G E PQ JR (3) where PS UT = =ILWV is a X24Y3 matrix. The measurement noise R is assume to be inepenent, with normal probability istribution" # A ') #-,/.[ZN'. In the onventional Kalman filter, the a priori estimate error \ is efine as the ifferene between the a priori estimate of the worl loation, an the atual worl loation /. Similarly, the a posteriori estimate error \ is efine as ifferene between the a posteriori estimate of the worl loation, an the atual worl loation. The stohasti error in measurements is haraterize by these errors. The a priori estimate error ovariane is the expetation of the outer prout of the a priori estimate error \ with itself given by >^ \ \. (4) an the a posteriori estimate error ovariane is the expetation of the outer prout of the a posteriori estimate error \ with itself given by _^ \ \ a` (5) Kalman filter minimizes the a posteriori estimate error ovariane given by equation Geometri Error To inlue geometri information into the traker, we efine a geometri error in aition to the stohasti error employe in the onventional Kalman filter. Let be ; f1.ag.1h an fji g be the 7!4N7 funamental matrix [7 between view f an view g. A row of zeros is ae to the funamental matries to get 3D4k7 matries le T b8e, V. The a priori geometri error m #nl o ; Mpl e is efine as ; ' #nl [ ; Mpl [ ; ' ` (6)

3 \ Ž The term #-lqor ; Mlqe ; ' represents the geometri error in view g an the term #-l[s ; M6l[ ; ' represents the geometri error in view f. This error in eah view an be visualize as the vetor ifferene between the atual an the preite epipolar lines [7. It is shown pitorially in igure 1. The a posteriori geometri error m is also efine on similar lines. or simpliity, we use two views of a sene in the formulation. It an be extene to more views by onsiering geometri errors orresponing to all the views. t u<vxwyvqz/vr{} ~ t vƒ v {} ~ t vƒ v {} ~ ig. 1. The ark lines in blak iniate the atual epipolar line, the ashe lines iniate the preite epipolar line. The ifferene between these lines is enote as the geometri error for that view. In the ual spae, ue to the uality of lines an points, the geometri error an be interprete as the istane between two points, one of whih iniates the atual value an the other iniates the preite value of the epipolar line. 3. KALMAN ILTERING IN TWO VIEWS The error in the measurements has two omponents stohasti an geometri. We an ombine these errors to efine a new formulation for the Kalman filter. We first erive a set of preitor-orretor equations aime at estimating the worl loation of a 3D point from the measurements in two views. Then, we show that this an be easily onverte to a form that results in a simple set of preitororretor equations for the loation of the projetions of the worl features in the two views without omputing the worl loations. The a priori geometri error m an the stohasti error M /q are ombine. With equations 2 an 6, we simplify the new a priori estimate error as M /q m # l L = L l L = '# M / ' # M ' (7) where, is a 3 4D3 ientity matrix an ˆ # l Lx=IL l L = '. This matrix embes the geometri information obtaine from the two views. Similarly, the a posteriori error is given by \ > # / M / ' (8) estimate an the measurement G E Given the a priori we woul like to orret our preition to get an a posteriori estimate. This orretion is one by the linear ombination of the a priori estimate an a weighte ifferene between the measurement G E an the preite measure- as ment P /< Š # E G M P ' ` () The 3<4 X matrix, the gain or blening fator minimizes the a posteriori error ovariane, i.e. we hoose a gain fator suh that the a posteriori error ovariane is minimize, as a onsequene of whih our estimate woul oinie with the atual value. Using equation 7 along with the a priori error ovariane in equation 4, we get ^ # M '# NŒ where Œ _^ # Intial estmates of v 6 v v 1. Projet the state 2. Projet the error ovariane B v_ 6 < v ~ ~ ~ J < j ~ M ' M '# M ', implying Œ!< > (10) an v PREDICT CORRECT v 6 v ~š ~ &œ š B vj ~ š ~ 1. Compute the Kalman Gain v v! vž 1Ÿ vk š v v v v š v6 k vqœ ~v ~ B v ~ š ~ ~v ~ J vqš B v( ~ š ~ ~v ~ 2. Upate the estmate wth measurement 3. Upate the error ovariane ig. 2. Time upate an measurement upate equations for Two View Kalman ilter. The loation of the worl point is preite by making use of the knowlege of the motion moel (Equation 1) as > /q ž` (11)

4 M G G M G When this preition equation (Equation 11) an the motion moel equation (Equation 1) are use, the a priori error ovariane (Equation 4) an be expresse as ^ # M 'x# M ' 2^ # $q M q M ' # $q M $q M ' By grouping the terms appropriately we get 2^ # q M q '# q M ' 0 D 0 (12) This simplifiation has been one to eliminate the parameters relate to the worl spae. It provies relation between the a priori error ovariane at time instant an the a posteriori error ovariane at time instant M. A similar simplifiation for the a posteriori error ovariane an be performe by manipulating equations 5, 8 an as ^K \ \ # Œ! M PYŒ Z M Œ! P < PYŒ! P ' M PY Z P PY P (13) We want to hoose the gain fator, k, that minimizes the trae of (Equation 13), whih is equivalent to minimizing sum of square errors. Thus, the stohasti an geometri errors are minimize together. or this we ifferentiate the trae with respet to an set it to zero, whih yiels the require as < > P #nz PY P ' (14) The proess of traking a worl point using the above formulation is state here. We begin by assigning values to the initial estimates. Then, the next state an the error ovariane are preite by using equations 11 an 12. We then orret our estimates using equations, 13 an 14. The various steps involve in using the two view Kalman traker have been summarize in igure TRACKING IN IMAGE SPACE In the previous setion, we outline a mehanism for utilizing the Kalman filter for preiting the 3D loation of a worl point unergoing linear motion by moeling stohasti an geometri errors. Preition an also be one in the image spae without expliitly taking measurements in the worl spae. This type of inepenene between the worl an image spaes gives an ae avantage. It eliminates the iffiult proess of measuring 3D worl loations [8. We show how the time an measurement upate equations for the worl point an be transforme to arrive at time an measurement upate equations for the projetions of the worl point. To ahieve this we take projetions of the a priori ( P ) an a posteriori ( P ) estimates of the loation of the worl point given by equations in igure 2 using the projetion equation 2. The error ovariane an the gain fator equations remain the same. The equations for the image spae are Time Upate Equations PY2P G q (15) 2 q 0 Measurement Upate Equations G P # G E M ' (16) P #nz PY P ' M Q PY Q Z P Q PY P Disussion We formulate the traking problem for an affine moel, whih is linear in nature. Extensions for the projetive moel an be worke out using a non-linear filtering mehanism. Traking of multiple objets is another possible extension. These are out of sope of this paper. Olusion an be hanle as follows. Consier a situation where we have three views.1h an 7 of the objet in known ameras. The pair-wise funamental matries l L, l L[ an l (refer setion 2) between these views is also known. The objet is trake in all the three views simultaneously. When it is olue in only one of the views, say view 2, for some uration, measurements in that view annot be obtaine. Sine the objet is seen in the other views, we an use the orresponing trakers to preit the loation of the objet in them. rom the two preite loations ; an ; ) we an fin the epipolar lines L ; ;. The point of intersetion of these epipolar lines in those views ( orresponing to these points in the thir view as l an l }L gives the preite loation of the objet in the thir view. This an be use as the measurements when the point is olue. 5. RESULTS We teste the Kalman filter base traker using geometri an stohasti errors by performing various experiments on syntheti as well as image ata. The performane of the one-view an two-view trakers is ompare in the rest of

the setion. The two-view traker performs better in terms of onvergene, auray, robustness to noise. 5.1. aster onvergene Kalman filter works by minimizing the trae of the a posteriori error ovariane.

igure 3 shows the value of the trae in one-view an two-view ases for the first 25 iterations of the algorithm.

5 the setion. The two-view traker performs better in terms of onvergene, auray, robustness to noise aster onvergene Kalman filter works by minimizing the trae of the a posteriori error ovariane. This gives a goo measure of the error in preition. Ieally, this error shoul approah zero in a few iterations. igure 3 shows the value of the trae in one-view an two-view ases for the first 25 iterations of the algorithm. As an be seen from the graph, the error in two-view ase approahes zero at a faster rate than it oes in the one-view ase. Real image ata was use for this (a) (b) ig. 4. Observing an event in two ameras (Courtesy - Kek Laboratory, University of Marylan, College Park) Error One view error Two view error rame ig. 3. Graph showing the eay of trae of a posteriori error ovariane in the ase of one-view an two-view Kalman filters. experiment. It onsiste of 25 frames eah of two views of an event, shown in igure 4. The left han finger tip of the person shown was trake using the one-view an two-view Kalman trakers (Setions 2 an 3). We alulate trae of the a posteriori error ovariane matries for both the trakers, normalize it by their orresponing initial trae values an plot it against the frame number. The error ereases at a faster rate in the two-view Kalman traker. This is ue to the fat that two measurements obtaine from the two views are being use to upate the a priori estimate that we have at eah time instant. This in a way orrets the error at a faster rate ompare to the stanar formulation. Similar results were observe for syntheti ata. A 3D moel of a ar moving in a linear fashion was use. The motion of the ar was apture using two affine ameras. Noisy measurements of the enter of mass of the ar were use to trak it Robustness to noise The aitional information we have in terms of two views of the same sene an be use to make the Kalman filter more robust to noise in aition to making it onverge faster. The robustness of the two-view Kalman filter has been teste with ifferent values of measurement an proess noise. Syntheti ata similar to that use in the above experiment was use so to provie flexibility in hanging the noise values. We alulate the trae of the a posteriori error ovariane matrix for ifferent pairs of measurement an proess noise. The resulte are shown in Table 1. The noise values are iniate by the ovarianes of the error istribution funtion as 0 «ª an Z ª L. The trakers exhibit similar behaviour even in the ase when ifferent ovariane values are use for eah imension. ª ª L Measurement Proess One-view Two-view Noise Noise Trae( ) Trae( ) Table 1. Comparison of the one-view an two-view error values for ifferent measurement an proess noise values after 25 iterations. rom Table 1 it an be observe that two-view Kalman filter proues lower error values ompare to the one-view ase, thereby onfirming our laim that two-view traking is more robust to noise. rame One-view Two-view Number Distane Distane Table 2. RMS errors in 1-view an 2-view formulations

6 5.3. Better auray It is important to know how aurately the objet is being trake. We quantifie the auray of the traker as the root mean square of the istanes (RMS istane) of all the preite points from their orresponing atual non-noisy measurements. Proess an measurement noise are inherent in real ata an hene annot be eliminate by any simple proess. Thus, to obtain the non-noisy measurements we generate the ata synthetially onforming to the equations in the previous setions. (Setions 2, 3). or this experiment, we performe traking using both the one-view an two-view formulations an measure the orresponing auray values by fining the RMS istane. Ieally, this istane shoul be zero for a ompletely aurate traker. The results are shown in Table Olusion hanling To stuy the effet of olusion, we onsiere three views of a sene along with their pair-wise funamental matries. After introuing olusion in one of the views, no measurement is taken in that view. Performane of the traking algorithm is given in the Table 3. In the one-view ase, no orretion term is applie uring the olue perio. In the three-view ase, the non-olue views preit the loation in this thir view using epipolar lines. The istane between the preition an the groun truth when the objet is visible again is shown in olumns 2 an 3 of the table for the one-view an three-view ases respetively. We an see that the two-view Kalman filter results are superior for hanling olusion. 6. CONCLUSIONS We inorporate the aitional information that is available from multiple views of a sene, in terms of geometri onstraints on the error funtion, into the Kalman filter framework in this paper. The oupling of stohasti an geometri errors using two views le to better traking. This formulation an be extene to multiple views by moifying the geometri error appropriately. The two-view base traker efine in this paper an be use for traking objets moving in a linear fashion. urther work is being one to ex- Olusion One-view Three-view uration Error Error Table 3. Performane of the traker uner olusions ten the traker for any general motion an any non-linear projetion. The possibility of inorporating other multiview onstraints suh as multilinear tensors in plae of funamental matrix is also being explore. 7. REERENCES [1 K. V. Ramahanra, Kalman iltering Tehniques for Raar Traking, SiTeh Publishing In., USA, [2 R. E. Kalman, A new approah to linear filtering an preition problems, Transations of the ASME - Journal of Basi Engineering, vol. 82(Series D), pp , 160. [3 M. Isar an A. Blake, Conensation - onitional ensity propagation for visual traking, International Journal of Computer Vision, vol. 28, no. 1, pp. 5 28, 18. [4 A. Azarbayejani an A. Pentlan, Reursive estimation of motion, struture, an foal length, IEEE Trans. PAMI, vol. 17, no. 6, pp , June 15. [5 A. o, A. Howar, an M. J. Matari, Laser-base people traking, Proeeings of the IEEE International Conferene on Robotis an Automation, pp , [6 L. Marenaro, M. errari, L. Marbesotti, an C. S. Regazzoni, Multiple objet traking uner heavy olusions by using kalman filters base on shape mathing, ICIP, vol. 3, pp , [7 R. Hartley an A. Zisserman, Multiple View Geometry in Computer Vision, Cambrige Univ. Press, [8 Y. Li, A. Hilton, an J. Illingworth, Towars reliable real-time multiview traking, IEEE Workshop on Multi-Objet Traking (WOMOT 01), pp , [ S. L. Dokstaer an A. Murat Tekalp, Multiple amera fusion for multi-objet traking, IEEE Workshop on Multi-Objet Traking (WOMOT 01), pp , [10 Q. Delamarre an O. augeras, 3 artiulate moels an multiview traking with physial fores, Computer Vision an Image Unerstaning, vol. 81, no. 3, pp , [11 R. Vial, Y. Ma, S. Hsu, an S. Sastry, Optimal motion estimation from multiview normalize epipolar onstraint, ICCV, vol. 1, pp , [12 G. Welh an G. Bishop, An Introution to the Kalman ilter, SIGGRAPH - Course 8, 2001.

Optimal Distributed Estimation Fusion with Transformed Data

Optimal Distributed Estimation Fusion with Transformed Data Optimal Distribute Estimation Fusion with Transforme Data Zhansheng Duan X. Rong Li Department of Eletrial Engineering University of New Orleans New Orleans LA 70148 U.S.A. Email: {zuanxli@uno.eu Abstrat