Video Geometry without Shape

Transcription

1 Video Geoety without Shape Retieving geoetical infoation in videos without any a pioi infoation about the iage stuctue o possible shapes: Taás Sziányi Registation of diffeent views, o io, o shadows though co-otion statistics MTA SZTAKI, Budapest Focus-ap though Bayesian iteations and a new eo etic. Stuctue fo conditional pobabilities Seaching fo statistical inteaction aong iage points. This statistical infoation is given by Conditional/Concuent Motion changes: caea egistation, vanishing point of io, shadow, hoizon Spatial coheence: elative focus depth; hee conditional pobabilities ae given by the light distibution via the iaging syste Lucy Richadson Bayesian iteation schea is used thee ties hee: Co-otion statistics fo coon points of two caeas Shadow odelling Focus depth though blind deconvolution 3 Co-otion: coelated otion in io and in steeo iage pai 4 The TASK fo Steeo Wide Baseline Video Registation Given two o oe views. Tack objects acoss diffeent views. Geneal schee. Backgound odeling.. Detection of featues. 3. Etaction of point-coespondences etaction of candidates, ejection of outlies. 4. Alignent of the caeas views. 5 6

2 The epipola geoety fo two caeas Concuently oving points fo atching Co-otion statistics Π M I I Motion statistics fo a given piel: Detected objects Motion ap Collected statistics e C e C i l k (R,ϕ l i k Co-otion statistics Collecting statistics of co-otions siultaneously fo two caeas/videos. 7 8 Motion, global otion statistics, co-otion statistics Etacting candidate point pais: Co-otion detection in local and eote views Inlies Outlie Caea Caea Caea Oveall otion statistics fo Caea Reote otion Statistics fo the othe caea consideing a point 9 Local otion co-otion statistics fo an iage point Local aius on the ceated statistical iages Resolution is Coposite view afte optiized alignen by using RANSAC Ehaustive seach ethod needs huge aounts of eoy fo stoing co-otion statistics fo each piel

3 Pobabilistic odel without shapes o tacks Change histoies Continuous otion Motion with significant sudden changes Noise flickeings (video noise The distibution of entopy fo noise flickeings (boes and deteinistic otions (cicles in eal life videos. - The line shows a selected theshold at which the popotion of potential outlies is 5%. - Estiated Gaussian distibutions Saples fo input videos and Results of the entopy based peselection of featue points 5 6 Paaetes of entopy distibution fo diffeent test videos. Last colun shows the popotion of noise flickeings aong piels of detected changes if the theshold value is 0.. Pobabilistic inteaction aong points of diffeent views fo otion / no-otion functions i k = T b i ( t b k ( t t= b ( t t= k T i k = j k k i j i j 7 8 3

4 Egodic egula Makov chain has a unique stationay distibution ( p p = ( p p Π Bayesian iteations of Egodic egula Makov chain with a unique stationay distibution i + = i k j k k i j k j k + = k i j i i k j i j 9 0 Saple point pais obtained by Bayesian iteations. The nealy coesponding points ae nubeed with the sae nube. Shot iteation length fo the eceeding point sets Afte fou of the double iteation steps the algoith is stopped and those featue-points ae selected fo which i and k ae geate than /N and /N. Iages show the esulting point sets of the featue etaction. OFV OFV Saples fo input videos. Results of the entopy based peselection of featue points. The change of the elative ipact (in % of featue points within the estiated OFV aeas elative to the whole iage befoe and afte Bayesian iteation. The popotion of estiated OFV points to the eal OFV points afte coelation based selection and Bayesian iteation. Result of Bayesian estiation of OFV

5 Reduction of ROI afte featue etaction steps Co-otion statistics 5 GELLÉRT video Sae caeas Sae zoo 60 0 esolution, 0 fps 6 Gounplane fitting of two views Final alignent of two views gound planes fo PETS Indoo gound planes: Alignent of two views detecting concuently oving shadows Rando otion fo egistation

6 Saple point-pais with coesponding epipola lines obtained fo ando otion videos Nueical esults of odel fitting fo diffeent epeients 3 3 Shadow detection with an iteation schee Results of iteative shadow pocess Based on this foula the following iteation schee can be witten (fo shadow piels: P ( j hi P ( j Pk + ( hi = Pk ( hi, i, j, k S j P ( j hk P ( hk k The key issue in the foula is the deteination of h conditional pobability. Accoding to the geoetical odel the coputation can be suaized as follows: Thee is unifo initialization value along the line. The j and k indices deonstate the cycles only. Advantages: well defined foulas fleible fo futhe paaetes can handle geoetical infoation Disadvantages: heavy coputation tie thee ae pobleatic situations on-line estiation of light diection is needed 33 It is a copleentay ethod with othes. 34 Indoo saple video Detection of huan walking

7 Non-ovelapping views detecting walkes leg Alignent non-ovelapping views Entance Aula Moe about Co-otion egistation fo two views Z. Szlávik, T. Sziányi, L. Havasi: Stochastic view egistation of ovelapping caeas based on abitay otion, IEEE T. Iage Pocessing, Mach, 007 Z. Szlávik, T. Sziányi, L. Havasi: Video caea egistation using accuulated co-otion aps, ISPRS J Photogaety and Reote Sensing, Januay, 007 L. Havasi, Z. Szlávik, T. Sziányi: Detection of Gait Chaacteistics fo Scene Registation in Video Suveillance Syste, IEEE T. Iage Pocessing, Febuay, 007 Vanishing Point in case of Mio: Oiginal object and its eflection ae pesented Havasi, L., Sziányi, T.: Estiation of Vanishing Point in Caea-Mio Scenes Using Video, Optics Lettes ( Use of Co-otion statistics fo vanishing point estiation in caea-io scenes and in case of cast shadow A eflective suface (e.g. a io, denoted by Ω, which lies in the -y plane (ight-handed syste. C denotes the caea cente, and the iage plane is denoted by Π (3-D points ae apped to this plane via cental pojection. Vanishing point descibes a skew syetic fundaental ati between two views: Ω C X y.. α z β c X Π C 4 Fundaental constaint The fundaental ati coesponds to the oiginal iage and the vitual iage in a caea-io scene. Consequently, F has degees of feedo and is identified with the VP. ( %( %( % F% = % c F % = %, c F % = 0 T T F 0 c = 0 c c c 0 4 7

8 Co-otion statistics in caea-io scenes P u w P u + w P u ( ( ( co nea coll = w Ν u,, Σ, whee w + w = i = and ( μ i i i μ < μ Co-otion statistics and its Gaussian itue odel estiation In case of a visible eflective suface two peaks ae pobable, thus the pdf is odeled with a siple Gaussian itue odel with two coponents: Coesponding point pais Depending on the scene configuation, not evey oving point will have a visible eflection. Rejection of outlies fo the Shop sequence. Only the diections coesponding to the ain peak (ode of the histoga (deteined fo the line diections will be used fo late coputations. a befoe ejection, c afte ejection; b and d show the coesponding histogas of angles The deteination of VP is caied out by using an objective function Goodness-of-fit function epesented using a contou gaph; the VP is aked with point δ ( u, = aga Pcoll ( v and μ% nea uv %,% = 0 v S VP = c = ag a Pg( Pcoll( δ (, u u S a u δ ( a, u VP Ν (, 3 u μ, Σ 3 a 3 a a 3 Epeiental esults on VP estiation The esults deonstate the collineaity constaint a Ν u, μ, Σ ( Ν a (, u μ, Σ

9 Co-otion suay We have shown that caeas can be egisteed in seveal athe iseable conditions, based on: Unpedictable otion without stuctued backgound o defined object shapes o Shadows of undefined stuctues in font of flickeing backgound. Detection of Vanishing Pont fo abitay otion in case of io o shadow Relative Focus Aea Etaction by Blind Deconvolution fo Defining Regions of Inteest It can joint to detection and seach fo video events of ulticaea systes Focus estiation with deconvolution Focus Aea Etaction by Blind Deconvolution fo Defining Regions of Inteest Blind deconvolution: Given obsevation g, give an estiation of the oiginal iage f and the bluing function (PSF h : g = f * h Stating fo Richadson s oiginal foula based on Bayesians: 5 5 Blind deconvolution with Lucy - Richadson double ieations The double iteation We ceate a localized double iteation schee fo locally vaying f and PSF estiation ( location vecto:

10 Autoatic el. focus ap etaction Fo the classic Richadson iteative blind deconvolution foula g obseved h PSF f unknown oiginal f and the PSF vay locally accoding to the aount of blu (distotion pesent on the iage locally Stop the double iteation at a finite step (hee #5 and check the eo between the easued and the estiated blued iage blocks: g g k Is MSE usable fo copaison the BD esidual eos of diffeent blocks? Constaints and ill-posedness In the local deconvolution we conside only a few constaints syeticity, non-negativity, zeo phase. and nothing about the iage content egulaization (e.g. edges. Localized deconvolution uns on sall blocks, ange of the PSF. Thus the ill-posed iteation pocess tends to be noisy. Fo the classification we stop at a low iteation count and we need a stable eo easue which gives diffeent values fo diffeently focused aeas, and which is not uch affected by the pocess s noisy natue. ADE : angle deviation eo Othogonality citeion: signal and noise ae independent ac sin g, g - g g.g - g k k In case of g - g k = [ +, -, -, +, -, , + ] g = [0, 0, 0, 0, 0, , 0] g - g k is high, while 57 < g, g - g k > zeo 58 Eo cuves fo 8 neighboing blocks (each cuve stands fo one block on a blued tetue saple (top fo the sae blu with ADE (left, and MSE (ight. Ideally, cuves of the sae easue should eain close to each othe. The eo function Localised blind deconvolution fo focus ap estiation: un local deconvolution with a low iteation count calculate local esidual eos, with contast weighting use the local esiduals fo elative classification of aeas

11 6 6 Find iages with siila elative focused objects: Follow the path of elative focus changes: L. Kovács, T. Sziányi: Iage / Video indeing atching saple iage o seantic desciption Iage Indeing by Focus Map

12 Painting detail vaiation based on etacted elative focus aps Application in Non Photoealistic Rendeing A deostation on A. Licsá, T. Sziányi: Use-adaptive hand gestue ecognition syste with inteactive taining, Iage and Vision Coputing, 005 and ACM MM WS, 006 Oct Tillao: an AJAX Based Folk Song Seach and Retieval Syste with Gestue Inteface Based on Kodály Hand Signs Thank you fo you attention! Distibuted Events Analysis (DEVA Reseach Goup sztaki.hu/depatent/eee/ Distibuted Events Analysis (DEVA Reseach Goup 7 7

13 Results fo diffeent scales Constaints: In the space doain: size of PSF (egion of suppot constaint: spuious eleents of h k outside the initial egion of suppot will be zeoed duing the iteation; piel aplitude bound 0<=f k <=55; non-negativity fo the PSF (h k and the iage (f k. In the fequency doain: zeo phase of h k, so as not to induce phase distotion when filteing the iage f k Rel. focus ap etaction - copaison on tetues blued ous in ous AC edge Apps: featue etaction Find iages with siila elative focused objects: Follow the path of elative focus changes: 77 3