Esimaion o complex moions in video signals Erhard Barh Insiue or Neuro- and Bioinormaics Universiy o Lübec, Germany Nonlinear analysis o mulidimensional signals: LOCOMOTOR LOcal adapive esimaion o COmplex MOTion and ORienaion paerns LOCOMOTOR Franur Heidelberg Im DFG Schwerpun 4: Mahemaical mehods or ime series analysis and digial image processing hp://www.mah.uni-bremen.de/zeem/dfg-schwerpun/ Projec Ba-76/7 Lübec
Inrinsic dimension in 3D i0d: consan in all direcions: id: consan in direcions: id: consan in one direcion: i3d: no consan direcion: (x, y,) = cons. (x, y,) = g(ξ) (x, y,) = g(ξ,ζ) ( x,y,) = g(ξ,ζ,τ) Inrinsic dimension id and i3d regions are he leas requen bu mos signiican. i0d id id i3d
The classical moion model (x,y,) image-sequence inensiy v = (v,v ) T moion vecor α (v) = v + v + = V = 0 x y r = ( v,v, ) Why complex moions? Basic problems wih naural image sequences: muliple moions, occlusions, noisy daa, eaures a dieren scales,... 3
Transparen moions Movie (op) and x plo (below). Moion model or ransparen moions Opical low or one moion (x,y,) image sequence, v = (v,v ) T moion vecor α (v) = v + v + = V = 0 x y wih he derivaive operaor: α (v) = v + v + x y Opical low or N moions: = g (x-v )+...+g N (x-v N ) α(v ) α(v ) α (v ) = 0 N 4
Mixed-moion parameers Example or wo moions u, v : (x,)=g (x-v )+g (x-u ) α(v) α(u) = u v + ( u v xx + u v + u v ) + ( u + v ) x yy xy + ( u + v ) We deine he mixed-moion parameers as: y + c = vu c = vu c = uv +uv xx yy xy c = u+v c = u+v c = x y Solving or he mixed-moion parameers Wih he mixed-moion parameers we obain: α(v) α (u) = c + c + c xx xx yy yy xy xy + c + c + c x x y y = V L= 0 For one moion we had V = 0 The above consrain can be used in a number o ways o derive he mixed moion parameers in V rom, e.g. by deining he generalized srucure ensor e.g. JV= N 0 and solving he sysem m i im i(m ) i J N = T h LL V (M, M,...,( ) M ) M ij, i =,...,m are he minors o J N 5
Separaion o he mixed-moion parameers We inerpre he moion vecors u and v as complex numbers (v = v +i v, u = u +i u ) and observe ha u v = A0 = cxx cyy + icxy u+ v= A = c + ic x y A 0 and A are homogeneous symmerical uncions o he coordinaes u and v and hereore by Viea s rule he coeicien o he complex polynomial Q(z) = (z u)(z v) = z Az + A0 ha has he complex roos u and v. In case o N moions Q(z) = z A z + + ( ) A N N N N 0 Fourier analysis o muliple ransparen moions Coninuous ime ( x, ) = g α( u) α( v) ( x u) + g = 0 F = G δ( u ω+ ω ) + G ( x v) δ( v ω+ ω ) Discree ime ( x) = g F ( x u) + g ( ω) = φ G ( ω) + φ ( x v) G ( ω) α (u ) = u + φ -ju ω = e 6
Bloc-maching or ransparen moions F ( ω) = φ G( ω) + φg ( ω) Layer eliminaion F a ( ) ( ) 0( ) ω a F ω + a F ω = 0 = φ +φ a = φ φ Bloc-maching consrain Bac o space domain x) ( x u) ( x v) + ( x u v) = 0 ( 0 Synheic resuls or bloc-maching SNR : 35 db Bloc size : 5 x 5 pixels 7
Fourier analysis o muliple occluded moions ( x, ) = χ( x u) g( x u) + χ( x u) g ( x v) g ( x) g ( x) occluding objec occluded objec χ characerisic uncion o he occluding objec χ = χ How does he Fourier ransorm o (x,) deviae rom he wo moion planes? α ( u ) α ( v ) ( x, )? Occluded moion α( u) α( v) ( x, ) = q( x, u, v, ) δ( B( x u)) q( x, u, v, ) = ( u v) N( x u) ( u v) g ( x v) Foreground moion Bacground moion Normal N o he border B(x)=0 The equaion is he same as or ransparen moions excep a he occluding boundary. q( x, u, v, ) is maximal i u - v is orhogonal o he boundary is null i u - v is angen o he boundary 8
Occluded moion in he Fourier domain F( ω, ω ) = A( ω) δ( u ω+ω ) + B( ω) δ( v ω+ω ) + C( ω, ω ) C( ω, ω ) d /( v ω+ω ) along he plane u ω+ω = c The disorion C has hyperbolic decay Log(+F) Log(+F) The hyperbolic decay o C has been irs recognized by Beauchemin a al. or he paricular case o sraigh line boundary Reerences C. Moa, I. Sue, T. Aach, and E. Barh: Spaio-emporal moion esimaion or ransparency and occlusions. Signal Processing: Image Communicaion. Elsevier Science, 005. C. Moa, I. Sue, T. Aach, and E. Barh: Divide-and-Conquer Sraegies or Esimaing Muliple Transparen Moions. In: Proceedings o he s Inernaional Worshop on Complex Moion, Schloss Reisensburg, Germany. Lecure Noes on Compuer Science, LNCS Vol. 347, 005. C. Moa, I. Sue, T. Aach, and E. Barh: Esimaion o muliple orienaions a corners and juncions. In: 6h Paern Recogniion Symposium (DAGM'04), Tübingen, 63-70, 004. C. Moa, T. Aach, I. Sue, and E. Barh: Esimaion o muliple orienaions in muli-dimensional signals. In: IEEE Inernaional Conerence on Image Processing, 665-668, 004. I. Sue, T. Aach, E. Barh, and C. Moa: Muliple-moion-esimaion by bloc-maching using MRF. Inernaional Journal o Compuer & Inormaion Science, Vol. 5, No., pp. 4-5, 004. I. Sue, T. Aach, E. Barh, and C. Moa: Muliple-moion esimaion by bloc-maching using Marov random ields. In: Inernaional Journal o Compuer & Inormaion Science, Vol. 5, No., pp. 4-5, June 004. E. Barh, I. Sue, T. Aach, and C. Moa: Spaio-Temporal Moion Esimaion or Transparency and Occlusions.In: Proceedings o IEEE Inernaional Conerence on Image Processing (ICIP 03), Barcelona, Spain, Sepember 4-7, 003, Vol. III, pp. 69-7. E. Barh, I. Sue, and C. Moa: Analysis o moion and curvaure in image sequences (invied paper). In: Proc. o he 5h IEEE Souhwes Symposium on Image Analysis and Inerpreaion, 06-0, 00. C. Moa, I. Sue, and E. Barh: Analyic Soluions or Muliple Moions. In: Proceedings o he IEEE Inernaional Conerence on Image Processing (ICIP 0), Thessalonii, Greece, Ocober 7-0, 00, Vol. II, pp. 97-90. 9