Lecture Topics VMSC Prof. Dr.-Ing. habil. Hermann Lödding Prof. Dr.-Ing. Wolfgang Hintze. PD Dr.-Ing. habil.

Transcription

1 Lecture Topcs 1. Introducton 2. Sensor Gudes Robots / Machnes 3. Motvaton Model Calbraton 4. 3D Vdeo Metrc (Geometrcal Camera Model) 5. Grey Level Pcture Processng for Poston Measurement 6. Lght and Percepton as well as Blac-and-Whte- and Colour Pctures 7. Model Calbraton 8. Vdeo Metrc Sensor Calbraton (Geometrcal Camera Model) 9. Vdeo Metrc Camera Model (Fourer 2D) VMSC

2 Calbraton LDT Camera Model VMSC

3 LDT Camera Calbraton I Known: Measured: Searched: Measure ponts (x,y,z ) t, = {1, 2,..., I}, I 6 n the world frame. Correspondng pxel coordnates (m,n ) t of the I measurng ponts LDT Parameter t VMSC

4 LDT Camera Calbraton II Approach: F HG m n I KJ F G H u v w I F J K G H m w b n w b w I J K edt j F G GG H x y z 1 I J JJ K b m b t x t y t z t t x t y t z t n b b t x t y t z t g t x t y t z t g VMSC

5 LDT Camera Calbraton III Separate LDT parameters FF HG HG x y z x y z x y z 1 x y z b g t t t t t t t t t t t t t b 34 FF HG H G m n II KJ KJ II KJ KJ F t LDT v VMSC

6 LDT Camera Calbraton IV The soluton s unambguously up to a scale factor V t 34 b and over the pseudo nverse matrx we determnes c h 1 t t tldt F F F v the parameter set. We receves numercally more stable results, as the pseudo nverse s computed over the SVD concept. VMSC

7 Vdeo Metrc 3D Measurement wth Several Camera Systems VMSC

8 Optcal Segmentaton Camera Retro-reflected ball Infrared slash lght Infrared optcal pass Retroreflecton ball Optcal segmented marer (Maxmzaton contrast envronment marer) Actuator Correspondng marer VMSC

9 3D Vdeo Metrc Measurng Procedure Correspondng marer must be on a lne (area around the lne) Grey level pcture data Search correspondng marer Eppolare Measure 2D Postons n the RAM Quadratc search Fast lnear search Inverse camera model Soluton 3D measurng equaton 3D Sensor coordnate Ideal undstorted sensor coordnate S R VMSC

10 Webcam Color Pcture pxel Lens Dstortons VMSC

11 Center of Area I g(x,y) x y s s The center of the grey level weghted area defnes the sub pxel poston of an ellptcal grey level marer pcture y x VMSC

12 Center of Area II Optcal low pass flterng g(x,y) Contnuous x g( x, y) x da s 1 A ys A g( x, y) y da A Dscrete M N ms 1 m g( m, n) n s MN m 1 n 1 n M N VMSC

13 Mult Camera Systems I u m t s ns v t Man Pont Scale Factor Sensor Orentaton Error Lens Dstorton VMSC th camera r S u Correspondng center of area M N ms 1 m g ( m, n) n s M N m n n v Inverse Camera Model t ms u n s v Correspondng sensor coordnates of the pnhole model u v t

14 Straght Lne n Space g r 1 r 2 3 g : r g0 g, g0, g, Two Pont Formula 3 g : r r1 ( r2 r1 ), r1, r2, VMSC

15 Intersecton of Straght Lnes n Space I Mult Lnes 3 g : r g0 g, g0, g,, {1,..., K} r S r g g g g S 0 0, VMSC

16 Intersecton of Straght Lnes n Space II g g g g x 0 x x 0 x g g g g y 0 y y 0 y g g g g z 0 z z 0 z gx g x gx0 g x 0 g y g y g y0 g y 0 gz g z gz 0 g z 0 g g g g 0 0 VMSC

17 Intersecton of Straght Lnes n Space III r S r g g g g S 0 0, rs g0 g, { } wth t 1 t g and g g Average r r r 2 g g g 0 0 VMSC

18 Intersecton of Straght Lnes n Space IV The pseudo nverse solves the least square problem. The predcted poston s at the mnmum dstance between the lnes. In the context to the systematc and random errors mostly there exst no ntersecton pont. r Do not use the average Use t only for measurement. for model calbraton. VMSC

19 Mult Camera Systems II u, v are the deal sensor coordnates (pnhole camera wth deal sensor placement) T j R y Sj v j u j x Sj Seeng Beams T y S R v u y R x R x S z S z R P S Objet Object Seeng beam goes through (0, 0, 0) t and (u, v, -b ) t Intersecton of the seeng beams defned the 3D coordnate VMSC

20 Mult Camera Systems III -th camera n -th camera frame = {1,...,K} Straght lne equaton n space (Two Ponts Formula) b g r ( ) r1 r2 r1, { 1,, K} Camera Seeng Beam r u v b t, {1,, K} Transformaton to the reference frame R t R r DR u v b t, {1,, K} VMSC

21 Mult Camera Systems IV Intersecton Equaton u u j R DR v DR v t j t j b b j R for all pars (, j) wthout repetton t R 3D coordnate R t R r DR u v b t, {1,, K} c h t 1 t R t We receves numercally more stable results, as the pseudo nverse s computed over the SVD concept. VMSC

22 Mult Camera Systems V Averagng r R H r (, j) for all pars wthout repetton R K 2 r R j VMSC

23 Vdeo Metrc 3D measurement LDT camera model VMSC

24 3D Measurement wth Mult-camera Arrangement I Known: Measured: Searched: LDT Parameter t j, = {1, 2,..., K} of the K 2 camera systems (K = 2 Stereo) Pxel coordnates (m, n ) t of the -th camera 3D Pont (x,y,z) t Approach: F G H m w n w w b b I F J K G H t t t t t t t t t t t t I J K F G GG H x y z 1 I J JJ K VMSC

25 3D Measurement wth Mult-camera Arrangement II Wthout restrcton we can use the dentfed LDT parameter for t 34 = 1 and b = 1: b b F G H m w n w w I F J K G H t t t t t t t t t t t t Rotaton Matrx m t x t y t z t t x t y t z t n t x t y t z t t x t y t z t g g I J K F G GG H x y z 1 I J JJ K Translaton vector VMSC

26 3D Measurement wth Mult-camera Arrangement III Separate x,y,z FF HG HG m t t m t t m t t n t t n t t n t t II KJ F KJ G H x I J K yj z FF HG HG t t m t n t II KJ KJ G r v LDT LDT c h 1 LDT LDT r G t t G G v LDT LDT We receves numercally more stable results, as the pseudo nverse s computed over the SVD concept. VMSC

27 3D Measurement wth Lght Plane and Camera I Intersecton of plane and 3D lne seeng beams t 3 p : D 0,,, n r n r D (1) projecton plane 3 l: 0, 0,, t (2) n to (1) r g g g g (2) p : n g g D 0 a ( b c) a b a c 0 t t p : n g0 n g D 0 t ng0d t for ng 0 t ng (3) VMSC

28 3D Measurement wth Lght Plane and Camera II (3) n to (2) t ng0d t I 0 for 0 t r g g n g ng seeng beams t ng0d t I 0 for 0 t r g g n g ng projecton plane VMSC

29 Sensor Calbraton Prncples VMSC

30 Sensor Calbraton wth moved balls Camera retroreflecton ball Actuator TCP 0 p C G HG f C0 y C0 x C0 0 mm 0 mm T TCP0 C 590 mm J KJ x r0 y r0 y TCP 0 z C0 RAM-KOS Frame T C0 C1 z TCP 0 y C1 Camera Kamera- F KOS x TCP 0 x r1 x C1 z C1 C 0 p C 1 G HG TCP Frame TCP-KOS f C1 480 mm 0 mm 0 mm y r1 J KJ VMSC

31 Sensor Calbraton Prncples I Sensor Frame Seeng Beams or Projecton Pattern Calbraton Features VMSC Reference Frame

32 Sensor Calbraton Prncples II Sensor Frame Seeng Beams Intersecton S r ps (, x) t Seeng beams equatons x r 2 l : r r, wth r u v b t 2 2 t 3D calbraton coordnates T r r ( p, x) 0 R R S S S Reference Frame VMSC

33 Sensor Calbraton Prncples III Sensor Frame Seeng Beams Structured Lght Intersecton Seeng beams and plane equatons l: r r 2 0t p : n ( r r0 ) 0 Calbraton Feature Intersecton of the 3 planes 0t p : n ( r r 0) 0, {1,2,3} Reference Frame defned T r r ( p, x) 0 R R S S S VMSC

34 Sensor Calbraton Prncples IV Intersecton Sensor Frame Seeng Beams Structured Lght Seeng beams and plane equatons l: r r 2 0t p : n ( r r0 ) 0 Mnmum of the mplct body Calbraton Feature Reference Frame models f ( p, T r ( p, x)) 0, S S K K K S S wth T r D and {1,..., N} S K f K VMSC

35 Sensor Calbraton wth Dstances I S S r S T m 0 T l 0 Measured Values: Correspondng RAM coordnates of the and 0 camera r x y x y r r r r r p 0 t p d I : Number Features P : Number of Poses {1,..., I} : Measurement Index p{1,..., P} : Pose Index Explct Sensor Functon: r r ( p, r ) C0 S r p S p Flyng Kamera VMSC

36 Sensor Calbraton wth Dstances II t C t t S C0 I C 0, wth p p p p b,,,, S, S, H, H C0 I 0 0 x0 y0 x0 y0 x0 y0 t C C t p,,,,,,,, C 0 pc 0 b S S H H x y x y x y t VMSC

37 Sensor Calbraton wth Dstances III Input Values (Measured Values): x r r r r x y x y M p 0 Model Parameter: p p S p t M Mnmzaton: 2 S S Mn p r ( p S S m, xm p) r ( ps m, xm 1 p) d, E {2,3,..., I}, p{1,..., P} 2 Dmp C 6 wth P 1 I 1 VMSC

38 Sensor Calbraton wth Actuator I S S T TCP0 S S TCP p S TCP 0 TCP TCP 0 p TCP p r S p T TCP p x n relaton to ntal pose Input Values (Measured Values): r r r r x y x y M p 0 Explct Sensor Functon: r( p) f ( p, x, y) 1 1 S r TCP TCP 0 S r ( ps, r ) p r T( ptcp p) T( ptcp 0) p p := Actuator pose p t M VMSC

39 Sensor Calbraton wth Actuator II t t t S C0 C1 p p p p p, b,,,, S, S, H, H TCP0 C0 C t 0 0 x0 y0 x0 y0 x0 y0 t p p, b,,,, S, S, H, H C1t C1 C0 1 1 x1 y1 x1 y1 x1 y1 t VMSC

40 Sensor Calbraton wth Actuator III Input Values (Measured Values): x p p p TCP t TCP t TCP t M TCP01 TCP0 p TCP0 P t M Model Parameter: p p r r r t TCP t TCP t TCP t S 1 I t Mnmzaton: t Mn ( ) ( ), P 2 3 p r p r p t t wth r( p) r( p) f ( p, x p, y) f( p, x p, y) p, Dm I p 1 VMSC

41 Sensor Calbraton wth xyz-actuator Measured Messdaten Data r S M r R M S R S S r S r R T R S VMSC

42 Sensor Calbraton wth Calbraton Bodes AICON AICON VMSC

43 Vdeo Metrc Camera Model VMSC

44 Vdeo Metrc Camera Model I The model of the pcture sgnal processng of the photo process descrbes the mappng of a space pont by the optcs ncludng the sgnal processng up to the sgnal value n the storage of the computer system. VMSC

45 Vdeo Metrc Camera Model II Focus Fousfehler Error Lnsenfehler Lens Aberraton Beugung Vgnetterung Camera Kamera-Gamma Dffracton örtlche Photonenrauschen Nose Spatal Samplng Spatal Integraton Photons Integraton Abtastung Verstärer- Amplfer and und Quantzaton Quantserungsrauschen Nose A/D RAM Bldspecher Flter Samplng Abtastung VMSC

46 Vdeo Metrc Camera Model III Spatal nvarant and Spatal varant as well as Pont and Surface related Operators VMSC

47 Influencng Effects I Spatal varant and surface related operators Irs/Aperture Dffracton Defocusng Local spatal ntegraton of the photons/charge n the sensor Lne orentated, horzontal low-pass flterng n the amplfer (only for analog nterfaces mportantly) Partly sgnal value dependent nfluences of the photons nose, thermal nose and quantzaton nose of the AD converter VMSC

48 Influencng Effects II Spatal varant and pont related operators Vgnettaton 1 Indvdual nonlnear characterstcs of the ndvdual sensor cells Indvdual nonlneartes and offset error of electroncs assgned to the sensor element. 1 Reducton of the brllance to the edge of the sensor surface (rs effect) VMSC

49 Influencng Effects II Spatal varant and pont related operators The nfluence of charge transfer leaage n the CCD sensor s varant because t depends on the number of memory chans to be passed through. Because of the to the power of charge transfer effectveness of approx % a spatal nvarance could assumed n frstorder approxmaton. The charge transfer leaage are at 1024 transfer tmes approx %. VMSC

50 END VMSC