Position & motion in 2D and 3D

Transcription

1 Position & motion in 2D and 3D! METR4202: Guest lecture 22 October 2014 Peter Corke Peter Corke Robotics, Vision and Control FUNDAMENTAL ALGORITHMS IN MATLAB 123 Sections 11.1, 15.2 Queensland University of Technology Peter Corke

2 Peter Corke

3 Peter Corke

4 The (amazing) sense of vision The trilobites were among the most successful of all early animals, appearing 521 million years ago and roaming the oceans for over 270 million years. Peter Corke

5 2D and 3D Queensland University of Technology Peter Corke

6

7 Cave paintings ~40,000 years ago Peter Corke

8 Piero della Francesca ( ) Jan Vredeman de Vries, Peter Corke

9 trompe l'oeil ˌtrômp ˈloi noun ( pl. trompe l'oeils pronunc. same ) visual illusion in art, esp. as used to trick the eye into perceiving a painted detail as a threedimensional object. Peter Corke

10 Peter Corke

11 Peter Corke

12 Peter Corke

13 Peter Corke

14 Peter Corke

15 Edgar Mueller Peter Corke

16 Peter Corke

17 Waterfall M.C. Escher 1961 Peter Corke

18 Image formation (in pictures) Queensland University of Technology Peter Corke

19 points in the Peter Corke

20 Image formation points in the image plane Peter Corke

21 Image formation dark inverted image points in the image plane Peter Corke

22 The pin hole camera Peter Corke

23 Pin hole images Peter Corke

24 Simple imaging Y Y x Z = y f y X Z f X Z = x f Image formation is the mapping of scene points to the image plane x = fx Z, y = fy Z (X,Y,Z) 7! (x,y) R 3 7! R 2 Queensland University of Technology Peter Corke

25 Image formation dark inverted image Peter Corke

26 Image formation f bigger area f brighter image small is good F = f /f image plane Peter Corke

27 Use a lens to gather more light George R. Lawrence 1900 Peter Corke

28 Thin lens model object pin hole ray equivalent pin hole z z o focal points f f z i inverted image 1 z o + 1 z i = 1 f ideal thin lens image plane Perspective projection 3D to 2D Focussing on distant objects (X,Y,Z) 7! (x,y) z o! z i! f R 3 7! R 2 Peter Corke

29 Peter Corke

30 Peter Corke

31 Peter Corke

32 Perspective projection Maps Lines lines parallel lines not necessarily parallel angles are not preserved Conics conics Peter Corke

33 No unique inverse

34

35 Image formation (in maths) Queensland University of Technology Peter Corke

36 Homogeneous coordinates Cartesian homogeneous P =(x,y) P 2 R 2 P =(x,y,1) P 2 P 2 homogeneous Cartesian P =( x,ỹ, z) P =(x,y) x = x z, y = ỹ z lines and points are duals Peter Corke

37 A line in homogeneous form such that p =( x,ỹ, z) =(l 1,l 2,l 3 ) `T p = 0 Point equation of a line y = mx + c Peter Corke

38 Line joining points p 2 =(d,e, f ) p 1 =(a,b,c) ` ` = p 1 p 2 Peter Corke

39 Intersecting lines 2 =(d,e, f ) p p = `1 `2 1 =(a,b,c) line equation of a point Peter Corke

40 Central projection model 0 x ỹa = z 0 1 f X 0 f 0 0AB Y ZA Peter Corke

41 optical axis Pin-hole model in homogeneous form P =(X, Y, Z) p x 0 ỹa = z 1 f f A X Z 1 1 C A z=f principal point image planey z C {C} y C x C camera origin x = fx,ỹ = fy, z = Z x = x z, y = ỹ z Central perspective model ) x = fx Z, y = fy Z Perspective transformation, with the pesky divide by Z, is linear in homogeneous coordinate form. Peter Corke

42 0 ỹa = z 1 0 3D to f f A X Z 1 1 C A scaling/ zooming f A B0 f 0 0 C A Peter Corke

43 Change of coordinates P =(X, Y, Z) W 1 u u = x r u + u 0 z optical axis 0 0 p v = y r v + v 0 H H 1 v (u 0,v 0 ) image plane z=f principal point W z C {C} y C x C camera origin ṽ w 1 A = 0 1 r u 0 u rv v C A ỹa z 1 scale from metres to pixels shift the origin to top left corner p = u v = ũ/ w ṽ/ w Peter Corke

44 Complete camera model P =(X, Y, Z) C P 0 x ỹa = z f f A X Z 1 1 C A {C} y ξ C z x {0} extrinsic parameters 0 P ṽ w 1 A = u r B u 0 1 C f v rv 0 A@ 0 f 0 0A R t {z } K 1 {z } intrinsic parameters C camera matrix 0 1 X BY ZA Peter Corke 1

45 Camera matrix Mapping points from the to an image (pixel) coordinate is simply a matrix multiplication using homogeneous coordinates ṽ w 1 A = C 11 C 12 C 13 C 14 C 21 C 22 C 23 C 24 C 31 C 32 C 33 C A X Z 1 1 C A u = ũ w, v = ṽ w Peter Corke

46 Scale invariance Consider an arbitrary scalar scale factor!!!!! ũ,ṽ, w will all be scaled by but! ṽ w 1 A = l u = ũ C 11 C 12 C 13 C 14 C 21 C 22 C 23 C 24 w, v = ṽ so the result is unchanged C 31 C 32 C 33 C 34 w l 0 1 A X Z 1 1 C A Peter Corke

47 Normalized camera matrix Since scale factor is arbitrary we can fix the value of one element, typically C(3,4) to one. focal length pixel size camera position & orientation ṽ w 1 A = C X 11 C 12 C 13 C C 21 C 22 C 23 C ABY C ZA C 31 C 32 C u = ũ w, v = ṽ w Peter Corke

48 Consider a moving camera Queensland University of Technology Peter Corke

49 Motion of a camera w x v x (u,v) v z w z v y (X,Y,Z) w y 6 degrees of freedom translate along x, y, z rotate about x, y, z

50 Optical flow patterns t x v (pixels) w z v (pixels) u (pixels) u (pixels) v (pixels) t z u (pixels) Pixel motion depends on pixel position camera motion Peter Corke

51 Optical flow patterns t x w y v (pixels) v (pixels) u (pixels) u (pixels) Rotation and translation in x and y cause very similar motion Peter Corke

52 Optical flow equation speed of pixel speed of camera (ū, v) are distances from principal point Peter Corke

53 Motion of multiple points Consider the case of three points, in matrix form ν = (v x, v y, v z, ω x, ω y, ω z ) R 6 ational velocity components. Th Peter Corke

54 Desired view Peter Corke

55 Current view Peter Corke

56 Image plane motion Peter Corke

57 Image plane motion ( u, v) (u, v) Peter Corke

58 Inverting the problem required point velocity to move from p to p* desired point current point Peter Corke

59 IBVS simulation Peter Corke

60 Applications Queensland University of Technology Peter Corke

61 Visual servoing: simple feedback Peter Corke

62 Non-holonomic visual servoing Peter Corke

63 Direct processing of wide-angle imagery K. Usher and J. Roberts and P. Corke and E. Duff, Vision-based navigational competencies for a car-like vehicles. In M. Ang and O. Khatib, editors, Experimental Robotics IX, volume 21 of Springer Tracts in Advanced Robotics (STAR), P. Corke and D. Symeonidis and K. Usher, Tracking road edges in the panospheric image plane. In Proc. Int. Conf on Intelligent Robots and Systems (IROS), Peter Corke

64 AUV visual odometry A hybrid AUV design for shallow water reef navigation. In Proc. IEEE Int. Conf. Robotics and Automation, M. Dunbabin, P. Corke, Visual motion estimation for an autonomous underwater reef monitoring robot. In P. Corke and S. Sukkariah, editors, Field and Service Robotics: Results of the 5th International Conference, I. Vasilescu, K. Kotay, D. Rus, M. Dunbabin, and P. Corke. Data collection, storage and retrieval with an underwater sensor network. In Proc. IEEE SenSys, Peter Corke

65 Crucible Finder The HMC must deal with uncertainty on crucible pose and vehicle approach position Use a pan/tilt/zoom camera Use visual fiducials on the crucible handle Peter Corke

66 Crucible pickup Peter Corke

67

68