IMAGE WARPING APPLICATIONS. register multiple images (registration) correct system distortions. register image to map (rectification)

Transcription

1 APPLICATIONS IMAGE WARPING N correct system distortions Rectification of Landsat TM image register multiple images (registration) original register image to map (rectification) N fill Speedway rectified Broadway Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax) Speedway Broadway

2 WHAT IS REGISTRATION? Alignment of two images such that pixels in each correspond to the same area examples of registered remote sensing images same sensor orbit j orbit k date 1 date different sensors orbit j date 1 and date 1+revisit period or orbit j and orbit k date 1, date with off-nadir pointing AVIRIS SPOT TM TM Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

3 THREE COMPONENTS TO WARPING appropriate mathematical distortion model(s) coordinate transformation resampling (interpolation) Dr. Robert A. Schowengerdt, (voice), (fax)

4 POLYNOMIAL DISTORTION MODELS Generic model useful for registration, rectification and geocoding Known as rubber sheet stretch Relates distorted coordinate system (x,y) and reference coordinate system (x ref,y ref ) N N i i j x = a ij x ref yref i = 0 N j = 0 i j y = b ij x ref yref i = 0 N i j = 0 For example, a quadratic polynomial is written as: x = a 00 + a 10 x ref + a 01 y ref + a 11 x ref y ref + a 0 x ref +a 0 y ref y = b 00 + b 10 x ref + b 01 y ref + b 11 x ref y ref + b 0 x ref +b 0 y ref Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

5 The coefficients in the polynomial can be associated with particular types of distortion coefficient warp component a 00 b 00 a 10 b 01 a 01 b 10 a 11 b 11 a 0 b 0 shift in x shift in y scale in x scale in y rotation shear in x shear in y y-dependent scale in x x-dependent scale in y nonlinear scale in x nonlinear scale in y Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

6 different types of distortion original shift scale in x shear y-dependent scale in x quadratic scale in x rotation quadratic Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

7 A linear (affine) polynomial (number of terms K = 3) Special case that can include: shift scale shear rotation x = a 00 + a 10 x ref + a 01 y ref y = b 00 + b 10 x ref + b 01 y ref In vector-matrix notation x y = a 10 a 01 x ref a + 00 b 10 b 01 y ref b 00 Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

8 GROUND CONTROL POINTS (GCPS) Use to control the polynomial, i.e. to determine its coefficients Characteristics: high contrast in all images of interest small feature size unchanging over time all are at the same elevation (unless topographic relief is being specifically addressed) Often located by visual examination, but can be automated to varying degree, depending on the problem Dr. Robert A. Schowengerdt, (voice), (fax)

9 Automated GCP Location Use image chips Small segments that contain one easily identified and well-located GCP Normalized cross-correlation between template chip T (reference) and search area S in image to be registered N N R ij = T mn S i+ m, j+ n K 1 K m = 1 n = 1 where N N T K1 mn = and K = S i m m = 1 n = 1 1 N m = 1 N n = 1 +, j+ n 1 normalization adjusts for changes in mean DN within area Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

10 Schematic for correlation target area T Cross-correlation of one area two relative shift positions search area S i,j = 0,0 i,j = 0,1 Chip layout over full scene T 1 + T + T 3 + S + S 3 S reference image distorted image Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

11 x 0 0 Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax) y search chip target chip search chip target chip Example with TM chips IMAGE WARPING

12 Using GCPs to find polynomial coefficients Set up system of simultaneous equations using GCPs and solve for polynomial coefficients Example with quadratic polynomial (number of terms K = 6) Given M pairs of GCPs For each GCP pair, m, create two equations x m = a 00 + a 10 x refm + a 01 y refm + a 11 x refm y refm + a 0 x refm + a 0 y refm y m = b 00 + b 10 x refm + b 01 y refm + b 11 x refm y refm + b 0 x refm + b 0 y refm Then, for all M GCP pairs, in vector-matrix notation x 1 x : x M = 1 x ref1 y ref1 x ref1 y ref1 x ref1 1 x ref y ref x ref y ref x ref y ref1 y ref : : : : : : 1 x refm y refm x refm y refm x refm y refm a 00 a 10 a 01 a 11 a 0 a 0 Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

13 So, for each set of GCP x- and y-coordinate pairs, we can write X = WA Y = WB Determined case (M = K, just enough GCPs) solution: M = K : A = W 1 X B = W 1 Y Exact solution which passes through GCPs, i.e. they are mapped exactly Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

14 Overdetermined case (M > K, more than enough GCPs): M K : X = WA + ε X Y = WB + ε Y Solution: M K : Â = ( W T W ) 1 W T X Bˆ = ( W T W ) 1 W T Y. ( W T W ) 1 W T is called the pseudoinverse of W This solution results in least-squares minimum error at GCPs: T min [ ε X εx ] = ( X WÂ ) T ( X WÂ ) T min [ ε Y εy ] = ( Y WBˆ ) T ( Y WBˆ ). Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

15 Dr. Robert A. Schowengerdt, (voice), (fax) x y y = x R= linear 1-D Example of Polynomial Curve Fitting IMAGE WARPING

16 Dr. Robert A. Schowengerdt, (voice), (fax) x y R M M M Y = M0 + M1*x +... M8*x 8 + M9*x 9 second order IMAGE WARPING

17 Dr. Robert A. Schowengerdt, (voice), (fax) x y R M M M M Y = M0 + M1*x +... M8*x 8 + M9*x 9 third order IMAGE WARPING

18 Dr. Robert A. Schowengerdt, (voice), (fax) x y R M e-05 M M M M Y = M0 + M1*x +... M8*x 8 + M9*x 9 fourth order IMAGE WARPING

19 Dr. Robert A. Schowengerdt, (voice), (fax) x y R 1 M e-08 M e-05 M M M M Y = M0 + M1*x +... M8*x 8 + M9*x 9 fifth order IMAGE WARPING

20 Example Application GCP1 GCP6 Register aerial photo to map in an urban area GCP GCP5 aerial photo (x,y) Locate 6 GCPs in each (black crosses) GCP3 GCP4 Locate 4 Ground Points (GPs) in each (red circles) not used for control, only for testing GCP1 GCP GCP6 GCP5 map reference (xref,y ref ) GCP3 GCP4 Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

21 Dr. Robert A. Schowengerdt, (voice), (fax) y (pixels) x (pixels) Mapping of GCPs IMAGE WARPING

22 Dr. Robert A. Schowengerdt, (voice), (fax) number of terms 0 1 RMS deviation (pixels) 3 4 GCPs GPs 5 Error at GCPs and GPs versus polynomial order IMAGE WARPING

23 Dr. Robert A. Schowengerdt, (voice), (fax) original six GCPs five GCPs, with outlier deleted deviation in x (pixels) deviation in x (pixels) -1-1 deviation in y (pixels) deviation in y (pixels) Refinement of GCPs by error analysis IMAGE WARPING

24 Final result Dr. Robert A. Schowengerdt, (voice), (fax)

25 Piecewise Polynomial Distortion Model IMAGE WARPING piecewise quadrilateral warping For severely-distorted images that can t be modeled with a single, global polynomial of reasonable order a d b c A B D C Example airborne scanner image piecewise triangle warping affine correction a b c A B C d D reference frame distorted frame quadratic correction Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

26 COORDINATE TRANSFORMATION Coordinates are mapped from the reference frame to the distorted image frame ( xy, ) = fx ( ref, y ref ) Distortion Correction Implementation 1. Create an empty output image (which is in the reference coordinate system). Step through the integer reference coordinates, one at a time, and calculate the coordinates in the distorted image (x,y) by Eq Estimate the pixel value to insert in the output image at (xref,y ref ) from the original image at (x,y) (resampling) Backwards mapping from (x ref,y ref ) > (x,y) avoids holes or overlaying of multiple pixels in the processed image Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

27 Coordinate transformation between two frames reference frame image frame column column (x,y) (x ref,y ref ) row row Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

28 RESAMPLING The (x,y) coordinates calculated by ( xy, ) = fx ( ref, y ref ) are generally between the integer pixel coordinates of the array Therefore, must estimate (interpolate or resample) a new pixel at the (x,y) location reference frame image frame column column DN r (x,y) DN(x ref,y ref ) row row Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

29 Dr. Robert A. Schowengerdt, (voice), (fax) (pixels) resample DN(x,y) between E and F 0 resample F between C and D 0. resample E between A and B 0.4 value n-n linear 1. implement -D bilinear resampling as two successive 1-D resamplings + [ xdn D + ( 1 x )DN C ] y DN r = xdn B 1 x [ + ( )DN A ] ( 1 y ) C F D bilinear: slower, but continuous 1 - y DN r (x,y) DN r = DN C y A E B nearest-neighbor: fast, but discontinuous INT(x,y) x 1 - x Three common weighting functions: Pixels are resampled using a weightedaverage of the neighboring pixels resampling distances IMAGE WARPING

30 Dr. Robert A. Schowengerdt, (voice), (fax) (pixels) value PCC, α = -1 PCC, α = -0.5 sinc 1. High-boost filter characteristics; amount of boost depends on amplitude of side-lobe, which is proportional to α standard bicubic is α = -1; superior bicubic is α = -0.5 where is the distance from (x,y) to the grid points in 1-D = 0, = α( ), 1 = ( α + ) 3 ( α + 3 ) + 1, 1 DN r ( α ; ) = PCC ( α, ) piecewise polynomial; special case of Parametric Cubic Convolution (PCC) bicubic: slowest, but results in sharpest image IMAGE WARPING

31 PCC resampling procedure resample along each row, A-D, E-H, I-L, M-P DN Q = DN A PCC ( x + 1 ; α ) + DN B PCC ( x ; α ) + DN C PCC ( ( 1 x ); α ) + DN D PCC ( x ; α ) etc. resample along new column Q-T DN r = DN Q PCC ( y + 1 ; α ) + DN R PCC ( y ; α ) + DN S PCC ( ( 1 y ); α ) + DN T PCC ( y ; α ) A B Q C D y x 1 - x E F R G H 1 - y DN r (x,y) I J S K L M N T O P Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

32 Comparison of nearestneighbor and bilinear resampling for image magnification 1X nearest-neighbor bilinear X 3X Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

33 Surface plots from previous figure nearest-neighbor bilinear Dr. Robert A. Schowengerdt, (voice), (fax) x y x y DN DN DN DN

34 Comparison of 4 resampling functions for image magnification nearest-neighbor bilinear PCC (α = 0.5) PCC (α = 1.0) Dr. Robert A. Schowengerdt, schowengerdt@ece.arizona.edu, (voice), (fax)

35 image difference maps resulting from different resampling functions nearest-neighbor bilinear bilinear PCC Note, resampling function affects local radiometric accuracy, while polynomial distortion model affects global geometric accuracy. Dr. Robert A. Schowengerdt, (voice), (fax)