Sisteme de viziune in robotica An2, Master Robotica

Transcription

1 Sisteme de viziune in robotica An, Master Robotica

2 Cuprins / Contents Grascale image processing. Proprietati statistice ale imaginilor grascale si aplicatii. Imbunatatirea calitatii imaginilor Statistical properties of grascale images and applications. Image Enhancement. Filtrarea imaginilor / filtre spatiale Image filtering 3. Modelarea si eliminarea zgomotelor Noise modeling & removal 4. Detectia muchiilor / metoda de segmentare bazata pe discontinuitati Edge detection & segmentation 5. Detectia colturilor Corner detection

3 Grascale image processing Statistical properties of grascale images and applications. Image Enhancement Proprietati statistice ale imaginilor grascale. Imbunatatirea calitatii imaginilor

4 Trasaturi statistice / Statistical features Statsitical features global features computed on the whole image or on ROIs Histograma intensitatilor Image histogram Gra level: g [0 L], L ma. level 8bits/piel images: L= hg=n g N g no. of piels in the image / ROI having the brightness level g

5 Trasaturi statistice / Statistical features Probabilit distribution function of the gra-levels functia distributiei de probabilitate a nivelelor de gri Pg := probabilit for the brightness in a region <= g: 0 <= Pg <= Pg monotonicall increasing dp/dg >=0 Probabilit densit function of the gra-levels: pg functia densitatii de probabilitate a nivelelor de gri pgg := probabilit for the brightness in a region being between g g+g p g g dp g g dg pg normalized histogram : p g h g, M with : M = image_height image_width p g dg p g 0 L h g, M g0 M M

6 Trasaturi statistice / Statistical features Mean media intensitatilor Measure of the average brightness in a region g g p g dg L g p g M g0 g0 L g h g M H W i0 j0 I i, j Image intunecata dark image Imagine luminoasa bright image

7 Trasaturi statistice / Statistical features Standard deviation deviatia standard Measure of the average contrast in a region L g0 g p g M H W I i, j i0 j0 Contrast ridicat high contrast Contrast scazut low contrast

8 Eemple: computing statistical features %Compute statistical properties: histogram, FDP, mean, standard deviation %read bitmap image from file cameraman.bmp I=imread'cameraman.bmp','BMP'; ColorDepth=56; [height,width]=sizei; M=height*width; %compute histogrm [h,] = imhisti,colordepth; %displa histogram figure; imhisti,colordepth %compute FDP p = h / M; %displa using stem function figure;stem,p; %displa histogram using plot function figure3; hold on; grid on; ploth,'k-';hold off; %histogram filtering with a median filter h=medfilth,0; %displa filtered histogram using plot function figure4; hold on; grid on; ploth,'k-';hold off; %compute brightness mean medie = meani %compute brightness standard deviation std_dev = stdi

9 Eemple: computing statistical features Histogram displaed using plot function Filtered histogram displaed using plot function medie = std_dev = 6.347

10 Grascale image segmentation b adaptive thresholding Automatic computation of the threshold T Can be applied on images with bi-modal histogram The algorithm Application: grascale image segmentation. Take an initial value for T: T 0 = object area = background area T 0 = g MAX + g MIN /. Step k: segment the image after T k- b dividing the image piels in groups: Foreground G: I[i,j] < T k- G Background G: I[i,j] > T k- G 3. T k = G + G / 4. Repeat -3 until T k - T k- < e Efficient implementation first the image histogram is computed then all computations Gi will be done on the histogram!!!

11 Eemple: adaptive thresholding function T=hist_thresholdI; % Compute binar treshold using the "The global adaptive threshold method" [height,width]=sizei; M=height*width; ColorDepth=56; [h,] = imhisti,colordepth; T=mamadoubleI+minmindoubleI/; Te=0.5; dt=56; i=0; while dt > Te, To=roundT; m=0; m=0; nr_pi=0; for z=:to, nr_pi=nr_pi+hz; m=m+z*hz; end m=m/nr_pi; nr_pi=0; for z=to+:colordepth, nr_pi=nr_pi+hz; m=m+z*hz; end m=m/nr_pi; T=m+m/; dt=absto-t; end T=roundT; %Usage I=imread'eight.bmp','BMP'; ColorDepth=56; T=hist_thresholdI; T_norm = T/ ColorDepth Ibw=imbwI, T_norm; Ibw=~Ibw; figure; imshowi;

12 Image enhancement / Imbunatatirea calitatii imaginilor Histogram slide / deplasarea histogramei SlideI[i,j] = I[i,j] + offset offset > 0 brighter image offset < 0 darker image

13 Image enhancement / Imbunatatirea calitatii imaginilor Histogram stretch/shrink latire/ingustare a histogramei Strecth/ShrinkI[i,j] =Final MIN + Final MAX - Final MIN *I[i,j]- g MIN / g MAX - g MIN shrink stretch shrink stretch

14 Image enhancement / Imbunatatirea calitatii imaginilor Grascale levels remapping using a transformation function g output = T g input E. - gamma correction: g c out g in sau g L out g c L in

15 Image enhancement / Imbunatatirea calitatii imaginilor E. - gamma correction: < : codificare/compresie gamma > : decodificare/decompresie gamma

16 Trasaturi statistice / Statistical features Information information associated to the gra-level g: I g log p g [ bits ] information is high for a gra level with smaller probabilit Entrop average information from the image: No. bits necessar to encode the gra-levels from the image H big grascale values are widel spread H ma = log L [bits] uniform PDF Energ how are the gra-levels distributed: E mica large number of gra-levels E ma = gra-level H E L g0 L g0 p g log p g [ bits ] p g

17 Procesari pe histograma / Histogram processing Histogram equalization/ Egalizarea histogramei Normalized gra-levels: g [0... L] r [0... ] Transformation functions: s = Tr a. Bijective and monotonicall increasing r = T - s b. 0 <= Tr <=

18 Procesari pe histograma / Histogram processing Histogram equalization/ Egalizarea histogramei p r r, p s s FDP of the source and destination image p r r, Tr are known and T - satisfies condition a ps s pr r dr ds Historama cumulativa / FDP cumulativa CDF s T r r 0 p r w dw T satisfies a & b Leibniz Rule: the derivative of an integral function superiorl defined is the integral function evaluated in the upper limit: ds dr dt r dr d dr r 0 p r w dw p r r 3

19 p s s: uniform PDF independent b p r r Histogram equalization algorithm Procesari pe histograma / Histogram processing 0, s r p r p ds dr r p s p r r r s Histogram equalization/ Egalizarea histogramei + 3 L k n n r p r T s L k r L k n n r p k j j k j j r k k k k k r 0..., /, 0..., 0 0 remapping the input image gra-levels: r k -> s k

20 Procesari pe histograma / Histogram processing Histogram equalization/ Egalizarea histogramei: rezultate I = imread'tire.tif'; J = histeqi; imshowi figure, imshowj figure; imhisti,56 figure; imhistj,56

21 Grascale image processing Image filtering space domain filters Filtrarea imaginilor filtre spatiale

22 Convolution This operator is the basis of the linear image filtering operation applied in the spatial image domain b directl manipulating image piels It implies the usage of a convolution kernel H usuall squared shape, b size w*w, with width/height = w=k+ which is applied on the image using a shift & multipl scheme: I D k k I H D I S, H i, j I i, j, 0.. Height, 0.. Width ik jk S

23 Convolutie - eemple Eample: filter for vertical edges detection computes the horizontal derivative of the image X Y = filterh,x

24 Filtre de tip trece-jos / low-pass filters Used for smoothing the gra-levels / noise reduction. The outcome is an averaging operation of the current piel b his neighbor s values blur effect on the image These filters have onl positive elements. From that reason the result of the convolution is normalized b dividing it with the sum of the filters' coefficients: k k i k k j S D j i I j i H c I,,, k k i k k j j i H c, Average filter 33: Gaussian filter 33:

25 Filtre de tip trece-jos / low-pass a. Original image b. Result obtained b c. Result obtained b appling a 33 mean filter. appling a 55 mean filter.

26 Filtre de tip trece-sus / high-pass Image regions / piel with local intensit variations are highlighted i.e. edge piels. The outcome is a high pass filtering image sharpening The filters / kernels can have positive an negative elements. Edge detection kernels must have a null sum: c. The result obtained b filtering the original image with the high-pass filter a. The result of appling the Laplace edge detection filter on the original image b. The result of appling the Laplace edge detection filter on the blurred image

27 Filtre de tip trece-sus / high-pass High-pass filters will have both positive and negative coefficients. You must ensure that the final result is an integer between 0 and 55! There are three possibilities to ensure that the resulting image fits the destination range.. The first one is to compute: S F, S F, S F k k 0 F 0 ma{ S, S } L I D u, v S F * IS u, v k k In the formula above represents the sum of positive filter coefficients and the sum of negative filter coefficients magnitudes. This result of appling the highpass filter alwas lies in the interval where L is the maimum image gra level 55. The result of this transform will place scale the result to [-L/, L/] and then move the 0 level to L/.

28 Filtre de tip trece-sus / high-pass. Another approach is to perform all operations using signed integers determine the minimum and maimum and then linearl transform the resulting values according to: D LS min ma min 3. The third approach is to compute the magnitude of the result and saturate everthing that eceeds the maimum level L.

29 Grascale image processing Modelarea si eliminarea zgomotelor Noise modeling & removal

30 Definitia zgomotului / Noise definition Noise := An process n that affects image f and is not part of the scene s: fi,j = si,j + ni,j additive noise model Causes:. Discrete nature of the radiation. Detector sensitivit variable sensitivit of the sensorial elements CCD/CMOS fied pattern noise dark current noise DCN & photon response nonuniformit PhRNU 3. Electrical noise 4. Data transmission errors 5. Air turbulences 6. Spatial resolution of the sensor 7. Digitization errors quantization of the color /gra levels Tpes of noise FDP pg shape: - Salt&pepper sare si piper - Uniform - Gaussian - Other distributions: Raleigh, Erlang/Gamma, Eponential etc.

31 Salt &Pepper noise sare si piper Causes Malfunctioning of sensor s cells Malfunctioning of memor cells Snchronization errors in the signal digitization process Bits loss on the communication channel Model PDF salt& pepper A for g a " pepper " B for g b " salt" In the salt&pepper noise model onl two possible values are possible, a and b, and the probabilit of obtaining each of them is less than 0. otherwise, the noise would vastl dominate the image. For an 8 bit/piel image, the tpical intensit value for pepper noise is close to 0 and for salt noise is close to 55.

32 Salt &Pepper noise sare si piper

33 Salt &Pepper noise sare si piper Eliminating the Salt&Pepper noise Median filter non-linear filter Ordered filters are based on a specific image statistic, called ordered statistic. The are called non-linear, because the cannot be applied as a linear operator such as a convolution kernel. These filters operate on small windows, and replace the value of the central piel similarl to convolution. The ordered statistic is a technique which arranges all the piels in sequential order, based on their gra-level value. The position of an element in this ordered set can be characterized b its rank. Given a NN window W, the piel values can be sorted in ascending order: I I I 3 N I I I, I,, I represent the intensit values of the piels located, 3 N within the NN window W {85, 88, 95, 00, 04, 04, 0, 0, 4}

34 Salt &Pepper noise sare si piper The median filter: selects the middle value from the ordered statistic and replaces the destination piel with it. In the eample above, the selected value would be 04. The median filter allows the elimination of salt&pepper noise.

35 Gaussian noise Gaussian noise is useful for modeling natural processes which introduce noise e.g. noise caused b the discrete nature of radiation and the conversion of the optical signal into an electrical one detector/shot noise, the electrical noise during acquisition sensor electrical signal amplification, etc.. For modelling these tpes of noises a poisson distribution should be used but it is to complicated to handle it mathematicall can be approimated b a Gaussian distribution Model where: g = gra level; g FDP Gaussian e = media zgomotului; = deviatia standard a zgomotului;

36 Gaussian noise

37 Design a variable size Gaussian kernel FDP for gaussian noise with 0 mean: G, = GG G, e The filter size w of such a filter is usuall 6 for eample, for a Gaussian noise with s=0.8 w = w 6

38 Matlab eample: Calculul nucleului gaussian cazul D = 0.8 w = 5

39 Filtering the Gaussian noise,,, I G G I G G I S S D 0 e G,,, I G I S D sau 0 0 G G Spatial domain filters using convolution kernels 0 e G

40 Detecting the presence of noise in the image Signal to Noise Ratio SNR Raportul semnal zgomot Additive noise model: fi,j = si,j + ni,j n zero mean <ni,j> = 0 and signal independent <si,jni,j> = 0 s i, j f i, j f s n Noise alters onl the standard deviation and not the mean of the image: SNR s n f n

41 SNR Eemples:

42 Computing SNR Single image case. Compute f on the whole image. Select a ROI with uniform intensit S =0 e: sk, water, wall etc. and compute f = n s f SNR n n f = n

43 Computing SNR images case successive images of a static scene: fi,j = si,j + ni,j gi,j = si,j + mi,j n and m have the same FDP: same mean 0 and standard deviation n and m are uncorrelated independent with the signal:<si,jni,j> = 0, <si,jmi,j> = 0 f and g are uncorrelated <fi,jgi,j> = 0 Normalized correlation between f and g SNR r r

44 Grascale image processing Features detection: edges & corners Detectia de trasaturi: Detectia punctelor de muchie. Detectia de colturi.

45 Motivation Purpose of edge detection? - It seems that human visual sstem uses edges as primitives in the perception/recognition process comple information color and teture are inferred afterwards - It is possible to recognize shapes/objects onl based on contours i.e. caricatures, bw comics / cartoons. Edge detection is an important step in the automated image analsis process Edge detection segmentation process Segmentation := from low level information piels / row data high level information is etracted: - edge points contours shape features analsis - edge points features for sparse stereo reconstruction

46 Definition Muchie / edge := the frontier that separates regions of different brightness usual the brightness has an abrupt variation at the edge How can we detect an edge?

47 Edge points intensit profile

48 Image gradient -st order derivative Gradient of a D function image gradient For a digital image: = =

49 Approimations of gradient Operatorul Roberts Operatorul Prewitt Operatorul Sobel

50 Magnitude Gradient features Directiion dir G arctg G f f Imaginea G Binarizare prag T

51 Eamples Cann G G

52 Metoda Cann de detectie a muchiilor Features of the Cann edge detector Maimizes the signal to noise ratio for a correct Good localization of the edge Minimization of the positive responses to a single edge non-edges elimination Algorithm. Gaussian filtering. Edge magnitude & direction computation 3. Non-maima suppression edge thinning 4. Hsteresis thresholding edge linking

53 . Gaussian filtering 0 0, e g, g g g Eample: = 0.8 w = 5 filter dimension w 6 0 e g * *,, *,, g g f g f f s 0 e g G, = [ ]

54 . Edge magnitude & direction G G f f,, f f,, * S * S,, Magnitude Directiion dir G arctg G f f

55 3. Non-maima suppression Edge thinning along the gradient direction piel thick Quantif the gradient directions: P is a local maima if: G 6 G and G G Where: G, G, G 6 are the gradient magnitudes in P, I, I 6. If P is a local maima is retained. Otherwise is eliminated.

56 4. Hsteresis thresholding Edge linking contour defragmentation. Two thresholds are used: q L low and q H and the following thresholding scheme is applied: Ever edge point with magnitude bellow q L is labeled as non-edge Ever edge point with magnitude above q H is labeled as strong edge Ever edge point with magnitude between q L and q H is labeled as weak edge. Appl an algorithm similar with the labelling one that marks weak edge points as strong if the are connected to strong edge points and eliminates weak edge points if the are not connected to strong edge points. a. Result after step : strong blue edges and weak green edges. b. Result after step

57 4. Hsteresis thresholding An efficient implementation of this step uses a queue to perform a breadth first search through WEAK_EDGE points connected to STRONG_EDGE points and mark them as STRONG_EDGE points. The algorithm would look like this:. Scan the image, top left to bottom right, pick the first STRONG_EDGE point encountered and push its coordinates in the queue.. While queue is not empt a. Etracts the first point from the queue b. Find all the WEAK_EDGE neighbors of the current point c. Label in the image all these neighbors as STRONG_EDGE points d. Push the image coordinates of these neighbors into the queue e. Continue to the net STRONG_EDGE point 3. Go to step considering the net STRONG_EDGE point. 4. Eliminate the remaining WEAK_EDGE points from the image b turning them to NON_EDGE 0

58 Results Magnitude image G Thresholding with T Non-maima suppression Hsteresis thresholding

59 -nd order derivative, f f f Laplacian *, f f

60 Laplacian of Gaussian LoG/ Mar-Hilderth 6 6 6,,,, e LoG e e e e LoG g g g g g LoG

61 Laplacianul Gaussianului LoG Aplicatie: stereo corelatie compenseza variatia de luminozitate dintre camera stanga si dreapta.

62 Corner detection Corner := a point where are intensit variation in at least different

63 Corner detection Harris method The intensit variation in a point, for a window w shifted with displacement u,v: Corners points where Eu,v has a local maima Talor series aproimation: w can be window of Gaussian weights!

64 M A C Corner detection Harris method C B Autocorrelation matri covariance of the derivatives contains all the differential operators that describe the geometr of the intensit surface in, The autocorrelation matri can be diagonalized b rotating the aes 0 ma 0 min, ma min 0 The eigenvalues of M: λ, λ response function R, measure of the cornerness in P, det M trace M AB C A B R, det M k trace M k =

65 Corner detection Harris method Harris algorithm:. For each piel P, compute the autocorrelation matri M.. Compute the map matri of the response function R, in ever piel P,. 3. Filter out edge points b thresholding e: if. R < T R=0. 4. Appl non-maimum suppression retain onl local maima others are eliminated: R=0. 5. Al remaining points R > 0 will be the reported corners. 6. Optional ou can also limit the maim no. of reported corners. [] A. Koschan, M. Abidi, Digital Color Image Processing, Wile & Sons, cap 6, pag 43-44