Nonlinearity optimization in nonlinear joint transform correlators

Transcription

1 Nonlinearity optimization in nonlinear joint transform correlators Leonid P. Yaroslavsky and Emanuel Marom Three types of nonlinear transformations of the joint spectrum in nonlinear joint transform correlators NLJTC s are investigated with the purpose of achieving the highest discrimination capability in target location in a cluttered background: logarithmic transformation and the 1 k th law transformation in combination with the limitation of the signal dynamic range and binarization by thresholding. By computer simulation carried out on a set of test images, it is shown that application of these transformations in NLJTC s may considerably improve the correlator s capacity to locate and recognize properly small objects on a cluttered background, provided there is proper selection of nonlinearity parameters. It is also shown that a moderate blur of the joint spectrum in such NLJTC s before nonlinear transformation is permissible, which simplifies the requirements of correlator optical alignment, the resolution power of correlator electronic components, or both Optical Society of America Key words: Optical correlators, matched filters, joint transform correlators. 1. Introduction Nonlinear joint transform correlators NLJTC s with input images joint spectra nonlinearly transformed by computer or in an electronic amplifier before modulating the output spatial light modulator SLM Fig. 1 have been proposed as a tool for real-time pattern recognition and target location. 1 7 Among them, the binary joint transform correlators BJTC s 2 4 that use a binary SLM at the Fourier plane have attracted special attention because of their improved discrimination capability and high light efficiency because of the availability of binary SLM s. However, a number of important issues on the design and implementation of such correlators have not been addressed as yet: What type of nonlinear transformation should one use in joint transform correlators JTC s to ensure the highest discrimination capability? How sensitive is the correlator s discrimination capability to such design factors as the accuracy of realization of the nonlinear transformation, the limitation of the dynamic range, the resolution power of The authors are with the Faculty of Engineering, Tel-Aviv University, Tel-Aviv 69978, Israel. Received 31 July 1996; revised manuscript received 30 January $ Optical Society of America the nonlinear optical media and electronic components used, and the accuracy of optical alignment? Why do BJTC s perform better than conventional ones and how sensitive is their performance to the binarization threshold? In this paper, we address these issues. We show that NLJTC s with a logarithmic nonlinearity approximate the optimal adaptive correlator OPAC 8 that guarantees the highest discrimination capability for target location in a cluttered background. We show also that the logarithmic, 1 k th law nonlinearity within a limited dynamic range and binary nonlinearity can be regarded as a version of nonlinearity that compresses the signal dynamic range and that JTC s with such nonlinearities may exhibit similar discrimination capabilities, provided an appropriate selection of the nonlinearity index k, the dynamic-range limitation and the binarization threshold, respectively. In Section 2, we provide a brief review of the relevant theory. In Section 3, we justify the use of a logarithmic nonlinearity in JTC s and experimentally investigate their discrimination capability and sensitivity to the limitation of the nonlinearity s dynamic range and resolution power. In Section 4, from analogy between the logarithmic and 1 k th law nonlinearities for k 1, we suggest using, in NLJTC s, the 1 k th law nonlinearity, demonstrate that the NLJTC s may perform slightly better than even the logarithmic JTC s, and investigate, by computer sim APPLIED OPTICS Vol. 36, No July 1997

2 etc., are involved in the localization, this requirement should be satisfied on average over these factors. The transfer function of the optimal filter H f was found 8 to be Fig. 1. Schematic diagram of the JTC with a nonlinear transformation of the joint spectrum. ulation, a trade-off between the nonlinearity index k and the nonlinearity s dynamic-range limitation threshold. Being motivated by the results for the NLJTC s with the 1 k th law nonlinearity and the limitation of the nonlinearity s dynamic range, in Section 5 we experimentally by computer simulation investigate BJTC s and show that, with an appropriately chosen binarization threshold, they may perform nearly as well as NLJTC s with logarithmic and 1 k th law nonlinearities and that they are rather tolerant to reasonable deviations of the binarization threshold from its optimal value. In the conclusion, we summarize the discussion. 2. Review of the Theory We base our analysis on the theory presented in Ref. 8. JTC s represent an attractive implementation of optical correlators. From the point of view of the theory, optical correlators can be regarded as a special case of localization recognition devices that consist of a linear filter followed by a unit for the localization of the signal global maximum at the filter output. It was shown in Ref. 8 that, for the problem of target location in a cluttered background, the requirement for the highest discrimination capability of such devices is practically equivalent to the requirement that the filter provide the maximal ratio of the signal value in the point of the output, where the target object is located, to the standard deviation of the signal in the background part of the output signal. Since a number of random factors, such as sensor noise, an unknown reference-object position, variability of the reference-object orientation, size, RO* f H opt f AV imsys AV xo B f 2, (1) where RO* f is the complex-conjugate Fourier spectrum of the reference target object, B f 2 is the power spectrum of the background component of the input image for object location or average power spectrum of objects to be rejected in object recognition, AV imsys denotes averaging over the imagingsystem sensor noise, and AV x0 denotes averaging over the unknown coordinates x 0 of the target. The filter described by Eq. 1, if realizable, is adaptive to the background component of the image and therefore provides the best possible performance for the given individual input image. We refer to it as the OPAC. The design of the OPAC requires that one know the power spectrum of the background objects averaged over the involved random factors. These data are not known before the target is located and have to be determined from the observed input image. As a zero-order approximation of the background power spectrum one may use the squared absolute value of the entire observed image spectrum IM f 2 : AV imsys AV x0 B f 2 IM f 2. (2) Approximation 2 is based on the assumption that the target object size is much smaller than the size of the entire image area of search. For a more accurate estimation of the background power spectrum, one can use the following two models for the representation of the image background component: b x w x x 0 im x, (3) b x im x ro x x 0, (4) where im x is the observed image, b x is its background component, ro x x 0 is the reference target object, and w x x 0 is a window function: w x x 0 0 within the target object, (5) 1 elsewhere For the model of Eq. 3, one can show that, with the assumption of a uniform a priori distribution of the object coordinates x 0 over the picture area S, AV imsys AV x0 B f 2 AV imsys IM f 2 W f 2 S, (6) where the symbol R denotes convolution and W 2 is the squared magnitude of the window function s Fourier spectrum. For the model of Eq. 4, one can show that, with the same assumption of a uniform distribution of the object coordinates x 0 over the picture area S, AV imsys AV x0 B f 2 AV imsys IM f 2 RO f 2. (7) 10 July 1997 Vol. 36, No. 20 APPLIED OPTICS 4817

3 With estimations 6 and 7 of the background power spectrum, the optimal filter can be implemented either as or as RO* f H opt f AV imsys IM f 2 W f 2 RO* f H opt AV imsys IM f 2 RO f 2. (9) Filtering schemes similar to those shown in expressions 8 and 9 were also discussed, although under different assumptions, in Refs Logarithmic Joint Transform Correlators For the filter of expression 9, signal filtering in a suboptimal adaptive correlator is described in the frequency domain by the formula IM f RO* f OUT f AV imsys IM f 2 RO f 2. (10) Let us now show that, with a logarithmic nonlinear transformation of the joint spectrum, the NLJTC of Fig. 1 approximates this filter. With the logarithmic nonlinearity, the transformed joint power spectrum OUT NLJTC at the output of this nonlinear device can be written as OUT f NLJTC log IM f RO f 2 log IM f 2 RO f 2 IM f RO* f IM* f RO f. (11) Since the size of the reference object is usually much smaller than the size of the input image, we can assume that, for the majority of the spectral components, IM f 2 RO f 2 IM f RO f. (12) With this assumption, OUT NLJTC f is approximately equal to OUT NLJTC f log IM f 2 RO f 2 IM f * RO f IM f 2 RO f 2 IM f RO* f IM f 2 RO f. (13) 2 When a JTC configuration is utilized, the two last terms displaying the correlation function are readily separated. The last term of expression 13 resembles expression 10 when it corresponds to the OPAC with estimation of the background image component power spectrum by expression 7 but without averaging AV imsys. Therefore, one can conclude that the nonlinear joint transform with the logarithmic nonlinearity correlator can be regarded as an approximation to the OPAC and therefore promises an (8) improved discrimination capability. Note that averaging AV imsys can be implemented by a kind of smoothing for instance, by a linear blur of the involved signal. To verify this conclusion, we conducted computersimulation experiments using sixteen pixel fragments of a pixel satellite photograph of an urban area the same images were used in Ref. 13 as a set of test images and a small circular spot with a diameter of approximately 5 pixels as the target object. The target object was embedded within each test image in such a way that a sector of 5 5 pixels at the center of each image was substituted for by the target image. Arrangement of the input images and the target object, which was used as an input for the JTC, is shown in Fig. 2 a. Pairwise arrangement of the test images allowed us to make experiments with two images of the set in parallel. To reduce boundary effects we inscribed the input images and the test image into a uniform background with a gray level equal to the average gray level of the images. Experiments with this set of test images were aimed at investigation of the discrimination capability of adaptive nonlinear correlators in images with the same target and different backgrounds. To verify the results obtained we also performed an experiment with stereoscopic images Fig. 2 b. In this case, the target object was a pixel fragment of one image shown in the box in the lowerimage illustration in Fig. 2 b and the test image in which this fragment was to be located was the second upper image. When designing nonlinear correlators one should also take into account that the dynamic range of SLM s is always limited. This feature can be described as a pointwise nonlinear transformation of the form LDR x x lim x lim otherwise. (14) The parameter lim defines the degree of the dynamicrange limitation. An important practical issue in the design of the JTC s of Fig. 1 is the required resolution power of the TV camera that reads out the joint spectrum. It is well known that the space bandwidth product of an optical system is usually much higher than that of electronic imaging devices such as TV cameras. This casts a restriction on the space bandwidth product of NLJTC s. On the other hand, the estimation of the background-image power spectrum required for the implementation of an OPAC according to expression 7 assumes that the input-image power spectrum is smoothed by convolution with an appropriate window function, as was mentioned above. Experiments reported in Ref. 13 provide evidence that appropriate spectrum smoothing may provide substantial improvement of the nonlinear correlators discrimination capabilities. In the setup shown in Fig. 1, the limited resolution power of the TV camera as reported in Ref. 11 causes smoothing of the 4818 APPLIED OPTICS Vol. 36, No July 1997

4 Fig. 3. Frequency response of the blur operator blr used in the experiments. into this issue was also included in the computer simulation. The computer model implemented the following signal transformation: coroutput IFT(CONV LDR log FT corinput 2, h blr ) 2. (15) Fig. 2. a Example of the input image of the JTC. b Stereoscopic images used in the experiments. joint spectrum. Averaging the joint-spectrum image power spectrum IM f RO f 2 by convolution with the point-spread function of the TV camera results in smoothing both the IM f 2 RO f 2 and the IM f RO* f terms in Eq. 11. Whereas the former term s smoothing is what one needs for the spectrum estimation and may increase the correlator s discrimination capability, the latter may have a negative effect. Therefore, one might expect that there exists an optimal smoothing degree. The investigation Here, coroutput and corinput are the output and input images, respectively, of the correlator, FT and IFT are direct and inverse discrete Fourier transforms, respectively, that were used as approximations to the optical Fourier transform, LDR is the dynamic-range limitation transformation Eq. 14, CONV...,h blr is a blur fold convolution operator with a point-spread function h. Frequency responses of the convolution operator for values of the blur parameter of blr 0, 1,..., 11 are shown in Fig. 3. The parameter lim of the dynamic-range limitation was selected as the 0.1 lim 1 th fraction of the signal maximum with lim 1, 2,..., 12. Thus, the case in which lim 1 corresponds to no dynamic-range limitation, and the case in which lim 12 corresponds to limitation at the level of th of the signal maximal value. Two performance measures of the correlator s discrimination capability were computed in each experiment: the ratio of the object s signal maximum exclusive of the output-signal dc component to the output signals standard deviation over the background area in the correlation plane SNRV, and the ratio of the object s signal maximum to the highest signal maximum over the background area in the correlation plane SNRM, 14 both exclusive of the output-signal dc component. The experimental results are presented in Figs July 1997 Vol. 36, No. 20 APPLIED OPTICS 4819

5 Fig. 4. Plots of the average SNRV and SNRM at the output of the logarithmic JTC versus the limitation threshold. Fig. 6. Illustration of the similarity between the logarithmic and the 1 k th law nonlinearities. and 5. The plots in Fig. 4 represent values of the SNRV and SNRM for the logarithmic JTC as functions of the dynamic-range limitation parameter lim averaged over the set of test images. They show that the logarithmic JTC is not very sensitive to the limitations up to 10 7 of the entire dynamic range of the joint spectrum. Similarly averaged plots in Fig. 5 show how the discrimination capability of the logarithmic JTC depends on the blur of the joint spectrum. One can observe the optimum degree of blur that tells that, although the expected gain in the correlator s discrimination capability derived from optimal jointspectrum smoothing is not high, the discrimination capability remains high over a rather broad range of the degree of smoothing. Note that the optimum in the SNRM is less pronounced than that in the SNRV. In two of 16 test images no optimum was observed, and the SNRM monotonically, although very slowly, decreased with the increase of the blur parameter blr. One can conclude from these data that the requirement of the resolution power of the TV camera is not very critical: The spatial bandwidth of the camera may be times less than that of the optics without noticeable losses in the correlator s discrimination capability. 4. Nonlinear Joint Transform Correlators with the 1 k th Law Nonlinearity The distinctive feature of the logarithmic signal transform is that it compresses the signal s dynamic range. Similar compression can be also achieved by the 1 k th law nonlinearity k, (16) where k 1. This similarity is illustrated in Fig. 6 where the logarithmic and 1 k th law nonlinearities are plotted together after normalization by a constant. Therefore, one can expect that NLJTC s with the 1 k th law nonlinearity and k 1 will perform nearly as well as does the logarithmic JTC. The simulation experiments with NLJTC s with the 1 k th law nonlinearity were carried out with the same computer model as that for the logarithmic nonlinearity, except the logarithmic transformation was substituted for by the 1 k th law transformation and spectrum smoothing was not applied: coroutput IFT(LDR FT corinput 2 k ) 2. (17) Fig. 5. Plots of the SNRV and SNRM at the output of the logarithmic JTC versus the blur parameter blr in the Fourier plane. The corresponding averaged experimental data for the SNRV are plotted in Fig. 7 a for the set of test images of Fig. 2 a and in Fig. 7 b for the stereoscopic images of Fig. 2 b as functions of the nonlinearity index k for the dynamic-range limitation parameter 4820 APPLIED OPTICS Vol. 36, No July 1997

6 Fig. 8. Plot of the SNRV at the output of the BJTC SNRV bin versus the fraction of the joint-spectrum energy under the binarization threshold for the set of test images. provided there is a proper selection of the nonlinearity index of k 1. A trade-off exists between the nonlinearity index k and the dynamic-range limitation parameter lim: A higher degree of dynamic-range limitation requires lower values of k. With this trade-off, the discrimination capability remains practically the same. The discrimination capability of the NLJTC with the 1 k th law nonlinearity does not depend noticeably on k, provided that k exceeds a minimal value determined by the dynamic-range limitation level. Fig. 7. Average SNRV SNRV av at the output of the JTC with the 1 k th law nonlinearity versus the nonlinearity index k for a the set of test images and b the set of stereoscopic images SNRV stereo. lim 1,..., 10. The results for the SNRM not shown are similar. These results show that NLJTC s with the 1 k th law nonlinearity may have a considerably improved discrimination capability in comparison with that of the JTC without nonlinear transformation of the joint spectrum case of k 1. In terms of the SNRM, the observed averaged gain is exceeded 3 times. Comparison of the corresponding data for the NLJTC s with the 1 k th law nonlinearity and the logarithmic JTC shows that, with an appropriate selection of the nonlinearity index k and the dynamic-range limitation threshold lim, the former performs slightly better. As for the logarithmic JTC, the discrimination capability of the NLJTC with the 1 k th law nonlinearity is not sensitive to the limitation threshold, 5. Binary Joint Transform Correlators Many nonlinear media are binary, that is, practically they can be in only two states: transparent or opaque. This feature can be described as hard limiting binarization : LDR h x 0 1 x lim otherwise. (18) From the experiments with the 1 k th law nonlinearity, one can conclude that even the simple dynamic-range limitation alone may substantially improve the NLJTC discrimination capability, provided the limitation threshold is properly chosen. This fact allows us to assume that BJTC s with the hard limitation according to Eq. 18 may also have a sufficiently high discrimination capability. The simulation results confirmed this conjecture. The simulation was carried out with the same set of test images according to the model: coroutput IFT(LDR h FT corinput 2 ) 2. (19) The simulation results are plotted in Fig. 8. The graphs represent the SNRV for all 16 images of the test set as functions of the fraction of the joint- 10 July 1997 Vol. 36, No. 20 APPLIED OPTICS 4821

7 spectrum energy under the binarization threshold. They demonstrate the variability of the SNRV for different images, and at the same time they clearly show that, for all the test images, there exists an optimal value of the binarization threshold for which the correlator s discrimination capability reaches levels close to those achievable for the logarithmic and 1 k th law NLJTC s. This optimal value corresponds to the binarization threshold in the range of approximately th to th fraction of the entire energy of the joint spectrum. This range is of the same order of magnitude as that of the ratio of the area occupied by the target object 5 5 pixels to the area of the input image pixels, which one would expect. One can also see that the discrimination capability of the BJTC is relatively tolerant to reasonable deviations of the binarization threshold from its optimal value. Note that a similar conclusion can be reached from the results reported in Ref. 4. The achievable SNRV and SNRM for BJTC s are lower than those in the optimal NLJTC but still are much higher than those for the matched filter lim 1, k 1 in Fig. 7 a. 6. Conclusion We have shown that different types of nonlinear transformations of the joint spectrum in JTC s can be used successfully for substantial improvement of the correlators discrimination capability, provided an appropriate choice of the transformation parameters: logarithmic transformation, the 1 k th law transformation in combination with limitation of the dynamic range, and thresholding. We also have demonstrated that a moderate blur of the joint spectrum before its nonlinear transformation in NLJTC s is admissible and may even slightly improve their discrimination capability, that all types of NLJTC s discussed are tolerant to reasonable deviations of nonlinearity parameters the limitation of the dynamic range for logarithmic NLJTC s, the nonlinearity index k and the limitation of the dynamic range for NLJTC s with the 1 k th law nonlinearity, and the binarization threshold for BJTC s. This permits a substantial weakening of the requirements on the electronic components of NLJTC s. Direct correspondence to L. Yaroslavsky, Faculty of Engineering, Department of Intedisciplinary Studies, Tel-Aviv University, Tel-Aviv 69978, Israel. address: yaro@eng.tau.ac.il. L. P. Yarosslavsky is on leave from the Institute of Information Transmission Problems, Russian Acadamy of Sciences, Bolshoy Karetney 19, Moscow , Russia. References 1. B. Javidi, Nonlinear joint power spectrum based optical correlation, Appl. Opt. 28, B. Javidi and C.-J. Kuo, Joint transform image correlation using a binary spatial light modulator at the Fourier plane, Appl. Opt. 27, F. T. S. Yu and T. Nagata, Binary phase-only joint transform correlator, Microwave Opt. Technol. Lett. 2, W. B. Hahn, Jr., and D. L. Flannery, Design elements of binary joint transform correlation and selected optimization techniques, Opt. Eng. 31, P. Refregier, V. Laude, and B. Javidi, Nonlinear jointtransform correlation: an optimal solution for adaptive image discrimination and input noise robustness, Opt. Lett. 19, H. Inbar, N. Konforti, and E. Marom, Modified joint transform correlator binarized by error diffusion. I. Spatially constant noise-dependent range limit, Appl. Opt. 33, H. Inbar and E. Marom, Modified joint transform correlator bynarized by error diffusion. II. Spatially variant range limit, Appl. Opt. 33, L. P. Yaroslavsky, The Theory of Optimal Methods for Localization of Objects in Pictures, Vol. 32 in Progress in Optics Series Elsevier, Amsterdam, 1993, pp B. Javidi and J. Wang, Optimum filter for detection of a target in nonoverlapping scene noise, Appl. Opt. 33, A. Mahalanobis, B. W. K. Vijaya Kumar, S. Song, S. R. F. Sims, and J. F. Epperson, Unconstrained correlation filters, Appl. Opt. 33, H. Inbar and E. Marom, A priori and adaptive Wiener filtering with joint transform correlators, Opt. Lett. 20, H. Inbar and E. Marom, New interpretations of Wiener filters for image recognition, J. Opt. Soc. Am. A 13, L. P. Yaroslavsky, Optical correlators with k th law nonlinearity: Optimal and suboptimal solutions, Appl. Opt. 34, The SNRM is sometimes called the peak-to-clutter ratio see, for instance, Ref APPLIED OPTICS Vol. 36, No July 1997