Study on Fusion Algorithm of Multi-source Image Based on Sensor and Computer Image Processing Technology

Transcription

1 Sensors & Transducers 3 by IFSA Study on Fusion Agorithm of Muti-source Image Based on Sensor and Computer Image Processing Technoogy Yao NAN, Wang KAISHENG, 3 Yu JIN The Information oom of Eectric Power esearch Institute in Jiangsu Eectric Power,, China Power Suppy and Mai Tunnes Company in Yangzhou, 55, China 3 Computer Science and Technoogy, Naning University,, China,, 3 Te.: , , , E-mai: nanyan7@63.com, @63.com eceived: 3 October 3 /Accepted: November 3 /Pubished: 3 December 3 Abstract: Muti-source image fusion as a research area with image as its obect in muti-source information fusion, is a combination of sensor, computer image processing, artificia inteigence and other discipines. This artice mainy focused on wor summary and new ideas based on muti-resoution decomposition and fusion method. Using variationa partia differentia equation for the fusion exporation based on feature eve fusion study of regiona and muti-resoution decomposition. Discussed how to reaize the obective evauation of image fusion performance and the specific appication of mutipe source image feature eve fusion in road target recognition. Copyright 3 IFSA. Keywords: Sensor technoogy, Computer image technoogy, Mutipe image fusion.. Introduction With the deveopment of image sensor technoogy, more and more sensors are used in various fieds. The increase of number of sensors eads to rapid increase in the amount of data received and presents diversity. Traditiona information processing methods cannot satisfy the new requirements. Therefore, muti-source information fusion technoogy emerges at the right moment in this context. Muti-source information fusion refers to carry on a comprehensive processing with muti eve s, muti-aspects and mutiayers of the muti-source information from mutipe sensors, so as to obtain more abundant, accurate and reiabe usefu information []. Its main idea is through the adoption of a certain agorithm, to fuse two or more source images with compementary or redundant features into a new image. Mae the image after fusion maximize the use of compementary information, reduce redundancy, to gain higher carity and understandabiity. Muti-source image fusion refers to the comprehensive processing of information in mutipe carriers in order to achieve a certain purpose. The scope of muti-source image fusion is very extensive. It can be concuded as a muti-source image fusion process. The genera concusion is obtained after the integration and abstraction of severa specia cases. Therefore, there exists essentia contact between inductive reasoning and information fusion. And inductive reasoning is the source of a the science and technoogy. In recent years, muti-source image fusion has received great attention in many fieds and muti-source image fusion technoogy deveoped rapidy. Artice number P_667 67

2 . Mutipe Image Fusion Technoogy Currenty, the imaging sensor mainy consists of infrared imaging, radar imaging and optica imaging. Infrared imaging has the characteristics of penetrating smoe, wor doube tides, etc, therefore, it is particuary vaued at cose range imaging tracing system. Active radar imaging sensor has the abiity to wor with a-weather, but due to its compex structure and arge equipment, its appication in many occasions is imited. And radar images are easiy infuenced by the ground cutter []. Visibe ight CCD imaging system is of sma size and simpe structure and with high resoution, strong anti-amming capabiity, but is more easiy affected by weather conditions. In contrast, muti-source image fusion integrated sensor, computer image processing, signa processing, and artificia inteigence and other discipines. Its main idea is through the adoption of a certain agorithm, to fuse two or more source images with compementary or redundant features into a new image. Mae the image after fusion maximize the use of compementary information, reduce redundancy, to gain higher carity and understandabiity. And provide more effective information for further image processing such as image segmentation, obect detection and recognition, batte damage assessment and understanding [3]. Fig. is redundant and compementary information between source images. Muti-source image fusion is not a simpe superposition, it produces a new image which contains more vaue information. Image fusion can not ony fuse images of the same ind of sensors, but aso can fuse images of different types of sensors. But sensor of the same type is with the same physica significance and the images are with the same properties. Images from different types of sensors have greater compementarity, So the meaning of the fusion of heterogeneous images is more apparent. Muti-source image fusion can improve the performance of the sensor system from the foowing aspects: ) Extend system coverage. Measured obect or environmenta information can be obtained more accuratey. And information received has higher accuracy and reiabiity. ) By compementation between each sensor, independent characteristic information is obtained which is beyond the abiity of singe sensor. 3) According to the prior nowedge of the system, by fusion processing, cassification, identification and decision-maing can be accompished. 4) Improve the system reiabiity. Because the data are from mutipe (ind of) sensors, when one or more sensors fai or mae mistaes, system continues to wor. Muti-source image fusion as a specific research area of muti-source information fusion, whie having the characteristics of information fusion, it aso exerts stricter requirements on preprocessing such as the image registration. Image fusion agorithm maes the fusion image contain the important information in its origina image. Image fusion agorithm shoud not introduce any miseading for human visua perception or error message of image processing [4]. 3. Basic Principe, System Structure and Agorithm of Muti-Source Image Fusion 3.. Basic Principe and System Structure of Muti-source Image Fusion Image fusion is an emerging technoogy with the integration of sensor, signa processing, image processing and artificia inteigence. Genera processing fow of muti-source image fusion is: Image preprocessing, image registration, image fusion, feature extraction and recognition and decision maing. Based on the stage of fusion in the processing fow and abstraction degree of the information, muti-source image fusion is generay divided into three eves: Pixe eve fusion, feature eve fusion and decision eve fusion [5]. The frame structure of decision eve fusion is shown in Fig Cassification of the Agorithm Fig.. Diagram of redundancy and compementary information between source images. The agorithm of muti-source image fusion at pixe eve can be divided into six types: inear weighted fusion, fase coor image fusion, image fusion based on the moduation, statistics-based image fusion, neura networ-based image fusion and image fusion based on muti-resoution decomposition. 68

3 d W s V d W d3 W 3 s V s s3 V3 V Fig. 4. Parse tree of fiter ban for muti-resoution anaysis. Fig.. Frame structure of decision eve fusion. 4. Muti-source Image Fusion Based on Muti-resoution Decomposition 4.. Orthogona Waveet Transform Digita signa in space can be proected onto each subspace. In many cases, it is much easier to anayze signa proection in each subspace than to anayze the origina signa. Signa muti-resoution anaysis refected in frequency space is shown in Fig. 3. H () If () t L ( ) meets the "permissibe" ˆ ( ) C, then () t is condition: caed a basic waveet. For a basic waveet () t, square integrabe space function f () t L ( ), its waveet transform definition is: - t-b W, f ( a, b) a f( t) ( ) dt,( a, b, a) () a - Of which, a> is the scae factor, b refects the dispacement, its vaue can be positive or negative, t is the continuous variabe. Therefore, it is nown as the continuous waveet transform. Discrete waveet () t, is: V V V () t a ( a t- b ), Z, Z () - -, V3 W 3 W W In the famiy of orthogona waveets structured by Daubechies, the decomposition and reconstruction fiter group is shown in Fig. 5. / 8 / 4 / Fig. 3 Muti-resoution frequency separation characteristics. Use a high-pass fiter g and a ow-pass fiter h to decompose the origina signa into two sub-bands. Then for the ow frequency sub-band, use ow-pass fiter and high-pass fiter recursivey to mae further decomposition [6]. Using a series of the FI fiter ban to reaize the above muti-resoution decomposition, as shown in Fig Fast Waveet Transform-Maat Agorithm Maat based on muti-resoution anaysis, puts forward the famous fast waveet transform agorithmmaat agorithm. By the preceding muti-resoution anaysis, expanded form of random signa f(t) in space is: ( ) ( ) (), ( - ) f t c t (3) 69

4 Of which, c, is scae factor, discrete approximation of f(t) on space V. d, is a waveet coefficient, discrete vaue of f(t) on space W i.e., the waveet transform W, f (, ) we require. (a) ow pass decomposition fiter { /, ( ),, ( ) ( ) ( ) c f t t f t t dt /, ( ),, ( ) ( ) ( ) d f t t f t t dt estructuring agorithm is: c h(-) n c g( n) d,,, (5) (6) Fig. 6 is Maat pyramida decomposition and reconstruction agorithm based on DB4 waveet. The source signa is eeccum signa in Matab. (b) High pass decomposition fiter (c) Low pass reconstruction fiter (d) High pass reconstruction fiter Fig. 5. Waveet decomposition fiter group. Mae a decomposition of f () t V, proection to V, W respectivey. Then: /,, f () t c ( t) d (t - ) (4) Fig. 6. Maat decomposition and reconstruction structure chart Muti-Spectra Image Fusion Enhancement Based on Biorthogona Mutiwaveet Mutiwaveet theory is the deveopment of the unit-waveet theory. Structuray, Mutiwaveet is with greater degrees of freedom, and can meet the requirements of orthogonaity and symmetry simutaneousy. If an orthogona basis is repaced by two biorthogona base, the biorthogona mutiwaveet is obtained. Two biorthogona base are made up of two mutua dua scaing functions. One for the decomposition, and the other for reconstruction. For each scae function () t and the corresponding waveet function () t, sti meet doube dimension equations as foows: m - () t C(t - ), m- () t D(- t ) (7) 63

5 Biorthogona mutiwaveet is structured on the basis of ordinary GHM orthogona mutiwaveet, as shown in Fig ( b) ( t)_ ( t) ( c) ( t)_ ( t ( d) ( t) _ ( t a) ( t) _ ( ) ) ) ( t Fig. 7. Biorthogona mutiwaveet (GHW base) scae function. Good characteristics of muti-resoution anaysis in signa anaysis wins widespread attention in the fied of image fusion. At present, most of the fusion agorithm unfods under the framewor of mutiresoution anaysis. Image fusion method based on muti-resoution anaysis is the mainstream research direction in the fied of image fusion [7]. In this chapter we put forward the use of the properties of the biorthogona mutiwaveet transform, combining with the fusion strategy of seection of ow frequency coefficients, reaize muti-spectra image fusion enhancement. It is a try to reaize image fusion by muti-resoution anaysis. On the other hand, for the researchers in the fied of image fusion, it is a foundation for further research on fusion fied to have in-depth nowedge of the framewor of reaizing image fusion based on mutiresoution anaysis. 5. Image Fusion Based on Variationa Partia Differentia Equation 5.. Variationa Method and Euer-Lagrange Equation ( ) [ ] x (,, ) x J y y F x y y y y dx () When, ( ) arrives the extreme vaue. By necessary conditions of extreme vaue, it can be obtained that: x () J[ yy] ( F ) x yyfyy dx () Note by integration by parts, y y, and: xx xx d d F ydx F y y Fdx y Fdx dx dx x x - x x y y y x y x x x () x d () J ( Fy - Fy ) ydx (3) x dx Function y(x) which maes J[ y ] arrives extreme vaue shoud meet the equation: F y d - F y (4) dx This equation is caed Euer-Lagrange eqution of functiona J[ y ]. 5.. Optica System Imaging Features Band width of the system function corresponding to burred image is narrower than that of sharpy focused image. Fig. 8 iustrates the genera characteristics of the optica imaging system. H ( f ) Functiona extremum is as foows: [ ( )] x (,, ) x J y x F x y y dy (8) f f Of which, satisfy the boundary condition:, ( ) [, ] and F C y x C x x H ( f ) yx ( ) y, yx ( ) y (9) f f Assume yx ( ), and J arrives its extreme vaue, consider a series of curves with as its parameter: y yx ( ) y. y is a variation of y( x ). When, y=y(x) is the curve which arrives functiona extremum. Appy y into the equation and the foowing can be obtained: Fig. 8. Optica system imaging features Coor Fusion Mode The coor fusion mode based on HSV coor space is shown in Fig. 9. HSV coor space can be described 63

6 as a cone. It consists of three components which are respectivey V(Vaue), S(Saturation) and H(Hue). They are independent of each other. H represents the ange formed by rotation around the center axis in the cone with range of (, 36), with order of red, yeow, green, bue and then bac to red again. Saturation S depends on the proportion of with-coor composition and decoorization composition (gray) with a range of (, ). The bigger the coor composition, the greater the degree of saturation. The bigger the decoorization composition, the smaer the degree of saturation. V can be regarded as a characterization of coor image on the gray eve domain, which contains the main space detais of coor image. 6. egion-based Muti-Source Image Fusion 6.. Image Segmentation Based on Graph Theory Define a metric area and, with a boundary predicate: tureifd(, ) MID(, ) D (, ) { faseotherwise () Of which, D(, ) represents the difference between and, denoted by: MID, ) = min( ID( ) + /, ID( ) + / ) (8) ( V i,.. N H S i,,... N Fig. 9. Fusion mode of coor muti-focused image. To sove the probem of coor distortion, We put forward triange average agorithm based on significant weighted. Tona component of band source image is denoted by H, and Synthesis of tona component H, then: H N = N = = Arg( w sinh, w cosh ) (5) S N = i = w S (6) Coor component processing by this method can effectivey avoid the coor distortion. It can be appied to any number of coor source image fusion. Considering from the perspective of spatia domain, visua information obtained by the image observer is reaized through the change of the image contrast, that is, the edge or gradient of the image. So we according to the importance of different features refected by source image, design different weights based on the characteristic diagram strength. Then according to the corresponding weighting of contrast information of the source image, cacuate the principa component as target gradient fied of the fusion. So as to reaize the image fusion of characteristic eeping. That is to design a specia contrast in advance as the target of the fusion, variationa probem is then used to cacuate the fusion resuts cose to the target. i Interna differences of is denoted by ID(), Greatest weight edge e of minimum spanning tree in is: ID( ) max we ( ) (9) emst(, E) egiona image obtained by the source image segmentation generates a oint area image as foows: 6.. Fusion Strategy Based on egiona Characteristics and Muti-Scae Decomposition Muti-scae decomposition is simuation of the human eye perception process in computer vision research. Muti-scae decomposition decompose image from bottom to top. Each ayer of image is formed through a tempate fitering of the former ayer of image. And muti-scae decomposition can effectivey perform many basic image arithmetic, produce a set of ow pass or band pass images. Through the interconnection between the ayers, provides the connection between the oca and goba processing. So combination of muti-scae decomposition and regiona characteristics of fusion strategy wi obtain better fusion effect. Waveet anaysis is a good method of muti-scae decomposition. Usuay discrete waveet is adopted in the fusion to mae muti-scae decomposition. Considering the mutiwaveet has better signa processing features, such as short support set, orthogonaity, symmetry and vanishing moment, therefore, biorthogona mutiwaveet is seected to mae mutiscae decomposition. Based on the biorthogona mutiwaveet, We put forward fusion agorithm with a combination of regiona characteristics and muti-scae decomposition, as shown in Fig.. 63

7 () () () 3 rough representation of the source image. It may inherits some texture and gray eve characteristics of the source image. So in many of the fusion method, scae coefficient of the composite image usuay is the weighted average of the scae coefficient of the source image. Waveet coefficients are seectivey merged Compatibiity Test (a) regiona segmentation image of source image. () Simiarity of area of source image A and B is denoted by M ( ), then: [ C A ( x, y) CB ( x, y)] x, y M ( ) = () S( A, ) + S( B, ) () S( I, ) = [ C ( x, y)], I = A B x, y I, () (b) regiona segmentation image of source image. ( ) ( ) In the case of ow simiarity, points with arger coefficient vaues in the source image shoud be converted into the composite image. When simiarity is high, mae average of the coefficient vaue of the source image. Simiarity threshod is denoted by T. When T M( ), ( ) 4 ( ) 5 ( ) 3 C( A ), ifs( A, ) S ( B, ) C( F ) { C( ),otherwise When T M( ), B () (c) Synthetic regiona figure. Fig.. Generating sampe of synthetic regiona figure. wmax ( ) C( A ) + wmin ( ) C( B ), ifs( A, ) S( B, ) C( F ) = { wmin ( ) C( A ) + wmax ( ) C( B ),otherwise (3) Of which, wmax ( ) and wmin ( ) are respectivey the maximum and minimum coefficient weight, defined by: - ( ) wmin ( ) - ( M ), wmax ( ) - wmin ( ) -T (4) Fig.. Fusion agorithm fow with a combination of regiona characteristics and biorthogona waveet decomposition. Because of the differences between scae coefficient and waveet coefficient in the physica sense, different fusion rues are needed in the fusion process. The point with arger absoute vaue in the waveet coefficient corresponds to the dramatic gray eve change. Which are the edge, inear features and regiona boundaries in the image. Scae factor is a Overa, the threshod vaue is sma. Because when the simiarity of source image is sma, sma threshod is more accord with human visua system. In image fusion, peope tend to focus on the image of the actua target or area, rather than a singe pixe. Moreover, Pixe-based fusion is greaty affected by source image noise and source image registration precision. However, region-based feature eve fusion method is not sensitive to these infuences, and more fexibe fusion strategy can be put forward according to the target or regiona characteristics and effectivey retain significant information of source image. 633

8 6. Concusion The rapid deveopment of computer science and microeectronics technoogy maes more and more sensors appied to various fieds. And because of the increase of quantity of sensors, the amount of data recived has increased dramaticay and presents diversity. The traditiona information processing method aready can not satisfy the new requirements. So the muti-source image fusion technoogy is proposed. Muti-source image fusion technoogy has been widey regarded as a very vauabe processing method in the image appication fied. With the increase of muti-source image fusion demand, a main method of image fusion quaity evauation is subective assessment, i.e, the visua evauation method. The reaization of muti-source image fusion roughy incudes three ayers which are pixe eve, feature eve and decision eve. Among which the pixe eve fusion research is the most widey and deepy. There are mainy six impementation method which are inear weighted fusion, fase coor image fusion, moduation-based fusion, statistica-based fusion, neura networ-based fusion, and fusion based on muti-resoution decomposition. This paper maes simpe anaysis for some fusion ways. The study has shown that the current existent fusion method can effectivey sove many probems encountered in image fusion and some practica systems have been successfuy deveoped. eferences []. A. A. Goshtasby, Image Fusion: Advances in the state of the art, Information Fusion, Vo. 8, Issue, 7, pp []. G. Simone, A. Farina, F. C. Morabito, et a., Image fusion techniques for remote sensing appications, Information Fusion, No. 3,, pp [3]. Liu Gang, Muti-sensor image fusion research based on the mutiresoution, Jiao Tong University, Shanghai, 5. [4]. Liu Yanyan, esearch on ey technoogies in mutisensor data fusion, China Science and Technoogy University, 6. [5]. Ni Guoqiang, esearch and new deveopment of muti-band image fusion agorithm, Optoeectronic Technoogy & Information, Vo. 4, Issue 5,, pp. -7. [6]. Liu Gang, Jin Zhongiang, Sun Shaoyuan, Image fusion based on the expectation maximum agorithm, Laser & Infrared, Vo. 35, Issue, 5, pp [7]. Wang Haihui, Peng Jiaxiong, Image fusion based on mutiwaveet transform, Journa of Image and Graphics, Vo. 9, Issue 8, 4, pp Copyright, Internationa Frequency Sensor Association (IFSA). A rights reserved. ( 634