Different Levels of Detail Display for Exposure Fusion Using Local Laplacian Filtering

Transcription

1 JOURNAL OF COMPUTERS, VOL. 9, NO. 9, SEPTEMBER Different Levels of Detail Display for Exposure Fusion Using Local Laplacian Filtering Chunmeng Wang an Changhe Tu School of Computer Science an Technolog Shanong Universit Jinan, China Abstract Exposure fusion is an efficient metho for irectly fusing multi-exposure images into a high-quality low ynamic range image, without the high ynamic range (HDR) prouction an tone mapping process. The previous exposure fusion methos only prouce an image that contains a fixe amount of etails, which can not satisfy further emans for more etail information. We introuce Local Laplacian Filtering (LLF) for ege-aware image processing to multi-resolution Laplacian pyrami weighte blening metho. Our metho not only preserves etails in the overexpose an unerexpose areas, but also shows multi-level etail manipulation by ajusting a simple parameter. Compare with previous methos, our interactive metho has a greater flexibility on etail isplay for the nees of ifferent users. Inex Terms high ynamic range image, HDR, exposure fusion, Laplacian pyrami, image enhancement I. INTRODUCTION The ynamic range of the real worl ramatically excees the capability of the common image isplay evices, so a single common image cannot show all the information an etails in the bright an ar areas. Besies, the current HDR capture evices [1] [] [3] are still not popular because of their high prices an harware requirements. Therefore, we have to capture a stac of ifferent expose images with a common camera, HDR image is generate by first recovering a HDR raiance map from these images [4] an then maing it isplayable by tone mapping algorithms [5] [6]. However, the above typical HDR process nees the camera response curve calibration an complicate tone mapping algorithms. So we can irectly fuse ifferently expose low ynamic range (LDR) images into a wellexpose tone-mappe lie image. This strategy is calle exposure fusion an it simplifies the HDR pipeline. Several exposure fusion methos have been propose in the past ecae. Goshtasby [7] propose a bloc-base fusion metho that selects the bloc contains the most information by computing the entropy. Raman an Chauhuri [8] propose a exposure fusion metho with the technique of ege-preserving bilateral filtering. Wei Zhang et al. [9] compose multi-exposure images with Manuscript receive January 6, 014; revise March 5, 014; accepte April 9, 014. the graient magnitue as the quality measures. The stateof-the-art metho was propose by Mertens et al. [10], which etermines the weight map of each image accoring to contrast, saturation an well-exposeness. On the basis of weight maps, they aopt multi-resolution Laplacian pyrami weighte blening to eliminate seams an halos an achieve perfect results in most cases. All the methos above prouce a certain fusion result with a changeless information an etails. However, it is uncertain that the fixe fusion result image is just what we nee because it is a ifficult problem to juge what result is best. For example, sometimes we expect as many etails as possible, but sometimes we focus more on overall bright an ar effect. Instea of evaluating the quality of HDR image base on complex methoology [1], we propose a new strategy that lets users choose the etail isplay level as they want freely by ajusting a simple parameter. Our metho is base on multi-resolution Laplacian pyrami weighte blening as [10], but we use Local Laplacian Filtering to reconstruct the Laplacian pyrami coefficients, an use a remapping function to manipulate the image etail level controlle by an user parameter. Our metho can generate clear an etail-preserving fuse result without halos an have a greater flexibility on etail isplay for the nees of ifferent users. II. OUR APPROACH Assume that there are N ifferent expose input images (I 1 I N ). These input images are all capture by a common camera from a static scene with no moving object or camera motion an have been aligne. Base on the multi-resolution exposure fusion, we use Local Laplacian Filtering (LLF) to construct the Laplacian pyrami coefficient one by one for each input image. The technique of Local Laplacian Filtering was propose by Paris et al. [13] for ege-aware image processing an has been applie to ege-preserving smoothing, etail enhancement an tone mapping. We combine this technique with multi-resolution exposure fusion to obtain multi-level etail isplay with a simple parameter ajustment accoring to ifferent etail emans. Our metho is ifferent from common methos of contrast enhancement because we implant the etail manipulation process to the generation of Laplacian pyrami coefficients. The overview pipeline of our approach is showe in Fig ACADEMY PUBLISHER oi: /jcp

2 18 JOURNAL OF COMPUTERS, VOL. 9, NO. 9, SEPTEMBER 014 Figure 1. The pipeline of our approach. A. Buil the Laplacian pyrami by Local Laplacian Filtering For each coefficient, we generate an intermeiate image processe locally by a point-wise remapping function R(I') g,σ, an then compute Laplacian pyrami of the intermeiate image, finally copy the corresponing coefficient to the output pyrami by L I y = L R I, y. (1) { } { ( )} ( ) ( ), g,σ where L{I(,} is the Laplacian pyrami of the -th image on the -th level. The size of intermeiate image I' is K K, an K is relate to the current pyrami level by K= The meaning of g an σ is explaine in the next section. Figure. S-shape function to manipulate etails. The value g 0 is the Gaussian pyrami coefficient of the 0-th level an the parameter σ is use to istinguish eges from etails. σ=0.05 for blac curve, σ=0.5 for green curve, σ= for blue curve an σ=1 for ash line. B. Detail Manipulation by S-shape Function The etail manipulation is controlle by the remapping function R(I') g,σ. For each coefficient () of the -th image, this remapping function epens on the local image value from the Gaussian pyrami g = G 0 ( an the parameter σ is use to istinguish eges from etails. Intensity variations smaller than σ shoul be consiere fine-scale etails an larger variations are eges. As a center point for this function we use the Gaussian pyrami coefficient of the 0-th level, which represents the image intensity at the location an scale where we compute the output pyrami coefficient. In this paper, we aim to manipulate etails an prouce multi-level etails, so we moify the image fine-scale etails only an leave the eges not treate. Therefore, the parameter σ controls what magnitue of variations shoul be consiere eges that are preserve constant. Large values allow the filter to manipulate larger portions of the etails an yiel larger visual changes. For the RGB color image in our metho, we efine etails as colors within a ball of raius σ centere at g an eges as the colors outsie it. That is, if i - g < σ, it is regare as etail, where i is a three-imensional vector for the RGB channels. So, the remapping function is efine as i g i g R ( i) = g + σ f ( ) () i g σ This smooth mapping function moifies the fine-scale etails by altering the amplitue aroun the vector g. The value σ is set to 0.5 in our metho. To moify the etails of an image we use an S-shape power curve function for the local manipulation of image contrast by f(δ)=δ α, where α is a user-efine parameter with the value of α>0. When α>1, the etails are ecrease. An when 0<α<1, the etails are increase. As α is smaller, the amplitue of etail increasing is larger (Fig. ). If users prefer richer an clearer etails, they can ajust this parameter closer to 0. When α=1, the etail is not moifie. In orer to reuce noise an artifacts, we limit the smallest Δ amplifie by f ( Δ ) = λδ α + ( 1- λ) Δ (3) where λ is a smooth step function equal to 0 if Δ is less than 1% of the maximum intensit 1 if it is more than %, with a smooth transition in between. In its basic form etail manipulation is applie at all scales, but one can also control which scales are affecte by limiting processing to a subset of the pyrami levels. The level of Laplacian pyrami controls the frequency of the etails that are manipulate. The low level pyrami processes the high frequency information an the high level pyrami processes the low frequency information. We only moify the lower levels of pyrami (0 to levels in this paper) by LLF to manipulate the high frequency etails an remain other levels as their original form. 014 ACADEMY PUBLISHER

3 JOURNAL OF COMPUTERS, VOL. 9, NO. 9, SEPTEMBER C. Weighte Fusion After generating the Laplacian pyrami coefficient of each input image, we aopt the multi-resolution exposure fusion metho [10] to eliminate seams an halos. This metho irectly fuses multiple exposure images to a highquality etail-preserving result without raiometric calibration of camera response function an the exposure setting parameters. This metho uses the weighte prouction of three quality measures incluing contrast, saturation an well-exposeness as the weight map by W ) wc ws we = ( C ) ( S ) ( E (4) where w C, w S an w E are corresponing weighting exponents (w C = w S = w E = 1 in this paper). Then we buil the normalize Gaussian pyrami of each weight map, an obtain the fuse Laplacian pyrami for each level separately by where L { I} is the Laplacian pyrami of the -th image on the -th level, G { Wˆ } is the normalize Gaussian pyrami of the corresponing weight map W x, y,. Finall the Laplacian pyrami L { F} is collapse to generate y the fuse image F. III. EXPERIMENT RESULTS All of our metho results are compute on a.33ghz Intel core Duo CPU, 1.96 GB memory. We have one large number of experiments to prove the effectiveness of our metho. For example, Fig. 3 shows three groups of ifferent expose HDR scenes. In each scene, we show 4 ifferent fusion results of ifferent parameter setting. Our metho not only preserves the etails of bright an ar areas, but also can generate ifferent levels of etail manipulation results by ajusting the user-efine (a) (b) (c) Figure. (a), (b), (c) are three groups of ifferent expose scenes. In each group, we show four ifferent fusion results of ifferent parameters. From left to right: α=, α=1, α=0.5, α=0.05. N L { F} y L{ I} G{ W ˆ } = 1 = (5) parameter α. When the parameter α becomes smaller to 0, the etails an the fine information are more visible, an the efinition of the result image is higher as α changes, such as the rawing boars on the wall (Fig. 3(a)), etails on the groun (Fig. 3(b)), clou an grass (Fig. 3(c)). So our metho has avantage over previous exposure fusion 014 ACADEMY PUBLISHER

4 184 JOURNAL OF COMPUTERS, VOL. 9, NO. 9, SEPTEMBER 014 methos for greater flexibility an etail iversity. We aopt the average graient to measure the amount of etails iversity. The average graient is greater, the more an clearer etails the image contains. The average graient is efine by g = H W x= 1 y= 1 xf( H W + y F( where H an W are the image height an with, respectively. xf( an yf( are the average horizontal an vertical erivatives of (, respectively. TABLEI shows the average graient of ifferent α for the fuse images in our paper. We can see that the average graient becomes larger when α changes smaller, which prove the theory of our approach by quantitative analysis. More fusion results are showe in Fig. 4. Our algorithm is implemente easily without camera calibration an tone mapping operations. It taes less than one minute to fuse three exposures with resolution of 1 megapixel using single threa. Fuse images TABLE I. AVERAGE GRADIENT VARIATION OF DIFFERENT PARAMETERS Average graient of ifferent α α= α=1 α=0.5 α=0.05 Fig. 3(a) Fig. 3(b) Fig. 3(c) (6) IV. CONCLUSION AND FUTURE WORK In this paper, we exten the traitional exposure fusion metho to a more flexible an ajustable version. We aopt Local Laplacian Filtering to generate an remappe intermeiate image an rebuil the low levels of Laplacian pyrami coefficients. The remapping function is an S-shape power curve function controlle by a simple user parameter to manipulate etails of ifferent levels. Our metho not only preserves the overexpose an unerexpose etails of the whole scene without halos, but also has a greater flexibility an interactivity because users can get their esire fuse image by ajusting the parameter easily. We woul lie to achieve the real-time fusion process with GPU implementation or a multi-core architecture in the future wor to improve the efficiency because our algorithm is highly ata parallel. We also want to esign other functions to manipulate Laplacian pyrami coefficients from more aspects besies the image etails because users not just pay attention to the image etails. For example, the richness of colors, the bright an ar visual effects an so on. We plan to provie more parameters to give users more freely to customize their esire fusion type. ACKNOWLEDGMENT Figure 3. More fusion results with ifferent parameter setting (α=1 for the left image an α=0.05 for the right image). This research was supporte by NSFC-Guangong Joint Fun Project (U ) an Grauate Innovation Founation of Shanong University (GIFSDU) yzc1140. REFERENCES [1] M. Aggarwal an N. Ahuja, Split aperture imaging for high ynamic range, Int. J. Comput. Vision, vol. 58, no. 1, pp , 004. [] H. Seetzen, W. Heirich, W. Stuerzlinger, G. War, L.Whitehea, an M. Trentacoste, High ynamic range isplay systems, in Proc. SIGGRAPH 04, ACM, 004, pp [3] M. D. Tocc, C. Kiser, N. Tocci, an P. Sen, A versatile hr vieo prouction system, in Proc. SIGGRAPH 11, ACM, 011, pp [4] P. E. Debeve an J. Mali, Recovering high ynamic range raiance maps from photographs, in Proc. SIGGRAPH 97, ACM, 1997, pp [5] F. Duran an J. Dorse Fast bilateral filtering for the isplay of high-ynamic-range images, in Proc. SIGGRAPH 0, ACM, 00, pp [6] R. Fattal, D. Lischinsi, an M. Werman, Graient omain high ynamic range compression, in Proc. SIGGRAPH 0, ACM, 00, pp ACADEMY PUBLISHER

5 JOURNAL OF COMPUTERS, VOL. 9, NO. 9, SEPTEMBER [7] A. A. Goshtasb Fusion of multi-exposure images, Image Vision Comput., vol. 3, no. 6, pp , 005. [8] S. Raman an S. Chauhuri, Bilateral Filter Base Compositing for Variable Exposure Photograph In Short Papers, Eurographics, 009, pp.1 4. [9] W. Zhang an W. K. Cham, Graient-irecte composition of multi-exposure images, in Proc. IEEE Conference on Computer Vision an Pattern Recognition, CVPR, 010, pp [10] T. Mertens, J. Kautz, an F. V. Reeth, Exposure fusion: A simple an practical alternative to high ynamic range photograph In Short Papers, Eurographics, 009, pp.1 4. [11] S. Raman an S. Chauhuri, Bilateral filter base compositing for variable exposure photograph Comput. Graph. Forum, vol. 8, no. 1, pp , 009. [1] J. Kuang, H. Yamaguchi, C. Liu, G. M. Johnson, an M. D. Fairchil, Evaluating HDR renering algorithms, ACM Trans. Appl. Percept, vol.4, no., 007. [13] S. Paris, S. W. Hasinoff, J. Kautz, Local laplacian filters: ege-aware image processing with a laplacian pyrami, in Proc. SIGGRAPH 11, ACM, 011, pp Chunmeng Wang receive the B.S. egree in Department of Software Engineering from Shanong Universit Jinan, China, in 009, an he is currently a Ph.D. stuent in Department of Computer Science an Technology at Shanong University. His research interests inclue computer graphics, image processing, high ynamic range image. Changhe Tu receive the B.S. an M.S. egrees in Department of Computer Science from Shanong Universit Jinan, China, in 1990 an Since 1993 he has jointe the faculty of Department of Computer Science an Technology at Shanong University an receive the Ph.D. egree. From 001 to 00, he wore on collaborative research with the Department of Computer Science of Hong Kong University in the fiel of geometric calculation an computer graphics. He is a Professor in Department of Computer Science an Technology at Shanong University. He is the program committee of GMP 006, 007, 006, PG 007, Entertainment Computing 007 CAD/Graphics 007, an the program committee chair of 14th CAD/CG. His research interests inclue geometric calculation, computer graphics, image processing an computer aie geometric esign (CAGD). 014 ACADEMY PUBLISHER