A STUDY OF EMBEDDED GRADIENT DOMAIN TONE MAPPING OPERATORS FOR HIGH DYNAMIC RANGE IMAGES. A Thesis. Presented to

Size: px
Start display at page:

Download "A STUDY OF EMBEDDED GRADIENT DOMAIN TONE MAPPING OPERATORS FOR HIGH DYNAMIC RANGE IMAGES. A Thesis. Presented to"

Transcription

1 A STUDY OF EMBEDDED GRADIENT DOMAIN TONE MAPPING OPERATORS FOR HIGH DYNAMIC RANGE IMAGES A Thesis Presented to The Graduate Faculty of The University of Akron In Partial Fulfillment of the Requirements for the Degree Master of Science Jian Liu December, 2013

2 A STUDY OF EMBEDDED GRADIENT DOMAIN TONE MAPPING OPERATORS FOR HIGH DYNAMIC RANGE IMAGES Jian Liu Thesis Approved: Accepted: Advisor Dr. Joan E. Carletta Dean of the College Dr. George K. Haritos Committee Member Dr. Robert Veillette Dean of the Graduate School Dr. George R. Newkome Committee Member Dr. Kye-Shin Lee Date Committee Member Dr. Firas Hassan Department Chair Dr. Alex De Abreu-Garcia ii

3 ABSTRACT A large class of techniques for image processing is based on manipulation of the gradients of an image. These techniques have high computational complexity, due to the need to solve an inverse problem, taking the form of a Poisson equation, to find the output image that best matches manipulated gradients. This thesis studies hardwarefriendly techniques, appropriate for implementation on a field-programmable gate array and embeddable inside a camera, for approximating this solution. Fattal s operator for the dynamic range compression of high dynamic range images is studied as an example application. A family of methods, inspired by Fattal s operator but with significantly lower computational complexity, solves the inverse problem in a moving window of small size. In this thesis, a study is conducted to understand the role that the boundary conditions and the size of the window play in the quality of the resulting output image and the cost of the hardware. The impact of using single-scale and multi-scale approaches to compute the manipulation factor needed for Fattal s operator is also considered. The result of the study is a better understanding of the tradeoff between image quality and hardware cost when designing an embedded implementation of Fattal s operator. iii

4 DEDICATION Dedicated to my parents, and Alison iv

5 ACKNOWLEDGEMENTS I would like to thank my committee members Dr. Joan E. Carletta, Dr. Robert Veillette and Dr. Dr. Kye-Shin Lee for their support and guidance throughout my Master s program. I would like to thank Dr. Firas Hassan for patiently helping me throughout the research work. I would also like to thank Dr. Joan Carletta for the help throughout the research work and shaping it into a good manuscript. I am truly thankful for the assistance provided by the Department of Electrical and Computer Engineering for supporting me as a graduate assistant. I appreciate Mrs. Gay Boden for her help during my time at the University of Akron. I would also like to thank my fellow graduate students and friends, Joe Davis, Pongpachara Limpisathian, Jason Lin for making my time in Akron memorable. I thank my parents Juefu and Hong Liu, my wife Alison for always being there for me. v

6 TABLE OF CONTENTS Page LIST OF TABLES... viii LIST OF FIGURES... ix CHAPTER I. INTRODUCTION...1 II. RELATED WORK Reinhard s Operator The Retinex Operator Fattal s Operator III. BACKGROUND The Poisson Solvers Poisson Solvers with Zero Dirichlet Boundary Condition Poisson Solvers with Neumann Boundary Condition TMO Quality Assessment IV. IMPLEMENTED TONE MAPPING OPERATORS Tone Mapping Operator with Zero Dirichlet Boundary Condition Gradient Transformation and Gradient Manipulation (Steps A through D) Inverse Gradient Transformation (Step E) Tone Mapping Operator with Neumann Boundary Condition Gradient Transformation and Gradient Manipulation (Steps A through D) Inverse Gradient Transformation (Step E) The Estimation and Correction of DC Value The DC Value in the Input Window vi

7 4.3.2 The DC Value Estimation The DC Value Correction Parameters for the Implemented TMOs V. SIMULATION RESULTS Global versus Local Poisson Solvers Single-Scale versus Five-Scale Manipulation Factor Computation Zero Dirichlet Boundary Condition versus Neumann Boundary Condition The Effects of Moving Window Size VI. HARDWARE ESTIMATION Gradient Transformation Calculation of One-Norms Calculation of Parameter Gradient Manipulation Inverse Gradient Transformation for Center Pixel DC Value Estimation or Correction Overall Hardware Cost for Each Local TMO VII. CONCLUSIONS...73 REFERENCES...76 vii

8 LIST OF TABLES Table... Page TABLE OF NOTATION... xi 1.1 CHOICES TO BE MADE IN DESIGNING A FATTAL-LIKE LOCAL TMO PARAMETERS FOR EACH IMPLEMENTED TMO QUALITY ASSESSMENT MEASURES FOR THE IMPLEMENTED TMOS ON THE NAVE IMAGE QUALITY ASSESSMENT MEASURES FOR THE IMPLEMENTED TMOS ON THE MEMORIAL IMAGE HARDWARE COST OF GRADIENT CALCULATION NUMBER OF ADDERS REQUIRED FOR COMPUTATION OF ONE-NORMS OF THE CENTRAL DIFFERENCE GRADIENTS NUMBER OF GRADIENT MANIPULATION FACTORS LUT REQUIREMENTS FOR MANIPULATION FACTOR CALCULATION SCALAR REQUIREMENTS FOR MANIPULATION FACTOR CALCULATION ADDER REQUIREMENTS FOR MANIPULATION FACTOR CALCULATION ADDER REQUIREMENTS FOR CALCULATION MULTIPLIER REQUIREMENTS FOR GRADIENT MANIPULATION ADDER REQUIREMENTS FOR POISSON SOLVER SCALAR REQUIREMENTS FOR POISSON SOLVER ADDER REQUIREMENTS FOR THE DC VALUE ESTIMATION OR CORRECTION SCALAR REQUIREMENTS FOR THE DC VALUE ESTIMATION OR CORRECTION HARDWARE COST FOR THE IMPLEMENTED TMOS viii

9 LIST OF FIGURES Figure Page 1.1 THE NAVE IMAGE AS TONE-MAPPED USING THE EIGHT-BIT RANGE "DISPLAY MAPPING WINDOW" TECHNIQUE FOR FOUR DIFFERENT SPECIFIED RANGES DATA FLOW FOR 3 3 MOVING WINDOW TMO WITH ZERO DIRICHLET BOUNDARY CONDITION DATA FLOW FOR 3 3 MOVING WINDOW TMO WITH NEUMANN BOUNDARY CONDITION THE DC VALUE ESTIMATING FUNCTION THE NAVE IMAGE AS PROCESSED BY THE SINGLE-SCALE TONE MAPPING OPERATORS THE DETAIL VISIBILITY CHANGE MAPS FOR THE NAVE IMAGE AS PROCESSED BY THE SINGLE-SCALE TONE MAPPING OPERATORS THE NAVE IMAGE AS PROCESSED BY THE FIVE-SCALE TONE MAPPING OPERATORS THE DETAIL VISIBILITY CHANGE MAPS FOR THE NAVE IMAGE AS PROCESSED BY THE FIVE-SCALE TONE MAPPING OPERATORS THE MEMORIAL IMAGES AS PROCESSED BY THE SINLE-SCALE TONE MAPPING OPERATORS THE DETAIL VISIBILITY CHANGE MAPS FOR THE MEMORIAL IMAGE AS PROCESSED BY THE SINGLE-SCALE TONE MAPPING OPERATORS THE MEMORIAL IMAGE AS PROCESSED BY THE FIVE-SCALE TONE MAPPING OPERATORS THE DETAIL VISIBILITY CHANGE MAPS FOR THE MEMORIAL IMAGE AS PROCESSED BY THE FIVE-SCALE TONE MAPPING OPERATORS BLOCK DIAGRAM OF TMO HARDWARE IMPLEMENTATION BOUNDARY EXTENSION REQUIRED TO COMPUTE THE FORWARD DIFFERENCES FOR A 3 3 MOVING WINDOW ix

10 6.3 BOUNDARY EXTENSION REQUIRED TO COMPUTE THE CENTRAL DIFFERENCES FOR A 3 3 MOVING WINDOW CENTRAL DIFFERENCE GRADIENTS AND ONE-NORMS FOR 3 3 TMO WITH ZERO DIRICHLET BOUNDARY CENTRAL DIFFERENCE GRADIENTS FOR 3 3 TMO WITH NEUMANN BOUNDARY GRADIENT SET FOR TMOS WITH ZERO DIRICHLET BOUNDARY AND 3 3 MOVING WINDOW GRADIENT SET FOR NEUMANN TMO x

11 TABLE OF NOTATION,,,,,, G G G G () Φ,,, The indices used for pixels The magnitude of R, G and B channels of HDR images The magnitude of R, G and B channels of tone-mapped images The linear domain luminance of HDR images The logarithm domain luminance of HDR images The size of images or moving windows The vertical gradients computed by forward difference The horizontal gradients computed by forward difference The manipulated vertical gradients The manipulated horizontal gradients The indices for the manipulation factor scale The vertical gradients computed by central difference The horizontal gradients computed by central difference The gradient manipulation factor at scale The gradient manipulation factor (all scales combined) The control parameters for calculating the gradient manipulation factor The indices within the surround of a pixel The weighted Gaussian average of Reinhard s Operator The weight factor of the Retinex operator The Gaussian kernel The normalization factor of the Retinex operator The two-dimensional Laplacian operator The one-dimensional vertical Laplacian operator xi

12 div C max min dst2, idst2 dct2, idct2 The one-dimensional horizontal Laplacian operator The eigenvector matrix of The eigenvector matrix of The eigenvalues of The eigenvalues of The divergence of manipulated gradients The tone-mapped luminance in logarithm domain The tone-mapped luminance in linear domain The global contrast change The maximum operator The minimum operator The two-dimensional discrete sine transform and inverse discrete sine transform The two-dimensional discrete cosine transform and inverse discrete cosine transform The mean pixel value of an HDR image in logarithm domain The mean pixel value of an LDR image in logarithm domain xii

13 CHAPTER I INTRODUCTION Currently, cameras and display devices are not sophisticated enough to fully represent what the human eye can see. The Human Visual System (HVS) allows us to perceive the world, from a bright sunny day to a dark moonless night, with a range of luminance of the order of 10 : 1. In other words, the HVS has a dynamic range, or ratio between the intensities of the darkest and brightest pixels, of 32 bits or more. However, standard cameras and display devices are capable of representing only Low Dynamic Range (LDR) images, which have an 8-bit dynamic range, for a range of luminance of the order of 256: 1. To capture scenes well with an LDR camera requires a complex adjustment of exposure time based on the lighting in the environment. In many situations, it is impossible to preserve details in both darker areas and brighter areas clearly in a single LDR image. High Dynamic Range (HDR) imaging possesses the potential to depict captured scenes more accurately by capturing more bits per pixel. There are two ways to produce HDR images. The first is to use an HDR camera with special image sensors that directly capture the data with more bits per pixel. The second is to use a standard LDR camera to capture multiple images of the same scene, each with a different exposure time, and then fuse the images together. As the dynamic range of common display devices such as monitors and printers is only eight bits per pixel, even if HDR images have been produced, most devices simply 1

14 (a) range 2 to 2 1 (b) range 2 to 2 1 (c) range 2 to 2 1 (d) range 2 to 2 1 FIGURE 1.1 THE NAVE IMAGE AS TONE-MAPPED USING THE EIGHT-BIT RANGE "DISPLAY MAPPING WINDOW" TECHNIQUE FOR FOUR DIFFERENT SPECIFIED RANGES cannot display them. Special HDR display devices are just starting to become available, but are too expensive for most applications. Display of an HDR image on a conventional (LDR) display device requires that the image first be processed to reduce its dynamic range; this should be done in a way that preserves all the important details in the original image. Many researchers [1]-[19] have proposed tone mapping operators (TMOs) that convert HDR images into displayable LDR images. The need for a quality tone mapping operator is illustrated with an example using a grayscale version of the Nave image, an HDR image with 32 bits per pixel from the Debevec library of HDR images [20]. Perhaps the simplest tone mapping operator simply maps each HDR pixel in a specified eight-bit range linearly to the values 0 to 255, saturating those HDR pixels whose values are too small or too big to fall within the 2

15 specified range. This technique can be seen as implementing a display mapping window in that the specified eight-bit range is mapped linearly to the eight-bit range of the LDR output; for example, if the specified eight-bit range is from 2 to 2 1, HDR pixels with values between 64 and are mapped linearly to LDR values from 0 to 255. HDR pixels with values less than 64 are mapped to the darkest LDR value, 0, and HDR pixels with values greater than are mapped to the brightest LDR value, 255. Figure 1.1 illustrates the result of implementing this display mapping window on the Nave image when mapping different specified eight-bit ranges from the HDR input. The image in Figure 1.1(a) depicts the result of mapping the eight-bit range from 2 to 2 1 to the values 0 to 255. The image in Figure 1.1(b) shows the result of mapping the eight-bit range from 2 to 2 1 to the values 0 to 255. Similarly, images in Figure 1.1(c) and Figure 1.1(d) show the result of mapping two additional eight-bit ranges, from 2 to 2 1, and from 2 to 2 1, respectively. It can be seen that the upper two images, for which low-order bits are mapped and therefore large luminance values are saturated, have bright areas overexposed and that the details in those areas are lost. In the lower two images, for which higher-order bits are mapped, although the details in the bright area are better preserved, those in the dark areas are underexposed and not visible anymore. Therefore, although an HDR image possesses detailed information in both bright and dark areas, the output of this simple tone mapping algorithm is unable to preserve those details in both bright and dark areas. Fattal s operator [1] is a more sophisticated tone mapping operator for HDR images. It compresses the dynamic range by manipulating the spatial gradients of luminance in the HDR image based on the observation that the HVS is highly sensitive to 3

16 gradients. In Fattal s operator, gradients with high magnitude are manipulated to reduce dynamic range, and gradients with lower magnitude are not reduced or even magnified to make details more visible [1]. This approach allows Fattal s operator to compress an HDR image to fit into the LDR display range while preserving much of the HDR image s details. Implementing tone mapping operators in real time is a challenge. For example, tone mapping operators running in software on powerful computers still require tens of seconds to tone map a single HDR image. If the tone mapping operator could be implemented in real time as an HDR image is produced within a camera, the user would be able to view the pictures as they are taken on an electronic viewfinder, without the need to download the image to a computer and process it. This would significantly enhance an HDR's camera usability. Furthermore, if a frame of HDR video could be tone mapped quickly enough, video could also be viewed as it is taken. This could significantly enhance the capability of many systems, including security camera systems. Several issues must be considered carefully in order to implement a tone mapping operator like Fattal's operator in real time. Fattal s operator was originally proposed in a way such that processing a single pixel requires a calculation that involves every pixel of the image; that is, a single global Poisson equation needs to be solved for the entire image at once, to find the tone-mapped output image that most closely matches the gradients after manipulation. Solving this big inverse problem requires a sizeable amount of hardware, for both computation and temporary buffering of image pixels. Driven by the need to implement the tone mapping operator in real time on an embedded system, a modification of Fattal s operator [2, 5] solves a series of small 4

17 inverse problems, one for each pixel in the output image, using just a 3 3 window of data around the pixel, instead of solving one global inverse problem. This local Poisson solver requires some redundant computations compared to a global solver. Each inverse problem takes nine gradients as input and returns one tone-mapped pixel value, but structures the computations such that computations can be done locally and in parallel, with simple hardware. It produces a tone-mapped image that does not match the one produced by the global solver, but that is similar in quality. Fattal s original operator uses a Neumann boundary condition when solving the global Poisson equation. The modified local Fattal s operator in [2, 5] instead applies a zero Dirichlet boundary around each moving window to avoid a problem with the Neumann boundary condition, namely, the insolvability of the Poisson equation because of the presence of one singular value. In this thesis, a method for dealing with the singularity introduced by the Neumann boundary condition will be proposed for the local Fattal-like TMO and the role that the choice of boundary condition plays in both computational load and quality of output will be explored. The first goal of this thesis is to understand the effects of using different boundary conditions. Fattal s original operator calls for a multi-scale approach in determining the gradient manipulation factor; the calculation of manipulation factor is based on central difference gradients calculated at several different spatial length scales [1]. The second goal of this thesis is to assess the impact of both single-scale and multi-scale approaches, motivated by the desire to avoid the extensive computation required to implement a multi-scale approach. 5

18 TABLE 1.1 CHOICES TO BE MADE IN DESIGNING A FATTAL-LIKE LOCAL TMO Manipulation Factor Computation Window Size Boundary Condition Single-Scale Five-Scale Neumann Zero Dirichlet Fattal utilized the Full Multigrid Algorithm [21] to solve the Poisson equation. In this thesis, alternative solvers that implement discrete sine transform and discrete cosine transform techniques are applied. The work in [2, 5] solves the Poisson equation locally in a 3 3 moving window, and produces a tone-mapped output of good quality; however, the question of what size moving window is best has not been answered. Taking into consideration the increasing number of pixels modern cameras capture, a bigger moving window might produce better tone-mapped images. The third goal of this thesis is to understand the effects of moving window size on computational load and quality of output. To achieve all three goals, a comprehensive study of effects of boundary condition, number of scales of manipulation factor computation, and moving window size is conducted. A total of twelve local TMOs are studied, using different choices for these three aspects. Table 1.1 lists the choices for each of the three aspects. Two different numbers of scales of manipulation are studied. Three different window sizes are studied. Two different boundary conditions are studied: a Neumann boundary condition, which uses symmetric extension at the borders of the moving window, and a zero Dirichlet boundary condition, using zeros at the border. Different local TMOs are defined by 6

19 making different choices for the three aspects. For example, the choices highlighted in red define a TMO that uses the 5 5 moving window with a Neumann boundary condition that computes the manipulation factor by the single-scale approach. For each of the twelve local TMOs, the quality of output is assessed using a sophisticated quality assessment system [22], and the hardware usage is estimated, in order to present a clear view of the balance between quality and expense among all the operators. The number of adders, multipliers, look-up tables and other hardware components affect the size, energy efficiency, and speed of the system. It is safe to assume that a TMO that uses a global Poisson solver requires more hardware than one that uses a local Poisson solver in a moving window, and also that the bigger the moving window, the more adders and multipliers are needed. The different boundary conditions affect the number of adders and multipliers as well. The organization of the reminder of the thesis is as follows. In Chapter II, the previously published work on hardware-friendly TMOs is described. Necessary background for implementing a Fattal-like TMO is given in Chapter III; this includes Poisson solvers and systems for assessing the quality of TMOs. In Chapter IV, all twelve variations of the gradient-domain TMO are developed, and their implementation is described. The simulation results and quality evaluations of these TMOs are presented in Chapter V. In Chapter VI, the hardware cost for each TMO is estimated by calculating the number of adders, scalar multipliers and look-up tables required by a fully parallel implementation. The conclusions are made in Chapter VII. Some of the work developed in this thesis is also documented in [24]. 7

20 CHAPTER II RELATED WORK Many researchers have proposed algorithms for tone mapping HDR images, but nearly all existing methods are far too computationally expensive to consider for embedded real-time implementation. The hardware implementation of three relatively hardware-friendly algorithms [1, 6, 13, 14], however, has been studied. In this chapter, the real-time implementations of these tone mapping operators, namely Reinhard s operator, the Retinex operator and Fattal s operator, are reviewed. The basic mathematic details of the three tone mapping operators are discussed in this chapter. Previous research focused on implementing these operators in real time is also reviewed. The advantages and disadvantages of each operator in terms of hardware implementations are identified. As a result, the real-time implementation of Fattal s operator is chosen to be studied further. 2.1 Reinhard s Operator One of the most widely studied tone mapping operators is Reinhard s operator [6]. This operator finds the LDR luminance, of the output pixel at pixel index (, ) with the global tone mapping equation, =, 1 +,, (2.1) 8

21 where, is the HDR luminance of the pixel, and, is a normalization factor. The normalization factor is found by taking a weighted Gaussian average of the luminance in a surround centered on the pixel; the Gaussian weighting function used is, = 1, (2.2) where is a scale chosen pixel-by-pixel, and and are indices relative to the pixel being processed, where = 0 and = 0 indicates the chosen pixel, which is at the center of the surround. Reinhard proposed considering successively larger surrounds and scales at each pixel location to find the best choice of scale and size of surround to normalize each pixel [6]. Together, the use of multiple surrounds and different scales affect both the quality of the tone-mapped output and the hardware cost. Hassan and Carletta [7, 8] have proposed modifications of Reinhard s operator to make the operator amenable to hardware implementation. The Gaussian filters of given scales and sizes of surround are approximated by simpler filters that can be easily implemented and require less computation. At the same time, the number of sizes of surround considered is reduced from nine to four in order to significantly reduce the computational load. To summarize Reinhard s operator, it is relatively easy to apply in real time; the most computationally expensive module, Gaussian filtering, has been well studied and can be successfully approximated and implemented in real time. However, it is worth mentioning that Reinhard s operator computes multiple tone-mapped values with differently sized surrounds and different scales at each pixel location, so that even with simplifications and the use of a reduced number of surrounds, the computational load is 9

22 still high. Thus, Reinhard s operator for tone mapping HDR images will not be considered further in this thesis. 2.2 The Retinex Operator The Retinex operator is an image enhancement algorithm that improves the contrast, brightness and perceived sharpness of images while simultaneously performing dynamic range compression [13]. Unlike many other HDR compression algorithms, the Retinex operator processes each color band separately, and hence maintains good color constancy but requires bigger computational load. The Retinex operators are categorized into the more hardware friendly single-scale Retinex (SSR) operator and the better performance multi-scale Retinex (MSR) operator. Using the Retinex operator, the tone-mapped values for the color components at a pixel location can be computed from the HDR versions of the color components of that pixel and its surrounding pixels, as, = log, log,, (2.3), = log, log,, (2.4), = log, log,, (2.5) where,,, and, are the values of the color components of the pixel,,,, and, are normalization factors, and and are gain and offset parameters. The normalization factor for a given color component is found by taking a Gaussian average of the color band in a surround centered on the pixel; the Gaussian weighting function used is, = (2.6) 10

23 where, are parameters to control the spread and height of the Gaussian function [13]. Both single-scale and multi-scale approaches to the normalization have been proposed; the multi-scale approach tone maps the red component, for example, using a sum of SSR operators at different scales [14]:, = log, log, (2.7) where is the number of scales being used. Each scale has its own weight and Gaussian parameters and are used to compute the normalization factor for the scale. Several research efforts have used Digital Signal Processors (DSPs) to implement the Retinex operator in video applications [13]. The reported frame rate for a single-scale Retinex operator varies from 20.7 to 34.1 frames per second, depending on the particular DSP device used. By implementing multi-scale Retinex operators in parallel hardware on a Field Programmable Gate Array (FPGA), researchers [15] were able to achieve 30 frames per second for full high definition video with a resolution of 1920 pixels by 1080 pixels. Tsutsui et al. [16] further improved the multi-scale Retinex operator on a FPGA device, achieving 60 frames per second with a resolution of 1920 pixels by 1200 pixels. To summarize the work on the Retinex operator, single-scale and multi-scale Retinex operators have been successfully implemented in real time. Although the singlescale Retinex operator is exceptional in preserving color constancy, the output can suffer from halo effects [14]. At the same time, the multi-scale Retinex operator results in a higher quality output image, but also has a larger hardware cost than the single-scale Retinex operator [14]. Furthermore, the Retinex operator processes each of the color 11

24 bands separately, and so potentially requires a much higher computation load than the luminance-based TMOs. Based on the consideration of both hardware consumption and quality performance, the Retinex operator is also not chosen for further study in this thesis. 2.3 Fattal s Operator Fattal s operator [1] is another TMO that has been implemented in real time. It does gradient-domain processing of an HDR image s logarithmic luminance. The main steps of Fattal s operator are to transform an HDR logarithmic-luminance image into the gradient domain, manipulate the gradients, and transform the image back into the logarithmic-luminance domain. The method of gradient-domain manipulation is designed to maintain the visible details while compressing the dynamic range. A color HDR image must first be converted to grayscale, so that each pixel s value represents the logarithm of the luminance or intensity of the pixel. One possible conversion defines the log-luminance, as [23], = 0.299, , ,, (2.8) where,,, and, indicate the magnitudes of the logarithms of the red, green and blue components of the original linear pixel. Fattal s operator is based on the observation that local details correspond to ratios between the luminances of neighboring pixels [1]. Ratios of neighboring linear domain pixels are gradients in the logarithm domain image; this is why processing the image in the gradient domain is efficient. Fattal s operator starts with the logarithmic luminance pixels of an image,,,,, (2.9) 12

25 where is the size of the image. The first step is to transform the log-luminance image to the gradient domain. In the gradient domain, the image is represented by two different matrices, one of horizontal gradients and one of vertical gradients, both computed by forward difference. The vertical gradients and horizontal gradients are computed by G, =,,, (2.10) G, =,,, (2.11) where = 0, 1, 2,, and = 0, 1, 2,,. Boundary extension is used so that the gradient-domain images have one more row and one more column of pixels than the original. In Fattal s original work [1], a Neumann boundary condition with even extension was used. The second step is to manipulate both the vertical gradients and horizontal gradients using G, G, = Φ, G,, (2.12) = Φ, G,, (2.13) for = 0, 1, 2,, and = 0, 1, 2,,, where Φ, is the gradient manipulation factor for the pixel at indices,. A manipulation factor is computed for each pixel in the gradient-domain image; Fattal calculated the manipulation factor using gradients at that pixel in the original image, and in all scale images; the gradients are calculated using central differences and a Neumann boundary condition [1]. The intent is to attenuate gradient-domain pixels of high magnitude; these correspond to parts of the image to which the eye is highly sensitive. Also, gradient-domain pixels of small magnitude are magnified so that fine details that would not otherwise be visible can be seen. The 13

26 manipulation factor for the vertical and horizontal gradient-domain pixels at indices (, ) is, +,, =, (2.14) for = 0, 1, 2,, and = 0, 1, 2,,. In (2.14),, and, are the (central difference) horizontal and vertical gradients at indices (i, j) and scale k, =,, 2, (2.15), =,, 2, (2.16) is the image at scale, and and are parameters used to control the degree of manipulation. Essentially, sets the threshold above which gradients are attenuated, while controls the degree of manipulation. Fattal has recommended that be set to one-tenth of the average gradient-domain pixel magnitude, and that be between 0.8 and 0.9. The third step of Fattal s operator is to convert the manipulated gradient-domain image back to the luminance domain. This involves solving a Poisson Equation [1] = div, (2.17) for the log-luminance image that has gradients closest to the manipulated gradients; is a two-dimensional Laplacian operator, and div is the divergence of the manipulated gradients. Finally, the tone-mapped output is converted back from log-luminance to luminance, and then to color; the result is a tone-mapped image appropriate for display on a standard low dynamic range device. 14

27 Fattal s operator, unlike Reinhard s operator, can be modified in a way such that every pixel is processed by only one surround of fixed size. At the same time, the solution of the most computationally expensive portion of Fattal s operator can be implemented by discrete sine transform or discrete cosine transform, which can be implemented efficiently in real time. As a luminance-based TMO, Fattal s operator does not process data in separate color bands. All these imply that Fattal s operator is a good candidate for real-time implementation. Published work on real-time implementation of Fattal s operator is described next. Tsun-Hsien Wang, et al., [3] have proposed a block-based Fattal s operator. In this TMO, the pre-processed HDR image is divided into 8 8 non-overlapping blocks that are processed separately. Neumann boundary extension is applied to each block. Wang, et al. have also proposed a hardware architecture for their TMO. However, the technique used to solve the Poisson equation did not match the boundary condition chosen; when a Neumann boundary condition is used, the Poisson solver requires a discrete cosine transform, but in [3] a discrete sine transform is applied. More importantly, while the block-based technique reduces computational load, it also affects the quality of the output; images produced using this operator tend to have a noticeable tile effect, with the borders between the eight-by-eight blocks clearly visible. Vytla et al. [2] have modified Fattal s operator with a local Poisson solver for the ease of real-time implementation. A 3 3 moving window with zero Dirichlet boundary condition is applied. Thus, each output pixel is found via manipulation of gradients in its immediate surroundings; this allows the moving window version of Fattal s operator to be more adaptive than the original. This moving window approach requires some 15

28 redundant computations compared to an approach that solves the Poisson equation globally, but requires simpler hardware by doing the computation locally and in parallel. Some additional modifications to Fattal s operator were made in [2]. For simpler hardware implementation, the gradient manipulation factor are computed using a onenorm rather than two-norm, so that (2.14) is replaced by:, = where = 0, 1, 2,, and = 0, 1, 2,,., +,, (2.18) Fattal's original operator computed the gradient manipulation factor using an approach that required successively down-sampling the input image to a size of 32 pixels by 32 pixels. Because the moving window size in [2] is already smaller than 32 pixels by 32 pixels, this approach is not feasible; instead, a series of full-sized versions of the original full-sized input image window is first produced by implementing Gaussian-like filters of different scales, and windows from those scale images are used. In this chapter, work related to real-time implementation of three tone mapping operators is reviewed. The advantages and disadvantages of each operator are assessed. Evidence is provided that Fattal s operator may be more hardware friendly than the competing algorithms. Thus, Fattal s operator is chosen for further study. Particular attention will be paid to issues that affect its hardware implementation in this thesis. 16

29 CHAPTER III BACKGROUND The most important and computationally expensive part of Fattal s operator is solving the Poisson equation. Methods for the solution are studied in detail in this chapter. Solutions when both zero Dirichlet and Neumann boundary conditions are specified are elaborated. The insolvability of the Poisson equation with a Neumann boundary condition is addressed mathematically. A quality assessment system developed in previous research [22] is also discussed in this chapter; it will be used to evaluate quality of the output images produced by the twelve local TMOs studied in Chapter V. 3.1 The Poisson Solvers The mathematic details of the Poisson solver depend on the boundary conditions applied, either around the full-size image in a global solver, or around the moving window in a local solver [25]. Those details are described in two separate subsections for the zero Dirichlet and Neumann boundary conditions, respectively Poisson Solvers with Zero Dirichlet Boundary Condition There are two ways to solve the Poisson equation with zero Dirichlet boundary condition to find the tone-mapped output image. The straightforward way is to directly apply the inverse of Laplacian operator to both sides of the Poisson equation = div, (3.1) 17

30 where is a Laplacian operator, div is the divergence of the gradients and is the tone-mapped output image. However, this is plausible in hardware only when the image size for a global solver, or window size for a moving window solver, is fixed; then, the matrix can be inverted once, off line. For a global Poisson solver working on images of different sizes, this approach is not plausible as the hardware of cost of inverting the matrix in real time is prohibitively high. The second way is to solve the Poisson equation with a zero Dirichlet boundary condition using the eigenvalue-eigenvector decomposition of the Laplacian operator. Since the two-dimensional Laplacian operator addresses a pixel and its neighboring pixels both vertically and horizontally, it can be rewritten as the sum of two onedimensional operators applied on the image: + = div, (3.2) where is the vertical operator and is the horizontal operator. Both and are tridiagonal matrices with 2s on the main diagonals and 1s on the superdiagonals and subdiagonals; the size of is, while that of is. Using the eigenvalue decomposition, any square matrix can be represented by the product of three matrices: its eigenvector matrix, its eigenvalue matrix, and the inverse of its eigenvector matrix. Thus, the Poisson equation can be further rewritten as + = div, (3.3) where and are diagonal matrices of eigenvalues of and respectively, and and are the corresponding orthogonal matrices of eigenvectors. After further manipulation, the equation becomes + = div. (3.4) 18

31 Since and have zeros everywhere but the main diagonal, they can be moved to the right-hand side of the equation, by writing the equation at a single pixel location (, ) as [ + ], = [ div ],, +,. (3.5) In the final step, it can be easily computed that where = [ ], (3.6), = [ div ],, +,. (3.7) Although in general the eigenvalue decomposition is computationally intensive, the eigenvalues and eigenvectors of the one-dimensional Laplacian operator are not difficult to compute. The eigenvectors follow the pattern [5], = sin if, sin if < (3.8) where, is the eigenvector matrix of the vertical Laplacian operator or the horizontal Laplacian operator, and indicates the size of the one-dimensional Laplacian operator, for = 1, 2,, and = 1, 2,,. In this respect, [ div ], = div, sin sin , (3.9) which is equivalent with a two-dimensional discrete sine transform applied on div. In the mean time, [ ], = , sin sin , (3.10) 19

32 which is equivalent to a two-dimensional inverse discrete sine transform applied on. The eigenvalue matrices and can be calculated by, = 2 1 cos + 1, (3.11), = 2 1 cos + 1, (3.12) Thus, the Poisson equation with a zero Dirichlet boundary condition can be solved by taking the two-dimensional inverse discrete sine transform of the matrix that is computed by point-wise division of the results of two-dimensional discrete sine transform of div and the sum of correlating and Poisson Solvers with Neumann Boundary Condition When a Neumann boundary condition is used, the solution of the Poisson equation is significantly different from the solution for a zero Dirichlet boundary condition. First of all, the Poisson equation cannot be solved directly by multiplying both sides by the inverse of the Laplacian operator because the Laplacian matrix is singular, and has no inverse. Secondly, through the eigenvalue-eigenvector decomposition approach, this solver requires the discrete cosine transformation rather than the discrete sine transformation. Again, (3.1) can be rewritten as the sum of two one-dimensional operators applied on the image: + = div, (3.13) where is the vertical operator and is the horizontal operator. Both and are tridiagonal matrices with [ ] on the main diagonal and 1s on the superdiagonals and subdiagonals; again the size of is and that of is. 20

33 Note that (3.13) looks identical to (3.2), but the Laplacian operators and are no longer the same due to the boundary condition. As demonstrated before, the output image can be solved by = [ ], (3.14) where, = [ div ],, +,. (3.15) However, when a Neumann boundary condition is used, (3.8) becomes: div, = div 1 2 1, cos 2 (3.16) cos, 2 1 if = 1 where = 2 if 1 1 if = 1 and = 2 if 1, which is equivalent to a two-dimensional discrete cosine transform applied on div. In the mean time, [ ], = 1 2 1, cos cos, 2 and the eigenvalues of the T matrices under a Neumann boundary condition are (3.17), = 2 1 cos, = 2 1 cos 1, (3.18) 1. (3.19) Thus, the Poisson equation with a Neumann boundary condition can be solved by taking the two-dimensional inverse discrete cosine transform of the matrix that is computed by 21

34 point-wise division of the matrix that is the two-dimensional discrete cosine transform of div and the sum by a matrix of sums of eigenvalues in and. The singularity of the Laplacian operator matrix for a Neumann boundary condition that made a direct solution impossible also complicates the eigenvalueeigenvector decomposition method of the solution. Two of the eigenvalues,, and,, are zero, so that dividing by their sum leads to an infinitely large value for,, the component that corresponds to the DC value of the output image. One of the contributions of this thesis is to address this problem by substituting an estimate of the DC level of the image for the DC component, according to the expected dynamic range of the tone-mapped output; this work is elaborated in the next chapter. 3.2 TMO Quality Assessment Assessing the performance of a TMO is a challenging topic. It is unavoidable that some information is lost in the process of tone mapping an HDR image with 32 bits per pixel or more to produce an LDR image with just eight bits per pixel. Smith, et al. [22] have proposed a system of quality assessment for TMOs that considers changes in global contrast and visible details as the image is tone mapped. The evaluation of TMOs in this thesis is done based on Smith s ideas. The system of Smith, et al. [22] creates a single metric for the global contrast change. The global contrast change C is measured straightforwardly by the log-luminance values before and after the tone mapping C = max min max min, (3.20) where max and min indicate the maximum and minimum log-luminance pixel values of the image, respectively. A global contrast change value that is smaller than one 22

35 implies a smaller global contrast in the tone-mapped image than in the original HDR image; a value greater than one denotes a larger global contrast than in the original HDR image. A global contrast change value that is closer to one is better than a global contrast change value that is farther from one. The detail visibility change is measured by comparing the detail visibility of the HDR image with the detail visibility of the tone-mapped LDR image at the same location. The mechanism of estimating detail visibility takes into consideration the different dynamic ranges of the HDR and LDR images, and is based on the Human Visual System response to high frequency contrast [26]. In this thesis, the detail visibility change of an image is represented by a binary map, with a value of 255 in a pixel location indicating that the visible detail is maintained after implementing the tone mapping operator and a value of 0 indicating that the visible detail is lost after the tone mapping. In this chapter, a computationally efficient method for approximating the solution to the Poisson equation is elaborated for different boundary conditions. This is important because solving the Poisson equation is the most computationally expensive portion of Fattal s operator. This chapter also presents the background information for the quality assessment of the tone mapping operators. With the two quality metrics demonstrated, it is possible to evaluate different TMOs or different parameter sets for a single TMO. 23

36 CHAPTER IV IMPLEMENTED TONE MAPPING OPERATORS A number of variations on Fattal s operator are proposed; each operator is specified by its window size, number of scales used in manipulation factor computation, and boundary condition. In this chapter, the computational steps of the TMOs are discussed. As discussed in Chapter III, the Poisson equation cannot be solved when a Neumann boundary condition is used because some eigenvalues of the Laplacian matrices are zero. A DC value estimation method is proposed in this chapter to provide a work-around that allows the Neumann boundary to be used. A DC value correction method to enhance the tone mapping quality with zero Dirichlet boundary condition is also developed. Finally, a series of experiments is conducted in order to determine appropriate values for the control parameters and for each discussed TMO for quality performance. 4.1 Tone Mapping Operator with Zero Dirichlet Boundary Condition Figure 4.1 depicts the data flow for one of the twelve local TMOs implemented in this work, the TMO with a single-scale manipulation factor computation, a 3 3 moving window, and a zero Dirichlet boundary condition. Note that gradients needed during the process are approximated using the same techniques that Fattal originally proposed in [1]; other approximations could also be considered. In Figure 4.1, the steps of the TMO are labeled with the letters A through E. These steps are: 24

37 ,,, 0 0 0,,, 0 0 0,,, B,,,,,,,,,,,,,,,,,,,,,,,, A,,,,,,,, C ,,,,,,,,,,,, 0,,, 0,,, 0,,, 0,,,,,,.,,,,,,,,,,,, D div, div, div, div, div, div, div, div, div, E,,,,,,,,, FIGURE 4.1 DATA FLOW FOR 3 3 MOVING WINDOW TMO WITH ZERO DIRICHLET BOUNDARY CONDITION 25

38 A. Calculation of forward-difference gradients, both vertical and horizontal, from the zero Dirichlet extended image window; B. Calculation of central-difference gradients, both vertical and horizontal, at each scale, from the zero Dirichlet extended image window; C. Calculation of gradient manipulation factor from the vertical and horizontal central-difference gradients; D. Manipulation of the forward-difference gradients according to the gradient manipulation factor, and calculation of the backward-difference divergence; E. Solving of the Poisson equation to produce the tone-mapped output image. These steps are described in more detail next. While the example shown uses a singlescale manipulation factor computation and a 3 3 window, it serves as a template for any Fattal-like TMO with a zero Dirichlet boundary condition; the other five local TMOs discussed in this thesis that use a zero Dirichlet boundary condition follow the same basic data flow Gradient Transformation and Gradient Manipulation (Steps A through D) An operator that solves the Poisson equation in a moving window of size requires that the window be extended to ; for example, an operator that solves the Poisson equation in a 3 3 moving window requires that the window be expanded to 6 6. When a zero Dirichlet boundary condition is used, the moving window is extended with zeros. The horizontal and vertical gradients used to represent the window in the gradient domain are matrices of size As an example, a 3 3 moving window has the vertical gradients G and horizontal gradients G with 26

39 ,, =,,,,,,,,,,,,,, 0,,, 0 =,,,,,,, (4.1) 0,,,,,, 0,,,,, =,,,,,,,,,,,,,, =,,,,,,,,,, h,,,,,,,, (4.2) when the zero Dirichlet boundary condition is used. Note that because of the boundary condition, the first column of vertical gradients and the first row of horizontal gradients are all zeros. The gradient manipulation factor can be calculated by, =, +,, (4.3) for = 0, 1, 2,, and = 0, 1, 2,,, where, and, are the horizontal and vertical gradients in central difference. In (4.3), K is set to 0 for the single-scale approach and to 4 for the five-scale approach. Multiplying the gradient at each pixel location with its corresponding gradient manipulation factor performs the process of gradient manipulation Inverse Gradient Transformation (Step E) Solving the Poisson equation (3.1) moves the image from the gradient domain back into the log-luminance domain. A close look reveals why this step performs the inverse gradient transformation. To begin with, the divergence of gradients is defined by divg, = G, G, + G, G,, (4.4) 27

40 where = 1, 2,, and = 1, 2,,. Applying the relationship between gradients and, described in (4.1) and (4.2), (4.4) can be rewritten as: divg, = 4,,,,,. (4.5) Note that this is precisely the same computation done as when the two-dimensional Laplacian operator (4.6) is applied at the pixel at location (, ) and its surrounding pixels,,,,,,. (4.7),,, Written in vector form, the Poisson equation for the pixel at location (, ) is,,,, [ ], = divg,.,,,, (4.8) If some of, s neighbors happen to lie in the zero extension of the original window, those neighbors do not contribute to the divergence calculation, or to the sum computed by the Laplacian operator. For example, when computing the divergence of,, (4.5) becomes divg, = 4,,,. (4.9) 28

41 When the Laplacian operator is centered on pixel,, the two-dimensional Laplacian operator is applied to the window ,,, (4.10) 0,, and the Poisson equation corresponding to this pixel in vector form is [ ], = divg,. (4.11), 0,, All together, the Poisson equations for the nine pixels in the 3 3 window can be written in matrix form as, div, , div, , div, , div, , = div,. (4.12) , div, , div, , div,, div, While (4.12) is specifically for a 3 3 window, a similar equation can be derived for windows of general size. In general, if the TMO solves the Poisson equation by window (or in a tiled block as done in [3]), the matrix on the left-hand side of (4.12) is. The structure of the matrix can be viewed as a combination of blocks, each of size : 29

42 L D Z Z Z Z D L D Z Z Z Z D L Z Z Z, (4.13) Z Z Z L D Z Z Z Z D L D Z Z Z Z D L where L is a tri-diagonal block with 4s in the main diagonal and 1s in the superdiagonal and subdiagonal, D is a diagonal block with 1s in the main diagonal, and Z is a block containing only zeros. 4.2 Tone Mapping Operator with Neumann Boundary Condition Figure 4.2 depicts the data flow for one of the twelve local TMOs implemented in this work, the TMO with a single-scale manipulation factor computation, a 3 3 moving window, and a Neumann boundary condition. In Figure 4.2, the steps of the TMO are labeled with the letters A through E, similar to those in Figure 4.1. These steps are described in more detail next. While the example shown uses a single-scale manipulation factor computation and a 3 3 window, it serves as a template for any Fattal-like TMO with a Neumann boundary condition; the other five local TMOs that use a Neumann boundary condition follow the same basic data flow Gradient Transformation and Gradient Manipulation (Steps A through D) The size of the window extension is the same whether a Neumann boundary or a zero Dirichlet boundary is used. However, when a Neumann boundary is used, the moving window is extended with pixel values. The gradient transformation for a TMO with a Neumann boundary condition also makes use of forward differences. However, the actual gradients are slightly different because of the boundary condition. The vertical gradients G and horizontal gradients G of the TMO with a 3 3 moving window are 30

43 ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, B,,,,,,,,,,,,,,,,,,,,,,,, A,,,,,,,, C 0,, 0 0,, 0 0,, 0 0,, ,,,,,,,, ,,,.,,,,,,,,,,,, D div, div, div, div, div, div, div, div, div, E,,,,,,,,, FIGURE 4.2 DATA FLOW FOR 3 3 MOVING WINDOW TMO WITH NEUMANN BOUNDARY CONDITION 31

44 G,, =,,,,,,,,,,,,,, =,,,,,,,,, (4.14),,,,,,,, , G, =,,,,,,,,,,,,,, 0,,,, 0 0 =,,,, 0. (4.15) 0,,, h, 0 0,,,, 0 Again the computation of gradient manipulation factor here is done using, =, +,, (4.16) where = 0, 1, 2,, and = 0, 1, 2,, ; K = 0 for the single-scale approach and K = 4 for the five-scale approach. Note that (4.16) is identical to (4.3), except that gradients are acquired with different boundary conditions Inverse Gradient Transformation (Step E) When a Neumann boundary condition is used, the solution of the Poisson equation is different than when a zero Dirichlet boundary condition is used. If some of, s neighbors happen to lie on the even extension of the original window, those neighbors cancel out with other terms in the divergence calculation, or in the sum computed by the Laplacian operator. For example, when computing the divergence at pixel (1, 1), (4.5) becomes divg, = 2,,,. (4.17) 32

45 When the Laplacian operator is centered on pixel,, the two-dimensional Laplacian operator is applied to the window and the Poisson equation corresponding to this pixel is,,,,,, (4.18),,,,,,, [ ], = divg,. (4.19),,,, All together, the Poisson equations for the nine pixels in a 3 3 window can be written in matrix form as, div, , div, , div, , div, , = div,. (4.20) , div, , div, , div,, div, As before, in general the Laplacian operator s size is ; however, for the Neumann boundary condition the structure of the matrix is different. It can be viewed as a combination of blocks, each of size : 33

46 L2 D Z Z Z Z D L1 D Z Z Z Z D L1 Z Z Z, (4.21) Z Z Z L1 D Z Z Z Z D L1 D Z Z Z Z D L2 where L1 is a tri-diagonal block with [ ] in the main diagonal and 1 s in the superdiagonal and subdiagonal; L2 is a tri-diagonal block with [ ] in the main diagonal and 1 s in the superdiagonal and subdiagonal; D is a diagonal block with 1s in the main diagonal, and Z is a block containing only 0s. 4.3 The Estimation and Correction of DC Value As mentioned in 3.1.2, when the Neumann boundary condition is used, the Poisson equation cannot be fully solved; in the eigenvalue-eigenvector approach of solving the Poisson equation in (4.20), the solution requires division by the eigenvalues of the matrix on the left-hand side. Because the smallest eigenvalue is zero, the division by the smallest eigenvalue yields infinity. In order to obtain a solution, the component that is produced by dividing by zero must be replaced with a finite value; this component corresponds to the DC level of luminance in the window of the (tone-mapped) output image. When the zero Dirichlet condition is used, the Poisson equation can be solved directly; however, experiments conducted on images from Debevec s library [20] show that the tone-mapped output images are typically darker than desired. A correction of the DC value allows the TMO to produce high quality output images even when the input HDR images vary in dynamic range. 34

47 In the rest of this section, the DC value of the input image window is examined to help explain how the DC value estimation and correction can be done for the output image. Techniques for DC value estimation for TMOs with Neumann boundary condition and DC value correction for TMOs with zero Dirichlet boundary condition are then further discussed The DC Value in the Input Window As a prerequisite to estimating or correcting the DC value of a window in the tone-mapped output image, the DC value in the corresponding window of the input image is first understood. By definition, the first element of the two-dimensional discrete cosine transform of the input window is a normalized DC value; it is [dct2], =, (4.22) where denotes the mean value in the window of the HDR image. The main idea used in developing techniques for DC level estimation and correction is that the DC value of a tone-mapped window in the output image should be related to the DC value of the corresponding window of the input image; if, for example, the input images have a 32-bit dynamic range and the output images have an eight-bit dynamic range, the relationship should involve a factor of about four, so that the DC value should be close to The DC Value Estimation In a global solver, the presence of one zero eigenvalue does not affect the output image much, as it affects only one element of matrix in (3.15). Based on a study of histograms of pixel intensities in the resulting tone-mapped output images from 35

48 FIGURE 4.3 THE DC VALUE ESTIMATING FUNCTION Debevec s library when the DC coefficient is replaced with different values of +, the DC coefficient is set using = + 2, (4.23) where µ is the mean log-luminance pixel value in the input HDR image. For the local solvers, the effect is much more significant, since there is a singularity in each small window or block that corresponds to a single output pixel. A fully dark window in the input image maps to a fully dark window in the output image, and so it makes sense to map a µ of zero to an estimate of zero for the DC coefficient of the same block in the output image. Similarly, a fully bright window in the input image maps to a fully bright window in the output image; because for the TMOs developed in this thesis the maximum value of a pixel in the logarithmic domain in the input image is 32, and the maximum value in the output image is eight, it makes sense to map a µ of 32 to an estimate of eight. A piecewise linear function of µ between these two endpoints was used to estimate the DC value of an output window. The choice for the piecewise linear function was informed by a study of the histograms of the µ values for windows of the sizes studied in this thesis in images from 36

49 Debevec s library [20]. In those images, most windows have µ between 6 and 15. For this reason, one linear function was used in this range. The particular piecewise linear function chosen is 0.75 if 6 = if (4.24) if It can be seen that the chosen DC value estimation function ensures that the mean value of each window is mapped between 0 and 8 in the log domain. The multi-linear approach used here further attenuates the gradients of large magnitudes The DC Value Correction For the TMOs with a zero Dirichlet boundary condition, it is not necessary to provide an estimate of the DC level; the Laplacian matrices have no zero eigenvalues, and the Poisson equation has a solution. However, during the experiments, it was seen that the output images are sometimes over-compressed. The reason for the overcompression is that the input HDR images from the Debevec library do not fully utilize the available 32-bit dynamic range. For example, the Nave image utilizes 28 bits, and the Memorial image utilizes 18 bits. However, the TMO is designed to compress the full 32- bit dynamic range to 8 bits. Thus, a correction should be done to offset this overcompressing effect. This is done by adding a constant to the DC value in the logarithm domain, where the constant value is chosen based on the dynamic range of the input image. For the Nave and Memorial images, constants of 2 and 3, respectively, were used. 4.4 Parameters for the Implemented TMOs The parameters and used to compute the gradient manipulation factor have significant impact on the global contrast and local details of the output image. For a good 37

50 TABLE 4.1 PARAMETERS FOR EACH IMPLEMENTED TMO Method single-scale zero Dirichlet five-scale zero Dirichlet Method single-scale Neumann five-scale Neumann Window Size avg avg avg 0.65 global 0.4 avg avg avg avg 0.87 global 0.1 avg 0.93 Window Size avg avg avg 0.80 global 0.1 avg avg avg avg 0.90 global 0.1 avg 0.90 quality output, it is important to choose these two parameters, as well as parameters controlling the DC level, carefully. A high quality output image should have good contrast and certain overall level of brightness, and should have clearly visible but not overemphasized details. For the global solvers, it can be difficult to set the parameters for a successful tone mapping. In the Nave image, for example, which is mostly dark but has a bright window, it is a challenge to set the overall DC level appropriately. If it is set high, it is difficult to select and to maintain details around the bright window, as significant manipulation must be applied to the gradients to avoid saturation in the window area. A lower DC level allows for more details to be seen around the window, but results in an image too dark to be visually appealing. One significant advantage of the 38

51 local solvers is that the DC level and parameter may be set locally, window by window, which allows the tone mapping operator to be more adaptive. A series of experiments was conducted to determine the best set of parameter values for each of the implemented TMOs. TMOs using different combinations of values for and were simulated for the Nave and Memorial images, and the quality of the output for each combination was judged, both qualitatively and quantitatively, using the detail visibility change maps. The chosen parameter values are shown in Table 4.1. The parameter is chosen to be a fraction of the average gradient magnitude, denoted avg. In the moving window TMOs, the parameter is set for each window independently using the average gradient within the window; for the global versions of the TMOs, it is set using the average gradient for the entire image. Note that for the TMOs using the Neumann boundary condition in moving windows, the average gradient in the window is smaller, because the gradients around the boundary are zero. This necessitates that the coefficient multiplier in the parameter be larger than for the other TMOs. In this chapter, the mathematic details of tone mapping operators are covered and elaborated with the 3 3 moving window as an example. By analyzing the computational steps of TMOs, all the details of the twelve TMOs are fully specified. A technique for the estimation of the DC level is proposed for the TMOs with Neumann boundary conditions, and a technique for the DC level correction is developed for the TMOs with zero Dirichlet boundary conditions, respectively. With the help of DC level estimation and DC level correction, the quality of the tone-mapped image is improved. The choices of control parameters for different TMOs are also made, in preparation for 39

52 the evaluation and comparison of all twelve TMOs with local Poisson solvers and four TMOs with global Poisson solvers in the next chapter. 40

53 CHAPTER V SIMULATION RESULTS A series of simulation-based experiments were done in Matlab to assess the quality of the implemented TMOs. Simulations of the twelve local TMOs were used to process the images in the Debevec library. The simulations were done for TMOs using local Poisson solvers with three different window sizes for both zero Dirichlet and Neumann boundary conditions, and using both single-scale and five-scale manipulation factor computations. For the purposes of comparison, simulations of TMOs using a global Poisson solver are also done; global TMOs are simulated using both Neumann and zero Dirichlet boundary conditions, and with both single- and five-scale manipulation factors. For each tone-mapped image, a global contrast change, using the definition given in Chapter III, is calculated. In addition, a detail maintenance percentage is computed based on the detail visibility change map developed in [23]; the detail maintenance percentage is a single number that gives the percentage of pixel locations for which visible details are maintained after the tone mapping. The study of these performance metrics and maps gives us insight into the performance of the various implemented TMOs. Figure 5.1 to Figure 5.8 show the output images produced by the different TMOs and their corresponding detail visibility change maps, while Table 5.1 and Table 5.2 list their assessed performance. Figure 5.1 shows the tone-mapped Nave images for the 41

54 single-scale TMOs, and Figure 5.2 shows the corresponding detail visibility change maps, 3 3 moving window with zero Dirichlet boundary 5 5 moving window with zero Dirichlet boundary 7 7 moving window with zero Dirichlet boundary 3 3 moving window with Neumann boundary 5 5 moving window with Neumann boundary 7 7 moving window with Neumann boundary global solver with zero Dirichlet boundary global solver with Neumann boundary FIGURE 5.1 THE NAVE IMAGE AS PROCESSED BY THE SINGLE-SCALE TONE MAPPING OPERATORS 42

55 3 3 moving window with zero Dirichlet boundary 5 5 moving window with zero Dirichlet boundary 7 7 moving window with zero Dirichlet boundary 3 3 moving window with Neumann boundary 5 5 moving window with Neumann boundary 7 7 moving window with Neumann boundary global solver with zero Dirichlet boundary global solver with Neumann boundary FIGURE 5.2 THE DETAIL VISIBILITY CHANGE MAPS FOR THE NAVE IMAGE AS PROCESSED BY THE SINGLE-SCALE TONE MAPPING OPERATORS 43

56 3 3 moving window with zero Dirichlet boundary 5 5 moving window with zero Dirichlet boundary 7 7 moving window with zero Dirichlet boundary 3 3 moving window with Neumann boundary 5 5 moving window with Neumann boundary 7 7 moving window with Neumann boundary global solver with zero Dirichlet boundary global solver with Neumann boundary FIGURE 5.3 THE NAVE IMAGE AS PROCESSED BY THE FIVE-SCALE TONE MAPPING OPERATORS 44

57 3 3 moving window with zero Dirichlet boundary 5 5 moving window with zero Dirichlet boundary 7 7 moving window with zero Dirichlet boundary 3 3 moving window with Neumann boundary 5 5 moving window with Neumann boundary 7 7 moving window with Neumann boundary global solver with zero Dirichlet boundary global solver with Neumann boundary FIGURE 5.4 THE DETAIL VISIBILITY CHANGE MAPS FOR THE NAVE IMAGE AS PROCESSED BY THE FIVE-SCALE TONE MAPPING OPERATORS 45

58 3 3 moving window with zero Dirichlet boundary 5 5 moving window with zero Dirichlet boundary 7 7 moving window with zero Dirichlet boundary 3 3 moving window with Neumann boundary 5 5 moving window with Neumann boundary 7 7 moving window with Neumann boundary global solver with zero Dirichlet boundary global solver with Neumann boundary FIGURE 5.5 THE MEMORIAL IMAGES AS PROCESSED BY THE SINLE-SCALE TONE MAPPING OPERATORS 46

59 3 3 moving window with zero Dirichlet boundary 5 5 moving window with zero Dirichlet boundary 7 7 moving window with zero Dirichlet boundary 3 3 moving window with Neumann boundary 5 5 moving window with Neumann boundary 7 7 moving window with Neumann boundary global solver with zero Dirichlet boundary global solver with Neumann boundary FIGURE 5.6 THE DETAIL VISIBILITY CHANGE MAPS FOR THE MEMORIAL IMAGE AS PROCESSED BY THE SINGLE-SCALE TONE MAPPING OPERATORS 47

60 3 3 moving window with zero Dirichlet boundary 5 5 moving window with zero Dirichlet boundary 7 7 moving window with zero Dirichlet boundary 3 3 moving window with Neumann boundary 5 5 moving window with Neumann boundary 7 7 moving window with Neumann boundary global solver with zero Dirichlet boundary global solver with Neumann boundary FIGURE 5.7 THE MEMORIAL IMAGE AS PROCESSED BY THE FIVE-SCALE TONE MAPPING OPERATORS 48

61 3 3 moving window with zero Dirichlet boundary 5 5 moving window with zero Dirichlet boundary 7 7 moving window with zero Dirichlet boundary 3 3 moving window with Neumann boundary 5 5 moving window with Neumann boundary 7 7 moving window with Neumann boundary global solver with zero Dirichlet boundary global solver with Neumann boundary FIGURE 5.8 THE DETAIL VISIBILITY CHANGE MAPS FOR THE MEMORIAL IMAGE AS PROCESSED BY THE FIVE-SCALE TONE MAPPING OPERATORS 49

Gradient Domain High Dynamic Range Compression

Gradient Domain High Dynamic Range Compression Gradient Domain High Dynamic Range Compression Raanan Fattal Dani Lischinski Michael Werman The Hebrew University of Jerusalem School of Computer Science & Engineering 1 In A Nutshell 2 Dynamic Range Quantity

More information

Edge Detection in Computer Vision Systems

Edge Detection in Computer Vision Systems 1 CS332 Visual Processing in Computer and Biological Vision Systems Edge Detection in Computer Vision Systems This handout summarizes much of the material on the detection and description of intensity

More information

SYDE 575: Introduction to Image Processing. Image Compression Part 2: Variable-rate compression

SYDE 575: Introduction to Image Processing. Image Compression Part 2: Variable-rate compression SYDE 575: Introduction to Image Processing Image Compression Part 2: Variable-rate compression Variable-rate Compression: Transform-based compression As mentioned earlier, we wish to transform image data

More information

8 The SVD Applied to Signal and Image Deblurring

8 The SVD Applied to Signal and Image Deblurring 8 The SVD Applied to Signal and Image Deblurring We will discuss the restoration of one-dimensional signals and two-dimensional gray-scale images that have been contaminated by blur and noise. After an

More information

IDENTIFICATION AND ANALYSIS OF TIME-VARYING MODAL PARAMETERS

IDENTIFICATION AND ANALYSIS OF TIME-VARYING MODAL PARAMETERS IDENTIFICATION AND ANALYSIS OF TIME-VARYING MODAL PARAMETERS By STEPHEN L. SORLEY A THESIS PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE

More information

Singular Value Decomposition and Digital Image Compression

Singular Value Decomposition and Digital Image Compression Singular Value Decomposition and Digital Image Compression Chris Bingham December 1, 016 Page 1 of Abstract The purpose of this document is to be a very basic introduction to the singular value decomposition

More information

ECE 592 Topics in Data Science

ECE 592 Topics in Data Science ECE 592 Topics in Data Science Final Fall 2017 December 11, 2017 Please remember to justify your answers carefully, and to staple your test sheet and answers together before submitting. Name: Student ID:

More information

8 The SVD Applied to Signal and Image Deblurring

8 The SVD Applied to Signal and Image Deblurring 8 The SVD Applied to Signal and Image Deblurring We will discuss the restoration of one-dimensional signals and two-dimensional gray-scale images that have been contaminated by blur and noise. After an

More information

6 The SVD Applied to Signal and Image Deblurring

6 The SVD Applied to Signal and Image Deblurring 6 The SVD Applied to Signal and Image Deblurring We will discuss the restoration of one-dimensional signals and two-dimensional gray-scale images that have been contaminated by blur and noise. After an

More information

Lossless Image and Intra-frame Compression with Integer-to-Integer DST

Lossless Image and Intra-frame Compression with Integer-to-Integer DST 1 Lossless Image and Intra-frame Compression with Integer-to-Integer DST Fatih Kamisli, Member, IEEE arxiv:1708.07154v1 [cs.mm] 3 Aug 017 Abstract Video coding standards are primarily designed for efficient

More information

What is Image Deblurring?

What is Image Deblurring? What is Image Deblurring? When we use a camera, we want the recorded image to be a faithful representation of the scene that we see but every image is more or less blurry, depending on the circumstances.

More information

Image Compression. 1. Introduction. Greg Ames Dec 07, 2002

Image Compression. 1. Introduction. Greg Ames Dec 07, 2002 Image Compression Greg Ames Dec 07, 2002 Abstract Digital images require large amounts of memory to store and, when retrieved from the internet, can take a considerable amount of time to download. The

More information

Contents. Acknowledgments

Contents. Acknowledgments Table of Preface Acknowledgments Notation page xii xx xxi 1 Signals and systems 1 1.1 Continuous and discrete signals 1 1.2 Unit step and nascent delta functions 4 1.3 Relationship between complex exponentials

More information

Screen-space processing Further Graphics

Screen-space processing Further Graphics Screen-space processing Rafał Mantiuk Computer Laboratory, University of Cambridge Cornell Box and tone-mapping Rendering Photograph 2 Real-world scenes are more challenging } The match could not be achieved

More information

arxiv: v1 [math.na] 5 May 2011

arxiv: v1 [math.na] 5 May 2011 ITERATIVE METHODS FOR COMPUTING EIGENVALUES AND EIGENVECTORS MAYSUM PANJU arxiv:1105.1185v1 [math.na] 5 May 2011 Abstract. We examine some numerical iterative methods for computing the eigenvalues and

More information

Implementation of CCSDS Recommended Standard for Image DC Compression

Implementation of CCSDS Recommended Standard for Image DC Compression Implementation of CCSDS Recommended Standard for Image DC Compression Sonika Gupta Post Graduate Student of Department of Embedded System Engineering G.H. Patel College Of Engineering and Technology Gujarat

More information

Econ Slides from Lecture 7

Econ Slides from Lecture 7 Econ 205 Sobel Econ 205 - Slides from Lecture 7 Joel Sobel August 31, 2010 Linear Algebra: Main Theory A linear combination of a collection of vectors {x 1,..., x k } is a vector of the form k λ ix i for

More information

c 2011 JOSHUA DAVID JOHNSTON ALL RIGHTS RESERVED

c 2011 JOSHUA DAVID JOHNSTON ALL RIGHTS RESERVED c 211 JOSHUA DAVID JOHNSTON ALL RIGHTS RESERVED ANALYTICALLY AND NUMERICALLY MODELING RESERVOIR-EXTENDED POROUS SLIDER AND JOURNAL BEARINGS INCORPORATING CAVITATION EFFECTS A Dissertation Presented to

More information

L. Yaroslavsky. Fundamentals of Digital Image Processing. Course

L. Yaroslavsky. Fundamentals of Digital Image Processing. Course L. Yaroslavsky. Fundamentals of Digital Image Processing. Course 0555.330 Lec. 6. Principles of image coding The term image coding or image compression refers to processing image digital data aimed at

More information

CHAPTER 4 PRINCIPAL COMPONENT ANALYSIS-BASED FUSION

CHAPTER 4 PRINCIPAL COMPONENT ANALYSIS-BASED FUSION 59 CHAPTER 4 PRINCIPAL COMPONENT ANALYSIS-BASED FUSION 4. INTRODUCTION Weighted average-based fusion algorithms are one of the widely used fusion methods for multi-sensor data integration. These methods

More information

Background Mathematics (2/2) 1. David Barber

Background Mathematics (2/2) 1. David Barber Background Mathematics (2/2) 1 David Barber University College London Modified by Samson Cheung (sccheung@ieee.org) 1 These slides accompany the book Bayesian Reasoning and Machine Learning. The book and

More information

Multimedia Networking ECE 599

Multimedia Networking ECE 599 Multimedia Networking ECE 599 Prof. Thinh Nguyen School of Electrical Engineering and Computer Science Based on lectures from B. Lee, B. Girod, and A. Mukherjee 1 Outline Digital Signal Representation

More information

Basic Principles of Video Coding

Basic Principles of Video Coding Basic Principles of Video Coding Introduction Categories of Video Coding Schemes Information Theory Overview of Video Coding Techniques Predictive coding Transform coding Quantization Entropy coding Motion

More information

On Compression Encrypted Data part 2. Prof. Ja-Ling Wu The Graduate Institute of Networking and Multimedia National Taiwan University

On Compression Encrypted Data part 2. Prof. Ja-Ling Wu The Graduate Institute of Networking and Multimedia National Taiwan University On Compression Encrypted Data part 2 Prof. Ja-Ling Wu The Graduate Institute of Networking and Multimedia National Taiwan University 1 Brief Summary of Information-theoretic Prescription At a functional

More information

Mobile Robotics 1. A Compact Course on Linear Algebra. Giorgio Grisetti

Mobile Robotics 1. A Compact Course on Linear Algebra. Giorgio Grisetti Mobile Robotics 1 A Compact Course on Linear Algebra Giorgio Grisetti SA-1 Vectors Arrays of numbers They represent a point in a n dimensional space 2 Vectors: Scalar Product Scalar-Vector Product Changes

More information

Stripey Squares. Kelly Delp Ithaca College Ithaca, NY 14850, USA Abstract

Stripey Squares. Kelly Delp Ithaca College Ithaca, NY 14850, USA Abstract Stripey Squares Kelly Delp Ithaca College Ithaca, NY 14850, USA kelly.delp@gmail.com Abstract A black and white image can be represented by a matrix containing numbers between 0 and 1; we describe a way

More information

Singular Value Decompsition

Singular Value Decompsition Singular Value Decompsition Massoud Malek One of the most useful results from linear algebra, is a matrix decomposition known as the singular value decomposition It has many useful applications in almost

More information

Learning goals: students learn to use the SVD to find good approximations to matrices and to compute the pseudoinverse.

Learning goals: students learn to use the SVD to find good approximations to matrices and to compute the pseudoinverse. Application of the SVD: Compression and Pseudoinverse Learning goals: students learn to use the SVD to find good approximations to matrices and to compute the pseudoinverse. Low rank approximation One

More information

Number Representation and Waveform Quantization

Number Representation and Waveform Quantization 1 Number Representation and Waveform Quantization 1 Introduction This lab presents two important concepts for working with digital signals. The first section discusses how numbers are stored in memory.

More information

Section 4.4 Reduction to Symmetric Tridiagonal Form

Section 4.4 Reduction to Symmetric Tridiagonal Form Section 4.4 Reduction to Symmetric Tridiagonal Form Key terms Symmetric matrix conditioning Tridiagonal matrix Similarity transformation Orthogonal matrix Orthogonal similarity transformation properties

More information

The Solution of Linear Systems AX = B

The Solution of Linear Systems AX = B Chapter 2 The Solution of Linear Systems AX = B 21 Upper-triangular Linear Systems We will now develop the back-substitution algorithm, which is useful for solving a linear system of equations that has

More information

Lecture 7 Predictive Coding & Quantization

Lecture 7 Predictive Coding & Quantization Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Lecture 7 Predictive Coding & Quantization June 3, 2009 Outline Predictive Coding Motion Estimation and Compensation Context-Based Coding Quantization

More information

9. Image filtering in the spatial and frequency domains

9. Image filtering in the spatial and frequency domains Image Processing - Laboratory 9: Image filtering in the spatial and frequency domains 9. Image filtering in the spatial and frequency domains 9.. Introduction In this laboratory the convolution operator

More information

PHASE RETRIEVAL OF SPARSE SIGNALS FROM MAGNITUDE INFORMATION. A Thesis MELTEM APAYDIN

PHASE RETRIEVAL OF SPARSE SIGNALS FROM MAGNITUDE INFORMATION. A Thesis MELTEM APAYDIN PHASE RETRIEVAL OF SPARSE SIGNALS FROM MAGNITUDE INFORMATION A Thesis by MELTEM APAYDIN Submitted to the Office of Graduate and Professional Studies of Texas A&M University in partial fulfillment of the

More information

A Comprehensive Framework for Modeling and Correcting Color Variations in Multi Projector Displays

A Comprehensive Framework for Modeling and Correcting Color Variations in Multi Projector Displays A Comprehensive Framework for Modeling and Correcting Color Variations in Multi Projector Displays Aditi Majumder Slide 1 Historical Perspective A dream computer in 1980 (Bill Joy and others) Megabytes

More information

Preface to the Second Edition. Preface to the First Edition

Preface to the Second Edition. Preface to the First Edition n page v Preface to the Second Edition Preface to the First Edition xiii xvii 1 Background in Linear Algebra 1 1.1 Matrices................................. 1 1.2 Square Matrices and Eigenvalues....................

More information

Word-length Optimization and Error Analysis of a Multivariate Gaussian Random Number Generator

Word-length Optimization and Error Analysis of a Multivariate Gaussian Random Number Generator Word-length Optimization and Error Analysis of a Multivariate Gaussian Random Number Generator Chalermpol Saiprasert, Christos-Savvas Bouganis and George A. Constantinides Department of Electrical & Electronic

More information

Computational Techniques Prof. Sreenivas Jayanthi. Department of Chemical Engineering Indian institute of Technology, Madras

Computational Techniques Prof. Sreenivas Jayanthi. Department of Chemical Engineering Indian institute of Technology, Madras Computational Techniques Prof. Sreenivas Jayanthi. Department of Chemical Engineering Indian institute of Technology, Madras Module No. # 05 Lecture No. # 24 Gauss-Jordan method L U decomposition method

More information

(Linear equations) Applied Linear Algebra in Geoscience Using MATLAB

(Linear equations) Applied Linear Algebra in Geoscience Using MATLAB Applied Linear Algebra in Geoscience Using MATLAB (Linear equations) Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots

More information

DESIGNING CNN GENES. Received January 23, 2003; Revised April 2, 2003

DESIGNING CNN GENES. Received January 23, 2003; Revised April 2, 2003 Tutorials and Reviews International Journal of Bifurcation and Chaos, Vol. 13, No. 10 (2003 2739 2824 c World Scientific Publishing Company DESIGNING CNN GENES MAKOTO ITOH Department of Information and

More information

Introduction to Video Compression H.261

Introduction to Video Compression H.261 Introduction to Video Compression H.6 Dirk Farin, Contact address: Dirk Farin University of Mannheim Dept. Computer Science IV L 5,6, 683 Mannheim, Germany farin@uni-mannheim.de D.F. YUV-Colorspace Computer

More information

Image Compression. Fundamentals: Coding redundancy. The gray level histogram of an image can reveal a great deal of information about the image

Image Compression. Fundamentals: Coding redundancy. The gray level histogram of an image can reveal a great deal of information about the image Fundamentals: Coding redundancy The gray level histogram of an image can reveal a great deal of information about the image That probability (frequency) of occurrence of gray level r k is p(r k ), p n

More information

Multivariate Gaussian Random Number Generator Targeting Specific Resource Utilization in an FPGA

Multivariate Gaussian Random Number Generator Targeting Specific Resource Utilization in an FPGA Multivariate Gaussian Random Number Generator Targeting Specific Resource Utilization in an FPGA Chalermpol Saiprasert, Christos-Savvas Bouganis and George A. Constantinides Department of Electrical &

More information

encoding without prediction) (Server) Quantization: Initial Data 0, 1, 2, Quantized Data 0, 1, 2, 3, 4, 8, 16, 32, 64, 128, 256

encoding without prediction) (Server) Quantization: Initial Data 0, 1, 2, Quantized Data 0, 1, 2, 3, 4, 8, 16, 32, 64, 128, 256 General Models for Compression / Decompression -they apply to symbols data, text, and to image but not video 1. Simplest model (Lossless ( encoding without prediction) (server) Signal Encode Transmit (client)

More information

Product Obsolete/Under Obsolescence. Quantization. Author: Latha Pillai

Product Obsolete/Under Obsolescence. Quantization. Author: Latha Pillai Application Note: Virtex and Virtex-II Series XAPP615 (v1.1) June 25, 2003 R Quantization Author: Latha Pillai Summary This application note describes a reference design to do a quantization and inverse

More information

Interpreting Deep Classifiers

Interpreting Deep Classifiers Ruprecht-Karls-University Heidelberg Faculty of Mathematics and Computer Science Seminar: Explainable Machine Learning Interpreting Deep Classifiers by Visual Distillation of Dark Knowledge Author: Daniela

More information

Automatic estimation of crowd size and target detection using Image processing

Automatic estimation of crowd size and target detection using Image processing Automatic estimation of crowd size and target detection using Image processing Asst Prof. Avinash Rai Dept. of Electronics and communication (UIT-RGPV) Bhopal avinashrai@rgtu.net Rahul Meshram Dept. of

More information

Vlad Estivill-Castro (2016) Robots for People --- A project for intelligent integrated systems

Vlad Estivill-Castro (2016) Robots for People --- A project for intelligent integrated systems 1 Vlad Estivill-Castro (2016) Robots for People --- A project for intelligent integrated systems V. Estivill-Castro 2 Perception Concepts Vision Chapter 4 (textbook) Sections 4.3 to 4.5 What is the course

More information

Over-enhancement Reduction in Local Histogram Equalization using its Degrees of Freedom. Alireza Avanaki

Over-enhancement Reduction in Local Histogram Equalization using its Degrees of Freedom. Alireza Avanaki Over-enhancement Reduction in Local Histogram Equalization using its Degrees of Freedom Alireza Avanaki ABSTRACT A well-known issue of local (adaptive) histogram equalization (LHE) is over-enhancement

More information

Optimization of new Chinese Remainder theorems using special moduli sets

Optimization of new Chinese Remainder theorems using special moduli sets Louisiana State University LSU Digital Commons LSU Master's Theses Graduate School 2010 Optimization of new Chinese Remainder theorems using special moduli sets Narendran Narayanaswamy Louisiana State

More information

Exercise Sheet 1. 1 Probability revision 1: Student-t as an infinite mixture of Gaussians

Exercise Sheet 1. 1 Probability revision 1: Student-t as an infinite mixture of Gaussians Exercise Sheet 1 1 Probability revision 1: Student-t as an infinite mixture of Gaussians Show that an infinite mixture of Gaussian distributions, with Gamma distributions as mixing weights in the following

More information

Lecture 7: Edge Detection

Lecture 7: Edge Detection #1 Lecture 7: Edge Detection Saad J Bedros sbedros@umn.edu Review From Last Lecture Definition of an Edge First Order Derivative Approximation as Edge Detector #2 This Lecture Examples of Edge Detection

More information

Review for Exam 1. Erik G. Learned-Miller Department of Computer Science University of Massachusetts, Amherst Amherst, MA

Review for Exam 1. Erik G. Learned-Miller Department of Computer Science University of Massachusetts, Amherst Amherst, MA Review for Exam Erik G. Learned-Miller Department of Computer Science University of Massachusetts, Amherst Amherst, MA 0003 March 26, 204 Abstract Here are some things you need to know for the in-class

More information

Least Squares with Examples in Signal Processing 1. 2 Overdetermined equations. 1 Notation. The sum of squares of x is denoted by x 2 2, i.e.

Least Squares with Examples in Signal Processing 1. 2 Overdetermined equations. 1 Notation. The sum of squares of x is denoted by x 2 2, i.e. Least Squares with Eamples in Signal Processing Ivan Selesnick March 7, 3 NYU-Poly These notes address (approimate) solutions to linear equations by least squares We deal with the easy case wherein the

More information

Solving the Yang-Baxter Matrix Equation

Solving the Yang-Baxter Matrix Equation The University of Southern Mississippi The Aquila Digital Community Honors Theses Honors College 5-7 Solving the Yang-Baxter Matrix Equation Mallory O Jennings Follow this and additional works at: http://aquilausmedu/honors_theses

More information

PDE Based Image Diffusion and AOS

PDE Based Image Diffusion and AOS PDE Based Image Diffusion and AOS by Jarno Ralli April 27, 204 Abstract This technical paper shows how AOS Additive Operator Splitting scheme, together with TDMA Tri-Diagonal Matrix Algorithm, can be used

More information

APPENDIX A. Background Mathematics. A.1 Linear Algebra. Vector algebra. Let x denote the n-dimensional column vector with components x 1 x 2.

APPENDIX A. Background Mathematics. A.1 Linear Algebra. Vector algebra. Let x denote the n-dimensional column vector with components x 1 x 2. APPENDIX A Background Mathematics A. Linear Algebra A.. Vector algebra Let x denote the n-dimensional column vector with components 0 x x 2 B C @. A x n Definition 6 (scalar product). The scalar product

More information

Final Examination. CS 205A: Mathematical Methods for Robotics, Vision, and Graphics (Fall 2013), Stanford University

Final Examination. CS 205A: Mathematical Methods for Robotics, Vision, and Graphics (Fall 2013), Stanford University Final Examination CS 205A: Mathematical Methods for Robotics, Vision, and Graphics (Fall 2013), Stanford University The exam runs for 3 hours. The exam contains eight problems. You must complete the first

More information

Designing Information Devices and Systems I Fall 2017 Official Lecture Notes Note 2

Designing Information Devices and Systems I Fall 2017 Official Lecture Notes Note 2 EECS 6A Designing Information Devices and Systems I Fall 07 Official Lecture Notes Note Introduction Previously, we introduced vectors and matrices as a way of writing systems of linear equations more

More information

Lecture 24: Principal Component Analysis. Aykut Erdem May 2016 Hacettepe University

Lecture 24: Principal Component Analysis. Aykut Erdem May 2016 Hacettepe University Lecture 4: Principal Component Analysis Aykut Erdem May 016 Hacettepe University This week Motivation PCA algorithms Applications PCA shortcomings Autoencoders Kernel PCA PCA Applications Data Visualization

More information

Contents. Preface... xi. Introduction...

Contents. Preface... xi. Introduction... Contents Preface... xi Introduction... xv Chapter 1. Computer Architectures... 1 1.1. Different types of parallelism... 1 1.1.1. Overlap, concurrency and parallelism... 1 1.1.2. Temporal and spatial parallelism

More information

Data Mining. Dimensionality reduction. Hamid Beigy. Sharif University of Technology. Fall 1395

Data Mining. Dimensionality reduction. Hamid Beigy. Sharif University of Technology. Fall 1395 Data Mining Dimensionality reduction Hamid Beigy Sharif University of Technology Fall 1395 Hamid Beigy (Sharif University of Technology) Data Mining Fall 1395 1 / 42 Outline 1 Introduction 2 Feature selection

More information

Detection of Motor Vehicles and Humans on Ocean Shoreline. Seif Abu Bakr

Detection of Motor Vehicles and Humans on Ocean Shoreline. Seif Abu Bakr Detection of Motor Vehicles and Humans on Ocean Shoreline Seif Abu Bakr Dec 14, 2009 Boston University Department of Electrical and Computer Engineering Technical report No.ECE-2009-05 BOSTON UNIVERSITY

More information

Lecture 8: Interest Point Detection. Saad J Bedros

Lecture 8: Interest Point Detection. Saad J Bedros #1 Lecture 8: Interest Point Detection Saad J Bedros sbedros@umn.edu Review of Edge Detectors #2 Today s Lecture Interest Points Detection What do we mean with Interest Point Detection in an Image Goal:

More information

Computer Vision Lecture 3

Computer Vision Lecture 3 Computer Vision Lecture 3 Linear Filters 03.11.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Demo Haribo Classification Code available on the class website...

More information

Linear Algebra Done Wrong. Sergei Treil. Department of Mathematics, Brown University

Linear Algebra Done Wrong. Sergei Treil. Department of Mathematics, Brown University Linear Algebra Done Wrong Sergei Treil Department of Mathematics, Brown University Copyright c Sergei Treil, 2004, 2009 Preface The title of the book sounds a bit mysterious. Why should anyone read this

More information

The New Graphic Description of the Haar Wavelet Transform

The New Graphic Description of the Haar Wavelet Transform he New Graphic Description of the Haar Wavelet ransform Piotr Porwik and Agnieszka Lisowska Institute of Informatics, Silesian University, ul.b dzi ska 39, 4-00 Sosnowiec, Poland porwik@us.edu.pl Institute

More information

Roadmap. Introduction to image analysis (computer vision) Theory of edge detection. Applications

Roadmap. Introduction to image analysis (computer vision) Theory of edge detection. Applications Edge Detection Roadmap Introduction to image analysis (computer vision) Its connection with psychology and neuroscience Why is image analysis difficult? Theory of edge detection Gradient operator Advanced

More information

Study Notes on Matrices & Determinants for GATE 2017

Study Notes on Matrices & Determinants for GATE 2017 Study Notes on Matrices & Determinants for GATE 2017 Matrices and Determinates are undoubtedly one of the most scoring and high yielding topics in GATE. At least 3-4 questions are always anticipated from

More information

ELIMINATION OF RAINDROPS EFFECTS IN INFRARED SENSITIVE CAMERA AHMAD SHARMI BIN ABDULLAH

ELIMINATION OF RAINDROPS EFFECTS IN INFRARED SENSITIVE CAMERA AHMAD SHARMI BIN ABDULLAH ELIMINATION OF RAINDROPS EFFECTS IN INFRARED SENSITIVE CAMERA AHMAD SHARMI BIN ABDULLAH A project report submitted in partial fulfillment of the requirements for the award of the degree of Master of Engineering

More information

Getting Started with Communications Engineering

Getting Started with Communications Engineering 1 Linear algebra is the algebra of linear equations: the term linear being used in the same sense as in linear functions, such as: which is the equation of a straight line. y ax c (0.1) Of course, if we

More information

Special Two-Semester Linear Algebra Course (Fall 2012 and Spring 2013)

Special Two-Semester Linear Algebra Course (Fall 2012 and Spring 2013) Special Two-Semester Linear Algebra Course (Fall 2012 and Spring 2013) The first semester will concentrate on basic matrix skills as described in MA 205, and the student should have one semester of calculus.

More information

Lecture 6: Edge Detection. CAP 5415: Computer Vision Fall 2008

Lecture 6: Edge Detection. CAP 5415: Computer Vision Fall 2008 Lecture 6: Edge Detection CAP 5415: Computer Vision Fall 2008 Announcements PS 2 is available Please read it by Thursday During Thursday lecture, I will be going over it in some detail Monday - Computer

More information

Contents. Preface for the Instructor. Preface for the Student. xvii. Acknowledgments. 1 Vector Spaces 1 1.A R n and C n 2

Contents. Preface for the Instructor. Preface for the Student. xvii. Acknowledgments. 1 Vector Spaces 1 1.A R n and C n 2 Contents Preface for the Instructor xi Preface for the Student xv Acknowledgments xvii 1 Vector Spaces 1 1.A R n and C n 2 Complex Numbers 2 Lists 5 F n 6 Digression on Fields 10 Exercises 1.A 11 1.B Definition

More information

Linear Algebra. and

Linear Algebra. and Instructions Please answer the six problems on your own paper. These are essay questions: you should write in complete sentences. 1. Are the two matrices 1 2 2 1 3 5 2 7 and 1 1 1 4 4 2 5 5 2 row equivalent?

More information

LOSSLESS INTRA CODING IN HEVC WITH INTEGER-TO-INTEGER DST. Fatih Kamisli. Middle East Technical University Ankara, Turkey

LOSSLESS INTRA CODING IN HEVC WITH INTEGER-TO-INTEGER DST. Fatih Kamisli. Middle East Technical University Ankara, Turkey LOSSLESS INTRA CODING IN HEVC WITH INTEGER-TO-INTEGER DST Fatih Kamisli Middle East Technical University Ankara, Turkey ABSTRACT It is desirable to support efficient lossless coding within video coding

More information

Matrix Algebra for Engineers Jeffrey R. Chasnov

Matrix Algebra for Engineers Jeffrey R. Chasnov Matrix Algebra for Engineers Jeffrey R. Chasnov The Hong Kong University of Science and Technology The Hong Kong University of Science and Technology Department of Mathematics Clear Water Bay, Kowloon

More information

Numerical Linear Algebra

Numerical Linear Algebra Numerical Linear Algebra By: David McQuilling; Jesus Caban Deng Li Jan.,31,006 CS51 Solving Linear Equations u + v = 8 4u + 9v = 1 A x b 4 9 u v = 8 1 Gaussian Elimination Start with the matrix representation

More information

Inverse Problems in Image Processing

Inverse Problems in Image Processing H D Inverse Problems in Image Processing Ramesh Neelamani (Neelsh) Committee: Profs. R. Baraniuk, R. Nowak, M. Orchard, S. Cox June 2003 Inverse Problems Data estimation from inadequate/noisy observations

More information

Matrices and Vectors. Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A =

Matrices and Vectors. Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A = 30 MATHEMATICS REVIEW G A.1.1 Matrices and Vectors Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A = a 11 a 12... a 1N a 21 a 22... a 2N...... a M1 a M2... a MN A matrix can

More information

Overview. Analog capturing device (camera, microphone) PCM encoded or raw signal ( wav, bmp, ) A/D CONVERTER. Compressed bit stream (mp3, jpg, )

Overview. Analog capturing device (camera, microphone) PCM encoded or raw signal ( wav, bmp, ) A/D CONVERTER. Compressed bit stream (mp3, jpg, ) Overview Analog capturing device (camera, microphone) Sampling Fine Quantization A/D CONVERTER PCM encoded or raw signal ( wav, bmp, ) Transform Quantizer VLC encoding Compressed bit stream (mp3, jpg,

More information

Advanced Introduction to Machine Learning CMU-10715

Advanced Introduction to Machine Learning CMU-10715 Advanced Introduction to Machine Learning CMU-10715 Principal Component Analysis Barnabás Póczos Contents Motivation PCA algorithms Applications Some of these slides are taken from Karl Booksh Research

More information

Linear Algebra Review. Fei-Fei Li

Linear Algebra Review. Fei-Fei Li Linear Algebra Review Fei-Fei Li 1 / 51 Vectors Vectors and matrices are just collections of ordered numbers that represent something: movements in space, scaling factors, pixel brightnesses, etc. A vector

More information

Wavefront reconstruction for adaptive optics. Marcos van Dam and Richard Clare W.M. Keck Observatory

Wavefront reconstruction for adaptive optics. Marcos van Dam and Richard Clare W.M. Keck Observatory Wavefront reconstruction for adaptive optics Marcos van Dam and Richard Clare W.M. Keck Observatory Friendly people We borrowed slides from the following people: Lisa Poyneer Luc Gilles Curt Vogel Corinne

More information

Lecture 8: Interest Point Detection. Saad J Bedros

Lecture 8: Interest Point Detection. Saad J Bedros #1 Lecture 8: Interest Point Detection Saad J Bedros sbedros@umn.edu Last Lecture : Edge Detection Preprocessing of image is desired to eliminate or at least minimize noise effects There is always tradeoff

More information

1 GSW Sets of Systems

1 GSW Sets of Systems 1 Often, we have to solve a whole series of sets of simultaneous equations of the form y Ax, all of which have the same matrix A, but each of which has a different known vector y, and a different unknown

More information

SIMON FRASER UNIVERSITY School of Engineering Science

SIMON FRASER UNIVERSITY School of Engineering Science SIMON FRASER UNIVERSITY School of Engineering Science Course Outline ENSC 810-3 Digital Signal Processing Calendar Description This course covers advanced digital signal processing techniques. The main

More information

Principal Components Analysis. Sargur Srihari University at Buffalo

Principal Components Analysis. Sargur Srihari University at Buffalo Principal Components Analysis Sargur Srihari University at Buffalo 1 Topics Projection Pursuit Methods Principal Components Examples of using PCA Graphical use of PCA Multidimensional Scaling Srihari 2

More information

Cyclic Reduction History and Applications

Cyclic Reduction History and Applications ETH Zurich and Stanford University Abstract. We discuss the method of Cyclic Reduction for solving special systems of linear equations that arise when discretizing partial differential equations. In connection

More information

[POLS 8500] Review of Linear Algebra, Probability and Information Theory

[POLS 8500] Review of Linear Algebra, Probability and Information Theory [POLS 8500] Review of Linear Algebra, Probability and Information Theory Professor Jason Anastasopoulos ljanastas@uga.edu January 12, 2017 For today... Basic linear algebra. Basic probability. Programming

More information

Application of Neugebauer-Based Models to Ceramic Printing

Application of Neugebauer-Based Models to Ceramic Printing Application of Neugebauer-Based Models to Ceramic Printing L Iovine, S Westland* and TLV Cheung* Colour and Imaging Institute, University of Derby, Derby, United Kingdom *Centre for Colour Design Technology,

More information

Introduction to Mobile Robotics Compact Course on Linear Algebra. Wolfram Burgard, Bastian Steder

Introduction to Mobile Robotics Compact Course on Linear Algebra. Wolfram Burgard, Bastian Steder Introduction to Mobile Robotics Compact Course on Linear Algebra Wolfram Burgard, Bastian Steder Reference Book Thrun, Burgard, and Fox: Probabilistic Robotics Vectors Arrays of numbers Vectors represent

More information

FILTERING IN THE FREQUENCY DOMAIN

FILTERING IN THE FREQUENCY DOMAIN 1 FILTERING IN THE FREQUENCY DOMAIN Lecture 4 Spatial Vs Frequency domain 2 Spatial Domain (I) Normal image space Changes in pixel positions correspond to changes in the scene Distances in I correspond

More information

Lecture 11. Advanced Dividers

Lecture 11. Advanced Dividers Lecture 11 Advanced Dividers Required Reading Behrooz Parhami, Computer Arithmetic: Algorithms and Hardware Design Chapter 15 Variation in Dividers 15.3, Combinational and Array Dividers Chapter 16, Division

More information

The Mathematics of Facial Recognition

The Mathematics of Facial Recognition William Dean Gowin Graduate Student Appalachian State University July 26, 2007 Outline EigenFaces Deconstruct a known face into an N-dimensional facespace where N is the number of faces in our data set.

More information

ECG782: Multidimensional Digital Signal Processing

ECG782: Multidimensional Digital Signal Processing Professor Brendan Morris, SEB 3216, brendan.morris@unlv.edu ECG782: Multidimensional Digital Signal Processing Filtering in the Frequency Domain http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Background

More information

Lecture Notes 5: Multiresolution Analysis

Lecture Notes 5: Multiresolution Analysis Optimization-based data analysis Fall 2017 Lecture Notes 5: Multiresolution Analysis 1 Frames A frame is a generalization of an orthonormal basis. The inner products between the vectors in a frame and

More information

When taking a picture, what color is a (Lambertian) surface?

When taking a picture, what color is a (Lambertian) surface? Shadow removal When taking a picture, what color is a (Lambertian) surface? What if it s not a cloudy day? Region lit by skylight only Region lit by sunlight and skylight What great things could we do

More information

STA 414/2104: Lecture 8

STA 414/2104: Lecture 8 STA 414/2104: Lecture 8 6-7 March 2017: Continuous Latent Variable Models, Neural networks With thanks to Russ Salakhutdinov, Jimmy Ba and others Outline Continuous latent variable models Background PCA

More information

Image Compression Using Singular Value Decomposition

Image Compression Using Singular Value Decomposition Image Compression Using Singular Value Decomposition Ian Cooper and Craig Lorenc December 15, 2006 Abstract Singular value decomposition (SVD) is an effective tool for minimizing data storage and data

More information