Perceptual Image Coding: Introduction of Φ SET Image Compression System

Transcription

1 Perceptual Image Coding: Introduction of Φ SET Image Compression System Jesús Jaime Moreno Escobar Communication and Electronics Engineering Department Superior School of Mechanical and Electrical Engineers National Polytechnic Institute, Mexico

2 1 Introduction This Chapter introduces perceptual criteria into the Hilbert Scanning of Embedded quadtrees Image Coder or Hi-SET (Moreno & Otazu, 2011a; Moreno & Otazu, 2011b), namely Perceptual-Hi-SET, PHi-SET or Φ SET. It is important to note that one of the biggest challenges of image compressors is the massive storage and ordering of coefficients coordinates. Some algorithms, like, SPECK (Pearlman & Said, 2008a; Pearlman & Said, 2008b; Pearlman et al., 2004) SPIHT (Said & Pearlman, 1996) and EZW (Shapiro, 1993), are based on the fact that the execution path will give the correct order, as a result of comparison of its branching points(usevitch, 2001). While Φ SET coder makes use of a Hilbert Scanning, which exploits the self-similarity of pixels. Hence, applying a Hilbert Scanning to Wavelet Transform coefficients takes the advantage of the similarity of neighbor pixels, helping to develop a optimal progressive transmission coder. In this way, at any step of the decoding process the quality of the recovered image is the best that can be achieved for the number of bits processed by the decoder up to that moment. Also the Hilbert s Space- Filling Curve gives by oneself its coordinate, since each branch belongs to a big one unless this is a root branch. Thereby, the decoder just needs the magnitudes in order to recover a wavelet coefficient. It is important to say that the JPEG2000 standard is intended to provide rate-distortion and subjective image quality performance superior to existing standards, as well as to supply functionality (Boliek et al., 2000). However, JPEG2000 does not provide the most relevant characteristics of the human visual system, since for removing information, in order to compress the image, mainly information theory criteria are applied. This information removal introduces artifacts to the image that are visible at high compression rates, because of many pixels with high perceptual significance have been discarded. Hence, it is necessary a model that removes information according to perceptual criteria, preserving the pixels with high perceptual relevance regardless of the numerical information. The Chromatic Induction Wavelet Model (CIWaM) presents some perceptual concepts that can be suitable for it. Both CIWaM and JPEG2000 use a Wavelet Transformation. CIWaM uses this kind of Transformation in order to generate an approximation to how every pixel is perceived from a certain distance taking into account the value of its neighboring pixels. By contrast, JPEG2000 applies a perceptual criteria for all coefficients in a certain spatial frequency independently of the values of its surrounding ones. In other words, JPEG2000 performs a global weighting of wavelet coefficients, while Φ SET performs a local one. CIWaM attenuates the details that the human visual system is not able to perceive, enhances those that are perceptually relevant and produces an approximation of the image that the brain visual cortex perceives. At long distances, as Figure 3(d) depicts, the lack of information does not produce the well-known compression artifacts, rather it is presented as a softened version, where the details with high perceptual value remain (for example, some edges (Murray et al., 2010)). The block diagram of the Φ SET engine for encoding and decoding is shown in Figure 1. The source data is a RGB image, which comprises three components, then a color transformation is first applied over all three components. After the color transformation, each component is decomposed with a discrete wavelet transform into a set of planes of different spatial frequencies by means of a Forward Wavelet Transformation (9/7 analysis Filter). Then, these coefficients are Forward Perceptually Quantized using CIWaM, for reducing the precision of data in order to make them more perceptually compressible. Perceptual Quantization is the only responsible that introduces imperceptible lossless distortion into the image data. Then, Hi-SET algorithm is employed for entropy encoding among quantized coefficients forming the output bit stream. The decoding process is the inverse of the encoding one. The bit stream is first entropy decoded by

3 means of Hi-SET, perceptually dequantized, inverse discrete wavelet transformed and finally inverse color transformed, getting as a result the reconstructed image data. Source Image Data Forward Transform Perceptual Quantization Hi-SET Coding Compressed Image Data Hi-SET Decoding Perceptual Inverse Quantization Inverse Transform Reconstructed Image Data Figure 1: General block diagram for Φ SET encoding and decoding. 2 Chromatic Induction Wavelet Model: Brief description. CIWaM is a low-level perceptual model of the HVS(Otazu et al., 2010). It estimates the image perceived by an observer at a distance d just by modeling the perceptual chromatic induction processes of the Human Visual System (HVS). That is, given an image I and an observation distance d, CIWaM obtains an estimation of the perceptual image I ρ that the observer perceives when observing I at distance d. CIWaM is based on just three important stimulus properties: spatial frequency, spatial orientation and surround contrast. These three properties allow to unify the chromatic assimilation and contrast phenomena, as well as some other perceptual processes such as saliency perceptual processes. (a) Figure 2: (a) Graphical representation of the e-csf (α s,o,i (r,ν))) for the luminance channel. (b) Some profiles of the same surface along the Spatial Frequency (ν) axis for different center surround contrast energy ratio values (r). The psychophysically measured CSF is a particular case of this family of curves (concretely for r = 1). CIWaM takes an input image I and decomposes it into a set of wavelet planes ω s,o of different spatial scales s (i.e., spatial frequency ν) and spatial orientations o. It is described as: n I = s=1 o=v,h,dgl (b) ω s,o +c n, (1)

4 where n is the number of wavelet planes, c n is the residual plane and o is the spatial orientation either vertical, horizontal or diagonal. The perceptual image I ρ is recovered by weighting these ω s,o wavelet coefficients using the extended Contrast Sensitivity Function (e-csf, Fig. 2). The e-csf is an extension of the psychophysical CSF (Mullen, 1985a) considering spatial surround information (denoted by r), visual frequency (denoted by ν, which is related to spatial frequency by observation distance) and observation distance (d). Perceptual image I ρ can be obtained by n I ρ = s=1 o=v,h,dgl α(ν,r) ω s,o +c n, (2) where α(ν,r) is the e-csf weighting function that tries to reproduce some perceptual properties of the HVS. The term α(ν,r) ω s,o ω s,o;ρ,d can be considered the perceptual wavelet coefficients of image I when observed at distance d and is written as: α(ν,r) = z ctr C d (ṡ)+c min (ṡ). (3) This function has a shape similar to the e-csf and the three terms that describe this function are defined as: z ctr Non-linear function and estimation of the central feature contrast relative to its surround contrast, oscillating from zero to one, defined by: z ctr = [ σ cen σ sur ] 2 1+[ σ cen σ sur ] 2 (4) being σ cen and σ sur the standard deviation of the wavelet coefficients in two concentric rings, which represent a center surround interaction around each coefficient. C d (ṡ) Weighting function that approximates to the perceptual e-csf, emulates some perceptual properties and is defined as a piecewise Gaussian function (Mullen, 1985b), such as: C d (ṡ) = e ṡ 2 e ṡ 2σ 2 1, ṡ = s s thr 0, 2 2σ 2 2, ṡ = s s thr > 0. (5) C min (ṡ) Term that avoids α(ν,r) function to be zero and is defined by: C min (ṡ) = ṡ e 2σ 2 1, ṡ = s s thr 0, 1 2, ṡ = s s thr > 0. taking σ 1 = 2 and σ 2 = 2σ 1. Both C min (ṡ) and C d (ṡ) depend on the factor s thr, which is the scale associated to 4 cycles per degree (cpd) when an image is observed from the distance d with a pixel size l p and one visual degree, whose expression is defined by Equation 7. Where s thr value is associated to the e-csf maximum value. (6) s thr = log 2 ( d tan(1 ) 4 l p ) (7) Figure 3 shows three examples of CIWaM images of Lenna, calculated by Equation 2 for a 19 inch monitor with 1280 pixels of horizontal resolution, at d = {30,100,200} centimeters.

5 (a) Original image (b) d=30 cm. (c) d=100 cm. (d) d=200 cm. Figure 3: (a) Original color image Lenna. (b)-(d) Perceptual images obtained by CIWaM at different observation distances d. 3 Image Quantization 3.1 State-of-the-art Quantization: Dead-zone Uniform Scalar Algorithm Dead-zone Uniform Scalar Forward Quantization Marcellin et al. summarize in (Marcellin et al., 2002), among others, the uniform scalar quantizer (SQ). This quantizer is described as a function that maps each element of a subset of the real numbers into a particular value, which ensures that more zeros result. This way, quantized values are uniformly spaced by step size except for the interval containing the zero value, which is called the dead-zone, that extends from to +. Thus, a dead-zone means that the quantization range around 0 is 2.

6 Taking a given wavelet plane ω o s, a particular quantization step size o s is used to quantize all the coefficients in that spatial frequency s and orientation o. Hence a particular quantized index is defined as: q = sign(y) y (8) o s where y is the input to the quantizer (i.e., the original wavelet coefficient value), sign(y) denotes the sign of y and q is the resulting quantized indexes, namely quantized coefficients Dead-zone Uniform Scalar Inverse Quantization The inverse quantizer or the reconstructed ŷ is given by: ŷ = (q+δ) o s, q > 0 (q δ) o s q < 0 0, q = 0 (9) where δ is a parameter often set to place the reconstruction value at the centroid of the quantization interval and varies form 0 to 1. The International Organization for Standardization recommends to adopt the mid-point reconstruction value, setting δ = 0.5 (Boliek et al., 2000). Experience indicates that some small improvements can be obtained by selecting a slightly smaller value. Pearlman and Said in (Pearlman & Said, 2008a) suggest δ = 0.375, especially for higher frequency subbands (e.g. high frequency wavelet planes). It is important to realize, when < y <, the quantizer level and reconstruction value are both 0. Since it is known that many coefficients in a wavelet transform are close to zero (usually those of higher frequencies), it implies that they are on the dead-zone, thus the quantizer is going to set them to q=0. Thus taking a wavelet plane o s, an array of quantized coefficients is obtained for further losslessly encoding, since the image degradations are induced only by the Quantization process. 3.2 Perceptual Quantization Forward Perceptual Quantization Quantization is the only cause that introduces distortion into a compression process. Since each transformed sample at the perceptual image I ρ (from Eq. 2) is mapped independently to a corresponding step size either s or n, thus I ρ is associated with a specific interval on the real line. Then, the perceptually quantized coefficients Q, from a known viewing distance d, are calculated as follows: n Q = s=1 o=v,h,dgl sign(ω s,o ) α(ν,r) ω s,o + c n (10) s n Unlike the classical techniques of Visual Frequency Weighting (VFW) on JPEG2000, which apply one CSF weight per sub-band (Boliek et al., 2000, Annex J.8), Perceptual Quantization using CIWaM (ρsq) applies one CSF weight per coefficient over all wavelet planes ω s,o. Equation 10 introduces the perceptual criteria of Equation 2 (Perceptual Images) to each quantized coefficient of Equation 8(Dead-zone Scalar Quantizer). A normalized quantization step size = 1/128 is used, namely the range between the minimal and maximal values at I ρ is divided into 128 intervals. Finally, the resultant indexes or perceptually quantized coefficients are entropy coded, before forming the output code stream or bitstream.

7 3.2.2 Inverse Perceptual Quantization The proposed Perceptual Quantization is a generalized method, which can be applied to wavelet-transformbased image compression algorithms such as EZW, SPIHT, SPECK or JPEG2000. In this chpater, both forward (F-ρSQ) and inverse perceptual quantization (I-ρSQ) into the Hi-SET coder are introduced. An advantage of introducing ρsq is to maintain the embedded features not only of Hi-SET algorithm but also of any wavelet-based image coder. Thus, Perceptual Quantization + Hi-SET = PHi-SET or Φ SET is proposed. Both JPEG2000 and Φ SET choose their VFWs according to a final viewing condition. When JPEG2000 modifies the quantization step size with a certain visual weight, it needs to explicitly specify the quantizer, which is not very suitable for embedded coding. While Φ SET needs neither to store the visual weights nor to necessarily specify a quantizer in order to keep its embedded coding properties. The main challenge underlies in to recover not only a good approximation of coefficients Q but also the visual weight α(ν,r)(eq. 10) that weighted them. A recovered approximation Q with a certain distortion Λ is decoded from the bitstream by the entropy decoding process. The VFWs were not encoded during the entropy encoding process, since it would increase the amount of stored data. A possible solution is to embed these weights α(ν,r) into Q. Thus, the goal is to recover the α(ν,r) weights only using the information from the bitstream, namely from the Forward quantized coefficients Q. The reduction of the dynamic range is uniformly made by the perceptual quantizer, thus the statistical properties of I are maintained in Q. Therefore, the approximation α(ν,r) of α(ν,r) can be recovered applying CIWaM to Q, with the same viewing conditions used in I. That is, α(ν,r) is the recovered e-csf. Thus, the inverse perceptual quantizer or the recovered α(ν,r) introduces perceptual criteria to 9 (Inverse Scalar Quantizer) and it is given by: Î = n sign( ω s,o ) s ( ω s,o +δ) + (ĉ n +δ) n s=1 o=v,h,dgl α(ν,r) ω s,o > 0 0, ω s,o = 0 (11) Figure 4(a) depicts the PSNR difference (db) of each color image of the CMU Image Database, that is, the gain in db of image quality after applying α(ν,r) at d = 2000 centimeters to the Q images. On average, this gain is about 15 db(signal Image Processing Institute, 1997). Then, the same experiment for color images is performed with d = 20, 40, 60, 80, 100, 200, 400, 800, 1000 and 2000 centimeters, in addition to test their objective image quality by means of the PSNR. Figure 4(b), green function denoted as F-ρSQ is the quality metrics of perceptual quantized images after applying α(ν,r), while blue function denoted as I-ρSQ is the quality metrics of recovered images after applying α(ν,r). Thus, for color images, PSNR estimation of the quantized image Q decrease regarding d, the longer d the greater the image quality decline. When the image decoder recovers Q and it is perceptually inverse quantized, the quality barely varies and it can be considered as a perceptually lossless image coding, no matter the distance.

8 (a) PSNR differences. (b) Viewing Distance vs PSNR of recovered color images. Figure 4: (a) PSNR difference between Q image after applying α(ν,r) and recovered Î after applying α(ν,r) for every color image of the CMU database. (b) PSNR assessments of compression of Color Images of the CMU image database. Green function denoted as F-ρSQ is the quality metrics of forward perceptual quantized images after applying α(ν, r), while blue function denoted as I-ρSQ is the quality metrics of recovered images after applying α(ν,r).

9 4 Image Entropy Encoding: Hi-SET Algorithm 4.1 Startup Considerations Hilbert space-filling Curve The Hilbert curve is an iterated function, which can be represented by a parallel rewriting system, more precisely an L-system. In general, the L-system structure is a tuple of four elements: 1. Alphabet: the variables or symbols to be replaced. 2. Constants: set of symbols that remain fixed. 3. Axiom or initiator: the initial state of the system. 4. Production rules: how variables are replaced. In order to describe the Hilbert curve alphabet let us denote the upper left, lower left, lower right, and upper right quadrants as W, X, Y and Z, respectively, and the variables as U (up, W X Y Z), L (left, W Z Y X ), R (right, Z W X Y), and D (down, X W Z Y). Where indicates a movement from a certain quadrant to another. Each variable represents not only a trajectory followed through the quadrants, but also a set of 4 m transformed pixels in m level. The structure of the proposed Hilbert Curve representation does not need fixed symbols, since it is just a linear indexing of pixels. It is appropriate to say that the original work made by David Hilbert (Hilbert, 1891), proposes an axiom with a D trajectory (Figure 5a), while it is proposed to start with an U trajectory (Figure 5b). This proposal is based on the most of the image energy is concentrated where the higher subbands with lower frequencies are, namely at the upper-left quadrant. The first three levels are portrayed in left-to-right order by Figures 5(a) and 5(b). The production rule set of the Hilbert curves is defined as follows: U is changed by the string LUUR, L by ULLD, R by DRRU and D by RDDL. In this way high order curves are recursively generated replacing each former level curve with the four later level curves. (a) Axiom = D (b) Axiom = U Figure 5: First three levels of a Hilbert Curve. (a) Axiom = D proposed by David Hilbert (Hilbert, 1891). (b) Axiom = U employed in this chapter. The Hilbert Curve process remains in an area as long as possible before moving to the neighboring region. Hence, the correlation between pixels is maximized, which is an important image compression issue. Since the higher correlation at the preprocessing, the more efficient the data compression.

10 4.1.2 Linear Indexing A linear indexing is developed in order to store the coefficient matrix into a vector. Let us define the Wavelet Transform coefficient matrix as H and the interleaved resultant vector as H, being 2 γ 2 γ be the size of H and 4 γ the size of H, where γ is the Hilbert curve level. Algorithm 1 generates a Hilbert mapping matrix θ with level γ, expressing each curve as four consecutive indexes. The level γ of θ is acquired concatenating four different θ transformations in the previous level γ 1. Algorithm 1 generates the Hilbert mapping matrix θ, where β refers a 180 degree rotation of β and β T is the linear algebraic transpose of β. Figure 6(b) shows an example of the mapping matrix θ at level γ = 3. Thus, each wavelet coefficient at H (i, j) is stored and ordered at H θ(i, j), being θ (i, j) the location index of it into H. Algorithm 1: Function to generate Hilbert mapping matrix θ of size 2 γ 2 γ. Input: γ Output: θ 1 if γ = 1 then 2 θ = [ ] 3 else 4 β = Algorithm 1 (γ 1) 5 θ = [ β T ( β) T + (3 4 γ 1 ) β + 4 γ 1 β + (2 4 γ 1 ) ] Significance Test A significance test is defined as the trial of whether a coefficient subset achieves the predetermined significance level or threshold in order to be a significant or insignificant. It defines how these subsets are formed and what are the coefficients considered significant. With the aim of recovering the original image at different qualities and compression ratios, it is not needed to sort and store all the coefficients H but just a subset of them: the subset of significant coefficients. Those coefficients H i such that 2 thr H i are called significant otherwise they are called insignificant. The smaller the thr, the better the final image quality and the lower the compression ratio. Let us define a bit-plane as the subset of coefficients S o such that 2 thr S o < 2 thr+1. The significance of a given subset S o amongst a particular bit-plane is store at Ĥsig and can be defined as: Ĥ sig = { 1, 2thr S o < 2 thr+1 0, otherwise (12) Algorithm 2 shows how a given subset S o is divided into four equal parts (line 6) and how the significance test (lines 7-12) is performed, resulting in four subsets (S 1, S 2, S 3 and S 4 ) with their respective significance stored at the end of Ĥsig. The subsets S 1, S 2, S 3 and S 4 are four 2 1 cell arrays. The fist cell of each array contains one of the four subsets extracted from S o (S i (1))and the second one stores its respective significance test result(s i (2)).

11 Algorithm 2: Subset Significance Test. Data: S o, thr Result: S 1, S 2, S 3, S 4 and Ĥsig 1 γ= log 4 (length o f S o) 2 The cell 1 of the subsets S 1, S 2, S 3 and S 4 is declared with 4 γ 1 elements, while the cell 2 with just one element. 3 i = 1 4 Ĥsig is emptied. 5 for j=1 to 4 γ do 6 Store S o [ f rom j to (i 4 γ 1 )] into S i (1). 7 if 2 thr max S i (1) < 2 thr+1 then 8 S i (2) = 1 9 Add 1 at the end of the Ĥsig. 10 else 11 S i (2) = 0 12 Add 0 at the end of the Ĥsig. 13 i and j are incremented by 1 and 4 γ 1, respectively. 4.2 Coding Algorithm Similarly to SPIHT and SPECK (Pearlman & Said, 2008a; Pearlman & Said, 2008b), Hi-SET considers three coding passes: Initialization, Sorting and Refinement, which are described in the following subsections. SPIHT uses three ordered lists, namely the list of insignificant pixels (LIP), the list of significant sets (LIS) and the list of significant pixels (LSP). The latter list represents just the individual coefficients, which are considered the most important ones. SPECK employs two of these lists, the LIS and the LSP. Whereas Hi-SET makes use of only one ordered list, the LSP. Using a single LSP place extra load on the memory requirements for the coder, since the total number of significant pixels remains the same even if the coding process is working in insignificant branches. That is why Hi-SET employs spare lists, storing significant pixels in several sub-lists. This smaller lists have the same length than significant coefficients founded in the processed branch. With the purpose of expediting the coding process it is used not only spare lists, but also spare cell arrays, both are denoted by an apostrophe, LSP, Ĥ or S 1, for instance Initialization Pass The first step is to define threshold thr as thr = log 2 (max{ H}), (13) that is, thr is the maximum integer power of two not exceeding the maximum value found at H. The second step is to apply Algorithm 2 with thr and H as input data, which divides H into four subsets of 4 γ 1 coefficients and adds their significance bits at the end of Ĥ.

12 4.2.2 Sorting Pass Algorithm 3 shows a simplified version of the classification or sorting step of the Hi-SET Coder. The Hi-SET sorting pass exploits the recursion of fractals. If a quadtree branch is significant it moves forward until finding an individual pixel, otherwise the algorithm stops and codes the entire branch as insignificant. Algorithm 3: Sorting Pass Data: S 1, S 2, S 3, S 4, thr and γ Result: LSP and Ĥ 1 LSP and Ĥ are emptied. 2 if γ = 0 then 3 for i = 4 to 1 do 4 if S i (2) is significant then 5 Add S i (1) at the beginning of the LSP. 6 if S i (1) is positive then 7 Add 0 at the beginning of the Ĥ. 8 else 9 Add 1 at the beginning of the Ĥ. 10 else 11 for i=1 to 4 do 12 if S i (2) is significant then 13 Call Algorithm 2 with S i (1) and thr as input data and Store the results into S 1, S 2, S 3, S 4 and Ĥ. 14 Add Ĥ at the end of the Ĥ. 15 Call Algorithm 3 with S 1, S 2, S 3, S 4, thr and γ 1 as input data and Store the results into Ĥ and LSP. 16 Add Ĥ at the end of the Ĥ. 17 Add LSP at the end of the LSP. Algorithm 3 is divided into two parts: Sign Coding (lines 2 to 9) and Branch Significance Coding (lines 11 to 16). The algorithm performs the Sign Coding by decomposing a given quadtree branch up to level γ = 0, i.e. the branch is represented by only 4 coefficients with at least one of them being significant. The initial value of γ is log 4 (length o f H) 1. Only the sign of the significant coefficients is coded, 0 for positives and 1 for negatives. Also, each significant coefficient is added into a spare LSP or LSP. The Branch Significance Coding calls the Algorithm 2 in order to quarter a branch in addition to call recursively an entire sorting pass at level γ 1 up to reach the elemental level when γ = 0. The Significance Test results of a current branch (obtained by the Algorithm 2) and the ones of next branches (acquired by Algorithm 3, denoted as Ĥ ) are added at the end of Ĥ. Also, all the significant coefficients found in previous branches (all the lists LSP ) are added at the end of the LSP. This process is repeated for all four subsets of H Refinement Pass At the end of Ĥ, the (thr 1)-th most significant bit of each ordered entry of the LSP, including those entries added in the last sorting pass, are added. Then, thr is decremented and another Sorting Pass is performed. The Sorting and Refinement steps are repeated up to thr = 1.

13 The decoder employs the same mechanism as the encoder, since it knows the fractal applied to the original image. When the bitstream Ĥ is received, by itself describes the significance of every variable of the fractal. Then with these bits, the decoder is able to reconstruct both partially and completely, the same fractal structure of the original image, refining the pixels progressively as the algorithm proceeds. 4.3 A Simple Example In order to highlight the operations employed by Hi-SET, a simple example is developed. The considered wavelet transform coefficient matrix H is depicted in Figure 6(a), which is a three scale transformation of an 8 8 image, namely γ = 3. The indexed vector H (Fig. 6(c)) is acquired interleaving H with a three-level Matrix θ (Fig. 6(b)). Table 1 shows the entire process until to code all coefficients at the first bit-plane. The eleven steps in Table 1 represent the three passes of the scheme. Initialization Pass is described by steps 1 and 2, Sorting Pass by 3 to 10 steps, while the last one illustrates Refinement Pass. 5 Φ SET Codestream Syntax The Φ SET Codestream Syntax is a representation of compressed image data, that contains all parameters used in the encoding process and is a linear stream of bits. This bit-stream is mainly divided into two consecutive groups: Headers and the Ĥ obtained in the coding process (Figure 7). Headers are subdivided in groups of Markers. So, these Headers are considered in two types: Mandatory and Complemental Headers. Figure 8(a) shows the structure of the Mandatory Header, that is a 16 bit fixed sized substream. This Header is fractionated in six Markers, namely Image size, thr max, w lev, Channels, w f ilter and Q step, which are described as follows: Image size (4 bits). If this marker is different to zero means that the processed image is squared with height and width equal to 2 Image size+1. Thus the overall size of a square image varies from 4 2 to 4 16 pixels. Otherwise when Image size = 0000 the markers Image height and Image width of the Complemental Header are used for establishing the image size. thr max (4 bits). It stores the maximum threshold thr 1 defined in the Equation 13, thus its value varies from 1 to 16. Hence Φ SET can process an image up to 16 bit-planes. w lev (3 bits). This marker contains the number of spatial frequencies minus one performed by the Wavelet Transform, thus its value varies from 1 to 8 Wavelet spatial frequencies. Channels (3 bits). The number of image color components minus one is stored in this marker, thus managing up to eight components. w filter (1 bit). If it is one a 9/7 Wavelet filter is used, otherwise the employed filter is a 5/3. Q step (1 bit). Quantization step marker indicates whether the coefficients are quantized or not. When they are quantized, the size of these Quantization steps are placed in a marker at the end of the Complemental Header.

14 (a) H (b) θ (c) H Figure 6: Example of Hilbert indexing of an 8 8 image. (a) Three-scale wavelet transform matrix H. (b) Hilbert Indexing matrix θ when γ = 3. (c) Interleaved resultant vector H. Step Former Current Bitstream Decoded Curve Curve(s) Ĥ LSP 1 3U 2 3U 2LUUR L 1ULLD U SIIS sign L SIII sign U 1LUUR R IIIS sign re f Table 1: The First bit-plane encoding using Hi-SET scheme. H, α and H are taken from Figure 6, with initial threshold thr = 5. Figure 7: Φ SET Codestream Syntax.

15 (a) Mandatory Header. (b) Complemental Header. Figure 8: Φ SET Headers with their Markers. Figure 8(b) depicts the Complemental Header, that is formed by three consecutive Marks: two for storing the size of a non-squared image and the other one for recovering the quantization steps. Image height (16 bits). It contains the height of a non-squared image. Thereby a pixel image of height can be supported. Image width (16 bits). It contains the width of a non-squared image. Thereby a pixel image of width can be supported. Qsteps orientation and frequencies ( bits). This marker is a collection of several sub-markers. Φ SET may retrieve a quantization step o s by each spatial frequency (indexed by s) and spatial orientation (indexed by o) into a wavelet plane ω o s, in addition to another one for the residual plane c wlev +1. Since the Codestream of Φ SET supports up to w lev + 1 spatial frequencies and three spatial orientations, there are 3 w lev +4 quantization steps. Each quantization step is represented by a two-byte-long sub-marker, which is divided in three parts: Sign, Exponent ε o s and Mantissa µ o s (Figure??). The most significant bit of the sub-marker is the sign of o s, whether 0 for positive or 1 for negative. The ten least significant bits are determined for the allocation of µ o s, which is defined by (Boliek et al., 2000) as: µ o s = 2 10 ( o s 1)+ 1 2 Ro s εo s 2 (14) Equation 15 expresses how ε o s is obtained, which is stored at the 5 remaining bits of the o s sub-marker ε o s = R o s log 2 o s (15) where R o s is the number of bits used to represent the peak coefficient inside ω o s, defined as follows: R o s = log 2 [max{ω o s }] (16) PQ (1 bit). If Q step = 1, PQ would specify if the wavelet coefficients were perceptually quantized or not. Fig.?? shows this marker. d (16 bits). This marker stores the observation distance d, which is represented by a two-byte long sub-marker and divided in two parts: Exponent ε d and Mantissa µ d (Fig.??). The eleven least significant bits are employed for the allocation of µ d, which is defined as: µ d = 2 11 ( d 2 R dmax ε d 1)+ 1 2 (17)

16 Equation (18) expresses how ε d is obtained, which is stored at the 5 remaining bits of the d marker ε d = R dmax log 2 (d) (18) where R dmax is the number of bits used to represent the peak permitted observation distance d < 2048H, being H the height of a pixel image presented in an M size LCD monitor with horizontal resolution of h res pixels and v res pixels of vertical resolution. Therefore, R dmax = bit 5 bits 10 bits Sign ε 0 s µ 0 s MSB (a) o s Sub-marker 1 bit LSB PQ (b) PQ Marker 5 bits 11 bits ε d µ d MSB (c) d Marker LSB Figure 9: (a) Structure of the o s Sub-marker, (b) Perceptual Quantization Marker and (b) Structure of Observation Distance Marker 6 Efficiency of Φ SET Coder comparing to JPEG2000 Standard For the sake of comparing the performance between the JPEG2000(Taubman & Marcellin, 2002) andφ SET coders, first a Φ SET compression with certain viewing conditions is performed, which gives a compressed image with a particular bit-rate (bits per pixel, bpp). Then, a JPEG2000 compression is performed with the same bit-rate. Once both algorithms recover their distorted images, they are compared with some numerical image quality estimators such as: MSSIM(Wang et al., 2003), PSNR(Huynh-Thu & Ghanbari, 2008), VIF(Wang et al., 2004) and WSNR(Mitsa & Varkur, 1993).

17 (a) MSSIM (b) PSNR (c) VIF (d) WSNR Figure 10: Comparison between Φ SET (green functions) and JPEG2000 (blue functions) image coders. Compression rate vs image quality assessed by (a) MSSIM, (b) PSNR, (c) VIF and (d) WSNR the CMU image database. This experiment is performed across the CMU Image Database. Image quality estimations are assessed by the four metrics before mentioned. Figures 10 show the perceptual quality, estimated by 10(a) MSSIM, 10(c)VIF and 10(d) WSNR, in addition to the objective quality 10(b) PSNR, of the recovered color images both for JPEG2000(Blue function) and Φ SET (Green function) as a function of their compression rate. For this experiment, the CMU Image Database and the Kakadu implementation for JPEG2000 compression(taubman, 2010) are employed. On the average, a color image compressed at 1.0 bpp (1:24 ratio, stored in 32 KBytes) by JPEG2000 coder has MSSIM=0.9424, VIF= and WSNR=29.2 of perceptual image quality, and PSNR=30.11 of objective image quality, while by Φ SET has MSSIM=0.9780, VIF=0.4387, WSNR=35.41 and PSNR= Figure 11 depicts these differences when images (a-b)lenna and (c-d)tiffany are compressed at bpp by JPEG2000 and Φ SET. For example for these two images, the average difference of MSSIM is in favor of Φ SET, namely 4% better. Therefore, for this image database, Φ SET has clearly improvement of visual quality than JPEG Conclusions The main goal of this chapter is to introduce perceptual criteria on the image compression process. These perceptual criteria are used to identify and to remove non-perceptual information of an image. These aspects are used to propose a perceptual image compression system. Additionally, the coder based on Hilbert

18 (a) JPEG2000, MSSIM= (b) ΦSET, MSSIM= (c) JPEG2000, MSSIM= (d) ΦSET, MSSIM= Figure 11: Example of recovered color images Lenna and Tiffany of the CMU image database compressed, both at bpp. Scanning (Hi-SET) is also presented. The Hi-SET coder, presented in Section 4, is based on Hilbert scanning of embedded quadtrees. It has low computational complexity and some important properties of modern image coders such as embedding and progressive transmission. This is achieved by using the principle of partial sorting by magnitude when a sequence of thresholds decreases. The desired compression rate can be controlled just by chunking the stream at the desired file length. When compared to other algorithms that use Hilbert scanning for pixel ordering, Hi-SET improves image quality by around 6.20 db. Hi-SET achieves higher compression rates than JPEG2000 coder not only for high and medium resolution images but also for low resolution ones where it is difficult to find redundancies among spatial frequencies. Table 2 summarizes the average improvements when compressing the TID2008 Image Database(Ponomarenko et al., 2009).

19 Components Y YC b C r Resolution Low Medium Low Medium Compression Ratio (bpp) Image Quality (db) Table 2: Average PSNR(dB) improvement of Hi-SET in front of JPEG2000 for TID2008 image database. The Hi-SET coder improves the image quality of the JPEG2000 coder around PSNR=1.16 db for gray-scale images and 1.43 db for color ones. Furthermore, it saves around bpp for high resolution gray-scale images. Thus, the results across the CMU image database resulted Hi-SET improved the results of JPEG2000 not only objectively but also by metrics like MSSIM, VIF or WPSNR, which are perceptual indicators. This is why in Section 3 Forward (F-ρSQ) and Inverse (I-ρSQ) perceptual quantizer using CIWaM are defined. When F-ρSQ and I-ρSQ are incorporated to Hi-SET, a perceptual image compression system Φ SET is proposed. In order to measure the effectiveness of the perceptual quantization, a performance analysis is done using the most important image quality assessments such as PSNR, MSSIM, VIF or WSNR. In addition, when both Forward and Inverse Perceptual Quantization are applied into Hi-SET, namely using Φ SET, it significatively improves the results regarding the JPEG2000 compression. Acknowledgement This work is supported by The National Polytechnic Institute of Mexico by means of a granted fund by the Committee of Operation and Promotion of Academic Activities (COFAA). References Boliek, M., Christopoulos, C., & Majani, E. (2000). Information Technology: JPEG2000 Image Coding System. ISO/IEC JTC1/SC29 WG1, JPEG 2000, JPEG 2000 Part I final committee draft version 1.0 edition. Hilbert, D. (1891). Über die stetige Abbildung einer Linie auf ein Flächenstück. Mathematische Annalen, 38(3), Huynh-Thu, Q. & Ghanbari, M. (2008). Scope of validity of PSNR in image/video quality assessment. Electronics Letters, 44(13), Marcellin, M. W., Lepley, M. A., Bilgin, A., Flohr, T. J., Chinen, T. T., & Kasner, J. H. (2002). An overview of quantizartion of JPEG2000. Signal Processing: Image Communication, 17(1), Mitsa, T. & Varkur, K. (1993). Evaluation of contrast sensitivity functions for formulation of quality measures incorporated in halftoning algorithms. IEEE International Conference on Acustics, Speech and Signal Processing, 5, Moreno, J. & Otazu, X. (2011a). Image coder based on Hilbert Scaning of Embedded quadtrees. IEEE Data Compression Conference, (pp. 470).

20 Moreno, J. & Otazu, X. (2011b). Image coder based on Hilbert Scaning of Embedded quadtrees: An introduction of Hi-SET coder. IEEE International Conference on Multimedia and Expo. Mullen, K. (1985a). The contrast sensitivity of human color vision to red-green and blue-yellow chromatic gratings. Journal of Physiology, 359, Mullen, K. T. (1985b). The contrast sensitivity of human colour vision to red-green and blue-yellow chromatic gratings. The Journal of Physiology, 359, Murray, N., Vanrell, M., Otazu, X., & Parraga, A. (2010). Saliency estimation using a non-parametric low-level vision model. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2011) (pp ). Otazu, X., Párraga, C., & Vanrell, M. (2010). Toward a unified chromatic induction model. Journal of Vision, 10(12)(6). Pearlman, W., Islam, A., Nagaraj, N., & Said, A. (2004). Efficient, low-complexity image coding with a Set-Partitioning Embedded block coder. IEEE Transactions on Circuits and Systems for Video Technology, 14(11), Pearlman, W. A. & Said, A. (2008a). Image wavelet coding systems: Part II of set partition coding and image wavelet coding systems. Foundations and Trends in Signal Processing, 2(3), Pearlman, W. A. & Said, A. (2008b). Set partition coding: Part I of set partition coding and image wavelet coding systems. Foundations and Trends in Signal Processing, 2(2), Ponomarenko, N., Lukin, V., Zelensky, A., Egiazarian, K., Carli, M., & Battisti, F. (2009). TID a database for evaluation of full-reference visual quality assessment metrics. Advances of Modern Radioelectronics, 10, Said, A. & Pearlman, W. (1996). A new, fast, and efficient image codec based on Set Partitioning In Hierarchical Trees. IEEE Transactions on Circuits and Systems for Video Technology, 6(3), Shapiro, J. (1993). Embedded image coding using Zerotrees of wavelet coefficients. IEEE Transactions on Acoustics, Speech, and Signal Processing, 41(12), Signal Image Processing Institute, U. o. S. C. (1997). The USC-SIPI image database. Taubman, D. (2010). Kakadu software. Taubman, D. S. & Marcellin, M. W. (2002). JPEG2000: Image Compression Fundamentals, Standards and Practice. ISBN: X. Kluwer Academic Publishers. Usevitch, B. E. (2001). A tutorial on modern lossy wavelet image compression: foundations of JPEG IEEE Signal Processing Magazine, 18(5), Wang, Z., Bovik, A., Sheikh, H., & Simoncelli, E. (2004). Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), Wang, Z., Simoncelli, E., & Bovik, A. (2003). Multiscale structural similarity for image quality assessment. In Conference Record of the Thirty-Seventh Asilomar Conference on Signals, Systems and Computers., volume 2 (pp Vol.2).