JPEG and JPEG2000 Image Coding Standards

Similar documents
<Outline> JPEG 2000 Standard - Overview. Modes of current JPEG. JPEG Part I. JPEG 2000 Standard

Wavelet Scalable Video Codec Part 1: image compression by JPEG2000

Objective: Reduction of data redundancy. Coding redundancy Interpixel redundancy Psychovisual redundancy Fall LIST 2

encoding without prediction) (Server) Quantization: Initial Data 0, 1, 2, Quantized Data 0, 1, 2, 3, 4, 8, 16, 32, 64, 128, 256

Image Compression - JPEG

EBCOT coding passes explained on a detailed example

L. Yaroslavsky. Fundamentals of Digital Image Processing. Course

Multimedia Networking ECE 599

SIGNAL COMPRESSION. 8. Lossy image compression: Principle of embedding

+ (50% contribution by each member)

Image Compression. Fundamentals: Coding redundancy. The gray level histogram of an image can reveal a great deal of information about the image

Basic Principles of Video Coding

JPEG2000 High-Speed SNR Progressive Decoding Scheme

Image Data Compression

- An Image Coding Algorithm

IMAGE COMPRESSION-II. Week IX. 03/6/2003 Image Compression-II 1

Embedded Zerotree Wavelet (EZW)

Module 4. Multi-Resolution Analysis. Version 2 ECE IIT, Kharagpur

RESOLUTION SCALABLE AND RANDOM ACCESS DECODABLE IMAGE CODING WITH LOW TIME COMPLEXITY

EE67I Multimedia Communication Systems

State of the art Image Compression Techniques

Introduction p. 1 Compression Techniques p. 3 Lossless Compression p. 4 Lossy Compression p. 5 Measures of Performance p. 5 Modeling and Coding p.

Multimedia & Computer Visualization. Exercise #5. JPEG compression

Fast Progressive Wavelet Coding

Fault Tolerance Technique in Huffman Coding applies to Baseline JPEG

Implementation of CCSDS Recommended Standard for Image DC Compression

Lecture 2: Introduction to Audio, Video & Image Coding Techniques (I) -- Fundaments

Module 5 EMBEDDED WAVELET CODING. Version 2 ECE IIT, Kharagpur

ECE533 Digital Image Processing. Embedded Zerotree Wavelet Image Codec

SYDE 575: Introduction to Image Processing. Image Compression Part 2: Variable-rate compression

Can the sample being transmitted be used to refine its own PDF estimate?

CSE 408 Multimedia Information System Yezhou Yang

Lecture 2: Introduction to Audio, Video & Image Coding Techniques (I) -- Fundaments. Tutorial 1. Acknowledgement and References for lectures 1 to 5

Compression. Encryption. Decryption. Decompression. Presentation of Information to client site

Inverse Problems in Image Processing

ECE 634: Digital Video Systems Wavelets: 2/21/17

Overview. Analog capturing device (camera, microphone) PCM encoded or raw signal ( wav, bmp, ) A/D CONVERTER. Compressed bit stream (mp3, jpg, )

Image and Multidimensional Signal Processing

Compression. What. Why. Reduce the amount of information (bits) needed to represent image Video: 720 x 480 res, 30 fps, color

Waveform-Based Coding: Outline

Progressive Wavelet Coding of Images

Lecture 7 Predictive Coding & Quantization

Introduction to Video Compression H.261

Digital communication system. Shannon s separation principle

IMAGE COMPRESSION IMAGE COMPRESSION-II. Coding Redundancy (contd.) Data Redundancy. Predictive coding. General Model

Image Compression Basis Sebastiano Battiato, Ph.D.

On Compression Encrypted Data part 2. Prof. Ja-Ling Wu The Graduate Institute of Networking and Multimedia National Taiwan University

Product Obsolete/Under Obsolescence. Quantization. Author: Latha Pillai

Information and Entropy

Lec 05 Arithmetic Coding

Introduction to Wavelet. Based on A. Mukherjee s lecture notes

BASICS OF COMPRESSION THEORY

Run-length & Entropy Coding. Redundancy Removal. Sampling. Quantization. Perform inverse operations at the receiver EEE

JPEG Standard Uniform Quantization Error Modeling with Applications to Sequential and Progressive Operation Modes

CHAPTER 3. Transformed Vector Quantization with Orthogonal Polynomials Introduction Vector quantization

4. Quantization and Data Compression. ECE 302 Spring 2012 Purdue University, School of ECE Prof. Ilya Pollak

Basics of DCT, Quantization and Entropy Coding

Compression and Coding

ECE472/572 - Lecture 11. Roadmap. Roadmap. Image Compression Fundamentals and Lossless Compression Techniques 11/03/11.

Vector Quantization and Subband Coding

Real-Time Audio and Video

6. H.261 Video Coding Standard

AN IMPROVED CONTEXT ADAPTIVE BINARY ARITHMETIC CODER FOR THE H.264/AVC STANDARD

Multimedia Information Systems

Compression and Coding. Theory and Applications Part 1: Fundamentals

A New Pipelined Architecture for JPEG2000 MQ-Coder

Basics of DCT, Quantization and Entropy Coding. Nimrod Peleg Update: Dec. 2005

Department of Electrical Engineering, Polytechnic University, Brooklyn Fall 05 EL DIGITAL IMAGE PROCESSING (I) Final Exam 1/5/06, 1PM-4PM

EMBEDDED ZEROTREE WAVELET COMPRESSION

ii Abstract Compression of digital images has been a topic of research for many years and a number of image compression standards has been created for

CSE 126 Multimedia Systems Midterm Exam (Form A)

Half-Pel Accurate Motion-Compensated Orthogonal Video Transforms

Rounding Transform. and Its Application for Lossless Pyramid Structured Coding ABSTRACT

Entropy Encoding Using Karhunen-Loève Transform

2018/5/3. YU Xiangyu

Image Compression. Qiaoyong Zhong. November 19, CAS-MPG Partner Institute for Computational Biology (PICB)

Title. Author(s)Lee, Kenneth K. C.; Chan, Y. K. Issue Date Doc URL. Type. Note. File Information

COSC460 Honours Report. A Fast Discrete Tchebichef Transform Algorithm for Image Compression

Enhanced Stochastic Bit Reshuffling for Fine Granular Scalable Video Coding

Image compression. Institute of Engineering & Technology, Ahmedabad University. October 20, 2015

Lifting Parameterisation of the 9/7 Wavelet Filter Bank and its Application in Lossless Image Compression

A NEW BASIS SELECTION PARADIGM FOR WAVELET PACKET IMAGE CODING

Lec 04 Variable Length Coding (VLC) in JPEG

Embedded Lossless Wavelet-Based Image Coder Based on Successive Partition and Hybrid Bit Scanning

BASIC COMPRESSION TECHNIQUES

repetition, part ii Ole-Johan Skrede INF Digital Image Processing

Module 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

LATTICE VECTOR QUANTIZATION FOR IMAGE CODING USING EXPANSION OF CODEBOOK

Study of Wavelet Functions of Discrete Wavelet Transformation in Image Watermarking

Wavelets & Mul,resolu,on Analysis

Detailed Review of H.264/AVC

Lossless Image and Intra-frame Compression with Integer-to-Integer DST

Compressing a 1D Discrete Signal

Reduce the amount of data required to represent a given quantity of information Data vs information R = 1 1 C

UNIT I INFORMATION THEORY. I k log 2

Compressing a 1D Discrete Signal

Efficient Alphabet Partitioning Algorithms for Low-complexity Entropy Coding

Compression and Coding. Theory and Applications Part 1: Fundamentals

Number Representation and Waveform Quantization

Vector Quantizers for Reduced Bit-Rate Coding of Correlated Sources

Transcription:

JPEG and JPEG2000 Image Coding Standards Yu Hen Hu

Outline Transform-based Image and Video Coding Linear Transformation DCT Quantization Scalar Quantization Vector Quantization Entropy Coding Discrete wavelet transform Perfect reconstruction Lifting formulation Context based bit-plane coding JPEG2000 (C) 2002-2013 by Yu Hen Hu 2

Image Coding Preliminaries A concise representation of an image so that it can be transmitted, stored more efficiently without sacrificing viewing quality. Exploiting Spatial correlations among neighboring pixels Limited sensitivity of human psycho-visual system (lossy coding) Converting original image into a symbol stream using transformed coding and context coding The symbol stream is converted into a binary stream using entropy coding. (C) 2002-2013 by Yu Hen Hu 3

linear transform Linear Transform Transformation is accomplished via weighted linear combination of input data Can be formulated as matrix-matrix multiplication Examples: K-L Expansion, Discrete Fourier Transform (DFT), Discrete cosine transform (DCT), Discrete wavelet transform (DWT) Fast and parallel algorithm formulations of linear transform have been well studied. Examples include fast Fourier transform (FFT) Energy compaction property: After transformation, energy of the original signal concentrates on few, large transformed coefficients Allow efficient exploitation of the spatial correlation of image data By eliminating small transformed coefficients, lossy compression of image data can be accomplished. (C) 2002-2013 by Yu Hen Hu 4

Block-based Image Coding An image is a 2D signal of pixel intensities (including colors). A block-based image coding scheme partitions the entire image into 8 by 8 or 16 by 16 (or other size) blocks. Then the coding algorithm is applied to individual blocks independently. Blocks may be overlapping or non-overlapping. Advantage: parallel processing can be applied to process individual blocks in parallel. For hand-held devices, only one block needs be loaded into main memory each time. (C) 2002-2013 by Yu Hen Hu 5

Transform-based Image Coding Input Image Linear Transform Quantization Entropy Coding Binary bit stream (C) 2002-2013 by Yu Hen Hu 6

JPEG Encoding f(x, y) F(u, v) DCT Quantization F q (u, v) 8 8 Quantization table Header Tables Coding tables DC EC DPCM DC Zig Zag Scan Data AC EC RLC AC (C) 2002-2013 by Yu Hen Hu 7

Pre-Processing Color sub-sampling A color image is converted from RGB to YCbCr color space. Each pixel in each dimension is 1 byte. Sub-sample Cb-Cr planes: 4:1:1 scheme. For every 16 by 16 block of a color image, six 8 by 8 blocks are encoded. Level shifting: Each pixel value is subtracted by 128 so it ranges ( 128, 127). Four 8 8 blocks of luminance pixels, plus two 8 8 sub-sampled chrominance components makes a 16 by 16 macro-block (C) 2002-2013 by Yu Hen Hu 8

YCbCr Color Conversion YCbCr or Y CbCr is a family of color spaces used as a part of the color image pipeline in video and digital photography systems. Y is the luma component and CB and CR are the blue-difference and red-difference chroma components. Y (with prime) is distinguished from Y which is luminance, meaning that light intensity is non-linearly encoded using gamma. R Y CR Y KR R KG G KB B K K C B Y G Y C C C R Y K K K 1 B R B B R KG KG R B Y CB R G B ITU-601 ITU-709 SMPTE K R.299.2126.212 K B.114.0722.087 http://discoverybiz.net/enu0/faq/faq_yuv_ycbcr_ypbpr.htm (C) 2002-2013 by Yu Hen Hu http://en.wikipedia.org/wiki/ycbcr 9

Discrete Cosine Transform 8 8 two-dimensional separable DCT: 7 7 1 (2m 1) u (2n 1) v F( u, v) C u C v f ( m, n) cos cos 4 16 16 C u 1 2 u 0; 1 u 0. m 0 m 0 DCT is chosen because it leads to superior energy compaction for natural images. F(0,0): DC coefficient ranges (-128x64/8,127x8) needs 11 bits to represent (including sign bit). 11 bits are more than enough for the remaining AC coefficients (u > 0, or v > 0) (C) 2002-2013 by Yu Hen Hu 10

Inverse DCT (IDCT) 8 8 two-dimensional separable IDCT: 7 7 1 (2u 1) m (2v 1) n f ( m, n) C u C v F( u, v) cos cos 4 16 16 u 0 v 0 1 2 u 0; C u 1 u 0. IDCT can be computed using the same routine as DCT (C) 2002-2013 by Yu Hen Hu 11

Quantization of DCT Coefficients (C) 2002-2013 by Yu Hen Hu 12

DPCM of DC coefficients DC coding: All DC coefficients of each 8 by 8 blocks of the entire image are combined to make a sequence of DC coefficients. Next, DPCM is applied: DiffDC(block i ) = DC(block i ) DC(block i 1 ) Then DiffDCs will be encoded using Hoffman entropy Example: Original: 1216 1232 1224 1248 1248 1208 After DPCM: 1216 1216 +16 8 +24 0 40 1232 1224 1248 1248 1208 (C) 2002-2013 by Yu Hen Hu 13

Huffman Encoding of DC Coefficients Encoding and decoding of Huffman code is done via look-up table. In JPEG, DC coefficients (after DPCM) are first grouped according to their magnitudes. Each category is assigned as a symbol and a Hoffman table is given. For example, 7 to 4 and 4 to 7 are listed as category 3 which has a code "00. If the number is positive, the binary representation of the number will be append to the Hoffman code of the category number directly. For example, 6 is encoded as 00 110. If the number is negative, the appended code is the 1 s complement of that number. For example, -5 is encoded as 00 010. Question: Given such a table, how to devise a dedicated hardware to implement the encoding procedure? (C) 2002-2013 by Yu Hen Hu 14

JPEG Huffman Table: Categories (C) 2002-2013 by Yu Hen Hu 15

JPEG DC Entropy Coding Example: -9: category 4. Hence Base code = 101 1 s complement of (-9) = 1C(1001) = 0110 Code word = 101 + 0110 = 1010110 Note that category 3 occurs most frequent and hence has shortest base code word. (C) 2002-2013 by Yu Hen Hu 16

AC Coefficients AC coefficients are first weighted with a quantization matrix: C(i,j)/q(i,j) = C q (i,j) Then quantized. Then they are scanned in a zigzag order into a 1D sequence to be subject to AC Huffman encoding. Question: Given a 8 by 8 array, how to convert it into a vector according to the zig-zag scan order? What is the algorithm? 1 2 6 7 15 16 28 29 3 5 8 14 17 27 30 43 4 9 13 18 26 31 42 44 10 12 19 25 32 41 45 54 11 20 24 33 40 46 53 55 21 23 34 39 47 52 56 61 22 35 38 48 51 57 60 62 36 37 49 50 58 59 63 64 Zig-Zag scan order (C) 2002-2013 by Yu Hen Hu 17

AC Coefficients Huffman Encoding The symbols for encoding AC coefficient consists both the number of significant bits, as well as runs of 0s preceding the nonzero AC coefficient. For example, 5 0 2 0 0 1 is encoded as: 100101 11100110 110110 This is according to the table below: Number Run/Category Base code Length Final code 5 0/3 100 6 100 101 02 1/2 111001 8 111001 10 00-1 2/1 11011 6 11011 0 (C) 2002-2013 by Yu Hen Hu 18

JPEG2000: An Introduction Yu Hen Hu University of Wisconsin - Madison Dept. Elec. & Comp. Engr. Madison, WI 53706 hu@engr.wisc.edu

Agenda Overview Transformed Image Coding: A Review JPEG2000 Wavelet transform EBCOT - JPEG2000 coefficient modeling and context encoding MQ arithmetic coding ROI: Region of Interests Enhancement, extension and related issues. (C) 2002-2013 by Yu Hen Hu 20

Overview (C) 2002-2013 by Yu Hen Hu 21

Introduction Joint Photographic Experts Group (JPEG) is an ISO standard committee with a mission on Coding and compression of still images. JPEG coding standard (1988): DCT (discrete cosine transform) based transform coding to compress bit-map images. JPEG2000 efforts started in 1996 to use new methods such as fractals or wavelets. The target deliver date is year 2000 and hence the name. (C) 2002-2013 by Yu Hen Hu 22

JPEG JPEG (Joint Photographic Experts Group) Digital Compression and Coding of Continuous-tone Still Images Joint ISO and ITU-T Published in 4 Parts: ISO/IEC 10918-1 ITU-T T.81 : Requirements and guidelines ISO/IEC 10918-2 ITU-T T.83 : Compliance testing ISO/IEC 10918-3 ITU-T T.84: Extensions ISO/IEC 10918-4 ITU-T T.86: Registration of JPEG Parameters, Profiles, Tags, Color Spaces, APPn Markers, Compression Types, and Registration Authorities (REGAUT) (C) 2002-2013 by Yu Hen Hu 23

JPEG derived standards JFIF (JPEG File Interchange Format, <xxxxxx.jpg>) JTIP (JPEG Tiled, Pyramid Format) TIFF (Tagged Image File Format) SPIFF (Still Picture Interchange File Format, JPEG Part 3) FlashPix Developed by Hewlett-Packard, Kodak, Microsoft, Live Picture (1996) Transferred to Digital Imaging Group (DIG), an industry consortium (C) 2002-2013 by Yu Hen Hu 24

JPEG2000: Objectives Low bit-rate compression performance Lossless and lossy compression Large images greater then 64K by 64K without tiling Single decompression architecture Transmission in noisy environments Beyond restart intervals Computer generated imagery Compound document text, natural image, and graphics mixed. Random code stream access and processing Open architecture Progressive transmission by pixel accuracy and resolution (C) 2002-2013 by Yu Hen Hu 25

Handling Large Images Partition in both spatial and frequency domain. Spatial Domain Partition: Tile, Frame bit streams of different tiles or frames are not independent. artifact may occur at boundaries Special wavelet transform: Spatially segmented wavelet transform (SSWT) Line based wavelet transform Block: Independent partition in frequency domain (wavelet coefficients) bit streams are independently generated. (C) 2002-2013 by Yu Hen Hu 26

Current Status Part 1 Core coding system Final Committee Draft 12/ 2000. Part 2 Extensions Final committee draft Part 3 Motion JPEG2000 committee draft Part 4 Conformance committee draft Part 5 reference software Final committee draft Part 6 compound image file format Committee draft Official website: http://www.jpeg.org/ Implementations Verification Model (not public) JJ2000 http://jj2000.epfl.ch/ JesPer http://www.imagepower.com / Kakadu http://maestro.ee.unsw.edu. au/taubman/kakadu/ (C) 2002-2013 by Yu Hen Hu 27

JPEG2000 features High compression efficiency Lossless color transformations Lossy and lossless coding in one algorithm Embedded lossy to lossless coding Progressive by resolution and quality Static and dynamic Region-of-Interest Error resilience Visual (fixed and progressive) coding Multiple component images Palletized Images (C) 2002-2013 by Yu Hen Hu 28

JPEG at 0.125 bpp (enlarged) C. Christopoulos, A. Skodras, T. Ebrahimi, JPEG2000 (online tutorial) (C) 2002-2013 by Yu Hen Hu 29

JPEG2000 at 0.125 bpp C. Christopoulos, A. Skodras, T. Ebrahimi, JPEG2000 (online tutorial) (C) 2002-2013 by Yu Hen Hu 30

Bit Plane Coding 3-1 7 0 0 1+ 1+ 0 1 1 1-1 4-5 2 1+ 1-0 0 0 1+ 0 1 0 6 1-2 1+ 0 0 1 0 1-0 1+ 0 MSB LSB Coefficients are represented in sign-magnitude format Bit plane starts from the most significant bit (MSB) Sign bit is encoded after the MSB is encoded. Context (surrounding bit patterns) at each bit plane is examined. Key: explore patterns in binary bit-plane. (C) 2002-2013 by Yu Hen Hu 31

Wavelet Based Image Coding (generic) Discrete Wavelet Transform Context-based Quantization Entropy coding 10 20 30 40 50 60 20 40 60 10 20 30 40 50 60 20 40 60 2D discrete wavelet transform converts images into sub-bands Upper left is the DC coefficient Lower right are higher frequency sub-bands. (C) 2002-2013 by Yu Hen Hu 32

1D Discrete Wavelet Transform x(n) z 1 H0 H1 HO: low pass digital filter, H1: high pass digital filter. Z -1 : delay, 2: down-sample by 2 2 2 z 1 H0 H1 y0 y1 y2 y3 2 2 z 1 H0 H1 2 2 y0 y1 y2 y3 Recursive application of wavelet transform in spatial domain corresponds to dyadic partition of data in the frequency domain. /8 /4 /2 (C) 2002-2013 by Yu Hen Hu 33

2D Separate DWT Image in spatial domain L H LL HL LH HH 1D DWT applied alternatively to vertical and horizontal direction line by line. The LL band is recursively decomposed, first vertically, and then horizontally. This is Mallat method. Other methods have also been proposed. HL LH HH HL LH HH (C) 2002-2013 by Yu Hen Hu 34

SPIHT Set Partitioning in Hierarchical Trees. Amir Said and William Pearlman (IEEE Trans. CSVT, 1996) Based on zero tree wavelet coding Main ideas: Partial magnitude sorting of wavelet transformation coefficients Ordered bit plane transmission Exploitation of the self-similarity among wavelet coefficients between sub-bands having parentdescendent relations. (C) 2002-2013 by Yu Hen Hu 35

JPEG2000 Image components, tiles, and sub-band structures Wavelet transform Coefficient modeling Arithmetic coding (C) 2002-2013 by Yu Hen Hu 36

Image Size Image size in pixels: (width, height) = (x 1 -x o, y 1 -y o ) XRsiz, YRsiz: x, y increment (C) 2002-2013 by Yu Hen Hu 37

Tiling XTOsiz + XTsiz > XOsiz, YTOsiz + YTsiz > YOsiz (C) 2002-2013 by Yu Hen Hu 38

DC level shift and component transform Purpose of component transform is to decorrelate among components. For multi-spectral images, PCA may be used. There are reversible and irreversible transforms., 2,, I0 x y I1 x y I2 x y Y0 x, y 4 Y x y I x y I x y,,,,,, 1 2 1 Y x y I x y I x y 2 0 1 Forward reversible component transform,,,,,, 0 2 1 I x y Y x y I x y 2 1 1,, Y1 x y Y2 x y I1 x, y Y0 x, y 4 I x y Y x y I x y Inverse reversible component transform (C) 2002-2013 by Yu Hen Hu 39

Reversible Color Transform R 2 G B Yr 4 U R G r r V B G Ur Vr G Yr 4 R U G r r B V G Make lossless color coding possible. All components must have identical sub-sampling parameters and same depth Example: Let R =7, G = 3, B = 9. Then Forward transform: Yr = floor((7+2*3+9)/4) = 5, Ur = 7-3 = 4, Vr = 9 3 = 6 Inverse transform: G =5-floor((4+6)/4) = 5-2 = 3 = G, R = 4 + 3 =7, B = 6 + 3 = 9 = B. (C) 2002-2013 by Yu Hen Hu 40

Periodic Symmetric Signal Extension (C) 2002-2013 by Yu Hen Hu 41

Lossless 1D DWT Forward transform 2 2 2 X ext n X ext n Y 2n 1 X ext 2n 1 2 Y 2n 1 Y 2n 1 2 Y 2n X ext 2n 4 Reverse transform Yext 2n 1 Yext 2n 1 2 X 2n Yext 2n 4 2 2 2 X n X n X 2n 1 Yext 2n 1 2 I 0 1 2n+1 < i 1 1; I 0 2n < i 1 ; I 0 1 2n < i 1 1; I 0 2n+1 < i 1 ; X ext (), Y ext (): symmetrically, cyclic extended signals. Reversible Integer DWT DWT coefficients are integers without any truncation error provided image component pixel values are also integer-valued. Transform is exactly reversible. Non-causal filter. (C) 2002-2013 by Yu Hen Hu 42

Lossy 1D DWT Daubechies (9,7) filter in the lifting format. 2 1 2 1 2 2 2 2 2 2 1 2 1 2 1 2 1 2 2 2 2 2 2 1 2 1 2 1 b 2 1 2 2 Y n X n X n X n a ext ext ext Y n X n Y n Y n a ext a a Y n Y n Y n Y n b a a a Y n Y n Y n Y n b a b b Y n K Y n Y n Y n K b 2 ext 2 2 1 ext 2 1 2 2 2 1 2 1 2 1 2 1 2 2 2 2 b 2 b 2 1 b 2 1 2 1 2 1 2 2 2 X n K Y n a X n Y n K a X n X n X n X n b a a a X n X n X n X n b a a a X n X n X n X n X n X b n X b n X b n Step 1: i 0 3 2n+1 < i 1 +3 Step 2: i 0 2 2n < i 1 +2 Step 3: i 0 1 2n+1 < i 1 +1 Step 4: i 0 2n < i 1 Step 5: i 0 2n+1 < i 1 Step 6: i 0 2n < i 1 = 1.586 134 342, = 0.052 980 118 = 0.882 911 075, = 0.443 506 852 K = 1.230 174 105 Step 1: i 0 3 2n < i 1 +3 Step 2: i 0 2 2n+1 < i 1 +2 Step 3: i 0 3 2n < i 1 +3 Step 4: i 0 2 2n+1 < i 1 +2 Step 5: i 0 1 2n < i 1 +1 Step 6: i 0 2n+1 < i 1 (C) 2002-2013 by Yu Hen Hu 43

Row-based Wavelet Transform Problem with traditional wavelet transform: filtering to be performed in both vertical and horizontal directions. While access in one direction is easy, access in the other will require whole image to be buffered Difficult for implementation on PDA or other hand-held devices with limited amount of main memory. Row-based wavelet transform consumes the minimum amount of resources, gives same results as traditional wavelet transform Method Use a rolling window for each decomposition level to keep enough number (five) rows of image data in on-chip memory. (C) 2002-2013 by Yu Hen Hu 44

JPEG2000: Context coding: EBCOT (C) 2002-2013 by Yu Hen Hu 45

Context Coding Algorithm: EBCOT Embedded Block Coding with Optimal Truncation Block Coding Divide each sub-band into code blocks of samples which are coded independently For each block, a separate bit-stream is generated without utilizing any information from any of the other blocks Optimal Truncation The bit-stream of each block can be truncated to a variety of discrete lengths, with associated distortion A post-processing step after all blocks are compressed determines truncation point for each block (C) 2002-2013 by Yu Hen Hu 46

EBCOT Block Coding Taubman and Zakhor (IEEE Trans. IP, Sep. 94). Layered Zero Coding with Fractional Bit-Planes. For each bit plane, the encoding is applied three passes. Four types of coding operations for Arithmetic Entropy Coding: Zero Coding (ZC) Run-Length Coding (RLC) Sign Coding (SC) Magnitude Refinement Usage rule: If a pixel is not yet significant, use ZC and RLC to encode whether it is significant in the current bit plane. If so, use SC to encode its sign. If a pixel is already significant, use Magnitude refinement to encode the new bit position. (C) 2002-2013 by Yu Hen Hu 47

Coefficient Bit Modeling Wavelet coefficients are associated with different subbands arising from the 2D separable transform applied These coefficients are then arranged into rectangular blocks within each sub-band, called code-blocks. These code-blocks are then coded a bit-plane at a time starting from the most significant bit-plane with a non-zero element to the least significant bit-plane. For each bit-plane in a code-block, a special code-block scan pattern is used for each of three coding passes. Each coefficient bit in the bit-plane is coded in only one of the three coding passes: significance propagation, magnitude refinement, and cleanup. (C) 2002-2013 by Yu Hen Hu 48

Significant Pass Three Passes Scanning Scanning all insignificant samples which have at least one significant neighbors to determine if it will become significant at current bit plane. Use ZC to encode if a sample is still insignificant. If a sample becomes significant, also apply SC to encode its sign bit. Magnitude Refinement Pass Scanning samples which became significant in a previous bit-plane using MR encoding. Normalization Pass Scanning all remaining samples and encode using ZC + RLC (C) 2002-2013 by Yu Hen Hu 49

Scanning Order within a code block Each bit plane with a code block is scanned during the context coding process in a specific order. All quantized transform coefficients are represented in sign-magnitude representation. For a particular sub-band, there is a maximum number of magnitude bits, M b. The significance state changes from insignificant to significant at the bit plane where the most significant 1 bit is found. For a code-block, the number of bit-planes starting from the most significant bit-plane that are all zero, is signaled in the packet header (C) 2002-2013 by Yu Hen Hu 50

Neighboring states used to form context Each coefficient in a codeblock has an associated binary state variable called its significance state. Significance states are initialized to 0 (coefficient is insignificant) and may become 1 (coefficient is significant) during the course of the coding of the code-block. Four different context formation rules are defined, one for each of the four coding operations: significance propagation pass: significance coding, sign coding, magnitude refinement pass magnitude refinement coding, cleanup pass Cleanup coding. The current context obtained during context coding is provided to the arithmetic MQ coder. (C) 2002-2013 by Yu Hen Hu 51

Bit plane encoding orders The number of bit-planes starting from the most significant bit that have no significant coefficients (only insignificant bits) is signaled in the packet headers. The first bit-plane with a non-zero element has a cleanup pass only. The remaining bit-planes are coded in three coding passes. Each coefficient bit is coded in exactly one of the three coding passes. Which pass a coefficient bit is coded in depends on the conditions for that pass. In general, the significance propagation pass includes the coefficients that are predicted, or most likely, to become significant and their sign bits, as appropriate. The magnitude refinement pass includes bits from already significant coefficients. The cleanup pass includes all the remaining coefficients. (C) 2002-2013 by Yu Hen Hu 52

Context of Significance and Cleanup Passes LL and LH sub-bands (vertical high pass) HL sub-bands (horizontal high pass) HH sub-bands (diagonally high pass) H V D H V D (H+V) D Context label 2 x x x 2 x x 3 8 1 1 x 1 1 x 1 2 7 1 0 1 0 1 1 0 2 6 1 0 0 0 1 0 2 1 5 0 2 x 2 0 x 1 1 4 0 1 x 1 0 x 0 1 3 0 0 2 0 0 2 2 0 2 0 0 1 0 0 1 1 0 1 0 0 0 0 0 0 0 0 0 x: don t care (C) 2002-2013 by Yu Hen Hu 53

Significance propagation pass The significance propagation pass includes only bits of coefficients that were insignificant (the significance bit has yet to be encountered) and have a non-zero context. All other coefficients are skipped. The context is delivered to the arithmetic decoder (along with the bit stream) and the decoded coefficient bit is returned. If the value of this bit is 1 then the significance state is set to 1 and the immediate next bit to be decoded is the sign bit for the coefficient. Otherwise, the significance state remains 0. When the contexts of successive coefficients and coding passes are considered, the most current significance state for this coefficient is used. (C) 2002-2013 by Yu Hen Hu 54

Sign Bit Coding Two phases: Summarize contributions of vertical and horizontal neighbors Reduces these contributions into 1 or 5 context labels The context labels are sent to MQ arithmetic coder. Signbit = AC(contextlabel) XORbit Signbit: sign bit of the current coefficient AC(contextlabel) is the valuate returned from arithmetic decoder given the context label and the bit stream. V 0 (or H 0 ) V 1 (or H 1 ) V (or H) contribution S (significant), P (positive) S, P 1 S, N (negative) S, P 0 I (insignificant) S, P 1 S, P S, N 0 S, N S, N 1 I S, N 1 S, P I 1 S, N I 1 I I 0 H contribution V contribution Context label XORbit 1 1 13 0 1 0 12 0 1 1 11 0 0 1 10 0 0 0 9 0 0 1 10 1 1 1 11 1 1 0 12 1 1 1 13 1 (C) 2002-2013 by Yu Hen Hu 55

Magnitude Refinement H + V + D 1st refinement for Context this coefficient label X False 16 1 True 15 0 True 14 The magnitude refinement pass includes the bits from coefficients that are already significant (except those that have just become significant in the immediately proceeding significance propagation pass). The context used is determined by the summation of the significance state of the horizontal, vertical, and diagonal neighbors. These are the states as currently known to the decoder, not the states used before the significance decoding pass. Further, it is dependent on whether this is the first refinement bit (the bit immediately after the significance and sign bits) or not. (C) 2002-2013 by Yu Hen Hu 56

Cleanup Pass The first pass and only coding pass for the first significant bit-plane. The third and the last pass of all the remaining bit-planes. Use both neighbor context as in significant propagation pass and runlength coding. (C) 2002-2013 by Yu Hen Hu 57

Context-based Arithmetic Entropy Coding The MQ-coder, a low complexity entropy coder is used. Contexts are based on the significance of horizontal, vertical, diagonal neighbors of the pixel concerned. Current there are 46 contexts. (C) 2002-2013 by Yu Hen Hu 58

Layers Bit-stream is a succession of layers. Layer contains the contributions from each code block. The block truncation associated with each layer are optimal in rate-distortion sense. Single layer can achieve progressive in resolution Multiple layers can achieve progressive in SNR (C) 2002-2013 by Yu Hen Hu 59

Features of EBCOT Low Memory Requirement Inherent Parallelism Efficient Rate Control High Compression Performance Region of Interests (ROI) and Degree of Interests (DOI) Error Resilience Simple Quantization Modest Complexity (C) 2002-2013 by Yu Hen Hu 60

Region of Interests Coding (ROI) (C) 2002-2013 by Yu Hen Hu 61

Region of Interests Coding An ROI is a part of an image that is coded earlier in the code stream than the rest of the image (the background). The coding is also done in such a way that the information associated with the ROI precedes the information associated with the background. The method used is the Maxshift method. ROI allows certain parts of the image to be coded in better quality Static: The ROI is decided and coded once for all at the encoder side Dynamic: The ROI can be decided and decoded on the fly from a same bit stream (C) 2002-2013 by Yu Hen Hu 62

Encoding MaxShift Method 1. Generate ROI mask, M(x,y). M(x,y) = 1, wavelet coefficient (x,y) is needed for ROI M(x,y) = 0, wavelet coefficient (x,y) belong to background pixels and can be sacrificed w/o affecting ROI. 2. Find the scaling value, s and scale up all ROI wavelet coefficients by s bits so that ROI coefficients > 2 s > background coefficient 3. Write the scaling value, s, into code stream using the RGN marker Decoding 1. Get s from RGN marker 2. Scale background wavelet coefficients by 2 s (C) 2002-2013 by Yu Hen Hu 63

ROI Mask Computation Must track wavelet coefficients that will contribute to ROI region pixels. C. Christopoulos, A. Skodras, T. Ebrahimi, JPEG2000 (online tutorial) (C) 2002-2013 by Yu Hen Hu 64

Scale Operation C. Christopoulos, A. Skodras, T. Ebrahimi, JPEG2000 (online tutorial) (C) 2002-2013 by Yu Hen Hu 65

Advantages of Maxshift method Support for arbitrary shaped ROI s with minimal complexity No need to send shape information No need for shape encoder and decoder No need for ROI mask at decoder side Decoder as simple as non-roi capable decoder Can decide in which sub band the ROI will begin therefore it can give similar results to the general scaling method (C) 2002-2013 by Yu Hen Hu 66

Conclusion JPEG2000 is an emerging image coding standard for the next generation of digital imaging. No IPR (intellectual property right) on part I of the standard (free licensing) More complex than JPEG but designed with hardware implementation in mind. Many companies are working to incorporate JP2 into the next generation of digital camera and scanners. (C) 2002-2013 by Yu Hen Hu 67