JPEG 000 tandard - Overview Ping-ing Tsai, Ph.D. <Outline> JPEG000 Background & Overview Part I JPEG000 oding ulti-omponent Transform Bit Plane oding (BP) Binary Arithmetic oding (BA) Bit-Rate ontrol odes of current JPEG Why JPEG000? 5. bpp equential Lossless mode decoded image is exact replica of the original equential DT based mode the simplest and widely used algorithm in this mode is the Baseline JPEG Progressive DT based mode encodes image in multiple scans Hierarchical ode encodes at multiple resolution lossless Original image b i t s t r e a m.89 bpp.84 bpp 3 4 JPEG 000 tandard Part I : ore oding ystem Part II : Extensions Part III : otion JPEG 000 Part IV : onformance Testing Part V : Reference oftware Part VI : ompound image file format ore parts are coming ulti-omponent Transform Discrete Wavelet Transform onvolution (9-7 filter) Lifting (5-3 filter) Quantization Image ulti component transform Tier- coding Bit-Plane oding Binary Arithmetic coding Tier- coding Tag-Tree coding packet header Region of Interest (ROI) Bit-Rate control (open issue) JPEG 000 - Part I Discrete Wavelet Transform Quantization Region of Interest Rate ontrol Tier- Encoder Tier- Encoder oded Image 5 6
JPEG000 Encoder Data Flow ROI tile ubbands ubband ode block tile ubbands ubband ubband ubband B B An Image omponent code block BP DWT ontext & Data ubband Tile BA ubband ompressed data ode block ubband Bit stream Layer formation and data formatting omponent of an image tile tile tile tile DWT + (Q) ubbands ubband ubband ubband ubbands ubband ubband ubband ubbands ubband ubband ubband B B B B BP + BA Bit treams 7 8 Thinking Parallel? Parallelism Opportunities ultiple omponent Image omponent omponent D level hifting D level hifting olor Transformation (Optional) JK Encoding JK Encoding ompressed Image Processing component(s) in an image Processing tile(s) in a component Processing subband(s) in a tile Processing code block(s) in a subband omponent 3 D level hifting JK Encoding 9 0 <Outline> JPEG000 Background & Overview Part I JPEG000 oding ulti-omponent Transform Bit Plane oding (BP) Binary Arithmetic oding (BA) Bit-Rate ontrol ulti-omponent Transform Part I allows color transformation on first three components Reversible olor Transform (RT) Irreversible olor Transform (IT)
Reversible olor Transform Irreversible olor Transform R G B Yr 4 Ur R G Vr B G R Ur G Ur Vr G Yr 4 B Vr G Y 0.99 0.587 0.4 R U 0.6875 0.336 0.5 * G V 0.5 0.4869 0.083 B R.0 0.40 Y G.0 0.3443 0.744 * U B.0.77 0 V 3 4 <Outline> JPEG000 Background & Overview Part I JPEG000 oding ulti-omponent Transform Bit Plane oding (BP) Binary Arithmetic oding (BA) Bit-Rate ontrol Discrete Wavelet Transform Two procedures onvolution based (9-7 filter) Lifting based (5-3 filter) DWT is generally more intensive computationally compared to DT emory requirement is high Different filters exist 5 6 Forward -D DWT Analysis Filter Bank The -D signal is separately filtered using one low-pass filter and one high-pass filter Both the filtered output signals are decimated by a factor of two to produce low-pass and highpass subbands. X(0 N-) L H X L (0 N/-) X H (0 N/-) Example - Forward DWT Analysis filters (5, 3): Low pass filter L : ( - 6 - )/8 High pass filter H : ( - - )/ -D ignal: 0 0 0 0 0 0 0 0 Filtered output before downsampling 0 0 8.75.5 8.75.5 0 0 0 0 0-5 5 0 0 0 ubbands after downsampling X L : 0.5.5 0 X H : 0 0 5 0 7 8 3
Inverse -D DWT ynthesis Filter Bank Both the subbands are interpolated by inserting 0 s in between two samples. The interpolated low-pass subband is low-pass filtered and the interpolated high-pass subband is high-pass filtered. Two filtered outputs are added X (0 N-) + L H X L (0 N/-) X H (0 N/-) Example - inverse DWT ynthesis filters (5, 3): Low pass filter L : ( )/ High pass filter H : ( - - 6 - - )/8 Interpolated subbands 0 0 0.5 0.5 0 0 0 0 0 0 5 0 0 0 Filtered outputs L : 0 0 0.65.5 6.5.5 0.65 0 H : 0 0-0.65 -.5 3.75 -.5-0.65 0 Reconstructed ignal: 0 0 0 0 0 0 0 0 9 0 Perfect Reconstruction Property Ideally, the analysis filter pair (L, H ) and the synthesis filter pair (L, H ) are chosen to yield zero overall distortion, i.e., X(0 N-) = X (0 N-). Dyadic DWT Decomposition In multi-level dyadic decomposition, the analysis filter banks are successively applied to the low-pass subband to further decompose it into two subbands. X(0 N-) L H L H + X (0 N-) L H L H L H Analysis filter Bank ynthesis Filter Bank DWT coefficient of G channel Decoded at different resolutions from the same bit-stream Level DWT 3 Level DWT Level DWT 3 4 4
Lifting scheme for DWT usually requires less computation and less memory can be adapted to integer-to-integer transform for lossless compression forward and backward transforms complexity are same does not require explicit signal extension at boundaries in-place computation Lifting Algorithm plit: The input signal, x k, is split into even and odd samples (lazy DWT) s k0 x k d k0 x k+ Lifting: This is executed in N sub-steps as polyphase filters d kn d n- k +P n (m).s n- m, n [,,, N] s kn s n- k +U n (m).d mn, n [,,, N] Normalization: Final output is normalized as s kn K s N k d kn K d N k 5 6 x i k N d k N Phase plitter K - K - Lifting Block Diagram k 0 d k 0 P (z) - U N (z) U (z) P N (z) Forward transform P N (z) - - U (z) Inverse transform U N (z) P (z) - k 0 d k 0 N k k Low-pass k d k N High-pass Phase joiner x i Input amples Lifting scheme for (5, 3) filter x 0 x x x 3 x 4 x 5 x 6 x 7 x 8-0.5-0.5-0.5-0.5-0.5-0.5-0.5-0.5 High-pass samples H 0 H H H 3 low-pass samples 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 L 0 L L L 3 L 4 H k = x k+ - [x k + x k+ ]/ L k = x k + [H k- + H k ]/4 7 8 Lifting scheme for (9, 7) filter x 0 x x x 3 x 4 x 5 x 6 x 7 x 8 p p p p p p p u u u u u u u u p p p p p p p u u u u u u u u p = -.5863434, u = -0.059808, k =.307404 p = +0.889075, u = +0.44350685, k = / k p p Input amples Intermediate k results k High-pass samples low-pass samples <Outline> JPEG000 Background & Overview Part I JPEG000 oding ulti-omponent Transform Bit Plane oding (BP) Binary Arithmetic oding (BA) Bit-Rate ontrol 9 30 5
Quantization Dead-Zone Quantization Uniform scalar quantization with deadzone No quantization in lossless mode Quantization rule: b b b b b b b y q sign( y) b -3 - - 0 + + +3 3 3 <Outline> JPEG000 Background & Overview Part I JPEG000 oding ulti-omponent Transform Bit Plane oding (BP) Binary Arithmetic oding (BA) Bit-Rate ontrol Tier- coding Fractional Bit Plane oding (BP) Binary Arithmetic oding (BA) 33 34 Bit Plane oding (BP) Embedded Block oding with Optimized Truncation (EBOT) algorithm 3 coding passes for each bit plane ignificance Propagation Pass (PP) agnitude Refinement Pass (RP) lean Up Pass (UP) 4 oding operations Zero oding (Z), ign oding (), agnitude Refinement oding (R), Run Length oding (RL) Output one or more (X, D) where X is a context and D is a binary data. BP Fixed can Pattern Example of scan pattern for a 5x0 code block 35 36 6
ign-agnitude Representation ign [ m, n] 0 0 0 0 0 0 0 0 0 0 0 0 0 Quantized DWT coefficients 4-3 5 0 4 0 9-5 4 7 8-5 3 agnitude v p [ m, n] 4 3 5 0 4 0 9 5 4 7 8 5 3 ignificance Each coefficient in a code-block has an associated binary state variable called its significance state ( [ m, n]). ignificance states are initialized to 0 and may become during the course of the coding of the code-block. Once the significance state for a bit position becomes, it stays throughout the encoding of the codeblock. 37 38 ignificance Propagation Pass Apply Z to the samples if the bit is in insignificant state and at least one of its eight neighbors is significant. Apply after above Z if the bit value is and this is the most significant bit of the pixel agnitude Refinement Pass Apply R to the samples if the sample has a bit value of in any of the previous bit planes and the sample has not been encoded by PP in the current bit plane 39 40 leanup Pass The samples that have not been coded in either PP or RP, in current bit plane, are coded in this pass. Apply RL or Z (w/ or w/o ) Zero oding Operation X is determined depending upon the significance state variables of the eight surrounding neighbors. D is equal to the bit value of the sample coefficient at the current location [m,n] at bit plane p. p D v [ m, n] 4 4 7
Zero oding ontext Table d v d h X h d v d LL & LH subbands h v d Note: HL & HH use different Z context tables X x x 8 >= x 7 0 >= 6 0 0 5 0 x 4 0 x 3 0 0 >= 0 0 0 0 0 0 ign oding Operation X is determined by a horizontal contribution value H and vertical contribution value V. Possible values of H & V are -, 0, + 0 implies both neighbors are insignificant or both are significant but has opposite signs implies that one or both neighbors are significant and positive -implies that one or both neighbors are significant and negative 43 44 ign oding (cont.) H V X 0 3 0 0-0 0 0 0 0 0 0 9 0-0 - - 0 - - 3 ˆ D ˆ [ m, n] agnitude Refinement oding Operation h v x >= 0 d First refinement for this coefficient false true true p D v [ m, n] X 6 5 4 45 46 Run Length oding Operation The RL is invoked in place of Z iff the following conditions hold: Four consecutive samples must have a zero significance state variable. All four samples must have identically zero neighborhood. The group of four samples must be aligned on the four-sample boundary within the scan. Run Length oding (cont.) ase all four bit values are zero (X, D) = (7, 0) ase else send (7, ) and two pairs (X, D) = (8, x) (X, D) = (8, y) where xy is the distance (in binary) of the first bit in the 4 bit strip 47 48 8
<Outline> JPEG000 Background & Overview Part I JPEG000 oding ulti-omponent Transform Bit Plane oding (BP) Binary Arithmetic oding (BA) Bit-Rate ontrol Binary Arithmetic oding JPEG 000 uses Q coder with 9 different contexts generated by EBOT. The data bit D in a (X, D) pair is coded by BA, X is used to select the necessary Q value from the Q-table. A predetermined fixed probability estimation state machine is provided by the standard. 49 50 <Outline> JPEG000 Background & Overview Part I JPEG000 oding ulti-omponent Transform Bit Plane oding (BP) Binary Arithmetic oding (BA) Bit-Rate ontrol Tier- coding Bit tream Formation Tag Tree oding 5 5 Layer 5 Layer 4 Layer 3 Layer ode-block contribution to bit-stream layers block bit-stream block bit-stream empty block 3 bit-stream block 4 bit-stream empty block 5 bit-stream empty empty empty empty block 6 bit-stream block 7 bit-stream empty Layer The coded data (bit-stream) of each codeblock is distributed across one or more layers in the code stream Each layer consists of some number of consecutive bit-plane coding passes from each codeblock in the tile, including all subbands of all components for that tile. Layer empty 53 54 9
Resolution Partition of subbands in one tile. (N L +) resolutions for N L levels DWT decomposition. r = 0 : (N L )LL subband only r = : (N L )LH, (N L )HL, (N L )HH r : (N L - r+)lh, (N L - r+)hl, (N L - r+)hh Precinct Partition in each resolution (formed in DWT domain). Power of in size (line up with codeblock size boundary). Don t cause block (tile) artifacts. r = N L : LH, HL, HH 55 56 Precinct (cont ) Progression Order LL HL LH HH LH 4 5 6 7 HL 0 3 HH 8 9 0 A precinct A codeblock Layer-Resolution-omponent-Precinct Resolution-Layer-omponent-Precinct Resolution-Precinct-omponent-Layer Precinct-omponent-Resolution-Layer omponent-precinct-resolution-layer 57 58 level DWT with 6 ode Blocks LL (B) LH (B3) (B9) (B) LH HL (B) HH (B4) (B0) (B) (B5) (B7) (B3) (B5) HL HH (B6) (B8) (B4) (B6) BP4 BP3 BP BP Assume maximum 4 Bit Planes for each ode Block B B B3 B4 B5 B6 B7 B8 B9 B9 B B B3 B4 B5 B6 LL HL LH HH HL LH HH : ignificance propagation pass : agnitude refinement pass : lean up pass 59 60 0
6 BP4 BP3 BP BP B B B3 B4 B5 B6 B7 B8 B9 B9 B B B3 B4 B5 B6 LL HL LH HH HL LH HH Bit stream with lower resolution image : ignificance propagation pass : agnitude refinement pass : lean up pass 6 BP4 BP3 BP BP B B B3 B4 B5 B6 B7 B8 B9 B9 B B B3 B4 B5 B6 LL HL LH HH HL LH HH Bit stream progressive in term of resolution : ignificance propagation pass : agnitude refinement pass : lean up pass 63 BP4 BP3 BP BP B B B3 B4 B5 B6 B7 B8 B9 B9 B B B3 B4 B5 B6 LL HL LH HH HL LH HH Bit stream progressive in term of quality : ignificance propagation pass : agnitude refinement pass : lean up pass 64 BP4 BP3 BP BP B B B3 B4 B5 B6 B7 B8 B9 B9 B B B3 B4 B5 B6 LL HL LH HH HL LH HH Bit stream w/ highest resolution & highest quality : ignificance propagation pass : agnitude refinement pass : lean up pass 65 BP4 BP3 BP BP B B B3 B4 B5 B6 B7 B8 B9 B9 B B B3 B4 B5 B6 LL HL LH HH HL LH HH Bit stream w/ highest resolution & target NR quality : ignificance propagation pass : agnitude refinement pass : lean up pass 66 BP4 BP3 BP BP B B B3 B4 B5 B6 B7 B8 B9 B9 B B B3 B4 B5 B6 LL HL LH HH HL LH HH Bit stream w/ highest resolution & target file size : ignificance propagation pass : agnitude refinement pass : lean up pass
Tag Tree oding Basic Tag Tree Data tructure Tag Tree - Hierarchical representation of a D array of non-negative integer values, where successively reduced resolutions form a tree. q 3 (0,0) 3 q 3 (,0) q 3 (,0) 3 q 3 (3,0) q 3 (4,0) 3 q 3 (5,0) 4 3 q (0,0) Usage in JPEG 000 (to carry the following information) Level 3 leaf nodes q 3 (x,y) (original array of numbers) Level internal nodes q (x,y) The bit-stream layers to which the code block contributes code words The length of these code words The most significant magnitude bit-plane at which any sample in the code block is non-zero The truncation point between the bit-stream layers q (0,0) q (,0) q (,0) Level internal nodes q (x,y) q 0 (0,0) Level 0 root node q 0 (x,y) 67 68 Basic Tag Tree Data tructure Basic Tag Tree Data tructure Tree structure root node 0 q0(0,0) 0 00 0 00 0 q(0,0) q(,0) 0 q(0,0) q(,0) q(0,) q(,) q(,0) (q 0 (0,0),q (0,0),q (0,0)) q 3 (0,0) q 3 (,0) (q (,0)) q 3 (,0) q 3 (3,0) (q (,0),q (,0) ) q 3 (4,0) 0 3 3 4 q3(0,0) 00 00 q3(,0) q3(,0) q3(3,0) 0 3 3 q3(4,0) q3(5,0) Bit tream generated 69 70 Tag Tree Usage in JPEG 000 Two independent tag trees for each precinct, resolution level, tile and component: One for code block first inclusion One for number of all zero bit planes <Outline> JPEG000 Background & Overview Part I JPEG000 oding ulti-omponent Transform Bit Plane oding (BP) Binary Arithmetic oding (BA) Bit-Rate ontrol 7 7
Bit-Rate control Post Processing ethod To meet a particular target bit-rate To assure the desired number of bytes is used by the code stream while assuring the highest image quality possible. Open issue 73 74 Original image.5 bpp with 4 Level DWT 75 76 3.5 bpp with 4 Level DWT ompressed at 4.9 bpp with 4 level DWT 77 78 3
<Outline> JPEG000 Background & Overview Part I JPEG000 oding ulti-omponent Transform Bit Plane oding (BP) Binary Arithmetic oding (BA) Bit-Rate ontrol Region of Interest (ROI) A binary mask generated in the wavelet domain for distinction of ROI and background ax-hift method (Part I) caling based (Part II) 79 80 ROI mask NO ROI ROI caling Based axhift R O I B R O I BG R O I BG BG BG BG BG LB 8 8 Original image ROI Example using ax_hift (encoded at 5.85 bpp) 83 84 4
Difference image (d > 5) (5.7 bpp) 85 86 (ROI w/ 3.4 bpp) (ROI w/ 3.0 bpp) 87 88 (ROI w/.5 bpp) Implementation hallenges emory management Data pipeline Power consumption omplexity 89 90 5