Information Theoretic Imaging WU Faculty: J. A. O Sullivan WU Doctoral Student: Naveen Singla Boeing Engineer: James Meany First Year Focus: Imaging for Data Storage Image Reconstruction Data Retrieval
Information Theoretic Imaging Theoretical and Implementation Issues in Image formation Image deblurring Automatic target recognition Data modeling Data Storage Systems Channel coding problems Graphical models for available data Binary rather than continuous data Cross-Fertilization Imaging approach to data storage Graphical models for image formation, ATR
Joint Detection and Decoding Schemes for 2D Recording J. A. O Sullivan, N. Singla, Y. Wu, R. S. Indeck Electronics Systems and Signals Research Laboratory Magnetic Information Science Center Washington University
Enabling Technology: Disk Drives ~,,x increase in 45 years! Magnetic disk storage areal density vs. year of IBM product introduction (From D. A. Thompson)
Problem Description Reliable data retrieval from channels having 2-D ISI Advanced storage media: Patterned media As bit aspect ratio reduces inter-track interference becomes significant Optical memories
Science enables: 6 million-million bits/square inch!
Conventional PRML System Timing & Tap Updating xn Magnetic Recording Channel Wide-Band Noise Receive Filter (low pass) nt+τ e Adaptive Filter yn ŷn Viterbi Detector xˆ n n P ( D ) = ( D )( + D ), n =,2,3,... No generalization to 2D
Joint Equalization and Decoding General 2D ISI Schemes Using 2D MMSE equalization and decoding Using novel message-passing algorithms that take advantage of the 2D dependence Separable 2D ISI Using turbo equalization Singla et al., Iterative decoding and equalization for 2-D recording channels, IEEE Trans. Magn., Sept. 22.
Channel Model a ( i, j) x ( i, j) r ( i, j) x ( i, j) ( i, j) LDPC Encoder Channel ISI Channel w( i, j) Equalizer x(i,j) {+,-} Channel ISI is 2D and linear Noise assumed to be AWGN LDPC Decoder a
2D Intersymbol Interference 2 2 2 Λ Λ Λ Μ Ο Μ Μ Ο Λ Λ Λ kk k k k x x x x x x x 2 + + + + + + + + + k k k k k k kk kk k k k k k k r r r r r r r r r r r r r r r r Λ Λ Λ Λ Μ Ο Μ Μ Μ Ο Μ Λ Λ Λ Λ Λ GUARD BAND ), ( ), ( ), ( ), ( 2, j w i m n h n j m i x j i r m n + = = ), ( j w i = c b c b a b c b c h
MMSE Equalization and Decoding a ( i, j) x ( i, j) r ( i, j) x ( i, j) ( i, j) LDPC Encoder Channel ISI w( i, j) Equalizer LDPC Decoder a Equalizer designed assuming inputs to be Gaussian
Performance MMSE Equalization and Decoding Bit Error Rate, Log Base - -2-3 -4-5 -6 h =.22.366.22.366.89.366 BIAWGN Capacity LDPC No ISI Wiener.22.366.22-7 2 3 4 5 6 7 SNR (db)
Iterative MMSE Equalization and Decoding a ( i, j) x ( i, j) r ( i, j) x ( i, j) ( i, j) LDPC Encoder Channel ISI Equalizer LDPC Decoder a w( i, j) Extrinsic Information Soft information, estimated mean of the codeword, passed from LDPC decoder to equalizer E[ x] x = Pr( x = ) Pr( x = ) = E[ x] + W **[ r h** E[ x]]
Performance Iterative MMSE Equalization and Decoding Bit Error Rate, Log Base - -2-3 -4-5 -6 h =.22.366.22.366.89.366 BIAWGN Capacity LDPC No ISI Itrw iener_ Wiener.22.366.22-7 2 3 4 5 6 7 SNR (db)
Full Graph Message-Passing Check Nodes Codeword Bit Nodes Data Nodes h =.5.5.25 r i, j = xi, j +.5xi, j +.5xi, j +. 25xi, j + w i, j
Full Graph x(i,j) x(i,j+) x(i,j+2) From Check Nodes x(i+,j) r(i,j) r(i+,j) x(i+,j+) r(i,j+) r(i+,j+) x(i+,j+2) x(i+2,j) x(i+2,j+) x(i+2,j+2)
Performance Full Graph Message-Passing Bit Error Rate, Log Base - -2-3 -4-5 -6 h =.22.366.22 BIAWGN Capacity LDPC No ISI Full graph_5 Itrw iener_ Wiener.366.89.366.22.366.22-7 2 3 4 5 6 7 SNR (db)
Full Graph Analysis Length 4 cycles present which degrade performance of message-passing algorithm x(i,j) x(i,j+) x(i,j+2) From Check Nodes x(i+,j) r(i,j) r(i+,j) x(i+,j+) r(i,j+) r(i+,j+) x(i+,j+2) x(i+2,j) x(i+2,j+) x(i+2,j+2) Kschischang et al., Factor graphs and the sum-product algorithm, IEEE Trans. Inform. Theory, Feb. 2.
Ordered Subsets Message-Passing From Imaging Data set is grouped into subsets to increase rate of convergence For Decoding Observed data is grouped into subsets to eliminate short length cycles in the Channel ISI graph H. M. Hudson, and R. S. Larkin, Accelerated image reconstruction using ordered subsets of projection data, IEEE Trans. Medical Imaging, Dec. 994
Grouped ISI Graph Labeling of data nodes into 4 subsets For each iteration use data nodes of one label only J. A. O Sullivan, and N. Singla, Ordered subsets message-passing, Submitted to Int l Symp. Inform. Theory 23.
Performance Ordered Subsets Message-Passing Bit Error Rate, Log Base - -2-3 -4-5 -6 h =.22.366.22.366.89.366.22.366.22 BIAWGN Capacity LDPC No ISI Ord Subsets_ Full graph_5 Itrw iener_ Wiener -7 2 3 4 5 6 7 SNR (db)
Performance Joint Equalization and Decoding Schemes Bit Error Rate, Log Base - -2-3 -4-5 -6 h =.9.98.9.98.98.98.9.98.9 BIAWGN Capacity LDPC No ISI Ord Subsets_ Full graph_5 Itrw iener_ Wiener -7.5.5 2 2.5 SNR (db)
Performance Joint Equalization and Decoding Schemes Bit Error Rate, Log Base - -2-3 -4-5 -6 h =.353.353.77.353.353 BIAWGN Capacity LDPC No ISI Ord Subsets_ Full graph_5 Itrw iener_ Wiener -7 2 3 4 5 6 7 SNR (db)
Performance Joint Equalization and Decoding Schemes Bit Error Rate, Log Base - -2-3 -4-5 -6 h =.98.98.98.98.98 BIAWGN Capacity LDPC No ISI Ord Subsets_ Full graph_5 Itrw iener_ Wiener -7.5.5 2 2.5 SNR (db)
Consider Separable 2D ISI Wu et al., Iterative detection and decoding for separable two-dimensional intersymbol interference Submitted to IEEE Trans. Magn., June. 22.
A Separable 2D ISI x(i, j) Row ISI y(i, j) Column ISI r(i, j) Channel Detector Separable Channel Response w(i,j) h =.5.5 =.25.5 (.5) Advantages of Separable 2D ISI Apply existing one-dimensional equalization methods Reduced Detector Complexity Tüchler et al., Turbo equalization: principles and new results, IEEE Trans. On Comm., May 22.
Row-by-Row Decoder r(i, j) Column Equalization MAP Detector LDPC Decoder Hard Decision Extrinsic Information Conventional One-Dimensional Iterative Decoder MMSE and Zero-forcing criteria used for Equalization
Performance Bit-Error-Rate In log - -2-3 -4-5 Row-by-Row Decoder.5 h =.5.25 LDPC No ISI Row -by-row : MMSE_iter Row -by-row : ZF_iter -6 2 3 4 5 SNR (db)
Row-and-Column Decoder r(i, j) Column MAP Detector Row MAP Detector LDPC Decoder Hard Decision Extrinsic Information Conventional One-Dimensional Iterative Detector Inputs to column detector are not binary
Performance - Row-and-Column Decoder h =.5.5.25 Bit Error Rate, Log Base -2-3 -4 LDPC No ISI Row -and-column_ Ordered Subsets_ Full Graph_5 Itrw iener_ Wiener Row -by-row : ZF_iter -5-6 2 3 4 5 SNR (db)
Conclusions MMSE equalization and decoding Good Performance with Iterative Equalization Message-passing algorithms Full graph algorithm performance deteriorated due to short cycles Ordered subsets message-passing gives best performance for general 2D ISI Separable ISI decoding Best performance for separable 2D ISI Low complexity Approximate channel response by separable response
Performance Block length regular (3,6) LDPC code Ignoring ISI Bit Error Rate in log - -2-3 -4-5 ISI-free ISI ignored -6 2 4 6 8 2 4 SNR [db]
Performance MMSE Equalization and Decoding Bit Error Rate in log - -2-3 -4-5 ISI-free Wiener ISI ignored -6 2 4 6 8 2 4 SNR [db]
Performance Iterative MMSE and Decoding Bit Error Rate in log - -2-3 -4-5 ISI-free Itr Wiener_ Wiener -6 2 3 4 5 SNR [db]
Performance Full Graph Message-Passing Bit Error Rate in log - -2-3 -4-5 ISI-free Full Graph_5 Itr Wiener_ Wiener -6 2 3 4 5 SNR [db]
Performance Ordered Subsets Message-Passing Bit Error Rate in log - -2-3 -4-5 ISI-free Ordered Subsets_ Full Graph_5 Itr Wiener_ Wiener -6 2 3 4 5 SNR [db]
Advanced Media