Adaptive transform coding of viewphone signals

Transcription

1 Loughborough University Institutional Repository Adaptive transform coding of viewphone signals This item was submitted to Loughborough University's Institutional Repository by the/an author. Additional Information: A Doctoral Thesis. Submitted in partial fullment of the requirements for the award of Doctor of Philosophy of Loughborough University. Metadata Record: Publisher: c W.C. Wong Please cite the published version.

2 This item was submitted to Loughborough University as a PhD thesis by the author and is made available in the Institutional Repository ( under the following Creative Commons Licence conditions. For the full text of this licence, please go to:

3 LOUGHBOROUGH UNIVERSITY OF TECHNOLOGY LIBRARY AUTHOR/FILING TiTlE i.! J-\:J I --AC(iS-SI(iN/-COPY--NO t;)o 'tsi CJ /C'L'. I : /-' VOL. NO. CLASS MARK I 1.' /' 3 0 /!\ ,cr i. J.) 1'(9.3.-L. J':' :0'.." -0., '.. " 3'1989 ) '\ \

4 ,I',,\, V I.

5 ADAPTIVE TRP,HSFORM CODING OF VIEHPHONE SIGHALS by,[. C. I-lONG A Doctoral Thesis Submitted in partial fulfilment of the requirements for the m/ard of Doctor of Philosophy of the J,oughborough University of 'J'eohnoloG"Y by,,-1. C. \".'OHG", February 1980 "'

6 L... hbarough Unlv... lty af T «""01""" Library J..-.L lio Cl.. Ace. O'tS\'I /02- Ne.,

7 (i) ABSTRACT The initial phase of the research programme involves the design, development and construction of a reliable and versatile Video AC'luisition and Display System (VADS) to provide a mealos of subjective evaluation of the results of computer simulations. An orthogonal transformation generates a set of uncorrelated coefficients and has, in addition, an energy compaction property which leads to more efficient coding of the source data. As an introduction to its application, its use in the d.etection and correction of transmission erors is cosidered. SimulatioDs are performed on two-dimensional Gauss-l1arkoy sequencen. One-dimensional Hadamard transformation is performed in the hori7.ontal direction and the vertical correlation of the simulated data is exploited to achieve partial error detection and correction. Viel<phone signals, particularly those derived from head and shoulder scenes, possess much spatial and temporal redundancy 'tlhich may be significantly reduced at the expense of a slight, but subjectively tolerable, degree of deg)'."ll:clation in picturj'l quality. The discret.e ',>',.. '. Cos ine transform offers a corriparati V-e ly' fas t and e fficien t -. -,.'./'.: ; transforma tion for '3.chi8ving this objec:t1vl", and i.8 used 8xtensi ve -' Y here, in conjunction >I1th various adaptple aleorithms -.;hich select and code the transform coefficients for transmission on the basis of the statistical and local content of the data. Initially, twodimensional adaptive tcchni'lues arc employed follo>led by an extension of the ideas developed to th time domain. It is demonstrated that a marked reduction il1 the signal transmirsic,n rate can be achieved by this method.

8 (ii l ACKNOWLEDGEt1ENTS I wish to express my most sincere thanks to Dr. Raymond Steele for his constant guidance and help throughout the course of this work. I am grateful to Roger Clarke for his understanding and. patience as well as his useful comments in the preparation of this thesis. In addition, I also wish to thank Phil Atkinson and Tony Erwood for their assistance'. Thanks are due to my parents for their constant encourllgement and support throughout my education. Finally, I gratefully Hckno;rledge the fin8.ncial support provided by the Electronic and Electrical En&::i.neering Department, LOl1Chbo:couCh University of Technology, and the Post Office Research Establishment, which made it possible for me to carry out this work.,.

9 (iii) LIST OF ABBREVIATIONS AND SYMBOLS a(m,:"t) a A A/D ACQST C(m) CYHEQ C1 los Transform coefficient amplitude of order (m,n,t) Transform coefficient vector Transform coefficient array matrix Analogue-to-digital Acquisition Discrete Cosine transform mul tiplica tive constant Cycle request Complementary metal oxide semiconductor kth decision level of quantizer Discrete Cosine transform D/A DDC DPCH Digital-to-analogue Difference detection and correction Differential pulse code modulation DSPLY Display...,.. " lwch 1-D 2-D 3-D ll.(l,m,n) Device command signal from HP 2100A minicomputer One-dimension?,l Two-dimensional Three-dimensional Scquency difference for th th 1 sequency in ill block of... " nth line Distortion threshold for bit assignment Distortion threshold for coefficient selection e q E(m,n,t) E { J E'(m,n,t) Nean square quantization error Mean energy of transform coefficient a(m,n,t) Expected value of { } Energy estim?te of coefficient a(m,n,t)

10 llvj E'D f(x,y,z) f F g G h H Even field drive Picture element intensity of coordinates (x,y,z) Image vector Image matrix Forward transform operator ],'orl-iard transform matrix Inverse transform oper,c.tor Inverse transform matrix (also Hadamard matrix, Section 3.2.2) Truncate fractional part of I J, i Identity matrix of dimension N Distortion threshold constant for bit assignment Distortion threshold constant for coefficient selection Covariance functicn of irj3rre array Covariance matrix of image array Line drive max I 11S nmse pdf PCH R r k RDSTB R/W RAM Haximum value of the variables in I J l1emory select normalized mean square error Probability density function Pulse code modulation Haar transfcrm matrix (also Rate distortion function in Section ) kth reconstructi.on level of quantizer Read strobe signal Read/rite signal Random access memory

11 I v I S SDDC TRTC TRFC VADS XTB YTB Slant trqnsform matrix Sequency difference detection and correction Transfer data to computer Transfer data from computer Video Acquisition and Display System Horizontal time base Vertical timebase Eigcnvalues of covariance function of image array er 2 a Variance of transform coefficient Error detection threshold (u sibifies Vertical correlation Horizontal correlation group number) Handom variable of coordinates (x,y) to drive t\,odimensional l'larkov field Hatrix transpose I-la trix inverse Complex conjneate

12 (vi) CONTENTS PAGE ABSTRACT ACKNO'lILEDGEl'lENTS LIS'r- OF ABBREVIA'l'IONS AND SYNBOLS i ii iii CHAPTER I - INTRODUCTION 1.1 Introduction 1.2 Background 1.3 Organization of Thesis CHJl.PTEH II - VIDEO ACQUISITION MlD DISPLAY SYSTE1 l (VADfi) 2.1 Introduction 2.2 System Functions Data Transfer Video Acquisition Video Display 2.3 System Description Data ffran:;;fer Video Acquisition Video Display 2.4 Circuit Details Frame Store Control Circuitry and Data Routing Analogue-to-digital Conversion Digital-to-analogue Conversion Time base "'ave form Generation 2.5 System Input/output Arrangements 2.6 Use of the Video Acquisition and Display System 2.7 Conclusj,on - -, ' (,

13 (vii) PAGE CHAPTER III - HlAGE TnfJS}'ORNS 3.1 Introduction 3.2 General Representation 3.3 Types of Image Transforms Fourier Transform Hadamard Transform Haar Transform Karhunen-Loeve Transform Slant Transform Discrete Cosine Transform 3.4 Computational Algorithms 3.5 Statistical Analysis Statistical'l1ean and Variance B E;nej,'gy Distribution Probabi li ty Densi ty J'lode Is for Image 1ransfor:ns Choice of Transform CHAP,['ER IV - DETECTION AND CORHEC'rION OP TRAHSl'JISSION 90 ERRORS 4.1 Introduction 4.2 'l';to-dimensional Gaus s-harkov Random Fie Ids 4.3 Error Detection 4.4 Error Correction 4.5 Hesults 4.6 Note on Publication 4.7 Conclusion CHAPTEH V - DP,TA COl-IPRESSION 5.1 Introduction 107

14 (viii) PAGE 5.2 Coding Considerations Type of Transformtion Transform Block Shape Transform Block Size Quantization Strategy 5.3 Adaptive Coefficient Selection and Quantization A Review Threshold Sampling Zonal Sampling Zonal Coding Other Adaptive Coding Nethods 5.4 Basis for Improvement 5.5 Two-dimensional Analysis Transform Coefficient Energy Distribution and Block Quantization Adaptive Coefficient Selection Quantization Effects and 'l'no-dimension::l.l Adaptive 'fransform Coders 5.6 Three-dimensional Analysis Transform Coefficient Energy Distribution and Block Quantization ,nergy O:stimation anll. AdfCptive Coeficient 191 Selection Three-dimensional Ada?tive Transform Coding 198 System 5.7 Note en Publication 5.8 Conclusion

15 ( ix) PAGE CHAPTER VI - RECAPITULATION AND CLOSING CO1MENTS 6.1 Introduction 6.2 Recapitulation Detection and Correction of Transmis,;ion Errors Data Compression Closing Comments REFERENCES APPENDIX I - DEVICE SPECIFICA'IONS APPEKDIX II - CQ!.PUTER PROGRJJ I FOR TJ-,,;: VADS

16 1 CHAPTER I - INTRODUCTION 1.1 INTRODUCTION A fundamental problem in the design of image coding systems for digital communication links is that of a suitable coding strategy which "ill minimize the number of code symbols required to describe an image. Such a coding strategy must not cause degradation in the quality of the decoded image beyond c8rtain predetermined fidelity limits and, furthermore, the coding method ",ust not oe prohibi tively complex from the practical point of view, nor should. it be particularly sensitive to channel errors. r':uch 'W'o:t'lt bas (1 9 ) been carried out in search of such image coding systems' and the use of transform coding is one of the more effective methods of overcoming the problem. Such a coding system is capabltj of achieving a reasonably large bandwidth reduction and also offers some irr.munity to channel errors without sienificant image clezraclation. 'l'he in trodl.lc tion of the Fas t FoiITier transform algorithm ( ) led to the investi;sa tion of. the Fourier transfor-m image coding technique, \-!hereby the tho--dimcnsional 'ourier transform ef the image was transmitted over the channel rather than the imagc itself(15-17). This investigation subsequcntly let to the study of a related cod ins technique in which an image 'Ins transformed by a Hadamard matrix operator(hl-20)..

17 2 The Fourier and Hadamard transforms are only two of a large number of transforms that have potential applications for image coding. Investigations have also been carried out into the application of t he dlscre - t e Karhunen-Loeve (21-25) and Haar (26-29) transforms. The Karhunen-Loeve transform, also known as the Hotelling transform, achieves best minimum mean square error performance, but unfortunately, requires statistical knmlledge of the image source and does not possess a fast computational algorithm. On the other hand, the Haar transform has n extremely efficient computational algorithm, but usually results in relatively large coding errors. None of the transforms mentioned above, however, has been expressly tailored to the subjective characteristics of an image. Enomoto and Shibata(30) have reported orthogol>al transforms containing a "slant" basis vector for data of vector lengths of four and eight. The slant vector is a discrete sawtooth waveform 'Thich is sui table for representing Gradual brightness changes in an image line. Pratt, Chen and l-ielch(3 1 ) subsequently developed a Generalized Slant transform algorithm for larger size vectors and arrays. Aned(32) has shown that the discrete Cosine transform, which possesses a fast algorithm, approaches the efficiency of the Karhunen-Loeve transform for l-iarkoll process image data, and Jain (24) has suggested a Sine transform with similar properties. A major attribute of an image transform is the fact that the transform process compacts the image energy into only a few of the transform domain coefficients. A high degree of energy compaction will result if the basis vectors of the transfcrm matrix "resemble" typical horizontal or vertical lines of image. For

18 3 example, if a line of the image is of nearly constant intensi ty, then transforming this line will result in a large amplitude for the d.c. ooeffioient while the a.c. coefficient amplitudes will be relatively insignificant. Thus the transform process has concentrated most of the energy in the original image data into the d.c. coefficient. In general, gradual changes in intensity are often encountered in real images so that there is a tendency for the energy to be concentrated in the lower order transform coefficients. The transform also aims at reducing the degree of correlation wi thin the original picture and generates a. set of unco'related, or nearly uncorrelated, transform coefi'icients. This improves encoding ei'i'iciency by allowing transi'orm coefficients to be processed independently of each other. The best transformation would, therefore, be one that results in statistically indep<"ndent variables. reasons. HOHever, such a transi'orm cannot be determined for hlo First, it evidently depends on very detailed statistics which are not easily obtained. and, in ally case, would change i'rom picture to picture. Second, even ii' such statistics were known, the problem of determi.ning, in general, a reversible transformation that results in independent coei'i'icients is unsolved. The best that can be done wi.th lin(>8.r transform"tions t'l produce " transformation that generates independent coei'ficients is one that results in uncorrelated, but not necessarily statistically independent, coefficients.

19 4 1.2 BACKGHOUND At the start of the research programme, the lack of a working video processing system posed a severe restriction on the achievement of useful results, the major drawbacks being the absence of any real image source data and of any facility for observing the vj.sual results of data processed. It was therefore decided to design and develop a digital interface system which could be linked to the department's Hewlett Packard 2100A minicomp'.l,ter. Containing a fran;" store, it >!ould have the facility not only to store but also to display single or multiple frames. The major constraint i!l the design of this system was financial and, as more funds became available at later stages, re-appraisalg of the design eventually led to the flexibility of the final system. This necessity to design and build a nel digital video interfaco took on an urgent priority and subsequently affected the timc that could be spent on actual research. Notwithstanding all the obstacles and drawbacks, an effort to familiarize with the use of image transforms I;as initiated. The possible use of image transforms, in particular, the Hadamard transform, in the field of error detection and correction,las first investigated. This investigation was a natural progression of earlier...!ork(33-37) on error detection and. correction and the use of transform coding. Fenwick, et al(3 6,57) used an error protection code for the d.c. sequency in a Walsh transformed picture to detect transmission errors, and a planar prediction technique, similar to the one described by Thompson(3 8 ) in connection with DPCM, was applied to correct such errors.

20 5 Rovrever, the application of such a protection code increases the bit rate by 100/N % for a data block of N samples. This was felt to be unacceptable and unnecessary and an investigation was made into the possible exploitation of the characteristics of the transform coefficients of picture data, or simulated picture data, to achieve error detection and correction. The success of such a technique would then mean that it would not be necessary to increase the transmitted bit rate. 'rhe investigation provided experience vlith image transformations which "'as to prove invaluable at later sta&es of the research programme when data compression using transform coding techniques "'ere used. As facilities improved, attention ",as directed to>lards investigation of data compression techniques. By courtesy of the Post Office. Research Centre at Ha:c"tlesharn Heath, and the Technical University of Hanover, some picture data of real images were obtained. 'l'hcse pictures consisted of monochrome head-and=-shoulder scenes "hich' were typical of vie"rphone signals. Simulations of data compression techniques Here performed using these data, and with the completion of the digital video interface equipment, the processed images could be viehed. Informal visu.al as"essment proved a valuable tool in directing the path of the investigations that were being conducted. Reported work by a number of researchers formed the basis for further consideration, in particular, (39-41 ) on recursive quantization and the papers by 1'escher, et al by Gimlett(4 2 ) ruld Claire(43) on the use of activity indices fm' adaptive transform coding to achieve 10vI transmitted bit rates. Not only are these techniques

21 6 shown to be relatively straightforward but also they exhibit impressive results in achieving redundancy reduction. The object of the present investigation is to produce a system that will allow the use of as Iowa bit rate as possible provided a satisfactory fidelity limit is met. In addition, the algoritlm should be simple enough to allow economic implementation,.although this last aspect is not fully considered in a hardware sense but rather provides a longterm guideline to the problems of data compression. 1.3 ORGANIZATION OF THESIS Follm-ling the present introducto'y chapter, Chapter II describes the hardware digital Video Acquisition and Display System (VADS) that was built to facilitate subjective assessment of the processed video data. This chapter does not seek to explain the exa.ct constructional details of the system, but rather is a general guide to its operation. The inclusion of circuit details and tabulated listings of connections and components is intended only to asuist future users of,the system. Prior to describing the actual research carried out, Chapter III provides a summarized introduction to orthogonal functions and image transforms. In this chapter, orthogonal trane forma -t ions commoi"'lly used for image processing are described and their statistical properties are examined in order to establish a basis for transform coding. Having established the.foundations for the use of ortho{;onal transforms, Chapter IV examines a possible application in the detection and correction of transmission errors. Such an investigation

22 7 is fruitful in relation to the understanding of the statistical characteristics of image transformations. This chapter also examines the use of Gauss-Harkov random fields to represent real images. Such simulation techniques, in fact, provide a useful means of investigation and are often employed, especially in the context of theoretical comparisons of codng algorithm(44-46). Chapter V deals with the main theme of transform coding - data compression. The chapter starts by introducing the coding considerations that need to be taken into account when using image transforms for data compression. An understanding of such considerations is essential, however, this must not be seen as being restrictive in the pursuit of new ideas. A review of transform coding techniques that have recently been used is then presented and, following this, the simula tions on two-dimensi onal and three-dimensiona.l a.dapti ve transform coding \-Ihich have been performed in the course of the research programme are described and their results discussed. The effectiveness and drawbacks of the individual algorithms are examined, and the chapter concludes by draing attention to the better algorithms and the bit rates achieveable using them... The thesis concludes with Chapter VI which collates the ideas and work that has ueen performed. in the course of the research programme. Suggestions are also made as to the possiuuities for future work in the transform coding fie Id.

23 8 CHAPTER 11- VIDEO ACQUISITION AND SYSTEM (VADS) DISPLAY 2.1 IN'rRODUCTION The Video Acquisition and Display System (VADS) is a p8ripheral unit that may be interfaced to any general purpose minicomputer, in particular the Hewlett Packard 2100A minicomputer >!hich is available for video signal processing 'IOrk in the Department of Electronic and Electrical Engineering of I.oughborough University of 1'echnology. The system accepts video data from a camera and formats it lor PAPER TAPE REA DER PAPER TAPE PUNCH HP2100A MINICOMPUTER TELETYPE VADS ( MONlroR) FIGURE Video Signal Processing System

24 storage on magnetic tapes via one of two magnetic tape units interfaced to the HP2100A. It also allows visual assessment of processed images resulting from simulations using the acquired data. Video data from external sources stored on magnetic tapes may also be transcribed into a format compatible Hith the HP2100A to be used for simulation work. Such a facility is useful at later stages of research when standard scenes are used to provide a comparative assessment of the effectiveness of algorithms that have been simulated. Ji'igure' 2.1 is a diagram of the complete video signal processing facility. 2.2 SYS'rEl1 FUNCTIONS The Video Acquisition and Display System i.s capable of performing the follo'ding functions : 1. Data transfer between VlillS and minicomputer 2. Video acquisition 3. Video display Data Transfer Input and output data transfer between the VADS and the HP2100A minicomputer is carried out in blocks of bit Hords (bytes), The starting address of each block of data is soft",are assignable, and there are 256-blocks giving a total capacity of bytes for the frame s tore in the V /,DS. In this.,:joy random access to the frame store is possible. Data transfer is performed synchronously so that the rate of transfer is entirely within the control of the '".

25 10 minicomputer, and it obviates any synchronization problems Video ACQ11isition The analogue video signal from a television camera is low-pass filtered, sampled and converted to digital form at the appropriate instances depending on the picture format required. Since the amount of data that can be stored is limited by the capacity of the frame store, the following picture formats are available: ( a) 1 frame of 256 pels by 256 lines (b) (c) (d) 4 frames of 128 pels by 128 lines 16 frames of 64 pels by 64 lines 64 frames of 32 pels by 32 lines 'fhe picture to be stored is ah;ays extracteo. from the first field of every frame when 2:1 interlacing is originally used. This is because the video data is displayed by thg system in a non-interlaced, field repeated mode Video Display display The data source for video,may originate from either the camera or the frame: store. IL both cases the sizes of thc picturc displayed. are as specified in the previous section. Por the display of data from a camcra, the analogue video information is first converted into a digital form. This digital data is then reconverted back into an analogue signal and is subsequently displayed. Such a process ensures that the data from the camera which is displayed is of tile

26 11 same form as that from the frame store. This facility is provided primarily to enable correct selection and focussing of the required subject field. In the case of display of data from the frame store, the format of the picture is already predetermined by the acquisition process. However, a further facility is available in that this data may be displayed in fast, normal or slow motion. This enables subjective evaluation not only of normal temporal defects but also careful consideration of spatial defects re sui tin" from any encoding algorithm performed. The slow motion facility is obtained by multiple repeti tion of each picture frame before a succeeding frrune is displayed. All system functions described above are sofhlare controllable. Initiation of any function is exercised by the transmission of an operation codeword together with the initiation signal (flag) to the peripheral. Also included is a general system reset l!hich is performed after the completion of each system function. This prepares the VADS for the next system function. 'l'he system reset m2.y also be software activated. 2.3 SYSTEJ.1 DESCTlIFTImr Although a general system description is possible, however, in the light of the numerous functions the VADS is capable of performing, it is best to segment such a description in order to get a clear impression of what is happening. Hence the system description is

27 12 categorized under the three system functions, namely (a) Data transfer (b) Video Acquisition (c) Video Display "2.3.1 Data Transfer The scheme for video data transfer is shown in Figure 2.2. Initiation of this mode of operation involves the transmisdon from the computer to the VAnS an initiation signal (flag) called DEVICE --1 COl 1l1AND (DVCH), together with the block starting address and operation codeword. The instructions are interpreted by the control logic in the VADS 11hich then sets the respective enablo and select status lines, clock signals and registers to perform the operation. If data is transferred from the computer to the frame store in the VADS, the data is buffered into a data selector and demultiplexed into 8 parallel 8-bi t lords before being wri tten into the frame store. The other input to the data selector consists of data arriving from the AID converter which in this case is ignored. The process of demul tiplexing the data and wri tine; it into the fj.'2.rne store in parallel is necessary since the random access memories (RAN IS) which have been utilized in the frame s tore have cycle times which are not sufficiently short. As the data is transferred, a data counter and a memory address register (both 01 which are included in the control logic) are incremented. th continues until the 256 This process sample has been transferred after which 1 The overhead line indicates negative logic.

28 13 the VADS >Till automatically reset to be ready to exeoute the next operation. DATA. FROM AID CONY ERTER., DATA J,. COMPUTER -v SELECTOR! BUFFER, ;. DVCM, BLOCK \ ADDRESS, OPERATION CODEWORD CONTROL LOGIC -v DEMULTlPLE X ER --v - FRAt1E STORE MULTIPLEXER, ",. BUFFER FIGURE Data Transfer Scheme }'or the transfer of data from the frame store to the computer, sixteen data points are read at a time from the frame store. The control logic supervises multiplexing of the parallel data words and transfers the 8-bi t samples to the computer one at" a time. Again mu.1tiplexing is necessary to overcome the handicap of lone access time of the random access memories in the fram" store. th is completed after the transfer of the 256 is reset. 'l'he operation sample and the system In both data transfer operations the clock signal is derived from the DV CM flae so that the speed of transfer becomes entirely dependent

29 14 on the computer. This is useful since no synchronisation problems will be encountered, but more important data transfer may be accomplished as quickly as possible, short of direct memory access (DNA) rates Video Acquisition The scheme for video acquisition is shown in Figure 2.3. 'rhe camera is driven by synchronization signals derived from a pulse generator in the control logic. ' A variety of synchronization signals may be used depending on the requirements of the camera, and these include line, field and frame drive signals and mixed synchronizing signal. On initiation by the computer the control logic sends the appropriate convert. command signal to the AID converter. 'Phis command sign!'l is specified by the required picture fol'mat and is defined by the operation code>lord from the computer. The digital video data is demul tiplexed before being \'lri tten into the frame store. counters and memory address registers >lithin the contol Neanwhile, logic are DATA CAMERAr---lAID CONVERTERI----v1 SELECTOR I DEMULTIPLEXER FRAME STORE DVCM, OPERATION CODE>.JORD COMPUTER FIGURE Video Acquisition Scheme

30 15 incremented as the data is stored until the frame store is filled. When this happens an automatic system reset is executed, thus signifying the end of video acquisition and prepares the VADS for the next system function Video Display The video display scheme is shown in Figure 2.4. Data from either the camera or frame store may be displayed. Again all the necessary control and clock signals are generated within the control logic section /i th commands and initialization from the computer. A point worth noting is that, in order to achieve full system control from the VADS, the synchronization of the drive circuits for the caera and monitor are completely provided by the control logic. Incidentally, this obviates the necessity of locking the video signals from the camera to the peripheral or the peripheral to the moni t.or, though in the latter case the problem does not arise....>.. DATA. -" o [ CAMER AID CONVERTER --y OrA CONVERTER SELECTOR /, ".. ' CONTROL DEMLLTlPlEXER LOG!C 0, i' MONITOR COMPUTER frame STORE FIGURE Video Display Scheme

31 CIRCUIT DETAILS In order to describe the system hardware clearly, the detailed description of the system is divided into the following sections (a) (b) (c) (d) (e) Frame store Control circuitry and data routing Analogue-to-diGital conve-rsion Digital-to-analogue conversion Timebase waveform generation Frame store The frame store for the VADS is built from static 1024-uords by 1-bit random access memories. It is organized into 16 boards of 4096-uords by 8-bits per board, giving it a total capacity of bit words. Due to cost limitations and the ease of implementing static rather than dynamic devices, a compromise has been found in the use of 1024-\;ords by 1-bi tram's, specifically the Tfo1S4035!hich has a minimum cycle time of 1 us. The slo'"ncss of these devices thus necessitates data demultiplexing at the input and multiplexing at the output for real-time data transfer at vdeo rates between the frame store and any other section of the VADS Timing reauirements The cycle times for reading and. writing data into a memory board ef 4k x 8 bits is shown in Figure 2.5. Whenever the memory board is accessed the HEHORY SELECT status lines, }ls1 and J.lS2, must go high

32 17 READ CYCLE TIME le10ry SSLECT /Address Valid \'---- ADDRRSS )()()()( '--X= I I..., 1 crusq } / DATA ou'r I l-t _I read ),(r RDSTB t 600 ns read \'---_/ HEHOHY SELECT \ I I, t ----_I I wri te 1 I H/W ;( I /Address Valid """'""...,,-...,,---, I r--'---, ADDRESS ><><>OK,-' X= 1 _--I l'- I CYREQ = I I DATA IN ><XXX 1 t write 1350 ns, tda ta 600 ns I-t -I 1 d a ta, """",,,r:-r:-7'c7'c--";--";--";c-r.:--;- FIGUHE Head and Vlri te eycle times for l1emory Boards

33 1 8 (i.e. logical 1). The mode of operation, i.e. whether a read or a '<rite, is selected by the READ/I1RI'i'E (R/VI) status line. The initiation of any cycle is performed with the CYCLE REUEST (CYREQ) signal which consists of a negative-going pulse. The address to be read into the memory board must be available at or prior to the arrival of the negative-going edge of the CYREQ signal. For the read mode, the correct data will appear at the output data latch of the memory board after a minimum interval of about 600 ns. Receipt of the HEAD STROBE (RDS'l'B) signal will then latch out the required data to be routed to,,,herever necessary. In the write mode, the minimum 'Iri te cycle time (t ) is 'Iri te approximately 1350 ns. During this time the address and data are latched into buffers before being read into the RAH's. The data to be written into the memory must be available prior to the application of the CYREQ signal and it must remain stable for a minimwn period of approximately 600 ns after the application of the CYREQ signal to ensure a clean ;Iri te operation Circuit Diar;ram ---' The circuit diagram for a memory board. of 4k x 8 bi ts j.b shown in }'igure 2.6. The 74'116 latches buffers the data to be vritten into, and read from, the memory board. It can be seen that the i'is1 si ;nal acts as a CLEAR signal for the latches. HS2 enables the multivibrator. The address for the memory board is strcbed into the and flip-flops and the most significant two

34 19 :::I!O', i:; n 0 M - N n 0 N r ; M IT -_ Ht'!L c: ;;; -. - N,. M.- IJ M. I I I :=1 I :::1 I := I \ i S(u';SH I S[O'ISJl S(Q'I$Wl I- I SEO'lSNl / :::I, I I N :::1 I :=1 I := I '"I \ ::J I S(O'J$I-H t SEC'ISH! 5[0';$W1 I SEO'l$Wl :=1 L I I :=1 I :::II :::II I S(O'lSl I SE0'15Wl I- SEC'lsm I SEO'J$l ::IL I f- I I I :=1 I :::1 "'I "I I \ I SEO'1SWl I S(O'I$1-11 I- S(O'JS)'ll I SEO*;SWi I- \ X ).-!" -- f c' I I I.,1 N :=1 I ::1 "'I :::1 I I S(O'1SWl I StOt/SW! te SEO'iSHj I SEC'!S!-l! " :::II I- I- A ;;IP.' '" Vl.- -Q I I T "I I ::: 1 :::1 l \ r;;1 ;:::. ::: r I SE O'J$WJ tii I S(O':SIlJ2 SEDiSWl I SE Q'lSHl t2 ;:.. l- f- N N I t!. l: I I I _ N :::1, :::I ::: I - - l S(O'lSIl. I. SEO';SU_ SE O'/SI.Jl I - SEO'1$Wl '" l- f- I- I --y- '" - " =1" -:-:1 :::II I SEO'ISl1 tit- I SEO':$I-ll 5(0';$1-11 I 5(0' rl L, '="0 s, O -"'0 G!G Po "'-0 L., l Doe,,,I:= " SNid S(O'1$Wl SS l '1l. '., "'r'-"j" -- I --.c,o-nv' I:I M IT IT -, I-Cr-- > (LOL t- SLlH -- ':' "6"'6 "!,!, "!'TC!i"!' 6"'f6 FIGURE Circui t diagram of ) lemory lloa.rd ',.,

35 20 bits, i.e. FSAD10 and FSAD11, are decoded to select one of the four rows of RAW s arranged in the fashion shol-tn. Each row is capable of storing 1k x 8 bits. The necessary signals or pulses to latch in the data and addresses, and the write pulse for the RAM's are generated by a dual monostable multivibrator (74123) when the CYREQ signal is applied together Hi th the associated select R/W and enable ls2 signals. The components are mounted on a double sided printed circuit board and sufficient de coupling of the supply line is ensured to minimize any interference that may arise. The list of components used is tabulated in Table 2.1 and a list of the input/output connections to the memory board is given 'able 2.2. Item No. Description (Juan ti ty TMS035 Dual 4-bit latches with Clear 2 Octal D-type Flip-flop with Clear 1 Quadruple edge-triggered D-type Flip-flop 1 Dual 2-line to 4-line decoder/demultiplexer 1 Dual retriggerable monostable mul tivibra tor word by 1-bi t static random access...-"" memory 32 Resistors 15 kohms 1 10 kohms 1 1 kohms ohms 1 Capacitors 150 pf uf 17 TABLE Components list for 409b-words by 8-bits Memory Board

36 21 Track No. Function Track No. Function V 30 DI V 31 DI O 9 FSAD8 32 DI4 10 FSAD9 33 DI2 11 FSAD10 34 DI1 12 FSAD11 35 DI3 15 D03 47 R/W D01 48 CYREQ. 18 D02 49 FSAD7 19 D04 50 FSAD4 20 DO O 51 FSADO 21 D06 52 FSAD3 22 D05 53 FSAD1 2 DO? 55 FSAD6 25 RDSTll 56 FSAD2 26 MS1 57 FSAD5 28 DI7 59 V 29 DI5 60 V 45 11S2 TABLE Input/Output Connections for 4096-words by 8-bits Hemory Board

37 Control Circuitry and Data Routing Figure 2.7 illustrates the overall system showing the routing of digital control signals and data. On receipt of the required initiation signal (i.e. the DVCN flag) from the computer and any relevant clock signals from the clock circuitry, a system function is initialized following the decoding of the operation codeword which has been assigned. The clock circuitry produces all the required clock, select and enable signals necessary for the execution on the desired system function. Data is written into the frame store during the video acquisition mode or when tra.nsferring data from the computer to the fral'l8 store. Data is read from the frame store during the display mode or when transferring data from the frame store to the computer. In the course of data moving into or out of the frame store, counters are incremented, and the output of these counters are sent to the Address Decoder. In addition to the output from the counters, during the transfer mode between the computer and the VADS, the starting address of each block of data to be tr'ansferred is also taken into consideration in the Address Decoder. Thus from the Address Decoder we obtain the addresses, FSADO to FSAD11, for each memory board together with the individual cont ol signals for selecting each of the sixteen memory boards. The Operation Codeword Decoder also supplied information for routing data into and out of the frame store.!hen data is!ritten into the frame store, it may COlJle from one of two sources, namely, the AID

38 [ CAMERA AID CONVERTtt< V MULTIPLEXER j.. DATA J.. rv V '[ SELECTOR D/A CONVERTER 0 MONITOR. CONTROL LOGIC Contains: 1. Initiation Circuitry I 2.. Clock Circuitry CONTROL FRAME 3. Operation Codeword Dec oder LOGIC STORE 4. Counters, registers. 8. c omparators, S. Frame Store Address G en era tor i> COMPUTER f, -" DATA.A DEMUL,IPLEXER v SELECTOR N W FIGURE Schematic of VADS

39 24 converter or the computer. The Data Selector selects the required data under instruction from the Operation Codeword Decoder, after which it is demultiplexed before being written into the frame store. On the other hand, when data is read from the frame store, the rate at which it is read is determined by the Operation Codeword Decoder and Clock circuitry as previously explained, but it is then multiplexed and routed to either the D/A converter or to the computer. The above is a simple description of the control circuit and data routing sequence. In reality, complications arise due to the fact that in video acquisition and display modes, and especially in the latter, there are several operational options, thereby rcqlliring Cl, greater complexity in the generation of the clock, select and enable signals as well as in the addressing of the frame store. Finally the data to be displayed may come from the camer2- through the A/D converter, or from the frame store Timing Haveforms The timing,;aveforms for the various modes of operation are now described for each of the separate modes, i.e. (a) Data transfer from HP2100A to the VADS (b) Data transfer from the VADS to HP2100A (c) Video Acquisi tion (d) Video Display

40 Timing durin data transfer from HP2100A to the VADS The main timing >Taveforms during this operation are functionally (i.e. not to scale) shown in Figure 2.8. The operation starts >Tith the receipt of the DV CM flag from the computer which results in CLKA strobing the Operation Codeword into the VADS. Following decoding of the Operation Codeword, the system is initialized to execute data transfer, as can be seen by a. 10>T-to-high level transition of the RUN status. The DV CM flag from the HP2100A, minicomputer supervises the data transfer rate and most other clock waveforms are derived from it. CLKB and CYREQ are used to demultiplex the data in blocks of eight samples before >Triting them into the frame store using CYREQ1 and CYREQ2. Heanwhile as the data is accepted, a set of counters is incremented by CLK1. The outputs of these counters are checked and when 256 samples have been transferred the system automatically resets. This is seen by the RUN status returning to a low level Timing durin data transfer from the VADS to HP2100A The main timing waveforms for this mode of operation are shown in Figure 2.9. Aceep" ance of the operation code\wrd and ini tializa tion procedares are similar to the previous case. Data is requested from the frame store by CIEQ and when sufficient time has elapsed to allow the data to reach the output data latches on the memory boards, the data is read using RDSTB. CLK1 increments a set of counters as data is read and transferred to the computer. It is also used in

41 DVCM ClKA Jl I RUN ClKB CYREO.1 -UU o I, I I, I U L _ JLJ1L 11 CYRECl2 11 Is- - U I C1K UUUUU 1 _ n 11 CYREO. rl 1 N 0- FIGURE 2.8 Timing waveforms for data transfer from computer

42 DV CM ClKA RUN CYREO. o LJU I I ULJLJ --- L I ---lflflji N -..I RDSTB I ClK I InnJ--- ',F igure 2.9 Timing waveforms for data transfer to computer

43 28 generating the required clock signals to select the data coming from the sixteen memory boards of the frame store. The RUN status indica tes the period during,,hich the system is enabled to perform the required operation. After 256 samples have been transferred, a reset is performed by the system, thus preparing it for the next operation Timing during Video Acouisition Figure 2.10 shows the timing waveforms for the acquisition of a single picture frame of size 256 pals by 256 lines. CLKA latches the operation codeword into the system and, following its decoding, the acquisition process is initiated by the even field drive, EFD1. Appropriate masking of the unwanted sections of the original video signal is determined by the decoded operation codeword. In this case, the first 32 lines of the 313 lines of the first field are neglected and the required command signal, ADeON, to sa.mple and convert the analogue video signal into digital form to be stored in the frame store is generated accordingly from a set of counters and comparators. Hithin an active line the first 63 sampling periods are blanked prior to actual sampling of the vidgo signal. Only the first field of each frame is used in the sampling operation. For other acquisition odes, the procedure is similar except that the degree of blanking is different for each specific case. Because of the limited capacity of the frame store the option of acquiring successive frames will unavoidably result in sub-pictures of smaller spatial dimensions. The successful execution of this operation is signified by a system reset >fhen the frame store fills to capacit)'.

44 DVC>-1I,---- ClKA _----'._---- Ef01 RUN I i eo ldrvulju rl _ JLU ENCo."11 LDRV 6l t-=-===1 I I, DeON ---- (YRE(l I11 I I i 11 I 1 I 11 I I I I I11 I 1 I I11 11 I I (.. = SamiJling per:od) JJIJJLI ILLL II I Illll 1 Illll I I N '" (VREQl E,CO_C":':" FIGURE Timing Iaveforms for Video Acquisition.".

45 DVC:11!-!---- (LY, FO 1 RUN o ld.v JLJLJLJLJLJL JU--.lLJLJLULJ fij(or-:1 J1JuLnJ1_ ld" 1--=::: r-16 '-' l-lu5,-1,.j I LI -+ +-'-'l_-;-, w <:> fl;(qm I I I ENCDM' 1 I, T (YREQ I Ri'fS'Ts ij _ I ' r--- I Fis-ure Timing ',raveforms for Video Display

46 Timing during Video Dianlay ENCOM1 and ENCOl12 are the major command signals during video display. When ENCOM1 is high data is requestad from the frame store by CYREQ.. Su'bsequently, the data is latched out of the frame store by RDSTB and mul tiplexed to yield the correct data sequence. Umlanted portions of the frame are blanked out by ENCOH2. The data is originally taken from the first field of each frame and is displayed in a noninterlaced mode at a frame rate of 50 Hz. In addition, the system is capable of repeating each frame up to 128 times before displaying the next frame. In this way a slow motion effect can be generated which may assist in subjective evaluation of the video information. This flexibility is made possible by a sl ight increase in complexity of the frame store address decoder. Finally, the option of display of data straight from the camera, if required, is carried out by channelling the digital data from the AID converter to the DI A converter. The blanking of un'"anted portions of the original analogue video signal as specified by the operation codeword is identical to that process' in the case of video acquisi tion and on display, blanking is again performed by ENC0i Circuit Di,a :rams The Control and Data routing circuitry is contained in 8 boards and may be roughly grouped as follows : Board 1 Board 2 Data selector and latches for D/A converter Initiation and clock circuitry

47 32 Board 3 Address decoder (Part I) Board 4 Operation code\'lord decoder Address decoder (Part 11) Frame store data selector/demultiplexer (Part 1) Board 5 Board 6 Frame store data selector/demultiplexer (Part II) Counters and comparators Board 7 Frame store data selector/multiplexer (Part I) Board 8 Frame store data selector/multiplexer (Part Il) Board 1 The circuit on Board 1 is shown in Figure As can be seen it carries out a relatively straightforward selection process on the data which originates either from the frame store or from the AID converter. The data from the A/D converter is blanked so that in each field' thereis only a maximum of 256 pels of 256 active lines. Selection of the data is controlled by control signals 8 and DSPLY 7 which have been decoded from the operation code\wrd Board 2 The initiation and clock circuitry is contained in Board 2 and is shown in Figure All the required line, field, frame and mixed synchronization drive signals are generated using a type ZNA134 TV Synchronizing Pulse Generator. In addition, all necessary clock and control signals for the entire system are eenerated on this board. A number of these clock and control si cnals are buffered as they may drive as many as thirty normalized TTL loads.

48 33 At the start of each operation cycle, the computer flags the VADS by sending the DVCM signal together with the required operation codeword. DVCM sets one of the flip-flops of D1. This indicates to the VADS that it is required to perform an operation. 'l'he operation, is initiated either by the even field drive (EFD1) or the DVC!'j signal itself. The choice of EFD1 as an operation initiator synchronizes the execution of video acquisition or display. For the transfer of data bet\een the VADS and the HP2100A minicomputer all the clock si,,'nals are derived from the DV CH signal, including system initialization Board 3 The circuit of Board' 3 forms the main section of the Address Decoder. The addressing for the various acquisition and display modes are determined by the control signals S10 and S14 which select the required clock signals generated by the address counter, i.e. AllCllT10 to ADCNT22. The truth table for this selection is sho,m in 'able 2.3. A logical 1 for the ADCNT line indicates that the line is selected, whereas a logical 0 means that it is omitted. The implementation of these conditions is simplified by the use of an 8-line to i-line data selector/multiplexer, type During acquisition only the modes referring to logical 0 for S12, 813 ru1d S14 are used, these lines must therefore be set accordingly. The circuit diagram for Board 3: is shown in Figure Board 4 After being buffered the data from the computer is decoded to indicate

49 34 ADCNT s14 S13 S12 S11 S , ' ' , TABLE 'l'ruth table for the Address counter outputs

50 35 the required operation to be performed. Control signals S7 to S15, DSPLY, ACQ.S'f, TRTC and TRFC are produced by the Operation Code"ord Decoder. Data selectors/multiplexers C4 and C5 select the data from ei ther the computer or the A/D converter to be subsequently,,,ri tten into: the frame store. Finally, the outputs from the address counter, the address decoder (Part r) and the Y counter are selected to yield the required addresses for the frame store, Le. FSADO to FSAD11. All the above are shown clearly in Figure Board 5 Board 5 contains a set of flip-flops which demultiplcxes the data into the frame store. 'rhe circuit for this operation is shol>!ll in Figure C1KB shifts the data through 7 successive stages and at the end of the 8 th sampling period all the data is parallel loaded through another set of flip-flops by the CYREQ clock, \-[hence the data may be written into the appropri"te storage locations in the frame store Board 6 Board 6 contains all the counters and comparators which eventually generate the convert commal,d signal for the A/D converter, i.e. ADCON, and the clock signals for generating the required addresses for the frame store. The counters and comparators may be divided into three sets, (a) X counters and comparators which take into account the number of pels per line

51 36 (b) Y counter3 and comparators which account for the number of active lines to be considered (c) Address counters which generate the clock signals for the address decoder The blanking of unwanted pels and lines is performed by the comparators,,",here the outputs from the counters are compared with values Hhich are initially determined from the operation codeword sent from the computer. Control signals 810 and 811 determine the spatial dimensions of the picture. Table 2.4 indicates the manner in which the X comparators operate. For a given condition of 810 and 811, denoted by M (n = 0, 1, 2, 3) n the picture: size is specified and 8TAHT COUNT indicates the position of the first active pel on an active line and E;{D COUNT shols the start of blanking after the active pel sequence. Thus the base count for the comparators, namely X8 m (m"= 0, 1,, 15) and XEm (m = 0,1,, 15) are determined from 810 and 811. Vlith the exception of X8 4 to X8 7 and XE 4 to XE8 all other states are 10" Denoted by Picture size 8TART couwr END COUNT (X8 m ) (XE ) m 0 0 MO 256 x x M2 64 x N3 32 x TABLE Truth table for X comparators

52 37 From the truth table the following relations result, XS 4 = NO + M3 XE 4 = fol O + H3 XS 5 = H1 XE5 = M = S10 XS 6 = 11 XE6 = XS 7 = = S11 XE 7 = H XES = 110 For the Y comparators a similar situation exists with the exception that the START COUNT for the picture size of 256 pels x 256 Hnes is 32 instead of 16. Table 2.5 illustrates the conditions that apply. Again all states are low with the exception of YS 4 to YS 7 and YE 4 to YES. S10 S11 Denoted by Picture size START COUN'r END OOUNT (YS ) (YE ) m m 0 0 l'l x S x 12S H2 64 x x TABLE Truth table for Y comparators Thus for the Y comparators the following apply, YS 4 = 113 YE4 = N 3 YS 5 '" H1 = S11 YE 5 -- MO JoI 3 YS 6 = 11, YE 6 = 11 + M2 YS 7 = = S11 YE7 '" M1 YES = MO + H + M3 2

53 38 Incorporating these two sets of results and combining conditions that are similar results in the hardware representation shown in Figure Finally it should be mentioned that with a sampling frequency of NHz and taking into account the line synchronizing pulse leaves a total of 296 pels (to the nearest pel) for each line, whereas for each field, including the field drive blanking, there are only 306 lines to be considered lloa.rds 7 and 8 Boards 7 and 8 are identical in configuration, "here lloard 7 multiplexes the data from the frame store to yield the 4 least sienificant bits, and Board 8 performs the same operation to giye the 4 most signifioalt date. bits. The clock signals for multiplexing tho data from the frame store are ADDRESSO to ADDRESS3. The multiplexed data is then fed to a set of D-type flop-flops and the resulting output may either be'displayed or transferred to the computer as required. The circuit diagram is shom in }'igure 2.18.

54 ENCOM2 DSPLY S7 FSDAT ADC7 6 5 It FSDAT3 2 1 o ADC3 2 1 o '"' J 7 3 r-- 9 Ln "" ::; ""..,. 11 r-- 18 r ,'fl r-- er> j- 11 r DAT o CLK2 FICURE Circuit on Board 1

55 .1 e;;..j : ;::; 9 h L.- " " :; r 7 X -SV RI 170. AJ EPOl z 5 IUC 4 N r:-. FDl rl. A3 Sr 10, "4 RJ.'SV lk 10k -5V l 6 J Rl1 III RlJ!.7C -SV 01 1'1 r---@ 10 R14 15 FORvl fdrvl - LDRVl lijrvl -sv -sv P9 5 QVUl N "' DCl FDMYZ Ra --Cl. (1 R6.SV s V.. in R5 i,.7 lu S.2i< -SV 560 il...l) ) ;uji X(NT3 Rl J 2 EFD1 rl 9, J EN(OM2 r--;: 1,1 ENCOMi '" " Il. t---" 1 13 I1I r 1 M51-5V 515 TRUN 11 o o(i I Uj----/ _ ---1"> ;:- XeNTQ 2 :I. y(nto '((NTl 2 YCtlTzo---l 4 Y(NU- TRFC (f"ol (YREo. < C> M5YN RUN (YRm2 S9. 3 0( , MSYN 5 i33 6 _ (LK2 p 2 0 (lk2 y( T 3 o-_:,,::{:::::::=.ol!-..o yent X(NTJX(NT3 1," SS "-!R/W) CLK10, 5 Al L-osa ( RIW) 01 a:.=ztx302 51SS15 BJ CLK2 A (C 5 T e-----li i3 AO(NT16o- MSZ 59' vn;t8 MS2 D:;PLY ENCOM2 X(T3!2.-0 RDSTBl TR7[ TR.':o-----!Q 11 :loo " '"'RJS1B2 D L ORSTB E.N(O2, Y(ARO 11 TR'(<>-1 ' P2 \...6 DOo---il F1 8 5 TRUr-: fo 59-0 'I YCt-; i 0, 3 J Z DDR[SS....:. :?IGBE 2. -: 3 - Circuit on Eord 2

56 41 \l 1 d 3 i [ i lij B3 [BI '2 1, ".lnbb 9 9 I. 4 J :;; 1 1 ADC010 5 :; :; ADCOl1 r f " " n ::< 0> " SI. Cl ' " a 11 3 I ADCD l 3 N 2 2 [01-5 u. 1 1 :; n (.OC01? I "" :=> 0 H :< 14 - " 13 1 J ?c- C2 6 ' j (1 C1 (1 6 '5' 11 I., d H ro 0 ;t1,:: 0 +' d ::l () H.-< 0.q,'< 1..0(014 f 5 I 1111'0 9 J :;; ID II '" " " i t.rj( 01S

57 42 '. DF(15 5 v ijfc1:'.5 V DF(13.5 v R33 R17 R31,!l Tk'l I,.!" I.1 Rl& R31 "'5 R30 Tc.': OFe V 5 'i--t± OF{l1.5 V. DF(10 5 V orc? 5 V Off'.5 OFC? 5 V R14 TU9 1 T'-" R13 R28 1 T'--'. Rll R11 1,2 l...'. Rl1 RZ6 1,-!. V WO R15 R, Rl4 ),.!! '-". -t-cl Of(6 0--, 5 V om 5 V DFC V or( 3.5 v DF(2 5 V OFC 5 v R' RB 1.2 Te. R1 Rn 1.J -1' tj Rb Rl1 I 3,Cl.' R5 RIO R4 019 ),.'.0 T <.3 1o-K V R3 R15 1,J RI _J '-'- '-'- 0 0 " 0 0,. -, < Q 0 " 0 0 N " 1, 11 cl cl-, 11 3, a , , N ;!. " 9 1, ;;; " Rl )30 5V 1 11 ClKh l- s 1,29 13 j.! 1... N cj , " " 1 't 12 '0 5, e N w '3 ;G t..' " R2 - R17 82C Ohms R18-R33330otrnc; 1 5v... tt " N B ;!. 9 1,, " elkb P cs -'-' :;; :; N ': OSPI), 'r-<> ADCNT7o- f-l' I" 9 8 fll --0 '0- J? Fl 50-f L'!,0-i-1 ;! YCNT7o- f2.l 1 3,,-2, '3 '-' N f'-4: G 50-1,-' '0-f2 H,' e! ADCD1;:= I-'" 12 '2.!- ",2} 14...@ f--o 1)0-5 F4 1'3, L'! J..! :; 13 "' 5 ;!. ::;."., '-" 10 t1--1._bs.,!-o '5, 3... l..! fl.!-o,.j.'5 l-'!l2r- ',.!?... ",11 - C> et- <0 Cl Cl '-' J <{ <{ 1'5 "' TRF( TH( AcaST rsaoli FS I.. D10 F')AJ9 FS,t"D8 rsa7 FSAD6 rst..d5 FS:"02 *, 't fl.--o AOC7 'o--j!! t' !,- f-'., :; " 5!1-f--o w j.!!-o 1--0 '<j' 'd >' cl (;. P'1 so: 0 +'... FS/,D4 ;:: tl H '.-l D FSt.::J3 tr, '"!: SA!J, r l :'00 C- h f"!] f-' ' t,-- t!! Lli AOOo---!l 1 w 10---' ).15 f-'lo fl-o Il--<> P-<> DO

58 43 5V 5V 81F17, 5 J 1 o B2F FI J 1 1 o BI,FI o 01 Ba 19 lfj 15 " 9 " ",, 1 S 1 o 0- -p , " " P hl N " :;;,- -, N ;! N '" 11 I" N ;.' U " R2 180 k " :;; " B B 9 7 7, I, L 5 "' 1 J, f :. n 8 7 L J 1? 1 S ,, ' A " 11 El k c!:!- t? i2 9, a 7 4 J L s 13 1 J ::; 12 6 B 1-7 7, 4 4.:; 5 J ).,.--l , L J ' 5 1 ' Bl DJ 7!, S 12 '} i a 7,q13 B 7 L J 18 i " N 12 6 B N J " 1 f---l I, 1 i<11t , BJ 71., Dl 7U73 " 1 S l " " !. 13 E 7 L J I, [i L 15 " N " "., N." 8 6 ;! 9, 9 7 7, 1 6 L 4 N v 5 4 v S J J 1 J 1 -<J B8FJ7, 5 I, o B7F17, 5 L 1 'J B6Fl7 6 5 U5F!7 1 o o FIGURE Circui t on Boa,rd 5

59 Rutio. Ri.. c;v 1 WIT - I [, r)tjl r:-r " 1, --I _I RU " 13 ADEN YCARO C bj H c:: ";U i:':l N -.J 0), ti o " I-' 0!- 0_ Vi V; 11 -L...!. ll.j. L1 0,2 " ii 1, 1 1 III 1 '3 m 1 5 :0 10_., ;'::1 '" -"J 1:>1..- D ::J tjj D I'l ti P- O'> 6 9 m Hf 1 11

60 45 58 B16F ,... 19, u;, '" n r UJ '" '" 0 O... NfY) '" 22, ,.. u; 1...,., r-- 2 " 3,.,,, "\ 8 B16F B16F :: :: 23 0 U; 1...,... 2 r n '" ,5 t t r " 18, lq.-e U;,., 1'\... 1 r-- 2.n 3, '",... 7 " 8 B16FOO FSDATO De2 P-fi u; c r ,., DTCO 6 0 o... c- 8, , FIGURE Circuit on Board 7 & 8

61 46 Components list for Board 1 Description Quanti ty Quadruple 2-input data selector/multiplexer 2 Hex inverter 1 Triple 3-input NAND 1 Capacitors: 0.1 uf ceramic 1 Components list for Board 2 Description Quanti ty 7437 Q;uadruple 2-input NAND buffer Hex inverter Dual D-type flip-flop Quadruple 2-input NAND bit synchronous binary counter Triple 3-input NAND Dual retrieeereable monostable multivibrator Quadruple 2-input AND input NPJm Quadruple 2-input data selector/multiplexer Dual 4-input positive NAND gates Triple 3-input AND 1 ZUA 134J Synchronizi ng Pulse Generator 1 C HHz crystal oscillator 1 ZTX302 General purpose npn silicon transistors 4 Capacitors : 330 pf (foil) 1000 pf (ceramic) 1 uf (foil) 1 2 1

62 47 Components list for Board 2 (cont.) Resistors 330 ohms ohms kohms ohms ohms 2 75 ohms ohms 1 10 kohms 1 1 kohms kohms :3 Components list for Board 3 1U Description Quantity 8-line to 1-line data se"lector/rnul tiplexer 6 Quadruple 2-input AND 3 Hex inverter 1 Capacitors: 0.1 uf (ceramic) 3 Components list for Board 4 1U. Description Quanti ty Octal D-type flip-flop Quadruple 2-input NAND Quadruple 2-input NAND buffer Quadruple 2-input data selector/multiplexer Dual 2-line to 4-line decoder/demultiplexer 1

63 48 Components li t for Board 4 (cont.) Description Quantity 7404 Hex inverter Resistors: 330 ohms 820 ohms Capacitors: 0.1 uf (ceramic) !3 Components list for Board Description Octal D-type flip-flop Resistor: 180 ohms Capacitor: 0.1 uf (ceramic) Quantitv Components list for Board 6 Description Qua.nti ty Synchronous 4-bi ts binary counter 12 4-bits binary comparators 12 Dual 2-line to 4-line decoder/demultiplexer 1 Quadruple 2-input HAlm buffer 1 Triple 3-input EAND 1 Hex inverter 1 Resistor : 75 ohms Capacitor: 0.1 uf (ceramic) 1 10

64 49 Components list for Boards 7 nnd Descrintion Quantity 16-line to 1-line data selector/multiplexer 4 Dual D-type flip-flop 2 Quadruple 2-input NAND buffer 1 Hex inverter 1 Capacitor: 0.1 uf (ceramic) 2 TABLE List of components used in Control and Data Routing Circuitry

65 50 Board Track Function Track Function Track Function No. No. No. 1 A2 ADCO A3 ADC1 A4 ADC2 A5 ADC3 A6 ADC4 A1 ADC5 A8 ADC6 A9 ADC1 A10 F8DATO A11 F8DAT1 A12 F8DAT2 A13. }'8DAT3 A14 F8DAT4 A15 F8DAT5 A16 F8DAT6 A11 F8DAT1 A18 DATO A19 DAT1 A20 DAT2 A21 DAT3 A22 DAT4 A23 DAT5 A24 DAT6 A25 DAT1 A26 S1 A21 ENCOH1 A28 ENC0!12 A29 DSPLY A30 CLK2 2 A2 EFD1 A3 ElilD1 A4 FDRV1 A5 FDRV1 A6 LDRV1. A1 LDRV1 A8 }18YN A9 H8YN A10 CLK2 A11 CLK2 A12 ADCARO A13 YCAHO A14 88 A15 CLK1 A16 CLR1 A11 CLH2 A18 XCNTO A19 XCNT1 A20 XCN'f2 A21 XCNT3 A22 89 A23 :PC A " A25 NS2 A26 H81 A21 ADDRESSO A28 AJ)DRg8S1 A29!JJDRESS2 A30 ADDRE883 A31 Cyp'gQ B2 815 B3 DVCM B4 HUN B5 ChXB B6 CLKA B7 80 B8 ENC0l11 B9 ADCNT16 B10 S7 B11 87 B12 YCN'EO B13 YCNT1 B14 YCNT2 B15 YCNT3 B16 ACQ8T - B11 YCAR1 B19 RUN B20 ADCON B21 D8PLY B22 TRTC B23 S8 B24 RDSTB B25 HD8TB B26 CYHEQ1 B27 CYHEQ2 B28 HI" B29 H/'II B30 FDHV2 B31 ENCOH1 B32 ENCOl'12 A32 DC2 3 A2 S12 A3 813 A4 S14 A5 ADCNT10 A6 ADCNT11 A7 ADCNT12 A8 ADCNT13 A9 ADCNT14 A10 ADCNT15 A11 ADCNT16 A12 ADCNT17 A13 ADCNT18 A14 ADCNT19 A15 AJ)CNT20 A16 ADCN'E21

66 51 Board Track Function Track Function Track Funetion No. No. No. A17 ADCNT22 A18 ADCD10 A19 ADCD11 A20 ADCD12 A21 ADCD13 A22 ADCD14 A23 ADCD15 A A A2 DFCO A3 DFC1 A4 DFC2 A5 DFC3 A6 DFC4 A7. DFC5 A8 DFC6 A9 DFC7 A10 DFC8 A11 m'c9 A12 DFC10 A13 DFC11 A14 DF'C12 A15 DFC13 A16 DFC14 A17 DFC15 A18 ADCO A19 ADC1 A20 ADC2 A21 ADC3 A22 ADC4 A23 ADC5 A24 ADC6 A25 ADC7 A26 DO A27 D1 A28 D2 A29 D3 A30 D4 A31 D5 A32 D6 A33 D7 A34 YGNT4 A35 YCNT5 A36 YCNT6 A38 YCN'1'7 B2 ADCN'1'4 B3 ADCNT5 B4 ADCNT6 B5 ADCNT7 B6 ADCNT8 B,( ADCNT9 B8 ADCD10 B9 ADCD11 B10 ADCD12 B11 ADCD13 B12 ADCD14 B13 ADCD15 B14 F8ADO B15 'SAD1 B16 FSAD2 B17 FSAD3 B18 FSAD4 B19 }'SAD5 B20 FSAD6 B21 FSAD7 B22 }'SAD8 B23 FSAD9 B24 FSAD10 B25 FSAD11 B26 S8 B27 S9 B28 S10 B29 S11 B30 S12 B31 S13 B32 S14 B34 CLKA B35 CLKB B36 S15 B38 DSPLY B39 ACQST B40 'l'rtc B41 TRFC B A2 B1FIO A3 BiFI1 A4 B1FI2 A5 B1FI3 A6 B1FI4 A7 B1FI5 A8 B1FI6 A9 B1FI7 A10 B2FIO A11 B2FI1 A12 B2FI2 A1, B2FI3 A14 B2FI4 A15 B2FI5 A16 B2FI6 A17 B2FI7 A18 B3FIO A19 B3FI1 A20 B3FI2 A21 B3FI3 A22 B3FI4

67 52 Board Track Function Track Function Track Function No. No. No. 5 A23 B3FI5 A24 B3FI6 A25 E3}'I7 A26 B4FIO 1.27 B4FI1 1>28 E4FI2 A29 E4FI3 A30 B4FI4 A31 B4FI5 A32 B4FI6 A33 B4]o'I7 A34 B5FIO A35 B5FI1 A36 B5FI2 A38 B5FI3 A39 B5FI4 MO B5FI5 M1 B5FI6 M2 B5FI7 B2 B6FIO B3 B6FI1 B4 B6FI2 B5 B6FI3 B6 E6FI4 B7 B6FI5 B8 B6FI6 B9 B6FI7 E10 B7FIO B11 B7}'l1 B12 B7FI2 B13 B7FI3 B14 B7FI4 B15 B7l'I5 B16 B7FI6 B17 B7FI7 B"18 B8FIO B19 B8FI1 B20 B8FI2 B21 B8FI3 B22 B8FI4 B23 B8FI5 B24 B8FI6 B25 B8FI7 B26 DO B27 D1 B28 D2 E29 D3 B30 D4 B31 D5 B32 D6 B33 D7 B34 C J,ICB B35 CYREQ. 6 A2 YCHTO A4 YCNT1 A6 YCN'f2 A8 YCNT3 A10 YCNT4 " A12 YCNT5 A14 YCNT6 A16 YCNT7 A18 YCAIW A20 YCNT8 A22 ADCARO A24 ADCNT16 A26 S8 A28 ENCOJ11 A30 XCH'l'O A32 XCNT1 A34 XCNT2 A36 XCNT3 B2 RUN B3 RUN B4 CIB1 B5 CLK1 B6 CLH2 B7 CLK2 B8 RUN B9 CLK2 B10 S10 B11 S11 B12 ADCNTO :313 ADCH'l.'1 B14 ADCNT2 Bi5 ADCNT3 B16 ADCNT4 Bi7 ADCNT5 Bi8 ADCNT6 B19 A11CNT7 B20 ADCNT8 B2i ADCNT9 B22 ADCNTi0 B23 1.DCNT11 B24 ADCNT12 B25 ADCNT13 B26 1.DCNTi4 B27 ADCNT15 B28 ADCHT16 B29 ADCNT17 B30 ADCNT18 B31 ADCNT19 B32 ADCNT20 B33 ADCN21 B34 ADCNT22 B35 AJ)EN

68 53 Board Track }'unction Track }unction Track Function No. li o. No. 1 A2 B9FOO A3 B10FOO A4 B11FOO A5 B12FOO A6 B13FOO A7 B14FOO A8 B15}'OO A9 B16FOO A10 B9F01 A11 B10F01 A12 B11 F01 A13 B12F01 A14 B13F01 A15 B14F01 A 16 B15F01 A17 B16F01 A18 B9F02 A19 B10F02 A20 B11F02 A21 B12F02 A22 B13P02 A23 B14F02 A24 B15F02 A25 B16]' B9}' B10F03 A28 B11F03 A29 B12}'03 A30 B13F04 A31 B14F03 A32 B15F03 A33 B16}'03 A34 ADDRESSO 1.35 ADDRESS1 A36. ADDRESS2' A;8 ADDRESS3 A39 FSDA'fO 1140 FSDAT1 A41 FSDAT FSDAT3 B2 B1FOO B3 B2FOO B4 B3}'OO B5 B4POO B6 B5FOO B7 B6FOO B8 B7FOO B9 B8FOO Jl10 B1F01 B11 B2F01 B12 B3}'01 B13 B4F01 B14 B5F01 B15 B6F01 B16 B7F01 B17 B8F01 B18 B1F02 B19 B2F02 B20 B3F02. B21 B4F'02 B22 B5F02 B23 B6F02 B24 B7F02 B25 B8F02 B26 BH'03 B27 B2F03 B28 B3F03 B29 B4F03 B30 B5F03 B31 B6}'03 B32 B7F03 B33 BB}'03 B34 D'l'CO B35 DTC1 B36 DTC2 B38 DTC3 B39 S8 B40 DC2 8 A2 B9F04 A3 B10F B11F04 A5 B12F04 A6 B13F04 A7 Jl14F04 A8 B15F04 A9 B16F04 A10 B9F05 A11 B1Q}'05 A12 B11F05 A13 B12F05 A14 B13F05 A15 B14F05 A16 B15l'05 A17 B16F05 A18 B9F06 A19 B10F06 A20 B1F06 A21 B12F B13F06 A23 B14F06 A24 B15F06 A25 B16F06 A26 B9}'(l7 A27 B10F07 A28 Jl11FO'! A29 B12F07 A30 B13F07 A31 B14F07 ';..

69 54 Board Track Function Track Function Track Function No. No. No. 8 A32 B15F07 A33 B16F07 A34 ADDRESSO" A35 ADDRESS1 A36 ADDRESS2 A38 ADDRESS3 A39 FSDAT4 MO FSDAT5 A41 FSDAT6 A42 ji'sda'r7 B2 B1F04 B3 B2F04 B4 B3F04 B5 B4,'04 B6 B5F04 B7 B6F04 B8 B7F04 B9 B8F04 B10 B1F05 B11 B2F05 B12 B3F05 B13 B4F05 B14 B5l"05 B15 B6F05 B16 B7l"05 B17 B8F05 B18 BH'06 B19 B2F06 B20 B3l"o6 B21 B4F06 B22 B5F06 B23 B6l"06 B24 B7F06 B25 B8F06 B26 B1F07 B27 D2]i'07 B28 B3F07 B29 B4F07 B30 B51'o7 B31 B6l"07 B32 B7l"07 B33 B8F07 B34 DTC4 B35 lltc5 B36 lltc6 B38 DTC7 B39 S8 B40 llc2 TABLE List of input/output connections to Boards1-8.-"

70 ss Analoljue-to-dip,"ital Conversion In the VADS the conversion of analogue video information from a -camera to digital data to be stored in the frame store is simplified by the use of the TDC1007PCB Video AID converter Evaluation Board from TR1 /. This device meets the intended requirements and its details are included in Appendix I. A few important and relevant points regarding its usage are: (a) \{ith regard to the analogue input range, the value of the resistors R1 and R2 have been chosen as 39 ohms for operation over an input range of 2V with an input impedance of 75 ohms. (b) Every possible effort has been made to ensure separate analogue and digital earth returns as specified 8.S a precauti.on by the manufacturer. There i.s only one common point at the earth terminal for the entire VADS system. (c) 'raok A, be ing ini tially unassigned, has been connected to the VII' input of the TDC1007.J to assis t, in calibration for any incoming video signal so as to ensure full use of its dt.amic range. It also obviates the use of an extender board Hhich may be subject to stray capacitance and susceptible to unhanted in terference. (d) All leads carrying analogue information are screened and kept as short as possible to reduce interferece and noise. (e) The output coding of the digital data is straight binary Hhich is compatible Hith that of the digital-to-analogue converter (see specification of device in Appendix I)

71 Digital-to-analor:ue Conversion The design of the digital-to-analogue converter is simplified by the use of a module, i.e. the DAC-FI8 supplied by Datel. Figure 2.19 shows the circuit implementation of the device. The only other external device required is a wide bandwidth operational amplifier which is used to buffer the output from the module. Resistora R1 and R2 adjust the offset and range of the resultant decoded analogue signal. A list of the components used on the digital-to-analogue conversion board is tabulated in Table 2.8 while Table 2.9 lists the input/output connections to the board. Specifica.tions for the D/A converter module is also included in Appendix I for reference purposes ' ;2'-1 o ,3'-i '-I o ''-l 6 D20---"-l '-i DO o-_s"-1 (1 r 10u ,1 I OAC- F I S (2 I 10u I""" R 1 t ( 3 A !.. 41sT 22p R3 1-'1-=-5---< 3'-l ; _ >.::...q-{::::r--0analdgue 13 -" R2 r=- - OUTPUT k FIGURE Digital-to-analogue Conversion Circuit

72 57 DAC-FIB 1434 Description D/A converter module lifideband operational amplifier Potentiometers : 500 ohms 2.2 kohms Pua.nti t;v Resistor Capacitors 75 ohms 10 uf (electrolytic) 22 pf (ceramic) TABLE 2.B - Components used in D/A Conversion Board Track No._ lilunction 6 7 Analogue Gro'.md B 11 AnaloGUe output 16-15V 18 DATO 19 DAT1 20 DAT2 21 DAT3 22 DA'f4 25 DAT5 26 DAT6 27 DAT V TABLE Input/output connections for D/ A Conversion Board

73 Timebase ':Iaveform Generatj.on Unlike a conventional TV monitor, the HP1330A X-Y display that has been incorporated in the VADS has provision for sepaate x, y and z inputs. As a result the line and field (or frame) scan has to be supplied, in addition to the analogue video signal which is fed to the z-input. A timebase waveform generation circuit in /hich s,eep linearity is achieved by exponential charging of a capacitor has been used Exnonential Sweep Circuit The exponential Sleep circuit is sho\m in Figure At t = 0 the swi tch S is opened and the sheep voltage v s is given by v s (2.1) "" FIGURE Exponential Sweep Circuit After an interval T s ' lhen Vs' the switch is closed. the sweep amplitude attains the value Assuming zero switch resistance, the resultant sweep waveform is lndica.ted in Fl.gure 2.21 a. If tire «1 expa.nding the exponential in Eqn. 2.1 gives

74 59 t 2RC + (2.2) v :;;10>-.,..... I / / ;' '" ( a) - Time - Time FIGUHE Exponential '1'leep Waveforms Since v s = V when t = T s' to a first approximation s V s T =.-!l RC V so that an approximately linear sweep is obtained provided T!HC<<.1. s The displacement error illustrated in Figure 2.21b is given by (v - v,) -,s"---,-,s,,---- max V s which serves as a useful criterion to check the linearity of the resul ting s1"eep voltage.

75 Circuit Implementation The timebase waveform generation circuit is shown in Figure The CI lqs 4016 Quad bilateral switch serves as a us.eful device in charging and discharging the capacitors. When operated with a supply voltage range of OV to 15V it has a typical turn-on resistance of 80 ohms and when switched off current leakage is typically 0.1 na. The requirement for a high input impedance operational amplifier is satisfied by the los/}'et operational amplifier which has an input impedance of typically ohms. Maximum possible linearity is achieved by ensuring that the maximum s\ifeep voltage Vs is only a small fraction of the total voltage V. The list of components used is tabulated in Table The displacement error for the line and field timebase are x x 10-3 V edline = x = x x 10-3 edfield = x = 9.88 x 10-4 V x which are both insignificant. In addition, V is 0.155V for the s_ line timeb9.se and O.119V for the field timebase, and they are amplified to the required range of 1V peak-to-peak for the x- and y-inputs of the X-Y display by adjusting the potentiometers R4 and R8. The components are laid out on a printed circuit board which has an extensive ground-ple to reduce noise and interference. }alogue and digital earth returns are. also kept separate as a further

76 61 precaution. The track configuration for the various input/output connections of the printed circuit board is tabulated in Table Description Quantity S HOS/FET operational amplifier 2 High slel-/-rate operational amplifier 1 Cl10S quad bilateral switch 1 Hex inverter 1 Resistors : 1 kohms 10 kohms 100 kohms 10 Hohms Capac'i tors 560 pf 22 uf.22 uf.1 uf TABLE Components for Timebase Ivaveform Genera ti.on Circuit Track No. Function 8 XTB V 12 YTB 14-15V 16 Analogue Ground 18 Analogue Ground 24 Digital Ground 26 FDRVl 28 LDRVl TABLE Input/output connections for Timebase Board

77 +15V R1 R2 ":l '-! 1k 10M <;1 fa t':l N N N,.,. '-3 s ldrv1 1 2 (1 T560P <1> 0- Il> +15V ril <1> 4 1 (2 22u I OXTB R3 'R4 10k 100k.; R10 III <: R5 R6 10k et>..., 1k 10M R10 0 t; + 10k () 3 4 ( S YTB I t; FDRV1 () O.22u (4.l io... 22u:t: <, R7 R8 10k 100k J. -15V 10k +1SV a- N

78 '.., "... (a) Board 1 (e) Board 5 "r i "I (b) Board 2 (f) Board 6 1, :1 I (e) Board 3 (g) Board 7 (d) Board 4 (h) Memory Board

79 (i) DAC Board (j) Timebase Board (k) Video Acquisi tion & Display System FIGURE 2.23

80 SYSTEI1 INPUT/OUTPUT ARRANGEN!<:WPS The Video Acquisition and Display System hard"are is contained in a metal cabinet 551 mm wide, 590 mm deep and mm high. Because of the high current consumption and the necessity for efficient heat dissipation, a fan unit has also been incorporated. The system is linked to the computer by two cables, each terlilinating in 25-way connectors. The first link is labelled CONNECTOn 1 OUTPUT for data transfer from the VADS to the IlP2100A minicomputer. The second link, labelled as CONNEc'rOR 2 INPUT, is for data input to the VADS from the RP2100A. Table 2.12 lists the pin functions in the two 25-\,ay connectors. In addition, the system also has provision for connections of the following (a) LINE DRIVE (b) (c) (d) FIELD DRIVE ]<'RANE muve MIXED SYHCIillONIZATION DRIVE (e) ANALOGUE VIDEO INPUT (r) ANALOGUE VIDEO OUTPUT.- The.first four d:-ive sienals permit synchronization of camera ard. an external television monitor to the system. The drive signals are TTL compatible. The analogue input and output connections are matched to 75 ohms impedance. The video output signal if; a true video signal and not a co mposi te signal COIl taining synchroniza tioil pulsen.

81 66 Connector Pin No. Colour Function 1 1 White & red DTCO. 2 Brown DTC1 3 Green DTC2 4 Red DTC3 5 Blue & black DTC4 6 Blue & red DTC5 7 Orange DTC6 8 Grey DTC7 2 1 Black DFCO 2 Brown & black DFC1 3 Red & black DFC2 4 Orange & black DFC3 5 Yellow & black DFC4 6 Green & black DFC5 7 Blue & black DFC6 e Purple & red D}'C7 9 Grey & red DFC8 10 \ lhi te & red DFC9 11 Black & red DFC10 12 Brown & red DFC11 13 Red DFC12 14 Blue & red DFC13 15 Green & red DFC14 16 Yellow & red DFC15 17 Orange & red DVCI1 TABLE Input/output Digita.l connections for the VADS

82 USE OP THE VIDEO ACQUISITION AND DISPLAY SYS'rEH The Video Acquisition and Display system is relatively simple to use. All system functions are software controlled so that the user has total control of the system via a keyboard (or an appropriate input/output terminal) interfaced to the I2100A minicomputer. Each system function is defined by a 16-bit operation codeword. The various operations and their associated codewords are tabulated in Table 2.13 and are explained below: (a) Por displ'ay and acquisition modes, bits 0 to 7 are unused and set to logical 0 with the exception of bit 7, which indicates the source of the data for display, i.e. either the frame store or a camera. (b) Par data transfer modes between the computer and the ystem, bits 0 to 7 indicate the address of the data block to or from which da+'a is transferred. (c) Bits 8 and 9 indicate the major modes of operation, namely (i) display (ii) acquisition (iii) data transfer to computer (iv) data transfer from computer --- (d) Bits 10 and 11 indicate the picture format during display or acquisition. These are unused during da+'a transfer. (e) Bits 12 to 14 indicate the number of repetitions of each frame before succeeding frames are displayed. The number of these repetitions is always a power of 2. A total of eight repeti-';;ion rates is available. (f) Bit 15 is used to reset the system, and may be issued at any time since it overides all other operations.

83 68 status Bit Function 0 Bit 0 1 Bit 1 2 Bit 2 3 Bit. 3 8 most signilicant bits 01 address 4 Bit 4 for lrame store during transler mode 5 Bit 5 6 Bit Bit (also used to determine display source) Display Acquisition Transfer data to computer Transfer d"ata lrom computer 256 x 256 x 1 picture format 128 x 128 x 4 picture format 64 x 64 x 16 picture format 32 x 32 x 64 pic'ture format No repetition (fast mod.e) Repetition x 2 (normal mode) Repetition x 4 Repetition x 8 Repetition x 16 Repetition x 32 Hepeti tion x 64 Repetition x System reset TABLE Operat ion Code'ford S ta tus

84 69 A listing of the program which controls the VADS is given in Appendix II. 2.7 CONCLUSION The Video Acquisition and Display System is a vital component in video signal processing. Its design and construction is intended \ to provide as much flexibility in use as cost limitations allow. The fact that all system functions are software controlled means greater user flexibility wi thou t recourse to manual control Hhich may be cumbersome and fault-sensitive. Vlith such a system sub,jective assessment of processed video data may be carried out which is a necessary feature in directing the path of subsequent investigations in digital video processing.

85 70 CHAPTER III - IMAGE TRANSFORMS 3.1 INTRODUCTION Figure 3.1 shows a block diagram of a generalized adaptive transform coding system. For the t>lo-dimensional case, an orieinal digital imag'e element (pel) is denoted by f(x,y) where x and y are the horizontal and vertical spatial co-ordinates respectively. These samples undergo a transformation either over the entire image or over sub-sections of the image called blocks. The resultant trn.l1sform coefficients, denoted by a(m,n), are then adaptively selected and quanti zed according to a set of conditions normally dependerlt on the statistics of the original image data. Those samples that are transmitted are decoded at the receiver and inversely transformed to form the reconstructed image r(x,y). f(x,yj ORIGINA L FORWARD TRANSFORM ADAPTIVE COEFFICIENT SELECTION & OUANTlZATlON BINARY ENCODER f(x,y J RECON ST RUCTED INVERSE TRANSFORM BINARY DECODER - CHANNEL FIGUHE A generalized adapti Ye.transform coding' Gys tern

86 GElmRAL REPRBSENTATION The general one-dimensional tra.nsform maps the image vector f(x) into a set of transform coefficients a(m) according to the equation a(m) = M-1 L f(x) g(x,m) x=o where, g(x,m) is the forward transform kernel and m = 0, 1,, H-1. This process is illustrated belo'l. DATA VECTOR TRANSFORM COEFFICIENT VECTOR f(o) f1) f(m-2) f(m-1) o() o)...0(1:\-2) a(-1) ! FIGURE GBnBralized one-dimensional trsformation Similarly, thb inverse transform is giver. by H-1 f(x) = I a(m) h(x,m) (3.2) m=o where h(x,m) is the inverse transform kernbl and x.. 0, 1, 1"-1. In the two-didlensional case the for;rard and :cnvers" iransfo:t'ids are given by the equations a(m,n) = L 11-1 N-1 L f(x,y) g(x,y,m,n) x=o y=o 11-1 N-1 f(x,y) = I I R(m,n) h(x,y,m,n) m=o n=o

87 72 where, as above, g(x,y,m,n) and h(x,y,m,n) are the forward and inverse transformation kernels respectively. The two-dimensional transformation is shown in Figure 3.3. fw,o) f(m-1,0) 0(0.0) a(m-1,0) c 0 0 TRANSFO fw, -1) f(m-1 N-1) aw,n-1) a(1-1,n-1) ' }'IGURE Generalized hlo-dimensional trancformation o '1'he for'<mrd kernel is said to be separable if g(x,y,m,n) = gl(x,m) g2(y,n) In addition, it is symmetric if gl is functionally equal to 52'.:hence g(x,y,m,n) = gl(x,m) gl(y,n) (3.6) A hlo-dimensional transform "i th a separable kernel can be computerl in hlo steps, each requiring a one-dimensional transform. First, the one-dimensional transform is taken along each row (i.e. y constant) of f(x,y) yielding T(y,m) H-1 ( ) = ): f x,y x,.,o

88 73 for y = 0, 1,, N-1. Next, a one-dimensional transform is taken along each column of T(y,m) (i.e. Iil constant) resulting in the expression a(m,n) = N-1 L T(y,m) B'2(y,n) y=o (3.8) The same final results are obtained if the transi'orm is taken along each column i'irst to obtain T(x,n) and then along each row to obtain a(m,n). Similar coffililents hold for the inverse transform if h(x,y,m,n) is separable. In the three-dimensional case for an image array f( x, y. z) "here. x, y and z are the horizontal, vertical and temporal co-ordinates the for"/a'd and inverse transforms are given by the equations H-1 N-1 T-1 a(m,n,t) = L L L f(x,y,z) &1(x,m) g2(y,n) g,(z,t) (3.9) ) x=o y=o z=o H-1 N-1 T-1 f(x,y,z) = L L l a(m,n,t) h1 (x.m) h 2 (y,n) h 3 (z,t) (3.10) m=o n=o t=o It is often useful to express transforms in vector-space or matrix form(4?). Let F and! denote the matrix and vector forms of the image array, and let A and a be the matrix and vector i'ordls of the transformed image. Then the transform >:ritten in vector form is given by a=gf (3.11) where. G is the forward transform matrix. The reverse transform is j=ha (3.12)

89 74,qhere H is the reverse transform matrix. Hence, If G is a unitary matrix, then by definition (3.14) where G* is the complex conjugate matrix of G and ",here G T is the matrix transpose of G. A real, unitary matrix is called an orthoconal matrix. For such a matrix In addition, if G is a symmetric orthogonal matrix, which is often the case for commonly used image transforms, then (3.16) If the transform kernel is separable such that lhere GC and G R are unitary matrices representing' the column and row transformations, then the transformed image matrix, A, can be obtained from the image matrix, F, by (3.18)

90 75 The inverse transformation is given by ( 3.19) where JI e -1 = Gc and}lu 3.3 TYPES OF nip-gb TRAHSFORI'-1S FOl<ricr Transform The discrete Fourier transform, with and >lithout efficient computntional als'ori thms, has long been used for signal 1. (10-14,48,49) ana YSlS, and has been pre-eminent in the application of transforms to ililage COding( 15-17, 50-55). The two-dimensional Fourier transform of,m image f(x,y) may be expressed in series form as a(m,n) 1 N-1 N-1 = N L L f(x,y) x=o y=o and its inverse as f(x,y) 1 N-1 N-1 = N L L a(m,n) exp [2i (mx + ny)] m=o n=o 0.21) where i = F1. Since the Fourier transform kernel is separable, both the two- and three-dimensional transform may be com9uted in sequential one-dimensional transform steps. The basis functions (or operator kernels) of the transforms, being complex exponentials, may be decomp(1sed into sine and cosine components. Figure 3.4 contains plots of the sine and cosine components of the onedimensional Fourier basis functions for N = 16.

91 7h WAVE NUMBER , 2 c::::'::::l,-tr='-l-l,--,-- --c:::;:::;::j c:;:::;j D Cl,...JL, 3 L..r' D L;J OL_ o 0 u-u 5 D..L:L-.O n n n 6 l.j-----u U U IO II r='-l c='-j c:;:::;j LW=' 15 lllullll!ljllllt o WAVE NUMBER o Cl =====:=1 ' : ::!od - er uld=tr--r=tp-cp 7 8 cr::rfl::rftrft ,.,Jt1 10 DOn D_D-D DUO U O---U- II 12 0 n D---.lJ o 0 0 U c:;:::;j Cl c:;:::;j r'l 15r-:-, IIIIIIIIIIIIIL_LU o Ibl 1 FIGlffiE P'ourier basis functions (a) sine component (b) cosine component Hadamard Transform The Hadamard transform (56) is based on the Hada.mard ma tr ix, \1hich is a square array of plus and minus ones whose rohs and columl1l:: are orthogonal. l'his transform is convenient to use becd.use of its computational simplicity - the generation of the transform coefficients or sequencies (Sequency is a term denoting the number of zero crossincs per basis function) only involves additions and subtractions. A normalized N x N Hadamard matrix, H, satisfies the rela tion, Acknowledgement - Figures are from Ref. 45

92 77 T' IT H = I C3. 22) Where I is the identity matrix. The lowest order Hadamard matrix is 1 [1 1] H2 - fi 1-1 and the construction of a Hadamard matrix of order n can be obtained by the follo\"ing recursive relationship wave NUMBER o FIGURE 3.5' - Hadamard transform basis functions for N = tci-t-t-i-i-t -'-j-t-t-' _-t-rho 4 j B "12 16

93 78 where; N = 2 n and n is an integer. Since the Hadamard matrix is real, symmetric and orthonormal, the forward and inverse transforms can be expressed as A = H F H F = H A H Figure 3.5 contains plots of the Hadamard transform basis waveforms of order Haar Transform The Haar transform(27) is based on the. Haar matrix,thich is a square array of plus and minus ones ;;.nd zero elements. A normalized Haar matrix of order 8 x 8 is shown belm; V2 V2 -{2 -:-V , 1 R8 =.rs tl tl -(Z -Il / (3.27) Based on this pattern the Haar matrices of order N (=2 n ; n = 1, 2, 3, ) can easily be constructed. Haar matrices are also real and orthonormal.

94 79 The two-dimensione.l forward Haar transformation in matrix form is (3.28 ) and its inverse is A sketch of the Haar transform basis functions of order 16 is shown in F'igure 3.6. Wf... ve NUMBER o ================ 2=L=, '=1= 4 9:::J =t:::r B % ::r Lb--- " 9:J rn :r J0'-lu 'm I I I I I I I I I I I I I I I I I o FIGURE Haar transform basis func tions, ], = Karhunen-Loeve Transform The Karhunen-Loeve (KL) transformation is a special case of an eigenvector matrix transformation(23, 57, 58) It takes on the

95 80 the general form 1'1-1 1'1-1 a(m,n) = L L f(x,y) g(x,y,m,n) x=o y=o for, rhich the kernel g(x,y,m,n) satisfies the equation 1'1-1 1'1-1 A(m,n) g(x,y,m,n) = L L kf(x,y,x',y') g(x',y',m,n) m=o n=o (3.31) "There kf(x,y,x',y') denotes the covariance' function of the image array and A(m,n) is a constant for fixed (m,n). The covariance function, kf(x,y,x',y') is defined as kf(x,y,x',y') = g{rf(x,y) - E\f(x,y)}l [f(x',y') - E{f(X',y')}lT} (3.32 ) where E {) denotes the expected value operator. The set of functions defined by the kernel, g(x,y,m,n), are the eigenvectors of the covariance function, and ii (m,n) represents the eigenvalue s of the covariance function. In matrix form, the transformation matrix G satisfies the relation 'There, K f is the covariance matrix of f, G is a matrix Hhose rohs are the ei(;envectors of K f, and A is a diagonal matrix of the form = A (1 ) o o o o o o...

96 81 Generally, K f is not separable. Figure 3.7 contains plots of the Karhunen-Loeve basis functions for a one-dimensional Narkov process with adjacent element correlation of 0.9. VECTOR NUMER =========== = FIGURE lcarhunen-loeve transform 9 10 JVt 11 d=bj=l,1=t:f=l::j='=tf I;> =tp=9::ftp=jij= 13 =t!-lp-jtp-.d=j:fb!4 =JtFl:-P-'9=R:fl:-r 15 -c:i=lf\:ftftfl::j=1-=f ror-r-1- r-h-h1tr-t-ri6 basis functions, N = Slant Transform 'l'he Slant transfo.cm(20, 31) is d",signed to possess the follo>ling properties: (1) a C0:1St2.nt (d.c.) basis vector (2) slant basis vectors (mono tonically decreasing in constant steps from maximum to minimum amplitude) (3) sequency property (number of zero crossings increases with increasing transform order) (4) fast computati.onal nlgorithrn and_ (5) high ene:r.gy compaction. The lowest order Slant transform matrix is identical to the lladamard transform

97 82 of order 2, i.e. 1 {i The Slant transform is orthonormal. Slan t matrices of higher order may be generated recursively from the generalized relation given by 1 I V o I rhere IK represents a K x K identity matrix. The constants, and b N are computed from the recursive relation a 2 = 1 (3.37a ) b N -- [1 ( ) 2] -t + 4 /2 (3.37b) an = 2 b N '/2 (3.3"lc) or from the formulae a 2N = ['-J 4N 2 _ 1 1t 2 b 2N = [ N - 4N 2-1 Figure 3.8 ShOl18 the Slant basis functions for N = 16.

98 83,---_._..._---_.._-----, WAvE NUMBER o =========== FIGURE Slant transform basis fu.nctions, N = 16.,, 1 o Discrete Cosine Transform The performance of the discrete cosine transform (DeT) has been shown to compare closely to that of the Karhunen-Loeve transfor(32). which is kno.m to be optimal with respect to the following perfor mance measure s : Variance distribution(59), estimation using the..t. (60,61) mean square error crl erlon, and the rate distortion function (62) Although the KL transform is optimal, there is no general algorithm that enables its fast computation. This is not the case for the discrete cosine transform and as such it is more attractive to use.

99 84 The discrete cosine transform of a data sequence f(x), x = 0, 1,.., l'i-1, is defined as a(o) = Vi N-1 r f(x) H x=o (3.39a) H-1 f(x) Cos [ m ;; (x + t)] a(m) = 2 L (3.39b) N M x=o for m = 0, 1,, M-1. Likewise, the inverse is given by 1'1-1 f(x) = 1 a(o) + L a(m) Cos [m'1l (x + t)]..0.40) fi m=1 1 1 In the two-dimensional case, the discrete cosine transform for a square image array f(x,y); x, y = 0, 1,, 1'1-1, is a(m,n) = 2 C(m) C(n) H "M-1 N-1 L L f(x,y) cos[(x+t)] Cos[(y+)] x=o y=o M N and its inverse is' H-1 1'1-1 f(x,y) = 2 L L C(m) C(n) a(m,n) cos[!'!e(x+)] cos[(y+j-)]. M m=o n=o M. M (3.42 ) where C(O) =.a and C(1), C(2),, C(H-1) = 1. It has been observed that the basis functions are actually a class of Chebychoff polynomials (32) The discrete cosine transform may also be computed by a Fourier transformation of f(x,y) over 2N points C32,45). Alternatively, a fast algorithm developed by Chen, et.al(65) provides an improvement in computational speed by a factor of six when compared with

100 85 conventional discrete cosine transform algorithms using the fast Fourier transform. Fieure 3.9 shows the cosine transform basis functions of order 16. WAVE NUMBER o C! =====::::J 5 L;J"""' '-Lr-' D 6 L? Cb f'-. 7 ""LI GJ -U I::J 8 =FI::::Fl::::FI n CL.Jl 9 0 l.r' 0 = Jl n n 1 0 OL I-U!!!2!!4!5!!"r[!!II!!!!'" o FIGURE Discrete cosine transform basis functions, 1'1 = COHPUTATIONAI. ALGOnITllNS For the general case, a one-dimensional image transform of order N.requires N 2 computational operations of additions and mul tiplica tions. This may become prohibitively high for large values of N. Fortunately, holever, efficient computational algori thros exist for mos t image. ( 1.1 ) ( 1 8 ) ( 64 ) transforms, including the Fourler., Hadamard,Haar', Cosine (65) and Slant(31). The exception is the Karhunen-Loeve transform, for Ihich approximate fast algorithms are available, but only for data with a simple exponential correlation(24,2 5 ).

101 86 Table' 3.1 shows the amount of computation involved for the various fast algorithms mentioned above. It is seen that most onedimensional' transforms require of the order of Nlog 2 N operations, with the exception of the Haar transform lihich can be computed with 2(N - 1) operations because of its sparseness. Tra.nsform Fourier Hadamard Haar KL Slant Cosine computational Comnlexity (NIOg 2 11) complex addtn. and multn. (NI0 52 N) additions 2(11-1) additions No fast alitorithm (Nlog 2 N + 2N - 4) addtn. and multn. le. 2 (1052N - 1) + 2 additions and (Nlog N - le. + 4) multiplications, 2 2 TABLE Computational complexity of image transforms 3.5 STf,TISTICAL ANAI,YSIS OF I!1AGE TRANSFORHS 'l'he development of efficient quantization and' coding methods for image transform coefficients requires an understanding of the statistical properties of transform domain samples statistical Hean and Variance Suppose each sample of an original image, denoted by the function f(x,y) over spatial coordinates (x,y), is considered as a twodimensional stochastic process. The spatial mean

102 67 E { i'(x,y)) - i'(x,y) and the covariance or their estimates are assumed known. Then, i'or a generalized i'orward transi'orm as shown in Eqn. 3.3, the mean of the transi'orm coei'i'icients can be written as E {a(m,nl) ;: a(m,n) = L L i'(x,y) g(x,y,m,n) x y 0.45) The covariance function oi' the transform samples is d.ei'ined as From Eqns. 3.3, 3.44, 3.45 and 3.46 it can be shown(45) that and the generalized expresslon i'or the variance oi' the transi'orm coefi'icients is thus ) If the covariance matrix of the original image and the transform kernel are both separable, then the transi'orm coefficient variance

103 88 can be computed as A knowledge of the variance of the transform coefficients is important in the design of quantizers to code the transform coefficients. The number of bits required to quantize any coeffi- ( cient is normally set proportional to the logarithm of its variance,or energy. In some cases,'the variance function is modelled according to a simple relation. One such model is a two-dimensional function where the amplitude of the variance is circularly distributed a':lout the ID and n directions, i.e.' it is circularly syuunetric. It has a maximum at the origin, decreases mono tonically away from the origin, and is given by(64) 2 rr (m,n) " a vhere S is an amplitude scaling constant arid 'p is a spread control constant....,' In general, it has been shown th2.t if f(x,y) is a highly correlated process, which is often the case for re2.1 im2.ges, then the variance of a(m,n) tends to be large for IOHer order coefficients and falls off rapidly with increasing coefficient order.

104 Energy Distribution lhile the dynamic range of variables in the transform domain is extremely large, it is interesting to note that only a few coefficients can actually take on large values since the total image energies in the spatial and transform domains are identical. This energy equivalence relationship can be derived from a generalization of Parsevals relationship(64) as, N-1 N-1 }: }: m=o n=o ia (m,n)i 2 = 1'1-1 N-1 }: }: x=o y=o (3.51) This relationship is useful for checking the various simulation algorithms which have been examined Probabili ty Dens;' ty Node Is for Imap,'e 'l'ransforms Generally, the probability density function (pdf) of the transform coefficients is difficult to obtain since the pdf of the original image is not usually well defined. Nevertheless, a model of the pdf is necessary to facilitate the design of appropria te quantizers based on minimum mean square error(66,67) or mini.nium nth power eror(68). Assuming that the f(x,y) samples are bounded and i.dentically distributed, the Central-limit theorem applies and in the limit the distribution of the function a(m,n) becomes normal(69). The first transform coefficient a(o,e) is a weighted summation of the original image points f(x,y) which are non-negative. As such a(e,e) is also non-negative and its pdf is often modelled by a

105 90 Rayleigh distribution given by p[a(o,o)] = a(o,o) a (0,0) exp - 2 J [ a. a cr 2(0,0) 2 a 2(0,0) It is also possible to use a uniform distribution to model the pdf of this coefficient. For transform coefficients a(m,n) where (m,n) (0,0) Gaussian or decaying exponential (Laplacian) probability densi ty functions have been used. 'rhe Gaussian pdf is given by p [a(m,n)] _ 1 exp [- a2m,n)] 2 er (m,n) a and the Laplacian pdf by p [a(m,n)] = 1 {i <r (m,n) a exp [- Ela(m,n)l] er (m,n) a. Other models Hhich have been reported include the gamma distribution (70) and a set of piece-wise linear approximated distributions(7 1 ). The present \lork uses a Rayleigh pdf for a(o,o) and a Gaussian pdf for all other transform coefficients. 3.6 CHOICE OF TRANSFORM The best transformation from the point of vie,l of (1) mean square error performance (2) subjective quality and (3) energy compaction is the KL transformation. However, two major problems are encountered in its use. They are the ne cessi ty to find the eigenmatrices,

106 91 and the computational complexity of generating KL transform coefficients which, in the two-imensional case, requires N 4 multiplication and addition operations. Although some fast KL transforms have been developed to reduce the required number of computations, these methods are only approximations and are only valid for data with a simple exponential correlation. The theoretical performances of transforms such as Hadamard, Cosine, Slant, llaar and Fourier, are close to that of the KL transform if the image is assumed to have a stationary separable correlation which is exponential in both the horizontal and vertical directions. HOlever, since these assumptions are' not strictly true far most images, particularly>!hen small data blocks are used, the simulated results usin these transforms are inferior to theoretical expectations. Comparative studies ha'fe been carried out(32,45, ) on'the better sub-optimal transforms. In the majority of cases dii'ferent criteria form the bases of these studies, among which the most commonly used include variance distribution, rate distortion, energy compaction efficiency, degree of decorrelation, "eighted and unweighted mean square error performance figures and subjective analysis. It is apparent from such results that the, Hadamard transform can be implemented effectively at the expense of a slight loss in decoded picture quality, while '"i th a slight increase in computational complexity, the discrete cosine transform is very attractive for both adaptive a.nd non-adaptive transform coding. As a result these two transforms have been used extensively in the simulations that will be described subsequently.

107 92 CHAPTER IV DETECTION AND CORRECTION OF TRANSMISSION ERRORS 4.1 INTRODUCTION In this chapter the possible use of transform coding in the detection and correction of transmission errors is ex&.lined. In a Hadamard transform system an analogue signal f(t) is sampled, the samples arranged sequentially in blocks I fk 1 of length N, and the blocks are transformed into N co efficients {a k 1 ' "here each coefficient corresponds to the amplitude of a seuency component. 'l'he coefficients are quantized, represented by binary '<lords, and transmitted. At the receiver, the reverse transform is performed to recover /f k \ plus some quantization noise. The procedure is repeated for subsequent blocks. In picture encoding, a hlock may either be a line (one-dimensional), or encompass an area of the picture (tllo-dimensional). FORWARD TRANSFORM u k Q BINARY ENCODER + CHANNEL NOISE - INVERSE fk TRANSFORM u k BINARY DECODER FIGURE A transform coding system

108 93 If a bit error occurs durin transmission such that a received sequency coefficient is erroneous, there is an error in every component in Irk1 resulting in a luminance error throughout the block. To protect against one error per block a simple block protection code has been applied(3 6,37), which results in an increase in the transmitted bit rate of (1 OO/N) %. Nethods have been employed(33-35) partially to correct the effects of transmission errors in PCI1 and DPCI1 systems, by detecting the errors in the recovered time signal on a statistical basis and replacing the e.rroneous sample using an interpolative technique. These stati3tical techniques are, now extended to the Hadamard transform system, operating not in the time but in the sequency domain, to attain an improved performance compared "ith the block protection encoding method "ithout any expansion in the bandi'jidth of the transmitted signal. he basis of the scheme is the fact that real images are highly correlated spatially and as such amenable to effective data compression techniques. HO\fever, if such redundancy reduction techniques are not fully exploited, then the remaining redundancies may, alternati.vely, be used in an efficient and novel manner for error detection and correction to maintain a certain predetermined fidelity in the received information. Pri.or to the detailed analysis of such a scheme, however, the subject of artificially generated images "ill first be discussed. At the onset of the research programme, facilities for digital video processing were very limited in that no digi tal video processing hardware Ias available to permit si.mulations using real i.mages. Thus the use

109 94 of artificially generated images having predetermined statistics emulating those of real images 'fere necessary to allow simulation work to proceed whilst a digital video processing system "las being developed. Experiments on the use of image transforms for error detection a.nd correction thus employed Gauss-Harkov random fields. 4.2 ",'IO-DIJ.n=::NSIONAL 'GAUSS-NARKOV RANDOI'l FIELDS(4 6, 78, 79) In the theory of random processes, the concept of a Harkov process is important since many physical processes can be modelled in such a way. This section defines a l1arkov random field on a discrete space,' 1-lhich, in the homogeneous Gaussian case, can be represented by a two-dimensional difference equation "driven" by a. homogeneous non-hhite noise source. The noise source is equivalent to the error in a best linear mean square error estimate of the field at a point in terms of its nearest neighbours. A P-order homogeneous Gauss-l1arkov random field is characterized by the correlations among the elements to a distance P. Let D represent the collection of pairs (i,j) D = {(O,1),(1,O),(1,1)} and r(x,y) be an estimate of the discrete random field f(x,y), i.e. f(x,y) = }: (i,j)ed c(i,j) f(x - i, y - j) where the coefficients e(i,j) are determined such that the mean

110 95 square estimation error e(x,y) = E{lf(x,y) _ r(x,y) 12} is minimized. A discrete random field f(x,y) is wide-sense j';arkov if and only if it satisfies the follo\-ling difference equation for all points (x,y) with x >1 and y> 1, f(x,y) - r(x,y) = (x,y) where (x,y) is a discrete random field of orthogonal random variables, i.e. E I (x,y) s(p,q) 1 = x fo p, y fo q For a zero-mean random field the random variables (x,y) a.re uncorrelated. For the points in the top row and extreme left-h"tnd column, f( 1,1) = (1,1) f(x,1 ) - ;z!.f(x-1,1) = (x, 1) j f( 1,y) -$jf(1,y-1) = (1,y) (4.6a) (4.Gb) (4.6c) where, if Rff(') is the autocorrelation function, then * = Rf1(1,O)/Rff(O,O) and S = R f1 (O,1 )/Rff(O,O). The random field of Eqn. 4.4 may be modelled to a good approximation if the autocorrelation function is known. The modelling is valid only if the autocorrelation function of (x,y) is positive definite.

111 96 The degree of correlation of (y',y) serves 8.S a measure of the goodness of the representation of the given random field (the 10l<er the correlation the better the representation). If the random field is assumed to be normalized in that the mean value of the random field has been subtracted from everj picture element and each picture element amplitude has been d.ivided by Rff(O,o) then the autocorrelation function can be \-Tritten as(4 6 ) or (4.8) where -c rh = e 1 and fv = e -c 2 are measures of the horizontal Rnd vertical correlation.respecti-,rely. Using this expression for Rff(') in the evaluation of the coefficients c(o,1), c(1,0) and c(1,1) yields the follohing solution c(1,0) = Ih c(0,1) = (J IV c(1,1) = - ih Iv Thus we have a simple way of e;enerating o.iscrete two-dimensional Markov fields if the horizontal and vertical correlations are kn01-in. 'J'he difference equation (Eqn. 4.4) is driven by a noise signal. (x,y) with a Gaussian probability density function, zero mean and a variance given by = (4.10)

112 ERROR DETECTION Consider a picture composed of one-dimensional blocks having N components. The blocks are sequential along the scan lines. In the sequency domain the same block formation occurs along the lines, but the blocks contain sequency coefficients. Let S(l,m,n) represent the Ith coefficient in the: mth sequency block on the nth line in the recovered picture at the receiver. Each coefficient is scrutinised by an error detector. The detector computes the absolute differences between corresponding sequency coefficients on adjacent lines, i.e. (l,m,n) = IS(l,m,n) - S'(1,m,n-1) I (.11) where- the prime symbol signifies that the coefficient has been previously examined and has either been accepted as correct or heen corrected. If this absolute difference (l,m,n) is statistically larger than might be expected then S(l,m,n) is considered to be erroneous and a correction procedure is employed. that S(l,m,n) is only consideed The error detection criterion is erroneous if Ml,ffi,n) > 'j. (m,n) (4.12) u where r (m,n) is the error detection threshold factor. Observe u that each coefficient difference (1 = 1 to N) is tested against a

113 98 threshold dependent on a block of data. We now consider the formulation of 1u (m, n). The absolute difference of the corrected sequencies in blocks m of the previous two lines (n-1) and (n-2) are formed: 6'(1,m,n-1) =IS'(1,m,n-1) - S'(1,m,n-2)1 The rms values a(m,n) and a'(m,n-1) related to the present and previous defference values between corresponding coefficients in adjacent lines computed for blocks m having N coefficients are then respectively, cr(m,n) = 1 N N L [Ml,m,n)] 1=1 2 cr'(m,n-1) = N 1 L [6' (1,m,n-1)] N 1=1 2 Typically, the 10Hor sequency coefficients have higher energies than those of higher sequency. This characteristic is acknowledged by the use of an adaptive threshold "hich assigns a sequency weighting factor dependent on the sequency number. N = 32 is a typical block size, and for this case, experiments have revealed that the sequency block may be satisfactorily partitioned into five groups. The weighted threshold is also made more adaptive by the inclusion of another scaling factor which is determined by the relative magnitudes of cr(m,n) and a'(m,n-1). It is to be expected that if there is a large difference between cr(m,n) and cr'(m,n-1) either an

114 99 error (or errors) has occurred or there is considerable activity between lines. In this situation the threshold J (m,n) is made u larger to ackno,lledge" the change in activity, but not so large as to prevent error detection. The determination of the optimum threshold as a function of the sequency number depends on the statistics of the signal. A twodimensional Gauss-Harkov source has been used Ihich has statistical properties not unlike many pictures, particularly those encountered in vim"phone systems. The follo\ing expression for t- (m,n) gives u good results although it is probably not optimum. (4.16) where u is an integer from 1 to 5, each value corresponding to one of the five sequency groups. l and LA are scaling factors, where l determines the Ieighting curve and LA adapts this curve to the activity of the data from line to line. LA is determined as follows : If cr(m,n).;; 0.5 (J""'(m,n-1) AND <T'(m,n-1»K then LA = 0.1l If 0.5O-'(m,n-1) <cr(m,n) < cr'(m,n-1) AND cr'(m,n-1»k then LA = 1.0 If cr(m,n) ;;'IT'(m,n-1) OR <T'(m,n-1)<K L1 and K are system parameters which are determined experimentally.

115 100 The inclusion of K means that a minimum criterion is set such that should cr' (m,n-1) < K an effort is made to increase 1- (m,n) to avoid u excessive false detection, i.e. to make LA = L1 > 1. This becomes apparent in the following example. Suppose we have a background with a nearly constant intensity for a feh lines prior to encountering scan l_ines wi th varying intensity. This results in very 10>1 values for cr' (m,n-1). vlhen the scan line containing varying intensity is encountered, because of the lo1 magnitude of cr'(m,n-1), the threshold X (m,n) will also be very u small and hence the activity in the current line Hill result in the detection of false errors. Hence the minimum bound K is necessary to prevent such occurrences. The weighting scale factor is found from o-(m,n) CT'(m,n-1) %o-'(m,n-1) cr(m,n) < CT'(m,n-1) i;3 cr'(m,n-1) (J"(m,n) < i;4 cr'(m,n-1) for i = 1, 2, 3 and (J"'(m,n-1) cr(m,n) \, = " = 1\.) = L 2, L }'2 + i \01 = L Thus when the acti;,"ity in the current block is 101 Lw= (4.18) L, 2 \'Ihioh increases to L ",hen the activity is high in order to prevent false detections. L 2, like L1 and K, is determined experimentally. The adaptively weighted threshold component of 1- (m,n) from u Eqn is,

116 101 where' the sequency weighting factor (2 5 - u - 1) in largest for the lowest sequency components. u identifies the sequency group. The sequency coefficients in groups u = 1, 2, 3,4 and 5 are 1 = 1; 1 = 2, 3; 1 = 4, 5, 6, 7; 1 = 8, 9,, 15; 1 = 16, 17,, 32 respectively. Thus each coefficient is tested against a threshold which adapts to its sequency number 1 and the local activity in the picture. Figure 4.2 illustrates the typical effect on the adaptively ;,eighted threshold of varying degrees of activity. Increasing Activity COEFFICIENT ORDER- FIGURE Effect on error detection threshold for varying degrees of activity 4.4 ERROR CorRECTION If Inequality 4.12 is satisfied, the coefficient S(l,m,n) is

117 102 considered to be in error ru1d is replaced by the estimate S'(l,m,n) = 0.5 [S'(1,m,n-1) + S(1,m,n+1)] This estimate is the mean value of the coefficient amplitude of the lth sequency in the mth block on the previous line (n-1) and the succeeding line (n+1). S(1,m,n+1) has not been tested yet, but the probability of S(l,m,n) and S(1,m,n+1) both being in error is assumed to be very 10\'1, thereby justifying this method of estimation. Eqn is employed except on the last line of the picture v;hen S'(l,m,Z) = S'(l,m,Z-1) (4.21) where Z is the total number of lines in the picture. 4.5 RESU LTS AND COHI'1EN'fS The above sequency difference detection and correction (SDDC) system is evaluated using a discrete two-dimensional Gauss-Harkov random field for the image. In the simulations, the values of fv and rh are selected to be typical of those for a head and shoulder pcture, namely The original picture elements f(l,m,n) in an 8-bit format are Hadamard transformed as S = H F where H is the Had.amard matrix of order 32, F is the one-dimensional

118 103 data vector containing f(l,m,n) and S is the resulting sequency vector. After bit errors have been introduced into S the SDDC system is used. Because j (m,n) should not be excessive for large u changes in cr' (m,n-1) and O"(m,n), it is limited in the simula.tions to 1024 for u = 1, 768 for u = 2, 512 for u = 3, 256 for u = 4 and 128 for u = 5. Naturally the choice of these ma.xima are dependent on the statistics of the data being processed. The results are displayed in Figures 4.3 and 4.4 for three sets of values for 1, 12 and K, for the situation of single and double 1 errors. Although the number of binary errors in the se'luency blocks is under the control of the experimenter, >Thich blocks receive these errors and which bit in a se'luency coefficient word is inverted is a random event, '['he figures shoh that the percentage normalized mean square error (nmse) is significantly improved by using the SDDC system. Curve (a) sho\'ls the effect when L 1, 12 and K are too low. The increase in nmse is the re!3ult of false detections, while the effect of making these parameters too high is shown in Curve (b). Here the SDDC s;)'stem can only detect the le.rgest errors in the secj.uency coefficients. Curve (c) has a set of parameters Ihich enables all errors in excess of 1.6 % of the amplitude range of the secj.uency components to be correctly identified. l;1hile errors of smaller magnitude are occasionally detected, particularly if they occur in the higher secj.uency components. The set of values for 1 1, 12 and K in Curve (c) appears to be nearly optimum for the SDDC system.

119 (a) L1 =1.2,L2=0.2,K=32 (b) L1=1.5,L2=0.4,K=160 (c) L1=2.15,L2=0.65,K=320 (d) without S.D.D.e. system (d) '" o u.j '" 1.0 z <! u.j :':0.1 u.j I Z u.j U u.j '" Q. (a) (b) (c) L PERCENTAGE SAMPLE ERROR RATE FIGURE Results for snnc system, single errors

120 105 (d) 10.0 (a) L1 =1.2,L2=0.2,K =32 (b) L1=1.5,L2=0.4,K=160 (c) L1=2.15, L2=0.65, K=320 (d) without S.O.O.C. system (b) (c) 1.0 0: o 0: 0:: LLJ LLJ 0: «:::> 0.1 z..:: LLJ L... ' <:J «I z LLJ LJ 0:: UJ Cl L- --'- -' PERCENTAGE SAMPLE ERROR RATE FIGURE Results for SDDC system, double errors

121 NOTE ON PUBLICATION The sequency difference detection and oorrection (SDDC) system described in this chapter has appeared as a paper in the Hay 1978 issue of Electronic Letters Vol. 14 No. "10. The paper was entitled "Partial correction of transmission errors in \-Ialsh transform ima.ge without recourse to error correction codes" and was jointly authored by W.C. \'Iong and R. Steele. 4.7 CONCLUSION The SDDC system exploits the correlation between adjacent lines of an image to reduce the percentage normalized mean square errcr by approximately 20 db for percentage sample error rates below 3 %. It performs almost as well in the presence of t,w errors per block as for one error per block. The SDlJC system does not rely on any channel protection codes yet has a superior performance to the block protection code technique(36,37). In addition, study of the SDDC system reveals the importance of the coefficient energy and amplitude distributions in relation to the order of the coefficient, and of the activity \oiithin a transform block in relation to the choice of a sui table algorithm to perform a particular operation.

122 107 CHAPTER V - DATA COMPRESSION 5.1 INTRODUCTION Data compression is one important area in Hhich the application of orthogonal transforms allo"s more efficient transmission of information. }<'or the original data f(x) Hhich has been transformed according to N-1 a(rn) = L f(x) g(x,m) x=o where' g(x,m) is the forl1brd transform kernel and a(m), the transform coefficient the objective is to select a subset of M components of a(m), "here H is substantially less than N (N is the total number of coeffioients' in a transform block). The,remaining (N - 1-1) components can then be discarded \11 thout introd.ucing objectionable error, and the signal is reconstructed using the retained J.1 components of a(m). 'fhis procedure involves the reduction 01 redundant inlormation present in the source signal. As a result of the inherently high pel-to-pel correlation of real imat;es, the energy in the transform domain tends to be concentrat"o. in a relatively small number of transform coefficients. This point is illustrated l.n Fl.gure 5.1, \lhich shohs the energy distribution of a t'iodimensional (16 x 16) discrete Cosine transform of the "GIRL" picture of Figure 5.2. transform coeficients To achieve data compression, low-magnitude ffi2.y be d.iscar.ded or grossly ql1antized without

123 108 Log(Energyl ENeRGY OISTR!9TIGN 'GIRL' PICTUR AZ I 11:) Hi ;, w i [lth " figure Energy distribution of the transfom coefficients of the "GIRl}' :picture.figu"e The. "GIRL" picture

124 109 introducing serious image degradation. In order to compare the results of such data compression techniques a suitable error criterion must be defined. An error criterion which is often used is the percentage normalized mean square error (nmse), i. e. N rl l If(x,y) - r(x,y) 1 2 x=o y=o nmse = 1' % (5.2) l l \f(x,y) 12 x=o y=o where' f(x,y) is the original sample having spatial coordinates (x,y) A and f(x,y) is the corresponding recovered signal. This criterion has been used in all the simulation experiments to be desc'ibed, and, although it is not entirely adequate. as an objective equivalent to subjective assessment, it nevertheless provides a useful basis for numerical comparison of the results. 5.2 CODING COHSID;;;RATIONS In transform coding, a number of considerations must be taken into account to ensure efficient data compression. These consid.erations include 1. the type of transformation 2. the transform block shape 3. the transform block size 4. the quantization strategy

125 Tvne of 'fransforma tion The choice of transformation has already been discussed in Section 3.6. Of course, in the pursuit of an efficient tran3form coding technique, the compromise between speed of computation and resulting quality is an important consideration, and these two opposing factors must be weighed appropriately in the light of current technological capability of implementing the transformation. In the experiments on data compression techniques the discrete Cosine transform has been used since this transform has the advantage of possessing a fast computational algorithm as ",ell as other properties "'hich are suitable for image coding. Moreover, its performance compares closely to that of the Km:hunen-Loeve Hhich is knohn to be optimal..2.2 Transform Block Shane T\'lo-dimensional transformation yields better performance th2.n tllc one-dimensional operation(45,79.) because both horizontal and vertical spatial corre la tions are taken in to accoun t.. )[OHever, the reported improvement is small, i.e. about 0.2 bit/pel. In moving to three-dimensional transformation, it is expected. that more improvement Hill be forthcoming since the high degree of temporal correlation can be exploited. However, inherent in the use of three-dimensional techniques is a time delay which is related to the order of the temporal dimension. If this d.elay is excessive it may be subjectively disturbing in a hlo-hay 80mmunication link.

126 111 Nevertheless, three-dimensional transform coding presents the prospect of achieving a substantial degree of data compression. Undoubtedly, three-dimensional transform coding is computationally more complicated than hlo-dimensional transform coding, so that the only justification for its use is when its performance exceeds that of the two-dimensional case by a convincing margin. Subsequent simula tions on data compression make use of both t\w- and threedimensional transforms and their relative advantages and disadvantages are compared Transform Block Size If nmse is the criterion used to assess the quality of decoded pictures, then performance should improve with increasing block size since the correlation bet\een more picture elements is taken into account. Ho\{ever, most pictures contain significant correlation between picture elements for only about 20 nearest elements, although this is dependent on the amount of detail in the picturc,(79). Hence a point of diminishing returns is reached' for block sizes of greater than 16 x 16 pels. If subjective quality is considered then n;. 4 is acceptable, \;here n is the block size. If n is too large adaptation techniques cannot be effectively applied, but on the other hand, if n is too small the correlation properties cannot be fully exploited. As a compromire, a 16 x 16 transform block size has been used for experiments on t\o-dimensional transform coding,,lhereas for the

127 112 three-dimensional case, various block sizes have been used, viz. 16 x 16 x 4, 16 x 16 x 8 and 8 x 8 x 8. The effects on coding algorithms due to the use of different block sizes are also examined Ouantization strategy An effective data compression algorithm requires appropriate Quantization of the transform coefficients. If the range of the transform coefficient amplitude is represented by the line H1H2 in Figure 5.3, coefficient a(m,n) is Quantized into one of K discrete levels, ",here the reconstruction levels are represented by r, r, 1 2, r, and the input intervals are represented by the decision K levels d, d,, d This means that if a coefficient at the 1 2 K input of the Quantizer has a va.lue bet>leen d and d + k k 1, it is assigned the value r k at the output. The process of quantization involves t"o operations, namely 1. Quantizer characteristic determination 2. Bit allocation ---- SAMPLE VALUE RANGE DECISION LEVELS d 1 d 2 dl< d K + 1 H I I I I I I I I I I I I I I I I I 1 I 1 H2 1 I I I I I I I I I I I I I I I I r 1 r 2 r K RECONSTRUCTION LEVELS }'IGURE Quantizer decision and reconstruction levels

128 (Juantizer Characteristics The selection of the quantizer decision and reconstruction levels depends on the error criterion 'lhich is used to assess the performance of the quantizer. Subjectively, it has been found that a given incremental brightness change in the reconstructed image is more noticeable if the brightness level is low than if it is high. This suggests that the density of quantization levels in the original spatial domain (i.e. the x-y plane) should be (ireater ",t lower amplitude levels; But since the brightness of every point of a reconstructed image is a function of the amplitude of a single transform coefficient, then by the same reasoning, the density of quantization levels should be greater at lower levels of the transform coefficient amplitude range. The human viewer L; also very sensi tive to the location of high frequcncy brightness "c'ratlsitions, but relatively insensitive to their actual magnitude. Jo'o:om this characteristic of subjective viewing it would seem that the density of quantization levels at low coefficient amplitudes should Output Output Input (a) ( b) FIGURE Quantization characteristics (a) Uniform linear (b) Non-linear

129 c 1 cc 1 cc 4 c be greater i'or higher order transi'orm coefficients than for 10Her order ones. This means that i'rom the standpoint of subjective quality, a quantizer Hith a non-linear characteristic will result in reconstructed images which are subjectively more acceptable. Figure 5.4(b) shows a typical quantizer characteristic which is non-linear in contrast to the uniform, linear quantizer characteristic 01' Figure 5.4(a). Optimum quantizers are derived i'rom a minimization of the distortion 01' a signal by a quantizer. A distortion measure \olhich is commonly used is the mean square error, e q 2,,given by 2 e q = K d r J k+1 (a - r )2 pea) da k=1 d k k where pea) is the probability density function (pdi') for the input transi'orm coefficient value which is represented by the continuous variable a. HaX< 66) has derived equations i'or the parameters 01' a quantizer \',i th miniinum distortion. For most common probabili ty density i'unctions encountered in the coding of transform coefficiets, these equations cannot be solved in a closed form and numerical techniques are required. Hax employed a systematic, recursive method for calculating the decision and reconstruction 'levels for the optimum quantizer in the case when the input signal has a Gaussian pdf wi th zero mean and unit variance, and when the end values d and d + are given by - eo and + 00 respectively. Table K 1 lists the decision and reconstruction levels for a quantizer designed aceording to the method of l';ax for signals wi th a Gaussian pdf. Approximations to the solutions for the decision and reconst-

130 115 Si t s d k 'k Si I s d k " lJOOOO G 167 Et, ;:80.033bB f.g S4 1j9. (, ;' BOG <)0 I I SO 1.0' \%0.OOOQv. 12 Si I) LS3tiO,.' ;: SO.7'HGO u 351;30 1.0<; (, <) '%00.44:2 BD.4H20 2 Hil S'jO.512Bu B' SS Hao (,37'50.535<)0.(,0500.b5S5Cl (, 7360 (, 7Q I !210G.e470 : ;6.746% 5 c)"n Bl,tO !JO (1.'0280 E; B05G, {;S50 I.4820 (j [ COO bO ,02900,9'S620.97::% ';0 1. Cll hoo ,, (,00, i..:WO 1, uDOO BOO,OG )00 1. IBOOO C, 1 L , i'OO, B ' u , GO ;;GO 1.4SGOO.54440, ;' (,' '54E.OO,6& <; is ,e '7BOO 1.70'SOO 'J20 1, nlod, g2eao I & aoo 1.6:7& ;:00 1.9u ;7'; :' OG 2.1';2ClO ::,,0(::( Cl (, {} OO 1.717DO ; '9DO 1.D ; ,.5HOO 2.S6:;;OO 2.0B Cl &00,, ,7aGOO & , :8;;00 2.B IBOD IOO 3. 06(;00 2.S t B600 3, O(j C u :3. % TABLE Placement of decision and reconstruction levels for Max Gaussinn Quantizer 7

131 116 ruction levels of the optimum quantizer have been proposed by such authors as Panter and Dite(67), \>loods(oo), Bruce(81) and Roe(82). It is also possible to perform non-linear quantizaticn by a companding operation, as shown in Figure 5.5, in which the sample is transformed non-linearly, uniform linear quantization is and the inverse non-linear transformation is taken(8 4,09). performed, In this system the pdf of the transformed samples, g, is forced to be uniform. Table 5.2 contains the companding transformations cmd their corresponding inverses for the Gaussian, Rayleigh and Lapla.cian probability density functions. NONLINEAR UNIFORM INVERSE - '-<? TRANSFORMATION 9 QUANTIZER 9 TRANSFORIAT!ON f FIGURE Companding quantizer Quantization characteristics may also be matched to subjective performance. Mounts,et al(85) developed a technique for obtaining subjec ti vely optimum quan tizers for Hadamard transformed still pictures in which a series of subjective tests were carried out to determine the visibility of impairment in the reconstructed picture when simulated quantization noise was added to the Hadamard coefficients. A design procedure for quantizers was developed using these visibility criteria.

132 Gaussian p(f) (2,...,-) '-w- 'xp -\:;-; rr'l -". Rayl!!igh pl{)"xp I 1 --, I' I if 2" Laplacian where erf(x)=..r;; 2 f' " 0 exp(-y2)dy TABLE 5.2 Probbility Density Forward Transformation Inverse Transformation gl crflu i =..fi" erf-' {211l g = -exp { - ::} p(f) =:; " exp {-a I III ghi-exp{-af)] 1",0 J2 (f =- 8-lEI-exp{af)] 1<0 " Companding quantization transformations(4sl i [J2,,' In [I/(l-gl]]'" _ 1 _ 1--ln[I-2g] "",0 u - 1 [ _] _ I=-In 1+2g g<o a --.J

133 118 M_ O "0 M "0 -N , M N,,,, ;:;- - N O "0 - - N " 0_ - I- V> 0 0 C '" -.. '" 0 U - - M N,, ';' 0, -V> C " "0 " 0 "- E ci " 0- E -<.., N 0 - N.-.- N U - E " C> - " 0> 0 V> :c N N,, ';' 0., " " E.. Z - I '" " ';' -D tn W 0:: ::> <:J j: u: - N - N. 0.I.)U;)na:i a... q:;la -,(JuanbaJ-j aaqojil/::i C,,

134 119 From the points made above, the design of a quantizer involves a knowledge of the range and statistical distribution of the transform coefficients to be quantized. Since this information is not available unless the transform is specified, quantization methods can only be investigated for particular transforms. In the simulatio.ns carried out, the discrete Cosine transform was used, and }'igure 5.6 sho<ls the histograms for the first eight transform coefficients vlhere a two-dimensional transform block size of 16 x 16 pels was implemented. The original data source was the "GIll L" picture sho,m in Figure 5.2. From these ohserva tions it was decided to assign a llayleigh pdf to the d.c. coefficient a(o,o), and a Gaussian pdf to all the a.c. coefficients. In addition, the d.c. coefficient used a companding quantizer whilst the a.c. coefficients used the optimum quantizer of Max Bit Allocation In defining the number of bits required to quantize a coefficient an understanding of the concept of the rate distortion function is necessary. The rate distortion function of a source with kno,m probability distribution determines the minimum channel capacity required to transmit the source output as a function of the desired minimum average distortion, where, the distortjon fullction (someti.mes called fidelity criterion) is a measure of agreement bet>leen source and system output specified by the user. Let the rate distortion. (87 88). function be denoted by ll, then the follo<lng theorem ' nolds If a memoryless source x is to be quantizecl wi th a mean squa.re

135 120 error not greater than D, then R "" H(x) - t log2(2 To ed) bits/sample where' H(x) is the entropy of x, e is the exponent and sample in this case means transform coefficient. If the source is Gaussian we have where a 2 is the variance of x. x The rate distortion function, R, in the above theorem, in particular for the case of a source with a Gaussian probability distributicn (Inequality 5.5) provides a useful lol"/er bound in determining the bit requirement for quantization. Although the transform coefficients may not all possess a Gaussian pdf, Inequality 5.5 does give an indication of the number of bits required per coefficient, given a knowledge of 'the distribution of the variances of the co efficients and the required distortion criterion. Thus, because of its simplicity, this result has been used extensively in the hit assignment of trapsform coefficier,ts, and the problem reduces to a determination or estimation of the variances or energies of the transform coefficients. As the transform coefficients differ signj,ficantly in magnitude from coefficiellt to coefficient, the number of bits required will also vary frnm coefficient to coefficient. An additional point of

136 121 interest here is that decorrelation of the coefficients due to transformation results in reduced tra.nsmission rates compared to when the picture data is directly coded as though it consisted of uncorrelated samples. 5.3 ADAPTIVE COEFFICIEWf SELECTION Al'lD (luml'frzation - A REVIEH An approach 11hich is often employed to achieve data compression using image transforms is adaptively to select and 'luantize the coefficients. Figure 5.7 shows a block diagram of an adaptive transform coding system, where the coding of the transform coefficients is made dependent on the short-term statistical characteristics of the image. Such a system normally results in a variable rate data stream so that for the system to be practical some form of d.ata buffering becomes necessary. In general, three of the more - STATISTICAL MEASUREMENT ORIGINAL IMAGE FORWARD TRANSFORM COEFFICIENT SELECTION QUANTIZER & CODER RECONSTR UC TED. IMAGE INVERSE TRANSFORM DECODER CHkNNEL FIGURE An adaptive transfom coding system

137 1 22 established techniques that have been employed are 1. Threshold sampling 2. Zonal sampling 3. Zonal coding Threshold Samnling In its simplest form this method involves the selection of a threshold level and only transform coefficients I,[hich are larger than this threshold are transmitted. The transform coefficients below the threshold level are set to zero at the receiver. This system is adaptive since the number 'and the location of the coefficients that are larger than a fixed threshold level change from one block to another depending upon picture details. HOHevcr, the system requires a relatively high bit rate for transmitting the addressing information. A more comprehensive study of the threshold coding method for bandhid th 'compression of image data '[as made by Anderson and lluang(5 1 ). Their proposed system used a tlo-dimensional j<'ourier transform of a block of 16 x 16 picture elements. The standard deviation er of the coefficients in each block was first measured. a Then amplitude, phase and position of the 11 transform coefficients wi th the largest amplitudes Here transmitted "here H '[as proportional to the standard deviation of the coefficients in each block. The adaptivity of the system \-Ias increaned by making the number of quantization levels in each block proportional to the standard deviation of the transform coefficients in that block. The addressing

138 123 information and position of the M largest Fourier coefficients t lere transmitted using a run-length coding algorithm. Good results were reported at 1.25 bits per picture element Zonal Samnling For most natural scenes, high spatial correlation results in a concentration of most of the image energy in relatively few transform coefficients, usually those of the lower order, such that substantial bandwidth reductions may be accomplished if the higher order coefficients are not transmitted. Discarding the higher order coefficients is equivalent to passing the image through a zonal low-pass filter. Several different zonal shapes may be used, for example, circular, (4'5) elliptical, rectangular or tria.ngutar. l!ohever, it has been shown.. that the optimum zone for a mean square error criterion is one of maximum variance, where. the transform coefficients having the largest va.riances for a given covariance model of the original image are selected. This zonal sampling operation may be expressed analytically as = (5.6) 'lhere S is the selection matrix, G the iwage transform, f the original data vector and f the reconstructed data vector. (S G) represents the forward zonal sampling process, and (G- 1 st) represents the inverse reconstruction process. Figure 5.8 contains a block diagram of the zonal sampling technique.

139 124 G S ST G-1 f FORWARD SELECTION RECONSTRUCTlO INVERSE A a at TRANSFORM MATRIX MATRIX a TRANSFORM. f FIGURE Zonal sampling operation Zonal Coding In the zona.l transform codine system a set of zones is established in each transform block. The zones may be derived from ah optimum block quantization algorithm where the number of quantization levels for each transform coefficient is set proportional to its expected variance Ruch that the q1.1antization error for each transform ccefficient is the same (21,9 0 ). Since in general the variances of the transform cqefficients are different the number of binary digits assigned to each coefficient,{ill be different. In addition, efficient encoding of these blocks requires more binary digits for areas of high detail and fe;ler binary digits for areas of low detail. If a constant word-length code is used, b(m,n) code bits are assigned to the coefficient a(m,n) resulting in a total of B bits per block, i.e. N-1 N-1 B = I r b(m,n) bits (5.7) m=o n=o liuang and Schultheiss(21) determined the optimum allocation of a total of B bits to the N 2 coefficients. They found that the nu;:lber of bits b(m,n) used to code coefficient a(m,n) should be proportional

140 125 to er 2(m,n). a An algorithm was obtained for computin" b(m,n) such that the mean square quantization error was minimized for a given B' and set of variances 0' 2(m,n); m, n = 0, 1,, N-1. a Ready and wintz(7 6 ) derived an algorithm for computing b(m,n) and B, such that the mean square quantization error was minimized for a given N 2 and set of variances er 2(m,n); m, n = 0, 1,, N-1. a It is also possible to achieve a slightly lower mean square error for a given channel rate by employing Huffman coding(9 1 ) of the quantized coefficients rather than constant word length coding, holever, this \ill j.ncrease the complexity of the coder other Adaptive Coding Hethods Gimlett(4 2 ) and Claire(43) have suggested the use of an activity index to classify transform blocks, where in each class a different coefficient selection and quantization procedure is used. TIW definitions of activity index have been used, the first being the sum of the squares of the 2.11 pli tudes of the transform coefficients and the second, the sum of their absolute amplitudes. A combination of zonal and threshold sampling is used for each class. In another system developed by Chen, et al(9 2,93), adaptivity results from a distribution of bits between classes that favour higher levels of activity. 'rhe fast discrete Cosine transform blocks are sorted into four classes according to the level of image activity measured by the total a.c. energy within each transform block. Tasto and lintz (94) have studied an adaptive transform coding system in which

141 1 26 image blocks are classified into three categories according to luminance activity, and blocks of each category are'matched to the category statistics. Similar methods have been developed by Cox and Tescher(39) and Netravali(95). The relative magnitudes of different transform coefficients also indicate the degree of activity in various directions and at va.rious frequencies. In a three-dimensional transformati'on, for example, the relative amplitudes of the three coefficients adjacent to the d.c. term reflect the image activity in the horizontal, vertical and temporal directions. Knauer(9 6 ) uses this idea adaptively to encode various degrees of movement using a 4 x 4 x 4 Hadamard transform. }'or rapid movements, an option giving high temporal fideli ty is utilized. 'rhe overall encoded picture quality is improved since the eye is relatively insensitive to spatial fidelity for. rapidly moving objects. other methods of block classification not using some form of activity. index have also been reported. The 3ystem of Ohira, et 0.1(97) has three kinds of bit assignment, the optimum one being adaptively selected according to the detail in the input signal. 11auersberger(9 8 ) considers the use of a two-dimensional variable which relates to the mean s'luare difference between adjacent pels. The computation of this variable provides an estimate of the horizontal and vertical correlation \-Thich is subse'luently used to classify the block, based on an optimization of a rate distortion criterion. Tescher, et al(4 0 ) have developed an adaptive Fourier transform

142 127 quantization technique in \1hich the magnitude and phase of each coefficient are adaptively coded. The number of quantization levels for the phase. and magnitude components are set proportional to the logarithm of their estimated variances, with the restriction that the number of phase levels is t\1ice the number of magnitude levels. The variance of the amplitude of the individual coefficients is estimated using a predictor that combines the variances of a number of adjacent quantized elements. This adaptive coding teclmique has also been extended to the coding of Hadamard transform coefficients. In another approach, Tescher and Cox(4 1 ) divide the image into blocks of 16 x 16 pels and SCan the transform coefficients diagonally, since this gives a smoother decay of the size of the variances of the transform coefficients. An estimate of the variance of the onedimensional data sequence is then made and a bi t assignment is set proportional to its logarithm. The estimate of the variance 01 the nth transform coefficient cr 2(n) is a (5.8) \1here a(n-1) is the quantized amplitude of the (n-1)th transform coefficient (in the one-dimensional sequence) and A1 is a \1eighting factor "hi ch is chosen as 0.75 in their experiments. Good results have been reported using thi s recursive quantizatiori method I;i th Cosine and Slant transforms. Hany of the adaptive coding techniques mentioned above result in variable rate systems,there the number of bits assigned to each

143 12 B block changes from block to block. This means that a buffer and buffer control logic are necessary for transmission over a fixed rate channel. To overcome this problem, Reader(99) and schaming(100) have considered algorithms resulting in fixed rates which make use of normalising constants related to the variances of the transform coe fficients. It is intended that the discussion presented in this section establishes a brief understanding of the variety of adaptive transform coding algorithms that have been examined 1fith the aim of achievi ng data. compression. In general, the bit rates \'Thich are achieveable using such systems are ),n the region of 2-1, ' , and bits/pel for the one-, t,io- and three- dimensional cases respectively. These figures represent substantial reduction in the informa.tion rate. However, there are still areas "'hi ch are insufficiently explored and in the follo\'ling section bas i s for improving the performance of adaptive transform coding systems are examined. 5.4 llm3ic; FOIl H1P110VEHEllT The performance of a basic transform coder can often be improved substantially by monitoring the short-term statistical changes 'Iithin an image, and then adaptively selecting and quantizing the transform coefficients. Despite the desirable property that image transforms have of producing a sequence of nearly uncorrelated coefficients from a highly correlated image field, it is nevertheless true that there is some justification for adaptive transform coefficient quantization and reconstruction. The review on adaptive

144 129 transform coding systems preoented in Section 5.3 provides an insight to the variety of algorithms that have been used to achieve, data compression. However, in these systems a number of shortcomings are apparent which limit their performance. Three important shortcomings which merit consideration are: 1. inflexibility or limited adaptivity of existing systems 2. necessity of transmission of side information 3. generation of highly variable data rates. In general, all these three points are inter-related Ifhen the overall performance of an adaptive coding system is considered. To obtain an understanding of the implications of these shortcomings, each will first be considered separately. 'ro ensure that a coding algorithm is responsive to measured statistical changes in the image field, it is essential that "llhe adaptivity of the coding algorithm must be sufficiently flexible to cope with such changes. In the case of systems using block classification techniques, adaptivity is limited to a set of predetermj.ned classes of transform blocks. Such classes are nor:nally geneated from statistics derived as an average of a set of ioages that may be "commonly" encountered. The intuitive choice of such a set of images generates uncertainty and debate about the performance of such 5ySt '.:U3 in a rcal-tir:.e environment. In many case5 the limitations are imposed by the simulation environment, nevertheless, it is still true that such a coding system has only been optimised for a specific range of data source images. The present work is also bounded by such limitations due to the simulation envircnment. HOHever, this consideration must be taken

145 130 into account and so any coding system "hi ch has been develeped should have sufficient flexibility to cope Id th a variety of images in an environment in which the system has been d.esigned to operate. To a certain extent, such a problem may be overcome by relaying to the receiver drastically changing source conditions thro"j.gh the transmission of side information. Block quantization techniques fall into this category. In such cases there is a need to transmit information about the variances of the transform coefficients so as to allow appropriate bit assigment. So the question nol1 is the duration over 11hich such statistics should be computed.. Natur3.11y, if the duration is short, adaptivity of the system is iffiproved since short-term statistics are taken into acoount. However, the drahback would be the necessity to transmit an excessive amount of sidcinformation relating to the variance:> of the transform ooefficients. On the other hand, statistios oomputed over a longer period would result in less side-information, but then the system beoomes less adaptive.!-lany other systems "hieh involve the transmission of side-information are constrained by those h,o opposing faotors, so that a oompromise is necessary to ar.hieve the best overall decoded pioture quality and transmission rate. A major problem wi+h many adaptive transform ced.ing systems is the resulting non-uniform data rate Hhich, in certain cases, may vary considerably from image to image. Such a situation poses a serious challenge to the economic and efficient use of channel capacity and although buffering is possible, it represents an additional complexity. Despite this drawback it does appear that the achievement of high

146 1 31 levels of data compression using adaptive transform coding techniques necessarily involves variable data rates. Hence the only option is to exercise care in designing an adaptive system 1'ith a data rate which, if not totally uniform, is certainly less erratic so that the actual data buffering needed is very elementary and inexpensive. The above three considerations imply that great care must be taken in designing an adapt ive transform coding system. In the follohing section, the work Hhich has been performed on data compression of viewphone-based pictures (Le. head and shoulder scenes) is discussed. The objective is to design a system (or systems) which requires as Imf a transmitted bit rate as possible, consistent Hith a satisfactory fidelity standard, and encompassing the considerations mentioned above. The simulations are described in h;o sections, the first on two-dimensional analysis and the second on three-dimensional analysis, Hhere tho- and three- dimensional adaptive coding techniques respectively are employed. 'l'he implications of the above three conflicting considerations are also examined in the light of the performance of the various algorithms simulated.

147 TWO-DIMENSIONAL ANALYSIS The present work is based on the assumption of the existence of some degree of correlation between the energy of the various coefficients. Conceptually this may seem contradictory, since image transforms are in tended to convert the irrage vector into a a se t of uncorre la ted coefficients. Jlmlever, on closer examination of the energy distribution of the transform coefficients (Figure 5.1) a general decrease in energy with increasing order of transform coefficient suggests a certain amount of truth in the assumption, and, in fact, the use of recursive quantization techniques by Tescher, et al(40,4 1 ) lends support to this. Basically h;o types of adaptive transform coding systems have been examined and these are shohn in Figures 5.9 and Essentially the difference in the t"lo systems is the manner in "hioh the estimated energy of the transform coeffiients is derived. To achieve data compression, the estimated energy provides the basis of adaptively selecting and quantizing the transform coefficients. It is seen that in the system shown in Figure 5.9, side-informati.on regardi.ng the mean energies (E(x,y) is necessary, \,hich thereby leads to an increase in the transmitted. bit rata. It is agsumed that the encoder for the mean energies is sufficiently accurate for a good representation to be obtained at the receiver. The energy encoder will not be dealt with in this analysis. On the other hand, the system shown in lligure 5.10 requires no side-information. Although more elegant, it may mean that the

148 f (x,y ENERGY E(m,n) ENERGY r- DISTRIBUTION MODEL LING ENCODER I FORWARD COEFFICIENT AMPLITUDE TRANSFORM SELECTION NORMALIZATION! t, Q I E'(m,n) MULTIPLEXE R,..- ENERGY ENERGY r- ESTIMATION ESTIMATION E'( m,n) UJ UJ f(x,y) INVERSE TRANSFORM COEF FIC lent,reconstruct! DEMULTIPLEXER CHANNEL El m,ni ENERGY DISTRIBUTION MODEL FIGURE in adaptive transfol'in coding system (1) (Side-information required)

149 ENERGY ESTIMATION E'( m,n) t fix y) " FORWARD COEFFICIENT A MPLlT UDE TRANSFORM SELECTION NORMALIZATION Q w... FIGURE An adaptive trar.sform codine system (2) (No side-information required)

150 135 system is less capable of achieving a high fidelity reconstructed image. Such considerations are taken into account in the design of various systems. This analysis is devided into three parts namely 1.Transform coefficient energy distribution and block quantization 2.Adaptive coefficient selection 3.Quantization effects and t,lo-dimensional adaptive transform coders The data source consists of the "GIRL" picture shown in Figure 5.2,,-,hich is a head and should.er scene typical of viewphone pictures, having 256 lines with 256 pels per line. It is segmented into blocks of 16 x 16 pels and a two-dimensional fast discrete Cosine transform(15) is implemented. Various adaptive algorithms are then simulated using these 16 x 16 transform blocks. The percentage normalized mean square error as defined in Eqn. 5.2 is utilized to assess the resulting quality of the processed images. The efficiency of a coding algori thm it> implicitly expressed. - \ l.n... le resulting bit rates, i.e. the number of bits per pel, for the case where the images have been quantized. In the unquantized case the effectiveness of data compression is expressed in the coefficient compression ratio which is given by coefficient compression ratio Total number of cofficionts No. of transmitted coefficients Informal visual assessment of the processed images is also carried out.

151 Transform Coefficient Rnerv.v Distribution and Block quantization For a transform coefficient a(m,n), ;hose ener,:;y (or its estimate) is given by E(m,n), the number of bits, b(m,n), required to code that coefficient is set prcportional to the logarithm of E(m,n), i.e. b(m,n) = INT [ 10g2 [E(m,n)/D ;} ] bits (5.10) where D1 represents the distortion criterion given by (5.11) E is the mean energy of the image data source and k1 is a mean threshold constant that determines the required fidelity in the reconstructed image. '.'hen b(m,n) is le.s than 1 bit the coefficient is set to zero. It is thus seen that, in order to d.etermine the number of bits required to code a coefficient, a knowledge of its expected energy is necessary. The analysis of energy distribution models for the transform coefficients is shown diagrammatically in Figure A total of five energy distribution models has been examined, namely 1 Mean energy ditribution model 2. 2-D directional acti.vi ty-related log-ari thmic model 3. 2-D circularly symmetric scheme ( 1 ) logarithmic model 4. 2-D circu.larly symmetric scheme (2) logarithmic moc.e D logarithmic model These models may be distingui.shed into two cat"lgories. In the first

152 1 37 category (f.1odel 1) the energy distribution of' the transf'orm coef'f'icients is derived directly f'rom average statistics of' the coef'f'icients themselves. In the second category (Models 2-5), the energy distribution is modelled according to a simple and f'ixed set of' rules which have been experimentally determined based on empirical data and analysis. Nodels in the second category are, strictly speaking, approximations to those in the f'irst category. ENERGY DISTRIBUTION MODELS DIRECT STATISTICS APPROXfMATIONS MEAN ENERGY MODEL 2-DIMENSIDNAL LOG t10dels 1-DlMENSIONAL LOG t10del DIREC T10NAL ACTIVITY - RELATED CIRCULARLY SYMME TRIC (1) CIRCULARLY SYMMETRI C (2) FIGURE Analysis of energy distribution models 'ean enerr:y distribution Hodel The mean energy of each coefficient a(rn,n) is computed as E(rn,n) 1 N = E N i=1 2 a (m,n) (5.12)

153 138 where N is the total number of blocks over which the mean is computed (in this case, a picture frame). Having established the mean energies, the bit allocation map is obtained using Eqn A typical bit allocation map is shown in Figure 5.12 for a 16 x 16 nct block. Coding of the transform coefficients then follm{s and the effect of such an energy distribution and block quantization is examined for various values of k1' the threshold constant. In addition, in deriving the bit allocation map a limitation is imosed ' , ' : o 0 o 0 o. 0 o o o 0 o 0 o 0 o o o o o 1 o 0 o 0 o o o 1 1 o 0 o 0 o o o o -0 o 0 o 0 o FIGURE Bit allocation map for 16 x 16 DCT block

154 139 on the maximum number of bi ts alloled for the transform coe fficien ts, thus, the d.c. coefficient may not, have more than 8 bits and'the a.c. coefficients more than 6 bits. The mean energies are also used to normalize the coefficient amplitudes prior to quantization. Since the quantization of the d.e. coefficient is carried out assuming that it has a Rayleigh pdf, the coefficient is scaled by a factor of je(0,0)/2, while for the a.c. coefficients \1ith a Gaussion pdf, the scaling factor is /E(m,n), for m, n = 0, 1,, 15 except when m = n = Energy Distribution Logarithmic Hodels The models to be considered here differ from the mean energy distribution model (Section ) in that these models are approximations to the mean energy distribution model. 'l'he statistics of the coefficients are analysed to derive a simple mathemati.cal expression for the energies of the transform coefficients. In all cases a linear relationship is sought since it is easiest to utilize. In trying to establish an approxi,mate model for the energy distribution of the transform coefficients the relationship bett-ieen the coefficient enereies and their order is examil'ed. From Figure which shows a plot of the energ'j distribution for the "GIRL" picture, a general decrease in energy with increasing coefficient order is observed, and this serves as a basis for ful't}1er analysis. Figure 5.13 sho"s a plot of the mean energy against the variable X,

155 ,...., ,... ; J." 1 :l" 10.0 QJ d w VI en 0 --' I""CL. F l-:::c :::; " 1= 1;::ji':Cil:7: I, "'", " LW 1.0 f 1::-:-::-: =r. ",. F='.. F" ;:Oo::.::)...-!... c:cr::::: I::cc;::: j::'.!-- :::, ,... 1 _... 0 I.. j:o'ih!::i;'i', 1"::;':;':"UYi il'" ';:'i"rl';'i',":, :;:','.'!,i,' 01 i',:,i" '"i:", L-. X(Log scale)_. FIGUHE Plots of meim energy against distance fae tor X

156 " c:::,"h' :'. ',("C.1:0:, I:::::;:,:',"8l ' "'c' : ':: '. I:::: 'lcvij:....,'.., :c... " i'5;i " ",' '. 1""1 I' +.'-.-..",i"'" 'I'" ",,,.'.... 'f'. ':.',.: ',;e'.:.:,. "',, ' -. it' :',' :., :.:',:'...,...,+ " ':+;"'cl;. Tc,! 10.0 'i.' "'1-,,: ; ";;: 1,...,"... w '" u v> 0\ o...j w ;" ::;: :;;1"1:;-.' " ' : :,,=, 1.,"":::: ' " 111,,.=:'''' ':". -==:i:.,.. ',,::::;... I"'"'''''', ""'-1", : : I" 0.1 f' '. '7. i ;...,.., :.,::::.. '1:::: I,, :1"" ';4:::. ::.:.:. ;"..," ::: '1:.:: c....,.:1 :.,. 'I:": 'I:'"!-' 10 : cc: I:::: '::::..,,,,, 1; :: ':, r:::': :=I::::: i (Log scale) -.: I:Co::::, 17'+:: I :'.:"1'",-.-' 1 : '::-: : FIGURE Plot mean energy against distance factor i

157 142 which is the distance of coefficient a(m,n) from the first. or d.c. coefficient (e.g. a(4,3) Hill have X = / = 5). Plots for the first ro\;, column and diagonal are sho,m. Figure 5.14 illustrates the relationship betheen the mean energy and the variable i, which is the distance of any coefficient from the first d.c. coefficient as scanning takes place along the diagonal path shown in Figure In both cases, linear curves have been approximately fitted (no regression analysis has been used) and the point of this exercise is to indicate the possibility of deriving a sim?le relationship behleen the mean energy and the variable X (in the two-dimensional case) and i (in the one-dimensional case). I t is seen that the fi t is reasonably satisfactory for most of the range of the variables X and i, For higher values of X and i the fit deteriorates, but m n / / / V IV IV / / / / / V V 1/ V L L 1/ / V V / / / /,/,/ 1/ IV V 1/ 1/ './ V V V / / / / 1/ / V V V L L V V V V V / / /' L / V ll V / / V V 1/ V / / / / / V V V V / / V V Vl/ / / / / / V V V V / V V V V IV / / / / / V V V V / V V IV IV IV / / V / V V V V / V 1/ 1/ i/ IV 1/ / / l/ / IV V V V / l,.t' L V V V V / V V / V 1/ V k": / V / / V V V / V V / / / V V / V / V V V V / V V / 1/ /' V V / V 1/ 1/ 1/ V 1/ / V V / / V V V V 1/ / V V V V V V V / V V V V / V 1/ 1/ 1/ V 1/ V V V / V 1/ 1/ V " FIGURE One-dimensional diagonal scanning

158 143 this is not a major problem since when data compression is considered the higher order coefficients are normally neglected anyway. Thus the simplicity of these approximately linear log-log relationships between E and X ( or i) is attractive, and in the folloliing sections, validity of these arguments are tested with four proposed models. Approximations are only derived for the energies of the a.e. eoeffielents. It is assumed that the d.c. coeffleient is ab-lays transmitted with a fixed number of bits, in this case Two-dimensional Directional Activity-related Lop: 110del Although the correlations in both the horizontal and vertical directions of real images are relatively high, they are rarely identical. The mean energy of the a.c. coefficients adjacent to the d.c. coefficient, i.e. g(1,o) and E(O,1), give an indication of the activity, or energy distribution, in the horizontal and vertical directions respectively. Based on the observations of Figure 5.13 and the arguments presented above a generalized model of the a.c. transf9rm coefficients is proposed ln [E(m,n)] = C 1 ln [ X(m,n)] + C 2 (5.13) for m, n = 0, 1,, 15 except m = n = 0, and where C 1 and C 2 represent the slope and intercept for the linear log-log relation bet\<een the estim,,-ted energy E(m,n) and the distance factor X(m,n). X(m,n) is related to the order (m,n) of the transform coefficient

159 144 according to (5.14) 'Ihere A1 and A2 are the directional Heighting constants, and A3 is a normalising factor. The follohing hold for A 1, A2 and A3 : A1 = 1/1n[E(1,O)] (5.15a) A2 = 1/1n [E( 0, 1)] ( 5.1 5b) A3 = [In [E(1,0)] + In[E(0,1 )1] (5.15c) C 1 and C 2 are inter-related, and the problem is to determine the end points EO' EN and X O ' X N as shohn in Figure In[E O 1 In[Xl- FIGURE Logarithmic energy distribution model

160 145 Hence, C 1 = In (EN) - In (EO) In (X N ) - In (X o ) (5,17) C 2 = In (EN) - C 1 In (X N ) (5,18) These values of EO' EN' Xo and X N should result in a model with the lowest number of bits required to code the transform coefficients for a given fidelity criterion, of which the measure is the percentage normalised mean square error (nmse). In the present case the folloldng condi tions hold, X N = X(15,15) (5,19;:;) EO = max [E( 1,0), E ( 0, 1 )] (5, 19b) Xo = [x corresponding to the coefficient giving EO] (5,19c) EN is a system parameter which is varied to yield the energy distribution surface which results in the minimum number of bits required to code the transform coefficients subject to a given fid.eli ty limit. In these experiments the best choice of EN ;:hich achieves these conditions is determined and used in subsequent simulations where such a model is employed. The number of bits used for coefficiet a(m,n) is determined using Eqn for a given distortion D 1, 'wo-dimensional Circularly Symmetric LOPjarithmic l"lodels Circularly symmetric energy distribution models have also been

161 146 examined and their performances compared with the directional activity related model explained above. These models are termed circularly symmetric because the energies are distributed in a circularly manner about the origin (0,0) and they are symmetrical about the two directions m and n. Such a model is illustrated in Figure I ri le) Inlm) In In) FIGUHE Two-dimensional circularly symmetric logarithmic model The generalized circularly symmetric model has the form In E(m,n) = C 3 In [Y(m,n)] + C 4 (5.20) In (EN) - In (EO) ""here C = (5.21a) 3 In if450'\ ') <:;.\') C 4 = In (EO) (5.21b) Y(m,n) =/m 2 + n 2 (5.21c) In this case tio schemes for EO have been used

162 147 Scheme 1 - EO = t [ E( 1,0) + E( 0,1)] (5.22 ) Scheme 2 - EO = max [E(1,O),F,(O,1)] (5.23). - The first scheme used the average of the energy of the first t\w a.c. coefficients adjacent to the d.c. coefficient, while the second scheme uses the maximum value of either E(1,O) or E(0,1). Again EN is a system parameter "hich is experimentally determined to produce a system "hich permits encoding of the transform coefficients with the least number of bits to achieve a given quality in the reconstructed image. Such values of EN will then be used in all subsequent experiments where these models are employed One-dimensional Logarithmic Model The results of Figure 5.14 suggest that the energy distribution of the coefficients of a two-dimensional transformation may be modelled on a diagonally scanned one-dimensional basis. Such a model is attractive sinoe its one-dimensional nature suggests that it may be computationally less complex and conceptually easier to visualie. The diagonally scanned one-dimensional logarithmic model takes on the general form : In[E(i)] = C 5 In(i) + C 6 (5.24) ;there C 5 = In (EN) - In(255) In (EO) (5.25a)

163 148 and i = 1, 2,, 255 is the distance factor along the scan direction. In addition, EO = max [E(1,0),E(0,1)] (5.26) and EN is the system parameter to be determined as before. The bit assignment for the transform coefficient again makes use of the rate distortion relation given in Eqn Results and Comments The "GIRL" picture sho\.jn in Figure 5.2 is used as data source. It is segmented into two-dimensional blocks of 16 x 16 picture elements and transformed using the DC'],. The energy models described in the previous sections have bcen investigated using these transform coefficients. Figures 5.18 to 5.22 plot the percentage normalized mean square error versus bit rate for the various models employed. In relating subjective picture quality to percentage normalize mean square error (nmse), it has been found that for quantized coefficients, a value of allout 0.1 % mi1se results in decode 1 pictures of good quality, where, although degradations are present, they are not immediately noticeable under "normal" viewing conp.itions. Although this relation is not totally reliable in all instances, it is found to be usually applicable and as such has been used to judge the fidelity of decoded pictures.

164 149 Generally it is seen that the quality of the decoded picture increases (i.e. the nmse value drops).lith increasing bit rate. To obtain an overall impression of the comparative performance behleen the various models, the respective bit rates for the case of 0.1 % nmse are listed in Table 5.3. lodel Mean energy distribution 2-D logarithmic directional activity-related 2-D logarithmic - circularly symmetric Scheme ( 1 ) 2-D logarithmic - circub.rly symmetric Scheme (2 ) Bits/pel l-d logarithmic - diagonally scanned 1.42 TABLE Results of 2-D energy distribution models From the above table it is evident that the best model is the mea.n energy distribution model since for a given % nmse it achieves the lo>lest bit rate. The circularly symmetric Scheme (2) model ancl the directiona.l activity-related model follo>l closely behind whilst the diagonally scanned one-d.imensional logarithmic model in the least efficient. These results are not entirely unexpected since the 199arithmic models are only approximations to the mean energy distribution model. Thi::J means that at best 'he logal'i thmic models can only equal the performance of the mean energy distribution mcdel. However, the use of the mean energy distribution model involves the need to relay the mean energy of all the transform coefficients to the receiver prior to decoding the coefficients. This represents

165 150 a significant amount of side information.which must be weighed against the scheme's coding performance. In order to assess what this might involve, consider a hypothetical case "here the transform. coefficient energies for every picture frame of 256 x 256 picture elements are transmitted. Using a 16 x 16 DC'f block there is a total of 256 coefficient energies and if each of these is described by a 1 O-bi t ;Iord on the average, say, then there "'ill be a total of 2560 bits of side-information per' frame. Stated in another way this means an increase of 2560/(256 x 256) or bits/pel in the transmitted bit rate. This may be reduced by the use of variable length Coding( 101,111) or lluffman coding( 9 1 ). Hence the quantization of the transform coefficients using the mean energy distribution model will require a total of not more than 0.93 bi ts/pel, Hhich is still less than the best tho-dimensional logari thmic mode 1. Of course, error protection of the sideinformation is necessary, so that the resulting total bit rate may approach that for the ti;o-dimensional logarithmic mode 1. From the results the appropriate choice of EN can also be determined. }'or the two-dimensional logarithmic mode Is, EN = 0.05 appar8ntly provides a satisfactory compromise between quantization error and bt rate. This is clearly seen in the directional activity-elated model where for EN < 0.05 the quantization error is higher at all bit rates, whilst for EN> 0.05 the quantization error has been reduced but at the expense of higher bit rates, thus resulting in curves, parts of which lie above that for EN = 'or the other logarithmic models the asymptotic nature of the curves indicates that no significant advantage can be gained in terms of lowering

166 cu VI E c if. C.Q , , BIT RATE (BITS IPEL) FIGURE Results for Nean EnerGY Distribution Hodel

167 (a) EN =0.01 (b)e N =0.03 (c)e N =0.05 (d) EN = 0.07 & V> E c a 0.07 b c d r-----r , o BIT RATE (BITS/PEL) FIGURE Results for 2-D Logarithmic Directional Activity-related energy distribtion Hodel

168 (a) EN= (b) EN= 0.03 (()EN , '" E c: a 0.07 b ( ,-----,-----,----'1 o BIT RATE ( BITS f PEL ) FIGUm Results of 2 -D Logarithmic Circularly Symmetric Scheme (1) 3nergy Distribution Nodel

169 (a) EN= 0.01 Cb) EN= 0.03 (c)e N =0.05 (d) EN = 0.Q cv <I) E c a 0.07 b ,----.,------; , o BIT RATE ( BIT Sf PEll c d FIGURE Results for 2-D Logarithmic Circularly Symmetric SGheme (2) Energy Distri.bution Hodel'

170 (a) EN= (b) E N =0.05 (c) EN= 0.07 (d) EN = a w Cl> E c: b c d ,.- ---, o BIT RATE ( BITS I PEL) FIGURE Results of 1-ll Loearithmic DiaGonally Scanned Energy Distribution t jodel

171 the bit rate for values of E greater than about 0.05 in the two N dimensional case, and 0.09 in the one-dimensional case. In contrast to the mean energy distribution model, \'Ihere the mean energy of all the transform coefficients must be relayed to the receiver prior to decoding the transform coefficients, the use of logarithmic models leads to a significant reduction in sideinformation, since the receiver only needs to know the slope and intercept of the model. Thus weighted against the mean energy distribution model is the simplicity of the two-dimensional logarithmic model. Having considered the requirements of side-information and t.he comparative performances of the various models, the mean energy distribution model is still the most attractive in terms of coding efficiency. The marginally lower performance of the two- dimensional logarithmic models justify further consideration of them, in particular of the directional activity-related loodel, because of its simplicity and versatility. A value of 0.05 has been chosen for EN in implementing this model. These two energy distribution models are examined further in conjunction with adaptive coding techniques described in later sections Adautive Coefficient Selection In transforming a hlo-dimensional block, a collection of transform blocks with varying degrees of activity is obtained. This activity manifests itself in the distribution of the coefficient amplitudes

172 157 and energies. Fixed zonal sampling does not take into' consideration such variations and thus leads to less efficient data compression. Adaptive transform coding takes into account the need to code each block and coefficient appropriately to achieve high data compression rates. In the previous section (Section 5.5.1) various energy distribution mode Is have been examined in re la tio!! to the ir ability to genera to appropriate bit allocation maps to quantize the transform coefficients. Such energy distribution models are derived, directly or indirectly, from time average statistics and so do not account for short-term changes that may occur from b).0ck to block. Once the energy distribution is determined the bit allocation map is fixed for all blocks over which the average statistics are computed. Since the activity may change considerably from one block to the next, it is apparent that a reduction in bandwid.th may be forthcomirlg if coefficients of insignificantly small amplitude in 10Hor activity blocks are neglected, despite the fact tha.t bits have he en a.llocated to these coefficients. In this section various schemes for adaptively selecting the coefficients are examined. Such schemes do not make use of any additional statistical information that might constitute side-information. The required decision criteria are derived entirely from the transmitted coefficient ampli tude s. The assumption that has been made to implement adaptive coefficient selection is the existence of some correlation in the distribution of the transform coefficient enere-ies. The validity of this

173 158 assumption is now examined through the 2.nalysis of results in relation to the possibility of achieving effective redundancy reduction. In addition, the extent to which such ideas may be practically utilized is considered. The strategy for determining whether or not a specific coefficient needs to be transmitted is to examine an estimate of its energy derived from the energies of coefficients in its vicinity. This estimated energy is trea.ted in a manner different from the energy defined in Section For the present, this estimate is used merely as a basis for enabling a decision to be made as to ",hether a coefficient is to be tra.nsmitted. Such an estimate is compared with a predetermined threshold "'hich is related to the required fidelity of the decoded image, i.e. if E' (m, n) :::: D2 then truncate where E'(rn,n) is the energy estimate and D2 is the distortion factor given by E mean (5.28) E is the mean energy of the original data source and k2 is a mean distortion threshold factor. If this estimate is below the threshold then the coefficient is considered to be insignificant and is not transmitted. In add.i tion, since coefficients of higher order than the first truncatec1 coeffi-

174 159 cient are usually of even smaller energy, these higher order coefficients are not transmitted either. In this way each transform block is individually examined and a variable zonal mapping results depending on the amount of significant energy spread, i.e. the activity. The problem is now one of choosing a representative and accurate energy estimate to enable proper selection of the transform coefficients. Several schemes are suggested and the initial task is to compare their relative effectiveness. In this analysis the coefficients have not been quantized as the initial intention is to examine the validity of these schemes in selecting the appropriate coefficients. This analysis is shown diagrammatically in Figure ENERGY ESTIMATION I- - ENERGY ESTIlATlON f 0-- FORWARD COEFFICIENT r- f-<.>-.,... COEFFICIENT INVERSE TRANSFORM SELECTION RECONSTRUCTIO f--- TRANSFORM f FIGURE Adaptive coefficient selection Scheme 1 In this scheme, the coefficients obtained by discrete Cosine transformation are scanned in a diagonal fashion i:'.g sho,m in Figure The first three coefficients are always transmitted as they are necessary to start the estimation process.

175 160 0(0) 0(1) o( i + 1) 0(2) 0(3 ) FIGlmE One-dimensional diagona,l Sca,n The estimated energy for the ith coefficient is given by for i = 3, 4,, 255, and where a(i-l) and a(i-2) are the a"plitudes. of the (i-l) th and (i-2) th coefficients respectively Scheme 2 This scheme is similar to Scheme 1 except that the estimated energy is"now given by the average of the squares of amplitudes of the three previous quantized coefficients, i.e. for i = 4, 5,, 255. In this case the first four coefficients are always transmitted.

176 Scheme 3 The coefficients are again scanned diagonally but the estimated energy is given by for i = 5, 6,, 255, and the first five cce fficients are ahmys transmi tted. Unlike the previous three schemes where the energy is estimaten in a one-dimensional manner, Scheme 4 involves estimation on a t,iodimensional basis, with the exception of the firot row and colunil1 in the transform block. The transform block is scanned horizontally from left to right, and when Ineqyality 5.27 is satisfied, tlat coefficient and all coefficients to its right on the same row are se t to zero. Es timation then s tarts anew on the next ro>;. This scheme is illustrated in Figure In- this case the first four coefficients a(o,o), a(0,1), a(1,1) and a(1,1) are always transmitted. 'rhe energies of the coefficients along the first row and co lumn are estimated prior to the consideration of the energies of other coefficients within the block. Fer the first column,

177 162 HORIZONTALLY SCANNED- 0(0,0) 0(1,0) 0(m-1,O) o(m,o) (0,1) 0(1,1) I I I I I o(o,[)-1) 0(m-1,n-1) 0(m,n-1) o o(o,n) o 0 0(m-1,n) o(m,n) 0-, FIGtmE If'wo-dimensional energy estimation (5.32b) E' (O,n) =.g 3 L j=1 for n = 4, 5,, 15 (5.32c) and for the first row, E' (3,0), = -3" [a 2 (1,0) + a 2 (2,O) 2 +a(1,1)] 3 EI(m,o) =.g L j=1 a 2 (m_j, o) m = 4, 5,.., 15 Truncation occurs vertically for the first column and horizontally

178 163 for the first row when Inequality 5.27 is satisfied; For the remaining coefficients in the block, where m, n = 1, 2, equal to 1..., 15 except for the case "hen both m and n are Scheme 5 This scheme is essentially identical to Scheme 4 except that the block is scanned vertically from top to bottom. The estimated energy from each of the five schemes suggested is then compared against a threshold (Inequality 5.27) >lhich is assignd experimentally and is a function of the required qua.li ty in the decoded imaee, "hich in this case is indicated by the % nmse. If the estimated energy is above this threshold it is considered that the activity in the vicinity of the coefficient is sufficiently high to warrant the transmission of that coefficient. On the other hand, if the estimated energy is beloh the threshold, then that coefficient will not be transmitted and all subsequent coefficients along the scanning path will he set to zero. The excepticns are Schemes 4 and 5 where energy estimation continues from the succeeding ro>! and column respectively...

179 Results and Comments The "GIRL" picture of Figure 5.2 is transformed using the DCT in blocks of 16 x 16 elements.. The above adaptive coefficient selection schemes are then applied to these blocks of data. The results of the five coefficient selection schemes are shown in Figures 5.26 and Figure 5.26 contains plots of % nmse against coefficient compression ratio (see Eqn. 5.9) while Figure 5.27 shows the performance of the various schemes plotted as % nmf;e against the threshold factor k? (see Inequality 5.27 and Eqn. 5.28). All these curves are obtained by varying the threshold factor k The 2 % ninso and the number of transmitted coefficients are subsequently measured. The latter value is used to compute the coefficient compresion ratio. It is seen from I<'igure 5.?7 that as k2 increases the % ninse falls, i. e. the picture Quality improves. Referd.ng to Inequality 5.27 and Eqn. 5.28, this is not surpri>:ing since a. higher k2 will mean a. 10>ler distortion criterion D2 and hence more coefficients I rill be transmitted prior to truncation. An interesting point is that the % nmse falls exponentially Id th increa.sing k Comparatively, 2 however, to achieve the same quality it is seen that the required threshold factor k2 varies considera.bly from one scheme to the next. This implies that those schemes which require higher k2 values, e.g. Schemes 1, 2 and 3, arc over-sensitive to changes in local activity within a block and as a result tend to truncate prematurely, thereby leading to excessive image degradation and higher nmse. This result may also be taken to mean that the energy

180 165 estimates of such schemes are insufficiently accurate) to permit efficient coefficient selection. In addition, another point to note is that if Schemes 1, 2 and 3 are forced to achieve the same % nmse by increasing the threshold factor k2' then from Figure 5.26, it is seen that the coefficient compression ratio is reduced and hence no advantage is gained in terms of achieving higher data compression. From Figure 5.26 it is observed that for a given % nmse, Schemes 4 and 5 perform better than Schemes 1, 2 and 3, since the former two schemes are capable of achieving higher coefficient compression ratios. Subjectively, it is noticed that for a nmse of less than about %. the degradations in the decoded image are hardly noticeable. On this basis the obtainable coefficient compression ratio is in the region of 5 for Schemes 1, 2 and 3. and 7 for Schemes 4 and 5. The similarity of the results of Schemes 1,? and 3 (Figure 5.26) is due to the fact that the truncation pro(;os8 is identical in all three schemes. This means tha.t for a given number of coefficients transmitted (i.e. the same coefficient compression ratio) then by virtue of the unique diagonal scan path the same % nmse >/ill result in all three schemes. On the other h8.j1d, for Schemes 4 and 5, the reasoning behind their similaritios is quite different. Since these t>io schemes ollly differ in the direction of scan of the energy estimation process, the similarity of their results indicates the symmetry of statistical properties in the horizontal and vertical directions. The superiority of Schemes 4 and. 5 in selecting the 8.I'propriate

181 166 '" N, Ul 0 -oz- -rl -to V> V> QI QI -a f.j E E QI QI..c:..c: C u u 0 (/) (/) -rl to..: Q) " 1-; E "" 0 () -to 0 C Q) l- -rl N «0 0: -rl Z... '--< B CJ (/) 0 (/) () LW 0: -to Co Cl. :>: " 0 -::J u r: l- t;;) Z r: u, u <0 u. "'" '2 u. LW C 0 u -" c> "" C :n +> 0 p., C> <0 -a...t N H - CO 0 0 <:> : aswu %...t '-!l "', "' ::;: CC -

182 ;,.:1. :' ":1',_ 1 ::1': '1,,;, 'r o, 1,. " t' I". I.'" I '. 1 4 l ",'1 "I,_:_,_--,_!J:' I.:.':' J'. :;;"\1".",,J,;:;:.. 1 "." :::/ii:.,.,'""'" ":':'1:';";.,,.'. ",,.".,.,J,'" "" 'i,:.,h,.".,. -, }, I i: "t:\ii!;.':: 11,.:I:JHi:t 'i ), J" 1 1Z 'H!.10:;; I U:'I)iN2'I-'. r_i'!"it:" '; I.,,:,.-...' r;-l;- _; 'c':s,.1 I ""," IT],. I " "...,'",...,.. L ".1 "."" ':i c, : ; i : ''-H.,;!"" I :;::,i,;' :i:'::: '.': IF.'.,.. :',,::,,':.,7'''''''1',;."-1 HE,,011 l,'li,:l:;:"!'i::'.1!f.'.'.i '.,!,-:II.,'." :':;-." I --:. l!.":'' 1:''1.. ; ;1" ; J-=-1-+-J:'; :;-: 11:' j:jl:!'! :."""",,,,,,.:;:.1;;1 r!ll::'i -I,-::;..;.*l:::'!:::: '::.".,.,.::.:,::::,r.1-r1 i 1-"-'".. ;,l"di:;;;;.,'.'",1";:"""::': "". I.:,',.I.1,,-,-"",_,_; ;.::; t ::.!...: : ji'+"'!' _u:._. l :r,;i,:'!:;j:,...".:.. ',i:' i;;,,:!! i ::iii,::ru-:; i! jl ti:':: f. :il :+I!!!;I::'IJUj-'- j-fi/ ij-i1m,:!ill! fi] i,k 'J,:H-JJHW'Wt mtf,ifr8 "--!li; L:.:! L:.uili:::I:g:r.:,:--,--,-,-. :.::'-'-W-:.::.: I ;.,;",,,,::.:-'.H.;Je.,!+ltil:l,-",I,j...,,,, "".,,,d;,:: "'''''':Ig.l.!.i.i. )1;,,;,,; :::: :U:!11-II:I+H*Jlm;,--::!:'IHiil1Ii'*k1 mmii,1\:1!::: ffil lllillm OB :-:=7:"1:'1=1'''',,,,.. -,... '\=l'."". """."r-;.,,'i''''-',,-'".".... "". ",,,. ' """"... =. ""I"'''' w. " 'IT" ".. """,, ---._,,,,,,,,.. "... "., '... ".' -".--'-''''''''''.".","" ""1""'''''''''''''' """""'1""!1,!'i!il'',T!mYbiill ;(;I;;,!:i:,;f2iinm1: 'i:m :i,,i frr:,,g.06 rl' ±ill rootmti IT,-...m m er...., it!ili!!iti!,!l:i1,ni:m!':'i!: Ij!'Ii :;"mr iml:!;i::::'!8i :n:'m:.04.,.,4:-;"!".. "."... -,.-."" -:==-:,...""".,.".".. " """'1'1-'" "."" """"'''S'CElEME-l' ".. "";-c. ::'[1;:::;;;;::;"::; :;",',;,ei';; ;:'::::.:."":ii'!':!:;;::lj;:'! ',;:"',';:,;:,, "-"".: ',.,: ;: ':",,;;;F;::: :: I,,::':''':':::;:.' ; :";"; :--: :' ::;: :::: :;;: -.-,-:---:--:----:-;--:---;-- -:"i-:--: ;-;";';"; ::::.. --1:-:-:::::-:; ";--:-;-7'.:;: -:-:-; ;:: :::: -,,-. -:-:- ".-,-. i. -:"it,:-':-:-:-:-: ;,.:.;::.:-:.; ;-;-;-; ;-;-:-:- :-:-:- :;-;-: ""j""i""i""i"",... ",.. ""'''''''''''' "',Tl;.. '""""", \" "'" '\"'1" "1 1".,,,,,,,,,,,..,,,,...,,,,,,,,,,. _:':':1 i::: -:':::':21 -; :! -: r: i :: r::;: i::: ::::::: L.. -::±:tr(t:::: :::i :,;,., ':,-.,1-;: _to: :: ; -: I,::;, ;:!::: :::; :,!:!; I;:: :::' :::: ijjjht'j3ei; 2'1_!: l! ;),';.,...j....._.j "<': I : : : ':!; i I:'::!' >i:::::!i;t>: I \:1:::.1::, :, i;:: I i ' ij\hlsl*&ts,i'::::: ::/::' ::; :::,!::::.:.i' ;_CI.0 2 h_:,i 'WW I'i +j_!_l \-!+!1.l:Tr i!;w jilli:'jfl;:;(l!g thf-lffi- ' ]; l ITI."..,,1 r--- 1m - 11,1"", " '1"'" I'"'' l'''' I "'1 'I',,, 1 I I" 1"'1'1'1"1""""1"11',,,,,,,,,,, ::!.: -:-:-j -;-:-H:- ;::; j:: ;:f:,: ;::l: ll.r:+,-i::'-( ;;: ; ; ; : : T-i+ --hif:'\ i:"!f:} ij';;;:;: T:. tt :;;: 1'''1 I,1 1",1,1... '''''''I'... 1."".11"'1 1 "1.. ",."1,.,,,.,, "1, 1.. 1'''1''\1'''''''''''''''''1'' 1''''''1", 0" """, '" ",,,'," """ '" """ I, L...-.:..:...!_., 1., I"",.."''''..,',,11. '."" "1-,, Threshold factor k 2 - vrclih: ; t:,.';>7 - Pled,:. of " nmic nr:;-)inf,t thrcr.}lold fa.ctot k., "

183 168 coefficients may be attributed to the manner in 'rhich the energy estimation is performed., Because of the proximity of the nearest three: coefficients in Schemes 4 and 5 the correlation is expected to be higher and hence the energy estimate to be more accurate. Ho,rever, the mean distances of the coefficients involved in estimating the energy are grea.ter in Schemes 1, 2 and 3, since they are cons trained by the one-dimensional diagonal scanning process. This results in an estimate 'rhich is less accurate and leads to less effective coefficient selection. In addition, the diagonal scan process assumes a triangular spread in the energy distribution of the transform coefficients which may not be strictly true. Having considered the various arguments for the five schemes, it was decided to continue investigation of schemes 2,3 and 4 in relation to later simulations involving adaptiye coefficient selection Quantization Effects and 'ri rd-dimensional Adaut:lYe Transform Coders The choice of appropriate quantizers forms an important aspect of the application of transform coding techniques. The present \rork does not examine in detail the use of quantizers which are optimal for transform coefficient characteristics, but those coonly encountered, such as that due to Max(66) are used, since the development of efficient coefficient selection schemes is the principal aim. In the simulations, two types of quantizers are used. Based on histogram analysis (<lee Section , Figure 5.6) a quantizer

184 169 with a Rayleigh pdf characteristic is used for the first (i.e. d.c.) coefficient, where the decision and reconstruction levels are computed by an approximate method due to Algazi(72). A Hax optimum quantizer with a Gaussian pdf characteristic is employed for the other coefficients. The decision and reconstruction levels for the Gaussian quantizer for various number of bits are listed in Table 5.1. In using these quantizers the transform coefficient amplitudes are normalized prior to quantization so as to reduce the probability of overloading the quantizers. For the d.c. coefficient a(o,o), the normalizing factor is /;;;(0,0)/2 lhere E(O,O) is its mean energy. :;'or the a.c. coefficients a(m,n) where m, n '" 0, 1,, 15 except when m '" n = 0, the normalizing factor is 'E(m,n) where E(m,n) is the mean energy or its estimate. This process of normalizing the coefficient amplitudes is an attractive and useful method of effectively quantizing a wide variety of signals having the same pdf's but different energies. In this section some of the more successful ideas explored in Sections and are consolidated to derive a number of complete two-dimensional adaptive transform coding syster:ls. In Section five block quantization strategies were examined. These involved the modelling of the energy distribution of the transform coefficient::; and the subsequent generation of the bit assignments for the vari.ous coeffi.cients. Section dealt >!ith methods of adaptively selecting coefficients without recourse to additional statistical data that may constitute side-information. These adaptive coefficient selection schemes >!ere considered in the absence of quantization effects in order to facilitate a clearer und.erstand.ing of their operation.

185 170 a ENERGY DISTRIBUTION MODELS 2-D ADAPTIVE CODING SYSTEMS MEAN ENERGY MODEL 1A 2-D DIRECT IONAL ACVITY RELATED MODEL.1B 2-D CIRCULRLY, SYMMETRIC ( 1),.' I MODEL 1C 2-D CIRCULARLY MODEL 10 SYMMETRIC (2) 'j 1-D LOGARITHMIC MODEL 1E r--- MODEL 1F MODEL 1G ENERGY ESTI MAT ION & COEF F. SELECTION : : r r.... SCHEME 1 I,._..., Energy SCHEME estimate, used for SC HE M E 3... quantization SCHEME , proces s SCHEME 5 FIGURE S G Analysis of 2-D adaptive coding systems

186 170 b Figure 5.28 shows the various adaptive transform coding systems that are examined. The systems come about as a result of the application of various adaptive coefficient selection schemes to the different block quantization models. In addition, the possible use of the energy estimate to influence quantization of the coefficient directly is also examined Vean Energy Distribution l'lodel Hith /,dantive Coefficient Selection (l1odels 1A and 1B) The mean energy of euch transform coefficient is first computed over the entire picture frame and the energy and bit allocaticn mappinbs are produced as described in Section This method of bit assignment is then used in conjunction with the energy estimation and coefficient selection schemes explained in Section In this case, of course, the energy estimate used in the adaptive coefficient selection schemes is 'based on averag'es of the squares of previously quanti7.ed coefficient amplitudes in contrast to the use of unquanti7.ed coefficient amplitudes as in the earlier case. This means that, assuming errorless transmission, identical operations of enerey estimation are performed at both the transmitter and. receiver. The energy estimate g' (i) (Eqn. 5.31) or E' (m,n) (Eqns. 5.32, 5.33,"nd 5.34) that is generated is compared Yli th the threshold D2 defined in Inequality 5.27 and Eqn In the simulations the number of bits allocated to the d.c. and a.c. coefficients have been limited to eight and six respectively. Additional experiments (not discussed here) have indicated this

187 171 arrangement to be more than adequate for most purposes. Only results for adaptive selection Schemes 3 and 4 are presented since it has been shown in the previous sections that Schemes 1 and 2 are inferior, and Scheme 5 has approximately similar characteristics to Scheme 4. Another point to consider when such coding schemes are employed is the necessity of informing the receiver of the energy distribution of the transform coefficients. This consideration has already been discussed briefly in Section and is found to be only a minor problem in the light of the.efficient performance of the coding system ]'ixcd I.ogari thmic Enerc::y ( lode 1 with Adanb.ve Coefficient Selection (Hodels lc and ld) This adaptive coding scheme utilizes the hlo-dimensional energy distribution model described in Section (see Eqns ). The two-dimensional model has been chosen in prcfel'ence to the one-dimensional model because of its ability' reproduce the cnc:?:!;:i distribution of the transform coefficients ];lore accurately at lovier bit ra.tes. Of the various two-dimensional models examined, the directional activi t.y-rela.ted model is used here in orncr to to.ke account of differences in horizontal and vertical correlations in the image. The hlo-dimensional di rectional acti vi ty-rela ted logarithmic energy distribution model, and its associated bit allocation map, are used

188 172 in conjunction with the adaptive coefficient selection schemes 3 and 4 to achieve redundancy reduction. In these cases, transmission of side-information is also necessary :,ut quantitatively this is reduced significantly since only information regarding the slope and intercept of the energy model (i.e. C 1 and C 2 in Eqn. 5.13) need to be transmitted. As a result such an approach is very attractive since there is less side-information to be protected against transmission error and the effectiveness of such coding schemes, if proven to be satisfactory, will be greater than that of systems requiring more supporting side-information Adantive Estimated Enerp:v l1apning and Coefficient Selection (Models 1E, 1F and 1G) So far systems ",here the bit assignments have been predetermi.ned in one form or another prior to adaptive seleoticn of the trrulsform coefficients have been dealt Id th. These bi t assignments or mappi;;gs have been either generated from the mean energy distribution of the transform coefficients or modelled acoording to some straightforward linear relationship ",here the modelling characteristics may be manipulated according to the activity of the picture. The efficient use of an energy estimation scheme poses t>10 questions; The first is whether such a scheme does, in fact, give an indication of the activity in the vicinity of the particular coeffioient in question Rnd the seoond is that, if this is true, whether this energy estimate is representative enough to influence the quantization of that coefficient directly. Results in Scction

189 173 indicate that the proposed schemes allow the first question to be answered in the affirmative, and in this section, the possibility of using this energy estimate directly to influence quantization y normalization of, and bit assignment to, the coefficient to be transmitted is investigated. Experiments have been performed on three methods "'hich are variations of the adaptive coefficient selection Schemes 2, 3 and 4. Nodel 1E This model makes use of energy estimation Scheme 2 described by Eqn In this case, of.course, the quantized coefficient ampli tudes are used in the compu ta tion of tht? enere;y es tima te E' (i) instead of unquantizod values in the earlier case. To start the estimation process coefficients a(o), a(1), a(2) and a(3) (see l'icure 5.24) are initially transmitted ",ith a fixed and predetermined number of bits, namely, 8 for a(o), 6 for'a(1) and a(2) and 5 for a(3). To determine the number of bits b(i) to be used in the transmission of a(i) the rate distortion equation is used, but ",hen ti) is less than 1, all subsequent coefficient::; along the scanning path are set to zero. In this instance threshold D2 in Inequality 5.27 is not used and the entire decision of coefficient selection is based on Eqn and b(i) being less than 1.

190 17/,!odel 1 F This model makes use of energy estimation Scheme 3 described by Eqn The operation of this model is similar to that of Model 1E except that a(4) is also initially transmitted with 5 bits in addition to the first four coefficients. The bit assignment is derived using Eqn and the coefficient is selected and coded whenever b(i) is greater than or equal to 1. Bodel 1G Here' the energy estimation procedure follows that of the adaptive coefficie,nt selection Scheme 4. The four coefficients a(o,o), a(0,1), a(1,0) and a(1,1) are first transmitted I<ith 8, 6, 6 and 5 bits respectively. The energy estimate E'(m,n) is again compu.ted froin previously quantized coefficient amplitudes in contrast to the use of unquantizcd ampli tud.e" in the previous' case. From the enere;y estimate the bit assignment for the coefficient a(m,n) is derived using the rate distortion equation, i.e. b(m,n) = INT [-it 10g2 [ g' (m,n)/d11 + t] The direction of scan in a transform block and. the manner in!hich E'(m,n) is computed are similar to those described in Section , >li th the exception tha.t truncation occurs I<hen b(m,n) is less the.n 1. The threshold D2 and Inequality 5.27 is not used as a decision criterion for truncation.

191 175 From the above descriptions of Models 1A to if, it is seen that Hodels 1A to 1D are of the type sho,in in Figure 5.9, where sideinformation is a necessary component in the reconstruction of the image at the receiver. Hodels if. to 1G, on the other hand, are of the type sho'in in Figure 5.10 \-lhere no additional side-information is necessary. In the subsequent analysis, the ability of these systems to achieve high data compression is examined. It must be noted, however, that the requirements of transmitting the sideinformation is not considered here in this analysis, thus all quoted bit rates do not include that figure RaSl.1ts and Comments' Figures 5.29 to 5.33 ShO,1 the results of the seven adaptive coefficient and selection algorithms described in Sectiono , and In models 1C and id where the directional activity-related logarithmic energy distribution model is implemented, a value of 0.05 is used for EN (sec Eqns and 5.18). It can be seen from these figures that a hie-h degrc,e of data eampression can be achieved and in all cases bit rates of less than 1 bits/pel result in decoded pictures of ood quality. For example, for 0. 1 % nmse, the follol-ling data rates are required: 0.65 bits/pel - Hodel 1A - mean energy mapping with adaptive coefficient selection Scheme bits/pel - J'-lodel 1B - mean energy mapping with adaptive coefficient selection Scheme bi ts/pel - l10del 1C - fixed logarithmic energy model wi th adaptive coefficient selection Scheme 3

192 bits/pel - Hodel 1D - fixed logarithmic energy model with adaptive coefficient selection Scheme bits/pel - Hodel 1E - adaptive estimated energy mapping and coefficient selection Hodel bi ts/pel - Hode); 1F - adaptive estimated energy mapping and coefficient selection Model bits/pel - Model 1G - adaptive estimated energy mapping and coefficient selection Model 3 In }lodels 1A to 1D the threshold f",ctor k1 determines the bit allocation map which is used for all the transform blocks (see Section Eqns and 5.11). An additional threshold factor k2 is used in the energy estimation and coefficient selection process (see Section Inequality 5.27 and Eqn. 5.28). The choice of k1 and k2 is critical in the encoding of transform coefficients for a given adaptive scheme. If k1 is too large then too many bits "ill be allocated to the coefficient prior to ad2.ptively selecting the coefficient, 2.lthough this could mean a reduction in quantization noise belm'l that level required for a given decoded picture fidelity. On the other hand, too 10" a value for k1 'Iill result in insufficient number of quantization levels to code the transform coefficient so that the energy estimation and adaptive coefficient selecti0n process is adversely affected, leading to greater mean square error in the decoded picture. The value of k2 is important in choosing the number of coefficients to be transmitted. Naturally, a low valt1.e of k2 Hill mean the selection of fewer coefficients and vice versa. The comments are

193 177 demonstrated in Figures 5.29 to For higher k1' improved % nmse values are obtainable but this is more evident at higher bit rates. At a level of 0.1 % nmse \'lhieh is considered to yield good deeoded picture fidelity, the asymptotic nature of the curves indicate little variation in bit rate with respect to the various k1 threshold constants. 'rhe variation of % nmse Hi th k2 has a negative slope - the lower the value of k 2, the greater the r-umber of coefficients transmitted. This leads to loler % nmse at higher bi t rates. At lower values of k1' the curves gradually f"latten out since when k2 k1 all the non-zero bit allocated coefficients are transmitted. l1ith regard to the type of energy distribution model used, there is little dif"feronce in performance between the mean energy model (l1odels 1A and 1n) and the logarithmic model (11odels 1C and 1D) Hhen these are used in conjunction Hi th the adaptive coefficient selection schemes. Nodels 1E and 1F exhibit poor results, the roason being that, in addi tion to the inferior energy estimation process (limited by the one-dimensional diagonal scan)t the energy estimate is insufficiently accurate to allow appropriate quantization of the transform coefficients. H0l1ever, this is not the case for Nodel 1G and, in fact, this adaptive algorithm is the best, giving a bit rate of 0.47 bi ts/pel for 0.1 % nmse. '1'he success in Nodel 1G again lies i.n its ability to give a truer representation of the coeff"icient energ:)' and hence allow a more accurate quantization of the transform coefficient. Figure 5.34 contains pictures of the "GIRL" image >1hich have been processed using thi s method. The del\rada tions due to coefficient trancation and quantization error are easily disting-

194 178 uishable at the Imler bit rates (FiL'Ures 5.34b and 5.34c). These degradations appear as visible transform block edge structures in addition to a more random but "structured" noise giving the impression that the picture is on a piece of canvas. At higher bit rates, these degradations are very much less apparent and the overall picture quality is improved. The adaptive algorithms indicate the potential of transform coding for achieving efficient redundancy reduction. leading to lml bit rate transmission. Rates belo,", 0.5 bits/pel have been obtained with relatively simple algorithms applied to two-dimensional net coefficients. Such data compression rates are especially encouraging in comparison with those of existing systems where the average value is about 1 bits/pel. In addition, these adaptive coding algorithms provide a basis for the analysis of three-d.i.mensional transform coding where not only the spatial, but the temporal correlation can also be exploited to achieve even lower bit rates.

195 ()J E '" c k1" k1 = k1 = , BIT RATE (BITS/PH) FIGURE Results for Iodel 1A

196 QJ.11 c: o k1 = k1 = k1 =4BOOOO r BIT RATE(BITS/PEl) }'IGURE :!esults for Node 1 1 B

197 QJ E '" c: k1 = k1 = k1 = 102OOO ,---, ,-----,-----, BIT RATE (BITSfPEL) FIGUE esults for Model 1e

198 k1 = /k1 = / //k1 = QJ V> E c r r BIT RATE (BITS/PH) FIGURE Results for Hodel 1D

199 QJ Vl E to.10 MODEL 1E.08 MODEL 1 F MODEL 1G.06 -t ;------,------, BIT RATE (BITS/PEL) FIGURE Results for Kodels 1E, 1F and 1G

200 (0) Origtnol Ib) k 1 =12800, 0.31 blt/pel ; 0.181%nmse (e) k 1 =38400; 0.42 bit /pel, 0.117%nmse (d) k.=76800, 0.52b, t/pel, 0.089%nmse (e k 1 =128000, 0.62bltlpel,O.073'/onmse FIGURE 5.34

201 THREE-DH1BNSIONAL ANf.LYSIS The results of two-dimensional adaptive coefficient selection and quantization are encouraging in that datn rates beloh 0.5 bits/pel are achievable l;i th a tolerable amount of degradation in the decod.ed image. Such techniques, hohever, do not exploit the high temporal corre lation that!!lay exis t in natural images. hree-dimensional transform cod.ing takes account of such effects and, in general, leads to more effective redundancy reduction and consequently lower transmitted bit rate. The techniques that have been developed using tho-dimensional transforms (Section 5.5) are now extended to the three-dimensional case. 'rhe pictures used in the' simulations are tlvo head and shoulder scenes Hhere the first consists of a moderate movement, and the second a rapid movement sequence. FiGUres 5.35 and 5.36 ShO'lls respectively, four typical successive fromes from these sequences. In examining the algorithms a sequence of 48 fra.mes (approximately 2 seconds) of picture information is used as the da ta source. ;;:ach frame consis ts of 96 pe Is by 128 line s. '" 1 thou{',':, it would. be preferable to use longer sequences this would involve expessive computing time. As a compromise, a sequence length of about 2 seconds is probably adequate and the best that can be achieved using existing hardware compute!' simulation facilities. In the simulations, various transform block sizes are used and their effects on the algorithms examined. In t\ojo-dimensional transform codjng a 16 x 16 transform block size is a good choice as a comprobise

202 (a) Frame 1 (b) Frame 2 (e) Frame 3 (d) Frame 4 FIGURE 5.35

203 (a) Frame 1 (b) Frame 2 (c) Frame 3 (d) Frame 4 FIG URE 5.36

204 1 B B bet",een high data compression and computational complexity. However, the situation 'Ii th regard to three-dimensional transform coding is not so clear and often the choice of transform bl.ock size is dictated by implementation restrictions. For a hio-i<ay c.ommunication link, another important point is the effect.of time delay in threedimensional transform coding. This is significant since excessive delay,;ill impede satisfactory c.ommunicati.on. Taking such considerations into account experiments have been performed using block sixes.of 16 x 16 x 4, 16 x 16 x 8 a.nd 8 x 8 x Transform C.oefficient 8nergy Distribution and Bl.ock Quantization The results of h!o-dimensional analysis have indicated that the energy of the transf.orm coefficients is best represented by the mean square values.of the c.oefficient amplitudes. Alth.ouC;h from empirical observations other models may be preposed to represent the energy distributi.on, their effectiveness fer data compressi.on is not as great. '['he use of the mean energy distribution techni"ue necessarily involves the transmission of a greater amount of sideinformation to the receiver but this problem is shol<n to be a minor one (see Section ). Based on these arglllnents the mean encrey distribution model for the transform coefficients is employed, extensively in the three-dimensional case. The effectiveness of such a model is examined by using it to derive a bit allocation map, through theapplicati.on of the rate dist.ortion relation, ad subsequently to quantize the coefficients. This procedure is repeated for various transform block sizes and both movement sequences.

205 1 89 Figure 5.37 shows the results of these simulations. It is obvious from this fi8ure that the larger the block size the higher the data compression factor for a given decoded picture fidelity. For example, for 0.1 % nmse in the decoded image of the moderate movement sequence, the 16 x 16 x 8 transformation requires an average of 0.24 bits/pel, whereas the 8 x 8 x 8 transformation needs about 0.41 bits/pel. This relation is also true for the rapid movement sequence. \o/i th regard to the rate of image movement, it is seen that there is a considerable increase in the amount of information wi th increase in movement. For ex3mple, for 0.1 % nmse ancl a blocl: size of 16 x 16 x 8 the bit rate increases from 0.24 bits/pel to 0.52 bits/pel, i.e. by a factor greater than 2. This is also true in the case of a block size of 16 x 16 x 4. \'Ihat is perhaps most noticeable, however, is that the transition from hlo-dimensi onal to three-dimensi onal transform c:oding results in a significant increase in the data compression lc?.. ctoro 'l'}1in is seen in the reduction in bit rate from about 0.89 bits/pel in the hlo-dimensional case (Figure 5.18) to about 0.24 bits/pel in 'he three-dimensional case (Figure 5.37) for 0.1 % nmse, using 2. similar energy distribution procedure. These results are );,ost encouraging and lend support to the use of three-dimensional transform coding in spite of its increased computational complexity. With such 101 data rates for simple non-adaptive coding as explained above, there is great scope for further reduction in bit rate when adaptive coding strategies are applied. Despite the poorer performance of the approximate energy

206 . 11,: (Ml-Modera te (R)- Rapid.12 QI <I> E.10 o '" o.08 16x16x4(R) 16x16x8(Ml 16 x16x4(ml Bx8xB(M) 16x16xB(R).06\ r ' ' 'r r BIT RATE (BITS IPE Ll FIGUllE Hesults of 3-D l 'ean er:9rgy distribution and Block Quatization

207 1 91 distribution models described in Section , they were again considered, in the context of three-dimensional coding, and variations of the directional activity-related model explained in Section ,.1 were examined in depth. Ho,>'ever, the results of these investigations fell short of expectation by a considerable margin. Since the original aim Has to derive a simple linear relationship between the coefficient energy and its order, 'i t Ias found not to be possible in this case because of the highly asjetrical nature of the energy distribution \d th respect to the three coordinate directions. As the mean energy distribution model is \>'ell suited to further analysis, the idea of developing such an approximate model for the three-dimensional case was eventually dropped since it was felt unlikely that its performance would be better than that of the mean energy d.istribution model, as illustrated in the hlo-dimensional case. 5.G2 nerry Estimation ana Adantj.ve Coefficient Sel.Bction. Following the same course of investigation ab in the case of twodimensi onal a,nalysis, the effects of energy estimation and adaptive coefficient selection are first examined without considering quantization noise. 'I'he ideas formulated for the tho-dimensional case, having been s:lo\m to be successful, are extended into th:::eedimensions. Two energy estimation schemes have been examined and are described belo<1. Figure 5.38 is a diagram of. the threedimensional transform coefficients to aid visualizing the manner in which the estimated energies are computed.

208 1 92 a(m-1,n-1,t-1)p------_ a(m,n-1,t-1l a(m-1,n-1,t)?---!---4 a(m,n-1,t) a( -1nt-1) ;,.," a(m,n,t-1) a(m-1,n,t)0' a(m,n,t). FIGURE Three-dimensional transform coefficients Let the transform ooefficient be represented in three-dimensions as a(m,n,t). Estimation of the coefficieht energy is first considered on the edge-faoes, i.e. the planes m-n, n-t and m-t, in a manner similar to Scheme 4 for the two-dimensional case. It is assumed that the first seven coefficients a(o,oo), a(1,0,0), a(0,1,0), a(0,0,1), a(1,1,0), a(1,0,1) and a(0,1,1) are ahlays transmitted to start the estimation process. Energy estimation and coefficient selection along the edges of the transform block arc first considered as follows }'or edge a(m,o,o) where m = 2, 3,, N-1 (5.37a).,.

209 193 E'(3,O,O) =} [a 2 (2,0,0) + a 2 (1,O,O) + -1J-[a 2 (1,1,a) + a 2 (1,O,1)l] (5.37b) for ID > 3 (5.37c) For edge a(a,n,o) where- n = 2, 3,, N-1 E'(0,2,0) 222 [a (0,1,0) + a (1,1,0) + a (0,1,1)] (5.38a) E' (0,3,0) = - [a 2 (a,2,0) + a 2 (0,1,0) + -1J-[a 2 (1, 1,a) + a 2 (O, 1, -1)1] (5. 38b) E' (a,n,a) = - [a 2 (a,n_1,a) + a 2 (a,n-2,a) + a 2 (a,n_3,a)] (S,38c) for n > 3 For edge a(a,a,t) where t = 2, 3,, T-1 E'(a,a,3) = + [a 2 (a,a,2) + a 2 (a,a,1) + t[a 2 (a,1,1) + a 2 (1,a,1)]] E'(a,a,t) -for t > ' = i a-(a,o,t-1) + a (a,a,t-2) + a (a,a,t-3) (5.39b) For edge-face a(m,n,a) where ID = 1,., M-1 n=1,oo,n-1 except for m = n = 1,

210 194 For edge-face a(o,n,t) where n = 1,, N-1 t = 1, T-1 except for n = t = 1, E' ( 0, n, t) = } [ a 2 ( 0, n-1, t) + a 2 ( 0, n, t-1) + a 2 (0, n-1, t-1 )] (5.41 ) For edge-face a(m,o,t) where m = 1,, M-1 t = 1,, T-1 except for m = t = 1, E'(m,O,t) = t [a 2 (m_1,o,t) + a 2 (m,o,t-l) + a 2 (m_1,o,t_1)] (5.42) Finally, the energies of the coefficients 'li thin the b lock are estimated as (5.43) wherem=1,,m-1 ;n=l,,h-l ; t = 1,.., T-l, and they are scanned sequcntially in the order m, nand t. 'l'he estimated energy 3 1 (m,n,t) is compared against a threshold D2 and truncation of the coefficients along the path of scanninc occurs 'lhen E' (m,n, t).::; D2 where D2 is defined by Eqn

211 Scheme 2 This scheme is essentially similar to Scheme 1 in its initial stages, with the exception that the energy estimation and coefficient selection 'Ii thin the interior of the transform block is given by 1 [ E'(m,n,t) = 6 a (m-1,n,t) + a (m,n-1,t) + a (m,n,t-1) + a 2 (m_1,n_1,t) + a 2 (m_1,n,t_1) + a 2 (m,n_1,t_1)] that is, the estimated energy is based on the av"rage of the nearest six coefficients. This estimate is compared Hith D2 and the truncation procedure is similar to that in Scheme Results and Comments The results of Schemes 1 and 2 for the mod.erate and rapid movement sequence s for various transform b lock sizes are shov/n }l'igures 5.39 and The transform block sizes used are 8 x 8 x 8, 16 x 16 x 4 and 16 x 16 x 8. Larger block sizes have not been implemented, o'ling to the limi ted core memory size of the minicomputer. On the other hand, a smaller block size "ould not fully exploit the spatial and temporal correlation in the image sequence. For the various block sizes considered, coefficient compression ) ratios of app roximately 10 to 16 are possible, in comparison Hi th about 5 to 7 for the two-dimensional case, thus giving al1 improvement of approximately two times. Scheme 1 appears to be marginally better than Scheme 2 but the difference is only very

212 // // 16x16x4 (R) 16x16x4 (M) 16 x16x8 (R) QI V> E.04 c 0 16X16X8(M) Bx8xB(M) '" (M) - Moderate (R) - Rapid o, COEFFICIENT COMPRESS ION FACTOR 16 1B FlcmlB Results of Energy estimation and Coefficient selection Scheme 1

213 x16 x4(m).06 // 16 x16x4(r) <lj on.04 0 // 16x16x8(R) 16 x16 x8 (M) // 8x8x8(M) -D -.:.02 (M) - Moderate (R) - Rapid o \ COEFFICIENT COMPRESSION FACTOR FIGIJRE Results of Nnergy estimati0 and Coefficient selection Scheme 2

214 198 slight. As for the effectiveness of the selection procedure in respect of different block sizes, the choice of temporal extent is and importantaappears to have greater weight than that of spatial extent of the transform block in achieving higher compression factors. This is very well illustrated by the superior performance of the 8 x 8 x 8 transformation in comparison with the 16 x 16 x 4 transformation. In fac, both Schemes 1 and 2 seem to be particularly well suited to the 8 x 8 x 8 transformation. Generally, compression ratios are higher for the moderate movement sequence a.nd this can be attributed to the higher temporal correlation in the original image. Rapid movement scenes result in large intensity variations both temporally and spatially, although in some instances the blurring effects of such movements in the spatial domain may assist in counteracting the increase in information in the temporal direc tion. Such effects may be an advr-mta,ge, providing et more rel5111ar and consistent total data or information rode by allowing allocation of limited transmission channel resources with appropriate emphasis on either the spatial or the temporal direction depending on the activity cf the ima.ge in those directions. This natur"lly further justifies the use of three-dimensional, compared with twodimensional, transform coding for achieving low bit rate tra.nsmission. HO'lever, three-dimensional transform coding does increase hardware complexity considerably, and thus a compromise between performance and block size must be found Three-dimensional Adantive Transform Codin" Systems In this section the ideas discussed and ex=ined in Sections

215 199 and are consolidated to derive several three-dimensional adaptive transform coding systems. Errors due to transform coei'i'iciant quantization manifest themselves in a manner quite unlike PCI1 quantization error. Such effects are examined here through the use of various quantization strategies which closely resemble those of the b.o-dimensional case. Essentially, four different schemes of three-dimensional adaptive transform coding ha.ve been considered, "here the coefficients are adaptively selected and coded taking account of local statistical variations from one transform block to the next. As in the case of t;lo-dimensi onal' analysis, tho types of quanti zers have been implemented. The first (or d.c.) coefficient utilizes a. quantizer with a Rayleigh pdf characteristic ;lhose decision and reconstruction levels have been ohtaire d. hy the approximate method due to Algazi(66). For the remaining a.c. coefficients a Hax Gaussian quantizer has been used. Both quantizers are, in fact, identical to those used in the t\'lo-dimensional analysis Hean Sner&"y Distribution!-lodel with i.da:otivc Coefficient Selection (Models 21. to 2H) The merits of the r.:ban energy distributioll model and its associated. bit allocation map have already been demonstrated in Sections , and It is obvious that activity varies from block to block, and if this variation is taken into consic.eration greater redundancy reduction may be obtained than in the case of fixed block quantization generated from the mean energy' di.strl.bution

216 200 model. The adaptive schemes to he investigated here are of the type shmm in Figure 5.9 where side-information about the mean energies is a necessary component in reconstructing the image at the receiver. The mean energy E(m,n,t) of the transform coefficient a(m,n,t) is first determined by averaing over K frames, and the bit allocation map is determined via the rate distortion equation, b(m,n,t) = INT [-Is 10g2 [E(m,n,t)/D11 + ] >there D1 follm-rs Eqn This is follm<ed by energy estimation and adaptive coefficient selection using the two schemes descri.bed in Section 5.6.2, >tith the difference that the energy estimates are based on averages of the squares cf the previously quantized coefficient amplitudes as opposed to the use of unquantized values in the former case. 'l'his ensures that the receiver follows an identical estimation procedure to that of the transmitter, thus overcoming the need to transmit excess side-information. The estimated energy E'(m,n,t) is compared with the threshold D2 and truncation occurs when Inequality 5.27 is satisfied. The first seven coefficients a(o,oo). a(0,0,1), a(0,1,0), a(1,0,0), a( 0,1,1), a( 1,0,1) and a( 1,1,0) are ahrays transmi tted "Ii th a predetermined number of bits assir;;ned by the bit allocation map. The receipt of these seven coefficients will then start the energy estimation and. coefficient selection process. This means that the minimum number of transmitted. coefficients is seven (corresponding

217 201 to low block activity) t whereas the maximum number is determined by the non-zero zone of the bit allocation map when the activity is high. A typical bit allocation map f.or an 8 x 8 x 8 transformation applied to the moderate movement sequence is illustrated in Figure In the simulations the number of bits assigned to the coefficients is limited to a maximum of 9 for the d.c" and 7 for the a,c, coefficients. It is seen here that a substantial number t = t = Cl t = o o 0 0 t = t o t = t = o o 0 o o o o o o o o o o o 0 t =., FIGURE Typical bi t allocation map for 8 x 8 x 8 DeT block of moderate movement sequence

218 202 of coeffecients have no bits allocated indicating the high degree of data compression that is achieveable using three-dimensional transform coding Adantive Eotima.ted Enerv Hanning and Coefficient Selection (Models 2G to 2L) Adaptive coefficient selection using an energy estimation scheme may be extended to adaptive bit assignment qf the transform coefficients. Such a technique will cbviate the need to relay information of the mean enereies of the coefficients so that the system "Till be of the form sho\m in Figure '1'he energy estimation scheme that has been implemented is similar to Scheme 1 (Section ) where the estimated energy of a transform coefficient within the block is an average of the squares of the amplitudes of' the nearest three coefficients, Le. E' (m,n, t) 22' 2 - t 1&-(m,n,t-1) + i-(m,n-1,t) + &'(m-1,n,t)] There m = 1,.., ; n = 1, I H -1 ; t = 1. ', T-1, '" i th the exception of the edge and edge-faces of the transf'orm block where, as explained in Section , one- and hto-dimensional energy estilnations are made. Scheme 2 (Section ) has not been implemented which it offers no advantaee over Scheme 1 a.nd, in addition, involves more arithmetic operations. Again the first seven coefficient a(o,o,o), a(0,o,1), a(0,1,o), a(1,0,0), a(o,1,1), a(1,0,1) and a(1,1,0) are ahtays transmitted

219 203 with a fixed and predetermined number of bits to start the energy estimation process. The case in v/hich only the firgt four coefficients, namely, a(o,o,o), a(o,o,f), a(0,1,0) and a(1,0,0) are always transmitted has also been examined. In such a case, the onedimensional energy estimation along the m-axis for E'(2,0,0) and E'(3,0,0) are given by (5.48a) E'{3,O,O) = S- [2(2,0,0) + 2(1,0,0) + }la 2 (0,O,1) + a 2 (O,1,O)J] (S.1\8b) while for the edge-face corresponding to the m-n plane, E'(1,1,0) is given by E'(1,1,0) '" - I a-(o,1,0) + a (1,0,0) + a-(0,o,1)] 'l'he estimation of the energy of the other coefficients in the m-n plane is then similar to that described in Section , Hith the exception that guantized values are used. Corresponding procedures are also employed for the other two-dimensional planes. The estimated energy is used to determine the number of bits required to code the coefficient using the equation b(m,n,t) = INT [i 10g2[E'(m,n,t)!D 1 I + i] \'Ihen b(m,n,t) is less than 1, truncation of the coefficients occurs. In addi hon, the square root of the estimated energy is used to

220 204 normalize the coefficient amplitudes in order to ensure that 110 excessive overloading of the quantizers occurs Results 8,nd Comments The effects cf quantization on the adaptive algorithms for threedimensional transforms are examined using various block sizes on the two moving sequences described, previously. Graphical results of the three-dimensi anal' adaptive coding algorithms are shmm in Figures 5.42 to In order to allow a comparative assessment of the various algorithms and the use of different transform block sizes, the bit rates of the different systems corresponding to 0.1 % nmse in the decoded picture are shown in 'fable 5.4. The results again reveal that the choice of an appropriate transfotm block size is an important factor in achieving high data compression. Undoubtedly, larger block sizes are better since a greater amount of spatial and temporal correlatio'n is taken into consideration. The rapidity of movement also has considerable influence on the information rate. In eeneral, with the present adaptive systems it is found that for the two moving sequences considered in-the simulations, there is an average increase of 50 % in the required bi t rat e from that for the moderate movement sequence to that for the rapid sequence. This incre2,se is qui te substantial and it is unfortunate that, despite the use of adaptive algorithms, such a large variation in information rate is not entirely, or even substantially, buffered by the adaptivi ty of the system. This, in part, ma.y be due to the non-optimum choice of the transform block size. Although larger '".

221 205 System Block size I-lode 1 Sequence Bit rate Movement (bit/pe 1) Mean energy mapping 8x8x8 2A Moderate 0.27 with adaptive 16x16x4 2B Moderate 0.24 coefficient selection Rapid 0.36 Scheme 1 16x16x8 2C Hoderate 0.17 Rapid 0.29 Mean energy mapping 8x8x8 2D lodera te 0.28 with adaptive 16x16x4 2E Nodera.te 0.25 coefficient selection Rapid 0.37 Scheme 2 16x16x8 2F Hoderate 0.19 Rapid 0.29 Adaptive estimated 8x8x8 2G Hodera te 0.49 energy mapping (7 16x16x4 2R l10dera te 0.38 initial coefficients) Rapid x16x8 2I Moderate 0.27 Rapid 0.25 Adaptive estimated 8x8x8 2J Noderate energy mapping ( If 16x16x4 2K Moderate 0.51 initial coefficients) Rapid 0.41.' 16x16x8 2L Moderate 0.28 Rapid 0.28 Table.4

222 206 block sizes may be more amenable to adaptive algorithms and hence achieve higher data compression, the fact that they are not practically applicable will negate SUCll advantages. Thus the choice of transform block dimension is, to a large extent, decided. by the difficulties in implementation. In contrast to two-dimensional analysis where the best performance is obtained for the totally adaptive estimated enerey mapping method (Hodel 1G), in the three-dimensional.case, the mean enerey distribution model Hith adaptive coefficient selection (Scheme1), Le. Hodel 2C, performs better than the adaptive estima.ted energy mapping methods, i.e. node I" 2G to 2L. The best performance figure' is obtained fol' Node I 2C Hhere, for the moderate movement sequenr.e, and 0.1 % nmse, a bit rate of only 0.17 bits/pel is necessary. The rapid movement sequence represents a vcry extreme form of movement I<hieh is not often encountered, although it does exist in real images. Even so, using Model 2C the bit rate is only 0.29 bits/pel Hhich is slightly more than on8-half of the best hto-dimensional adaptive transform coding sy'" tem consillered in the previous sections. 'fhe reason for the poorer performance of the totally adaptive estimated energy mapping models, i.e, Hodels '2G to 2L, in comparison to Eodels 2A to 21;', is probably the quantization process. This is not to say, though, that l'lodels 2G to 2L are i'leffective, on the contrary, with Model 21, a value of only 0.25 bits/pel is obtained, "hich is just over one-half that obtained,1i tll the best tho-dimensional system. Since the quantization process in these cases is dependent on the esb.mated energy, then the accuracy of this es tiroa te is crucial. 'fhe non-uniformity of the di.mensions of

223 207 the transform block might necessitate an appropriate airectional weighting in the ccmputation of the energy estimate. However, this has not been examined further primarily because, even if such an optimum weighting factor,iere' determined (.on the basis of extensive empirical data) it ;rould be quite unique and totally depeno:ent on the original source data. In addition, existing simulation facilities are not sufficiently flexible and versatile to permit rigorous investigations of such a nature. Hm'lever, despite the lower performance of Ilodels 2G to 2L, the algori thm is more effective in maintaining a more constant flml of data. For example, VlithJ<1odel 2I, for 0.1 '/bnmse, the bit rate for the, moderate movement sequence is 0.27 bits/pel while for the rapid movement sequence it is 0.25 bits/pel, i.e. a change of only 0.02 bits/pel. V/hen contrasted Hith Bodels 2A to 2F, the potential'of the totally adaptive system is recognised. rig-ures 5.54 and 5.55 sho" decoded sub-piet-jres using j'10dels 2C!?nd 2L respectively. Each sub-picture consists of 64 pels by 61j lines, and eight successive frames from th8 tl-io movem8nt sequences are illustr3,ted far different threshold parameters (k and k2 in Nodel 1 2C and k1 only in Nodel 2L). Although these sub-pictures reproduced here are not of gogd quality, some degradations are noticeable \'lnen the threshold factors are not chosen properly. The intention of including these pictorial results is mer81y to serve as in indication 01 the decoded picture quality. In actual fact, these degradations, "hich appear 80 prominently here, are very much le ss noticeable when viewed in real-time, since the distortions from one frame to

224 V> E.12 c, k 1 = k1 = k1 = r T ' BIT RATE (BlTS/PEL) FIGU3E Results of Model 2A for Moderate movement sequence

225 V> '" E c '1 = r r r r BIT RATE{BITS/PEl) FIGURE Results of Model 2B for Moderate movement sequence

226 w Vl E c 0.10 Kl= K 1= OB K1 = BIT RATE (BITSJPEl) FIGURE Results of Nodel 2C for l'ioderate movement sequence

227 QJ V> E co cf k1 = k 1 = k1 = rl Ir I BIT RATE (BITS IPEl) FIGURE Hesults for f.lodel 2B for rapid movement sequence

228 (U Vl E c k1 = k 1 = k1 = , BIT RATE (BITS/PEL) FIGURE Reaults of Model 2C for rapid movement sequence

229 <lj V> E c:.10 t k 1 = k 1 = BIT RATE (BITS IPELI FIGUHE Results of Nodel 2D for moderate movement sequence

230 k1 = OB k1 = k1 = r r ; BI T RATE (BITS IPEL) FIGUHE Results of Model 2E for moderate movement sequence

231 '" Vl E c: 'if k 1 = OB ---- k, = k1 = r BIT RATE (BITS/PEl) FIGURE Results of Model 2F for moderate movement sequence

232 QJ V> E 10- c k1 = k1 = ; , r ,----_ BIT R ATE (B ITS I P EL ) FIGURE Results of }lodel 21' for rapid movement sequence

233 QJ Vl E c 0.10 k1 = k1 = k1 = r ' BIT RATE (BITS IPEl) FIGURE Results of Model 2F for rapid movement sequence 'J'.

234 '" c E 10 o (M) -Moderate (R)-Rapid N a>.08 16x16x4(M) 16x16x8(R) 16x16x8(M) 16x16x4(R) 8x8x8(M).06 I I SIT RATE (!3!TS/PEl) FIGllHE :" ne su 1 ts of I';rode Is 2G, 211 n,nd 21. I

235 V> '" E c 10 o IM) -Modera te IR)-Rapid N..,.OS."". 16x16x81R) 16x16x8(M) 16x16x4(R) Sx8x81M) 16x16x41M) r r BIT RATE IBITS/PEl) ji'igutie Results of l'!odels 2J, 2K a.nd 2L

236 (c) Ortgmct (bl kl k211oo11 (cl "'2S6000. k2j1ooo (dl k 1 ds6000. k 26t.OOO bit lpet 0.133%nmse 0.1B4 bill pet O.092%nmse bi t Ipet 0.086%nmse FIGU RE 5.5 "

237 (01 Orl}ool I bl k1" bit/pet O.111%nmse lel k," IM t/pei O.085%nmse Idl k 1 " o.3sob, t /pel O.OS8%nrnse FIGURE 5.55