Integrating Knowledge Sources in Devanagari Text Recognition System

Transcription

1 500 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART A: SYSTEMS AND HUMANS, VOL. 30, NO. 4, JULY 2000 [16] U. Ophthalmology Branch, Clinical sciences division, Aerospace Ophtalmology, 1988, vol. II, ch. 15. [17] G. Wyszecki and W. S. Stiles, Color Science. New York: Wiley, [18] J. Neitz, M. Neitz, and P. M. Kainz, Visual pigment gene structure and the severity of color vision defects, Science, vol. 274, pp , [19] S. K. Shevell and J. C. He, The visual photopigments of simple deuteranomalous trichromats inferred from color matching, Vis. Res., vol. 37, no. 9, [20] J. Lutze, N. J. Cox, V. C. Smith, and J. Pokorny, Genetic studies of variation in Rayleigh and photometric matches in normal trichromats, Vis. Res., vol. 30, no. 1, [21] H. Scheibner and T. Kremer, Deuteranomaly studied with four perceptual criteria, Vision Research, vol. 36, no. 19, [22] C. M. Cicerone, A. L. Nagy, and J. L. Nerger, Equilibrium hue judgements of dichromats, Vis. Res., vol. 27, no. 6, [23] A. B. Poirson and B. A. Wandell, Appearance of colored patterns: Pattern-color separability, J. Opt. Soc. Amer., vol. 10, no. 12, pp , [24] I. Dvorine, The Dvorine Color Perception Test. Baltimore, MD: Waverly, [25] C. E. Martin, S. K. Rogers, and M. Kabrisky, Digital production of color Mach bands using a color human visual system model, IEEE Trans. Syst., Man, Cybern., Aug. 1996, submitted for publication. Integrating Knowledge Sources in Devanagari Text Recognition System Veena Bansal and R. M. K. Sinha Abstract The reading process has been widely studied and there is a general agreement among researchers that knowledge in different forms and at different levels plays a vital role. This is the underlying philosophy of the Devanagari document recognition system described in this work. The knowledge sources we use are mostly statistical in nature or in the form of a word dictionary tailored specifically for optical character recognition (OCR). We do not perform any reasoning on these. However, we explore their relative importance and role in the hierarchy. Some of the knowledge sources are acquired a priori by an automated training process while others are extracted from the text as it is processed. A complete Devanagari OCR system has been designed and tested with real-life printed documents of varying size and font. Most of the documents used were photocopies of the original. A performance of approximately 90% correct recognition is achieved. Index Terms Devanagari document processing, knowledge-based systems, optical character recognition. I. INTRODUCTION Segmentation and classification are the two primary phases of a text recognition system. The segmentation process extracts recognition units from the text. The recognition unit is usually a character. The classification process computes certain features for each isolated character and each character is classified to a class which may be the true class (correct recognition), wrong class (substitution error), Manuscript received July 21, 1998; revised August 2, The review of this paper was recommended by Associate Editor S. Lakshmivarahan. V. Bansal is with the Department of Indistrial and Management Engineering, Indian Institute of Technology, Kanpur India ( veena@iitk.ac.in). R. M. K. Sinha is with the Department of Computer Science and Engineering, Indian Institute of Technology, Kanpur , India ( rmk@iitk.ac.in). Publisher Item Identifier S (00) or an unknown class (rejection error). Based on the output of the classification process, resegmentation of selected units is attempted [7], [14] and reclassification is done. However, some classification errors still remain which many researchers have tried to correct using contextual knowledge [12], [16], [22] [25]. More than one classifying feature have been employed for classification to take care of different discriminating features under varying real-life situation [11], [13]. In other words, the contextual knowledge is used during the classification phase also [1]. In this paper, our concern is Devanagari script. It is the script for Hindi which is official language of India. It is also the script for Sanskrit, Marathi, and Nepali languages. The script is used by more than 450 million people on the globe. The algorithms which perform well for other scripts can be applied only after extensive preprocessing which makes simple adaptation ineffective. Therefore, the research work has to be done independently for Devanagari script. Sinha et al. [15], [20], [21] and some other researchers [17], [18] have reported various aspects of Devanagari script recognition. However, none of these works have considered real-life documents consisting of character fusions and noisy environment. An optical character recognition (OCR) for Devanagari and Bangla (an Indian language script) printed script has been described by Chaudhuri and Pal [9], [10]. The emphasis is on recognizing the vowels and consonants (basic characters) which constitute 94 96% of the text. The basic characters are described in terms of nine types of strokes. A stroke is described in terms of its length, height and expected angle. These angles are very much font specific. Moreover, the fading of some black pixels may mislead the stroke extraction process. Even the hand-crafted character specific features may not sustain their utility across various fonts. Our approach to Devanagari script recognition is to use a number of knowledge sources and integrate them in hierarchical manner. There is no reasoning performed with the knowledge sources. These are mostly statistical in nature, or in the form of word dictionary tailored specifically for OCR. The character classification is based on a hybrid approach. The decision for further segmentation of an image box is based on the outcome of the classification process and the statistical analysis of width of the image box. We integrated the knowledge sources using the blackboard model primarily because it facilitates easy modifiability of the system. Some of the knowledge sources are acquired by a separate automated training phase. Whereas, the transient knowledge sources are constituted from the knowledge extracted from the text as it is processed. The performance is studied and presented in Section V. The overall system architecture and the knowledge sources are presented in Section III. Section IV describes the implementation details. We introduce Devanagari script from the OCR viewpoint in the next section. II. DEVANAGARI SCRIPT FROM OCR VIEW POINT Devanagari script is a logical composition of its constituent symbols in two dimensions. It is an alphabetic script. Devanagari has 11 vowels and 33 simple consonants. Besides the consonants and the vowels, other constituent symbols in Devanagari are set of vowel modifiers called matra (placed to the left, right, above, or at the bottom of a character or conjunct), pure-consonant (also called half-letters) which when combined with other consonants yield conjuncts. A horizontal line called shirorekha (a headerline) runs through the entire span of work. Some illustrations are given in Figs. 1 and 2. Devanagari script is a derivative of ancient Brahmi script which is mother of almost all Indian scripts. Word formation in Indian scripts /00$ IEEE

2 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART A: SYSTEMS AND HUMANS, VOL. 30, NO. 4, JULY Fig. 3. A simplied Devanagari script composition grammar in BNF. Fig. 1. Characters and symbols of Devanagari script. Character set of Devanagari script is divided into three groups based on the coverage of the region of the core strip. These sets are shown in Fig. 6(a). The full box characters are further divided into three groups based on the presence and position of the vertical bar. These sets are shown in Fig. 6(b). The end-bar set of core characters is large. These characters are subdivided in two groups based on the joining pattern of the character with the header line [Fig. 6(c)]. Classification based on these features is robust and remains consistent over a large number of fonts and sizes. Fig. 2. Sample Hindi text written in Devanagari script. follows a definite script composition rule for which there is no counterpart in Roman. A simplified Devanagari script composition grammar in BNF is summarized in Fig. 3. The script composition grammar imposes constraints on the symbols recognized by the OCR [21]. III. SYSTEM ARCHITECTURE AND KNOWLEDGE SOURCES Fig. 4 depicts the system architecture and its components for Devanagari text recognition system. The system consists of a solution blackboard and various knowledge sources. There is a set of tools that is used for applying knowledge sources. Each component is outlined in the following subsections. A. Statistical Information: Transient Knowledge (KS1 and KS3) The segmentation done by smearing or profiling leaves the overlapping text lines and touching characters unsegmented. Selection of image regions for further segmentation in our system is based on statistical analysis of height or width depending on the context. The segmentation process based on this strategy is described in [3]. The most frequent line height of the threshold line height which constitute KS1. The most frequent height of the image box is stored as threshold character height. Similarly, width information is stored in threshold character width. This information constitute KS3. B. Structural Properties of the Script and Characters (KS2 and KS4) 1) Header Line: The presence of more than one header line indicates the fusion of two text lines. Similarly, the absence of a header line indicates that the image is part of a text line. The header line property is also used for identification of word boundaries and skew correction. 2) Three Strips of the Word: It is convenient to visualize a Devanagari word in terms of three strips: a core strip, an upper strip, and a bottom strip. The upper strip is the region above the header line which contains all the top modifiers and easily extracted. The core strip contains all the characters and the vowel modifier j (see Fig. 5). The lower modifiers are placed below the core characters. This is referred to as the lower strip. For separating the lower strip from the core strip, we need to estimate the height of the core strip which we explain in the next section. C. Statistical Features of Symbols In this section, the statistical classifying features for Devanagari characters are discussed. 1) Horizontal Zero Crossings (KS5): Bao-Chang et al. [5] use the sequence S which represents horizontal zero crossings for a character image as a classification feature. We have modified the feature to make it font-independent and noise resilient. We divide the character into three horizontal segments of equal height. Each segment is represented by number of zero-transitions that is most frequent in the segment. Further, the most frequent number is sequence S represents the sequence. The feature vector consisting of these four numbers is robust and remains invariant over a large number of fonts and sizes. 2) Moments (KS7): Two dimensional moments have been widely studied as a feature for character classification [8]. We compute size and translation invariant moments up to an order of two from real-life samples using computation formulae given in [8]. The cardinal distance between the moment vector for an unknown character and a known candidate character is computed. This distance is used for ranking the candidate characters. 3) Vertex Points and Pixel Density in 9-Zones (KS6 and KS8): A character box is divided into nine (3 * 3) zones. These zones are used to record the position of end points (also called vertex points). A pixel having only one neighbor is defined as an end point. The position of these end points is recorded as a nine bit string denoting the presence/absence in the nine zones in the given order. A character class may have multiple nine-bit string. Further, the digitized pixel density in each of the nine zones is also used for classification. Number of dark pixels exceeding 50% of total pixels in a zone is represented by 1 in the feature vector. Less than 50% dark pixels correspond to a 0 in the feature vector. This feature vector is specially used in recognition of lower and upper modifier symbols. D. Structural Prototypes of Characters (KS9) We describe Devanagari characters in terms of a certain number of primitives with their positional information encoded in the form of nine-zones of the minimum sized up-right enclosing rectangular box and also used certain features specific to Devanagari characters. One such feature is presence of a vertical bar. This vertical bar besides acting as a filter in reducing the set of characters over which the search is

3 502 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART A: SYSTEMS AND HUMANS, VOL. 30, NO. 4, JULY 2000 Fig. 4. Blackboard architecture for Devanagari text recognition system. Fig. 5. Three strips of a Devanagari word. on the unknown character for which a description has been obtained using the similar process as used in construction of the prototypes. We construct the prototypes on characters obtained through the process of segmentation on real-life words. The description schema is applicable to any font, size and also to handwritten characters. The matching process yields a distance figure between the structural representation of the unknown character and the prototype of the known character. This distance figure is used for ordering the candidate characters. This KS is described in detail in [4]. E. Character Confidence Information (KS10) The confidence figure is either 0 or 1 for a feature of binary nature (match/no match); such as modified horizontal zero crossing vector, position of vertex points, and position of vertical bar. Whereas, structural description comparison gives a distance from the reference prototype depicting the degree of mismatch in the description. The moments feature also yields a distance figure. The maximum confidence figure is 1.0 corresponding to zero distance and for all other values of d, c is inversely proportional to the distance d; i.e., c =1=d. An empirically designed function integrates these confidence figures. F. Character Confusion Matrix (KS11) There are certain characters which isolated optical character recognition (IOCR) has difficulty in classifying due to noise or wrong segmentation. In addition, inefficacy of classifying features for certain character classes has also been observed due to their similarity. For instance, if the output of IOCR is X2, the true character could be any one from the set Fig. 6. Visually extracted properties of Devanagari characters. made, also gives a reference with respect to which correspondence between junction points are established. We perform a predictive parsing

4 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART A: SYSTEMS AND HUMANS, VOL. 30, NO. 4, JULY Fig. 7. An example showing contents of the solution blackboard for Devanagari word recognition, asterisk represents a rejected character. During the testing phase, these confusions are collected and stored as character confusion matrix. The character confusion matrix is consulted when a correction is made to map a word composed from the classification process output to a dictionary word. The cost of correction is less when the substitution is made by a known confusion than an arbitrary substitution. J. Character Pair Expert This is shown under Tools in the Fig. 4. The character pair expert is designed for each closely resembling character pair which classification process is unable to resolve most of the time for example G. Script Composition Rules (KS12) A composition process is used to compose back the word from its constituent characters and symbols. A set of rules guide the process of composition [20]. These rules guide the composition processor in identifying the symbol sequences that are syntactically correct. H. Word Envelop Information (KS13) The envelop of a word contains the following information: 1) number of character boxes; 2) number of vertical bars; 3) number of upper modifier boxes; 4) number of lower modifier boxes; 5) vector giving position of vertical bars; 6) vector giving type and position of each character box. The word envelop information is used to select candidate words from a word dictionary which has been partitioned using word envelop information in a hierarchical way. The hypothesis set for each character box is constituted by corresponding character of each selected word. I. Word Dictionary (KS14) The word level knowledge of Hindi language is represented as a word dictionary. The partitioning of the dictionary [2] is done based on certain features extracted during the OCR process. Further details on dictionary design and its use for post-processing can be had from [2]. We use a set of hand-crafted rules for each pair which are used for giving decision in favor of one of the character. IV. IMPLEMENTATION DETAILS A serious effort has been made to develop a modular system. Addition of a new features requires almost no change anywhere. Interface is clean because all interactions take place through the solution blackboard. We assume that a preprocessing stage extracts the uniform text zones from the document image. Also, the tilt correction has been done and portrait printing mode has been used. The image is scanned at 200 dpi. For most of the printed text, 200 dpi suits well. An example has been used to show the contents of the solution blackboard in Fig. 7. The binary image of document is at the lowest level. Word hypotheses are at the top level. The image segmented into characters and symbols by initial segmentation process are placed at the level above the text image. The segmented units are linearized. Initial hypothesis for each character/symbol are posted on the blackboard. Various transient statistical information which in turn trigger creation of corresponding transient knowledge source are posted on the blackboard as they become available. As the recognition process progresses, different knowledge sources are invoked which filter out some of the hypotheses and associate a confidence figure with the remaining hypotheses. These are composed into words which are verified and corrected if necessary.

5 504 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART A: SYSTEMS AND HUMANS, VOL. 30, NO. 4, JULY 2000 Fig. 9. The input text. Fig. 10. The output of the classification process; the word has been underlined if the true word is the second or third choice; incorrect words are shown in fg. This threshold value is learned by the system during testing phase. A rejection error causes control module to invoke the decomposition processor. The decomposed version and the earlier character box, both are retained as alternative representation. This process is recursively applied if the decomposed version is also suspected to be a composite character. If a character after decomposition also is not supported by character level hypothesis, it is rejected. The word dictionary is help of rest of the recognized characters and symbols of the word. Fig. 8. Recognition module for core characters. The CoreRecognizer process is outlined in Fig. 8. The recognition process for lower and upper modifiers is less involved due to small number of classes and is not presented here. The features used to substitute the rejected characters with the KS4 and KS5 are used as filters. It is ensured that the true character is not missed by allowing other characters in each class. The features KS6, KS7, and KS8 are used for ranking the candidate characters. When an invoked KS supports a hypothesis, it increments the confidence figure of the hypothesis. The control mechanism doesn t check the preconditions for a knowledge source. It is the responsibility of the knowledge source to check its preconditions and make possible modifications to the solution blackboard. The recognition process is stopped when the confidence figure associated with a character reaches a preset threshold value. V. EXPERIMENTATION AND SYSTEM PERFORMANCE A sample input text page shown in Fig. 9. The corresponding output of the OCR is shown in Fig. 10. We tested our Devanagari document reading system on various printed documents and gathered various results. In the first run, the font knowledge was not used. Consequently, moments feature, which is a font specific feature, is not used. The character level performance is approximately 70% which improves to 80% when the font information is made available to the system and the moments feature is used for classification. The performance further improves to approximately 87% after corrections are made using word dictionary. We tested the performance of our system on photocopies rather than the clean originals. This was done to test the robustness of our design schema. In photocopies, the upper and lower modifier symbols get fused with the header line and core strip area, respectively. This results in hypothesization of words from incorrect dictionary partition as the number of modifier symbols is used as one of the parameters in creating the dictionary partition.

6 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART A: SYSTEMS AND HUMANS, VOL. 30, NO. 4, JULY Another source of incorrect recognition has been system s inability to trigger segmentation leading to substitution error. This can be improved by forcing segmentation resulting in over segmentation in many cases and then use composition to yield possible multiple word hypotheses. In case of short (one or two characters in the core-strip) words, a large number of competing words are hypothesized and the distance criterion below threshold is at times erroneous. This can be improved by incorporating further layers of knowledge in the form of natural language syntactic constraints and semantics. VI. CONCLUSION In this paper, we have outlined a schema for integrating diversified knowledge sources for recognition of Devanagari script. These knowledge sources are mostly statistical in nature or a word dictionary tailored for OCR. The recognition schema integrates the different layers of the process, segmentation, character and symbol recognition, word hypothesization, and verification in a hierarchical manner with interactions with each other. The system has been tested on different fonts and sizes with photocopied printed text. The schema presented in this paper is equally applicable to all other Indian scripts as they have similar structure as Devanagari. REFERENCES [1] V. Bansal and R. M. K. Sinha, On integrating diverse knowledge sources in optical reading of Devanagari script, in Proc. Int. Conf. Information Systems Analysis and Synthesis (ISAS 96), Orlando, FL, [2], Partitioning and searching dictionary for correction of optically-read Devanagari character strings, in Proc. Int. Conf. Document Analysis and Recognition (ICDAR 99), Bangalore, India, [3], Segmentation of touching Devanagari characters, in Proc. Indian Conf. Computer Vision, Graphics, and Image Processing (CVGIP 98), Delhi, India, [4], On how to describe shapes of Devanagari characters and use them for recognition, in Proc. Int. Conf. Document Analysis and Recognition (ICDAR 99), Bangalore, India, [5] P. Bao-Chang, W. Si-Chang, and Y. Guang-Yi, A method of recognizing hand printed characters, in Computer Recognition and Human Production of Handwriting, R. Plamondon, C. Y. Suen, and M. L. Simner, Eds. Cleveland, OH: World Scientific, 1989, pp [6] T. Bayer et al., Toward the understanding of printed documents, in Structured Document Image Analysis, H. S. Baird, H. Bunke, and K. Yamamoto, Eds. New York: Springer-Verlag, [7] R. G. Casey and D. R. Furgson, Intelligent forms processing, IBM Syst. J., vol. 29, [8] G. C. Cash and M. Hatamian, Optical character recognition by the method of moments, Comput. Vis., Graph. Image Processing, vol. 39, pp , [9] B. B. Chaudhuri and U. Pal, A complete printed Bangla OCR system, Pattern Recognit., vol. 31, no. 5, pp , [10], An OCR system to read two Indian language scripts: Bangla and Devanagari, in Proc. 4th Int. Conf. Document Analysis and Recognition, 1997, pp [11] T. K. Ho, J. J. Hull, and S. N. Srihari, Decision combination in multiple classifier systems, IEEE Trans. Pattern Anal. Machine Intell., vol. 16, pp , [12] J. J. Hull, S. N. Srihari, and R. Choudhari, An integrated algorithm for text recognition: Comparison with cascaded algorithm, IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-5, pp , [13] S. Kahan, T. Pavlidis, and H. S. Baird, On the recognition of printed characters of any font and size, IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-9, pp , [14] S. Liang, M. Shridhar, and M. Ahmad, Segmentation of touching characters in printed document recognition, Pattern Recognit., vol. 27, pp , [15] S. S. Marwah, S. K. Mullick, and R. M. K. Sinha, Recognition of Devanagari characters using a hierarchical binary decision tree classifier, IEEE Int. Conf. Syst., Man, Cybern., Oct [16] E. M. Riseman and A. R. Hanson, A contextual post processing system for error correction using binary n-grams, IEEE Trans. Comput., vol. C-23, pp , [17] J. C. Sant and S. K. Mullick, Handwritten Devanagari script recognition using CTNNSE algorithm, in Int. Conf. Application of Information Technology in South Asian Language, Feb [18] I. K. Sethi and B. Chatterjee, Machine recognition of hand printed Devanagari numerals, J. Inst. Electron. Telecommun. Eng., vol. 22, pp , [19] I. K. Sethi, Machine recognition of constrained hand printed Devanagari, Pattern Recognit., vol. 9, pp , [20] R. M. K. Sinha, Rule based contextual post-processing for Devanagari text recognition, Pattern Recognit., vol. 20, pp , [21] R. M. K. Sinha and H. N. Mahabala, Machine recognition of Devanagari script, IEEE Trans. Syst., Man, Cybern., vol. SMC-9, pp , [22] R. M. K. Sinha, B. Prasada, G. Houle, and M. Sabourin, Hybrid contextual text recognition with string matching, IEEE Trans. Pattern Anal. Machine Intell., vol. 15, pp , [23] R. Shinghal, A hybrid algorithm for contextual text recognition, Pattern Recognit., vol. 16, pp , [24] S. N. Srihari, Integrating diverse knowledge sources in text recognition, ACM Trans. Off. Inf. Syst., vol. 1, pp , [25] H. Takahashi, N. Itoh, T. Amano, and A. Yamashita, A spelling correction method and its application to an OCR system, Pattern Recognit., vol. 23, pp , 1990.