Two Dimensional Principal Component Analysis for Online Tamil Character Recognition

Size: px

Start display at page:

Download "Two Dimensional Principal Component Analysis for Online Tamil Character Recognition"

Cordelia Atkinson
6 years ago
Views:

1 Two Dimensional Prinipal Component Analysis for Online Tamil Charater Reognition Suresh Sunaram, A G Ramarishnan Inian Institute of Siene,Bangalore, Inia suresh@ee.iis.ernet.in, ramiag@ee.iis.ernet.in Abstrat This paper presents a new applation of two imensional Prinipal Component Analysis (DPCA to the problem of online harater reognition in Tamil Sript. A novel set of features employing polynomial fits an quartiles in ombination with onventional features are erive for eah sample point of the Tamil harater obtaine after smoothing an resampling. These are stae to form a matrix, using whh a ovariane matrix is onstrute. A subset of the eigenvetors of the ovariane matrix is employe to get the features in the reue sub spae. Eah harater is moele as a separate subspae an a moifie form of the ahalanobis istane is erive to lassify a given harater. Results inate that the reognition auray using the DPCA sheme shows an approximate 3% improvement over the onventional PCA tehnique. Keywors: Prinipal Component Analysis (PCA, DPCA, ahalanobis Distane.. Introution In an online hanwriting reognition system, a methoology is evelope to reognize the writing when a user writes on a pressure sensitive sreen using a stylus that aptures the temporal information. Online hanwritten sript reognition engines exist for languages lie Latin [], Chinese [] an Japanese [3]. However, little attention has been evote to evelop similar engines for Inian languages. In this paper, we attempt to evolve an online reognition system for Tamil haraters using a tehnique alle two imensional Prinipal Component Analysis (DPCA. Tamil is a lassal South Inian language spoen by a segment of the population in ountries suh as Singapore, alaysia an Sri Lana apart from Inia. The Tamil alphabet omprises of 47 letters (onsonants, vowels an onsonant vowel ombinations. Eah letter is represente either as a separate symbol or as a ombination of isrete symbols, whh we refer to as haraters in this wor. Only 56 istint haraters are suffient to reognize all the 47 letters [4]. Samples of eah of these haraters form a separate lass. As far as the wor on online hanwriting reognition for Tamil is onerne, Niranjan et al. [5] have propose elast mathing shemes. Dimensionality reution tehniques lie Prinipal Component Analysis [6] have also been employe for reognition. In this wor, we propose an aaptation of the DPCA tehnique [7] for harater feature extration in a reue subspae. Eah of the 56 lasses is separately moele as a subspae. Contrary to the onventional PCA, the DPCA operates on matres rather than D vetors. A set of loal features (basally a novel set of features ombine with onventional features are erive for eah sample point of the preproesse harater. The features orresponing to a sample point are stae to form the rows of a matrix, referre to as the harater matrix in this wor. A ovariane matrix of a signifantly smaller size as ompare to the one obtaine in PCA is onstrute from the harater matrix. In orer to represent the features in a reue subspae, we projet the harater matrix onto a subset of the eigenvetors of the ovariane matrix. For the lassifation of a harater, we have employe a moifie form of the ahalanobis / Euliean istane. To the best of our nowlege, there have been no attempts in the literature of applying the DPCA tehnique to the ontext of online harater reognition till ate. ost of the applations for whh this tehnique has been propose have been image-base suh as fae reognition [7].

2 . Preproessing Prior to feature extration an reognition, the input raw harater is smoothene to minimize the effet of noise. The harater is then resample to obtain a onstant number of points uniformly sample in spae following whh it is normalize by entering an resaling [6]. 3. Feature Extration Let the number of sample points in the preproesse harater be N p. At eah sample point (x i,y i for i N p of the resample harater, we extrat a set of loal features esribe in Setion 3.. Let F represent the j th feature erive from the i th sample point of the harater. This notation has been aopte here merely to inex the features an not to assign any weightage to them. In ase of multistroe haraters, we onatenate the stroes into a single stroe, retaining the stroe orer, before feature extration. 3. Loal Features Normalize x-y oorinates: The normalize x an y oorinates of the sample point are i i use as features an are enote by F an F. Raial Distane an Polar Angle: The raial istane an angle in raians of the sample point with respet to the entroi of the harater are ompute to form two features i i F 3 an F 4. Raial istane an polar angle from the segment mean: We fin the length of the preproesse harater an ivie it into 4 segments. Samples lying within a segment are use to ompute the mean for that segment. The raial istane an polar angle of the sample point of the harater uner onsieration is ompute as follows: when it lies in segment, ( 4 its istane an angle from the mean of that segment is the i i feature F 5 an F6. Polynomial fit oeffients: At every sample point, we inten to relate its position with respet to its immeiate neighbors. In orer to exploit this loal property, we tae a sliing winow of size ( o entere on the sample point an perform an N th orer polynomial fit on the samples within the winow using numeral tehniques. We use i j the resulting N+ polynomial oeffients as the features. For our wor, we tae =3, N= (quarat fit an aoringly enote the features as F 7 i, F8 i an F9 i. Autoregressive (AR Coeffients: We separately moel the x an y oorinates of the sample point by two N th orer autoregressive (AR proesses an use the resultant AR oeffients also as features. We employ a n orer AR proess an aoringly obtain the features F 0 i, F i, F i, F 3 i, F4 i an F5 i. It is to be explitly state that for obtaining the polynomial an AR oeffients of the first an last sample points of the harater, we assume that the last sample point of the last stroe is onnete to the first sample point of the first stroe. Suh a onnetion ensures that the notion of neighborhoo is not lost while omputing the polynomial fit features for the first an last sample point of the harater. The set of 5 features obtaine at a sample point (x i,y i are onatenate to form a feature vetor FV i of size X 5. FV = F F... F ( i i i i 5 We then onstrut a matrix C (referre to as the harater matrix in this wor by staing the feature vetors of the sample points of the preproesse harater. FV FV C ( =... N FV p It an be observe that the i th row of the harater matrix C orrespons to the feature vetor erive for the i th sample point. Therefore the size of matrix C is N p The DPCA Tehnique The main priniple behin the DPCA metho lies in projeting the harater matrix C onto an 5 imensional projetion vetor X to yiel a N p imensional feature vetor Y. We refer to Y as the projete feature vetor or the prinipal omponent vetor. Y = CX (3

3 The best projetion vetor X is the iretion along whh the total satter of the projete samples is maximum. The total satter of the projete samples an be haraterize by the trae of the ovariane matrix S Y of the prinipal omponent vetors. Aoringly we see to fin the iretion X for whh the riterion J(X is maximize. J ( X = trae( S Y (4 It has been shown in [7] that the projetion vetor X that maximizes the riterion J(X is the eigenvetor orresponing to the largest eigenvalue of the harater ovariane matrix G t efine below. G = ( C C ( C C (5 T t j j j = It is to be borne in min that we attempt to moel eah harater as a separate subspae. Aoringly, one an interpret C, C,..., C to be the training harater matres of a partular lass an C as the mean harater matrix of that lass. It an be easily verifie that for our wor, the size of the harater ovariane matrix G t is 5 5. However, in atual prate, we selet a set of projetion axes { X subjet to, X,..., X } being orthonormal to one another an maximizing the riterion J(X. These projetion axes turn out to be the orthonormal eigenvetors of G t orresponing to the first largest eigenvalues. On applying the propose DPCA tehnique to the harater matrix C, we get a family of prinipal omponent analysis vetors {Y,Y,Y } as efine below Y,,... p = CX p p= (6 For the ase where < 5, a subset of the eigenvetors of the ovariane matrix G t is employe to get the features in the reue subspae The prinipal omponent vetors an be stae olumn-wise to form an N p matrix B referre to as the harater feature matrix. = [... ] (7 B Y Y Y If instea of the DPCA tehnique, the PCA is use for feature extration [6], we first onatenate the olumns of matrix C to form an 5 N p imensional feature vetor. We then use the eigenvetors orresponing to the (where <= 5 N p largest eigenvalues of the harater ovariane matrix as the projetion axes. The size of the ovariane matrix in the PCA is (5 N p (5 N p whh is very large ompare to the 5 5 ovariane matrix G t in the DPCA metho. The signifantly smaller size of G t in turn spees up the feature extration proess in the DPCA tehnique ompare to the PCA. 5. Classifation Sheme Assume that we have training samples of a lass (harater ω. After transformation by DPCA, we obtain feature matres of the form B = [ Y Y... Y ] =,,.. From Eq. 8, we an interpret { Y } as the set of i th prinipal omponent vetors orresponing to the training samples of the lass ω. These prinipal omponent vetors have been obtaine by projeting the harater matres C, C,..., C onto the eigenvetor orresponing to the i th largest eigenvalue of the harater ovariane matrix G t efine in Eq. 5. We assume that the set of N p imensional prinipal omponent vetors { Y } are rawn inepenently from a multivariate Gaussian probability istribution funtion of the form [8]: p( Y = where ( π Y Np = e = Y ( T Y Y ( Y Y (9

4 an = T ( Y Y ( Y Y = are the estimate mean vetor an ovariane matrix of the i th prinipal omponent vetors of the lass ω. Eq. 9 gives the lielihoo of the i th prinipal omponent vetor Y for the given lass ω. For simplity, we mae an assumption that any set of prinipal omponent vetors of lass ω, { Y } an { Y } ( m m n n are inepenent of eah other. Therefore, using this we an write the lielihoo of the prinipal omponent vetors in the subspaes in whh they lie as: p( B = p( Y (0 Using Eq. 9 we an write p( B = e where = ( π T ( Y Y ( Y Y N p ( an B = [ Y Y... Y ] ( Let ω, ω ω 56 be the labels of the lasses orresponing to the 56 Tamil haraters. Given a harater, we an now onstrut a feature matrix of the form B = [ Y Y... Y ] =,...56 ( by projeting it to eah of the 56 subspaes using the DPCA. B refers to the feature matrix obtaine by projeting the harater onto the subspae of lass ω. Using Eq. we see that T ( ( Y Y Y Y p( B = e (3 The harater is assigne the lass ω for whh the following onition is satisfie. ω = arg max p( B (4 It an be reaily verifie from Eq. 3 that we assign the harater to the lass for whh the moifie ahalanobis istane is minimize. Let T = ( ( + log D Y Y Y Y then we an write ω = arg min (5 D (6 On the other han, if we employ a simple nearest neighbor Euliean istane base approah [7] for the lassifation of the ata, we first ompute the istane of the harater to its losest training sample in the subspae of lass ω as j j m m m= D = min Y Y j=,... (7 Given the set of istanes { D, D,..., D 56}, the harater is assigne the lass ω for whh the following onition is satisfie. ω = arg min D 6. Experiments an Results Data base of Tamil haraters was ollete from 5 native Tamil writers using a ustom applation running on a tablet PC. Eah writer input 0 samples of eah of the 56 istint haraters. To avoi the problem of segmentation, users wrote eah harater in a bouning box. We present our results for the writer inepenent senario, with eah lass omprising of 50 samples. The haraters are resample to 60 points an normalize to [0, ]. In the first experiment, we ompute the features liste in Setion 3 an onstrut the harater matres of size We then extrat the features in a lower imensional subspae by performing the DPCA algorithm on the training samples of eah lass separately. The size of the harater ovariane matrix use to erive the projetion axes is 5 5. On applying the algorithm, we get a ifferent subspae for eah Tamil harater. Given a

5 harater, we form its harater matrix an projet it on to eah of the 56 subspaes. The harater is assigne to the subspae for whh the moifie ahalanobis istane (Eq. 6 is least. As our seon experiment, to ompare the performane of the DPCA algorithm against the existing PCA tehnique, all features obtaine at eah sample point of a training harater are onatenate to form a 900 length feature vetor. PCA is performe on the poole training samples an a simple nearest neighbor lassifier is use to lassify the harater in the projete spae. It is to be note that the size of the harater ovariane matrix N T T R = ( x x( x x (9 x i i N T i = use to erive the projetion axes in the PCA is , whh is 60 times larger than the size of the ovariane matrix G t use in the DPCA algorithm. Herein lies the avantage of the DPCA algorithm over the PCA-the signifantly smaller size of the harater ovariane matrix G t in DPCA enables the extration of features muh faster while at the same time proviing an improvement in reognition auray over the PCA. The set { x, x,.., x } in Eq. 9 are the onatenate N T 900-imensional feature vetors of the poole training samples an x is the mean feature vetor. Tables an respetively present the reognition auray an time taen (in ses for feature extration by employing the DPCA an PCA tehniques for varying number of training samples per lass. The values in the parantheses in Table inate the number of eigenvetors require for retaining 98% of the total satter of projete samples in the DPCA. The reognition auray using the DPCA is better than that of the onventional PCA for the ase where eigenvetors of R x are hosen so that at least 98% of the total variane is retaine. (Table. We also evaluate the performane of the DPCA algorithm on the IWFHR 006 Tamil Competition Dataset [4] an foun that top reognition auray using the DPCA shows an improvement of up to 3% as against the onventional PCA (Table 3. For this ataset, we Table. Comparison of reognition auray of the DPCA versus the PCA. Value in parentheses inate the number of eigenvetors use to retain 98% of the total satter of projete samples in D PCA an 98% of the total variane in PCA. #of Training Samples per lass DPCA (ahalanobis metr % PCA 80.6% ( % 84.8% ( % 86.6% (48 Table. Comparison of the feature extration time (in ses of the DPCA versus the PCA uring training. # of Training Samples per lass DPCA PCA Table 3. Comparison of the top reognition auray of the DPCA versus the PCA on the IWFHR 06 Tamil Database [4 ]. DPCA (Euliean metr 87.5% PCA 84.% (67 use a nearest neighbor approah for lassifation of the sample (Eq. 8. Some mislassifations that our in both the algorithms an be attribute to the visual similarities of a few haraters an therefore in suh ases, the istane metrs may not be powerful enough to tra variations that mae these haraters istint. It is worth mentioning that though the DPCA metho has been e on Tamil haraters in this wor, no sript speif features have been use in the feature extration step, thereby

6 suggesting that the metho an be applie to other sripts as well. 7. Conlusion an Future Wor In this paper we have attempte to apply the reently reporte DPCA tehnique to the ontext of online harater reognition. A set of loal features is erive for eah sample point of the preproesse harater to form the harater matrix, using whh a ovariane matrix is onstrute. The smaller size of the ovariane matrix in DPCA maes the proess of feature extration muh faster ompare to the onventional PCA. Experimental results inate that the reognition auray using the DPCA sheme shows an approximate 3% improvement over the onventional PCA tehnique while at the same time being more omputationally effient. Though the algorithm has been e for reognizing Tamil haraters, it an be wiely use for the reognition of other sripts as well. Further potential areas of researh are to evelop other imensionality reution shemes that tae into aount the lass isriminatory haraterists for the online harater reognition problem, to possibly improve the overall lassifation auray. [6] Deepu V, Sriganesh ahavanath an A G Ramarishnan. Prinipal Component Analysis for Online Hanwritten Charater Reognition, Pro. of the 7 th Intl Conf.Pattern Reognition (ICPR 004 : pp , August 004. [7] Jiang Yang, Davi Zhang, Alejanro. F.Frangi an Jing yu Yang, Two Dimensional PCA: a New Approah to Appearane Base Fae Representation an Reognition. IEEE Trans. on Pattern. Analysis an ahine Intelligene, 6 (, pp.3-37, January 004. [8] Anrew Webb, Statistal Pattern Reognition. Seon Eition. John Wiley an Sons Lt, 00 Referenes [] C.C.Tappert, C.Y.Suen an T. Waahara. The state of online hanwriting reognition. IEEE Trans. on Pattern. Analysis an ahine Intelligene,, pp , August 990. [] Cheng-Lin Liu, Stefan Jaeger an asai Naagawa. Reognition of Chinese Charaters: The State-of-the-Art. IEEE Trans.on Pattern Analysis an ahine Intelligene, 6 (, pp.88-3, 004. [3] S. Jaeger, C.-L. Liu an. Naagawa.The state of the art in Japanese online hanwriting reognition ompare to tehniques in western hanwriting reognition. Intl Journal on Doument Analysis an Reognition, Springer Berlin 6 (: pp , Otober 003. [4] HP Labs Isolate Hanwritten Tamil Charater Dataset. [5] Niranjan Joshi, G Sita, A G Ramarishnan an Sriganesh ahavanath, Comparison of Elast athing Algorithms for Online Tamil Hanwritten Charater Reognition. Proeeings of the 8 th Intl Worshop on Frontiers in Hanwriting Reognition (IWFHR-8 pp , Otober 004.

Some Useful Results for Spherical and General Displacements

E 5 Fall 997 V. Kumar Some Useful Results for Spherial an General Displaements. Spherial Displaements.. Eulers heorem We have seen that a spherial isplaement or a pure rotation is esribe by a 3 3 rotation