ensors & Transducers, Vol 164, Issue, February 014, pp 5-30 ensors & Transducers 014 by IFA Publshng, L http://wwwsensorsportalcom ensor Calbraton Method Based on Numercal Roundng Youcheng WU, Jan WANG and Yan HAN Natonal Key Laboratory for Electronc Measurement Technology, North Unversty of Chna, Tayuan 030051, Chna E-mal: wuyoucheng@vpsnacom Receved: 8 November 013 /Accepted: 8 January 014 /Publshed: 8 February 014 Abstract: The roundng and operaton rules of sgnfcant fgures n tests are ntroduced The method for determnng the number of sgnfcant fgures, the roundng nterval based on the precson of test nstruments and sensors calbraton calculaton and analyss are studed n ths paper The detaled analyss process of the varous stages of the testng examples s gven as well Copyrght 014 IFA Publshng, L Keywords: Numercal roundng, gnfcant fgures, Precson, Error, ensor calbraton 1 Introducton In experments and tests, a large number of orgnal data are recorded After complcated data computaton and processng by usng a varety of algorthms, the requred test results can be fnally obtaned The precson of the orgnal data and ntermedate computaton results would drectly nfluence the precson of the fnal result Thus, the precson of data would greatly affect the judgment as to whether a system has reached the desgn requrements The orgnal data are obtaned from varous measurng nstruments Ther precsons are lmted by the precson of nstruments Theoretcally, only the approxmated measured value can be obtaned, and are usually represented by a value wth a certan number of sgnfcant fgures The sgnfcant fgures of a number are those dgts ndcatng the precson of measurement Durng the recordng and computaton, the number of sgnfcant fgures should be determned accordng to the precson of measurng nstrument In general, most expermental results are submtted after a seres of computaton of data Therefore, the choce of number of sgnfcant fgures s crucal n the process of computaton [1] All tests and computatons for tests nvolve choces of number of sgnfcant fgures A correct recordng, roundng and computaton of the sgnfcant fgures are requred for all researchers It s also an mportant factor n scentfc researches [], but s often gnored by most researchers, whch results n devaton of expermental results, or even an error n judgment of the truth [3, 4] Numercal Roundng and the Operaton Rule of gnfcant Fgures The choce of number of sgnfcant fgures means numercal roundng, whch s specfed n the latest Chnese natonal standard GB/T 8170-008 "Rules of Roundng off for Numercal Values & Expresson and Judgment of Lmtng Value" A detaled llustraton of the defnton of numercal roundng and the rules of roundng s gven n the standard document Dfferent from the conventonal rule of "roundng half up", the new rule adopted n the standard s the so-called "banker's roundng" rule: when the leftmost dgt of the dgts to be dropped n a number s smaller than 5, t s dropped, and the rest Artcle number P_188 5
ensors & Transducers, Vol 164, Issue, February 014, pp 5-30 dgts left to t are retaned; when the leftmost dgt of the dgts to be dropped s 5 wth a non-zero dgt on ts mmedate rght, the dgt s rounded up; when the leftmost dgt of the dgts to be dropped s 5 wth no more dgts or all zeros on ts rght, f the rghtmost dgt of the dgts to be retaned s an old number, then the leftmost dgt to be dropped s rounded up, or else, t s rounded down [5] Accordng to statstcs, the tradtonal "roundng half up", accordng to whch a dgt s rounded up as long as the last fgure s equal to or greater than 5, would lead to a larger result; whereas the "banker's roundng" adopted n the new standard s more scentfc[1] gnfcant fgures consst of relable dgts and doubtful (estmated) dgts [6-8], and the operaton rules are lsted below [9] In operatons of addton and subtracton, the datum wth the least number of decmal places should be treated as a reference Other data can be rounded to the number of decmal places more than the least number by one However, the fnal result should be rounded to the least number of decmal places In operatons of multplcaton and dvson, the datum wth the least number of sgnfcant fgures should be treated as a reference n roundng other data The data wth number of sgnfcant fgures greater than the least number obtaned prevously should be rounded to the number greater than the least number by one The fnal result should be rounded to the least number of sgnfcant fgures In square or square root operatons, the same treatment as n operaton of multplcaton s applcable to the former, whle the same treatment as n operaton of dvson s applcable to the square root operaton, as an nverse operaton of the square operaton In logarthmc operaton, data wth n sgnfcant fgures should be dealt wth n-dgt logarthmc table, or (n+1)-dgt logarthmc table, wth a purpose of avodng loss of precson In trgonometrc operaton, the number of sgnfcant fgures to be rounded to for the functon value should ncrease wth the reducng of angular error When the angular error s '', 1'', 01'', and 001'', the correspondng number of sgnfcant fgures to be rounded to for the functon value should be 5, 6, 7, and 8, respectvely If the calculaton result needs to be operated agan (as an ntermedate result), then the number of sgnfcant fgures could be ncreased by one temporarly 3 Precson of Measurng Instrument and Determnaton of gnfcant Fgures n the Expermental Result The rule n determnng the number of fgure retaned n the test result s as follows: the last dgt s unrelable, whle the last but one dgt should be a relable [9] Hence, the number of sgnfcant fgures s determned by the precson of measurng nstrument That s, the last dgt should be of the same order of magntude wth the precson of testng equpment For the same target, the number of sgnfcant fgures adopted would be dfferent wth nstruments havng dfferent precsons (mnmum scale) For example, the precson of some pressure transducers s expressed wth percentage of scale span If ts precson s 15 % F, and full scale s 0-00 KPa, then the measurng precson would reach 00*15/0=3 kpa If 16873 kpa s the measurng result of some test, then the actual pressure would be 16873±3 kpa It s seen, the unt dgt 8 s already an unrelable dgt, lettng alone the two decmal dgts after the decmal pont Accordng to the precson of nstrument, the test result should be rounded to unt dgt, wth the fnal result beng 169 kpa A total of three sgnfcant fgures are retaned, ncludng two relable dgts and one doubtful dgt 4 Experment Method of ensor Calbraton and Determnaton of Roundng Interval A statc pressure calbraton experment s performed on four ar pressure sensors The experment method s as follows: ar wth pressure of 0 kpa, kpa,, 0 kpa s generated by the ar pressure controller, and then the ar pressure sensors convert the ar pressure nto a voltage sgnal The voltage sgnal s subsequently converted to dgtal sgnal by an A/D data acquston nstrument and enters a computer to be stored for computaton Fnally the relatonshp between ar pressure and the output voltage of sensor would be obtaned The process stated above s adopted n calbraton of many sensors Hence, analyss and calculaton of the process s of great mportance Fg 1 s the schematc dagram of the calbraton experment of ar pressure sensor Ar Controller Ar ensor A/D Data Acquston Instrument Computer Fg 1 chematc dagram of calbraton experment of ar pressure sensors 6
ensors & Transducers, Vol 164, Issue, February 014, pp 5-30 Table 1 shows the man techncal ndces of testng nstruments Table 1 Man techncal ndces of testng nstrument Instrument Ar Controller Ar ensor A/D Data Acquston Instrument Test scale Output scale 0-0 kpa 0-0 kpa 00 %F 0-0 kpa 0-5 V (analog) 15-45 V (analog) 0-5 V (dgtal) Accuracy Remark 05 %F 0 %F Gauge Gauge 1 bt, 0-150 khz The nput and output of each secton durng the testng are calculated as follows: 41 Ar Controller Assumng the maxmum measurng span of the ar pressure controller s 1, and the precson s 1, the maxmum error would be 1 1 When the ar pressure s controlled to be P, the actual output ar pressure s P 1 1 4 Ar ensor Assumng the maxmum nput of the ar pressure sensor s P, the maxmum output s V, and the precson s, the maxmum error of output would be V The nput of ar pressure sensor s just the output of the ar pressure controller, whch s P 1 1 Wthout consderng the non-lnear condtons, the V output voltage s ( P1 1) V P If the maxmum output of ar pressure controller and the maxmum nput of ar pressure sensor s the same 1 P, then the equaton above would be V smplfed as P V ( 1 ) P 43 A/D Data Acquston Instrument A/D data acquston smply converts the analog sgnal to dgtal sgnal The scale of ts nput and output are equal Assumng the full scale of A/D data acquston nstrument s 3, and the precson s 3, the maxmum error of output would be 3 3 Its nput s the nput voltage of ar pressure sensor: ( P ), (1) V 1 1 V P Its output (a value rounded to a specfed number of sgnfcant fgures) s ( P ), () V 1 1 V 3 3 P The maxmum error of test result s V P, (3) 1 1 V 3 3 If 1 P, V 3, then the equaton would be V smplfed as P V ( 1 3) P Accordng to the techncal ndces of the test nstrument, 1 P Thus, the maxmum error of test data s as follows: V ( 1 ) 33 45 (0000 0005) 5000, (4) 00334( V ) Namely, t s approprate that the orgnal voltage value measured by A/D s rounded to 001 V 5 ensor Calbraton Method Based on Numercal Roundng 51 Roundng of Orgnal Data Table shows a part of the orgnal data when the four ar pressure sensors are at 30 kpa The orgnal data have 5 sgnfcant fgures, and need to be rounded to 3 sgnfcant fgures Table Orgnal data of output voltage wth 1-4# sensors beng at 30 kpa (unt: V) 1# # 3# 4# 3705 3338 3900 358 3705 3314 3900 3558 3631 365 3875 3509 358 316 3778 3460 3631 365 380 3509 3558 316 3778 3460 Table 3 shows a part of the orgnal data when the four ar pressure sensors are at 30 kpa 7
ensors & Transducers, Vol 164, Issue, February 014, pp 5-30 Table 3 Roundng result of output voltage wth 1-4# sensors beng at 30 kpa (unt: V) 1# # 3# 4# 37 33 39 36 37 33 39 36 36 33 39 35 36 3 38 35 36 33 38 35 36 3 38 35 5 Elmnaton of Rough Error The error exceedng the antcpated error under specfed condtons caused by carelessness of experment staff, or sudden changes of envronmental condtons, s defned as a rough error The value of rough error s relatvely large and would obvously dstort the test results Therefore, errors of ths type must be elmnated When the data volume s large, the 3σ rule s normally exploted, meanng that the datum whose dfference wth the mean s larger than 3 tmes of the standard devaton s elmnated Generally, the 3σ rule s utlzed for several tmes, untl the condton s satsfed The waveform comparson among orgnal data, rounded data and the data after rough error beng elmnated s shown n Fg 53 Calbraton of Ar ensor After the orgnal data have been rounded and the rough errors are elmnated, the output mean of the four sensors under 11 pressures s selected for subsequent operaton Because these means are ntermedate results, there would be one more sgnfcant number retaned, namely, the roundng nterval s 0001 V, accordng to the operaton rules The results can be seen n Table 4 Table 4 Outputs of 1-4# ar pressure sensors under 11 pressures Test 1# # 3# 4# 0 kpa 1540 15 1569 1548 kpa 1815 178 1839 1815 0 kpa 090 058 111 080 30 kpa 367 33 386 356 40 kpa 644 609 659 66 50 kpa 9 888 97 898 60 kpa 304 316 30 3168 70 kpa 3481 3438 347 3437 80 kpa 3759 3716 3745 3709 90 kpa 4035 3991 4017 3980 0 kpa 431 470 489 456 In the test stated above, the nput pressure of a certan sensor s assumed to be P, whle the output voltage s assumed as V When good lnearty of the sensor s guaranteed, P kv a where k, a s the undetermned coeffcent Fg Waveform comparson of orgnal data and rounded data The test data obtaned under 11 pressures are substtuted nto the above equaton, and the followng formulas are obtaned: P0 kv0 a P1 kv1 a, (5) P kv a They are over-determned equatons, for whch the optmal soluton ˆk, â would be derved wth the least square method The formulas are shown as follows 8
ensors & Transducers, Vol 164, Issue, February 014, pp 5-30 kˆ 0 0 ( V V) P ( V V), (6) Table 5 Errors of ˆk and â of 1-4# ar pressure sensors 1# # 3# 4# ˆk -09-09 -3-4 â 6 604 654 648 aˆ V ( VV) P n, (7) ( V V) 0 0 0 In the test, ˆk and â so excessve and does not reflect accurately the approprate relatonshp between varables because of the nherent error of V and P o ˆk and â should also be rounded and the roundng nterval should be determned by the error of V and P as follows ˆ ˆ ˆ k k dk dp V P ( V V) P 0 0 ( V V) 0 0 0 ( V V) 0 0 0 ( V V) P V, (8) ( V V) dp ( V V) a a daˆ dp V P V ( V V) ( V) P 0 0 0 11 ( V V) 0 V ( VV) P V 0 0 11 0 ( V V) 0 V ( VV ) 0 0 11 dp ( V V) 0, (9) Accordng to the result above, the roundng nterval of ˆk and â should be taken to be 1 The calculaton result of ˆk and â for four sensors s shown n Table 6 Table 6 ˆk and â of 1-4# ar pressure sensors 1# # 3# 4# ˆk 36 36 37 37 â -55-55 -58-57 Durng the current calculaton, the roundng nterval of V s 0001 V, then the error of P can be obtaned from the followng formula dp k, () The result of error of P s shown n Table 7 Table 7 Error of P of 1-4# ar pressure sensors 1# # 3# 4# dp 0036 0036 0037 0037 Therefore, n the calculaton process, the roundng nterval of the pressure P s should be 001 Each sensor has a group of optmal solutons ˆk, â and the lnear fttng result s shown n Fg 3 From the above results, ˆk and â error can be seen n Table 5 Fg 3 Lnear approxmaton result of nputs and outputs of the sensor 9
ensors & Transducers, Vol 164, Issue, February 014, pp 5-30 6 Conclusons References The test data be properly rounded, can make the fnal test results more n lne wth the actual stuaton On the other hand, data bts after roundng reduced to decrease the amount of calculaton, t s great convenence for the calculaton contans lots computaton and some computng requrements of real-tme In ths study, the problem of data roundng accordng to the precson of test nstrument s dscussed Wth the method, a sensor calbraton s performed n experment, and scentfcally relable results are obtaned Acknowledgements Ths work was partally supported by grants from the Natonal cence Foundaton of Chna (No 617003, 61171179, 114744 and 671193) and the cence Foundaton of hanx Provnce (No 0010-1, 0010- and 011-) It was also supported by the 973 Program (No 011CB311804), RFDP (011401006) and TTIT [1] Wang Huan, Reconsderaton of roundng and algorthm for sgnfcant fgures, Guangdong Chemcal Industry, Vol 39, No 17, 01, pp 150-151 [] Chen Jan-Me, Zhu Fang, Gu Hao, The past, present stuaton and suggeston of the sgnfcant fgures, cence & Technology Informaton, Vol 30, 007, pp 153-154 [3] Zhang hu-jan, Data processng and determnaton n nspecton report, Modern Economc Informaton, No 4, 01, pp 4 [4] Chen Zh, Wang Dongfang, Dealng wth sgnfcant dgts n academc papers n edtng work, Acta Edtologca, Issue 1, 004, pp 31-33 [5] GB/T 8170-008, Rules of roundng off for numercal values & expresson and judgment of lmtng values, Chna tandard Organzaton, 008 [6] L Chao-Rong, Xu Png, Tang Fang, Wang Mu-Bng, Physcs experment (revson), 1st edton, Bejng Unversty of Aeronautcs and Astronautcs Press, eptember 0, pp 6 [7] Yang hu-wu, General physcs experment (mechancal & thermal), 3rd edton, Hgher Educaton Press, March 000, pp 19 [8] Ln hu, Gong Zhen-Xong, General physcs experment, 1st edton, People's Educaton Press, eptember 1981, pp 6 [9] Fe Ye-Ta, Error theory and data processng, 4th edton, Machnery Industry Press, 000, pp 8-9 014 Copyrght, Internatonal Frequency ensor Assocaton (IFA) Publshng, L All rghts reserved (http://wwwsensorsportalcom) 30