ABSTRACT Circular Imaging Block Keywords:

Transcription

1

2

3 ABSTRACT Phgrammeric 3D measuring prcedure needs careful planning, especially in he clse-range case, in rder fulfill requiremens ih respec accuracy and reliabiliy f measuremens. In he special case f indr envirnmen, here imaging is be aken inside he bjec space, sme difficulies cncerning he imaging prcedure can be epeced. In hese envirnmens a special aenin has be paid he arrangemen f sensible imaging gemery, hich ill guaranee he precisin f bservains and he reliabiliy f esimaes. Smeimes, he divisin f he measuring ask in smaller sub-asks cann be avided. This, hever, requires mre planning in respec f daa regisering in rder ge he sub-mdels in he same crdinae sysem. In his research he issue saed abve is sudied and a sluin he prblems is searched and fund via adjusing he imaging prcedure suiable his special case. Grea aenin is paid he gemerical aspec f imaging fr 3D measuremens and rbusness f he sluin. In his research a ne Circular Imaging Blck mehd has been develped fr measuring asks in he inside scene envirnmen. The ne mehd is based n cnsrained imaging and leas squares esimain. ne bjecive f he research has been simplify he planning sage f he phgrammeric measuring prcedure in special circumsances. The cnrlled imaging prcedure imprves he ppruniy assess he accuracy f measuremens befrehand, and diminishes he need f assisance ih an uneperienced user design and accmplish he imaging. Als, he number f undesirable crdinae ransfrmains can be decreased, since all measuremens frm ne imaging sain ill be in ne and he same crdinae sysem. Resuls frm real-rld eperimens verify ha an adequae level f accuracy f measuremens fr bjec recnsrucin in general is aainable ih his mehd. Als, ess indicae ha he level f reliabiliy, hich is epeced in ypical clserange measuring cases, can be reached. The advanages f he mehd can be encapsulaed as he sraighfrardness f imaging, n need f cnrl daa, and he use f assised aumaic prcedures in image measuremens. Keyrds: phgrammery, clse-range, esimain, image blck, blck adjusmen, accuracy, reliabiliy 3

4 4

5 PREFACE This rk is a resul f develpmen in number f years creae and es ne ideas in he field f phgrammery. This rk sared in he psgraduae schl es paikkaiejrjeselmiss (Fress in GIS) and as funded by esmiesen siö, hich I armly hank. I as able cninue he rk as a par f research prjec funded by he inisry f Agriculure and Fresry f Finland, hm I am very graeful. Als, I uld like epress my appreciain fundains Jenny and Ani Wihuri Fundain and aanmiausalan Edismissiö, hich have given financial suppr finish my hesis. The financial suppr and resurces carry u he research have been essenial cmplee his rk. Specially, I uld like epress my graiude Prf. Henrik Haggrén h has arranged fr me hese ppruniies and als supervised my rk a he Insiue f Phgrammery and Reme Sensing. Als, I uld like hank Prf. Clive S. Fraser and Prf. Thmas Luhmann fr revieing he manuscrip and fr heir cnsrucive suggesins fr imprvemens. I have mainly carried u his research as a ne-man prjec. Neverheless, I have received unselfish help frm my clleagues a he insiue and fr his I uld like epress my graiude. Especially, I uld like hank Veli-ai Salminen, h assised me build he mrised imaging sysem and special arges fr he eperimens. Als, I am graeful Jaakk Jrvinen h carried u he gedeic reference measuremens fr he eperimens. N cmplains ere heard even he mensurain lased in he lae hurs. I uld like epress my deepes appreciain Keij Inkil, ih hm I had many fruiful discussins cncerning he esimain hery. The lve science ha I have fel is much due Jan Heikkil h guided me and my fell sudens in my early sudies. Try reach yur limis and srech hem even furher ere insrucins e learned. This aiude has helped me ever since, s hank yu Jani. I an give my armes hanks all he peple and insances, menined abve r n, h have advanced his research, friends and relaives fr n asking many imes When d yu ge he hesis cmpleed? Abve all, I sincerely uld like epress my graiude my dear ife Ania, h encuraged and suppred me hrughu his rk. Yu have been my esing field f ne ideas and hery, a fe benefis f sharing he same prfessinal backgrund. Esp, Augus 2005 Jussi Heikkinen 5

6 6

7 Cnens Lis f symbls and abbreviains 11 1 Backgrund Srucure f he hesis The bjecives f he hesis A revie f clse-range phgrammeric measuring mehds fr bjec recnsrucin Special case: Inside scene imaging fr bjec mdelling Inrducin Bundle blck esimain Cncep f Phgrammeric Nerk Design Prblem Cnsrained Imaging Circular Imaging Blck Inrducin he Circular Imaging Blck Cncep Definiin f Circular Imaging Blck Image Blck Cnsrucin Esimain f he del frm a Circular Imaging Blck Perspecive prjecin Camera mdel Esimain Prblem, Apprach I Esimain Prblem, Apprach II Image Blck Esimain Simulain Selecin f simulain parameers Nise level Lengh f radius Number f phs in blck Qualiy f iniial values

8 5 Verificain f he develped mehd Verificain mehdlgy Targes Image measuremens Sraegy f image measuremens Real rld eperimens Eperimen I Eperimen II Camera calibrain Resuls and Analysis Refinemen f he mahemaical mdel Cmparisn reference R ean Square Difference Crrelain f parameers Reliabiliy esimaes Inernal and eernal reliabiliy Reliabiliy f eperimens Discussin Applicabiliy f he mehd Direcin fr furher imprvemen Cnclusin 105 References 107 APPENDIX I 115 APPENDIX II 127 8

9 Lis f Figures 1 Imaging gemery f panramic imaging creaed in ars Eplrain Rver (ER), redran frm (aki e al. 2003) Iniial imaging gemery f a circular imaging blck accrding definiin Circular imaging blck gemery ih camera urned angenial direcin T c-cenric circular imaging blcks. The imaging gemery used in his hesis Effec f ne image bservain n nrmal mari Pr inersecin gemery Effec f nise n bjec pin accuracy Effec f lengh f radius n bjec pin accuracy Effec f number f images in blck n bjec pin accuracy Used arges in eperimens Crrespndence f image pins (shn by he perar) beeen image frm Blck I and II Back-prjecin f an bjec pin n images f ne image blck Disribuin f arge pins aached he measuring ples in Eperimen I Camera seup in Eperimen I Sep mr driven imaging sysem The enrance hall here he eperimen k place Three-dimensinal calibrain field used in calibrain Used plumb-lines in camera calibrain Pssible ransversal il f bar during imaging Heigh variain f prjecin cenre during imaging Pssible il f he camera in direcin f pical sysem pimal case, Eperimens I The lengh f pin differences respec bjec disance, indr case, Eperimen II Nn-cenraliy f hypheses and ih seleced significance level and per f es Cmparisn f panramic imaging and Circular Imaging Blcks in a ficiius indr envirnmen ih brusive pillars and eensins.101 9

10 Lis f Tables 1 Sandard deviains f blck parameers, Blck I Sandard deviains f blck parameers, Blck II pimized case RSD values in meers (m), Eperimens I Inerir space RSD values in meers (m), Eperimen II pimized case, Eperimen I, Blck I pimized case, Eperimen I, Blck II Inerir space, Eperimen II, Blck I Inerir space, Eperimen II, Blck II pimized case, Eperimen I, Blck I pimized case, Eperimen I, Blck II Inerir space, Eperimen II, Blck I Inerir space, Eperimen II, Blck II pimized case, Eperimen I, Blck I (lef) and Blck II (righ) Inerir space case, Eperimen II,Blck I (lef) and Blck II (righ) Residuals & sandardized residuals (pimized case, Eperimen I) Residuals & sandardized residuals (inerir space, Eperimen II) Cnrllabiliy, sensiiviy facrs and redundancy numbers f bservains (pimized case, Eperimen I) Cnrllabiliy, sensiiviy facrs and redundancy numbers f bservains (inerir space, Eperimen II) Cnrllabiliy facrs respec bjec disance; Eperimen I: pimized case (upper), Eperimen II: Inerir space (ler), Cnrllabiliy separaed - and -cmpnens (Eperimen I, pimized case) Cnrllabiliy separaed - and -cmpnens (Eperimen II, inerir space)

11 $ % ( & %, J J J J J J G LIST F SYBLS AND ABBREVIATINS Symbls residual vecr parameer cefficien mari unknn parameer vecr vecr f adjused bservains eigh mari nrmal mari reference variance variance-cvariance mari f bservains cfacr mari f bservains eigh f bservain variance f bservain number f unknn parameers!#" redundancy f esimain number f bservains pserir esimae f reference variance '& ( esimaed variance-cvariance mari f unknn parameers *)+) variance-cvariance mari f residuals esimaed unknn parameer vecr.- mean sandard deviain f bjec pin crdinaes./ sandard errr f inciden angles srengh f he nerk gemery scale number number f addiinal images mean bjec disance shif f prjecin cenre crdinaes 9;:<5=9;> 4 rain angle f -ais f lef and righ image respecively 9?5 4A@ 5 4CB rain angle difference amng rain angles.d5 7 image crdinaes E;FHG cnsrain f base vecr lengh IKJ lengh f he individual base vecr IL mean lengh f he base vecr N : prjecin cenre X-crdinae f lef image f image pair N > prjecin cenre X-crdinae f righ image f P image pair Q : prjecin cenre Y-crdinae f lef image f image pair Q > prjecin cenre Y-crdinae f righ image f image pair R : prjecin cenre Z-crdinae f lef image f image pair R > prjecin cenre Z-crdinae f righ image f image pair S ( angle difference f -aes f lef and righ image; image pair 11

12 c ƒ Ÿ i y ˆ ˆ i ˆ ˆ ˆ TVUXW angle difference f y-aes f lef and righ image; YPZ[ image pair T]\^W angle difference f z-aes f lef and righ image; Y Z[ image pair TV_a` mean angle difference f -aes f image pairs TVUb` mean angle difference f y-aes f image pairs T]\` mean angle difference f z-aes f image pairs ced fhgji chk fhgli i=1,2,3; j=1,2,3 rain mari elemens f lef and righ image mn=p W i mjnrq W i mjn=s W cnsrain f rienain angle differences radius f camera plane rain in image sequence vu g bjec pin zy { g prjecin cenre pin } f~ image pin scale facr beeen 3D and 2D image spaces rhnrmal rain mari camera cnsan Au g i u g ia u g bjec pin crdinaes Aˆ i i prjecin cenre crdinaes Š ia iaœ rains f he camera crdinae aes,y and z respecively Ž i Ž linear disrin crreced camera cenred image crdinaes y i principal pin crdinaes crrecin erm f affiniy crrecin erm f rhgnaliy angle beeen ske y-ais and crreced ne. i radial disrin crreced image crdinaes i decenring disrin crreced image crdinaes c# radial disance f image pin frm he principal pin c ˆ radial disance frm he pin f bes symmery š œ parameers f radial disrin } i } parameers f decenering disrin u{{ i u {{ crreced camera cenred image crdinaes c#ž linear disrin crreced radial disance f prjecin cenre pin f ž Z[ image in image sequence nrmal vecr f rain plane bservain cefficien mari cnsrain cefficien mari Š ˆ i i Œ camera rain angles f he firs image in circular image blck f i i=1,2,..., e plane rain angle f ž Z[ image in circular image blck? j r = ªa rain mari f camera ih respec crdinae sysem «lplane rain mari? j ªacamera rain mari f žz[ image f±g i i=1,2,3; j=1,2,3 rain mari j ªa elemens f i i=1,2,..., j³ plane rain angle f ž Z[ image in secnd circular image blck f±g 3D disance f bjec pins } f and } g Š deviain f Š ˆ angles deviain f angles 12

13 ¾ ½ ë ¹» Ý ã ã ã ã Ö Û µ] deviain f. angles µp deviain f radius µº¹ mean deviain f»6¼ angles crrelain cefficien ¾H ÀXÁ=ÂDÃVÄ gray-level value f piel in (i,j) lcain f emplae image pach ¾eÅlÀXÁrÂÃ]Ä gray-level value f piel in (i,j) lcain f arge image pach ¾H  ¾Å ¹ Æ mean gray-level values f emplae and arge image paches ÀÇ6Â È Ä gray-level bservain frm emplae image ÉPÀÇ6Â È Ä randm nise in emplae and arge image ¾À Ç6Â È Ä ranfrmain funcin fr he arge image Ê Â ÊË Â ÊšÌlÂ Í Â ÍÎË Â Í Ì affine ransfrmain parameers used in LSQ-image maching Ç Â È iniial emplae lcain fr LSQ-image maching ÀÇ6Â È Ä iniial ranfrmain funcin fr he arge image ļ shif cmpnen in linear radimeric ransfrmain ÎÐ scale facr in linear radimeric ransfrmain» line direcin in parameric 2D line presenain µ line disance parameer in parameric 2D line presenain µ Ç disrin crrecin erm f -crdinaes µ È disrin crrecin erm f y-crdinaes Ç Ò crrecin vecr f unknn parameers ¼ addiinal parameer, ransversal il angle Ój¼ addiinal parameer, il angle f pical ais Ô ÕlÖ ØÖ plane rain mari, ih addiinal Ò ¼ angle Ô ÕlÖ ÙDÖ ØÖ plane rain mari ih addiinal Ó¼ and Ò ¼ angles ÚÜÛ effec f bservainal errr n he residual ÚAÞ bservainal errr Ô ß redundancy mari àâ esimaed cfacr mari f residuals Ö variance f Á Ðå residual Ö sandard deviain f Á Ðå bservain çk¼ sandardized residual è null hyphesis è é alernaive hyphesis ¼ lcal redundancy number cnneced ih Á Ðå residual ê ¼ ê nn-cenraliy parameer saisically derived nn-cenraliy parameer» significance level f saisical es per f he saisical es êjì cnrllabiliy facr Ú Þ ¼ size f grss errr n eceeding he limi f rejecin í ¼ sensiiviy facr 13

14 Abbreviains 3D 2D CCD ICP CD NRC RSE ER NASA IU LSQ ZD FD SD TD PAL NTSC LAPACK RSD RSDX RSDY RSDZ Three dimensinal T dimensinal Charge Cupled Device Ieraive Clses Pin Cncepual Daa del Nainal Research Cuncil f Canada R ean Square Errr ars Eplrain Rver Nainal Aernauics and Space Adminisrain Agency Inerial easuremen Uni Leas Squares Zer-rder design Firs-rder design Secnd-rder design Third-rder design Phase Alernaing Line (vide sandard) Nainal Televisin Sysem Cmmiee (vide sandard) Linear Algebra PACKage R ean Square Difference R ean Square Difference f -crdinaes R ean Square Difference f y-crdinaes R ean Square Difference f z-crdinaes 14

15 1 BACKGRUND The cncep f a mdel can be undersd in varius ays, depending n he cne in hich i is menined. The mdel iself can be cnsidered be a descripin f a phenmenn rien in he frm f a mahemaical frmula. A mdel can als be regarded as a prduc f a planning r design prcess. I migh be a mdel f a manufacuring prcess r i can epress he shape and size f a designed bjec, building r arifac. In his hesis, he rd mdel denes a gemerical realizain f an eising bjec, r grup f bjecs, and heir relainships in hree dimensinal space. The erm bjec recnsrucin can be cnsidered be a synnym fr he rd mdelling in his sense. bjec recnsrucin r gemerical mdelling can be said cnsis f deermining he gemerical prperies f an eising bjec and is relain is surrundings a a given insan in ime. In his hesis, he mdelling prcess is cnfined a saic bjec, r bjecs, and heir gemeric relain each her. An bjec mdel cnsiss f he gemerical prperies f he bjec, is relain her bjecs, is aribues cnneced ih epressing is prperies, is maerial, clr prperies and funcinaliy ec. relaed cerain applicains. The fcus f his hesis is n acquiring hree-dimensinal crdinae infrmain abu an bjec frm a sequence f phgraphs. In a image-sequence analysis, fen a saic camera pse is assumed and he mvemen f bjecs is he arge f ineres, hereas, here, he bjec is assumed be saic hile he camera is mved beeen cnsecuive epsures. In cnras research bjecives in he field f cmpuer visin, here he fcus f research in bjec recnsrucin has been n he accuracy and precisin f esimaes. The aumain f measuremens and sensr-rienain deerminain has been f nly secndary ineres. The fleibiliy f he measuring sysem is, hever, he cncern f he research, bu n a he cs f accuracy. The cncep f accuracy is undersd here as he qualiy f he resuls, hile precisin is undersd as a qualiy f he measuring sysem iself. Accuracy represens he gdness f resuls in respec sme sandards, hile precisin quanifies he suiabiliy f bservain be used in a mahemaical mdel. Smeimes, accuracy is presened as a relaive number f esimaed bjec crdinae accuracy in respec he maimum dimensin f he bjec, fr eample îðï*î#ñœñœñœñ. This kind f represenain f he accuracy number is quie cmmn, especially in clse-range phgrammery. The main fcus hrughu he hesis ill be n eamining a specific mehd f baining hree-dimensinal gemeric infrmain n an bjec r bjecs. The research is limied clse-range phgrammeric measuring mehds; all measuremens are herefre epeced be made using erresrial-based, raher han airbrne, mehds. 15

16 The image-based 3D measuring mehd develped in his research prjec is a nvel mehd designed fr use in special cndiins fr he purpse f bjec recnsrucin. This special case means ha imaging is be aken inside he bjec space and he gal is recnsruc he surrunding bjec(s). This ype f imaging case ill be dened in his hesis as a inside scene imaging frm n n. The mehd develped in his research is called Circular Imaging Blck -mehd. The mehd can be cnsidered be an addiinal mehd clse-range phgrammeric measuring mehds, especially designed fr inside scene imaging case. The 3D measuremens made by applying his mehd are based n image bservains frm sequence f images. The sequence is aken symmerically ih respec ne navel pin and he lcain f he sequence ill be referenced as imaging sain. This is dra a disincin beeen i and camera sain ha is a lcain f a camera, here a single image is aken. The flling revie in Secin 1.3 f phgrammeric measuring mehds is a crss-secin f curren mehds designed acquire 3D infrmain fr he purpse f bjec recnsrucin. The revie is especially cncerned ih he acquisiin prcess relaed his rk in he field f clse-range phgrammery, n abu bjec mdelling iself. 1.1 Srucure f he hesis In his hesis he specific clse-range phgrammeric prblem regarding imaging gemery design is discussed. The subjec is reaed frm an imaging design pin f vie in specific inside scene bjec recnsrucin cases. In Chaper 1 a clser lk is aken a he echniques fr prviding measuremens fr bjec recnsrucin. re aenin is paid hse mehds ha are applied in inside scene envirnmens. Chaper 2 gives an vervie f he mahemaical backgrund f he esimain hery f phgrammeric measuremens. In addiin, he hery and cncep f phgrammeric nerk design is discussed. Als, research rk dealing ih cnsrains in clse-range phgrammeric measuremens is revieed. In Chaper 3 he descripin f he Circular Imaging Blck -mehd is prvided. In his chaper he mahemaical backgrund f he develped mehd and assumpins made are revealed. The mahemaical cnsiderain is based n hery presened in Chaper 2. The subjec f Chaper 4 cncerns he simulain ess accmplished in rder demnsrae he nerk design pins ih he presened mehd. The es arrangemens f he accmplished real-rld eperimens and he acquired reference daa are presened in Chaper 5. The presenain als includes a descripin f he echniques used in he image bservain acquisiin. In Chaper 6 he refinemen f he presened mehd is given. The imprved mahemaical mdel and he bained resuls f cmpuains are analysed frm 16

17 he pin f vie f measuring accuracy and he reliabiliy f he measuremens. A brief revie f reliabiliy analysis is prvided fr clarificain f he echniques used. The feasibiliy f he develped mehd is discussed in Chaper 7. A fe aspecs f he usage f he develped mehd are highlighed and a cmparisn ih panramic imaging is made. The discussin cncenraes n he applicabiliy f he mehd in an inside scene imaging case. In addiin, suggesins fr furher develpmen and direcin f research are given. In Chaper 8 cnclusins are dran and sme recmmendains are given. 1.2 The bjecives f he hesis In his hesis, he prblem f clse-range nerk design in special cndiins ill be sudied a a deep level. The aim f he research is find and develp a ne measuring mehd, hich is based slely, n image infrmain, prvide 3D measuremens in inside scene imaging case. This research ill cncenrae n mehd develpmen, in rder prvide a fleible measuring sysem fr users n necessarily having a backgrund in phgrammery. The gemerical characerisics f he measuring cndiins, fr hich he measuring sysem is be designed, are cnsidered be difficul frm he phgrammeric nerk design pin f vie. A sluin he prblem f meeing he inside scene bjec recnsrucin needs, here he imaging ill be carried u frm inside uards, ill be sugh. The saring pin fr he research is simplify he clse-range phgrammeric design prcess in his paricular measuring case. The sluin ill be sugh hrugh verdeerminain ih redundan image infrmain and by regulaing he ay he imaging is accmplished. The priri infrmain f imaging ill be used in he frm f cnsrains in he image blck adjusmen. As a cnsequence f he simplificain f he imaging design, sme degradain f accuracy in bjec recnsrucin cmpared an pimal sluin can be assumed. Therefre, he flling quesin is psed: Can a circular imaging blck be rbus enugh and prvide bjec measuremens fr he purpse f phgrammeric bjec recnsrucin? The anser his quesin ill be sugh using mehds invlving leas square esimain and accuracy assessmen. The reliabiliy f esimaes and qualiy f measuremens ill be defined by means f saisical esing. 17

18 1.3 A revie f clse-range phgrammeric measuring mehds fr bjec recnsrucin In clse-range phgrammery, measuring asks can be planned accrding he required accuracy f he bjec mdel, he required cmpleeness f he mdel, presenain f he mdel, and prperies f he bjec iself. T a large een, he purpse f he measuremens dicaes hich mehd is apprpriae fr he ask. Is ime he resricing elemen fr mensurain r can he measuremens be made n an ff-line basis? Wha is he usage f he resuling bjec mdel? Will i nly be used fr visualizain r ill any measures be derived frm he mdel? Are e ineresed in he lcain f a fe disinc pins n he surface f he bjec and heir relaive psiin in respec a ime span, r is i mre impran creae a realisic-lking cmprehensive mdel f an bjec ha cmprmises ulimae gemeric accuracy? Archaelgical dcumenain: Smeimes he mdel required des n have be in 3D. I is fairly cmmn n archaelgical sies ha images are aken as a basis fr 2D skeches and s 2D is sufficien. ccasinally, i als means sere pairs r recified images f he relics f aniquiy r msaics are required. The use f phgrammery in archaelgy is quie fen resriced baining phgraphs ih a knn image scale fr dcumenain purpses. Seldm has he cmplee gemerical mdel f he sie been regarded as necessary, alhugh recenly sme ineres has been arused in recnsrucing 3D mdels f archaelgical sies as par f he dcumenain prcess (gleby 2001; Kisinen e al. 2001; Pllefeys e al. 2003). A hree-dimensinal visualized mdel f he curren srucure f he sie has been recgnized by archaelgiss as giving an addiinal l fr inerpreing he sie. The sub-mdels f he arifacs r heir sere pairs are als made in rder sre hem in a daabase (Chikasu and Anai 1998). Virual realiy mdels: Recenly, i has, perhaps, msly been virual-realiy mdels ha have inspired researchers all ver he rld develp mehds fr bjec mdel recnsrucin. Virual mdels are required in he cnsrucin indusry fr faciliy-managemen purpses and in he mining indusry fr aunmus vehicle r machinery ineracin, als knn as eleperaing (El- Hakim e al. 1997, 1998; Sequeira e al. 1999). Hever, perhaps he larges penial can be seen in he enerainmen indusry and, especially, in he gaming indusry. Creaing a virual bjec mdel des n necessarily require any real measuremens be made, bu can be accmplished frm scrach even hugh i has been realized ha demand fr real-rld-based, raher han synheically-generaed cnen mdels is rapidly increasing. This is because real-rld daa have he penial generae realisic lking mdels in a mre aumaic and faser manner han heir graphic-based cuner-pars, hich are als mre labr-inensive prduce (El-Hakim e al. 1998). In many cases, he virual mdel is a cmbinain f acual measuremens ih real-rld image eure and parial graphical ma- 18

19 nipulain by he perar ih synheic eure. This is especially he case ih virual mdels f archaelgical sies creaed here nly remains f he mnumens are presen. The cmplee srucure f he sie has be clleced frm her surces, frm lieraure, fr eample (gleby 2001). The gemeric cnen f he mdel des n have be cmplee fr all applicains here virual mdels are used. Fr applicains here sme measures are be derived frm he mdel, he cnsisency f he mdel ih respec scale and derived feaures has be cnrllable. Quie fen, deailed pars f he mdel are firs mdelled in very rugh r generalized frm, hile real-rld image daa are used fr surface eure in rder cmpensae fr he incnsisency f he gemeric mdel. This is quie a perful echnique and an accepable apprach if he mdel is used fr visualizain nly. bjec recnsrucin: The measuremens be carried u fr recnsrucing an bjec mdel can be accmplished by using a variey f insrumens. The mdel can be based enirely n phgrammeric measuremens r measuremens migh be carried u using a hybrid mehd invlving differen ypes f sensr daa. Recenly, he laer seems be he ms ppular mehd amng he laes research prjecs (Ng e al. 1998; El-Hakim e al. 1997, 1998; Brenner and Haala 1998; gleby 2001). In many prjecs, he cmbinain f laser scanner and vide imaging is used (El-Hakim e al. 1997, 1998; Ng e al. 1998), bu here are her cmbinains, such as a achemeer and sill vide images (gleby 2001), airbrne laser scanner daa, 2D map daa and aerial and erresrial images (Brenner and Haala 1998). Als, a fully image-based apprach bjec recnsrucin has been suggesed (Seales and Faugeras 1995), here a sainary bjec is recnsruced frm an image sequence. The camera mvemen is esimaed n he basis f cmmn feaures n he images and n bjec pins deermined frm silhuee pins. Laser scanning: Laser scanning is a perful echnique fr eracing 3D gemeric infrmain f bjecs. Here, he laser scanner is undersd be a device hich aumaically cllecs 3D infrmain frm an bjec surface in a sysemaic paern. I is essenial nice ha daa are clleced in direcins and usually sred in a frm f a grid. This clarificain is make a disincin beeen laser-based devices ha cllec daa nly in ne direcin hile daa in he her direcin is bained hrugh he mvemen f he device iself r f he bjec. Cmmn all hese devices is he accmplishmen f he cllecin f daa pins a a very high rae (hundreds r husands f pins per secnd), alhugh, differences can be seen beeen 3D laser scanners in regard f heir peraing range and he echnique hey are based upn. Cmmnly, hese devices can be divided in hree caegries ih respec peraing range: 1) shr-range, 2) mid-range, 3) lng-range. Wih regard he echnique laser scanners are based upn, insrumens can be divided in devices based n riangulain echniques and devices based n imef-fligh. The firs caegry cmprises a cmbinain f a laser-beam prjecr and 19

20 a imaging sensr device. The disance measuremen in hese devices is based upn angle bservains ih respec he base disance f he laser and imaging sensr. The 3D pin crdinaes are slved based n measuremens f pan- and il-angular sensrs and derived pin disance value. These devices ypically fall in he caegry f shr-range devices, bu here are als devices ha can be cnsidered be in he mid-range caegry. The devices based n a ime-f-fligh echnique emi a laser pulse and he ime passed unil he refleced pulse arrives back a he receiver is measured. In sme devices, he phase difference f emied and received pulses are measured and a mre accurae esimae fr ime difference can be calculaed. In general, devices based n riangulain can be cnsidered be mre accurae han laser scanners based n a ime-f-fligh echnique. Single-pin accuracy can be even ò óõô#ö ö in he bes case ih riangulain-based devices, bu accuracy ill drp abruply hen he bjec disance increases. A a disance f en meers, he accuracy can degrade several millimeers (Behler and arbs 2002). Wih ime-f-fligh scanners, he accuracy f disance measuremen is quie cnsan, regardless f bjec disance. Hever, a single 3D pin accuracy is n cnsan since measuremen is a funcin f disance and angular bservains. The accuracy ih hese devices is ypically ens f millimeers in relain a single-pin lcain. Unlike riangulain-based devices, hese scanners are able measure up a fe hundred meers. In cmparing cmmercial scanner brand ypes, differences can be fund in he ay hey perae. In addiin peraing range, scanning speed and angular resluin, here are aribues ha are scanner-ype relaed. Als, he maimum field f vie, here a single scanning sessin is bund be cnduced, des vary. There are scanner ypes ha all he scanning f nearly a full sphere, hile ih her devices he scanning is resriced a field f vie f he size f øšòœù in ne direcin. The use f a narr field f vie in scanning a large cmple bjec ill generally lead muliple scanning sessins in rder cver he hle bjec. Separae scannings mus parly verlap be able merge daa ses in a single bjec mdel. This increases he number f scanning sessins even mre. Daa merging and daa regisering, has becme a challenge ha has inspired many researchers find a feasible mehd in rder preserve bjec cnsisency in bjec recnsrucin. Wih ime-f-fligh based devices, he accuracy f measured disance is nearly cnsan ihin a fe ens f meers, bu here is n pssibiliy f evaluaing he accuracy f an individual 3D pin bservain. This is because i is hard, r even impssible, rack he precise lcain, here he laser beam is refleced frm. Smeimes, special 3D regular arges are used in rder esimae he accuracy f a se f 3D pin bservains. This can be cnsidered be a draback f laser scanning mehds, in cmparisn ver-deermined image-bundle measuremens. In invesigaing laser scanning accuracy, i has been niced ha he measuremen nise increases epnenially ih he angle beeen he sur- 20

21 face nrmal and laser beam (Sequeira e al. 1999). The same phenmenn f reduced scanning accuracy has been discvered hile cmparing measuremens achieved by laser scanning ih digial phgrammeric measuremens (Lichi e al. 2002). This knledge f inaccuracy f bservains n he surface near he cplanar--ranging beam has been eplied in range daa regisrain (Sequeira e al. 1999). In he case f verlapping daa, nly hse pins frm he scanning ha have a beer vieing angle ih respec he surface nrmal hey belng have been included in he mdel. In ms f he cases, he bjec cann be mdelled frm ne viepin nly, s several insrumen sains are needed. T esimae crdinae ransfrmain, daa ses mus verlap. Because f he naure f laser scanning, n pin--pin crrespndences can be esablished, s daa ses have be eamined in rder find deph discninuiies fr 3D feaure eracin. In verlapping areas, he same 3D feaures have be idenified in bh daa ses. Transfrmain can be hen based n minimizing he pin disances in he verlapping area ih an Ieraive Clses Pin (ICP) r equivalen algrihm (Ng e al. 1998; Sequeira e al. 1999). The prblem ih cmbining he daa in he ay described is ha here is n cnrllabiliy f he ransfrmain and is accuracy unless he crdinae ransfrmain is esablished in anher ay. T slve his prblem, special 3Darges have been develped. Targeing as in phgrammery increases he need fr careful planning f he mensurain ask; his reduces he fleibiliy f he measuring mehd. Sme laser scanning devices prvide he 2D reflecance image f a scanning grid. Fe reprs have been given here his image has been used successfully help in he deerminain f ransfrmain beeen separae scans. Als, an apprach has been suggesed here a discree rhimage is calculaed frm image daa and an inerplaed rhimage grid is cmpared ih inensiy values received frm laser device (Wend 2004). In he lympia prjec, he digializain f he saue f Zeus as esablished by aaching a laser scanner he arm f a crdinae measuring device (gleby 2001). This guaraneed ha he pins cluds ceised in he same crdinae frame. Unfrunaely, his kind f apprach can nly be used ih fairly small bjecs. The uncerainy f sub-mdel regisrain based purely n laser measuremens has been he reasn fr lking fr her alernaives. In sme research prjecs, he sluin has been cmbine differen measuring echniques in rder achieve a beer verall accuracy f he mdel. In cmbining laser range daa and digial images, laser scanning is used ge ms f he gemeric daa f he scene, hile images are used nly fr achieving a naural eure fr he mdel (Sequeira e al. 1999). Hever, here are prjecs, here image daa are als used fr geing gemeric daa fr daa fusin, r even fr eracing he bjec srucure alng ih laser daa. In cases here phgrammeric measuremens have been regisered geher ih range daa, a beer and mre precise crdinae sysem fr he cnrl nerk has been bained (Guidi e al. 2002; El-Hakim e al. 2002; Beraldin e al. 2002). Bu here are als papers repring he cmbinain f achemeer measuremens and laser scanning, r f all hree echniques, in 21

22 he same prjec (Balzani e al. 2002; Brg and Cannaaci 2002). The general endency has been use mre precise echniques creae a cmmn crdinae sysem fr measuremens, and mdel nly dminan feaures ih phgrammeric mehds, leaving mre deailed mdelling be carried u ih laser scanning. Image-based bjec recnsrucin: In ms accurae clse-range phgrammeric measuremens up úüûðú#ýœýœýœýœýþ ú ûðúlýœýœýhýœýœý imaging is based n a muli-sain cnvergen camera cnsellain (Fraser 1992). This requires a grea deal f eperise in phgrammeric nerk design. Usually, all pins be measured are pre-argeed ih rerreflecing r high-cnras arges. The recnsruced bjec mdel ill hen be based n hese defined bjec pins. Typically, hese pins hemselves are disincive pins f he bjec mdel, r hey are disribued n he bjec surface bes describe he shape f he bjec mdel. The measuring asks requiring such accuracy are usually unique asks n be carried u regularly, unless i is abu peridic inspecin; fr eample in bjec defrmain measuremens. The argeing can als be carried u by prjecing srucured ligh n he bjec surface r by using naural pins in measuremens. This slighly degrades he achieved accuracy, bu enables he measuremens be made n a regular basis. Sme research has been cnduced, especially in he field f cmpuer visin, uilize perspecive prperies as vanishing pins in rder reslve camera rienain ih respec he bjec. The idea is based n he assumpin f parallel and rhgnal lines r f a symmeric paern n he bjec srucure. This apprach des n sui he recnsrucin f all ypes f bjec, bu here have been encuraging resuls, especially in he field f archiecural phgrammery (van den Heuvel 1998, 2003). Image-based mdelling frm an image sequence has been f ineres many researchers seeking recver he shape f real bjecs, especially in he field f cmpuer visin. In sme research aciviy, he min f he bjec r camera is epeced be rigid (Seales and Faugeras 1995) e.g., he bjec is raed arund ne rain ais r he camera min is assumed fll a defined rack. ne research line ha deserves a clser lk is an apprach here priri infrmain abu a camera r scene is ally disregarded and bjec recnsrucin is perfrmed aumaically. In he research f a Belgium grup frm Leuven (Pllefeys e al. 2004, 2003, 2000), he fcus has been n aumaic recnsrucin and he mehdlgy used is prbably unfamiliar many phgrammeriss. The image sequence is assumed be aken ih a hand-held camera and he mvemen f he camera is alled be quie arbirary. The mdelling apprach is based n he idea f firs recvering he min f camera and, afer ha, deermining he bjec parameers f cerain feaure pins. The feaure pin selecin is based n using he ineres perar in rder find gd feaure pins fr racking in he image sequence. The camera min esimain and main srucural recvery is accmplished in prjecive space. The esimain is iniialized firs ih an image pair, hich is seleced have an apprpriaely ide separ- 22

23 ain f image vie, i.e., a gd base rai. The crdinae frame is iniialized and her images f he sequence are included gradually in he cmpuain, hile he bjec srucure is updaed accrdingly. In his sage, nly a basic camera mdel is assumed. Afer including all images f a sequence, and nce heir subsequen epiplar gemery is reslved, a bundle adjusmen is perfrmed, hile inrinsic camera parameers are als aken in accun. The adjusmen is made in prjecive space; in rder ransfrm bjec srucure in meric space, cnsrains n he camera inrinsic parameers are impsed. A mre-dense bjec srucure is recnsruced frm recified sere image pairs f subsequen images. The resuls f image maching are dense dispariy maps and hese are hen cmbined in a glbal deph map by linking pins frm separae sere mdels. In he final sage, frm his deph map, a surface mdel in he frm f a riangular mesh is creaed. uliple image vies ensure ha he eure frm images mapped n he surface mdel creaes a visually realisic lking mdel f he bjec srucure. The basis f his research has been creae an image-based bjec recnsrucin mehd, here n sric resricins fr imaging are assigned. The emphasis has been n he epliain f aumaic prcedures. Aspecs f imaging gemery n he accuracy f he bjec mdel have n been a big cncern in research. The fcus seems have been n he develpmen f rapid mehds f bjec recnsrucin prvide visual mdels frm images. The resuling bjec mdels cnsruced ih his mehd have been very impressive. The research rk is als encuraging ih respec he develpmen f aumaic recnsrucin mehds Special case: Inside scene imaging fr bjec mdelling Especially in he indr envirnmen, he imaging sraegy plays a very impran rle. This as niced by Dr. Framii h cnsruced an imaging sysem fr archiecural phgraphic recrding in he early 1970s in Ausria. The bjecive f his imaging sysem as prvide an image sequence in a sensible rder recrd archiecural deails f a scene fr dcumenain(framii and Ackler 1976). The imaging sysem cnsised f a bar aached n a ripd and a camera aached he end f he bar. By urning he bar arund he ripd and iling he camera, he culd phgraph he srucures in an inside scene envirnmen. Hever, n phgrammeric measuremens ere accmplished. His invenin as made in he 1970s and he phgraphs ere analg images; he insrumenain f analgue phgrammeric sere-plers, used in hse days, resriced he use f heavily iled phgraphs fr phgrammeric mensurain. The cmpleiy f inside scene imaging and bjec recnsrucin has inspired many researchers since Framii search fr ne ideas and mehds achieve realisic and accurae mdels f he indr envirnmen. Sme f hese effrs have cncenraed n he mdelling par, such as in research by Haggrén and aila (Haggrén and aila 1997), here he cncep f mdelling as represened as an bjec-space-driven apprach. The indr scene as mdelled 23

24 ih he help f an bjec-mdel library and he pse and rienain f he images ere defined ih respec bjec mdels. The mdelling as dne ineracively and he crdinae sysem as esablished during he mdelling prcess. The research fcused n mdelling nly a par f he scene. Free-ne adjusmen as accmplished aferards in rder reduce he pssible defrmains f bjec space during he prcess f ineracin. A similar philsphy f bjec recnsrucin as adped by Khalil and Grussenmeyer (Khalil and Grussenmeyer 2002) in an apprach bjec mdelling frm a single image. Their saring pin fr he sudy as slve he image eerir rienain n he basis f a deerminain f vanishing pins. S he idea is basically he same as ha in he rk f Van den Heuvel in he field f archiecural phgrammery (van den Heuvel 1998). The sysem design f he sudy invlves he cncep f a Cncepual Daa del (CD) in he recnsrucin f indr scenes. Infrmain is represened in a daa mdel n hree levels; gemeric, plgical, and semanic. Infrmain n he cplanariy and parallelism f bjecs sred in a relainal daabase are hen eplied in he mdelling f he bjec scene. In his mehd, hree disance measures are required calculae he scale facrs he aes f a lcal crdinae sysem. A mre ambiius prjec prduce an accurae mdel in an indr envirnmen has been se up by he Nainal Research Cuncil f Canada (NRC) (El-Hakim e al. 1997, 1998). There has been develped an aunmus mapping sysem designed especially fr recnsrucing a mdel f a crridr r unnel-ype indr scene. The sysem is carried u using a hybrid mehd by cmbining differen ypes f sensr daa. The sysem cnsiss f heel encders, a laser range sensr and 8 CCD cameras. The cameras are fied in a frame, hile cameras pin sideays cver an image srip f a half circle. The range sensr is muned n a pan and il uni, s i can cver he same area as he CCD sensrs. All daa is gahered aunmusly. Hever, sme assisance is required n image feaure acquisiin. The pse f he sysem is reslved n he basis f infrmain frm all sensrs. Fr bjec recnsrucin nly, he image and range daa are used. The sysem is unique, since all daa is emplyed in a cmmn nerk adjusmen slve he pse and rienain f he mapping uni and he bjec mdel simulaneusly. The accuracy achieved in eperimens is hich is admirable in such an envirnmen f high cmpleiy. Hever, pre-calibrain f he sysem ih help frm a calibrain frame is necessary in rder slve he relaive lcain and rienain f individual sensrs. This infrmain is hen used as priri knledge in he nerk adjusmen prcess sabilize he cmpuain. Panramic imaging: The use f panramic images in bjec recnsrucin has been f ineres in many papers recenly. Panramic images can be cnsidered be ideal media fr acquiring images in inside scene envirnmen. The idea f panramic images has been knn fr ver a cenury, bu using his cncep fr measuremens in clse-range phgrammery has nly recenly been f ineres. 24

25 Panramic images can be bained by merging muliple cenral prjecive frame images in a single panramic image r by using cameras especially designed acquire he inensiy infrmain ih panramic prjecin. In rder acquire panramic image infrmain frm muliple cenral prjecive frame images, i is necessary have a single prjecin cenre ha all images share. This can be accmplished by esimaing mahemaically he pssible eccenrical difference beeen prjecin cenres and aking in accun his effec in cnsrucing he panramic image (Weser-Ebbinghaus 1982; Harley 1993; Luhmann and Tecklenburg 2002), r by rying place he camera in a rainal mun piece s ha n eccenrical difference eiss beeen image perspecive cenres (Pöninen 1999; Kukk 2004). In he case f special panramic cameras, he imaging is based n a single array sensr. The panramic image is cnsruced hile applying plane rain fr he verically aligned sensr and acquiring he inensiy infrmain during rain. The spaial resluin f he image is hen dependen n a minimum rainal resluin in he hriznal direcin, hile, in he verical direcin, he resluin is limied by he number f sensr elemens in he array. The sensr array is assumed be perpendicular he ais f rain, bu here have been reprs f his assumpin having been fund be incrrec (Schneider and aas 2005). Als, here have been bservains f uneven rain f he sensr in he direcin f rain and f vilains f a planariy cnsrain (Parian and Gruen 2004). By applying crrecin values based n calibrain, i has been pssible reduce he errr f he single bservain frm piels (Schneider and aas 2005). Hever, bjec recnsrucin, based upn he image ray inersecin, cann be achieved frm a single panramic image. The image bservains frm a panramic image can be equaed ih he hriznal and verical angle bservains frm he hedlie. Cnsequenly, be able measure 3D pin lcain by means f image ray inersecin, a leas panramic images have be creaed ih a difference in lcain. In rder have bservainal redundancy, hree panramas are he minimum. Befre bjec recnsrucin, images have be relaively riened r riened ih respec a chsen crdinae sysem. Wih he use f phgrammeric bundle adjusmen, he rienains can be cmpued in a rigrus ay. Luhmann and Tecklenburg have ned in heir invesigain ha generally ie pins are sufficien fr a cmplee rm(luhmann and Tecklenburg 2002). Fr bjec measuremens, he crrespnding image-pin bservains have be made in he same ay as hen using cenral prjecive frame images. Hever, in cnras frame images, he epiplar line, here crrespnden pins shuld ceis, is n a sraigh line bu a sinusidal curve. This is due he presenain f he panramic image prjeced n he surface f a cylinder. The benefi f panramic images in inside scene measuremens by imaging uards frm inside is heir cverage f full. This means ha an bjec pin can 25

26 pssibly be seen in all panramic images f he prjec. Hever, he image scale can vary largely frm image image. This is hy, in many research prjecs, an image managemen and brsing sysem has had be creaed in rder find all pssible images here he bjec pin can be seen (Luhmann and Tecklenburg 2004; Chapman e al. 2004). In very cmple envirnmens, he image managemen sysem is a sysem cmpnen ha is clearly crucial in geing he ask accmplished (Chapman e al. 2004). The accuracy achieved in bjec recnsrucin by using panramic imaging has generally been quie adequae fr he requiremens f an bjec mdel. By using fur panramas cnsruced frm a sequence f cenral prjecive images fr bjec measuremens, he r mean square errr (RSE) has been f a size f, hen bjec dimensins have been "! #!%$ (Luhmann and Tecklenburg 2004) and &! &! ' (Schneider and aas 2005). A similar accuracy range f (*),+.-0/ has been repred ih he use f special panramic cameras (Schneider and aas 2005). Even hugh, ih panramic imaging, alms he hle scene can be cvered, depending n he srucure f bjec scene, here can sill be ccluded areas here n image infrmain is bained. In rder recver frm his shrcming, an image adjusmen sysem has been develped, here cenral prjecive frame image bservains are adjused simulaneusly ih panramic image bservains (Schneider and aas 2005). Panramic imaging has als arused ineres in he cmpuer visin cmmuniy. Panramic imaging is smeimes called mnidirecinal imaging in cmpuer visin lieraure. Research rk has been underaken in he develpmen f cnsrucin mehds f panramic images as ell as in 3D bjec recnsrucin. The main bjecive f research in hese research prjecs has cncenraed n varius ays recver he camera mvemen during imaging. The ulimae gal f he research has been reslve he camera rain in an aumaic ay (Kang and Szeliski 1997; Jiang e al. 2005). In bjec recnsrucin, a 3D mesh has been generaed accrding all pssible piel lcains here he crrespnden image pin has been fund n her panramic images. Als, here, aumain f he prcess has played he main rle in invesigains. Unlike ih research rk in he phgrammeric cmmuniy, he gemery f imaging regarding bjec recnsrucin has n been an impran par f he research. Sere panramic imaging: Sme variain frm he sric panramic imaging rule has been suggesed here imaging des n share a cmmn prjecin cenre. The aim f research in ha case has been creae a sere sysem ih a panramic vieing capabiliy (Peleg and Ben-Ezra 1999). The cnsrucin f sere panramas is achieved frm image sequences. Each image sequence is bained by raing he camera arund a saic ais alng a circular pah. The lking direcin f he camera is angenial he circular pah during imaging. The secnd image sequence is creaed in he same ay, ecep fr he ppsie angenial vieing direcin f he camera. The aim is creae a panrama im- 26

27 age ha cnsiss f image srips frm bh image sequences. Fr he purpse f sere vieing, he image is prjeced n a planar surface as a sere pair ih a chsen vieing direcin. Anher apprach epliing similar imaging gemery is called mnivergen sere (Seiz e al. 2002). The idea f imaging image sequences is he same as ha fund in he rk f Peleg and Ben-Ezra. Als in his rk, he aim has been reduce redundan image infrmain and nly sre hse image rays angenial he circular pah. Alng ih basic image cnsrucin, he spherical mnivergen image mdel, here clleced image rays are sred accrding heir hriznal and verical angle direcin values, is presened. By sring image ray infrmain s ha image rays ih he same hriznal angle are sred in he same image clumn, and image rays ih equal verical angles are sred n same r, he epiplar line ill g alng he scan-line and he crrespndence f image pins can be fund n he same r. This is a benefi hen using sandard sere maching algrihms fr bjec recnsrucin. In addiin, an apprach acquiring images is presened fr creaing mnivergen sere: cenre-srip sere, based n a single image srip cllecin (equivalen he rk f Peleg and Ben-Ezra), and dual-srip sere, epliing image acquisiin frm symmerical ff-cenre sli images. In bh panrama sere and mnivergen sere, ideal imaging cndiins are assumed in research and n discrepancy f epeced imaging gemery is cnsidered. Als, he qualiy f he bained bjec pin crdinaes is n eamined. ne research prjec, r mre like a missin, be menined is he ars Eplrain Rver (ER) missin ha has cnnecins he repred research area (aki e al. 2003; Bell e al. 2003). In early 2004 he ER missin landed a pair f rvers n he surface f plane ars. The missin as launched by he Nainal Aernauics and Space Adminisrain Agency (NASA). Algeher 12 cameras are placed n each rver eplre he arian surface. Frm he 12 cameras sere camera pairs are he arge f ineres cncerning his research; namely he Navcam and Pancam sere cameras. The sere cameras are muned n a mas abve he surface in a pan- and il-uni f he rver. The Navcam cameras are ihin a disance f apar and he Pancam cameras have separain in camera bar. The imaging sysem resembles he imaging sysem used in he earlier ars missins f ars Pahfinder and Viking. The primary bjecives f he Navcam camera sysem is acquire an end f he day panrama f he lcal errain afer a rver raverse. This errain mdel is used in planning he rue f he rver in he ne sage. The sere image sysem can prduce a sere vie cvering he hle 9 : 2 ; in rder bain a errain mdel f he scene. A ypical Navcam panrama cnsiss f a sequence f 12 sere pairs, spaced apar by 9 : ;. In rder acquire a errain mdel, images are reprjeced epiplar images and an image crrelain echnique is used prduce dispariy maps f he scene. The dispariy maps bained frm separae sere images are hen cmbined using angular infrmain gained frm he angular 27

28 Figure 1. Imaging gemery f panramic imaging creaed in ars Eplrain Rver (ER), redran frm (aki e al. 2003). acuars f he pan-il-uni. Hever, if image msaics f he scene are be creaed, ie pin measuremens beeen images need als be made. The rver is equipped ih a Inerial easuremen Uni (IU), and in cmbinain ih infrmain bained frm rain and rienain f he rver heels, his akes care f psiining f he rver in respec previus lcains. When needed, he psiin infrmain is updaed based n image measuremens f bjecs, seen frm bh imaging sies. The imaging sysem used fr 3D bjec measuremens, here measuremens are based n image bservains made frm image sequences f camera pair aached a raing bar, is depiced in Figure (1), hich ehibis he imaging gemery. The imaging gemery is cmparable imaging accmplished in mnidirecinal sere and panrama sere. The imaging sysem f ars Eplrain Rver cann be cnsidered belng under he ile f indr imaging, bu surely demnsraes he imaging cndiins f inside scene envirnmen, defined earlier in his secin. T summarise, in his Chaper a revie f phgrammeric clse-range measuring mehds aiming 3D bjec recnsrucin have been made. Especially, mre clse lk n mehds applicable inside scene envirnmen is aken. The reader is encuraged bear in mind he imaging gemery presened in Figure (1) and described in he las par f he secin dealing ih sere panramic imaging, since i resembles he imaging prcedure used ih circular imaging blcks. A cmparisn f he develped mehd ih mehds presened here in his secin is be made laer in Chaper 3. 28

29 2 INTRDUCTIN 2.1 Bundle blck esimain The recnsrucin f an bjec purely n he basis f image infrmain needs be perfrmed ih prper imaging gemery in rder bain a cnsisen and undisred bjec mdel. Smeimes, especially fr visualizain, i is enugh ge a mdel ha resembles a real bjec. In cases here many sub-bjec mdels have be merged in a larger and mre cmplicaed mdel, here is a risk f majr prblems if he cnsisency f mdel gemery has n been deal ih prperly. Preferably, he hle mdel shuld be hmgeneus in erms f accuracy f measuremens. This, hever, is quie difficul achieve. In rder ge an apprpriae sluin, imaging has be planned carefully. In phgrammeric lieraure, special aenin has been paid he prblem f achieving a gd imaging nerk (Fraser 1982, 1984; asn 1995; Fraser 1996). Fr sme applicains, requiremens fr accuracy can be very sric. There have been reprs f image-based bjec measuremens achieving an accuracy f ne par in ne millin ih respec he size f he bjec (Fraser 1992). In such cases, many planning ierains may be needed achieve an pimized nerk. The accuracy f a final 3D mdel depends upn: 1. he accuracy f image bservains, 2. he gemery f imaging nerk, 3. he number f bservains, and 4. he crrecness f camera mdel. The effec f image bservain accuracy is apparen n bjec-parameer deerminain. H he gemery f inersecing image rays has an effec n bjec accuracy can als be clearly shn ih errr prpagain. An increase in he number f bservains abve he number ha is necessary slve he bjec parameers des n nly imprve he accuracy f bjec measuremens, bu als gives us a l ih hich esimae he precisin f ur measuremens ihu any eerir reference. By applying a crrec camera mdel, he ccurrence f sysemaic errrs n bjec mdel crdinaes can be prevened. The recnsrucin f an bjec mdel can be accmplished frm muliple sere image pairs. Frm each sere image pair, bjec pins can be measured and a sub-mdel represening a parial bjec creaed. Usually he gal is recnsruc he bjec as a hle and his iniiaes a need have all sub-mdels ransfrmed in he same crdinae sysem. A rigid bdy, cnfrmal ransfrmain 29

30 is fen used fr ransferring he sub-mdels in a cmmn crdinae sysem. Unfrunaely, his kind f apprach generaes quie a number f sub-mdels and herefre sme kind f image handling sysem is required manage he hle measuring prjec. Als, mre effr has be pu in he search fr crrespnding bjec feaures fr he purpse f crdinae ransfrmain. These numerus sub-blcks are difficul handle in he same prjec and heir rienain can be quie arbirary in bjec space. In sere measuremens, an bjec pin is measured n bh images. Smeimes, he relaive psiin and rienain f he image pair have been reslved befrehand and he deerminain f mdel crdinaes is cmpue he inersecin pin f image rays in 3D space. In an ideal sere pair case, crrespnding image rays ill inersec in ne unambiguus 3D pin lcain. The disadvanage f he sere measuremen apprach is ha i des n prvide real qualiy infrmain f bjec pin deerminain. Sme feed-back f he qualiy f he relaive rienain f an image pair is available in he frm f < -parala, hugh. Als, sere imaging is n gemerically he ms accurae imaging cnfigurain, even hugh i is idely used. In clse-range phgrammeric measuremens, he mulisain cnvergen cnfigurain has been fund be he pimal sluin fr imaging gemery (Fraser 1984, 1996). As previusly menined in his secin, he number f bservains has an effec n bjec mdel accuracy. In an verdeermined case, here are mre image bservains f each bjec pin han he minimum requiremen. Due randm nise n image bservains r pssible sysemaic errr, he crrespnding image rays frm muliple images d n necessary inersec a he same pin. S he prblem is slve he 3D pin lcain, here he bservains have he bes fi. In he field f gedesy and phgrammery, he sluin he prblem has been slved ih he Leas Squares (LSQ) Esimain mdel (Slama 1980). bjec pin crdinaes can be slved by minimizing he eighed squared sum f bservain errrs in he image space. The image rienain values are als cmmnly slved in he same adjusmen. The prblem is hen find he bes fi f he bundle f image rays ih respec each her and pssibly ih respec he pre-knn bjec pins r feaures. This esimain mdel is called he bundle adjusmen mehd (Brn 1976). Adjusmen mehds have been idely used, firs in aerial phgrammery applicains (Brn 1976) and laer in clserange phgrammeric measuremens (Weser-Ebbinghaus 1978; Fraser 1984). Recenly, he cmpuer visin cmmuniy has als adped he bundle mehd in heir cmpuain mdels (Pllefeys e al. 2004). By using he bundle esimain mehd in an verdeermined case, i is pssible assess he precisin and reliabiliy f measuremens, plus he qualiy f measuremens, ihu any eerir reference daa. 30

31 c The esimain mdel can be represened in he frm f leas squares in he flling ay: =?>A@CBEDGF (1) The funcinal mdel can be derived as an bservain equain, Equain (1), here he discrepancy beeen he linear funcin f parameer values B and bservains F is presened as a residual vecr =. The is a cefficien mari f unknn parameer vecr f his linear funcin. This bservain equain is an eplici funcin here bservains can be epressed as a linear cmbinain f parameers. This, hever, cann be cnsruced in every case; hen i cann be, bservains and parameers have be adjused in he same adjusmen by using a general adjusmen mdel (ikhail 1976). The bjecive f his LSQ esimain is minimize he sum f squares f eighed residuals. If he firs derivae f Equain (2) ih respec he unknn parameer vecr B is cmpued and se zer, e end up ih Equain (3). This ill guaranee he minimum crieria and he sluin fr unknns B can be cmpued. In he lieraure, i is cmmn dene he symmeric in Equain (3) as K and he as (ikhail 1976). The symmeric squared mari K is fen designaed he nrmal mari. = HJI*=N> P@QBRDGFPSTHJIUV@QBDGFPS (2) BW> P@ H H I^F > K Y\[ (3),here I_> `ba ced Y\[ fgf >h Y\[ fif (4) The eigh mari I in Equain (4) is an inverse f he cfacr mari. The cfacr mari h fif is a variance cvariance mari f image bservains d fif scaled by he reference variance f he adjusmen. This infrmain is hardly ever available and has be creaed n he basis f prir eperience. Since he image bservains are assumed be independen, he mari ill be reduced a diagnal mari, hse diagnal elemens are in a frm f jlkm> ` a ` a kk (5) 31

32 here nb p is he reference variance f adjusmen and nq rr is he variance f he equivalen bservain. fen he reference variance n p is seleced as he variance f image bservains, in hich case he eigh f a single image bservain is uniy, r uni eigh. If nly image bservains are included in he adjusmen, he eigh mari s ill hen be he ideniy mari. Hever, her ypes f bservain can be included in he esimain as ell. They can have differen variance prperies as image bservains and als image bservains can have differen variance values, depending n heher he bservain is frm a argeed pin r a naural bjec pin. Then he ideniy mari ill n be adequae as a eigh mari fr he adjusmen. The equivalence f he mari presenain f he bundle mdel can be fund ih he fur facrs presened previusly in his secin. 1. The eigh mari s includes he bservain accuracy infrmain. 2. The cefficien mari f unknn parameers cnains he effec f imaging gemery. The disribuin f bjec pins and feaures, as ell as he relaive psiin and rienain f image bundles, have a srng effec n he srucure f he mari. Tha is hy he mari is fen called he design mari. 3. The redundancy uv f he adjusmen can be calculaed frm he number f bservains and number f rs in he mari (in he assumpin f eplici equains) cmpared he number f unknn parameers y, i.e., size f vecr z. S redundancy is uv {}~y. 4. The precisin f he camera mdel can be evaluaed frm camera calibrain measuremens r, if he inerir rienain parameers f he camera are included as unknns, he precisin values can be derived frm he parameer variance-cvariance mari Z, as discussed bel. The pssibiliy f assessing he qualiy f measuremens is ne majr advanage f using LSQ esimain. bjec space uncerainy can be evaluaed by deriving he bservain uncerainy ƒiƒ by use f errr prpagain in he variancecvariance f unknn parameers Z. The uncerainy f he bjec feaure parameers is als dependen upn he gemery f he imaging nerk. Accrdingly, he esimae f variance-cvariances f parameers is a funcin f he design mari and eigh mari s in he frm f he inverse f he nrmal mari scaled by he pserir esimae f he reference variance f uni eigh p. I is given as n ˆ{ n p Š \ (6) The variance-cvariance mari f unknn parameers N Œ, Equain (6), can be used in he evaluain f adjusmen resuls. The race Ž]u N Œ has a direc 32

33 relain R ean Square (RS) values and is herefre used in he assessmen f resuls, as ell as in he simulain f he imaging nerk design (Fraser 1982). The qualiy f bservains can be revieed by eamining he variancecvariance mari f residuals ]. ehds deec blunders, r uliers, are cmmnly based n residual variances and cvariances. This cmpac presenain f he bundle mehd is very general and is designed mivae he adpin f he mehd in image-based measuring cnfigurains. A mre deailed descripin can be fund in he lieraure (ikhail 1976; ikhail e al. 2001). 2.2 Cncep f Phgrammeric Nerk Design Prblem In phgrammeric measuremens, he accuracy f 3D measuremens depends highly n he camera cnfigurain. The bes resul can be achieved in he case here images are epsed a he cnvergence angle f. This is n a sric requiremen, bu having an inciden angle f inersecing image rays f.šg frm r muliple images (Fraser 1992), ill alms resul in isrpic precisin in all hree crdinae direcins. Unfrunaely, his is ccasinally difficul arrange in pracice. In cases here he pimum gemery cann be cnsruced, her crieria fr he esimain mdel are used achieve a sable sluin. In aerial phgraphy, he verlap f images shuld be adequae guaranee he accuracy f he mdel. The achievable accuracy f a phgrammeric measuring sysem largely depends n he lcalizain and disribuin f he cnrl pins r cnrl feaures be used. The imaging device, is accuracy, and gemerical sabiliy have be knn hen planning he mensurain sysem. These all have an effec n he final measuremen precisin. Changing ne cmpnen migh have a grea impac n sme her cmpnen and in ha ay affecs he accuracy f he hle measuremen sysem. The sandard prcedure is adjus he measuremen cndiins ieraively mee he accuracy requiremens. The design f he measuremen sysem can be divided in ZD, FD, SD and TD levels f planning, accrding Grafarend (Grafarend 1974) and Fraser (Fraser 1984). zer-rder design (ZD): he daum prblem firs-rder design (FD): he cnfigurain prblem secnd-rder design (SD): he eigh prblem hird-rder design (TD): he densificain prblem Accrding Fraser (Fraser 1984, 1996), in clse-range phgrammeric nerks, his classificain is n really applicable and he ZD- and SD-level design are grealy simplified cmpared sages in gedeic nerk design, here 33

34 he design classificain as firs presened (Grafarend 1974). The ZD-level planning is abu he fiing f seven apprpriae bjec space parameers in rder remve he nerks daum defec. The variance-cvariance mari f bjec space feaures œž Ÿ Ÿ is highly dependen n he chice f his minimum cnrl. Even hugh he shape f he nerk ill remain unchanged, he changes can be seen in he numerical values f œž Ÿ Ÿ as a resul f his chice. Hever, he ZDlevel planning can be ignred hen nly he shape f he bjec is needed. I is als pssible slve he daum defec prblem by applying numerical mehds. The sluin is fund by cnsraining he minimum f he linear equain sysem. ne apprach is based n inner cnsrains in adjusmen. Alernaively, same sluin can be achieved by using he singular value decmpsiin algrihm. The sluin based n inner cnsrains has prved be mre feasible in pracice (Fraser 1984; ikhail e al. 2001), hile he laer apprach has had a mainly hereical value (Inkil and Laih 1989). The ms demanding phase in clse-range phgrammeric nerk design is prbably planning he camera sain cnfigurain: he FD level. As an indicar f pin riangulain precisin in a cnvergen, muli-sain phgrammeric nerk, Fraser has presened a frmula (Fraser 1984): b ª «ª \ (7) Frm Equain (7) e can see ha he mean sandard deviain f bjec pin crdinaes b is dependen upn he image crdinae sandard errr ; scale number «, hich is a relain f mean bjec disance and camera cnsan; and a erm epressing he srengh f he nerk gemery. ª The effec f he number f images ª n bjec pin accuracy is inversely prprinal he square r f, assuming ha he addiinal images d n essenially imprve he gemery f he nerk. If he bservain accuracy is epressed in a frm f sandard errr f inciden angles ±, he scale number reduces mean bjec disance. Values f are epeced range frm ³g ³µ. The value migh be epeced fr a eakly cnvergen nerk. In phgrammeric prjecs redundan measuremens are msly used. The redundan measuremens cmbined ih he bundle f rays mehd prvide he capabiliy esimae he precisin and reliabiliy f he measuremen perain. This definiely gives us feed-back infrmain frm he nerk design prblem. In he case f bundle esimain and cnvergen image gemery, here he bjec Ÿ Ÿ is epeced be diagnally dminan, he value variance-cvariance mari œn 34

35 f can be hen epressed ih a sub-diagnal mari relaed he 3D pin parameer cefficiens in he design mari. I can be ned ha planning can be a ime-cnsuming prcess and can require a gd knledge f h phgrammeric nerk design can be achieved successfully. Hever, research rk has been carried u in rder simplify he prcess by implemening an eper sysem fr nerk design (asn 1995), here he cnvergen, muli-sain phgrammeric nerk has been assumed. This basically means ha e are able pse he camera sains arund he bjec, and images can be aken ih cnvergen rienain. The placemen f he camera sains may fen be cnsrained by varius facrs such as: image scale, deph f field, inciden angle and rkplace envirnmen (asn 1995). The use f geneic algrihms fr aumaing he phgrammeric nerk design prcess has als been prpsed (lague 2002). In sme applicains here he precisin requesed is n s high and he imaging cndiins are cmplicaed, sme her kind f apprach fr imaging design can be chsen. Smeimes, cnsrucing a cnvergen image bundle is n even pssible, especially in archaelgical and archiecural applicains. This can als be he case in indusrial applicains (Fraser and allisn 1992; Fraser 1996), here he prblem f recnsrucing he mdel f an inside scene fen ccurs. The reasn hy he image blck is n creaed frm inside scene is ha, especially in cncave crner areas, he imaging gemery is pr. The resul f pr gemery cmbined ih nisy bservains is a defrmed bjec mdel. In ms cases, he prblem has been slved by aking sere images r creaing sub-nerks and recnsrucing he sub-mdels frm hem. The hle mdel is hen recreaed by cmbining hese mdels. This is dne by slving he similariy r rigid cnfrmal ransfrmain beeen sub-mdels ih he help f cmmn bjec feaures. An alernaive ay cmbine mdels is use ICP r similar algrihms, here he disance beeen pin ses is minimized. This apprach, hever, assumes ha a subsanial number f pins are included in ransfrmain esimain. 2.3 Cnsrained Imaging In rder sabilize he adjusmen prcess f he image blck ih pr imaging cndiins, sme imprvemens have been suggesed. The imprvemens have mainly cncerned he inrducin f he prir knledge f he imaging r bjec prperies in an adjusmen. Papers dealing ih his issue have ms cmmnly arrived a a sluin ha applies cnsrains in bjecs space (Yucai and Haralick 1999). Cnsrains have been esablished beeen bjec pins revealing knn relainships in bjec space. The epliain f bjec infrmain has been aken a lile furher by phgrammeric frmulains, including parameric presenain f 3D linear feaures such as lines, circles, and ellipses. her cnic secins, as ell as b-splines in bundle adjusmen, have als been presened (ulaa 1989; ulaa and ikhail 1988). Presenain binds geher 35

36 he bjec feaure parameers, image eerir rienain parameers and image bservains. In his apprach, a srng cnnecin is creaed beeen bjec feaures and image ray bundles. Als, iner-feaure relains, such as parallelism, perpendiculariy, cplanariy, ec. have been included in he esimain mdel. The sabilizing effec f using linear feaures in place f pins is based n beer lcalizain f feaures n images, mre rigrus sluins f ransfrmain beeen image space and bjec space, as ell as imprved redundancy in adjusmen (cglne 1995; Heikkinen 1994). Als, in he cmpuain f eerir rienain based n a linear ransfrmain mdel, he same apprach has been flled (Ji e al. 2000). Less effr has been pu in making use f knn infrmain abu imaging cndiins. In sme cases, pre-cmpued relaive rienain f sere image pairs has been included in he adjusmen via eighed bservains f rienain parameers. Usually, relaive rienain parameer values are cnsidered be fied, hile nly measured bjec pins are aken in he cmpuain f he cmmn adjusmen. Bu here are eamples here a bundle adjusmen has been eplied in an eensive ay. In he Nainal Research Cuncil f Canada (NRC), an aunmus mapping vehicle has been develped here, geher ih image sensrs, a range-sensr, as ell as navigain-sensr daa are included in cmmn bundle adjusmen. The relaive psiin and rienain f image sensrs ih respec he vehicle crdinae sysem is pre-calibraed and his infrmain is hen used as cnsrains in adjusmen (El-Hakim e al. 1997, 1998). In research cnduced by King, auiliary infrmain f he imaging cndiin as included in he adjusmen (King 1994). In his research, he bundle adjusmen as cmpued ih cnsrained sere pairs. This ype f case resembles he adjusmen f independen mdels. Accrding he hery f independen mdels (Schidefsky and Ackermann 1978; Slama 1980), he hree dimensinal similariy ransfrmain is be esimaed amng mdel and cnrl pin crdinaes. In his cmpuain mdel, sere mdels are he cmpuing unis, hile seven-parameer ransfrmain is slved assuming nly randm nise in mdel crdinaes. In Kings invesigain, image crdinaes ere he primary bservains, n mdel crdinaes, and he priri infrmain f he sere case as aken in accun as addiinal bservains. In his mdel, camera relaive rains and he shif vecr beeen cameras ere inrduced in adjusmen as addiinal bservain equains. By assigning differen eighs fr camera parameer bservains, he resriciveness f he sere pair gemery case culd be uned. This is, in fac, he same as ih eended independen mdels, here an addiinal parameer se is used in rder cmpensae he effec f sysemaic errrs in sere mdel crdinaes. Hever, he se f unknn parameers in Kings mehd as differen. Anher sluin fr he adjusmen f he blck f sere pairs ha King presened is use cnsrains beeen camera parameers. The cnsrains can 36

37 be assigned prjecin cenre crdinaes as a cnsan shif (¹,¹ y,¹ z) inside a sere pair and cnsan rain angle difference amng rain angles (º¼» ½¾ºÀ ÂÁ¹žº,¹ÄÃ,¹ÄÅ ). If LSQ-ype esimain is be applied, he cnsrains can be given as a cnsrain equain. Abslue cnsrain equains are mre sric han eighed bservains and using hem leads up an addiinal degree f freedm (ikhail 1976; ikhail e al. 2001). This rks ell if base vecrs f all sere pairs are cllinear r parallel ih each her. In rder all he sere pairs have varying rienain, differen kinds f cnsrains have be defined. ÆÀÇÈ ÁÉ Ê˽GÉQÌ%ÁÍ (8) É ÊÎÁÐ ÓÒ» Ê ½ÔÒ ÊÖÕØ ÀÙ PÚ» Ê ½GÚ ÊÕ] Ù ÜÛ» Ê ½Û ÊÖÕ] (9) ÉQÌ%ÁÞÝ ß ÊãâJ à É Ê (10) If a cnsrain f a base vecr É lengh is applied insead f a shif difference f prjecin cenre crdinaes, he varying rienain f sere pairs can be alled. King used his cnsrain in his rk, bu insead f using a cnsan value fr base lengh, he alled his value als vary. By requiring he lengh f he individual base vecr É Ê difference mean lengh ÉˆÌ f base vecr be minimized, he cnsrained culd be se as depiced in Equain (8-10). inimizain f rienain-angle differences mean difference values culd n be used. This as because he rain angles f he cameras ere epressed ih respec he bjec space crdinae sysem. Fr he bundle f sere pairs ih fied relaive rains, he cnvergence angles mus be invarian fr all sere pairs, irrespecive f he rienains f he camera aes ih respec he bjec space crdinae sysem. Assuming he rhnrmal rain mari å, he cnvergen angles culd be derived by cmpuing he d prduc f crdinae aes and applying he inverse csine f he prduc, as depiced in he flling equain. ç È Á}èêé ëêì Ví» T È í T È Ù í» È í È Ù í» Pî È í Pî È Õ ï È ÁAèðé ë ì Pí» È í È Ù í» È T í È T Ù í» î È í î È Õ ñ È Á}èðé ëòì Pí îã» È í îã È Ù í î» È í î È Ù í îtî» È í îtî È Õ The nain í å in he Equain (11) marks he rain mari elemen frm he righ í Ê and lef í Ê» image f he ó ôõ sere mdel, respecively. 37 (11)

38 ö ùøvúxûaü ýòþ Ôü ý ú ö úxû ü ]þš^ü ú ö ú û ü þ ^ü ú (12) In addiin, King included he mean values f cnvergence angles in cnsrain equains flling he same principle as ha f prjecin cenre crdinaes, see Equain (12). As mean values are used in he cnsrain equain insead f cnsan values, he means need be updaed afer every ierain in he leas squares mdel esimain fr his nnlinear case. The mehds described are eamples f echniques ha can be eplied hen imaging gemery cann be pimized. The reasn fr n having pimal gemery can be due he srucure f he bjec iself; he envirnmen, hich resrics he pssible imaging sains, r he cmpuainal speed required. The ramificains f n aking care f sabiliy in he recnsrucin prcess can lead a disred bjec mdel. 38

39 3 CIRCULAR IAGING BLCK In ms accurae bjec recnsrucin prjecs, he nerk design is based n cnvergen imaging and argeed bjec pins. There are many cases hen his is n pssible. The imaging mehd presened here is designed fr special cndiins here he radiinal apprach in he nerk design prblem (Fraser 1984; asn 1995) mees is limiains, hen, fr eample, visibiliy is smeh cmprmised, as ih very cmple bjec srucures (Chapman e al. 2004; Leru e al. 2002). The nly sluin his prblem is fr he imaging be accmplished inside he bjec space, n arund he bjec, i.e., inside scene imaging. The develped mehd is designed especially fr inside scene imaging ype f mdelling cases. The applicains fr his imaging sysem are cngruen ih applicains f mehds presened in Secin This nvel mehd is based n image measuremens made frm an image sequence, bu differs frm bjec recnsrucin mehds based n image sequences (Pllefeys e al. 2004, 2003, 2000) presened in Secin 1.3, in he ay imaging is accmplished. In he image sequence research, a hand-held camera as mved arbirarily, hereas in his apprach he camera is suppsed be mved in a mre rigrus ay. Here he sequence is assumed cver he hle scene f. Nneheless, here, he quesin is n abu panramic imaging, here a single image f panramic vie is creaed. In his mehd, he measuremens are based n image ray bundles f cenral prjecive images f a frame camera. The images in he sequence d n share a cmmn prjecive cenre like images in he creain prcess f a panramic image frm muliple-perspecive images (Weser-Ebbinghaus 1982; Pöninen 1999). The capabiliy f measuring 3D bjecs is merely based n image bservains made frm images having a displacemen f prjecin cenres n subsequen images. All bservains frm he hle image sequence are epeced be handled in a cmmn bundle adjusmen. In he cmpuain f bjec-pin r feaure parameers, he cnsrained relaive psiin and rienain f images in sequence ill als be reslved. 3.1 Inrducin he Circular Imaging Blck Cncep The nvel mehd f Circular Imaging Blck uilizes he cnsrains beeen camera sains. The idea is minimize he rklad and need f phgrammeric eperise in phgrammeric nerk design. The number f parameers be used in specifying he imaging gemery is reduced jus a fe. This als gives rise he pssibiliy ha nn-phgrammeriss migh design he phgrammeric nerk ihu any knledge f accuracy aspecs f he phgrammeric measuremens and he hery f errr prpagain in he LSQ esi- 39

40 main mdel. In rder achieve his gal, he imaging has be accmplished in a specific ay. Earlier in radiinal erresrial phgrammery, images ere fen cnsrained be aken s ha he se f camera sains uld cmply ih nrmal sere phgraphy. This as parly because nly analgue plers ere used fr crdinae measuremens in hse days. The resricin f hse devices frced he cnvergence f he images be ihin fied limis. Als, large image scale differences culd n be accmmdaed. Wih analyical mehds, he use f cnsrains beeen camera sains in bundle adjusmen became mre fleible. In pracice, nly he measured disances beeen camera sains ere used as addiinal bservains r cnsrains. Hever, cnsrains in he frm f equal heigh f camera sains have als been applied, as ell as lining up he se f camera sains ih equal rienain. In he laer case, he arranged camera cnfigurain f imaging culd be inrduced in cmpuain by fiing ne r crdinaes f he prjecin cenres in adjusmen. re fen, hese bjec space bservains ere nly used in rder ge beer iniial values fr camera pse esimaes Definiin f Circular Imaging Blck In his research, n nly cnsrains are applied beeen camera sains, bu he chsen parameers epress camera pse rely n he assumpin ha images are aken in a pre-defined ay. The imaging is especially designed be used in applicains here he measuremen is mean be made inside he bjecspace. This means ha bjec feaures hse psiin, size, and rienain, are be measured are disribued arund sme pre-defined area, fr eample, he rm f a building. Insead f aking muliple sere pairs and cmbining hese sub-mdels as a final mdel, in his apprach nly a fe imaging sains are needed. A each imaging sain, ens f images are be aken, depending upn he disance f bjec feaures frm he camera and he field-f-vie f he used camera. In rder bain hree-dimensinal measuremens frm a single imaging sain, here mus be an ffse beeen he recrded images. This paralla can be achieved by mving he camera alng a pre-defined rajecry beeen successive epsures. In his research, he rajecry is assumed be a circle n an arbirary plane. In pracice, his can be arranged by fiing he camera a he end f a suppring bar. The her end f he bar is hen psiined prvide a navel pin fr revluin. The navel pin can be aached, fr eample, a ripd, hich has been he case in real-rld eperimens, as ill be described in Secin 5.3. The assumpins fr he imaging are as flls: he bar is raed nly n a plane r n a cnic surface here he peak pin f he cne ceis ih he navel pin f revluin and he ais f revluin ges alng ih ais f he cne, ye aking care ha nly plane rain is applied he camera a he end f he bar. 40

41 ne pin in a bar ill be in a cnsan psiin, i.e., he navel pin. he rienain f he camera ih respec he bar is cnsan hrughu he imaging sequence. subsequen images mus verlap s ha ie pins can be measured beeen images. sequence f images ill cver s ha ie pins beeen he firs and he las image in sequence can be measured. The image blck fulfilling hese assumpins is called a Circular Imaging Blck, Figure (2). The minimum number f images required in sequence is dependen upn camera bjec disance and field f vie. Als, he camera disance frm he navel pin and he rienain in respec he bar have an effec n his minimum number. If using cninuus imaging, fr eample, vide imaging and making measuremens n each frame f he vide sequence, enugh verlap beeen subsequen images can be guaraneed. This undubedly increases he redundancy f bservains. The era rk caused by adding mre images n he image blck is eviden, bu, by applying aunmus image measuring algrihms, he rklad can be reduced subsanially. The camera des n necessarily have be a vide r digial sill camera; analgue film camera images can als be aken. Hever, he amun f image mensurain needed favrs he use f digial images. r α 1 α 2 Figure 2. Iniial imaging gemery f a circular imaging blck accrding definiin. This ype f imaging design des n guaranee ideal measuring gemery, neiher des i avid all he pssible cclusins. Usually, muliple image blcks, imaging sains, f his kind mus be aken saisfy he cndiin f recnsrucing he hle bjec. Unlike he case ih panramic images, 3D measuremens can be made frm a single image sequence, displacemen beeen prjecin cenres d eis. This mehd prvides measuremens ha are sufficien fr 41

42 bjec recnsrucin, r hich are a gd base be used in a design f mre precise measuremens in cmple mdelling cndiins. Benefi f using circular imaging blcks include fas daa acquisiin and, als, in he case f vide imaging, he recrding f increased amun f daa. The increased amun f daa can be used fr subsiuing he l qualiy f images sme een. By using l-cs ape recrding devices, he prvisin f sufficien srage space is rarely a prblem. If a vide camera is used, all images in a sequence can be used recnsruc he bjec insead f selecing he images ih he bes gemery. The develpmen f digial sill cameras and digial srage devices has pened ne perspecives n he use f digial imaging. The capaciy f srage devices fr digial images is nadays sufficien, hile beer spaial resluin gives an advanage in using digial cameras ver vide devices, here he resluin f images is resriced by vide sandards (PAL, NTSC ec) Image Blck Cnsrucin I is impran ha he circular imaging sequence is clsed, since he lcal crdinae sysem ill be creaed based n he bservains frm he imaging sequence. Tha means ha n cnrl pin nerk, r measuremens fr camera pse, are required. The enclsure f imaging rks as in a gedeic leveling chain; he heigh difference f pins are measured ice, preferably n differen rues, clsing a he sar pin. Insead f adjusing measuremens direcly, in he case f image bservains, he adjusmen is based n bservain equains f indirec bservains. The crdinae sysem is defined by fiing ne pin and direcins. The navel pin f revluin funcins as he rigin and crdinaes aes f he rhgnal crdinae sysem are he nrmal vecr f plane rain and is rhnrmal vecr cnaining he cenre f prjecin f he firs camera pse in sequence. Hever, sme scale measuremens are required creae a crdinae sysem in he meric rld. The resriced imaging arrangemen frces he prjecin cenres f each camera psiin lie n he same plane and ihin he same disance frm he navel pin. Frm he pin f vie f 3D measuremens, a circular imaging blck is perhaps n he perfec imaging cnfigurain fr he ask. In imaging design, he camera is lking uard frm he navel pin, as in Figure (2). The cnsecuive camera psiins ill hen have diverging pical aes. The impac f bservains f images ih divergen image gemery is a decrease f reliabiliy f he camera rienain parameer esimaes. The sluin is urn he camera he angenial direcin f he circle pah, Figure (3). Sill, he pical aes f he subsequen images ill be divergen. By cnsrucing such circular imaging blcks n he same ripd psiin ih a difference in camera rienain, beer gemery can be achieved. Beeen image sequences, he camera is urned a he end f he bar, as shn in Figure (4). These 42

43 r α i Figure 3. Circular imaging blck gemery ih camera urned angenial direcin image blcks are c-cenred and ceis in he same crdinae sysem. The bjec pin can be seen in images frm bh image blcks, and he camera pical ais n he firs blck images is cnvergen ih camera psiins n he secnd blck. N he imaging gemery ih respec he previus case has subsanially imprved. Images f his pair f image blcks ill be, a ms, imes as far frm each her as he lengh f he bar. This is he imaging gemery used in his rk and implemened fr he eperimens. Imaging gemery resembles he implemenain depiced is Secin and cmparisn ih mehds presened in Secin ill be made laer in his Chaper. r Blck II %% && % &' ( Z X!!! "" " # $ Blck I Blck II Blck I * )) ** α i Figure 4. T c-cenric circular imaging blcks. The imaging gemery used in his hesis. Frm he phgrammeric nerk design pin f vie, he ZD-level f design is very much simplified ih circular imaging blcks. The daum prblem is slved by creaing an n crdinae sysem, hile he scale fr he nerk is based n simple disance measuremen beeen r mre 3D pins. The FD-level design cnsiss f he selecin f ie pins frm images and he lengh - f he bar used. The seleced ie pins can be image pins f naural feaures r argeed pins. Using argeed pins means era rk, bu imprvemen 43

44 f measuring accuracy als imprves he precisin f blck esimaes. In ms cases, especially hen he image plane is chsen be perpendicular he plane f rain, i is desirable selec pins frm he side areas f he images. This is because gemerically hey have ms reslving per in deermining he rienain angles f images. The reasn hy i is favrable have he image plane near perpendicular he plane f rain, and ne f he image plane aes parallel he rain plane, is ha he crrespnding pins beeen subsequen images ill hen ms likely be fund in he verlapping area. In he ppsie case, he verlapping area ill be decreased because f gaps beeen subsequen images. The same phenmenn is knn in aerial phgraphy as drif, hen he airplane is urned aay frm he flying direcin. The pimum chice f lengh. is mainly dependen upn mean bjec disance. The bunding values fr. ill be resriced by real imaging circumsances; cnsrucin f a sable rain sysem is cmprmised due large a radius.. The chice f he inciden angle f he camera pical aes and he camera psiin angle as chsen earlier be /01 i.e. angenial direcin. This is a reasnable chice in he case f using c-cenric circular imaging blcks fr measuremens. The phgrammeric nerk design prblem is subsanially simplified in his imaging mdel. The effec f differen chices f circular imaging blck parameers n 3D bjec esimaes is discussed in Chaper 4. This kind f apprach is applicable in mdelling asks here here is n reference sysem nearby and he camera sain cnfigurain is difficul design r build. 3.2 Esimain f he del frm a Circular Imaging Blck In rder deermine bjec pin crdinaes, he eac rienain f he camera a he ime f epsure has be slved ih respec he chsen crdinae sysem. Since he crdinae sysem ill be creaed frm scrach ihu any crdinae reference sysem, he nly bservains included are image bservains. The nly priri infrmain hich is aken in accun is ne r mre bjec pin disances. Thse disances can be cnsidered as bservains r hey can be seen as a cnsrain beeen crdinaes f bjec pin pairs. The purpse f scale measuremen is bain bjec recnsrucin in meric unis. Since he crdinae sysem is parially based n a predefined mvemen f he camera, i is lgical slve he camera rienains simulaneusly. In rder ensure he accuracy f bjec recnsrucin, ver-deerminain (i.e. making mre bservains han he minimum requiremen) is eplied. By using verdeerminain, i is als pssible assess he precisin and reliabiliy f bjec measuremens plus he qualiy f measuremens. Due ver-deerminain, adjusmen f bservains is required. As menined in Secin 2.1, he verdeermined prblem slved ih an LSQ esimain has prved be saisically 44

45 ^] ^] rigrus and herefre, in he discussin f he esimain mdel furher n in his hesis, he use f he leas squares mdel is assumed. 3.3 Perspecive prjecin The camera mdel epeced here is based n a cenral perspecive camera mdel. S he mahemaical mdel f 3D--2D ransfrmain is a perspecive prjecin. The crdinae ransfrmain frm 2D 3D image space ill hen be in he frm: 2R :9;2=<?>@7BADCFEHG IKJ LNPRQ denes a 3D pin crdinae vecr and 2F<S>T7 he camera prjecin cenre crdinaes in he chsen crdinae sysem in Equain (13). The 3D rain mari I is assumed be rhnrmal and he image crdinae bservain vecr LNU4Q is suppsed be image cenre crdinaes. The scalar C is a scale facr beeen 3D and 2D image spaces. The inverse ransfrmain derived frm Equain (13) yields: VWYX Z [K\ 9;C VẀ_ GaG bg dg Gcb bab dab Gcd bad dad ]^evẁ f [ fhg i=36587 [ i g jk36587 [ j g Eplici in Equain (14) is ha he hree pins 2l<S>T7mS2R36587 and LNURQ are cllinear. Tha is he reasn hy i is idely called he cllineariy cndiin. The linear equain sysem in Equain (14) can als be rien as separae equains, here he scale facr C ill be eliminaed: f X n9 [K\ GaGp 3q587 [ fhg?r A Gcbs i=3q587 [ i g?r A _ Gcds jk3q587 [ j g?r f dgp 3q587 [ fhg?r A dabs i=3q587 [ i g?r A _ dads jk3q587 [ j g?r f Z l9 [K\ bgp [ fgsr A babs i=36587 [ i gsr A _ bads jk36587 [ j gsr (15) f dgp [ fgsr A dabs i=36587 [ i gsr A _ dads jk36587 [ j gsr denes an elemen f rhgnal rain mari Iv The 3D rain mari I is an rhnrmal mari, hich is uniquely defined by m?yzm?{ hree independen rain angles : In Equain (15) he _ u7 (13) (14) I 9 }VW~ y ~p { ~ ƒ {A ƒ ƒ y ~ { ƒ ƒ { [ ~p ƒ y ~p { [ ~p y ƒ { ~ ~p { [ ƒ ƒ y ƒ { ƒ ~ { A ~ ƒ y ƒ { ƒ y [ ƒ ~ y ~p ~ y ] ˆ ^ (16) 45

46 This ype f rain mari, Equain (16), is prved be saisically ms rigrus hen cmbined ih redundan bservains and an LSQ esimain (Cper and Rbsn 1996). The rains ŠŒ S4?Ž f he crdinae aes z = S, respecively, are assumed have psiive direcin n clckise rain hile lking frm he rigin alng he direcin f he respecive crdinae aes. The disadvanage f such a parameerizain f he rain mari is ha he bservain equain, Equain (1), presened in Secin 2.1 becmes nn-linear. Nn-lineariy f he funcin can be slved by linearizing he equain and slving parameers ih ieraive mehds. Then, hever, he iniial values fr parameers have be bained. This characerisic f he ransfrmain mdel has resriced is use in many real-ime applicains. Because in his research he rigrusness f he sluin is favred ver he speed f cmpuain, he nn-linear prjecin mdel is used. 3.4 Camera mdel The camera crdinae sysem is defined as a righ handed, rhnrmal 3D crdinae sysem. The rigin f he crdinae sysem is in he prjecin cenre f he camera and he image plane is psiively riened, i.e., i lies beeen prjecin cenre and bjec. In he previus secin, he perspecive prjecin as based, essenially, n a pin-hle camera mdel. This is an ideal mdel f ligh raveling frm a 3D bjec pin hrugh he prjecin cenre in he image plane alng a sraigh line. Unfrunaely, his is n he case in realiy, hen ligh has penerae hrugh he lens sysem n he image plane. A descripin f ligh raveling inside he camera is called he inerir rienain. The discrepancies f his ideal mdel can be divided in linear and nn-linear cmpnens. In rder accmplish precise phgrammeric measuremens, hese errrs have be eliminaed r heir effec n measuremens have be cmpensaed. The prcess f deermining hese sysemaic errrs is called camera calibrain. The defrmains are due many differen errr surces. The nnperpendiculariy f he pical ais and image plane cause crdinae aes f he image plane be nn-rhgnal and s here can be a scale difference beeen he - and -aes. Als, nn-linear defrmain ill be bserved, bu he amun f defrmain usually is s small ha nly linear cmpnens are cnsidered. The scale difference can als be an effec f he aspec rai f a piel and nnrhgnaliy f he effec f he misrienain f sensr elemens in rs and clumns in he case f a digial sensr. In calibrain, he lcain f he prjecin cenre ih respec he image plane is described by he principal pin crdinaes and. The perpendicular disance f he prjecin cenre frm he image plane is hen epressed by he camera cnsan. The scale difference is usually epressed by ne number, hich 46

47 ž ª ª ± ª ± ± ª ª ª ª ª ± ± ± ª ª «± ± mdels affiniy. The nn-rhgnaliy f image crdinae aes, r ske, can be deermined by ne value. The nn-rhgnaliy can als be rien ih he help f angle, hich denes he discrepancy in rhgnaliy and affiniy as š Sœ (Niini 2000). The camera cenred image crdinaes afer he crrecin f linear disrins can be hen given as In Equain (17) Ÿ and ª ž Ÿ ŸH = Ÿ cª cª ««(17) denes he camera cenred and linear disrin crreced image bservains. The nn-linear cmpnens f inerir rienain mainly are caused by he camera lens sysem. When ligh cmes frm bjec pins he image plane, i has g hrugh an pical lens sysem ih several lenses ha ill change he direcin f he ligh beam sysemaically due differen lens maerials r misalignmen f hese lenses. This disrin is bserved be nn-linear. The disrin mdel can be divided in radial and decenring disrin. Radial disrin causes image pins mve aay frm he principal pin (pincushin disrin) r ard he principal pin (barrel disrin). This disrin is knn be circular symmeric in respec he pin f bes symmery. This disrin can be mdelled by a hird degree f plynmial (Brn 1971): ŸF ŸH ¹ª ŸH = Ÿ s««q p c s± cª º p c s± «µ ³± «µ ³± c c «µ s «µ s c s c s ««³ ³ ««(18) ŸH = Ÿ s««here, ¼» cª ±, and denes he radial disance frm he pin f bes symmery, see Equain (18). The radial disrin is zer in he pin f bes symmery, bu i can be frced be zer in any freely-chsen radius. The differen chice f ill have an effec n he values f cefficiens and camera cnsan, bu he al effec f changed parameer values ill cmpensae he radial disrin equally ell (Brn 1971). The decenring disrin is caused by an imprper alignmen f lens elemens in a cmpund lens sysem, ih is crrecin being given as (Brn 1966): ž½ÿ ¾ ¾ ŸH DÀÂÁ ÄÀÂÁ Ū ŸH l šÿ s«ÿh = ÆŸ s«cª cª «µ ÃÁ «µ ÇÁ c ± c ± ÄÀ DÀ ŸH Ÿ s«h «cª «(19) Nrmally, in camera calibrain, all hese parameers are deermined simulaneusly. Niini (Niini 2000) pins u in his invesigains ha lens disrins shuld be aken in accun befre he disrin raised in he imaging media is crreced. This is because he lens disrins ccur befre he ligh beam is 47

48 Ö ß ß ß â â â ß epsed her disrins. In pracice, his means subsiuing he È=ÉnÊYÈ Ë and Ì É Ê Ì Ë bservains by heir crreced values Ì Î Ð in Equain (18) and Equain (19) and slving he crrecins linearly disred crdinaes. This subsiuin yields an ieraive sluin f calibrain parameers. Niini, hever, discvered ha, if he effec f radial disrin is less han Ò piels in size (ih he camera in quesin), he difference f hese esimain mdels is less han uni measuremen precisin, here epeced be Ò ÓÔÒÕ piels. The difference can especially be seen ih devices ha have subsanial affine disrin, like vide cameras. Bu even ih vide cameras, he effec f lens disrin is unlikely be greaer han he menined bunding effec. Since e are n primarily ineresed in he acual values f calibrain parameers, bu mre in he abiliy cmpensae disrins f image bservains, e can safely use he flling crrecin mdel, here all camera-calibrain parameers are slved simulaneusly: È TØqÙ6ÙÛÚ¹ÈÎÜYÈÎ8Ý ÞHß TÎ Íqà Ì TØqÙqÙçÚ Ì ÎÜ Ì ÎèÝÞHß cî ͺà Ýs TÎ ÜDàâ ݳã TÎ ÜDà ݳ TÎ ÜDàâSÝsã cî ÜÄà Ý TΠеÜÃå ÍTÝ TÎ ÜÄ È Î ÐµÜDÂåHâÈÎ Ì Î Ý³ cî еÜÃåHâsÍcÝ TÎ ÜÄ Ì â ΠеÜÄ?å ÈÎ Ì Î here Ýé TÎRÚëê È â Î Ü Ì â Î, see Equain (20). her surces f nn-linear disrin migh be due a discrepancy f sensr r imaging media frm he recangular fla image plane. In he case f film, he defrmain f he media can ccur a he ime f epsure if he film is n precisely in cnac ih he image plae. The nn-linear defrmain f film can als be a resul f he develping prcess r film-srage envirnmen. Wih digial sensrs, he nn-linear defrmain can be cnsidered as a manufacuring imperfecin. Sensr elemens in a CCD array migh n be perfecly lined up r hey migh n be evenly spaced. The flaness f he sensr array migh als be cmprmised. The nn-linear defrmains f he imaging media are quie difficul mdel. Usually sme plynmial funcins have been applied in rder cmpensae fr he effec f hese anmalies n image bservains. The prblem ih highrder plynmials is ha hey require a subsanial number f measuremens be made in rder be precisely defined. Fr his reasn, réseau grids ih knn image crdinaes have been used ih film cameras. In mdern digial sensrs, he nn-lineariy has been cnsidered s small cmpared he image bservain accuracy ha i has been ignred. In addiin, he nn-rhgnaliy cmpnen has been fund mdel he defrmain f digial sensr irregulariies als. In addiin, he insabiliy f he sensr lcain inside he camera has been ned (Shris e al. 1998, 2001). The calibrain unknns can be divided in blck-invarian and -varian parameer. eaning ha blck-varian parameers (e.g. principal pin crdinaes) are independenly derived fr each epsure and all her calibrain parameers are in cmmn fr all epsures in image blck. This apprach is gd fr cases here imaging gemery is adequae fr self-calibrain. An imprved prcedure is suggesed, here blckinvarian parameers are defined ih help f finie elemen crrecin grid (Has- 48 (20)

49 ed e al. 2002). In his mdel als unflaness f sensr can be defined. Hever, if self-calibrain is n apprpriae fr he ask, his sabiliy prblem has be aken care f ih pracical means, such as frequen calibrain, r ensuring physically sable CCD chip psiining in he camera bdy. In his research, he linear and nn-linear defrmains are epeced be crreced based n pre-calibrain f he camera. Self-calibrain by including inrinsic camera parameers in a circular imaging blck-adjusmen can als be applied, bu, due resriced imaging gemery, deerminain f all calibrain parameer values is n feasible. The deerminain abiliy als depends very much n he bjec pin disribuin. 3.5 Esimain Prblem, Apprach I We can see he esimain prblem as ìçí½î number f relaive rienains, r even ì number, if e include relaive rienain beeen he firs and las image in sequence. By chsing he mdel f independen sere mdels, e can avid reslving he bjec pin unknns. T d s is advanageus, since e are primarily ineresed in image lcains and rienains in an image blck. The idea behind his pririizain is ha e firs an ge he image blck and crdinae sysem creaed. Then e carry u measuremens n images in rder recnsruc bjecs as in he rdinary mapping prcess, raher han argeing all bjec pins needed recnsruc he bjec and reslve bjec pins simulaneusly ih blck parameers. Alernaively, e can end up using he bundle blck esimain and applying he cllineariy cndiin. Then e ill have ì number f eerir rienains be slved. The unfrunae siuain is ha n e have assign iniial values fr all f ur ie pins, because f he nn-linear mdel. In bh cases e have a daum ha is insufficien. We can apply a free-ne ype apprach and use a minimum nrm sluin r e can fi sme parameers in rder ge he daum becme sufficien. We d n epec have any eerir pins knn in any crdinae sysem. As menined in Secin 3.1.2, e creae a lcal crdinae sysem n sie fr ur measuremens. By selecing he navel pin f he image blck as an rigin, and fiing he ï -ais f a defined crdinae sysem in he direcin f he firs image prjecin cenre hile he rainplane nrmal assigns he secnd direcin f he rhgnal crdinae sysem, he daum prblem ill be slved. As he camera is raed arund he rigin, he rienain ill change ih respec he crdinae sysem, bu he angle beeen he image plane and psiin vecr f he prjecin cenre ill say cnsan. By applying his knledge, and he fac menined earlier ha all prjecin cenres ðbñ lie n he pah f he same circle, e can se cnsrains sabilize he esimain prcess. 49

50 6 þ øãù ôœúdû ò¾ózôò³õåö ù ôüúdý ù ô õåö (21) (22) Equains (21) and (22) sae ha all prjecin cenres are a he same disance ö frm he navel pin, bu his des n say anyhing abu hem lying n he same plane. This can be epressed by seing a cnsrain beeen he prjecin cenres and a nrmal vecr þ f he rain plane: ó4ô õ (23) We can als cnsider þ as a ne parameer vecr be esimaed. By giving a large eigh in he LSQ esimain fr his bservain equain e can frce he sysem reain his cndiin, given in Equain (23). The cnsan angle beeen he psiin vecr f he prjecin cenre and he pical ais f he camera can be frced by adding he flling cnsrain in esimain: ô ózô ò¾ózôò õ (24) In Equain (24), ô denes he rhnrmal rain mari f he image in sequence and órô he equivalen prjecin cenre psiin vecr. By using independen sere mdels ö and a cplanariy cndiin, here ill be ú unknn parameers (, þ and cnsan value are included as unknns) r unknns plus "! $ cnsrain equains. Wih he bundle f rays mdel, e ill ú% '! end up # plus & unknn parameers, here & denes he number f ie pins. Since sere mdels are based n he cplanariy f crrespnding image rays and base vecr, n eplici bservain funcin can be rien here nly ne bservain is invlved. This leads he cnsrucin f he cndiin equain and he general adjusmen mdel (ikhail 1976): (*)+-, ú/.10µú/2"3 õ õ87 (25) ) here he mari denes he cefficien mari f bservains and he mari cnains he cnsrain cndiins, see Equain (25). The bundle f image rays mdel is based n he cllineariy cndiin and eplici bservain equains. 50

51 X < < I J Y d P P E = X P P S P P Š R The equain is equivalen Equain (1) ih he addiin f cnsrains, see Equain (26). 9;:"<>=@?BADC <>=GF (26) 3.6 Esimain Prblem, Apprach II The image blck esimain based n he bundle f rays mdel can als be rien by using a differen se f parameers. As e are n using any eerir cnrl pins, bu are cnsrucing ur n crdinae sysem, e can make sme assumpins. As e have already defined he rigin be he navel pin f revluin and he -ais be in he direcin f he firs ph prjecin cenre, e can als sae ha he prjecin cenre f he camera in differen <I camera pses ill lie n a plane parallel a crdinae plane. We can chse he H -ais pin upards s ha all prjecin cenres ill lie n he -plane. This chice is as gd as any her, and idely used in erresrial phgrammeric nerks (Slama 1980). N e can fi he H -crdinae f all prjecin cenres be a cnsan and epress he - and -crdinaes in a plar crdinae sysem insead f Caresian crdinaes: KLNQP =@R STUWVX =@Z[]\_^`ab\c` =8R SeVgfihjX N e d n need pu any cnsrains frce parameer values f he prjecin cenres lie n a plane nr be a a disance f frm he cenre f revluin, because all his infrmain is included in Equain (27). Wha e have n ye included in he mdel is he cnsan angle beeen he pical ais f he camera and he psiin vecr f he prjecin cenre. In a special case here e have zer il kml and spin nl angles, ur rain mari differs frm he firs camera rain mari prq]sg usv s nly by he difference p"q]sg usvy{z s. (Ne ha he } increases in he ppsie direcin ). re generally, he rain f he camera by P can be epressed in a 2D rain n he plane: (27) p"q]~ u ~ ~ p"q]sg usv s pbze~ (28) here, pbze~ = T UWVƒX Vvf h"x Vvf hjx P TUWVƒX ˆ P (29) 51

52 Á The elemens f he Œ"]Ž Ž Ž ill hen be: š œ žÿwƒ ŸWc BŸW gï jªgï " «žÿwƒ ŸW ± v r g ³ v r BŸW µ g j g Ÿ žÿwƒ gï ¹ v " ŸW gï " BŸW ƒµm gï jÿ ŸW b«g œ * ºŸW Bv " b«š«žÿw Ÿ gï ³ g r Bv " b«š žg ³ ŸW ª/ ŸW gï " Bg " bģ œ žg jª ŸW BŸ ƒ ª/ ŸWƒv " b š«žg j> Ÿ v " gï ¹ v " BŸ ƒ ƒµ» ¼ŸWv ³ ŸW b š žg j> gï v " Q ŸW gï r BŸWƒ ƒµ_ÿw ŸW ŸW (30) Since he rain angles ¾½, ƒ½, ½ depend n he firs image rain mari by he angle _½ and prjecin cenre crdinaes f a single camera psiin are dependen n he values f and À½, here is n need esimae hese dependen parameers. Insead, he cllineariy Equain (15) can be rien accrding hese independen parameers as * B Á ÌQ Í râ š à ÄÆÅÇÈ Ÿ ƒ ½µ_D «-ɃÄÆÅdz ʃµ D ] -ËœÄÅÇ» g "_½ µ bģ à ÄÆÅÇÈ Ÿ ƒ ½µ_Db š«-éƒäæåç³ Êƒµ Db š ] -ËœÄÅÇ» g "_½ µ b«g à ÄÆÅdz Ÿ c_½ µ_/b«š«-éƒäæåçè /ʃµ_Db«š -ËœÄÆÅÇÎ v "_½ µ bģ à ÄÆÅdz Ÿ c_½ µ_/b š«-éƒäæåçè /ʃµ_Db š -ËœÄÆÅÇÎ v "_½ µ (31) This nnlinear bservain equain, Equain (31), is hen linearized ih respec he firs image rain angles með Ð ±, radius and he angle À½, Ò ÐÓ1ÐÕÔÖÔiÐg. Als, bjec pin crdinaes are epeced be unknns and linearizain is be carried u ih respec hse bjec parameers à ÄÅÇÐØɃÄÆÅÇeÐË»ÄÆÅÇ as ell. Value Ê is sme arbirary cnsan. The resul f linearizain is presened in Appendi I. I is essenial nice ha all image bservains frm all images have an effec n he deerminain f he firs image rienain angles ¾eÐ Ð ± and radius. All camera rienains are dependen n hse parameer values and herefre hey can be cnsidered as cmmn parameers f he image blck. N, he al number f image unknns ill be Ù 8. The assumpin f he firs image prjecin cenre be in he direcin f he ais means ha» ill be fied zer and he number f unknns reduced by ne Ù Ú Û Ò. The al number f unknns is hen Ù Ü ÒeµÞÝ ßà. The effec f ne image bservain n he nrmal mari is depiced in Figure (5). I can be seen ha he number f unknn blck parameers is less han ih using he parameer se presened in Secin 3.5. Als e can avid he use f cnsrain equains. Eerir rienain parameers f dependen camera pses can easily be derived back sandard presenain in a Caresian crdinae sysem by applying Equains (27) and (28). 52

53 â ååå ω n 1 m 3 φκ r α XjYj Zj 0 i 0 ççç 0 0 Figure 5. Effec f ne image bservain n nrmal mari. I as menined earlier ha pin inersecin uld be pr unless e d n use image bservains frm a secnd c-cenric image blck. The difference beeen hese crdinae sysems is nly an angle beeen heir è é aes. S he angle can be esimaed frm bservains f cmmn pins. Wih a single blck esimain, e can find ha he same gemerical prblem ill appear. The inersecin angle f image rays fr he unknn 3D ie pin ill be raher small, as shn in Figure (6). S, he psiin accuracy fr such a pin is quesinable. Even hugh hse ie pins are n be used fr mdelling purpses, he unreliabiliy f hese bservains als affecs he deerminain f camera rienain. r α i Figure 6. Pr inersecin gemery. Based n his aspec, bh image blcks mus be esimaed simulaneusly. This ay e can bain gd bservains frm cnvergen images, as depiced in 53

54 Figure (4). The number f addiinal unknns ih respec ph parameers is ì íúîcï. The firs fur unknn parameers cnsis f rain angles f he firs camera psiin n he secnd blck and radius ð]ï f he secnd circular imaging blck. The angle ñò (equivalen óîò in he firs blck) is an angle beeen he firs image prjecin cenre and he ô -ais. I is be emphasized ha bh blcks are be bund in he same crdinae sysem and herefre all camera rains ñ±õ, hen öî ùøwúû1úøü1úõýiýöýþîï, are measured frm he ô -ais. The number f ie pins des n need increase much as bh sequences ill include mre r less he same scenery. The ie pins are epeced be bserved frm bh image blcks since herise he gemery f inersecin ill be pr. The assumpin made is ha he camera is urned by ø a he end f he suppring bar, as suggesed in Secin and shn in Figure (4). The camera prjecin cenres ill sill be n he same ô -plane as he firs image blck. herise crrecin fr he heigh, r mre precisely -crdinae difference, has be inrduced fr he prjecin cenre crdinaes f he secnd blck. As e ill cnsruc ur n crdinae sysem, and as e d n have any eerir knledge, e have have a scale fr ur measuremens. Disance measures can be included in he esimain as a cnsrain equain r as a nrmal bservain ih a large eigh. Fr defining scale fr he measuremens, e can include a disance measure beeen bjec pins ƒõ and as an bservain equain in he frm f Equain (32). õ ï í õ ï í õ ï Wõ (32) The mdel presened here resembles he imaging mdel f mnivergen sere (Seiz e al. 2002) and sere panramas (Peleg and Ben-Ezra 1999). In bh cases, he bjecive is recnsruc image msaics, here he bjec pin can be seen in panrama images having image rays ih cnvergen angles. Imaging is assumed cmply sricly ih he imaging mdel; n variain is alled. In cnras hse appraches, in his research he measuremens are be made frm muliple perspecive prjeced images and he bjecive is epli he ver-deerminain and bundle mehd in bjec measuremens. Als, deerminain f he eac rienain f he camera a he individual ime f epsure is he purpse f muliple image bservains. In bh he mnivergen sere and sere panramas mehds, he gal is reduce he redundan infrmain in he image sequence and epli he epiplar gemery and sere maching algrihms. Als, aspecs f he accuracy f bjec measuremens have n been reaed in hese mdels. 54

55 3.7 Image Blck Esimain The mahemaical mdel used is based n image bundle blcks and a cllineariy cndiin. An alernaive mehd uld have been use independen sere mdels as primary cmpuain unis. I is rue ha blck adjusmen based n sere mdels and a cplanariy cndiin des n include unknn 3D bjec pins in he esimain, hich as saed earlier. Bu, hinking gemerically, hse unknn pins are sill here, and in he case f a bundle f rays ne can alays eliminae he unknn 3D pins frm a LSQ ype esimain by creaing a reduced nrmal mari (ikhail 1976; ikhail e al. 2001). The design mari and nrmal mari can be pariined in respec rienain unknns and bjec crdinae unknns as flls:! "$#&% (33) '(*) +-, ). / ) $# )#. / )# $#10 -, 2342#.#54.#3#20 (34), 63 /6# 7#5 /7#3# 0,*8 8 # 0,$9 9 # 0 (35) The design mari is pariined in sub-marices here he clumns f he mari represen he cefficiens f he image rienain r 3D pin crdinae values as depiced in Equain (33). By using a similar nain, he sluin vecr can als be divided in pars 8 cnsising f he parameer values f eerir rienain parameers and 8 # he 3D pin crdinae values, respecively, see Equains (35). The nrmal mari can hen be reduced he size f he sub-mari 63 f he riginal nrmal mari (ikhail 1976): : 23 <;=7#5 >@? #3# 2#BA 8C 9EDF7#5G@? #3# 9 # (36) The nain 9 and 9 # are resul f crrespnding pariin f righ-handside f he nrmal equain in Equain (36). By using he linear mdel, e d n need re-eliminae he 3D pin unknn parameers, bu, since e have chsen use a nnlinear ype mdel, e are frced reslve crrecins apprimains f pin unknn parameers als. The back subsiuin bain 8 # in given as 8 #H #3#!I? 9 #CDF7#5 (37) 85J 55

56 Equain (37). The idea f eliminaing he unknn pin parameers frm he esimain is beneficial. Since e are ging have numerus images included in a single circular imaging blck, e ill ms likely have numerus unknn 3D pins as ell. In sandard clse-range blck adjusmens, e usually eliminae he image parameers frm he nrmal mari in cases here e have numerus images, since ne image increases he diagnal elemen f he nrmal mari by si. In his mdel, ne addiinal image increases he number f unknn parameers by ne, bu ne addiinal 3D pin increases he number f unknns by hree. Als, in he case f cnvergen imaging, he same bjec pin can be seen in muliple images, smeimes even frm all f hem. In circular imaging, he scene frm image image changes, and ne pin can be seen nly frm a small subse f images, herefre he number f ie pins increases significanly. As he nrmal mari K can be updaed sequenially by bservain equains, here is n need cnsruc he design mari L a all. Als, he unknn parameers can be eliminaed sequenially. S he eliminain can be perfrmed pin-by-pin. Alhugh e have ake care ha all bservains aached ha pin are updaed cnsecuively he sub-mari fkn andk73. Afer his, he reduced nrmal mari can be updaed by using Equain (36) and e can cninue by prcessing he bservains f he ne pin. The number f seps cnsruc he reduced nrmal mari des n differ much frm he number f seps used cnsruc he riginal K. Calculaing he back subsiuin increases he number f seps, bu he cmpuing ime in his ask is very shr cmpared he ime spen fr cmpuing he LSQ sluin fr a large nrmal marik. 56

57 4 SIULATIN The ms desirable ucme f a phgrammeric measuring sysem is he abiliy creae a cnsisen and reliable mdel f he bjec scene. The highes aainable accuracy f bjec measuremens can be evaluaed by he use f simulain. Simulain is a l verify he crrecness f a mahemaical mdel in an arificial siuain, smeimes due unknn siuains, lack f eperience, r reprduce essenial feaures r characerisics f a phenmenn. fen he bjecive is prvide he circumsances ha are as clse as pssible hse f a real siuain. In his research, he basic measuring cndiins are mainained hrughu esing hile nly ne parameer value f he sysem a ne ime is alered in each eperimen. Due his alerain f muliple variables, he impac f change f an individual parameer value n bjec measuremens is hard r unreliable deermine. In many cases, he unreliabiliy f assessmen is due a high crrelain f individual parameer values. In simulains carried u in his research, he mainained cndiins ere: he number and disribuin f bjec pins in a scene; he camera mdel used; and he rienain f camera ih respec he suppring bar. The bjec pin se used in he simulains as cmpuer generaed, cnsising fprq Q bjec pins. The bjec disance frm he rigin varied frm fifeen meers and he mean bjec disance as abu eigh meers. Generain f he pin se as accmplished by using a randm number generar generae a nrmal disribued daase. The nly resricin fr bjec pin generain as ha bjec pins ere alled be siuaed n farher ff han he maimum disance and n clser han he minimum disance. Als, i as ensured ha all pins fied he heigh range accrding he field f vie f he used camera mdel. This as guaranee bjec pins uld be visible n images f he ficiius image blck. The camera gemery as chsen be SQTPVUXW(SYPRZ Q piels ih a camera cnsan f S[U Q Q piels. This camera mdel resembles he gemery f he real camera lympus Camedia C-1400L, hich as available a ha ime a he insiue here he research as carried u. The piel resluin and field f vie have an effec n bjec measuring accuracy as ell. Tesing he measuring accuracy ih differen camera gemeries culd have been ne es pin, bu, as he chice f camera cann be cnsidered relae paricularly his phgrammeric measuring mehd, nly ne camera mdel as used. Since n prminen camera gemery n erresrial imaging can be assigned, his selecin can be cnsidered as gd as any her chice. The piel size f his paricular camera mdel as annunced by he manufacurer be P \^]_U `ba and as used in cnvering he piel nise level values crrespnding values in micrmeers in Tables (1) and (2). The rienain f he camera ih respec he plane f rain, and he direcin f he camera pical ais ih respec he rain pah, ere kep 57

58 cnsan. The pical ais f he camera as parallel he plane f rain and als angenial he circular pah f prjecin cenres during he rain. The rain angles f he camera in he firs camera pse in he firs image blck ere herefre cedfggihjdkfggilmdkf and fr he secnd blck cedfggihjdnp ftqrgilsd+f. herise, he camera as raed arund a fied rain pin ih equal angular seps in he ficiius image blck, i.e., nly he h -angle as changed. I is be ned ha he -ais as parallel ih he nrmal vecr f he rain plane. Simulaed image bservains ere creaed by back-prjecing hse 3D bjec pins, hich fied he field f vie, n he fcal plane in each camera pse. Randm nise f sme predefined level as added bservains and he adjusmen f he blck as perfrmed. In adjusmen, bh image blcks ere cmpued simulaneusly ih bjec pin crdinaes. In addiin hese arrangemens, ne scale bservain in he size f meers as included in he cmpuain. These arrangemens remained he same hrughu all simulain ess. In rder aain reliabiliy in erms f saisical variables, nf f simulain runs ere carried u. Saisical parameers as he mean value and variance f blck parameers and bjec crdinaes ere cmpued frm he resuls f nf f simulain runs f ess. 4.1 Selecin f simulain parameers The perfrmance f circular imaging blcks can be uned by changing a fe facrs in he cnsrucin f he blck. The accuracy f a final 3D mdel depends n accuracy f image bservains, gemery f imaging nerk, number f bservains, and precisin f camera mdel. These are all facrs ms fen menined in he lieraure dealing ih he accuracy f clse-range phgrammery (Fraser 1984, 1989; asn 1995). The effec f image bservain accuracy n bjec parameer deerminain is apparen. Als, i can clearly be shn ih errr prpagain, h he gemery f inersecing image rays affecs bjec accuracy. An increase in he number f bservains abve he number ha is necessary slve he bjec parameers des n nly imprve he accuracy f bjec measuremens, bu als gives us a l ih hich esimae he precisin f ur measuremens ihu any eerir reference. By applying a crrec and precise camera mdel he ccurrence f sysemaic errrs n he bjec mdel can be prevened. In his kind f cnsrained imaging sysem, he nly facrs having an influence n imaging gemery are he lengh f he used bar and he iniial rienain f he camera ih respec rain. The rienain f he camera has an influence n imaging gemery a clse range. Since he bjec disance in hese ess spans ver nuv he lengh f he bar ill have mre effec n he imaging gemery han he rienain. The accuracy f measuremens has been simulaed by adding sme nise n crrec image bservains. The number f bservains can nly be increased by shrening he angular sep beeen camera 58

59 pses in he image blck. The crrec camera mdel has an influence n crdinae values, as in any phgrammeric nerk. In a free-ne ype phgrammeric nerk, he incrrec camera mdel may cause minr effecs n bjec crdinae values, since par f he sysemaic errr ill be absrbed in rienain parameer values. In his kind f cnsrained imaging sysem especially, he crrecness f he camera mdel has an essenial imprance, since sysemaic errr is mre likely be presen in crdinae values han in cnsrained rienain parameer values. Neverheless, in hese simulain ess, he camera mdel is assumed be crrec ihu any sysemaic errrs. 4.2 Nise level The flling prcedure as adaped hile esing he nise level f he image measuremens. This nise level es is simulae he use f differen cameras r bjec pins f differen qualiy. The chsen nise levels ere Gy{z, Gy{, Gy}Tz and Gy}T piels. The firs cases can be cnsidered simulae bad and gd image measuremens f naural bjec pins and he laer as bad and gd bservains f argeed bjec pins. The nise levels can als be hugh f as cameras f differen qualiy r resluin. In hese simulain ess he radius as a cnsan zrr~ and R images ere included in bh image blcks. The bjec pins ere back-prjeced n he fcal plane as image pin bservains accrding he rienain infrmain. The qualiy f image bservains as hen degraded by adding nrmal disribued nise n image crdinae values via a randm number generar. In he generain f nrmal disribued nise, he algrihm depiced in (Knuh 1981) as flled. The nise generaed had a zer mean ih a sandard deviain fgyz, Gy{,Gy}Tz and^yt piels, respecively. Frm 100 runs f ess, he mean values and deviain f blck parameer and bjec 3D crdinae values ere calculaed in each es case. The sandard deviain f blck parameers have been clleced in Tables (1) and (2) and an equivalen presenain f he bjec crdinae sandard deviain is depiced in Figure (7). In Tables (1) and (2), he sandard deviain values f ƒ -angles are averaged mean sandard deviains. ˆŠ Œ ŽR Ž r Œ Ž Ž r Œ Ž Ž r Œ ŽT jœ Ž Ž r Œ GyzR š Gy}TŒ Gy R z^y Œ Gy}Tz š y} TŒ Gy}T Gyz Œ Table 1. Sandard deviains f blck parameers, Blck I 59

60 œšžÿb R ž r G 2ž r G ž r G T Hžj ªjž r G «+ G R žš ± } «+ G R ž ^ Y «+ G } T žš ±T «+ G } T ž G Y Table 2. Sandard deviains f blck parameers, Blck II Frm Table (1), e can see ha sandard deviain declines firs half and hen Y³^ and Y³ Rfrm he firs lised values. The linear change in sandard deviains ih changing «is enirely as prediced and cnsisen ih epecains. The same phenmenn can be seen in Table (2). The nly difference beeen hese ables can be seen in clumn ª, here values in Table (2) are slighly higher. Tha is parly due he esimain f he angular difference f -ais in he firs and secnd blck. This uncerainy f esimae affecs he esimaes n ª<µ -values in he secnd blck. In he firs image blck -ais as fied. In rder evaluae he ie pin accuracy, mean values f pin crdinae sandard deviains ere cmpued and indeed in respec 3D pin nminal disances frm he cenre f measuremens. Secnd-rder plynmials ere hen fied pin disances and crdinae mean deviains. Crrespnden graphs f differen nise levels are shn in Figure (7) σ = 0.5 σ = 0.2 σ = 0.05 σ = Sd.Dev (mm) Sd.Dev (mm) Disance (m) 0 Figure 7. Effec f nise n bjec pin accuracy Frm Figure (7), he accuracy f such a measuring cnfigurain can be esimaed be. R R fr he bes case and º¹ in he rs case. These relaive accuracy numbers are derived assuming he maimum dimensin f he bjec be ±. As his imaging sysem is symmerical, he maimum disance frm he imaging sain is hen». 60

61 4.3 Lengh f radius The lengh f he bar here has he same meaning as he base lengh in serescpy. In principle, he lnger he bar, he beer he precisin. Hever, in pracice, he imaging envirnmen and mechanical cnsrains can limi he lengh f he bar. This can ccur hen here is nly a cnsriced space fr he camera be raed r hen eending he lengh; he vie ill als change, and bjecs clser he cenre ill be u f sigh, n ms f he images r hen he cnsrucin f such a rain sysem ill be unsable. Ficiius circular imaging blcks ih differing radii ere cnsruced in he cenre f a randmly generaed bjec pin se. The field f vie accrding he camera mdel as ¼T½ ¾º (¼TÀ ¾ depending n heher i as evaluaed in he direcin f Á - r  -ais. In each case, he number f images in a blck as à ½. S, in each cmpuain, ÄTÅ blck parameers and Ä ½ Ä crdinae values ere subjec esimain. The number f bservains depended n he pse f he camera ih respec he bjec pin se. nly hse bjec pins ha fied inside he field f vie, ere back-prjeced n he image plane as image bservains. The difference in number f bservains beeen he maimum and minimum case as abu Æ %. This variain as due he change f imaging gemery and can be cnsidered insignifican in erms f redundancy f esimain. Sd.Dev (mm) r = 0.2m r = 0.3m r = 0.4m r = 0.5m r = 0.6m r = 0.7m r = 0.8m r = 0.9m r = 1.0m Sd.Dev (mm) Disance (m) 0 Figure 8. Effec f lengh f radius n bjec pin accuracy The sandard deviains f crdinae values ih respec disance are depiced in Figure (8). The sandard deviains ere calculaed frm Æ[½ ½ simulaed es runs ih nrmal disribued nise added a a level f ½GÇÈ piels. The plynmial curve as fied he daa, here he mean crdinae sandard deviain as depiced n he rdinae and he disance f he 3D pin n he abscissa. A curve as fied all daa ses f differen radii. The resul f he simulain shed ha he effec as n ally linear. The furher he bjec 61

62 pins lcae, he mre significan is he imprvemen f pin precisin. Afer sme limi, he eensin f he radius did n imprve he resul significanly. In his case, he limi seemed be arund ÉRÊ ËÌ and as, ihu quesin, als dependen n he srucure f he bjec pin se and chsen camera mdel. 4.4 Number f phs in blck The change f number f images in a blck as als esed in a similar ay. Nrmally, i can be saed ha increasing he number f images des n imprve he precisin much, unless he hle imaging gemery f he phgrammeric nerk is imprved. This hlds in radiinal nerks, here each ne camera psiin brings si ne parameers in esimain. In his apprach, ne ne camera psiin adds nly a single parameer Í Î in adjusmen, hile redundancy is essenially larger. This is due he use f plar crdinaes fr presening he prjecin cenres in he frmulain f he mahemaical mdel f he circular imaging blck. Frm Figure (9), i can be seen ha increasing he number f images and bservains imprved he precisin up Ê Ê frames per blck, bu, afer ha n significan imprvemen culd be seen. Hever, heher i is rhhile nearly duble he number f frames in image prcessing if he epeced accuracy imprvemen is less han Ð %, as i is hen increasing he number f frames in a blck frm Ê ÒRÊ frames (i.e., frm Ó Ê Ê Ê images, ally) shuld be cnsidered. The amun f addiinal prcessing is dependen n he chsen image measuring sraegy and has be assessed in each individual case. Sd.Dev (mm) frames 20 frames 30 frames 50 frames 100 frames 150 frames 200 frames Sd.Dev (mm) Disance (m) 0 Figure 9. Effec f number f images in blck n bjec pin accuracy 62

63 4.5 Qualiy f iniial values In simulain ess, he iniial values ere se clse he crrec values. The reasn fr his as ha here hen uld be n risk f he adjusmen n cnverging a lcal minimum. The simulain envirnmen as als used fr esing he limis f he gdness f iniial values. The parameer values ere slighly changed frm heir crrec values and nly ne parameer as alernaed a a ime. The es as firs accmplished ihu nise, and hen nly a small amun f nise as added he image bservains. The rienain angles f he firs camera ere mre sensiive he incrrecness f he iniial values han he Ô -angle f each ph r lengh f radius Õ. Fr Ö* iø iù -angles, he iniial values ere required be beer hanú ÛEÜÞÝRÛ in rder mee cnvergence. Fr Ô<ß -angles, ÝRÛ as generally gd enugh, and fr lengh f radius Õ he iniial value à.ýrâ as accepable. When evaluaing he sysem sensiiviy ih respec he iniial values f 3D pins, i as discvered ha he discrepancy f a fe decimeers up a meer frm crrec crdinae values ere sill accepable. In general, i can be ned ha his phgrammeric measuring mehd requires mre accurae iniial values fr parameers han he measuring apprach based n cnvergence imaging. Acquiring he iniial values fr cmmn blck parameerö* iø BÙ -values is an especially demanding ask. The reasn ha hese parameers are mre sric ih heir iniial values can be deduced frm he fac ha all rain marices are derived frm hese parameers. Als, all image bservains affec he deerminain f he values f hese unknn parameers in he adjusmen prcess. Hever, baining such accurae iniial values fr his kind f regulaed imaging sraegy is easily achievable by he use f simple auiliary insrumens, i.e., measuring ape, angle measuring devices, ec. 63

64 5 VERIFICATIN F THE DEVELPED ETHD By using simulain, ne can verify he sysem as being able rk in general. Bu i is quie usual ha n all pssible variains frm he ideal sae f he sysem can be simulaed. This has als been he case in simulains described in Chaper 4. A pssible surce f errrs in imaging sysems is sysemaic errr in he imaging devices, i.e., in he camera and lens sysems. Als, i is presumed in he mahemaical mdel ha n deviain f camera pse ih respec he rain plane r rienain f he pical ais f he camera ih respec he pah f rain can eis. In rder verify he resuls derived frm simulaed ess, field ess ih real images ere carried u. The aim f he ess cnduced as cmpare he bjec crdinae values received frm he measuring sysem ih an eerir reference. T ess ere accmplished; ne as cnduced in an indr envirnmen and he her in he pen-air, in mre pimal cndiins. 5.1 Verificain mehdlgy The decisin use reference daa as made parly verify ha he develped mahemaical mdel uld apply in real cndiins and parly gain an esimae f achievable accuracy. The precisin f measuremens can be evaluaed by use f errr prpagain. This, hever, uld n necessarily reveal he pssible sysemaic errrs inside he esimain mdel. The reference infrmain mus be acquired ih a degree f accuracy beer han ha f he bserved daa i is cmpared, in rder guaranee he reliabiliy f es resuls. In his research, he eerir reference has been aained by achemeer measuremens. The same insrumen, a Gedimeer 600, has been used in bh ess in he acquisiin f reference daa. The 3D bjec arges ere measured mainly frm a single measuring sain by means f verical and hriznal angle measuremens and ne disance bservain. As is cmmnly knn, he achemeer can prvide accurae measuremens f arge bjecs if he bjec disance is abve en meers. A shrer ranges, disance measuremens especially becme less accurae. Hever, a cnfidence check culd be made in he calibrain measuremens f he insrumen in disances shrer han en meers. Because he imaging is symmerical in naure, he accuracy f bjec pin measuremens is suppsed be equal fr all pins a he same disance frm he imaging sain. This as realized earlier ih simulaed ess. Fr his reasn, he reference achemeer measuremens ere made frm he same pin as he phgraphic imaging. The bjec disance in such arrangemens is he same in bh mehds, hich are herefre mre cmparable ih each her. 64

65 The achemeer measuremens can be cnsidered be mre accurae han he image-based mehds f he circular image bck fr bjec disances lnger han five meers. Wihin shrer disances, he reliabiliy f achemeer measuremens is reduced and he accuracy ill be n he same level, r even rse, han ih image-based mehds. This is a mild draback, since, in rder assess he accuracy f his mehd in he hle range, sme accuracy infrmain is needed fr he shrer disances as ell. In rder vercme he prblem, bjec pins a clse range ere measured frm mre han ne measuring sain and crdinae sysems ere hen bund in a cmmn crdinae sysem. Unfrunaely, his culd be dne nly ih he es cnduced udrs. In he indr case, he pin disribuin and cmpleiy f he measuring envirnmen made he equivalen arrangemen impracical realize. Fr his reasn, he resuls cncerning he case ih shrer disances shuld be sudied ih cauin. In he pimized es case, he accuracy f reference can be esimaed be beer han millimeers, since ha as he maimum residual in he esimain f he crdinae ransfrmain f separae pin ses in a cmmn crdinae sysem Targes The bjecive in argeing is be able measure he same bjec pins in bh reference and phgrammeric crdinae frames. In he indr es case, he used arges ere prined black circles n hie backgrunds aached cardbard as shn in Figure (10a). In he middle f he black circle here as a hie sp, hich as used assis in seing he field prism in he cenre f he arge. Three differen sizes f arges ere used in he es cmpensae fr he scale difference in he image measuremens. The diameers f he circles ere ã, å and millimeers. The furher he pin as lcaed frm he imaging sain, he larger as he arge size used. In he pimized es case, he used arges ere rer-arges r rer-prisms. The rer-arges ere made f rer-reflecing maerial and ere designed especially fr achemeer measuremens. The arges ere riginally f a square shape, bu ere cvered ih black adhesive paper ih a rund hle in he middle, as indicaed in Figure (10b). The idea as prvide favrable cndiins fr image maching, since a circular shape in image maching is mre invarian perspecive disrin han a recangular. In his case, he size f he arges as fied. There uld have been differen sizes f rer-arges available, bu since he difference uld have been s small, nly ç èjè size arges ere used. The disance bservains ere esed ih achemeer rer-arge cmbinain a several disances (é èšêbëèšê RèŠê ëèšê ç è ) and he sandard deviain f disance bservains asì2 íèjè, hich is quie accepable. 65

66 ( a ) Indr Case: Cardbard arges ( b ) pimized Case: Rerarges in black backgrund Figure 10. Used arges in eperimens. 5.2 Image measuremens In phgrammeric measuremen sysems, he image pin crrespndence prblem is he ms demanding ask frm he image measuremen pin f vie. In general, he chice, f hich f he pins are be measured is n as impran as h precisely he image pins can be lcaed and h reliably i can be verified ha crrespnding image pin bservains are frm he same bjec pin. I is n sricly rue say ha he chice as hich pins are measured uld have n imprance. The disribuin f image pins n an image des affec he accuracy f blck parameer esimaes. Bu ha is mean here is ha pins measured in his sage d n have have any relevance frm he bjec mdelling pin f vie. The nly purpse f pin measuremens is be able deermine he camera pse ih respec he crdinae sysem. Alhugh, in hese ess, he aim as cmpare daa ses a he same bjec pin lcain, and herefre he bjec measuremens ere carried u ih argeed pins. Hever, a fe image bservains ere made n unargeed naural pins as ell. S, in selecing an image measuremen sraegy, ne gd chice culd be feaurebased image maching. In a feaure-based sluin, he prminen ell-defined image pin se is measured aumaically. Eracin and selecin f image pins n images is based n lcal disinc prperies f he gray-level funcin. The ms significan algrihms, s-called ineres perars, fr eracing image pins, have been develped by ravec, Harris and Försner (Försner and Gülch 1987; Försner 1986; ikhail e al. 2001). The crrespndence f image pins n her images can ne be deermined by eamining he feaure prperies. The decisin n he crrec mach can be based purely n gemerical reasning and 66

67 he feaure prperies hemselves. In addiin, he feaure prperies f ineres pins in he viciniy can be aken in accun in decisin making. There is a grup f sraegies fll in rder slve he crrespndence prblem in a rbus manner. If sme priri knledge f imaging gemery is available, his can be aken in accun as ell. In his research, he iniial apprimaes f camera pse can easily be prduced due develped imaging sraegy; applying he feaure-based mehd fr image pin measuremens is herefre sensible. As menined previusly in his secin, he bjecive as be able measure he same bjec pins bh in a gedeic and phgrammeric manner in rder cmpare he daa ses. This as he main argumen fr using area-based image maching apprach. This ay e can have cnrl ver bjec pin measuremens. Als, he precisin f individual image bservains based n he areabased mehd has been discvered be beer han in he feaure-based mehd (ikhail e al. 2001). Area-based maching, smeimes called emplae maching, is a mehd fr searching a similar image pach (emplae) f he surce image n he arge image. The emplae image is a small image pach eraced in a paricular lcain frm a surce image. The surce image is an image hse crrespnding image pins ill be searched n anher image - here designaed as he arge image. The mehd des n necessarily require human ineracin. The area-based maching mehd can be applied in cmbinain ih he feaure-based apprach by firs selecing he suiable image paches n he surce image ih he help f an ineres perar and hen applying he emplae maching a he lcain indicaed by he crrespndence mach. Bu nce again, since he paricular bjec pins are he subjec f ineres, human ineracin is required pin u he equivalen image pin a leas n ne image Sraegy f image measuremens The flled prcedure in image bservain acquisiin can be cnsidered be semiaumaic. The gdness f iniial values f blck parameers ere eplied in he aumain f image measuremens. The bjec pin as firs measured frm ne image f he blck by he human perar, and afer ha he same image pin as used as a emplae fr maching n an image f he secnd blck, as indicaed by Figure (11). The idea as ha he camera pse and rienain ere apprimaely knn and he bjec pin crdinaes culd easily be calculaed frm hese image pin bservains. This carse apprimain f bjec pin lcain culd subsequenly be back-prjeced n her images f bh image blcks; see Figure (12). The back-prjecin f he bjec pin indicaed he image area here he crrespnding image pin uld be lcaed n her images. H clsely he back-prjecin ill succeed in epsing he real lcain f crrespnding image pins is heavily dependen upn blck parameer apprimains. This prcedure as flled unil all preferred bjec pins ere measured. 67

68 Figure 11. Crrespndence f image pins (shn by he perar) beeen image frm Blck I and II. Figure 12. Back-prjecin f an bjec pin n images f ne image blck. The selecin f he image pin n he firs image as accmplished manually. The bjec pin culd be measured ih sub-piel accuracy by uilizing image magnificain. Ne, an image pach as eraced frm he image and he em- 68

69 plae as generaed frm ha image pach, s ha he measured image pin lcaed eacly a he cenre f he emplae. In emplae generain, he bilinear inerplain mehd as used derive ne gray-level values fr he ne piel lcains in all hree channels (red, green, blue). The size f he emplae migh als be changed. A disadvanage f he area-based maching mehd is ha i cann lerae big perspecive disrins beeen mached images. If he bjec surface is a fla surface, he disrin can be cnsidered as a 2D ransfrmain prblem. An assumpin f a fla bjec surface can be made nly if small image emplaes are used. In general, a smh bjec surface can be apprimaed by number f small fla paches. Since he used arges ere fla, n vilain ha resricin f he mehd as made. Als, pssible disrin culd be diminished by selecing a arge image n anher blck hse direcin f vie as clse he surce image. The reasn hy he arge image as chsen frm anher image blck, and n amng cnsecuive images n he same image blck in his firs sage, as due beer imaging gemery. The image pair chsen is clse he sere case; he larger he base--disance rai, he beer he accuracy in pin deerminain. S geing beer apprimaes fr bjec pin crdinaes helps us acquire he res f he crrespnding image pin bservains mre reliably. Fr emplae maching, he Leas Squares-mehd (LSQ) as applied. In LSQ maching, he gray-level funcin f he emplae is mached agains arge image gray-levels a a given lcain. The emplae is mved n he arge image unil he sum f leas squares f gray-level value differences is minimized. This gives he lcain f he crrespnding image pin f he emplae sub-piel accuracy n he arge image. Since he maching prblem is urned nnlinear frm, he sluin can nly be achieved ih he help f an ierain prcess. Hever, he maching prcess needs gd iniial values fr ierain and he perar herefre has give he saring pin ihin a fe piels frm he crrec lcain. Fr he res f he image pin crrespndences, he calculaed apprimae lcain f he bjec pin culd be used indicae he iniial lcains by means f back-prjecin. Hever, hese iniial lcains culd n be regarded as accurae enugh fr applying he LSQ-maching direcly. A beer sar-pin fr ierain as acquired, herefre, by use f image crrelain. The crss-crrelain f emplae image and arge image as cmpued fr every lcain in he search area. The back-prjecin sraegy rked a his sage in favr f reducing he size f he search area n he arge image, as illusraed in Figure (12). If n priri knledge had been available, he crss-crrelain f emplae and arge image uld have had be cmpued a every piel lcain f he arge image. This ay, he amun f cmpuain as reduced and he search area culd be se arund he back-prjeced image pin. The size f he search space culd be changed a he perars discrein. The calculain f he crrelain 69

70 ð cefficien based n he equain f he nrmalized crss-crrelain cefficien is given (Schenk 1999): î2ï ñò óöõ- üvý and ñòôóöõø ñùúóöõrûübý[ûþú übý û ü ûþú ü ð ñù5óöõrû übý[ûþ5 ü ý ñòôóöõ- ñùúóöõrûü ûþú ü ð ü In Equain (38) are mean gray-level values in he emplae and maching ind, respecively. The value f he crrelain cefficien î ill lie in he range f î. n ccasins hen he back-prjeced pin as lcaed near he edge f he image, here as dub if he bjec pin as u f he field f vie. In rder vercme his, he hreshld value as se fr he maimum crrelain cefficien based n empirical bservains. The maimum crrelain cefficien value ihin he search ind had be eceeded befre prceeding he final measuring sage. herise he cnclusin as ha ha pin had been u f sigh, and he search fr he cnjugae image pin as haled. If he hreshld fr crrelain cefficien as eceeded, he piel lcain ha had he larges crrelain value as hen used as a sar pin fr he final LSQ-maching sub-piel accuracy. The emplae maching as run sequenially in such a ay ha he emplae as eraced alays n he previus image and as nly used fr a subsequen image in a blck. S afer every successful mach, a ne emplae as eraced in ha lcain and i as nly used in maching n he ne image. This ay, he effec f perspecive disrin culd be minimized in he maching prcedure. The LSQ-maching can be cnsidered as û< a! -dimensinal signal fiing prcess. In he leas squares cne, he is a gray-level bservain frm a emplae image in lcain û<! and übû<! is a ransfrmain funcin fr he arge image, hich minimizes he gray-level differences beeen images. The bservain equain in he leas squares sense is hen û"<#$% '& û<! ï ü û"<! (38) (39) The cmpnen & û< $ in Equain (39) describes he randm effec (nise) in bh images. This is an LSQ-image maching idea, hich as presened in he early 1980s. The esimain mdel has been presened bh by Grün (Grün and Balsavias 1985; Grün 1996) and in parallel Grüns invesigain by Prf. Ackermann and his research grup a he Universiy f Sugar (Ackermann 1984). 70

71 ? 3? 3? Transfrmain is suppsed recify he arge image pach in a emplae image piel crdinae sysem. If he surce image and arge image are aken subsanially frm differen vie angles, and he bjec is evidenly hree dimensinal, he recificain uld n saisfy he perspecive disrin beeen images. Hever, by assuming he bjec surface is smh and reaing nly a small pach f he image in maching, even he flling affine ransfrmain can be cnsidered be adequae fr he ask (Ackermann 1984): (*),+-/. 01-!2)43506-/ :+<;. 01;=2)43%01; 7 83 (40) Since i is a case f separae phgraphs aken ih differen epsures, he radimeric difference beeen images als has be aken in accun. In his research, as in he riginal research f Ackermann (Ackermann 1984), linear cmpnens f radimeric ransfrmain ere cnsidered be adequae, as given by: >@? )BA 8!C%D'E )BA 8$CF+HG )BA#8$CI0'JLK0G )BA 8!CJN (41) The linear radimeric ransfrmain included he shif JPK and he scale facr JQN as depiced in Equain (41). The LSQ-image maching algrihm as implemened ih a simple image shif perain als. Insead f esimaing all affine parameers, nly he displacemens in bh ) - and 8 -direcins ere included. This kind f simplificain is jusified if n perspecive disrin is epeced. Als, his sluin des n all any significan image rain beeen images. Neverheless, a simplified versin f he LSQ-image maching mehd as used successfully, since he used arges ere he shape f a circle and herefre invarian fr image rain. Als, he scale difference and effec f vieing angle ere cnsidered be small enugh be disregarded. The reasn fr finally using a reduced parameer se is parly because f he knn risk f verparameerizain in he esimain prcess. In sme cases, he arge size as as small as 99 piels, hich can be cnsidered be small an image area reliably slve all si affine and radimeric parameers. Hever, he affine mdel as als used hen he pserir esimae f sandard deviain S. R f gray-level bservain as deeced be large fr he use f shif parameers nly. Als, he chice f esimain mdel as based n visual inspecin made by he perar. Fr bserving all cnjugae image pins f a single bjec pin, he nly acins required frm he perar ere measure he bjec pin frm ne image and pin u is equivalen lcain n an image f he her blck, as indicaed in Figure (11). The res f he hmlgus image pins ere fund aumaically 71

72 and heir relain each her and he bjec pin culd be sred fr furher cmpuain. All image measuring perains and calculains ere implemened in a cmpuer prgram rien by he auhr f his hesis. A he ime, he research rk as sared, here as n sfare n he marke ha culd handle he image sequence and measure hmlgus pins ih sub-piel accuracy; hse ha ere available ere n fleible enugh be alered fr he special needs f his measuring mehd. The implemenain as carried u in he Linu Red- Ha 9.0 plafrm ih kernel. The prgramming as accmplished in C++ prgramming language and, fr sme graphical perains and user inerfaces, Q library funcins ere uilized. Als, sme mari perains in he esimain par ere prgrammed by epliing he library funcins f he mahemaical prgramming library package, LAPACK (Lapack 2005). 5.3 Real rld eperimens The real rld eperimens ere made in differen seups, ne f hich included fairly pimized cndiins, hile he her included mre realisic measuring cndiins. The eperimen accmplished in pimized cndiins ill be designaed frm n n as Eperimen I and he ne made in he indr envirnmen as Eperimen II. Fr bh eperimens, seleced bjec pins ere argeed and reference measuremens made ih gedeic echniques fr he purpse f cmparisn as depiced in Secin Eperimen I In his eperimen measuring cndiins ere designed be as pimal as pssible, i.e., pimized regarding bh phgrammeric measuremens and he gedeic reference sysem. The recangular rer-arges ere cvered ih black adhesive paper ih a rund hle in he middle, as depiced earlier in Secin and Figure (10b). All arges ere faced cenre parly bain an pimal rienain fr achemeer measuremens and parly prvide favrable cndiins fr he seleced image maching mehd. The disribuin f pins as designed s ha here uld be enugh pins ihin differen bjec disances. Special aenin as paid bserve he effec f bjec disance n pin accuracy since he accuracy is independen f he direcin he bjec pin, due symmeric imaging. Targes ere aached he measuring ples a differen heighs, as shn in Figure (13). The reasn as ge adequae pin disribuin in he verical direcin als. This is essenial in rder deermine camera rienain reliably. 72

73 Figure 13. Disribuin f arge pins aached he measuring ples in Eperimen I. 73

74 W The camera seup as aached he same ripd ha as used in he achemeer measuremens. This ay, he rigin f bh crdinae sysems culd be apprimaely cenred n he same pin. I has be emphasized here ha gedeically measured crdinaes ere never used as a cnrl daum, bu nly fr he purpse f cmparisn. The rienain f bh lcal crdinae sysems as fied by using he same bjec pin in rder define he reference direcin fr plane crdinae aes. Figure 14. Camera seup in Eperimen I. The navel cenre as equipped ih a cenring device ha had a bubble leveling cmpensar. Leveling f he imaging sysem as n necessary by definiin, bu i gave feedback as h sable he sysem as during he imaging sequence. The camera used in he eperimen as an lympus E-10 ( pi, c=2350pi) digial sill camera and his as aached he end f he meal bar TVUXWYZY frm he cenre, as shn in Figure (14). The her end f he bar as supplied ih an adequae cunerbalance in rder mainain sabiliy during imaging. Images in sequence ere aken ih equiangular seps. The plane angle beeen subsequen images as []\X^ resuling in _ images per image blck. The camera as riggered ih ireless reme cnrl and he fcus se infiniy; he aperure as als predeermined accrding seup values used in a previus camera calibrain. The epsure ime as alled be deermined by he aumaic funcin f he camera. Fr he secnd blck, he camera as urned UXW arund [ ^ and imaging as sared a he same sar pin as he firs image sequence. Naurally, he real lcain f he prjecin cenre inside he camera culd n be precisely deermined a his sage. S he minr plane rain difference beeen image blcks had be acceped. The required iniial values fr blck parameers culd be bained by using simple insrumenain. The radius f rain culd be evaluaed by use f a 74

75 b f ape measure and, fr plane rain angle readu a graduaed scale aached he cenring device culd be used. The angle f pical ais ih respec he suppring bar as fied he perpendicular alignmen ih he help f a simple l. There ere, in al, `Va arges used in he eperimen and he disance range f arges as frm b9c up bxb9c. The disribuin f pins as designed in such a ay ha he arge disances ere disribued evenly. A leas arges ere aached each measuring ple. The measuring ples ere divided acrss he arge field s ha arges culd be used as ie pins in he mensurain f image blcks. The scale bar used fr geing scale fr measuremens as se up a he disance f d9c frm he rigin. The nminal value f he calibraed disance as smeha under b meers. The iniial values fr unknn parameers ere acquired quie easily. The lengh f he bar as measured ih he accuracy f a cenimeer and rain an accuracy f ne degree. In his eperimen, he camera as fied perpendicularly a end f he bar. Wih he help f a simple l, his culd be accmplished ihin elfhg. The lengh f he scalebar as added in he esimain as a cnsrain, he lengh having been verified befrehand in labrary cndiins Eperimen II The secnd eperimen as cnduced in an indr envirnmen. The arge area as an enrance hall, here he maimum disance inside he area as apprimaely `Xi meers. Cndiins ere n as pimal as in Eperimen I. Cmprmises had be made as regards he phgrammeric measuring sysem and acquisiin f gedeic reference daa. Targes used in he eperimen ere aached alls and surfaces f clumns inside he es area. This ime, rer-arges ere n used, since hey culd n be measured reliably ih a achemeer. In hese circumsances, i as knn ha difficulies uld ccur ih disance measuremens. When using rers, i is required ha he inciden angle f he measuring ray shuld be equal r near he nrmal he arge surface receive an impulse f a refleced ray f adequae size; his cann be guaraneed in indr space. Therefre, a sandard field prism as used fr reference measuremens and he arges used ere prined arges made f cardbard. Three differen sizes f arges ere used in Eperimen II fr differen bjec disances. A mre deailed descripin as presened in Secin and in Figure (10a). In his eperimen, he fcus as se n mre rbus imaging sysem develpmen. The plane rain is a hard cndiin be fulfilled. Human inervenin in he rain f he camera frm ne epsure he ne, and riggering he camera, can cause unaned discrepancy f suppsed camera rienain. Sme abnrmaliy in he heigh f he prjecin cenres frm he plane f rain during imaging can easily ccur, if sufficien precauins are n aken. 75

76 In rder fulfill requiremens and receive beer iniial values, a ball-bearing ype f rain sysem as designed and assembled. Beer rain cnrl as achieved by supplying he sysem ih a rm gear and a sep mr, Figure (15a). The sep mr as cnrlled by he cmpuer rae he camera ih equal-angled seps, Figure (15b). The camera as als riggered aumaically under cmpuer cnrl. In rder bain her required iniial values, he same prcedure as in Eperimen I as flled. ( a ) Sep mr fied in rm gear. ( b ) Cmpuer cnlled imaging sysem. Figure 15. Sep mr driven imaging sysem This ype f sysem design prvided fully-aumaic imaging ihu human inervenin. The camera as he same as ha used in Eperimen I and he same seup f camera parameers as used in he earlier camera calibrain as adped. The eperimen as run in inerir space in an enrance hall, hich cnsised f a crridr, rund clumns and a saircase, as seen in Figure (16). S he cndiins ere represenaive f hse ne migh mee hen carrying u a ypical measuremen ask in inerir space: blind angles, small angles beeen all surfaces and vieing angle plus varying illuminain. The imaging as made frm he same sp as he achemeer measuremens. The rain cenre f imaging differed nly a fe millimeers frm he achemeer crdinae sysem. This as verified aferards ih a crdinae sysem ransfrmain. Imaging as accmplished ih a cmpuer cnrlled sysem ih jvk phs per image blck, he camera ih a perpendicular vieing angle being aached he suppring bar apprimaely l/mnz frm he rain cenre. The camera seings ere fied he same fcus and aperure values as used in camera calibrain. The epsure ime as alled be deermined by he aumaic funcin f he camera. The imaging as carried u 76

77 u Figure 16. The enrance hall here he eperimen k place. under arificial illuminain cndiins and here as n cnrl ver h he flurescen lamps impaced upn image qualiy in any individual sh. Fr his reasn, sme images ere ver-epsured and sme under-epsured. Neverheless, arges ere able be measured n images reasnably ell. The scale bar ih a lengh f prqps prvided he scale fr he 3-D measuremens and as se a a disance f Xs frm he cenre f measuremens. 5.4 Camera calibrain The camera used in bh eperimens as an lympus E-10, a ypical nn-prfessinal digial sill-camera. Accrding he manufacurer, he camera has a ulvxxzy qxqp piel resluin ih a {V PsZs~} szs zm lens. The camera as used in eperimens nly ih fcal lenghs in ide-angle mde ih a singleaperure sp value. The camera as herefre calibraed nly ih hse seings. In camera calibrain, linear sensr defrmains and nnlinear lens disrin crrecins ere calculaed accrding he disrin mdel presened in Secin 3.4 in Equains (17) and (20). Fr he calibrain, a hree dimensinal calibrain field as phgraphed frm five imaging sains, see Figure (17). In each imaging sain, hree images ere aken, having apprimaely rll beeen epsures. S, all geher u images ere aken and arund argeed bjec pins ere measured. The esimain f unknn camera calibrain parameers as carried u via free-ne adjusmen ih a single disance measure. In free-ne adjusmen, inner cnsrains ere applied. 77

78 Figure 17. Three-dimensinal calibrain field used in calibrain. In bh eperimens, he reslved calibrain parameers ere kep fied in he blck cmpuain and nly blck parameers and bjec crdinae values ere esimaed. In he laer analysis, i as suspeced ha radial disrin parameers ere n deermined ih adequae accuracy afer all. Therefre, in addiin, a plumb-line calibrain as accmplished in rder bain beer esimaes f he amun f disrin f he lens. The prblem ih camera calibrain carried u ih he calibrain field is he high crrelain f decenring lens disrin parameers and he principal pin, especially ih Lƒ -cmpnen and. Hever, his prblem can be vercme by slving he lens disrin parameers, ih, fr eample, plumb-line calibrain and he res f he calibrain parameers ih es field calibrain. Fr he plumb-line calibrain, ]ˆ plumb-lines ere se hang frm he ceiling in frn f a dark backgrund, see Figure (18). The plumb lines ere made f a heavy fishing line, having small eighs ied a he her end and leing hem hang freely. Three images ere aken a a single psiin ih Š rll beeen epsures. The disance plumb lines as nly meers, bu he camera fcus as se infiniy; her seings ere he same as ih earlier calibrain. The prcedure flled in plumb line calibrain as similar ha repred by Fryer (Fryer e al. 1994). The cenre f he lines ere bserved frm images by means f cenrid cmpuain f gray-levels in he ransversal direcin. Samples f he cenrid lcains ere bserved a evenly separaed pins alng he line. 78

79 Ÿ š š Figure 18. Used plumb-lines in camera calibrain. In cmpuain f camera calibrain, nly he firs cefficiens f radial disrin parameers Œ!ŽŒX and decenring disrin parameers IŽX ere esimaed. In addiin, he line parameers % and V had be esimaed. The sraigh line equain in a parameric presenain is given as š ]œ=ž 5/Ÿ' Lž B! ' V (42) In Equain (42), š and are crreced image bservains accrding he disrin mdel. The lens disrin is included via erms and V as flls: š š š/ Ÿ6 Ÿ6 Ÿ1 V (43) The Equain (43) presens he crreced image bservains. I is be ned ha he adjusmen mdel differs frm ha depiced in Equain (1) since he bjec funcin is in implici frm. This means ha Equain (42) has be differeniaed ih respec bservains als: ªZ«Ÿ6 Ÿ6 ± š, ³ (44) 79

80 In cmpuain f ne calibrain parameers, he crrecins ere made disrin parameers and line parameers afer every ierain rund, unil he crrecins ere small enugh be negleced. Afer his, ne values ere applied in he calibrain mdel and a ne esimain f her calibrain parameers ere cmpued, hile lens disrin parameers ere se fied ih bservains made n images f a hree-dimensinal calibrain field, Figure (17). This as repeaed unil n crrecins ere required fr any calibrain parameers. The reasn fr his kind f prcedure as ha i is n pssible slve all calibrain parameers ih plumb line calibrain and he change f lens disrin parameers have an effec n her calibrain parameers and vice versa. 80

81 ¼½ 6 RESULTS AND ANALYSIS 6.1 Refinemen f he mahemaical mdel The mahemaical mdel presened in Secin 3.6 as flled in he cmpuains. As menined earlier, ver-deerminain as eplied in measuremens, parly imprve he precisin f esimaes and parly be able evaluae he qualiy f measuremens aferards. The esimain as based n linear leas squares, hich guaraneed he pimal sluin ih respec he bservain errr. The cllineariy cndiin Equain (31) is nn-linear ih respec he parameer se based n plar crdinaes and is dependen n rain angles. S, slving he unknn parameer values leads an ieraive sluin and he need fr iniial parameer values. Iniial values can be acquired in a relaively easy manner, as eplained in Secin 5.3. Cmpuain cnverged nicely afer a fe ierains. In earlier eperimens, here iniial values ere n accurae enugh, he cmpuain mdel had be alered sme een. The parameer se had be cmpued in grups in rder assure he cnvergence f cmpuain. Firs, nly he plane rain angles µb ere se free, hile he her parameer values ere kep fied. Ierain as cninued unil he cnvergence as reached. Afer his, he cmmn camera rain angles Q¹º4 Q¹»$ ere als se free ne by ne and, in he final sage, radii values f bh image blcks ere se free as ell, and a cmmn adjusmen as cmpued. The magniude f he maimum residual as fund be larger han epeced. I as fund be large hen cmpared hrugh eperience r by means f equivalen insrumenain her clse-range measuremens; his as especially he case hen he fac ha argeed pins ere used as aken in accun. The pserir esimae f he sandard deviain f uni eigh bservain culd be cmpued frm he residual values as ¾ÁÀ ÂXÃÄ ÅZÆÇ (45) This esimae says a l abu he finess f he measuremens ih he chsen mdel. In his case, he sandard deviain f uni eigh as ver ne piel, s i as eviden ha eiher he mdel as incrrec r ha here ere grss errrs amng he bservains. When residuals ere eamined gemerically, he verical alignmen f residual vecrs culd be niced direcly. This clear sysemaic paern indicaed ha eiher camera sensr plus lens calibrain ere n crrec r he physical mdel 81

82 presened ih he chsen parameer se, Equain (31), culd n depic he acual imaging case. Since calibrain as carried u accrding sandard mehds and camera calibrain parameers ere kep fied during cmpuain, he cnclusin as ha he used mahemaical mdel culd n enirely eplain he physical mdel. In rder verify he bserved shrcming f he mahemaical mdel, he phgrammerically acquired 3D pin crdinaes ere ransfrmed in he same crdinae sysem as he achemeer measuremens fr cmparisn. Fr ransfrmain, a rigid bdy 3D ransfrmain ih hree rains and ranslains as used. When cmparing he crdinaes, i as immediaely niced ha here ere big differences n crdinaes f he pins a greaes disance frm he camera. The resricins se fr imaging required ha all prjecin cenres shuld lie n he same plane and he rienain f he camera ih respec he suppring bar and rain plane shuld remain saic during imaging. If he requiremen f prjecin cenres lying n he same plane as vilaed, he effec f his discrepancy shuld be visible n pins ih shrer disances als. Because he difference in bjec pin crdinaes ih respec reference daa as much larger ih pins far aay, i as lgically deduced ha he reasn fr his discrepancy as mre likely due unepeced rienain angle values han simple verical ranslain. dificain in he mahemaical mdel as carried u in rder ake in accun pssible variain f he plane f rain f he camera arund he navel pin. The variain as suspeced be due ransversal il f he suppring bar ihin he imaging sessin. Figure (19) describes he epeced alerain f camera heigh frm nminal rain plane during imaging due iling f he bar. È=É Figure 19. Pssible ransversal il f bar during imaging. An addiinal parameer È as added he mahemaical mdel describe he heigh difference frm he nminal plane f rain n each camera pse, see Figure (19). The number f parameers as increased by Ê@ËÍÌ frm he riginal se f parameers in Equains (27) and (28). nly he firs camera f he firs blck as suppsed have a fied value ÈzÎÐ defining he nminal plane. The al 82

83 â Ù Û number f parameers as nearly dubled, bu sill he sysem as subsanially verdeermined. I is essenial nice ha his gemerical inerpreain affecs bh prjecin cenre crdinae values as ell as rain mari cnsrucin, as is depiced bel: ÒÔÓÖÕØ Ù Ù Ù ÚÜÛLÝÞXß4àB 4ÛLÝÞXß4= ÚÜÛLßã å= ÚÜÛLßã åà 4ÛLÝ=ÞXß4= (46) The change made in cmpuain f he rain mari fr each individual camera pse culd be derived frm he cmmn parameer angles f he blck ih muliplicain f rain marices: çéè]êë ìêíë îê çéè]ï#ë ìðïë îï çñlê"ë òóê (47) This refinemen f he mdel, presened in Equains (46) and (47), reduced he sandard errr f uni eigh ôõ4ö frm Xøúù9ûVü piels ü!øþýx piels in he cmmn adjusmen. The effec f inrducing a ne parameer in he mdel can be illusraed in he heigh variain f he prjecin cenre frm he nminal plane during he imaging, hich is shn in Figure (20). The flucuain inside ne image blck is quie small, bu a clear shif can be niced beeen image blcks. This shif f culd have happened hen he camera as urned in he ppsie direcin a he end f he bar beeen he firs and secnd image sequence. Similar flucuain f an array sensr frm he nminal plane rain during imaging has als been repred in invesigains f panrama-cameras (Parian and Gruen 2004) Blck I Blck II Heigh(m) α angle (Deg) Figure 20. Heigh variain f prjecin cenre during imaging. 83

84 Afer his mdel refinemen, he maimum lengh f he residual vecr as sill ver fur piels and residuals in -direcin ere sill slighly bigger han in - direcin. Als, he cmparisn reference shed ha he 3D pins ere sysemaically farher aay han pins measured by a achemeer. The cnclusin dran frm his discrepancy as ha he camera mus have been iled in he direcin f he pical ais during imaging as ell. Figure (21) demnsraes his suspeced phenmenn. Figure 21. Pssible il f he camera in direcin f pical sysem. The iling can be cnsidered as a verical plane rain. Sme verical and hriznal displacemen f a prjecin cenre may have ccurred as ell as a cnsequence f his il. This, hever, can be negleced, since he maimum value f he il as less han and he maimum effec f he il as less han a millimeer in he prjecin cenre crdinae value. S, he effec f he il can be appended as an addiinal angle in he calculain f he rain mari:!#" $% &' (48) Wih his las addiin, given in Equain (48), he sysemaic paern n residuals seemed have disappeared, a leas hen visually eamined. Als, he sandard deviain f uni eigh )+* ( reduced,.-0/ 1 piels, hich is quie ypical in clserange phgrammeric cases and crrelaes ih earlier eperimens. The refined mahemaical mdel defined in Equains (46) and (48) as adped fr all furher cmpuains repred in his hesis. The cllineariy cndiin hen changed in he frm: ;:=<>?>A@CBED?FHGIKJ.LNP.Q'R LNSAQT'UV<>XW@CYZD'FHGIKJ[R ]\0^KSAQ_T?UV<>]`@abD?FHGcUdJ.'\e^fP.Q'R LNSAQgT < D?FHG IKJ.LNP Q R LNS Q T'UV< D'FHG IKJ[R ]\0^KS Q T?UV< D?FHG UdJ.'\e^fP Q R LNS Q T hi7 9;: < D?FHG IKJ.LNP Q R LNS Q T?UV< D'FHG IKJ[R ]\0^KS Q T?UV< D?FHG UdJ.'\e^fP Q R LjkS Q T <[`%>A@CBED?FHGIKJ.LNP.Q'R LNSAQgT?UV<[`'W@CYZD'FHGIKJ[R ]\0^KSAQ_T?UV<k`?`@abD?FHGcUdJ.'\e^fP.QXR LjkSAQ_T (49) here erms l nm dene he mari elemens f rain mari # in Equain (48). The linearizain f he cllineariy cndiin, Equain (49), ih respec he ne unknns is presened in Appendi II. 84

85 ne issue ye be deal ih is he eccenriciy f he rain sysem. In a panramic imaging sysem, he eccenriciy f he rain has be deermined r i has be minimized in rder achieve he gemery f ideal panramic imaging. In his research, he crdinae sysem is creaed n sie, defining ha he -ais mus cnverge ih he cenre f rain (navel, rigin) and he prjecin cenre f he firs camera pse in sequence, hich is n necessarily he direcin f he suppring bar. The rain is defined ih respec his crdinae sysem by definiin. Since he bar-camera mun is assumed be a rigid cnsrucin, he nly surce f eccenriciy can be he ball bearing f he rain r he arbirary mvemen f he hle imaging sysem during he imaging sequence. The pssible ani-symmeric rain caused by a ball bearing sysem can be cnsidered be s small cmpared her surces f errr ha i can be negleced. The pssible arbirary mvemen f he hle sysem during he imaging sequence is quie hard deermine. The siuain is here he same as ih panramic imaging r ih laser scanning; he insabiliy f he imaging plafrm has be deal ih by her means. Als, here he quesin is handled by muning he imaging sysem n a plafrm, fr eample, n a ripd made as sable as pssible, and, in his ay, ensuring ha he pssible vibrain f he hle sysem is negligible. 6.2 Cmparisn reference In rder cmpare phgrammeric daa ih reference daa, a rigid bdy 3D ransfrmain beeen daa ses fr Eperimen I as calculaed. The lengh f pin--pin differences, shn in Figure (22), represens he abslue crdinae difference beeen daa ses including inaccuracies f bh measuring mehds and he crdinae ransfrmain. The pin differences near he rigin are clearly greaer due he unsuiabiliy f a achemeer fr measuring ver shr disances. Whereas he achemeer crdinaes f far-ff pins are mre reliable, he differences beeen daa ses are due mre he limiain f phgrammeric mehds. The secnd rder curve, depiced in Figure (22), as fied he daa se f pin differences. The differences are presened ih respec he nminal disance f pins frm he rigin. I can be cmpared ih he equivalen represenain f simulaed daa in Figure (7), alhugh i has be remembered ha he camera mdel and he imaging cnfigurain ere n enirely equivalen. In bh real-rld eperimens, he radius f he imaging sequence as near p q rs and he epeced bservain sandard deviain as q.vuy+z' crrespnding {ZeZ }s, hich is equivalen he used nise level f q.0qzpy+z'i~{ 0 } s in simulain. The field f vie as he same in bh camera mdels (simulaed - rue). In he pimal case, he siuain can be cnsidered be near he simulaed case in erms f gemery (all arges facing he cenre). Hever, he number f pins in he real-rld eperimen is essenially smaller han in he simulain. 85

86 0.1 Difference in Disance Difference (m) Disance (m) Figure 22. pimal case, Eperimens I Als, i is be ned ha in he simulain he image bservain accuracy is assumed be cnsan regardless f pin disance. In he real-rld case, he image scale des have an effec n he accuracy f bservains, especially hen he size f he arge is he same, despie heir lcain. In he simulaed case, nly randm nise is assumed be fund n bservains, bu, in he real-rld eperimen, i is n pssible guaranee ha all sysemaic errr cmpnens can be enirely cmpensaed in cmpuain. In Figure (22), he crdinae difference beeen daa ses is larger han he sandard deviain f pin crdinaes in he simulaed case in he same disance range ( ƒ ˆ ), cmpare Figure (7). In he simulaed case, he maimum sandard deviain is (nise Š Œ0 Ž+' a disance f, bu, in realrld eperimen, he crdinae difference is arund a his disance. Hever, differences bel ihin he same disance ere recrded. The derived values a he maimum disance (Z ) give he crdinae difference f hich is quie small cmpared bjec disance. In he case f inerir space measuremens, Eperimen II, he effec f imperfecin f reference daa ih shrer disances is eviden, Figure (23). The variain f crdinae differences ihin he same disance says smehing abu he lack f cnfidence in he reference values. Hever, he mean r derived value frm he fied secnd-rder plynmial curve gives an insigh in he epeced accuracy f he measuremens. Cmpared equivalen resuls f he pimized case, Eperimens I, larger variain hrughu he disance range can be seen. 86

87 0.1 Difference in disance Difference (m) Disance (m) Figure 23. The lengh f pin differences respec bjec disance, indr case, Eperimen II. Sme discrepancies can be cnsidered be due he iled angle f he arge surface ih respec he vie angle r inadequae illuminain f he arge area. Insufficien illuminain culd be verified fr he furherms pins. In general, illuminain can be seen as ne f he ms resricing elemens hen rying achieve he ms accurae measuremens in indr envirnmens. When cmparing he indr case ih he pimized case, crdinae differences ih respec reference in he indr case are nly slighly rse han ih he pimized case. Hever, he variain f differences is clearly larger in he indr case R ean Square Difference Since he accuracy f 3D pin measuremens ih his mehd is highly dependen n he pin disance frm he rigin, disinc crdinae differences shuld be bserved fr differen disance ranges. Anher presenain f he bjec crdinae accuracy can be given in he frm f R ean Square Differences (RSD). In Tables (3) and (4), 3D crdinae RSDs are depiced fr fur disance ranges, apprimaely five meers each. Hever, in he firs range caegry, he clses pins are n clser han meers. In Tables (3) and (4), he firs clumn presens he al pin RSD values and he ne hree clumns presen he RSD values in respec each crdinae. The las clumn liss he number f pins falling in each disance range. 87