SINGLE EYE AND CAMERA WITH DEPTH PERCEPTION

Transcription

1 SINGLE EYE AND CAMERA WITH DEPTH PERCEPTION by Bart Farell a, Dawit Negussey b, James Flattery and Philipp Kornreih a - Institute of Sensory Researh,, Syrause University, Syrause, NY 1344 b - Civil and Environmental Engineering Department, Syrause University, Syrause, NY 1344 Eletrial Engineering and Computer Siene Department, Syrause University, Syrause, NY 1344 It is possible that eah light sensor pixel in the eye has the apability of measuring the distane to the part of the objet in fous at the pixel. One an also onstrut an eletroni amera where eah pixel an measure the distane to the portion of the objet in fous at the pixel. That is, these devies have depth pereption It is possible that eah light sensor pixel in the eye has the apability of measuring the distane to the part of the objet in fous at the pixel. The rods and ones whih are atually ell ilia or flagella that at as short lightguides. It is these ilia that failitate the depth pereption. One an also distane Objet L Image Fig. 1. A group of lightwaves of different wavelength will all have the same phase at the image as they had at the objet where the light was emitted. Thus, if Photonis North 009, edited by Réal Vallée, Pro. of SPIE Vol. 7386, 73861D 009 SPIE CCC ode: X/09/$18 doi: / Pro. of SPIE Vol D-1

2 the wavelength and phase of eah omponent of the light is known at the image the distane L to the objet an be determined. onstrut an eletroni amera where eah pixel an measure the distane to the portion of the objet in fous at the pixel, that is, build a amera with depth pereption. The devie funtions as follows: A group of lightwaves of different wavelength as shown in Fig. 1 will all have the same phase at the image as they had at the objet where the light was emitted. Thus, if the wavelength and phase of eah omponent of the light is known at the image the distane L to the objet an be determined. However, some kind of guidane mehanism is required for the lightwaves that were emitted by an objet point to be guided to the image point. This an readily be ahieved with an ordinary onvex lens. Indeed, ameras and animal eyes have been using lenses very suessfully for this purpose. Light takes the same time to traverse any path from an objet point through the lens to an image point. Negleting aberrations of the lens, by adding all the lightwave omponents of different wavelength at an image point one obtains an image if all the lightwave omponents have the same phase as they had at the objet. All this is also true if the light is only partially oherent. Coherene an be illustrated by two light beams originating at the same time from the same objet point that travel over different path length to a single image point. The lightwaves will form an interferene pattern if the differene in the length of the two light path is less than the oherene length of the light. If the differene Mode sensitive Detetors Lightguide Lightwaves of various wavelength from a single objet point arrive with different phases Fig.. A simple apparatus that is both sensitive to the wavelength and phase of the arriving light is a short piee of lightguide. In order to obtain the wavelength and phase information of the light omponents measurements of a number of modes of the lightguide are made.. Pro. of SPIE Vol D-

3 path length is longer than the oherene length of the lightwaves the light will form a more or less uniform bright spot when observed over a length of time. All lasers have some finite oherene length. Even white light has a oherene length of a few wavelengths. If this were not so lenses would not work. The distane from an image point to its objet point an be determined from the phase of a number of omponents of light at different wavelengths even if the light is only oherent for a distane of a few wavelengths. In order to determine the distane from eah point of an image to its orresponding objet point one needs a simple apparatus at eah image point that an measure the phase and wavelength of eah omponent of the light. A short resonant waveguide setion as shown in Fig. with an detetor array at its far end is apable of both deteting wavelength and phase. The detetor array an detet the various modes in the lightguide. The information measured by the light sensors at the end of the lightguide also ontains the brightness of the light radiated by the objet point. To eliminate the brightness information and retain the phase information measurements of at least two lightguide modes are made by an detetor array at the far end of the lightguide. Thus there has to be a light sensor array at the far end of the lightguide apable of distinguishing between the modes in the lightguide. Lens r o r 1 Retina r Cilia Objet a Eye b Fig. 3. Single pixel imaging sensor. Pro. of SPIE Vol D-3

4 The partially oherene of the light is taken in onsideration in this alulation. To aomplish this the orrelation between parts of the light waves has to be performed. Fortunately the light intensity is expressed as the produt of the eletri field and magneti flux density of the light eletromagneti field. Thus a orrelation between the light eletri field and the light magneti flux density an be performed. In the mathematial model, low frequenies orresponding to long wavelength give the largest interation terms with long distanes. Light wavelength are muh smaller than the distanes to be measured. However, again, sine the orrelation between the light eletri field and light magneti flux density is alulated the sum and differene frequenies appear. It is the low differene frequenies orresponding to long wavelengths that give the largest interation terms. First one has to alulate the light eletri field at the input of the lightguide. Sine in this part of the problem all distanes are muh larger than a light wavelength a salar diffration theory model an be used. Also, the paraxial Fresnell approximation is used in this alulation. The salar light eletri field u 1 (r 1 ) at the input plane of the lens is: u 1 (r 1 ) = ω jπa exp j ωa So ds o u o (r o )exp ω a r 1 r o (1.1) where u o (r o ) is the salar light eletri field at the objet and the integral is over the objet plane S o. The salar light eletri field u (r 1 ) at the output plane of the lens is: ωr u (r 1 ) = u 1 (r 1 )exp 1 (1.) f The salar light eletri field u 3 (r ) at the input to the lightguide setion is: u 3 (r ) = ω jπb exp ωn 1 b ωn j ds S1 1 u (r 1 )exp 1 b r r 1 (1.3) where n 1 is the index of refration of the material in the eye and the integral is over the pupil of the eye or amera. By substituting equation 1.1 into equation 1. and the resulting expression into equation 1.3 one obtains: Pro. of SPIE Vol D-4

5 ω ω(a + bn ) 1 u 3 (r ) = 4π exp j ab So ds o S1 ds 1 u 0 (r o )exp ω(r 1 r. ro + r ) 1 o a ωr 1 f + ωn 1 (r 1 r 1. r + r ) ] (1.4) b The image will be in fous on the objet if: 1 a + n 1 b 1 f = 0 (1.5) By substituting equation 1.5 into equation 1.4 one obtains: u 3 (r ) = ω ω(a + n 1 b) 4π exp j ab ds So o u 0 (r o )exp ω j n a + 1 r b r o ds 1 exp j ω S1 a r an. 1 1 r o + b r (1.6) Sine the pupil diameter is muh larger than a light wavelength the integral over S 1, the pupil, an be approximated by a delta funtion. ds 1 exp j ω S1 a r. 1 ro + an 1 b r - 4π a ω δ ro + an 1 b r (1.7) By substituting equation 1.7 into equation 1.6 one obtains: u 3 (r ) = a b exp j ω(a + n 1 b) So ds o u 0 (r o )exp ω j n a + 1 r b r o δ r an 1 o + b r (1.8) By integrating one obtains for the light eletri field at the input to the light guide: Pro. of SPIE Vol D-5

6 u 3 (r ) = a b u an 1 0 b r exp j ω a + n1 b + 1 n 1 a + n 1 r b b (1.9) The amplitude the light eletri field at the entrane to the lightguide and the effetive distane L from the objet to the lightguide entrane is: a) E o a b u an 1 o b r b) L = a + n 1 b + 1 n 1 a + n 1 r b b (1.10) Next, the propagation of the light trough the lightgude is alulated. Here the salar paraxial approximation is no longer valid. In order to inlude the fat that the light has a limited oherene time η the light eletri field vetor in the air in front of the lightguide is formulated as a random proess: The light eletri field entering the light guide is a omponent of the expansion of the light eletri field of equation 1.9 in the modes of the light guide. E 1 = á φ E o exp[ jmφ]s(r) exp j η t z + Rexp j η t + z (1.11) where it was assumed that some light is refleted at the entrane to the lightguide. Here ω is the osillating frequeny of the light, m is an integer, η is a oherene time and θ is uniformly distributed random phase. The probability density p(θ) of the random phase is: p(θ) = 1 π for π < θ π (1.1) For simpliity TE like modes are assumed. The orresponding magneti flax density pseudo vetor omponent in the air between the objet and the light guide an be obtained by substituting the eletri field into the Faraday Maxwell equation. The magneti flax density pseudo vetor has also the form of a random proess. Pro. of SPIE Vol D-6

7 E o B 1 = á r exp [ jmφ]s(r) exp j η t z Rexp j η t + z + jâ z E o exp[ jmφ] 1 ωr Ž Žr [ rs(r)] exp j η t z + Rexp j η t + z (1.13) Here the funtion S(r) of the transverse oordinate r an be of the following form for a fiber like lightguide. S(r) = J m (rq m ) (1.14) where J m (rq m ) is the m th order Bessel funtion. The light eletri field vetor in the ilia light guide is:: E = á φ E o S(r)exp[ jmφ] Fexp j θ η t n(z L) + Gexp j θ η t + n(z L) } (1.15) The orresponding magneti flax density pseudo vetor in the ilia lightguide, similar to equation 1.3 is: E o n B = á r exp [ jmφ]s(r) Fexp j η n(z L) t Gexp j η n(z L) t + } + jâ z E o exp[ jmφ] 1 Ž ωr Žr [ rs(r)] Fexp j η n(z L) t + Gexp j η n(z L) t + } (1.16) where the dimension less quantity n is the mode dependent effetive index of refration of the lightguide. The dispersion relation of the light guide. Pro. of SPIE Vol D-7

8 q m + n ω = ω n (1.17) and where q m is the transverse m dependent mode wave vetor, see equation The light eletri field vetor in the detetor is: E 3 = á φ E o S(r)exp[ jmφ]texp j η t (β jα)(z L b) (1.18) Again, similar to equation 1.13 the orresponding magneti flax density pseudo vetor in the detetor is: E o (β jα) B 3 = á r exp[ jmφ]s(r)texp j η t (β jα)(z L b) + jâ z E o exp[ jmφ] 1 ωr Ž Žr [ rs(r)]texp j θ η t (β jα)(z L b) (1.19) The dispersion relation in the detetor has the following form: q m + ω (β jα) = ω n 3 + jωσμ o (1.0) where σ is the effetive ondutivity at the light frequeny. By subtrating equation 1.17 from equation 1.0 and multiplying by one obtains for the real and imaginary parts: ω a) β n n 3 + n α = 0 b) αβ = σ ωε o (1.1) By substituting equation 1.1b into equation 1.1a for ß and solving the resulting equation for ß one obtains: Pro. of SPIE Vol D-8

9 β = n + n 3 n σ ω ε o ( n + n 3 n ) (1.) An expression for a an be obtained by substituting equation 1. into equation 1.1b. σ α = n + n 3 n ωε o σ ω ε o ( n + n 3 n ) (1.3) BOUNDARY CONDITIONS At z = L the tangential eletri field is ontinuous: exp j η L + Rexp j η L = F + G (1.4) At z = L the perpendiular omponent, the r omponent, of the magneti flux density pseudo vetor is ontinuous: exp j η L Rexp j η L = n(f G) (1.5) It is true that the tangential omponent of the magneti flux density pseudo vetor is also ontinuous at the boundary. However, it yields the same information as the one obtained from the ontinuity of the tangential eletri field, equation 1.3. By adding equations 1.4 and 1.5 and solving for F one obtains: F = n exp j η L 1 + R 1 n exp j η L (1.6) Pro. of SPIE Vol D-9

10 By subtrating equation 1.5 from equation 1.4 and solving for G one obtains: G = n exp j η L 1 + R 1 + n exp j η L (1.7) At z = L + b the tangential omponent of the eletri field vetor is ontinuous; Fexp j η nb + Gexp j η nb = T (1.8) At z = L + b the perpendiular omponent, the r omponent, of the magneti flux density pseudo vetor is ontinuous. n Fexp j η nb Gexp j η nb = (β jα)t (1.9) Adding equations 1.8 and 1.9 and solving for F one obtains: F = 1 1+ β jα Texp j n η nb (1.30) Subtrating equation 1.9 from 1.8 and solving for G one obtains: G = 1 1 β jα Texp j n η nb (1.31) The onstant F an be eliminated by equating equations 1.6 and n exp j η L + R 1 1 n exp j η L = β jα 1 + n Texp j η nb (1.3) The onstant G an be eliminated by equating equations 1.7 and Pro. of SPIE Vol D-10

11 1 1 n exp j η L + R n exp j η L = β jα 1 n Texp j η nb (1.33) This leaves the onstants R and T. One next solves equations 1.3 and 1.3 for R 1 + β jα n 1 1 n Texp j η L nb n 1 1 n exp j η L = R (1.34) ανδ 1 β jα n n Texp j η L + nb 1 1 n n exp j η L = R (1.35) The onstant R an be eliminated by equating equations.34 and.35. By equating equations.4 and.35 and olleting terms one obtains: 1 T 1 + n (n + β jα)exp j η nb 1 1 n (n β + jα)exp j η nb = 4nexp j η L (1.36) By solving for T one obtains: T = 4exp j η L P(ω)exp j η nb Q(ω)exp j η nb (1.37) where: a) P(ω) = n ( n + β jα) b) Q(ω) = 1 1 n (n β + jα) (1.38) Pro. of SPIE Vol D-11

12 and where n, ß, and α are funtions of ω. The light detetor response is proportional to the z omponent of the eletromagneti power density S z or Pointing s vetor evaluated at z = L + b. S z = 1 μ o E φt (z = L + b)b * rt ( z = L + b) (1.39) where E φt is the φ omponent of the total eletri field, the eletri field vetor omponent integrated over the bandwidth of the impinging light and B rt is the omponent of the total magneti flux density pseudo vetor, the magneti flux density pseudo vetor omponent integrated over the bandwidth of the impinging light a) E φt = 1 o ω o E. âφ 3 dω b) B rt = 1 o ω o B 3. âr dω (1.40) By substituting equations 1.18, 1.19 into equations 1.40 and substituting the resulting expression and equation 1.37 into equation 1.39 one obtains. S z = E o z o 16 B w ω o ω o + dω ω o + ω o dω(β + jα)exp j (Ω ω) L P(ω)P *(Ω)exp j (ω Ω) nb + Q(ω)Q * (Ω))exp j (ω Ω) nb Q(ω)P * (Ω)exp j Ω + η nb P(ω)Q * (Ω)exp j Ω + η nb } 1 (1.41) where ω o is the enter frequeny and is the bandwidth. Here P and Q are funtions of ω and P* and Q* are funtions of Ω. Note that; z o = μ o (1.4) One an make the following transformation of variables that denote the sum and differene frequenies: Pro. of SPIE Vol D-1

13 a) ξ = ω Ω b) Ξ = Ω (1.43) By inverting equations 1.40 one obtains: a) ω = ξ + Ξ b) Ω = Ω ω (1.44) By substituting equations 1.43 into equation 1.41 one obtains: S z = E o 16 z o B w dξ B o w dξ (β + jα)exp j ξl ω B P (ω)p * (Ω)exp j ξnb + o w Q(ω)Q * (Ω))exp ξnb j Q(ω)P * (Ω)exp j Ξ + η nb P(ω)Q * (Ω)exp j Ξ + η nb } 1 (1.45) P and Q are funtions of ω and P* and Q* are funtions of Ω. However the parameters n, ß and α do not hange muh with frequeny ω. Thus one an approximate n, ß, α, P and Q by onstants. The bandwidth ontains a very large number of yles of the frequeny ξ. Therefore the integral over ξ an be assumed to go from minus to plus infinity. In order to perform the integral over ξ the following transformation of variables is made: a) z = exp j ξnb b) dξ = j dz nb z (1.46) sine the exponential is a onstant z is a omplex variable. Note that the magnitude of z is onstant and equal to one. Thus equation 1.45 an be rewritten as follows: S z = E o z o where ω o + 16 nb dξ β + jα ω B o w PP * Unit irle z L nb dz z U(Ξ,θ)z + QQ* PP * (1.47) Pro. of SPIE Vol D-13

14 1 a) U(Ξ,θ) = PP * * PQ exp j Ξ + nb η + QP * exp j Ξ + nb η b) U(Ξ,θ) = PP * 1 1 n (n β α )os Ξ + η nb + αnsin Ξ + θ η nb (1.48) For an effetive objet distane L equal to 1 m, a 10 µm long lightguide with an effetive index of refration n of , L is approximately equal to The denominator of equation nb 1.47 an be expended as follows: S z = E o z o ω o + 16 nb dξ β + jα ω B o w PP * Unit irle z L nb dz (z z 1 )(z z ) (1.49) where: : a) z 1 = U(Ξ,θ) U (Ξ,θ) QQ* PP * and a) z = U(Ξ,θ) + U (Ξ,θ) QQ * PP * (1.50) Estimated values of z 1 and z an be alulated from equation They are z 1 is approximately equal to and z is approximately equal to x Sine both z 1 and z are less than 1 in magnitude equation 1.49 an be integrated by using the method of residues: B S z = j E o w o 3π z o nb dξ β + jα ω B o w PP * z L nb 1 z L nb z 1 z (1.51) Pro. of SPIE Vol D-14

15 10 9 Ratio of amplitudes of first and zero order mode Distane in m Fig. 4. Plot of the ratio of the amplitude of the zero-order and first-order modes as a funtion of the distane to the objet point that is in fous at the partiular pixel. Spikes are quantization noise in the alulation. Both z 1 and z are funtions of the random phase θ. Thus the average value of Poynting's vetor over the random phase θ has to be alulated next. <S z > = j E o z o 16 nb ω o + dξ ω B o w π dθ β + jα π PP * z L nb 1 z L nb z 1 z (1.5) The integral over θ has to be evaluated numerially. Pro. of SPIE Vol D-15

16 In order to eliminate the amplitude term E and only retain the phase information the light z o power density <S z > or Poynting's vetor is measured for two different modes. The ratio of these two measurements is alulated. The amplitude term anels in the ratio of the amplitudes of the two modes. The following parameters where used in the alulation: n = 1.33 n 3 = 3.5 σ = J 0 (aq 1 ) = 0 at aq 1 = J 1 (aq 1 ) = 0 at aq 1 = where a =.5 µm is the radius of the lightguide setion. A 10 µm long light guide was used. A plot of the of the ratio of the amplitudes of the zero order and first order modes is shown in Fig. 4. Note that the amplitude ratio inreases with the distane from the amera or eye pixel to the point on the objet that is in fous at the pixel. APPENDIX A The url in ylindrial oordinates is: F = â 1 r r ŽF z Žϕ ŽF φ Žz + â ŽF r φ Žz ŽF z Žr + â z 1 r Ž Žr ( rf φ ) 1 r ŽF r Žφ (A.1) REFERENCES 1. The Stiles and Crowford effet shows the eye to be sensitive to the diretion from whih ollimated white light beams enter the eye. First measurements were made by W. S. Stiles and B. H. Crawford in Pro. of SPIE Vol D-16

17 . "The Luminous Effiieny of Rays Entering he Eye Pupil at Different Points" by W. S. Stiles and B. H. Crawford, Proeedings of the Royal Soiety Series B, Vol. 11 No. 778, ( Marh 1933) pp 48 to "Guided Light and Diffration Model of Human-Eye Photoreeptors" by B. Vohnsen, I. Iglesias and P. Artel, Journal of the Optial Soiety of Ameria A, Vol., No. 11 November 005, pp 318 to 38. Pro. of SPIE Vol D-17