Appenix: Proof of Spatial erivative of Clear Rainrop Shaoi You Robby T. Tan The University of Tokyo {yous,rei,ki}@cvl.iis.u-tokyo.ac.jp Rei Kawakami Katsushi Ikeuchi Utrecht University R.T.Tan@uu.nl Layout Section 1 forms the imagery of clear rainrop with more etaile notations. Two conclusions use in the manuscript (Eq. (4, after Eq. (6 are at the en of Section 2 an Section 3 corresponingly. 1. Geometric erivative of Clear Rainrop Continuous mapping As shown in Fig. 1 (a, the appearance of each rainrop is a contracte image of the backgroun, as if it is taken from a cataioptric camera. The numeric values inicate in Fig. 1 (c are the contraction ratios between the original image an the image insie the rainrops, calculate from the black an white patterns. The contraction ratio is aroun 20 to 30, meaning that the motion observe insie the rainrops will be 1/30 to 1/20 smaller than the other areas in the image. This significant motion ifference can be use as a clue for rainrop etection. Mathematically, for a given rainrop, we escribe the smooth expan mapping start from rainrop area Ω r into the environment scene Ω e as ϕ: ϕ : Ω r Ω e. (12 The appearance of the rainrop an the environment share the same image plane an coorinates. In orer to istinguish, we enote the points an coorinates in rainrop Ω r as: = (u, v an the corresponing points an coorinates in environment Ω e as P e = (x, y. Then ϕ can be expresse as: P e = (x, y = ϕ( = ϕ(u, v = (ϕ 1 (u, v, ϕ 2 (u, v. (13 Local erivative (Jacobian with: ϕ 1 u(u, v = ϕ1 (u,v u. The local ifferentials (Jacobian of ϕ at = (u, v is efine as: ( ϕ 1 u (u, v J ϕ ( = J ϕ (u, v = ϕ 1 v(u, v ϕ 2 u(u, v ϕ 2 v(u, v, (14 1
Image plane Rainrop (Image insie a rainrop Mapping φ 18 35 32 21 22 Environment (Image without rainrops (a Observation (b Continuous mapping (c Contraction ratio Figure 1. (a The appearance of each rainrop is a contracte image of the backgroun. (b On the image plane, there is a smooth mapping ϕ starting from the rainrop into the environment scene. (c Contraction ratios from backgroun to rainrop are significant. 26 Rainrop θ o Environment point P e Image plane θ i Equivalent lens θ e Rainrop plane Image system axis Figure 2. Refraction moel of a pair of corresponing points on an image plane. There are two refractions on the light path through a rainrop. (The camera lens cover or protecting shiel is assume to be a thin plane an thus neglecte. The local motion at (u, v, enote as (δu, δv T, an the local motion at (x, y, enote as (δx, δy T, is linearly associate by J ϕ (u, v: ( δx δy ( δu = J ϕ (u, v δv. (15 Instea of moeling ϕ or J ϕ (u, v, we are intereste in the non-irectional scale ratio between (δx, δy an (δu, δv. Accoring to Eq.(15: (δx, δy 2 = (δx, δy T (δx, δy = (δu, δv T (J ϕ (u, v T J ϕ (u, v(δu, δv, (16 with (J ϕ (u, v T J ϕ (u, v is symmetric an positive-semiefinite, an can be iagonalize as: (J ϕ (u, v T J ϕ (u, v = E T ( λ 2 1 (u, v λ 2 2(u, v E, (17 where E is an orthogonal matrix, an 0 λ 1 (u, v < λ 2 (u, v. Therefore, accoring to Eqs.(16 an (17, for any irectional motion (δu, δv at (u, v: (δx, δy (δu, δv λ 1(u, v. (18
A simplifie version of Eq. (18 in the manuscript (Eq.(3 is given as: ϕ(u + δu, v + δv ϕ(u, v E ϕ (u, v, δu, δv = lim (δu,δv 0 (u + δu, v + δv (u, v. We can give a lower bounary, enote as λ lower, for all λ 1 (u, v insie the rainrop area Ω r : We call it the lower bounary of the expansion ratio. λ lower min{λ 1 (u, v (u, v Ω r } (19 2. Proof of Expansion Ratio Lower bounary of the expansion ratio The lower bounary of the expansion ratio is given by: λ lower n a 1 n a R +, (20 where an n a are refraction inices of water an air, is the istance from the lens to the rainrop, is the istance from the rainrop to the backgroun, an R is the lower bounary of rainrop curvature raius. Proof A light ray passing through a rainrop unergoes two refractions: first, the refraction from the air to the water, an secon, the refraction from the water to the air. Thus, the mapping function, ϕ, can be separate as two continuous mappings: ϕ = a w ϕ w a ϕ, (21 inex a stans for air, an w stans for water. Assuming the contact surface between the camera lens cover an rainrop is flat, a w ϕ shoul be analytically solvable. Accoring to Snell s law, where n a sinθ i = sinθ o, we can have: P e = tanθ e tanθ i = n a 1 + nw na 1 + = constant, (22 where the notation is efine in Fig. 2, an nw n a is approximately 4 3 > 1. Thus the ratio: P e = P e = n a 1 + nw na 1 + Hence, a lower bounary of the expansion (contraction here ratio is: > n a. (23 a w λ lower = n a. (24
n water n air θ o θ i k 2 k 1 S θ r θ e P e R (a (b Figure 3. Simplifie refraction moel of the secon refraction using principle curvature. (a Given a point on rainrop surface S, its two principle curvatures vectors k 1, k 2 an the normal n are orthogonal to each other. (b Refraction moel of the secon refractiohen assuming the normal of refraction is very close to the image system axis. The notation are same with Fig. 2, R is the curvature raius at the place an irectiohere the refraction happens. Now, we estimate the expansion ratio of the secon refractio a ϕ. Note that, referring to Fig. 2, although in the first refraction the irection an position of the emergence light trace coul be analytically solve, the position an angle of the incient light of the secon refraction is still unsolvable. This is because we have no knowlege of the position an shape of the rainrop. To estimate the expansion ratio of the secon refraction, we start from the ifferential geometry on the outer surface of the rainrop. For a given position (u, v on the surface of the rainrop, its up to secon orer ifferential geometry values are illustrate as in Fig. 3(a [2]. The upper principle curvature vector, k 1, points to the irectiohere the rainrop surface bens most. An the lower principle curvature vector, k 2, points to the irectiohere the surface bens least. The curvature vector of any other irection, k, is the linear combination of k 1 an k 2. The values of any curvature vector k is boune by k 1 an k 2 : k 2 k k 1. (25 The reciprocal of curvature is calle curvature raius: R = 1. In any irection, it is boune by two k principle curvature raius: R 1 R R 2. As illustrate in Fig. 2, we now consier the secon refraction locally at given point (u, v. Mention that there is no knowlege about how this local coorinates is aligne to the global coorination. First, we try to estimate the angular ratio θo. θ i Accoring to Snell s law, we have: θ o θ i = na cosθ i (1 ( nw n a 2 sin 2 θ i 1 2, (26 where we know that nw n a > 1, thus Eq.(26 gets its minimum when θ i =0: ( θo min = θ o θ i θ i (27 θi =0. This is in accorance with the observation of real rainrop image shown in Fig. 1(a.
As illustrate in Fig. 3(b, accoring to Eq.(27, we may put the normal of the rainrop surface consierably close to the image system axis. Assuming every angle is significantly small: θ i 1, θ o 1, θ e 1, θ r 1. (28 Accoring to Eq.(28, we can use the following approximation: sinθ = tanθ = θ, Pe expansion ratio is estimate as: = Pe. The P e = P e = θ e = 1 θ r n a R +. (29 Combine Eqs. (24 an (29, we have: λ lower n a 1 n a R +. (30 Parameter estimation The three parameters in Eq. (30,, an R, are estimate as following: 1. is the istance from rainrop panel to backgroun, > 1m is simply satisfie in outoor vision system. 2. is the istance from rainrop panel to the equivalent camera lens center. Usually, < 200mm. 3. R is the raius curvature on the smoothest place of the rainrop. The bigger the rainrop is, the smoother the rainrop surface is. However a rainrop cannot be too big without sliing own a vertical panel. Accoring to our observation (Fig. 1(a, an static of size of falling rainrops in Garg. an Nayer s paper [1], usually, R < 2.5mm. Thus, R < 5mm is a very safe estimation. Substitute the estimation of,, R an na = 3 to (20, we have: 4 The simplifie version in the manuscript (Eq. (4 is: λ lower > 10. (31 E ϕ > 10 1. 3. Proof of Areal Expansion Ratio Lower boun of areal contraction ratio The lower boun of areal contraction ratio is: λ 2 lower. (32 Mention that ϕ (Eq. (12 is not conformal, the proof of Eq. (32 is given as following.
Proof Observing Eqs.(17 an (31, both eigenvalues of (J ϕ (u, v T J ϕ (u, v is greater than 0 which means (J ϕ (u, v T J ϕ (u, v is positive efine, an thus, J ϕ (u, v is reversible at (u, v: ( δu δv ( δx = Jϕ 1 (x, y δy An at the neighborhoo of (u, v, enote as V (u, v, ϕ is a one to one mapping which maps V (u, v to V (x, y = ϕ(v (u, v. If we select a pixel in V (x, y, its area can be efine as: (33 x y sinθ = x y (34 where x = ( x, 0 T an y = (0, y T enotes the irectional with an height of the pixel, θ is the angle between x an y. For a square pixel, x is orthogonal to y, thus sinθ = 1. For the given pixel area, its corresponing area on the rainrop can be calculate using Eq.(33: Jϕ 1 (x, y x Jϕ 1 (x, y y sinθ (35 where θ is the angle between Jϕ 1 (x, y x an Jϕ 1 (x, y y. Using the lower boun of linear expansion ratio in Eq.(31, we have: J 1 ϕ Thus, the areal ratio is: (x, y x J 1 (x, y y sinθ J ϕ (x, y 1 x J ϕ (x, y 1 y ϕ λ 1 lower x λ 1 lower y = ( 1 λ lower 2 x y (36 The simplifie version use in the manuscript (after Eq. (6 is: References λ 2 lower. (37 E 2 ϕ > 100 1. [1] K. Garg an S. Nayar. Photometric moel of a rainrop. CMU Technical Report, 2003. 5 [2] V. Zorich an R. Cooke. Mathematical analysis. Springer, 2004. 4