Tree Crown Extraction using Marked Point Processes

Transcription

1 Tree Crown Extraction using Marked Point Processes G. Perrin 12 PhD Advisors : C. Saguez 1 J. Zerubia 2 1 : MAS Laboratory (Ecole Centrale Paris) 2 : Ariana Res. Group (CNRS/INRIA/UNSA) prenom.nom@mas.ecp.fr prenom.nom@inria.fr Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

2 1 Introduction 2 Marked point processes : definition and examples 3 Description of the mpp models for our application 4 Results on dense vegetation 5 Sparse vegetation 6 Conclusion and Future Work Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

3 French Inventory / French forests French forests cover 15 millions of ha (27% of the territory). French National Inventory (IFN) : created in Photo interpretation + ground measurements for : cartography (land cover, main species, forest structures) : minimum mapped surface = 2.25 ha. statistics (volume per species, ecological measurements,...). grid on the territory : 10% of the grid points explored annualy. Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

4 Data : Colour InfraRed (CIR) images Colour Infrared aerial photographs, 1/ June September. Scanned to a 50cm per pixel resolution. Near future : National Geographic Institute Orthophotos Database (4 band camera : colour + near infrared), satellite images. Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

5 Goals of this work Helping the inventories in their work in order to : get some statistics at the scale of the tree (increasing demands). automate or semi-automate some tasks which would be very costly if done by an operator. Some existing methods in tree crown extraction with CIR images : local maxima approaches : [Pinz91,Dralle96], pixel-based approaches : [Gougeon98,Erikson04], or object-based approaches : [Larsen97]. none of them get position and size of the trees on both dense and sparse vegetation. Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

6 Proposed method Object-based approach : model the trees and their interaction at the scale of the stand. Marked point processes : random configuration of an unknown number of geometric objects whose position and marks are unknown. Find the best configuration of objects : Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

7 1 Introduction 2 Marked point processes : definition and examples 3 Description of the mpp models for our application 4 Results on dense vegetation 5 Sparse vegetation 6 Conclusion and Future Work Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

8 Some notations Configuration x = {x 1,..., x n } : an unordered set of points in some space S. Ex : {0, 2.3, π, 3 11 } is a configuration of points in S = R. Configuration space : we write N f the space of locally finite (finite number of points in any bounded Borel set A S) and simple (distinct points) configurations. Let ν(.) be a locally finite Borelian measure on S, and Nn f = {x N f : N x (S) = n}. We have : ν(n f ) = ν(nn f ) = ν(s) n n! n=0 n=0 = e ν(s) (1) Point process X : a mapping from a probability space into N f. Marked point process X : the objects of the process are in S = P M. Both the process on S and the underlying process on P are well-defined points processes. Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

9 Point processes : a first example Poisson process : E B [X ] = ν(b) = B λ(x)λ(dx) <. Independently distributed points. Homogeneous Poisson process, uniform intensity λ(.). Inhomogeneous Poisson process, λ(.) higher in bottom right. Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

10 Point processes : a first example How to sample a configuration of the Poisson process : compute the number of points : P[N(X ) = n] = e ν(s) ν(s) n generate each point wrt ν(.) ν(s). n! Homogeneous Poisson process. Inhomogeneous Poisson process. Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

11 Marked point processes : the general framework Density of a mpp : A mpp X is defined by its density f (.) with respect to the reference Poisson process probability distribution µ(.) : P X (dx) = f (x)π ν (dx) We have : π ν (B) = e ν(s) (1 [ B] + n=1 ) π νn (B) n! with π νn (B) = S 1 [{x1,..., x n} B]ν(dx 1 )... ν(dx n ). S Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

12 Marked point processes : other examples Cox processes : doubly stochastic Poisson process : the intensity of the Poisson Process is itself randomly distributed. Problem : they do not guarantee a hard-core distance. λ(x) = µ φ (x p i ) i=1 Gibbs processes : f (x) = 1 Z exp ( U(x)). Constraints on the objects and their interactions. U(x) = U ri (u) + U i (u i v) i u x i u i v Example : Strauss process U(x) = γ u,v x 1 d(u,v)<r Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

13 Marked point processes : definition and examples Neyman-Scott process. λ(x) = n(p) i=1 1 b(p i,r)(x) Strauss process : repulsion btw the points : γ > 0. Strauss process : attraction btw the points : γ < 0. Density f (.) is not well defined the number of points needs to be bounded. Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

14 Simulation How to sample a marked point process in the general case? Markov Chain Monte Carlo algorithms. Reversible Jump MCMC algorithm [Green95] = iterative algorithm : current state Xt = x : we propose a perturbation y Q(x,.). birth and death (add or remove one point),... accept the proposition with a probability α = min{1, R(x y)}, with R(x y) = Q(y,dx) f (y) π ν(dy) Q(x,dy) f (x) π ν(dx) Ergodic convergence to the probability distribution f (x)π ν (dx). Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

15 1 Introduction 2 Marked point processes : definition and examples 3 Description of the mpp models for our application 4 Results on dense vegetation 5 Sparse vegetation 6 Conclusion and Future Work Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

16 Energy of the marked point process Goal : to extract the trees with random geometric objects. Bayesian framework = A priori + Likelihood : f (x) = f p (x)l(y = I X = x) Non Bayesian framework = Regularizing term + Data term : U(x) = U p (x) + U d (x) Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

17 Prior Energy U p (x) for the ellipse model Constraints on the objects + Pairwise interactions btw objects. Goal : to model the pattern of the trees in a stand. Penalize flat ellipses ( a b >> 1). Penalize overlapping objects : R(x) = λ P (x 1 x 2 ) x i r x j max(λ P (x 1 ),λ P (x 2 )) Favor clustering or alignments (for example in poplar plantations)... Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

18 Data energy term U d (x) U d (x) : data term = fitting the object into the image. Bayesian framework : likelihood L(Y = I X = x), Gaussian classes (vegetation and background : parameters of these classes calculated with a k-means algorithm). non Bayesian framework : Ud (x) = γ d x i x U d(x i ) Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

19 Computing the data energy term What is typical of the presence of a tree? high reflectance in the near infrared ++, shadow ++, neighbourhood The data term is negative for good objects, and positive for bad objects. In dense vegetation : objects = ellipses merged shadows, shadow area = all around the tree. calc. of the pixel distribution inside the object and its shadow area. favour high Bhattacharya distances db btw these 2 distributions. S 2 Dark pixels in the boundary F(x) ρ Bright pixels inside the ellipse x Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

20 Simulation and Optimization Proposition kernel = birth and death, translation, rotation, split/merge, birth and death in a neighbourhood,... Simulated annealing : f (x) 1 Tn π ν (dx). Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

21 1 Introduction 2 Marked point processes : definition and examples 3 Description of the mpp models for our application 4 Results on dense vegetation 5 Sparse vegetation 6 Conclusion and Future Work Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

22 Results with the 2D model (1) Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

23 Results with the 2D model (2) False alarms : color model, border of the plantation, point of view. Taking into account the sun position improves the extraction : Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

24 Results with the 2D model (3) Non Bayesian model is more robust when the image contains non vegetation area. Bayesian model puts some objects where the near infrared is high. Eure et Loire c IFN Bayesian model Non Bayesian model Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

25 Results with the 2D model (4) Timber forest makes segmentation more difficult (even for a human operator : impossible to count precisely the trees). Bayesian model works better. Not enough shadow around the trees in very dense areas for the non Bayesian model. Timber forest c IFN Non Bayesian model Bayesian model Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

26 1 Introduction 2 Marked point processes : definition and examples 3 Description of the mpp models for our application 4 Results on dense vegetation 5 Sparse vegetation 6 Conclusion and Future Work Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

27 See the trees as 3D objects Application : sparse vegetation, border trees in plantation, mixed height stands. Hypotheses : trees close to the Nadir and flat ground (no deformation). Results : position, crown diameter, height... Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

28 3D model New objects = ellipsoids. New data energy U d (x) : drop shadows, shadow area = in the direction of the sun light. favour high Bhattacharya distances db + gradient on the contour. Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

29 Results with the 3D model (1) 3D model extraction in sparse vegetation. 2.5 ha (Alpes Maritimes) c IFN. 3D model extraction. Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

30 Results with the 3D model (2) Applications for sparse vegetation : density of the sparse vegetation : 19%. estimation of the height of the trees. Binary image of the vegetation Height of the trees. Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

31 Results with the 3D model (3) Applications for plantations : estimation of the wood volume in the plantations : 3D reconstruction with AMAP Orchestra Software c CIRAD. estimation of the age by calculating the mean height. Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

32 Results with the 3D model (4) Coppice-with-standards. Information on the timber forest density : 15%. Mixed height stand (3 ha) c IFN. 3D model extraction. Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

33 1 Introduction 2 Marked point processes : definition and examples 3 Description of the mpp models for our application 4 Results on dense vegetation 5 Sparse vegetation 6 Conclusion and Future Work Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

34 Conclusion Advantages of the proposed approach : gives some information on the stand at the scale of the tree. includes interactions btw the objects (interactions knowledge can be embedded in the model wrt to the stand). automatic computation of statistics : number of trees, cover density, height of the trees and wood volume (for sparse vegetation and plantations). works both on dense forest and sparse vegetation. Principal limit = geometry of the objects. Comparison with other algorithms : template matching (TM), region growing (RG). Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

35 Comparison (1) Plantations : object approach leads to less false alarms. MPP : 866 trees RG : 1023 trees Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

36 Comparison (2) Dense forest : more objects extracted by the pixel-based approachs. MPP : 491 trees TM : 411 trees RG : 800 trees Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

37 Comparison (3) Sparse vegetation : object approachs are the only ones to work. MPP : 302 trees TM : 288 trees Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

38 Future Work Future work : I I I I I I propose a colour model. cooperation btw models. embed texture classification. best image resolution? new 3D templates for the trees take into account the topography Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39

39 References and links MAS Laboratory : Project Ariana : Larsen,M. : Crown Modelling to Find Tree Top Positions in Aerial Photographs, Proc. 3 rd Int. Airborne Remote Sensing Conference and Exhibition, pages , vol. 2, Erikson, M. : Segmentation and Classification of Individual Tree Crowns, Swedish University of Agricultural Sciences, Uppsala, Sweden, Green, P.J. : Reversible Jump Markov Chain Monte Carlo Computation and Bayesian Model Determination, Biometrika 82, , Guillaume Perrin (ECP / INRIA) LIAMA - August, 28th / 39