Grundlagen Fernerkundung - 12

Transcription

1 Grundlagen Fernerkundung - 12 Image classification GEO123.1, FS2015 Michael Schaepman, Rogier de Jong, Reik Leiterer 5/7/15 Page 1

2 5/7/15 Grundlagen Fernerkundung FS2011 Page 2

3 5/7/15 Grundlagen Fernerkundung FS2011 Page 3

4 Department of Geography Landcover Classification Improvement 75.0% 90.0% 92.1%

5 Today s topics & learning goals Feature space How are land cover classes separated from each other? Classification algorithms Which algorithms exist and how do they differ? Accuracy measures How valid is a classification result?

6 Feature Space (Eigenschaftsraum / Merkmalsraum) Albertz, 2001

7 Feature space: NIR vs RED

8 Department of Geography Feature space: NIR vs RED

9 Statistical description of spectral clusters Histogram / distribution curve Min, Mean, Max values Standard deviations

10 Spectral clustering using the feature space Ground features are defined based on their spectral response Points 1-3 are unknown features that need to be assigned to A-E! = wavelength Messwerte = reflectance

11 Method 1: Parallelepiped classification OG = Obergrenze (upper limit) UG = Untergrenze (lower limit) OG A2 UG A2 UG A1 OG A1

12 Method 1: Parallelepiped classification OG = Obergrenze (upper limit) UG = Untergrenze (lower limit) OG E2 OG A2 UG A2 = UG E2 UG A1 OG A1 UG E1 OG E1

13 Method 1: Parallelepiped classification Features B and C cannot be distinguished. You ll need to try other spectral channels for this purpose. Point 2 is therefore the most uncertain but with only this information classified as B

14 Method 2: Minimum-distance classification Criterium: smallest Euclidean distance to the mean value of the class (+) Boundaries are perpendiculars ( Mittelsenkrechte )! = wavelength Messwerte = reflectance

15 Method 2: Minimum-distance classification Without a max-distance threshold, point 3 changed from unclassified to E Point 2 has changed from B to C Less overlap between class B and C but more misclassifications of Bs as Cs?

16 Method 3: Maximum-likelihood classification Criterium: maximum likelihood ( Mutmasslichkeit ) according to a probabilitydensity function ( Wahrscheinlichkeitsdichte-Funktion ) Boundaries: number of standard deviations (sigma)

17 Method 3: Maximum-likelihood classification Point 3: more than x sigma distance and thus unclassified 1 sigma probability 2 sigma etc. Point 2: class B with 2 sigma distance

18 Common classification algorithms: overview Parallelepiped Minimum Distanz Maximum Likelihood

19 User supervision ( Überwachung ) Unsupervised classification The user does not provide information with respect to the classes Classes are defined based on statistical properties Given the same dataset and same method, each user obtains the identical result Supervised classification The user defines training classes to which unclassified pixels are being compared The result depends on the definition of the training classes and is therefore likely to change between users

20 Method 4: Unsupervised clustering Option 1: no a-priori information at all, (number of) clusters are estimated from statistics Cluster 1 Cluster 2 Cluster 4 Cluster 3

21 Method 4: Unsupervised clustering Option 2: the user defines (only) the number of classes, i.e. 5 Cluster 1 Cluster 2 After classification (option 1 or 2), clusters may be assigned to features of interest based on user knowledge Cluster 5 Cluster 3 Cluster 4

22 Classification methods Albertz, 2001 Unsupervised classification Supervised classification Clustering, (Segmentation)... Parallelepiped, Minimum distance, Maximum likelihood...

23 How accurate is your classification? Landsat Thematic Mapper, K 2,4,7 Looks good! Seems plausible! Is that enough though?

24 A classification is not complete until it has been assessed. (R.G. CONGALTON 1991:35)

25

26 For a given point, how likely is it that the classification represents the reality? How accurate are the various classes? How can we communicate the (in)accuracy to the users of our maps?

27 Options for accurcay assessment Vergleich mit Bild (visuell) -> sehr unzuverlässig, nicht-quantitativ Vergleich mit Karte (visuell) -> unzuverlässig, nicht-quantitativ Vergleich mit ground truth (visuell) -> bedingt abschätzbar, nicht-quantitativ Vergleich mit ground truth (digital) -> zuverlässig, häufig angewendet, quantitativ Unabhängige Verwendung von -> zuverlässig, häufig angewendet, quantitativ ground truth für Training des Klassifikators und Validierung (digital) Vergleich mit unabhängiger/ statistisch basierter Referenz (digital) -> best practice, sehr zuverlässig, quantitativ

28 Department of Geography Accuracy assessment Aerial image HRSC-AX Visual comparison: How accurate has Vegetation been classified? Digital comparison with ground truth map: Only 56% of the actual Vegetation has been classified as Vegetation Classes: Water, Vegetation, Bare / built-up

29 Error matrix ( Fehlermatrix ) (synonym: confusion matrix, matching matrix, contingency table) Classification Reference / ground truth Forest Arable Other vegetation Bare / built-up

30 Error matrix Referenz/ ground truth A B C D! Referenz/ ground truth A B C D! Forest Arable Other vegetation Bare / built-up

31 Error matrix Referenz/ ground truth Referenz/ ground truth A B C D! A B C D ! Forest Other vegetation Arable Bare / built-up

32 Error matrix Referenz/ ground truth A B C D! A n AA nx AB nx AC nx AD n A+ X X X B n BA n BB n BC n BD n B+ X X X C n CA n CB n CC n CD n C+ D n X DA nx DB n DC n DD n D+ X! n +A n +B n +C n +D n ++ N A+ Number of pixels classified as A N +A Number of pixels found to be A in reference Correctly classified (classification == reference) Incorrectly classified (classification " reference) Grand total (total pixel count)

33 Error matrix Referenz/ ground truth A Referenz/ ground truth A B C D! B C D ! Forest Arable Other vegetation Bare / built-up Correctly classified classification == reference Incorrectly classified classification " reference

34 Type 1 (commission) and type 2 (omission) error Referenz/ ground truth Referenz/ ground truth A B C D! A B C D ! Incorrectly classified classification " reference -! Type 1: Commission Error Pixels were included in the class, although they should not have been -! Type 2: Omission Error Pixels were not included in the class, although they should have been

35 Type 1 (commission) and type 2 (omission) error Type 1 (commission error) example for class Forest Referenz/ ground truth Type 2 (ommission error) example for class Forest Referenz/ ground truth

36 Accuracy metrics (or: error metrics) Referenz/ ground truth A B C D! A n AA nx AB nx AC nx AD n A+ X X X B n BA n BB n BC n BD n B+ X X X C n CA n CB n CC n CD n C+ D n X DA nx DB n DC n DD n D+ X! n +A n +B n +C n +D n ++ N A+ Number of pixels classified as A N +A Number of pixels found to be A in reference Correctly classified (classification == reference) Incorrectly classified (classification " reference) Grand total (total pixel count) Accuracy metrics: measures for classification accuracy based on the error matrix

37 Accuracy metrics English Deutsch Overall Accuracy Gesamtgenauigkeit Producer Accuracy Produzenten-Genauigkeit User Accuracy Nutzer-Genauigkeit Average Accuracy durchschnittliche Genauigkeit Mean Accuracy mittlere Genauigkeit Kappa Coefficient Kappa Koeffizient

38 Overall/ Total Accuracy (OA) Referenz/ ground truth A B C D! A B C D Referenz! Overall Accuracy = Count of correctly classified pixels Grand total (total pixel count) = = (~90.6%)

39 Genauigkeitsmasse Overall/ Total Accuracy (OA) (GLC2000) Referenz/ ground truth A B C D! A Die Overall Accuracy ist ein einfaches Genauigkeitsmass, aber wenig aussagekräftig! B Fehler 1. Art (Commission) und C Fehler Art 24 (Omission) 2 30 werden nicht berücksichtigt! D Referenz! Overall die Summe der richtig klassifizierten Pixel Accuracy = Gesamtanzahl aller Pixel/ Grundgesamtheit = = (~90.6%)

40 Producer s Accuracy (PA) Referenz/ ground truth A B C D! A B C The producer needs to know how well her/his classification (e.g. for class A) matches with the reference Referenz D ! PA == Omission Error How many pixels should have been in the class but were not Producer s Accuracy = count of correctly classified pixels in class count of pixels in same reference class = = (~83.3%)

41 User s Accuracy (UA) Referenz/ ground truth A B C D! UA A B C The user needs to know how well a class (e.g. A) matches with the reality Referenz D ! == Commission Error How many pixels are in the class but should not have been User s Accuracy = count of correctly classified pixels in class count of all pixels in that class = = 1 (100%)

42 Accuracy metrics OA, PA, UA Referenz/ ground truth A B C D! UA [%] A If I classified a pixel as forest (A), there is in 100% of the cases indeed forest at that location B C I have captured 100% of the existing arable land (C) with my classification Referenz D ! But: 20% of the classified arable land (C) pixels has another land cover in reality PA [%] OA [%] But: 16.67% of the existing forest (A) was not captured

43 Accuracy metrics OA, PA, UA Referenz/ ground truth A B C D! UA [%] A B C % of all classified pixels matches with the reference / reality D Referenz! PA [%] OA [%] 90.63

44 Hands-on! A A A B B A A A B B A A A B B B B B B B A A A A B A A A B B A A A B B B B B B B Referenz/ ground truth PA [%] OA [%] Referenz A B! A B! 20 UA [%]

45 Hands-on! A A A B B A A A B B A A A B B B B B B B A A A A B A A A B B A A A B B B B B B B Referenz/ ground truth Referenz A B! A B ! PA [%] OA [%] UA [%]

46 Mean UA (or: average accuracy) Referenz/ ground truth A B C D! UA [%] Overall Accuracy % A B C Mean UA % D Referenz! PA [%] Average Accuracy = sum of all user accuracies number of classes = 380% 4 = 95%

47 Mean classification accuracy Referenz/ ground truth A B C D! A B C UA [%] Overall Accuracy % Mean UA % Mean Accuracy % D Referenz! PA [%] Mean Overall Accuracy + Mean UA Accuracy = 2 = = 92.82%

48 Genauigkeitsmasse Kappa Coefficient/ KHAT statistic Referenz Referenz/ ground truth A B C D! A B C D Unterschied der Übereinstimmung zwischen > Referenz und Zufallsklassifikation > Referenz. Zu welchem Anteil sind die korrekt klassifizierten Pixel zufällig richtig? Zufalls-! Um welchen Prozentsatz ist meine besser im Vergleich zu einer rein zufälligen?

49 Genauigkeitsmasse Kappa Coefficient Referenz/ ground truth A B C D! A B C D Referenz! Kappa Coefficient ^ ( k ) = Overall Accuracy 1 - Zufallsübereinstimmung - Zufallsübereinstimmung

50 Genauigkeitsmasse Kappa Coefficient Referenz/ ground truth A B C D! A B C Summe der richtig klassifizierten Pixel Grundgesamtheit = = 58 = 64 Overall Accuracy = D Referenz! Kappa Coefficient = Overall Accuracy 1 - Zufallsübereinstimmung - Zufallsübereinstimmung

51 Genauigkeitsmasse Kappa Coefficient Referenz/ ground truth A B C D! A Overall Accuracy = B C D Zufalls- = Summe der übereinstimmung Produkte der Zeilen- und Spaltensummen Referenz! Grundgesamtheit 2 Kappa Coefficient = Zufallsübereinstimmung - Zufallsübereinstimmung

52 Genauigkeitsmasse Kappa Coefficient Referenz/ ground truth A B C D! A Overall Accuracy = B C D Zufalls- = Summe der übereinstimmung Produkte der Zeilen- und Spaltensummen Referenz! Grundgesamtheit 2 Kappa Coefficient = Zufallsübereinstimmung - Zufallsübereinstimmung

53 Genauigkeitsmasse Kappa Coefficient Referenz/ ground truth A B C D! A Overall Accuracy = B C D Zufalls- = Summe der übereinstimmung Produkte der Zeilen- und Spaltensummen Referenz! Kappa Coefficient = Zufallsübereinstimmung - Zufallsübereinstimmung

54 Genauigkeitsmasse Kappa Coefficient Referenz/ ground truth A B C D! A Overall Accuracy = B C D Zufalls- = (15x18)+ übereinstimmung (9x10)+ (30x24)+ (10x12) Referenz! Kappa Coefficient = Zufallsübereinstimmung - Zufallsübereinstimmung

55 Genauigkeitsmasse Kappa Coefficient Referenz/ ground truth A B C D! A Overall Accuracy = B C Zufallsübereinstimmung = = D Referenz! Kappa Coefficient = Zufallsübereinstimmung - Zufallsübereinstimmung

56 Genauigkeitsmasse Kappa Coefficient Referenz/ ground truth A B C D! A Overall Accuracy = B C Zufalls- = übereinstimmung D Referenz Kappa Coefficient = ! = Kann interpretiert werden als: Meine ist 86.74% besser im Vergleich zu einer rein zufälligen!

57 Genauigkeitsmasse Kappa Coefficient -! vielfältig interpretierbar -> es existieren verschiedene Richtwerte für die Beurteilung des Koeffizienten Kappa - Coefficient Level of Agreement to Reference 0.00 poor / slight / schwach fair / leicht moderate / mittelmäßig substantial / gut almost perfect / sehr gut!"#$%&'()'*)+,-).// ! keine allgemeingültigen Aussagen möglich!

58 Genauigkeitsmasse Kappa Coefficient -! keine allgemeingültigen Aussagen möglich! A A A A A A A A A A A A A A A A A A A B Wald Sonstige Vegetation A A A A A A A A A C Ackerland Vegetationsfrei A A A A A A A A A D Referenz/ ground truth Kappa Coefficient = 0 OA = 85% Kappa - Coefficient Level of Agreement to Reference 0.00 poor / slight / schwach fair / leicht moderate / mittelmäßig substantial / gut almost perfect / sehr gut

59 Accuracy assessment summary Referenz A B C D! A n AA n AB n AC n AD n A+ Overall Accuracy = q! i = 1 n n ii PA = n ii n +i UA = n ii n i+ B n BA n BB n BC n BD n B+ C n CA n CB n CC n CD n C+ D n DA n DB n DC n DD n D+! N +A N +B N +C N +D n Mean UA Accuracy = q! UA i = 1 n Mean UA OA + Mean Accuracy = Accuracy 2 q = Anzahl an Klassen n = Grundgesamtheit n ii = Anzahl übereinstimmender Pixel n i+ = Summe der Zeilenwerte n +i = Summe der Spaltenwerte

60 Accuracy metrics applying them correctly Accuracy metrics depend on the sampling scheme of the reference x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x OA = 95.6% Forest ("20%) Forest (< 20%) Grassland Water Referenz/ ground truth A B C D! A B C D !

61 Accuracy metrics applying them correctly Accuracy information depend on the choice of metric x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x OA = 95.6% Mean user accuracy = 72.0% Mean classification accuracy = 83.9% Forest ("20%) Forest (< 20%) Grassland Water PA UA

62 Accuracy metrics applying them correctly Accuracy information depend on the interpretation of the metric

63 Accuracy Take-Home Messages A classification is not complete until it has been assessed. You need multiple accuracy metrics to get a good representation of the actual accuracy; a single metric may lead to an incorrect assessment The reference dataset must be representative for the classification Be critical when you communicate your classification to your users

64 Requirements regarding the previous two lectures (on Image analysis and Image classification ) Image analysis Understand the concepts of (visualizing) spectral bands Understand simple band operations and what they can be used for Know various change-detection techniques (concepts, no tech. details) Be able to describe and recognize low-pass and high-pass filters Classification Know different (types of) algorithms (characteristics, pros and cons), be able to describe them and to conceptually apply them Be able to describe and apply the various accuracy metrics

65 Thank you for your attention!