Training Course Textbook Emma Woolliams Andreas Hueni Javier Gorroño. Intermediate Uncertainty Analysis

Transcription

1 Intermedate Uncertanty Analyss for Earth Observaton Instrment Calbraton Modle Tranng Corse Textbook Emma Woollams Andreas Hen Javer Gorroño

2

3 Ttle: Intermedate Uncertanty Analyss for Earth Observaton (Instrment Calbraton) Reference: EMRP-ENV04-D5.._textbook Verson: 0.1 Date: Febrary 014 Dssemnaton level: Athors: Approved by: Keywords: PUBLIC Emma Woollams, Natonal Physcal Laboratory Andreas Hen, Unversty of Zrch Javer Gorroño, Natonal Physcal Laboratory Teresa Goodman, Natonal Physcal Laboratory Uncertanty analyss, Uncertanty bdget, Radometrc Calbraton, Imager Cal/Val Contact: Abot the EMRP The Eropean Metrology Research Programme (EMRP) s a metrology-focsed Eropean programme of coordnated R&D that facltates closer ntegraton of the natonal research programmes. The EMRP s jontly spported by the Eropean Commsson and the partcpatng contres wthn the Eropean Assocaton of Natonal Metrology Instttes (EURAMET e.v.). The EMRP wll ensre collaboraton between Natonal Measrement Instttes, redcng dplcaton and ncreasng mpact. The overall goal of the EMRP s to accelerate nnovaton and compettveness n Erope whlst contnng to provde essental spport to nderpn the qalty of or lves. See for more nformaton Abot the EMRP JRP Metrology for Earth Observaton and Clmate (METEOC) Ths corse has been prodced for the MetEOC (Eropean Metrology for Earth Observaton and Clmate) project, fnded by the Eropean Metrology Research Programme. MetEOC s developng new nfrastrctre and methods to allow hgher, traceable, accracy to be delvered to the Eropean calbraton and valdaton commnty. Ths ncldes: Charactersaton of the stray-lght propertes of an arborne hyperspectral mager sng tneable laser radaton. Development of a faclty whch allows ocean color sensor calbratons to be performed at an NMI, at a cstomer ste, n the feld, n ar and n a vacm. Measrements to spport the n-flght calbraton of the arcraft based atmospherc lmb sondng experment GLORIA. The desgn of a set of Novel LED based (self-calbratng) radometers enable atonomos low cost montorng of test stes for post lanch calbraton and valdaton of satelltes. Development of a strategy to enable SI Traceablty to be assessed and assgned to a radaton transfer model sng a vrtal smlaton of a real, traceably calbrated, optcal, geometrc and mechancal, 3D target. MetEOC s a three year project, endng n September 014. A follow on project, MetEOC, has been fnded by the EMRP and wll begn n late 014. For more nformaton abot the project vst:

4

5 Contents Contents Introdcton... 1 Key concepts for ncertanty analyss....1 QA4EO.... The ISO and BIPM Gde to the Expresson of Uncertanty n Measrement (GUM) Traceablty and SI Correctons, errors and ncertantes The law of propagaton of ncertantes Classfcatons Random and Systematc Effects Type A and Type B Absolte and relatve ncertantes Wrtng abot ncertantes Understandng the Law of Propagaton of Uncertantes Senstvty coeffcents Addng n qadratre Takng an average of ndependent measred vales Applyng the law of propagaton of ncertantes Averagng enogh readngs Allan devaton/allan varance Takng an average of partally correlated measred vales Correlaton Analytcal approach by modellng ot the correlaton: absolte ncertantes Analytcal approach by calclatng covarance: absolte ncertantes Analytcal approach by modellng ot the correlaton: relatve ncertantes Covarance term n the law of propagaton of ncertantes Covarance... 5

6 3.5. Covarance from an error model Estmatng the covarance from expermental and modelled data Range of possbltes for correlaton The steps to an ncertanty bdget Steps Step 1: Descrbng the traceablty chan Step : Wrtng down the calclaton eqatons Step 3: Consderng the sorces of ncertanty Steps 1 to 3 for a space-borne sensor Step 4: Creatng the measrement eqatons Step 5: Determnng the senstvty coeffcents Step 6: Assgnng ncertantes Step 7: Combnng and propagatng ncertantes Step 8: Expanded ncertantes Case stdy: APEX mager calbraton (smplfed) The APEX mager calbraton Step 1: Descrbng the traceablty chan Step : Wrtng down the calclaton eqatons For the calbraton of the sphere For the calbraton of the APEX mager For the ser of the APEX mager to measre scene radance Step 3: Consderng the sorces of ncertanty For the calbraton of the SVC spectrometer aganst the RASTA For the calbraton of the sphere wth the SVC spectrometer For the combned sorce radance from the sphere and flters For the calbraton of the APEX gan from the measrement of the sphere-flter sorce For the observed scene radance Step 4: Creatng the measrement eqatons... 56

7 5.6 Step 5: Determnng the senstvty coeffcents Step 6: Assgnng ncertantes Step 7: Combnng and propagatng ncertantes Step 8: Expandng ncertantes Ths s jst the start of the process Straght lne calbraton eqatons A straght lne calbraton eqaton Uncertanty analyss overvew Calbraton data and ncertantes Determnng the ft coeffcents (dong the ft) Approaches to take Unweghted ft and Monte Carlo Weghted ft and analytcal covarance Rgoros analyss Valdatng the ft Usng the ft Analytcal covarance when the ft s sed Straght lne nterpolatons Spectral Selecton Spectral response fncton Uncertanty assocated wth the spectral response fncton Spectral ntegrals and convolton Orgn of spectral ntegrals Calclatng the ntegral The ncertanty assocated wth the ntegral Stray lght (ot of band) Spectral and spatal effects Post-lanch calbraton and Level-1 EO prodcts radometrc ncertanty... 89

8 8.1 Imager changes n orbt On-board calbraton systems Vcaros cal/val Uncertanty analyss Example: PTFE dffser Use of the solar dffser n orbt Step 1: Descrbng the Traceablty Chan Step : Wrtng down the calclaton eqatons Step 3: Consderng the sorces of ncertanty Step 4: Creatng the measrement eqaton Step 5: Determnng the senstvty coeffcents Step 6: Assgnng ncertantes Step 7: Combnng and propagatng ncertantes Step 8: Expandng ncertantes Level-1 EO prodcts radometrc ncertanty Radance-to-reflectance converson Image orthorectfcaton Conclsons

9 1 Introdcton Ths s the corse textbook for the tranng corse developed nder the EMRP project MetEOC-1 as an ntermedate level tranng corse for ncertanty analyss wth emphass on radometrc nstrment calbraton and charactersaton for Earth Observaton. The textbook does not follow the corse lectres completely, bt s desgned as a standalone teach yorself gde whch s spplemented by the lectres. In partclar the textbook ncldes topcs that are not taght drng the corse (n partclar straght lne calbraton eqatons and spectral effects) and does not nclde other topcs that are taght n the corse (most notably vcaros calbraton). ~ 1 ~

10 Key concepts for ncertanty analyss Ths ntermedate-level corse blds on some key deas for ncertanty analyss that are provded n begnner level texts, for example the NPL Best Practce Gde nmber 11 by Stephane Bell. Ths s freely downloadable at: Also consder NPL s e-corses (see the Frther Stdy secton on page 119. Ths secton of ths textbook emphasses some of the key deas of ncertanty analyss that are assmed for the rest of the corse..1 QA4EO The Qalty Assrance Framework for Earth Observaton (QA4EO) was wrtten and endorsed by the Commttee on Earth Observaton Satelltes (CEOS) and conssts of a key prncple, spported by a set of gdelnes. The QA4EO Key Prncple states: Data and derved prodcts shall have assocated wth them a flly traceable ndcator of ther qalty Wth the defntons: Qalty Indcator A Qalty Indcator (QI) shall provde sffcent nformaton to allow all sers to readly evalate the ftness for prpose of the data or derved prodct Traceablty A QI shall be based on a docmented and qantfable assessment of evdence demonstratng the level of traceablty to nternatonally agreed (where possble SI) reference standards Its am s to provde EO data sers wth sffcent (smple) nformaton to enable them to evalate the ftness-for-prpose of data/nformaton for ther applcatons throgh the assgnment of Qalty Indcators (QI) to data and derved prodcts. QA4EO encorages the qantfcaton of ncertantes and the docmentaton of spportng evdence. There are seven gdelnes wrtten to ad the ser n ths process, provdng examples, templates and detals of the sort of nformaton and evdence whch shold be made avalable. These gdelnes are largely derved from exstng best practce n both the EO sector and elsewhere, and are lsted n Table 1. Ths corse text s flly complant wth QA4EO. ~ ~

11 Table 1 Lst of QA4EO Gdelnes. QA4EO-QAEO-GEN-DQK-001 QA4EO-QAEO-GEN-DQK-00 QA4EO-QAEO-GEN-DQK-003 QA4EO-QAEO-GEN-DQK-004 QA4EO-QAEO-GEN-DQK-005 QA4EO-QAEO-GEN-DQK-006 QA4EO-QAEO-GEN-DQK-007 A gde to establsh a Qalty Indcator on a satellte sensor derved data prodct A gde to content of a docmentary procedre to meet the Qalty Assrance reqrements of CEOS A gde to reference standards n spport of Qalty Assrance reqrements of QA4EO A gde to comparsons: organsaton, operaton and analyss to establsh measrement eqvalence to nderpn the Qalty Assrance reqrements of QA4EO A gde to establshng valdated models, algorthms and software to nderpn the Qalty Assrance reqrements of QA4EO A gde to expresson of ncertanty of measrements A gde to establshng qanttatve evdence of traceablty to nderpn the Qalty Assrance reqrements of QA4EO. The ISO and BIPM Gde to the Expresson of Uncertanty n Measrement (GUM) The Gde to the Expresson of Uncertanty n Measrement, known as the GUM, provdes gdance on how to determne, combne and express ncertanty [1]. It was developed by the JCGM (Jont Commttee for Gdes n Metrology), a jont commttee of all the relevant standards organsatons (e.g. ISO) and the BIPM (Brea Internatonal des Pods et Mesres). Ths hertage gves the GUM athorty and recognton. The JCGM contnes to develop the GUM and has recently prodced a nmber of spplements. All of these, as well as the VIM (Internatonal Vocablary of Metrology, []) are freely downloadable from the BIPM webste 1. QA4EO Gdelne 006 attempts to explan the man prncples of the GUM to the EO commnty. Ths corse s flly complant wth the GUM..3 Traceablty and SI Traceablty s defned by the Commttee for Earth Observaton Satelltes (CEOS) as the Property of a measrement reslt relatng the reslt to a stated metrologcal reference (free defnton and not necessarly SI) throgh an nbroken chan of calbratons of a measrng system or comparsons, each contrbtng to the stated measrement ncertanty. Traceablty ncldes both an nbroken chan (.e. t s calbrated aganst X, whch was calbrated aganst Y, whch was calbrated aganst Z, all the way back to SI, or, perhaps, a commnty reference) and the docmentary evdence that each step was done n a relable way (deally adted, at least thoroghly peer-revewed). 1 ~ 3 ~

12 Traceablty shold, deally, be to the Internatonal System of Unts, known as the SI from ts French name, le Système nternatonal d ntés. The SI nts provde a coherent system of nts of measrement blt arond seven base nts and coherent derved nts. A coherent system of nts means that a qantty s vale does not depend on how t was measred. The SI s an evolvng system, wth the responsblty for ensrng long term consstency wth the General Conference on Weghts and Measres (CGPM), rn throgh the Internatonal Brea of Weghts and Measres, the BIPM, and mantaned natonally throgh the Natonal Metrology Instttes (NMIs). The Mtal Recognton Arrangement (MRA) sgned n 1999 between the NMIs ensres that measrements made traceably to any NMI wthn the MRA are recognsed by other NMIs. Ths s enforced by both formal nternatonal comparsons and a process of adtng and peer-revewng statements of calbraton capablty. For the ser, ths means that traceablty to SI can be acheved throgh any NMI wthn the MRA..4 Correctons, errors and ncertantes The terms error and ncertanty are not synonyms, althogh they are often confsed. To nderstand the dstncton, consder the reslt of a measrement the measred vale. The vale wll dffer from the tre vale for several reasons, some of whch we may know abot. In these cases, we apply a correcton. A correcton s appled to a measred vale to accont for known dfferences, for example the measred vale may be mltpled by a gan determned drng the nstrment s calbraton, or a measred optcal sgnal may have a dark readng sbtracted. Ths correcton wll never be perfectly known and there wll also be other effects that cannot be corrected, so after correcton there wll always be a resdal, nknown error an nknown dfference between the measred vale and the (nknown) tre vale. The specfc error n the reslt of a partclar measrement cannot be known, bt we descrbe t as a draw from a probablty dstrbton fncton. The ncertanty assocated wth the measred vale s a measre of that probablty dstrbton fncton; n partclar, the standard ncertanty s the standard devaton of the probablty dstrbton, and the eqvalent of ths for other dstrbtons. There are generally several sorces of ncertanty that jontly contrbte to the ncertanty assocated wth the measred vale. These wll nclde ncertantes assocated wth the way the measrement s set p, the vales ndcated by nstrments, and resdal ncertantes assocated wth correctons appled. The fnal (nknown) error on the measred vale s drawn from the overall probablty dstrbton descrbed by the ncertanty assocated wth the measred vale. Ths s blt p from the probablty dstrbtons assocated wth all the dfferent sorces of ncertanty. The se of the words error and ncertanty descrbed here s consstent wth paragraph..4 of the GUM. See also Secton.6.1 and Fgre 1. See the note box n secton 0. ~ 4 ~

13 .5 The law of propagaton of ncertantes The am of ncertanty analyss s to estmate the ncertanty assocated wth the measred vale, whch may be the reslt of a process nvolvng several dfferent parameters beng controlled and set or measred, and a calclaton. To obtan the fnal ncertanty, ncertantes de to each element n the process that affect the fnal reslt mst be combned.e. they mst be propagated throgh ths process. The GUM gves the Law of Propagaton of Uncertanty: n n 1 f n f f c x 1 x 1 j 1 x = = = + j ( ) = ( ) + (, j), (.1) y x x x whch apples for a measrement model of the form where an estmate x of qantty (,,,,, ) Y f X1 X X3 X = (.) X has an assocated ncertanty ( x ). The qantty c ( y) s the sqared standard ncertanty (standard devaton of the probablty dstrbton) assocated wth the measred vale y whch comes from a combnaton of the ncertantes assocated wth all the dfferent effects, x. The sqare of the standard ncertanty s also known as the varance. The Law of Propagaton of ncertantes s dscssed n detal n Secton 3. It can help to wrte t n terms of senstvty coeffcents as n n 1 n c = 1 = 1 j= + 1 ( ) = ( ) + j (, j), (.3) y c x cc x x where the senstvty coeffcent c = f x. The senstvty coeffcent s a translaton from one varable to another. It answers the qeston: how senstve s y to an ncertanty assocated wth x? The law of propagaton of ncertantes s wrtten n ths slghtly complex notaton of two parts to separate two terms: The frst term s the sm of the sqares of the standard ncertantes ( x ) (the sm of the varances) assocated wth each ndvdal effect mltpled by the relevant senstvty coeffcent (the partal dervatve). Ths frst term s what s meant by the descrpton addng n qadratre. The second term deals wth the covarance of correlated qanttes. The covarance s a measre of how mch the two qanttes vary together. See also Secton 3.5. Note that the covarance term covers all pars of dfferent qanttes, e.g.,,,,,, x, x = x, x, the smmaton s ( x x ) ( x x ) ( x x ). Snce the covarance ( ) ( ) ~ 5 ~

14 only over the combnatons where acconts for the opposte cases..6 Classfcatons.6.1 Random and Systematc Effects < j (.e. only half the cases). The n front of ths term Correlaton wll be ntrodced whenever there s somethng n common between two measred vales that wll be combned (.e. two vales that wll be averaged, or two qanttes sed n a measrement eqaton, or vales at dfferent wavelengths that wll be combned throgh nterpolaton or ntegraton). The smplest way to descrbe ths s n terms of random and systematc effects. Random effects are those that are not common to the mltple measrements beng combned. A common example s nose: two measred vales may both sffer from nose, bt the effect of nose wll be dfferent for each of the two measred vales (for example, f nose has ncreased one measred vale, ths provdes no nformaton abot whether any other measred vale s ncreased or decreased by that nose, nor by what extent). Systematc effects are those that are common to all measred vales. If one measred vale has been ncreased as a reslt of a systematc effect, then we can make a relable predcton regardng whether any other measred vale wll be ncreased, and by how mch. For example each tme the dstance s set for an rradance measrement sng a partclar lamp, there wll be a (normally small) error n that dstance. Ths wll eqally affect all measrements of that lamp ntl the next algnment. If mltple measred vales are averaged wthot realgnment, or measred vales at dfferent wavelengths are combned n an ntegral, then the dstance error wll be common to all those measred vales. Ths s a systematc effect. Some effects, sch as nose, are always random; other effects can be ether random or systematc dependng on the measrement process. For example, f three measred vales of a lamp are combned n an average and the lamp s realgned between each measrement, then algnment/dstance s a random effect. If the lamp s not realgned between measrements, then algnment/dstance s a systematc effect. The error n the measred vale de to a random effect wll change from one measred vale to another. In ths case the ncertanty assocated wth the effect may be the same for each measred vale (the probablty dstrbton for the effect s the same for each measred vale), bt each measred vale s ndependent of each other measred vale, as nflenced by ths effect. The nknown random error at each measred vale s an ndependent draw from the probablty dstrbton, meanng that the error de to the random effect s not only dfferent from, bt also ndependent of, the error at any other wavelength. The standard ncertanty assocated wth random effects s sally (bt not always) determned by calclatng the standard devaton of repeated measred vales. The error n the measred vale de to a systematc effect wll be the same from one measred vale to another. The ncertanty assocated wth the effect s the same for each ~ 6 ~

15 measred vale and the error s the same draw from the probablty dstrbton for all measred vales. The standard ncertanty assocated wth systematc effects cannot be determned by repeat measrements, nless the effect s ntentonally altered between repeats (e.g. by realgnng a sorce mltple tmes sng a seres of dfferent extreme bt acceptable algnments 3 n an experment to characterse the mpact of sorce algnment). Fgre 1: Representng a measrement where there s a known correcton, an nknown systematc effect and random effects. Fgre 1 represents a measrement process where there s a known correcton, an nknown systematc effect and random effects. A measrement s made (obtanng the vale represented by the golden crcle at 6). We know of a correcton a systematc bas de to, e.g. a dark readng and apply ths correcton, obtanng the vale of the dotted crcle here abot 3. There s stll an nknown error from the tre vale of zero. If we make many measrements we get the probablty dstrbton fncton shown n ble. The spread of ths, the standard devaton of the normal dstrbton, s the standard ncertanty assocated wth random effects those effects that change from measrement to measrement. Or measred vale s a draw from ths probablty dstrbton fncton. If we take mltple measrements we obtan dfferent draws. The average wll tend towards the vale at the peak of ths dstrbton. When the known correcton s appled, the reslt wll be close to the tre vale, bt dffer from t by an nknown systematc error common to all the measred vales. Ths comes from ts own probablty dstrbton fncton and all measred vales have the same draw from that dstrbton (not shown n the fgre, bt ths wll take the 3 See secton 3.1. ~ 7 ~

16 form of a probablty dstrbton centred at the tre vale wth a standard devaton eqal to the ncertanty assocated wth systematc effects)..6. Type A and Type B The terms Type A and Type B are sed wth ncertanty analyss. Ths se comes from the GUM, whch defnes:.3. Type A evalaton (of ncertanty) method of evalaton of ncertanty by the statstcal analyss of seres of observatons.3.3 Type B evalaton (of ncertanty) method of evalaton of ncertanty by means other than the statstcal analyss of seres of observatons Type A evalaton ses statstcal methods to determne ncertantes. Commonly ths means takng repeat measrements and determnng the standard devaton of those measrements. Ths method can only treat ncertantes assocated wth random effects, for example the ncertanty assocated wth measrement nose. Type B evalaton ses 'any other method' to determne the ncertantes. Ths can nclde estmates of systematc effects from prevos experments or the scentst's pror knowledge. It can also nclde random effects determned 'by any other method'. For example we may model room temperatre by a random varable n the nterval from 19 C to 1 C the temperatre range of the ar-condtonng settngs. Smlarly, we may say that a voltmeter wth dgts after the decmal place has an ncertanty assocated wth resolton of V becase we know the rondng range. It s common to assme that Type A evalaton s for random effects and Type B evalaton s for systematc effects. Ths s generally, bt not always, the case. For example, a Type A method may be sed to determne the ncertanty assocated wth algnment: a lamp may be realgned ten tmes and the standard devaton of those ten measrements sed to determne an ncertanty assocated wth algnment 4. In a later expermental set-p, measrements may be taken at mltple wavelengths and these combned n a spectral ntegral. For that ntegral, algnment s a systematc effect (the lamp s not realgned from wavelength to wavelength) even thogh the determnaton of the assocated ncertanty was performed sng Type A methods. Smlarly, the ncertanty assocated wth a random effect may be estmated from pror knowledge, or a measrement certfcate, and ths by a Type B method..6.3 Absolte and relatve ncertantes The ncertantes gven n the law of propagaton of ncertantes by the symbol ( x ) are always standard absolte ncertantes. The term standard ncertanty means that t s a sngle standard devaton of the probablty dstrbton fncton assocated wth that 4 If ths s done, care mst be taken to avod doble contng any random effect de to, e.g. nose. ~ 8 ~

17 qantty. The term absolte ncertanty means that t has the same nt as the measrand. In other words, f the sgnal s n volts, the absolte ncertanty wll also be n volts. If the dstance s n metres, the absolte ncertanty wll also be n metres. It s common n radometrc calbratons to descrbe relatve ncertantes, wth nts of per cent. The relatve ncertanty s the absolte ncertanty dvded by the qantty,.e. x x. ( ).7 Wrtng abot ncertantes In casal langage we talk abot 'averagng a set of measrements' or 'the ncertanty n the measrement s 0.5 %'. In metrology these words are defned careflly to redce msnderstandng. We cannot 'average a set of measrements' bt we can 'average the measred vales' obtaned from those measrements. The measrement has no ncertanty, there s an ncertanty assocated wth the measred vale. For a non-specalst, sch defntons can seem pedantc, as wth jargon n all felds; bt for a specalst, sch carefl se of words s a sorce of clarty. The words are defned throgh the VIM: the nternatonal vocablary of metrology []. A measrement s made (nstrments set p and vale recorded) of a measrand (a qantty, sch as radance) to obtan a measred vale (e.g. 0.5 W m - sr -1 nm -1 ) wth an assocated ncertanty (e.g. 0.5 %). The VIM defnes measrement as the process of expermentally obtanng one or more qantty vales that can reasonably be attrbted to a qantty The most mportant word here s process: t defnes measrement as the act of measrng. A measrement s not a qantty nor a reslt. The VIM defnes measrand as the qantty ntended to be measred In trn, qantty s the property of a phenomenon, body or sbstance, where the property has a magntde that can be expressed by a nmber and a reference. Ths qanttes are thngs lke length, mass, reflectance, rradance, nstrment gan, etc. When yo measre a qantty, that qantty s the measrand of the measrement. The measrement reslt s defned by the VIM as the set of qantty vales beng attrbted to a measrand together wth any other avalable relevant nformaton ~ 9 ~

18 The "other avalable relevant nformaton" refers to the assocated ncertanty, perhaps expressed drectly, perhaps as a probablty densty fncton, or perhaps mpled by the nmber of dgts provded wth the reslt (the latter provdng less relable nformaton). The qantty vale s a nmber and reference together expressng magntde of a qantty The reference sally means the nt. The measred qantty vale (often shortened to measred vale) s the qantty vale that s the partclar measrement reslt. ~ 10 ~

19 3 Understandng the Law of Propagaton of Uncertantes The Law of Propagaton of Uncertantes s gven n the GUM, and was provded above as Eqaton (.1). Ths secton descrbes some basc concepts behnd ths law and how to apply t. Ths secton s reasonably theoretcal. All the concepts here wll be frther explaned n the examples and case stdes of sbseqent chapters. 3.1 Senstvty coeffcents Central to the law of propagaton of ncertantes (Eqaton (.3)) are the senstvty coeffcents, wrtten as c f =. (3.1) x The senstvty coeffcent s a measre of how senstve the measrand (the reslt),y, calclated from the eqaton Y = f ( X1, X, X3,, X, ) ) s to the npt qantty 5 X. In other words, t answers the qeston: How mch does ths effect nflence the fnal measred vale? The ncertanty assocated wth Y de to X s y = = x. (3.) ( ) y_ deto_ x y:: x x So, for example, f we calclate a sgnal as a lght readng mns a dark readng, we have the eqaton VS = Vlght Vdark. (3.3) The ncertanty assocated wth the sgnal de to the lght readng s V = = V = V ( ) 1 ( ) S VS _ deto_ Vlght VS:: Vlght lght lght Vlght. (3.4) There are three methods for determnng senstvty coeffcents and they are all eqally vald and all approved by the GUM. These are: Mathematcally (dfferentatng the measrement eqatons) Nmercally (modellng throgh an nstrment model n software, or changng the npt parameters to the measrement eqaton) 5 Note, as descrbed n Secton.5 that n formal mathematcal notaton of the Law of Propagaton of Uncertantes, a captal letter s sed to denote a qantty (measrand) and a small letter s sed to denote a specfc measred vale of that measrand. Ths notaton s not sed n later stages of ths book, where tradtonal physcs notaton s sed, e.g. that radance s represented by L. ~ 11 ~

20 Expermentally (changng the effect n the lab and seeng how mch the measred vale changes) In the development of a real-world ncertanty bdget, all three wll be sed. Throgh dfferentaton (mathematcally): Where the measrement eqaton shows a straghtforward relatonshp, then often the smplest method s to dfferentate. Ths s the partal dervatve term n the Law of Propagaton of Uncertantes (n the GUM). Consder the radance of a whte dffser panel llmnated by an FEL 6 lamp. The radance of the dffser tle, vewed at an angle of 45 and for normal ncdence llmnaton, s gven by L EFELβ0 :45 dcal s π dse = (3.5) where L s s the sorce radance, EFEL s the lamp rradance and β 0 :45 s the dffser radance factor. The calbraton dstance for the FEL lamp s d cal and t s set a dstance d se from the dffser. Note that here the wavelength dependence s not explctly descrbed, n practce L ( λ ), E ( λ ) and β ( λ) are all fnctons of wavelength, λ. s FEL 0 :45 Ths, the senstvty coeffcent of the dffser radance de to the rradance of the lamp s c EFEL L E s = (3.6) FEL From Eqaton (3.5) we calclate ths as c EFEL Ls β0 :45 dcal FEL π dse = = E L = E s FEL. (3.7) The frst lne calclates the dervatve. The second lne shows that ths can be expressed more smply. Ths s sgnfcant becase the meanng of the senstvty coeffcent s that the ncertanty assocated wth the dffser radance, de to the ncertanty assocated wth the lamp rradance s = c. (3.8) Ls : EFEL EFEL EFEL Ths means that the absolte ncertanty (nts [W m - sr -1 nm -1 ]) assocated wth the lampdffser radance de to the lamp rradance s the senstvty coeffcent tmes the ncertanty assocated wth the lamp rradance (whch has nts [W m - nm -1 ]). If the smpler, second lne of (3.7) s sed, ths gves 6 FEL s an ANSI standard desgnaton denotng a specfc 1 kw doble-coled tngsten halogen lamp, operatng at 110 V, wth a specfc base. The letters are an arbtrary code and not an acronym. ~ 1 ~

21 L = (3.9) s Ls : EFEL EFEL EFEL Rearrangng ths, gves Ls : EFEL EFEL L s = (3.10) E FEL.e. the relatve ncertanty assocated wth radance of the lamp-dffser (sally wth ncertanty expressed n %) de to the lamp rradance s eqal to the relatve ncertanty assocated wth the FEL rradance (also expressed n %). A smlar relatonshp can be fond for all the other parameters. Note that for the dstance of se (the lamp to dffser dstance) c dse Ls EFELβ0 :45 dcal 3 se d π se = = d Ls =. d se (3.11) And therefore Ls : dse dse L s = (3.1) d se The relatve ncertanty assocated wth the radance of the lamp-dffser (n %) de to the lamp-dffser dstance s twce 7 the relatve ncertanty assocated wth the lamp-dffser dstance (also n %). Throgh modellng (nmercally): Sometmes the most approprate way to determne senstvty coeffcents s throgh modellng. Ths may be from a smple realsaton of the measrement eqaton n a spreadsheet program, or t may be from a consderable pece of software that models the optcal aberratons, pxel cross-talk, etc. for a mltspectral mager. An example of the latter s gven n the sectons below. A smple measrement eqaton or model may be sed to determne the senstvty coeffcents nmercally. Ths may be sefl f the eqaton s dffclt to dfferentate analytcally. Consder as a smple example, the eqaton (3.5). A spreadsheet can be set p to calclate the lamp-dffser radance from the npt parameters. Then each parameter n trn can be altered by ts ncertanty and the change n lamp-dffser radance recorded. It s mportant here that only one parameter s changed at a tme. The am s to determne the senstvty of the calclated reslt to an ncertanty assocated wth a sngle parameter. If 7 The mns sgn n front of the s gnored here becase t wll be sqared. The negatve senstvty coeffcents wll be needed when correlaton s taken nto accont. ~ 13 ~

22 mltple parameters are changed at the same tme, then ths wold mply a correlaton between them (e.g. that ncreasng one atomatcally ncreases the other). Changng one parameter at a tme provdes a senstvty coeffcent to that sngle parameter. An alternatve modellng approach, Monte Carlo smlaton, s descrbed n Appendx 9Appendx B. In Monte Carlo smlaton all the parameters are altered smltaneosly, bt by a random amont, wth the random nmber drawn from the probablty dstrbton descrbed by the ncertanty. Many (sally tens of thosands of) smlatons are rn; each smlaton havng dfferent ncertanty draws. The advantages of Monte Carlo smlaton are that correlatons can be more easly dealt wth, that statstcal parameters can be determned from the end reslts and that the probablty dstrbton fnctons do not have to be Gassan (and therefore realstc probablty dstrbton fnctons can be nclded). The man dsadvantages are the comptatonal tme and the fact that ndvdal senstvty coeffcents are not determned as everythng s altered smltaneosly. Ths means that the expermentalst does not easly obtan an nderstandng of what the most sgnfcant ncertantes are. Throgh laboratory testng (expermentally). In some cases t s not possble to wrte the fll measrement eqaton. For example, the senstvty to a lamp algnment wold have to be wrtten as some fncton of the tlt and roll angles of the lamp, as well as the vertcal and horzontal dsplacement of the lamp. Smlarly, the lamp stablty effect s a fncton of the tme snce calbraton, whether the lamp has been transported, etc. These fnctons cannot be wrtten ot as explct eqatons that can be dfferentated or sed n a mathematcal model. Instead, the senstvty s determned expermentally. Expermental methods are based on repeat measrements where the effect s changed and the effect on the measred reslt s analysed. Ths s very good for statons where the effect can be easly controlled expermentally and there s no easy mathematcal model relatng the effect to the measrand. The senstvty to algnment, for example, may be determned by realgnng the lamp ten tmes and comparng the standard devaton of those ten measred vales wth the standard devaton of ten measred vales where the lamp s not realgned between measrements 8. Ths wold be an example of a Type A determnaton of an ncertanty (whch, n the fnal measrements cold be a systematc effect f the lamp s not realgned then, or a random effect, f the lamp s realgned). Sometmes, a more systematc approach may be preferred. If the room temperatre may vary from 19.5 C to 0.5 C, then t may be approprate to determne an nstrment s responsvty (e.g. spectrometer s gan) at 17 C, 18 C, 19 C, 0 C, 1 C, C to check the senstvty to 8 The ncertanty assocated wth the algnment wold then be calclated as ( ) 1 ~ 14 ~ = s s, so as algn realgned not_realgned not to doble cont the straghtforward measrement repeatablty. Note also that the measred vales obtaned throgh mltple measrements wthot realgnng the lamp are correlated wth each other becase of the common algnment.

23 temperatre and whether ths s lnear across a temperatre range wder than that whch s lkely n practce. Ths can be done, for example, by wrappng the spectrometer wth soft ppng, throgh whch water at each temperatre n trn s sent. The spectrometer s set to vew a stable sorce whle ts temperatre s changed and the change n readngs s determned. Consder the case of estmatng the senstvty of the lamp rradance to lamp crrent by expermentally varyng the crrent. If we make too small a change n lamp crrent then we wll not get a reasonable estmate of the effect for two reasons. Frst, we wll not be able to control the crrent sffcently well to get a clear separaton of the two measred crrents. For example f the crrent s stable to (8.100 ± 0.004) A, then makng a measrement at (8.104 ± 0.004) A wold case too mch overlap between the actally provded crrents (n both statons a crrent of 8.10 A cold natrally arse). For ths reason t s sensble to change the crrent by at least ten tmes the measred effect. Secondly, too small a change may mean any effect s hdden behnd the nose de to other effects. For example, a change of 40 ma may affect the measred rradance mperceptbly compared to the more domnant effect of detector nose. Ths, t s mportant that before sch expermental determnatons of a senstvty coeffcent are carred ot, the natral repeatablty of the measrements s determned. In ths case t wold be approprate to make 5 consectve measrements wth a lamp crrent of A, before repeatng, agan perhaps mltple tmes, wth a lamp crrent of A. If the change as a reslt of the change n crrent s mch smaller than the nose cased by other effects, then there are several optons, we cold consder the ncertanty assocated wth stablty n the lamp crrent to be an nsgnfcant ncertanty component average several measrements at each of the crrents sed n the senstvty coeffcent nvestgaton to redce the mpact of the nose on the measrements ncrease the crrent step sed ntl an effect s seen, for example the effect may be smaller than the nose wth a 40 ma change, bt larger than the nose for a 100 ma change. Any of these approaches may be vald. As well as the rsks of too small a crrent change, t s necessary to consder the rsk of too large a crrent change. As well as the extreme examples, e.g. f the lamp crrent s ncreased too hgh the lamp flament may be destroyed or ts calbraton may be changed, a more modest and safe change may also provde nrelable reslts. The process descrbed above: makng a change ten tmes larger than that expected and dvdng the observed change n the rradance by ten to obtan a senstvty coeffcent, assmes that the process of change s lnear. Ths s not the case wth a lamp crrent: a doblng of the lamp crrent does not lead to a doblng of lamp rradance. For small changes, of arond 100 ma, lnearty may be a reasonable approxmaton, bt for larger changes, t s not. ~ 15 ~

24 It s therefore sensble to make measrements at a slghtly smaller and a slghtly larger crrent, ths provdng three data ponts and gvng a better nderstandng of how lnear the lamp's behavor s over the range. Smlar consderatons apply for all senstvty coeffcent examples. It s mportant to make the smallest possble change sch that the observed effect can "come ot of the nose",.e. be seen as a real effect, bt wthot changng the nderlyng behavoral relatonshp that s beng nvestgated. 3. Addng n qadratre From the senstvty coeffcents we determne the ncertanty assocated wth the measrand (the answer calclated from the measrement eqaton) de to each component n trn. The Law of Propagaton of Uncertantes then combnes these. If there s no assocated correlaton (.e. f all the components are ndependent of each other), then only the frst half of Eqaton (.3) needed n c = 1 ( ) ( ) y = c x. (3.13) Ths adds ncertantes n qadratre. That means that the ncertanty assocated wth the measrand de to each component n trn s sqared, they are smmed and fnally a sqare root s taken. The reason that ncertantes are added n qadratre s that t s statstcally mprobable that all the errors (.e. all the draws from the probablty dstrbton fnctons descrbed by the ncertantes) are all at the extreme of the probablty dstrbton fncton and n the same drecton. It s more lkely that some wll ncrease and others decrease the measred vale and that some errors wll be smaller than ther average vale and others larger. Addng n qadratre provdes a far combnaton that s statstcally robst 9. Consder two of the examples gven n the prevos secton. Frst the sgnal calclated from a lght and dark readng (eqaton (3.3)). The senstvty coeffcents are V V s lght V = = V s 1; 1 dark. (3.14) Therefore, ( ) ( 1) ( ) ( 1) ( ) ( ) ( ) V = V + V = V + V. (3.15) S lght dark lght dark Note here that these are absolte ncertantes and all n the nts of the measrand, e.g. volts (or dgtal nmbers, or amperes). 9 It s a prncple of statstcs that t s varances that are added n combnng effects. ~ 16 ~

25 For the lamp-llmnated dffser tle, eqaton (3.5), the calbraton dstance s assmed to have no assocated ncertanty (t s gven on the provded certfcate), and the other terms have the relatve senstvty coeffcents calclated as for eqatons (3.7) and (3.11), ths ( ) ( ) ( β ) ( ) = ( ) + ( β ) + ( ) ( ) FEL 0 :45 se L E d = + + ( ) L E β d FEL 0 :45 se L E d rel rel FEL rel 0 :45 rel se (3.16) The second lne provdes a verson n terms of relatve ncertantes, expressed n nts of percent. 3.3 Takng an average of ndependent measred vales Applyng the law of propagaton of ncertantes Consder the mean, M, of three ndependent 10 calclated as measred vales A, B and C. Ths s M A+ B+ C = (3.17) 3 The senstvty coeffcents are therefore all M M M = = = A B C 1 3. (3.18) We can also assme that these three measred vales have the same assocated ncertanty 11 ( A) = ( B) = ( C) ( x). (3.19) Therefore, applyng the law of propagaton of ncertantes ( ) = ( ) + ( ) + ( ) M A B C = 3 3 ( ) x = 3 ( x). (3.0) 10 Independent here means that the measred vales are taken separately and have no assocated correlaton. If there are systematc effects, ths shold be consdered separately (Secton 3.4). 11 If the three measrements are dentcal, t s a reasonable assmpton that they have the same assocated ncertanty. If they are not dentcal (taken n dfferent ways) and there are known dfferences n the ncertanty assocated wth dfferent methods, then t wold be sensble to se a weghted mean, rather than a smple mean. ~ 17 ~

26 Hence, takng a mean of three readngs redces the ncertanty assocated wth a sngle readng by the sqare root of three. Consder the sgnal calclated from the lght and dark readngs, bt ths tme we assme N lght readngs and M dark readngs. Ths N M 1 1 V = V V. (3.1) s lght, dark, j N = 1 M = 1 To apply the law of propagaton of ncertanty to Eq. (3.1), we need to determne the senstvty coeffcents. These are c Vlght, c Vdark, j V 1 = =, = 1,, N V N lght, V 1 = =, j = 1,, M. V M dark, j (3.) If we treat the N lght readngs as random draws from the same probablty dstrbton fncton (.e. they are ndependent realsatons of the lght readng), then we can say that for all readngs, lght, = ; (3.3) lght.e. they have the same ncertanty. And smlarly, dark, j =. (3.4) dark Applyng the law of propagaton of ncertantes to Eq. (3.1), we obtan N M 1 1 V = V + lght, Vdark, j = 1 N j= 1 M 1 1 V = V + lght Vdark N M N M V V lght V dark = + N M. (3.5) In other words, n order to calclate the ncertanty assocated wth the measred sgnal, the ncertanty assocated wth a sngle lght readng s dvded by the sqare root of the nmber of lght readngs and added n qadratre wth the ncertanty assocated wth a sngle dark readng dvded by the sqare root of the nmber of dark readngs. Note frst, that V lght and V dark are absolte ncertantes, wth nts ([volts], or [Dgtal Nmbers]). If the lght sgnal has a vale of 5000 DN ± 500 DN, and the dark sgnal has a vale 1000 DN ± 10 DN, and assmng these ncertantes are already redced by the sqare ~ 18 ~

27 root of the nmber of measrements averaged, then the sgnal s ( ) DN ( ) ± DN, or 4000 DN ± DN. Expressed n relatve terms, ths ncertanty s =.1% Averagng enogh readngs In order to apply the law of propagaton of ncertantes, t s necessary to obtan a good estmate of and. Assmng N and M are sffcently large 1, then the standard V lght V dark devaton of the N and M readngs, gves a good estmate of V lght and V dark respectvely. Where, for example, 1000 readngs are averaged sng an atomatc data acqston process, then the standard devaton of those readngs s a good estmate of the ncertanty assocated wth the ndvdal measred vales, de to random effects 13 (effects that vary from one readng to the next) becase there are enogh readngs. Sometmes, however, N and M are relatvely small. Ths s tre wherever a manal process s nvolved (for example where a lamp s realgned from measrement to measrement), or when a measrement s slow. When N and M are relatvely small, then the standard devaton calclated from the measred vales provdes an nrelable estmate of the ncertanty assocated wth each ndvdal measred vale. The GUM deals wth ths throgh the Welch-Satterthwate Eqaton (see Secton 4.10). The planned revson to the GUM wll, however, treat ths n a dfferent way, that s, perhaps more helpfl for an nttve nderstandng of whether sffcent measrements have been taken. In the GUM revson, the sqared standard ncertanty assocated wth the mean of the lght readngs wold be calclated from the standard devaton, s lght, sng lght,mean N 1 s. N 3 N lght = (3.6) Ths eqaton estmates the standard varance (sqared ncertanty) for the fll dstrbton based on a sqared standard devaton calclated from the N measred vales. It ncreases t a bt becase a few ponts are may nderestmate the standard devaton. Note that the ( 1 N ) term here s the same term as n Eq. (3.5). Clearly, the larger the vale of N, the closer the ncertanty assocated wth the mean s to the standard devaton of the measrements dvded by N. If N = 5, then the standard 1 Bt not so large that drft domnates, see the dscsson on Allan Varance n Secton Note that 1000 readngs taken n a short tme perod wll have a standard devaton that depends only on short-term random nose. Any longer term flctatons wll not be nclded, nor wll any effects that are common to all those 1000 readngs, sch as those de to e.g. crrent settngs on a lamp sed, or the algnment of a dffser panel. It may be that even where 1000 readngs are taken smltaneosly, there wll need to be a smaller nmber (say 5-10) separate averages of those 1000 readngs taken over a longer tme nterval, or wth lamps trned off and back on, or wth the dffser panel realgned. There s almost always a set of measrements whose standard devaton s determned for a very small nmber of measred vales. ~ 19 ~

28 devaton s mltpled by ~1.41, f N = 10, then t s mltpled by ~1.13, and f N s 5, then t s mltpled by ~1.04. If N s 1000, then the standard devaton s mltpled by It s always preferable to se the best possble estmate of the standard devaton, from as many measrements as possble, so that the correcton of Eq. (3.6) s as small as possble. It may not be practcal, however, to make 5 or more measrements on every occason. Under those crcmstances, there are several thngs that cold be done to obtan a better estmate of the standard devaton, ether ndvdally or combned: In the case of a spectral measrement, data from adjacent wavelength ponts can provde a better estmate. Frst, determne a standard devaton on a wavelength-bywavelength bass. Ths wll have some strctre becase of the random nose. Smooth ot ths strctre sng the data from neghborng wavelengths. If N = 5 at a sngle wavelength, then sng data from 5 neghborng wavelengths to smooth ot the standard devaton wll effectvely ncrease N to somethng closer 14 to 5. Drng a commssonng phase for yor nstrmentaton take 5 readngs. Compare the standard devaton of the 5 readngs taken on a sngle day to that of the 5 readngs taken drng the commssonng phase to check that t s stll vald, bt se the standard devaton of the 5 commssonng readngs as yor best estmate of the ncertanty of a sngle readng. Determne the dfferences of the 5 readngs to the average (mean) of those fve readngs. Do ths on 5 sccessve measrement days. Take the standard devaton of the dfferences for all fve days 15. Usng the dfference from the daly mean takes ot the day-to-day varaton and looks only at the varablty. Any of these optons wold generally be preferable to ncreasng the ncertanty sng Eqaton (3.6). Note that these wll gve yo an estmate of the standard ncertanty assocated wth a sngle readng. The ncertanty assocated wth the mean of today s 5 readngs wll be the ncertanty assocated wth a sngle readng dvded by the sqare root of 5 (the nmber of measrements taken today) Allan devaton/allan varance The dscsson above assmes that t s always preferable to average more and more measrements. Eqaton (3.5) redces the ncertanty by the sqare root of the nmber of measrements taken. Consder an extreme example f measrements are taken every mnte for a week, wold ths mean the ncertanty was effectvely zero? Inttvely we nderstand that ths s not the case and there are two reasons for ths. The frst s that there may be systematc effects common to all the measrements. The second s becase the nstrments are lkely to drft over the week. There comes a tme when the nose s no longer 14 How mch closer depends on the smoothng algorthm sed. Do not worry too mch abot the exact vale of N, the pont here s to make t sffcently large to gve a good estmate and for the correcton of Eq. (3.6) to be small. 15 Agan N = 5 n ths example. ~ 0 ~

29 whte nose that can be averaged, bt some form of drft, the effect of whch jst gets worse wth averagng over a longer and longer perod. The Allan Varance [3] s a means of determnng whether the nose s whte nose or drft (or somethng else). Calclatng the Allan Varance reqres a seres of measrements at reglar tme ntervals t. The Allan Varance calclates the dfference between sccessve data ponts, sqares them and takes the average, before dvdng the answer by two: σ 1 1 N ( t) = ( y y ) + ( y y ) + + ( y y ) N N 1 (3.7) The pars of data are then averaged, so now yo have half as many data ponts wth a tme nterval of t and the Allan Varance s calclated agan. Ths s repeated, each tme averagng the pars that were sed before. After ths has been completed, the data s plotted wth the vertcal axs gvng the Allan Varance and the horzontal axs gvng the tme nterval. Both axes shold be represented on a log scale. For whte nose, the Allan devaton les on a straght lne on ths log-log plot, wth a slope of -0.5, correspondng to a redcton n the ncertanty accordng to the sqare root of tme. However, f the sgnals are accompaned by 1 f nose or drft (random walk), the slope wll become postve. Ths sggests that averagng for tmes greater than the Allan Devaton mnmm ncreases rather than decreases ncertanty. σ(τ) AlaVar 5. Allan STD DEV tme / s Prodced by AlaVar 5. ADEV Lne Ft Lower Bond Upper Bond Fgre Otpt plot from the Allan Varance software showng the Allan varance of a very stable sgnal Free software for evalatng the Allan Varance s avalable 16. Ths provdes plots as shown n Fgre 3. In ths example, the sgnal s stable wth whte nose for p to ~6000 s, after whch drft domnates and frther averagng makes the staton worse. From ths graph we can see that f we average for 100 s the ncertanty assocated wth the mean s ~0.07 %, f we average for 1000 s, the ncertanty assocated wth the mean s ~0.03 % ~ 1 ~

30 3.4 Takng an average of partally correlated measred vales Correlaton Correlaton s ntrodced whenever there s somethng n common between mltple measred vales de to a common effect n the measrement process. When there s somethng n common, then that correlaton needs to be taken nto accont. There are two ways of dealng wth that correlaton n the law of propagaton of ncertantes. The frst s to descrbe the correlaton explctly n the measrement eqaton; the second s to se the second half of the law of propagaton of ncertantes. Both are descrbed here Analytcal approach by modellng ot the correlaton: absolte ncertantes Consder the staton where a lamp rradance s determned by averagng three measred vales of ts rradance. The measred vales are taken seqentally and the lamp s not realgned between measrements. We can model the reslt of the th measrement as Here, E T, s the (nknown) tre rradance of the lamp. E = E + R + S. (3.8) T R s the random error n the th measrement. It s a draw from the probablty dstrbton descrbed by the ncertanty ( R ) ; the ncertanty assocated wth the random effects. Smlarly, S s the systematc error n all the measrements. It s a draw from the probablty dstrbton descrbed by the ncertanty ( S ) ; the ncertanty assocated wth systematc effects. The expectaton vales of R and S are zero (we are not applyng a correcton). However there s an ncertanty assocated wth the measred vale de to these effects. The R error comes from all random effects probably predomnantly de to measrement nose, bt also sorce stablty, temperatre flctatons etc. The S comes from all systematc effects that ddn t change between measrements; for example, algnment or the calbraton of the reference lamp. If these ncertantes are determned separately, then the ncertanty assocated wth S can be calclated as the qadratre sm of the ndvdal ncertantes. The mean of the three measred vales s whch we can wrte, by applyng (3.8) as ( ) E = E + E + E (3.9) M R + R + R = + + (3.30) EM ET S We now have an eqaton wth for ndependent varables: R1, R, R3, S and, snce these are ndependent,.e. ncorrelated, we can apply the frst half of the law of propagaton of ncertantes ~ ~

31 ( ) R ( EM ) = + S 3 ( ). (3.31) The ncertanty assocated wth random effects s redced by the sqare root of the nmber of measrements; the ncertanty assocated wth systematc effects remans nchanged by the averagng. Ths s as we may nttvely expect. No matter how many measrements we make, we cannot redce the ncertanty assocated wth systematc effects by averagng Analytcal approach by calclatng covarance: absolte ncertantes In order to nderstand the second part of the law of propagaton of ncertantes t s worth dervng Eqaton (3.31) sng the fll law of propagaton of ncertantes, Eqaton (.1). To E, E assocated wth a par of measred do ths, we need to determne the covarance ( j) vales, n ths case a par of rradance vales. The covarance s a measre of the ncertanty common to the two measred vales. And wth the model Eqaton (3.8) ths s ( S ). Therefore ( j) ( ) E, E = S ; j. (3.3) Ths, ncldng the senstvty coeffcents from (3.18), the term sed n the second part of Eqaton (.1) s EM EM 1 1 E E S E E 3 3 j (, j) = ( ). (3.33) The ncertanty assocated wth any ndvdal rradance vale s gven by ( ) ( ) ( ) E = R + S. (3.34) The law of propagaton of ncertantes n fll for ths example s M 1 3 ( ) ( ) ( ) ( ) E = E + E + E ( E1, E) + ( E1, E3) + ( E, E3) (3.35) Combnng (3.33), (3.34) and (3.35) gves ~ 3 ~

32 1 1 1 M ( ) = 3 ( ) + 3 ( ) + 6 ( ) E R S S ( ) ( ) R 1 = R = + 3 ( S). ( S) (3.36) Whch, as expected, s the same as Eqaton (3.31) Analytcal approach by modellng ot the correlaton: relatve ncertantes The example gven above assmes that the systematc error s an addtve effect, wth the error and ncertanty havng the same nts as the measrand (.e. [W m - nm -1 ]). In radometrc measrements ncertantes are more lkely to be relatve ncertantes n % and errors have a mltplcatve effect. Therefore nstead of Eqaton (3.8), the error model s better descrbed by ( )( ) E 1 1 = ET + R + S (3.37) where the error terms R and S have an expectaton vale of nty (one), and a relatve ncertanty assocated wth that. In ths case the mean s 3+ R1+ R + R3 EM = ET ( 1+ S) 3. (3.38) Once agan, we have an eqaton wth for ndependent varables, so we can se the frst part only of the law of propagaton of ncertantes. The senstvty coeffcents are EM E = S + M ( 1 S) E ( 1+ S) EM T EM = = R 3 3+ R + R + R 1 3 (3.39) and the law of propagaton of ncertantes gves E M E ( M) = ( 1) + ( ) + ( 3) + 3+ R1+ R + R3 1+ S M ( ) ( ) E R R R S And therefore, makng the reasonable assmpton that ( R ) ( R ) ( R ) ( R) = = =, 1 3. (3.40) ( M ) 3 ( ) E ( 3+ R + R + R ) ( ) E R S = + 1+ S M 1 3 (3.41) ~ 4 ~

33 To take ths a step frther, we have to nderstand that f the expected vale of R = 0, then the term R1+ R + R3 = 0. In takng the average, that s the assmpton that we are makng. Smlarly 17, S = 0 ; ths ( ) ( R) E = + E M M 3 ( S) (3.4).e. for a mltplcatve model wth relatve ncertantes, the relatve ncertantes behave exactly as the absolte ncertantes for an addtve model n (3.31). 3.5 Covarance term n the law of propagaton of ncertantes Covarance In order to apply the law of propagaton of ncertantes, we need to deal wth correlaton n one of the followng manners: By wrtng the correlaton explctly nto an error model and rearrangng the measrement eqaton so that the qanttes are no longer correlated, ths s the approach descrbed n Sectons 3.4. and By calclatng the covarance from an error model. Ths s the approach descrbed n Secton above and 3.5. By determnng the correlaton expermentally or nmercally as descrbed n Secton By obtanng a range of possbltes for the covarance as descrbed n Secton Note that as wth Type A and Type B evalatons of ncertanty, the frst and second of these are a Type B evalaton of covarance the covarance s estmated by an explct measrement model. The thrd method s a Type A evalaton of ncertanty statstcal method are appled to the data tself to estmate the covarance. The fnal method does not estmate the covarance, bt t does gve the opportnty to consder t neglgble or sgnfcant and to decde whether frther analyss s reqred. In Secton 3.4 the emphass was on averagng smlar measred vales (e.g. combnng mltple vales for the lamp rradance). Here correlaton related to whether those ndvdal measred vales were obtaned nder smlar or dfferent condtons, e.g. was the lamp realgned between measrements (f so, lamp algnment was a random effect wth no assocated correlaton, f not, then lamp algnment s a systematc effect whch ntrodces a correlaton). 17 Ths s the assmpton we are makng, that or best estmate of the systematc effect s that there sn t one. If we had a better estmate, we wold remove t wth a correcton. After all correctons have been appled, there s some nknown systematc error that s as lkely to be postve as negatve. We take t as havng an expected vale of 0 wth an ncertanty assocated wth that of ( S ). ~ 5 ~

34 It s also mportant to estmate correlaton between measred vales from dfferent parameters. For example, consder a spectral ntegral (from for example the convolton of a spectral response fncton and a scene radance). Here vales from dfferent wavelengths are combned. Some components n the ncertanty bdget (sorces of ncertanty) wll be common from one wavelength to the next, and other components wll vary from wavelength to wavelength. Correlaton can also occr between effects that are dfferent. For example f a spectrometer and a flter are both temperatre senstve, then the responsvty of the spectrometer may be correlated wth the flter transmttance throgh the laboratory temperatre. Sometmes sch correlatons can be explctly descrbed n a measrement eqaton (e.g. by ncldng a term that s a fncton of temperatre n both places), and then the Type B methods can be sed for these examples too. Sometmes sch a correlaton can only be estmated by Type A methods, from a statstcal analyss of the measrement data tself Covarance from an error model The covarance can be calclated from an error model by seeng what terms are common to e.g. eqatons beng combned or measred vales beng averaged. Sometmes the correlaton can be wrtten explctly so that the terms can be treated as ndependent, as n Sectons 3.4. and There are occasons, however, where t s essental to descrbe the covarance analytcally and n the form of a covarance matrx (See Appendx A). A covarance matrx s needed where data s manplated throgh a modellng process, or, for example, a least sqares algorthm s sed. In these cases textbooks can provde nformaton on how to manplate a covarance matrx n, for example, the least sqares algorthm. What s mssng s how to form a covarance matrx from a typcal expermental ncertanty bdget n the frst place. To form sch a covarance matrx t s necessary to form an error eqaton, smlar to that of Eqaton (3.8). In sch an eqaton, R represents the nknown random error whch s an nknown draw from the probablty dstrbton descrbed by the ncertanty assocated wth (a combnaton of) random effects. To obtan that ncertanty t s necessary to add n qadratre the ncertantes assocated wth ndvdal random effects (e.g. nose, sorce stablty, etc). Smlarly, S represents the nknown systematc error whch s an nknown draw from the probablty dstrbton descrbed by the ncertanty assocated wth (a combnaton of) systematc effects. To obtan that ncertanty t s necessary to add n qadratre the ncertantes assocated wth ndvdal systematc effects (e.g. system algnment, calbraton certfcate vales etc). For the error model of the form of (3.8) the covarance s gven by (3.3). In ths case the ncertantes are n the same nts as the measred vale. Where there s a relatve model and ncertantes n percent, the covarance has to nclde the actal vale. Ths for the relatve error model (3.37) the covarance s ( ) ( ) ( ) ( ) E, E = S E S E = S EE ; j. (3.43) j j j ~ 6 ~

35 .e. the covarance s the sqared relatve ncertanty that s common (and n percent) mltpled by the rradance vales at the two vales that are beng combned. Error models can be made as complex as reqred to descrbe the correlaton between measred vales. Consder, for example the more complex error model ( )( )( ) E = ET + R + S + cf + r + s (3.44) whch has a random mltplcatve error, R, a systematc mltplcatve error, S, and a systematc mltplcate error, F, that has a measrement dependent senstvty coeffcent 18 c, as well as two addtve errors one random, r and one systematc, s. In ths case the covarance s gven by ( j) ( ) j ( ) j j ( ) E, E = S EE + F cc EE + s ; j (3.45) The random effects are ndependent and do not contrbte to the covarance. The relatve ncertantes assocated wth mltplcatve systematc effects are mltpled by the measred vale to convert them to absolte ncertantes, and where approprate the senstvty coeffcent. The absolte ncertantes assocated wth addtve systematc effects are sed drectly Estmatng the covarance from expermental and modelled data In some statons t s approprate to estmate the covarance from data. Ths wold be the case when yo assme there mst be some correlaton between separate varables. For example, both an nstrment s responsvty and a flter s transmttance may be senstve to room temperatre, and therefore the two parameters may be correlated f they were determned at the same tme. In ths case, the covarance can be calclated statstcally from the pars of data. The correlaton coeffcent, r( xy, ), s calclated from (, ) r xy where n s the nmber of data pars ( X, Y ) 1 X X Y Y = n s s n 1 = 1 X Y (3.46), X s the mean of the x vales and Y s the mean of the y vales and s X,sY are the standard devatons of the x vales and y vales respectvely. The covarance s then calclated as (, ) x ( ) ( yr ) ( xy, ) xy = (3.47) 18 For example, f the dfferent vales are at dfferent wavelengths, ths cold correspond to somethng lke lamp crrent whch has an assocated ncertanty and an error n lamp crrent wll affect all wavelengths n the same drecton, bt wll have a bgger mpact on the short wavelengths than on the long wavelengths. ~ 7 ~

36 where, as sal, the ncertantes correspond to the absolte standard ncertantes (n the same nts as the measred vales f relatve standard ncertantes are known, these shold be mltpled by the average measred vales). Ths eqaton can also be sed to determne the covarance of a par of data ponts obtaned throgh Monte Carlo smlaton, see Appendx B Range of possbltes for correlaton Generally speakng, there are three extreme statons: Entrely ncorrelated data (the term r( xy, ) n Eq. (3.47) s 0) Entrely correlated data (the term r( xy, ) n Eq. (3.47) s +1) Entrely correlated data (the term r( xy, ) n Eq. (3.47) s -1) If t s not clear what the correlaton coeffcent s, t s approprate to consder the extreme cases and say that the ncertanty les wthn the range of those extremes. Where the senstvty coeffcents for both parameters are postve (.e. a postve error n each parameter means a postve error n the reslt calclated from the measrement eqaton), or both negatve (.e. a negatve error n each parameter means a postve error n the reslt), then correlaton wll always ncrease the ncertanty relatve to ncorrelated data. Where the senstvty coeffcents are opposte sgns (one s postve and the other negatve), then the correlaton wll decrease the ncertanty relatve to ncorrelated data. Consder the very smple eqaton x= x + x (3.48) meas whch can represent, for example, a dstance between two apertres beng a measred dstance pls an offset (e.g. for the thckness of the apertre f the measrement s to the x x meas =+ 1. What ths back srface). Here the senstvty coeffcent s +1 for both terms ( ) means s that correlaton between the two vales represents makng the same error twce and t wll ncrease the assocated ncertanty. Therefore the worst case scenaro s for the two terms to be entrely correlated and the best case scenaro s for them to be entrely ncorrelated. For ths eqaton, the fll law of propagaton of ncertantes s offset n n 1 n f f f c = 1 = 1 j= + 1 j = xmeas + xoffset + xmeas xoffset ( ) = ( ) + (, j) x x x (3.49) y x x x Note that, ( meas, offset ) r( x, x ) = 1, ( x, x ) ( x ) ( x ) meas offset ( ) ( ) ( ),. x x s gven by Eqaton (3.47), and snce here we are assmng =. meas offset meas offset On the other hand, consder the smple eqaton ~ 8 ~

37 Vsgnal = Vlght Vdark (3.50).e. that a sgnal s a lght readng mns a dark readng. Here, the senstvty coeffcent s +1 for the lght readng and -1 for the dark readng. What ths means s that correlaton between the two vales wll be cancelled ot and t wll decrease the assocated ncertanty. Ths makes sense, becase f, for example there was a common offset error n both the lght and dark readngs, sbtractng the dark readng wll remove the error. Ths, the fll law of propagaton of ncertantes s n n 1 n c f f f y x x x ( ) = ( ) + (, j) x 1 x 1 1 x = = j= + j (3.51) = ( Vlght ) + ( Vdark ) ( Vlght, Vdark ), where the mns sgn comes from the negatve senstvty coeffcent. Note that, ( xlght, xdark ) s gven by Eqaton (3.47), and snce here we are assmng r( xlght, x dark ) = 1, ( xlght, xdark ) = ( xlght ) ( xdark ). ~ 9 ~

38 4 The steps to an ncertanty bdget 4.1 Steps Ths corse consders eght steps to an ncertanty bdget. Other wrters have sggested somewhat dfferent steps; however, what s common s a general framework arond three areas: The frst few steps relate to nderstandng the problem The second set of steps relate to determnng the formal relatonshps the measrement eqaton and senstvty coeffcents The fnal steps relate to dong the mathematcs and propagatng the ncertantes Most ncertanty textbooks and tranng emphasse the fnal steps the Law of Propagaton of Uncertantes, and how to do the mathematcs n calclatng senstvty coeffcents and propagatng ncertantes. There s often very lttle gdance on how to do the early stages, even thogh they are often both the most challengng and, argably, the most mportant part of ncertanty analyss. Ths s becase the technqes are sally very specfc to ndvdal measrement felds. Fortnately, the sklls reqred for the frst step are the sklls that an expermentalst who nderstands hs or her measrement faclty wll natrally have. Ths corse attempts to provde examples, qestons and ways of approachng a problem that each partcpant can apply to hs or her own laboratory. The steps are ntrodced here, and then appled to case stdes n the sbseqent sectons of ths textbook. Understandng the problem o Step 1: Descrbng the traceablty chan o Step : Wrtng down the calclaton eqatons o Step 3: Consderng the sorces of ncertanty Determnng the formal relatonshps o Step 4: Creatng the measrement eqaton o Step 5: Determnng the senstvty coeffcents o Step 6: Assgnng ncertantes Propagatng the ncertantes o Step 7: Combnng and propagatng ncertantes o Step 8: Expandng ncertantes 4. Step 1: Descrbng the traceablty chan The prpose of the frst three steps s to get a very clear nderstandng of the ncertanty problem. The emphass shold be on obtanng the overvew and seeng where the sorces of ncertanty come from. It s easy to become overwhelmed at ths stage and to get worred abot whether an ncertanty s beng doble conted, what mght be beng mssed ot (the ~ 30 ~

39 nknown-nknowns), beng concerned that t s not clear whch step an ncertanty component comes n at. The frst stage s to determne the traceablty rote. That s to show, preferably dagrammatcally, the rote from the measrements beng performed to the SI (or, f approprate, commnty reference). For example, consder calbratng an nstrment that measres rradance (e.g. a down wellng rradance meter for ocean color). The nstrment may be calbrated n a laboratory n comparson wth an FEL lamp. An example traceablty chan s provded here. Here, a test and measrement laboratory (perhaps the nstrment manfactrer) obtans an FEL lamp from a Natonal Metrology Insttte (NMI). In order to avod over-se of the NMI-provded lamp, the test and measrement laboratory may then se t to calbrate a workng standard lamp, whch s then sed to calbrate the nstrment. Fgre 4 Smple traceablty chan for an FEL lamp calbraton of an nstrment Ths process obtans the herarchcal calbraton chan. The am of the herarchcal approach s to work systematcally from the end to the begnnng p all the necessary branches of the traceablty tree 19. How far back ths process s taken depends on where, for the specfc measrement prposes, the vale provded by someone or somethng else can be trsted. For feld measrements, ths pont of trst may be the certfcate provded by a calbraton and testng laboratory (especally f that certfcate s athorsed by an accredtaton body). In a calbraton laboratory ths may be the certfcate provded by the natonal measrement nsttte. In a natonal measrement nsttte ths may be the prmary SI nstrment (for radometry most often a cryogenc radometer). Sch a reference provdes the begnnng of the herarchcal calbraton chan and the measred vale for whch the ncertanty bdget s beng prodced provdes the end of the chan. It s not necessary to make the chan go all the way back to fndamental constants, ncldng the fll traceablty chan wthn, for example, an NMI. It s necessary to go back to the pont of trst the pont for whch yo have a calbraton, wth calbraton certfcate ncldng a fll ncertanty statement from an accredted laboratory. 19 The traceablty may be a chan f there s a sngle artefact that s calbrated n comparson to a reference artefact that was calbrated by a hgher ter laboratory, etc. There may be a tree f for example a qantty s calclated by combnng electrcal, optcal, thermal and dmensonal measrements where traceablty wll be ndependently to SI for each of those parameters. ~ 31 ~

40 Qestons to ask yorself: What s the traceablty chan for my measrements? What references do I se? How were the references set p for the calbraton? Am I relyng on other secondary measrements (temperatre, tme, dstance)? How are they calbrated? Are there ntermedate steps f I m comparng two sorces (lamps), what detector am I sng? If I m comparng two detectors, what sorce am I sng? How far back do I need to go before I read the answer off a certfcate? 4.3 Step : Wrtng down the calclaton eqatons Each arrow n the traceablty chan s lkely to be calclated by combnng dfferent vales n a calclaton eqaton. For example, consder the radance of a dffser tle llmnated by an FEL lamp. The radance of the dffser tle, vewed at an angle of 45 and for normal ncdence llmnaton s gven by: EFELβ0 :45 Ls = (4.1) π where, L s s the sorce radance, E FEL s the lamp rradance and β 0 :45 s the dffser reflectance factor. Eqaton (3.5) forms the bass of the measrement eqaton and s the calclaton eqaton;.e. the eqaton sed to calclate the answer n the laboratory. The calclaton eqaton wll be expanded to form a fll measrement eqaton n Step 4. Bt at ths stage the am s to wrte down the calclaton that s performed at each step explctly. Qestons to ask yorself: What s the eqaton sed to calclate each box n my traceablty chan? Is t a sngle step or mltple step process? Do I rely on other nformaton (e.g. a dstance measrement) that s not nclded n my traceablty chan? At ths stage do I need to go back to step 1 and refne my traceablty chan? ~ 3 ~

41 4.4 Step 3: Consderng the sorces of ncertanty The am here s to consder, for each step of the traceablty chan, what the sorces of ncertantes are. In the fnal ncertanty bdget, these wll provde the rows of the ncertanty bdget. Often a good startng pont s to branstorm whether that s n a lst, or sng a more graphcal approach (e.g. Fgre 5). It s helpfl to se other people s examples as a check-lst. Fgre 5 Branstorm mndmap of ncertantes assocated wth a spectral rradance measrement of a test lamp n comparson wth a reference lamp Another method that can be sed s to consder the calclaton eqaton for each step of the traceablty chan, as determned n Step. Each term n that eqaton wll have assocated ncertantes and those shold be lsted. For example, for the calclaton eqaton (4.1), we mmedately determne the frst sorces of ncertanty: The rradance of the FEL lamp (as measred by, e.g. an NMI) The reflectance factor of the dffser (as measred, by e.g. an NMI) Fnally, t s mportant to nderstand the nderlyng (often nstated) assmptons behnd the calbraton process. Many calbraton processes nvolve a comparson, e.g. that the rradance of the test lamp s compared wth the rradance of the reference lamp sng a transfer spectrometer. The assmpton here s that the transfer spectrometer behaves n exactly the same way to the reference lamp and test lamp. However, that may not be the case. For example: ~ 33 ~

42 The spectrometer may be non-lnear and the test lamp mght be mch hgher or lower radance than the reference lamp. The spectrometer may be temperatre senstve and one lamp may heat the room p more than the other, or the second lamp measrement may take place a day after the frst lamp measrement and the room has changed temperatre. The spectrometer may have changed n beng moved from one sorce to another. The spectrometer may be senstve to stray lght from other wavelengths. The reference lamp (say an FEL) may emt more lght at those wavelengths than the test lamp (say an LED-based sorce). The spectrometer s npt optcs has a dffser on t. The dffser s not perfectly Lambertan and the test lamp may llmnate t at a greater range of angles than the reference lamp does. The two sorces may have dfferent ltravolet otpts and the dffser may floresce. All these examples are where the comparson makes nderlyng assmptons of eqvalence, whereas the actal measrement system may not be eqvalent for the two sorces. At ths stage, these effects shold be lsted. Qestons to ask yorself (for each step of my traceablty chan): How dd I get ths reslt? What prevos nformaton dd I se? What else mght have affected the reslt? What s the eqaton I m sng to calclate the answer? Where do the vales for each of the varables come from? What ncertanty s assocated wth each of those? What hdden assmptons are there? Am I dong a comparson? What am I assmng s the same for that comparson? Am I comparng apples and oranges? 4.5 Steps 1 to 3 for a space-borne sensor There s nothng fndamentally dfferent abot a sensor that s n orbt to an nstrment that s calbrated for measrements on an arcraft, n a feld or a laboratory. The steps descrbed here wll apply n all statons, bt may need some translaton and nterpretaton specfc to the applcaton. Probably the most sgnfcant barrer to applyng ths to satellte sensors s the complexty of the algorthms and the scale of the task. It s therefore mportant to be able to determne a smplfed verson of the ncertanty problem and then add n complexty n stages. The frst three steps are abot nderstandng the problem and wrtng down what s known abot the measrement system. There wll stll be a traceablty chan, albet perhaps one that s broken by the lanch process (see e.g. Secton 8.5.). It can help to separate the analyss nto two ndependent sectons the frst that descrbes the traceablty of the pre- ~ 34 ~

43 lanch calbraton and charactersaton of the nstrments and the second that descrbes the process of post-lanch cal/val sng on-board and vcaros calbraton processes. In any calbraton chan, even those of well-nderstood rotne calbratons n an NMI laboratory, there wll be aspects that are not flly nderstood. There are always ncertanty components that are estmated, or even gessed, based on experence, hstorcal records or models of worst case scenaros. In a space-borne nstrment, there wll sally be more sch gesses thngs that cannot be known (e.g. how has the spectral response fncton of the nstrment changed n orbt?). However, post-lanch cal/val processes can pt pper lmts on probable changes and assocate ncertantes wth them. The NMIs are well aware that there can always be systematc effects not flly nderstood the nknown nknowns n any ncertanty bdget. The NMI commnty attempts to determne whether sch factors are present throgh formal blnd nternatonal comparsons 0. Smlarly sensor-to-sensor and sensor-to-grond comparsons can gve an estmate of the post-lanch changes n the nstrments on satelltes as well as ndcatng any problems wth the calbraton (nknown nknowns). The QA4EO Gdelne 4 advses on how to rn smlar comparsons for EO measrements whether between sensors or between grond measrement technqes. The frst step n ncertanty analyss for a satellte sensor nvolves an explct descrpton of the n-orbt traceablty chan ncldng both the prelanch calbraton processes and the post-lanch cal/val. The second step s to wrte down the calclaton eqatons both for the calbraton processes and for the data processng n orbt, e.g. how the dgtal nmbers obtaned n orbt are trned nto the Level 1 prodct (e.g. top-of-atmosphere reflectance). Ths s sally done n the Algorthm Theoretcal Bass Docment and/or the Detaled Processng Model and shold nclde the correctons appled to the data, ncldng prelanch and post-lanch calbraton coeffcents. The thrd step s to wrte down the sorces of ncertanty. Ths wll nclde: Calbraton processes n the pre-flght calbraton ncldng ncertantes assocated wth references, wth the process of calbraton and the hdden assmptons n the calbraton (e.g. smlar to those descrbed n Secton 4.4). Calbraton processes n the post-lanch calbraton/valdaton ncldng ncertantes assocated wth references, the process of calbraton and the hdden assmptons n the calbraton In-orbt degradaton of all nstrments and artefacts, especally those that cannot be checked by the nflght calbraton/valdaton In-orbt degradaton of nflght references 0 As part of the Mtal Recognton Arrangement whch ensres that measrements made traceably to SI at one NMI can be consdered legally traceable to another NMI wthn a degree of eqvalence there are reglar comparsons between NMIs of all sgnfcant qanttes. The comparsons are organsed n a formal manner to ensre rgoros mpartalty and the reslts are pblshed on the Key Comparson Database ( ~ 35 ~

44 A really helpfl example s gven for the MERIS calbraton. The docment MERIS Instrment Calbraton s freely downloadable [4]. Ths report revews the MERIS calbraton process. It dstngshes pre-flght and on-orbt calbraton n two separate sectons. The frst two sectons of the MERIS calbraton report are a revew of the nstrment and the calbraton prncple. The report does not explctly provde a traceablty chan. However t s possble to work t ot from the provded nformaton. Fgre 1 provdes the calbraton processng chan and effectvely s dong Step of ths ncertanty process the calbraton processng chan s the calclaton eqaton. The branstorm of sorces of ncertanty of step 3 s also straghtforward from the ttles of the report. 4.6 Step 4: Creatng the measrement eqatons The measrement eqaton s an extended verson of the calclaton eqaton that also explctly descrbes the other sorces of ncertanty. It s sally sensble to create a measrement eqaton for the each sbsecton of the traceablty chan, althogh n some relatvely smple cases, a fll measrement eqaton for the whole process can be made. Ths wll bld on the hdden assmptons descrbed n Step 3. For example, for the calclaton eqaton (4.1) we have the followng hdden assmptons: The lamp-dffser dstance, d se s the same as the dstance at whch the lamp rradance was calbrated, d cal. Assmng the FEL lamp obeys the nverse sqare law 1 [5], then there needs to be an addtonal term dcal dse n Eqaton (3.5). The lamp has been stable snce calbraton and E FEL s an accrate representaton of the lamp rradance at the tme of the new measrements. There wll actally be some varatons de to: o Lamp stablty (short term).e. random flctatons o Lamp stablty (long term).e. changes snce calbraton, drft o Lamp algnment (rotaton, postonng) o Lamp crrent control: accracy and stablty The dffser has been stable snce calbraton, and hence β 0 :45 s an accrate representaton of the dffser radance factor at the tme of the measrements. There may be some varatons de to: o Dffser stablty (e.g. de to gettng drty or absorbng hydrocarbons) Accracy of angles set drng calbraton and se (s t really a 0 /45 geometry?) The rradance patch s nform over the feld-of-vew of the nstrment vewng the dffser panel. 1 Whch t doesn t! Actally FEL lamps, for dstances greater than 500 mm, obey the modfed nverse sqare law and the cal offset se offset term shold be ( d d ) ( d d ) + +, where the offset dstance s the dstance from the flament to the reference plane and s determned by testng the nverse sqare law behavor sng a detector wth a clearly defned reference plane. For shorter dstances a frther term s reqred to accont for the physcal dmenson of the flament. Note also that for a blk dffser, sch as Spectralon TM, there s an addtonal dffser offset dstance. ~ 36 ~

45 These concepts shold, somehow be nclded n the measrement eqaton (whch s an expanded verson of the calclaton eqaton). The dstance term can be drectly nclded nto the eqaton, expandng t to: L EFELβ0 :45 dcal s π dse =. (4.) Note that f the calbraton and se dstances are nomnally the same, the fnal term has an expected vale of nty (one), bt there wll nevertheless be an ncertanty assocated wth takng ths vale of nty that mst be consdered. All the terms assocated wth the assmptons made can be nclded n the eqaton. For example, we may wrte EFELβ0 :45 dcal s lampstab algn crrent dffstab nf π dse L = K K K K K. (4.3) Here the K terms relate to the dfferent effects. These terms all have an expected (nomnal) vale K = 1. In other words, we are assmng that the vale of each s nty 3. However, there s an ncertanty assocated wth that assmpton. Eqaton (4.3) forms the fll measrement eqaton. Note that there are stll ncertantes assocated wth the man terms EFEL, β 0 :45 and there are addtonal ncertantes assocated wth the assmptons n the K terms. If there s correlaton, then the measrement eqaton also provdes an opportnty to descrbe the correlaton, often n sch a way that the wrtes-ot the correlaton and means that the remanng terms have no assocated correlaton (they are ndependent of each other), as we saw n Secton 3.4. Qestons to ask yorself: Have I nclded each sorce of ncertanty n my measrement eqaton? Are these addtve or mltplcatve? Note that f the dffser s only 500 mm from the lamp, the corners of a 300 mm by 300 mm panel are 543 mm from the centre of the flament and from the nverse sqare law wold be expected to be 15 % lower radance than the centre of the panel. In practce the radance non-nformty can be better or worse than ths, dependng on the flament dmenson. 3 In practce, some of these may be correctons. We may make a correcton for the fact that we know the lamp crrent was set wrongly or that we know the nformty s not perfect. In ths case the terms have an assgned vale (not nty), and an ncertanty assocated wth the correcton. It can be helpfl to se a dfferent symbol, say C for correctons. nf ~ 37 ~

46 4.7 Step 5: Determnng the senstvty coeffcents The senstvty coeffcent s the senstvty of the calclated reslt to an error n each of the parameters of the measrement eqaton n trn. As descrbed n Secton 3.1, senstvty coeffcents can be determned: Mathematcally, by dfferentatng the measrement eqaton Nmercally, by modellng the effect of a change n that qantty sng a system model Expermentally, by varyng that parameter n the laboratory All three methods are typcally sed n the development of any ncertanty bdget. The method chosen depends on what nformaton s avalable: f a model exsts as software, t s relatvely straghtforward to modfy that to nclde a nmercal estmate of senstvty coeffcents. If the measrement eqaton has explct relatonshps, then often t s most straghtforward to dfferentate t drectly. If the measrement eqaton cannot be wrtten down explctly, then the senstvty coeffcents shold be determned expermentally. Qestons to ask yorself: Can I dfferentate the measrement eqaton? Can I determne the senstvty coeffcent nmercally / throgh modellng? What expermental tests can I do to determne the senstvty coeffcent? 4.8 Step 6: Assgnng ncertantes The fnal am of ths secton s to obtan an ncertanty bdget table that lsts the ncertanty components, ther assocated ncertantes and the ncertanty assocated wth the fnal measred vale de to each of these effects n trn. Separately t s mportant to state clearly the correlatons (or lack of correlaton). For the example of the lamp-dffser radance, an example table s gven as Table. Table Example ncertanty bdget for a lamp-dffser combnaton Uncertanty component Assocated ncertanty (relatve) Uncertanty absolte relatve Senstvty coeffcent assocated wth radance de to ths Lamp rradance (calbraton) 0.30% % Dffser radance factor (calbraton) 0.30% % Lamp-dffser dstance (same as calbraton dstance for lamp)? 1 mm n 500 mm 0.0% 0.40% Stablty of lamp (short term) 0.10% % Stablty of lamp (drft/ageng) 0.10% % Algnment of lamp 0.05% Crrent stablty of lamp (at 350 nm) 3 ma 0.9% Dffser stablty (ageng) 0.10% % Unformty of dffser 0.50% % ~ 38 ~

47 Note here: For some effects (rradance, reflectance factor, stablty, nformty), the estmated ncertanty assocated wth the parameter s a relatve ncertanty n %, the senstvty coeffcent s determned mathematcally (Eqaton (3.7) for example) and s the relatve senstvty coeffcent. The fnal colmn s determned by mltplyng the ncertanty assocated wth the component wth the senstvty coeffcent. For the dstance the same apples, here the relatve senstvty coeffcent s determned mathematcally (Eqaton (3.11), note the negatve sgn s gnored as ths wll be sqared 4 ). Ths reqres the relatve ncertanty assocated wth dstance, whch s calclated from the absolte ncertanty assocated wth dstance. The algnment senstvty for the lamp was determned expermentally. Therefore there s nothng n the frst three colmns as t was calclated drectly from the standard devaton of mltple measrements wth the lamp realgned between measrements. Wth expermental determnatons, the senstvty coeffcent and the assgned ncertanty are often calbrated smltaneosly. The crrent senstvty was estmated to be 3 ma. Ths was determned to affect the lamp rradance at 350 nm by 0.9 %. In ths case ths was evalated throgh modellng and the reslt provded straght nto the fnal colmn. Alternatvely, t cold have been done expermentally by changng the crrent by ten tmes more (see page 14). In that case a senstvty coeffcent wold have been calclated from the measred vales for ten tmes the change and then dvded by ten. It s not therefore necessary to fll n every sqare n ths table. What matters s the fnal colmn. Ths secton has descrbed dfferent methods for estmatng that fnal colmn. Qestons to ask yorself (for each sorce of ncertanty): How can I assess how large the ncertanty s? Is t small (< 1/5 th ) compared wth the largest ncertanty? What s the probablty dstrbton fncton? Is t correlated wth other ncertanty contrbtons? 4 Keep the mns sgns n f there s any correlaton between ths and another parameter, as the second half of the law of propagaton of ncertantes does not sqare the senstvty coeffcents. ~ 39 ~

48 Gassan and other probablty dstrbton fnctons Note that here we have assmed that the ncertantes assocated wth each of the parameters are consdered to descrbe Gassan (normal) probablty dstrbton fnctons. Ths the parameter s more lkely to be close to the expected vale than frther from t. There are tmes when a rectanglar probablty dstrbton fncton s more approprate. Ths means that we know that the vale wll defntely le between two lmts, wth an eqal probablty of beng anywhere between those lmts, and no probablty of beng otsde ths. Ths cold be the case, for example, where the room temperatre s controlled between 19.5 C and 0.5 C by the ar-condtonng system and wll not be otsde ths. It s also the case for rondng errors on dgtal dsplays. If a dsplay provdes the vale 3.84 t s eqally lkely that the vale falls anywhere n the range between and Rectanglar probablty dstrbton fnctons can be represented n ncertanty tables as standard (Gassan) ncertantes, by dvdng the half range by 3. Sometmes ncertanty bdgets wll have an addtonal colmn labelled dvsor that wll nclde these for rectanglar dstrbtons. 3 vales Smlarly, f a vale s read off a certfcate, then the certfcate wll probably provde expanded ncertantes (say at the 95 % confdence level) and may nclde a statement sayng, for k =, for example. A standard ncertanty (as needed for the ncertanty bdget) wll be obtaned by dvdng the expanded ncertanty by the vale for k. 4.9 Step 7: Combnng and propagatng ncertantes Havng obtaned the ncertanty assocated wth the measred vale de to each ncertanty parameter n trn t s necessary frst to combne the ncertantes to obtan the ncertanty assocated wth the measred vale de to all these ncertanty components, and then to expand that calclated standard ncertanty to an approprate confdence level. These steps are dscssed here. Uncertantes are combned sng the Law of Propagaton of Uncertantes gven n the GUM, and gven above as Eqaton (.1). If the dfferent parameters gven n the ncertanty bdget table are ncorrelated, then the frst half of the eqaton apples and f the fnal colmn n the ncertanty bdget provdes c, the combned standard ncertanty s obtaned by addng the colmn n qadratre,.e. takng the sqare root of the sm of the sqares 5, whch for the table above s 0.84 %. If there are correlatons between the npt parameters, then the second half of the Law of Propagaton of Uncertantes s reqred. Qestons to ask yorself (for each sorce of ncertanty): 5 In Excel =sqrt(smsq(f10:f18)) for example ~ 40 ~

49 Wll the smallest ncertantes have neglgble mpact? 4.10 Step 8: Expanded ncertantes The central lmt theorem states that the arthmetc mean of a sffcently large nmber of ndependent random varables, each wth a well-defned expected vale and well-defned varance, wll be approxmately normally (Gassan) dstrbted 6. What ths means for ncertanty analyss s that f there s a reasonably large nmber of npt parameters, wth smlar enogh (well-defned) ncertantes, then the probablty dstrbton of the otpt parameter wll be approxmately Gassan, no matter what are the probablty dstrbtons of the npt parameters. Ths means that t s straghtforward to expand ncertantes assmng that the standard ncertanty obtaned by combnaton relates to a Gassan dstrbton. For a Gassan dstrbton, the standard ncertanty represents a coverage probablty of approxmately 66 %, therefore the tre vale wll be wthn the standard ncertanty of the measred vale approxmately 66 % of the tme. It s more common to provde (approxmately) 95 % confdence ntervals. For a Gassan dstrbton, these are obtaned by mltplyng the standard ncertanty by the coverage factor k =. If the dstrbton s not Gassan, then a dfferent coverage factor s needed. A dfferent coverage factor s also needed f the standard ncertantes n the ncertanty bdget table are not sffcently well known. Secton 3.3. descrbed ncreasng the estmate of ncertanty obtaned from the standard devaton of a nmber of repeat readngs accordng to Eqaton (3.6) to accont for the fact that the standard devaton of a small nmber of readngs s nrelable. If ths s done, then t s reasonable to assme that k =. Alternatvely, the GUM provdes the Welch-Satterthwate Eqaton, whch calclates the effectve degrees of freedom of the combned standard ncertanty (.e. a weghted answer to the qeston: how many ndependent measrements are nvolved n estmatng the ncertanty? ). The Welch-Satterthwate eqaton calclates the effectve degrees of freedom sng the followng expresson Here, c ( ) ( y) c ( x ) ν 4 c eff = N 4 = 1 ( y) ( y) ν. (4.4) y s the combned standard ncertanty assocated wth the vale y and = s the ncertanty assocated wth y de to one of ts contrbtng effects, 6 Provded the dstrbton of no one random varable domnates ~ 41 ~

50 x,.e. the standard ncertanty assocated wth x mltpled by the magntde of the senstvty coeffcent. ν s the nmber of degrees of freedom assocated wth the estmate of ( ) y. For Type A determnatons of ncertanty,.e. where the ncertanty s determned by takng the standard devaton of mltple measrements, the degrees of freedom s the nmber of measrements mns one 7 : ν = N 1 (4.5) Note that f yo had a commssonng phase (see Secton 3.3.) where yo made large nmbers of measrements to estmate the ncertanty, then the N to se here s the nmber of measrements combned n that commssonng phase to estmate the standard devaton and hence ncertanty. If for the rotne measrements only a small nmber of measrements are averaged, then the degrees of freedom are stll calclated from the orgnal nmber of commssonng measrements, as ths s abot how good the ncertanty estmate s. Note however, that ths ncertanty assocated wth a sngle readng wll only be redced n averagng by the sqare root of the nmber of readngs actally averaged today. For Type B determnatons of ncertanty,.e. where the ncertanty s determned by other means, for example pror knowledge, certfcates etc, the nmber of degrees of freedom s harder to defne. There are two ways of dong so: 1. By assmng that the Type B ncertanty s known perfectly and the degree of freedom s nfnte. Ths comes from the atttde that yo re gven a Type B ncertanty. In ths example the bottom lne of the Welch-Satterthwate eqaton 4 y = 0. ends p wth a term ( ). We estmate the ncertanty n the ncertanty. Ths s, of corse, more of an art than a scence, bt f we have a feel for an ncertanty range, then the GUM provdes an eqaton ( x) ( ( )) 1 ν = x (4.6) So, for example, f we have a Type B ncertanty that s 3 %, bt or experence and confdence consders that what we mean by ths s that the ncertanty les n the range from % to 4 %, we may say the ncertanty n the ncertanty s 1 % absolte (or the ncertanty s relable to abot 0.33). And 7 The mns one comes from the fact that f a mean s calclated from N measrements, then the mean and 1 N measrements flly descrbe the problem: the fnal measrement can be calclated from the mean and the others..e. the mean ses p one of the degrees of freedom. ~ 4 ~

51 1 3% ν = 4.5 = 1% (4.7) The fnal step nvolved n evalatng an expanded ncertanty s to determne the vale of k,.e. the vale to mltply a standard ncertanty by n order to trn t nto an expanded ncertanty wth a 95 % confdence nterval. Ths s obtaned from the t-dstrbton, and for a 95 % confdence nterval the vales are gven n Table 3. More vales are gven n the GUM. Table 3 Degrees of freedom and vale of the coverage factor for a 95 % confdence nterval Degrees of freedom Vale of k reqred for from the Welch- a 95 % confdence Sattherwate formla nterval (For nfnte degrees of freedom, the coverage nterval for k = s %). ~ 43 ~

52 5 Case stdy: APEX mager calbraton (smplfed) 5.1 The APEX mager calbraton Ths case stdy descrbes a somewhat smplfed verson of the APEX mager calbraton. Later sectons consder some of the more nvolved aspects. It s strongly recommended that n prodcng an ncertanty bdget, partcpants start wth a smplfed verson of ther own traceablty and then add complexty as they become more famlar wth the concepts. Althogh the detals n ths secton apply to the arborne APEX nstrment specfcally, the concepts they ntrodce are more general than ths and apply to all EO magers. ESA s Arborne Imagng Spectrometer APEX (Arborne Prsm Experment) was developed nder the PRODEX (PROgramme de Développement d'expérences scentfqes) program by a Swss-Belgan consortm and entered ts operatonal phase at the end of 010. It s an magng spectrometer that s sally flown on a DLR arcraft to obtan hgh resolton mages of the grond wth typcal pxel szes rangng between 1.5 m to.5 m. APEX featres p to 53 spectral bands n fll spectral mode (provdng spectral nformaton from 387 nm to 500 nm). It also has spectral programmablty to acheve hgher Sgnal-to-Nose-Ratos (SNR) by redcng the nmber of bands n a bnned confgraton. Data [6, 7] are acqred n 1000 pxels across track wth a FOV of 8. Fgre 6 APEX mager on board arcraft and captred hyperspectral mage 8 8 The data cbe shown here s avalable for free download from the APEX web ste ( as APEX Open Scence Dataset. ~ 44 ~

53 The APEX calbraton s carred ot at the CHB (Calbraton Home Base) stated at DLR Oberpfaffenhofen [8]. The standard APEX calbraton s carred ot n an operatonal manner at least once a year, sally at the begnnng of the flght season. The calbraton comprses measrements on an optcal bench for the geometrc and spectral calbraton, sng a collmator-slt setp and a monochromator respectvely, and measrements sng a small and bg ntegratng sphere for absolte radometrc calbraton and flat-feldng respectvely. Fgre 7 Calbraton Home Base at DLR A fll sensor calbraton at the CHB can be acheved wthn 3-4 days, ncldng sensor nstallaton and algnment on the calbraton bench. Raw calbraton data are stored n the APEX Calbraton Informaton System (CAL IS) [9] and processed to generate calbraton cbes, holdng calbraton coeffcent on a pxel per pxel bass (Fgre 8). APEX magery s calbrated to radance n the APEX Processng and Archvng Faclty (PAF)[10], applyng the calbraton coeffcents prodced by the CAL IS to the mage cbes. ~ 45 ~

54 Fgre 8: APEX Data Acqston to Prodct Chan 5. Step 1: Descrbng the traceablty chan The APEX nstrment s calbrated, at a few radance levels, sng a small ntegratng sphere sorce wth netral densty flters placed n front of the sorce. By makng the calbraton for dfferent radance levels, the lnearty and any bas offset for the nstrment can also be determned. The ntegratng sphere sorce s too large to be easly calbrated at an NMI. So nstead, a hand-held spectrometer s sed to transfer a calbraton from a sorce engneered by DLR and calbrated by PTB (the RASTA sorce) to the ntegratng sphere sorce. The netral densty flters were calbrated at NPL drectly. Therefore the traceablty chan, smplfed, s shown n Fgre 9. ~ 46 ~

55 Fgre 9 Traceablty chan (smplfed) for the APEX calbraton In the ble secton on the left of Fgre 9, the calbraton s transferred from the RASTA sorce, to a portable spectrometer and then to the ntegratng sphere sed for the APEX calbraton. The RASTA sorce [11] conssts of a whte dffser tle llmnated by an FEL lamp at normal ncdence and vewed at 45. The sorce also ncldes approprate bafflng to redce and control stray lght and several flter radometers that montor the short and longterm stablty of the RASTA sorce. The sorce was calbrated at an NMI, n ths case PTB, and that calbraton provdes the traceablty to SI. Fgre 10 From [11]: Mechancal set-p of RASTA (left). RASTA after ts algnment for calbraton at PTB (rght). The flter radometers are shown n red n the mechancal set-p dagram. The lamp s n the octagonal hosng and the dffser panel s monted on the plate on the left of the dagram and pctre. ~ 47 ~

56 The spectrometer sed s the SVC 104 spectrometer 9. Ths s frst calbrated radometrcally aganst the RASTA and then sed to calbrate the ntegratng sphere. The ntegratng sphere [8] s 500 mm n dameter, wth a port of 40 mm 00 mm. The sphere s operated so that the ext port s at the top of the sphere whch allows APEX to be calbrated n ts flght orentaton (Fgre 11). Fgre 11 APEX monted above the ntegratng sphere sed for ts calbraton In order to perform the APEX calbraton at several radance levels, netral densty flters are sed to redce the otpt of the sphere. These netral densty flters were drectly calbrated for transmttance at an NMI, n ths case, NPL, and therefore have straghtforward traceablty to SI. The prple rectangle n Fgre 9 s where the compted radance for the sphere-flter combnaton s calclated. Ths s a straghtforward mltplcaton of the sphere radance and the flter transmttance. The green secton transfers the calbraton to the APEX nstrment. Frst the radometrc gan of the APEX nstrment (.e. the converson factor from ts otpt dgtal nmbers to radance) s calclated from the APEX measrement of the sphere-flter sorce, and then the measred dgtal nmbers of a partclar scene are converted to radance sng the nstrment gan. 9 ~ 48 ~

57 5.3 Step : Wrtng down the calclaton eqatons For the calbraton of the sphere The traceablty chan for ths secton s shown to the left. In the frst step PTB provdes a calbraton of the RASTA sorce (comprsng a lamp and dffser and approprate bafflng). Ths creates an SI-traceable calbraton of the radance of the RASTA sorce,. PTB provdes a calbraton certfcate for the radance, wth an assocated relatve ncertanty, expressed as an expanded (95 % confdence, k = ) ncertanty n per cent. In the second step the SVC transfer spectrometer vews the RASTA sorce and obtans a spectrm (n dgtal nmbers [DN] as a fncton of wavelength). Ths measrement s performed as the average of several lght readngs mns the average of several dark readngs (wth the entrance port of the spectrometer closed). From ths, the gan of the SVC (nts: [W m - sr -1 nm -1 DN -1 ]) s determned. Fgre 1 In the thrd step, the SVC transfer spectrometer vews the APEX calbraton sphere sorce. The radance of the APEX sphere sorce s calclated from the SVC gan and the measred DN for the sphere sorce, agan an average of several lght readngs mns an average of several dark readngs. From ths the spectral radance of the APEX calbraton sphere sorce s determned. Both parts of ths process can be consdered comparson calbratons. In the second step of Fgre 1, the SVC gan 30 s calclated from the calclaton eqaton, (5.1) where, s the PTB-calbrated radance of the RASTA sorce and s the measred dgtal nmbers on the SVC spectrometer when vewng the RASTA sorce. Note that all three qanttes n Eqaton (5.1) are spectral qanttes and are fnctons of wavelength. The eqaton s vald for a specfed wavelength, bt the wavelength dependence s not specfcally wrtten here to smplfy the presentaton. 30 An alternatve wold be to calclate the spectral radance responsvty of the SVC nstrment. The spectral radance responsvty, wth nts [DN / (W m - sr -1 nm -1 )], s the nverse of the gan, nstrment per nt radance. and s the response of the ~ 49 ~

58 The thrd step s smlar. The radance of the sphere s calbrated sng the SVC spectrometer and s calclated sng the calclaton eqaton, (5.) where, s the sphere radance, s the measred dgtal nmbers when the SVC vews the sphere and s the SVC gan, calclated from (5.1). Agan, all three qanttes are spectral qanttes, and ths eqaton s calclated at each wavelength n trn For the calbraton of the APEX mager The radance of the sphere-flter combnaton s calclated as (5.3) where s the transmttance of the flter, and s calclated n Eqaton (5.). Agan, these are spectral qanttes and ths s calclated for each wavelength n trn. The sgnal on the APEX mager for a partclar wavelength,, n dgtal nmbers, when vewng the sphere-flter combnaton s then sed to calclate the APEX gan as, (5.4) where. (5.5) For the ser of the APEX mager to measre scene radance The radance of an observed scene s then, (5.6) where 5.4 Step 3: Consderng the sorces of ncertanty. (5.7) Each stage of the transfer from the PTB radance vales to the observed scene radance s some form of comparson calbraton, and therefore there are ncertantes assocated wth both the explct terms n the calclaton eqaton, and the hdden assmptons that ~ 50 ~

59 everythng s eqvalent on both halves of the comparson. For each calclaton eqaton, we can prepare an ncertanty table. These are gven below, For the calbraton of the SVC spectrometer aganst the RASTA Calclaton Eqaton (5.1) s a comparson between the vales assgned to RASTA by PTB sng ther measrement nstrment RASTA and the SVC s measred vales for the RASTA. It s mportant to revew three sorces of ncertanty: those de to L RASTA, those de to DN RASTA and those de to mpled assmptons hdden wthn ths comparson. Table 4 Uncertantes assocated wth Calclaton Eqaton (5.1) and ts hdden assmptons G SVC Uncertanty component Comments Uncertantes assocated wth L RASTA LRASTA PTB calbraton of L RASTA L RASTA Uncertantes assocated wth DN DN RASTA_ Nose n lght readng DN RASTA_ DN DN RASTA_d RASTA_d Nose n dark readng RASTA Ths wll be read off the certfcate. See the addtonal notes box on the next page. Ths wll be obtaned throgh Type A methods (.e. from a standard devaton see Secton.6..) Ths wll be obtaned throgh Type A methods Uncertantes assocated wth the comparson assmptons K RASTA_age Ageng of RASTA snce PTB calbraton K RASTA_age stray stray Stablty of RASTA (short term) External stray lght nflencng the SVC calbraton The RASTA sorce has nblt flter radometers that montor the sorce stablty. The varaton of these sgnals can be sed to estmate ths ncertanty component, and, f approprate, apply a correcton. Ths wll get nclded n the standard devaton of the lght readngs, and does not need doble contng here Ths s lght that contrbtes to the SVC sgnal and comes from otsde ts feld-of-vew. K nf nf Any envronmental senstvtes of RASTA (temperatre, pressre, hmdty) Unformty of RASTA and any dfferences n the feld-of-vew for PTB s calbraton and the SVC vew These are assmed to be neglgble (and therefore gven an ncertanty of 0 %). RASTA conssts of a tngsten lamp, whch s nsenstve to temperatre, and a dffser, whch s also nsenstve 31 to mnor temperatre changes. Understandng ths reqres a nformty scan of the RASTA sorce, along wth knowledge of the felds-of-vew of the PTB calbraton (from the measrement certfcate) and the SVC sorce. 31 Spectralon does show a phase transton at ~19 C, whch affects ts reflectance by approxmately 0.1 %. However, the lamp s lkely to heat the dffser above ths temperatre, and ths s mnor compared to other ncertanty components. ~ 51 ~

60 5.4. For the calbraton of the sphere wth the SVC spectrometer Calclaton Eqaton (5.) s a comparson between the SVC s measrement of RASTA and the SVC s measrement of the sphere. It s mportant to revew three sorces of ncertanty: those de to DN, those de to G SVC and those de to the mpled assmptons hdden sphere wthn ths comparson. Table 5 Uncertantes assocated wth Calclaton Eqaton (5.) and ts hdden assmptons L sphere Uncertanty component Uncertantes assocated wth G SVC GSVC As above Uncertantes assocated wth DN sphere_ DNsphere_ DN sphere_d DNsphere_d Nose n lght readng Nose n dark readng G SVC DNsphere Comments Ths wll come from the prevos step. Uncertantes assocated wth the comparson assmptons K K K K K SVC_dft SVC_dft stray stray stray_n stray_n temp temp ln ln Change of SVC between measrements Stablty of SVC (short term) External stray lght nflencng the SVC drng calbraton aganst RASTA and se wth the sphere Internal stray lght nflencng the SVC drng calbraton aganst RASTA and se wth the sphere Any envronmental senstvtes of the SVC (temperatre, pressre, hmdty) Lnearty of SVC Ths wll be obtaned throgh Type A methods (.e. from a standard devaton). Ths wll be obtaned throgh Type A methods (.e. from a standard devaton). We assme that the two measrements are reasonably close together n tme. Ths means that there s no ageng between one set of measrements and the next. There may be some senstvty, however, to physcally movng the SVC between the devces, for example. Ths drft can be estmated by smlatng the movement and remeasrng the same sorce wth the SVC. Ths term shold only be nclded f t s larger than the nose. Ths wll get nclded n the standard devaton of the lght readngs, and does not need doble contng here Ths s lght that contrbtes to the SVC sgnal and comes from otsde ts feld-of-vew. What matters s the change n ths between the calbraton and se. Ths s lght that s scattered wthn the SVC onto the pxel for the wrong wavelength. Ths shold be charactersed for the SVC spectrometer and ts effect wll depend on the dfferent spectral shape of the two sorce radances what matters s the dfference between the RASTA and sphere sorce spectral radances If the external condtons vary from the calbraton of the SVC vs the RASTA to ts se to calbrate the sphere, then t s necessary to accont for the senstvty of the SVC to those changes. Spectrometers can be very temperatre senstve and local room temperatres can be hgher near hgh power tngsten sorces. Spectrometers can be non-lnear, ether n terms of ther ntegraton tme (does doblng the ntegraton tme doble the sgnal), or n terms of ther response to doble the radance. Ths wll be a problem f the SVC s sed on a dfferent ntegraton tme for each sorce and/or f the sorces are dfferent radance levels. ~ 5 ~

61 5.4.3 For the combned sorce radance from the sphere and flters Calclaton eqaton (5.3) assmes that the radance of the sorce s a prodct of the radance of the sphere and the transmttance of the flter. Ths also has some hdden assmptons. Table 6 Uncertantes assocated wth Calclaton Eqaton (5.3) and ts hdden assmptons L sph-flt Uncertanty component Uncertantes assocated wth L sphere Lsphere As above Lsphere Uncertantes assocated wth τ flter τ flter Transmttance of the flter τ flter Comments Ths wll come from the prevos step. Uncertantes assocated wth the hdden assmptons K K K reflect reflect fl_temp fl_temp sph-stab sph-stab Optcal nterreflectons between the sphere and the flters Temperatre senstvty of the flter and changes when n front of the sphere The stablty of the sphere between ts calbraton wth the SVC and ts se wth the flter Ths wll be obtaned from NPL s calbraton certfcate. When the flter s ntrodced n front of the sphere, t may be that some lght from the sphere s reflected back nto the sphere and ths alters the radance of the sphere. Ths coeffcent s to accont for any sch nterreflecton effects. To estmate sch effects, the dstance between the sphere and flter can be vared, or the flter angled slghtly so that reflectons change drecton (beng aware that for many flters changng the angle wll tself change the transmttance). These are expermental methods for estmatng the senstvty coeffcent. Flters are sally temperatre senstve. The flter may be at a dfferent temperatre n front of the sphere than t was drng ts calbraton at NPL. The temperatre of the flter shold be measred n front of the sphere, e.g. wth a thermocople. The flter cold be recalbrated at dfferent temperatres, or nformaton obtaned from the manfactrer on the temperatre stablty of the flter. Ths can be estmated by the repeatablty of mltple readngs of the sphere alone over ths tme perod K flt_age flt_age Ageng of the flters snce calbraton There s a tme delay between the flter calbraton and ther se. In ths tme they are rradated wth UV radaton, bt even storage can alter flters. The flters shold be recalbrated at reglar ntervals to estmate the lkely ageng snce calbraton For the calbraton of the APEX gan from the measrement of the sphere-flter sorce Calclaton Eqaton (5.4) s a comparson between the SVC measrement of the sphereflter combnaton and the APEX nstrment measrng the same sorce. There are, agan, mpled assmptons hdden wthn ths comparson. ~ 53 ~

62 Table 7 Uncertantes assocated wth Calclaton Eqaton (5.4) and ts hdden assmptons G APEX Uncertanty component Comments Uncertantes assocated wth L sph-flt Lsph-flt As above Uncertantes assocated wth DN APEX,cal,lght DNAPEX,cal,lght DN APEX,cal,dark DNAPEX,cal,dark Nose n lght readng Nose n dark readng L sph-flt DNAPEX,cal Calclated above. Uncertantes assocated wth the comparson assmptons K K K stray stray hmd hmd nf nf Stablty of sphere-flter (short term) External stray lght nflencng the APEX calbraton Any envronmental senstvtes of the sphere-flter (temperatre, pressre, hmdty) Unformty of the sphere-flter and any dfferences n the feld-of-vew for the SVC calbraton and the APEX vew For the observed scene radance Ths wll be obtaned throgh Type A methods (.e. from a standard devaton). Ths wll be obtaned throgh Type A methods (.e. from a standard devaton). Ths wll get nclded n the standard devaton of the lght readngs, and does not need doble contng here Ths s from lght that contrbtes to the APEX sgnal and comes from otsde ts feld-of-vew. Temperatre and pressre effects are assmed to be neglgble (and therefore gven an ncertanty of 0 %). The sphere has hgh power lamps n t and the temperatre of the sphere wll be set by the sphere lamps, not the room condtons. There may be some senstvty at certan wavelengths to hmdty. A sphere has a very long effectve path length, and therefore shows absorpton at the water absorpton lnes. Ths wll change wth room hmdty. Understandng ths reqres a nformty scan of the sorce, along wth knowledge of the felds-of-vew of the SVC spectrometer and the APEX spectrometer. The calclaton eqaton (5.6) s a comparson between the calbraton of APEX sng the sphere and APEX s measrement of the scene. There are several assmptons. ~ 54 ~

63 Table 8 Uncertantes assocated wth Calclaton Eqaton (5.6) and ts hdden assmptons L scene Uncertanty component Comments Uncertantes assocated wth GAPEX As above GAPEX Uncertantes assocated wth DNAPEX,scene,lght DNAPEX,scene,lght DN APEX,scene,dark DNAPEX,scene,dark Nose n lght readng Nose n dark readng G APEX DNsphere Ths wll come from the prevos step. Uncertantes assocated wth the comparson assmptons K K K K K APEX_dft APEX_dft stray stray stray_n stray_n temp temp ln ln Change of APEX between measrements Stablty of APEX (short term) External stray lght nflencng APEX drng ts se wth the sphere Internal stray lght nflencng APEX drng calbraton and se. Also known as cross-talk Any envronmental senstvtes of the APEX (temperatre, pressre, hmdty) Lnearty of APEX Ths wll be obtaned throgh Type A methods (.e. from a standard devaton ). Ths wll be obtaned throgh Type A methods (.e. from a standard devaton). We assme that the two measrements are reasonably close together n tme. Ths means that there s no ageng between one set of measrements and the next. APEX may be senstve, however, to beng transported to the arcraft, monted n the arcraft, etc. Ths change can be estmated by remeasrng the same sorce for a second calbraton after the measrement campagn (bt ensre that only effects bgger than nose are nclded to prevent doble contng). Ths wll get nclded n the standard devaton of the lght readngs, and does not need doble contng here Ths s lght that contrbtes to the APEX sgnal and comes from otsde ts feld-of-vew. Ths s specfc to the staton n the arcraft, whch s lkely to be very dfferent from n the laboratory calbraton and t s the dfference n stray lght that s nclded here. Ths s lght that that s scattered wthn the APEX onto the pxel for the wrong wavelength. Ths shold be charactersed for a spectrometer and wll depend on the dfferent spectral shape of the two sorce radances. In ths case there are both spatal and spectral dmensons to consder. Agan, t s the change n nternal stray lght between the calbraton sorce and the scene measrements that matters. The external condtons of the APEX are very dfferent on the arcraft to n the laboratory. These changes shold be smlated and the changes nderstood. Even f certan correctons are made, there wll be a resdal ncertanty. Spectrometers can be non-lnear, ether n terms of ther ntegraton tme (does doblng the ntegraton tme doble the sgnal), or n terms of ther response to doble the radance. Ths s tested by makng the calbraton wth dfferent flters n front of the sphere ~ 55 ~

64 5.5 Step 4: Creatng the measrement eqatons The calclaton eqatons need to be expanded nto fll measrement eqatons by ncldng all the hdden assmpton terms. Here we are treatng all these terms as a mltplcatve factors K. All the K have an expected vale of nty (one), wth an ncertanty assocated wth that, descrbed by. Ths, calclaton eqaton (5.1) becomes measrement eqaton G SVC = L K K K RASTA RASTA_age stray nf DN RASTA_ DN RASTA_d. (5.8) Calclaton eqaton (5.) becomes measrement eqaton ( ) L = DN DN G K K K K K sphere sphere_ sphere_d SVC SVC_dft stray stray_n temp ln Calclaton eqaton (5.3) becomes measrement eqaton ( λ ) ( λ ) τ ( λ ). (5.9) L L K K K K K =. (5.10) cal 0 sphere 0 flter 0 sph_stab sph_temp reflect flt_age flt_temp Calclaton eqaton (5.4) becomes measrement eqaton G APEX = DN L K K K sph-flt stray hmd nf APEX,cal,lght DN APEX,cal,dark. (5.11) Calclaton eqaton (5.6) becomes measrement eqaton Lscene = GAPEX DNAPEX,sceneKAPEX_dft KstrayKstray_n KtempKln. (5.1) In all these measrement eqatons the terms are ndependent of each other, and are ndependent from lne to lne. Ths means that correlaton does not need to be taken nto accont. 5.6 Step 5: Determnng the senstvty coeffcents For these straghtforward measrement eqatons, the smplest way to determne the senstvty coeffcents s throgh dfferentatng. Consder frst eqaton (5.8) for L RASTA Therefore, the ncertanty assocated wth G K K K G (5.13) L DN DN L SVC RASTA_age stray nf SVC = = RASTA RASTA_ RASTA_d RASTA G SVC de to L RASTA s G = (5.14) SVC GSVC: LRASTA LRASTA LRASTA ~ 56 ~

65 And therefore, GSVC: LRASTA LRASTA G SVC = ; (5.15) L RASTA.e. the relatve ncertanty assocated wth the SVC gan de to the RASTA radance s eqal to the relatve ncertanty assocated wth the RASTA radance. Therefore the relatve senstvty coeffcent s 1. The same relatonshp holds for all the parameters n the nmerator of Eqaton (5.8). The denomnator of Eqaton (5.8) s DNRASTA = DNRASTA_ DNRASTA_d ;.e. a lght sgnal mns a dark sgnal. Ths was dscssed n detal n Secton For now, we consder ths as a sngle qantty, DN. Therefore RASTA G DN SVC RASTA L K K K = = RASTA RASTA_age stray nf ( DN ) RASTA G DN SVC RASTA (5.16) and the relatve ncertanty assocated wth the gan, de to the relatve ncertanty assocated wth the dgtal nmber sgnal s gven by GSVC:DNRASTA DNRASTA G SVC DN = (5.17) RASTA and the relatve senstvty coeffcent s Step 6: Assgnng ncertantes The fnal am of ths secton s to obtan an ncertanty bdget table that lsts the ncertanty components, ther assocated ncertantes and the ncertanty assocated wth the fnal measred vale de to each of these effects n trn. Some methods for determnng the ncertanty assocated wth the ndvdal parameters were descrbed n the tables above. The se of NMI calbraton certfcates A calbraton certfcate from an NMI generally provdes ncertantes wth a statement smlar to: The reported expanded ncertanty s based on a standard ncertanty mltpled by a coverage factor k =, provdng a coverage probablty of approxmately 95 %. The concept of expanded ncertantes s descrbed n more detal n Secton When sng the vales, the ncertantes shold be converted back to standard ncertantes. Ths means dvdng by the provded vale for k, whch s generally, bt not always. Each table n Secton 5.4 can be expanded wth extra colmns for sze of effect, (relatve) senstvty coeffcent and ncertanty assocated wth Y de to ths effect (here Y represents the qantty calclated by each calclaton eqaton n trn). For example, for the ~ 57 ~

66 second step n the chan, the determnaton of the gan of the SVC, ths wold be as n Table 9. Table 9 Uncertanty bdget table for the gan of the SVC G SVC Uncertanty component Sze of effect (relatve) senstvty coeffcent Uncertantes assocated wth L RASTA PTB calbraton of L RASTA L RASTA 1 Uncertantes assocated wth DN DN RASTA_ Nose n lght readng RASTA Combned, relatve: Uncertanty assocated wth SVC gan de to ths effect DN RASTA_d Nose n dark readng Uncertantes assocated wth the comparson (apples and oranges) -1 RASTA_age Ageng of RASTA snce PTB calbraton 1 Stablty of RASTA (short term) 0 1 stray External stray lght nflencng the SVC calbraton Any envronmental senstvtes of RASTA (temperatre, pressre, hmdty) nf Unformty of RASTA and any dfferences n the feld-of-vew for PTB s calbraton and the SVC vew Smlar tables can be prepared for the other parameters; however n all cases becase of the smple measrement eqatons, the relatve senstvty coeffcent s 1 (or occasonally -1). The parameters themselves are spectral qanttes they take a dfferent vale n each of the spectral bands. And therefore a more helpfl way to present them s graphcally, as n Fgre 13. ~ 58 ~

67 Fgre 13 Uncertanty components for the APEX radometrc calbraton presented graphcally on a logarthmc scale. 5.8 Step 7: Combnng and propagatng ncertantes Here the qanttes are mtally ndependent (no correlaton) and the ncertanty s propagated from one level to the next by beng nclded as an npt ncertanty at the next eqaton. The combned relatve standard ncertanty assocated wth the scene radance s therefore obtaned by addng all the relatve standard ncertantes n qadratre. Ths provdes the green lne n Fgre Step 8: Expandng ncertantes Almost always, to obtan the 95 % confdence nterval, the standard ncertanty s mltpled by k = and ths has been sed here Ths s jst the start of the process The ncertanty analyss descrbed here s a sgnfcant smplfcaton of the actal calbraton process for the APEX mager. It s, however, mportant to start wth the smplfcaton and then bld n the complexty. Later sectons n ths report descrbe, for example, how the lnearty measrements can be sed to obtan a straght lne calbraton eqaton and how the spectral calbraton of APEX s done and how t relates to the ncertanty analyss of ths chapter. As complexty s ntrodced t can be consdered a new modle that mproves or measrement model and nderstandng of the ncertanty bdget. Often ntally sch analyss s done to nderstand the ncertanty to assgn to a component, later, as the analyss becomes more sophstcated, t can be sed to calclate a correcton and assocated ncertanty. For example, the lnearty measrements may ntally be performed to estmate ~ 59 ~

68 the ncertanty assocated wth the assmpton that lnearty s nsgnfcant (.e. that K ln s Eqaton (5.1) takes the vale nty). It may be that from those tests a more sophstcated lnearty model s developed and that latter a lnearty correcton s appled, wth an assocated ncertanty. It s worth notng that becase ncertantes are added n qadratre, an ncertanty that s 1/5 the sze of the largest ncertanty wll have a relatve weghtng on the ncertanty bdget of 1/5 th. Generally there s a pont where an ncertanty becomes nsgnfcant. Usally at the end of Step 3 of the steps to an ncertanty bdget t s possble to elmnate several terms as nsgnfcant. At Step 6 others may be classfed n ths way. ~ 60 ~

69 6 Straght lne calbraton eqatons Ths chapter focses mostly on a straght lne calbraton eqaton where the response of the nstrment to dfferent levels of npt s ftted wth a straght lne. The fnal secton, Secton 6.6, consders nterpolatng expermental data by straght lne nterpolaton. 6.1 A straght lne calbraton eqaton In Eqaton (5.4) n Secton 5.3., the gan of the APEX mager s determned sng the ntegratng sphere and flter, and then sed n Eqaton (5.6) to convert the APEX sgnal ( DN ) when vewng a scene nto scene radance, L. Ths approach oversmplfed the staton. In practce, the process s repeated wth several dfferent flters to obtan a response of APEX to dfferent sgnal levels. The sphere-flter radance (as a spectral qantty) and the sgnal response of APEX to those radance levels are shown n Fgre 14. Fgre 14 Dfferent radance levels presented to APEX for dfferent flters n front of the sphere (left) and the sgnal on APEX for each of these levels (rght). From these reslts, at any partclar wavelength, vales are obtaned for the sgnal at dfferent radance levels. Example reslts are shown n Fgre 15. A straght lne calbraton fncton s ftted to these reslts and from ths two parameters are obtaned the slope and the orgn offset. The measred vales at dfferent radance levels provde the calbraton eqaton L = GAPEX DN + L0 (6.1) where L s the measred radance n a partclar band, L 0 s the offset radance, G APEX s the gan for that bandand DN s the measred sgnal (n dgtal nmbers) n that band. Ths can also be wrtten as DN L = + DN0, (6.) G APEX where DN0 = L0 G s the sgnal offset. APEX ~ 61 ~

70 Fgre 15 Example ft for a partclar band of the sgnal on APEX and the sorce radance Ths approach s smlar to that descrbed n Secton There a sngle measred vale s sed to determne the gan and the offset s assmed to be zero (or that DN 0 s the dark readng that s sbtracted from the measred sgnal). The straght lne calbraton eqaton expands ths concept slghtly by sng data from other measred radances and allowng for any offset that sn t flly removed by the dark readng. When the nstrment s then sed to measre a scene, Eqaton (6.1) replaces (5.6). Straght lne calbraton eqatons are often sed for calbratng nstrments. If the nstrment has a small non-lnearty, then the ncertanty assocated wth the assmpton of lnearty can be predcted by the qalty of sch a ft. If the nstrment s sgnfcantly non-lnear, then hgher order calbraton eqatons may be sed nstead. Althogh the analyss below s specfc for a straght lne calbraton eqaton, many of the concepts apply to hgher order eqatons too. 6. Uncertanty analyss overvew The straght lne calbraton eqaton s a specfc example of the more general problem of fttng a model to expermentally obtaned data. Here the model s a straght lne. For all fttng problems we mst consder that: There s ncertanty assocated wth the npt vales (both on the horzontal and the vertcal axes) There may be some correlaton assocated wth npt vales The otpt of the fttng problem wll nclde the model parameters (for the straght lne, the slope and the ntercept) and ther covarance matrx (provdng the ncertanty assocated wth each of them and the correlaton between them). The ~ 6 ~

71 otpt parameters wll always have some assocated correlaton becase they were derved from the same npt vales. The ncertanty obtaned from the ft assmes that the model s correct. We also need to consder any ncertanty assocated wth the stablty of the model sed. The straght lne ft s consdered n detal n the sbseqent sbsectons. Ths sbsecton provdes an nttve overvew of the analyss. If the measred data vales that wll be ftted by the straght lne have only random errors,.e. the ncertanty assocated wth those data vales s assocated only wth random effects, then the straght lne wll average ot some of those effects. The ft wll go throgh the data wth ponts ether sde of t. Becase of ths, the ncertanty assocated wth a vale determned from the straght lne s smaller than the ncertanty assocated wth any one measred vale that went nto the ft (Fgre 16, left). One way to nderstand that s that the ftted parameters are determned from all the measred data, jst as n an average, the average s determned from all the vales. And therefore, as n an average, the ncertanty assocated wth the ftted pont s generally smaller than the ncertanty assocated wth the ndvdal measred vales. If the measred data ponts that wll be ftted by the straght lne have systematc errors,.e. the ncertanty assocated wth those data vales s at least partally assocated wth systematc effects, then the ftted lne wll not correct for these common effects 3. Ths means that ths ncertanty assocated wth systematc effects wll be blt nto the straght lne obtaned, and the ncertanty obtaned from ponts on the straght lne wll not be redced throgh averagng (Fgre 16, rght). Fgre 16 A straght lne ft, on the left wth random errors only. Here the ft s mch closer to the tre vale than are ndvdal ponts. On the rght wth systematc and random errors. Here the ft shows an offset from the tre vale becase of the systematc effect common to all the measred data ponts. 3 Note that systematc effects can have dfferent vales from pont to pont f there s a senstvty coeffcent that vares, for example a systematc stray lght effect may affect hgher radance levels more than lower radance levels, provdng a tlt to the straght lne. ~ 63 ~

72 Ths s why t s mportant to nderstand the dependence of the npt data vales on random and systematc effects by assgnng a covarance matrx to them. That s done for the APEX sensor as dscssed n Secton 6.3. The otpts of a fttng process are the model parameters n ths case the slope and ntercept of the straght lne. Whether or not the npt vales have assocated correlaton, these otpt parameters wll always be correlated becase they were derved from the same npt vales. Parameters are derved from the ft, so n ths case, when the APEX nstrment s n flght, the measred scene dgtal nmbers are converted to a scene radance sng the gan and offset (Eqaton (6.1)). Becase the gan and offset are correlated de to the ft, the ncertanty assocated wth the otpt qantty (scene radance) mst consder the fll law of propagaton of ncertantes. Ths s dscssed n Secton 6.6. The fnal consderaton s whether the model we se, n ths case the straght lne, s an approprate ft to the data. The ncertantes assocated wth vales obtaned from the model are sally srprsngly small, becase of the averagng effect descrbed above. Ths may not be a tre representaton of the ncertanty assocated wth the otpt, however, f the ft s not a good ft to the measred data. Ths s dscssed n Secton Calbraton data and ncertantes The overvew n the prevos secton shows the mportance of nderstandng the covarance of the npt measred vales. There are three ways to estmate ths: If they have come from a prevos step nvolvng fttng or nterpolaton, the assocated covarance matrx wll be calclated n that step. If they have come from expermental data or data modelled n a Monte Carlo smlaton, then the correlaton can be estmated sng the eqatons n Secton If they come from expermental data and yo have a relable nstrment model, yo wll be able to estmate the covarance from yor nderstandng of the processes, as descrbed n Secton The thrd approach s often the most sefl and t s ths approach that s sed n the APEX example. The gan and offset for the APEX nstrment (at each spectral channel analysed ndependently) are determned from measrements made of the response of APEX to lght of dfferent radance levels. The dfferent radance levels come from ntrodcng netral densty flters n front of the ntegratng sphere. The two varables are sorce radance and the sgnal (dgtal nmbers) on APEX. Whch s consdered the x-varable and whch the y-varable depends on whether Eqaton (6.1) or (6.) s sed. The second of these eqatons s more nttve we vary the radance level and see the measred sgnal, bt the former s more easly appled to a scene radance calclaton later as t drectly provdes the gan. ~ 64 ~

73 Ether way, n order to ft the straght lne, we need to determne the ncertanty assocated wth measred vales of the two varables: sgnal and sorce radance. In order to determne the covarance we need to determne what s common to the measred vales and what vares from measrement to measrement. Consderng frst the sgnal (dgtal nmbers); as dscssed n Secton the ncertanty here comes from nose on the lght sgnal and nose on the dark sgnal. Nose, by defnton, changes from measrement to measrement, and therefore the sgnal vales are consdered entrely ncorrelated 33 for all the sgnal-radance level pars. The ncertanty assocated wth the sorce radance s dscssed n Secton It comes from the sphere radance ncertanty (tself a reslt of the chan descrbed generally n Secton 5). It also comes from the ncertanty assocated wth the flter transmttance measrement, optcal nterreflectons between the sphere and the flters, temperatre senstvtes of the flters n front of the spheres, sphere stablty and flter ageng. The domnant ncertanty here s that assocated wth the sphere radance. Ths s common to the measrements at all radance levels as t s the same sphere sed. The next most domnant ncertanty s the calbraton of the flters by NPL. Ths wll ntrodce partal correlaton becase NPL has measred all the flters on the same faclty sng the same technqe. Generally speakng, measrements at an NMI wll mnmse the ncertanty assocated wth random effects, by takng repeat readngs. Therefore t s lkely that althogh ths s partal correlaton, the correlaton coeffcent wll be hgh. Interreflectons are lkely to be smlar for all flters and snce all the flters are the same type (netral densty flters), ther temperatre senstvtes and ageng rates may be smlar. There may be some flterdependent ageng and the sphere nstablty. Overall, the measred vales can be consdered hghly correlated and as a frst approxmaton, flly correlated. Ths we have the staton where the sgnal levels (dgtal nmbers) are entrely ncorrelated and the sorce radance levels are entrely correlated. The covarance matrx s therefore smple to prodce. The covarance matrx s sqare wth n rows and colmns where n s the nmber of ( x, y ) measred ponts sed for the ft. It effectvely has for qarters, each sqare wth n n vales. The top left qarter represents the covarance between the dfferent x-vales. The bottom rght qarter represents the covarance between the dfferent y-vales. The bottom left and top rght qarters represent the covarance between the x-vales and the y-vales. These are ncorrelated and these qarters have zeroes throghot. Down the dagonal s the varance (the sqared ncertanty) assocated wth each measred vale ndvdally. 33 Note, that f the same dark readng s sbtracted for each radance level, then there wll be a correlaton ntrodced by ths choce. Sch a decson may be made f the dark sgnal s relatvely stable, as a way of savng tme. Generally, ths decson s made f the dark sgnal s small n comparson to the lght sgnal. And therefore the ntrodced correlaton wll also be small. ~ 65 ~

74 The x terms or rather, the sorce radance levels, L sph-flt,, are consdered flly correlated. If they have an assocated relatve ncertanty (expressed n percent) of rel ( L sph-flt ) rel ( L sph-flt ) sph-flt, elements are ( ), then for that qarter of the covarance matrx the dagonal terms are L (.e. the absolte ncertanty sqared) and all off-dagonal L L (.e. the prodct of the two absolte rel L sph-flt sph-flt, sph-flt, j ncertantes). The y terms or rather, the sgnal (dgtal nmber) terms DN APEX,cal,, are consdered DN, then ncorrelated. If they have an assocated relatve ncertanty of rel ( APEX,cal, ) ( ) rel APEX,cal, APEX,cal, for that segment of the covarance matrx, the dagonal terms are DN DN (.e. the absolte ncertanty sqared) and all offdagonal elements are 0. Therefore, wrtng for compactness ( ) ( ) rel APEX,cal, APEX,cal, DN, L L = LL and rel L sph-flt sph-flt, sph-flt, j s f j DN DN = DN, the fll covarance matrx 34 s U x x x 1 = n y y y 1 x x x y y y 1 n 1 s fl1 s fll 1 s fll 1 n s fll1 s fl s flln L L L L L s f n 1 s f n s f n DN 0 0 DN, DN 0 DN, n DNn, n n DN (6.3).e. the block n the top left corner has terms to accont for the covarance. The block n the bottom rght corner has vales down the dagonal only, and the bottom-left and top-rght corners are all zeroes. Note that ths process has to be repeated separately for each spectral band. The covarance matrx for each wll be dfferent, even f the relatve ncertantes are the same at each wavelength, snce the absolte vale of radance (and hence absolte ncertantes) are dfferent. 34 The red terms n Eqaton (6.3) are helpfl labels that wold not normally be nclded n ths eqaton. They show how the dfferent rows and colmns represent the dfferent x vales and then y vales. ~ 66 ~

75 6.4 Determnng the ft coeffcents (dong the ft) Approaches to take There are dfferent approaches that can be taken to estmatng the ft parameters and ther assocated ncertanty. Most of these wll be some form of least sqares analyss and least sqares analyss can be performed at dfferent levels of complexty. The least complex wold be nweghted least sqares (the npt vales are gven eqal weght and no accont s made of ther assocated ncertanty or correlaton), then weghted least sqares (the npt vales are gven dfferent weghts dependng on ther assocated ncertanty) and fnally generalsed least sqares (the covarance matrx of the npt vales s taken nto accont n the ft). The assocated ncertanty analyss can also be performed for these dfferent levels of complexty. The most rgoros approach wold se generalsed least sqares for the ft and obtan a covarance matrx for the ft parameters from ths analyss. Here we consder the followng levels of complexty: 1. Usng an nweghted ft and calclate the assocated ncertantes and covarance sng Monte Carlo smlaton.. Usng a weghted ft and obtan the ncertantes and covarance from the process 3. Usng a generalsed ft. The second and thrd of these are based on the concepts descrbed n Brtsh Standard DD ISO/TS 8037:010 Determnaton and se of straght-lne calbraton fnctons, whch descrbes how to deal wth all these cases and provdes a recpe for determnng both the ft parameters and ther assocated covarance. Matlab code to apply those recpes s avalable, for free download, at: Unweghted ft and Monte Carlo Dong the ft Fttng a straght lne to measred data ponts s a common procedre that s blt nto most programmng langages. For example, n Excel, the formla 35 : =INDEX(LINEST($D$5:$D$14,$A$5:$A$14),1) wll gve yo the slope of a straght lne ft throgh the ponts wth x-vales n A5:A14 and y-vales n D5:D14. The offset s gven by: =INDEX(LINEST($D$5:$D$14,$A$5:$A$14),) 35 INDEX(,1) gves the frst vale n an array. INDEX(,) gves the second vale. LINEST(y-vales,x-vales) retrns an array where the frst vale s the slope and second vale the offset. ~ 67 ~

76 In Matlab the eqvalent command s wrtten c = polyft(x, y, 1); where x s a vector of measred x-vales (here dgtal nmbers) for one band at several radance ntenstes. The length of x s N where N s the nmber of radance levels mltpled by the nmber of measred vales for each level (.e. the total nmber of measrements). Y s a vector of the measred y-vales. Here a vector of the APEX convolved npt radance, of same length as x wth replcatons of the same vales for each radance level. c s a vector wth two elements holdng the slope ( 1 G APEX ) and offset ( 0 ) ft. DN of the lnear Both Excel and Matlab se least sqares analyss to determne the ft, althogh wth some mnor dfferences n approach. Least sqares analyss s a standard approach to determne the best solton to a problem where there are more eqatons than nknowns. It determnes the overall solton that mnmses the sm of the sqares of the resdals (the dfferences between the raw data and the ftted lne). So, for a straght lne, descrbed by y = a + bx, the fncton beng mnmsed 36 s N ( ). (6.4) S = y a + bx = 1 For a straght lne, and for ordnary, nweghted, least sqares, the solton can be calclated analytcally sng the followng steps: Step 1: Calclate the means of x and xmean x N ymean = y N Step : Calclate the dfference between each data pont and the mean and sm the prodcts: N = 1 N = 1 y : = ( ), ( ) ( )( ) T = x x y y mean ( ) B= x x mean Step 3: Calclate the slope and ntercepts: N = 1 mean ( x xmean )( y ymean ) N ( ) T b = = B x x = 1 mean 36 Ths s not exactly how matlab does t. The polyft rotne can deal wth polynomals and the straght lne s a smple case. ~ 68 ~

77 a = y bx (6.5) mean mean Ths nsophstcated straght lne ft assmes that all the data ponts have the same weght, that the varablty s de to random nose only and there s no ncertanty assocated wth the x vales. It also does not provde an ncertanty assocated wth the calclated slope and ntercept. Uncertanty estmate sng Monte Carlo The nsophstcated straght lne ft descrbed above, and calclated sng Eqaton (6.5) or the Excel or Matlab expressons gven above, does not provde the ncertantes and covarance assocated wth the slope and ntercept of the ftted straght lne. Whle t s possble to calclate the ncertanty and covarance analytcally (as descrbed n the sectons below), t s also worth 37 consderng an alternatve the se of Monte Carlo smlaton. In Monte Carlo smlaton, many (1000 or more) smlatons are rn, n each of whch the npt qanttes are adjsted by a random varable wthn ther ncertantes. In the example beng consdered here, we create a model for a perfect tre answer by assgnng an arbtrary bt realstc vale to the gan and offset and calclatng tre sgnal-system radance x, y vales. We then rn a loop where n each loop errors are drawn from the pars ( ) approprate probablty dstrbton fnctons (sally a Gassan dstrbton wth a standard devaton eqal to the assocated standard ncertanty). In ths case, for the sgnal (whch s ncorrelated from pont to pont), a dfferent error s drawn for each y wthn one loop. For the system radance (whch s flly correlated) the same error s drawn for all x (or the same relatve error s drawn and mltpled by each x ) wthn one loop. In the next loop, new random nmbers are drawn from the dstrbtons. At the end of each loop, a ft s appled to the data, obtanng a par for the slope and ntercept of the straght lne. These pars can be plotted, as shown n Fgre 17. Havng rn a large nmber of loops, the ncertanty assocated wth each (slope and ntercept) can be determned from the standard devaton of the vales for that parameter. The covarance can be calclated sng Eqatons (3.46) and (3.47). 37 The solton descrbed here s pragmatc. It s often the case that some form of fttng or nterpolaton or other algorthm s sed n a processng chan and that the algorthm has been determned wth lttle thoght for rgoros ncertanty analyss. Later an ncertanty bdget s pt together. In those cases, Monte Carlo provdes a smple method to pgrade the analyss to nclde ncertantes wthot rethnk. The Monte Carlo analyss descrbed here does not gve the same answer as the more sophstcated approaches of the sbseqent sectons becase the ntal ft s stll done wth an nsophstcated nweghted least sqares rotne. The more sophstcated approaches take the ncertantes (and covarance) nto accont n the ft. ~ 69 ~

78 Fgre 17 Plots of offset and gan for dfferent loops of the Monte Carlo smlaton. Ths shows a possble mld negatve correlaton (hgh vales of one parameter tend to occr wth low vales of the other parameter) Weghted ft and analytcal covarance Dong the ft In ths case we consder fttng a straght lne to measred data ponts wth some assocated ncertantes whch may dffer from pont to pont. For example, f the ncertanty assocated wth nose s hgher at the lower radance level measrements than for the hgher radance level measrements, then the ft shold have a stronger weghtng at the hgher radance level measrements. The method descrbed here calclates both a slope and ntercept for the straght lne and ther assocated ncertantes and covarance sng the ncertantes assocated wth the y-vales. There s assmed to be no ncertanty assocated wth the x- vales. For the example consdered here, t therefore makes sense to have the x-axs the radance levels and the y-axs the sgnal (dgtal nmbers) becase the dfferent radance levels are (assmed to be) flly correlated, whereas the sgnal vales have ncertantes assocated wth random effects. The calclaton of the slope and offset s as follows: The weghts 38 are defned as 38 Becase ths term s sqared n the sbseqent eqaton, the actal weght s nversely proportonal to the sqare of the ncertanty. ~ 70 ~

79 1 w y = (6.6) where ( y ) s the ncertanty assocated wth the measred vale y (here the sgnal) at the set vale x (for a gven radance level). The reference vales are gven by ( ) x N N wx 1 wy = = 1 0 =, y N 0 = N w 1 w = = 1. (6.7) The slope (here 1 G APEX ) s then calclated as b = N = 1 ( 0)( 0) w ( x x ) w x x y y N = 1 0 (6.8) and the ntercept (here DN 0 ) as a = y0 bx0. (6.9) The varance (sqared ncertanty) and covarance assocated wth the slope and ntercept are gven by Uncertanty analyss ( b) N w ( ) 1 x x = 0 1 x0 ( a) = + N N w = 1 w ( 0 ) 1 x x = ( ) ab = 1 x 0, =. N w ( 0 ) 1 x x =,, (6.10) Here the ncertanty (and covarance) s gven by Eqaton (6.10) nder the ntal condtons,.e. that the ncertantes are only assocated wth random effects n the y-vales (the sgnal). We dd not take nto accont the ncertantes assocated wth the x-vales (the radance levels). Earler we conclded that the radance levels can be consdered flly correlated. Therefore there s a systematc effect on the x-vales. If the error s an absolte error, an addtve effect n the same nts, then t wold have the effect of shftng the entre straght lne to the left or rght. Ths wold have no effect on the slope (hence nstrment gan) or ts assocated ncertanty. It wold, however, affect the offset (Fgre 18). A rgoros ncertanty analyss reqres the fll covarance matrx, bt an ndcatve one can be determned by treatng ths as an ncertanty assocated wth the mean vale, x 0 n Eqaton (6.9). The ncertanty assocated wth the ntercept n Eqaton (6.10) wold have an ~ 71 ~

80 addtonal term ( ) ndcatve ncertanty. + b x 0. Ths s by no means a rgoros analyss, bt t provdes an Fgre 18 An error n the x that s common to all vales wll shft the ftted crve (n ths case to hgher vales). Ths has no effect on the slope, bt wll change the offset Rgoros analyss A rgoros analyss wold take nto accont the ncertantes and covarance assocated wth both the x and y. A recpe for dong ths calclaton s gven n the Brtsh Standard Standard DD ISO/TS 8037:010 Determnaton and se of straght-lne calbraton fnctons and Matlab code s gven n the lnk on page 67. The most general case s gven n TS8037_GGMR1. The software reqres as npt: The x-vales n a vector The y-vales n a vector A covarance matrx for the x-vales (the top left qarter of the matrx n Eqaton (6.3)) A covarance matrx for the y-vales (the bottom rght qarter of the matrx n Eqaton (6.3)) As otpt, the software provdes: The vale of the slope (b ) The vale of the ntercept ( a ) The varance (sqared ncertanty) of the slope ( b ) The varance of the ntercept ( a ) The covarance of the slope and ntercept ( ab, ) Valdaton nformaton for the model n the form of a ch-sqared test (see below). ~ 7 ~

81 6.5 Valdatng the ft The determnaton of the assocated ncertantes gven above assmes that the straght lne ft s a good ft to the measred data. Ths shold be tested, sng a statstc sch as the chsqared statstc. Applyng the ch-sqared test nvolves two parts: determnng the observed ch-sqared, χ, and then comparng t wth an approprate ch-sqared dstrbton. obs The observed ch-sqared s calclated as the sm of sqares of the weghted resdals from the measred data to the ft. For N measred vales, we calclate ( ) ( ) y a + bx = N χ obs = 1 y (6.11) Becase t s calclated from statstcal data, we expect the vale of χ obs that we obtan to be a sample from the approprate statstcal dstrbton. The dstrbton depends on the nmber of degrees of freedom. Ths s the nmber of measrements mns the nmber of ft parameters, or here ν = N. As the nmber of degrees of freedom ncreases (.e. as more and more vales were sed to determne the ft), then the χ dstrbton becomes more symmetrc and shfts to the rght. The centre of mass of the dstrbton s the degrees of freedom (that s the shft to the rght). ν Fgre 19 Ch sqared dstrbtons for dfferent degrees of freedom The ch-sqared test evalates the probablty that a partclar observed vale, χ obs came from the expected dstrbton χ ν. There are two common versons of ths test. The frst tests whether the vale s less than the 95 % percentle of the expected dstrbton (.e. t wold only be faled 5 % of the tme for good data). The second, the Brge rato, compares the observed ch-sqared wth the expectaton vale (.e. the nmber of degrees of freedom). ~ 73 ~

82 When the ch-sqared test s faled t mples that the model does not adeqately explan the data and a better model s reqred. 6.6 Usng the ft Analytcal covarance when the ft s sed However we determne the covarance, whether 39 analytcally or throgh Monte Carlo smlaton, at the end of ths process we have the eqaton y = a + bx wth the ncertantes assocated wth a and b and ther covarance. The fnal step s to calclate for a gven, measred y the assocated x. For the APEX calbraton, we have determned the parameters sng the provded radance levels L sph-flt, as the x and the measred sgnal DN APEX,cal, for that radance as the y. We now need to calclate the radance of a scene observed by APEX from a measred sgnal. Ths s done sng Eqaton (6.1). Consder frst the generc eqaton: y = a + bx ( ) x= y a b (6.1) Therefore, later, when the nstrment s sed, we measre a sgnal y (wth an assocated ncertanty) and convert t to the parameter x. Here we measre ( y ) the scene sgnal n dgtal nmbers (wth nose n that scene sgnal) and we convert t to the scene radance ( x ), sng the second verson of Eqaton (6.1). We have the senstvty coeffcents c c c y a b x 1 = = y b x 1 = = a b x = = b b ( y a) (6.13) Applyng the fll law of propagaton of ncertantes, gves ( ) = ( ) + ( ) + ( ) + (, ) x c y c a c b cc ab y a b a b ( ) ( ) ( ) ( ) ( ) (, ) y a y a b y a a b = b b b b. (6.14) It may help codng ths nto a langage lke MATLAB to wrte ths same eqaton n matrx notaton: 39 Note that there won t be exactly the same vales from these two processes, bt they are lkely to be smlar ~ 74 ~

83 T ( x) = c Vc, ( ) ( ) ( ) ( ) c a a ab, 0 c = cb, V = ab, b 0. c y 0 0 ( y) (6.15) Translatng ths for the APEX calbraton nto ts notaton, a = DN0 = L0 GAPEX and b 1 GAPEX. x Lscene, y DNscene, 6.7 Straght lne nterpolatons The prevos sectons of ths chapter have related to sng a ft. Sometmes, however, data s nterpolated rather than ftted. Ths s done when the data ponts are a relable ndcaton of the desred fncton and addtonal nterpolated data ponts are reqred. It s often the means to obtan spectral nformaton where measrements are made at a sbset of the fll set of wavelengths. The smplest nterpolaton s a straght lne (lnear) nterpolaton between data ponts (jonng the dots n a graph wth straght lnes). A lnear nterpolaton wll take the form y = y + x x ( ) y x y x 1 0 (6.16) where the measred data ponts ( x, y ) and (, ) 0 0 x1 y 1 have a spacng x= x1 x0 horzontally, and y = y1 y0 vertcally. We are nterested n the ncertanty assocated wth the nterpolated vale y de to ncertantes assocated wth y 0 and y 1. We assme the horzontal scale has no assocated ncertanty for smplcty. The senstvty coeffcents are ( x x ) ( ) y x x = = y x x x x 0 1 1, y x x = y x x (6.17) Note that both these senstvty coeffcents are fractons of the horzontal axs spacng. The frst one, the senstvty of the nterpolated vale to the frst (left hand) pont, s the fractonal dstance from the nterpolaton wavelength to the rght hand pont.e. the frther t s from the rght hand pont, the more senstve t s to varatons n the left hand pont. And vce versa for the second senstvty coeffcent. For a pont n the mddle, where both senstvty coeffcents are 1, the determned nterpolated vale, s eqally senstve to both ends. The ncertanty assocated wth the nterpolated vale s therefore gven as ~ 75 ~

84 1 1 y y y ( mdpt ) = ( 0 ) + ( 1) 1 (6.18) And, f the two measred data ponts have eqal ncertantes, the ncertanty assocated wth the nterpolated vale s redced by. Ths s clearly nderstood, as the nterpolated vale s the average of the two end ponts and we have already seen that averagng redces the ncertanty by the sqare root of the nmber of measred ponts. When the nterpolated pont s elsewhere than the mdpont, t wll be more senstve to the measred vale t s closer to, and the ncertanty redcton wll be smaller (assmng both measred data ponts have eqal ncertantes). Ths can be seen graphcally n the followng dagram. The lnes and sold ponts represent dfferent random measrements of the two end ponts. Clearly the nterpolated ponts are closer together than the two end ponts. Interpolaton always appears 40 to redce ncertantes and the longer the dstance over whch the nterpolaton s carred ot the more the redcton (the central ponts have smaller ncertantes than those closer to measred vales). Fgre 0 A lnear nterpolaton between measred vales at 300 nm and 350 nm. The spread of vales at the two ends (at 300 nm and at 350 nm) s de to measrement ncertanty. It s clear that a pont n the centre experences a smaller range of vales (and hence a smaller assocated ncertanty) than the orgnal measred vales. Inttvely thogh, we realse that nterpolaton does not really redce ncertantes. As wth fttng, a blnd applcaton of the Law of Propagaton of Uncertantes wthot thoght can falsely nderestmate ncertantes. Here, also, we mst consder whether a straght lne ft s an approprate ft to the measred data. Consder, for example the followng dagram. Ths s 40 See below. Interpolaton redces ncertantes from a mathematcal perspectve, assmng that the nterpolaton fncton s approprate. ~ 76 ~

85 the spectral measred vales for the external qantm effcency of a trap detector, a standard reference slcon detector sed at NPL. The ble dots are measred vales, and the red lne a lnear ft. Fgre 1 Measred and nterpolated data for the qantm effcency of a reference slcon trap detector. The 476 nm measred vale s essental to determne the correct shape. If the measred vale at 476 nm s removed, then the straght lne ft wold provde the dashed nterpolaton, wth low ncertantes at 476 nm (as t s almost half way between the other measred data ponts). Ths ncertanty wold woeflly nderestmate the tre error at the mssng data pont. Agan, as wth fttng, some test needs to be performed to estmate the qalty of the nterpolaton. Methods to do ths nclde: Use a hgher order ft as well (say a cbc splne), and compare the nterpolated vales for the dfferent ft parameters If there s a large nmber of data ponts, nterpolate takng ot every other data pont and compare the reslts wth those ponts mssng to the reslts wth the ponts nclded Take addtonal expermental vales where yo nttvely feel that the ft qalty may be poor (or hand drawn lnes are generally closer to the tre vale than lnear fts). ~ 77 ~

86 7 Spectral Selecton Most radometrc earth observaton sensors have some spectral selecton to obtan spectral nformaton of the scene. Whether ths s n the form of a few broad spectral bands, defned by a flter, or of a hgh resolton spectral dsperson n a spectrometer, spectral selecton provdes wavelength-dependent nformaton and has ts own ncertanty reqrements. Ths secton dscsses the ncertanty aspects of spectral selecton. In many magng systems a two-dmensonal array provdes smltaneos spectral and spatal selecton. Many of the concepts descrbed here for the spectral channels also apply to the spatal channels. 7.1 Spectral response fncton The two most common methods for spectral selecton are to place a flter n front of an ndvdal detector, or to have a dspersve element (a gratng, or occasonally prsm) and an array detector. Whchever method s sed, the nstrment s defned by the spectral response fncton (SRF), whch s the spectral responsvty of the combned nstrment (ncldng both the transmttance or reflectance of the flter or dspersve element and the responsvty of the detector and spectral transmttance throgh any other optcal element n the path e.g. telescope). The most accrate way to determne the spectral response fncton s to scan a spectrally tneable laser across the spectral response regon of nterest of the nstrment, and a lttle beyond, and compare the response of the nstrment at each laser wavelength to the response of a reference detector of known spectral responsblty. However, althogh deal, tneable lasers are not always readly avalable and sometmes the added complexty s nnecessary for the reqred accracy. For a flter system a more common alternatve s to obtan tneable monochromatc radaton sng a monochromator llmnated by a lamp. Here t s necessary to ensre that the bandwdth of the monochromator s sffcently narrow not to dstort the measred spectral response fncton. For a spectrometer-based nstrment a smlar process can be followed, or, makng the assmpton that the spectrometer s responsvty s constant over a bandwdth, the sgnal from adjacent spectrometer pxels can be sed to determne the spectral response fncton. Where a spectrometer has many pxels, for example for the APEX mager descrbed n Secton 5, whch has p to 530 spectral bands for each of 1000 spatal channels, then t s not possble to determne the SRF ndependently for all bands (pxels). Instead the SRF s determned for several pxels sng a lamp llmnated monochromator that s tned across the spectral band of the measred pxels. A Gassan s ftted to the measred data as the monochromator wavelength s tned and from the Gassan fts two parameters are obtaned: the bandwdth and the centre wavelength. These are plotted as a fncton of pxel nmber and therefore Gassan fnctons can be determned for all other pxels by nterpolatng of extrapolatng the measred data. A smlar method was sed for the MERIS satellte mager [4]. ~ 78 ~

87 7. Uncertanty assocated wth the spectral response fncton The ncertanty assocated wth the determned spectral response fncton comes from several sorces: Uncertantes relatng to the spectral measrements o Uncertantes assocated wth the wavelength of the monochromatc lght sed (partclarly the wavelength scale of the monochromator) o Uncertantes assocated wth the bandwdth of the monochromatc lght sed o Uncertantes assocated wth nose n the measrement of the spectral response fncton Uncertantes relatng to the nterpolaton o Where a spectral response fncton s determned for measred data at a set of wavelengths, ths relates to the nterpolaton between the measred data ponts (the monochromatc llmnaton sed). Were sffcent wavelengths sed for the llmnaton sorce to determne the fll shape of the spectral response fncton? o Where a Gassan (or smlar fncton) s ftted to the measred spectral response fncton ths relates to to what extent a Gassan s a good ft to the measred data ponts. o Where the SRF of only a few pxels of an array are flly charactersed, and these propertes are then nterpolated for other pxels of the array, t relates to the qalty of the SRF ths determned compared to the tre SRF for those ntermedate pxels. Uncertantes relatng to changes snce calbraton[1] o The SRF may change smply de to storage ths s partclarly tre for nterference-flters. o The SRF may change de to vbratons on transportaton and lanch. o The SRF may change de to changes of temperatre, hmdty and the move to vacm. o The SRF may change de to both ltravolet and hgh energy solar radaton damage. o Contamnaton flms can change spectral and absolte levels of transmttance. It s beyond the scope of ths docment to provde a rgoros analyss of all of these concepts. The sze of each effect mst be estmated for an nstrment of nterest and ths can be done throgh expermentaton or modellng. For example, an estmate of the nose on ndvdal SRF data vales can be obtaned by repeatng the calbraton mltple tmes and observng the spread of obtaned reslts and an estmate of the effect of nterpolaton can be obtaned by sng dfferent nterpolaton fnctons (see Secton 6.6). The qalty of the ft can be tested sng the types of test descrbed n Secton 6.3, or by tryng dfferent models. Estmates of change snce calbraton can be obtaned by calbraton before and after se or before and after smlated exposre to a space-lke envronment. ~ 79 ~

88 Note also, that where two parameters are ftted from the same data, for example the central wavelength and bandwdth of the Gassan, as n Error! Reference sorce not fond., then there wll be some correlaton between the two ft parameters. What we are really nterested n estmatng s the ncertanty assocated wth the measred vale obtaned by the nstrment (of e.g. spectral radance) de to the ncertanty assocated wth the SRF (tself a reslt of all these effects). To nderstand ths we need to realse that any spectral measrement reles on a spectral ntegral or convolton wth the spectral characterstcs of the scene nder observaton and ths s the sbject of the next secton. 7.3 Spectral ntegrals and convolton Orgn of spectral ntegrals In Secton 5, the APEX nstrment calclaton eqaton was gven by Eqaton (5.6): Lscene = GAPEX DNAPEX,scene. If we convert ths to gve the measred sgnal on APEX as a fncton of the radance of the scene we get the eqaton DNAPEX,scene = Lscene GAPEX = RAPEX Lscene. (6.19) where R APEX = 1 G APEX s the APEX responsvty. In practce, ths eqaton oversmplfes the process becase a pxel of the APEX mager does not measre a sngle wavelength, bt has a SRF. Therefore eqaton (6.19) shold be wrtten as where ( ; ) SRF,APEX 0 ( ) ( ) DN = R λλ ; L λ dλ (6.0) APEX,scene SRF,APEX 0 scene R λλ s the spectral response fncton of the APEX pxel centred on wavelength λ 0. Whatever the nstrment, whether t ses a spectrometer or a fltered system to defne the wavelength band, the real measred vale wll be some eqaton that s of the form of (6.0). The spectral response fncton mplctly creates a weghted ntegral of the scene radance. Ths s sometmes descrbed as a spectral convolton. In some cases, where the SRF s narrow band, the sbseqent analyss treats the measred vale as thogh t were made at the centre wavelength and was trly monochromatc. In these cases, the problem s treated as one of nstrment bandwdth and the ncertanty analyss shold consder the ncertanty assocated wth the assmpton that the measrement s effectvely made at a sngle wavelength. Generally, thogh, for both magng spectrometers and fltered nstrments, the measred sgnal mst be treated as a band-ntegrated qantty and nterpretaton of the reslts mst acknowledge the mplct ntegral descrbed by Eqaton (6.0). Ths nterpretaton sally reles on some knowledge of the spectral radance of the scene at a hgher resolton than ~ 80 ~

89 the measrement provdes. For example, n Fgre we model the otpt (red dots) of the Landsat-ETM nstrment by ntegratng a scene radance measred wth a hgher resolton ASD spectrometer (ble) and the SRF of the Landsat-ETM bands (green). Fgre : Example of a broadband sensor convolton, synthessng a radance spectrm measred wth an ASD spectrometer wth Landsat ETM. (Note red lne for ndcaton only, only the dots have meanng they provde band-ntegrated radance vales) Calclatng the ntegral In the analyss of the measred vales from any sensor, there s a need to determne ntegrals of the form of Eqaton (6.0). Generally the relevant spectra are provded as dscrete vales at gven wavelengths. A common method for evalatng sch ntervals s the trapezm (or trapezodal) rle whch approxmates the ntegral by treatng the ntegrand as varyng lnearly between adjacent measrement ponts (Fgre 3). ~ 81 ~

90 Fgre 3 Trapezm rle dagram The trapezm rle replaces the ntegral n Eqaton (6.0) wth a smmaton (6.1) where, whch depends on the wavelength spacng ether sde of, s an approprate weghtng term. If the data are evenly spaced, sch that for all wavelength steps, then, for the trapezm rle (6.) Hgher order rles (that ft dfferent pecewse polynomals to the data) are also avalable [13] and may be stable where the trapezm rle over-smplfes the analyss. Some care s needed f the two spectra mltpled (the scene radance and the nstrment SRF) are at dfferent wavelength steps. In these cases at least one 41 of the spectra shold be nterpolated to the wavelengths of the other spectrm. Where one spectrm s smooth and the other spky then the smoother spectrm shold be nterpolated. Ths s a strong case for replacng the nstrment SRF wth a ftted Gassan, whch can then be defned at any wavelength, when t s then mltpled by a measred scene radance wth consderable strctre (for example de to atmospherc or solar atomc/moleclar absorpton lnes) The ncertanty assocated wth the ntegral A detaled analyss on determnng the ncertanty assocated wth ntegrated qanttes s avalable for download at That report was wrtten for lghtng applcatons bt many of the concepts apply eqally for satellte bands 4. Secton 7. lsted dfferent sorces of ncertanty assocated wth the SRF tself. These dfferent categores of ncertanty have dfferent effects on the ntegral. 41 There are occasons when t s approprate to nterpolate both spectra to the wavelengths of each other 4 Emma Woollams hopes to wrte a verson of that report for satellte band applcatons sometme n the latter half of 014. In the vocablary of that report, the ntegral gven above as Eqaton (6.0) s an expermental prodct ntegral. ~ 8 ~

91 Uncertantes relatng to the spectral measrements When the ntegral s determned sng the trapezm rle from the raw measred SRF, then nose n the ndvdal measred vales of the SRF wll affect the determned ntegral. However, the natre of nose s that t s random, therefore whle one pont may be determned a lttle hgher than the tre vale, there s lkely to be another pont determned a lttle lower than ts tre vale. Ths to some extent the nose wll be averaged ot by the ntegraton process. Integraton, lke averagng, redces the effect of nose. The extent to whch the nose s redced depends on the nmber of measred vales descrbng the SRF (more ponts ncreases the averagng ot effect) and whether the redcton s sffcent to make nose neglgble depends on how nosy the data are n the frst place, however, generally speakng, nose n the determnaton of the SRF s a tny contrbton to typcal ncertanty bdgets. The ncertanty assocated wth the ntegral de to nose can be calclated ether throgh Monte Carlo smlaton (ntrodcng nose to all data ponts as separate draws from a Gassan probablty dstrbton fncton) or analytcally. An analytcal expresson s obtaned from the smmaton n Eqaton (6.1). If we wrte the frst three terms ot, we get DN R L R L R L (6.3) APEX,scene = where RSRF,APEX λ1; λ0 Lscene λ1 1. If we apply the law of propagaton of ncertantes to ths for an ncertanty assocated wth R 1, we need the senstvty coeffcent RL s shorthand for ( ) ( ) c DN = = L (6.4) APEX.scene R1 1 1 R1 And ths the law of propagaton of ncertantes (consderng only ncertantes assocated wth the R ) s ( ) ( ) ( ) ( ) ( ) ( ) ( ) DN = L R + L R + L R + (6.5) Whch, wrtten n smmaton notaton makes N ( DNAPEX,scene ) = ( RSRF,APEX ( λ; λ0 )) Lscene ( λ). (6.6) = 1 Uncertantes relatng to the nterpolaton Ths category descrbes ncertantes assocated wth the orgnal determnaton of the spectral response fncton and the assmptons made n ts determnaton. Where dscrete measred vales are sed to estmate the SRF ths relates to whether sffcent measred ~ 83 ~

92 vales have been made to descrbe the SRF 43. Where the SRF s modelled by a Gassan, ths relates to the stablty of that Gassan approxmaton and any loss of nformaton from trncatng the Gassan over a fnte range. Where the SRF s only determned for some bands and the SRF of ntermedate bands s estmated throgh nterpolaton of, for example centre wavelength and bandwdth, ths relates to the approxmatons ntrodced by those processes. It s sally not possble to gve a defntve ncertanty assocated wth these effects. Sch a defntve ncertanty mples beng able to compare the SRF sed wth a more accrate one bt f a more accrate one were avalable, t wold be sed. There are, however, technqes that can be sed to estmate some of these ncertantes. To estmate the ncertanty assocated wth whether sffcent measred ponts are avalable to descrbe the SRF and to se the trapezm rle can be estmated by comparng the trapezm rle vale wth the vale obtaned from a hgher order ntegraton rle, or by comparng a trapezm rle calclaton wth the raw data nterpolated frst sng a hgher order rle (sch as a cbc splne nterpolaton). It may also be sefl to recalclate the ntegral wth half the measred data ponts (every other pont) and compare those vales. None of these methods wll gve a defntve answer, bt they wll ndcate whether or not the effect can be consdered neglgble. Where the SRF s modelled by a Gassan, then an estmate of the ncertanty assocated wth ths assmpton can be made by comparng the ntegral calclated wth the Gassan to that calclated from the measred SRF sng the trapezm rle. Becase the ntegral nvolves the scene radance, and real scene radances are lkely to be qte spky, ths shold be done by nterpolatng (sng e.g. lnear nterpolaton) the measred SRF to the wavelengths at whch the scene radance s defned. To estmate the ncertanty assocated wth the nterpolaton of the Gassan defnng parameters (bandwdth and centre wavelength) from measrements for a few pxels, t s necessary to model the process. Ths can be done, for example, by rnnng a Monte Carlo smlaton on the measred data. For the APEX calbraton, a Monte Carlo smlaton was rn on the fll process. Frst nose was added to the raw measred SRF and dfferent Gassans were obtaned ths way (Fgre 4). 43 Whch n trn relates to whether the trapezm rle s an approprate method for ntegraton becase that assmes that yo can jon the measred ponts wth straght lnes. ~ 84 ~

93 Fgre 4 Dfferent Monte Carlo smlatons of the ftted Gassan to the raw SRF Then addtonal nose was added to accont for systematc effects, and a smlaton rn of the nterpolaton to non-charactersed pxels (Fgre 5). Fgre 5 Dfferent Monte Carlo smlatons to the nterpolaton From ths a probablty densty clod was obtaned for the centre wavelength and bandwdth of one of the ntermedate pxels (Fgre 6). Ths clod provded ncertantes assocated wth both the nterpolated bandwdth and the nterpolated centre wavelength, as well as the covarance (see Secton 3.5.3). ~ 85 ~

94 Fgre 6 Probablty densty clod of the bandwdth and central wavelength from the Monte Carlo Smlaton for band 68 of the APEX VNIR detector From the ncertanty assocated wth the bandwdth and central wavelength, we need to determne the ncertanty assocated wth the ntegrated qantty calclated n Eqaton (6.0). Ths can be done ether by contnng the Monte Carlo smlaton and determnng the ntegral nmercally for each Gassan obtaned, or by dfferentatng the Gassan analytcal eqaton wth respect to the centre wavelength and bandwdth respectvely and ths obtanng the senstvty coeffcents needed for the law of propagaton of ncertantes. Uncertantes relatng to the wavelength scale of the SRF Where the SRF s determned expermentally sng a lamp-llmnated monochromator (rather than a laser), there wll be ncertantes assocated wth the wavelength scale of the monochromator and wth the bandwdth of the monochromator makng the measrements. The monochromator bandwdth wll have the effect of compressng the peak of the SRF, makng t broader and shorter. Wavelength ncertantes fall nto three 44 categores those de to a systematc spectral offset that apples to measrements at all wavelengths (a wavelength scale error) those de to a wavelength dependent offset where the same error occrs every tme the monochromator s set to a gven wavelength, bt ths error s random from one wavelength to the next a random effect (the reprodcblty of the wavelength scale over short tme perods). 44 See also secton 7.3 of the Integral Uncertanty report fond at whch descrbes three types of wavelength effect. ~ 86 ~

95 Generally speakng, the systematc spectral offset, and some of the wavelength-dependent offset wll be corrected throgh calbraton of the wavelength scale of the monochromator. There may, however, be a resdal ncertanty assocated wth these calbratons. When the SRF s determned on mltple occasons, then random errors n the wavelength scale wll show p as random effects n the measred vales and these can be treated as descrbed above. A systematc spectral offset (common to all wavelengths) wll shft the SRF to longer or shorter wavelengths. How sgnfcant ths s depends on how qckly the scene radance changes as a fncton of wavelength. Consder, for example the case of a SRF that s close to an atmospherc absorpton featre. A small wavelength shft cold have a sgnfcant mpact on the nterpretaton of a measred sgnal f the shft changes whether or not that absorpton featre s consdered to be wthn the bandpass of the sensor. In contrast, f the scene radance s only slowly changng wth wavelength, then a small wavelength shft wll have almost no mpact. It s best to estmate the ncertanty assocated wth spectral effects by modellng sch a wavelength shft for typcal scene spectra. A spectral offset that vares wth wavelength wll almost always have a smaller mpact than a systematc spectral offset especally when there s no correlaton from one wavelength to the next. Ths s becase of the averagng effect of the ntegral and the fact that t s reasonable to assme that some offsets wll be to longer wavelengths and others to shorter wavelengths. Agan, the ncertanty assocated wth the ntegral de to ths effect wll depend on the scene radance tself and how that changes wth wavelength. It s often reasonable to assme ths effect s neglgble, althogh modellng cold be sed to estmate the sze of t. Uncertantes related to changes snce calbraton The dfferent effects descrbed n Secton 7. as ncertantes relatng to changes snce calbraton wll all affect ether the absolte level of the SRF (the nstrment gan), or they wll change the shape of the SRF shftng t n wavelength, broadenng t or creatng a spectral tlt. The ncertanty assocated wth changes snce calbraton can only be estmated from an nderstandng of the lkely changes and modellng what effect that has on the ntegral[1]. The mpact of these effects wll be larger f the scene radance has sharp spectral featres. 7.4 Stray lght (ot of band) Ot-of-band stray lght s where lght s scattered onto the wrong pxel. Spectral stray lght s where lght of one wavelength s measred as thogh t were at a dfferent (wrong) wavelength, de to scatterng mechansms wthn the nstrment. One method to correct for ot of band stray lght s to se a ct-on flter to remove all lght below the flter s ct-on wavelength. Sgnals measred at the lower wavelengths are entrely de to stray lght and can be removed by sbtracton, perhaps weghtng the sbtracted sgnal by the long wavelength transmsson of the ct-on flter. Usng a seres of bandpass or blockng flters to restrct the range of wavelengths enterng the spectrometer allows measrements to be made over a trncated spectral range wthot the nflence of stray radaton at wavelengths ~ 87 ~

96 otsde ths range. If a seres of sch bandpass flters are sed, each talored for a gven spectral regon, t s possble to make measrements over a broad spectral range whlst stll ensrng good stray lght performance s acheved [14]. Stray lght can also be charactersed and corrected sng a monochromatc sorce. A tneable laser (or, wth care, a narrow bandwdth monochromator) s sed to scan seqentally throgh each wavelength n trn. The response at other wavelengths to ths monochromatc lght s sed to create a stray lght correcton matrx whch can then be sed to correct stray lght n any measred spectrm [15-17]. If ncorrected, stray lght can case very large errors n the spectrm measred by an array spectrometer, even errors greater than 100 % n the ble end of a solar spectrm measred sng an array spectrometer wth sgnfcant short wavelength stray lght. Sch an error wll change the vale of any ntegral calclated from that spectrm, and the sze of the effect wll depend on where n the spectrm the stray lght occrs. If corrected, the correcton method wll ntrodce mathematcal correlaton to the measred spectral rradance, whch s best descrbed sng a covarance matrx. Stray lght can also come from ot-of-feld (spatal stray lght). Ths can be sgnfcant, especally f the scene vewed s nhomogeneos. For example, f one pxel s vewng vegetaton and an adjacent pxel a clod or water there cold be a sgnfcant effect on the observaton wth the brght pxel scatterng lght nto the dm pxel. Ths effect can be modelled from a pre-flght determnaton of the Modlar Transfer Fncton (MTF) of the nstrment. Some post-lanch evalaton can be carred ot, partclarly for hgh spatal resolton sensors where the mpact s most prononced, sng cheqer board or edge targets wth sharp hgh contrast edges. 7.5 Spectral and spatal effects Ths chapter has provded an overvew of some of the consderatons reqred for ncertanty analyss for spectral and spatal effects. Generally, these effects have to be modelled wth a good nstrment model and the senstvty coeffcents determned nmercally. Ths s stll the sbject of research, and ths chapter has smply ntrodced some methods that can be sed. ~ 88 ~

97 8 Post-lanch calbraton and Level-1 EO prodcts radometrc ncertanty 8.1 Imager changes n orbt The spectral response fncton of a satellte sensor s lkely to change on the transton to orbt (throgh the vbraton and temperatre cycles of lanch, throgh the dfference between vacm and ar operaton) and once n orbt (reversbly de to temperatre cycles, or rreversbly de to solarsaton and other damage). Sch changes affect the SRF, and smlar effects wll also alter the nstrment gan, dark offset and lnearty as components, sch as mrrors or electronc systems are damaged by hgh energy radaton and vbraton. An example of reversble effect s the mpact on the Dark Sgnal levels when measrng near an area named the Soth Atlantc Anomaly (SAA), Fgre 7. At ths area, hgh energy partcles are confned n the nner Van Allen belt whch s translated nto spontaneos peaks n the readngs levels as detected by the MERIS sensor n [4]. Fgre 7 MERIS Dark offset drng orbt 5, OCL-ON (left), orbt 9, OCL-ON (rght), band 16 (smear band), vs. pxels (front axs) and tme. Spkes are clearly vsble at the rght mage when crossng the Soth Atlantc Anomaly (SAA). However, ths same effect of space radaton as well as other effects lke the radaton of the sn tself, the changes of temperatre and so on; can prodce long term drfts n the nstrment gan and spectral responses whch are acconted as rreversble effects. The example n Fgre 8, from [18] shows how the gan of MODIS Terra and Aqa gans as changed drng the msson lfetme as montored by the sn dffser calbraton system. ~ 89 ~

98 Fgre 8 Trends of Terra and Aqa MODIS gans for bands 1 7 and 6 obtaned from the SD/SDSM measrements. The sold and dashed lnes are for mrror sdes 1 and, respectvely, from [18] It s nevertheless tre that some of these changes can be well charactersed and modelled pre-flght. However, these models have a lmted relablty and cannot accont for all scenaros (e.g. contngences drng the operatonal phase) and they typcally need to rely on a valdaton and/or pdate n-flght. Sch an pdate s done sng an optcs degradaton model [19] whch provdes an exponental trend for the degradaton of the optcs n space envronments. It was frstly sed for the SeaWFS msson and later on n the MERIS msson, n both cases t was necessary ts valdaton and/or pdate sng ether Earth, moon or sn dffser measrements. The pre-flght calbraton and charactersaton provdes a record of the nstrment performance pror to lanch and t s a sefl ndcaton and/or correcton of the performance once n orbt. However, t s necessary to montor the nstrment performance n-flght so that the effects (reversble or rreversble) of workng n a space envronment are well valdated and/or pdated n the nstrment calbraton and charactersaton. 8. On-board calbraton systems One method for charactersng changes drng flght, s to provde on-board calbraton systems. For example, the on-board spectroradometrc calbraton assembly for MODIS provdes a charactersaton of the SRF (spectral response fncton) and also a radometrc and spatal calbraton. MODIS also has a blackbody as a prme calbraton sorce for the md and long-wave nfrared bands (3.5 µm 14.4 µm). In-flght blackbodes operatng at temperatres arond the nstrment ambent temperatre (typcally K 315 K) are commonly sed as references for nfrared nstrments as these can be made to be 45 althogh other temperatres are sed dependng on the applcaton ~ 90 ~

99 reasonably stable. Fnally MODIS, lke many other earth observaton satelltes, has an onboard dffser whch can be placed n front of the earth mager to reflect (dffsely) snlght. Ths provdes a radance-based reference, allowng top of atmosphere (TOA) radance to be determned by reference to a vale of the solar spectral rradance 46. MERIS had an addtonal pnk dffser for spectral calbraton (wth absorpton lnes sed to check the MERIS spectrometer wavelength scale) as well as the whte dffser. Some satelltes have on-board lamps as radometrc references. The most mportant thng to note wth on-board calbraton systems s that they themselves also reqre pre-flght radometrc charactersaton and are also sbject to change on transfer to orbt, and once n orbt de to the same mechansms that affect the mager tself. Some sch changes are more sgnfcant than others for example, an on-board blackbody wll sally be less lable to change drng orbt than an on-board dffser 47. One method to deal wth ths s to have a spare on-board calbraton reference or a montorng system that s sed less often than the man calbraton reference. For example, MERIS had a second whte dffser plate that was deployed every three months to montor the degradaton of the freqently sed plate 48. A second approach s to have addtonal nstrments to montor the degradaton. The MODIS satellte carres a solar dffser stablty montor an ntegratng sphere wth nne fltered detectors (for nne narrow SRFs from 400 nm to 1000 nm) monted on t that vew a dark scene, drect snlght and then the solar dffser. The reslts of sng these two systems to montor the dffser degradaton are shown n Fgre 9. a) b) Fgre 9 On-board solar dffser degradaton a) for the dfferent spectral bands of MERIS, b) for MODIS AQUA at 41 nm. Fgres taken from a) MERIS 65 th Cyclc Report [0]. And b) [1] 46 Usally obtaned from other mssons althogh the choce of solar spectrm, and how to convolve t wth the nstrment SRF s tself the sbject of some debate and reslts can be sgnfcantly changed when dfferent choces are made. See also secton Althogh care does need to be taken regardng the contact thermometers that measre ts contact temperatre and are sed to predct the effectve radaton temperatre. Changes n thermometers can occr, and ther physcal contact wth the blackbody can change, prodcng gradents and bases. 48 Ths s based on the assmpton that most changes n the dffser are de to llmnaton of the panel by the sn, and n partclar de to the ltravolet exposre and therefore a dffser sed less freqently wll be degraded mch more slowly ~ 91 ~

100 Of corse any spare calbraton reference and any montor to check that reference wll also degrade n orbt. The expectaton s that the degradaton of the spares and montors wll be sgnfcantly less than that of the prmary reference and can be consdered small (and deally neglgble) wthn the ncertantes soght by the msson. Most of the effects descrbed n ths chapter are corrected for by the nstrment scence teams. For example, the raw spectral dffser degradaton for MODIS shown n the ble crcles of Fgre 9b, leads to a correcton (the red damonds) that s appled to the pre-flght calbraton of the dffser to obtan the vales for n-flght se. In secton 8.5 the PTFE dffser calbraton wll be sed and generalsed as mch as possble to provde a smplfed example of a post-lanch ncertanty bdget. 8.3 Vcaros cal/val In addton to (or, n many cases, nstead of) on-board calbraton systems, satelltes se vcaros calbraton and valdaton (cal/val) as a means of checkng the radometrc stablty of ther on-board nstrments, as well as provdng comparsons between dfferent satelltes and therefore nderstandng nter-satellte bas. In some cases, ths vcaros calbraton can provde absolte radometrc gan calbraton coeffcents. Valdaton of satellte data prodcts s also performed comparng the hgher level satellte prodcts (e.g. sea srface temperatre, ocean color, vegetaton ndces, etc.) wth n-st measrements of these prodcts. Radometrc calbraton and valdaton of basc satellte prodcts (reflectance, radance), and the harmonsaton of satellte records to correct for bases between dfferent satelltes, are performed sng one of the followng methods: Smltaneos Nadr Overpass: Ths s where two satelltes see the same scene wthn a few (~5) mntes of each other anywhere on the globe. Brght psedo-nvarant stes: These are brght stes that are consdered extremely stable over tme (they are sally naccessble deserts, for example the centre of the Sahara). Satellte measrements are compared to one another for smlar vew and llmnaton geometres (e.g. solar zenth angle) and provde satellte-to-satellte comparson or long-term stablty testng for a sngle satellte. In ths case, care also has to be taken of potental errors de to changes n the atmospherc transmttance. Ths s dealt wth ether throgh averagng, or by applyng correctons based on external measrements of key atmospherc parameters sch as aerosols, ozone and hmdty. Natral featres sch as convectve clods, Raylegh scatterng over the ocean, snglnt etc. can provde a stable reference. It s not always predctable exactly where and when these featres are stable for observaton, bt they have well-defned optcal propertes and can be relably sed for satellte-to-satellte comparson, and, n the case of Raylegh scatterng, absolte radometrc gan and band-to-band normalsaton. ~ 9 ~

101 The Moon: lke the psedo-nvarant stes, the moon s stable over tme 49 and provdes a brght nform reference sorce for satellte-to-satellte comparson or long-term stablty testng of a sngle satellte. Instrmented reference stes: There are a small nmber of nform stes whch are constantly montored by nstrments on the grond to check both the grond reflectance and the optcal propertes of the atmosphere (aerosol optcal depth, for example). These stes can be sed for satellte-to-satellte comparson, sng the onthe-grond nstrments as references to correct for changes of the ste or the atmosphere, and also for satellte-to-grond comparson. In addton, these grond measrements can be made sng eqpment (e.g. a feld radometer) whch reles on a SI-traceable calbraton. Crrently a workng grop of CEOS-WGCV-IVOS s developng a grop of these stes nto a global network known as RADCALNET. The RADCALNET project s n a two-year prototype stage. Note that all of these methods are stll nder development and the sbject of actve research. The methods themselves are beng developed and fll ncertanty analyss s generally not yet performed! 8.4 Uncertanty analyss It s mportant to nderstand that the process of n-orbt calbraton and valdaton, whether performed by on-board references or throgh vcaros calbraton, wll always ntrodce addtonal ncertanty components of ts own. Understandng those ncertantes s, conceptally, no dfferent from performng ncertanty analyss on grond-based calbratons and the technqes ntrodced n Secton 4 of ths textbook apply eqally well to post-lanch calbraton. The man dfference s that lanch tself ntrodces a fndamental break n the traceablty chan and many more of the ncertanty components after lanch wll have to be estmated n-flght and/or smply gessed. Ths cold mean that the calbraton procedre does not rgorosly follow a SI-traceable chan. The objectve shold be to nderstand as many ncertanty contrbtors as possble, ncldng the orgn sorce for each, and to lnk these to the measrement model. These ncertanty contrbtons can then be ndvdally assessed and any that are consdered to be neglgble can be removed, so smplfyng the ncertanty analyss. 8.5 Example: PTFE dffser As an example, some prelmnary (smplfed and basc) ncertanty analyss s provded for a PTFE 50 solar dffser sed as a reference on board an earth observaton magng satellte. Ths example s smplfed and s not meant to be an exhastve stdy of the ncertantes for sch a measrement, bt an example of how to apply Secton 4 to a post-lanch staton. 49 Provdng de care s taken to correct for lnar phases, sng for example the ROLO model 50 Pressed PTFE powder s avalable nder the tradename Spectralon from Labsphere and nder the tradename OP.DI.MA from Ggahertz Optk ~ 93 ~

102 8.5.1 Use of the solar dffser n orbt The solar dffser provdes a reflectance reference n orbt and s sed to check the radance calbraton of the nstrment. The radance of the solar dffser, when llmnated by the sn, s gven by where, L ( ) SD, L ( pb, ) ( ) E ( b) ρ θ, φ, θ, φ cosθ = (7.1) r r sn SD π dsn pb s the radance of the dffser as observed by a partclar pxel, p n a spectral band b. ρ θ, φ, θr, φr s the dffser radance factor 51 for llmnaton at the nomnal solar, θ, φ. It s a rato of the dffser BRDF ( ) angle ( θ φ ) and vewng at the pxel angles ( r r) (charactersed on-grond) wth respect to the deal lambertan BRDF ( 1 π ). E ( ) sn b s the band-ntegrated solar rradance at a standard dstance (one astronomcal nt). θ s the angle from the normal axs of the dffser to the sn-to-dffser axs d sn s the actal satellte-to-sn dstance n astronomcal nts Note that ths s to some extent senstve to the vewng pxel (and/or detector, dependng on the nstrment type) as ths wll determne the vewng angles to select the approprate reflectance factor. It s also dependent on the SRF of the pxel, the nstrment band. The band-ntegrated solar rradance s gven by 5 E sn ( b) E 0 = ( ) S( b ) S( b; λ) d λ ; λ dλ λ (7.) where, E ( ) λ 0 λ s the average exo-atmospherc solar spectral rradance 53 [] at wavelength 51 Ths term s sometmes ncorrectly referred to as reflectance. Reflectance s the rato of the flx reflected n a gven drecton to the flx n the ncdent beam. Reflectance factor converts ths to a rato between the measred reflectance and that from a perfect dffser. Radance factor s both relatve to a perfect dffser and for nfntesmal angles. BRDF s defned as the fncton descrbng the change wth angle of rradaton and angle of vew, of the qotent of the radance of a srface element n the gven drecton of vew, by the rradance ncdent on the medm from the gven drecton of rradaton. For a Lambertan dffser, BRDF s nmercally eqvalent to the anglar dstrbton (wth angle of llmnaton and vew) of the radance factor dvded by π. (Note all these qanttes also vary wth wavelength.) 5 Note that f the SRF s normalsed to have nt area, the denomnator s nty. Often ths assmpton s made and the eqaton s wrtten wth the nomnator only. 53 There s consderable debate n the relevant commntes as to whch solar spectrm s to be sed and how ths ntegraton s performed. One common method s to se the solar spectrm gven by the Thller model, ref n text. ~ 94 ~

103 S( b; λ ) s the SRF of the nstrment beng calbrated for band b. More generally, t cold be also depend on the detector and/or pxel. Note that eqatons (7.1) and (7.) are calclaton eqatons to se the termnology of Secton 4 of ths textbook. The fll measrement eqatons wold take nto accont all the sorces of ncertanty. The dffser radance s sed to perform, an absolte radance responsvty calbraton (or an absolte reflectance responsvty calbraton, see secton 8.6.1) to obtan the calbraton coeffcents for radance or reflectance measrements. The radance responsvty gan coeffcent 54 s gven by where (, ) Gbp Q bp, = L (7.3) sd ( bp, ) Q bp, s the dark-corrected sgnal (dgtal cont) measred by the nstrment. Typcally ths sgnal wll be the reslt of averagng several samples (e.g samples) n order to redce the nstrment nose or other random effects. L bp s the calclated solar dffser radance, as gven n Eqaton (7.1). ( ) sd, 8.5. Step 1: Descrbng the Traceablty Chan The traceablty chan for the determnaton of the nstrment gan (Eqaton (7.3)) s gven n Fgre 30. Note the break n the traceablty chan de to lanch and post-lanch ageng effects. It s conceptally helpfl to draw ths nto the traceablty chan to ensre that the ncertantes assocated wth sch changes are consdered. In Fgre 30, there s an assmpton of some on-board dffser montorng. Ths s ntentonally vage to allow for several possbltes. However, for sbseqent dscssons n ths secton, we wll assme that ths s acheved the MERIS way,.e. throgh the se of a reference dffser that s only rarely llmnated by the sn. Ths the preflght calbraton of dffser montorng system here means the preflght BRDF calbraton of the reference dffser. 54 In practce ths may be averaged across pxels wthn a band, for example. ~ 95 ~

104 Fgre 30 Traceablty chan for the nstrment radance gan coeffcent calbraton sng the nflght dffser. Note that lanch and post-lanch ageng effects break the fll traceablty to SI Step : Wrtng down the calclaton eqatons The calclaton eqatons have been gven above as Eqatons (7.3), (7.1) and (7.). In addton t s necessary to calclate the dffser radance factor from the BRDF. The BRDF s essentally the anglar dstrbton fncton for the radance/reflectance factor dvded by π, so ths s nomnally a reqrement to read off the correct BRDF or radance factor vale for a gven nstrment vewng angle and solar llmnaton angle. In practce t may be necessary to nterpolate a vale from dscrete measred BRDF vales or ft those to a crve. Another opton s to se polynomal and hgher order models whch are tned sng expermental data lke hemreflectance and scatterng propertes of the PTFE materal. It s also necessary to have a model to accont for the dffser degradaton n orbt. We assme here that the degradaton n orbt s determned from a second dffser that s sed only rarely. Ths s the MERIS method and reslts obtaned gve the correcton crve of Fgre 4a. We assme that a correcton s appled, sch that the n-orbt radance/reflectance factor s calclated from the pre-flght radance factor sng (7.4) where ~ 96 ~

105 A s the slope of the lnear fncton ftted to the correcton crve of Fgre 9a t s the tme snce lanch n the nts of Fgre 9a Step 3: Consderng the sorces of ncertanty The am here s to consder what all the sorces of ncertanty are. Ths lst s not ntended to be exhastve or defntve for ths partclar example, bt s ndcatve of the types of effect consdered. Preflght: ρ( θ, φ, θ, φ ) r r The man dffser wll have been calbrated for BRDF drng the pre-flght calbraton campagn as well as the reference dffser and the nstrment angles. There wll be several ncertanty contrbtors assocated wth ths process: Uncertanty assocated wth the absolte BRDF charactersaton: The dffser radance/reflectance factor s calclated from the modelled dffser BRDF and the known vewng angles for the relevant pxel and the solar llmnaton angle. The absolte dffser BRDF charactersaton wll be performed by the calbraton laboratory drng the preflght calbraton process. Ths laboratory wll ether be a natonal metrology nsttte, or the measrements wll be traceable to a natonal metrology nsttte. For example, NPL s BRDF measrements are performed on the Natonal Reference Reflectometer (NRR) and a fll ncertanty bdget has been pblshed [3]. The NRR performs radance factor measrements sng an npt beam whose geometrc extent s accrately defned wth an apertre of known area, and measrements made (n both polarsatons) of the drect beam and the lght reflected from the sample. These measrements wll typcally nterpolated or ft nto a model provdng a pre-flght ncertanty of the BRDF model. The reference dffser (the one that s only occasonally sed n orbt) wll be calbrated pre-flght n a smlar manner, wth smlar ncertanty components. Uncertanty assocated to the nformty of the PTFE srface: Ths s de to the srface non-nformty of the PTFE materal that can prodce small varatons n the BRDF dependng on the poston. At nstrment level, each pxel wll be vewng dfferent postons of the dffser srface across and along the FoV. In ths case we wll also report the vale provded by the MERIS dffser charactersaton. In that case, t was fond a varablty <0.5 % across the dffser srface [4]. Uncertanty assocated wth the pxel vewng and ncdent angles. The nstrment and vewng angles for each pxel wll be defned by charactersng (or settng) the geometrc arrangement preflght. There wll be an ncertanty assocated ~ 97 ~

106 wth these angles de to the system tself or the nflence of other systems (e.g. vbraton de to an nternal gyroscope system or the dffser flatness sng tomographc mages). Ths wll translate to an ncertanty assocated wth the radance/reflectance factor that wll depend on how rapdly the BRDF changes wth angle. For a good qalty dffser, the radance/reflectance factor s lkely to change slowly wth angle and therefore small ncertantes assocated wth the angle are lkely to create neglgble ncertantes assocated wth the BRDF. For example, as mentoned before the mcro-vbratons can be assessed pre-flght provdng a Gassan angle error dstrbton of e.g. 0.1º (1σ). The ncertanty n the angle can be smply propagated by sng a Monte Carlo evalaton. Smplfed for ths example, we assme that the radance factor changes by ~0.3 % per degree and, proportonally a change of ~0.1 % n angle wold reslt n an ncertanty 0.03 % (1σ) Uncertanty assocated wth the stablty of the BRDF model. As commented n step (secton 8.5.3), the dffser BRDF s ftted to an analytcal model (or any other technqe) that s sed to estmate the radance factor at the actal angles of llmnaton and vew. The man ncertanty component wll be that de to the stablty of the model. The MERIS calbraton report [4] descrbes the root mean sqare resdal 55 (dfference from model and measred vales) as 0.3 %. In addton to ths ncertanty t has been notced that the calclated gan (Eqaton(7.3)) has some senstvty to sn azmth angle. There s no physcal process that wold create sch dependence, and t can be assmed that ths dependence s therefore an artefact of ncorrected errors n the BRDF model. Ths provdes a frther ncertanty component at ~0.5 %. Uncertanty assocated wth the spectral nterpolaton of the BRDF. The BRDF vales are measred at certan wavelengths only. At other wavelengths there s a need to nterpolate. As dscssed n Secton 6.6, nterpolaton can have the nexpected effect of redcng the ncertantes as ntermedate ponts are some form of average of the npt data. However, that assmes that the nterpolaton s approprate, a more sefl measre of the ncertanty assocated wth nterpolaton s to take some addtonal measrements (or perform the nterpolaton wth some measred vales mssng) and calclate the resdal from the nterpolaton for the extra measred vales. 55 The peak dfference s 1 %. Ths ncertanty cold be consdered a Gassan dstrbton wth a standard devaton of 0.3 % or a rectanglar dstrbton wth a half wdth of 1 %. For a rectanglar dstrbton to obtan the eqvalent standard ncertanty t s necessary to dvde by 3 (see Secton 0). Ths wold gve ~0.6 %. Whether 0.3 % or 0.6 % s sed n the ncertanty bdget depends on whether the resdal dstrbton s rectanglar or Gassan. ~ 98 ~

107 Postlanch: ρ ( θ, φ, θ, φ ) n-orbt r r The post-lanch dffser BRDF s calclated from the preflght BRDF calbraton and the known degradaton n orbt, sng Eqaton (7.4). There are the followng sorces of ncertanty for the degradaton correcton: Reference dffser stablty. The assmpton that the reference dffser s stable n orbt. In actal fact, t too wll degrade and probably at a smlar rate. The MERIS nstrment calbraton report has calclated that for the tme scale of Fgre 9a, the reference dffser had 37 mntes of exposre (compared to 370 mntes for the man dffser). Applyng the same correcton over the shorter tme perod, sggests a change of ~0. %. Ths can be consdered the ncertanty assocated wth the dffser correcton de to reference dffser stablty. Correcton model. The correcton assmes that a lnear trend relably descrbes the dffser ageng snce lanch based on pre-lanch assessments [4]. The measred data vales do not perfectly ft a lnear trend, and t has been noted n the MERIS calbraton report that the mnor flctatons from the lnear trend are correlated wth solar azmth varaton and ths sggests that t s de to resdal errors of the BRDF models (see below). As an ntal estmate, the root-mean-sqare resdal (devaton from the lnear ft for the measred vales) can be sed as the ncertanty assocated wth the lnear trend assmpton (ths s also ~0. %). It may be that for the shortest tme perods an addtonal ncertanty component s reqred. Radance of the dffser L ( pb ) SD, The radance of the dffser s calclated sng Eqaton (7.1). Jst above we have descrbed ρ θ, φ, θ, φ and ts mpact the man ncertanty sorces related to the reflectance factor ( ) r r on the estmaton of the radance of the dffser. The rest of components n ths eqaton wll also ntrodce addtonal ncertantes: d sn ; The sn-satellte dstance can be consdered determnstc (t s a fncton of the date of acqston and the earth s orbt). The ncertanty assocated wth ths can be consdered neglgble. E λ ; The sn rradance model for MERIS s based on SOLSPEC measrements ( ) 0 and the paper by Thller et al provdes an ncertanty estmate wth a clear ncertanty bdget. There s, however, consderable debate as to whch solar model to se. For TOA reflectance measrements, ths wll cancel ot (see secton 8.6.1), bt for radance t s sgnfcant. The ncertanty ntrodced here shold nclde both the ncertanty assocated wth the model (as n the paper) and the stablty of the model choce (by comparng the otpts of dfferent models to dentfy bases between dfferent models) ~ 99 ~

108 For ths example, we wll smply provde the relatve standard ncertanty reported by the Thller sn rradance model of ~.5 % S( b; λ ) ; The SRF of the band s charactersed pre-lanch. As dscssed n Secton 7 ths can change n orbt and whle there are some possble n-orbt methods to recharacterse t, they have ther own assocated ncertantes. There may also be some effects (.e. spectral nose) from any assmpton that the SRF s the same for all spatal pxels. Note that the SRF s sed wthn an ntegral n Eqaton (7.) and therefore the analyss descrbed n Secton 7 apples. The mltplyng fncton s the solar spectral rradance. Ths means that the analyss wll be most senstve to changes n the SRF where the SRF s close to one of the solar Franhofer lnes, and therefore the solar spectral rradance s rapdly changng wth wavelength. To make t very smplstc jst assme that the spectral nose across the pxels n an specfc detector and band s ±0.5 nm shft of central wavelength (note that a proper assessment wold nvolve the ntegral over the whole band range n Eqaton (7.)). The spectral response s also corrected n-flght and the stablty correcton resdal s ±0. nm (1σ) for the central pxel. Ths leads to an assocated standard ncertanty of ~0.54 nm. Fnally, the evalaton of ths ncertanty n Eqaton (7.) wold propagate to an ncertanty on the band-ntegrated solar rradance of 0. % (the nmbers are only llstratve) cosθ ; lambertan term. The same ncertanty on the ncdent angle knowledge that had an mpact on the radance factor wll have a drect mpact on the estmated dffser radance. Here the same angle ncertanty apples as n the prevos ncertanty contrbtor (standard devaton of 0.1º). For a dffser measrng at a nomnal ncdence angle θ = 60, the mpact on the cosne term wold be: cosθ = 0.3% ( ) Measred nstrment sgnal: Q bp, The measred sgnal wll be a lght cont mns a dark cont. There wll be ncertantes assocated wth: Instrment nose ths can be estmated from the standard devaton of the lght measrements. Typcally the measred sgnal wll be an average of several samples that wll redce the random nose to a neglgble level. For example f the nstrment nose at the radance calbraton L cal s 1 % for an specfc band and the measred sgnal at calbraton s averaged over 1000 samples, the nose ntrodced wll be ~0.03 %, whch s neglgble. Qantsaton nose the effect of trncatng the sgnal for the bandwdth of the nstrment commncaton. The radance measred at calbraton L cal wll be typcally mch hgher than the mnmm radance L mn measred by the nstrment (e.g. TOA ~ 100 ~

109 ocean radance levels). Wth a trncaton of e.g. 1 bts, the qantsaton nose wll be neglgble at the measred sgnal. Internal spatal stray lght ths effect wll be acconted as neglgble here snce the npt sgnal s largely nform. The spread, back-reflecton or any smlar mechansm wll contamnate the pxel neghborhood. However, the same mechansm wll apply for all the pxels n the FoV. Therefore, for each pxel the sgnal losses wll be largely compensated by the sgnal ncreases from the rest of pxels meanng that the mpact on the ncertanty s neglgble. Internal spectral stray lght the lght from other spectral wavelengths may have some effect on the sgnal wth a spectral mager system. External straylght drng the calbraton t may orgnate from reflectons n the Sn dffser assembly or other nstrment parts. It s possble that the Earth lght or other parts of the system contamnate the measrement. Ths lght wll be translated nto a bas where the frst component s proportonal to Lcal and the second one s an offset of Q bp,. Let s assme that ths have been well charactersed (e.g. sng a ray tracng model) and the total bas at L cal s n the order of 1 %. A correcton term shold be added n Eqaton (7.1), K ext-stray that acconts for ths bas. Nonetheless, the correcton tself ntrodces a resdal ncertanty of ~0. %. Note that the resdal ncertanty wll be lnked not to the measred sgnal bt to the estmated dffser radance (see Table 10). Polarsaton error PTFE materal s not a perfect scrambler. Ths, the sn-reflected sgnal n the nstrment cold have a degree of lnear polarsaton e.g. of 3 %. If the nstrment does not nclde a depolarsaton stage, a typcal vale of polarsaton senstvty cold be 5 %. Ths, the error prodced by ths contrbtor wll be of 0.15 %. Dark sgnal accracy and stablty the frst one acconts for the relablty of the dark sgnal measrement (e.g. sng a shtter) whereas the second one acconts for the standard devaton of the dark sgnal and also for the flctatons of dark sgnals taken at dfferent tmes. There may be some senstvty to the nstrment temperatre. The radance at calbraton L cal wll be mch hgher compared to the dgtal sgnal levels and any relatve ncertanty (ntrodced as a %) can be neglected. In addton, t s expectable that the same calbraton procedre ncldes e.g. a dark sgnal measrement before and after the measrement that mnmses the stablty error. ~ 101 ~

110 Uncertanty bdget From the dscsson n ths secton t s possble to wrte the rows of the ncertanty bdget and assgn ncertantes to each contrbtor. These are gven n Table 10. Note that the symbols gven are for the ncertantes assocated wth the effect. Table 10 Uncertanty bdget otlne for post-lanch radance calbraton sng a dffser Symbol Effect Dffser Radance Factor ρ( θ, φ, θ, φ ) r r Step 4: Creatng the measrement eqaton Uncertanty assocated wth ths effect pf-brdf Pre-flght dffser absolte BRDF calbraton 0.10 % BRDF-nf Unformty of the PTFE srface 0.50 % BRDF-ang Angle errors and BRDF anglar senstvty 0.05 % BRDF-model BRDF model stablty 0.50 % BRDF- λ nt Spectral nterpolaton of BRDF 0.0 % Dffser Reflectance Factor correcton ρn-orbt ( θ, φ, θr, φ r ) ref-stab Post-lanch stablty of reference dffser 0.0 % ln-corr Stablty of post-lanch correcton (lnear trend assmpton) L pb Dffser radance calclaton ( ) SD, 0.0 % d Sn-satellte dstance 0.00 % Esn Solar rradance model.50 % SRF of nstrment spectral nose 0.0 % SRF ( cos ) θ Incdent angle knowledge mpact 0.30 % ( K ext-stray ) External stray lght correcton resdal 0.0 % Measred nstrment sgnal Q bp, nose Instrment nose 0.03 % trnc Qantsaton nose 0.00 % nt-stray Internal stray lght 0.00 % pol Polarsaton error 0.15 % dark Dark sgnal accracy and stablty 0.00 % The smplest way to approach the measrement eqaton, a method smlar to that for Eqaton (4.3) n Secton 4.6, s to se an addtonal symbol for each of these ncertanty ~ 10 ~

111 components that represent the error n the gan de to ths effect. In ths case, the measred sgnal s best treated as havng addtve errors, ths 56 Q = Q Q + δ + δ + δ + δ + δ (7.5) bp, bp,,lght bp,,dark nose trnc crosstalk dark dark-stab Here all the δ terms have a nomnal (expected) vale of zero, bt an ncertanty assocated wth that vale that s gven by the eqvalent term. Each of these parameters wll have nts of [dgtal nmbers] and the assocated ncertantes shold also have nts of [dgtal nmbers]. For all other parameters, a smlar approach s taken wth a mltplcatve effect, hence, for example (, ) ( p p) Esn ( b) ρ θ, φ cosθ L pb = K K K (7.6) sn SD SRF SRF-Change ext-stray π dsn Here the K terms have a nomnal vale of nty, and an ncertanty assocated wth that gven by the eqvalent terms. Note that SRF s the ncertanty assocated wth the solar dffser radance de to the SRF. It s not the ncertanty assocated wth the SRF. Here all the terms are mltplcatve and the ncertanty s a relatve ncertanty, sally expressed n %. The parameters n the calclaton eqaton also have assocated ncertantes. In the case of ρ θ, φ, the ncertanty assocated wth ths comes from all the the dffser radance factor, ( p p) effects lsted n Table 10. The fnal eqaton s also mltplcatve, (, ) Q bp, Gbp = Kext-strayKnt-stray L (7.7) sd ( bp, ) Step 5: Determnng the senstvty coeffcents Wth the approach descrbed n the prevos secton, the eqatons gven are smple addtve (Eqaton (7.5)) or mltplcatve (Eqaton (7.6), (7.7)) expressons. The senstvty coeffcents for these eqatons are therefore smple. For Eqaton (7.5), the senstvty coeffcents for all terms are nty for ncertantes expressed n dgtal nmbers. To se ths ncertanty n Eqaton (7.7), the resltant absolte Q Q 56 Here, the frst two terms, bp,,lght bp,,dark are the nomnal tre vale and the remanng terms are those wth assocated ncertantes. It s a matter of personal preference whether to treat these as ncertantes assocated wth the lght and dark sgnals, respectvely, and therefore wrte bp, bp,,lght bp,,dark other ncertantes nblt nto ( Q bp,,lght ) and ( Q bp,,dark ) and pt all ncertantes nto δ terms. Q = Q Q as the measrement eqaton, wth the, or to treat these as terms wth no assocated ncertanty ~ 103 ~

112 ncertanty (n dgtal nmbers) mst be converted to a relatve ncertanty (n per cent). Ths s acheved by dvdng the resltant ncertanty by the measred sgnal. Ths, from (7.5) And ( bp) Q = (7.8), nose trnc crosstalk dark dark-stab ( ) ( ) Q = Q Q (7.9) rel bp, bp, bp, In eqaton (7.6) all the terms are mltplcatve. In ths case t s the relatve ncertantes that are added n qadratre. Treatng relatve ncertantes, most senstvty coeffcents are nty. The exceptons are for the cosθ sn and d sn terms. For the dstance term the senstvty coeffcent s straghtforward to derve, as n Secton 3.1, eqaton (3.11) L d SD sn L = d SD sn. (7.10) Therefore the relatve senstvty coeffcent s -. For the cosne term t s slghtly more complex, as the dfferental of cosθ s snθ and for angles arond θ = 0, ths sggests the senstvty coeffcent (and therefore the ncertanty contrbton) s neglgble. Ths s an example of where the n-blt assmpton n the Law of Propagaton of Uncertantes, of lnear fnctons, breaks down. The GUM does nclde advce on how to deal wth ths 57 bt often ths s an example of where a senstvty coeffcent calclated nmercally s the smplest and most satsfactory method for estmatng t. Ths can be done by calclatng the reslt of Eqaton (7.6) frst wth the nomnal solar angle and then agan wth a slghtly modfed verson. The propagaton of a Gassan dstrbton throgh cosθ wold derve n a typcal -shaped fncton f the lmts of the dstrbton accont for a bg or the whole part of the cosne perod. However, n ths case, the changes are so small (0.1 degrees) that ts propagaton s lnear prodcng a Gassan otpt. Note that althogh the relatve senstvty coeffcents for the K vales are straghtforward and nomnally nty, ths s becase the K vales have been defned as the error n the radance (Eqaton (7.6)) or gan (Eqaton (7.7)) de to ths effect. The senstvty of radance to the ncertanty assocated wth the SRF, or the senstvty of gan to nternal stray lght s mch more easly modelled than wrtten n an analytcal eqaton, and therefore the hard work of workng ot the senstvty coeffcents s already done n the modellng Step 6: Assgnng ncertantes Ths step nvolves provdng the nmbers n Table 10 of Secton Many of the methods for estmatng these assocated ncertantes are gven n that secton. It s mportant to be 57 See, e.g. GUM secton F..4.4, and the note to 5.1. ~ 104 ~

113 clear n the table whether the ncertantes provded are relatve (n percent) or absolte (wth the same nts as the qantty consdered). It s also mportant, partclarly wth relatve ncertantes to be clear whether ths s the relatve ncertanty assocated wth the qantty tself (e.g. dstance) or the ncertanty assocated wth the measrand de to ths effect (.e. whether the senstvty coeffcent has already been taken nto accont). Strctly, the table shold provde only relatve or only absolte ncertantes. And strctly, the table shold provde the ncertanty assocated wth each parameter n trn, wth a separate colmn provdng the senstvty coeffcent and a fnal colmn mltplyng the two together. However, as wth the K factors descrbed above, often ths s not a clear ct decson and n real, expermental, ncertanty bdgets ths often gets messy! The most mportant thng s to make t clear n the wordng. There shold defntely be a colmn descrbed as the ncertanty assocated wth the measrand de to ths effect. In some cases, ths alone s flled n. In other cases ths wll be a prodct of a senstvty coeffcent and the ncertanty assocated wth the qantty tself Step 7: Combnng and propagatng ncertantes If the table provdes a colmn labelled the ncertanty assocated wth the measrand [here gan] de to ths effect then the combnaton of ncertanty s smple t reqres that that colmn be added n qadratre. Ths does make the assmpton that there s no assocated correlaton and the second half of the law of propagaton of ncertantes s not reqred. In ths example, ths can be consdered the case. If, however, the reflectance (Eqaton (7.11), secton 8.6.1) s the prodct of nterest, then note that the scene radance L s correlated wth the band-ntegrated solar E b. Ths s becase t s calclated from the measred sgnal and gan, whch rradance ( ) sn s tself calclated from the solar dffser measrement (Eqaton (7.3)), whch depends on the band-ntegrated solar rradance. It s possble to calclate the correlaton coeffcent between these two parameters. Alternatvely, and more straghtforwardly, the eqatons shold be wrtten ot n fll and the term cancelled before ncertanty analyss s carred ot Step 8: Expandng ncertantes Here the ncertanty analyss has been straghtforward and t s very lkely that an expanson to a 95 % confdence nterval s acheved by mltplyng by k =. If any ncertanty components are determned on a Type A evalaton based on a small nmber of readngs, then ether the ncertanty shold be ncreased sng Eqaton (3.6) n Secton 3.3., or the Welch-Satterthwate Eqaton, Eqaton (4.4) shold be sed. For Type B evalatons t s also possble to consder, n the Welch-Satterthwate eqaton the ncertanty n the ncertanty, as n Secton Level-1 EO prodcts radometrc ncertanty Althogh the man objectve of the corse s abot nderstandng the man ncertantes assocated to EO nstrment and ts calbraton (both pre-flght and post-lanch), t s worth ~ 105 ~

114 to menton how these nstrment measrements are typcally dssemnated to the EO sers and the conseqences n terms of ncertanty assessment and propagaton. The dea behnd ths secton s not provde a detaled assessment of the specfc L1 grond processng related to EO prodcts. That wold not only mply the nderstandng of the absolte calbraton ncertanty as n secton 8.5 bt other grond processng correctons: DS, lnearty, nformty, temperatre, etc The objectve here s to help both nstrment desgners and EO end-sers nderstand of the mplcatons that the prodct format has n the EO ncertanty analyss and ts propagaton to hgher level prodcts. We wll descrbe here two typcal examples n EO prodct format that demonstrate the specfc constrants assocated wth the ncertanty assessment and propagaton: the radance-to-reflectance converson and the mage orthorectfcaton Radance-to-reflectance converson The PTFE dffser example n secton 8.5 has been sed to exemplfy a typcal post-lanch calbraton of EO systems. Ths type of calbraton provdes a radance gan as ndcated n Eqaton (7.3) One of the man satellte level-1 prodcts s the top-of-atmosphere reflectance. Top-ofatmosphere reflectance s calclated from the measred radance sng: ρ TOA ( ) (, ) ( ) π d L bp = sn_to_srface TOA. (7.11) cosθ sn p E b sn where d sn_to_srface represents the Sn-to-Earth srface dstance n astronomcal nts and θ p s the sn zenth angle that corresponds to the Earth srface projecton for ( ) sn each pxel. Note the smlarty between Eqaton (7.11) and Eqaton (7.1). Ths means that for the reflectance prodct, n effect the reflectance of the Earth s compared to the reflectance of the panel drectly and some of the ncertanty components descrbed above are cancelled. In partclar, the same ntegral of the prodct of the SRF of the nstrment and the defned solar rradance s sed. Ths means that reflectance prodcts are nsenstve to errors n these parameters 58. Frthermore t removes the cosne effect of dfferent solar zenth angles de to tme acqston dfferences between mages as well as the Earth-to-sn dstance changes throgh the year (translaton) and day (rotaton). 58 If the SRF has changed or s not the one assmed, then t wll cancel ot n the solar convolton ntegrals n Eqatons (7.11) and (7.1). That does not mean that changes n the SRF are nsgnfcant, however! The calclated vale ρ n eqaton (7.11) s the band-ntegrated earth top-of-atmosphere reflectance. If that prodct s sbseqently sed or compared wth other prodcts, then the band-ntegraton mst be taken nto accont. Becase that wll depend on the tre SRF and the tre grond spectral reflectance, then errors n the assmed SRF wll have a sgnfcant effect. ~ 106 ~

115 Althogh the reflectance approach corrects for the several sses as descrbed before, the ρ plb only acconts for a specfc ncdent and reflected geometry. radance factor ( ) TOA,, The exact vewng geometry wll be dfferent from pxel-to-pxel across the feld of vew, whch may be dfferent for dfferent orbts. Some of the Earth s srfaces (e.g. desert areas) can provde BRDF characterstcs relatvely close to the deal Lambertan one whch mnmses the drectonal dependency of the measred radance. However, ths s not the staton n most cases where the srface BRDF presents a speclar component and an mportant ansotropy. Several models have been (and are) sed to characterse the dfferent Earth srface BRDF. Recent mssons, e.g. POLDER/PARASOL, estmate the drectonalty of the Earth srfaceatmosphere (TOA) reflectance. The BRDF correcton s a sorce of ncertanty whch s acconted for n hgher level prodcts (e.g. srface reflectance prodcts). E.g. [5] To sm p, t s mportant to be clear abot the qantty of measrement and assmptons n the EO prodcts, snce, for example, the ncertantes assocated wth reflectance are dfferent from those assocated wth radance. These dfferences wll propagate throgh to hgher-level prodcts Image orthorectfcaton Provdng an mage at the nstrment pxel vewng mposes several dffcltes to the EO end sers. For example, at the sensor geometry each ndvdal pxel does not accont for the srface elevaton varatons or the dfference n projecton over the srface from pxels at nadr and off-nadr as shown n Fgre 31. Typcally, EO prodcts are resampled drng the grond processng (or sng an external tool) that provdes pxel mages orthorectfed to world geodetc models (e.g. WGS 84). Ths provdes a nform sze of the pxel (pxel sze provded n meters rather than mrad) whle a dgtal elevaton model (DEM) corrects (p to a certan extent) for the srface elevaton varatons. Althogh ths s a beneft to EO end sers t also mposes a dffclty when propagatng the ncertantes from the nstrment otpt geometry to the Earth geometry. ~ 107 ~

116 Fgre 31 Compared to a nadr vew (lookng straght down), a vew from the sde (30 ) sbtly dstorts the terran. Satellte data mst be careflly corrected for even small dstortons de to terran or vewng angle f scentsts want to detect changes n the landscape over tme. (NASA Earth Observatory mages by Robert Smmon, based on the USGS Natonal Elevaton Dataset.) The dffclty arses from the fact that the pxel mage does not represent anymore the focal plane geometry. That means, that dfferences between several detectors, flters etc. are not (a pror) traceable anymore. Nonetheless, the capacty to propagate the nstrment ncertanty to the resampled prodcts reles n a great extent to the radometrc resamplng algorthm. Here we make a dstncton between two typcal radometrc resamplng algorthms and ts dfferent mpact n terms of ncertanty: Cbc convolton resamplng algorthm ses a weghted average of the 16 pxels nearest to the focal cell and prodces the smoothest (or most contnos) mage compared to nearest neghbor resamplng. It provdes a great performance n terms of pxel nterpolaton and smoothness bt lmts the propagaton of the nstrment ncertantes to the prodct (no correspondence between pxels). It s also one of the most comptatonally ntensve algorthms. The propagaton of the radometrc ncertanty from the nstrment mage to the resampled mage becomes especally complcated for systems whch nclde several detectors n the focal plane. In that case, t complcates the determnaton of the detector nose or the spectral response not only at pxel level bt also at detector level. ~ 108 ~

117 For example, ths s the case of Landsat-8 Level 1 T- Terran Corrected prodcts whch have been resampled sng a cbc nterpolaton algorthm. The resampled pxels have no drect eqvalent pxel n the nstrment mage. In addton, each pxel cannot be, a pror 59, drectly assocated to any of the 14 detector modles that the OLI nstrment ncldes. The Fgre 3 corresponds to the per-modle average spectral response of the CA band Landsat 8 OLI nstrment. From the fgre, t s possble to apprecate the dfference n average spectral response between dfferent modles de to dfferent flter wafers. Fgre 3 Landsat-8 OLI Spectral Response normalsed and averaged for each one of the 14 modles of the Coastal Aerosol band [6]. For the resampled mage Level 1 T prodct, the per-modle spectral response wll not be possble to assocate (a pror) to the mage bt the average spectral response for the whole FoV (.e. the 14 detectors). Ths, an error wold be ntrodced compared to the non-resampled mage whch wold accont for the dfference n the convolton 59 It s possble to mplement mechansms that defne the footprnt of the detectors n the resampled mage wth some lmtatons. ~ 109 ~

118 from one modle to another (smlarly to Eqaton (7.)) and wll depend on the type of scene measred 60. Nearest Neghbor resamplng algorthm works by matchng a pxel from the orgnal mage to ts correspondng poston n the reszed mage. If no correspondng pxel s avalable, the pxel nearest s sed nstead. Ths type of performance has been sed n several mssons lke ENVISAT/MERIS. It does not prodce the smoothest mage bt redces the processng [7] by more than 10 tmes compared to the cbc convolton nterpolaton and t s a reversble process. It means that each resampled pxel corresponds to a specfc pxel of the nstrment focal plane. The Fgre 33 shows an example of that pxel correspondence for the ENVISAT/MERIS msson. Fgre 33 MERIS correspondency between the resampled pxels and the focal plane pxels. Re-blds deal swath from actal MERIS FOV: slghtly msalgned plane + nter-camera dsperson [Ldovc Borg, MERIS Level 1b processng, MERIS US Workshop, 14 Jly 008] 60 Note that the level of error for ths case has not been calclated. The bas ntrodced cold be partly compensated or cold have lmted mpact on the radometrc ncertanty bdget. However, ths dstncton at TOA level wold be of extremely beneft for the ncertanty propagaton to hgher level terms ~ 110 ~

119 9 Conclsons Ths corse textbook has provded an ntrodcton to ncertanty analyss for earth observaton nstrment calbraton. It has provded a sggested step-by-step methodcal approach to ncertanty analyss and gven some examples of how to apply ths n real earth observaton calbratons. One of the frst barrers many scentsts and engneers have to ncertanty analyss s that t seems theoretcal and complcated. Or am n ths corse has been to break down that barrer and present a practcal and pragmatc approach to ncertanty analyss. Once scentsts and engneers begn to apply these tools, they often reach a second barrer the realsaton of jst how complex the expermental systems are and jst how many sorces of ncertanty they are 61. Once they start to prodce ncertanty bdgets, they also start to see the sbtle correlatons for example that f both the detector responsvty and the flter transmttance are temperatre senstve, then there s a correlaton between detector responsvty and flter transmttance. Ths ncreased nderstandng can lead scentsts and engneers to become overwhelmed wth the problem and therefore pt ncertanty analyss off to be tackled another day. Or recommendaton s that yo consder yor frst ncertanty bdget a smplstc one. Start wth the concepts that are most crtcal. It s lkely that yo have an nttve nderstandng of where yor domnant ncertanty components wll come from. Later ths ntal bdget can be refned and agmented. Later t wll be possble to replace what are gesses now wth formal ncertanty estmates based on expermental testng or modellng. It s Emma s experence that ntal ncertanty bdgets based on gesswork are pretty close to the formal ncertanty bdget that comes after a year or more of rgoros analyss. It s also her experence that ncertanty bdgets tend to ncrease over tme, as more nknown nknowns are recognsed and nclded. Don t let ths pt yo off! 61 Ths realsaton s, of corse, also behnd the frst barrer, althogh at that stage t s harder to be explct abot what the concern s. Once the frst three steps sggested n Secton 4 are completed, thogh, t can be qte dantng to thnk abot all the effects that are now lsted. ~ 111 ~

120 Appendx A Makng and sng covarance matrces When t s possble to splt the ncertanty bdget nto systematc effects and random effects, t s not necessary to se the fll form of the law of propagaton of ncertanty (Eqaton (.1)). In general, as dscssed n Secton 3.5, the systematc and random effects can separately be explctly descrbed n an error-model form of the measrement eqaton. In other words as the correlaton s blt nto the error model, the terms wthn that error model are entrely ncorrelated wth each other. Ths approach s often the smplest, partclarly when data analyss s carred ot sng a spreadsheet program. There are, however, occasons when sch an analyss s nsffcent. Sometmes correlaton cannot be descrbed by smply separatng random and systematc components. Sch separaton s napplcable whenever mathematcal correlaton has been ntrodced, for example throgh nterpolaton, averagng, smoothng, bandwdth correcton and smlar processes that combne dfferent data ponts (e.g. from dfferent wavelengths, or from dfferent radance levels). In these statons a covarance matrx provdes the straghtforward means to apply the fll form of the law of propagaton of ncertanty. The law of propagaton of ncertanty (eqaton (.1)) s wrtten n matrx form as (10.1) where (10.) s a colmn vector of senstvty coeffcents. ( represents transpose,.e that vector s wrtten as a colmn, rather than as wrtten here, for space reasons, as a row) andu x s the covarance matrx. Ths form s gven and sed n the Spplement to the GUM 6 as eqaton (3) on page 15. Once a covarance matrx has been formed, Expresson (10.1) s smple to mplement n a programmng langage, partclarly a matrx-based langage sch as MATLAB. Ths formlaton can handle complcated analyss problems and partally correlated data. A.1 How to create a covarance matrx A covarance matrx s a sqare matrx that descrbes the covarance of the measred vales, the x. Each row and each colmn represents a dfferent x. The dagonal elements are the x, x. Note that varance, ( ) x, the non-dagonal elements represent the covarance, ( j) 6 Freely downloadable at ~ 11 ~

121 the covarance matrx s generally symmetrcal as ( x, xj) ( xj, x) =. If two qanttes have no assocated covarance, (.e. they are not correlated), then the covarance term s 0. As an example, consder the case of data that wll sbseqently be ntegrated spectrally, the covarance matrx we need to develop s that showng the covarance assocated wth (e.g.) spectral rradance vales determned at dfferent wavelengths. Ths example s sed becase t s often n spectral problems that covarance matrces are needed. Each row (or colmn) of the covarance matrx wll ths represent a dfferent measrement wavelength. The dagonal terms gve the varance ( E ) and the off-dagonals the E, E between the rradance at the row wavelength and the rradance at the covarance ( j) colmn wavelength. The covarance matrx takes the form U E 1 n ( E1) ( E1, E) ( E1, En ) (, ) ( ) (, ) 1 = E E1 E E En n ( En, E1) ( En, E) ( En) (10.3) The vales shown n red smply specfy the row and colmn nmbers and wold not normally be ndcated. ( E ) on the dagonal represents the varance: the sqare of the standard ncertanty assocated wth the spectral rradance vale at λ. Ths varance s the sqare of the combned standard ncertanty, obtaned by combnng n qadratre the standard ncertantes assocated wth all effects, whether systematc, random or mxed. The off-dagonal terms represent the covarance assocated wth the measred vales at two dfferent wavelengths, as explaned, throgh examples, below. Radometrc ncertantes are sally expressed as relatve ncertantes (fractonally or n percent) rather than absolte ncertantes (wth the same nts as the measrand). The covarance matrx reqres absolte varances and covarances. Ths a covarance matrx for spectral rradance, whch has nts W m nm 1, wll have terms wth nts 63 (W m nm 1 ). The dagonal terms can be calclated as the sqare of the prodct of the relatve standard ncertanty and the spectral rradance vale at that wavelength. In Secton 3.5 we ntrodced an error model where the rradance (here at one wavelength, λ ) can be expressed as a tre rradance wth nknown random and systematc errors (draws from the probablty dstrbton descrbed by the ncertanty assocated wth random and systematc effects, respectvely) E = E + R + S Repeat of (3.8) T 63 Ths non-standard notaton for the nts s to ad nderstandng ~ 113 ~

122 Wth ths model, the varances (dagonal terms n the covarance matrx) are gven by The off-dagonal terms, wrtten ( E, E j) ( ( λ) ) = ( ) + ( ) ( λ) E S R E vale determned at λ and that determned at. (10.4), gve the covarance between the spectral rradance λ j. Ths covarance wll arse only from those effects that are common to both measred vales: the systematc effects. Ths for the error model n (3.8), only the term ( S ) s nclded. The off-dagonal covarance vales for ths error model are ( ( λ) ( λj) ) ( ) ( λ) ( λj) E, E = S E E. (10.5) Therefore the covarance matrx, expresson (10.3), takes the (symmetrc) form U E 1 n ( ) + ( ) ( ) ( ) 1 S R1 E1 S E1E S E1E n = ( S) EE 1 ( S) ( R) E ( S) EE + n. n ( S) EnE1 ( S) EnE ( S) + ( Rn) En (10.6) To create ths matrx, t s necessary to examne each row of the ncertanty bdget and consder whether that row corresponds to a systematc or random effect wth wavelength. The ncertanty assocated wth systematc effects shold be combned n qadratre to create a sngle ncertanty, whch becomes ( S ). Smlarly, on a wavelength-by-wavelength bass, the ncertanty assocated wth random effects shold be combned to obtan a sngle standard ncertanty, whch becomes ( R ). Sometmes a more detaled error-model s needed. If there are both mltplcatve and addtve effects n an error model, the error model that replaces (3.8) s ( λ ) ( λ )( )( ) E = ET 1+ S 1 + R + s + r. (10.7) The mltplcatve effect S wll nclde terms sch as dstance effects, algnment effects, etc. The mltplcatve effect R wll nclde terms sch as lght-sgnal nose, and rapdly varyng electrcal crrent stablty, temperatre senstvty, etc. The addtve term s wll be a constant offset at all wavelengths. Ths term may correspond to a common dark readng sbtracted at all wavelengths, stray lght, etc. The addtve term r corresponds to a random offset: the nose n the dark sgnal, varatons n stray lght, etc. The covarance matrx formed from the error model (10.7) has for (th row, jth colmn) U E, j ( ) ( ) ( ) ( ) E S R + + s + r ( = j) = EEj ( S) + ( s ) ( j). (10.8) ~ 114 ~

123 Note that the ncertanty assocated wth addtve effects wll generally be descrbed as an absolte ncertanty and does not need to be mltpled by the spectral rradance vale. It s possble to extend ths concept frther to accont for covarance between the measred vales of a test lamp and those of a reference lamp. ~ 115 ~

124 Appendx B Appendx: Monte Carlo Analyss Ths appendx provdes a recpe on how to mplement a Monte Carlo smlaton and ses code examples gven for Matlab. What yo need before yo start: 1. A model of the effect yo are tryng to analyse, takng a nmber of parameters and prodcng an otpt. Ths s essentally a bt of a compter code, deally a fncton that encapslates the whole model wth npt parameters and accordng otpt.. An ncertanty estmate for each parameter nclded n the ncertanty modellng. 3. A probablty dstrbton fncton (PDF) for each parameter. Most commonly these wold be a Gassan dstrbton, bt t may depend on the process that creates the parameter n the frst place. For example, ncertantes de to samplng of a contnos varable by a dgtser wll have a PDF n the form of a top hat (rectanglar dstrbton). The actal process of the Monte Carlo Smlaton s llstrated n Fgre 34 and comprses the followng steps: 1. Generate N random vales for each parameter as samples from the correct PDF, mean vale and standard devaton. These are the realsatons of the npt parameter.. Enter a loop that wll be terated N tmes. For each teraton, the -th realsaton s chosen as an npt parameter, the model s parametersed wth the -th vale and the otpt stored. 3. The ncertanty of the model s estmated from the standard devaton of the model otpt. Fgre 34: Monte Carlo Smlaton processes Table 11 ntrodces a Matlab code for a smple example of a Monte Carlo Smlaton: 1. Realsatons of an npt parameter wth a mean = and a standard devaton,.e. ncertanty, of 0. are calclated by employng the Matlab random nmber generator, creatng 1000 random vales (Fgre 35). The relatve ncertanty s ths 0./*100 = 10 %. ~ 116 ~

125 . These 1000 random vales representng possble vales the parameter cold assme are fed nto the model n a loop. The model s n ths case the sqared logarthm of the npt parameter The otpt of the model (Fgre 36) s sed to estmate the model ncertanty by comptng the standard devaton of the otpt. In the demonstrated rn the ncertanty of the model s wth a mean vale of The relatve ncertanty s ths 0.137/0.48*100 = 8 %. Table 11: Matlab code for a smple Monte Carlo Smlaton m=; % defne mean vale of PDF sgma=0.; % defne standard devaton of PDF parameter_vales = (m + sgma.*randn(1000, 1)); % generate realsatons of the parameter % plot as hstogram fgre hst(parameter_vales, 5) % plot vector of the random vales (parameter realsatons) fgre plot(parameter_vales, 'LneWdth',lnewdth) % smple model to draw from the generated parameter vales otpt = zeros(sze(parameter_vales)); % allocate otpt vector % loop over all realsatons and calclate the model otpt for =1:length(parameter_vales) parameter = parameter_vales(); otpt() = (log(parameter))^; end % plot otpt as vector fgre plot(otpt, 'LneWdth',lnewdth) % plot otpt as hstogram fgre hst(otpt, 5) % get statstcs of the otpt mean_otpt = mean(otpt) stddev otpt = std(otpt) 64 Note that the for loop s sed here for edcatonal prposes only. Ths smple model cold of corse be compted n one vector operaton by otpt = (log(parameter_vales)).^. Ths wll of corse not always be possble for more complex models, n whch case a loop wll be reqred. ~ 117 ~

126 Fgre 35: Model npt realsatons as hstogram (left) and vale vector plot (rght) Fgre 36: Model otpt as hstogram (left) and vale vector plot (rght) ~ 118 ~