1 Measurement and Instrumentation, Data Analysis
2 Error in Scientific Measurement means the Inevitable Uncertainty that attends all measurements -Fritschen and Gay Uncertainties are ubiquitous and therefore no reflection on the usefulness of the measurement or the competence of the measurer If we are to rationally use a measurement the uncertainties must be known quantitatively
3 Precision vs Accuracy
4
5
6
7 Measurement Systems INSTRUMENT: a device that contains at least a sensor, a signal conditioning device and a data display SENSOR: interacts with variable to be measured (measurand) and generates an output signal proportional to that variable TRANSDUCER: a device that converts energy from one form to another (an instrument may include several + the primary transducer (sensor)
8 Sources of Error Static: measured when input is held steady and after calibration applied eg. random electrical noise or effects of unwanted inputs such as temperature Dynamic: due to changing inputs eg. time lag Drift: physical changes in sensor over time Exposure: imperfect coupling between sensor and measurand eg. temperature: radiation, conduction, dead air around sensor..
9 Instrument Platforms
10 Data-sets and data analysis. Keep it simple - UCAR Digital Image Library
11 A classical research approach. Creative Part Hands-on Documentation Design Hypothesis, motivation, possibilities Experiment design Analysis plan Information Project Proposal Experiment Data Lab- and field measurements Experiment documentation Analysis Conclusions, hypothesis verification / falsification Data analysis Analysis documentation Report
12 Documentation : Meta-data Document your ideas and plans! proposal. Document your instrumentation (specifications, manufactures, serial numbers). Document your sampling strategy (how, where and when...) Document your data files (parameter, units, time-zone, location) Document your analysis (data filtering, data selection criteria, statistical methods) log- or field note book Document and prove your conclusions! report
13 How will your data look like? - Data format Boolean data Classification Continuous data data dimension Yes Yes No No No No No No No No Yes data dimension Cold Cold Cool Cool Warm Warm Hot Warm Cool Cool Cold data dimension -5.24-3.20 2.32 9.63 14.15 17.83 27.08 16.34 10.22 4.43-1.32
14 Field data in physical geography. Any natural environment is a complex, multivariate web of interacting variables. We can never measure continuously everything, everywhere. We sample selected data with an appropriate strategy.
15 Dimensions. Strictly speaking, we sample any physical, chemical or biological variable! in a four dimensional setting, i.e. as a function of time t and space x (x, y, z). Luckily, quite often we are not interested in all four dimensions. Likely, we focus on a single or two dimensions due to logistical reasons or because your study object allows this. You might assume that variability in one dimension is much smaller than in another one (homogeneity, stationarity critearia).! In your project you will implicitly include certain dimensions / exclude others.
16 Dimensions. Basic dimension Data-set Resolution Examples of 1-D data Time Space Time series Horizontal profile Vertical profile Temporal resolution Spatial resolution Spatial resolution (Frequency)* Spectrum Spectral resolution One day of 10 min temperature measurements at a climate station One year of hourly discharge measurements from a stream Vegetation classification along a traverse. A horizontal profile of snow water content along a line. A tethered balloon run measuring wind with height. Temperature change in soil with depth. A histogram of different grain sizes in sediments Irradiance in different wavelengths *This is strictly speaking not a basic dimension, but a transformation of time or space
17 Examples of data sets - time series. time dimension Example: Carbon dioxide in a forest as a function of time of a day
18 Examples of data sets - horizontal profiles. horizontal dimension horizontal dimension Example: Horizontal transect through Vegetation Example: Horizontal transect showing air temperature Ecosystems of BC / T.R. Oke (1987): 'Boundary Layer Climates' 2 nd Edition.
19 Examples of data sets - vertical profiles. vertical dimension vertical dimension Univ. Stuttgart / T.R. Oke (1987): 'Boundary Layer Climates' 2 nd Edition.
20 Examples of two dimensional data sets. space-space (map) Example: land use time-space Example: temperatures in a lake as a function of time of year and water depth horizontal dimension vertical dimension horizontal dimension yearly course (time) time dimension
21 Examples of two dimensional data sets. time-space Example: temperatures in the air of a forest as function of time over one hour time dimension vertical dimension Christen et al. (2001),
22 Resolution. Temporal and spatial resolution: How many data-points per unit of a dimension? Temporal resolution and spatial resolution, i.e 1 measurement a day vs. 1440 measurements a day, or 1 measurement per km vs. 1000 measurements per km. Data depth: How accurately can we distinguish between different physical values, i.e. 0.02 vs. 0.0214523. Illustration: Wikipedia
23 Integration and interpolation. Integration refers to the process of combining or accumulating - or more generally to methods of upscaling - data from an existing set of measured data points. Interpolation refers to the process of splitting down or fill-in data to constructing new data points - or generally to methods of downscaling - an existing set of measured data points. Both can be done in time and space domains, and there are various methods.
24 Gridded vs. irregular data regular irregular North (space) North (space). Data Points East (space) Voronoi tessellation East (space)
25 Example of a regular grid in vegetation studies. Photo: http://www.marine.gov/
26 Your data-set? Choose a physical parameter or a classification of interest in your potential project: Data format? Dimensions? Resolution? Regular or irregular? Assumptions?
27 Data processing. Correcting sensors with data from lab calibrations or field intercomparisons (mainly climatology). Plausibility checks - define criteria for errors, experiment disturbances, etc. Flag data - remove data that fulfill the above criteria (never delete data forever, just flag it - and backup raw data!). Integrate or interpolate data - only if your data are not at the scale required, or if you have to compare two data sets with different resolution. Select data for further analysis if you have made assumptions to fulfill certain criteria.
28 Intercompare your sensors. If you are interested in spatial or temporal difference, and you use multiple sensors at different locations in space or in different time slots of your experiment, you have to ensure that these sensors are comparable. Pre experimental Lab- or field intercomparison Field intercomparison Field measurements Post-experimental Lab- or field intercomparison Some sensors need recalibration during field experiments.
29 Check your data - potential approaches. Global criteria (minimum, maximum,...). Local criteria (rate of change,...). Statistical criteria. CO2-concentration Standard deviation Manual data flagging.
30 Analysis tools. Describe data distribution - statistical probability of occurrence, histograms, statistical moments,... Find events - peak detection, integration,... Find and quantify correlations (same variable at different locations, same variable at compared different times, between two variables, correlation between model and measured values) - correlation, regressions, curve-fitting, statistical tests Find groups and dominating dimensions - Clustering, principal component analysis, Find process dominating scales - Spectral analysis finds process dominating time and length scales, wavelets.
31 Data analysis tools. Method / System Advantage Limitations Example of a system Manual Analysis very simple, fast up to a few 10s of datapoints, no large data-sets, no modelling Calculator, paper, pen... Spread-sheet software simple analysis and graph tools Limited # of data points, limited statistics, modelling, and automation & slow. Microsoft Excel GIS system complex spatial analysis and modelling Expert knowledge. Expensive. Workstation with ArcGIS Statistical software and programming languages complex and fast time series analysis, automation, modelling Programming skills. Expensive. Workstation with Matlab, R, IDL...
32 Your data-analysis? Think about your potential project: Correction, calibrations? Data checks? Analysis concept? Software needs? Hardware needs? again: Keep it simple!