Luca Fiorani, Bertrand Calpini, Laurent Jaquet, Hubert Van den Bergh, and Eric Durieux

Correction scheme for experimental biases in differential absorption lidar tropospheric ozone measurements based on the analysis of shot per shot data samples Luca Fiorani, Bertrand Calpini, Laurent Jaquet, Hubert Van den Bergh, and Eric Durieux From lidar signals detected with a shot per shot differential absorption lidar instrument tuned for tropospheric ozone measurements and recording each individual return, we reconstruct histograms of their sampled values for each channel of our digitizers. The analysis of their shape permits the correction of our measurements for experimental biases. In particular, a negative correlation is found between the skew of the histograms and the intensity of the backscattered light. The skew comes from a tail at high sampled values, interpreted as due to a raise of the relative contribution to the signal of a signal-induced noise when this intensity diminishes. By fitting a Gaussian function to the histograms without considering their tails, we calculate average signals unbiased by the corresponding noise. This approach allows us to increase the range of our ozone profiles, up to as much as double it in some cases. 1997 Optical Society of America Key words: Lidar, ozone, troposphere, atmospheric fluctuations, signal-induced bias. 1. Introduction Lidar systems have been widely used to probe many parameters of the chemical and physical dynamics of the atmosphere by following different approaches. 1 Our instrument is designed for tropospheric ozone profiling so that the backscattered intensity from laser light pulses transmitted to the atmosphere is recorded after direct detection. In such measurements a few thousand lidar signals are typically averaged to obtain a signal-to-noise ratio SNR sufficient to reconstruct range-resolved concentrations with a spatial resolution of the order of 10 100 m. 2 This averaging is usually achieved from a hardware summation of the signals successively produced by the repetition of laser triggering. The information contained in the signal fluctuation from one laser shot to the following is therefore not recorded. 3 7 The authors are with the Shot per Shot Lidar Group, Laboratory for Air and Soil Pollution, Ecole Polytechnique Federal Lausanne, CH-1015 Lausanne, Switzerland. Received 2 July 1996; revised manuscript received 23 December 1996. 0003-6935 97 276857-07$10.00 0 1997 Optical Society of America Recently, it was shown to be usable for the measurement of ozone fluxes. 8 In this paper we discuss how we correct our measurements for experimental biases by taking advantage of the capability of our system to record and store the lidar signal for each laser pulse on a shot per shot basis. Hardware averaging schemes can be used to calculate the mean value of each sampling channel of an analog-to-digital converter ADC by signal summation. By doing so, one can assume that the histograms of the ADC values obtained for all the lidar signals recorded to retrieve one profile follow, at least approximately, a distribution whose mean can be estimated by an average without biasing the measurement. Following our shot per shot methodology, this assumption is replaced by an actual histogram reconstruction so that we can question its validity. Typically we observe that it is usually true for regions of high SNR when it is possible to fit a Gaussian function to the histograms. Then we use Gaussian estimators to calculate the mean and standard deviation of our signals as their average and root-mean-square deviation, respectively. Some experimental factors, such as the energy fluctuation of our laser sources, have been shown to affect the latter, and we could measure the efficiency of our energy normalization 20 September 1997 Vol. 36, No. 27 APPLIED OPTICS 6857

scheme by comparing its value before and after normalization of the signals. However, the histograms have been found not to present a Gaussian shape at long range when the SNR is lowered. In our interpretation this is due mainly to a signal-induced noise whose relative contribution to the lidar signal increases as the number of detected photons diminishes. The photomultiplier tubes PMT s used for light conversion in lidar experiments are known to produce a signal-induced bias caused by an afterpulse effect. 9 11 The lidar signal dynamic is such that the PMT s first produce a large signal corresponding to the short range of the measurement. When the light signal becomes weaker for the long range, some electrons are produced by the tube itself because of its prior excitation in addition to those generated by the photocathode. The importance of this effect at long range varies from shot to shot and is responsible for the distortion of reconstructed histograms in the form of a tail for high ADC values. The effect becomes more important as the range increases. The calculation of a mean signal from the fit of a Gaussian function to those distributions, in replacement of the usual averaging by summation, makes it possible to get rid of this effect and consequently to increase the range of the measurements. This approach has already been followed elsewhere for random saturation of the ADC s to remove the corresponding peak on the histograms from the determination of the mean signals. 12 2. Experimental System In a differential absorption lidar DIAL 13 two laser light pulses are emitted toward the atmosphere at two different wavelengths. This light is far more absorbed at the on wavelength by the probed species, the ozone molecule in our case, whose concentration is reconstructed from the analysis of this differential absorption. The photons backscattered by aerosols and molecules in the air are collected by a telescope and conducted to a PMT. The latter produces an analog signal that is digitized by an ADC and transferred to a processing unit. The optics and the electronics of our system have been detailed elsewhere. 12,14 The light generation is based on two frequency-doubled excimer-pumped dye lasers for the emission of on and off pulses at 272.2 nm 1.1 mj and 291.65 nm 0.9 mj, respectively. For the measurements discussed below, 3000 5000 laser shots were fired at a 49-Hz repetition rate. A plate is mounted between the laser output and the light emission optics to direct 1% of the shot toward a five-stage PMT whose output is digitized by a charge-integrating ADC and is used for the energy normalization of the lidar signals on a shot per shot basis. When entering the atmosphere, the beam diameter is 100 mm and the divergence is 0.05 mrad. The axes for the light emission and collection are separated by 430 mm, the latter being carried out by a 600-mm Cassegrain telescope. An optical fiber is used to conduct the light to a spectrograph, and the Fig. 1. Temporal evolution of the lidar signal at the on wavelength, averaged between 570 and 1485 m. light conversion is achieved by a single 12-dynode PMT at both wavelengths. An interference filter has been placed after the spectrograph on the optical path of the off wavelength to ensure that both on and off signals have approximately the same intensity. The shot per shot sampling, recording, and treatment of the signals is ensured by a front-end electronics embedded into a Versa Module Eurocard (VME) architecture. Samplers with a 12-bit vertical resolution and a 10-MHz sampling rate corresponding to a 15-m spatial resolution have been used for digitization of the signals. The whole system was assembled in a trailer and engaged in the Mediterranean Campaign of Photochemical Tracers (MEDCAPHOT) synchronized field campaign, in which air pollution was measured over the great Athens area during September 1994. 15 3. Histogram Reconstruction The temporal evolution of our lidar signal at the on wavelength, averaged over a range from 570 to 1485 m, is shown in Fig. 1 for a typical profile. The signal intensity is seen to decrease over the first laser shots, following an evolution comparable with that of the excimer laser output energy. A histogram of this energy, expressed in ADC counts, is presented in Fig. 2. If the signals are normalized by the emitted Fig. 2. Histogram of the ADC counts for energy monitoring. 6858 APPLIED OPTICS Vol. 36, No. 27 20 September 1997

Fig. 4. Evolution of histogram skew for both wavelength ranges. Only one to 12 sampling channels has been represented. clearly visible. The lidar signals have not been normalized by the output energy whose histogram fits well to a Gaussian function as in the following, if not otherwise stated. A Gaussian function with mean and standard deviation reconstructed from the data with Gaussian estimators has been superimposed onto each histogram. At a short distance, in regions of high SNR, the agreement is good. However, agreement is not mutual from a far distance that varies depending on the experimental conditions. This distance is always farther for the off wavelength because the on radiation is attenuated by ozone absorption. The skew s of a probability distribution function F p of a variable p is defined as Fig. 3. Histograms of the ADC counts for the lidar signal recorded at three ranges. A Gaussian curve with mean and standard deviation parameters set equal to those calculated from the data by using Gaussian estimators is superimposed on each histogram. energy, the time evolution of the corrected signals does not start with values systematically above the mean any more. They fluctuate around a constant level and their standard deviation is reduced by 15%. From our shot per shot data sample, histograms have been reconstructed for each sampling channel of our ADC converters. A mean value and a standard deviation have been calculated from them with average and root-mean-square deviation Gaussian estimators to reconstruct ozone profiles. 16 This was achieved in a range from 570 to 1485 m by taking advantage of the approximate Gaussian shape of the histograms within that interval. However, significant deviations from a Gaussian behavior occur as illustrated in Fig. 3, in which histograms of the ADC counts for 2990 laser shots are shown for three sampling channels at the on wavelength. The ADC counts have been grouped in bins of 80, 6, and 1 value s for the 570-, 1470-, and 2370-m ranges, respectively, in order to make their overall shape more with s p p p 3 F p dp p p 2 F p dp 3 2 pf p dp. The degree to which a histogram is skewed gives a quantitative indication of its deviation from a symmetric distribution. Negative and positive skews can indicate a tail below and above the mean, respectively. A typical measurement of this variable is shown in Fig. 4 for the two wavelengths and the complete range in which a lidar signal is still measurable under our conditions. A positive skew, increasing with the distance, is found for the on wavelength over the whole range. The same evolution is found at the off wavelength, except that the skew is found to be slightly negative for the first bin. 20 September 1997 Vol. 36, No. 27 APPLIED OPTICS 6859

4. Calculation of Corrected Mean Values and Standard Deviations The fluctuations of the atmosphere, 17 the generation of photoelectrons by the PMT, 18 the transmitted energy variation when not compensated for by a normalization, and the signal-induced noise 11 are the main sources of variation of the lidar signals in our experimental conditions. The physical processes that underly these effects are therefore dominant to originate the shape and the width of the histograms, although they can also be affected by other causes, such as ADC saturation. 12 The numerical modeling of these different processes to simulate accurately their effect on the observed distributions is complex and difficult to implement as part of systematic data processing. Therefore in this research we have chosen to follow an empirical approach to calculate mean values and standard deviations, taking into account the shape of the observed histograms. As the histogram skew increases with the distance, even if the lidar signals are normalized by the transmitted energy, it is not due to a drift of the laser energy output. Furthermore this increase has been found to be linearly correlated to the signal intensity and the skew is approximately the same at the on and the off wavelengths for equal mean ADC counts. Within our data set this correlation was not found to depend strongly on atmospheric conditions. It was retrieved for measurements limited to the free troposphere and or including the boundary layer. Moreover, the dependency of the skew on the signal intensity is continuous; no breaking point is apparent, which would make it possible to explain because of a change in statistics at low signal. Therefore we have interpreted the observed correlation as being caused by a larger contribution of the signal-induced noise at low SNR. The entry of the emitted light beam within the field of view of the telescope causes a brusque exposure of the PMT to a large amount of backscattered light. This is known to generate an afterpulse effect that is responsible for artificial enhancement of the lidar signal at greater distances. 9 11 The intensity of this effect can vary from one laser shot to another. Consequently at a given distance the recorded signal is more or less biased, and this increases the probability of obtaining larger ADC counts than expected. By opposition the light backscattering can be so intense at short distances for the off wavelength that the photoelectron production of the PMT is no longer proportional to the number of received photons. In such cases the frequency of low ADC counts increases and the skew of the histogram becomes negative. It is visible in Fig. 3 that the skew of the histograms is due mainly to a tail at high ADC counts that Fig. 5. Comparison between the fitted Gaussian curve and the curve calculated from the application of Gaussian estimators to the data for an extreme case of large skew at the on wavelength. is not present in the region of high SNR. Moreover, it also appears, as mentioned above, that their shape is compatible with a Gaussian function in that region. As our goal was not to introduce a complete modeling of the physical processes at the origin of this shape in our data processing, each histogram has been interpreted as combining a Gaussian-like signal and a noise with the form of a tail for high ADC counts. By fitting a Gaussian function to the raising edge and to the upper half of the falling edge of the histograms, mean values and standard deviations independent from the noisy tail have been calculated. An example of such a fit is shown in Fig. 5, where the on signal at a far range has been represented because it corresponds to the most extreme degree of skew. This procedure is based on the assumption, validated for high SNR, that the deviation of the actual distribution from a Gaussian distribution is small, so that the accuracy of the mean signal reconstructed after removal of the noisy tail is improved for low SNR by more than the difference in mean values owing to this deviation. The difference between the mean values we obtained by using a classical averaging or by this Gaussian fit increases with the distance, as shown in Fig. 6. This range evolution varies with the conditions of measurements and consequently from profile to profile. However, there is good correlation between the skew of the histograms and the increase in SNR obtained by the fitting procedure instead of calculating and from Gaussian estimators. This is Fig. 6. Effect of the correction scheme on the histogram mean value at both wavelengths for one to 12 sampling channels. 6860 APPLIED OPTICS Vol. 36, No. 27 20 September 1997

Fig. 7. Evolution with the histogram skew of the gain in SNR obtained when we applied the correction scheme. Symbols mark different experimental conditions. illustrated in Fig. 7, where the ratio of the relative error calculated with Gaussian estimators to the relative error calculated with and coming directly from the fit is shown to increase with skew for measurements taken in different conditions at both wavelengths. 5. Application If no correction scheme is applied, the mean ADC count is overestimated at both wavelengths. This effect increases with the distance and is more important for the on wavelength. Therefore it causes a decrease of the reconstructed ozone concentration because at a given distance the light pulse at the on wavelength appears to be less attenuated. The actual measurements follow this trend, as illustrated in Fig. 8, where a lidar profile is compared with simultaneous airborne measurements for 12 September at 1:00 p.m. The distance between the atmospheric column probed by the airplane and our lidar system was of the order of a few kilometers. The measurement conditions were comparable because it was a Fig. 8. Comparison of corrected and uncorrected ozone profiles to simultaneous airplane data that have a 57.4-ppb average. Only statistical error bars are reported on the data points. Fig. 9. Comparison of corrected and uncorrected ozone profiles to a constant ozone level of 55 5 ppb, typical of a well-mixed atmosphere over greater Athens in the first two weeks of September 1994. day of good atmospheric mixing. The boundary layer limit has been estimated from our backscattering measurements to be 2.8 km. By opposition, fitting a Gaussian function to the appropriate part of the histograms offers a good way to correct measurements for this bias. After correction the lidar values become compatible with the mean ozone concentration measured by the airplane as 57.4 ppb. Furthermore a mean background ozone concentration of 55 5 ppb has been measured by combining our lidar profiles and airplane measurements for conditions of good atmospheric mixing during the first two weeks of September. The measurement conditions were comparable for 12 and 13 September, and Fig. 9 gives another illustration of the capacity of the correction scheme to retrieve ozone concentrations compatible with that mean value to the top of the boundary layer. Only the statistical errors on the ozone profiles are presented in Figs. 8 and 9. They have been obtained by the propagation of a mean signal error along the ozone retrieval scheme. 16 The average total uncertainty on our measurement has been evaluated as lower than 20% below 1500 m. 14 The fitting procedure proposed here makes it possible to correct the measurement for a systematic effect, which otherwise prevents a reliable ozone concentration determination above 1500 m. The sensitivity of our calculated concentrations at long range to the fit has been evaluated by varying the portion of the histogram to which the Gaussian curve has been fitted. Three cases have been considered: the function is fitted only to the upper halves of the raising and falling edges; it is fitted to the raising edge and to the upper third of the falling edge; it is fitted to the raising edge and to the upper two thirds of the falling edge. The final ozone concentrations do not vary by more than 5% from the results obtained with the actual choice. The range increase of the measurement is not the only advantage of the correction. Physical effects 20 September 1997 Vol. 36, No. 27 APPLIED OPTICS 6861

of measurement, and it was then subtracted from the actual signals. Such an exponential behavior was reported to be not always visible, limiting the possibility of application of this correcting method. Unfortunately in our data set this exponential decrease could never be observed, so that it has not been possible for us to compare the results of both correction schemes. Fig. 10. Identification of ozone accumulation in the planetary boundary layer derived from a corrected profile. can also become visible in regions where the validity of the lidar measurement is not easily determined with a classical approach. Thus, in some cases, patterns appear on the profiles, making interpretation difficult because they are not always statistically significant. For example, the slight raise observed in the ozone concentration on the uncorrected profile of Fig. 10, immediately followed by a decrease, would be difficult to assert with confidence if the analysis were to stay at this stage for this measurement on 8 September at 4:00 p.m. With the corrected profile a clear step in the ozone concentration becomes visible. An ozone profile recorded by airplane on 10 September at 1:00 p.m. confirms that such steps correspond to a known physical effect in the region of our measurements. The ozone concentration measured by the airplane is higher because more ozone was present on 10 September. This is confirmed by ground measurement only because unfortunately our lidar instrument was not operational that day. Furthermore the airplane flight was at 1:00 p.m., and this time usually corresponds to a peak of the ozone concentration owing to photochemical effects. As reported elsewhere, 14 our lidar measurements have also shown the existence of such a midday maximum of the ozone concentration during a photochemical episode observed on 14 September. The highest ozone concentrations are reached at the top of a zone in which pollutants and particles have been trapped because of an inversion of the temperature gradient. This accumulation is at the origin of the peak observed also for the atmospheric backscattering reconstructed from our raw lidar signals 14 and is also shown in Fig. 10. An alternative mean to correct for signal-induced bias effects has been described in the literature. 11 It was based on the observation of an exponential decrease of the lidar signal following the region over which backscattered photons were collected. An exponential function was fitted to the tail of the measured signal, its value was extrapolated to the region 6. Conclusions The shot per shot data recording of lidar signals for tropospheric ozone profiling makes it possible to reconstruct histograms of the ADC values obtained for each sampling channel of the converters. The analysis of their shape permits us to correct the measurements for experimental effects. A systematic raise of histogram skew with the range has been observed. It has been interpreted as caused by an after-pulse effect of the PMT. A correction scheme has been developed based on the fit of a Gaussian function to the noncorrupted part of the histograms. Consequently we have been able to increase the range of our measurements, to reduce relative statistical errors, and to get more accurate profiles even at intermediate distances, making it possible to identify significant structures linked to the atmospheric conditions. The authors thank F. Fenter for his comments on this manuscript and O. Klemm for providing us with airplane data. This research was supported by the Federal Office for Science and Education, the Swiss Commission for the Promotion of Applied Research, and LeCroy SA. References 1. R. M. Measures, Laser Remote Sensing Krieger, Malabar, Fla., 1992. 2. I. S. McDermid, D. A. Haner, M. M. Kleiman, T. D. Walsh, and M. L. White, Differential absorption lidar systems for tropospheric and stratospheric ozone measurements, Opt. Eng. 30, 22 30 1991. 3. H. Edner, K. Fredriksson, A. Sunesson, S. Svanberg, L. Unéus, and W. Wendt, Mobile remote sensing system for atmospheric monitoring, Appl. Opt. 26, 4330 4338 1987. 4. G. Ancellet, A. Papayannis, J. Pelon, and G. Mégie, DIAL tropospheric ozone measurement using Nd:YAG laser and the Raman shifting technique, J. Atmos. Oceanic Technol. 6, 832 839 1989. 5. J. Bösenberg, G. Ancellet, A. Apituley, H. Bergwerff, G. von Cossart, H. Edner, J. Fiedler, B. Galle, C. de Jonge, J. Mellqvist, V. Mitev, T. Schaberl, G. Sonnemann, J. Spakman, D. Swart, and E. Wallinder, Tropospheric ozone lidar intercomparison experiment, TROLIX 91, field phase report, Rep. No. 102 Max-Planck-Institut für Meteorologie, Hamburg, Germany, 1993. 6. U. Kempfer, W. Carnuth, R. Lotz, and T. Trickl, A wide-range ultraviolet lidar system for tropospheric ozone measurements: development and application, Rev. Sci. Instrum. 65, 3145 3164 1994. 7. A. Papayannis, Instruments, in Instrument Development for Atmospheric Research and Monitoring, J. Bösenberg, D. Brassington, and P. Simon, eds. Springer-Verlag, Berlin, Germany, 1997, pp. 33 88. 8. L. Fiorani, B. Calpini, L. Jaquet, H. Van den Bergh, and E. 6862 APPLIED OPTICS Vol. 36, No. 27 20 September 1997

Durieux, A combined determination of wind velocities and ozone concentrations for a first measurement of ozone fluxes with a DIAL instrument during the MEDCAPHOT-TRACE campaign, Atmos. Environ. to be published. 9. R. E. W. Pettifer, Signal induced noise in lidar experiments, J. Atmos. Terr. Phys. 37, 669 673 1975. 10. C. J. Grund and E. W. Eloranta, University of Wisconsin high spectral resolution lidar, Opt. Eng. 30, 6 12 1991. 11. J. A. Sunesson, A. Apituley, and D. P. J. Swart, Differential absorption lidar system for routine monitoring of tropospheric ozone, Appl. Opt. 33, 7045 7058 1994. 12. E. Durieux, B. Calpini, A. Clappier, L. Fiorani, M. Flamm, L. Jaquet, and H. Van den Bergh, An innovative high repetition rate shot per shot acquisition system and its application to a new kind of tropospheric lidar measurements, in Excimer Lasers, L. D. Laude, ed. Kluwer, Dordrecht, The Netherlands, 1994, pp. 245 267. 13. E. V. Browell, Lidar measurements of tropospheric gases, Opt. Eng. 21, 128 132 1982. 14. E. Durieux, L. Fiorani, B. Calpini, M. Flamm, L. Jaquet, and H. Van den Bergh, Tropospheric ozone measurements over the Great Athens Area during the MEDCAPHOT-TRACE campaign with a new shot-per-shot DIAL instrument. Experimental system and results, Atmos. Environ. to be published. 15. I. Ziomas, Proposal for the Mediterranean campaign of photochemical tracers transport and chemical evolution MEDCAPHOT-TRACE, 1993 Aristotle University, Thessaloniki, Greece. 16. E. Durieux and L. Fiorani, Data analysis, in Instrument Development for Atmospheric Research and Monitoring, J. Bösenberg, D. Brassington, and P. Simon, eds. Springer- Verlag, Berlin, Germany, 1997, pp. 89 116. 17. M. Elbaum and P. Diament, SNR in the photocounting images of rough objects in partially coherent light, Appl. Opt. 15, 2268 2275 1976. 18. J. Pelon and G. Mégie, Ozone monitoring in the troposphere and lower stratosphere: evaluation and operation of a ground-based lidar station, J. Geophys. Res. 87, 4947 4955 1982. 20 September 1997 Vol. 36, No. 27 APPLIED OPTICS 6863