Cold Regions Research and Engineering Laboratory. Probability and Statistics in Sensor Performance Modeling ERDC/CRREL TR-10-12

Transcription

1 ERDC/CRREL TR-- Geospatial Research and Engineering Proaility and Statistics in Sensor Performance Modeling Kenneth K. Yamamoto, D. Keith Wilson, and Chris L. Pettit Decemer Cold Regions Research and Engineering Laoratory Approved for pulic release; distriution is unlimited.

2

3 Geospatial Research and Engineering ERDC/CRREL TR-- Decemer Proaility and Statistics in Sensor Performance Modeling Kenneth K. Yamamoto, D. Keith Wilson, and Chris L. Pettit Cold Regions Research and Engineering Laoratory U.S. Army Engineer Research and Development Center 7 Lyme Road Hanover, NH Final report Approved for pulic release; distriution is unlimited. Prepared for Under U.S. Army Corps of Engineers Washington, DC 34- AT4 GEOINT Eploitation in Man-Made Environments: Nations to Insurgents (GEMENI)

4 ERDC/CRREL TR-- ii Astract: Signals from many military targets of interest are often strongly randomized, due to the irregular mechanisms y which the signals are generated and propagated. In particular, complicated and dynamic terrestrial/atmospheric environments (with man-made ojects, vegetation, and turulence) randomize signals through random atmospheric and terrestrial processes affecting the propagation. Signals may also e considered random due to uncertainties in the knowledge of the propagation environment and target-sensor geometry. Predictions of sensor performance and recommendations of sensor types and placements derived from them, thus, should account for the random nature of the sensed signals. This report discusses software-modeling approaches for characterizing signals suject to random generation and propagation mechanisms. By representing signals with random variales, they are manipulated statistically to make proailistic predictions of sensor performance. Both the theory and implementation in a general, ojectoriented software design for attlefield signal transmission and sensing is eplained. The Java-language software program is called Environmental Awareness for Sensor and Emitter Employment. Some important numerical issues in the implementation are also discussed. DISCLAIMER: The contents of this report are not to e used for advertising, pulication, or promotional purposes. Citation of trade names does not constitute an official endorsement or approval of the use of such commercial products. All product names and trademarks cited are the property of their respective owners. The findings of this report are not to e construed as an official Department of the Army position unless so designated y other authorized documents. DESTROY THIS REPORT WHEN NO LONGER NEEDED. DO NOT RETURN IT TO THE ORIGINATOR.

5 ERDC/CRREL TR-- iii Contents Acronyms... iv Preface... v Introduction... Statistical Modeling of Signals... 3 Signature data features... 3 Proailistic modeling of features... 3 Statistical analysis for measuring sensor performance Software Implementation... 7 Signal model ojects... 7 Signal model inheritance relationships... 8 Representing a specific feature via a signal model Sums of Random Variales... A very simple scheme... Using the Laplace approimation... 4 The convolution approach... 4 Central limit theorem Numerical Issues... 7 Computing the quantile using the isection method... 7 Computing the cdf using Gauss-Legendre quadrature on the pdf... 8 Numerical staility and well-posed prolems... Illustrative eamples of numerically stale statistical computations... 6 Conclusions... 8 References... 9 Appendi: Floating-Point Implementations of Statistical Formulae... 3 Floating point... 3 Stale statistical implementations Gaussian distriution Lognormal distriution Eponential distriution... 4 Gamma distriution... 4 X X transformed Rice-Nakagami distriution Report Documentation Page... 55

6 ERDC/CRREL TR-- iv Acronyms ccdf cdf DST EASEE FFT IEEE NaN pdf complementary cumulative distriution function cumulative distriution function decision-support tool Environmental Awareness of Sensor and Emitter Employment fast Fourier transform Institute of Electrical and Electronics Engineers not a numer (value) proaility density function

7 ERDC/CRREL TR-- v Preface This study was conducted for the U.S. Army Corps of Engineers. Funding was provided y the Engineer Research and Development Center (ERDC) Geospatial Intelligence (GEOINT) program, Eploitation in Man-made Environments: Nations to Insurgents (GEMENI). The work was performed y the Signature Physics Branch (RR-D) of the Research and Engineering Division (RR) at ERDC s Cold Regions Research and Engineering Laoratory (ERDC-CRREL). The principal investigator was Dr. D. Keith Wilson (RR-D). The authors thank Dr. George G. Koenig of ERDC-CRREL s Terrestrial and Cryospheric Sciences Branch (RR-G) and Mr. Michael Parker of ERDC-CRREL s Force Projection and Sustainment Branch (RR-H) for providing helpful comments on a draft version of this report. At the time of pulication, Dr. Lindamae Peck was Chief, RR-D; Dr. Justin B. Berman was Chief, RR; and Mr. Dale R. Hill was the Acting Technical Director for Geospatial Research and Engineering. The Deputy Director of ERDC-CRREL was Dr. Lance D. Hansen, and the Director was Dr. Roert E. Davis. COL Kevin J. Wilson was the Commander and Eecutive Director of ERDC, and Dr. Jeffery P. Holland was the Director.

8

9 ERDC/CRREL TR-- Introduction A variety of modern-day Army missions rely on effective sensing capailities that provide intelligence on the adversary and to protect friendly forces from enemy detection. Sensors that are stationary (e.g., microphones, geophones, and ground-ased radars) and moving (e.g., cameras on unattended aerial vehicles and ground vehicles) assist operations such as persistent surveillance of small, forward-operating ases and rapid covert troop maneuvers in the air and on the ground. When advantageous, sensing is often performed in multiple signal modalities including visile, infrared, acoustic, seismic, radiofrequency, chemical, and iological. Yet, despite the increasing assortment of sensors availale, knowledge and epertise of how to use them efficiently in particular environments and missions is quite frequently lacking (Hie et al. 7). Since terrain and weather effects on signals are comple and oftentimes counterintuitive, computational simulations are valuale to oth facilitate quick and accurate decision making when planning the use of sensors in an actual mission as well as to guide the development of sensor technologies in general. When multiple diverse signal modalities are involved, it would e ideal to integrate them all into a single simulation environment rather than having one for each modality. Environmental Awareness of Sensor and Emitter Employment (EASEE) is a software framework that provides a single environment for analyzing sensor performance involving many different signal modalities. This aility for multimodal signal analysis then enales EASEE to also perform higher-level data synthesis needed to answer critical sensing questions such as the sensor types and locations est suited for accomplishing mission ojectives within the constraints of a particular environment. A comprehensive description of the software design and structure of EASEE is provided in Wilson et al. (9). In addition to reconciling models of many, differing signal modalities into a common software framework, a decision-support tool (DST) must also systematically assess the uncertainty with its predictions. Accounting for and measuring uncertainty is often overlooked in current software tools predicting sensor performance, ut the reliaility of a DST s recommenda-

10 ERDC/CRREL TR-- tions is oviously key to well-informed decision making. In the complicated process of modeling environmental effects on signal transmission and sensing, many types and levels of uncertainty are involved. A thorough eplanation of practical uncertainty issues is availale in Wilson et al. (8). Generally, uncertainties may e due to incomplete and/or inaccurate knowledge (of, say, the weather for a particular time and location) or due to processes that are purely stochastic (e.g., sensitivity of wave propagation to irresolvale and highly variale environmental details). It is the latter type of uncertainty namely stochastic/random uncertainty with which this report is concerned. (Note that sensor performance is also affected y non-stochastic processes not addressed here.) In principle, propagation of acoustic, seismic, and electromagnetic (including visual and infrared) waves as well as the dispersion of transient gases are still sensitive to many fine-scale and even dynamic environmental variations that cannot e known or determined. Practically speaking, second-to-second variaility in signal characteristics (e.g., sound level, seismic energy, concentration of a chemical agent, etc.) caused y unoservale and unpredictale environmental features (e.g., turulent eddies, vegetation, dust particles, precipitation, small uran structures) are essentially random, so statistical models are necessary to represent signal and noise (Strohehn 978; Andrews and Phillips 5). Wilson et al. (8) outlines specific eamples in greater detail. Proailistic information aout signal and noise may then e processed to compute statistical metrics on sensor performance and signal-emitter detectaility. The report egins y first eplaining the premise and theory of modeling signals statistically in Chapter, which also descries how statistical modeling of signals lead to proailistic predictions on sensor performance. Then, Chapter 3 eplains how the statistical modeling is implemented efficiently in EASEE s general, oject-oriented structure. A discussion on various approaches for summing multiple random variales is given in Chapter 4, which is required when simulating multiple target and noise signals arriving at a sensor. Some general numerical issues in programming statistical computations are illustrated in Chapter 5, while the Appendi supplies specific information on how particular statistical calculations were implemented.

11 ERDC/CRREL TR-- 3 Statistical Modeling of Signals Signature data features The EASEE software design (Wilson et al. 9) operates on asic, identifying qualities of a signal called signature data features (or features for short). The features can e thought of as the most essential characteristics of signals that might identify their source (i.e., an oservale that is a function of time at the source, sensor, or somewhere in etween). Generally, features are etracted from raw sensor data after some low-level processing (e.g., calirations to remove sensor response or filtering into spectral ands), typically eing a conservative quantity (such as energy or mass). Eamples include: sound power of a harmonic line or in a standard octave and; infrared rightness in near, shortwave, midwave, longwave, or far ands; components of an electromagnetic field vector; and concentration of a particular chemical or iological species. By using features rather than raw signals, simulations like EASEE are made not only quicker and much more efficient ut also more general and nonspecific to any particular sensor system. Proailistic modeling of features As discussed earlier, irregular generation mechanisms and random propagation processes may make it impractical to predict feature characteristics deterministically; rather, they can only e descried proailistically. Thus, signal and noise features may e regarded as random variales. Then, statistical distriutions of signal features are generated, propagated, and processed. A variety of statistical models are appropriate for representing sensor data. For eample, the eponential distriution descries a single, strongly scattered signal, which often arises when acoustic or electromagnetic waves are scattered y oth ojects and turulent wind. A version of the Rice- Nakagami model (specifically with a variale transformation of X X ) may closely approimate signal power distriutions of certain acoustic and seismic signals. Since the lognormal distriution models variales that can e the multiplicative product of many independent, positive random variales, atmospheric scientists often use it to descrie plume sizes and frequency distriutions of transient gases from turulent processes in the air

12 ERDC/CRREL TR-- 4 (Baker et al. 983; Limpert et al. ). Generally, signals are represented with proaility density functions (pdfs) that have nonnegative support ecause signal power or concentration can never e negative. However, if the mean is positive and much larger than the standard deviation, pdfs with real-numer support like the Gaussian may also e suitale for representing a signal. Currently, EASEE implements five continuous statistical models namely, the Gaussian, lognormal, eponential, gamma, and the X X transformed Rice-Nakagami as well as a discrete model. (Other eamples of statistical models for signals are in Burdic 984; Wilson et. al. ; and Nadarajah 8) For consistency, all of these models are used to descrie distriutions of signal power (or intensity) rather than amplitude. Thus, while the untransformed Rice-Nakagami model may descrie the distriution of signal amplitudes of various acoustic and seismic signals, it is necessary to find the corresponding model that would represent the distriution of the signal powers. Since signal power is related to the square of the amplitude, the correct model for the signal-power distriution is the Rice-Nakagami with a variale transformation of X X, which is what is coded in EASEE. (Whenever the transformed Rice-Nakagami is referred to in this report, the X X transformed Rice-Nakagami model is meant.) Details, including the derivation of the X X transformed Rice-Nakagami random vari- ale, are provided in the Appendi. Statistical analysis for measuring sensor performance When features are descried proailistically, it is susequently possile to derive statistical metrics of sensor performance. There are actually four proailities that are generally of interest:. P cd proaility that noise only is present when the sensor determines that noise only is present (a correct dismissal). P fa proaility that noise only is present when the sensor determines that a target is present (a false alarm) 3. P fd proaility that a target is present when the sensor determines that noise only is present (a false dismissal) 4. P d proaility that a target is present when the sensor determines that a target is present (a correct detection)

13 ERDC/CRREL TR-- 5 These proailities are given y the following integrals: P P P P cd fa fd d f f f d P d P cd d P f d Pfd d fa where: f pdf of the noise signal alone, f pdf of the target and noise signals together, and the detection threshold set y the detection algorithm. A detection algorithm computes an appropriate that ideally optimizes sensor performance y increasing proaility of detection and decreasing proaility of false alarm. Some eamples (which are currently implemented in EASEE) include:. Neyman-Pearson (constant false-alarm rate) criterion. asolute threshold detection 3. relative threshold detection 4. error minimization 5. Bayes risk minimization As the integrals aove suggest, various proailities of interest are computed y manipulating pdfs of signal and noise. In fact, integrations of pdfs elow and aove a threshold value are the cumulative distriution function (cdf) and the complementary cumulative distriution function (ccdf), respectively. Thus, we may also write: P P P P cd fa fd d f f f f d F ( ) C d F ( ) F ( ) d F ( ) C d F ( ) F ( )

14 ERDC/CRREL TR-- 6 where: f pdf, F cdf, and F C ccdf with the suscripts and denoting the pdf, cdf, or ccdf of the noise signal and comined target and noise signal, respectively. Strictly speaking, the cdf is actually the integral of the pdf from negative infinity to the threshold value; however, the integral from zero would only e negligily different (if at all) when signals are represented with appropriate statistical models, whose measure limits to zero (or is even noneistent) in the negative domain, since negative signal power or concentration is impossile. The aforementioned detection algorithms also compute the quantile (or inverse-cdf) function when determining signal-power (or concentration) thresholds corresponding to specified proailities. To otain a particular statistical representation, oth the type of random variale and the values for its parameters are needed. While the pdf and cdf for each type of random variale are coded to use its unique parameters (e.g., location, shape, scale, etc.), it is useful to also descrie random variales with the universal parameters, mean and variance. For twoparameter random variales, closed-form epressions may e found to convert its unique parameters to/from mean and variance. Finally, the signal pdfs within the integrals aove are often joint pdfs, since multiple signals of interest and noise may arrive at a sensor simultaneously. When multiple signal sources are present within a sensor s vicinity, random variales representing each signal source must e summed at the receiver. Depending on the type of random variales eing summed, computing the eact summation can e complicated and intensive. Thus, otaining a quick approimation may e more practical. A couple of approimating approaches and the most efficient method for computing eact results are descried later in this report.

15 ERDC/CRREL TR Software Implementation The EASEE software design is formulated within the conceptual framework of oject-oriented programming in the Java language. For a comprehensive description on the oject-oriented structure and approach to signal transmission and sensing in EASEE, it is recommended to consult Wilson et al. (9). Here, only a more thorough and updated eplanation of the signal model ojects will e given. Signal model ojects As previously descried, random environmental effects on transmitted and received features are accounted for y representing them as random variales. Parametric descriptions of these random variales are programmed in EASEE within Java classes. These Java classes are denoted as signal models and instances of them are signal model ojects. Signal model ojects are key in the architecture of EASEE, since they are what are actually transmitted, received, and processed, as outlined in Figure. The purple arrows in the figure specify where signal models are passed from one of the five components (illustrated in oes) to another. Specifically, the feature generator produces a signal model oject that is inputted and then altered y the feature propagator and feature sensor efore it is finally analyzed y the feature processor to produce an inference, which represents desired information derived from the data features such as proaility of detection or error of target location estimates. Signal model classes contain methods for the various, previously descried statistical operations necessary for making proailistic predictions on sensor performance, including:. setting the mean and variance. converting from mean and variance to unique parameter values 3. computing the pdf, cdf, and quantile 4. summing a random variale descried y an instance of the class with another one.

16 ERDC/CRREL TR-- 8 Figure. Generalized flow diagram for transmitting, receiving, and processing signature data features in EASEE. These signal model ojects also store arrays of parameter values defining different random variales of the same type, which may later e distriuted across a terrain grid. Computations for each array component are parallelized when possile to improve computational efficiency. As mentioned earlier, EASEE currently has implementations for five continuous statistical models namely, the Gaussian, lognormal, eponential, gamma and the X X transformed Rice-Nakagami as well as a discrete model. Each of these models is represented via its own signal model class. Signal model inheritance relationships The overall signal-transmission and sensing process in EASEE is descried generally enough to apply for all signal modalities descried y potentially very different statistics so that each of the generalized components in the aove figure is coded to work with an astract representation of all statistical models. This astract representation is the parent astract Java class of all signal models and is called the astract signal model. It defines astract methods for the various required statistical operations, which are then individually implemented or overridden y suclass signal models representing different statistical models. Figure shows the inheritance relationships of the signal models currently availale in EASEE. Figure. Inheritance tree of statistical signal models.

17 ERDC/CRREL TR-- 9 The constant signal model that etends the ase astract signal model represent constant (i.e., time-invariant) signals involving only one variale for the signal mean. A collection of variale signal models etend the constant signal. Presently, the inheritance relationships are organized so that deeper suclasses add more characteristics to their parent classes. Suclasses inherit and then modify the fields and/or methods of its parent class. For eample, nonconstant signal models allow nonzero variance; whereas, the constant signal model does not. The gamma distriution generalizes the eponential distriution, which is, in fact, a special case of the gamma distriution with shape parameter equal to and the scale parameter equal to the mean. It may e useful, however, to structure inheritance relationships differently. For eample, it may e useful to somehow categorize various distriutions into similar statistical families so the sum of multiple closely related random variales may e approimated y a parametric pdf within the same family. One of the simplest methods for summing multiple random variales (descried later) may, in fact, e made more efficient if inheritance is organized in such a way. Otherwise, it may e more appropriate to use interfaces to delineate statistical families within the current signal model inheritance tree, which can e used to make the descried approimation method for summing random variales more efficient. While the generalized components in the simulation scheme are coded to accept and return astract signal models, specific suclass signal models will actually e used in actual simulations as appropriate via polymorphism. This programming approach using inheritance and polymorphism greatly improves software efficiency and the potential for EASEE to continuously develop and include new capailities. Namely, eisting signalgeneration, propagation, and processing algorithms would automatically operate on all new signal models that etend the astract signal model and, conversely, new processing methods that work with the astract signal model would instantly apply to all eisting and future suclass signal models. Representing a specific feature via a signal model While the signal models fully descrie a parametric statistical model, they are not eplicitly associated with any particular signature data feature. Features are separately defined as Java-enumerated types, whose representation y a signal model are assigned y a given suclass implementa-

18 ERDC/CRREL TR-- tion of the feature generator as appropriate. Thus, the signal data transmissions passed from one generalized component to another in Figure are essentially packets of information assemled y the feature generator that lists transmitted features and their corresponding signal-model representations. For organization and utility, features are defined and grouped in different Java enum classes, typically y modality. For instance, all seismic features and all radiofrequency features are enumerated in their own, separate classes. When appropriate, further distinctions are defined within a single signal modality such as with acoustic features, which have separate enum classes for acoustic octave ands (oth standard and third-octave) and linearly spaced spectral ands.

19 ERDC/CRREL TR-- 4 Sums of Random Variales When simulating multiple signals of interest and noise arriving at a sensor, proaility-of-detection calculations are performed on the pdf of the sum of multiple signals. Thus, the final pdf that is analyzed at the sensor represents a sum of multiple random variales, where each individual signal is descried y a unique pdf. The calculation is trivial for summing multiple Gaussian random variales, since the pdf of the sum is another Gaussian pdf with mean and variance equal to the sum of the means and variances, respectively, of each contriuting random variale. In fact, the mean and variance of the sum of any (even potentially dissimilar) types of random variales is always, y definition, the sum of the means and variances of each summed random variale. (This property is referred to in the rest of this report y saying that the means and variances of the summed random variales are conserved. ) Yet, despite this property, implementation for summing multiple non-gaussian random variales is not straightforward ecause an analytical form for the pdf of the sum is usually not availale. A very simple scheme One of the simplest methods for representing the sum of multiple random variales of related types is to approimate the sum with a pdf that conserves the mean and variance of the original contriutions. The pdf for the sum may e the same as one of the original random variales, or a generalized form of the pdf of the original variales. This approach is eact if all the summed random variales are constant or Gaussian, and also for some other very particular cases, such as for gamma random variales with the same scale parameter. Otherwise, it is only an approimation that is not necessarily accurate in all cases. For instance, Stewart et al. (7) oserve that this approach is only reasonaly accurate when summing two gamma random variales if the shape parameters are not lower than. and the scale parameters do not differ y more than a factor of. Accuracy of approimating the sum of two lognormal random variales y another lognormal random variale may e adequate in one tail of the summed distriution ut not the necessarily the other (Mehta et al. 7). The accuracy of this approimation with other distriutions such as the transformed Rice-Nakagami is unverified. While the method seems to make intuitive sense, the accuracy of the approach is difficult to confirm or predict

20 ERDC/CRREL TR-- for various distriutions and cases. It is reasonale to assume that there are limitations and that it is not good in all cases. However, this method is a very simple alternative to other more roust yet computationally intensive approaches and much easier to implement. It is also important to note that parametric pdfs representing various signals are themselves only estimates to egin with ecause oth the models and the parameters in the models used to generate them are approimate due to idealizations of the signal physics and uncertainties in the weather, ground conditions, source signature and directivity, noise ackground, etc. Especially if it will greatly increase computational intensity, it is not necessarily worthwhile to compute the sum of multiple, imprecise random variales with very high precision. Although the eact sum of multiple random variales of related type cannot typically e represented y an analytical pdf, this method essentially suggests selecting one that will serve as a good approimation with mean and variance conserved. Generally, the sum is est approimated y the pdf of one of the summed random variales, ut sometimes it is etter to use a pdf that is different ut still a generalized form of the summed random variales. For eample, the sum of multiple eponential random variales would e etter approimated y a gamma distriution rather than another eponential distriution, where means and variances are conserved. In fact, this would e eact if the eponential random variales were all identical (with the same mean). Likewise, the summation of multiple random variales of different ut related type is represented as a pdf of the most generalized form of the related random variales. An eample of this is the sum of an eponential random variale and a gamma random variale, which would e a gamma random variale with the means and variances conserved. Here, the eponential and gamma distriutions are closely related, elonging to the eponential class of density functions, where the gamma distriution with shape parameter equal to reduces to the eponential distriution with a mean equal to the scale parameter (i.e., the eponential distriution is a special case of the gamma distriution). So, in fact, the sum of eponential and gamma random variales is essentially just a sum of gamma random variales. Conceivaly (ut not necessarily), the method could well approimate the sum of many different kinds of random variales, so long as a parametric distriution that generalizes a diverse set of random variales is availale.

21 ERDC/CRREL TR-- 3 For instance, the sum of any comination of two or more of the lognormal, eponential, Weiull, and gamma distriutions may e well approimated y a generalized gamma distriution with means and variances conserved, since the aforementioned, well-known distriutions are actually special cases of the generalized, three-parameter gamma distriution. Indeed, there are other three-parameter distriutions that generalize various wellknown distriutions such as the generalized and skew normal distriutions, oth of which can mimic other skewed distriutions like the gamma, lognormal, and Weiull distriutions ut also includes the normal distriution as a special case. Though there is an added complication in representing the sum of random variales with a three-parameter distriution, since the three parameters must e estimated numerically via maimum likelihood or method of moments, it is possile and potentially worthwhile if the process proves to e relatively less computationally intensive (Stacy and Mihram 965; Ashkar et al. 988; Varanasi and Aazhang 989). When using the method of moments, it is helpful to notice that the third unstandardized central moment can e computed eactly for the sum, since they are also conserved like the means and variances, which, in fact, are the first raw moment and the second unstandardized central moment, respectively. This results in three moments that can e computed eactly for the sum, which can e used to estimate the required three parameters to otain the generalized distriution. Otherwise, it may e advantages to compute a few particular cumulants of the generalized distriution of the sum to estimate its parameters instead, since all cumulants are conserved. As descried, this method only applies for summing multiple random variales of related type. If no generalized distriution for two or more diverse random variales is availale, it is impossile to predict the appropriate analytical pdf that est approimates their sum. For eample, the lognormal and the transformed Rice-Nakagami random variales are not of related type and do not have a parametric distriution that generalizes oth of them, so this method does not apply in this case. Finally, this method is oth commutative and associative in that the order in which multiple random variales are summed together does not affect the final representation of their sum. This follows first from the fact that the summation of means and variances (and the third unstandardized central moment, if necessary) is itself commutative and associative. Then, the

22 ERDC/CRREL TR-- 4 method selects the analytical pdf of the same type or of a more generalized type than that which descries the two summed random variales, regardless of which one is presented first in the method s algorithm. Using the Laplace approimation The Laplace approimation is another approach that introduces the approimation at a different stage in the summation of random variales. Rather than approimating the resulting sum of multiple random variales with an appropriate analytical pdf as in the previously descried method, it is possile to approimate each non-gaussian random variale addend y a Gaussian pdf. Then, the sum of these Gaussian approimations may e computed eactly, since the eact sum of multiple Gaussian random variales is simply another Gaussian random variale with means and variances conserved. Since many familiar, unimodal random variales may e roughly approimated y a Gaussian pdf, the Laplace approimation is widely used in many situations requiring manipulation of multiple random variales that must e or is greatly preferred to e Gaussian. In our case, otaining a good Gaussian approimation of multiple random variales greatly simplifies their summation y avoiding the use of a more computationally intensive, numerical approach. It is also likely that the Laplace approimation is within the precision errors inherent in the idealizations and uncertainties of the models and parameters used in simulating the signal-transmission and sensing process. Since the Laplace method involves the second-degree Taylor epansion of the given, non-gaussian pdf s logarithm centered on the mode, oth the mode and the second derivative of the pdf s logarithm at the mode must e computed. Often these can e calculated with simple, closed-form epressions. If not, quick numerical computations can e devised noting that the mode is defined as where the maimum of the pdf occurs. Details on the theory and definition of the Laplace approimation can e found in Bishop (6). The convolution approach Though it is the most computationally intense, it is possile to always compute the pdf of the sum of two independent random variales via the convolution of their pdfs. Since all pdfs are compactly supported and

23 ERDC/CRREL TR-- 5 locally integrale, the convolution of two pdfs, f and g, denoted as f * g, is the following well-defined and continuous integral transform: def ( f g)( t) f ( τ ) g( t τ ) R The preferred method for computing the convolution is via the forward and inverse discrete Fourier transforms, for which fast Fourier transform algorithms are availale. According to the convolution theorem, the Fourier transform of a convolution is the pointwise product of the Fourier transforms of the two functions eing convoluted together: F dτ [ f * g] k F[ f ] F[ g] where k is a constant that depends on the normalization of the Fourier transform and F [ f ] denotes the Fourier transform of f. Then, the inverse Fourier transform may e applied to otain the convolution, f * g: f * g F [ F[ f * g ] F [ k F[ f ] F[ g ] kf [ F[ f ] F[ g ] Since the characteristic function of a random variale is a Fourier transform of its pdf and is availale for every type of random variale, it is then possile to compute the sum of multiple random variales using the inverse Fourier transform and characteristic functions y the general formula aove. By using the convolution theorem and the fast Fourier transform (FFT) algorithms, an implementation for computing the sum of multiple random variales via the numerical convolution of two pdfs may e made practical. In practice, the FFT algorithm must e implemented carefully to avoid errors from discretization and truncation of integrals as much as possile. This can e quite difficult, ut, if done correctly, the convolution approach would make it possile to sum multiple random variales of even vastly different type. It is also commutative and associative, as required. Central limit theorem As more independent random variales are summed, the joint random variale monotonically converges into a Gaussian distriution given certain conditions. By the classical central limit theorem, the sum must e of many independent and identically distriuted random variales, each having positive variance with oth mean and variance eing finite.

24 ERDC/CRREL TR-- 6 However, the theorem actually also holds for non-identically distriuted random variales if Lyapunov s condition is satisfied, which further implies that Lindeerg s condition is also met (Ash and Doléans-Dade ). Though the rate of convergence depends on the type(s) of the identical or non-identically distriuted random variale addends, it is conceivale that the sum of even three to five non-gaussian random variales may already e well approimated y a Gaussian distriution. Hence, in cases where enough random variales are added together, where at least one of them is non-gaussian, the sum may e simply represented y a Gaussian distriution with means and variances conserved. Any implementation for summing multiple random variales should take this limit theorem into account to improve oth accuracy as well as computational speed. When the theorem applies, the sum should e directly represented as Gaussian with means and variances conserved rather than computing many Laplace approimations, numerically evaluating many convolutions, or using a different pdf for the final representation.

25 ERDC/CRREL TR Numerical Issues Although most statistical operations required for the pdfs descried in this report are relatively straightforward to program, it is helpful to illustrate some of the operations that make use of certain numerical algorithms. It is also worthwhile to descrie how more complicated operations must e programmed carefully so that they compute properly using floating-point arithmetic on a computer. For the latter concern, only a selection of eamples is presented to illustrate some common prolems that may arise. The Appendi includes further details. Computing the quantile using the isection method With the eception of the eponential model, for which a closed-form formula for evaluating the quantile is availale, all other statistical models are coded to numerically solve for the quantile,, y approimating the root, y, of the following function, Q ( y), using the isection method for a given P cdf with the tolerance set to the lowest representale numer in inary doule (64-it) floating-point precision (approimately ): Q ( y) cdf ( y) P where the root, y, is equal to the quantile, : ( y ) cdf ( y ) P cdf ( y ) cdf cdf ( y ) y Q cdf The lower- and upper-ounds used for the isection method is set to span the entire domain supported y a particular statistical distriution that is representale in inary doule (64-it) floating-point precision. Specifically, the lower- and upper-ounds for the lognormal, gamma, and X X transformed Rice-Nakagami models are set to zero and the highest representale numer (approimately.8 38 ). For random variales with real-numer support like the Gaussian, the lower- and upper-ound is set to the lowest and highest representale numer (approimately and.8 38 ), respectively. The isection method used to solve for the quantile requires that the cdf e always computale within the ounds. Since the ounds are set to span very large domains, the cdf must e programmed carefully so that it remains computale for all input values within the domain. Concerns related to how the cdf should e programmed to satisfy this required condition are discussed in the last two

26 ERDC/CRREL TR-- 8 sections of this chapter ( Numerical staility and well-posed prolems and Illustrative eamples of numerically stale statistical computations ). Although the isection method only converges linearly (i.e., the asolute error of the true and estimated roots is halved at each step), it is preferred here ecause of it roustness. Unfortunately, other methods with higher convergence rates such as the Newton and Secant methods fail due to the nature of the cdf curve, which is asymptotically a horizontal line (with the first derivative equal to zero) at oth ends. An optimized algorithm with a miture of different methods is conceivale ut does not necessarily improve the convergence rate apprecialy. Computing the cdf using Gauss-Legendre quadrature on the pdf Usually there is an epression for the cdf that could e directly implemented, ut, if it is complicated (e.g., it contains a special function that is difficult to program), numerical integration of the pdf may e a simpler alternative, although it is generally more computationally intensive: cdf pdf ( t) The Gauss-Legendre quadrature rule is a very efficient method for performing the integration. Basically, the method approimates the definite integral of a function via a weighted sum of function values at specified points within the domain of integration. More details aout staly implementing this method for aritrary integration domains are provided in the Appendi. In addition to the lower- and upper-ounds of the integration, the method requires specification of the numer of weighted function values, N, to e summed. The appropriate numer of points for the Gauss-Legendre rule, N, so that the integration of the pdf is computed to a sufficient level of precision depends on the nature of the pdf curve over the specified range of integration. Since the theory ehind the N-point Gauss-Legendre rule ensures that it computes eact integrals for the class of polynomials of degree N- or less, one must essentially predict the degree of the polynomial that adequately approimates the pdf over the specified range of integration to find the appropriate N. As N is increased, the numerical integration ecomes more computationally intensive (and time-consuming), so it is dt

27 ERDC/CRREL TR-- 9 important to choose an N that is not greater than what suffices to otain results of sufficient precision. To minimize the numer of points necessary in the Gauss-Legendre rule and still compute cdf pdf ( t) dt with sufficient precision, it is helpful to minimize the range of integration as much as possile y programming cdf computation differently depending on : cdf pdf l u pdf ( t) dt, for pdf, for pdf ( t) dt, for pdf, for pdf and < mean > and < mean > and mean and > mean where l and u are approimations of the lowest and highest values for with nonzero pdf values, respectively. More precisely, l and u are computed as the first integer standard deviate elow and aove the mean with a pdf value of zero, respectively. If l and/or u is eyond the given statistical distriution s representale support domain, it is set as the lowest and highest representale value of the support domain. Then, empirical investigations suggest that the 5-point Gauss-Legendre rule would e sufficiently precise for computing any desired cdf value (Figure 3). Although all parameters cannot e tested, it is reasonale to assume that the general ehavior of all pdfs of the same type is similar enough that the 5-point rule would always suffice. This approach is currently used for the X X transformed Rice- Nakagami model to avoid implementing the first-order Marcum Q- Function in its cdf epression. It is also included in the gamma model as an alternative in the potential event that the implementation for the direct approimation fails for certain inputs. As it turns out, some care must e taken in programming floating-point implementations of certain pdfs and cdfs (including the gamma cdf), as will e discussed in the following sections.

28 ERDC/CRREL TR-- Figure 3. Comparison of cdf generated using different N-point Gauss-Legendre rules to integrate the X X transformed Rice-Nakagami pdf with mean and var 7. Vertical ais: pdf(); horizontal ais:. From left to right on the top row: N, 5, and. From left to right on the ottom row: N, 5, and. As N is increased, accuracy of the pdf integration is improved and a smooth cdf curve is generated. Numerical staility and well-posed prolems When programming the various statistical operations, it is important to code calculations so that they function properly when performed using floating-point arithmetic on a computer. For programming very simple calculations, it is not necessarily important to understand the mechanics of floating-point arithmetic. It is, however, essential to understand when formulating more complicated functions and numerical algorithms. When a calculation is programmed to function properly within the limitations of floating point, it is called numerically stale. Golderg (99) has a much more comprehensive treatment of the suject than that which is provided here. The asic prolem is caused y computers with limited memory that can only represent (and, therefore, work with) a finite, discrete set of numers. Floating point descries a system for allocating computer memory efficiently so that a very large set of numers can e made representale. A more thorough ackground on floating point is provided in the Appendi. A variety of issues due to floating point are well known. First, there is round-off error, which is the difference etween the calculated approimation of a numer and its eact mathematical value. This type of error is in-

29 ERDC/CRREL TR-- troduced when the inputs and outputs of an arithmetic operation must e rounded to the nearest representale numer. Typically, round-off errors are insignificant in simple computations, ecept in relatively rare instances (e.g., when two large, nearly equal numers are sutracted, causing the numer of accurate significant digits in the difference to e greatly reduced, sometimes called a catastrophic cancellation). But, small round-off errors may accumulate unacceptaly when performing operations that require a sequence of many calculations (e.g., large summations, matri inversion, eigenvector computation, and solving differential equations). Another concern is more critical in our case, particularly when programming the pdf, cdf, and quantile. Given that the final result of a computation is indeed representale, it is important that the results of the various arithmetic operations within that computation remain within the given floating-point representation s domain of representale numers. Ideally, a computation should e programmed so that this condition is met for all representale inputs. When this condition is not met, the final result is typically either very inaccurate or not computale at all, since an arithmetic operation has resulted in an overflow or underflow (which is denoted y saying that the computation has suffered from premature overflow or underflow, respectively). Then, the computation could unpredictaly return an undefined or infinite numer (e.g.,, -, or NaN), although the true, mathematical solution to the computation is within the representale range. For eample, computation of the average of two nonzero numers, and y, that are within the representale domain using the formula, ( y) /, may return or - if ( y) overflows, although the average would always lie within the representale range. To address this latter issue, it is helpful to notice that sometimes a single calculation can e achieved in several ways. In the previous eample for computing the average of the two numers, and y, it is etter to program the distriuted formula, / y /, which does not have the overflow prolem. But, this formula is now susceptile to underflow y the terms, / or y /, for small enough or y. Since premature underflow would only present, at most, an asolute error of twice the smallest representale numer, it is more important to program against premature overflow than premature underflow in this case for other situations when underflow may present a division-y-zero operation, premature underflow is much more important to prevent. The final, numerically stale algorithm for averaging the two numers, and y, would compute the distriuted

30 ERDC/CRREL TR-- formula, / y /, only if the original formula, ( y) /, results in an overflow. If a computation is rather complicated, a careful, numerically stale floating-point computer implementation of it can e quite difficult to program. Although various algeraic manipulations are commonly used, each computation is unique and requires different techniques. While it is easier to program a computation that must e stale only within a limited domain of inputs, the inputs that the computation must e ale to handle may e difficult to know or predict. Illustrative eamples of numerically stale statistical computations Since they share some general similarities, many statistical formulae suffer from common issues. For instance, many pdfs are of the following form: a pdf a ep( cd ). ep ( cd ) A simple, direct implementation of the pdf epression may not normally present issues with ordinary values for a,, c, and d. However, this formulation is certainly not roust for all representale a,, c, and d. Namely, the numerator and denominator may oth overflow, although their ratio is a perfectly ordinary value (that always lies within the representale range). This potential prolem is easily remedied y a single, straightforward algeraic rearrangement. Specifically, the division of two potentially large numers, A and B, is etter encoded via the difference of their logarithms: A B ep [ ln( A) ln( B) ] so that the ratio ecomes computale for a much more epansive domain of A and B, since it is only required that their logarithms not overflow. Using this rearrangement, the general pdf epression aove ecomes: pdf a a ep ln ep ln ln ep( cd ) ep( cd ) [ ( a) ln( ep( cd ))] ep[ ln( a) cd] ep[ ln( a) cd] It is important to note here that not all overflows are necessarily an issue ut that they ecome prolematic only when inaccuracies may result. For eample, the potential overflow of the term, cd, in the aove rearrangement is not an issue ecause it would always imply that the pdf is zero

31 ERDC/CRREL TR-- 3 and, in fact, actually results in a zero to e returned for the pdf. (It is helpful to notice here that the rearranged pdf would always equal zero when the term, cd, ecomes large enough even efore it overflows.) Overflow of A and/or B in the previous epression is troulesome ecause it does not imply anything aout the true value of A / B and inaccurately returns,, or NaN. Since computing eponentials and logarithms are more computationally epensive, it is etter to calculate the stale rearrangement as an alternative only when the floating-point computation of A / B fails, if the simpler formulation is only rarely unstale. In certain cases, however, it may e that the straightforward formula, A / B, is actually more often unstale than it is stale. This is true, for eample, with the gamma cdf, which will e descried shortly. In these cases, it would e more efficient to just compute the stale rearrangement directly. The Appendi includes specific details on how to apply this rearrangement for each type of distriution. As just stated, this technique is useful in coding the gamma cdf, defined for, >, and > : cdf, P, Γ( ) where P is the regularized incomplete gamma function: P ( s, ) ( s, ) s r e Γ s Γ s and Γ and are the gamma function and the lower incomplete gamma function, respectively. In this formula, the individual terms,, and Γ ( ), overflow at modest values of,, and, although the regularized incomplete gamma function itself has the limiting values: P ( s, ) and ( s, ) P. Thus, it is also helpful in this case to implement the eponentiation of the difference of the logarithms of the two terms: r dr

32 ERDC/CRREL TR-- 4 cdf,, P, ep ln Γ Γ ep ln, ln. ( Γ) Though they are used less frequently, various other manipulations are useful in more specific circumstances (and often in comination with the first technique). For instance, after using this first technique, the Gaussian cdf still contains a term that resemles the following: ln ( a ). For all representale a and, the true value of this term is always within the representale range, ut would e returned whenever (a ) overflows. This issue can e avoided y programming the following: a a a ln c c c c c ( a ) ln c ln c ln ln( c) where c is set as the largest representale numer in the given floatingpoint system. A similar distriutive approach is used in multiple other situations, including the formula for the gamma pdf, which presents the following general epression: a c. When implemented directly, the sutraction y c is ignored whenever the term, a, overflows and results in the entire epression eing automatically equated to or -. This is unfortunate if c is of the same sign as a, since the entire epression may very well e within the representale range (if c is of large enough magnitude). This prolem can e easily avoided y programming the following instead: a c a c a c d d d d d a where d is selected to e large enough so that the term, d, does not overflow. In many instances, it is only important to simply pay attention to the order a of operations. This, in fact, applies for programming the term, d, in the

33 ERDC/CRREL TR-- 5 technique eplained aove. It also applies in many other situations, including when programming a term in the lognormal cdf, which is of the following form: a. c For even a simple epression like this, there are many ways to program it: a c a c a a a a c a. c c ( c) c With some quick investigation, it is relatively easy to confirm that (usually) only one of these implementations (if any) is stale given certain limitations on the domain of a,, and c. But, in a couple of special cases of this general epression, it is always est to implement them in the following way: a a and a a a. Simply changing the order of operations is often sufficient throughout parts of various statistical formulae; however, attention to this detail is quite easily overlooked. Application of this technique in specific cases is descried within the Appendi. When no particular ordering of operations is universally stale for the given domains of a,, and c, it is always possile to resort to the following general rearrangement that is stale for all representale a,, and c: a c a ep ln ep[ ln( a) ln ln( c) ]. c Finally, it is helpful to note certain consequences of finite-precision arithmetic when analyzing and implementing stale mathematical epressions in floating point. Essentially, any floating-point system represents a finite, discrete set of numers along the real numer line that retains the same relative difference only etween each representale numer (Figure 4).

34 ERDC/CRREL TR-- 6 This results in additions or sutractions y sufficiently small numers to e ignored ecause the floating-point system is not precise enough to represent the difference. This oservation is useful when programming, for instance, the following general epression, which arises in lognormal parameter conversions: ln ( a c) which ecomes the following for sufficiently large differences in magnitude etween a and c: a << c ( a c) ln( c) ln ln( c). ln This is a simpler alternative to the earlier, distriutive technique: a a a ln c c c c c ( a ) ln c ln c ln ln( c) if it can e known that there will always e a sufficiently large magnitude difference etween the added or sutracted terms. This oservation is often useful in analyzing the staility of an implementation and concluding that certain potential underflow issues are acceptale to ignore. Specifically, the magnitude difference of a given representale numer and an underflowing numer may e large enough that the underflow to zero would not affect their summation, since the contriution y the underflowing numer would e ignored anyways even if it was not necessarily underflowing. However, there are situations when it is prudent Figure 4. Floating-point numer line (Recktenwald 6).