Estimation of Performance Measures from Simulation

A Monte Carlo simulation run can be viewed as a statistical experiment which is the software counterpart of an experiment on a real system. The objective of the experiment is to allow us to make inferences about one or more performance parameters. The observations (measurements) consist of discretely spaced values of a finite-duration segment of a random process at some point in the system. Hence, these measurements are inherently random. Therefore, the inferences made can only be statistical. In this chapter we study the statistical aspect of measuring performance parameters within the simulation context. The parameters that we shall specifically be concerned with are the average level, average power, signal-to-noise ratio, probability distribution and density function, bit (symbol) error probability and power spectral density. We will also briefly review some popular visual indicators of signal quality, which are often generated in a simulation to provide a qualitative sense of the performance of a digital system, but which can also be used to provide somewhat loose quantitative bounds on performance.

[1]  S. Weinstein.  Estimation of Small Probabilities by Linearization of the Tail of a Probability Distribution Function , 1971 .

[2]  Nirode Mohanty.  Random Signals Estimation and Identification , 1986 .

[3]  Milton Abramowitz,et al.  Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables , 1964 .

[4]  M. C. Jeruchim,et al.  Estimation of the signal-to-noise ratio (SNR) in communication simulation , 1989, IEEE Global Telecommunications Conference, 1989, and Exhibition. 'Communications Technology for the 1990s and Beyond.

[5]  Stephen B. Weinstein,et al.  Theory and application of some classical and generalized asymptotic distributions of extreme values , 1973, IEEE Trans. Inf. Theory.

[6]  Letizia Lo Presti,et al.  Semianalytic BER evaluation by simulation for noisy nonlinear bandpass channels , 1988, IEEE J. Sel. Areas Commun..

[7]  William Gardner,et al.  Rice's Representation for Cyclostationary Processes , 1987, IEEE Trans. Commun..

[8]  M. Jeruchim,et al.  Spacing Limitations of Geostationary Satellites Using Multilevel Coherent PSK Signals , 1972, IEEE Trans. Commun..

[9]  Paul Wagner,et al.  Least reliable bits coding (LRBC) for high data rate satellite communications , 1992 .

[10]  Michael Devetsikiotis,et al.  A useful and general technique for improving the efficiency of Monte Carlo simulation of digital communication systems , 1990, [Proceedings] GLOBECOM '90: IEEE Global Telecommunications Conference and Exhibition.

[11]  Heinz-Josef Schlebusch.  On the asymptotic efficiency of importance sampling techniques , 1993, IEEE Trans. Inf. Theory.

[12]  Jean C. Walrand,et al.  Review of 'Large Deviation Techniques in Decision, Simulation, and Estimation' (Bucklew, J.A.; 1990) , 1991, IEEE Trans. Inf. Theory.

[13]  G. W. Lank.  Theoretical aspects of importance sampling applied to false alarms , 1983, IEEE Trans. Inf. Theory.

[14]  Kung Yao,et al.  On Importance Sampling in Digital Communications - Part I: Fundamentals , 1993, IEEE J. Sel. Areas Commun..

[15]  V. Prabhu,et al.  Error-Rate Considerations for Digital Phase-Modulation Systems , 1969 .

[16]  D. Gooding.  Performance Monitor Techniques for Digital Receivers Based on Extrapolation of Error Rate , 1968 .

[17]  R. Tapia,et al.  Nonparametric Probability Density Estimation , 1978 .

[18]  P. Balaban,et al.  A Modified Monte-Carlo Simulation Technique for the Evaluation of Error Rate in Digital Communication Systems , 1980, IEEE Trans. Commun..

[19]  N. Blachman.  The Power Spectrum of a Digital Signal , 1974, IEEE Trans. Commun..

[20]  Michel C. Jeruchim,et al.  On optimum and suboptimum biasing procedures for importance sampling in communication simulation , 1990, IEEE Trans. Commun..

[21]  John S. Sadowsky,et al.  A new method for Viterbi decoder simulation using importance sampling , 1990, IEEE Trans. Commun..

[22]  Sheldon M. Ross,et al.  Stochastic Processes , 2018, Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics.

[23]  A. Papoulis.  Signal Analysis , 1977 .

[24]  Khaled Ben Letaief,et al.  Computing bit-error probabilities for avalanche photodiode receivers by large deviations theory , 1992, IEEE Trans. Inf. Theory.

[25]  D. Slepian,et al.  On bandwidth , 1976, Proceedings of the IEEE.

[26]  Michael Devetsikiotis,et al.  An algorithmic approach to the optimization of importance sampling parameters in digital communication system simulation , 1993, IEEE Trans. Commun..

[27]  Maurizio Longo,et al.  Comparative performance analysis of some extrapolative estimators of probability tails , 1988, IEEE J. Sel. Areas Commun..

[28]  F. Amoroso,et al.  The bandwidth of digital data signal , 1980, IEEE Communications Magazine.

[29]  A.R.K. Sastry,et al.  Models for channels with memory and their applications to error control , 1978, Proceedings of the IEEE.

[30]  P. Balaban,et al.  Statistical evaluation of the error rate of the fiberguide repeater using importance sampling , 1976, The Bell System Technical Journal.

[31]  V. K. Prabhu,et al.  Spectra of digital phase modulation by matrix methods , 1974 .

[32]  Don J. Torrieri.  The Information-Bit Error Rate for Block Codes , 1984, IEEE Trans. Commun..

[33]  J. Wolfowitz,et al.  Introduction to the Theory of Statistics. , 1951 .

[34]  Vijay K. Rohatgi,et al.  Statistical Inference , 1984 .

[35]  Michel C. Jeruchim.  On the Coding Gain for Degraded Channels , 1986, IEEE Trans. Commun..

[36]  K. Shanmugan,et al.  Random Signals: Detection, Estimation and Data Analysis , 1988 .

[37]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[38]  Andrew J. Viterbi,et al.  Convolutional Codes and Their Performance in Communication Systems , 1971 .

[39]  H. E. Rowe,et al.  Signals and Noise in Communication Systems , 1965 .

[40]  Qiang Wang,et al.  On the Application of Importance Sampling to BER Estimation in the Simulation of Digital Communication Systems , 1987, IEEE Trans. Commun..

[41]  Michel C. Jeruchim.  On the Application of Importance Sampling to the Simulation of Digital Satellite and Multihop Links , 1984, IEEE Trans. Commun..

[42]  Michel C. Jeruchim,et al.  An experimental investigation of conventional and efficient importance sampling , 1989, IEEE Trans. Commun..

[43]  John S. Sadowsky,et al.  On Importance Sampling in Digital Communications - Part II: Trellis-Coded Modulation , 1993, IEEE J. Sel. Areas Commun..

[44]  V. K. Prabhu,et al.  Error rate considerations for coherent phase-shift keyed systems with co-channel interference , 1969 .

[45]  Michel C. Jeruchim,et al.  Techniques for Estimating the Bit Error Rate in the Simulation of Digital Communication Systems , 1984, IEEE J. Sel. Areas Commun..

[46]  William Gardner,et al.  Common Pitfalls in the Application of Stationary Process Theory to Time-Sampled and Modulated Signals , 1987, IEEE Trans. Commun..

[47]  S.M. Kay,et al.  Spectrum analysis—A modern perspective , 1981, Proceedings of the IEEE.

[48]  Kung Yao,et al.  Improved importance sampling technique for efficient simulation of digital communication systems , 1988, IEEE J. Sel. Areas Commun..

[49]  P. Welch.  The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms , 1967 .

[50]  E. S. Pearson.  Biometrika tables for statisticians , 1967 .

[51]  M. D. Knowles.  Bit error rate estimation for channels with memory , 1988, IEEE Trans. Commun..

[52]  J. S. Sadowsky,et al.  Direct-sequence spread-spectrum multiple-access communications with random signature sequences: A large deviations analysis , 1991, IEEE Trans. Inf. Theory.

[53]  William Turin,et al.  Performance Analysis of Digital Transmission Systems , 1990 .

[54]  V. Johannes.  Improving on bit error rate , 1984, IEEE Communications Magazine.

[55]  Bruce R. Davis.  An Improved Importance Sampling Method for Digital Communication System Simulations , 1986, IEEE Trans. Commun..

[56]  D. Middleton.  An Introduction to Statistical Communication Theory , 1960 .

[57]  Mansour I. Irshid,et al.  Bit error probability for coherent M-ary PSK systems , 1991, IEEE Trans. Commun..

[58]  E. J. Gumbel,et al.  Statistics of Extremes. , 1960 .

[59]  R.L. Mitchell.  Importance Sampling Applied to Simulation of False Alarm Statistics , 1981, IEEE Transactions on Aerospace and Electronic Systems.

[60]  S. Pasupathy,et al.  Error rate monitoring for digital communications , 1982, Proceedings of the IEEE.

[61]  R. Gagliardi,et al.  PCM Data Reliability Monitoring Through Estimation of Signal-to-Noise Ratio , 1968 .

[62]  Michel C. Jeruchim,et al.  Developments in the Theory and Application of Importance Sampling , 1987, IEEE Trans. Commun..

[63]  F. Harris.  On the use of windows for harmonic analysis with the discrete Fourier transform , 1978, Proceedings of the IEEE.

[64]  J.S. Sadowsky,et al.  On large deviations theory and asymptotically efficient Monte Carlo estimation , 1990, IEEE Trans. Inf. Theory.

[65]  Michael Devetsikiotis,et al.  Importance Sampling Methodologies for Simulation of Communication Systems with Time-Varying Channels and Adaptive Equalizers , 1993, IEEE J. Sel. Areas Commun..