Proceedings of the 2000 winter simulation conference

This paper reviews statistical methods for analyzing output data from computer simulations. First, it focuses on the estimation of steady-state system parameters. The estimation techniques include the replication/deletion approach, the regenerative method, the batch means method, and methods based on standardized time series. Second, it reviews recent statistical procedures to find the best system among a set of competing alternatives.

[1]  D. Daley.  The serial correlation coefficients of waiting times in a stationary single server queue , 1968, Journal of the Australian Mathematical Society.

[2]  Marvin K. Nakayama,et al.  Two-stage multiple-comparison procedures for steady-state simulations , 1999, TOMC.

[3]  S. D. Roberts,et al.  A Voice Assisted Simulation-Animation Architecture , 2000, 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165).

[4]  M. Schlesinger,et al.  When we don't know the costs or the benefits: Adaptive strategies for abating climate change , 1997 .

[5]  Farhad Azadivar,et al.  Simulation optimization methodologies , 1999, WSC '99.

[6]  Lee W. Schruben.  Simulation optimization using simultaneous replications and event time dilation , 1997, WSC '97.

[7]  W.-N. Yang.  Optimization using common random numbers, control variates and multiple comparisons with the best , 1989, WSC '89.

[8]  L. Schruben,et al.  Properties of standardized time series weighted area variance estimators , 1990 .

[9]  S. Andradóttir,et al.  A Simulated Annealing Algorithm with Constant Temperature for Discrete Stochastic Optimization , 1999 .

[10]  Jerry Banks.  Introduction to simulation , 1999, WSC '99.

[11]  Yu-Chi Ho,et al.  The problem of large search space in stochastic optimization , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[12]  Jörg Desel,et al.  Validation of Process Models by Construction of Process Nets , 2000, Business Process Management.

[13]  Keebom Kang,et al.  An Investigation of Finite-Sample Behavior of Confidence Interval Estimators , 1992, Oper. Res..

[14]  James R. Wilson,et al.  A revised simplex search procedure for stochastic simulation response-surface optimization , 1998, WSC '98.

[15]  Mark Fleischer.  Simulated annealing: past, present, and future , 1995, WSC '95.

[16]  Paul K. Davis,et al.  Experiments In Multiresolution Modeling (MRM) , 1998 .

[17]  Halim Damerdji,et al.  Strong Consistency of the Variance Estimator in Steady-State Simulation Output Analysis , 1994, Math. Oper. Res..

[18]  C. Dunnett.  A Multiple Comparison Procedure for Comparing Several Treatments with a Control , 1955 .

[19]  Lee W. Schruben,et al.  Note-New Confidence Interval Estimators Using Standardized Time Series , 1990 .

[20]  Gunar E. Liepins,et al.  Genetic algorithms: Foundations and applications , 1990 .

[21]  Lee W. Schruben,et al.  Designing simultaneous simulation experiments , 1999, WSC '99.

[22]  S. Andradóttir.  A Scaled Stochastic Approximation Algorithm , 1996 .

[23]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[24]  Jerry Banks.  Software for simulation , 1995, WSC '95.

[25]  Russell R. Barton,et al.  Modifications of the Nelder-Mead simplex method for stochastic simulation response optimization , 1991, 1991 Winter Simulation Conference Proceedings..

[26]  James R. Wilson.  Modeling dependencies in stochastic simulation inputs , 1997, WSC '97.

[27]  James R. Wilson,et al.  Convergence Properties of the Batch Means Method for Simulation Output Analysis , 2001, INFORMS J. Comput..

[28]  Michael C. Fu,et al.  Using perturbation analysis for gradient estimation, averaging and updating in a stochastic approximation algorithm , 1988, WSC '88.

[29]  Yu-Chi Ho,et al.  Effect of correlated estimation errors in ordinal optimization , 1992, WSC '92.

[30]  James R. Wilson,et al.  Evaluation of startup policies in simulation experiments , 1978 .

[31]  Paul T. Kidd,et al.  Agile Manufacturing: Forging New Frontiers , 1994 .

[32]  Barry L. Nelson,et al.  Ranking, selection and multiple comparisons in computer simulation , 1994, Proceedings of Winter Simulation Conference.

[33]  Yu-Chi Ho.  Overview of ordinal optimization , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[34]  Donald L. Iglehart,et al.  Large-sample theory for standardized time series: an overview , 1985, WSC '85.

[35]  Paul K. Davis,et al.  Adaptiveness in National Defense: The Basis of a New Framework , 1996 .

[36]  Andrew F. Seila,et al.  Multivariate inference in stationary simulation using batch means , 1987, WSC '87.

[37]  George S. Fishman,et al.  An Implementation of the Batch Means Method , 1997, INFORMS J. Comput..

[38]  C. Adami,et al.  Introduction To Artificial Life , 1997, IEEE Trans. Evol. Comput..

[39]  A. Tamhane.  Design and Analysis of Experiments for Statistical Selection, Screening, and Multiple Comparisons , 1995 .

[40]  J. S. Dagpunar,et al.  Principles of Discrete Event Simulation , 1980 .

[41]  A. Dalcher,et al.  A Simple Biosphere Model (SIB) for Use within General Circulation Models , 1986 .

[42]  S. Yakowitz.  A globally convergent stochastic approximation , 1993 .

[43]  James R. Wilson,et al.  A survey of research on the simulation startup problem , 1978 .

[44]  B. Schmeiser,et al.  Optimal mean-squared-error batch sizes , 1995 .

[45]  Leyuan Shi,et al.  Simultaneous simulation experiments and nested partition for discrete resource allocation in supply chain management , 1999, WSC '99.

[46]  Douglas J. Morrice,et al.  Sensitivity analysis in ranking and selection for multiple performance measures , 1999, WSC '99.

[47]  Douglas J. Morrice,et al.  An approach to ranking and selection for multiple performance measures , 1998, 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274).

[48]  Wei-Ning Yang,et al.  Using Common Random Numbers and Control Variates in Multiple-Comparison Procedures , 1991, Oper. Res..

[49]  Michael A. Crane,et al.  Simulating Stable Stochastic Systems: III. Regenerative Processes and Discrete-Event Simulations , 1975, Oper. Res..

[50]  Christopher G. Langton,et al.  Artificial Life , 2019, Philosophical Posthumanism.

[51]  Sigrún Andradóttir,et al.  A projected stochastic approximation algorithm , 1991, 1991 Winter Simulation Conference Proceedings..

[52]  B. Nelson,et al.  Using common random numbers for indifference-zone selection and multiple comparisons in simulation , 1995 .

[53]  P. Glynn.  Optimization of stochastic systems via simulation , 1989, WSC '89.

[54]  Miron Livny,et al.  Priority in Distributed Systems , 1985, RTSS.

[55]  Paul K. Davis,et al.  Effects of Terrain, Maneuver Tactics, and C4ISR on the Effectiveness of Long-Range Precision Fires , 2000 .

[56]  Chun-Hung Chen,et al.  An effective approach to smartly allocate computing budget for discrete event simulation , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[57]  Chun-Hung Chen,et al.  A gradient approach for smartly allocating computing budget for discrete event simulation , 1996, Winter Simulation Conference.

[58]  Michael C. Fu,et al.  A tutorial review of techniques for simulation optimization , 1994, Proceedings of Winter Simulation Conference.

[59]  D. B. Preston.  Spectral Analysis and Time Series , 1983 .

[60]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[61]  Barry L. Nelson,et al.  Ranking and selection for steady-state simulation , 2000, 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165).

[62]  A. F. Seila,et al.  A Batching Approach to Quantile Estimation in Regenerative Simulations , 1982 .

[63]  Brian Rooks.  Off‐line programming: a success for the automotive industry , 1997 .

[64]  William K. Horton,et al.  Designing Web based training: how to teach anyone anything anywhere anytime , 2000 .

[65]  James R. Wilson,et al.  Experimental performance evaluation of batch means procedures for simulation output analysis , 2000, 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165).

[66]  Leyuan Shi,et al.  An integrated framework for deterministic and stochastic optimization , 1997, WSC '97.

[67]  P. M. Pelagagge,et al.  Simulation tool for design and management optimization of automated interbay material handling and storage systems for large wafer fab , 1995 .

[68]  Ricardo Jiménez-Peris,et al.  New Technologies in Computer Science Education , 2000 .

[69]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[70]  Hiroshi Morita,et al.  Economical analysis of high volume flexible manufacturing system for agile manufacturing , 1998, Strategic Management of the Manufacturing Value Chain.

[71]  Jorge Haddock,et al.  Application of a simulation optimization system for a continuous review inventory model , 1987, WSC '87.

[72]  Shirish S. Joshi,et al.  An improved response surface methodology algorithm with an application to traffic signal optimization for urban networks , 1995, WSC '95.

[73]  Xiaolan Xie.  Dynamics and convergence rate of ordinal comparison of stochastic discrete-event systems , 1997, IEEE Trans. Autom. Control..

[74]  Chun-Hung Chen.  A lower bound for the correct subset-selection probability and its application to discrete-event system simulations , 1996, IEEE Trans. Autom. Control..

[75]  Michael A. Vidulich,et al.  Computer Modeling of Operator Mental Workload and Situational Awareness in Simulated Air-to-Ground Combat: An Assessment of Predictive Validity , 1998 .

[76]  Michael C. Fu,et al.  Simulation optimization of (s,S) inventory systems , 1992, WSC '92.

[77]  George S. Fishman,et al.  Statistical Analysis for Queueing Simulations , 1973 .

[78]  Jörg Desel,et al.  Modeling, Simulation and Analysis of Business Processes , 2000, Business Process Management.

[79]  Optimizing discrete event systems with the simultaneous perturbation stochastic approximation algorithm , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[80]  K. Preston White,et al.  An Effective Truncation Heuristic for Bias Reduction in Simulation Output , 1997, Simul..

[81]  A. Law,et al.  A procedure for selecting a subset of size m containing the l best of k independent normal populations, with applications to simulation , 1985 .

[82]  Yun Bae Kim,et al.  Single run optimization using the reverse-simulation method , 1997, WSC '97.

[83]  Leyuan Shi,et al.  Optimization via adaptive sampling and regenerative simulation , 1999, WSC '99.

[84]  Lee W. Schruben,et al.  Detecting Initialization Bias in Simulation Output , 1982, Oper. Res..

[85]  Randall P. Sadowski,et al.  Simulation with Arena , 1998 .

[86]  Farhad Azadivar,et al.  Genetic algorithms in optimizing simulated systems , 1995, WSC '95.

[87]  Gül Gürkan,et al.  Sample-path optimization in simulation , 1994, Proceedings of Winter Simulation Conference.

[88]  L. Dai.  Convergence properties of ordinal comparison in the simulation of discrete event dynamic systems , 1995 .

[89]  W. David Kelton,et al.  Simulation-based estimation of quantiles , 1999, WSC '99.

[90]  Russell R. Barton,et al.  Sample size selection for improved Nelder-Mead performance , 1995, WSC '95.

[91]  David Krahl,et al.  A message-based discrete event simulation architecture , 1997, WSC '97.

[92]  B.W.M. Bettonvil.  A formal description of discrete event dynamic systems including infinitesimal perturbation analysis , 1989 .

[93]  Rajan Suri,et al.  Single run optimization of a SIMAN model for closed loop flexible assembly systems , 1987, WSC '87.

[94]  John W. Fowler,et al.  Measurable improvements in cycle-time-constrained capacity , 1997, 1997 IEEE International Symposium on Semiconductor Manufacturing Conference Proceedings (Cat. No.97CH36023).

[95]  Hiroshi Morita,et al.  A basic study on high volume flexible manufacturing system for agile manufacturing , 1996, APMS.

[96]  Yorai Wardi,et al.  Application of descent algorithms with Armijo stepsizes to simulation-based optimization of queueing networks , 1991, [1991] Proceedings of the 30th IEEE Conference on Decision and Control.

[97]  Paul K. Davis,et al.  Exploratory Analysis of The Halt Problem: A Briefing on Methods and Initial Insights , 1998 .

[98]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[99]  Richard Hillestad,et al.  Transforming the Force: Suggestions for DoD Strategy , 1998 .

[100]  James O. Henriksen,et al.  SLX: pyramid power , 1999, WSC '99.

[101]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[102]  Michael Paul Griscik.  A New Algorithm for Stochastic Approximation , 1970 .

[103]  Barry L. Nelson,et al.  Input modeling tools for complex problems , 1998, 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274).

[104]  George S. Fishman,et al.  LABATCH.2: software for statistical analysis of simulation sample path data , 1998, 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274).

[105]  A. Gaivoronski.  Optimization of Stochastic Discrete Event Dynamic Systems: A Survey of Some Recent Results , 1992 .

[106]  J. S. Ivey,et al.  Nelder-Mead simplex modifications for simulation optimization , 1996 .

[107]  S. Jacobson,et al.  A harmonic analysis approach to simulation sensitivity analysis , 1999 .

[108]  P. Davis.  New Challenges for Defense Planning: Rethinking How Much is Enough , 1995 .

[109]  Lee W. Schruben,et al.  Confidence Interval Estimation Using Standardized Time Series , 1983, Oper. Res..

[110]  C. Cassandras,et al.  Sample path properties of timed discrete event systems , 1989, Proc. IEEE.

[111]  Kevin J. Healy.  Identifying policies that are most likely to produce a desirable outcome , 1994, Proceedings of Winter Simulation Conference.

[112]  Jerry Banks,et al.  Discrete Event Simulation , 2002, Encyclopedia of Information Systems.

[113]  S. Gupta.  On Some Multiple Decision (Selection and Ranking) Rules , 1965 .

[114]  Jack P. C. Kleijnen,et al.  A Java-based simulation manager for optimization and response surface methodology in multiple-response parallel simulation , 1999, WSC '99.

[115]  Thomas J. Schriber,et al.  An introduction to simulation using GPSS/H , 1991 .

[116]  P. Glynn,et al.  Stochastic Optimization by Simulation: Convergence Proofs for the GI/G/1 Queue in Steady-State , 1994 .

[117]  R. Jordan.  A One-dimensional temperature model for a snow cover : technical documentation for SNTHERM.89 , 1991 .

[118]  Richard E. Barlow,et al.  Statistical Theory of Reliability and Life Testing: Probability Models , 1976 .

[119]  Lee W. Schruben,et al.  Retrospective simulation response optimization , 1991, 1991 Winter Simulation Conference Proceedings..

[120]  Chun-Hung Chen,et al.  An empirical evaluation of several methods to select the best system , 1999, TOMC.

[121]  Paul K. Davis,et al.  Case history of using entity-level simulation as imperfect experimental data for informing and calibrating simpler analytical models for interdiction , 2000, Defense, Security, and Sensing.

[122]  Philip Heidelberger,et al.  Quantile Estimation in Dependent Sequences , 1984, Oper. Res..

[123]  R. Rubinstein.  How to optimize discrete-event systems from a single sample path by the score function method , 1991 .

[124]  P. Davies,et al.  Kendall's Advanced Theory of Statistics. Volume 1. Distribution Theory , 1988 .

[125]  Peter J. Denning,et al.  Computing the profession , 1999, SIGCSE '99.

[126]  Steve Bankes,et al.  Exploratory Modeling for Policy Analysis , 1993, Oper. Res..

[127]  Melba M. Crawford,et al.  Modeling and simulation of a nonhomogeneous poisson process having cyclic behavior , 1991 .

[128]  Yu-Chi Ho,et al.  Iterative ordinal optimization and its applications , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[129]  Richard W. Conway,et al.  Some Tactical Problems in Digital Simulation , 1963 .

[130]  Stephen D. Roberts,et al.  A time-varying Poisson arrival process generator , 1984 .

[131]  Halim Damerdji,et al.  On the batch means and area variance estimators , 1994, Proceedings of Winter Simulation Conference.

[132]  Natalie M. Steiger,et al.  Output analysis: ASAP2: an improved batch means procedure for simulation output analysis , 2002, WSC '02.

[133]  Marvin K. Nakayama.  Selecting the best system in steady-state simulations using batch means , 1995, WSC '95.

[134]  Thomas W. Lucas,et al.  The Stochastic Versus Deterministic Argument for Combat Simulations: Tales of When the Average Won't Do , 2000 .

[135]  Yolanda Carson,et al.  Simulation optimization: methods and applications , 1997, WSC '97.

[136]  Ward Whitt,et al.  Estimating the asymptotic variance with batch means , 1991, Oper. Res. Lett..

[137]  Stephen E. Chick,et al.  New Two-Stage and Sequential Procedures for Selecting the Best Simulated System , 2001, Oper. Res..

[138]  L. Schruben,et al.  A Review of Techniques for Simulation Optimization , 1986 .

[139]  Loo Hay Lee,et al.  Explanation of goal softening in ordinal optimization , 1999, IEEE Trans. Autom. Control..

[140]  MICHAEL C. FU,et al.  Techniques for optimization via simulation: an experimental study on an (s,S) inventory system , 1997 .

[141]  Lee W. Schruben,et al.  Optimal Tests for Initialization Bias in Simulation Output , 1983, Oper. Res..

[142]  Bruce W. Schmeiser,et al.  Batch Size Effects in the Analysis of Simulation Output , 1982, Oper. Res..

[143]  Sigurdur Ólafsson,et al.  Iterative ranking-and-selection for large-scale optimization , 1999, WSC '99.

[144]  Xi-Ren Cao,et al.  Perturbation analysis of discrete event dynamic systems , 1991 .

[145]  Bruce W. Schmeiser,et al.  Overlapping batch means: something for nothing? , 1984, WSC '84.

[146]  Donald L. Iglehart,et al.  Simulating Stable Stochastic Systems, VI: Quantile Estimation , 1976, JACM.

[147]  M. Kendall,et al.  Kendall's advanced theory of statistics , 1995 .

[148]  Keebom Kang,et al.  Cramér-von Mises variance estimators for simulations , 1991, WSC '91.

[149]  Rajan Suri,et al.  Finite-time behavior of two simulation optimization algorithms , 1990, 1990 Winter Simulation Conference Proceedings.

[150]  Averill M. Law,et al.  A Sequential Procedure for Determining the Length of a Steady-State Simulation , 1979, Oper. Res..

[151]  David Goldsman,et al.  Large-Sample Results for Batch Means , 1997 .

[152]  Chun-Hung Chen,et al.  New development of optimal computing budget allocation for discrete event simulation , 1997, WSC '97.

[153]  Alexandre Dolgui,et al.  A stochastic method for discrete and continuous optimization in manufacturing systems , 1997, J. Intell. Manuf..

[154]  R. Durrett.  Probability: Theory and Examples , 1993 .

[155]  Barry L. Nelson,et al.  Multiple comparisons with the best for steady-state simulation , 1993, TOMC.

[156]  S. Colbeck,et al.  A Theory of Water Percolation in Snow , 1972, Journal of Glaciology.

[157]  Rajarshi Das,et al.  A Study of Control Parameters Affecting Online Performance of Genetic Algorithms for Function Optimization , 1989, ICGA.

[158]  D. Goldsman,et al.  ASAP2: an improved batch means procedure for simulation output analysis , 2002, Proceedings of the Winter Simulation Conference.

[159]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[160]  Daniel H. Ockerman.  Initialization bias tests for stationary stochastic processes based upon standardized time series techniques , 1995 .

[161]  Mary R. Albert,et al.  A fast, physically based point snowmelt model for use in distributed applications , 1998 .

[162]  Wil M. P. van der Aalst,et al.  Modelling logistic Systems with ExSpect , 1990, DMIS.

[163]  Barry L. Nelson,et al.  Statistical screening, selection, and multiple comparison procedures in computer simulation , 1998, 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274).

[164]  Gerald W. Evans,et al.  Multicriteria optimization of simulation models , 1991, 1991 Winter Simulation Conference Proceedings..

[165]  S. Dalal,et al.  ALLOCATION OF OBSERVATIONS IN RANKING AND SELECTION WITH UNEQUAL VARIANCES , 1971 .

[166]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[167]  Y. Rinott.  On two-stage selection procedures and related probability-inequalities , 1978 .

[168]  Theodore B. Aldrich,et al.  The Development and Application of Models to Predict Operator Workload During System Design , 1989 .

[169]  Y. Mualem,et al.  Hydraulic conductivity of unsaturated porous media: Generalized macroscopic approach , 1978 .

[170]  van der Wmp Wil Aalst,et al.  A Petri-net-based Tool to analyze workflows , 1997 .

[171]  J H McCracken,et al.  Analyses of Selected LHX Mission Functions: Implications for Operator Workload and System Automation Goals , 1984 .

[172]  Daniel H. Kim.  Introduction to Systems Thinking , 1999 .

[173]  Peter D. Welch,et al.  On the relationship between batch means, overlapping means and spectral estimation , 1987, WSC '87.

[174]  H. Robbins,et al.  ON THE ASYMPTOTIC THEORY OF FIXED-WIDTH SEQUENTIAL CONFIDENCE INTERVALS FOR THE MEAN. , 1965 .

[175]  Wheyming Tina Song,et al.  On the estimation of optimal batch sizes in the analysis of simulation output , 1996 .

[176]  Azer Bestavros,et al.  Timeliness via speculation for real-time databases , 1994, 1994 Proceedings Real-Time Systems Symposium.

[177]  Chiahon Chien.  Small-sample theory for steady state confidence intervals , 1988, WSC '88.

[178]  John M. Charnes,et al.  Power comparisons for the multivariate batch-means method , 1990, 1990 Winter Simulation Conference Proceedings.

[179]  M. Fu.  Convergence of a stochastic approximation algorithm for the GI/G/1 queue using infinitesimal perturbation analysis , 1990 .

[180]  Keebom Kang,et al.  Cycle time reduction for naval aviation depots , 1998, 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274).

[181]  M. Hossein Safizadeh,et al.  Optimization in simulation: Current issues and the future outlook , 1990 .

[182]  James R. Wilson,et al.  Improved batching for confidence interval construction in steady-state simulation , 1999, WSC '99.

[183]  J. Hsu.  Constrained Simultaneous Confidence Intervals for Multiple Comparisons with the Best , 1984 .

[184]  Benjamin S. Blanchard,et al.  Logistics engineering and management , 1986 .

[185]  Max Henrion,et al.  Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis , 1990 .

[186]  Eric A. Anderson,et al.  National Weather Service river forecast system: snow accumulation and ablation model , 1973 .

[187]  Colin E. Bell,et al.  WHEN THE NUMBER OF ALTERNATIVES IS LARGE , 1981 .

[188]  Chris P. Tsokos,et al.  Parameter estimation of the Weibull probability distribution , 1994 .

[189]  Peter Bosch.  Effective multithreading , 2000 .

[190]  Jorge Haddock,et al.  Simulation optimization using simulated annealing , 1992 .

[191]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[192]  Peter W. Glynn,et al.  Likelilood ratio gradient estimation: an overview , 1987, WSC '87.

[193]  D. Iglehart.  Simulating stable stochastic systems, V: Comparison of ratio estimators , 1975 .

[194]  Barry L. Nelson,et al.  Two-Stage Multiple Comparisons with the Best for Computer Simulation , 1995, Oper. Res..

[195]  H. Martz.  Bayesian reliability analysis , 1982 .

[196]  Sigrún Andradóttir,et al.  A review of simulation optimization techniques , 1998, 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274).

[197]  W. Philipp,et al.  Almost sure invariance principles for partial sums of weakly dependent random variables , 1975 .

[198]  J. Lawless.  Statistical Models and Methods for Lifetime Data , 2002 .

[199]  Michael C. Fu,et al.  Optimization via simulation: A review , 1994, Ann. Oper. Res..

[200]  Mahmoud H. Alrefaei,et al.  A modification of the stochastic ruler method for discrete stochastic optimization , 2001, Eur. J. Oper. Res..

[201]  J. J. Swain,et al.  Tests for transient means in simulated time series , 1994 .

[202]  Barry L. Nelson,et al.  Optimization over a finite number of system designs with one-stage sampling and multiple comparisons with the best , 1988, WSC '88.

[203]  Sigrún Andradóttir,et al.  A Stochastic Approximation Algorithm with Varying Bounds , 1995, Oper. Res..

[204]  G. Hornberger,et al.  Empirical equations for some soil hydraulic properties , 1978 .

[205]  M.K. Nakayama.  Selecting the best system in steady-state simulations using batch means , 1995, Winter Simulation Conference Proceedings, 1995..

[206]  Marvin K. Nakayama,et al.  Multiple-comparison procedures for steady-state simulations , 1997 .

[207]  R. Kaplan,et al.  The Balanced Scorecard: Translating Strategy into Action , 1996 .

[208]  John M. Charnes,et al.  Multivariate simulation output analysis , 1991, 1991 Winter Simulation Conference Proceedings..

[209]  S. Chick.  Selecting The Best System: A Decision-theoretic Approach , 1997, Winter Simulation Conference Proceedings,.

[210]  Barry L. Nelson,et al.  Input modeling when simple models fail , 1995, WSC '95.

[211]  Rajan Suri,et al.  Single Run Optimization of Discrete Event Simulations—An Empirical Study Using the M/M/l Queue , 1989 .

[212]  Hossein Arsham.  STOCHASTIC OPTIMIZATION OF DISCRETE EVENT SYSTEMS SIMULATION , 1996 .

[213]  J. Neumann.  Distribution of the Ratio of the Mean Square Successive Difference to the Variance , 1941 .

[214]  R. Wilcox.  A Table for Rinott's Selection Procedure , 1984 .

[215]  S. Andradóttir.  Optimization of the transient and steady state behavior of discrete event systems , 1996 .

[216]  Barry L. Nelson,et al.  Simultaneous ranking, selection and multiple comparisons for simulation , 1993, WSC '93.

[217]  Barry L. Nelson,et al.  A fully sequential procedure for indifference-zone selection in simulation , 2001, TOMC.

[218]  T. W. E. Lau,et al.  Universal Alignment Probabilities and Subset Selection for Ordinal Optimization , 1997 .