The Virtual Margin of Error - On the Limits of Virtual Machines in Scientific Research

Using Virtual Machines from public cloud providers, researchers gain access to a large pool of experimental infrastructure at comparatively low cost. However, as it is shown in this position paper based on dedicated experiments using real-life systems, Virtual Machines often do not provide accurate time measurements. These limitations are problematic for a variety of use cases, such as the runtime comparison of algorithms in the computer science domain.

[1]  Gerard Briscoe,et al.  Digital ecosystems in the clouds: Towards community cloud computing , 2009, 2009 3rd IEEE International Conference on Digital Ecosystems and Technologies.

[2]  Raj Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[3]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[4]  Edmund Kohlwey,et al.  Leveraging the Cloud for Big Data Biometrics: Meeting the Performance Requirements of the Next Generation Biometric Systems , 2011, 2011 IEEE World Congress on Services.

[5]  F. O R M A T I O N G U I D.  Timekeeping in VMware Virtual Machines , 2004 .

[6]  Ulrich Lampe,et al.  Optimizing the Distribution of Software Services in Infrastructure Clouds , 2011, 2011 IEEE World Congress on Services.

[7]  Randy H. Katz,et al.  A view of cloud computing , 2010, CACM.

[8]  Swati Sharma,et al.  An Empirical study of clock skew behavior in modern mobile and hand-held devices , 2011, 2011 Third International Conference on Communication Systems and Networks (COMSNETS 2011).

[9]  Luís Moura Silva,et al.  Evaluating the performance and intrusiveness of virtual machines for desktop grid computing , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[10]  Hai Jin,et al.  XenHVMAcct: Accurate CPU Time Accounting for Hardware-Assisted Virtual Machine , 2010, 2010 International Conference on Parallel and Distributed Computing, Applications and Technologies.

[11]  En-Jui Lee,et al.  Rapid 3D Seismic Source Inversion Using Windows Azure and Amazon EC2 , 2011, 2011 IEEE World Congress on Services.

[12]  Rajkumar Buyya,et al.  Article in Press Future Generation Computer Systems ( ) – Future Generation Computer Systems Cloud Computing and Emerging It Platforms: Vision, Hype, and Reality for Delivering Computing as the 5th Utility , 2022 .

[13]  Joaquim Marques de Sá,et al.  Applied Statistics Using SPSS, STATISTICA, MATLAB and R , 2003 .