Benchmarking Web API Quality

Web APIs are increasingly becoming an integral part of web or mobile applications. As a consequence, performance characteristics and availability of the APIs used directly impact the user experience of end users. Still, quality of web APIs is largely ignored and simply assumed to be sufficiently good and stable. Especially considering geo-mobility of today’s client devices, this can lead to negative surprises at runtime.

[1]  David Bermbach,et al.  Benchmarking Eventual Consistency: Lessons Learned from Long-Term Experimental Studies , 2014, 2014 IEEE International Conference on Cloud Engineering.

[2]  Philippe Suter,et al.  Inferring Web API Descriptions from Usage Data , 2015, 2015 Third IEEE Workshop on Hot Topics in Web Systems and Technologies (HotWeb).

[3]  Simson L. Garfinkel,et al.  An Evaluation of Amazon's Grid Computing Services: EC2, S3, and SQS , 2007 .

[4]  Markus Klems,et al.  Consistency Benchmarking: Evaluating the Consistency Behavior of Middleware Services in the Cloud , 2010, ICSOC.

[5]  Andy Zaidman,et al.  Web API Fragility: How Robust is Your Mobile Application? , 2015, 2015 2nd ACM International Conference on Mobile Software Engineering and Systems.

[6]  David Bermbach.  Benchmarking eventually consistent distributed storage systems , 2014 .

[7]  Bu-Sung Lee,et al.  WPress: An Application-Driven Performance Benchmark for Cloud-Based Virtual Machines , 2014, 2014 IEEE 18th International Enterprise Distributed Object Computing Conference.

[8]  Kevin Lee,et al.  Data Consistency Properties and the Trade-offs in Commercial Cloud Storage: the Consumers' Perspective , 2011, CIDR.

[9]  Shaohua Wang,et al.  How Do Developers React to RESTful API Evolution? , 2014, ICSOC.

[10]  Adam Silberstein,et al.  Benchmarking cloud serving systems with YCSB , 2010, SoCC '10.

[11]  Jacky W. Keung,et al.  Evaluating Cloud Platform Architecture with the CARE Framework , 2010, 2010 Asia Pacific Software Engineering Conference.

[12]  Samuel Kounev,et al.  Performance evaluation of message-oriented middleware using the SPECjms2007 benchmark , 2009, Perform. Evaluation.

[13]  David Bermbach,et al.  An Extendable Toolkit for Managing Quality of Human-Based Electronic Services , 2011, Human Computation.

[14]  Aleksander Slominski,et al.  A Graph-Based Data Model for API Ecosystem Insights , 2014, 2014 IEEE International Conference on Web Services.

[15]  Matjaz B. Juric,et al.  Comparison of performance of Web services, WS-Security, RMI, and RMI-SSL , 2006, J. Syst. Softw..

[16]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[17]  David Bermbach,et al.  Consistency in Distributed Storage Systems - An Overview of Models, Metrics and Measurement Approaches , 2013, NETYS.

[18]  Valérie Issarny,et al.  Dynamic Service Substitution in Service-Oriented Architectures , 2008, 2008 IEEE Congress on Services - Part I.

[19]  M. Kunze,et al.  Cloud Federation , 2011 .

[20]  Lin Xiao,et al.  YCSB++: benchmarking and performance debugging advanced features in scalable table stores , 2011, SoCC.

[21]  Dan S. Wallach,et al.  Performance analysis of TLS Web servers , 2006, TOCS.

[22]  Carsten Binnig,et al.  How is the weather tomorrow?: towards a benchmark for the cloud , 2009, DBTest '09.

[23]  David Bermbach,et al.  Benchmarking the Performance Impact of Transport Layer Security in Cloud Database Systems , 2014, 2014 IEEE International Conference on Cloud Engineering.

[24]  Sherif Sakr,et al.  Towards Comprehensive Measurement of Consistency Guarantees for Cloud-Hosted Data Storage Services , 2013, TPCTC.

[25]  Tilmann Rabl,et al.  Solving Big Data Challenges for Enterprise Application Performance Management , 2012, Proc. VLDB Endow..

[26]  Jörn Kuhlenkamp,et al.  Benchmarking Scalability and Elasticity of Distributed Database Systems , 2014, Proc. VLDB Endow..

[27]  Tim Kraska,et al.  An evaluation of alternative architectures for transaction processing in the cloud , 2010, SIGMOD Conference.

[28]  David Bermbach,et al.  Eventual consistency: How soon is eventual? An evaluation of Amazon S3's consistency behavior , 2011, MW4SOC '11.

[29]  David Bermbach,et al.  A Runtime Quality Measurement Framework for Cloud Database Service Systems , 2012, 2012 Eighth International Conference on the Quality of Information and Communications Technology.

[30]  Stefan Tai,et al.  What Are You Paying For? Performance Benchmarking for Infrastructure-as-a-Service Offerings , 2011, 2011 IEEE 4th International Conference on Cloud Computing.

[31]  Bettina Kemme,et al.  How consistent is your cloud application? , 2012, SoCC '12.

[32]  David Bermbach,et al.  AISLE: Assessment of Provisioned Service Levels in Public IaaS-Based Database Systems , 2015, ICSOC.

[33]  Frank Maurer,et al.  A Case Study of Web API Evolution , 2015, 2015 IEEE World Congress on Services.