Large Scale Application Response Time Measurement Using Image Recognition and Deep Learning

Application response time is a critical performance metric to assess the quality of software products. It is also an objective metric for user experience evaluation. In this paper, we present a novel method named CVART (Computer Vision–based Application Response Time measurement) for measuring the response time (latency) of an application. In our solution, we use image recognition and deep learning techniques to capture visible changes in the display of the device running the application to compute the application response time of an operation that triggers these visual changes. Appling CVART can bring multiple benefits compared to traditional methods. First, it allows measuring the response time that reflects a real user experience. Second, the solution enables the measurement of operations that are extremely hard or impossible to measure when using traditional methods. Third, it does not require application instrumentation, which is infeasible in many use cases. Finally, the method does not depend on any specific application or software platform, which allows building performance measurement and application monitoring tools that work on multiple platforms and on multiple devices. For demonstration, we present one use case of applying CVART to measure the application response time of virtual desktops hosted in the cloud or datacenter, and we evaluate its efficiency on measurement at large scale.

[1]  Anthony Steed,et al.  Measuring Latency in Virtual Environments , 2014, IEEE Transactions on Visualization and Computer Graphics.

[2]  Wojciech Zaremba,et al.  Recurrent Neural Network Regularization , 2014, ArXiv.

[3]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[4]  Lan Vu,et al.  An Automation Framework for Benchmarking and Optimizing Performance of Remote Desktops in the Cloud , 2017, 2017 International Conference on High Performance Computing & Simulation (HPCS).

[5]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[6]  Tim R. Norton.  End-To-End Response-Time: Where to Measure? , 1999, Int. CMG Conference.

[7]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[8]  Nickolai Zeldovich,et al.  Interactive Performance Measurement with VNCPlay , 2005, USENIX Annual Technical Conference, FREENIX Track.

[9]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[10]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Yichuan Tang,et al.  Deep Learning using Linear Support Vector Machines , 2013, 1306.0239.

[12]  Prasad Calyam,et al.  VDBench: A Benchmarking Toolkit for Thin-Client Based Virtual Desktop Environments , 2010, 2010 IEEE Second International Conference on Cloud Computing Technology and Science.

[13]  Lan Vu,et al.  Machine Learning Using Virtualized GPUs in Cloud Environments , 2017, ISC Workshops.

[14]  Shie-Yuan Wang,et al.  Benchmarking the Performance of XenDesktop Virtual DeskTop Infrastructure (VDI) Platform , 2015 .

[15]  Michael Seufert,et al.  Quality of experience in remote virtual desktop services , 2013, 2013 IFIP/IEEE International Symposium on Integrated Network Management (IM 2013).