Web based multi-platform benchmark program construction in smartphone

A benchmark program provides objective and quantitative experimental data that is used in HW architecture design and as performance criteria. For that reason, a benchmark point must be fairly calculated to make a right decision. It is difficult to benchmark with fairness in embedded systems such as smartphones since there are many HW and SW development environments. For example, iOS adopts ObjectC language and Android supports Java and C/C++ to develop applications. Therefore, it is impossible to apply the same native workload without modification to both iOS and Android platforms. Web-based benchmark programs can solve the problem of fairness. If a platform supports a browsing system, then we can run the same web benchmark program on that platform. In this paper, we construct a web-based benchmark program with JavaScript and HTML5 to evaluate an entire HW platform. The JavaScript workload has three components and the HTML5 workload has two components. Experiment shows that similar runtime characteristics are maintained between native and web workloads. Moreover, we use a compared ratio method to relieve result distortion that comes from unexpected system overheads. Finally, we run our benchmark program on the iOS and many Android platforms.

[1]  Steve Souders.  High-performance web sites , 2008, CACM.

[2]  Jesse James Garrett.  Ajax: A New Approach to Web Applications , 2007 .

[3]  John R. Mashey,et al.  War of the benchmark means: time for a truce , 2004, CARN.

[4]  W. Meira,et al.  The USAR characterization model , 2004, IEEE International Workshop on Workload Characterization, 2004. WWC-7. 2004.

[5]  Andre Charland,et al.  Mobile application development , 2011, Commun. ACM.

[6]  Thomas M. Conte,et al.  A Benchmark Characterization of the EEMBC Benchmark Suite , 2009, IEEE Micro.

[7]  Aaron Weiss,et al.  WebOS: say goodbye to desktop applications , 2005, NTWK.

[8]  Cloyce D. Spradling.  SPEC CPU2006 benchmark tools , 2007, CARN.

[9]  Bo Huang,et al.  Browser workload characterization for an Ajax-based commercial online service , 2009, 2009 IEEE International Symposium on Workload Characterization (IISWC).

[10]  Todd M. Austin,et al.  The SimpleScalar tool set, version 2.0 , 1997, CARN.

[11]  Jack J. Dongarra,et al.  The LINPACK Benchmark: past, present and future , 2003, Concurr. Comput. Pract. Exp..

[12]  Trevor Mudge,et al.  MiBench: A free, commercially representative embedded benchmark suite , 2001 .

[13]  Miodrag Potkonjak,et al.  MediaBench: a tool for evaluating and synthesizing multimedia and communications systems , 1997, Proceedings of 30th Annual International Symposium on Microarchitecture.

[14]  Melody Y. Ivory,et al.  Evolution of web site design patterns , 2005, TOIS.

[15]  Alexandra Fedorova,et al.  Base Vectors : A Potential Technique for Micro-architectural Classification of Applications , 2007 .

[16]  JoAnn M. Paul,et al.  Webpage-based benchmarks for mobile device design , 2008, 2008 Asia and South Pacific Design Automation Conference.

[17]  Lizy Kurian John,et al.  Improved automatic testcase synthesis for performance model validation , 2005, ICS '05.

[18]  Donald E. Thomas,et al.  Scenario-oriented design for single-chip heterogeneous multiprocessors , 2006, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[19]  Daniel Citron,et al.  The harmonic or geometric mean: does it really matter? , 2006, CARN.