Less Users More Confidence: How AOIs Don't Affect Scanpath Trend Analysis

User studies are typically difficult, recruiting enough users is often problematic and each experiment takes a considerable amount of time to be completed. In these studies, eye tracking is increasingly used which often increases time, therefore, the lower the number of users required for these studies the better for making these kinds of studies more practical in terms of economics and time expended. The possibility of achieving almost the same results with fewer users has already been raised. Specifically, the possibility of achieving 75% similarity to the results of 65 users with 27 users for searching tasks and 34 users for browsing tasks has been observed in scanpath trend analysis which discovers the most commonly followed path on a particular web page in terms of its visual elements or areas of interest (AOIs). Different approaches are available to segment or divide web pages into their visual elements or AOIs. In this paper, we investigate whether the possibility raised by the previous work is restricted to a particular page segmentation approach by replicating the experiments with two other segmentation approaches. The results are consistent with ~5% difference for the searching tasks and ~10% difference for the browsing tasks.

[1]  James R. Lewis Testing Small System Customer Set-Up , 1982 .

[2]  Jakob Nielsen,et al.  A mathematical model of the finding of usability problems , 1993, INTERCHI.

[3]  Carolyn Snyder,et al.  Web Site Usability: A Designer's Guide , 1997 .

[4]  Relaxing the homogeneity assumption in usability testing , 2001, Behav. Inf. Technol..

[5]  Jared M. Spool,et al.  Testing web sites: five users is nowhere near enough , 2001, CHI Extended Abstracts.

[6]  Michael E. Holmes,et al.  Visual attention to repeated internet images: testing the scanpath theory on the world wide web , 2002, ETRA.

[7]  L. Faulkner Beyond the five-user assumption: Benefits of increased sample sizes in usability testing , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[8]  Jakob Nielsen,et al.  The "magic number 5": is it enough for web testing? , 2002, CHI Extended Abstracts.

[9]  Eleni Michailidou,et al.  ViCRAM: visual complexity rankings and accessibility metrics , 2006, ASAC.

[10]  Haruhiko Takeuchi,et al.  A Quantitative Method for Analyzing Scan Path Data Obtained by Eye Tracker , 2007, 2007 IEEE Symposium on Computational Intelligence and Data Mining.

[11]  Stephanie Wilson,et al.  Identifying web usability problems from eye-tracking data , 2007, BCS HCI.

[12]  Sean Bechhofer,et al.  Visual complexity and aesthetic perception of web pages , 2008, SIGDOC '08.

[13]  Geoffrey M. Underwood,et al.  Knowledge-Based Patterns of Remembering: Eye Movement Scanpaths Reflect Domain Experience , 2008, USAB.

[14]  Caroline Jay,et al.  End User Evaluations , 2008, Web Accessibility.

[15]  Gavriel Salvendy,et al.  Number of people required for usability evaluation , 2010, Commun. ACM.

[16]  Andy Brown,et al.  Audio access to calendars , 2010, W4A.

[17]  Jeffrey Johnson,et al.  Designing with the Mind in Mind: Simple Guide to Understanding User Interface Design Rules , 2010 .

[18]  Peter Fankhauser,et al.  Boilerplate detection using shallow text features , 2010, WSDM '10.

[19]  Barbara S. Chaparro,et al.  Text Advertising Blindness: The New Banner Blindness? , 2011 .

[20]  Martin Schmettow,et al.  Sample size in usability studies , 2012, Commun. ACM.

[21]  N. Hari Narayanan,et al.  Visual attention patterns during program debugging with an IDE , 2012, ETRA '12.

[22]  M. Elgin Akpinar,et al.  Vision Based Page Segmentation Algorithm: Extended and Perceived Success , 2013, ICWE Workshops.

[23]  Yeliz Yesilada,et al.  Experiential transcoding: an EyeTracking approach , 2013, W4A.

[24]  Stéphane Gançarski,et al.  Block-o-Matic: a Web Page Segmentation Tool and its Evaluation , 2013 .

[25]  Pam J. Mayhew,et al.  How many participants are really enough for usability studies? , 2014, 2014 Science and Information Conference.

[26]  Andres Sanoja,et al.  Block-o-Matic: A web page segmentation framework , 2014, 2014 International Conference on Multimedia Computing and Systems (ICMCS).

[27]  Jer Lang Hong,et al.  Webpage Segmentation Using Ontology and Word Matching , 2014, ICONIP.

[28]  Yeliz Yesilada,et al.  Web Page Segmentation: A Review , 2014 .

[29]  Lidong Bing,et al.  Web page segmentation with structured prediction and its application in web page classification , 2014, SIGIR.

[30]  Tingting Wei,et al.  Web page segmentation based on the hough transform and vision cues , 2015, 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA).

[31]  Dragan Gasevic,et al.  Automated classification and localization of daily deal content from the Web , 2015, Appl. Soft Comput..

[32]  Jurriaan Hage,et al.  A Quantitative Comparison of Semantic Web Page Segmentation Approaches , 2015, ICWE.

[33]  Yeliz Yesilada,et al.  Eye tracking scanpath analysis techniques on web pages: A survey, evaluation and comparison , 2015 .

[34]  M. Elgin Akpinar,et al.  "Old habits die hard!": eyetracking based experiential transcoding: a study with mobile users , 2015, W4A.

[35]  Yeliz Yesilada,et al.  Scanpath Trend Analysis on Web Pages , 2016, ACM Trans. Web.

[36]  Wenzhe Zhang,et al.  Web Page Segmentation and Its Application for Web Information Crawling , 2016, 2016 IEEE 28th International Conference on Tools with Artificial Intelligence (ICTAI).

[37]  Yeliz Yesilada,et al.  Eye tracking scanpath analysis on web pages: how many users? , 2016, ETRA.

[38]  Christian S. Collberg,et al.  Repeatability in computer systems research , 2016, Commun. ACM.

[39]  Esther Parra,et al.  Estimating sample size for usability testing , 2017 .

[40]  Jaroslav Zendulka,et al.  Box clustering segmentation: A new method for vision-based web page preprocessing , 2017, Inf. Process. Manag..

[41]  Christus,et al.  A General Method Applicable to the Search for Similarities in the Amino Acid Sequence of Two Proteins , 2022 .