Analysis of review quality by using gaze data during document review

In software development, deliverables in an upstream process are reviewed to ensure their quality and to reduce error propagation to the downstream process. Methods are available for evaluating the review quality. In this study, we considered the defect detection process in a review of Requirement Definition Documents and tested a potential relationship between the gaze patterns and review quality. Specifically, we analyzed the relationship between the gaze patterns, with a primary focus on the blink rate, in a review of RDDs and detection accuracy. A significant nonlinear correlation between the blink rate and the detection accuracy was observed; moreover, the subsequent regression analysis also verified the blink rate as the best predictor of the review quality, notwithstanding the use of other gaze patterns. This result indicates that the blink rate is a major predictor of a type of review performance.

[1]  Kai Kunze,et al.  Towards Extraction of Subjective Reading Incomprehension: Analysis of Eye Gaze Features , 2015, CHI Extended Abstracts.

[2]  Paul P. Maglio,et al.  A robust algorithm for reading detection , 2001, PUI '01.

[3]  Michael Mitzenmacher,et al.  Detecting Novel Associations in Large Data Sets , 2011, Science.

[4]  Sidney K. D'Mello,et al.  Automatic Gaze-Based Detection of Mind Wandering with Metacognitive Awareness , 2015, UMAP.

[5]  F. E. Grubbs Procedures for Detecting Outlying Observations in Samples , 1969 .

[6]  Akito Monden,et al.  Exploiting Eye Movements for Evaluating Reviewer's Performance in Software Review , 2007, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[7]  A. Bentivoglio,et al.  Analysis of blink rate patterns in normal subjects , 1997, Movement disorders : official journal of the Movement Disorder Society.

[8]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[9]  Kai Kunze,et al.  Estimation of english skill with a mobile eye tracker , 2016, UbiComp Adjunct.

[10]  J. Kleinman,et al.  Speaking, thinking, and blinking , 1981, Psychiatry Research.

[11]  Yann-Gaël Guéhéneuc,et al.  A systematic literature review on the usage of eye-tracking in software engineering , 2015, Inf. Softw. Technol..

[12]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[13]  P Cho,et al.  Baseline blink rates and the effect of visual task difficulty and position of gaze. , 2000, Current eye research.

[14]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[15]  Hans Gruber,et al.  Eye Tracking Metrics in Software Engineering , 2018, ECSEE.