Online Study Reveals the Multimodal Effects of Discrete Auditory Cues in Moving Target Estimation Task

Tasks that require temporal accuracy and precision are common in interactive systems. One such task is the moving target selection task in which visual motion cues are used to estimate timing. Our online study investigates the multimodal effects of discrete auditory cues in such a task in which the input timing has to be estimated from visual targets thrown at different speeds and intervals (moving target estimation). The auditory cues provide timing anticipation of the interception from the interval between the beats of two beeps. This intervention is novel in that it enriches the visual motion information and independently gives timing anticipation. Furthermore, we show that such cues can decrease systematic errors in the moving target estimation task. As the effectiveness of auditory cues is demonstrated in online experiments, we can expect to apply such intervention to interactive systems in various environments.

[1]  Weiyi Pan,et al.  Effect of Rhythm of Music Therapy on Gait in Patients with Stroke. , 2020, Journal of stroke and cerebrovascular diseases : the official journal of National Stroke Association.

[2]  Hans Forssberg,et al.  Listening to rhythms activates motor and premotor cortices , 2009, Cortex.

[3]  Jin Huang,et al.  Modeling Error Rates in Spatiotemporal Moving Target Selection , 2019, CHI Extended Abstracts.

[4]  Eli Brenner,et al.  Errors in interception can be predicted from errors in perception , 2018, Cortex.

[5]  Daiichiro Kuroki,et al.  A new jsPsych plugin for psychophysics, providing accurate display duration and stimulus onset asynchrony , 2020, Behavior Research Methods.

[6]  Feng Tian,et al.  Modeling the Uncertainty in Pointing of Moving Targets with Arbitrary Shapes , 2020, CHI Extended Abstracts.

[7]  Byungjoo Lee,et al.  Geometrically Compensating Effect of End-to-End Latency in Moving-Target Selection Games , 2019, CHI.

[8]  R. Ulrich,et al.  Perceptual learning in temporal discrimination: asymmetric cross-modal transfer from audition to vision , 2012, Experimental Brain Research.

[9]  Ryota Kanai,et al.  Modality-independent role of the primary auditory cortex in time estimation , 2011, Experimental Brain Research.

[10]  Feng Tian,et al.  Modeling the Endpoint Uncertainty in Crossing-based Moving Target Selection , 2020, CHI.

[11]  R. Ivry,et al.  Moving outside the lab: The viability of conducting sensorimotor learning studies online , 2021, bioRxiv.

[12]  Antti Oulasvirta,et al.  Moving Target Selection: A Cue Integration Model , 2018, CHI.

[13]  Joshua de Leeuw,et al.  jsPsych: A JavaScript library for creating behavioral experiments in a Web browser , 2014, Behavior Research Methods.

[14]  David P. McGovern,et al.  Task-specific transfer of perceptual learning across sensory modalities , 2016, Current Biology.

[15]  Shiau-Chuen Chiou,et al.  Bimanual Coordination Learning with Different Augmented Feedback Modalities and Information Types , 2016, PloS one.

[16]  Aaron R. Seitz,et al.  Benefits of multisensory learning , 2008, Trends in Cognitive Sciences.

[17]  Alain Pitiot,et al.  The timing mega-study: comparing a range of experiment generators, both lab-based and online , 2020, PeerJ.

[18]  Aaron R. Seitz,et al.  Visual rhythm perception improves through auditory but not visual training , 2015, Current Biology.

[19]  Josef B. Spjut,et al.  Esports and High Performance HCI , 2021, CHI Extended Abstracts.

[20]  Robert J. Teather,et al.  Is 60 FPS better than 30?: the impact of frame rate and latency on moving target selection , 2014, CHI Extended Abstracts.

[21]  Feng Tian,et al.  Influence of multi-modality on moving target selection in virtual reality , 2019, Virtual Real. Intell. Hardw..

[22]  Mehrdad Jazayeri,et al.  Integration of speed and time for estimating time to contact , 2018, Proceedings of the National Academy of Sciences.

[23]  Eli Brenner,et al.  Continuously updating one's predictions underlies successful interception. , 2018, Journal of neurophysiology.