Design and Application of the BiVib Audio-Tactile Piano Sample Library

A library of piano samples composed of binaural recordings and keyboard vibrations has been built, with the aim of sharing accurate data that in recent years have successfully advanced the knowledge on several aspects about the musical keyboard and its multimodal feedback to the performer. All samples were recorded using calibrated measurement equipment on two Yamaha Disklavier pianos, one grand and one upright model. This paper documents the sample acquisition procedure, with related calibration data. Then, for sound and vibration analysis, it is shown how physical quantities such as sound intensity and vibration acceleration can be inferred from the recorded samples. Finally, the paper describes how the samples can be used to correctly reproduce binaural sound and keyboard vibrations. The library has potential to support experimental research about the psycho-physical, cognitive and experiential effects caused by the keyboard’s multimodal feedback in musicians and other users, or, outside the laboratory, to enable an immersive personal piano performance.

[1]  Stefano Papetti,et al.  Vibrotactile Sensitivity in Active Touch: Effect of Pressing Force , 2017, IEEE Transactions on Haptics.

[2]  Hareo Hamada,et al.  Fast deconvolution of multichannel systems using regularization , 1998, IEEE Trans. Speech Audio Process..

[3]  Gaurav Sharma,et al.  Creating a Multitrack Classical Music Performance Dataset for Multimodal Music Analysis: Challenges, Insights, and Applications , 2016, IEEE Transactions on Multimedia.

[4]  Marcelo M. Wanderley,et al.  Enriched Multimodal Representations of Music Performances: Online Access and Visualization , 2017, IEEE MultiMedia.

[5]  Hideo Suzuki.  Vibration and sound radiation of a piano soundboard , 1985 .

[6]  Federico Fontana,et al.  Detection of keyboard vibrations and effects on perceived piano quality. , 2017, The Journal of the Acoustical Society of America.

[7]  Erik V. Jansson,et al.  On Vibration Sensation and Finger Touch in Stringed Instrument Playing , 1992 .

[8]  Thomas L. Rhea,et al.  Evolution of the Keyboard Interface: The Bosendorfer 290 SE Recording Piano and the Moog Multiply-Touch-Sensitive Keyboards , 1990 .

[9]  Stefano Papetti,et al.  Multi-point vibrotactile feedback for an expressive musical interface , 2015, NIME.

[10]  Siddharth Kalra,et al.  Infrared Powered Vibro-Haptic Piano Training System for the Visually Impaired , 2017 .

[11]  Anders Askenfelt.  Speech , Music and Hearing Quarterly Progress and Status Report Observations on the transient components of the piano tone , 2007 .

[12]  Jacques Poitevineau,et al.  Influence of vibrotactile feedback on some perceptual features of violins. , 2014, The Journal of the Acoustical Society of America.

[13]  D. Murray Campbell.  Evaluating musical instruments , 2014 .

[14]  Karlheinz Blankenbach.  Spatial Effects , 2012, Handbook of Visual Display Technology.

[15]  Gordon Cheng,et al.  Effects of short-term piano training on measures of finger tapping, somatosensory perception and motor-related brain activity in patients with cerebral palsy , 2017, Neuropsychiatric disease and treatment.

[16]  A. Pick,et al.  How is a trumpet known? The "basic object level" concept and perception of musical instruments. , 1989, The American journal of psychology.

[17]  W. Hairston,et al.  Aspects of Multisensory Perception: The Integration of Visual and Auditory Information in Musical Experiences , 2005, Annals of the New York Academy of Sciences.

[18]  R. T. Verrillo,et al.  Vibration Sensation in Humans , 1992 .

[19]  M. Keane,et al.  Subjective Assessment of Upright Piano Key Vibrations , 2011 .

[20]  Joëlle Tilmanne,et al.  Airpiano: a multi-touch keyboard with hovering control , 2015, NIME.

[21]  M. Eimer,et al.  Tactile enhancement of auditory detection and perceived loudness , 2007, Brain Research.

[22]  E. O. Turner,et al.  Touch and Tone-Quality: The Pianist's Illusion , 1939 .

[23]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[24]  N. Logothetis,et al.  Integration of Touch and Sound in Auditory Cortex , 2005, Neuron.

[25]  Michael S. Beauchamp,et al.  Sound enhances touch perception , 2009, Experimental Brain Research.

[26]  Edgar Berdahl,et al.  Advancements in Actuated Musical Instruments , 2011, Organised Sound.

[27]  Roberto Bresin,et al.  Measurement and reproduction accuracy of computer-controlled grand pianos. , 2003, The Journal of the Acoustical Society of America.