Low-Latency Audio Pitch Tracking: A Multi-Modal Sensor-Assisted Approach

This paper presents a multi-modal approach to musical instrument pitch tracking combining audio and position sensor data. Finger location on a violin fingerboard is measured using resistive sensors, allowing rapid detection of approximate pitch. The initial pitch estimate is then used to restrict the search space of an audio pitch tracking algorithm. Most audio pitch tracking algorithms face a fundamental tradeoff between accuracy and latency, with longer analysis windows producing better pitch estimates at the cost of noticeable lag in a live performance environment. Conversely, sensor-only strategies struggle to achieve the fine pitch accuracy a human listener would expect. While this paper is violin centric, it demonstrates a more general concept for augmented instruments that by combining the two differing approaches, high accuracy and low latency pitch can be simultaneiously achieved.