A neural decoder for learned vocal behavior

Brain Machine Interfaces (BMIs) hold promise to restore impaired motor function and, because they decode neural signals to infer behavior, can serve as powerful tools to understand the neural mechanisms of motor control. Yet complex behaviors, such as vocal communication, exceed state-of-the-art decoding technologies which are currently restricted to comparatively simple motor actions. Here we present a BMI for birdsong, that decodes a complex, learned vocal behavior directly from neural activity.

[1]  F Goller,et al.  Dynamical origin of spectrally rich vocalizations in birdsong. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[2]  I. Titze.  The physics of small-amplitude oscillation of the vocal folds. , 1988, The Journal of the Acoustical Society of America.

[3]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[4]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[5]  Yonatan Sanz Perl,et al.  Reconstruction of physiological instructions from Zebra finch song. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[6]  S. S. Stevens.  A scale for the measurement of a psychological magnitude: loudness. , 1936 .

[7]  J. Wild,et al.  Neural pathways for the control of birdsong production. , 1997, Journal of neurobiology.

[8]  P. Kuhl,et al.  Birdsong and human speech: common themes and mechanisms. , 1999, Annual review of neuroscience.

[9]  Anish A. Sarma,et al.  Clinical translation of a high-performance neural prosthesis , 2015, Nature Medicine.

[10]  J. Spall.  STOCHASTIC OPTIMIZATION , 2002 .

[11]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[12]  Atsushi Iriki,et al.  Convergent transcriptional specializations in the brains of humans and song-learning birds , 2014, Science.

[13]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[14]  L. Nathan Perkins,et al.  Unstable neurons underlie a stable learned behavior , 2016, Nature Neuroscience.

[15]  J. Goldberg,et al.  Dopamine neurons encode performance error in singing birds , 2016, Science.

[16]  F. Goller,et al.  A new mechanism of sound generation in songbirds. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[17]  Jae Lim,et al.  Signal estimation from modified short-time Fourier transform , 1984 .

[18]  Andrew S. Whitford,et al.  Cortical control of a prosthetic arm for self-feeding , 2008, Nature.

[19]  N. Fletcher,et al.  Songbirds tune their vocal tract to the fundamental frequency of their song. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[20]  Matteo Carandini,et al.  Kilosort: realtime spike-sorting for extracellular electrophysiology with hundreds of channels , 2016, bioRxiv.

[21]  Jae S. Lim,et al.  Signal estimation from modified short-time Fourier transform , 1983, ICASSP.

[22]  Yonatan Sanz Perl,et al.  Elemental gesture dynamics are encoded by song premotor cortical neurons , 2013, Nature.

[23]  Douglas D. O'Shaughnessy,et al.  Speech communication : human and machine , 1987 .

[24]  P. Holmes,et al.  The developmental dynamics of marmoset monkey vocal production , 2015, Science.

[25]  Ezequiel M. Arneodo,et al.  Prosthetic Avian Vocal Organ Controlled by a Freely Behaving Bird Based on a Low Dimensional Model of the Biomechanical Periphery , 2012, PLoS Comput. Biol..

[26]  A. C. Yu,et al.  Temporal Hierarchical Control of Singing in Birds , 1996, Science.

[27]  Yonatan Sanz Perl,et al.  Automatic reconstruction of physiological gestures used in a model of birdsong production. , 2015, Journal of neurophysiology.

[28]  Sam E Benezra,et al.  Supplemental Information Population-level Representation of a Temporal Sequence Underlying Song Production in the Zebra Finch , 2022 .

[29]  A. Leonardo,et al.  Ensemble Coding of Vocal Control in Birdsong , 2005, The Journal of Neuroscience.

[30]  Alexander Hanuschkin,et al.  Rhythmic Continuous-Time Coding in the Songbird Analog of Vocal Motor Cortex , 2016, Neuron.