EthoLoop: automated closed-loop neuroethology in naturalistic environments

Accurate tracking and analysis of animal behavior is crucial for modern systems neuroscience. However, following freely moving animals in naturalistic, three-dimensional (3D) or nocturnal environments remains a major challenge. Here, we present EthoLoop, a framework for studying the neuroethology of freely roaming animals. Combining real-time optical tracking and behavioral analysis with remote-controlled stimulus-reward boxes, this system allows direct interactions with animals in their habitat. EthoLoop continuously provides close-up views of the tracked individuals and thus allows high-resolution behavioral analysis using deep-learning methods. The behaviors detected on the fly can be automatically reinforced either by classical conditioning or by optogenetic stimulation via wirelessly controlled portable devices. Finally, by combining 3D tracking with wireless neurophysiology we demonstrate the existence of place-cell-like activity in the hippocampus of freely moving primates. Taken together, we show that the EthoLoop framework enables interactive, well-controlled and reproducible neuroethological studies in large-field naturalistic settings.

[1]  B L McNaughton,et al.  Dynamics of the hippocampal ensemble code for space. , 1993, Science.

[2]  Antonio Iera,et al.  The Internet of Things: A survey , 2010, Comput. Networks.

[3]  D. Albe-Fessard,et al.  A stereotaxic atlas of the grey lesser mouse lemur brain (Microcebus murinus) , 1998, Brain Research Bulletin.

[4]  D. Robert,et al.  Tracking of flying insects using pan-tilt cameras , 2000, Journal of Neuroscience Methods.

[5]  Jean-Christophe Olivo-Marin,et al.  Real-time analysis of the behaviour of groups of mice via a depth-sensing camera and machine learning , 2019, Nature Biomedical Engineering.

[6]  Christopher Zach,et al.  Robust Bundle Adjustment Revisited , 2014, ECCV.

[7]  Michael H. Dickinson,et al.  TrackFly: Virtual reality for a behavioral system analysis in free-flying fruit flies , 2008, Journal of Neuroscience Methods.

[8]  Lin Tian,et al.  Functional imaging of hippocampal place cells at cellular resolution during virtual navigation , 2010, Nature Neuroscience.

[9]  Daniel A. Dombeck,et al.  An olfactory virtual reality system for mice , 2018, Nature Communications.

[10]  Karel Svoboda,et al.  Natural Whisker-Guided Behavior by Head-Fixed Mice in Tactile Virtual Reality , 2014, The Journal of Neuroscience.

[11]  D. Stork Optics and realism in Renaissance art. , 2004, Scientific American.

[12]  Jakob N. Foerster,et al.  Three-dimensional head-direction coding in the bat brain , 2014, Nature.

[13]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Iain D. Couzin,et al.  Virtual Reality for Freely Moving Animals , 2017, Nature Methods.

[15]  J. Duhamel,et al.  A real-time 3D video tracking system for monitoring primate groups , 2014, Journal of Neuroscience Methods.

[16]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[17]  M. Silcox,et al.  Major Questions in the Study of Primate Origins , 2017 .

[18]  Mayank R Mehta,et al.  Impaired spatial selectivity and intact phase precession in two-dimensional virtual reality , 2014, Nature Neuroscience.

[19]  Nachum Ulanovsky,et al.  Neuroscience: How Is Three-Dimensional Space Encoded in the Brain? , 2011, Current Biology.

[20]  M. Perret Change in Photoperiodic Cycle Affects Life Span in a Prosimian Primate (Microcebus murinus , 1997, Journal of biological rhythms.

[21]  Tansu Celikel,et al.  Real-time contextual feedback for close-loop control of navigation. , 2019, Journal of neural engineering.

[22]  David J. Anderson,et al.  Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning , 2015, Proceedings of the National Academy of Sciences.

[23]  Tucker Balch,et al.  An outdoor 3-D visual tracking system for the study of spatial navigation and memory in rhesus monkeys , 2005, Behavior research methods.

[24]  Philipp Häfliger,et al.  Open source modules for tracking animal behavior and closed-loop stimulation based on Open Ephys and Bonsai , 2018, bioRxiv.

[25]  Dmitriy Aronov,et al.  Engagement of Neural Circuits Underlying 2D Spatial Navigation in a Rodent Virtual Reality System , 2014, Neuron.

[26]  Ryan P. Adams,et al.  Mapping Sub-Second Structure in Mouse Behavior , 2015, Neuron.

[27]  Zengcai V. Guo,et al.  Procedures for Behavioral Experiments in Head-Fixed Mice , 2014, PloS one.

[28]  Georg B. Keller,et al.  Sensorimotor Mismatch Signals in Primary Visual Cortex of the Behaving Mouse , 2012, Neuron.

[29]  R. Strauss,et al.  Processing of artificial visual feedback in the walking fruit fly Drosophila melanogaster. , 1997, The Journal of experimental biology.

[30]  Nachum Ulanovsky,et al.  Representation of Three-Dimensional Space in the Hippocampus of Flying Bats , 2013, Science.

[31]  Elad Schneidman,et al.  Correction: High-order social interactions in groups of mice , 2014, eLife.

[32]  Kevin M. Cury,et al.  DeepLabCut: markerless pose estimation of user-defined body parts with deep learning , 2018, Nature Neuroscience.

[33]  Nachum Ulanovsky,et al.  Large-scale navigational map in a mammal , 2011, Proceedings of the National Academy of Sciences.

[34]  Salma Bougacha,et al.  Digital templates and brain atlas dataset for the mouse lemur primate , 2018, Data in brief.

[35]  B. Dickson,et al.  FlyMAD: rapid thermogenetic control of neuronal activity in freely walking Drosophila , 2014, Nature Methods.

[36]  L. Nadel,et al.  The Hippocampus as a Cognitive Map , 1978 .

[37]  Kate Jeffery,et al.  Horizontal biases in rats’ use of three-dimensional space , 2011, Behavioural Brain Research.

[38]  K. Deisseroth,et al.  Phasic Firing in Dopaminergic Neurons Is Sufficient for Behavioral Conditioning , 2009, Science.

[39]  J. Krebs,et al.  Behavioural Ecology: An Evolutionary Approach , 1978 .

[40]  Mikhail Kislin,et al.  Fast animal pose estimation using deep neural networks , 2018, Nature Methods.

[41]  A Schnee,et al.  Rats are able to navigate in virtual environments , 2005, Journal of Experimental Biology.

[42]  A. Pérez-Escudero,et al.  idTracker: tracking individuals in a group by automatic identification of unmarked animals , 2014, Nature Methods.

[43]  A. Cressant,et al.  Computerized video analysis of social interactions in mice , 2012, Nature Methods.

[44]  Michael H. Dickinson,et al.  Multi-camera real-time three-dimensional tracking of multiple flying animals , 2010, Journal of The Royal Society Interface.

[45]  C. Harvey,et al.  Neuroscience: Virtual reality explored , 2016, Nature.

[46]  T. Ono,et al.  A 3D-Video-Based Computerized Analysis of Social and Sexual Interactions in Rats , 2013, PloS one.

[47]  D. Tank,et al.  Imaging Large-Scale Neural Activity with Cellular Resolution in Awake, Mobile Mice , 2007, Neuron.

[48]  Mattia G. Bergomi,et al.  idtracker.ai: tracking all individuals in small or large collectives of unmarked animals , 2019, Nature Methods.

[49]  R. Jacobs,et al.  Three-dimensional digital mouse atlas using high-resolution MRI. , 2001, Developmental biology.

[50]  M. Théry,et al.  Increased Late Night Response to Light Controls the Circadian Pacemaker in a Nocturnal Primate , 2010, Journal of biological rhythms.

[51]  Joel s. Brown,et al.  Foraging : behavior and ecology , 2007 .

[52]  K. Breland,et al.  A field of applied animal psychology. , 1951, The American psychologist.

[53]  Weiwei Chen,et al.  Virtual Reality system for freely-moving rodents , 2017, bioRxiv.

[54]  Marie-Claude Grobéty,et al.  Spatial learning in a three-dimensional maze , 1992, Animal Behaviour.

[55]  R. Hartley Triangulation, Computer Vision and Image Understanding , 1997 .

[56]  M. A. MacIver,et al.  Neuroscience Needs Behavior: Correcting a Reductionist Bias , 2017, Neuron.

[57]  O. Feinerman,et al.  Automated long-term tracking and social behavioural phenotyping of animal colonies within a semi-natural environment , 2013, Nature Communications.

[58]  D. Tank,et al.  Intracellular dynamics of hippocampal place cells during virtual navigation , 2009, Nature.