Big behavior: challenges and opportunities in a new era of deep behavior profiling

The assessment of rodent behavior forms a cornerstone of preclinical assessment in neuroscience research. Nonetheless, the true and almost limitless potential of behavioral analysis has been inaccessible to scientists until very recently. Now, in the age of machine vision and deep learning, it is possible to extract and quantify almost infinite numbers of behavioral variables, to break behaviors down into subcategories and even into small behavioral units, syllables or motifs. However, the rapidly growing field of behavioral neuroethology is experiencing birthing pains. The community has not yet consolidated its methods, and new algorithms transfer poorly between labs. Benchmarking experiments as well as the large, well-annotated behavior datasets required are missing. Meanwhile, big data problems have started arising and we currently lack platforms for sharing large datasets—akin to sequencing repositories in genomics. Additionally, the average behavioral research lab does not have access to the latest tools to extract and analyze behavior, as their implementation requires advanced computational skills. Even so, the field is brimming with excitement and boundless opportunity. This review aims to highlight the potential of recent developments in the field of behavioral analysis, whilst trying to guide a consensus on practical issues concerning data collection and data sharing.

[1]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[2]  Peter Seibt Algorithmic Information Theory: Mathematics of Digital Information Processing , 2010 .

[3]  A. Pérez-Escudero,et al.  idTracker: tracking individuals in a group by automatic identification of unmarked animals , 2014, Nature Methods.

[4]  D. Ringach,et al.  Animal Research in Neuroscience: A Duty to Engage , 2016, Neuron.

[5]  Stephen V. Stehman,et al.  Selecting and interpreting measures of thematic classification accuracy , 1997 .

[6]  Vittorio Murino,et al.  Automatic Visual Tracking and Social Behaviour Analysis with Multiple Mice , 2013, PloS one.

[7]  Marcel A. J. van Gerven,et al.  Deep Learning Improves Automated Rodent Behavior Recognition Within a Specific Experimental Setup , 2019, Journal of Neuroscience Methods.

[8]  C. Darwin The Expression of the Emotions in Man and Animals , .

[9]  Douglas Wahlsten,et al.  In search of a better mouse test , 2003, Trends in Neurosciences.

[10]  Kenji Satou,et al.  Applying Deep Learning Models to Mouse Behavior Recognition , 2019, Journal of Biomedical Science and Engineering.

[11]  Pascal Fua,et al.  DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila , 2019, eLife.

[12]  Nejc Dolensek,et al.  Facial expressions of emotion states and their neuronal correlates in mice , 2020, Science.

[13]  Samantha R White,et al.  The Future Is Open: Open-Source Tools for Behavioral Neuroscience Research , 2019, eNeuro.

[14]  Mikhail Kislin,et al.  Fast animal pose estimation using deep neural networks , 2018, Nature Methods.

[15]  Jacob M. Graving,et al.  DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning , 2019, bioRxiv.

[16]  Yijie Wang,et al.  High Performance Implementation of 3D Convolutional Neural Networks on a GPU , 2017, Comput. Intell. Neurosci..

[17]  Remco C. Veltkamp,et al.  Learning to recognize rat social behavior: Novel dataset and cross-dataset application , 2017, Journal of Neuroscience Methods.

[18]  Andrew D. Zaharia,et al.  The Detection of Visual Contrast in the Behaving Mouse , 2011, The Journal of Neuroscience.

[19]  Monitoring technologies and genetic engineering are producing a growing array of animal models for psychiatric disorders , .

[20]  David J. Anderson,et al.  Computational Neuroethology: A Call to Action , 2019, Neuron.

[21]  Laura J. Grundy,et al.  A dictionary of behavioral motifs reveals clusters of genes affecting Caenorhabditis elegans locomotion , 2012, Proceedings of the National Academy of Sciences.

[22]  C. S. Hall,et al.  Emotional behavior in the rat. I. Defecation and urination as measures of individual differences in emotionality. , 1934 .

[23]  Matthijs Verhage,et al.  A solution to dependency: using multilevel analysis to accommodate nested data , 2014, Nature Neuroscience.

[24]  Ugne Klibaite,et al.  An unsupervised method for quantifying the behavior of paired animals , 2016, Physical biology.

[25]  S. Hyman,et al.  Animal models of neuropsychiatric disorders , 2010, Nature Neuroscience.

[26]  W. H. Gispen,et al.  Classification of rat behavior with an image-processing method and a neural network , 2000, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[27]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[28]  H. Lipp,et al.  Automated dissection of permanent effects of hippocampal or prefrontal lesions on performance at spatial, working memory and circadian timing tasks of C57BL/6 mice in IntelliCage , 2017, Behavioural Brain Research.

[29]  Neir Eshel,et al.  Simple Behavioral Analysis (SimBA) – an open source toolkit for computer classification of complex social behaviors in experimental animals , 2020, bioRxiv.

[30]  Kartikeya Murari,et al.  Recurrent 3D Convolutional Network for Rodent Behavior Recognition , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[31]  David J. Anderson,et al.  Toward a Science of Computational Ethology , 2014, Neuron.

[32]  P. Germain,et al.  Exploratory rearing: a context- and stress-sensitive behavior recorded in the open-field test , 2018, Stress.

[33]  G. Bove,et al.  Mechanical sensory threshold testing using nylon monofilaments: The pain field’s “Tin Standard” , 2006, Pain.

[34]  J. Archer Tests for emotionality in rats and mice: a review. , 1973, Animal behaviour.

[35]  R. Kerr,et al.  Discovery of Brainwide Neural-Behavioral Maps via Multiscale Unsupervised Structure Learning , 2014, Science.

[36]  Benjamin F. Grewe,et al.  Deep learning based behavioral analysis enables high precision rodent tracking and is capable of outperforming commercial solutions , 2020, bioRxiv.

[37]  Gordon J. Berman,et al.  Measuring behavior across scales , 2017, BMC Biology.

[38]  B. Skinner,et al.  The Behavior of Organisms: An Experimental Analysis , 2016 .

[39]  Robert W. Williams,et al.  Reproducibility and replicability of rodent phenotyping in preclinical studies , 2016, Neuroscience & Biobehavioral Reviews.

[40]  Kevin M. Cury,et al.  DeepLabCut: markerless pose estimation of user-defined body parts with deep learning , 2018, Nature Neuroscience.

[41]  C. Braak,et al.  An automated system for the recognition of various specific rat behaviours , 2013, Journal of Neuroscience Methods.

[42]  Kristin Branson,et al.  JAABA: interactive machine learning for automatic annotation of animal behavior , 2013, Nature Methods.

[43]  W. Russell,et al.  Ethical and Scientific Considerations Regarding Animal Testing and Research , 2011, PloS one.

[44]  C. Belzung,et al.  The open field as a paradigm to measure the effects of drugs on anxiety-like behaviors: a review. , 2003, European journal of pharmacology.

[45]  Douglas Wahlsten,et al.  Experimenter effects on behavioral test scores of eight inbred mouse strains under the influence of ethanol , 2014, Behavioural Brain Research.

[46]  Greg J. Stephens,et al.  Dimensionality and Dynamics in the Behavior of C. elegans , 2007, PLoS Comput. Biol..

[47]  Thomas Serre,et al.  Automated home-cage behavioural phenotyping of mice. , 2010, Nature communications.

[48]  A. J Spink,et al.  The EthoVision video tracking system—A tool for behavioral phenotyping of transgenic mice , 2001, Physiology & Behavior.

[49]  A. Zaslaver,et al.  A multi-animal tracker for studying complex behaviors , 2017, BMC Biology.

[50]  Bastijn J. G. van den Boom,et al.  Automated classification of self-grooming in mice using open-source software , 2017, Journal of Neuroscience Methods.

[51]  I. Pavlov Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex , 1929 .

[52]  David J. Anderson,et al.  Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning , 2015, Proceedings of the National Academy of Sciences.

[53]  Michael B. Reiser,et al.  Mapping the Neural Substrates of Behavior , 2017, Cell.

[54]  Pietro Perona,et al.  Social behavior recognition in continuous video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Alex E. Lash,et al.  Gene Expression Omnibus: NCBI gene expression and hybridization array data repository , 2002, Nucleic Acids Res..

[56]  F. Guhad Introduction to the 3Rs (refinement, reduction and replacement). , 2005, Contemporary topics in laboratory animal science.

[57]  R N Walsh,et al.  The Open-Field Test: a critical review. , 1976, Psychological bulletin.

[58]  Matthias Bethge,et al.  Using DeepLabCut for 3D markerless pose estimation across species and behaviors , 2018 .

[59]  N. Sousa,et al.  PhenoWorld: a new paradigm to screen rodent behavior , 2014, Translational Psychiatry.

[60]  Philip G. Zimbardo,et al.  Effects of “Free-Environment” Rearing upon Exploratory Behavior , 1957 .

[61]  A. Ennaceur Tests of unconditioned anxiety — Pitfalls and disappointments , 2014, Physiology & Behavior.

[62]  J. Cryan,et al.  The age of anxiety: role of animal models of anxiolytic action in drug discovery , 2011, British journal of pharmacology.

[63]  Yoram Ben-Shaul,et al.  OptiMouse: a comprehensive open source program for reliable detection and analysis of mouse body and nose positions , 2017, BMC Biology.

[64]  Y. Benjamini,et al.  THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY , 2001 .

[65]  William Bialek,et al.  Mapping the stereotyped behaviour of freely moving fruit flies , 2013, Journal of The Royal Society Interface.

[66]  Wim E Crusio,et al.  QTL and systems genetics analysis of mouse grooming and behavioral responses to novelty in an open field , 2017, Genes, brain, and behavior.

[67]  Ryan P. Adams,et al.  Mapping Sub-Second Structure in Mouse Behavior , 2015, Neuron.

[68]  Eric A. Yttri,et al.  B-SOiD: An Open Source Unsupervised Algorithm for Discovery of Spontaneous Behaviors , 2019, bioRxiv.

[69]  Jean-Christophe Olivo-Marin,et al.  Real-time analysis of the behaviour of groups of mice via a depth-sensing camera and machine learning , 2019, Nature Biomedical Engineering.

[70]  Scott W. Linderman,et al.  The Striatum Organizes 3D Behavior via Moment-to-Moment Action Selection , 2018, Cell.

[71]  Vitor Augusto Garcia,et al.  Assessment of observers' stability and reliability — A tool for evaluation of intra- and inter-concordance in animal behavioral recordings , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[72]  Espen A. Sjoberg Logical fallacies in animal model research , 2017, Behavioral and Brain Functions.

[73]  Daniel T. Sangiamo,et al.  Ultrasonic signals associated with different types of social behavior of mice. , 2020, Nature Neuroscience.

[74]  Liang-Gee Chen,et al.  Predictive watershed: a fast watershed algorithm for video segmentation , 2003, IEEE Trans. Circuits Syst. Video Technol..

[75]  Pascal Fua,et al.  DeepFly3D: A deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila , 2019, bioRxiv.

[76]  Robert C. Wolpert,et al.  A Review of the , 1985 .

[77]  J. O’Keefe,et al.  Rearing on Hind Legs, Environmental Novelty, and the Hippocampal Formation , 2006, Reviews in the neurosciences.

[78]  I. Pavlov,et al.  Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex , 2010, Annals of Neurosciences.

[79]  Joseph J. Paton,et al.  Big behavioral data: psychology, ethology and the foundations of neuroscience , 2014, Nature Neuroscience.