Toward Testing for Multimodal Perception of Mating Signals

Many mating signals consist of multimodal components that need decoding by several sensory modalities on the receiver’s side. For methodological and conceptual reasons, the communicative functions of these signals are often investigated only one at a time. Likewise, variation of single signal traits are frequently correlated by researchers with senders’ quality or receivers’ behavioural responses. Consequently, the two classic and still dominating hypotheses regarding the communicative meaning of multimodal mating signals postulate that different components either serve as back-up messages or provide multiple meanings. Here we discuss how this conceptual dichotomy might have hampered a more integrative, perception encompassing understanding of multimodal communication: neither the multiple message nor the back-up signal hypotheses address the possibility that multimodal signals are integrated neurally into one percept. Therefore when studying multimodal mating signals, we should be aware that multimodal signals can give rise to multimodal percepts, meaning that receivers can gain access to additional information inherent to only combined signal components (‘the whole is something different than the sum of its parts’). We review the evidence for the importance of multimodal percepts and outline potential avenues for discovery of multimodal percepts in animal communication.

[1]  Sarah R Partan,et al.  Issues in the Classification of Multimodal Communication Signals , 2005, The American Naturalist.

[2]  D. Papaj,et al.  Complex signal function: developing a framework of testable hypotheses , 2004, Behavioral Ecology and Sociobiology.

[3]  C. Scharff,et al.  Waltzing Taeniopygia: integration of courtship song and dance in the domesticated Australian zebra finch , 2016, Animal Behaviour.

[4]  E. A. Whitchurch,et al.  Combined auditory and visual stimuli facilitate head saccades in the barn owl (Tyto alba). , 2006, Journal of neurophysiology.

[5]  A. Barron,et al.  A systems approach to animal communication , 2016, Proceedings of the Royal Society B: Biological Sciences.

[6]  A. Møller,et al.  Why have birds got multiple sexual ornaments? , 1993, Behavioral Ecology and Sociobiology.

[7]  Ryan C. Taylor,et al.  Crossmodal Comparisons of Signal Components Allow for Relative-Distance Assessment , 2014, Current Biology.

[8]  E. Hebets,et al.  Female mate choice for multimodal courtship and the importance of the signaling background for selection on male ornamentation , 2013 .

[9]  Melissa Bateson,et al.  Comparative evaluation and its implications for mate choice. , 2005, Trends in ecology & evolution.

[10]  M. Amorim,et al.  Noise Affects Multimodal Communication During Courtship in a Marine Fish , 2018, Front. Ecol. Evol..

[11]  P. Narins,et al.  Cross-modal integration in a dart-poison frog. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[12]  N. Estramil,et al.  Cichlids respond to conspecific sounds but females exhibit no phonotaxis without the presence of live males , 2014 .

[13]  U. Candolin The use of multiple cues in mate choice , 2003, Biological reviews of the Cambridge Philosophical Society.

[14]  C. Rowe Receiver psychology and the evolution of multicomponent signals , 1999, Animal Behaviour.

[15]  H. Williams Choreography of song, dance and beak movements in the zebra finch (Taeniopygia guttata). , 2001, The Journal of experimental biology.

[16]  P. Ręk,et al.  Deceptive vocal duets and multimodal display in a songbird , 2017, Proceedings of the Royal Society B: Biological Sciences.

[17]  L. S. Shestakov,et al.  Loser in Fight but Winner in Love: How Does Inter-Male Competition Determine the Pattern and Outcome of Courtship in Cricket Gryllus bimaculatus? , 2018, Front. Ecol. Evol..

[18]  M. Hall A Review of Vocal Duetting in Birds , 2009 .

[19]  P. Marler,et al.  Communication Goes Multimodal , 1999, Science.

[20]  M. Knörnschild,et al.  Songs, scents, and senses: sexual selection in the greater sac-winged bat, Saccopteryx bilineata , 2008 .

[21]  Rufus A. Johnstone,et al.  Multiple Displays in Animal Communication:`Backup Signals' and `Multiple Messages' , 1996 .

[22]  Tim Guilford,et al.  Hidden colour aversions in domestic chicks triggered by pyrazine odours of insect warning displays , 1996, Nature.

[23]  C. Spence Crossmodal correspondences: A tutorial review , 2011, Attention, perception & psychophysics.

[24]  J. Mappes,et al.  Multimodal Aposematic Signals and Their Emerging Role in Mate Attraction , 2018, Front. Ecol. Evol..

[25]  T. Stanford,et al.  Development of multisensory integration from the perspective of the individual neuron , 2014, Nature Reviews Neuroscience.

[26]  Daniel Robert,et al.  Detection and Learning of Floral Electric Fields by Bumblebees , 2013, Science.

[27]  M. Ryan,et al.  Interactions of Multisensory Components Perceptually Rescue Túngara Frog Mating Signals , 2013, Science.

[28]  T. Chapman,et al.  Males Use Multiple, Redundant Cues to Detect Mating Rivals , 2011, Current Biology.

[29]  Peter M. Narins,et al.  Clinal variation in anuran advertisement calls: basis for acoustic isolation? , 1986, Behavioral Ecology and Sociobiology.

[30]  J. Barber,et al.  Fireflies thwart bat attack with multisensory warnings , 2018, Science Advances.

[31]  M. Ryan,et al.  Multimodal cues improve prey localization under complex environmental conditions , 2015, Proceedings of the Royal Society B: Biological Sciences.

[32]  Gil G. Rosenthal,et al.  Technical and conceptual considerations for using animated stimuli in studies of animal behavior , 2016, Current zoology.

[33]  Asif A. Ghazanfar,et al.  Integration of Bimodal Looming Signals through Neuronal Coherence in the Temporal Lobe , 2008, Current Biology.

[34]  D. A. Mclennan The importance of olfactory signals in the gasterosteid mating system: sticklebacks go multimodal , 2003 .

[35]  J. Vroomen,et al.  Intersensory binding across space and time: A tutorial review , 2013, Attention, Perception, & Psychophysics.

[36]  Wouter Halfwerk,et al.  Pollution going multimodal: the complex impact of the human-altered sensory environment on animal perception and performance , 2015, Biology Letters.

[37]  G. Uetz,et al.  Cross-modal integration of multimodal courtship signals in a wolf spider , 2016, Animal Cognition.

[38]  G. Uetz,et al.  Multisensory Cues and Multimodal Communication in Spiders: Insights from Video/Audio Playback Studies , 2002, Brain, Behavior and Evolution.

[39]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[40]  S. Shimojo,et al.  Illusions: What you see is what you hear , 2000, Nature.

[41]  J. Ratcliffe,et al.  Multimodal warning signals for a multiple predator world , 2008, Nature.

[42]  W. Hödl,et al.  From uni- to multimodality: towards an integrative view on anuran communication , 2014, Journal of Comparative Physiology A.

[43]  Ryan C. Taylor,et al.  Interactions between complex multisensory signal components result in unexpected mate choice responses , 2017, Animal Behaviour.

[44]  Ryan C. Taylor,et al.  ‘Crazy love’: nonlinearity and irrationality in mate choice , 2019, Animal Behaviour.

[45]  Gesa Feenders,et al.  Temporal Ventriloquism Effect in European Starlings: Evidence for Two Parallel Processing Pathways , 2017, Behavioral neuroscience.

[46]  B. R. Smith,et al.  Multimodal communication and spatial binding in pied currawongs (Strepera graculina) , 2008, Animal Cognition.

[47]  B. Stein,et al.  Spatial determinants of multisensory integration in cat superior colliculus neurons. , 1996, Journal of neurophysiology.

[48]  Christoph Kayser,et al.  Natural asynchronies in audiovisual communication signals regulate neuronal multisensory interactions in voice-sensitive cortex , 2014, Proceedings of the National Academy of Sciences.

[49]  R. Peters,et al.  Dance Choreography Is Coordinated with Song Repertoire in a Complex Avian Display , 2013, Current Biology.