Assessment of information content in visual signal: analysis of optical flow fractal complexity

ABSTRACT We make a first attempt at distinguishing an information-carrying visual signal by comparing visual characteristics of American Sign Language to everyday human motion, to identify what clues might be available in one but not in the other. The comparison indicated significantly higher fractal complexity in sign language across tested frequency bands (0.01–15 Hz), as compared to everyday motion. A comparison of our results with other work showing high fractal complexity in the speech signal allows us to suggest the underlying properties of linguistic signals which allow babies to “tune into” a specific channel, or modality, during language acquisition.

[1]  Ronnie B. Wilbur,et al.  Phonological and prosodic layering of nonmanuals in American Sign Language. , 2000 .

[2]  Bruce J. West,et al.  The Living Matter Way to exchange information. , 2010, Medical hypotheses.

[3]  Evie Malaia,et al.  Kinematic Signatures of Telic and Atelic Events in ASL Predicates , 2012, Language and speech.

[4]  Evguenia Malaia,et al.  It Still Isn't Over: Event Boundaries in Language and Perception , 2014, Lang. Linguistics Compass.

[5]  Evie Malaia,et al.  Kinematic parameters of signed verbs. , 2013, Journal of speech, language, and hearing research : JSLHR.

[6]  Jordan Fenlon,et al.  Seeing sentence boundaries , 2007 .

[7]  J. A. Wise,et al.  Perceptual and physiological responses to the visual complexity of fractal patterns. , 2005, Nonlinear dynamics, psychology, and life sciences.

[8]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[9]  Frank E. Pollick,et al.  Event Segmentation and Biological Motion Perception in Watching Dance , 2014 .

[10]  Philippe Schlenker,et al.  Event representations constrain the structure of language: Sign language as a window into universally accessible linguistic biases , 2015, Proceedings of the National Academy of Sciences.

[11]  K. Linkenkaer-Hansen,et al.  Long-Range Temporal Correlations and Scaling Behavior in Human Brain Oscillations , 2001, The Journal of Neuroscience.

[12]  N. C. Singh,et al.  Modulation spectra of natural sounds and ethological theories of auditory processing. , 2003, The Journal of the Acoustical Society of America.

[13]  Yuguo Yu,et al.  Preference of sensory neural coding for 1/f signals. , 2005, Physical review letters.

[14]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[15]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[16]  J. Zacks,et al.  Using movement and intentions to understand human activity , 2009, Cognition.

[17]  R. J. Baken.  Irregularity of vocal period and amplitude: A first approach to the fractal analysis of voice , 1990 .

[18]  Evguenia Malaia,et al.  Analysis of ASL Motion Capture Data towards Identification of Verb Type , 2008, STEP.

[19]  Evguenia Malaia,et al.  Event segmentation in a visual language: Neural bases of processing American Sign Language predicates , 2012, NeuroImage.

[20]  David Ostry,et al.  Language rhythms in baby hand movements , 2001, Nature.

[21]  Evguenia Malaia,et al.  Contributions of Sign Language Research to Gesture Understanding: What Can Multimodal Computational Systems Learn from Sign Language Research , 2008, Int. J. Semantic Comput..

[22]  Yuen Ren Chao,et al.  Human Behavior and the Principle of Least Effort: An Introduction to Human Ecology , 1950 .

[23]  Antti Oulasvirta,et al.  Information capacity of full-body movements , 2013, CHI.

[24]  Alejandrina Cristia,et al.  Why the body comes first: effects of experimenter touch on infants' word finding. , 2015, Developmental science.

[25]  George Kingsley Zipf,et al.  Human Behaviour and the Principle of Least Effort: an Introduction to Human Ecology , 2012 .

[26]  Diane Brentari,et al.  A Prosodic Model of Sign Language Phonology , 1999 .

[27]  Michael D. Tyler,et al.  The Edge Factor in Early Word Segmentation: Utterance-Level Prosody Enables Word Form Extraction by 6-Month-Olds , 2014, PloS one.