Shape-and-Behavior Encoded Tracking of Bee Dances

Behavior analysis of social insects has garnered impetus in recent years and has led to some advances in fields like control systems and flight navigation. Manual labeling of insect motions required for analyzing the behaviors of insects requires significant investment of time and effort. In this paper, we propose certain general principles that help in simultaneous automatic tracking and behavior analysis, with applications in tracking bees and recognizing specific behaviors that they exhibit. The state space for tracking is defined using the position, orientation, and current behavior of the insect being tracked. The position and the orientation are parameterized using a shape model, whereas the behavior is explicitly modeled using a three-tier hierarchical motion model. The first tier (dynamics) models the local motions exhibited, and the models built in this tier act as a vocabulary for behavior modeling. The second tier is a Markov motion model built on top of the local motion vocabulary, which serves as the behavior model. The third tier of the hierarchy models the switching between behaviors, and this is also modeled as a Markov model. We address issues in learning the three-tier behavioral model, in discriminating between models, and in detecting and modeling abnormal behaviors. Another important aspect of this work is that it leads to joint tracking and behavior analysis instead of the traditional "track-and-then-recognize" approach. We apply these principles for tracking bees in a hive while they are executing the waggle dance and the round dance.

[1]  Hsi-Jian Lee,et al.  Determination of 3D human body postures from a single view , 1985, Comput. Vis. Graph. Image Process..

[2]  L. R. Rabiner,et al.  A probabilistic distance measure for hidden Markov models , 1985, AT&T Technical Journal.

[3]  R. Chellappa,et al.  Recursive 3-D motion estimation from a monocular image sequence , 1990 .

[4]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[5]  R. Morse.  The Dance Language and Orientation of Bees , 1994 .

[6]  Fabrizio Mura,et al.  Visual control of altitude and speed in a flying agent , 1994 .

[7]  Zhang,et al.  Honeybee navigation en route to the goal: visual flight control and odometry , 1996, The Journal of experimental biology.

[8]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[9]  Christoph Bregler,et al.  Learning and recognizing human dynamics in video sequences , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Jun S. Liu,et al.  Sequential Monte Carlo methods for dynamic systems , 1997 .

[11]  Michael J. Black,et al.  A probabilistic frameworkfor matching temporal trajectories , 1998, ICCV 1998.

[12]  H. Damasio,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence: Special Issue on Perceptual Organization in Computer Vision , 1998 .

[13]  Michael Isard,et al.  Learning Multi-Class Dynamics , 1998, NIPS.

[14]  Michael Isard,et al.  A mixed-state condensation tracker with automatic model-switching , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[15]  Gregory D. Hager,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Vladimir Pavlovic,et al.  A dynamic Bayesian network approach to figure tracking using learned dynamic models , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[17]  José M. F. Moura,et al.  Capture and Representation of Human Walking in Live Video Sequences , 1999, IEEE Trans. Multim..

[18]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[19]  Michael Isard,et al.  Learning and Classification of Complex Dynamics , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Cristian Sminchisescu,et al.  Covariance scaled sampling for monocular 3D body tracking , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[21]  Heinrich H. Bülthoff,et al.  Insect Inspired Visual Control of Translatory Flight , 2001, ECAL.

[22]  Panos J. Antsaklis,et al.  Hierarchical Control of Piecewise Linear Hybrid Dynamical Systems Based on Discrete Abstractions , 2001 .

[23]  Ramakant Nevatia,et al.  3D tracking of human locomotion: a tracking as recognition approach , 2002, Object recognition supported by user interaction for service robots.

[24]  H. Shum,et al.  Learning A Highly Structured Motion Model for 3D Human Tracking , 2002 .

[25]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[26]  F. Dellaert,et al.  A Rao-Blackwellized particle filter for EigenTracking , 2004, CVPR 2004.

[27]  Rama Chellappa,et al.  Visual tracking and recognition using appearance-adaptive models in particle filters , 2004, IEEE Transactions on Image Processing.

[28]  N. Vaswani,et al.  Change detection in partially observed nonlinear dynamic systems with unknown change parameters , 2004, Proceedings of the 2004 American Control Conference.

[29]  T. Seeley.  The tremble dance of the honey bee: message and meanings , 1992, Behavioral Ecology and Sociobiology.

[30]  Yoram Singer,et al.  The Hierarchical Hidden Markov Model: Analysis and Applications , 1998, Machine Learning.

[31]  James M. Rehg,et al.  Learning and inference in parametric switching linear dynamic systems , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[32]  Daniel Freedman,et al.  Illumination-invariant tracking via graph cuts , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[33]  James M. Rehg,et al.  Parameterized Duration Mmodeling for Switching Linear Dynamic Systems , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[34]  Namrata Vaswani,et al.  Additive Change Detection in Nonlinear Systems With Unknown Change Parameters , 2007, IEEE Transactions on Signal Processing.

[35]  James M. Rehg,et al.  Learning and Inferring Motion Patterns using Parametric Segmental Switching Linear Dynamic Systems , 2008, International Journal of Computer Vision.

[36]  Amit K. Roy-Chowdhury,et al.  Integrating Motion, Illumination, and Structure in Video Sequences with Applications in Illumination-Invariant Tracking , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.