O fly, where art thou?

In this paper, the design of a real-time image acquisition system for tracking the movement of Drosophila in three-dimensional space is presented. The system uses three calibrated and synchronized cameras to detect multiple flies and integrates the detected fly silhouettes to construct the three-dimensional visual hull models of each fly. We used an extended Kalman filter to estimate the state of each fly, given past positions from the reconstructed fly visual hulls. The results show that our approach constructs the three-dimensional visual hull of each fly from the detected image silhouettes and robustly tracks them at real-time rates. The system is suitable for a more detailed analysis of fly behaviour.

[1]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[2]  Robert F. Murphy,et al.  Automated interpretation of subcellular location patterns , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[3]  Michael H Dickinson,et al.  The influence of visual landscape on the free flight behavior of the fruit fly Drosophila melanogaster. , 2002, The Journal of experimental biology.

[4]  Isaac Cohen,et al.  Posture and Gesture Recognition using 3D Body Shapes Decomposition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[5]  A. Gruen,et al.  Least squares 3D surface and curve matching , 2005 .

[6]  Gareth M. James.  Curve alignment by moments , 2007, 0712.1425.

[7]  H. Preisler,et al.  Modeling animal movements using stochastic differential equations , 2004 .

[8]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Mohan M. Trivedi,et al.  Articulated body posture estimation from multi-camera voxel data , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[10]  Paulo R. S. Mendonça,et al.  Model-based 3D tracking of an articulated hand , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[11]  KhanZia,et al.  MCMC-Based Particle Filtering for Tracking a Variable Number of Interacting Targets , 2005 .

[12]  J. L. Roux.  An Introduction to the Kalman Filter , 2003 .

[13]  Isaac Cohen,et al.  Inference of human postures by classification of 3D human body shape , 2003, 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443).

[14]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Michael Potmesil.  Generating octree models of 3D objects from their silhouettes in a sequence of images , 1987, Comput. Vis. Graph. Image Process..

[16]  Naoufel Werghi,et al.  Posture recognition and segmentation from 3D human body scans , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[17]  Steven V. Viscido,et al.  Self-Organized Fish Schools: An Examination of Emergent Properties , 2002, The Biological Bulletin.

[18]  G. Parisi,et al.  Interaction ruling animal collective behavior depends on topological rather than metric distance: Evidence from a field study , 2007, Proceedings of the National Academy of Sciences.

[19]  J. M. Gerzso,et al.  Computer graphics and interactive techniques: 15th-17th July 1974. Boulder, Colorado, USA. Sponsored by the University of Colorado Computing Centre and ACM/SIGGRAPH , 1975, Comput. Aided Des..

[20]  H.F. Durrant-Whyte,et al.  A new approach for filtering nonlinear systems , 1995, Proceedings of 1995 American Control Conference - ACC'95.

[21]  R. Y. Tsai,et al.  An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision , 1986, CVPR 1986.

[22]  Jun S. Liu,et al.  Mixture Kalman filters , 2000 .

[23]  Mubarak Shah.  Tracking people in presence of occlusion , 2000 .

[24]  Paulo R. S. Mendonça,et al.  Model-Based Hand Tracking Using an Unscented Kalman Filter , 2001, BMVC.

[25]  Frank Dellaert,et al.  MCMC-based particle filtering for tracking a variable number of interacting targets , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  S.N. Fry,et al.  A 6000 Hz Computer Vision System for Real-Time Wing Beat Analysis of Drosophila , 2006, The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006..

[27]  C. Dyer.  Volumetric Scene Reconstruction from Multiple Views , 2001 .

[28]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[29]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[30]  Simon Tavaré,et al.  Hydrogen Peroxide Stimulates Activity and Alters Behavior in Drosophila melanogaster , 2009, PloS one.

[31]  Jean Ponce,et al.  Automatic model construction, pose estimation, and object recognition from photographs using triangular splines , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[32]  J. Parrish,et al.  Animal Groups in Three Dimensions: Individual decisions, traffic rules, and emergent pattern in schooling fish , 1997 .

[33]  Ramesh C. Jain,et al.  Reality modeling and visualization from multiple video sequences , 1996, IEEE Computer Graphics and Applications.

[34]  Arun K. Pujari,et al.  Volume intersection with optimal set of directions , 1991, Pattern Recognit. Lett..

[35]  Richard Szeliski,et al.  Rapid octree construction from image sequences , 1993 .

[36]  Ingemar J. Cox,et al.  Modeling a Dynamic Environment Using a Bayesian Multiple Hypothesis Approach , 1994, Artif. Intell..

[37]  Bruce G. Baumgart,et al.  Geometric modeling for computer vision. , 1974 .

[38]  Gregory D. Hager,et al.  Probabilistic Data Association Methods for Tracking Complex Visual Objects , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Wojciech Matusik,et al.  Polyhedral Visual Hulls for Real-Time Rendering , 2001, Rendering Techniques.

[40]  Tucker R. Balch,et al.  Representing Honey Bee Behavior for Recognition Using Human Trainable Models , 2004, Adapt. Behav..

[41]  Manuela M. Veloso,et al.  Automatically tracking and analyzing the behavior of live insect colonies , 2001, AGENTS '01.

[42]  Rachid Deriche,et al.  Tracking line segments , 1990, Image Vis. Comput..

[43]  Tim J. Ellis,et al.  Illumination-Invariant Motion Detection Using Colour Mixture Models , 2001, BMVC.