An appearance-based representation of action

A new view-based approach to the representation of action is presented. Our underlying representations are view-based descriptions of the coarse image motion associated with viewing given actions from particular directions. Using these descriptions, we propose an appearance-based action-recognition strategy comprised of two stages: 1) a motion energy image (MEI) is computed that grossly describes the spatial distribution of motion energy for a given view of a given action, and the input MEI is matched against stored models which span the range of views of known actions; 2) any models that plausibly match the input are tested for a coarse, categorical agreement between a stored motion model of the action and a parametrization of the input motion. Using a "sitting" action as an example, and using a manually placed stick model, we develop a representation and verification technique that collapses the temporal variations of the motion parameters into a single, low-order vector.

[1]  Shimon Ullman,et al.  Analysis of Visual Motion by Biological and Computer Systems , 1981, Computer.

[2]  Larry S. Davis,et al.  Computing spatio-temporal representations of human faces , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[3]  David C. Hogg.  Model-based vision: a program to see a walking person , 1983, Image Vis. Comput..

[4]  Koichiro Akita,et al.  Image sequence analysis of real world human motion , 1984, Pattern Recognit..

[5]  Aaron F. Bobick,et al.  Recognition of human body motion using phase space constraints , 1995, Proceedings of IEEE International Conference on Computer Vision.

[6]  Mubarak Shah,et al.  Motion-based recognition a survey , 1995, Image Vis. Comput..

[7]  Alex Pentland,et al.  Space-time gestures , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[9]  Junji Yamato,et al.  Recognizing human action in time-sequential images using hidden Markov model , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Pietro Perona,et al.  Monocular tracking of the human arm in 3D , 1995, Proceedings of IEEE International Conference on Computer Vision.

[11]  William Grimson,et al.  Object recognition by computer - the role of geometric constraints , 1991 .

[12]  Michael J. Black,et al.  Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion , 1995, Proceedings of IEEE International Conference on Computer Vision.

[13]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[14]  Takeo Kanade,et al.  Model-based tracking of self-occluding articulated objects , 1995, Proceedings of IEEE International Conference on Computer Vision.

[15]  K. Rohr.  Towards model-based recognition of human movements in image sequences , 1994 .

[16]  Trevor Darrell,et al.  A novel environment for situated vision and behavior , 1994 .

[17]  K. Ikeuchi,et al.  Determining linear shape change: toward automatic generation of object recognition programs , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Yuntao Cui,et al.  Learning-based hand sign recognition using SHOSLIF-M , 1995, Proceedings of IEEE International Conference on Computer Vision.

[19]  Dmitry B. Goldgof,et al.  The scale space aspect graph , 1992, CVPR.

[20]  R. Nelson,et al.  Low level recognition of human motion (or how to get your man without finding his body parts) , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[21]  William T. Freeman,et al.  Orientation Histograms for Hand Gesture Recognition , 1995 .