Character‐Object Interaction Retrieval using the Interaction Bisector Surface

In this paper, we propose a novel approach for the classification and retrieval of interactions between human characters and objects. We propose to use the interaction bisector surface (IBS) between the body and the object as a feature of the interaction. We define a multi‐resolution representation of the body structure, and compute a correspondence matrix hierarchy that describes which parts of the character's skeleton take part in the composition of the IBS and how much they contribute to the interaction. Key‐frames of the interactions are extracted based on the evolution of the IBS and used to align the query interaction with the interaction in the database. Through the experimental results, we show that our approach outperforms existing techniques in motion classification and retrieval, which implies that the contextual information plays a significant role for scene and interaction description. Our method also shows better performance than other techniques that use features based on the spatial relations between the body parts, or the body parts and the object. Our method can be applied for character motion synthesis and robot motion planning.

[1]  Chi-Keung Tang,et al.  Make it home: automatic optimization of furniture arrangement , 2011, ACM Trans. Graph..

[2]  Kang Chen,et al.  Automatic semantic modeling of indoor scenes from low-quality RGB-D data using contextual information , 2014, ACM Trans. Graph..

[3]  Jitendra Malik,et al.  Robust computation of optical flow in a multi-scale differential framework , 2005, International Journal of Computer Vision.

[4]  Luc Van Gool,et al.  What makes a chair a chair? , 2011, CVPR 2011.

[5]  Zhaoqi Wang,et al.  Efficient motion data indexing and retrieval with local similarity measure of motion strings , 2009, The Visual Computer.

[6]  Chih-Yi Chiu,et al.  Content-based retrieval for human motion data , 2004, J. Vis. Commun. Image Represent..

[7]  Taku Komura,et al.  Relationship templates for creating scene variations , 2016, ACM Trans. Graph..

[8]  Meinard Müller,et al.  Information retrieval for music and motion , 2007 .

[9]  Niloy J. Mitra,et al.  Creating consistent scene graphs using a probabilistic grammar , 2014, ACM Trans. Graph..

[10]  Meinard Müller,et al.  Efficient content-based retrieval of motion capture data , 2005, SIGGRAPH '05.

[11]  Rama Chellappa,et al.  Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Pat Hanrahan,et al.  Example-based synthesis of 3D object arrangements , 2012, ACM Trans. Graph..

[13]  Taku Komura,et al.  Spatial relationship preserving character motion adaptation , 2010, SIGGRAPH 2010.

[14]  Nico Blodow,et al.  Persistent Point Feature Histograms for 3D Point Clouds , 2008 .

[15]  Elsevier Sdol,et al.  Journal of Visual Communication and Image Representation , 2009 .

[16]  Richard Szeliski,et al.  Multi-image matching using multi-scale oriented patches , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[17]  Yasuhiko Sakamoto,et al.  Motion map: image-based retrieval and segmentation of motion data , 2004, SCA '04.

[18]  Hubert P. H. Shum,et al.  Emulating human perception of motion similarity , 2008 .

[19]  Taku Komura,et al.  Indexing 3D Scenes Using the Interaction Bisector Surface , 2014, ACM Trans. Graph..

[20]  Lizhuang Ma,et al.  Content-Based Human Motion Retrieval with Automatic Transition , 2006, Computer Graphics International.

[21]  Leonidas J. Guibas,et al.  Shape2Pose , 2014, ACM Trans. Graph..

[22]  Yun Jiang,et al.  Hallucinated Humans as the Hidden Context for Labeling 3D Scenes , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Rui Ma,et al.  Organizing heterogeneous scene collections through contextual focal points , 2014, ACM Trans. Graph..

[24]  Meinard Müller.  DTW-Based Motion Comparison and Retrieval , 2007 .

[25]  Jessica K. Hodgins,et al.  Performance animation from low-dimensional control signals , 2005, ACM Trans. Graph..

[26]  David P. Dobkin,et al.  The quickhull algorithm for convex hulls , 1996, TOMS.

[27]  Edmond S. L. Ho,et al.  Spatial relationship preserving character motion adaptation , 2010, ACM Trans. Graph..

[28]  Dimitrios Gunopulos,et al.  Indexing Large Human-Motion Databases , 2004, VLDB.

[29]  Taku Komura,et al.  Interaction Retrieval by Spacetime Proximity Graphs , 2012, Comput. Graph. Forum.

[30]  Lucas Kovar,et al.  Automated extraction and parameterization of motions in large data sets , 2004, ACM Trans. Graph..

[31]  Pat Hanrahan,et al.  Characterizing structural relationships in scenes using graph kernels , 2011, ACM Trans. Graph..

[32]  Leonidas J. Guibas,et al.  Understanding and Exploiting Object Interaction Landscapes , 2016, ACM Trans. Graph..

[33]  Ligang Liu,et al.  Interaction context (ICON) , 2015, ACM Trans. Graph..

[34]  HuShi-Min,et al.  Automatic semantic modeling of indoor scenes from low-quality RGB-D data using contextual information , 2014 .

[35]  Nancy Argüelles,et al.  Author ' s , 2008 .

[36]  Pat Hanrahan,et al.  SceneGrok: inferring action maps in 3D environments , 2014, ACM Trans. Graph..

[37]  Meng Li,et al.  3D human motion retrieval using graph kernels based on adaptive graph construction , 2016, Comput. Graph..

[38]  Zhigang Deng,et al.  Perceptually consistent example-based human motion retrieval , 2009, I3D '09.

[39]  Chi-Keung Tang,et al.  Make it home: automatic optimization of furniture arrangement , 2011, SIGGRAPH 2011.

[40]  Eugene Fiume,et al.  An efficient search algorithm for motion data using weighted PCA , 2005, SCA '05.

[41]  Taku Komura,et al.  Interaction capture using magnetic sensors , 2013, Comput. Animat. Virtual Worlds.

[42]  Taku Komura,et al.  Emulating human perception of motion similarity , 2008, Comput. Animat. Virtual Worlds.

[43]  Ariel Shamir,et al.  Learning how objects function via co-analysis of interactions , 2016, ACM Trans. Graph..

[44]  Pat Hanrahan,et al.  Context-based search for 3D models , 2010, ACM Trans. Graph..

[45]  Ariel Shamir,et al.  Filling Your Shelves: Synthesizing Diverse Style-Preserving Artifact Arrangements , 2014, IEEE Transactions on Visualization and Computer Graphics.

[46]  Kun Zhou,et al.  An interactive approach to semantic modeling of indoor scenes with an RGBD camera , 2012, ACM Trans. Graph..