Toward human motion capturing with an ultra-wide fisheye camera on the chest

We are interested in utilizing egocentric view from a wearable camera and are working on MonoEye system, a novel system to estimate the wearer's motion using a chest-mounted camera equipped with an ultra-wide fisheye lens. Because our system has a wide field of view, it provides a balanced capacity of recognizable pose types and broad egocentric view. The prototype deep neural network estimates camera wearer's 3D pose and acquires motion without complex configuration like conventional motion capture systems.

[1]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[2]  Takeo Kanade,et al.  Learning scene-specific pedestrian detectors without real data , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Hans-Peter Seidel,et al.  EgoCap , 2016, ACM Trans. Graph..

[4]  Da-Yuan Huang,et al.  Cyclops: Wearable and Single-Piece Full-Body Gesture Input Devices , 2015, CHI.

[5]  Michael J. Black,et al.  SMPL: A Skinned Multi-Person Linear Model , 2023 .

[6]  Pascal Fua,et al.  Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera , 2018, IEEE Transactions on Visualization and Computer Graphics.

[7]  Cordelia Schmid,et al.  Learning from Synthetic Humans , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Kris M. Kitani,et al.  3D Ego-Pose Estimation via Imitation Learning , 2018, ECCV.

[9]  Kristen Grauman,et al.  Seeing Invisible Poses: Estimating 3D Body Pose from Egocentric Video , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Yaser Sheikh,et al.  Motion capture from body-mounted cameras , 2011, SIGGRAPH 2011.

[11]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.