Metadata-Based Activity Analysis in 3D Tele-Immersion

In view of the necessity of activity analysis on Gbps-scale 3D Tele-Immersive (3DTI) content bundle and petabyte-scale 3DTI recordings, this work proposes and verifies the feasibility of replacing high-latency intrusive analysis with light-weighted metadata-based analysis. For real-time in-session use case, result shows that metadata-based analysis module, when personalized with user's body index, can achieve accuracies in 90 percentile on classifying various activity classes. For offline cross-session use case, we propose a hybrid analysis scheme which combines the advantages of intrusive analysis (i.e., conventional activity analysis on content level) and metadata-based analysis and achieves high accuracy with low computation latency.

[1]  BajcsyRuzena,et al.  Enabling multi-party 3D tele-immersive environments with ViewCast , 2010 .

[2]  Klara Nahrstedt,et al.  4D TeleCast: Towards Large Scale Multi-site and Multi-view Dissemination of 3DTI Contents , 2012, 2012 IEEE 32nd International Conference on Distributed Computing Systems.

[3]  Ruzena Bajcsy,et al.  Upper extremity 3‐dimensional reachable workspace analysis in dystrophinopathy using Kinect , 2015, Muscle & nerve.

[4]  Ruzena Bajcsy,et al.  Color-plus-depth level-of-detail in 3D tele-immersive video: a psychophysical approach , 2011, MM '11.

[5]  Greg Welch,et al.  Smart instrumented training ranges: bringing automated system solutions to support critical domain needs , 2013 .

[6]  Indranil Gupta,et al.  3DTI Amphitheater , 2015, ACM Trans. Multim. Comput. Commun. Appl..

[7]  Luc Van Gool,et al.  Sparse Flow: Sparse Matching for Small to Large Displacement Optical Flow , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[8]  Hans-Peter Seidel,et al.  Performance capture from sparse multi-view video , 2008, ACM Trans. Graph..

[9]  Hans-Peter Seidel,et al.  Marker-Less 3D Feature Tracking for Mesh-Based Human Motion Capture , 2007, Workshop on Human Motion.

[10]  Indranil Gupta,et al.  3DTI amphitheater: a manageable 3DTI environment with hierarchical stream prioritization , 2014, MMSys '14.

[11]  Klara Nahrstedt,et al.  Activity-based synthesized frame generation in 3DTI video , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[12]  Klara Nahrstedt,et al.  Activity-aware adaptive compression: a morphing-based frame synthesis application in 3DTI , 2013, ACM Multimedia.

[13]  Ruzena Bajcsy,et al.  Computational Model of Human Creativity in Dance Choreography , 2008, AAAI Spring Symposium: Creative Intelligent Systems.

[14]  Ruzena Bajcsy,et al.  Enabling multiparty 3D tele-immersive environments with ViewCast , 2010, TOMCCAP.

[15]  Jiawei Han,et al.  DIAMOND: Correlation-Based Anomaly Monitoring Daemon for DIME , 2010, 2010 IEEE International Symposium on Multimedia.

[16]  Klara Nahrstedt,et al.  FreeViewer: An Intelligent Director for 3D Tele-Immersion System , 2014, ACM Multimedia.

[17]  Klara Nahrstedt,et al.  3D teleimmersive activity classification based on application-system metadata , 2013, MM '13.

[18]  Klara Nahrstedt,et al.  M-TEEVE: real-time 3D video interaction and broadcasting framework for mobile devices , 2009, IMMERSCOM.

[19]  Reflection Coefficients,et al.  A. real-time , 1982 .

[20]  S. Goldsack,et al.  IN REAL-TIME , 2008 .

[21]  Ruzena Bajcsy,et al.  Classification and Analysis of 3 D Tele-immersive Activities , 2012 .

[22]  Michele Sanna,et al.  A 3D tele-immersion system based on live captured mesh geometry , 2013, MMSys.

[23]  Ruzena Bajcsy,et al.  Enabling multi-party 3D tele-immersive environments with ViewCast , 2010, TOMCCAP.

[25]  Klara Nahrstedt,et al.  OmniViewer: Enabling Multi-modal 3D DASH , 2015, ACM Multimedia.

[26]  Ruzena Bajcsy,et al.  Classification and Analysis of 3D Teleimmersive Activities , 2013, IEEE MultiMedia.

[27]  Shannon Chen.  Exploiting Contextual Information to Enable Efficient Content Delivery for 3D Tele-Immersion Applications , 2015, ACM Multimedia.

[28]  Gregorij Kurillo,et al.  Teleimmersive Archaeology: Simulation and Cognitive Impact , 2010, EuroMed.

[29]  Robert M. Haralick,et al.  Multiscale 3D feature extraction and matching with an application to 3D face recognition , 2013, Graph. Model..

[30]  Graham Coleman,et al.  Detection and explanation of anomalous activities: representing activities as bags of event n-grams , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[31]  Klara Nahrstedt,et al.  Prioritized evolutionary optimization in open session management for 3D tele-immersion , 2013, MMSys.

[32]  Jitendra Malik,et al.  Large displacement optical flow , 2009, CVPR.

[33]  Cordelia Schmid,et al.  DeepFlow: Large Displacement Optical Flow with Deep Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[34]  B. Prabhakaran,et al.  A real time, distributed system with haptic interfaces for fine motor skill rehabilitation and its quality of experience , 2014, 2014 IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE) Proceedings.

[35]  Horst Bischof,et al.  A Duality Based Approach for Realtime TV-L1 Optical Flow , 2007, DAGM-Symposium.