On the impact of wireless multimedia network for multi-modal activity recognition

Due to the rapid development of sensor devices, many types of multimedia sensor data can be collected and analyzed to develop useful multimedia applications such as human activity recognition. Our objective of this research work is to study and analyze the impact of the wireless multimedia network quality on human activity recognition accuracy in an end-to-end networked multimedia system environment - Ocha House. We utilized the new improved Generalized Time Warping (GTW) algorithm and its variants for temporally aligning multi-modal sequences from multiple subjects performing similar human activities. In Ocha House, we captured the human activity data with camera sensors which are transmitted via wireless network for server processing and analysis. We evaluated the human activity recognition accuracy using GTW and its variants by taking into consideration of the wireless network data packet loss phenomenon due to common wireless network protocol behaviors as a result of network flow control and congestion management. We discovered that GTW and its variants are sensitive to wireless network packet loss and observed interesting characteristics resulting in deteriorated human activity recognition accuracy - when wireless network packet loss rate is nominal, the human activity recognition accuracy decreases dramatically.

[1]  Svetha Venkatesh,et al.  Multi-modal emotive computing in a smart house environment , 2007, Pervasive Mob. Comput..

[2]  Fernando De la Torre,et al.  Generalized time warping for multi-modal alignment of human motion , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Gérard G. Medioni,et al.  Dynamic Manifold Warping for view invariant action recognition , 2011, 2011 International Conference on Computer Vision.

[4]  H. Vincent Poor,et al.  Constrained Energy-Aware AP Placement with Rate Adaptation in WLAN Mesh Networks , 2011, 2011 IEEE Global Telecommunications Conference - GLOBECOM 2011.

[5]  Jian Lu,et al.  Recognizing multi-user activities using wearable sensors in a smart home , 2011, Pervasive Mob. Comput..

[6]  Fernando De la Torre,et al.  Canonical Time Warping for Alignment of Human Behavior , 2009, NIPS.

[7]  Kevin C. Almeroth,et al.  Rate Adaptation in Congested Wireless Networks through Real-Time Measurements , 2010, IEEE Transactions on Mobile Computing.

[8]  Stefano Tubaro,et al.  Subjective Quality Assessment of H.264/AVC Video Streaming with Packet Losses , 2011, EURASIP J. Image Video Process..

[9]  K. Mardia,et al.  Statistical Shape Analysis , 1998 .

[10]  Kari Pulli,et al.  Style translation for human motion , 2005, SIGGRAPH 2005.

[11]  Tutomu Murase,et al.  Throughput analysis and measurement on real terminal in multi-rate wireless LAN , 2012, ICUIMC '12.

[12]  Eamonn J. Keogh,et al.  Derivative Dynamic Time Warping , 2001, SDM.