A Quality-of-Content-Based Joint Source and Channel Coding for Human Detections in a Mobile Surveillance Cloud

More than 70% of consumer mobile Internet traffic will be mobile video transmissions by 2019. The development of wireless video transmission technologies has been boosted by the rapidly increasing demand of video streaming applications. Although more and more videos are delivered for video analysis (e.g., object detection/tracking and action recognition), most existing wireless video transmission schemes are developed to optimize human perception quality and are suboptimal for video analysis. In mobile surveillance networks, a cloud server collects videos from multiple moving cameras and detects suspicious persons in all camera views. Camera mobility in smartphones or dash cameras implies that video is to be uploaded through bandwidth-limited and error-prone wireless networks, which may cause quality degradation of the decoded videos and jeopardize the performance of video analyses. In this paper, we propose an effective rate-allocation scheme for multiple moving cameras in order to improve human detection (content) performance. Therefore, the optimization criterion of the proposed rate-allocation scheme is driven by quality of content (QoC). Both video source coding and application layer forward error correction coding rates are jointly optimized. Moreover, the proposed rate-allocation problem is formulated as a convex optimization problem and can be efficiently solved by standard solvers. Many simulations using High Efficiency Video Coding standard compression of video sequences and the deformable part model object detector are carried, and results demonstrate the effectiveness and favorable performance of our proposed QoC-driven scheme under different pedestrian densities and wireless conditions.

[1]  Jenq-Neng Hwang,et al.  OLM: Opportunistic Layered Multicasting for Scalable IPTV over Mobile WiMAX , 2012, IEEE Transactions on Mobile Computing.

[2]  Yue Yang,et al.  PCF Scheme for Periodic Data Transmission in Smart Metering Network with Cognitive Radio , 2014, GLOBECOM 2014.

[3]  Jenq-Neng Hwang,et al.  Deformable multiple-kernel based human tracking using a moving camera , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  T. Stockhammer,et al.  Application Layer FEC in IPTV Services , 2008 .

[5]  Ming-Ting Sun,et al.  Capture-to-display delay measurement for visual communication applications , 2015, APSIPA Transactions on Signal and Information Processing.

[6]  Jenq-Neng Hwang,et al.  Driving recorder based on-road pedestrian tracking using visual SLAM and Constrained Multiple-Kernel , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[7]  Jenq-Neng Hwang,et al.  On-Road Pedestrian Tracking Across Multiple Driving Recorders , 2015, IEEE Transactions on Multimedia.

[8]  Jenq-Neng Hwang,et al.  A Near Optimal QoE-Driven Power Allocation Scheme for Scalable Video Transmissions Over MIMO Systems , 2015, IEEE Journal of Selected Topics in Signal Processing.

[9]  Alan C. Bovik,et al.  Wireless Video Quality Assessment: A Study of Subjective Scores and Objective Algorithms , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Jun Huang,et al.  Joint source-channel coding and optimization for mobile video streaming in heterogeneous wireless networks , 2013, EURASIP J. Wirel. Commun. Netw..

[11]  Jenq-Neng Hwang,et al.  Quality-Driven Joint Rate and Power Adaptation for Scalable Video Transmissions Over MIMO Systems , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[13]  Huang-Chia Shih,et al.  A robust occupancy detection and tracking algorithm for the automatic monitoring and commissioning of a building , 2014 .

[14]  Jenq-Neng Hwang,et al.  A QoE-based APP layer scheduling scheme for scalable video transmissions over multi-RAT systems? , 2015, 2015 IEEE International Conference on Communications (ICC).

[15]  Thomas Stockhammer,et al.  IPTV Systems, Standards and Architectures: Part II - Application Layer FEC In IPTV Services , 2008, IEEE Communications Magazine.

[16]  Bernt Schiele,et al.  Robust Object Detection with Interleaved Categorization and Segmentation , 2008, International Journal of Computer Vision.

[17]  Jenq-Neng Hwang,et al.  A QoE-driven FEC rate adaptation scheme for scalable video transmissions over MIMO systems , 2015, 2015 IEEE International Conference on Communications (ICC).

[18]  Yue Yang Contributions to Smart Metering Protocol Design and Data Analytics , 2015 .

[19]  Xiang Chen,et al.  Quality-Driven Cross Layer Design of Video Transmissions over MIMO Systems , 2015 .

[20]  Jenq-Neng Hwang,et al.  A near optimal QoE-driven power allocation scheme for SVC-based video transmissions over MIMO systems , 2014, 2014 IEEE International Conference on Communications (ICC).

[21]  Roberto Rinaldo,et al.  A saliency-based rate control for people detection in video , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[22]  Bernd Girod,et al.  Mobile Visual Search , 2011, IEEE Signal Processing Magazine.

[23]  Yue Yang,et al.  On the number of relays for orthogonalize-and-forward relaying , 2011, 2011 International Conference on Wireless Communications and Signal Processing (WCSP).

[24]  Jenq-Neng Hwang,et al.  Optimal Power Allocation and Rate Adaptation for Scalable Video over Multi-User MIMO , 2014, 2015 IEEE Global Communications Conference (GLOBECOM).

[25]  Yue Yang,et al.  PMU deployment for optimal state estimation performance , 2012, 2012 IEEE Globecom Workshops.

[26]  Jenq-Neng Hwang,et al.  Model-Based Vehicle Localization Based on 3-D Constrained Multiple-Kernel Tracking , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Madhukar Budagavi,et al.  Improvements on Intra Block Copy in natural content video coding , 2015, 2015 IEEE International Symposium on Circuits and Systems (ISCAS).

[28]  Pascal Frossard,et al.  Forward Error Correction for Multipath Media Streaming , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  Ming-Ting Sun,et al.  A capture-to-display delay measurement system for visual communication applications , 2013, 2013 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference.

[30]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[31]  Gabriella Olmo,et al.  Slice Sorting for Unequal Loss Protection of Video Streams , 2008, IEEE Signal Processing Letters.

[32]  Homer H. Chen,et al.  Perceptual Rate-Distortion Optimization Using Structural Similarity Index as Quality Metric , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[33]  Markus Fiedler,et al.  A generic quantitative relationship between quality of experience and quality of service , 2010, IEEE Network.

[34]  Jenq-Neng Hwang Multimedia Networking: From Theory to Practice , 2009 .

[35]  Marco Tagliasacchi,et al.  Rate-accuracy optimization in visual wireless sensor networks , 2012, 2012 19th IEEE International Conference on Image Processing.

[36]  Antonio Iera,et al.  The Internet of Things: A survey , 2010, Comput. Networks.

[37]  Jenq-Neng Hwang,et al.  Fully Unsupervised Learning of Camera Link Models for Tracking Humans Across Nonoverlapping Cameras , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[38]  R. G. Purandare,et al.  QoS-Optimized Adaptive Multi-layer (OQAM) architecture of wireless network for high quality digital video transmission , 2015, J. Vis. Commun. Image Represent..

[39]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Elias Yaacoub,et al.  QoE Enhancement of SVC Video Streaming Over Vehicular Networks Using Cooperative LTE/802.11p Communications , 2015, IEEE Journal of Selected Topics in Signal Processing.

[41]  Huang-Chia Shih,et al.  A robust vehicle model construction and identification system using local feature alignment , 2013, 2013 IEEE International Symposium on Consumer Electronics (ISCE).

[42]  Jenq-Neng Hwang,et al.  Quality-of-content (QoC)-driven rate allocation for video analysis in mobile surveillance networks , 2015, 2015 IEEE 17th International Workshop on Multimedia Signal Processing (MMSP).

[43]  Marimuthu Palaniswami,et al.  Internet of Things (IoT): A vision, architectural elements, and future directions , 2012, Future Gener. Comput. Syst..

[44]  Yue Yang,et al.  Grouping-Based MAC Protocols for EV Charging Data Transmission in Smart Metering Network , 2014, IEEE Journal on Selected Areas in Communications.

[45]  Yue Yang,et al.  PMU placement for optimal three-phase state estimation performance , 2013, 2013 IEEE International Conference on Smart Grid Communications (SmartGridComm).

[46]  Colin Perkins,et al.  RTP: Audio and Video for the Internet , 2003 .

[47]  Eckehard G. Steinbach,et al.  A Novel Rate Control Framework for SIFT/SURF Feature Preservation in H.264/AVC Video Compression , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[48]  Ming-Ting Sun,et al.  Adaptive intra-refresh for low-delay error-resilient video coding , 2014, Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2014 Asia-Pacific.

[49]  Jenq-Neng Hwang,et al.  Adaptive mode and modulation coding switching scheme in MIMO multicasting system , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[50]  Tobias Hoßfeld,et al.  From Packets to People: Quality of Experience as a New Measurement Challenge , 2013, Data Traffic Monitoring and Analysis.

[51]  Luc Van Gool,et al.  A mobile vision system for robust multi-person tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Jenq-Neng Hwang,et al.  Multiple-kernel adaptive segmentation and tracking (MAST) for robust object tracking , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[53]  Bechir Hamdaoui,et al.  A Survey on Energy-Efficient Routing Techniques with QoS Assurances for Wireless Multimedia Sensor Networks , 2012, IEEE Communications Surveys & Tutorials.

[54]  Bing Zeng,et al.  New Transforms Tightly Bounded by DCT and KLT , 2012, IEEE Signal Processing Letters.

[55]  Huang-Chia Shih,et al.  SPiraL Aggregation Map (SPLAM): A new descriptor for robust template matching with fast algorithm , 2015, Pattern Recognit..

[56]  Jenq-Neng Hwang,et al.  An efficient CQI feedback resource allocation scheme for wireless video multicast services , 2013, 2013 IEEE Global Communications Conference (GLOBECOM).

[57]  Henry Stark,et al.  Probability, Statistics, and Random Processes for Engineers , 2011 .

[58]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.