metrics and methods of video quality assessment: a brief review

With rapid development of video acquisition devices and wide applications of video data, more and more requirements are established to use video data query. video quality assessment and improvement become popular and important research issues which attract lots of researchers. The video quality can influence technique application, user experience, and application results. This paper firstly reviews research work on video query based applications. Then, various metrics of video quality assessment are reviewed according to the requirement of reference video, including full-reference metrics, no-reference metrics and reduced-reference metrics. In addition, methods of video quality assessment are reviewed by the features, which include visual features, bitstream-based or packet-based features, data features. Finally, a number of video quality improvement methods are briefly introduced.

[1]  Alan C. Bovik,et al.  Motion-based perceptual quality assessment of video , 2009, Electronic Imaging.

[2]  Lucjan Janowski,et al.  Assessing quality of experience for high definition video streaming under diverse packet loss patterns , 2013, Signal Process. Image Commun..

[3]  Yingjie Xia,et al.  Towards improving quality of video-based vehicle counting method for traffic flow estimation , 2016, Signal Process..

[4]  Christos Faloutsos,et al.  Efficient Similarity Search In Sequence Databases , 1993, FODO.

[5]  Iain E. Garden Richardson,et al.  A Full Reference Quality Metric for Compressed Video Based on Mean Squared Error and Video Content , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Yingjie Xia,et al.  Scalable Constrained Spectral Clustering , 2015, IEEE Transactions on Knowledge and Data Engineering.

[7]  Aldo Maalouf,et al.  A no-reference color video quality metric based on a 3D multispectral wavelet transform , 2010, 2010 Second International Workshop on Quality of Multimedia Experience (QoMEX).

[8]  Ling Shao,et al.  Perceptually Guided Photo Retargeting , 2017, IEEE Transactions on Cybernetics.

[9]  Ofer Hadar,et al.  A metric for no-reference video quality assessment for HD TV delivery based on saliency maps , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[10]  Mei Yu,et al.  New no-reference blocking artifacts metric based on human visual system , 2009, 2009 International Conference on Wireless Communications & Signal Processing.

[11]  Tao Liu,et al.  Saliency Inspired Full-Reference Quality Metrics for Packet-Loss-Impaired Video , 2011, IEEE Transactions on Broadcasting.

[12]  Vladimir Zlokolica,et al.  Salient Motion Features for Video Quality Assessment , 2011, IEEE Transactions on Image Processing.

[13]  Weisi Lin,et al.  Low-Complexity Video Quality Assessment Using Temporal Quality Variations , 2012, IEEE Transactions on Multimedia.

[14]  Peipei Hou,et al.  Fusing Incomplete Multisensor Heterogeneous Data to Estimate Urban Traffic , 2016, IEEE MultiMedia.

[15]  Janghyun Yoon,et al.  Relevance feedback for semantics based image retrieval , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[16]  Luming Zhang,et al.  An Effective Video Summarization Framework Toward Handheld Devices , 2015, IEEE Transactions on Industrial Electronics.

[17]  Luming Zhang,et al.  Recognizing multi-view objects with occlusions using a deep architecture , 2015, Inf. Sci..

[18]  Ralf Steinmetz,et al.  Metadata Standards for Web-Based Resources , 2001, IEEE Multim..

[19]  Mikel Izal,et al.  Improving Video Quality in Network Paths with Bursty Losses , 2011, 2011 IEEE Global Telecommunications Conference - GLOBECOM 2011.

[20]  Yueting Zhuang,et al.  Accommodating hybrid retrieval in a comprehensive video database management system , 2002, IEEE Trans. Multim..

[21]  Mohammed Ghanbari,et al.  Modelling of spatio-temporal interaction for video quality assessment , 2010, Signal Process. Image Commun..

[22]  Stefano Tubaro,et al.  A reduced-reference structural similarity approximation for videos corrupted by channel errors , 2010, Multimedia Tools and Applications.

[23]  Jie Yao,et al.  No-Reference Video Quality Assessment Using Statistical Features Along Temporal Trajectory , 2012 .

[24]  Xuelong Li,et al.  Adaptive Multimedia Data Forwarding for Privacy Preservation in Vehicular Ad-Hoc Networks , 2017, IEEE Transactions on Intelligent Transportation Systems.

[25]  Shin Miura,et al.  Managing a media server content directory in absence of reliable metadata , 2009, IEEE Transactions on Consumer Electronics.

[26]  Rajiv Soundararajan,et al.  Study of Subjective and Objective Quality Assessment of Video , 2010, IEEE Transactions on Image Processing.

[27]  Chaofeng Li,et al.  Content-weighted video quality assessment using a three-component image model , 2010, J. Electronic Imaging.

[28]  Is-Haka Mkwawa,et al.  Content-Based Video Quality Prediction for HEVC Encoded Videos Streamed Over Packet Networks , 2015, IEEE Transactions on Multimedia.

[29]  Fernando Pereira,et al.  MPEG-7 the generic multimedia content description standard, part 1 - Multimedia, IEEE , 2001 .

[30]  Z. Meral Özsoyoglu,et al.  Design and Implementation of a Semantic Query Optimizer , 1989, IEEE Trans. Knowl. Data Eng..

[31]  Dong-Gyu Sim,et al.  Real-time objective quality assessment based on coding parameters extracted from H.264/AVC bitstream , 2010, IEEE Transactions on Consumer Electronics.

[32]  Mohammed Ghanbari,et al.  Reduced-Reference Video Quality Assessment Using Discriminative Local Harmonic Strength With Motion Consideration , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[33]  Martin Reisslein,et al.  Objective Video Quality Assessment Methods: A Classification, Review, and Performance Comparison , 2011, IEEE Transactions on Broadcasting.

[34]  A. S. Krishnakumar,et al.  Measuring video quality degradation using face detection , 2012, 2012 35th IEEE Sarnoff Symposium.

[35]  Joel H. Saltz,et al.  Optimizing the execution of multiple data analysis queries on parallel and distributed environments , 2004, IEEE Transactions on Parallel and Distributed Systems.

[36]  Xuelong Li,et al.  A Fine-Grained Image Categorization System by Cellet-Encoded Spatial Pyramid Modeling , 2015, IEEE Transactions on Industrial Electronics.

[37]  Alan C. Bovik,et al.  Motion Tuned Spatio-Temporal Quality Assessment of Natural Videos , 2010, IEEE Transactions on Image Processing.

[38]  Xuelong Li,et al.  Weakly Supervised Multimodal Kernel for Categorizing Aerial Photographs , 2017, IEEE Transactions on Image Processing.

[39]  Yuanchun Shi,et al.  A conformance test suite of localized LOM model , 2003, Proceedings 3rd IEEE International Conference on Advanced Technologies.

[40]  Keita Hirai,et al.  SV-CIELAB: Video Quality Assessment using Spatio-Velocity Contrast Sensitivity Function , 2010, Color Imaging Conference.

[41]  Yongdong Zhang,et al.  Visual quality assessment for web videos , 2010, J. Vis. Commun. Image Represent..

[42]  Charlotte Jenkins,et al.  Server-side automatic metadata generation using qualified Dublin Core and RDF , 2000, Proceedings 2000 Kyoto International Conference on Digital Libraries: Research and Practice.

[43]  V. Bhateja,et al.  A full-reference image quality metric for objective evaluation in spatial domain , 2011, 2011 International Conference on Communication and Industrial Application.

[44]  Jaime Lloret,et al.  A MPEG-2/MPEG-4 Quantizer to Improve the Video Quality in IPTV Services , 2010, 2010 Sixth International Conference on Networking and Services.

[45]  Lucjan Janowski,et al.  Lightweight implementation of No-Reference (NR) perceptual quality assessment of H.264/AVC compression , 2015, Signal Process. Image Commun..

[46]  Ying Zhang,et al.  Geometric discriminative features for aerial image retrieval in social media , 2014, Multimedia Systems.

[47]  Mi Hee,et al.  Hybrid video system supporting content-based retrieval , 1999, Proceedings Third International Conference on Computational Intelligence and Multimedia Applications. ICCIMA'99 (Cat. No.PR00300).

[48]  Jiro Katto,et al.  Quality assessment of compressed video sequences having blocking artifacts by cepstrum analysis , 2013, 2013 IEEE International Conference on Consumer Electronics (ICCE).

[49]  Stuart Weibel,et al.  The Dublin Core: A Simple Content Description Model for Electronic Resources , 2005 .

[50]  Touradj Ebrahimi,et al.  Balancing Attended and Global Stimuli in Perceived Video Quality Assessment , 2011, IEEE Transactions on Multimedia.

[51]  Zhou Wang,et al.  Video quality assessment based on structural distortion measurement , 2004, Signal Process. Image Commun..

[52]  Farah Torkamani-Azar,et al.  Video quality measurement based on 3-D. Singular value decomposition , 2015, J. Vis. Commun. Image Represent..

[53]  Xuelong Li,et al.  Toward solving the Steiner travelling salesman problem on urban road maps using the branch decomposition of graphs , 2016, Inf. Sci..

[54]  Tihao Chiang,et al.  An efficient FGS to mpeg‐1/2/4 single layer transcoder with R‐D optimized multi‐layer streaming technique for video quality improvement , 2007 .

[55]  Tao Liu,et al.  Saliency based objective quality assessment of decoded video affected by packet losses , 2008, 2008 15th IEEE International Conference on Image Processing.

[56]  Christian Keimel,et al.  No-reference video quality metric for HDTV based on H.264/AVC bitstream features , 2011, 2011 18th IEEE International Conference on Image Processing.

[57]  Hans-Peter Kriegel,et al.  Effective similarity search in multimedia databases using multiple representations , 2006, 2006 12th International Multi-Media Modelling Conference.

[58]  Lina J. Karam,et al.  A no-reference perceptual image sharpness metric based on a cumulative probability of blur detection , 2009, 2009 International Workshop on Quality of Multimedia Experience.

[59]  Yi Yang,et al.  Weakly Supervised Human Fixations Prediction , 2016, IEEE Transactions on Cybernetics.

[60]  Luming Zhang,et al.  Media Quality Assessment by Perceptual Gaze-Shift Patterns Discovery , 2017, IEEE Transactions on Multimedia.

[61]  Kwanghoon Sohn,et al.  Depth perception and motion cue based 3D video quality assessment , 2012, IEEE international Symposium on Broadband Multimedia Systems and Broadcasting.

[62]  Chin-Tser Huang,et al.  Semantic Retrieval of Instructional Videos , 2007, Ninth IEEE International Symposium on Multimedia Workshops (ISMW 2007).

[63]  Kai Zeng,et al.  Temporal motion smoothness measurement for reduced-reference video quality assessment , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[64]  Cristina Ribeiro,et al.  A multimedia database workbench for content and context retrieval , 2004, IEEE 6th Workshop on Multimedia Signal Processing, 2004..

[65]  Andrew Perkis,et al.  Spatial and temporal pooling of image quality metrics for perceptual video quality assessment on packet loss streams , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[66]  Beng Chin Ooi,et al.  Towards effective indexing for very large video sequence database , 2005, SIGMOD '05.

[67]  Su-Cheng Haw,et al.  A survey on optimizing video and audio query retrieval in multimedia databases , 2010, 2010 3rd International Conference on Advanced Computer Theory and Engineering(ICACTE).

[68]  C.-C. Jay Kuo,et al.  On-line knowledge- and rule-based video classification system for video indexing and dissemination , 2002, Inf. Syst..

[69]  Munchurl Kim,et al.  A perceptual quality assessment metric using temporal complexity and disparity information for stereoscopic video , 2011, 2011 18th IEEE International Conference on Image Processing.

[70]  Xuelong Li,et al.  Actively Learning Human Gaze Shifting Paths for Semantics-Aware Photo Cropping , 2014, IEEE Transactions on Image Processing.

[71]  Rabab Kreidieh Ward,et al.  Semi-blind quality estimation of compressed videos using digital watermarking , 2013, Digit. Signal Process..

[72]  Chao Xu,et al.  Big traffic data processing framework for intelligent monitoring and recording systems , 2016, Neurocomputing.

[73]  Jean-Luc Dugelay,et al.  Efficient ocular expression analysis for synthetic reproduction , 2006, IEEE Transactions on Multimedia.

[74]  J. Leon Zhao,et al.  Content-based object organization for efficient image retrieval in image databases , 2006, Decis. Support Syst..

[75]  Yue Gao,et al.  Representative Discovery of Structure Cues for Weakly-Supervised Image Segmentation , 2014, IEEE Transactions on Multimedia.

[76]  G. Sreelekha,et al.  Quality improvement of H.264/AVC FRExt by incorporating perceptual models , 2011, 2011 Annual IEEE India Conference.

[77]  Paola Lecca,et al.  On the Performances in Simulation of Parallel Databases: An Overview on the Most Recent Techniques for Query Optimization , 2009, 2009 International Workshop on High Performance Computational Systems Biology.

[78]  Snjezana Rimac-Drlje,et al.  Review of objective video quality metrics and performance comparison using different databases , 2013, Signal Process. Image Commun..

[79]  Liming Zhang,et al.  New strategy for image and video quality assessment , 2010, J. Electronic Imaging.

[80]  Xuelong Li,et al.  Spatial-Aware Object-Level Saliency Prediction by Learning Graphlet Hierarchies , 2015, IEEE Transactions on Industrial Electronics.

[81]  Chaminda T. E. R. Hewage,et al.  Reduced-reference quality evaluation for compressed depth maps associated with colour plus depth 3D video , 2010, 2010 IEEE International Conference on Image Processing.

[82]  Lan Zhang,et al.  SVD-Based 3D Image Quality Assessment by Using Depth Information , 2014, 2014 IEEE 17th International Conference on Computational Science and Engineering.

[83]  Yi Yang,et al.  Weakly Supervised Multilabel Clustering and its Applications in Computer Vision , 2016, IEEE Transactions on Cybernetics.

[84]  Li She,et al.  Effectively identifying the influential spreaders in large-scale social networks , 2014, Multimedia Tools and Applications.

[85]  Mohamed Daoudi,et al.  A Bayesian 3-D Search Engine Using Adaptive Views Clustering , 2007, IEEE Transactions on Multimedia.

[86]  Luming Zhang,et al.  Large-Scale Aerial Image Categorization by Multi-Task Discriminative Topologies Discovery , 2014, WISMM '14.

[87]  BovikAlan Conrad,et al.  Motion tuned spatio-temporal quality assessment of natural videos , 2010 .

[88]  Ira Assent,et al.  Efficient EMD-based similarity search in multimedia databases via flexible dimensionality reduction , 2008, SIGMOD Conference.