Video key-frame extraction for smart phones

In this paper, the key-frame extraction for video clips which are captured by smart phones are investigated. Different from existing work which exploits the image contents, this work utilizes the sensors embedded in the smart phone, such as accelerometers and the touch screen surface, to infer the user’s intention. Those intentions are further analyzed to extract the meaningful video key-frames. The proposed method is not only fast enough for on-device implementation, but it also can improve the key-frame extraction performance. Finally, a prototype is developed in a smart phone and extensive experimental validations are provided to show the advantages of the proposed method.

[1]  Marco Roccetti,et al.  Embedded key frame extraction in UGC scenarios , 2013, 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[2]  Danny Crookes,et al.  Hierarchical video summarization in reference subspace , 2009, IEEE Transactions on Consumer Electronics.

[3]  Hyunseung Choo,et al.  Picture browsing non-touch interaction methods for smartphones using an accelerometer and camera with a focus on phone dialing , 2013, Multimedia Tools and Applications.

[4]  Lei Yang,et al.  OpenSesame: Unlocking smart phone through handshaking biometrics , 2013, 2013 Proceedings IEEE INFOCOM.

[5]  Moncef Gabbouj,et al.  Multi-sensor fusion for sport genre classification of user generated mobile videos , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[6]  Per-Erik Forssén,et al.  Stabilizing cell phone video using inertial measurement sensors , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[7]  Sung-Jea Ko,et al.  Feature point classification based global motion estimation for video stabilization , 2013, IEEE Transactions on Consumer Electronics.

[8]  Ying-Wen Bai,et al.  Design and implementation of a fall monitor system by using a 3-axis accelerometer in a smart phone , 2012, 2012 IEEE 16th International Symposium on Consumer Electronics.

[9]  Wen-Shyong Hsieh,et al.  Automatic feature-based global motion estimation in video sequences , 2004, IEEE Transactions on Consumer Electronics.

[10]  Zygmunt Pizlo,et al.  Camera Motion-Based Analysis of User Generated Video , 2010, IEEE Transactions on Multimedia.

[11]  Martin K. Purvis,et al.  Wildlife video key-frame extraction based on novelty detection in semantic context , 2011, Multimedia Tools and Applications.

[12]  Mubarak Shah,et al.  Detection and representation of scenes in videos , 2005, IEEE Transactions on Multimedia.

[13]  José Eugenio Naranjo,et al.  Modeling the Driving Behavior of Electric Vehicles Using Smartphones and Neural Networks , 2014, IEEE Intelligent Transportation Systems Magazine.

[14]  Klaus Schöffmann,et al.  Keyframe extraction in endoscopic video , 2015, Multimedia Tools and Applications.

[15]  Yue Gao,et al.  A video summarization tool using two-level redundancy detection for personal video recorders , 2008, IEEE Transactions on Consumer Electronics.

[16]  Athanasios Voulodimos,et al.  A System for Multicamera Task Recognition and Summarization for Structured Environments , 2013, IEEE Transactions on Industrial Informatics.

[17]  Ba Tu Truong,et al.  Video abstraction: A systematic review and classification , 2007, TOMCCAP.

[18]  Moncef Gabbouj,et al.  Multimodal extraction of events and of information about the recording activity in user generated videos , 2012, Multimedia Tools and Applications.

[19]  Vincent Lepetit,et al.  Video-Based In Situ Tagging on Mobile Phones , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Gang Hua,et al.  Introduction to the Special Issue on Mobile Vision , 2011, International Journal of Computer Vision.

[21]  Cecilio Angulo,et al.  Online motion recognition using an accelerometer in a mobile device , 2012, Expert Syst. Appl..

[22]  Jiebo Luo,et al.  Towards Extracting Semantically Meaningful Key Frames From Personal Video Clips: From Humans to Computers , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  Yunhao Liu,et al.  Towards Accurate Object Localization with Smartphones , 2014, IEEE Transactions on Parallel and Distributed Systems.

[24]  Emiliano Miluzzo,et al.  A survey of mobile phone sensing , 2010, IEEE Communications Magazine.

[25]  L. Atzori,et al.  Indoor navigation system using image and sensor data processing on a smartphone , 2012, 2012 13th International Conference on Optimization of Electrical and Electronic Equipment (OPTIM).

[26]  Jian Ma,et al.  An enhanced personal photo recommendation system by fusing contextual and textual features on mobile device , 2013, IEEE Transactions on Consumer Electronics.