Image Selection in Photo Albums

The selection of the best photos in personal albums is a task that is often faced by photographers. This task can become laborious when the photo collection is large and it contains multiple similar photos. Recent advances on image aesthetics and photo importance evaluation has led to the creation of different metrics for automatically assessing a given image. However, these metrics are intended for the independent assessment of an image, without considering the possible context implicitly present within photo albums. In this work, we perform a user study for assessing how users select photos when provided with a complete photo album---a task that better reflects how users may review their personal photos and collections. Using the data provided by our study, we evaluate how existing state-of-the-art photo assessment methods perform relative to user selection, focusing in particular on deep learning based approaches. Finally, we explore a recent framework for adapting independent image scores to collections and evaluate in which scenarios such an adaptation can prove beneficial.

[1]  Susanne Boll,et al.  Employing Aesthetic Principles for Automatic Photo Book Layout , 2011, MMM.

[2]  David A. Shamma,et al.  YFCC100M , 2015, Commun. ACM.

[3]  Gabriela Csurka,et al.  Assessing the aesthetic quality of photographs using generic image descriptors , 2011, 2011 International Conference on Computer Vision.

[4]  Sabine Süsstrunk,et al.  Image aesthetic predictors based on weighted CNNs , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[5]  Andreas E. Savakis,et al.  Automated event clustering and quality screening of consumer pictures for digital albuming , 2003, IEEE Trans. Multim..

[6]  Adam Finkelstein,et al.  Automatic triage for a photo series , 2016, ACM Trans. Graph..

[7]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[8]  James Ze Wang,et al.  Studying Aesthetics in Photographic Images Using a Computational Approach , 2006, ECCV.

[9]  Antonio Torralba,et al.  Understanding and Predicting Image Memorability at a Large Scale , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[10]  Steffen Staab,et al.  Smart photo selection: interpret gaze as personal interest , 2014, CHI.

[11]  Yale Song,et al.  To Click or Not To Click: Automatic Selection of Beautiful Thumbnails from Videos , 2016, CIKM.

[12]  Wei-Ta Chu,et al.  Automatic selection of representative photo and smart thumbnailing using near-duplicate detection , 2008, ACM Multimedia.

[13]  Matthieu Guillaumin,et al.  Event Recognition in Photo Collections with a Stopwatch HMM , 2013, 2013 IEEE International Conference on Computer Vision.

[14]  Radomír Mech,et al.  Deep Multi-patch Aggregation Network for Image Style, Aesthetics, and Quality Estimation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Wei Luo,et al.  Content-Based Photo Quality Assessment , 2013, IEEE Trans. Multim..

[16]  Xian-Sheng Hua,et al.  The role of attractiveness in web image search , 2011, ACM Multimedia.

[17]  Mohamed Abdel-Mottaleb,et al.  Image browsing using hierarchical clustering , 1999, Proceedings IEEE International Symposium on Computers and Communications (Cat. No.PR00250).

[18]  Jacob Cohen,et al.  The Equivalence of Weighted Kappa and the Intraclass Correlation Coefficient as Measures of Reliability , 1973 .

[19]  Radomír Mech,et al.  Photo Aesthetics Ranking Network with Attributes and Content Adaptation , 2016, ECCV.

[20]  Andreas Girgensohn,et al.  Temporal event clustering for digital photo collections , 2003, ACM Multimedia.

[21]  Hailin Jin,et al.  Composition-Preserving Deep Photo Aesthetics Assessment , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Pratibha Mishra,et al.  Advanced Engineering Mathematics , 2013 .

[23]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[24]  Yonatan Wexler,et al.  Hierarchical photo organization using geo-relevance , 2007, GIS.

[25]  M. Kendall,et al.  The Problem of $m$ Rankings , 1939 .

[26]  Mats Sjöberg,et al.  Predicting Interestingness of Visual Content , 2017, Visual Content Indexing and Retrieval with Psycho-Visual Models.

[27]  Vasileios Mezaris,et al.  A comprehensive aesthetic quality assessment method for natural images using basic rules of photography , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[28]  Xiaoou Tang,et al.  Photo and Video Quality Evaluation: Focusing on the Subject , 2008, ECCV.

[29]  Tania Pouli,et al.  Context-aware clustering and assessment of photo collections , 2017, CAE '17.

[30]  Jianxiong Xiao,et al.  What makes an image memorable , 2011 .

[31]  Maria Klara Wolters,et al.  The art of deleting snapshots , 2014, CHI Extended Abstracts.

[32]  Radomír Mech,et al.  Event-Specific Image Importance , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Miriam Redi,et al.  The beauty of capturing faces: Rating the quality of digital portraits , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[34]  Florent Perronnin,et al.  Learning beautiful (and ugly) attributes , 2013, BMVC.

[35]  Ming Ouhyoung,et al.  Personalized photograph ranking and selection system , 2010, ACM Multimedia.

[36]  Claudia Niederée,et al.  Investigating human behaviors in selecting personal photos to preserve memories , 2015, 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[37]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[38]  Henning Müller,et al.  Result diversification in social image retrieval: a benchmarking framework , 2014, Multimedia Tools and Applications.

[39]  Luc Van Gool,et al.  The Interestingness of Images , 2013, 2013 IEEE International Conference on Computer Vision.

[40]  Peyman Milanfar,et al.  NIMA: Neural Image Assessment , 2017, IEEE Transactions on Image Processing.

[41]  Seunghoon Hong,et al.  Personalized Image Aesthetic Quality Assessment by Joint Regression and Ranking , 2017, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[42]  James Zijun Wang,et al.  RAPID: Rating Pictorial Aesthetics using Deep Learning , 2014, ACM Multimedia.

[43]  Claudia Niederée,et al.  To Keep or not to Keep: An Expectation-oriented Photo Selection Method for Personal Photo Collections , 2015, ICMR.

[44]  Mary Czerwinski,et al.  PhotoTOC: automatic clustering for browsing personal photographs , 2003, Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint.