Context in Photo Albums

Recent progress in digital photography and storage availability has drastically changed our approach to photo creation. While in the era of film cameras, careful forethought would usually precede the capture of a photo; nowadays, a large number of pictures can be taken with little effort. One of the consequences is the creation of numerous photos depicting the same moment in slightly different ways, which makes the process of organizing photos laborious for the photographer. Nevertheless, photo collection organization is important both for exploring photo albums and for simplifying the ultimate task of selecting the best photos. In this work, we conduct a user study to explore how users tend to organize or cluster similar photos in albums, to what extent different users agree in their clustering decisions, and to investigate how the clustering-defined photo context affects the subsequent photo-selection process. We also propose an automatic hierarchical clustering solution for modeling user clustering decisions. To demonstrate the usefulness of our approach, we apply it to the task of automatic photo evaluation within photo albums and propose a clustering-based context adaptation.

[1]  Andreas Girgensohn,et al.  Temporal event clustering for digital photo collections , 2003, ACM Multimedia.

[2]  David A. Shamma,et al.  YFCC100M , 2015, Commun. ACM.

[3]  Sabine Süsstrunk,et al.  Image aesthetic predictors based on weighted CNNs , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[4]  Naila Murray,et al.  AVA: A large-scale database for aesthetic visual analysis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Radomír Mech,et al.  Deep Multi-patch Aggregation Network for Image Style, Aesthetics, and Quality Estimation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[6]  Yonatan Wexler,et al.  Hierarchical photo organization using geo-relevance , 2007, GIS.

[7]  Tania Pouli,et al.  Context-aware clustering and assessment of photo collections , 2017, CAE '17.

[8]  Radomír Mech,et al.  Event-Specific Image Importance , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Yan Ke,et al.  The Design of High-Level Features for Photo Quality Assessment , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[10]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[11]  Vasileios Mezaris,et al.  A comprehensive aesthetic quality assessment method for natural images using basic rules of photography , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[12]  Xiaoou Tang,et al.  Photo and Video Quality Evaluation: Focusing on the Subject , 2008, ECCV.

[13]  Brendan J. Frey,et al.  Non-metric affinity propagation for unsupervised image categorization , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[14]  Gabriela Csurka,et al.  Assessing the aesthetic quality of photographs using generic image descriptors , 2011, 2011 International Conference on Computer Vision.

[15]  Miriam Redi,et al.  The beauty of capturing faces: Rating the quality of digital portraits , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[16]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Min-Yen Kan,et al.  Hidden Markov Model for Event Photo Stream Segmentation , 2012, 2012 IEEE International Conference on Multimedia and Expo Workshops.

[18]  Florent Perronnin,et al.  Learning beautiful (and ugly) attributes , 2013, BMVC.

[19]  Andreas E. Savakis,et al.  Automated event clustering and quality screening of consumer pictures for digital albuming , 2003, IEEE Trans. Multim..

[20]  Abigail Sellen,et al.  Understanding photowork , 2006, CHI.

[21]  Ben Shneiderman,et al.  Meaningful presentations of photo libraries: rationale and applications of bi-level radial quantum layouts , 2005, Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '05).

[22]  Takayuki Itoh,et al.  CAT: A Hierarchical Image Browser Using a Rectangle Packing Technique , 2008, 2008 12th International Conference Information Visualisation.

[23]  Peyman Milanfar,et al.  NIMA: Neural Image Assessment , 2017, IEEE Transactions on Image Processing.

[24]  Seunghoon Hong,et al.  Personalized Image Aesthetic Quality Assessment by Joint Regression and Ranking , 2017, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[25]  Claudia Niederée,et al.  Investigating human behaviors in selecting personal photos to preserve memories , 2015, 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[26]  James Zijun Wang,et al.  RAPID: Rating Pictorial Aesthetics using Deep Learning , 2014, ACM Multimedia.

[27]  Zhiwu Lu,et al.  From comparing clusterings to combining clusterings , 2008, AAAI 2008.

[28]  Claudia Niederée,et al.  To Keep or not to Keep: An Expectation-oriented Photo Selection Method for Personal Photo Collections , 2015, ICMR.

[29]  Mary Czerwinski,et al.  PhotoTOC: automatic clustering for browsing personal photographs , 2003, Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint.

[30]  James Ze Wang,et al.  Studying Aesthetics in Photographic Images Using a Computational Approach , 2006, ECCV.

[31]  Steffen Staab,et al.  Smart photo selection: interpret gaze as personal interest , 2014, CHI.

[32]  M. Cugmas,et al.  On comparing partitions , 2015 .

[33]  Tania Pouli,et al.  Image Selection in Photo Albums , 2018, ICMR.

[34]  Jiebo Luo,et al.  Annotating collections of photos using hierarchical event and scene models , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  J. H. Ward Hierarchical Grouping to Optimize an Objective Function , 1963 .

[36]  Wei-Ta Chu,et al.  Automatic selection of representative photo and smart thumbnailing using near-duplicate detection , 2008, ACM Multimedia.

[37]  Florent Perronnin,et al.  Large-scale image retrieval with compressed Fisher vectors , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[38]  Matthieu Guillaumin,et al.  Event Recognition in Photo Collections with a Stopwatch HMM , 2013, 2013 IEEE International Conference on Computer Vision.

[39]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[40]  Ming Ouhyoung,et al.  Personalized photograph ranking and selection system , 2010, ACM Multimedia.

[41]  Andrew D. Miller,et al.  Give and take: a study of consumer photo-sharing culture and practice , 2007, CHI.

[42]  Pratibha Mishra,et al.  Advanced Engineering Mathematics , 2013 .

[43]  Giorgos Tolias,et al.  Fine-Tuning CNN Image Retrieval with No Human Annotation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Yi Li,et al.  Convolutional Neural Networks for No-Reference Image Quality Assessment , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Radomír Mech,et al.  Photo Aesthetics Ranking Network with Attributes and Content Adaptation , 2016, ECCV.

[46]  Kenneth Levenberg A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[47]  Xiaogang Wang,et al.  Content-based photo quality assessment , 2011, 2011 International Conference on Computer Vision.

[48]  Kerry Rodden,et al.  How do people manage their digital photographs? , 2003, CHI '03.

[49]  Mohamed Abdel-Mottaleb,et al.  Image browsing using hierarchical clustering , 1999, Proceedings IEEE International Symposium on Computers and Communications (Cat. No.PR00250).

[50]  Adam Finkelstein,et al.  Automatic triage for a photo series , 2016, ACM Trans. Graph..