Autonomous robot photographer with KL divergence optimization of image composition and human facial direction

Abstract In this paper, we propose a robot photography system that can autonomously search for the optimal target viewpoint. Two technical issues of scene composition evaluation and viewpoint selection are solved by this system. A composition evaluation method for photos is developed using well-known composition rules based on Kullback–Leibler (KL) divergence, considering the directional information of each target. To reduce the calculation cost of the composition evaluation in the case where the number of targets is large, automatic target grouping is conducted via variational Bayes. The optimal viewpoint with respect to the composition is selected from a number of candidate viewpoints around the targets based on KL divergence. Finally, the fact that better composed photos can be autonomously photographed by the proposed system is validated via experiments and human evaluations.

[1]  Kousuke Sekiyama,et al.  Autonomous Viewpoint Selection of Robot Based on Aesthetic Evaluation of a Scene , 2016, J. Artif. Intell. Soft Comput. Res..

[2]  Fumio Miyazaki,et al.  A stable tracking control method for a non-holonomic mobile robot , 1991, Proceedings IROS '91:IEEE/RSJ International Workshop on Intelligent Robots and Systems '91.

[3]  Masato Ito,et al.  Optimal viewpoint selection for cooperative visual assistance in multi-robot systems , 2015, 2015 IEEE/SICE International Symposium on System Integration (SII).

[4]  Jean-Philippe Thiran,et al.  Lower and upper bounds for approximation of the Kullback-Leibler divergence between Gaussian Mixture Models , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[5]  William D. Smart,et al.  Say Cheese! Experiences with a Robot Photographer , 2004, AI Mag..

[6]  Jun-Wei Huang,et al.  A photographer robot with multiview face detector , 2016, 2016 IEEE International Conference on Industrial Technology (ICIT).

[7]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[8]  Shengcai Liao,et al.  A Fast and Accurate Unconstrained Face Detector , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Stephen Cameron,et al.  Luke: An autonomous robot photographer , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Jaeyeon Lee,et al.  A Robot Photographer with User Interactivity , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Ren C. Luo,et al.  Intelligent robot photographer: Help people taking pictures using their own camera , 2014, 2014 IEEE/SICE International Symposium on System Integration.

[12]  William D. Smart,et al.  An autonomous robot photographer , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[13]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[14]  Daniel Cohen-Or,et al.  Optimizing Photo Composition , 2010, Comput. Graph. Forum.

[15]  Philippe Souères,et al.  Robust path-following control with exponential stability for mobile robots , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[16]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[17]  H. Risken.  Fokker-Planck Equation , 1996 .

[18]  Kanti V. Mardia,et al.  Bayesian inference for the von Mises-Fisher distribution , 1976 .

[19]  M. Abramowitz,et al.  Handbook of Mathematical Functions With Formulas, Graphs and Mathematical Tables (National Bureau of Standards Applied Mathematics Series No. 55) , 1965 .

[20]  Jizhong Xiao,et al.  An autonomous flyer photographer , 2016, 2016 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER).