Augmented Reality Based Recommendations Based on Perceptual Shape Style Compatibility with Objects in the Viewpoint and Color Compatibility with the Background

Augmented Reality (AR) has been heralded as the next frontier in retail, but so far, has been mostly used to advertise or market products in a gimmicky way and its true potential in digital marketing remains unexploited. In this work, we leverage richer data coming from AR usage to make re-targeting much more persuasive via viewpoint image augmentation. Based on the user's purchase viewpoint visual, we identify relevant objects/products present in the viewpoint along with their style such that products with more style compatibility with those surrounding real-world objects can be recommended. We also use color compatibility with the background of the user's purchase viewpoint to select suitable product textures. We embed the recommended products in the viewpoint at the location of the initially browsed product with similar pose and scale. This makes the recommendations much more personalized and relevant which can increase conversions. Evaluation with user studies show that our system is able to make recommendations better than tag-based recommendations, and targeting using the viewpoint is better than that of usual product catalogs.

[1]  Kumar Ayush,et al.  [POSTER] Enhanced Personalized Targeting Using Augmented Reality , 2017, 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct).

[2]  Mateu Sbert,et al.  Automatic View Selection Using Viewpoint Entropy and its Application to Image‐Based Modelling , 2003, Comput. Graph. Forum.

[3]  Andreas M. Kunz,et al.  Using Head Tracking Data for Robust Short Term Path Prediction of Human Locomotion , 2013, Trans. Comput. Sci..

[4]  Wilmot Li,et al.  Style compatibility for 3D furniture models , 2015, ACM Trans. Graph..

[5]  Pan Hui,et al.  When Augmented Reality meets Big Data , 2017, 2017 IEEE 37th International Conference on Distributed Computing Systems Workshops (ICDCSW).

[6]  Aaron Hertzmann,et al.  Color compatibility from large datasets , 2011, SIGGRAPH 2011.

[7]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Dameng Dai.  Stylized rendering for virtual furniture layout , 2011, 2011 International Conference on Multimedia Technology.

[9]  Altigran Soares da Silva,et al.  Finding Similar Products in E-commerce Sites Based on Attributes , 2015, AMW.

[10]  Roberto Pierdicca,et al.  Automatic Analysis of Eye-Tracking Data for Augmented Reality Applications: A Prospective Outlook , 2016, AVR.

[11]  Przemyslaw Kazienko,et al.  Integration of relational databases and Web site content for product and page recommendation , 2004, Proceedings. International Database Engineering and Applications Symposium, 2004. IDEAS '04..

[12]  J. Bobadilla,et al.  Recommender systems survey , 2013, Knowl. Based Syst..

[13]  Alexei A. Efros,et al.  Seeing 3D Chairs: Exemplar Part-Based 2D-3D Alignment Using a Large Dataset of CAD Models , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Lukasz Raczkowski,et al.  Visual Recommendation Use Case for an Online Marketplace Platform: allegro.pl , 2016, SIGIR.

[15]  Stephan Steglich,et al.  A Generic Multipurpose recommender System for Contextual Recommendations , 2007, Eighth International Symposium on Autonomous Decentralized Systems (ISADS'07).

[16]  Andy Cockburn,et al.  User-defined gestures for augmented reality , 2013, INTERACT.

[17]  Afzal Godil,et al.  A benchmark for best view selection of 3D objects , 2010, 3DOR '10.

[18]  Xavier Bonaventura Brugués.  Perceptual information-theoretic measures for viewpoint selection and object recognition , 2015 .

[19]  Bruce Krulwich.  Intelligent User Profiling Using Large-Scale Demographic Data1 , 1997 .

[20]  Hui Xiong,et al.  Point-of-Interest Recommendation in Location Based Social Networks with Topic and Location Awareness , 2013, SDM.