An IoT Edge-Fog-Cloud Architecture for Vision Based Planogram Integrity

A planogram is a detailed visual map of the products in the shelves of a retail store. The planogram wants to provide the best location of products on the shelves, with the aim to improve the customer experience and satisfaction, to increase sales and profits and to better manage the products on the shelves. However, an important aspect is not only to design the best planogram, but mostly to maintain the right position, orientation and, quantity of the products in the shelves according to the accepted planogram. For this purpose, we propose a fog computing architecture consisted of edge nodes which are low-cost cameras able to transmit in wireless mode shelf photos to local fog nodes in the same store. These last examine the images coming from the edges and sends results to the cloud for further data aggregation and analysis. The experimental results derive from a real environment considering a two-month observation period.

[1]  Priya Raghubir,et al.  Shelf space schemas: Myth or reality? ☆ , 2013 .

[2]  Marshall L. Fisher,et al.  Introduction to Focused Issue: Retail Operations Management , 2001, Manuf. Serv. Oper. Manag..

[3]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[4]  Emanuele Frontoni,et al.  Convolutional Networks for Semantic Heads Segmentation using Top-View Depth Data in Crowded Environment , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[5]  Emanuele Frontoni,et al.  Robotic retail surveying by deep learning visual and textual data , 2019, Robotics Auton. Syst..

[6]  Songul Albayrak,et al.  A survey of product recognition in shelf images , 2017, 2017 International Conference on Computer Science and Engineering (UBMK).

[7]  Emanuele Frontoni,et al.  Modelling and Forecasting Customer Navigation in Intelligent Retail Environments , 2018, J. Intell. Robotic Syst..

[8]  Emanuele Frontoni,et al.  Mobile robot for retail surveying and inventory using visual and textual analysis of monocular pictures based on deep learning , 2017, 2017 European Conference on Mobile Robots (ECMR).

[9]  Michele Merler,et al.  Recognizing Groceries in situ Using in vitro Training Data , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Emanuele Frontoni,et al.  Real time out of shelf detection using embedded sensor network , 2014, 2014 IEEE/ASME 10th International Conference on Mechatronic and Embedded Systems and Applications (MESA).

[11]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Song Liu,et al.  Planogram Compliance Checking Using Recurring Patterns , 2015, 2015 IEEE International Symposium on Multimedia (ISM).

[13]  Emanuele Frontoni,et al.  People Detection and Tracking from an RGB-D Camera in Top-View Configuration: Review of Challenges and Applications , 2017, ICIAP Workshops.

[14]  Emanuele Frontoni,et al.  Shelf space re-allocation for out of stock reduction , 2017, Comput. Ind. Eng..

[15]  Gül Varol,et al.  Product placement detection based on image processing , 2014, 2014 22nd Signal Processing and Communications Applications Conference (SIU).

[16]  Emanuele Frontoni,et al.  Machine Learning approach for Predictive Maintenance in Industry 4.0 , 2018, 2018 14th IEEE/ASME International Conference on Mechatronic and Embedded Systems and Applications (MESA).

[17]  Primo Zingaretti,et al.  An automatic analysis of shoppers behaviour using a distributed RGB-D cameras system , 2014, 2014 IEEE/ASME 10th International Conference on Mechatronic and Embedded Systems and Applications (MESA).

[18]  Heinrich Kuhn,et al.  Retail category management: State-of-the-art review of quantitative research and software applications in assortment and shelf space management , 2012 .

[19]  Emanuele Frontoni,et al.  Smart Vision System for Shelf Analysis in Intelligent Retail Environments , 2013 .

[20]  Emanuele Frontoni,et al.  Information Management for Intelligent Retail Environment: The Shelf Detector System , 2014, Inf..

[21]  Dipti Prasad Mukherjee,et al.  A comprehensive survey on computer vision based approaches for automatic identification of products in retail store , 2019, Image Vis. Comput..

[22]  C. Harland.  Supply Chain Management: Relationships, Chains and Networks , 1996 .

[23]  Emanuele Frontoni,et al.  Robotic platform for deep change detection for rail safety and security , 2017, 2017 European Conference on Mobile Robots (ECMR).

[24]  Rubaiya Hafiz,et al.  Image based drinks identification for dietary assessment , 2016, 2016 International Workshop on Computational Intelligence (IWCI).

[25]  Emanuele Frontoni,et al.  A heuristic approach to evaluate occurrences of products for the planogram maintenance , 2014, 2014 IEEE/ASME 10th International Conference on Mechatronic and Embedded Systems and Applications (MESA).

[26]  Emanuele Frontoni,et al.  A business application of RTLS technology in Intelligent Retail Environment: Defining the shopper's preferred path and its segmentation , 2019, Journal of Retailing and Consumer Services.

[27]  Narayanan Vijaykrishnan,et al.  Visual co-occurrence network: using context for large-scale object recognition in retail , 2015, 2015 13th IEEE Symposium on Embedded Systems For Real-time Multimedia (ESTIMedia).

[28]  Roberto Pierdicca,et al.  Automatic Classification for Anti Mixup Events in Advanced Manufacturing System , 2015 .

[29]  S. Lennon,et al.  Consumer response to online apparel stockouts , 2011 .

[30]  Daniel Corsten,et al.  Stock-Outs Cause Walkouts , 2004 .

[31]  W. Zinn,et al.  CONSUMER RESPONSE TO RETAIL STOCKOUTS , 2001 .

[32]  Rajeev Gandhi,et al.  Challenges and Opportunities for Embedded Computing in Retail Environments , 2012, S-CUBE.