Image Graph Production by Dense Captioning

Image captioning is the process of analyzing an image and generating a textual description according to objects and actions in the image. Thus, both image processing and natural language understanding are required for an image captioning system. Applications of image captioning can vary from assisting visually impaired people to detecting fake news in social media. One of significant utilizations of image captioning would be the detection of particular actions in images. In this paper, we use image captioning to produce a textual description from an image. Then we exploit a natural language processing algorithm to extract main components in the produced description. Finally we generate a general graph according to detected components in descriptions of the image. The generated graph shows objects and pairwise relationship between them along with their attributes. Thus, it can be used to determine if there is any particular relation in a sequence of input images.

[1]  Ramin Sahba,et al.  Improving IPC in simultaneous multi-threading (SMT) processors by capping IQ utilization according to dispatched memory instructions , 2014, 2014 World Automation Congress (WAC).

[2]  Ramin Sahba.  Hashing for fast IP address lookup utilizing inter-key correlation , 2013 .

[3]  Javad Poshtan,et al.  Simulative and experimental investigation on stator winding turn and unbalanced supply voltage fault diagnosis in induction motors using Artificial Neural Networks. , 2015, ISA transactions.

[4]  Mo Jamshidi,et al.  Applications of Big Data Analytics Tools for Data Management , 2016 .

[5]  Paul Rad,et al.  Deep learning control for complex and large scale cloud systems , 2017, Intell. Autom. Soft Comput..

[6]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[7]  Jiaxuan Wang,et al.  HICO: A Benchmark for Recognizing Human-Object Interactions in Images , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[8]  Berat A. Erol,et al.  Proposed testbed for the modeling and control of a system of autonomous vehicles , 2016, 2016 11th System of Systems Engineering Conference (SoSE).

[9]  Lee H. Keel,et al.  Secured formation control for multi-agent systems under DoS attacks , 2016, 2016 IEEE Symposium on Technologies for Homeland Security (HST).

[10]  Li Fei-Fei,et al.  DenseCap: Fully Convolutional Localization Networks for Dense Captioning , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Farshid Sahba,et al.  The diagnosis of lumbar disc disorder by MR image processing and data mining , 2016, 2016 World Automation Congress (WAC).

[12]  Javad Poshtan,et al.  Induction motor stator fault detection via fuzzy logic , 2013, 2013 21st Iranian Conference on Electrical Engineering (ICEE).

[13]  Farshid Sahba.  Wireless Sensors and RFID in Garden Automation , 2014 .

[14]  Morteza Dabbaghjamanesh,et al.  High Performance Control of Grid Connected Cascaded H-Bridge Active Rectifier Based on Type II-Fuzzy Logic Controller with Low Frequency Modulation Technique , 2016 .

[15]  P. Shahmaleki,et al.  Real time experimental study of truck backer upper problem with fuzzy controller , 2008, 2008 World Automation Congress.

[16]  Patrick Benavidez,et al.  Pedestrian detection system for smart communities using deep Convolutional Neural Networks , 2017, 2017 12th System of Systems Engineering Conference (SoSE).

[17]  Mo Jamshidi,et al.  Stock market prediction by using artificial neural network , 2014, 2014 World Automation Congress (WAC).

[18]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[20]  Majid Poshtan,et al.  Experimental validation on stator fault detection via fuzzy logic , 2013, 2013 3rd International Conference on Electric Power and Energy Conversion Systems.

[21]  Michael S. Bernstein,et al.  Image retrieval using scene graphs , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Hui Lin,et al.  Multi-agent System for Detecting False Data Injection Attacks Against the Power Grid , 2016, ICSS '16.

[23]  Berat A. Erol,et al.  Cloud-based control and vSLAM through cooperative Mapping and Localization , 2016, 2016 World Automation Congress (WAC).

[24]  Berat A. Erol,et al.  Improved Deep Neural Network Object Tracking System for Applications in Home Robotics , 2018 .

[25]  Paul Rad,et al.  AI Thinking for Cloud Education Platform with Personalized Learning , 2018, HICSS.

[26]  Mahdi Bahaghighat,et al.  Text-dependent Speaker Recognition by Combination of LBG VQ and DTW for Persian Language , 2012 .

[27]  Morteza Dabbaghjamanesh,et al.  Design of networked polynomial control systems with random delays: sum of squares approach , 2016, Int. J. Autom. Control..

[28]  P. Shahmaleki,et al.  Designing a hierarchical fuzzy controller for backing-up a four wheel autonomous robot , 2008, 2008 American Control Conference.

[29]  Mojtaba Mahzoon,et al.  Designing Fuzzy Controller and Real Time Experimental Studies on a Nonholonomic Robot , 2009, CTS 2009.

[30]  Lee H. Keel,et al.  Model-based resilient control for a multi-agent system against Denial of Service attacks , 2016, 2016 World Automation Congress (WAC).

[31]  Morteza Dabbaghjamanesh,et al.  An Optimization Technique Based on Profit of Investment and Market Clearing in Wind Power Systems , 2016 .

[32]  Farshid Sahba,et al.  Museum automation with RFID , 2014, 2014 World Automation Congress (WAC).

[33]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[34]  Hamed Bouzary,et al.  Service optimal selection and composition in cloud manufacturing: a comprehensive survey , 2018 .

[35]  Alireza Kazemi,et al.  Vision-Based Hierarchical Fuzzy Controller and Real Time Results for a Wheeled Autonomous Robot , 2010 .

[36]  Mehdi Madani,et al.  Real time object detection using a novel adaptive color thresholding method , 2011, Ubi-MUI '11.

[37]  Farshid Sahba,et al.  Smart tractors in pistachio orchards equipped with RFID , 2016, 2016 World Automation Congress (WAC).

[38]  Li-Jia Li,et al.  Dense Captioning with Joint Inference and Visual Context , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).