Visual understanding industrial workflows under uncertainty on distributed service oriented architectures

Service Oriented Architecture (SOA) is an evolution of distributed computing based on the request/reply design paradigm for synchronous and asynchronous applications. In such architectures, application developers or system integrators can build scenarios by composing one or more services without knowing the services' underlying implementations. In this paper, we adopt a SOA framework for on-line scheduling complex industrial workflows. The proposed SOA consists of three main layers; the input layer, the benchmarking layer, the prediction layer and the scheduler. The main innovation of our SOA architecture is the prediction layer that supports computer vision tools able to visually observe how an industrial workflow is actually executed. This way, we can automatically estimate the actual execution time for a process. In particular a new self-initialized visual tracker algorithm is proposed in this paper to robustly trace workers' trajectory in a plant via visual observations. Then, part-to-whole curve matching is presented so as to find correspondences among the traced curve and the ideal one and thus improving scheduler efficiency. The input layer interoperably describes industrial operations using the XPDL (an XML-based) format. The benchmarker evaluates much faster than real-time and in an off-line mode how long it takes for an industrial workflow to be executed on a given resource guaranteeing an almost real-time implementation of the video processing algorithms on the plant. The last component of the proposed SOA is the scheduler with the goal to assign the workflows to the available resources. In this paper, a Maximum Benefit First (MBF) scheduler is presented which maximizes the total gain received by the industry when completing execution of all the beneficial operations with minimum violations of their delivery deadlines (that is with the minimum compensations). Experiments have been conducted on a real-world industrial plant of Nissan Iberica automobile construction indicating the efficiency of the proposed system.

[1]  Daniel Cremers,et al.  Integral Invariants for Shape Matching , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Florian Michahelles,et al.  Increasing Supply-Chain Visibility with Rule-Based RFID Data Analysis , 2009, IEEE Internet Computing.

[3]  Farzin Mokhtarian,et al.  A Theory of Multiscale, Curvature-Based Shape Representation for Planar Curves , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Ronald L. Rivest,et al.  Introduction to Algorithms, Second Edition , 2001 .

[5]  David M. Booth,et al.  Web Services Architecture , 2004 .

[6]  Wen Gao,et al.  Modeling Background and Segmenting Moving Objects from Compressed Video , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Kamlendu Kumar Pandey,et al.  Design of SOA based service stack for collaborative wireless sensor network , 2009, 2009 Fifth International Conference on Wireless Communication and Sensor Networks (WCSN).

[8]  Eduardo Huedo,et al.  A modular meta-scheduling architecture for interfacing with pre-WS and WS Grid resource management services , 2007, Future Gener. Comput. Syst..

[9]  B. Cyganek.  An Introduction to 3D Computer Vision Techniques and Algorithms , 2009 .

[10]  Jeffrey W. Herrmann,et al.  Handbook of production scheduling , 2006 .

[11]  Mandyam D. Srinath,et al.  Partial Shape Classification Using Contour Matching in Distance Transformation , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Gregory D. Hager,et al.  Probabilistic Data Association Methods for Tracking Complex Visual Objects , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Lakmal D. Seneviratne,et al.  On the Recognition and Location of Partially Occluded Objects , 1999, J. Intell. Robotic Syst..

[14]  Jörg Hähner,et al.  PhD forum: Competing agents for distributed object-tracking in smart camera networks , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[15]  Tommaso Cucinotta,et al.  A Real-Time Service-Oriented Architecture for Industrial Automation , 2009, IEEE Transactions on Industrial Informatics.

[16]  Giovanni Caire,et al.  WADE: a software platform to develop mission critical applications exploiting agents and workflows , 2008, AAMAS.

[17]  Shervin Shirmohammadi,et al.  A distributed camera network architecture supporting video adaptation , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[18]  Markus Ulrich,et al.  Machine Vision Algorithms and Applications , 2007 .

[19]  Datong Chen,et al.  Robust Object Tracking Via Online Dynamic Spatial Bias Appearance Models , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Wesley E. Snyder,et al.  Application of Affine-Invariant Fourier Descriptors to Recognition of 3-D Objects , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Sudeep Sarkar,et al.  Background subtraction in varying illuminations using an ensemble based on an enlarged feature set , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[23]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[24]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[25]  Miguel Contreras,et al.  Industrial application integration using the unification approach to agent-enabled semantic SOA , 2008 .

[26]  Hau L. Lee,et al.  Unlocking the Value of RFID , 2007 .

[27]  Luís M. M. Custódio,et al.  Production planning and scheduling using a fuzzy decision system , 1994, IEEE Trans. Robotics Autom..

[28]  Frédérique Biennier,et al.  A Service Oriented Architecture to Support Industrial Information Systems , 2006, APMS.

[29]  Zoran Zivkovic,et al.  Improved adaptive Gaussian mixture model for background subtraction , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[30]  Clifford Stein,et al.  Introduction to Algorithms, 2nd edition. , 2001 .

[31]  Serkan Kaygin,et al.  Shape recognition using attributed string matching with polygon vertices as the primitives , 2002, Pattern Recognit. Lett..

[32]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[34]  I. Foster,et al.  Service-Oriented Science , 2005, Science.

[35]  Roman Barták.  On the Boundary of Planning and Scheduling: A Study , 1999 .

[36]  Omri Serlin,et al.  Multiprogramming for Hybrid Computation , 1899 .

[37]  Tamás Kis,et al.  Computing lower and upper bounds for a large-scale industrial job shop scheduling problem , 2009, Eur. J. Oper. Res..

[38]  José T. Palma,et al.  Scheduling of maintenance work: A constraint-based approach , 2010, Expert Syst. Appl..

[39]  Theodora A. Varvarigou,et al.  Adaptive for Interactive , 2003 .

[40]  Yuichi Motai,et al.  Salient feature extraction of industrial objects for an automated assembly system , 2005, Comput. Ind..

[41]  Andrea Cavallaro,et al.  Multi-camera track-before-detect , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[42]  Pedro M. Castro,et al.  Dynamic modelling and scheduling of an industrial batch system , 2002 .

[43]  S. V. Patel,et al.  A Design of Sensor Web Registry for Wireless Sensor Networks with SOA Approach , 2009, 2009 First International Conference on Computational Intelligence, Communication Systems and Networks.

[44]  Sherali Zeadally,et al.  RFID Infrastructure Design: A Case Study of Two Australian RFID Projects , 2009, IEEE Internet Computing.

[45]  Christophe Kolski,et al.  Towards new web services based supervisory systems in complex industrial organizations: Basic principles and case study , 2010, Comput. Ind..

[46]  Euripides G. M. Petrakis,et al.  A survey on industrial vision systems, applications, tools , 2003, Image Vis. Comput..

[47]  Seppo Kuikka,et al.  Value Added Web Services for Industrial Operations and Maintenance , 2005, ICEIS.

[48]  Y. N. Sotskov,et al.  Software for production scheduling based on the mixed [multi]graph approach , 1996 .

[49]  Johan Montagnat,et al.  A Service-Oriented Architecture enabling dynamic service grouping for optimizing distributed workflow execution , 2008, Future Gener. Comput. Syst..

[50]  Christodoulos A. Floudas,et al.  Production scheduling of a large-scale industrial continuous plant: Short-term and medium-term scheduling , 2007 .

[51]  Senem Velipasalar,et al.  Detection of composite events spanning multiple camera views with wireless embedded smart cameras , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[52]  Anastasios D. Doulamis,et al.  Dynamic tracking re-adjustment: a method for automatic tracking recovery in complex visual environments , 2010, Multimedia Tools and Applications.

[53]  Jorge González Fernández,et al.  Transf-ID: Automatic ID and Data Capture for Rail Freight Asset Management , 2009, IEEE Internet Computing.

[54]  Hairong Qi,et al.  Target detection and counting using a progressive certainty map in distributed visual sensor networks , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[55]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[56]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[57]  Jean-Marie Proth,et al.  Scheduling: New trends in industrial environment , 2007, Annu. Rev. Control..

[58]  Klaus Fischer,et al.  Agents in manufacturing: online scheduling and production plant configuration , 1999, Proceedings. First and Third International Symposium on Agent Systems Applications, and Mobile Agents.

[59]  Nikolaos F. Matsatsinis,et al.  A Joint Optimization Algorithm for Dispatching Tasks in Agent-Based Workflow Management Systems , 2008, ICEIS.

[60]  Stefan Fischer,et al.  Using graphical process modeling for realizing SOA programming paradigms in sensor networks , 2009, 2009 Sixth International Conference on Wireless On-Demand Network Systems and Services.

[61]  Jose L. Martinez Lastra,et al.  Runtime Modeling of Flow for Dynamic Deadlock-free Scheduling in Service-oriented Factory Automation Systems , 2009 .

[62]  Peter Wonka,et al.  Curve matching for open 2D curves , 2009, Pattern Recognit. Lett..

[63]  Ki-Tae Kim,et al.  An on-line production scheduler using neural network and simulator based on manufacturing system states , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[64]  Theodora A. Varvarigou,et al.  Workflows Recognition through Multi Agents in Surveillance Systems , 2010, IFIP AI.

[65]  Theodora A. Varvarigou,et al.  Adaptive Algorithms for Interactive Multimedia , 2003, IEEE Multim..

[66]  Klas Nilsson,et al.  Experiments with service-oriented architectures for industrial robotic cells programming , 2009 .

[67]  George Roussos,et al.  RFID Meets the Internet , 2009, IEEE Internet Computing.

[68]  Nikolaos F. Matsatsinis,et al.  Optimizing Resource Conflicts in Workflow Management Systems , 2011, IEEE Transactions on Knowledge and Data Engineering.