Improving Human-Robot-Interaction Utilizing Learning and Intelligence

Several decades of development in the fields of robotics and automation have resulted in human–robot interaction is commonplace, and the subject of intense study. These interactions are particularly prevalent in manufacturing, where human operators (HOs) have been employed in numerous robotics and automation tasks. The presence of HOs continues to be a source of uncertainty in such systems, despite the study of human factors, in an attempt to better understand these variations in performance. Concurrent developments in intelligent manufacturing present opportunities for adaptability within robotic control. This article examines relevant human factors and develops a framework for integrating the necessary elements of intelligent control and data processing to provide appropriate adaptability to robotic elements, consequently improving collaborative interaction with human colleagues. A neural network-based learning approach is used to predict the influence on human task performance and use these predictions to make informed changes to programed behavior, and a methodology developed to explore the application of learning techniques to this area further. This article is supported by an example case study, in which a simulation model is used to explore the application of the developed system, and its performance in a real-world production scenario. The simulation results reveal that adaptability can be realized with some relatively simple techniques and models if applied in the right manner and that such adaptability is helpful to tackle the issue of performance disparity in manufacturing operations. Note to Practitioners—This article presents research into the application of intelligent methodologies to this problem and builds a framework to describe how this information can be captured, generated, and used within manufacturing production processes. This framework helps identify which areas require further research and serves as a basis for the development of a methodology, by which a control system may enable adaptable behavior to reduce the impact of human performance variation and improve human–machine interaction (HMI). This article also presents a simulation-based case study to support the development and evaluate the presented control system on a representative real-world problem. The methodology makes use of a machine-learning approach to identify the complex influence of several identified human factors on human performance. This knowledge can be used to adjust the robotic behavior to match the predicted performance of multiple different operators over different scenarios. This adaptability reduces performance disparity by reducing idle times and enabling leaner production through workpiece-in-progress reduction. Future work will focus on expanding the intelligent capabilities of the proposed system to deal with uncertainty and improve decision-making ability.

[1]  John E. Laird,et al.  A Standard Model of the Mind: Toward a Common Computational Framework across Artificial Intelligence, Cognitive Science, Neuroscience, and Robotics , 2017, AI Mag..

[2]  Bogdan-Constantin Pirvu,et al.  Engineering insights from an anthropocentric cyber-physical system: A case study for an assembly station , 2016 .

[3]  Jay Lee,et al.  A Cyber-Physical Systems architecture for Industry 4.0-based manufacturing systems , 2015 .

[4]  G. R. J. Hockey Compensatory control in the regulation of human performance under stress and high workload: A cognitive-energetical framework , 1997, Biological Psychology.

[5]  Iveta Zolotova,et al.  Comparison between multi-class classifiers and deep learning with focus on industry 4.0 , 2016, 2016 Cybernetics & Informatics (K&I).

[6]  Jay Lee,et al.  Smart Agents in Industrial Cyber–Physical Systems , 2016, Proceedings of the IEEE.

[7]  P A Hancock,et al.  Influence of task demand characteristics on workload and performance. , 1995, The International journal of aviation psychology.

[8]  Andrea Vinci,et al.  Pattern Detection in Cyber-Physical Systems , 2015, ANT/SEIT.

[9]  A. Moniz,et al.  Robots Working with Humans or Humans Working with Robots? Searching for Social Dimensions in New Human-Robot Interaction in Industry , 2016 .

[10]  Konrad Wegener,et al.  Human-robot Collaboration in Decentralized Manufacturing Systems: An Approach for Simulation-based Evaluation of Future Intelligent Production ☆ , 2017 .

[11]  Joshua B. Tenenbaum,et al.  Beating the World's Best at Super Smash Bros. with Deep Reinforcement Learning , 2017, ArXiv.

[12]  R. Clarisse,et al.  Time-of-day and day-of-week effects on mnemonic performance. , 1999, Chronobiology international.

[13]  Amanda K. Emo,et al.  Emotional intelligence, personality, and task-induced stress. , 2006, Journal of experimental psychology. Applied.

[14]  A. Lowden,et al.  Eating and shift work - effects on habits, metabolism and performance. , 2010, Scandinavian journal of work, environment & health.

[15]  H. Berthoud,et al.  Interactions between the “cognitive” and “metabolic” brain in the control of food intake , 2007, Physiology & Behavior.

[16]  Bruce Blumberg,et al.  A Layered Brain Architecture for Synthetic Creatures , 2001, IJCAI.

[17]  P A Bell,et al.  Effects of Noise and Heat Stress on Primary and Subsidiary Task Performance , 1978, Human factors.

[18]  László Monostori,et al.  ScienceDirect Variety Management in Manufacturing . Proceedings of the 47 th CIRP Conference on Manufacturing Systems Cyber-physical production systems : Roots , expectations and R & D challenges , 2014 .

[19]  Danica Kragic,et al.  Human-Robot Collaboration: From Psychology to Social Robotics , 2017, ArXiv.

[20]  Syed Imran Shafiq,et al.  Virtual Engineering Factory: Creating Experience Base for Industry 4.0 , 2016, Cybern. Syst..

[21]  José Barbosa,et al.  Standardization in cyber-physical systems: The ARUM case , 2015, 2015 IEEE International Conference on Industrial Technology (ICIT).

[22]  Carme Torras,et al.  Learning Physical Collaborative Robot Behaviors From Human Demonstrations , 2016, IEEE Transactions on Robotics.

[23]  D. Dawson,et al.  Enhancement of nighttime alertness and performance with bright ambient light , 1990, Physiology & Behavior.

[24]  W. Patrick Neumann,et al.  Integrating human factors into discrete event simulation: a proactive approach to simultaneously design for system performance and employees’ well being , 2016 .

[25]  Diolino J. Santos Filho,et al.  Modeling of reconfigurable distributed manufacturing control systems , 2015 .

[26]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[27]  Yue Wang,et al.  Human-Collaborative Schemes in the Motion Control of Single and Multiple Mobile Robots Mobile robot , 2017 .

[28]  Gerard A. Kerkhof,et al.  Inter-individual differences in the human circadian system: A review , 1985, Biological Psychology.

[29]  Conrad S. Tucker,et al.  Machine learning classification of design team members' body language patterns for real time emotional state detection , 2015 .

[30]  Luc Bongaerts,et al.  Reference architecture for holonic manufacturing systems: PROSA , 1998 .

[31]  Stephen M. Fiore,et al.  Enabling robotic social intelligence by engineering human social-cognitive mechanisms , 2017, Cognitive Systems Research.

[32]  M. J. Blake,et al.  Time of day effects on performance in a range of tasks , 1967 .

[33]  Vladimír Marík,et al.  Industrial adoption of agent-based technologies , 2005, IEEE Intelligent Systems.

[34]  M. Koslowsky,et al.  Meta-analysis of the relationship between total sleep deprivation and performance. , 1992, Chronobiology international.

[35]  S. Nieuwenhuis,et al.  Mental fatigue and task control: planning and preparation. , 2000, Psychophysiology.

[36]  Ehud Sharlin,et al.  Evaluating Human-Robot Interaction , 2011, Int. J. Soc. Robotics.

[37]  A. Pack,et al.  Cumulative sleepiness, mood disturbance, and psychomotor vigilance performance decrements during a week of sleep restricted to 4-5 hours per night. , 1997, Sleep.

[38]  Jonathan Evans,et al.  Science Perspectives on Psychological , 2022 .

[39]  Erwin R. Boer,et al.  Toward an Integrated Model of Driver Behavior in Cognitive Architecture , 2001 .

[40]  Allison S. Gabriel,et al.  The effects of intrinsic and extrinsic sources of motivation on well-being depend on time of day: The moderating effects of workday accumulation , 2015 .

[41]  John R. Anderson ACT: A simple theory of complex cognition. , 1996 .

[42]  Paulo Leitão,et al.  Petri net based Methodology for the Development of Collaborative Production Systems , 2006, 2006 IEEE Conference on Emerging Technologies and Factory Automation.

[43]  Wenwu Yu,et al.  An Overview of Recent Progress in the Study of Distributed Multi-Agent Coordination , 2012, IEEE Transactions on Industrial Informatics.

[44]  Jessie Y. C. Chen,et al.  A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction , 2011, Hum. Factors.

[45]  Tyler H. Shaw,et al.  From ‘automation’ to ‘autonomy’: the importance of trust repair in human–machine interaction , 2018, Ergonomics.

[46]  Luigi Carro,et al.  Distributed real-time embedded systems: Recent advances, future trends and their impact on manufacturing plant control , 2007, Annu. Rev. Control..

[47]  Jeremy L. Rickli,et al.  A Framework for Collaborative Robot (CoBot) Integration in Advanced Manufacturing Systems , 2016 .

[48]  R. Sundar,et al.  A Review on Lean Manufacturing Implementation Techniques , 2018 .

[49]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[50]  Edward A. Lee Cyber Physical Systems: Design Challenges , 2008, 2008 11th IEEE International Symposium on Object and Component-Oriented Real-Time Distributed Computing (ISORC).

[51]  Allen Newell,et al.  SOAR: An Architecture for General Intelligence , 1987, Artif. Intell..

[52]  Christian Brecher,et al.  Self-optimizing Production Systems☆ , 2016 .

[53]  D. Vernon Artificial Cognitive Systems: A Primer , 2014 .

[54]  Lenz Belzner,et al.  Optimization of global production scheduling with deep reinforcement learning , 2018 .

[55]  P A Hancock,et al.  Imposing limits on autonomous systems , 2017, Ergonomics.

[56]  Robin R. Murphy,et al.  Human-Robot Interaction , 2012 .

[57]  Jay Lee,et al.  Introduction to cyber manufacturing , 2016 .

[58]  Eric Mayer,et al.  Stress And Human Performance , 2016 .

[59]  Peter Pagel,et al.  Cognitive Computing , 2018, Informatik-Spektrum.

[60]  Andreas Buchheim,et al.  Investigation of an automated dry fiber preforming process for an aircraft fuselage demonstrator using collaborating robots , 2016 .

[61]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[62]  R. Wilkinson,et al.  Some factors influencing the effect of environmental stressors upon performance. , 1969, Psychological bulletin.

[63]  Lihui Wang,et al.  Gesture recognition for human-robot collaboration: A review , 2017, International Journal of Industrial Ergonomics.

[64]  Axel Cleeremans,et al.  The relationship between human agency and embodiment , 2015, Consciousness and Cognition.

[65]  Ronay Ak,et al.  A Survey of the Advancing Use and Development of Machine Learning in Smart Manufacturing. , 2018, Journal of manufacturing systems.

[66]  Levent Gürgen,et al.  Self-aware cyber-physical systems and applications in smart buildings and cities , 2013, 2013 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[67]  Stephen M. Fiore,et al.  Leveraging Social Judgment Theory to Examine the Relationship between Social Cues and Signals in Human-Robot Interactions , 2014 .

[68]  Daqiang Zhang,et al.  Towards smart factory for industry 4.0: a self-organized multi-agent system with big data based feedback and coordination , 2016, Comput. Networks.

[69]  Yi Wu,et al.  Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments , 2017, NIPS.

[70]  Inge Zijdewind,et al.  Motor fatigue and cognitive task performance in humans , 2002, The Journal of physiology.

[71]  Paulo Leitão,et al.  ADACOR: A holonic architecture for agile and adaptive manufacturing control , 2006, Comput. Ind..

[72]  Jacques Duchateau,et al.  Translating Fatigue to Human Performance. , 2016, Medicine and science in sports and exercise.

[73]  Antonio Franchi,et al.  Human-Collaborative Schemes in the Motion Control of Single and Multiple Mobile Robots Mobile robot , 2017 .

[74]  Ahmed K. Noor,et al.  Potential of Cognitive Computing and Cognitive Systems , 2014 .

[75]  Don Joven Agravante,et al.  Collaborative human-humanoid carrying using vision and haptic sensing , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[76]  John E. Laird,et al.  Cognitive Architecture Requirements for Achieving AGI , 2010, AGI 2010.

[77]  Günther Schuh,et al.  Collaboration Mechanisms to Increase Productivity in the Context of Industrie 4.0 , 2014 .

[78]  Stephen M. Fiore,et al.  Towards Modeling Social-Cognitive Mechanisms in Robots to Facilitate Human-Robot Teaming , 2013, Proceedings of the Human Factors and Ergonomics Society Annual Meeting.