Psychophysiologically Based Real-Time Adaptive General Type 2 Fuzzy Modeling and Self-Organizing Control of Operator's Performance Undertaking a Cognitive Task

This paper presents a new modeling and control fuzzy-based framework validated with real-time experiments on human participants experiencing stress via mental arithmetic cognitive tasks identified through psychophysiological markers. The ultimate aim of the modeling/control framework is to prevent performance breakdown in human–computer interactive systems with a special focus on human performance. Two designed modeling/control experiments which consist of carrying-out arithmetic operations of varying difficulty levels were performed by ten participants (operators) in the study. With this new technique, modeling is achieved through a new adaptive, self-organizing, and interpretable modeling framework based on general Type-2 fuzzy sets. This framework is able to learn in real time through the implementation of a restructured performance learning algorithm that identifies important features in the data without the need for prior training. The information learnt by the model is later exploited via an energy model based controller that infers adequate control actions by changing the difficulty level of the arithmetic operations in the human–computer interaction system; these actions being based on the most current psychophysiological state of the subject under study. The real-time implementation of the proposed modeling and control configurations for the human–machine interaction under study shows superior performance as compared to other forms of modeling and control, with minimal intervention in terms of model retraining or parameter retuning to deal with uncertainties, disturbances, and inter/intrasubject parameter variability.

[1]  Peter A. Hancock,et al.  On the Process of Automation Transition in Multitask Human–Machine Systems , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[2]  A. Garde,et al.  Effects of mental and physical demands on heart rate variability during computer work , 2002, European Journal of Applied Physiology.

[3]  Najmedin Meshkati Control rooms' design in industrial facilities , 2003 .

[4]  Frank Chung-Hoon Rhee,et al.  Uncertain Fuzzy Clustering: Interval Type-2 Fuzzy Approach to $C$-Means , 2007, IEEE Transactions on Fuzzy Systems.

[5]  Robert Ivor John,et al.  A Fast Geometric Method for Defuzzification of Type-2 Fuzzy Sets , 2008, IEEE Transactions on Fuzzy Systems.

[6]  B. Lorenz,et al.  Human operator functional states in automated systems: the role of compensatory control strategies , 2003 .

[7]  Chao Li,et al.  Realization of stress detection using psychophysiological signals for improvement of human-computer interactions , 2005, Proceedings. IEEE SoutheastCon, 2005..

[8]  George Panoutsos,et al.  Real-Time Adaptive Automation System Based on Identification of Operator Functional State in Simulated Process Control Operations , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[9]  J. Cummings,et al.  Executive control function: a review of its promise and challenges for clinical research. A report from the Committee on Research of the American Neuropsychiatric Association. , 2002, The Journal of neuropsychiatry and clinical neurosciences.

[10]  Jerry M. Mendel,et al.  Type-2 fuzzy logic systems , 1999, IEEE Trans. Fuzzy Syst..

[11]  C. Isik,et al.  Fuzzy modeling and control based on maximum entropy self-organizing nets and cell state mapping , 1997, 1997 Annual Meeting of the North American Fuzzy Information Processing Society - NAFIPS (Cat. No.97TH8297).

[12]  Christoph F. Eick,et al.  Adaptive Clustering : Better Representatives with Reinforcement Learning , 2005 .

[13]  Jerry M. Mendel,et al.  Type-2 fuzzy sets made simple , 2002, IEEE Trans. Fuzzy Syst..

[14]  M. Er,et al.  Online adaptive fuzzy neural identification and control of a class of MIMO nonlinear systems , 2003, IEEE Trans. Fuzzy Syst..

[15]  Jerry Mendel,et al.  Type-2 Fuzzy Sets and Systems: An Overview [corrected reprint] , 2007, IEEE Computational Intelligence Magazine.

[16]  Chia-Feng Juang,et al.  Data-Driven Interval Type-2 Neural Fuzzy System With High Learning Accuracy and Improved Model Interpretability , 2013, IEEE Transactions on Cybernetics.

[17]  Antonio F. Gómez-Skarmeta,et al.  A fuzzy clustering-based rapid prototyping for fuzzy rule-based modeling , 1997, IEEE Trans. Fuzzy Syst..

[18]  Chia-Feng Juang,et al.  A Fuzzy System Constructed by Rule Generation and Iterative Linear SVR for Antecedent and Consequent Parameter Optimization , 2012, IEEE Transactions on Fuzzy Systems.

[19]  Ying Gao,et al.  Affective assessment of computer users based on processing the pupil diameter signal , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[20]  P. Sleight,et al.  Effects of controlled breathing, mental activity and mental stress with or without verbalization on heart rate variability. , 2000, Journal of the American College of Cardiology.

[21]  Dongrui Wu,et al.  Comparison and practical implementation of type-reduction algorithms for type-2 fuzzy sets and systems , 2011, 2011 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011).

[22]  Daphne N. Yu,et al.  High-resolution EEG mapping of cortical activation related to working memory: effects of task difficulty, type of processing, and practice. , 1997, Cerebral cortex.

[23]  J. Bezdek,et al.  FCM: The fuzzy c-means clustering algorithm , 1984 .

[24]  Charles E. Billings,et al.  Aviation Automation: The Search for A Human-centered Approach , 1996 .

[25]  Robert Ivor John,et al.  Type-2 Fuzzy Logic: Challenges and Misconceptions [Discussion Forum] , 2012, IEEE Computational Intelligence Magazine.

[26]  Jerry M. Mendel,et al.  Interval type-2 fuzzy logic systems , 2000, Ninth IEEE International Conference on Fuzzy Systems. FUZZ- IEEE 2000 (Cat. No.00CH37063).

[27]  Hani Hagras,et al.  Adaptive Non-singleton Type-2 Fuzzy Logic Systems: A Way Forward for Handling Numerical Uncertainties in Real World Applications , 2011, Int. J. Comput. Commun. Control.

[28]  Lotfi A. Zadeh,et al.  The Concepts of a Linguistic Variable and its Application to Approximate Reasoning , 1975 .

[29]  Y. Kuriyagawa,et al.  A modeling of heart rate variability to estimate mental work load , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[30]  P. Hancock,et al.  Stress, Workload, and Fatigue , 2009 .

[31]  A. K. Blangsted,et al.  The effect of mental stress on heart rate variability and blood pressure during computer work , 2004, European Journal of Applied Physiology.

[32]  E. Katkin,et al.  Vagal Rebound and Recovery From Psychological Stress , 2001, Psychosomatic medicine.

[33]  Mahdi Mahfouf,et al.  Pupil diameter size marker for incremental mental stress detection , 2015, 2015 17th International Conference on E-health Networking, Application & Services (HealthCom).

[34]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[35]  F. Craik,et al.  Relations among memory performance, mental workload and cardiovascular responses. , 1996, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[36]  R. Cohen,et al.  Power spectrum analysis of heart rate fluctuation: a quantitative probe of beat-to-beat cardiovascular control. , 1981, Science.

[37]  M. Posner,et al.  Anterior cingulate cortex regulation of sympathetic activity. , 2003, Brain : a journal of neurology.

[38]  Myriam Regattieri Delgado,et al.  General Type-2 Fuzzy Inference Systems: Analysis, Design and Computational Aspects , 2007, 2007 IEEE International Fuzzy Systems Conference.

[39]  Mahdi Mahfouf,et al.  A new computer control system for mental stress management using fuzzy logic , 2015, 2015 IEEE International Conference on Evolving and Adaptive Intelligent Systems (EAIS).

[40]  Mohammad R. Akbarzadeh-Totonchi,et al.  Mental Stress Detection Based on Soft Computing Techniques , 2011, 2011 IEEE International Conference on Bioinformatics and Biomedicine.

[41]  Michael E. Smith,et al.  Monitoring Task Loading with Multivariate EEG Measures during Complex Forms of Human-Computer Interaction , 2001, Hum. Factors.

[42]  I. Burhan Türksen,et al.  Discrete Interval Type 2 Fuzzy System Models Using Uncertainty in Learning Parameters , 2007, IEEE Transactions on Fuzzy Systems.

[43]  Mahdi Mahfouf,et al.  Interval Type-2 Fuzzy Logic Adaptive Modelling for Human Operators Undergoing Mental Stress , 2014 .

[44]  M. Brdys,et al.  Adaptive fuzzy modelling and control for discrete-time nonlinear uncertain systems , 2005, Proceedings of the 2005, American Control Conference, 2005..

[45]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..