Predicting human design decisions with deep recurrent neural network combining static and dynamic data

Computational modeling of the human sequential design process and successful prediction of future design decisions are fundamental to design knowledge extraction, transfer, and the development of artificial design agents. However, it is often difficult to obtain designer-related attributes (static data) in design practices, and the research based on combining static and dynamic data (design action sequences) in engineering design is still underexplored. This paper presents an approach that combines both static and dynamic data for human design decision prediction using two different methods. The first method directly combines the sequential design actions with static data in a recurrent neural network (RNN) model, while the second method integrates a feed-forward neural network that handles static data separately, yet in parallel with RNN. This study contributes to the field from three aspects: (a) we developed a method of utilizing designers’ cluster information as a surrogate static feature to combine with a design action sequence in order to tackle the challenge of obtaining designer-related attributes; (b) we devised a method that integrates the function–behavior–structure design process model with the one-hot vectorization in RNN to transform design action data to design process stages where the insights into design thinking can be drawn; (c) to the best of our knowledge, it is the first time that two methods of combining static and dynamic data in RNN are compared, which provides new knowledge about the utility of different combination methods in studying sequential design decisions. The approach is demonstrated in two case studies on solar energy system design. The results indicate that with appropriate kernel models, the RNN with both static and dynamic data outperforms traditional models that only rely on design action sequences, thereby better supporting design research where static features, such as human characteristics, often play an important role.

[1]  Corey Schimpf,et al.  A Computer-Aided Design Based Research Platform for Design Thinking Studies , 2019, Journal of Mechanical Design.

[2]  John S. Gero,et al.  An Empirical Foundation for Design Patterns in Parametric Design , 2015 .

[3]  Christopher McComb,et al.  Capturing Human Sequence-Learning Abilities in Configuration Design Tasks through Markov Chains , 2017 .

[4]  Thurston Sexton,et al.  Learning an Optimization Algorithm through Human Design Iterations , 2016, 1608.06984.

[5]  Andrew G. Barto,et al.  Optimal learning: computational procedures for bayes-adaptive markov decision processes , 2002 .

[6]  John S. Gero,et al.  Analysing Design Protocols: Development of Methods and Tools , 2011 .

[7]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[8]  P. Papalambros,et al.  Measuring Systems Engineering and Design Thinking Attitudes , 2019, Proceedings of the Design Society: International Conference on Engineering Design.

[9]  Jeff W T Kan,et al.  Using the FBS Ontology to Capture Semantic Design Information in Design Protocol Studies , 2007 .

[10]  Zhenghui Sha,et al.  Automatic Clustering of Sequential Design Behaviors , 2018 .

[11]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[12]  Anuj Sharma A combined static and dynamic feature extraction technique to recognize handwritten digits , 2014, Vietnam Journal of Computer Science.

[13]  Aitor Almeida,et al.  Predicting Human Behaviour with Recurrent Neural Networks , 2018 .

[14]  Lei Shi,et al.  Computer aided design user interaction as a sensor for monitoring engineers and the engineering design process , 2016 .

[15]  William P. Anthony,et al.  The Influence of Tacit Knowledge and Collective Mind on Strategic Planning , 1998 .

[16]  Volker Tresp,et al.  Predicting Clinical Events by Combining Static and Dynamic Information Using Recurrent Neural Networks , 2016, 2016 IEEE International Conference on Healthcare Informatics (ICHI).

[17]  Tyson R. Browning,et al.  Design Process Sequencing With Competent Genetic Algorithms , 2007 .

[18]  Christopher McComb,et al.  Mining Process Heuristics From Designer Action Data via Hidden Markov Models , 2017 .

[19]  C. Lingard,et al.  Book Review: The Challenge of Red China , 1946 .

[20]  Zachary Chase Lipton A Critical Review of Recurrent Neural Networks for Sequence Learning , 2015, ArXiv.

[21]  Ali A. Yassine,et al.  An Introduction to Modeling and Analyzing Complex Product Development Processes Using the Design Structure Matrix (DSM) Method , 2001 .

[22]  Masaki Suwa,et al.  What do architects and students perceive in their design sketches? A protocol analysis , 1997 .

[23]  Jitesh H. Panchal,et al.  Behavioral Experimentation and Game Theory in Engineering Systems Design , 2015 .

[24]  Gabriela Goldschmidt Linkography: Unfolding the Design Process , 2014 .

[25]  Trupti M. Kodinariya,et al.  Review on determining number of Cluster in K-Means Clustering , 2013 .

[26]  John S. Gero,et al.  The Situated Function — Behaviour — Structure Framework , 2004 .

[27]  Jitesh H. Panchal,et al.  Understanding Design Decisions Under Competition Using Games With Information Acquisition and a Behavioral Experiment , 2017 .

[28]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[29]  Zhenghui Sha,et al.  A Deep Learning Based Approach to Predict Sequential Design Decisions , 2019, Volume 1: 39th Computers and Information in Engineering Conference.

[30]  Jie Chao,et al.  Learning and teaching engineering design through modeling and simulation on a CAD platform , 2018, Comput. Appl. Eng. Educ..

[31]  John S. Gero,et al.  Design Prototypes: A Knowledge Representation Schema for Design , 1990, AI Mag..

[32]  Timothy W. Simpson,et al.  Two Applications of Design As a Sequential Decision Process , 2017, Design Automation Conference.

[33]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[34]  Jitesh H. Panchal,et al.  An Experimental Study of Human Decisions in Sequential Information Acquisition in Design: Impact of Cost and Task Complexity , 2019, Smart Innovation, Systems and Technologies.

[35]  Timothy W. Simpson,et al.  Design as a sequential decision process , 2017 .

[36]  Dong Yu,et al.  Deep Learning: Methods and Applications , 2014, Found. Trends Signal Process..

[37]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[38]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[39]  Christopher McComb,et al.  Design Strategy Transfer in Cognitively-Inspired Agents , 2018, Volume 2A: 44th Design Automation Conference.

[40]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[41]  Nicky J Welton,et al.  Exploring the Research Decision Space: The Expected Value of Information for Sequential Research Designs , 2010, Medical decision making : an international journal of the Society for Medical Decision Making.

[42]  Sanyam Shukla,et al.  Analysis of k-Fold Cross-Validation over Hold-Out Validation on Colossal Datasets for Quality Classification , 2016, 2016 IEEE 6th International Conference on Advanced Computing (IACC).

[43]  Robert P. Smith,et al.  A predictive model of sequential iteration in engineering design , 1997 .

[44]  Jami J. Shah,et al.  Mechanical Engineering Design Complexity Metrics: Size, Coupling, and Solvability , 2010 .

[45]  Katia Kermanidis,et al.  Combining LSTM and Feed Forward Neural Networks for Conditional Rhythm Composition , 2017, EANN.

[46]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..