A theory of reverse engineering and its application to Boolean systems

To reverse engineer a system is to infer how its underlying mechanism works. This paper presents a theory of the process, which postulates that individuals rely on an initial strategy of either focusing on the outputs of a system one by one, or on the components of the system one by one. They then try to assemble the system guided by both local and global constraints. The theory predicts that three main factors should affect the difficulty of reverse engineering: the number of variable components in the system, the number of their settings that yield an output, and, most importantly, the interdependence of components on one another in yielding outputs. Five experiments corroborated these predictions, using a test bed of electric light circuits and water-flow systems based on Boolean logic.

[1]  L. Hood,et al.  Reverse Engineering of Biological Complexity , 2007 .

[2]  J. Fleck,et al.  The use of verbal protocols as data: An analysis of insight in the candle problem , 2004, Memory & cognition.

[3]  Timothy D. Wilson,et al.  Telling more than we can know: Verbal reports on mental processes. , 1977 .

[4]  Geoffrey P. Goodwin,et al.  Conceptual illusions , 2010, Cognition.

[5]  J. Kruschke,et al.  ALCOVE: an exemplar-based connectionist model of category learning. , 1992, Psychological review.

[6]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[7]  P. Chandler,et al.  Why Some Material Is Difficult to Learn , 1994 .

[8]  H. Simon,et al.  The sciences of the artificial (3rd ed.) , 1996 .

[9]  Dietrich Dorner,et al.  The Logic of Failure: Why Things Go Wrong and What We Can Do to Make Them Right , 1996 .

[10]  Gary Klein,et al.  Decision-Centered Training , 1995 .

[11]  R. Jeffrey.  Formal Logic: Its Scope and Limits , 1981 .

[12]  P. Johnson-Laird.  Flying bicycles: How the Wright brothers invented the airplane , 2005 .

[13]  R. Nosofsky,et al.  Comparing modes of rule-based classification learning: A replication and extension of Shepard, Hovland, and Jenkins (1961) , 1994, Memory & cognition.

[14]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[15]  P. Johnson-Laird,et al.  PSYCHOLOGICAL SCIENCE Research Article HOW DIAGRAMS CAN IMPROVE REASONING , 2022 .

[16]  Steven A. Sloman,et al.  Beyond covariation: Cues to causal structure. , 2010 .

[17]  N. Wetherick.  The Computer and the Mind: An Introduction to Cognitive Science. By Philip N. Johnson‐Laird. Pp. 444. (Harvard University Press, 1988.) £23.50. (Also available from Fontana (1988), £6.95.) , 1988 .

[18]  David M. Sobel,et al.  Causal learning mechanisms in very young children: two-, three-, and four-year-olds infer causal relations from patterns of variation and covariation. , 2001, Developmental psychology.

[19]  Wolfgang A. Linke,et al.  Reverse engineering of the giant muscle protein titin , 2002, Nature.

[20]  Jeffrey D. Ullman,et al.  Formal languages and their relation to automata , 1969, Addison-Wesley series in computer science and information processing.

[21]  P. N. Johnson-Laird,et al.  Strategic changes in problem solving , 2013 .

[22]  H. Simon,et al.  Perception in chess , 1973 .

[23]  David Klahr,et al.  Dual Space Search During Scientific Reasoning , 1988, Cogn. Sci..

[24]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[25]  P. Johnson-Laird,et al.  Illusory inferences: a novel class of erroneous deductions , 1999, Cognition.

[26]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[27]  Michael R. Waldmann,et al.  How temporal assumptions influence causal judgments , 2002, Memory & cognition.

[28]  Jonathan Evans,et al.  Human Reasoning: The Psychology Of Deduction , 1993 .

[29]  P N Johnson-Laird,et al.  Reasoning about relations. , 2005, Psychological review.

[30]  J. Sweller,et al.  Cognitive Load Theory and Complex Learning: Recent Developments and Future Directions , 2005 .

[31]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[32]  J. G. Taylor,et al.  From Wetware to Hardware: Reverse Engineering Using Probabilistic RAMs , 1992 .

[33]  S. Phillips,et al.  Processing capacity defined by relational complexity: implications for comparative, developmental, and cognitive psychology. , 1998, The Behavioral and brain sciences.

[34]  Philip N. Johnson-Laird,et al.  Strategies in sentential reasoning , 2002, Cogn. Sci..

[35]  A. Gopnik,et al.  Causal learning across domains. , 2004, Developmental psychology.

[36]  Philip N. Johnson-Laird,et al.  Diagnosis of Ambiguous Faults in Simple Networks , 2005 .

[37]  William B. Rouse,et al.  Human Problem Solving in Fault Diagnosis Tasks , 1986 .

[38]  Ralph R. Martin,et al.  Reverse engineering of geometric models - an introduction , 1997, Comput. Aided Des..

[39]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[40]  Howard Greisdorf,et al.  Exploring Science: The Cognition and Development of Discovery Processes , 2003, J. Documentation.

[41]  P. N. Johnson-Laird,et al.  Mental models of Boolean concepts , 2011, Cognitive Psychology.

[42]  Eldad Eilam,et al.  Reversing: Secrets of Reverse Engineering , 2005 .

[43]  Peter A. White,et al.  Use of prior beliefs in the assignment of causal roles: Causal powers versus regularity-based accounts , 1995, Memory & cognition.

[44]  Ronaldo Vigo,et al.  Journal of Mathematical Psychology Categorical Invariance and Structural Complexity in Human Concept Learning , 2022 .

[45]  Daniel L. Schwartz,et al.  Analog Imagery in Mental Model Reasoning: Depictive Models , 1996, Cognitive Psychology.

[46]  M. Hegarty.  Mental animation: inferring motion from static displays of mechanical systems. , 1992, Journal of experimental psychology. Learning, memory, and cognition.

[47]  R. Shepard,et al.  Learning and memorization of classifications. , 1961 .

[48]  P. Chandler,et al.  Learning and understanding science instructional material. , 2003 .

[49]  J. Hasty,et al.  Reverse engineering gene networks: Integrating genetic perturbations with dynamical modeling , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[50]  D. Medin,et al.  SUSTAIN: a network model of category learning. , 2004, Psychological review.

[51]  Naomi Miyake,et al.  Constructive Interaction and the Iterative Process of Understanding , 1986, Cogn. Sci..

[52]  S. Sloman,et al.  The advantage of timely intervention. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[53]  Kevin M. Brooks,et al.  Thoughts beyond words : When language overshadows insight , 1993 .

[54]  J. Feldman.  An algebra of human concept learning , 2006 .

[55]  H. Simon,et al.  Models Of Man : Social And Rational , 1957 .

[56]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[57]  D. Gopher,et al.  Attention and performance XVII: Cognitive regulation of performance: Interaction of theory and application. , 1999 .

[58]  Gary Klein,et al.  Developing Expertise in Decision Making , 1997 .

[59]  Jacob Feldman,et al.  Minimization of Boolean complexity in human concept learning , 2000, Nature.

[60]  S. Kay.  On the Nature of Expertise. , 1992 .

[61]  John H. Holland,et al.  Induction: Processes of Inference, Learning, and Discovery , 1987, IEEE Expert.

[62]  P. Johnson-Laird.  How We Reason , 2006 .

[63]  P. Johnson-Laird,et al.  Naïve deontics: A theory of meaning, representation, and reasoning , 2005, Cognitive Psychology.

[64]  Richard Reviewer-Granger.  Unified Theories of Cognition , 1991, Journal of Cognitive Neuroscience.