Design with shape grammars and reinforcement learning

Shape grammars are a powerful and appealing formalism for automatic shape generation in computer-based design systems. This paper presents a proposal complementing the generative power of shape grammars with reinforcement learning techniques. We use simple (naive) shape grammars capable of generating a large variety of different designs. In order to generate those designs that comply with given design requirements, the grammar is subject to a process of machine learning using reinforcement learning techniques. Based on this method, we have developed a system for architectural design, aimed at generating two-dimensional layout schemes of single-family housing units. Using relatively simple grammar rules, we learn to generate schemes that satisfy a set of requirements stated in a design guideline. Obtained results are presented and discussed.

[1]  Jonathan Cagan,et al.  Innovative dome design: Applying geodesic patterns with shape annealing , 1997, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[2]  Dieter Fensel,et al.  Knowledge Engineering: Principles and Methods , 1998, Data Knowl. Eng..

[3]  U Flemming,et al.  More Than the Sum of Parts: The Grammar of Queen Anne Houses , 1987 .

[4]  Kenneth N. Brown,et al.  Describing process plans as the formal semantics of a language of shape , 1996, Artif. Intell. Eng..

[5]  Abdul Sattar,et al.  Reinforcement learning of iterative behaviour with multiple sensors , 2004, Applied Intelligence.

[6]  John S. Gero,et al.  Evolutionary learning of novel grammars for design improvement , 1994, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[7]  T W Knight,et al.  Shape Grammars: Six Types , 1999 .

[8]  Jonathan Cagan,et al.  Languages and semantics of grammatical discrete structures , 1999, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[9]  John S. Gero,et al.  Evolving Building Blocks for Design Using Genetic Engineering: A Formal Approach , 1996 .

[10]  Alan de Pennington,et al.  COMBINING EVOLUTIONARY ALGORITHMS AND SHAPE GRAMMARS TO GENERATE BRANDED PRODUCT DESIGN , 2006 .

[11]  Sebastian Thrun,et al.  Issues in Using Function Approximation for Reinforcement Learning , 1999 .

[12]  Vivek S. Borkar,et al.  Stochastic Approximation for Nonexpansive Maps: Application to Q-Learning Algorithms , 1997, SIAM J. Control. Optim..

[13]  M. Kosorok,et al.  Reinforcement learning design for cancer clinical trials , 2009, Statistics in medicine.

[14]  David Silver,et al.  Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (2008) Achieving Master Level Play in 9 × 9 Computer Go , 2022 .

[15]  George Stiny,et al.  Shape: Talking about Seeing and Doing , 2006 .

[16]  Mehmet Emin Aydin,et al.  Dynamic job-shop scheduling using reinforcement learning agents , 2000, Robotics Auton. Syst..

[17]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[18]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[19]  Herbert A. Simon,et al.  The Structure of Ill Structured Problems , 1973, Artif. Intell..

[20]  Nada Y. Philip,et al.  Medical QoS provision based on reinforcement learning in ultrasound streaming over 3.5G wireless systems , 2009, IEEE Journal on Selected Areas in Communications.

[21]  Kemal Leblebicioglu,et al.  Free gait generation with reinforcement learning for a six-legged robot , 2008, Robotics Auton. Syst..

[22]  George Stiny,et al.  Pictorial and Formal Aspects of Shape and Shape Grammars , 1975 .

[23]  G. Stiny Introduction to Shape and Shape Grammars , 1980 .

[24]  Scott Curland Chase,et al.  A model for user interaction in grammar-based design systems , 2002 .

[25]  Ming Xi Tang,et al.  Artificial Intelligence for Engineering Design, Analysis and Manufacturing Evolving Product Form Designs Using Parametric Shape Grammars Integrated with Genetic Programming Evolving Product Form Designs Using Parametric Shape Grammars Integrated with Genetic Programming , 2022 .

[26]  Dimitri P. Bertsekas,et al.  Reinforcement Learning for Dynamic Channel Allocation in Cellular Telephone Systems , 1996, NIPS.

[27]  Ashutosh Saxena,et al.  High speed obstacle avoidance using monocular vision and reinforcement learning , 2005, ICML.

[28]  Michael P. Wellman,et al.  Nash Q-Learning for General-Sum Stochastic Games , 2003, J. Mach. Learn. Res..

[29]  José Pinto Duarte,et al.  A Discursive Grammar for Customizing Mass Housing - The case of Siza´s houses at Malagueira , 2005, eCAADe proceedings.

[30]  Yi-Chi Wang,et al.  Application of reinforcement learning for agent-based production scheduling , 2005, Eng. Appl. Artif. Intell..

[31]  S.-M. Senouci,et al.  Dynamic channel assignment in cellular networks: a reinforcement learning solution , 2003, 10th International Conference on Telecommunications, 2003. ICT 2003..

[32]  Bir Bhanu,et al.  Closed-Loop Object Recognition Using Reinforcement Learning , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Charles M. Eastman,et al.  Cognitive processes and ill-defined problems: a case study from design , 1969, IJCAI 1969.

[34]  William J. Mitchell,et al.  The Palladian Grammar , 1978 .

[35]  Bojan Nemec,et al.  Learning of a ball-in-a-cup playing robot , 2010, 19th International Workshop on Robotics in Alpe-Adria-Danube Region (RAAD 2010).

[36]  H. Sebastian Seung,et al.  Stochastic policy gradient reinforcement learning on a simple 3D biped , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[37]  Jay McCormack,et al.  Speaking the Buick Language: Capturing, Understanding, and Exploring Brand Identity With Shape Grammars , 2004 .

[38]  Jonathan Cagan,et al.  Influencing generative design through continuous evaluation: Associating costs with the coffeemaker shape grammar , 1999, Artif. Intell. Eng. Des. Anal. Manuf..

[39]  Jun Morimoto,et al.  Learning CPG-based Biped Locomotion with a Policy Gradient Method: Application to a Humanoid Robot , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[40]  Andrew Tridgell,et al.  Learning to Play Chess Using Temporal Differences , 2000, Machine Learning.

[41]  Jonathan Schaeffer,et al.  Temporal Difference Learning Applied to a High-Performance Game-Playing Program , 2001, IJCAI.

[42]  Lucas Paletta,et al.  Active object recognition by view integration and reinforcement learning , 2000, Robotics Auton. Syst..

[43]  Pieter Abbeel,et al.  An Application of Reinforcement Learning to Aerobatic Helicopter Flight , 2006, NIPS.

[44]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[45]  Oliver Kroemer,et al.  Towards Motor Skill Learning for Robotics , 2007, ISRR.

[46]  Gerald Tesauro,et al.  Temporal Difference Learning and TD-Gammon , 1995, J. Int. Comput. Games Assoc..

[47]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[48]  Pieter Spronck,et al.  Monte-Carlo Tree Search in Settlers of Catan , 2009, ACG.

[49]  Kutluyil Dogançay,et al.  Dynamic channel allocation for mobile cellular traffic using reduced-state reinforcement learning , 2004, 2004 IEEE Wireless Communications and Networking Conference (IEEE Cat. No.04TH8733).

[50]  Johannes Fürnkranz,et al.  Learning the Piece Values for Three Chess Variants , 2008, J. Int. Comput. Games Assoc..

[51]  Jonathan Cagan,et al.  Capturing a rebel: modeling the Harley-Davidson brand through a motorcycle shape grammar , 2002 .

[52]  George Stiny,et al.  Shape Grammars and the Generative Specification of Painting and Sculpture , 1971, IFIP Congress.

[53]  Anthony Brabazon,et al.  Evolutionary design using grammatical evolution and shape grammars: designing a shelter , 2010 .

[54]  Stefan Schaal,et al.  Learning variable impedance control , 2011, Int. J. Robotics Res..

[55]  Sebastian Thrun,et al.  Learning to Play the Game of Chess , 1994, NIPS.

[56]  Wei Zhang,et al.  A Reinforcement Learning Approach to job-shop Scheduling , 1995, IJCAI.

[57]  G. Stiny Shape , 1999 .

[58]  Jonathan Cagan,et al.  Optimally Directed Shape Generation by Shape Annealing , 1993 .

[59]  Minoru Asada,et al.  Purposive Behavior Acquisition for a Real Robot by Vision-Based Reinforcement Learning , 2005, Machine Learning.

[60]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[61]  Colin de la Higuera,et al.  Grammatical Inference: Learning Automata and Grammars , 2010 .

[62]  Gülen Çaǧdaş,et al.  A shape grammar model for designing row-houses , 1996 .

[63]  T. Knight,et al.  Applications in architectural design and education and practice , 1999 .

[64]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[65]  Terrence J. Sejnowski,et al.  Learning to evaluate Go positions via temporal difference methods , 2001 .

[66]  Bruno Scherrer,et al.  Building Controllers for Tetris , 2009, J. Int. Comput. Games Assoc..

[67]  Maja J. Mataric,et al.  Reinforcement Learning in the Multi-Robot Domain , 1997, Auton. Robots.

[68]  Luke Fletcher,et al.  Reinforcement learning for a vision based mobile robot , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[69]  Jonathan Cagan,et al.  The design of novel roof trusses with shape annealing: assessing the ability of a computational method in aiding structural designers with varying design intent , 1999 .

[70]  Leslie Pack Kaelbling,et al.  Effective reinforcement learning for mobile robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[71]  Kristina Shea,et al.  Design-to-fabrication automation for the cognitive machine shop , 2010, Adv. Eng. Informatics.

[72]  Joel Veness,et al.  Bootstrapping from Game Tree Search , 2009, NIPS.

[73]  Robert Levinson,et al.  Chess Neighborhoods, Function Combination, and Reinforcement Learning , 2000, Computers and Games.

[74]  Walter Daelemans Colin de la Higuera: Grammatical inference: learning automata and grammars , 2011, Machine Translation.