Parameter Search for Aesthetic Design and Composition

This thesis is about algorithmic creation in the arts – where an artist, designer or composer uses a formal generative process to assist in crafting forms and patterns – and approaches to finding effective input parameter values to these generative processes for aesthetic ends. Framed in three practical studies, approaches to navigating the aesthetic possibilities of generative processes in sound and visuals are presented, and strategies for eliciting the preferences of the consumers of the generated output are explored. The first study presents a musical interface that enables navigation of the possibilities of a stochastic generative process with respect to measures of subjective predictability. Through a mobile phone version of the application, aesthetic preferences are crowd-sourced. The second study presents an eye-tracking based framework for the exploration of the possibilities afforded by generative designs; the interaction between the viewers’ gaze patterns and the system engendering a fluid navigation of the state-space of the visual forms. The third study presents a crowd-sourced interactive evolutionary system, where populations of abstract colour images are shaped by thousands of preference selections from users worldwide For each study, the results of analyses eliciting the attributes of the generated outputs – and their associated parameter values – that are most preferred by the consumers/users of these systems are presented. Placed in a historical and theoretical context, a refined perspective on the complex interrelationships between generative processes, input parameters and perceived aesthetic value is presented. Contributions to knowledge include identified trends in objective aesthetic preferences in colour combinations and their arrangements, theoretical insights relating perceptual mechanisms to generative system design and analysis, strategies for effectively leveraging evolutionary computation in an empirical aesthetic context, and a novel eye-tracking based framework for the exploration of visual generative designs. Submitted for the degree of Doctor of Philosophy Queen Mary, University of London

[1]  Arne Eigenfeldt,et al.  Evolving structures for electronic dance music , 2013, GECCO '13.

[2]  Keld K. Jensen,et al.  Timbre Models of Musical Sounds , 1999 .

[3]  A. E. Eiben Evolutionary Reproduction of Dutch Masters: The Mondriaan and Escher Evolvers , 2008, The Art of Artificial Evolution.

[4]  Kolby McHale Ada Lovelace: The Unauthorized Guide to the History, Life and Accomplishments of the First Computer Programmer , 2011 .

[5]  A. M. Turing,et al.  The chemical basis of morphogenesis , 1952, Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences.

[6]  Mark D. Plumbley,et al.  Information dynamics: patterns of expectation and surprise in the perception of music , 2009, Connect. Sci..

[7]  Scott Draves,et al.  Evolution and Collective Intelligence of the Electric Sheep , 2008, The Art of Artificial Evolution.

[8]  Alexis Kirke,et al.  A Multi-Agent Emotional Society Whose Melodies Represent its Emergent Social Hierarchy and Are Generated by Agent Communications , 2015, J. Artif. Soc. Soc. Simul..

[9]  Lester C. Loschky,et al.  User performance with gaze contingent multiresolutional displays , 2000, ETRA.

[10]  Elaine P. Miller,et al.  Philosophy of the Arts: An Introduction to Aesthetics , 2000 .

[11]  R. Arnheim Art and visual perception: A psychology of the creative eye, New version , 1955 .

[12]  Kate Reed,et al.  Aesthetic Measures for Evolutionary Vase Design , 2013, EvoMUSART.

[13]  Mats G. Nordahl,et al.  An evolutionary architecture for a humanoid robot , 1999 .

[14]  Michael Vitale,et al.  The Wisdom of Crowds , 2015, Cell.

[15]  Benjamin B. Bederson,et al.  Human computation: a survey and taxonomy of a growing field , 2011, CHI.

[16]  Kemal Ebcioglu,et al.  An Expert System for Harmonizing Chorales in the Style of J. S. Bach , 1990, J. Log. Program..

[17]  R. Drake Effects of gaze manipulation on aesthetic judgments: hemisphere priming of affect. , 1987, Acta psychologica.

[18]  J. Itten The art of color : the subjective experience and objective rationale of color , 1973 .

[19]  Trevor Wishart,et al.  On Sonic Art , 1996 .

[20]  Christian Jacob,et al.  Evolutionary and Swarm Design in Science, Art, and Music , 2008, The Art of Artificial Evolution.

[21]  Rolf G. Kuehni,et al.  The early development of the Munsell system , 2002 .

[22]  George E. Lewis Too Many Notes: Computers, Complexity and Culture in Voyager , 2000, Leonardo Music Journal.

[23]  Penousal Machado,et al.  Zipf's Law, Music Classification, and Aesthetics , 2005, Computer Music Journal.

[24]  Una-May O'Reilly,et al.  Genr8: Architects' Experience with an Emergent Design Tool , 2008, The Art of Artificial Evolution.

[25]  Peter M. Todd,et al.  Connectionism and Musiconomy , 2003 .

[26]  Gary Greenfield,et al.  Co-evolutionary Methods in Evolutionary Art , 2008, The Art of Artificial Evolution.

[27]  R. Davis An Evaluation and Test of Birkhoff's Aesthetic Measure Formula , 1936 .

[28]  R. MacCallum,et al.  Evolution of music by public choice , 2012, Proceedings of the National Academy of Sciences.

[29]  Penousal Machado,et al.  The Art of Artificial Evolution: A Handbook on Evolutionary Art and Music , 2007 .

[30]  G. W. Granger An Experimental Study of Colour Preferences , 1955 .

[31]  Yasuaki Sakamoto,et al.  Feature Selection in Crowd Creativity , 2011, HCI.

[32]  Leonard M. Isaacson,et al.  Experimental Music. Composition with an Electronic Computer , 1959 .

[33]  Geraint A. Wiggins,et al.  Auditory Expectation: The Information Dynamics of Music Perception and Cognition , 2012, Top. Cogn. Sci..

[34]  Eduardo Reck Miranda,et al.  At the Crossroads of Evolutionary Computation and Music: Self-Programming Synthesizers, Swarm Orchestras and the Origins of Melody , 2004, Evolutionary Computation.

[35]  Johannes M. Zanker,et al.  Using an oculomotor signature as an indicator of aesthetic preference , 2012, i-Perception.

[36]  Robert J. K. Jacob,et al.  Interacting with eye movements in virtual environments , 2000, CHI.

[37]  Kirti Trivedi,et al.  Hindu temples: Models of a fractal universe , 1989, The Visual Computer.

[38]  Craig W. Reynolds Flocks, herds, and schools: a distributed behavioral model , 1998 .

[39]  H. Atlan Entre le cristal et la fumée : essai sur l'organisation du vivant , 1979 .

[40]  Robert J. K. Jacob,et al.  What you look at is what you get: eye movement-based interaction techniques , 1990, CHI '90.

[41]  E. Robinson Cybernetics, or Control and Communication in the Animal and the Machine , 1963 .

[42]  Henri Matisse,et al.  Matisse on Art , 1973 .

[43]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[44]  Francis Heylighen,et al.  The Science of Self-Organization and Adaptivity , 1999 .

[45]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[46]  Elaine Cohen,et al.  Computer Generated Celtic Design , 2003, Rendering Techniques.

[47]  Jon McCormack,et al.  Art, emergence, and the computational sublime , 2001 .

[48]  Paul Doornbusch A Brief Survey of Mapping in Algorithmic Composition , 2002, ICMC.

[49]  Douglas DeCarlo,et al.  Stylization and abstraction of photographs , 2002, ACM Trans. Graph..

[50]  Christopher G. Langton,et al.  Computation at the edge of chaos: Phase transitions and emergent computation , 1990 .

[51]  Joel E. Cohen,et al.  Information theory and music , 2007 .

[52]  Penousal Machado,et al.  On the development of evolutionary artificial artists , 2007, Comput. Graph..

[53]  Poika Isokoski,et al.  Now Dasher! Dash away!: longitudinal study of fast text entry by Eye Gaze , 2008, ETRA.

[54]  Penousal Machado,et al.  Developing Fitness Functions for Pleasant Music: Zipf's Law and Interactive Evolution Systems , 2005, EvoWorkshops.

[55]  John M. Chowning,et al.  The Synthesis of Complex Audio Spectra by Means of Frequency Modulation , 1973 .

[56]  Paul Brown,et al.  Stepping stones in the mist , 2001 .

[57]  Kenneth O. Stanley,et al.  Scaffolding for Interactively Evolving Novel Drum Tracks for Existing Songs , 2008, EvoWorkshops.

[58]  Ron Eglash,et al.  African Fractals: Modern Computing and Indigenous Design , 1999 .

[59]  Alan Dorin A Survey of Virtual Ecosystems in Generative Electronic Art , 2008, The Art of Artificial Evolution.

[60]  Makis Solomos Cellular automata in Xenakis's music. Theory and Practice , 2005 .

[61]  Geraint A. Wiggins,et al.  The Four-Part Harmonisation Problem : A comparison between Genetic Algorithms and a Rule-Based System , 1999 .

[62]  Laura Chamberlain Eye Tracking Methodology; Theory and Practice , 2007 .

[63]  Petri Toiviainen Modeling the Target-Note Technique of Bebop-Style Jazz Improvisation: An Artificial Neural Network Approach , 1995 .

[64]  Andrew T Duchowski,et al.  A breadth-first survey of eye-tracking applications , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[65]  Claudia Mello-Thoms,et al.  Visual interest in pictorial art during an aesthetic experience. , 2007, Spatial vision.

[66]  Ebroul Izquierdo,et al.  Reading Users' Minds From Their Eyes: A Method for Implicit Image Annotation , 2012, IEEE Transactions on Multimedia.

[67]  Stephen Todd,et al.  Evolutionary Art and Computers , 1992 .

[68]  Kitsuchart Pasupa,et al.  Image ranking with implicit feedback from eye movements , 2010, ETRA.

[69]  George Papadopoulos,et al.  A Genetic Algorithm for the Generation of Jazz Melodies , 2000 .

[70]  Ji-Hyun Lee,et al.  A study of emergence in the generation of Islamic geometric patterns , 2010, CAADRIA proceedings.

[71]  K. Rayner Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[72]  Stephen Schwanauer,et al.  Simulating Musical Skills by Digital Computer , 1993 .

[73]  Michael W. Levine,et al.  Fundamentals of sensation and perception , 1981 .

[74]  David Hume Of the Standard of Taste , 2017, Reading David Hume’s 'Of the Standard of Taste'.

[75]  Lee Ellis,et al.  Color preferences according to gender and sexual orientation , 2001 .

[76]  Christopher K. I. Williams,et al.  Harmonising Chorales by Probabilistic Inference , 2004, NIPS.

[77]  Matthew R. Lewis Evolutionary Visual Art and Design , 2008, The Art of Artificial Evolution.

[78]  E. Adelson Perceptual organization and the judgment of brightness. , 1993, Science.

[79]  Richard A. Bolt,et al.  A gaze-responsive self-disclosing display , 1990, CHI '90.

[80]  Penousal Machado,et al.  A Corpus-Based Hybrid Approach to Music Analysis and Composition , 2007, AAAI.

[81]  Yôiti Suzuki,et al.  Equal-loudness-level contours for pure tones. , 2004, The Journal of the Acoustical Society of America.

[82]  C. Ehrenfels,et al.  Foundations of Gestalt Theory , 1988 .

[83]  John H. Conway,et al.  The game of life. , 1996, The Hastings Center report.

[84]  Agostino Di Scipio ‘Sound is the interface’: from interactive to ecosystemic signal processing , 2003 .

[85]  Steve Garner,et al.  Evaluating an eye tracking interface for a two-dimensional sketch editor , 2013, Comput. Aided Des..

[86]  W. Stolz Information Theory and Esthetic Perception. , 1967 .

[87]  A. Gocer A Hypothesis Concerning the Character of Islamic Art , 1999 .

[88]  Adolfo Maia,et al.  Granular synthesis of sounds through Markov Chains with fuzzy control , 2005, ICMC.

[89]  Owen Green Pondering Value in the Performance Ecosystem , 2008 .

[90]  Kevin N Laland,et al.  Transmission fidelity is the key to the build-up of cumulative culture , 2012, Philosophical Transactions of the Royal Society B: Biological Sciences.

[91]  Harold J. McWhinnie,et al.  A review of research on aesthetic measure , 1968 .

[92]  J. Meyer Minimalism: Art and Polemics in the Sixties , 2001 .

[93]  Matti Karjalainen,et al.  Body Modeling Techniques for String Instrument Synthesis , 1996, ICMC.

[94]  Chris Langton,et al.  Artificial Life , 2017, Encyclopedia of Machine Learning and Data Mining.

[95]  G. Pask An Approach To Cybernetics , 1961 .

[96]  Donald A. and Wigging Glenn Schon,et al.  Kinds of Seeing and Their Functions in Designing , 1988 .

[97]  R. Ascott Is There Love in the Telematic Embrace , 1990 .

[98]  Kemal Ebcioglu,et al.  An Expert System for Chorale Harmonization , 1986, AAAI.

[99]  Gareth Loy,et al.  Composing with computers: a survey of some compositional formalisms and music programming languages , 1989 .

[100]  Richard N. Aslin,et al.  Statistical learning of tonal structure by adults and infants , 1999 .

[101]  Raimund Dachselt,et al.  Look & touch: gaze-supported target acquisition , 2012, CHI.

[102]  Jasia Reichardt,et al.  Cybernetic serendipity : the computer and the arts , 1969 .

[103]  Issam El-Said,et al.  Islamic Art and Architecture: The System of Geometric Design , 1998 .

[104]  Christoph Klütsch INFORMATION AESTHETICS AND THE STUTTGART SCHOOL , 2012 .

[105]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 1997, Texts in Computer Science.

[106]  Lester C. Loschky,et al.  Gaze-Contingent Multiresolutional Displays: An Integrative Review , 2003, Hum. Factors.

[107]  Karl Sims,et al.  Artificial evolution for computer graphics , 1991, SIGGRAPH.

[108]  Penousal Machado,et al.  Experiments in Computational Aesthetics , 2008, The Art of Artificial Evolution.

[109]  Eduardo Miranda,et al.  Granular Synthesis of Sounds by Means of a Cellular Automaton , 2017 .

[110]  Chris Mellish,et al.  Statistical Learning of Harmonic Movement , 1999 .

[111]  Penousal Machado,et al.  On the Development of Critics in Evolutionary Computation Artists , 2003, EvoWorkshops.

[112]  Anthony G. Pipe,et al.  Convergence Synthesis of Dynamic Frequency Modulation Tones Using an Evolution Strategy , 2005, EvoWorkshops.

[113]  Penousal Machado,et al.  Adaptive Critics for Evolutionary Artists , 2004, EvoWorkshops.

[114]  Ricardo A. Garcia Growing Sound Synthesizers using Evolutionary Methods , 2001 .

[115]  Mary Nielsen,et al.  The Creation of the sRGB ICC Profile , 1998, Color Imaging Conference.

[116]  Steve Garner,et al.  Shape exploration of designs in a style: Toward generation of product designs , 2006, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[117]  A. L. Yarbus Eye Movements During Perception of Complex Objects , 1967 .

[118]  Shlomo Dubnov,et al.  Spectral Anticipations , 2006, Computer Music Journal.

[119]  J. V. Goethe Theory of Colours , 1810 .

[120]  Michael Hensel Emergence : morphogenetic design strategies , 2004 .

[121]  Eduardo Reck Miranda,et al.  Emergent Sound Repertoires in Virtual Societies , 2002, Computer Music Journal.

[122]  T. Schnier,et al.  From Mondrian to Frank Lloyd Wright: Transforming Evolving Representations , 1998 .

[123]  Geraint A. Wiggins,et al.  Towards Greater Objectivity in Music Theory: Information-Dynamic Analysis of Minimalist Music , 2007 .

[124]  Peter M. Todd,et al.  Frankensteinian methods for evolutionary music composition , 1999 .

[125]  Mateu Sbert,et al.  Conceptualizing Birkhoff's Aesthetic Measure Using Shannon Entropy and Kolmogorov Complexity , 2007, CAe.

[126]  Karlheinz Essl Algorithmic composition , 2012, The Cambridge Companion to Electronic Music.

[127]  James W. Beauchamp,et al.  Timbre Breeding , 1993, ICMC.

[128]  Iestyn Jowers,et al.  DESIGN EXPLORATION WITH USELESS RULES AND EYE TRACKING , 2011 .

[129]  L. Sivik,et al.  A Theory of Colors in Combination-A Descriptive Model Related to the NCS Color-Order System , 2001 .

[130]  Phil M. Henry,et al.  Colour Harmony , 2007 .

[131]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[132]  Shumin Zhai,et al.  Manual and gaze input cascaded (MAGIC) pointing , 1999, CHI '99.

[133]  Hugh Dubberly,et al.  What is interaction ? Are there different types ? , 2014 .

[134]  Christoph Klutsch,et al.  Computer GraphicAesthetic Experiments between Two Cultures , 2007, Leonardo.

[135]  Alex Kosorukoff,et al.  Human based genetic algorithm , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[136]  S. Shevell The Science of Color , 2003 .

[137]  George Stiny,et al.  Shape Grammars and the Generative Specification of Painting and Sculpture , 1971, IFIP Congress.

[138]  Steve Reich,et al.  Writings on Music, 1965-2000 , 2002 .

[139]  Tom Davis,et al.  Gestural Control of Sonic Swarms: Composing with Grouped Sound Objects , 2007 .

[140]  Critical Aesthetic Realism , 2011 .

[141]  J. Guilford,et al.  A system of color-preferences. , 1949, The American journal of psychology.

[142]  Jeffrey V. Nickerson,et al.  Structures for Creativity: The crowdsourcing of design , 2011 .

[143]  Parry Moon,et al.  Aesthetic Measure Applied to Color Harmony , 1944 .

[144]  Terri Gullickson The Creative Mind: Myths and Mechanisms. , 1995 .