A visual editing tool supporting the production of 3D interactive graphics assets for public exhibitions

Abstract The introduction of interactive assets in public exhibitions is capable to significantly enhance the visitors’ user experience. However, the creation of interactive applications could represent a challenging task, especially for users lacking computer skills. Visual programming languages (VPLs) – one of the instruments belonging to the broad categories of methods and tools devised to support end-user development (EUD) – promise to offer an intuitive way to overcome these limitations, by providing easy-to-use and efficient interfaces for encoding applications’ logic. Moving from these considerations, this paper first analyses pros and cons of tools devised so far to support the generation of interactive contents. Then, it presents the design of a new tool named Visual Scene Editor (VSE), which allows users with little to no programming skills to create 3D interactive applications by combining available assets through an interactive, visual process. Both objective and subjective measurements have been collected with both skilled and unskilled users to evaluate the performance of the proposed tool. A comparison with existing solutions shows a reduction in the time required to complete the assigned tasks, of the complexity of the logic created, as well as of the number of errors made, confirming the suitability of the VSE for the said purpose.

[1]  Maria Roussou,et al.  A versatile large-scale multimodal VR system for cultural heritage visualization , 2006, VRST '06.

[2]  John Maloney,et al.  The Scratch Programming Language and Environment , 2010, TOCE.

[3]  Peter M. Thompson,et al.  SketchBio: a scientist’s 3D interface for molecular modeling and animation , 2014, BMC Bioinformatics.

[4]  Claude Ghaoui,et al.  Encyclopedia of Human Computer Interaction , 2005 .

[5]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[6]  Chad Baker,et al.  Development of a video game that teaches the fundamentals of computer programming , 2015, SoutheastCon 2015.

[7]  Wan Chul Yoon,et al.  Supporting the cognitive process of user interface design with reusable design cases , 2005, Int. J. Hum. Comput. Stud..

[8]  Chi Thanh Vi,et al.  Not just seeing, but also feeling art: Mid-air haptic experiences integrated in a multisensory art exhibition , 2017, Int. J. Hum. Comput. Stud..

[9]  Maurice Hendrix,et al.  Scenario-based serious games repurposing , 2011, SIGDOC '11.

[10]  Ivan E. Sutherland,et al.  Sketchpad a Man-Machine Graphical Communication System , 1964 .

[11]  Hiroyuki Tominaga,et al.  A Practical Lesson of Introductory Programming Exercise with LEGO Robot Control and Game Project , 2009 .

[12]  Fabio Paternò,et al.  End User Development: Survey of an Emerging Field for Empowering People , 2013 .

[13]  Nazlena Mohamad Ali,et al.  A Conceptual Framework for Designing Virtual Heritage Environment for Cultural Learning , 2018, ACM Journal on Computing and Cultural Heritage.

[14]  Dieter Schmalstieg,et al.  Unified Patterns for Realtime Interactive Simulation in Games and Digital Storytelling , 2019, IEEE Computer Graphics and Applications.

[15]  Krzysztof Walczak,et al.  Building Virtual and Augmented Reality museum exhibitions , 2004, Web3D '04.

[16]  T. O. Ellis,et al.  The GRAIL Language and Operations , 1969 .

[17]  Roberto Pierdicca,et al.  A Survey of Augmented, Virtual, and Mixed Reality for Cultural Heritage , 2018, ACM Journal on Computing and Cultural Heritage.

[18]  Francisco José García-Peñalvo,et al.  Building, coding and programming 3D models via a visual programming environment , 2018 .

[19]  Gavin Sim,et al.  A Virtual Reality Platform for Analyzing Remote Archaeological Sites , 2019, Interacting with computers.

[20]  So-Yeon Yoon,et al.  User attributes in processing 3D VR-enabled showroom: Gender, visual cognitive styles, and the sense of presence , 2015, Int. J. Hum. Comput. Stud..

[21]  B. R. Barricelli,et al.  End-user development, end-user programming and end-user software engineering: A systematic mapping study , 2019, J. Syst. Softw..

[22]  Paolo Montuschi,et al.  Is Immersive Virtual Reality the Ultimate Interface for 3D Animators? , 2020, Computer.

[23]  Michael S. Hsiao,et al.  A Natural Language Programming Application for Lego Mindstorms EV3 , 2018, 2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR).

[24]  Margaret M. Burnett,et al.  End-User Development in the Internet of Things Era , 2015, CHI Extended Abstracts.

[25]  David C. Webb,et al.  AgentCubes: Incremental 3D end-user development , 2009, J. Vis. Lang. Comput..

[26]  Cheng-Li Liu,et al.  A study of detecting and combating cybersickness with fuzzy control for the elderly within 3D virtual stores , 2014, Int. J. Hum. Comput. Stud..

[27]  Antonella De Angeli,et al.  End-User Development for Serious Games , 2017, New Perspectives in End-User Development.

[28]  Emiliano Pastorelli,et al.  Virtual Reality Visualization for Photogrammetric 3D Reconstructions of Cultural Heritage , 2014, AVR.

[29]  Ranjitha Kumar,et al.  Bricolage: example-based retargeting for web design , 2011, CHI.

[30]  Andrew M. Dearden,et al.  Quando: Enabling Museum and Art Gallery Practitioners to Develop Interactive Digital Exhibits , 2017, IS-EUD.

[31]  Hirokazu Kato,et al.  Handheld Guides in Inspection Tasks: Augmented Reality versus Picture , 2018, IEEE Transactions on Visualization and Computer Graphics.

[32]  Matthew MacLaurin,et al.  The design of kodu: a tiny visual programming language for children on the Xbox 360 , 2011, POPL '11.

[33]  Fabio Paternò,et al.  Trigger-Action Programming for Personalising Humanoid Robot Behaviour , 2019, CHI.

[34]  Huhn Kim,et al.  Effective organization of design guidelines reflecting designer's design strategies , 2010 .

[35]  Morten Lind,et al.  Using the blender game engine for real-time emulation of production devices , 2012 .

[36]  Carmelo Ardito,et al.  Empowering End Users to Customize their Smart Environments , 2017, ACM Trans. Comput. Hum. Interact..

[37]  Andrea Sanna,et al.  Developing Touch-Less Interfaces to Interact with 3D Contents in Public Exhibitions , 2016, AVR.

[38]  Vicente Pelechano,et al.  Towards an End-User Development Approach for Web Engineering Methods , 2006, CAiSE.

[39]  Jaime Lloret,et al.  Implementation of end-user development success factors in mashup development environments , 2016, Comput. Stand. Interfaces.

[40]  Fabio Paternò,et al.  Puzzle: A mobile application development environment using a jigsaw metaphor , 2014, J. Vis. Lang. Comput..

[41]  Hideyuki Nakanishi,et al.  FreeWalk: a social interaction platform for group behaviour in a virtual space , 2004, Int. J. Hum. Comput. Stud..

[42]  Toni Robertson,et al.  Design representations of moving bodies for interactive, motion-sensing spaces , 2009, Int. J. Hum. Comput. Stud..

[43]  Myounghoon Jeon,et al.  From rituals to magic: Interactive art and HCI of the past, present, and future , 2019, Int. J. Hum. Comput. Stud..

[44]  David H. Rogers,et al.  ColorMoves: Real-time Interactive Colormap Construction for Scientific Visualization , 2018, IEEE Computer Graphics and Applications.

[45]  Cesare Pautasso,et al.  End-User Development of Mashups with NaturalMash , 2014, J. Vis. Lang. Comput..

[46]  Volker Wulf,et al.  End-User Development of Enterprise Widgets , 2009, IS-EUD.

[47]  Jetmir Xhembulla,et al.  Integrating a Location-Based Mobile Game in the Museum Visit , 2015, ACM Journal on Computing and Cultural Heritage.

[48]  Ivan Martinovic,et al.  Digital heritage application as an edutainment tool , 2004, VRCAI '04.

[49]  Carmelo Ardito,et al.  Composition of situational interactive spaces by end users: a case for cultural heritage , 2012, NordiCHI.

[50]  Teale Fristoe,et al.  Kodu Game Lab: a programming environment , 2012, The Computer Games Journal.

[51]  Mira Dontcheva,et al.  Rewire: Interface Design Assistance from Examples , 2018, CHI.

[52]  Carmelo Ardito,et al.  From smart objects to smart experiences: An end-user development approach , 2017, Int. J. Hum. Comput. Stud..

[53]  Michael Zyda,et al.  From visual simulation to virtual reality to games , 2005, Computer.

[54]  Giulio Jacucci,et al.  Extreme Co-design: Prototyping with and by the User for Appropriation of Web-connected Tags , 2015, IS-EUD.

[55]  Kathryn T. Stolee,et al.  Expressing computer science concepts through Kodu game lab , 2011, SIGCSE.

[56]  Esteban Vázquez-Cano,et al.  Visual programming languages integrated across the curriculum in elementary school: A two year case study using "Scratch" in five schools , 2016, Comput. Educ..

[57]  Vittorio Scarano,et al.  A Framework to Design, Develop, and Evaluate Immersive and Collaborative Serious Games in Cultural Heritage , 2017, ACM Journal on Computing and Cultural Heritage.

[58]  Mario Alaguero,et al.  A flexible platform for the creation of 3D semi-immersive environments to teach Cultural Heritage , 2015, Digit. Appl. Archaeol. Cult. Heritage.

[59]  Michele Risi,et al.  Iconic languages: Towards end-user programming of mobile applications , 2017, J. Vis. Lang. Comput..

[60]  Miklós Maróti,et al.  A Visual Programming Environment for Learning Distributed Programming , 2017, SIGCSE.

[61]  Fabio Paternò,et al.  Cicero Designer: An Environment for End-User Development of Multi-Device Museum Guides , 2009, IS-EUD.

[62]  Paul Gault,et al.  DiCER: A distributed consumer experience research method for use in public spaces , 2015, Int. J. Hum. Comput. Stud..

[63]  Monica G. Maceli,et al.  Tools of the Trade: A Survey of Technologies in End-User Development Literature , 2017, IS-EUD.

[64]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[65]  Markus Ketterl,et al.  Graphical Programming Environments for Educational Robots: Open Roberta - Yet Another One? , 2014, 2014 IEEE International Symposium on Multimedia.

[66]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[67]  Soufiane Lahmine,et al.  Learning Basic Programming Concepts by Creating Games with Scratch Programming Environment , 2015 .

[68]  Ronen I. Brafman,et al.  Designing with interactive example galleries , 2010, CHI.

[69]  Francesco De Pace,et al.  A Comparison Between Two Different Approaches for a Collaborative Mixed-Virtual Environment in Industrial Maintenance , 2019, Front. Robot. AI.

[70]  Fabio Paternò,et al.  Personalization of Context-Dependent Applications Through Trigger-Action Rules , 2017, ACM Trans. Comput. Hum. Interact..