An Architectural Framework for the Design, Analysis and Implementation of Interactive Systems

This article presents the Software Architecture for Immersipresence (SAI) framework for the design, analysis and implementation of interactive software systems. SAI defines an architectural style whose data and processing models capture temporal properties of computational primitives. SAI adopts an asynchronous concurrent processing model in order to allow designing for optimal system latency and throughput. The SAI style is modular to enable scalability, and to facilitate incremental system design, integration and evolution, as well as code development, testing and reuse. A graph-based notation for architectural designs aims to communicate system architectures in an intuitive and scalable manner. The open-source Modular Flow Scheduling Middleware (MFSM) provides a multi-threaded, cross-platform implementation of SAI's primitives. The SAI/MFSM framework has been used in the design and implementation of numerous interactive systems in research, education and artistic performance settings. This article illustrates the definition and use of SAI with examples from computer vision.

[1]  Alexandre R. J. François,et al.  Software Architecture for Immersipresence , 2003 .

[2]  David Lorge Parnas,et al.  Modularity in the new millenium: a panel summary , 2003, ICSE.

[3]  Elaine Chew,et al.  An Architectural Framework for Interactive Music Systems , 2006, NIME.

[4]  David H. Eberly,et al.  3D game engine design - a practical approach to real-time computer graphics , 2000 .

[5]  David Garlan,et al.  Acme: an architecture description interchange language , 2010, CASCON.

[6]  Carl Hewitt,et al.  Viewing Control Structures as Patterns of Passing Messages , 1977, Artif. Intell..

[7]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Robert J. Allen.  A formal approach to software architecture , 1997 .

[9]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[10]  Gul A. Agha,et al.  ACTORS - a model of concurrent computation in distributed systems , 1985, MIT Press series in artificial intelligence.

[11]  Elaine Chew,et al.  Visual feedback in performer-machine interaction for musical improvisation , 2007, NIME '07.

[12]  Lawrence A. Rowe,et al.  Design and performance of the Berkeley Continuous Media Toolkit , 1997, Electronic Imaging.

[13]  Elaine Chew,et al.  Interactive multi-scale visualizations of tonal evolution in MuSA.RT Opus 2 , 2005, CIE.

[14]  Frank Eliassen,et al.  Supporting timeliness and accuracy in distributed real-time content-based video analysis , 2003, MULTIMEDIA '03.

[15]  Walter Mann,et al.  Correction to "Specification and Analysis of System Architecture Using Rapide" , 1995, IEEE Trans. Software Eng..

[16]  Jie Liu,et al.  ESP: A Driving Interface for Expression Synthesis , 2005, NIME.

[17]  Alexandre R. J. François,et al.  A handheld virtual mirror , 2002, SIGGRAPH '02.

[18]  Frank Eliassen,et al.  Scalable Independent Multi-level Distribution in Multimedia Content Analysis , 2002, IDMS/PROMS.

[19]  Jie Liu,et al.  ESP: roadmaps as constructed interpretations and guides to expressive performance , 2006, AMCMM '06.

[20]  Elaine Chew,et al.  MuSA.RT: music on the spiral array. real-time , 2003, ACM Multimedia.

[21]  David L. Tennenhouse,et al.  The VuSystem: A Programming System for Compute-Intensive Multimedia , 1996, IEEE J. Sel. Areas Commun..

[22]  E. Gat.  On Three-Layer Architectures , 1997 .

[23]  David H. Eberly.  3D game engine architecture - engineering real-time applications with wild magic , 2005, The Morgan Kaufmann series in interactive 3D technology.

[24]  Dennis McLeod,et al.  Integrated media systems , 1999 .

[25]  Gregory R. Andrews,et al.  Foundations of Multithreaded, Parallel, and Distributed Programming , 1999 .

[26]  M. Turk,et al.  Perceptual Interfaces , 2003 .

[27]  Gérard G. Medioni,et al.  Robust real-time vision for a personal service robot , 2007, Comput. Vis. Image Underst..

[28]  David C. Luckham,et al.  An Event-Based Architecture Definition Language , 1995, IEEE Trans. Software Eng..

[29]  C. A. R. Hoare,et al.  Communicating sequential processes , 1978, CACM.

[30]  Gérard G. Medioni,et al.  A Modular Software Architecture for Real-Time Video Processing , 2001, ICVS.

[31]  Alexandre R. J. François,et al.  A hybrid architectural style for distributed parallel processing of generic data streams , 2004, Proceedings. 26th International Conference on Software Engineering.

[32]  Sing Bing Kang,et al.  Emerging Topics in Computer Vision , 2004 .

[33]  Leslie Lamport,et al.  The ``Hoare Logic'' of CSP, and All That , 1984, TOPL.

[34]  Mary Shaw,et al.  Software architecture - perspectives on an emerging discipline , 1996 .

[35]  Gary Bradski,et al.  Computer Vision Face Tracking For Use in a Perceptual User Interface , 1998 .

[36]  Jan A. Bergstra,et al.  ACTtau: A Universal Axiom System for Process Specification , 1987, Algebraic Methods.

[37]  Richard N. Taylor,et al.  A Classification and Comparison Framework for Software Architecture Description Languages , 2000, IEEE Trans. Software Eng..

[38]  Gul Agha,et al.  Abstracting Interaction Patterns: A Programming Paradigm for Open Distributed Systems , 1997 .

[39]  Roger B. Dannenberg.  A Language for Interactive Audio Applications , 2002, ICMC.

[40]  Miller S. Puckette.  A divide between 'compositional' and 'performative' aspects of Pd ⁄ , 2004 .

[41]  Nenad Medvidovic,et al.  Composing architectural styles from architectural primitives , 2003, ESEC/FSE-11.

[42]  Philipp Slusallek,et al.  An Open Middleware Architecture for Network-Integrated Multimedia , 2002, IDMS/PROMS.

[43]  Gérard G. Medioni,et al.  A Vision System for Personal Service Robots : Resilient Detection and Tracking of People , 2006 .

[44]  Alexandre R. J. François,et al.  CAMSHIFT Tracker Design Experiments With Intel OpenCV and SAI , 2004 .

[45]  T. C. Nicholas Graham,et al.  Integrating Support for Temporal Media into an Architecture for Graphical User Interfaces , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[46]  David Garlan,et al.  A Formal Approach to Software Architectures , 1992, IFIP Congress.

[47]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[48]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .

[49]  Jie Liu,et al.  From driving to expressive music performance: ensuring tempo smoothness , 2006, ACE '06.

[50]  Xiaolei Qian,et al.  Correct Architecture Refinement , 1995, IEEE Trans. Software Eng..

[51]  Richard N. Taylor,et al.  A Component- and Message-Based Architectural Style for GUI Software , 1995, 1995 17th International Conference on Software Engineering.

[52]  David Garlan,et al.  A formal basis for architectural connection , 1997, TSEM.

[53]  Robin Milner,et al.  Communicating and mobile systems - the Pi-calculus , 1999 .

[54]  Alexandre R. Francois.  Software Architecture for Computer Vision: Beyond Pipes and Filters , 2003 .

[55]  Robin Milner,et al.  A Calculus of Communicating Systems , 1980, Lecture Notes in Computer Science.

[56]  Richard N. Taylor,et al.  A language and environment for architecture-based software development and evolution , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[57]  Alexandre R. J. François,et al.  Components for immersion , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[58]  Alexandre R. J. François,et al.  A handheld mirror simulation , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[59]  Robert A. Riemenschneider,et al.  Introduction to SADL 1.0: A Language for Specifying Software Architecture Hierarchies , 1997 .