Components of the visual computer: A review of relevant technologies

Visual computing emhpasizes visibility of information of interest and direct manipulation of visible information. It has become a dominant style of computing in recent years. In this paper, we review three major technologies that provide a platform for visual computing. These technologies reflect the needs of various people who use visual computers: programmers, end users, and scientists. We begin with a discussion of important developments in visual programming and follow with discussions of visual interfaces and visualization. We conclude with a summary of visual computing's current status and identify critical areas of research that should be emphasized in future work.

[1]  M. C. Pong,et al.  PIGS—A system for programming with interactive graphical support , 1983, Softw. Pract. Exp..

[2]  Marc Levoy,et al.  Volume rendering in radiation treatment planning , 1990, [1990] Proceedings of the First Conference on Visualization in Biomedical Computing.

[3]  Larry Masinter,et al.  The Interlisp Programming Environment , 1981, Computer.

[4]  Gurminder Singh,et al.  Visual user interface design tools , 2005, The Visual Computer.

[5]  Marita Franzke,et al.  Should we or shouldn't we use spoken commands in voice interfaces? , 1991, CHI '91.

[6]  Robert Sedgewick,et al.  A system for algorithm animation , 1984, SIGGRAPH.

[7]  J Rhyne Dialogue management for gestural interfaces , 1987, COMG.

[8]  Daniel T. Ling,et al.  Dialogue structures for virtual worlds , 1991, CHI.

[9]  John Stasko,et al.  Understanding and Characterizing Program Visualization Systems , 1991 .

[10]  Nan C. Shu Formal: A Forms-Oriented, Visual-Directed Application Development System , 1985, Computer.

[11]  Alan W. Biermann,et al.  Constructing Programs from Example Computations , 1976, IEEE Transactions on Software Engineering.

[12]  Ephraim P. Glinert Towards Software Metrics for Visual Programming , 1989, Int. J. Man Mach. Stud..

[13]  Warren Robinett,et al.  Virtual environment display system , 1987, I3D '86.

[14]  B. E. J. Clark,et al.  A Graphically Interacting Program Monitor , 1983, Comput. J..

[15]  Craig Upson,et al.  Volumetric Visualization Techniques , 1991 .

[16]  Gurminder Singh Vu: visual user-interface design , 2006, The Visual Computer.

[17]  van DamAndries,et al.  The Application Visualization System , 1989 .

[18]  A. Marcus,et al.  Design principles for the enhanced presentation of computer program source text , 1986, CHI '86.

[19]  Bruce H. McCormick,et al.  Visualization: expanding scientific and engineering research opportunities , 1989, Computer.

[20]  Pat Hanrahan,et al.  Volume Rendering , 2020, Definitions.

[21]  Paolo Sabella,et al.  A rendering algorithm for visualizing 3D scalar fields , 1988, SIGGRAPH.

[22]  David H. Laidlaw,et al.  The application visualization system: a computational environment for scientific visualization , 1989, IEEE Computer Graphics and Applications.

[23]  Steven P. Reiss,et al.  PECAN: Program Development Systems that Support Multiple Views , 1984, IEEE Transactions on Software Engineering.

[24]  Fred Lakin,et al.  Spatial Parsing for Visual Languages , 1986 .

[25]  Christopher F. Herot,et al.  Program Visualization: Graphics Support for Software Development , 1983, 20th Design Automation Conference Proceedings.

[26]  Jock D. Mackinlay,et al.  Automating the design of graphical presentations of relational information , 1986, TOGS.

[27]  Austin Henderson,et al.  On the Implementation of AMBIT/G: A Graphical Programming Language , 1969, IJCAI.

[28]  Christopher F. Herot Spatial Management Of Data , 1979, Fifth International Conference on Very Large Data Bases, 1979..

[29]  James C. Browne,et al.  TDFL: A Task-Level Dataflow Language , 1990, J. Parallel Distributed Comput..

[30]  Mark Chignell,et al.  Intelligent databases - object-oriented deductive hypermedia technologies , 1989 .

[31]  Larry Press Personal computing: compuvision or teleputer , 1990, CACM.

[32]  Steven S. Muchnick,et al.  Dbxtool: A window‐based symbolic debugger for sun workstations , 1986, Softw. Pract. Exp..

[33]  Gerry Souter,et al.  What is Multimedia , 1997 .

[34]  Christopher D. Shaw,et al.  The DataPaper: living in the virtual world , 1990 .

[35]  Larry Press,et al.  Compuvision or Teleputer , 1990 .

[36]  Charles Richter,et al.  VERDI: A Visual Environment for Designing Distributed Systems , 1990, J. Parallel Distributed Comput..

[37]  Tom G. Zimmerman,et al.  A hand gesture interface device , 1987, CHI '87.

[38]  Chris Shaw,et al.  The decoupled simulation model for virtual reality systems , 1992, CHI '92.

[39]  Brad A. Myers INCENSE: A system for displaying data structures , 1983, SIGGRAPH.

[40]  D. A. Duce,et al.  Visualization in Scientific Computing , 1994, Focus on Computer Graphics.

[41]  Ben Shneiderman,et al.  Direct Manipulation: A Step Beyond Programming Languages , 1983, Computer.

[42]  Shi-Kuo Chang,et al.  Visual Languages: A Tutorial and Survey , 1986, IEEE Software.

[43]  William M. Newman,et al.  A system for interactive graphical programming , 1968, AFIPS Spring Joint Computing Conference.

[44]  Edward A. Fox,et al.  Standards and the emergence of digital multimedia systems , 1991, CACM.

[45]  Henry Lieberman,et al.  Mondrian: a teachable graphical editor , 1993, INTERCHI.

[46]  Brad Allen Myers,et al.  Displaying data structures for interactive debugging , 1980 .

[47]  Dan R. Olsen,et al.  Interface usage measurements in a user interface management system , 1988, UIST '88.

[48]  Craig Upson,et al.  V-buffer: visible volume rendering , 1988, SIGGRAPH.

[49]  Nan C. Shu,et al.  Visual Programming: Perspectives and Approaches , 1989, IBM Syst. J..

[50]  Brad A. Myers,et al.  Invisible programming , 1990, Proceedings of the 1990 IEEE Workshop on Visual Languages.

[51]  Luca Cardelli,et al.  Building user interfaces by direct manipulation , 1988, UIST '88.

[52]  Alan R. Hevner,et al.  FORMANAGER: an office forms management system , 1984, TOIS.

[53]  Austin Henderson,et al.  Rooms: the use of multiple virtual workspaces to reduce space contention in a window-based graphical user interface , 1986, TOGS.

[54]  Andreas Pommert,et al.  Investigation of medical 3D-rendering algorithms , 1990, IEEE Computer Graphics and Applications.

[55]  Zary Segall,et al.  Visualizing performance debugging , 1989, Computer.

[56]  Takao Shimomura,et al.  Linked-list visualization for debugging , 1991, IEEE Software.

[57]  Shelley V. Goldman Sorting out sorting , 1982 .

[58]  George C. Feth Workshop Report: Impact of Emerging Technology , 1975, Computer.

[59]  James D. Hollan,et al.  Direct Manipulation Interfaces , 1985, Hum. Comput. Interact..

[60]  Brad A. Myers,et al.  A taxonomy of window manager user interfaces , 1988, IEEE Computer Graphics and Applications.

[61]  Alan Borning Graphically Defining New Building Blocks in Thinglab , 1987, SGCH.

[62]  Jakob Nielsen,et al.  Hypertext and hypermedia , 1990 .

[63]  Mark H. Chignell,et al.  A Manifesto for Hypermedia Usability Research , 1989, Hypermedia.

[64]  Johannes Sametinger,et al.  User‐adaptable prettyprinting , 1989, Softw. Pract. Exp..

[65]  Gabor Herman,et al.  Display of 3-D Digital Images: Computational Foundations and Medical Applications , 1983, IEEE Computer Graphics and Applications.

[66]  Gurminder Singh,et al.  Automating the lexical and syntactic design of graphical user interfaces: the UofA* UIMS , 1991, TOGS.

[67]  John T. Stasko Using direct manipulation to build algorithm animations by demonstration , 1991, CHI '91.

[68]  Anthony I. Wasserman,et al.  Extending State Transition Diagrams for the Specification of Human–Computer Interaction , 1985, IEEE Transactions on Software Engineering.

[69]  Andrea A. diSessa,et al.  A Principled Design for an Integrated Computational Environment , 1985, Hum. Comput. Interact..

[70]  James A. Pittman,et al.  Recognizing handwritten text , 1991, CHI.

[71]  Gurminder Singh,et al.  Druid: a system for demonstrational rapid user interface development , 1990, UIST '90.

[72]  Marc Levoy,et al.  A hybrid ray tracer for rendering polygon and volume data , 1990, IEEE Computer Graphics and Applications.

[73]  James C. Browne,et al.  CODE: a unified approach to parallel programming , 1989, IEEE Software.

[74]  Robert Duisberg,et al.  Animating Programs Using Smalltalk , 1985, Computer.

[75]  Robert Duisberg Visual Programming of Program Visualizations , 1990 .

[76]  Srdjan Kovacevic,et al.  Defining interfaces at a high level of abstraction , 1989, IEEE Software.

[77]  Miren Begoña Albizuri-Romero GRASE: a graphical syntax-directed editor for structured programming , 1984, SIGP.

[78]  James Arthur Kohl,et al.  A Tool to Aid in the Design, Implementation, and Understanding of Matrix Algorithms for Parallel Processors , 1990, J. Parallel Distributed Comput..

[79]  Mark Green,et al.  SIGGRAPH '90 Workshop report: software architectures and metaphors for non-WIMP user interfaces , 1991, COMG.

[80]  M. H. Brown Perspectives on algorithm animation , 1988, CHI '88.

[81]  Aaron Marcus,et al.  User-interface developments for the nineties , 1991, Computer.

[82]  B. Shneiderman,et al.  Flowchart techniques for structured programming , 1973, SIGP.

[83]  Moshé M. Zloof QBE/OBE: A Language for Office and Business Automation , 1981, Computer.

[84]  Brad A. Myers,et al.  Taxonomies of visual programming and program visualization , 1990, J. Vis. Lang. Comput..

[85]  Andries van Dam PHIGS+ functional description revision , 1988, SIGGRAPH 1988.

[86]  Gregory M. Nielson Visualization in scientific and engineering computation , 1991, Computer.

[87]  Ephraim P. Glinert,et al.  Pict: An Interactive Graphical Programming Environment , 1984, Computer.

[88]  Lawrence A. Rowe,et al.  A browser for directed graphs , 1987, Softw. Pract. Exp..

[89]  Steven P. Reiss,et al.  Using GELO to visualize software systems , 1989, UIST '89.

[90]  Brad A. Myers,et al.  Creating user interfaces using programming by example, visual programming, and constraints , 1990, TOPL.

[91]  Lionel M. Ni,et al.  Special Issue on Software Tools for Parallel Programming and Visualization: Guest Editors' Introduction , 1990, J. Parallel Distributed Comput..

[92]  Nan C. Shu,et al.  Visual Programming: Perspectives and Approaches , 1999, IBM Syst. J..

[93]  Jock D. Mackinlay,et al.  The cognitive coprocessor architecture for interactive user interfaces , 1989, UIST '89.

[94]  Thomas J. Leblanc,et al.  Analyzing Parallel Program Executions Using Multiple Views , 1990, J. Parallel Distributed Comput..

[95]  D. Scott Dyer,et al.  A dataflow toolkit for visualization , 1990, IEEE Computer Graphics and Applications.

[96]  Ephraim P. Glinert,et al.  Visual Programming Environments: Paradigms and Systems , 1990 .

[97]  Dan Gordon,et al.  Back-to-Front Display of Voxel Based Objects , 1985, IEEE Computer Graphics and Applications.

[98]  Kouichi Murakami,et al.  Gesture recognition using recurrent neural networks , 1991, CHI.

[99]  Jane Wilhelms,et al.  A coherent projection approach for direct volume rendering , 1991, SIGGRAPH.

[100]  Deborah Hix,et al.  Human-computer interface development tools: a methodology for their evaluation , 1991, CACM.

[101]  Clayton Lewis,et al.  Making usable, useful, productivity-enhancing computer applications , 1991, CACM.

[102]  Ephraim P. Glinert,et al.  Exploring the general-purpose visual alternative , 1990, J. Vis. Lang. Comput..

[103]  Frederick P. Brooks,et al.  Walkthrough—a dynamic graphics system for simulating virtual buildings , 1987, I3D '86.

[104]  Wolfgang Klas,et al.  Introduction to Special Issue on Multimedia Information Systems , 2004, Multimedia Systems.

[105]  John A. Begoray An Introduction to Hypermedia Issues, Systems and Application Areas , 1990, Int. J. Man Mach. Stud..

[106]  Paul E. Haeberli,et al.  ConMan: a visual programming language for interactive graphics , 1988, SIGGRAPH.

[107]  William Buxton,et al.  The use of non-speech audio at the interface , 1988, CHI 1988.

[108]  Mukkai S. Krishnamoorthy,et al.  Program tools for algorithm animation , 1989, Softw. Pract. Exp..

[109]  Roger B. Dannenberg,et al.  Garnet: comprehensive support for graphical, highly interactive user interfaces , 1990, Computer.

[110]  Randy F. Pausch,et al.  Tailor: creating custom user interfaces based on gesture , 1990, UIST '90.

[111]  Eileen Kraemer,et al.  Applying Program Visualization Techniques to Aid Parallel and Distributed Program Development , 1991 .

[112]  Larry Cook,et al.  A Three-Dimensional Display System for Diagnostic Imaging Applications , 1983, IEEE Computer Graphics and Applications.

[113]  Gary Bishop,et al.  A visualization programming environment for multicomputers , 1990, IEEE Computer Graphics and Applications.

[114]  Robert M. Keller,et al.  Data Flow Program Graphs , 1982, Computer.

[115]  Moshé M. Zloof Query-by-Example: A Data Base Language , 1977, IBM Syst. J..

[116]  Myron W. Krueger,et al.  Artificial reality II , 1991 .

[117]  Ted G. Lewis,et al.  Prototypes from standard user interface management systems , 1989, Computer.

[118]  Frederick P. Brooks,et al.  Towards image realism with interactive update rates in complex virtual building environments , 1990, I3D '90.

[119]  Robert J. K. Jacob,et al.  A State Transition Diagram Language for Visual Programming , 1985, Computer.

[120]  M. Green Virtual reality user interface: tools and techniques , 1990 .

[121]  Gurminder Singh,et al.  Creating graphical interfaces through graphical interaction , 1990, Proceedings of the 1990 IEEE Workshop on Visual Languages.

[122]  David Canfield Smith,et al.  Pygmalion: A COMPUTER PROGRAM TO Model and Stimulate Creative Thought , 1975 .

[123]  Lawrence Koved,et al.  Design for interactive performance in a virtual laboratory , 1990, I3D '90.

[124]  Daniel L. Weller,et al.  A graphics-based programming-support system , 1978, SIGGRAPH.

[125]  Marc Levoy,et al.  Display of surfaces from volume data , 1988, IEEE Computer Graphics and Applications.

[126]  John T. Stasko Tango: A Framework and System for Algorithm Animation , 1990, Computer.

[127]  J. Nielsen,et al.  Coordinating user interfaces for consistency , 2001, SGCH.

[128]  Gurminder Singh,et al.  Requirements for user interface programming languages , 1992 .

[129]  Andrew V. Thompson,et al.  Micro-CAP: An Analog Circuit Design System for Personal Computers , 1984, IEEE Computer Graphics and Applications.

[130]  Robert Sedgewick,et al.  SYSTEM FOR ALGORITHM ANIMATION. , 1984 .

[131]  Roger B. Dannenberg,et al.  Garnet: comprehensive support for graphical, highly interactive user interfaces , 1995 .

[132]  J.T. Stasko,et al.  Tango: a framework and system for algorithm animation , 1990, Computer.

[133]  Mark H. Chignell,et al.  The HEFTI Model of Text to Hypertext Conversion , 1991, Hypermedia.

[134]  Rich Gossweiler,et al.  Virtual Reality on Five Dollars a Day , 1991 .

[135]  Ian H. Witten,et al.  Metamouse: specifying graphical procedures by example , 1989, SIGGRAPH.

[136]  David L. Donoho,et al.  MacSpin: dynamic graphics on a desktop computer , 1988, IEEE Computer Graphics and Applications.

[137]  Eric Harslem,et al.  Designing the STAR User Interface , 1987, ECICS.

[138]  T. O. Ellis,et al.  THE GRAIL PROJECT: AN EXPERIMENT IN MAN-MACHINE COMMUNICATIONS , 1969 .

[139]  N. Cunniff,et al.  Does programming language affect the type of conceptual bugs in beginners' programs? A comparison of FPL and Pascal , 1986, CHI '86.

[140]  Robert J. Glushko,et al.  Transforming text into hypertext for a compact disc encyclopedia , 1989, CHI '89.

[141]  Gurminder Singh Automating the lexical and syntactic design of graphical user interfaces , 1989 .

[142]  Gabriel Robins,et al.  The isi grapher: a portable tool for displaying graphs pictorially , 1987 .

[143]  Meredith Bricken,et al.  Virtual reality learning environments: potentials and challenges , 1991, COMG.

[144]  Ron Baecker Two systems which produce animated representations of the execution of computer programs , 1975, SIGCSE '75.