Capturing and indexing computer-based activities with virtual network computing

In this paper, we present a new technique to capture and index computer-based activities, without hindering natural humancomputer interactions. This technique is based on the Virtual Network Computing (VNC) technology, which is an ultra-thinclient/server computing model that separates the display interface from the application logic in windowing systems. The server executes all the applications and the client simply presents the frame buffer updates to the user and accepts user input. We record the frame buffer updates for work review, and store the user and system events as potential indices into the recording.