Multimodal exploration of the fourth dimension

We present a multimodal paradigm for exploring topological surfaces embedded in four dimensions; we exploit haptic methods in particular to overcome the intrinsic limitations of 3D graphics images and 3D physical models. The basic problem is that, just as 2D shadows of 3D curves lose structure where lines cross, 3D graphics projections of smooth 4D topological surfaces are interrupted where one surface intersects another. Furthermore, if one attempts to trace real knotted ropes or a plastic models of self-intersecting surfaces with a fingertip, one inevitably collides with parts of the physical artifact. In this work, we exploit the free motion of a computer-based haptic probe to support a continuous motion that follows the local continuity of the object being explored. For our principal test case of 4D-embedded surfaces projected to 3D, this permits us to follow the full local continuity of the surface as though in fact we were touching an actual 4D object. We exploit additional sensory cues to provide supplementary or redundant information. For example, we can use audio tags to note the relative 4D depth of illusory 3D surface intersections produced by projection from 4D, as well as providing automated refinement of the tactile exploration path to eliminate jitter and snagging, resulting in a much cleaner exploratory motion than a bare uncorrected motion. Visual enhancements provide still further improvement to the feedback: by opening a view-direction-defined cutaway into the interior of the 3D surface projection, we allow the viewer to keep the haptic probe continuously in view as it traverses any touchable part of the object. Finally, we extend the static tactile exploration framework using a dynamic mode that links each stylus motion to a change in orientation that creates at each instant a maximal-area screen projection of a neighborhood of the current point of interest. This minimizes 4D distortion and permits true metric sizes to be deduced locally at any point. All these methods combine to reveal the full richness of the complex spatial relationships of the target shapes, and to overcome many expected perceptual limitations in 4D visualization.