Merging virtual objects with the real world: seeing ultrasound imagery within the patient

We describe initial results which show “live” ultrasound echography data visualized within a pregnant human subject. The visualization is achieved by using a small video camera mounted in front of a conventional head-mounted display worn by an observer. The camera’s video images are composite with computer-generated ones that contain one or more 2D ultrasound images properly transformed to the observer’s current viewing position. As the observer walks around the subject. the ultrasound images appear stationary in 3-space within the subject. This kind of enhancement of the observer’s vision may have many other applications, e.g., image guided surgical procedures and on location 3D interactive architecture preview. CR Categories: 1.3.7 [Three-Dimensional Graphics and Realism] Virtual Reality, 1,3.I [Hardware architecture]: Three-dimensional displays, 1.3.6 [Methodology and Techniques]: Interaction techniques, J.3 ILife and Medical Sciences]: Medical information systems. Additional

[1]  Stephen M. Pizer,et al.  Boundary Estimation in Ultrasound Images , 1991, IPMI.

[2]  E. Geiser,et al.  A mechanical arm for spatial registration of two-dimensional echocardiographic sections. , 1982, Catheterization and cardiovascular diagnosis.

[3]  J. Thijssen,et al.  Texture in tissue echograms. Speckle or information? , 1990, Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine.

[4]  F.L. Thurstone,et al.  Biomedical Ultrasonics , 1970, IEEE Transactions on Industrial Electronics and Control Instrumentation.

[5]  Marc Levoy,et al.  Display of surfaces from volume data , 1988, IEEE Computer Graphics and Applications.

[6]  Ryutarou Ohbuchi,et al.  Incremental Volume Rendering Algorithm for Interactive 3D Ultrasound Imaging , 1991, IPMI.

[7]  Frederick P. Brooks,et al.  Fast spheres, shadows, textures, transparencies, and imgage enhancements in pixel-planes , 1985, Advances in Computer Graphics.

[8]  W. Moritz,et al.  An Ultrasonic Technique for Imaging the Ventricle in Three Dimensions and Calculating Its Volume , 1983, IEEE Transactions on Biomedical Engineering.

[9]  D. King,et al.  Three‐dimensional spatial registration and interactive display of position and orientation of real‐time ultrasound images. , 1990, Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine.

[10]  Donald P. Greenberg,et al.  Echocardiographic Three-Dimensional Visualization of the Heart , 1990 .

[11]  Hiroshi Abe,et al.  Three‐dimensional echocardiography for spatial visualization and volume calculation of cardiac structures , 1981, Journal of clinical ultrasound : JCU.

[12]  S.W. Smith,et al.  High-speed ultrasound volumetric imaging system. II. Parallel processing and image display , 1991, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[13]  Thomas W. Parsons,et al.  Digital signal processing: theory, applications, and hardware , 1991 .

[14]  L S Wann,et al.  An analysis of three-dimensional reconstructive echocardiography. , 1984, Ultrasound in medicine & biology.

[15]  N. Nanda,et al.  Three-dimensional reconstruction of echo-cardiographic images using the rotation method. , 1982, Ultrasound in medicine & biology.

[16]  John G. Eyles,et al.  PixelFlow: high-speed rendering using image composition , 1992, SIGGRAPH.

[17]  E A Geiser,et al.  Dynamic three-dimensional echocardiographic reconstruction of the intact human left ventricle: technique and initial observations in patients. , 1982, American heart journal.

[18]  G T Herman,et al.  Dynamic three-dimensional reconstruction of the left ventricle from two-dimensional echocardiograms. , 1986, Journal of the American College of Cardiology.

[19]  Chris Shaw,et al.  On temporal-spatial realism in the virtual reality environment , 1991, UIST '91.

[20]  Paolo Sabella,et al.  A rendering algorithm for visualizing 3D scalar fields , 1988, SIGGRAPH.

[21]  Henry Fuchs,et al.  Incremental 3D ultrasound imaging from a 2D scanner , 1990, [1990] Proceedings of the First Conference on Visualization in Biomedical Computing.

[22]  Warren Robinett,et al.  A Computational Model for the Stereoscopic Optics of a Head-Mounted Display , 1991, Presence: Teleoperators & Virtual Environments.

[23]  Henry Fuchs,et al.  3D ultrasound display using optical tracking , 1990, [1990] Proceedings of the First Conference on Visualization in Biomedical Computing.

[24]  John F. Hughes,et al.  Sculpting: an interactive volumetric modeling technique , 1991, SIGGRAPH.

[25]  A. Collet Billon,et al.  3D Echography: Status and Perspective , 1990 .

[26]  Ellen C. Hildreth,et al.  The detection of intensity changes by computer and biological vision systems , 1983, Comput. Vis. Graph. Image Process..

[27]  Roy Leipnik.  The Extended Entropy Uncertainty Principle , 1960, Inf. Control..

[28]  Ronald Azuma,et al.  A demonstrated optical tracker with scalable work area for head-mounted display systems , 1992, I3D '92.

[29]  Craig Upson,et al.  V-buffer: visible volume rendering , 1988, SIGGRAPH.

[30]  Shohei Nakamura,et al.  Three-Dimensional Digital Display of Ultrasonograms , 1984, IEEE Computer Graphics and Applications.

[31]  Hooshang Kangarloo,et al.  Three-Dimensional Reconstruction Of Ultrasound Images , 1989, Medical Imaging.

[32]  M Halliwell,et al.  Ultimate limits in ultrasonic imaging resolution. , 1991, Ultrasound in medicine & biology.

[33]  Christian Barillot,et al.  Multidimensional ultrasonic imaging for cardiology , 1988, Proc. IEEE.

[34]  J F Brinkley,et al.  Ultrasonic three-dimensional imaging and volume from a series of arbitrary sector scans. , 1978, Ultrasound in medicine & biology.

[35]  Marc Levoy,et al.  A hybrid ray tracer for rendering polygon and volume data , 1990, IEEE Computer Graphics and Applications.

[36]  S.W. Smith,et al.  High-speed ultrasound volumetric imaging system. I. Transducer design and beam steering , 1991, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[37]  Pat Hanrahan,et al.  Volume Rendering , 2020, Definitions.

[38]  Henry Fuchs,et al.  Pixel-planes 5: a heterogeneous multiprocessor graphics system using processor-enhanced memories , 1989, SIGGRAPH.

[39]  B. Fornage,et al.  Sonographic appearance and ultrasound‐guided fine‐needle aspiration biopsy of breast carcinomas smaller than 1 cm3. , 1990, Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine.

[40]  M Itoh,et al.  A computer-aided three-dimensional display system for ultrasonic diagnosis of a breast tumour. , 1979, Ultrasonics.

[41]  O T von Ramm,et al.  Explososcan: A Parallel Processing Technique For High Speed Ultrasound Imaging With Linear Phased Arrays , 1985, Medical Imaging.